[2025-10-20T17:00:02.334531] [INFO ] [MAIN ] Using last completed trading day: 2025-10-20 [2025-10-20T17:00:02.518040] [DEBUG] [R-STATUS ] Registered 'tdm' as a dependent service. [2025-10-20T17:00:02.518206] [INFO ] [STATE ] Waiting for all dependent services to be clear... [2025-10-20T17:00:02.704747] [WARN ] [tdm-status] Service 'tdm' is not running. [2025-10-20T17:00:02.705159] [WARN ] [STATE ] State set to: RUNNING [2025-10-20T17:00:02.705223] [INFO ] [MAIN ] Refreshing symbols list before update... [2025-10-20T17:00:02.705261] [INFO ] [MAIN ] Starting TDM symbol refresh process. [2025-10-20T17:00:02.705340] [INFO ] [MAIN ] Fetching latest NYSE symbols list. [2025-10-20T17:00:02.705375] [DEBUG] [SYMBOL ] Starting fetch of NYSE Symbols. [2025-10-20T17:00:04.348978] [DEBUG] [SYMBOL ] Found 2419 valid NYSE symbols. [2025-10-20T17:00:04.349149] [INFO ] [SYMBOL ] Completed fetch of NYSE Symbols. [2025-10-20T17:00:04.349348] [INFO ] [MAIN ] Filtered out 54 blacklisted symbols. [2025-10-20T17:00:04.349895] [INFO ] [LOCAL ] Symbols cache updated with 2367 symbols. [2025-10-20T17:00:04.349951] [INFO ] [MAIN ] Successfully refreshed symbols list. [2025-10-20T17:00:04.349982] [INFO ] [MAIN ] Finished TDM symbol refresh process. [2025-10-20T17:00:04.350036] [INFO ] [MAIN ] Pruning stale symbols... [2025-10-20T17:00:04.350068] [INFO ] [MAIN ] Starting TDM stale symbol pruning process. [2025-10-20T17:00:04.350097] [INFO ] [LOCAL ] Reading cached symbols list. [2025-10-20T17:00:04.350475] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-10-20T17:00:04.350526] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-10-20T17:00:04.350702] [INFO ] [MAIN ] Including 54 blacklisted symbols for pruning. [2025-10-20T17:00:04.350757] [INFO ] [MAIN ] Pruning data for symbols not in current list of 2367 valid symbols. [2025-10-20T17:00:04.350789] [INFO ] [LOCAL ] Starting pruning of local symbol history data for inactive/invalid symbols. [2025-10-20T17:00:04.350819] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/primary_storage' [2025-10-20T17:00:04.370323] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/primary_storage/BTCM.csv (symbol BTCM no longer active). [2025-10-20T17:00:04.376525] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/primary_storage/SAND.csv (symbol SAND no longer active). [2025-10-20T17:00:04.377825] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/derivative_storage' [2025-10-20T17:00:04.397126] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/derivative_storage/BTCM.csv (symbol BTCM no longer active). [2025-10-20T17:00:04.403000] [DEBUG] [LOCAL ] Deleted /opt/tdm/output/data/derivative_storage/SAND.csv (symbol SAND no longer active). [2025-10-20T17:00:04.404326] [INFO ] [LOCAL ] Deleted 4 obsolete data files. [2025-10-20T17:00:04.404380] [INFO ] [LOCAL ] Completed pruning of unlisted local symbol history data. [2025-10-20T17:00:04.404413] [INFO ] [MAIN ] Successfully pruned stale symbol data. [2025-10-20T17:00:04.404441] [INFO ] [MAIN ] Finished TDM stale symbol pruning process. [2025-10-20T17:00:04.404496] [INFO ] [LOCAL ] Reading cached symbols list. [2025-10-20T17:00:04.404873] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-10-20T17:00:04.404924] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-10-20T17:00:04.404957] [INFO ] [MAIN ] Starting incremental update for 2367 symbols. [2025-10-20T17:00:04.404984] [INFO ] [MAIN ] Running in parallel mode [2025-10-20T17:00:04.405151] [INFO ] [MAIN ] Starting parallel primary update with 4 worker threads for 2367 symbols [2025-10-20T17:00:05.038992] [INFO ] [MAIN ] AAMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.038131] [INFO ] [MAIN ] A: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.068031] [INFO ] [MAIN ] AA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.066449] [DEBUG] [LOCAL ] AAMI: No existing data file found in primary storage. [2025-10-20T17:00:05.103732] [WARN ] [MAIN ] AAMI: No existing data found, performing full refresh instead. [2025-10-20T17:00:05.107633] [INFO ] [MAIN ] AAMI: Wiping existing primary data before full refresh. [2025-10-20T17:00:05.134255] [INFO ] [LOCAL ] AAMI: Starting deletion of primary data. [2025-10-20T17:00:05.132897] [INFO ] [MAIN ] AAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:05.134381] [DEBUG] [LOCAL ] AAMI: No primary data file found to delete. [2025-10-20T17:00:05.134483] [INFO ] [MAIN ] AAMI: Starting full primary refresh. [2025-10-20T17:00:05.134534] [DEBUG] [LOCAL ] AAM: No existing data file found in primary storage. [2025-10-20T17:00:05.140386] [WARN ] [MAIN ] AAM: No existing data found, performing full refresh instead. [2025-10-20T17:00:05.208869] [INFO ] [MAIN ] AAM: Wiping existing primary data before full refresh. [2025-10-20T17:00:05.225485] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-10-20T17:00:05.239586] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-10-20T17:00:05.240002] [INFO ] [MAIN ] AAM: Starting full primary refresh. [2025-10-20T17:00:05.763366] [DEBUG] [LOCAL ] A: Loaded 51523 existing records from primary storage. [2025-10-20T17:00:05.818078] [INFO ] [REMOTE ] AAMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:05.845594] [DEBUG] [LOCAL ] AA: Loaded 54401 existing records from primary storage. [2025-10-20T17:00:05.851241] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-01. [2025-10-20T17:00:05.851444] [INFO ] [MAIN ] A: Loaded 51523 existing records. [2025-10-20T17:00:05.863040] [INFO ] [MAIN ] AA: Loaded 54401 existing records. [2025-10-20T17:00:06.121022] [INFO ] [REMOTE ] AAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:06.147968] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-01. [2025-10-20T17:00:06.264550] [WARN ] [MAIN ] A: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:06.275035] [INFO ] [MAIN ] A: Wiping existing primary data before full refresh. [2025-10-20T17:00:06.280236] [INFO ] [LOCAL ] A: Starting deletion of primary data. [2025-10-20T17:00:06.294375] [DEBUG] [LOCAL ] A: Deleted primary data file. [2025-10-20T17:00:06.299849] [INFO ] [MAIN ] A: Starting full primary refresh. [2025-10-20T17:00:06.413418] [WARN ] [MAIN ] AA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:06.437835] [INFO ] [MAIN ] AA: Wiping existing primary data before full refresh. [2025-10-20T17:00:06.440327] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-10-20T17:00:06.440880] [DEBUG] [LOCAL ] AA: Deleted primary data file. [2025-10-20T17:00:06.446261] [INFO ] [MAIN ] AA: Starting full primary refresh. [2025-10-20T17:00:07.030268] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T17:00:07.032681] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-02. [2025-10-20T17:00:07.036815] [INFO ] [REMOTE ] A: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:07.037805] [INFO ] [REMOTE ] A: Fetching primary data for 2025-01. [2025-10-20T17:00:07.122411] [INFO ] [REMOTE ] AA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:07.122893] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-01. [2025-10-20T17:00:07.292735] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:00:07.296932] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-02. [2025-10-20T17:00:07.542599] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T17:00:07.543554] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-03. [2025-10-20T17:00:08.162073] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T17:00:08.163040] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-04. [2025-10-20T17:00:08.514797] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T17:00:08.515778] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-05. [2025-10-20T17:00:08.713940] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:00:08.717946] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-03. [2025-10-20T17:00:08.755257] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T17:00:08.756083] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-06. [2025-10-20T17:00:09.015543] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-10-20T17:00:09.016583] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-07. [2025-10-20T17:00:09.073728] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:00:09.084189] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-02. [2025-10-20T17:00:09.338312] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T17:00:09.339351] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-08. [2025-10-20T17:00:09.504898] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-01 (2.47 seconds). [2025-10-20T17:00:09.514552] [INFO ] [REMOTE ] A: Fetching primary data for 2025-02. [2025-10-20T17:00:09.627735] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T17:00:09.628804] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-09. [2025-10-20T17:00:09.873819] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-09 (0.25 seconds). [2025-10-20T17:00:09.874728] [INFO ] [REMOTE ] AAM: Fetching primary data for 2025-10. [2025-10-20T17:00:09.982202] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:00:09.987135] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-04. [2025-10-20T17:00:10.244887] [INFO ] [REMOTE ] AAM: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T17:00:10.245520] [INFO ] [REMOTE ] AAM: Full fetch complete (918 records). [2025-10-20T17:00:10.245590] [DEBUG] [VALIDATOR ] AAM: Validating primary data. [2025-10-20T17:00:10.419815] [WARN ] [VALIDATOR ] AAM: Validation failed - insufficient historical data. Have 184 unique days, need 200 days [2025-10-20T17:00:10.419975] [FATAL] [MAIN ] AAM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:10.420021] [WARN ] [MAIN ] AAM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:10.420061] [INFO ] [LOCAL ] AAM: Starting deletion of primary data. [2025-10-20T17:00:10.420114] [DEBUG] [LOCAL ] AAM: No primary data file found to delete. [2025-10-20T17:00:10.420156] [INFO ] [LOCAL ] AAM: Starting deletion of derivative data. [2025-10-20T17:00:10.420195] [DEBUG] [LOCAL ] AAM: No derivative data file found to delete. [2025-10-20T17:00:10.420246] [INFO ] [MAIN ] Update Progress: 1/2367 symbols completed. AAM took 6.0s. Estimated remaining: 3h 57m 12s. Rate: 10.0 symbols/min [2025-10-20T17:00:10.585806] [INFO ] [MAIN ] AAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:10.638242] [DEBUG] [LOCAL ] AAP: Loaded 52578 existing records from primary storage. [2025-10-20T17:00:10.638579] [INFO ] [MAIN ] AAP: Loaded 52578 existing records. [2025-10-20T17:00:10.815126] [WARN ] [MAIN ] AAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:10.815343] [INFO ] [MAIN ] AAP: Wiping existing primary data before full refresh. [2025-10-20T17:00:10.815389] [INFO ] [LOCAL ] AAP: Starting deletion of primary data. [2025-10-20T17:00:10.815971] [DEBUG] [LOCAL ] AAP: Deleted primary data file. [2025-10-20T17:00:10.816030] [INFO ] [MAIN ] AAP: Starting full primary refresh. [2025-10-20T17:00:11.181452] [INFO ] [REMOTE ] AAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:11.182429] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-01. [2025-10-20T17:00:11.239052] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T17:00:11.238078] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:00:11.258760] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-03. [2025-10-20T17:00:11.259002] [INFO ] [REMOTE ] A: Fetching primary data for 2025-03. [2025-10-20T17:00:11.574505] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:00:11.581040] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-05. [2025-10-20T17:00:12.835299] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:00:12.845854] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-02. [2025-10-20T17:00:12.866347] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:00:12.871980] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-06. [2025-10-20T17:00:13.040833] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:00:13.051341] [INFO ] [REMOTE ] A: Fetching primary data for 2025-04. [2025-10-20T17:00:13.181104] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:00:13.191961] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-04. [2025-10-20T17:00:13.857329] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:00:13.862327] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-07. [2025-10-20T17:00:14.568464] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:00:14.578563] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-03. [2025-10-20T17:00:14.741533] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:00:14.752475] [INFO ] [REMOTE ] A: Fetching primary data for 2025-05. [2025-10-20T17:00:14.885030] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:00:14.890834] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-08. [2025-10-20T17:00:15.439044] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-10-20T17:00:15.451741] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-05. [2025-10-20T17:00:15.782214] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:00:15.787854] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-09. [2025-10-20T17:00:16.145221] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:00:16.157178] [INFO ] [REMOTE ] A: Fetching primary data for 2025-06. [2025-10-20T17:00:16.493789] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:00:16.505263] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-04. [2025-10-20T17:00:17.092151] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:00:17.098392] [INFO ] [REMOTE ] AAMI: Fetching primary data for 2025-10. [2025-10-20T17:00:17.341107] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:00:17.353377] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-06. [2025-10-20T17:00:17.393014] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T17:00:17.404421] [INFO ] [REMOTE ] A: Fetching primary data for 2025-07. [2025-10-20T17:00:18.355357] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:00:18.368727] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-05. [2025-10-20T17:00:18.389695] [INFO ] [REMOTE ] AAMI: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:00:18.394178] [INFO ] [REMOTE ] AAMI: Full fetch complete (33226 records). [2025-10-20T17:00:18.394354] [DEBUG] [VALIDATOR ] AAMI: Validating primary data. [2025-10-20T17:00:18.828935] [DEBUG] [VALIDATOR ] AAMI: All expected trading days are present. [2025-10-20T17:00:18.829121] [DEBUG] [VALIDATOR ] AAMI: Primary data validation passed. [2025-10-20T17:00:18.829547] [INFO ] [MAIN ] AAMI: Fetched primary data is valid. [2025-10-20T17:00:18.829615] [DEBUG] [LOCAL ] AAMI: Saving primary data. [2025-10-20T17:00:18.830903] [DEBUG] [LOCAL ] AAMI: Trimmed 33226 records to 24762 (keeping 200 days of data) [2025-10-20T17:00:18.995126] [DEBUG] [LOCAL ] AAMI: Primary data saved (24762 records). [2025-10-20T17:00:18.995337] [INFO ] [MAIN ] AAMI: Full history saved. [2025-10-20T17:00:18.995496] [INFO ] [MAIN ] Update Progress: 2/2367 symbols completed. AAMI took 14.5s. Estimated remaining: 4h 47m 33s. Rate: 8.2 symbols/min [2025-10-20T17:00:19.186982] [INFO ] [MAIN ] AAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:19.246334] [DEBUG] [LOCAL ] AAT: Loaded 30862 existing records from primary storage. [2025-10-20T17:00:19.247392] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:00:19.258605] [INFO ] [MAIN ] AAT: Loaded 30862 existing records. [2025-10-20T17:00:19.292432] [INFO ] [REMOTE ] A: Fetching primary data for 2025-08. [2025-10-20T17:00:19.450067] [WARN ] [MAIN ] AAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:19.450358] [INFO ] [MAIN ] AAT: Wiping existing primary data before full refresh. [2025-10-20T17:00:19.450532] [INFO ] [LOCAL ] AAT: Starting deletion of primary data. [2025-10-20T17:00:19.450762] [DEBUG] [LOCAL ] AAT: Deleted primary data file. [2025-10-20T17:00:19.450817] [INFO ] [MAIN ] AAT: Starting full primary refresh. [2025-10-20T17:00:19.773737] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T17:00:19.807012] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-07. [2025-10-20T17:00:19.874730] [INFO ] [REMOTE ] AAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:19.875502] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-01. [2025-10-20T17:00:20.276482] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T17:00:20.289627] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-06. [2025-10-20T17:00:20.880994] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:00:20.894427] [INFO ] [REMOTE ] A: Fetching primary data for 2025-09. [2025-10-20T17:00:20.915535] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:00:20.921192] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-02. [2025-10-20T17:00:21.640175] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:00:21.652980] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-08. [2025-10-20T17:00:21.822733] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:00:21.828797] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-03. [2025-10-20T17:00:22.019551] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:00:22.032432] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-07. [2025-10-20T17:00:22.375083] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:00:22.387209] [INFO ] [REMOTE ] A: Fetching primary data for 2025-10. [2025-10-20T17:00:23.218166] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:00:23.226217] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-04. [2025-10-20T17:00:23.585040] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:00:23.597379] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-09. [2025-10-20T17:00:23.689029] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:00:23.700729] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-08. [2025-10-20T17:00:24.407283] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T17:00:24.415054] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-05. [2025-10-20T17:00:24.483707] [INFO ] [REMOTE ] A: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:00:24.491444] [INFO ] [REMOTE ] A: Full fetch complete (73577 records). [2025-10-20T17:00:24.491693] [DEBUG] [VALIDATOR ] A: Validating primary data. [2025-10-20T17:00:24.926245] [DEBUG] [VALIDATOR ] A: All expected trading days are present. [2025-10-20T17:00:24.926348] [DEBUG] [VALIDATOR ] A: Primary data validation passed. [2025-10-20T17:00:24.927033] [INFO ] [MAIN ] A: Fetched primary data is valid. [2025-10-20T17:00:24.927102] [DEBUG] [LOCAL ] A: Saving primary data. [2025-10-20T17:00:24.929088] [DEBUG] [LOCAL ] A: Trimmed 73577 records to 50738 (keeping 200 days of data) [2025-10-20T17:00:25.250828] [DEBUG] [LOCAL ] A: Primary data saved (50738 records). [2025-10-20T17:00:25.250993] [INFO ] [MAIN ] A: Full history saved. [2025-10-20T17:00:25.251306] [INFO ] [MAIN ] Update Progress: 3/2367 symbols completed. A took 20.8s. Estimated remaining: 4h 33m 47s. Rate: 8.6 symbols/min [2025-10-20T17:00:25.466509] [INFO ] [MAIN ] AAUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:25.466880] [DEBUG] [LOCAL ] AAUC: No existing data file found in primary storage. [2025-10-20T17:00:25.467117] [WARN ] [MAIN ] AAUC: No existing data found, performing full refresh instead. [2025-10-20T17:00:25.467429] [INFO ] [MAIN ] AAUC: Wiping existing primary data before full refresh. [2025-10-20T17:00:25.467884] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-10-20T17:00:25.468274] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-10-20T17:00:25.468355] [INFO ] [MAIN ] AAUC: Starting full primary refresh. [2025-10-20T17:00:25.785025] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T17:00:25.887951] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:00:25.896724] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-09. [2025-10-20T17:00:25.942727] [INFO ] [REMOTE ] AAUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:25.944395] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-06. [2025-10-20T17:00:25.945309] [INFO ] [REMOTE ] AAUC: Fetching primary data for 2025-01. [2025-10-20T17:00:25.946879] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-10-20T17:00:25.959953] [INFO ] [REMOTE ] AA: Fetching primary data for 2025-10. [2025-10-20T17:00:26.157623] [DEBUG] [REMOTE ] AAUC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:00:26.158069] [WARN ] [REMOTE ] AAUC: No data returned for month 2025-01 [2025-10-20T17:00:26.158160] [FATAL] [REMOTE ] AAUC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:00:26.158265] [FATAL] [MAIN ] AAUC: Failed to fetch primary data. [2025-10-20T17:00:26.158338] [WARN ] [MAIN ] AAUC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:00:26.158405] [INFO ] [LOCAL ] AAUC: Starting deletion of primary data. [2025-10-20T17:00:26.158492] [DEBUG] [LOCAL ] AAUC: No primary data file found to delete. [2025-10-20T17:00:26.158604] [INFO ] [LOCAL ] AAUC: Starting deletion of derivative data. [2025-10-20T17:00:26.158734] [DEBUG] [LOCAL ] AAUC: No derivative data file found to delete. [2025-10-20T17:00:26.158835] [INFO ] [MAIN ] Update Progress: 4/2367 symbols completed. AAUC took 0.9s. Estimated remaining: 3h 34m 11s. Rate: 11.0 symbols/min [2025-10-20T17:00:26.327235] [INFO ] [MAIN ] AB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:26.327405] [DEBUG] [LOCAL ] AB: No existing data file found in primary storage. [2025-10-20T17:00:26.327442] [WARN ] [MAIN ] AB: No existing data found, performing full refresh instead. [2025-10-20T17:00:26.327552] [INFO ] [MAIN ] AB: Wiping existing primary data before full refresh. [2025-10-20T17:00:26.327594] [INFO ] [LOCAL ] AB: Starting deletion of primary data. [2025-10-20T17:00:26.327679] [DEBUG] [LOCAL ] AB: No primary data file found to delete. [2025-10-20T17:00:26.327750] [INFO ] [MAIN ] AB: Starting full primary refresh. [2025-10-20T17:00:26.674038] [INFO ] [REMOTE ] AB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:26.674367] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-01. [2025-10-20T17:00:27.110479] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:00:27.116819] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-07. [2025-10-20T17:00:27.276125] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:00:27.286906] [INFO ] [REMOTE ] AAP: Fetching primary data for 2025-10. [2025-10-20T17:00:27.895169] [INFO ] [REMOTE ] AA: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:00:27.902475] [INFO ] [REMOTE ] AA: Full fetch complete (77343 records). [2025-10-20T17:00:27.902613] [DEBUG] [VALIDATOR ] AA: Validating primary data. [2025-10-20T17:00:27.903744] [FATAL] [VALIDATOR ] AA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:27.903832] [FATAL] [MAIN ] AA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:27.903874] [WARN ] [MAIN ] AA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:27.903937] [INFO ] [LOCAL ] AA: Starting deletion of primary data. [2025-10-20T17:00:27.904000] [DEBUG] [LOCAL ] AA: No primary data file found to delete. [2025-10-20T17:00:27.904035] [INFO ] [LOCAL ] AA: Starting deletion of derivative data. [2025-10-20T17:00:27.904161] [DEBUG] [LOCAL ] AA: Deleted derivative data file. [2025-10-20T17:00:27.904477] [INFO ] [MAIN ] Update Progress: 5/2367 symbols completed. AA took 23.5s. Estimated remaining: 3h 5m 1s. Rate: 12.8 symbols/min [2025-10-20T17:00:28.073489] [INFO ] [MAIN ] ABBV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:28.126527] [DEBUG] [LOCAL ] ABBV: Loaded 54465 existing records from primary storage. [2025-10-20T17:00:28.126889] [INFO ] [MAIN ] ABBV: Loaded 54465 existing records. [2025-10-20T17:00:28.328940] [WARN ] [MAIN ] ABBV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:28.329341] [INFO ] [MAIN ] ABBV: Wiping existing primary data before full refresh. [2025-10-20T17:00:28.331794] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-10-20T17:00:28.333382] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:00:28.341348] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-02. [2025-10-20T17:00:28.340575] [DEBUG] [LOCAL ] ABBV: Deleted primary data file. [2025-10-20T17:00:28.341673] [INFO ] [MAIN ] ABBV: Starting full primary refresh. [2025-10-20T17:00:28.519287] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:00:28.624979] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-08. [2025-10-20T17:00:28.734987] [INFO ] [REMOTE ] ABBV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:28.735336] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-01. [2025-10-20T17:00:29.206152] [INFO ] [REMOTE ] AAP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:00:29.214290] [INFO ] [REMOTE ] AAP: Full fetch complete (75154 records). [2025-10-20T17:00:29.214465] [DEBUG] [VALIDATOR ] AAP: Validating primary data. [2025-10-20T17:00:29.615441] [DEBUG] [VALIDATOR ] AAP: All expected trading days are present. [2025-10-20T17:00:29.615545] [DEBUG] [VALIDATOR ] AAP: Primary data validation passed. [2025-10-20T17:00:29.616272] [INFO ] [MAIN ] AAP: Fetched primary data is valid. [2025-10-20T17:00:29.616339] [DEBUG] [LOCAL ] AAP: Saving primary data. [2025-10-20T17:00:29.619108] [DEBUG] [LOCAL ] AAP: Trimmed 75154 records to 51783 (keeping 200 days of data) [2025-10-20T17:00:29.935184] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:00:29.947282] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-09. [2025-10-20T17:00:29.985111] [DEBUG] [LOCAL ] AAP: Primary data saved (51783 records). [2025-10-20T17:00:29.987152] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:00:29.987495] [INFO ] [MAIN ] AAP: Full history saved. [2025-10-20T17:00:29.996488] [INFO ] [MAIN ] Update Progress: 6/2367 symbols completed. AAP took 19.6s. Estimated remaining: 2h 47m 50s. Rate: 14.1 symbols/min [2025-10-20T17:00:29.996885] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-03. [2025-10-20T17:00:30.174270] [INFO ] [MAIN ] ABCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:30.202496] [DEBUG] [LOCAL ] ABCB: Loaded 31020 existing records from primary storage. [2025-10-20T17:00:30.202794] [INFO ] [MAIN ] ABCB: Loaded 31020 existing records. [2025-10-20T17:00:30.381448] [WARN ] [MAIN ] ABCB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:30.381790] [INFO ] [MAIN ] ABCB: Wiping existing primary data before full refresh. [2025-10-20T17:00:30.381894] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-10-20T17:00:30.382155] [DEBUG] [LOCAL ] ABCB: Deleted primary data file. [2025-10-20T17:00:30.382207] [INFO ] [MAIN ] ABCB: Starting full primary refresh. [2025-10-20T17:00:30.742926] [INFO ] [REMOTE ] ABCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:30.743299] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-01. [2025-10-20T17:00:30.783288] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:00:30.793556] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-02. [2025-10-20T17:00:30.848996] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T17:00:30.855269] [INFO ] [REMOTE ] AAT: Fetching primary data for 2025-10. [2025-10-20T17:00:31.148272] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T17:00:31.157842] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-04. [2025-10-20T17:00:32.134153] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:00:32.140978] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-02. [2025-10-20T17:00:32.252851] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:00:32.261070] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-05. [2025-10-20T17:00:32.349079] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:00:32.364974] [INFO ] [REMOTE ] AAT: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T17:00:32.371323] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-03. [2025-10-20T17:00:32.372756] [INFO ] [REMOTE ] AAT: Full fetch complete (43029 records). [2025-10-20T17:00:32.372844] [DEBUG] [VALIDATOR ] AAT: Validating primary data. [2025-10-20T17:00:32.765721] [DEBUG] [VALIDATOR ] AAT: All expected trading days are present. [2025-10-20T17:00:32.765812] [DEBUG] [VALIDATOR ] AAT: Primary data validation passed. [2025-10-20T17:00:32.766252] [INFO ] [MAIN ] AAT: Fetched primary data is valid. [2025-10-20T17:00:32.766293] [DEBUG] [LOCAL ] AAT: Saving primary data. [2025-10-20T17:00:32.767304] [DEBUG] [LOCAL ] AAT: Trimmed 43029 records to 30441 (keeping 200 days of data) [2025-10-20T17:00:32.954035] [DEBUG] [LOCAL ] AAT: Primary data saved (30441 records). [2025-10-20T17:00:32.954173] [INFO ] [MAIN ] AAT: Full history saved. [2025-10-20T17:00:32.956434] [INFO ] [MAIN ] Update Progress: 7/2367 symbols completed. AAT took 14.0s. Estimated remaining: 2h 40m 26s. Rate: 14.7 symbols/min [2025-10-20T17:00:33.125078] [INFO ] [MAIN ] ABEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:33.177005] [DEBUG] [LOCAL ] ABEV: Loaded 54055 existing records from primary storage. [2025-10-20T17:00:33.177364] [INFO ] [MAIN ] ABEV: Loaded 54055 existing records. [2025-10-20T17:00:33.366919] [WARN ] [MAIN ] ABEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:33.367361] [INFO ] [MAIN ] ABEV: Wiping existing primary data before full refresh. [2025-10-20T17:00:33.367830] [INFO ] [LOCAL ] ABEV: Starting deletion of primary data. [2025-10-20T17:00:33.368494] [DEBUG] [LOCAL ] ABEV: Deleted primary data file. [2025-10-20T17:00:33.368635] [INFO ] [MAIN ] ABEV: Starting full primary refresh. [2025-10-20T17:00:33.484867] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:00:33.515377] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T17:00:33.537222] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-06. [2025-10-20T17:00:33.573559] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-03. [2025-10-20T17:00:33.813966] [INFO ] [REMOTE ] ABEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:33.814295] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-01. [2025-10-20T17:00:34.463234] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T17:00:34.474893] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-04. [2025-10-20T17:00:34.488556] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:00:34.493312] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-07. [2025-10-20T17:00:35.044476] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:00:35.051671] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-04. [2025-10-20T17:00:35.484459] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:00:35.496093] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-02. [2025-10-20T17:00:35.636424] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:00:35.642103] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-08. [2025-10-20T17:00:36.252521] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:00:36.264297] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-05. [2025-10-20T17:00:36.391301] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:00:36.399913] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-05. [2025-10-20T17:00:36.416156] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T17:00:36.424258] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-09. [2025-10-20T17:00:37.059003] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:00:37.069188] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-03. [2025-10-20T17:00:37.241351] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T17:00:37.246757] [INFO ] [REMOTE ] AB: Fetching primary data for 2025-10. [2025-10-20T17:00:37.485489] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T17:00:37.491851] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-06. [2025-10-20T17:00:38.117338] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:00:38.128342] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-06. [2025-10-20T17:00:38.211557] [INFO ] [REMOTE ] AB: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T17:00:38.215089] [INFO ] [REMOTE ] AB: Full fetch complete (44285 records). [2025-10-20T17:00:38.215174] [DEBUG] [VALIDATOR ] AB: Validating primary data. [2025-10-20T17:00:38.546347] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:00:38.630402] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-07. [2025-10-20T17:00:38.649118] [DEBUG] [VALIDATOR ] AB: All expected trading days are present. [2025-10-20T17:00:38.649255] [DEBUG] [VALIDATOR ] AB: Primary data validation passed. [2025-10-20T17:00:38.649979] [INFO ] [MAIN ] AB: Fetched primary data is valid. [2025-10-20T17:00:38.665706] [DEBUG] [LOCAL ] AB: Saving primary data. [2025-10-20T17:00:38.682227] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:00:38.694208] [DEBUG] [LOCAL ] AB: Trimmed 44285 records to 25030 (keeping 200 days of data) [2025-10-20T17:00:38.694785] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-04. [2025-10-20T17:00:38.850535] [DEBUG] [LOCAL ] AB: Primary data saved (25030 records). [2025-10-20T17:00:38.850718] [INFO ] [MAIN ] AB: Full history saved. [2025-10-20T17:00:38.850804] [INFO ] [MAIN ] Update Progress: 8/2367 symbols completed. AB took 12.7s. Estimated remaining: 2h 49m 17s. Rate: 13.9 symbols/min [2025-10-20T17:00:39.025914] [INFO ] [MAIN ] ABG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:39.044685] [DEBUG] [LOCAL ] ABG: Loaded 18603 existing records from primary storage. [2025-10-20T17:00:39.044986] [INFO ] [MAIN ] ABG: Loaded 18603 existing records. [2025-10-20T17:00:39.229507] [WARN ] [MAIN ] ABG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:39.229819] [INFO ] [MAIN ] ABG: Wiping existing primary data before full refresh. [2025-10-20T17:00:39.229879] [INFO ] [LOCAL ] ABG: Starting deletion of primary data. [2025-10-20T17:00:39.230069] [DEBUG] [LOCAL ] ABG: Deleted primary data file. [2025-10-20T17:00:39.230126] [INFO ] [MAIN ] ABG: Starting full primary refresh. [2025-10-20T17:00:39.591712] [INFO ] [REMOTE ] ABG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:39.592463] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-01. [2025-10-20T17:00:40.125165] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:00:40.144581] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T17:00:40.153112] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-08. [2025-10-20T17:00:40.152824] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-07. [2025-10-20T17:00:40.537535] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T17:00:40.539953] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:00:40.556957] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-02. [2025-10-20T17:00:40.558245] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-05. [2025-10-20T17:00:41.251265] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T17:00:41.256755] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-03. [2025-10-20T17:00:41.402577] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:00:41.409445] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-09. [2025-10-20T17:00:42.065916] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:00:42.070830] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-04. [2025-10-20T17:00:42.442000] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-10-20T17:00:42.453722] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-08. [2025-10-20T17:00:42.683936] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T17:00:42.696782] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-06. [2025-10-20T17:00:42.943343] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:00:42.950501] [INFO ] [REMOTE ] ABCB: Fetching primary data for 2025-10. [2025-10-20T17:00:43.258383] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:00:43.264912] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-05. [2025-10-20T17:00:43.979118] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T17:00:43.982596] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-06. [2025-10-20T17:00:44.284770] [INFO ] [REMOTE ] ABCB: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:00:44.289988] [INFO ] [REMOTE ] ABCB: Full fetch complete (44683 records). [2025-10-20T17:00:44.290190] [DEBUG] [VALIDATOR ] ABCB: Validating primary data. [2025-10-20T17:00:44.290967] [FATAL] [VALIDATOR ] ABCB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:44.291065] [FATAL] [MAIN ] ABCB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:44.291140] [WARN ] [MAIN ] ABCB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:44.291229] [INFO ] [LOCAL ] ABCB: Starting deletion of primary data. [2025-10-20T17:00:44.291320] [DEBUG] [LOCAL ] ABCB: No primary data file found to delete. [2025-10-20T17:00:44.291362] [INFO ] [LOCAL ] ABCB: Starting deletion of derivative data. [2025-10-20T17:00:44.291524] [DEBUG] [LOCAL ] ABCB: Deleted derivative data file. [2025-10-20T17:00:44.293143] [INFO ] [MAIN ] Update Progress: 9/2367 symbols completed. ABCB took 14.3s. Estimated remaining: 2h 54m 11s. Rate: 13.5 symbols/min [2025-10-20T17:00:44.468874] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T17:00:44.533189] [INFO ] [MAIN ] ABM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:44.535058] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-09. [2025-10-20T17:00:44.565124] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T17:00:44.577113] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-07. [2025-10-20T17:00:44.606444] [DEBUG] [LOCAL ] ABM: Loaded 32640 existing records from primary storage. [2025-10-20T17:00:44.606744] [INFO ] [MAIN ] ABM: Loaded 32640 existing records. [2025-10-20T17:00:44.792818] [WARN ] [MAIN ] ABM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:44.793068] [INFO ] [MAIN ] ABM: Wiping existing primary data before full refresh. [2025-10-20T17:00:44.793137] [INFO ] [LOCAL ] ABM: Starting deletion of primary data. [2025-10-20T17:00:44.793349] [DEBUG] [LOCAL ] ABM: Deleted primary data file. [2025-10-20T17:00:44.793410] [INFO ] [MAIN ] ABM: Starting full primary refresh. [2025-10-20T17:00:45.165884] [INFO ] [REMOTE ] ABM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:45.166926] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:00:45.167202] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-01. [2025-10-20T17:00:45.172074] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-07. [2025-10-20T17:00:46.297465] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T17:00:46.302475] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-08. [2025-10-20T17:00:46.767725] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T17:00:46.780629] [INFO ] [REMOTE ] ABBV: Fetching primary data for 2025-10. [2025-10-20T17:00:46.824157] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-10-20T17:00:46.836522] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-08. [2025-10-20T17:00:46.970991] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:00:46.980224] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-02. [2025-10-20T17:00:47.374698] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:00:47.379762] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-09. [2025-10-20T17:00:47.623091] [INFO ] [REMOTE ] ABBV: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:00:47.631778] [INFO ] [REMOTE ] ABBV: Full fetch complete (77458 records). [2025-10-20T17:00:47.631979] [DEBUG] [VALIDATOR ] ABBV: Validating primary data. [2025-10-20T17:00:47.633156] [FATAL] [VALIDATOR ] ABBV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:00:47.633282] [FATAL] [MAIN ] ABBV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:00:47.633326] [WARN ] [MAIN ] ABBV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:00:47.633366] [INFO ] [LOCAL ] ABBV: Starting deletion of primary data. [2025-10-20T17:00:47.633426] [DEBUG] [LOCAL ] ABBV: No primary data file found to delete. [2025-10-20T17:00:47.633461] [INFO ] [LOCAL ] ABBV: Starting deletion of derivative data. [2025-10-20T17:00:47.633600] [DEBUG] [LOCAL ] ABBV: Deleted derivative data file. [2025-10-20T17:00:47.636195] [INFO ] [MAIN ] Update Progress: 10/2367 symbols completed. ABBV took 19.7s. Estimated remaining: 2h 49m 50s. Rate: 13.9 symbols/min [2025-10-20T17:00:47.823574] [INFO ] [MAIN ] ABR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:47.878021] [DEBUG] [LOCAL ] ABR: Loaded 52735 existing records from primary storage. [2025-10-20T17:00:47.878422] [INFO ] [MAIN ] ABR: Loaded 52735 existing records. [2025-10-20T17:00:48.065605] [WARN ] [MAIN ] ABR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:48.066376] [INFO ] [MAIN ] ABR: Wiping existing primary data before full refresh. [2025-10-20T17:00:48.066481] [INFO ] [LOCAL ] ABR: Starting deletion of primary data. [2025-10-20T17:00:48.072254] [DEBUG] [LOCAL ] ABR: Deleted primary data file. [2025-10-20T17:00:48.081699] [INFO ] [MAIN ] ABR: Starting full primary refresh. [2025-10-20T17:00:48.115991] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:00:48.155980] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-03. [2025-10-20T17:00:48.268498] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:00:48.299737] [INFO ] [REMOTE ] ABG: Fetching primary data for 2025-10. [2025-10-20T17:00:48.486544] [INFO ] [REMOTE ] ABR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:48.486981] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-01. [2025-10-20T17:00:48.768869] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:00:48.780683] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-09. [2025-10-20T17:00:49.447323] [INFO ] [REMOTE ] ABG: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:00:49.450436] [INFO ] [REMOTE ] ABG: Full fetch complete (26719 records). [2025-10-20T17:00:49.450587] [DEBUG] [VALIDATOR ] ABG: Validating primary data. [2025-10-20T17:00:49.846789] [DEBUG] [VALIDATOR ] ABG: All expected trading days are present. [2025-10-20T17:00:49.846991] [DEBUG] [VALIDATOR ] ABG: Primary data validation passed. [2025-10-20T17:00:49.847414] [INFO ] [MAIN ] ABG: Fetched primary data is valid. [2025-10-20T17:00:49.847550] [DEBUG] [LOCAL ] ABG: Saving primary data. [2025-10-20T17:00:49.868633] [DEBUG] [LOCAL ] ABG: Trimmed 26719 records to 18128 (keeping 200 days of data) [2025-10-20T17:00:49.869694] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:00:49.878149] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-04. [2025-10-20T17:00:50.001840] [DEBUG] [LOCAL ] ABG: Primary data saved (18128 records). [2025-10-20T17:00:50.002191] [INFO ] [MAIN ] ABG: Full history saved. [2025-10-20T17:00:50.002277] [INFO ] [MAIN ] Update Progress: 11/2367 symbols completed. ABG took 11.2s. Estimated remaining: 2h 42m 46s. Rate: 14.5 symbols/min [2025-10-20T17:00:50.182179] [INFO ] [MAIN ] ABT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:50.182478] [DEBUG] [LOCAL ] ABT: No existing data file found in primary storage. [2025-10-20T17:00:50.182524] [WARN ] [MAIN ] ABT: No existing data found, performing full refresh instead. [2025-10-20T17:00:50.182634] [INFO ] [MAIN ] ABT: Wiping existing primary data before full refresh. [2025-10-20T17:00:50.182718] [INFO ] [LOCAL ] ABT: Starting deletion of primary data. [2025-10-20T17:00:50.182767] [DEBUG] [LOCAL ] ABT: No primary data file found to delete. [2025-10-20T17:00:50.182800] [INFO ] [MAIN ] ABT: Starting full primary refresh. [2025-10-20T17:00:50.546179] [INFO ] [REMOTE ] ABT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:50.546847] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-01. [2025-10-20T17:00:50.583757] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:00:50.594766] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-02. [2025-10-20T17:00:50.759804] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:00:50.772326] [INFO ] [REMOTE ] ABEV: Fetching primary data for 2025-10. [2025-10-20T17:00:51.539808] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:00:51.547415] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-05. [2025-10-20T17:00:52.439032] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:00:52.450060] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-03. [2025-10-20T17:00:52.481762] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:00:52.492401] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-02. [2025-10-20T17:00:52.758890] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:00:52.764780] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-06. [2025-10-20T17:00:52.912989] [INFO ] [REMOTE ] ABEV: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T17:00:52.921913] [INFO ] [REMOTE ] ABEV: Full fetch complete (77346 records). [2025-10-20T17:00:52.922110] [DEBUG] [VALIDATOR ] ABEV: Validating primary data. [2025-10-20T17:00:53.319593] [DEBUG] [VALIDATOR ] ABEV: All expected trading days are present. [2025-10-20T17:00:53.319750] [DEBUG] [VALIDATOR ] ABEV: Primary data validation passed. [2025-10-20T17:00:53.320492] [INFO ] [MAIN ] ABEV: Fetched primary data is valid. [2025-10-20T17:00:53.320547] [DEBUG] [LOCAL ] ABEV: Saving primary data. [2025-10-20T17:00:53.322236] [DEBUG] [LOCAL ] ABEV: Trimmed 77346 records to 53291 (keeping 200 days of data) [2025-10-20T17:00:53.629220] [DEBUG] [LOCAL ] ABEV: Primary data saved (53291 records). [2025-10-20T17:00:53.629377] [INFO ] [MAIN ] ABEV: Full history saved. [2025-10-20T17:00:53.630805] [INFO ] [MAIN ] Update Progress: 12/2367 symbols completed. ABEV took 20.7s. Estimated remaining: 2h 41m 1s. Rate: 14.6 symbols/min [2025-10-20T17:00:53.805744] [INFO ] [MAIN ] ACA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:00:53.832095] [DEBUG] [LOCAL ] ACA: Loaded 23255 existing records from primary storage. [2025-10-20T17:00:53.832414] [INFO ] [MAIN ] ACA: Loaded 23255 existing records. [2025-10-20T17:00:54.019766] [WARN ] [MAIN ] ACA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:00:54.021025] [INFO ] [MAIN ] ACA: Wiping existing primary data before full refresh. [2025-10-20T17:00:54.021135] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-10-20T17:00:54.027958] [DEBUG] [LOCAL ] ACA: Deleted primary data file. [2025-10-20T17:00:54.038723] [INFO ] [MAIN ] ACA: Starting full primary refresh. [2025-10-20T17:00:54.049360] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:00:54.117512] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-07. [2025-10-20T17:00:54.488213] [INFO ] [REMOTE ] ACA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:00:54.489020] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-01. [2025-10-20T17:00:54.518680] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T17:00:54.558297] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-03. [2025-10-20T17:00:54.561432] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:00:54.572983] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-04. [2025-10-20T17:00:55.671287] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:00:55.676671] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-02. [2025-10-20T17:00:55.767118] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:00:55.775258] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-08. [2025-10-20T17:00:56.359020] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:00:56.370605] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-05. [2025-10-20T17:00:56.626432] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T17:00:56.638089] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-04. [2025-10-20T17:00:56.755548] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:00:56.761913] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-03. [2025-10-20T17:00:56.788750] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:00:56.797943] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-09. [2025-10-20T17:00:57.815609] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T17:00:57.822479] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-04. [2025-10-20T17:00:58.087306] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:00:58.096613] [INFO ] [REMOTE ] ABM: Fetching primary data for 2025-10. [2025-10-20T17:00:58.343104] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:00:58.354899] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-06. [2025-10-20T17:00:58.938917] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:00:58.945039] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-05. [2025-10-20T17:00:59.226752] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-04 (2.59 seconds). [2025-10-20T17:00:59.238586] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-05. [2025-10-20T17:00:59.378978] [INFO ] [REMOTE ] ABM: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:00:59.384217] [INFO ] [REMOTE ] ABM: Full fetch complete (45009 records). [2025-10-20T17:00:59.384327] [DEBUG] [VALIDATOR ] ABM: Validating primary data. [2025-10-20T17:00:59.818744] [DEBUG] [VALIDATOR ] ABM: All expected trading days are present. [2025-10-20T17:00:59.818858] [DEBUG] [VALIDATOR ] ABM: Primary data validation passed. [2025-10-20T17:00:59.819380] [INFO ] [MAIN ] ABM: Fetched primary data is valid. [2025-10-20T17:00:59.819425] [DEBUG] [LOCAL ] ABM: Saving primary data. [2025-10-20T17:00:59.820629] [DEBUG] [LOCAL ] ABM: Trimmed 45009 records to 32177 (keeping 200 days of data) [2025-10-20T17:01:00.047882] [DEBUG] [LOCAL ] ABM: Primary data saved (32177 records). [2025-10-20T17:01:00.048097] [INFO ] [MAIN ] ABM: Full history saved. [2025-10-20T17:01:00.049313] [INFO ] [MAIN ] Update Progress: 13/2367 symbols completed. ABM took 15.8s. Estimated remaining: 2h 47m 56s. Rate: 14.0 symbols/min [2025-10-20T17:01:00.084226] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:01:00.181155] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-07. [2025-10-20T17:01:00.249269] [INFO ] [MAIN ] ACCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:00.264481] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:01:00.270440] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-06. [2025-10-20T17:01:00.305431] [DEBUG] [LOCAL ] ACCO: Loaded 37509 existing records from primary storage. [2025-10-20T17:01:00.305762] [INFO ] [MAIN ] ACCO: Loaded 37509 existing records. [2025-10-20T17:01:00.478401] [WARN ] [MAIN ] ACCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:00.478598] [INFO ] [MAIN ] ACCO: Wiping existing primary data before full refresh. [2025-10-20T17:01:00.478676] [INFO ] [LOCAL ] ACCO: Starting deletion of primary data. [2025-10-20T17:01:00.478994] [DEBUG] [LOCAL ] ACCO: Deleted primary data file. [2025-10-20T17:01:00.479055] [INFO ] [MAIN ] ACCO: Starting full primary refresh. [2025-10-20T17:01:00.826903] [INFO ] [REMOTE ] ACCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:00.827234] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-01. [2025-10-20T17:01:01.167189] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T17:01:01.178194] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-06. [2025-10-20T17:01:01.215161] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:01:01.222011] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-07. [2025-10-20T17:01:01.998031] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:01:02.003773] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-08. [2025-10-20T17:01:02.119520] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:01:02.131173] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-08. [2025-10-20T17:01:02.816281] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:01:02.823929] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-02. [2025-10-20T17:01:02.902937] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:01:02.919294] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-07. [2025-10-20T17:01:03.393032] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:01:03.451503] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-09. [2025-10-20T17:01:04.312670] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T17:01:04.323985] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-09. [2025-10-20T17:01:04.363573] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:01:04.437826] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:01:05.051453] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-03. [2025-10-20T17:01:05.174769] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:01:05.851350] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-08. [2025-10-20T17:01:05.936413] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:01:06.588241] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:01:06.651340] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-04. [2025-10-20T17:01:07.451365] [INFO ] [REMOTE ] ACA: Fetching primary data for 2025-10. [2025-10-20T17:01:07.909777] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T17:01:08.251610] [INFO ] [REMOTE ] ABR: Fetching primary data for 2025-10. [2025-10-20T17:01:08.295188] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:01:08.504349] [INFO ] [REMOTE ] ACA: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:01:08.509277] [INFO ] [REMOTE ] ACA: Full fetch complete (34169 records). [2025-10-20T17:01:08.509420] [DEBUG] [VALIDATOR ] ACA: Validating primary data. [2025-10-20T17:01:08.510899] [FATAL] [VALIDATOR ] ACA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:01:08.511021] [FATAL] [MAIN ] ACA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:01:08.511099] [WARN ] [MAIN ] ACA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:01:08.511179] [INFO ] [LOCAL ] ACA: Starting deletion of primary data. [2025-10-20T17:01:08.511281] [DEBUG] [LOCAL ] ACA: No primary data file found to delete. [2025-10-20T17:01:08.511345] [INFO ] [LOCAL ] ACA: Starting deletion of derivative data. [2025-10-20T17:01:08.511614] [DEBUG] [LOCAL ] ACA: Deleted derivative data file. [2025-10-20T17:01:08.511746] [INFO ] [MAIN ] Update Progress: 14/2367 symbols completed. ACA took 14.9s. Estimated remaining: 2h 59m 35s. Rate: 13.1 symbols/min [2025-10-20T17:01:08.735256] [INFO ] [MAIN ] ACEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:08.766708] [DEBUG] [LOCAL ] ACEL: Loaded 27028 existing records from primary storage. [2025-10-20T17:01:08.767145] [INFO ] [MAIN ] ACEL: Loaded 27028 existing records. [2025-10-20T17:01:08.970443] [WARN ] [MAIN ] ACEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:08.970733] [INFO ] [MAIN ] ACEL: Wiping existing primary data before full refresh. [2025-10-20T17:01:08.970792] [INFO ] [LOCAL ] ACEL: Starting deletion of primary data. [2025-10-20T17:01:08.971001] [DEBUG] [LOCAL ] ACEL: Deleted primary data file. [2025-10-20T17:01:08.971044] [INFO ] [MAIN ] ACEL: Starting full primary refresh. [2025-10-20T17:01:09.056792] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-05. [2025-10-20T17:01:09.388599] [INFO ] [REMOTE ] ACEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:09.851433] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-09. [2025-10-20T17:01:09.965309] [INFO ] [REMOTE ] ABR: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T17:01:09.973703] [INFO ] [REMOTE ] ABR: Full fetch complete (75420 records). [2025-10-20T17:01:09.973885] [DEBUG] [VALIDATOR ] ABR: Validating primary data. [2025-10-20T17:01:10.434832] [DEBUG] [VALIDATOR ] ABR: All expected trading days are present. [2025-10-20T17:01:10.434990] [DEBUG] [VALIDATOR ] ABR: Primary data validation passed. [2025-10-20T17:01:10.436130] [INFO ] [MAIN ] ABR: Fetched primary data is valid. [2025-10-20T17:01:10.436222] [DEBUG] [LOCAL ] ABR: Saving primary data. [2025-10-20T17:01:10.463798] [DEBUG] [LOCAL ] ABR: Trimmed 75420 records to 51949 (keeping 200 days of data) [2025-10-20T17:01:10.465471] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T17:01:10.672031] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-01. [2025-10-20T17:01:10.825366] [DEBUG] [LOCAL ] ABR: Primary data saved (51949 records). [2025-10-20T17:01:10.825566] [INFO ] [MAIN ] ABR: Full history saved. [2025-10-20T17:01:10.826174] [INFO ] [MAIN ] Update Progress: 15/2367 symbols completed. ABR took 23.2s. Estimated remaining: 2h 53m 35s. Rate: 13.5 symbols/min [2025-10-20T17:01:11.014860] [INFO ] [MAIN ] ACHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:11.073809] [DEBUG] [LOCAL ] ACHR: Loaded 54598 existing records from primary storage. [2025-10-20T17:01:11.074272] [INFO ] [MAIN ] ACHR: Loaded 54598 existing records. [2025-10-20T17:01:11.286981] [WARN ] [MAIN ] ACHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:11.287272] [INFO ] [MAIN ] ACHR: Wiping existing primary data before full refresh. [2025-10-20T17:01:11.287322] [INFO ] [LOCAL ] ACHR: Starting deletion of primary data. [2025-10-20T17:01:11.287790] [DEBUG] [LOCAL ] ACHR: Deleted primary data file. [2025-10-20T17:01:11.287848] [INFO ] [MAIN ] ACHR: Starting full primary refresh. [2025-10-20T17:01:11.455414] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-06. [2025-10-20T17:01:11.712848] [INFO ] [REMOTE ] ACHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:11.811905] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:01:11.948194] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T17:01:12.251422] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-02. [2025-10-20T17:01:13.051425] [INFO ] [REMOTE ] ABT: Fetching primary data for 2025-10. [2025-10-20T17:01:13.190248] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:01:13.336701] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:01:13.533916] [INFO ] [REMOTE ] ABT: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:01:13.545005] [INFO ] [REMOTE ] ABT: Full fetch complete (77769 records). [2025-10-20T17:01:13.545208] [DEBUG] [VALIDATOR ] ABT: Validating primary data. [2025-10-20T17:01:13.859396] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-03. [2025-10-20T17:01:13.990939] [DEBUG] [VALIDATOR ] ABT: All expected trading days are present. [2025-10-20T17:01:13.991031] [DEBUG] [VALIDATOR ] ABT: Primary data validation passed. [2025-10-20T17:01:13.991752] [INFO ] [MAIN ] ABT: Fetched primary data is valid. [2025-10-20T17:01:13.991818] [DEBUG] [LOCAL ] ABT: Saving primary data. [2025-10-20T17:01:13.993587] [DEBUG] [LOCAL ] ABT: Trimmed 77769 records to 53622 (keeping 200 days of data) [2025-10-20T17:01:14.359356] [DEBUG] [LOCAL ] ABT: Primary data saved (53622 records). [2025-10-20T17:01:14.359513] [INFO ] [MAIN ] ABT: Full history saved. [2025-10-20T17:01:14.359587] [INFO ] [MAIN ] Update Progress: 16/2367 symbols completed. ABT took 24.4s. Estimated remaining: 2h 51m 19s. Rate: 13.7 symbols/min [2025-10-20T17:01:14.555107] [INFO ] [MAIN ] ACI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:14.613768] [DEBUG] [LOCAL ] ACI: Loaded 54062 existing records from primary storage. [2025-10-20T17:01:14.614167] [INFO ] [MAIN ] ACI: Loaded 54062 existing records. [2025-10-20T17:01:14.659396] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-07. [2025-10-20T17:01:14.822425] [WARN ] [MAIN ] ACI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:14.822708] [INFO ] [MAIN ] ACI: Wiping existing primary data before full refresh. [2025-10-20T17:01:14.822789] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-10-20T17:01:14.823264] [DEBUG] [LOCAL ] ACI: Deleted primary data file. [2025-10-20T17:01:14.823321] [INFO ] [MAIN ] ACI: Starting full primary refresh. [2025-10-20T17:01:15.219975] [INFO ] [REMOTE ] ACI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:15.236984] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:01:15.451403] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-04. [2025-10-20T17:01:15.909283] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:01:16.251403] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-08. [2025-10-20T17:01:16.838000] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:01:17.051384] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-05. [2025-10-20T17:01:17.659577] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:01:17.851364] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-01. [2025-10-20T17:01:18.230612] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T17:01:18.651391] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-06. [2025-10-20T17:01:19.451424] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-09. [2025-10-20T17:01:19.623870] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T17:01:19.780231] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:01:20.251352] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-07. [2025-10-20T17:01:20.723710] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:01:21.051425] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-02. [2025-10-20T17:01:21.436040] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:01:21.851374] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-01. [2025-10-20T17:01:22.650132] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:01:22.659822] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-08. [2025-10-20T17:01:23.451375] [INFO ] [REMOTE ] ACCO: Fetching primary data for 2025-10. [2025-10-20T17:01:24.010453] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:01:24.251351] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-03. [2025-10-20T17:01:24.560375] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-01 (2.71 seconds). [2025-10-20T17:01:24.799994] [INFO ] [REMOTE ] ACCO: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:01:24.805457] [INFO ] [REMOTE ] ACCO: Full fetch complete (54045 records). [2025-10-20T17:01:24.805591] [DEBUG] [VALIDATOR ] ACCO: Validating primary data. [2025-10-20T17:01:25.056603] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-09. [2025-10-20T17:01:25.194381] [DEBUG] [VALIDATOR ] ACCO: All expected trading days are present. [2025-10-20T17:01:25.194471] [DEBUG] [VALIDATOR ] ACCO: Primary data validation passed. [2025-10-20T17:01:25.195018] [INFO ] [MAIN ] ACCO: Fetched primary data is valid. [2025-10-20T17:01:25.195081] [DEBUG] [LOCAL ] ACCO: Saving primary data. [2025-10-20T17:01:25.196941] [DEBUG] [LOCAL ] ACCO: Trimmed 54045 records to 36838 (keeping 200 days of data) [2025-10-20T17:01:25.410380] [DEBUG] [LOCAL ] ACCO: Primary data saved (36838 records). [2025-10-20T17:01:25.410561] [INFO ] [MAIN ] ACCO: Full history saved. [2025-10-20T17:01:25.410679] [INFO ] [MAIN ] Update Progress: 17/2367 symbols completed. ACCO took 25.4s. Estimated remaining: 3h 6m 38s. Rate: 12.6 symbols/min [2025-10-20T17:01:25.589811] [INFO ] [MAIN ] ACM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:25.632410] [DEBUG] [LOCAL ] ACM: Loaded 43560 existing records from primary storage. [2025-10-20T17:01:25.632771] [INFO ] [MAIN ] ACM: Loaded 43560 existing records. [2025-10-20T17:01:25.817895] [WARN ] [MAIN ] ACM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:25.818226] [INFO ] [MAIN ] ACM: Wiping existing primary data before full refresh. [2025-10-20T17:01:25.818301] [INFO ] [LOCAL ] ACM: Starting deletion of primary data. [2025-10-20T17:01:25.818751] [DEBUG] [LOCAL ] ACM: Deleted primary data file. [2025-10-20T17:01:25.818809] [INFO ] [MAIN ] ACM: Starting full primary refresh. [2025-10-20T17:01:25.862567] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-02. [2025-10-20T17:01:26.226795] [INFO ] [REMOTE ] ACM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:26.260553] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:01:26.380759] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:01:26.651372] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-01. [2025-10-20T17:01:27.451389] [INFO ] [REMOTE ] ACEL: Fetching primary data for 2025-10. [2025-10-20T17:01:28.251406] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-04. [2025-10-20T17:01:28.447022] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-02 (2.58 seconds). [2025-10-20T17:01:28.487150] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:01:28.629799] [INFO ] [REMOTE ] ACEL: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:01:28.635116] [INFO ] [REMOTE ] ACEL: Full fetch complete (38833 records). [2025-10-20T17:01:28.635304] [DEBUG] [VALIDATOR ] ACEL: Validating primary data. [2025-10-20T17:01:29.050751] [DEBUG] [VALIDATOR ] ACEL: All expected trading days are present. [2025-10-20T17:01:29.051076] [DEBUG] [VALIDATOR ] ACEL: Primary data validation passed. [2025-10-20T17:01:29.051534] [INFO ] [MAIN ] ACEL: Fetched primary data is valid. [2025-10-20T17:01:29.051599] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-02. [2025-10-20T17:01:29.052796] [DEBUG] [LOCAL ] ACEL: Saving primary data. [2025-10-20T17:01:29.053776] [DEBUG] [LOCAL ] ACEL: Trimmed 38833 records to 26577 (keeping 200 days of data) [2025-10-20T17:01:29.220626] [DEBUG] [LOCAL ] ACEL: Primary data saved (26577 records). [2025-10-20T17:01:29.220814] [INFO ] [MAIN ] ACEL: Full history saved. [2025-10-20T17:01:29.220898] [INFO ] [MAIN ] Update Progress: 18/2367 symbols completed. ACEL took 20.7s. Estimated remaining: 3h 4m 28s. Rate: 12.7 symbols/min [2025-10-20T17:01:29.407239] [INFO ] [MAIN ] ACN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:29.460497] [DEBUG] [LOCAL ] ACN: Loaded 54216 existing records from primary storage. [2025-10-20T17:01:29.460942] [INFO ] [MAIN ] ACN: Loaded 54216 existing records. [2025-10-20T17:01:29.679128] [WARN ] [MAIN ] ACN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:29.679372] [INFO ] [MAIN ] ACN: Wiping existing primary data before full refresh. [2025-10-20T17:01:29.679421] [INFO ] [LOCAL ] ACN: Starting deletion of primary data. [2025-10-20T17:01:29.679967] [DEBUG] [LOCAL ] ACN: Deleted primary data file. [2025-10-20T17:01:29.680028] [INFO ] [MAIN ] ACN: Starting full primary refresh. [2025-10-20T17:01:29.859808] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-03. [2025-10-20T17:01:30.096588] [INFO ] [REMOTE ] ACN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:30.132901] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:01:30.307213] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T17:01:30.651502] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-01. [2025-10-20T17:01:31.451379] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-03. [2025-10-20T17:01:32.251354] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-05. [2025-10-20T17:01:32.440497] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:01:32.665613] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-03 (2.81 seconds). [2025-10-20T17:01:33.051353] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-02. [2025-10-20T17:01:33.087592] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:01:33.851359] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-04. [2025-10-20T17:01:34.061693] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:01:34.651368] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-04. [2025-10-20T17:01:34.824228] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T17:01:35.227510] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:01:35.451374] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-03. [2025-10-20T17:01:36.251424] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-06. [2025-10-20T17:01:37.051369] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-05. [2025-10-20T17:01:37.402686] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-04 (2.75 seconds). [2025-10-20T17:01:37.492291] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T17:01:37.851358] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-05. [2025-10-20T17:01:37.993816] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:01:38.661725] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-04. [2025-10-20T17:01:38.665010] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:01:39.451396] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-07. [2025-10-20T17:01:40.251419] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-06. [2025-10-20T17:01:40.436615] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-05 (2.59 seconds). [2025-10-20T17:01:40.743757] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T17:01:41.051416] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-06. [2025-10-20T17:01:41.429618] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:01:41.743500] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:01:41.851438] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-05. [2025-10-20T17:01:42.651398] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-07. [2025-10-20T17:01:43.439834] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-06 (2.39 seconds). [2025-10-20T17:01:43.451336] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-07. [2025-10-20T17:01:43.758922] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T17:01:44.251348] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-06. [2025-10-20T17:01:44.348756] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T17:01:45.051466] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-08. [2025-10-20T17:01:45.851432] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-08. [2025-10-20T17:01:45.933040] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:01:46.326571] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-07 (2.88 seconds). [2025-10-20T17:01:46.581926] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:01:46.651597] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-09. [2025-10-20T17:01:47.383939] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:01:47.451436] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-08. [2025-10-20T17:01:48.116936] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:01:48.251414] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-07. [2025-10-20T17:01:49.051377] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-09. [2025-10-20T17:01:49.851400] [INFO ] [REMOTE ] ACI: Fetching primary data for 2025-10. [2025-10-20T17:01:49.932225] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-08 (2.48 seconds). [2025-10-20T17:01:50.496226] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-07 (2.24 seconds). [2025-10-20T17:01:50.651352] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-08. [2025-10-20T17:01:50.793859] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:01:51.451389] [INFO ] [REMOTE ] ACM: Fetching primary data for 2025-10. [2025-10-20T17:01:51.958555] [INFO ] [REMOTE ] ACI: Incremental fetch complete for 2025-10 (2.11 seconds). [2025-10-20T17:01:51.966510] [INFO ] [REMOTE ] ACI: Full fetch complete (76843 records). [2025-10-20T17:01:51.966711] [DEBUG] [VALIDATOR ] ACI: Validating primary data. [2025-10-20T17:01:51.967586] [FATAL] [VALIDATOR ] ACI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:01:51.967702] [FATAL] [MAIN ] ACI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:01:51.967752] [WARN ] [MAIN ] ACI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:01:51.967820] [INFO ] [LOCAL ] ACI: Starting deletion of primary data. [2025-10-20T17:01:51.967916] [DEBUG] [LOCAL ] ACI: No primary data file found to delete. [2025-10-20T17:01:51.967979] [INFO ] [LOCAL ] ACI: Starting deletion of derivative data. [2025-10-20T17:01:51.968159] [DEBUG] [LOCAL ] ACI: Deleted derivative data file. [2025-10-20T17:01:51.970074] [INFO ] [MAIN ] Update Progress: 19/2367 symbols completed. ACI took 37.6s. Estimated remaining: 3h 41m 33s. Rate: 10.6 symbols/min [2025-10-20T17:01:52.165402] [INFO ] [MAIN ] ACP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:52.166534] [DEBUG] [LOCAL ] ACP: No existing data file found in primary storage. [2025-10-20T17:01:52.166587] [WARN ] [MAIN ] ACP: No existing data found, performing full refresh instead. [2025-10-20T17:01:52.166736] [INFO ] [MAIN ] ACP: Wiping existing primary data before full refresh. [2025-10-20T17:01:52.166783] [INFO ] [LOCAL ] ACP: Starting deletion of primary data. [2025-10-20T17:01:52.166827] [DEBUG] [LOCAL ] ACP: No primary data file found to delete. [2025-10-20T17:01:52.166860] [INFO ] [MAIN ] ACP: Starting full primary refresh. [2025-10-20T17:01:52.258763] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-09. [2025-10-20T17:01:52.546065] [INFO ] [REMOTE ] ACP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:52.613987] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T17:01:52.856917] [INFO ] [REMOTE ] ACM: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:01:52.863500] [INFO ] [REMOTE ] ACM: Full fetch complete (63289 records). [2025-10-20T17:01:52.863665] [DEBUG] [VALIDATOR ] ACM: Validating primary data. [2025-10-20T17:01:53.052883] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-09. [2025-10-20T17:01:53.271862] [DEBUG] [VALIDATOR ] ACM: All expected trading days are present. [2025-10-20T17:01:53.271949] [DEBUG] [VALIDATOR ] ACM: Primary data validation passed. [2025-10-20T17:01:53.272578] [INFO ] [MAIN ] ACM: Fetched primary data is valid. [2025-10-20T17:01:53.272622] [DEBUG] [LOCAL ] ACM: Saving primary data. [2025-10-20T17:01:53.273961] [DEBUG] [LOCAL ] ACM: Trimmed 63289 records to 42890 (keeping 200 days of data) [2025-10-20T17:01:53.548779] [DEBUG] [LOCAL ] ACM: Primary data saved (42890 records). [2025-10-20T17:01:53.548919] [INFO ] [MAIN ] ACM: Full history saved. [2025-10-20T17:01:53.550278] [INFO ] [MAIN ] Update Progress: 20/2367 symbols completed. ACM took 28.1s. Estimated remaining: 3h 33m 28s. Rate: 11.0 symbols/min [2025-10-20T17:01:53.727462] [INFO ] [MAIN ] ACR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:01:53.736496] [DEBUG] [LOCAL ] ACR: Loaded 5512 existing records from primary storage. [2025-10-20T17:01:53.736754] [INFO ] [MAIN ] ACR: Loaded 5512 existing records. [2025-10-20T17:01:53.856513] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-01. [2025-10-20T17:01:53.929308] [WARN ] [MAIN ] ACR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:01:53.929511] [INFO ] [MAIN ] ACR: Wiping existing primary data before full refresh. [2025-10-20T17:01:53.929561] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-10-20T17:01:53.929727] [DEBUG] [LOCAL ] ACR: Deleted primary data file. [2025-10-20T17:01:53.929785] [INFO ] [MAIN ] ACR: Starting full primary refresh. [2025-10-20T17:01:54.311698] [INFO ] [REMOTE ] ACR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:01:54.651332] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-01. [2025-10-20T17:01:55.054838] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-09 (2.80 seconds). [2025-10-20T17:01:55.141144] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T17:01:55.451414] [INFO ] [REMOTE ] ACHR: Fetching primary data for 2025-10. [2025-10-20T17:01:55.504992] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:01:55.504848] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:01:56.251438] [INFO ] [REMOTE ] ACN: Fetching primary data for 2025-10. [2025-10-20T17:01:57.051444] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-02. [2025-10-20T17:01:57.851380] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-02. [2025-10-20T17:01:58.269484] [INFO ] [REMOTE ] ACHR: Incremental fetch complete for 2025-10 (2.82 seconds). [2025-10-20T17:01:58.278112] [INFO ] [REMOTE ] ACHR: Full fetch complete (77998 records). [2025-10-20T17:01:58.278273] [DEBUG] [VALIDATOR ] ACHR: Validating primary data. [2025-10-20T17:01:58.577355] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T17:01:58.661186] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-03. [2025-10-20T17:01:58.647940] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:01:58.732123] [DEBUG] [VALIDATOR ] ACHR: All expected trading days are present. [2025-10-20T17:01:58.732473] [DEBUG] [VALIDATOR ] ACHR: Primary data validation passed. [2025-10-20T17:01:58.733334] [INFO ] [MAIN ] ACHR: Fetched primary data is valid. [2025-10-20T17:01:58.733414] [DEBUG] [LOCAL ] ACHR: Saving primary data. [2025-10-20T17:01:58.734992] [DEBUG] [LOCAL ] ACHR: Trimmed 77998 records to 53818 (keeping 200 days of data) [2025-10-20T17:01:58.842544] [INFO ] [REMOTE ] ACN: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T17:01:58.863036] [INFO ] [REMOTE ] ACN: Full fetch complete (77401 records). [2025-10-20T17:01:58.864693] [DEBUG] [VALIDATOR ] ACN: Validating primary data. [2025-10-20T17:01:59.463709] [DEBUG] [VALIDATOR ] ACN: All expected trading days are present. [2025-10-20T17:01:59.458944] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-03. [2025-10-20T17:01:59.463834] [DEBUG] [VALIDATOR ] ACN: Primary data validation passed. [2025-10-20T17:01:59.475105] [INFO ] [MAIN ] ACN: Fetched primary data is valid. [2025-10-20T17:01:59.479310] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:01:59.483716] [DEBUG] [LOCAL ] ACN: Saving primary data. [2025-10-20T17:01:59.491549] [DEBUG] [LOCAL ] ACN: Trimmed 77401 records to 53440 (keeping 200 days of data) [2025-10-20T17:01:59.563831] [DEBUG] [LOCAL ] ACHR: Primary data saved (53818 records). [2025-10-20T17:01:59.594094] [INFO ] [MAIN ] ACHR: Full history saved. [2025-10-20T17:01:59.603034] [INFO ] [MAIN ] Update Progress: 21/2367 symbols completed. ACHR took 48.8s. Estimated remaining: 3h 34m 29s. Rate: 10.9 symbols/min [2025-10-20T17:01:59.874812] [INFO ] [MAIN ] ACRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:00.042087] [DEBUG] [LOCAL ] ACRE: Loaded 34100 existing records from primary storage. [2025-10-20T17:02:00.047743] [INFO ] [MAIN ] ACRE: Loaded 34100 existing records. [2025-10-20T17:02:00.159608] [DEBUG] [LOCAL ] ACN: Primary data saved (53440 records). [2025-10-20T17:02:00.162234] [INFO ] [MAIN ] ACN: Full history saved. [2025-10-20T17:02:00.163053] [INFO ] [MAIN ] Update Progress: 22/2367 symbols completed. ACN took 30.9s. Estimated remaining: 3h 25m 39s. Rate: 11.4 symbols/min [2025-10-20T17:02:00.265842] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-04. [2025-10-20T17:02:00.320405] [WARN ] [MAIN ] ACRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:00.350479] [INFO ] [MAIN ] ACRE: Wiping existing primary data before full refresh. [2025-10-20T17:02:00.366956] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-10-20T17:02:00.425691] [DEBUG] [LOCAL ] ACRE: Deleted primary data file. [2025-10-20T17:02:00.425615] [INFO ] [MAIN ] ACV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:00.425940] [INFO ] [MAIN ] ACRE: Starting full primary refresh. [2025-10-20T17:02:00.553282] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:02:00.563938] [DEBUG] [LOCAL ] ACV: Loaded 7718 existing records from primary storage. [2025-10-20T17:02:00.575810] [INFO ] [MAIN ] ACV: Loaded 7718 existing records. [2025-10-20T17:02:00.890831] [WARN ] [MAIN ] ACV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:00.908453] [INFO ] [MAIN ] ACV: Wiping existing primary data before full refresh. [2025-10-20T17:02:00.928855] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-10-20T17:02:00.953379] [DEBUG] [LOCAL ] ACV: Deleted primary data file. [2025-10-20T17:02:00.986581] [INFO ] [MAIN ] ACV: Starting full primary refresh. [2025-10-20T17:02:01.057701] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-04. [2025-10-20T17:02:01.073714] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:02:01.245418] [INFO ] [REMOTE ] ACRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:01.519619] [INFO ] [REMOTE ] ACV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:01.851356] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-01. [2025-10-20T17:02:02.636395] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:02:02.651402] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-05. [2025-10-20T17:02:03.007876] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T17:02:03.351261] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:02:03.451404] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-06. [2025-10-20T17:02:03.872501] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T17:02:04.251409] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-07. [2025-10-20T17:02:04.720878] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T17:02:05.051369] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-08. [2025-10-20T17:02:05.564021] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T17:02:05.851365] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-09. [2025-10-20T17:02:06.233270] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T17:02:06.651385] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-05. [2025-10-20T17:02:07.451449] [INFO ] [REMOTE ] ACR: Fetching primary data for 2025-10. [2025-10-20T17:02:07.899792] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:02:08.018491] [INFO ] [REMOTE ] ACR: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T17:02:08.020122] [INFO ] [REMOTE ] ACR: Full fetch complete (8534 records). [2025-10-20T17:02:08.020220] [DEBUG] [VALIDATOR ] ACR: Validating primary data. [2025-10-20T17:02:08.020811] [FATAL] [VALIDATOR ] ACR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:08.021064] [FATAL] [MAIN ] ACR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:08.021120] [WARN ] [MAIN ] ACR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:08.021202] [INFO ] [LOCAL ] ACR: Starting deletion of primary data. [2025-10-20T17:02:08.021283] [DEBUG] [LOCAL ] ACR: No primary data file found to delete. [2025-10-20T17:02:08.021332] [INFO ] [LOCAL ] ACR: Starting deletion of derivative data. [2025-10-20T17:02:08.021498] [DEBUG] [LOCAL ] ACR: Deleted derivative data file. [2025-10-20T17:02:08.021611] [INFO ] [MAIN ] Update Progress: 23/2367 symbols completed. ACR took 14.5s. Estimated remaining: 3h 29m 58s. Rate: 11.2 symbols/min [2025-10-20T17:02:08.197998] [INFO ] [MAIN ] ACVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:08.198176] [DEBUG] [LOCAL ] ACVA: No existing data file found in primary storage. [2025-10-20T17:02:08.198213] [WARN ] [MAIN ] ACVA: No existing data found, performing full refresh instead. [2025-10-20T17:02:08.198319] [INFO ] [MAIN ] ACVA: Wiping existing primary data before full refresh. [2025-10-20T17:02:08.198363] [INFO ] [LOCAL ] ACVA: Starting deletion of primary data. [2025-10-20T17:02:08.198450] [DEBUG] [LOCAL ] ACVA: No primary data file found to delete. [2025-10-20T17:02:08.198490] [INFO ] [MAIN ] ACVA: Starting full primary refresh. [2025-10-20T17:02:08.259244] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-06. [2025-10-20T17:02:08.558750] [INFO ] [REMOTE ] ACVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:09.051415] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-01. [2025-10-20T17:02:09.270207] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:02:09.500370] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:02:09.851381] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-02. [2025-10-20T17:02:10.352011] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T17:02:10.651384] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-07. [2025-10-20T17:02:11.451404] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-01. [2025-10-20T17:02:11.599091] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:02:12.251382] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-08. [2025-10-20T17:02:12.837311] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:02:13.051358] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-02. [2025-10-20T17:02:13.508216] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:02:13.851384] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-02. [2025-10-20T17:02:14.495487] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T17:02:14.651383] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-09. [2025-10-20T17:02:15.031273] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T17:02:15.451404] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-03. [2025-10-20T17:02:15.771494] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:02:16.251375] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-03. [2025-10-20T17:02:17.002567] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:02:17.051378] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-04. [2025-10-20T17:02:17.671816] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:02:17.851387] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-04. [2025-10-20T17:02:18.651355] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-03. [2025-10-20T17:02:18.843078] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:02:19.319519] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:02:19.404861] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T17:02:19.451338] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-04. [2025-10-20T17:02:20.251417] [INFO ] [REMOTE ] ACP: Fetching primary data for 2025-10. [2025-10-20T17:02:20.355277] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T17:02:21.051381] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-05. [2025-10-20T17:02:21.665125] [INFO ] [REMOTE ] ACP: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:02:21.670400] [INFO ] [REMOTE ] ACP: Full fetch complete (46459 records). [2025-10-20T17:02:21.670587] [DEBUG] [VALIDATOR ] ACP: Validating primary data. [2025-10-20T17:02:21.857117] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-05. [2025-10-20T17:02:22.065230] [DEBUG] [VALIDATOR ] ACP: All expected trading days are present. [2025-10-20T17:02:22.065318] [DEBUG] [VALIDATOR ] ACP: Primary data validation passed. [2025-10-20T17:02:22.065812] [INFO ] [MAIN ] ACP: Fetched primary data is valid. [2025-10-20T17:02:22.065873] [DEBUG] [LOCAL ] ACP: Saving primary data. [2025-10-20T17:02:22.066887] [DEBUG] [LOCAL ] ACP: Trimmed 46459 records to 31109 (keeping 200 days of data) [2025-10-20T17:02:22.237721] [DEBUG] [LOCAL ] ACP: Primary data saved (31109 records). [2025-10-20T17:02:22.237859] [INFO ] [MAIN ] ACP: Full history saved. [2025-10-20T17:02:22.237932] [INFO ] [MAIN ] Update Progress: 24/2367 symbols completed. ACP took 30.3s. Estimated remaining: 3h 44m 16s. Rate: 10.4 symbols/min [2025-10-20T17:02:22.415198] [INFO ] [MAIN ] AD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:22.415408] [DEBUG] [LOCAL ] AD: No existing data file found in primary storage. [2025-10-20T17:02:22.415451] [WARN ] [MAIN ] AD: No existing data found, performing full refresh instead. [2025-10-20T17:02:22.415563] [INFO ] [MAIN ] AD: Wiping existing primary data before full refresh. [2025-10-20T17:02:22.415604] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-10-20T17:02:22.415669] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-10-20T17:02:22.415724] [INFO ] [MAIN ] AD: Starting full primary refresh. [2025-10-20T17:02:22.673422] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-05. [2025-10-20T17:02:22.690493] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:02:22.823313] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:02:22.821314] [INFO ] [REMOTE ] AD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:23.451408] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-06. [2025-10-20T17:02:24.079866] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T17:02:24.235928] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:02:24.251366] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-06. [2025-10-20T17:02:25.051387] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-06. [2025-10-20T17:02:25.690830] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:02:25.851360] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-07. [2025-10-20T17:02:26.493859] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:02:26.651408] [INFO ] [REMOTE ] AD: Fetching primary data for 2025-01. [2025-10-20T17:02:26.862071] [DEBUG] [REMOTE ] AD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:02:26.862387] [WARN ] [REMOTE ] AD: No data returned for month 2025-01 [2025-10-20T17:02:26.862473] [FATAL] [REMOTE ] AD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:02:26.862548] [FATAL] [MAIN ] AD: Failed to fetch primary data. [2025-10-20T17:02:26.862632] [WARN ] [MAIN ] AD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:02:26.862729] [INFO ] [LOCAL ] AD: Starting deletion of primary data. [2025-10-20T17:02:26.862813] [DEBUG] [LOCAL ] AD: No primary data file found to delete. [2025-10-20T17:02:26.862851] [INFO ] [LOCAL ] AD: Starting deletion of derivative data. [2025-10-20T17:02:26.862891] [DEBUG] [LOCAL ] AD: No derivative data file found to delete. [2025-10-20T17:02:26.862937] [INFO ] [MAIN ] Update Progress: 25/2367 symbols completed. AD took 4.6s. Estimated remaining: 3h 42m 25s. Rate: 10.5 symbols/min [2025-10-20T17:02:27.039542] [INFO ] [MAIN ] ADC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:27.081620] [DEBUG] [LOCAL ] ADC: Loaded 47056 existing records from primary storage. [2025-10-20T17:02:27.081951] [INFO ] [MAIN ] ADC: Loaded 47056 existing records. [2025-10-20T17:02:27.274434] [WARN ] [MAIN ] ADC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:27.274899] [INFO ] [MAIN ] ADC: Wiping existing primary data before full refresh. [2025-10-20T17:02:27.275001] [INFO ] [LOCAL ] ADC: Starting deletion of primary data. [2025-10-20T17:02:27.275280] [DEBUG] [LOCAL ] ADC: Deleted primary data file. [2025-10-20T17:02:27.275338] [INFO ] [MAIN ] ADC: Starting full primary refresh. [2025-10-20T17:02:27.454010] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-07. [2025-10-20T17:02:27.661667] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:02:27.693528] [INFO ] [REMOTE ] ADC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:28.239636] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:02:28.251322] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-08. [2025-10-20T17:02:29.027470] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T17:02:29.051436] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-08. [2025-10-20T17:02:29.851390] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-09. [2025-10-20T17:02:30.493768] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:02:30.651405] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-09. [2025-10-20T17:02:30.675947] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T17:02:31.451377] [INFO ] [REMOTE ] ACV: Fetching primary data for 2025-10. [2025-10-20T17:02:32.251406] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-07. [2025-10-20T17:02:32.359256] [INFO ] [REMOTE ] ACV: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:02:32.361700] [INFO ] [REMOTE ] ACV: Full fetch complete (10342 records). [2025-10-20T17:02:32.361799] [DEBUG] [VALIDATOR ] ACV: Validating primary data. [2025-10-20T17:02:32.362224] [FATAL] [VALIDATOR ] ACV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:32.362314] [FATAL] [MAIN ] ACV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:32.362425] [WARN ] [MAIN ] ACV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:32.362723] [INFO ] [LOCAL ] ACV: Starting deletion of primary data. [2025-10-20T17:02:32.362813] [DEBUG] [LOCAL ] ACV: No primary data file found to delete. [2025-10-20T17:02:32.362859] [INFO ] [LOCAL ] ACV: Starting deletion of derivative data. [2025-10-20T17:02:32.362996] [DEBUG] [LOCAL ] ACV: Deleted derivative data file. [2025-10-20T17:02:32.363070] [INFO ] [MAIN ] Update Progress: 26/2367 symbols completed. ACV took 32.2s. Estimated remaining: 3h 42m 2s. Rate: 10.5 symbols/min [2025-10-20T17:02:32.598925] [INFO ] [MAIN ] ADCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:32.599116] [DEBUG] [LOCAL ] ADCT: No existing data file found in primary storage. [2025-10-20T17:02:32.599339] [WARN ] [MAIN ] ADCT: No existing data found, performing full refresh instead. [2025-10-20T17:02:32.600900] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T17:02:32.600993] [INFO ] [MAIN ] ADCT: Wiping existing primary data before full refresh. [2025-10-20T17:02:32.610938] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-10-20T17:02:32.611100] [DEBUG] [LOCAL ] ADCT: No primary data file found to delete. [2025-10-20T17:02:32.611153] [INFO ] [MAIN ] ADCT: Starting full primary refresh. [2025-10-20T17:02:32.982165] [INFO ] [REMOTE ] ADCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:33.051586] [INFO ] [REMOTE ] ACVA: Fetching primary data for 2025-10. [2025-10-20T17:02:33.505792] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:02:33.851410] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-01. [2025-10-20T17:02:34.651388] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-01. [2025-10-20T17:02:34.995542] [INFO ] [REMOTE ] ACVA: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:02:35.002957] [INFO ] [REMOTE ] ACVA: Full fetch complete (70994 records). [2025-10-20T17:02:35.003073] [DEBUG] [VALIDATOR ] ACVA: Validating primary data. [2025-10-20T17:02:35.363096] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T17:02:35.425161] [DEBUG] [VALIDATOR ] ACVA: All expected trading days are present. [2025-10-20T17:02:35.425231] [DEBUG] [VALIDATOR ] ACVA: Primary data validation passed. [2025-10-20T17:02:35.425904] [INFO ] [MAIN ] ACVA: Fetched primary data is valid. [2025-10-20T17:02:35.425967] [DEBUG] [LOCAL ] ACVA: Saving primary data. [2025-10-20T17:02:35.427599] [DEBUG] [LOCAL ] ACVA: Trimmed 70994 records to 49252 (keeping 200 days of data) [2025-10-20T17:02:35.474353] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-02. [2025-10-20T17:02:35.714464] [DEBUG] [LOCAL ] ACVA: Primary data saved (49252 records). [2025-10-20T17:02:35.714618] [INFO ] [MAIN ] ACVA: Full history saved. [2025-10-20T17:02:35.714726] [INFO ] [MAIN ] Update Progress: 27/2367 symbols completed. ACVA took 27.7s. Estimated remaining: 3h 38m 34s. Rate: 10.7 symbols/min [2025-10-20T17:02:35.893778] [INFO ] [MAIN ] ADM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:35.946279] [DEBUG] [LOCAL ] ADM: Loaded 54178 existing records from primary storage. [2025-10-20T17:02:35.946833] [INFO ] [MAIN ] ADM: Loaded 54178 existing records. [2025-10-20T17:02:36.133839] [WARN ] [MAIN ] ADM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:36.134614] [INFO ] [MAIN ] ADM: Wiping existing primary data before full refresh. [2025-10-20T17:02:36.134730] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-10-20T17:02:36.135006] [DEBUG] [LOCAL ] ADM: Deleted primary data file. [2025-10-20T17:02:36.135067] [INFO ] [MAIN ] ADM: Starting full primary refresh. [2025-10-20T17:02:36.254582] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-08. [2025-10-20T17:02:36.497637] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T17:02:36.538076] [INFO ] [REMOTE ] ADM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:36.589783] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:02:37.051445] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-01. [2025-10-20T17:02:37.637081] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:02:37.851404] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-02. [2025-10-20T17:02:38.651403] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-03. [2025-10-20T17:02:38.997295] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:02:39.248207] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:02:39.451389] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-03. [2025-10-20T17:02:39.530670] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:02:40.251440] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-09. [2025-10-20T17:02:41.051386] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-04. [2025-10-20T17:02:41.091698] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:02:41.182383] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T17:02:41.851899] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T17:02:41.851971] [INFO ] [REMOTE ] ACRE: Fetching primary data for 2025-10. [2025-10-20T17:02:42.651407] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-05. [2025-10-20T17:02:43.103412] [INFO ] [REMOTE ] ACRE: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:02:43.108205] [INFO ] [REMOTE ] ACRE: Full fetch complete (49468 records). [2025-10-20T17:02:43.108350] [DEBUG] [VALIDATOR ] ACRE: Validating primary data. [2025-10-20T17:02:43.109023] [FATAL] [VALIDATOR ] ACRE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:43.109108] [FATAL] [MAIN ] ACRE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:43.109153] [WARN ] [MAIN ] ACRE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:43.109233] [INFO ] [LOCAL ] ACRE: Starting deletion of primary data. [2025-10-20T17:02:43.109312] [DEBUG] [LOCAL ] ACRE: No primary data file found to delete. [2025-10-20T17:02:43.109360] [INFO ] [LOCAL ] ACRE: Starting deletion of derivative data. [2025-10-20T17:02:43.109576] [DEBUG] [LOCAL ] ACRE: Deleted derivative data file. [2025-10-20T17:02:43.110227] [INFO ] [MAIN ] Update Progress: 28/2367 symbols completed. ACRE took 43.5s. Estimated remaining: 3h 40m 58s. Rate: 10.6 symbols/min [2025-10-20T17:02:43.284805] [INFO ] [MAIN ] ADNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:43.329255] [DEBUG] [LOCAL ] ADNT: Loaded 47379 existing records from primary storage. [2025-10-20T17:02:43.329580] [INFO ] [MAIN ] ADNT: Loaded 47379 existing records. [2025-10-20T17:02:43.456558] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-02. [2025-10-20T17:02:43.522754] [WARN ] [MAIN ] ADNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:43.523340] [INFO ] [MAIN ] ADNT: Wiping existing primary data before full refresh. [2025-10-20T17:02:43.523404] [INFO ] [LOCAL ] ADNT: Starting deletion of primary data. [2025-10-20T17:02:43.523833] [DEBUG] [LOCAL ] ADNT: Deleted primary data file. [2025-10-20T17:02:43.523892] [INFO ] [MAIN ] ADNT: Starting full primary refresh. [2025-10-20T17:02:43.876590] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:02:43.952605] [INFO ] [REMOTE ] ADNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:44.251464] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-01. [2025-10-20T17:02:44.707611] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:02:45.051393] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-06. [2025-10-20T17:02:45.851395] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-03. [2025-10-20T17:02:45.989855] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:02:46.651393] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-02. [2025-10-20T17:02:46.877888] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T17:02:47.356216] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:02:47.451372] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-07. [2025-10-20T17:02:47.810937] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:02:48.251379] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-03. [2025-10-20T17:02:48.837508] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:02:49.051461] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-08. [2025-10-20T17:02:49.788389] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:02:49.851401] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-04. [2025-10-20T17:02:50.311784] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:02:50.651437] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-04. [2025-10-20T17:02:51.451366] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-04. [2025-10-20T17:02:51.783949] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T17:02:51.916908] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:02:52.251377] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-05. [2025-10-20T17:02:53.033617] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:02:53.051333] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-05. [2025-10-20T17:02:53.851372] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-09. [2025-10-20T17:02:54.651356] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-05. [2025-10-20T17:02:54.823980] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-05 (2.57 seconds). [2025-10-20T17:02:55.009529] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T17:02:55.451379] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-06. [2025-10-20T17:02:55.917141] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:02:56.047637] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:02:56.251397] [INFO ] [REMOTE ] ADCT: Fetching primary data for 2025-10. [2025-10-20T17:02:56.880198] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T17:02:57.051454] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-07. [2025-10-20T17:02:57.842730] [INFO ] [REMOTE ] ADCT: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T17:02:57.849314] [INFO ] [REMOTE ] ADCT: Full fetch complete (51212 records). [2025-10-20T17:02:57.849456] [DEBUG] [VALIDATOR ] ADCT: Validating primary data. [2025-10-20T17:02:57.850214] [FATAL] [VALIDATOR ] ADCT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:02:57.850311] [FATAL] [MAIN ] ADCT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:02:57.850363] [WARN ] [MAIN ] ADCT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:02:57.850442] [INFO ] [LOCAL ] ADCT: Starting deletion of primary data. [2025-10-20T17:02:57.850761] [DEBUG] [LOCAL ] ADCT: No primary data file found to delete. [2025-10-20T17:02:57.851017] [INFO ] [LOCAL ] ADCT: Starting deletion of derivative data. [2025-10-20T17:02:57.851084] [DEBUG] [LOCAL ] ADCT: No derivative data file found to delete. [2025-10-20T17:02:57.851192] [INFO ] [MAIN ] Update Progress: 29/2367 symbols completed. ADCT took 25.5s. Estimated remaining: 3h 53m 3s. Rate: 10.0 symbols/min [2025-10-20T17:02:57.851390] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-06. [2025-10-20T17:02:58.024261] [INFO ] [MAIN ] ADT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:02:58.073040] [DEBUG] [LOCAL ] ADT: Loaded 54217 existing records from primary storage. [2025-10-20T17:02:58.073383] [INFO ] [MAIN ] ADT: Loaded 54217 existing records. [2025-10-20T17:02:58.248251] [WARN ] [MAIN ] ADT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:02:58.248459] [INFO ] [MAIN ] ADT: Wiping existing primary data before full refresh. [2025-10-20T17:02:58.248547] [INFO ] [LOCAL ] ADT: Starting deletion of primary data. [2025-10-20T17:02:58.249413] [DEBUG] [LOCAL ] ADT: Deleted primary data file. [2025-10-20T17:02:58.249470] [INFO ] [MAIN ] ADT: Starting full primary refresh. [2025-10-20T17:02:58.646159] [INFO ] [REMOTE ] ADT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:02:58.658899] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-06. [2025-10-20T17:02:58.681302] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:02:59.044033] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:02:59.451468] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-07. [2025-10-20T17:03:00.251359] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-01. [2025-10-20T17:03:00.434979] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:03:00.805779] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T17:03:01.051356] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-08. [2025-10-20T17:03:01.670079] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:03:01.851360] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-07. [2025-10-20T17:03:02.651426] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-08. [2025-10-20T17:03:02.696400] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:03:03.451385] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-09. [2025-10-20T17:03:04.056017] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:03:04.102215] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-07 (2.25 seconds). [2025-10-20T17:03:04.251403] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-09. [2025-10-20T17:03:04.826917] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:03:05.051433] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-08. [2025-10-20T17:03:05.771559] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T17:03:05.851393] [INFO ] [REMOTE ] ADNT: Fetching primary data for 2025-10. [2025-10-20T17:03:06.651417] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-02. [2025-10-20T17:03:07.451477] [INFO ] [REMOTE ] ADC: Fetching primary data for 2025-10. [2025-10-20T17:03:07.517705] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-08 (2.47 seconds). [2025-10-20T17:03:07.706433] [INFO ] [REMOTE ] ADNT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:03:07.712936] [INFO ] [REMOTE ] ADNT: Full fetch complete (67975 records). [2025-10-20T17:03:07.713119] [DEBUG] [VALIDATOR ] ADNT: Validating primary data. [2025-10-20T17:03:08.169069] [DEBUG] [VALIDATOR ] ADNT: All expected trading days are present. [2025-10-20T17:03:08.169191] [DEBUG] [VALIDATOR ] ADNT: Primary data validation passed. [2025-10-20T17:03:08.169987] [INFO ] [MAIN ] ADNT: Fetched primary data is valid. [2025-10-20T17:03:08.170070] [DEBUG] [LOCAL ] ADNT: Saving primary data. [2025-10-20T17:03:08.171752] [DEBUG] [LOCAL ] ADNT: Trimmed 67975 records to 46530 (keeping 200 days of data) [2025-10-20T17:03:08.252908] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-09. [2025-10-20T17:03:08.295559] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:03:08.520509] [DEBUG] [LOCAL ] ADNT: Primary data saved (46530 records). [2025-10-20T17:03:08.520691] [INFO ] [MAIN ] ADNT: Full history saved. [2025-10-20T17:03:08.521784] [INFO ] [MAIN ] Update Progress: 30/2367 symbols completed. ADNT took 25.4s. Estimated remaining: 3h 59m 3s. Rate: 9.8 symbols/min [2025-10-20T17:03:08.728515] [INFO ] [MAIN ] ADX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:08.763756] [DEBUG] [LOCAL ] ADX: Loaded 28160 existing records from primary storage. [2025-10-20T17:03:08.764217] [INFO ] [MAIN ] ADX: Loaded 28160 existing records. [2025-10-20T17:03:08.980032] [INFO ] [REMOTE ] ADC: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:03:09.014479] [WARN ] [MAIN ] ADX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:09.015148] [INFO ] [MAIN ] ADX: Wiping existing primary data before full refresh. [2025-10-20T17:03:09.015216] [INFO ] [LOCAL ] ADX: Starting deletion of primary data. [2025-10-20T17:03:09.015707] [DEBUG] [LOCAL ] ADX: Deleted primary data file. [2025-10-20T17:03:09.015785] [INFO ] [MAIN ] ADX: Starting full primary refresh. [2025-10-20T17:03:09.015667] [INFO ] [REMOTE ] ADC: Full fetch complete (66615 records). [2025-10-20T17:03:09.015886] [DEBUG] [VALIDATOR ] ADC: Validating primary data. [2025-10-20T17:03:09.054120] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-03. [2025-10-20T17:03:09.865202] [INFO ] [REMOTE ] ADX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:09.886950] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-01. [2025-10-20T17:03:09.908588] [DEBUG] [VALIDATOR ] ADC: All expected trading days are present. [2025-10-20T17:03:09.909007] [DEBUG] [VALIDATOR ] ADC: Primary data validation passed. [2025-10-20T17:03:09.909828] [INFO ] [MAIN ] ADC: Fetched primary data is valid. [2025-10-20T17:03:09.910109] [DEBUG] [LOCAL ] ADC: Saving primary data. [2025-10-20T17:03:09.911995] [DEBUG] [LOCAL ] ADC: Trimmed 66615 records to 46431 (keeping 200 days of data) [2025-10-20T17:03:10.070545] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T17:03:10.250563] [DEBUG] [LOCAL ] ADC: Primary data saved (46431 records). [2025-10-20T17:03:10.250752] [INFO ] [MAIN ] ADC: Full history saved. [2025-10-20T17:03:10.251470] [INFO ] [MAIN ] Update Progress: 31/2367 symbols completed. ADC took 43.4s. Estimated remaining: 3h 53m 24s. Rate: 10.0 symbols/min [2025-10-20T17:03:10.423692] [INFO ] [MAIN ] AEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:10.423946] [DEBUG] [LOCAL ] AEE: No existing data file found in primary storage. [2025-10-20T17:03:10.424009] [WARN ] [MAIN ] AEE: No existing data found, performing full refresh instead. [2025-10-20T17:03:10.424510] [INFO ] [MAIN ] AEE: Wiping existing primary data before full refresh. [2025-10-20T17:03:10.424611] [INFO ] [LOCAL ] AEE: Starting deletion of primary data. [2025-10-20T17:03:10.424721] [DEBUG] [LOCAL ] AEE: No primary data file found to delete. [2025-10-20T17:03:10.424835] [INFO ] [MAIN ] AEE: Starting full primary refresh. [2025-10-20T17:03:10.665885] [INFO ] [REMOTE ] ADM: Fetching primary data for 2025-10. [2025-10-20T17:03:10.821442] [INFO ] [REMOTE ] AEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:10.826896] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:03:11.177133] [INFO ] [REMOTE ] ADM: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:03:11.185293] [INFO ] [REMOTE ] ADM: Full fetch complete (77067 records). [2025-10-20T17:03:11.185472] [DEBUG] [VALIDATOR ] ADM: Validating primary data. [2025-10-20T17:03:11.186573] [FATAL] [VALIDATOR ] ADM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:03:11.186690] [FATAL] [MAIN ] ADM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:03:11.186743] [WARN ] [MAIN ] ADM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:03:11.186813] [INFO ] [LOCAL ] ADM: Starting deletion of primary data. [2025-10-20T17:03:11.186882] [DEBUG] [LOCAL ] ADM: No primary data file found to delete. [2025-10-20T17:03:11.186919] [INFO ] [LOCAL ] ADM: Starting deletion of derivative data. [2025-10-20T17:03:11.187054] [DEBUG] [LOCAL ] ADM: Deleted derivative data file. [2025-10-20T17:03:11.187734] [INFO ] [MAIN ] Update Progress: 32/2367 symbols completed. ADM took 35.5s. Estimated remaining: 3h 47m 9s. Rate: 10.3 symbols/min [2025-10-20T17:03:11.380627] [INFO ] [MAIN ] AEFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:11.393213] [DEBUG] [LOCAL ] AEFC: Loaded 8348 existing records from primary storage. [2025-10-20T17:03:11.393459] [INFO ] [MAIN ] AEFC: Loaded 8348 existing records. [2025-10-20T17:03:11.458027] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-04. [2025-10-20T17:03:11.584764] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:03:11.611581] [WARN ] [MAIN ] AEFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:11.612724] [INFO ] [MAIN ] AEFC: Wiping existing primary data before full refresh. [2025-10-20T17:03:11.612780] [INFO ] [LOCAL ] AEFC: Starting deletion of primary data. [2025-10-20T17:03:11.612948] [DEBUG] [LOCAL ] AEFC: Deleted primary data file. [2025-10-20T17:03:11.612985] [INFO ] [MAIN ] AEFC: Starting full primary refresh. [2025-10-20T17:03:12.007630] [INFO ] [REMOTE ] AEFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:12.251375] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-01. [2025-10-20T17:03:12.978933] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:03:13.025124] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:03:13.051362] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-02. [2025-10-20T17:03:13.759834] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:03:13.851379] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-01. [2025-10-20T17:03:14.651452] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-05. [2025-10-20T17:03:15.173105] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:03:15.451399] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-02. [2025-10-20T17:03:15.968234] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:03:16.163846] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:03:16.251390] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-03. [2025-10-20T17:03:16.777853] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-10-20T17:03:17.051368] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-04. [2025-10-20T17:03:17.848165] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T17:03:17.851325] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-05. [2025-10-20T17:03:18.506067] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T17:03:18.651353] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-06. [2025-10-20T17:03:19.421787] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:03:19.451353] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-07. [2025-10-20T17:03:20.183688] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T17:03:20.251389] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-08. [2025-10-20T17:03:20.963386] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T17:03:21.051375] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-06. [2025-10-20T17:03:21.851335] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-09. [2025-10-20T17:03:22.597907] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:03:22.628867] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:03:22.651378] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-07. [2025-10-20T17:03:23.451395] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-02. [2025-10-20T17:03:24.251351] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-03. [2025-10-20T17:03:24.299242] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:03:25.010389] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T17:03:25.051387] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-03. [2025-10-20T17:03:25.851438] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-08. [2025-10-20T17:03:26.265535] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:03:26.651351] [INFO ] [REMOTE ] AEFC: Fetching primary data for 2025-10. [2025-10-20T17:03:26.879858] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T17:03:27.280417] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:03:27.352561] [INFO ] [REMOTE ] AEFC: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T17:03:27.354408] [INFO ] [REMOTE ] AEFC: Full fetch complete (11640 records). [2025-10-20T17:03:27.354542] [DEBUG] [VALIDATOR ] AEFC: Validating primary data. [2025-10-20T17:03:27.456716] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-04. [2025-10-20T17:03:27.781027] [DEBUG] [VALIDATOR ] AEFC: All expected trading days are present. [2025-10-20T17:03:27.781116] [DEBUG] [VALIDATOR ] AEFC: Primary data validation passed. [2025-10-20T17:03:27.781392] [INFO ] [MAIN ] AEFC: Fetched primary data is valid. [2025-10-20T17:03:27.781459] [DEBUG] [LOCAL ] AEFC: Saving primary data. [2025-10-20T17:03:27.782133] [DEBUG] [LOCAL ] AEFC: Trimmed 11640 records to 8025 (keeping 200 days of data) [2025-10-20T17:03:27.840106] [DEBUG] [LOCAL ] AEFC: Primary data saved (8025 records). [2025-10-20T17:03:27.840254] [INFO ] [MAIN ] AEFC: Full history saved. [2025-10-20T17:03:27.840339] [INFO ] [MAIN ] Update Progress: 33/2367 symbols completed. AEFC took 16.7s. Estimated remaining: 3h 59m 48s. Rate: 9.7 symbols/min [2025-10-20T17:03:28.066030] [INFO ] [MAIN ] AEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:28.118187] [DEBUG] [LOCAL ] AEG: Loaded 50300 existing records from primary storage. [2025-10-20T17:03:28.118555] [INFO ] [MAIN ] AEG: Loaded 50300 existing records. [2025-10-20T17:03:28.256576] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-04. [2025-10-20T17:03:28.313461] [WARN ] [MAIN ] AEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:28.313752] [INFO ] [MAIN ] AEG: Wiping existing primary data before full refresh. [2025-10-20T17:03:28.313823] [INFO ] [LOCAL ] AEG: Starting deletion of primary data. [2025-10-20T17:03:28.314311] [DEBUG] [LOCAL ] AEG: Deleted primary data file. [2025-10-20T17:03:28.314368] [INFO ] [MAIN ] AEG: Starting full primary refresh. [2025-10-20T17:03:28.712573] [INFO ] [REMOTE ] AEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:28.725566] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:03:29.051392] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-09. [2025-10-20T17:03:29.512313] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:03:29.851385] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-01. [2025-10-20T17:03:30.651353] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-05. [2025-10-20T17:03:30.884504] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:03:31.451356] [INFO ] [REMOTE ] ADT: Fetching primary data for 2025-10. [2025-10-20T17:03:31.495722] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T17:03:31.522697] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T17:03:32.251381] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-05. [2025-10-20T17:03:32.861674] [INFO ] [REMOTE ] ADT: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:03:32.869226] [INFO ] [REMOTE ] ADT: Full fetch complete (77363 records). [2025-10-20T17:03:32.869324] [DEBUG] [VALIDATOR ] ADT: Validating primary data. [2025-10-20T17:03:33.058975] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-06. [2025-10-20T17:03:33.269533] [DEBUG] [VALIDATOR ] ADT: All expected trading days are present. [2025-10-20T17:03:33.269688] [DEBUG] [VALIDATOR ] ADT: Primary data validation passed. [2025-10-20T17:03:33.271097] [INFO ] [MAIN ] ADT: Fetched primary data is valid. [2025-10-20T17:03:33.271886] [DEBUG] [LOCAL ] ADT: Saving primary data. [2025-10-20T17:03:33.273708] [DEBUG] [LOCAL ] ADT: Trimmed 77363 records to 53438 (keeping 200 days of data) [2025-10-20T17:03:33.601480] [DEBUG] [LOCAL ] ADT: Primary data saved (53438 records). [2025-10-20T17:03:33.601678] [INFO ] [MAIN ] ADT: Full history saved. [2025-10-20T17:03:33.602751] [INFO ] [MAIN ] Update Progress: 34/2367 symbols completed. ADT took 35.8s. Estimated remaining: 3h 59m 15s. Rate: 9.8 symbols/min [2025-10-20T17:03:33.819202] [INFO ] [MAIN ] AEM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:33.819531] [DEBUG] [LOCAL ] AEM: No existing data file found in primary storage. [2025-10-20T17:03:33.819603] [WARN ] [MAIN ] AEM: No existing data found, performing full refresh instead. [2025-10-20T17:03:33.819877] [INFO ] [MAIN ] AEM: Wiping existing primary data before full refresh. [2025-10-20T17:03:33.819964] [INFO ] [LOCAL ] AEM: Starting deletion of primary data. [2025-10-20T17:03:33.820052] [DEBUG] [LOCAL ] AEM: No primary data file found to delete. [2025-10-20T17:03:33.820213] [INFO ] [MAIN ] AEM: Starting full primary refresh. [2025-10-20T17:03:33.862397] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-02. [2025-10-20T17:03:33.967995] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:03:34.235266] [INFO ] [REMOTE ] AEM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:34.252764] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:03:34.651386] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-06. [2025-10-20T17:03:35.351714] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:03:35.451374] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-07. [2025-10-20T17:03:36.048370] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:03:36.251366] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-07. [2025-10-20T17:03:36.543472] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:03:37.051380] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-03. [2025-10-20T17:03:37.851370] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-08. [2025-10-20T17:03:37.923294] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T17:03:38.338202] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T17:03:38.651422] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-01. [2025-10-20T17:03:39.043797] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:03:39.451467] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-08. [2025-10-20T17:03:40.251362] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-09. [2025-10-20T17:03:40.368541] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:03:41.051354] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-02. [2025-10-20T17:03:41.117552] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:03:41.343108] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T17:03:41.851423] [INFO ] [REMOTE ] ADX: Fetching primary data for 2025-10. [2025-10-20T17:03:42.651440] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-09. [2025-10-20T17:03:42.693517] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:03:42.786994] [INFO ] [REMOTE ] ADX: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T17:03:42.791984] [INFO ] [REMOTE ] ADX: Full fetch complete (41970 records). [2025-10-20T17:03:42.792168] [DEBUG] [VALIDATOR ] ADX: Validating primary data. [2025-10-20T17:03:43.208880] [DEBUG] [VALIDATOR ] ADX: All expected trading days are present. [2025-10-20T17:03:43.209174] [DEBUG] [VALIDATOR ] ADX: Primary data validation passed. [2025-10-20T17:03:43.209724] [INFO ] [MAIN ] ADX: Fetched primary data is valid. [2025-10-20T17:03:43.209796] [DEBUG] [LOCAL ] ADX: Saving primary data. [2025-10-20T17:03:43.211342] [DEBUG] [LOCAL ] ADX: Trimmed 41970 records to 27841 (keeping 200 days of data) [2025-10-20T17:03:43.382081] [DEBUG] [LOCAL ] ADX: Primary data saved (27841 records). [2025-10-20T17:03:43.382238] [INFO ] [MAIN ] ADX: Full history saved. [2025-10-20T17:03:43.382769] [INFO ] [MAIN ] Update Progress: 35/2367 symbols completed. ADX took 34.9s. Estimated remaining: 4h 3m 10s. Rate: 9.6 symbols/min [2025-10-20T17:03:43.456500] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-04. [2025-10-20T17:03:43.574932] [INFO ] [MAIN ] AEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:43.642878] [DEBUG] [LOCAL ] AEO: Loaded 54346 existing records from primary storage. [2025-10-20T17:03:43.643252] [INFO ] [MAIN ] AEO: Loaded 54346 existing records. [2025-10-20T17:03:43.833414] [WARN ] [MAIN ] AEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:43.833670] [INFO ] [MAIN ] AEO: Wiping existing primary data before full refresh. [2025-10-20T17:03:43.833734] [INFO ] [LOCAL ] AEO: Starting deletion of primary data. [2025-10-20T17:03:43.833997] [DEBUG] [LOCAL ] AEO: Deleted primary data file. [2025-10-20T17:03:43.834056] [INFO ] [MAIN ] AEO: Starting full primary refresh. [2025-10-20T17:03:44.238461] [INFO ] [REMOTE ] AEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:44.263112] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-01. [2025-10-20T17:03:44.285443] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:03:45.051446] [INFO ] [REMOTE ] AEE: Fetching primary data for 2025-10. [2025-10-20T17:03:45.302731] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:03:45.851403] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-05. [2025-10-20T17:03:46.071019] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:03:46.651441] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-03. [2025-10-20T17:03:46.940715] [INFO ] [REMOTE ] AEE: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T17:03:46.948930] [INFO ] [REMOTE ] AEE: Full fetch complete (73026 records). [2025-10-20T17:03:46.949072] [DEBUG] [VALIDATOR ] AEE: Validating primary data. [2025-10-20T17:03:47.354638] [DEBUG] [VALIDATOR ] AEE: All expected trading days are present. [2025-10-20T17:03:47.354771] [DEBUG] [VALIDATOR ] AEE: Primary data validation passed. [2025-10-20T17:03:47.355985] [INFO ] [MAIN ] AEE: Fetched primary data is valid. [2025-10-20T17:03:47.356065] [DEBUG] [LOCAL ] AEE: Saving primary data. [2025-10-20T17:03:47.358342] [DEBUG] [LOCAL ] AEE: Trimmed 73026 records to 50155 (keeping 200 days of data) [2025-10-20T17:03:47.458760] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-02. [2025-10-20T17:03:47.490403] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T17:03:47.718890] [DEBUG] [LOCAL ] AEE: Primary data saved (50155 records). [2025-10-20T17:03:47.719031] [INFO ] [MAIN ] AEE: Full history saved. [2025-10-20T17:03:47.719101] [INFO ] [MAIN ] Update Progress: 36/2367 symbols completed. AEE took 37.5s. Estimated remaining: 4h 0m 60s. Rate: 9.7 symbols/min [2025-10-20T17:03:47.910161] [INFO ] [MAIN ] AER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:03:47.965111] [DEBUG] [LOCAL ] AER: Loaded 48795 existing records from primary storage. [2025-10-20T17:03:47.965595] [INFO ] [MAIN ] AER: Loaded 48795 existing records. [2025-10-20T17:03:48.144913] [WARN ] [MAIN ] AER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:03:48.145124] [INFO ] [MAIN ] AER: Wiping existing primary data before full refresh. [2025-10-20T17:03:48.145171] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-10-20T17:03:48.145384] [DEBUG] [LOCAL ] AER: Deleted primary data file. [2025-10-20T17:03:48.145439] [INFO ] [MAIN ] AER: Starting full primary refresh. [2025-10-20T17:03:48.252267] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-06. [2025-10-20T17:03:48.546955] [INFO ] [REMOTE ] AER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:03:48.611758] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:03:49.051389] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-04. [2025-10-20T17:03:49.091857] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T17:03:49.753744] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:03:49.851380] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-07. [2025-10-20T17:03:50.651415] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-03. [2025-10-20T17:03:51.036936] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:03:51.242609] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T17:03:51.451371] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-05. [2025-10-20T17:03:52.039993] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:03:52.251378] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-08. [2025-10-20T17:03:53.057952] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-04. [2025-10-20T17:03:53.064958] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:03:53.861287] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-06. [2025-10-20T17:03:53.884617] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T17:03:54.651468] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-09. [2025-10-20T17:03:54.839811] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:03:55.392070] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:03:55.451448] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-01. [2025-10-20T17:03:56.215503] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:03:56.251368] [INFO ] [REMOTE ] AEG: Fetching primary data for 2025-10. [2025-10-20T17:03:56.906690] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:03:57.051460] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-05. [2025-10-20T17:03:57.851386] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-07. [2025-10-20T17:03:57.959094] [INFO ] [REMOTE ] AEG: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T17:03:57.967617] [INFO ] [REMOTE ] AEG: Full fetch complete (70650 records). [2025-10-20T17:03:57.967891] [DEBUG] [VALIDATOR ] AEG: Validating primary data. [2025-10-20T17:03:58.430259] [DEBUG] [VALIDATOR ] AEG: All expected trading days are present. [2025-10-20T17:03:58.430389] [DEBUG] [VALIDATOR ] AEG: Primary data validation passed. [2025-10-20T17:03:58.431259] [INFO ] [MAIN ] AEG: Fetched primary data is valid. [2025-10-20T17:03:58.431334] [DEBUG] [LOCAL ] AEG: Saving primary data. [2025-10-20T17:03:58.433743] [DEBUG] [LOCAL ] AEG: Trimmed 70650 records to 49602 (keeping 200 days of data) [2025-10-20T17:03:58.671882] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-02. [2025-10-20T17:03:58.742954] [DEBUG] [LOCAL ] AEG: Primary data saved (49602 records). [2025-10-20T17:03:58.743548] [INFO ] [MAIN ] AEG: Full history saved. [2025-10-20T17:03:58.744922] [INFO ] [MAIN ] Update Progress: 37/2367 symbols completed. AEG took 30.9s. Estimated remaining: 4h 5m 57s. Rate: 9.5 symbols/min [2025-10-20T17:03:58.980843] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T17:03:59.451388] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-06. [2025-10-20T17:03:59.757356] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T17:03:59.942508] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:04:00.251389] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-03. [2025-10-20T17:04:01.056566] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-08. [2025-10-20T17:04:01.083129] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T17:04:01.803168] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:04:01.851380] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-04. [2025-10-20T17:04:02.651396] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-07. [2025-10-20T17:04:02.732440] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:04:03.451432] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-09. [2025-10-20T17:04:03.553681] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:04:04.263564] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-05. [2025-10-20T17:04:04.270840] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T17:04:05.051482] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-08. [2025-10-20T17:04:05.186559] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:04:05.737581] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:04:05.851368] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-06. [2025-10-20T17:04:06.651426] [INFO ] [REMOTE ] AEM: Fetching primary data for 2025-10. [2025-10-20T17:04:07.192478] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T17:04:07.451408] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-09. [2025-10-20T17:04:07.709959] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:04:08.251406] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-07. [2025-10-20T17:04:08.574606] [INFO ] [REMOTE ] AEM: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:04:08.582604] [INFO ] [REMOTE ] AEM: Full fetch complete (77457 records). [2025-10-20T17:04:08.582836] [DEBUG] [VALIDATOR ] AEM: Validating primary data. [2025-10-20T17:04:09.005931] [DEBUG] [VALIDATOR ] AEM: All expected trading days are present. [2025-10-20T17:04:09.006055] [DEBUG] [VALIDATOR ] AEM: Primary data validation passed. [2025-10-20T17:04:09.006978] [INFO ] [MAIN ] AEM: Fetched primary data is valid. [2025-10-20T17:04:09.007048] [DEBUG] [LOCAL ] AEM: Saving primary data. [2025-10-20T17:04:09.009822] [DEBUG] [LOCAL ] AEM: Trimmed 77457 records to 53477 (keeping 200 days of data) [2025-10-20T17:04:09.379013] [DEBUG] [LOCAL ] AEM: Primary data saved (53477 records). [2025-10-20T17:04:09.379211] [INFO ] [MAIN ] AEM: Full history saved. [2025-10-20T17:04:09.379302] [INFO ] [MAIN ] Update Progress: 38/2367 symbols completed. AEM took 35.8s. Estimated remaining: 4h 10m 14s. Rate: 9.3 symbols/min [2025-10-20T17:04:09.846999] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-09 (2.40 seconds). [2025-10-20T17:04:09.860682] [INFO ] [REMOTE ] AEO: Fetching primary data for 2025-10. [2025-10-20T17:04:10.258842] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:04:10.270922] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-08. [2025-10-20T17:04:11.707520] [INFO ] [REMOTE ] AEO: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T17:04:11.717506] [INFO ] [REMOTE ] AEO: Full fetch complete (77648 records). [2025-10-20T17:04:11.717982] [DEBUG] [VALIDATOR ] AEO: Validating primary data. [2025-10-20T17:04:12.172720] [DEBUG] [VALIDATOR ] AEO: All expected trading days are present. [2025-10-20T17:04:12.172879] [DEBUG] [VALIDATOR ] AEO: Primary data validation passed. [2025-10-20T17:04:12.173875] [INFO ] [MAIN ] AEO: Fetched primary data is valid. [2025-10-20T17:04:12.173979] [DEBUG] [LOCAL ] AEO: Saving primary data. [2025-10-20T17:04:12.176238] [DEBUG] [LOCAL ] AEO: Trimmed 77648 records to 53565 (keeping 200 days of data) [2025-10-20T17:04:12.307073] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T17:04:12.347871] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-09. [2025-10-20T17:04:12.564079] [DEBUG] [LOCAL ] AEO: Primary data saved (53565 records). [2025-10-20T17:04:12.564287] [INFO ] [MAIN ] AEO: Full history saved. [2025-10-20T17:04:12.565481] [INFO ] [MAIN ] Update Progress: 39/2367 symbols completed. AEO took 29.2s. Estimated remaining: 4h 6m 53s. Rate: 9.4 symbols/min [2025-10-20T17:04:14.109104] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:04:14.120263] [INFO ] [REMOTE ] AER: Fetching primary data for 2025-10. [2025-10-20T17:04:15.803254] [INFO ] [REMOTE ] AER: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:04:15.811044] [INFO ] [REMOTE ] AER: Full fetch complete (69618 records). [2025-10-20T17:04:15.811280] [DEBUG] [VALIDATOR ] AER: Validating primary data. [2025-10-20T17:04:15.812485] [FATAL] [VALIDATOR ] AER: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:04:15.812667] [FATAL] [MAIN ] AER: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:04:15.812730] [WARN ] [MAIN ] AER: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:04:15.813073] [INFO ] [LOCAL ] AER: Starting deletion of primary data. [2025-10-20T17:04:15.813151] [DEBUG] [LOCAL ] AER: No primary data file found to delete. [2025-10-20T17:04:15.813192] [INFO ] [LOCAL ] AER: Starting deletion of derivative data. [2025-10-20T17:04:15.813426] [DEBUG] [LOCAL ] AER: Deleted derivative data file. [2025-10-20T17:04:15.814811] [INFO ] [MAIN ] Update Progress: 40/2367 symbols completed. AER took 28.1s. Estimated remaining: 4h 3m 46s. Rate: 9.5 symbols/min [2025-10-20T17:04:16.481102] [INFO ] [MAIN ] AFB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.618315] [INFO ] [MAIN ] AES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.606838] [INFO ] [MAIN ] AEXA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.638389] [INFO ] [MAIN ] AESI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:16.638778] [DEBUG] [LOCAL ] AEXA: No existing data file found in primary storage. [2025-10-20T17:04:16.639578] [WARN ] [MAIN ] AEXA: No existing data found, performing full refresh instead. [2025-10-20T17:04:16.644844] [INFO ] [MAIN ] AEXA: Wiping existing primary data before full refresh. [2025-10-20T17:04:16.645258] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-10-20T17:04:16.645417] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-10-20T17:04:16.645463] [INFO ] [MAIN ] AEXA: Starting full primary refresh. [2025-10-20T17:04:16.734278] [DEBUG] [LOCAL ] AFB: Loaded 11315 existing records from primary storage. [2025-10-20T17:04:16.772952] [INFO ] [MAIN ] AFB: Loaded 11315 existing records. [2025-10-20T17:04:17.212828] [DEBUG] [LOCAL ] AES: Loaded 54448 existing records from primary storage. [2025-10-20T17:04:17.244455] [INFO ] [MAIN ] AES: Loaded 54448 existing records. [2025-10-20T17:04:17.297274] [DEBUG] [LOCAL ] AESI: Loaded 50515 existing records from primary storage. [2025-10-20T17:04:17.396131] [INFO ] [MAIN ] AESI: Loaded 50515 existing records. [2025-10-20T17:04:17.414832] [WARN ] [MAIN ] AFB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.576919] [INFO ] [MAIN ] AFB: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.588426] [INFO ] [REMOTE ] AEXA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:17.658537] [INFO ] [REMOTE ] AEXA: Fetching primary data for 2025-01. [2025-10-20T17:04:17.647985] [INFO ] [LOCAL ] AFB: Starting deletion of primary data. [2025-10-20T17:04:17.757932] [DEBUG] [LOCAL ] AFB: Deleted primary data file. [2025-10-20T17:04:17.784296] [INFO ] [MAIN ] AFB: Starting full primary refresh. [2025-10-20T17:04:17.838303] [WARN ] [MAIN ] AES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.843941] [INFO ] [MAIN ] AES: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.874777] [INFO ] [LOCAL ] AES: Starting deletion of primary data. [2025-10-20T17:04:17.936427] [DEBUG] [LOCAL ] AES: Deleted primary data file. [2025-10-20T17:04:17.936332] [DEBUG] [REMOTE ] AEXA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:04:17.969384] [WARN ] [MAIN ] AESI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:17.980920] [INFO ] [MAIN ] AESI: Wiping existing primary data before full refresh. [2025-10-20T17:04:17.975323] [INFO ] [MAIN ] AES: Starting full primary refresh. [2025-10-20T17:04:17.975143] [WARN ] [REMOTE ] AEXA: No data returned for month 2025-01 [2025-10-20T17:04:17.996675] [INFO ] [LOCAL ] AESI: Starting deletion of primary data. [2025-10-20T17:04:18.007395] [FATAL] [REMOTE ] AEXA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:04:18.067089] [FATAL] [MAIN ] AEXA: Failed to fetch primary data. [2025-10-20T17:04:18.077787] [WARN ] [MAIN ] AEXA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:04:18.113259] [INFO ] [LOCAL ] AEXA: Starting deletion of primary data. [2025-10-20T17:04:18.107863] [DEBUG] [LOCAL ] AESI: Deleted primary data file. [2025-10-20T17:04:18.126690] [INFO ] [MAIN ] AESI: Starting full primary refresh. [2025-10-20T17:04:18.137431] [DEBUG] [LOCAL ] AEXA: No primary data file found to delete. [2025-10-20T17:04:18.192853] [INFO ] [LOCAL ] AEXA: Starting deletion of derivative data. [2025-10-20T17:04:18.262607] [DEBUG] [LOCAL ] AEXA: No derivative data file found to delete. [2025-10-20T17:04:18.301844] [INFO ] [MAIN ] Update Progress: 41/2367 symbols completed. AEXA took 2.4s. Estimated remaining: 4h 0m 4s. Rate: 9.7 symbols/min [2025-10-20T17:04:19.192670] [INFO ] [REMOTE ] AES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.189411] [INFO ] [MAIN ] AFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:19.184946] [INFO ] [REMOTE ] AFB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.233470] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-01. [2025-10-20T17:04:19.243918] [DEBUG] [LOCAL ] AFG: No existing data file found in primary storage. [2025-10-20T17:04:19.250173] [WARN ] [MAIN ] AFG: No existing data found, performing full refresh instead. [2025-10-20T17:04:19.244598] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-01. [2025-10-20T17:04:19.262058] [INFO ] [MAIN ] AFG: Wiping existing primary data before full refresh. [2025-10-20T17:04:19.281882] [INFO ] [LOCAL ] AFG: Starting deletion of primary data. [2025-10-20T17:04:19.287484] [DEBUG] [LOCAL ] AFG: No primary data file found to delete. [2025-10-20T17:04:19.287597] [INFO ] [MAIN ] AFG: Starting full primary refresh. [2025-10-20T17:04:19.455980] [INFO ] [REMOTE ] AESI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.482536] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-01. [2025-10-20T17:04:19.761347] [INFO ] [REMOTE ] AFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:19.761681] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-01. [2025-10-20T17:04:19.948453] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:04:19.951321] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-02. [2025-10-20T17:04:20.722215] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:04:20.725086] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-03. [2025-10-20T17:04:20.897873] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T17:04:20.905028] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-02. [2025-10-20T17:04:20.962267] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:04:20.971470] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-02. [2025-10-20T17:04:21.028466] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:04:21.038956] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-02. [2025-10-20T17:04:21.287631] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:04:21.290956] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-04. [2025-10-20T17:04:21.946994] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:04:21.956306] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-03. [2025-10-20T17:04:21.973599] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:04:21.978888] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-05. [2025-10-20T17:04:22.502947] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T17:04:22.562680] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:04:22.651334] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-03. [2025-10-20T17:04:22.758266] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:04:23.451440] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-03. [2025-10-20T17:04:23.647388] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:04:23.852424] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:04:24.251447] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-04. [2025-10-20T17:04:25.041940] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T17:04:25.053090] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-04. [2025-10-20T17:04:25.712829] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:04:25.851362] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-05. [2025-10-20T17:04:26.651475] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-06. [2025-10-20T17:04:26.913700] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:04:27.096887] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:04:27.316314] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T17:04:27.451455] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-07. [2025-10-20T17:04:28.251344] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-05. [2025-10-20T17:04:28.334333] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T17:04:29.051433] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-04. [2025-10-20T17:04:29.766217] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:04:29.851360] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-06. [2025-10-20T17:04:30.651422] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-06. [2025-10-20T17:04:30.921709] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:04:31.451402] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-05. [2025-10-20T17:04:31.498352] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:04:32.095199] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:04:32.251419] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-08. [2025-10-20T17:04:33.051363] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-07. [2025-10-20T17:04:33.126844] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:04:33.391899] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:04:33.851375] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-06. [2025-10-20T17:04:34.411231] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:04:34.651368] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-08. [2025-10-20T17:04:35.451356] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-09. [2025-10-20T17:04:35.852901] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:04:36.004379] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:04:36.007367] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:04:36.251401] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-09. [2025-10-20T17:04:37.051425] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-07. [2025-10-20T17:04:37.777458] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:04:37.851371] [INFO ] [REMOTE ] AFG: Fetching primary data for 2025-10. [2025-10-20T17:04:38.651378] [INFO ] [REMOTE ] AFB: Fetching primary data for 2025-10. [2025-10-20T17:04:39.012688] [INFO ] [REMOTE ] AFG: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:04:39.017924] [INFO ] [REMOTE ] AFG: Full fetch complete (46719 records). [2025-10-20T17:04:39.018111] [DEBUG] [VALIDATOR ] AFG: Validating primary data. [2025-10-20T17:04:39.404487] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T17:04:39.456996] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-08. [2025-10-20T17:04:39.508516] [DEBUG] [VALIDATOR ] AFG: All expected trading days are present. [2025-10-20T17:04:39.508589] [DEBUG] [VALIDATOR ] AFG: Primary data validation passed. [2025-10-20T17:04:39.509079] [INFO ] [MAIN ] AFG: Fetched primary data is valid. [2025-10-20T17:04:39.509253] [DEBUG] [LOCAL ] AFG: Saving primary data. [2025-10-20T17:04:39.510369] [DEBUG] [LOCAL ] AFG: Trimmed 46719 records to 31954 (keeping 200 days of data) [2025-10-20T17:04:39.611026] [INFO ] [REMOTE ] AFB: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T17:04:39.636781] [INFO ] [REMOTE ] AFB: Full fetch complete (16449 records). [2025-10-20T17:04:39.648970] [DEBUG] [VALIDATOR ] AFB: Validating primary data. [2025-10-20T17:04:39.825435] [DEBUG] [LOCAL ] AFG: Primary data saved (31954 records). [2025-10-20T17:04:39.841296] [INFO ] [MAIN ] AFG: Full history saved. [2025-10-20T17:04:39.851707] [INFO ] [MAIN ] Update Progress: 42/2367 symbols completed. AFG took 21.5s. Estimated remaining: 4h 14m 8s. Rate: 9.1 symbols/min [2025-10-20T17:04:40.254270] [INFO ] [MAIN ] AFGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:40.260579] [DEBUG] [LOCAL ] AFGB: No existing data file found in primary storage. [2025-10-20T17:04:40.254387] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-07. [2025-10-20T17:04:40.264600] [WARN ] [MAIN ] AFGB: No existing data found, performing full refresh instead. [2025-10-20T17:04:40.290751] [INFO ] [MAIN ] AFGB: Wiping existing primary data before full refresh. [2025-10-20T17:04:40.303490] [INFO ] [LOCAL ] AFGB: Starting deletion of primary data. [2025-10-20T17:04:40.303911] [DEBUG] [LOCAL ] AFGB: No primary data file found to delete. [2025-10-20T17:04:40.304082] [DEBUG] [VALIDATOR ] AFB: All expected trading days are present. [2025-10-20T17:04:40.304179] [INFO ] [MAIN ] AFGB: Starting full primary refresh. [2025-10-20T17:04:40.304274] [DEBUG] [VALIDATOR ] AFB: Primary data validation passed. [2025-10-20T17:04:40.304752] [INFO ] [MAIN ] AFB: Fetched primary data is valid. [2025-10-20T17:04:40.310025] [DEBUG] [LOCAL ] AFB: Saving primary data. [2025-10-20T17:04:40.371756] [DEBUG] [LOCAL ] AFB: Trimmed 16449 records to 11115 (keeping 200 days of data) [2025-10-20T17:04:40.563131] [DEBUG] [LOCAL ] AFB: Primary data saved (11115 records). [2025-10-20T17:04:40.600211] [INFO ] [MAIN ] AFB: Full history saved. [2025-10-20T17:04:40.613727] [INFO ] [MAIN ] Update Progress: 43/2367 symbols completed. AFB took 24.8s. Estimated remaining: 4h 8m 48s. Rate: 9.3 symbols/min [2025-10-20T17:04:40.927166] [INFO ] [REMOTE ] AFGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:40.947885] [INFO ] [MAIN ] AFGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:40.953212] [DEBUG] [LOCAL ] AFGC: Loaded 2584 existing records from primary storage. [2025-10-20T17:04:40.953400] [INFO ] [MAIN ] AFGC: Loaded 2584 existing records. [2025-10-20T17:04:41.057317] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-01. [2025-10-20T17:04:41.131114] [WARN ] [MAIN ] AFGC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:41.131667] [INFO ] [MAIN ] AFGC: Wiping existing primary data before full refresh. [2025-10-20T17:04:41.131756] [INFO ] [LOCAL ] AFGC: Starting deletion of primary data. [2025-10-20T17:04:41.131919] [DEBUG] [LOCAL ] AFGC: Deleted primary data file. [2025-10-20T17:04:41.131977] [INFO ] [MAIN ] AFGC: Starting full primary refresh. [2025-10-20T17:04:41.523226] [INFO ] [REMOTE ] AFGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:41.567469] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:04:41.604297] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:04:41.858963] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-02. [2025-10-20T17:04:41.865442] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:04:42.184353] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T17:04:42.651381] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-09. [2025-10-20T17:04:43.451427] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-01. [2025-10-20T17:04:43.871961] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:04:44.251446] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-02. [2025-10-20T17:04:44.692085] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:04:44.754442] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T17:04:45.051416] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-03. [2025-10-20T17:04:45.552710] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:04:45.851372] [INFO ] [REMOTE ] AES: Fetching primary data for 2025-10. [2025-10-20T17:04:46.651442] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-03. [2025-10-20T17:04:47.208508] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:04:47.451451] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-04. [2025-10-20T17:04:47.815553] [INFO ] [REMOTE ] AES: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:04:47.824420] [INFO ] [REMOTE ] AES: Full fetch complete (77846 records). [2025-10-20T17:04:47.824605] [DEBUG] [VALIDATOR ] AES: Validating primary data. [2025-10-20T17:04:47.940382] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T17:04:48.242297] [DEBUG] [VALIDATOR ] AES: All expected trading days are present. [2025-10-20T17:04:48.242389] [DEBUG] [VALIDATOR ] AES: Primary data validation passed. [2025-10-20T17:04:48.243211] [INFO ] [MAIN ] AES: Fetched primary data is valid. [2025-10-20T17:04:48.243278] [DEBUG] [LOCAL ] AES: Saving primary data. [2025-10-20T17:04:48.244903] [DEBUG] [LOCAL ] AES: Trimmed 77846 records to 53668 (keeping 200 days of data) [2025-10-20T17:04:48.300690] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-08. [2025-10-20T17:04:48.592680] [DEBUG] [LOCAL ] AES: Primary data saved (53668 records). [2025-10-20T17:04:48.592842] [INFO ] [MAIN ] AES: Full history saved. [2025-10-20T17:04:48.592930] [INFO ] [MAIN ] Update Progress: 44/2367 symbols completed. AES took 32.8s. Estimated remaining: 4h 10m 4s. Rate: 9.3 symbols/min [2025-10-20T17:04:48.765706] [INFO ] [MAIN ] AFGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:48.769932] [DEBUG] [LOCAL ] AFGD: Loaded 1791 existing records from primary storage. [2025-10-20T17:04:48.770097] [INFO ] [MAIN ] AFGD: Loaded 1791 existing records. [2025-10-20T17:04:48.961545] [WARN ] [MAIN ] AFGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:48.961784] [INFO ] [MAIN ] AFGD: Wiping existing primary data before full refresh. [2025-10-20T17:04:48.961833] [INFO ] [LOCAL ] AFGD: Starting deletion of primary data. [2025-10-20T17:04:48.961966] [DEBUG] [LOCAL ] AFGD: Deleted primary data file. [2025-10-20T17:04:48.962042] [INFO ] [MAIN ] AFGD: Starting full primary refresh. [2025-10-20T17:04:49.060405] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-05. [2025-10-20T17:04:49.322712] [INFO ] [REMOTE ] AFGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:49.545277] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:04:49.851380] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-06. [2025-10-20T17:04:49.904121] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T17:04:50.309392] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:04:50.651408] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-09. [2025-10-20T17:04:51.451363] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-01. [2025-10-20T17:04:52.052446] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:04:52.111694] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:04:52.251394] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-07. [2025-10-20T17:04:52.819874] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:04:53.051464] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-04. [2025-10-20T17:04:53.473635] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T17:04:53.851395] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-08. [2025-10-20T17:04:54.275507] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T17:04:54.651392] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-05. [2025-10-20T17:04:55.055053] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T17:04:55.451419] [INFO ] [REMOTE ] AESI: Fetching primary data for 2025-10. [2025-10-20T17:04:56.251420] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-06. [2025-10-20T17:04:56.777805] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:04:57.051386] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-09. [2025-10-20T17:04:57.308696] [INFO ] [REMOTE ] AESI: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:04:57.316337] [INFO ] [REMOTE ] AESI: Full fetch complete (71403 records). [2025-10-20T17:04:57.316448] [DEBUG] [VALIDATOR ] AESI: Validating primary data. [2025-10-20T17:04:57.420052] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T17:04:57.774424] [DEBUG] [VALIDATOR ] AESI: All expected trading days are present. [2025-10-20T17:04:57.774544] [DEBUG] [VALIDATOR ] AESI: Primary data validation passed. [2025-10-20T17:04:57.775326] [INFO ] [MAIN ] AESI: Fetched primary data is valid. [2025-10-20T17:04:57.775398] [DEBUG] [LOCAL ] AESI: Saving primary data. [2025-10-20T17:04:57.777305] [DEBUG] [LOCAL ] AESI: Trimmed 71403 records to 49828 (keeping 200 days of data) [2025-10-20T17:04:57.869430] [INFO ] [REMOTE ] AFGC: Fetching primary data for 2025-10. [2025-10-20T17:04:58.095561] [DEBUG] [LOCAL ] AESI: Primary data saved (49828 records). [2025-10-20T17:04:58.095794] [INFO ] [MAIN ] AESI: Full history saved. [2025-10-20T17:04:58.095900] [INFO ] [MAIN ] Update Progress: 45/2367 symbols completed. AESI took 42.3s. Estimated remaining: 4h 12m 34s. Rate: 9.2 symbols/min [2025-10-20T17:04:58.275250] [INFO ] [MAIN ] AFGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:58.279513] [INFO ] [REMOTE ] AFGC: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T17:04:58.280933] [INFO ] [REMOTE ] AFGC: Full fetch complete (3606 records). [2025-10-20T17:04:58.281028] [DEBUG] [VALIDATOR ] AFGC: Validating primary data. [2025-10-20T17:04:58.367947] [DEBUG] [LOCAL ] AFGE: Loaded 2387 existing records from primary storage. [2025-10-20T17:04:58.394890] [INFO ] [MAIN ] AFGE: Loaded 2387 existing records. [2025-10-20T17:04:58.653800] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-07. [2025-10-20T17:04:58.833612] [WARN ] [MAIN ] AFGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:58.883427] [INFO ] [MAIN ] AFGE: Wiping existing primary data before full refresh. [2025-10-20T17:04:58.888742] [DEBUG] [VALIDATOR ] AFGC: All expected trading days are present. [2025-10-20T17:04:58.888631] [INFO ] [LOCAL ] AFGE: Starting deletion of primary data. [2025-10-20T17:04:58.888942] [DEBUG] [VALIDATOR ] AFGC: Primary data validation passed. [2025-10-20T17:04:58.889215] [INFO ] [MAIN ] AFGC: Fetched primary data is valid. [2025-10-20T17:04:58.889289] [DEBUG] [LOCAL ] AFGE: Deleted primary data file. [2025-10-20T17:04:58.889501] [INFO ] [MAIN ] AFGE: Starting full primary refresh. [2025-10-20T17:04:58.889408] [DEBUG] [LOCAL ] AFGC: Saving primary data. [2025-10-20T17:04:58.953850] [DEBUG] [LOCAL ] AFGC: Trimmed 3606 records to 2561 (keeping 200 days of data) [2025-10-20T17:04:59.024074] [DEBUG] [LOCAL ] AFGC: Primary data saved (2561 records). [2025-10-20T17:04:59.024183] [INFO ] [MAIN ] AFGC: Full history saved. [2025-10-20T17:04:59.024332] [INFO ] [MAIN ] Update Progress: 46/2367 symbols completed. AFGC took 18.4s. Estimated remaining: 4h 7m 45s. Rate: 9.4 symbols/min [2025-10-20T17:04:59.246538] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:04:59.414774] [INFO ] [MAIN ] AFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:04:59.462686] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-02. [2025-10-20T17:04:59.513617] [INFO ] [REMOTE ] AFGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:04:59.558691] [DEBUG] [LOCAL ] AFL: Loaded 53965 existing records from primary storage. [2025-10-20T17:04:59.559057] [INFO ] [MAIN ] AFL: Loaded 53965 existing records. [2025-10-20T17:04:59.731269] [WARN ] [MAIN ] AFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:04:59.731507] [INFO ] [MAIN ] AFL: Wiping existing primary data before full refresh. [2025-10-20T17:04:59.731565] [INFO ] [LOCAL ] AFL: Starting deletion of primary data. [2025-10-20T17:04:59.731770] [DEBUG] [LOCAL ] AFL: Deleted primary data file. [2025-10-20T17:04:59.731831] [INFO ] [MAIN ] AFL: Starting full primary refresh. [2025-10-20T17:04:59.904673] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:05:00.081135] [INFO ] [REMOTE ] AFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:00.251379] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-03. [2025-10-20T17:05:00.757472] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-10-20T17:05:01.051384] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-01. [2025-10-20T17:05:01.642122] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T17:05:01.851389] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-02. [2025-10-20T17:05:02.315336] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T17:05:02.651413] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-04. [2025-10-20T17:05:03.119715] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T17:05:03.451417] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-03. [2025-10-20T17:05:03.814797] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T17:05:04.251415] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-05. [2025-10-20T17:05:04.569798] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:05:05.051439] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-01. [2025-10-20T17:05:05.851422] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-08. [2025-10-20T17:05:06.301889] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T17:05:06.651387] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-09. [2025-10-20T17:05:06.976958] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:05:07.142732] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T17:05:07.451419] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-04. [2025-10-20T17:05:07.876442] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:05:08.251412] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-05. [2025-10-20T17:05:08.671120] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T17:05:09.051402] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-02. [2025-10-20T17:05:09.851431] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-06. [2025-10-20T17:05:10.345206] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:05:10.553768] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T17:05:10.651361] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-03. [2025-10-20T17:05:11.451373] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-07. [2025-10-20T17:05:12.022162] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:05:12.251375] [INFO ] [REMOTE ] AFGB: Fetching primary data for 2025-10. [2025-10-20T17:05:12.699072] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T17:05:12.733622] [INFO ] [REMOTE ] AFGB: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:05:12.734569] [INFO ] [REMOTE ] AFGB: Full fetch complete (3467 records). [2025-10-20T17:05:12.734637] [DEBUG] [VALIDATOR ] AFGB: Validating primary data. [2025-10-20T17:05:13.060473] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-04. [2025-10-20T17:05:13.093834] [DEBUG] [VALIDATOR ] AFGB: All expected trading days are present. [2025-10-20T17:05:13.093930] [DEBUG] [VALIDATOR ] AFGB: Primary data validation passed. [2025-10-20T17:05:13.094100] [INFO ] [MAIN ] AFGB: Fetched primary data is valid. [2025-10-20T17:05:13.094143] [DEBUG] [LOCAL ] AFGB: Saving primary data. [2025-10-20T17:05:13.094498] [DEBUG] [LOCAL ] AFGB: Trimmed 3467 records to 2245 (keeping 200 days of data) [2025-10-20T17:05:13.109278] [DEBUG] [LOCAL ] AFGB: Primary data saved (2245 records). [2025-10-20T17:05:13.109425] [INFO ] [MAIN ] AFGB: Full history saved. [2025-10-20T17:05:13.109504] [INFO ] [MAIN ] Update Progress: 47/2367 symbols completed. AFGB took 33.2s. Estimated remaining: 4h 13m 58s. Rate: 9.1 symbols/min [2025-10-20T17:05:13.291296] [INFO ] [MAIN ] AG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:13.339856] [DEBUG] [LOCAL ] AG: Loaded 54423 existing records from primary storage. [2025-10-20T17:05:13.340184] [INFO ] [MAIN ] AG: Loaded 54423 existing records. [2025-10-20T17:05:13.520087] [WARN ] [MAIN ] AG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:13.520315] [INFO ] [MAIN ] AG: Wiping existing primary data before full refresh. [2025-10-20T17:05:13.520362] [INFO ] [LOCAL ] AG: Starting deletion of primary data. [2025-10-20T17:05:13.520577] [DEBUG] [LOCAL ] AG: Deleted primary data file. [2025-10-20T17:05:13.520633] [INFO ] [MAIN ] AG: Starting full primary refresh. [2025-10-20T17:05:13.864578] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-06. [2025-10-20T17:05:13.905955] [INFO ] [REMOTE ] AG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:14.359543] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:05:14.647687] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:05:14.658364] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-01. [2025-10-20T17:05:15.451401] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-05. [2025-10-20T17:05:16.264110] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-07. [2025-10-20T17:05:16.272021] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:05:16.725211] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T17:05:17.051393] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-08. [2025-10-20T17:05:17.390704] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T17:05:17.465002] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-10-20T17:05:17.851434] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-08. [2025-10-20T17:05:18.449485] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T17:05:18.651352] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-02. [2025-10-20T17:05:19.451401] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-09. [2025-10-20T17:05:19.917570] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T17:05:20.251384] [INFO ] [REMOTE ] AFGD: Fetching primary data for 2025-10. [2025-10-20T17:05:20.603482] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-10-20T17:05:20.682927] [INFO ] [REMOTE ] AFGD: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T17:05:20.683783] [INFO ] [REMOTE ] AFGD: Full fetch complete (2778 records). [2025-10-20T17:05:20.683851] [DEBUG] [VALIDATOR ] AFGD: Validating primary data. [2025-10-20T17:05:21.045110] [DEBUG] [VALIDATOR ] AFGD: All expected trading days are present. [2025-10-20T17:05:21.045188] [DEBUG] [VALIDATOR ] AFGD: Primary data validation passed. [2025-10-20T17:05:21.045286] [INFO ] [MAIN ] AFGD: Fetched primary data is valid. [2025-10-20T17:05:21.045324] [DEBUG] [LOCAL ] AFGD: Saving primary data. [2025-10-20T17:05:21.045664] [DEBUG] [LOCAL ] AFGD: Trimmed 2778 records to 1775 (keeping 200 days of data) [2025-10-20T17:05:21.054001] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-06. [2025-10-20T17:05:21.057026] [DEBUG] [LOCAL ] AFGD: Primary data saved (1775 records). [2025-10-20T17:05:21.057984] [INFO ] [MAIN ] AFGD: Full history saved. [2025-10-20T17:05:21.058126] [INFO ] [MAIN ] Update Progress: 48/2367 symbols completed. AFGD took 32.5s. Estimated remaining: 4h 14m 58s. Rate: 9.1 symbols/min [2025-10-20T17:05:21.227784] [INFO ] [MAIN ] AGCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:21.271191] [DEBUG] [LOCAL ] AGCO: Loaded 42964 existing records from primary storage. [2025-10-20T17:05:21.271519] [INFO ] [MAIN ] AGCO: Loaded 42964 existing records. [2025-10-20T17:05:21.447054] [WARN ] [MAIN ] AGCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:21.447280] [INFO ] [MAIN ] AGCO: Wiping existing primary data before full refresh. [2025-10-20T17:05:21.447336] [INFO ] [LOCAL ] AGCO: Starting deletion of primary data. [2025-10-20T17:05:21.447883] [DEBUG] [LOCAL ] AGCO: Deleted primary data file. [2025-10-20T17:05:21.447939] [INFO ] [MAIN ] AGCO: Starting full primary refresh. [2025-10-20T17:05:21.801110] [INFO ] [REMOTE ] AGCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:21.851329] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-01. [2025-10-20T17:05:22.512574] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:05:22.651398] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-09. [2025-10-20T17:05:23.146806] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:05:23.413849] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:05:23.451327] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-02. [2025-10-20T17:05:24.251391] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-07. [2025-10-20T17:05:24.907936] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:05:25.051399] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-03. [2025-10-20T17:05:25.851348] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-03. [2025-10-20T17:05:25.983229] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:05:26.656712] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-08. [2025-10-20T17:05:26.676206] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:05:27.463954] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-04. [2025-10-20T17:05:27.494356] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:05:28.251350] [INFO ] [REMOTE ] AFGE: Fetching primary data for 2025-10. [2025-10-20T17:05:28.500463] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:05:28.726465] [INFO ] [REMOTE ] AFGE: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T17:05:28.727688] [INFO ] [REMOTE ] AFGE: Full fetch complete (3365 records). [2025-10-20T17:05:28.727798] [DEBUG] [VALIDATOR ] AFGE: Validating primary data. [2025-10-20T17:05:29.058131] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-09. [2025-10-20T17:05:29.151371] [DEBUG] [VALIDATOR ] AFGE: All expected trading days are present. [2025-10-20T17:05:29.151604] [DEBUG] [VALIDATOR ] AFGE: Primary data validation passed. [2025-10-20T17:05:29.151834] [INFO ] [MAIN ] AFGE: Fetched primary data is valid. [2025-10-20T17:05:29.151967] [DEBUG] [LOCAL ] AFGE: Saving primary data. [2025-10-20T17:05:29.152511] [DEBUG] [LOCAL ] AFGE: Trimmed 3365 records to 2313 (keeping 200 days of data) [2025-10-20T17:05:29.184631] [DEBUG] [LOCAL ] AFGE: Primary data saved (2313 records). [2025-10-20T17:05:29.196953] [INFO ] [MAIN ] AFGE: Full history saved. [2025-10-20T17:05:29.197132] [INFO ] [MAIN ] Update Progress: 49/2367 symbols completed. AFGE took 31.1s. Estimated remaining: 4h 16m 5s. Rate: 9.1 symbols/min [2025-10-20T17:05:29.243871] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:05:29.394634] [INFO ] [MAIN ] AGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:29.414200] [DEBUG] [LOCAL ] AGD: Loaded 19496 existing records from primary storage. [2025-10-20T17:05:29.414483] [INFO ] [MAIN ] AGD: Loaded 19496 existing records. [2025-10-20T17:05:29.590463] [WARN ] [MAIN ] AGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:29.590789] [INFO ] [MAIN ] AGD: Wiping existing primary data before full refresh. [2025-10-20T17:05:29.590850] [INFO ] [LOCAL ] AGD: Starting deletion of primary data. [2025-10-20T17:05:29.591761] [DEBUG] [LOCAL ] AGD: Deleted primary data file. [2025-10-20T17:05:29.591824] [INFO ] [MAIN ] AGD: Starting full primary refresh. [2025-10-20T17:05:29.861350] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-04. [2025-10-20T17:05:29.941453] [INFO ] [REMOTE ] AGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:30.651392] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-01. [2025-10-20T17:05:30.917732] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:05:31.464495] [INFO ] [REMOTE ] AFL: Fetching primary data for 2025-10. [2025-10-20T17:05:31.467176] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T17:05:31.668966] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:05:32.251410] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-05. [2025-10-20T17:05:33.051364] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-05. [2025-10-20T17:05:33.361380] [INFO ] [REMOTE ] AFL: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:05:33.369769] [INFO ] [REMOTE ] AFL: Full fetch complete (77065 records). [2025-10-20T17:05:33.369919] [DEBUG] [VALIDATOR ] AFL: Validating primary data. [2025-10-20T17:05:33.782527] [DEBUG] [VALIDATOR ] AFL: All expected trading days are present. [2025-10-20T17:05:33.782872] [DEBUG] [VALIDATOR ] AFL: Primary data validation passed. [2025-10-20T17:05:33.783890] [INFO ] [MAIN ] AFL: Fetched primary data is valid. [2025-10-20T17:05:33.783982] [DEBUG] [LOCAL ] AFL: Saving primary data. [2025-10-20T17:05:33.786428] [DEBUG] [LOCAL ] AFL: Trimmed 77065 records to 53182 (keeping 200 days of data) [2025-10-20T17:05:33.869962] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-02. [2025-10-20T17:05:33.927747] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:05:34.176154] [DEBUG] [LOCAL ] AFL: Primary data saved (53182 records). [2025-10-20T17:05:34.176296] [INFO ] [MAIN ] AFL: Full history saved. [2025-10-20T17:05:34.177635] [INFO ] [MAIN ] Update Progress: 50/2367 symbols completed. AFL took 35.2s. Estimated remaining: 4h 14m 42s. Rate: 9.1 symbols/min [2025-10-20T17:05:34.347316] [INFO ] [MAIN ] AGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:34.347499] [DEBUG] [LOCAL ] AGI: No existing data file found in primary storage. [2025-10-20T17:05:34.347543] [WARN ] [MAIN ] AGI: No existing data found, performing full refresh instead. [2025-10-20T17:05:34.347677] [INFO ] [MAIN ] AGI: Wiping existing primary data before full refresh. [2025-10-20T17:05:34.347730] [INFO ] [LOCAL ] AGI: Starting deletion of primary data. [2025-10-20T17:05:34.347772] [DEBUG] [LOCAL ] AGI: No primary data file found to delete. [2025-10-20T17:05:34.347804] [INFO ] [MAIN ] AGI: Starting full primary refresh. [2025-10-20T17:05:34.646214] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:05:34.654721] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-06. [2025-10-20T17:05:34.737003] [INFO ] [REMOTE ] AGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:35.162599] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T17:05:35.451442] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-03. [2025-10-20T17:05:35.848986] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:05:36.254074] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-01. [2025-10-20T17:05:36.258148] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:05:37.051441] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-04. [2025-10-20T17:05:37.698312] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:05:37.851464] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-06. [2025-10-20T17:05:38.055236] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:05:38.651432] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-07. [2025-10-20T17:05:39.451415] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-05. [2025-10-20T17:05:39.517587] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:05:40.176398] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:05:40.251346] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-08. [2025-10-20T17:05:40.508191] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:05:41.051381] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-06. [2025-10-20T17:05:41.531791] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T17:05:41.851369] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-07. [2025-10-20T17:05:41.939269] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T17:05:42.651458] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-02. [2025-10-20T17:05:43.451374] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-07. [2025-10-20T17:05:44.004370] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:05:44.051918] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-10-20T17:05:44.251387] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-08. [2025-10-20T17:05:44.482734] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:05:45.051426] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-09. [2025-10-20T17:05:45.852126] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-03. [2025-10-20T17:05:46.012801] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T17:05:46.651457] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-08. [2025-10-20T17:05:46.789901] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:05:47.326831] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T17:05:47.451386] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-09. [2025-10-20T17:05:47.826323] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T17:05:48.251406] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-04. [2025-10-20T17:05:49.051375] [INFO ] [REMOTE ] AGCO: Fetching primary data for 2025-10. [2025-10-20T17:05:49.518258] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:05:49.851395] [INFO ] [REMOTE ] AG: Fetching primary data for 2025-10. [2025-10-20T17:05:50.190698] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T17:05:50.651394] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-09. [2025-10-20T17:05:50.772761] [INFO ] [REMOTE ] AGCO: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:05:50.779370] [INFO ] [REMOTE ] AGCO: Full fetch complete (61605 records). [2025-10-20T17:05:50.779576] [DEBUG] [VALIDATOR ] AGCO: Validating primary data. [2025-10-20T17:05:51.185366] [DEBUG] [VALIDATOR ] AGCO: All expected trading days are present. [2025-10-20T17:05:51.185491] [DEBUG] [VALIDATOR ] AGCO: Primary data validation passed. [2025-10-20T17:05:51.186130] [INFO ] [MAIN ] AGCO: Fetched primary data is valid. [2025-10-20T17:05:51.186194] [DEBUG] [LOCAL ] AGCO: Saving primary data. [2025-10-20T17:05:51.187512] [DEBUG] [LOCAL ] AGCO: Trimmed 61605 records to 42303 (keeping 200 days of data) [2025-10-20T17:05:51.319511] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:05:51.453266] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-05. [2025-10-20T17:05:51.470459] [DEBUG] [LOCAL ] AGCO: Primary data saved (42303 records). [2025-10-20T17:05:51.470731] [INFO ] [MAIN ] AGCO: Full history saved. [2025-10-20T17:05:51.470853] [INFO ] [MAIN ] Update Progress: 51/2367 symbols completed. AGCO took 30.4s. Estimated remaining: 4h 22m 41s. Rate: 8.8 symbols/min [2025-10-20T17:05:51.664746] [INFO ] [MAIN ] AGL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:51.714930] [DEBUG] [LOCAL ] AGL: Loaded 53357 existing records from primary storage. [2025-10-20T17:05:51.715262] [INFO ] [MAIN ] AGL: Loaded 53357 existing records. [2025-10-20T17:05:51.900301] [WARN ] [MAIN ] AGL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:51.900567] [INFO ] [MAIN ] AGL: Wiping existing primary data before full refresh. [2025-10-20T17:05:51.900677] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-10-20T17:05:51.901052] [DEBUG] [LOCAL ] AGL: Deleted primary data file. [2025-10-20T17:05:51.901259] [INFO ] [MAIN ] AGL: Starting full primary refresh. [2025-10-20T17:05:52.253548] [INFO ] [REMOTE ] AG: Incremental fetch complete for 2025-10 (2.40 seconds). [2025-10-20T17:05:52.265685] [INFO ] [REMOTE ] AGD: Fetching primary data for 2025-10. [2025-10-20T17:05:52.320617] [INFO ] [REMOTE ] AG: Full fetch complete (77784 records). [2025-10-20T17:05:52.328214] [INFO ] [REMOTE ] AGL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:52.328591] [DEBUG] [VALIDATOR ] AG: Validating primary data. [2025-10-20T17:05:52.737382] [DEBUG] [VALIDATOR ] AG: All expected trading days are present. [2025-10-20T17:05:52.737478] [DEBUG] [VALIDATOR ] AG: Primary data validation passed. [2025-10-20T17:05:52.738323] [INFO ] [MAIN ] AG: Fetched primary data is valid. [2025-10-20T17:05:52.738392] [DEBUG] [LOCAL ] AG: Saving primary data. [2025-10-20T17:05:52.739991] [DEBUG] [LOCAL ] AG: Trimmed 77784 records to 53644 (keeping 200 days of data) [2025-10-20T17:05:53.059074] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-01. [2025-10-20T17:05:53.084782] [DEBUG] [LOCAL ] AG: Primary data saved (53644 records). [2025-10-20T17:05:53.085018] [INFO ] [MAIN ] AG: Full history saved. [2025-10-20T17:05:53.085986] [INFO ] [MAIN ] Update Progress: 52/2367 symbols completed. AG took 40.0s. Estimated remaining: 4h 18m 43s. Rate: 8.9 symbols/min [2025-10-20T17:05:53.284523] [INFO ] [MAIN ] AGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:53.307380] [INFO ] [REMOTE ] AGD: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:05:53.312407] [DEBUG] [LOCAL ] AGM: Loaded 10847 existing records from primary storage. [2025-10-20T17:05:53.317240] [INFO ] [MAIN ] AGM: Loaded 10847 existing records. [2025-10-20T17:05:53.332910] [INFO ] [REMOTE ] AGD: Full fetch complete (26597 records). [2025-10-20T17:05:53.349947] [DEBUG] [VALIDATOR ] AGD: Validating primary data. [2025-10-20T17:05:53.642452] [WARN ] [MAIN ] AGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:53.655190] [INFO ] [MAIN ] AGM: Wiping existing primary data before full refresh. [2025-10-20T17:05:53.655691] [INFO ] [LOCAL ] AGM: Starting deletion of primary data. [2025-10-20T17:05:53.666303] [DEBUG] [LOCAL ] AGM: Deleted primary data file. [2025-10-20T17:05:53.707965] [INFO ] [MAIN ] AGM: Starting full primary refresh. [2025-10-20T17:05:54.165412] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-05 (2.71 seconds). [2025-10-20T17:05:54.188996] [DEBUG] [VALIDATOR ] AGD: All expected trading days are present. [2025-10-20T17:05:54.254207] [DEBUG] [VALIDATOR ] AGD: Primary data validation passed. [2025-10-20T17:05:54.304022] [INFO ] [MAIN ] AGD: Fetched primary data is valid. [2025-10-20T17:05:54.339058] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-06. [2025-10-20T17:05:54.340047] [DEBUG] [LOCAL ] AGD: Saving primary data. [2025-10-20T17:05:54.391704] [DEBUG] [LOCAL ] AGD: Trimmed 26597 records to 19307 (keeping 200 days of data) [2025-10-20T17:05:54.604375] [DEBUG] [LOCAL ] AGD: Primary data saved (19307 records). [2025-10-20T17:05:54.620142] [INFO ] [MAIN ] AGD: Full history saved. [2025-10-20T17:05:54.630923] [INFO ] [MAIN ] Update Progress: 53/2367 symbols completed. AGD took 25.4s. Estimated remaining: 4h 14m 51s. Rate: 9.1 symbols/min [2025-10-20T17:05:54.690256] [INFO ] [REMOTE ] AGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:54.690622] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-01. [2025-10-20T17:05:54.900812] [INFO ] [MAIN ] AGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:05:54.899819] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:05:54.938325] [DEBUG] [LOCAL ] AGO: Loaded 28374 existing records from primary storage. [2025-10-20T17:05:54.938621] [INFO ] [MAIN ] AGO: Loaded 28374 existing records. [2025-10-20T17:05:55.114466] [WARN ] [MAIN ] AGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:05:55.114702] [INFO ] [MAIN ] AGO: Wiping existing primary data before full refresh. [2025-10-20T17:05:55.114757] [INFO ] [LOCAL ] AGO: Starting deletion of primary data. [2025-10-20T17:05:55.114929] [DEBUG] [LOCAL ] AGO: Deleted primary data file. [2025-10-20T17:05:55.114965] [INFO ] [MAIN ] AGO: Starting full primary refresh. [2025-10-20T17:05:55.475280] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-02. [2025-10-20T17:05:55.494045] [INFO ] [REMOTE ] AGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:05:55.494767] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T17:05:56.263471] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-01. [2025-10-20T17:05:56.272181] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T17:05:57.051381] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-02. [2025-10-20T17:05:57.156594] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:05:57.305638] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:05:57.518725] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:05:57.851418] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-02. [2025-10-20T17:05:58.651389] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-03. [2025-10-20T17:05:58.930075] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:05:59.451420] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-03. [2025-10-20T17:06:00.120831] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:06:00.251373] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-04. [2025-10-20T17:06:00.404081] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:06:01.051374] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-04. [2025-10-20T17:06:01.853237] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-03. [2025-10-20T17:06:01.859545] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:06:02.586103] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:06:02.600939] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T17:06:02.651374] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-04. [2025-10-20T17:06:03.389208] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:06:03.451331] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-05. [2025-10-20T17:06:04.141668] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:06:04.251373] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-06. [2025-10-20T17:06:05.015621] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:06:05.051382] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-05. [2025-10-20T17:06:05.851376] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-07. [2025-10-20T17:06:06.272402] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:06:06.651363] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-05. [2025-10-20T17:06:06.793968] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:06:07.451401] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-07. [2025-10-20T17:06:08.055142] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:06:08.251374] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-06. [2025-10-20T17:06:09.051379] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-08. [2025-10-20T17:06:09.224750] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:06:09.851353] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-08. [2025-10-20T17:06:09.897966] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:06:09.926326] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:06:10.651378] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-06. [2025-10-20T17:06:11.451347] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-09. [2025-10-20T17:06:11.789732] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T17:06:12.012376] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:06:12.034956] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:06:12.251482] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-07. [2025-10-20T17:06:13.051389] [INFO ] [REMOTE ] AGM: Fetching primary data for 2025-10. [2025-10-20T17:06:13.567587] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T17:06:13.833562] [INFO ] [REMOTE ] AGM: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:06:13.837089] [INFO ] [REMOTE ] AGM: Full fetch complete (13325 records). [2025-10-20T17:06:13.837279] [DEBUG] [VALIDATOR ] AGM: Validating primary data. [2025-10-20T17:06:13.858887] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-07. [2025-10-20T17:06:14.199951] [DEBUG] [VALIDATOR ] AGM: All expected trading days are present. [2025-10-20T17:06:14.200037] [DEBUG] [VALIDATOR ] AGM: Primary data validation passed. [2025-10-20T17:06:14.200290] [INFO ] [MAIN ] AGM: Fetched primary data is valid. [2025-10-20T17:06:14.200327] [DEBUG] [LOCAL ] AGM: Saving primary data. [2025-10-20T17:06:14.200839] [DEBUG] [LOCAL ] AGM: Trimmed 13325 records to 10702 (keeping 200 days of data) [2025-10-20T17:06:14.266526] [DEBUG] [LOCAL ] AGM: Primary data saved (10702 records). [2025-10-20T17:06:14.266698] [INFO ] [MAIN ] AGM: Full history saved. [2025-10-20T17:06:14.266788] [INFO ] [MAIN ] Update Progress: 54/2367 symbols completed. AGM took 21.2s. Estimated remaining: 4h 24m 2s. Rate: 8.8 symbols/min [2025-10-20T17:06:14.435791] [INFO ] [MAIN ] AGRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:14.467985] [DEBUG] [LOCAL ] AGRO: Loaded 33606 existing records from primary storage. [2025-10-20T17:06:14.468264] [INFO ] [MAIN ] AGRO: Loaded 33606 existing records. [2025-10-20T17:06:14.641598] [WARN ] [MAIN ] AGRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:14.641816] [INFO ] [MAIN ] AGRO: Wiping existing primary data before full refresh. [2025-10-20T17:06:14.641863] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-10-20T17:06:14.642032] [DEBUG] [LOCAL ] AGRO: Deleted primary data file. [2025-10-20T17:06:14.642069] [INFO ] [MAIN ] AGRO: Starting full primary refresh. [2025-10-20T17:06:14.656440] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-09. [2025-10-20T17:06:14.994235] [INFO ] [REMOTE ] AGRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:15.118836] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T17:06:15.451398] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-08. [2025-10-20T17:06:16.251431] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-08. [2025-10-20T17:06:16.685752] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T17:06:17.051398] [INFO ] [REMOTE ] AGI: Fetching primary data for 2025-10. [2025-10-20T17:06:17.133550] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:06:17.512025] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:06:17.851403] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-01. [2025-10-20T17:06:18.651421] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-09. [2025-10-20T17:06:18.685839] [INFO ] [REMOTE ] AGI: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:06:18.694308] [INFO ] [REMOTE ] AGI: Full fetch complete (77306 records). [2025-10-20T17:06:18.694494] [DEBUG] [VALIDATOR ] AGI: Validating primary data. [2025-10-20T17:06:19.111524] [DEBUG] [VALIDATOR ] AGI: All expected trading days are present. [2025-10-20T17:06:19.111622] [DEBUG] [VALIDATOR ] AGI: Primary data validation passed. [2025-10-20T17:06:19.112390] [INFO ] [MAIN ] AGI: Fetched primary data is valid. [2025-10-20T17:06:19.112447] [DEBUG] [LOCAL ] AGI: Saving primary data. [2025-10-20T17:06:19.113912] [DEBUG] [LOCAL ] AGI: Trimmed 77306 records to 53423 (keeping 200 days of data) [2025-10-20T17:06:19.349379] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:06:19.451827] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-02. [2025-10-20T17:06:19.471367] [DEBUG] [LOCAL ] AGI: Primary data saved (53423 records). [2025-10-20T17:06:19.471519] [INFO ] [MAIN ] AGI: Full history saved. [2025-10-20T17:06:19.471592] [INFO ] [MAIN ] Update Progress: 55/2367 symbols completed. AGI took 45.3s. Estimated remaining: 4h 22m 46s. Rate: 8.8 symbols/min [2025-10-20T17:06:19.645732] [INFO ] [MAIN ] AGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:19.677624] [DEBUG] [LOCAL ] AGX: Loaded 33316 existing records from primary storage. [2025-10-20T17:06:19.677932] [INFO ] [MAIN ] AGX: Loaded 33316 existing records. [2025-10-20T17:06:19.853565] [WARN ] [MAIN ] AGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:19.853841] [INFO ] [MAIN ] AGX: Wiping existing primary data before full refresh. [2025-10-20T17:06:19.853903] [INFO ] [LOCAL ] AGX: Starting deletion of primary data. [2025-10-20T17:06:19.854195] [DEBUG] [LOCAL ] AGX: Deleted primary data file. [2025-10-20T17:06:19.854269] [INFO ] [MAIN ] AGX: Starting full primary refresh. [2025-10-20T17:06:20.224326] [INFO ] [REMOTE ] AGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:20.254195] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-01. [2025-10-20T17:06:20.257301] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:06:20.413494] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:06:21.051359] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-09. [2025-10-20T17:06:21.851435] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-03. [2025-10-20T17:06:22.040602] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:06:22.092380] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:06:22.651413] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-02. [2025-10-20T17:06:23.329624] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:06:23.451392] [INFO ] [REMOTE ] AGO: Fetching primary data for 2025-10. [2025-10-20T17:06:24.098738] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:06:24.251477] [INFO ] [REMOTE ] AGL: Fetching primary data for 2025-10. [2025-10-20T17:06:24.311576] [INFO ] [REMOTE ] AGO: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T17:06:24.316635] [INFO ] [REMOTE ] AGO: Full fetch complete (39782 records). [2025-10-20T17:06:24.316828] [DEBUG] [VALIDATOR ] AGO: Validating primary data. [2025-10-20T17:06:24.733713] [DEBUG] [VALIDATOR ] AGO: All expected trading days are present. [2025-10-20T17:06:24.733827] [DEBUG] [VALIDATOR ] AGO: Primary data validation passed. [2025-10-20T17:06:24.734307] [INFO ] [MAIN ] AGO: Fetched primary data is valid. [2025-10-20T17:06:24.734352] [DEBUG] [LOCAL ] AGO: Saving primary data. [2025-10-20T17:06:24.735377] [DEBUG] [LOCAL ] AGO: Trimmed 39782 records to 27832 (keeping 200 days of data) [2025-10-20T17:06:24.900415] [DEBUG] [LOCAL ] AGO: Primary data saved (27832 records). [2025-10-20T17:06:24.900566] [INFO ] [MAIN ] AGO: Full history saved. [2025-10-20T17:06:24.901279] [INFO ] [MAIN ] Update Progress: 56/2367 symbols completed. AGO took 30.2s. Estimated remaining: 4h 21m 42s. Rate: 8.8 symbols/min [2025-10-20T17:06:25.069026] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-03. [2025-10-20T17:06:25.073391] [INFO ] [MAIN ] AHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:25.113253] [DEBUG] [LOCAL ] AHH: Loaded 41118 existing records from primary storage. [2025-10-20T17:06:25.113711] [INFO ] [MAIN ] AHH: Loaded 41118 existing records. [2025-10-20T17:06:25.296712] [WARN ] [MAIN ] AHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:25.296977] [INFO ] [MAIN ] AHH: Wiping existing primary data before full refresh. [2025-10-20T17:06:25.297030] [INFO ] [LOCAL ] AHH: Starting deletion of primary data. [2025-10-20T17:06:25.297240] [DEBUG] [LOCAL ] AHH: Deleted primary data file. [2025-10-20T17:06:25.297279] [INFO ] [MAIN ] AHH: Starting full primary refresh. [2025-10-20T17:06:25.657936] [INFO ] [REMOTE ] AHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:25.851420] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-04. [2025-10-20T17:06:25.884869] [INFO ] [REMOTE ] AGL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:06:25.893930] [INFO ] [REMOTE ] AGL: Full fetch complete (75529 records). [2025-10-20T17:06:25.894119] [DEBUG] [VALIDATOR ] AGL: Validating primary data. [2025-10-20T17:06:25.894985] [FATAL] [VALIDATOR ] AGL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:06:25.895081] [FATAL] [MAIN ] AGL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:06:25.895125] [WARN ] [MAIN ] AGL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:06:25.895191] [INFO ] [LOCAL ] AGL: Starting deletion of primary data. [2025-10-20T17:06:25.895257] [DEBUG] [LOCAL ] AGL: No primary data file found to delete. [2025-10-20T17:06:25.895292] [INFO ] [LOCAL ] AGL: Starting deletion of derivative data. [2025-10-20T17:06:25.895478] [DEBUG] [LOCAL ] AGL: Deleted derivative data file. [2025-10-20T17:06:25.895569] [INFO ] [MAIN ] Update Progress: 57/2367 symbols completed. AGL took 34.4s. Estimated remaining: 4h 17m 40s. Rate: 9.0 symbols/min [2025-10-20T17:06:26.065097] [INFO ] [MAIN ] AHL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:26.065310] [DEBUG] [LOCAL ] AHL: No existing data file found in primary storage. [2025-10-20T17:06:26.065348] [WARN ] [MAIN ] AHL: No existing data found, performing full refresh instead. [2025-10-20T17:06:26.065468] [INFO ] [MAIN ] AHL: Wiping existing primary data before full refresh. [2025-10-20T17:06:26.065511] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-10-20T17:06:26.065554] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-10-20T17:06:26.065587] [INFO ] [MAIN ] AHL: Starting full primary refresh. [2025-10-20T17:06:26.432476] [INFO ] [REMOTE ] AHL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:26.635007] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:06:26.651350] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-04. [2025-10-20T17:06:27.456527] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-01. [2025-10-20T17:06:27.475148] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T17:06:28.125842] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:06:28.251395] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-05. [2025-10-20T17:06:28.725281] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:06:29.051407] [INFO ] [REMOTE ] AHL: Fetching primary data for 2025-01. [2025-10-20T17:06:29.245377] [DEBUG] [REMOTE ] AHL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:06:29.245711] [WARN ] [REMOTE ] AHL: No data returned for month 2025-01 [2025-10-20T17:06:29.245798] [FATAL] [REMOTE ] AHL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:06:29.245844] [FATAL] [MAIN ] AHL: Failed to fetch primary data. [2025-10-20T17:06:29.245900] [WARN ] [MAIN ] AHL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:06:29.245972] [INFO ] [LOCAL ] AHL: Starting deletion of primary data. [2025-10-20T17:06:29.246036] [DEBUG] [LOCAL ] AHL: No primary data file found to delete. [2025-10-20T17:06:29.246072] [INFO ] [LOCAL ] AHL: Starting deletion of derivative data. [2025-10-20T17:06:29.246111] [DEBUG] [LOCAL ] AHL: No derivative data file found to delete. [2025-10-20T17:06:29.246155] [INFO ] [MAIN ] Update Progress: 58/2367 symbols completed. AHL took 3.4s. Estimated remaining: 4h 15m 21s. Rate: 9.0 symbols/min [2025-10-20T17:06:29.424517] [INFO ] [MAIN ] AHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:29.424805] [DEBUG] [LOCAL ] AHR: No existing data file found in primary storage. [2025-10-20T17:06:29.424857] [WARN ] [MAIN ] AHR: No existing data found, performing full refresh instead. [2025-10-20T17:06:29.424969] [INFO ] [MAIN ] AHR: Wiping existing primary data before full refresh. [2025-10-20T17:06:29.425011] [INFO ] [LOCAL ] AHR: Starting deletion of primary data. [2025-10-20T17:06:29.425052] [DEBUG] [LOCAL ] AHR: No primary data file found to delete. [2025-10-20T17:06:29.425083] [INFO ] [MAIN ] AHR: Starting full primary refresh. [2025-10-20T17:06:29.786554] [INFO ] [REMOTE ] AHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:29.807278] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:06:29.851360] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-06. [2025-10-20T17:06:30.651399] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-01. [2025-10-20T17:06:31.401524] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:06:31.451357] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-07. [2025-10-20T17:06:32.122956] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T17:06:32.251389] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-02. [2025-10-20T17:06:32.573716] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:06:33.051381] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-05. [2025-10-20T17:06:33.714221] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:06:33.851344] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-03. [2025-10-20T17:06:34.279206] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:06:34.651399] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-08. [2025-10-20T17:06:35.225416] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:06:35.451378] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-04. [2025-10-20T17:06:36.016953] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:06:36.251413] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-09. [2025-10-20T17:06:36.703616] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T17:06:37.051429] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-02. [2025-10-20T17:06:37.836846] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T17:06:37.851375] [INFO ] [REMOTE ] AGX: Fetching primary data for 2025-10. [2025-10-20T17:06:38.344468] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T17:06:38.651448] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-05. [2025-10-20T17:06:39.327611] [INFO ] [REMOTE ] AGX: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:06:39.332505] [INFO ] [REMOTE ] AGX: Full fetch complete (48956 records). [2025-10-20T17:06:39.332674] [DEBUG] [VALIDATOR ] AGX: Validating primary data. [2025-10-20T17:06:39.456555] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-06. [2025-10-20T17:06:39.769001] [DEBUG] [VALIDATOR ] AGX: All expected trading days are present. [2025-10-20T17:06:39.769097] [DEBUG] [VALIDATOR ] AGX: Primary data validation passed. [2025-10-20T17:06:39.769685] [INFO ] [MAIN ] AGX: Fetched primary data is valid. [2025-10-20T17:06:39.769753] [DEBUG] [LOCAL ] AGX: Saving primary data. [2025-10-20T17:06:39.771167] [DEBUG] [LOCAL ] AGX: Trimmed 48956 records to 32688 (keeping 200 days of data) [2025-10-20T17:06:39.979872] [DEBUG] [LOCAL ] AGX: Primary data saved (32688 records). [2025-10-20T17:06:39.980073] [INFO ] [MAIN ] AGX: Full history saved. [2025-10-20T17:06:39.980964] [INFO ] [MAIN ] Update Progress: 59/2367 symbols completed. AGX took 20.5s. Estimated remaining: 4h 17m 54s. Rate: 8.9 symbols/min [2025-10-20T17:06:40.159909] [INFO ] [MAIN ] AHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:40.198690] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:06:40.214078] [DEBUG] [LOCAL ] AHT: Loaded 6493 existing records from primary storage. [2025-10-20T17:06:40.215539] [INFO ] [MAIN ] AHT: Loaded 6493 existing records. [2025-10-20T17:06:40.259075] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-06. [2025-10-20T17:06:40.405834] [WARN ] [MAIN ] AHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:40.406062] [INFO ] [MAIN ] AHT: Wiping existing primary data before full refresh. [2025-10-20T17:06:40.406110] [INFO ] [LOCAL ] AHT: Starting deletion of primary data. [2025-10-20T17:06:40.406267] [DEBUG] [LOCAL ] AHT: Deleted primary data file. [2025-10-20T17:06:40.406323] [INFO ] [MAIN ] AHT: Starting full primary refresh. [2025-10-20T17:06:40.759246] [INFO ] [REMOTE ] AHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:40.784819] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:06:41.051408] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-03. [2025-10-20T17:06:41.847899] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:06:41.858302] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-07. [2025-10-20T17:06:42.651429] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-01. [2025-10-20T17:06:43.159506] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:06:43.298188] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:06:43.451352] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-04. [2025-10-20T17:06:43.524973] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:06:44.251414] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-07. [2025-10-20T17:06:45.051379] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-08. [2025-10-20T17:06:45.112173] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:06:45.851379] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-02. [2025-10-20T17:06:46.043100] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:06:46.440344] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:06:46.568726] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T17:06:46.651433] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-03. [2025-10-20T17:06:47.105093] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T17:06:47.451600] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-08. [2025-10-20T17:06:48.251441] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-04. [2025-10-20T17:06:48.781477] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T17:06:48.980279] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:06:49.051438] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-05. [2025-10-20T17:06:49.751095] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:06:49.851379] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-06. [2025-10-20T17:06:50.581894] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T17:06:50.651357] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-09. [2025-10-20T17:06:51.451402] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-05. [2025-10-20T17:06:51.998574] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:06:52.251425] [INFO ] [REMOTE ] AGRO: Fetching primary data for 2025-10. [2025-10-20T17:06:52.714442] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:06:53.051479] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-06. [2025-10-20T17:06:53.638699] [INFO ] [REMOTE ] AGRO: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:06:53.643484] [INFO ] [REMOTE ] AGRO: Full fetch complete (48855 records). [2025-10-20T17:06:53.643604] [DEBUG] [VALIDATOR ] AGRO: Validating primary data. [2025-10-20T17:06:53.644203] [FATAL] [VALIDATOR ] AGRO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:06:53.644289] [FATAL] [MAIN ] AGRO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:06:53.644331] [WARN ] [MAIN ] AGRO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:06:53.644369] [INFO ] [LOCAL ] AGRO: Starting deletion of primary data. [2025-10-20T17:06:53.644425] [DEBUG] [LOCAL ] AGRO: No primary data file found to delete. [2025-10-20T17:06:53.644458] [INFO ] [LOCAL ] AGRO: Starting deletion of derivative data. [2025-10-20T17:06:53.644711] [DEBUG] [LOCAL ] AGRO: Deleted derivative data file. [2025-10-20T17:06:53.644938] [INFO ] [MAIN ] Update Progress: 60/2367 symbols completed. AGRO took 39.4s. Estimated remaining: 4h 22m 15s. Rate: 8.8 symbols/min [2025-10-20T17:06:53.823963] [INFO ] [MAIN ] AI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:53.856387] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-09. [2025-10-20T17:06:53.880173] [DEBUG] [LOCAL ] AI: Loaded 54545 existing records from primary storage. [2025-10-20T17:06:53.880513] [INFO ] [MAIN ] AI: Loaded 54545 existing records. [2025-10-20T17:06:54.067006] [WARN ] [MAIN ] AI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:54.067613] [INFO ] [MAIN ] AI: Wiping existing primary data before full refresh. [2025-10-20T17:06:54.067694] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-10-20T17:06:54.067919] [DEBUG] [LOCAL ] AI: Deleted primary data file. [2025-10-20T17:06:54.067974] [INFO ] [MAIN ] AI: Starting full primary refresh. [2025-10-20T17:06:54.437699] [INFO ] [REMOTE ] AI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:54.460196] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T17:06:54.651345] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-01. [2025-10-20T17:06:55.451417] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-07. [2025-10-20T17:06:55.700238] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:06:56.251414] [INFO ] [REMOTE ] AHR: Fetching primary data for 2025-10. [2025-10-20T17:06:56.608289] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:06:56.949489] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-01 (2.30 seconds). [2025-10-20T17:06:57.051368] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-02. [2025-10-20T17:06:57.740256] [INFO ] [REMOTE ] AHR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:06:57.747256] [INFO ] [REMOTE ] AHR: Full fetch complete (70806 records). [2025-10-20T17:06:57.747363] [DEBUG] [VALIDATOR ] AHR: Validating primary data. [2025-10-20T17:06:57.856537] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-08. [2025-10-20T17:06:58.130448] [DEBUG] [VALIDATOR ] AHR: All expected trading days are present. [2025-10-20T17:06:58.130521] [DEBUG] [VALIDATOR ] AHR: Primary data validation passed. [2025-10-20T17:06:58.131206] [INFO ] [MAIN ] AHR: Fetched primary data is valid. [2025-10-20T17:06:58.131267] [DEBUG] [LOCAL ] AHR: Saving primary data. [2025-10-20T17:06:58.132503] [DEBUG] [LOCAL ] AHR: Trimmed 70806 records to 49006 (keeping 200 days of data) [2025-10-20T17:06:58.418524] [DEBUG] [LOCAL ] AHR: Primary data saved (49006 records). [2025-10-20T17:06:58.418621] [INFO ] [MAIN ] AHR: Full history saved. [2025-10-20T17:06:58.418728] [INFO ] [MAIN ] Update Progress: 61/2367 symbols completed. AHR took 29.2s. Estimated remaining: 4h 20m 51s. Rate: 8.8 symbols/min [2025-10-20T17:06:58.582732] [INFO ] [MAIN ] AIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:06:58.629932] [DEBUG] [LOCAL ] AIG: Loaded 54247 existing records from primary storage. [2025-10-20T17:06:58.630246] [INFO ] [MAIN ] AIG: Loaded 54247 existing records. [2025-10-20T17:06:58.661050] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-07. [2025-10-20T17:06:58.836731] [WARN ] [MAIN ] AIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:06:58.836987] [INFO ] [MAIN ] AIG: Wiping existing primary data before full refresh. [2025-10-20T17:06:58.837040] [INFO ] [LOCAL ] AIG: Starting deletion of primary data. [2025-10-20T17:06:58.837383] [DEBUG] [LOCAL ] AIG: Deleted primary data file. [2025-10-20T17:06:58.837440] [INFO ] [MAIN ] AIG: Starting full primary refresh. [2025-10-20T17:06:59.218207] [INFO ] [REMOTE ] AIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:06:59.271943] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:06:59.281733] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-02 (2.23 seconds). [2025-10-20T17:06:59.451385] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-09. [2025-10-20T17:06:59.460705] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T17:07:00.251387] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-08. [2025-10-20T17:07:00.398215] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T17:07:01.003182] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T17:07:01.051385] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-09. [2025-10-20T17:07:01.718959] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:07:01.851352] [INFO ] [REMOTE ] AHH: Fetching primary data for 2025-10. [2025-10-20T17:07:02.651372] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-01. [2025-10-20T17:07:03.451426] [INFO ] [REMOTE ] AHT: Fetching primary data for 2025-10. [2025-10-20T17:07:03.522802] [INFO ] [REMOTE ] AHH: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:07:03.528529] [INFO ] [REMOTE ] AHH: Full fetch complete (59561 records). [2025-10-20T17:07:03.528662] [DEBUG] [VALIDATOR ] AHH: Validating primary data. [2025-10-20T17:07:03.916743] [DEBUG] [VALIDATOR ] AHH: All expected trading days are present. [2025-10-20T17:07:03.916844] [DEBUG] [VALIDATOR ] AHH: Primary data validation passed. [2025-10-20T17:07:03.917498] [INFO ] [MAIN ] AHH: Fetched primary data is valid. [2025-10-20T17:07:03.917562] [DEBUG] [LOCAL ] AHH: Saving primary data. [2025-10-20T17:07:03.918903] [DEBUG] [LOCAL ] AHH: Trimmed 59561 records to 40379 (keeping 200 days of data) [2025-10-20T17:07:04.159910] [DEBUG] [LOCAL ] AHH: Primary data saved (40379 records). [2025-10-20T17:07:04.160074] [INFO ] [MAIN ] AHH: Full history saved. [2025-10-20T17:07:04.160939] [INFO ] [MAIN ] Update Progress: 62/2367 symbols completed. AHH took 39.3s. Estimated remaining: 4h 20m 5s. Rate: 8.9 symbols/min [2025-10-20T17:07:04.256613] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-03. [2025-10-20T17:07:04.370690] [INFO ] [MAIN ] AII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:04.371037] [DEBUG] [LOCAL ] AII: No existing data file found in primary storage. [2025-10-20T17:07:04.371084] [WARN ] [MAIN ] AII: No existing data found, performing full refresh instead. [2025-10-20T17:07:04.371200] [INFO ] [MAIN ] AII: Wiping existing primary data before full refresh. [2025-10-20T17:07:04.371242] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-10-20T17:07:04.371283] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-10-20T17:07:04.371317] [INFO ] [MAIN ] AII: Starting full primary refresh. [2025-10-20T17:07:04.679908] [INFO ] [REMOTE ] AHT: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T17:07:04.716387] [INFO ] [REMOTE ] AHT: Full fetch complete (9114 records). [2025-10-20T17:07:04.753454] [DEBUG] [VALIDATOR ] AHT: Validating primary data. [2025-10-20T17:07:04.759199] [INFO ] [REMOTE ] AII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:05.063993] [INFO ] [REMOTE ] AII: Fetching primary data for 2025-01. [2025-10-20T17:07:05.112310] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-01 (2.46 seconds). [2025-10-20T17:07:05.200332] [DEBUG] [VALIDATOR ] AHT: All expected trading days are present. [2025-10-20T17:07:05.200407] [DEBUG] [VALIDATOR ] AHT: Primary data validation passed. [2025-10-20T17:07:05.200638] [INFO ] [MAIN ] AHT: Fetched primary data is valid. [2025-10-20T17:07:05.200718] [DEBUG] [LOCAL ] AHT: Saving primary data. [2025-10-20T17:07:05.201123] [DEBUG] [LOCAL ] AHT: Trimmed 9114 records to 6438 (keeping 200 days of data) [2025-10-20T17:07:05.246870] [DEBUG] [LOCAL ] AHT: Primary data saved (6438 records). [2025-10-20T17:07:05.247085] [INFO ] [MAIN ] AHT: Full history saved. [2025-10-20T17:07:05.247197] [INFO ] [MAIN ] Update Progress: 63/2367 symbols completed. AHT took 25.3s. Estimated remaining: 4h 16m 31s. Rate: 9.0 symbols/min [2025-10-20T17:07:05.344584] [DEBUG] [REMOTE ] AII: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:07:05.357195] [WARN ] [REMOTE ] AII: No data returned for month 2025-01 [2025-10-20T17:07:05.357359] [FATAL] [REMOTE ] AII: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:07:05.357414] [FATAL] [MAIN ] AII: Failed to fetch primary data. [2025-10-20T17:07:05.357977] [WARN ] [MAIN ] AII: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:07:05.368746] [INFO ] [LOCAL ] AII: Starting deletion of primary data. [2025-10-20T17:07:05.384253] [DEBUG] [LOCAL ] AII: No primary data file found to delete. [2025-10-20T17:07:05.399366] [INFO ] [LOCAL ] AII: Starting deletion of derivative data. [2025-10-20T17:07:05.400801] [DEBUG] [LOCAL ] AII: No derivative data file found to delete. [2025-10-20T17:07:05.400898] [INFO ] [MAIN ] Update Progress: 64/2367 symbols completed. AII took 1.2s. Estimated remaining: 4h 12m 29s. Rate: 9.1 symbols/min [2025-10-20T17:07:05.506767] [INFO ] [MAIN ] AIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:05.628015] [INFO ] [MAIN ] AIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:05.639690] [DEBUG] [LOCAL ] AIN: Loaded 24822 existing records from primary storage. [2025-10-20T17:07:05.640072] [INFO ] [MAIN ] AIN: Loaded 24822 existing records. [2025-10-20T17:07:05.757769] [DEBUG] [LOCAL ] AIO: Loaded 13513 existing records from primary storage. [2025-10-20T17:07:05.761775] [INFO ] [MAIN ] AIO: Loaded 13513 existing records. [2025-10-20T17:07:05.861698] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-02. [2025-10-20T17:07:05.960858] [WARN ] [MAIN ] AIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:05.971987] [INFO ] [MAIN ] AIN: Wiping existing primary data before full refresh. [2025-10-20T17:07:05.972610] [INFO ] [LOCAL ] AIN: Starting deletion of primary data. [2025-10-20T17:07:06.007971] [DEBUG] [LOCAL ] AIN: Deleted primary data file. [2025-10-20T17:07:06.017547] [INFO ] [MAIN ] AIN: Starting full primary refresh. [2025-10-20T17:07:06.117199] [WARN ] [MAIN ] AIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:06.134382] [INFO ] [MAIN ] AIO: Wiping existing primary data before full refresh. [2025-10-20T17:07:06.145092] [INFO ] [LOCAL ] AIO: Starting deletion of primary data. [2025-10-20T17:07:06.155937] [DEBUG] [LOCAL ] AIO: Deleted primary data file. [2025-10-20T17:07:06.156034] [INFO ] [MAIN ] AIO: Starting full primary refresh. [2025-10-20T17:07:06.713767] [INFO ] [REMOTE ] AIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:06.732035] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-01. [2025-10-20T17:07:06.837951] [INFO ] [REMOTE ] AIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:07.138443] [INFO ] [REMOTE ] AI: Incremental fetch complete for 2025-03 (2.88 seconds). [2025-10-20T17:07:07.454425] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-01. [2025-10-20T17:07:07.524215] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:07:07.655847] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:07:08.251421] [INFO ] [REMOTE ] AI: Fetching primary data for 2025-04. [2025-10-20T17:07:08.358691] [DEBUG] [REMOTE ] AI: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T17:07:08.359068] [WARN ] [REMOTE ] AI: No data returned for month 2025-04 [2025-10-20T17:07:08.359177] [FATAL] [REMOTE ] AI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:07:08.359261] [FATAL] [MAIN ] AI: Failed to fetch primary data. [2025-10-20T17:07:08.359296] [WARN ] [MAIN ] AI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:07:08.359331] [INFO ] [LOCAL ] AI: Starting deletion of primary data. [2025-10-20T17:07:08.359387] [DEBUG] [LOCAL ] AI: No primary data file found to delete. [2025-10-20T17:07:08.359420] [INFO ] [LOCAL ] AI: Starting deletion of derivative data. [2025-10-20T17:07:08.359613] [DEBUG] [LOCAL ] AI: Deleted derivative data file. [2025-10-20T17:07:08.360067] [INFO ] [MAIN ] Update Progress: 65/2367 symbols completed. AI took 14.7s. Estimated remaining: 4h 10m 15s. Rate: 9.2 symbols/min [2025-10-20T17:07:08.552785] [INFO ] [MAIN ] AIR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:08.585523] [DEBUG] [LOCAL ] AIR: Loaded 29335 existing records from primary storage. [2025-10-20T17:07:08.586067] [INFO ] [MAIN ] AIR: Loaded 29335 existing records. [2025-10-20T17:07:08.729314] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:07:08.817495] [WARN ] [MAIN ] AIR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:08.817759] [INFO ] [MAIN ] AIR: Wiping existing primary data before full refresh. [2025-10-20T17:07:08.817816] [INFO ] [LOCAL ] AIR: Starting deletion of primary data. [2025-10-20T17:07:08.818066] [DEBUG] [LOCAL ] AIR: Deleted primary data file. [2025-10-20T17:07:08.818123] [INFO ] [MAIN ] AIR: Starting full primary refresh. [2025-10-20T17:07:09.058261] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-03. [2025-10-20T17:07:09.220428] [INFO ] [REMOTE ] AIR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:09.851379] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-02. [2025-10-20T17:07:10.651361] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-02. [2025-10-20T17:07:10.695732] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:07:10.807194] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:07:11.451370] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-03. [2025-10-20T17:07:11.570381] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T17:07:12.251357] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-01. [2025-10-20T17:07:12.560186] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:07:13.051423] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-03. [2025-10-20T17:07:13.323629] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T17:07:13.691628] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:07:13.851400] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-04. [2025-10-20T17:07:14.651437] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-04. [2025-10-20T17:07:15.399812] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:07:15.451356] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-02. [2025-10-20T17:07:15.761810] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:07:16.251373] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-05. [2025-10-20T17:07:16.693050] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T17:07:16.919741] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T17:07:17.051395] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-04. [2025-10-20T17:07:17.851404] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-03. [2025-10-20T17:07:18.439108] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:07:18.651438] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-05. [2025-10-20T17:07:19.118013] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:07:19.451430] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-06. [2025-10-20T17:07:19.584033] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T17:07:20.251351] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-05. [2025-10-20T17:07:20.476810] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:07:21.051441] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-07. [2025-10-20T17:07:21.682769] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T17:07:21.838820] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:07:21.851340] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-06. [2025-10-20T17:07:22.651402] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-04. [2025-10-20T17:07:23.451348] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-08. [2025-10-20T17:07:23.639340] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:07:23.730267] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:07:23.991219] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T17:07:24.251393] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-05. [2025-10-20T17:07:25.051427] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-09. [2025-10-20T17:07:25.101391] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T17:07:25.851388] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-07. [2025-10-20T17:07:26.037468] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:07:26.651416] [INFO ] [REMOTE ] AIO: Fetching primary data for 2025-10. [2025-10-20T17:07:27.451378] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-06. [2025-10-20T17:07:27.810283] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T17:07:27.811241] [INFO ] [REMOTE ] AIO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:07:27.827721] [INFO ] [REMOTE ] AIO: Full fetch complete (20099 records). [2025-10-20T17:07:27.827985] [DEBUG] [VALIDATOR ] AIO: Validating primary data. [2025-10-20T17:07:28.219595] [DEBUG] [VALIDATOR ] AIO: All expected trading days are present. [2025-10-20T17:07:28.219705] [DEBUG] [VALIDATOR ] AIO: Primary data validation passed. [2025-10-20T17:07:28.220082] [INFO ] [MAIN ] AIO: Fetched primary data is valid. [2025-10-20T17:07:28.220123] [DEBUG] [LOCAL ] AIO: Saving primary data. [2025-10-20T17:07:28.220723] [DEBUG] [LOCAL ] AIO: Trimmed 20099 records to 13123 (keeping 200 days of data) [2025-10-20T17:07:28.257238] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-08. [2025-10-20T17:07:28.306982] [DEBUG] [LOCAL ] AIO: Primary data saved (13123 records). [2025-10-20T17:07:28.307247] [INFO ] [MAIN ] AIO: Full history saved. [2025-10-20T17:07:28.307362] [INFO ] [MAIN ] Update Progress: 66/2367 symbols completed. AIO took 22.9s. Estimated remaining: 4h 17m 56s. Rate: 8.9 symbols/min [2025-10-20T17:07:28.499545] [INFO ] [MAIN ] AIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:28.530926] [DEBUG] [LOCAL ] AIT: Loaded 27357 existing records from primary storage. [2025-10-20T17:07:28.531333] [INFO ] [MAIN ] AIT: Loaded 27357 existing records. [2025-10-20T17:07:28.741436] [WARN ] [MAIN ] AIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:28.757278] [INFO ] [MAIN ] AIT: Wiping existing primary data before full refresh. [2025-10-20T17:07:28.758175] [INFO ] [LOCAL ] AIT: Starting deletion of primary data. [2025-10-20T17:07:28.759041] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:07:28.764706] [DEBUG] [LOCAL ] AIT: Deleted primary data file. [2025-10-20T17:07:28.765437] [INFO ] [MAIN ] AIT: Starting full primary refresh. [2025-10-20T17:07:29.056544] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-07. [2025-10-20T17:07:29.165108] [INFO ] [REMOTE ] AIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:29.851385] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-06. [2025-10-20T17:07:29.921769] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:07:30.249583] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:07:30.656474] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-09. [2025-10-20T17:07:30.664706] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:07:31.451471] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-01. [2025-10-20T17:07:32.251381] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-07. [2025-10-20T17:07:32.297345] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T17:07:32.833084] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T17:07:33.051458] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-02. [2025-10-20T17:07:33.484879] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:07:33.851438] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-08. [2025-10-20T17:07:33.965357] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:07:34.651361] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-08. [2025-10-20T17:07:34.722860] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:07:35.451392] [INFO ] [REMOTE ] AIG: Fetching primary data for 2025-10. [2025-10-20T17:07:35.941472] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:07:36.031745] [INFO ] [REMOTE ] AIG: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T17:07:36.039594] [INFO ] [REMOTE ] AIG: Full fetch complete (77569 records). [2025-10-20T17:07:36.039784] [DEBUG] [VALIDATOR ] AIG: Validating primary data. [2025-10-20T17:07:36.258149] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-09. [2025-10-20T17:07:36.452443] [DEBUG] [VALIDATOR ] AIG: All expected trading days are present. [2025-10-20T17:07:36.452535] [DEBUG] [VALIDATOR ] AIG: Primary data validation passed. [2025-10-20T17:07:36.453420] [INFO ] [MAIN ] AIG: Fetched primary data is valid. [2025-10-20T17:07:36.453488] [DEBUG] [LOCAL ] AIG: Saving primary data. [2025-10-20T17:07:36.455090] [DEBUG] [LOCAL ] AIG: Trimmed 77569 records to 53465 (keeping 200 days of data) [2025-10-20T17:07:36.801690] [DEBUG] [LOCAL ] AIG: Primary data saved (53465 records). [2025-10-20T17:07:36.801884] [INFO ] [MAIN ] AIG: Full history saved. [2025-10-20T17:07:36.801980] [INFO ] [MAIN ] Update Progress: 67/2367 symbols completed. AIG took 38.4s. Estimated remaining: 4h 18m 50s. Rate: 8.9 symbols/min [2025-10-20T17:07:36.993317] [INFO ] [MAIN ] AIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:37.036915] [DEBUG] [LOCAL ] AIV: Loaded 42194 existing records from primary storage. [2025-10-20T17:07:37.037419] [INFO ] [MAIN ] AIV: Loaded 42194 existing records. [2025-10-20T17:07:37.060116] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-03. [2025-10-20T17:07:37.235567] [WARN ] [MAIN ] AIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:37.236159] [INFO ] [MAIN ] AIV: Wiping existing primary data before full refresh. [2025-10-20T17:07:37.236395] [INFO ] [LOCAL ] AIV: Starting deletion of primary data. [2025-10-20T17:07:37.236969] [DEBUG] [LOCAL ] AIV: Deleted primary data file. [2025-10-20T17:07:37.237033] [INFO ] [MAIN ] AIV: Starting full primary refresh. [2025-10-20T17:07:37.629739] [INFO ] [REMOTE ] AIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:37.656276] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:07:37.851393] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-01. [2025-10-20T17:07:38.508925] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T17:07:38.651383] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-04. [2025-10-20T17:07:39.074078] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:07:39.451408] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-02. [2025-10-20T17:07:40.109365] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:07:40.251394] [INFO ] [REMOTE ] AIR: Fetching primary data for 2025-10. [2025-10-20T17:07:40.568226] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:07:41.051397] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-09. [2025-10-20T17:07:41.851368] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-03. [2025-10-20T17:07:41.943453] [INFO ] [REMOTE ] AIR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:07:41.949266] [INFO ] [REMOTE ] AIR: Full fetch complete (40306 records). [2025-10-20T17:07:41.949466] [DEBUG] [VALIDATOR ] AIR: Validating primary data. [2025-10-20T17:07:42.369451] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:07:42.408348] [DEBUG] [VALIDATOR ] AIR: All expected trading days are present. [2025-10-20T17:07:42.408439] [DEBUG] [VALIDATOR ] AIR: Primary data validation passed. [2025-10-20T17:07:42.409085] [INFO ] [MAIN ] AIR: Fetched primary data is valid. [2025-10-20T17:07:42.409152] [DEBUG] [LOCAL ] AIR: Saving primary data. [2025-10-20T17:07:42.410300] [DEBUG] [LOCAL ] AIR: Trimmed 40306 records to 28742 (keeping 200 days of data) [2025-10-20T17:07:42.591942] [DEBUG] [LOCAL ] AIR: Primary data saved (28742 records). [2025-10-20T17:07:42.592125] [INFO ] [MAIN ] AIR: Full history saved. [2025-10-20T17:07:42.592210] [INFO ] [MAIN ] Update Progress: 68/2367 symbols completed. AIR took 34.2s. Estimated remaining: 4h 18m 11s. Rate: 8.9 symbols/min [2025-10-20T17:07:42.656541] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-05. [2025-10-20T17:07:42.765541] [INFO ] [MAIN ] AIZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:42.765752] [DEBUG] [LOCAL ] AIZ: No existing data file found in primary storage. [2025-10-20T17:07:42.765798] [WARN ] [MAIN ] AIZ: No existing data found, performing full refresh instead. [2025-10-20T17:07:42.765909] [INFO ] [MAIN ] AIZ: Wiping existing primary data before full refresh. [2025-10-20T17:07:42.765950] [INFO ] [LOCAL ] AIZ: Starting deletion of primary data. [2025-10-20T17:07:42.765989] [DEBUG] [LOCAL ] AIZ: No primary data file found to delete. [2025-10-20T17:07:42.766021] [INFO ] [MAIN ] AIZ: Starting full primary refresh. [2025-10-20T17:07:43.131488] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T17:07:43.150467] [INFO ] [REMOTE ] AIZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:43.451412] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-04. [2025-10-20T17:07:43.940689] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T17:07:44.251440] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-01. [2025-10-20T17:07:44.925578] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:07:45.051428] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-05. [2025-10-20T17:07:45.209125] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:07:45.851374] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-02. [2025-10-20T17:07:46.200898] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:07:46.651388] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-06. [2025-10-20T17:07:47.179605] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:07:47.451390] [INFO ] [REMOTE ] AIN: Fetching primary data for 2025-10. [2025-10-20T17:07:47.593626] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:07:48.251373] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-07. [2025-10-20T17:07:48.475391] [INFO ] [REMOTE ] AIN: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T17:07:48.479188] [INFO ] [REMOTE ] AIN: Full fetch complete (33704 records). [2025-10-20T17:07:48.479261] [DEBUG] [VALIDATOR ] AIN: Validating primary data. [2025-10-20T17:07:48.863408] [DEBUG] [VALIDATOR ] AIN: All expected trading days are present. [2025-10-20T17:07:48.863485] [DEBUG] [VALIDATOR ] AIN: Primary data validation passed. [2025-10-20T17:07:48.863952] [INFO ] [MAIN ] AIN: Fetched primary data is valid. [2025-10-20T17:07:48.864013] [DEBUG] [LOCAL ] AIN: Saving primary data. [2025-10-20T17:07:48.864801] [DEBUG] [LOCAL ] AIN: Trimmed 33704 records to 24466 (keeping 200 days of data) [2025-10-20T17:07:49.008435] [DEBUG] [LOCAL ] AIN: Primary data saved (24466 records). [2025-10-20T17:07:49.008597] [INFO ] [MAIN ] AIN: Full history saved. [2025-10-20T17:07:49.008703] [INFO ] [MAIN ] Update Progress: 69/2367 symbols completed. AIN took 43.8s. Estimated remaining: 4h 17m 53s. Rate: 8.9 symbols/min [2025-10-20T17:07:49.056521] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-03. [2025-10-20T17:07:49.185363] [INFO ] [MAIN ] AIZN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:49.190474] [DEBUG] [LOCAL ] AIZN: Loaded 3995 existing records from primary storage. [2025-10-20T17:07:49.190630] [INFO ] [MAIN ] AIZN: Loaded 3995 existing records. [2025-10-20T17:07:49.377803] [WARN ] [MAIN ] AIZN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:49.378092] [INFO ] [MAIN ] AIZN: Wiping existing primary data before full refresh. [2025-10-20T17:07:49.378154] [INFO ] [LOCAL ] AIZN: Starting deletion of primary data. [2025-10-20T17:07:49.378373] [DEBUG] [LOCAL ] AIZN: Deleted primary data file. [2025-10-20T17:07:49.378419] [INFO ] [MAIN ] AIZN: Starting full primary refresh. [2025-10-20T17:07:49.643699] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:07:49.773396] [INFO ] [REMOTE ] AIZN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:49.851367] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-08. [2025-10-20T17:07:50.599005] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:07:50.651375] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-06. [2025-10-20T17:07:51.408325] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:07:51.451351] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-09. [2025-10-20T17:07:51.525486] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:07:52.251399] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-04. [2025-10-20T17:07:53.051450] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-07. [2025-10-20T17:07:53.178760] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T17:07:53.472929] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:07:53.851395] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-01. [2025-10-20T17:07:54.443687] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:07:54.519679] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:07:54.651373] [INFO ] [REMOTE ] AIV: Fetching primary data for 2025-10. [2025-10-20T17:07:55.451375] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-05. [2025-10-20T17:07:56.251367] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-08. [2025-10-20T17:07:56.391961] [INFO ] [REMOTE ] AIV: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:07:56.399141] [INFO ] [REMOTE ] AIV: Full fetch complete (60395 records). [2025-10-20T17:07:56.399274] [DEBUG] [VALIDATOR ] AIV: Validating primary data. [2025-10-20T17:07:56.805327] [DEBUG] [VALIDATOR ] AIV: All expected trading days are present. [2025-10-20T17:07:56.805419] [DEBUG] [VALIDATOR ] AIV: Primary data validation passed. [2025-10-20T17:07:56.806065] [INFO ] [MAIN ] AIV: Fetched primary data is valid. [2025-10-20T17:07:56.806129] [DEBUG] [LOCAL ] AIV: Saving primary data. [2025-10-20T17:07:56.807398] [DEBUG] [LOCAL ] AIV: Trimmed 60395 records to 41799 (keeping 200 days of data) [2025-10-20T17:07:57.032260] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:07:57.054360] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-02. [2025-10-20T17:07:57.081165] [DEBUG] [LOCAL ] AIV: Primary data saved (41799 records). [2025-10-20T17:07:57.081301] [INFO ] [MAIN ] AIV: Full history saved. [2025-10-20T17:07:57.081382] [INFO ] [MAIN ] Update Progress: 70/2367 symbols completed. AIV took 20.3s. Estimated remaining: 4h 18m 31s. Rate: 8.9 symbols/min [2025-10-20T17:07:57.263511] [INFO ] [MAIN ] AJG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:07:57.305169] [DEBUG] [LOCAL ] AJG: Loaded 47258 existing records from primary storage. [2025-10-20T17:07:57.305499] [INFO ] [MAIN ] AJG: Loaded 47258 existing records. [2025-10-20T17:07:57.495333] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:07:57.501137] [WARN ] [MAIN ] AJG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:07:57.506781] [INFO ] [MAIN ] AJG: Wiping existing primary data before full refresh. [2025-10-20T17:07:57.507174] [INFO ] [LOCAL ] AJG: Starting deletion of primary data. [2025-10-20T17:07:57.507574] [DEBUG] [LOCAL ] AJG: Deleted primary data file. [2025-10-20T17:07:57.507632] [INFO ] [MAIN ] AJG: Starting full primary refresh. [2025-10-20T17:07:57.831708] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:07:57.863933] [INFO ] [REMOTE ] AJG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:07:57.856961] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-09. [2025-10-20T17:07:58.651380] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-06. [2025-10-20T17:07:58.812785] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:07:59.451376] [INFO ] [REMOTE ] AIT: Fetching primary data for 2025-10. [2025-10-20T17:07:59.940357] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:08:00.251381] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-07. [2025-10-20T17:08:00.279824] [INFO ] [REMOTE ] AIT: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:08:00.285179] [INFO ] [REMOTE ] AIT: Full fetch complete (39190 records). [2025-10-20T17:08:00.285440] [DEBUG] [VALIDATOR ] AIT: Validating primary data. [2025-10-20T17:08:00.710924] [DEBUG] [VALIDATOR ] AIT: All expected trading days are present. [2025-10-20T17:08:00.711014] [DEBUG] [VALIDATOR ] AIT: Primary data validation passed. [2025-10-20T17:08:00.711507] [INFO ] [MAIN ] AIT: Fetched primary data is valid. [2025-10-20T17:08:00.711548] [DEBUG] [LOCAL ] AIT: Saving primary data. [2025-10-20T17:08:00.712571] [DEBUG] [LOCAL ] AIT: Trimmed 39190 records to 26804 (keeping 200 days of data) [2025-10-20T17:08:00.877127] [DEBUG] [LOCAL ] AIT: Primary data saved (26804 records). [2025-10-20T17:08:00.877275] [INFO ] [MAIN ] AIT: Full history saved. [2025-10-20T17:08:00.877353] [INFO ] [MAIN ] Update Progress: 71/2367 symbols completed. AIT took 32.6s. Estimated remaining: 4h 16m 48s. Rate: 8.9 symbols/min [2025-10-20T17:08:01.054788] [INFO ] [MAIN ] AKA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:01.052851] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-01. [2025-10-20T17:08:01.055981] [DEBUG] [LOCAL ] AKA: No existing data file found in primary storage. [2025-10-20T17:08:01.056049] [WARN ] [MAIN ] AKA: No existing data found, performing full refresh instead. [2025-10-20T17:08:01.056195] [INFO ] [MAIN ] AKA: Wiping existing primary data before full refresh. [2025-10-20T17:08:01.056250] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-10-20T17:08:01.056327] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-10-20T17:08:01.056371] [INFO ] [MAIN ] AKA: Starting full primary refresh. [2025-10-20T17:08:01.483463] [INFO ] [REMOTE ] AKA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:01.645353] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:08:01.851373] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-08. [2025-10-20T17:08:02.651397] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-01. [2025-10-20T17:08:02.867014] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:08:03.168708] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:08:03.243904] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:08:03.451372] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-09. [2025-10-20T17:08:04.251434] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-03. [2025-10-20T17:08:04.851490] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:08:04.959774] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T17:08:05.051373] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-04. [2025-10-20T17:08:05.620753] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T17:08:05.851372] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-05. [2025-10-20T17:08:06.413535] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:08:06.651378] [INFO ] [REMOTE ] AIZ: Fetching primary data for 2025-10. [2025-10-20T17:08:07.451376] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-02. [2025-10-20T17:08:07.645828] [INFO ] [REMOTE ] AIZ: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:08:07.649821] [INFO ] [REMOTE ] AIZ: Full fetch complete (42798 records). [2025-10-20T17:08:07.649950] [DEBUG] [VALIDATOR ] AIZ: Validating primary data. [2025-10-20T17:08:08.036171] [DEBUG] [VALIDATOR ] AIZ: All expected trading days are present. [2025-10-20T17:08:08.036249] [DEBUG] [VALIDATOR ] AIZ: Primary data validation passed. [2025-10-20T17:08:08.036729] [INFO ] [MAIN ] AIZ: Fetched primary data is valid. [2025-10-20T17:08:08.036789] [DEBUG] [LOCAL ] AIZ: Saving primary data. [2025-10-20T17:08:08.037699] [DEBUG] [LOCAL ] AIZ: Trimmed 42798 records to 29471 (keeping 200 days of data) [2025-10-20T17:08:08.217771] [DEBUG] [LOCAL ] AIZ: Primary data saved (29471 records). [2025-10-20T17:08:08.217911] [INFO ] [MAIN ] AIZ: Full history saved. [2025-10-20T17:08:08.217981] [INFO ] [MAIN ] Update Progress: 72/2367 symbols completed. AIZ took 25.6s. Estimated remaining: 4h 17m 2s. Rate: 8.9 symbols/min [2025-10-20T17:08:08.256632] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-06. [2025-10-20T17:08:08.399559] [INFO ] [MAIN ] AKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:08.442467] [DEBUG] [LOCAL ] AKR: Loaded 46788 existing records from primary storage. [2025-10-20T17:08:08.442799] [INFO ] [MAIN ] AKR: Loaded 46788 existing records. [2025-10-20T17:08:08.623513] [WARN ] [MAIN ] AKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:08.623782] [INFO ] [MAIN ] AKR: Wiping existing primary data before full refresh. [2025-10-20T17:08:08.623832] [INFO ] [LOCAL ] AKR: Starting deletion of primary data. [2025-10-20T17:08:08.624030] [DEBUG] [LOCAL ] AKR: Deleted primary data file. [2025-10-20T17:08:08.624114] [INFO ] [MAIN ] AKR: Starting full primary refresh. [2025-10-20T17:08:08.965088] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:08:08.995363] [INFO ] [REMOTE ] AKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:09.051332] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-01. [2025-10-20T17:08:09.133592] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:08:09.851380] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-02. [2025-10-20T17:08:10.164305] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T17:08:10.657774] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-07. [2025-10-20T17:08:10.660181] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:08:11.003430] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T17:08:11.451385] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-02. [2025-10-20T17:08:12.251387] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-08. [2025-10-20T17:08:12.540403] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:08:12.818864] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T17:08:13.051415] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-03. [2025-10-20T17:08:13.604133] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:08:13.851395] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-03. [2025-10-20T17:08:14.651395] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-09. [2025-10-20T17:08:15.305839] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T17:08:15.451396] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-03. [2025-10-20T17:08:15.566105] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:08:16.251408] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-04. [2025-10-20T17:08:16.944377] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:08:17.051411] [INFO ] [REMOTE ] AIZN: Fetching primary data for 2025-10. [2025-10-20T17:08:17.591200] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:08:17.617487] [INFO ] [REMOTE ] AIZN: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T17:08:17.619347] [INFO ] [REMOTE ] AIZN: Full fetch complete (5826 records). [2025-10-20T17:08:17.619457] [DEBUG] [VALIDATOR ] AIZN: Validating primary data. [2025-10-20T17:08:17.856447] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-05. [2025-10-20T17:08:18.043703] [DEBUG] [VALIDATOR ] AIZN: All expected trading days are present. [2025-10-20T17:08:18.043791] [DEBUG] [VALIDATOR ] AIZN: Primary data validation passed. [2025-10-20T17:08:18.044015] [INFO ] [MAIN ] AIZN: Fetched primary data is valid. [2025-10-20T17:08:18.044055] [DEBUG] [LOCAL ] AIZN: Saving primary data. [2025-10-20T17:08:18.044444] [DEBUG] [LOCAL ] AIZN: Trimmed 5826 records to 3848 (keeping 200 days of data) [2025-10-20T17:08:18.069802] [DEBUG] [LOCAL ] AIZN: Primary data saved (3848 records). [2025-10-20T17:08:18.069941] [INFO ] [MAIN ] AIZN: Full history saved. [2025-10-20T17:08:18.070022] [INFO ] [MAIN ] Update Progress: 73/2367 symbols completed. AIZN took 29.1s. Estimated remaining: 4h 18m 33s. Rate: 8.9 symbols/min [2025-10-20T17:08:18.255796] [INFO ] [MAIN ] AL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:18.297360] [DEBUG] [LOCAL ] AL: Loaded 45438 existing records from primary storage. [2025-10-20T17:08:18.297686] [INFO ] [MAIN ] AL: Loaded 45438 existing records. [2025-10-20T17:08:18.475199] [WARN ] [MAIN ] AL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:18.475496] [INFO ] [MAIN ] AL: Wiping existing primary data before full refresh. [2025-10-20T17:08:18.475552] [INFO ] [LOCAL ] AL: Starting deletion of primary data. [2025-10-20T17:08:18.475856] [DEBUG] [LOCAL ] AL: Deleted primary data file. [2025-10-20T17:08:18.475912] [INFO ] [MAIN ] AL: Starting full primary refresh. [2025-10-20T17:08:18.657457] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-04. [2025-10-20T17:08:18.844937] [INFO ] [REMOTE ] AL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:19.451401] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-01. [2025-10-20T17:08:19.610982] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:08:20.251417] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-06. [2025-10-20T17:08:20.405798] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:08:21.051356] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-04. [2025-10-20T17:08:21.116078] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:08:21.349059] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:08:21.596163] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:08:21.851358] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-07. [2025-10-20T17:08:22.651386] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-05. [2025-10-20T17:08:23.192169] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T17:08:23.214303] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:08:23.451417] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-06. [2025-10-20T17:08:23.916909] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T17:08:24.251408] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-08. [2025-10-20T17:08:25.051423] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-05. [2025-10-20T17:08:25.851461] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-07. [2025-10-20T17:08:26.340978] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T17:08:26.651464] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-08. [2025-10-20T17:08:26.702116] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-10-20T17:08:26.825533] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:08:26.981405] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-10-20T17:08:27.451391] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-09. [2025-10-20T17:08:28.251387] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-06. [2025-10-20T17:08:28.867509] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T17:08:29.051402] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-02. [2025-10-20T17:08:29.765195] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T17:08:29.851410] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-07. [2025-10-20T17:08:30.560031] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T17:08:30.651379] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-03. [2025-10-20T17:08:31.347469] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:08:31.451372] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-08. [2025-10-20T17:08:32.251405] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-09. [2025-10-20T17:08:32.304933] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:08:32.569607] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T17:08:32.974961] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:08:33.051400] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-09. [2025-10-20T17:08:33.851434] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-04. [2025-10-20T17:08:34.659073] [INFO ] [REMOTE ] AKA: Fetching primary data for 2025-10. [2025-10-20T17:08:34.677147] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:08:35.451391] [INFO ] [REMOTE ] AKR: Fetching primary data for 2025-10. [2025-10-20T17:08:35.501415] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:08:35.501714] [INFO ] [REMOTE ] AKA: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:08:35.514188] [INFO ] [REMOTE ] AKA: Full fetch complete (2801 records). [2025-10-20T17:08:35.514280] [DEBUG] [VALIDATOR ] AKA: Validating primary data. [2025-10-20T17:08:35.726213] [WARN ] [VALIDATOR ] AKA: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T17:08:35.726458] [FATAL] [MAIN ] AKA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:08:35.726530] [WARN ] [MAIN ] AKA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:08:35.726583] [INFO ] [LOCAL ] AKA: Starting deletion of primary data. [2025-10-20T17:08:35.726736] [DEBUG] [LOCAL ] AKA: No primary data file found to delete. [2025-10-20T17:08:35.726782] [INFO ] [LOCAL ] AKA: Starting deletion of derivative data. [2025-10-20T17:08:35.726825] [DEBUG] [LOCAL ] AKA: No derivative data file found to delete. [2025-10-20T17:08:35.726900] [INFO ] [MAIN ] Update Progress: 74/2367 symbols completed. AKA took 34.8s. Estimated remaining: 4h 24m 4s. Rate: 8.7 symbols/min [2025-10-20T17:08:35.905719] [INFO ] [MAIN ] ALB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:35.962188] [DEBUG] [LOCAL ] ALB: Loaded 54123 existing records from primary storage. [2025-10-20T17:08:35.962525] [INFO ] [MAIN ] ALB: Loaded 54123 existing records. [2025-10-20T17:08:36.152975] [WARN ] [MAIN ] ALB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:36.153231] [INFO ] [MAIN ] ALB: Wiping existing primary data before full refresh. [2025-10-20T17:08:36.153282] [INFO ] [LOCAL ] ALB: Starting deletion of primary data. [2025-10-20T17:08:36.153497] [DEBUG] [LOCAL ] ALB: Deleted primary data file. [2025-10-20T17:08:36.153552] [INFO ] [MAIN ] ALB: Starting full primary refresh. [2025-10-20T17:08:36.257130] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-05. [2025-10-20T17:08:36.531334] [INFO ] [REMOTE ] ALB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:37.051383] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-01. [2025-10-20T17:08:37.237679] [INFO ] [REMOTE ] AKR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T17:08:37.244463] [INFO ] [REMOTE ] AKR: Full fetch complete (65453 records). [2025-10-20T17:08:37.244593] [DEBUG] [VALIDATOR ] AKR: Validating primary data. [2025-10-20T17:08:37.626288] [DEBUG] [VALIDATOR ] AKR: All expected trading days are present. [2025-10-20T17:08:37.626398] [DEBUG] [VALIDATOR ] AKR: Primary data validation passed. [2025-10-20T17:08:37.627166] [INFO ] [MAIN ] AKR: Fetched primary data is valid. [2025-10-20T17:08:37.627251] [DEBUG] [LOCAL ] AKR: Saving primary data. [2025-10-20T17:08:37.628838] [DEBUG] [LOCAL ] AKR: Trimmed 65453 records to 46127 (keeping 200 days of data) [2025-10-20T17:08:37.763243] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:08:37.853099] [INFO ] [REMOTE ] AJG: Fetching primary data for 2025-10. [2025-10-20T17:08:37.965493] [DEBUG] [LOCAL ] AKR: Primary data saved (46127 records). [2025-10-20T17:08:37.965718] [INFO ] [MAIN ] AKR: Full history saved. [2025-10-20T17:08:37.966272] [INFO ] [MAIN ] Update Progress: 75/2367 symbols completed. AKR took 29.7s. Estimated remaining: 4h 21m 34s. Rate: 8.8 symbols/min [2025-10-20T17:08:38.155902] [INFO ] [MAIN ] ALC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:38.205466] [DEBUG] [LOCAL ] ALC: Loaded 51959 existing records from primary storage. [2025-10-20T17:08:38.205824] [INFO ] [MAIN ] ALC: Loaded 51959 existing records. [2025-10-20T17:08:38.384080] [WARN ] [MAIN ] ALC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:38.384293] [INFO ] [MAIN ] ALC: Wiping existing primary data before full refresh. [2025-10-20T17:08:38.384340] [INFO ] [LOCAL ] ALC: Starting deletion of primary data. [2025-10-20T17:08:38.384540] [DEBUG] [LOCAL ] ALC: Deleted primary data file. [2025-10-20T17:08:38.384577] [INFO ] [MAIN ] ALC: Starting full primary refresh. [2025-10-20T17:08:38.651662] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-06. [2025-10-20T17:08:38.745911] [INFO ] [REMOTE ] ALC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:38.902766] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T17:08:39.451426] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-01. [2025-10-20T17:08:39.733348] [INFO ] [REMOTE ] AJG: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T17:08:39.741208] [INFO ] [REMOTE ] AJG: Full fetch complete (68999 records). [2025-10-20T17:08:39.741408] [DEBUG] [VALIDATOR ] AJG: Validating primary data. [2025-10-20T17:08:40.177713] [DEBUG] [VALIDATOR ] AJG: All expected trading days are present. [2025-10-20T17:08:40.178268] [DEBUG] [VALIDATOR ] AJG: Primary data validation passed. [2025-10-20T17:08:40.179062] [INFO ] [MAIN ] AJG: Fetched primary data is valid. [2025-10-20T17:08:40.181316] [DEBUG] [LOCAL ] AJG: Saving primary data. [2025-10-20T17:08:40.181210] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:08:40.192328] [DEBUG] [LOCAL ] AJG: Trimmed 68999 records to 46490 (keeping 200 days of data) [2025-10-20T17:08:40.255810] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-07. [2025-10-20T17:08:40.497308] [DEBUG] [LOCAL ] AJG: Primary data saved (46490 records). [2025-10-20T17:08:40.497559] [INFO ] [MAIN ] AJG: Full history saved. [2025-10-20T17:08:40.498492] [INFO ] [MAIN ] Update Progress: 76/2367 symbols completed. AJG took 43.4s. Estimated remaining: 4h 19m 18s. Rate: 8.8 symbols/min [2025-10-20T17:08:40.673374] [INFO ] [MAIN ] ALE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:08:40.705202] [DEBUG] [LOCAL ] ALE: Loaded 32507 existing records from primary storage. [2025-10-20T17:08:40.705479] [INFO ] [MAIN ] ALE: Loaded 32507 existing records. [2025-10-20T17:08:40.887563] [WARN ] [MAIN ] ALE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:08:40.887827] [INFO ] [MAIN ] ALE: Wiping existing primary data before full refresh. [2025-10-20T17:08:40.887878] [INFO ] [LOCAL ] ALE: Starting deletion of primary data. [2025-10-20T17:08:40.888108] [DEBUG] [LOCAL ] ALE: Deleted primary data file. [2025-10-20T17:08:40.888169] [INFO ] [MAIN ] ALE: Starting full primary refresh. [2025-10-20T17:08:41.062133] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-02. [2025-10-20T17:08:41.254680] [INFO ] [REMOTE ] ALE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:08:41.285320] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:08:41.394869] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:08:41.851417] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-02. [2025-10-20T17:08:42.651446] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-01. [2025-10-20T17:08:42.854679] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:08:43.451451] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-03. [2025-10-20T17:08:43.522729] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T17:08:43.695224] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:08:44.251373] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-02. [2025-10-20T17:08:45.041777] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T17:08:45.051321] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-03. [2025-10-20T17:08:45.143944] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:08:45.851413] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-03. [2025-10-20T17:08:46.155767] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:08:46.651434] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-08. [2025-10-20T17:08:47.451414] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-04. [2025-10-20T17:08:47.595473] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:08:47.848903] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T17:08:48.251390] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-09. [2025-10-20T17:08:49.051400] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-04. [2025-10-20T17:08:49.496522] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T17:08:49.851388] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-05. [2025-10-20T17:08:49.963484] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:08:50.651383] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-04. [2025-10-20T17:08:50.831152] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:08:51.451353] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-05. [2025-10-20T17:08:51.853530] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:08:51.971575] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T17:08:52.251449] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-05. [2025-10-20T17:08:53.051393] [INFO ] [REMOTE ] AL: Fetching primary data for 2025-10. [2025-10-20T17:08:53.290786] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:08:53.573899] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:08:53.851365] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-06. [2025-10-20T17:08:54.651392] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-06. [2025-10-20T17:08:54.777682] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:08:54.809944] [INFO ] [REMOTE ] AL: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:08:54.816880] [INFO ] [REMOTE ] AL: Full fetch complete (63611 records). [2025-10-20T17:08:54.816978] [DEBUG] [VALIDATOR ] AL: Validating primary data. [2025-10-20T17:08:55.221788] [DEBUG] [VALIDATOR ] AL: All expected trading days are present. [2025-10-20T17:08:55.221889] [DEBUG] [VALIDATOR ] AL: Primary data validation passed. [2025-10-20T17:08:55.222577] [INFO ] [MAIN ] AL: Fetched primary data is valid. [2025-10-20T17:08:55.222670] [DEBUG] [LOCAL ] AL: Saving primary data. [2025-10-20T17:08:55.224105] [DEBUG] [LOCAL ] AL: Trimmed 63611 records to 44827 (keeping 200 days of data) [2025-10-20T17:08:55.452393] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-06. [2025-10-20T17:08:55.496000] [DEBUG] [LOCAL ] AL: Primary data saved (44827 records). [2025-10-20T17:08:55.496197] [INFO ] [MAIN ] AL: Full history saved. [2025-10-20T17:08:55.496884] [INFO ] [MAIN ] Update Progress: 77/2367 symbols completed. AL took 37.4s. Estimated remaining: 4h 23m 15s. Rate: 8.7 symbols/min [2025-10-20T17:08:56.251352] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-07. [2025-10-20T17:08:56.661555] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T17:08:56.973866] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:08:57.051366] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-07. [2025-10-20T17:08:57.732838] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:08:57.851401] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-08. [2025-10-20T17:08:58.651404] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-07. [2025-10-20T17:08:58.708934] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:08:59.038499] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:08:59.451369] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-09. [2025-10-20T17:09:00.251372] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-08. [2025-10-20T17:09:00.913511] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:09:01.051370] [INFO ] [REMOTE ] ALE: Fetching primary data for 2025-10. [2025-10-20T17:09:01.139220] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-10-20T17:09:01.642133] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:09:01.851349] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-09. [2025-10-20T17:09:02.515385] [INFO ] [REMOTE ] ALE: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:09:02.521719] [INFO ] [REMOTE ] ALE: Full fetch complete (44169 records). [2025-10-20T17:09:02.521890] [DEBUG] [VALIDATOR ] ALE: Validating primary data. [2025-10-20T17:09:02.656576] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-08. [2025-10-20T17:09:02.958043] [DEBUG] [VALIDATOR ] ALE: All expected trading days are present. [2025-10-20T17:09:02.958148] [DEBUG] [VALIDATOR ] ALE: Primary data validation passed. [2025-10-20T17:09:02.958790] [INFO ] [MAIN ] ALE: Fetched primary data is valid. [2025-10-20T17:09:02.958869] [DEBUG] [LOCAL ] ALE: Saving primary data. [2025-10-20T17:09:02.960060] [DEBUG] [LOCAL ] ALE: Trimmed 44169 records to 32104 (keeping 200 days of data) [2025-10-20T17:09:03.163417] [DEBUG] [LOCAL ] ALE: Primary data saved (32104 records). [2025-10-20T17:09:03.163546] [INFO ] [MAIN ] ALE: Full history saved. [2025-10-20T17:09:03.163625] [INFO ] [MAIN ] Update Progress: 78/2367 symbols completed. ALE took 22.7s. Estimated remaining: 4h 23m 30s. Rate: 8.7 symbols/min [2025-10-20T17:09:03.878769] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T17:09:03.889814] [INFO ] [REMOTE ] ALC: Fetching primary data for 2025-10. [2025-10-20T17:09:05.039492] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T17:09:05.051171] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-09. [2025-10-20T17:09:05.609375] [INFO ] [REMOTE ] ALC: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:09:05.617220] [INFO ] [REMOTE ] ALC: Full fetch complete (73767 records). [2025-10-20T17:09:05.617392] [DEBUG] [VALIDATOR ] ALC: Validating primary data. [2025-10-20T17:09:06.007692] [DEBUG] [VALIDATOR ] ALC: All expected trading days are present. [2025-10-20T17:09:06.007795] [DEBUG] [VALIDATOR ] ALC: Primary data validation passed. [2025-10-20T17:09:06.008530] [INFO ] [MAIN ] ALC: Fetched primary data is valid. [2025-10-20T17:09:06.008620] [DEBUG] [LOCAL ] ALC: Saving primary data. [2025-10-20T17:09:06.010122] [DEBUG] [LOCAL ] ALC: Trimmed 73767 records to 51212 (keeping 200 days of data) [2025-10-20T17:09:06.318039] [DEBUG] [LOCAL ] ALC: Primary data saved (51212 records). [2025-10-20T17:09:06.318195] [INFO ] [MAIN ] ALC: Full history saved. [2025-10-20T17:09:06.318899] [INFO ] [MAIN ] Update Progress: 79/2367 symbols completed. ALC took 28.4s. Estimated remaining: 4h 21m 35s. Rate: 8.7 symbols/min [2025-10-20T17:09:07.089122] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T17:09:07.100461] [INFO ] [REMOTE ] ALB: Fetching primary data for 2025-10. [2025-10-20T17:09:09.719955] [INFO ] [REMOTE ] ALB: Incremental fetch complete for 2025-10 (2.62 seconds). [2025-10-20T17:09:09.727828] [INFO ] [REMOTE ] ALB: Full fetch complete (77075 records). [2025-10-20T17:09:09.727983] [DEBUG] [VALIDATOR ] ALB: Validating primary data. [2025-10-20T17:09:10.130460] [DEBUG] [VALIDATOR ] ALB: All expected trading days are present. [2025-10-20T17:09:10.130573] [DEBUG] [VALIDATOR ] ALB: Primary data validation passed. [2025-10-20T17:09:10.131480] [INFO ] [MAIN ] ALB: Fetched primary data is valid. [2025-10-20T17:09:10.131555] [DEBUG] [LOCAL ] ALB: Saving primary data. [2025-10-20T17:09:10.133245] [DEBUG] [LOCAL ] ALB: Trimmed 77075 records to 53378 (keeping 200 days of data) [2025-10-20T17:09:10.450065] [DEBUG] [LOCAL ] ALB: Primary data saved (53378 records). [2025-10-20T17:09:10.450216] [INFO ] [MAIN ] ALB: Full history saved. [2025-10-20T17:09:10.450912] [INFO ] [MAIN ] Update Progress: 80/2367 symbols completed. ALB took 34.7s. Estimated remaining: 4h 20m 10s. Rate: 8.8 symbols/min [2025-10-20T17:09:11.063336] [INFO ] [MAIN ] ALIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.090972] [INFO ] [MAIN ] ALG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.212609] [INFO ] [MAIN ] ALH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.213290] [DEBUG] [LOCAL ] ALH: No existing data file found in primary storage. [2025-10-20T17:09:11.228602] [WARN ] [MAIN ] ALH: No existing data found, performing full refresh instead. [2025-10-20T17:09:11.232310] [INFO ] [MAIN ] ALEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:11.232688] [INFO ] [MAIN ] ALH: Wiping existing primary data before full refresh. [2025-10-20T17:09:11.233586] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-10-20T17:09:11.233695] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-10-20T17:09:11.233751] [INFO ] [MAIN ] ALH: Starting full primary refresh. [2025-10-20T17:09:11.347826] [DEBUG] [LOCAL ] ALG: Loaded 9877 existing records from primary storage. [2025-10-20T17:09:11.354800] [INFO ] [MAIN ] ALG: Loaded 9877 existing records. [2025-10-20T17:09:11.585114] [DEBUG] [LOCAL ] ALEX: Loaded 31633 existing records from primary storage. [2025-10-20T17:09:11.669013] [INFO ] [MAIN ] ALEX: Loaded 31633 existing records. [2025-10-20T17:09:11.790239] [WARN ] [MAIN ] ALG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:11.849027] [INFO ] [MAIN ] ALG: Wiping existing primary data before full refresh. [2025-10-20T17:09:11.830236] [DEBUG] [LOCAL ] ALIT: Loaded 53560 existing records from primary storage. [2025-10-20T17:09:11.854337] [INFO ] [LOCAL ] ALG: Starting deletion of primary data. [2025-10-20T17:09:11.908151] [DEBUG] [LOCAL ] ALG: Deleted primary data file. [2025-10-20T17:09:11.875737] [INFO ] [MAIN ] ALIT: Loaded 53560 existing records. [2025-10-20T17:09:11.908400] [INFO ] [MAIN ] ALG: Starting full primary refresh. [2025-10-20T17:09:12.038761] [INFO ] [REMOTE ] ALH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:12.125679] [INFO ] [REMOTE ] ALH: Fetching primary data for 2025-01. [2025-10-20T17:09:12.247852] [WARN ] [MAIN ] ALEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:12.277082] [INFO ] [MAIN ] ALEX: Wiping existing primary data before full refresh. [2025-10-20T17:09:12.277420] [INFO ] [LOCAL ] ALEX: Starting deletion of primary data. [2025-10-20T17:09:12.308723] [DEBUG] [LOCAL ] ALEX: Deleted primary data file. [2025-10-20T17:09:12.398750] [INFO ] [MAIN ] ALEX: Starting full primary refresh. [2025-10-20T17:09:12.402564] [WARN ] [MAIN ] ALIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:12.467372] [INFO ] [MAIN ] ALIT: Wiping existing primary data before full refresh. [2025-10-20T17:09:12.468220] [INFO ] [LOCAL ] ALIT: Starting deletion of primary data. [2025-10-20T17:09:12.484825] [DEBUG] [REMOTE ] ALH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:09:12.493943] [WARN ] [REMOTE ] ALH: No data returned for month 2025-01 [2025-10-20T17:09:12.499237] [FATAL] [REMOTE ] ALH: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:09:12.520121] [FATAL] [MAIN ] ALH: Failed to fetch primary data. [2025-10-20T17:09:12.549766] [DEBUG] [LOCAL ] ALIT: Deleted primary data file. [2025-10-20T17:09:12.625680] [INFO ] [MAIN ] ALIT: Starting full primary refresh. [2025-10-20T17:09:12.615171] [WARN ] [MAIN ] ALH: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:09:12.641548] [INFO ] [LOCAL ] ALH: Starting deletion of primary data. [2025-10-20T17:09:12.734620] [DEBUG] [LOCAL ] ALH: No primary data file found to delete. [2025-10-20T17:09:12.852471] [INFO ] [LOCAL ] ALH: Starting deletion of derivative data. [2025-10-20T17:09:12.937624] [DEBUG] [LOCAL ] ALH: No derivative data file found to delete. [2025-10-20T17:09:13.029435] [INFO ] [REMOTE ] ALG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.080158] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-01. [2025-10-20T17:09:13.079829] [INFO ] [MAIN ] Update Progress: 81/2367 symbols completed. ALH took 2.6s. Estimated remaining: 4h 18m 5s. Rate: 8.9 symbols/min [2025-10-20T17:09:13.446950] [INFO ] [REMOTE ] ALEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.468409] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-01. [2025-10-20T17:09:13.580311] [INFO ] [MAIN ] ALK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:13.635165] [INFO ] [REMOTE ] ALIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:13.635787] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-01. [2025-10-20T17:09:13.693362] [DEBUG] [LOCAL ] ALK: Loaded 53860 existing records from primary storage. [2025-10-20T17:09:13.693890] [INFO ] [MAIN ] ALK: Loaded 53860 existing records. [2025-10-20T17:09:13.898279] [WARN ] [MAIN ] ALK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:13.898947] [INFO ] [MAIN ] ALK: Wiping existing primary data before full refresh. [2025-10-20T17:09:13.899054] [INFO ] [LOCAL ] ALK: Starting deletion of primary data. [2025-10-20T17:09:13.899602] [DEBUG] [LOCAL ] ALK: Deleted primary data file. [2025-10-20T17:09:13.899810] [INFO ] [MAIN ] ALK: Starting full primary refresh. [2025-10-20T17:09:14.281153] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:09:14.326706] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-02. [2025-10-20T17:09:14.325803] [INFO ] [REMOTE ] ALK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:14.328078] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-01. [2025-10-20T17:09:14.676187] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:09:14.682208] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-02. [2025-10-20T17:09:15.209463] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:09:15.212556] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-03. [2025-10-20T17:09:15.463795] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:09:15.475129] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-02. [2025-10-20T17:09:15.490849] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:09:15.496337] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-03. [2025-10-20T17:09:15.852761] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:09:15.856400] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-04. [2025-10-20T17:09:16.331189] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:09:16.341628] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-02. [2025-10-20T17:09:16.692121] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:09:16.713794] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:09:16.714913] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-03. [2025-10-20T17:09:16.718876] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-05. [2025-10-20T17:09:16.860820] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:09:17.051402] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-04. [2025-10-20T17:09:17.408926] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:09:17.674935] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:09:17.851413] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-03. [2025-10-20T17:09:18.316839] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:09:18.474206] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T17:09:18.651370] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-06. [2025-10-20T17:09:19.456458] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-05. [2025-10-20T17:09:19.472989] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:09:19.562777] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T17:09:20.251394] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-04. [2025-10-20T17:09:20.628327] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T17:09:21.051424] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-07. [2025-10-20T17:09:21.710521] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T17:09:21.825859] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:09:21.851340] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-05. [2025-10-20T17:09:22.651371] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-06. [2025-10-20T17:09:23.452671] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-08. [2025-10-20T17:09:23.455291] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:09:23.496491] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:09:24.035425] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:09:24.251380] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-07. [2025-10-20T17:09:25.051358] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-04. [2025-10-20T17:09:25.324418] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:09:25.851419] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-08. [2025-10-20T17:09:26.651449] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-06. [2025-10-20T17:09:27.149450] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:09:27.184771] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T17:09:27.451379] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-09. [2025-10-20T17:09:28.062918] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:09:28.136352] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:09:28.251415] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-05. [2025-10-20T17:09:29.051442] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-07. [2025-10-20T17:09:29.851367] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-09. [2025-10-20T17:09:29.982230] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T17:09:30.651370] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-06. [2025-10-20T17:09:30.895728] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:09:31.026752] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:09:31.451386] [INFO ] [REMOTE ] ALG: Fetching primary data for 2025-10. [2025-10-20T17:09:32.000268] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:09:32.204903] [INFO ] [REMOTE ] ALG: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T17:09:32.206577] [INFO ] [REMOTE ] ALG: Full fetch complete (15295 records). [2025-10-20T17:09:32.206686] [DEBUG] [VALIDATOR ] ALG: Validating primary data. [2025-10-20T17:09:32.260331] [INFO ] [REMOTE ] ALEX: Fetching primary data for 2025-10. [2025-10-20T17:09:32.577437] [DEBUG] [VALIDATOR ] ALG: All expected trading days are present. [2025-10-20T17:09:32.577518] [DEBUG] [VALIDATOR ] ALG: Primary data validation passed. [2025-10-20T17:09:32.577777] [INFO ] [MAIN ] ALG: Fetched primary data is valid. [2025-10-20T17:09:32.577838] [DEBUG] [LOCAL ] ALG: Saving primary data. [2025-10-20T17:09:32.578325] [DEBUG] [LOCAL ] ALG: Trimmed 15295 records to 9748 (keeping 200 days of data) [2025-10-20T17:09:32.639854] [DEBUG] [LOCAL ] ALG: Primary data saved (9748 records). [2025-10-20T17:09:32.640004] [INFO ] [MAIN ] ALG: Full history saved. [2025-10-20T17:09:32.640082] [INFO ] [MAIN ] Update Progress: 82/2367 symbols completed. ALG took 22.2s. Estimated remaining: 4h 23m 54s. Rate: 8.7 symbols/min [2025-10-20T17:09:32.815027] [INFO ] [MAIN ] ALL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:32.865990] [DEBUG] [LOCAL ] ALL: Loaded 50503 existing records from primary storage. [2025-10-20T17:09:32.866351] [INFO ] [MAIN ] ALL: Loaded 50503 existing records. [2025-10-20T17:09:33.060516] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-07. [2025-10-20T17:09:33.066253] [WARN ] [MAIN ] ALL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:33.067518] [INFO ] [MAIN ] ALL: Wiping existing primary data before full refresh. [2025-10-20T17:09:33.067570] [INFO ] [LOCAL ] ALL: Starting deletion of primary data. [2025-10-20T17:09:33.067940] [DEBUG] [LOCAL ] ALL: Deleted primary data file. [2025-10-20T17:09:33.068000] [INFO ] [MAIN ] ALL: Starting full primary refresh. [2025-10-20T17:09:33.432971] [INFO ] [REMOTE ] ALL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:33.627098] [INFO ] [REMOTE ] ALEX: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:09:33.632350] [INFO ] [REMOTE ] ALEX: Full fetch complete (44539 records). [2025-10-20T17:09:33.632527] [DEBUG] [VALIDATOR ] ALEX: Validating primary data. [2025-10-20T17:09:33.856784] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-08. [2025-10-20T17:09:34.063897] [DEBUG] [VALIDATOR ] ALEX: All expected trading days are present. [2025-10-20T17:09:34.063992] [DEBUG] [VALIDATOR ] ALEX: Primary data validation passed. [2025-10-20T17:09:34.064532] [INFO ] [MAIN ] ALEX: Fetched primary data is valid. [2025-10-20T17:09:34.064573] [DEBUG] [LOCAL ] ALEX: Saving primary data. [2025-10-20T17:09:34.065745] [DEBUG] [LOCAL ] ALEX: Trimmed 44539 records to 31005 (keeping 200 days of data) [2025-10-20T17:09:34.265026] [DEBUG] [LOCAL ] ALEX: Primary data saved (31005 records). [2025-10-20T17:09:34.265183] [INFO ] [MAIN ] ALEX: Full history saved. [2025-10-20T17:09:34.265268] [INFO ] [MAIN ] Update Progress: 83/2367 symbols completed. ALEX took 23.8s. Estimated remaining: 4h 21m 21s. Rate: 8.7 symbols/min [2025-10-20T17:09:34.457498] [INFO ] [MAIN ] ALLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:34.504281] [DEBUG] [LOCAL ] ALLE: Loaded 44772 existing records from primary storage. [2025-10-20T17:09:34.504724] [INFO ] [MAIN ] ALLE: Loaded 44772 existing records. [2025-10-20T17:09:34.667739] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-01. [2025-10-20T17:09:34.706562] [WARN ] [MAIN ] ALLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:34.708058] [INFO ] [MAIN ] ALLE: Wiping existing primary data before full refresh. [2025-10-20T17:09:34.708181] [INFO ] [LOCAL ] ALLE: Starting deletion of primary data. [2025-10-20T17:09:34.708807] [DEBUG] [LOCAL ] ALLE: Deleted primary data file. [2025-10-20T17:09:34.708891] [INFO ] [MAIN ] ALLE: Starting full primary refresh. [2025-10-20T17:09:34.989813] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:09:35.122499] [INFO ] [REMOTE ] ALLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:35.451375] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-01. [2025-10-20T17:09:35.560257] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:09:36.251422] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-09. [2025-10-20T17:09:36.744994] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T17:09:36.814265] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:09:37.051396] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-02. [2025-10-20T17:09:37.851384] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-02. [2025-10-20T17:09:38.143879] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T17:09:38.651378] [INFO ] [REMOTE ] ALK: Fetching primary data for 2025-10. [2025-10-20T17:09:38.742680] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:09:39.451402] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-08. [2025-10-20T17:09:39.500142] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:09:40.251342] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-03. [2025-10-20T17:09:40.620200] [INFO ] [REMOTE ] ALK: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T17:09:40.628062] [INFO ] [REMOTE ] ALK: Full fetch complete (77025 records). [2025-10-20T17:09:40.628202] [DEBUG] [VALIDATOR ] ALK: Validating primary data. [2025-10-20T17:09:41.036173] [DEBUG] [VALIDATOR ] ALK: All expected trading days are present. [2025-10-20T17:09:41.036274] [DEBUG] [VALIDATOR ] ALK: Primary data validation passed. [2025-10-20T17:09:41.037049] [INFO ] [MAIN ] ALK: Fetched primary data is valid. [2025-10-20T17:09:41.037111] [DEBUG] [LOCAL ] ALK: Saving primary data. [2025-10-20T17:09:41.038587] [DEBUG] [LOCAL ] ALK: Trimmed 77025 records to 53085 (keeping 200 days of data) [2025-10-20T17:09:41.088076] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-03. [2025-10-20T17:09:41.344343] [DEBUG] [LOCAL ] ALK: Primary data saved (53085 records). [2025-10-20T17:09:41.344496] [INFO ] [MAIN ] ALK: Full history saved. [2025-10-20T17:09:41.344580] [INFO ] [MAIN ] Update Progress: 84/2367 symbols completed. ALK took 28.3s. Estimated remaining: 4h 21m 20s. Rate: 8.7 symbols/min [2025-10-20T17:09:41.532770] [INFO ] [MAIN ] ALLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:41.563517] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:09:41.619192] [DEBUG] [LOCAL ] ALLY: Loaded 54000 existing records from primary storage. [2025-10-20T17:09:41.619539] [INFO ] [MAIN ] ALLY: Loaded 54000 existing records. [2025-10-20T17:09:41.796280] [WARN ] [MAIN ] ALLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:41.796526] [INFO ] [MAIN ] ALLY: Wiping existing primary data before full refresh. [2025-10-20T17:09:41.796573] [INFO ] [LOCAL ] ALLY: Starting deletion of primary data. [2025-10-20T17:09:41.797031] [DEBUG] [LOCAL ] ALLY: Deleted primary data file. [2025-10-20T17:09:41.797087] [INFO ] [MAIN ] ALLY: Starting full primary refresh. [2025-10-20T17:09:41.879102] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-09. [2025-10-20T17:09:42.174263] [INFO ] [REMOTE ] ALLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:42.210663] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:09:42.651373] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-04. [2025-10-20T17:09:42.988512] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:09:43.451363] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-04. [2025-10-20T17:09:43.755255] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T17:09:44.251385] [INFO ] [REMOTE ] ALIT: Fetching primary data for 2025-10. [2025-10-20T17:09:44.426928] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:09:45.051382] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-05. [2025-10-20T17:09:45.230893] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:09:45.742543] [INFO ] [REMOTE ] ALIT: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:09:45.750166] [INFO ] [REMOTE ] ALIT: Full fetch complete (76618 records). [2025-10-20T17:09:45.750278] [DEBUG] [VALIDATOR ] ALIT: Validating primary data. [2025-10-20T17:09:45.856544] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-01. [2025-10-20T17:09:46.135169] [DEBUG] [VALIDATOR ] ALIT: All expected trading days are present. [2025-10-20T17:09:46.135249] [DEBUG] [VALIDATOR ] ALIT: Primary data validation passed. [2025-10-20T17:09:46.135988] [INFO ] [MAIN ] ALIT: Fetched primary data is valid. [2025-10-20T17:09:46.136053] [DEBUG] [LOCAL ] ALIT: Saving primary data. [2025-10-20T17:09:46.137416] [DEBUG] [LOCAL ] ALIT: Trimmed 76618 records to 52790 (keeping 200 days of data) [2025-10-20T17:09:46.435393] [DEBUG] [LOCAL ] ALIT: Primary data saved (52790 records). [2025-10-20T17:09:46.435542] [INFO ] [MAIN ] ALIT: Full history saved. [2025-10-20T17:09:46.436093] [INFO ] [MAIN ] Update Progress: 85/2367 symbols completed. ALIT took 36.0s. Estimated remaining: 4h 20m 26s. Rate: 8.8 symbols/min [2025-10-20T17:09:46.607056] [INFO ] [MAIN ] ALSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:09:46.646097] [DEBUG] [LOCAL ] ALSN: Loaded 44062 existing records from primary storage. [2025-10-20T17:09:46.646389] [INFO ] [MAIN ] ALSN: Loaded 44062 existing records. [2025-10-20T17:09:46.676938] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-05. [2025-10-20T17:09:46.828598] [WARN ] [MAIN ] ALSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:09:46.828936] [INFO ] [MAIN ] ALSN: Wiping existing primary data before full refresh. [2025-10-20T17:09:46.828981] [INFO ] [LOCAL ] ALSN: Starting deletion of primary data. [2025-10-20T17:09:46.829232] [DEBUG] [LOCAL ] ALSN: Deleted primary data file. [2025-10-20T17:09:46.829270] [INFO ] [MAIN ] ALSN: Starting full primary refresh. [2025-10-20T17:09:47.220069] [INFO ] [REMOTE ] ALSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:09:47.239515] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-10-20T17:09:47.451362] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-06. [2025-10-20T17:09:47.653103] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:09:48.251437] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-02. [2025-10-20T17:09:48.304161] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T17:09:48.744993] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:09:49.051379] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-06. [2025-10-20T17:09:49.851422] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-01. [2025-10-20T17:09:50.042772] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:09:50.244766] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:09:50.651371] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-03. [2025-10-20T17:09:51.038900] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:09:51.451349] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-07. [2025-10-20T17:09:52.144907] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:09:52.251364] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-04. [2025-10-20T17:09:53.051369] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-02. [2025-10-20T17:09:53.354943] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T17:09:53.665275] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T17:09:53.851395] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-08. [2025-10-20T17:09:54.507635] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:09:54.651363] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-03. [2025-10-20T17:09:55.237488] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:09:55.451383] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-07. [2025-10-20T17:09:55.865771] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:09:56.251377] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-05. [2025-10-20T17:09:57.051345] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-09. [2025-10-20T17:09:57.160679] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:09:57.715083] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:09:57.851401] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-06. [2025-10-20T17:09:58.651345] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-08. [2025-10-20T17:09:58.721740] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T17:09:59.451384] [INFO ] [REMOTE ] ALL: Fetching primary data for 2025-10. [2025-10-20T17:09:59.633255] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:10:00.251453] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-04. [2025-10-20T17:10:00.394635] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:10:01.051381] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-09. [2025-10-20T17:10:01.406509] [INFO ] [REMOTE ] ALL: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:10:01.414282] [INFO ] [REMOTE ] ALL: Full fetch complete (72555 records). [2025-10-20T17:10:01.414476] [DEBUG] [VALIDATOR ] ALL: Validating primary data. [2025-10-20T17:10:01.852254] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-07. [2025-10-20T17:10:01.877410] [DEBUG] [VALIDATOR ] ALL: All expected trading days are present. [2025-10-20T17:10:01.877531] [DEBUG] [VALIDATOR ] ALL: Primary data validation passed. [2025-10-20T17:10:01.878917] [INFO ] [MAIN ] ALL: Fetched primary data is valid. [2025-10-20T17:10:01.879136] [DEBUG] [LOCAL ] ALL: Saving primary data. [2025-10-20T17:10:01.881174] [DEBUG] [LOCAL ] ALL: Trimmed 72555 records to 49721 (keeping 200 days of data) [2025-10-20T17:10:02.167271] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T17:10:02.256074] [DEBUG] [LOCAL ] ALL: Primary data saved (49721 records). [2025-10-20T17:10:02.256231] [INFO ] [MAIN ] ALL: Full history saved. [2025-10-20T17:10:02.257286] [INFO ] [MAIN ] Update Progress: 86/2367 symbols completed. ALL took 29.6s. Estimated remaining: 4h 24m 17s. Rate: 8.6 symbols/min [2025-10-20T17:10:02.434345] [INFO ] [MAIN ] ALTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:02.457984] [DEBUG] [LOCAL ] ALTG: Loaded 23632 existing records from primary storage. [2025-10-20T17:10:02.458237] [INFO ] [MAIN ] ALTG: Loaded 23632 existing records. [2025-10-20T17:10:02.641067] [WARN ] [MAIN ] ALTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:02.641350] [INFO ] [MAIN ] ALTG: Wiping existing primary data before full refresh. [2025-10-20T17:10:02.641405] [INFO ] [LOCAL ] ALTG: Starting deletion of primary data. [2025-10-20T17:10:02.641674] [DEBUG] [LOCAL ] ALTG: Deleted primary data file. [2025-10-20T17:10:02.641744] [INFO ] [MAIN ] ALTG: Starting full primary refresh. [2025-10-20T17:10:02.673866] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-05. [2025-10-20T17:10:02.960676] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T17:10:03.110138] [INFO ] [REMOTE ] ALTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:03.451409] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-01. [2025-10-20T17:10:03.506056] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:10:04.215912] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T17:10:04.251370] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-06. [2025-10-20T17:10:04.656348] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:10:05.051425] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-02. [2025-10-20T17:10:05.605939] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:10:05.670833] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:10:05.851434] [INFO ] [REMOTE ] ALLE: Fetching primary data for 2025-10. [2025-10-20T17:10:06.651393] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-07. [2025-10-20T17:10:07.451365] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-03. [2025-10-20T17:10:07.677586] [INFO ] [REMOTE ] ALLE: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:10:07.684856] [INFO ] [REMOTE ] ALLE: Full fetch complete (64858 records). [2025-10-20T17:10:07.685046] [DEBUG] [VALIDATOR ] ALLE: Validating primary data. [2025-10-20T17:10:08.101437] [DEBUG] [VALIDATOR ] ALLE: All expected trading days are present. [2025-10-20T17:10:08.101532] [DEBUG] [VALIDATOR ] ALLE: Primary data validation passed. [2025-10-20T17:10:08.102220] [INFO ] [MAIN ] ALLE: Fetched primary data is valid. [2025-10-20T17:10:08.102285] [DEBUG] [LOCAL ] ALLE: Saving primary data. [2025-10-20T17:10:08.103758] [DEBUG] [LOCAL ] ALLE: Trimmed 64858 records to 44026 (keeping 200 days of data) [2025-10-20T17:10:08.284034] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-08. [2025-10-20T17:10:08.389126] [DEBUG] [LOCAL ] ALLE: Primary data saved (44026 records). [2025-10-20T17:10:08.389694] [INFO ] [MAIN ] ALLE: Full history saved. [2025-10-20T17:10:08.390795] [INFO ] [MAIN ] Update Progress: 87/2367 symbols completed. ALLE took 34.1s. Estimated remaining: 4h 23m 49s. Rate: 8.6 symbols/min [2025-10-20T17:10:08.441463] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:10:08.629081] [INFO ] [MAIN ] ALUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:08.630607] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:10:08.648423] [DEBUG] [LOCAL ] ALUR: Loaded 9640 existing records from primary storage. [2025-10-20T17:10:08.651598] [INFO ] [MAIN ] ALUR: Loaded 9640 existing records. [2025-10-20T17:10:08.841449] [WARN ] [MAIN ] ALUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:08.841670] [INFO ] [MAIN ] ALUR: Wiping existing primary data before full refresh. [2025-10-20T17:10:08.841729] [INFO ] [LOCAL ] ALUR: Starting deletion of primary data. [2025-10-20T17:10:08.841872] [DEBUG] [LOCAL ] ALUR: Deleted primary data file. [2025-10-20T17:10:08.841926] [INFO ] [MAIN ] ALUR: Starting full primary refresh. [2025-10-20T17:10:09.060442] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-08. [2025-10-20T17:10:09.203789] [INFO ] [REMOTE ] ALUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:09.851353] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-04. [2025-10-20T17:10:10.399932] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T17:10:10.651497] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-09. [2025-10-20T17:10:10.773734] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:10:11.037188] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:10:11.451398] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-05. [2025-10-20T17:10:12.093454] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:10:12.180635] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T17:10:12.251389] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-01. [2025-10-20T17:10:13.051395] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-09. [2025-10-20T17:10:13.753741] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:10:13.851361] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-02. [2025-10-20T17:10:14.651362] [INFO ] [REMOTE ] ALLY: Fetching primary data for 2025-10. [2025-10-20T17:10:14.762817] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:10:14.823439] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:10:15.451448] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-06. [2025-10-20T17:10:16.251380] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-03. [2025-10-20T17:10:16.349241] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T17:10:16.729508] [INFO ] [REMOTE ] ALLY: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T17:10:16.737238] [INFO ] [REMOTE ] ALLY: Full fetch complete (77235 records). [2025-10-20T17:10:16.737377] [DEBUG] [VALIDATOR ] ALLY: Validating primary data. [2025-10-20T17:10:17.056565] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-07. [2025-10-20T17:10:17.127937] [DEBUG] [VALIDATOR ] ALLY: All expected trading days are present. [2025-10-20T17:10:17.128026] [DEBUG] [VALIDATOR ] ALLY: Primary data validation passed. [2025-10-20T17:10:17.128868] [INFO ] [MAIN ] ALLY: Fetched primary data is valid. [2025-10-20T17:10:17.128969] [DEBUG] [LOCAL ] ALLY: Saving primary data. [2025-10-20T17:10:17.140577] [DEBUG] [LOCAL ] ALLY: Trimmed 77235 records to 53224 (keeping 200 days of data) [2025-10-20T17:10:17.141484] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:10:17.469480] [DEBUG] [LOCAL ] ALLY: Primary data saved (53224 records). [2025-10-20T17:10:17.469688] [INFO ] [MAIN ] ALLY: Full history saved. [2025-10-20T17:10:17.469817] [INFO ] [MAIN ] Update Progress: 88/2367 symbols completed. ALLY took 36.1s. Estimated remaining: 4h 24m 37s. Rate: 8.6 symbols/min [2025-10-20T17:10:17.643308] [INFO ] [MAIN ] ALV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:17.684441] [DEBUG] [LOCAL ] ALV: Loaded 42843 existing records from primary storage. [2025-10-20T17:10:17.684761] [INFO ] [MAIN ] ALV: Loaded 42843 existing records. [2025-10-20T17:10:17.851602] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-04. [2025-10-20T17:10:17.870855] [WARN ] [MAIN ] ALV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:17.871838] [INFO ] [MAIN ] ALV: Wiping existing primary data before full refresh. [2025-10-20T17:10:17.871927] [INFO ] [LOCAL ] ALV: Starting deletion of primary data. [2025-10-20T17:10:17.872135] [DEBUG] [LOCAL ] ALV: Deleted primary data file. [2025-10-20T17:10:17.872196] [INFO ] [MAIN ] ALV: Starting full primary refresh. [2025-10-20T17:10:18.236321] [INFO ] [REMOTE ] ALV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:18.450188] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:10:18.520860] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T17:10:18.651357] [INFO ] [REMOTE ] ALSN: Fetching primary data for 2025-10. [2025-10-20T17:10:19.451476] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-08. [2025-10-20T17:10:20.177128] [INFO ] [REMOTE ] ALSN: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:10:20.183575] [INFO ] [REMOTE ] ALSN: Full fetch complete (62570 records). [2025-10-20T17:10:20.183706] [DEBUG] [VALIDATOR ] ALSN: Validating primary data. [2025-10-20T17:10:20.258033] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-05. [2025-10-20T17:10:20.569506] [DEBUG] [VALIDATOR ] ALSN: All expected trading days are present. [2025-10-20T17:10:20.569584] [DEBUG] [VALIDATOR ] ALSN: Primary data validation passed. [2025-10-20T17:10:20.570222] [INFO ] [MAIN ] ALSN: Fetched primary data is valid. [2025-10-20T17:10:20.570283] [DEBUG] [LOCAL ] ALSN: Saving primary data. [2025-10-20T17:10:20.571459] [DEBUG] [LOCAL ] ALSN: Trimmed 62570 records to 43332 (keeping 200 days of data) [2025-10-20T17:10:20.684768] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:10:20.855911] [DEBUG] [LOCAL ] ALSN: Primary data saved (43332 records). [2025-10-20T17:10:20.856045] [INFO ] [MAIN ] ALSN: Full history saved. [2025-10-20T17:10:20.856698] [INFO ] [MAIN ] Update Progress: 89/2367 symbols completed. ALSN took 34.4s. Estimated remaining: 4h 22m 58s. Rate: 8.7 symbols/min [2025-10-20T17:10:21.031621] [INFO ] [MAIN ] ALX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:21.040675] [DEBUG] [LOCAL ] ALX: Loaded 7141 existing records from primary storage. [2025-10-20T17:10:21.040911] [INFO ] [MAIN ] ALX: Loaded 7141 existing records. [2025-10-20T17:10:21.062893] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-09. [2025-10-20T17:10:21.232479] [WARN ] [MAIN ] ALX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:21.232739] [INFO ] [MAIN ] ALX: Wiping existing primary data before full refresh. [2025-10-20T17:10:21.232849] [INFO ] [LOCAL ] ALX: Starting deletion of primary data. [2025-10-20T17:10:21.233034] [DEBUG] [LOCAL ] ALX: Deleted primary data file. [2025-10-20T17:10:21.233084] [INFO ] [MAIN ] ALX: Starting full primary refresh. [2025-10-20T17:10:21.552889] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:10:21.618895] [INFO ] [REMOTE ] ALX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:21.841679] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:10:21.851332] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-06. [2025-10-20T17:10:22.438810] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T17:10:22.651387] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-07. [2025-10-20T17:10:23.451386] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-01. [2025-10-20T17:10:23.627156] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:10:24.088531] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T17:10:24.251431] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-02. [2025-10-20T17:10:24.864172] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:10:25.051377] [INFO ] [REMOTE ] ALTG: Fetching primary data for 2025-10. [2025-10-20T17:10:25.851472] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-01. [2025-10-20T17:10:26.191156] [INFO ] [REMOTE ] ALTG: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T17:10:26.195215] [INFO ] [REMOTE ] ALTG: Full fetch complete (33412 records). [2025-10-20T17:10:26.195321] [DEBUG] [VALIDATOR ] ALTG: Validating primary data. [2025-10-20T17:10:26.625464] [DEBUG] [VALIDATOR ] ALTG: All expected trading days are present. [2025-10-20T17:10:26.625565] [DEBUG] [VALIDATOR ] ALTG: Primary data validation passed. [2025-10-20T17:10:26.626060] [INFO ] [MAIN ] ALTG: Fetched primary data is valid. [2025-10-20T17:10:26.626145] [DEBUG] [LOCAL ] ALTG: Saving primary data. [2025-10-20T17:10:26.627009] [DEBUG] [LOCAL ] ALTG: Trimmed 33412 records to 23275 (keeping 200 days of data) [2025-10-20T17:10:26.674633] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-03. [2025-10-20T17:10:26.764360] [DEBUG] [LOCAL ] ALTG: Primary data saved (23275 records). [2025-10-20T17:10:26.764501] [INFO ] [MAIN ] ALTG: Full history saved. [2025-10-20T17:10:26.764585] [INFO ] [MAIN ] Update Progress: 90/2367 symbols completed. ALTG took 24.5s. Estimated remaining: 4h 22m 26s. Rate: 8.7 symbols/min [2025-10-20T17:10:26.964522] [INFO ] [MAIN ] AM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:27.016318] [DEBUG] [LOCAL ] AM: Loaded 52670 existing records from primary storage. [2025-10-20T17:10:27.016691] [INFO ] [MAIN ] AM: Loaded 52670 existing records. [2025-10-20T17:10:27.199056] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T17:10:27.222711] [WARN ] [MAIN ] AM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:27.223041] [INFO ] [MAIN ] AM: Wiping existing primary data before full refresh. [2025-10-20T17:10:27.223228] [INFO ] [LOCAL ] AM: Starting deletion of primary data. [2025-10-20T17:10:27.223581] [DEBUG] [LOCAL ] AM: Deleted primary data file. [2025-10-20T17:10:27.223763] [INFO ] [MAIN ] AM: Starting full primary refresh. [2025-10-20T17:10:27.407432] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:10:27.457716] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-04. [2025-10-20T17:10:27.623198] [INFO ] [REMOTE ] AM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:28.135963] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:10:28.251406] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-08. [2025-10-20T17:10:29.051454] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-02. [2025-10-20T17:10:29.090410] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:10:29.851367] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-09. [2025-10-20T17:10:30.233325] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T17:10:30.296330] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T17:10:30.651397] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-05. [2025-10-20T17:10:31.211840] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T17:10:31.451358] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-01. [2025-10-20T17:10:32.251365] [INFO ] [REMOTE ] ALUR: Fetching primary data for 2025-10. [2025-10-20T17:10:33.013780] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:10:33.051331] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-02. [2025-10-20T17:10:33.083044] [INFO ] [REMOTE ] ALUR: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:10:33.084611] [INFO ] [REMOTE ] ALUR: Full fetch complete (18629 records). [2025-10-20T17:10:33.084710] [DEBUG] [VALIDATOR ] ALUR: Validating primary data. [2025-10-20T17:10:33.436107] [DEBUG] [VALIDATOR ] ALUR: All expected trading days are present. [2025-10-20T17:10:33.436185] [DEBUG] [VALIDATOR ] ALUR: Primary data validation passed. [2025-10-20T17:10:33.436459] [INFO ] [MAIN ] ALUR: Fetched primary data is valid. [2025-10-20T17:10:33.436502] [DEBUG] [LOCAL ] ALUR: Saving primary data. [2025-10-20T17:10:33.437036] [DEBUG] [LOCAL ] ALUR: Trimmed 18629 records to 9345 (keeping 200 days of data) [2025-10-20T17:10:33.487613] [DEBUG] [LOCAL ] ALUR: Primary data saved (9345 records). [2025-10-20T17:10:33.487779] [INFO ] [MAIN ] ALUR: Full history saved. [2025-10-20T17:10:33.487861] [INFO ] [MAIN ] Update Progress: 91/2367 symbols completed. ALUR took 25.1s. Estimated remaining: 4h 22m 14s. Rate: 8.7 symbols/min [2025-10-20T17:10:33.662682] [INFO ] [MAIN ] AMBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:33.696190] [DEBUG] [LOCAL ] AMBC: Loaded 36767 existing records from primary storage. [2025-10-20T17:10:33.696563] [INFO ] [MAIN ] AMBC: Loaded 36767 existing records. [2025-10-20T17:10:33.856552] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-03. [2025-10-20T17:10:33.888542] [WARN ] [MAIN ] AMBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:33.889612] [INFO ] [MAIN ] AMBC: Wiping existing primary data before full refresh. [2025-10-20T17:10:33.889719] [INFO ] [LOCAL ] AMBC: Starting deletion of primary data. [2025-10-20T17:10:33.890025] [DEBUG] [LOCAL ] AMBC: Deleted primary data file. [2025-10-20T17:10:33.890096] [INFO ] [MAIN ] AMBC: Starting full primary refresh. [2025-10-20T17:10:34.239351] [INFO ] [REMOTE ] AMBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:34.389807] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:10:34.651401] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-03. [2025-10-20T17:10:35.377626] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:10:35.451422] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-06. [2025-10-20T17:10:36.091704] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:10:36.251453] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-04. [2025-10-20T17:10:36.390616] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T17:10:37.051414] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-04. [2025-10-20T17:10:37.851383] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-07. [2025-10-20T17:10:38.102264] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:10:38.382893] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T17:10:38.651357] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-08. [2025-10-20T17:10:38.839191] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:10:39.104731] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T17:10:39.451373] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-01. [2025-10-20T17:10:40.251373] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-05. [2025-10-20T17:10:40.397109] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T17:10:41.051408] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-02. [2025-10-20T17:10:41.851413] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-09. [2025-10-20T17:10:42.024540] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:10:42.334785] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T17:10:42.379531] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:10:42.651374] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-03. [2025-10-20T17:10:43.451399] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-06. [2025-10-20T17:10:44.251913] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-05. [2025-10-20T17:10:44.258699] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:10:44.908057] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:10:45.051365] [INFO ] [REMOTE ] ALX: Fetching primary data for 2025-10. [2025-10-20T17:10:45.762125] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:10:45.851354] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-04. [2025-10-20T17:10:46.051318] [INFO ] [REMOTE ] ALX: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:10:46.054372] [INFO ] [REMOTE ] ALX: Full fetch complete (7985 records). [2025-10-20T17:10:46.054476] [DEBUG] [VALIDATOR ] ALX: Validating primary data. [2025-10-20T17:10:46.435488] [DEBUG] [VALIDATOR ] ALX: All expected trading days are present. [2025-10-20T17:10:46.435577] [DEBUG] [VALIDATOR ] ALX: Primary data validation passed. [2025-10-20T17:10:46.435834] [INFO ] [MAIN ] ALX: Fetched primary data is valid. [2025-10-20T17:10:46.435895] [DEBUG] [LOCAL ] ALX: Saving primary data. [2025-10-20T17:10:46.436381] [DEBUG] [LOCAL ] ALX: Trimmed 7985 records to 7183 (keeping 200 days of data) [2025-10-20T17:10:46.482456] [DEBUG] [LOCAL ] ALX: Primary data saved (7183 records). [2025-10-20T17:10:46.482579] [INFO ] [MAIN ] ALX: Full history saved. [2025-10-20T17:10:46.482676] [INFO ] [MAIN ] Update Progress: 92/2367 symbols completed. ALX took 25.6s. Estimated remaining: 4h 24m 37s. Rate: 8.6 symbols/min [2025-10-20T17:10:46.651110] [INFO ] [MAIN ] AMBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:46.651404] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-07. [2025-10-20T17:10:46.692729] [DEBUG] [LOCAL ] AMBP: Loaded 45132 existing records from primary storage. [2025-10-20T17:10:46.693062] [INFO ] [MAIN ] AMBP: Loaded 45132 existing records. [2025-10-20T17:10:46.877562] [WARN ] [MAIN ] AMBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:46.877782] [INFO ] [MAIN ] AMBP: Wiping existing primary data before full refresh. [2025-10-20T17:10:46.877831] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-10-20T17:10:46.878086] [DEBUG] [LOCAL ] AMBP: Deleted primary data file. [2025-10-20T17:10:46.878142] [INFO ] [MAIN ] AMBP: Starting full primary refresh. [2025-10-20T17:10:47.244636] [INFO ] [REMOTE ] AMBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:47.321703] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:10:47.451451] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-05. [2025-10-20T17:10:48.251430] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-06. [2025-10-20T17:10:48.472403] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T17:10:48.705204] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:10:49.051359] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-06. [2025-10-20T17:10:49.645174] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T17:10:49.851421] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-01. [2025-10-20T17:10:50.339308] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:10:50.651408] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-08. [2025-10-20T17:10:51.231591] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T17:10:51.451377] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-07. [2025-10-20T17:10:52.202874] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:10:52.251360] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-09. [2025-10-20T17:10:53.051380] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-07. [2025-10-20T17:10:53.098558] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:10:53.687541] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:10:53.851411] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-08. [2025-10-20T17:10:54.136397] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:10:54.651377] [INFO ] [REMOTE ] AM: Fetching primary data for 2025-10. [2025-10-20T17:10:55.450314] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T17:10:55.458518] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-02. [2025-10-20T17:10:56.251372] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-09. [2025-10-20T17:10:56.570455] [INFO ] [REMOTE ] AM: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:10:56.578429] [INFO ] [REMOTE ] AM: Full fetch complete (75691 records). [2025-10-20T17:10:56.578617] [DEBUG] [VALIDATOR ] AM: Validating primary data. [2025-10-20T17:10:56.944313] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:10:57.064577] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-03. [2025-10-20T17:10:57.068798] [DEBUG] [VALIDATOR ] AM: All expected trading days are present. [2025-10-20T17:10:57.069827] [DEBUG] [VALIDATOR ] AM: Primary data validation passed. [2025-10-20T17:10:57.070696] [INFO ] [MAIN ] AM: Fetched primary data is valid. [2025-10-20T17:10:57.070763] [DEBUG] [LOCAL ] AM: Saving primary data. [2025-10-20T17:10:57.072423] [DEBUG] [LOCAL ] AM: Trimmed 75691 records to 51887 (keeping 200 days of data) [2025-10-20T17:10:57.395525] [DEBUG] [LOCAL ] AM: Primary data saved (51887 records). [2025-10-20T17:10:57.395756] [INFO ] [MAIN ] AM: Full history saved. [2025-10-20T17:10:57.395879] [INFO ] [MAIN ] Update Progress: 93/2367 symbols completed. AM took 30.6s. Estimated remaining: 4h 26m 7s. Rate: 8.5 symbols/min [2025-10-20T17:10:57.534800] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T17:10:57.612932] [INFO ] [MAIN ] AMBQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:57.613112] [DEBUG] [LOCAL ] AMBQ: No existing data file found in primary storage. [2025-10-20T17:10:57.613149] [WARN ] [MAIN ] AMBQ: No existing data found, performing full refresh instead. [2025-10-20T17:10:57.613258] [INFO ] [MAIN ] AMBQ: Wiping existing primary data before full refresh. [2025-10-20T17:10:57.613300] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-10-20T17:10:57.613340] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-10-20T17:10:57.613371] [INFO ] [MAIN ] AMBQ: Starting full primary refresh. [2025-10-20T17:10:57.858150] [INFO ] [REMOTE ] ALV: Fetching primary data for 2025-10. [2025-10-20T17:10:57.974182] [INFO ] [REMOTE ] AMBQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:58.651365] [INFO ] [REMOTE ] AMBQ: Fetching primary data for 2025-01. [2025-10-20T17:10:58.741563] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:10:58.849984] [DEBUG] [REMOTE ] AMBQ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:10:58.850289] [WARN ] [REMOTE ] AMBQ: No data returned for month 2025-01 [2025-10-20T17:10:58.850372] [FATAL] [REMOTE ] AMBQ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:10:58.850420] [FATAL] [MAIN ] AMBQ: Failed to fetch primary data. [2025-10-20T17:10:58.850478] [WARN ] [MAIN ] AMBQ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:10:58.850552] [INFO ] [LOCAL ] AMBQ: Starting deletion of primary data. [2025-10-20T17:10:58.850615] [DEBUG] [LOCAL ] AMBQ: No primary data file found to delete. [2025-10-20T17:10:58.850692] [INFO ] [LOCAL ] AMBQ: Starting deletion of derivative data. [2025-10-20T17:10:58.850794] [DEBUG] [LOCAL ] AMBQ: No derivative data file found to delete. [2025-10-20T17:10:58.850881] [INFO ] [MAIN ] Update Progress: 94/2367 symbols completed. AMBQ took 1.5s. Estimated remaining: 4h 23m 45s. Rate: 8.6 symbols/min [2025-10-20T17:10:59.026887] [INFO ] [MAIN ] AMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:10:59.080887] [DEBUG] [LOCAL ] AMC: Loaded 54542 existing records from primary storage. [2025-10-20T17:10:59.081260] [INFO ] [MAIN ] AMC: Loaded 54542 existing records. [2025-10-20T17:10:59.284087] [WARN ] [MAIN ] AMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:10:59.284382] [INFO ] [MAIN ] AMC: Wiping existing primary data before full refresh. [2025-10-20T17:10:59.284435] [INFO ] [LOCAL ] AMC: Starting deletion of primary data. [2025-10-20T17:10:59.284612] [DEBUG] [LOCAL ] AMC: Deleted primary data file. [2025-10-20T17:10:59.284689] [INFO ] [MAIN ] AMC: Starting full primary refresh. [2025-10-20T17:10:59.458470] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-08. [2025-10-20T17:10:59.652245] [INFO ] [REMOTE ] ALV: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T17:10:59.656514] [INFO ] [REMOTE ] AMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:10:59.663012] [INFO ] [REMOTE ] ALV: Full fetch complete (62304 records). [2025-10-20T17:10:59.663104] [DEBUG] [VALIDATOR ] ALV: Validating primary data. [2025-10-20T17:11:00.052164] [DEBUG] [VALIDATOR ] ALV: All expected trading days are present. [2025-10-20T17:11:00.052254] [DEBUG] [VALIDATOR ] ALV: Primary data validation passed. [2025-10-20T17:11:00.053041] [INFO ] [MAIN ] ALV: Fetched primary data is valid. [2025-10-20T17:11:00.053116] [DEBUG] [LOCAL ] ALV: Saving primary data. [2025-10-20T17:11:00.054349] [DEBUG] [LOCAL ] ALV: Trimmed 62304 records to 42190 (keeping 200 days of data) [2025-10-20T17:11:00.265986] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-04. [2025-10-20T17:11:00.315679] [DEBUG] [LOCAL ] ALV: Primary data saved (42190 records). [2025-10-20T17:11:00.315803] [INFO ] [MAIN ] ALV: Full history saved. [2025-10-20T17:11:00.316592] [INFO ] [MAIN ] Update Progress: 95/2367 symbols completed. ALV took 42.8s. Estimated remaining: 4h 21m 27s. Rate: 8.7 symbols/min [2025-10-20T17:11:00.493938] [INFO ] [MAIN ] AMCR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:00.547386] [DEBUG] [LOCAL ] AMCR: Loaded 54449 existing records from primary storage. [2025-10-20T17:11:00.547797] [INFO ] [MAIN ] AMCR: Loaded 54449 existing records. [2025-10-20T17:11:00.737906] [WARN ] [MAIN ] AMCR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:00.738292] [INFO ] [MAIN ] AMCR: Wiping existing primary data before full refresh. [2025-10-20T17:11:00.738353] [INFO ] [LOCAL ] AMCR: Starting deletion of primary data. [2025-10-20T17:11:00.758912] [DEBUG] [LOCAL ] AMCR: Deleted primary data file. [2025-10-20T17:11:00.759055] [INFO ] [MAIN ] AMCR: Starting full primary refresh. [2025-10-20T17:11:00.831841] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:11:01.056559] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-01. [2025-10-20T17:11:01.141203] [INFO ] [REMOTE ] AMCR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:01.633490] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:11:01.851363] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-01. [2025-10-20T17:11:02.651373] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-05. [2025-10-20T17:11:03.451355] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-09. [2025-10-20T17:11:03.535539] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-01 (2.48 seconds). [2025-10-20T17:11:03.709972] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:11:04.251358] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-02. [2025-10-20T17:11:04.404446] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:11:04.835286] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:11:05.051365] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-02. [2025-10-20T17:11:05.650813] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:11:05.851382] [INFO ] [REMOTE ] AMBC: Fetching primary data for 2025-10. [2025-10-20T17:11:06.651372] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-06. [2025-10-20T17:11:06.956590] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:11:07.259048] [INFO ] [REMOTE ] AMBC: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:11:07.264837] [INFO ] [REMOTE ] AMBC: Full fetch complete (51907 records). [2025-10-20T17:11:07.265058] [DEBUG] [VALIDATOR ] AMBC: Validating primary data. [2025-10-20T17:11:07.456506] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-03. [2025-10-20T17:11:07.648712] [DEBUG] [VALIDATOR ] AMBC: All expected trading days are present. [2025-10-20T17:11:07.648813] [DEBUG] [VALIDATOR ] AMBC: Primary data validation passed. [2025-10-20T17:11:07.649447] [INFO ] [MAIN ] AMBC: Fetched primary data is valid. [2025-10-20T17:11:07.649527] [DEBUG] [LOCAL ] AMBC: Saving primary data. [2025-10-20T17:11:07.650796] [DEBUG] [LOCAL ] AMBC: Trimmed 51907 records to 36189 (keeping 200 days of data) [2025-10-20T17:11:07.789178] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:11:07.893128] [DEBUG] [LOCAL ] AMBC: Primary data saved (36189 records). [2025-10-20T17:11:07.893284] [INFO ] [MAIN ] AMBC: Full history saved. [2025-10-20T17:11:07.894134] [INFO ] [MAIN ] Update Progress: 96/2367 symbols completed. AMBC took 34.4s. Estimated remaining: 4h 21m 36s. Rate: 8.7 symbols/min [2025-10-20T17:11:08.066739] [INFO ] [MAIN ] AME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:08.116390] [DEBUG] [LOCAL ] AME: Loaded 49338 existing records from primary storage. [2025-10-20T17:11:08.116712] [INFO ] [MAIN ] AME: Loaded 49338 existing records. [2025-10-20T17:11:08.257226] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-07. [2025-10-20T17:11:08.304575] [WARN ] [MAIN ] AME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:08.304862] [INFO ] [MAIN ] AME: Wiping existing primary data before full refresh. [2025-10-20T17:11:08.304917] [INFO ] [LOCAL ] AME: Starting deletion of primary data. [2025-10-20T17:11:08.305143] [DEBUG] [LOCAL ] AME: Deleted primary data file. [2025-10-20T17:11:08.305200] [INFO ] [MAIN ] AME: Starting full primary refresh. [2025-10-20T17:11:08.705877] [INFO ] [REMOTE ] AME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:09.051391] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-03. [2025-10-20T17:11:09.686903] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T17:11:09.757925] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:11:09.851332] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-08. [2025-10-20T17:11:10.606056] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:11:10.651347] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-01. [2025-10-20T17:11:11.082574] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:11:11.451398] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-04. [2025-10-20T17:11:12.251362] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-09. [2025-10-20T17:11:12.421296] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T17:11:13.051396] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-02. [2025-10-20T17:11:13.596424] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:11:13.624887] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T17:11:13.851377] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-05. [2025-10-20T17:11:14.410092] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:11:14.651385] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-04. [2025-10-20T17:11:15.451460] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-03. [2025-10-20T17:11:16.251378] [INFO ] [REMOTE ] AMBP: Fetching primary data for 2025-10. [2025-10-20T17:11:16.338775] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-05 (2.49 seconds). [2025-10-20T17:11:16.600624] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:11:16.912162] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:11:17.051396] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-04. [2025-10-20T17:11:17.694277] [INFO ] [REMOTE ] AMBP: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T17:11:17.701389] [INFO ] [REMOTE ] AMBP: Full fetch complete (64868 records). [2025-10-20T17:11:17.701511] [DEBUG] [VALIDATOR ] AMBP: Validating primary data. [2025-10-20T17:11:17.702208] [FATAL] [VALIDATOR ] AMBP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:11:17.702291] [FATAL] [MAIN ] AMBP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:11:17.702334] [WARN ] [MAIN ] AMBP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:11:17.702403] [INFO ] [LOCAL ] AMBP: Starting deletion of primary data. [2025-10-20T17:11:17.702466] [DEBUG] [LOCAL ] AMBP: No primary data file found to delete. [2025-10-20T17:11:17.702503] [INFO ] [LOCAL ] AMBP: Starting deletion of derivative data. [2025-10-20T17:11:17.702722] [DEBUG] [LOCAL ] AMBP: Deleted derivative data file. [2025-10-20T17:11:17.703186] [INFO ] [MAIN ] Update Progress: 97/2367 symbols completed. AMBP took 31.2s. Estimated remaining: 4h 22m 37s. Rate: 8.6 symbols/min [2025-10-20T17:11:17.851898] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-05. [2025-10-20T17:11:17.871845] [INFO ] [MAIN ] AMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:17.892866] [DEBUG] [LOCAL ] AMG: Loaded 20485 existing records from primary storage. [2025-10-20T17:11:17.893117] [INFO ] [MAIN ] AMG: Loaded 20485 existing records. [2025-10-20T17:11:18.067621] [WARN ] [MAIN ] AMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:18.067842] [INFO ] [MAIN ] AMG: Wiping existing primary data before full refresh. [2025-10-20T17:11:18.067889] [INFO ] [LOCAL ] AMG: Starting deletion of primary data. [2025-10-20T17:11:18.068125] [DEBUG] [LOCAL ] AMG: Deleted primary data file. [2025-10-20T17:11:18.068181] [INFO ] [MAIN ] AMG: Starting full primary refresh. [2025-10-20T17:11:18.419753] [INFO ] [REMOTE ] AMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:18.651397] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-06. [2025-10-20T17:11:18.870861] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T17:11:19.451355] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-01. [2025-10-20T17:11:19.665051] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:11:20.251382] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-06. [2025-10-20T17:11:20.360937] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:11:20.974813] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-10-20T17:11:21.051344] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-07. [2025-10-20T17:11:21.664856] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T17:11:21.851433] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-07. [2025-10-20T17:11:22.651453] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-05. [2025-10-20T17:11:23.342687] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-10-20T17:11:23.451386] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-08. [2025-10-20T17:11:23.764937] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T17:11:24.251354] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-02. [2025-10-20T17:11:24.434091] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:11:24.919210] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T17:11:25.051347] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-06. [2025-10-20T17:11:25.851406] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-03. [2025-10-20T17:11:25.971110] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T17:11:26.651371] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-09. [2025-10-20T17:11:26.856701] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:11:27.033779] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:11:27.451392] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-04. [2025-10-20T17:11:28.251394] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-07. [2025-10-20T17:11:28.373568] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:11:28.984001] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T17:11:29.051367] [INFO ] [REMOTE ] AMC: Fetching primary data for 2025-10. [2025-10-20T17:11:29.851414] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-08. [2025-10-20T17:11:30.082090] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:11:30.651425] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-05. [2025-10-20T17:11:31.451340] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-08. [2025-10-20T17:11:31.572153] [INFO ] [REMOTE ] AMC: Incremental fetch complete for 2025-10 (2.52 seconds). [2025-10-20T17:11:31.571478] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T17:11:31.590771] [INFO ] [REMOTE ] AMC: Full fetch complete (77930 records). [2025-10-20T17:11:31.590929] [DEBUG] [VALIDATOR ] AMC: Validating primary data. [2025-10-20T17:11:31.768771] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T17:11:32.005767] [DEBUG] [VALIDATOR ] AMC: All expected trading days are present. [2025-10-20T17:11:32.005859] [DEBUG] [VALIDATOR ] AMC: Primary data validation passed. [2025-10-20T17:11:32.006590] [INFO ] [MAIN ] AMC: Fetched primary data is valid. [2025-10-20T17:11:32.006635] [DEBUG] [LOCAL ] AMC: Saving primary data. [2025-10-20T17:11:32.008133] [DEBUG] [LOCAL ] AMC: Trimmed 77930 records to 53766 (keeping 200 days of data) [2025-10-20T17:11:32.256975] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-09. [2025-10-20T17:11:32.304350] [DEBUG] [LOCAL ] AMC: Primary data saved (53766 records). [2025-10-20T17:11:32.304450] [INFO ] [MAIN ] AMC: Full history saved. [2025-10-20T17:11:32.304524] [INFO ] [MAIN ] Update Progress: 98/2367 symbols completed. AMC took 33.5s. Estimated remaining: 4h 25m 27s. Rate: 8.5 symbols/min [2025-10-20T17:11:32.477608] [INFO ] [MAIN ] AMH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:32.536004] [DEBUG] [LOCAL ] AMH: Loaded 52597 existing records from primary storage. [2025-10-20T17:11:32.536408] [INFO ] [MAIN ] AMH: Loaded 52597 existing records. [2025-10-20T17:11:32.732176] [WARN ] [MAIN ] AMH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:32.732504] [INFO ] [MAIN ] AMH: Wiping existing primary data before full refresh. [2025-10-20T17:11:32.732565] [INFO ] [LOCAL ] AMH: Starting deletion of primary data. [2025-10-20T17:11:32.732952] [DEBUG] [LOCAL ] AMH: Deleted primary data file. [2025-10-20T17:11:32.733011] [INFO ] [MAIN ] AMH: Starting full primary refresh. [2025-10-20T17:11:33.053729] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-06. [2025-10-20T17:11:33.117355] [INFO ] [REMOTE ] AMH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:33.148380] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:11:33.851355] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-01. [2025-10-20T17:11:33.994583] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:11:34.103402] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:11:34.651406] [INFO ] [REMOTE ] AMCR: Fetching primary data for 2025-10. [2025-10-20T17:11:35.058231] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:11:35.451397] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-07. [2025-10-20T17:11:36.165387] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:11:36.251369] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-08. [2025-10-20T17:11:36.861069] [INFO ] [REMOTE ] AMCR: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:11:36.870088] [INFO ] [REMOTE ] AMCR: Full fetch complete (77842 records). [2025-10-20T17:11:36.870255] [DEBUG] [VALIDATOR ] AMCR: Validating primary data. [2025-10-20T17:11:37.061719] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-09. [2025-10-20T17:11:37.359519] [DEBUG] [VALIDATOR ] AMCR: All expected trading days are present. [2025-10-20T17:11:37.359626] [DEBUG] [VALIDATOR ] AMCR: Primary data validation passed. [2025-10-20T17:11:37.360421] [INFO ] [MAIN ] AMCR: Fetched primary data is valid. [2025-10-20T17:11:37.365684] [DEBUG] [LOCAL ] AMCR: Saving primary data. [2025-10-20T17:11:37.371956] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:11:37.376913] [DEBUG] [LOCAL ] AMCR: Trimmed 77842 records to 53670 (keeping 200 days of data) [2025-10-20T17:11:37.697677] [DEBUG] [LOCAL ] AMCR: Primary data saved (53670 records). [2025-10-20T17:11:37.697858] [INFO ] [MAIN ] AMCR: Full history saved. [2025-10-20T17:11:37.698639] [INFO ] [MAIN ] Update Progress: 99/2367 symbols completed. AMCR took 37.4s. Estimated remaining: 4h 24m 43s. Rate: 8.6 symbols/min [2025-10-20T17:11:37.856444] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-02. [2025-10-20T17:11:37.883490] [INFO ] [MAIN ] AMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:37.883714] [DEBUG] [LOCAL ] AMN: No existing data file found in primary storage. [2025-10-20T17:11:37.883772] [WARN ] [MAIN ] AMN: No existing data found, performing full refresh instead. [2025-10-20T17:11:37.883895] [INFO ] [MAIN ] AMN: Wiping existing primary data before full refresh. [2025-10-20T17:11:37.883959] [INFO ] [LOCAL ] AMN: Starting deletion of primary data. [2025-10-20T17:11:37.884018] [DEBUG] [LOCAL ] AMN: No primary data file found to delete. [2025-10-20T17:11:37.884064] [INFO ] [MAIN ] AMN: Starting full primary refresh. [2025-10-20T17:11:38.253688] [INFO ] [REMOTE ] AMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:38.651419] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-01. [2025-10-20T17:11:38.875582] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T17:11:39.263902] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:11:39.451435] [INFO ] [REMOTE ] AME: Fetching primary data for 2025-10. [2025-10-20T17:11:39.518748] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:11:40.251379] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-09. [2025-10-20T17:11:41.051352] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-02. [2025-10-20T17:11:41.129531] [INFO ] [REMOTE ] AME: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:11:41.136801] [INFO ] [REMOTE ] AME: Full fetch complete (69996 records). [2025-10-20T17:11:41.136937] [DEBUG] [VALIDATOR ] AME: Validating primary data. [2025-10-20T17:11:41.480552] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:11:41.559401] [DEBUG] [VALIDATOR ] AME: All expected trading days are present. [2025-10-20T17:11:41.559479] [DEBUG] [VALIDATOR ] AME: Primary data validation passed. [2025-10-20T17:11:41.560188] [INFO ] [MAIN ] AME: Fetched primary data is valid. [2025-10-20T17:11:41.560251] [DEBUG] [LOCAL ] AME: Saving primary data. [2025-10-20T17:11:41.561572] [DEBUG] [LOCAL ] AME: Trimmed 69996 records to 48570 (keeping 200 days of data) [2025-10-20T17:11:41.857800] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-03. [2025-10-20T17:11:41.889772] [DEBUG] [LOCAL ] AME: Primary data saved (48570 records). [2025-10-20T17:11:41.889896] [INFO ] [MAIN ] AME: Full history saved. [2025-10-20T17:11:41.890866] [INFO ] [MAIN ] Update Progress: 100/2367 symbols completed. AME took 34.0s. Estimated remaining: 4h 23m 32s. Rate: 8.6 symbols/min [2025-10-20T17:11:42.065409] [INFO ] [MAIN ] AMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:42.098746] [DEBUG] [LOCAL ] AMP: Loaded 32543 existing records from primary storage. [2025-10-20T17:11:42.099028] [INFO ] [MAIN ] AMP: Loaded 32543 existing records. [2025-10-20T17:11:42.284678] [WARN ] [MAIN ] AMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:42.284889] [INFO ] [MAIN ] AMP: Wiping existing primary data before full refresh. [2025-10-20T17:11:42.284935] [INFO ] [LOCAL ] AMP: Starting deletion of primary data. [2025-10-20T17:11:42.285218] [DEBUG] [LOCAL ] AMP: Deleted primary data file. [2025-10-20T17:11:42.285275] [INFO ] [MAIN ] AMP: Starting full primary refresh. [2025-10-20T17:11:42.661082] [INFO ] [REMOTE ] AMG: Fetching primary data for 2025-10. [2025-10-20T17:11:42.666670] [INFO ] [REMOTE ] AMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:42.696846] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:11:43.340801] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:11:43.451363] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-01. [2025-10-20T17:11:43.872737] [INFO ] [REMOTE ] AMG: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:11:43.876059] [INFO ] [REMOTE ] AMG: Full fetch complete (29606 records). [2025-10-20T17:11:43.876141] [DEBUG] [VALIDATOR ] AMG: Validating primary data. [2025-10-20T17:11:44.251899] [DEBUG] [VALIDATOR ] AMG: All expected trading days are present. [2025-10-20T17:11:44.251759] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-04. [2025-10-20T17:11:44.252997] [DEBUG] [VALIDATOR ] AMG: Primary data validation passed. [2025-10-20T17:11:44.253407] [INFO ] [MAIN ] AMG: Fetched primary data is valid. [2025-10-20T17:11:44.253469] [DEBUG] [LOCAL ] AMG: Saving primary data. [2025-10-20T17:11:44.254295] [DEBUG] [LOCAL ] AMG: Trimmed 29606 records to 20174 (keeping 200 days of data) [2025-10-20T17:11:44.380788] [DEBUG] [LOCAL ] AMG: Primary data saved (20174 records). [2025-10-20T17:11:44.380918] [INFO ] [MAIN ] AMG: Full history saved. [2025-10-20T17:11:44.380990] [INFO ] [MAIN ] Update Progress: 101/2367 symbols completed. AMG took 26.7s. Estimated remaining: 4h 21m 44s. Rate: 8.7 symbols/min [2025-10-20T17:11:44.554150] [INFO ] [MAIN ] AMPX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:11:44.601915] [DEBUG] [LOCAL ] AMPX: Loaded 53118 existing records from primary storage. [2025-10-20T17:11:44.602264] [INFO ] [MAIN ] AMPX: Loaded 53118 existing records. [2025-10-20T17:11:44.786255] [WARN ] [MAIN ] AMPX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:11:44.786616] [INFO ] [MAIN ] AMPX: Wiping existing primary data before full refresh. [2025-10-20T17:11:44.786699] [INFO ] [LOCAL ] AMPX: Starting deletion of primary data. [2025-10-20T17:11:44.787074] [DEBUG] [LOCAL ] AMPX: Deleted primary data file. [2025-10-20T17:11:44.787129] [INFO ] [MAIN ] AMPX: Starting full primary refresh. [2025-10-20T17:11:45.058507] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-03. [2025-10-20T17:11:45.187327] [INFO ] [REMOTE ] AMPX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:11:45.214810] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T17:11:45.851369] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-01. [2025-10-20T17:11:45.990436] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T17:11:46.651353] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-05. [2025-10-20T17:11:46.753575] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T17:11:47.451378] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-04. [2025-10-20T17:11:47.643838] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:11:48.251354] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-02. [2025-10-20T17:11:48.426323] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:11:48.982443] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:11:49.051340] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-05. [2025-10-20T17:11:49.738165] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:11:49.851344] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-03. [2025-10-20T17:11:50.539874] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:11:50.651391] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-06. [2025-10-20T17:11:51.451419] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-02. [2025-10-20T17:11:51.871057] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:11:51.942149] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:11:52.251364] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-07. [2025-10-20T17:11:52.533228] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:11:53.051346] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-06. [2025-10-20T17:11:53.851378] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-03. [2025-10-20T17:11:54.045507] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:11:54.399558] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:11:54.651373] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-07. [2025-10-20T17:11:55.158164] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:11:55.451378] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-08. [2025-10-20T17:11:56.151274] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:11:56.251333] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-04. [2025-10-20T17:11:57.051390] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-08. [2025-10-20T17:11:57.185432] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T17:11:57.851508] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-09. [2025-10-20T17:11:57.887163] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:11:58.651394] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-04. [2025-10-20T17:11:58.830498] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T17:11:59.451370] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-09. [2025-10-20T17:11:59.615274] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:12:00.109376] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:12:00.251431] [INFO ] [REMOTE ] AMH: Fetching primary data for 2025-10. [2025-10-20T17:12:00.802037] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:12:01.051383] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-05. [2025-10-20T17:12:01.851397] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-05. [2025-10-20T17:12:02.198143] [INFO ] [REMOTE ] AMH: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:12:02.206023] [INFO ] [REMOTE ] AMH: Full fetch complete (75082 records). [2025-10-20T17:12:02.206188] [DEBUG] [VALIDATOR ] AMH: Validating primary data. [2025-10-20T17:12:02.617465] [DEBUG] [VALIDATOR ] AMH: All expected trading days are present. [2025-10-20T17:12:02.617683] [DEBUG] [VALIDATOR ] AMH: Primary data validation passed. [2025-10-20T17:12:02.618751] [INFO ] [MAIN ] AMH: Fetched primary data is valid. [2025-10-20T17:12:02.618842] [DEBUG] [LOCAL ] AMH: Saving primary data. [2025-10-20T17:12:02.644637] [DEBUG] [LOCAL ] AMH: Trimmed 75082 records to 51831 (keeping 200 days of data) [2025-10-20T17:12:02.646339] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:12:02.657071] [INFO ] [REMOTE ] AMN: Fetching primary data for 2025-10. [2025-10-20T17:12:02.980394] [DEBUG] [LOCAL ] AMH: Primary data saved (51831 records). [2025-10-20T17:12:02.980536] [INFO ] [MAIN ] AMH: Full history saved. [2025-10-20T17:12:02.980614] [INFO ] [MAIN ] Update Progress: 102/2367 symbols completed. AMH took 30.7s. Estimated remaining: 4h 25m 57s. Rate: 8.5 symbols/min [2025-10-20T17:12:03.148708] [INFO ] [MAIN ] AMPY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:03.185970] [DEBUG] [LOCAL ] AMPY: Loaded 39876 existing records from primary storage. [2025-10-20T17:12:03.186273] [INFO ] [MAIN ] AMPY: Loaded 39876 existing records. [2025-10-20T17:12:03.369699] [WARN ] [MAIN ] AMPY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:03.370003] [INFO ] [MAIN ] AMPY: Wiping existing primary data before full refresh. [2025-10-20T17:12:03.370112] [INFO ] [LOCAL ] AMPY: Starting deletion of primary data. [2025-10-20T17:12:03.370404] [DEBUG] [LOCAL ] AMPY: Deleted primary data file. [2025-10-20T17:12:03.370516] [INFO ] [MAIN ] AMPY: Starting full primary refresh. [2025-10-20T17:12:03.468236] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-06. [2025-10-20T17:12:03.682444] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T17:12:03.786702] [INFO ] [REMOTE ] AMPY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:04.251428] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-06. [2025-10-20T17:12:04.438867] [INFO ] [REMOTE ] AMN: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T17:12:04.445178] [INFO ] [REMOTE ] AMN: Full fetch complete (60282 records). [2025-10-20T17:12:04.445293] [DEBUG] [VALIDATOR ] AMN: Validating primary data. [2025-10-20T17:12:04.918903] [DEBUG] [VALIDATOR ] AMN: All expected trading days are present. [2025-10-20T17:12:04.919123] [DEBUG] [VALIDATOR ] AMN: Primary data validation passed. [2025-10-20T17:12:04.920089] [INFO ] [MAIN ] AMN: Fetched primary data is valid. [2025-10-20T17:12:04.920183] [DEBUG] [LOCAL ] AMN: Saving primary data. [2025-10-20T17:12:04.942114] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:12:04.949764] [DEBUG] [LOCAL ] AMN: Trimmed 60282 records to 41731 (keeping 200 days of data) [2025-10-20T17:12:05.053764] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-07. [2025-10-20T17:12:05.216301] [DEBUG] [LOCAL ] AMN: Primary data saved (41731 records). [2025-10-20T17:12:05.216471] [INFO ] [MAIN ] AMN: Full history saved. [2025-10-20T17:12:05.216557] [INFO ] [MAIN ] Update Progress: 103/2367 symbols completed. AMN took 27.5s. Estimated remaining: 4h 24m 4s. Rate: 8.6 symbols/min [2025-10-20T17:12:05.407417] [INFO ] [MAIN ] AMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:05.441735] [DEBUG] [LOCAL ] AMR: Loaded 29322 existing records from primary storage. [2025-10-20T17:12:05.442217] [INFO ] [MAIN ] AMR: Loaded 29322 existing records. [2025-10-20T17:12:05.656571] [WARN ] [MAIN ] AMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:05.656894] [INFO ] [MAIN ] AMR: Wiping existing primary data before full refresh. [2025-10-20T17:12:05.656947] [INFO ] [LOCAL ] AMR: Starting deletion of primary data. [2025-10-20T17:12:05.657226] [DEBUG] [LOCAL ] AMR: Deleted primary data file. [2025-10-20T17:12:05.657288] [INFO ] [MAIN ] AMR: Starting full primary refresh. [2025-10-20T17:12:05.858255] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-01. [2025-10-20T17:12:06.083144] [INFO ] [REMOTE ] AMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:06.167555] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:12:06.406906] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T17:12:06.651388] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-07. [2025-10-20T17:12:07.159856] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T17:12:07.451393] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-02. [2025-10-20T17:12:08.251369] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-01. [2025-10-20T17:12:08.553107] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T17:12:08.606330] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:12:09.051400] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-08. [2025-10-20T17:12:09.594339] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:12:09.851387] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-08. [2025-10-20T17:12:10.651391] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-03. [2025-10-20T17:12:10.772025] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T17:12:11.456788] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-02. [2025-10-20T17:12:11.462494] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-08 (2.41 seconds). [2025-10-20T17:12:11.903421] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T17:12:12.251420] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-04. [2025-10-20T17:12:12.433120] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:12:13.051381] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-09. [2025-10-20T17:12:13.742113] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:12:13.851443] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-03. [2025-10-20T17:12:14.084332] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T17:12:14.651453] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-09. [2025-10-20T17:12:15.147695] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T17:12:15.451360] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-04. [2025-10-20T17:12:16.251436] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-05. [2025-10-20T17:12:16.664868] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:12:16.707121] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:12:17.051412] [INFO ] [REMOTE ] AMPX: Fetching primary data for 2025-10. [2025-10-20T17:12:17.389238] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:12:17.851410] [INFO ] [REMOTE ] AMP: Fetching primary data for 2025-10. [2025-10-20T17:12:18.651374] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-06. [2025-10-20T17:12:19.004299] [INFO ] [REMOTE ] AMPX: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:12:19.012897] [INFO ] [REMOTE ] AMPX: Full fetch complete (75220 records). [2025-10-20T17:12:19.013053] [DEBUG] [VALIDATOR ] AMPX: Validating primary data. [2025-10-20T17:12:19.412720] [DEBUG] [VALIDATOR ] AMPX: All expected trading days are present. [2025-10-20T17:12:19.412886] [DEBUG] [VALIDATOR ] AMPX: Primary data validation passed. [2025-10-20T17:12:19.413680] [INFO ] [MAIN ] AMPX: Fetched primary data is valid. [2025-10-20T17:12:19.413848] [DEBUG] [LOCAL ] AMPX: Saving primary data. [2025-10-20T17:12:19.427475] [INFO ] [REMOTE ] AMP: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:12:19.432549] [DEBUG] [LOCAL ] AMPX: Trimmed 75220 records to 52385 (keeping 200 days of data) [2025-10-20T17:12:19.433147] [INFO ] [REMOTE ] AMP: Full fetch complete (47590 records). [2025-10-20T17:12:19.433230] [DEBUG] [VALIDATOR ] AMP: Validating primary data. [2025-10-20T17:12:19.456476] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-05. [2025-10-20T17:12:20.096283] [DEBUG] [VALIDATOR ] AMP: All expected trading days are present. [2025-10-20T17:12:20.102478] [DEBUG] [VALIDATOR ] AMP: Primary data validation passed. [2025-10-20T17:12:20.103243] [INFO ] [MAIN ] AMP: Fetched primary data is valid. [2025-10-20T17:12:20.103438] [DEBUG] [LOCAL ] AMP: Saving primary data. [2025-10-20T17:12:20.201427] [DEBUG] [LOCAL ] AMPX: Primary data saved (52385 records). [2025-10-20T17:12:20.203153] [INFO ] [MAIN ] AMPX: Full history saved. [2025-10-20T17:12:20.203375] [INFO ] [MAIN ] Update Progress: 104/2367 symbols completed. AMPX took 35.8s. Estimated remaining: 4h 26m 51s. Rate: 8.5 symbols/min [2025-10-20T17:12:20.203025] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:12:20.218296] [DEBUG] [LOCAL ] AMP: Trimmed 47590 records to 31952 (keeping 200 days of data) [2025-10-20T17:12:20.353502] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-07. [2025-10-20T17:12:20.508055] [INFO ] [MAIN ] AMRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:20.595755] [DEBUG] [LOCAL ] AMRC: Loaded 39116 existing records from primary storage. [2025-10-20T17:12:20.620032] [INFO ] [MAIN ] AMRC: Loaded 39116 existing records. [2025-10-20T17:12:20.633673] [DEBUG] [LOCAL ] AMP: Primary data saved (31952 records). [2025-10-20T17:12:20.633931] [INFO ] [MAIN ] AMP: Full history saved. [2025-10-20T17:12:20.634753] [INFO ] [MAIN ] Update Progress: 105/2367 symbols completed. AMP took 38.7s. Estimated remaining: 4h 24m 20s. Rate: 8.6 symbols/min [2025-10-20T17:12:20.959825] [WARN ] [MAIN ] AMRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:20.986881] [INFO ] [MAIN ] AMRC: Wiping existing primary data before full refresh. [2025-10-20T17:12:21.007791] [INFO ] [MAIN ] AMRZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:21.008088] [DEBUG] [LOCAL ] AMRZ: No existing data file found in primary storage. [2025-10-20T17:12:21.008301] [WARN ] [MAIN ] AMRZ: No existing data found, performing full refresh instead. [2025-10-20T17:12:21.008532] [INFO ] [MAIN ] AMRZ: Wiping existing primary data before full refresh. [2025-10-20T17:12:21.008639] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-10-20T17:12:21.008719] [INFO ] [LOCAL ] AMRC: Starting deletion of primary data. [2025-10-20T17:12:21.008946] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-10-20T17:12:21.009136] [INFO ] [MAIN ] AMRZ: Starting full primary refresh. [2025-10-20T17:12:21.052708] [DEBUG] [LOCAL ] AMRC: Deleted primary data file. [2025-10-20T17:12:21.079202] [INFO ] [MAIN ] AMRC: Starting full primary refresh. [2025-10-20T17:12:21.231571] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:12:21.270283] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-06. [2025-10-20T17:12:21.745842] [INFO ] [REMOTE ] AMRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:21.788205] [INFO ] [REMOTE ] AMRZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:21.814559] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T17:12:21.851333] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-08. [2025-10-20T17:12:22.315224] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:12:22.651426] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-07. [2025-10-20T17:12:23.097342] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:12:23.451389] [INFO ] [REMOTE ] AMRZ: Fetching primary data for 2025-01. [2025-10-20T17:12:23.642391] [DEBUG] [REMOTE ] AMRZ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:12:23.642762] [WARN ] [REMOTE ] AMRZ: No data returned for month 2025-01 [2025-10-20T17:12:23.642854] [FATAL] [REMOTE ] AMRZ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:12:23.642902] [FATAL] [MAIN ] AMRZ: Failed to fetch primary data. [2025-10-20T17:12:23.642940] [WARN ] [MAIN ] AMRZ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:12:23.643000] [INFO ] [LOCAL ] AMRZ: Starting deletion of primary data. [2025-10-20T17:12:23.643080] [DEBUG] [LOCAL ] AMRZ: No primary data file found to delete. [2025-10-20T17:12:23.643137] [INFO ] [LOCAL ] AMRZ: Starting deletion of derivative data. [2025-10-20T17:12:23.643179] [DEBUG] [LOCAL ] AMRZ: No derivative data file found to delete. [2025-10-20T17:12:23.643224] [INFO ] [MAIN ] Update Progress: 106/2367 symbols completed. AMRZ took 3.0s. Estimated remaining: 4h 22m 48s. Rate: 8.6 symbols/min [2025-10-20T17:12:23.855177] [INFO ] [MAIN ] AMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:23.918407] [DEBUG] [LOCAL ] AMT: Loaded 53232 existing records from primary storage. [2025-10-20T17:12:23.918838] [INFO ] [MAIN ] AMT: Loaded 53232 existing records. [2025-10-20T17:12:24.116769] [WARN ] [MAIN ] AMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:24.117042] [INFO ] [MAIN ] AMT: Wiping existing primary data before full refresh. [2025-10-20T17:12:24.117117] [INFO ] [LOCAL ] AMT: Starting deletion of primary data. [2025-10-20T17:12:24.117491] [DEBUG] [LOCAL ] AMT: Deleted primary data file. [2025-10-20T17:12:24.117553] [INFO ] [MAIN ] AMT: Starting full primary refresh. [2025-10-20T17:12:24.278963] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-09. [2025-10-20T17:12:24.342353] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:12:24.532477] [INFO ] [REMOTE ] AMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:25.051427] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-08. [2025-10-20T17:12:25.382402] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T17:12:25.851392] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-01. [2025-10-20T17:12:26.191454] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T17:12:26.651429] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-01. [2025-10-20T17:12:27.451390] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-09. [2025-10-20T17:12:27.846584] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:12:28.037633] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:12:28.251592] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-02. [2025-10-20T17:12:28.280150] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:12:29.051391] [INFO ] [REMOTE ] AMR: Fetching primary data for 2025-10. [2025-10-20T17:12:29.851447] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-02. [2025-10-20T17:12:29.982829] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T17:12:30.315286] [INFO ] [REMOTE ] AMR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:12:30.320072] [INFO ] [REMOTE ] AMR: Full fetch complete (40691 records). [2025-10-20T17:12:30.320185] [DEBUG] [VALIDATOR ] AMR: Validating primary data. [2025-10-20T17:12:30.659235] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-03. [2025-10-20T17:12:30.713701] [DEBUG] [VALIDATOR ] AMR: All expected trading days are present. [2025-10-20T17:12:30.713829] [DEBUG] [VALIDATOR ] AMR: Primary data validation passed. [2025-10-20T17:12:30.714380] [INFO ] [MAIN ] AMR: Fetched primary data is valid. [2025-10-20T17:12:30.714426] [DEBUG] [LOCAL ] AMR: Saving primary data. [2025-10-20T17:12:30.715589] [DEBUG] [LOCAL ] AMR: Trimmed 40691 records to 28978 (keeping 200 days of data) [2025-10-20T17:12:30.894396] [DEBUG] [LOCAL ] AMR: Primary data saved (28978 records). [2025-10-20T17:12:30.894630] [INFO ] [MAIN ] AMR: Full history saved. [2025-10-20T17:12:30.895332] [INFO ] [MAIN ] Update Progress: 107/2367 symbols completed. AMR took 25.7s. Estimated remaining: 4h 22m 47s. Rate: 8.6 symbols/min [2025-10-20T17:12:31.076589] [INFO ] [MAIN ] AMTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:31.101017] [DEBUG] [LOCAL ] AMTB: Loaded 20848 existing records from primary storage. [2025-10-20T17:12:31.101348] [INFO ] [MAIN ] AMTB: Loaded 20848 existing records. [2025-10-20T17:12:31.227452] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:12:31.317039] [WARN ] [MAIN ] AMTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:31.317328] [INFO ] [MAIN ] AMTB: Wiping existing primary data before full refresh. [2025-10-20T17:12:31.317379] [INFO ] [LOCAL ] AMTB: Starting deletion of primary data. [2025-10-20T17:12:31.317564] [DEBUG] [LOCAL ] AMTB: Deleted primary data file. [2025-10-20T17:12:31.317603] [INFO ] [MAIN ] AMTB: Starting full primary refresh. [2025-10-20T17:12:31.456621] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-03. [2025-10-20T17:12:31.686396] [INFO ] [REMOTE ] AMTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:32.224058] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:12:32.251414] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-01. [2025-10-20T17:12:33.051436] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-04. [2025-10-20T17:12:33.280762] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:12:33.658291] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:12:33.851464] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-02. [2025-10-20T17:12:34.510339] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:12:34.548749] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T17:12:34.651439] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-04. [2025-10-20T17:12:35.451382] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-05. [2025-10-20T17:12:35.863435] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:12:36.251416] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-03. [2025-10-20T17:12:36.901862] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T17:12:37.051398] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-05. [2025-10-20T17:12:37.268058] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T17:12:37.851397] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-06. [2025-10-20T17:12:38.185463] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:12:38.651407] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-06. [2025-10-20T17:12:39.200318] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:12:39.451359] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-04. [2025-10-20T17:12:39.901848] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:12:40.251386] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-07. [2025-10-20T17:12:40.499524] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T17:12:41.051381] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-07. [2025-10-20T17:12:41.851390] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-05. [2025-10-20T17:12:42.107078] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T17:12:42.651403] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-08. [2025-10-20T17:12:43.451388] [INFO ] [REMOTE ] AMPY: Fetching primary data for 2025-10. [2025-10-20T17:12:44.171615] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-10-20T17:12:44.251395] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-06. [2025-10-20T17:12:44.347348] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-07 (3.30 seconds). [2025-10-20T17:12:44.949397] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-08 (2.30 seconds). [2025-10-20T17:12:44.971824] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T17:12:45.051452] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-09. [2025-10-20T17:12:45.273613] [INFO ] [REMOTE ] AMPY: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:12:45.285978] [INFO ] [REMOTE ] AMPY: Full fetch complete (53157 records). [2025-10-20T17:12:45.286149] [DEBUG] [VALIDATOR ] AMPY: Validating primary data. [2025-10-20T17:12:45.697394] [DEBUG] [VALIDATOR ] AMPY: All expected trading days are present. [2025-10-20T17:12:45.697478] [DEBUG] [VALIDATOR ] AMPY: Primary data validation passed. [2025-10-20T17:12:45.698242] [INFO ] [MAIN ] AMPY: Fetched primary data is valid. [2025-10-20T17:12:45.698305] [DEBUG] [LOCAL ] AMPY: Saving primary data. [2025-10-20T17:12:45.699496] [DEBUG] [LOCAL ] AMPY: Trimmed 53157 records to 39461 (keeping 200 days of data) [2025-10-20T17:12:45.855418] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-07. [2025-10-20T17:12:45.926213] [DEBUG] [LOCAL ] AMPY: Primary data saved (39461 records). [2025-10-20T17:12:45.926354] [INFO ] [MAIN ] AMPY: Full history saved. [2025-10-20T17:12:45.926431] [INFO ] [MAIN ] Update Progress: 108/2367 symbols completed. AMPY took 42.9s. Estimated remaining: 4h 25m 28s. Rate: 8.5 symbols/min [2025-10-20T17:12:46.104370] [INFO ] [MAIN ] AMTD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:46.109588] [DEBUG] [LOCAL ] AMTD: Loaded 3828 existing records from primary storage. [2025-10-20T17:12:46.109791] [INFO ] [MAIN ] AMTD: Loaded 3828 existing records. [2025-10-20T17:12:46.317294] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:12:46.321354] [WARN ] [MAIN ] AMTD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:46.329488] [INFO ] [MAIN ] AMTD: Wiping existing primary data before full refresh. [2025-10-20T17:12:46.329661] [INFO ] [LOCAL ] AMTD: Starting deletion of primary data. [2025-10-20T17:12:46.330051] [DEBUG] [LOCAL ] AMTD: Deleted primary data file. [2025-10-20T17:12:46.330107] [INFO ] [MAIN ] AMTD: Starting full primary refresh. [2025-10-20T17:12:46.663754] [INFO ] [REMOTE ] AMRC: Fetching primary data for 2025-10. [2025-10-20T17:12:46.710423] [INFO ] [REMOTE ] AMTD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:47.108156] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:12:47.451380] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-08. [2025-10-20T17:12:48.251390] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-08. [2025-10-20T17:12:48.355964] [INFO ] [REMOTE ] AMRC: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:12:48.363696] [INFO ] [REMOTE ] AMRC: Full fetch complete (54461 records). [2025-10-20T17:12:48.363906] [DEBUG] [VALIDATOR ] AMRC: Validating primary data. [2025-10-20T17:12:48.738875] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:12:48.787126] [DEBUG] [VALIDATOR ] AMRC: All expected trading days are present. [2025-10-20T17:12:48.787218] [DEBUG] [VALIDATOR ] AMRC: Primary data validation passed. [2025-10-20T17:12:48.787807] [INFO ] [MAIN ] AMRC: Fetched primary data is valid. [2025-10-20T17:12:48.787871] [DEBUG] [LOCAL ] AMRC: Saving primary data. [2025-10-20T17:12:48.789194] [DEBUG] [LOCAL ] AMRC: Trimmed 54461 records to 38558 (keeping 200 days of data) [2025-10-20T17:12:49.023582] [DEBUG] [LOCAL ] AMRC: Primary data saved (38558 records). [2025-10-20T17:12:49.023781] [INFO ] [MAIN ] AMRC: Full history saved. [2025-10-20T17:12:49.023874] [INFO ] [MAIN ] Update Progress: 109/2367 symbols completed. AMRC took 28.8s. Estimated remaining: 4h 23m 60s. Rate: 8.6 symbols/min [2025-10-20T17:12:49.056501] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-01. [2025-10-20T17:12:49.202487] [INFO ] [MAIN ] AMTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:49.249431] [DEBUG] [LOCAL ] AMTM: Loaded 51234 existing records from primary storage. [2025-10-20T17:12:49.249777] [INFO ] [MAIN ] AMTM: Loaded 51234 existing records. [2025-10-20T17:12:49.431096] [WARN ] [MAIN ] AMTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:49.431345] [INFO ] [MAIN ] AMTM: Wiping existing primary data before full refresh. [2025-10-20T17:12:49.431402] [INFO ] [LOCAL ] AMTM: Starting deletion of primary data. [2025-10-20T17:12:49.431791] [DEBUG] [LOCAL ] AMTM: Deleted primary data file. [2025-10-20T17:12:49.431848] [INFO ] [MAIN ] AMTM: Starting full primary refresh. [2025-10-20T17:12:49.802869] [INFO ] [REMOTE ] AMTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:49.824711] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:12:49.851368] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-09. [2025-10-20T17:12:50.030529] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T17:12:50.651367] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-09. [2025-10-20T17:12:50.801589] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T17:12:51.451375] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-02. [2025-10-20T17:12:51.856414] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:12:52.251368] [INFO ] [REMOTE ] AMTB: Fetching primary data for 2025-10. [2025-10-20T17:12:52.699332] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T17:12:53.051390] [INFO ] [REMOTE ] AMT: Fetching primary data for 2025-10. [2025-10-20T17:12:53.305168] [INFO ] [REMOTE ] AMTB: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:12:53.309109] [INFO ] [REMOTE ] AMTB: Full fetch complete (29719 records). [2025-10-20T17:12:53.309367] [DEBUG] [VALIDATOR ] AMTB: Validating primary data. [2025-10-20T17:12:53.735382] [DEBUG] [VALIDATOR ] AMTB: All expected trading days are present. [2025-10-20T17:12:53.735502] [DEBUG] [VALIDATOR ] AMTB: Primary data validation passed. [2025-10-20T17:12:53.736039] [INFO ] [MAIN ] AMTB: Fetched primary data is valid. [2025-10-20T17:12:53.736108] [DEBUG] [LOCAL ] AMTB: Saving primary data. [2025-10-20T17:12:53.737684] [DEBUG] [LOCAL ] AMTB: Trimmed 29719 records to 20539 (keeping 200 days of data) [2025-10-20T17:12:53.858167] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-03. [2025-10-20T17:12:53.868514] [DEBUG] [LOCAL ] AMTB: Primary data saved (20539 records). [2025-10-20T17:12:53.868627] [INFO ] [MAIN ] AMTB: Full history saved. [2025-10-20T17:12:53.868745] [INFO ] [MAIN ] Update Progress: 110/2367 symbols completed. AMTB took 23.0s. Estimated remaining: 4h 23m 8s. Rate: 8.6 symbols/min [2025-10-20T17:12:54.100876] [INFO ] [MAIN ] AMWL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:54.112345] [DEBUG] [LOCAL ] AMWL: Loaded 9827 existing records from primary storage. [2025-10-20T17:12:54.112589] [INFO ] [MAIN ] AMWL: Loaded 9827 existing records. [2025-10-20T17:12:54.299678] [WARN ] [MAIN ] AMWL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:54.299927] [INFO ] [MAIN ] AMWL: Wiping existing primary data before full refresh. [2025-10-20T17:12:54.299975] [INFO ] [LOCAL ] AMWL: Starting deletion of primary data. [2025-10-20T17:12:54.300307] [DEBUG] [LOCAL ] AMWL: Deleted primary data file. [2025-10-20T17:12:54.300343] [INFO ] [MAIN ] AMWL: Starting full primary refresh. [2025-10-20T17:12:54.403772] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:12:54.664108] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-01. [2025-10-20T17:12:54.678478] [INFO ] [REMOTE ] AMWL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:54.950522] [INFO ] [REMOTE ] AMT: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:12:54.958321] [INFO ] [REMOTE ] AMT: Full fetch complete (76173 records). [2025-10-20T17:12:54.958459] [DEBUG] [VALIDATOR ] AMT: Validating primary data. [2025-10-20T17:12:55.339336] [DEBUG] [VALIDATOR ] AMT: All expected trading days are present. [2025-10-20T17:12:55.339427] [DEBUG] [VALIDATOR ] AMT: Primary data validation passed. [2025-10-20T17:12:55.340189] [INFO ] [MAIN ] AMT: Fetched primary data is valid. [2025-10-20T17:12:55.340253] [DEBUG] [LOCAL ] AMT: Saving primary data. [2025-10-20T17:12:55.341772] [DEBUG] [LOCAL ] AMT: Trimmed 76173 records to 52454 (keeping 200 days of data) [2025-10-20T17:12:55.456703] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-04. [2025-10-20T17:12:55.668406] [DEBUG] [LOCAL ] AMT: Primary data saved (52454 records). [2025-10-20T17:12:55.668548] [INFO ] [MAIN ] AMT: Full history saved. [2025-10-20T17:12:55.669460] [INFO ] [MAIN ] Update Progress: 111/2367 symbols completed. AMT took 32.0s. Estimated remaining: 4h 21m 15s. Rate: 8.6 symbols/min [2025-10-20T17:12:55.839305] [INFO ] [MAIN ] AMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:12:55.883968] [DEBUG] [LOCAL ] AMX: Loaded 49908 existing records from primary storage. [2025-10-20T17:12:55.884293] [INFO ] [MAIN ] AMX: Loaded 49908 existing records. [2025-10-20T17:12:55.988697] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T17:12:56.066168] [WARN ] [MAIN ] AMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:12:56.066402] [INFO ] [MAIN ] AMX: Wiping existing primary data before full refresh. [2025-10-20T17:12:56.066449] [INFO ] [LOCAL ] AMX: Starting deletion of primary data. [2025-10-20T17:12:56.066756] [DEBUG] [LOCAL ] AMX: Deleted primary data file. [2025-10-20T17:12:56.066811] [INFO ] [MAIN ] AMX: Starting full primary refresh. [2025-10-20T17:12:56.269278] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-05. [2025-10-20T17:12:56.435736] [INFO ] [REMOTE ] AMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:12:56.464273] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:12:56.850511] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T17:12:57.051389] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-06. [2025-10-20T17:12:57.751509] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:12:57.851432] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-01. [2025-10-20T17:12:58.574378] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T17:12:58.651548] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-02. [2025-10-20T17:12:59.451413] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-02. [2025-10-20T17:12:59.477809] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T17:13:00.251386] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-07. [2025-10-20T17:13:00.966361] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:13:01.051489] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-08. [2025-10-20T17:13:01.373050] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T17:13:01.508610] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T17:13:01.851376] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-03. [2025-10-20T17:13:02.651341] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-03. [2025-10-20T17:13:03.047943] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:13:03.451426] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-04. [2025-10-20T17:13:04.251465] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-01. [2025-10-20T17:13:04.692595] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:13:05.051433] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-05. [2025-10-20T17:13:05.286726] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-03 (2.64 seconds). [2025-10-20T17:13:05.568935] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:13:05.845310] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T17:13:05.851335] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-06. [2025-10-20T17:13:06.581239] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T17:13:06.651380] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-07. [2025-10-20T17:13:07.451401] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-02. [2025-10-20T17:13:07.514354] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T17:13:08.251423] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-08. [2025-10-20T17:13:08.865865] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T17:13:08.992441] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:13:09.051365] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-09. [2025-10-20T17:13:09.852993] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-03. [2025-10-20T17:13:09.857224] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:13:10.651458] [INFO ] [REMOTE ] AMWL: Fetching primary data for 2025-10. [2025-10-20T17:13:11.451366] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-04. [2025-10-20T17:13:11.528800] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:13:11.605023] [INFO ] [REMOTE ] AMWL: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T17:13:11.608140] [INFO ] [REMOTE ] AMWL: Full fetch complete (14622 records). [2025-10-20T17:13:11.608218] [DEBUG] [VALIDATOR ] AMWL: Validating primary data. [2025-10-20T17:13:12.013317] [DEBUG] [VALIDATOR ] AMWL: All expected trading days are present. [2025-10-20T17:13:12.013393] [DEBUG] [VALIDATOR ] AMWL: Primary data validation passed. [2025-10-20T17:13:12.013663] [INFO ] [MAIN ] AMWL: Fetched primary data is valid. [2025-10-20T17:13:12.013724] [DEBUG] [LOCAL ] AMWL: Saving primary data. [2025-10-20T17:13:12.014184] [DEBUG] [LOCAL ] AMWL: Trimmed 14622 records to 9705 (keeping 200 days of data) [2025-10-20T17:13:12.066158] [DEBUG] [LOCAL ] AMWL: Primary data saved (9705 records). [2025-10-20T17:13:12.066305] [INFO ] [MAIN ] AMWL: Full history saved. [2025-10-20T17:13:12.066384] [INFO ] [MAIN ] Update Progress: 112/2367 symbols completed. AMWL took 18.2s. Estimated remaining: 4h 24m 19s. Rate: 8.5 symbols/min [2025-10-20T17:13:12.241543] [INFO ] [MAIN ] AN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:12.259592] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-04. [2025-10-20T17:13:12.275177] [DEBUG] [LOCAL ] AN: Loaded 31816 existing records from primary storage. [2025-10-20T17:13:12.275505] [INFO ] [MAIN ] AN: Loaded 31816 existing records. [2025-10-20T17:13:12.459008] [WARN ] [MAIN ] AN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:12.459294] [INFO ] [MAIN ] AN: Wiping existing primary data before full refresh. [2025-10-20T17:13:12.459348] [INFO ] [LOCAL ] AN: Starting deletion of primary data. [2025-10-20T17:13:12.459579] [DEBUG] [LOCAL ] AN: Deleted primary data file. [2025-10-20T17:13:12.459636] [INFO ] [MAIN ] AN: Starting full primary refresh. [2025-10-20T17:13:12.865535] [INFO ] [REMOTE ] AN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:13.051425] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-09. [2025-10-20T17:13:13.167835] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:13:13.782760] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T17:13:13.852149] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-01. [2025-10-20T17:13:13.852057] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:13:14.651435] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-05. [2025-10-20T17:13:14.905287] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:13:15.451425] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-02. [2025-10-20T17:13:16.251465] [INFO ] [REMOTE ] AMTD: Fetching primary data for 2025-10. [2025-10-20T17:13:16.333150] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:13:16.995836] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:13:17.051420] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-06. [2025-10-20T17:13:17.062433] [INFO ] [REMOTE ] AMTD: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T17:13:17.064901] [INFO ] [REMOTE ] AMTD: Full fetch complete (5467 records). [2025-10-20T17:13:17.065123] [DEBUG] [VALIDATOR ] AMTD: Validating primary data. [2025-10-20T17:13:17.512190] [DEBUG] [VALIDATOR ] AMTD: All expected trading days are present. [2025-10-20T17:13:17.512272] [DEBUG] [VALIDATOR ] AMTD: Primary data validation passed. [2025-10-20T17:13:17.512446] [INFO ] [MAIN ] AMTD: Fetched primary data is valid. [2025-10-20T17:13:17.512487] [DEBUG] [LOCAL ] AMTD: Saving primary data. [2025-10-20T17:13:17.513200] [DEBUG] [LOCAL ] AMTD: Trimmed 5467 records to 3824 (keeping 200 days of data) [2025-10-20T17:13:17.537633] [DEBUG] [LOCAL ] AMTD: Primary data saved (3824 records). [2025-10-20T17:13:17.537867] [INFO ] [MAIN ] AMTD: Full history saved. [2025-10-20T17:13:17.537976] [INFO ] [MAIN ] Update Progress: 113/2367 symbols completed. AMTD took 31.6s. Estimated remaining: 4h 23m 41s. Rate: 8.5 symbols/min [2025-10-20T17:13:17.747628] [INFO ] [MAIN ] ANET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:17.810379] [DEBUG] [LOCAL ] ANET: Loaded 54546 existing records from primary storage. [2025-10-20T17:13:17.810855] [INFO ] [MAIN ] ANET: Loaded 54546 existing records. [2025-10-20T17:13:17.856619] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-03. [2025-10-20T17:13:18.041120] [WARN ] [MAIN ] ANET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:18.041417] [INFO ] [MAIN ] ANET: Wiping existing primary data before full refresh. [2025-10-20T17:13:18.041471] [INFO ] [LOCAL ] ANET: Starting deletion of primary data. [2025-10-20T17:13:18.041896] [DEBUG] [LOCAL ] ANET: Deleted primary data file. [2025-10-20T17:13:18.041958] [INFO ] [MAIN ] ANET: Starting full primary refresh. [2025-10-20T17:13:18.469376] [INFO ] [REMOTE ] ANET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:18.651380] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-01. [2025-10-20T17:13:18.744890] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:13:19.430869] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:13:19.451347] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-04. [2025-10-20T17:13:20.251452] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-05. [2025-10-20T17:13:20.947318] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:13:21.058545] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-07. [2025-10-20T17:13:21.064501] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-01 (2.41 seconds). [2025-10-20T17:13:21.516952] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:13:21.851363] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-06. [2025-10-20T17:13:22.422602] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:13:22.651416] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-02. [2025-10-20T17:13:23.451454] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-08. [2025-10-20T17:13:23.702267] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T17:13:24.251458] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-05. [2025-10-20T17:13:25.041303] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:13:25.051566] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-07. [2025-10-20T17:13:25.093155] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-02 (2.44 seconds). [2025-10-20T17:13:25.421199] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T17:13:25.851420] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-06. [2025-10-20T17:13:26.651345] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-03. [2025-10-20T17:13:26.714132] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T17:13:26.774875] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:13:27.451626] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-08. [2025-10-20T17:13:28.251389] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-07. [2025-10-20T17:13:28.755476] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T17:13:29.051389] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-09. [2025-10-20T17:13:29.186744] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:13:29.692888] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:13:29.851371] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-08. [2025-10-20T17:13:30.651358] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-09. [2025-10-20T17:13:30.793261] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:13:30.968124] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:13:31.451382] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-09. [2025-10-20T17:13:32.128902] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:13:32.251379] [INFO ] [REMOTE ] AMTM: Fetching primary data for 2025-10. [2025-10-20T17:13:32.685404] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:13:33.051414] [INFO ] [REMOTE ] AN: Fetching primary data for 2025-10. [2025-10-20T17:13:33.851422] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-04. [2025-10-20T17:13:33.897306] [INFO ] [REMOTE ] AMTM: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T17:13:33.905458] [INFO ] [REMOTE ] AMTM: Full fetch complete (73747 records). [2025-10-20T17:13:33.905585] [DEBUG] [VALIDATOR ] AMTM: Validating primary data. [2025-10-20T17:13:34.301939] [DEBUG] [VALIDATOR ] AMTM: All expected trading days are present. [2025-10-20T17:13:34.302051] [DEBUG] [VALIDATOR ] AMTM: Primary data validation passed. [2025-10-20T17:13:34.302963] [INFO ] [MAIN ] AMTM: Fetched primary data is valid. [2025-10-20T17:13:34.308288] [DEBUG] [LOCAL ] AMTM: Saving primary data. [2025-10-20T17:13:34.320756] [INFO ] [REMOTE ] AN: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:13:34.326956] [INFO ] [REMOTE ] AN: Full fetch complete (47093 records). [2025-10-20T17:13:34.327129] [DEBUG] [VALIDATOR ] AN: Validating primary data. [2025-10-20T17:13:34.327230] [DEBUG] [LOCAL ] AMTM: Trimmed 73747 records to 50401 (keeping 200 days of data) [2025-10-20T17:13:34.653571] [INFO ] [REMOTE ] AMX: Fetching primary data for 2025-10. [2025-10-20T17:13:34.978560] [DEBUG] [VALIDATOR ] AN: All expected trading days are present. [2025-10-20T17:13:34.980556] [DEBUG] [VALIDATOR ] AN: Primary data validation passed. [2025-10-20T17:13:34.982273] [INFO ] [MAIN ] AN: Fetched primary data is valid. [2025-10-20T17:13:34.982347] [DEBUG] [LOCAL ] AN: Saving primary data. [2025-10-20T17:13:34.995589] [DEBUG] [LOCAL ] AN: Trimmed 47093 records to 31264 (keeping 200 days of data) [2025-10-20T17:13:35.147474] [DEBUG] [LOCAL ] AMTM: Primary data saved (50401 records). [2025-10-20T17:13:35.151966] [INFO ] [MAIN ] AMTM: Full history saved. [2025-10-20T17:13:35.152366] [INFO ] [MAIN ] Update Progress: 114/2367 symbols completed. AMTM took 46.1s. Estimated remaining: 4h 27m 3s. Rate: 8.4 symbols/min [2025-10-20T17:13:35.347060] [INFO ] [MAIN ] ANF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:35.444750] [DEBUG] [LOCAL ] AN: Primary data saved (31264 records). [2025-10-20T17:13:35.457951] [INFO ] [MAIN ] AN: Full history saved. [2025-10-20T17:13:35.458962] [INFO ] [MAIN ] Update Progress: 115/2367 symbols completed. AN took 23.4s. Estimated remaining: 4h 24m 43s. Rate: 8.5 symbols/min [2025-10-20T17:13:35.593362] [DEBUG] [LOCAL ] ANF: Loaded 52835 existing records from primary storage. [2025-10-20T17:13:35.609144] [INFO ] [MAIN ] ANF: Loaded 52835 existing records. [2025-10-20T17:13:35.767843] [INFO ] [MAIN ] ANGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:35.768083] [DEBUG] [LOCAL ] ANGX: No existing data file found in primary storage. [2025-10-20T17:13:35.774000] [WARN ] [MAIN ] ANGX: No existing data found, performing full refresh instead. [2025-10-20T17:13:35.807606] [INFO ] [MAIN ] ANGX: Wiping existing primary data before full refresh. [2025-10-20T17:13:35.820783] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-10-20T17:13:35.829281] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-10-20T17:13:35.850437] [WARN ] [MAIN ] ANF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:35.850502] [INFO ] [MAIN ] ANGX: Starting full primary refresh. [2025-10-20T17:13:35.850732] [INFO ] [MAIN ] ANF: Wiping existing primary data before full refresh. [2025-10-20T17:13:35.850861] [INFO ] [LOCAL ] ANF: Starting deletion of primary data. [2025-10-20T17:13:35.926768] [DEBUG] [LOCAL ] ANF: Deleted primary data file. [2025-10-20T17:13:35.943704] [INFO ] [MAIN ] ANF: Starting full primary refresh. [2025-10-20T17:13:36.636871] [INFO ] [REMOTE ] ANGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:36.661981] [INFO ] [REMOTE ] ANGX: Fetching primary data for 2025-01. [2025-10-20T17:13:36.673799] [INFO ] [REMOTE ] ANF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:36.683415] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-01. [2025-10-20T17:13:36.697377] [INFO ] [REMOTE ] AMX: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T17:13:36.705452] [INFO ] [REMOTE ] AMX: Full fetch complete (71036 records). [2025-10-20T17:13:36.705556] [DEBUG] [VALIDATOR ] AMX: Validating primary data. [2025-10-20T17:13:36.962401] [DEBUG] [REMOTE ] ANGX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:13:37.035538] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-04 (3.18 seconds). [2025-10-20T17:13:37.041945] [WARN ] [REMOTE ] ANGX: No data returned for month 2025-01 [2025-10-20T17:13:37.055416] [FATAL] [REMOTE ] ANGX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:13:37.075809] [FATAL] [MAIN ] ANGX: Failed to fetch primary data. [2025-10-20T17:13:37.076379] [WARN ] [MAIN ] ANGX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:13:37.081966] [INFO ] [LOCAL ] ANGX: Starting deletion of primary data. [2025-10-20T17:13:37.087279] [DEBUG] [LOCAL ] ANGX: No primary data file found to delete. [2025-10-20T17:13:37.106682] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-05. [2025-10-20T17:13:37.143809] [INFO ] [LOCAL ] ANGX: Starting deletion of derivative data. [2025-10-20T17:13:37.165888] [DEBUG] [VALIDATOR ] AMX: All expected trading days are present. [2025-10-20T17:13:37.161832] [DEBUG] [LOCAL ] ANGX: No derivative data file found to delete. [2025-10-20T17:13:37.166206] [INFO ] [MAIN ] Update Progress: 116/2367 symbols completed. ANGX took 1.7s. Estimated remaining: 4h 22m 52s. Rate: 8.6 symbols/min [2025-10-20T17:13:37.166279] [DEBUG] [VALIDATOR ] AMX: Primary data validation passed. [2025-10-20T17:13:37.167241] [INFO ] [MAIN ] AMX: Fetched primary data is valid. [2025-10-20T17:13:37.167320] [DEBUG] [LOCAL ] AMX: Saving primary data. [2025-10-20T17:13:37.273796] [DEBUG] [LOCAL ] AMX: Trimmed 71036 records to 49207 (keeping 200 days of data) [2025-10-20T17:13:37.442531] [INFO ] [MAIN ] ANRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:13:37.467579] [DEBUG] [LOCAL ] ANRO: Loaded 19753 existing records from primary storage. [2025-10-20T17:13:37.468621] [INFO ] [MAIN ] ANRO: Loaded 19753 existing records. [2025-10-20T17:13:37.674902] [WARN ] [MAIN ] ANRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:13:37.677850] [INFO ] [MAIN ] ANRO: Wiping existing primary data before full refresh. [2025-10-20T17:13:37.677925] [INFO ] [LOCAL ] ANRO: Starting deletion of primary data. [2025-10-20T17:13:37.678119] [DEBUG] [LOCAL ] ANRO: Deleted primary data file. [2025-10-20T17:13:37.678160] [INFO ] [MAIN ] ANRO: Starting full primary refresh. [2025-10-20T17:13:38.212322] [DEBUG] [LOCAL ] AMX: Primary data saved (49207 records). [2025-10-20T17:13:38.227544] [INFO ] [MAIN ] AMX: Full history saved. [2025-10-20T17:13:38.245808] [INFO ] [MAIN ] Update Progress: 117/2367 symbols completed. AMX took 42.6s. Estimated remaining: 4h 20m 51s. Rate: 8.6 symbols/min [2025-10-20T17:13:38.251450] [INFO ] [REMOTE ] ANRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:13:38.251797] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-01. [2025-10-20T17:13:38.491189] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:13:38.651381] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-02. [2025-10-20T17:13:39.117063] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T17:13:39.307894] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-10-20T17:13:39.451373] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-06. [2025-10-20T17:13:40.251514] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-02. [2025-10-20T17:13:40.447163] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T17:13:41.051481] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-03. [2025-10-20T17:13:41.352333] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:13:41.447706] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:13:41.851450] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-03. [2025-10-20T17:13:42.651377] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-07. [2025-10-20T17:13:42.946005] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:13:43.045781] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:13:43.451443] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-04. [2025-10-20T17:13:44.251414] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-04. [2025-10-20T17:13:44.681996] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T17:13:45.051401] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-08. [2025-10-20T17:13:45.288600] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T17:13:45.343717] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:13:45.851442] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-05. [2025-10-20T17:13:46.651400] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-05. [2025-10-20T17:13:46.724133] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:13:47.451399] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-09. [2025-10-20T17:13:47.706995] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:13:47.747002] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T17:13:48.251418] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-06. [2025-10-20T17:13:49.051414] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-06. [2025-10-20T17:13:49.524226] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T17:13:49.851365] [INFO ] [REMOTE ] ANET: Fetching primary data for 2025-10. [2025-10-20T17:13:50.048066] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T17:13:50.279285] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:13:50.519361] [INFO ] [REMOTE ] ANET: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:13:50.527400] [INFO ] [REMOTE ] ANET: Full fetch complete (77945 records). [2025-10-20T17:13:50.527575] [DEBUG] [VALIDATOR ] ANET: Validating primary data. [2025-10-20T17:13:50.656527] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-07. [2025-10-20T17:13:50.951719] [DEBUG] [VALIDATOR ] ANET: All expected trading days are present. [2025-10-20T17:13:50.951873] [DEBUG] [VALIDATOR ] ANET: Primary data validation passed. [2025-10-20T17:13:50.953042] [INFO ] [MAIN ] ANET: Fetched primary data is valid. [2025-10-20T17:13:50.953119] [DEBUG] [LOCAL ] ANET: Saving primary data. [2025-10-20T17:13:50.955412] [DEBUG] [LOCAL ] ANET: Trimmed 77945 records to 53766 (keeping 200 days of data) [2025-10-20T17:13:51.291982] [DEBUG] [LOCAL ] ANET: Primary data saved (53766 records). [2025-10-20T17:13:51.292174] [INFO ] [MAIN ] ANET: Full history saved. [2025-10-20T17:13:51.292274] [INFO ] [MAIN ] Update Progress: 118/2367 symbols completed. ANET took 33.8s. Estimated remaining: 4h 22m 40s. Rate: 8.6 symbols/min [2025-10-20T17:13:51.451461] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-07. [2025-10-20T17:13:51.730521] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:13:52.251432] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-08. [2025-10-20T17:13:53.160162] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T17:13:53.164366] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-09. [2025-10-20T17:13:53.291882] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:13:53.662303] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:13:53.851398] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-08. [2025-10-20T17:13:54.651412] [INFO ] [REMOTE ] ANRO: Fetching primary data for 2025-10. [2025-10-20T17:13:55.839770] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T17:13:55.851001] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-09. [2025-10-20T17:13:55.931410] [INFO ] [REMOTE ] ANRO: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:13:55.936304] [INFO ] [REMOTE ] ANRO: Full fetch complete (30045 records). [2025-10-20T17:13:55.936399] [DEBUG] [VALIDATOR ] ANRO: Validating primary data. [2025-10-20T17:13:56.353817] [DEBUG] [VALIDATOR ] ANRO: All expected trading days are present. [2025-10-20T17:13:56.353923] [DEBUG] [VALIDATOR ] ANRO: Primary data validation passed. [2025-10-20T17:13:56.354397] [INFO ] [MAIN ] ANRO: Fetched primary data is valid. [2025-10-20T17:13:56.354442] [DEBUG] [LOCAL ] ANRO: Saving primary data. [2025-10-20T17:13:56.355346] [DEBUG] [LOCAL ] ANRO: Trimmed 30045 records to 19540 (keeping 200 days of data) [2025-10-20T17:13:56.469986] [DEBUG] [LOCAL ] ANRO: Primary data saved (19540 records). [2025-10-20T17:13:56.470189] [INFO ] [MAIN ] ANRO: Full history saved. [2025-10-20T17:13:56.470324] [INFO ] [MAIN ] Update Progress: 119/2367 symbols completed. ANRO took 19.3s. Estimated remaining: 4h 21m 58s. Rate: 8.6 symbols/min [2025-10-20T17:13:57.598673] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T17:13:57.611425] [INFO ] [REMOTE ] ANF: Fetching primary data for 2025-10. [2025-10-20T17:13:59.074492] [INFO ] [REMOTE ] ANF: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:13:59.082070] [INFO ] [REMOTE ] ANF: Full fetch complete (75784 records). [2025-10-20T17:13:59.082197] [DEBUG] [VALIDATOR ] ANF: Validating primary data. [2025-10-20T17:13:59.509764] [DEBUG] [VALIDATOR ] ANF: All expected trading days are present. [2025-10-20T17:13:59.509854] [DEBUG] [VALIDATOR ] ANF: Primary data validation passed. [2025-10-20T17:13:59.510574] [INFO ] [MAIN ] ANF: Fetched primary data is valid. [2025-10-20T17:13:59.510613] [DEBUG] [LOCAL ] ANF: Saving primary data. [2025-10-20T17:13:59.511986] [DEBUG] [LOCAL ] ANF: Trimmed 75784 records to 52054 (keeping 200 days of data) [2025-10-20T17:13:59.811570] [DEBUG] [LOCAL ] ANF: Primary data saved (52054 records). [2025-10-20T17:13:59.811743] [INFO ] [MAIN ] ANF: Full history saved. [2025-10-20T17:13:59.811828] [INFO ] [MAIN ] Update Progress: 120/2367 symbols completed. ANF took 24.7s. Estimated remaining: 4h 20m 43s. Rate: 8.6 symbols/min [2025-10-20T17:14:00.556193] [INFO ] [MAIN ] AOMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.553082] [INFO ] [MAIN ] ANVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.575211] [INFO ] [MAIN ] AOMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.598301] [DEBUG] [LOCAL ] AOMD: No existing data file found in primary storage. [2025-10-20T17:14:00.598958] [WARN ] [MAIN ] AOMD: No existing data found, performing full refresh instead. [2025-10-20T17:14:00.603622] [INFO ] [MAIN ] AOMD: Wiping existing primary data before full refresh. [2025-10-20T17:14:00.603545] [INFO ] [MAIN ] AOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:00.603786] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-10-20T17:14:00.604256] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-10-20T17:14:00.604788] [INFO ] [MAIN ] AOMD: Starting full primary refresh. [2025-10-20T17:14:00.610148] [DEBUG] [LOCAL ] AOD: No existing data file found in primary storage. [2025-10-20T17:14:00.635986] [WARN ] [MAIN ] AOD: No existing data found, performing full refresh instead. [2025-10-20T17:14:00.682161] [INFO ] [MAIN ] AOD: Wiping existing primary data before full refresh. [2025-10-20T17:14:00.694191] [INFO ] [LOCAL ] AOD: Starting deletion of primary data. [2025-10-20T17:14:00.699467] [DEBUG] [LOCAL ] AOD: No primary data file found to delete. [2025-10-20T17:14:00.699751] [INFO ] [MAIN ] AOD: Starting full primary refresh. [2025-10-20T17:14:00.768941] [DEBUG] [LOCAL ] AOMR: Loaded 16681 existing records from primary storage. [2025-10-20T17:14:00.825796] [INFO ] [MAIN ] AOMR: Loaded 16681 existing records. [2025-10-20T17:14:00.842073] [DEBUG] [LOCAL ] ANVS: Loaded 24303 existing records from primary storage. [2025-10-20T17:14:00.902253] [INFO ] [MAIN ] ANVS: Loaded 24303 existing records. [2025-10-20T17:14:01.609853] [WARN ] [MAIN ] ANVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:01.660492] [INFO ] [REMOTE ] AOMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:01.691893] [INFO ] [REMOTE ] AOMD: Fetching primary data for 2025-01. [2025-10-20T17:14:01.692004] [INFO ] [MAIN ] ANVS: Wiping existing primary data before full refresh. [2025-10-20T17:14:01.753665] [WARN ] [MAIN ] AOMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:01.755063] [INFO ] [LOCAL ] ANVS: Starting deletion of primary data. [2025-10-20T17:14:01.755364] [INFO ] [MAIN ] AOMR: Wiping existing primary data before full refresh. [2025-10-20T17:14:01.755438] [DEBUG] [LOCAL ] ANVS: Deleted primary data file. [2025-10-20T17:14:01.755547] [INFO ] [LOCAL ] AOMR: Starting deletion of primary data. [2025-10-20T17:14:01.768710] [INFO ] [MAIN ] ANVS: Starting full primary refresh. [2025-10-20T17:14:01.811137] [DEBUG] [LOCAL ] AOMR: Deleted primary data file. [2025-10-20T17:14:01.817438] [INFO ] [MAIN ] AOMR: Starting full primary refresh. [2025-10-20T17:14:02.053983] [INFO ] [REMOTE ] AOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.098721] [DEBUG] [REMOTE ] AOMD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:14:02.098276] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-01. [2025-10-20T17:14:02.120216] [WARN ] [REMOTE ] AOMD: No data returned for month 2025-01 [2025-10-20T17:14:02.143922] [FATAL] [REMOTE ] AOMD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:14:02.149674] [FATAL] [MAIN ] AOMD: Failed to fetch primary data. [2025-10-20T17:14:02.161894] [WARN ] [MAIN ] AOMD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:14:02.222285] [INFO ] [LOCAL ] AOMD: Starting deletion of primary data. [2025-10-20T17:14:02.307615] [DEBUG] [LOCAL ] AOMD: No primary data file found to delete. [2025-10-20T17:14:02.330928] [INFO ] [LOCAL ] AOMD: Starting deletion of derivative data. [2025-10-20T17:14:02.398548] [DEBUG] [LOCAL ] AOMD: No derivative data file found to delete. [2025-10-20T17:14:02.437677] [INFO ] [MAIN ] Update Progress: 121/2367 symbols completed. AOMD took 2.6s. Estimated remaining: 4h 19m 16s. Rate: 8.7 symbols/min [2025-10-20T17:14:02.813902] [INFO ] [REMOTE ] ANVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.868078] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-01. [2025-10-20T17:14:02.990503] [INFO ] [REMOTE ] AOMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:02.985986] [INFO ] [MAIN ] AON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:02.990919] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-01. [2025-10-20T17:14:03.037100] [DEBUG] [LOCAL ] AON: Loaded 45813 existing records from primary storage. [2025-10-20T17:14:03.037538] [INFO ] [MAIN ] AON: Loaded 45813 existing records. [2025-10-20T17:14:03.225543] [WARN ] [MAIN ] AON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:03.225863] [INFO ] [MAIN ] AON: Wiping existing primary data before full refresh. [2025-10-20T17:14:03.225920] [INFO ] [LOCAL ] AON: Starting deletion of primary data. [2025-10-20T17:14:03.226181] [DEBUG] [LOCAL ] AON: Deleted primary data file. [2025-10-20T17:14:03.226275] [INFO ] [MAIN ] AON: Starting full primary refresh. [2025-10-20T17:14:03.597880] [INFO ] [REMOTE ] AON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:03.598264] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-01. [2025-10-20T17:14:03.686084] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:14:03.693798] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-02. [2025-10-20T17:14:03.926620] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:14:03.929395] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-02. [2025-10-20T17:14:04.315370] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:14:04.324543] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-02. [2025-10-20T17:14:04.829599] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:14:04.836455] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-03. [2025-10-20T17:14:04.876034] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:14:04.883000] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-03. [2025-10-20T17:14:05.043205] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:14:05.052596] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-02. [2025-10-20T17:14:05.323624] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:14:05.858387] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-03. [2025-10-20T17:14:05.865166] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T17:14:05.952276] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:14:06.590098] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:14:06.651358] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-04. [2025-10-20T17:14:06.715606] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T17:14:07.451432] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-04. [2025-10-20T17:14:07.708507] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T17:14:08.251398] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-05. [2025-10-20T17:14:08.590782] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T17:14:09.051363] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-05. [2025-10-20T17:14:09.288032] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:14:09.851389] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-04. [2025-10-20T17:14:10.266157] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:14:10.651415] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-03. [2025-10-20T17:14:11.051226] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T17:14:11.451438] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-06. [2025-10-20T17:14:12.180035] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T17:14:12.251390] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-06. [2025-10-20T17:14:12.635871] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:14:12.894277] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:14:13.051427] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-07. [2025-10-20T17:14:13.851446] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-04. [2025-10-20T17:14:14.316406] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T17:14:14.651432] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-05. [2025-10-20T17:14:15.451399] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-07. [2025-10-20T17:14:15.703710] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T17:14:15.832114] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:14:16.251361] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-05. [2025-10-20T17:14:16.402125] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:14:17.051414] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-08. [2025-10-20T17:14:17.851407] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-08. [2025-10-20T17:14:17.911541] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T17:14:18.427529] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:14:18.651367] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-09. [2025-10-20T17:14:19.068847] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T17:14:19.451430] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-09. [2025-10-20T17:14:20.197673] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T17:14:20.251399] [INFO ] [REMOTE ] AOD: Fetching primary data for 2025-10. [2025-10-20T17:14:20.486577] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:14:21.051423] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-06. [2025-10-20T17:14:21.733149] [INFO ] [REMOTE ] AOD: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:14:21.738286] [INFO ] [REMOTE ] AOD: Full fetch complete (44782 records). [2025-10-20T17:14:21.738366] [DEBUG] [VALIDATOR ] AOD: Validating primary data. [2025-10-20T17:14:21.856544] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-06. [2025-10-20T17:14:22.135290] [DEBUG] [VALIDATOR ] AOD: All expected trading days are present. [2025-10-20T17:14:22.135546] [DEBUG] [VALIDATOR ] AOD: Primary data validation passed. [2025-10-20T17:14:22.136240] [INFO ] [MAIN ] AOD: Fetched primary data is valid. [2025-10-20T17:14:22.136362] [DEBUG] [LOCAL ] AOD: Saving primary data. [2025-10-20T17:14:22.157069] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:14:22.164478] [DEBUG] [LOCAL ] AOD: Trimmed 44782 records to 30963 (keeping 200 days of data) [2025-10-20T17:14:22.349582] [DEBUG] [LOCAL ] AOD: Primary data saved (30963 records). [2025-10-20T17:14:22.349752] [INFO ] [MAIN ] AOD: Full history saved. [2025-10-20T17:14:22.349831] [INFO ] [MAIN ] Update Progress: 122/2367 symbols completed. AOD took 22.5s. Estimated remaining: 4h 23m 8s. Rate: 8.5 symbols/min [2025-10-20T17:14:22.520703] [INFO ] [MAIN ] AORT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:22.549753] [DEBUG] [LOCAL ] AORT: Loaded 29531 existing records from primary storage. [2025-10-20T17:14:22.550048] [INFO ] [MAIN ] AORT: Loaded 29531 existing records. [2025-10-20T17:14:22.656474] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-07. [2025-10-20T17:14:22.729860] [WARN ] [MAIN ] AORT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:22.730131] [INFO ] [MAIN ] AORT: Wiping existing primary data before full refresh. [2025-10-20T17:14:22.730213] [INFO ] [LOCAL ] AORT: Starting deletion of primary data. [2025-10-20T17:14:22.730390] [DEBUG] [LOCAL ] AORT: Deleted primary data file. [2025-10-20T17:14:22.730429] [INFO ] [MAIN ] AORT: Starting full primary refresh. [2025-10-20T17:14:23.083727] [INFO ] [REMOTE ] AORT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:23.451365] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-01. [2025-10-20T17:14:23.530474] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:14:23.636311] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:14:24.251416] [INFO ] [REMOTE ] AOMR: Fetching primary data for 2025-10. [2025-10-20T17:14:24.775599] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:14:25.051450] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-02. [2025-10-20T17:14:25.163636] [INFO ] [REMOTE ] AOMR: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:14:25.166118] [INFO ] [REMOTE ] AOMR: Full fetch complete (23200 records). [2025-10-20T17:14:25.166230] [DEBUG] [VALIDATOR ] AOMR: Validating primary data. [2025-10-20T17:14:25.563799] [DEBUG] [VALIDATOR ] AOMR: All expected trading days are present. [2025-10-20T17:14:25.563903] [DEBUG] [VALIDATOR ] AOMR: Primary data validation passed. [2025-10-20T17:14:25.564311] [INFO ] [MAIN ] AOMR: Fetched primary data is valid. [2025-10-20T17:14:25.564380] [DEBUG] [LOCAL ] AOMR: Saving primary data. [2025-10-20T17:14:25.565363] [DEBUG] [LOCAL ] AOMR: Trimmed 23200 records to 16433 (keeping 200 days of data) [2025-10-20T17:14:25.666523] [DEBUG] [LOCAL ] AOMR: Primary data saved (16433 records). [2025-10-20T17:14:25.666701] [INFO ] [MAIN ] AOMR: Full history saved. [2025-10-20T17:14:25.666794] [INFO ] [MAIN ] Update Progress: 123/2367 symbols completed. AOMR took 25.8s. Estimated remaining: 4h 21m 53s. Rate: 8.6 symbols/min [2025-10-20T17:14:25.833374] [INFO ] [MAIN ] AOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:25.856546] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-07. [2025-10-20T17:14:25.878035] [DEBUG] [LOCAL ] AOS: Loaded 49475 existing records from primary storage. [2025-10-20T17:14:25.878336] [INFO ] [MAIN ] AOS: Loaded 49475 existing records. [2025-10-20T17:14:26.071546] [WARN ] [MAIN ] AOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:26.071829] [INFO ] [MAIN ] AOS: Wiping existing primary data before full refresh. [2025-10-20T17:14:26.071907] [INFO ] [LOCAL ] AOS: Starting deletion of primary data. [2025-10-20T17:14:26.072146] [DEBUG] [LOCAL ] AOS: Deleted primary data file. [2025-10-20T17:14:26.072219] [INFO ] [MAIN ] AOS: Starting full primary refresh. [2025-10-20T17:14:26.464979] [INFO ] [REMOTE ] AOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:26.485034] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T17:14:26.651429] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-08. [2025-10-20T17:14:27.451348] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-01. [2025-10-20T17:14:27.634915] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:14:27.668031] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:14:28.251444] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-03. [2025-10-20T17:14:29.051362] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-08. [2025-10-20T17:14:29.175396] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:14:29.589424] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T17:14:29.851366] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-02. [2025-10-20T17:14:30.651415] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-09. [2025-10-20T17:14:30.766617] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T17:14:31.361831] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T17:14:31.418332] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T17:14:31.451417] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-03. [2025-10-20T17:14:32.251395] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-09. [2025-10-20T17:14:33.051406] [INFO ] [REMOTE ] ANVS: Fetching primary data for 2025-10. [2025-10-20T17:14:33.158873] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:14:33.851378] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-04. [2025-10-20T17:14:33.887634] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T17:14:34.475285] [INFO ] [REMOTE ] ANVS: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:14:34.480138] [INFO ] [REMOTE ] ANVS: Full fetch complete (36901 records). [2025-10-20T17:14:34.480215] [DEBUG] [VALIDATOR ] ANVS: Validating primary data. [2025-10-20T17:14:34.658564] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-04. [2025-10-20T17:14:34.860158] [DEBUG] [VALIDATOR ] ANVS: All expected trading days are present. [2025-10-20T17:14:34.860325] [DEBUG] [VALIDATOR ] ANVS: Primary data validation passed. [2025-10-20T17:14:34.861035] [INFO ] [MAIN ] ANVS: Fetched primary data is valid. [2025-10-20T17:14:34.861106] [DEBUG] [LOCAL ] ANVS: Saving primary data. [2025-10-20T17:14:34.862576] [DEBUG] [LOCAL ] ANVS: Trimmed 36901 records to 24001 (keeping 200 days of data) [2025-10-20T17:14:35.003863] [DEBUG] [LOCAL ] ANVS: Primary data saved (24001 records). [2025-10-20T17:14:35.004067] [INFO ] [MAIN ] ANVS: Full history saved. [2025-10-20T17:14:35.004190] [INFO ] [MAIN ] Update Progress: 124/2367 symbols completed. ANVS took 35.2s. Estimated remaining: 4h 22m 28s. Rate: 8.5 symbols/min [2025-10-20T17:14:35.252353] [INFO ] [MAIN ] AP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:35.252705] [DEBUG] [LOCAL ] AP: No existing data file found in primary storage. [2025-10-20T17:14:35.252764] [WARN ] [MAIN ] AP: No existing data found, performing full refresh instead. [2025-10-20T17:14:35.252886] [INFO ] [MAIN ] AP: Wiping existing primary data before full refresh. [2025-10-20T17:14:35.252929] [INFO ] [LOCAL ] AP: Starting deletion of primary data. [2025-10-20T17:14:35.252971] [DEBUG] [LOCAL ] AP: No primary data file found to delete. [2025-10-20T17:14:35.253005] [INFO ] [MAIN ] AP: Starting full primary refresh. [2025-10-20T17:14:35.457531] [INFO ] [REMOTE ] AON: Fetching primary data for 2025-10. [2025-10-20T17:14:35.667894] [INFO ] [REMOTE ] AP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:35.698413] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T17:14:36.193904] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T17:14:36.251418] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-05. [2025-10-20T17:14:37.051346] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-01. [2025-10-20T17:14:37.143965] [INFO ] [REMOTE ] AON: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:14:37.150581] [INFO ] [REMOTE ] AON: Full fetch complete (65161 records). [2025-10-20T17:14:37.150736] [DEBUG] [VALIDATOR ] AON: Validating primary data. [2025-10-20T17:14:37.560940] [DEBUG] [VALIDATOR ] AON: All expected trading days are present. [2025-10-20T17:14:37.561093] [DEBUG] [VALIDATOR ] AON: Primary data validation passed. [2025-10-20T17:14:37.561848] [INFO ] [MAIN ] AON: Fetched primary data is valid. [2025-10-20T17:14:37.561928] [DEBUG] [LOCAL ] AON: Saving primary data. [2025-10-20T17:14:37.566967] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:14:37.568931] [DEBUG] [LOCAL ] AON: Trimmed 65161 records to 45175 (keeping 200 days of data) [2025-10-20T17:14:37.683111] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T17:14:37.852222] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-02. [2025-10-20T17:14:37.869059] [DEBUG] [LOCAL ] AON: Primary data saved (45175 records). [2025-10-20T17:14:37.869157] [INFO ] [MAIN ] AON: Full history saved. [2025-10-20T17:14:37.869230] [INFO ] [MAIN ] Update Progress: 125/2367 symbols completed. AON took 35.4s. Estimated remaining: 4h 21m 6s. Rate: 8.6 symbols/min [2025-10-20T17:14:38.047976] [INFO ] [MAIN ] APAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:38.083257] [DEBUG] [LOCAL ] APAM: Loaded 36578 existing records from primary storage. [2025-10-20T17:14:38.083605] [INFO ] [MAIN ] APAM: Loaded 36578 existing records. [2025-10-20T17:14:38.273604] [WARN ] [MAIN ] APAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:38.273849] [INFO ] [MAIN ] APAM: Wiping existing primary data before full refresh. [2025-10-20T17:14:38.273897] [INFO ] [LOCAL ] APAM: Starting deletion of primary data. [2025-10-20T17:14:38.274134] [DEBUG] [LOCAL ] APAM: Deleted primary data file. [2025-10-20T17:14:38.274173] [INFO ] [MAIN ] APAM: Starting full primary refresh. [2025-10-20T17:14:38.522304] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T17:14:38.647739] [INFO ] [REMOTE ] APAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:38.651339] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-01. [2025-10-20T17:14:39.451371] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-03. [2025-10-20T17:14:39.777880] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T17:14:40.031671] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:14:40.251368] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-06. [2025-10-20T17:14:41.051353] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-02. [2025-10-20T17:14:41.444474] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:14:41.851372] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-05. [2025-10-20T17:14:42.446123] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:14:42.651377] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-03. [2025-10-20T17:14:43.451390] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-04. [2025-10-20T17:14:43.530253] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:14:44.088329] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:14:44.114464] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T17:14:44.251358] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-04. [2025-10-20T17:14:45.051385] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-07. [2025-10-20T17:14:45.734698] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:14:45.851369] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-05. [2025-10-20T17:14:46.386293] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:14:46.651384] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-06. [2025-10-20T17:14:47.304095] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:14:47.451365] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-05. [2025-10-20T17:14:48.040902] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T17:14:48.251343] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-06. [2025-10-20T17:14:48.306946] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:14:48.990217] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T17:14:49.051338] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-07. [2025-10-20T17:14:49.851426] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-08. [2025-10-20T17:14:49.879859] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T17:14:50.651386] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-07. [2025-10-20T17:14:51.226463] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:14:51.451406] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-06. [2025-10-20T17:14:52.251359] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-08. [2025-10-20T17:14:52.284809] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:14:52.735324] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T17:14:52.919273] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:14:53.051344] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-08. [2025-10-20T17:14:53.851389] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-07. [2025-10-20T17:14:54.492589] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:14:54.651432] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-09. [2025-10-20T17:14:55.274012] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:14:55.398435] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T17:14:55.451334] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-09. [2025-10-20T17:14:56.251380] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-08. [2025-10-20T17:14:57.056456] [INFO ] [REMOTE ] AP: Fetching primary data for 2025-10. [2025-10-20T17:14:57.068042] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T17:14:57.161164] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T17:14:57.855186] [INFO ] [REMOTE ] AP: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T17:14:57.854352] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-09. [2025-10-20T17:14:57.857406] [INFO ] [REMOTE ] AP: Full fetch complete (9046 records). [2025-10-20T17:14:57.857581] [DEBUG] [VALIDATOR ] AP: Validating primary data. [2025-10-20T17:14:58.212470] [DEBUG] [VALIDATOR ] AP: All expected trading days are present. [2025-10-20T17:14:58.212546] [DEBUG] [VALIDATOR ] AP: Primary data validation passed. [2025-10-20T17:14:58.212772] [INFO ] [MAIN ] AP: Fetched primary data is valid. [2025-10-20T17:14:58.212879] [DEBUG] [LOCAL ] AP: Saving primary data. [2025-10-20T17:14:58.213386] [DEBUG] [LOCAL ] AP: Trimmed 9046 records to 7408 (keeping 200 days of data) [2025-10-20T17:14:58.254599] [DEBUG] [LOCAL ] AP: Primary data saved (7408 records). [2025-10-20T17:14:58.254727] [INFO ] [MAIN ] AP: Full history saved. [2025-10-20T17:14:58.254800] [INFO ] [MAIN ] Update Progress: 126/2367 symbols completed. AP took 23.3s. Estimated remaining: 4h 24m 58s. Rate: 8.5 symbols/min [2025-10-20T17:14:58.428914] [INFO ] [MAIN ] APD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:14:58.477472] [DEBUG] [LOCAL ] APD: Loaded 46187 existing records from primary storage. [2025-10-20T17:14:58.477903] [INFO ] [MAIN ] APD: Loaded 46187 existing records. [2025-10-20T17:14:58.651790] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-09. [2025-10-20T17:14:58.680374] [WARN ] [MAIN ] APD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:14:58.680887] [INFO ] [MAIN ] APD: Wiping existing primary data before full refresh. [2025-10-20T17:14:58.680950] [INFO ] [LOCAL ] APD: Starting deletion of primary data. [2025-10-20T17:14:58.681220] [DEBUG] [LOCAL ] APD: Deleted primary data file. [2025-10-20T17:14:58.681277] [INFO ] [MAIN ] APD: Starting full primary refresh. [2025-10-20T17:14:59.068773] [INFO ] [REMOTE ] APD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:14:59.307563] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:14:59.451360] [INFO ] [REMOTE ] APAM: Fetching primary data for 2025-10. [2025-10-20T17:15:00.079911] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:15:00.251364] [INFO ] [REMOTE ] AORT: Fetching primary data for 2025-10. [2025-10-20T17:15:00.689184] [INFO ] [REMOTE ] APAM: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T17:15:00.694805] [INFO ] [REMOTE ] APAM: Full fetch complete (52123 records). [2025-10-20T17:15:00.694942] [DEBUG] [VALIDATOR ] APAM: Validating primary data. [2025-10-20T17:15:01.072444] [INFO ] [REMOTE ] AOS: Fetching primary data for 2025-10. [2025-10-20T17:15:01.090924] [DEBUG] [VALIDATOR ] APAM: All expected trading days are present. [2025-10-20T17:15:01.091003] [DEBUG] [VALIDATOR ] APAM: Primary data validation passed. [2025-10-20T17:15:01.091578] [INFO ] [MAIN ] APAM: Fetched primary data is valid. [2025-10-20T17:15:01.091620] [DEBUG] [LOCAL ] APAM: Saving primary data. [2025-10-20T17:15:01.093189] [DEBUG] [LOCAL ] APAM: Trimmed 52123 records to 36007 (keeping 200 days of data) [2025-10-20T17:15:01.315925] [DEBUG] [LOCAL ] APAM: Primary data saved (36007 records). [2025-10-20T17:15:01.316145] [INFO ] [MAIN ] APAM: Full history saved. [2025-10-20T17:15:01.316737] [INFO ] [MAIN ] Update Progress: 127/2367 symbols completed. APAM took 23.4s. Estimated remaining: 4h 23m 40s. Rate: 8.5 symbols/min [2025-10-20T17:15:01.508263] [INFO ] [MAIN ] APG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:01.532373] [INFO ] [REMOTE ] AORT: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:15:01.537006] [INFO ] [REMOTE ] AORT: Full fetch complete (42377 records). [2025-10-20T17:15:01.537168] [DEBUG] [VALIDATOR ] AORT: Validating primary data. [2025-10-20T17:15:01.857815] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-01. [2025-10-20T17:15:01.920989] [DEBUG] [LOCAL ] APG: Loaded 51343 existing records from primary storage. [2025-10-20T17:15:01.930415] [INFO ] [MAIN ] APG: Loaded 51343 existing records. [2025-10-20T17:15:02.167081] [DEBUG] [VALIDATOR ] AORT: All expected trading days are present. [2025-10-20T17:15:02.167227] [DEBUG] [VALIDATOR ] AORT: Primary data validation passed. [2025-10-20T17:15:02.167870] [INFO ] [MAIN ] AORT: Fetched primary data is valid. [2025-10-20T17:15:02.167957] [DEBUG] [LOCAL ] AORT: Saving primary data. [2025-10-20T17:15:02.219821] [DEBUG] [LOCAL ] AORT: Trimmed 42377 records to 28981 (keeping 200 days of data) [2025-10-20T17:15:02.252901] [WARN ] [MAIN ] APG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:02.298955] [INFO ] [MAIN ] APG: Wiping existing primary data before full refresh. [2025-10-20T17:15:02.299030] [INFO ] [LOCAL ] APG: Starting deletion of primary data. [2025-10-20T17:15:02.334502] [DEBUG] [LOCAL ] APG: Deleted primary data file. [2025-10-20T17:15:02.335044] [INFO ] [MAIN ] APG: Starting full primary refresh. [2025-10-20T17:15:02.814437] [INFO ] [REMOTE ] APG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:02.817903] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-01. [2025-10-20T17:15:02.825441] [DEBUG] [LOCAL ] AORT: Primary data saved (28981 records). [2025-10-20T17:15:02.825538] [INFO ] [MAIN ] AORT: Full history saved. [2025-10-20T17:15:02.825669] [INFO ] [MAIN ] Update Progress: 128/2367 symbols completed. AORT took 40.5s. Estimated remaining: 4h 21m 55s. Rate: 8.5 symbols/min [2025-10-20T17:15:03.007848] [INFO ] [MAIN ] APH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:03.061323] [DEBUG] [LOCAL ] APH: Loaded 54471 existing records from primary storage. [2025-10-20T17:15:03.061755] [INFO ] [MAIN ] APH: Loaded 54471 existing records. [2025-10-20T17:15:03.269053] [WARN ] [MAIN ] APH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:03.280870] [INFO ] [MAIN ] APH: Wiping existing primary data before full refresh. [2025-10-20T17:15:03.281666] [INFO ] [LOCAL ] APH: Starting deletion of primary data. [2025-10-20T17:15:03.282621] [INFO ] [REMOTE ] AOS: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:15:03.289629] [DEBUG] [LOCAL ] APH: Deleted primary data file. [2025-10-20T17:15:03.289876] [INFO ] [MAIN ] APH: Starting full primary refresh. [2025-10-20T17:15:03.337522] [INFO ] [REMOTE ] AOS: Full fetch complete (71026 records). [2025-10-20T17:15:03.366825] [DEBUG] [VALIDATOR ] AOS: Validating primary data. [2025-10-20T17:15:04.085826] [INFO ] [REMOTE ] APH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:04.096807] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-01. [2025-10-20T17:15:04.186967] [DEBUG] [VALIDATOR ] AOS: All expected trading days are present. [2025-10-20T17:15:04.187141] [DEBUG] [VALIDATOR ] AOS: Primary data validation passed. [2025-10-20T17:15:04.188760] [INFO ] [MAIN ] AOS: Fetched primary data is valid. [2025-10-20T17:15:04.188912] [DEBUG] [LOCAL ] AOS: Saving primary data. [2025-10-20T17:15:04.191397] [DEBUG] [LOCAL ] AOS: Trimmed 71026 records to 48689 (keeping 200 days of data) [2025-10-20T17:15:04.311575] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-01 (2.45 seconds). [2025-10-20T17:15:04.343243] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-02. [2025-10-20T17:15:04.548174] [DEBUG] [LOCAL ] AOS: Primary data saved (48689 records). [2025-10-20T17:15:04.548370] [INFO ] [MAIN ] AOS: Full history saved. [2025-10-20T17:15:04.548499] [INFO ] [MAIN ] Update Progress: 129/2367 symbols completed. AOS took 38.9s. Estimated remaining: 4h 20m 16s. Rate: 8.6 symbols/min [2025-10-20T17:15:04.738443] [INFO ] [MAIN ] APLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:04.798936] [DEBUG] [LOCAL ] APLE: Loaded 52733 existing records from primary storage. [2025-10-20T17:15:04.799452] [INFO ] [MAIN ] APLE: Loaded 52733 existing records. [2025-10-20T17:15:05.007719] [WARN ] [MAIN ] APLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:05.028673] [INFO ] [MAIN ] APLE: Wiping existing primary data before full refresh. [2025-10-20T17:15:05.030356] [INFO ] [LOCAL ] APLE: Starting deletion of primary data. [2025-10-20T17:15:05.030203] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-01 (2.21 seconds). [2025-10-20T17:15:05.040069] [DEBUG] [LOCAL ] APLE: Deleted primary data file. [2025-10-20T17:15:05.040153] [INFO ] [MAIN ] APLE: Starting full primary refresh. [2025-10-20T17:15:05.073852] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-02. [2025-10-20T17:15:05.422851] [INFO ] [REMOTE ] APLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:05.851463] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-01. [2025-10-20T17:15:06.094837] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:15:06.197021] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:15:06.651355] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-02. [2025-10-20T17:15:06.758126] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:15:07.451442] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-03. [2025-10-20T17:15:07.708749] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:15:08.259949] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-03. [2025-10-20T17:15:08.265712] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:15:08.909387] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:15:09.051436] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-03. [2025-10-20T17:15:09.851374] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-04. [2025-10-20T17:15:10.078118] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:15:10.589798] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:15:10.651351] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-04. [2025-10-20T17:15:11.451395] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-04. [2025-10-20T17:15:11.599533] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:15:12.251431] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-05. [2025-10-20T17:15:12.634618] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:15:13.051438] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-05. [2025-10-20T17:15:13.105291] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:15:13.698801] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:15:13.851413] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-02. [2025-10-20T17:15:14.651354] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-06. [2025-10-20T17:15:15.052700] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:15:15.451419] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-05. [2025-10-20T17:15:15.509470] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:15:16.250052] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T17:15:16.259521] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-03. [2025-10-20T17:15:17.025685] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:15:17.051563] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-06. [2025-10-20T17:15:17.851541] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-06. [2025-10-20T17:15:18.278559] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:15:18.651350] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-07. [2025-10-20T17:15:19.005279] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T17:15:19.161780] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:15:19.451481] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-04. [2025-10-20T17:15:20.251361] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-07. [2025-10-20T17:15:20.451479] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:15:21.051397] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-08. [2025-10-20T17:15:21.419486] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:15:21.749928] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:15:21.851377] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-05. [2025-10-20T17:15:22.651397] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-08. [2025-10-20T17:15:22.752953] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:15:23.237970] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:15:23.451364] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-09. [2025-10-20T17:15:23.834236] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T17:15:24.251393] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-06. [2025-10-20T17:15:25.051492] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-09. [2025-10-20T17:15:25.130137] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:15:25.851411] [INFO ] [REMOTE ] APG: Fetching primary data for 2025-10. [2025-10-20T17:15:26.015815] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T17:15:26.651367] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-07. [2025-10-20T17:15:26.703305] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:15:27.451423] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-07. [2025-10-20T17:15:27.737928] [INFO ] [REMOTE ] APG: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T17:15:27.747408] [INFO ] [REMOTE ] APG: Full fetch complete (72197 records). [2025-10-20T17:15:27.747676] [DEBUG] [VALIDATOR ] APG: Validating primary data. [2025-10-20T17:15:28.155257] [DEBUG] [VALIDATOR ] APG: All expected trading days are present. [2025-10-20T17:15:28.155353] [DEBUG] [VALIDATOR ] APG: Primary data validation passed. [2025-10-20T17:15:28.156119] [INFO ] [MAIN ] APG: Fetched primary data is valid. [2025-10-20T17:15:28.156185] [DEBUG] [LOCAL ] APG: Saving primary data. [2025-10-20T17:15:28.157710] [DEBUG] [LOCAL ] APG: Trimmed 72197 records to 50628 (keeping 200 days of data) [2025-10-20T17:15:28.254061] [INFO ] [REMOTE ] APD: Fetching primary data for 2025-10. [2025-10-20T17:15:28.460072] [DEBUG] [LOCAL ] APG: Primary data saved (50628 records). [2025-10-20T17:15:28.460290] [INFO ] [MAIN ] APG: Full history saved. [2025-10-20T17:15:28.461008] [INFO ] [MAIN ] Update Progress: 130/2367 symbols completed. APG took 27.1s. Estimated remaining: 4h 25m 1s. Rate: 8.4 symbols/min [2025-10-20T17:15:28.672075] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:15:28.675046] [INFO ] [MAIN ] APO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:28.686065] [DEBUG] [LOCAL ] APO: No existing data file found in primary storage. [2025-10-20T17:15:28.686342] [WARN ] [MAIN ] APO: No existing data found, performing full refresh instead. [2025-10-20T17:15:28.686612] [INFO ] [MAIN ] APO: Wiping existing primary data before full refresh. [2025-10-20T17:15:28.686694] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-10-20T17:15:28.686757] [DEBUG] [LOCAL ] APO: No primary data file found to delete. [2025-10-20T17:15:28.687073] [INFO ] [MAIN ] APO: Starting full primary refresh. [2025-10-20T17:15:29.053527] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-08. [2025-10-20T17:15:29.060594] [INFO ] [REMOTE ] APO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:29.201185] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:15:29.690324] [INFO ] [REMOTE ] APD: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T17:15:29.697500] [INFO ] [REMOTE ] APD: Full fetch complete (67393 records). [2025-10-20T17:15:29.697638] [DEBUG] [VALIDATOR ] APD: Validating primary data. [2025-10-20T17:15:29.854479] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-08. [2025-10-20T17:15:30.092179] [DEBUG] [VALIDATOR ] APD: All expected trading days are present. [2025-10-20T17:15:30.092307] [DEBUG] [VALIDATOR ] APD: Primary data validation passed. [2025-10-20T17:15:30.093050] [INFO ] [MAIN ] APD: Fetched primary data is valid. [2025-10-20T17:15:30.093113] [DEBUG] [LOCAL ] APD: Saving primary data. [2025-10-20T17:15:30.094667] [DEBUG] [LOCAL ] APD: Trimmed 67393 records to 45522 (keeping 200 days of data) [2025-10-20T17:15:30.382080] [DEBUG] [LOCAL ] APD: Primary data saved (45522 records). [2025-10-20T17:15:30.382233] [INFO ] [MAIN ] APD: Full history saved. [2025-10-20T17:15:30.382972] [INFO ] [MAIN ] Update Progress: 131/2367 symbols completed. APD took 32.1s. Estimated remaining: 4h 23m 25s. Rate: 8.5 symbols/min [2025-10-20T17:15:30.564396] [INFO ] [MAIN ] APOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:30.564739] [DEBUG] [LOCAL ] APOS: No existing data file found in primary storage. [2025-10-20T17:15:30.564815] [WARN ] [MAIN ] APOS: No existing data found, performing full refresh instead. [2025-10-20T17:15:30.564952] [INFO ] [MAIN ] APOS: Wiping existing primary data before full refresh. [2025-10-20T17:15:30.565015] [INFO ] [LOCAL ] APOS: Starting deletion of primary data. [2025-10-20T17:15:30.565193] [DEBUG] [LOCAL ] APOS: No primary data file found to delete. [2025-10-20T17:15:30.565276] [INFO ] [MAIN ] APOS: Starting full primary refresh. [2025-10-20T17:15:30.656492] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-01. [2025-10-20T17:15:30.861258] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:15:30.967033] [INFO ] [REMOTE ] APOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:31.451429] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-01. [2025-10-20T17:15:31.760419] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:15:31.998256] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:15:32.251414] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-02. [2025-10-20T17:15:32.705315] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:15:32.733866] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:15:33.051367] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-02. [2025-10-20T17:15:33.851365] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-09. [2025-10-20T17:15:34.651380] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-09. [2025-10-20T17:15:34.939607] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T17:15:35.451400] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-03. [2025-10-20T17:15:35.715875] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:15:36.251340] [INFO ] [REMOTE ] APLE: Fetching primary data for 2025-10. [2025-10-20T17:15:36.264282] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:15:36.822967] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:15:37.051413] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-04. [2025-10-20T17:15:37.802041] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:15:37.851395] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-05. [2025-10-20T17:15:37.893944] [INFO ] [REMOTE ] APLE: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:15:37.901790] [INFO ] [REMOTE ] APLE: Full fetch complete (74742 records). [2025-10-20T17:15:37.901955] [DEBUG] [VALIDATOR ] APLE: Validating primary data. [2025-10-20T17:15:38.296258] [DEBUG] [VALIDATOR ] APLE: All expected trading days are present. [2025-10-20T17:15:38.296337] [DEBUG] [VALIDATOR ] APLE: Primary data validation passed. [2025-10-20T17:15:38.297096] [INFO ] [MAIN ] APLE: Fetched primary data is valid. [2025-10-20T17:15:38.297159] [DEBUG] [LOCAL ] APLE: Saving primary data. [2025-10-20T17:15:38.298556] [DEBUG] [LOCAL ] APLE: Trimmed 74742 records to 51975 (keeping 200 days of data) [2025-10-20T17:15:38.392733] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T17:15:38.619752] [DEBUG] [LOCAL ] APLE: Primary data saved (51975 records). [2025-10-20T17:15:38.619887] [INFO ] [MAIN ] APLE: Full history saved. [2025-10-20T17:15:38.619979] [INFO ] [MAIN ] Update Progress: 132/2367 symbols completed. APLE took 34.1s. Estimated remaining: 4h 23m 38s. Rate: 8.5 symbols/min [2025-10-20T17:15:38.661765] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-06. [2025-10-20T17:15:38.806997] [INFO ] [MAIN ] APTV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:38.863291] [DEBUG] [LOCAL ] APTV: Loaded 52719 existing records from primary storage. [2025-10-20T17:15:38.863751] [INFO ] [MAIN ] APTV: Loaded 52719 existing records. [2025-10-20T17:15:39.064400] [WARN ] [MAIN ] APTV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:39.064701] [INFO ] [MAIN ] APTV: Wiping existing primary data before full refresh. [2025-10-20T17:15:39.064763] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-10-20T17:15:39.065152] [DEBUG] [LOCAL ] APTV: Deleted primary data file. [2025-10-20T17:15:39.065238] [INFO ] [MAIN ] APTV: Starting full primary refresh. [2025-10-20T17:15:39.277794] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:15:39.465054] [INFO ] [REMOTE ] APH: Fetching primary data for 2025-10. [2025-10-20T17:15:39.472803] [INFO ] [REMOTE ] APTV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:40.251417] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-07. [2025-10-20T17:15:40.697566] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T17:15:41.051376] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-08. [2025-10-20T17:15:41.449832] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-10-20T17:15:41.817866] [INFO ] [REMOTE ] APH: Incremental fetch complete for 2025-10 (2.35 seconds). [2025-10-20T17:15:41.826718] [INFO ] [REMOTE ] APH: Full fetch complete (77869 records). [2025-10-20T17:15:41.826913] [DEBUG] [VALIDATOR ] APH: Validating primary data. [2025-10-20T17:15:41.858685] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-01. [2025-10-20T17:15:42.266876] [DEBUG] [VALIDATOR ] APH: All expected trading days are present. [2025-10-20T17:15:42.267061] [DEBUG] [VALIDATOR ] APH: Primary data validation passed. [2025-10-20T17:15:42.268117] [INFO ] [MAIN ] APH: Fetched primary data is valid. [2025-10-20T17:15:42.268187] [DEBUG] [LOCAL ] APH: Saving primary data. [2025-10-20T17:15:42.270633] [DEBUG] [LOCAL ] APH: Trimmed 77869 records to 53691 (keeping 200 days of data) [2025-10-20T17:15:42.644350] [DEBUG] [LOCAL ] APH: Primary data saved (53691 records). [2025-10-20T17:15:42.644590] [INFO ] [MAIN ] APH: Full history saved. [2025-10-20T17:15:42.644726] [INFO ] [MAIN ] Update Progress: 133/2367 symbols completed. APH took 39.8s. Estimated remaining: 4h 22m 40s. Rate: 8.5 symbols/min [2025-10-20T17:15:42.658244] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-03. [2025-10-20T17:15:42.865197] [INFO ] [MAIN ] AQN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:42.865404] [DEBUG] [LOCAL ] AQN: No existing data file found in primary storage. [2025-10-20T17:15:42.865444] [WARN ] [MAIN ] AQN: No existing data found, performing full refresh instead. [2025-10-20T17:15:42.865564] [INFO ] [MAIN ] AQN: Wiping existing primary data before full refresh. [2025-10-20T17:15:42.865606] [INFO ] [LOCAL ] AQN: Starting deletion of primary data. [2025-10-20T17:15:42.865677] [DEBUG] [LOCAL ] AQN: No primary data file found to delete. [2025-10-20T17:15:42.865732] [INFO ] [MAIN ] AQN: Starting full primary refresh. [2025-10-20T17:15:43.213851] [INFO ] [REMOTE ] AQN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:43.451395] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-01. [2025-10-20T17:15:43.507628] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:15:44.251398] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-02. [2025-10-20T17:15:44.459911] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:15:45.019853] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:15:45.051360] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-04. [2025-10-20T17:15:45.835460] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:15:45.851339] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-03. [2025-10-20T17:15:46.651365] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-02. [2025-10-20T17:15:46.933796] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:15:47.451351] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-09. [2025-10-20T17:15:47.675085] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:15:48.251351] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-04. [2025-10-20T17:15:48.333835] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T17:15:48.661082] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T17:15:49.051389] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-05. [2025-10-20T17:15:49.800873] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:15:49.851334] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-05. [2025-10-20T17:15:50.651380] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-03. [2025-10-20T17:15:51.072148] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-10-20T17:15:51.451381] [INFO ] [REMOTE ] APOS: Fetching primary data for 2025-10. [2025-10-20T17:15:51.575140] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:15:52.025466] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:15:52.127842] [INFO ] [REMOTE ] APOS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:15:52.129158] [INFO ] [REMOTE ] APOS: Full fetch complete (10240 records). [2025-10-20T17:15:52.129227] [DEBUG] [VALIDATOR ] APOS: Validating primary data. [2025-10-20T17:15:52.264903] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-04. [2025-10-20T17:15:52.484265] [DEBUG] [VALIDATOR ] APOS: All expected trading days are present. [2025-10-20T17:15:52.484333] [DEBUG] [VALIDATOR ] APOS: Primary data validation passed. [2025-10-20T17:15:52.484505] [INFO ] [MAIN ] APOS: Fetched primary data is valid. [2025-10-20T17:15:52.484543] [DEBUG] [LOCAL ] APOS: Saving primary data. [2025-10-20T17:15:52.484992] [DEBUG] [LOCAL ] APOS: Trimmed 10240 records to 7187 (keeping 200 days of data) [2025-10-20T17:15:52.527491] [DEBUG] [LOCAL ] APOS: Primary data saved (7187 records). [2025-10-20T17:15:52.527579] [INFO ] [MAIN ] APOS: Full history saved. [2025-10-20T17:15:52.527662] [INFO ] [MAIN ] Update Progress: 134/2367 symbols completed. APOS took 22.1s. Estimated remaining: 4h 23m 20s. Rate: 8.5 symbols/min [2025-10-20T17:15:52.695347] [INFO ] [MAIN ] AQNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:15:52.701429] [DEBUG] [LOCAL ] AQNB: Loaded 4088 existing records from primary storage. [2025-10-20T17:15:52.701633] [INFO ] [MAIN ] AQNB: Loaded 4088 existing records. [2025-10-20T17:15:52.875321] [WARN ] [MAIN ] AQNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:15:52.875501] [INFO ] [MAIN ] AQNB: Wiping existing primary data before full refresh. [2025-10-20T17:15:52.875547] [INFO ] [LOCAL ] AQNB: Starting deletion of primary data. [2025-10-20T17:15:52.875710] [DEBUG] [LOCAL ] AQNB: Deleted primary data file. [2025-10-20T17:15:52.875768] [INFO ] [MAIN ] AQNB: Starting full primary refresh. [2025-10-20T17:15:53.054422] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-06. [2025-10-20T17:15:53.235637] [INFO ] [REMOTE ] AQNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:15:53.851372] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-01. [2025-10-20T17:15:54.078476] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:15:54.427171] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T17:15:54.600144] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:15:54.651381] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-07. [2025-10-20T17:15:55.451436] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-06. [2025-10-20T17:15:56.251381] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-02. [2025-10-20T17:15:56.421618] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:15:56.672903] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T17:15:57.051394] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-08. [2025-10-20T17:15:57.238442] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:15:57.851374] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-03. [2025-10-20T17:15:58.495100] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T17:15:58.651373] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-05. [2025-10-20T17:15:58.802495] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:15:59.451416] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-04. [2025-10-20T17:15:59.830418] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-04 (0.38 seconds). [2025-10-20T17:16:00.251384] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-05. [2025-10-20T17:16:00.407914] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:16:01.020721] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T17:16:01.051330] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-06. [2025-10-20T17:16:01.839294] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:16:01.851323] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-07. [2025-10-20T17:16:02.443533] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:16:02.651369] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-06. [2025-10-20T17:16:03.451370] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-09. [2025-10-20T17:16:04.251454] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-08. [2025-10-20T17:16:04.466867] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T17:16:04.965334] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T17:16:05.051349] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-07. [2025-10-20T17:16:05.108012] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:16:05.851413] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-09. [2025-10-20T17:16:06.419366] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:16:06.651431] [INFO ] [REMOTE ] AQNB: Fetching primary data for 2025-10. [2025-10-20T17:16:06.846323] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:16:07.110215] [INFO ] [REMOTE ] AQNB: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T17:16:07.111206] [INFO ] [REMOTE ] AQNB: Full fetch complete (5420 records). [2025-10-20T17:16:07.111276] [DEBUG] [VALIDATOR ] AQNB: Validating primary data. [2025-10-20T17:16:07.459405] [DEBUG] [VALIDATOR ] AQNB: All expected trading days are present. [2025-10-20T17:16:07.459509] [DEBUG] [VALIDATOR ] AQNB: Primary data validation passed. [2025-10-20T17:16:07.459818] [INFO ] [MAIN ] AQNB: Fetched primary data is valid. [2025-10-20T17:16:07.459894] [DEBUG] [LOCAL ] AQNB: Saving primary data. [2025-10-20T17:16:07.454585] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-08. [2025-10-20T17:16:07.461111] [DEBUG] [LOCAL ] AQNB: Trimmed 5420 records to 3972 (keeping 200 days of data) [2025-10-20T17:16:07.484996] [DEBUG] [LOCAL ] AQNB: Primary data saved (3972 records). [2025-10-20T17:16:07.485092] [INFO ] [MAIN ] AQNB: Full history saved. [2025-10-20T17:16:07.485167] [INFO ] [MAIN ] Update Progress: 135/2367 symbols completed. AQNB took 15.0s. Estimated remaining: 4h 25m 23s. Rate: 8.4 symbols/min [2025-10-20T17:16:07.663367] [INFO ] [MAIN ] AR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:07.714479] [DEBUG] [LOCAL ] AR: Loaded 54366 existing records from primary storage. [2025-10-20T17:16:07.714852] [INFO ] [MAIN ] AR: Loaded 54366 existing records. [2025-10-20T17:16:07.887265] [WARN ] [MAIN ] AR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:07.887479] [INFO ] [MAIN ] AR: Wiping existing primary data before full refresh. [2025-10-20T17:16:07.887526] [INFO ] [LOCAL ] AR: Starting deletion of primary data. [2025-10-20T17:16:07.887911] [DEBUG] [LOCAL ] AR: Deleted primary data file. [2025-10-20T17:16:07.887967] [INFO ] [MAIN ] AR: Starting full primary refresh. [2025-10-20T17:16:08.237284] [INFO ] [REMOTE ] AR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:08.251335] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-01. [2025-10-20T17:16:08.785819] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T17:16:09.051379] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-09. [2025-10-20T17:16:09.851356] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-07. [2025-10-20T17:16:10.258183] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T17:16:10.405612] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:16:10.651343] [INFO ] [REMOTE ] AQN: Fetching primary data for 2025-10. [2025-10-20T17:16:11.451408] [INFO ] [REMOTE ] APTV: Fetching primary data for 2025-10. [2025-10-20T17:16:11.830229] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T17:16:12.235728] [INFO ] [REMOTE ] APTV: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:16:12.243037] [INFO ] [REMOTE ] APTV: Full fetch complete (75535 records). [2025-10-20T17:16:12.243214] [DEBUG] [VALIDATOR ] APTV: Validating primary data. [2025-10-20T17:16:12.244018] [FATAL] [VALIDATOR ] APTV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:16:12.244115] [FATAL] [MAIN ] APTV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:16:12.244157] [WARN ] [MAIN ] APTV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:16:12.244221] [INFO ] [LOCAL ] APTV: Starting deletion of primary data. [2025-10-20T17:16:12.244285] [DEBUG] [LOCAL ] APTV: No primary data file found to delete. [2025-10-20T17:16:12.244319] [INFO ] [LOCAL ] APTV: Starting deletion of derivative data. [2025-10-20T17:16:12.244491] [DEBUG] [LOCAL ] APTV: Deleted derivative data file. [2025-10-20T17:16:12.244583] [INFO ] [MAIN ] Update Progress: 136/2367 symbols completed. APTV took 33.6s. Estimated remaining: 4h 24m 37s. Rate: 8.4 symbols/min [2025-10-20T17:16:12.257565] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-08. [2025-10-20T17:16:12.420471] [INFO ] [MAIN ] ARCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:12.460941] [DEBUG] [LOCAL ] ARCO: Loaded 45143 existing records from primary storage. [2025-10-20T17:16:12.461285] [INFO ] [MAIN ] ARCO: Loaded 45143 existing records. [2025-10-20T17:16:12.679966] [WARN ] [MAIN ] ARCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:12.680285] [INFO ] [MAIN ] ARCO: Wiping existing primary data before full refresh. [2025-10-20T17:16:12.680433] [INFO ] [LOCAL ] ARCO: Starting deletion of primary data. [2025-10-20T17:16:12.687471] [DEBUG] [LOCAL ] ARCO: Deleted primary data file. [2025-10-20T17:16:12.700284] [INFO ] [REMOTE ] AQN: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T17:16:12.699696] [INFO ] [MAIN ] ARCO: Starting full primary refresh. [2025-10-20T17:16:12.760738] [INFO ] [REMOTE ] AQN: Full fetch complete (75371 records). [2025-10-20T17:16:12.789152] [DEBUG] [VALIDATOR ] AQN: Validating primary data. [2025-10-20T17:16:13.055818] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-02. [2025-10-20T17:16:13.585811] [DEBUG] [VALIDATOR ] AQN: All expected trading days are present. [2025-10-20T17:16:13.585529] [INFO ] [REMOTE ] ARCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:13.585977] [DEBUG] [VALIDATOR ] AQN: Primary data validation passed. [2025-10-20T17:16:13.587153] [INFO ] [MAIN ] AQN: Fetched primary data is valid. [2025-10-20T17:16:13.587221] [DEBUG] [LOCAL ] AQN: Saving primary data. [2025-10-20T17:16:13.588787] [DEBUG] [LOCAL ] AQN: Trimmed 75371 records to 51714 (keeping 200 days of data) [2025-10-20T17:16:13.866508] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-01. [2025-10-20T17:16:13.885685] [DEBUG] [LOCAL ] AQN: Primary data saved (51714 records). [2025-10-20T17:16:13.886013] [INFO ] [MAIN ] AQN: Full history saved. [2025-10-20T17:16:13.886179] [INFO ] [MAIN ] Update Progress: 137/2367 symbols completed. AQN took 31.2s. Estimated remaining: 4h 23m 1s. Rate: 8.5 symbols/min [2025-10-20T17:16:14.063320] [INFO ] [MAIN ] ARDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:14.063496] [DEBUG] [LOCAL ] ARDC: No existing data file found in primary storage. [2025-10-20T17:16:14.063626] [WARN ] [MAIN ] ARDC: No existing data found, performing full refresh instead. [2025-10-20T17:16:14.063797] [INFO ] [MAIN ] ARDC: Wiping existing primary data before full refresh. [2025-10-20T17:16:14.063915] [INFO ] [LOCAL ] ARDC: Starting deletion of primary data. [2025-10-20T17:16:14.063982] [DEBUG] [LOCAL ] ARDC: No primary data file found to delete. [2025-10-20T17:16:14.064016] [INFO ] [MAIN ] ARDC: Starting full primary refresh. [2025-10-20T17:16:14.379797] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T17:16:14.466202] [INFO ] [REMOTE ] ARDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:14.651365] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-09. [2025-10-20T17:16:14.747047] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:16:15.451395] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-03. [2025-10-20T17:16:15.546515] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T17:16:16.251443] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-02. [2025-10-20T17:16:16.612518] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:16:17.051396] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-01. [2025-10-20T17:16:17.349748] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:16:17.805774] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:16:17.851336] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-03. [2025-10-20T17:16:17.955668] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T17:16:18.651382] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-02. [2025-10-20T17:16:19.430291] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:16:19.456498] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-03. [2025-10-20T17:16:19.469797] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:16:20.149955] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T17:16:20.251365] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-04. [2025-10-20T17:16:21.051377] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-04. [2025-10-20T17:16:21.851399] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-04. [2025-10-20T17:16:22.036277] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:16:22.258863] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T17:16:22.651376] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-05. [2025-10-20T17:16:23.148005] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:16:23.451386] [INFO ] [REMOTE ] APO: Fetching primary data for 2025-10. [2025-10-20T17:16:24.251396] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-05. [2025-10-20T17:16:24.514227] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:16:25.055748] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-06. [2025-10-20T17:16:25.060802] [INFO ] [REMOTE ] APO: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T17:16:25.068838] [INFO ] [REMOTE ] APO: Full fetch complete (77077 records). [2025-10-20T17:16:25.069004] [DEBUG] [VALIDATOR ] APO: Validating primary data. [2025-10-20T17:16:25.069937] [FATAL] [VALIDATOR ] APO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:16:25.070030] [FATAL] [MAIN ] APO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:16:25.070073] [WARN ] [MAIN ] APO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:16:25.070140] [INFO ] [LOCAL ] APO: Starting deletion of primary data. [2025-10-20T17:16:25.070206] [DEBUG] [LOCAL ] APO: No primary data file found to delete. [2025-10-20T17:16:25.070241] [INFO ] [LOCAL ] APO: Starting deletion of derivative data. [2025-10-20T17:16:25.070279] [DEBUG] [LOCAL ] APO: No derivative data file found to delete. [2025-10-20T17:16:25.070336] [INFO ] [MAIN ] Update Progress: 138/2367 symbols completed. APO took 56.6s. Estimated remaining: 4h 23m 60s. Rate: 8.4 symbols/min [2025-10-20T17:16:25.252548] [INFO ] [MAIN ] ARDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:25.279025] [DEBUG] [LOCAL ] ARDT: Loaded 28112 existing records from primary storage. [2025-10-20T17:16:25.279294] [INFO ] [MAIN ] ARDT: Loaded 28112 existing records. [2025-10-20T17:16:25.453447] [WARN ] [MAIN ] ARDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:25.453702] [INFO ] [MAIN ] ARDT: Wiping existing primary data before full refresh. [2025-10-20T17:16:25.453769] [INFO ] [LOCAL ] ARDT: Starting deletion of primary data. [2025-10-20T17:16:25.454222] [DEBUG] [LOCAL ] ARDT: Deleted primary data file. [2025-10-20T17:16:25.454282] [INFO ] [MAIN ] ARDT: Starting full primary refresh. [2025-10-20T17:16:25.828939] [INFO ] [REMOTE ] ARDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:25.851344] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-05. [2025-10-20T17:16:25.988285] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:16:26.651381] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-06. [2025-10-20T17:16:26.714627] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:16:27.098730] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-10-20T17:16:27.451410] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-01. [2025-10-20T17:16:27.952249] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:16:28.251373] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-07. [2025-10-20T17:16:28.750867] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T17:16:29.051376] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-07. [2025-10-20T17:16:29.576528] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:16:29.851399] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-02. [2025-10-20T17:16:30.555309] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:16:30.651346] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-08. [2025-10-20T17:16:30.992374] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:16:31.451397] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-06. [2025-10-20T17:16:32.251368] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-08. [2025-10-20T17:16:32.482856] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:16:32.647586] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T17:16:33.051391] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-03. [2025-10-20T17:16:33.665319] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:16:33.851375] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-09. [2025-10-20T17:16:34.132548] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:16:34.651411] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-04. [2025-10-20T17:16:35.012069] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:16:35.451421] [INFO ] [REMOTE ] ARCO: Fetching primary data for 2025-10. [2025-10-20T17:16:35.985328] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:16:36.251434] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-09. [2025-10-20T17:16:36.760424] [INFO ] [REMOTE ] ARCO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:16:36.768453] [INFO ] [REMOTE ] ARCO: Full fetch complete (65147 records). [2025-10-20T17:16:36.768710] [DEBUG] [VALIDATOR ] ARCO: Validating primary data. [2025-10-20T17:16:37.056555] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-07. [2025-10-20T17:16:37.182697] [DEBUG] [VALIDATOR ] ARCO: All expected trading days are present. [2025-10-20T17:16:37.182832] [DEBUG] [VALIDATOR ] ARCO: Primary data validation passed. [2025-10-20T17:16:37.183576] [INFO ] [MAIN ] ARCO: Fetched primary data is valid. [2025-10-20T17:16:37.183622] [DEBUG] [LOCAL ] ARCO: Saving primary data. [2025-10-20T17:16:37.185744] [DEBUG] [LOCAL ] ARCO: Trimmed 65147 records to 44434 (keeping 200 days of data) [2025-10-20T17:16:37.470263] [DEBUG] [LOCAL ] ARCO: Primary data saved (44434 records). [2025-10-20T17:16:37.470501] [INFO ] [MAIN ] ARCO: Full history saved. [2025-10-20T17:16:37.470636] [INFO ] [MAIN ] Update Progress: 139/2367 symbols completed. ARCO took 25.2s. Estimated remaining: 4h 25m 18s. Rate: 8.4 symbols/min [2025-10-20T17:16:37.652667] [INFO ] [MAIN ] ARE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:37.652908] [DEBUG] [LOCAL ] ARE: No existing data file found in primary storage. [2025-10-20T17:16:37.652949] [WARN ] [MAIN ] ARE: No existing data found, performing full refresh instead. [2025-10-20T17:16:37.653075] [INFO ] [MAIN ] ARE: Wiping existing primary data before full refresh. [2025-10-20T17:16:37.653118] [INFO ] [LOCAL ] ARE: Starting deletion of primary data. [2025-10-20T17:16:37.653160] [DEBUG] [LOCAL ] ARE: No primary data file found to delete. [2025-10-20T17:16:37.653194] [INFO ] [MAIN ] ARE: Starting full primary refresh. [2025-10-20T17:16:37.856688] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-05. [2025-10-20T17:16:38.067406] [INFO ] [REMOTE ] ARE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:38.197709] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T17:16:38.312373] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:16:38.651432] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-01. [2025-10-20T17:16:38.834176] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T17:16:39.451387] [INFO ] [REMOTE ] AR: Fetching primary data for 2025-10. [2025-10-20T17:16:40.251375] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-06. [2025-10-20T17:16:40.530337] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T17:16:41.051343] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-08. [2025-10-20T17:16:41.421847] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:16:41.541000] [INFO ] [REMOTE ] AR: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:16:41.549528] [INFO ] [REMOTE ] AR: Full fetch complete (77737 records). [2025-10-20T17:16:41.549770] [DEBUG] [VALIDATOR ] AR: Validating primary data. [2025-10-20T17:16:41.740069] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:16:41.856511] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-02. [2025-10-20T17:16:41.973450] [DEBUG] [VALIDATOR ] AR: All expected trading days are present. [2025-10-20T17:16:41.973532] [DEBUG] [VALIDATOR ] AR: Primary data validation passed. [2025-10-20T17:16:41.974493] [INFO ] [MAIN ] AR: Fetched primary data is valid. [2025-10-20T17:16:41.974566] [DEBUG] [LOCAL ] AR: Saving primary data. [2025-10-20T17:16:41.976510] [DEBUG] [LOCAL ] AR: Trimmed 77737 records to 53590 (keeping 200 days of data) [2025-10-20T17:16:42.318725] [DEBUG] [LOCAL ] AR: Primary data saved (53590 records). [2025-10-20T17:16:42.318878] [INFO ] [MAIN ] AR: Full history saved. [2025-10-20T17:16:42.319824] [INFO ] [MAIN ] Update Progress: 140/2367 symbols completed. AR took 34.8s. Estimated remaining: 4h 24m 34s. Rate: 8.4 symbols/min [2025-10-20T17:16:42.496190] [INFO ] [MAIN ] ARES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:42.555602] [DEBUG] [LOCAL ] ARES: Loaded 50486 existing records from primary storage. [2025-10-20T17:16:42.555964] [INFO ] [MAIN ] ARES: Loaded 50486 existing records. [2025-10-20T17:16:42.656629] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-07. [2025-10-20T17:16:42.740302] [WARN ] [MAIN ] ARES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:42.740551] [INFO ] [MAIN ] ARES: Wiping existing primary data before full refresh. [2025-10-20T17:16:42.740621] [INFO ] [LOCAL ] ARES: Starting deletion of primary data. [2025-10-20T17:16:42.740900] [DEBUG] [LOCAL ] ARES: Deleted primary data file. [2025-10-20T17:16:42.740960] [INFO ] [MAIN ] ARES: Starting full primary refresh. [2025-10-20T17:16:43.112221] [INFO ] [REMOTE ] ARES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:43.451412] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-09. [2025-10-20T17:16:43.549504] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:16:44.081047] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:16:44.251364] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-08. [2025-10-20T17:16:44.441915] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:16:45.051381] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-01. [2025-10-20T17:16:45.446997] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T17:16:45.851380] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-09. [2025-10-20T17:16:46.651385] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-03. [2025-10-20T17:16:46.707023] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:16:46.956439] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:16:47.451388] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-02. [2025-10-20T17:16:48.251352] [INFO ] [REMOTE ] ARDC: Fetching primary data for 2025-10. [2025-10-20T17:16:48.295965] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T17:16:48.749794] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T17:16:49.051375] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-03. [2025-10-20T17:16:49.409248] [INFO ] [REMOTE ] ARDC: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:16:49.412440] [INFO ] [REMOTE ] ARDC: Full fetch complete (21441 records). [2025-10-20T17:16:49.412530] [DEBUG] [VALIDATOR ] ARDC: Validating primary data. [2025-10-20T17:16:49.811950] [DEBUG] [VALIDATOR ] ARDC: All expected trading days are present. [2025-10-20T17:16:49.812048] [DEBUG] [VALIDATOR ] ARDC: Primary data validation passed. [2025-10-20T17:16:49.812423] [INFO ] [MAIN ] ARDC: Fetched primary data is valid. [2025-10-20T17:16:49.812496] [DEBUG] [LOCAL ] ARDC: Saving primary data. [2025-10-20T17:16:49.813101] [DEBUG] [LOCAL ] ARDC: Trimmed 21441 records to 15280 (keeping 200 days of data) [2025-10-20T17:16:49.855060] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-04. [2025-10-20T17:16:49.918186] [DEBUG] [LOCAL ] ARDC: Primary data saved (15280 records). [2025-10-20T17:16:49.918551] [INFO ] [MAIN ] ARDC: Full history saved. [2025-10-20T17:16:49.918886] [INFO ] [MAIN ] Update Progress: 141/2367 symbols completed. ARDC took 36.0s. Estimated remaining: 4h 24m 34s. Rate: 8.4 symbols/min [2025-10-20T17:16:50.114579] [INFO ] [MAIN ] ARI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:50.155425] [DEBUG] [LOCAL ] ARI: Loaded 44332 existing records from primary storage. [2025-10-20T17:16:50.155753] [INFO ] [MAIN ] ARI: Loaded 44332 existing records. [2025-10-20T17:16:50.370731] [WARN ] [MAIN ] ARI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:50.370953] [INFO ] [MAIN ] ARI: Wiping existing primary data before full refresh. [2025-10-20T17:16:50.371001] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-10-20T17:16:50.371277] [DEBUG] [LOCAL ] ARI: Deleted primary data file. [2025-10-20T17:16:50.371334] [INFO ] [MAIN ] ARI: Starting full primary refresh. [2025-10-20T17:16:50.655980] [INFO ] [REMOTE ] ARDT: Fetching primary data for 2025-10. [2025-10-20T17:16:50.767517] [INFO ] [REMOTE ] ARI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:51.011465] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:16:51.451435] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-01. [2025-10-20T17:16:52.021549] [INFO ] [REMOTE ] ARDT: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:16:52.026497] [INFO ] [REMOTE ] ARDT: Full fetch complete (41543 records). [2025-10-20T17:16:52.026704] [DEBUG] [VALIDATOR ] ARDT: Validating primary data. [2025-10-20T17:16:52.263695] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-04. [2025-10-20T17:16:52.505023] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-04 (2.65 seconds). [2025-10-20T17:16:52.505582] [DEBUG] [VALIDATOR ] ARDT: All expected trading days are present. [2025-10-20T17:16:52.516830] [DEBUG] [VALIDATOR ] ARDT: Primary data validation passed. [2025-10-20T17:16:52.517849] [INFO ] [MAIN ] ARDT: Fetched primary data is valid. [2025-10-20T17:16:52.517963] [DEBUG] [LOCAL ] ARDT: Saving primary data. [2025-10-20T17:16:52.519715] [DEBUG] [LOCAL ] ARDT: Trimmed 41543 records to 27434 (keeping 200 days of data) [2025-10-20T17:16:52.722361] [DEBUG] [LOCAL ] ARDT: Primary data saved (27434 records). [2025-10-20T17:16:52.724473] [INFO ] [MAIN ] ARDT: Full history saved. [2025-10-20T17:16:52.724340] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:16:52.724677] [INFO ] [MAIN ] Update Progress: 142/2367 symbols completed. ARDT took 27.7s. Estimated remaining: 4h 23m 19s. Rate: 8.4 symbols/min [2025-10-20T17:16:52.926266] [INFO ] [MAIN ] ARLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:16:52.970430] [DEBUG] [LOCAL ] ARLO: Loaded 44394 existing records from primary storage. [2025-10-20T17:16:52.970894] [INFO ] [MAIN ] ARLO: Loaded 44394 existing records. [2025-10-20T17:16:53.056597] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-02. [2025-10-20T17:16:53.161990] [WARN ] [MAIN ] ARLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:16:53.162303] [INFO ] [MAIN ] ARLO: Wiping existing primary data before full refresh. [2025-10-20T17:16:53.162359] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-10-20T17:16:53.162603] [DEBUG] [LOCAL ] ARLO: Deleted primary data file. [2025-10-20T17:16:53.162677] [INFO ] [MAIN ] ARLO: Starting full primary refresh. [2025-10-20T17:16:53.558174] [INFO ] [REMOTE ] ARLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:16:53.735408] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:16:53.851369] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-05. [2025-10-20T17:16:54.639865] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:16:54.651333] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-03. [2025-10-20T17:16:55.451369] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-05. [2025-10-20T17:16:55.694477] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:16:55.776370] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:16:56.251417] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-01. [2025-10-20T17:16:57.051379] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-06. [2025-10-20T17:16:57.416496] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:16:57.720160] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T17:16:57.851421] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-06. [2025-10-20T17:16:58.522964] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T17:16:58.651410] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-07. [2025-10-20T17:16:59.451402] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-04. [2025-10-20T17:16:59.544044] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:17:00.251374] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-07. [2025-10-20T17:17:00.476767] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T17:17:01.051489] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-08. [2025-10-20T17:17:01.851403] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-02. [2025-10-20T17:17:01.936561] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-04 (2.49 seconds). [2025-10-20T17:17:02.573861] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-10-20T17:17:02.651446] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-05. [2025-10-20T17:17:03.101464] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T17:17:03.139211] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T17:17:03.451408] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-08. [2025-10-20T17:17:04.251408] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-09. [2025-10-20T17:17:04.411171] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:17:05.051430] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-06. [2025-10-20T17:17:05.216770] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T17:17:05.765757] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:17:05.851378] [INFO ] [REMOTE ] ARE: Fetching primary data for 2025-10. [2025-10-20T17:17:06.550118] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:17:06.651403] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-09. [2025-10-20T17:17:07.451366] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-03. [2025-10-20T17:17:07.526788] [INFO ] [REMOTE ] ARE: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:17:07.534950] [INFO ] [REMOTE ] ARE: Full fetch complete (73020 records). [2025-10-20T17:17:07.535083] [DEBUG] [VALIDATOR ] ARE: Validating primary data. [2025-10-20T17:17:07.958875] [DEBUG] [VALIDATOR ] ARE: All expected trading days are present. [2025-10-20T17:17:07.958980] [DEBUG] [VALIDATOR ] ARE: Primary data validation passed. [2025-10-20T17:17:07.959808] [INFO ] [MAIN ] ARE: Fetched primary data is valid. [2025-10-20T17:17:07.959908] [DEBUG] [LOCAL ] ARE: Saving primary data. [2025-10-20T17:17:07.961500] [DEBUG] [LOCAL ] ARE: Trimmed 73020 records to 50965 (keeping 200 days of data) [2025-10-20T17:17:08.263458] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-07. [2025-10-20T17:17:08.284516] [DEBUG] [LOCAL ] ARE: Primary data saved (50965 records). [2025-10-20T17:17:08.285931] [INFO ] [MAIN ] ARE: Full history saved. [2025-10-20T17:17:08.286111] [INFO ] [MAIN ] Update Progress: 143/2367 symbols completed. ARE took 30.8s. Estimated remaining: 4h 25m 24s. Rate: 8.4 symbols/min [2025-10-20T17:17:08.472841] [INFO ] [MAIN ] ARMK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:08.504206] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:17:08.555695] [DEBUG] [LOCAL ] ARMK: Loaded 51136 existing records from primary storage. [2025-10-20T17:17:08.556047] [INFO ] [MAIN ] ARMK: Loaded 51136 existing records. [2025-10-20T17:17:08.732809] [WARN ] [MAIN ] ARMK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:08.733019] [INFO ] [MAIN ] ARMK: Wiping existing primary data before full refresh. [2025-10-20T17:17:08.733067] [INFO ] [LOCAL ] ARMK: Starting deletion of primary data. [2025-10-20T17:17:08.733456] [DEBUG] [LOCAL ] ARMK: Deleted primary data file. [2025-10-20T17:17:08.733514] [INFO ] [MAIN ] ARMK: Starting full primary refresh. [2025-10-20T17:17:09.056584] [INFO ] [REMOTE ] ARES: Fetching primary data for 2025-10. [2025-10-20T17:17:09.096993] [INFO ] [REMOTE ] ARMK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:09.359716] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T17:17:09.513466] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:17:09.851428] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-04. [2025-10-20T17:17:10.651372] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-08. [2025-10-20T17:17:10.923059] [INFO ] [REMOTE ] ARES: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:17:10.931437] [INFO ] [REMOTE ] ARES: Full fetch complete (71606 records). [2025-10-20T17:17:10.931699] [DEBUG] [VALIDATOR ] ARES: Validating primary data. [2025-10-20T17:17:11.247429] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:17:11.374052] [DEBUG] [VALIDATOR ] ARES: All expected trading days are present. [2025-10-20T17:17:11.374138] [DEBUG] [VALIDATOR ] ARES: Primary data validation passed. [2025-10-20T17:17:11.374847] [INFO ] [MAIN ] ARES: Fetched primary data is valid. [2025-10-20T17:17:11.374910] [DEBUG] [LOCAL ] ARES: Saving primary data. [2025-10-20T17:17:11.376300] [DEBUG] [LOCAL ] ARES: Trimmed 71606 records to 49703 (keeping 200 days of data) [2025-10-20T17:17:11.468472] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-05. [2025-10-20T17:17:11.689191] [DEBUG] [LOCAL ] ARES: Primary data saved (49703 records). [2025-10-20T17:17:11.689330] [INFO ] [MAIN ] ARES: Full history saved. [2025-10-20T17:17:11.689964] [INFO ] [MAIN ] Update Progress: 144/2367 symbols completed. ARES took 29.4s. Estimated remaining: 4h 24m 19s. Rate: 8.4 symbols/min [2025-10-20T17:17:11.871468] [INFO ] [MAIN ] AROC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:11.899343] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:17:11.950069] [DEBUG] [LOCAL ] AROC: Loaded 49789 existing records from primary storage. [2025-10-20T17:17:11.950405] [INFO ] [MAIN ] AROC: Loaded 49789 existing records. [2025-10-20T17:17:12.128680] [WARN ] [MAIN ] AROC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:12.128939] [INFO ] [MAIN ] AROC: Wiping existing primary data before full refresh. [2025-10-20T17:17:12.129002] [INFO ] [LOCAL ] AROC: Starting deletion of primary data. [2025-10-20T17:17:12.129228] [DEBUG] [LOCAL ] AROC: Deleted primary data file. [2025-10-20T17:17:12.129285] [INFO ] [MAIN ] AROC: Starting full primary refresh. [2025-10-20T17:17:12.256564] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-01. [2025-10-20T17:17:12.498159] [INFO ] [REMOTE ] AROC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:13.020899] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:17:13.051340] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-06. [2025-10-20T17:17:13.605380] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T17:17:13.851405] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-09. [2025-10-20T17:17:14.651357] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-02. [2025-10-20T17:17:14.722531] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:17:15.443365] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T17:17:15.452456] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-01. [2025-10-20T17:17:16.251469] [INFO ] [REMOTE ] ARI: Fetching primary data for 2025-10. [2025-10-20T17:17:16.304527] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:17:17.051429] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-03. [2025-10-20T17:17:17.233633] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:17:17.851464] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-02. [2025-10-20T17:17:18.074792] [INFO ] [REMOTE ] ARI: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:17:18.081776] [INFO ] [REMOTE ] ARI: Full fetch complete (62121 records). [2025-10-20T17:17:18.081939] [DEBUG] [VALIDATOR ] ARI: Validating primary data. [2025-10-20T17:17:18.082782] [FATAL] [VALIDATOR ] ARI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:18.082883] [FATAL] [MAIN ] ARI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:18.082966] [WARN ] [MAIN ] ARI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:18.083059] [INFO ] [LOCAL ] ARI: Starting deletion of primary data. [2025-10-20T17:17:18.083139] [DEBUG] [LOCAL ] ARI: No primary data file found to delete. [2025-10-20T17:17:18.083179] [INFO ] [LOCAL ] ARI: Starting deletion of derivative data. [2025-10-20T17:17:18.083377] [DEBUG] [LOCAL ] ARI: Deleted derivative data file. [2025-10-20T17:17:18.083477] [INFO ] [MAIN ] Update Progress: 145/2367 symbols completed. ARI took 28.2s. Estimated remaining: 4h 24m 0s. Rate: 8.4 symbols/min [2025-10-20T17:17:18.263341] [INFO ] [MAIN ] ARR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:18.318396] [DEBUG] [LOCAL ] ARR: Loaded 52880 existing records from primary storage. [2025-10-20T17:17:18.318851] [INFO ] [MAIN ] ARR: Loaded 52880 existing records. [2025-10-20T17:17:18.509944] [WARN ] [MAIN ] ARR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:18.510217] [INFO ] [MAIN ] ARR: Wiping existing primary data before full refresh. [2025-10-20T17:17:18.510267] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-10-20T17:17:18.510637] [DEBUG] [LOCAL ] ARR: Deleted primary data file. [2025-10-20T17:17:18.510733] [INFO ] [MAIN ] ARR: Starting full primary refresh. [2025-10-20T17:17:18.655629] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-07. [2025-10-20T17:17:18.902803] [INFO ] [REMOTE ] ARR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:18.933070] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:17:19.063049] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:17:19.451390] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-03. [2025-10-20T17:17:20.251437] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-01. [2025-10-20T17:17:20.388923] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:17:21.051520] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-08. [2025-10-20T17:17:21.102689] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:17:21.851389] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-04. [2025-10-20T17:17:22.272974] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:17:22.651341] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-04. [2025-10-20T17:17:22.753841] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T17:17:23.141023] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T17:17:23.451373] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-09. [2025-10-20T17:17:24.238756] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:17:24.251380] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-05. [2025-10-20T17:17:24.740381] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:17:25.051417] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-02. [2025-10-20T17:17:25.851480] [INFO ] [REMOTE ] ARLO: Fetching primary data for 2025-10. [2025-10-20T17:17:26.024730] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:17:26.651389] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-06. [2025-10-20T17:17:26.895606] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:17:27.451409] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-03. [2025-10-20T17:17:27.578180] [INFO ] [REMOTE ] ARLO: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:17:27.584575] [INFO ] [REMOTE ] ARLO: Full fetch complete (62940 records). [2025-10-20T17:17:27.584775] [DEBUG] [VALIDATOR ] ARLO: Validating primary data. [2025-10-20T17:17:27.585808] [FATAL] [VALIDATOR ] ARLO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:27.585906] [FATAL] [MAIN ] ARLO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:27.585952] [WARN ] [MAIN ] ARLO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:27.585996] [INFO ] [LOCAL ] ARLO: Starting deletion of primary data. [2025-10-20T17:17:27.586061] [DEBUG] [LOCAL ] ARLO: No primary data file found to delete. [2025-10-20T17:17:27.586100] [INFO ] [LOCAL ] ARLO: Starting deletion of derivative data. [2025-10-20T17:17:27.586497] [DEBUG] [LOCAL ] ARLO: Deleted derivative data file. [2025-10-20T17:17:27.587285] [INFO ] [MAIN ] Update Progress: 146/2367 symbols completed. ARLO took 34.9s. Estimated remaining: 4h 24m 29s. Rate: 8.4 symbols/min [2025-10-20T17:17:27.768722] [INFO ] [MAIN ] ARW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:27.804256] [DEBUG] [LOCAL ] ARW: Loaded 33602 existing records from primary storage. [2025-10-20T17:17:27.804525] [INFO ] [MAIN ] ARW: Loaded 33602 existing records. [2025-10-20T17:17:27.987243] [WARN ] [MAIN ] ARW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:27.987803] [INFO ] [MAIN ] ARW: Wiping existing primary data before full refresh. [2025-10-20T17:17:27.988158] [INFO ] [LOCAL ] ARW: Starting deletion of primary data. [2025-10-20T17:17:27.988822] [DEBUG] [LOCAL ] ARW: Deleted primary data file. [2025-10-20T17:17:27.989173] [INFO ] [MAIN ] ARW: Starting full primary refresh. [2025-10-20T17:17:28.242175] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:17:28.267436] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-05. [2025-10-20T17:17:28.450474] [INFO ] [REMOTE ] ARW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:29.051396] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-01. [2025-10-20T17:17:29.303418] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:17:29.851407] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-04. [2025-10-20T17:17:30.038296] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:17:30.299271] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T17:17:30.651402] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-06. [2025-10-20T17:17:31.451404] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-02. [2025-10-20T17:17:31.843717] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:17:31.905273] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:17:32.251377] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-07. [2025-10-20T17:17:32.804395] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:17:33.051383] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-05. [2025-10-20T17:17:33.851360] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-07. [2025-10-20T17:17:34.026743] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:17:34.651423] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-03. [2025-10-20T17:17:35.214891] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-05 (2.16 seconds). [2025-10-20T17:17:35.290269] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:17:35.451369] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-08. [2025-10-20T17:17:35.797592] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T17:17:36.251411] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-06. [2025-10-20T17:17:36.923888] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:17:37.051369] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-09. [2025-10-20T17:17:37.851374] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-04. [2025-10-20T17:17:37.904850] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:17:38.651358] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-07. [2025-10-20T17:17:38.698768] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:17:39.430537] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:17:39.451339] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-05. [2025-10-20T17:17:40.251394] [INFO ] [REMOTE ] ARMK: Fetching primary data for 2025-10. [2025-10-20T17:17:40.912514] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:17:41.053570] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-08. [2025-10-20T17:17:41.058897] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:17:41.851367] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-08. [2025-10-20T17:17:42.170561] [INFO ] [REMOTE ] ARMK: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:17:42.179263] [INFO ] [REMOTE ] ARMK: Full fetch complete (73006 records). [2025-10-20T17:17:42.179523] [DEBUG] [VALIDATOR ] ARMK: Validating primary data. [2025-10-20T17:17:42.670691] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-06. [2025-10-20T17:17:42.671134] [DEBUG] [VALIDATOR ] ARMK: All expected trading days are present. [2025-10-20T17:17:42.671464] [DEBUG] [VALIDATOR ] ARMK: Primary data validation passed. [2025-10-20T17:17:42.672576] [INFO ] [MAIN ] ARMK: Fetched primary data is valid. [2025-10-20T17:17:42.673774] [DEBUG] [LOCAL ] ARMK: Saving primary data. [2025-10-20T17:17:42.675631] [DEBUG] [LOCAL ] ARMK: Trimmed 73006 records to 50421 (keeping 200 days of data) [2025-10-20T17:17:42.855171] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:17:43.047450] [DEBUG] [LOCAL ] ARMK: Primary data saved (50421 records). [2025-10-20T17:17:43.047663] [INFO ] [MAIN ] ARMK: Full history saved. [2025-10-20T17:17:43.047772] [INFO ] [MAIN ] Update Progress: 147/2367 symbols completed. ARMK took 34.8s. Estimated remaining: 4h 26m 28s. Rate: 8.3 symbols/min [2025-10-20T17:17:43.245371] [INFO ] [MAIN ] ARX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:43.245549] [DEBUG] [LOCAL ] ARX: No existing data file found in primary storage. [2025-10-20T17:17:43.245586] [WARN ] [MAIN ] ARX: No existing data found, performing full refresh instead. [2025-10-20T17:17:43.245730] [INFO ] [MAIN ] ARX: Wiping existing primary data before full refresh. [2025-10-20T17:17:43.245775] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-10-20T17:17:43.245815] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-10-20T17:17:43.245846] [INFO ] [MAIN ] ARX: Starting full primary refresh. [2025-10-20T17:17:43.458720] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-09. [2025-10-20T17:17:43.613773] [INFO ] [REMOTE ] ARX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:43.657609] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:17:43.697560] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T17:17:44.251378] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-09. [2025-10-20T17:17:45.051376] [INFO ] [REMOTE ] ARX: Fetching primary data for 2025-01. [2025-10-20T17:17:45.155734] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:17:45.259695] [DEBUG] [REMOTE ] ARX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:17:45.260040] [WARN ] [REMOTE ] ARX: No data returned for month 2025-01 [2025-10-20T17:17:45.260145] [FATAL] [REMOTE ] ARX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:17:45.260231] [FATAL] [MAIN ] ARX: Failed to fetch primary data. [2025-10-20T17:17:45.260310] [WARN ] [MAIN ] ARX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:17:45.260399] [INFO ] [LOCAL ] ARX: Starting deletion of primary data. [2025-10-20T17:17:45.260506] [DEBUG] [LOCAL ] ARX: No primary data file found to delete. [2025-10-20T17:17:45.260588] [INFO ] [LOCAL ] ARX: Starting deletion of derivative data. [2025-10-20T17:17:45.260671] [DEBUG] [LOCAL ] ARX: No derivative data file found to delete. [2025-10-20T17:17:45.260746] [INFO ] [MAIN ] Update Progress: 148/2367 symbols completed. ARX took 2.2s. Estimated remaining: 4h 25m 6s. Rate: 8.4 symbols/min [2025-10-20T17:17:45.484214] [INFO ] [MAIN ] AS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:45.540018] [DEBUG] [LOCAL ] AS: Loaded 53954 existing records from primary storage. [2025-10-20T17:17:45.540515] [INFO ] [MAIN ] AS: Loaded 53954 existing records. [2025-10-20T17:17:45.732790] [WARN ] [MAIN ] AS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:45.733091] [INFO ] [MAIN ] AS: Wiping existing primary data before full refresh. [2025-10-20T17:17:45.733156] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-10-20T17:17:45.733692] [DEBUG] [LOCAL ] AS: Deleted primary data file. [2025-10-20T17:17:45.733771] [INFO ] [MAIN ] AS: Starting full primary refresh. [2025-10-20T17:17:45.856790] [INFO ] [REMOTE ] AROC: Fetching primary data for 2025-10. [2025-10-20T17:17:46.176104] [INFO ] [REMOTE ] AS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:46.393575] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-10-20T17:17:46.651424] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-07. [2025-10-20T17:17:47.451422] [INFO ] [REMOTE ] ARR: Fetching primary data for 2025-10. [2025-10-20T17:17:47.654270] [INFO ] [REMOTE ] AROC: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:17:47.663527] [INFO ] [REMOTE ] AROC: Full fetch complete (71331 records). [2025-10-20T17:17:47.663786] [DEBUG] [VALIDATOR ] AROC: Validating primary data. [2025-10-20T17:17:48.030582] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:17:48.119571] [DEBUG] [VALIDATOR ] AROC: All expected trading days are present. [2025-10-20T17:17:48.119691] [DEBUG] [VALIDATOR ] AROC: Primary data validation passed. [2025-10-20T17:17:48.120420] [INFO ] [MAIN ] AROC: Fetched primary data is valid. [2025-10-20T17:17:48.120465] [DEBUG] [LOCAL ] AROC: Saving primary data. [2025-10-20T17:17:48.121898] [DEBUG] [LOCAL ] AROC: Trimmed 71331 records to 49092 (keeping 200 days of data) [2025-10-20T17:17:48.295332] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-01. [2025-10-20T17:17:48.420846] [DEBUG] [LOCAL ] AROC: Primary data saved (49092 records). [2025-10-20T17:17:48.420991] [INFO ] [MAIN ] AROC: Full history saved. [2025-10-20T17:17:48.421699] [INFO ] [MAIN ] Update Progress: 149/2367 symbols completed. AROC took 36.7s. Estimated remaining: 4h 23m 59s. Rate: 8.4 symbols/min [2025-10-20T17:17:48.597544] [INFO ] [MAIN ] ASA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:48.615873] [DEBUG] [LOCAL ] ASA: Loaded 15654 existing records from primary storage. [2025-10-20T17:17:48.616177] [INFO ] [MAIN ] ASA: Loaded 15654 existing records. [2025-10-20T17:17:48.794307] [WARN ] [MAIN ] ASA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:48.794547] [INFO ] [MAIN ] ASA: Wiping existing primary data before full refresh. [2025-10-20T17:17:48.794596] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-10-20T17:17:48.794773] [DEBUG] [LOCAL ] ASA: Deleted primary data file. [2025-10-20T17:17:48.794830] [INFO ] [MAIN ] ASA: Starting full primary refresh. [2025-10-20T17:17:49.056507] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-08. [2025-10-20T17:17:49.165504] [INFO ] [REMOTE ] ASA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:49.461321] [INFO ] [REMOTE ] ARR: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T17:17:49.471525] [INFO ] [REMOTE ] ARR: Full fetch complete (75097 records). [2025-10-20T17:17:49.471769] [DEBUG] [VALIDATOR ] ARR: Validating primary data. [2025-10-20T17:17:49.472862] [FATAL] [VALIDATOR ] ARR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:17:49.473002] [FATAL] [MAIN ] ARR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:17:49.473059] [WARN ] [MAIN ] ARR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:17:49.473139] [INFO ] [LOCAL ] ARR: Starting deletion of primary data. [2025-10-20T17:17:49.473223] [DEBUG] [LOCAL ] ARR: No primary data file found to delete. [2025-10-20T17:17:49.473273] [INFO ] [LOCAL ] ARR: Starting deletion of derivative data. [2025-10-20T17:17:49.473482] [DEBUG] [LOCAL ] ARR: Deleted derivative data file. [2025-10-20T17:17:49.473598] [INFO ] [MAIN ] Update Progress: 150/2367 symbols completed. ARR took 31.4s. Estimated remaining: 4h 22m 22s. Rate: 8.5 symbols/min [2025-10-20T17:17:49.672432] [INFO ] [MAIN ] ASAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:49.727369] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T17:17:49.754531] [DEBUG] [LOCAL ] ASAN: Loaded 53507 existing records from primary storage. [2025-10-20T17:17:49.754923] [INFO ] [MAIN ] ASAN: Loaded 53507 existing records. [2025-10-20T17:17:49.856512] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-01. [2025-10-20T17:17:49.938983] [WARN ] [MAIN ] ASAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:17:49.939181] [INFO ] [MAIN ] ASAN: Wiping existing primary data before full refresh. [2025-10-20T17:17:49.939228] [INFO ] [LOCAL ] ASAN: Starting deletion of primary data. [2025-10-20T17:17:49.939611] [DEBUG] [LOCAL ] ASAN: Deleted primary data file. [2025-10-20T17:17:49.939679] [INFO ] [MAIN ] ASAN: Starting full primary refresh. [2025-10-20T17:17:50.306268] [INFO ] [REMOTE ] ASAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:50.418492] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T17:17:50.584578] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:17:50.651473] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-01. [2025-10-20T17:17:51.451407] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-02. [2025-10-20T17:17:52.251361] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-09. [2025-10-20T17:17:52.602062] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T17:17:53.051351] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-02. [2025-10-20T17:17:53.161739] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T17:17:53.821927] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T17:17:53.851350] [INFO ] [REMOTE ] ARW: Fetching primary data for 2025-10. [2025-10-20T17:17:54.651356] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-03. [2025-10-20T17:17:54.972837] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T17:17:55.065788] [INFO ] [REMOTE ] ARW: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:17:55.071320] [INFO ] [REMOTE ] ARW: Full fetch complete (48613 records). [2025-10-20T17:17:55.071465] [DEBUG] [VALIDATOR ] ARW: Validating primary data. [2025-10-20T17:17:55.457727] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-03. [2025-10-20T17:17:55.463854] [DEBUG] [VALIDATOR ] ARW: All expected trading days are present. [2025-10-20T17:17:55.464891] [DEBUG] [VALIDATOR ] ARW: Primary data validation passed. [2025-10-20T17:17:55.465510] [INFO ] [MAIN ] ARW: Fetched primary data is valid. [2025-10-20T17:17:55.465584] [DEBUG] [LOCAL ] ARW: Saving primary data. [2025-10-20T17:17:55.467012] [DEBUG] [LOCAL ] ARW: Trimmed 48613 records to 33236 (keeping 200 days of data) [2025-10-20T17:17:55.664346] [DEBUG] [LOCAL ] ARW: Primary data saved (33236 records). [2025-10-20T17:17:55.664477] [INFO ] [MAIN ] ARW: Full history saved. [2025-10-20T17:17:55.664937] [INFO ] [MAIN ] Update Progress: 151/2367 symbols completed. ARW took 28.1s. Estimated remaining: 4h 22m 1s. Rate: 8.5 symbols/min [2025-10-20T17:17:55.836873] [INFO ] [MAIN ] ASB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:17:55.837071] [DEBUG] [LOCAL ] ASB: No existing data file found in primary storage. [2025-10-20T17:17:55.837110] [WARN ] [MAIN ] ASB: No existing data found, performing full refresh instead. [2025-10-20T17:17:55.837249] [INFO ] [MAIN ] ASB: Wiping existing primary data before full refresh. [2025-10-20T17:17:55.837318] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-10-20T17:17:55.837384] [DEBUG] [LOCAL ] ASB: No primary data file found to delete. [2025-10-20T17:17:55.837430] [INFO ] [MAIN ] ASB: Starting full primary refresh. [2025-10-20T17:17:56.211862] [INFO ] [REMOTE ] ASB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:17:56.251516] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:17:56.261807] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-02. [2025-10-20T17:17:57.051927] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-04. [2025-10-20T17:17:57.056417] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:17:57.081900] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:17:57.851398] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-03. [2025-10-20T17:17:58.651411] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-01. [2025-10-20T17:17:58.740825] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:17:58.844498] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:17:59.451397] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-04. [2025-10-20T17:18:00.185757] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T17:18:00.251340] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-02. [2025-10-20T17:18:01.051441] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-04. [2025-10-20T17:18:01.818123] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-04 (2.37 seconds). [2025-10-20T17:18:01.851628] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-05. [2025-10-20T17:18:01.903807] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:18:01.945583] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T17:18:02.651416] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-05. [2025-10-20T17:18:03.331030] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T17:18:03.451352] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-06. [2025-10-20T17:18:03.573084] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:18:04.005852] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T17:18:04.251394] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-05. [2025-10-20T17:18:05.051366] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-07. [2025-10-20T17:18:05.851352] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-03. [2025-10-20T17:18:05.998829] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:18:06.032748] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:18:06.651344] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-06. [2025-10-20T17:18:07.451351] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-06. [2025-10-20T17:18:07.583903] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T17:18:08.251423] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-08. [2025-10-20T17:18:08.312356] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:18:08.950709] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T17:18:09.051351] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-07. [2025-10-20T17:18:09.330067] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T17:18:09.851373] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-04. [2025-10-20T17:18:10.651401] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-07. [2025-10-20T17:18:10.983329] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:18:11.163094] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T17:18:11.451428] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-08. [2025-10-20T17:18:12.251358] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-05. [2025-10-20T17:18:12.698092] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T17:18:13.051354] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-08. [2025-10-20T17:18:13.336292] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T17:18:13.801471] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:18:13.851345] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-06. [2025-10-20T17:18:14.651429] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-09. [2025-10-20T17:18:14.893713] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:18:14.990194] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:18:15.451413] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-09. [2025-10-20T17:18:16.251404] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-09. [2025-10-20T17:18:16.417722] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:18:17.051363] [INFO ] [REMOTE ] AS: Fetching primary data for 2025-10. [2025-10-20T17:18:17.268324] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T17:18:17.601590] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:18:17.851387] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-07. [2025-10-20T17:18:18.651368] [INFO ] [REMOTE ] ASAN: Fetching primary data for 2025-10. [2025-10-20T17:18:19.148257] [INFO ] [REMOTE ] AS: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:18:19.156053] [INFO ] [REMOTE ] AS: Full fetch complete (76526 records). [2025-10-20T17:18:19.156207] [DEBUG] [VALIDATOR ] AS: Validating primary data. [2025-10-20T17:18:19.157043] [FATAL] [VALIDATOR ] AS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:19.157135] [FATAL] [MAIN ] AS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:19.157178] [WARN ] [MAIN ] AS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:19.157217] [INFO ] [LOCAL ] AS: Starting deletion of primary data. [2025-10-20T17:18:19.157292] [DEBUG] [LOCAL ] AS: No primary data file found to delete. [2025-10-20T17:18:19.157329] [INFO ] [LOCAL ] AS: Starting deletion of derivative data. [2025-10-20T17:18:19.157483] [DEBUG] [LOCAL ] AS: Deleted derivative data file. [2025-10-20T17:18:19.157573] [INFO ] [MAIN ] Update Progress: 152/2367 symbols completed. AS took 33.9s. Estimated remaining: 4h 25m 53s. Rate: 8.3 symbols/min [2025-10-20T17:18:19.355671] [INFO ] [MAIN ] ASBA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:19.361912] [DEBUG] [LOCAL ] ASBA: Loaded 3617 existing records from primary storage. [2025-10-20T17:18:19.362128] [INFO ] [MAIN ] ASBA: Loaded 3617 existing records. [2025-10-20T17:18:19.458377] [INFO ] [REMOTE ] ASA: Fetching primary data for 2025-10. [2025-10-20T17:18:19.545095] [WARN ] [MAIN ] ASBA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:19.545365] [INFO ] [MAIN ] ASBA: Wiping existing primary data before full refresh. [2025-10-20T17:18:19.545531] [INFO ] [LOCAL ] ASBA: Starting deletion of primary data. [2025-10-20T17:18:19.545781] [DEBUG] [LOCAL ] ASBA: Deleted primary data file. [2025-10-20T17:18:19.545918] [INFO ] [MAIN ] ASBA: Starting full primary refresh. [2025-10-20T17:18:19.770572] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:18:19.956980] [INFO ] [REMOTE ] ASBA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:20.251377] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-08. [2025-10-20T17:18:20.507864] [INFO ] [REMOTE ] ASA: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:18:20.512425] [INFO ] [REMOTE ] ASA: Full fetch complete (20356 records). [2025-10-20T17:18:20.512515] [DEBUG] [VALIDATOR ] ASA: Validating primary data. [2025-10-20T17:18:20.512987] [FATAL] [VALIDATOR ] ASA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:20.513073] [FATAL] [MAIN ] ASA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:20.513115] [WARN ] [MAIN ] ASA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:20.513152] [INFO ] [LOCAL ] ASA: Starting deletion of primary data. [2025-10-20T17:18:20.513213] [DEBUG] [LOCAL ] ASA: No primary data file found to delete. [2025-10-20T17:18:20.513271] [INFO ] [LOCAL ] ASA: Starting deletion of derivative data. [2025-10-20T17:18:20.513423] [DEBUG] [LOCAL ] ASA: Deleted derivative data file. [2025-10-20T17:18:20.513509] [INFO ] [MAIN ] Update Progress: 153/2367 symbols completed. ASA took 32.1s. Estimated remaining: 4h 24m 21s. Rate: 8.4 symbols/min [2025-10-20T17:18:20.687525] [INFO ] [MAIN ] ASC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:20.722219] [DEBUG] [LOCAL ] ASC: Loaded 37898 existing records from primary storage. [2025-10-20T17:18:20.722521] [INFO ] [MAIN ] ASC: Loaded 37898 existing records. [2025-10-20T17:18:20.909256] [WARN ] [MAIN ] ASC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:20.909522] [INFO ] [MAIN ] ASC: Wiping existing primary data before full refresh. [2025-10-20T17:18:20.909614] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-10-20T17:18:20.909943] [DEBUG] [LOCAL ] ASC: Deleted primary data file. [2025-10-20T17:18:20.910008] [INFO ] [MAIN ] ASC: Starting full primary refresh. [2025-10-20T17:18:21.031740] [INFO ] [REMOTE ] ASAN: Incremental fetch complete for 2025-10 (2.38 seconds). [2025-10-20T17:18:21.058223] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-01. [2025-10-20T17:18:21.068974] [INFO ] [REMOTE ] ASAN: Full fetch complete (76635 records). [2025-10-20T17:18:21.095020] [DEBUG] [VALIDATOR ] ASAN: Validating primary data. [2025-10-20T17:18:21.588019] [INFO ] [REMOTE ] ASC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:21.750170] [DEBUG] [VALIDATOR ] ASAN: All expected trading days are present. [2025-10-20T17:18:21.750255] [DEBUG] [VALIDATOR ] ASAN: Primary data validation passed. [2025-10-20T17:18:21.750965] [INFO ] [MAIN ] ASAN: Fetched primary data is valid. [2025-10-20T17:18:21.751030] [DEBUG] [LOCAL ] ASAN: Saving primary data. [2025-10-20T17:18:21.752607] [DEBUG] [LOCAL ] ASAN: Trimmed 76635 records to 52732 (keeping 200 days of data) [2025-10-20T17:18:21.854714] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-01. [2025-10-20T17:18:21.975083] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:18:22.094252] [DEBUG] [LOCAL ] ASAN: Primary data saved (52732 records). [2025-10-20T17:18:22.094492] [INFO ] [MAIN ] ASAN: Full history saved. [2025-10-20T17:18:22.094583] [INFO ] [MAIN ] Update Progress: 154/2367 symbols completed. ASAN took 32.6s. Estimated remaining: 4h 22m 54s. Rate: 8.4 symbols/min [2025-10-20T17:18:22.263329] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T17:18:22.336748] [INFO ] [MAIN ] ASG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:22.354585] [DEBUG] [LOCAL ] ASG: Loaded 19812 existing records from primary storage. [2025-10-20T17:18:22.354867] [INFO ] [MAIN ] ASG: Loaded 19812 existing records. [2025-10-20T17:18:22.541146] [WARN ] [MAIN ] ASG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:22.541371] [INFO ] [MAIN ] ASG: Wiping existing primary data before full refresh. [2025-10-20T17:18:22.541420] [INFO ] [LOCAL ] ASG: Starting deletion of primary data. [2025-10-20T17:18:22.541686] [DEBUG] [LOCAL ] ASG: Deleted primary data file. [2025-10-20T17:18:22.541746] [INFO ] [MAIN ] ASG: Starting full primary refresh. [2025-10-20T17:18:22.656550] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-02. [2025-10-20T17:18:22.964842] [INFO ] [REMOTE ] ASG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:23.173088] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:18:23.451373] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-01. [2025-10-20T17:18:23.716023] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:18:24.251437] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-09. [2025-10-20T17:18:24.430542] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:18:25.051366] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-02. [2025-10-20T17:18:25.554155] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:18:25.851384] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-02. [2025-10-20T17:18:26.082742] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T17:18:26.651367] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-03. [2025-10-20T17:18:27.451413] [INFO ] [REMOTE ] ASB: Fetching primary data for 2025-10. [2025-10-20T17:18:27.500683] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:18:27.538375] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:18:28.251393] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-03. [2025-10-20T17:18:29.051427] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-03. [2025-10-20T17:18:29.177611] [INFO ] [REMOTE ] ASB: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:18:29.184974] [INFO ] [REMOTE ] ASB: Full fetch complete (71771 records). [2025-10-20T17:18:29.185115] [DEBUG] [VALIDATOR ] ASB: Validating primary data. [2025-10-20T17:18:29.185888] [FATAL] [VALIDATOR ] ASB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:29.185974] [FATAL] [MAIN ] ASB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:29.186018] [WARN ] [MAIN ] ASB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:29.186057] [INFO ] [LOCAL ] ASB: Starting deletion of primary data. [2025-10-20T17:18:29.186112] [DEBUG] [LOCAL ] ASB: No primary data file found to delete. [2025-10-20T17:18:29.186148] [INFO ] [LOCAL ] ASB: Starting deletion of derivative data. [2025-10-20T17:18:29.186188] [DEBUG] [LOCAL ] ASB: No derivative data file found to delete. [2025-10-20T17:18:29.186243] [INFO ] [MAIN ] Update Progress: 155/2367 symbols completed. ASB took 33.5s. Estimated remaining: 4h 22m 46s. Rate: 8.4 symbols/min [2025-10-20T17:18:29.362001] [INFO ] [MAIN ] ASGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:29.385215] [DEBUG] [LOCAL ] ASGI: Loaded 23311 existing records from primary storage. [2025-10-20T17:18:29.385494] [INFO ] [MAIN ] ASGI: Loaded 23311 existing records. [2025-10-20T17:18:29.576463] [WARN ] [MAIN ] ASGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:29.580945] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-10-20T17:18:29.582968] [INFO ] [MAIN ] ASGI: Wiping existing primary data before full refresh. [2025-10-20T17:18:29.583150] [INFO ] [LOCAL ] ASGI: Starting deletion of primary data. [2025-10-20T17:18:29.583352] [DEBUG] [LOCAL ] ASGI: Deleted primary data file. [2025-10-20T17:18:29.583392] [INFO ] [MAIN ] ASGI: Starting full primary refresh. [2025-10-20T17:18:29.860039] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-04. [2025-10-20T17:18:29.956926] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:18:29.997662] [INFO ] [REMOTE ] ASGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:30.307764] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T17:18:30.651401] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-05. [2025-10-20T17:18:31.126490] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:18:31.451370] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-06. [2025-10-20T17:18:32.021584] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T17:18:32.251373] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-04. [2025-10-20T17:18:33.051419] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-01. [2025-10-20T17:18:33.558986] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T17:18:33.851388] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-04. [2025-10-20T17:18:34.300443] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T17:18:34.651366] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-02. [2025-10-20T17:18:35.151795] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:18:35.371631] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T17:18:35.451332] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-03. [2025-10-20T17:18:36.251351] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-05. [2025-10-20T17:18:36.405571] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:18:37.051393] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-05. [2025-10-20T17:18:37.708506] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:18:37.851374] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-06. [2025-10-20T17:18:38.523915] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:18:38.651362] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-04. [2025-10-20T17:18:38.858988] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:18:39.454298] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-07. [2025-10-20T17:18:39.459090] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:18:40.251371] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-06. [2025-10-20T17:18:40.509394] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T17:18:41.051385] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-05. [2025-10-20T17:18:41.727007] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:18:41.851360] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-07. [2025-10-20T17:18:41.880022] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:18:42.503150] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T17:18:42.651361] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-08. [2025-10-20T17:18:43.061804] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T17:18:43.451416] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-08. [2025-10-20T17:18:44.251356] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-07. [2025-10-20T17:18:44.429481] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:18:45.051361] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-09. [2025-10-20T17:18:45.495722] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T17:18:45.851348] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-06. [2025-10-20T17:18:46.253334] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T17:18:46.651425] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-09. [2025-10-20T17:18:46.784186] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:18:47.451402] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-08. [2025-10-20T17:18:47.584096] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T17:18:48.251429] [INFO ] [REMOTE ] ASG: Fetching primary data for 2025-10. [2025-10-20T17:18:48.840031] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:18:49.051403] [INFO ] [REMOTE ] ASBA: Fetching primary data for 2025-10. [2025-10-20T17:18:49.251913] [INFO ] [REMOTE ] ASG: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:18:49.254987] [INFO ] [REMOTE ] ASG: Full fetch complete (30907 records). [2025-10-20T17:18:49.255081] [DEBUG] [VALIDATOR ] ASG: Validating primary data. [2025-10-20T17:18:49.617435] [DEBUG] [VALIDATOR ] ASG: All expected trading days are present. [2025-10-20T17:18:49.617513] [DEBUG] [VALIDATOR ] ASG: Primary data validation passed. [2025-10-20T17:18:49.617917] [INFO ] [MAIN ] ASG: Fetched primary data is valid. [2025-10-20T17:18:49.617979] [DEBUG] [LOCAL ] ASG: Saving primary data. [2025-10-20T17:18:49.618718] [DEBUG] [LOCAL ] ASG: Trimmed 30907 records to 19504 (keeping 200 days of data) [2025-10-20T17:18:49.709326] [INFO ] [REMOTE ] ASBA: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T17:18:49.711722] [INFO ] [REMOTE ] ASBA: Full fetch complete (5279 records). [2025-10-20T17:18:49.712296] [DEBUG] [VALIDATOR ] ASBA: Validating primary data. [2025-10-20T17:18:49.831967] [DEBUG] [LOCAL ] ASG: Primary data saved (19504 records). [2025-10-20T17:18:49.834537] [INFO ] [MAIN ] ASG: Full history saved. [2025-10-20T17:18:49.834628] [INFO ] [MAIN ] Update Progress: 156/2367 symbols completed. ASG took 27.7s. Estimated remaining: 4h 25m 51s. Rate: 8.3 symbols/min [2025-10-20T17:18:49.860304] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-07. [2025-10-20T17:18:50.206240] [INFO ] [MAIN ] ASGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:18:50.272887] [DEBUG] [VALIDATOR ] ASBA: All expected trading days are present. [2025-10-20T17:18:50.272968] [DEBUG] [VALIDATOR ] ASBA: Primary data validation passed. [2025-10-20T17:18:50.273121] [INFO ] [MAIN ] ASBA: Fetched primary data is valid. [2025-10-20T17:18:50.273159] [DEBUG] [LOCAL ] ASBA: Saving primary data. [2025-10-20T17:18:50.273516] [DEBUG] [LOCAL ] ASBA: Trimmed 5279 records to 3520 (keeping 200 days of data) [2025-10-20T17:18:50.306483] [DEBUG] [LOCAL ] ASBA: Primary data saved (3520 records). [2025-10-20T17:18:50.306585] [INFO ] [MAIN ] ASBA: Full history saved. [2025-10-20T17:18:50.306690] [INFO ] [MAIN ] Update Progress: 157/2367 symbols completed. ASBA took 31.1s. Estimated remaining: 4h 24m 9s. Rate: 8.4 symbols/min [2025-10-20T17:18:50.320832] [DEBUG] [LOCAL ] ASGN: Loaded 37201 existing records from primary storage. [2025-10-20T17:18:50.321111] [INFO ] [MAIN ] ASGN: Loaded 37201 existing records. [2025-10-20T17:18:50.517620] [WARN ] [MAIN ] ASGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:18:50.517940] [INFO ] [MAIN ] ASGN: Wiping existing primary data before full refresh. [2025-10-20T17:18:50.517994] [INFO ] [LOCAL ] ASGN: Starting deletion of primary data. [2025-10-20T17:18:50.518227] [DEBUG] [LOCAL ] ASGN: Deleted primary data file. [2025-10-20T17:18:50.518270] [INFO ] [MAIN ] ASGN: Starting full primary refresh. [2025-10-20T17:18:50.656562] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-09. [2025-10-20T17:18:50.955941] [INFO ] [REMOTE ] ASGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:18:51.024974] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:18:51.451385] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-08. [2025-10-20T17:18:52.120676] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:18:52.251370] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-01. [2025-10-20T17:18:52.562788] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:18:53.051428] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-09. [2025-10-20T17:18:53.776805] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T17:18:53.851360] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-02. [2025-10-20T17:18:54.161410] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:18:54.651394] [INFO ] [REMOTE ] ASGI: Fetching primary data for 2025-10. [2025-10-20T17:18:55.250577] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:18:55.451239] [INFO ] [REMOTE ] ASGI: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T17:18:55.451427] [INFO ] [REMOTE ] ASC: Fetching primary data for 2025-10. [2025-10-20T17:18:55.456075] [INFO ] [REMOTE ] ASGI: Full fetch complete (33240 records). [2025-10-20T17:18:55.456145] [DEBUG] [VALIDATOR ] ASGI: Validating primary data. [2025-10-20T17:18:55.830933] [DEBUG] [VALIDATOR ] ASGI: All expected trading days are present. [2025-10-20T17:18:55.831056] [DEBUG] [VALIDATOR ] ASGI: Primary data validation passed. [2025-10-20T17:18:55.831493] [INFO ] [MAIN ] ASGI: Fetched primary data is valid. [2025-10-20T17:18:55.831535] [DEBUG] [LOCAL ] ASGI: Saving primary data. [2025-10-20T17:18:55.832341] [DEBUG] [LOCAL ] ASGI: Trimmed 33240 records to 23006 (keeping 200 days of data) [2025-10-20T17:18:55.968375] [DEBUG] [LOCAL ] ASGI: Primary data saved (23006 records). [2025-10-20T17:18:55.968549] [INFO ] [MAIN ] ASGI: Full history saved. [2025-10-20T17:18:55.968631] [INFO ] [MAIN ] Update Progress: 158/2367 symbols completed. ASGI took 26.8s. Estimated remaining: 4h 23m 40s. Rate: 8.4 symbols/min [2025-10-20T17:18:56.251463] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-03. [2025-10-20T17:18:57.089065] [INFO ] [REMOTE ] ASC: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:18:57.094378] [INFO ] [REMOTE ] ASC: Full fetch complete (55853 records). [2025-10-20T17:18:57.094480] [DEBUG] [VALIDATOR ] ASC: Validating primary data. [2025-10-20T17:18:57.095235] [FATAL] [VALIDATOR ] ASC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:18:57.095326] [FATAL] [MAIN ] ASC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:18:57.095394] [WARN ] [MAIN ] ASC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:18:57.095474] [INFO ] [LOCAL ] ASC: Starting deletion of primary data. [2025-10-20T17:18:57.095531] [DEBUG] [LOCAL ] ASC: No primary data file found to delete. [2025-10-20T17:18:57.095569] [INFO ] [LOCAL ] ASC: Starting deletion of derivative data. [2025-10-20T17:18:57.095739] [DEBUG] [LOCAL ] ASC: Deleted derivative data file. [2025-10-20T17:18:57.096340] [INFO ] [MAIN ] Update Progress: 159/2367 symbols completed. ASC took 36.6s. Estimated remaining: 4h 22m 9s. Rate: 8.4 symbols/min [2025-10-20T17:18:57.753347] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:18:57.761349] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-04. [2025-10-20T17:18:58.865237] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:18:58.877781] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-05. [2025-10-20T17:19:00.294964] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:19:00.302078] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-06. [2025-10-20T17:19:01.994602] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:19:02.002441] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-07. [2025-10-20T17:19:03.871625] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T17:19:03.881138] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-08. [2025-10-20T17:19:06.577323] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-08 (2.70 seconds). [2025-10-20T17:19:06.585703] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-09. [2025-10-20T17:19:08.265102] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:19:08.273997] [INFO ] [REMOTE ] ASGN: Fetching primary data for 2025-10. [2025-10-20T17:19:09.889024] [INFO ] [REMOTE ] ASGN: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:19:09.895211] [INFO ] [REMOTE ] ASGN: Full fetch complete (53460 records). [2025-10-20T17:19:09.895312] [DEBUG] [VALIDATOR ] ASGN: Validating primary data. [2025-10-20T17:19:10.293148] [DEBUG] [VALIDATOR ] ASGN: All expected trading days are present. [2025-10-20T17:19:10.293246] [DEBUG] [VALIDATOR ] ASGN: Primary data validation passed. [2025-10-20T17:19:10.293863] [INFO ] [MAIN ] ASGN: Fetched primary data is valid. [2025-10-20T17:19:10.293925] [DEBUG] [LOCAL ] ASGN: Saving primary data. [2025-10-20T17:19:10.295181] [DEBUG] [LOCAL ] ASGN: Trimmed 53460 records to 36724 (keeping 200 days of data) [2025-10-20T17:19:10.503123] [DEBUG] [LOCAL ] ASGN: Primary data saved (36724 records). [2025-10-20T17:19:10.503260] [INFO ] [MAIN ] ASGN: Full history saved. [2025-10-20T17:19:10.503341] [INFO ] [MAIN ] Update Progress: 160/2367 symbols completed. ASGN took 20.7s. Estimated remaining: 4h 23m 29s. Rate: 8.4 symbols/min [2025-10-20T17:19:11.127178] [INFO ] [MAIN ] ASR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.206121] [INFO ] [MAIN ] ASH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.209487] [INFO ] [MAIN ] ASPN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.233857] [INFO ] [MAIN ] ASIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:11.244419] [DEBUG] [LOCAL ] ASR: Loaded 9527 existing records from primary storage. [2025-10-20T17:19:11.244851] [INFO ] [MAIN ] ASR: Loaded 9527 existing records. [2025-10-20T17:19:11.417133] [DEBUG] [LOCAL ] ASIX: Loaded 23435 existing records from primary storage. [2025-10-20T17:19:11.447091] [INFO ] [MAIN ] ASIX: Loaded 23435 existing records. [2025-10-20T17:19:11.492504] [WARN ] [MAIN ] ASR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:11.522997] [INFO ] [MAIN ] ASR: Wiping existing primary data before full refresh. [2025-10-20T17:19:11.556247] [DEBUG] [LOCAL ] ASH: Loaded 38148 existing records from primary storage. [2025-10-20T17:19:11.550459] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-10-20T17:19:11.561764] [DEBUG] [LOCAL ] ASR: Deleted primary data file. [2025-10-20T17:19:11.579575] [INFO ] [MAIN ] ASR: Starting full primary refresh. [2025-10-20T17:19:11.620688] [INFO ] [MAIN ] ASH: Loaded 38148 existing records. [2025-10-20T17:19:11.727926] [DEBUG] [LOCAL ] ASPN: Loaded 50648 existing records from primary storage. [2025-10-20T17:19:11.862029] [INFO ] [MAIN ] ASPN: Loaded 50648 existing records. [2025-10-20T17:19:12.035289] [WARN ] [MAIN ] ASIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.134912] [WARN ] [MAIN ] ASH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.142002] [INFO ] [MAIN ] ASIX: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.194484] [INFO ] [LOCAL ] ASIX: Starting deletion of primary data. [2025-10-20T17:19:12.199720] [INFO ] [MAIN ] ASH: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.257732] [INFO ] [LOCAL ] ASH: Starting deletion of primary data. [2025-10-20T17:19:12.278475] [DEBUG] [LOCAL ] ASIX: Deleted primary data file. [2025-10-20T17:19:12.288947] [INFO ] [MAIN ] ASIX: Starting full primary refresh. [2025-10-20T17:19:12.397266] [DEBUG] [LOCAL ] ASH: Deleted primary data file. [2025-10-20T17:19:12.413144] [INFO ] [MAIN ] ASH: Starting full primary refresh. [2025-10-20T17:19:12.391960] [WARN ] [MAIN ] ASPN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:12.511561] [INFO ] [MAIN ] ASPN: Wiping existing primary data before full refresh. [2025-10-20T17:19:12.650491] [INFO ] [LOCAL ] ASPN: Starting deletion of primary data. [2025-10-20T17:19:12.659541] [INFO ] [REMOTE ] ASR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:12.699669] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-01. [2025-10-20T17:19:12.699752] [DEBUG] [LOCAL ] ASPN: Deleted primary data file. [2025-10-20T17:19:12.743822] [INFO ] [MAIN ] ASPN: Starting full primary refresh. [2025-10-20T17:19:13.424722] [INFO ] [REMOTE ] ASIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.444115] [INFO ] [REMOTE ] ASH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.468927] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-01. [2025-10-20T17:19:13.495331] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-01. [2025-10-20T17:19:13.599454] [INFO ] [REMOTE ] ASPN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:13.602206] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-01. [2025-10-20T17:19:13.605764] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:19:13.607766] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-02. [2025-10-20T17:19:14.260332] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T17:19:14.264345] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-02. [2025-10-20T17:19:14.351464] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T17:19:14.353750] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-03. [2025-10-20T17:19:14.725010] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T17:19:14.733326] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-02. [2025-10-20T17:19:14.889596] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T17:19:14.891876] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-04. [2025-10-20T17:19:15.210800] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:19:15.214395] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-03. [2025-10-20T17:19:15.454939] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:19:15.458591] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-05. [2025-10-20T17:19:15.501896] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:19:15.512683] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-02. [2025-10-20T17:19:15.849435] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:19:15.857397] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-03. [2025-10-20T17:19:15.872609] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:19:15.877525] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-04. [2025-10-20T17:19:15.946345] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T17:19:15.950128] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-06. [2025-10-20T17:19:16.736940] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:19:16.744351] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-05. [2025-10-20T17:19:16.784134] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T17:19:16.787578] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-07. [2025-10-20T17:19:17.052402] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:19:17.062142] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-04. [2025-10-20T17:19:17.265660] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:19:17.275631] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-03. [2025-10-20T17:19:17.387988] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:19:17.552819] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:19:17.851376] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-06. [2025-10-20T17:19:18.459590] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:19:18.493075] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:19:18.657741] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-05. [2025-10-20T17:19:18.663377] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:19:19.451387] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-04. [2025-10-20T17:19:19.946082] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T17:19:20.251370] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-08. [2025-10-20T17:19:21.045281] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T17:19:21.051320] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-09. [2025-10-20T17:19:21.173635] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:19:21.851407] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-06. [2025-10-20T17:19:21.883619] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:19:22.651376] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-07. [2025-10-20T17:19:22.844874] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:19:23.451342] [INFO ] [REMOTE ] ASR: Fetching primary data for 2025-10. [2025-10-20T17:19:23.855412] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:19:24.251371] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-05. [2025-10-20T17:19:24.424346] [INFO ] [REMOTE ] ASR: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:19:24.425873] [INFO ] [REMOTE ] ASR: Full fetch complete (12384 records). [2025-10-20T17:19:24.425942] [DEBUG] [VALIDATOR ] ASR: Validating primary data. [2025-10-20T17:19:24.426323] [FATAL] [VALIDATOR ] ASR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:19:24.426405] [FATAL] [MAIN ] ASR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:19:24.426448] [WARN ] [MAIN ] ASR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:19:24.426486] [INFO ] [LOCAL ] ASR: Starting deletion of primary data. [2025-10-20T17:19:24.426538] [DEBUG] [LOCAL ] ASR: No primary data file found to delete. [2025-10-20T17:19:24.426580] [INFO ] [LOCAL ] ASR: Starting deletion of derivative data. [2025-10-20T17:19:24.426769] [DEBUG] [LOCAL ] ASR: Deleted derivative data file. [2025-10-20T17:19:24.426852] [INFO ] [MAIN ] Update Progress: 161/2367 symbols completed. ASR took 13.9s. Estimated remaining: 4h 24m 54s. Rate: 8.3 symbols/min [2025-10-20T17:19:24.594060] [INFO ] [MAIN ] ASX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:24.642752] [DEBUG] [LOCAL ] ASX: Loaded 53653 existing records from primary storage. [2025-10-20T17:19:24.643057] [INFO ] [MAIN ] ASX: Loaded 53653 existing records. [2025-10-20T17:19:24.822689] [WARN ] [MAIN ] ASX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:24.822918] [INFO ] [MAIN ] ASX: Wiping existing primary data before full refresh. [2025-10-20T17:19:24.822977] [INFO ] [LOCAL ] ASX: Starting deletion of primary data. [2025-10-20T17:19:24.823210] [DEBUG] [LOCAL ] ASX: Deleted primary data file. [2025-10-20T17:19:24.823267] [INFO ] [MAIN ] ASX: Starting full primary refresh. [2025-10-20T17:19:25.062563] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-07. [2025-10-20T17:19:25.179127] [INFO ] [REMOTE ] ASX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:25.851419] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-08. [2025-10-20T17:19:25.994362] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:19:26.599013] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:19:26.651420] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-06. [2025-10-20T17:19:27.451371] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-08. [2025-10-20T17:19:27.468518] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:19:28.251387] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-01. [2025-10-20T17:19:28.634779] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T17:19:28.894318] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T17:19:29.051369] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-09. [2025-10-20T17:19:29.856873] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-07. [2025-10-20T17:19:29.875510] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:19:30.414995] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:19:30.651370] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-09. [2025-10-20T17:19:31.357923] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:19:31.451396] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-08. [2025-10-20T17:19:31.705895] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:19:32.251375] [INFO ] [REMOTE ] ASIX: Fetching primary data for 2025-10. [2025-10-20T17:19:32.848065] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:19:33.051332] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-09. [2025-10-20T17:19:33.113605] [INFO ] [REMOTE ] ASIX: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T17:19:33.117408] [INFO ] [REMOTE ] ASIX: Full fetch complete (31091 records). [2025-10-20T17:19:33.117477] [DEBUG] [VALIDATOR ] ASIX: Validating primary data. [2025-10-20T17:19:33.477755] [DEBUG] [VALIDATOR ] ASIX: All expected trading days are present. [2025-10-20T17:19:33.477830] [DEBUG] [VALIDATOR ] ASIX: Primary data validation passed. [2025-10-20T17:19:33.478216] [INFO ] [MAIN ] ASIX: Fetched primary data is valid. [2025-10-20T17:19:33.478254] [DEBUG] [LOCAL ] ASIX: Saving primary data. [2025-10-20T17:19:33.479016] [DEBUG] [LOCAL ] ASIX: Trimmed 31091 records to 23205 (keeping 200 days of data) [2025-10-20T17:19:33.610832] [DEBUG] [LOCAL ] ASIX: Primary data saved (23205 records). [2025-10-20T17:19:33.610961] [INFO ] [MAIN ] ASIX: Full history saved. [2025-10-20T17:19:33.611520] [INFO ] [MAIN ] Update Progress: 162/2367 symbols completed. ASIX took 23.1s. Estimated remaining: 4h 25m 14s. Rate: 8.3 symbols/min [2025-10-20T17:19:33.781162] [INFO ] [MAIN ] ATEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:33.817993] [DEBUG] [LOCAL ] ATEN: Loaded 38815 existing records from primary storage. [2025-10-20T17:19:33.818309] [INFO ] [MAIN ] ATEN: Loaded 38815 existing records. [2025-10-20T17:19:33.860303] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-02. [2025-10-20T17:19:33.997452] [WARN ] [MAIN ] ATEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:33.997693] [INFO ] [MAIN ] ATEN: Wiping existing primary data before full refresh. [2025-10-20T17:19:33.997749] [INFO ] [LOCAL ] ATEN: Starting deletion of primary data. [2025-10-20T17:19:33.997995] [DEBUG] [LOCAL ] ATEN: Deleted primary data file. [2025-10-20T17:19:33.998033] [INFO ] [MAIN ] ATEN: Starting full primary refresh. [2025-10-20T17:19:34.344449] [INFO ] [REMOTE ] ATEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:34.651348] [INFO ] [REMOTE ] ASH: Fetching primary data for 2025-10. [2025-10-20T17:19:34.830256] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:19:35.460468] [INFO ] [REMOTE ] ASPN: Fetching primary data for 2025-10. [2025-10-20T17:19:35.467728] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:19:36.150004] [INFO ] [REMOTE ] ASH: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T17:19:36.155919] [INFO ] [REMOTE ] ASH: Full fetch complete (55741 records). [2025-10-20T17:19:36.156019] [DEBUG] [VALIDATOR ] ASH: Validating primary data. [2025-10-20T17:19:36.256470] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-01. [2025-10-20T17:19:36.546668] [DEBUG] [VALIDATOR ] ASH: All expected trading days are present. [2025-10-20T17:19:36.546754] [DEBUG] [VALIDATOR ] ASH: Primary data validation passed. [2025-10-20T17:19:36.547321] [INFO ] [MAIN ] ASH: Fetched primary data is valid. [2025-10-20T17:19:36.547360] [DEBUG] [LOCAL ] ASH: Saving primary data. [2025-10-20T17:19:36.548460] [DEBUG] [LOCAL ] ASH: Trimmed 55741 records to 37563 (keeping 200 days of data) [2025-10-20T17:19:36.772757] [DEBUG] [LOCAL ] ASH: Primary data saved (37563 records). [2025-10-20T17:19:36.772904] [INFO ] [MAIN ] ASH: Full history saved. [2025-10-20T17:19:36.772982] [INFO ] [MAIN ] Update Progress: 163/2367 symbols completed. ASH took 26.3s. Estimated remaining: 4h 24m 12s. Rate: 8.3 symbols/min [2025-10-20T17:19:36.943000] [INFO ] [MAIN ] ATGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:36.975349] [DEBUG] [LOCAL ] ATGE: Loaded 31377 existing records from primary storage. [2025-10-20T17:19:36.975735] [INFO ] [MAIN ] ATGE: Loaded 31377 existing records. [2025-10-20T17:19:37.052639] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-03. [2025-10-20T17:19:37.168103] [WARN ] [MAIN ] ATGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:19:37.191616] [INFO ] [MAIN ] ATGE: Wiping existing primary data before full refresh. [2025-10-20T17:19:37.191782] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-10-20T17:19:37.193305] [INFO ] [REMOTE ] ASPN: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:19:37.193353] [DEBUG] [LOCAL ] ATGE: Deleted primary data file. [2025-10-20T17:19:37.200905] [INFO ] [MAIN ] ATGE: Starting full primary refresh. [2025-10-20T17:19:37.206477] [INFO ] [REMOTE ] ASPN: Full fetch complete (72615 records). [2025-10-20T17:19:37.217621] [DEBUG] [VALIDATOR ] ASPN: Validating primary data. [2025-10-20T17:19:38.026864] [DEBUG] [VALIDATOR ] ASPN: All expected trading days are present. [2025-10-20T17:19:38.050832] [DEBUG] [VALIDATOR ] ASPN: Primary data validation passed. [2025-10-20T17:19:38.051666] [INFO ] [MAIN ] ASPN: Fetched primary data is valid. [2025-10-20T17:19:38.050572] [INFO ] [REMOTE ] ATGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:38.051926] [DEBUG] [LOCAL ] ASPN: Saving primary data. [2025-10-20T17:19:38.052345] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-01. [2025-10-20T17:19:38.076796] [DEBUG] [LOCAL ] ASPN: Trimmed 72615 records to 49915 (keeping 200 days of data) [2025-10-20T17:19:38.084970] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:19:38.371492] [DEBUG] [LOCAL ] ASPN: Primary data saved (49915 records). [2025-10-20T17:19:38.371675] [INFO ] [MAIN ] ASPN: Full history saved. [2025-10-20T17:19:38.371775] [INFO ] [MAIN ] Update Progress: 164/2367 symbols completed. ASPN took 27.9s. Estimated remaining: 4h 22m 50s. Rate: 8.4 symbols/min [2025-10-20T17:19:38.550061] [INFO ] [MAIN ] ATHM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:19:38.550326] [DEBUG] [LOCAL ] ATHM: No existing data file found in primary storage. [2025-10-20T17:19:38.550476] [WARN ] [MAIN ] ATHM: No existing data found, performing full refresh instead. [2025-10-20T17:19:38.550888] [INFO ] [MAIN ] ATHM: Wiping existing primary data before full refresh. [2025-10-20T17:19:38.551050] [INFO ] [LOCAL ] ATHM: Starting deletion of primary data. [2025-10-20T17:19:38.551202] [DEBUG] [LOCAL ] ATHM: No primary data file found to delete. [2025-10-20T17:19:38.551268] [INFO ] [MAIN ] ATHM: Starting full primary refresh. [2025-10-20T17:19:38.656678] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-02. [2025-10-20T17:19:38.905726] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:19:38.990356] [INFO ] [REMOTE ] ATHM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:19:39.451373] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-04. [2025-10-20T17:19:39.626227] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:19:40.180953] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:19:40.251381] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-02. [2025-10-20T17:19:41.003437] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:19:41.051404] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-01. [2025-10-20T17:19:41.637335] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:19:41.851366] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-03. [2025-10-20T17:19:42.613279] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:19:42.651367] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-02. [2025-10-20T17:19:43.272339] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:19:43.451427] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-05. [2025-10-20T17:19:43.751245] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:19:44.251443] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-03. [2025-10-20T17:19:45.051355] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-03. [2025-10-20T17:19:45.552450] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-05 (2.10 seconds). [2025-10-20T17:19:45.621583] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:19:45.851422] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-04. [2025-10-20T17:19:46.538818] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:19:46.651382] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-04. [2025-10-20T17:19:47.013748] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T17:19:47.451456] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-04. [2025-10-20T17:19:47.869574] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:19:48.251460] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-05. [2025-10-20T17:19:49.029357] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:19:49.051334] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-05. [2025-10-20T17:19:49.721701] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:19:49.851371] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-05. [2025-10-20T17:19:50.519934] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:19:50.651369] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-06. [2025-10-20T17:19:51.385305] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:19:51.451358] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-06. [2025-10-20T17:19:51.867334] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:19:52.251410] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-07. [2025-10-20T17:19:52.402933] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:19:53.051364] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-06. [2025-10-20T17:19:53.575245] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T17:19:53.851367] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-06. [2025-10-20T17:19:54.651474] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-07. [2025-10-20T17:19:54.899017] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T17:19:55.255090] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:19:55.451384] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-07. [2025-10-20T17:19:55.978849] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:19:56.251413] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-08. [2025-10-20T17:19:56.893831] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:19:57.051351] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-08. [2025-10-20T17:19:57.584701] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T17:19:57.851398] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-08. [2025-10-20T17:19:58.429380] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T17:19:58.651428] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-07. [2025-10-20T17:19:59.337392] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T17:19:59.451385] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-09. [2025-10-20T17:20:00.251373] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-09. [2025-10-20T17:20:00.538469] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:20:00.612154] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:20:00.967542] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:20:01.051369] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-09. [2025-10-20T17:20:01.851379] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-08. [2025-10-20T17:20:02.545371] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T17:20:02.651406] [INFO ] [REMOTE ] ATEN: Fetching primary data for 2025-10. [2025-10-20T17:20:03.451444] [INFO ] [REMOTE ] ATHM: Fetching primary data for 2025-10. [2025-10-20T17:20:03.622550] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T17:20:04.110966] [INFO ] [REMOTE ] ATEN: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:20:04.117142] [INFO ] [REMOTE ] ATEN: Full fetch complete (56825 records). [2025-10-20T17:20:04.117277] [DEBUG] [VALIDATOR ] ATEN: Validating primary data. [2025-10-20T17:20:04.256618] [INFO ] [REMOTE ] ATGE: Fetching primary data for 2025-10. [2025-10-20T17:20:04.517594] [DEBUG] [VALIDATOR ] ATEN: All expected trading days are present. [2025-10-20T17:20:04.518743] [INFO ] [REMOTE ] ATHM: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T17:20:04.518172] [DEBUG] [VALIDATOR ] ATEN: Primary data validation passed. [2025-10-20T17:20:04.523469] [INFO ] [MAIN ] ATEN: Fetched primary data is valid. [2025-10-20T17:20:04.523869] [DEBUG] [LOCAL ] ATEN: Saving primary data. [2025-10-20T17:20:04.524390] [INFO ] [REMOTE ] ATHM: Full fetch complete (49799 records). [2025-10-20T17:20:04.524710] [DEBUG] [VALIDATOR ] ATHM: Validating primary data. [2025-10-20T17:20:04.584784] [DEBUG] [LOCAL ] ATEN: Trimmed 56825 records to 38022 (keeping 200 days of data) [2025-10-20T17:20:05.077847] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-09. [2025-10-20T17:20:05.185081] [DEBUG] [VALIDATOR ] ATHM: All expected trading days are present. [2025-10-20T17:20:05.193922] [DEBUG] [VALIDATOR ] ATHM: Primary data validation passed. [2025-10-20T17:20:05.226982] [INFO ] [MAIN ] ATHM: Fetched primary data is valid. [2025-10-20T17:20:05.238354] [DEBUG] [LOCAL ] ATHM: Saving primary data. [2025-10-20T17:20:05.253330] [DEBUG] [LOCAL ] ATHM: Trimmed 49799 records to 31516 (keeping 200 days of data) [2025-10-20T17:20:05.256830] [DEBUG] [LOCAL ] ATEN: Primary data saved (38022 records). [2025-10-20T17:20:05.257153] [INFO ] [MAIN ] ATEN: Full history saved. [2025-10-20T17:20:05.257400] [INFO ] [MAIN ] Update Progress: 165/2367 symbols completed. ATEN took 31.6s. Estimated remaining: 4h 27m 6s. Rate: 8.2 symbols/min [2025-10-20T17:20:05.599199] [INFO ] [MAIN ] ATHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:05.629141] [INFO ] [REMOTE ] ATGE: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:20:05.635639] [INFO ] [REMOTE ] ATGE: Full fetch complete (45740 records). [2025-10-20T17:20:05.636580] [DEBUG] [VALIDATOR ] ATGE: Validating primary data. [2025-10-20T17:20:05.638133] [FATAL] [VALIDATOR ] ATGE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:20:05.641187] [FATAL] [MAIN ] ATGE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:20:05.642396] [WARN ] [MAIN ] ATGE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:20:05.648965] [INFO ] [LOCAL ] ATGE: Starting deletion of primary data. [2025-10-20T17:20:05.655686] [DEBUG] [LOCAL ] ATHS: Loaded 6713 existing records from primary storage. [2025-10-20T17:20:05.657615] [DEBUG] [LOCAL ] ATGE: No primary data file found to delete. [2025-10-20T17:20:05.662143] [INFO ] [MAIN ] ATHS: Loaded 6713 existing records. [2025-10-20T17:20:05.657897] [INFO ] [LOCAL ] ATGE: Starting deletion of derivative data. [2025-10-20T17:20:05.674036] [DEBUG] [LOCAL ] ATHM: Primary data saved (31516 records). [2025-10-20T17:20:05.690604] [INFO ] [MAIN ] ATHM: Full history saved. [2025-10-20T17:20:05.713076] [INFO ] [MAIN ] Update Progress: 166/2367 symbols completed. ATHM took 27.3s. Estimated remaining: 4h 25m 28s. Rate: 8.3 symbols/min [2025-10-20T17:20:05.706275] [DEBUG] [LOCAL ] ATGE: Deleted derivative data file. [2025-10-20T17:20:05.758956] [INFO ] [MAIN ] Update Progress: 167/2367 symbols completed. ATGE took 29.0s. Estimated remaining: 4h 23m 46s. Rate: 8.3 symbols/min [2025-10-20T17:20:06.089866] [WARN ] [MAIN ] ATHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:06.109299] [INFO ] [MAIN ] ATHS: Wiping existing primary data before full refresh. [2025-10-20T17:20:06.119778] [INFO ] [LOCAL ] ATHS: Starting deletion of primary data. [2025-10-20T17:20:06.125484] [DEBUG] [LOCAL ] ATHS: Deleted primary data file. [2025-10-20T17:20:06.136009] [INFO ] [MAIN ] ATHS: Starting full primary refresh. [2025-10-20T17:20:06.227417] [INFO ] [MAIN ] ATI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:06.347923] [INFO ] [MAIN ] ATKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:06.636451] [DEBUG] [LOCAL ] ATI: Loaded 52088 existing records from primary storage. [2025-10-20T17:20:06.652094] [INFO ] [MAIN ] ATI: Loaded 52088 existing records. [2025-10-20T17:20:06.706829] [DEBUG] [LOCAL ] ATKR: Loaded 35684 existing records from primary storage. [2025-10-20T17:20:06.782743] [INFO ] [MAIN ] ATKR: Loaded 35684 existing records. [2025-10-20T17:20:06.937570] [INFO ] [REMOTE ] ATHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:07.018020] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-01. [2025-10-20T17:20:07.100147] [WARN ] [MAIN ] ATI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:07.124319] [INFO ] [MAIN ] ATI: Wiping existing primary data before full refresh. [2025-10-20T17:20:07.147341] [WARN ] [MAIN ] ATKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:07.147477] [INFO ] [LOCAL ] ATI: Starting deletion of primary data. [2025-10-20T17:20:07.148164] [INFO ] [MAIN ] ATKR: Wiping existing primary data before full refresh. [2025-10-20T17:20:07.148333] [INFO ] [LOCAL ] ATKR: Starting deletion of primary data. [2025-10-20T17:20:07.148457] [DEBUG] [LOCAL ] ATI: Deleted primary data file. [2025-10-20T17:20:07.148610] [INFO ] [MAIN ] ATI: Starting full primary refresh. [2025-10-20T17:20:07.161149] [DEBUG] [LOCAL ] ATKR: Deleted primary data file. [2025-10-20T17:20:07.161372] [INFO ] [MAIN ] ATKR: Starting full primary refresh. [2025-10-20T17:20:07.566354] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T17:20:07.717267] [INFO ] [REMOTE ] ASX: Fetching primary data for 2025-10. [2025-10-20T17:20:07.868505] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:20:07.905336] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-02. [2025-10-20T17:20:07.977284] [INFO ] [REMOTE ] ATKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:08.020721] [INFO ] [REMOTE ] ATI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:08.251435] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-01. [2025-10-20T17:20:08.491908] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T17:20:09.051416] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-03. [2025-10-20T17:20:09.580830] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T17:20:09.844006] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:20:09.851336] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-04. [2025-10-20T17:20:10.184903] [INFO ] [REMOTE ] ASX: Incremental fetch complete for 2025-10 (2.47 seconds). [2025-10-20T17:20:10.192841] [INFO ] [REMOTE ] ASX: Full fetch complete (76790 records). [2025-10-20T17:20:10.192990] [DEBUG] [VALIDATOR ] ASX: Validating primary data. [2025-10-20T17:20:10.499135] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:20:10.597121] [DEBUG] [VALIDATOR ] ASX: All expected trading days are present. [2025-10-20T17:20:10.597204] [DEBUG] [VALIDATOR ] ASX: Primary data validation passed. [2025-10-20T17:20:10.597964] [INFO ] [MAIN ] ASX: Fetched primary data is valid. [2025-10-20T17:20:10.598029] [DEBUG] [LOCAL ] ASX: Saving primary data. [2025-10-20T17:20:10.599872] [DEBUG] [LOCAL ] ASX: Trimmed 76790 records to 52882 (keeping 200 days of data) [2025-10-20T17:20:10.658739] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-02. [2025-10-20T17:20:10.907892] [DEBUG] [LOCAL ] ASX: Primary data saved (52882 records). [2025-10-20T17:20:10.908029] [INFO ] [MAIN ] ASX: Full history saved. [2025-10-20T17:20:10.908108] [INFO ] [MAIN ] Update Progress: 168/2367 symbols completed. ASX took 46.5s. Estimated remaining: 4h 23m 12s. Rate: 8.4 symbols/min [2025-10-20T17:20:11.074502] [INFO ] [MAIN ] ATMU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:11.108585] [DEBUG] [LOCAL ] ATMU: Loaded 39475 existing records from primary storage. [2025-10-20T17:20:11.108904] [INFO ] [MAIN ] ATMU: Loaded 39475 existing records. [2025-10-20T17:20:11.287073] [WARN ] [MAIN ] ATMU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:11.287299] [INFO ] [MAIN ] ATMU: Wiping existing primary data before full refresh. [2025-10-20T17:20:11.287345] [INFO ] [LOCAL ] ATMU: Starting deletion of primary data. [2025-10-20T17:20:11.287590] [DEBUG] [LOCAL ] ATMU: Deleted primary data file. [2025-10-20T17:20:11.287680] [INFO ] [MAIN ] ATMU: Starting full primary refresh. [2025-10-20T17:20:11.460322] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-01. [2025-10-20T17:20:11.641600] [INFO ] [REMOTE ] ATMU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:12.251391] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-01. [2025-10-20T17:20:12.316841] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:20:13.051387] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-05. [2025-10-20T17:20:13.650732] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:20:13.851462] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-06. [2025-10-20T17:20:14.485600] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T17:20:14.574099] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-01 (3.11 seconds). [2025-10-20T17:20:14.651482] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-07. [2025-10-20T17:20:14.939857] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-01 (2.69 seconds). [2025-10-20T17:20:15.356202] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:20:15.451396] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-03. [2025-10-20T17:20:16.251397] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-08. [2025-10-20T17:20:16.722219] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T17:20:16.811952] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:20:17.051477] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-02. [2025-10-20T17:20:17.851416] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-02. [2025-10-20T17:20:18.467611] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:20:18.651371] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-03. [2025-10-20T17:20:19.449371] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:20:19.457668] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-09. [2025-10-20T17:20:19.929012] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T17:20:20.170853] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:20:20.251404] [INFO ] [REMOTE ] ATHS: Fetching primary data for 2025-10. [2025-10-20T17:20:20.929466] [INFO ] [REMOTE ] ATHS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:20:20.930879] [INFO ] [REMOTE ] ATHS: Full fetch complete (9606 records). [2025-10-20T17:20:20.930970] [DEBUG] [VALIDATOR ] ATHS: Validating primary data. [2025-10-20T17:20:21.056512] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-03. [2025-10-20T17:20:21.327832] [DEBUG] [VALIDATOR ] ATHS: All expected trading days are present. [2025-10-20T17:20:21.327913] [DEBUG] [VALIDATOR ] ATHS: Primary data validation passed. [2025-10-20T17:20:21.328147] [INFO ] [MAIN ] ATHS: Fetched primary data is valid. [2025-10-20T17:20:21.328210] [DEBUG] [LOCAL ] ATHS: Saving primary data. [2025-10-20T17:20:21.328701] [DEBUG] [LOCAL ] ATHS: Trimmed 9606 records to 6572 (keeping 200 days of data) [2025-10-20T17:20:21.367354] [DEBUG] [LOCAL ] ATHS: Primary data saved (6572 records). [2025-10-20T17:20:21.367480] [INFO ] [MAIN ] ATHS: Full history saved. [2025-10-20T17:20:21.367559] [INFO ] [MAIN ] Update Progress: 169/2367 symbols completed. ATHS took 16.1s. Estimated remaining: 4h 23m 48s. Rate: 8.3 symbols/min [2025-10-20T17:20:21.536406] [INFO ] [MAIN ] ATO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:21.577300] [DEBUG] [LOCAL ] ATO: Loaded 42359 existing records from primary storage. [2025-10-20T17:20:21.577596] [INFO ] [MAIN ] ATO: Loaded 42359 existing records. [2025-10-20T17:20:21.765013] [WARN ] [MAIN ] ATO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:21.765230] [INFO ] [MAIN ] ATO: Wiping existing primary data before full refresh. [2025-10-20T17:20:21.765277] [INFO ] [LOCAL ] ATO: Starting deletion of primary data. [2025-10-20T17:20:21.765620] [DEBUG] [LOCAL ] ATO: Deleted primary data file. [2025-10-20T17:20:21.765715] [INFO ] [MAIN ] ATO: Starting full primary refresh. [2025-10-20T17:20:21.856457] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-04. [2025-10-20T17:20:22.123621] [INFO ] [REMOTE ] ATO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:22.656830] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-04. [2025-10-20T17:20:22.659470] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:20:23.415784] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T17:20:23.451367] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-01. [2025-10-20T17:20:24.038799] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:20:24.251419] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-04. [2025-10-20T17:20:25.051379] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-05. [2025-10-20T17:20:25.122672] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:20:25.755700] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:20:25.851382] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-02. [2025-10-20T17:20:26.651414] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-05. [2025-10-20T17:20:26.816852] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:20:27.278918] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T17:20:27.451388] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-05. [2025-10-20T17:20:27.765436] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T17:20:28.251389] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-06. [2025-10-20T17:20:28.797221] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T17:20:29.051412] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-06. [2025-10-20T17:20:29.860913] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-03. [2025-10-20T17:20:29.867597] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:20:30.321921] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T17:20:30.651385] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-06. [2025-10-20T17:20:31.451370] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-07. [2025-10-20T17:20:31.529396] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:20:31.610100] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:20:32.251422] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-07. [2025-10-20T17:20:32.886150] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T17:20:33.051375] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-08. [2025-10-20T17:20:33.734051] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:20:33.851396] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-07. [2025-10-20T17:20:34.627549] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:20:34.651380] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-08. [2025-10-20T17:20:35.331849] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:20:35.451374] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-08. [2025-10-20T17:20:36.074624] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:20:36.251369] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-09. [2025-10-20T17:20:37.056501] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-09. [2025-10-20T17:20:37.070870] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:20:37.851386] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-09. [2025-10-20T17:20:37.988587] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:20:38.292077] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T17:20:38.651369] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-04. [2025-10-20T17:20:39.285061] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:20:39.451374] [INFO ] [REMOTE ] ATMU: Fetching primary data for 2025-10. [2025-10-20T17:20:40.098893] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T17:20:40.251409] [INFO ] [REMOTE ] ATI: Fetching primary data for 2025-10. [2025-10-20T17:20:41.027299] [INFO ] [REMOTE ] ATMU: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:20:41.033287] [INFO ] [REMOTE ] ATMU: Full fetch complete (56862 records). [2025-10-20T17:20:41.033442] [DEBUG] [VALIDATOR ] ATMU: Validating primary data. [2025-10-20T17:20:41.056544] [INFO ] [REMOTE ] ATKR: Fetching primary data for 2025-10. [2025-10-20T17:20:41.450390] [DEBUG] [VALIDATOR ] ATMU: All expected trading days are present. [2025-10-20T17:20:41.450486] [DEBUG] [VALIDATOR ] ATMU: Primary data validation passed. [2025-10-20T17:20:41.451197] [INFO ] [MAIN ] ATMU: Fetched primary data is valid. [2025-10-20T17:20:41.451265] [DEBUG] [LOCAL ] ATMU: Saving primary data. [2025-10-20T17:20:41.452571] [DEBUG] [LOCAL ] ATMU: Trimmed 56862 records to 38808 (keeping 200 days of data) [2025-10-20T17:20:41.693339] [DEBUG] [LOCAL ] ATMU: Primary data saved (38808 records). [2025-10-20T17:20:41.693511] [INFO ] [MAIN ] ATMU: Full history saved. [2025-10-20T17:20:41.693603] [INFO ] [MAIN ] Update Progress: 170/2367 symbols completed. ATMU took 30.8s. Estimated remaining: 4h 26m 30s. Rate: 8.2 symbols/min [2025-10-20T17:20:41.852574] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-05. [2025-10-20T17:20:41.912814] [INFO ] [MAIN ] ATR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:41.942782] [DEBUG] [LOCAL ] ATR: Loaded 30974 existing records from primary storage. [2025-10-20T17:20:41.943056] [INFO ] [MAIN ] ATR: Loaded 30974 existing records. [2025-10-20T17:20:42.123634] [INFO ] [REMOTE ] ATI: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:20:42.180506] [WARN ] [MAIN ] ATR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:42.180754] [INFO ] [REMOTE ] ATI: Full fetch complete (74018 records). [2025-10-20T17:20:42.181115] [INFO ] [MAIN ] ATR: Wiping existing primary data before full refresh. [2025-10-20T17:20:42.181200] [INFO ] [LOCAL ] ATR: Starting deletion of primary data. [2025-10-20T17:20:42.181381] [DEBUG] [VALIDATOR ] ATI: Validating primary data. [2025-10-20T17:20:42.181681] [DEBUG] [LOCAL ] ATR: Deleted primary data file. [2025-10-20T17:20:42.181757] [INFO ] [MAIN ] ATR: Starting full primary refresh. [2025-10-20T17:20:43.125195] [INFO ] [REMOTE ] ATR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:43.120444] [INFO ] [REMOTE ] ATKR: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T17:20:43.157292] [DEBUG] [VALIDATOR ] ATI: All expected trading days are present. [2025-10-20T17:20:43.157011] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-01. [2025-10-20T17:20:43.157447] [DEBUG] [VALIDATOR ] ATI: Primary data validation passed. [2025-10-20T17:20:43.159603] [INFO ] [MAIN ] ATI: Fetched primary data is valid. [2025-10-20T17:20:43.159989] [DEBUG] [LOCAL ] ATI: Saving primary data. [2025-10-20T17:20:43.160557] [INFO ] [REMOTE ] ATKR: Full fetch complete (53302 records). [2025-10-20T17:20:43.161317] [DEBUG] [VALIDATOR ] ATKR: Validating primary data. [2025-10-20T17:20:43.184527] [DEBUG] [LOCAL ] ATI: Trimmed 74018 records to 51311 (keeping 200 days of data) [2025-10-20T17:20:43.875939] [DEBUG] [VALIDATOR ] ATKR: All expected trading days are present. [2025-10-20T17:20:43.889040] [DEBUG] [VALIDATOR ] ATKR: Primary data validation passed. [2025-10-20T17:20:43.894906] [INFO ] [MAIN ] ATKR: Fetched primary data is valid. [2025-10-20T17:20:43.895035] [DEBUG] [LOCAL ] ATKR: Saving primary data. [2025-10-20T17:20:43.896668] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T17:20:43.960960] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-06. [2025-10-20T17:20:43.965835] [DEBUG] [LOCAL ] ATKR: Trimmed 53302 records to 35028 (keeping 200 days of data) [2025-10-20T17:20:44.039259] [DEBUG] [LOCAL ] ATI: Primary data saved (51311 records). [2025-10-20T17:20:44.049951] [INFO ] [MAIN ] ATI: Full history saved. [2025-10-20T17:20:44.077074] [INFO ] [MAIN ] Update Progress: 171/2367 symbols completed. ATI took 38.3s. Estimated remaining: 4h 25m 20s. Rate: 8.3 symbols/min [2025-10-20T17:20:44.352749] [DEBUG] [LOCAL ] ATKR: Primary data saved (35028 records). [2025-10-20T17:20:44.385593] [INFO ] [MAIN ] ATKR: Full history saved. [2025-10-20T17:20:44.412998] [INFO ] [MAIN ] ATS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:44.413263] [DEBUG] [LOCAL ] ATS: No existing data file found in primary storage. [2025-10-20T17:20:44.409471] [INFO ] [MAIN ] Update Progress: 172/2367 symbols completed. ATKR took 38.6s. Estimated remaining: 4h 23m 44s. Rate: 8.3 symbols/min [2025-10-20T17:20:44.418811] [WARN ] [MAIN ] ATS: No existing data found, performing full refresh instead. [2025-10-20T17:20:44.448900] [INFO ] [MAIN ] ATS: Wiping existing primary data before full refresh. [2025-10-20T17:20:44.477075] [INFO ] [LOCAL ] ATS: Starting deletion of primary data. [2025-10-20T17:20:44.528799] [DEBUG] [LOCAL ] ATS: No primary data file found to delete. [2025-10-20T17:20:44.529561] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T17:20:44.547181] [INFO ] [MAIN ] ATS: Starting full primary refresh. [2025-10-20T17:20:44.553554] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-02. [2025-10-20T17:20:44.778290] [INFO ] [MAIN ] ATUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:44.788925] [DEBUG] [LOCAL ] ATUS: No existing data file found in primary storage. [2025-10-20T17:20:44.794182] [WARN ] [MAIN ] ATUS: No existing data found, performing full refresh instead. [2025-10-20T17:20:44.830616] [INFO ] [MAIN ] ATUS: Wiping existing primary data before full refresh. [2025-10-20T17:20:44.850672] [INFO ] [LOCAL ] ATUS: Starting deletion of primary data. [2025-10-20T17:20:44.930768] [DEBUG] [LOCAL ] ATUS: No primary data file found to delete. [2025-10-20T17:20:44.944874] [INFO ] [MAIN ] ATUS: Starting full primary refresh. [2025-10-20T17:20:45.182727] [INFO ] [REMOTE ] ATS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:45.230372] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-01. [2025-10-20T17:20:45.455365] [INFO ] [REMOTE ] ATUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:45.481567] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:20:45.838558] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:20:45.851344] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-03. [2025-10-20T17:20:46.138887] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:20:46.651354] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-02. [2025-10-20T17:20:47.220054] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:20:47.236828] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T17:20:47.451385] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-04. [2025-10-20T17:20:48.251452] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-01. [2025-10-20T17:20:48.710347] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:20:49.051381] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-07. [2025-10-20T17:20:49.637554] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:20:49.851376] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-02. [2025-10-20T17:20:50.625117] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T17:20:50.651329] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-08. [2025-10-20T17:20:51.116043] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T17:20:51.451382] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-05. [2025-10-20T17:20:51.909140] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:20:52.251377] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-09. [2025-10-20T17:20:52.775528] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:20:53.051345] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-06. [2025-10-20T17:20:53.850037] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:20:53.859177] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-03. [2025-10-20T17:20:54.332058] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:20:54.546931] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T17:20:54.651384] [INFO ] [REMOTE ] ATO: Fetching primary data for 2025-10. [2025-10-20T17:20:55.451415] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-03. [2025-10-20T17:20:56.251368] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-04. [2025-10-20T17:20:56.472379] [INFO ] [REMOTE ] ATO: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:20:56.478670] [INFO ] [REMOTE ] ATO: Full fetch complete (60745 records). [2025-10-20T17:20:56.478821] [DEBUG] [VALIDATOR ] ATO: Validating primary data. [2025-10-20T17:20:56.872551] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:20:56.917862] [DEBUG] [VALIDATOR ] ATO: All expected trading days are present. [2025-10-20T17:20:56.917950] [DEBUG] [VALIDATOR ] ATO: Primary data validation passed. [2025-10-20T17:20:56.918700] [INFO ] [MAIN ] ATO: Fetched primary data is valid. [2025-10-20T17:20:56.918769] [DEBUG] [LOCAL ] ATO: Saving primary data. [2025-10-20T17:20:56.920064] [DEBUG] [LOCAL ] ATO: Trimmed 60745 records to 41744 (keeping 200 days of data) [2025-10-20T17:20:57.051387] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-04. [2025-10-20T17:20:57.191884] [DEBUG] [LOCAL ] ATO: Primary data saved (41744 records). [2025-10-20T17:20:57.192041] [INFO ] [MAIN ] ATO: Full history saved. [2025-10-20T17:20:57.192811] [INFO ] [MAIN ] Update Progress: 173/2367 symbols completed. ATO took 35.8s. Estimated remaining: 4h 24m 48s. Rate: 8.3 symbols/min [2025-10-20T17:20:57.369525] [INFO ] [MAIN ] AU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:20:57.418899] [DEBUG] [LOCAL ] AU: Loaded 53857 existing records from primary storage. [2025-10-20T17:20:57.419213] [INFO ] [MAIN ] AU: Loaded 53857 existing records. [2025-10-20T17:20:57.584504] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:20:57.620126] [WARN ] [MAIN ] AU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:20:57.620308] [INFO ] [MAIN ] AU: Wiping existing primary data before full refresh. [2025-10-20T17:20:57.620354] [INFO ] [LOCAL ] AU: Starting deletion of primary data. [2025-10-20T17:20:57.620754] [DEBUG] [LOCAL ] AU: Deleted primary data file. [2025-10-20T17:20:57.620810] [INFO ] [MAIN ] AU: Starting full primary refresh. [2025-10-20T17:20:57.859158] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-05. [2025-10-20T17:20:57.979037] [INFO ] [REMOTE ] AU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:20:58.651366] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-07. [2025-10-20T17:20:58.777491] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T17:20:58.886461] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:20:59.451374] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-06. [2025-10-20T17:21:00.029135] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:21:00.161180] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:21:00.251373] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-05. [2025-10-20T17:21:01.051410] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-07. [2025-10-20T17:21:01.761621] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:21:01.851360] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-08. [2025-10-20T17:21:01.958109] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:21:02.651394] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-01. [2025-10-20T17:21:02.824445] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T17:21:03.451395] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-08. [2025-10-20T17:21:04.251395] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-06. [2025-10-20T17:21:04.399713] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T17:21:04.427773] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T17:21:05.051389] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-09. [2025-10-20T17:21:05.851356] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-09. [2025-10-20T17:21:05.966571] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:21:05.999745] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T17:21:06.651436] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-02. [2025-10-20T17:21:07.308005] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:21:07.451440] [INFO ] [REMOTE ] ATS: Fetching primary data for 2025-10. [2025-10-20T17:21:08.251405] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-07. [2025-10-20T17:21:08.529954] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T17:21:09.051451] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-03. [2025-10-20T17:21:09.851354] [INFO ] [REMOTE ] ATR: Fetching primary data for 2025-10. [2025-10-20T17:21:10.096536] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:21:10.651400] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-08. [2025-10-20T17:21:10.739356] [INFO ] [REMOTE ] ATS: Incremental fetch complete for 2025-10 (3.29 seconds). [2025-10-20T17:21:10.741825] [INFO ] [REMOTE ] ATS: Full fetch complete (25476 records). [2025-10-20T17:21:10.741913] [DEBUG] [VALIDATOR ] ATS: Validating primary data. [2025-10-20T17:21:11.151490] [DEBUG] [VALIDATOR ] ATS: All expected trading days are present. [2025-10-20T17:21:11.151738] [DEBUG] [VALIDATOR ] ATS: Primary data validation passed. [2025-10-20T17:21:11.152242] [INFO ] [MAIN ] ATS: Fetched primary data is valid. [2025-10-20T17:21:11.152324] [DEBUG] [LOCAL ] ATS: Saving primary data. [2025-10-20T17:21:11.153311] [DEBUG] [LOCAL ] ATS: Trimmed 25476 records to 17325 (keeping 200 days of data) [2025-10-20T17:21:11.273592] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-03 (2.22 seconds). [2025-10-20T17:21:11.294679] [DEBUG] [LOCAL ] ATS: Primary data saved (17325 records). [2025-10-20T17:21:11.295019] [INFO ] [MAIN ] ATS: Full history saved. [2025-10-20T17:21:11.295127] [INFO ] [MAIN ] Update Progress: 174/2367 symbols completed. ATS took 27.2s. Estimated remaining: 4h 26m 7s. Rate: 8.2 symbols/min [2025-10-20T17:21:11.342547] [INFO ] [REMOTE ] ATR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:21:11.396798] [INFO ] [REMOTE ] ATR: Full fetch complete (43989 records). [2025-10-20T17:21:11.417976] [DEBUG] [VALIDATOR ] ATR: Validating primary data. [2025-10-20T17:21:11.456545] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-04. [2025-10-20T17:21:11.552974] [INFO ] [MAIN ] AUB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:11.596279] [DEBUG] [LOCAL ] AUB: No existing data file found in primary storage. [2025-10-20T17:21:11.599787] [WARN ] [MAIN ] AUB: No existing data found, performing full refresh instead. [2025-10-20T17:21:11.600058] [INFO ] [MAIN ] AUB: Wiping existing primary data before full refresh. [2025-10-20T17:21:11.605732] [INFO ] [LOCAL ] AUB: Starting deletion of primary data. [2025-10-20T17:21:11.610976] [DEBUG] [LOCAL ] AUB: No primary data file found to delete. [2025-10-20T17:21:11.616170] [INFO ] [MAIN ] AUB: Starting full primary refresh. [2025-10-20T17:21:12.179389] [DEBUG] [VALIDATOR ] ATR: All expected trading days are present. [2025-10-20T17:21:12.179530] [DEBUG] [VALIDATOR ] ATR: Primary data validation passed. [2025-10-20T17:21:12.180319] [INFO ] [MAIN ] ATR: Fetched primary data is valid. [2025-10-20T17:21:12.196041] [DEBUG] [LOCAL ] ATR: Saving primary data. [2025-10-20T17:21:12.260587] [INFO ] [REMOTE ] AUB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:12.261421] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-01. [2025-10-20T17:21:12.261609] [DEBUG] [LOCAL ] ATR: Trimmed 43989 records to 30554 (keeping 200 days of data) [2025-10-20T17:21:12.394057] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:21:12.508009] [DEBUG] [LOCAL ] ATR: Primary data saved (30554 records). [2025-10-20T17:21:12.508159] [INFO ] [MAIN ] ATR: Full history saved. [2025-10-20T17:21:12.508267] [INFO ] [MAIN ] Update Progress: 175/2367 symbols completed. ATR took 30.8s. Estimated remaining: 4h 24m 44s. Rate: 8.3 symbols/min [2025-10-20T17:21:12.694492] [INFO ] [MAIN ] AUNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:12.708609] [DEBUG] [LOCAL ] AUNA: Loaded 5186 existing records from primary storage. [2025-10-20T17:21:12.709078] [INFO ] [MAIN ] AUNA: Loaded 5186 existing records. [2025-10-20T17:21:12.912693] [WARN ] [MAIN ] AUNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:12.912929] [INFO ] [MAIN ] AUNA: Wiping existing primary data before full refresh. [2025-10-20T17:21:12.912982] [INFO ] [LOCAL ] AUNA: Starting deletion of primary data. [2025-10-20T17:21:12.913123] [DEBUG] [LOCAL ] AUNA: Deleted primary data file. [2025-10-20T17:21:12.913162] [INFO ] [MAIN ] AUNA: Starting full primary refresh. [2025-10-20T17:21:13.062872] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-09. [2025-10-20T17:21:13.303383] [INFO ] [REMOTE ] AUNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:13.812469] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-04 (2.36 seconds). [2025-10-20T17:21:13.851489] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-05. [2025-10-20T17:21:13.861217] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:21:14.651453] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-02. [2025-10-20T17:21:14.747124] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:21:15.451341] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-01. [2025-10-20T17:21:15.968065] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T17:21:16.033349] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:21:16.251378] [INFO ] [REMOTE ] ATUS: Fetching primary data for 2025-10. [2025-10-20T17:21:16.258623] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:21:17.051360] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-02. [2025-10-20T17:21:17.500503] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:21:17.851372] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-03. [2025-10-20T17:21:18.090332] [INFO ] [REMOTE ] ATUS: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:21:18.097908] [INFO ] [REMOTE ] ATUS: Full fetch complete (71392 records). [2025-10-20T17:21:18.098078] [DEBUG] [VALIDATOR ] ATUS: Validating primary data. [2025-10-20T17:21:18.521337] [DEBUG] [VALIDATOR ] ATUS: All expected trading days are present. [2025-10-20T17:21:18.521424] [DEBUG] [VALIDATOR ] ATUS: Primary data validation passed. [2025-10-20T17:21:18.522190] [INFO ] [MAIN ] ATUS: Fetched primary data is valid. [2025-10-20T17:21:18.522253] [DEBUG] [LOCAL ] ATUS: Saving primary data. [2025-10-20T17:21:18.523616] [DEBUG] [LOCAL ] ATUS: Trimmed 71392 records to 49947 (keeping 200 days of data) [2025-10-20T17:21:18.611345] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T17:21:18.695518] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-04. [2025-10-20T17:21:18.823674] [DEBUG] [LOCAL ] ATUS: Primary data saved (49947 records). [2025-10-20T17:21:18.823816] [INFO ] [MAIN ] ATUS: Full history saved. [2025-10-20T17:21:18.823887] [INFO ] [MAIN ] Update Progress: 176/2367 symbols completed. ATUS took 34.4s. Estimated remaining: 4h 24m 25s. Rate: 8.3 symbols/min [2025-10-20T17:21:19.007611] [INFO ] [MAIN ] AVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:19.057849] [DEBUG] [LOCAL ] AVA: Loaded 39279 existing records from primary storage. [2025-10-20T17:21:19.058254] [INFO ] [MAIN ] AVA: Loaded 39279 existing records. [2025-10-20T17:21:19.285161] [WARN ] [MAIN ] AVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:19.288846] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:21:19.288969] [INFO ] [MAIN ] AVA: Wiping existing primary data before full refresh. [2025-10-20T17:21:19.289103] [INFO ] [LOCAL ] AVA: Starting deletion of primary data. [2025-10-20T17:21:19.290794] [DEBUG] [LOCAL ] AVA: Deleted primary data file. [2025-10-20T17:21:19.291044] [INFO ] [MAIN ] AVA: Starting full primary refresh. [2025-10-20T17:21:19.452784] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-06. [2025-10-20T17:21:19.712785] [INFO ] [REMOTE ] AVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:20.251384] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-01. [2025-10-20T17:21:21.051422] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-03. [2025-10-20T17:21:21.456957] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:21:21.851432] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-05. [2025-10-20T17:21:22.063881] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:21:22.469243] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T17:21:22.651371] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-02. [2025-10-20T17:21:22.705758] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T17:21:23.451379] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-06. [2025-10-20T17:21:23.976612] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:21:24.245173] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:21:24.251338] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-07. [2025-10-20T17:21:24.606473] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-10-20T17:21:25.051403] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-08. [2025-10-20T17:21:25.588170] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T17:21:25.851388] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-09. [2025-10-20T17:21:26.651414] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-03. [2025-10-20T17:21:26.742454] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:21:27.451469] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-04. [2025-10-20T17:21:28.088031] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:21:28.251440] [INFO ] [REMOTE ] AUNA: Fetching primary data for 2025-10. [2025-10-20T17:21:28.980221] [INFO ] [REMOTE ] AUNA: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T17:21:28.982169] [INFO ] [REMOTE ] AUNA: Full fetch complete (7779 records). [2025-10-20T17:21:28.982247] [DEBUG] [VALIDATOR ] AUNA: Validating primary data. [2025-10-20T17:21:29.058053] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-07. [2025-10-20T17:21:29.411771] [DEBUG] [VALIDATOR ] AUNA: All expected trading days are present. [2025-10-20T17:21:29.411888] [DEBUG] [VALIDATOR ] AUNA: Primary data validation passed. [2025-10-20T17:21:29.412396] [INFO ] [MAIN ] AUNA: Fetched primary data is valid. [2025-10-20T17:21:29.412503] [DEBUG] [LOCAL ] AUNA: Saving primary data. [2025-10-20T17:21:29.413382] [DEBUG] [LOCAL ] AUNA: Trimmed 7779 records to 5212 (keeping 200 days of data) [2025-10-20T17:21:29.474922] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:21:29.494028] [DEBUG] [LOCAL ] AUNA: Primary data saved (5212 records). [2025-10-20T17:21:29.494415] [INFO ] [MAIN ] AUNA: Full history saved. [2025-10-20T17:21:29.494571] [INFO ] [MAIN ] Update Progress: 177/2367 symbols completed. AUNA took 17.0s. Estimated remaining: 4h 25m 0s. Rate: 8.3 symbols/min [2025-10-20T17:21:29.682370] [INFO ] [MAIN ] AVAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:29.693272] [DEBUG] [LOCAL ] AVAL: Loaded 8705 existing records from primary storage. [2025-10-20T17:21:29.693570] [INFO ] [MAIN ] AVAL: Loaded 8705 existing records. [2025-10-20T17:21:29.856514] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-05. [2025-10-20T17:21:29.892565] [WARN ] [MAIN ] AVAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:29.892902] [INFO ] [MAIN ] AVAL: Wiping existing primary data before full refresh. [2025-10-20T17:21:29.892974] [INFO ] [LOCAL ] AVAL: Starting deletion of primary data. [2025-10-20T17:21:29.893331] [DEBUG] [LOCAL ] AVAL: Deleted primary data file. [2025-10-20T17:21:29.893371] [INFO ] [MAIN ] AVAL: Starting full primary refresh. [2025-10-20T17:21:30.295921] [INFO ] [REMOTE ] AVAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:30.651391] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-04. [2025-10-20T17:21:30.861705] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:21:31.451391] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-01. [2025-10-20T17:21:31.610886] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:21:32.155153] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:21:32.251407] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-08. [2025-10-20T17:21:32.297585] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:21:33.051391] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-02. [2025-10-20T17:21:33.851406] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-05. [2025-10-20T17:21:34.039591] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T17:21:34.222326] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T17:21:34.651409] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-06. [2025-10-20T17:21:35.090836] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T17:21:35.451416] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-03. [2025-10-20T17:21:35.965078] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:21:36.251385] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-07. [2025-10-20T17:21:36.442812] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:21:37.051404] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-06. [2025-10-20T17:21:37.816133] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:21:37.851433] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-04. [2025-10-20T17:21:38.574904] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:21:38.651457] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-08. [2025-10-20T17:21:38.711978] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:21:39.451460] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-09. [2025-10-20T17:21:40.251349] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-07. [2025-10-20T17:21:40.304710] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:21:41.051389] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-05. [2025-10-20T17:21:41.675209] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:21:41.753289] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:21:41.790969] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:21:41.851338] [INFO ] [REMOTE ] AU: Fetching primary data for 2025-10. [2025-10-20T17:21:42.651470] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-08. [2025-10-20T17:21:43.451709] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-09. [2025-10-20T17:21:44.057870] [INFO ] [REMOTE ] AU: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:21:44.065807] [INFO ] [REMOTE ] AU: Full fetch complete (76604 records). [2025-10-20T17:21:44.065971] [DEBUG] [VALIDATOR ] AU: Validating primary data. [2025-10-20T17:21:44.252115] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-06. [2025-10-20T17:21:44.381330] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T17:21:44.523682] [DEBUG] [VALIDATOR ] AU: All expected trading days are present. [2025-10-20T17:21:44.523925] [DEBUG] [VALIDATOR ] AU: Primary data validation passed. [2025-10-20T17:21:44.524769] [INFO ] [MAIN ] AU: Fetched primary data is valid. [2025-10-20T17:21:44.524847] [DEBUG] [LOCAL ] AU: Saving primary data. [2025-10-20T17:21:44.526531] [DEBUG] [LOCAL ] AU: Trimmed 76604 records to 53089 (keeping 200 days of data) [2025-10-20T17:21:44.773884] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:21:44.889018] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:21:44.904110] [DEBUG] [LOCAL ] AU: Primary data saved (53089 records). [2025-10-20T17:21:44.904263] [INFO ] [MAIN ] AU: Full history saved. [2025-10-20T17:21:44.904709] [INFO ] [MAIN ] Update Progress: 178/2367 symbols completed. AU took 47.7s. Estimated remaining: 4h 26m 33s. Rate: 8.2 symbols/min [2025-10-20T17:21:45.056513] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-09. [2025-10-20T17:21:45.082428] [INFO ] [MAIN ] AVB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:45.133366] [DEBUG] [LOCAL ] AVB: Loaded 42866 existing records from primary storage. [2025-10-20T17:21:45.133739] [INFO ] [MAIN ] AVB: Loaded 42866 existing records. [2025-10-20T17:21:45.359163] [WARN ] [MAIN ] AVB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:45.359408] [INFO ] [MAIN ] AVB: Wiping existing primary data before full refresh. [2025-10-20T17:21:45.359456] [INFO ] [LOCAL ] AVB: Starting deletion of primary data. [2025-10-20T17:21:45.359868] [DEBUG] [LOCAL ] AVB: Deleted primary data file. [2025-10-20T17:21:45.359922] [INFO ] [MAIN ] AVB: Starting full primary refresh. [2025-10-20T17:21:45.739575] [INFO ] [REMOTE ] AVB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:45.851392] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-07. [2025-10-20T17:21:46.213239] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:21:46.623371] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T17:21:46.651336] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-08. [2025-10-20T17:21:47.379250] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T17:21:47.451401] [INFO ] [REMOTE ] AVA: Fetching primary data for 2025-10. [2025-10-20T17:21:48.251377] [INFO ] [REMOTE ] AUB: Fetching primary data for 2025-10. [2025-10-20T17:21:48.793312] [INFO ] [REMOTE ] AVA: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T17:21:48.799021] [INFO ] [REMOTE ] AVA: Full fetch complete (55483 records). [2025-10-20T17:21:48.799163] [DEBUG] [VALIDATOR ] AVA: Validating primary data. [2025-10-20T17:21:49.056615] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-09. [2025-10-20T17:21:49.182872] [DEBUG] [VALIDATOR ] AVA: All expected trading days are present. [2025-10-20T17:21:49.182961] [DEBUG] [VALIDATOR ] AVA: Primary data validation passed. [2025-10-20T17:21:49.183911] [INFO ] [MAIN ] AVA: Fetched primary data is valid. [2025-10-20T17:21:49.183974] [DEBUG] [LOCAL ] AVA: Saving primary data. [2025-10-20T17:21:49.185144] [DEBUG] [LOCAL ] AVA: Trimmed 55483 records to 38503 (keeping 200 days of data) [2025-10-20T17:21:49.418302] [DEBUG] [LOCAL ] AVA: Primary data saved (38503 records). [2025-10-20T17:21:49.418473] [INFO ] [MAIN ] AVA: Full history saved. [2025-10-20T17:21:49.418561] [INFO ] [MAIN ] Update Progress: 179/2367 symbols completed. AVA took 30.6s. Estimated remaining: 4h 25m 52s. Rate: 8.2 symbols/min [2025-10-20T17:21:49.597314] [INFO ] [MAIN ] AVBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:49.598291] [DEBUG] [LOCAL ] AVBC: No existing data file found in primary storage. [2025-10-20T17:21:49.598418] [WARN ] [MAIN ] AVBC: No existing data found, performing full refresh instead. [2025-10-20T17:21:49.598989] [INFO ] [MAIN ] AVBC: Wiping existing primary data before full refresh. [2025-10-20T17:21:49.607126] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-10-20T17:21:49.607226] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-10-20T17:21:49.630332] [INFO ] [MAIN ] AVBC: Starting full primary refresh. [2025-10-20T17:21:49.639311] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:21:49.661238] [INFO ] [REMOTE ] AUB: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:21:49.771106] [INFO ] [REMOTE ] AUB: Full fetch complete (66264 records). [2025-10-20T17:21:49.771374] [DEBUG] [VALIDATOR ] AUB: Validating primary data. [2025-10-20T17:21:49.857212] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-01. [2025-10-20T17:21:50.342450] [INFO ] [REMOTE ] AVBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:50.500369] [DEBUG] [VALIDATOR ] AUB: All expected trading days are present. [2025-10-20T17:21:50.500466] [DEBUG] [VALIDATOR ] AUB: Primary data validation passed. [2025-10-20T17:21:50.501145] [INFO ] [MAIN ] AUB: Fetched primary data is valid. [2025-10-20T17:21:50.501210] [DEBUG] [LOCAL ] AUB: Saving primary data. [2025-10-20T17:21:50.503075] [DEBUG] [LOCAL ] AUB: Trimmed 66264 records to 45950 (keeping 200 days of data) [2025-10-20T17:21:50.685021] [INFO ] [REMOTE ] AVBC: Fetching primary data for 2025-01. [2025-10-20T17:21:50.790411] [DEBUG] [LOCAL ] AUB: Primary data saved (45950 records). [2025-10-20T17:21:50.790559] [INFO ] [MAIN ] AUB: Full history saved. [2025-10-20T17:21:50.790633] [INFO ] [MAIN ] Update Progress: 180/2367 symbols completed. AUB took 39.5s. Estimated remaining: 4h 24m 33s. Rate: 8.3 symbols/min [2025-10-20T17:21:50.897343] [DEBUG] [REMOTE ] AVBC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:21:50.898247] [WARN ] [REMOTE ] AVBC: No data returned for month 2025-01 [2025-10-20T17:21:50.914233] [FATAL] [REMOTE ] AVBC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:21:50.940049] [FATAL] [MAIN ] AVBC: Failed to fetch primary data. [2025-10-20T17:21:50.954234] [WARN ] [MAIN ] AVBC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:21:50.972865] [INFO ] [MAIN ] AVD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:50.973019] [INFO ] [LOCAL ] AVBC: Starting deletion of primary data. [2025-10-20T17:21:50.973749] [DEBUG] [LOCAL ] AVBC: No primary data file found to delete. [2025-10-20T17:21:50.973832] [INFO ] [LOCAL ] AVBC: Starting deletion of derivative data. [2025-10-20T17:21:50.973892] [DEBUG] [LOCAL ] AVBC: No derivative data file found to delete. [2025-10-20T17:21:50.973976] [INFO ] [MAIN ] Update Progress: 181/2367 symbols completed. AVBC took 1.6s. Estimated remaining: 4h 22m 60s. Rate: 8.3 symbols/min [2025-10-20T17:21:51.195479] [INFO ] [MAIN ] AVK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:51.196828] [DEBUG] [LOCAL ] AVD: Loaded 22021 existing records from primary storage. [2025-10-20T17:21:51.197879] [INFO ] [MAIN ] AVD: Loaded 22021 existing records. [2025-10-20T17:21:51.429601] [WARN ] [MAIN ] AVD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:51.430877] [DEBUG] [LOCAL ] AVK: Loaded 18858 existing records from primary storage. [2025-10-20T17:21:51.431554] [INFO ] [MAIN ] AVD: Wiping existing primary data before full refresh. [2025-10-20T17:21:51.432044] [INFO ] [LOCAL ] AVD: Starting deletion of primary data. [2025-10-20T17:21:51.431926] [INFO ] [MAIN ] AVK: Loaded 18858 existing records. [2025-10-20T17:21:51.437868] [DEBUG] [LOCAL ] AVD: Deleted primary data file. [2025-10-20T17:21:51.456994] [INFO ] [MAIN ] AVD: Starting full primary refresh. [2025-10-20T17:21:51.456598] [INFO ] [REMOTE ] AVAL: Fetching primary data for 2025-10. [2025-10-20T17:21:51.884867] [WARN ] [MAIN ] AVK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:51.926311] [INFO ] [MAIN ] AVK: Wiping existing primary data before full refresh. [2025-10-20T17:21:51.942959] [INFO ] [LOCAL ] AVK: Starting deletion of primary data. [2025-10-20T17:21:51.997125] [DEBUG] [LOCAL ] AVK: Deleted primary data file. [2025-10-20T17:21:52.005158] [INFO ] [MAIN ] AVK: Starting full primary refresh. [2025-10-20T17:21:52.007088] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-01 (2.15 seconds). [2025-10-20T17:21:52.259036] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-02. [2025-10-20T17:21:52.301523] [INFO ] [REMOTE ] AVD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:52.435792] [INFO ] [REMOTE ] AVAL: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T17:21:52.493799] [INFO ] [REMOTE ] AVAL: Full fetch complete (14958 records). [2025-10-20T17:21:52.520820] [DEBUG] [VALIDATOR ] AVAL: Validating primary data. [2025-10-20T17:21:52.758325] [INFO ] [REMOTE ] AVK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:53.065812] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-01. [2025-10-20T17:21:53.079627] [DEBUG] [VALIDATOR ] AVAL: All expected trading days are present. [2025-10-20T17:21:53.079764] [DEBUG] [VALIDATOR ] AVAL: Primary data validation passed. [2025-10-20T17:21:53.080168] [INFO ] [MAIN ] AVAL: Fetched primary data is valid. [2025-10-20T17:21:53.080389] [DEBUG] [LOCAL ] AVAL: Saving primary data. [2025-10-20T17:21:53.081049] [DEBUG] [LOCAL ] AVAL: Trimmed 14958 records to 8528 (keeping 200 days of data) [2025-10-20T17:21:53.135895] [DEBUG] [LOCAL ] AVAL: Primary data saved (8528 records). [2025-10-20T17:21:53.136091] [INFO ] [MAIN ] AVAL: Full history saved. [2025-10-20T17:21:53.136194] [INFO ] [MAIN ] Update Progress: 182/2367 symbols completed. AVAL took 23.6s. Estimated remaining: 4h 21m 52s. Rate: 8.3 symbols/min [2025-10-20T17:21:53.337358] [INFO ] [MAIN ] AVNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:21:53.375765] [DEBUG] [LOCAL ] AVNS: Loaded 34111 existing records from primary storage. [2025-10-20T17:21:53.376172] [INFO ] [MAIN ] AVNS: Loaded 34111 existing records. [2025-10-20T17:21:53.576917] [WARN ] [MAIN ] AVNS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:21:53.577299] [INFO ] [MAIN ] AVNS: Wiping existing primary data before full refresh. [2025-10-20T17:21:53.577357] [INFO ] [LOCAL ] AVNS: Starting deletion of primary data. [2025-10-20T17:21:53.577778] [DEBUG] [LOCAL ] AVNS: Deleted primary data file. [2025-10-20T17:21:53.577839] [INFO ] [MAIN ] AVNS: Starting full primary refresh. [2025-10-20T17:21:53.855416] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-01. [2025-10-20T17:21:53.989901] [INFO ] [REMOTE ] AVNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:21:54.014337] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T17:21:54.406372] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:21:54.651385] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-01. [2025-10-20T17:21:54.888058] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T17:21:55.451364] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-02. [2025-10-20T17:21:55.842807] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:21:56.251384] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-02. [2025-10-20T17:21:56.464059] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:21:57.051409] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-03. [2025-10-20T17:21:57.261545] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:21:57.851518] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-02. [2025-10-20T17:21:57.940677] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T17:21:58.651377] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-04. [2025-10-20T17:21:59.065161] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:21:59.451496] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-03. [2025-10-20T17:21:59.866127] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:22:00.251366] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-03. [2025-10-20T17:22:00.697217] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T17:22:01.051451] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-04. [2025-10-20T17:22:01.782572] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T17:22:01.851432] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-05. [2025-10-20T17:22:02.057682] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T17:22:02.651381] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-05. [2025-10-20T17:22:02.927085] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:22:03.451351] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-06. [2025-10-20T17:22:03.757679] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T17:22:04.251365] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-04. [2025-10-20T17:22:04.494019] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:22:05.051445] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-06. [2025-10-20T17:22:05.669248] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:22:05.851474] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-03. [2025-10-20T17:22:06.205326] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T17:22:06.651479] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-05. [2025-10-20T17:22:07.423573] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:22:07.451367] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-04. [2025-10-20T17:22:08.132120] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:22:08.251414] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-07. [2025-10-20T17:22:09.052228] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-06. [2025-10-20T17:22:09.059819] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:22:09.435383] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:22:09.851369] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-08. [2025-10-20T17:22:09.974816] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:22:10.651443] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-07. [2025-10-20T17:22:11.140129] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:22:11.451438] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-05. [2025-10-20T17:22:12.019856] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T17:22:12.251399] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-08. [2025-10-20T17:22:12.845260] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:22:13.051403] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-07. [2025-10-20T17:22:13.676071] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:22:13.851449] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-06. [2025-10-20T17:22:14.332418] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:22:14.651575] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-09. [2025-10-20T17:22:15.044630] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:22:15.451425] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-07. [2025-10-20T17:22:15.732972] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:22:16.251440] [INFO ] [REMOTE ] AVD: Fetching primary data for 2025-10. [2025-10-20T17:22:16.725608] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:22:17.051409] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-08. [2025-10-20T17:22:17.563530] [INFO ] [REMOTE ] AVD: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:22:17.566842] [INFO ] [REMOTE ] AVD: Full fetch complete (32733 records). [2025-10-20T17:22:17.566964] [DEBUG] [VALIDATOR ] AVD: Validating primary data. [2025-10-20T17:22:17.852406] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-09. [2025-10-20T17:22:17.942688] [DEBUG] [VALIDATOR ] AVD: All expected trading days are present. [2025-10-20T17:22:17.942790] [DEBUG] [VALIDATOR ] AVD: Primary data validation passed. [2025-10-20T17:22:17.943247] [INFO ] [MAIN ] AVD: Fetched primary data is valid. [2025-10-20T17:22:17.943288] [DEBUG] [LOCAL ] AVD: Saving primary data. [2025-10-20T17:22:17.944206] [DEBUG] [LOCAL ] AVD: Trimmed 32733 records to 21480 (keeping 200 days of data) [2025-10-20T17:22:18.066162] [DEBUG] [LOCAL ] AVD: Primary data saved (21480 records). [2025-10-20T17:22:18.066382] [INFO ] [MAIN ] AVD: Full history saved. [2025-10-20T17:22:18.066509] [INFO ] [MAIN ] Update Progress: 183/2367 symbols completed. AVD took 27.3s. Estimated remaining: 4h 25m 16s. Rate: 8.2 symbols/min [2025-10-20T17:22:18.217113] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T17:22:18.276135] [INFO ] [MAIN ] AVNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:18.321925] [DEBUG] [LOCAL ] AVNT: Loaded 39562 existing records from primary storage. [2025-10-20T17:22:18.322499] [INFO ] [MAIN ] AVNT: Loaded 39562 existing records. [2025-10-20T17:22:18.527163] [WARN ] [MAIN ] AVNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:18.527737] [INFO ] [MAIN ] AVNT: Wiping existing primary data before full refresh. [2025-10-20T17:22:18.527796] [INFO ] [LOCAL ] AVNT: Starting deletion of primary data. [2025-10-20T17:22:18.528065] [DEBUG] [LOCAL ] AVNT: Deleted primary data file. [2025-10-20T17:22:18.528122] [INFO ] [MAIN ] AVNT: Starting full primary refresh. [2025-10-20T17:22:18.656634] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-08. [2025-10-20T17:22:18.982963] [INFO ] [REMOTE ] AVNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:19.352132] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T17:22:19.451382] [INFO ] [REMOTE ] AVNS: Fetching primary data for 2025-10. [2025-10-20T17:22:20.251374] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-09. [2025-10-20T17:22:20.514787] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T17:22:20.601584] [INFO ] [REMOTE ] AVNS: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:22:20.607578] [INFO ] [REMOTE ] AVNS: Full fetch complete (48430 records). [2025-10-20T17:22:20.607781] [DEBUG] [VALIDATOR ] AVNS: Validating primary data. [2025-10-20T17:22:21.048340] [DEBUG] [VALIDATOR ] AVNS: All expected trading days are present. [2025-10-20T17:22:21.048579] [DEBUG] [VALIDATOR ] AVNS: Primary data validation passed. [2025-10-20T17:22:21.054755] [INFO ] [MAIN ] AVNS: Fetched primary data is valid. [2025-10-20T17:22:21.054875] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-09. [2025-10-20T17:22:21.062888] [DEBUG] [LOCAL ] AVNS: Saving primary data. [2025-10-20T17:22:21.065569] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:22:21.072601] [DEBUG] [LOCAL ] AVNS: Trimmed 48430 records to 33631 (keeping 200 days of data) [2025-10-20T17:22:21.291065] [DEBUG] [LOCAL ] AVNS: Primary data saved (33631 records). [2025-10-20T17:22:21.291301] [INFO ] [MAIN ] AVNS: Full history saved. [2025-10-20T17:22:21.291455] [INFO ] [MAIN ] Update Progress: 184/2367 symbols completed. AVNS took 28.2s. Estimated remaining: 4h 24m 21s. Rate: 8.3 symbols/min [2025-10-20T17:22:21.472708] [INFO ] [MAIN ] AVTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:21.537445] [DEBUG] [LOCAL ] AVTR: Loaded 54352 existing records from primary storage. [2025-10-20T17:22:21.537933] [INFO ] [MAIN ] AVTR: Loaded 54352 existing records. [2025-10-20T17:22:21.758701] [WARN ] [MAIN ] AVTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:21.759055] [INFO ] [MAIN ] AVTR: Wiping existing primary data before full refresh. [2025-10-20T17:22:21.759113] [INFO ] [LOCAL ] AVTR: Starting deletion of primary data. [2025-10-20T17:22:21.759711] [DEBUG] [LOCAL ] AVTR: Deleted primary data file. [2025-10-20T17:22:21.759771] [INFO ] [MAIN ] AVTR: Starting full primary refresh. [2025-10-20T17:22:21.861416] [INFO ] [REMOTE ] AVK: Fetching primary data for 2025-10. [2025-10-20T17:22:22.149892] [INFO ] [REMOTE ] AVTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:22.651402] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-01. [2025-10-20T17:22:22.838315] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:22:23.041885] [INFO ] [REMOTE ] AVK: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:22:23.045689] [INFO ] [REMOTE ] AVK: Full fetch complete (26698 records). [2025-10-20T17:22:23.045788] [DEBUG] [VALIDATOR ] AVK: Validating primary data. [2025-10-20T17:22:23.431194] [DEBUG] [VALIDATOR ] AVK: All expected trading days are present. [2025-10-20T17:22:23.431295] [DEBUG] [VALIDATOR ] AVK: Primary data validation passed. [2025-10-20T17:22:23.431753] [INFO ] [MAIN ] AVK: Fetched primary data is valid. [2025-10-20T17:22:23.431817] [DEBUG] [LOCAL ] AVK: Saving primary data. [2025-10-20T17:22:23.432630] [DEBUG] [LOCAL ] AVK: Trimmed 26698 records to 18702 (keeping 200 days of data) [2025-10-20T17:22:23.483849] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-01. [2025-10-20T17:22:23.562171] [DEBUG] [LOCAL ] AVK: Primary data saved (18702 records). [2025-10-20T17:22:23.563171] [INFO ] [MAIN ] AVK: Full history saved. [2025-10-20T17:22:23.563283] [INFO ] [MAIN ] Update Progress: 185/2367 symbols completed. AVK took 32.6s. Estimated remaining: 4h 23m 15s. Rate: 8.3 symbols/min [2025-10-20T17:22:23.746351] [INFO ] [MAIN ] AVY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:23.788518] [DEBUG] [LOCAL ] AVY: Loaded 38653 existing records from primary storage. [2025-10-20T17:22:23.788952] [INFO ] [MAIN ] AVY: Loaded 38653 existing records. [2025-10-20T17:22:23.994710] [WARN ] [MAIN ] AVY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:23.994942] [INFO ] [MAIN ] AVY: Wiping existing primary data before full refresh. [2025-10-20T17:22:23.994992] [INFO ] [LOCAL ] AVY: Starting deletion of primary data. [2025-10-20T17:22:23.995284] [DEBUG] [LOCAL ] AVY: Deleted primary data file. [2025-10-20T17:22:23.995323] [INFO ] [MAIN ] AVY: Starting full primary refresh. [2025-10-20T17:22:24.252249] [INFO ] [REMOTE ] AVB: Fetching primary data for 2025-10. [2025-10-20T17:22:24.479568] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:22:24.532033] [INFO ] [REMOTE ] AVY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:24.593406] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T17:22:25.051378] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-02. [2025-10-20T17:22:25.851383] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-02. [2025-10-20T17:22:25.953002] [INFO ] [REMOTE ] AVB: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T17:22:25.960225] [INFO ] [REMOTE ] AVB: Full fetch complete (59219 records). [2025-10-20T17:22:25.960737] [DEBUG] [VALIDATOR ] AVB: Validating primary data. [2025-10-20T17:22:26.375583] [DEBUG] [VALIDATOR ] AVB: All expected trading days are present. [2025-10-20T17:22:26.375714] [DEBUG] [VALIDATOR ] AVB: Primary data validation passed. [2025-10-20T17:22:26.376383] [INFO ] [MAIN ] AVB: Fetched primary data is valid. [2025-10-20T17:22:26.376427] [DEBUG] [LOCAL ] AVB: Saving primary data. [2025-10-20T17:22:26.377866] [DEBUG] [LOCAL ] AVB: Trimmed 59219 records to 42300 (keeping 200 days of data) [2025-10-20T17:22:26.667507] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-01. [2025-10-20T17:22:26.678064] [DEBUG] [LOCAL ] AVB: Primary data saved (42300 records). [2025-10-20T17:22:26.678353] [INFO ] [MAIN ] AVB: Full history saved. [2025-10-20T17:22:26.679020] [INFO ] [MAIN ] Update Progress: 186/2367 symbols completed. AVB took 41.8s. Estimated remaining: 4h 22m 19s. Rate: 8.3 symbols/min [2025-10-20T17:22:26.906033] [INFO ] [MAIN ] AWF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:26.912879] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T17:22:26.951334] [DEBUG] [LOCAL ] AWF: Loaded 24387 existing records from primary storage. [2025-10-20T17:22:26.951671] [INFO ] [MAIN ] AWF: Loaded 24387 existing records. [2025-10-20T17:22:27.150039] [WARN ] [MAIN ] AWF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:27.150375] [INFO ] [MAIN ] AWF: Wiping existing primary data before full refresh. [2025-10-20T17:22:27.150774] [INFO ] [LOCAL ] AWF: Starting deletion of primary data. [2025-10-20T17:22:27.151021] [DEBUG] [LOCAL ] AWF: Deleted primary data file. [2025-10-20T17:22:27.151101] [INFO ] [MAIN ] AWF: Starting full primary refresh. [2025-10-20T17:22:27.431270] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:22:27.456502] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-03. [2025-10-20T17:22:27.559807] [INFO ] [REMOTE ] AWF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:28.238732] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:22:28.251356] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-02. [2025-10-20T17:22:29.051434] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-03. [2025-10-20T17:22:29.343385] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:22:29.381989] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T17:22:29.851372] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-01. [2025-10-20T17:22:30.492949] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T17:22:30.651392] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-04. [2025-10-20T17:22:31.068685] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:22:31.451475] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-02. [2025-10-20T17:22:31.798590] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T17:22:32.251429] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-04. [2025-10-20T17:22:32.543467] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:22:33.051377] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-03. [2025-10-20T17:22:33.734488] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:22:33.851448] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-05. [2025-10-20T17:22:34.265352] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:22:34.651438] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-03. [2025-10-20T17:22:35.164135] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:22:35.451410] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-05. [2025-10-20T17:22:35.837074] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T17:22:36.251353] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-06. [2025-10-20T17:22:37.051352] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-04. [2025-10-20T17:22:37.231574] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:22:37.319626] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T17:22:37.851431] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-07. [2025-10-20T17:22:38.660296] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-04. [2025-10-20T17:22:38.663539] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:22:39.139214] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:22:39.451470] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-06. [2025-10-20T17:22:39.530819] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T17:22:40.251448] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-05. [2025-10-20T17:22:41.051387] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-08. [2025-10-20T17:22:41.129112] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:22:41.851400] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-07. [2025-10-20T17:22:41.939232] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T17:22:42.517978] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:22:42.651379] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-06. [2025-10-20T17:22:43.255274] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:22:43.451375] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-09. [2025-10-20T17:22:44.016792] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:22:44.251397] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-05. [2025-10-20T17:22:44.826506] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:22:45.051380] [INFO ] [REMOTE ] AVNT: Fetching primary data for 2025-10. [2025-10-20T17:22:45.458331] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T17:22:45.851446] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-08. [2025-10-20T17:22:46.356438] [INFO ] [REMOTE ] AVNT: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:22:46.363312] [INFO ] [REMOTE ] AVNT: Full fetch complete (56309 records). [2025-10-20T17:22:46.363450] [DEBUG] [VALIDATOR ] AVNT: Validating primary data. [2025-10-20T17:22:46.656530] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-07. [2025-10-20T17:22:46.780444] [DEBUG] [VALIDATOR ] AVNT: All expected trading days are present. [2025-10-20T17:22:46.780513] [DEBUG] [VALIDATOR ] AVNT: Primary data validation passed. [2025-10-20T17:22:46.781245] [INFO ] [MAIN ] AVNT: Fetched primary data is valid. [2025-10-20T17:22:46.781311] [DEBUG] [LOCAL ] AVNT: Saving primary data. [2025-10-20T17:22:46.782570] [DEBUG] [LOCAL ] AVNT: Trimmed 56309 records to 38938 (keeping 200 days of data) [2025-10-20T17:22:47.017901] [DEBUG] [LOCAL ] AVNT: Primary data saved (38938 records). [2025-10-20T17:22:47.018055] [INFO ] [MAIN ] AVNT: Full history saved. [2025-10-20T17:22:47.018136] [INFO ] [MAIN ] Update Progress: 187/2367 symbols completed. AVNT took 29.0s. Estimated remaining: 4h 24m 45s. Rate: 8.2 symbols/min [2025-10-20T17:22:47.213767] [INFO ] [MAIN ] AWI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:47.242135] [DEBUG] [LOCAL ] AWI: Loaded 25722 existing records from primary storage. [2025-10-20T17:22:47.242413] [INFO ] [MAIN ] AWI: Loaded 25722 existing records. [2025-10-20T17:22:47.427777] [WARN ] [MAIN ] AWI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:47.428224] [INFO ] [MAIN ] AWI: Wiping existing primary data before full refresh. [2025-10-20T17:22:47.428385] [INFO ] [LOCAL ] AWI: Starting deletion of primary data. [2025-10-20T17:22:47.428666] [DEBUG] [LOCAL ] AWI: Deleted primary data file. [2025-10-20T17:22:47.428724] [INFO ] [MAIN ] AWI: Starting full primary refresh. [2025-10-20T17:22:47.460362] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-06. [2025-10-20T17:22:47.844608] [INFO ] [REMOTE ] AWI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:47.879758] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T17:22:48.127400] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:22:48.251411] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-08. [2025-10-20T17:22:48.582524] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:22:49.051484] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-09. [2025-10-20T17:22:49.498093] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T17:22:49.851399] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-09. [2025-10-20T17:22:50.651469] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-07. [2025-10-20T17:22:50.902439] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:22:51.303141] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:22:51.367686] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T17:22:51.451387] [INFO ] [REMOTE ] AVTR: Fetching primary data for 2025-10. [2025-10-20T17:22:52.251407] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-08. [2025-10-20T17:22:53.033198] [INFO ] [REMOTE ] AVTR: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:22:53.042259] [INFO ] [REMOTE ] AVTR: Full fetch complete (77650 records). [2025-10-20T17:22:53.042433] [DEBUG] [VALIDATOR ] AVTR: Validating primary data. [2025-10-20T17:22:53.057245] [INFO ] [REMOTE ] AVY: Fetching primary data for 2025-10. [2025-10-20T17:22:53.522607] [DEBUG] [VALIDATOR ] AVTR: All expected trading days are present. [2025-10-20T17:22:53.527978] [DEBUG] [VALIDATOR ] AVTR: Primary data validation passed. [2025-10-20T17:22:53.538199] [INFO ] [MAIN ] AVTR: Fetched primary data is valid. [2025-10-20T17:22:53.538990] [DEBUG] [LOCAL ] AVTR: Saving primary data. [2025-10-20T17:22:53.539675] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T17:22:53.546842] [DEBUG] [LOCAL ] AVTR: Trimmed 77650 records to 53572 (keeping 200 days of data) [2025-10-20T17:22:53.853833] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-01. [2025-10-20T17:22:53.867991] [DEBUG] [LOCAL ] AVTR: Primary data saved (53572 records). [2025-10-20T17:22:53.868204] [INFO ] [MAIN ] AVTR: Full history saved. [2025-10-20T17:22:53.868308] [INFO ] [MAIN ] Update Progress: 188/2367 symbols completed. AVTR took 32.6s. Estimated remaining: 4h 24m 33s. Rate: 8.2 symbols/min [2025-10-20T17:22:54.084715] [INFO ] [MAIN ] AWK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:54.137001] [DEBUG] [LOCAL ] AWK: Loaded 49233 existing records from primary storage. [2025-10-20T17:22:54.137381] [INFO ] [MAIN ] AWK: Loaded 49233 existing records. [2025-10-20T17:22:54.323151] [WARN ] [MAIN ] AWK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:54.323409] [INFO ] [MAIN ] AWK: Wiping existing primary data before full refresh. [2025-10-20T17:22:54.323459] [INFO ] [LOCAL ] AWK: Starting deletion of primary data. [2025-10-20T17:22:54.323729] [DEBUG] [LOCAL ] AWK: Deleted primary data file. [2025-10-20T17:22:54.323786] [INFO ] [MAIN ] AWK: Starting full primary refresh. [2025-10-20T17:22:54.660680] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-09. [2025-10-20T17:22:54.690145] [INFO ] [REMOTE ] AWK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:54.788239] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T17:22:54.901598] [INFO ] [REMOTE ] AVY: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:22:54.907981] [INFO ] [REMOTE ] AVY: Full fetch complete (56634 records). [2025-10-20T17:22:54.908105] [DEBUG] [VALIDATOR ] AVY: Validating primary data. [2025-10-20T17:22:55.297431] [DEBUG] [VALIDATOR ] AVY: All expected trading days are present. [2025-10-20T17:22:55.297565] [DEBUG] [VALIDATOR ] AVY: Primary data validation passed. [2025-10-20T17:22:55.298211] [INFO ] [MAIN ] AVY: Fetched primary data is valid. [2025-10-20T17:22:55.298275] [DEBUG] [LOCAL ] AVY: Saving primary data. [2025-10-20T17:22:55.299581] [DEBUG] [LOCAL ] AVY: Trimmed 56634 records to 38172 (keeping 200 days of data) [2025-10-20T17:22:55.479482] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-02. [2025-10-20T17:22:55.576521] [DEBUG] [LOCAL ] AVY: Primary data saved (38172 records). [2025-10-20T17:22:55.578037] [INFO ] [MAIN ] AVY: Full history saved. [2025-10-20T17:22:55.577920] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:22:55.578226] [INFO ] [MAIN ] Update Progress: 189/2367 symbols completed. AVY took 32.0s. Estimated remaining: 4h 23m 21s. Rate: 8.3 symbols/min [2025-10-20T17:22:55.831290] [INFO ] [MAIN ] AWP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:55.863086] [DEBUG] [LOCAL ] AWP: Loaded 21887 existing records from primary storage. [2025-10-20T17:22:55.863480] [INFO ] [MAIN ] AWP: Loaded 21887 existing records. [2025-10-20T17:22:56.144371] [WARN ] [MAIN ] AWP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:56.144681] [INFO ] [MAIN ] AWP: Wiping existing primary data before full refresh. [2025-10-20T17:22:56.144745] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-10-20T17:22:56.144985] [DEBUG] [LOCAL ] AWP: Deleted primary data file. [2025-10-20T17:22:56.145042] [INFO ] [MAIN ] AWP: Starting full primary refresh. [2025-10-20T17:22:56.256838] [INFO ] [REMOTE ] AWF: Fetching primary data for 2025-10. [2025-10-20T17:22:56.621795] [INFO ] [REMOTE ] AWP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:56.700975] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:22:57.051444] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-01. [2025-10-20T17:22:57.173364] [INFO ] [REMOTE ] AWF: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T17:22:57.177426] [INFO ] [REMOTE ] AWF: Full fetch complete (35941 records). [2025-10-20T17:22:57.177525] [DEBUG] [VALIDATOR ] AWF: Validating primary data. [2025-10-20T17:22:57.576659] [DEBUG] [VALIDATOR ] AWF: All expected trading days are present. [2025-10-20T17:22:57.576754] [DEBUG] [VALIDATOR ] AWF: Primary data validation passed. [2025-10-20T17:22:57.577406] [INFO ] [MAIN ] AWF: Fetched primary data is valid. [2025-10-20T17:22:57.577502] [DEBUG] [LOCAL ] AWF: Saving primary data. [2025-10-20T17:22:57.578710] [DEBUG] [LOCAL ] AWF: Trimmed 35941 records to 24036 (keeping 200 days of data) [2025-10-20T17:22:57.728191] [DEBUG] [LOCAL ] AWF: Primary data saved (24036 records). [2025-10-20T17:22:57.728334] [INFO ] [MAIN ] AWF: Full history saved. [2025-10-20T17:22:57.728443] [INFO ] [MAIN ] Update Progress: 190/2367 symbols completed. AWF took 31.0s. Estimated remaining: 4h 22m 15s. Rate: 8.3 symbols/min [2025-10-20T17:22:57.856485] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-01. [2025-10-20T17:22:57.913631] [INFO ] [MAIN ] AWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:22:57.934858] [DEBUG] [LOCAL ] AWR: Loaded 23114 existing records from primary storage. [2025-10-20T17:22:57.935158] [INFO ] [MAIN ] AWR: Loaded 23114 existing records. [2025-10-20T17:22:58.127286] [WARN ] [MAIN ] AWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:22:58.127534] [INFO ] [MAIN ] AWR: Wiping existing primary data before full refresh. [2025-10-20T17:22:58.127611] [INFO ] [LOCAL ] AWR: Starting deletion of primary data. [2025-10-20T17:22:58.127847] [DEBUG] [LOCAL ] AWR: Deleted primary data file. [2025-10-20T17:22:58.127939] [INFO ] [MAIN ] AWR: Starting full primary refresh. [2025-10-20T17:22:58.393059] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:22:58.556331] [INFO ] [REMOTE ] AWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:22:58.651425] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-02. [2025-10-20T17:22:59.451371] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-03. [2025-10-20T17:22:59.668713] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T17:22:59.754234] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:23:00.251429] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-01. [2025-10-20T17:23:00.707564] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:23:01.051400] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-02. [2025-10-20T17:23:01.574120] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:23:01.851434] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-03. [2025-10-20T17:23:02.647988] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:23:02.656902] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-04. [2025-10-20T17:23:02.696298] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T17:23:03.451405] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-04. [2025-10-20T17:23:03.987621] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:23:04.251425] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-02. [2025-10-20T17:23:04.482288] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T17:23:05.051432] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-03. [2025-10-20T17:23:05.362556] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:23:05.851421] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-03. [2025-10-20T17:23:06.651365] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-05. [2025-10-20T17:23:06.839395] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:23:07.081156] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:23:07.451596] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-04. [2025-10-20T17:23:07.552793] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:23:08.251398] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-04. [2025-10-20T17:23:09.051409] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-05. [2025-10-20T17:23:09.319343] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:23:09.369153] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:23:09.851601] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-05. [2025-10-20T17:23:10.354543] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T17:23:10.651469] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-06. [2025-10-20T17:23:10.996621] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:23:11.451389] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-05. [2025-10-20T17:23:11.803730] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T17:23:12.251413] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-06. [2025-10-20T17:23:13.051378] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-06. [2025-10-20T17:23:13.353448] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:23:13.395241] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:23:13.851393] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-07. [2025-10-20T17:23:13.973122] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T17:23:14.651405] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-07. [2025-10-20T17:23:15.164332] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:23:15.451369] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-08. [2025-10-20T17:23:15.604425] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:23:16.251368] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-07. [2025-10-20T17:23:16.342136] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:23:17.051385] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-06. [2025-10-20T17:23:17.384929] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T17:23:17.851384] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-08. [2025-10-20T17:23:18.347145] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:23:18.651399] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-07. [2025-10-20T17:23:18.985293] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:23:19.451425] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-09. [2025-10-20T17:23:20.049218] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:23:20.251425] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-08. [2025-10-20T17:23:20.475497] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:23:21.051377] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-09. [2025-10-20T17:23:21.263952] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:23:21.851416] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-09. [2025-10-20T17:23:22.186639] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:23:22.651356] [INFO ] [REMOTE ] AWP: Fetching primary data for 2025-10. [2025-10-20T17:23:22.981393] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:23:23.451397] [INFO ] [REMOTE ] AWR: Fetching primary data for 2025-10. [2025-10-20T17:23:23.782730] [INFO ] [REMOTE ] AWP: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T17:23:23.786833] [INFO ] [REMOTE ] AWP: Full fetch complete (31702 records). [2025-10-20T17:23:23.786938] [DEBUG] [VALIDATOR ] AWP: Validating primary data. [2025-10-20T17:23:23.787568] [FATAL] [VALIDATOR ] AWP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:23:23.787688] [FATAL] [MAIN ] AWP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:23:23.787759] [WARN ] [MAIN ] AWP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:23:23.787838] [INFO ] [LOCAL ] AWP: Starting deletion of primary data. [2025-10-20T17:23:23.787935] [DEBUG] [LOCAL ] AWP: No primary data file found to delete. [2025-10-20T17:23:23.787992] [INFO ] [LOCAL ] AWP: Starting deletion of derivative data. [2025-10-20T17:23:23.788243] [DEBUG] [LOCAL ] AWP: Deleted derivative data file. [2025-10-20T17:23:23.788354] [INFO ] [MAIN ] Update Progress: 191/2367 symbols completed. AWP took 28.2s. Estimated remaining: 4h 25m 43s. Rate: 8.2 symbols/min [2025-10-20T17:23:24.001257] [INFO ] [MAIN ] AX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:24.001453] [DEBUG] [LOCAL ] AX: No existing data file found in primary storage. [2025-10-20T17:23:24.001490] [WARN ] [MAIN ] AX: No existing data found, performing full refresh instead. [2025-10-20T17:23:24.001629] [INFO ] [MAIN ] AX: Wiping existing primary data before full refresh. [2025-10-20T17:23:24.001706] [INFO ] [LOCAL ] AX: Starting deletion of primary data. [2025-10-20T17:23:24.001758] [DEBUG] [LOCAL ] AX: No primary data file found to delete. [2025-10-20T17:23:24.001790] [INFO ] [MAIN ] AX: Starting full primary refresh. [2025-10-20T17:23:24.256545] [INFO ] [REMOTE ] AWI: Fetching primary data for 2025-10. [2025-10-20T17:23:24.360122] [INFO ] [REMOTE ] AX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:24.601988] [INFO ] [REMOTE ] AWR: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:23:24.607348] [INFO ] [REMOTE ] AWR: Full fetch complete (32381 records). [2025-10-20T17:23:24.607480] [DEBUG] [VALIDATOR ] AWR: Validating primary data. [2025-10-20T17:23:24.988467] [DEBUG] [VALIDATOR ] AWR: All expected trading days are present. [2025-10-20T17:23:24.988589] [DEBUG] [VALIDATOR ] AWR: Primary data validation passed. [2025-10-20T17:23:24.989143] [INFO ] [MAIN ] AWR: Fetched primary data is valid. [2025-10-20T17:23:24.989210] [DEBUG] [LOCAL ] AWR: Saving primary data. [2025-10-20T17:23:24.990467] [DEBUG] [LOCAL ] AWR: Trimmed 32381 records to 22778 (keeping 200 days of data) [2025-10-20T17:23:25.052818] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-01. [2025-10-20T17:23:25.157314] [DEBUG] [LOCAL ] AWR: Primary data saved (22778 records). [2025-10-20T17:23:25.157543] [INFO ] [MAIN ] AWR: Full history saved. [2025-10-20T17:23:25.157673] [INFO ] [MAIN ] Update Progress: 192/2367 symbols completed. AWR took 27.4s. Estimated remaining: 4h 24m 28s. Rate: 8.2 symbols/min [2025-10-20T17:23:25.345770] [INFO ] [MAIN ] AXL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:25.404781] [DEBUG] [LOCAL ] AXL: Loaded 51268 existing records from primary storage. [2025-10-20T17:23:25.405286] [INFO ] [MAIN ] AXL: Loaded 51268 existing records. [2025-10-20T17:23:25.550242] [INFO ] [REMOTE ] AWI: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:23:25.576858] [INFO ] [REMOTE ] AWI: Full fetch complete (36037 records). [2025-10-20T17:23:25.592499] [DEBUG] [VALIDATOR ] AWI: Validating primary data. [2025-10-20T17:23:25.752145] [WARN ] [MAIN ] AXL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:25.754722] [INFO ] [MAIN ] AXL: Wiping existing primary data before full refresh. [2025-10-20T17:23:25.754797] [INFO ] [LOCAL ] AXL: Starting deletion of primary data. [2025-10-20T17:23:25.760437] [DEBUG] [LOCAL ] AXL: Deleted primary data file. [2025-10-20T17:23:25.776013] [INFO ] [MAIN ] AXL: Starting full primary refresh. [2025-10-20T17:23:25.877789] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-08. [2025-10-20T17:23:26.404267] [DEBUG] [VALIDATOR ] AWI: All expected trading days are present. [2025-10-20T17:23:26.404004] [INFO ] [REMOTE ] AXL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:26.404785] [DEBUG] [VALIDATOR ] AWI: Primary data validation passed. [2025-10-20T17:23:26.405364] [INFO ] [MAIN ] AWI: Fetched primary data is valid. [2025-10-20T17:23:26.405448] [DEBUG] [LOCAL ] AWI: Saving primary data. [2025-10-20T17:23:26.406391] [DEBUG] [LOCAL ] AWI: Trimmed 36037 records to 25285 (keeping 200 days of data) [2025-10-20T17:23:26.566627] [DEBUG] [LOCAL ] AWI: Primary data saved (25285 records). [2025-10-20T17:23:26.566894] [INFO ] [MAIN ] AWI: Full history saved. [2025-10-20T17:23:26.567044] [INFO ] [MAIN ] Update Progress: 193/2367 symbols completed. AWI took 39.5s. Estimated remaining: 4h 23m 14s. Rate: 8.3 symbols/min [2025-10-20T17:23:26.658983] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-01. [2025-10-20T17:23:26.754873] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:23:26.792668] [INFO ] [MAIN ] AXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:26.848149] [DEBUG] [LOCAL ] AXP: Loaded 53945 existing records from primary storage. [2025-10-20T17:23:26.848620] [INFO ] [MAIN ] AXP: Loaded 53945 existing records. [2025-10-20T17:23:27.026046] [WARN ] [MAIN ] AXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:27.026248] [INFO ] [MAIN ] AXP: Wiping existing primary data before full refresh. [2025-10-20T17:23:27.026292] [INFO ] [LOCAL ] AXP: Starting deletion of primary data. [2025-10-20T17:23:27.026511] [DEBUG] [LOCAL ] AXP: Deleted primary data file. [2025-10-20T17:23:27.026566] [INFO ] [MAIN ] AXP: Starting full primary refresh. [2025-10-20T17:23:27.371010] [INFO ] [REMOTE ] AXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:27.451390] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-02. [2025-10-20T17:23:27.781785] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T17:23:28.251413] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-01. [2025-10-20T17:23:28.463194] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:23:28.682734] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:23:29.051401] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-02. [2025-10-20T17:23:29.851419] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-03. [2025-10-20T17:23:30.651401] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-09. [2025-10-20T17:23:30.701685] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-01 (2.45 seconds). [2025-10-20T17:23:30.750621] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:23:31.171411] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T17:23:31.451454] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-03. [2025-10-20T17:23:32.014205] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:23:32.251380] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-02. [2025-10-20T17:23:33.051441] [INFO ] [REMOTE ] AWK: Fetching primary data for 2025-10. [2025-10-20T17:23:33.121363] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:23:33.851434] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-04. [2025-10-20T17:23:34.227881] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:23:34.651388] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-04. [2025-10-20T17:23:34.789578] [INFO ] [REMOTE ] AWK: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:23:34.797510] [INFO ] [REMOTE ] AWK: Full fetch complete (70910 records). [2025-10-20T17:23:34.797678] [DEBUG] [VALIDATOR ] AWK: Validating primary data. [2025-10-20T17:23:35.279032] [DEBUG] [VALIDATOR ] AWK: All expected trading days are present. [2025-10-20T17:23:35.279823] [DEBUG] [VALIDATOR ] AWK: Primary data validation passed. [2025-10-20T17:23:35.280581] [INFO ] [MAIN ] AWK: Fetched primary data is valid. [2025-10-20T17:23:35.281933] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:23:35.282044] [DEBUG] [LOCAL ] AWK: Saving primary data. [2025-10-20T17:23:35.291964] [DEBUG] [LOCAL ] AWK: Trimmed 70910 records to 48510 (keeping 200 days of data) [2025-10-20T17:23:35.474820] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-05. [2025-10-20T17:23:35.599627] [DEBUG] [LOCAL ] AWK: Primary data saved (48510 records). [2025-10-20T17:23:35.599803] [INFO ] [MAIN ] AWK: Full history saved. [2025-10-20T17:23:35.600493] [INFO ] [MAIN ] Update Progress: 194/2367 symbols completed. AWK took 41.7s. Estimated remaining: 4h 23m 27s. Rate: 8.2 symbols/min [2025-10-20T17:23:35.768277] [INFO ] [MAIN ] AXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:35.773116] [DEBUG] [LOCAL ] AXR: Loaded 2975 existing records from primary storage. [2025-10-20T17:23:35.773474] [INFO ] [MAIN ] AXR: Loaded 2975 existing records. [2025-10-20T17:23:35.955791] [WARN ] [MAIN ] AXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:35.956099] [INFO ] [MAIN ] AXR: Wiping existing primary data before full refresh. [2025-10-20T17:23:35.956156] [INFO ] [LOCAL ] AXR: Starting deletion of primary data. [2025-10-20T17:23:35.956283] [DEBUG] [LOCAL ] AXR: Deleted primary data file. [2025-10-20T17:23:35.956321] [INFO ] [MAIN ] AXR: Starting full primary refresh. [2025-10-20T17:23:36.274498] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-03. [2025-10-20T17:23:36.356484] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:23:36.428844] [INFO ] [REMOTE ] AXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:36.460265] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T17:23:37.051465] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-06. [2025-10-20T17:23:37.851430] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-01. [2025-10-20T17:23:37.986346] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:23:38.449866] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T17:23:38.565229] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T17:23:38.651397] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-02. [2025-10-20T17:23:39.117826] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:23:39.451452] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-07. [2025-10-20T17:23:40.251455] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-04. [2025-10-20T17:23:40.687936] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:23:41.051455] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-03. [2025-10-20T17:23:41.622159] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T17:23:41.851425] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-08. [2025-10-20T17:23:42.618152] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-04 (2.37 seconds). [2025-10-20T17:23:42.656076] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-05. [2025-10-20T17:23:42.664108] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:23:43.451444] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-09. [2025-10-20T17:23:44.106993] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:23:44.251418] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-05. [2025-10-20T17:23:44.772809] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T17:23:45.051423] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-04. [2025-10-20T17:23:45.777137] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T17:23:45.851389] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-05. [2025-10-20T17:23:46.382237] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T17:23:46.541140] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:23:46.651440] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-06. [2025-10-20T17:23:47.451500] [INFO ] [REMOTE ] AX: Fetching primary data for 2025-10. [2025-10-20T17:23:48.239868] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T17:23:48.251325] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-07. [2025-10-20T17:23:48.862860] [INFO ] [REMOTE ] AX: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T17:23:48.868030] [INFO ] [REMOTE ] AX: Full fetch complete (42245 records). [2025-10-20T17:23:48.868204] [DEBUG] [VALIDATOR ] AX: Validating primary data. [2025-10-20T17:23:49.054432] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-06. [2025-10-20T17:23:49.267917] [DEBUG] [VALIDATOR ] AX: All expected trading days are present. [2025-10-20T17:23:49.268012] [DEBUG] [VALIDATOR ] AX: Primary data validation passed. [2025-10-20T17:23:49.268634] [INFO ] [MAIN ] AX: Fetched primary data is valid. [2025-10-20T17:23:49.268730] [DEBUG] [LOCAL ] AX: Saving primary data. [2025-10-20T17:23:49.269845] [DEBUG] [LOCAL ] AX: Trimmed 42245 records to 28103 (keeping 200 days of data) [2025-10-20T17:23:49.431054] [DEBUG] [LOCAL ] AX: Primary data saved (28103 records). [2025-10-20T17:23:49.431196] [INFO ] [MAIN ] AX: Full history saved. [2025-10-20T17:23:49.431270] [INFO ] [MAIN ] Update Progress: 195/2367 symbols completed. AX took 25.6s. Estimated remaining: 4h 24m 33s. Rate: 8.2 symbols/min [2025-10-20T17:23:49.600392] [INFO ] [MAIN ] AXS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:49.635254] [DEBUG] [LOCAL ] AXS: Loaded 39196 existing records from primary storage. [2025-10-20T17:23:49.635564] [INFO ] [MAIN ] AXS: Loaded 39196 existing records. [2025-10-20T17:23:49.819682] [WARN ] [MAIN ] AXS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:49.820142] [INFO ] [MAIN ] AXS: Wiping existing primary data before full refresh. [2025-10-20T17:23:49.820252] [INFO ] [LOCAL ] AXS: Starting deletion of primary data. [2025-10-20T17:23:49.820461] [DEBUG] [LOCAL ] AXS: Deleted primary data file. [2025-10-20T17:23:49.820528] [INFO ] [MAIN ] AXS: Starting full primary refresh. [2025-10-20T17:23:49.852529] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-06. [2025-10-20T17:23:49.931720] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T17:23:50.046152] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:23:50.248900] [INFO ] [REMOTE ] AXS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:50.651437] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-01. [2025-10-20T17:23:51.451390] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-07. [2025-10-20T17:23:51.497914] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T17:23:52.072758] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T17:23:52.251398] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-07. [2025-10-20T17:23:52.342788] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:23:53.051425] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-08. [2025-10-20T17:23:53.475460] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T17:23:53.851347] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-09. [2025-10-20T17:23:54.257197] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T17:23:54.535741] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-07 (2.28 seconds). [2025-10-20T17:23:54.651434] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-02. [2025-10-20T17:23:55.451376] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-08. [2025-10-20T17:23:55.749144] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:23:56.251394] [INFO ] [REMOTE ] AXR: Fetching primary data for 2025-10. [2025-10-20T17:23:56.761831] [INFO ] [REMOTE ] AXR: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:23:56.762988] [INFO ] [REMOTE ] AXR: Full fetch complete (4413 records). [2025-10-20T17:23:56.763066] [DEBUG] [VALIDATOR ] AXR: Validating primary data. [2025-10-20T17:23:57.056855] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-08. [2025-10-20T17:23:57.152333] [DEBUG] [VALIDATOR ] AXR: All expected trading days are present. [2025-10-20T17:23:57.152472] [DEBUG] [VALIDATOR ] AXR: Primary data validation passed. [2025-10-20T17:23:57.152733] [INFO ] [MAIN ] AXR: Fetched primary data is valid. [2025-10-20T17:23:57.152806] [DEBUG] [LOCAL ] AXR: Saving primary data. [2025-10-20T17:23:57.153183] [DEBUG] [LOCAL ] AXR: Trimmed 4413 records to 2925 (keeping 200 days of data) [2025-10-20T17:23:57.170471] [DEBUG] [LOCAL ] AXR: Primary data saved (2925 records). [2025-10-20T17:23:57.170684] [INFO ] [MAIN ] AXR: Full history saved. [2025-10-20T17:23:57.170788] [INFO ] [MAIN ] Update Progress: 196/2367 symbols completed. AXR took 21.6s. Estimated remaining: 4h 24m 30s. Rate: 8.2 symbols/min [2025-10-20T17:23:57.352863] [INFO ] [MAIN ] AXTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:23:57.413562] [DEBUG] [LOCAL ] AXTA: Loaded 53031 existing records from primary storage. [2025-10-20T17:23:57.413959] [INFO ] [MAIN ] AXTA: Loaded 53031 existing records. [2025-10-20T17:23:57.596708] [WARN ] [MAIN ] AXTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:23:57.597100] [INFO ] [MAIN ] AXTA: Wiping existing primary data before full refresh. [2025-10-20T17:23:57.597292] [INFO ] [LOCAL ] AXTA: Starting deletion of primary data. [2025-10-20T17:23:57.597610] [DEBUG] [LOCAL ] AXTA: Deleted primary data file. [2025-10-20T17:23:57.597711] [INFO ] [MAIN ] AXTA: Starting full primary refresh. [2025-10-20T17:23:57.858455] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-03. [2025-10-20T17:23:57.951574] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-08 (2.50 seconds). [2025-10-20T17:23:58.059160] [INFO ] [REMOTE ] AXTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:23:58.567634] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:23:58.651440] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-01. [2025-10-20T17:23:59.451456] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-09. [2025-10-20T17:23:59.531994] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:24:00.106463] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:24:00.251389] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-02. [2025-10-20T17:24:01.051523] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-04. [2025-10-20T17:24:01.170677] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:24:01.851346] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-09. [2025-10-20T17:24:02.082676] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T17:24:02.419967] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:24:02.651439] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-05. [2025-10-20T17:24:03.451401] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-03. [2025-10-20T17:24:03.938956] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T17:24:04.151130] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T17:24:04.251352] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-06. [2025-10-20T17:24:05.056500] [INFO ] [REMOTE ] AXP: Fetching primary data for 2025-10. [2025-10-20T17:24:05.069304] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:24:05.767053] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:24:05.851387] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-07. [2025-10-20T17:24:06.651387] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-04. [2025-10-20T17:24:06.928093] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T17:24:07.456631] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-08. [2025-10-20T17:24:07.490001] [INFO ] [REMOTE ] AXP: Incremental fetch complete for 2025-10 (2.43 seconds). [2025-10-20T17:24:07.500981] [INFO ] [REMOTE ] AXP: Full fetch complete (77124 records). [2025-10-20T17:24:07.501211] [DEBUG] [VALIDATOR ] AXP: Validating primary data. [2025-10-20T17:24:08.011830] [DEBUG] [VALIDATOR ] AXP: All expected trading days are present. [2025-10-20T17:24:08.012047] [DEBUG] [VALIDATOR ] AXP: Primary data validation passed. [2025-10-20T17:24:08.013188] [INFO ] [MAIN ] AXP: Fetched primary data is valid. [2025-10-20T17:24:08.013410] [DEBUG] [LOCAL ] AXP: Saving primary data. [2025-10-20T17:24:08.015828] [DEBUG] [LOCAL ] AXP: Trimmed 77124 records to 53169 (keeping 200 days of data) [2025-10-20T17:24:08.259616] [INFO ] [REMOTE ] AXL: Fetching primary data for 2025-10. [2025-10-20T17:24:08.432860] [DEBUG] [LOCAL ] AXP: Primary data saved (53169 records). [2025-10-20T17:24:08.437176] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:24:08.437393] [INFO ] [MAIN ] AXP: Full history saved. [2025-10-20T17:24:08.448787] [INFO ] [MAIN ] Update Progress: 197/2367 symbols completed. AXP took 41.9s. Estimated remaining: 4h 25m 6s. Rate: 8.2 symbols/min [2025-10-20T17:24:08.617259] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:24:09.051459] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-05. [2025-10-20T17:24:09.851414] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-09. [2025-10-20T17:24:10.180843] [INFO ] [REMOTE ] AXL: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:24:10.191383] [INFO ] [REMOTE ] AXL: Full fetch complete (73085 records). [2025-10-20T17:24:10.191685] [DEBUG] [VALIDATOR ] AXL: Validating primary data. [2025-10-20T17:24:10.618141] [DEBUG] [VALIDATOR ] AXL: All expected trading days are present. [2025-10-20T17:24:10.618291] [DEBUG] [VALIDATOR ] AXL: Primary data validation passed. [2025-10-20T17:24:10.619354] [INFO ] [MAIN ] AXL: Fetched primary data is valid. [2025-10-20T17:24:10.619467] [DEBUG] [LOCAL ] AXL: Saving primary data. [2025-10-20T17:24:10.621692] [DEBUG] [LOCAL ] AXL: Trimmed 73085 records to 50539 (keeping 200 days of data) [2025-10-20T17:24:10.766034] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:24:10.787209] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-06. [2025-10-20T17:24:10.982057] [DEBUG] [LOCAL ] AXL: Primary data saved (50539 records). [2025-10-20T17:24:10.982419] [INFO ] [MAIN ] AXL: Full history saved. [2025-10-20T17:24:10.982546] [INFO ] [MAIN ] Update Progress: 198/2367 symbols completed. AXL took 45.8s. Estimated remaining: 4h 24m 7s. Rate: 8.2 symbols/min [2025-10-20T17:24:11.027923] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T17:24:11.451397] [INFO ] [REMOTE ] AXS: Fetching primary data for 2025-10. [2025-10-20T17:24:12.569678] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:24:12.581241] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-07. [2025-10-20T17:24:13.085732] [INFO ] [REMOTE ] AXS: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:24:13.096083] [INFO ] [REMOTE ] AXS: Full fetch complete (56020 records). [2025-10-20T17:24:13.096346] [DEBUG] [VALIDATOR ] AXS: Validating primary data. [2025-10-20T17:24:13.521440] [DEBUG] [VALIDATOR ] AXS: All expected trading days are present. [2025-10-20T17:24:13.521565] [DEBUG] [VALIDATOR ] AXS: Primary data validation passed. [2025-10-20T17:24:13.522448] [INFO ] [MAIN ] AXS: Fetched primary data is valid. [2025-10-20T17:24:13.522519] [DEBUG] [LOCAL ] AXS: Saving primary data. [2025-10-20T17:24:13.524385] [DEBUG] [LOCAL ] AXS: Trimmed 56020 records to 38702 (keeping 200 days of data) [2025-10-20T17:24:13.776292] [DEBUG] [LOCAL ] AXS: Primary data saved (38702 records). [2025-10-20T17:24:13.776463] [INFO ] [MAIN ] AXS: Full history saved. [2025-10-20T17:24:13.776554] [INFO ] [MAIN ] Update Progress: 199/2367 symbols completed. AXS took 24.3s. Estimated remaining: 4h 23m 10s. Rate: 8.2 symbols/min [2025-10-20T17:24:14.206416] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T17:24:14.218450] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-08. [2025-10-20T17:24:15.838324] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:24:15.851003] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-09. [2025-10-20T17:24:17.632983] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:24:17.645174] [INFO ] [REMOTE ] AXTA: Fetching primary data for 2025-10. [2025-10-20T17:24:19.557910] [INFO ] [REMOTE ] AXTA: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T17:24:19.566786] [INFO ] [REMOTE ] AXTA: Full fetch complete (75415 records). [2025-10-20T17:24:19.567002] [DEBUG] [VALIDATOR ] AXTA: Validating primary data. [2025-10-20T17:24:19.992164] [DEBUG] [VALIDATOR ] AXTA: All expected trading days are present. [2025-10-20T17:24:19.992327] [DEBUG] [VALIDATOR ] AXTA: Primary data validation passed. [2025-10-20T17:24:19.993209] [INFO ] [MAIN ] AXTA: Fetched primary data is valid. [2025-10-20T17:24:19.993278] [DEBUG] [LOCAL ] AXTA: Saving primary data. [2025-10-20T17:24:19.995996] [DEBUG] [LOCAL ] AXTA: Trimmed 75415 records to 52286 (keeping 200 days of data) [2025-10-20T17:24:20.317385] [DEBUG] [LOCAL ] AXTA: Primary data saved (52286 records). [2025-10-20T17:24:20.317540] [INFO ] [MAIN ] AXTA: Full history saved. [2025-10-20T17:24:20.317621] [INFO ] [MAIN ] Update Progress: 200/2367 symbols completed. AXTA took 23.1s. Estimated remaining: 4h 22m 55s. Rate: 8.2 symbols/min [2025-10-20T17:24:21.104042] [INFO ] [MAIN ] AZO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.175934] [INFO ] [MAIN ] B: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.170600] [INFO ] [MAIN ] AZZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.178701] [INFO ] [MAIN ] AYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:21.176120] [DEBUG] [LOCAL ] B: No existing data file found in primary storage. [2025-10-20T17:24:21.178869] [WARN ] [MAIN ] B: No existing data found, performing full refresh instead. [2025-10-20T17:24:21.179450] [INFO ] [MAIN ] B: Wiping existing primary data before full refresh. [2025-10-20T17:24:21.179688] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-10-20T17:24:21.179785] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-10-20T17:24:21.179911] [INFO ] [MAIN ] B: Starting full primary refresh. [2025-10-20T17:24:21.261346] [DEBUG] [LOCAL ] AZO: Loaded 11148 existing records from primary storage. [2025-10-20T17:24:21.279422] [INFO ] [MAIN ] AZO: Loaded 11148 existing records. [2025-10-20T17:24:21.485873] [DEBUG] [LOCAL ] AYI: Loaded 25175 existing records from primary storage. [2025-10-20T17:24:21.550966] [DEBUG] [LOCAL ] AZZ: Loaded 23075 existing records from primary storage. [2025-10-20T17:24:21.565828] [INFO ] [MAIN ] AYI: Loaded 25175 existing records. [2025-10-20T17:24:21.701082] [INFO ] [MAIN ] AZZ: Loaded 23075 existing records. [2025-10-20T17:24:21.787695] [WARN ] [MAIN ] AZO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:21.922569] [INFO ] [MAIN ] AZO: Wiping existing primary data before full refresh. [2025-10-20T17:24:21.998201] [INFO ] [LOCAL ] AZO: Starting deletion of primary data. [2025-10-20T17:24:22.083593] [DEBUG] [LOCAL ] AZO: Deleted primary data file. [2025-10-20T17:24:22.130941] [INFO ] [MAIN ] AZO: Starting full primary refresh. [2025-10-20T17:24:22.213431] [WARN ] [MAIN ] AYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:22.270768] [INFO ] [REMOTE ] B: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:22.281098] [INFO ] [MAIN ] AYI: Wiping existing primary data before full refresh. [2025-10-20T17:24:22.312961] [INFO ] [REMOTE ] B: Fetching primary data for 2025-01. [2025-10-20T17:24:22.323350] [INFO ] [LOCAL ] AYI: Starting deletion of primary data. [2025-10-20T17:24:22.346207] [DEBUG] [LOCAL ] AYI: Deleted primary data file. [2025-10-20T17:24:22.355860] [INFO ] [MAIN ] AYI: Starting full primary refresh. [2025-10-20T17:24:22.524521] [WARN ] [MAIN ] AZZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:22.540482] [INFO ] [MAIN ] AZZ: Wiping existing primary data before full refresh. [2025-10-20T17:24:22.540674] [INFO ] [LOCAL ] AZZ: Starting deletion of primary data. [2025-10-20T17:24:22.604692] [DEBUG] [LOCAL ] AZZ: Deleted primary data file. [2025-10-20T17:24:22.610126] [INFO ] [MAIN ] AZZ: Starting full primary refresh. [2025-10-20T17:24:23.432506] [INFO ] [REMOTE ] AZO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.465317] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-01. [2025-10-20T17:24:23.694114] [INFO ] [REMOTE ] AYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.690472] [INFO ] [REMOTE ] AZZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:23.694602] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-01. [2025-10-20T17:24:23.694831] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-01. [2025-10-20T17:24:23.728299] [INFO ] [REMOTE ] B: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:24:23.735206] [INFO ] [REMOTE ] B: Fetching primary data for 2025-02. [2025-10-20T17:24:23.943137] [DEBUG] [REMOTE ] B: Error fetching primary data for 2025-02: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:24:23.943561] [WARN ] [REMOTE ] B: No data returned for month 2025-02 [2025-10-20T17:24:23.943680] [FATAL] [REMOTE ] B: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:24:23.943777] [FATAL] [MAIN ] B: Failed to fetch primary data. [2025-10-20T17:24:23.943825] [WARN ] [MAIN ] B: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:24:23.943868] [INFO ] [LOCAL ] B: Starting deletion of primary data. [2025-10-20T17:24:23.943934] [DEBUG] [LOCAL ] B: No primary data file found to delete. [2025-10-20T17:24:23.943971] [INFO ] [LOCAL ] B: Starting deletion of derivative data. [2025-10-20T17:24:23.944011] [DEBUG] [LOCAL ] B: No derivative data file found to delete. [2025-10-20T17:24:23.944084] [INFO ] [MAIN ] Update Progress: 201/2367 symbols completed. B took 3.6s. Estimated remaining: 4h 22m 8s. Rate: 8.3 symbols/min [2025-10-20T17:24:24.130199] [INFO ] [MAIN ] BA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:24.190725] [DEBUG] [LOCAL ] BA: Loaded 54570 existing records from primary storage. [2025-10-20T17:24:24.191137] [INFO ] [MAIN ] BA: Loaded 54570 existing records. [2025-10-20T17:24:24.401091] [WARN ] [MAIN ] BA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:24.402382] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:24:24.407178] [INFO ] [MAIN ] BA: Wiping existing primary data before full refresh. [2025-10-20T17:24:24.407082] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-02. [2025-10-20T17:24:24.408535] [INFO ] [LOCAL ] BA: Starting deletion of primary data. [2025-10-20T17:24:24.408848] [DEBUG] [LOCAL ] BA: Deleted primary data file. [2025-10-20T17:24:24.408910] [INFO ] [MAIN ] BA: Starting full primary refresh. [2025-10-20T17:24:24.817599] [INFO ] [REMOTE ] BA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:24.818253] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-01. [2025-10-20T17:24:24.833990] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T17:24:24.838893] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-02. [2025-10-20T17:24:25.276271] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T17:24:25.279057] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-03. [2025-10-20T17:24:25.358735] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:24:25.366042] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-02. [2025-10-20T17:24:25.915835] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:24:25.923218] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-03. [2025-10-20T17:24:26.371807] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:24:26.376058] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-04. [2025-10-20T17:24:26.707613] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T17:24:26.714032] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-04. [2025-10-20T17:24:26.824790] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T17:24:26.835954] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-02. [2025-10-20T17:24:26.912807] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:24:26.918018] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-03. [2025-10-20T17:24:27.390842] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T17:24:27.451461] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-05. [2025-10-20T17:24:28.011500] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T17:24:28.146477] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:24:28.251555] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-05. [2025-10-20T17:24:28.401615] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:24:28.657473] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T17:24:29.051467] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-04. [2025-10-20T17:24:29.312103] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:24:29.851401] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-06. [2025-10-20T17:24:30.510198] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T17:24:30.539556] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:24:30.651595] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-05. [2025-10-20T17:24:31.451386] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-07. [2025-10-20T17:24:31.509047] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:24:32.251612] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-06. [2025-10-20T17:24:32.857293] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T17:24:33.051443] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-06. [2025-10-20T17:24:33.534269] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:24:33.851450] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-03. [2025-10-20T17:24:33.996884] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:24:34.651466] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-08. [2025-10-20T17:24:35.451503] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-07. [2025-10-20T17:24:35.809181] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:24:36.258698] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-09. [2025-10-20T17:24:36.269001] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-03 (2.42 seconds). [2025-10-20T17:24:36.416189] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T17:24:37.051440] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-04. [2025-10-20T17:24:37.398882] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:24:37.851396] [INFO ] [REMOTE ] AZZ: Fetching primary data for 2025-10. [2025-10-20T17:24:38.651453] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-07. [2025-10-20T17:24:39.241169] [INFO ] [REMOTE ] AZZ: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:24:39.246682] [INFO ] [REMOTE ] AZZ: Full fetch complete (32093 records). [2025-10-20T17:24:39.246855] [DEBUG] [VALIDATOR ] AZZ: Validating primary data. [2025-10-20T17:24:39.461046] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-08. [2025-10-20T17:24:39.706626] [DEBUG] [VALIDATOR ] AZZ: All expected trading days are present. [2025-10-20T17:24:39.706879] [DEBUG] [VALIDATOR ] AZZ: Primary data validation passed. [2025-10-20T17:24:39.707515] [INFO ] [MAIN ] AZZ: Fetched primary data is valid. [2025-10-20T17:24:39.707602] [DEBUG] [LOCAL ] AZZ: Saving primary data. [2025-10-20T17:24:39.709392] [DEBUG] [LOCAL ] AZZ: Trimmed 32093 records to 22882 (keeping 200 days of data) [2025-10-20T17:24:39.865020] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-04 (2.81 seconds). [2025-10-20T17:24:39.930467] [DEBUG] [LOCAL ] AZZ: Primary data saved (22882 records). [2025-10-20T17:24:39.930893] [INFO ] [MAIN ] AZZ: Full history saved. [2025-10-20T17:24:39.931118] [INFO ] [MAIN ] Update Progress: 202/2367 symbols completed. AZZ took 19.6s. Estimated remaining: 4h 23m 34s. Rate: 8.2 symbols/min [2025-10-20T17:24:40.028979] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:24:40.176216] [INFO ] [MAIN ] BABA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:40.189965] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T17:24:40.261102] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-08. [2025-10-20T17:24:40.267018] [DEBUG] [LOCAL ] BABA: Loaded 54600 existing records from primary storage. [2025-10-20T17:24:40.267591] [INFO ] [MAIN ] BABA: Loaded 54600 existing records. [2025-10-20T17:24:40.474000] [WARN ] [MAIN ] BABA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:40.474365] [INFO ] [MAIN ] BABA: Wiping existing primary data before full refresh. [2025-10-20T17:24:40.474418] [INFO ] [LOCAL ] BABA: Starting deletion of primary data. [2025-10-20T17:24:40.474893] [DEBUG] [LOCAL ] BABA: Deleted primary data file. [2025-10-20T17:24:40.474952] [INFO ] [MAIN ] BABA: Starting full primary refresh. [2025-10-20T17:24:40.898237] [INFO ] [REMOTE ] BABA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:41.051466] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-09. [2025-10-20T17:24:41.395509] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:24:41.851368] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-05. [2025-10-20T17:24:41.916999] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:24:42.651462] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-01. [2025-10-20T17:24:43.451449] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-09. [2025-10-20T17:24:43.989750] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T17:24:44.251427] [INFO ] [REMOTE ] AZO: Fetching primary data for 2025-10. [2025-10-20T17:24:44.526583] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:24:45.022076] [INFO ] [REMOTE ] AZO: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:24:45.024544] [INFO ] [REMOTE ] AZO: Full fetch complete (15936 records). [2025-10-20T17:24:45.024634] [DEBUG] [VALIDATOR ] AZO: Validating primary data. [2025-10-20T17:24:45.059259] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-06. [2025-10-20T17:24:45.466893] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-01 (2.82 seconds). [2025-10-20T17:24:45.513045] [DEBUG] [VALIDATOR ] AZO: All expected trading days are present. [2025-10-20T17:24:45.513127] [DEBUG] [VALIDATOR ] AZO: Primary data validation passed. [2025-10-20T17:24:45.513469] [INFO ] [MAIN ] AZO: Fetched primary data is valid. [2025-10-20T17:24:45.513510] [DEBUG] [LOCAL ] AZO: Saving primary data. [2025-10-20T17:24:45.514352] [DEBUG] [LOCAL ] AZO: Trimmed 15936 records to 10945 (keeping 200 days of data) [2025-10-20T17:24:45.592464] [DEBUG] [LOCAL ] AZO: Primary data saved (10945 records). [2025-10-20T17:24:45.592626] [INFO ] [MAIN ] AZO: Full history saved. [2025-10-20T17:24:45.592954] [INFO ] [MAIN ] Update Progress: 203/2367 symbols completed. AZO took 25.3s. Estimated remaining: 4h 23m 10s. Rate: 8.2 symbols/min [2025-10-20T17:24:45.796526] [INFO ] [MAIN ] BAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:45.849481] [DEBUG] [LOCAL ] BAC: Loaded 54588 existing records from primary storage. [2025-10-20T17:24:45.849933] [INFO ] [MAIN ] BAC: Loaded 54588 existing records. [2025-10-20T17:24:45.861634] [INFO ] [REMOTE ] AYI: Fetching primary data for 2025-10. [2025-10-20T17:24:46.066386] [WARN ] [MAIN ] BAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:46.066680] [INFO ] [MAIN ] BAC: Wiping existing primary data before full refresh. [2025-10-20T17:24:46.066929] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-10-20T17:24:46.067363] [DEBUG] [LOCAL ] BAC: Deleted primary data file. [2025-10-20T17:24:46.067407] [INFO ] [MAIN ] BAC: Starting full primary refresh. [2025-10-20T17:24:46.456836] [INFO ] [REMOTE ] BAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:46.651479] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-01. [2025-10-20T17:24:47.113215] [INFO ] [REMOTE ] AYI: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:24:47.118920] [INFO ] [REMOTE ] AYI: Full fetch complete (36997 records). [2025-10-20T17:24:47.119147] [DEBUG] [VALIDATOR ] AYI: Validating primary data. [2025-10-20T17:24:47.456704] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-02. [2025-10-20T17:24:47.635928] [DEBUG] [VALIDATOR ] AYI: All expected trading days are present. [2025-10-20T17:24:47.639830] [DEBUG] [VALIDATOR ] AYI: Primary data validation passed. [2025-10-20T17:24:47.640741] [INFO ] [MAIN ] AYI: Fetched primary data is valid. [2025-10-20T17:24:47.641075] [DEBUG] [LOCAL ] AYI: Saving primary data. [2025-10-20T17:24:47.645170] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-06 (2.59 seconds). [2025-10-20T17:24:47.657782] [DEBUG] [LOCAL ] AYI: Trimmed 36997 records to 24589 (keeping 200 days of data) [2025-10-20T17:24:47.832719] [DEBUG] [LOCAL ] AYI: Primary data saved (24589 records). [2025-10-20T17:24:47.832902] [INFO ] [MAIN ] AYI: Full history saved. [2025-10-20T17:24:47.832987] [INFO ] [MAIN ] Update Progress: 204/2367 symbols completed. AYI took 27.5s. Estimated remaining: 4h 22m 9s. Rate: 8.3 symbols/min [2025-10-20T17:24:48.027788] [INFO ] [MAIN ] BAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:24:48.096025] [DEBUG] [LOCAL ] BAH: Loaded 52467 existing records from primary storage. [2025-10-20T17:24:48.096399] [INFO ] [MAIN ] BAH: Loaded 52467 existing records. [2025-10-20T17:24:48.280735] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-07. [2025-10-20T17:24:48.287801] [WARN ] [MAIN ] BAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:24:48.289695] [INFO ] [MAIN ] BAH: Wiping existing primary data before full refresh. [2025-10-20T17:24:48.289760] [INFO ] [LOCAL ] BAH: Starting deletion of primary data. [2025-10-20T17:24:48.290008] [DEBUG] [LOCAL ] BAH: Deleted primary data file. [2025-10-20T17:24:48.290065] [INFO ] [MAIN ] BAH: Starting full primary refresh. [2025-10-20T17:24:48.734159] [INFO ] [REMOTE ] BAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:24:48.783709] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-01 (2.13 seconds). [2025-10-20T17:24:49.051425] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-02. [2025-10-20T17:24:49.851472] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-01. [2025-10-20T17:24:50.102124] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-02 (2.65 seconds). [2025-10-20T17:24:50.544431] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:24:50.651372] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-03. [2025-10-20T17:24:50.890486] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:24:51.120666] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:24:51.451427] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-03. [2025-10-20T17:24:52.251515] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-02. [2025-10-20T17:24:53.051368] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-08. [2025-10-20T17:24:53.465971] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-03 (2.81 seconds). [2025-10-20T17:24:53.856233] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-04. [2025-10-20T17:24:53.863475] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-10-20T17:24:54.027088] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T17:24:54.651462] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-04. [2025-10-20T17:24:55.458014] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-03. [2025-10-20T17:24:55.484592] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-08 (2.43 seconds). [2025-10-20T17:24:56.251425] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-09. [2025-10-20T17:24:56.752587] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-04 (2.90 seconds). [2025-10-20T17:24:57.051385] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-05. [2025-10-20T17:24:57.238816] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:24:57.290470] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-04 (2.64 seconds). [2025-10-20T17:24:57.851404] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-04. [2025-10-20T17:24:58.467184] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:24:58.651406] [INFO ] [REMOTE ] BA: Fetching primary data for 2025-10. [2025-10-20T17:24:59.190509] [INFO ] [REMOTE ] BA: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T17:24:59.199792] [INFO ] [REMOTE ] BA: Full fetch complete (77969 records). [2025-10-20T17:24:59.200016] [DEBUG] [VALIDATOR ] BA: Validating primary data. [2025-10-20T17:24:59.466405] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-05. [2025-10-20T17:24:59.679418] [DEBUG] [VALIDATOR ] BA: All expected trading days are present. [2025-10-20T17:24:59.679764] [DEBUG] [VALIDATOR ] BA: Primary data validation passed. [2025-10-20T17:24:59.680960] [INFO ] [MAIN ] BA: Fetched primary data is valid. [2025-10-20T17:24:59.681268] [DEBUG] [LOCAL ] BA: Saving primary data. [2025-10-20T17:24:59.683888] [DEBUG] [LOCAL ] BA: Trimmed 77969 records to 53790 (keeping 200 days of data) [2025-10-20T17:24:59.820087] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-05 (2.77 seconds). [2025-10-20T17:24:59.997191] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T17:25:00.149998] [DEBUG] [LOCAL ] BA: Primary data saved (53790 records). [2025-10-20T17:25:00.150414] [INFO ] [MAIN ] BA: Full history saved. [2025-10-20T17:25:00.150512] [INFO ] [MAIN ] Update Progress: 205/2367 symbols completed. BA took 36.2s. Estimated remaining: 4h 22m 55s. Rate: 8.2 symbols/min [2025-10-20T17:25:00.256773] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-06. [2025-10-20T17:25:00.345408] [INFO ] [MAIN ] BAK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:00.393276] [DEBUG] [LOCAL ] BAK: Loaded 39871 existing records from primary storage. [2025-10-20T17:25:00.393706] [INFO ] [MAIN ] BAK: Loaded 39871 existing records. [2025-10-20T17:25:00.599772] [WARN ] [MAIN ] BAK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:00.600107] [INFO ] [MAIN ] BAK: Wiping existing primary data before full refresh. [2025-10-20T17:25:00.600158] [INFO ] [LOCAL ] BAK: Starting deletion of primary data. [2025-10-20T17:25:00.600497] [DEBUG] [LOCAL ] BAK: Deleted primary data file. [2025-10-20T17:25:00.600561] [INFO ] [MAIN ] BAK: Starting full primary refresh. [2025-10-20T17:25:01.052958] [INFO ] [REMOTE ] BAK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:01.053041] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-05. [2025-10-20T17:25:01.851492] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-01. [2025-10-20T17:25:02.167718] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-05 (2.70 seconds). [2025-10-20T17:25:02.651488] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-06. [2025-10-20T17:25:02.992065] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:25:03.169089] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-06 (2.91 seconds). [2025-10-20T17:25:03.447079] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:25:03.456767] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-07. [2025-10-20T17:25:04.251425] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-02. [2025-10-20T17:25:05.051407] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-06. [2025-10-20T17:25:05.145958] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-06 (2.49 seconds). [2025-10-20T17:25:05.774552] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:25:05.851394] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-07. [2025-10-20T17:25:05.965367] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T17:25:06.507166] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:25:06.651408] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-03. [2025-10-20T17:25:07.451397] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-08. [2025-10-20T17:25:08.070779] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:25:08.251387] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-04. [2025-10-20T17:25:08.476135] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-07 (2.62 seconds). [2025-10-20T17:25:09.051388] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-08. [2025-10-20T17:25:09.369071] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:25:09.851584] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-05. [2025-10-20T17:25:10.248588] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-08 (2.80 seconds). [2025-10-20T17:25:10.651463] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-07. [2025-10-20T17:25:10.980820] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:25:11.451380] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-09. [2025-10-20T17:25:11.499903] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-10-20T17:25:12.251403] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-09. [2025-10-20T17:25:12.617133] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T17:25:13.051395] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-08. [2025-10-20T17:25:13.851474] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-06. [2025-10-20T17:25:13.957243] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-09 (2.51 seconds). [2025-10-20T17:25:14.589334] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-09 (2.34 seconds). [2025-10-20T17:25:14.651461] [INFO ] [REMOTE ] BABA: Fetching primary data for 2025-10. [2025-10-20T17:25:14.710319] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:25:15.207409] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T17:25:15.451363] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-07. [2025-10-20T17:25:16.251410] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-09. [2025-10-20T17:25:16.947004] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:25:17.051412] [INFO ] [REMOTE ] BAC: Fetching primary data for 2025-10. [2025-10-20T17:25:17.455494] [INFO ] [REMOTE ] BABA: Incremental fetch complete for 2025-10 (2.80 seconds). [2025-10-20T17:25:17.464417] [INFO ] [REMOTE ] BABA: Full fetch complete (78000 records). [2025-10-20T17:25:17.464668] [DEBUG] [VALIDATOR ] BABA: Validating primary data. [2025-10-20T17:25:17.852759] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-08. [2025-10-20T17:25:17.931342] [DEBUG] [VALIDATOR ] BABA: All expected trading days are present. [2025-10-20T17:25:17.931518] [DEBUG] [VALIDATOR ] BABA: Primary data validation passed. [2025-10-20T17:25:17.932621] [INFO ] [MAIN ] BABA: Fetched primary data is valid. [2025-10-20T17:25:17.932731] [DEBUG] [LOCAL ] BABA: Saving primary data. [2025-10-20T17:25:17.965387] [DEBUG] [LOCAL ] BABA: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T17:25:17.968006] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:25:18.325924] [DEBUG] [LOCAL ] BABA: Primary data saved (53820 records). [2025-10-20T17:25:18.326083] [INFO ] [MAIN ] BABA: Full history saved. [2025-10-20T17:25:18.326212] [INFO ] [MAIN ] Update Progress: 206/2367 symbols completed. BABA took 38.4s. Estimated remaining: 4h 24m 41s. Rate: 8.2 symbols/min [2025-10-20T17:25:18.496988] [INFO ] [MAIN ] BALL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:18.549357] [DEBUG] [LOCAL ] BALL: Loaded 52656 existing records from primary storage. [2025-10-20T17:25:18.549729] [INFO ] [MAIN ] BALL: Loaded 52656 existing records. [2025-10-20T17:25:18.652106] [INFO ] [REMOTE ] BAH: Fetching primary data for 2025-10. [2025-10-20T17:25:18.732605] [WARN ] [MAIN ] BALL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:18.732890] [INFO ] [MAIN ] BALL: Wiping existing primary data before full refresh. [2025-10-20T17:25:18.732939] [INFO ] [LOCAL ] BALL: Starting deletion of primary data. [2025-10-20T17:25:18.733206] [DEBUG] [LOCAL ] BALL: Deleted primary data file. [2025-10-20T17:25:18.733264] [INFO ] [MAIN ] BALL: Starting full primary refresh. [2025-10-20T17:25:19.081634] [INFO ] [REMOTE ] BALL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:19.371241] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:25:19.451529] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-09. [2025-10-20T17:25:19.706482] [INFO ] [REMOTE ] BAC: Incremental fetch complete for 2025-10 (2.66 seconds). [2025-10-20T17:25:19.714779] [INFO ] [REMOTE ] BAC: Full fetch complete (77598 records). [2025-10-20T17:25:19.714967] [DEBUG] [VALIDATOR ] BAC: Validating primary data. [2025-10-20T17:25:19.716028] [FATAL] [VALIDATOR ] BAC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:25:19.716126] [FATAL] [MAIN ] BAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:25:19.716172] [WARN ] [MAIN ] BAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:25:19.716219] [INFO ] [LOCAL ] BAC: Starting deletion of primary data. [2025-10-20T17:25:19.716287] [DEBUG] [LOCAL ] BAC: No primary data file found to delete. [2025-10-20T17:25:19.716326] [INFO ] [LOCAL ] BAC: Starting deletion of derivative data. [2025-10-20T17:25:19.716514] [DEBUG] [LOCAL ] BAC: Deleted derivative data file. [2025-10-20T17:25:19.716907] [INFO ] [MAIN ] Update Progress: 207/2367 symbols completed. BAC took 34.1s. Estimated remaining: 4h 23m 32s. Rate: 8.2 symbols/min [2025-10-20T17:25:19.928002] [INFO ] [MAIN ] BALY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:19.928231] [DEBUG] [LOCAL ] BALY: No existing data file found in primary storage. [2025-10-20T17:25:19.928274] [WARN ] [MAIN ] BALY: No existing data found, performing full refresh instead. [2025-10-20T17:25:19.928477] [INFO ] [MAIN ] BALY: Wiping existing primary data before full refresh. [2025-10-20T17:25:19.928527] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-10-20T17:25:19.928572] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-10-20T17:25:19.928606] [INFO ] [MAIN ] BALY: Starting full primary refresh. [2025-10-20T17:25:20.252559] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-01. [2025-10-20T17:25:20.336459] [INFO ] [REMOTE ] BALY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:20.654252] [INFO ] [REMOTE ] BAH: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T17:25:20.663500] [INFO ] [REMOTE ] BAH: Full fetch complete (75242 records). [2025-10-20T17:25:20.663690] [DEBUG] [VALIDATOR ] BAH: Validating primary data. [2025-10-20T17:25:21.057459] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-01. [2025-10-20T17:25:21.099431] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:25:21.127909] [DEBUG] [VALIDATOR ] BAH: All expected trading days are present. [2025-10-20T17:25:21.128035] [DEBUG] [VALIDATOR ] BAH: Primary data validation passed. [2025-10-20T17:25:21.129235] [INFO ] [MAIN ] BAH: Fetched primary data is valid. [2025-10-20T17:25:21.129975] [DEBUG] [LOCAL ] BAH: Saving primary data. [2025-10-20T17:25:21.131747] [DEBUG] [LOCAL ] BAH: Trimmed 75242 records to 51683 (keeping 200 days of data) [2025-10-20T17:25:21.459307] [DEBUG] [LOCAL ] BAH: Primary data saved (51683 records). [2025-10-20T17:25:21.459454] [INFO ] [MAIN ] BAH: Full history saved. [2025-10-20T17:25:21.459541] [INFO ] [MAIN ] Update Progress: 208/2367 symbols completed. BAH took 33.6s. Estimated remaining: 4h 22m 27s. Rate: 8.2 symbols/min [2025-10-20T17:25:21.630530] [INFO ] [MAIN ] BAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:21.679163] [DEBUG] [LOCAL ] BAM: Loaded 52877 existing records from primary storage. [2025-10-20T17:25:21.679489] [INFO ] [MAIN ] BAM: Loaded 52877 existing records. [2025-10-20T17:25:21.859821] [INFO ] [REMOTE ] BAK: Fetching primary data for 2025-10. [2025-10-20T17:25:21.865717] [WARN ] [MAIN ] BAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:21.867174] [INFO ] [MAIN ] BAM: Wiping existing primary data before full refresh. [2025-10-20T17:25:21.867369] [INFO ] [LOCAL ] BAM: Starting deletion of primary data. [2025-10-20T17:25:21.868063] [DEBUG] [LOCAL ] BAM: Deleted primary data file. [2025-10-20T17:25:21.868198] [INFO ] [MAIN ] BAM: Starting full primary refresh. [2025-10-20T17:25:22.059925] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:25:22.284500] [INFO ] [REMOTE ] BAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:22.558038] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:25:22.651370] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-02. [2025-10-20T17:25:23.454996] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-01. [2025-10-20T17:25:23.459070] [INFO ] [REMOTE ] BAK: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T17:25:23.467155] [INFO ] [REMOTE ] BAK: Full fetch complete (56869 records). [2025-10-20T17:25:23.467305] [DEBUG] [VALIDATOR ] BAK: Validating primary data. [2025-10-20T17:25:23.928768] [DEBUG] [VALIDATOR ] BAK: All expected trading days are present. [2025-10-20T17:25:23.928894] [DEBUG] [VALIDATOR ] BAK: Primary data validation passed. [2025-10-20T17:25:23.929552] [INFO ] [MAIN ] BAK: Fetched primary data is valid. [2025-10-20T17:25:23.929623] [DEBUG] [LOCAL ] BAK: Saving primary data. [2025-10-20T17:25:23.931008] [DEBUG] [LOCAL ] BAK: Trimmed 56869 records to 39438 (keeping 200 days of data) [2025-10-20T17:25:24.163130] [DEBUG] [LOCAL ] BAK: Primary data saved (39438 records). [2025-10-20T17:25:24.163399] [INFO ] [MAIN ] BAK: Full history saved. [2025-10-20T17:25:24.163571] [INFO ] [MAIN ] Update Progress: 209/2367 symbols completed. BAK took 24.0s. Estimated remaining: 4h 21m 32s. Rate: 8.3 symbols/min [2025-10-20T17:25:24.262122] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-02. [2025-10-20T17:25:24.305887] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:25:24.391306] [INFO ] [MAIN ] BANC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:24.391507] [DEBUG] [LOCAL ] BANC: No existing data file found in primary storage. [2025-10-20T17:25:24.391545] [WARN ] [MAIN ] BANC: No existing data found, performing full refresh instead. [2025-10-20T17:25:24.391682] [INFO ] [MAIN ] BANC: Wiping existing primary data before full refresh. [2025-10-20T17:25:24.391735] [INFO ] [LOCAL ] BANC: Starting deletion of primary data. [2025-10-20T17:25:24.391777] [DEBUG] [LOCAL ] BANC: No primary data file found to delete. [2025-10-20T17:25:24.391809] [INFO ] [MAIN ] BANC: Starting full primary refresh. [2025-10-20T17:25:24.760103] [INFO ] [REMOTE ] BANC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:25.051356] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-03. [2025-10-20T17:25:25.109951] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T17:25:25.142263] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:25:25.851382] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-02. [2025-10-20T17:25:26.651362] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-03. [2025-10-20T17:25:26.803970] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:25:27.386303] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:25:27.455419] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-01. [2025-10-20T17:25:27.461552] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:25:28.251371] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-04. [2025-10-20T17:25:28.861315] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:25:29.051396] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-02. [2025-10-20T17:25:29.851360] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-03. [2025-10-20T17:25:30.014944] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T17:25:30.605106] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:25:30.651380] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-03. [2025-10-20T17:25:31.115709] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:25:31.451423] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-04. [2025-10-20T17:25:32.251399] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-04. [2025-10-20T17:25:32.454935] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:25:32.851280] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:25:32.988219] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:25:33.051367] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-04. [2025-10-20T17:25:33.851390] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-05. [2025-10-20T17:25:34.656485] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-05. [2025-10-20T17:25:34.665878] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:25:35.346399] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:25:35.451460] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-06. [2025-10-20T17:25:35.633881] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T17:25:36.251468] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-06. [2025-10-20T17:25:37.051438] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-05. [2025-10-20T17:25:37.096682] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:25:37.368114] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:25:37.851526] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-05. [2025-10-20T17:25:38.651468] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-07. [2025-10-20T17:25:38.817856] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:25:39.431837] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:25:39.451340] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-08. [2025-10-20T17:25:39.646147] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:25:40.106990] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T17:25:40.251400] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-06. [2025-10-20T17:25:41.051449] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-06. [2025-10-20T17:25:41.829855] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T17:25:41.851393] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-07. [2025-10-20T17:25:42.651411] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-09. [2025-10-20T17:25:42.739635] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:25:43.341137] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T17:25:43.451370] [INFO ] [REMOTE ] BALY: Fetching primary data for 2025-10. [2025-10-20T17:25:43.622351] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T17:25:44.083287] [INFO ] [REMOTE ] BALY: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T17:25:44.085768] [INFO ] [REMOTE ] BALY: Full fetch complete (16198 records). [2025-10-20T17:25:44.085843] [DEBUG] [VALIDATOR ] BALY: Validating primary data. [2025-10-20T17:25:44.258915] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-08. [2025-10-20T17:25:44.302553] [WARN ] [VALIDATOR ] BALY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T17:25:44.302975] [FATAL] [MAIN ] BALY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:25:44.304003] [WARN ] [MAIN ] BALY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:25:44.304083] [INFO ] [LOCAL ] BALY: Starting deletion of primary data. [2025-10-20T17:25:44.304147] [DEBUG] [LOCAL ] BALY: No primary data file found to delete. [2025-10-20T17:25:44.304185] [INFO ] [LOCAL ] BALY: Starting deletion of derivative data. [2025-10-20T17:25:44.304226] [DEBUG] [LOCAL ] BALY: No derivative data file found to delete. [2025-10-20T17:25:44.304290] [INFO ] [MAIN ] Update Progress: 210/2367 symbols completed. BALY took 24.6s. Estimated remaining: 4h 23m 37s. Rate: 8.2 symbols/min [2025-10-20T17:25:44.515270] [INFO ] [MAIN ] BAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:44.545381] [DEBUG] [LOCAL ] BAP: Loaded 26192 existing records from primary storage. [2025-10-20T17:25:44.545756] [INFO ] [MAIN ] BAP: Loaded 26192 existing records. [2025-10-20T17:25:44.771901] [WARN ] [MAIN ] BAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:44.772172] [INFO ] [MAIN ] BAP: Wiping existing primary data before full refresh. [2025-10-20T17:25:44.772222] [INFO ] [LOCAL ] BAP: Starting deletion of primary data. [2025-10-20T17:25:44.772626] [DEBUG] [LOCAL ] BAP: Deleted primary data file. [2025-10-20T17:25:44.772726] [INFO ] [MAIN ] BAP: Starting full primary refresh. [2025-10-20T17:25:45.061373] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-07. [2025-10-20T17:25:45.220980] [INFO ] [REMOTE ] BAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:45.851417] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-07. [2025-10-20T17:25:45.970570] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:25:46.651444] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-01. [2025-10-20T17:25:47.121749] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T17:25:47.451392] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-08. [2025-10-20T17:25:47.539083] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:25:47.705629] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:25:48.251438] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-02. [2025-10-20T17:25:48.925928] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T17:25:49.051515] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-08. [2025-10-20T17:25:49.233195] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:25:49.851413] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-09. [2025-10-20T17:25:50.651389] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-03. [2025-10-20T17:25:50.787134] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:25:51.451430] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-09. [2025-10-20T17:25:51.588737] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:25:51.727966] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:25:52.251461] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-09. [2025-10-20T17:25:53.051354] [INFO ] [REMOTE ] BANC: Fetching primary data for 2025-10. [2025-10-20T17:25:53.151169] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:25:53.851385] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-04. [2025-10-20T17:25:53.940318] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:25:54.651630] [INFO ] [REMOTE ] BAM: Fetching primary data for 2025-10. [2025-10-20T17:25:54.774638] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:25:54.877671] [INFO ] [REMOTE ] BANC: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:25:54.885975] [INFO ] [REMOTE ] BANC: Full fetch complete (74580 records). [2025-10-20T17:25:54.886134] [DEBUG] [VALIDATOR ] BANC: Validating primary data. [2025-10-20T17:25:55.321690] [DEBUG] [VALIDATOR ] BANC: All expected trading days are present. [2025-10-20T17:25:55.321833] [DEBUG] [VALIDATOR ] BANC: Primary data validation passed. [2025-10-20T17:25:55.322637] [INFO ] [MAIN ] BANC: Fetched primary data is valid. [2025-10-20T17:25:55.322727] [DEBUG] [LOCAL ] BANC: Saving primary data. [2025-10-20T17:25:55.324306] [DEBUG] [LOCAL ] BANC: Trimmed 74580 records to 51387 (keeping 200 days of data) [2025-10-20T17:25:55.504926] [INFO ] [REMOTE ] BALL: Fetching primary data for 2025-10. [2025-10-20T17:25:55.683710] [DEBUG] [LOCAL ] BANC: Primary data saved (51387 records). [2025-10-20T17:25:55.683886] [INFO ] [MAIN ] BANC: Full history saved. [2025-10-20T17:25:55.683967] [INFO ] [MAIN ] Update Progress: 211/2367 symbols completed. BANC took 31.5s. Estimated remaining: 4h 24m 11s. Rate: 8.2 symbols/min [2025-10-20T17:25:55.913947] [INFO ] [MAIN ] BARK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:55.961525] [DEBUG] [LOCAL ] BARK: Loaded 38766 existing records from primary storage. [2025-10-20T17:25:55.961990] [INFO ] [MAIN ] BARK: Loaded 38766 existing records. [2025-10-20T17:25:56.234141] [WARN ] [MAIN ] BARK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:56.234551] [INFO ] [MAIN ] BARK: Wiping existing primary data before full refresh. [2025-10-20T17:25:56.234694] [INFO ] [LOCAL ] BARK: Starting deletion of primary data. [2025-10-20T17:25:56.235057] [DEBUG] [LOCAL ] BARK: Deleted primary data file. [2025-10-20T17:25:56.235125] [INFO ] [MAIN ] BARK: Starting full primary refresh. [2025-10-20T17:25:56.260353] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-05. [2025-10-20T17:25:56.777761] [INFO ] [REMOTE ] BAM: Incremental fetch complete for 2025-10 (2.13 seconds). [2025-10-20T17:25:56.788882] [INFO ] [REMOTE ] BARK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:56.797140] [INFO ] [REMOTE ] BAM: Full fetch complete (75138 records). [2025-10-20T17:25:56.797330] [DEBUG] [VALIDATOR ] BAM: Validating primary data. [2025-10-20T17:25:57.067021] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-01. [2025-10-20T17:25:57.234960] [DEBUG] [VALIDATOR ] BAM: All expected trading days are present. [2025-10-20T17:25:57.235082] [DEBUG] [VALIDATOR ] BAM: Primary data validation passed. [2025-10-20T17:25:57.235966] [INFO ] [MAIN ] BAM: Fetched primary data is valid. [2025-10-20T17:25:57.236172] [DEBUG] [LOCAL ] BAM: Saving primary data. [2025-10-20T17:25:57.237850] [DEBUG] [LOCAL ] BAM: Trimmed 75138 records to 52103 (keeping 200 days of data) [2025-10-20T17:25:57.443029] [INFO ] [REMOTE ] BALL: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:25:57.507032] [INFO ] [REMOTE ] BALL: Full fetch complete (75632 records). [2025-10-20T17:25:57.515256] [DEBUG] [VALIDATOR ] BALL: Validating primary data. [2025-10-20T17:25:57.630695] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:25:57.797511] [DEBUG] [LOCAL ] BAM: Primary data saved (52103 records). [2025-10-20T17:25:57.812408] [INFO ] [MAIN ] BAM: Full history saved. [2025-10-20T17:25:57.812862] [INFO ] [MAIN ] Update Progress: 212/2367 symbols completed. BAM took 36.4s. Estimated remaining: 4h 23m 11s. Rate: 8.2 symbols/min [2025-10-20T17:25:57.863823] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-06. [2025-10-20T17:25:58.152745] [INFO ] [MAIN ] BAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:58.237810] [DEBUG] [VALIDATOR ] BALL: All expected trading days are present. [2025-10-20T17:25:58.237969] [DEBUG] [VALIDATOR ] BALL: Primary data validation passed. [2025-10-20T17:25:58.238999] [INFO ] [MAIN ] BALL: Fetched primary data is valid. [2025-10-20T17:25:58.239358] [DEBUG] [LOCAL ] BALL: Saving primary data. [2025-10-20T17:25:58.241720] [DEBUG] [LOCAL ] BALL: Trimmed 75632 records to 51927 (keeping 200 days of data) [2025-10-20T17:25:58.388980] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T17:25:58.591635] [DEBUG] [LOCAL ] BAX: Loaded 54309 existing records from primary storage. [2025-10-20T17:25:58.628271] [INFO ] [MAIN ] BAX: Loaded 54309 existing records. [2025-10-20T17:25:58.665811] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-02. [2025-10-20T17:25:58.726339] [DEBUG] [LOCAL ] BALL: Primary data saved (51927 records). [2025-10-20T17:25:58.760235] [INFO ] [MAIN ] BALL: Full history saved. [2025-10-20T17:25:58.760375] [INFO ] [MAIN ] Update Progress: 213/2367 symbols completed. BALL took 40.4s. Estimated remaining: 4h 21m 59s. Rate: 8.2 symbols/min [2025-10-20T17:25:59.115480] [INFO ] [MAIN ] BB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:25:59.094753] [WARN ] [MAIN ] BAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:59.116026] [INFO ] [MAIN ] BAX: Wiping existing primary data before full refresh. [2025-10-20T17:25:59.116236] [INFO ] [LOCAL ] BAX: Starting deletion of primary data. [2025-10-20T17:25:59.117340] [DEBUG] [LOCAL ] BAX: Deleted primary data file. [2025-10-20T17:25:59.117412] [INFO ] [MAIN ] BAX: Starting full primary refresh. [2025-10-20T17:25:59.400338] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:25:59.458257] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-07. [2025-10-20T17:25:59.484010] [DEBUG] [LOCAL ] BB: Loaded 53673 existing records from primary storage. [2025-10-20T17:25:59.486350] [INFO ] [MAIN ] BB: Loaded 53673 existing records. [2025-10-20T17:25:59.670877] [INFO ] [REMOTE ] BAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:25:59.764161] [WARN ] [MAIN ] BB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:25:59.764677] [INFO ] [MAIN ] BB: Wiping existing primary data before full refresh. [2025-10-20T17:25:59.764746] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-10-20T17:25:59.765020] [DEBUG] [LOCAL ] BB: Deleted primary data file. [2025-10-20T17:25:59.765078] [INFO ] [MAIN ] BB: Starting full primary refresh. [2025-10-20T17:26:00.248814] [INFO ] [REMOTE ] BB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:00.251166] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:26:00.258414] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-01. [2025-10-20T17:26:00.741431] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:26:01.051396] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-01. [2025-10-20T17:26:01.851397] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-08. [2025-10-20T17:26:01.998846] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:26:02.656509] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-03. [2025-10-20T17:26:02.675924] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:26:02.982696] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:26:03.451376] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-02. [2025-10-20T17:26:03.745503] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:26:04.251382] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-04. [2025-10-20T17:26:05.051481] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-02. [2025-10-20T17:26:05.580264] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T17:26:05.718324] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:26:05.851403] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-03. [2025-10-20T17:26:06.651356] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-09. [2025-10-20T17:26:06.754942] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:26:07.451365] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-05. [2025-10-20T17:26:07.861986] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:26:07.878831] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:26:08.251377] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-03. [2025-10-20T17:26:08.854477] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:26:09.051434] [INFO ] [REMOTE ] BAP: Fetching primary data for 2025-10. [2025-10-20T17:26:09.851363] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-04. [2025-10-20T17:26:10.127577] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:26:10.313027] [INFO ] [REMOTE ] BAP: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:26:10.317266] [INFO ] [REMOTE ] BAP: Full fetch complete (37382 records). [2025-10-20T17:26:10.317375] [DEBUG] [VALIDATOR ] BAP: Validating primary data. [2025-10-20T17:26:10.656487] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-06. [2025-10-20T17:26:10.733281] [DEBUG] [VALIDATOR ] BAP: All expected trading days are present. [2025-10-20T17:26:10.733362] [DEBUG] [VALIDATOR ] BAP: Primary data validation passed. [2025-10-20T17:26:10.733868] [INFO ] [MAIN ] BAP: Fetched primary data is valid. [2025-10-20T17:26:10.733929] [DEBUG] [LOCAL ] BAP: Saving primary data. [2025-10-20T17:26:10.734825] [DEBUG] [LOCAL ] BAP: Trimmed 37382 records to 25879 (keeping 200 days of data) [2025-10-20T17:26:10.904924] [DEBUG] [LOCAL ] BAP: Primary data saved (25879 records). [2025-10-20T17:26:10.905066] [INFO ] [MAIN ] BAP: Full history saved. [2025-10-20T17:26:10.905144] [INFO ] [MAIN ] Update Progress: 214/2367 symbols completed. BAP took 26.6s. Estimated remaining: 4h 22m 40s. Rate: 8.2 symbols/min [2025-10-20T17:26:11.081136] [INFO ] [MAIN ] BBAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:11.136323] [DEBUG] [LOCAL ] BBAI: Loaded 54600 existing records from primary storage. [2025-10-20T17:26:11.136895] [INFO ] [MAIN ] BBAI: Loaded 54600 existing records. [2025-10-20T17:26:11.325903] [WARN ] [MAIN ] BBAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:11.326123] [INFO ] [MAIN ] BBAI: Wiping existing primary data before full refresh. [2025-10-20T17:26:11.326181] [INFO ] [LOCAL ] BBAI: Starting deletion of primary data. [2025-10-20T17:26:11.326424] [DEBUG] [LOCAL ] BBAI: Deleted primary data file. [2025-10-20T17:26:11.326480] [INFO ] [MAIN ] BBAI: Starting full primary refresh. [2025-10-20T17:26:11.466456] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-04. [2025-10-20T17:26:11.690065] [INFO ] [REMOTE ] BBAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:11.720037] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T17:26:12.026621] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:26:12.251404] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-01. [2025-10-20T17:26:13.051433] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-05. [2025-10-20T17:26:13.416489] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:26:13.851478] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-07. [2025-10-20T17:26:14.651439] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-05. [2025-10-20T17:26:14.892216] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-01 (2.64 seconds). [2025-10-20T17:26:14.949795] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:26:15.292918] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T17:26:15.451384] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-08. [2025-10-20T17:26:16.217425] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:26:16.251368] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-06. [2025-10-20T17:26:16.516971] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:26:17.051420] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-06. [2025-10-20T17:26:17.851398] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-09. [2025-10-20T17:26:18.280009] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:26:18.651350] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-07. [2025-10-20T17:26:18.719092] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:26:19.319807] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:26:19.451387] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-02. [2025-10-20T17:26:20.251386] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-07. [2025-10-20T17:26:20.506134] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:26:21.051366] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-08. [2025-10-20T17:26:21.851357] [INFO ] [REMOTE ] BARK: Fetching primary data for 2025-10. [2025-10-20T17:26:22.050850] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-02 (2.60 seconds). [2025-10-20T17:26:22.143497] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:26:22.629037] [INFO ] [REMOTE ] BB: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:26:22.651350] [INFO ] [REMOTE ] BB: Fetching primary data for 2025-09. [2025-10-20T17:26:22.752391] [DEBUG] [REMOTE ] BB: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-10-20T17:26:22.752718] [WARN ] [REMOTE ] BB: No data returned for month 2025-09 [2025-10-20T17:26:22.752804] [FATAL] [REMOTE ] BB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:26:22.752882] [FATAL] [MAIN ] BB: Failed to fetch primary data. [2025-10-20T17:26:22.752919] [WARN ] [MAIN ] BB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:26:22.752959] [INFO ] [LOCAL ] BB: Starting deletion of primary data. [2025-10-20T17:26:22.753014] [DEBUG] [LOCAL ] BB: No primary data file found to delete. [2025-10-20T17:26:22.753048] [INFO ] [LOCAL ] BB: Starting deletion of derivative data. [2025-10-20T17:26:22.753176] [DEBUG] [LOCAL ] BB: Deleted derivative data file. [2025-10-20T17:26:22.753239] [INFO ] [MAIN ] Update Progress: 215/2367 symbols completed. BB took 24.0s. Estimated remaining: 4h 23m 18s. Rate: 8.2 symbols/min [2025-10-20T17:26:22.969419] [INFO ] [MAIN ] BBAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:23.013070] [DEBUG] [LOCAL ] BBAR: Loaded 42985 existing records from primary storage. [2025-10-20T17:26:23.013460] [INFO ] [MAIN ] BBAR: Loaded 42985 existing records. [2025-10-20T17:26:23.233140] [WARN ] [MAIN ] BBAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:23.233563] [INFO ] [MAIN ] BBAR: Wiping existing primary data before full refresh. [2025-10-20T17:26:23.233636] [INFO ] [LOCAL ] BBAR: Starting deletion of primary data. [2025-10-20T17:26:23.234001] [DEBUG] [LOCAL ] BBAR: Deleted primary data file. [2025-10-20T17:26:23.234065] [INFO ] [MAIN ] BBAR: Starting full primary refresh. [2025-10-20T17:26:23.466768] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-03. [2025-10-20T17:26:23.514948] [INFO ] [REMOTE ] BARK: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T17:26:23.589400] [INFO ] [REMOTE ] BARK: Full fetch complete (56339 records). [2025-10-20T17:26:23.603520] [DEBUG] [VALIDATOR ] BARK: Validating primary data. [2025-10-20T17:26:23.783893] [INFO ] [REMOTE ] BBAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:24.169203] [DEBUG] [VALIDATOR ] BARK: All expected trading days are present. [2025-10-20T17:26:24.169302] [DEBUG] [VALIDATOR ] BARK: Primary data validation passed. [2025-10-20T17:26:24.170030] [INFO ] [MAIN ] BARK: Fetched primary data is valid. [2025-10-20T17:26:24.170098] [DEBUG] [LOCAL ] BARK: Saving primary data. [2025-10-20T17:26:24.171405] [DEBUG] [LOCAL ] BARK: Trimmed 56339 records to 38200 (keeping 200 days of data) [2025-10-20T17:26:24.252728] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-08. [2025-10-20T17:26:24.395612] [DEBUG] [LOCAL ] BARK: Primary data saved (38200 records). [2025-10-20T17:26:24.395809] [INFO ] [MAIN ] BARK: Full history saved. [2025-10-20T17:26:24.395917] [INFO ] [MAIN ] Update Progress: 216/2367 symbols completed. BARK took 28.7s. Estimated remaining: 4h 22m 14s. Rate: 8.2 symbols/min [2025-10-20T17:26:24.574895] [INFO ] [MAIN ] BBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:24.632126] [DEBUG] [LOCAL ] BBD: Loaded 54039 existing records from primary storage. [2025-10-20T17:26:24.632484] [INFO ] [MAIN ] BBD: Loaded 54039 existing records. [2025-10-20T17:26:24.815901] [WARN ] [MAIN ] BBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:24.816154] [INFO ] [MAIN ] BBD: Wiping existing primary data before full refresh. [2025-10-20T17:26:24.816203] [INFO ] [LOCAL ] BBD: Starting deletion of primary data. [2025-10-20T17:26:24.816400] [DEBUG] [LOCAL ] BBD: Deleted primary data file. [2025-10-20T17:26:24.816437] [INFO ] [MAIN ] BBD: Starting full primary refresh. [2025-10-20T17:26:25.074346] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-01. [2025-10-20T17:26:25.199934] [INFO ] [REMOTE ] BBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:25.851442] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-01. [2025-10-20T17:26:26.166179] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:26:26.394882] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-03 (2.93 seconds). [2025-10-20T17:26:26.651390] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-04. [2025-10-20T17:26:26.929776] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:26:27.451419] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-02. [2025-10-20T17:26:27.638739] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:26:28.251489] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-02. [2025-10-20T17:26:28.731961] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:26:29.051413] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-03. [2025-10-20T17:26:29.292955] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-04 (2.64 seconds). [2025-10-20T17:26:29.851473] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-05. [2025-10-20T17:26:29.930219] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T17:26:30.120067] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:26:30.651576] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-04. [2025-10-20T17:26:31.451435] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-09. [2025-10-20T17:26:32.114360] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:26:32.251470] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-05. [2025-10-20T17:26:32.492553] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-05 (2.64 seconds). [2025-10-20T17:26:32.920816] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T17:26:33.051597] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-06. [2025-10-20T17:26:33.729997] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:26:33.851469] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-06. [2025-10-20T17:26:34.651518] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-03. [2025-10-20T17:26:35.418805] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T17:26:35.451338] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-07. [2025-10-20T17:26:35.632674] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-06 (2.58 seconds). [2025-10-20T17:26:35.977875] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:26:36.251398] [INFO ] [REMOTE ] BAX: Fetching primary data for 2025-10. [2025-10-20T17:26:36.785251] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T17:26:37.051386] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-04. [2025-10-20T17:26:37.851387] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-08. [2025-10-20T17:26:37.985573] [INFO ] [REMOTE ] BAX: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:26:37.994215] [INFO ] [REMOTE ] BAX: Full fetch complete (77616 records). [2025-10-20T17:26:37.994579] [DEBUG] [VALIDATOR ] BAX: Validating primary data. [2025-10-20T17:26:38.416365] [DEBUG] [VALIDATOR ] BAX: All expected trading days are present. [2025-10-20T17:26:38.416448] [DEBUG] [VALIDATOR ] BAX: Primary data validation passed. [2025-10-20T17:26:38.417236] [INFO ] [MAIN ] BAX: Fetched primary data is valid. [2025-10-20T17:26:38.417297] [DEBUG] [LOCAL ] BAX: Saving primary data. [2025-10-20T17:26:38.418832] [DEBUG] [LOCAL ] BAX: Trimmed 77616 records to 53537 (keeping 200 days of data) [2025-10-20T17:26:38.651451] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-07. [2025-10-20T17:26:38.779380] [DEBUG] [LOCAL ] BAX: Primary data saved (53537 records). [2025-10-20T17:26:38.779703] [INFO ] [MAIN ] BAX: Full history saved. [2025-10-20T17:26:38.779858] [INFO ] [MAIN ] Update Progress: 217/2367 symbols completed. BAX took 41.0s. Estimated remaining: 4h 23m 17s. Rate: 8.2 symbols/min [2025-10-20T17:26:38.990248] [INFO ] [MAIN ] BBDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:39.023915] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:26:39.060183] [DEBUG] [LOCAL ] BBDC: Loaded 33616 existing records from primary storage. [2025-10-20T17:26:39.060453] [INFO ] [MAIN ] BBDC: Loaded 33616 existing records. [2025-10-20T17:26:39.248604] [WARN ] [MAIN ] BBDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:39.248887] [INFO ] [MAIN ] BBDC: Wiping existing primary data before full refresh. [2025-10-20T17:26:39.249110] [INFO ] [LOCAL ] BBDC: Starting deletion of primary data. [2025-10-20T17:26:39.249356] [DEBUG] [LOCAL ] BBDC: Deleted primary data file. [2025-10-20T17:26:39.249423] [INFO ] [MAIN ] BBDC: Starting full primary refresh. [2025-10-20T17:26:39.463959] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-05. [2025-10-20T17:26:39.527846] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:26:39.691004] [INFO ] [REMOTE ] BBDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:40.251634] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-01. [2025-10-20T17:26:41.051407] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-09. [2025-10-20T17:26:41.267401] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T17:26:41.312468] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T17:26:41.773351] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-07 (3.12 seconds). [2025-10-20T17:26:41.851382] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-08. [2025-10-20T17:26:42.651377] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-02. [2025-10-20T17:26:42.739550] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:26:43.451421] [INFO ] [REMOTE ] BBAR: Fetching primary data for 2025-10. [2025-10-20T17:26:43.600133] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:26:44.251378] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-06. [2025-10-20T17:26:44.528515] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-08 (2.68 seconds). [2025-10-20T17:26:45.051365] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-09. [2025-10-20T17:26:45.367541] [INFO ] [REMOTE ] BBAR: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:26:45.374369] [INFO ] [REMOTE ] BBAR: Full fetch complete (61404 records). [2025-10-20T17:26:45.374466] [DEBUG] [VALIDATOR ] BBAR: Validating primary data. [2025-10-20T17:26:45.762463] [DEBUG] [VALIDATOR ] BBAR: All expected trading days are present. [2025-10-20T17:26:45.762545] [DEBUG] [VALIDATOR ] BBAR: Primary data validation passed. [2025-10-20T17:26:45.763229] [INFO ] [MAIN ] BBAR: Fetched primary data is valid. [2025-10-20T17:26:45.763290] [DEBUG] [LOCAL ] BBAR: Saving primary data. [2025-10-20T17:26:45.764462] [DEBUG] [LOCAL ] BBAR: Trimmed 61404 records to 42660 (keeping 200 days of data) [2025-10-20T17:26:45.854353] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-03. [2025-10-20T17:26:46.033365] [DEBUG] [LOCAL ] BBAR: Primary data saved (42660 records). [2025-10-20T17:26:46.045124] [INFO ] [MAIN ] BBAR: Full history saved. [2025-10-20T17:26:46.045225] [INFO ] [MAIN ] Update Progress: 218/2367 symbols completed. BBAR took 23.3s. Estimated remaining: 4h 23m 9s. Rate: 8.2 symbols/min [2025-10-20T17:26:46.172442] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:26:46.255053] [INFO ] [MAIN ] BBDO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:46.264165] [DEBUG] [LOCAL ] BBDO: Loaded 5104 existing records from primary storage. [2025-10-20T17:26:46.264317] [INFO ] [MAIN ] BBDO: Loaded 5104 existing records. [2025-10-20T17:26:46.449490] [WARN ] [MAIN ] BBDO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:46.449738] [INFO ] [MAIN ] BBDO: Wiping existing primary data before full refresh. [2025-10-20T17:26:46.449796] [INFO ] [LOCAL ] BBDO: Starting deletion of primary data. [2025-10-20T17:26:46.449933] [DEBUG] [LOCAL ] BBDO: Deleted primary data file. [2025-10-20T17:26:46.449972] [INFO ] [MAIN ] BBDO: Starting full primary refresh. [2025-10-20T17:26:46.661719] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-07. [2025-10-20T17:26:46.817510] [INFO ] [REMOTE ] BBDO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:47.327821] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:26:47.451356] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-04. [2025-10-20T17:26:47.747255] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-09 (2.70 seconds). [2025-10-20T17:26:48.197689] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:26:48.251339] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-08. [2025-10-20T17:26:48.926824] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:26:49.051413] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-05. [2025-10-20T17:26:49.851446] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-01. [2025-10-20T17:26:50.053676] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:26:50.419927] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:26:50.586528] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:26:50.651435] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-06. [2025-10-20T17:26:51.451426] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-09. [2025-10-20T17:26:52.156353] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:26:52.251364] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-02. [2025-10-20T17:26:52.848195] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T17:26:53.051372] [INFO ] [REMOTE ] BBAI: Fetching primary data for 2025-10. [2025-10-20T17:26:53.207091] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T17:26:53.851373] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-07. [2025-10-20T17:26:54.651346] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-03. [2025-10-20T17:26:55.118026] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:26:55.349087] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:26:55.451367] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-08. [2025-10-20T17:26:55.754090] [INFO ] [REMOTE ] BBAI: Incremental fetch complete for 2025-10 (2.70 seconds). [2025-10-20T17:26:55.761886] [INFO ] [REMOTE ] BBAI: Full fetch complete (78000 records). [2025-10-20T17:26:55.762021] [DEBUG] [VALIDATOR ] BBAI: Validating primary data. [2025-10-20T17:26:56.155658] [DEBUG] [VALIDATOR ] BBAI: All expected trading days are present. [2025-10-20T17:26:56.155752] [DEBUG] [VALIDATOR ] BBAI: Primary data validation passed. [2025-10-20T17:26:56.156751] [INFO ] [MAIN ] BBAI: Fetched primary data is valid. [2025-10-20T17:26:56.156814] [DEBUG] [LOCAL ] BBAI: Saving primary data. [2025-10-20T17:26:56.158624] [DEBUG] [LOCAL ] BBAI: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T17:26:56.251824] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-04. [2025-10-20T17:26:56.520692] [DEBUG] [LOCAL ] BBAI: Primary data saved (53820 records). [2025-10-20T17:26:56.522307] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:26:56.522620] [INFO ] [MAIN ] BBAI: Full history saved. [2025-10-20T17:26:56.529461] [INFO ] [MAIN ] Update Progress: 219/2367 symbols completed. BBAI took 45.6s. Estimated remaining: 4h 23m 32s. Rate: 8.2 symbols/min [2025-10-20T17:26:56.701299] [INFO ] [MAIN ] BBN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:26:56.727074] [DEBUG] [LOCAL ] BBN: Loaded 18233 existing records from primary storage. [2025-10-20T17:26:56.727319] [INFO ] [MAIN ] BBN: Loaded 18233 existing records. [2025-10-20T17:26:56.850811] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T17:26:56.934576] [WARN ] [MAIN ] BBN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:26:56.934832] [INFO ] [MAIN ] BBN: Wiping existing primary data before full refresh. [2025-10-20T17:26:56.934879] [INFO ] [LOCAL ] BBN: Starting deletion of primary data. [2025-10-20T17:26:56.935044] [DEBUG] [LOCAL ] BBN: Deleted primary data file. [2025-10-20T17:26:56.935083] [INFO ] [MAIN ] BBN: Starting full primary refresh. [2025-10-20T17:26:57.056445] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-09. [2025-10-20T17:26:57.318744] [INFO ] [REMOTE ] BBN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:26:57.851367] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-05. [2025-10-20T17:26:58.292923] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T17:26:58.293624] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T17:26:58.651378] [INFO ] [REMOTE ] BBD: Fetching primary data for 2025-10. [2025-10-20T17:26:59.451435] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-01. [2025-10-20T17:27:00.251356] [INFO ] [REMOTE ] BBDC: Fetching primary data for 2025-10. [2025-10-20T17:27:00.627452] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:27:00.750445] [INFO ] [REMOTE ] BBD: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:27:00.758819] [INFO ] [REMOTE ] BBD: Full fetch complete (77276 records). [2025-10-20T17:27:00.758953] [DEBUG] [VALIDATOR ] BBD: Validating primary data. [2025-10-20T17:27:01.057763] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-02. [2025-10-20T17:27:01.179904] [DEBUG] [VALIDATOR ] BBD: All expected trading days are present. [2025-10-20T17:27:01.180020] [DEBUG] [VALIDATOR ] BBD: Primary data validation passed. [2025-10-20T17:27:01.181398] [INFO ] [MAIN ] BBD: Fetched primary data is valid. [2025-10-20T17:27:01.181474] [DEBUG] [LOCAL ] BBD: Saving primary data. [2025-10-20T17:27:01.183697] [DEBUG] [LOCAL ] BBD: Trimmed 77276 records to 53276 (keeping 200 days of data) [2025-10-20T17:27:01.534750] [DEBUG] [LOCAL ] BBD: Primary data saved (53276 records). [2025-10-20T17:27:01.534901] [INFO ] [MAIN ] BBD: Full history saved. [2025-10-20T17:27:01.534985] [INFO ] [MAIN ] Update Progress: 220/2367 symbols completed. BBD took 37.1s. Estimated remaining: 4h 23m 2s. Rate: 8.2 symbols/min [2025-10-20T17:27:01.725092] [INFO ] [MAIN ] BBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:01.725307] [DEBUG] [LOCAL ] BBT: No existing data file found in primary storage. [2025-10-20T17:27:01.725349] [WARN ] [MAIN ] BBT: No existing data found, performing full refresh instead. [2025-10-20T17:27:01.725482] [INFO ] [MAIN ] BBT: Wiping existing primary data before full refresh. [2025-10-20T17:27:01.725527] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-10-20T17:27:01.725571] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-10-20T17:27:01.725608] [INFO ] [MAIN ] BBT: Starting full primary refresh. [2025-10-20T17:27:01.853201] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-06. [2025-10-20T17:27:01.919123] [INFO ] [REMOTE ] BBDC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:27:02.005145] [INFO ] [REMOTE ] BBDC: Full fetch complete (48865 records). [2025-10-20T17:27:02.037835] [DEBUG] [VALIDATOR ] BBDC: Validating primary data. [2025-10-20T17:27:02.130443] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T17:27:02.263819] [INFO ] [REMOTE ] BBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:02.567046] [DEBUG] [VALIDATOR ] BBDC: All expected trading days are present. [2025-10-20T17:27:02.567136] [DEBUG] [VALIDATOR ] BBDC: Primary data validation passed. [2025-10-20T17:27:02.567703] [INFO ] [MAIN ] BBDC: Fetched primary data is valid. [2025-10-20T17:27:02.567765] [DEBUG] [LOCAL ] BBDC: Saving primary data. [2025-10-20T17:27:02.568992] [DEBUG] [LOCAL ] BBDC: Trimmed 48865 records to 33048 (keeping 200 days of data) [2025-10-20T17:27:02.655132] [INFO ] [REMOTE ] BBT: Fetching primary data for 2025-01. [2025-10-20T17:27:02.667243] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:27:02.779528] [DEBUG] [LOCAL ] BBDC: Primary data saved (33048 records). [2025-10-20T17:27:02.779700] [INFO ] [MAIN ] BBDC: Full history saved. [2025-10-20T17:27:02.779803] [INFO ] [MAIN ] Update Progress: 221/2367 symbols completed. BBDC took 24.0s. Estimated remaining: 4h 21m 55s. Rate: 8.2 symbols/min [2025-10-20T17:27:02.879305] [DEBUG] [REMOTE ] BBT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:27:02.882385] [WARN ] [REMOTE ] BBT: No data returned for month 2025-01 [2025-10-20T17:27:02.887876] [FATAL] [REMOTE ] BBT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:27:02.898182] [FATAL] [MAIN ] BBT: Failed to fetch primary data. [2025-10-20T17:27:02.908506] [WARN ] [MAIN ] BBT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:27:02.934905] [INFO ] [LOCAL ] BBT: Starting deletion of primary data. [2025-10-20T17:27:02.952838] [INFO ] [MAIN ] BBU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:02.950163] [DEBUG] [LOCAL ] BBT: No primary data file found to delete. [2025-10-20T17:27:02.953018] [INFO ] [LOCAL ] BBT: Starting deletion of derivative data. [2025-10-20T17:27:02.953399] [DEBUG] [LOCAL ] BBT: No derivative data file found to delete. [2025-10-20T17:27:02.953497] [INFO ] [MAIN ] Update Progress: 222/2367 symbols completed. BBT took 1.4s. Estimated remaining: 4h 20m 39s. Rate: 8.2 symbols/min [2025-10-20T17:27:03.061834] [DEBUG] [LOCAL ] BBU: Loaded 5708 existing records from primary storage. [2025-10-20T17:27:03.065071] [INFO ] [MAIN ] BBU: Loaded 5708 existing records. [2025-10-20T17:27:03.265690] [INFO ] [MAIN ] BBUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:03.360402] [WARN ] [MAIN ] BBU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:03.365973] [DEBUG] [LOCAL ] BBUC: Loaded 14077 existing records from primary storage. [2025-10-20T17:27:03.366252] [INFO ] [MAIN ] BBU: Wiping existing primary data before full refresh. [2025-10-20T17:27:03.366517] [INFO ] [MAIN ] BBUC: Loaded 14077 existing records. [2025-10-20T17:27:03.399399] [INFO ] [LOCAL ] BBU: Starting deletion of primary data. [2025-10-20T17:27:03.456498] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-03. [2025-10-20T17:27:03.444546] [DEBUG] [LOCAL ] BBU: Deleted primary data file. [2025-10-20T17:27:03.474210] [INFO ] [MAIN ] BBU: Starting full primary refresh. [2025-10-20T17:27:03.661849] [WARN ] [MAIN ] BBUC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:03.664396] [INFO ] [MAIN ] BBUC: Wiping existing primary data before full refresh. [2025-10-20T17:27:03.664472] [INFO ] [LOCAL ] BBUC: Starting deletion of primary data. [2025-10-20T17:27:03.680431] [DEBUG] [LOCAL ] BBUC: Deleted primary data file. [2025-10-20T17:27:03.690722] [INFO ] [MAIN ] BBUC: Starting full primary refresh. [2025-10-20T17:27:04.194032] [INFO ] [REMOTE ] BBU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:04.257118] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-01. [2025-10-20T17:27:04.300781] [INFO ] [REMOTE ] BBUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:04.404230] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:27:04.993996] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:27:05.051389] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-02. [2025-10-20T17:27:05.622251] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T17:27:05.851388] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-04. [2025-10-20T17:27:06.651334] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-01. [2025-10-20T17:27:06.672048] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T17:27:07.451397] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-03. [2025-10-20T17:27:07.703937] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:27:08.003913] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T17:27:08.251402] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-04. [2025-10-20T17:27:08.659812] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T17:27:09.051365] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-05. [2025-10-20T17:27:09.529045] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:27:09.851416] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-06. [2025-10-20T17:27:10.323001] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T17:27:10.651374] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-05. [2025-10-20T17:27:11.451387] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-07. [2025-10-20T17:27:11.791237] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:27:11.888277] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T17:27:12.251385] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-08. [2025-10-20T17:27:12.767179] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T17:27:13.051408] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-09. [2025-10-20T17:27:13.798295] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T17:27:13.851405] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-02. [2025-10-20T17:27:14.398979] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T17:27:14.651398] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-03. [2025-10-20T17:27:15.451361] [INFO ] [REMOTE ] BBU: Fetching primary data for 2025-10. [2025-10-20T17:27:15.643183] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:27:16.251376] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-06. [2025-10-20T17:27:16.355339] [INFO ] [REMOTE ] BBU: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:27:16.357869] [INFO ] [REMOTE ] BBU: Full fetch complete (7467 records). [2025-10-20T17:27:16.357938] [DEBUG] [VALIDATOR ] BBU: Validating primary data. [2025-10-20T17:27:16.779364] [DEBUG] [VALIDATOR ] BBU: All expected trading days are present. [2025-10-20T17:27:16.779446] [DEBUG] [VALIDATOR ] BBU: Primary data validation passed. [2025-10-20T17:27:16.779697] [INFO ] [MAIN ] BBU: Fetched primary data is valid. [2025-10-20T17:27:16.779758] [DEBUG] [LOCAL ] BBU: Saving primary data. [2025-10-20T17:27:16.780275] [DEBUG] [LOCAL ] BBU: Trimmed 7467 records to 5727 (keeping 200 days of data) [2025-10-20T17:27:16.815879] [DEBUG] [LOCAL ] BBU: Primary data saved (5727 records). [2025-10-20T17:27:16.816032] [INFO ] [MAIN ] BBU: Full history saved. [2025-10-20T17:27:16.816126] [INFO ] [MAIN ] Update Progress: 223/2367 symbols completed. BBU took 14.0s. Estimated remaining: 4h 21m 35s. Rate: 8.2 symbols/min [2025-10-20T17:27:16.998387] [INFO ] [MAIN ] BBVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:17.054182] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-07. [2025-10-20T17:27:17.060608] [DEBUG] [LOCAL ] BBVA: Loaded 49862 existing records from primary storage. [2025-10-20T17:27:17.061336] [INFO ] [MAIN ] BBVA: Loaded 49862 existing records. [2025-10-20T17:27:17.303387] [WARN ] [MAIN ] BBVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:17.303808] [INFO ] [MAIN ] BBVA: Wiping existing primary data before full refresh. [2025-10-20T17:27:17.303941] [INFO ] [LOCAL ] BBVA: Starting deletion of primary data. [2025-10-20T17:27:17.315799] [DEBUG] [LOCAL ] BBVA: Deleted primary data file. [2025-10-20T17:27:17.315871] [INFO ] [MAIN ] BBVA: Starting full primary refresh. [2025-10-20T17:27:17.316610] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:27:17.737493] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T17:27:17.756783] [INFO ] [REMOTE ] BBVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:17.851414] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-01. [2025-10-20T17:27:18.651448] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-08. [2025-10-20T17:27:19.004766] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-10-20T17:27:19.451405] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-09. [2025-10-20T17:27:19.629487] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:27:20.121021] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:27:20.251420] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-04. [2025-10-20T17:27:21.051450] [INFO ] [REMOTE ] BBDO: Fetching primary data for 2025-10. [2025-10-20T17:27:21.079549] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T17:27:21.494085] [INFO ] [REMOTE ] BBDO: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T17:27:21.495549] [INFO ] [REMOTE ] BBDO: Full fetch complete (7511 records). [2025-10-20T17:27:21.495621] [DEBUG] [VALIDATOR ] BBDO: Validating primary data. [2025-10-20T17:27:21.861709] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-07. [2025-10-20T17:27:21.862742] [DEBUG] [VALIDATOR ] BBDO: All expected trading days are present. [2025-10-20T17:27:21.862793] [DEBUG] [VALIDATOR ] BBDO: Primary data validation passed. [2025-10-20T17:27:21.862987] [INFO ] [MAIN ] BBDO: Fetched primary data is valid. [2025-10-20T17:27:21.863025] [DEBUG] [LOCAL ] BBDO: Saving primary data. [2025-10-20T17:27:21.863446] [DEBUG] [LOCAL ] BBDO: Trimmed 7511 records to 5012 (keeping 200 days of data) [2025-10-20T17:27:21.893322] [DEBUG] [LOCAL ] BBDO: Primary data saved (5012 records). [2025-10-20T17:27:21.893442] [INFO ] [MAIN ] BBDO: Full history saved. [2025-10-20T17:27:21.893518] [INFO ] [MAIN ] Update Progress: 224/2367 symbols completed. BBDO took 35.8s. Estimated remaining: 4h 21m 6s. Rate: 8.2 symbols/min [2025-10-20T17:27:22.069774] [INFO ] [MAIN ] BBW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:22.098104] [DEBUG] [LOCAL ] BBW: Loaded 27830 existing records from primary storage. [2025-10-20T17:27:22.098412] [INFO ] [MAIN ] BBW: Loaded 27830 existing records. [2025-10-20T17:27:22.285447] [WARN ] [MAIN ] BBW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:22.285614] [INFO ] [MAIN ] BBW: Wiping existing primary data before full refresh. [2025-10-20T17:27:22.285690] [INFO ] [LOCAL ] BBW: Starting deletion of primary data. [2025-10-20T17:27:22.285879] [DEBUG] [LOCAL ] BBW: Deleted primary data file. [2025-10-20T17:27:22.285919] [INFO ] [MAIN ] BBW: Starting full primary refresh. [2025-10-20T17:27:22.638076] [INFO ] [REMOTE ] BBW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:22.651340] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-01. [2025-10-20T17:27:23.015541] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:27:23.451390] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-02. [2025-10-20T17:27:23.965938] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T17:27:24.251389] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-02. [2025-10-20T17:27:24.703170] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:27:25.051378] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-03. [2025-10-20T17:27:25.132542] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:27:25.851368] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-08. [2025-10-20T17:27:26.651364] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-03. [2025-10-20T17:27:26.795089] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:27:26.928191] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T17:27:27.451385] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-04. [2025-10-20T17:27:27.599730] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:27:28.251380] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-04. [2025-10-20T17:27:29.051364] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-05. [2025-10-20T17:27:29.257825] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T17:27:29.586145] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T17:27:29.722477] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:27:29.851357] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-06. [2025-10-20T17:27:30.641075] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:27:30.651324] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-07. [2025-10-20T17:27:31.451381] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-09. [2025-10-20T17:27:31.520517] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T17:27:32.172957] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:27:32.251396] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-08. [2025-10-20T17:27:33.051406] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-05. [2025-10-20T17:27:33.077676] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:27:33.851381] [INFO ] [REMOTE ] BBN: Fetching primary data for 2025-10. [2025-10-20T17:27:34.651400] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-09. [2025-10-20T17:27:34.823880] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:27:34.961639] [INFO ] [REMOTE ] BBN: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:27:34.965177] [INFO ] [REMOTE ] BBN: Full fetch complete (25135 records). [2025-10-20T17:27:34.965271] [DEBUG] [VALIDATOR ] BBN: Validating primary data. [2025-10-20T17:27:35.358813] [DEBUG] [VALIDATOR ] BBN: All expected trading days are present. [2025-10-20T17:27:35.367447] [DEBUG] [VALIDATOR ] BBN: Primary data validation passed. [2025-10-20T17:27:35.368305] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:27:35.368776] [INFO ] [MAIN ] BBN: Fetched primary data is valid. [2025-10-20T17:27:35.372420] [DEBUG] [LOCAL ] BBN: Saving primary data. [2025-10-20T17:27:35.373398] [DEBUG] [LOCAL ] BBN: Trimmed 25135 records to 17924 (keeping 200 days of data) [2025-10-20T17:27:35.451872] [INFO ] [REMOTE ] BBUC: Fetching primary data for 2025-10. [2025-10-20T17:27:35.483829] [DEBUG] [LOCAL ] BBN: Primary data saved (17924 records). [2025-10-20T17:27:35.483969] [INFO ] [MAIN ] BBN: Full history saved. [2025-10-20T17:27:35.484047] [INFO ] [MAIN ] Update Progress: 225/2367 symbols completed. BBN took 39.0s. Estimated remaining: 4h 21m 58s. Rate: 8.2 symbols/min [2025-10-20T17:27:35.653730] [INFO ] [MAIN ] BBWI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:35.653896] [DEBUG] [LOCAL ] BBWI: No existing data file found in primary storage. [2025-10-20T17:27:35.653934] [WARN ] [MAIN ] BBWI: No existing data found, performing full refresh instead. [2025-10-20T17:27:35.654048] [INFO ] [MAIN ] BBWI: Wiping existing primary data before full refresh. [2025-10-20T17:27:35.654101] [INFO ] [LOCAL ] BBWI: Starting deletion of primary data. [2025-10-20T17:27:35.654145] [DEBUG] [LOCAL ] BBWI: No primary data file found to delete. [2025-10-20T17:27:35.654194] [INFO ] [MAIN ] BBWI: Starting full primary refresh. [2025-10-20T17:27:36.009804] [INFO ] [REMOTE ] BBWI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:36.251348] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-05. [2025-10-20T17:27:36.556474] [INFO ] [REMOTE ] BBUC: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:27:36.560018] [INFO ] [REMOTE ] BBUC: Full fetch complete (19160 records). [2025-10-20T17:27:36.560129] [DEBUG] [VALIDATOR ] BBUC: Validating primary data. [2025-10-20T17:27:36.938756] [DEBUG] [VALIDATOR ] BBUC: All expected trading days are present. [2025-10-20T17:27:36.938834] [DEBUG] [VALIDATOR ] BBUC: Primary data validation passed. [2025-10-20T17:27:36.939148] [INFO ] [MAIN ] BBUC: Fetched primary data is valid. [2025-10-20T17:27:36.939185] [DEBUG] [LOCAL ] BBUC: Saving primary data. [2025-10-20T17:27:36.939728] [DEBUG] [LOCAL ] BBUC: Trimmed 19160 records to 13935 (keeping 200 days of data) [2025-10-20T17:27:37.020094] [DEBUG] [LOCAL ] BBUC: Primary data saved (13935 records). [2025-10-20T17:27:37.020222] [INFO ] [MAIN ] BBUC: Full history saved. [2025-10-20T17:27:37.020297] [INFO ] [MAIN ] Update Progress: 226/2367 symbols completed. BBUC took 34.1s. Estimated remaining: 4h 20m 56s. Rate: 8.2 symbols/min [2025-10-20T17:27:37.056967] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-06. [2025-10-20T17:27:37.198725] [INFO ] [MAIN ] BBY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:37.239443] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T17:27:37.262147] [DEBUG] [LOCAL ] BBY: Loaded 54218 existing records from primary storage. [2025-10-20T17:27:37.262451] [INFO ] [MAIN ] BBY: Loaded 54218 existing records. [2025-10-20T17:27:37.443815] [WARN ] [MAIN ] BBY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:37.444058] [INFO ] [MAIN ] BBY: Wiping existing primary data before full refresh. [2025-10-20T17:27:37.444124] [INFO ] [LOCAL ] BBY: Starting deletion of primary data. [2025-10-20T17:27:37.444745] [DEBUG] [LOCAL ] BBY: Deleted primary data file. [2025-10-20T17:27:37.444817] [INFO ] [MAIN ] BBY: Starting full primary refresh. [2025-10-20T17:27:37.824959] [INFO ] [REMOTE ] BBY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:37.851422] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-06. [2025-10-20T17:27:38.656494] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-01. [2025-10-20T17:27:38.670680] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T17:27:38.719453] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:27:39.451381] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-07. [2025-10-20T17:27:40.251415] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-07. [2025-10-20T17:27:40.470730] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T17:27:40.509965] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T17:27:41.051375] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-08. [2025-10-20T17:27:41.773757] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:27:41.851379] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-02. [2025-10-20T17:27:42.315407] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T17:27:42.651362] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-01. [2025-10-20T17:27:43.451388] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-09. [2025-10-20T17:27:43.589171] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:27:44.251413] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-03. [2025-10-20T17:27:44.571757] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:27:44.863811] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T17:27:45.051408] [INFO ] [REMOTE ] BBW: Fetching primary data for 2025-10. [2025-10-20T17:27:45.859607] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-08. [2025-10-20T17:27:45.862729] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:27:46.438369] [INFO ] [REMOTE ] BBW: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:27:46.443450] [INFO ] [REMOTE ] BBW: Full fetch complete (39732 records). [2025-10-20T17:27:46.443529] [DEBUG] [VALIDATOR ] BBW: Validating primary data. [2025-10-20T17:27:46.658545] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-04. [2025-10-20T17:27:46.827074] [DEBUG] [VALIDATOR ] BBW: All expected trading days are present. [2025-10-20T17:27:46.827156] [DEBUG] [VALIDATOR ] BBW: Primary data validation passed. [2025-10-20T17:27:46.827666] [INFO ] [MAIN ] BBW: Fetched primary data is valid. [2025-10-20T17:27:46.827730] [DEBUG] [LOCAL ] BBW: Saving primary data. [2025-10-20T17:27:46.828704] [DEBUG] [LOCAL ] BBW: Trimmed 39732 records to 27378 (keeping 200 days of data) [2025-10-20T17:27:46.992148] [DEBUG] [LOCAL ] BBW: Primary data saved (27378 records). [2025-10-20T17:27:46.992322] [INFO ] [MAIN ] BBW: Full history saved. [2025-10-20T17:27:46.992417] [INFO ] [MAIN ] Update Progress: 227/2367 symbols completed. BBW took 25.1s. Estimated remaining: 4h 21m 14s. Rate: 8.2 symbols/min [2025-10-20T17:27:47.187994] [INFO ] [MAIN ] BC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:47.230395] [DEBUG] [LOCAL ] BC: Loaded 44654 existing records from primary storage. [2025-10-20T17:27:47.230704] [INFO ] [MAIN ] BC: Loaded 44654 existing records. [2025-10-20T17:27:47.407202] [WARN ] [MAIN ] BC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:47.407472] [INFO ] [MAIN ] BC: Wiping existing primary data before full refresh. [2025-10-20T17:27:47.407708] [INFO ] [LOCAL ] BC: Starting deletion of primary data. [2025-10-20T17:27:47.407999] [DEBUG] [LOCAL ] BC: Deleted primary data file. [2025-10-20T17:27:47.408058] [INFO ] [MAIN ] BC: Starting full primary refresh. [2025-10-20T17:27:47.464270] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-02. [2025-10-20T17:27:47.733418] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T17:27:47.836866] [INFO ] [REMOTE ] BC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:48.251383] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-09. [2025-10-20T17:27:48.398326] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T17:27:49.051387] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-01. [2025-10-20T17:27:49.207992] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:27:49.851380] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-05. [2025-10-20T17:27:49.956181] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:27:50.651363] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-03. [2025-10-20T17:27:50.748691] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:27:51.451390] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-02. [2025-10-20T17:27:51.805867] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:27:52.251360] [INFO ] [REMOTE ] BBVA: Fetching primary data for 2025-10. [2025-10-20T17:27:52.463341] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T17:27:52.573861] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:27:53.051388] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-06. [2025-10-20T17:27:53.851393] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-03. [2025-10-20T17:27:54.068319] [INFO ] [REMOTE ] BBVA: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T17:27:54.076614] [INFO ] [REMOTE ] BBVA: Full fetch complete (71846 records). [2025-10-20T17:27:54.076868] [DEBUG] [VALIDATOR ] BBVA: Validating primary data. [2025-10-20T17:27:54.473557] [DEBUG] [VALIDATOR ] BBVA: All expected trading days are present. [2025-10-20T17:27:54.473686] [DEBUG] [VALIDATOR ] BBVA: Primary data validation passed. [2025-10-20T17:27:54.474582] [INFO ] [MAIN ] BBVA: Fetched primary data is valid. [2025-10-20T17:27:54.474631] [DEBUG] [LOCAL ] BBVA: Saving primary data. [2025-10-20T17:27:54.476215] [DEBUG] [LOCAL ] BBVA: Trimmed 71846 records to 49119 (keeping 200 days of data) [2025-10-20T17:27:54.652186] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-04. [2025-10-20T17:27:54.771717] [DEBUG] [LOCAL ] BBVA: Primary data saved (49119 records). [2025-10-20T17:27:54.771852] [INFO ] [MAIN ] BBVA: Full history saved. [2025-10-20T17:27:54.771946] [INFO ] [MAIN ] Update Progress: 228/2367 symbols completed. BBVA took 38.0s. Estimated remaining: 4h 21m 11s. Rate: 8.2 symbols/min [2025-10-20T17:27:54.945531] [INFO ] [MAIN ] BCAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:27:54.988064] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T17:27:55.024547] [DEBUG] [LOCAL ] BCAT: Loaded 34802 existing records from primary storage. [2025-10-20T17:27:55.024860] [INFO ] [MAIN ] BCAT: Loaded 34802 existing records. [2025-10-20T17:27:55.198175] [WARN ] [MAIN ] BCAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:27:55.198465] [INFO ] [MAIN ] BCAT: Wiping existing primary data before full refresh. [2025-10-20T17:27:55.198522] [INFO ] [LOCAL ] BCAT: Starting deletion of primary data. [2025-10-20T17:27:55.198881] [DEBUG] [LOCAL ] BCAT: Deleted primary data file. [2025-10-20T17:27:55.198940] [INFO ] [MAIN ] BCAT: Starting full primary refresh. [2025-10-20T17:27:55.456482] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-07. [2025-10-20T17:27:55.575456] [INFO ] [REMOTE ] BCAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:27:55.604430] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:27:56.251355] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-04. [2025-10-20T17:27:56.417514] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:27:57.051419] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-01. [2025-10-20T17:27:57.499761] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T17:27:57.851347] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-08. [2025-10-20T17:27:58.043572] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:27:58.098670] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:27:58.651456] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-02. [2025-10-20T17:27:59.253029] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:27:59.451465] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-05. [2025-10-20T17:27:59.988928] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:28:00.251382] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-05. [2025-10-20T17:28:00.983095] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:28:01.051387] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-06. [2025-10-20T17:28:01.851369] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-03. [2025-10-20T17:28:02.211197] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T17:28:02.651365] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-06. [2025-10-20T17:28:02.682414] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T17:28:02.952043] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:28:03.451413] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-09. [2025-10-20T17:28:04.251380] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-07. [2025-10-20T17:28:04.369741] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:28:05.051408] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-07. [2025-10-20T17:28:05.245233] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T17:28:05.672239] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:28:05.851377] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-08. [2025-10-20T17:28:06.651415] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-04. [2025-10-20T17:28:06.994209] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:28:07.301555] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:28:07.451450] [INFO ] [REMOTE ] BBY: Fetching primary data for 2025-10. [2025-10-20T17:28:08.112490] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T17:28:08.251377] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-05. [2025-10-20T17:28:09.011492] [INFO ] [REMOTE ] BBY: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T17:28:09.019872] [INFO ] [REMOTE ] BBY: Full fetch complete (77395 records). [2025-10-20T17:28:09.020048] [DEBUG] [VALIDATOR ] BBY: Validating primary data. [2025-10-20T17:28:09.056621] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-08. [2025-10-20T17:28:09.447468] [DEBUG] [VALIDATOR ] BBY: All expected trading days are present. [2025-10-20T17:28:09.447564] [DEBUG] [VALIDATOR ] BBY: Primary data validation passed. [2025-10-20T17:28:09.448372] [INFO ] [MAIN ] BBY: Fetched primary data is valid. [2025-10-20T17:28:09.448439] [DEBUG] [LOCAL ] BBY: Saving primary data. [2025-10-20T17:28:09.449923] [DEBUG] [LOCAL ] BBY: Trimmed 77395 records to 53436 (keeping 200 days of data) [2025-10-20T17:28:09.784466] [DEBUG] [LOCAL ] BBY: Primary data saved (53436 records). [2025-10-20T17:28:09.784783] [INFO ] [MAIN ] BBY: Full history saved. [2025-10-20T17:28:09.784897] [INFO ] [MAIN ] Update Progress: 229/2367 symbols completed. BBY took 32.8s. Estimated remaining: 4h 22m 15s. Rate: 8.2 symbols/min [2025-10-20T17:28:09.855721] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-09. [2025-10-20T17:28:09.965689] [INFO ] [MAIN ] BCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:09.984577] [DEBUG] [LOCAL ] BCC: No existing data file found in primary storage. [2025-10-20T17:28:09.985418] [WARN ] [MAIN ] BCC: No existing data found, performing full refresh instead. [2025-10-20T17:28:09.985622] [INFO ] [MAIN ] BCC: Wiping existing primary data before full refresh. [2025-10-20T17:28:09.986573] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:28:09.986694] [INFO ] [LOCAL ] BCC: Starting deletion of primary data. [2025-10-20T17:28:09.994792] [DEBUG] [LOCAL ] BCC: No primary data file found to delete. [2025-10-20T17:28:09.995596] [INFO ] [MAIN ] BCC: Starting full primary refresh. [2025-10-20T17:28:10.359750] [INFO ] [REMOTE ] BCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:10.651365] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-06. [2025-10-20T17:28:10.856831] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:28:11.451440] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-09. [2025-10-20T17:28:11.489420] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:28:12.051249] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T17:28:12.251391] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-07. [2025-10-20T17:28:12.977798] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:28:13.051370] [INFO ] [REMOTE ] BC: Fetching primary data for 2025-10. [2025-10-20T17:28:13.353048] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T17:28:13.851441] [INFO ] [REMOTE ] BBWI: Fetching primary data for 2025-10. [2025-10-20T17:28:14.659440] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-08. [2025-10-20T17:28:14.674566] [INFO ] [REMOTE ] BC: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:28:14.681456] [INFO ] [REMOTE ] BC: Full fetch complete (63673 records). [2025-10-20T17:28:14.681607] [DEBUG] [VALIDATOR ] BC: Validating primary data. [2025-10-20T17:28:15.076151] [DEBUG] [VALIDATOR ] BC: All expected trading days are present. [2025-10-20T17:28:15.076260] [DEBUG] [VALIDATOR ] BC: Primary data validation passed. [2025-10-20T17:28:15.077019] [INFO ] [MAIN ] BC: Fetched primary data is valid. [2025-10-20T17:28:15.077083] [DEBUG] [LOCAL ] BC: Saving primary data. [2025-10-20T17:28:15.078396] [DEBUG] [LOCAL ] BC: Trimmed 63673 records to 43999 (keeping 200 days of data) [2025-10-20T17:28:15.344510] [DEBUG] [LOCAL ] BC: Primary data saved (43999 records). [2025-10-20T17:28:15.344669] [INFO ] [MAIN ] BC: Full history saved. [2025-10-20T17:28:15.344767] [INFO ] [MAIN ] Update Progress: 230/2367 symbols completed. BC took 28.4s. Estimated remaining: 4h 21m 51s. Rate: 8.2 symbols/min [2025-10-20T17:28:15.456498] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-01. [2025-10-20T17:28:15.522586] [INFO ] [MAIN ] BCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:15.568877] [DEBUG] [LOCAL ] BCE: Loaded 53022 existing records from primary storage. [2025-10-20T17:28:15.569190] [INFO ] [MAIN ] BCE: Loaded 53022 existing records. [2025-10-20T17:28:15.756597] [WARN ] [MAIN ] BCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:15.757052] [INFO ] [MAIN ] BCE: Wiping existing primary data before full refresh. [2025-10-20T17:28:15.757115] [INFO ] [LOCAL ] BCE: Starting deletion of primary data. [2025-10-20T17:28:15.775323] [DEBUG] [LOCAL ] BCE: Deleted primary data file. [2025-10-20T17:28:15.775478] [INFO ] [MAIN ] BCE: Starting full primary refresh. [2025-10-20T17:28:15.844891] [INFO ] [REMOTE ] BBWI: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:28:15.892249] [INFO ] [REMOTE ] BBWI: Full fetch complete (77629 records). [2025-10-20T17:28:15.918754] [DEBUG] [VALIDATOR ] BBWI: Validating primary data. [2025-10-20T17:28:16.340279] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:28:16.382026] [INFO ] [REMOTE ] BCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:16.402832] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-01. [2025-10-20T17:28:16.590062] [DEBUG] [VALIDATOR ] BBWI: All expected trading days are present. [2025-10-20T17:28:16.590157] [DEBUG] [VALIDATOR ] BBWI: Primary data validation passed. [2025-10-20T17:28:16.590933] [INFO ] [MAIN ] BBWI: Fetched primary data is valid. [2025-10-20T17:28:16.590994] [DEBUG] [LOCAL ] BBWI: Saving primary data. [2025-10-20T17:28:16.592382] [DEBUG] [LOCAL ] BBWI: Trimmed 77629 records to 53584 (keeping 200 days of data) [2025-10-20T17:28:16.911020] [DEBUG] [LOCAL ] BBWI: Primary data saved (53584 records). [2025-10-20T17:28:16.911180] [INFO ] [MAIN ] BBWI: Full history saved. [2025-10-20T17:28:16.911361] [INFO ] [MAIN ] Update Progress: 231/2367 symbols completed. BBWI took 41.4s. Estimated remaining: 4h 20m 50s. Rate: 8.2 symbols/min [2025-10-20T17:28:16.996524] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:28:17.056478] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-02. [2025-10-20T17:28:17.103495] [INFO ] [MAIN ] BCH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:17.135298] [DEBUG] [LOCAL ] BCH: Loaded 30818 existing records from primary storage. [2025-10-20T17:28:17.135627] [INFO ] [MAIN ] BCH: Loaded 30818 existing records. [2025-10-20T17:28:17.309122] [WARN ] [MAIN ] BCH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:17.309332] [INFO ] [MAIN ] BCH: Wiping existing primary data before full refresh. [2025-10-20T17:28:17.309387] [INFO ] [LOCAL ] BCH: Starting deletion of primary data. [2025-10-20T17:28:17.309605] [DEBUG] [LOCAL ] BCH: Deleted primary data file. [2025-10-20T17:28:17.309699] [INFO ] [MAIN ] BCH: Starting full primary refresh. [2025-10-20T17:28:17.672275] [INFO ] [REMOTE ] BCH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:17.851386] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-09. [2025-10-20T17:28:18.283826] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:28:18.454045] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:28:18.651434] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-01. [2025-10-20T17:28:19.235869] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:28:19.451397] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-03. [2025-10-20T17:28:19.572947] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:28:20.251454] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-02. [2025-10-20T17:28:20.556459] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:28:21.051403] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-04. [2025-10-20T17:28:21.851418] [INFO ] [REMOTE ] BCAT: Fetching primary data for 2025-10. [2025-10-20T17:28:21.938887] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:28:22.042557] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T17:28:22.651369] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-05. [2025-10-20T17:28:23.183683] [INFO ] [REMOTE ] BCAT: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:28:23.189165] [INFO ] [REMOTE ] BCAT: Full fetch complete (50025 records). [2025-10-20T17:28:23.189254] [DEBUG] [VALIDATOR ] BCAT: Validating primary data. [2025-10-20T17:28:23.456584] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-03. [2025-10-20T17:28:23.574812] [DEBUG] [VALIDATOR ] BCAT: All expected trading days are present. [2025-10-20T17:28:23.575004] [DEBUG] [VALIDATOR ] BCAT: Primary data validation passed. [2025-10-20T17:28:23.575739] [INFO ] [MAIN ] BCAT: Fetched primary data is valid. [2025-10-20T17:28:23.575819] [DEBUG] [LOCAL ] BCAT: Saving primary data. [2025-10-20T17:28:23.577093] [DEBUG] [LOCAL ] BCAT: Trimmed 50025 records to 34298 (keeping 200 days of data) [2025-10-20T17:28:23.802743] [DEBUG] [LOCAL ] BCAT: Primary data saved (34298 records). [2025-10-20T17:28:23.811281] [INFO ] [MAIN ] BCAT: Full history saved. [2025-10-20T17:28:23.811401] [INFO ] [MAIN ] Update Progress: 232/2367 symbols completed. BCAT took 29.0s. Estimated remaining: 4h 20m 39s. Rate: 8.2 symbols/min [2025-10-20T17:28:23.905819] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:28:24.033429] [INFO ] [MAIN ] BCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:24.033624] [DEBUG] [LOCAL ] BCO: No existing data file found in primary storage. [2025-10-20T17:28:24.033693] [WARN ] [MAIN ] BCO: No existing data found, performing full refresh instead. [2025-10-20T17:28:24.033811] [INFO ] [MAIN ] BCO: Wiping existing primary data before full refresh. [2025-10-20T17:28:24.033853] [INFO ] [LOCAL ] BCO: Starting deletion of primary data. [2025-10-20T17:28:24.033893] [DEBUG] [LOCAL ] BCO: No primary data file found to delete. [2025-10-20T17:28:24.033924] [INFO ] [MAIN ] BCO: Starting full primary refresh. [2025-10-20T17:28:24.260701] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-02. [2025-10-20T17:28:24.404240] [INFO ] [REMOTE ] BCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:25.058122] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-06. [2025-10-20T17:28:25.061277] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:28:25.532125] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:28:25.851366] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-01. [2025-10-20T17:28:26.187459] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:28:26.651443] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-04. [2025-10-20T17:28:27.041883] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:28:27.451470] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-02. [2025-10-20T17:28:28.251347] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-07. [2025-10-20T17:28:28.350666] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:28:28.417410] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:28:29.051375] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-03. [2025-10-20T17:28:29.324726] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T17:28:29.851344] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-08. [2025-10-20T17:28:30.188803] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T17:28:30.651372] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-04. [2025-10-20T17:28:31.386218] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:28:31.451372] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-09. [2025-10-20T17:28:32.251403] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-05. [2025-10-20T17:28:32.564571] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T17:28:33.051410] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-05. [2025-10-20T17:28:33.099431] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:28:33.851388] [INFO ] [REMOTE ] BCC: Fetching primary data for 2025-10. [2025-10-20T17:28:34.613778] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-10-20T17:28:34.651336] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-06. [2025-10-20T17:28:34.725254] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T17:28:35.218736] [INFO ] [REMOTE ] BCC: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:28:35.224827] [INFO ] [REMOTE ] BCC: Full fetch complete (41522 records). [2025-10-20T17:28:35.224984] [DEBUG] [VALIDATOR ] BCC: Validating primary data. [2025-10-20T17:28:35.461765] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-06. [2025-10-20T17:28:35.603077] [DEBUG] [VALIDATOR ] BCC: All expected trading days are present. [2025-10-20T17:28:35.603167] [DEBUG] [VALIDATOR ] BCC: Primary data validation passed. [2025-10-20T17:28:35.603823] [INFO ] [MAIN ] BCC: Fetched primary data is valid. [2025-10-20T17:28:35.603886] [DEBUG] [LOCAL ] BCC: Saving primary data. [2025-10-20T17:28:35.604878] [DEBUG] [LOCAL ] BCC: Trimmed 41522 records to 28671 (keeping 200 days of data) [2025-10-20T17:28:35.796575] [DEBUG] [LOCAL ] BCC: Primary data saved (28671 records). [2025-10-20T17:28:35.796747] [INFO ] [MAIN ] BCC: Full history saved. [2025-10-20T17:28:35.796828] [INFO ] [MAIN ] Update Progress: 233/2367 symbols completed. BCC took 26.0s. Estimated remaining: 4h 21m 14s. Rate: 8.2 symbols/min [2025-10-20T17:28:35.986736] [INFO ] [MAIN ] BCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:36.039700] [DEBUG] [LOCAL ] BCS: Loaded 54222 existing records from primary storage. [2025-10-20T17:28:36.040055] [INFO ] [MAIN ] BCS: Loaded 54222 existing records. [2025-10-20T17:28:36.239370] [WARN ] [MAIN ] BCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:36.240273] [INFO ] [MAIN ] BCS: Wiping existing primary data before full refresh. [2025-10-20T17:28:36.240552] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-10-20T17:28:36.240842] [DEBUG] [LOCAL ] BCS: Deleted primary data file. [2025-10-20T17:28:36.240892] [INFO ] [MAIN ] BCS: Starting full primary refresh. [2025-10-20T17:28:36.269864] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-03. [2025-10-20T17:28:36.592477] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:28:36.650291] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T17:28:36.725039] [INFO ] [REMOTE ] BCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:37.051357] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-07. [2025-10-20T17:28:37.781430] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T17:28:37.851415] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-04. [2025-10-20T17:28:38.207392] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:28:38.651466] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-07. [2025-10-20T17:28:39.238807] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:28:39.451434] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-08. [2025-10-20T17:28:40.251337] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-05. [2025-10-20T17:28:40.353124] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T17:28:40.718252] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:28:41.051414] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-08. [2025-10-20T17:28:41.113466] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T17:28:41.851379] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-06. [2025-10-20T17:28:42.414480] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T17:28:42.612021] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:28:42.651368] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-09. [2025-10-20T17:28:43.451378] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-07. [2025-10-20T17:28:44.251416] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-09. [2025-10-20T17:28:44.425638] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:28:44.763505] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:28:44.990539] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T17:28:45.051392] [INFO ] [REMOTE ] BCE: Fetching primary data for 2025-10. [2025-10-20T17:28:45.851380] [INFO ] [REMOTE ] BCO: Fetching primary data for 2025-10. [2025-10-20T17:28:46.651366] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-01. [2025-10-20T17:28:46.861060] [INFO ] [REMOTE ] BCE: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T17:28:46.868917] [INFO ] [REMOTE ] BCE: Full fetch complete (76119 records). [2025-10-20T17:28:46.869045] [DEBUG] [VALIDATOR ] BCE: Validating primary data. [2025-10-20T17:28:47.240511] [INFO ] [REMOTE ] BCO: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:28:47.282059] [INFO ] [REMOTE ] BCO: Full fetch complete (36223 records). [2025-10-20T17:28:47.293211] [DEBUG] [VALIDATOR ] BCO: Validating primary data. [2025-10-20T17:28:47.377230] [DEBUG] [VALIDATOR ] BCE: All expected trading days are present. [2025-10-20T17:28:47.383684] [DEBUG] [VALIDATOR ] BCE: Primary data validation passed. [2025-10-20T17:28:47.404339] [INFO ] [MAIN ] BCE: Fetched primary data is valid. [2025-10-20T17:28:47.426273] [DEBUG] [LOCAL ] BCE: Saving primary data. [2025-10-20T17:28:47.461138] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-08. [2025-10-20T17:28:47.542579] [DEBUG] [LOCAL ] BCE: Trimmed 76119 records to 52271 (keeping 200 days of data) [2025-10-20T17:28:47.861017] [DEBUG] [VALIDATOR ] BCO: All expected trading days are present. [2025-10-20T17:28:47.864681] [DEBUG] [VALIDATOR ] BCO: Primary data validation passed. [2025-10-20T17:28:47.867089] [INFO ] [MAIN ] BCO: Fetched primary data is valid. [2025-10-20T17:28:47.868893] [DEBUG] [LOCAL ] BCO: Saving primary data. [2025-10-20T17:28:47.934044] [DEBUG] [LOCAL ] BCO: Trimmed 36223 records to 25219 (keeping 200 days of data) [2025-10-20T17:28:48.228635] [DEBUG] [LOCAL ] BCE: Primary data saved (52271 records). [2025-10-20T17:28:48.231850] [INFO ] [MAIN ] BCE: Full history saved. [2025-10-20T17:28:48.231963] [INFO ] [MAIN ] Update Progress: 234/2367 symbols completed. BCE took 32.9s. Estimated remaining: 4h 21m 53s. Rate: 8.1 symbols/min [2025-10-20T17:28:48.340817] [DEBUG] [LOCAL ] BCO: Primary data saved (25219 records). [2025-10-20T17:28:48.340984] [INFO ] [MAIN ] BCO: Full history saved. [2025-10-20T17:28:48.346822] [INFO ] [MAIN ] Update Progress: 235/2367 symbols completed. BCO took 24.5s. Estimated remaining: 4h 20m 40s. Rate: 8.2 symbols/min [2025-10-20T17:28:48.517753] [INFO ] [MAIN ] BCSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:48.590795] [INFO ] [MAIN ] BCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:48.630557] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:28:48.645693] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-02. [2025-10-20T17:28:48.663216] [DEBUG] [LOCAL ] BCX: Loaded 22404 existing records from primary storage. [2025-10-20T17:28:48.662880] [DEBUG] [LOCAL ] BCSF: Loaded 28764 existing records from primary storage. [2025-10-20T17:28:48.663541] [INFO ] [MAIN ] BCX: Loaded 22404 existing records. [2025-10-20T17:28:48.684016] [INFO ] [MAIN ] BCSF: Loaded 28764 existing records. [2025-10-20T17:28:48.973731] [WARN ] [MAIN ] BCX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:49.005050] [INFO ] [MAIN ] BCX: Wiping existing primary data before full refresh. [2025-10-20T17:28:49.037303] [INFO ] [LOCAL ] BCX: Starting deletion of primary data. [2025-10-20T17:28:49.040177] [WARN ] [MAIN ] BCSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:49.041475] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:28:49.041588] [DEBUG] [LOCAL ] BCX: Deleted primary data file. [2025-10-20T17:28:49.047412] [INFO ] [MAIN ] BCX: Starting full primary refresh. [2025-10-20T17:28:49.059821] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-09. [2025-10-20T17:28:49.047614] [INFO ] [MAIN ] BCSF: Wiping existing primary data before full refresh. [2025-10-20T17:28:49.150160] [INFO ] [LOCAL ] BCSF: Starting deletion of primary data. [2025-10-20T17:28:49.152935] [DEBUG] [LOCAL ] BCSF: Deleted primary data file. [2025-10-20T17:28:49.158317] [INFO ] [MAIN ] BCSF: Starting full primary refresh. [2025-10-20T17:28:49.718073] [INFO ] [REMOTE ] BCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:49.800236] [INFO ] [REMOTE ] BCSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:49.851331] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-01. [2025-10-20T17:28:50.114238] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:28:50.223589] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:28:50.651385] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-01. [2025-10-20T17:28:51.117782] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:28:51.451374] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-02. [2025-10-20T17:28:52.168086] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:28:52.251368] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-02. [2025-10-20T17:28:52.700851] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:28:53.051372] [INFO ] [REMOTE ] BCH: Fetching primary data for 2025-10. [2025-10-20T17:28:53.303183] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T17:28:53.851402] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-03. [2025-10-20T17:28:54.232721] [INFO ] [REMOTE ] BCH: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T17:28:54.238131] [INFO ] [REMOTE ] BCH: Full fetch complete (42007 records). [2025-10-20T17:28:54.238222] [DEBUG] [VALIDATOR ] BCH: Validating primary data. [2025-10-20T17:28:54.614457] [DEBUG] [VALIDATOR ] BCH: All expected trading days are present. [2025-10-20T17:28:54.614567] [DEBUG] [VALIDATOR ] BCH: Primary data validation passed. [2025-10-20T17:28:54.615093] [INFO ] [MAIN ] BCH: Fetched primary data is valid. [2025-10-20T17:28:54.615157] [DEBUG] [LOCAL ] BCH: Saving primary data. [2025-10-20T17:28:54.616106] [DEBUG] [LOCAL ] BCH: Trimmed 42007 records to 30410 (keeping 200 days of data) [2025-10-20T17:28:54.662593] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-03. [2025-10-20T17:28:54.791027] [DEBUG] [LOCAL ] BCH: Primary data saved (30410 records). [2025-10-20T17:28:54.791173] [INFO ] [MAIN ] BCH: Full history saved. [2025-10-20T17:28:54.791505] [INFO ] [MAIN ] Update Progress: 236/2367 symbols completed. BCH took 37.9s. Estimated remaining: 4h 20m 25s. Rate: 8.2 symbols/min [2025-10-20T17:28:54.959988] [INFO ] [MAIN ] BDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:28:54.975219] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:28:55.002667] [DEBUG] [LOCAL ] BDC: Loaded 24777 existing records from primary storage. [2025-10-20T17:28:55.002995] [INFO ] [MAIN ] BDC: Loaded 24777 existing records. [2025-10-20T17:28:55.176097] [WARN ] [MAIN ] BDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:28:55.176313] [INFO ] [MAIN ] BDC: Wiping existing primary data before full refresh. [2025-10-20T17:28:55.176358] [INFO ] [LOCAL ] BDC: Starting deletion of primary data. [2025-10-20T17:28:55.176578] [DEBUG] [LOCAL ] BDC: Deleted primary data file. [2025-10-20T17:28:55.176634] [INFO ] [MAIN ] BDC: Starting full primary refresh. [2025-10-20T17:28:55.453420] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-03. [2025-10-20T17:28:55.532439] [INFO ] [REMOTE ] BDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:28:55.981372] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T17:28:56.251364] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-04. [2025-10-20T17:28:57.051452] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-04. [2025-10-20T17:28:57.112525] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T17:28:57.464703] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:28:57.851396] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-05. [2025-10-20T17:28:58.392260] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:28:58.651350] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-01. [2025-10-20T17:28:59.041682] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T17:28:59.451359] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-04. [2025-10-20T17:28:59.944226] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T17:29:00.251425] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-06. [2025-10-20T17:29:01.051440] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-05. [2025-10-20T17:29:01.681753] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T17:29:01.732928] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T17:29:01.851362] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-02. [2025-10-20T17:29:02.148907] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T17:29:02.651453] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-06. [2025-10-20T17:29:02.727862] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:29:03.451473] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-07. [2025-10-20T17:29:03.932100] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T17:29:04.251355] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-07. [2025-10-20T17:29:04.604777] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:29:05.051349] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-05. [2025-10-20T17:29:05.130105] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T17:29:05.851424] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-08. [2025-10-20T17:29:06.591532] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T17:29:06.651343] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-03. [2025-10-20T17:29:06.957795] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:07.451393] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-09. [2025-10-20T17:29:07.578165] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T17:29:08.251381] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-08. [2025-10-20T17:29:08.422233] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:29:09.051432] [INFO ] [REMOTE ] BCX: Fetching primary data for 2025-10. [2025-10-20T17:29:09.359845] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:09.851375] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-04. [2025-10-20T17:29:10.362623] [INFO ] [REMOTE ] BCX: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:29:10.366858] [INFO ] [REMOTE ] BCX: Full fetch complete (33678 records). [2025-10-20T17:29:10.366935] [DEBUG] [VALIDATOR ] BCX: Validating primary data. [2025-10-20T17:29:10.658121] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-06. [2025-10-20T17:29:10.752225] [DEBUG] [VALIDATOR ] BCX: All expected trading days are present. [2025-10-20T17:29:10.752304] [DEBUG] [VALIDATOR ] BCX: Primary data validation passed. [2025-10-20T17:29:10.752787] [INFO ] [MAIN ] BCX: Fetched primary data is valid. [2025-10-20T17:29:10.752861] [DEBUG] [LOCAL ] BCX: Saving primary data. [2025-10-20T17:29:10.753587] [DEBUG] [LOCAL ] BCX: Trimmed 33678 records to 22104 (keeping 200 days of data) [2025-10-20T17:29:10.881816] [DEBUG] [LOCAL ] BCX: Primary data saved (22104 records). [2025-10-20T17:29:10.881938] [INFO ] [MAIN ] BCX: Full history saved. [2025-10-20T17:29:10.882010] [INFO ] [MAIN ] Update Progress: 237/2367 symbols completed. BCX took 22.5s. Estimated remaining: 4h 21m 36s. Rate: 8.1 symbols/min [2025-10-20T17:29:11.143219] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T17:29:11.451388] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-05. [2025-10-20T17:29:12.251375] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-09. [2025-10-20T17:29:12.578858] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:29:12.640166] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T17:29:13.051424] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-07. [2025-10-20T17:29:13.758694] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:29:13.851398] [INFO ] [REMOTE ] BCSF: Fetching primary data for 2025-10. [2025-10-20T17:29:14.651403] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-06. [2025-10-20T17:29:15.059864] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:29:15.157221] [INFO ] [REMOTE ] BCSF: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:29:15.162660] [INFO ] [REMOTE ] BCSF: Full fetch complete (39744 records). [2025-10-20T17:29:15.162755] [DEBUG] [VALIDATOR ] BCSF: Validating primary data. [2025-10-20T17:29:15.451602] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-08. [2025-10-20T17:29:15.540969] [DEBUG] [VALIDATOR ] BCSF: All expected trading days are present. [2025-10-20T17:29:15.541104] [DEBUG] [VALIDATOR ] BCSF: Primary data validation passed. [2025-10-20T17:29:15.541721] [INFO ] [MAIN ] BCSF: Fetched primary data is valid. [2025-10-20T17:29:15.541781] [DEBUG] [LOCAL ] BCSF: Saving primary data. [2025-10-20T17:29:15.542714] [DEBUG] [LOCAL ] BCSF: Trimmed 39744 records to 28311 (keeping 200 days of data) [2025-10-20T17:29:15.711144] [DEBUG] [LOCAL ] BCSF: Primary data saved (28311 records). [2025-10-20T17:29:15.711277] [INFO ] [MAIN ] BCSF: Full history saved. [2025-10-20T17:29:15.711352] [INFO ] [MAIN ] Update Progress: 238/2367 symbols completed. BCSF took 27.5s. Estimated remaining: 4h 21m 6s. Rate: 8.2 symbols/min [2025-10-20T17:29:15.791134] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:29:16.251459] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-07. [2025-10-20T17:29:17.365470] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:29:17.377299] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-09. [2025-10-20T17:29:17.465587] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T17:29:17.851495] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-08. [2025-10-20T17:29:18.966396] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:29:18.972316] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-09. [2025-10-20T17:29:19.290834] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T17:29:19.451428] [INFO ] [REMOTE ] BCS: Fetching primary data for 2025-10. [2025-10-20T17:29:20.084195] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T17:29:20.251398] [INFO ] [REMOTE ] BDC: Fetching primary data for 2025-10. [2025-10-20T17:29:21.221559] [INFO ] [REMOTE ] BCS: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T17:29:21.229000] [INFO ] [REMOTE ] BCS: Full fetch complete (77131 records). [2025-10-20T17:29:21.229163] [DEBUG] [VALIDATOR ] BCS: Validating primary data. [2025-10-20T17:29:21.229997] [FATAL] [VALIDATOR ] BCS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:29:21.230091] [FATAL] [MAIN ] BCS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:29:21.230135] [WARN ] [MAIN ] BCS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:29:21.230199] [INFO ] [LOCAL ] BCS: Starting deletion of primary data. [2025-10-20T17:29:21.230279] [DEBUG] [LOCAL ] BCS: No primary data file found to delete. [2025-10-20T17:29:21.230330] [INFO ] [LOCAL ] BCS: Starting deletion of derivative data. [2025-10-20T17:29:21.230535] [DEBUG] [LOCAL ] BCS: Deleted derivative data file. [2025-10-20T17:29:21.230681] [INFO ] [MAIN ] Update Progress: 239/2367 symbols completed. BCS took 45.4s. Estimated remaining: 4h 20m 42s. Rate: 8.2 symbols/min [2025-10-20T17:29:21.359947] [INFO ] [REMOTE ] BDC: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:29:21.363814] [INFO ] [REMOTE ] BDC: Full fetch complete (35779 records). [2025-10-20T17:29:21.363914] [DEBUG] [VALIDATOR ] BDC: Validating primary data. [2025-10-20T17:29:21.783918] [DEBUG] [VALIDATOR ] BDC: All expected trading days are present. [2025-10-20T17:29:21.784005] [DEBUG] [VALIDATOR ] BDC: Primary data validation passed. [2025-10-20T17:29:21.784448] [INFO ] [MAIN ] BDC: Fetched primary data is valid. [2025-10-20T17:29:21.784489] [DEBUG] [LOCAL ] BDC: Saving primary data. [2025-10-20T17:29:21.785431] [DEBUG] [LOCAL ] BDC: Trimmed 35779 records to 24267 (keeping 200 days of data) [2025-10-20T17:29:21.939211] [DEBUG] [LOCAL ] BDC: Primary data saved (24267 records). [2025-10-20T17:29:21.939373] [INFO ] [MAIN ] BDC: Full history saved. [2025-10-20T17:29:21.939468] [INFO ] [MAIN ] Update Progress: 240/2367 symbols completed. BDC took 27.1s. Estimated remaining: 4h 19m 36s. Rate: 8.2 symbols/min [2025-10-20T17:29:22.578092] [INFO ] [MAIN ] BDJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.562524] [INFO ] [MAIN ] BDX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.628370] [INFO ] [MAIN ] BE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.685771] [INFO ] [MAIN ] BDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:22.686202] [DEBUG] [LOCAL ] BDN: No existing data file found in primary storage. [2025-10-20T17:29:22.686259] [WARN ] [MAIN ] BDN: No existing data found, performing full refresh instead. [2025-10-20T17:29:22.686363] [INFO ] [MAIN ] BDN: Wiping existing primary data before full refresh. [2025-10-20T17:29:22.686407] [INFO ] [LOCAL ] BDN: Starting deletion of primary data. [2025-10-20T17:29:22.686455] [DEBUG] [LOCAL ] BDN: No primary data file found to delete. [2025-10-20T17:29:22.686514] [INFO ] [MAIN ] BDN: Starting full primary refresh. [2025-10-20T17:29:22.931464] [DEBUG] [LOCAL ] BDJ: Loaded 29832 existing records from primary storage. [2025-10-20T17:29:23.043203] [INFO ] [MAIN ] BDJ: Loaded 29832 existing records. [2025-10-20T17:29:23.044213] [DEBUG] [LOCAL ] BDX: Loaded 52951 existing records from primary storage. [2025-10-20T17:29:23.104579] [INFO ] [MAIN ] BDX: Loaded 52951 existing records. [2025-10-20T17:29:23.227556] [INFO ] [REMOTE ] BDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:23.301159] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-01. [2025-10-20T17:29:23.370550] [DEBUG] [LOCAL ] BE: Loaded 54441 existing records from primary storage. [2025-10-20T17:29:23.396614] [INFO ] [MAIN ] BE: Loaded 54441 existing records. [2025-10-20T17:29:23.508447] [WARN ] [MAIN ] BDJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.542318] [INFO ] [MAIN ] BDJ: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.549430] [INFO ] [LOCAL ] BDJ: Starting deletion of primary data. [2025-10-20T17:29:23.600460] [DEBUG] [LOCAL ] BDJ: Deleted primary data file. [2025-10-20T17:29:23.612303] [INFO ] [MAIN ] BDJ: Starting full primary refresh. [2025-10-20T17:29:23.612095] [WARN ] [MAIN ] BDX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.634036] [INFO ] [MAIN ] BDX: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.656408] [INFO ] [LOCAL ] BDX: Starting deletion of primary data. [2025-10-20T17:29:23.684404] [DEBUG] [LOCAL ] BDX: Deleted primary data file. [2025-10-20T17:29:23.684523] [INFO ] [MAIN ] BDX: Starting full primary refresh. [2025-10-20T17:29:23.889041] [WARN ] [MAIN ] BE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:23.942468] [INFO ] [MAIN ] BE: Wiping existing primary data before full refresh. [2025-10-20T17:29:23.954144] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-10-20T17:29:23.990418] [DEBUG] [LOCAL ] BE: Deleted primary data file. [2025-10-20T17:29:24.015417] [INFO ] [MAIN ] BE: Starting full primary refresh. [2025-10-20T17:29:24.677136] [INFO ] [REMOTE ] BDX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.791372] [INFO ] [REMOTE ] BDJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.791942] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-01. [2025-10-20T17:29:24.819113] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-01. [2025-10-20T17:29:24.943175] [INFO ] [REMOTE ] BE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:24.943499] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-01. [2025-10-20T17:29:25.167136] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T17:29:25.176968] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-02. [2025-10-20T17:29:26.273358] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:29:26.279822] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-02. [2025-10-20T17:29:26.453128] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:29:26.463739] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-02. [2025-10-20T17:29:26.488370] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T17:29:26.651424] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-03. [2025-10-20T17:29:27.134206] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-01 (2.19 seconds). [2025-10-20T17:29:27.451403] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-02. [2025-10-20T17:29:27.521896] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T17:29:27.885825] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:29:28.179012] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:29:28.251381] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-03. [2025-10-20T17:29:29.051349] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-04. [2025-10-20T17:29:29.172746] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:29:29.851346] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-03. [2025-10-20T17:29:30.132623] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:29:30.651357] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-03. [2025-10-20T17:29:30.759303] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:29:31.011769] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T17:29:31.451393] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-05. [2025-10-20T17:29:32.251429] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-04. [2025-10-20T17:29:32.516223] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T17:29:32.760317] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:29:33.051394] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-04. [2025-10-20T17:29:33.291783] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:29:33.851419] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-06. [2025-10-20T17:29:34.651366] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-04. [2025-10-20T17:29:35.020886] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T17:29:35.451418] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-05. [2025-10-20T17:29:35.536700] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:29:36.251431] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-07. [2025-10-20T17:29:36.295159] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:29:36.410708] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T17:29:37.051372] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-06. [2025-10-20T17:29:37.851401] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-05. [2025-10-20T17:29:37.998737] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:29:38.314487] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T17:29:38.651408] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-07. [2025-10-20T17:29:39.451442] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-05. [2025-10-20T17:29:39.767796] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T17:29:40.040593] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:29:40.251433] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-08. [2025-10-20T17:29:41.051354] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-08. [2025-10-20T17:29:41.183310] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T17:29:41.496218] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:29:41.851370] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-09. [2025-10-20T17:29:42.295494] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:29:42.651390] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-06. [2025-10-20T17:29:42.972918] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:29:43.451380] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-09. [2025-10-20T17:29:44.032893] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:29:44.251379] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-07. [2025-10-20T17:29:45.051358] [INFO ] [REMOTE ] BDJ: Fetching primary data for 2025-10. [2025-10-20T17:29:45.130736] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:29:45.851377] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-06. [2025-10-20T17:29:46.182420] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T17:29:46.344340] [INFO ] [REMOTE ] BDJ: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:29:46.349126] [INFO ] [REMOTE ] BDJ: Full fetch complete (42853 records). [2025-10-20T17:29:46.349220] [DEBUG] [VALIDATOR ] BDJ: Validating primary data. [2025-10-20T17:29:46.653487] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-08. [2025-10-20T17:29:46.748926] [DEBUG] [VALIDATOR ] BDJ: All expected trading days are present. [2025-10-20T17:29:46.749006] [DEBUG] [VALIDATOR ] BDJ: Primary data validation passed. [2025-10-20T17:29:46.749555] [INFO ] [MAIN ] BDJ: Fetched primary data is valid. [2025-10-20T17:29:46.749605] [DEBUG] [LOCAL ] BDJ: Saving primary data. [2025-10-20T17:29:46.750835] [DEBUG] [LOCAL ] BDJ: Trimmed 42853 records to 29444 (keeping 200 days of data) [2025-10-20T17:29:46.922787] [DEBUG] [LOCAL ] BDJ: Primary data saved (29444 records). [2025-10-20T17:29:46.922998] [INFO ] [MAIN ] BDJ: Full history saved. [2025-10-20T17:29:46.923087] [INFO ] [MAIN ] Update Progress: 241/2367 symbols completed. BDJ took 25.0s. Estimated remaining: 4h 22m 5s. Rate: 8.1 symbols/min [2025-10-20T17:29:47.102574] [INFO ] [MAIN ] BEDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:47.102768] [DEBUG] [LOCAL ] BEDU: No existing data file found in primary storage. [2025-10-20T17:29:47.102812] [WARN ] [MAIN ] BEDU: No existing data found, performing full refresh instead. [2025-10-20T17:29:47.102923] [INFO ] [MAIN ] BEDU: Wiping existing primary data before full refresh. [2025-10-20T17:29:47.102963] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-10-20T17:29:47.103004] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-10-20T17:29:47.103037] [INFO ] [MAIN ] BEDU: Starting full primary refresh. [2025-10-20T17:29:47.469235] [INFO ] [REMOTE ] BDN: Fetching primary data for 2025-10. [2025-10-20T17:29:47.472922] [INFO ] [REMOTE ] BEDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:47.507627] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:29:48.251392] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-07. [2025-10-20T17:29:48.366452] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:29:48.922312] [INFO ] [REMOTE ] BDN: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T17:29:48.929490] [INFO ] [REMOTE ] BDN: Full fetch complete (69409 records). [2025-10-20T17:29:48.929622] [DEBUG] [VALIDATOR ] BDN: Validating primary data. [2025-10-20T17:29:49.056494] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-09. [2025-10-20T17:29:49.361542] [DEBUG] [VALIDATOR ] BDN: All expected trading days are present. [2025-10-20T17:29:49.361626] [DEBUG] [VALIDATOR ] BDN: Primary data validation passed. [2025-10-20T17:29:49.362384] [INFO ] [MAIN ] BDN: Fetched primary data is valid. [2025-10-20T17:29:49.362438] [DEBUG] [LOCAL ] BDN: Saving primary data. [2025-10-20T17:29:49.363967] [DEBUG] [LOCAL ] BDN: Trimmed 69409 records to 47639 (keeping 200 days of data) [2025-10-20T17:29:49.640869] [DEBUG] [LOCAL ] BDN: Primary data saved (47639 records). [2025-10-20T17:29:49.641014] [INFO ] [MAIN ] BDN: Full history saved. [2025-10-20T17:29:49.641086] [INFO ] [MAIN ] Update Progress: 242/2367 symbols completed. BDN took 27.7s. Estimated remaining: 4h 21m 16s. Rate: 8.1 symbols/min [2025-10-20T17:29:49.844521] [INFO ] [MAIN ] BEKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:49.851394] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-01. [2025-10-20T17:29:49.910030] [DEBUG] [LOCAL ] BEKE: Loaded 53980 existing records from primary storage. [2025-10-20T17:29:49.910540] [INFO ] [MAIN ] BEKE: Loaded 53980 existing records. [2025-10-20T17:29:50.176075] [WARN ] [MAIN ] BEKE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:50.176744] [INFO ] [MAIN ] BEKE: Wiping existing primary data before full refresh. [2025-10-20T17:29:50.176938] [INFO ] [LOCAL ] BEKE: Starting deletion of primary data. [2025-10-20T17:29:50.177369] [DEBUG] [LOCAL ] BEKE: Deleted primary data file. [2025-10-20T17:29:50.177434] [INFO ] [MAIN ] BEKE: Starting full primary refresh. [2025-10-20T17:29:50.378499] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T17:29:50.657434] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-02. [2025-10-20T17:29:50.673361] [INFO ] [REMOTE ] BEKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:50.724087] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-10-20T17:29:50.785055] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T17:29:51.027560] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T17:29:51.451443] [INFO ] [REMOTE ] BDX: Fetching primary data for 2025-10. [2025-10-20T17:29:52.251440] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-01. [2025-10-20T17:29:52.985770] [INFO ] [REMOTE ] BDX: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:29:52.993214] [INFO ] [REMOTE ] BDX: Full fetch complete (75508 records). [2025-10-20T17:29:52.993344] [DEBUG] [VALIDATOR ] BDX: Validating primary data. [2025-10-20T17:29:53.056523] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-08. [2025-10-20T17:29:53.404230] [DEBUG] [VALIDATOR ] BDX: All expected trading days are present. [2025-10-20T17:29:53.404308] [DEBUG] [VALIDATOR ] BDX: Primary data validation passed. [2025-10-20T17:29:53.405069] [INFO ] [MAIN ] BDX: Fetched primary data is valid. [2025-10-20T17:29:53.405131] [DEBUG] [LOCAL ] BDX: Saving primary data. [2025-10-20T17:29:53.406750] [DEBUG] [LOCAL ] BDX: Trimmed 75508 records to 52166 (keeping 200 days of data) [2025-10-20T17:29:53.729475] [DEBUG] [LOCAL ] BDX: Primary data saved (52166 records). [2025-10-20T17:29:53.729601] [INFO ] [MAIN ] BDX: Full history saved. [2025-10-20T17:29:53.729699] [INFO ] [MAIN ] Update Progress: 243/2367 symbols completed. BDX took 31.8s. Estimated remaining: 4h 20m 40s. Rate: 8.1 symbols/min [2025-10-20T17:29:53.854088] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-03. [2025-10-20T17:29:53.904442] [INFO ] [MAIN ] BEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:29:53.954352] [DEBUG] [LOCAL ] BEN: Loaded 53896 existing records from primary storage. [2025-10-20T17:29:53.954697] [INFO ] [MAIN ] BEN: Loaded 53896 existing records. [2025-10-20T17:29:54.131779] [WARN ] [MAIN ] BEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:29:54.132153] [INFO ] [MAIN ] BEN: Wiping existing primary data before full refresh. [2025-10-20T17:29:54.132242] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-10-20T17:29:54.148055] [DEBUG] [LOCAL ] BEN: Deleted primary data file. [2025-10-20T17:29:54.161218] [INFO ] [MAIN ] BEN: Starting full primary refresh. [2025-10-20T17:29:54.238407] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:29:54.417823] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T17:29:54.530940] [INFO ] [REMOTE ] BEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:29:54.651338] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-01. [2025-10-20T17:29:55.451357] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-04. [2025-10-20T17:29:55.621581] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-08 (2.57 seconds). [2025-10-20T17:29:55.879819] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:29:56.251346] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-02. [2025-10-20T17:29:56.515612] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T17:29:57.051459] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-05. [2025-10-20T17:29:57.387291] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-10-20T17:29:57.851419] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-09. [2025-10-20T17:29:58.147177] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T17:29:58.651355] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-03. [2025-10-20T17:29:59.451397] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-02. [2025-10-20T17:30:00.071550] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:30:00.251421] [INFO ] [REMOTE ] BE: Fetching primary data for 2025-10. [2025-10-20T17:30:00.603361] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T17:30:00.835361] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T17:30:01.051406] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-06. [2025-10-20T17:30:01.533463] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:30:01.851371] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-07. [2025-10-20T17:30:02.104687] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-10-20T17:30:02.651389] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-08. [2025-10-20T17:30:02.857926] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-08 (0.21 seconds). [2025-10-20T17:30:03.451405] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-03. [2025-10-20T17:30:03.561632] [INFO ] [REMOTE ] BE: Incremental fetch complete for 2025-10 (3.31 seconds). [2025-10-20T17:30:03.570713] [INFO ] [REMOTE ] BE: Full fetch complete (77426 records). [2025-10-20T17:30:03.570960] [DEBUG] [VALIDATOR ] BE: Validating primary data. [2025-10-20T17:30:03.572174] [FATAL] [VALIDATOR ] BE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:30:03.572292] [FATAL] [MAIN ] BE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:03.572369] [WARN ] [MAIN ] BE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:03.572461] [INFO ] [LOCAL ] BE: Starting deletion of primary data. [2025-10-20T17:30:03.572572] [DEBUG] [LOCAL ] BE: No primary data file found to delete. [2025-10-20T17:30:03.572700] [INFO ] [LOCAL ] BE: Starting deletion of derivative data. [2025-10-20T17:30:03.572894] [DEBUG] [LOCAL ] BE: Deleted derivative data file. [2025-10-20T17:30:03.573005] [INFO ] [MAIN ] Update Progress: 244/2367 symbols completed. BE took 41.6s. Estimated remaining: 4h 20m 54s. Rate: 8.1 symbols/min [2025-10-20T17:30:03.760820] [INFO ] [MAIN ] BEP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:03.806077] [DEBUG] [LOCAL ] BEP: Loaded 39645 existing records from primary storage. [2025-10-20T17:30:03.806409] [INFO ] [MAIN ] BEP: Loaded 39645 existing records. [2025-10-20T17:30:03.990742] [WARN ] [MAIN ] BEP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:03.991021] [INFO ] [MAIN ] BEP: Wiping existing primary data before full refresh. [2025-10-20T17:30:03.991077] [INFO ] [LOCAL ] BEP: Starting deletion of primary data. [2025-10-20T17:30:03.991304] [DEBUG] [LOCAL ] BEP: Deleted primary data file. [2025-10-20T17:30:03.991360] [INFO ] [MAIN ] BEP: Starting full primary refresh. [2025-10-20T17:30:04.256694] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-04. [2025-10-20T17:30:04.363561] [INFO ] [REMOTE ] BEP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:05.051369] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-01. [2025-10-20T17:30:05.425523] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T17:30:05.851408] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-04. [2025-10-20T17:30:06.401724] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T17:30:06.651357] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-05. [2025-10-20T17:30:06.813973] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T17:30:07.451345] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-02. [2025-10-20T17:30:07.682566] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T17:30:08.251360] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-05. [2025-10-20T17:30:08.487012] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:30:08.901534] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:30:09.051375] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-09. [2025-10-20T17:30:09.368229] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T17:30:09.851415] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-06. [2025-10-20T17:30:10.268592] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-10-20T17:30:10.651397] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-03. [2025-10-20T17:30:11.451376] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-06. [2025-10-20T17:30:11.649919] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T17:30:12.062599] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:30:12.251447] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-07. [2025-10-20T17:30:13.051368] [INFO ] [REMOTE ] BEDU: Fetching primary data for 2025-10. [2025-10-20T17:30:13.224230] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T17:30:13.605575] [INFO ] [REMOTE ] BEDU: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:30:13.606931] [INFO ] [REMOTE ] BEDU: Full fetch complete (1320 records). [2025-10-20T17:30:13.607022] [DEBUG] [VALIDATOR ] BEDU: Validating primary data. [2025-10-20T17:30:13.797840] [WARN ] [VALIDATOR ] BEDU: Validation failed - insufficient historical data. Have 169 unique days, need 200 days [2025-10-20T17:30:13.798071] [FATAL] [MAIN ] BEDU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:13.798140] [WARN ] [MAIN ] BEDU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:13.798227] [INFO ] [LOCAL ] BEDU: Starting deletion of primary data. [2025-10-20T17:30:13.798289] [DEBUG] [LOCAL ] BEDU: No primary data file found to delete. [2025-10-20T17:30:13.798324] [INFO ] [LOCAL ] BEDU: Starting deletion of derivative data. [2025-10-20T17:30:13.798411] [DEBUG] [LOCAL ] BEDU: No derivative data file found to delete. [2025-10-20T17:30:13.798473] [INFO ] [MAIN ] Update Progress: 245/2367 symbols completed. BEDU took 26.9s. Estimated remaining: 4h 21m 12s. Rate: 8.1 symbols/min [2025-10-20T17:30:13.859560] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-04. [2025-10-20T17:30:13.978126] [INFO ] [MAIN ] BEPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:14.021218] [DEBUG] [LOCAL ] BEPC: Loaded 47230 existing records from primary storage. [2025-10-20T17:30:14.021556] [INFO ] [MAIN ] BEPC: Loaded 47230 existing records. [2025-10-20T17:30:14.186599] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T17:30:14.254632] [WARN ] [MAIN ] BEPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:14.254997] [INFO ] [MAIN ] BEPC: Wiping existing primary data before full refresh. [2025-10-20T17:30:14.255358] [INFO ] [LOCAL ] BEPC: Starting deletion of primary data. [2025-10-20T17:30:14.255577] [DEBUG] [LOCAL ] BEPC: Deleted primary data file. [2025-10-20T17:30:14.255616] [INFO ] [MAIN ] BEPC: Starting full primary refresh. [2025-10-20T17:30:14.629190] [INFO ] [REMOTE ] BEPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:14.651383] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-01. [2025-10-20T17:30:15.456609] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-07. [2025-10-20T17:30:15.469543] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:30:16.251448] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-08. [2025-10-20T17:30:16.580753] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:30:17.051521] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-02. [2025-10-20T17:30:17.335885] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T17:30:17.851465] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-08. [2025-10-20T17:30:18.651357] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-05. [2025-10-20T17:30:18.751590] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T17:30:19.451486] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-03. [2025-10-20T17:30:19.699684] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T17:30:20.222389] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-08 (3.97 seconds). [2025-10-20T17:30:20.256849] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-09. [2025-10-20T17:30:20.261022] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:30:20.843959] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:30:21.051389] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-06. [2025-10-20T17:30:21.851368] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-04. [2025-10-20T17:30:22.106554] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:30:22.602463] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:30:22.651397] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-07. [2025-10-20T17:30:23.451406] [INFO ] [REMOTE ] BEKE: Fetching primary data for 2025-10. [2025-10-20T17:30:23.541155] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:30:24.075602] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:30:24.251373] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-08. [2025-10-20T17:30:25.051375] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-09. [2025-10-20T17:30:25.192436] [INFO ] [REMOTE ] BEKE: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:30:25.200295] [INFO ] [REMOTE ] BEKE: Full fetch complete (77332 records). [2025-10-20T17:30:25.200439] [DEBUG] [VALIDATOR ] BEKE: Validating primary data. [2025-10-20T17:30:25.594879] [DEBUG] [VALIDATOR ] BEKE: All expected trading days are present. [2025-10-20T17:30:25.595073] [DEBUG] [VALIDATOR ] BEKE: Primary data validation passed. [2025-10-20T17:30:25.596051] [INFO ] [MAIN ] BEKE: Fetched primary data is valid. [2025-10-20T17:30:25.596136] [DEBUG] [LOCAL ] BEKE: Saving primary data. [2025-10-20T17:30:25.597706] [DEBUG] [LOCAL ] BEKE: Trimmed 77332 records to 53198 (keeping 200 days of data) [2025-10-20T17:30:25.713356] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T17:30:25.856494] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-09. [2025-10-20T17:30:25.955500] [DEBUG] [LOCAL ] BEKE: Primary data saved (53198 records). [2025-10-20T17:30:25.955679] [INFO ] [MAIN ] BEKE: Full history saved. [2025-10-20T17:30:25.955781] [INFO ] [MAIN ] Update Progress: 246/2367 symbols completed. BEKE took 36.3s. Estimated remaining: 4h 21m 45s. Rate: 8.1 symbols/min [2025-10-20T17:30:26.141313] [INFO ] [MAIN ] BEPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:26.155419] [DEBUG] [LOCAL ] BEPH: Loaded 5233 existing records from primary storage. [2025-10-20T17:30:26.155689] [INFO ] [MAIN ] BEPH: Loaded 5233 existing records. [2025-10-20T17:30:26.358604] [WARN ] [MAIN ] BEPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:26.359251] [INFO ] [MAIN ] BEPH: Wiping existing primary data before full refresh. [2025-10-20T17:30:26.359425] [INFO ] [LOCAL ] BEPH: Starting deletion of primary data. [2025-10-20T17:30:26.359616] [DEBUG] [LOCAL ] BEPH: Deleted primary data file. [2025-10-20T17:30:26.359712] [INFO ] [MAIN ] BEPH: Starting full primary refresh. [2025-10-20T17:30:26.653416] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-05. [2025-10-20T17:30:26.720912] [INFO ] [REMOTE ] BEPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:26.751271] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:30:27.416261] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:30:27.451376] [INFO ] [REMOTE ] BEN: Fetching primary data for 2025-10. [2025-10-20T17:30:27.913685] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:30:28.251426] [INFO ] [REMOTE ] BEP: Fetching primary data for 2025-10. [2025-10-20T17:30:29.051425] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-01. [2025-10-20T17:30:29.317149] [INFO ] [REMOTE ] BEN: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:30:29.324758] [INFO ] [REMOTE ] BEN: Full fetch complete (76695 records). [2025-10-20T17:30:29.324946] [DEBUG] [VALIDATOR ] BEN: Validating primary data. [2025-10-20T17:30:29.325836] [FATAL] [VALIDATOR ] BEN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:30:29.325937] [FATAL] [MAIN ] BEN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:30:29.325981] [WARN ] [MAIN ] BEN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:30:29.326021] [INFO ] [LOCAL ] BEN: Starting deletion of primary data. [2025-10-20T17:30:29.326101] [DEBUG] [LOCAL ] BEN: No primary data file found to delete. [2025-10-20T17:30:29.326137] [INFO ] [LOCAL ] BEN: Starting deletion of derivative data. [2025-10-20T17:30:29.326268] [DEBUG] [LOCAL ] BEN: Deleted derivative data file. [2025-10-20T17:30:29.326339] [INFO ] [MAIN ] Update Progress: 247/2367 symbols completed. BEN took 35.6s. Estimated remaining: 4h 21m 3s. Rate: 8.1 symbols/min [2025-10-20T17:30:29.501883] [INFO ] [MAIN ] BEPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:29.507921] [DEBUG] [LOCAL ] BEPI: Loaded 3648 existing records from primary storage. [2025-10-20T17:30:29.508139] [INFO ] [MAIN ] BEPI: Loaded 3648 existing records. [2025-10-20T17:30:29.689235] [WARN ] [MAIN ] BEPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:29.689539] [INFO ] [MAIN ] BEPI: Wiping existing primary data before full refresh. [2025-10-20T17:30:29.689593] [INFO ] [LOCAL ] BEPI: Starting deletion of primary data. [2025-10-20T17:30:29.689766] [DEBUG] [LOCAL ] BEPI: Deleted primary data file. [2025-10-20T17:30:29.689820] [INFO ] [MAIN ] BEPI: Starting full primary refresh. [2025-10-20T17:30:29.867746] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-06. [2025-10-20T17:30:29.948067] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T17:30:29.988270] [INFO ] [REMOTE ] BEP: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:30:30.032068] [INFO ] [REMOTE ] BEP: Full fetch complete (59297 records). [2025-10-20T17:30:30.042579] [DEBUG] [VALIDATOR ] BEP: Validating primary data. [2025-10-20T17:30:30.134106] [INFO ] [REMOTE ] BEPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:30.488242] [DEBUG] [VALIDATOR ] BEP: All expected trading days are present. [2025-10-20T17:30:30.488353] [DEBUG] [VALIDATOR ] BEP: Primary data validation passed. [2025-10-20T17:30:30.489020] [INFO ] [MAIN ] BEP: Fetched primary data is valid. [2025-10-20T17:30:30.489085] [DEBUG] [LOCAL ] BEP: Saving primary data. [2025-10-20T17:30:30.490323] [DEBUG] [LOCAL ] BEP: Trimmed 59297 records to 39006 (keeping 200 days of data) [2025-10-20T17:30:30.654370] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-01. [2025-10-20T17:30:30.720866] [DEBUG] [LOCAL ] BEP: Primary data saved (39006 records). [2025-10-20T17:30:30.721005] [INFO ] [MAIN ] BEP: Full history saved. [2025-10-20T17:30:30.721374] [INFO ] [MAIN ] Update Progress: 248/2367 symbols completed. BEP took 27.1s. Estimated remaining: 4h 20m 5s. Rate: 8.1 symbols/min [2025-10-20T17:30:30.887549] [INFO ] [MAIN ] BEPJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:30.893285] [DEBUG] [LOCAL ] BEPJ: Loaded 3122 existing records from primary storage. [2025-10-20T17:30:30.893476] [INFO ] [MAIN ] BEPJ: Loaded 3122 existing records. [2025-10-20T17:30:31.067140] [WARN ] [MAIN ] BEPJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:31.067370] [INFO ] [MAIN ] BEPJ: Wiping existing primary data before full refresh. [2025-10-20T17:30:31.067424] [INFO ] [LOCAL ] BEPJ: Starting deletion of primary data. [2025-10-20T17:30:31.067585] [DEBUG] [LOCAL ] BEPJ: Deleted primary data file. [2025-10-20T17:30:31.067639] [INFO ] [MAIN ] BEPJ: Starting full primary refresh. [2025-10-20T17:30:31.254896] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:30:31.430322] [INFO ] [REMOTE ] BEPJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:31.451330] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-01. [2025-10-20T17:30:31.609716] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:30:32.234861] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:30:32.251346] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-02. [2025-10-20T17:30:32.690147] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:30:33.051370] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-02. [2025-10-20T17:30:33.630705] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:30:33.851377] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-07. [2025-10-20T17:30:34.651388] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-02. [2025-10-20T17:30:35.122736] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:30:35.451409] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-03. [2025-10-20T17:30:35.577800] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:30:35.740821] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T17:30:36.251453] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-08. [2025-10-20T17:30:37.051451] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-04. [2025-10-20T17:30:37.761092] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:30:37.814246] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T17:30:37.851362] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-05. [2025-10-20T17:30:38.529723] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T17:30:38.651372] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-03. [2025-10-20T17:30:39.058806] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T17:30:39.451398] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-09. [2025-10-20T17:30:40.251387] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-03. [2025-10-20T17:30:40.708243] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:30:41.051343] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-06. [2025-10-20T17:30:41.156186] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:30:41.746837] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:30:41.851377] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-04. [2025-10-20T17:30:42.280625] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T17:30:42.651432] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-04. [2025-10-20T17:30:43.281950] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T17:30:43.451474] [INFO ] [REMOTE ] BEPC: Fetching primary data for 2025-10. [2025-10-20T17:30:44.251380] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-07. [2025-10-20T17:30:44.811490] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:30:45.051374] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-05. [2025-10-20T17:30:45.088037] [INFO ] [REMOTE ] BEPC: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:30:45.096587] [INFO ] [REMOTE ] BEPC: Full fetch complete (68882 records). [2025-10-20T17:30:45.096867] [DEBUG] [VALIDATOR ] BEPC: Validating primary data. [2025-10-20T17:30:45.521787] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T17:30:45.547634] [DEBUG] [VALIDATOR ] BEPC: All expected trading days are present. [2025-10-20T17:30:45.547781] [DEBUG] [VALIDATOR ] BEPC: Primary data validation passed. [2025-10-20T17:30:45.548817] [INFO ] [MAIN ] BEPC: Fetched primary data is valid. [2025-10-20T17:30:45.548913] [DEBUG] [LOCAL ] BEPC: Saving primary data. [2025-10-20T17:30:45.550855] [DEBUG] [LOCAL ] BEPC: Trimmed 68882 records to 46428 (keeping 200 days of data) [2025-10-20T17:30:45.854999] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-06. [2025-10-20T17:30:45.870207] [DEBUG] [LOCAL ] BEPC: Primary data saved (46428 records). [2025-10-20T17:30:45.870449] [INFO ] [MAIN ] BEPC: Full history saved. [2025-10-20T17:30:45.870600] [INFO ] [MAIN ] Update Progress: 249/2367 symbols completed. BEPC took 32.1s. Estimated remaining: 4h 21m 4s. Rate: 8.1 symbols/min [2025-10-20T17:30:46.052367] [INFO ] [MAIN ] BFAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:46.052634] [DEBUG] [LOCAL ] BFAM: No existing data file found in primary storage. [2025-10-20T17:30:46.052711] [WARN ] [MAIN ] BFAM: No existing data found, performing full refresh instead. [2025-10-20T17:30:46.052868] [INFO ] [MAIN ] BFAM: Wiping existing primary data before full refresh. [2025-10-20T17:30:46.052927] [INFO ] [LOCAL ] BFAM: Starting deletion of primary data. [2025-10-20T17:30:46.052971] [DEBUG] [LOCAL ] BFAM: No primary data file found to delete. [2025-10-20T17:30:46.053005] [INFO ] [MAIN ] BFAM: Starting full primary refresh. [2025-10-20T17:30:46.418716] [INFO ] [REMOTE ] BFAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:46.651391] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-05. [2025-10-20T17:30:46.710049] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:30:47.253374] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:30:47.451368] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-07. [2025-10-20T17:30:48.036773] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T17:30:48.251357] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-08. [2025-10-20T17:30:48.686388] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T17:30:49.051374] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-01. [2025-10-20T17:30:49.851385] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-06. [2025-10-20T17:30:50.443606] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T17:30:50.475798] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:30:50.651404] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-07. [2025-10-20T17:30:51.225772] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:30:51.451618] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-08. [2025-10-20T17:30:52.035748] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:30:52.251448] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-09. [2025-10-20T17:30:52.830581] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:30:53.051421] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-09. [2025-10-20T17:30:53.610629] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:30:53.851632] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-02. [2025-10-20T17:30:54.651411] [INFO ] [REMOTE ] BEPH: Fetching primary data for 2025-10. [2025-10-20T17:30:55.209857] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:30:55.429874] [INFO ] [REMOTE ] BEPH: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:30:55.431142] [INFO ] [REMOTE ] BEPH: Full fetch complete (7524 records). [2025-10-20T17:30:55.431214] [DEBUG] [VALIDATOR ] BEPH: Validating primary data. [2025-10-20T17:30:55.460178] [INFO ] [REMOTE ] BEPJ: Fetching primary data for 2025-10. [2025-10-20T17:30:55.793165] [DEBUG] [VALIDATOR ] BEPH: All expected trading days are present. [2025-10-20T17:30:55.793245] [DEBUG] [VALIDATOR ] BEPH: Primary data validation passed. [2025-10-20T17:30:55.793478] [INFO ] [MAIN ] BEPH: Fetched primary data is valid. [2025-10-20T17:30:55.793516] [DEBUG] [LOCAL ] BEPH: Saving primary data. [2025-10-20T17:30:55.793932] [DEBUG] [LOCAL ] BEPH: Trimmed 7524 records to 4950 (keeping 200 days of data) [2025-10-20T17:30:55.824724] [DEBUG] [LOCAL ] BEPH: Primary data saved (4950 records). [2025-10-20T17:30:55.824868] [INFO ] [MAIN ] BEPH: Full history saved. [2025-10-20T17:30:55.824948] [INFO ] [MAIN ] Update Progress: 250/2367 symbols completed. BEPH took 29.9s. Estimated remaining: 4h 21m 18s. Rate: 8.1 symbols/min [2025-10-20T17:30:55.994394] [INFO ] [MAIN ] BFH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:56.032812] [DEBUG] [LOCAL ] BFH: Loaded 42165 existing records from primary storage. [2025-10-20T17:30:56.033149] [INFO ] [MAIN ] BFH: Loaded 42165 existing records. [2025-10-20T17:30:56.116754] [INFO ] [REMOTE ] BEPJ: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T17:30:56.139892] [INFO ] [REMOTE ] BEPJ: Full fetch complete (4367 records). [2025-10-20T17:30:56.140017] [DEBUG] [VALIDATOR ] BEPJ: Validating primary data. [2025-10-20T17:30:56.282577] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-08. [2025-10-20T17:30:56.315216] [WARN ] [MAIN ] BFH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:56.328816] [INFO ] [MAIN ] BFH: Wiping existing primary data before full refresh. [2025-10-20T17:30:56.328949] [INFO ] [LOCAL ] BFH: Starting deletion of primary data. [2025-10-20T17:30:56.339927] [DEBUG] [LOCAL ] BFH: Deleted primary data file. [2025-10-20T17:30:56.345220] [INFO ] [MAIN ] BFH: Starting full primary refresh. [2025-10-20T17:30:56.906210] [DEBUG] [VALIDATOR ] BEPJ: All expected trading days are present. [2025-10-20T17:30:56.916873] [DEBUG] [VALIDATOR ] BEPJ: Primary data validation passed. [2025-10-20T17:30:56.917123] [INFO ] [MAIN ] BEPJ: Fetched primary data is valid. [2025-10-20T17:30:56.917539] [DEBUG] [LOCAL ] BEPJ: Saving primary data. [2025-10-20T17:30:56.939569] [DEBUG] [LOCAL ] BEPJ: Trimmed 4367 records to 3084 (keeping 200 days of data) [2025-10-20T17:30:57.007753] [INFO ] [REMOTE ] BFH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:57.021430] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T17:30:57.026702] [DEBUG] [LOCAL ] BEPJ: Primary data saved (3084 records). [2025-10-20T17:30:57.026771] [INFO ] [MAIN ] BEPJ: Full history saved. [2025-10-20T17:30:57.026844] [INFO ] [MAIN ] Update Progress: 251/2367 symbols completed. BEPJ took 26.3s. Estimated remaining: 4h 20m 18s. Rate: 8.1 symbols/min [2025-10-20T17:30:57.058595] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-03. [2025-10-20T17:30:57.206339] [INFO ] [MAIN ] BFK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:30:57.224259] [DEBUG] [LOCAL ] BFK: Loaded 13862 existing records from primary storage. [2025-10-20T17:30:57.224599] [INFO ] [MAIN ] BFK: Loaded 13862 existing records. [2025-10-20T17:30:57.410892] [WARN ] [MAIN ] BFK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:30:57.411190] [INFO ] [MAIN ] BFK: Wiping existing primary data before full refresh. [2025-10-20T17:30:57.411252] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-10-20T17:30:57.411435] [DEBUG] [LOCAL ] BFK: Deleted primary data file. [2025-10-20T17:30:57.411475] [INFO ] [MAIN ] BFK: Starting full primary refresh. [2025-10-20T17:30:57.792404] [INFO ] [REMOTE ] BFK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:30:57.851445] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-01. [2025-10-20T17:30:58.460420] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:30:58.618411] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:30:58.651397] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-02. [2025-10-20T17:30:59.067530] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T17:30:59.451409] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-09. [2025-10-20T17:30:59.961798] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-10-20T17:31:00.251424] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-03. [2025-10-20T17:31:01.051480] [INFO ] [REMOTE ] BEPI: Fetching primary data for 2025-10. [2025-10-20T17:31:01.310732] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T17:31:01.851382] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-04. [2025-10-20T17:31:01.923158] [INFO ] [REMOTE ] BEPI: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T17:31:01.924600] [INFO ] [REMOTE ] BEPI: Full fetch complete (4683 records). [2025-10-20T17:31:01.924773] [DEBUG] [VALIDATOR ] BEPI: Validating primary data. [2025-10-20T17:31:02.363815] [DEBUG] [VALIDATOR ] BEPI: All expected trading days are present. [2025-10-20T17:31:02.363901] [DEBUG] [VALIDATOR ] BEPI: Primary data validation passed. [2025-10-20T17:31:02.364056] [INFO ] [MAIN ] BEPI: Fetched primary data is valid. [2025-10-20T17:31:02.364096] [DEBUG] [LOCAL ] BEPI: Saving primary data. [2025-10-20T17:31:02.364562] [DEBUG] [LOCAL ] BEPI: Trimmed 4683 records to 3594 (keeping 200 days of data) [2025-10-20T17:31:02.393934] [DEBUG] [LOCAL ] BEPI: Primary data saved (3594 records). [2025-10-20T17:31:02.394107] [INFO ] [MAIN ] BEPI: Full history saved. [2025-10-20T17:31:02.394203] [INFO ] [MAIN ] Update Progress: 252/2367 symbols completed. BEPI took 33.1s. Estimated remaining: 4h 19m 54s. Rate: 8.1 symbols/min [2025-10-20T17:31:02.614575] [INFO ] [MAIN ] BFLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:02.651388] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-04. [2025-10-20T17:31:02.672157] [DEBUG] [LOCAL ] BFLY: Loaded 52605 existing records from primary storage. [2025-10-20T17:31:02.672522] [INFO ] [MAIN ] BFLY: Loaded 52605 existing records. [2025-10-20T17:31:02.881614] [WARN ] [MAIN ] BFLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:02.881856] [INFO ] [MAIN ] BFLY: Wiping existing primary data before full refresh. [2025-10-20T17:31:02.881902] [INFO ] [LOCAL ] BFLY: Starting deletion of primary data. [2025-10-20T17:31:02.882100] [DEBUG] [LOCAL ] BFLY: Deleted primary data file. [2025-10-20T17:31:02.882138] [INFO ] [MAIN ] BFLY: Starting full primary refresh. [2025-10-20T17:31:03.263569] [INFO ] [REMOTE ] BFLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:03.431763] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:31:03.451351] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-05. [2025-10-20T17:31:04.060768] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T17:31:04.251367] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-01. [2025-10-20T17:31:04.320362] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T17:31:05.051377] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-05. [2025-10-20T17:31:05.851413] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-06. [2025-10-20T17:31:05.881071] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:31:06.413179] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:31:06.536096] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:31:06.651418] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-07. [2025-10-20T17:31:07.365589] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T17:31:07.451428] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-02. [2025-10-20T17:31:08.251368] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-06. [2025-10-20T17:31:08.912900] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:31:09.051436] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-03. [2025-10-20T17:31:09.601830] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T17:31:09.851399] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-07. [2025-10-20T17:31:10.651432] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-08. [2025-10-20T17:31:10.729731] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T17:31:11.248296] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T17:31:11.451381] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-04. [2025-10-20T17:31:11.486855] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:31:12.251383] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-09. [2025-10-20T17:31:12.886613] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T17:31:12.939528] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T17:31:13.051387] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-08. [2025-10-20T17:31:13.851441] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-05. [2025-10-20T17:31:14.656518] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-01. [2025-10-20T17:31:14.673622] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T17:31:15.347830] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T17:31:15.451385] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-09. [2025-10-20T17:31:16.251381] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-06. [2025-10-20T17:31:16.632397] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:31:16.833939] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T17:31:17.051425] [INFO ] [REMOTE ] BFK: Fetching primary data for 2025-10. [2025-10-20T17:31:17.832622] [INFO ] [REMOTE ] BFK: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T17:31:17.835812] [INFO ] [REMOTE ] BFK: Full fetch complete (18924 records). [2025-10-20T17:31:17.836059] [DEBUG] [VALIDATOR ] BFK: Validating primary data. [2025-10-20T17:31:17.856577] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-02. [2025-10-20T17:31:17.860800] [FATAL] [VALIDATOR ] BFK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:31:17.862523] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T17:31:17.870960] [FATAL] [MAIN ] BFK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:31:17.871749] [WARN ] [MAIN ] BFK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:31:17.871833] [INFO ] [LOCAL ] BFK: Starting deletion of primary data. [2025-10-20T17:31:17.871947] [DEBUG] [LOCAL ] BFK: No primary data file found to delete. [2025-10-20T17:31:17.872356] [INFO ] [LOCAL ] BFK: Starting deletion of derivative data. [2025-10-20T17:31:17.872545] [DEBUG] [LOCAL ] BFK: Deleted derivative data file. [2025-10-20T17:31:17.872685] [INFO ] [MAIN ] Update Progress: 253/2367 symbols completed. BFK took 20.8s. Estimated remaining: 4h 20m 54s. Rate: 8.1 symbols/min [2025-10-20T17:31:18.042565] [INFO ] [MAIN ] BFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:18.042808] [DEBUG] [LOCAL ] BFS: No existing data file found in primary storage. [2025-10-20T17:31:18.042860] [WARN ] [MAIN ] BFS: No existing data found, performing full refresh instead. [2025-10-20T17:31:18.042970] [INFO ] [MAIN ] BFS: Wiping existing primary data before full refresh. [2025-10-20T17:31:18.043010] [INFO ] [LOCAL ] BFS: Starting deletion of primary data. [2025-10-20T17:31:18.043050] [DEBUG] [LOCAL ] BFS: No primary data file found to delete. [2025-10-20T17:31:18.043081] [INFO ] [MAIN ] BFS: Starting full primary refresh. [2025-10-20T17:31:18.396145] [INFO ] [REMOTE ] BFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:18.651467] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-07. [2025-10-20T17:31:19.396778] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:31:19.451466] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-03. [2025-10-20T17:31:20.154598] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:31:20.251362] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-08. [2025-10-20T17:31:20.948405] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T17:31:21.051385] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-01. [2025-10-20T17:31:21.289456] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T17:31:21.758748] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T17:31:21.851396] [INFO ] [REMOTE ] BFAM: Fetching primary data for 2025-10. [2025-10-20T17:31:22.651437] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-04. [2025-10-20T17:31:23.308138] [INFO ] [REMOTE ] BFAM: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T17:31:23.313689] [INFO ] [REMOTE ] BFAM: Full fetch complete (47340 records). [2025-10-20T17:31:23.313806] [DEBUG] [VALIDATOR ] BFAM: Validating primary data. [2025-10-20T17:31:23.461269] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-09. [2025-10-20T17:31:23.709224] [DEBUG] [VALIDATOR ] BFAM: All expected trading days are present. [2025-10-20T17:31:23.709342] [DEBUG] [VALIDATOR ] BFAM: Primary data validation passed. [2025-10-20T17:31:23.710106] [INFO ] [MAIN ] BFAM: Fetched primary data is valid. [2025-10-20T17:31:23.710172] [DEBUG] [LOCAL ] BFAM: Saving primary data. [2025-10-20T17:31:23.711433] [DEBUG] [LOCAL ] BFAM: Trimmed 47340 records to 33897 (keeping 200 days of data) [2025-10-20T17:31:23.918625] [DEBUG] [LOCAL ] BFAM: Primary data saved (33897 records). [2025-10-20T17:31:23.918816] [INFO ] [MAIN ] BFAM: Full history saved. [2025-10-20T17:31:23.918904] [INFO ] [MAIN ] Update Progress: 254/2367 symbols completed. BFAM took 38.0s. Estimated remaining: 4h 20m 35s. Rate: 8.1 symbols/min [2025-10-20T17:31:24.092418] [INFO ] [MAIN ] BFZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:24.107081] [DEBUG] [LOCAL ] BFZ: Loaded 12201 existing records from primary storage. [2025-10-20T17:31:24.107347] [INFO ] [MAIN ] BFZ: Loaded 12201 existing records. [2025-10-20T17:31:24.265671] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-02. [2025-10-20T17:31:24.301045] [WARN ] [MAIN ] BFZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:24.301324] [INFO ] [MAIN ] BFZ: Wiping existing primary data before full refresh. [2025-10-20T17:31:24.301416] [INFO ] [LOCAL ] BFZ: Starting deletion of primary data. [2025-10-20T17:31:24.301579] [DEBUG] [LOCAL ] BFZ: Deleted primary data file. [2025-10-20T17:31:24.301857] [INFO ] [MAIN ] BFZ: Starting full primary refresh. [2025-10-20T17:31:24.453091] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T17:31:24.710319] [INFO ] [REMOTE ] BFZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:25.025811] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T17:31:25.060384] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-05. [2025-10-20T17:31:25.065614] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:31:25.851401] [INFO ] [REMOTE ] BFH: Fetching primary data for 2025-10. [2025-10-20T17:31:26.651403] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-03. [2025-10-20T17:31:27.027488] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T17:31:27.451360] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-06. [2025-10-20T17:31:27.480418] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T17:31:27.710065] [INFO ] [REMOTE ] BFH: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:31:27.716329] [INFO ] [REMOTE ] BFH: Full fetch complete (60198 records). [2025-10-20T17:31:27.716447] [DEBUG] [VALIDATOR ] BFH: Validating primary data. [2025-10-20T17:31:28.098498] [DEBUG] [VALIDATOR ] BFH: All expected trading days are present. [2025-10-20T17:31:28.098590] [DEBUG] [VALIDATOR ] BFH: Primary data validation passed. [2025-10-20T17:31:28.099254] [INFO ] [MAIN ] BFH: Fetched primary data is valid. [2025-10-20T17:31:28.099319] [DEBUG] [LOCAL ] BFH: Saving primary data. [2025-10-20T17:31:28.100590] [DEBUG] [LOCAL ] BFH: Trimmed 60198 records to 41581 (keeping 200 days of data) [2025-10-20T17:31:28.267387] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-04. [2025-10-20T17:31:28.344619] [DEBUG] [LOCAL ] BFH: Primary data saved (41581 records). [2025-10-20T17:31:28.344776] [INFO ] [MAIN ] BFH: Full history saved. [2025-10-20T17:31:28.344856] [INFO ] [MAIN ] Update Progress: 255/2367 symbols completed. BFH took 32.5s. Estimated remaining: 4h 20m 3s. Rate: 8.1 symbols/min [2025-10-20T17:31:28.515960] [INFO ] [MAIN ] BG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:28.562028] [DEBUG] [LOCAL ] BG: Loaded 51286 existing records from primary storage. [2025-10-20T17:31:28.562335] [INFO ] [MAIN ] BG: Loaded 51286 existing records. [2025-10-20T17:31:28.753249] [WARN ] [MAIN ] BG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:28.753502] [INFO ] [MAIN ] BG: Wiping existing primary data before full refresh. [2025-10-20T17:31:28.753561] [INFO ] [LOCAL ] BG: Starting deletion of primary data. [2025-10-20T17:31:28.753779] [DEBUG] [LOCAL ] BG: Deleted primary data file. [2025-10-20T17:31:28.753835] [INFO ] [MAIN ] BG: Starting full primary refresh. [2025-10-20T17:31:29.056117] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-01. [2025-10-20T17:31:29.122915] [INFO ] [REMOTE ] BG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:29.130563] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:31:29.176399] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:31:29.851342] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-05. [2025-10-20T17:31:29.976009] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:31:30.365638] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T17:31:30.651421] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-02. [2025-10-20T17:31:31.425004] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:31:31.451386] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-07. [2025-10-20T17:31:32.251374] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-06. [2025-10-20T17:31:33.051377] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-01. [2025-10-20T17:31:33.238750] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:31:33.240863] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T17:31:33.851377] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-08. [2025-10-20T17:31:34.402359] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T17:31:34.651408] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-07. [2025-10-20T17:31:35.439300] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:31:35.451352] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-08. [2025-10-20T17:31:35.757694] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:31:35.979352] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T17:31:36.251387] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-02. [2025-10-20T17:31:37.051439] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-03. [2025-10-20T17:31:37.649616] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T17:31:37.775895] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:31:37.851423] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-03. [2025-10-20T17:31:38.651393] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-09. [2025-10-20T17:31:39.345602] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T17:31:39.451399] [INFO ] [REMOTE ] BFS: Fetching primary data for 2025-10. [2025-10-20T17:31:39.600776] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:31:40.179955] [INFO ] [REMOTE ] BFS: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T17:31:40.182217] [INFO ] [REMOTE ] BFS: Full fetch complete (13095 records). [2025-10-20T17:31:40.182295] [DEBUG] [VALIDATOR ] BFS: Validating primary data. [2025-10-20T17:31:40.256510] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-04. [2025-10-20T17:31:40.551290] [DEBUG] [VALIDATOR ] BFS: All expected trading days are present. [2025-10-20T17:31:40.551368] [DEBUG] [VALIDATOR ] BFS: Primary data validation passed. [2025-10-20T17:31:40.551692] [INFO ] [MAIN ] BFS: Fetched primary data is valid. [2025-10-20T17:31:40.551752] [DEBUG] [LOCAL ] BFS: Saving primary data. [2025-10-20T17:31:40.552243] [DEBUG] [LOCAL ] BFS: Trimmed 13095 records to 9614 (keeping 200 days of data) [2025-10-20T17:31:40.609623] [DEBUG] [LOCAL ] BFS: Primary data saved (9614 records). [2025-10-20T17:31:40.609788] [INFO ] [MAIN ] BFS: Full history saved. [2025-10-20T17:31:40.609861] [INFO ] [MAIN ] Update Progress: 256/2367 symbols completed. BFS took 22.7s. Estimated remaining: 4h 20m 36s. Rate: 8.1 symbols/min [2025-10-20T17:31:40.775874] [INFO ] [MAIN ] BGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:40.794801] [DEBUG] [LOCAL ] BGB: Loaded 18257 existing records from primary storage. [2025-10-20T17:31:40.795056] [INFO ] [MAIN ] BGB: Loaded 18257 existing records. [2025-10-20T17:31:40.968994] [WARN ] [MAIN ] BGB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:31:40.969330] [INFO ] [MAIN ] BGB: Wiping existing primary data before full refresh. [2025-10-20T17:31:40.969402] [INFO ] [LOCAL ] BGB: Starting deletion of primary data. [2025-10-20T17:31:40.969565] [DEBUG] [LOCAL ] BGB: Deleted primary data file. [2025-10-20T17:31:40.969604] [INFO ] [MAIN ] BGB: Starting full primary refresh. [2025-10-20T17:31:41.051964] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-09. [2025-10-20T17:31:41.260925] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T17:31:41.352153] [INFO ] [REMOTE ] BGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:41.851352] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-05. [2025-10-20T17:31:42.281176] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T17:31:42.651382] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-04. [2025-10-20T17:31:42.818827] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:31:43.451375] [INFO ] [REMOTE ] BFLY: Fetching primary data for 2025-10. [2025-10-20T17:31:43.979258] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:31:44.251381] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-05. [2025-10-20T17:31:45.051392] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-06. [2025-10-20T17:31:45.685802] [INFO ] [REMOTE ] BFLY: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T17:31:45.693828] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:31:45.697531] [INFO ] [REMOTE ] BFLY: Full fetch complete (75525 records). [2025-10-20T17:31:45.697960] [DEBUG] [VALIDATOR ] BFLY: Validating primary data. [2025-10-20T17:31:45.857489] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-07. [2025-10-20T17:31:46.095988] [DEBUG] [VALIDATOR ] BFLY: All expected trading days are present. [2025-10-20T17:31:46.096089] [DEBUG] [VALIDATOR ] BFLY: Primary data validation passed. [2025-10-20T17:31:46.096861] [INFO ] [MAIN ] BFLY: Fetched primary data is valid. [2025-10-20T17:31:46.096921] [DEBUG] [LOCAL ] BFLY: Saving primary data. [2025-10-20T17:31:46.099174] [DEBUG] [LOCAL ] BFLY: Trimmed 75525 records to 51857 (keeping 200 days of data) [2025-10-20T17:31:46.339928] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-10-20T17:31:46.425386] [DEBUG] [LOCAL ] BFLY: Primary data saved (51857 records). [2025-10-20T17:31:46.425535] [INFO ] [MAIN ] BFLY: Full history saved. [2025-10-20T17:31:46.425636] [INFO ] [MAIN ] Update Progress: 257/2367 symbols completed. BFLY took 44.0s. Estimated remaining: 4h 20m 16s. Rate: 8.1 symbols/min [2025-10-20T17:31:46.599196] [INFO ] [MAIN ] BGH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:31:46.599457] [DEBUG] [LOCAL ] BGH: No existing data file found in primary storage. [2025-10-20T17:31:46.599508] [WARN ] [MAIN ] BGH: No existing data found, performing full refresh instead. [2025-10-20T17:31:46.599660] [INFO ] [MAIN ] BGH: Wiping existing primary data before full refresh. [2025-10-20T17:31:46.599717] [INFO ] [LOCAL ] BGH: Starting deletion of primary data. [2025-10-20T17:31:46.599763] [DEBUG] [LOCAL ] BGH: No primary data file found to delete. [2025-10-20T17:31:46.599796] [INFO ] [MAIN ] BGH: Starting full primary refresh. [2025-10-20T17:31:46.655678] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-06. [2025-10-20T17:31:46.710944] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:31:46.968078] [INFO ] [REMOTE ] BGH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:31:47.451346] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-01. [2025-10-20T17:31:48.225260] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T17:31:48.251343] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-01. [2025-10-20T17:31:48.639953] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:31:49.051412] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-02. [2025-10-20T17:31:49.104398] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:31:49.835230] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T17:31:49.851335] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-02. [2025-10-20T17:31:50.416227] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T17:31:50.651383] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-07. [2025-10-20T17:31:51.451387] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-03. [2025-10-20T17:31:52.150778] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:31:52.188147] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:31:52.251398] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-04. [2025-10-20T17:31:53.041722] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T17:31:53.051360] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-05. [2025-10-20T17:31:53.851393] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-03. [2025-10-20T17:31:53.991586] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T17:31:54.651373] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-06. [2025-10-20T17:31:54.852032] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:31:55.411392] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:31:55.451344] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-08. [2025-10-20T17:31:56.251422] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-07. [2025-10-20T17:31:57.057744] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-04. [2025-10-20T17:31:57.060251] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T17:31:57.101705] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:31:57.851417] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-08. [2025-10-20T17:31:58.232186] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T17:31:58.651345] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-08. [2025-10-20T17:31:58.728278] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:31:59.451345] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-09. [2025-10-20T17:31:59.766333] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:32:00.251423] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-09. [2025-10-20T17:32:00.897269] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:32:01.051436] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-05. [2025-10-20T17:32:01.218785] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:32:01.851379] [INFO ] [REMOTE ] BFZ: Fetching primary data for 2025-10. [2025-10-20T17:32:02.004845] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:32:02.651348] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-06. [2025-10-20T17:32:02.816933] [INFO ] [REMOTE ] BFZ: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:32:02.819097] [INFO ] [REMOTE ] BFZ: Full fetch complete (17496 records). [2025-10-20T17:32:02.819171] [DEBUG] [VALIDATOR ] BFZ: Validating primary data. [2025-10-20T17:32:03.184775] [DEBUG] [VALIDATOR ] BFZ: All expected trading days are present. [2025-10-20T17:32:03.184876] [DEBUG] [VALIDATOR ] BFZ: Primary data validation passed. [2025-10-20T17:32:03.185193] [INFO ] [MAIN ] BFZ: Fetched primary data is valid. [2025-10-20T17:32:03.185231] [DEBUG] [LOCAL ] BFZ: Saving primary data. [2025-10-20T17:32:03.185805] [DEBUG] [LOCAL ] BFZ: Trimmed 17496 records to 12050 (keeping 200 days of data) [2025-10-20T17:32:03.261289] [DEBUG] [LOCAL ] BFZ: Primary data saved (12050 records). [2025-10-20T17:32:03.261529] [INFO ] [MAIN ] BFZ: Full history saved. [2025-10-20T17:32:03.261626] [INFO ] [MAIN ] Update Progress: 258/2367 symbols completed. BFZ took 39.3s. Estimated remaining: 4h 21m 26s. Rate: 8.1 symbols/min [2025-10-20T17:32:03.431903] [INFO ] [MAIN ] BGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:03.448478] [DEBUG] [LOCAL ] BGR: Loaded 13773 existing records from primary storage. [2025-10-20T17:32:03.448759] [INFO ] [MAIN ] BGR: Loaded 13773 existing records. [2025-10-20T17:32:03.460738] [INFO ] [REMOTE ] BG: Fetching primary data for 2025-10. [2025-10-20T17:32:03.644459] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T17:32:03.663872] [WARN ] [MAIN ] BGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:03.667513] [INFO ] [MAIN ] BGR: Wiping existing primary data before full refresh. [2025-10-20T17:32:03.667882] [INFO ] [LOCAL ] BGR: Starting deletion of primary data. [2025-10-20T17:32:03.668121] [DEBUG] [LOCAL ] BGR: Deleted primary data file. [2025-10-20T17:32:03.668178] [INFO ] [MAIN ] BGR: Starting full primary refresh. [2025-10-20T17:32:04.027216] [INFO ] [REMOTE ] BGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:04.251392] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-01. [2025-10-20T17:32:05.051359] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-09. [2025-10-20T17:32:05.113988] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T17:32:05.138475] [INFO ] [REMOTE ] BG: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T17:32:05.146282] [INFO ] [REMOTE ] BG: Full fetch complete (73050 records). [2025-10-20T17:32:05.146422] [DEBUG] [VALIDATOR ] BG: Validating primary data. [2025-10-20T17:32:05.566230] [DEBUG] [VALIDATOR ] BG: All expected trading days are present. [2025-10-20T17:32:05.566335] [DEBUG] [VALIDATOR ] BG: Primary data validation passed. [2025-10-20T17:32:05.567104] [INFO ] [MAIN ] BG: Fetched primary data is valid. [2025-10-20T17:32:05.567167] [DEBUG] [LOCAL ] BG: Saving primary data. [2025-10-20T17:32:05.568867] [DEBUG] [LOCAL ] BG: Trimmed 73050 records to 50585 (keeping 200 days of data) [2025-10-20T17:32:05.867128] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-07. [2025-10-20T17:32:05.905875] [DEBUG] [LOCAL ] BG: Primary data saved (50585 records). [2025-10-20T17:32:05.906243] [INFO ] [MAIN ] BG: Full history saved. [2025-10-20T17:32:05.906339] [INFO ] [MAIN ] Update Progress: 259/2367 symbols completed. BG took 37.6s. Estimated remaining: 4h 20m 39s. Rate: 8.1 symbols/min [2025-10-20T17:32:06.093630] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:32:06.097105] [INFO ] [MAIN ] BGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:06.149612] [DEBUG] [LOCAL ] BGS: Loaded 51299 existing records from primary storage. [2025-10-20T17:32:06.149972] [INFO ] [MAIN ] BGS: Loaded 51299 existing records. [2025-10-20T17:32:06.334741] [WARN ] [MAIN ] BGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:06.334988] [INFO ] [MAIN ] BGS: Wiping existing primary data before full refresh. [2025-10-20T17:32:06.335058] [INFO ] [LOCAL ] BGS: Starting deletion of primary data. [2025-10-20T17:32:06.335266] [DEBUG] [LOCAL ] BGS: Deleted primary data file. [2025-10-20T17:32:06.335323] [INFO ] [MAIN ] BGS: Starting full primary refresh. [2025-10-20T17:32:06.665131] [INFO ] [REMOTE ] BGH: Fetching primary data for 2025-10. [2025-10-20T17:32:06.711286] [INFO ] [REMOTE ] BGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:06.991481] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:32:07.451396] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-01. [2025-10-20T17:32:07.819798] [INFO ] [REMOTE ] BGH: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:32:07.823024] [INFO ] [REMOTE ] BGH: Full fetch complete (22894 records). [2025-10-20T17:32:07.823104] [DEBUG] [VALIDATOR ] BGH: Validating primary data. [2025-10-20T17:32:08.221888] [DEBUG] [VALIDATOR ] BGH: All expected trading days are present. [2025-10-20T17:32:08.221993] [DEBUG] [VALIDATOR ] BGH: Primary data validation passed. [2025-10-20T17:32:08.222586] [INFO ] [MAIN ] BGH: Fetched primary data is valid. [2025-10-20T17:32:08.222638] [DEBUG] [LOCAL ] BGH: Saving primary data. [2025-10-20T17:32:08.223382] [DEBUG] [LOCAL ] BGH: Trimmed 22894 records to 17201 (keeping 200 days of data) [2025-10-20T17:32:08.268004] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-08. [2025-10-20T17:32:08.334343] [DEBUG] [LOCAL ] BGH: Primary data saved (17201 records). [2025-10-20T17:32:08.334541] [INFO ] [MAIN ] BGH: Full history saved. [2025-10-20T17:32:08.334629] [INFO ] [MAIN ] Update Progress: 260/2367 symbols completed. BGH took 21.9s. Estimated remaining: 4h 19m 51s. Rate: 8.1 symbols/min [2025-10-20T17:32:08.514707] [INFO ] [MAIN ] BGSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:08.524709] [DEBUG] [LOCAL ] BGSF: Loaded 8117 existing records from primary storage. [2025-10-20T17:32:08.524992] [INFO ] [MAIN ] BGSF: Loaded 8117 existing records. [2025-10-20T17:32:08.718042] [WARN ] [MAIN ] BGSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:08.718352] [INFO ] [MAIN ] BGSF: Wiping existing primary data before full refresh. [2025-10-20T17:32:08.718419] [INFO ] [LOCAL ] BGSF: Starting deletion of primary data. [2025-10-20T17:32:08.718605] [DEBUG] [LOCAL ] BGSF: Deleted primary data file. [2025-10-20T17:32:08.718694] [INFO ] [MAIN ] BGSF: Starting full primary refresh. [2025-10-20T17:32:09.062108] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-02. [2025-10-20T17:32:09.107557] [INFO ] [REMOTE ] BGSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:09.229446] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:32:09.378751] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:32:09.851391] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-09. [2025-10-20T17:32:10.134166] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T17:32:10.651348] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-03. [2025-10-20T17:32:10.807503] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:32:11.451443] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-01. [2025-10-20T17:32:11.474868] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:32:12.117511] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:32:12.251431] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-02. [2025-10-20T17:32:12.694090] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:32:13.051447] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-04. [2025-10-20T17:32:13.851388] [INFO ] [REMOTE ] BGB: Fetching primary data for 2025-10. [2025-10-20T17:32:14.089412] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:32:14.651357] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-02. [2025-10-20T17:32:15.024332] [INFO ] [REMOTE ] BGB: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T17:32:15.028473] [INFO ] [REMOTE ] BGB: Full fetch complete (26786 records). [2025-10-20T17:32:15.028614] [DEBUG] [VALIDATOR ] BGB: Validating primary data. [2025-10-20T17:32:15.395431] [DEBUG] [VALIDATOR ] BGB: All expected trading days are present. [2025-10-20T17:32:15.395505] [DEBUG] [VALIDATOR ] BGB: Primary data validation passed. [2025-10-20T17:32:15.395894] [INFO ] [MAIN ] BGB: Fetched primary data is valid. [2025-10-20T17:32:15.395954] [DEBUG] [LOCAL ] BGB: Saving primary data. [2025-10-20T17:32:15.396582] [DEBUG] [LOCAL ] BGB: Trimmed 26786 records to 17876 (keeping 200 days of data) [2025-10-20T17:32:15.472435] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-05. [2025-10-20T17:32:15.504758] [DEBUG] [LOCAL ] BGB: Primary data saved (17876 records). [2025-10-20T17:32:15.504898] [INFO ] [MAIN ] BGB: Full history saved. [2025-10-20T17:32:15.504977] [INFO ] [MAIN ] Update Progress: 261/2367 symbols completed. BGB took 34.9s. Estimated remaining: 4h 19m 42s. Rate: 8.1 symbols/min [2025-10-20T17:32:15.683093] [INFO ] [MAIN ] BGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:15.699332] [DEBUG] [LOCAL ] BGT: Loaded 16635 existing records from primary storage. [2025-10-20T17:32:15.699555] [INFO ] [MAIN ] BGT: Loaded 16635 existing records. [2025-10-20T17:32:15.883152] [WARN ] [MAIN ] BGT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:15.883554] [INFO ] [MAIN ] BGT: Wiping existing primary data before full refresh. [2025-10-20T17:32:15.883721] [INFO ] [LOCAL ] BGT: Starting deletion of primary data. [2025-10-20T17:32:15.884070] [DEBUG] [LOCAL ] BGT: Deleted primary data file. [2025-10-20T17:32:15.884255] [INFO ] [MAIN ] BGT: Starting full primary refresh. [2025-10-20T17:32:16.217392] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T17:32:16.278195] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-03. [2025-10-20T17:32:16.303303] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T17:32:16.325258] [INFO ] [REMOTE ] BGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:16.946312] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:32:17.051364] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-06. [2025-10-20T17:32:17.851380] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-01. [2025-10-20T17:32:18.006695] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:32:18.651356] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-07. [2025-10-20T17:32:19.035837] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:32:19.233090] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T17:32:19.451413] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-02. [2025-10-20T17:32:20.251431] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-08. [2025-10-20T17:32:20.357705] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:32:21.051398] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-04. [2025-10-20T17:32:21.141751] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:32:21.526228] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T17:32:21.851384] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-03. [2025-10-20T17:32:22.651353] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-09. [2025-10-20T17:32:22.722030] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:32:23.451371] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-04. [2025-10-20T17:32:23.496543] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T17:32:24.103634] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:32:24.251379] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-03. [2025-10-20T17:32:25.051440] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-05. [2025-10-20T17:32:25.851416] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-05. [2025-10-20T17:32:25.984006] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T17:32:26.036993] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:32:26.362629] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T17:32:26.651397] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-04. [2025-10-20T17:32:27.451377] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-06. [2025-10-20T17:32:28.251431] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-06. [2025-10-20T17:32:28.422448] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:32:28.465618] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:32:29.051383] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-07. [2025-10-20T17:32:29.059181] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T17:32:29.851365] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-07. [2025-10-20T17:32:30.017035] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T17:32:30.600309] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T17:32:30.651338] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-08. [2025-10-20T17:32:31.184992] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T17:32:31.451362] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-09. [2025-10-20T17:32:32.251433] [INFO ] [REMOTE ] BGR: Fetching primary data for 2025-10. [2025-10-20T17:32:32.424563] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:32:32.917235] [INFO ] [REMOTE ] BGR: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:32:32.919507] [INFO ] [REMOTE ] BGR: Full fetch complete (19891 records). [2025-10-20T17:32:32.919580] [DEBUG] [VALIDATOR ] BGR: Validating primary data. [2025-10-20T17:32:33.056465] [INFO ] [REMOTE ] BGSF: Fetching primary data for 2025-10. [2025-10-20T17:32:33.283499] [DEBUG] [VALIDATOR ] BGR: All expected trading days are present. [2025-10-20T17:32:33.283575] [DEBUG] [VALIDATOR ] BGR: Primary data validation passed. [2025-10-20T17:32:33.283912] [INFO ] [MAIN ] BGR: Fetched primary data is valid. [2025-10-20T17:32:33.283974] [DEBUG] [LOCAL ] BGR: Saving primary data. [2025-10-20T17:32:33.284506] [DEBUG] [LOCAL ] BGR: Trimmed 19891 records to 13505 (keeping 200 days of data) [2025-10-20T17:32:33.365176] [DEBUG] [LOCAL ] BGR: Primary data saved (13505 records). [2025-10-20T17:32:33.365311] [INFO ] [MAIN ] BGR: Full history saved. [2025-10-20T17:32:33.365386] [INFO ] [MAIN ] Update Progress: 262/2367 symbols completed. BGR took 30.1s. Estimated remaining: 4h 20m 59s. Rate: 8.1 symbols/min [2025-10-20T17:32:33.534790] [INFO ] [MAIN ] BGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:33.548112] [DEBUG] [LOCAL ] BGX: Loaded 11350 existing records from primary storage. [2025-10-20T17:32:33.548399] [INFO ] [MAIN ] BGX: Loaded 11350 existing records. [2025-10-20T17:32:33.721768] [WARN ] [MAIN ] BGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:33.721975] [INFO ] [MAIN ] BGX: Wiping existing primary data before full refresh. [2025-10-20T17:32:33.722022] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-10-20T17:32:33.722177] [DEBUG] [LOCAL ] BGX: Deleted primary data file. [2025-10-20T17:32:33.722215] [INFO ] [MAIN ] BGX: Starting full primary refresh. [2025-10-20T17:32:33.860022] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-08. [2025-10-20T17:32:34.079957] [INFO ] [REMOTE ] BGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:34.357135] [INFO ] [REMOTE ] BGSF: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T17:32:34.359787] [INFO ] [REMOTE ] BGSF: Full fetch complete (9556 records). [2025-10-20T17:32:34.359858] [DEBUG] [VALIDATOR ] BGSF: Validating primary data. [2025-10-20T17:32:34.658896] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-05. [2025-10-20T17:32:34.659847] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:32:34.745444] [DEBUG] [VALIDATOR ] BGSF: All expected trading days are present. [2025-10-20T17:32:34.745544] [DEBUG] [VALIDATOR ] BGSF: Primary data validation passed. [2025-10-20T17:32:34.745841] [INFO ] [MAIN ] BGSF: Fetched primary data is valid. [2025-10-20T17:32:34.745902] [DEBUG] [LOCAL ] BGSF: Saving primary data. [2025-10-20T17:32:34.746362] [DEBUG] [LOCAL ] BGSF: Trimmed 9556 records to 8128 (keeping 200 days of data) [2025-10-20T17:32:34.792017] [DEBUG] [LOCAL ] BGSF: Primary data saved (8128 records). [2025-10-20T17:32:34.792138] [INFO ] [MAIN ] BGSF: Full history saved. [2025-10-20T17:32:34.792214] [INFO ] [MAIN ] Update Progress: 263/2367 symbols completed. BGSF took 26.5s. Estimated remaining: 4h 20m 3s. Rate: 8.1 symbols/min [2025-10-20T17:32:34.963353] [INFO ] [MAIN ] BGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:34.963524] [DEBUG] [LOCAL ] BGY: No existing data file found in primary storage. [2025-10-20T17:32:34.963562] [WARN ] [MAIN ] BGY: No existing data found, performing full refresh instead. [2025-10-20T17:32:34.963699] [INFO ] [MAIN ] BGY: Wiping existing primary data before full refresh. [2025-10-20T17:32:34.963753] [INFO ] [LOCAL ] BGY: Starting deletion of primary data. [2025-10-20T17:32:34.963800] [DEBUG] [LOCAL ] BGY: No primary data file found to delete. [2025-10-20T17:32:34.963833] [INFO ] [MAIN ] BGY: Starting full primary refresh. [2025-10-20T17:32:35.321982] [INFO ] [REMOTE ] BGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:35.451352] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-01. [2025-10-20T17:32:36.251354] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-01. [2025-10-20T17:32:36.424887] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:32:36.665788] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:32:37.051413] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-02. [2025-10-20T17:32:37.140608] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:32:37.851384] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-09. [2025-10-20T17:32:38.050739] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:32:38.531129] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-10-20T17:32:38.651396] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-02. [2025-10-20T17:32:39.260755] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:32:39.451369] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-03. [2025-10-20T17:32:40.148409] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T17:32:40.251357] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-03. [2025-10-20T17:32:41.036869] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:32:41.051320] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-04. [2025-10-20T17:32:41.851348] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-06. [2025-10-20T17:32:42.165229] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:32:42.651369] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-04. [2025-10-20T17:32:43.153773] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T17:32:43.451381] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-05. [2025-10-20T17:32:43.530843] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T17:32:44.251447] [INFO ] [REMOTE ] BGT: Fetching primary data for 2025-10. [2025-10-20T17:32:44.465289] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:32:45.051344] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-06. [2025-10-20T17:32:45.236114] [INFO ] [REMOTE ] BGT: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T17:32:45.238786] [INFO ] [REMOTE ] BGT: Full fetch complete (24259 records). [2025-10-20T17:32:45.238870] [DEBUG] [VALIDATOR ] BGT: Validating primary data. [2025-10-20T17:32:45.612718] [DEBUG] [VALIDATOR ] BGT: All expected trading days are present. [2025-10-20T17:32:45.612800] [DEBUG] [VALIDATOR ] BGT: Primary data validation passed. [2025-10-20T17:32:45.613175] [INFO ] [MAIN ] BGT: Fetched primary data is valid. [2025-10-20T17:32:45.613214] [DEBUG] [LOCAL ] BGT: Saving primary data. [2025-10-20T17:32:45.613902] [DEBUG] [LOCAL ] BGT: Trimmed 24259 records to 16314 (keeping 200 days of data) [2025-10-20T17:32:45.714226] [DEBUG] [LOCAL ] BGT: Primary data saved (16314 records). [2025-10-20T17:32:45.714367] [INFO ] [MAIN ] BGT: Full history saved. [2025-10-20T17:32:45.714443] [INFO ] [MAIN ] Update Progress: 264/2367 symbols completed. BGT took 30.2s. Estimated remaining: 4h 20m 24s. Rate: 8.1 symbols/min [2025-10-20T17:32:45.860328] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-05. [2025-10-20T17:32:45.891607] [INFO ] [MAIN ] BH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:32:45.914833] [DEBUG] [LOCAL ] BH: Loaded 16442 existing records from primary storage. [2025-10-20T17:32:45.915104] [INFO ] [MAIN ] BH: Loaded 16442 existing records. [2025-10-20T17:32:46.145573] [WARN ] [MAIN ] BH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:32:46.145897] [INFO ] [MAIN ] BH: Wiping existing primary data before full refresh. [2025-10-20T17:32:46.145978] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-10-20T17:32:46.146185] [DEBUG] [LOCAL ] BH: Deleted primary data file. [2025-10-20T17:32:46.146230] [INFO ] [MAIN ] BH: Starting full primary refresh. [2025-10-20T17:32:46.337396] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:32:46.655303] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-07. [2025-10-20T17:32:46.741943] [INFO ] [REMOTE ] BH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:32:46.743724] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T17:32:47.451402] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-07. [2025-10-20T17:32:48.251376] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-01. [2025-10-20T17:32:48.710107] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T17:32:48.738231] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:32:48.838753] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T17:32:49.051394] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-02. [2025-10-20T17:32:49.460796] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:32:49.851422] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-03. [2025-10-20T17:32:50.618559] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T17:32:50.651395] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-04. [2025-10-20T17:32:51.392525] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T17:32:51.451413] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-06. [2025-10-20T17:32:52.251434] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-08. [2025-10-20T17:32:52.424976] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T17:32:53.051431] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-08. [2025-10-20T17:32:53.258896] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:32:53.851428] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-09. [2025-10-20T17:32:54.651459] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-07. [2025-10-20T17:32:54.873291] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:32:54.877455] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:32:55.350137] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:32:55.451386] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-09. [2025-10-20T17:32:56.251359] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-08. [2025-10-20T17:32:57.009488] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T17:32:57.051324] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-09. [2025-10-20T17:32:57.312421] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:32:57.642061] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:32:57.851402] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-05. [2025-10-20T17:32:58.651367] [INFO ] [REMOTE ] BGY: Fetching primary data for 2025-10. [2025-10-20T17:32:58.799461] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:32:59.451422] [INFO ] [REMOTE ] BGS: Fetching primary data for 2025-10. [2025-10-20T17:32:59.937593] [INFO ] [REMOTE ] BGY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T17:32:59.941168] [INFO ] [REMOTE ] BGY: Full fetch complete (33055 records). [2025-10-20T17:32:59.941247] [DEBUG] [VALIDATOR ] BGY: Validating primary data. [2025-10-20T17:33:00.256594] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-06. [2025-10-20T17:33:00.331715] [DEBUG] [VALIDATOR ] BGY: All expected trading days are present. [2025-10-20T17:33:00.331975] [DEBUG] [VALIDATOR ] BGY: Primary data validation passed. [2025-10-20T17:33:00.332679] [INFO ] [MAIN ] BGY: Fetched primary data is valid. [2025-10-20T17:33:00.332761] [DEBUG] [LOCAL ] BGY: Saving primary data. [2025-10-20T17:33:00.334198] [DEBUG] [LOCAL ] BGY: Trimmed 33055 records to 21632 (keeping 200 days of data) [2025-10-20T17:33:00.477600] [DEBUG] [LOCAL ] BGY: Primary data saved (21632 records). [2025-10-20T17:33:00.477806] [INFO ] [MAIN ] BGY: Full history saved. [2025-10-20T17:33:00.477906] [INFO ] [MAIN ] Update Progress: 265/2367 symbols completed. BGY took 25.7s. Estimated remaining: 4h 21m 14s. Rate: 8.0 symbols/min [2025-10-20T17:33:00.679029] [INFO ] [MAIN ] BHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:00.720750] [DEBUG] [LOCAL ] BHC: Loaded 47402 existing records from primary storage. [2025-10-20T17:33:00.721137] [INFO ] [MAIN ] BHC: Loaded 47402 existing records. [2025-10-20T17:33:00.922306] [WARN ] [MAIN ] BHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:00.922571] [INFO ] [MAIN ] BHC: Wiping existing primary data before full refresh. [2025-10-20T17:33:00.922639] [INFO ] [LOCAL ] BHC: Starting deletion of primary data. [2025-10-20T17:33:00.923155] [DEBUG] [LOCAL ] BHC: Deleted primary data file. [2025-10-20T17:33:00.923227] [INFO ] [MAIN ] BHC: Starting full primary refresh. [2025-10-20T17:33:01.056617] [INFO ] [REMOTE ] BGX: Fetching primary data for 2025-10. [2025-10-20T17:33:01.327770] [INFO ] [REMOTE ] BHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:01.467883] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:33:01.681276] [INFO ] [REMOTE ] BGS: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T17:33:01.689433] [INFO ] [REMOTE ] BGS: Full fetch complete (72666 records). [2025-10-20T17:33:01.689590] [DEBUG] [VALIDATOR ] BGS: Validating primary data. [2025-10-20T17:33:01.856466] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-01. [2025-10-20T17:33:02.132433] [DEBUG] [VALIDATOR ] BGS: All expected trading days are present. [2025-10-20T17:33:02.133184] [INFO ] [REMOTE ] BGX: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:33:02.132967] [DEBUG] [VALIDATOR ] BGS: Primary data validation passed. [2025-10-20T17:33:02.137152] [INFO ] [MAIN ] BGS: Fetched primary data is valid. [2025-10-20T17:33:02.137271] [DEBUG] [LOCAL ] BGS: Saving primary data. [2025-10-20T17:33:02.137540] [INFO ] [REMOTE ] BGX: Full fetch complete (16417 records). [2025-10-20T17:33:02.137957] [DEBUG] [VALIDATOR ] BGX: Validating primary data. [2025-10-20T17:33:02.139255] [FATAL] [VALIDATOR ] BGX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:02.139414] [FATAL] [MAIN ] BGX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:02.139525] [WARN ] [MAIN ] BGX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:02.139755] [INFO ] [LOCAL ] BGX: Starting deletion of primary data. [2025-10-20T17:33:02.139921] [DEBUG] [LOCAL ] BGX: No primary data file found to delete. [2025-10-20T17:33:02.140034] [INFO ] [LOCAL ] BGX: Starting deletion of derivative data. [2025-10-20T17:33:02.140421] [DEBUG] [LOCAL ] BGX: Deleted derivative data file. [2025-10-20T17:33:02.140540] [INFO ] [MAIN ] Update Progress: 266/2367 symbols completed. BGX took 28.8s. Estimated remaining: 4h 20m 21s. Rate: 8.1 symbols/min [2025-10-20T17:33:02.173775] [DEBUG] [LOCAL ] BGS: Trimmed 72666 records to 50571 (keeping 200 days of data) [2025-10-20T17:33:02.514226] [INFO ] [MAIN ] BHE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:02.657051] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-07. [2025-10-20T17:33:02.671917] [DEBUG] [LOCAL ] BHE: Loaded 24249 existing records from primary storage. [2025-10-20T17:33:02.677129] [INFO ] [MAIN ] BHE: Loaded 24249 existing records. [2025-10-20T17:33:02.700542] [DEBUG] [LOCAL ] BGS: Primary data saved (50571 records). [2025-10-20T17:33:02.748944] [INFO ] [MAIN ] BGS: Full history saved. [2025-10-20T17:33:02.759983] [INFO ] [MAIN ] Update Progress: 267/2367 symbols completed. BGS took 56.9s. Estimated remaining: 4h 19m 20s. Rate: 8.1 symbols/min [2025-10-20T17:33:03.013116] [WARN ] [MAIN ] BHE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:03.036601] [INFO ] [MAIN ] BHE: Wiping existing primary data before full refresh. [2025-10-20T17:33:03.037507] [INFO ] [LOCAL ] BHE: Starting deletion of primary data. [2025-10-20T17:33:03.047856] [DEBUG] [LOCAL ] BHE: Deleted primary data file. [2025-10-20T17:33:03.056896] [INFO ] [MAIN ] BHK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:03.058811] [INFO ] [MAIN ] BHE: Starting full primary refresh. [2025-10-20T17:33:03.333088] [DEBUG] [LOCAL ] BHK: Loaded 23994 existing records from primary storage. [2025-10-20T17:33:03.348634] [INFO ] [MAIN ] BHK: Loaded 23994 existing records. [2025-10-20T17:33:03.632568] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T17:33:03.636731] [INFO ] [REMOTE ] BHE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:03.657975] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-01. [2025-10-20T17:33:03.738179] [WARN ] [MAIN ] BHK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:03.738739] [INFO ] [MAIN ] BHK: Wiping existing primary data before full refresh. [2025-10-20T17:33:03.738832] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-10-20T17:33:03.739309] [DEBUG] [LOCAL ] BHK: Deleted primary data file. [2025-10-20T17:33:03.739375] [INFO ] [MAIN ] BHK: Starting full primary refresh. [2025-10-20T17:33:03.885319] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:33:04.146457] [INFO ] [REMOTE ] BHK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:04.251425] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-01. [2025-10-20T17:33:04.774565] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:33:05.051360] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-02. [2025-10-20T17:33:05.227015] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:33:05.851364] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-08. [2025-10-20T17:33:06.627104] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:33:06.651334] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-03. [2025-10-20T17:33:06.689955] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:33:07.451371] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-09. [2025-10-20T17:33:08.251443] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-02. [2025-10-20T17:33:08.343025] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:33:08.395638] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:33:08.875862] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:33:09.051359] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-04. [2025-10-20T17:33:09.851370] [INFO ] [REMOTE ] BH: Fetching primary data for 2025-10. [2025-10-20T17:33:10.651355] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-02. [2025-10-20T17:33:10.840846] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:33:10.857773] [INFO ] [REMOTE ] BH: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:33:10.860246] [INFO ] [REMOTE ] BH: Full fetch complete (19462 records). [2025-10-20T17:33:10.860328] [DEBUG] [VALIDATOR ] BH: Validating primary data. [2025-10-20T17:33:10.860833] [FATAL] [VALIDATOR ] BH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:10.860936] [FATAL] [MAIN ] BH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:10.861003] [WARN ] [MAIN ] BH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:10.861075] [INFO ] [LOCAL ] BH: Starting deletion of primary data. [2025-10-20T17:33:10.861137] [DEBUG] [LOCAL ] BH: No primary data file found to delete. [2025-10-20T17:33:10.861173] [INFO ] [LOCAL ] BH: Starting deletion of derivative data. [2025-10-20T17:33:10.861331] [DEBUG] [LOCAL ] BH: Deleted derivative data file. [2025-10-20T17:33:10.861419] [INFO ] [MAIN ] Update Progress: 268/2367 symbols completed. BH took 25.1s. Estimated remaining: 4h 19m 18s. Rate: 8.1 symbols/min [2025-10-20T17:33:11.035957] [INFO ] [MAIN ] BHP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:11.082739] [DEBUG] [LOCAL ] BHP: Loaded 54152 existing records from primary storage. [2025-10-20T17:33:11.083071] [INFO ] [MAIN ] BHP: Loaded 54152 existing records. [2025-10-20T17:33:11.265837] [WARN ] [MAIN ] BHP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:11.266113] [INFO ] [MAIN ] BHP: Wiping existing primary data before full refresh. [2025-10-20T17:33:11.266161] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-10-20T17:33:11.266606] [DEBUG] [LOCAL ] BHP: Deleted primary data file. [2025-10-20T17:33:11.266698] [INFO ] [MAIN ] BHP: Starting full primary refresh. [2025-10-20T17:33:11.458700] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-05. [2025-10-20T17:33:11.643208] [INFO ] [REMOTE ] BHP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:11.904352] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:33:12.251410] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-01. [2025-10-20T17:33:12.981396] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T17:33:13.051340] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-06. [2025-10-20T17:33:13.851366] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-03. [2025-10-20T17:33:14.272662] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:33:14.598381] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:33:14.645911] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T17:33:14.651326] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-04. [2025-10-20T17:33:15.366191] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:33:15.451382] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-05. [2025-10-20T17:33:16.251343] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-03. [2025-10-20T17:33:16.356029] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:33:17.051378] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-02. [2025-10-20T17:33:17.734346] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T17:33:17.851391] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-04. [2025-10-20T17:33:18.499015] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:33:18.651415] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-06. [2025-10-20T17:33:19.211871] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T17:33:19.451426] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-07. [2025-10-20T17:33:19.809282] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:33:20.251379] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-03. [2025-10-20T17:33:20.839954] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:33:21.051355] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-07. [2025-10-20T17:33:21.851396] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-05. [2025-10-20T17:33:22.094984] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:33:22.278239] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T17:33:22.651352] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-08. [2025-10-20T17:33:23.003530] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T17:33:23.451354] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-04. [2025-10-20T17:33:23.656635] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:33:24.251387] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-08. [2025-10-20T17:33:25.051465] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-06. [2025-10-20T17:33:25.591147] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T17:33:25.652684] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:33:25.851428] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-09. [2025-10-20T17:33:26.276118] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:33:26.651467] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-07. [2025-10-20T17:33:27.451361] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-09. [2025-10-20T17:33:27.765421] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T17:33:27.771678] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:33:28.251361] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-05. [2025-10-20T17:33:28.725018] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T17:33:29.051396] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-08. [2025-10-20T17:33:29.851364] [INFO ] [REMOTE ] BHC: Fetching primary data for 2025-10. [2025-10-20T17:33:30.040354] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T17:33:30.372977] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T17:33:30.651372] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-09. [2025-10-20T17:33:31.207026] [INFO ] [REMOTE ] BHC: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T17:33:31.213833] [INFO ] [REMOTE ] BHC: Full fetch complete (67520 records). [2025-10-20T17:33:31.213984] [DEBUG] [VALIDATOR ] BHC: Validating primary data. [2025-10-20T17:33:31.461304] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-06. [2025-10-20T17:33:31.608727] [DEBUG] [VALIDATOR ] BHC: All expected trading days are present. [2025-10-20T17:33:31.608848] [DEBUG] [VALIDATOR ] BHC: Primary data validation passed. [2025-10-20T17:33:31.609579] [INFO ] [MAIN ] BHC: Fetched primary data is valid. [2025-10-20T17:33:31.609726] [DEBUG] [LOCAL ] BHC: Saving primary data. [2025-10-20T17:33:31.611097] [DEBUG] [LOCAL ] BHC: Trimmed 67520 records to 46695 (keeping 200 days of data) [2025-10-20T17:33:31.875693] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:33:31.903047] [DEBUG] [LOCAL ] BHC: Primary data saved (46695 records). [2025-10-20T17:33:31.903358] [INFO ] [MAIN ] BHC: Full history saved. [2025-10-20T17:33:31.903456] [INFO ] [MAIN ] Update Progress: 269/2367 symbols completed. BHC took 31.4s. Estimated remaining: 4h 20m 57s. Rate: 8.0 symbols/min [2025-10-20T17:33:32.070570] [INFO ] [MAIN ] BHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:32.090710] [DEBUG] [LOCAL ] BHR: Loaded 22426 existing records from primary storage. [2025-10-20T17:33:32.090985] [INFO ] [MAIN ] BHR: Loaded 22426 existing records. [2025-10-20T17:33:32.265034] [INFO ] [REMOTE ] BHE: Fetching primary data for 2025-10. [2025-10-20T17:33:32.269944] [WARN ] [MAIN ] BHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:32.271321] [INFO ] [MAIN ] BHR: Wiping existing primary data before full refresh. [2025-10-20T17:33:32.271408] [INFO ] [LOCAL ] BHR: Starting deletion of primary data. [2025-10-20T17:33:32.271688] [DEBUG] [LOCAL ] BHR: Deleted primary data file. [2025-10-20T17:33:32.271747] [INFO ] [MAIN ] BHR: Starting full primary refresh. [2025-10-20T17:33:32.652010] [INFO ] [REMOTE ] BHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:33.051369] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-01. [2025-10-20T17:33:33.356359] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:33:33.571382] [INFO ] [REMOTE ] BHE: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T17:33:33.574890] [INFO ] [REMOTE ] BHE: Full fetch complete (36362 records). [2025-10-20T17:33:33.574970] [DEBUG] [VALIDATOR ] BHE: Validating primary data. [2025-10-20T17:33:33.855479] [INFO ] [REMOTE ] BHK: Fetching primary data for 2025-10. [2025-10-20T17:33:33.973389] [DEBUG] [VALIDATOR ] BHE: All expected trading days are present. [2025-10-20T17:33:33.973534] [DEBUG] [VALIDATOR ] BHE: Primary data validation passed. [2025-10-20T17:33:33.974084] [INFO ] [MAIN ] BHE: Fetched primary data is valid. [2025-10-20T17:33:33.974166] [DEBUG] [LOCAL ] BHE: Saving primary data. [2025-10-20T17:33:33.988305] [DEBUG] [LOCAL ] BHE: Trimmed 36362 records to 23853 (keeping 200 days of data) [2025-10-20T17:33:33.989878] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T17:33:34.134973] [DEBUG] [LOCAL ] BHE: Primary data saved (23853 records). [2025-10-20T17:33:34.135107] [INFO ] [MAIN ] BHE: Full history saved. [2025-10-20T17:33:34.135184] [INFO ] [MAIN ] Update Progress: 270/2367 symbols completed. BHE took 32.0s. Estimated remaining: 4h 20m 9s. Rate: 8.1 symbols/min [2025-10-20T17:33:34.302031] [INFO ] [MAIN ] BHV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:34.302246] [DEBUG] [LOCAL ] BHV: No existing data file found in primary storage. [2025-10-20T17:33:34.302295] [WARN ] [MAIN ] BHV: No existing data found, performing full refresh instead. [2025-10-20T17:33:34.302428] [INFO ] [MAIN ] BHV: Wiping existing primary data before full refresh. [2025-10-20T17:33:34.302472] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-10-20T17:33:34.302514] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-10-20T17:33:34.302550] [INFO ] [MAIN ] BHV: Starting full primary refresh. [2025-10-20T17:33:34.652145] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-02. [2025-10-20T17:33:34.652082] [INFO ] [REMOTE ] BHV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:34.975356] [INFO ] [REMOTE ] BHK: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T17:33:34.978806] [INFO ] [REMOTE ] BHK: Full fetch complete (30338 records). [2025-10-20T17:33:34.978877] [DEBUG] [VALIDATOR ] BHK: Validating primary data. [2025-10-20T17:33:34.979323] [FATAL] [VALIDATOR ] BHK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:34.979405] [FATAL] [MAIN ] BHK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:34.979447] [WARN ] [MAIN ] BHK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:34.979528] [INFO ] [LOCAL ] BHK: Starting deletion of primary data. [2025-10-20T17:33:34.979590] [DEBUG] [LOCAL ] BHK: No primary data file found to delete. [2025-10-20T17:33:34.979625] [INFO ] [LOCAL ] BHK: Starting deletion of derivative data. [2025-10-20T17:33:34.979792] [DEBUG] [LOCAL ] BHK: Deleted derivative data file. [2025-10-20T17:33:34.979868] [INFO ] [MAIN ] Update Progress: 271/2367 symbols completed. BHK took 32.2s. Estimated remaining: 4h 19m 10s. Rate: 8.1 symbols/min [2025-10-20T17:33:35.146283] [INFO ] [MAIN ] BHVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:35.192139] [DEBUG] [LOCAL ] BHVN: Loaded 51375 existing records from primary storage. [2025-10-20T17:33:35.192468] [INFO ] [MAIN ] BHVN: Loaded 51375 existing records. [2025-10-20T17:33:35.390893] [WARN ] [MAIN ] BHVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:35.391130] [INFO ] [MAIN ] BHVN: Wiping existing primary data before full refresh. [2025-10-20T17:33:35.391183] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-10-20T17:33:35.391615] [DEBUG] [LOCAL ] BHVN: Deleted primary data file. [2025-10-20T17:33:35.391706] [INFO ] [MAIN ] BHVN: Starting full primary refresh. [2025-10-20T17:33:35.456940] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-07. [2025-10-20T17:33:35.794039] [INFO ] [REMOTE ] BHVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:35.806169] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:33:36.251400] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-01. [2025-10-20T17:33:37.051431] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-01. [2025-10-20T17:33:37.373760] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:33:37.388017] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-10-20T17:33:37.851406] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-08. [2025-10-20T17:33:37.904458] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:33:38.651384] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-03. [2025-10-20T17:33:39.451413] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-02. [2025-10-20T17:33:39.644260] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:33:39.747038] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T17:33:39.801588] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-10-20T17:33:40.251419] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-02. [2025-10-20T17:33:41.051372] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-04. [2025-10-20T17:33:41.609630] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T17:33:41.851392] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-03. [2025-10-20T17:33:42.143695] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:33:42.651429] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-03. [2025-10-20T17:33:43.056126] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T17:33:43.405840] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:33:43.451391] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-04. [2025-10-20T17:33:44.251482] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-05. [2025-10-20T17:33:45.051462] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-04. [2025-10-20T17:33:45.223155] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T17:33:45.251962] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:33:45.407708] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T17:33:45.851432] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-05. [2025-10-20T17:33:46.651443] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-09. [2025-10-20T17:33:47.451390] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-05. [2025-10-20T17:33:47.709588] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:33:48.142896] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T17:33:48.251362] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-06. [2025-10-20T17:33:48.429026] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:33:49.051390] [INFO ] [REMOTE ] BHP: Fetching primary data for 2025-10. [2025-10-20T17:33:49.851466] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-06. [2025-10-20T17:33:49.923697] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T17:33:50.651382] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-06. [2025-10-20T17:33:50.891426] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:33:50.908106] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-06 (0.26 seconds). [2025-10-20T17:33:51.045673] [INFO ] [REMOTE ] BHP: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:33:51.053096] [INFO ] [REMOTE ] BHP: Full fetch complete (77052 records). [2025-10-20T17:33:51.053234] [DEBUG] [VALIDATOR ] BHP: Validating primary data. [2025-10-20T17:33:51.053975] [FATAL] [VALIDATOR ] BHP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:33:51.054078] [FATAL] [MAIN ] BHP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:33:51.054149] [WARN ] [MAIN ] BHP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:33:51.054244] [INFO ] [LOCAL ] BHP: Starting deletion of primary data. [2025-10-20T17:33:51.054322] [DEBUG] [LOCAL ] BHP: No primary data file found to delete. [2025-10-20T17:33:51.054358] [INFO ] [LOCAL ] BHP: Starting deletion of derivative data. [2025-10-20T17:33:51.054481] [DEBUG] [LOCAL ] BHP: Deleted derivative data file. [2025-10-20T17:33:51.054767] [INFO ] [MAIN ] Update Progress: 272/2367 symbols completed. BHP took 40.2s. Estimated remaining: 4h 20m 10s. Rate: 8.1 symbols/min [2025-10-20T17:33:51.239598] [INFO ] [MAIN ] BILL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:33:51.290797] [DEBUG] [LOCAL ] BILL: Loaded 53277 existing records from primary storage. [2025-10-20T17:33:51.291196] [INFO ] [MAIN ] BILL: Loaded 53277 existing records. [2025-10-20T17:33:51.451870] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-07. [2025-10-20T17:33:51.475059] [WARN ] [MAIN ] BILL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:33:51.475464] [INFO ] [MAIN ] BILL: Wiping existing primary data before full refresh. [2025-10-20T17:33:51.475534] [INFO ] [LOCAL ] BILL: Starting deletion of primary data. [2025-10-20T17:33:51.476060] [DEBUG] [LOCAL ] BILL: Deleted primary data file. [2025-10-20T17:33:51.476113] [INFO ] [MAIN ] BILL: Starting full primary refresh. [2025-10-20T17:33:51.827190] [INFO ] [REMOTE ] BILL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:33:52.251391] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-01. [2025-10-20T17:33:53.051416] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-07. [2025-10-20T17:33:53.176689] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:33:53.851434] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-08. [2025-10-20T17:33:53.954953] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T17:33:53.991815] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:33:54.651353] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-08. [2025-10-20T17:33:55.205712] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:33:55.451374] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-02. [2025-10-20T17:33:55.775737] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:33:56.251402] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-09. [2025-10-20T17:33:57.051382] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-07. [2025-10-20T17:33:57.163524] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T17:33:57.395231] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T17:33:57.851422] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-09. [2025-10-20T17:33:58.096957] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:33:58.651406] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-08. [2025-10-20T17:33:58.834413] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:33:59.019975] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-08 (0.37 seconds). [2025-10-20T17:33:59.451373] [INFO ] [REMOTE ] BHR: Fetching primary data for 2025-10. [2025-10-20T17:34:00.251441] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-09. [2025-10-20T17:34:00.643403] [INFO ] [REMOTE ] BHR: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T17:34:00.647437] [INFO ] [REMOTE ] BHR: Full fetch complete (31338 records). [2025-10-20T17:34:00.647515] [DEBUG] [VALIDATOR ] BHR: Validating primary data. [2025-10-20T17:34:00.808998] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T17:34:01.027806] [DEBUG] [VALIDATOR ] BHR: All expected trading days are present. [2025-10-20T17:34:01.027888] [DEBUG] [VALIDATOR ] BHR: Primary data validation passed. [2025-10-20T17:34:01.028346] [INFO ] [MAIN ] BHR: Fetched primary data is valid. [2025-10-20T17:34:01.028412] [DEBUG] [LOCAL ] BHR: Saving primary data. [2025-10-20T17:34:01.029159] [DEBUG] [LOCAL ] BHR: Trimmed 31338 records to 22056 (keeping 200 days of data) [2025-10-20T17:34:01.076307] [INFO ] [REMOTE ] BHVN: Fetching primary data for 2025-10. [2025-10-20T17:34:01.155227] [DEBUG] [LOCAL ] BHR: Primary data saved (22056 records). [2025-10-20T17:34:01.155355] [INFO ] [MAIN ] BHR: Full history saved. [2025-10-20T17:34:01.155428] [INFO ] [MAIN ] Update Progress: 273/2367 symbols completed. BHR took 29.3s. Estimated remaining: 4h 20m 23s. Rate: 8.0 symbols/min [2025-10-20T17:34:01.333337] [INFO ] [MAIN ] BIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:01.355892] [DEBUG] [LOCAL ] BIO: Loaded 24345 existing records from primary storage. [2025-10-20T17:34:01.356159] [INFO ] [MAIN ] BIO: Loaded 24345 existing records. [2025-10-20T17:34:01.528363] [WARN ] [MAIN ] BIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:01.528601] [INFO ] [MAIN ] BIO: Wiping existing primary data before full refresh. [2025-10-20T17:34:01.528678] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-10-20T17:34:01.529021] [DEBUG] [LOCAL ] BIO: Deleted primary data file. [2025-10-20T17:34:01.529076] [INFO ] [MAIN ] BIO: Starting full primary refresh. [2025-10-20T17:34:01.856541] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-03. [2025-10-20T17:34:01.892272] [INFO ] [REMOTE ] BIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:02.651445] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-01. [2025-10-20T17:34:03.277817] [INFO ] [REMOTE ] BHVN: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T17:34:03.285761] [INFO ] [REMOTE ] BHVN: Full fetch complete (70178 records). [2025-10-20T17:34:03.285959] [DEBUG] [VALIDATOR ] BHVN: Validating primary data. [2025-10-20T17:34:03.286910] [FATAL] [VALIDATOR ] BHVN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:03.287003] [FATAL] [MAIN ] BHVN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:03.287047] [WARN ] [MAIN ] BHVN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:03.287114] [INFO ] [LOCAL ] BHVN: Starting deletion of primary data. [2025-10-20T17:34:03.287178] [DEBUG] [LOCAL ] BHVN: No primary data file found to delete. [2025-10-20T17:34:03.287217] [INFO ] [LOCAL ] BHVN: Starting deletion of derivative data. [2025-10-20T17:34:03.287427] [DEBUG] [LOCAL ] BHVN: Deleted derivative data file. [2025-10-20T17:34:03.287519] [INFO ] [MAIN ] Update Progress: 274/2367 symbols completed. BHVN took 28.3s. Estimated remaining: 4h 19m 34s. Rate: 8.1 symbols/min [2025-10-20T17:34:03.459305] [INFO ] [REMOTE ] BHV: Fetching primary data for 2025-10. [2025-10-20T17:34:03.491843] [INFO ] [MAIN ] BIP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:03.526428] [DEBUG] [LOCAL ] BIP: Loaded 39830 existing records from primary storage. [2025-10-20T17:34:03.526739] [INFO ] [MAIN ] BIP: Loaded 39830 existing records. [2025-10-20T17:34:03.709701] [WARN ] [MAIN ] BIP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:03.709961] [INFO ] [MAIN ] BIP: Wiping existing primary data before full refresh. [2025-10-20T17:34:03.710031] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-10-20T17:34:03.710422] [DEBUG] [LOCAL ] BIP: Deleted primary data file. [2025-10-20T17:34:03.710539] [INFO ] [MAIN ] BIP: Starting full primary refresh. [2025-10-20T17:34:03.863899] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T17:34:03.988415] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:34:04.056374] [INFO ] [REMOTE ] BHV: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T17:34:04.073031] [INFO ] [REMOTE ] BHV: Full fetch complete (1373 records). [2025-10-20T17:34:04.078266] [DEBUG] [VALIDATOR ] BHV: Validating primary data. [2025-10-20T17:34:04.083948] [FATAL] [VALIDATOR ] BHV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:04.113431] [FATAL] [MAIN ] BHV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:04.117466] [WARN ] [MAIN ] BHV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:04.134637] [INFO ] [LOCAL ] BHV: Starting deletion of primary data. [2025-10-20T17:34:04.138766] [INFO ] [REMOTE ] BIP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:04.139167] [DEBUG] [LOCAL ] BHV: No primary data file found to delete. [2025-10-20T17:34:04.139231] [INFO ] [LOCAL ] BHV: Starting deletion of derivative data. [2025-10-20T17:34:04.139281] [DEBUG] [LOCAL ] BHV: No derivative data file found to delete. [2025-10-20T17:34:04.139341] [INFO ] [MAIN ] Update Progress: 275/2367 symbols completed. BHV took 30.0s. Estimated remaining: 4h 18m 37s. Rate: 8.1 symbols/min [2025-10-20T17:34:04.256549] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-02. [2025-10-20T17:34:04.306499] [INFO ] [MAIN ] BIPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:04.342255] [DEBUG] [LOCAL ] BIPC: Loaded 41904 existing records from primary storage. [2025-10-20T17:34:04.342544] [INFO ] [MAIN ] BIPC: Loaded 41904 existing records. [2025-10-20T17:34:04.514311] [WARN ] [MAIN ] BIPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:04.514519] [INFO ] [MAIN ] BIPC: Wiping existing primary data before full refresh. [2025-10-20T17:34:04.514566] [INFO ] [LOCAL ] BIPC: Starting deletion of primary data. [2025-10-20T17:34:04.514929] [DEBUG] [LOCAL ] BIPC: Deleted primary data file. [2025-10-20T17:34:04.514984] [INFO ] [MAIN ] BIPC: Starting full primary refresh. [2025-10-20T17:34:04.859124] [INFO ] [REMOTE ] BIPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:05.051366] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-01. [2025-10-20T17:34:05.391851] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T17:34:05.851382] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-03. [2025-10-20T17:34:06.610174] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:34:06.651417] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-04. [2025-10-20T17:34:06.838501] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:34:07.451414] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-04. [2025-10-20T17:34:08.251450] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-02. [2025-10-20T17:34:08.612166] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T17:34:08.669559] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:34:09.051438] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-01. [2025-10-20T17:34:09.671488] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:34:09.851400] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-05. [2025-10-20T17:34:10.311216] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T17:34:10.651417] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-03. [2025-10-20T17:34:11.451381] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-05. [2025-10-20T17:34:11.710269] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:34:12.017195] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:34:12.251392] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-02. [2025-10-20T17:34:12.667726] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:34:13.051427] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-04. [2025-10-20T17:34:13.501565] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:34:13.851429] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-03. [2025-10-20T17:34:14.575375] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T17:34:14.651431] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-05. [2025-10-20T17:34:15.451396] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-06. [2025-10-20T17:34:15.697712] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T17:34:16.252841] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-06. [2025-10-20T17:34:16.256279] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:34:16.744082] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:34:17.051436] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-07. [2025-10-20T17:34:17.706639] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:34:17.851411] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-04. [2025-10-20T17:34:18.228446] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T17:34:18.651380] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-08. [2025-10-20T17:34:19.195933] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:34:19.451339] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-05. [2025-10-20T17:34:19.469537] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T17:34:20.251366] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-09. [2025-10-20T17:34:20.956557] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T17:34:20.997558] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T17:34:21.051348] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-06. [2025-10-20T17:34:21.851353] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-07. [2025-10-20T17:34:22.430401] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T17:34:22.651391] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-07. [2025-10-20T17:34:23.451368] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-06. [2025-10-20T17:34:23.859919] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T17:34:24.229201] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:34:24.251395] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-08. [2025-10-20T17:34:24.627311] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T17:34:25.051378] [INFO ] [REMOTE ] BIO: Fetching primary data for 2025-10. [2025-10-20T17:34:25.430775] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T17:34:25.851393] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-09. [2025-10-20T17:34:26.149449] [INFO ] [REMOTE ] BIO: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:34:26.152502] [INFO ] [REMOTE ] BIO: Full fetch complete (34239 records). [2025-10-20T17:34:26.152633] [DEBUG] [VALIDATOR ] BIO: Validating primary data. [2025-10-20T17:34:26.153261] [FATAL] [VALIDATOR ] BIO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:26.153348] [FATAL] [MAIN ] BIO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:26.153393] [WARN ] [MAIN ] BIO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:26.153431] [INFO ] [LOCAL ] BIO: Starting deletion of primary data. [2025-10-20T17:34:26.153517] [DEBUG] [LOCAL ] BIO: No primary data file found to delete. [2025-10-20T17:34:26.153572] [INFO ] [LOCAL ] BIO: Starting deletion of derivative data. [2025-10-20T17:34:26.153881] [DEBUG] [LOCAL ] BIO: Deleted derivative data file. [2025-10-20T17:34:26.153971] [INFO ] [MAIN ] Update Progress: 276/2367 symbols completed. BIO took 25.0s. Estimated remaining: 4h 20m 20s. Rate: 8.0 symbols/min [2025-10-20T17:34:26.342344] [INFO ] [MAIN ] BIPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:26.348336] [DEBUG] [LOCAL ] BIPH: Loaded 2538 existing records from primary storage. [2025-10-20T17:34:26.348551] [INFO ] [MAIN ] BIPH: Loaded 2538 existing records. [2025-10-20T17:34:26.532242] [WARN ] [MAIN ] BIPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:26.532504] [INFO ] [MAIN ] BIPH: Wiping existing primary data before full refresh. [2025-10-20T17:34:26.532552] [INFO ] [LOCAL ] BIPH: Starting deletion of primary data. [2025-10-20T17:34:26.532734] [DEBUG] [LOCAL ] BIPH: Deleted primary data file. [2025-10-20T17:34:26.532790] [INFO ] [MAIN ] BIPH: Starting full primary refresh. [2025-10-20T17:34:26.656730] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-07. [2025-10-20T17:34:26.909556] [INFO ] [REMOTE ] BIPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:27.280245] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:34:27.451352] [INFO ] [REMOTE ] BIP: Fetching primary data for 2025-10. [2025-10-20T17:34:27.928997] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:34:28.251381] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-08. [2025-10-20T17:34:28.975798] [INFO ] [REMOTE ] BIP: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T17:34:28.982598] [INFO ] [REMOTE ] BIP: Full fetch complete (57336 records). [2025-10-20T17:34:28.982795] [DEBUG] [VALIDATOR ] BIP: Validating primary data. [2025-10-20T17:34:28.984082] [FATAL] [VALIDATOR ] BIP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:34:28.984190] [FATAL] [MAIN ] BIP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:34:28.984235] [WARN ] [MAIN ] BIP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:34:28.984280] [INFO ] [LOCAL ] BIP: Starting deletion of primary data. [2025-10-20T17:34:28.984345] [DEBUG] [LOCAL ] BIP: No primary data file found to delete. [2025-10-20T17:34:28.984382] [INFO ] [LOCAL ] BIP: Starting deletion of derivative data. [2025-10-20T17:34:28.984531] [DEBUG] [LOCAL ] BIP: Deleted derivative data file. [2025-10-20T17:34:28.984612] [INFO ] [MAIN ] Update Progress: 277/2367 symbols completed. BIP took 25.7s. Estimated remaining: 4h 19m 38s. Rate: 8.1 symbols/min [2025-10-20T17:34:29.051388] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-01. [2025-10-20T17:34:29.504461] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:34:29.851398] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-02. [2025-10-20T17:34:29.921590] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:34:30.222937] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T17:34:30.651380] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-08. [2025-10-20T17:34:31.451408] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-03. [2025-10-20T17:34:31.820468] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T17:34:32.159416] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:34:32.251415] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-09. [2025-10-20T17:34:33.051440] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-09. [2025-10-20T17:34:33.394752] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T17:34:33.851387] [INFO ] [REMOTE ] BIPC: Fetching primary data for 2025-10. [2025-10-20T17:34:34.652624] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-04. [2025-10-20T17:34:34.660282] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T17:34:35.277289] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T17:34:35.451384] [INFO ] [REMOTE ] BILL: Fetching primary data for 2025-10. [2025-10-20T17:34:35.518568] [INFO ] [REMOTE ] BIPC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:34:35.525377] [INFO ] [REMOTE ] BIPC: Full fetch complete (60293 records). [2025-10-20T17:34:35.525551] [DEBUG] [VALIDATOR ] BIPC: Validating primary data. [2025-10-20T17:34:35.934759] [DEBUG] [VALIDATOR ] BIPC: All expected trading days are present. [2025-10-20T17:34:35.934846] [DEBUG] [VALIDATOR ] BIPC: Primary data validation passed. [2025-10-20T17:34:35.935485] [INFO ] [MAIN ] BIPC: Fetched primary data is valid. [2025-10-20T17:34:35.935526] [DEBUG] [LOCAL ] BIPC: Saving primary data. [2025-10-20T17:34:35.936798] [DEBUG] [LOCAL ] BIPC: Trimmed 60293 records to 41246 (keeping 200 days of data) [2025-10-20T17:34:36.180288] [DEBUG] [LOCAL ] BIPC: Primary data saved (41246 records). [2025-10-20T17:34:36.180425] [INFO ] [MAIN ] BIPC: Full history saved. [2025-10-20T17:34:36.180499] [INFO ] [MAIN ] Update Progress: 278/2367 symbols completed. BIPC took 32.0s. Estimated remaining: 4h 19m 28s. Rate: 8.1 symbols/min [2025-10-20T17:34:36.251395] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-05. [2025-10-20T17:34:36.699169] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T17:34:37.051415] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-06. [2025-10-20T17:34:37.498775] [INFO ] [REMOTE ] BILL: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T17:34:37.506587] [INFO ] [REMOTE ] BILL: Full fetch complete (76124 records). [2025-10-20T17:34:37.506758] [DEBUG] [VALIDATOR ] BILL: Validating primary data. [2025-10-20T17:34:37.603834] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T17:34:37.856700] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-07. [2025-10-20T17:34:37.901357] [DEBUG] [VALIDATOR ] BILL: All expected trading days are present. [2025-10-20T17:34:37.901486] [DEBUG] [VALIDATOR ] BILL: Primary data validation passed. [2025-10-20T17:34:37.902374] [INFO ] [MAIN ] BILL: Fetched primary data is valid. [2025-10-20T17:34:37.902443] [DEBUG] [LOCAL ] BILL: Saving primary data. [2025-10-20T17:34:37.904573] [DEBUG] [LOCAL ] BILL: Trimmed 76124 records to 52513 (keeping 200 days of data) [2025-10-20T17:34:38.234979] [DEBUG] [LOCAL ] BILL: Primary data saved (52513 records). [2025-10-20T17:34:38.235192] [INFO ] [MAIN ] BILL: Full history saved. [2025-10-20T17:34:38.235620] [INFO ] [MAIN ] Update Progress: 279/2367 symbols completed. BILL took 47.2s. Estimated remaining: 4h 18m 40s. Rate: 8.1 symbols/min [2025-10-20T17:34:38.359230] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T17:34:38.651402] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-08. [2025-10-20T17:34:39.147818] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T17:34:39.451485] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-09. [2025-10-20T17:34:39.824020] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T17:34:40.251414] [INFO ] [REMOTE ] BIPH: Fetching primary data for 2025-10. [2025-10-20T17:34:40.656271] [INFO ] [REMOTE ] BIPH: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T17:34:40.657148] [INFO ] [REMOTE ] BIPH: Full fetch complete (3896 records). [2025-10-20T17:34:40.657216] [DEBUG] [VALIDATOR ] BIPH: Validating primary data. [2025-10-20T17:34:41.022490] [DEBUG] [VALIDATOR ] BIPH: All expected trading days are present. [2025-10-20T17:34:41.022570] [DEBUG] [VALIDATOR ] BIPH: Primary data validation passed. [2025-10-20T17:34:41.022725] [INFO ] [MAIN ] BIPH: Fetched primary data is valid. [2025-10-20T17:34:41.022785] [DEBUG] [LOCAL ] BIPH: Saving primary data. [2025-10-20T17:34:41.023129] [DEBUG] [LOCAL ] BIPH: Trimmed 3896 records to 2395 (keeping 200 days of data) [2025-10-20T17:34:41.037580] [DEBUG] [LOCAL ] BIPH: Primary data saved (2395 records). [2025-10-20T17:34:41.037684] [INFO ] [MAIN ] BIPH: Full history saved. [2025-10-20T17:34:41.037768] [INFO ] [MAIN ] Update Progress: 280/2367 symbols completed. BIPH took 14.9s. Estimated remaining: 4h 17m 58s. Rate: 8.1 symbols/min [2025-10-20T17:34:41.715123] [INFO ] [MAIN ] BIPJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.753614] [INFO ] [MAIN ] BIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.762329] [INFO ] [MAIN ] BIPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.840542] [INFO ] [MAIN ] BIRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:34:41.845985] [DEBUG] [LOCAL ] BIPJ: Loaded 3197 existing records from primary storage. [2025-10-20T17:34:41.849781] [INFO ] [MAIN ] BIPJ: Loaded 3197 existing records. [2025-10-20T17:34:41.849460] [DEBUG] [LOCAL ] BIPI: Loaded 3038 existing records from primary storage. [2025-10-20T17:34:41.863956] [INFO ] [MAIN ] BIPI: Loaded 3038 existing records. [2025-10-20T17:34:42.096042] [DEBUG] [LOCAL ] BIT: Loaded 24078 existing records from primary storage. [2025-10-20T17:34:42.117967] [INFO ] [MAIN ] BIT: Loaded 24078 existing records. [2025-10-20T17:34:42.303679] [WARN ] [MAIN ] BIPJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.334864] [WARN ] [MAIN ] BIPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.335149] [INFO ] [MAIN ] BIPJ: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.361511] [INFO ] [LOCAL ] BIPJ: Starting deletion of primary data. [2025-10-20T17:34:42.356213] [INFO ] [MAIN ] BIPI: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.361906] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-10-20T17:34:42.367216] [DEBUG] [LOCAL ] BIPJ: Deleted primary data file. [2025-10-20T17:34:42.382418] [DEBUG] [LOCAL ] BIPI: Deleted primary data file. [2025-10-20T17:34:42.367448] [INFO ] [MAIN ] BIPJ: Starting full primary refresh. [2025-10-20T17:34:42.383730] [INFO ] [MAIN ] BIPI: Starting full primary refresh. [2025-10-20T17:34:42.467299] [DEBUG] [LOCAL ] BIRK: Loaded 51882 existing records from primary storage. [2025-10-20T17:34:42.657750] [INFO ] [MAIN ] BIRK: Loaded 51882 existing records. [2025-10-20T17:34:42.793173] [WARN ] [MAIN ] BIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:42.829200] [INFO ] [MAIN ] BIT: Wiping existing primary data before full refresh. [2025-10-20T17:34:42.856853] [INFO ] [LOCAL ] BIT: Starting deletion of primary data. [2025-10-20T17:34:42.877931] [DEBUG] [LOCAL ] BIT: Deleted primary data file. [2025-10-20T17:34:42.903922] [INFO ] [MAIN ] BIT: Starting full primary refresh. [2025-10-20T17:34:43.434702] [WARN ] [MAIN ] BIRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:34:43.491043] [INFO ] [MAIN ] BIRK: Wiping existing primary data before full refresh. [2025-10-20T17:34:43.530889] [INFO ] [LOCAL ] BIRK: Starting deletion of primary data. [2025-10-20T17:34:43.664083] [DEBUG] [LOCAL ] BIRK: Deleted primary data file. [2025-10-20T17:34:43.691268] [INFO ] [REMOTE ] BIPJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:43.740372] [INFO ] [REMOTE ] BIPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:43.763502] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-01. [2025-10-20T17:34:43.764180] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-01. [2025-10-20T17:34:43.763676] [INFO ] [MAIN ] BIRK: Starting full primary refresh. [2025-10-20T17:34:44.150700] [INFO ] [REMOTE ] BIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:44.169445] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-01. [2025-10-20T17:34:44.367909] [INFO ] [REMOTE ] BIRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:34:44.368189] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:34:44.368823] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-01. [2025-10-20T17:34:44.370601] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-02. [2025-10-20T17:34:44.781557] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:34:44.847140] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:34:45.051420] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-02. [2025-10-20T17:34:45.301615] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T17:34:45.647093] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T17:34:45.662395] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:34:45.851399] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-02. [2025-10-20T17:34:46.651376] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-03. [2025-10-20T17:34:46.973168] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:34:47.230193] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:34:47.451392] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-03. [2025-10-20T17:34:47.871259] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T17:34:48.251414] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-03. [2025-10-20T17:34:49.051363] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-04. [2025-10-20T17:34:49.457958] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T17:34:49.583362] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:34:49.851392] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-04. [2025-10-20T17:34:50.651397] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-02. [2025-10-20T17:34:51.463027] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-05. [2025-10-20T17:34:51.484554] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:34:51.496008] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:34:51.774187] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T17:34:52.251376] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-03. [2025-10-20T17:34:53.051443] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-05. [2025-10-20T17:34:53.262874] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:34:53.851402] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-04. [2025-10-20T17:34:54.471223] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:34:54.651418] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-04. [2025-10-20T17:34:54.805007] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T17:34:55.055051] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-04 (0.40 seconds). [2025-10-20T17:34:55.451417] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-05. [2025-10-20T17:34:55.880349] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T17:34:56.251446] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-06. [2025-10-20T17:34:56.867685] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:34:57.051445] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-05. [2025-10-20T17:34:57.851375] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-06. [2025-10-20T17:34:58.092669] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:34:58.651397] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-06. [2025-10-20T17:34:59.103231] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T17:34:59.183355] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:34:59.451388] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-06. [2025-10-20T17:35:00.056434] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T17:35:00.251364] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-07. [2025-10-20T17:35:01.051351] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-07. [2025-10-20T17:35:01.697069] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T17:35:01.851367] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-07. [2025-10-20T17:35:02.092282] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:35:02.413668] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:35:02.651362] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-07. [2025-10-20T17:35:03.451404] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-08. [2025-10-20T17:35:03.469736] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T17:35:04.044085] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T17:35:04.251447] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-08. [2025-10-20T17:35:05.051432] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-08. [2025-10-20T17:35:05.747949] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:35:05.746165] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:35:05.851380] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-08. [2025-10-20T17:35:06.651393] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-09. [2025-10-20T17:35:07.451388] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-09. [2025-10-20T17:35:08.251454] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-09. [2025-10-20T17:35:08.966175] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:35:09.051408] [INFO ] [REMOTE ] BIPJ: Fetching primary data for 2025-10. [2025-10-20T17:35:09.266620] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:35:09.562795] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-08 (3.71 seconds). [2025-10-20T17:35:09.600544] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-09 (2.95 seconds). [2025-10-20T17:35:09.603442] [INFO ] [REMOTE ] BIPJ: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:35:09.612160] [INFO ] [REMOTE ] BIPJ: Full fetch complete (4204 records). [2025-10-20T17:35:09.612235] [DEBUG] [VALIDATOR ] BIPJ: Validating primary data. [2025-10-20T17:35:09.856584] [INFO ] [REMOTE ] BIPI: Fetching primary data for 2025-10. [2025-10-20T17:35:10.003283] [DEBUG] [VALIDATOR ] BIPJ: All expected trading days are present. [2025-10-20T17:35:10.003519] [DEBUG] [VALIDATOR ] BIPJ: Primary data validation passed. [2025-10-20T17:35:10.003914] [INFO ] [MAIN ] BIPJ: Fetched primary data is valid. [2025-10-20T17:35:10.004032] [DEBUG] [LOCAL ] BIPJ: Saving primary data. [2025-10-20T17:35:10.004925] [DEBUG] [LOCAL ] BIPJ: Trimmed 4204 records to 3133 (keeping 200 days of data) [2025-10-20T17:35:10.023638] [DEBUG] [LOCAL ] BIPJ: Primary data saved (3133 records). [2025-10-20T17:35:10.023732] [INFO ] [MAIN ] BIPJ: Full history saved. [2025-10-20T17:35:10.023816] [INFO ] [MAIN ] Update Progress: 281/2367 symbols completed. BIPJ took 29.0s. Estimated remaining: 4h 20m 31s. Rate: 8.0 symbols/min [2025-10-20T17:35:10.189244] [INFO ] [MAIN ] BJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:10.235562] [DEBUG] [LOCAL ] BJ: Loaded 52353 existing records from primary storage. [2025-10-20T17:35:10.235894] [INFO ] [MAIN ] BJ: Loaded 52353 existing records. [2025-10-20T17:35:10.409072] [WARN ] [MAIN ] BJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:10.409300] [INFO ] [MAIN ] BJ: Wiping existing primary data before full refresh. [2025-10-20T17:35:10.409346] [INFO ] [LOCAL ] BJ: Starting deletion of primary data. [2025-10-20T17:35:10.409931] [DEBUG] [LOCAL ] BJ: Deleted primary data file. [2025-10-20T17:35:10.409986] [INFO ] [MAIN ] BJ: Starting full primary refresh. [2025-10-20T17:35:10.494775] [INFO ] [REMOTE ] BIPI: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T17:35:10.517960] [INFO ] [REMOTE ] BIPI: Full fetch complete (4623 records). [2025-10-20T17:35:10.524089] [DEBUG] [VALIDATOR ] BIPI: Validating primary data. [2025-10-20T17:35:10.545541] [FATAL] [VALIDATOR ] BIPI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:35:10.568845] [FATAL] [MAIN ] BIPI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:35:10.579051] [WARN ] [MAIN ] BIPI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:35:10.628961] [INFO ] [LOCAL ] BIPI: Starting deletion of primary data. [2025-10-20T17:35:10.664438] [DEBUG] [LOCAL ] BIPI: No primary data file found to delete. [2025-10-20T17:35:10.658932] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-09. [2025-10-20T17:35:10.688239] [INFO ] [LOCAL ] BIPI: Starting deletion of derivative data. [2025-10-20T17:35:10.700714] [DEBUG] [LOCAL ] BIPI: Deleted derivative data file. [2025-10-20T17:35:10.722040] [INFO ] [MAIN ] Update Progress: 282/2367 symbols completed. BIPI took 29.7s. Estimated remaining: 4h 19m 33s. Rate: 8.0 symbols/min [2025-10-20T17:35:10.842378] [INFO ] [REMOTE ] BJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:10.985710] [INFO ] [MAIN ] BK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:11.042297] [DEBUG] [LOCAL ] BK: Loaded 54323 existing records from primary storage. [2025-10-20T17:35:11.042887] [INFO ] [MAIN ] BK: Loaded 54323 existing records. [2025-10-20T17:35:11.245056] [WARN ] [MAIN ] BK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:11.245365] [INFO ] [MAIN ] BK: Wiping existing primary data before full refresh. [2025-10-20T17:35:11.245415] [INFO ] [LOCAL ] BK: Starting deletion of primary data. [2025-10-20T17:35:11.245619] [DEBUG] [LOCAL ] BK: Deleted primary data file. [2025-10-20T17:35:11.245708] [INFO ] [MAIN ] BK: Starting full primary refresh. [2025-10-20T17:35:11.463711] [INFO ] [REMOTE ] BIT: Fetching primary data for 2025-10. [2025-10-20T17:35:11.624894] [INFO ] [REMOTE ] BK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:12.251403] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-01. [2025-10-20T17:35:12.582166] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:35:12.935683] [INFO ] [REMOTE ] BIT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T17:35:12.940252] [INFO ] [REMOTE ] BIT: Full fetch complete (32123 records). [2025-10-20T17:35:12.940329] [DEBUG] [VALIDATOR ] BIT: Validating primary data. [2025-10-20T17:35:13.056512] [INFO ] [REMOTE ] BIRK: Fetching primary data for 2025-10. [2025-10-20T17:35:13.314271] [DEBUG] [VALIDATOR ] BIT: All expected trading days are present. [2025-10-20T17:35:13.314352] [DEBUG] [VALIDATOR ] BIT: Primary data validation passed. [2025-10-20T17:35:13.314796] [INFO ] [MAIN ] BIT: Fetched primary data is valid. [2025-10-20T17:35:13.314856] [DEBUG] [LOCAL ] BIT: Saving primary data. [2025-10-20T17:35:13.315534] [DEBUG] [LOCAL ] BIT: Trimmed 32123 records to 23780 (keeping 200 days of data) [2025-10-20T17:35:13.457378] [DEBUG] [LOCAL ] BIT: Primary data saved (23780 records). [2025-10-20T17:35:13.457518] [INFO ] [MAIN ] BIT: Full history saved. [2025-10-20T17:35:13.457599] [INFO ] [MAIN ] Update Progress: 283/2367 symbols completed. BIT took 32.4s. Estimated remaining: 4h 18m 51s. Rate: 8.1 symbols/min [2025-10-20T17:35:13.631991] [INFO ] [MAIN ] BKD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:13.677383] [DEBUG] [LOCAL ] BKD: Loaded 51205 existing records from primary storage. [2025-10-20T17:35:13.677714] [INFO ] [MAIN ] BKD: Loaded 51205 existing records. [2025-10-20T17:35:13.851749] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-01. [2025-10-20T17:35:13.854887] [WARN ] [MAIN ] BKD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:13.856093] [INFO ] [MAIN ] BKD: Wiping existing primary data before full refresh. [2025-10-20T17:35:13.856144] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-10-20T17:35:13.856360] [DEBUG] [LOCAL ] BKD: Deleted primary data file. [2025-10-20T17:35:13.856430] [INFO ] [MAIN ] BKD: Starting full primary refresh. [2025-10-20T17:35:14.216715] [INFO ] [REMOTE ] BKD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:14.247880] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:35:14.651418] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-01. [2025-10-20T17:35:14.934590] [INFO ] [REMOTE ] BIRK: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T17:35:14.942308] [INFO ] [REMOTE ] BIRK: Full fetch complete (73389 records). [2025-10-20T17:35:14.942428] [DEBUG] [VALIDATOR ] BIRK: Validating primary data. [2025-10-20T17:35:15.335988] [DEBUG] [VALIDATOR ] BIRK: All expected trading days are present. [2025-10-20T17:35:15.336102] [DEBUG] [VALIDATOR ] BIRK: Primary data validation passed. [2025-10-20T17:35:15.337160] [INFO ] [MAIN ] BIRK: Fetched primary data is valid. [2025-10-20T17:35:15.337230] [DEBUG] [LOCAL ] BIRK: Saving primary data. [2025-10-20T17:35:15.338815] [DEBUG] [LOCAL ] BIRK: Trimmed 73389 records to 51198 (keeping 200 days of data) [2025-10-20T17:35:15.452953] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-02. [2025-10-20T17:35:15.484570] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:35:15.671459] [DEBUG] [LOCAL ] BIRK: Primary data saved (51198 records). [2025-10-20T17:35:15.671614] [INFO ] [MAIN ] BIRK: Full history saved. [2025-10-20T17:35:15.671732] [INFO ] [MAIN ] Update Progress: 284/2367 symbols completed. BIRK took 34.6s. Estimated remaining: 4h 18m 5s. Rate: 8.1 symbols/min [2025-10-20T17:35:15.845669] [INFO ] [MAIN ] BKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:15.845859] [DEBUG] [LOCAL ] BKE: No existing data file found in primary storage. [2025-10-20T17:35:15.845896] [WARN ] [MAIN ] BKE: No existing data found, performing full refresh instead. [2025-10-20T17:35:15.846013] [INFO ] [MAIN ] BKE: Wiping existing primary data before full refresh. [2025-10-20T17:35:15.846088] [INFO ] [LOCAL ] BKE: Starting deletion of primary data. [2025-10-20T17:35:15.846150] [DEBUG] [LOCAL ] BKE: No primary data file found to delete. [2025-10-20T17:35:15.846183] [INFO ] [MAIN ] BKE: Starting full primary refresh. [2025-10-20T17:35:16.198917] [INFO ] [REMOTE ] BKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:16.224821] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:35:16.251332] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-01. [2025-10-20T17:35:16.972138] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:35:17.051442] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-02. [2025-10-20T17:35:17.458378] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:35:17.851377] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-03. [2025-10-20T17:35:18.595344] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T17:35:18.651357] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-03. [2025-10-20T17:35:19.451406] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-02. [2025-10-20T17:35:19.721207] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T17:35:19.986444] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T17:35:20.251427] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-02. [2025-10-20T17:35:21.051484] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-04. [2025-10-20T17:35:21.687983] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-10-20T17:35:21.851375] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-03. [2025-10-20T17:35:22.465985] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T17:35:22.651451] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-03. [2025-10-20T17:35:23.451369] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-04. [2025-10-20T17:35:23.778538] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-04 (2.73 seconds). [2025-10-20T17:35:23.961843] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T17:35:24.051958] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:35:24.251366] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-04. [2025-10-20T17:35:25.051346] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-05. [2025-10-20T17:35:25.442993] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:35:25.755115] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:35:25.851346] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-05. [2025-10-20T17:35:26.617129] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:35:26.651337] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-06. [2025-10-20T17:35:27.236438] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T17:35:27.451397] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-06. [2025-10-20T17:35:28.256496] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-05. [2025-10-20T17:35:28.269794] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:35:28.757342] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:35:29.051387] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-07. [2025-10-20T17:35:29.626345] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:35:29.851423] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-04. [2025-10-20T17:35:30.440736] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:35:30.651365] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-07. [2025-10-20T17:35:31.451385] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-08. [2025-10-20T17:35:31.844732] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T17:35:32.251346] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-05. [2025-10-20T17:35:32.339187] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:35:32.569302] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:35:33.051422] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-08. [2025-10-20T17:35:33.716980] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T17:35:33.851435] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-06. [2025-10-20T17:35:34.651433] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-09. [2025-10-20T17:35:34.797666] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:35:35.168397] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T17:35:35.451386] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-09. [2025-10-20T17:35:36.082755] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:35:36.251374] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-07. [2025-10-20T17:35:37.051364] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-06. [2025-10-20T17:35:37.107463] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:35:37.820505] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T17:35:37.851402] [INFO ] [REMOTE ] BKD: Fetching primary data for 2025-10. [2025-10-20T17:35:38.651412] [INFO ] [REMOTE ] BKE: Fetching primary data for 2025-10. [2025-10-20T17:35:38.966525] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:35:39.451356] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-07. [2025-10-20T17:35:39.545591] [INFO ] [REMOTE ] BKD: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:35:39.552699] [INFO ] [REMOTE ] BKD: Full fetch complete (71776 records). [2025-10-20T17:35:39.552835] [DEBUG] [VALIDATOR ] BKD: Validating primary data. [2025-10-20T17:35:39.553570] [FATAL] [VALIDATOR ] BKD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:35:39.553690] [FATAL] [MAIN ] BKD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:35:39.553758] [WARN ] [MAIN ] BKD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:35:39.553844] [INFO ] [LOCAL ] BKD: Starting deletion of primary data. [2025-10-20T17:35:39.553923] [DEBUG] [LOCAL ] BKD: No primary data file found to delete. [2025-10-20T17:35:39.553972] [INFO ] [LOCAL ] BKD: Starting deletion of derivative data. [2025-10-20T17:35:39.554206] [DEBUG] [LOCAL ] BKD: Deleted derivative data file. [2025-10-20T17:35:39.554316] [INFO ] [MAIN ] Update Progress: 285/2367 symbols completed. BKD took 26.1s. Estimated remaining: 4h 19m 58s. Rate: 8.0 symbols/min [2025-10-20T17:35:39.723457] [INFO ] [MAIN ] BKH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:39.756103] [DEBUG] [LOCAL ] BKH: Loaded 37744 existing records from primary storage. [2025-10-20T17:35:39.756411] [INFO ] [MAIN ] BKH: Loaded 37744 existing records. [2025-10-20T17:35:39.940943] [WARN ] [MAIN ] BKH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:39.941216] [INFO ] [MAIN ] BKH: Wiping existing primary data before full refresh. [2025-10-20T17:35:39.941266] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-10-20T17:35:39.941561] [DEBUG] [LOCAL ] BKH: Deleted primary data file. [2025-10-20T17:35:39.941617] [INFO ] [MAIN ] BKH: Starting full primary refresh. [2025-10-20T17:35:40.261504] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-08. [2025-10-20T17:35:40.278854] [INFO ] [REMOTE ] BKE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:35:40.338814] [INFO ] [REMOTE ] BKH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:40.339373] [INFO ] [REMOTE ] BKE: Full fetch complete (50414 records). [2025-10-20T17:35:40.339452] [DEBUG] [VALIDATOR ] BKE: Validating primary data. [2025-10-20T17:35:40.734827] [DEBUG] [VALIDATOR ] BKE: All expected trading days are present. [2025-10-20T17:35:40.734935] [DEBUG] [VALIDATOR ] BKE: Primary data validation passed. [2025-10-20T17:35:40.735613] [INFO ] [MAIN ] BKE: Fetched primary data is valid. [2025-10-20T17:35:40.735686] [DEBUG] [LOCAL ] BKE: Saving primary data. [2025-10-20T17:35:40.736907] [DEBUG] [LOCAL ] BKE: Trimmed 50414 records to 33953 (keeping 200 days of data) [2025-10-20T17:35:40.949778] [DEBUG] [LOCAL ] BKE: Primary data saved (33953 records). [2025-10-20T17:35:40.949922] [INFO ] [MAIN ] BKE: Full history saved. [2025-10-20T17:35:40.949997] [INFO ] [MAIN ] Update Progress: 286/2367 symbols completed. BKE took 25.3s. Estimated remaining: 4h 19m 6s. Rate: 8.0 symbols/min [2025-10-20T17:35:41.059591] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-01. [2025-10-20T17:35:41.135894] [INFO ] [MAIN ] BKKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:41.171008] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:35:41.209673] [DEBUG] [LOCAL ] BKKT: Loaded 37334 existing records from primary storage. [2025-10-20T17:35:41.209951] [INFO ] [MAIN ] BKKT: Loaded 37334 existing records. [2025-10-20T17:35:41.398756] [WARN ] [MAIN ] BKKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:41.398962] [INFO ] [MAIN ] BKKT: Wiping existing primary data before full refresh. [2025-10-20T17:35:41.399007] [INFO ] [LOCAL ] BKKT: Starting deletion of primary data. [2025-10-20T17:35:41.399219] [DEBUG] [LOCAL ] BKKT: Deleted primary data file. [2025-10-20T17:35:41.399275] [INFO ] [MAIN ] BKKT: Starting full primary refresh. [2025-10-20T17:35:41.794083] [INFO ] [REMOTE ] BKKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:41.851427] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-08. [2025-10-20T17:35:41.936738] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:35:42.624823] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T17:35:42.651463] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-01. [2025-10-20T17:35:43.451393] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-02. [2025-10-20T17:35:43.764869] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:35:44.094587] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:35:44.251358] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-02. [2025-10-20T17:35:44.796103] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:35:45.051461] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-03. [2025-10-20T17:35:45.341736] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T17:35:45.851412] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-03. [2025-10-20T17:35:46.123008] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:35:46.651357] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-09. [2025-10-20T17:35:47.201549] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T17:35:47.451421] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-04. [2025-10-20T17:35:48.251386] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-04. [2025-10-20T17:35:48.431121] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T17:35:48.851192] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:35:49.051433] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-09. [2025-10-20T17:35:49.482830] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T17:35:49.851421] [INFO ] [REMOTE ] BK: Fetching primary data for 2025-10. [2025-10-20T17:35:50.651435] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-05. [2025-10-20T17:35:51.211869] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T17:35:51.451466] [INFO ] [REMOTE ] BJ: Fetching primary data for 2025-10. [2025-10-20T17:35:51.826937] [INFO ] [REMOTE ] BK: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:35:51.835995] [INFO ] [REMOTE ] BK: Full fetch complete (77678 records). [2025-10-20T17:35:51.836228] [DEBUG] [VALIDATOR ] BK: Validating primary data. [2025-10-20T17:35:52.225088] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:35:52.279688] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-05. [2025-10-20T17:35:52.311513] [DEBUG] [VALIDATOR ] BK: All expected trading days are present. [2025-10-20T17:35:52.312568] [DEBUG] [VALIDATOR ] BK: Primary data validation passed. [2025-10-20T17:35:52.313411] [INFO ] [MAIN ] BK: Fetched primary data is valid. [2025-10-20T17:35:52.313475] [DEBUG] [LOCAL ] BK: Saving primary data. [2025-10-20T17:35:52.315063] [DEBUG] [LOCAL ] BK: Trimmed 77678 records to 53546 (keeping 200 days of data) [2025-10-20T17:35:52.642054] [DEBUG] [LOCAL ] BK: Primary data saved (53546 records). [2025-10-20T17:35:52.642189] [INFO ] [MAIN ] BK: Full history saved. [2025-10-20T17:35:52.642261] [INFO ] [MAIN ] Update Progress: 287/2367 symbols completed. BK took 41.9s. Estimated remaining: 4h 19m 29s. Rate: 8.0 symbols/min [2025-10-20T17:35:52.813022] [INFO ] [MAIN ] BKN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:52.823814] [DEBUG] [LOCAL ] BKN: Loaded 7394 existing records from primary storage. [2025-10-20T17:35:52.823986] [INFO ] [MAIN ] BKN: Loaded 7394 existing records. [2025-10-20T17:35:53.006410] [WARN ] [MAIN ] BKN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:53.006707] [INFO ] [MAIN ] BKN: Wiping existing primary data before full refresh. [2025-10-20T17:35:53.006853] [INFO ] [LOCAL ] BKN: Starting deletion of primary data. [2025-10-20T17:35:53.007066] [DEBUG] [LOCAL ] BKN: Deleted primary data file. [2025-10-20T17:35:53.007200] [INFO ] [MAIN ] BKN: Starting full primary refresh. [2025-10-20T17:35:53.056779] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-06. [2025-10-20T17:35:53.210395] [INFO ] [REMOTE ] BJ: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:35:53.302985] [INFO ] [REMOTE ] BJ: Full fetch complete (74444 records). [2025-10-20T17:35:53.323693] [DEBUG] [VALIDATOR ] BJ: Validating primary data. [2025-10-20T17:35:53.504811] [INFO ] [REMOTE ] BKN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:53.836985] [DEBUG] [VALIDATOR ] BJ: All expected trading days are present. [2025-10-20T17:35:53.837136] [DEBUG] [VALIDATOR ] BJ: Primary data validation passed. [2025-10-20T17:35:53.838018] [INFO ] [MAIN ] BJ: Fetched primary data is valid. [2025-10-20T17:35:53.838118] [DEBUG] [LOCAL ] BJ: Saving primary data. [2025-10-20T17:35:53.842760] [DEBUG] [LOCAL ] BJ: Trimmed 74444 records to 51581 (keeping 200 days of data) [2025-10-20T17:35:53.892327] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-01. [2025-10-20T17:35:53.985399] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:35:54.181271] [DEBUG] [LOCAL ] BJ: Primary data saved (51581 records). [2025-10-20T17:35:54.181414] [INFO ] [MAIN ] BJ: Full history saved. [2025-10-20T17:35:54.181492] [INFO ] [MAIN ] Update Progress: 288/2367 symbols completed. BJ took 44.2s. Estimated remaining: 4h 18m 39s. Rate: 8.0 symbols/min [2025-10-20T17:35:54.353098] [INFO ] [MAIN ] BKSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:35:54.398690] [DEBUG] [LOCAL ] BKSY: Loaded 49470 existing records from primary storage. [2025-10-20T17:35:54.399046] [INFO ] [MAIN ] BKSY: Loaded 49470 existing records. [2025-10-20T17:35:54.582086] [WARN ] [MAIN ] BKSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:35:54.582479] [INFO ] [MAIN ] BKSY: Wiping existing primary data before full refresh. [2025-10-20T17:35:54.582734] [INFO ] [LOCAL ] BKSY: Starting deletion of primary data. [2025-10-20T17:35:54.589921] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:35:54.593006] [DEBUG] [LOCAL ] BKSY: Deleted primary data file. [2025-10-20T17:35:54.593133] [INFO ] [MAIN ] BKSY: Starting full primary refresh. [2025-10-20T17:35:54.658350] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-06. [2025-10-20T17:35:54.720977] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:35:54.976456] [INFO ] [REMOTE ] BKSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:35:55.451354] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-01. [2025-10-20T17:35:56.108542] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:35:56.251337] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-07. [2025-10-20T17:35:56.904879] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:35:57.051373] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-02. [2025-10-20T17:35:57.663632] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:35:57.851352] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-03. [2025-10-20T17:35:57.941769] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T17:35:58.581403] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:35:58.651368] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-02. [2025-10-20T17:35:59.451482] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-08. [2025-10-20T17:36:00.251392] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-04. [2025-10-20T17:36:00.528990] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T17:36:01.051427] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-03. [2025-10-20T17:36:01.071825] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T17:36:01.285694] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:36:01.851372] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-05. [2025-10-20T17:36:02.591895] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:36:02.651393] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-06. [2025-10-20T17:36:02.974687] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:36:03.153188] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T17:36:03.451399] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-07. [2025-10-20T17:36:04.251417] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-04. [2025-10-20T17:36:05.051410] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-07. [2025-10-20T17:36:05.135012] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T17:36:05.639072] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:36:05.717447] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T17:36:05.851370] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-08. [2025-10-20T17:36:06.522638] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T17:36:06.651373] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-09. [2025-10-20T17:36:07.241348] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:36:07.451436] [INFO ] [REMOTE ] BKN: Fetching primary data for 2025-10. [2025-10-20T17:36:08.251399] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-05. [2025-10-20T17:36:08.300258] [INFO ] [REMOTE ] BKN: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T17:36:08.302137] [INFO ] [REMOTE ] BKN: Full fetch complete (11139 records). [2025-10-20T17:36:08.302224] [DEBUG] [VALIDATOR ] BKN: Validating primary data. [2025-10-20T17:36:08.684195] [DEBUG] [VALIDATOR ] BKN: All expected trading days are present. [2025-10-20T17:36:08.684284] [DEBUG] [VALIDATOR ] BKN: Primary data validation passed. [2025-10-20T17:36:08.684562] [INFO ] [MAIN ] BKN: Fetched primary data is valid. [2025-10-20T17:36:08.684601] [DEBUG] [LOCAL ] BKN: Saving primary data. [2025-10-20T17:36:08.685056] [DEBUG] [LOCAL ] BKN: Trimmed 11139 records to 7285 (keeping 200 days of data) [2025-10-20T17:36:08.729956] [DEBUG] [LOCAL ] BKN: Primary data saved (7285 records). [2025-10-20T17:36:08.730089] [INFO ] [MAIN ] BKN: Full history saved. [2025-10-20T17:36:08.730164] [INFO ] [MAIN ] Update Progress: 289/2367 symbols completed. BKN took 16.1s. Estimated remaining: 4h 19m 22s. Rate: 8.0 symbols/min [2025-10-20T17:36:08.906887] [INFO ] [MAIN ] BKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:08.919847] [DEBUG] [LOCAL ] BKT: Loaded 12781 existing records from primary storage. [2025-10-20T17:36:08.920100] [INFO ] [MAIN ] BKT: Loaded 12781 existing records. [2025-10-20T17:36:09.056524] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-08. [2025-10-20T17:36:09.115844] [WARN ] [MAIN ] BKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:09.116102] [INFO ] [MAIN ] BKT: Wiping existing primary data before full refresh. [2025-10-20T17:36:09.116150] [INFO ] [LOCAL ] BKT: Starting deletion of primary data. [2025-10-20T17:36:09.116360] [DEBUG] [LOCAL ] BKT: Deleted primary data file. [2025-10-20T17:36:09.116414] [INFO ] [MAIN ] BKT: Starting full primary refresh. [2025-10-20T17:36:09.480295] [INFO ] [REMOTE ] BKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:09.851406] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-01. [2025-10-20T17:36:10.201301] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:36:10.396483] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T17:36:10.599907] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T17:36:10.651381] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-09. [2025-10-20T17:36:11.451392] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-09. [2025-10-20T17:36:11.672023] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:36:12.251469] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-02. [2025-10-20T17:36:13.051435] [INFO ] [REMOTE ] BKH: Fetching primary data for 2025-10. [2025-10-20T17:36:13.075750] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:36:13.591000] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-10-20T17:36:13.851411] [INFO ] [REMOTE ] BKKT: Fetching primary data for 2025-10. [2025-10-20T17:36:14.543226] [INFO ] [REMOTE ] BKH: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:36:14.549670] [INFO ] [REMOTE ] BKH: Full fetch complete (52726 records). [2025-10-20T17:36:14.549864] [DEBUG] [VALIDATOR ] BKH: Validating primary data. [2025-10-20T17:36:14.550805] [FATAL] [VALIDATOR ] BKH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:36:14.550900] [FATAL] [MAIN ] BKH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:36:14.550946] [WARN ] [MAIN ] BKH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:36:14.550995] [INFO ] [LOCAL ] BKH: Starting deletion of primary data. [2025-10-20T17:36:14.551059] [DEBUG] [LOCAL ] BKH: No primary data file found to delete. [2025-10-20T17:36:14.551093] [INFO ] [LOCAL ] BKH: Starting deletion of derivative data. [2025-10-20T17:36:14.551276] [DEBUG] [LOCAL ] BKH: Deleted derivative data file. [2025-10-20T17:36:14.551380] [INFO ] [MAIN ] Update Progress: 290/2367 symbols completed. BKH took 35.0s. Estimated remaining: 4h 19m 3s. Rate: 8.0 symbols/min [2025-10-20T17:36:14.657094] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-03. [2025-10-20T17:36:14.741345] [INFO ] [MAIN ] BKU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:14.787369] [DEBUG] [LOCAL ] BKU: Loaded 42897 existing records from primary storage. [2025-10-20T17:36:14.787962] [INFO ] [MAIN ] BKU: Loaded 42897 existing records. [2025-10-20T17:36:15.000143] [WARN ] [MAIN ] BKU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:15.002049] [INFO ] [MAIN ] BKU: Wiping existing primary data before full refresh. [2025-10-20T17:36:15.002139] [INFO ] [LOCAL ] BKU: Starting deletion of primary data. [2025-10-20T17:36:15.003157] [DEBUG] [LOCAL ] BKU: Deleted primary data file. [2025-10-20T17:36:15.003219] [INFO ] [MAIN ] BKU: Starting full primary refresh. [2025-10-20T17:36:15.382208] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:36:15.415321] [INFO ] [REMOTE ] BKU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:15.451443] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-01. [2025-10-20T17:36:15.867123] [INFO ] [REMOTE ] BKKT: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T17:36:15.878244] [INFO ] [REMOTE ] BKKT: Full fetch complete (51700 records). [2025-10-20T17:36:15.878499] [DEBUG] [VALIDATOR ] BKKT: Validating primary data. [2025-10-20T17:36:16.256793] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-04. [2025-10-20T17:36:16.302444] [DEBUG] [VALIDATOR ] BKKT: All expected trading days are present. [2025-10-20T17:36:16.302676] [DEBUG] [VALIDATOR ] BKKT: Primary data validation passed. [2025-10-20T17:36:16.303609] [INFO ] [MAIN ] BKKT: Fetched primary data is valid. [2025-10-20T17:36:16.303708] [DEBUG] [LOCAL ] BKKT: Saving primary data. [2025-10-20T17:36:16.305376] [DEBUG] [LOCAL ] BKKT: Trimmed 51700 records to 37052 (keeping 200 days of data) [2025-10-20T17:36:16.538896] [DEBUG] [LOCAL ] BKKT: Primary data saved (37052 records). [2025-10-20T17:36:16.539089] [INFO ] [MAIN ] BKKT: Full history saved. [2025-10-20T17:36:16.539662] [INFO ] [MAIN ] Update Progress: 291/2367 symbols completed. BKKT took 35.6s. Estimated remaining: 4h 18m 16s. Rate: 8.0 symbols/min [2025-10-20T17:36:16.723369] [INFO ] [MAIN ] BKV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:16.757019] [DEBUG] [LOCAL ] BKV: Loaded 30117 existing records from primary storage. [2025-10-20T17:36:16.757372] [INFO ] [MAIN ] BKV: Loaded 30117 existing records. [2025-10-20T17:36:16.982092] [WARN ] [MAIN ] BKV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:16.982505] [INFO ] [MAIN ] BKV: Wiping existing primary data before full refresh. [2025-10-20T17:36:16.982592] [INFO ] [LOCAL ] BKV: Starting deletion of primary data. [2025-10-20T17:36:16.983089] [DEBUG] [LOCAL ] BKV: Deleted primary data file. [2025-10-20T17:36:16.983286] [INFO ] [MAIN ] BKV: Starting full primary refresh. [2025-10-20T17:36:17.056277] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-06. [2025-10-20T17:36:17.122078] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T17:36:17.250960] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T17:36:17.454367] [INFO ] [REMOTE ] BKV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:17.851405] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-02. [2025-10-20T17:36:18.651392] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-05. [2025-10-20T17:36:18.954972] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:36:19.305050] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T17:36:19.353495] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:36:19.451388] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-03. [2025-10-20T17:36:20.251384] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-06. [2025-10-20T17:36:21.051385] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-07. [2025-10-20T17:36:21.108125] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T17:36:21.108475] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:36:21.851384] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-04. [2025-10-20T17:36:22.651413] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-07. [2025-10-20T17:36:23.184836] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T17:36:23.410608] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T17:36:23.453862] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-01. [2025-10-20T17:36:23.460786] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:36:24.251448] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-05. [2025-10-20T17:36:24.913979] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T17:36:25.051413] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-02. [2025-10-20T17:36:25.646687] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:36:25.851362] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-06. [2025-10-20T17:36:26.359121] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T17:36:26.651411] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-08. [2025-10-20T17:36:27.061421] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:36:27.212901] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T17:36:27.451392] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-07. [2025-10-20T17:36:28.251365] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-08. [2025-10-20T17:36:28.977172] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:36:29.051383] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-08. [2025-10-20T17:36:29.803784] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:36:29.851385] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-09. [2025-10-20T17:36:30.007086] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:36:30.651371] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-09. [2025-10-20T17:36:31.451363] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-03. [2025-10-20T17:36:31.857365] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T17:36:32.103121] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T17:36:32.251366] [INFO ] [REMOTE ] BKU: Fetching primary data for 2025-10. [2025-10-20T17:36:32.648238] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:36:33.051374] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-09. [2025-10-20T17:36:33.525427] [INFO ] [REMOTE ] BKU: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:36:33.532279] [INFO ] [REMOTE ] BKU: Full fetch complete (60914 records). [2025-10-20T17:36:33.532427] [DEBUG] [VALIDATOR ] BKU: Validating primary data. [2025-10-20T17:36:33.854966] [INFO ] [REMOTE ] BKSY: Fetching primary data for 2025-10. [2025-10-20T17:36:33.931625] [DEBUG] [VALIDATOR ] BKU: All expected trading days are present. [2025-10-20T17:36:33.931735] [DEBUG] [VALIDATOR ] BKU: Primary data validation passed. [2025-10-20T17:36:33.932380] [INFO ] [MAIN ] BKU: Fetched primary data is valid. [2025-10-20T17:36:33.932420] [DEBUG] [LOCAL ] BKU: Saving primary data. [2025-10-20T17:36:33.933591] [DEBUG] [LOCAL ] BKU: Trimmed 60914 records to 42263 (keeping 200 days of data) [2025-10-20T17:36:34.166547] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:36:34.205410] [DEBUG] [LOCAL ] BKU: Primary data saved (42263 records). [2025-10-20T17:36:34.205551] [INFO ] [MAIN ] BKU: Full history saved. [2025-10-20T17:36:34.205629] [INFO ] [MAIN ] Update Progress: 292/2367 symbols completed. BKU took 19.7s. Estimated remaining: 4h 19m 21s. Rate: 8.0 symbols/min [2025-10-20T17:36:34.371864] [INFO ] [MAIN ] BLCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:34.404484] [DEBUG] [LOCAL ] BLCO: Loaded 35655 existing records from primary storage. [2025-10-20T17:36:34.404784] [INFO ] [MAIN ] BLCO: Loaded 35655 existing records. [2025-10-20T17:36:34.580795] [WARN ] [MAIN ] BLCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:34.581014] [INFO ] [MAIN ] BLCO: Wiping existing primary data before full refresh. [2025-10-20T17:36:34.581060] [INFO ] [LOCAL ] BLCO: Starting deletion of primary data. [2025-10-20T17:36:34.581243] [DEBUG] [LOCAL ] BLCO: Deleted primary data file. [2025-10-20T17:36:34.581281] [INFO ] [MAIN ] BLCO: Starting full primary refresh. [2025-10-20T17:36:34.656470] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-04. [2025-10-20T17:36:35.027399] [INFO ] [REMOTE ] BLCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:35.451453] [INFO ] [REMOTE ] BKT: Fetching primary data for 2025-10. [2025-10-20T17:36:35.776034] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T17:36:36.063117] [INFO ] [REMOTE ] BKSY: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T17:36:36.071533] [INFO ] [REMOTE ] BKSY: Full fetch complete (70525 records). [2025-10-20T17:36:36.071779] [DEBUG] [VALIDATOR ] BKSY: Validating primary data. [2025-10-20T17:36:36.251396] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-05. [2025-10-20T17:36:36.488992] [DEBUG] [VALIDATOR ] BKSY: All expected trading days are present. [2025-10-20T17:36:36.489079] [DEBUG] [VALIDATOR ] BKSY: Primary data validation passed. [2025-10-20T17:36:36.489849] [INFO ] [MAIN ] BKSY: Fetched primary data is valid. [2025-10-20T17:36:36.489912] [DEBUG] [LOCAL ] BKSY: Saving primary data. [2025-10-20T17:36:36.491379] [DEBUG] [LOCAL ] BKSY: Trimmed 70525 records to 48890 (keeping 200 days of data) [2025-10-20T17:36:36.712052] [INFO ] [REMOTE ] BKT: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:36:36.728606] [INFO ] [REMOTE ] BKT: Full fetch complete (16574 records). [2025-10-20T17:36:36.740234] [DEBUG] [VALIDATOR ] BKT: Validating primary data. [2025-10-20T17:36:36.912284] [DEBUG] [LOCAL ] BKSY: Primary data saved (48890 records). [2025-10-20T17:36:36.912797] [INFO ] [MAIN ] BKSY: Full history saved. [2025-10-20T17:36:36.918414] [INFO ] [MAIN ] Update Progress: 293/2367 symbols completed. BKSY took 42.7s. Estimated remaining: 4h 18m 40s. Rate: 8.0 symbols/min [2025-10-20T17:36:37.074381] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-01. [2025-10-20T17:36:37.290876] [INFO ] [MAIN ] BLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:37.379981] [DEBUG] [VALIDATOR ] BKT: All expected trading days are present. [2025-10-20T17:36:37.380075] [DEBUG] [VALIDATOR ] BKT: Primary data validation passed. [2025-10-20T17:36:37.380438] [INFO ] [MAIN ] BKT: Fetched primary data is valid. [2025-10-20T17:36:37.380476] [DEBUG] [LOCAL ] BKT: Saving primary data. [2025-10-20T17:36:37.381275] [DEBUG] [LOCAL ] BKT: Trimmed 16574 records to 12778 (keeping 200 days of data) [2025-10-20T17:36:37.484409] [DEBUG] [LOCAL ] BLD: Loaded 27717 existing records from primary storage. [2025-10-20T17:36:37.484855] [DEBUG] [LOCAL ] BKT: Primary data saved (12778 records). [2025-10-20T17:36:37.485517] [INFO ] [MAIN ] BLD: Loaded 27717 existing records. [2025-10-20T17:36:37.485191] [INFO ] [MAIN ] BKT: Full history saved. [2025-10-20T17:36:37.517759] [INFO ] [MAIN ] Update Progress: 294/2367 symbols completed. BKT took 28.8s. Estimated remaining: 4h 17m 44s. Rate: 8.0 symbols/min [2025-10-20T17:36:37.825457] [WARN ] [MAIN ] BLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:37.853668] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:36:37.850721] [INFO ] [MAIN ] BLD: Wiping existing primary data before full refresh. [2025-10-20T17:36:37.877750] [INFO ] [LOCAL ] BLD: Starting deletion of primary data. [2025-10-20T17:36:37.902162] [DEBUG] [LOCAL ] BLD: Deleted primary data file. [2025-10-20T17:36:37.901323] [INFO ] [MAIN ] BLDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:37.902293] [INFO ] [MAIN ] BLD: Starting full primary refresh. [2025-10-20T17:36:37.903005] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-06. [2025-10-20T17:36:38.277369] [INFO ] [REMOTE ] BLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:38.295071] [DEBUG] [LOCAL ] BLDR: Loaded 52629 existing records from primary storage. [2025-10-20T17:36:38.295436] [INFO ] [MAIN ] BLDR: Loaded 52629 existing records. [2025-10-20T17:36:38.488814] [WARN ] [MAIN ] BLDR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:38.489188] [INFO ] [MAIN ] BLDR: Wiping existing primary data before full refresh. [2025-10-20T17:36:38.489367] [INFO ] [LOCAL ] BLDR: Starting deletion of primary data. [2025-10-20T17:36:38.489723] [DEBUG] [LOCAL ] BLDR: Deleted primary data file. [2025-10-20T17:36:38.505623] [INFO ] [MAIN ] BLDR: Starting full primary refresh. [2025-10-20T17:36:38.557491] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:36:38.656624] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-01. [2025-10-20T17:36:38.913003] [INFO ] [REMOTE ] BLDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:39.354330] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T17:36:39.451400] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-01. [2025-10-20T17:36:39.867096] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:36:40.251391] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-02. [2025-10-20T17:36:41.051425] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-02. [2025-10-20T17:36:41.093033] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T17:36:41.288251] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:36:41.851387] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-02. [2025-10-20T17:36:42.453858] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T17:36:42.651373] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-03. [2025-10-20T17:36:43.265255] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:36:43.451438] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-03. [2025-10-20T17:36:43.693158] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T17:36:44.251432] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-07. [2025-10-20T17:36:45.051431] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-03. [2025-10-20T17:36:45.282176] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T17:36:45.488367] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:36:45.851403] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-04. [2025-10-20T17:36:46.331532] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T17:36:46.651483] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-08. [2025-10-20T17:36:47.242018] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:36:47.451381] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-05. [2025-10-20T17:36:47.500134] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T17:36:48.251401] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-04. [2025-10-20T17:36:49.051365] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-09. [2025-10-20T17:36:49.173622] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:36:49.851366] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-06. [2025-10-20T17:36:50.290520] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:36:50.344539] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:36:50.651428] [INFO ] [REMOTE ] BKV: Fetching primary data for 2025-10. [2025-10-20T17:36:51.451401] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-05. [2025-10-20T17:36:51.473253] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:36:52.081067] [INFO ] [REMOTE ] BKV: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T17:36:52.086594] [INFO ] [REMOTE ] BKV: Full fetch complete (44924 records). [2025-10-20T17:36:52.086774] [DEBUG] [VALIDATOR ] BKV: Validating primary data. [2025-10-20T17:36:52.253305] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-04. [2025-10-20T17:36:52.462344] [DEBUG] [VALIDATOR ] BKV: All expected trading days are present. [2025-10-20T17:36:52.462436] [DEBUG] [VALIDATOR ] BKV: Primary data validation passed. [2025-10-20T17:36:52.462986] [INFO ] [MAIN ] BKV: Fetched primary data is valid. [2025-10-20T17:36:52.463054] [DEBUG] [LOCAL ] BKV: Saving primary data. [2025-10-20T17:36:52.464199] [DEBUG] [LOCAL ] BKV: Trimmed 44924 records to 29615 (keeping 200 days of data) [2025-10-20T17:36:52.633984] [DEBUG] [LOCAL ] BKV: Primary data saved (29615 records). [2025-10-20T17:36:52.634127] [INFO ] [MAIN ] BKV: Full history saved. [2025-10-20T17:36:52.634226] [INFO ] [MAIN ] Update Progress: 295/2367 symbols completed. BKV took 36.1s. Estimated remaining: 4h 18m 30s. Rate: 8.0 symbols/min [2025-10-20T17:36:52.801060] [INFO ] [MAIN ] BLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:36:52.814871] [DEBUG] [LOCAL ] BLE: Loaded 13930 existing records from primary storage. [2025-10-20T17:36:52.815083] [INFO ] [MAIN ] BLE: Loaded 13930 existing records. [2025-10-20T17:36:52.988111] [WARN ] [MAIN ] BLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:36:52.988353] [INFO ] [MAIN ] BLE: Wiping existing primary data before full refresh. [2025-10-20T17:36:52.988409] [INFO ] [LOCAL ] BLE: Starting deletion of primary data. [2025-10-20T17:36:52.988601] [DEBUG] [LOCAL ] BLE: Deleted primary data file. [2025-10-20T17:36:52.988638] [INFO ] [MAIN ] BLE: Starting full primary refresh. [2025-10-20T17:36:53.062050] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-07. [2025-10-20T17:36:53.346138] [INFO ] [REMOTE ] BLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:36:53.445155] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T17:36:53.683214] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:36:53.851418] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-05. [2025-10-20T17:36:54.226191] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:36:54.651426] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-06. [2025-10-20T17:36:55.067875] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T17:36:55.451412] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-01. [2025-10-20T17:36:56.251368] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-06. [2025-10-20T17:36:56.429931] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:36:56.460333] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T17:36:57.051370] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-08. [2025-10-20T17:36:57.586627] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:36:57.851428] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-07. [2025-10-20T17:36:58.330839] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T17:36:58.651381] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-02. [2025-10-20T17:36:59.392676] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T17:36:59.451343] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-03. [2025-10-20T17:37:00.066116] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-10-20T17:37:00.251370] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-09. [2025-10-20T17:37:00.400676] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:37:01.051411] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-07. [2025-10-20T17:37:01.382076] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:37:01.851350] [INFO ] [REMOTE ] BLCO: Fetching primary data for 2025-10. [2025-10-20T17:37:02.075600] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:37:02.651421] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-04. [2025-10-20T17:37:02.847210] [INFO ] [REMOTE ] BLCO: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:37:02.852007] [INFO ] [REMOTE ] BLCO: Full fetch complete (52342 records). [2025-10-20T17:37:02.852099] [DEBUG] [VALIDATOR ] BLCO: Validating primary data. [2025-10-20T17:37:03.248765] [DEBUG] [VALIDATOR ] BLCO: All expected trading days are present. [2025-10-20T17:37:03.248842] [DEBUG] [VALIDATOR ] BLCO: Primary data validation passed. [2025-10-20T17:37:03.249438] [INFO ] [MAIN ] BLCO: Fetched primary data is valid. [2025-10-20T17:37:03.249477] [DEBUG] [LOCAL ] BLCO: Saving primary data. [2025-10-20T17:37:03.251101] [DEBUG] [LOCAL ] BLCO: Trimmed 52342 records to 35097 (keeping 200 days of data) [2025-10-20T17:37:03.448041] [DEBUG] [LOCAL ] BLCO: Primary data saved (35097 records). [2025-10-20T17:37:03.448218] [INFO ] [MAIN ] BLCO: Full history saved. [2025-10-20T17:37:03.448340] [INFO ] [MAIN ] Update Progress: 296/2367 symbols completed. BLCO took 29.2s. Estimated remaining: 4h 18m 46s. Rate: 8.0 symbols/min [2025-10-20T17:37:03.459378] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-08. [2025-10-20T17:37:03.570614] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T17:37:03.641839] [INFO ] [MAIN ] BLK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:03.673009] [DEBUG] [LOCAL ] BLK: Loaded 34882 existing records from primary storage. [2025-10-20T17:37:03.673282] [INFO ] [MAIN ] BLK: Loaded 34882 existing records. [2025-10-20T17:37:03.846745] [WARN ] [MAIN ] BLK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:03.846970] [INFO ] [MAIN ] BLK: Wiping existing primary data before full refresh. [2025-10-20T17:37:03.847017] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-10-20T17:37:03.847227] [DEBUG] [LOCAL ] BLK: Deleted primary data file. [2025-10-20T17:37:03.847264] [INFO ] [MAIN ] BLK: Starting full primary refresh. [2025-10-20T17:37:04.193245] [INFO ] [REMOTE ] BLK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:04.251326] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-01. [2025-10-20T17:37:05.051421] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-05. [2025-10-20T17:37:05.652187] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T17:37:05.851412] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-09. [2025-10-20T17:37:05.865537] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:37:06.130718] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T17:37:06.651444] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-06. [2025-10-20T17:37:07.451345] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-02. [2025-10-20T17:37:07.502823] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:37:07.767012] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:37:08.251409] [INFO ] [REMOTE ] BLDR: Fetching primary data for 2025-10. [2025-10-20T17:37:09.056534] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-08. [2025-10-20T17:37:09.087238] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:37:09.851371] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-03. [2025-10-20T17:37:09.939970] [INFO ] [REMOTE ] BLDR: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:37:09.948622] [INFO ] [REMOTE ] BLDR: Full fetch complete (74700 records). [2025-10-20T17:37:09.948840] [DEBUG] [VALIDATOR ] BLDR: Validating primary data. [2025-10-20T17:37:10.369022] [DEBUG] [VALIDATOR ] BLDR: All expected trading days are present. [2025-10-20T17:37:10.370499] [DEBUG] [VALIDATOR ] BLDR: Primary data validation passed. [2025-10-20T17:37:10.371444] [INFO ] [MAIN ] BLDR: Fetched primary data is valid. [2025-10-20T17:37:10.371528] [DEBUG] [LOCAL ] BLDR: Saving primary data. [2025-10-20T17:37:10.370408] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:37:10.379755] [DEBUG] [LOCAL ] BLDR: Trimmed 74700 records to 51892 (keeping 200 days of data) [2025-10-20T17:37:10.668413] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-07. [2025-10-20T17:37:10.691333] [DEBUG] [LOCAL ] BLDR: Primary data saved (51892 records). [2025-10-20T17:37:10.691545] [INFO ] [MAIN ] BLDR: Full history saved. [2025-10-20T17:37:10.691635] [INFO ] [MAIN ] Update Progress: 297/2367 symbols completed. BLDR took 33.1s. Estimated remaining: 4h 18m 37s. Rate: 8.0 symbols/min [2025-10-20T17:37:10.858207] [INFO ] [MAIN ] BLND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:10.900221] [DEBUG] [LOCAL ] BLND: Loaded 50014 existing records from primary storage. [2025-10-20T17:37:10.900516] [INFO ] [MAIN ] BLND: Loaded 50014 existing records. [2025-10-20T17:37:11.072883] [WARN ] [MAIN ] BLND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:11.073129] [INFO ] [MAIN ] BLND: Wiping existing primary data before full refresh. [2025-10-20T17:37:11.073176] [INFO ] [LOCAL ] BLND: Starting deletion of primary data. [2025-10-20T17:37:11.073585] [DEBUG] [LOCAL ] BLND: Deleted primary data file. [2025-10-20T17:37:11.073623] [INFO ] [MAIN ] BLND: Starting full primary refresh. [2025-10-20T17:37:11.390784] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T17:37:11.455064] [INFO ] [REMOTE ] BLND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:11.455135] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-09. [2025-10-20T17:37:11.855209] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T17:37:12.251348] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-08. [2025-10-20T17:37:12.329797] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:37:12.718037] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T17:37:13.051390] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-01. [2025-10-20T17:37:13.851374] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-04. [2025-10-20T17:37:14.651400] [INFO ] [REMOTE ] BLD: Fetching primary data for 2025-10. [2025-10-20T17:37:14.878975] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:37:15.451369] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-02. [2025-10-20T17:37:15.501853] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T17:37:15.539131] [INFO ] [REMOTE ] BLD: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T17:37:15.542881] [INFO ] [REMOTE ] BLD: Full fetch complete (39851 records). [2025-10-20T17:37:15.542966] [DEBUG] [VALIDATOR ] BLD: Validating primary data. [2025-10-20T17:37:15.918537] [DEBUG] [VALIDATOR ] BLD: All expected trading days are present. [2025-10-20T17:37:15.918637] [DEBUG] [VALIDATOR ] BLD: Primary data validation passed. [2025-10-20T17:37:15.919164] [INFO ] [MAIN ] BLD: Fetched primary data is valid. [2025-10-20T17:37:15.919206] [DEBUG] [LOCAL ] BLD: Saving primary data. [2025-10-20T17:37:15.920617] [DEBUG] [LOCAL ] BLD: Trimmed 39851 records to 27298 (keeping 200 days of data) [2025-10-20T17:37:16.081876] [DEBUG] [LOCAL ] BLD: Primary data saved (27298 records). [2025-10-20T17:37:16.082008] [INFO ] [MAIN ] BLD: Full history saved. [2025-10-20T17:37:16.082089] [INFO ] [MAIN ] Update Progress: 298/2367 symbols completed. BLD took 39.2s. Estimated remaining: 4h 18m 14s. Rate: 8.0 symbols/min [2025-10-20T17:37:16.253856] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-05. [2025-10-20T17:37:16.253733] [INFO ] [MAIN ] BLSH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:16.255114] [DEBUG] [LOCAL ] BLSH: No existing data file found in primary storage. [2025-10-20T17:37:16.255187] [WARN ] [MAIN ] BLSH: No existing data found, performing full refresh instead. [2025-10-20T17:37:16.255267] [INFO ] [MAIN ] BLSH: Wiping existing primary data before full refresh. [2025-10-20T17:37:16.255303] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-10-20T17:37:16.255345] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-10-20T17:37:16.255379] [INFO ] [MAIN ] BLSH: Starting full primary refresh. [2025-10-20T17:37:16.622137] [INFO ] [REMOTE ] BLSH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:16.866902] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:37:17.051348] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-03. [2025-10-20T17:37:17.692534] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T17:37:17.851391] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-06. [2025-10-20T17:37:18.402912] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T17:37:18.651394] [INFO ] [REMOTE ] BLSH: Fetching primary data for 2025-01. [2025-10-20T17:37:18.838591] [DEBUG] [REMOTE ] BLSH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:37:18.838930] [WARN ] [REMOTE ] BLSH: No data returned for month 2025-01 [2025-10-20T17:37:18.839022] [FATAL] [REMOTE ] BLSH: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:37:18.839069] [FATAL] [MAIN ] BLSH: Failed to fetch primary data. [2025-10-20T17:37:18.839102] [WARN ] [MAIN ] BLSH: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:37:18.839165] [INFO ] [LOCAL ] BLSH: Starting deletion of primary data. [2025-10-20T17:37:18.839226] [DEBUG] [LOCAL ] BLSH: No primary data file found to delete. [2025-10-20T17:37:18.839260] [INFO ] [LOCAL ] BLSH: Starting deletion of derivative data. [2025-10-20T17:37:18.839298] [DEBUG] [LOCAL ] BLSH: No derivative data file found to delete. [2025-10-20T17:37:18.839344] [INFO ] [MAIN ] Update Progress: 299/2367 symbols completed. BLSH took 2.8s. Estimated remaining: 4h 17m 34s. Rate: 8.0 symbols/min [2025-10-20T17:37:19.012444] [INFO ] [MAIN ] BLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:19.028028] [DEBUG] [LOCAL ] BLW: Loaded 13364 existing records from primary storage. [2025-10-20T17:37:19.028269] [INFO ] [MAIN ] BLW: Loaded 13364 existing records. [2025-10-20T17:37:19.207277] [WARN ] [MAIN ] BLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:19.207580] [INFO ] [MAIN ] BLW: Wiping existing primary data before full refresh. [2025-10-20T17:37:19.207696] [INFO ] [LOCAL ] BLW: Starting deletion of primary data. [2025-10-20T17:37:19.207951] [DEBUG] [LOCAL ] BLW: Deleted primary data file. [2025-10-20T17:37:19.208008] [INFO ] [MAIN ] BLW: Starting full primary refresh. [2025-10-20T17:37:19.450684] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T17:37:19.457193] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-04. [2025-10-20T17:37:19.636607] [INFO ] [REMOTE ] BLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:20.251419] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-07. [2025-10-20T17:37:21.051356] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-09. [2025-10-20T17:37:21.179677] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T17:37:21.851414] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-01. [2025-10-20T17:37:21.990433] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:37:22.060186] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T17:37:22.651396] [INFO ] [REMOTE ] BLE: Fetching primary data for 2025-10. [2025-10-20T17:37:22.781498] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T17:37:23.451361] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-08. [2025-10-20T17:37:23.489890] [INFO ] [REMOTE ] BLE: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:37:23.491984] [INFO ] [REMOTE ] BLE: Full fetch complete (19600 records). [2025-10-20T17:37:23.492051] [DEBUG] [VALIDATOR ] BLE: Validating primary data. [2025-10-20T17:37:23.856217] [DEBUG] [VALIDATOR ] BLE: All expected trading days are present. [2025-10-20T17:37:23.856298] [DEBUG] [VALIDATOR ] BLE: Primary data validation passed. [2025-10-20T17:37:23.856627] [INFO ] [MAIN ] BLE: Fetched primary data is valid. [2025-10-20T17:37:23.856697] [DEBUG] [LOCAL ] BLE: Saving primary data. [2025-10-20T17:37:23.857229] [DEBUG] [LOCAL ] BLE: Trimmed 19600 records to 13804 (keeping 200 days of data) [2025-10-20T17:37:23.935942] [DEBUG] [LOCAL ] BLE: Primary data saved (13804 records). [2025-10-20T17:37:23.936072] [INFO ] [MAIN ] BLE: Full history saved. [2025-10-20T17:37:23.936150] [INFO ] [MAIN ] Update Progress: 300/2367 symbols completed. BLE took 31.3s. Estimated remaining: 4h 17m 10s. Rate: 8.0 symbols/min [2025-10-20T17:37:24.101894] [INFO ] [MAIN ] BLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:24.118027] [DEBUG] [LOCAL ] BLX: Loaded 15775 existing records from primary storage. [2025-10-20T17:37:24.118272] [INFO ] [MAIN ] BLX: Loaded 15775 existing records. [2025-10-20T17:37:24.256459] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-05. [2025-10-20T17:37:24.298976] [WARN ] [MAIN ] BLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:24.299159] [INFO ] [MAIN ] BLX: Wiping existing primary data before full refresh. [2025-10-20T17:37:24.299215] [INFO ] [LOCAL ] BLX: Starting deletion of primary data. [2025-10-20T17:37:24.299405] [DEBUG] [LOCAL ] BLX: Deleted primary data file. [2025-10-20T17:37:24.299463] [INFO ] [MAIN ] BLX: Starting full primary refresh. [2025-10-20T17:37:24.653309] [INFO ] [REMOTE ] BLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:25.007202] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:37:25.051340] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-09. [2025-10-20T17:37:25.838259] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:37:25.851319] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-06. [2025-10-20T17:37:26.647501] [INFO ] [REMOTE ] BLK: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:37:26.653930] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-02. [2025-10-20T17:37:27.062292] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:37:27.183396] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:37:27.451425] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-03. [2025-10-20T17:37:28.251443] [INFO ] [REMOTE ] BLK: Fetching primary data for 2025-10. [2025-10-20T17:37:28.380481] [DEBUG] [REMOTE ] BLK: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T17:37:28.380918] [WARN ] [REMOTE ] BLK: No data returned for month 2025-10 [2025-10-20T17:37:28.381042] [FATAL] [REMOTE ] BLK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:37:28.381149] [FATAL] [MAIN ] BLK: Failed to fetch primary data. [2025-10-20T17:37:28.381232] [WARN ] [MAIN ] BLK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:37:28.381323] [INFO ] [LOCAL ] BLK: Starting deletion of primary data. [2025-10-20T17:37:28.381414] [DEBUG] [LOCAL ] BLK: No primary data file found to delete. [2025-10-20T17:37:28.381474] [INFO ] [LOCAL ] BLK: Starting deletion of derivative data. [2025-10-20T17:37:28.381715] [DEBUG] [LOCAL ] BLK: Deleted derivative data file. [2025-10-20T17:37:28.381824] [INFO ] [MAIN ] Update Progress: 301/2367 symbols completed. BLK took 24.9s. Estimated remaining: 4h 16m 42s. Rate: 8.0 symbols/min [2025-10-20T17:37:28.487275] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T17:37:28.639265] [INFO ] [MAIN ] BMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:28.692433] [DEBUG] [LOCAL ] BMA: Loaded 36733 existing records from primary storage. [2025-10-20T17:37:28.692922] [INFO ] [MAIN ] BMA: Loaded 36733 existing records. [2025-10-20T17:37:28.925042] [WARN ] [MAIN ] BMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:28.925360] [INFO ] [MAIN ] BMA: Wiping existing primary data before full refresh. [2025-10-20T17:37:28.925431] [INFO ] [LOCAL ] BMA: Starting deletion of primary data. [2025-10-20T17:37:28.925737] [DEBUG] [LOCAL ] BMA: Deleted primary data file. [2025-10-20T17:37:28.925809] [INFO ] [MAIN ] BMA: Starting full primary refresh. [2025-10-20T17:37:29.056569] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-07. [2025-10-20T17:37:29.440131] [INFO ] [REMOTE ] BMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:29.851393] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-01. [2025-10-20T17:37:30.651377] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-01. [2025-10-20T17:37:30.763563] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:37:31.215630] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:37:31.455817] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-08. [2025-10-20T17:37:31.459228] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:37:32.251397] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-02. [2025-10-20T17:37:33.051455] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-04. [2025-10-20T17:37:33.118369] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:37:33.160473] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T17:37:33.851439] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-02. [2025-10-20T17:37:33.889971] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T17:37:34.651396] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-05. [2025-10-20T17:37:35.016066] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:37:35.109303] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T17:37:35.451402] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-03. [2025-10-20T17:37:36.251392] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-09. [2025-10-20T17:37:36.401380] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:37:37.051394] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-06. [2025-10-20T17:37:37.851388] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-04. [2025-10-20T17:37:37.917175] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T17:37:38.169483] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:37:38.651418] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-03. [2025-10-20T17:37:39.272006] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:37:39.451409] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-05. [2025-10-20T17:37:39.577279] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T17:37:40.251362] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-04. [2025-10-20T17:37:40.518616] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:37:41.051367] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-06. [2025-10-20T17:37:41.341204] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T17:37:41.851393] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-05. [2025-10-20T17:37:42.118698] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:37:42.598561] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T17:37:42.651402] [INFO ] [REMOTE ] BLND: Fetching primary data for 2025-10. [2025-10-20T17:37:43.451373] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-06. [2025-10-20T17:37:44.044945] [INFO ] [REMOTE ] BLND: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:37:44.052272] [INFO ] [REMOTE ] BLND: Full fetch complete (72274 records). [2025-10-20T17:37:44.052419] [DEBUG] [VALIDATOR ] BLND: Validating primary data. [2025-10-20T17:37:44.261347] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-07. [2025-10-20T17:37:44.452284] [DEBUG] [VALIDATOR ] BLND: All expected trading days are present. [2025-10-20T17:37:44.457526] [DEBUG] [VALIDATOR ] BLND: Primary data validation passed. [2025-10-20T17:37:44.458626] [INFO ] [MAIN ] BLND: Fetched primary data is valid. [2025-10-20T17:37:44.458744] [DEBUG] [LOCAL ] BLND: Saving primary data. [2025-10-20T17:37:44.460394] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:37:44.465770] [DEBUG] [LOCAL ] BLND: Trimmed 72274 records to 49271 (keeping 200 days of data) [2025-10-20T17:37:44.731191] [DEBUG] [LOCAL ] BLND: Primary data saved (49271 records). [2025-10-20T17:37:44.731342] [INFO ] [MAIN ] BLND: Full history saved. [2025-10-20T17:37:44.731434] [INFO ] [MAIN ] Update Progress: 302/2367 symbols completed. BLND took 34.0s. Estimated remaining: 4h 17m 36s. Rate: 8.0 symbols/min [2025-10-20T17:37:44.896736] [INFO ] [MAIN ] BME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:44.908272] [DEBUG] [LOCAL ] BME: Loaded 11421 existing records from primary storage. [2025-10-20T17:37:44.908475] [INFO ] [MAIN ] BME: Loaded 11421 existing records. [2025-10-20T17:37:45.056513] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-07. [2025-10-20T17:37:45.084379] [WARN ] [MAIN ] BME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:37:45.084799] [INFO ] [MAIN ] BME: Wiping existing primary data before full refresh. [2025-10-20T17:37:45.084857] [INFO ] [LOCAL ] BME: Starting deletion of primary data. [2025-10-20T17:37:45.085016] [DEBUG] [LOCAL ] BME: Deleted primary data file. [2025-10-20T17:37:45.085056] [INFO ] [MAIN ] BME: Starting full primary refresh. [2025-10-20T17:37:45.207886] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:37:45.456471] [INFO ] [REMOTE ] BME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:45.851424] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-01. [2025-10-20T17:37:46.229490] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T17:37:46.651357] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-08. [2025-10-20T17:37:46.892880] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:37:47.451340] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-02. [2025-10-20T17:37:47.533370] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:37:48.198446] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:37:48.251367] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-09. [2025-10-20T17:37:49.051341] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-03. [2025-10-20T17:37:49.349639] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T17:37:49.835720] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T17:37:49.851339] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-04. [2025-10-20T17:37:50.651382] [INFO ] [REMOTE ] BLW: Fetching primary data for 2025-10. [2025-10-20T17:37:50.728227] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:37:51.451372] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-05. [2025-10-20T17:37:51.619431] [INFO ] [REMOTE ] BLW: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:37:51.622217] [INFO ] [REMOTE ] BLW: Full fetch complete (19198 records). [2025-10-20T17:37:51.622302] [DEBUG] [VALIDATOR ] BLW: Validating primary data. [2025-10-20T17:37:51.989295] [DEBUG] [VALIDATOR ] BLW: All expected trading days are present. [2025-10-20T17:37:51.989375] [DEBUG] [VALIDATOR ] BLW: Primary data validation passed. [2025-10-20T17:37:51.989738] [INFO ] [MAIN ] BLW: Fetched primary data is valid. [2025-10-20T17:37:51.989798] [DEBUG] [LOCAL ] BLW: Saving primary data. [2025-10-20T17:37:51.990334] [DEBUG] [LOCAL ] BLW: Trimmed 19198 records to 13124 (keeping 200 days of data) [2025-10-20T17:37:52.072464] [DEBUG] [LOCAL ] BLW: Primary data saved (13124 records). [2025-10-20T17:37:52.072701] [INFO ] [MAIN ] BLW: Full history saved. [2025-10-20T17:37:52.072792] [INFO ] [MAIN ] Update Progress: 303/2367 symbols completed. BLW took 33.2s. Estimated remaining: 4h 17m 27s. Rate: 8.0 symbols/min [2025-10-20T17:37:52.239545] [INFO ] [MAIN ] BMEZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:37:52.239715] [DEBUG] [LOCAL ] BMEZ: No existing data file found in primary storage. [2025-10-20T17:37:52.239763] [WARN ] [MAIN ] BMEZ: No existing data found, performing full refresh instead. [2025-10-20T17:37:52.239898] [INFO ] [MAIN ] BMEZ: Wiping existing primary data before full refresh. [2025-10-20T17:37:52.239938] [INFO ] [LOCAL ] BMEZ: Starting deletion of primary data. [2025-10-20T17:37:52.239978] [DEBUG] [LOCAL ] BMEZ: No primary data file found to delete. [2025-10-20T17:37:52.240009] [INFO ] [MAIN ] BMEZ: Starting full primary refresh. [2025-10-20T17:37:52.259942] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-07. [2025-10-20T17:37:52.351562] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:37:52.609202] [INFO ] [REMOTE ] BMEZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:37:53.051371] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-06. [2025-10-20T17:37:53.114290] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:37:53.851431] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-01. [2025-10-20T17:37:53.885428] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T17:37:54.651426] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-07. [2025-10-20T17:37:54.951453] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T17:37:55.451511] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-02. [2025-10-20T17:37:55.451987] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T17:37:56.251377] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-08. [2025-10-20T17:37:56.669095] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:37:57.051370] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-08. [2025-10-20T17:37:57.637178] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:37:57.665003] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T17:37:57.851372] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-09. [2025-10-20T17:37:58.651377] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-08. [2025-10-20T17:37:59.335613] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T17:37:59.451621] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-09. [2025-10-20T17:37:59.452726] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:38:00.252479] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-09. [2025-10-20T17:38:00.253485] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T17:38:00.864533] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:38:01.051383] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-03. [2025-10-20T17:38:01.851365] [INFO ] [REMOTE ] BME: Fetching primary data for 2025-10. [2025-10-20T17:38:02.418500] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:38:02.651406] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-04. [2025-10-20T17:38:02.756795] [INFO ] [REMOTE ] BME: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T17:38:02.763547] [INFO ] [REMOTE ] BME: Full fetch complete (15804 records). [2025-10-20T17:38:02.763745] [DEBUG] [VALIDATOR ] BME: Validating primary data. [2025-10-20T17:38:03.216193] [DEBUG] [VALIDATOR ] BME: All expected trading days are present. [2025-10-20T17:38:03.216288] [DEBUG] [VALIDATOR ] BME: Primary data validation passed. [2025-10-20T17:38:03.216851] [INFO ] [MAIN ] BME: Fetched primary data is valid. [2025-10-20T17:38:03.216928] [DEBUG] [LOCAL ] BME: Saving primary data. [2025-10-20T17:38:03.217759] [DEBUG] [LOCAL ] BME: Trimmed 15804 records to 11258 (keeping 200 days of data) [2025-10-20T17:38:03.287003] [DEBUG] [LOCAL ] BME: Primary data saved (11258 records). [2025-10-20T17:38:03.287140] [INFO ] [MAIN ] BME: Full history saved. [2025-10-20T17:38:03.287217] [INFO ] [MAIN ] Update Progress: 304/2367 symbols completed. BME took 18.6s. Estimated remaining: 4h 17m 45s. Rate: 8.0 symbols/min [2025-10-20T17:38:03.460370] [INFO ] [REMOTE ] BLX: Fetching primary data for 2025-10. [2025-10-20T17:38:03.465860] [INFO ] [MAIN ] BMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:03.490803] [DEBUG] [LOCAL ] BMI: Loaded 24957 existing records from primary storage. [2025-10-20T17:38:03.491075] [INFO ] [MAIN ] BMI: Loaded 24957 existing records. [2025-10-20T17:38:03.679706] [WARN ] [MAIN ] BMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:03.680952] [INFO ] [MAIN ] BMI: Wiping existing primary data before full refresh. [2025-10-20T17:38:03.681011] [INFO ] [LOCAL ] BMI: Starting deletion of primary data. [2025-10-20T17:38:03.681741] [DEBUG] [LOCAL ] BMI: Deleted primary data file. [2025-10-20T17:38:03.681800] [INFO ] [MAIN ] BMI: Starting full primary refresh. [2025-10-20T17:38:04.043258] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:38:04.063462] [INFO ] [REMOTE ] BMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:04.251370] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-01. [2025-10-20T17:38:04.510029] [INFO ] [REMOTE ] BLX: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:38:04.513479] [INFO ] [REMOTE ] BLX: Full fetch complete (22539 records). [2025-10-20T17:38:04.513582] [DEBUG] [VALIDATOR ] BLX: Validating primary data. [2025-10-20T17:38:04.926505] [DEBUG] [VALIDATOR ] BLX: All expected trading days are present. [2025-10-20T17:38:04.926613] [DEBUG] [VALIDATOR ] BLX: Primary data validation passed. [2025-10-20T17:38:04.927173] [INFO ] [MAIN ] BLX: Fetched primary data is valid. [2025-10-20T17:38:04.927233] [DEBUG] [LOCAL ] BLX: Saving primary data. [2025-10-20T17:38:04.928105] [DEBUG] [LOCAL ] BLX: Trimmed 22539 records to 15575 (keeping 200 days of data) [2025-10-20T17:38:05.038715] [DEBUG] [LOCAL ] BLX: Primary data saved (15575 records). [2025-10-20T17:38:05.038958] [INFO ] [MAIN ] BLX: Full history saved. [2025-10-20T17:38:05.039088] [INFO ] [MAIN ] Update Progress: 305/2367 symbols completed. BLX took 41.1s. Estimated remaining: 4h 16m 59s. Rate: 8.0 symbols/min [2025-10-20T17:38:05.062067] [INFO ] [REMOTE ] BMA: Fetching primary data for 2025-10. [2025-10-20T17:38:05.225905] [INFO ] [MAIN ] BMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:05.230535] [DEBUG] [LOCAL ] BMN: Loaded 2635 existing records from primary storage. [2025-10-20T17:38:05.230736] [INFO ] [MAIN ] BMN: Loaded 2635 existing records. [2025-10-20T17:38:05.439887] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:38:05.447387] [WARN ] [MAIN ] BMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:05.451805] [INFO ] [MAIN ] BMN: Wiping existing primary data before full refresh. [2025-10-20T17:38:05.452563] [INFO ] [LOCAL ] BMN: Starting deletion of primary data. [2025-10-20T17:38:05.452842] [DEBUG] [LOCAL ] BMN: Deleted primary data file. [2025-10-20T17:38:05.452902] [INFO ] [MAIN ] BMN: Starting full primary refresh. [2025-10-20T17:38:05.806909] [INFO ] [REMOTE ] BMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:05.851415] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-01. [2025-10-20T17:38:06.271038] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:38:06.589187] [INFO ] [REMOTE ] BMA: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T17:38:06.595545] [INFO ] [REMOTE ] BMA: Full fetch complete (52294 records). [2025-10-20T17:38:06.595735] [DEBUG] [VALIDATOR ] BMA: Validating primary data. [2025-10-20T17:38:06.658626] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-02. [2025-10-20T17:38:06.985215] [DEBUG] [VALIDATOR ] BMA: All expected trading days are present. [2025-10-20T17:38:06.985346] [DEBUG] [VALIDATOR ] BMA: Primary data validation passed. [2025-10-20T17:38:06.985974] [INFO ] [MAIN ] BMA: Fetched primary data is valid. [2025-10-20T17:38:06.986041] [DEBUG] [LOCAL ] BMA: Saving primary data. [2025-10-20T17:38:06.987221] [DEBUG] [LOCAL ] BMA: Trimmed 52294 records to 36470 (keeping 200 days of data) [2025-10-20T17:38:07.226858] [DEBUG] [LOCAL ] BMA: Primary data saved (36470 records). [2025-10-20T17:38:07.227072] [INFO ] [MAIN ] BMA: Full history saved. [2025-10-20T17:38:07.227179] [INFO ] [MAIN ] Update Progress: 306/2367 symbols completed. BMA took 38.8s. Estimated remaining: 4h 16m 15s. Rate: 8.0 symbols/min [2025-10-20T17:38:07.409428] [INFO ] [MAIN ] BMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:07.452841] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-05. [2025-10-20T17:38:07.455262] [DEBUG] [LOCAL ] BMO: Loaded 41935 existing records from primary storage. [2025-10-20T17:38:07.455826] [INFO ] [MAIN ] BMO: Loaded 41935 existing records. [2025-10-20T17:38:07.647807] [WARN ] [MAIN ] BMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:07.648237] [INFO ] [MAIN ] BMO: Wiping existing primary data before full refresh. [2025-10-20T17:38:07.648363] [INFO ] [LOCAL ] BMO: Starting deletion of primary data. [2025-10-20T17:38:07.648670] [DEBUG] [LOCAL ] BMO: Deleted primary data file. [2025-10-20T17:38:07.648844] [INFO ] [MAIN ] BMO: Starting full primary refresh. [2025-10-20T17:38:07.761097] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:38:08.043701] [INFO ] [REMOTE ] BMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:08.251364] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-03. [2025-10-20T17:38:08.594132] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:38:09.051370] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-06. [2025-10-20T17:38:09.257588] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:38:09.851436] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-01. [2025-10-20T17:38:10.114572] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T17:38:10.651437] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-07. [2025-10-20T17:38:11.157598] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T17:38:11.451371] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-02. [2025-10-20T17:38:11.602766] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:38:12.251391] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-02. [2025-10-20T17:38:12.673452] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:38:12.779571] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:38:13.051425] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-03. [2025-10-20T17:38:13.437043] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:38:13.851437] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-04. [2025-10-20T17:38:14.651382] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-04. [2025-10-20T17:38:15.093215] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:38:15.197051] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T17:38:15.451390] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-05. [2025-10-20T17:38:15.773193] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:38:16.251466] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-06. [2025-10-20T17:38:16.781257] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:38:17.051377] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-08. [2025-10-20T17:38:17.851420] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-07. [2025-10-20T17:38:18.147370] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:38:18.364312] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T17:38:18.651376] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-09. [2025-10-20T17:38:19.451385] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-03. [2025-10-20T17:38:19.957196] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:38:20.251358] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-05. [2025-10-20T17:38:20.710541] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:38:21.051426] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-04. [2025-10-20T17:38:21.158031] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:38:21.851453] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-06. [2025-10-20T17:38:22.651411] [INFO ] [REMOTE ] BMEZ: Fetching primary data for 2025-10. [2025-10-20T17:38:22.762046] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:38:22.951583] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T17:38:23.451378] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-07. [2025-10-20T17:38:23.585818] [INFO ] [REMOTE ] BMEZ: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T17:38:23.590441] [INFO ] [REMOTE ] BMEZ: Full fetch complete (39854 records). [2025-10-20T17:38:23.590618] [DEBUG] [VALIDATOR ] BMEZ: Validating primary data. [2025-10-20T17:38:23.988446] [DEBUG] [VALIDATOR ] BMEZ: All expected trading days are present. [2025-10-20T17:38:23.988568] [DEBUG] [VALIDATOR ] BMEZ: Primary data validation passed. [2025-10-20T17:38:23.989297] [INFO ] [MAIN ] BMEZ: Fetched primary data is valid. [2025-10-20T17:38:23.989363] [DEBUG] [LOCAL ] BMEZ: Saving primary data. [2025-10-20T17:38:23.990844] [DEBUG] [LOCAL ] BMEZ: Trimmed 39854 records to 25392 (keeping 200 days of data) [2025-10-20T17:38:24.169883] [DEBUG] [LOCAL ] BMEZ: Primary data saved (25392 records). [2025-10-20T17:38:24.170088] [INFO ] [MAIN ] BMEZ: Full history saved. [2025-10-20T17:38:24.170184] [INFO ] [MAIN ] Update Progress: 307/2367 symbols completed. BMEZ took 32.1s. Estimated remaining: 4h 17m 12s. Rate: 8.0 symbols/min [2025-10-20T17:38:24.254383] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-08. [2025-10-20T17:38:24.366795] [INFO ] [MAIN ] BMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:24.413953] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T17:38:24.437459] [DEBUG] [LOCAL ] BMY: Loaded 54537 existing records from primary storage. [2025-10-20T17:38:24.437816] [INFO ] [MAIN ] BMY: Loaded 54537 existing records. [2025-10-20T17:38:24.618819] [WARN ] [MAIN ] BMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:24.619135] [INFO ] [MAIN ] BMY: Wiping existing primary data before full refresh. [2025-10-20T17:38:24.619186] [INFO ] [LOCAL ] BMY: Starting deletion of primary data. [2025-10-20T17:38:24.619822] [DEBUG] [LOCAL ] BMY: Deleted primary data file. [2025-10-20T17:38:24.620123] [INFO ] [MAIN ] BMY: Starting full primary refresh. [2025-10-20T17:38:24.804611] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T17:38:24.989308] [INFO ] [REMOTE ] BMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:25.051340] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-01. [2025-10-20T17:38:25.851372] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-05. [2025-10-20T17:38:26.651373] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-09. [2025-10-20T17:38:27.112623] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T17:38:27.274661] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T17:38:27.299715] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T17:38:27.451362] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-08. [2025-10-20T17:38:28.251419] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-02. [2025-10-20T17:38:28.686534] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T17:38:29.051368] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-09. [2025-10-20T17:38:29.851359] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-06. [2025-10-20T17:38:30.232052] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:38:30.299470] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T17:38:30.651391] [INFO ] [REMOTE ] BMN: Fetching primary data for 2025-10. [2025-10-20T17:38:31.175373] [INFO ] [REMOTE ] BMN: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T17:38:31.176876] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:38:31.178026] [INFO ] [REMOTE ] BMN: Full fetch complete (3672 records). [2025-10-20T17:38:31.185246] [DEBUG] [VALIDATOR ] BMN: Validating primary data. [2025-10-20T17:38:31.456631] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-03. [2025-10-20T17:38:31.560890] [DEBUG] [VALIDATOR ] BMN: All expected trading days are present. [2025-10-20T17:38:31.560968] [DEBUG] [VALIDATOR ] BMN: Primary data validation passed. [2025-10-20T17:38:31.561096] [INFO ] [MAIN ] BMN: Fetched primary data is valid. [2025-10-20T17:38:31.561133] [DEBUG] [LOCAL ] BMN: Saving primary data. [2025-10-20T17:38:31.561473] [DEBUG] [LOCAL ] BMN: Trimmed 3672 records to 2592 (keeping 200 days of data) [2025-10-20T17:38:31.577498] [DEBUG] [LOCAL ] BMN: Primary data saved (2592 records). [2025-10-20T17:38:31.577603] [INFO ] [MAIN ] BMN: Full history saved. [2025-10-20T17:38:31.577701] [INFO ] [MAIN ] Update Progress: 308/2367 symbols completed. BMN took 26.5s. Estimated remaining: 4h 17m 4s. Rate: 8.0 symbols/min [2025-10-20T17:38:31.746518] [INFO ] [MAIN ] BN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:31.794701] [DEBUG] [LOCAL ] BN: Loaded 53768 existing records from primary storage. [2025-10-20T17:38:31.795086] [INFO ] [MAIN ] BN: Loaded 53768 existing records. [2025-10-20T17:38:31.984393] [WARN ] [MAIN ] BN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:31.984693] [INFO ] [MAIN ] BN: Wiping existing primary data before full refresh. [2025-10-20T17:38:31.984751] [INFO ] [LOCAL ] BN: Starting deletion of primary data. [2025-10-20T17:38:31.985210] [DEBUG] [LOCAL ] BN: Deleted primary data file. [2025-10-20T17:38:31.985266] [INFO ] [MAIN ] BN: Starting full primary refresh. [2025-10-20T17:38:32.260920] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-07. [2025-10-20T17:38:32.347048] [INFO ] [REMOTE ] BN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:33.051379] [INFO ] [REMOTE ] BMI: Fetching primary data for 2025-10. [2025-10-20T17:38:33.453333] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:38:33.557181] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T17:38:33.851411] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-08. [2025-10-20T17:38:34.470080] [INFO ] [REMOTE ] BMI: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:38:34.474429] [INFO ] [REMOTE ] BMI: Full fetch complete (33800 records). [2025-10-20T17:38:34.474566] [DEBUG] [VALIDATOR ] BMI: Validating primary data. [2025-10-20T17:38:34.660528] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-04. [2025-10-20T17:38:34.849547] [DEBUG] [VALIDATOR ] BMI: All expected trading days are present. [2025-10-20T17:38:34.849624] [DEBUG] [VALIDATOR ] BMI: Primary data validation passed. [2025-10-20T17:38:34.850108] [INFO ] [MAIN ] BMI: Fetched primary data is valid. [2025-10-20T17:38:34.850212] [DEBUG] [LOCAL ] BMI: Saving primary data. [2025-10-20T17:38:34.851017] [DEBUG] [LOCAL ] BMI: Trimmed 33800 records to 24793 (keeping 200 days of data) [2025-10-20T17:38:35.017509] [DEBUG] [LOCAL ] BMI: Primary data saved (24793 records). [2025-10-20T17:38:35.017669] [INFO ] [MAIN ] BMI: Full history saved. [2025-10-20T17:38:35.017766] [INFO ] [MAIN ] Update Progress: 309/2367 symbols completed. BMI took 31.7s. Estimated remaining: 4h 16m 29s. Rate: 8.0 symbols/min [2025-10-20T17:38:35.211550] [INFO ] [MAIN ] BNED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:35.239199] [DEBUG] [LOCAL ] BNED: Loaded 28914 existing records from primary storage. [2025-10-20T17:38:35.239478] [INFO ] [MAIN ] BNED: Loaded 28914 existing records. [2025-10-20T17:38:35.421273] [WARN ] [MAIN ] BNED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:35.421640] [INFO ] [MAIN ] BNED: Wiping existing primary data before full refresh. [2025-10-20T17:38:35.421835] [INFO ] [LOCAL ] BNED: Starting deletion of primary data. [2025-10-20T17:38:35.422061] [DEBUG] [LOCAL ] BNED: Deleted primary data file. [2025-10-20T17:38:35.422214] [INFO ] [MAIN ] BNED: Starting full primary refresh. [2025-10-20T17:38:35.453621] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-01. [2025-10-20T17:38:35.557375] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:38:35.854142] [INFO ] [REMOTE ] BNED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:36.251363] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-09. [2025-10-20T17:38:36.644247] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:38:37.051494] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-05. [2025-10-20T17:38:37.146161] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:38:37.783929] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:38:37.851473] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-02. [2025-10-20T17:38:38.651387] [INFO ] [REMOTE ] BMO: Fetching primary data for 2025-10. [2025-10-20T17:38:39.451440] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-01. [2025-10-20T17:38:39.615538] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-05 (2.56 seconds). [2025-10-20T17:38:39.658068] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T17:38:40.168239] [INFO ] [REMOTE ] BMO: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T17:38:40.174504] [INFO ] [REMOTE ] BMO: Full fetch complete (60042 records). [2025-10-20T17:38:40.174626] [DEBUG] [VALIDATOR ] BMO: Validating primary data. [2025-10-20T17:38:40.256505] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-06. [2025-10-20T17:38:40.570661] [DEBUG] [VALIDATOR ] BMO: All expected trading days are present. [2025-10-20T17:38:40.570835] [DEBUG] [VALIDATOR ] BMO: Primary data validation passed. [2025-10-20T17:38:40.571531] [INFO ] [MAIN ] BMO: Fetched primary data is valid. [2025-10-20T17:38:40.571576] [DEBUG] [LOCAL ] BMO: Saving primary data. [2025-10-20T17:38:40.572879] [DEBUG] [LOCAL ] BMO: Trimmed 60042 records to 41223 (keeping 200 days of data) [2025-10-20T17:38:40.830534] [DEBUG] [LOCAL ] BMO: Primary data saved (41223 records). [2025-10-20T17:38:40.830695] [INFO ] [MAIN ] BMO: Full history saved. [2025-10-20T17:38:40.830788] [INFO ] [MAIN ] Update Progress: 310/2367 symbols completed. BMO took 33.6s. Estimated remaining: 4h 16m 11s. Rate: 8.0 symbols/min [2025-10-20T17:38:40.990369] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:38:41.056576] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-02. [2025-10-20T17:38:41.085625] [INFO ] [MAIN ] BNH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:41.096150] [DEBUG] [LOCAL ] BNH: Loaded 5065 existing records from primary storage. [2025-10-20T17:38:41.096389] [INFO ] [MAIN ] BNH: Loaded 5065 existing records. [2025-10-20T17:38:41.336161] [WARN ] [MAIN ] BNH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:41.336389] [INFO ] [MAIN ] BNH: Wiping existing primary data before full refresh. [2025-10-20T17:38:41.336436] [INFO ] [LOCAL ] BNH: Starting deletion of primary data. [2025-10-20T17:38:41.336600] [DEBUG] [LOCAL ] BNH: Deleted primary data file. [2025-10-20T17:38:41.336693] [INFO ] [MAIN ] BNH: Starting full primary refresh. [2025-10-20T17:38:41.752532] [INFO ] [REMOTE ] BNH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:41.820271] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T17:38:41.851477] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-01. [2025-10-20T17:38:41.918091] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:38:42.540286] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T17:38:42.651441] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-03. [2025-10-20T17:38:43.451467] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-02. [2025-10-20T17:38:43.997988] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T17:38:44.251379] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-07. [2025-10-20T17:38:44.455053] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T17:38:45.051434] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-04. [2025-10-20T17:38:45.851460] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-03. [2025-10-20T17:38:46.015752] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T17:38:46.326742] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T17:38:46.651374] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-08. [2025-10-20T17:38:46.804176] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:38:47.451440] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-04. [2025-10-20T17:38:48.047792] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T17:38:48.251414] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-05. [2025-10-20T17:38:48.453126] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T17:38:49.028750] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T17:38:49.051403] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-03. [2025-10-20T17:38:49.851442] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-06. [2025-10-20T17:38:50.394893] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T17:38:50.479759] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T17:38:50.651439] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-09. [2025-10-20T17:38:51.451436] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-07. [2025-10-20T17:38:51.862919] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-10-20T17:38:52.251396] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-04. [2025-10-20T17:38:52.875483] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T17:38:53.051414] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-08. [2025-10-20T17:38:53.362605] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T17:38:53.683180] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T17:38:53.851454] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-05. [2025-10-20T17:38:54.651459] [INFO ] [REMOTE ] BMY: Fetching primary data for 2025-10. [2025-10-20T17:38:55.161597] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T17:38:55.451413] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-09. [2025-10-20T17:38:55.477317] [INFO ] [REMOTE ] BMY: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:38:55.486065] [INFO ] [REMOTE ] BMY: Full fetch complete (77937 records). [2025-10-20T17:38:55.486247] [DEBUG] [VALIDATOR ] BMY: Validating primary data. [2025-10-20T17:38:55.901110] [DEBUG] [VALIDATOR ] BMY: All expected trading days are present. [2025-10-20T17:38:55.901939] [DEBUG] [VALIDATOR ] BMY: Primary data validation passed. [2025-10-20T17:38:55.903315] [INFO ] [MAIN ] BMY: Fetched primary data is valid. [2025-10-20T17:38:55.903424] [DEBUG] [LOCAL ] BMY: Saving primary data. [2025-10-20T17:38:55.902231] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:38:55.907586] [DEBUG] [LOCAL ] BMY: Trimmed 77937 records to 53757 (keeping 200 days of data) [2025-10-20T17:38:56.249892] [DEBUG] [LOCAL ] BMY: Primary data saved (53757 records). [2025-10-20T17:38:56.250033] [INFO ] [MAIN ] BMY: Full history saved. [2025-10-20T17:38:56.250108] [INFO ] [MAIN ] Update Progress: 311/2367 symbols completed. BMY took 32.1s. Estimated remaining: 4h 16m 56s. Rate: 8.0 symbols/min [2025-10-20T17:38:56.260390] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-05. [2025-10-20T17:38:56.421166] [INFO ] [MAIN ] BNJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:38:56.426224] [DEBUG] [LOCAL ] BNJ: Loaded 2639 existing records from primary storage. [2025-10-20T17:38:56.426377] [INFO ] [MAIN ] BNJ: Loaded 2639 existing records. [2025-10-20T17:38:56.603067] [WARN ] [MAIN ] BNJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:38:56.603302] [INFO ] [MAIN ] BNJ: Wiping existing primary data before full refresh. [2025-10-20T17:38:56.603350] [INFO ] [LOCAL ] BNJ: Starting deletion of primary data. [2025-10-20T17:38:56.603507] [DEBUG] [LOCAL ] BNJ: Deleted primary data file. [2025-10-20T17:38:56.603560] [INFO ] [MAIN ] BNJ: Starting full primary refresh. [2025-10-20T17:38:56.952126] [INFO ] [REMOTE ] BNJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:38:57.051406] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-01. [2025-10-20T17:38:57.650518] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T17:38:57.851403] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-02. [2025-10-20T17:38:58.069755] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:38:58.299934] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:38:58.651441] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-06. [2025-10-20T17:38:59.451466] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-06. [2025-10-20T17:39:00.251412] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-03. [2025-10-20T17:39:00.399324] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T17:39:00.442210] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:39:00.640570] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:39:01.051426] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-07. [2025-10-20T17:39:01.851455] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-04. [2025-10-20T17:39:02.338954] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:39:02.424158] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T17:39:02.651443] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-07. [2025-10-20T17:39:03.451441] [INFO ] [REMOTE ] BNH: Fetching primary data for 2025-10. [2025-10-20T17:39:04.251403] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-05. [2025-10-20T17:39:04.296724] [INFO ] [REMOTE ] BNH: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T17:39:04.298267] [INFO ] [REMOTE ] BNH: Full fetch complete (6804 records). [2025-10-20T17:39:04.298376] [DEBUG] [VALIDATOR ] BNH: Validating primary data. [2025-10-20T17:39:04.695801] [DEBUG] [VALIDATOR ] BNH: All expected trading days are present. [2025-10-20T17:39:04.695917] [DEBUG] [VALIDATOR ] BNH: Primary data validation passed. [2025-10-20T17:39:04.696226] [INFO ] [MAIN ] BNH: Fetched primary data is valid. [2025-10-20T17:39:04.696308] [DEBUG] [LOCAL ] BNH: Saving primary data. [2025-10-20T17:39:04.697269] [DEBUG] [LOCAL ] BNH: Trimmed 6804 records to 4731 (keeping 200 days of data) [2025-10-20T17:39:04.772725] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T17:39:04.771941] [DEBUG] [LOCAL ] BNH: Primary data saved (4731 records). [2025-10-20T17:39:04.776526] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T17:39:04.778857] [INFO ] [MAIN ] BNH: Full history saved. [2025-10-20T17:39:04.779227] [INFO ] [MAIN ] Update Progress: 312/2367 symbols completed. BNH took 23.9s. Estimated remaining: 4h 16m 55s. Rate: 8.0 symbols/min [2025-10-20T17:39:04.971337] [INFO ] [MAIN ] BNL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:05.016390] [DEBUG] [LOCAL ] BNL: Loaded 47981 existing records from primary storage. [2025-10-20T17:39:05.016780] [INFO ] [MAIN ] BNL: Loaded 47981 existing records. [2025-10-20T17:39:05.056759] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-08. [2025-10-20T17:39:05.202166] [WARN ] [MAIN ] BNL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:05.202404] [INFO ] [MAIN ] BNL: Wiping existing primary data before full refresh. [2025-10-20T17:39:05.202454] [INFO ] [LOCAL ] BNL: Starting deletion of primary data. [2025-10-20T17:39:05.202697] [DEBUG] [LOCAL ] BNL: Deleted primary data file. [2025-10-20T17:39:05.202755] [INFO ] [MAIN ] BNL: Starting full primary refresh. [2025-10-20T17:39:05.558121] [INFO ] [REMOTE ] BNL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:05.851403] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-08. [2025-10-20T17:39:06.248871] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T17:39:06.651480] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-06. [2025-10-20T17:39:07.082002] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T17:39:07.451418] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-01. [2025-10-20T17:39:07.687016] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T17:39:08.251388] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-09. [2025-10-20T17:39:08.791908] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:39:09.051376] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-09. [2025-10-20T17:39:09.377421] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T17:39:09.851457] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-02. [2025-10-20T17:39:10.651464] [INFO ] [REMOTE ] BNED: Fetching primary data for 2025-10. [2025-10-20T17:39:11.222775] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:39:11.417141] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T17:39:11.451351] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-03. [2025-10-20T17:39:12.022148] [INFO ] [REMOTE ] BNED: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T17:39:12.025786] [INFO ] [REMOTE ] BNED: Full fetch complete (42265 records). [2025-10-20T17:39:12.025914] [DEBUG] [VALIDATOR ] BNED: Validating primary data. [2025-10-20T17:39:12.256579] [INFO ] [REMOTE ] BN: Fetching primary data for 2025-10. [2025-10-20T17:39:12.419841] [DEBUG] [VALIDATOR ] BNED: All expected trading days are present. [2025-10-20T17:39:12.419922] [DEBUG] [VALIDATOR ] BNED: Primary data validation passed. [2025-10-20T17:39:12.420454] [INFO ] [MAIN ] BNED: Fetched primary data is valid. [2025-10-20T17:39:12.420495] [DEBUG] [LOCAL ] BNED: Saving primary data. [2025-10-20T17:39:12.421390] [DEBUG] [LOCAL ] BNED: Trimmed 42265 records to 28483 (keeping 200 days of data) [2025-10-20T17:39:12.586547] [DEBUG] [LOCAL ] BNED: Primary data saved (28483 records). [2025-10-20T17:39:12.586740] [INFO ] [MAIN ] BNED: Full history saved. [2025-10-20T17:39:12.586831] [INFO ] [MAIN ] Update Progress: 313/2367 symbols completed. BNED took 37.6s. Estimated remaining: 4h 16m 49s. Rate: 8.0 symbols/min [2025-10-20T17:39:12.759817] [INFO ] [MAIN ] BNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:12.805446] [DEBUG] [LOCAL ] BNS: Loaded 52255 existing records from primary storage. [2025-10-20T17:39:12.805773] [INFO ] [MAIN ] BNS: Loaded 52255 existing records. [2025-10-20T17:39:13.007138] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:39:13.013691] [WARN ] [MAIN ] BNS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:13.013948] [INFO ] [MAIN ] BNS: Wiping existing primary data before full refresh. [2025-10-20T17:39:13.014020] [INFO ] [LOCAL ] BNS: Starting deletion of primary data. [2025-10-20T17:39:13.017477] [DEBUG] [LOCAL ] BNS: Deleted primary data file. [2025-10-20T17:39:13.023960] [INFO ] [MAIN ] BNS: Starting full primary refresh. [2025-10-20T17:39:13.055504] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-07. [2025-10-20T17:39:13.391724] [INFO ] [REMOTE ] BNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:13.577815] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T17:39:13.851339] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-01. [2025-10-20T17:39:14.345132] [INFO ] [REMOTE ] BN: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:39:14.353203] [INFO ] [REMOTE ] BN: Full fetch complete (76951 records). [2025-10-20T17:39:14.353359] [DEBUG] [VALIDATOR ] BN: Validating primary data. [2025-10-20T17:39:14.656549] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-08. [2025-10-20T17:39:14.743830] [DEBUG] [VALIDATOR ] BN: All expected trading days are present. [2025-10-20T17:39:14.743911] [DEBUG] [VALIDATOR ] BN: Primary data validation passed. [2025-10-20T17:39:14.744692] [INFO ] [MAIN ] BN: Fetched primary data is valid. [2025-10-20T17:39:14.744756] [DEBUG] [LOCAL ] BN: Saving primary data. [2025-10-20T17:39:14.746228] [DEBUG] [LOCAL ] BN: Trimmed 76951 records to 52986 (keeping 200 days of data) [2025-10-20T17:39:15.065289] [DEBUG] [LOCAL ] BN: Primary data saved (52986 records). [2025-10-20T17:39:15.065412] [INFO ] [MAIN ] BN: Full history saved. [2025-10-20T17:39:15.065760] [INFO ] [MAIN ] Update Progress: 314/2367 symbols completed. BN took 43.5s. Estimated remaining: 4h 16m 9s. Rate: 8.0 symbols/min [2025-10-20T17:39:15.238735] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:39:15.264396] [INFO ] [MAIN ] BNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:15.271881] [DEBUG] [LOCAL ] BNT: Loaded 3594 existing records from primary storage. [2025-10-20T17:39:15.272093] [INFO ] [MAIN ] BNT: Loaded 3594 existing records. [2025-10-20T17:39:15.449186] [WARN ] [MAIN ] BNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:15.449439] [INFO ] [MAIN ] BNT: Wiping existing primary data before full refresh. [2025-10-20T17:39:15.449490] [INFO ] [LOCAL ] BNT: Starting deletion of primary data. [2025-10-20T17:39:15.449615] [DEBUG] [LOCAL ] BNT: Deleted primary data file. [2025-10-20T17:39:15.449697] [INFO ] [MAIN ] BNT: Starting full primary refresh. [2025-10-20T17:39:15.461079] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-04. [2025-10-20T17:39:15.834840] [INFO ] [REMOTE ] BNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:15.852341] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:39:16.251391] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-09. [2025-10-20T17:39:16.698946] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:39:16.885524] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:39:17.051374] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-02. [2025-10-20T17:39:17.851359] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-01. [2025-10-20T17:39:18.468429] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T17:39:18.645222] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:39:18.654590] [INFO ] [REMOTE ] BNJ: Fetching primary data for 2025-10. [2025-10-20T17:39:19.247381] [INFO ] [REMOTE ] BNJ: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T17:39:19.248529] [INFO ] [REMOTE ] BNJ: Full fetch complete (3894 records). [2025-10-20T17:39:19.248608] [DEBUG] [VALIDATOR ] BNJ: Validating primary data. [2025-10-20T17:39:19.456497] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-05. [2025-10-20T17:39:19.646047] [DEBUG] [VALIDATOR ] BNJ: All expected trading days are present. [2025-10-20T17:39:19.646134] [DEBUG] [VALIDATOR ] BNJ: Primary data validation passed. [2025-10-20T17:39:19.646284] [INFO ] [MAIN ] BNJ: Fetched primary data is valid. [2025-10-20T17:39:19.646323] [DEBUG] [LOCAL ] BNJ: Saving primary data. [2025-10-20T17:39:19.646809] [DEBUG] [LOCAL ] BNJ: Trimmed 3894 records to 2531 (keeping 200 days of data) [2025-10-20T17:39:19.666392] [DEBUG] [LOCAL ] BNJ: Primary data saved (2531 records). [2025-10-20T17:39:19.666571] [INFO ] [MAIN ] BNJ: Full history saved. [2025-10-20T17:39:19.666700] [INFO ] [MAIN ] Update Progress: 315/2367 symbols completed. BNJ took 23.4s. Estimated remaining: 4h 15m 43s. Rate: 8.0 symbols/min [2025-10-20T17:39:19.839215] [INFO ] [MAIN ] BNY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:19.847792] [DEBUG] [LOCAL ] BNY: Loaded 7307 existing records from primary storage. [2025-10-20T17:39:19.847981] [INFO ] [MAIN ] BNY: Loaded 7307 existing records. [2025-10-20T17:39:20.029799] [WARN ] [MAIN ] BNY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:20.030078] [INFO ] [MAIN ] BNY: Wiping existing primary data before full refresh. [2025-10-20T17:39:20.030128] [INFO ] [LOCAL ] BNY: Starting deletion of primary data. [2025-10-20T17:39:20.030296] [DEBUG] [LOCAL ] BNY: Deleted primary data file. [2025-10-20T17:39:20.030352] [INFO ] [MAIN ] BNY: Starting full primary refresh. [2025-10-20T17:39:20.259358] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-02. [2025-10-20T17:39:20.400077] [INFO ] [REMOTE ] BNY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:20.794707] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T17:39:20.945249] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T17:39:21.051361] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-03. [2025-10-20T17:39:21.851428] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-01. [2025-10-20T17:39:22.405935] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T17:39:22.651424] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-02. [2025-10-20T17:39:22.797186] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:39:23.090259] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T17:39:23.451370] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-03. [2025-10-20T17:39:23.952524] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:39:24.251361] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-06. [2025-10-20T17:39:25.051380] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-04. [2025-10-20T17:39:25.770103] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T17:39:25.851334] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-03. [2025-10-20T17:39:26.582831] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T17:39:26.651364] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-04. [2025-10-20T17:39:27.070449] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:39:27.165205] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T17:39:27.451430] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-05. [2025-10-20T17:39:28.154300] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:39:28.251366] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-04. [2025-10-20T17:39:29.000781] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:39:29.051382] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-05. [2025-10-20T17:39:29.851406] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-05. [2025-10-20T17:39:30.394709] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T17:39:30.421848] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T17:39:30.651384] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-06. [2025-10-20T17:39:31.040012] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T17:39:31.451476] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-07. [2025-10-20T17:39:32.015087] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T17:39:32.251426] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-07. [2025-10-20T17:39:33.051390] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-08. [2025-10-20T17:39:33.537072] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T17:39:33.706451] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T17:39:33.851400] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-09. [2025-10-20T17:39:34.469485] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T17:39:34.651393] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-08. [2025-10-20T17:39:35.451411] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-06. [2025-10-20T17:39:36.000459] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:39:36.251367] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-09. [2025-10-20T17:39:37.051451] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-06. [2025-10-20T17:39:37.235256] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:39:37.368499] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T17:39:37.532139] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:39:37.851479] [INFO ] [REMOTE ] BNT: Fetching primary data for 2025-10. [2025-10-20T17:39:38.524849] [INFO ] [REMOTE ] BNT: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:39:38.526157] [INFO ] [REMOTE ] BNT: Full fetch complete (4796 records). [2025-10-20T17:39:38.526223] [DEBUG] [VALIDATOR ] BNT: Validating primary data. [2025-10-20T17:39:38.663857] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-07. [2025-10-20T17:39:38.889298] [DEBUG] [VALIDATOR ] BNT: All expected trading days are present. [2025-10-20T17:39:38.889379] [DEBUG] [VALIDATOR ] BNT: Primary data validation passed. [2025-10-20T17:39:38.889540] [INFO ] [MAIN ] BNT: Fetched primary data is valid. [2025-10-20T17:39:38.889578] [DEBUG] [LOCAL ] BNT: Saving primary data. [2025-10-20T17:39:38.889958] [DEBUG] [LOCAL ] BNT: Trimmed 4796 records to 3553 (keeping 200 days of data) [2025-10-20T17:39:38.911986] [DEBUG] [LOCAL ] BNT: Primary data saved (3553 records). [2025-10-20T17:39:38.912118] [INFO ] [MAIN ] BNT: Full history saved. [2025-10-20T17:39:38.912198] [INFO ] [MAIN ] Update Progress: 316/2367 symbols completed. BNT took 23.8s. Estimated remaining: 4h 16m 52s. Rate: 8.0 symbols/min [2025-10-20T17:39:39.081062] [INFO ] [MAIN ] BOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:39.099403] [DEBUG] [LOCAL ] BOC: Loaded 18798 existing records from primary storage. [2025-10-20T17:39:39.099691] [INFO ] [MAIN ] BOC: Loaded 18798 existing records. [2025-10-20T17:39:39.297033] [WARN ] [MAIN ] BOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:39.297341] [INFO ] [MAIN ] BOC: Wiping existing primary data before full refresh. [2025-10-20T17:39:39.297395] [INFO ] [LOCAL ] BOC: Starting deletion of primary data. [2025-10-20T17:39:39.297592] [DEBUG] [LOCAL ] BOC: Deleted primary data file. [2025-10-20T17:39:39.297633] [INFO ] [MAIN ] BOC: Starting full primary refresh. [2025-10-20T17:39:39.456536] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-07. [2025-10-20T17:39:39.613302] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:39:39.705376] [INFO ] [REMOTE ] BOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:40.251391] [INFO ] [REMOTE ] BNL: Fetching primary data for 2025-10. [2025-10-20T17:39:41.001448] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:39:41.051429] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-08. [2025-10-20T17:39:41.851387] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-01. [2025-10-20T17:39:41.879944] [INFO ] [REMOTE ] BNL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:39:41.887236] [INFO ] [REMOTE ] BNL: Full fetch complete (67279 records). [2025-10-20T17:39:41.887387] [DEBUG] [VALIDATOR ] BNL: Validating primary data. [2025-10-20T17:39:42.263590] [DEBUG] [VALIDATOR ] BNL: All expected trading days are present. [2025-10-20T17:39:42.263689] [DEBUG] [VALIDATOR ] BNL: Primary data validation passed. [2025-10-20T17:39:42.264315] [INFO ] [MAIN ] BNL: Fetched primary data is valid. [2025-10-20T17:39:42.264356] [DEBUG] [LOCAL ] BNL: Saving primary data. [2025-10-20T17:39:42.265558] [DEBUG] [LOCAL ] BNL: Trimmed 67279 records to 47210 (keeping 200 days of data) [2025-10-20T17:39:42.546544] [DEBUG] [LOCAL ] BNL: Primary data saved (47210 records). [2025-10-20T17:39:42.546955] [INFO ] [MAIN ] BNL: Full history saved. [2025-10-20T17:39:42.547042] [INFO ] [MAIN ] Update Progress: 317/2367 symbols completed. BNL took 37.8s. Estimated remaining: 4h 16m 19s. Rate: 8.0 symbols/min [2025-10-20T17:39:42.588461] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T17:39:42.651414] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-08. [2025-10-20T17:39:42.974607] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:39:43.344915] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:39:43.451472] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-02. [2025-10-20T17:39:44.251426] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-09. [2025-10-20T17:39:44.410162] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:39:45.051379] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-03. [2025-10-20T17:39:45.711374] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:39:45.799324] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T17:39:45.851462] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-09. [2025-10-20T17:39:46.589074] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T17:39:46.651392] [INFO ] [REMOTE ] BNY: Fetching primary data for 2025-10. [2025-10-20T17:39:47.356108] [INFO ] [REMOTE ] BNY: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T17:39:47.358789] [INFO ] [REMOTE ] BNY: Full fetch complete (10426 records). [2025-10-20T17:39:47.358871] [DEBUG] [VALIDATOR ] BNY: Validating primary data. [2025-10-20T17:39:47.458875] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-04. [2025-10-20T17:39:47.746376] [DEBUG] [VALIDATOR ] BNY: All expected trading days are present. [2025-10-20T17:39:47.746482] [DEBUG] [VALIDATOR ] BNY: Primary data validation passed. [2025-10-20T17:39:47.746921] [INFO ] [MAIN ] BNY: Fetched primary data is valid. [2025-10-20T17:39:47.746988] [DEBUG] [LOCAL ] BNY: Saving primary data. [2025-10-20T17:39:47.747566] [DEBUG] [LOCAL ] BNY: Trimmed 10426 records to 7267 (keeping 200 days of data) [2025-10-20T17:39:47.796287] [DEBUG] [LOCAL ] BNY: Primary data saved (7267 records). [2025-10-20T17:39:47.796497] [INFO ] [MAIN ] BNY: Full history saved. [2025-10-20T17:39:47.796625] [INFO ] [MAIN ] Update Progress: 318/2367 symbols completed. BNY took 28.1s. Estimated remaining: 4h 15m 57s. Rate: 8.0 symbols/min [2025-10-20T17:39:48.251458] [INFO ] [REMOTE ] BNS: Fetching primary data for 2025-10. [2025-10-20T17:39:48.664569] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:39:49.051429] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-05. [2025-10-20T17:39:49.827346] [INFO ] [REMOTE ] BNS: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:39:49.836275] [INFO ] [REMOTE ] BNS: Full fetch complete (74670 records). [2025-10-20T17:39:49.836535] [DEBUG] [VALIDATOR ] BNS: Validating primary data. [2025-10-20T17:39:50.184637] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:39:50.215884] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-06. [2025-10-20T17:39:50.298498] [DEBUG] [VALIDATOR ] BNS: All expected trading days are present. [2025-10-20T17:39:50.298626] [DEBUG] [VALIDATOR ] BNS: Primary data validation passed. [2025-10-20T17:39:50.299868] [INFO ] [MAIN ] BNS: Fetched primary data is valid. [2025-10-20T17:39:50.299939] [DEBUG] [LOCAL ] BNS: Saving primary data. [2025-10-20T17:39:50.302200] [DEBUG] [LOCAL ] BNS: Trimmed 74670 records to 51449 (keeping 200 days of data) [2025-10-20T17:39:50.638162] [DEBUG] [LOCAL ] BNS: Primary data saved (51449 records). [2025-10-20T17:39:50.638397] [INFO ] [MAIN ] BNS: Full history saved. [2025-10-20T17:39:50.638507] [INFO ] [MAIN ] Update Progress: 319/2367 symbols completed. BNS took 38.1s. Estimated remaining: 4h 15m 20s. Rate: 8.0 symbols/min [2025-10-20T17:39:51.323613] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:39:51.328830] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-07. [2025-10-20T17:39:52.307173] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:39:52.311921] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-08. [2025-10-20T17:39:53.445389] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:39:53.452080] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-09. [2025-10-20T17:39:54.390452] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:39:54.394999] [INFO ] [REMOTE ] BOC: Fetching primary data for 2025-10. [2025-10-20T17:39:55.409028] [INFO ] [REMOTE ] BOC: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:39:55.412819] [INFO ] [REMOTE ] BOC: Full fetch complete (25815 records). [2025-10-20T17:39:55.412911] [DEBUG] [VALIDATOR ] BOC: Validating primary data. [2025-10-20T17:39:55.814306] [DEBUG] [VALIDATOR ] BOC: All expected trading days are present. [2025-10-20T17:39:55.814412] [DEBUG] [VALIDATOR ] BOC: Primary data validation passed. [2025-10-20T17:39:55.815130] [INFO ] [MAIN ] BOC: Fetched primary data is valid. [2025-10-20T17:39:55.815194] [DEBUG] [LOCAL ] BOC: Saving primary data. [2025-10-20T17:39:55.816045] [DEBUG] [LOCAL ] BOC: Trimmed 25815 records to 18418 (keeping 200 days of data) [2025-10-20T17:39:55.932776] [DEBUG] [LOCAL ] BOC: Primary data saved (18418 records). [2025-10-20T17:39:55.933061] [INFO ] [MAIN ] BOC: Full history saved. [2025-10-20T17:39:55.933184] [INFO ] [MAIN ] Update Progress: 320/2367 symbols completed. BOC took 17.0s. Estimated remaining: 4h 14m 58s. Rate: 8.0 symbols/min [2025-10-20T17:39:56.599970] [INFO ] [MAIN ] BOE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.730820] [INFO ] [MAIN ] BOOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.713693] [INFO ] [MAIN ] BOND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.732497] [INFO ] [MAIN ] BOH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:39:56.757316] [DEBUG] [LOCAL ] BOE: Loaded 18435 existing records from primary storage. [2025-10-20T17:39:56.757872] [INFO ] [MAIN ] BOE: Loaded 18435 existing records. [2025-10-20T17:39:56.910045] [DEBUG] [LOCAL ] BOND: Loaded 32938 existing records from primary storage. [2025-10-20T17:39:56.969517] [INFO ] [MAIN ] BOND: Loaded 32938 existing records. [2025-10-20T17:39:56.969259] [DEBUG] [LOCAL ] BOOT: Loaded 40347 existing records from primary storage. [2025-10-20T17:39:57.011564] [WARN ] [MAIN ] BOE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.057513] [INFO ] [MAIN ] BOOT: Loaded 40347 existing records. [2025-10-20T17:39:57.052139] [INFO ] [MAIN ] BOE: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.076786] [DEBUG] [LOCAL ] BOH: Loaded 31214 existing records from primary storage. [2025-10-20T17:39:57.092257] [INFO ] [LOCAL ] BOE: Starting deletion of primary data. [2025-10-20T17:39:57.092526] [INFO ] [MAIN ] BOH: Loaded 31214 existing records. [2025-10-20T17:39:57.126027] [DEBUG] [LOCAL ] BOE: Deleted primary data file. [2025-10-20T17:39:57.147998] [INFO ] [MAIN ] BOE: Starting full primary refresh. [2025-10-20T17:39:57.699124] [WARN ] [MAIN ] BOND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.695956] [WARN ] [MAIN ] BOOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.720695] [INFO ] [MAIN ] BOND: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.746135] [INFO ] [LOCAL ] BOND: Starting deletion of primary data. [2025-10-20T17:39:57.760637] [INFO ] [MAIN ] BOOT: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.794322] [WARN ] [MAIN ] BOH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:39:57.820071] [DEBUG] [LOCAL ] BOND: Deleted primary data file. [2025-10-20T17:39:57.812917] [INFO ] [LOCAL ] BOOT: Starting deletion of primary data. [2025-10-20T17:39:57.827547] [INFO ] [MAIN ] BOND: Starting full primary refresh. [2025-10-20T17:39:57.838354] [INFO ] [MAIN ] BOH: Wiping existing primary data before full refresh. [2025-10-20T17:39:57.838007] [DEBUG] [LOCAL ] BOOT: Deleted primary data file. [2025-10-20T17:39:57.904008] [INFO ] [MAIN ] BOOT: Starting full primary refresh. [2025-10-20T17:39:57.919797] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-10-20T17:39:57.950244] [DEBUG] [LOCAL ] BOH: Deleted primary data file. [2025-10-20T17:39:57.955626] [INFO ] [MAIN ] BOH: Starting full primary refresh. [2025-10-20T17:39:58.386637] [INFO ] [REMOTE ] BOE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:58.471025] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-01. [2025-10-20T17:39:59.244836] [INFO ] [REMOTE ] BOH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.209533] [INFO ] [REMOTE ] BOND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.281976] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-01. [2025-10-20T17:39:59.281481] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-01. [2025-10-20T17:39:59.318322] [INFO ] [REMOTE ] BOOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:39:59.321148] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-01. [2025-10-20T17:39:59.535754] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T17:39:59.540406] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-02. [2025-10-20T17:40:00.444849] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T17:40:00.452051] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-02. [2025-10-20T17:40:00.525550] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T17:40:00.531379] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-03. [2025-10-20T17:40:00.706383] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:40:00.714991] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-02. [2025-10-20T17:40:00.772523] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:40:00.781994] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-02. [2025-10-20T17:40:01.431323] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:40:01.437597] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-03. [2025-10-20T17:40:01.469088] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T17:40:01.851470] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-04. [2025-10-20T17:40:02.044718] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:40:02.149719] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T17:40:02.456063] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T17:40:02.651453] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-03. [2025-10-20T17:40:02.679051] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T17:40:03.451387] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-04. [2025-10-20T17:40:04.214251] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:40:04.251439] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-04. [2025-10-20T17:40:04.439833] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T17:40:05.051384] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-03. [2025-10-20T17:40:05.468115] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T17:40:05.851382] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-05. [2025-10-20T17:40:06.575412] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:40:06.651447] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-05. [2025-10-20T17:40:07.232884] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T17:40:07.451378] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-04. [2025-10-20T17:40:07.621874] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T17:40:08.251405] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-06. [2025-10-20T17:40:08.690137] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T17:40:09.051367] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-05. [2025-10-20T17:40:09.104936] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:40:09.851377] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-07. [2025-10-20T17:40:10.085121] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:40:10.651391] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-06. [2025-10-20T17:40:11.408579] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:40:11.451354] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-08. [2025-10-20T17:40:11.737480] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:40:12.251413] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-05. [2025-10-20T17:40:12.590143] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T17:40:13.051424] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-09. [2025-10-20T17:40:13.851390] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-07. [2025-10-20T17:40:14.100574] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:40:14.475258] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T17:40:14.651369] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-06. [2025-10-20T17:40:14.869723] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:40:15.451363] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-08. [2025-10-20T17:40:15.766489] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:40:16.251376] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-06. [2025-10-20T17:40:16.346988] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:40:17.051424] [INFO ] [REMOTE ] BOND: Fetching primary data for 2025-10. [2025-10-20T17:40:17.419638] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T17:40:17.851363] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-07. [2025-10-20T17:40:18.439593] [INFO ] [REMOTE ] BOND: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T17:40:18.444684] [INFO ] [REMOTE ] BOND: Full fetch complete (46586 records). [2025-10-20T17:40:18.444771] [DEBUG] [VALIDATOR ] BOND: Validating primary data. [2025-10-20T17:40:18.656591] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-09. [2025-10-20T17:40:18.812985] [DEBUG] [VALIDATOR ] BOND: All expected trading days are present. [2025-10-20T17:40:18.813063] [DEBUG] [VALIDATOR ] BOND: Primary data validation passed. [2025-10-20T17:40:18.813598] [INFO ] [MAIN ] BOND: Fetched primary data is valid. [2025-10-20T17:40:18.813638] [DEBUG] [LOCAL ] BOND: Saving primary data. [2025-10-20T17:40:18.814614] [DEBUG] [LOCAL ] BOND: Trimmed 46586 records to 32481 (keeping 200 days of data) [2025-10-20T17:40:19.012079] [DEBUG] [LOCAL ] BOND: Primary data saved (32481 records). [2025-10-20T17:40:19.012232] [INFO ] [MAIN ] BOND: Full history saved. [2025-10-20T17:40:19.012323] [INFO ] [MAIN ] Update Progress: 321/2367 symbols completed. BOND took 23.1s. Estimated remaining: 4h 16m 30s. Rate: 8.0 symbols/min [2025-10-20T17:40:19.179168] [INFO ] [MAIN ] BORR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:19.225546] [DEBUG] [LOCAL ] BORR: Loaded 51855 existing records from primary storage. [2025-10-20T17:40:19.226037] [INFO ] [MAIN ] BORR: Loaded 51855 existing records. [2025-10-20T17:40:19.331540] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:40:19.433103] [WARN ] [MAIN ] BORR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:19.433312] [INFO ] [MAIN ] BORR: Wiping existing primary data before full refresh. [2025-10-20T17:40:19.433361] [INFO ] [LOCAL ] BORR: Starting deletion of primary data. [2025-10-20T17:40:19.433813] [DEBUG] [LOCAL ] BORR: Deleted primary data file. [2025-10-20T17:40:19.433873] [INFO ] [MAIN ] BORR: Starting full primary refresh. [2025-10-20T17:40:19.456447] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-07. [2025-10-20T17:40:19.708988] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T17:40:19.822732] [INFO ] [REMOTE ] BORR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:20.251383] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-01. [2025-10-20T17:40:21.052782] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-08. [2025-10-20T17:40:21.058305] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T17:40:21.851357] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-08. [2025-10-20T17:40:22.072586] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T17:40:22.148133] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:40:22.651440] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-09. [2025-10-20T17:40:23.312464] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T17:40:23.451436] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-02. [2025-10-20T17:40:23.802017] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T17:40:24.251397] [INFO ] [REMOTE ] BOE: Fetching primary data for 2025-10. [2025-10-20T17:40:25.051487] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-09. [2025-10-20T17:40:25.145605] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:40:25.591392] [INFO ] [REMOTE ] BOE: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T17:40:25.596963] [INFO ] [REMOTE ] BOE: Full fetch complete (26835 records). [2025-10-20T17:40:25.597139] [DEBUG] [VALIDATOR ] BOE: Validating primary data. [2025-10-20T17:40:25.856770] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-03. [2025-10-20T17:40:25.972367] [DEBUG] [VALIDATOR ] BOE: All expected trading days are present. [2025-10-20T17:40:25.972441] [DEBUG] [VALIDATOR ] BOE: Primary data validation passed. [2025-10-20T17:40:25.972843] [INFO ] [MAIN ] BOE: Fetched primary data is valid. [2025-10-20T17:40:25.972904] [DEBUG] [LOCAL ] BOE: Saving primary data. [2025-10-20T17:40:25.973523] [DEBUG] [LOCAL ] BOE: Trimmed 26835 records to 18109 (keeping 200 days of data) [2025-10-20T17:40:26.077731] [DEBUG] [LOCAL ] BOE: Primary data saved (18109 records). [2025-10-20T17:40:26.077864] [INFO ] [MAIN ] BOE: Full history saved. [2025-10-20T17:40:26.077940] [INFO ] [MAIN ] Update Progress: 322/2367 symbols completed. BOE took 30.1s. Estimated remaining: 4h 16m 20s. Rate: 8.0 symbols/min [2025-10-20T17:40:26.247073] [INFO ] [MAIN ] BOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:26.269017] [DEBUG] [LOCAL ] BOW: Loaded 23579 existing records from primary storage. [2025-10-20T17:40:26.269258] [INFO ] [MAIN ] BOW: Loaded 23579 existing records. [2025-10-20T17:40:26.447933] [WARN ] [MAIN ] BOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:26.460387] [INFO ] [MAIN ] BOW: Wiping existing primary data before full refresh. [2025-10-20T17:40:26.472943] [INFO ] [LOCAL ] BOW: Starting deletion of primary data. [2025-10-20T17:40:26.476187] [DEBUG] [LOCAL ] BOW: Deleted primary data file. [2025-10-20T17:40:26.479069] [INFO ] [MAIN ] BOW: Starting full primary refresh. [2025-10-20T17:40:26.479009] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:40:26.655777] [INFO ] [REMOTE ] BOOT: Fetching primary data for 2025-10. [2025-10-20T17:40:26.847982] [INFO ] [REMOTE ] BOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:27.451361] [INFO ] [REMOTE ] BOH: Fetching primary data for 2025-10. [2025-10-20T17:40:27.483284] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:40:28.251433] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-04. [2025-10-20T17:40:28.286711] [INFO ] [REMOTE ] BOOT: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:40:28.292327] [INFO ] [REMOTE ] BOOT: Full fetch complete (60126 records). [2025-10-20T17:40:28.292450] [DEBUG] [VALIDATOR ] BOOT: Validating primary data. [2025-10-20T17:40:28.685742] [DEBUG] [VALIDATOR ] BOOT: All expected trading days are present. [2025-10-20T17:40:28.685838] [DEBUG] [VALIDATOR ] BOOT: Primary data validation passed. [2025-10-20T17:40:28.686619] [INFO ] [MAIN ] BOOT: Fetched primary data is valid. [2025-10-20T17:40:28.686718] [DEBUG] [LOCAL ] BOOT: Saving primary data. [2025-10-20T17:40:28.687980] [DEBUG] [LOCAL ] BOOT: Trimmed 60126 records to 39698 (keeping 200 days of data) [2025-10-20T17:40:28.924964] [DEBUG] [LOCAL ] BOOT: Primary data saved (39698 records). [2025-10-20T17:40:28.925115] [INFO ] [MAIN ] BOOT: Full history saved. [2025-10-20T17:40:28.926059] [INFO ] [MAIN ] Update Progress: 323/2367 symbols completed. BOOT took 33.0s. Estimated remaining: 4h 15m 43s. Rate: 8.0 symbols/min [2025-10-20T17:40:29.056637] [INFO ] [REMOTE ] BOH: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T17:40:29.076479] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-01. [2025-10-20T17:40:29.097581] [INFO ] [REMOTE ] BOH: Full fetch complete (45178 records). [2025-10-20T17:40:29.125498] [INFO ] [MAIN ] BOX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:29.126017] [DEBUG] [VALIDATOR ] BOH: Validating primary data. [2025-10-20T17:40:29.127496] [FATAL] [VALIDATOR ] BOH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:40:29.127610] [FATAL] [MAIN ] BOH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:40:29.127700] [WARN ] [MAIN ] BOH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:40:29.127772] [INFO ] [LOCAL ] BOH: Starting deletion of primary data. [2025-10-20T17:40:29.127853] [DEBUG] [LOCAL ] BOH: No primary data file found to delete. [2025-10-20T17:40:29.128030] [INFO ] [LOCAL ] BOH: Starting deletion of derivative data. [2025-10-20T17:40:29.128234] [DEBUG] [LOCAL ] BOH: Deleted derivative data file. [2025-10-20T17:40:29.128368] [INFO ] [MAIN ] Update Progress: 324/2367 symbols completed. BOH took 33.2s. Estimated remaining: 4h 14m 49s. Rate: 8.0 symbols/min [2025-10-20T17:40:29.314357] [INFO ] [MAIN ] BP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:29.334271] [DEBUG] [LOCAL ] BOX: Loaded 50897 existing records from primary storage. [2025-10-20T17:40:29.334712] [INFO ] [MAIN ] BOX: Loaded 50897 existing records. [2025-10-20T17:40:29.536209] [WARN ] [MAIN ] BOX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:29.551105] [INFO ] [MAIN ] BOX: Wiping existing primary data before full refresh. [2025-10-20T17:40:29.551187] [INFO ] [LOCAL ] BOX: Starting deletion of primary data. [2025-10-20T17:40:29.552054] [DEBUG] [LOCAL ] BOX: Deleted primary data file. [2025-10-20T17:40:29.552013] [DEBUG] [LOCAL ] BP: Loaded 54488 existing records from primary storage. [2025-10-20T17:40:29.552161] [INFO ] [MAIN ] BOX: Starting full primary refresh. [2025-10-20T17:40:29.583604] [INFO ] [MAIN ] BP: Loaded 54488 existing records. [2025-10-20T17:40:29.963665] [WARN ] [MAIN ] BP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:29.987184] [INFO ] [MAIN ] BP: Wiping existing primary data before full refresh. [2025-10-20T17:40:29.997851] [INFO ] [LOCAL ] BP: Starting deletion of primary data. [2025-10-20T17:40:30.023476] [DEBUG] [LOCAL ] BP: Deleted primary data file. [2025-10-20T17:40:30.044214] [INFO ] [MAIN ] BP: Starting full primary refresh. [2025-10-20T17:40:30.169697] [INFO ] [REMOTE ] BOX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:30.236573] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-01. [2025-10-20T17:40:30.259675] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:40:30.301761] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T17:40:30.535316] [INFO ] [REMOTE ] BP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:30.651405] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-01. [2025-10-20T17:40:31.451421] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-02. [2025-10-20T17:40:31.830605] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:40:32.256629] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-05. [2025-10-20T17:40:32.259778] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:40:32.360478] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:40:33.051428] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-03. [2025-10-20T17:40:33.851373] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-02. [2025-10-20T17:40:34.145316] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:40:34.317293] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:40:34.651376] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-04. [2025-10-20T17:40:35.080526] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:40:35.451340] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-06. [2025-10-20T17:40:35.862761] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:40:36.251403] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-05. [2025-10-20T17:40:36.995435] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:40:37.051414] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-03. [2025-10-20T17:40:37.322989] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:40:37.851373] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-06. [2025-10-20T17:40:38.641571] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:40:38.651349] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-07. [2025-10-20T17:40:38.724477] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:40:39.451401] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-02. [2025-10-20T17:40:39.598874] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T17:40:40.251368] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-08. [2025-10-20T17:40:41.051383] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-07. [2025-10-20T17:40:41.238387] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T17:40:41.263969] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:40:41.851436] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-09. [2025-10-20T17:40:42.651356] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-03. [2025-10-20T17:40:42.774003] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:40:42.792549] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T17:40:43.451407] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-08. [2025-10-20T17:40:44.251451] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-04. [2025-10-20T17:40:44.879900] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T17:40:44.951361] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:40:45.051361] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-09. [2025-10-20T17:40:45.851353] [INFO ] [REMOTE ] BOW: Fetching primary data for 2025-10. [2025-10-20T17:40:45.927812] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:40:46.514452] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:40:46.651372] [INFO ] [REMOTE ] BORR: Fetching primary data for 2025-10. [2025-10-20T17:40:46.748057] [INFO ] [REMOTE ] BOW: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:40:46.752610] [INFO ] [REMOTE ] BOW: Full fetch complete (32083 records). [2025-10-20T17:40:46.752787] [DEBUG] [VALIDATOR ] BOW: Validating primary data. [2025-10-20T17:40:47.133575] [DEBUG] [VALIDATOR ] BOW: All expected trading days are present. [2025-10-20T17:40:47.133680] [DEBUG] [VALIDATOR ] BOW: Primary data validation passed. [2025-10-20T17:40:47.134157] [INFO ] [MAIN ] BOW: Fetched primary data is valid. [2025-10-20T17:40:47.134218] [DEBUG] [LOCAL ] BOW: Saving primary data. [2025-10-20T17:40:47.134951] [DEBUG] [LOCAL ] BOW: Trimmed 32083 records to 22961 (keeping 200 days of data) [2025-10-20T17:40:47.265502] [DEBUG] [LOCAL ] BOW: Primary data saved (22961 records). [2025-10-20T17:40:47.265620] [INFO ] [MAIN ] BOW: Full history saved. [2025-10-20T17:40:47.265725] [INFO ] [MAIN ] Update Progress: 325/2367 symbols completed. BOW took 21.2s. Estimated remaining: 4h 15m 49s. Rate: 8.0 symbols/min [2025-10-20T17:40:47.432948] [INFO ] [MAIN ] BR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:47.452572] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-04. [2025-10-20T17:40:47.467235] [DEBUG] [LOCAL ] BR: Loaded 35340 existing records from primary storage. [2025-10-20T17:40:47.467493] [INFO ] [MAIN ] BR: Loaded 35340 existing records. [2025-10-20T17:40:47.645542] [WARN ] [MAIN ] BR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:40:47.645781] [INFO ] [MAIN ] BR: Wiping existing primary data before full refresh. [2025-10-20T17:40:47.645830] [INFO ] [LOCAL ] BR: Starting deletion of primary data. [2025-10-20T17:40:47.646190] [DEBUG] [LOCAL ] BR: Deleted primary data file. [2025-10-20T17:40:47.646245] [INFO ] [MAIN ] BR: Starting full primary refresh. [2025-10-20T17:40:47.997569] [INFO ] [REMOTE ] BR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:48.251365] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-01. [2025-10-20T17:40:48.500163] [INFO ] [REMOTE ] BORR: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T17:40:48.507953] [INFO ] [REMOTE ] BORR: Full fetch complete (74559 records). [2025-10-20T17:40:48.508088] [DEBUG] [VALIDATOR ] BORR: Validating primary data. [2025-10-20T17:40:48.922580] [DEBUG] [VALIDATOR ] BORR: All expected trading days are present. [2025-10-20T17:40:48.922687] [DEBUG] [VALIDATOR ] BORR: Primary data validation passed. [2025-10-20T17:40:48.923483] [INFO ] [MAIN ] BORR: Fetched primary data is valid. [2025-10-20T17:40:48.923525] [DEBUG] [LOCAL ] BORR: Saving primary data. [2025-10-20T17:40:48.924885] [DEBUG] [LOCAL ] BORR: Trimmed 74559 records to 51118 (keeping 200 days of data) [2025-10-20T17:40:49.076934] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-05. [2025-10-20T17:40:49.208241] [DEBUG] [LOCAL ] BORR: Primary data saved (51118 records). [2025-10-20T17:40:49.208424] [INFO ] [MAIN ] BORR: Full history saved. [2025-10-20T17:40:49.208507] [INFO ] [MAIN ] Update Progress: 326/2367 symbols completed. BORR took 30.2s. Estimated remaining: 4h 15m 6s. Rate: 8.0 symbols/min [2025-10-20T17:40:49.385584] [INFO ] [MAIN ] BRBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:40:49.386007] [DEBUG] [LOCAL ] BRBR: No existing data file found in primary storage. [2025-10-20T17:40:49.386078] [WARN ] [MAIN ] BRBR: No existing data found, performing full refresh instead. [2025-10-20T17:40:49.386304] [INFO ] [MAIN ] BRBR: Wiping existing primary data before full refresh. [2025-10-20T17:40:49.386462] [INFO ] [LOCAL ] BRBR: Starting deletion of primary data. [2025-10-20T17:40:49.386540] [DEBUG] [LOCAL ] BRBR: No primary data file found to delete. [2025-10-20T17:40:49.386586] [INFO ] [MAIN ] BRBR: Starting full primary refresh. [2025-10-20T17:40:49.620419] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T17:40:49.798212] [INFO ] [REMOTE ] BRBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:40:49.831745] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-10-20T17:40:49.851346] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-01. [2025-10-20T17:40:50.651350] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-05. [2025-10-20T17:40:50.868728] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:40:51.458876] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-06. [2025-10-20T17:40:51.462045] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:40:52.251442] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-02. [2025-10-20T17:40:52.605095] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:40:53.051409] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-02. [2025-10-20T17:40:53.115989] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:40:53.224781] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T17:40:53.851453] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-07. [2025-10-20T17:40:54.600809] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:40:54.651350] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-06. [2025-10-20T17:40:55.451405] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-03. [2025-10-20T17:40:55.491523] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T17:40:56.251441] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-03. [2025-10-20T17:40:56.680813] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T17:40:57.051520] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-08. [2025-10-20T17:40:57.090828] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T17:40:57.711215] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T17:40:57.851367] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-04. [2025-10-20T17:40:58.397441] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:40:58.651376] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-07. [2025-10-20T17:40:59.451418] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-04. [2025-10-20T17:40:59.480678] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:41:00.251382] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-05. [2025-10-20T17:41:00.914189] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T17:41:01.051415] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-08. [2025-10-20T17:41:01.159795] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:41:01.851407] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-09. [2025-10-20T17:41:01.975683] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:41:02.651437] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-05. [2025-10-20T17:41:02.836612] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T17:41:03.451368] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-06. [2025-10-20T17:41:03.551506] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:41:04.069997] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:41:04.251390] [INFO ] [REMOTE ] BOX: Fetching primary data for 2025-10. [2025-10-20T17:41:05.051414] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-06. [2025-10-20T17:41:05.265002] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:41:05.851433] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-09. [2025-10-20T17:41:06.083792] [INFO ] [REMOTE ] BOX: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T17:41:06.091687] [INFO ] [REMOTE ] BOX: Full fetch complete (72341 records). [2025-10-20T17:41:06.091869] [DEBUG] [VALIDATOR ] BOX: Validating primary data. [2025-10-20T17:41:06.516313] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T17:41:06.544361] [DEBUG] [VALIDATOR ] BOX: All expected trading days are present. [2025-10-20T17:41:06.544561] [DEBUG] [VALIDATOR ] BOX: Primary data validation passed. [2025-10-20T17:41:06.545387] [INFO ] [MAIN ] BOX: Fetched primary data is valid. [2025-10-20T17:41:06.545489] [DEBUG] [LOCAL ] BOX: Saving primary data. [2025-10-20T17:41:06.546977] [DEBUG] [LOCAL ] BOX: Trimmed 72341 records to 50187 (keeping 200 days of data) [2025-10-20T17:41:06.658987] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-07. [2025-10-20T17:41:06.847578] [DEBUG] [LOCAL ] BOX: Primary data saved (50187 records). [2025-10-20T17:41:06.847798] [INFO ] [MAIN ] BOX: Full history saved. [2025-10-20T17:41:06.847895] [INFO ] [MAIN ] Update Progress: 327/2367 symbols completed. BOX took 37.9s. Estimated remaining: 4h 16m 2s. Rate: 8.0 symbols/min [2025-10-20T17:41:07.015961] [INFO ] [MAIN ] BRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:07.043021] [DEBUG] [LOCAL ] BRC: Loaded 18893 existing records from primary storage. [2025-10-20T17:41:07.043242] [INFO ] [MAIN ] BRC: Loaded 18893 existing records. [2025-10-20T17:41:07.218164] [WARN ] [MAIN ] BRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:07.218428] [INFO ] [MAIN ] BRC: Wiping existing primary data before full refresh. [2025-10-20T17:41:07.218487] [INFO ] [LOCAL ] BRC: Starting deletion of primary data. [2025-10-20T17:41:07.218712] [DEBUG] [LOCAL ] BRC: Deleted primary data file. [2025-10-20T17:41:07.218769] [INFO ] [MAIN ] BRC: Starting full primary refresh. [2025-10-20T17:41:07.459325] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-07. [2025-10-20T17:41:07.583966] [INFO ] [REMOTE ] BRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:08.123290] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-09 (2.27 seconds). [2025-10-20T17:41:08.159112] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T17:41:08.251372] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-08. [2025-10-20T17:41:09.051377] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-01. [2025-10-20T17:41:09.342971] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T17:41:09.817860] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T17:41:09.851386] [INFO ] [REMOTE ] BP: Fetching primary data for 2025-10. [2025-10-20T17:41:10.270007] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T17:41:10.651411] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-09. [2025-10-20T17:41:11.451402] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-08. [2025-10-20T17:41:12.011075] [INFO ] [REMOTE ] BP: Incremental fetch complete for 2025-10 (2.16 seconds). [2025-10-20T17:41:12.019442] [INFO ] [REMOTE ] BP: Full fetch complete (77882 records). [2025-10-20T17:41:12.019634] [DEBUG] [VALIDATOR ] BP: Validating primary data. [2025-10-20T17:41:12.257245] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-02. [2025-10-20T17:41:12.366247] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:41:12.462353] [DEBUG] [VALIDATOR ] BP: All expected trading days are present. [2025-10-20T17:41:12.462434] [DEBUG] [VALIDATOR ] BP: Primary data validation passed. [2025-10-20T17:41:12.463287] [INFO ] [MAIN ] BP: Fetched primary data is valid. [2025-10-20T17:41:12.463360] [DEBUG] [LOCAL ] BP: Saving primary data. [2025-10-20T17:41:12.465034] [DEBUG] [LOCAL ] BP: Trimmed 77882 records to 53708 (keeping 200 days of data) [2025-10-20T17:41:12.796103] [DEBUG] [LOCAL ] BP: Primary data saved (53708 records). [2025-10-20T17:41:12.796306] [INFO ] [MAIN ] BP: Full history saved. [2025-10-20T17:41:12.796824] [INFO ] [MAIN ] Update Progress: 328/2367 symbols completed. BP took 43.7s. Estimated remaining: 4h 15m 45s. Rate: 8.0 symbols/min [2025-10-20T17:41:12.979590] [INFO ] [MAIN ] BRCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:13.015294] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T17:41:13.061943] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-09. [2025-10-20T17:41:13.066448] [DEBUG] [LOCAL ] BRCC: Loaded 40784 existing records from primary storage. [2025-10-20T17:41:13.067170] [INFO ] [MAIN ] BRCC: Loaded 40784 existing records. [2025-10-20T17:41:13.160835] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:41:13.287035] [WARN ] [MAIN ] BRCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:13.287305] [INFO ] [MAIN ] BRCC: Wiping existing primary data before full refresh. [2025-10-20T17:41:13.287356] [INFO ] [LOCAL ] BRCC: Starting deletion of primary data. [2025-10-20T17:41:13.287579] [DEBUG] [LOCAL ] BRCC: Deleted primary data file. [2025-10-20T17:41:13.287617] [INFO ] [MAIN ] BRCC: Starting full primary refresh. [2025-10-20T17:41:13.644132] [INFO ] [REMOTE ] BRCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:13.851400] [INFO ] [REMOTE ] BR: Fetching primary data for 2025-10. [2025-10-20T17:41:14.651458] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-03. [2025-10-20T17:41:14.765371] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T17:41:15.391120] [INFO ] [REMOTE ] BR: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T17:41:15.398870] [INFO ] [REMOTE ] BR: Full fetch complete (50107 records). [2025-10-20T17:41:15.399150] [DEBUG] [VALIDATOR ] BR: Validating primary data. [2025-10-20T17:41:15.459612] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-01. [2025-10-20T17:41:15.813911] [DEBUG] [VALIDATOR ] BR: All expected trading days are present. [2025-10-20T17:41:15.814044] [DEBUG] [VALIDATOR ] BR: Primary data validation passed. [2025-10-20T17:41:15.814848] [INFO ] [MAIN ] BR: Fetched primary data is valid. [2025-10-20T17:41:15.814932] [DEBUG] [LOCAL ] BR: Saving primary data. [2025-10-20T17:41:15.832702] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:41:15.838353] [DEBUG] [LOCAL ] BR: Trimmed 50107 records to 34741 (keeping 200 days of data) [2025-10-20T17:41:16.063941] [DEBUG] [LOCAL ] BR: Primary data saved (34741 records). [2025-10-20T17:41:16.064183] [INFO ] [MAIN ] BR: Full history saved. [2025-10-20T17:41:16.064287] [INFO ] [MAIN ] Update Progress: 329/2367 symbols completed. BR took 28.8s. Estimated remaining: 4h 15m 11s. Rate: 8.0 symbols/min [2025-10-20T17:41:16.246464] [INFO ] [MAIN ] BRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:16.251344] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-04. [2025-10-20T17:41:16.300635] [DEBUG] [LOCAL ] BRO: Loaded 52839 existing records from primary storage. [2025-10-20T17:41:16.300990] [INFO ] [MAIN ] BRO: Loaded 52839 existing records. [2025-10-20T17:41:16.491429] [WARN ] [MAIN ] BRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:16.491685] [INFO ] [MAIN ] BRO: Wiping existing primary data before full refresh. [2025-10-20T17:41:16.491747] [INFO ] [LOCAL ] BRO: Starting deletion of primary data. [2025-10-20T17:41:16.491983] [DEBUG] [LOCAL ] BRO: Deleted primary data file. [2025-10-20T17:41:16.492041] [INFO ] [MAIN ] BRO: Starting full primary refresh. [2025-10-20T17:41:16.899903] [INFO ] [REMOTE ] BRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:17.057414] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-01. [2025-10-20T17:41:17.063721] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:41:17.127830] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:41:17.851391] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-02. [2025-10-20T17:41:18.651351] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-05. [2025-10-20T17:41:18.844848] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:41:19.189200] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T17:41:19.374365] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T17:41:19.451408] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-06. [2025-10-20T17:41:20.251390] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-03. [2025-10-20T17:41:20.744221] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:41:21.051400] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-02. [2025-10-20T17:41:21.425468] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T17:41:21.851347] [INFO ] [REMOTE ] BRBR: Fetching primary data for 2025-10. [2025-10-20T17:41:22.326192] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T17:41:22.651420] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-04. [2025-10-20T17:41:23.451450] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-07. [2025-10-20T17:41:23.772023] [INFO ] [REMOTE ] BRBR: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:41:23.784448] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T17:41:23.793772] [INFO ] [REMOTE ] BRBR: Full fetch complete (73114 records). [2025-10-20T17:41:23.794569] [DEBUG] [VALIDATOR ] BRBR: Validating primary data. [2025-10-20T17:41:24.202618] [DEBUG] [VALIDATOR ] BRBR: All expected trading days are present. [2025-10-20T17:41:24.202801] [DEBUG] [VALIDATOR ] BRBR: Primary data validation passed. [2025-10-20T17:41:24.203806] [INFO ] [MAIN ] BRBR: Fetched primary data is valid. [2025-10-20T17:41:24.203879] [DEBUG] [LOCAL ] BRBR: Saving primary data. [2025-10-20T17:41:24.205473] [DEBUG] [LOCAL ] BRBR: Trimmed 73114 records to 51831 (keeping 200 days of data) [2025-10-20T17:41:24.256523] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-03. [2025-10-20T17:41:24.429120] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:41:24.536899] [DEBUG] [LOCAL ] BRBR: Primary data saved (51831 records). [2025-10-20T17:41:24.537042] [INFO ] [MAIN ] BRBR: Full history saved. [2025-10-20T17:41:24.537112] [INFO ] [MAIN ] Update Progress: 330/2367 symbols completed. BRBR took 35.3s. Estimated remaining: 4h 15m 9s. Rate: 8.0 symbols/min [2025-10-20T17:41:24.725370] [INFO ] [MAIN ] BROS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:24.725589] [DEBUG] [LOCAL ] BROS: No existing data file found in primary storage. [2025-10-20T17:41:24.725663] [WARN ] [MAIN ] BROS: No existing data found, performing full refresh instead. [2025-10-20T17:41:24.725782] [INFO ] [MAIN ] BROS: Wiping existing primary data before full refresh. [2025-10-20T17:41:24.725823] [INFO ] [LOCAL ] BROS: Starting deletion of primary data. [2025-10-20T17:41:24.725867] [DEBUG] [LOCAL ] BROS: No primary data file found to delete. [2025-10-20T17:41:24.725921] [INFO ] [MAIN ] BROS: Starting full primary refresh. [2025-10-20T17:41:25.062543] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-08. [2025-10-20T17:41:25.081672] [INFO ] [REMOTE ] BROS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:25.851368] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-01. [2025-10-20T17:41:26.152988] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T17:41:26.190585] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:41:26.651380] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-04. [2025-10-20T17:41:27.451363] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-09. [2025-10-20T17:41:27.641786] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T17:41:28.223115] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:41:28.245524] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T17:41:28.251318] [INFO ] [REMOTE ] BRC: Fetching primary data for 2025-10. [2025-10-20T17:41:28.993905] [INFO ] [REMOTE ] BRC: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T17:41:28.996922] [INFO ] [REMOTE ] BRC: Full fetch complete (28472 records). [2025-10-20T17:41:28.996990] [DEBUG] [VALIDATOR ] BRC: Validating primary data. [2025-10-20T17:41:29.057824] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-05. [2025-10-20T17:41:29.367282] [DEBUG] [VALIDATOR ] BRC: All expected trading days are present. [2025-10-20T17:41:29.367380] [DEBUG] [VALIDATOR ] BRC: Primary data validation passed. [2025-10-20T17:41:29.367854] [INFO ] [MAIN ] BRC: Fetched primary data is valid. [2025-10-20T17:41:29.367918] [DEBUG] [LOCAL ] BRC: Saving primary data. [2025-10-20T17:41:29.368785] [DEBUG] [LOCAL ] BRC: Trimmed 28472 records to 18533 (keeping 200 days of data) [2025-10-20T17:41:29.478899] [DEBUG] [LOCAL ] BRC: Primary data saved (18533 records). [2025-10-20T17:41:29.479040] [INFO ] [MAIN ] BRC: Full history saved. [2025-10-20T17:41:29.479119] [INFO ] [MAIN ] Update Progress: 331/2367 symbols completed. BRC took 22.6s. Estimated remaining: 4h 14m 46s. Rate: 8.0 symbols/min [2025-10-20T17:41:29.651800] [INFO ] [MAIN ] BRSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:29.651967] [DEBUG] [LOCAL ] BRSL: No existing data file found in primary storage. [2025-10-20T17:41:29.652004] [WARN ] [MAIN ] BRSL: No existing data found, performing full refresh instead. [2025-10-20T17:41:29.652134] [INFO ] [MAIN ] BRSL: Wiping existing primary data before full refresh. [2025-10-20T17:41:29.652211] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-10-20T17:41:29.652272] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-10-20T17:41:29.652305] [INFO ] [MAIN ] BRSL: Starting full primary refresh. [2025-10-20T17:41:29.857788] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-02. [2025-10-20T17:41:30.011741] [INFO ] [REMOTE ] BRSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:30.661327] [INFO ] [REMOTE ] BRSL: Fetching primary data for 2025-01. [2025-10-20T17:41:30.665858] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T17:41:30.862913] [DEBUG] [REMOTE ] BRSL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:41:30.863220] [WARN ] [REMOTE ] BRSL: No data returned for month 2025-01 [2025-10-20T17:41:30.863311] [FATAL] [REMOTE ] BRSL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:41:30.863398] [FATAL] [MAIN ] BRSL: Failed to fetch primary data. [2025-10-20T17:41:30.863528] [WARN ] [MAIN ] BRSL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:41:30.863601] [INFO ] [LOCAL ] BRSL: Starting deletion of primary data. [2025-10-20T17:41:30.863697] [DEBUG] [LOCAL ] BRSL: No primary data file found to delete. [2025-10-20T17:41:30.863751] [INFO ] [LOCAL ] BRSL: Starting deletion of derivative data. [2025-10-20T17:41:30.863794] [DEBUG] [LOCAL ] BRSL: No derivative data file found to delete. [2025-10-20T17:41:30.863840] [INFO ] [MAIN ] Update Progress: 332/2367 symbols completed. BRSL took 1.4s. Estimated remaining: 4h 14m 1s. Rate: 8.0 symbols/min [2025-10-20T17:41:31.037932] [INFO ] [MAIN ] BRSP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:31.038120] [DEBUG] [LOCAL ] BRSP: No existing data file found in primary storage. [2025-10-20T17:41:31.038158] [WARN ] [MAIN ] BRSP: No existing data found, performing full refresh instead. [2025-10-20T17:41:31.038295] [INFO ] [MAIN ] BRSP: Wiping existing primary data before full refresh. [2025-10-20T17:41:31.038351] [INFO ] [LOCAL ] BRSP: Starting deletion of primary data. [2025-10-20T17:41:31.038395] [DEBUG] [LOCAL ] BRSP: No primary data file found to delete. [2025-10-20T17:41:31.038427] [INFO ] [MAIN ] BRSP: Starting full primary refresh. [2025-10-20T17:41:31.397312] [INFO ] [REMOTE ] BRSP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:31.451352] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-01. [2025-10-20T17:41:31.549108] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T17:41:32.251380] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-06. [2025-10-20T17:41:32.873406] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:41:33.051420] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-05. [2025-10-20T17:41:33.622954] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:41:33.851437] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-03. [2025-10-20T17:41:34.651425] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-02. [2025-10-20T17:41:34.879606] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T17:41:35.415913] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:41:35.451353] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-04. [2025-10-20T17:41:35.865462] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:41:36.251440] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-06. [2025-10-20T17:41:37.051400] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-07. [2025-10-20T17:41:37.092834] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:41:37.851452] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-03. [2025-10-20T17:41:38.149461] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T17:41:38.200721] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T17:41:38.651404] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-08. [2025-10-20T17:41:39.144376] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T17:41:39.451438] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-05. [2025-10-20T17:41:40.241401] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:41:40.251704] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-04. [2025-10-20T17:41:41.051383] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-07. [2025-10-20T17:41:41.138319] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T17:41:41.839739] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:41:41.851342] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-05. [2025-10-20T17:41:42.651433] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-06. [2025-10-20T17:41:42.781474] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:41:42.988022] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:41:43.451420] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-08. [2025-10-20T17:41:44.251370] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-06. [2025-10-20T17:41:44.385438] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:41:44.859157] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T17:41:45.051373] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-07. [2025-10-20T17:41:45.097592] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:41:45.851468] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-09. [2025-10-20T17:41:46.651374] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-09. [2025-10-20T17:41:47.068627] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:41:47.456479] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-08. [2025-10-20T17:41:47.478761] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T17:41:47.860222] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:41:48.251435] [INFO ] [REMOTE ] BRCC: Fetching primary data for 2025-10. [2025-10-20T17:41:49.051376] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-07. [2025-10-20T17:41:49.211836] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T17:41:49.803525] [INFO ] [REMOTE ] BRCC: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T17:41:49.809363] [INFO ] [REMOTE ] BRCC: Full fetch complete (57453 records). [2025-10-20T17:41:49.809503] [DEBUG] [VALIDATOR ] BRCC: Validating primary data. [2025-10-20T17:41:49.863602] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-09. [2025-10-20T17:41:50.219911] [DEBUG] [VALIDATOR ] BRCC: All expected trading days are present. [2025-10-20T17:41:50.220054] [DEBUG] [VALIDATOR ] BRCC: Primary data validation passed. [2025-10-20T17:41:50.221013] [INFO ] [MAIN ] BRCC: Fetched primary data is valid. [2025-10-20T17:41:50.221109] [DEBUG] [LOCAL ] BRCC: Saving primary data. [2025-10-20T17:41:50.246218] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:41:50.255420] [DEBUG] [LOCAL ] BRCC: Trimmed 57453 records to 40137 (keeping 200 days of data) [2025-10-20T17:41:50.495309] [DEBUG] [LOCAL ] BRCC: Primary data saved (40137 records). [2025-10-20T17:41:50.495483] [INFO ] [MAIN ] BRCC: Full history saved. [2025-10-20T17:41:50.495579] [INFO ] [MAIN ] Update Progress: 333/2367 symbols completed. BRCC took 37.7s. Estimated remaining: 4h 15m 7s. Rate: 8.0 symbols/min [2025-10-20T17:41:50.670145] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-08. [2025-10-20T17:41:50.674582] [INFO ] [MAIN ] BRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:50.682718] [DEBUG] [LOCAL ] BRT: Loaded 5316 existing records from primary storage. [2025-10-20T17:41:50.682947] [INFO ] [MAIN ] BRT: Loaded 5316 existing records. [2025-10-20T17:41:50.869844] [WARN ] [MAIN ] BRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:50.870310] [INFO ] [MAIN ] BRT: Wiping existing primary data before full refresh. [2025-10-20T17:41:50.870367] [INFO ] [LOCAL ] BRT: Starting deletion of primary data. [2025-10-20T17:41:50.870777] [DEBUG] [LOCAL ] BRT: Deleted primary data file. [2025-10-20T17:41:50.870846] [INFO ] [MAIN ] BRT: Starting full primary refresh. [2025-10-20T17:41:51.262604] [INFO ] [REMOTE ] BRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:51.451378] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-01. [2025-10-20T17:41:52.121262] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T17:41:52.125072] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:41:52.251362] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-09. [2025-10-20T17:41:52.347349] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T17:41:53.051466] [INFO ] [REMOTE ] BROS: Fetching primary data for 2025-10. [2025-10-20T17:41:53.783513] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:41:53.851476] [INFO ] [REMOTE ] BRO: Fetching primary data for 2025-10. [2025-10-20T17:41:54.651418] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-02. [2025-10-20T17:41:55.101145] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:41:55.144571] [INFO ] [REMOTE ] BROS: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T17:41:55.152802] [INFO ] [REMOTE ] BROS: Full fetch complete (77271 records). [2025-10-20T17:41:55.152940] [DEBUG] [VALIDATOR ] BROS: Validating primary data. [2025-10-20T17:41:55.452729] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-03. [2025-10-20T17:41:55.563354] [DEBUG] [VALIDATOR ] BROS: All expected trading days are present. [2025-10-20T17:41:55.563445] [DEBUG] [VALIDATOR ] BROS: Primary data validation passed. [2025-10-20T17:41:55.564395] [INFO ] [MAIN ] BROS: Fetched primary data is valid. [2025-10-20T17:41:55.564463] [DEBUG] [LOCAL ] BROS: Saving primary data. [2025-10-20T17:41:55.565970] [DEBUG] [LOCAL ] BROS: Trimmed 77271 records to 53300 (keeping 200 days of data) [2025-10-20T17:41:55.801345] [INFO ] [REMOTE ] BRO: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T17:41:55.814710] [INFO ] [REMOTE ] BRO: Full fetch complete (75054 records). [2025-10-20T17:41:55.818196] [DEBUG] [VALIDATOR ] BRO: Validating primary data. [2025-10-20T17:41:56.116139] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T17:41:56.254115] [INFO ] [REMOTE ] BRSP: Fetching primary data for 2025-10. [2025-10-20T17:41:56.358579] [DEBUG] [VALIDATOR ] BRO: All expected trading days are present. [2025-10-20T17:41:56.360212] [DEBUG] [VALIDATOR ] BRO: Primary data validation passed. [2025-10-20T17:41:56.361077] [INFO ] [MAIN ] BRO: Fetched primary data is valid. [2025-10-20T17:41:56.364568] [DEBUG] [LOCAL ] BRO: Saving primary data. [2025-10-20T17:41:56.377168] [DEBUG] [LOCAL ] BROS: Primary data saved (53300 records). [2025-10-20T17:41:56.377604] [INFO ] [MAIN ] BROS: Full history saved. [2025-10-20T17:41:56.377767] [INFO ] [MAIN ] Update Progress: 334/2367 symbols completed. BROS took 31.8s. Estimated remaining: 4h 14m 50s. Rate: 8.0 symbols/min [2025-10-20T17:41:56.377883] [DEBUG] [LOCAL ] BRO: Trimmed 75054 records to 52094 (keeping 200 days of data) [2025-10-20T17:41:56.680189] [INFO ] [MAIN ] BRW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:56.723877] [DEBUG] [LOCAL ] BRW: Loaded 21211 existing records from primary storage. [2025-10-20T17:41:56.724146] [INFO ] [MAIN ] BRW: Loaded 21211 existing records. [2025-10-20T17:41:57.006629] [WARN ] [MAIN ] BRW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:57.016934] [INFO ] [MAIN ] BRW: Wiping existing primary data before full refresh. [2025-10-20T17:41:57.018501] [INFO ] [LOCAL ] BRW: Starting deletion of primary data. [2025-10-20T17:41:57.028292] [DEBUG] [LOCAL ] BRW: Deleted primary data file. [2025-10-20T17:41:57.028739] [INFO ] [MAIN ] BRW: Starting full primary refresh. [2025-10-20T17:41:57.061471] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-04. [2025-10-20T17:41:57.450994] [DEBUG] [LOCAL ] BRO: Primary data saved (52094 records). [2025-10-20T17:41:57.451254] [INFO ] [MAIN ] BRO: Full history saved. [2025-10-20T17:41:57.452889] [INFO ] [MAIN ] Update Progress: 335/2367 symbols completed. BRO took 41.4s. Estimated remaining: 4h 14m 3s. Rate: 8.0 symbols/min [2025-10-20T17:41:57.637709] [INFO ] [REMOTE ] BRW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:57.715666] [INFO ] [MAIN ] BRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:57.735874] [INFO ] [REMOTE ] BRSP: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T17:41:57.743383] [INFO ] [REMOTE ] BRSP: Full fetch complete (54172 records). [2025-10-20T17:41:57.743583] [DEBUG] [VALIDATOR ] BRSP: Validating primary data. [2025-10-20T17:41:57.857566] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-01. [2025-10-20T17:41:57.875939] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T17:41:58.125214] [DEBUG] [LOCAL ] BRX: Loaded 52572 existing records from primary storage. [2025-10-20T17:41:58.136748] [INFO ] [MAIN ] BRX: Loaded 52572 existing records. [2025-10-20T17:41:58.343014] [DEBUG] [VALIDATOR ] BRSP: All expected trading days are present. [2025-10-20T17:41:58.348305] [DEBUG] [VALIDATOR ] BRSP: Primary data validation passed. [2025-10-20T17:41:58.354212] [INFO ] [MAIN ] BRSP: Fetched primary data is valid. [2025-10-20T17:41:58.359478] [DEBUG] [LOCAL ] BRSP: Saving primary data. [2025-10-20T17:41:58.416511] [DEBUG] [LOCAL ] BRSP: Trimmed 54172 records to 36685 (keeping 200 days of data) [2025-10-20T17:41:58.420949] [WARN ] [MAIN ] BRX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:58.421783] [INFO ] [MAIN ] BRX: Wiping existing primary data before full refresh. [2025-10-20T17:41:58.421861] [INFO ] [LOCAL ] BRX: Starting deletion of primary data. [2025-10-20T17:41:58.422495] [DEBUG] [LOCAL ] BRX: Deleted primary data file. [2025-10-20T17:41:58.422753] [INFO ] [MAIN ] BRX: Starting full primary refresh. [2025-10-20T17:41:58.654870] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-05. [2025-10-20T17:41:59.024069] [INFO ] [REMOTE ] BRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:41:59.056425] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:41:59.079680] [DEBUG] [LOCAL ] BRSP: Primary data saved (36685 records). [2025-10-20T17:41:59.079864] [INFO ] [MAIN ] BRSP: Full history saved. [2025-10-20T17:41:59.079953] [INFO ] [MAIN ] Update Progress: 336/2367 symbols completed. BRSP took 28.2s. Estimated remaining: 4h 13m 20s. Rate: 8.0 symbols/min [2025-10-20T17:41:59.267194] [INFO ] [MAIN ] BSAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:41:59.300541] [DEBUG] [LOCAL ] BSAC: Loaded 33351 existing records from primary storage. [2025-10-20T17:41:59.300942] [INFO ] [MAIN ] BSAC: Loaded 33351 existing records. [2025-10-20T17:41:59.411516] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T17:41:59.458197] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-01. [2025-10-20T17:41:59.510306] [WARN ] [MAIN ] BSAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:41:59.510600] [INFO ] [MAIN ] BSAC: Wiping existing primary data before full refresh. [2025-10-20T17:41:59.510715] [INFO ] [LOCAL ] BSAC: Starting deletion of primary data. [2025-10-20T17:41:59.511032] [DEBUG] [LOCAL ] BSAC: Deleted primary data file. [2025-10-20T17:41:59.511093] [INFO ] [MAIN ] BSAC: Starting full primary refresh. [2025-10-20T17:41:59.890862] [INFO ] [REMOTE ] BSAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:00.251413] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-01. [2025-10-20T17:42:01.051403] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-06. [2025-10-20T17:42:01.174852] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T17:42:01.492356] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T17:42:01.563777] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T17:42:01.851392] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-02. [2025-10-20T17:42:02.651534] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-07. [2025-10-20T17:42:03.008602] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:42:03.412911] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:42:03.451399] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-03. [2025-10-20T17:42:04.251357] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-08. [2025-10-20T17:42:05.051424] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-02. [2025-10-20T17:42:05.410069] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T17:42:05.572328] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T17:42:05.851438] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-02. [2025-10-20T17:42:06.327086] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T17:42:06.651425] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-09. [2025-10-20T17:42:07.228404] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:42:07.451450] [INFO ] [REMOTE ] BRT: Fetching primary data for 2025-10. [2025-10-20T17:42:08.251425] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-03. [2025-10-20T17:42:08.282498] [INFO ] [REMOTE ] BRT: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T17:42:08.284433] [INFO ] [REMOTE ] BRT: Full fetch complete (7077 records). [2025-10-20T17:42:08.284540] [DEBUG] [VALIDATOR ] BRT: Validating primary data. [2025-10-20T17:42:08.569209] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-02 (2.72 seconds). [2025-10-20T17:42:08.741930] [DEBUG] [VALIDATOR ] BRT: All expected trading days are present. [2025-10-20T17:42:08.742042] [DEBUG] [VALIDATOR ] BRT: Primary data validation passed. [2025-10-20T17:42:08.742303] [INFO ] [MAIN ] BRT: Fetched primary data is valid. [2025-10-20T17:42:08.742346] [DEBUG] [LOCAL ] BRT: Saving primary data. [2025-10-20T17:42:08.743153] [DEBUG] [LOCAL ] BRT: Trimmed 7077 records to 5249 (keeping 200 days of data) [2025-10-20T17:42:08.779733] [DEBUG] [LOCAL ] BRT: Primary data saved (5249 records). [2025-10-20T17:42:08.779917] [INFO ] [MAIN ] BRT: Full history saved. [2025-10-20T17:42:08.780015] [INFO ] [MAIN ] Update Progress: 337/2367 symbols completed. BRT took 18.3s. Estimated remaining: 4h 13m 26s. Rate: 8.0 symbols/min [2025-10-20T17:42:08.971466] [INFO ] [MAIN ] BSBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:09.011518] [DEBUG] [LOCAL ] BSBR: Loaded 36277 existing records from primary storage. [2025-10-20T17:42:09.011988] [INFO ] [MAIN ] BSBR: Loaded 36277 existing records. [2025-10-20T17:42:09.056715] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-03. [2025-10-20T17:42:09.230066] [WARN ] [MAIN ] BSBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:09.230603] [INFO ] [MAIN ] BSBR: Wiping existing primary data before full refresh. [2025-10-20T17:42:09.230706] [INFO ] [LOCAL ] BSBR: Starting deletion of primary data. [2025-10-20T17:42:09.231062] [DEBUG] [LOCAL ] BSBR: Deleted primary data file. [2025-10-20T17:42:09.231122] [INFO ] [MAIN ] BSBR: Starting full primary refresh. [2025-10-20T17:42:09.354247] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:42:09.629353] [INFO ] [REMOTE ] BSBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:09.851378] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-04. [2025-10-20T17:42:10.613260] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:42:10.651384] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-04. [2025-10-20T17:42:11.117365] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T17:42:11.451403] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-04. [2025-10-20T17:42:12.251384] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-01. [2025-10-20T17:42:12.327497] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:42:12.531487] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:42:13.051374] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-05. [2025-10-20T17:42:13.458956] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:42:13.851374] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-05. [2025-10-20T17:42:14.116685] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:42:14.651476] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-02. [2025-10-20T17:42:14.736420] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T17:42:15.451453] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-06. [2025-10-20T17:42:15.870469] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T17:42:16.251451] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-05. [2025-10-20T17:42:16.503484] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:42:17.051376] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-06. [2025-10-20T17:42:17.851350] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-03. [2025-10-20T17:42:17.894812] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T17:42:18.283303] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T17:42:18.651335] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-07. [2025-10-20T17:42:18.749574] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T17:42:19.451425] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-07. [2025-10-20T17:42:20.075531] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T17:42:20.251404] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-04. [2025-10-20T17:42:20.361573] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T17:42:21.051394] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-06. [2025-10-20T17:42:21.832947] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:42:21.851346] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-05. [2025-10-20T17:42:22.610826] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T17:42:22.651415] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-08. [2025-10-20T17:42:23.035365] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T17:42:23.451444] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-06. [2025-10-20T17:42:23.774139] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:42:24.251439] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-08. [2025-10-20T17:42:24.808986] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T17:42:25.051485] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-07. [2025-10-20T17:42:25.111625] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:42:25.851413] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-09. [2025-10-20T17:42:26.535213] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:42:26.651412] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-09. [2025-10-20T17:42:27.214928] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T17:42:27.451429] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-07. [2025-10-20T17:42:27.712293] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T17:42:28.251438] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-08. [2025-10-20T17:42:28.828507] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:42:29.051378] [INFO ] [REMOTE ] BSAC: Fetching primary data for 2025-10. [2025-10-20T17:42:29.550429] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:42:29.851378] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-09. [2025-10-20T17:42:30.017048] [INFO ] [REMOTE ] BSAC: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T17:42:30.022108] [INFO ] [REMOTE ] BSAC: Full fetch complete (45052 records). [2025-10-20T17:42:30.022225] [DEBUG] [VALIDATOR ] BSAC: Validating primary data. [2025-10-20T17:42:30.389808] [DEBUG] [VALIDATOR ] BSAC: All expected trading days are present. [2025-10-20T17:42:30.389889] [DEBUG] [VALIDATOR ] BSAC: Primary data validation passed. [2025-10-20T17:42:30.390421] [INFO ] [MAIN ] BSAC: Fetched primary data is valid. [2025-10-20T17:42:30.390461] [DEBUG] [LOCAL ] BSAC: Saving primary data. [2025-10-20T17:42:30.391574] [DEBUG] [LOCAL ] BSAC: Trimmed 45052 records to 32970 (keeping 200 days of data) [2025-10-20T17:42:30.580096] [DEBUG] [LOCAL ] BSAC: Primary data saved (32970 records). [2025-10-20T17:42:30.580239] [INFO ] [MAIN ] BSAC: Full history saved. [2025-10-20T17:42:30.580714] [INFO ] [MAIN ] Update Progress: 338/2367 symbols completed. BSAC took 31.5s. Estimated remaining: 4h 14m 45s. Rate: 8.0 symbols/min [2025-10-20T17:42:30.658436] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-08. [2025-10-20T17:42:30.758340] [INFO ] [MAIN ] BSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:30.758507] [DEBUG] [LOCAL ] BSL: No existing data file found in primary storage. [2025-10-20T17:42:30.758545] [WARN ] [MAIN ] BSL: No existing data found, performing full refresh instead. [2025-10-20T17:42:30.758683] [INFO ] [MAIN ] BSL: Wiping existing primary data before full refresh. [2025-10-20T17:42:30.758737] [INFO ] [LOCAL ] BSL: Starting deletion of primary data. [2025-10-20T17:42:30.758781] [DEBUG] [LOCAL ] BSL: No primary data file found to delete. [2025-10-20T17:42:30.758814] [INFO ] [MAIN ] BSL: Starting full primary refresh. [2025-10-20T17:42:31.143255] [INFO ] [REMOTE ] BSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:31.158595] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:42:31.451374] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-01. [2025-10-20T17:42:32.154598] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:42:32.191866] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:42:32.251449] [INFO ] [REMOTE ] BRW: Fetching primary data for 2025-10. [2025-10-20T17:42:33.051412] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-09. [2025-10-20T17:42:33.293440] [INFO ] [REMOTE ] BRW: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:42:33.297357] [INFO ] [REMOTE ] BRW: Full fetch complete (30137 records). [2025-10-20T17:42:33.297439] [DEBUG] [VALIDATOR ] BRW: Validating primary data. [2025-10-20T17:42:33.699697] [DEBUG] [VALIDATOR ] BRW: All expected trading days are present. [2025-10-20T17:42:33.699791] [DEBUG] [VALIDATOR ] BRW: Primary data validation passed. [2025-10-20T17:42:33.700390] [INFO ] [MAIN ] BRW: Fetched primary data is valid. [2025-10-20T17:42:33.700465] [DEBUG] [LOCAL ] BRW: Saving primary data. [2025-10-20T17:42:33.701551] [DEBUG] [LOCAL ] BRW: Trimmed 30137 records to 20943 (keeping 200 days of data) [2025-10-20T17:42:33.834753] [DEBUG] [LOCAL ] BRW: Primary data saved (20943 records). [2025-10-20T17:42:33.834886] [INFO ] [MAIN ] BRW: Full history saved. [2025-10-20T17:42:33.835029] [INFO ] [MAIN ] Update Progress: 339/2367 symbols completed. BRW took 37.5s. Estimated remaining: 4h 14m 11s. Rate: 8.0 symbols/min [2025-10-20T17:42:33.859915] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-02. [2025-10-20T17:42:34.043521] [INFO ] [MAIN ] BSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:34.075234] [DEBUG] [LOCAL ] BSM: Loaded 32822 existing records from primary storage. [2025-10-20T17:42:34.075546] [INFO ] [MAIN ] BSM: Loaded 32822 existing records. [2025-10-20T17:42:34.283119] [WARN ] [MAIN ] BSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:34.283338] [INFO ] [MAIN ] BSM: Wiping existing primary data before full refresh. [2025-10-20T17:42:34.283385] [INFO ] [LOCAL ] BSM: Starting deletion of primary data. [2025-10-20T17:42:34.283622] [DEBUG] [LOCAL ] BSM: Deleted primary data file. [2025-10-20T17:42:34.283705] [INFO ] [MAIN ] BSM: Starting full primary refresh. [2025-10-20T17:42:34.677439] [INFO ] [REMOTE ] BSBR: Fetching primary data for 2025-10. [2025-10-20T17:42:34.677342] [INFO ] [REMOTE ] BSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:34.679944] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T17:42:34.772136] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T17:42:35.451370] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-03. [2025-10-20T17:42:35.899473] [INFO ] [REMOTE ] BSBR: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T17:42:35.906583] [INFO ] [REMOTE ] BSBR: Full fetch complete (51147 records). [2025-10-20T17:42:35.906784] [DEBUG] [VALIDATOR ] BSBR: Validating primary data. [2025-10-20T17:42:36.035117] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T17:42:36.256558] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-04. [2025-10-20T17:42:36.324956] [DEBUG] [VALIDATOR ] BSBR: All expected trading days are present. [2025-10-20T17:42:36.325044] [DEBUG] [VALIDATOR ] BSBR: Primary data validation passed. [2025-10-20T17:42:36.325692] [INFO ] [MAIN ] BSBR: Fetched primary data is valid. [2025-10-20T17:42:36.325759] [DEBUG] [LOCAL ] BSBR: Saving primary data. [2025-10-20T17:42:36.327007] [DEBUG] [LOCAL ] BSBR: Trimmed 51147 records to 35912 (keeping 200 days of data) [2025-10-20T17:42:36.533531] [DEBUG] [LOCAL ] BSBR: Primary data saved (35912 records). [2025-10-20T17:42:36.533725] [INFO ] [MAIN ] BSBR: Full history saved. [2025-10-20T17:42:36.533826] [INFO ] [MAIN ] Update Progress: 340/2367 symbols completed. BSBR took 27.8s. Estimated remaining: 4h 13m 35s. Rate: 8.0 symbols/min [2025-10-20T17:42:36.715903] [INFO ] [MAIN ] BST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:36.716130] [DEBUG] [LOCAL ] BST: No existing data file found in primary storage. [2025-10-20T17:42:36.716194] [WARN ] [MAIN ] BST: No existing data found, performing full refresh instead. [2025-10-20T17:42:36.716346] [INFO ] [MAIN ] BST: Wiping existing primary data before full refresh. [2025-10-20T17:42:36.716417] [INFO ] [LOCAL ] BST: Starting deletion of primary data. [2025-10-20T17:42:36.716489] [DEBUG] [LOCAL ] BST: No primary data file found to delete. [2025-10-20T17:42:36.716549] [INFO ] [MAIN ] BST: Starting full primary refresh. [2025-10-20T17:42:37.062916] [INFO ] [REMOTE ] BRX: Fetching primary data for 2025-10. [2025-10-20T17:42:37.106255] [INFO ] [REMOTE ] BST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:37.116288] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T17:42:37.851409] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-01. [2025-10-20T17:42:38.651468] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-01. [2025-10-20T17:42:38.866017] [INFO ] [REMOTE ] BRX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:42:38.874787] [INFO ] [REMOTE ] BRX: Full fetch complete (75144 records). [2025-10-20T17:42:38.874994] [DEBUG] [VALIDATOR ] BRX: Validating primary data. [2025-10-20T17:42:39.214923] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:42:39.322769] [DEBUG] [VALIDATOR ] BRX: All expected trading days are present. [2025-10-20T17:42:39.322899] [DEBUG] [VALIDATOR ] BRX: Primary data validation passed. [2025-10-20T17:42:39.323920] [INFO ] [MAIN ] BRX: Fetched primary data is valid. [2025-10-20T17:42:39.323989] [DEBUG] [LOCAL ] BRX: Saving primary data. [2025-10-20T17:42:39.326045] [DEBUG] [LOCAL ] BRX: Trimmed 75144 records to 51836 (keeping 200 days of data) [2025-10-20T17:42:39.457961] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-05. [2025-10-20T17:42:39.669532] [DEBUG] [LOCAL ] BRX: Primary data saved (51836 records). [2025-10-20T17:42:39.669773] [INFO ] [MAIN ] BRX: Full history saved. [2025-10-20T17:42:39.669885] [INFO ] [MAIN ] Update Progress: 341/2367 symbols completed. BRX took 42.2s. Estimated remaining: 4h 13m 2s. Rate: 8.0 symbols/min [2025-10-20T17:42:39.865373] [INFO ] [MAIN ] BSTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:39.898617] [DEBUG] [LOCAL ] BSTZ: Loaded 25243 existing records from primary storage. [2025-10-20T17:42:39.899029] [INFO ] [MAIN ] BSTZ: Loaded 25243 existing records. [2025-10-20T17:42:40.049226] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T17:42:40.142758] [WARN ] [MAIN ] BSTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:40.143042] [INFO ] [MAIN ] BSTZ: Wiping existing primary data before full refresh. [2025-10-20T17:42:40.143093] [INFO ] [LOCAL ] BSTZ: Starting deletion of primary data. [2025-10-20T17:42:40.143348] [DEBUG] [LOCAL ] BSTZ: Deleted primary data file. [2025-10-20T17:42:40.143406] [INFO ] [MAIN ] BSTZ: Starting full primary refresh. [2025-10-20T17:42:40.254083] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-02. [2025-10-20T17:42:40.486940] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:42:40.551689] [INFO ] [REMOTE ] BSTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:41.051339] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-06. [2025-10-20T17:42:41.293528] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T17:42:41.730528] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:42:41.851400] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-01. [2025-10-20T17:42:42.651396] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-03. [2025-10-20T17:42:43.188440] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T17:42:43.451357] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-07. [2025-10-20T17:42:43.851777] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T17:42:44.251423] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-04. [2025-10-20T17:42:44.375238] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T17:42:45.051427] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-08. [2025-10-20T17:42:45.350134] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:42:45.696991] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:42:45.851380] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-09. [2025-10-20T17:42:46.612952] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T17:42:46.651369] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-02. [2025-10-20T17:42:47.451432] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-02. [2025-10-20T17:42:47.700472] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T17:42:48.251432] [INFO ] [REMOTE ] BSL: Fetching primary data for 2025-10. [2025-10-20T17:42:48.801211] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T17:42:49.051374] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-03. [2025-10-20T17:42:49.060883] [INFO ] [REMOTE ] BSL: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T17:42:49.063758] [INFO ] [REMOTE ] BSL: Full fetch complete (13050 records). [2025-10-20T17:42:49.063862] [DEBUG] [VALIDATOR ] BSL: Validating primary data. [2025-10-20T17:42:49.460220] [DEBUG] [VALIDATOR ] BSL: All expected trading days are present. [2025-10-20T17:42:49.460330] [DEBUG] [VALIDATOR ] BSL: Primary data validation passed. [2025-10-20T17:42:49.460899] [INFO ] [MAIN ] BSL: Fetched primary data is valid. [2025-10-20T17:42:49.460965] [DEBUG] [LOCAL ] BSL: Saving primary data. [2025-10-20T17:42:49.461536] [DEBUG] [LOCAL ] BSL: Trimmed 13050 records to 8235 (keeping 200 days of data) [2025-10-20T17:42:49.520451] [DEBUG] [LOCAL ] BSL: Primary data saved (8235 records). [2025-10-20T17:42:49.520745] [INFO ] [MAIN ] BSL: Full history saved. [2025-10-20T17:42:49.520940] [INFO ] [MAIN ] Update Progress: 342/2367 symbols completed. BSL took 18.9s. Estimated remaining: 4h 13m 8s. Rate: 8.0 symbols/min [2025-10-20T17:42:49.721246] [INFO ] [MAIN ] BSX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:42:49.786124] [DEBUG] [LOCAL ] BSX: Loaded 54468 existing records from primary storage. [2025-10-20T17:42:49.786733] [INFO ] [MAIN ] BSX: Loaded 54468 existing records. [2025-10-20T17:42:49.860433] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-03. [2025-10-20T17:42:49.993880] [WARN ] [MAIN ] BSX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:42:49.994187] [INFO ] [MAIN ] BSX: Wiping existing primary data before full refresh. [2025-10-20T17:42:49.994241] [INFO ] [LOCAL ] BSX: Starting deletion of primary data. [2025-10-20T17:42:49.994520] [DEBUG] [LOCAL ] BSX: Deleted primary data file. [2025-10-20T17:42:49.994579] [INFO ] [MAIN ] BSX: Starting full primary refresh. [2025-10-20T17:42:50.409125] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T17:42:50.416414] [INFO ] [REMOTE ] BSX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:42:50.651417] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-04. [2025-10-20T17:42:50.978033] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T17:42:51.451376] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-04. [2025-10-20T17:42:51.978620] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:42:52.251384] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-05. [2025-10-20T17:42:52.600064] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T17:42:53.051409] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-05. [2025-10-20T17:42:53.310878] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T17:42:53.851418] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-05. [2025-10-20T17:42:54.317500] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:42:54.651395] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-06. [2025-10-20T17:42:54.734503] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T17:42:55.451375] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-06. [2025-10-20T17:42:55.657069] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:42:56.251379] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-01. [2025-10-20T17:42:56.750680] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:42:57.051417] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-06. [2025-10-20T17:42:57.851371] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-07. [2025-10-20T17:42:58.168604] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T17:42:58.211674] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T17:42:58.651413] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-07. [2025-10-20T17:42:58.881248] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:42:59.451421] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-07. [2025-10-20T17:42:59.878506] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:43:00.251371] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-08. [2025-10-20T17:43:00.929855] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T17:43:01.051355] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-08. [2025-10-20T17:43:01.356192] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T17:43:01.851447] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-09. [2025-10-20T17:43:02.651410] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-08. [2025-10-20T17:43:02.702967] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:43:02.924552] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T17:43:03.451435] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-02. [2025-10-20T17:43:03.808069] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T17:43:04.251471] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-09. [2025-10-20T17:43:04.921159] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T17:43:05.051331] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-03. [2025-10-20T17:43:05.146632] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T17:43:05.851390] [INFO ] [REMOTE ] BSTZ: Fetching primary data for 2025-10. [2025-10-20T17:43:06.651401] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-09. [2025-10-20T17:43:06.994280] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T17:43:07.285972] [INFO ] [REMOTE ] BSTZ: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T17:43:07.290103] [INFO ] [REMOTE ] BSTZ: Full fetch complete (36549 records). [2025-10-20T17:43:07.290185] [DEBUG] [VALIDATOR ] BSTZ: Validating primary data. [2025-10-20T17:43:07.456486] [INFO ] [REMOTE ] BST: Fetching primary data for 2025-10. [2025-10-20T17:43:07.724381] [DEBUG] [VALIDATOR ] BSTZ: All expected trading days are present. [2025-10-20T17:43:07.724591] [DEBUG] [VALIDATOR ] BSTZ: Primary data validation passed. [2025-10-20T17:43:07.725264] [INFO ] [MAIN ] BSTZ: Fetched primary data is valid. [2025-10-20T17:43:07.725361] [DEBUG] [LOCAL ] BSTZ: Saving primary data. [2025-10-20T17:43:07.745809] [DEBUG] [LOCAL ] BSTZ: Trimmed 36549 records to 24687 (keeping 200 days of data) [2025-10-20T17:43:07.745692] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T17:43:07.898144] [DEBUG] [LOCAL ] BSTZ: Primary data saved (24687 records). [2025-10-20T17:43:07.898300] [INFO ] [MAIN ] BSTZ: Full history saved. [2025-10-20T17:43:07.898398] [INFO ] [MAIN ] Update Progress: 343/2367 symbols completed. BSTZ took 28.2s. Estimated remaining: 4h 14m 5s. Rate: 8.0 symbols/min [2025-10-20T17:43:08.065295] [INFO ] [MAIN ] BTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:08.073053] [DEBUG] [LOCAL ] BTA: Loaded 5810 existing records from primary storage. [2025-10-20T17:43:08.073245] [INFO ] [MAIN ] BTA: Loaded 5810 existing records. [2025-10-20T17:43:08.254307] [WARN ] [MAIN ] BTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:08.254388] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-04. [2025-10-20T17:43:08.255469] [INFO ] [MAIN ] BTA: Wiping existing primary data before full refresh. [2025-10-20T17:43:08.255527] [INFO ] [LOCAL ] BTA: Starting deletion of primary data. [2025-10-20T17:43:08.255741] [DEBUG] [LOCAL ] BTA: Deleted primary data file. [2025-10-20T17:43:08.255797] [INFO ] [MAIN ] BTA: Starting full primary refresh. [2025-10-20T17:43:08.356603] [INFO ] [REMOTE ] BST: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:43:08.370520] [INFO ] [REMOTE ] BST: Full fetch complete (28416 records). [2025-10-20T17:43:08.381114] [DEBUG] [VALIDATOR ] BST: Validating primary data. [2025-10-20T17:43:08.964532] [INFO ] [REMOTE ] BTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:09.041815] [DEBUG] [VALIDATOR ] BST: All expected trading days are present. [2025-10-20T17:43:09.041888] [DEBUG] [VALIDATOR ] BST: Primary data validation passed. [2025-10-20T17:43:09.042295] [INFO ] [MAIN ] BST: Fetched primary data is valid. [2025-10-20T17:43:09.042334] [DEBUG] [LOCAL ] BST: Saving primary data. [2025-10-20T17:43:09.043003] [DEBUG] [LOCAL ] BST: Trimmed 28416 records to 18723 (keeping 200 days of data) [2025-10-20T17:43:09.090434] [INFO ] [REMOTE ] BSM: Fetching primary data for 2025-10. [2025-10-20T17:43:09.155239] [DEBUG] [LOCAL ] BST: Primary data saved (18723 records). [2025-10-20T17:43:09.155359] [INFO ] [MAIN ] BST: Full history saved. [2025-10-20T17:43:09.155427] [INFO ] [MAIN ] Update Progress: 344/2367 symbols completed. BST took 32.6s. Estimated remaining: 4h 13m 20s. Rate: 8.0 symbols/min [2025-10-20T17:43:09.326960] [INFO ] [MAIN ] BTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:09.327181] [DEBUG] [LOCAL ] BTE: No existing data file found in primary storage. [2025-10-20T17:43:09.327232] [WARN ] [MAIN ] BTE: No existing data found, performing full refresh instead. [2025-10-20T17:43:09.327422] [INFO ] [MAIN ] BTE: Wiping existing primary data before full refresh. [2025-10-20T17:43:09.327472] [INFO ] [LOCAL ] BTE: Starting deletion of primary data. [2025-10-20T17:43:09.327552] [DEBUG] [LOCAL ] BTE: No primary data file found to delete. [2025-10-20T17:43:09.327627] [INFO ] [MAIN ] BTE: Starting full primary refresh. [2025-10-20T17:43:09.680609] [INFO ] [REMOTE ] BTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:09.851375] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-01. [2025-10-20T17:43:10.446897] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T17:43:10.469483] [INFO ] [REMOTE ] BSM: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:43:10.474054] [INFO ] [REMOTE ] BSM: Full fetch complete (46955 records). [2025-10-20T17:43:10.474134] [DEBUG] [VALIDATOR ] BSM: Validating primary data. [2025-10-20T17:43:10.662456] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-05. [2025-10-20T17:43:10.890400] [DEBUG] [VALIDATOR ] BSM: All expected trading days are present. [2025-10-20T17:43:10.890553] [DEBUG] [VALIDATOR ] BSM: Primary data validation passed. [2025-10-20T17:43:10.891553] [INFO ] [MAIN ] BSM: Fetched primary data is valid. [2025-10-20T17:43:10.891622] [DEBUG] [LOCAL ] BSM: Saving primary data. [2025-10-20T17:43:10.893242] [DEBUG] [LOCAL ] BSM: Trimmed 46955 records to 32482 (keeping 200 days of data) [2025-10-20T17:43:11.105251] [DEBUG] [LOCAL ] BSM: Primary data saved (32482 records). [2025-10-20T17:43:11.105510] [INFO ] [MAIN ] BSM: Full history saved. [2025-10-20T17:43:11.105622] [INFO ] [MAIN ] Update Progress: 345/2367 symbols completed. BSM took 37.3s. Estimated remaining: 4h 12m 40s. Rate: 8.0 symbols/min [2025-10-20T17:43:11.291175] [INFO ] [MAIN ] BTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:11.346674] [DEBUG] [LOCAL ] BTI: Loaded 54375 existing records from primary storage. [2025-10-20T17:43:11.347245] [INFO ] [MAIN ] BTI: Loaded 54375 existing records. [2025-10-20T17:43:11.459206] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-01. [2025-10-20T17:43:11.557451] [WARN ] [MAIN ] BTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:11.558540] [INFO ] [MAIN ] BTI: Wiping existing primary data before full refresh. [2025-10-20T17:43:11.559076] [INFO ] [LOCAL ] BTI: Starting deletion of primary data. [2025-10-20T17:43:11.559616] [DEBUG] [LOCAL ] BTI: Deleted primary data file. [2025-10-20T17:43:11.559703] [INFO ] [MAIN ] BTI: Starting full primary refresh. [2025-10-20T17:43:11.984566] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-01 (2.13 seconds). [2025-10-20T17:43:12.061910] [INFO ] [REMOTE ] BTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:12.251395] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-02. [2025-10-20T17:43:12.476417] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:43:12.848908] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-05 (2.19 seconds). [2025-10-20T17:43:13.051444] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-06. [2025-10-20T17:43:13.851425] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-01. [2025-10-20T17:43:13.906930] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:43:14.651398] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-03. [2025-10-20T17:43:14.740731] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T17:43:15.451420] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-02. [2025-10-20T17:43:15.950194] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:43:15.980706] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:43:16.251416] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-03. [2025-10-20T17:43:16.337943] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T17:43:16.742228] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T17:43:17.051450] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-04. [2025-10-20T17:43:17.492572] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T17:43:17.851379] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-04. [2025-10-20T17:43:18.651376] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-07. [2025-10-20T17:43:19.451400] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-05. [2025-10-20T17:43:19.653959] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T17:43:19.825792] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T17:43:20.251394] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-02. [2025-10-20T17:43:20.672508] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:43:21.051405] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-05. [2025-10-20T17:43:21.851425] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-06. [2025-10-20T17:43:22.232057] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T17:43:22.645343] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:43:22.648020] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T17:43:22.654412] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-03. [2025-10-20T17:43:23.451418] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-08. [2025-10-20T17:43:24.270947] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:43:24.256774] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-06. [2025-10-20T17:43:25.051411] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-04. [2025-10-20T17:43:25.277776] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:43:25.851371] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-09. [2025-10-20T17:43:25.913594] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T17:43:26.651385] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-07. [2025-10-20T17:43:26.742288] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:43:27.451370] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-07. [2025-10-20T17:43:27.731550] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T17:43:28.047937] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:43:28.124867] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:43:28.251422] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-08. [2025-10-20T17:43:28.610506] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T17:43:29.051384] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-08. [2025-10-20T17:43:29.851414] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-09. [2025-10-20T17:43:30.354985] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:43:30.465271] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T17:43:30.651411] [INFO ] [REMOTE ] BTA: Fetching primary data for 2025-10. [2025-10-20T17:43:31.203530] [INFO ] [REMOTE ] BTA: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T17:43:31.205794] [INFO ] [REMOTE ] BTA: Full fetch complete (9469 records). [2025-10-20T17:43:31.205897] [DEBUG] [VALIDATOR ] BTA: Validating primary data. [2025-10-20T17:43:31.456716] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-09. [2025-10-20T17:43:31.564271] [DEBUG] [VALIDATOR ] BTA: All expected trading days are present. [2025-10-20T17:43:31.564347] [DEBUG] [VALIDATOR ] BTA: Primary data validation passed. [2025-10-20T17:43:31.564550] [INFO ] [MAIN ] BTA: Fetched primary data is valid. [2025-10-20T17:43:31.564592] [DEBUG] [LOCAL ] BTA: Saving primary data. [2025-10-20T17:43:31.565010] [DEBUG] [LOCAL ] BTA: Trimmed 9469 records to 5756 (keeping 200 days of data) [2025-10-20T17:43:31.597524] [DEBUG] [LOCAL ] BTA: Primary data saved (5756 records). [2025-10-20T17:43:31.597589] [INFO ] [MAIN ] BTA: Full history saved. [2025-10-20T17:43:31.597683] [INFO ] [MAIN ] Update Progress: 346/2367 symbols completed. BTA took 23.7s. Estimated remaining: 4h 13m 49s. Rate: 8.0 symbols/min [2025-10-20T17:43:31.763741] [INFO ] [MAIN ] BTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:31.774288] [DEBUG] [LOCAL ] BTO: Loaded 9046 existing records from primary storage. [2025-10-20T17:43:31.774487] [INFO ] [MAIN ] BTO: Loaded 9046 existing records. [2025-10-20T17:43:31.945133] [WARN ] [MAIN ] BTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:31.945283] [INFO ] [MAIN ] BTO: Wiping existing primary data before full refresh. [2025-10-20T17:43:31.945326] [INFO ] [LOCAL ] BTO: Starting deletion of primary data. [2025-10-20T17:43:31.945474] [DEBUG] [LOCAL ] BTO: Deleted primary data file. [2025-10-20T17:43:31.945510] [INFO ] [MAIN ] BTO: Starting full primary refresh. [2025-10-20T17:43:32.254691] [INFO ] [REMOTE ] BSX: Fetching primary data for 2025-10. [2025-10-20T17:43:32.288626] [INFO ] [REMOTE ] BTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:33.051427] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-01. [2025-10-20T17:43:33.141837] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T17:43:33.172941] [INFO ] [REMOTE ] BSX: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T17:43:33.180973] [INFO ] [REMOTE ] BSX: Full fetch complete (77861 records). [2025-10-20T17:43:33.181142] [DEBUG] [VALIDATOR ] BSX: Validating primary data. [2025-10-20T17:43:33.563608] [DEBUG] [VALIDATOR ] BSX: All expected trading days are present. [2025-10-20T17:43:33.563723] [DEBUG] [VALIDATOR ] BSX: Primary data validation passed. [2025-10-20T17:43:33.564514] [INFO ] [MAIN ] BSX: Fetched primary data is valid. [2025-10-20T17:43:33.564559] [DEBUG] [LOCAL ] BSX: Saving primary data. [2025-10-20T17:43:33.566001] [DEBUG] [LOCAL ] BSX: Trimmed 77861 records to 53688 (keeping 200 days of data) [2025-10-20T17:43:33.867394] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-05. [2025-10-20T17:43:33.896105] [DEBUG] [LOCAL ] BSX: Primary data saved (53688 records). [2025-10-20T17:43:33.896254] [INFO ] [MAIN ] BSX: Full history saved. [2025-10-20T17:43:33.896544] [INFO ] [MAIN ] Update Progress: 347/2367 symbols completed. BSX took 44.4s. Estimated remaining: 4h 13m 11s. Rate: 8.0 symbols/min [2025-10-20T17:43:34.015174] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:43:34.085150] [INFO ] [MAIN ] BTT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:34.085325] [DEBUG] [LOCAL ] BTT: No existing data file found in primary storage. [2025-10-20T17:43:34.085362] [WARN ] [MAIN ] BTT: No existing data found, performing full refresh instead. [2025-10-20T17:43:34.085471] [INFO ] [MAIN ] BTT: Wiping existing primary data before full refresh. [2025-10-20T17:43:34.085511] [INFO ] [LOCAL ] BTT: Starting deletion of primary data. [2025-10-20T17:43:34.085574] [DEBUG] [LOCAL ] BTT: No primary data file found to delete. [2025-10-20T17:43:34.085606] [INFO ] [MAIN ] BTT: Starting full primary refresh. [2025-10-20T17:43:34.443964] [INFO ] [REMOTE ] BTT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:34.651369] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-01. [2025-10-20T17:43:35.451422] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-02. [2025-10-20T17:43:35.691565] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T17:43:35.808299] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:43:36.201481] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:43:36.251378] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-03. [2025-10-20T17:43:37.051425] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-06. [2025-10-20T17:43:37.079423] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T17:43:37.851418] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-02. [2025-10-20T17:43:38.505467] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:43:38.651413] [INFO ] [REMOTE ] BTE: Fetching primary data for 2025-10. [2025-10-20T17:43:39.082811] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T17:43:39.451456] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-03. [2025-10-20T17:43:40.251457] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-07. [2025-10-20T17:43:40.453292] [INFO ] [REMOTE ] BTE: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:43:40.471601] [INFO ] [REMOTE ] BTE: Full fetch complete (76364 records). [2025-10-20T17:43:40.472556] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T17:43:40.472696] [DEBUG] [VALIDATOR ] BTE: Validating primary data. [2025-10-20T17:43:40.871325] [DEBUG] [VALIDATOR ] BTE: All expected trading days are present. [2025-10-20T17:43:40.871422] [DEBUG] [VALIDATOR ] BTE: Primary data validation passed. [2025-10-20T17:43:40.872241] [INFO ] [MAIN ] BTE: Fetched primary data is valid. [2025-10-20T17:43:40.872314] [DEBUG] [LOCAL ] BTE: Saving primary data. [2025-10-20T17:43:40.873895] [DEBUG] [LOCAL ] BTE: Trimmed 76364 records to 52728 (keeping 200 days of data) [2025-10-20T17:43:41.052330] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-04. [2025-10-20T17:43:41.219213] [DEBUG] [LOCAL ] BTE: Primary data saved (52728 records). [2025-10-20T17:43:41.219405] [INFO ] [MAIN ] BTE: Full history saved. [2025-10-20T17:43:41.219503] [INFO ] [MAIN ] Update Progress: 348/2367 symbols completed. BTE took 32.1s. Estimated remaining: 4h 13m 2s. Rate: 8.0 symbols/min [2025-10-20T17:43:41.401134] [INFO ] [MAIN ] BTU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:41.451778] [DEBUG] [LOCAL ] BTU: Loaded 54104 existing records from primary storage. [2025-10-20T17:43:41.452159] [INFO ] [MAIN ] BTU: Loaded 54104 existing records. [2025-10-20T17:43:41.626812] [WARN ] [MAIN ] BTU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:41.627035] [INFO ] [MAIN ] BTU: Wiping existing primary data before full refresh. [2025-10-20T17:43:41.627082] [INFO ] [LOCAL ] BTU: Starting deletion of primary data. [2025-10-20T17:43:41.627540] [DEBUG] [LOCAL ] BTU: Deleted primary data file. [2025-10-20T17:43:41.627578] [INFO ] [MAIN ] BTU: Starting full primary refresh. [2025-10-20T17:43:41.862840] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-04. [2025-10-20T17:43:42.062748] [INFO ] [REMOTE ] BTU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:42.133926] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T17:43:42.270312] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T17:43:42.651381] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-08. [2025-10-20T17:43:42.793980] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T17:43:43.451389] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-05. [2025-10-20T17:43:44.254716] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-05. [2025-10-20T17:43:44.260112] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T17:43:44.763478] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T17:43:44.850597] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T17:43:45.051420] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-06. [2025-10-20T17:43:45.808493] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T17:43:45.851430] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-09. [2025-10-20T17:43:46.651439] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-01. [2025-10-20T17:43:47.451364] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-07. [2025-10-20T17:43:47.812881] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T17:43:47.940868] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T17:43:48.192081] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T17:43:48.251368] [INFO ] [REMOTE ] BTI: Fetching primary data for 2025-10. [2025-10-20T17:43:49.051392] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-08. [2025-10-20T17:43:49.851734] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-06. [2025-10-20T17:43:49.851968] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:43:50.166717] [INFO ] [REMOTE ] BTI: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T17:43:50.175169] [INFO ] [REMOTE ] BTI: Full fetch complete (77645 records). [2025-10-20T17:43:50.175339] [DEBUG] [VALIDATOR ] BTI: Validating primary data. [2025-10-20T17:43:50.574122] [DEBUG] [VALIDATOR ] BTI: All expected trading days are present. [2025-10-20T17:43:50.574228] [DEBUG] [VALIDATOR ] BTI: Primary data validation passed. [2025-10-20T17:43:50.575045] [INFO ] [MAIN ] BTI: Fetched primary data is valid. [2025-10-20T17:43:50.575116] [DEBUG] [LOCAL ] BTI: Saving primary data. [2025-10-20T17:43:50.576556] [DEBUG] [LOCAL ] BTI: Trimmed 77645 records to 53594 (keeping 200 days of data) [2025-10-20T17:43:50.665821] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-02. [2025-10-20T17:43:50.891556] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:43:50.919822] [DEBUG] [LOCAL ] BTI: Primary data saved (53594 records). [2025-10-20T17:43:50.919972] [INFO ] [MAIN ] BTI: Full history saved. [2025-10-20T17:43:50.920055] [INFO ] [MAIN ] Update Progress: 349/2367 symbols completed. BTI took 39.8s. Estimated remaining: 4h 13m 7s. Rate: 8.0 symbols/min [2025-10-20T17:43:51.087528] [INFO ] [MAIN ] BTX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:51.087717] [DEBUG] [LOCAL ] BTX: No existing data file found in primary storage. [2025-10-20T17:43:51.087768] [WARN ] [MAIN ] BTX: No existing data found, performing full refresh instead. [2025-10-20T17:43:51.087931] [INFO ] [MAIN ] BTX: Wiping existing primary data before full refresh. [2025-10-20T17:43:51.087976] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-10-20T17:43:51.088036] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-10-20T17:43:51.088071] [INFO ] [MAIN ] BTX: Starting full primary refresh. [2025-10-20T17:43:51.448107] [INFO ] [REMOTE ] BTX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:51.451333] [INFO ] [REMOTE ] BTX: Fetching primary data for 2025-01. [2025-10-20T17:43:51.653471] [DEBUG] [REMOTE ] BTX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:43:51.653839] [WARN ] [REMOTE ] BTX: No data returned for month 2025-01 [2025-10-20T17:43:51.653925] [FATAL] [REMOTE ] BTX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:43:51.653971] [FATAL] [MAIN ] BTX: Failed to fetch primary data. [2025-10-20T17:43:51.654028] [WARN ] [MAIN ] BTX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:43:51.654097] [INFO ] [LOCAL ] BTX: Starting deletion of primary data. [2025-10-20T17:43:51.654160] [DEBUG] [LOCAL ] BTX: No primary data file found to delete. [2025-10-20T17:43:51.654194] [INFO ] [LOCAL ] BTX: Starting deletion of derivative data. [2025-10-20T17:43:51.654232] [DEBUG] [LOCAL ] BTX: No derivative data file found to delete. [2025-10-20T17:43:51.654277] [INFO ] [MAIN ] Update Progress: 350/2367 symbols completed. BTX took 0.7s. Estimated remaining: 4h 12m 20s. Rate: 8.0 symbols/min [2025-10-20T17:43:51.822507] [INFO ] [MAIN ] BTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:51.845451] [DEBUG] [LOCAL ] BTZ: Loaded 22828 existing records from primary storage. [2025-10-20T17:43:51.845756] [INFO ] [MAIN ] BTZ: Loaded 22828 existing records. [2025-10-20T17:43:52.018943] [WARN ] [MAIN ] BTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:52.019158] [INFO ] [MAIN ] BTZ: Wiping existing primary data before full refresh. [2025-10-20T17:43:52.019204] [INFO ] [LOCAL ] BTZ: Starting deletion of primary data. [2025-10-20T17:43:52.019450] [DEBUG] [LOCAL ] BTZ: Deleted primary data file. [2025-10-20T17:43:52.019506] [INFO ] [MAIN ] BTZ: Starting full primary refresh. [2025-10-20T17:43:52.261469] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-07. [2025-10-20T17:43:52.373577] [INFO ] [REMOTE ] BTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:52.405287] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T17:43:53.051419] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-01. [2025-10-20T17:43:53.341927] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T17:43:53.851361] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-08. [2025-10-20T17:43:54.173908] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:43:54.651458] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-09. [2025-10-20T17:43:54.901255] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T17:43:55.127957] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T17:43:55.451407] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-09. [2025-10-20T17:43:56.251442] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-02. [2025-10-20T17:43:56.426734] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:43:57.061778] [INFO ] [REMOTE ] BTO: Fetching primary data for 2025-10. [2025-10-20T17:43:57.064732] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:43:57.851359] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-03. [2025-10-20T17:43:57.906350] [INFO ] [REMOTE ] BTO: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T17:43:57.908790] [INFO ] [REMOTE ] BTO: Full fetch complete (13315 records). [2025-10-20T17:43:57.908912] [DEBUG] [VALIDATOR ] BTO: Validating primary data. [2025-10-20T17:43:58.284044] [DEBUG] [VALIDATOR ] BTO: All expected trading days are present. [2025-10-20T17:43:58.284149] [DEBUG] [VALIDATOR ] BTO: Primary data validation passed. [2025-10-20T17:43:58.284634] [INFO ] [MAIN ] BTO: Fetched primary data is valid. [2025-10-20T17:43:58.284718] [DEBUG] [LOCAL ] BTO: Saving primary data. [2025-10-20T17:43:58.285327] [DEBUG] [LOCAL ] BTO: Trimmed 13315 records to 8905 (keeping 200 days of data) [2025-10-20T17:43:58.344142] [DEBUG] [LOCAL ] BTO: Primary data saved (8905 records). [2025-10-20T17:43:58.344345] [INFO ] [MAIN ] BTO: Full history saved. [2025-10-20T17:43:58.344451] [INFO ] [MAIN ] Update Progress: 351/2367 symbols completed. BTO took 26.7s. Estimated remaining: 4h 12m 8s. Rate: 8.0 symbols/min [2025-10-20T17:43:58.532224] [INFO ] [MAIN ] BUD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:43:58.589515] [DEBUG] [LOCAL ] BUD: Loaded 53369 existing records from primary storage. [2025-10-20T17:43:58.590002] [INFO ] [MAIN ] BUD: Loaded 53369 existing records. [2025-10-20T17:43:58.660558] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-03. [2025-10-20T17:43:58.804108] [WARN ] [MAIN ] BUD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:43:58.805456] [INFO ] [MAIN ] BUD: Wiping existing primary data before full refresh. [2025-10-20T17:43:58.805522] [INFO ] [LOCAL ] BUD: Starting deletion of primary data. [2025-10-20T17:43:58.806355] [DEBUG] [LOCAL ] BUD: Deleted primary data file. [2025-10-20T17:43:58.806414] [INFO ] [MAIN ] BUD: Starting full primary refresh. [2025-10-20T17:43:59.168249] [INFO ] [REMOTE ] BUD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:43:59.451385] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-01. [2025-10-20T17:43:59.794480] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:43:59.852369] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T17:44:00.251377] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-04. [2025-10-20T17:44:01.051402] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-04. [2025-10-20T17:44:01.130665] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T17:44:01.851351] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-02. [2025-10-20T17:44:02.031131] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T17:44:02.426793] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:44:02.651380] [INFO ] [REMOTE ] BTT: Fetching primary data for 2025-10. [2025-10-20T17:44:03.438181] [INFO ] [REMOTE ] BTT: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T17:44:03.440422] [INFO ] [REMOTE ] BTT: Full fetch complete (22470 records). [2025-10-20T17:44:03.440493] [DEBUG] [VALIDATOR ] BTT: Validating primary data. [2025-10-20T17:44:03.461998] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-05. [2025-10-20T17:44:03.831431] [DEBUG] [VALIDATOR ] BTT: All expected trading days are present. [2025-10-20T17:44:03.831669] [DEBUG] [VALIDATOR ] BTT: Primary data validation passed. [2025-10-20T17:44:03.832135] [INFO ] [MAIN ] BTT: Fetched primary data is valid. [2025-10-20T17:44:03.832206] [DEBUG] [LOCAL ] BTT: Saving primary data. [2025-10-20T17:44:03.843350] [DEBUG] [LOCAL ] BTT: Trimmed 22470 records to 15488 (keeping 200 days of data) [2025-10-20T17:44:03.866934] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T17:44:03.969370] [DEBUG] [LOCAL ] BTT: Primary data saved (15488 records). [2025-10-20T17:44:03.969560] [INFO ] [MAIN ] BTT: Full history saved. [2025-10-20T17:44:03.969639] [INFO ] [MAIN ] Update Progress: 352/2367 symbols completed. BTT took 30.1s. Estimated remaining: 4h 11m 50s. Rate: 8.0 symbols/min [2025-10-20T17:44:04.143677] [INFO ] [MAIN ] BUI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:04.156786] [DEBUG] [LOCAL ] BUI: Loaded 12102 existing records from primary storage. [2025-10-20T17:44:04.157002] [INFO ] [MAIN ] BUI: Loaded 12102 existing records. [2025-10-20T17:44:04.257445] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-03. [2025-10-20T17:44:04.340014] [WARN ] [MAIN ] BUI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:04.340261] [INFO ] [MAIN ] BUI: Wiping existing primary data before full refresh. [2025-10-20T17:44:04.340309] [INFO ] [LOCAL ] BUI: Starting deletion of primary data. [2025-10-20T17:44:04.340516] [DEBUG] [LOCAL ] BUI: Deleted primary data file. [2025-10-20T17:44:04.340573] [INFO ] [MAIN ] BUI: Starting full primary refresh. [2025-10-20T17:44:04.710459] [INFO ] [REMOTE ] BUI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:04.724898] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T17:44:05.051370] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-01. [2025-10-20T17:44:05.851356] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-05. [2025-10-20T17:44:05.912396] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T17:44:06.002421] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:44:06.651379] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-02. [2025-10-20T17:44:07.400520] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:44:07.451366] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-04. [2025-10-20T17:44:07.768125] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T17:44:08.251412] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-06. [2025-10-20T17:44:09.051375] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-03. [2025-10-20T17:44:09.326893] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T17:44:09.401575] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:44:09.528323] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T17:44:09.851420] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-07. [2025-10-20T17:44:10.651420] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-06. [2025-10-20T17:44:10.817247] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T17:44:11.451391] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-08. [2025-10-20T17:44:12.251491] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-05. [2025-10-20T17:44:12.584190] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:44:12.638490] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T17:44:13.051390] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-09. [2025-10-20T17:44:13.851442] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-04. [2025-10-20T17:44:13.999968] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T17:44:14.038261] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T17:44:14.651420] [INFO ] [REMOTE ] BTZ: Fetching primary data for 2025-10. [2025-10-20T17:44:14.728955] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:44:15.451434] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-07. [2025-10-20T17:44:15.549437] [INFO ] [REMOTE ] BTZ: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:44:15.553837] [INFO ] [REMOTE ] BTZ: Full fetch complete (33268 records). [2025-10-20T17:44:15.554009] [DEBUG] [VALIDATOR ] BTZ: Validating primary data. [2025-10-20T17:44:15.946250] [DEBUG] [VALIDATOR ] BTZ: All expected trading days are present. [2025-10-20T17:44:15.946363] [DEBUG] [VALIDATOR ] BTZ: Primary data validation passed. [2025-10-20T17:44:15.946872] [INFO ] [MAIN ] BTZ: Fetched primary data is valid. [2025-10-20T17:44:15.946934] [DEBUG] [LOCAL ] BTZ: Saving primary data. [2025-10-20T17:44:15.947789] [DEBUG] [LOCAL ] BTZ: Trimmed 33268 records to 22556 (keeping 200 days of data) [2025-10-20T17:44:16.082670] [DEBUG] [LOCAL ] BTZ: Primary data saved (22556 records). [2025-10-20T17:44:16.082808] [INFO ] [MAIN ] BTZ: Full history saved. [2025-10-20T17:44:16.082893] [INFO ] [MAIN ] Update Progress: 353/2367 symbols completed. BTZ took 24.4s. Estimated remaining: 4h 12m 9s. Rate: 8.0 symbols/min [2025-10-20T17:44:16.253540] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-06. [2025-10-20T17:44:16.255186] [INFO ] [MAIN ] BUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:16.299697] [DEBUG] [LOCAL ] BUR: Loaded 46071 existing records from primary storage. [2025-10-20T17:44:16.300018] [INFO ] [MAIN ] BUR: Loaded 46071 existing records. [2025-10-20T17:44:16.479970] [WARN ] [MAIN ] BUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:16.480177] [INFO ] [MAIN ] BUR: Wiping existing primary data before full refresh. [2025-10-20T17:44:16.480224] [INFO ] [LOCAL ] BUR: Starting deletion of primary data. [2025-10-20T17:44:16.480415] [DEBUG] [LOCAL ] BUR: Deleted primary data file. [2025-10-20T17:44:16.480454] [INFO ] [MAIN ] BUR: Starting full primary refresh. [2025-10-20T17:44:16.851244] [INFO ] [REMOTE ] BUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:17.051524] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-05. [2025-10-20T17:44:17.542031] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T17:44:17.693932] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T17:44:17.851445] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-07. [2025-10-20T17:44:17.891266] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T17:44:18.651459] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-06. [2025-10-20T17:44:19.451407] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-01. [2025-10-20T17:44:19.666303] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:44:19.929883] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T17:44:20.251371] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-08. [2025-10-20T17:44:21.049529] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:44:21.058456] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-07. [2025-10-20T17:44:21.851435] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-08. [2025-10-20T17:44:21.909882] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:44:22.116147] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T17:44:22.651429] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-02. [2025-10-20T17:44:23.268958] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T17:44:23.451350] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-09. [2025-10-20T17:44:23.950890] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T17:44:24.251467] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-08. [2025-10-20T17:44:25.051797] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-03. [2025-10-20T17:44:25.051620] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T17:44:25.446335] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T17:44:25.851415] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-09. [2025-10-20T17:44:26.616421] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T17:44:26.651415] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-04. [2025-10-20T17:44:27.451395] [INFO ] [REMOTE ] BUD: Fetching primary data for 2025-10. [2025-10-20T17:44:27.834320] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:44:28.251455] [INFO ] [REMOTE ] BTU: Fetching primary data for 2025-10. [2025-10-20T17:44:28.347448] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:44:29.051346] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-05. [2025-10-20T17:44:29.518244] [INFO ] [REMOTE ] BUD: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T17:44:29.526637] [INFO ] [REMOTE ] BUD: Full fetch complete (76492 records). [2025-10-20T17:44:29.526882] [DEBUG] [VALIDATOR ] BUD: Validating primary data. [2025-10-20T17:44:29.864691] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-09. [2025-10-20T17:44:29.983972] [DEBUG] [VALIDATOR ] BUD: All expected trading days are present. [2025-10-20T17:44:29.984138] [DEBUG] [VALIDATOR ] BUD: Primary data validation passed. [2025-10-20T17:44:29.985162] [INFO ] [MAIN ] BUD: Fetched primary data is valid. [2025-10-20T17:44:29.985233] [DEBUG] [LOCAL ] BUD: Saving primary data. [2025-10-20T17:44:29.987381] [DEBUG] [LOCAL ] BUD: Trimmed 76492 records to 52605 (keeping 200 days of data) [2025-10-20T17:44:30.369939] [INFO ] [REMOTE ] BTU: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T17:44:30.384932] [INFO ] [REMOTE ] BTU: Full fetch complete (77280 records). [2025-10-20T17:44:30.386198] [DEBUG] [VALIDATOR ] BTU: Validating primary data. [2025-10-20T17:44:30.481900] [DEBUG] [LOCAL ] BUD: Primary data saved (52605 records). [2025-10-20T17:44:30.487634] [INFO ] [MAIN ] BUD: Full history saved. [2025-10-20T17:44:30.498774] [INFO ] [MAIN ] Update Progress: 354/2367 symbols completed. BUD took 32.2s. Estimated remaining: 4h 12m 41s. Rate: 8.0 symbols/min [2025-10-20T17:44:30.861514] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T17:44:30.886455] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T17:44:30.939856] [INFO ] [MAIN ] BURL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:30.954716] [INFO ] [REMOTE ] BUI: Fetching primary data for 2025-10. [2025-10-20T17:44:31.033146] [DEBUG] [VALIDATOR ] BTU: All expected trading days are present. [2025-10-20T17:44:31.033236] [DEBUG] [VALIDATOR ] BTU: Primary data validation passed. [2025-10-20T17:44:31.034093] [INFO ] [MAIN ] BTU: Fetched primary data is valid. [2025-10-20T17:44:31.034212] [DEBUG] [LOCAL ] BTU: Saving primary data. [2025-10-20T17:44:31.035912] [DEBUG] [LOCAL ] BTU: Trimmed 77280 records to 53336 (keeping 200 days of data) [2025-10-20T17:44:31.178915] [DEBUG] [LOCAL ] BURL: Loaded 46304 existing records from primary storage. [2025-10-20T17:44:31.181331] [INFO ] [MAIN ] BURL: Loaded 46304 existing records. [2025-10-20T17:44:31.461682] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-06. [2025-10-20T17:44:31.467189] [WARN ] [MAIN ] BURL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:31.469114] [INFO ] [MAIN ] BURL: Wiping existing primary data before full refresh. [2025-10-20T17:44:31.469184] [INFO ] [LOCAL ] BURL: Starting deletion of primary data. [2025-10-20T17:44:31.470442] [DEBUG] [LOCAL ] BURL: Deleted primary data file. [2025-10-20T17:44:31.472159] [INFO ] [MAIN ] BURL: Starting full primary refresh. [2025-10-20T17:44:32.015826] [INFO ] [REMOTE ] BURL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:32.020249] [DEBUG] [LOCAL ] BTU: Primary data saved (53336 records). [2025-10-20T17:44:32.027504] [INFO ] [MAIN ] BTU: Full history saved. [2025-10-20T17:44:32.028575] [INFO ] [REMOTE ] BUI: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T17:44:32.028311] [INFO ] [MAIN ] Update Progress: 355/2367 symbols completed. BTU took 50.8s. Estimated remaining: 4h 11m 59s. Rate: 8.0 symbols/min [2025-10-20T17:44:32.037038] [INFO ] [REMOTE ] BUI: Full fetch complete (16655 records). [2025-10-20T17:44:32.075548] [DEBUG] [VALIDATOR ] BUI: Validating primary data. [2025-10-20T17:44:32.256677] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-01. [2025-10-20T17:44:32.354113] [INFO ] [MAIN ] BUXX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:32.565957] [DEBUG] [LOCAL ] BUXX: Loaded 15594 existing records from primary storage. [2025-10-20T17:44:32.598942] [INFO ] [MAIN ] BUXX: Loaded 15594 existing records. [2025-10-20T17:44:32.683220] [DEBUG] [VALIDATOR ] BUI: All expected trading days are present. [2025-10-20T17:44:32.690820] [DEBUG] [VALIDATOR ] BUI: Primary data validation passed. [2025-10-20T17:44:32.691254] [INFO ] [MAIN ] BUI: Fetched primary data is valid. [2025-10-20T17:44:32.700460] [DEBUG] [LOCAL ] BUI: Saving primary data. [2025-10-20T17:44:32.728303] [DEBUG] [LOCAL ] BUI: Trimmed 16655 records to 11949 (keeping 200 days of data) [2025-10-20T17:44:32.917316] [WARN ] [MAIN ] BUXX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:32.921505] [INFO ] [MAIN ] BUXX: Wiping existing primary data before full refresh. [2025-10-20T17:44:32.925708] [INFO ] [LOCAL ] BUXX: Starting deletion of primary data. [2025-10-20T17:44:32.925571] [DEBUG] [LOCAL ] BUI: Primary data saved (11949 records). [2025-10-20T17:44:32.925984] [INFO ] [MAIN ] BUI: Full history saved. [2025-10-20T17:44:32.926100] [INFO ] [MAIN ] Update Progress: 356/2367 symbols completed. BUI took 29.0s. Estimated remaining: 4h 11m 14s. Rate: 8.0 symbols/min [2025-10-20T17:44:32.926130] [DEBUG] [LOCAL ] BUXX: Deleted primary data file. [2025-10-20T17:44:32.926314] [INFO ] [MAIN ] BUXX: Starting full primary refresh. [2025-10-20T17:44:33.323045] [INFO ] [MAIN ] BV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:44:33.479199] [INFO ] [REMOTE ] BUXX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:33.479775] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-01. [2025-10-20T17:44:33.531578] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-06 (2.07 seconds). [2025-10-20T17:44:33.557671] [DEBUG] [LOCAL ] BV: Loaded 39100 existing records from primary storage. [2025-10-20T17:44:33.558040] [INFO ] [MAIN ] BV: Loaded 39100 existing records. [2025-10-20T17:44:33.771705] [WARN ] [MAIN ] BV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:44:33.772028] [INFO ] [MAIN ] BV: Wiping existing primary data before full refresh. [2025-10-20T17:44:33.772132] [INFO ] [LOCAL ] BV: Starting deletion of primary data. [2025-10-20T17:44:33.772367] [DEBUG] [LOCAL ] BV: Deleted primary data file. [2025-10-20T17:44:33.772424] [INFO ] [MAIN ] BV: Starting full primary refresh. [2025-10-20T17:44:33.856629] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-07. [2025-10-20T17:44:34.160320] [INFO ] [REMOTE ] BV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:44:34.180241] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:44:34.179910] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:44:34.651421] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-02. [2025-10-20T17:44:35.444796] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T17:44:35.454586] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-01. [2025-10-20T17:44:35.464628] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T17:44:36.251385] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-02. [2025-10-20T17:44:36.466289] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T17:44:37.051495] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-02. [2025-10-20T17:44:37.782889] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:44:37.851412] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-03. [2025-10-20T17:44:37.992874] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T17:44:38.651371] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-08. [2025-10-20T17:44:39.451431] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-03. [2025-10-20T17:44:39.594633] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:44:40.258457] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-03. [2025-10-20T17:44:40.286303] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T17:44:40.576891] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:44:41.051410] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-04. [2025-10-20T17:44:41.069986] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:44:41.851445] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-04. [2025-10-20T17:44:42.417967] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:44:42.651489] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-05. [2025-10-20T17:44:43.249596] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:44:43.451439] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-05. [2025-10-20T17:44:44.251343] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-04. [2025-10-20T17:44:44.501878] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T17:44:44.937426] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T17:44:44.979396] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T17:44:45.051386] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-06. [2025-10-20T17:44:45.851400] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-09. [2025-10-20T17:44:45.986047] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T17:44:46.651377] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-07. [2025-10-20T17:44:47.288217] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T17:44:47.451349] [INFO ] [REMOTE ] BUR: Fetching primary data for 2025-10. [2025-10-20T17:44:48.207040] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:44:48.251371] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-08. [2025-10-20T17:44:49.051415] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-06. [2025-10-20T17:44:49.172180] [INFO ] [REMOTE ] BUR: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T17:44:49.179418] [INFO ] [REMOTE ] BUR: Full fetch complete (64342 records). [2025-10-20T17:44:49.179596] [DEBUG] [VALIDATOR ] BUR: Validating primary data. [2025-10-20T17:44:49.576528] [DEBUG] [VALIDATOR ] BUR: All expected trading days are present. [2025-10-20T17:44:49.576632] [DEBUG] [VALIDATOR ] BUR: Primary data validation passed. [2025-10-20T17:44:49.577463] [INFO ] [MAIN ] BUR: Fetched primary data is valid. [2025-10-20T17:44:49.577522] [DEBUG] [LOCAL ] BUR: Saving primary data. [2025-10-20T17:44:49.578867] [DEBUG] [LOCAL ] BUR: Trimmed 64342 records to 45494 (keeping 200 days of data) [2025-10-20T17:44:49.802920] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:44:49.852583] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-09. [2025-10-20T17:44:49.872838] [DEBUG] [LOCAL ] BUR: Primary data saved (45494 records). [2025-10-20T17:44:49.872923] [INFO ] [MAIN ] BUR: Full history saved. [2025-10-20T17:44:49.872997] [INFO ] [MAIN ] Update Progress: 357/2367 symbols completed. BUR took 33.8s. Estimated remaining: 4h 11m 60s. Rate: 8.0 symbols/min [2025-10-20T17:44:50.353128] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T17:44:50.651380] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-07. [2025-10-20T17:44:51.054694] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T17:44:51.451471] [INFO ] [REMOTE ] BV: Fetching primary data for 2025-10. [2025-10-20T17:44:52.251383] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-05. [2025-10-20T17:44:52.296561] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T17:44:52.609806] [INFO ] [REMOTE ] BV: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:44:52.618802] [INFO ] [REMOTE ] BV: Full fetch complete (54895 records). [2025-10-20T17:44:52.618970] [DEBUG] [VALIDATOR ] BV: Validating primary data. [2025-10-20T17:44:52.999720] [DEBUG] [VALIDATOR ] BV: All expected trading days are present. [2025-10-20T17:44:52.999805] [DEBUG] [VALIDATOR ] BV: Primary data validation passed. [2025-10-20T17:44:53.000414] [INFO ] [MAIN ] BV: Fetched primary data is valid. [2025-10-20T17:44:53.000456] [DEBUG] [LOCAL ] BV: Saving primary data. [2025-10-20T17:44:53.001533] [DEBUG] [LOCAL ] BV: Trimmed 54895 records to 38556 (keeping 200 days of data) [2025-10-20T17:44:53.056386] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-08. [2025-10-20T17:44:53.249772] [DEBUG] [LOCAL ] BV: Primary data saved (38556 records). [2025-10-20T17:44:53.250248] [INFO ] [MAIN ] BV: Full history saved. [2025-10-20T17:44:53.250388] [INFO ] [MAIN ] Update Progress: 358/2367 symbols completed. BV took 20.3s. Estimated remaining: 4h 11m 29s. Rate: 8.0 symbols/min [2025-10-20T17:44:53.251512] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:44:53.851484] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-06. [2025-10-20T17:44:54.530924] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T17:44:54.661172] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-07. [2025-10-20T17:44:54.667610] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T17:44:55.451356] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-09. [2025-10-20T17:44:55.680214] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T17:44:56.251428] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-08. [2025-10-20T17:44:56.842545] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T17:44:57.051414] [INFO ] [REMOTE ] BURL: Fetching primary data for 2025-10. [2025-10-20T17:44:57.077592] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:44:57.851384] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-09. [2025-10-20T17:44:58.429461] [INFO ] [REMOTE ] BURL: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:44:58.436099] [INFO ] [REMOTE ] BURL: Full fetch complete (65872 records). [2025-10-20T17:44:58.436229] [DEBUG] [VALIDATOR ] BURL: Validating primary data. [2025-10-20T17:44:58.630374] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:44:58.693929] [INFO ] [REMOTE ] BUXX: Fetching primary data for 2025-10. [2025-10-20T17:44:58.886018] [DEBUG] [VALIDATOR ] BURL: All expected trading days are present. [2025-10-20T17:44:58.886107] [DEBUG] [VALIDATOR ] BURL: Primary data validation passed. [2025-10-20T17:44:58.886812] [INFO ] [MAIN ] BURL: Fetched primary data is valid. [2025-10-20T17:44:58.886874] [DEBUG] [LOCAL ] BURL: Saving primary data. [2025-10-20T17:44:58.888246] [DEBUG] [LOCAL ] BURL: Trimmed 65872 records to 45583 (keeping 200 days of data) [2025-10-20T17:44:59.170316] [DEBUG] [LOCAL ] BURL: Primary data saved (45583 records). [2025-10-20T17:44:59.170458] [INFO ] [MAIN ] BURL: Full history saved. [2025-10-20T17:44:59.170537] [INFO ] [MAIN ] Update Progress: 359/2367 symbols completed. BURL took 28.7s. Estimated remaining: 4h 11m 13s. Rate: 8.0 symbols/min [2025-10-20T17:44:59.689799] [INFO ] [REMOTE ] BUXX: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:44:59.692553] [INFO ] [REMOTE ] BUXX: Full fetch complete (20711 records). [2025-10-20T17:44:59.692686] [DEBUG] [VALIDATOR ] BUXX: Validating primary data. [2025-10-20T17:45:00.134137] [DEBUG] [VALIDATOR ] BUXX: All expected trading days are present. [2025-10-20T17:45:00.134243] [DEBUG] [VALIDATOR ] BUXX: Primary data validation passed. [2025-10-20T17:45:00.134720] [INFO ] [MAIN ] BUXX: Fetched primary data is valid. [2025-10-20T17:45:00.134783] [DEBUG] [LOCAL ] BUXX: Saving primary data. [2025-10-20T17:45:00.135685] [DEBUG] [LOCAL ] BUXX: Trimmed 20711 records to 15466 (keeping 200 days of data) [2025-10-20T17:45:00.232675] [DEBUG] [LOCAL ] BUXX: Primary data saved (15466 records). [2025-10-20T17:45:00.232833] [INFO ] [MAIN ] BUXX: Full history saved. [2025-10-20T17:45:00.232923] [INFO ] [MAIN ] Update Progress: 360/2367 symbols completed. BUXX took 28.2s. Estimated remaining: 4h 10m 29s. Rate: 8.0 symbols/min [2025-10-20T17:45:00.749090] [INFO ] [MAIN ] BWA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.867636] [INFO ] [MAIN ] BWG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.882103] [DEBUG] [LOCAL ] BWG: No existing data file found in primary storage. [2025-10-20T17:45:00.942590] [INFO ] [MAIN ] BVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.929385] [WARN ] [MAIN ] BWG: No existing data found, performing full refresh instead. [2025-10-20T17:45:00.966044] [INFO ] [MAIN ] BWG: Wiping existing primary data before full refresh. [2025-10-20T17:45:00.965820] [INFO ] [MAIN ] BW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:00.966721] [INFO ] [LOCAL ] BWG: Starting deletion of primary data. [2025-10-20T17:45:00.966820] [DEBUG] [LOCAL ] BWG: No primary data file found to delete. [2025-10-20T17:45:00.967184] [INFO ] [MAIN ] BWG: Starting full primary refresh. [2025-10-20T17:45:01.305967] [DEBUG] [LOCAL ] BWA: Loaded 52901 existing records from primary storage. [2025-10-20T17:45:01.322011] [INFO ] [MAIN ] BWA: Loaded 52901 existing records. [2025-10-20T17:45:01.345112] [DEBUG] [LOCAL ] BVN: Loaded 45065 existing records from primary storage. [2025-10-20T17:45:01.391691] [DEBUG] [LOCAL ] BW: Loaded 41213 existing records from primary storage. [2025-10-20T17:45:01.404468] [INFO ] [MAIN ] BVN: Loaded 45065 existing records. [2025-10-20T17:45:01.415516] [INFO ] [MAIN ] BW: Loaded 41213 existing records. [2025-10-20T17:45:01.725368] [INFO ] [REMOTE ] BWG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:01.756949] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-01. [2025-10-20T17:45:01.991542] [WARN ] [MAIN ] BWA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.077039] [INFO ] [MAIN ] BWA: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.076952] [WARN ] [MAIN ] BVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.120030] [INFO ] [MAIN ] BVN: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.119867] [WARN ] [MAIN ] BW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:02.113508] [INFO ] [LOCAL ] BWA: Starting deletion of primary data. [2025-10-20T17:45:02.120211] [INFO ] [MAIN ] BW: Wiping existing primary data before full refresh. [2025-10-20T17:45:02.120309] [INFO ] [LOCAL ] BVN: Starting deletion of primary data. [2025-10-20T17:45:02.120442] [INFO ] [LOCAL ] BW: Starting deletion of primary data. [2025-10-20T17:45:02.120730] [DEBUG] [LOCAL ] BWA: Deleted primary data file. [2025-10-20T17:45:02.120790] [INFO ] [MAIN ] BWA: Starting full primary refresh. [2025-10-20T17:45:02.131788] [DEBUG] [LOCAL ] BW: Deleted primary data file. [2025-10-20T17:45:02.131914] [DEBUG] [LOCAL ] BVN: Deleted primary data file. [2025-10-20T17:45:02.132088] [INFO ] [MAIN ] BVN: Starting full primary refresh. [2025-10-20T17:45:02.131984] [INFO ] [MAIN ] BW: Starting full primary refresh. [2025-10-20T17:45:03.182224] [INFO ] [REMOTE ] BW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.183128] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-01. [2025-10-20T17:45:03.294144] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:45:03.346688] [INFO ] [REMOTE ] BVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.361594] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-02. [2025-10-20T17:45:03.361901] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-01. [2025-10-20T17:45:03.386341] [INFO ] [REMOTE ] BWA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:03.387497] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-01. [2025-10-20T17:45:04.241773] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T17:45:04.245771] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-03. [2025-10-20T17:45:04.784766] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T17:45:04.782524] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T17:45:04.794034] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-02. [2025-10-20T17:45:05.051378] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-04. [2025-10-20T17:45:05.098377] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T17:45:05.487792] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:45:05.749617] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T17:45:05.851381] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-02. [2025-10-20T17:45:06.393608] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:45:06.651449] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-03. [2025-10-20T17:45:07.269869] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:45:07.451456] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-02. [2025-10-20T17:45:08.189123] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T17:45:08.251405] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-04. [2025-10-20T17:45:08.656830] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:45:09.051370] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-03. [2025-10-20T17:45:09.851367] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-03. [2025-10-20T17:45:10.077815] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T17:45:10.651348] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-05. [2025-10-20T17:45:10.826288] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:45:11.435514] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T17:45:11.451324] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-04. [2025-10-20T17:45:11.971595] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T17:45:12.251427] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-05. [2025-10-20T17:45:13.051363] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-04. [2025-10-20T17:45:13.143053] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:45:13.173500] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T17:45:13.851441] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-06. [2025-10-20T17:45:14.657623] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-05. [2025-10-20T17:45:14.662955] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:45:15.047576] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T17:45:15.451378] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-06. [2025-10-20T17:45:15.886683] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:45:16.253036] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-07. [2025-10-20T17:45:16.255126] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T17:45:17.051441] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-07. [2025-10-20T17:45:17.637348] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:45:17.830040] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T17:45:17.851345] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-08. [2025-10-20T17:45:18.651393] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-05. [2025-10-20T17:45:18.754099] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:45:19.451437] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-09. [2025-10-20T17:45:20.256505] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-06. [2025-10-20T17:45:20.262817] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T17:45:20.390689] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:45:21.051420] [INFO ] [REMOTE ] BWG: Fetching primary data for 2025-10. [2025-10-20T17:45:21.744883] [INFO ] [REMOTE ] BWG: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T17:45:21.747506] [INFO ] [REMOTE ] BWG: Full fetch complete (17726 records). [2025-10-20T17:45:21.747809] [DEBUG] [VALIDATOR ] BWG: Validating primary data. [2025-10-20T17:45:21.853554] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-08. [2025-10-20T17:45:21.976971] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:45:22.187936] [DEBUG] [VALIDATOR ] BWG: All expected trading days are present. [2025-10-20T17:45:22.188021] [DEBUG] [VALIDATOR ] BWG: Primary data validation passed. [2025-10-20T17:45:22.188385] [INFO ] [MAIN ] BWG: Fetched primary data is valid. [2025-10-20T17:45:22.188445] [DEBUG] [LOCAL ] BWG: Saving primary data. [2025-10-20T17:45:22.188999] [DEBUG] [LOCAL ] BWG: Trimmed 17726 records to 12624 (keeping 200 days of data) [2025-10-20T17:45:22.261724] [DEBUG] [LOCAL ] BWG: Primary data saved (12624 records). [2025-10-20T17:45:22.261865] [INFO ] [MAIN ] BWG: Full history saved. [2025-10-20T17:45:22.261936] [INFO ] [MAIN ] Update Progress: 361/2367 symbols completed. BWG took 22.0s. Estimated remaining: 4h 11m 43s. Rate: 8.0 symbols/min [2025-10-20T17:45:22.428786] [INFO ] [MAIN ] BWLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:22.459740] [DEBUG] [LOCAL ] BWLP: Loaded 32641 existing records from primary storage. [2025-10-20T17:45:22.460011] [INFO ] [MAIN ] BWLP: Loaded 32641 existing records. [2025-10-20T17:45:22.641065] [WARN ] [MAIN ] BWLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:22.641318] [INFO ] [MAIN ] BWLP: Wiping existing primary data before full refresh. [2025-10-20T17:45:22.641366] [INFO ] [LOCAL ] BWLP: Starting deletion of primary data. [2025-10-20T17:45:22.641584] [DEBUG] [LOCAL ] BWLP: Deleted primary data file. [2025-10-20T17:45:22.641673] [INFO ] [MAIN ] BWLP: Starting full primary refresh. [2025-10-20T17:45:22.661759] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-06. [2025-10-20T17:45:22.995823] [INFO ] [REMOTE ] BWLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:23.058858] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T17:45:23.451397] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-07. [2025-10-20T17:45:23.873477] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:45:24.251433] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-01. [2025-10-20T17:45:25.043225] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T17:45:25.051935] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-07. [2025-10-20T17:45:25.483904] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T17:45:25.851429] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-08. [2025-10-20T17:45:26.355998] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T17:45:26.651395] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-08. [2025-10-20T17:45:27.451348] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-02. [2025-10-20T17:45:27.557207] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T17:45:27.925557] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:45:28.251383] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-09. [2025-10-20T17:45:28.432537] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:45:29.051424] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-09. [2025-10-20T17:45:29.712818] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T17:45:29.851352] [INFO ] [REMOTE ] BWA: Fetching primary data for 2025-10. [2025-10-20T17:45:30.619347] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T17:45:30.651328] [INFO ] [REMOTE ] BW: Fetching primary data for 2025-10. [2025-10-20T17:45:31.451393] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-03. [2025-10-20T17:45:31.608267] [INFO ] [REMOTE ] BWA: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:45:31.616453] [INFO ] [REMOTE ] BWA: Full fetch complete (75908 records). [2025-10-20T17:45:31.616625] [DEBUG] [VALIDATOR ] BWA: Validating primary data. [2025-10-20T17:45:32.024393] [DEBUG] [VALIDATOR ] BWA: All expected trading days are present. [2025-10-20T17:45:32.024483] [DEBUG] [VALIDATOR ] BWA: Primary data validation passed. [2025-10-20T17:45:32.025414] [INFO ] [MAIN ] BWA: Fetched primary data is valid. [2025-10-20T17:45:32.025489] [DEBUG] [LOCAL ] BWA: Saving primary data. [2025-10-20T17:45:32.027139] [DEBUG] [LOCAL ] BWA: Trimmed 75908 records to 52108 (keeping 200 days of data) [2025-10-20T17:45:32.252164] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-09. [2025-10-20T17:45:32.363764] [DEBUG] [LOCAL ] BWA: Primary data saved (52108 records). [2025-10-20T17:45:32.363908] [INFO ] [MAIN ] BWA: Full history saved. [2025-10-20T17:45:32.364089] [INFO ] [MAIN ] Update Progress: 362/2367 symbols completed. BWA took 32.1s. Estimated remaining: 4h 11m 49s. Rate: 8.0 symbols/min [2025-10-20T17:45:32.534624] [INFO ] [MAIN ] BWMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:32.535025] [DEBUG] [LOCAL ] BWMX: No existing data file found in primary storage. [2025-10-20T17:45:32.535135] [WARN ] [MAIN ] BWMX: No existing data found, performing full refresh instead. [2025-10-20T17:45:32.535286] [INFO ] [MAIN ] BWMX: Wiping existing primary data before full refresh. [2025-10-20T17:45:32.535410] [INFO ] [LOCAL ] BWMX: Starting deletion of primary data. [2025-10-20T17:45:32.535527] [DEBUG] [LOCAL ] BWMX: No primary data file found to delete. [2025-10-20T17:45:32.535600] [INFO ] [MAIN ] BWMX: Starting full primary refresh. [2025-10-20T17:45:32.671532] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T17:45:32.753109] [INFO ] [REMOTE ] BW: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:45:32.865221] [INFO ] [REMOTE ] BW: Full fetch complete (59407 records). [2025-10-20T17:45:32.876300] [DEBUG] [VALIDATOR ] BW: Validating primary data. [2025-10-20T17:45:33.069801] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-04. [2025-10-20T17:45:33.094708] [INFO ] [REMOTE ] BWMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:33.387424] [DEBUG] [VALIDATOR ] BW: All expected trading days are present. [2025-10-20T17:45:33.387513] [DEBUG] [VALIDATOR ] BW: Primary data validation passed. [2025-10-20T17:45:33.388197] [INFO ] [MAIN ] BW: Fetched primary data is valid. [2025-10-20T17:45:33.388260] [DEBUG] [LOCAL ] BW: Saving primary data. [2025-10-20T17:45:33.389536] [DEBUG] [LOCAL ] BW: Trimmed 59407 records to 40452 (keeping 200 days of data) [2025-10-20T17:45:33.614800] [DEBUG] [LOCAL ] BW: Primary data saved (40452 records). [2025-10-20T17:45:33.614928] [INFO ] [MAIN ] BW: Full history saved. [2025-10-20T17:45:33.615003] [INFO ] [MAIN ] Update Progress: 363/2367 symbols completed. BW took 33.4s. Estimated remaining: 4h 11m 7s. Rate: 8.0 symbols/min [2025-10-20T17:45:33.814489] [INFO ] [MAIN ] BWNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:33.820945] [DEBUG] [LOCAL ] BWNB: Loaded 4225 existing records from primary storage. [2025-10-20T17:45:33.821181] [INFO ] [MAIN ] BWNB: Loaded 4225 existing records. [2025-10-20T17:45:33.857677] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-01. [2025-10-20T17:45:34.005034] [WARN ] [MAIN ] BWNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:34.005320] [INFO ] [MAIN ] BWNB: Wiping existing primary data before full refresh. [2025-10-20T17:45:34.005417] [INFO ] [LOCAL ] BWNB: Starting deletion of primary data. [2025-10-20T17:45:34.014952] [DEBUG] [LOCAL ] BWNB: Deleted primary data file. [2025-10-20T17:45:34.028970] [INFO ] [MAIN ] BWNB: Starting full primary refresh. [2025-10-20T17:45:34.078008] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:45:34.418271] [INFO ] [REMOTE ] BWNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:34.535978] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T17:45:34.624306] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T17:45:34.651375] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-02. [2025-10-20T17:45:35.227721] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:45:35.451417] [INFO ] [REMOTE ] BVN: Fetching primary data for 2025-10. [2025-10-20T17:45:36.251375] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-03. [2025-10-20T17:45:36.601256] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T17:45:37.051436] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-01. [2025-10-20T17:45:37.188835] [INFO ] [REMOTE ] BVN: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T17:45:37.196407] [INFO ] [REMOTE ] BVN: Full fetch complete (63402 records). [2025-10-20T17:45:37.196588] [DEBUG] [VALIDATOR ] BVN: Validating primary data. [2025-10-20T17:45:37.574794] [DEBUG] [VALIDATOR ] BVN: All expected trading days are present. [2025-10-20T17:45:37.574871] [DEBUG] [VALIDATOR ] BVN: Primary data validation passed. [2025-10-20T17:45:37.575555] [INFO ] [MAIN ] BVN: Fetched primary data is valid. [2025-10-20T17:45:37.575594] [DEBUG] [LOCAL ] BVN: Saving primary data. [2025-10-20T17:45:37.576816] [DEBUG] [LOCAL ] BVN: Trimmed 63402 records to 44477 (keeping 200 days of data) [2025-10-20T17:45:37.670956] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T17:45:37.838924] [DEBUG] [LOCAL ] BVN: Primary data saved (44477 records). [2025-10-20T17:45:37.839063] [INFO ] [MAIN ] BVN: Full history saved. [2025-10-20T17:45:37.839143] [INFO ] [MAIN ] Update Progress: 364/2367 symbols completed. BVN took 37.6s. Estimated remaining: 4h 10m 41s. Rate: 8.0 symbols/min [2025-10-20T17:45:37.859460] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-04. [2025-10-20T17:45:38.014583] [INFO ] [MAIN ] BWSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:38.021051] [DEBUG] [LOCAL ] BWSN: Loaded 5235 existing records from primary storage. [2025-10-20T17:45:38.021241] [INFO ] [MAIN ] BWSN: Loaded 5235 existing records. [2025-10-20T17:45:38.201532] [WARN ] [MAIN ] BWSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:38.201800] [INFO ] [MAIN ] BWSN: Wiping existing primary data before full refresh. [2025-10-20T17:45:38.201850] [INFO ] [LOCAL ] BWSN: Starting deletion of primary data. [2025-10-20T17:45:38.201993] [DEBUG] [LOCAL ] BWSN: Deleted primary data file. [2025-10-20T17:45:38.202034] [INFO ] [MAIN ] BWSN: Starting full primary refresh. [2025-10-20T17:45:38.567778] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:45:38.585542] [INFO ] [REMOTE ] BWSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:38.651379] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-05. [2025-10-20T17:45:39.393107] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:45:39.451357] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-06. [2025-10-20T17:45:40.188818] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T17:45:40.251398] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-07. [2025-10-20T17:45:41.051356] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-05. [2025-10-20T17:45:41.150595] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T17:45:41.851416] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-01. [2025-10-20T17:45:42.270132] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T17:45:42.617391] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:45:42.651367] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-06. [2025-10-20T17:45:43.451371] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-02. [2025-10-20T17:45:43.923956] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T17:45:44.024689] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:45:44.251387] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-03. [2025-10-20T17:45:44.896553] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T17:45:45.051360] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-04. [2025-10-20T17:45:45.608204] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:45:45.851401] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-08. [2025-10-20T17:45:46.651399] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-07. [2025-10-20T17:45:46.898081] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T17:45:47.451392] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-09. [2025-10-20T17:45:47.694800] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:45:48.047235] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T17:45:48.251422] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-08. [2025-10-20T17:45:49.051411] [INFO ] [REMOTE ] BWMX: Fetching primary data for 2025-10. [2025-10-20T17:45:49.600367] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:45:49.822432] [INFO ] [REMOTE ] BWMX: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:45:49.824217] [INFO ] [REMOTE ] BWMX: Full fetch complete (9952 records). [2025-10-20T17:45:49.824295] [DEBUG] [VALIDATOR ] BWMX: Validating primary data. [2025-10-20T17:45:49.856546] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-05. [2025-10-20T17:45:50.208223] [DEBUG] [VALIDATOR ] BWMX: All expected trading days are present. [2025-10-20T17:45:50.208402] [DEBUG] [VALIDATOR ] BWMX: Primary data validation passed. [2025-10-20T17:45:50.208806] [INFO ] [MAIN ] BWMX: Fetched primary data is valid. [2025-10-20T17:45:50.208871] [DEBUG] [LOCAL ] BWMX: Saving primary data. [2025-10-20T17:45:50.209569] [DEBUG] [LOCAL ] BWMX: Trimmed 9952 records to 7887 (keeping 200 days of data) [2025-10-20T17:45:50.260113] [DEBUG] [LOCAL ] BWMX: Primary data saved (7887 records). [2025-10-20T17:45:50.260316] [INFO ] [MAIN ] BWMX: Full history saved. [2025-10-20T17:45:50.260410] [INFO ] [MAIN ] Update Progress: 365/2367 symbols completed. BWMX took 17.9s. Estimated remaining: 4h 11m 1s. Rate: 8.0 symbols/min [2025-10-20T17:45:50.453261] [INFO ] [MAIN ] BWXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:50.508958] [DEBUG] [LOCAL ] BWXT: Loaded 47875 existing records from primary storage. [2025-10-20T17:45:50.509455] [INFO ] [MAIN ] BWXT: Loaded 47875 existing records. [2025-10-20T17:45:50.658886] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-02. [2025-10-20T17:45:50.705458] [WARN ] [MAIN ] BWXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:50.707042] [INFO ] [MAIN ] BWXT: Wiping existing primary data before full refresh. [2025-10-20T17:45:50.707193] [INFO ] [LOCAL ] BWXT: Starting deletion of primary data. [2025-10-20T17:45:50.707553] [DEBUG] [LOCAL ] BWXT: Deleted primary data file. [2025-10-20T17:45:50.707616] [INFO ] [MAIN ] BWXT: Starting full primary refresh. [2025-10-20T17:45:50.801561] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T17:45:51.093777] [INFO ] [REMOTE ] BWXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:51.223241] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T17:45:51.451386] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-03. [2025-10-20T17:45:51.954753] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T17:45:52.251417] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-01. [2025-10-20T17:45:53.051394] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-09. [2025-10-20T17:45:53.851371] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-04. [2025-10-20T17:45:54.095590] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T17:45:54.413923] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T17:45:54.541604] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T17:45:54.651396] [INFO ] [REMOTE ] BWLP: Fetching primary data for 2025-10. [2025-10-20T17:45:55.451379] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-02. [2025-10-20T17:45:56.028347] [INFO ] [REMOTE ] BWLP: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:45:56.033727] [INFO ] [REMOTE ] BWLP: Full fetch complete (47924 records). [2025-10-20T17:45:56.033847] [DEBUG] [VALIDATOR ] BWLP: Validating primary data. [2025-10-20T17:45:56.256556] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-05. [2025-10-20T17:45:56.429407] [DEBUG] [VALIDATOR ] BWLP: All expected trading days are present. [2025-10-20T17:45:56.429488] [DEBUG] [VALIDATOR ] BWLP: Primary data validation passed. [2025-10-20T17:45:56.430066] [INFO ] [MAIN ] BWLP: Fetched primary data is valid. [2025-10-20T17:45:56.430163] [DEBUG] [LOCAL ] BWLP: Saving primary data. [2025-10-20T17:45:56.431296] [DEBUG] [LOCAL ] BWLP: Trimmed 47924 records to 32227 (keeping 200 days of data) [2025-10-20T17:45:56.621477] [DEBUG] [LOCAL ] BWLP: Primary data saved (32227 records). [2025-10-20T17:45:56.621741] [INFO ] [MAIN ] BWLP: Full history saved. [2025-10-20T17:45:56.621845] [INFO ] [MAIN ] Update Progress: 366/2367 symbols completed. BWLP took 34.4s. Estimated remaining: 4h 10m 47s. Rate: 8.0 symbols/min [2025-10-20T17:45:56.797854] [INFO ] [MAIN ] BX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:45:56.846878] [DEBUG] [LOCAL ] BX: Loaded 54263 existing records from primary storage. [2025-10-20T17:45:56.847224] [INFO ] [MAIN ] BX: Loaded 54263 existing records. [2025-10-20T17:45:57.052055] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T17:45:57.055605] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-06. [2025-10-20T17:45:57.055263] [WARN ] [MAIN ] BX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:45:57.072288] [INFO ] [MAIN ] BX: Wiping existing primary data before full refresh. [2025-10-20T17:45:57.073098] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T17:45:57.072873] [INFO ] [LOCAL ] BX: Starting deletion of primary data. [2025-10-20T17:45:57.076638] [DEBUG] [LOCAL ] BX: Deleted primary data file. [2025-10-20T17:45:57.076728] [INFO ] [MAIN ] BX: Starting full primary refresh. [2025-10-20T17:45:57.435221] [INFO ] [REMOTE ] BX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:45:57.587250] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T17:45:57.851379] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-01. [2025-10-20T17:45:58.651374] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-07. [2025-10-20T17:45:59.075263] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T17:45:59.451361] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-03. [2025-10-20T17:45:59.582768] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T17:46:00.251375] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-02. [2025-10-20T17:46:01.051386] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-06. [2025-10-20T17:46:01.170686] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T17:46:01.700387] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T17:46:01.857996] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-08. [2025-10-20T17:46:01.869310] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T17:46:02.602375] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T17:46:02.651419] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-09. [2025-10-20T17:46:03.219129] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:46:03.451425] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-03. [2025-10-20T17:46:04.251417] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-07. [2025-10-20T17:46:04.863352] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T17:46:05.051377] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-08. [2025-10-20T17:46:05.644568] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-10-20T17:46:05.851345] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-04. [2025-10-20T17:46:05.914054] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:46:06.651388] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-09. [2025-10-20T17:46:07.166425] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T17:46:07.451370] [INFO ] [REMOTE ] BWNB: Fetching primary data for 2025-10. [2025-10-20T17:46:07.511337] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T17:46:07.956543] [INFO ] [REMOTE ] BWNB: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:46:07.957488] [INFO ] [REMOTE ] BWNB: Full fetch complete (5579 records). [2025-10-20T17:46:07.957577] [DEBUG] [VALIDATOR ] BWNB: Validating primary data. [2025-10-20T17:46:08.259762] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-04. [2025-10-20T17:46:08.349393] [DEBUG] [VALIDATOR ] BWNB: All expected trading days are present. [2025-10-20T17:46:08.349467] [DEBUG] [VALIDATOR ] BWNB: Primary data validation passed. [2025-10-20T17:46:08.349631] [INFO ] [MAIN ] BWNB: Fetched primary data is valid. [2025-10-20T17:46:08.349707] [DEBUG] [LOCAL ] BWNB: Saving primary data. [2025-10-20T17:46:08.350092] [DEBUG] [LOCAL ] BWNB: Trimmed 5579 records to 4101 (keeping 200 days of data) [2025-10-20T17:46:08.374742] [DEBUG] [LOCAL ] BWNB: Primary data saved (4101 records). [2025-10-20T17:46:08.374840] [INFO ] [MAIN ] BWNB: Full history saved. [2025-10-20T17:46:08.374923] [INFO ] [MAIN ] Update Progress: 367/2367 symbols completed. BWNB took 34.8s. Estimated remaining: 4h 11m 3s. Rate: 8.0 symbols/min [2025-10-20T17:46:08.564341] [INFO ] [MAIN ] BXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:08.577371] [DEBUG] [LOCAL ] BXC: Loaded 13776 existing records from primary storage. [2025-10-20T17:46:08.577596] [INFO ] [MAIN ] BXC: Loaded 13776 existing records. [2025-10-20T17:46:08.762681] [WARN ] [MAIN ] BXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:08.762923] [INFO ] [MAIN ] BXC: Wiping existing primary data before full refresh. [2025-10-20T17:46:08.762970] [INFO ] [LOCAL ] BXC: Starting deletion of primary data. [2025-10-20T17:46:08.763191] [DEBUG] [LOCAL ] BXC: Deleted primary data file. [2025-10-20T17:46:08.763248] [INFO ] [MAIN ] BXC: Starting full primary refresh. [2025-10-20T17:46:09.059887] [INFO ] [REMOTE ] BWSN: Fetching primary data for 2025-10. [2025-10-20T17:46:09.155758] [INFO ] [REMOTE ] BXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:09.821119] [INFO ] [REMOTE ] BWSN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T17:46:09.822451] [INFO ] [REMOTE ] BWSN: Full fetch complete (6353 records). [2025-10-20T17:46:09.822518] [DEBUG] [VALIDATOR ] BWSN: Validating primary data. [2025-10-20T17:46:09.856439] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-01. [2025-10-20T17:46:10.197788] [DEBUG] [VALIDATOR ] BWSN: All expected trading days are present. [2025-10-20T17:46:10.197905] [DEBUG] [VALIDATOR ] BWSN: Primary data validation passed. [2025-10-20T17:46:10.198187] [INFO ] [MAIN ] BWSN: Fetched primary data is valid. [2025-10-20T17:46:10.198340] [DEBUG] [LOCAL ] BWSN: Saving primary data. [2025-10-20T17:46:10.198901] [DEBUG] [LOCAL ] BWSN: Trimmed 6353 records to 5120 (keeping 200 days of data) [2025-10-20T17:46:10.233875] [DEBUG] [LOCAL ] BWSN: Primary data saved (5120 records). [2025-10-20T17:46:10.234006] [INFO ] [MAIN ] BWSN: Full history saved. [2025-10-20T17:46:10.234084] [INFO ] [MAIN ] Update Progress: 368/2367 symbols completed. BWSN took 32.4s. Estimated remaining: 4h 10m 24s. Rate: 8.0 symbols/min [2025-10-20T17:46:10.403265] [INFO ] [MAIN ] BXMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:10.436305] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T17:46:10.483696] [DEBUG] [LOCAL ] BXMT: Loaded 48048 existing records from primary storage. [2025-10-20T17:46:10.484017] [INFO ] [MAIN ] BXMT: Loaded 48048 existing records. [2025-10-20T17:46:10.661908] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-05. [2025-10-20T17:46:10.668066] [WARN ] [MAIN ] BXMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:10.669444] [INFO ] [MAIN ] BXMT: Wiping existing primary data before full refresh. [2025-10-20T17:46:10.669538] [INFO ] [LOCAL ] BXMT: Starting deletion of primary data. [2025-10-20T17:46:10.674142] [DEBUG] [LOCAL ] BXMT: Deleted primary data file. [2025-10-20T17:46:10.675443] [INFO ] [MAIN ] BXMT: Starting full primary refresh. [2025-10-20T17:46:10.676291] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T17:46:11.050710] [INFO ] [REMOTE ] BXMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:11.451517] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-01. [2025-10-20T17:46:12.251440] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-02. [2025-10-20T17:46:12.434836] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:46:12.779511] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T17:46:12.905331] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T17:46:13.051429] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-02. [2025-10-20T17:46:13.851428] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-05. [2025-10-20T17:46:14.651401] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-03. [2025-10-20T17:46:14.702381] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T17:46:15.451444] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-06. [2025-10-20T17:46:15.463008] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:46:15.752568] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:46:16.251434] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-03. [2025-10-20T17:46:17.051471] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-04. [2025-10-20T17:46:17.177817] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:46:17.851364] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-07. [2025-10-20T17:46:18.017088] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T17:46:18.023186] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:46:18.651402] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-05. [2025-10-20T17:46:19.451422] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-04. [2025-10-20T17:46:19.546344] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T17:46:19.796876] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T17:46:20.251360] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-06. [2025-10-20T17:46:21.051402] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-06. [2025-10-20T17:46:21.210554] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T17:46:21.259819] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T17:46:21.851379] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-07. [2025-10-20T17:46:22.651347] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-05. [2025-10-20T17:46:22.789724] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:46:22.981257] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T17:46:23.451439] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-08. [2025-10-20T17:46:24.263504] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-07. [2025-10-20T17:46:24.284232] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T17:46:24.759913] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:46:25.051361] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-09. [2025-10-20T17:46:25.851351] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-06. [2025-10-20T17:46:26.110840] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T17:46:26.651431] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-08. [2025-10-20T17:46:26.969568] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:46:27.016011] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:46:27.451426] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-08. [2025-10-20T17:46:27.466203] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:46:28.251382] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-07. [2025-10-20T17:46:28.899259] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:46:29.051429] [INFO ] [REMOTE ] BWXT: Fetching primary data for 2025-10. [2025-10-20T17:46:29.831328] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:46:29.851341] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-08. [2025-10-20T17:46:30.651421] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-09. [2025-10-20T17:46:30.955996] [INFO ] [REMOTE ] BWXT: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:46:30.963910] [INFO ] [REMOTE ] BWXT: Full fetch complete (68036 records). [2025-10-20T17:46:30.964094] [DEBUG] [VALIDATOR ] BWXT: Validating primary data. [2025-10-20T17:46:31.372491] [DEBUG] [VALIDATOR ] BWXT: All expected trading days are present. [2025-10-20T17:46:31.372616] [DEBUG] [VALIDATOR ] BWXT: Primary data validation passed. [2025-10-20T17:46:31.373630] [INFO ] [MAIN ] BWXT: Fetched primary data is valid. [2025-10-20T17:46:31.373951] [DEBUG] [LOCAL ] BWXT: Saving primary data. [2025-10-20T17:46:31.383948] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T17:46:31.388167] [DEBUG] [LOCAL ] BWXT: Trimmed 68036 records to 47285 (keeping 200 days of data) [2025-10-20T17:46:31.453189] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-09. [2025-10-20T17:46:31.529350] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T17:46:31.718280] [DEBUG] [LOCAL ] BWXT: Primary data saved (47285 records). [2025-10-20T17:46:31.718453] [INFO ] [MAIN ] BWXT: Full history saved. [2025-10-20T17:46:31.719206] [INFO ] [MAIN ] Update Progress: 369/2367 symbols completed. BWXT took 41.5s. Estimated remaining: 4h 11m 32s. Rate: 7.9 symbols/min [2025-10-20T17:46:31.904216] [INFO ] [MAIN ] BXMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:31.927063] [DEBUG] [LOCAL ] BXMX: Loaded 19717 existing records from primary storage. [2025-10-20T17:46:31.927433] [INFO ] [MAIN ] BXMX: Loaded 19717 existing records. [2025-10-20T17:46:32.142303] [WARN ] [MAIN ] BXMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:32.142566] [INFO ] [MAIN ] BXMX: Wiping existing primary data before full refresh. [2025-10-20T17:46:32.142614] [INFO ] [LOCAL ] BXMX: Starting deletion of primary data. [2025-10-20T17:46:32.142844] [DEBUG] [LOCAL ] BXMX: Deleted primary data file. [2025-10-20T17:46:32.142898] [INFO ] [MAIN ] BXMX: Starting full primary refresh. [2025-10-20T17:46:32.256696] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-09. [2025-10-20T17:46:32.521635] [INFO ] [REMOTE ] BXMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:33.051414] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-01. [2025-10-20T17:46:33.679394] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T17:46:33.851379] [INFO ] [REMOTE ] BXC: Fetching primary data for 2025-10. [2025-10-20T17:46:34.091520] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T17:46:34.134983] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T17:46:34.651415] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-02. [2025-10-20T17:46:34.750938] [INFO ] [REMOTE ] BXC: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T17:46:34.754129] [INFO ] [REMOTE ] BXC: Full fetch complete (18287 records). [2025-10-20T17:46:34.754231] [DEBUG] [VALIDATOR ] BXC: Validating primary data. [2025-10-20T17:46:35.135305] [DEBUG] [VALIDATOR ] BXC: All expected trading days are present. [2025-10-20T17:46:35.135405] [DEBUG] [VALIDATOR ] BXC: Primary data validation passed. [2025-10-20T17:46:35.135773] [INFO ] [MAIN ] BXC: Fetched primary data is valid. [2025-10-20T17:46:35.135833] [DEBUG] [LOCAL ] BXC: Saving primary data. [2025-10-20T17:46:35.136391] [DEBUG] [LOCAL ] BXC: Trimmed 18287 records to 13571 (keeping 200 days of data) [2025-10-20T17:46:35.213040] [DEBUG] [LOCAL ] BXC: Primary data saved (13571 records). [2025-10-20T17:46:35.213177] [INFO ] [MAIN ] BXC: Full history saved. [2025-10-20T17:46:35.213251] [INFO ] [MAIN ] Update Progress: 370/2367 symbols completed. BXC took 26.8s. Estimated remaining: 4h 11m 3s. Rate: 8.0 symbols/min [2025-10-20T17:46:35.381948] [INFO ] [MAIN ] BXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:35.428237] [DEBUG] [LOCAL ] BXP: Loaded 50594 existing records from primary storage. [2025-10-20T17:46:35.428598] [INFO ] [MAIN ] BXP: Loaded 50594 existing records. [2025-10-20T17:46:35.459570] [INFO ] [REMOTE ] BXMT: Fetching primary data for 2025-10. [2025-10-20T17:46:35.632474] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:46:35.636592] [WARN ] [MAIN ] BXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:35.641673] [INFO ] [MAIN ] BXP: Wiping existing primary data before full refresh. [2025-10-20T17:46:35.641752] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-10-20T17:46:35.641952] [DEBUG] [LOCAL ] BXP: Deleted primary data file. [2025-10-20T17:46:35.641992] [INFO ] [MAIN ] BXP: Starting full primary refresh. [2025-10-20T17:46:36.021250] [INFO ] [REMOTE ] BXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:36.251399] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-01. [2025-10-20T17:46:37.051436] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-03. [2025-10-20T17:46:37.317093] [INFO ] [REMOTE ] BXMT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T17:46:37.324820] [INFO ] [REMOTE ] BXMT: Full fetch complete (69895 records). [2025-10-20T17:46:37.324988] [DEBUG] [VALIDATOR ] BXMT: Validating primary data. [2025-10-20T17:46:37.719488] [DEBUG] [VALIDATOR ] BXMT: All expected trading days are present. [2025-10-20T17:46:37.719632] [DEBUG] [VALIDATOR ] BXMT: Primary data validation passed. [2025-10-20T17:46:37.720581] [INFO ] [MAIN ] BXMT: Fetched primary data is valid. [2025-10-20T17:46:37.720690] [DEBUG] [LOCAL ] BXMT: Saving primary data. [2025-10-20T17:46:37.722159] [DEBUG] [LOCAL ] BXMT: Trimmed 69895 records to 47292 (keeping 200 days of data) [2025-10-20T17:46:37.855048] [INFO ] [REMOTE ] BX: Fetching primary data for 2025-10. [2025-10-20T17:46:37.862953] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T17:46:38.065752] [DEBUG] [LOCAL ] BXMT: Primary data saved (47292 records). [2025-10-20T17:46:38.067326] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T17:46:38.066154] [INFO ] [MAIN ] BXMT: Full history saved. [2025-10-20T17:46:38.067501] [INFO ] [MAIN ] Update Progress: 371/2367 symbols completed. BXMT took 27.8s. Estimated remaining: 4h 10m 30s. Rate: 8.0 symbols/min [2025-10-20T17:46:38.248102] [INFO ] [MAIN ] BXSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:38.289873] [DEBUG] [LOCAL ] BXSL: Loaded 48894 existing records from primary storage. [2025-10-20T17:46:38.290161] [INFO ] [MAIN ] BXSL: Loaded 48894 existing records. [2025-10-20T17:46:38.465248] [WARN ] [MAIN ] BXSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:38.465479] [INFO ] [MAIN ] BXSL: Wiping existing primary data before full refresh. [2025-10-20T17:46:38.465557] [INFO ] [LOCAL ] BXSL: Starting deletion of primary data. [2025-10-20T17:46:38.465810] [DEBUG] [LOCAL ] BXSL: Deleted primary data file. [2025-10-20T17:46:38.465880] [INFO ] [MAIN ] BXSL: Starting full primary refresh. [2025-10-20T17:46:38.656525] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-04. [2025-10-20T17:46:38.829092] [INFO ] [REMOTE ] BXSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:39.451495] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-02. [2025-10-20T17:46:39.937314] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T17:46:39.977410] [INFO ] [REMOTE ] BX: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T17:46:39.985885] [INFO ] [REMOTE ] BX: Full fetch complete (77591 records). [2025-10-20T17:46:39.986050] [DEBUG] [VALIDATOR ] BX: Validating primary data. [2025-10-20T17:46:40.256941] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-05. [2025-10-20T17:46:40.388813] [DEBUG] [VALIDATOR ] BX: All expected trading days are present. [2025-10-20T17:46:40.388897] [DEBUG] [VALIDATOR ] BX: Primary data validation passed. [2025-10-20T17:46:40.389685] [INFO ] [MAIN ] BX: Fetched primary data is valid. [2025-10-20T17:46:40.389746] [DEBUG] [LOCAL ] BX: Saving primary data. [2025-10-20T17:46:40.391181] [DEBUG] [LOCAL ] BX: Trimmed 77591 records to 53483 (keeping 200 days of data) [2025-10-20T17:46:40.719312] [DEBUG] [LOCAL ] BX: Primary data saved (53483 records). [2025-10-20T17:46:40.719452] [INFO ] [MAIN ] BX: Full history saved. [2025-10-20T17:46:40.719530] [INFO ] [MAIN ] Update Progress: 372/2367 symbols completed. BX took 44.1s. Estimated remaining: 4h 9m 56s. Rate: 8.0 symbols/min [2025-10-20T17:46:40.896406] [INFO ] [MAIN ] BY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:40.942282] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:46:40.940764] [DEBUG] [LOCAL ] BY: Loaded 22175 existing records from primary storage. [2025-10-20T17:46:40.952735] [INFO ] [MAIN ] BY: Loaded 22175 existing records. [2025-10-20T17:46:41.056503] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-01. [2025-10-20T17:46:41.138132] [WARN ] [MAIN ] BY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:41.138480] [INFO ] [MAIN ] BY: Wiping existing primary data before full refresh. [2025-10-20T17:46:41.138529] [INFO ] [LOCAL ] BY: Starting deletion of primary data. [2025-10-20T17:46:41.138846] [DEBUG] [LOCAL ] BY: Deleted primary data file. [2025-10-20T17:46:41.138902] [INFO ] [MAIN ] BY: Starting full primary refresh. [2025-10-20T17:46:41.521589] [INFO ] [REMOTE ] BY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:41.523020] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T17:46:41.851424] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-03. [2025-10-20T17:46:42.551095] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T17:46:42.651486] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-01. [2025-10-20T17:46:43.451431] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-06. [2025-10-20T17:46:43.520135] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:46:43.559991] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:46:44.251361] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-04. [2025-10-20T17:46:44.330498] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T17:46:45.051432] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-02. [2025-10-20T17:46:45.851456] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-07. [2025-10-20T17:46:45.949274] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T17:46:46.037804] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:46:46.495593] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T17:46:46.651441] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-02. [2025-10-20T17:46:47.451469] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-03. [2025-10-20T17:46:48.245710] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T17:46:48.254975] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-05. [2025-10-20T17:46:48.423407] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T17:46:49.051371] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-08. [2025-10-20T17:46:49.851394] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-03. [2025-10-20T17:46:49.908591] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T17:46:49.976960] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:46:50.651372] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-06. [2025-10-20T17:46:51.218874] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T17:46:51.451387] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-04. [2025-10-20T17:46:52.251344] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-09. [2025-10-20T17:46:52.366217] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T17:46:52.840422] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T17:46:53.051377] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-07. [2025-10-20T17:46:53.259604] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T17:46:53.851425] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-05. [2025-10-20T17:46:54.425464] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T17:46:54.651366] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-08. [2025-10-20T17:46:54.923359] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:46:55.451374] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-06. [2025-10-20T17:46:56.251370] [INFO ] [REMOTE ] BXMX: Fetching primary data for 2025-10. [2025-10-20T17:46:56.315185] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T17:46:56.582905] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T17:46:57.051345] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-07. [2025-10-20T17:46:57.117898] [INFO ] [REMOTE ] BXMX: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T17:46:57.122619] [INFO ] [REMOTE ] BXMX: Full fetch complete (28676 records). [2025-10-20T17:46:57.122770] [DEBUG] [VALIDATOR ] BXMX: Validating primary data. [2025-10-20T17:46:57.557607] [DEBUG] [VALIDATOR ] BXMX: All expected trading days are present. [2025-10-20T17:46:57.557768] [DEBUG] [VALIDATOR ] BXMX: Primary data validation passed. [2025-10-20T17:46:57.558427] [INFO ] [MAIN ] BXMX: Fetched primary data is valid. [2025-10-20T17:46:57.558496] [DEBUG] [LOCAL ] BXMX: Saving primary data. [2025-10-20T17:46:57.559751] [DEBUG] [LOCAL ] BXMX: Trimmed 28676 records to 19355 (keeping 200 days of data) [2025-10-20T17:46:57.681978] [DEBUG] [LOCAL ] BXMX: Primary data saved (19355 records). [2025-10-20T17:46:57.682185] [INFO ] [MAIN ] BXMX: Full history saved. [2025-10-20T17:46:57.682287] [INFO ] [MAIN ] Update Progress: 373/2367 symbols completed. BXMX took 26.0s. Estimated remaining: 4h 10m 39s. Rate: 8.0 symbols/min [2025-10-20T17:46:57.862715] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-09. [2025-10-20T17:46:57.866451] [INFO ] [MAIN ] BYD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:46:57.916980] [DEBUG] [LOCAL ] BYD: Loaded 45593 existing records from primary storage. [2025-10-20T17:46:57.917486] [INFO ] [MAIN ] BYD: Loaded 45593 existing records. [2025-10-20T17:46:58.131716] [WARN ] [MAIN ] BYD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:46:58.146842] [INFO ] [MAIN ] BYD: Wiping existing primary data before full refresh. [2025-10-20T17:46:58.147044] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-10-20T17:46:58.147689] [DEBUG] [LOCAL ] BYD: Deleted primary data file. [2025-10-20T17:46:58.148207] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T17:46:58.148461] [INFO ] [MAIN ] BYD: Starting full primary refresh. [2025-10-20T17:46:58.538495] [INFO ] [REMOTE ] BYD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:46:58.651448] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-01. [2025-10-20T17:46:59.444236] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:46:59.458127] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-08. [2025-10-20T17:47:00.064813] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T17:47:00.251357] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-02. [2025-10-20T17:47:00.338967] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T17:47:01.051370] [INFO ] [REMOTE ] BXP: Fetching primary data for 2025-10. [2025-10-20T17:47:01.851436] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-04. [2025-10-20T17:47:01.912626] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T17:47:02.651387] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-03. [2025-10-20T17:47:02.893805] [INFO ] [REMOTE ] BXP: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T17:47:02.901371] [INFO ] [REMOTE ] BXP: Full fetch complete (71886 records). [2025-10-20T17:47:02.901515] [DEBUG] [VALIDATOR ] BXP: Validating primary data. [2025-10-20T17:47:02.902361] [FATAL] [VALIDATOR ] BXP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:02.902452] [FATAL] [MAIN ] BXP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:02.902496] [WARN ] [MAIN ] BXP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:02.902537] [INFO ] [LOCAL ] BXP: Starting deletion of primary data. [2025-10-20T17:47:02.902598] [DEBUG] [LOCAL ] BXP: No primary data file found to delete. [2025-10-20T17:47:02.902635] [INFO ] [LOCAL ] BXP: Starting deletion of derivative data. [2025-10-20T17:47:02.902879] [DEBUG] [LOCAL ] BXP: Deleted derivative data file. [2025-10-20T17:47:02.903206] [INFO ] [MAIN ] Update Progress: 374/2367 symbols completed. BXP took 27.7s. Estimated remaining: 4h 10m 19s. Rate: 8.0 symbols/min [2025-10-20T17:47:03.089181] [INFO ] [MAIN ] BYM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:03.103663] [DEBUG] [LOCAL ] BYM: Loaded 11842 existing records from primary storage. [2025-10-20T17:47:03.103904] [INFO ] [MAIN ] BYM: Loaded 11842 existing records. [2025-10-20T17:47:03.277215] [WARN ] [MAIN ] BYM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:03.277604] [INFO ] [MAIN ] BYM: Wiping existing primary data before full refresh. [2025-10-20T17:47:03.277680] [INFO ] [LOCAL ] BYM: Starting deletion of primary data. [2025-10-20T17:47:03.277861] [DEBUG] [LOCAL ] BYM: Deleted primary data file. [2025-10-20T17:47:03.277899] [INFO ] [MAIN ] BYM: Starting full primary refresh. [2025-10-20T17:47:03.462175] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-09. [2025-10-20T17:47:03.634851] [INFO ] [REMOTE ] BYM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:03.733388] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T17:47:04.256685] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-05. [2025-10-20T17:47:04.258118] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T17:47:04.305700] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T17:47:05.051362] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-04. [2025-10-20T17:47:05.720709] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T17:47:05.851393] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-06. [2025-10-20T17:47:06.530396] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:47:06.651374] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-05. [2025-10-20T17:47:07.385229] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T17:47:07.451351] [INFO ] [REMOTE ] BY: Fetching primary data for 2025-10. [2025-10-20T17:47:08.215918] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:47:08.251332] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-06. [2025-10-20T17:47:08.448762] [INFO ] [REMOTE ] BY: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:47:08.452152] [INFO ] [REMOTE ] BY: Full fetch complete (29694 records). [2025-10-20T17:47:08.452237] [DEBUG] [VALIDATOR ] BY: Validating primary data. [2025-10-20T17:47:08.851464] [DEBUG] [VALIDATOR ] BY: All expected trading days are present. [2025-10-20T17:47:08.851580] [DEBUG] [VALIDATOR ] BY: Primary data validation passed. [2025-10-20T17:47:08.852064] [INFO ] [MAIN ] BY: Fetched primary data is valid. [2025-10-20T17:47:08.852125] [DEBUG] [LOCAL ] BY: Saving primary data. [2025-10-20T17:47:08.853099] [DEBUG] [LOCAL ] BY: Trimmed 29694 records to 21868 (keeping 200 days of data) [2025-10-20T17:47:08.982605] [DEBUG] [LOCAL ] BY: Primary data saved (21868 records). [2025-10-20T17:47:08.982763] [INFO ] [MAIN ] BY: Full history saved. [2025-10-20T17:47:08.982847] [INFO ] [MAIN ] Update Progress: 375/2367 symbols completed. BY took 28.3s. Estimated remaining: 4h 10m 4s. Rate: 8.0 symbols/min [2025-10-20T17:47:09.060009] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-07. [2025-10-20T17:47:09.159350] [INFO ] [MAIN ] BZH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:09.189917] [DEBUG] [LOCAL ] BZH: Loaded 32798 existing records from primary storage. [2025-10-20T17:47:09.190209] [INFO ] [MAIN ] BZH: Loaded 32798 existing records. [2025-10-20T17:47:09.379786] [WARN ] [MAIN ] BZH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:09.380011] [INFO ] [MAIN ] BZH: Wiping existing primary data before full refresh. [2025-10-20T17:47:09.380059] [INFO ] [LOCAL ] BZH: Starting deletion of primary data. [2025-10-20T17:47:09.380272] [DEBUG] [LOCAL ] BZH: Deleted primary data file. [2025-10-20T17:47:09.380332] [INFO ] [MAIN ] BZH: Starting full primary refresh. [2025-10-20T17:47:09.757222] [INFO ] [REMOTE ] BZH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:09.791780] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T17:47:09.851440] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-01. [2025-10-20T17:47:10.627578] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:47:10.651350] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-02. [2025-10-20T17:47:10.730157] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T17:47:11.304352] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:47:11.451441] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-07. [2025-10-20T17:47:12.251461] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-08. [2025-10-20T17:47:13.054485] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-03. [2025-10-20T17:47:13.062451] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:47:13.851389] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-08. [2025-10-20T17:47:13.996955] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T17:47:14.046975] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T17:47:14.651438] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-01. [2025-10-20T17:47:15.244100] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T17:47:15.451441] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-09. [2025-10-20T17:47:15.918743] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T17:47:16.251382] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-02. [2025-10-20T17:47:16.855605] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T17:47:17.051460] [INFO ] [REMOTE ] BYD: Fetching primary data for 2025-10. [2025-10-20T17:47:17.742611] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T17:47:17.851416] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-03. [2025-10-20T17:47:18.302030] [INFO ] [REMOTE ] BYD: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:47:18.309193] [INFO ] [REMOTE ] BYD: Full fetch complete (65028 records). [2025-10-20T17:47:18.309411] [DEBUG] [VALIDATOR ] BYD: Validating primary data. [2025-10-20T17:47:18.310741] [FATAL] [VALIDATOR ] BYD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:18.310863] [FATAL] [MAIN ] BYD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:18.310932] [WARN ] [MAIN ] BYD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:18.311035] [INFO ] [LOCAL ] BYD: Starting deletion of primary data. [2025-10-20T17:47:18.311125] [DEBUG] [LOCAL ] BYD: No primary data file found to delete. [2025-10-20T17:47:18.311166] [INFO ] [LOCAL ] BYD: Starting deletion of derivative data. [2025-10-20T17:47:18.311618] [DEBUG] [LOCAL ] BYD: Deleted derivative data file. [2025-10-20T17:47:18.312082] [INFO ] [MAIN ] Update Progress: 376/2367 symbols completed. BYD took 20.6s. Estimated remaining: 4h 10m 6s. Rate: 8.0 symbols/min [2025-10-20T17:47:18.513076] [INFO ] [MAIN ] C: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:18.567807] [DEBUG] [LOCAL ] C: Loaded 54520 existing records from primary storage. [2025-10-20T17:47:18.568219] [INFO ] [MAIN ] C: Loaded 54520 existing records. [2025-10-20T17:47:18.656563] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-04. [2025-10-20T17:47:18.782907] [WARN ] [MAIN ] C: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:18.783122] [INFO ] [MAIN ] C: Wiping existing primary data before full refresh. [2025-10-20T17:47:18.783169] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-10-20T17:47:18.783670] [DEBUG] [LOCAL ] C: Deleted primary data file. [2025-10-20T17:47:18.783729] [INFO ] [MAIN ] C: Starting full primary refresh. [2025-10-20T17:47:19.160808] [INFO ] [REMOTE ] C: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:19.258182] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:47:19.451421] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-09. [2025-10-20T17:47:19.637212] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:47:20.251378] [INFO ] [REMOTE ] C: Fetching primary data for 2025-01. [2025-10-20T17:47:21.051404] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-04. [2025-10-20T17:47:21.292612] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:47:21.851382] [INFO ] [REMOTE ] BXSL: Fetching primary data for 2025-10. [2025-10-20T17:47:22.356322] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T17:47:22.474040] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T17:47:22.651404] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-05. [2025-10-20T17:47:23.451365] [INFO ] [REMOTE ] C: Fetching primary data for 2025-02. [2025-10-20T17:47:23.808410] [INFO ] [REMOTE ] BXSL: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T17:47:23.816287] [INFO ] [REMOTE ] BXSL: Full fetch complete (70017 records). [2025-10-20T17:47:23.816439] [DEBUG] [VALIDATOR ] BXSL: Validating primary data. [2025-10-20T17:47:24.129802] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:47:24.230281] [DEBUG] [VALIDATOR ] BXSL: All expected trading days are present. [2025-10-20T17:47:24.230378] [DEBUG] [VALIDATOR ] BXSL: Primary data validation passed. [2025-10-20T17:47:24.231147] [INFO ] [MAIN ] BXSL: Fetched primary data is valid. [2025-10-20T17:47:24.231213] [DEBUG] [LOCAL ] BXSL: Saving primary data. [2025-10-20T17:47:24.232596] [DEBUG] [LOCAL ] BXSL: Trimmed 70017 records to 48208 (keeping 200 days of data) [2025-10-20T17:47:24.284922] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-05. [2025-10-20T17:47:24.538113] [DEBUG] [LOCAL ] BXSL: Primary data saved (48208 records). [2025-10-20T17:47:24.538278] [INFO ] [MAIN ] BXSL: Full history saved. [2025-10-20T17:47:24.538423] [INFO ] [MAIN ] Update Progress: 377/2367 symbols completed. BXSL took 46.5s. Estimated remaining: 4h 9m 52s. Rate: 8.0 symbols/min [2025-10-20T17:47:24.712572] [INFO ] [MAIN ] CAAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:24.733067] [DEBUG] [LOCAL ] CAAP: Loaded 20139 existing records from primary storage. [2025-10-20T17:47:24.733308] [INFO ] [MAIN ] CAAP: Loaded 20139 existing records. [2025-10-20T17:47:24.905463] [WARN ] [MAIN ] CAAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:24.905632] [INFO ] [MAIN ] CAAP: Wiping existing primary data before full refresh. [2025-10-20T17:47:24.905710] [INFO ] [LOCAL ] CAAP: Starting deletion of primary data. [2025-10-20T17:47:24.905915] [DEBUG] [LOCAL ] CAAP: Deleted primary data file. [2025-10-20T17:47:24.905971] [INFO ] [MAIN ] CAAP: Starting full primary refresh. [2025-10-20T17:47:25.062767] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-06. [2025-10-20T17:47:25.196216] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:47:25.279955] [INFO ] [REMOTE ] CAAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:25.318628] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T17:47:25.851432] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-06. [2025-10-20T17:47:26.389924] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:47:26.651350] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-01. [2025-10-20T17:47:26.699783] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T17:47:27.451353] [INFO ] [REMOTE ] C: Fetching primary data for 2025-03. [2025-10-20T17:47:27.840997] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:47:28.251538] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-07. [2025-10-20T17:47:29.056228] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-07. [2025-10-20T17:47:29.058165] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T17:47:29.296020] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T17:47:29.851388] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-08. [2025-10-20T17:47:30.417253] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:47:30.431024] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T17:47:30.651355] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-08. [2025-10-20T17:47:31.451383] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-09. [2025-10-20T17:47:31.772264] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:47:32.251357] [INFO ] [REMOTE ] C: Fetching primary data for 2025-04. [2025-10-20T17:47:32.310390] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T17:47:33.051394] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-09. [2025-10-20T17:47:33.851424] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-02. [2025-10-20T17:47:34.385572] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T17:47:34.433816] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:47:34.651439] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-03. [2025-10-20T17:47:34.726803] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T17:47:35.451431] [INFO ] [REMOTE ] BYM: Fetching primary data for 2025-10. [2025-10-20T17:47:35.648315] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:47:36.251427] [INFO ] [REMOTE ] C: Fetching primary data for 2025-05. [2025-10-20T17:47:36.439972] [INFO ] [REMOTE ] BYM: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:47:36.442272] [INFO ] [REMOTE ] BYM: Full fetch complete (17337 records). [2025-10-20T17:47:36.442399] [DEBUG] [VALIDATOR ] BYM: Validating primary data. [2025-10-20T17:47:36.858673] [DEBUG] [VALIDATOR ] BYM: All expected trading days are present. [2025-10-20T17:47:36.858768] [DEBUG] [VALIDATOR ] BYM: Primary data validation passed. [2025-10-20T17:47:36.859099] [INFO ] [MAIN ] BYM: Fetched primary data is valid. [2025-10-20T17:47:36.859140] [DEBUG] [LOCAL ] BYM: Saving primary data. [2025-10-20T17:47:36.859781] [DEBUG] [LOCAL ] BYM: Trimmed 17337 records to 11543 (keeping 200 days of data) [2025-10-20T17:47:36.927313] [DEBUG] [LOCAL ] BYM: Primary data saved (11543 records). [2025-10-20T17:47:36.927448] [INFO ] [MAIN ] BYM: Full history saved. [2025-10-20T17:47:36.927527] [INFO ] [MAIN ] Update Progress: 378/2367 symbols completed. BYM took 34.0s. Estimated remaining: 4h 10m 10s. Rate: 8.0 symbols/min [2025-10-20T17:47:37.056559] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-04. [2025-10-20T17:47:37.102535] [INFO ] [MAIN ] CABO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:37.119220] [DEBUG] [LOCAL ] CABO: Loaded 16919 existing records from primary storage. [2025-10-20T17:47:37.119468] [INFO ] [MAIN ] CABO: Loaded 16919 existing records. [2025-10-20T17:47:37.299941] [WARN ] [MAIN ] CABO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:37.300167] [INFO ] [MAIN ] CABO: Wiping existing primary data before full refresh. [2025-10-20T17:47:37.300217] [INFO ] [LOCAL ] CABO: Starting deletion of primary data. [2025-10-20T17:47:37.300379] [DEBUG] [LOCAL ] CABO: Deleted primary data file. [2025-10-20T17:47:37.300420] [INFO ] [MAIN ] CABO: Starting full primary refresh. [2025-10-20T17:47:37.661724] [INFO ] [REMOTE ] CABO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:37.851418] [INFO ] [REMOTE ] BZH: Fetching primary data for 2025-10. [2025-10-20T17:47:37.989522] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T17:47:38.013677] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T17:47:38.651476] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-01. [2025-10-20T17:47:39.185537] [INFO ] [REMOTE ] BZH: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:47:39.190591] [INFO ] [REMOTE ] BZH: Full fetch complete (47167 records). [2025-10-20T17:47:39.190785] [DEBUG] [VALIDATOR ] BZH: Validating primary data. [2025-10-20T17:47:39.456514] [INFO ] [REMOTE ] C: Fetching primary data for 2025-06. [2025-10-20T17:47:39.607382] [DEBUG] [VALIDATOR ] BZH: All expected trading days are present. [2025-10-20T17:47:39.616758] [DEBUG] [VALIDATOR ] BZH: Primary data validation passed. [2025-10-20T17:47:39.617956] [INFO ] [MAIN ] BZH: Fetched primary data is valid. [2025-10-20T17:47:39.618377] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T17:47:39.621878] [DEBUG] [LOCAL ] BZH: Saving primary data. [2025-10-20T17:47:39.623575] [DEBUG] [LOCAL ] BZH: Trimmed 47167 records to 32291 (keeping 200 days of data) [2025-10-20T17:47:39.814925] [DEBUG] [LOCAL ] BZH: Primary data saved (32291 records). [2025-10-20T17:47:39.815074] [INFO ] [MAIN ] BZH: Full history saved. [2025-10-20T17:47:39.815153] [INFO ] [MAIN ] Update Progress: 379/2367 symbols completed. BZH took 30.8s. Estimated remaining: 4h 9m 38s. Rate: 8.0 symbols/min [2025-10-20T17:47:39.982410] [INFO ] [MAIN ] CACI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:40.004163] [DEBUG] [LOCAL ] CACI: Loaded 23405 existing records from primary storage. [2025-10-20T17:47:40.004450] [INFO ] [MAIN ] CACI: Loaded 23405 existing records. [2025-10-20T17:47:40.181063] [WARN ] [MAIN ] CACI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:40.181289] [INFO ] [MAIN ] CACI: Wiping existing primary data before full refresh. [2025-10-20T17:47:40.181336] [INFO ] [LOCAL ] CACI: Starting deletion of primary data. [2025-10-20T17:47:40.181526] [DEBUG] [LOCAL ] CACI: Deleted primary data file. [2025-10-20T17:47:40.181564] [INFO ] [MAIN ] CACI: Starting full primary refresh. [2025-10-20T17:47:40.256590] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-02. [2025-10-20T17:47:40.544039] [INFO ] [REMOTE ] CACI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:41.031260] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T17:47:41.051364] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-01. [2025-10-20T17:47:41.275925] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T17:47:41.851412] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-05. [2025-10-20T17:47:42.385003] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T17:47:42.651396] [INFO ] [REMOTE ] C: Fetching primary data for 2025-07. [2025-10-20T17:47:42.699519] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T17:47:43.451441] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-03. [2025-10-20T17:47:44.211512] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T17:47:44.251392] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-06. [2025-10-20T17:47:44.836471] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T17:47:44.893779] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T17:47:45.051388] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-02. [2025-10-20T17:47:45.851363] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-04. [2025-10-20T17:47:46.488519] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T17:47:46.651409] [INFO ] [REMOTE ] C: Fetching primary data for 2025-08. [2025-10-20T17:47:46.723395] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T17:47:47.451435] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-05. [2025-10-20T17:47:48.251419] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-07. [2025-10-20T17:47:48.582690] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:47:48.645672] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T17:47:49.051386] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-06. [2025-10-20T17:47:49.097821] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T17:47:49.851357] [INFO ] [REMOTE ] C: Fetching primary data for 2025-09. [2025-10-20T17:47:50.187213] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T17:47:50.651374] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-07. [2025-10-20T17:47:51.451343] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-08. [2025-10-20T17:47:51.550132] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T17:47:51.697059] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:47:52.251378] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-08. [2025-10-20T17:47:52.465430] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:47:53.051350] [INFO ] [REMOTE ] C: Fetching primary data for 2025-10. [2025-10-20T17:47:53.207057] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:47:53.851377] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-09. [2025-10-20T17:47:54.651356] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-09. [2025-10-20T17:47:55.018062] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T17:47:55.461368] [INFO ] [REMOTE ] CAAP: Fetching primary data for 2025-10. [2025-10-20T17:47:55.466427] [INFO ] [REMOTE ] C: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T17:47:55.483688] [INFO ] [REMOTE ] C: Full fetch complete (77528 records). [2025-10-20T17:47:55.484777] [DEBUG] [VALIDATOR ] C: Validating primary data. [2025-10-20T17:47:55.484693] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T17:47:55.489352] [FATAL] [VALIDATOR ] C: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:47:55.489504] [FATAL] [MAIN ] C: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:47:55.489581] [WARN ] [MAIN ] C: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:47:55.489695] [INFO ] [LOCAL ] C: Starting deletion of primary data. [2025-10-20T17:47:55.489783] [DEBUG] [LOCAL ] C: No primary data file found to delete. [2025-10-20T17:47:55.489834] [INFO ] [LOCAL ] C: Starting deletion of derivative data. [2025-10-20T17:47:55.490005] [DEBUG] [LOCAL ] C: Deleted derivative data file. [2025-10-20T17:47:55.490109] [INFO ] [MAIN ] Update Progress: 380/2367 symbols completed. C took 37.2s. Estimated remaining: 4h 10m 13s. Rate: 7.9 symbols/min [2025-10-20T17:47:55.660075] [INFO ] [MAIN ] CADE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:55.706399] [DEBUG] [LOCAL ] CADE: Loaded 50257 existing records from primary storage. [2025-10-20T17:47:55.706767] [INFO ] [MAIN ] CADE: Loaded 50257 existing records. [2025-10-20T17:47:55.883069] [WARN ] [MAIN ] CADE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:55.883285] [INFO ] [MAIN ] CADE: Wiping existing primary data before full refresh. [2025-10-20T17:47:55.883332] [INFO ] [LOCAL ] CADE: Starting deletion of primary data. [2025-10-20T17:47:55.883554] [DEBUG] [LOCAL ] CADE: Deleted primary data file. [2025-10-20T17:47:55.883609] [INFO ] [MAIN ] CADE: Starting full primary refresh. [2025-10-20T17:47:56.238770] [INFO ] [REMOTE ] CADE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:56.251344] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-01. [2025-10-20T17:47:56.591272] [INFO ] [REMOTE ] CAAP: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T17:47:56.594990] [INFO ] [REMOTE ] CAAP: Full fetch complete (27978 records). [2025-10-20T17:47:56.595068] [DEBUG] [VALIDATOR ] CAAP: Validating primary data. [2025-10-20T17:47:56.960632] [DEBUG] [VALIDATOR ] CAAP: All expected trading days are present. [2025-10-20T17:47:56.960954] [DEBUG] [VALIDATOR ] CAAP: Primary data validation passed. [2025-10-20T17:47:56.961380] [INFO ] [MAIN ] CAAP: Fetched primary data is valid. [2025-10-20T17:47:56.961421] [DEBUG] [LOCAL ] CAAP: Saving primary data. [2025-10-20T17:47:56.962084] [DEBUG] [LOCAL ] CAAP: Trimmed 27978 records to 19913 (keeping 200 days of data) [2025-10-20T17:47:57.053549] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-03. [2025-10-20T17:47:57.076048] [DEBUG] [LOCAL ] CAAP: Primary data saved (19913 records). [2025-10-20T17:47:57.076151] [INFO ] [MAIN ] CAAP: Full history saved. [2025-10-20T17:47:57.076226] [INFO ] [MAIN ] Update Progress: 381/2367 symbols completed. CAAP took 32.5s. Estimated remaining: 4h 9m 34s. Rate: 8.0 symbols/min [2025-10-20T17:47:57.246808] [INFO ] [MAIN ] CAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:57.280828] [DEBUG] [LOCAL ] CAE: Loaded 39813 existing records from primary storage. [2025-10-20T17:47:57.281137] [INFO ] [MAIN ] CAE: Loaded 39813 existing records. [2025-10-20T17:47:57.460170] [WARN ] [MAIN ] CAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:57.460408] [INFO ] [MAIN ] CAE: Wiping existing primary data before full refresh. [2025-10-20T17:47:57.460456] [INFO ] [LOCAL ] CAE: Starting deletion of primary data. [2025-10-20T17:47:57.460943] [DEBUG] [LOCAL ] CAE: Deleted primary data file. [2025-10-20T17:47:57.461002] [INFO ] [MAIN ] CAE: Starting full primary refresh. [2025-10-20T17:47:57.852062] [INFO ] [REMOTE ] CABO: Fetching primary data for 2025-10. [2025-10-20T17:47:57.852002] [INFO ] [REMOTE ] CAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:47:57.878442] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:47:58.651354] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-01. [2025-10-20T17:47:58.728246] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T17:47:58.838993] [INFO ] [REMOTE ] CABO: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T17:47:58.841418] [INFO ] [REMOTE ] CABO: Full fetch complete (23148 records). [2025-10-20T17:47:58.841552] [DEBUG] [VALIDATOR ] CABO: Validating primary data. [2025-10-20T17:47:59.216630] [DEBUG] [VALIDATOR ] CABO: All expected trading days are present. [2025-10-20T17:47:59.216735] [DEBUG] [VALIDATOR ] CABO: Primary data validation passed. [2025-10-20T17:47:59.217147] [INFO ] [MAIN ] CABO: Fetched primary data is valid. [2025-10-20T17:47:59.217187] [DEBUG] [LOCAL ] CABO: Saving primary data. [2025-10-20T17:47:59.217818] [DEBUG] [LOCAL ] CABO: Trimmed 23148 records to 16723 (keeping 200 days of data) [2025-10-20T17:47:59.315241] [DEBUG] [LOCAL ] CABO: Primary data saved (16723 records). [2025-10-20T17:47:59.315376] [INFO ] [MAIN ] CABO: Full history saved. [2025-10-20T17:47:59.315458] [INFO ] [MAIN ] Update Progress: 382/2367 symbols completed. CABO took 22.4s. Estimated remaining: 4h 8m 59s. Rate: 8.0 symbols/min [2025-10-20T17:47:59.460279] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-02. [2025-10-20T17:47:59.493747] [INFO ] [MAIN ] CAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:47:59.499927] [DEBUG] [LOCAL ] CAF: Loaded 3545 existing records from primary storage. [2025-10-20T17:47:59.500136] [INFO ] [MAIN ] CAF: Loaded 3545 existing records. [2025-10-20T17:47:59.686512] [WARN ] [MAIN ] CAF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:47:59.686797] [INFO ] [MAIN ] CAF: Wiping existing primary data before full refresh. [2025-10-20T17:47:59.686869] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-10-20T17:47:59.687094] [DEBUG] [LOCAL ] CAF: Deleted primary data file. [2025-10-20T17:47:59.687162] [INFO ] [MAIN ] CAF: Starting full primary refresh. [2025-10-20T17:48:00.084752] [INFO ] [REMOTE ] CAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:00.187062] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:48:00.251446] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-02. [2025-10-20T17:48:00.777971] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T17:48:01.051381] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-04. [2025-10-20T17:48:01.457710] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T17:48:01.851381] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-01. [2025-10-20T17:48:02.555366] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T17:48:02.658084] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-03. [2025-10-20T17:48:02.685601] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T17:48:03.451431] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-03. [2025-10-20T17:48:03.895024] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T17:48:04.251474] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-04. [2025-10-20T17:48:05.051483] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-02. [2025-10-20T17:48:05.203343] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T17:48:05.320895] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T17:48:05.757050] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:48:05.851475] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-03. [2025-10-20T17:48:06.239410] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T17:48:06.651408] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-05. [2025-10-20T17:48:07.451444] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-04. [2025-10-20T17:48:07.998317] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T17:48:08.251435] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-04. [2025-10-20T17:48:08.883110] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T17:48:09.051435] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-05. [2025-10-20T17:48:09.198863] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T17:48:09.531166] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T17:48:09.851452] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-06. [2025-10-20T17:48:10.332835] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T17:48:10.651444] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-05. [2025-10-20T17:48:11.451454] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-06. [2025-10-20T17:48:12.251448] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-07. [2025-10-20T17:48:13.051422] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-05. [2025-10-20T17:48:13.921788] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-06 (2.47 seconds). [2025-10-20T17:48:13.926793] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-07. [2025-10-20T17:48:14.031178] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:48:14.132825] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-05 (3.48 seconds). [2025-10-20T17:48:14.651516] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-06. [2025-10-20T17:48:15.086919] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T17:48:15.456485] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-08. [2025-10-20T17:48:15.471818] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-05 (2.42 seconds). [2025-10-20T17:48:16.251419] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-08. [2025-10-20T17:48:16.407500] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T17:48:16.513314] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T17:48:16.741036] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T17:48:17.051389] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-07. [2025-10-20T17:48:17.851467] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-06. [2025-10-20T17:48:18.651347] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-09. [2025-10-20T17:48:19.006121] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T17:48:19.144865] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:48:19.451364] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-08. [2025-10-20T17:48:19.623749] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T17:48:20.251408] [INFO ] [REMOTE ] CACI: Fetching primary data for 2025-10. [2025-10-20T17:48:21.051358] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-07. [2025-10-20T17:48:21.117841] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T17:48:21.364550] [INFO ] [REMOTE ] CACI: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T17:48:21.367241] [INFO ] [REMOTE ] CACI: Full fetch complete (38294 records). [2025-10-20T17:48:21.367313] [DEBUG] [VALIDATOR ] CACI: Validating primary data. [2025-10-20T17:48:21.754375] [DEBUG] [VALIDATOR ] CACI: All expected trading days are present. [2025-10-20T17:48:21.754453] [DEBUG] [VALIDATOR ] CACI: Primary data validation passed. [2025-10-20T17:48:21.754965] [INFO ] [MAIN ] CACI: Fetched primary data is valid. [2025-10-20T17:48:21.755026] [DEBUG] [LOCAL ] CACI: Saving primary data. [2025-10-20T17:48:21.755918] [DEBUG] [LOCAL ] CACI: Trimmed 38294 records to 22704 (keeping 200 days of data) [2025-10-20T17:48:21.854175] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-09. [2025-10-20T17:48:21.898853] [DEBUG] [LOCAL ] CACI: Primary data saved (22704 records). [2025-10-20T17:48:21.898977] [INFO ] [MAIN ] CACI: Full history saved. [2025-10-20T17:48:21.899055] [INFO ] [MAIN ] Update Progress: 383/2367 symbols completed. CACI took 42.1s. Estimated remaining: 4h 10m 9s. Rate: 7.9 symbols/min [2025-10-20T17:48:22.081216] [INFO ] [MAIN ] CAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:22.081469] [DEBUG] [LOCAL ] CAG: No existing data file found in primary storage. [2025-10-20T17:48:22.081523] [WARN ] [MAIN ] CAG: No existing data found, performing full refresh instead. [2025-10-20T17:48:22.081699] [INFO ] [MAIN ] CAG: Wiping existing primary data before full refresh. [2025-10-20T17:48:22.081770] [INFO ] [LOCAL ] CAG: Starting deletion of primary data. [2025-10-20T17:48:22.081842] [DEBUG] [LOCAL ] CAG: No primary data file found to delete. [2025-10-20T17:48:22.081877] [INFO ] [MAIN ] CAG: Starting full primary refresh. [2025-10-20T17:48:22.435701] [INFO ] [REMOTE ] CAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:22.448633] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T17:48:22.540156] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T17:48:22.651353] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-01. [2025-10-20T17:48:23.451391] [INFO ] [REMOTE ] CAF: Fetching primary data for 2025-10. [2025-10-20T17:48:24.034378] [INFO ] [REMOTE ] CAF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T17:48:24.035492] [INFO ] [REMOTE ] CAF: Full fetch complete (5509 records). [2025-10-20T17:48:24.035559] [DEBUG] [VALIDATOR ] CAF: Validating primary data. [2025-10-20T17:48:24.035928] [FATAL] [VALIDATOR ] CAF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:48:24.036008] [FATAL] [MAIN ] CAF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:48:24.036049] [WARN ] [MAIN ] CAF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:48:24.036085] [INFO ] [LOCAL ] CAF: Starting deletion of primary data. [2025-10-20T17:48:24.036137] [DEBUG] [LOCAL ] CAF: No primary data file found to delete. [2025-10-20T17:48:24.036170] [INFO ] [LOCAL ] CAF: Starting deletion of derivative data. [2025-10-20T17:48:24.036347] [DEBUG] [LOCAL ] CAF: Deleted derivative data file. [2025-10-20T17:48:24.036430] [INFO ] [MAIN ] Update Progress: 384/2367 symbols completed. CAF took 24.7s. Estimated remaining: 4h 9m 34s. Rate: 7.9 symbols/min [2025-10-20T17:48:24.211888] [INFO ] [MAIN ] CAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:24.244852] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T17:48:24.255176] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-08. [2025-10-20T17:48:24.299853] [DEBUG] [LOCAL ] CAH: Loaded 52214 existing records from primary storage. [2025-10-20T17:48:24.300166] [INFO ] [MAIN ] CAH: Loaded 52214 existing records. [2025-10-20T17:48:24.478278] [WARN ] [MAIN ] CAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:24.478514] [INFO ] [MAIN ] CAH: Wiping existing primary data before full refresh. [2025-10-20T17:48:24.478561] [INFO ] [LOCAL ] CAH: Starting deletion of primary data. [2025-10-20T17:48:24.478792] [DEBUG] [LOCAL ] CAH: Deleted primary data file. [2025-10-20T17:48:24.478847] [INFO ] [MAIN ] CAH: Starting full primary refresh. [2025-10-20T17:48:24.846458] [INFO ] [REMOTE ] CAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:25.051346] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-02. [2025-10-20T17:48:25.778143] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:48:25.851336] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-09. [2025-10-20T17:48:26.651435] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-09. [2025-10-20T17:48:27.451353] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-01. [2025-10-20T17:48:27.563614] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-02 (2.51 seconds). [2025-10-20T17:48:27.908448] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T17:48:28.251389] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-03. [2025-10-20T17:48:29.051383] [INFO ] [REMOTE ] CAE: Fetching primary data for 2025-10. [2025-10-20T17:48:29.153173] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-09 (2.50 seconds). [2025-10-20T17:48:29.828668] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-01 (2.38 seconds). [2025-10-20T17:48:29.851354] [INFO ] [REMOTE ] CADE: Fetching primary data for 2025-10. [2025-10-20T17:48:29.964996] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T17:48:30.609045] [INFO ] [REMOTE ] CAE: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T17:48:30.615293] [INFO ] [REMOTE ] CAE: Full fetch complete (56551 records). [2025-10-20T17:48:30.615410] [DEBUG] [VALIDATOR ] CAE: Validating primary data. [2025-10-20T17:48:30.659612] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-02. [2025-10-20T17:48:31.015243] [DEBUG] [VALIDATOR ] CAE: All expected trading days are present. [2025-10-20T17:48:31.015350] [DEBUG] [VALIDATOR ] CAE: Primary data validation passed. [2025-10-20T17:48:31.016056] [INFO ] [MAIN ] CAE: Fetched primary data is valid. [2025-10-20T17:48:31.016120] [DEBUG] [LOCAL ] CAE: Saving primary data. [2025-10-20T17:48:31.017301] [DEBUG] [LOCAL ] CAE: Trimmed 56551 records to 39223 (keeping 200 days of data) [2025-10-20T17:48:31.242281] [DEBUG] [LOCAL ] CAE: Primary data saved (39223 records). [2025-10-20T17:48:31.242413] [INFO ] [MAIN ] CAE: Full history saved. [2025-10-20T17:48:31.242490] [INFO ] [MAIN ] Update Progress: 385/2367 symbols completed. CAE took 34.2s. Estimated remaining: 4h 9m 25s. Rate: 7.9 symbols/min [2025-10-20T17:48:31.416090] [INFO ] [MAIN ] CAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:31.437289] [INFO ] [REMOTE ] CADE: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T17:48:31.445387] [INFO ] [REMOTE ] CADE: Full fetch complete (70658 records). [2025-10-20T17:48:31.445594] [DEBUG] [VALIDATOR ] CADE: Validating primary data. [2025-10-20T17:48:31.457674] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-04. [2025-10-20T17:48:31.676691] [DEBUG] [LOCAL ] CAL: Loaded 43547 existing records from primary storage. [2025-10-20T17:48:31.737636] [INFO ] [MAIN ] CAL: Loaded 43547 existing records. [2025-10-20T17:48:32.071537] [DEBUG] [VALIDATOR ] CADE: All expected trading days are present. [2025-10-20T17:48:32.071313] [WARN ] [MAIN ] CAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:32.071728] [DEBUG] [VALIDATOR ] CADE: Primary data validation passed. [2025-10-20T17:48:32.071884] [INFO ] [MAIN ] CAL: Wiping existing primary data before full refresh. [2025-10-20T17:48:32.071974] [INFO ] [LOCAL ] CAL: Starting deletion of primary data. [2025-10-20T17:48:32.072862] [INFO ] [MAIN ] CADE: Fetched primary data is valid. [2025-10-20T17:48:32.072998] [DEBUG] [LOCAL ] CADE: Saving primary data. [2025-10-20T17:48:32.073203] [DEBUG] [LOCAL ] CAL: Deleted primary data file. [2025-10-20T17:48:32.073276] [INFO ] [MAIN ] CAL: Starting full primary refresh. [2025-10-20T17:48:32.156723] [DEBUG] [LOCAL ] CADE: Trimmed 70658 records to 49658 (keeping 200 days of data) [2025-10-20T17:48:32.635235] [INFO ] [REMOTE ] CAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:32.663159] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-01. [2025-10-20T17:48:32.708048] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T17:48:32.792773] [DEBUG] [LOCAL ] CADE: Primary data saved (49658 records). [2025-10-20T17:48:32.792920] [INFO ] [MAIN ] CADE: Full history saved. [2025-10-20T17:48:32.793000] [INFO ] [MAIN ] Update Progress: 386/2367 symbols completed. CADE took 37.3s. Estimated remaining: 4h 8m 46s. Rate: 8.0 symbols/min [2025-10-20T17:48:32.994964] [INFO ] [MAIN ] CALX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:33.028143] [DEBUG] [LOCAL ] CALX: Loaded 38084 existing records from primary storage. [2025-10-20T17:48:33.028459] [INFO ] [MAIN ] CALX: Loaded 38084 existing records. [2025-10-20T17:48:33.053404] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-03. [2025-10-20T17:48:33.223629] [WARN ] [MAIN ] CALX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:33.223984] [INFO ] [MAIN ] CALX: Wiping existing primary data before full refresh. [2025-10-20T17:48:33.224036] [INFO ] [LOCAL ] CALX: Starting deletion of primary data. [2025-10-20T17:48:33.224243] [DEBUG] [LOCAL ] CALX: Deleted primary data file. [2025-10-20T17:48:33.224287] [INFO ] [MAIN ] CALX: Starting full primary refresh. [2025-10-20T17:48:33.602602] [INFO ] [REMOTE ] CALX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:33.632281] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T17:48:33.851344] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-05. [2025-10-20T17:48:34.297963] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:48:34.651400] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-01. [2025-10-20T17:48:35.000817] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T17:48:35.451367] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-04. [2025-10-20T17:48:35.623237] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:48:35.699773] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T17:48:36.251427] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-06. [2025-10-20T17:48:37.051399] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-02. [2025-10-20T17:48:37.358838] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T17:48:37.851516] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-05. [2025-10-20T17:48:38.058779] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T17:48:38.155093] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T17:48:38.651440] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-07. [2025-10-20T17:48:39.369813] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T17:48:39.451351] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-06. [2025-10-20T17:48:40.251378] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-03. [2025-10-20T17:48:40.644696] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T17:48:41.051435] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-08. [2025-10-20T17:48:41.247604] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T17:48:41.818016] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T17:48:41.851363] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-04. [2025-10-20T17:48:42.362417] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:48:42.651384] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-07. [2025-10-20T17:48:43.451374] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-09. [2025-10-20T17:48:43.549412] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:48:44.251403] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-02. [2025-10-20T17:48:44.568886] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:48:44.985806] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T17:48:45.051376] [INFO ] [REMOTE ] CAG: Fetching primary data for 2025-10. [2025-10-20T17:48:45.771015] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T17:48:45.851387] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-03. [2025-10-20T17:48:46.651381] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-05. [2025-10-20T17:48:47.152827] [INFO ] [REMOTE ] CAG: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T17:48:47.161586] [INFO ] [REMOTE ] CAG: Full fetch complete (77826 records). [2025-10-20T17:48:47.161861] [DEBUG] [VALIDATOR ] CAG: Validating primary data. [2025-10-20T17:48:47.456837] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-08. [2025-10-20T17:48:47.614370] [DEBUG] [VALIDATOR ] CAG: All expected trading days are present. [2025-10-20T17:48:47.623985] [DEBUG] [VALIDATOR ] CAG: Primary data validation passed. [2025-10-20T17:48:47.642408] [INFO ] [MAIN ] CAG: Fetched primary data is valid. [2025-10-20T17:48:47.645090] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T17:48:47.645259] [DEBUG] [LOCAL ] CAG: Saving primary data. [2025-10-20T17:48:47.658439] [DEBUG] [LOCAL ] CAG: Trimmed 77826 records to 53665 (keeping 200 days of data) [2025-10-20T17:48:48.015778] [DEBUG] [LOCAL ] CAG: Primary data saved (53665 records). [2025-10-20T17:48:48.016007] [INFO ] [MAIN ] CAG: Full history saved. [2025-10-20T17:48:48.016117] [INFO ] [MAIN ] Update Progress: 387/2367 symbols completed. CAG took 26.1s. Estimated remaining: 4h 9m 18s. Rate: 7.9 symbols/min [2025-10-20T17:48:48.202482] [INFO ] [MAIN ] CANG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:48.230511] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T17:48:48.251337] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-06. [2025-10-20T17:48:48.264565] [DEBUG] [LOCAL ] CANG: Loaded 27428 existing records from primary storage. [2025-10-20T17:48:48.264964] [INFO ] [MAIN ] CANG: Loaded 27428 existing records. [2025-10-20T17:48:48.455721] [WARN ] [MAIN ] CANG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:48.456057] [INFO ] [MAIN ] CANG: Wiping existing primary data before full refresh. [2025-10-20T17:48:48.456108] [INFO ] [LOCAL ] CANG: Starting deletion of primary data. [2025-10-20T17:48:48.456391] [DEBUG] [LOCAL ] CANG: Deleted primary data file. [2025-10-20T17:48:48.456451] [INFO ] [MAIN ] CANG: Starting full primary refresh. [2025-10-20T17:48:48.845732] [INFO ] [REMOTE ] CANG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:49.036777] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:48:49.051337] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-04. [2025-10-20T17:48:49.590502] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T17:48:49.851379] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-07. [2025-10-20T17:48:50.651353] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-09. [2025-10-20T17:48:50.750692] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T17:48:51.296739] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T17:48:51.451434] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-08. [2025-10-20T17:48:52.251364] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-01. [2025-10-20T17:48:52.357779] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T17:48:52.881518] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:48:53.051424] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-09. [2025-10-20T17:48:53.397341] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T17:48:53.851384] [INFO ] [REMOTE ] CAH: Fetching primary data for 2025-10. [2025-10-20T17:48:54.034346] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:48:54.651419] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-05. [2025-10-20T17:48:55.460174] [INFO ] [REMOTE ] CALX: Fetching primary data for 2025-10. [2025-10-20T17:48:55.485833] [INFO ] [REMOTE ] CAH: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T17:48:55.493986] [INFO ] [REMOTE ] CAH: Full fetch complete (74843 records). [2025-10-20T17:48:55.494119] [DEBUG] [VALIDATOR ] CAH: Validating primary data. [2025-10-20T17:48:55.933553] [DEBUG] [VALIDATOR ] CAH: All expected trading days are present. [2025-10-20T17:48:55.933702] [DEBUG] [VALIDATOR ] CAH: Primary data validation passed. [2025-10-20T17:48:55.934906] [INFO ] [MAIN ] CAH: Fetched primary data is valid. [2025-10-20T17:48:55.934977] [DEBUG] [LOCAL ] CAH: Saving primary data. [2025-10-20T17:48:55.937120] [DEBUG] [LOCAL ] CAH: Trimmed 74843 records to 51463 (keeping 200 days of data) [2025-10-20T17:48:56.256472] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-02. [2025-10-20T17:48:56.303372] [DEBUG] [LOCAL ] CAH: Primary data saved (51463 records). [2025-10-20T17:48:56.303625] [INFO ] [MAIN ] CAH: Full history saved. [2025-10-20T17:48:56.304125] [INFO ] [MAIN ] Update Progress: 388/2367 symbols completed. CAH took 32.3s. Estimated remaining: 4h 9m 14s. Rate: 7.9 symbols/min [2025-10-20T17:48:56.510873] [INFO ] [MAIN ] CAPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:56.530158] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T17:48:56.546276] [DEBUG] [LOCAL ] CAPL: Loaded 5696 existing records from primary storage. [2025-10-20T17:48:56.546581] [INFO ] [MAIN ] CAPL: Loaded 5696 existing records. [2025-10-20T17:48:56.740402] [WARN ] [MAIN ] CAPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:56.740720] [INFO ] [MAIN ] CAPL: Wiping existing primary data before full refresh. [2025-10-20T17:48:56.740791] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-10-20T17:48:56.741023] [DEBUG] [LOCAL ] CAPL: Deleted primary data file. [2025-10-20T17:48:56.741087] [INFO ] [MAIN ] CAPL: Starting full primary refresh. [2025-10-20T17:48:57.053363] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-06. [2025-10-20T17:48:57.100016] [INFO ] [REMOTE ] CALX: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T17:48:57.167097] [INFO ] [REMOTE ] CALX: Full fetch complete (54173 records). [2025-10-20T17:48:57.169896] [DEBUG] [VALIDATOR ] CALX: Validating primary data. [2025-10-20T17:48:57.194151] [INFO ] [REMOTE ] CAPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:57.255239] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T17:48:57.653303] [DEBUG] [VALIDATOR ] CALX: All expected trading days are present. [2025-10-20T17:48:57.653409] [DEBUG] [VALIDATOR ] CALX: Primary data validation passed. [2025-10-20T17:48:57.654220] [INFO ] [MAIN ] CALX: Fetched primary data is valid. [2025-10-20T17:48:57.654310] [DEBUG] [LOCAL ] CALX: Saving primary data. [2025-10-20T17:48:57.655715] [DEBUG] [LOCAL ] CALX: Trimmed 54173 records to 37496 (keeping 200 days of data) [2025-10-20T17:48:57.880119] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-01. [2025-10-20T17:48:57.914350] [DEBUG] [LOCAL ] CALX: Primary data saved (37496 records). [2025-10-20T17:48:57.914611] [INFO ] [MAIN ] CALX: Full history saved. [2025-10-20T17:48:57.914758] [INFO ] [MAIN ] Update Progress: 389/2367 symbols completed. CALX took 25.1s. Estimated remaining: 4h 8m 36s. Rate: 8.0 symbols/min [2025-10-20T17:48:58.136773] [INFO ] [MAIN ] CARR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:48:58.198274] [DEBUG] [LOCAL ] CARR: Loaded 54382 existing records from primary storage. [2025-10-20T17:48:58.198739] [INFO ] [MAIN ] CARR: Loaded 54382 existing records. [2025-10-20T17:48:58.496381] [WARN ] [MAIN ] CARR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:48:58.498016] [INFO ] [MAIN ] CARR: Wiping existing primary data before full refresh. [2025-10-20T17:48:58.498407] [INFO ] [LOCAL ] CARR: Starting deletion of primary data. [2025-10-20T17:48:58.499009] [DEBUG] [LOCAL ] CARR: Deleted primary data file. [2025-10-20T17:48:58.499086] [INFO ] [MAIN ] CARR: Starting full primary refresh. [2025-10-20T17:48:58.660983] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T17:48:58.659385] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-03. [2025-10-20T17:48:58.785603] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T17:48:58.990249] [INFO ] [REMOTE ] CARR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:48:59.451433] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-02. [2025-10-20T17:48:59.789517] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T17:48:59.815133] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T17:49:00.251425] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-01. [2025-10-20T17:49:01.051431] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-03. [2025-10-20T17:49:01.851416] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-07. [2025-10-20T17:49:01.934885] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:49:02.272678] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T17:49:02.651429] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-04. [2025-10-20T17:49:03.297857] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:49:03.326277] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:49:03.451389] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-02. [2025-10-20T17:49:04.251384] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-05. [2025-10-20T17:49:04.748585] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-10-20T17:49:04.915224] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:49:05.051418] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-03. [2025-10-20T17:49:05.851418] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-08. [2025-10-20T17:49:06.569479] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:49:06.651420] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-04. [2025-10-20T17:49:07.069336] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T17:49:07.451462] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-04. [2025-10-20T17:49:08.251463] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-06. [2025-10-20T17:49:08.508337] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T17:49:08.666307] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:49:08.715704] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:49:09.051397] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-05. [2025-10-20T17:49:09.851451] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-05. [2025-10-20T17:49:10.651384] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-07. [2025-10-20T17:49:11.451448] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-09. [2025-10-20T17:49:11.658430] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:49:12.176711] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:49:12.251465] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-08. [2025-10-20T17:49:12.503888] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-05 (3.45 seconds). [2025-10-20T17:49:12.937258] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:49:13.051410] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-09. [2025-10-20T17:49:13.299297] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T17:49:13.553449] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T17:49:13.851427] [INFO ] [REMOTE ] CAL: Fetching primary data for 2025-10. [2025-10-20T17:49:14.651393] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-06. [2025-10-20T17:49:15.063980] [INFO ] [REMOTE ] CAL: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T17:49:15.070861] [INFO ] [REMOTE ] CAL: Full fetch complete (61837 records). [2025-10-20T17:49:15.071241] [DEBUG] [VALIDATOR ] CAL: Validating primary data. [2025-10-20T17:49:15.456731] [INFO ] [REMOTE ] CAPL: Fetching primary data for 2025-10. [2025-10-20T17:49:15.504895] [DEBUG] [VALIDATOR ] CAL: All expected trading days are present. [2025-10-20T17:49:15.505004] [DEBUG] [VALIDATOR ] CAL: Primary data validation passed. [2025-10-20T17:49:15.505945] [INFO ] [MAIN ] CAL: Fetched primary data is valid. [2025-10-20T17:49:15.506013] [DEBUG] [LOCAL ] CAL: Saving primary data. [2025-10-20T17:49:15.507741] [DEBUG] [LOCAL ] CAL: Trimmed 61837 records to 42891 (keeping 200 days of data) [2025-10-20T17:49:15.788319] [DEBUG] [LOCAL ] CAL: Primary data saved (42891 records). [2025-10-20T17:49:15.788503] [INFO ] [MAIN ] CAL: Full history saved. [2025-10-20T17:49:15.788602] [INFO ] [MAIN ] Update Progress: 390/2367 symbols completed. CAL took 44.5s. Estimated remaining: 4h 9m 21s. Rate: 7.9 symbols/min [2025-10-20T17:49:15.984050] [INFO ] [MAIN ] CARS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:16.031994] [DEBUG] [LOCAL ] CARS: Loaded 40383 existing records from primary storage. [2025-10-20T17:49:16.032387] [INFO ] [MAIN ] CARS: Loaded 40383 existing records. [2025-10-20T17:49:16.233713] [WARN ] [MAIN ] CARS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:16.234034] [INFO ] [MAIN ] CARS: Wiping existing primary data before full refresh. [2025-10-20T17:49:16.234223] [INFO ] [LOCAL ] CARS: Starting deletion of primary data. [2025-10-20T17:49:16.234567] [DEBUG] [LOCAL ] CARS: Deleted primary data file. [2025-10-20T17:49:16.234626] [INFO ] [MAIN ] CARS: Starting full primary refresh. [2025-10-20T17:49:16.266321] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-06. [2025-10-20T17:49:16.559571] [INFO ] [REMOTE ] CAPL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:49:16.621091] [INFO ] [REMOTE ] CARS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:16.623046] [INFO ] [REMOTE ] CAPL: Full fetch complete (8692 records). [2025-10-20T17:49:16.623539] [DEBUG] [VALIDATOR ] CAPL: Validating primary data. [2025-10-20T17:49:16.624039] [FATAL] [VALIDATOR ] CAPL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:49:16.624186] [FATAL] [MAIN ] CAPL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:49:16.624522] [WARN ] [MAIN ] CAPL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:49:16.624628] [INFO ] [LOCAL ] CAPL: Starting deletion of primary data. [2025-10-20T17:49:16.624957] [DEBUG] [LOCAL ] CAPL: No primary data file found to delete. [2025-10-20T17:49:16.625031] [INFO ] [LOCAL ] CAPL: Starting deletion of derivative data. [2025-10-20T17:49:16.625280] [DEBUG] [LOCAL ] CAPL: Deleted derivative data file. [2025-10-20T17:49:16.625370] [INFO ] [MAIN ] Update Progress: 391/2367 symbols completed. CAPL took 20.3s. Estimated remaining: 4h 8m 40s. Rate: 7.9 symbols/min [2025-10-20T17:49:16.843705] [INFO ] [MAIN ] CAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:16.846607] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-06 (2.20 seconds). [2025-10-20T17:49:16.906124] [DEBUG] [LOCAL ] CAT: Loaded 53736 existing records from primary storage. [2025-10-20T17:49:16.906448] [INFO ] [MAIN ] CAT: Loaded 53736 existing records. [2025-10-20T17:49:17.056754] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-01. [2025-10-20T17:49:17.085826] [WARN ] [MAIN ] CAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:17.086156] [INFO ] [MAIN ] CAT: Wiping existing primary data before full refresh. [2025-10-20T17:49:17.086207] [INFO ] [LOCAL ] CAT: Starting deletion of primary data. [2025-10-20T17:49:17.086743] [DEBUG] [LOCAL ] CAT: Deleted primary data file. [2025-10-20T17:49:17.086798] [INFO ] [MAIN ] CAT: Starting full primary refresh. [2025-10-20T17:49:17.448225] [INFO ] [REMOTE ] CAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:17.467516] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T17:49:17.851445] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-01. [2025-10-20T17:49:18.479968] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:49:18.651441] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-07. [2025-10-20T17:49:19.451359] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-07. [2025-10-20T17:49:19.835964] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T17:49:20.251471] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-02. [2025-10-20T17:49:20.637931] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T17:49:20.744409] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T17:49:21.051382] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-08. [2025-10-20T17:49:21.580102] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T17:49:21.851445] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-02. [2025-10-20T17:49:22.651366] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-03. [2025-10-20T17:49:22.970807] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T17:49:23.451397] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-09. [2025-10-20T17:49:23.864750] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T17:49:24.197260] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:49:24.251433] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-04. [2025-10-20T17:49:25.028903] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:49:25.051354] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-03. [2025-10-20T17:49:25.764116] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T17:49:25.851389] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-08. [2025-10-20T17:49:26.651420] [INFO ] [REMOTE ] CARR: Fetching primary data for 2025-10. [2025-10-20T17:49:26.853386] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T17:49:26.930556] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T17:49:27.451375] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-04. [2025-10-20T17:49:28.251436] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-09. [2025-10-20T17:49:28.588992] [INFO ] [REMOTE ] CARR: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T17:49:28.597261] [INFO ] [REMOTE ] CARR: Full fetch complete (77772 records). [2025-10-20T17:49:28.597425] [DEBUG] [VALIDATOR ] CARR: Validating primary data. [2025-10-20T17:49:29.009212] [DEBUG] [VALIDATOR ] CARR: All expected trading days are present. [2025-10-20T17:49:29.009327] [DEBUG] [VALIDATOR ] CARR: Primary data validation passed. [2025-10-20T17:49:29.010543] [INFO ] [MAIN ] CARR: Fetched primary data is valid. [2025-10-20T17:49:29.010615] [DEBUG] [LOCAL ] CARR: Saving primary data. [2025-10-20T17:49:29.012793] [DEBUG] [LOCAL ] CARR: Trimmed 77772 records to 53603 (keeping 200 days of data) [2025-10-20T17:49:29.069141] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-05. [2025-10-20T17:49:29.374458] [DEBUG] [LOCAL ] CARR: Primary data saved (53603 records). [2025-10-20T17:49:29.374740] [INFO ] [MAIN ] CARR: Full history saved. [2025-10-20T17:49:29.374907] [INFO ] [MAIN ] Update Progress: 392/2367 symbols completed. CARR took 31.5s. Estimated remaining: 4h 8m 58s. Rate: 7.9 symbols/min [2025-10-20T17:49:29.578682] [INFO ] [MAIN ] CATO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:29.602490] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T17:49:29.617985] [DEBUG] [LOCAL ] CATO: Loaded 6799 existing records from primary storage. [2025-10-20T17:49:29.619097] [INFO ] [MAIN ] CATO: Loaded 6799 existing records. [2025-10-20T17:49:29.814051] [WARN ] [MAIN ] CATO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:29.814453] [INFO ] [MAIN ] CATO: Wiping existing primary data before full refresh. [2025-10-20T17:49:29.814534] [INFO ] [LOCAL ] CATO: Starting deletion of primary data. [2025-10-20T17:49:29.815011] [DEBUG] [LOCAL ] CATO: Deleted primary data file. [2025-10-20T17:49:29.842083] [INFO ] [MAIN ] CATO: Starting full primary refresh. [2025-10-20T17:49:29.858225] [INFO ] [REMOTE ] CANG: Fetching primary data for 2025-10. [2025-10-20T17:49:29.884249] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-04 (2.43 seconds). [2025-10-20T17:49:30.225040] [INFO ] [REMOTE ] CATO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:30.651368] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-05. [2025-10-20T17:49:30.746938] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:49:30.942495] [INFO ] [REMOTE ] CANG: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:49:30.947664] [INFO ] [REMOTE ] CANG: Full fetch complete (39819 records). [2025-10-20T17:49:30.947774] [DEBUG] [VALIDATOR ] CANG: Validating primary data. [2025-10-20T17:49:31.346946] [DEBUG] [VALIDATOR ] CANG: All expected trading days are present. [2025-10-20T17:49:31.347064] [DEBUG] [VALIDATOR ] CANG: Primary data validation passed. [2025-10-20T17:49:31.347618] [INFO ] [MAIN ] CANG: Fetched primary data is valid. [2025-10-20T17:49:31.347692] [DEBUG] [LOCAL ] CANG: Saving primary data. [2025-10-20T17:49:31.348529] [DEBUG] [LOCAL ] CANG: Trimmed 39819 records to 27136 (keeping 200 days of data) [2025-10-20T17:49:31.453123] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-06. [2025-10-20T17:49:31.491583] [DEBUG] [LOCAL ] CANG: Primary data saved (27136 records). [2025-10-20T17:49:31.491702] [INFO ] [MAIN ] CANG: Full history saved. [2025-10-20T17:49:31.491786] [INFO ] [MAIN ] Update Progress: 393/2367 symbols completed. CANG took 43.5s. Estimated remaining: 4h 8m 23s. Rate: 7.9 symbols/min [2025-10-20T17:49:31.657097] [INFO ] [MAIN ] CAVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:31.657240] [DEBUG] [LOCAL ] CAVA: No existing data file found in primary storage. [2025-10-20T17:49:31.657276] [WARN ] [MAIN ] CAVA: No existing data found, performing full refresh instead. [2025-10-20T17:49:31.657383] [INFO ] [MAIN ] CAVA: Wiping existing primary data before full refresh. [2025-10-20T17:49:31.657423] [INFO ] [LOCAL ] CAVA: Starting deletion of primary data. [2025-10-20T17:49:31.657462] [DEBUG] [LOCAL ] CAVA: No primary data file found to delete. [2025-10-20T17:49:31.657492] [INFO ] [MAIN ] CAVA: Starting full primary refresh. [2025-10-20T17:49:32.003973] [INFO ] [REMOTE ] CAVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:32.251351] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-01. [2025-10-20T17:49:32.942122] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:49:32.972031] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-10-20T17:49:33.051376] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-06. [2025-10-20T17:49:33.851390] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-01. [2025-10-20T17:49:34.299190] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T17:49:34.371589] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T17:49:34.651367] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-07. [2025-10-20T17:49:35.058198] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T17:49:35.451387] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-02. [2025-10-20T17:49:36.059537] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T17:49:36.251411] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-07. [2025-10-20T17:49:37.059549] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-08. [2025-10-20T17:49:37.094479] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T17:49:37.851418] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-03. [2025-10-20T17:49:38.439151] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T17:49:38.568792] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:49:38.651405] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-09. [2025-10-20T17:49:39.451378] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-02. [2025-10-20T17:49:39.904689] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T17:49:39.950808] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T17:49:40.159378] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T17:49:40.251375] [INFO ] [REMOTE ] CARS: Fetching primary data for 2025-10. [2025-10-20T17:49:41.051388] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-03. [2025-10-20T17:49:41.720035] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:49:41.851406] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-04. [2025-10-20T17:49:41.918380] [INFO ] [REMOTE ] CARS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T17:49:41.928231] [INFO ] [REMOTE ] CARS: Full fetch complete (56359 records). [2025-10-20T17:49:41.928464] [DEBUG] [VALIDATOR ] CARS: Validating primary data. [2025-10-20T17:49:42.315050] [DEBUG] [VALIDATOR ] CARS: All expected trading days are present. [2025-10-20T17:49:42.315141] [DEBUG] [VALIDATOR ] CARS: Primary data validation passed. [2025-10-20T17:49:42.315970] [INFO ] [MAIN ] CARS: Fetched primary data is valid. [2025-10-20T17:49:42.316042] [DEBUG] [LOCAL ] CARS: Saving primary data. [2025-10-20T17:49:42.317324] [DEBUG] [LOCAL ] CARS: Trimmed 56359 records to 39802 (keeping 200 days of data) [2025-10-20T17:49:42.530478] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T17:49:42.559690] [DEBUG] [LOCAL ] CARS: Primary data saved (39802 records). [2025-10-20T17:49:42.559867] [INFO ] [MAIN ] CARS: Full history saved. [2025-10-20T17:49:42.559956] [INFO ] [MAIN ] Update Progress: 394/2367 symbols completed. CARS took 26.8s. Estimated remaining: 4h 8m 33s. Rate: 7.9 symbols/min [2025-10-20T17:49:42.656492] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-04. [2025-10-20T17:49:42.730476] [INFO ] [MAIN ] CB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:42.774717] [DEBUG] [LOCAL ] CB: Loaded 50716 existing records from primary storage. [2025-10-20T17:49:42.775055] [INFO ] [MAIN ] CB: Loaded 50716 existing records. [2025-10-20T17:49:42.946589] [WARN ] [MAIN ] CB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:42.946844] [INFO ] [MAIN ] CB: Wiping existing primary data before full refresh. [2025-10-20T17:49:42.946891] [INFO ] [LOCAL ] CB: Starting deletion of primary data. [2025-10-20T17:49:42.947189] [DEBUG] [LOCAL ] CB: Deleted primary data file. [2025-10-20T17:49:42.947248] [INFO ] [MAIN ] CB: Starting full primary refresh. [2025-10-20T17:49:43.293843] [INFO ] [REMOTE ] CB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:43.451355] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-01. [2025-10-20T17:49:44.251377] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-08. [2025-10-20T17:49:44.887217] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T17:49:44.994604] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T17:49:45.051334] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-02. [2025-10-20T17:49:45.851391] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-05. [2025-10-20T17:49:46.426388] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T17:49:46.513816] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:49:46.651341] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-03. [2025-10-20T17:49:47.451421] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-09. [2025-10-20T17:49:47.748839] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T17:49:48.251415] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-05. [2025-10-20T17:49:48.474927] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T17:49:48.877761] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T17:49:49.051355] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-06. [2025-10-20T17:49:49.193218] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:49:49.851408] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-06. [2025-10-20T17:49:50.455443] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T17:49:50.651450] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-07. [2025-10-20T17:49:50.912412] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:49:51.196012] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T17:49:51.451437] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-08. [2025-10-20T17:49:52.225219] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T17:49:52.251353] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-09. [2025-10-20T17:49:53.051369] [INFO ] [REMOTE ] CAT: Fetching primary data for 2025-10. [2025-10-20T17:49:53.124934] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T17:49:53.851435] [INFO ] [REMOTE ] CATO: Fetching primary data for 2025-10. [2025-10-20T17:49:54.338290] [INFO ] [REMOTE ] CATO: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T17:49:54.340046] [INFO ] [REMOTE ] CATO: Full fetch complete (10733 records). [2025-10-20T17:49:54.340127] [DEBUG] [VALIDATOR ] CATO: Validating primary data. [2025-10-20T17:49:54.656483] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-04. [2025-10-20T17:49:54.773490] [DEBUG] [VALIDATOR ] CATO: All expected trading days are present. [2025-10-20T17:49:54.773599] [DEBUG] [VALIDATOR ] CATO: Primary data validation passed. [2025-10-20T17:49:54.774119] [INFO ] [MAIN ] CATO: Fetched primary data is valid. [2025-10-20T17:49:54.774191] [DEBUG] [LOCAL ] CATO: Saving primary data. [2025-10-20T17:49:54.774874] [DEBUG] [LOCAL ] CATO: Trimmed 10733 records to 6648 (keeping 200 days of data) [2025-10-20T17:49:54.823947] [DEBUG] [LOCAL ] CATO: Primary data saved (6648 records). [2025-10-20T17:49:54.824096] [INFO ] [MAIN ] CATO: Full history saved. [2025-10-20T17:49:54.824186] [INFO ] [MAIN ] Update Progress: 395/2367 symbols completed. CATO took 25.4s. Estimated remaining: 4h 8m 49s. Rate: 7.9 symbols/min [2025-10-20T17:49:55.046993] [INFO ] [MAIN ] CBAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:55.060424] [DEBUG] [LOCAL ] CBAN: Loaded 10619 existing records from primary storage. [2025-10-20T17:49:55.060716] [INFO ] [MAIN ] CBAN: Loaded 10619 existing records. [2025-10-20T17:49:55.298489] [WARN ] [MAIN ] CBAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:55.298808] [INFO ] [MAIN ] CBAN: Wiping existing primary data before full refresh. [2025-10-20T17:49:55.298930] [INFO ] [LOCAL ] CBAN: Starting deletion of primary data. [2025-10-20T17:49:55.299321] [DEBUG] [LOCAL ] CBAN: Deleted primary data file. [2025-10-20T17:49:55.299437] [INFO ] [MAIN ] CBAN: Starting full primary refresh. [2025-10-20T17:49:55.476561] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-07. [2025-10-20T17:49:55.488003] [INFO ] [REMOTE ] CAT: Incremental fetch complete for 2025-10 (2.44 seconds). [2025-10-20T17:49:55.655794] [INFO ] [REMOTE ] CAT: Full fetch complete (76899 records). [2025-10-20T17:49:55.687919] [DEBUG] [VALIDATOR ] CAT: Validating primary data. [2025-10-20T17:49:55.922687] [INFO ] [REMOTE ] CBAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:56.251967] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-01. [2025-10-20T17:49:56.257400] [DEBUG] [VALIDATOR ] CAT: All expected trading days are present. [2025-10-20T17:49:56.257719] [DEBUG] [VALIDATOR ] CAT: Primary data validation passed. [2025-10-20T17:49:56.259162] [INFO ] [MAIN ] CAT: Fetched primary data is valid. [2025-10-20T17:49:56.259574] [DEBUG] [LOCAL ] CAT: Saving primary data. [2025-10-20T17:49:56.261811] [DEBUG] [LOCAL ] CAT: Trimmed 76899 records to 52966 (keeping 200 days of data) [2025-10-20T17:49:56.445791] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:49:56.677441] [DEBUG] [LOCAL ] CAT: Primary data saved (52966 records). [2025-10-20T17:49:56.677663] [INFO ] [MAIN ] CAT: Full history saved. [2025-10-20T17:49:56.678190] [INFO ] [MAIN ] Update Progress: 396/2367 symbols completed. CAT took 40.1s. Estimated remaining: 4h 8m 13s. Rate: 7.9 symbols/min [2025-10-20T17:49:56.866431] [INFO ] [MAIN ] CBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:49:56.885985] [DEBUG] [LOCAL ] CBL: Loaded 16467 existing records from primary storage. [2025-10-20T17:49:56.886324] [INFO ] [MAIN ] CBL: Loaded 16467 existing records. [2025-10-20T17:49:57.054421] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-05. [2025-10-20T17:49:57.089535] [WARN ] [MAIN ] CBL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:49:57.090837] [INFO ] [MAIN ] CBL: Wiping existing primary data before full refresh. [2025-10-20T17:49:57.090892] [INFO ] [LOCAL ] CBL: Starting deletion of primary data. [2025-10-20T17:49:57.091148] [DEBUG] [LOCAL ] CBL: Deleted primary data file. [2025-10-20T17:49:57.091207] [INFO ] [MAIN ] CBL: Starting full primary refresh. [2025-10-20T17:49:57.214412] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T17:49:57.488554] [INFO ] [REMOTE ] CBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:49:57.851381] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-02. [2025-10-20T17:49:57.903622] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-10-20T17:49:58.651391] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-01. [2025-10-20T17:49:58.835979] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T17:49:59.003159] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T17:49:59.451449] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-06. [2025-10-20T17:49:59.481158] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T17:50:00.251467] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-03. [2025-10-20T17:50:01.009841] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T17:50:01.051480] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-08. [2025-10-20T17:50:01.465715] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T17:50:01.851401] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-07. [2025-10-20T17:50:02.651725] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-02. [2025-10-20T17:50:03.257383] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T17:50:03.451491] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-09. [2025-10-20T17:50:03.726768] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:50:03.935529] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T17:50:04.251434] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-03. [2025-10-20T17:50:05.051362] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-08. [2025-10-20T17:50:05.326112] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T17:50:05.327060] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T17:50:05.851390] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-04. [2025-10-20T17:50:06.580612] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:50:06.606029] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:50:06.651390] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-04. [2025-10-20T17:50:07.451461] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-05. [2025-10-20T17:50:07.865488] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:50:08.251433] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-09. [2025-10-20T17:50:08.355921] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T17:50:09.051392] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-05. [2025-10-20T17:50:09.851404] [INFO ] [REMOTE ] CAVA: Fetching primary data for 2025-10. [2025-10-20T17:50:10.022826] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:50:10.052325] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T17:50:10.651467] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-06. [2025-10-20T17:50:11.359809] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:50:11.451370] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-07. [2025-10-20T17:50:12.251388] [INFO ] [REMOTE ] CB: Fetching primary data for 2025-10. [2025-10-20T17:50:12.477260] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T17:50:13.051444] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-08. [2025-10-20T17:50:13.851497] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-06. [2025-10-20T17:50:14.001768] [INFO ] [REMOTE ] CB: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T17:50:14.009129] [INFO ] [REMOTE ] CB: Full fetch complete (72841 records). [2025-10-20T17:50:14.009239] [DEBUG] [VALIDATOR ] CB: Validating primary data. [2025-10-20T17:50:14.159478] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T17:50:14.522807] [DEBUG] [VALIDATOR ] CB: All expected trading days are present. [2025-10-20T17:50:14.523161] [DEBUG] [VALIDATOR ] CB: Primary data validation passed. [2025-10-20T17:50:14.524803] [INFO ] [MAIN ] CB: Fetched primary data is valid. [2025-10-20T17:50:14.532490] [DEBUG] [LOCAL ] CB: Saving primary data. [2025-10-20T17:50:14.541179] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T17:50:14.558895] [INFO ] [REMOTE ] CAVA: Incremental fetch complete for 2025-10 (4.71 seconds). [2025-10-20T17:50:14.567299] [DEBUG] [LOCAL ] CB: Trimmed 72841 records to 49967 (keeping 200 days of data) [2025-10-20T17:50:14.567612] [INFO ] [REMOTE ] CAVA: Full fetch complete (77453 records). [2025-10-20T17:50:14.568234] [DEBUG] [VALIDATOR ] CAVA: Validating primary data. [2025-10-20T17:50:14.659893] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-07. [2025-10-20T17:50:15.313850] [DEBUG] [VALIDATOR ] CAVA: All expected trading days are present. [2025-10-20T17:50:15.326972] [DEBUG] [VALIDATOR ] CAVA: Primary data validation passed. [2025-10-20T17:50:15.338691] [INFO ] [MAIN ] CAVA: Fetched primary data is valid. [2025-10-20T17:50:15.338994] [DEBUG] [LOCAL ] CAVA: Saving primary data. [2025-10-20T17:50:15.338859] [DEBUG] [LOCAL ] CB: Primary data saved (49967 records). [2025-10-20T17:50:15.339469] [INFO ] [MAIN ] CB: Full history saved. [2025-10-20T17:50:15.339620] [INFO ] [MAIN ] Update Progress: 397/2367 symbols completed. CB took 32.8s. Estimated remaining: 4h 9m 1s. Rate: 7.9 symbols/min [2025-10-20T17:50:15.341165] [DEBUG] [LOCAL ] CAVA: Trimmed 77453 records to 53552 (keeping 200 days of data) [2025-10-20T17:50:15.453685] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-09. [2025-10-20T17:50:15.662013] [DEBUG] [LOCAL ] CAVA: Primary data saved (53552 records). [2025-10-20T17:50:15.662185] [INFO ] [MAIN ] CAVA: Full history saved. [2025-10-20T17:50:15.662387] [INFO ] [MAIN ] Update Progress: 398/2367 symbols completed. CAVA took 44.2s. Estimated remaining: 4h 8m 17s. Rate: 7.9 symbols/min [2025-10-20T17:50:15.827397] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T17:50:16.251406] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-08. [2025-10-20T17:50:16.368365] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T17:50:17.051416] [INFO ] [REMOTE ] CBL: Fetching primary data for 2025-10. [2025-10-20T17:50:17.123549] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T17:50:17.851363] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-09. [2025-10-20T17:50:18.155341] [INFO ] [REMOTE ] CBL: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T17:50:18.158073] [INFO ] [REMOTE ] CBL: Full fetch complete (24903 records). [2025-10-20T17:50:18.158146] [DEBUG] [VALIDATOR ] CBL: Validating primary data. [2025-10-20T17:50:18.526482] [DEBUG] [VALIDATOR ] CBL: All expected trading days are present. [2025-10-20T17:50:18.526622] [DEBUG] [VALIDATOR ] CBL: Primary data validation passed. [2025-10-20T17:50:18.527067] [INFO ] [MAIN ] CBL: Fetched primary data is valid. [2025-10-20T17:50:18.527114] [DEBUG] [LOCAL ] CBL: Saving primary data. [2025-10-20T17:50:18.527842] [DEBUG] [LOCAL ] CBL: Trimmed 24903 records to 16230 (keeping 200 days of data) [2025-10-20T17:50:18.613755] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T17:50:18.642024] [DEBUG] [LOCAL ] CBL: Primary data saved (16230 records). [2025-10-20T17:50:18.642164] [INFO ] [MAIN ] CBL: Full history saved. [2025-10-20T17:50:18.642242] [INFO ] [MAIN ] Update Progress: 399/2367 symbols completed. CBL took 22.0s. Estimated remaining: 4h 7m 47s. Rate: 7.9 symbols/min [2025-10-20T17:50:18.651333] [INFO ] [REMOTE ] CBAN: Fetching primary data for 2025-10. [2025-10-20T17:50:20.195826] [INFO ] [REMOTE ] CBAN: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T17:50:20.197852] [INFO ] [REMOTE ] CBAN: Full fetch complete (16230 records). [2025-10-20T17:50:20.197938] [DEBUG] [VALIDATOR ] CBAN: Validating primary data. [2025-10-20T17:50:20.575264] [DEBUG] [VALIDATOR ] CBAN: All expected trading days are present. [2025-10-20T17:50:20.575350] [DEBUG] [VALIDATOR ] CBAN: Primary data validation passed. [2025-10-20T17:50:20.575761] [INFO ] [MAIN ] CBAN: Fetched primary data is valid. [2025-10-20T17:50:20.575826] [DEBUG] [LOCAL ] CBAN: Saving primary data. [2025-10-20T17:50:20.576372] [DEBUG] [LOCAL ] CBAN: Trimmed 16230 records to 10299 (keeping 200 days of data) [2025-10-20T17:50:20.637598] [DEBUG] [LOCAL ] CBAN: Primary data saved (10299 records). [2025-10-20T17:50:20.637769] [INFO ] [MAIN ] CBAN: Full history saved. [2025-10-20T17:50:20.637853] [INFO ] [MAIN ] Update Progress: 400/2367 symbols completed. CBAN took 25.8s. Estimated remaining: 4h 7m 12s. Rate: 8.0 symbols/min [2025-10-20T17:50:21.179793] [INFO ] [MAIN ] CBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.315733] [INFO ] [MAIN ] CBNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.393014] [INFO ] [MAIN ] CBRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.389393] [INFO ] [MAIN ] CBO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:21.394078] [DEBUG] [LOCAL ] CBO: No existing data file found in primary storage. [2025-10-20T17:50:21.396667] [DEBUG] [LOCAL ] CBNA: Loaded 2942 existing records from primary storage. [2025-10-20T17:50:21.397281] [INFO ] [MAIN ] CBNA: Loaded 2942 existing records. [2025-10-20T17:50:21.396891] [WARN ] [MAIN ] CBO: No existing data found, performing full refresh instead. [2025-10-20T17:50:21.408709] [INFO ] [MAIN ] CBO: Wiping existing primary data before full refresh. [2025-10-20T17:50:21.446203] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-10-20T17:50:21.501898] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-10-20T17:50:21.520783] [INFO ] [MAIN ] CBO: Starting full primary refresh. [2025-10-20T17:50:21.559017] [DEBUG] [LOCAL ] CBT: Loaded 31588 existing records from primary storage. [2025-10-20T17:50:21.578550] [INFO ] [MAIN ] CBT: Loaded 31588 existing records. [2025-10-20T17:50:21.791406] [WARN ] [MAIN ] CBNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:21.860358] [DEBUG] [LOCAL ] CBRE: Loaded 51089 existing records from primary storage. [2025-10-20T17:50:21.898404] [INFO ] [MAIN ] CBNA: Wiping existing primary data before full refresh. [2025-10-20T17:50:21.932457] [INFO ] [MAIN ] CBRE: Loaded 51089 existing records. [2025-10-20T17:50:21.924599] [INFO ] [LOCAL ] CBNA: Starting deletion of primary data. [2025-10-20T17:50:22.024982] [DEBUG] [LOCAL ] CBNA: Deleted primary data file. [2025-10-20T17:50:22.144124] [INFO ] [MAIN ] CBNA: Starting full primary refresh. [2025-10-20T17:50:22.229377] [WARN ] [MAIN ] CBT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:22.295139] [INFO ] [MAIN ] CBT: Wiping existing primary data before full refresh. [2025-10-20T17:50:22.342299] [INFO ] [LOCAL ] CBT: Starting deletion of primary data. [2025-10-20T17:50:22.376733] [DEBUG] [LOCAL ] CBT: Deleted primary data file. [2025-10-20T17:50:22.460670] [INFO ] [MAIN ] CBT: Starting full primary refresh. [2025-10-20T17:50:22.550754] [WARN ] [MAIN ] CBRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:22.566541] [INFO ] [REMOTE ] CBO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:22.576560] [INFO ] [REMOTE ] CBO: Fetching primary data for 2025-01. [2025-10-20T17:50:22.618071] [INFO ] [MAIN ] CBRE: Wiping existing primary data before full refresh. [2025-10-20T17:50:22.657289] [INFO ] [LOCAL ] CBRE: Starting deletion of primary data. [2025-10-20T17:50:22.733107] [DEBUG] [LOCAL ] CBRE: Deleted primary data file. [2025-10-20T17:50:22.760436] [INFO ] [MAIN ] CBRE: Starting full primary refresh. [2025-10-20T17:50:22.919430] [DEBUG] [REMOTE ] CBO: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:50:22.961935] [WARN ] [REMOTE ] CBO: No data returned for month 2025-01 [2025-10-20T17:50:22.978226] [FATAL] [REMOTE ] CBO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:50:22.992168] [FATAL] [MAIN ] CBO: Failed to fetch primary data. [2025-10-20T17:50:23.009295] [WARN ] [MAIN ] CBO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:50:23.064280] [INFO ] [LOCAL ] CBO: Starting deletion of primary data. [2025-10-20T17:50:23.198998] [INFO ] [REMOTE ] CBNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.242324] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-01. [2025-10-20T17:50:23.195710] [DEBUG] [LOCAL ] CBO: No primary data file found to delete. [2025-10-20T17:50:23.266560] [INFO ] [LOCAL ] CBO: Starting deletion of derivative data. [2025-10-20T17:50:23.381346] [DEBUG] [LOCAL ] CBO: No derivative data file found to delete. [2025-10-20T17:50:23.401089] [INFO ] [MAIN ] Update Progress: 401/2367 symbols completed. CBO took 2.8s. Estimated remaining: 4h 6m 41s. Rate: 8.0 symbols/min [2025-10-20T17:50:23.677074] [INFO ] [REMOTE ] CBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.722959] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-01. [2025-10-20T17:50:23.717476] [INFO ] [REMOTE ] CBRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:23.737209] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-01. [2025-10-20T17:50:23.841565] [INFO ] [MAIN ] CBU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:23.866178] [DEBUG] [LOCAL ] CBU: Loaded 23143 existing records from primary storage. [2025-10-20T17:50:23.866479] [INFO ] [MAIN ] CBU: Loaded 23143 existing records. [2025-10-20T17:50:24.080730] [WARN ] [MAIN ] CBU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:24.085639] [INFO ] [MAIN ] CBU: Wiping existing primary data before full refresh. [2025-10-20T17:50:24.085737] [INFO ] [LOCAL ] CBU: Starting deletion of primary data. [2025-10-20T17:50:24.086568] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T17:50:24.086713] [DEBUG] [LOCAL ] CBU: Deleted primary data file. [2025-10-20T17:50:24.087061] [INFO ] [MAIN ] CBU: Starting full primary refresh. [2025-10-20T17:50:24.122339] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-02. [2025-10-20T17:50:24.483663] [INFO ] [REMOTE ] CBU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:24.483963] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-01. [2025-10-20T17:50:24.577241] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T17:50:24.579927] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-03. [2025-10-20T17:50:24.841208] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:50:25.036565] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:50:25.051384] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-04. [2025-10-20T17:50:25.346810] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:50:25.665365] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T17:50:25.711696] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T17:50:25.851447] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-02. [2025-10-20T17:50:26.651397] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-05. [2025-10-20T17:50:27.289261] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T17:50:27.355900] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T17:50:27.451378] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-06. [2025-10-20T17:50:27.862965] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T17:50:28.251389] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-02. [2025-10-20T17:50:29.051392] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-03. [2025-10-20T17:50:29.645410] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:50:29.851383] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-03. [2025-10-20T17:50:30.466575] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:50:30.651368] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-04. [2025-10-20T17:50:31.231629] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:50:31.451369] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-02. [2025-10-20T17:50:32.149237] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T17:50:32.251380] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-07. [2025-10-20T17:50:32.507707] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T17:50:32.619064] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T17:50:33.051377] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-03. [2025-10-20T17:50:33.851417] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-04. [2025-10-20T17:50:34.142570] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:50:34.651408] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-05. [2025-10-20T17:50:35.035969] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T17:50:35.451372] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-08. [2025-10-20T17:50:35.881601] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T17:50:36.157826] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:50:36.251382] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-04. [2025-10-20T17:50:37.051375] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-09. [2025-10-20T17:50:37.478237] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T17:50:37.633305] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:50:37.851364] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-05. [2025-10-20T17:50:38.651431] [INFO ] [REMOTE ] CBNA: Fetching primary data for 2025-10. [2025-10-20T17:50:39.074302] [INFO ] [REMOTE ] CBNA: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T17:50:39.075241] [INFO ] [REMOTE ] CBNA: Full fetch complete (4335 records). [2025-10-20T17:50:39.075312] [DEBUG] [VALIDATOR ] CBNA: Validating primary data. [2025-10-20T17:50:39.412343] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:50:39.454350] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-06. [2025-10-20T17:50:39.490757] [DEBUG] [VALIDATOR ] CBNA: All expected trading days are present. [2025-10-20T17:50:39.490833] [DEBUG] [VALIDATOR ] CBNA: Primary data validation passed. [2025-10-20T17:50:39.490973] [INFO ] [MAIN ] CBNA: Fetched primary data is valid. [2025-10-20T17:50:39.491013] [DEBUG] [LOCAL ] CBNA: Saving primary data. [2025-10-20T17:50:39.491416] [DEBUG] [LOCAL ] CBNA: Trimmed 4335 records to 2900 (keeping 200 days of data) [2025-10-20T17:50:39.509947] [DEBUG] [LOCAL ] CBNA: Primary data saved (2900 records). [2025-10-20T17:50:39.510082] [INFO ] [MAIN ] CBNA: Full history saved. [2025-10-20T17:50:39.510203] [INFO ] [MAIN ] Update Progress: 402/2367 symbols completed. CBNA took 18.9s. Estimated remaining: 4h 7m 16s. Rate: 7.9 symbols/min [2025-10-20T17:50:39.695275] [INFO ] [MAIN ] CBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:39.695442] [DEBUG] [LOCAL ] CBX: No existing data file found in primary storage. [2025-10-20T17:50:39.695483] [WARN ] [MAIN ] CBX: No existing data found, performing full refresh instead. [2025-10-20T17:50:39.695595] [INFO ] [MAIN ] CBX: Wiping existing primary data before full refresh. [2025-10-20T17:50:39.695637] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-10-20T17:50:39.695726] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-10-20T17:50:39.695765] [INFO ] [MAIN ] CBX: Starting full primary refresh. [2025-10-20T17:50:40.057063] [INFO ] [REMOTE ] CBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:40.251374] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-05. [2025-10-20T17:50:40.825271] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T17:50:41.051363] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-07. [2025-10-20T17:50:41.262940] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:50:41.851417] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-06. [2025-10-20T17:50:42.655253] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-06. [2025-10-20T17:50:42.660393] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T17:50:42.693877] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T17:50:43.451455] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-07. [2025-10-20T17:50:43.630905] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:50:44.251373] [INFO ] [REMOTE ] CBX: Fetching primary data for 2025-01. [2025-10-20T17:50:44.441632] [DEBUG] [REMOTE ] CBX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:50:44.442004] [WARN ] [REMOTE ] CBX: No data returned for month 2025-01 [2025-10-20T17:50:44.442106] [FATAL] [REMOTE ] CBX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:50:44.442191] [FATAL] [MAIN ] CBX: Failed to fetch primary data. [2025-10-20T17:50:44.442277] [WARN ] [MAIN ] CBX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:50:44.442376] [INFO ] [LOCAL ] CBX: Starting deletion of primary data. [2025-10-20T17:50:44.442460] [DEBUG] [LOCAL ] CBX: No primary data file found to delete. [2025-10-20T17:50:44.442499] [INFO ] [LOCAL ] CBX: Starting deletion of derivative data. [2025-10-20T17:50:44.442537] [DEBUG] [LOCAL ] CBX: No derivative data file found to delete. [2025-10-20T17:50:44.442581] [INFO ] [MAIN ] Update Progress: 403/2367 symbols completed. CBX took 4.9s. Estimated remaining: 4h 6m 55s. Rate: 8.0 symbols/min [2025-10-20T17:50:44.651187] [INFO ] [MAIN ] CBZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:44.685236] [DEBUG] [LOCAL ] CBZ: Loaded 34777 existing records from primary storage. [2025-10-20T17:50:44.685532] [INFO ] [MAIN ] CBZ: Loaded 34777 existing records. [2025-10-20T17:50:44.839871] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T17:50:44.899476] [WARN ] [MAIN ] CBZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:44.899966] [INFO ] [MAIN ] CBZ: Wiping existing primary data before full refresh. [2025-10-20T17:50:44.900014] [INFO ] [LOCAL ] CBZ: Starting deletion of primary data. [2025-10-20T17:50:44.900188] [DEBUG] [LOCAL ] CBZ: Deleted primary data file. [2025-10-20T17:50:44.900227] [INFO ] [MAIN ] CBZ: Starting full primary refresh. [2025-10-20T17:50:45.076113] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-07. [2025-10-20T17:50:45.260871] [INFO ] [REMOTE ] CBZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:45.851413] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-08. [2025-10-20T17:50:46.267063] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:50:46.651411] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-01. [2025-10-20T17:50:47.119716] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:50:47.451387] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-09. [2025-10-20T17:50:47.818373] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T17:50:48.251396] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-08. [2025-10-20T17:50:48.765843] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T17:50:49.051431] [INFO ] [REMOTE ] CBT: Fetching primary data for 2025-10. [2025-10-20T17:50:49.791775] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T17:50:49.851360] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-08. [2025-10-20T17:50:50.402021] [INFO ] [REMOTE ] CBT: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:50:50.407752] [INFO ] [REMOTE ] CBT: Full fetch complete (45846 records). [2025-10-20T17:50:50.407938] [DEBUG] [VALIDATOR ] CBT: Validating primary data. [2025-10-20T17:50:50.657067] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-09. [2025-10-20T17:50:50.790085] [DEBUG] [VALIDATOR ] CBT: All expected trading days are present. [2025-10-20T17:50:50.790173] [DEBUG] [VALIDATOR ] CBT: Primary data validation passed. [2025-10-20T17:50:50.791012] [INFO ] [MAIN ] CBT: Fetched primary data is valid. [2025-10-20T17:50:50.791075] [DEBUG] [LOCAL ] CBT: Saving primary data. [2025-10-20T17:50:50.792530] [DEBUG] [LOCAL ] CBT: Trimmed 45846 records to 31150 (keeping 200 days of data) [2025-10-20T17:50:51.014387] [DEBUG] [LOCAL ] CBT: Primary data saved (31150 records). [2025-10-20T17:50:51.014538] [INFO ] [MAIN ] CBT: Full history saved. [2025-10-20T17:50:51.014619] [INFO ] [MAIN ] Update Progress: 404/2367 symbols completed. CBT took 30.4s. Estimated remaining: 4h 6m 43s. Rate: 8.0 symbols/min [2025-10-20T17:50:51.203439] [INFO ] [MAIN ] CC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:51.220721] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:50:51.281875] [DEBUG] [LOCAL ] CC: Loaded 53789 existing records from primary storage. [2025-10-20T17:50:51.282234] [INFO ] [MAIN ] CC: Loaded 53789 existing records. [2025-10-20T17:50:51.457112] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-02. [2025-10-20T17:50:51.462219] [WARN ] [MAIN ] CC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:51.463615] [INFO ] [MAIN ] CC: Wiping existing primary data before full refresh. [2025-10-20T17:50:51.463779] [INFO ] [LOCAL ] CC: Starting deletion of primary data. [2025-10-20T17:50:51.464758] [DEBUG] [LOCAL ] CC: Deleted primary data file. [2025-10-20T17:50:51.464837] [INFO ] [MAIN ] CC: Starting full primary refresh. [2025-10-20T17:50:51.886339] [INFO ] [REMOTE ] CC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:52.084505] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T17:50:52.251354] [INFO ] [REMOTE ] CBRE: Fetching primary data for 2025-10. [2025-10-20T17:50:52.653481] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T17:50:53.051495] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-09. [2025-10-20T17:50:53.513864] [INFO ] [REMOTE ] CBRE: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T17:50:53.521793] [INFO ] [REMOTE ] CBRE: Full fetch complete (73785 records). [2025-10-20T17:50:53.521960] [DEBUG] [VALIDATOR ] CBRE: Validating primary data. [2025-10-20T17:50:53.861018] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-01. [2025-10-20T17:50:53.928605] [DEBUG] [VALIDATOR ] CBRE: All expected trading days are present. [2025-10-20T17:50:53.928746] [DEBUG] [VALIDATOR ] CBRE: Primary data validation passed. [2025-10-20T17:50:53.929585] [INFO ] [MAIN ] CBRE: Fetched primary data is valid. [2025-10-20T17:50:53.929631] [DEBUG] [LOCAL ] CBRE: Saving primary data. [2025-10-20T17:50:53.931686] [DEBUG] [LOCAL ] CBRE: Trimmed 73785 records to 50305 (keeping 200 days of data) [2025-10-20T17:50:54.266481] [DEBUG] [LOCAL ] CBRE: Primary data saved (50305 records). [2025-10-20T17:50:54.267339] [INFO ] [MAIN ] CBRE: Full history saved. [2025-10-20T17:50:54.269170] [INFO ] [MAIN ] Update Progress: 405/2367 symbols completed. CBRE took 33.6s. Estimated remaining: 4h 6m 15s. Rate: 8.0 symbols/min [2025-10-20T17:50:54.269014] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:50:54.466727] [INFO ] [MAIN ] CCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:54.520993] [DEBUG] [LOCAL ] CCI: Loaded 53890 existing records from primary storage. [2025-10-20T17:50:54.521346] [INFO ] [MAIN ] CCI: Loaded 53890 existing records. [2025-10-20T17:50:54.662008] [INFO ] [REMOTE ] CBU: Fetching primary data for 2025-10. [2025-10-20T17:50:54.703738] [WARN ] [MAIN ] CCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:50:54.704819] [INFO ] [MAIN ] CCI: Wiping existing primary data before full refresh. [2025-10-20T17:50:54.704872] [INFO ] [LOCAL ] CCI: Starting deletion of primary data. [2025-10-20T17:50:54.705380] [DEBUG] [LOCAL ] CCI: Deleted primary data file. [2025-10-20T17:50:54.705436] [INFO ] [MAIN ] CCI: Starting full primary refresh. [2025-10-20T17:50:55.075844] [INFO ] [REMOTE ] CCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:55.451404] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-03. [2025-10-20T17:50:55.480308] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:50:55.744300] [INFO ] [REMOTE ] CBU: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T17:50:55.751113] [INFO ] [REMOTE ] CBU: Full fetch complete (32722 records). [2025-10-20T17:50:55.751304] [DEBUG] [VALIDATOR ] CBU: Validating primary data. [2025-10-20T17:50:56.165138] [DEBUG] [VALIDATOR ] CBU: All expected trading days are present. [2025-10-20T17:50:56.165220] [DEBUG] [VALIDATOR ] CBU: Primary data validation passed. [2025-10-20T17:50:56.165736] [INFO ] [MAIN ] CBU: Fetched primary data is valid. [2025-10-20T17:50:56.165798] [DEBUG] [LOCAL ] CBU: Saving primary data. [2025-10-20T17:50:56.166638] [DEBUG] [LOCAL ] CBU: Trimmed 32722 records to 22862 (keeping 200 days of data) [2025-10-20T17:50:56.255967] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-02. [2025-10-20T17:50:56.308634] [DEBUG] [LOCAL ] CBU: Primary data saved (22862 records). [2025-10-20T17:50:56.308789] [INFO ] [MAIN ] CBU: Full history saved. [2025-10-20T17:50:56.308869] [INFO ] [MAIN ] Update Progress: 406/2367 symbols completed. CBU took 32.9s. Estimated remaining: 4h 5m 41s. Rate: 8.0 symbols/min [2025-10-20T17:50:56.480036] [INFO ] [MAIN ] CCIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:50:56.480227] [DEBUG] [LOCAL ] CCIA: No existing data file found in primary storage. [2025-10-20T17:50:56.480334] [WARN ] [MAIN ] CCIA: No existing data found, performing full refresh instead. [2025-10-20T17:50:56.480456] [INFO ] [MAIN ] CCIA: Wiping existing primary data before full refresh. [2025-10-20T17:50:56.480499] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-10-20T17:50:56.480540] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-10-20T17:50:56.480574] [INFO ] [MAIN ] CCIA: Starting full primary refresh. [2025-10-20T17:50:56.850431] [INFO ] [REMOTE ] CCIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:50:56.870282] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:50:57.051404] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-01. [2025-10-20T17:50:57.283867] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T17:50:57.623957] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T17:50:57.851423] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-04. [2025-10-20T17:50:58.651377] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-01. [2025-10-20T17:50:59.179516] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T17:50:59.451438] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-02. [2025-10-20T17:50:59.769454] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T17:51:00.251468] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-05. [2025-10-20T17:51:00.326002] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T17:51:01.051430] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-03. [2025-10-20T17:51:01.661693] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T17:51:01.851449] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-02. [2025-10-20T17:51:02.651419] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-06. [2025-10-20T17:51:02.968256] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T17:51:03.315615] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:51:03.451453] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-03. [2025-10-20T17:51:03.646596] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T17:51:03.797687] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T17:51:04.251383] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-04. [2025-10-20T17:51:04.546606] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T17:51:05.051394] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-04. [2025-10-20T17:51:05.851397] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-03. [2025-10-20T17:51:06.651369] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-05. [2025-10-20T17:51:06.944056] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T17:51:07.114488] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T17:51:07.451355] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-07. [2025-10-20T17:51:07.658836] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T17:51:08.251373] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-04. [2025-10-20T17:51:08.809988] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:51:09.051387] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-05. [2025-10-20T17:51:09.851431] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-08. [2025-10-20T17:51:10.291511] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T17:51:10.609626] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T17:51:10.651335] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-06. [2025-10-20T17:51:11.353743] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:51:11.451351] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-05. [2025-10-20T17:51:12.251449] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-06. [2025-10-20T17:51:12.288666] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:51:12.713778] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T17:51:13.051366] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-09. [2025-10-20T17:51:13.262219] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T17:51:13.851565] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-06. [2025-10-20T17:51:14.268262] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:51:14.651379] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-07. [2025-10-20T17:51:15.094974] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T17:51:15.451341] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-07. [2025-10-20T17:51:15.558041] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T17:51:16.251380] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-07. [2025-10-20T17:51:17.051355] [INFO ] [REMOTE ] CBZ: Fetching primary data for 2025-10. [2025-10-20T17:51:17.561449] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T17:51:17.683995] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T17:51:17.851433] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-08. [2025-10-20T17:51:18.139566] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T17:51:18.496822] [INFO ] [REMOTE ] CBZ: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T17:51:18.504597] [INFO ] [REMOTE ] CBZ: Full fetch complete (46238 records). [2025-10-20T17:51:18.504793] [DEBUG] [VALIDATOR ] CBZ: Validating primary data. [2025-10-20T17:51:18.657636] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-08. [2025-10-20T17:51:18.888135] [DEBUG] [VALIDATOR ] CBZ: All expected trading days are present. [2025-10-20T17:51:18.888215] [DEBUG] [VALIDATOR ] CBZ: Primary data validation passed. [2025-10-20T17:51:18.891773] [INFO ] [MAIN ] CBZ: Fetched primary data is valid. [2025-10-20T17:51:18.891868] [DEBUG] [LOCAL ] CBZ: Saving primary data. [2025-10-20T17:51:18.892950] [DEBUG] [LOCAL ] CBZ: Trimmed 46238 records to 34284 (keeping 200 days of data) [2025-10-20T17:51:19.084997] [DEBUG] [LOCAL ] CBZ: Primary data saved (34284 records). [2025-10-20T17:51:19.085146] [INFO ] [MAIN ] CBZ: Full history saved. [2025-10-20T17:51:19.085235] [INFO ] [MAIN ] Update Progress: 407/2367 symbols completed. CBZ took 34.6s. Estimated remaining: 4h 6m 47s. Rate: 7.9 symbols/min [2025-10-20T17:51:19.259548] [INFO ] [MAIN ] CCIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:19.274062] [DEBUG] [LOCAL ] CCIF: Loaded 15711 existing records from primary storage. [2025-10-20T17:51:19.274297] [INFO ] [MAIN ] CCIF: Loaded 15711 existing records. [2025-10-20T17:51:19.458761] [WARN ] [MAIN ] CCIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:19.458906] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-09. [2025-10-20T17:51:19.459303] [INFO ] [MAIN ] CCIF: Wiping existing primary data before full refresh. [2025-10-20T17:51:19.460268] [INFO ] [LOCAL ] CCIF: Starting deletion of primary data. [2025-10-20T17:51:19.460455] [DEBUG] [LOCAL ] CCIF: Deleted primary data file. [2025-10-20T17:51:19.460496] [INFO ] [MAIN ] CCIF: Starting full primary refresh. [2025-10-20T17:51:19.815188] [INFO ] [REMOTE ] CCIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:20.033840] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T17:51:20.251391] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-08. [2025-10-20T17:51:20.476259] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T17:51:21.051382] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-01. [2025-10-20T17:51:21.766922] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T17:51:21.851356] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-02. [2025-10-20T17:51:22.037395] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T17:51:22.503818] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:51:22.651358] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-03. [2025-10-20T17:51:23.451364] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-09. [2025-10-20T17:51:23.469820] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T17:51:24.251410] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-04. [2025-10-20T17:51:25.051385] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-09. [2025-10-20T17:51:25.259296] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T17:51:25.284868] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T17:51:25.851381] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-05. [2025-10-20T17:51:26.651434] [INFO ] [REMOTE ] CCIA: Fetching primary data for 2025-10. [2025-10-20T17:51:26.858523] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T17:51:26.994615] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:51:27.116317] [INFO ] [REMOTE ] CCIA: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T17:51:27.117429] [INFO ] [REMOTE ] CCIA: Full fetch complete (2166 records). [2025-10-20T17:51:27.117503] [DEBUG] [VALIDATOR ] CCIA: Validating primary data. [2025-10-20T17:51:27.316754] [WARN ] [VALIDATOR ] CCIA: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T17:51:27.316965] [FATAL] [MAIN ] CCIA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:51:27.317033] [WARN ] [MAIN ] CCIA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:51:27.317084] [INFO ] [LOCAL ] CCIA: Starting deletion of primary data. [2025-10-20T17:51:27.317148] [DEBUG] [LOCAL ] CCIA: No primary data file found to delete. [2025-10-20T17:51:27.317184] [INFO ] [LOCAL ] CCIA: Starting deletion of derivative data. [2025-10-20T17:51:27.317223] [DEBUG] [LOCAL ] CCIA: No derivative data file found to delete. [2025-10-20T17:51:27.317283] [INFO ] [MAIN ] Update Progress: 408/2367 symbols completed. CCIA took 31.0s. Estimated remaining: 4h 6m 43s. Rate: 7.9 symbols/min [2025-10-20T17:51:27.461823] [INFO ] [REMOTE ] CC: Fetching primary data for 2025-10. [2025-10-20T17:51:27.487842] [INFO ] [MAIN ] CCJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:27.539312] [DEBUG] [LOCAL ] CCJ: Loaded 54468 existing records from primary storage. [2025-10-20T17:51:27.539778] [INFO ] [MAIN ] CCJ: Loaded 54468 existing records. [2025-10-20T17:51:27.726532] [WARN ] [MAIN ] CCJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:27.726776] [INFO ] [MAIN ] CCJ: Wiping existing primary data before full refresh. [2025-10-20T17:51:27.726828] [INFO ] [LOCAL ] CCJ: Starting deletion of primary data. [2025-10-20T17:51:27.727164] [DEBUG] [LOCAL ] CCJ: Deleted primary data file. [2025-10-20T17:51:27.727221] [INFO ] [MAIN ] CCJ: Starting full primary refresh. [2025-10-20T17:51:28.081960] [INFO ] [REMOTE ] CCJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:28.251389] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-01. [2025-10-20T17:51:29.051355] [INFO ] [REMOTE ] CCI: Fetching primary data for 2025-10. [2025-10-20T17:51:29.151105] [INFO ] [REMOTE ] CC: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T17:51:29.160477] [INFO ] [REMOTE ] CC: Full fetch complete (76146 records). [2025-10-20T17:51:29.160743] [DEBUG] [VALIDATOR ] CC: Validating primary data. [2025-10-20T17:51:29.569287] [DEBUG] [VALIDATOR ] CC: All expected trading days are present. [2025-10-20T17:51:29.569384] [DEBUG] [VALIDATOR ] CC: Primary data validation passed. [2025-10-20T17:51:29.570244] [INFO ] [MAIN ] CC: Fetched primary data is valid. [2025-10-20T17:51:29.570309] [DEBUG] [LOCAL ] CC: Saving primary data. [2025-10-20T17:51:29.571756] [DEBUG] [LOCAL ] CC: Trimmed 76146 records to 53030 (keeping 200 days of data) [2025-10-20T17:51:29.863170] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-06. [2025-10-20T17:51:29.888252] [DEBUG] [LOCAL ] CC: Primary data saved (53030 records). [2025-10-20T17:51:29.888627] [INFO ] [MAIN ] CC: Full history saved. [2025-10-20T17:51:29.888751] [INFO ] [MAIN ] Update Progress: 409/2367 symbols completed. CC took 38.9s. Estimated remaining: 4h 6m 11s. Rate: 8.0 symbols/min [2025-10-20T17:51:30.060256] [INFO ] [MAIN ] CCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:30.100439] [DEBUG] [LOCAL ] CCK: Loaded 47154 existing records from primary storage. [2025-10-20T17:51:30.100772] [INFO ] [MAIN ] CCK: Loaded 47154 existing records. [2025-10-20T17:51:30.278218] [WARN ] [MAIN ] CCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:30.278517] [INFO ] [MAIN ] CCK: Wiping existing primary data before full refresh. [2025-10-20T17:51:30.278580] [INFO ] [LOCAL ] CCK: Starting deletion of primary data. [2025-10-20T17:51:30.278872] [DEBUG] [LOCAL ] CCK: Deleted primary data file. [2025-10-20T17:51:30.278927] [INFO ] [MAIN ] CCK: Starting full primary refresh. [2025-10-20T17:51:30.644116] [INFO ] [REMOTE ] CCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:30.658431] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-01. [2025-10-20T17:51:30.683745] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-10-20T17:51:30.901121] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:51:31.128002] [INFO ] [REMOTE ] CCI: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T17:51:31.135897] [INFO ] [REMOTE ] CCI: Full fetch complete (77098 records). [2025-10-20T17:51:31.136019] [DEBUG] [VALIDATOR ] CCI: Validating primary data. [2025-10-20T17:51:31.464965] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-07. [2025-10-20T17:51:31.523727] [DEBUG] [VALIDATOR ] CCI: All expected trading days are present. [2025-10-20T17:51:31.523815] [DEBUG] [VALIDATOR ] CCI: Primary data validation passed. [2025-10-20T17:51:31.524614] [INFO ] [MAIN ] CCI: Fetched primary data is valid. [2025-10-20T17:51:31.524691] [DEBUG] [LOCAL ] CCI: Saving primary data. [2025-10-20T17:51:31.526302] [DEBUG] [LOCAL ] CCI: Trimmed 77098 records to 53116 (keeping 200 days of data) [2025-10-20T17:51:31.851370] [DEBUG] [LOCAL ] CCI: Primary data saved (53116 records). [2025-10-20T17:51:31.851510] [INFO ] [MAIN ] CCI: Full history saved. [2025-10-20T17:51:31.851835] [INFO ] [MAIN ] Update Progress: 410/2367 symbols completed. CCI took 37.6s. Estimated remaining: 4h 5m 37s. Rate: 8.0 symbols/min [2025-10-20T17:51:32.020289] [INFO ] [MAIN ] CCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:32.071108] [DEBUG] [LOCAL ] CCL: Loaded 54580 existing records from primary storage. [2025-10-20T17:51:32.071419] [INFO ] [MAIN ] CCL: Loaded 54580 existing records. [2025-10-20T17:51:32.255267] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-02. [2025-10-20T17:51:32.262623] [WARN ] [MAIN ] CCL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:32.281600] [INFO ] [MAIN ] CCL: Wiping existing primary data before full refresh. [2025-10-20T17:51:32.282304] [INFO ] [LOCAL ] CCL: Starting deletion of primary data. [2025-10-20T17:51:32.283612] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T17:51:32.292457] [DEBUG] [LOCAL ] CCL: Deleted primary data file. [2025-10-20T17:51:32.294070] [INFO ] [MAIN ] CCL: Starting full primary refresh. [2025-10-20T17:51:32.505286] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T17:51:32.701201] [INFO ] [REMOTE ] CCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:33.051407] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-08. [2025-10-20T17:51:33.851453] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-01. [2025-10-20T17:51:33.947863] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T17:51:34.217434] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T17:51:34.651418] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-09. [2025-10-20T17:51:35.433326] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T17:51:35.451349] [INFO ] [REMOTE ] CCIF: Fetching primary data for 2025-10. [2025-10-20T17:51:36.188446] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-01 (2.34 seconds). [2025-10-20T17:51:36.220664] [INFO ] [REMOTE ] CCIF: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T17:51:36.223085] [INFO ] [REMOTE ] CCIF: Full fetch complete (21876 records). [2025-10-20T17:51:36.223161] [DEBUG] [VALIDATOR ] CCIF: Validating primary data. [2025-10-20T17:51:36.256460] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-02. [2025-10-20T17:51:36.594543] [DEBUG] [VALIDATOR ] CCIF: All expected trading days are present. [2025-10-20T17:51:36.594622] [DEBUG] [VALIDATOR ] CCIF: Primary data validation passed. [2025-10-20T17:51:36.595026] [INFO ] [MAIN ] CCIF: Fetched primary data is valid. [2025-10-20T17:51:36.595078] [DEBUG] [LOCAL ] CCIF: Saving primary data. [2025-10-20T17:51:36.595732] [DEBUG] [LOCAL ] CCIF: Trimmed 21876 records to 15294 (keeping 200 days of data) [2025-10-20T17:51:36.684893] [DEBUG] [LOCAL ] CCIF: Primary data saved (15294 records). [2025-10-20T17:51:36.685037] [INFO ] [MAIN ] CCIF: Full history saved. [2025-10-20T17:51:36.685116] [INFO ] [MAIN ] Update Progress: 411/2367 symbols completed. CCIF took 17.6s. Estimated remaining: 4h 5m 17s. Rate: 8.0 symbols/min [2025-10-20T17:51:36.878926] [INFO ] [MAIN ] CCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:36.920565] [DEBUG] [LOCAL ] CCO: Loaded 49089 existing records from primary storage. [2025-10-20T17:51:36.920934] [INFO ] [MAIN ] CCO: Loaded 49089 existing records. [2025-10-20T17:51:37.060437] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-03. [2025-10-20T17:51:37.103242] [WARN ] [MAIN ] CCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:37.103729] [INFO ] [MAIN ] CCO: Wiping existing primary data before full refresh. [2025-10-20T17:51:37.103798] [INFO ] [LOCAL ] CCO: Starting deletion of primary data. [2025-10-20T17:51:37.104067] [DEBUG] [LOCAL ] CCO: Deleted primary data file. [2025-10-20T17:51:37.104123] [INFO ] [MAIN ] CCO: Starting full primary refresh. [2025-10-20T17:51:37.473871] [INFO ] [REMOTE ] CCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:51:37.680245] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:51:37.851377] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-03. [2025-10-20T17:51:38.651330] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-02. [2025-10-20T17:51:39.161339] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:51:39.208897] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T17:51:39.451372] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-04. [2025-10-20T17:51:40.251392] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-04. [2025-10-20T17:51:40.877218] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-02 (2.23 seconds). [2025-10-20T17:51:41.051407] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-03. [2025-10-20T17:51:41.133153] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T17:51:41.799225] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:51:41.851399] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-05. [2025-10-20T17:51:42.651385] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-05. [2025-10-20T17:51:43.451361] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-01. [2025-10-20T17:51:43.524449] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-03 (2.47 seconds). [2025-10-20T17:51:43.627159] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T17:51:43.846977] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T17:51:44.251423] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-06. [2025-10-20T17:51:45.056842] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-06. [2025-10-20T17:51:45.061382] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:51:45.851439] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-02. [2025-10-20T17:51:46.116057] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T17:51:46.250714] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T17:51:46.651412] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-07. [2025-10-20T17:51:47.244447] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T17:51:47.451364] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-03. [2025-10-20T17:51:48.251423] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-04. [2025-10-20T17:51:48.652958] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T17:51:48.857150] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T17:51:49.051486] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-04. [2025-10-20T17:51:49.851458] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-08. [2025-10-20T17:51:50.456275] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-10-20T17:51:50.651397] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-05. [2025-10-20T17:51:50.845419] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:51:51.451429] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-05. [2025-10-20T17:51:51.777068] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T17:51:52.251433] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-07. [2025-10-20T17:51:52.653398] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T17:51:53.051456] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-09. [2025-10-20T17:51:53.170182] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T17:51:53.492558] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T17:51:53.851458] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-08. [2025-10-20T17:51:54.651434] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-06. [2025-10-20T17:51:55.033231] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T17:51:55.359573] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T17:51:55.451360] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-09. [2025-10-20T17:51:56.251420] [INFO ] [REMOTE ] CCJ: Fetching primary data for 2025-10. [2025-10-20T17:51:56.982509] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-06 (2.33 seconds). [2025-10-20T17:51:57.051343] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-07. [2025-10-20T17:51:57.096432] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T17:51:57.851370] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-06. [2025-10-20T17:51:58.651437] [INFO ] [REMOTE ] CCK: Fetching primary data for 2025-10. [2025-10-20T17:51:58.762110] [INFO ] [REMOTE ] CCJ: Incremental fetch complete for 2025-10 (2.51 seconds). [2025-10-20T17:51:58.770442] [INFO ] [REMOTE ] CCJ: Full fetch complete (77840 records). [2025-10-20T17:51:58.770615] [DEBUG] [VALIDATOR ] CCJ: Validating primary data. [2025-10-20T17:51:59.176738] [DEBUG] [VALIDATOR ] CCJ: All expected trading days are present. [2025-10-20T17:51:59.177105] [DEBUG] [VALIDATOR ] CCJ: Primary data validation passed. [2025-10-20T17:51:59.178017] [INFO ] [MAIN ] CCJ: Fetched primary data is valid. [2025-10-20T17:51:59.178082] [DEBUG] [LOCAL ] CCJ: Saving primary data. [2025-10-20T17:51:59.179622] [DEBUG] [LOCAL ] CCJ: Trimmed 77840 records to 53690 (keeping 200 days of data) [2025-10-20T17:51:59.529631] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-07 (2.48 seconds). [2025-10-20T17:51:59.535549] [DEBUG] [LOCAL ] CCJ: Primary data saved (53690 records). [2025-10-20T17:51:59.546709] [INFO ] [MAIN ] CCJ: Full history saved. [2025-10-20T17:51:59.547736] [INFO ] [MAIN ] Update Progress: 412/2367 symbols completed. CCJ took 32.2s. Estimated remaining: 4h 6m 22s. Rate: 7.9 symbols/min [2025-10-20T17:51:59.576598] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-08. [2025-10-20T17:51:59.642560] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T17:51:59.763087] [INFO ] [MAIN ] CCRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:51:59.772890] [DEBUG] [LOCAL ] CCRD: Loaded 9647 existing records from primary storage. [2025-10-20T17:51:59.773117] [INFO ] [MAIN ] CCRD: Loaded 9647 existing records. [2025-10-20T17:51:59.947999] [WARN ] [MAIN ] CCRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:51:59.948233] [INFO ] [MAIN ] CCRD: Wiping existing primary data before full refresh. [2025-10-20T17:51:59.948289] [INFO ] [LOCAL ] CCRD: Starting deletion of primary data. [2025-10-20T17:51:59.948478] [DEBUG] [LOCAL ] CCRD: Deleted primary data file. [2025-10-20T17:51:59.948530] [INFO ] [MAIN ] CCRD: Starting full primary refresh. [2025-10-20T17:52:00.251942] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-07. [2025-10-20T17:52:00.326911] [INFO ] [REMOTE ] CCRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:00.409850] [INFO ] [REMOTE ] CCK: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:52:00.417103] [INFO ] [REMOTE ] CCK: Full fetch complete (68007 records). [2025-10-20T17:52:00.417261] [DEBUG] [VALIDATOR ] CCK: Validating primary data. [2025-10-20T17:52:00.809770] [DEBUG] [VALIDATOR ] CCK: All expected trading days are present. [2025-10-20T17:52:00.809855] [DEBUG] [VALIDATOR ] CCK: Primary data validation passed. [2025-10-20T17:52:00.810573] [INFO ] [MAIN ] CCK: Fetched primary data is valid. [2025-10-20T17:52:00.810612] [DEBUG] [LOCAL ] CCK: Saving primary data. [2025-10-20T17:52:00.811898] [DEBUG] [LOCAL ] CCK: Trimmed 68007 records to 46482 (keeping 200 days of data) [2025-10-20T17:52:01.053708] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-01. [2025-10-20T17:52:01.109596] [DEBUG] [LOCAL ] CCK: Primary data saved (46482 records). [2025-10-20T17:52:01.109773] [INFO ] [MAIN ] CCK: Full history saved. [2025-10-20T17:52:01.109866] [INFO ] [MAIN ] Update Progress: 413/2367 symbols completed. CCK took 31.2s. Estimated remaining: 4h 5m 46s. Rate: 8.0 symbols/min [2025-10-20T17:52:01.280881] [INFO ] [MAIN ] CCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:01.310352] [DEBUG] [LOCAL ] CCS: Loaded 30600 existing records from primary storage. [2025-10-20T17:52:01.310627] [INFO ] [MAIN ] CCS: Loaded 30600 existing records. [2025-10-20T17:52:01.487523] [WARN ] [MAIN ] CCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:01.487755] [INFO ] [MAIN ] CCS: Wiping existing primary data before full refresh. [2025-10-20T17:52:01.487806] [INFO ] [LOCAL ] CCS: Starting deletion of primary data. [2025-10-20T17:52:01.488176] [DEBUG] [LOCAL ] CCS: Deleted primary data file. [2025-10-20T17:52:01.488233] [INFO ] [MAIN ] CCS: Starting full primary refresh. [2025-10-20T17:52:01.794286] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T17:52:01.854953] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-02. [2025-10-20T17:52:01.909840] [INFO ] [REMOTE ] CCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:02.062510] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T17:52:02.110446] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T17:52:02.336206] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T17:52:02.651374] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-01. [2025-10-20T17:52:03.451446] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-08. [2025-10-20T17:52:04.127931] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T17:52:04.251450] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-03. [2025-10-20T17:52:04.825729] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T17:52:04.825566] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T17:52:05.051403] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-09. [2025-10-20T17:52:05.851484] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-02. [2025-10-20T17:52:06.630042] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T17:52:06.651355] [INFO ] [REMOTE ] CCO: Fetching primary data for 2025-10. [2025-10-20T17:52:06.868145] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T17:52:07.451379] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-09. [2025-10-20T17:52:08.251404] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-04. [2025-10-20T17:52:08.425580] [INFO ] [REMOTE ] CCO: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T17:52:08.433087] [INFO ] [REMOTE ] CCO: Full fetch complete (67203 records). [2025-10-20T17:52:08.433235] [DEBUG] [VALIDATOR ] CCO: Validating primary data. [2025-10-20T17:52:08.888806] [DEBUG] [VALIDATOR ] CCO: All expected trading days are present. [2025-10-20T17:52:08.893231] [DEBUG] [VALIDATOR ] CCO: Primary data validation passed. [2025-10-20T17:52:08.894055] [INFO ] [MAIN ] CCO: Fetched primary data is valid. [2025-10-20T17:52:08.894800] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T17:52:08.894617] [DEBUG] [LOCAL ] CCO: Saving primary data. [2025-10-20T17:52:08.898179] [DEBUG] [LOCAL ] CCO: Trimmed 67203 records to 48483 (keeping 200 days of data) [2025-10-20T17:52:09.057558] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-05. [2025-10-20T17:52:09.165772] [DEBUG] [LOCAL ] CCO: Primary data saved (48483 records). [2025-10-20T17:52:09.165892] [INFO ] [MAIN ] CCO: Full history saved. [2025-10-20T17:52:09.165965] [INFO ] [MAIN ] Update Progress: 414/2367 symbols completed. CCO took 32.5s. Estimated remaining: 4h 5m 41s. Rate: 7.9 symbols/min [2025-10-20T17:52:09.333826] [INFO ] [MAIN ] CCU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:09.373953] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T17:52:09.395146] [DEBUG] [LOCAL ] CCU: Loaded 22640 existing records from primary storage. [2025-10-20T17:52:09.395484] [INFO ] [MAIN ] CCU: Loaded 22640 existing records. [2025-10-20T17:52:09.604603] [WARN ] [MAIN ] CCU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:09.604882] [INFO ] [MAIN ] CCU: Wiping existing primary data before full refresh. [2025-10-20T17:52:09.604951] [INFO ] [LOCAL ] CCU: Starting deletion of primary data. [2025-10-20T17:52:09.605146] [DEBUG] [LOCAL ] CCU: Deleted primary data file. [2025-10-20T17:52:09.605203] [INFO ] [MAIN ] CCU: Starting full primary refresh. [2025-10-20T17:52:09.777110] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T17:52:09.856830] [INFO ] [REMOTE ] CCL: Fetching primary data for 2025-10. [2025-10-20T17:52:10.026067] [INFO ] [REMOTE ] CCU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:10.651369] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-03. [2025-10-20T17:52:11.451368] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-06. [2025-10-20T17:52:12.054499] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:52:12.070590] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:52:12.251396] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-01. [2025-10-20T17:52:12.417372] [INFO ] [REMOTE ] CCL: Incremental fetch complete for 2025-10 (2.56 seconds). [2025-10-20T17:52:12.425927] [INFO ] [REMOTE ] CCL: Full fetch complete (77980 records). [2025-10-20T17:52:12.426096] [DEBUG] [VALIDATOR ] CCL: Validating primary data. [2025-10-20T17:52:12.826543] [DEBUG] [VALIDATOR ] CCL: All expected trading days are present. [2025-10-20T17:52:12.826696] [DEBUG] [VALIDATOR ] CCL: Primary data validation passed. [2025-10-20T17:52:12.827517] [INFO ] [MAIN ] CCL: Fetched primary data is valid. [2025-10-20T17:52:12.827568] [DEBUG] [LOCAL ] CCL: Saving primary data. [2025-10-20T17:52:12.829100] [DEBUG] [LOCAL ] CCL: Trimmed 77980 records to 53800 (keeping 200 days of data) [2025-10-20T17:52:13.060090] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-07. [2025-10-20T17:52:13.156831] [DEBUG] [LOCAL ] CCL: Primary data saved (53800 records). [2025-10-20T17:52:13.156987] [INFO ] [MAIN ] CCL: Full history saved. [2025-10-20T17:52:13.157447] [INFO ] [MAIN ] Update Progress: 415/2367 symbols completed. CCL took 41.3s. Estimated remaining: 4h 5m 16s. Rate: 8.0 symbols/min [2025-10-20T17:52:13.330222] [INFO ] [MAIN ] CCZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:13.330499] [DEBUG] [LOCAL ] CCZ: No existing data file found in primary storage. [2025-10-20T17:52:13.330632] [WARN ] [MAIN ] CCZ: No existing data found, performing full refresh instead. [2025-10-20T17:52:13.330774] [INFO ] [MAIN ] CCZ: Wiping existing primary data before full refresh. [2025-10-20T17:52:13.330818] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-10-20T17:52:13.330876] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-10-20T17:52:13.330986] [INFO ] [MAIN ] CCZ: Starting full primary refresh. [2025-10-20T17:52:13.453360] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:52:13.738877] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T17:52:13.744513] [INFO ] [REMOTE ] CCZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:13.851362] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-08. [2025-10-20T17:52:14.651368] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-04. [2025-10-20T17:52:14.861321] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T17:52:15.451383] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-02. [2025-10-20T17:52:16.181847] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T17:52:16.251374] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-05. [2025-10-20T17:52:16.564567] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:52:17.051371] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-09. [2025-10-20T17:52:17.551604] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T17:52:17.851403] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-01. [2025-10-20T17:52:17.935974] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T17:52:18.181274] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T17:52:18.651383] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-06. [2025-10-20T17:52:19.451440] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-03. [2025-10-20T17:52:19.667232] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T17:52:20.251423] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-02. [2025-10-20T17:52:20.538107] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T17:52:20.761497] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T17:52:21.051407] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-04. [2025-10-20T17:52:21.851392] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-03. [2025-10-20T17:52:22.074492] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-03 (0.22 seconds). [2025-10-20T17:52:22.396386] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T17:52:22.651376] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-05. [2025-10-20T17:52:23.451422] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-07. [2025-10-20T17:52:24.018711] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:52:24.251369] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-06. [2025-10-20T17:52:24.810525] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T17:52:25.051351] [INFO ] [REMOTE ] CCRD: Fetching primary data for 2025-10. [2025-10-20T17:52:25.286597] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:52:25.726336] [INFO ] [REMOTE ] CCRD: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:52:25.728142] [INFO ] [REMOTE ] CCRD: Full fetch complete (11827 records). [2025-10-20T17:52:25.728210] [DEBUG] [VALIDATOR ] CCRD: Validating primary data. [2025-10-20T17:52:25.865737] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-04. [2025-10-20T17:52:26.105069] [DEBUG] [VALIDATOR ] CCRD: All expected trading days are present. [2025-10-20T17:52:26.105144] [DEBUG] [VALIDATOR ] CCRD: Primary data validation passed. [2025-10-20T17:52:26.105425] [INFO ] [MAIN ] CCRD: Fetched primary data is valid. [2025-10-20T17:52:26.105464] [DEBUG] [LOCAL ] CCRD: Saving primary data. [2025-10-20T17:52:26.106024] [DEBUG] [LOCAL ] CCRD: Trimmed 11827 records to 9603 (keeping 200 days of data) [2025-10-20T17:52:26.165018] [DEBUG] [LOCAL ] CCRD: Primary data saved (9603 records). [2025-10-20T17:52:26.165144] [INFO ] [MAIN ] CCRD: Full history saved. [2025-10-20T17:52:26.165218] [INFO ] [MAIN ] Update Progress: 416/2367 symbols completed. CCRD took 26.6s. Estimated remaining: 4h 5m 35s. Rate: 7.9 symbols/min [2025-10-20T17:52:26.213369] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T17:52:26.345738] [INFO ] [MAIN ] CDE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:26.393561] [DEBUG] [LOCAL ] CDE: Loaded 54423 existing records from primary storage. [2025-10-20T17:52:26.393894] [INFO ] [MAIN ] CDE: Loaded 54423 existing records. [2025-10-20T17:52:26.568354] [WARN ] [MAIN ] CDE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:26.568559] [INFO ] [MAIN ] CDE: Wiping existing primary data before full refresh. [2025-10-20T17:52:26.568604] [INFO ] [LOCAL ] CDE: Starting deletion of primary data. [2025-10-20T17:52:26.568863] [DEBUG] [LOCAL ] CDE: Deleted primary data file. [2025-10-20T17:52:26.568919] [INFO ] [MAIN ] CDE: Starting full primary refresh. [2025-10-20T17:52:26.656544] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-07. [2025-10-20T17:52:26.939744] [INFO ] [REMOTE ] CDE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:27.451424] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-01. [2025-10-20T17:52:27.928492] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T17:52:28.251418] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-08. [2025-10-20T17:52:29.051411] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-08. [2025-10-20T17:52:29.157598] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:52:29.289382] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T17:52:29.851390] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-05. [2025-10-20T17:52:30.051693] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T17:52:30.162808] [INFO ] [REMOTE ] CCZ: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T17:52:30.651390] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-09. [2025-10-20T17:52:31.451466] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-09. [2025-10-20T17:52:31.939867] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T17:52:32.251435] [INFO ] [REMOTE ] CCS: Fetching primary data for 2025-10. [2025-10-20T17:52:32.433453] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T17:52:33.051425] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-02. [2025-10-20T17:52:33.548171] [INFO ] [REMOTE ] CCS: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T17:52:33.555539] [INFO ] [REMOTE ] CCS: Full fetch complete (44811 records). [2025-10-20T17:52:33.555689] [DEBUG] [VALIDATOR ] CCS: Validating primary data. [2025-10-20T17:52:33.856521] [INFO ] [REMOTE ] CCZ: Fetching primary data for 2025-06. [2025-10-20T17:52:33.932158] [DEBUG] [VALIDATOR ] CCS: All expected trading days are present. [2025-10-20T17:52:33.932298] [DEBUG] [VALIDATOR ] CCS: Primary data validation passed. [2025-10-20T17:52:33.932933] [INFO ] [MAIN ] CCS: Fetched primary data is valid. [2025-10-20T17:52:33.932996] [DEBUG] [LOCAL ] CCS: Saving primary data. [2025-10-20T17:52:33.934101] [DEBUG] [LOCAL ] CCS: Trimmed 44811 records to 30084 (keeping 200 days of data) [2025-10-20T17:52:34.110881] [DEBUG] [LOCAL ] CCS: Primary data saved (30084 records). [2025-10-20T17:52:34.111011] [INFO ] [MAIN ] CCS: Full history saved. [2025-10-20T17:52:34.111088] [INFO ] [MAIN ] Update Progress: 417/2367 symbols completed. CCS took 33.0s. Estimated remaining: 4h 5m 29s. Rate: 7.9 symbols/min [2025-10-20T17:52:34.179115] [DEBUG] [REMOTE ] CCZ: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:52:34.200837] [WARN ] [REMOTE ] CCZ: No data returned for month 2025-06 [2025-10-20T17:52:34.211346] [FATAL] [REMOTE ] CCZ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:52:34.211439] [FATAL] [MAIN ] CCZ: Failed to fetch primary data. [2025-10-20T17:52:34.211809] [WARN ] [MAIN ] CCZ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:52:34.211892] [INFO ] [LOCAL ] CCZ: Starting deletion of primary data. [2025-10-20T17:52:34.222441] [DEBUG] [LOCAL ] CCZ: No primary data file found to delete. [2025-10-20T17:52:34.227664] [INFO ] [LOCAL ] CCZ: Starting deletion of derivative data. [2025-10-20T17:52:34.237957] [DEBUG] [LOCAL ] CCZ: No derivative data file found to delete. [2025-10-20T17:52:34.250428] [INFO ] [MAIN ] Update Progress: 418/2367 symbols completed. CCZ took 21.1s. Estimated remaining: 4h 4m 47s. Rate: 8.0 symbols/min [2025-10-20T17:52:34.347000] [INFO ] [MAIN ] CDLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:34.444830] [DEBUG] [LOCAL ] CDLR: Loaded 10294 existing records from primary storage. [2025-10-20T17:52:34.462718] [INFO ] [MAIN ] CDLR: Loaded 10294 existing records. [2025-10-20T17:52:34.463574] [INFO ] [MAIN ] CDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:34.650084] [WARN ] [MAIN ] CDLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:34.650736] [INFO ] [MAIN ] CDLR: Wiping existing primary data before full refresh. [2025-10-20T17:52:34.650924] [INFO ] [LOCAL ] CDLR: Starting deletion of primary data. [2025-10-20T17:52:34.651131] [DEBUG] [LOCAL ] CDLR: Deleted primary data file. [2025-10-20T17:52:34.651222] [INFO ] [MAIN ] CDLR: Starting full primary refresh. [2025-10-20T17:52:34.662421] [INFO ] [REMOTE ] CCU: Fetching primary data for 2025-10. [2025-10-20T17:52:34.963552] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:52:34.960695] [DEBUG] [LOCAL ] CDP: Loaded 43123 existing records from primary storage. [2025-10-20T17:52:34.970264] [INFO ] [MAIN ] CDP: Loaded 43123 existing records. [2025-10-20T17:52:35.267182] [INFO ] [REMOTE ] CDLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:35.290677] [WARN ] [MAIN ] CDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:35.290916] [INFO ] [MAIN ] CDP: Wiping existing primary data before full refresh. [2025-10-20T17:52:35.290967] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-10-20T17:52:35.291194] [DEBUG] [LOCAL ] CDP: Deleted primary data file. [2025-10-20T17:52:35.291249] [INFO ] [MAIN ] CDP: Starting full primary refresh. [2025-10-20T17:52:35.455390] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-01. [2025-10-20T17:52:35.662511] [INFO ] [REMOTE ] CDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:35.934065] [INFO ] [REMOTE ] CCU: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T17:52:35.937701] [INFO ] [REMOTE ] CCU: Full fetch complete (32526 records). [2025-10-20T17:52:35.937804] [DEBUG] [VALIDATOR ] CCU: Validating primary data. [2025-10-20T17:52:36.256695] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-03. [2025-10-20T17:52:36.345839] [DEBUG] [VALIDATOR ] CCU: All expected trading days are present. [2025-10-20T17:52:36.346299] [DEBUG] [VALIDATOR ] CCU: Primary data validation passed. [2025-10-20T17:52:36.346194] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:52:36.347286] [INFO ] [MAIN ] CCU: Fetched primary data is valid. [2025-10-20T17:52:36.349148] [DEBUG] [LOCAL ] CCU: Saving primary data. [2025-10-20T17:52:36.351061] [DEBUG] [LOCAL ] CCU: Trimmed 32526 records to 21982 (keeping 200 days of data) [2025-10-20T17:52:36.488267] [DEBUG] [LOCAL ] CCU: Primary data saved (21982 records). [2025-10-20T17:52:36.488441] [INFO ] [MAIN ] CCU: Full history saved. [2025-10-20T17:52:36.488533] [INFO ] [MAIN ] Update Progress: 419/2367 symbols completed. CCU took 27.3s. Estimated remaining: 4h 4m 15s. Rate: 8.0 symbols/min [2025-10-20T17:52:36.660565] [INFO ] [MAIN ] CDRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:52:36.683682] [DEBUG] [LOCAL ] CDRE: Loaded 25250 existing records from primary storage. [2025-10-20T17:52:36.683962] [INFO ] [MAIN ] CDRE: Loaded 25250 existing records. [2025-10-20T17:52:36.861453] [WARN ] [MAIN ] CDRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:52:36.861714] [INFO ] [MAIN ] CDRE: Wiping existing primary data before full refresh. [2025-10-20T17:52:36.861770] [INFO ] [LOCAL ] CDRE: Starting deletion of primary data. [2025-10-20T17:52:36.862084] [DEBUG] [LOCAL ] CDRE: Deleted primary data file. [2025-10-20T17:52:36.862139] [INFO ] [MAIN ] CDRE: Starting full primary refresh. [2025-10-20T17:52:37.056618] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-01. [2025-10-20T17:52:37.226995] [INFO ] [REMOTE ] CDRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:52:37.851458] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-01. [2025-10-20T17:52:37.998219] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:52:38.651367] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-04. [2025-10-20T17:52:38.719592] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:52:38.745527] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T17:52:39.451437] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-02. [2025-10-20T17:52:40.251478] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-02. [2025-10-20T17:52:40.811986] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T17:52:40.868033] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:52:40.983696] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T17:52:41.051389] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-03. [2025-10-20T17:52:41.851424] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-05. [2025-10-20T17:52:42.385528] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T17:52:42.651437] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-03. [2025-10-20T17:52:43.451424] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-02. [2025-10-20T17:52:43.520713] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:52:43.789172] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T17:52:44.102841] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:52:44.251460] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-04. [2025-10-20T17:52:45.051422] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-04. [2025-10-20T17:52:45.856602] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-03. [2025-10-20T17:52:45.887298] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:52:45.926757] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T17:52:46.651444] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-05. [2025-10-20T17:52:46.757052] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T17:52:47.451383] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-06. [2025-10-20T17:52:48.048823] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T17:52:48.251407] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-05. [2025-10-20T17:52:49.051391] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-04. [2025-10-20T17:52:49.282793] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T17:52:49.375009] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T17:52:49.801623] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T17:52:49.851342] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-05. [2025-10-20T17:52:50.472060] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T17:52:50.651422] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-06. [2025-10-20T17:52:51.451460] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-06. [2025-10-20T17:52:52.153865] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T17:52:52.224334] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:52:52.251385] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-07. [2025-10-20T17:52:53.051397] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-07. [2025-10-20T17:52:53.110268] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T17:52:53.851436] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-07. [2025-10-20T17:52:54.282618] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T17:52:54.651386] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-08. [2025-10-20T17:52:55.451471] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-08. [2025-10-20T17:52:56.001460] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T17:52:56.100361] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T17:52:56.251443] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-08. [2025-10-20T17:52:56.408008] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:52:57.051377] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-09. [2025-10-20T17:52:57.851431] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-06. [2025-10-20T17:52:58.067747] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T17:52:58.222080] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T17:52:58.644376] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T17:52:58.651333] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-07. [2025-10-20T17:52:59.174634] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T17:52:59.451369] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-09. [2025-10-20T17:53:00.251439] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-08. [2025-10-20T17:53:01.051429] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-09. [2025-10-20T17:53:01.077074] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T17:53:01.607017] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T17:53:01.851367] [INFO ] [REMOTE ] CDP: Fetching primary data for 2025-10. [2025-10-20T17:53:02.273428] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T17:53:02.651480] [INFO ] [REMOTE ] CDE: Fetching primary data for 2025-10. [2025-10-20T17:53:03.173074] [INFO ] [REMOTE ] CDP: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T17:53:03.179467] [INFO ] [REMOTE ] CDP: Full fetch complete (61721 records). [2025-10-20T17:53:03.179681] [DEBUG] [VALIDATOR ] CDP: Validating primary data. [2025-10-20T17:53:03.180505] [FATAL] [VALIDATOR ] CDP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T17:53:03.180596] [FATAL] [MAIN ] CDP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:53:03.180664] [WARN ] [MAIN ] CDP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:53:03.180744] [INFO ] [LOCAL ] CDP: Starting deletion of primary data. [2025-10-20T17:53:03.180809] [DEBUG] [LOCAL ] CDP: No primary data file found to delete. [2025-10-20T17:53:03.180844] [INFO ] [LOCAL ] CDP: Starting deletion of derivative data. [2025-10-20T17:53:03.181032] [DEBUG] [LOCAL ] CDP: Deleted derivative data file. [2025-10-20T17:53:03.181370] [INFO ] [MAIN ] Update Progress: 420/2367 symbols completed. CDP took 28.9s. Estimated remaining: 4h 5m 36s. Rate: 7.9 symbols/min [2025-10-20T17:53:03.361271] [INFO ] [MAIN ] CE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:03.409155] [DEBUG] [LOCAL ] CE: Loaded 52065 existing records from primary storage. [2025-10-20T17:53:03.409467] [INFO ] [MAIN ] CE: Loaded 52065 existing records. [2025-10-20T17:53:03.459152] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-09. [2025-10-20T17:53:03.617604] [WARN ] [MAIN ] CE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:03.617907] [INFO ] [MAIN ] CE: Wiping existing primary data before full refresh. [2025-10-20T17:53:03.617956] [INFO ] [LOCAL ] CE: Starting deletion of primary data. [2025-10-20T17:53:03.618346] [DEBUG] [LOCAL ] CE: Deleted primary data file. [2025-10-20T17:53:03.618404] [INFO ] [MAIN ] CE: Starting full primary refresh. [2025-10-20T17:53:03.991951] [INFO ] [REMOTE ] CE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:04.251451] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-01. [2025-10-20T17:53:04.345600] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T17:53:04.639592] [INFO ] [REMOTE ] CDE: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T17:53:04.647702] [INFO ] [REMOTE ] CDE: Full fetch complete (77787 records). [2025-10-20T17:53:04.647831] [DEBUG] [VALIDATOR ] CDE: Validating primary data. [2025-10-20T17:53:05.053260] [INFO ] [REMOTE ] CDLR: Fetching primary data for 2025-10. [2025-10-20T17:53:05.106915] [DEBUG] [VALIDATOR ] CDE: All expected trading days are present. [2025-10-20T17:53:05.106996] [DEBUG] [VALIDATOR ] CDE: Primary data validation passed. [2025-10-20T17:53:05.107866] [INFO ] [MAIN ] CDE: Fetched primary data is valid. [2025-10-20T17:53:05.107927] [DEBUG] [LOCAL ] CDE: Saving primary data. [2025-10-20T17:53:05.109430] [DEBUG] [LOCAL ] CDE: Trimmed 77787 records to 53643 (keeping 200 days of data) [2025-10-20T17:53:05.408463] [DEBUG] [LOCAL ] CDE: Primary data saved (53643 records). [2025-10-20T17:53:05.408597] [INFO ] [MAIN ] CDE: Full history saved. [2025-10-20T17:53:05.408695] [INFO ] [MAIN ] Update Progress: 421/2367 symbols completed. CDE took 39.2s. Estimated remaining: 4h 5m 4s. Rate: 7.9 symbols/min [2025-10-20T17:53:05.579286] [INFO ] [MAIN ] CEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:05.586700] [DEBUG] [LOCAL ] CEE: Loaded 5462 existing records from primary storage. [2025-10-20T17:53:05.586879] [INFO ] [MAIN ] CEE: Loaded 5462 existing records. [2025-10-20T17:53:05.765538] [WARN ] [MAIN ] CEE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:05.765860] [INFO ] [MAIN ] CEE: Wiping existing primary data before full refresh. [2025-10-20T17:53:05.766021] [INFO ] [LOCAL ] CEE: Starting deletion of primary data. [2025-10-20T17:53:05.766202] [DEBUG] [LOCAL ] CEE: Deleted primary data file. [2025-10-20T17:53:05.766265] [INFO ] [MAIN ] CEE: Starting full primary refresh. [2025-10-20T17:53:05.862514] [INFO ] [REMOTE ] CDRE: Fetching primary data for 2025-10. [2025-10-20T17:53:05.998015] [INFO ] [REMOTE ] CDLR: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T17:53:06.078788] [INFO ] [REMOTE ] CDLR: Full fetch complete (14119 records). [2025-10-20T17:53:06.106373] [DEBUG] [VALIDATOR ] CDLR: Validating primary data. [2025-10-20T17:53:06.177717] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T17:53:06.181275] [INFO ] [REMOTE ] CEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:06.578372] [DEBUG] [VALIDATOR ] CDLR: All expected trading days are present. [2025-10-20T17:53:06.578466] [DEBUG] [VALIDATOR ] CDLR: Primary data validation passed. [2025-10-20T17:53:06.578988] [INFO ] [MAIN ] CDLR: Fetched primary data is valid. [2025-10-20T17:53:06.579050] [DEBUG] [LOCAL ] CDLR: Saving primary data. [2025-10-20T17:53:06.579572] [DEBUG] [LOCAL ] CDLR: Trimmed 14119 records to 10221 (keeping 200 days of data) [2025-10-20T17:53:06.639120] [DEBUG] [LOCAL ] CDLR: Primary data saved (10221 records). [2025-10-20T17:53:06.639273] [INFO ] [MAIN ] CDLR: Full history saved. [2025-10-20T17:53:06.639356] [INFO ] [MAIN ] Update Progress: 422/2367 symbols completed. CDLR took 32.5s. Estimated remaining: 4h 4m 27s. Rate: 8.0 symbols/min [2025-10-20T17:53:06.662048] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-01. [2025-10-20T17:53:06.820571] [INFO ] [MAIN ] CEPU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:06.848498] [DEBUG] [LOCAL ] CEPU: Loaded 27225 existing records from primary storage. [2025-10-20T17:53:06.848817] [INFO ] [MAIN ] CEPU: Loaded 27225 existing records. [2025-10-20T17:53:07.036668] [WARN ] [MAIN ] CEPU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:07.037048] [INFO ] [MAIN ] CEPU: Wiping existing primary data before full refresh. [2025-10-20T17:53:07.037172] [INFO ] [LOCAL ] CEPU: Starting deletion of primary data. [2025-10-20T17:53:07.037410] [DEBUG] [LOCAL ] CEPU: Deleted primary data file. [2025-10-20T17:53:07.042667] [INFO ] [MAIN ] CEPU: Starting full primary refresh. [2025-10-20T17:53:07.114570] [INFO ] [REMOTE ] CDRE: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T17:53:07.154291] [INFO ] [REMOTE ] CDRE: Full fetch complete (35110 records). [2025-10-20T17:53:07.156877] [DEBUG] [VALIDATOR ] CDRE: Validating primary data. [2025-10-20T17:53:07.470625] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-02. [2025-10-20T17:53:07.512052] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T17:53:07.729251] [INFO ] [REMOTE ] CEPU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:07.807705] [DEBUG] [VALIDATOR ] CDRE: All expected trading days are present. [2025-10-20T17:53:07.807789] [DEBUG] [VALIDATOR ] CDRE: Primary data validation passed. [2025-10-20T17:53:07.808294] [INFO ] [MAIN ] CDRE: Fetched primary data is valid. [2025-10-20T17:53:07.808364] [DEBUG] [LOCAL ] CDRE: Saving primary data. [2025-10-20T17:53:07.809131] [DEBUG] [LOCAL ] CDRE: Trimmed 35110 records to 24879 (keeping 200 days of data) [2025-10-20T17:53:07.958083] [DEBUG] [LOCAL ] CDRE: Primary data saved (24879 records). [2025-10-20T17:53:07.958225] [INFO ] [MAIN ] CDRE: Full history saved. [2025-10-20T17:53:07.958310] [INFO ] [MAIN ] Update Progress: 423/2367 symbols completed. CDRE took 31.5s. Estimated remaining: 4h 3m 51s. Rate: 8.0 symbols/min [2025-10-20T17:53:08.138146] [INFO ] [MAIN ] CF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:08.184016] [DEBUG] [LOCAL ] CF: Loaded 52902 existing records from primary storage. [2025-10-20T17:53:08.184329] [INFO ] [MAIN ] CF: Loaded 52902 existing records. [2025-10-20T17:53:08.256732] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-02. [2025-10-20T17:53:08.359798] [WARN ] [MAIN ] CF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:08.359996] [INFO ] [MAIN ] CF: Wiping existing primary data before full refresh. [2025-10-20T17:53:08.360043] [INFO ] [LOCAL ] CF: Starting deletion of primary data. [2025-10-20T17:53:08.360226] [DEBUG] [LOCAL ] CF: Deleted primary data file. [2025-10-20T17:53:08.360264] [INFO ] [MAIN ] CF: Starting full primary refresh. [2025-10-20T17:53:08.702235] [INFO ] [REMOTE ] CF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:09.051415] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-01. [2025-10-20T17:53:09.188732] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T17:53:09.219258] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T17:53:09.851437] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-01. [2025-10-20T17:53:10.651425] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-03. [2025-10-20T17:53:10.744093] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:53:11.274060] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:53:11.451362] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-02. [2025-10-20T17:53:12.251347] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-02. [2025-10-20T17:53:12.564772] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T17:53:12.608165] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:53:13.051373] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-03. [2025-10-20T17:53:13.785707] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:53:13.851365] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-03. [2025-10-20T17:53:13.926295] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T17:53:14.651387] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-04. [2025-10-20T17:53:14.770868] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T17:53:15.451439] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-04. [2025-10-20T17:53:16.097422] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T17:53:16.251409] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-05. [2025-10-20T17:53:16.603828] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T17:53:16.959824] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T17:53:17.051385] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-03. [2025-10-20T17:53:17.851373] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-04. [2025-10-20T17:53:18.574536] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T17:53:18.644358] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T17:53:18.651448] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-05. [2025-10-20T17:53:19.451421] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-06. [2025-10-20T17:53:19.902614] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T17:53:20.226227] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T17:53:20.251391] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-05. [2025-10-20T17:53:21.051356] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-04. [2025-10-20T17:53:21.851379] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-06. [2025-10-20T17:53:22.023072] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T17:53:22.651372] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-06. [2025-10-20T17:53:22.979988] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T17:53:23.008907] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T17:53:23.451424] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-05. [2025-10-20T17:53:24.259387] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-07. [2025-10-20T17:53:24.287620] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:53:25.051380] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-07. [2025-10-20T17:53:25.317866] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T17:53:25.461544] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:53:25.752766] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T17:53:25.851394] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-08. [2025-10-20T17:53:26.504752] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:53:26.651395] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-07. [2025-10-20T17:53:27.451423] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-06. [2025-10-20T17:53:28.251399] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-09. [2025-10-20T17:53:28.486779] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T17:53:28.831746] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T17:53:28.997422] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:53:29.051352] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-07. [2025-10-20T17:53:29.851403] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-08. [2025-10-20T17:53:30.523982] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T17:53:30.651374] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-08. [2025-10-20T17:53:30.974353] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T17:53:31.451408] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-08. [2025-10-20T17:53:32.047926] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T17:53:32.251385] [INFO ] [REMOTE ] CEE: Fetching primary data for 2025-10. [2025-10-20T17:53:32.930622] [INFO ] [REMOTE ] CEE: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:53:32.931840] [INFO ] [REMOTE ] CEE: Full fetch complete (10175 records). [2025-10-20T17:53:32.931911] [DEBUG] [VALIDATOR ] CEE: Validating primary data. [2025-10-20T17:53:33.056773] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-09. [2025-10-20T17:53:33.342337] [DEBUG] [VALIDATOR ] CEE: All expected trading days are present. [2025-10-20T17:53:33.342533] [DEBUG] [VALIDATOR ] CEE: Primary data validation passed. [2025-10-20T17:53:33.342899] [INFO ] [MAIN ] CEE: Fetched primary data is valid. [2025-10-20T17:53:33.342989] [DEBUG] [LOCAL ] CEE: Saving primary data. [2025-10-20T17:53:33.343780] [DEBUG] [LOCAL ] CEE: Trimmed 10175 records to 5304 (keeping 200 days of data) [2025-10-20T17:53:33.411751] [DEBUG] [LOCAL ] CEE: Primary data saved (5304 records). [2025-10-20T17:53:33.412805] [INFO ] [MAIN ] CEE: Full history saved. [2025-10-20T17:53:33.413429] [INFO ] [MAIN ] Update Progress: 424/2367 symbols completed. CEE took 28.0s. Estimated remaining: 4h 5m 5s. Rate: 7.9 symbols/min [2025-10-20T17:53:33.417845] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T17:53:33.605668] [INFO ] [MAIN ] CFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:33.656264] [DEBUG] [LOCAL ] CFG: Loaded 54403 existing records from primary storage. [2025-10-20T17:53:33.656622] [INFO ] [MAIN ] CFG: Loaded 54403 existing records. [2025-10-20T17:53:33.842891] [WARN ] [MAIN ] CFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:33.843109] [INFO ] [MAIN ] CFG: Wiping existing primary data before full refresh. [2025-10-20T17:53:33.843156] [INFO ] [LOCAL ] CFG: Starting deletion of primary data. [2025-10-20T17:53:33.843516] [DEBUG] [LOCAL ] CFG: Deleted primary data file. [2025-10-20T17:53:33.843555] [INFO ] [MAIN ] CFG: Starting full primary refresh. [2025-10-20T17:53:33.856580] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-09. [2025-10-20T17:53:34.226219] [INFO ] [REMOTE ] CFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:34.268132] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:53:34.651392] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-01. [2025-10-20T17:53:35.451351] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-09. [2025-10-20T17:53:35.627619] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:53:36.170335] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:53:36.251386] [INFO ] [REMOTE ] CEPU: Fetching primary data for 2025-10. [2025-10-20T17:53:37.050758] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T17:53:37.060982] [INFO ] [REMOTE ] CF: Fetching primary data for 2025-10. [2025-10-20T17:53:37.314058] [INFO ] [REMOTE ] CEPU: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T17:53:37.319369] [INFO ] [REMOTE ] CEPU: Full fetch complete (39845 records). [2025-10-20T17:53:37.319516] [DEBUG] [VALIDATOR ] CEPU: Validating primary data. [2025-10-20T17:53:37.685601] [DEBUG] [VALIDATOR ] CEPU: All expected trading days are present. [2025-10-20T17:53:37.685707] [DEBUG] [VALIDATOR ] CEPU: Primary data validation passed. [2025-10-20T17:53:37.686248] [INFO ] [MAIN ] CEPU: Fetched primary data is valid. [2025-10-20T17:53:37.686293] [DEBUG] [LOCAL ] CEPU: Saving primary data. [2025-10-20T17:53:37.687126] [DEBUG] [LOCAL ] CEPU: Trimmed 39845 records to 26925 (keeping 200 days of data) [2025-10-20T17:53:37.836410] [DEBUG] [LOCAL ] CEPU: Primary data saved (26925 records). [2025-10-20T17:53:37.836604] [INFO ] [MAIN ] CEPU: Full history saved. [2025-10-20T17:53:37.836752] [INFO ] [MAIN ] Update Progress: 425/2367 symbols completed. CEPU took 31.2s. Estimated remaining: 4h 4m 43s. Rate: 7.9 symbols/min [2025-10-20T17:53:37.857445] [INFO ] [REMOTE ] CE: Fetching primary data for 2025-10. [2025-10-20T17:53:38.006278] [INFO ] [MAIN ] CFND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:38.006414] [DEBUG] [LOCAL ] CFND: No existing data file found in primary storage. [2025-10-20T17:53:38.006451] [WARN ] [MAIN ] CFND: No existing data found, performing full refresh instead. [2025-10-20T17:53:38.006581] [INFO ] [MAIN ] CFND: Wiping existing primary data before full refresh. [2025-10-20T17:53:38.006637] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-10-20T17:53:38.006735] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-10-20T17:53:38.006773] [INFO ] [MAIN ] CFND: Starting full primary refresh. [2025-10-20T17:53:38.368534] [INFO ] [REMOTE ] CFND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:38.651393] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-02. [2025-10-20T17:53:38.963047] [INFO ] [REMOTE ] CF: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T17:53:38.975992] [INFO ] [REMOTE ] CF: Full fetch complete (75588 records). [2025-10-20T17:53:38.976300] [DEBUG] [VALIDATOR ] CF: Validating primary data. [2025-10-20T17:53:39.467174] [INFO ] [REMOTE ] CFND: Fetching primary data for 2025-01. [2025-10-20T17:53:39.544570] [DEBUG] [VALIDATOR ] CF: All expected trading days are present. [2025-10-20T17:53:39.544784] [DEBUG] [VALIDATOR ] CF: Primary data validation passed. [2025-10-20T17:53:39.545745] [INFO ] [MAIN ] CF: Fetched primary data is valid. [2025-10-20T17:53:39.545811] [DEBUG] [LOCAL ] CF: Saving primary data. [2025-10-20T17:53:39.547378] [DEBUG] [LOCAL ] CF: Trimmed 75588 records to 52126 (keeping 200 days of data) [2025-10-20T17:53:39.803085] [DEBUG] [REMOTE ] CFND: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:53:39.808827] [WARN ] [REMOTE ] CFND: No data returned for month 2025-01 [2025-10-20T17:53:39.813816] [FATAL] [REMOTE ] CFND: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:53:39.817169] [FATAL] [MAIN ] CFND: Failed to fetch primary data. [2025-10-20T17:53:39.824337] [WARN ] [MAIN ] CFND: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:53:39.826930] [INFO ] [LOCAL ] CFND: Starting deletion of primary data. [2025-10-20T17:53:39.827290] [DEBUG] [LOCAL ] CFND: No primary data file found to delete. [2025-10-20T17:53:39.827419] [INFO ] [LOCAL ] CFND: Starting deletion of derivative data. [2025-10-20T17:53:39.867314] [DEBUG] [LOCAL ] CFND: No derivative data file found to delete. [2025-10-20T17:53:39.899839] [INFO ] [MAIN ] Update Progress: 426/2367 symbols completed. CFND took 2.1s. Estimated remaining: 4h 4m 11s. Rate: 7.9 symbols/min [2025-10-20T17:53:39.912207] [DEBUG] [LOCAL ] CF: Primary data saved (52126 records). [2025-10-20T17:53:39.942566] [INFO ] [MAIN ] CF: Full history saved. [2025-10-20T17:53:40.015219] [INFO ] [MAIN ] Update Progress: 427/2367 symbols completed. CF took 32.1s. Estimated remaining: 4h 3m 30s. Rate: 8.0 symbols/min [2025-10-20T17:53:40.027970] [INFO ] [REMOTE ] CE: Incremental fetch complete for 2025-10 (2.17 seconds). [2025-10-20T17:53:40.116906] [INFO ] [REMOTE ] CE: Full fetch complete (74838 records). [2025-10-20T17:53:40.126832] [DEBUG] [VALIDATOR ] CE: Validating primary data. [2025-10-20T17:53:40.333460] [INFO ] [MAIN ] CFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:40.450111] [INFO ] [MAIN ] CGAU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:40.750088] [DEBUG] [VALIDATOR ] CE: All expected trading days are present. [2025-10-20T17:53:40.759304] [DEBUG] [VALIDATOR ] CE: Primary data validation passed. [2025-10-20T17:53:40.760279] [INFO ] [MAIN ] CE: Fetched primary data is valid. [2025-10-20T17:53:40.760625] [DEBUG] [LOCAL ] CE: Saving primary data. [2025-10-20T17:53:40.800297] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T17:53:40.798671] [DEBUG] [LOCAL ] CE: Trimmed 74838 records to 51312 (keeping 200 days of data) [2025-10-20T17:53:40.798472] [DEBUG] [LOCAL ] CFR: Loaded 33178 existing records from primary storage. [2025-10-20T17:53:40.822603] [INFO ] [MAIN ] CFR: Loaded 33178 existing records. [2025-10-20T17:53:40.823850] [DEBUG] [LOCAL ] CGAU: Loaded 44311 existing records from primary storage. [2025-10-20T17:53:40.887936] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-03. [2025-10-20T17:53:40.908601] [INFO ] [MAIN ] CGAU: Loaded 44311 existing records. [2025-10-20T17:53:41.253218] [WARN ] [MAIN ] CFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:41.277476] [INFO ] [MAIN ] CFR: Wiping existing primary data before full refresh. [2025-10-20T17:53:41.336393] [INFO ] [LOCAL ] CFR: Starting deletion of primary data. [2025-10-20T17:53:41.376116] [WARN ] [MAIN ] CGAU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:41.403639] [DEBUG] [LOCAL ] CFR: Deleted primary data file. [2025-10-20T17:53:41.404748] [INFO ] [MAIN ] CGAU: Wiping existing primary data before full refresh. [2025-10-20T17:53:41.404614] [INFO ] [MAIN ] CFR: Starting full primary refresh. [2025-10-20T17:53:41.404878] [INFO ] [LOCAL ] CGAU: Starting deletion of primary data. [2025-10-20T17:53:41.423813] [DEBUG] [LOCAL ] CGAU: Deleted primary data file. [2025-10-20T17:53:41.446456] [INFO ] [MAIN ] CGAU: Starting full primary refresh. [2025-10-20T17:53:42.166792] [INFO ] [REMOTE ] CFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:42.126285] [DEBUG] [LOCAL ] CE: Primary data saved (51312 records). [2025-10-20T17:53:42.171638] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-01. [2025-10-20T17:53:42.172597] [INFO ] [MAIN ] CE: Full history saved. [2025-10-20T17:53:42.173382] [INFO ] [MAIN ] Update Progress: 428/2367 symbols completed. CE took 39.0s. Estimated remaining: 4h 2m 58s. Rate: 8.0 symbols/min [2025-10-20T17:53:42.359741] [INFO ] [REMOTE ] CGAU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:42.376402] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-01. [2025-10-20T17:53:42.431241] [INFO ] [MAIN ] CGV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:53:42.435181] [DEBUG] [LOCAL ] CGV: Loaded 1418 existing records from primary storage. [2025-10-20T17:53:42.435334] [INFO ] [MAIN ] CGV: Loaded 1418 existing records. [2025-10-20T17:53:42.612696] [WARN ] [MAIN ] CGV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:53:42.613079] [INFO ] [MAIN ] CGV: Wiping existing primary data before full refresh. [2025-10-20T17:53:42.613141] [INFO ] [LOCAL ] CGV: Starting deletion of primary data. [2025-10-20T17:53:42.613263] [DEBUG] [LOCAL ] CGV: Deleted primary data file. [2025-10-20T17:53:42.613301] [INFO ] [MAIN ] CGV: Starting full primary refresh. [2025-10-20T17:53:43.011257] [INFO ] [REMOTE ] CGV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:53:43.014433] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-01. [2025-10-20T17:53:43.016923] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-03 (2.13 seconds). [2025-10-20T17:53:43.393251] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-01 (0.38 seconds). [2025-10-20T17:53:43.451431] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-04. [2025-10-20T17:53:43.789397] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T17:53:44.251412] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-02. [2025-10-20T17:53:44.292807] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T17:53:44.578890] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T17:53:44.931268] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T17:53:45.051338] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-05. [2025-10-20T17:53:45.851385] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-02. [2025-10-20T17:53:46.651384] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-02. [2025-10-20T17:53:46.929974] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T17:53:47.084690] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T17:53:47.451390] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-06. [2025-10-20T17:53:47.765927] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T17:53:48.251389] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-03. [2025-10-20T17:53:49.051435] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-03. [2025-10-20T17:53:49.187950] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:53:49.537291] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T17:53:49.656095] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T17:53:49.851375] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-04. [2025-10-20T17:53:50.651430] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-07. [2025-10-20T17:53:51.280956] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T17:53:51.451380] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-05. [2025-10-20T17:53:51.927897] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T17:53:52.251469] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-03. [2025-10-20T17:53:52.824421] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T17:53:53.051453] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-04. [2025-10-20T17:53:53.506614] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-10-20T17:53:53.736760] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T17:53:53.851417] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-04. [2025-10-20T17:53:54.651393] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-05. [2025-10-20T17:53:55.006477] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-05 (0.36 seconds). [2025-10-20T17:53:55.225574] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T17:53:55.451381] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-08. [2025-10-20T17:53:56.251416] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-06. [2025-10-20T17:53:56.682152] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T17:53:57.051409] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-06. [2025-10-20T17:53:57.285404] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T17:53:57.851378] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-09. [2025-10-20T17:53:58.651403] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-05. [2025-10-20T17:53:58.689670] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T17:53:59.347669] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T17:53:59.451395] [INFO ] [REMOTE ] CFG: Fetching primary data for 2025-10. [2025-10-20T17:54:00.014881] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:54:00.251438] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-07. [2025-10-20T17:54:01.051754] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-06. [2025-10-20T17:54:01.417942] [INFO ] [REMOTE ] CFG: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T17:54:01.426824] [INFO ] [REMOTE ] CFG: Full fetch complete (77760 records). [2025-10-20T17:54:01.426997] [DEBUG] [VALIDATOR ] CFG: Validating primary data. [2025-10-20T17:54:01.854800] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-07. [2025-10-20T17:54:01.879172] [DEBUG] [VALIDATOR ] CFG: All expected trading days are present. [2025-10-20T17:54:01.880956] [DEBUG] [VALIDATOR ] CFG: Primary data validation passed. [2025-10-20T17:54:01.882022] [INFO ] [MAIN ] CFG: Fetched primary data is valid. [2025-10-20T17:54:01.882116] [DEBUG] [LOCAL ] CFG: Saving primary data. [2025-10-20T17:54:01.895271] [DEBUG] [LOCAL ] CFG: Trimmed 77760 records to 53623 (keeping 200 days of data) [2025-10-20T17:54:01.970912] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T17:54:02.260826] [DEBUG] [LOCAL ] CFG: Primary data saved (53623 records). [2025-10-20T17:54:02.260961] [INFO ] [MAIN ] CFG: Full history saved. [2025-10-20T17:54:02.261038] [INFO ] [MAIN ] Update Progress: 429/2367 symbols completed. CFG took 28.8s. Estimated remaining: 4h 3m 47s. Rate: 7.9 symbols/min [2025-10-20T17:54:02.382039] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:54:02.457773] [INFO ] [MAIN ] CHCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:02.451848] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T17:54:02.489019] [DEBUG] [LOCAL ] CHCT: Loaded 24518 existing records from primary storage. [2025-10-20T17:54:02.489411] [INFO ] [MAIN ] CHCT: Loaded 24518 existing records. [2025-10-20T17:54:02.667765] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-08. [2025-10-20T17:54:02.673085] [WARN ] [MAIN ] CHCT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:02.674377] [INFO ] [MAIN ] CHCT: Wiping existing primary data before full refresh. [2025-10-20T17:54:02.674440] [INFO ] [LOCAL ] CHCT: Starting deletion of primary data. [2025-10-20T17:54:02.675024] [DEBUG] [LOCAL ] CHCT: Deleted primary data file. [2025-10-20T17:54:02.675299] [INFO ] [MAIN ] CHCT: Starting full primary refresh. [2025-10-20T17:54:03.061713] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T17:54:03.067660] [INFO ] [REMOTE ] CHCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:03.451491] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-07. [2025-10-20T17:54:04.251470] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-08. [2025-10-20T17:54:04.451885] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T17:54:05.051494] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-08. [2025-10-20T17:54:05.782075] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T17:54:05.851334] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-09. [2025-10-20T17:54:06.550799] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T17:54:06.651355] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-09. [2025-10-20T17:54:07.103795] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T17:54:07.451354] [INFO ] [REMOTE ] CGV: Fetching primary data for 2025-10. [2025-10-20T17:54:07.517154] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T17:54:07.961978] [INFO ] [REMOTE ] CGV: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T17:54:07.962933] [INFO ] [REMOTE ] CGV: Full fetch complete (2243 records). [2025-10-20T17:54:07.963006] [DEBUG] [VALIDATOR ] CGV: Validating primary data. [2025-10-20T17:54:08.256875] [INFO ] [REMOTE ] CGAU: Fetching primary data for 2025-10. [2025-10-20T17:54:08.369084] [DEBUG] [VALIDATOR ] CGV: All expected trading days are present. [2025-10-20T17:54:08.369187] [DEBUG] [VALIDATOR ] CGV: Primary data validation passed. [2025-10-20T17:54:08.369329] [INFO ] [MAIN ] CGV: Fetched primary data is valid. [2025-10-20T17:54:08.369396] [DEBUG] [LOCAL ] CGV: Saving primary data. [2025-10-20T17:54:08.369796] [DEBUG] [LOCAL ] CGV: Trimmed 2243 records to 1384 (keeping 200 days of data) [2025-10-20T17:54:08.379250] [DEBUG] [LOCAL ] CGV: Primary data saved (1384 records). [2025-10-20T17:54:08.379391] [INFO ] [MAIN ] CGV: Full history saved. [2025-10-20T17:54:08.379480] [INFO ] [MAIN ] Update Progress: 430/2367 symbols completed. CGV took 26.2s. Estimated remaining: 4h 3m 33s. Rate: 8.0 symbols/min [2025-10-20T17:54:08.565432] [INFO ] [MAIN ] CHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:08.632140] [DEBUG] [LOCAL ] CHD: Loaded 53066 existing records from primary storage. [2025-10-20T17:54:08.632637] [INFO ] [MAIN ] CHD: Loaded 53066 existing records. [2025-10-20T17:54:08.892897] [WARN ] [MAIN ] CHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:08.893227] [INFO ] [MAIN ] CHD: Wiping existing primary data before full refresh. [2025-10-20T17:54:08.893305] [INFO ] [LOCAL ] CHD: Starting deletion of primary data. [2025-10-20T17:54:08.893774] [DEBUG] [LOCAL ] CHD: Deleted primary data file. [2025-10-20T17:54:08.893835] [INFO ] [MAIN ] CHD: Starting full primary refresh. [2025-10-20T17:54:09.056542] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-09. [2025-10-20T17:54:09.353310] [INFO ] [REMOTE ] CHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:09.851443] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-01. [2025-10-20T17:54:10.233993] [INFO ] [REMOTE ] CGAU: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:54:10.242345] [INFO ] [REMOTE ] CGAU: Full fetch complete (61920 records). [2025-10-20T17:54:10.242574] [DEBUG] [VALIDATOR ] CGAU: Validating primary data. [2025-10-20T17:54:10.443104] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T17:54:10.656009] [DEBUG] [VALIDATOR ] CGAU: All expected trading days are present. [2025-10-20T17:54:10.655812] [INFO ] [REMOTE ] CFR: Fetching primary data for 2025-10. [2025-10-20T17:54:10.656864] [DEBUG] [VALIDATOR ] CGAU: Primary data validation passed. [2025-10-20T17:54:10.657677] [INFO ] [MAIN ] CGAU: Fetched primary data is valid. [2025-10-20T17:54:10.657738] [DEBUG] [LOCAL ] CGAU: Saving primary data. [2025-10-20T17:54:10.659049] [DEBUG] [LOCAL ] CGAU: Trimmed 61920 records to 43818 (keeping 200 days of data) [2025-10-20T17:54:10.943947] [DEBUG] [LOCAL ] CGAU: Primary data saved (43818 records). [2025-10-20T17:54:10.944196] [INFO ] [MAIN ] CGAU: Full history saved. [2025-10-20T17:54:10.944404] [INFO ] [MAIN ] Update Progress: 431/2367 symbols completed. CGAU took 30.9s. Estimated remaining: 4h 3m 3s. Rate: 8.0 symbols/min [2025-10-20T17:54:11.125712] [INFO ] [MAIN ] CHE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:11.143670] [DEBUG] [LOCAL ] CHE: Loaded 15548 existing records from primary storage. [2025-10-20T17:54:11.144031] [INFO ] [MAIN ] CHE: Loaded 15548 existing records. [2025-10-20T17:54:11.347320] [WARN ] [MAIN ] CHE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:11.351055] [INFO ] [MAIN ] CHE: Wiping existing primary data before full refresh. [2025-10-20T17:54:11.351164] [INFO ] [LOCAL ] CHE: Starting deletion of primary data. [2025-10-20T17:54:11.350923] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:54:11.357436] [DEBUG] [LOCAL ] CHE: Deleted primary data file. [2025-10-20T17:54:11.357582] [INFO ] [MAIN ] CHE: Starting full primary refresh. [2025-10-20T17:54:11.456590] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-01. [2025-10-20T17:54:11.751130] [INFO ] [REMOTE ] CHE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:12.251344] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-02. [2025-10-20T17:54:12.653841] [INFO ] [REMOTE ] CFR: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T17:54:12.658688] [INFO ] [REMOTE ] CFR: Full fetch complete (48533 records). [2025-10-20T17:54:12.658807] [DEBUG] [VALIDATOR ] CFR: Validating primary data. [2025-10-20T17:54:13.046575] [DEBUG] [VALIDATOR ] CFR: All expected trading days are present. [2025-10-20T17:54:13.046676] [DEBUG] [VALIDATOR ] CFR: Primary data validation passed. [2025-10-20T17:54:13.047262] [INFO ] [MAIN ] CFR: Fetched primary data is valid. [2025-10-20T17:54:13.047333] [DEBUG] [LOCAL ] CFR: Saving primary data. [2025-10-20T17:54:13.048382] [DEBUG] [LOCAL ] CFR: Trimmed 48533 records to 32637 (keeping 200 days of data) [2025-10-20T17:54:13.102857] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-01. [2025-10-20T17:54:13.282143] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T17:54:13.297433] [DEBUG] [LOCAL ] CFR: Primary data saved (32637 records). [2025-10-20T17:54:13.297569] [INFO ] [MAIN ] CFR: Full history saved. [2025-10-20T17:54:13.297671] [INFO ] [MAIN ] Update Progress: 432/2367 symbols completed. CFR took 33.4s. Estimated remaining: 4h 2m 32s. Rate: 8.0 symbols/min [2025-10-20T17:54:13.474875] [INFO ] [MAIN ] CHGG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:13.518762] [DEBUG] [LOCAL ] CHGG: Loaded 48419 existing records from primary storage. [2025-10-20T17:54:13.519056] [INFO ] [MAIN ] CHGG: Loaded 48419 existing records. [2025-10-20T17:54:13.693020] [WARN ] [MAIN ] CHGG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:13.693233] [INFO ] [MAIN ] CHGG: Wiping existing primary data before full refresh. [2025-10-20T17:54:13.693282] [INFO ] [LOCAL ] CHGG: Starting deletion of primary data. [2025-10-20T17:54:13.693539] [DEBUG] [LOCAL ] CHGG: Deleted primary data file. [2025-10-20T17:54:13.693596] [INFO ] [MAIN ] CHGG: Starting full primary refresh. [2025-10-20T17:54:13.851736] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-02. [2025-10-20T17:54:14.076058] [INFO ] [REMOTE ] CHGG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:14.091859] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T17:54:14.313293] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T17:54:14.651466] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-01. [2025-10-20T17:54:15.452271] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-03. [2025-10-20T17:54:15.457688] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T17:54:16.251351] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-03. [2025-10-20T17:54:16.454384] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:54:16.546572] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T17:54:17.051428] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-02. [2025-10-20T17:54:17.851361] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-04. [2025-10-20T17:54:18.014035] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T17:54:18.651424] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-04. [2025-10-20T17:54:18.797924] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:54:19.102954] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T17:54:19.451456] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-03. [2025-10-20T17:54:20.251365] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-02. [2025-10-20T17:54:20.318173] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T17:54:20.901499] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:54:21.001817] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T17:54:21.051372] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-03. [2025-10-20T17:54:21.851406] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-04. [2025-10-20T17:54:21.929221] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T17:54:22.651414] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-04. [2025-10-20T17:54:23.451383] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-05. [2025-10-20T17:54:23.490851] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:54:23.720906] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T17:54:24.251378] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-05. [2025-10-20T17:54:24.963061] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T17:54:25.051394] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-05. [2025-10-20T17:54:25.074839] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T17:54:25.851412] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-06. [2025-10-20T17:54:26.479915] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T17:54:26.651382] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-05. [2025-10-20T17:54:27.348832] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T17:54:27.451387] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-06. [2025-10-20T17:54:27.745464] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T17:54:28.151074] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:54:28.251452] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-07. [2025-10-20T17:54:29.051444] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-07. [2025-10-20T17:54:29.563619] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T17:54:29.851406] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-06. [2025-10-20T17:54:30.651396] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-08. [2025-10-20T17:54:30.831677] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:54:30.833962] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T17:54:31.451375] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-06. [2025-10-20T17:54:31.541568] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T17:54:32.251400] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-07. [2025-10-20T17:54:33.051354] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-09. [2025-10-20T17:54:33.341745] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T17:54:33.434355] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T17:54:33.851409] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-07. [2025-10-20T17:54:34.087916] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T17:54:34.651493] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-08. [2025-10-20T17:54:35.451455] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-08. [2025-10-20T17:54:35.486836] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T17:54:35.598099] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T17:54:36.251369] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-09. [2025-10-20T17:54:36.970712] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:54:37.051410] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-09. [2025-10-20T17:54:37.330434] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T17:54:37.851418] [INFO ] [REMOTE ] CHE: Fetching primary data for 2025-10. [2025-10-20T17:54:38.651376] [INFO ] [REMOTE ] CHCT: Fetching primary data for 2025-10. [2025-10-20T17:54:38.889660] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T17:54:39.000454] [INFO ] [REMOTE ] CHE: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T17:54:39.003491] [INFO ] [REMOTE ] CHE: Full fetch complete (20409 records). [2025-10-20T17:54:39.003804] [DEBUG] [VALIDATOR ] CHE: Validating primary data. [2025-10-20T17:54:39.369801] [DEBUG] [VALIDATOR ] CHE: All expected trading days are present. [2025-10-20T17:54:39.369891] [DEBUG] [VALIDATOR ] CHE: Primary data validation passed. [2025-10-20T17:54:39.370298] [INFO ] [MAIN ] CHE: Fetched primary data is valid. [2025-10-20T17:54:39.370338] [DEBUG] [LOCAL ] CHE: Saving primary data. [2025-10-20T17:54:39.370919] [DEBUG] [LOCAL ] CHE: Trimmed 20409 records to 15304 (keeping 200 days of data) [2025-10-20T17:54:39.451722] [INFO ] [REMOTE ] CHD: Fetching primary data for 2025-10. [2025-10-20T17:54:39.465179] [DEBUG] [LOCAL ] CHE: Primary data saved (15304 records). [2025-10-20T17:54:39.466710] [INFO ] [MAIN ] CHE: Full history saved. [2025-10-20T17:54:39.466817] [INFO ] [MAIN ] Update Progress: 433/2367 symbols completed. CHE took 28.5s. Estimated remaining: 4h 3m 48s. Rate: 7.9 symbols/min [2025-10-20T17:54:39.638828] [INFO ] [MAIN ] CHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:39.669387] [DEBUG] [LOCAL ] CHH: Loaded 32905 existing records from primary storage. [2025-10-20T17:54:39.669698] [INFO ] [MAIN ] CHH: Loaded 32905 existing records. [2025-10-20T17:54:39.844162] [WARN ] [MAIN ] CHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:39.844362] [INFO ] [MAIN ] CHH: Wiping existing primary data before full refresh. [2025-10-20T17:54:39.844449] [INFO ] [LOCAL ] CHH: Starting deletion of primary data. [2025-10-20T17:54:39.844743] [DEBUG] [LOCAL ] CHH: Deleted primary data file. [2025-10-20T17:54:39.844806] [INFO ] [MAIN ] CHH: Starting full primary refresh. [2025-10-20T17:54:40.201900] [INFO ] [REMOTE ] CHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:40.251346] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-01. [2025-10-20T17:54:40.275288] [INFO ] [REMOTE ] CHCT: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T17:54:40.279531] [INFO ] [REMOTE ] CHCT: Full fetch complete (34788 records). [2025-10-20T17:54:40.279609] [DEBUG] [VALIDATOR ] CHCT: Validating primary data. [2025-10-20T17:54:40.652203] [DEBUG] [VALIDATOR ] CHCT: All expected trading days are present. [2025-10-20T17:54:40.652289] [DEBUG] [VALIDATOR ] CHCT: Primary data validation passed. [2025-10-20T17:54:40.652890] [INFO ] [MAIN ] CHCT: Fetched primary data is valid. [2025-10-20T17:54:40.652952] [DEBUG] [LOCAL ] CHCT: Saving primary data. [2025-10-20T17:54:40.653814] [DEBUG] [LOCAL ] CHCT: Trimmed 34788 records to 24343 (keeping 200 days of data) [2025-10-20T17:54:40.799417] [DEBUG] [LOCAL ] CHCT: Primary data saved (24343 records). [2025-10-20T17:54:40.799560] [INFO ] [MAIN ] CHCT: Full history saved. [2025-10-20T17:54:40.799661] [INFO ] [MAIN ] Update Progress: 434/2367 symbols completed. CHCT took 38.5s. Estimated remaining: 4h 3m 13s. Rate: 7.9 symbols/min [2025-10-20T17:54:40.973719] [INFO ] [MAIN ] CHMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:40.994714] [DEBUG] [LOCAL ] CHMI: Loaded 21855 existing records from primary storage. [2025-10-20T17:54:40.994965] [INFO ] [MAIN ] CHMI: Loaded 21855 existing records. [2025-10-20T17:54:41.059311] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-08. [2025-10-20T17:54:41.181780] [WARN ] [MAIN ] CHMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:41.181982] [INFO ] [MAIN ] CHMI: Wiping existing primary data before full refresh. [2025-10-20T17:54:41.182028] [INFO ] [LOCAL ] CHMI: Starting deletion of primary data. [2025-10-20T17:54:41.182195] [DEBUG] [LOCAL ] CHMI: Deleted primary data file. [2025-10-20T17:54:41.182232] [INFO ] [MAIN ] CHMI: Starting full primary refresh. [2025-10-20T17:54:41.598832] [INFO ] [REMOTE ] CHMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:41.603676] [INFO ] [REMOTE ] CHD: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T17:54:41.616618] [INFO ] [REMOTE ] CHD: Full fetch complete (74954 records). [2025-10-20T17:54:41.616909] [DEBUG] [VALIDATOR ] CHD: Validating primary data. [2025-10-20T17:54:41.747277] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T17:54:41.855383] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-02. [2025-10-20T17:54:42.096870] [DEBUG] [VALIDATOR ] CHD: All expected trading days are present. [2025-10-20T17:54:42.096988] [DEBUG] [VALIDATOR ] CHD: Primary data validation passed. [2025-10-20T17:54:42.098019] [INFO ] [MAIN ] CHD: Fetched primary data is valid. [2025-10-20T17:54:42.098092] [DEBUG] [LOCAL ] CHD: Saving primary data. [2025-10-20T17:54:42.100140] [DEBUG] [LOCAL ] CHD: Trimmed 74954 records to 52329 (keeping 200 days of data) [2025-10-20T17:54:42.443682] [DEBUG] [LOCAL ] CHD: Primary data saved (52329 records). [2025-10-20T17:54:42.443886] [INFO ] [MAIN ] CHD: Full history saved. [2025-10-20T17:54:42.444306] [INFO ] [MAIN ] Update Progress: 435/2367 symbols completed. CHD took 34.1s. Estimated remaining: 4h 2m 39s. Rate: 8.0 symbols/min [2025-10-20T17:54:42.630785] [INFO ] [MAIN ] CHN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:42.640980] [DEBUG] [LOCAL ] CHN: Loaded 5065 existing records from primary storage. [2025-10-20T17:54:42.641286] [INFO ] [MAIN ] CHN: Loaded 5065 existing records. [2025-10-20T17:54:42.657717] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-01. [2025-10-20T17:54:42.845949] [WARN ] [MAIN ] CHN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:42.846295] [INFO ] [MAIN ] CHN: Wiping existing primary data before full refresh. [2025-10-20T17:54:42.846355] [INFO ] [LOCAL ] CHN: Starting deletion of primary data. [2025-10-20T17:54:42.846502] [DEBUG] [LOCAL ] CHN: Deleted primary data file. [2025-10-20T17:54:42.846539] [INFO ] [MAIN ] CHN: Starting full primary refresh. [2025-10-20T17:54:43.193747] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T17:54:43.317068] [INFO ] [REMOTE ] CHN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:43.320785] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T17:54:43.451388] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-03. [2025-10-20T17:54:43.680120] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T17:54:44.251398] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-02. [2025-10-20T17:54:44.838265] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T17:54:44.897586] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T17:54:45.051360] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-04. [2025-10-20T17:54:45.851344] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-01. [2025-10-20T17:54:46.651361] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-03. [2025-10-20T17:54:46.653384] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T17:54:46.663726] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T17:54:47.451391] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-02. [2025-10-20T17:54:47.830010] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T17:54:47.968832] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T17:54:48.251451] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-09. [2025-10-20T17:54:49.051424] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-05. [2025-10-20T17:54:49.851380] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-03. [2025-10-20T17:54:50.109880] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T17:54:50.192994] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T17:54:50.304007] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T17:54:50.651376] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-06. [2025-10-20T17:54:51.451412] [INFO ] [REMOTE ] CHGG: Fetching primary data for 2025-10. [2025-10-20T17:54:52.071792] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T17:54:52.251429] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-04. [2025-10-20T17:54:52.704544] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T17:54:53.051442] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-07. [2025-10-20T17:54:53.176918] [INFO ] [REMOTE ] CHGG: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T17:54:53.183965] [INFO ] [REMOTE ] CHGG: Full fetch complete (70163 records). [2025-10-20T17:54:53.184076] [DEBUG] [VALIDATOR ] CHGG: Validating primary data. [2025-10-20T17:54:53.609503] [DEBUG] [VALIDATOR ] CHGG: All expected trading days are present. [2025-10-20T17:54:53.609602] [DEBUG] [VALIDATOR ] CHGG: Primary data validation passed. [2025-10-20T17:54:53.610542] [INFO ] [MAIN ] CHGG: Fetched primary data is valid. [2025-10-20T17:54:53.610623] [DEBUG] [LOCAL ] CHGG: Saving primary data. [2025-10-20T17:54:53.612257] [DEBUG] [LOCAL ] CHGG: Trimmed 70163 records to 47673 (keeping 200 days of data) [2025-10-20T17:54:53.852854] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-04. [2025-10-20T17:54:53.916539] [DEBUG] [LOCAL ] CHGG: Primary data saved (47673 records). [2025-10-20T17:54:53.916722] [INFO ] [MAIN ] CHGG: Full history saved. [2025-10-20T17:54:53.916821] [INFO ] [MAIN ] Update Progress: 436/2367 symbols completed. CHGG took 40.6s. Estimated remaining: 4h 2m 49s. Rate: 8.0 symbols/min [2025-10-20T17:54:54.087791] [INFO ] [MAIN ] CHPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:54:54.132817] [DEBUG] [LOCAL ] CHPT: Loaded 48210 existing records from primary storage. [2025-10-20T17:54:54.133152] [INFO ] [MAIN ] CHPT: Loaded 48210 existing records. [2025-10-20T17:54:54.312765] [WARN ] [MAIN ] CHPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:54:54.313274] [INFO ] [MAIN ] CHPT: Wiping existing primary data before full refresh. [2025-10-20T17:54:54.313388] [INFO ] [LOCAL ] CHPT: Starting deletion of primary data. [2025-10-20T17:54:54.313727] [DEBUG] [LOCAL ] CHPT: Deleted primary data file. [2025-10-20T17:54:54.313892] [INFO ] [MAIN ] CHPT: Starting full primary refresh. [2025-10-20T17:54:54.615500] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T17:54:54.656516] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-05. [2025-10-20T17:54:54.713818] [INFO ] [REMOTE ] CHPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:54:55.245633] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T17:54:55.405028] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T17:54:55.451408] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-05. [2025-10-20T17:54:56.249203] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T17:54:56.254041] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-08. [2025-10-20T17:54:57.051449] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-01. [2025-10-20T17:54:57.549795] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T17:54:57.851445] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-06. [2025-10-20T17:54:58.554664] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T17:54:58.651469] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-07. [2025-10-20T17:54:59.052515] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T17:54:59.223133] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T17:54:59.451390] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-08. [2025-10-20T17:54:59.890842] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T17:55:00.251412] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-09. [2025-10-20T17:55:00.920592] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T17:55:01.051450] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-02. [2025-10-20T17:55:01.851470] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-09. [2025-10-20T17:55:02.651450] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-06. [2025-10-20T17:55:03.364769] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T17:55:03.441071] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-02 (2.39 seconds). [2025-10-20T17:55:03.451332] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-03. [2025-10-20T17:55:03.757830] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T17:55:04.251383] [INFO ] [REMOTE ] CHH: Fetching primary data for 2025-10. [2025-10-20T17:55:05.051471] [INFO ] [REMOTE ] CHN: Fetching primary data for 2025-10. [2025-10-20T17:55:05.544545] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T17:55:05.604985] [INFO ] [REMOTE ] CHH: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T17:55:05.610691] [INFO ] [REMOTE ] CHH: Full fetch complete (47486 records). [2025-10-20T17:55:05.610796] [DEBUG] [VALIDATOR ] CHH: Validating primary data. [2025-10-20T17:55:05.805192] [INFO ] [REMOTE ] CHN: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T17:55:05.823621] [INFO ] [REMOTE ] CHN: Full fetch complete (7551 records). [2025-10-20T17:55:05.856707] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-04. [2025-10-20T17:55:05.862105] [DEBUG] [VALIDATOR ] CHN: Validating primary data. [2025-10-20T17:55:06.215944] [DEBUG] [VALIDATOR ] CHH: All expected trading days are present. [2025-10-20T17:55:06.231264] [DEBUG] [VALIDATOR ] CHH: Primary data validation passed. [2025-10-20T17:55:06.287494] [INFO ] [MAIN ] CHH: Fetched primary data is valid. [2025-10-20T17:55:06.311571] [DEBUG] [LOCAL ] CHH: Saving primary data. [2025-10-20T17:55:06.362170] [DEBUG] [LOCAL ] CHH: Trimmed 47486 records to 32329 (keeping 200 days of data) [2025-10-20T17:55:06.494225] [DEBUG] [VALIDATOR ] CHN: All expected trading days are present. [2025-10-20T17:55:06.519073] [DEBUG] [VALIDATOR ] CHN: Primary data validation passed. [2025-10-20T17:55:06.519560] [INFO ] [MAIN ] CHN: Fetched primary data is valid. [2025-10-20T17:55:06.520795] [DEBUG] [LOCAL ] CHN: Saving primary data. [2025-10-20T17:55:06.527509] [DEBUG] [LOCAL ] CHN: Trimmed 7551 records to 5002 (keeping 200 days of data) [2025-10-20T17:55:06.584531] [DEBUG] [LOCAL ] CHN: Primary data saved (5002 records). [2025-10-20T17:55:06.586969] [INFO ] [MAIN ] CHN: Full history saved. [2025-10-20T17:55:06.589680] [INFO ] [MAIN ] Update Progress: 437/2367 symbols completed. CHN took 24.1s. Estimated remaining: 4h 3m 4s. Rate: 7.9 symbols/min [2025-10-20T17:55:06.652190] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-07. [2025-10-20T17:55:06.666811] [DEBUG] [LOCAL ] CHH: Primary data saved (32329 records). [2025-10-20T17:55:06.666896] [INFO ] [MAIN ] CHH: Full history saved. [2025-10-20T17:55:06.666968] [INFO ] [MAIN ] Update Progress: 438/2367 symbols completed. CHH took 27.2s. Estimated remaining: 4h 2m 24s. Rate: 8.0 symbols/min [2025-10-20T17:55:07.832897] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T17:55:07.855877] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-05. [2025-10-20T17:55:07.856870] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T17:55:08.251420] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-08. [2025-10-20T17:55:08.838929] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T17:55:09.051466] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-09. [2025-10-20T17:55:10.165720] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-05 (2.31 seconds). [2025-10-20T17:55:10.178109] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-06. [2025-10-20T17:55:10.209782] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T17:55:10.651415] [INFO ] [REMOTE ] CHMI: Fetching primary data for 2025-10. [2025-10-20T17:55:12.031795] [INFO ] [REMOTE ] CHMI: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T17:55:12.035143] [INFO ] [REMOTE ] CHMI: Full fetch complete (31632 records). [2025-10-20T17:55:12.035213] [DEBUG] [VALIDATOR ] CHMI: Validating primary data. [2025-10-20T17:55:12.399808] [DEBUG] [VALIDATOR ] CHMI: All expected trading days are present. [2025-10-20T17:55:12.399897] [DEBUG] [VALIDATOR ] CHMI: Primary data validation passed. [2025-10-20T17:55:12.400384] [INFO ] [MAIN ] CHMI: Fetched primary data is valid. [2025-10-20T17:55:12.400423] [DEBUG] [LOCAL ] CHMI: Saving primary data. [2025-10-20T17:55:12.401129] [DEBUG] [LOCAL ] CHMI: Trimmed 31632 records to 21466 (keeping 200 days of data) [2025-10-20T17:55:12.519211] [DEBUG] [LOCAL ] CHMI: Primary data saved (21466 records). [2025-10-20T17:55:12.519327] [INFO ] [MAIN ] CHMI: Full history saved. [2025-10-20T17:55:12.519397] [INFO ] [MAIN ] Update Progress: 439/2367 symbols completed. CHMI took 31.7s. Estimated remaining: 4h 2m 9s. Rate: 8.0 symbols/min [2025-10-20T17:55:12.900697] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-06 (2.72 seconds). [2025-10-20T17:55:12.911366] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-07. [2025-10-20T17:55:14.984917] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T17:55:14.996452] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-08. [2025-10-20T17:55:16.521377] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T17:55:16.530352] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-09. [2025-10-20T17:55:18.011618] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T17:55:18.020127] [INFO ] [REMOTE ] CHPT: Fetching primary data for 2025-10. [2025-10-20T17:55:19.715711] [INFO ] [REMOTE ] CHPT: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T17:55:19.721686] [INFO ] [REMOTE ] CHPT: Full fetch complete (71419 records). [2025-10-20T17:55:19.721798] [DEBUG] [VALIDATOR ] CHPT: Validating primary data. [2025-10-20T17:55:20.119812] [DEBUG] [VALIDATOR ] CHPT: All expected trading days are present. [2025-10-20T17:55:20.119921] [DEBUG] [VALIDATOR ] CHPT: Primary data validation passed. [2025-10-20T17:55:20.120731] [INFO ] [MAIN ] CHPT: Fetched primary data is valid. [2025-10-20T17:55:20.120793] [DEBUG] [LOCAL ] CHPT: Saving primary data. [2025-10-20T17:55:20.122193] [DEBUG] [LOCAL ] CHPT: Trimmed 71419 records to 47293 (keeping 200 days of data) [2025-10-20T17:55:20.385283] [DEBUG] [LOCAL ] CHPT: Primary data saved (47293 records). [2025-10-20T17:55:20.385412] [INFO ] [MAIN ] CHPT: Full history saved. [2025-10-20T17:55:20.385499] [INFO ] [MAIN ] Update Progress: 440/2367 symbols completed. CHPT took 26.5s. Estimated remaining: 4h 2m 2s. Rate: 8.0 symbols/min [2025-10-20T17:55:21.010268] [INFO ] [MAIN ] CIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.024510] [INFO ] [MAIN ] CHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.088732] [INFO ] [MAIN ] CI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.092734] [INFO ] [MAIN ] CHWY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:21.115564] [DEBUG] [LOCAL ] CIA: Loaded 16820 existing records from primary storage. [2025-10-20T17:55:21.116184] [INFO ] [MAIN ] CIA: Loaded 16820 existing records. [2025-10-20T17:55:21.192416] [DEBUG] [LOCAL ] CHT: Loaded 19756 existing records from primary storage. [2025-10-20T17:55:21.215899] [INFO ] [MAIN ] CHT: Loaded 19756 existing records. [2025-10-20T17:55:21.408355] [WARN ] [MAIN ] CIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:21.443840] [INFO ] [MAIN ] CIA: Wiping existing primary data before full refresh. [2025-10-20T17:55:21.470197] [INFO ] [LOCAL ] CIA: Starting deletion of primary data. [2025-10-20T17:55:21.468828] [DEBUG] [LOCAL ] CI: Loaded 50399 existing records from primary storage. [2025-10-20T17:55:21.486042] [INFO ] [MAIN ] CI: Loaded 50399 existing records. [2025-10-20T17:55:21.539013] [DEBUG] [LOCAL ] CHWY: Loaded 54435 existing records from primary storage. [2025-10-20T17:55:21.538234] [DEBUG] [LOCAL ] CIA: Deleted primary data file. [2025-10-20T17:55:21.544382] [INFO ] [MAIN ] CIA: Starting full primary refresh. [2025-10-20T17:55:21.562480] [INFO ] [MAIN ] CHWY: Loaded 54435 existing records. [2025-10-20T17:55:21.645718] [WARN ] [MAIN ] CHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:21.781324] [INFO ] [MAIN ] CHT: Wiping existing primary data before full refresh. [2025-10-20T17:55:21.800103] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-10-20T17:55:21.851767] [DEBUG] [LOCAL ] CHT: Deleted primary data file. [2025-10-20T17:55:21.855013] [INFO ] [MAIN ] CHT: Starting full primary refresh. [2025-10-20T17:55:22.066317] [WARN ] [MAIN ] CI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:22.134204] [INFO ] [MAIN ] CI: Wiping existing primary data before full refresh. [2025-10-20T17:55:22.144586] [INFO ] [LOCAL ] CI: Starting deletion of primary data. [2025-10-20T17:55:22.148954] [WARN ] [MAIN ] CHWY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:22.188196] [DEBUG] [LOCAL ] CI: Deleted primary data file. [2025-10-20T17:55:22.181263] [INFO ] [MAIN ] CHWY: Wiping existing primary data before full refresh. [2025-10-20T17:55:22.239841] [INFO ] [MAIN ] CI: Starting full primary refresh. [2025-10-20T17:55:22.234636] [INFO ] [LOCAL ] CHWY: Starting deletion of primary data. [2025-10-20T17:55:22.282366] [DEBUG] [LOCAL ] CHWY: Deleted primary data file. [2025-10-20T17:55:22.419668] [INFO ] [MAIN ] CHWY: Starting full primary refresh. [2025-10-20T17:55:22.691948] [INFO ] [REMOTE ] CIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:22.800585] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-01. [2025-10-20T17:55:23.225180] [INFO ] [REMOTE ] CHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.262634] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-01. [2025-10-20T17:55:23.334894] [INFO ] [REMOTE ] CI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.347407] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-01. [2025-10-20T17:55:23.424340] [INFO ] [REMOTE ] CHWY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:23.425119] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-01. [2025-10-20T17:55:23.769423] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T17:55:23.774357] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-02. [2025-10-20T17:55:24.354423] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T17:55:24.358984] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-03. [2025-10-20T17:55:24.412521] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T17:55:24.416283] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-02. [2025-10-20T17:55:24.783386] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:55:24.793463] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-02. [2025-10-20T17:55:25.024710] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T17:55:25.028730] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-03. [2025-10-20T17:55:25.369972] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:55:25.373969] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-04. [2025-10-20T17:55:25.536931] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T17:55:25.547397] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-02. [2025-10-20T17:55:25.940953] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T17:55:25.946907] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-04. [2025-10-20T17:55:25.961090] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:55:25.965446] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-05. [2025-10-20T17:55:26.216813] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:55:26.230548] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-03. [2025-10-20T17:55:26.612790] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T17:55:26.616324] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-06. [2025-10-20T17:55:26.961985] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T17:55:26.972638] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-03. [2025-10-20T17:55:27.050635] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T17:55:27.451355] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-05. [2025-10-20T17:55:27.593788] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T17:55:28.003847] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:55:28.251461] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-07. [2025-10-20T17:55:28.536738] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T17:55:28.988548] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T17:55:29.012768] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:55:29.051329] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-08. [2025-10-20T17:55:29.851400] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-04. [2025-10-20T17:55:30.013527] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T17:55:30.651373] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-06. [2025-10-20T17:55:31.422533] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T17:55:31.451327] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-05. [2025-10-20T17:55:31.870399] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T17:55:32.251390] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-07. [2025-10-20T17:55:33.051388] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-09. [2025-10-20T17:55:33.191616] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:55:33.339361] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T17:55:33.851385] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-08. [2025-10-20T17:55:34.118603] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T17:55:34.651423] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-06. [2025-10-20T17:55:34.704632] [INFO ] [REMOTE ] CHT: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T17:55:35.451404] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-04. [2025-10-20T17:55:36.251443] [INFO ] [REMOTE ] CHT: Fetching primary data for 2025-09. [2025-10-20T17:55:36.358897] [DEBUG] [REMOTE ] CHT: Error fetching primary data for 2025-09: 'API call returned an empty response.' [2025-10-20T17:55:36.359229] [WARN ] [REMOTE ] CHT: No data returned for month 2025-09 [2025-10-20T17:55:36.359324] [FATAL] [REMOTE ] CHT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:55:36.359400] [FATAL] [MAIN ] CHT: Failed to fetch primary data. [2025-10-20T17:55:36.359462] [WARN ] [MAIN ] CHT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:55:36.359757] [INFO ] [LOCAL ] CHT: Starting deletion of primary data. [2025-10-20T17:55:36.359856] [DEBUG] [LOCAL ] CHT: No primary data file found to delete. [2025-10-20T17:55:36.359920] [INFO ] [LOCAL ] CHT: Starting deletion of derivative data. [2025-10-20T17:55:36.360129] [DEBUG] [LOCAL ] CHT: Deleted derivative data file. [2025-10-20T17:55:36.360214] [INFO ] [MAIN ] Update Progress: 441/2367 symbols completed. CHT took 16.0s. Estimated remaining: 4h 2m 32s. Rate: 7.9 symbols/min [2025-10-20T17:55:36.585676] [INFO ] [MAIN ] CIB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:36.623543] [DEBUG] [LOCAL ] CIB: Loaded 30952 existing records from primary storage. [2025-10-20T17:55:36.623992] [INFO ] [MAIN ] CIB: Loaded 30952 existing records. [2025-10-20T17:55:36.871959] [WARN ] [MAIN ] CIB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:36.872333] [INFO ] [MAIN ] CIB: Wiping existing primary data before full refresh. [2025-10-20T17:55:36.872520] [INFO ] [LOCAL ] CIB: Starting deletion of primary data. [2025-10-20T17:55:36.872827] [DEBUG] [LOCAL ] CIB: Deleted primary data file. [2025-10-20T17:55:36.872898] [INFO ] [MAIN ] CIB: Starting full primary refresh. [2025-10-20T17:55:37.056946] [INFO ] [REMOTE ] CIA: Fetching primary data for 2025-10. [2025-10-20T17:55:37.095142] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T17:55:37.438372] [INFO ] [REMOTE ] CIB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:37.470543] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T17:55:37.851414] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-05. [2025-10-20T17:55:38.091729] [INFO ] [REMOTE ] CIA: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T17:55:38.095147] [INFO ] [REMOTE ] CIA: Full fetch complete (23370 records). [2025-10-20T17:55:38.095274] [DEBUG] [VALIDATOR ] CIA: Validating primary data. [2025-10-20T17:55:38.538026] [DEBUG] [VALIDATOR ] CIA: All expected trading days are present. [2025-10-20T17:55:38.538139] [DEBUG] [VALIDATOR ] CIA: Primary data validation passed. [2025-10-20T17:55:38.538983] [INFO ] [MAIN ] CIA: Fetched primary data is valid. [2025-10-20T17:55:38.539050] [DEBUG] [LOCAL ] CIA: Saving primary data. [2025-10-20T17:55:38.540097] [DEBUG] [LOCAL ] CIA: Trimmed 23370 records to 16550 (keeping 200 days of data) [2025-10-20T17:55:38.641390] [DEBUG] [LOCAL ] CIA: Primary data saved (16550 records). [2025-10-20T17:55:38.641541] [INFO ] [MAIN ] CIA: Full history saved. [2025-10-20T17:55:38.641854] [INFO ] [MAIN ] Update Progress: 442/2367 symbols completed. CIA took 18.3s. Estimated remaining: 4h 2m 1s. Rate: 8.0 symbols/min [2025-10-20T17:55:38.656618] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-01. [2025-10-20T17:55:38.846619] [INFO ] [MAIN ] CICB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:38.846869] [DEBUG] [LOCAL ] CICB: No existing data file found in primary storage. [2025-10-20T17:55:38.846914] [WARN ] [MAIN ] CICB: No existing data found, performing full refresh instead. [2025-10-20T17:55:38.847034] [INFO ] [MAIN ] CICB: Wiping existing primary data before full refresh. [2025-10-20T17:55:38.847076] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-10-20T17:55:38.847119] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-10-20T17:55:38.847153] [INFO ] [MAIN ] CICB: Starting full primary refresh. [2025-10-20T17:55:39.234268] [INFO ] [REMOTE ] CICB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:39.451366] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-01. [2025-10-20T17:55:39.727789] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T17:55:39.900402] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T17:55:40.178771] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T17:55:40.251466] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-02. [2025-10-20T17:55:41.051452] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-07. [2025-10-20T17:55:41.497459] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T17:55:41.851447] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-03. [2025-10-20T17:55:42.651442] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-06. [2025-10-20T17:55:42.854579] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T17:55:43.120694] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T17:55:43.451397] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-02. [2025-10-20T17:55:43.720128] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-02 (0.27 seconds). [2025-10-20T17:55:43.906319] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T17:55:44.251366] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-03. [2025-10-20T17:55:44.650070] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T17:55:45.051415] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-04. [2025-10-20T17:55:45.851454] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-07. [2025-10-20T17:55:46.654875] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-08. [2025-10-20T17:55:46.665261] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T17:55:47.451390] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-05. [2025-10-20T17:55:47.742303] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T17:55:48.251410] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-08. [2025-10-20T17:55:48.520635] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T17:55:48.562831] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T17:55:49.051420] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-09. [2025-10-20T17:55:49.851449] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-06. [2025-10-20T17:55:49.993618] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T17:55:50.651431] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-09. [2025-10-20T17:55:50.811165] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T17:55:51.038058] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T17:55:51.451377] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-04. [2025-10-20T17:55:51.869530] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T17:55:52.251373] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-07. [2025-10-20T17:55:52.394979] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T17:55:53.040054] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T17:55:53.051345] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-08. [2025-10-20T17:55:53.851436] [INFO ] [REMOTE ] CI: Fetching primary data for 2025-10. [2025-10-20T17:55:54.362474] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T17:55:54.488269] [INFO ] [REMOTE ] CI: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T17:55:54.496488] [INFO ] [REMOTE ] CI: Full fetch complete (72106 records). [2025-10-20T17:55:54.496764] [DEBUG] [VALIDATOR ] CI: Validating primary data. [2025-10-20T17:55:54.656578] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-09. [2025-10-20T17:55:54.911289] [DEBUG] [VALIDATOR ] CI: All expected trading days are present. [2025-10-20T17:55:54.911386] [DEBUG] [VALIDATOR ] CI: Primary data validation passed. [2025-10-20T17:55:54.912228] [INFO ] [MAIN ] CI: Fetched primary data is valid. [2025-10-20T17:55:54.912298] [DEBUG] [LOCAL ] CI: Saving primary data. [2025-10-20T17:55:54.913738] [DEBUG] [LOCAL ] CI: Trimmed 72106 records to 49691 (keeping 200 days of data) [2025-10-20T17:55:55.214398] [DEBUG] [LOCAL ] CI: Primary data saved (49691 records). [2025-10-20T17:55:55.214530] [INFO ] [MAIN ] CI: Full history saved. [2025-10-20T17:55:55.214608] [INFO ] [MAIN ] Update Progress: 443/2367 symbols completed. CI took 34.8s. Estimated remaining: 4h 2m 33s. Rate: 7.9 symbols/min [2025-10-20T17:55:55.380200] [INFO ] [MAIN ] CIEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:55:55.426675] [DEBUG] [LOCAL ] CIEN: Loaded 52910 existing records from primary storage. [2025-10-20T17:55:55.426975] [INFO ] [MAIN ] CIEN: Loaded 52910 existing records. [2025-10-20T17:55:55.457601] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-05. [2025-10-20T17:55:55.604161] [WARN ] [MAIN ] CIEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:55:55.604358] [INFO ] [MAIN ] CIEN: Wiping existing primary data before full refresh. [2025-10-20T17:55:55.604405] [INFO ] [LOCAL ] CIEN: Starting deletion of primary data. [2025-10-20T17:55:55.604701] [DEBUG] [LOCAL ] CIEN: Deleted primary data file. [2025-10-20T17:55:55.604757] [INFO ] [MAIN ] CIEN: Starting full primary refresh. [2025-10-20T17:55:55.893034] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T17:55:55.976273] [INFO ] [REMOTE ] CIEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:55:56.026607] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T17:55:56.251431] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-01. [2025-10-20T17:55:57.051414] [INFO ] [REMOTE ] CHWY: Fetching primary data for 2025-10. [2025-10-20T17:55:57.851376] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-06. [2025-10-20T17:55:57.915524] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T17:55:58.336746] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T17:55:58.651452] [INFO ] [REMOTE ] CIB: Fetching primary data for 2025-10. [2025-10-20T17:55:59.115945] [INFO ] [REMOTE ] CHWY: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T17:55:59.124533] [INFO ] [REMOTE ] CHWY: Full fetch complete (77781 records). [2025-10-20T17:55:59.124722] [DEBUG] [VALIDATOR ] CHWY: Validating primary data. [2025-10-20T17:55:59.464758] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-07. [2025-10-20T17:55:59.526528] [DEBUG] [VALIDATOR ] CHWY: All expected trading days are present. [2025-10-20T17:55:59.526717] [DEBUG] [VALIDATOR ] CHWY: Primary data validation passed. [2025-10-20T17:55:59.527706] [INFO ] [MAIN ] CHWY: Fetched primary data is valid. [2025-10-20T17:55:59.527845] [DEBUG] [LOCAL ] CHWY: Saving primary data. [2025-10-20T17:55:59.529834] [DEBUG] [LOCAL ] CHWY: Trimmed 77781 records to 53656 (keeping 200 days of data) [2025-10-20T17:55:59.651956] [INFO ] [REMOTE ] CIB: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T17:55:59.661872] [INFO ] [REMOTE ] CIB: Full fetch complete (45339 records). [2025-10-20T17:55:59.662876] [DEBUG] [VALIDATOR ] CIB: Validating primary data. [2025-10-20T17:55:59.911691] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T17:56:00.231821] [DEBUG] [VALIDATOR ] CIB: All expected trading days are present. [2025-10-20T17:56:00.232246] [DEBUG] [VALIDATOR ] CIB: Primary data validation passed. [2025-10-20T17:56:00.233572] [INFO ] [MAIN ] CIB: Fetched primary data is valid. [2025-10-20T17:56:00.233910] [DEBUG] [LOCAL ] CIB: Saving primary data. [2025-10-20T17:56:00.240044] [DEBUG] [LOCAL ] CIB: Trimmed 45339 records to 30372 (keeping 200 days of data) [2025-10-20T17:56:00.294701] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-02. [2025-10-20T17:56:00.454888] [DEBUG] [LOCAL ] CHWY: Primary data saved (53656 records). [2025-10-20T17:56:00.456898] [INFO ] [MAIN ] CHWY: Full history saved. [2025-10-20T17:56:00.457886] [INFO ] [MAIN ] Update Progress: 444/2367 symbols completed. CHWY took 40.1s. Estimated remaining: 4h 2m 15s. Rate: 7.9 symbols/min [2025-10-20T17:56:00.674827] [DEBUG] [LOCAL ] CIB: Primary data saved (30372 records). [2025-10-20T17:56:00.697964] [INFO ] [MAIN ] CIB: Full history saved. [2025-10-20T17:56:00.701418] [INFO ] [MAIN ] Update Progress: 445/2367 symbols completed. CIB took 24.3s. Estimated remaining: 4h 1m 36s. Rate: 8.0 symbols/min [2025-10-20T17:56:00.701058] [INFO ] [MAIN ] CIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:00.816118] [DEBUG] [LOCAL ] CIF: Loaded 4087 existing records from primary storage. [2025-10-20T17:56:00.837181] [INFO ] [MAIN ] CIF: Loaded 4087 existing records. [2025-10-20T17:56:00.987955] [INFO ] [MAIN ] CIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:01.056545] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-08. [2025-10-20T17:56:01.073041] [WARN ] [MAIN ] CIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:01.073462] [INFO ] [MAIN ] CIF: Wiping existing primary data before full refresh. [2025-10-20T17:56:01.073534] [INFO ] [LOCAL ] CIF: Starting deletion of primary data. [2025-10-20T17:56:01.073750] [DEBUG] [LOCAL ] CIF: Deleted primary data file. [2025-10-20T17:56:01.073808] [INFO ] [MAIN ] CIF: Starting full primary refresh. [2025-10-20T17:56:01.322273] [DEBUG] [LOCAL ] CIG: Loaded 45005 existing records from primary storage. [2025-10-20T17:56:01.355732] [INFO ] [MAIN ] CIG: Loaded 45005 existing records. [2025-10-20T17:56:01.653852] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T17:56:01.730146] [INFO ] [REMOTE ] CIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:01.752718] [WARN ] [MAIN ] CIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:01.754565] [INFO ] [MAIN ] CIG: Wiping existing primary data before full refresh. [2025-10-20T17:56:01.754633] [INFO ] [LOCAL ] CIG: Starting deletion of primary data. [2025-10-20T17:56:01.755297] [DEBUG] [LOCAL ] CIG: Deleted primary data file. [2025-10-20T17:56:01.755371] [INFO ] [MAIN ] CIG: Starting full primary refresh. [2025-10-20T17:56:01.856545] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-09. [2025-10-20T17:56:02.221243] [INFO ] [REMOTE ] CIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:02.264602] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-10-20T17:56:02.314201] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T17:56:02.651420] [INFO ] [REMOTE ] CICB: Fetching primary data for 2025-10. [2025-10-20T17:56:03.092595] [INFO ] [REMOTE ] CICB: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T17:56:03.093567] [INFO ] [REMOTE ] CICB: Full fetch complete (1970 records). [2025-10-20T17:56:03.093636] [DEBUG] [VALIDATOR ] CICB: Validating primary data. [2025-10-20T17:56:03.273359] [WARN ] [VALIDATOR ] CICB: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T17:56:03.273533] [FATAL] [MAIN ] CICB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:56:03.273598] [WARN ] [MAIN ] CICB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:56:03.273690] [INFO ] [LOCAL ] CICB: Starting deletion of primary data. [2025-10-20T17:56:03.273759] [DEBUG] [LOCAL ] CICB: No primary data file found to delete. [2025-10-20T17:56:03.273795] [INFO ] [LOCAL ] CICB: Starting deletion of derivative data. [2025-10-20T17:56:03.273833] [DEBUG] [LOCAL ] CICB: No derivative data file found to delete. [2025-10-20T17:56:03.273886] [INFO ] [MAIN ] Update Progress: 446/2367 symbols completed. CICB took 24.6s. Estimated remaining: 4h 1m 7s. Rate: 8.0 symbols/min [2025-10-20T17:56:03.443942] [INFO ] [MAIN ] CII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:03.453428] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-01. [2025-10-20T17:56:03.461889] [DEBUG] [LOCAL ] CII: Loaded 15546 existing records from primary storage. [2025-10-20T17:56:03.462331] [INFO ] [MAIN ] CII: Loaded 15546 existing records. [2025-10-20T17:56:03.639440] [WARN ] [MAIN ] CII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:03.639664] [INFO ] [MAIN ] CII: Wiping existing primary data before full refresh. [2025-10-20T17:56:03.639724] [INFO ] [LOCAL ] CII: Starting deletion of primary data. [2025-10-20T17:56:03.639918] [DEBUG] [LOCAL ] CII: Deleted primary data file. [2025-10-20T17:56:03.639973] [INFO ] [MAIN ] CII: Starting full primary refresh. [2025-10-20T17:56:03.992544] [INFO ] [REMOTE ] CII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:04.251388] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-01. [2025-10-20T17:56:04.981750] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T17:56:05.015850] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T17:56:05.051329] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-02. [2025-10-20T17:56:05.851378] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-01. [2025-10-20T17:56:06.369854] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T17:56:06.651405] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-03. [2025-10-20T17:56:06.679638] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T17:56:07.451385] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-02. [2025-10-20T17:56:08.251377] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-03. [2025-10-20T17:56:09.051376] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-02. [2025-10-20T17:56:09.361058] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T17:56:09.851395] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-03. [2025-10-20T17:56:10.227090] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-03 (3.58 seconds). [2025-10-20T17:56:10.282708] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T17:56:10.284559] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:56:10.307423] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T17:56:10.651550] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-04. [2025-10-20T17:56:11.363710] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T17:56:11.451384] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-04. [2025-10-20T17:56:12.251424] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-03. [2025-10-20T17:56:13.051409] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-05. [2025-10-20T17:56:13.094446] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T17:56:13.322053] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T17:56:13.456860] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T17:56:13.851439] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-04. [2025-10-20T17:56:14.651638] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-06. [2025-10-20T17:56:15.151928] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T17:56:15.451378] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-05. [2025-10-20T17:56:15.710826] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T17:56:16.251400] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-05. [2025-10-20T17:56:16.876350] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T17:56:17.051391] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-07. [2025-10-20T17:56:17.663113] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T17:56:17.851405] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-08. [2025-10-20T17:56:17.923551] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T17:56:18.416113] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T17:56:18.651607] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-09. [2025-10-20T17:56:19.252910] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T17:56:19.451357] [INFO ] [REMOTE ] CIF: Fetching primary data for 2025-10. [2025-10-20T17:56:19.994479] [INFO ] [REMOTE ] CIF: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T17:56:19.995808] [INFO ] [REMOTE ] CIF: Full fetch complete (6576 records). [2025-10-20T17:56:19.995885] [DEBUG] [VALIDATOR ] CIF: Validating primary data. [2025-10-20T17:56:20.259545] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-04. [2025-10-20T17:56:20.386233] [DEBUG] [VALIDATOR ] CIF: All expected trading days are present. [2025-10-20T17:56:20.386338] [DEBUG] [VALIDATOR ] CIF: Primary data validation passed. [2025-10-20T17:56:20.386677] [INFO ] [MAIN ] CIF: Fetched primary data is valid. [2025-10-20T17:56:20.386743] [DEBUG] [LOCAL ] CIF: Saving primary data. [2025-10-20T17:56:20.387197] [DEBUG] [LOCAL ] CIF: Trimmed 6576 records to 4025 (keeping 200 days of data) [2025-10-20T17:56:20.411476] [DEBUG] [LOCAL ] CIF: Primary data saved (4025 records). [2025-10-20T17:56:20.411613] [INFO ] [MAIN ] CIF: Full history saved. [2025-10-20T17:56:20.411742] [INFO ] [MAIN ] Update Progress: 447/2367 symbols completed. CIF took 20.0s. Estimated remaining: 4h 1m 41s. Rate: 7.9 symbols/min [2025-10-20T17:56:20.596844] [INFO ] [MAIN ] CIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:20.638045] [DEBUG] [LOCAL ] CIM: Loaded 38246 existing records from primary storage. [2025-10-20T17:56:20.638533] [INFO ] [MAIN ] CIM: Loaded 38246 existing records. [2025-10-20T17:56:20.881973] [WARN ] [MAIN ] CIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:20.882306] [INFO ] [MAIN ] CIM: Wiping existing primary data before full refresh. [2025-10-20T17:56:20.882363] [INFO ] [LOCAL ] CIM: Starting deletion of primary data. [2025-10-20T17:56:20.883027] [DEBUG] [LOCAL ] CIM: Deleted primary data file. [2025-10-20T17:56:20.883088] [INFO ] [MAIN ] CIM: Starting full primary refresh. [2025-10-20T17:56:21.062571] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-06. [2025-10-20T17:56:21.296289] [INFO ] [REMOTE ] CIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:21.448972] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T17:56:21.851373] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-05. [2025-10-20T17:56:22.651375] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-01. [2025-10-20T17:56:22.806822] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T17:56:22.928908] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:56:23.451370] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-06. [2025-10-20T17:56:24.251410] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-06. [2025-10-20T17:56:24.776241] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T17:56:25.051397] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-07. [2025-10-20T17:56:25.815416] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T17:56:25.851407] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-07. [2025-10-20T17:56:26.200570] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-01 (3.55 seconds). [2025-10-20T17:56:26.388071] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-10-20T17:56:26.651434] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-07. [2025-10-20T17:56:27.386880] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T17:56:27.451394] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-08. [2025-10-20T17:56:28.251383] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-08. [2025-10-20T17:56:28.359496] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T17:56:28.385791] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T17:56:29.051416] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-09. [2025-10-20T17:56:29.851369] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-02. [2025-10-20T17:56:29.900100] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:56:30.058452] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T17:56:30.651387] [INFO ] [REMOTE ] CII: Fetching primary data for 2025-10. [2025-10-20T17:56:30.972206] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T17:56:31.451367] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-08. [2025-10-20T17:56:31.715933] [INFO ] [REMOTE ] CII: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T17:56:31.720612] [INFO ] [REMOTE ] CII: Full fetch complete (22942 records). [2025-10-20T17:56:31.720824] [DEBUG] [VALIDATOR ] CII: Validating primary data. [2025-10-20T17:56:32.150678] [DEBUG] [VALIDATOR ] CII: All expected trading days are present. [2025-10-20T17:56:32.150792] [DEBUG] [VALIDATOR ] CII: Primary data validation passed. [2025-10-20T17:56:32.151219] [INFO ] [MAIN ] CII: Fetched primary data is valid. [2025-10-20T17:56:32.151259] [DEBUG] [LOCAL ] CII: Saving primary data. [2025-10-20T17:56:32.151926] [DEBUG] [LOCAL ] CII: Trimmed 22942 records to 15328 (keeping 200 days of data) [2025-10-20T17:56:32.244616] [DEBUG] [LOCAL ] CII: Primary data saved (15328 records). [2025-10-20T17:56:32.244789] [INFO ] [MAIN ] CII: Full history saved. [2025-10-20T17:56:32.244873] [INFO ] [MAIN ] Update Progress: 448/2367 symbols completed. CII took 29.0s. Estimated remaining: 4h 1m 52s. Rate: 7.9 symbols/min [2025-10-20T17:56:32.257930] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-09. [2025-10-20T17:56:32.419571] [INFO ] [MAIN ] CIMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:32.419769] [DEBUG] [LOCAL ] CIMN: No existing data file found in primary storage. [2025-10-20T17:56:32.419814] [WARN ] [MAIN ] CIMN: No existing data found, performing full refresh instead. [2025-10-20T17:56:32.419927] [INFO ] [MAIN ] CIMN: Wiping existing primary data before full refresh. [2025-10-20T17:56:32.419969] [INFO ] [LOCAL ] CIMN: Starting deletion of primary data. [2025-10-20T17:56:32.420008] [DEBUG] [LOCAL ] CIMN: No primary data file found to delete. [2025-10-20T17:56:32.420039] [INFO ] [MAIN ] CIMN: Starting full primary refresh. [2025-10-20T17:56:32.782824] [INFO ] [REMOTE ] CIMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:33.004958] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T17:56:33.051437] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-01. [2025-10-20T17:56:33.690863] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T17:56:33.851373] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-03. [2025-10-20T17:56:34.032089] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T17:56:34.651475] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-02. [2025-10-20T17:56:35.268433] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T17:56:35.305549] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T17:56:35.451384] [INFO ] [REMOTE ] CIG: Fetching primary data for 2025-10. [2025-10-20T17:56:36.251401] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-09. [2025-10-20T17:56:36.869941] [INFO ] [REMOTE ] CIG: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T17:56:36.879575] [INFO ] [REMOTE ] CIG: Full fetch complete (62509 records). [2025-10-20T17:56:36.879867] [DEBUG] [VALIDATOR ] CIG: Validating primary data. [2025-10-20T17:56:37.057487] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-04. [2025-10-20T17:56:37.318776] [DEBUG] [VALIDATOR ] CIG: All expected trading days are present. [2025-10-20T17:56:37.318879] [DEBUG] [VALIDATOR ] CIG: Primary data validation passed. [2025-10-20T17:56:37.319715] [INFO ] [MAIN ] CIG: Fetched primary data is valid. [2025-10-20T17:56:37.319784] [DEBUG] [LOCAL ] CIG: Saving primary data. [2025-10-20T17:56:37.321089] [DEBUG] [LOCAL ] CIG: Trimmed 62509 records to 44324 (keeping 200 days of data) [2025-10-20T17:56:37.570443] [DEBUG] [LOCAL ] CIG: Primary data saved (44324 records). [2025-10-20T17:56:37.570570] [INFO ] [MAIN ] CIG: Full history saved. [2025-10-20T17:56:37.570673] [INFO ] [MAIN ] Update Progress: 449/2367 symbols completed. CIG took 36.9s. Estimated remaining: 4h 1m 35s. Rate: 7.9 symbols/min [2025-10-20T17:56:37.758325] [INFO ] [MAIN ] CIMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:37.758488] [DEBUG] [LOCAL ] CIMO: No existing data file found in primary storage. [2025-10-20T17:56:37.758529] [WARN ] [MAIN ] CIMO: No existing data found, performing full refresh instead. [2025-10-20T17:56:37.758666] [INFO ] [MAIN ] CIMO: Wiping existing primary data before full refresh. [2025-10-20T17:56:37.758726] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-10-20T17:56:37.758773] [DEBUG] [LOCAL ] CIMO: No primary data file found to delete. [2025-10-20T17:56:37.758812] [INFO ] [MAIN ] CIMO: Starting full primary refresh. [2025-10-20T17:56:37.856523] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-03. [2025-10-20T17:56:38.147189] [INFO ] [REMOTE ] CIMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:38.192368] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:56:38.224613] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T17:56:38.622394] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T17:56:38.651412] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-01. [2025-10-20T17:56:39.451402] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-05. [2025-10-20T17:56:39.572754] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T17:56:40.251428] [INFO ] [REMOTE ] CIEN: Fetching primary data for 2025-10. [2025-10-20T17:56:41.051367] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-04. [2025-10-20T17:56:41.637820] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T17:56:41.851381] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-05. [2025-10-20T17:56:41.893770] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-05 (2.44 seconds). [2025-10-20T17:56:42.459051] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T17:56:42.651369] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-06. [2025-10-20T17:56:42.984594] [INFO ] [REMOTE ] CIEN: Incremental fetch complete for 2025-10 (2.73 seconds). [2025-10-20T17:56:42.993522] [INFO ] [REMOTE ] CIEN: Full fetch complete (75839 records). [2025-10-20T17:56:42.993964] [DEBUG] [VALIDATOR ] CIEN: Validating primary data. [2025-10-20T17:56:43.432160] [DEBUG] [VALIDATOR ] CIEN: All expected trading days are present. [2025-10-20T17:56:43.432283] [DEBUG] [VALIDATOR ] CIEN: Primary data validation passed. [2025-10-20T17:56:43.433482] [INFO ] [MAIN ] CIEN: Fetched primary data is valid. [2025-10-20T17:56:43.433552] [DEBUG] [LOCAL ] CIEN: Saving primary data. [2025-10-20T17:56:43.435858] [DEBUG] [LOCAL ] CIEN: Trimmed 75839 records to 52129 (keeping 200 days of data) [2025-10-20T17:56:43.487282] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-02. [2025-10-20T17:56:43.802255] [DEBUG] [LOCAL ] CIEN: Primary data saved (52129 records). [2025-10-20T17:56:43.802481] [INFO ] [MAIN ] CIEN: Full history saved. [2025-10-20T17:56:43.802591] [INFO ] [MAIN ] Update Progress: 450/2367 symbols completed. CIEN took 48.6s. Estimated remaining: 4h 1m 21s. Rate: 7.9 symbols/min [2025-10-20T17:56:43.901337] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T17:56:44.023251] [INFO ] [MAIN ] CIMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:44.042889] [DEBUG] [LOCAL ] CIMP: No existing data file found in primary storage. [2025-10-20T17:56:44.044268] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T17:56:44.044398] [WARN ] [MAIN ] CIMP: No existing data found, performing full refresh instead. [2025-10-20T17:56:44.052421] [INFO ] [MAIN ] CIMP: Wiping existing primary data before full refresh. [2025-10-20T17:56:44.052896] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-10-20T17:56:44.052973] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-10-20T17:56:44.053014] [INFO ] [MAIN ] CIMP: Starting full primary refresh. [2025-10-20T17:56:44.260051] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-06. [2025-10-20T17:56:44.476430] [INFO ] [REMOTE ] CIMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:44.879173] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T17:56:45.051393] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-03. [2025-10-20T17:56:45.522481] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T17:56:45.851386] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-07. [2025-10-20T17:56:46.651396] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-04. [2025-10-20T17:56:47.230276] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T17:56:47.262044] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T17:56:47.451388] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-05. [2025-10-20T17:56:47.772760] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-05 (0.32 seconds). [2025-10-20T17:56:48.251391] [INFO ] [REMOTE ] CIMP: Fetching primary data for 2025-01. [2025-10-20T17:56:48.469734] [DEBUG] [REMOTE ] CIMP: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:56:48.469987] [WARN ] [REMOTE ] CIMP: No data returned for month 2025-01 [2025-10-20T17:56:48.470074] [FATAL] [REMOTE ] CIMP: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:56:48.470149] [FATAL] [MAIN ] CIMP: Failed to fetch primary data. [2025-10-20T17:56:48.470213] [WARN ] [MAIN ] CIMP: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:56:48.470285] [INFO ] [LOCAL ] CIMP: Starting deletion of primary data. [2025-10-20T17:56:48.470366] [DEBUG] [LOCAL ] CIMP: No primary data file found to delete. [2025-10-20T17:56:48.470437] [INFO ] [LOCAL ] CIMP: Starting deletion of derivative data. [2025-10-20T17:56:48.470481] [DEBUG] [LOCAL ] CIMP: No derivative data file found to delete. [2025-10-20T17:56:48.470526] [INFO ] [MAIN ] Update Progress: 451/2367 symbols completed. CIMP took 4.7s. Estimated remaining: 4h 1m 2s. Rate: 7.9 symbols/min [2025-10-20T17:56:48.637911] [INFO ] [MAIN ] CINT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:48.657576] [DEBUG] [LOCAL ] CINT: Loaded 20304 existing records from primary storage. [2025-10-20T17:56:48.657883] [INFO ] [MAIN ] CINT: Loaded 20304 existing records. [2025-10-20T17:56:48.836342] [WARN ] [MAIN ] CINT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:48.836542] [INFO ] [MAIN ] CINT: Wiping existing primary data before full refresh. [2025-10-20T17:56:48.836591] [INFO ] [LOCAL ] CINT: Starting deletion of primary data. [2025-10-20T17:56:48.836818] [DEBUG] [LOCAL ] CINT: Deleted primary data file. [2025-10-20T17:56:48.836876] [INFO ] [MAIN ] CINT: Starting full primary refresh. [2025-10-20T17:56:49.056514] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-07. [2025-10-20T17:56:49.201031] [INFO ] [REMOTE ] CINT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:49.544451] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T17:56:49.851370] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-06. [2025-10-20T17:56:50.433794] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T17:56:50.651422] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-07. [2025-10-20T17:56:51.115666] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T17:56:51.451381] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-08. [2025-10-20T17:56:51.935145] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T17:56:52.251370] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-09. [2025-10-20T17:56:52.887082] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T17:56:53.051389] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-01. [2025-10-20T17:56:53.851443] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-08. [2025-10-20T17:56:54.028162] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T17:56:54.651416] [INFO ] [REMOTE ] CIMO: Fetching primary data for 2025-10. [2025-10-20T17:56:55.321518] [INFO ] [REMOTE ] CIMO: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:56:55.323082] [INFO ] [REMOTE ] CIMO: Full fetch complete (4633 records). [2025-10-20T17:56:55.323175] [DEBUG] [VALIDATOR ] CIMO: Validating primary data. [2025-10-20T17:56:55.433701] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T17:56:55.460266] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-08. [2025-10-20T17:56:55.547525] [WARN ] [VALIDATOR ] CIMO: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T17:56:55.548135] [FATAL] [MAIN ] CIMO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T17:56:55.548205] [WARN ] [MAIN ] CIMO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T17:56:55.548278] [INFO ] [LOCAL ] CIMO: Starting deletion of primary data. [2025-10-20T17:56:55.548372] [DEBUG] [LOCAL ] CIMO: No primary data file found to delete. [2025-10-20T17:56:55.548429] [INFO ] [LOCAL ] CIMO: Starting deletion of derivative data. [2025-10-20T17:56:55.548473] [DEBUG] [LOCAL ] CIMO: No derivative data file found to delete. [2025-10-20T17:56:55.548531] [INFO ] [MAIN ] Update Progress: 452/2367 symbols completed. CIMO took 18.0s. Estimated remaining: 4h 0m 52s. Rate: 8.0 symbols/min [2025-10-20T17:56:55.722760] [INFO ] [MAIN ] CIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:56:55.742499] [DEBUG] [LOCAL ] CIO: Loaded 21338 existing records from primary storage. [2025-10-20T17:56:55.742789] [INFO ] [MAIN ] CIO: Loaded 21338 existing records. [2025-10-20T17:56:55.919690] [WARN ] [MAIN ] CIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:56:55.919895] [INFO ] [MAIN ] CIO: Wiping existing primary data before full refresh. [2025-10-20T17:56:55.919941] [INFO ] [LOCAL ] CIO: Starting deletion of primary data. [2025-10-20T17:56:55.920207] [DEBUG] [LOCAL ] CIO: Deleted primary data file. [2025-10-20T17:56:55.920264] [INFO ] [MAIN ] CIO: Starting full primary refresh. [2025-10-20T17:56:56.105577] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T17:56:56.254871] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-02. [2025-10-20T17:56:56.291427] [INFO ] [REMOTE ] CIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:56:57.051372] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-01. [2025-10-20T17:56:57.099664] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T17:56:57.851362] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-09. [2025-10-20T17:56:58.078579] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T17:56:58.651400] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-02. [2025-10-20T17:56:59.191149] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T17:56:59.451424] [INFO ] [REMOTE ] CIM: Fetching primary data for 2025-10. [2025-10-20T17:56:59.583440] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T17:57:00.251371] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-03. [2025-10-20T17:57:00.940216] [INFO ] [REMOTE ] CIM: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T17:57:00.946139] [INFO ] [REMOTE ] CIM: Full fetch complete (54902 records). [2025-10-20T17:57:00.946238] [DEBUG] [VALIDATOR ] CIM: Validating primary data. [2025-10-20T17:57:01.053770] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-03. [2025-10-20T17:57:01.356339] [DEBUG] [VALIDATOR ] CIM: All expected trading days are present. [2025-10-20T17:57:01.357083] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T17:57:01.357179] [DEBUG] [VALIDATOR ] CIM: Primary data validation passed. [2025-10-20T17:57:01.361674] [INFO ] [MAIN ] CIM: Fetched primary data is valid. [2025-10-20T17:57:01.362012] [DEBUG] [LOCAL ] CIM: Saving primary data. [2025-10-20T17:57:01.363006] [DEBUG] [LOCAL ] CIM: Trimmed 54902 records to 37626 (keeping 200 days of data) [2025-10-20T17:57:01.593260] [DEBUG] [LOCAL ] CIM: Primary data saved (37626 records). [2025-10-20T17:57:01.593405] [INFO ] [MAIN ] CIM: Full history saved. [2025-10-20T17:57:01.593484] [INFO ] [MAIN ] Update Progress: 453/2367 symbols completed. CIM took 41.2s. Estimated remaining: 4h 0m 38s. Rate: 8.0 symbols/min [2025-10-20T17:57:01.769957] [INFO ] [MAIN ] CION: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:01.795110] [DEBUG] [LOCAL ] CION: Loaded 28822 existing records from primary storage. [2025-10-20T17:57:01.795398] [INFO ] [MAIN ] CION: Loaded 28822 existing records. [2025-10-20T17:57:01.861150] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-04. [2025-10-20T17:57:02.015934] [WARN ] [MAIN ] CION: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:02.016453] [INFO ] [MAIN ] CION: Wiping existing primary data before full refresh. [2025-10-20T17:57:02.016530] [INFO ] [LOCAL ] CION: Starting deletion of primary data. [2025-10-20T17:57:02.016927] [DEBUG] [LOCAL ] CION: Deleted primary data file. [2025-10-20T17:57:02.016987] [INFO ] [MAIN ] CION: Starting full primary refresh. [2025-10-20T17:57:02.147159] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T17:57:02.412608] [INFO ] [REMOTE ] CION: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:02.651379] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-01. [2025-10-20T17:57:02.775580] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T17:57:03.451386] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-05. [2025-10-20T17:57:03.854328] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T17:57:04.251399] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-04. [2025-10-20T17:57:04.362138] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:57:04.976055] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T17:57:05.051385] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-09. [2025-10-20T17:57:05.537862] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T17:57:05.851427] [INFO ] [REMOTE ] CIMN: Fetching primary data for 2025-10. [2025-10-20T17:57:06.501802] [INFO ] [REMOTE ] CIMN: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T17:57:06.503631] [INFO ] [REMOTE ] CIMN: Full fetch complete (3934 records). [2025-10-20T17:57:06.504035] [DEBUG] [VALIDATOR ] CIMN: Validating primary data. [2025-10-20T17:57:06.656627] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-06. [2025-10-20T17:57:06.899936] [DEBUG] [VALIDATOR ] CIMN: All expected trading days are present. [2025-10-20T17:57:06.900038] [DEBUG] [VALIDATOR ] CIMN: Primary data validation passed. [2025-10-20T17:57:06.900201] [INFO ] [MAIN ] CIMN: Fetched primary data is valid. [2025-10-20T17:57:06.900242] [DEBUG] [LOCAL ] CIMN: Saving primary data. [2025-10-20T17:57:06.900636] [DEBUG] [LOCAL ] CIMN: Trimmed 3934 records to 2756 (keeping 200 days of data) [2025-10-20T17:57:06.918437] [DEBUG] [LOCAL ] CIMN: Primary data saved (2756 records). [2025-10-20T17:57:06.918573] [INFO ] [MAIN ] CIMN: Full history saved. [2025-10-20T17:57:06.918676] [INFO ] [MAIN ] Update Progress: 454/2367 symbols completed. CIMN took 34.7s. Estimated remaining: 4h 0m 21s. Rate: 8.0 symbols/min [2025-10-20T17:57:07.113051] [INFO ] [MAIN ] CIVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:07.165606] [DEBUG] [LOCAL ] CIVI: Loaded 53478 existing records from primary storage. [2025-10-20T17:57:07.166010] [INFO ] [MAIN ] CIVI: Loaded 53478 existing records. [2025-10-20T17:57:07.403145] [WARN ] [MAIN ] CIVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:07.403370] [INFO ] [MAIN ] CIVI: Wiping existing primary data before full refresh. [2025-10-20T17:57:07.403421] [INFO ] [LOCAL ] CIVI: Starting deletion of primary data. [2025-10-20T17:57:07.404050] [DEBUG] [LOCAL ] CIVI: Deleted primary data file. [2025-10-20T17:57:07.404106] [INFO ] [MAIN ] CIVI: Starting full primary refresh. [2025-10-20T17:57:07.461261] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-02. [2025-10-20T17:57:07.745496] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:57:07.815799] [INFO ] [REMOTE ] CIVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:08.251419] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-01. [2025-10-20T17:57:08.618254] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T17:57:09.051433] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-03. [2025-10-20T17:57:09.853438] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-07. [2025-10-20T17:57:09.857219] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T17:57:10.049828] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T17:57:10.651393] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-04. [2025-10-20T17:57:10.831361] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T17:57:11.451385] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-05. [2025-10-20T17:57:12.064845] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T17:57:12.251428] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-08. [2025-10-20T17:57:12.362925] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T17:57:13.057598] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-02. [2025-10-20T17:57:13.060151] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:57:13.851397] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-09. [2025-10-20T17:57:14.499349] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T17:57:14.651386] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-03. [2025-10-20T17:57:15.063114] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T17:57:15.451406] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-05. [2025-10-20T17:57:16.251424] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-06. [2025-10-20T17:57:16.541690] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T17:57:16.733152] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T17:57:16.971352] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T17:57:17.051407] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-06. [2025-10-20T17:57:17.851581] [INFO ] [REMOTE ] CINT: Fetching primary data for 2025-10. [2025-10-20T17:57:18.336534] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T17:57:18.651457] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-04. [2025-10-20T17:57:18.886809] [INFO ] [REMOTE ] CINT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T17:57:18.893830] [INFO ] [REMOTE ] CINT: Full fetch complete (28264 records). [2025-10-20T17:57:18.893996] [DEBUG] [VALIDATOR ] CINT: Validating primary data. [2025-10-20T17:57:19.265862] [DEBUG] [VALIDATOR ] CINT: All expected trading days are present. [2025-10-20T17:57:19.265935] [DEBUG] [VALIDATOR ] CINT: Primary data validation passed. [2025-10-20T17:57:19.266375] [INFO ] [MAIN ] CINT: Fetched primary data is valid. [2025-10-20T17:57:19.266413] [DEBUG] [LOCAL ] CINT: Saving primary data. [2025-10-20T17:57:19.267090] [DEBUG] [LOCAL ] CINT: Trimmed 28264 records to 20229 (keeping 200 days of data) [2025-10-20T17:57:19.374742] [DEBUG] [LOCAL ] CINT: Primary data saved (20229 records). [2025-10-20T17:57:19.374861] [INFO ] [MAIN ] CINT: Full history saved. [2025-10-20T17:57:19.374938] [INFO ] [MAIN ] Update Progress: 455/2367 symbols completed. CINT took 30.9s. Estimated remaining: 4h 0m 34s. Rate: 7.9 symbols/min [2025-10-20T17:57:19.456547] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-07. [2025-10-20T17:57:19.547827] [INFO ] [MAIN ] CL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:19.595930] [DEBUG] [LOCAL ] CL: Loaded 54429 existing records from primary storage. [2025-10-20T17:57:19.596254] [INFO ] [MAIN ] CL: Loaded 54429 existing records. [2025-10-20T17:57:19.772861] [WARN ] [MAIN ] CL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:19.773017] [INFO ] [MAIN ] CL: Wiping existing primary data before full refresh. [2025-10-20T17:57:19.773064] [INFO ] [LOCAL ] CL: Starting deletion of primary data. [2025-10-20T17:57:19.773351] [DEBUG] [LOCAL ] CL: Deleted primary data file. [2025-10-20T17:57:19.773410] [INFO ] [MAIN ] CL: Starting full primary refresh. [2025-10-20T17:57:20.131109] [INFO ] [REMOTE ] CL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:20.230823] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T17:57:20.251360] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-07. [2025-10-20T17:57:20.672882] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T17:57:21.051350] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-01. [2025-10-20T17:57:21.263741] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T17:57:21.851392] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-05. [2025-10-20T17:57:22.542220] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T17:57:22.651372] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-02. [2025-10-20T17:57:23.451408] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-08. [2025-10-20T17:57:23.561778] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:57:24.251389] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-06. [2025-10-20T17:57:24.404064] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T17:57:24.683447] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:57:25.051394] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-03. [2025-10-20T17:57:25.851420] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-09. [2025-10-20T17:57:25.947958] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T17:57:26.651475] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-07. [2025-10-20T17:57:26.992062] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T17:57:27.043886] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T17:57:27.451488] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-08. [2025-10-20T17:57:28.106266] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T17:57:28.251447] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-04. [2025-10-20T17:57:28.683973] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T17:57:29.051436] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-08. [2025-10-20T17:57:29.851407] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-09. [2025-10-20T17:57:29.965988] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T17:57:30.484673] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T17:57:30.651363] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-05. [2025-10-20T17:57:30.869017] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T17:57:31.451450] [INFO ] [REMOTE ] CIO: Fetching primary data for 2025-10. [2025-10-20T17:57:32.251364] [INFO ] [REMOTE ] CION: Fetching primary data for 2025-10. [2025-10-20T17:57:32.500144] [INFO ] [REMOTE ] CIO: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T17:57:32.505327] [INFO ] [REMOTE ] CIO: Full fetch complete (29666 records). [2025-10-20T17:57:32.505453] [DEBUG] [VALIDATOR ] CIO: Validating primary data. [2025-10-20T17:57:32.860448] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T17:57:32.959985] [DEBUG] [VALIDATOR ] CIO: All expected trading days are present. [2025-10-20T17:57:32.960070] [DEBUG] [VALIDATOR ] CIO: Primary data validation passed. [2025-10-20T17:57:32.960527] [INFO ] [MAIN ] CIO: Fetched primary data is valid. [2025-10-20T17:57:32.960566] [DEBUG] [LOCAL ] CIO: Saving primary data. [2025-10-20T17:57:32.961347] [DEBUG] [LOCAL ] CIO: Trimmed 29666 records to 21090 (keeping 200 days of data) [2025-10-20T17:57:33.060961] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-06. [2025-10-20T17:57:33.077947] [DEBUG] [LOCAL ] CIO: Primary data saved (21090 records). [2025-10-20T17:57:33.078091] [INFO ] [MAIN ] CIO: Full history saved. [2025-10-20T17:57:33.078170] [INFO ] [MAIN ] Update Progress: 456/2367 symbols completed. CIO took 37.5s. Estimated remaining: 4h 0m 53s. Rate: 7.9 symbols/min [2025-10-20T17:57:33.245662] [INFO ] [MAIN ] CLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:33.276324] [DEBUG] [LOCAL ] CLB: Loaded 33985 existing records from primary storage. [2025-10-20T17:57:33.276599] [INFO ] [MAIN ] CLB: Loaded 33985 existing records. [2025-10-20T17:57:33.455773] [WARN ] [MAIN ] CLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:33.456031] [INFO ] [MAIN ] CLB: Wiping existing primary data before full refresh. [2025-10-20T17:57:33.456210] [INFO ] [LOCAL ] CLB: Starting deletion of primary data. [2025-10-20T17:57:33.456622] [DEBUG] [LOCAL ] CLB: Deleted primary data file. [2025-10-20T17:57:33.456713] [INFO ] [MAIN ] CLB: Starting full primary refresh. [2025-10-20T17:57:33.581901] [INFO ] [REMOTE ] CION: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:57:33.633965] [INFO ] [REMOTE ] CION: Full fetch complete (40693 records). [2025-10-20T17:57:33.649375] [DEBUG] [VALIDATOR ] CION: Validating primary data. [2025-10-20T17:57:33.852248] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-09. [2025-10-20T17:57:33.969040] [INFO ] [REMOTE ] CLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:34.235976] [DEBUG] [VALIDATOR ] CION: All expected trading days are present. [2025-10-20T17:57:34.236083] [DEBUG] [VALIDATOR ] CION: Primary data validation passed. [2025-10-20T17:57:34.236794] [INFO ] [MAIN ] CION: Fetched primary data is valid. [2025-10-20T17:57:34.236857] [DEBUG] [LOCAL ] CION: Saving primary data. [2025-10-20T17:57:34.238131] [DEBUG] [LOCAL ] CION: Trimmed 40693 records to 28316 (keeping 200 days of data) [2025-10-20T17:57:34.413743] [DEBUG] [LOCAL ] CION: Primary data saved (28316 records). [2025-10-20T17:57:34.413931] [INFO ] [MAIN ] CION: Full history saved. [2025-10-20T17:57:34.414028] [INFO ] [MAIN ] Update Progress: 457/2367 symbols completed. CION took 32.8s. Estimated remaining: 4h 0m 19s. Rate: 7.9 symbols/min [2025-10-20T17:57:34.599145] [INFO ] [MAIN ] CLCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:34.615125] [DEBUG] [LOCAL ] CLCO: Loaded 13484 existing records from primary storage. [2025-10-20T17:57:34.615464] [INFO ] [MAIN ] CLCO: Loaded 13484 existing records. [2025-10-20T17:57:34.681723] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-01. [2025-10-20T17:57:34.836246] [WARN ] [MAIN ] CLCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:34.842535] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T17:57:34.853041] [INFO ] [MAIN ] CLCO: Wiping existing primary data before full refresh. [2025-10-20T17:57:34.853719] [INFO ] [LOCAL ] CLCO: Starting deletion of primary data. [2025-10-20T17:57:34.854356] [DEBUG] [LOCAL ] CLCO: Deleted primary data file. [2025-10-20T17:57:34.854437] [INFO ] [MAIN ] CLCO: Starting full primary refresh. [2025-10-20T17:57:35.251882] [INFO ] [REMOTE ] CLCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:35.451386] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-01. [2025-10-20T17:57:35.742582] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T17:57:36.072084] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:57:36.251399] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-02. [2025-10-20T17:57:36.639622] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T17:57:37.051388] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-02. [2025-10-20T17:57:37.261444] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T17:57:37.851358] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-03. [2025-10-20T17:57:38.002003] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T17:57:38.651399] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-03. [2025-10-20T17:57:38.801828] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T17:57:39.451447] [INFO ] [REMOTE ] CIVI: Fetching primary data for 2025-10. [2025-10-20T17:57:39.661825] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:57:40.251403] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-04. [2025-10-20T17:57:41.051415] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-04. [2025-10-20T17:57:41.320036] [INFO ] [REMOTE ] CIVI: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T17:57:41.329059] [INFO ] [REMOTE ] CIVI: Full fetch complete (76040 records). [2025-10-20T17:57:41.329318] [DEBUG] [VALIDATOR ] CIVI: Validating primary data. [2025-10-20T17:57:41.766400] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T17:57:41.841096] [DEBUG] [VALIDATOR ] CIVI: All expected trading days are present. [2025-10-20T17:57:41.841209] [DEBUG] [VALIDATOR ] CIVI: Primary data validation passed. [2025-10-20T17:57:41.842430] [INFO ] [MAIN ] CIVI: Fetched primary data is valid. [2025-10-20T17:57:41.842501] [DEBUG] [LOCAL ] CIVI: Saving primary data. [2025-10-20T17:57:41.844980] [DEBUG] [LOCAL ] CIVI: Trimmed 76040 records to 52736 (keeping 200 days of data) [2025-10-20T17:57:41.851340] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-05. [2025-10-20T17:57:42.098300] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T17:57:42.219123] [DEBUG] [LOCAL ] CIVI: Primary data saved (52736 records). [2025-10-20T17:57:42.219339] [INFO ] [MAIN ] CIVI: Full history saved. [2025-10-20T17:57:42.219451] [INFO ] [MAIN ] Update Progress: 458/2367 symbols completed. CIVI took 35.3s. Estimated remaining: 4h 0m 13s. Rate: 7.9 symbols/min [2025-10-20T17:57:42.397626] [INFO ] [MAIN ] CLDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:42.427685] [DEBUG] [LOCAL ] CLDT: Loaded 25824 existing records from primary storage. [2025-10-20T17:57:42.428191] [INFO ] [MAIN ] CLDT: Loaded 25824 existing records. [2025-10-20T17:57:42.623200] [WARN ] [MAIN ] CLDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:42.623532] [INFO ] [MAIN ] CLDT: Wiping existing primary data before full refresh. [2025-10-20T17:57:42.623701] [INFO ] [LOCAL ] CLDT: Starting deletion of primary data. [2025-10-20T17:57:42.624021] [DEBUG] [LOCAL ] CLDT: Deleted primary data file. [2025-10-20T17:57:42.624103] [INFO ] [MAIN ] CLDT: Starting full primary refresh. [2025-10-20T17:57:42.657827] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-05. [2025-10-20T17:57:43.009567] [INFO ] [REMOTE ] CLDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:43.233314] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T17:57:43.451351] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-06. [2025-10-20T17:57:43.523747] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T17:57:44.251334] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-06. [2025-10-20T17:57:44.664266] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T17:57:45.051369] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-07. [2025-10-20T17:57:45.115818] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T17:57:45.851482] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-07. [2025-10-20T17:57:46.651476] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-01. [2025-10-20T17:57:46.971159] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T17:57:47.427456] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T17:57:47.451881] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-07. [2025-10-20T17:57:47.454169] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T17:57:48.251407] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-08. [2025-10-20T17:57:48.325702] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T17:57:49.051461] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-08. [2025-10-20T17:57:49.378530] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T17:57:49.672212] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T17:57:49.851439] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-02. [2025-10-20T17:57:50.651426] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-09. [2025-10-20T17:57:50.678855] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T17:57:51.451424] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-03. [2025-10-20T17:57:51.995584] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T17:57:52.251370] [INFO ] [REMOTE ] CLB: Fetching primary data for 2025-10. [2025-10-20T17:57:52.707471] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T17:57:53.051422] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-08. [2025-10-20T17:57:53.653695] [INFO ] [REMOTE ] CLB: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T17:57:53.659596] [INFO ] [REMOTE ] CLB: Full fetch complete (47275 records). [2025-10-20T17:57:53.659783] [DEBUG] [VALIDATOR ] CLB: Validating primary data. [2025-10-20T17:57:53.853128] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-09. [2025-10-20T17:57:54.085836] [DEBUG] [VALIDATOR ] CLB: All expected trading days are present. [2025-10-20T17:57:54.085957] [DEBUG] [VALIDATOR ] CLB: Primary data validation passed. [2025-10-20T17:57:54.086920] [INFO ] [MAIN ] CLB: Fetched primary data is valid. [2025-10-20T17:57:54.087031] [DEBUG] [LOCAL ] CLB: Saving primary data. [2025-10-20T17:57:54.088614] [DEBUG] [LOCAL ] CLB: Trimmed 47275 records to 33606 (keeping 200 days of data) [2025-10-20T17:57:54.319820] [DEBUG] [LOCAL ] CLB: Primary data saved (33606 records). [2025-10-20T17:57:54.320017] [INFO ] [MAIN ] CLB: Full history saved. [2025-10-20T17:57:54.320411] [INFO ] [MAIN ] Update Progress: 459/2367 symbols completed. CLB took 21.2s. Estimated remaining: 4h 0m 24s. Rate: 7.9 symbols/min [2025-10-20T17:57:54.506789] [INFO ] [MAIN ] CLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:54.568743] [DEBUG] [LOCAL ] CLF: Loaded 54521 existing records from primary storage. [2025-10-20T17:57:54.569199] [INFO ] [MAIN ] CLF: Loaded 54521 existing records. [2025-10-20T17:57:54.666324] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-04. [2025-10-20T17:57:54.769864] [WARN ] [MAIN ] CLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:54.770435] [INFO ] [MAIN ] CLF: Wiping existing primary data before full refresh. [2025-10-20T17:57:54.770522] [INFO ] [LOCAL ] CLF: Starting deletion of primary data. [2025-10-20T17:57:54.770930] [DEBUG] [LOCAL ] CLF: Deleted primary data file. [2025-10-20T17:57:54.770992] [INFO ] [MAIN ] CLF: Starting full primary refresh. [2025-10-20T17:57:55.114388] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T17:57:55.204680] [INFO ] [REMOTE ] CLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:55.240061] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T17:57:55.451399] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-01. [2025-10-20T17:57:56.049314] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:57:56.251428] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-09. [2025-10-20T17:57:57.051406] [INFO ] [REMOTE ] CLCO: Fetching primary data for 2025-10. [2025-10-20T17:57:57.436990] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T17:57:57.851443] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-02. [2025-10-20T17:57:58.212076] [INFO ] [REMOTE ] CLCO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T17:57:58.217184] [INFO ] [REMOTE ] CLCO: Full fetch complete (22860 records). [2025-10-20T17:57:58.217452] [DEBUG] [VALIDATOR ] CLCO: Validating primary data. [2025-10-20T17:57:58.639677] [DEBUG] [VALIDATOR ] CLCO: All expected trading days are present. [2025-10-20T17:57:58.639847] [DEBUG] [VALIDATOR ] CLCO: Primary data validation passed. [2025-10-20T17:57:58.640622] [INFO ] [MAIN ] CLCO: Fetched primary data is valid. [2025-10-20T17:57:58.640876] [DEBUG] [LOCAL ] CLCO: Saving primary data. [2025-10-20T17:57:58.642066] [DEBUG] [LOCAL ] CLCO: Trimmed 22860 records to 13189 (keeping 200 days of data) [2025-10-20T17:57:58.660428] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-05. [2025-10-20T17:57:58.727958] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T17:57:58.783063] [DEBUG] [LOCAL ] CLCO: Primary data saved (13189 records). [2025-10-20T17:57:58.783282] [INFO ] [MAIN ] CLCO: Full history saved. [2025-10-20T17:57:58.783394] [INFO ] [MAIN ] Update Progress: 460/2367 symbols completed. CLCO took 24.4s. Estimated remaining: 4h 0m 4s. Rate: 7.9 symbols/min [2025-10-20T17:57:58.970234] [INFO ] [MAIN ] CLH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:57:59.003053] [DEBUG] [LOCAL ] CLH: Loaded 28288 existing records from primary storage. [2025-10-20T17:57:59.003488] [INFO ] [MAIN ] CLH: Loaded 28288 existing records. [2025-10-20T17:57:59.193595] [WARN ] [MAIN ] CLH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:57:59.193883] [INFO ] [MAIN ] CLH: Wiping existing primary data before full refresh. [2025-10-20T17:57:59.193932] [INFO ] [LOCAL ] CLH: Starting deletion of primary data. [2025-10-20T17:57:59.194182] [DEBUG] [LOCAL ] CLH: Deleted primary data file. [2025-10-20T17:57:59.194239] [INFO ] [MAIN ] CLH: Starting full primary refresh. [2025-10-20T17:57:59.460697] [INFO ] [REMOTE ] CL: Fetching primary data for 2025-10. [2025-10-20T17:57:59.573815] [INFO ] [REMOTE ] CLH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:57:59.886366] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T17:58:00.064784] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T17:58:00.251375] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-06. [2025-10-20T17:58:01.051439] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-01. [2025-10-20T17:58:01.290535] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T17:58:01.613542] [INFO ] [REMOTE ] CL: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T17:58:01.625626] [INFO ] [REMOTE ] CL: Full fetch complete (77817 records). [2025-10-20T17:58:01.625855] [DEBUG] [VALIDATOR ] CL: Validating primary data. [2025-10-20T17:58:01.859400] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-07. [2025-10-20T17:58:02.052737] [DEBUG] [VALIDATOR ] CL: All expected trading days are present. [2025-10-20T17:58:02.053054] [DEBUG] [VALIDATOR ] CL: Primary data validation passed. [2025-10-20T17:58:02.053987] [INFO ] [MAIN ] CL: Fetched primary data is valid. [2025-10-20T17:58:02.054055] [DEBUG] [LOCAL ] CL: Saving primary data. [2025-10-20T17:58:02.055702] [DEBUG] [LOCAL ] CL: Trimmed 77817 records to 53649 (keeping 200 days of data) [2025-10-20T17:58:02.414689] [DEBUG] [LOCAL ] CL: Primary data saved (53649 records). [2025-10-20T17:58:02.414853] [INFO ] [MAIN ] CL: Full history saved. [2025-10-20T17:58:02.414939] [INFO ] [MAIN ] Update Progress: 461/2367 symbols completed. CL took 43.0s. Estimated remaining: 3h 59m 40s. Rate: 8.0 symbols/min [2025-10-20T17:58:02.614398] [INFO ] [MAIN ] CLPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:02.614798] [DEBUG] [LOCAL ] CLPR: No existing data file found in primary storage. [2025-10-20T17:58:02.614928] [WARN ] [MAIN ] CLPR: No existing data found, performing full refresh instead. [2025-10-20T17:58:02.615170] [INFO ] [MAIN ] CLPR: Wiping existing primary data before full refresh. [2025-10-20T17:58:02.622376] [INFO ] [LOCAL ] CLPR: Starting deletion of primary data. [2025-10-20T17:58:02.622470] [DEBUG] [LOCAL ] CLPR: No primary data file found to delete. [2025-10-20T17:58:02.622542] [INFO ] [MAIN ] CLPR: Starting full primary refresh. [2025-10-20T17:58:02.657066] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-03. [2025-10-20T17:58:02.701610] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T17:58:03.037095] [INFO ] [REMOTE ] CLPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:03.050396] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T17:58:03.451384] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-01. [2025-10-20T17:58:04.253697] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-08. [2025-10-20T17:58:04.257492] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T17:58:04.365683] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T17:58:05.051343] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-02. [2025-10-20T17:58:05.335112] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T17:58:05.851429] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-09. [2025-10-20T17:58:06.474864] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T17:58:06.570210] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T17:58:06.651380] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-03. [2025-10-20T17:58:07.451368] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-02. [2025-10-20T17:58:08.028773] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T17:58:08.202127] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T17:58:08.251376] [INFO ] [REMOTE ] CLDT: Fetching primary data for 2025-10. [2025-10-20T17:58:09.051421] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-03. [2025-10-20T17:58:09.530852] [INFO ] [REMOTE ] CLDT: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T17:58:09.535213] [INFO ] [REMOTE ] CLDT: Full fetch complete (37558 records). [2025-10-20T17:58:09.535293] [DEBUG] [VALIDATOR ] CLDT: Validating primary data. [2025-10-20T17:58:09.856509] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-04. [2025-10-20T17:58:09.904117] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T17:58:09.980228] [DEBUG] [VALIDATOR ] CLDT: All expected trading days are present. [2025-10-20T17:58:09.980338] [DEBUG] [VALIDATOR ] CLDT: Primary data validation passed. [2025-10-20T17:58:09.980990] [INFO ] [MAIN ] CLDT: Fetched primary data is valid. [2025-10-20T17:58:09.981056] [DEBUG] [LOCAL ] CLDT: Saving primary data. [2025-10-20T17:58:09.982071] [DEBUG] [LOCAL ] CLDT: Trimmed 37558 records to 25365 (keeping 200 days of data) [2025-10-20T17:58:10.135699] [DEBUG] [LOCAL ] CLDT: Primary data saved (25365 records). [2025-10-20T17:58:10.135831] [INFO ] [MAIN ] CLDT: Full history saved. [2025-10-20T17:58:10.135908] [INFO ] [MAIN ] Update Progress: 462/2367 symbols completed. CLDT took 27.9s. Estimated remaining: 3h 59m 33s. Rate: 8.0 symbols/min [2025-10-20T17:58:10.302754] [INFO ] [MAIN ] CLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:10.302913] [DEBUG] [LOCAL ] CLS: No existing data file found in primary storage. [2025-10-20T17:58:10.302949] [WARN ] [MAIN ] CLS: No existing data found, performing full refresh instead. [2025-10-20T17:58:10.303056] [INFO ] [MAIN ] CLS: Wiping existing primary data before full refresh. [2025-10-20T17:58:10.303095] [INFO ] [LOCAL ] CLS: Starting deletion of primary data. [2025-10-20T17:58:10.303136] [DEBUG] [LOCAL ] CLS: No primary data file found to delete. [2025-10-20T17:58:10.303166] [INFO ] [MAIN ] CLS: Starting full primary refresh. [2025-10-20T17:58:10.653931] [INFO ] [REMOTE ] CLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:10.654040] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-04. [2025-10-20T17:58:11.451387] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-04. [2025-10-20T17:58:11.634858] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T17:58:12.256070] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-05. [2025-10-20T17:58:12.261925] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-04 (2.41 seconds). [2025-10-20T17:58:12.889705] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T17:58:12.998600] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T17:58:13.051334] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-06. [2025-10-20T17:58:13.761752] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T17:58:13.851412] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-05. [2025-10-20T17:58:14.651411] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-05. [2025-10-20T17:58:14.983812] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T17:58:15.451443] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-01. [2025-10-20T17:58:16.242234] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T17:58:16.253482] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-06. [2025-10-20T17:58:17.051456] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-07. [2025-10-20T17:58:17.321230] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T17:58:17.857247] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-06. [2025-10-20T17:58:17.871324] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-01 (2.42 seconds). [2025-10-20T17:58:17.994829] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T17:58:18.651383] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-07. [2025-10-20T17:58:19.451348] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-02. [2025-10-20T17:58:19.904126] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T17:58:19.955569] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T17:58:20.251447] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-08. [2025-10-20T17:58:21.051527] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-07. [2025-10-20T17:58:21.065841] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T17:58:21.582296] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T17:58:21.851415] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-03. [2025-10-20T17:58:22.651379] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-08. [2025-10-20T17:58:23.409335] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T17:58:23.451411] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-09. [2025-10-20T17:58:23.831512] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T17:58:23.919529] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T17:58:23.975510] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T17:58:24.251388] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-08. [2025-10-20T17:58:25.051408] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-04. [2025-10-20T17:58:25.851450] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-09. [2025-10-20T17:58:26.205425] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T17:58:26.651359] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-09. [2025-10-20T17:58:26.812062] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T17:58:27.451354] [INFO ] [REMOTE ] CLH: Fetching primary data for 2025-10. [2025-10-20T17:58:27.600239] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T17:58:28.251363] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-05. [2025-10-20T17:58:28.588529] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T17:58:28.784958] [INFO ] [REMOTE ] CLH: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T17:58:28.789508] [INFO ] [REMOTE ] CLH: Full fetch complete (42231 records). [2025-10-20T17:58:28.789588] [DEBUG] [VALIDATOR ] CLH: Validating primary data. [2025-10-20T17:58:29.056545] [INFO ] [REMOTE ] CLPR: Fetching primary data for 2025-10. [2025-10-20T17:58:29.188199] [DEBUG] [VALIDATOR ] CLH: All expected trading days are present. [2025-10-20T17:58:29.188280] [DEBUG] [VALIDATOR ] CLH: Primary data validation passed. [2025-10-20T17:58:29.188964] [INFO ] [MAIN ] CLH: Fetched primary data is valid. [2025-10-20T17:58:29.189026] [DEBUG] [LOCAL ] CLH: Saving primary data. [2025-10-20T17:58:29.190008] [DEBUG] [LOCAL ] CLH: Trimmed 42231 records to 27808 (keeping 200 days of data) [2025-10-20T17:58:29.353905] [DEBUG] [LOCAL ] CLH: Primary data saved (27808 records). [2025-10-20T17:58:29.354045] [INFO ] [MAIN ] CLH: Full history saved. [2025-10-20T17:58:29.354158] [INFO ] [MAIN ] Update Progress: 463/2367 symbols completed. CLH took 30.6s. Estimated remaining: 4h 0m 13s. Rate: 7.9 symbols/min [2025-10-20T17:58:29.519289] [INFO ] [MAIN ] CLVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:29.519451] [DEBUG] [LOCAL ] CLVT: No existing data file found in primary storage. [2025-10-20T17:58:29.519488] [WARN ] [MAIN ] CLVT: No existing data found, performing full refresh instead. [2025-10-20T17:58:29.519599] [INFO ] [MAIN ] CLVT: Wiping existing primary data before full refresh. [2025-10-20T17:58:29.519659] [INFO ] [LOCAL ] CLVT: Starting deletion of primary data. [2025-10-20T17:58:29.519727] [DEBUG] [LOCAL ] CLVT: No primary data file found to delete. [2025-10-20T17:58:29.519760] [INFO ] [MAIN ] CLVT: Starting full primary refresh. [2025-10-20T17:58:29.853127] [INFO ] [REMOTE ] CLF: Fetching primary data for 2025-10. [2025-10-20T17:58:29.874586] [INFO ] [REMOTE ] CLVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:30.064317] [INFO ] [REMOTE ] CLPR: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T17:58:30.065972] [INFO ] [REMOTE ] CLPR: Full fetch complete (16046 records). [2025-10-20T17:58:30.066042] [DEBUG] [VALIDATOR ] CLPR: Validating primary data. [2025-10-20T17:58:30.427914] [DEBUG] [VALIDATOR ] CLPR: All expected trading days are present. [2025-10-20T17:58:30.427995] [DEBUG] [VALIDATOR ] CLPR: Primary data validation passed. [2025-10-20T17:58:30.428372] [INFO ] [MAIN ] CLPR: Fetched primary data is valid. [2025-10-20T17:58:30.428439] [DEBUG] [LOCAL ] CLPR: Saving primary data. [2025-10-20T17:58:30.429037] [DEBUG] [LOCAL ] CLPR: Trimmed 16046 records to 10162 (keeping 200 days of data) [2025-10-20T17:58:30.487959] [DEBUG] [LOCAL ] CLPR: Primary data saved (10162 records). [2025-10-20T17:58:30.488085] [INFO ] [MAIN ] CLPR: Full history saved. [2025-10-20T17:58:30.488155] [INFO ] [MAIN ] Update Progress: 464/2367 symbols completed. CLPR took 28.1s. Estimated remaining: 3h 59m 39s. Rate: 7.9 symbols/min [2025-10-20T17:58:30.657181] [INFO ] [MAIN ] CLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:30.657431] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-01. [2025-10-20T17:58:30.678272] [DEBUG] [LOCAL ] CLW: Loaded 21809 existing records from primary storage. [2025-10-20T17:58:30.678569] [INFO ] [MAIN ] CLW: Loaded 21809 existing records. [2025-10-20T17:58:30.867058] [WARN ] [MAIN ] CLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:30.867481] [INFO ] [MAIN ] CLW: Wiping existing primary data before full refresh. [2025-10-20T17:58:30.867558] [INFO ] [LOCAL ] CLW: Starting deletion of primary data. [2025-10-20T17:58:30.867865] [DEBUG] [LOCAL ] CLW: Deleted primary data file. [2025-10-20T17:58:30.867927] [INFO ] [MAIN ] CLW: Starting full primary refresh. [2025-10-20T17:58:31.084037] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-05 (2.83 seconds). [2025-10-20T17:58:31.283852] [INFO ] [REMOTE ] CLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:31.451431] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-06. [2025-10-20T17:58:32.148007] [INFO ] [REMOTE ] CLF: Incremental fetch complete for 2025-10 (2.29 seconds). [2025-10-20T17:58:32.156962] [INFO ] [REMOTE ] CLF: Full fetch complete (77918 records). [2025-10-20T17:58:32.157145] [DEBUG] [VALIDATOR ] CLF: Validating primary data. [2025-10-20T17:58:32.258817] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-01. [2025-10-20T17:58:32.362854] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T17:58:32.609881] [DEBUG] [VALIDATOR ] CLF: All expected trading days are present. [2025-10-20T17:58:32.610039] [DEBUG] [VALIDATOR ] CLF: Primary data validation passed. [2025-10-20T17:58:32.610926] [INFO ] [MAIN ] CLF: Fetched primary data is valid. [2025-10-20T17:58:32.610990] [DEBUG] [LOCAL ] CLF: Saving primary data. [2025-10-20T17:58:32.612442] [DEBUG] [LOCAL ] CLF: Trimmed 77918 records to 53741 (keeping 200 days of data) [2025-10-20T17:58:32.915091] [DEBUG] [LOCAL ] CLF: Primary data saved (53741 records). [2025-10-20T17:58:32.915224] [INFO ] [MAIN ] CLF: Full history saved. [2025-10-20T17:58:32.915571] [INFO ] [MAIN ] Update Progress: 465/2367 symbols completed. CLF took 38.6s. Estimated remaining: 3h 59m 11s. Rate: 8.0 symbols/min [2025-10-20T17:58:33.061328] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-02. [2025-10-20T17:58:33.087898] [INFO ] [MAIN ] CLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:33.138559] [DEBUG] [LOCAL ] CLX: Loaded 52408 existing records from primary storage. [2025-10-20T17:58:33.138897] [INFO ] [MAIN ] CLX: Loaded 52408 existing records. [2025-10-20T17:58:33.321363] [WARN ] [MAIN ] CLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:33.322090] [INFO ] [MAIN ] CLX: Wiping existing primary data before full refresh. [2025-10-20T17:58:33.322414] [INFO ] [LOCAL ] CLX: Starting deletion of primary data. [2025-10-20T17:58:33.322734] [DEBUG] [LOCAL ] CLX: Deleted primary data file. [2025-10-20T17:58:33.322794] [INFO ] [MAIN ] CLX: Starting full primary refresh. [2025-10-20T17:58:33.676260] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T17:58:33.742840] [INFO ] [REMOTE ] CLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:33.777603] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-06 (2.33 seconds). [2025-10-20T17:58:33.851362] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-07. [2025-10-20T17:58:34.288087] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:58:34.651436] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-02. [2025-10-20T17:58:35.451392] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-01. [2025-10-20T17:58:35.780384] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T17:58:36.229050] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T17:58:36.251342] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-08. [2025-10-20T17:58:37.051415] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-03. [2025-10-20T17:58:37.183379] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T17:58:37.851437] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-02. [2025-10-20T17:58:38.287552] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T17:58:38.660742] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-09. [2025-10-20T17:58:38.686088] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:58:39.451412] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-03. [2025-10-20T17:58:39.621187] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T17:58:40.251399] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-04. [2025-10-20T17:58:40.551215] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T17:58:40.830718] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T17:58:41.051382] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-04. [2025-10-20T17:58:41.511503] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T17:58:41.851409] [INFO ] [REMOTE ] CLS: Fetching primary data for 2025-10. [2025-10-20T17:58:41.888906] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T17:58:42.651371] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-05. [2025-10-20T17:58:43.451442] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-05. [2025-10-20T17:58:44.251366] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-03. [2025-10-20T17:58:44.399590] [INFO ] [REMOTE ] CLS: Incremental fetch complete for 2025-10 (2.55 seconds). [2025-10-20T17:58:44.408297] [INFO ] [REMOTE ] CLS: Full fetch complete (77697 records). [2025-10-20T17:58:44.408556] [DEBUG] [VALIDATOR ] CLS: Validating primary data. [2025-10-20T17:58:44.532876] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T17:58:44.824502] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T17:58:44.875666] [DEBUG] [VALIDATOR ] CLS: All expected trading days are present. [2025-10-20T17:58:44.875775] [DEBUG] [VALIDATOR ] CLS: Primary data validation passed. [2025-10-20T17:58:44.877494] [INFO ] [MAIN ] CLS: Fetched primary data is valid. [2025-10-20T17:58:44.877561] [DEBUG] [LOCAL ] CLS: Saving primary data. [2025-10-20T17:58:44.879097] [DEBUG] [LOCAL ] CLS: Trimmed 77697 records to 53569 (keeping 200 days of data) [2025-10-20T17:58:45.053798] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-06. [2025-10-20T17:58:45.208263] [DEBUG] [LOCAL ] CLS: Primary data saved (53569 records). [2025-10-20T17:58:45.208463] [INFO ] [MAIN ] CLS: Full history saved. [2025-10-20T17:58:45.208561] [INFO ] [MAIN ] Update Progress: 466/2367 symbols completed. CLS took 35.1s. Estimated remaining: 3h 59m 23s. Rate: 7.9 symbols/min [2025-10-20T17:58:45.385448] [INFO ] [MAIN ] CM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:45.442573] [DEBUG] [LOCAL ] CM: Loaded 49434 existing records from primary storage. [2025-10-20T17:58:45.443221] [INFO ] [MAIN ] CM: Loaded 49434 existing records. [2025-10-20T17:58:45.631094] [WARN ] [MAIN ] CM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:45.631349] [INFO ] [MAIN ] CM: Wiping existing primary data before full refresh. [2025-10-20T17:58:45.631556] [INFO ] [LOCAL ] CM: Starting deletion of primary data. [2025-10-20T17:58:45.632077] [DEBUG] [LOCAL ] CM: Deleted primary data file. [2025-10-20T17:58:45.632138] [INFO ] [MAIN ] CM: Starting full primary refresh. [2025-10-20T17:58:45.856616] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-06. [2025-10-20T17:58:45.992583] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T17:58:46.087434] [INFO ] [REMOTE ] CM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:58:46.100117] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T17:58:46.651380] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-04. [2025-10-20T17:58:47.344076] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T17:58:47.451398] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-07. [2025-10-20T17:58:48.251358] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-01. [2025-10-20T17:58:48.548908] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T17:58:48.573262] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T17:58:49.051387] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-07. [2025-10-20T17:58:49.612596] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T17:58:49.851391] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-05. [2025-10-20T17:58:50.651404] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-02. [2025-10-20T17:58:50.786019] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:58:51.328530] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T17:58:51.451379] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-06. [2025-10-20T17:58:52.183810] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T17:58:52.251363] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-03. [2025-10-20T17:58:53.001396] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T17:58:53.051399] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-08. [2025-10-20T17:58:53.851348] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-08. [2025-10-20T17:58:53.881970] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T17:58:54.120773] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T17:58:54.651427] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-09. [2025-10-20T17:58:55.451356] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-04. [2025-10-20T17:58:55.656544] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T17:58:55.839896] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T17:58:56.251404] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-07. [2025-10-20T17:58:57.044851] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T17:58:57.054339] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-09. [2025-10-20T17:58:57.851352] [INFO ] [REMOTE ] CLW: Fetching primary data for 2025-10. [2025-10-20T17:58:58.214440] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T17:58:58.285538] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T17:58:58.520891] [INFO ] [REMOTE ] CLW: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T17:58:58.524147] [INFO ] [REMOTE ] CLW: Full fetch complete (33796 records). [2025-10-20T17:58:58.524223] [DEBUG] [VALIDATOR ] CLW: Validating primary data. [2025-10-20T17:58:58.659734] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-05. [2025-10-20T17:58:58.888754] [DEBUG] [VALIDATOR ] CLW: All expected trading days are present. [2025-10-20T17:58:58.888836] [DEBUG] [VALIDATOR ] CLW: Primary data validation passed. [2025-10-20T17:58:58.889336] [INFO ] [MAIN ] CLW: Fetched primary data is valid. [2025-10-20T17:58:58.889375] [DEBUG] [LOCAL ] CLW: Saving primary data. [2025-10-20T17:58:58.890139] [DEBUG] [LOCAL ] CLW: Trimmed 33796 records to 21460 (keeping 200 days of data) [2025-10-20T17:58:59.011709] [DEBUG] [LOCAL ] CLW: Primary data saved (21460 records). [2025-10-20T17:58:59.011834] [INFO ] [MAIN ] CLW: Full history saved. [2025-10-20T17:58:59.011908] [INFO ] [MAIN ] Update Progress: 467/2367 symbols completed. CLW took 28.5s. Estimated remaining: 3h 59m 41s. Rate: 7.9 symbols/min [2025-10-20T17:58:59.177518] [INFO ] [MAIN ] CMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:58:59.220431] [DEBUG] [LOCAL ] CMA: Loaded 50902 existing records from primary storage. [2025-10-20T17:58:59.220754] [INFO ] [MAIN ] CMA: Loaded 50902 existing records. [2025-10-20T17:58:59.391612] [WARN ] [MAIN ] CMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:58:59.391843] [INFO ] [MAIN ] CMA: Wiping existing primary data before full refresh. [2025-10-20T17:58:59.391889] [INFO ] [LOCAL ] CMA: Starting deletion of primary data. [2025-10-20T17:58:59.392185] [DEBUG] [LOCAL ] CMA: Deleted primary data file. [2025-10-20T17:58:59.392223] [INFO ] [MAIN ] CMA: Starting full primary refresh. [2025-10-20T17:58:59.456618] [INFO ] [REMOTE ] CLVT: Fetching primary data for 2025-10. [2025-10-20T17:58:59.741070] [INFO ] [REMOTE ] CMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:00.251395] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-08. [2025-10-20T17:59:00.521402] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T17:59:01.051350] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-01. [2025-10-20T17:59:01.435093] [INFO ] [REMOTE ] CLVT: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T17:59:01.447257] [INFO ] [REMOTE ] CLVT: Full fetch complete (75393 records). [2025-10-20T17:59:01.447490] [DEBUG] [VALIDATOR ] CLVT: Validating primary data. [2025-10-20T17:59:01.852693] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-06. [2025-10-20T17:59:01.871295] [DEBUG] [VALIDATOR ] CLVT: All expected trading days are present. [2025-10-20T17:59:01.872587] [DEBUG] [VALIDATOR ] CLVT: Primary data validation passed. [2025-10-20T17:59:01.873507] [INFO ] [MAIN ] CLVT: Fetched primary data is valid. [2025-10-20T17:59:01.873710] [DEBUG] [LOCAL ] CLVT: Saving primary data. [2025-10-20T17:59:01.875270] [DEBUG] [LOCAL ] CLVT: Trimmed 75393 records to 51923 (keeping 200 days of data) [2025-10-20T17:59:02.171076] [DEBUG] [LOCAL ] CLVT: Primary data saved (51923 records). [2025-10-20T17:59:02.171408] [INFO ] [MAIN ] CLVT: Full history saved. [2025-10-20T17:59:02.171563] [INFO ] [MAIN ] Update Progress: 468/2367 symbols completed. CLVT took 32.8s. Estimated remaining: 3h 59m 15s. Rate: 7.9 symbols/min [2025-10-20T17:59:02.345031] [INFO ] [MAIN ] CMBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:02.387243] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T17:59:02.439016] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T17:59:02.436833] [DEBUG] [LOCAL ] CMBT: Loaded 26862 existing records from primary storage. [2025-10-20T17:59:02.451043] [INFO ] [MAIN ] CMBT: Loaded 26862 existing records. [2025-10-20T17:59:02.622180] [WARN ] [MAIN ] CMBT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:02.622328] [INFO ] [MAIN ] CMBT: Wiping existing primary data before full refresh. [2025-10-20T17:59:02.622373] [INFO ] [LOCAL ] CMBT: Starting deletion of primary data. [2025-10-20T17:59:02.623109] [DEBUG] [LOCAL ] CMBT: Deleted primary data file. [2025-10-20T17:59:02.623165] [INFO ] [MAIN ] CMBT: Starting full primary refresh. [2025-10-20T17:59:02.656577] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-09. [2025-10-20T17:59:02.965365] [INFO ] [REMOTE ] CMBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:03.456493] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-02. [2025-10-20T17:59:03.473727] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:59:04.217677] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T17:59:04.251356] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-07. [2025-10-20T17:59:04.684300] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T17:59:05.051442] [INFO ] [REMOTE ] CLX: Fetching primary data for 2025-10. [2025-10-20T17:59:05.851344] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-01. [2025-10-20T17:59:05.984994] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T17:59:06.651484] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-03. [2025-10-20T17:59:06.855997] [INFO ] [REMOTE ] CLX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T17:59:06.863459] [INFO ] [REMOTE ] CLX: Full fetch complete (73725 records). [2025-10-20T17:59:06.863624] [DEBUG] [VALIDATOR ] CLX: Validating primary data. [2025-10-20T17:59:06.974541] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T17:59:07.264255] [DEBUG] [VALIDATOR ] CLX: All expected trading days are present. [2025-10-20T17:59:07.264331] [DEBUG] [VALIDATOR ] CLX: Primary data validation passed. [2025-10-20T17:59:07.265142] [INFO ] [MAIN ] CLX: Fetched primary data is valid. [2025-10-20T17:59:07.265205] [DEBUG] [LOCAL ] CLX: Saving primary data. [2025-10-20T17:59:07.266491] [DEBUG] [LOCAL ] CLX: Trimmed 73725 records to 51778 (keeping 200 days of data) [2025-10-20T17:59:07.473874] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-02. [2025-10-20T17:59:07.580577] [DEBUG] [LOCAL ] CLX: Primary data saved (51778 records). [2025-10-20T17:59:07.580751] [INFO ] [MAIN ] CLX: Full history saved. [2025-10-20T17:59:07.580842] [INFO ] [MAIN ] Update Progress: 469/2367 symbols completed. CLX took 34.7s. Estimated remaining: 3h 58m 59s. Rate: 7.9 symbols/min [2025-10-20T17:59:07.762977] [INFO ] [MAIN ] CMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:07.801821] [DEBUG] [LOCAL ] CMC: Loaded 46259 existing records from primary storage. [2025-10-20T17:59:07.802107] [INFO ] [MAIN ] CMC: Loaded 46259 existing records. [2025-10-20T17:59:07.982527] [WARN ] [MAIN ] CMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:07.982809] [INFO ] [MAIN ] CMC: Wiping existing primary data before full refresh. [2025-10-20T17:59:07.982962] [INFO ] [LOCAL ] CMC: Starting deletion of primary data. [2025-10-20T17:59:07.983180] [DEBUG] [LOCAL ] CMC: Deleted primary data file. [2025-10-20T17:59:07.983237] [INFO ] [MAIN ] CMC: Starting full primary refresh. [2025-10-20T17:59:08.251544] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-08. [2025-10-20T17:59:08.365393] [INFO ] [REMOTE ] CMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:08.366855] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T17:59:08.425459] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T17:59:09.051411] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-03. [2025-10-20T17:59:09.851384] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-01. [2025-10-20T17:59:09.904882] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T17:59:10.060600] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T17:59:10.651412] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-04. [2025-10-20T17:59:11.451435] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-04. [2025-10-20T17:59:11.545506] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:59:12.052367] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T17:59:12.251390] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-02. [2025-10-20T17:59:12.495982] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T17:59:13.051398] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-09. [2025-10-20T17:59:13.805303] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T17:59:13.851347] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-03. [2025-10-20T17:59:14.651408] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-05. [2025-10-20T17:59:14.736089] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T17:59:15.320240] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T17:59:15.451388] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-05. [2025-10-20T17:59:15.695996] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T17:59:16.251442] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-04. [2025-10-20T17:59:16.811364] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T17:59:17.051379] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-06. [2025-10-20T17:59:17.851365] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-06. [2025-10-20T17:59:18.040816] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:59:18.651430] [INFO ] [REMOTE ] CM: Fetching primary data for 2025-10. [2025-10-20T17:59:18.768697] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T17:59:18.940692] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T17:59:19.451439] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-07. [2025-10-20T17:59:20.234742] [INFO ] [REMOTE ] CM: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T17:59:20.245048] [INFO ] [REMOTE ] CM: Full fetch complete (69866 records). [2025-10-20T17:59:20.245271] [DEBUG] [VALIDATOR ] CM: Validating primary data. [2025-10-20T17:59:20.258501] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-05. [2025-10-20T17:59:20.649110] [DEBUG] [VALIDATOR ] CM: All expected trading days are present. [2025-10-20T17:59:20.649228] [DEBUG] [VALIDATOR ] CM: Primary data validation passed. [2025-10-20T17:59:20.650035] [INFO ] [MAIN ] CM: Fetched primary data is valid. [2025-10-20T17:59:20.650097] [DEBUG] [LOCAL ] CM: Saving primary data. [2025-10-20T17:59:20.651497] [DEBUG] [LOCAL ] CM: Trimmed 69866 records to 48575 (keeping 200 days of data) [2025-10-20T17:59:20.947281] [DEBUG] [LOCAL ] CM: Primary data saved (48575 records). [2025-10-20T17:59:20.947423] [INFO ] [MAIN ] CM: Full history saved. [2025-10-20T17:59:20.947501] [INFO ] [MAIN ] Update Progress: 470/2367 symbols completed. CM took 35.7s. Estimated remaining: 3h 59m 15s. Rate: 7.9 symbols/min [2025-10-20T17:59:21.053600] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-07. [2025-10-20T17:59:21.128746] [INFO ] [MAIN ] CMCM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:21.135426] [DEBUG] [LOCAL ] CMCM: Loaded 4894 existing records from primary storage. [2025-10-20T17:59:21.135767] [INFO ] [MAIN ] CMCM: Loaded 4894 existing records. [2025-10-20T17:59:21.275020] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T17:59:21.389971] [WARN ] [MAIN ] CMCM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:21.390236] [INFO ] [MAIN ] CMCM: Wiping existing primary data before full refresh. [2025-10-20T17:59:21.390293] [INFO ] [LOCAL ] CMCM: Starting deletion of primary data. [2025-10-20T17:59:21.390442] [DEBUG] [LOCAL ] CMCM: Deleted primary data file. [2025-10-20T17:59:21.390500] [INFO ] [MAIN ] CMCM: Starting full primary refresh. [2025-10-20T17:59:21.761677] [INFO ] [REMOTE ] CMCM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:21.851368] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-01. [2025-10-20T17:59:22.017281] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T17:59:22.167848] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T17:59:22.170984] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-01 (0.32 seconds). [2025-10-20T17:59:22.651451] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-06. [2025-10-20T17:59:23.451424] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-08. [2025-10-20T17:59:24.257635] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-02. [2025-10-20T17:59:24.271593] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T17:59:24.860942] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T17:59:25.051439] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-07. [2025-10-20T17:59:25.340780] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T17:59:25.851398] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-03. [2025-10-20T17:59:26.517073] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T17:59:26.585768] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T17:59:26.651343] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-04. [2025-10-20T17:59:27.012480] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T17:59:27.451371] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-09. [2025-10-20T17:59:28.251379] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-08. [2025-10-20T17:59:29.051427] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-05. [2025-10-20T17:59:29.114867] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T17:59:29.397940] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T17:59:29.846061] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T17:59:29.854512] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-06. [2025-10-20T17:59:30.450602] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T17:59:30.651429] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-07. [2025-10-20T17:59:31.084941] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T17:59:31.451461] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-08. [2025-10-20T17:59:32.251460] [INFO ] [REMOTE ] CMA: Fetching primary data for 2025-10. [2025-10-20T17:59:32.796882] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T17:59:33.051382] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-09. [2025-10-20T17:59:33.851399] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-08. [2025-10-20T17:59:33.898979] [INFO ] [REMOTE ] CMA: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T17:59:33.906939] [INFO ] [REMOTE ] CMA: Full fetch complete (73483 records). [2025-10-20T17:59:33.907103] [DEBUG] [VALIDATOR ] CMA: Validating primary data. [2025-10-20T17:59:34.299592] [DEBUG] [VALIDATOR ] CMA: All expected trading days are present. [2025-10-20T17:59:34.299695] [DEBUG] [VALIDATOR ] CMA: Primary data validation passed. [2025-10-20T17:59:34.300524] [INFO ] [MAIN ] CMA: Fetched primary data is valid. [2025-10-20T17:59:34.300568] [DEBUG] [LOCAL ] CMA: Saving primary data. [2025-10-20T17:59:34.301979] [DEBUG] [LOCAL ] CMA: Trimmed 73483 records to 50177 (keeping 200 days of data) [2025-10-20T17:59:34.538864] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T17:59:34.641581] [DEBUG] [LOCAL ] CMA: Primary data saved (50177 records). [2025-10-20T17:59:34.641868] [INFO ] [MAIN ] CMA: Full history saved. [2025-10-20T17:59:34.642028] [INFO ] [MAIN ] Update Progress: 471/2367 symbols completed. CMA took 35.6s. Estimated remaining: 3h 59m 32s. Rate: 7.9 symbols/min [2025-10-20T17:59:34.656555] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-09. [2025-10-20T17:59:34.850683] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T17:59:34.853608] [INFO ] [MAIN ] CMDB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:34.863436] [DEBUG] [LOCAL ] CMDB: No existing data file found in primary storage. [2025-10-20T17:59:34.864180] [WARN ] [MAIN ] CMDB: No existing data found, performing full refresh instead. [2025-10-20T17:59:34.864751] [INFO ] [MAIN ] CMDB: Wiping existing primary data before full refresh. [2025-10-20T17:59:34.864839] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-10-20T17:59:34.864929] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-10-20T17:59:34.865416] [INFO ] [MAIN ] CMDB: Starting full primary refresh. [2025-10-20T17:59:35.210510] [INFO ] [REMOTE ] CMDB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:35.451434] [INFO ] [REMOTE ] CMC: Fetching primary data for 2025-10. [2025-10-20T17:59:35.578739] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T17:59:36.251412] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-09. [2025-10-20T17:59:37.051370] [INFO ] [REMOTE ] CMCM: Fetching primary data for 2025-10. [2025-10-20T17:59:37.214710] [INFO ] [REMOTE ] CMC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T17:59:37.221664] [INFO ] [REMOTE ] CMC: Full fetch complete (67272 records). [2025-10-20T17:59:37.221783] [DEBUG] [VALIDATOR ] CMC: Validating primary data. [2025-10-20T17:59:37.623113] [DEBUG] [VALIDATOR ] CMC: All expected trading days are present. [2025-10-20T17:59:37.623501] [DEBUG] [VALIDATOR ] CMC: Primary data validation passed. [2025-10-20T17:59:37.624661] [INFO ] [MAIN ] CMC: Fetched primary data is valid. [2025-10-20T17:59:37.624731] [DEBUG] [LOCAL ] CMC: Saving primary data. [2025-10-20T17:59:37.626972] [DEBUG] [LOCAL ] CMC: Trimmed 67272 records to 45569 (keeping 200 days of data) [2025-10-20T17:59:37.730119] [INFO ] [REMOTE ] CMCM: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T17:59:37.741458] [INFO ] [REMOTE ] CMCM: Full fetch complete (7026 records). [2025-10-20T17:59:37.741683] [DEBUG] [VALIDATOR ] CMCM: Validating primary data. [2025-10-20T17:59:37.911428] [INFO ] [REMOTE ] CMDB: Fetching primary data for 2025-01. [2025-10-20T17:59:38.332466] [DEBUG] [REMOTE ] CMDB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T17:59:38.334608] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-10-20T17:59:38.383213] [DEBUG] [VALIDATOR ] CMCM: All expected trading days are present. [2025-10-20T17:59:38.386065] [DEBUG] [VALIDATOR ] CMCM: Primary data validation passed. [2025-10-20T17:59:38.386486] [INFO ] [MAIN ] CMCM: Fetched primary data is valid. [2025-10-20T17:59:38.386625] [WARN ] [REMOTE ] CMDB: No data returned for month 2025-01 [2025-10-20T17:59:38.390412] [DEBUG] [LOCAL ] CMCM: Saving primary data. [2025-10-20T17:59:38.393115] [FATAL] [REMOTE ] CMDB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T17:59:38.395295] [FATAL] [MAIN ] CMDB: Failed to fetch primary data. [2025-10-20T17:59:38.401915] [WARN ] [MAIN ] CMDB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T17:59:38.403953] [DEBUG] [LOCAL ] CMC: Primary data saved (45569 records). [2025-10-20T17:59:38.401436] [DEBUG] [LOCAL ] CMCM: Trimmed 7026 records to 4827 (keeping 200 days of data) [2025-10-20T17:59:38.404152] [INFO ] [LOCAL ] CMDB: Starting deletion of primary data. [2025-10-20T17:59:38.404321] [INFO ] [MAIN ] CMC: Full history saved. [2025-10-20T17:59:38.405065] [DEBUG] [LOCAL ] CMDB: No primary data file found to delete. [2025-10-20T17:59:38.405210] [INFO ] [LOCAL ] CMDB: Starting deletion of derivative data. [2025-10-20T17:59:38.405669] [INFO ] [MAIN ] Update Progress: 472/2367 symbols completed. CMC took 30.8s. Estimated remaining: 3h 59m 9s. Rate: 7.9 symbols/min [2025-10-20T17:59:38.406231] [DEBUG] [LOCAL ] CMDB: No derivative data file found to delete. [2025-10-20T17:59:38.406485] [INFO ] [MAIN ] Update Progress: 473/2367 symbols completed. CMDB took 3.8s. Estimated remaining: 3h 58m 31s. Rate: 7.9 symbols/min [2025-10-20T17:59:38.583042] [DEBUG] [LOCAL ] CMCM: Primary data saved (4827 records). [2025-10-20T17:59:38.630990] [INFO ] [MAIN ] CMCM: Full history saved. [2025-10-20T17:59:38.657902] [INFO ] [REMOTE ] CMBT: Fetching primary data for 2025-10. [2025-10-20T17:59:38.679361] [INFO ] [MAIN ] Update Progress: 474/2367 symbols completed. CMCM took 17.7s. Estimated remaining: 3h 57m 54s. Rate: 8.0 symbols/min [2025-10-20T17:59:38.904071] [INFO ] [MAIN ] CMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:38.959989] [INFO ] [MAIN ] CMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:39.038010] [INFO ] [MAIN ] CMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:39.101623] [DEBUG] [LOCAL ] CMI: Loaded 42340 existing records from primary storage. [2025-10-20T17:59:39.102049] [INFO ] [MAIN ] CMI: Loaded 42340 existing records. [2025-10-20T17:59:39.235013] [DEBUG] [LOCAL ] CMG: Loaded 54501 existing records from primary storage. [2025-10-20T17:59:39.246701] [DEBUG] [LOCAL ] CMP: Loaded 36229 existing records from primary storage. [2025-10-20T17:59:39.280215] [INFO ] [MAIN ] CMG: Loaded 54501 existing records. [2025-10-20T17:59:39.286069] [INFO ] [MAIN ] CMP: Loaded 36229 existing records. [2025-10-20T17:59:39.317631] [WARN ] [MAIN ] CMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.430629] [INFO ] [MAIN ] CMI: Wiping existing primary data before full refresh. [2025-10-20T17:59:39.449001] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-10-20T17:59:39.458806] [DEBUG] [LOCAL ] CMI: Deleted primary data file. [2025-10-20T17:59:39.495139] [INFO ] [MAIN ] CMI: Starting full primary refresh. [2025-10-20T17:59:39.908002] [WARN ] [MAIN ] CMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.941395] [WARN ] [MAIN ] CMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T17:59:39.995167] [INFO ] [MAIN ] CMP: Wiping existing primary data before full refresh. [2025-10-20T17:59:39.995526] [INFO ] [LOCAL ] CMP: Starting deletion of primary data. [2025-10-20T17:59:39.995390] [INFO ] [MAIN ] CMG: Wiping existing primary data before full refresh. [2025-10-20T17:59:40.006063] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-10-20T17:59:40.017883] [DEBUG] [LOCAL ] CMP: Deleted primary data file. [2025-10-20T17:59:40.023193] [INFO ] [MAIN ] CMP: Starting full primary refresh. [2025-10-20T17:59:40.160424] [DEBUG] [LOCAL ] CMG: Deleted primary data file. [2025-10-20T17:59:40.197160] [INFO ] [MAIN ] CMG: Starting full primary refresh. [2025-10-20T17:59:40.506084] [INFO ] [REMOTE ] CMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:40.541873] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-01. [2025-10-20T17:59:40.894021] [INFO ] [REMOTE ] CMBT: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-10-20T17:59:40.897615] [INFO ] [REMOTE ] CMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:40.931625] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-01. [2025-10-20T17:59:40.943335] [INFO ] [REMOTE ] CMBT: Full fetch complete (33365 records). [2025-10-20T17:59:40.959171] [DEBUG] [VALIDATOR ] CMBT: Validating primary data. [2025-10-20T17:59:41.107056] [INFO ] [REMOTE ] CMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:41.121615] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-01. [2025-10-20T17:59:41.420846] [DEBUG] [VALIDATOR ] CMBT: All expected trading days are present. [2025-10-20T17:59:41.420973] [DEBUG] [VALIDATOR ] CMBT: Primary data validation passed. [2025-10-20T17:59:41.422328] [INFO ] [MAIN ] CMBT: Fetched primary data is valid. [2025-10-20T17:59:41.422401] [DEBUG] [LOCAL ] CMBT: Saving primary data. [2025-10-20T17:59:41.423804] [DEBUG] [LOCAL ] CMBT: Trimmed 33365 records to 27012 (keeping 200 days of data) [2025-10-20T17:59:41.612250] [DEBUG] [LOCAL ] CMBT: Primary data saved (27012 records). [2025-10-20T17:59:41.612503] [INFO ] [MAIN ] CMBT: Full history saved. [2025-10-20T17:59:41.612630] [INFO ] [MAIN ] Update Progress: 475/2367 symbols completed. CMBT took 39.4s. Estimated remaining: 3h 57m 29s. Rate: 8.0 symbols/min [2025-10-20T17:59:41.796896] [INFO ] [MAIN ] CMPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T17:59:41.797174] [DEBUG] [LOCAL ] CMPO: No existing data file found in primary storage. [2025-10-20T17:59:41.797218] [WARN ] [MAIN ] CMPO: No existing data found, performing full refresh instead. [2025-10-20T17:59:41.797351] [INFO ] [MAIN ] CMPO: Wiping existing primary data before full refresh. [2025-10-20T17:59:41.797401] [INFO ] [LOCAL ] CMPO: Starting deletion of primary data. [2025-10-20T17:59:41.797446] [DEBUG] [LOCAL ] CMPO: No primary data file found to delete. [2025-10-20T17:59:41.797481] [INFO ] [MAIN ] CMPO: Starting full primary refresh. [2025-10-20T17:59:42.201273] [INFO ] [REMOTE ] CMPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T17:59:42.201917] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-01. [2025-10-20T17:59:42.226966] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T17:59:42.439355] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T17:59:42.566376] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:59:42.651346] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-02. [2025-10-20T17:59:43.451337] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-02. [2025-10-20T17:59:43.640092] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T17:59:44.230272] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T17:59:44.251339] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-03. [2025-10-20T17:59:45.051341] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-02. [2025-10-20T17:59:45.211845] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T17:59:45.851436] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-02. [2025-10-20T17:59:45.977161] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T17:59:46.515004] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T17:59:46.651371] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-03. [2025-10-20T17:59:47.005787] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T17:59:47.451440] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-04. [2025-10-20T17:59:48.256538] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-03. [2025-10-20T17:59:48.269547] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T17:59:49.051375] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-04. [2025-10-20T17:59:49.142977] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T17:59:49.673965] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T17:59:49.851400] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-04. [2025-10-20T17:59:50.258571] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T17:59:50.651386] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-03. [2025-10-20T17:59:51.227543] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T17:59:51.451407] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-05. [2025-10-20T17:59:52.251478] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-05. [2025-10-20T17:59:52.427952] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T17:59:53.051351] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-04. [2025-10-20T17:59:53.166347] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T17:59:53.824114] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T17:59:53.851365] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-06. [2025-10-20T17:59:54.651397] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-05. [2025-10-20T17:59:54.841260] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T17:59:55.159502] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T17:59:55.451388] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-07. [2025-10-20T17:59:56.251387] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-06. [2025-10-20T17:59:56.334797] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T17:59:56.971409] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T17:59:57.051369] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-08. [2025-10-20T17:59:57.579303] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T17:59:57.851387] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-05. [2025-10-20T17:59:58.141456] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T17:59:58.651372] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-06. [2025-10-20T17:59:59.451352] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-09. [2025-10-20T17:59:59.637083] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T17:59:59.704997] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:00:00.251381] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-07. [2025-10-20T18:00:00.946882] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:00:01.051374] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-06. [2025-10-20T18:00:01.851361] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-07. [2025-10-20T18:00:01.927802] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:00:02.651357] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-08. [2025-10-20T18:00:02.966303] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T18:00:03.076722] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T18:00:03.451419] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-07. [2025-10-20T18:00:04.251400] [INFO ] [REMOTE ] CMPO: Fetching primary data for 2025-10. [2025-10-20T18:00:04.369743] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:00:05.051444] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-08. [2025-10-20T18:00:05.636818] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T18:00:05.801103] [INFO ] [REMOTE ] CMPO: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T18:00:05.806334] [INFO ] [REMOTE ] CMPO: Full fetch complete (58376 records). [2025-10-20T18:00:05.806487] [DEBUG] [VALIDATOR ] CMPO: Validating primary data. [2025-10-20T18:00:05.861534] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-09. [2025-10-20T18:00:06.191121] [DEBUG] [VALIDATOR ] CMPO: All expected trading days are present. [2025-10-20T18:00:06.191300] [DEBUG] [VALIDATOR ] CMPO: Primary data validation passed. [2025-10-20T18:00:06.192052] [INFO ] [MAIN ] CMPO: Fetched primary data is valid. [2025-10-20T18:00:06.192125] [DEBUG] [LOCAL ] CMPO: Saving primary data. [2025-10-20T18:00:06.193428] [DEBUG] [LOCAL ] CMPO: Trimmed 58376 records to 38711 (keeping 200 days of data) [2025-10-20T18:00:06.294507] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:00:06.438200] [DEBUG] [LOCAL ] CMPO: Primary data saved (38711 records). [2025-10-20T18:00:06.438342] [INFO ] [MAIN ] CMPO: Full history saved. [2025-10-20T18:00:06.438414] [INFO ] [MAIN ] Update Progress: 476/2367 symbols completed. CMPO took 24.8s. Estimated remaining: 3h 58m 30s. Rate: 7.9 symbols/min [2025-10-20T18:00:06.603196] [INFO ] [MAIN ] CMRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:06.634968] [DEBUG] [LOCAL ] CMRE: Loaded 36418 existing records from primary storage. [2025-10-20T18:00:06.635256] [INFO ] [MAIN ] CMRE: Loaded 36418 existing records. [2025-10-20T18:00:06.656788] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-08. [2025-10-20T18:00:06.815326] [WARN ] [MAIN ] CMRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:06.815555] [INFO ] [MAIN ] CMRE: Wiping existing primary data before full refresh. [2025-10-20T18:00:06.815609] [INFO ] [LOCAL ] CMRE: Starting deletion of primary data. [2025-10-20T18:00:06.816021] [DEBUG] [LOCAL ] CMRE: Deleted primary data file. [2025-10-20T18:00:06.816076] [INFO ] [MAIN ] CMRE: Starting full primary refresh. [2025-10-20T18:00:07.161204] [INFO ] [REMOTE ] CMRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:07.427226] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:00:07.451349] [INFO ] [REMOTE ] CMI: Fetching primary data for 2025-10. [2025-10-20T18:00:08.251441] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-01. [2025-10-20T18:00:08.992078] [INFO ] [REMOTE ] CMI: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:00:08.998398] [INFO ] [REMOTE ] CMI: Full fetch complete (60719 records). [2025-10-20T18:00:08.998547] [DEBUG] [VALIDATOR ] CMI: Validating primary data. [2025-10-20T18:00:08.999376] [FATAL] [VALIDATOR ] CMI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:00:08.999461] [FATAL] [MAIN ] CMI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:00:08.999502] [WARN ] [MAIN ] CMI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:00:08.999571] [INFO ] [LOCAL ] CMI: Starting deletion of primary data. [2025-10-20T18:00:08.999696] [DEBUG] [LOCAL ] CMI: No primary data file found to delete. [2025-10-20T18:00:08.999748] [INFO ] [LOCAL ] CMI: Starting deletion of derivative data. [2025-10-20T18:00:09.000138] [DEBUG] [LOCAL ] CMI: Deleted derivative data file. [2025-10-20T18:00:09.000227] [INFO ] [MAIN ] Update Progress: 477/2367 symbols completed. CMI took 30.6s. Estimated remaining: 3h 58m 2s. Rate: 7.9 symbols/min [2025-10-20T18:00:09.048533] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-10-20T18:00:09.059266] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-09. [2025-10-20T18:00:09.305147] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T18:00:09.851478] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-02. [2025-10-20T18:00:10.499069] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:00:10.651395] [INFO ] [REMOTE ] CMP: Fetching primary data for 2025-10. [2025-10-20T18:00:10.984481] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:00:11.451373] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-03. [2025-10-20T18:00:11.709938] [INFO ] [REMOTE ] CMP: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:00:11.716094] [INFO ] [REMOTE ] CMP: Full fetch complete (51646 records). [2025-10-20T18:00:11.716233] [DEBUG] [VALIDATOR ] CMP: Validating primary data. [2025-10-20T18:00:12.129396] [DEBUG] [VALIDATOR ] CMP: All expected trading days are present. [2025-10-20T18:00:12.129484] [DEBUG] [VALIDATOR ] CMP: Primary data validation passed. [2025-10-20T18:00:12.130201] [INFO ] [MAIN ] CMP: Fetched primary data is valid. [2025-10-20T18:00:12.130270] [DEBUG] [LOCAL ] CMP: Saving primary data. [2025-10-20T18:00:12.131470] [DEBUG] [LOCAL ] CMP: Trimmed 51646 records to 35799 (keeping 200 days of data) [2025-10-20T18:00:12.252814] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-09. [2025-10-20T18:00:12.349015] [DEBUG] [LOCAL ] CMP: Primary data saved (35799 records). [2025-10-20T18:00:12.349130] [INFO ] [MAIN ] CMP: Full history saved. [2025-10-20T18:00:12.349220] [INFO ] [MAIN ] Update Progress: 478/2367 symbols completed. CMP took 33.7s. Estimated remaining: 3h 57m 38s. Rate: 7.9 symbols/min [2025-10-20T18:00:12.795384] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:00:13.051497] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-04. [2025-10-20T18:00:14.473103] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:00:14.482838] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-05. [2025-10-20T18:00:14.574117] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-09 (2.32 seconds). [2025-10-20T18:00:14.651418] [INFO ] [REMOTE ] CMG: Fetching primary data for 2025-10. [2025-10-20T18:00:15.494854] [INFO ] [REMOTE ] CMG: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:00:15.502341] [INFO ] [REMOTE ] CMG: Full fetch complete (77503 records). [2025-10-20T18:00:15.502493] [DEBUG] [VALIDATOR ] CMG: Validating primary data. [2025-10-20T18:00:15.503471] [FATAL] [VALIDATOR ] CMG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:00:15.503562] [FATAL] [MAIN ] CMG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:00:15.503604] [WARN ] [MAIN ] CMG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:00:15.503675] [INFO ] [LOCAL ] CMG: Starting deletion of primary data. [2025-10-20T18:00:15.503757] [DEBUG] [LOCAL ] CMG: No primary data file found to delete. [2025-10-20T18:00:15.503794] [INFO ] [LOCAL ] CMG: Starting deletion of derivative data. [2025-10-20T18:00:15.503947] [DEBUG] [LOCAL ] CMG: Deleted derivative data file. [2025-10-20T18:00:15.504034] [INFO ] [MAIN ] Update Progress: 479/2367 symbols completed. CMG took 37.1s. Estimated remaining: 3h 57m 13s. Rate: 8.0 symbols/min [2025-10-20T18:00:15.920707] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:00:15.928521] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-06. [2025-10-20T18:00:17.340606] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:00:17.348696] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-07. [2025-10-20T18:00:18.879214] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:00:18.887438] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-08. [2025-10-20T18:00:20.468882] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:00:20.477052] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-09. [2025-10-20T18:00:21.756469] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:00:21.765930] [INFO ] [REMOTE ] CMRE: Fetching primary data for 2025-10. [2025-10-20T18:00:22.922282] [INFO ] [REMOTE ] CMRE: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T18:00:22.930881] [INFO ] [REMOTE ] CMRE: Full fetch complete (51327 records). [2025-10-20T18:00:22.931085] [DEBUG] [VALIDATOR ] CMRE: Validating primary data. [2025-10-20T18:00:23.368818] [DEBUG] [VALIDATOR ] CMRE: All expected trading days are present. [2025-10-20T18:00:23.368926] [DEBUG] [VALIDATOR ] CMRE: Primary data validation passed. [2025-10-20T18:00:23.369582] [INFO ] [MAIN ] CMRE: Fetched primary data is valid. [2025-10-20T18:00:23.369626] [DEBUG] [LOCAL ] CMRE: Saving primary data. [2025-10-20T18:00:23.370752] [DEBUG] [LOCAL ] CMRE: Trimmed 51327 records to 35844 (keeping 200 days of data) [2025-10-20T18:00:23.575635] [DEBUG] [LOCAL ] CMRE: Primary data saved (35844 records). [2025-10-20T18:00:23.575802] [INFO ] [MAIN ] CMRE: Full history saved. [2025-10-20T18:00:23.575880] [INFO ] [MAIN ] Update Progress: 480/2367 symbols completed. CMRE took 17.1s. Estimated remaining: 3h 57m 8s. Rate: 8.0 symbols/min [2025-10-20T18:00:24.331745] [INFO ] [MAIN ] CMSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.336276] [INFO ] [MAIN ] CMSC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.363726] [INFO ] [MAIN ] CMSD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.412221] [INFO ] [MAIN ] CMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:24.412439] [DEBUG] [LOCAL ] CMS: No existing data file found in primary storage. [2025-10-20T18:00:24.418701] [DEBUG] [LOCAL ] CMSA: Loaded 3570 existing records from primary storage. [2025-10-20T18:00:24.413841] [WARN ] [MAIN ] CMS: No existing data found, performing full refresh instead. [2025-10-20T18:00:24.419021] [INFO ] [MAIN ] CMSA: Loaded 3570 existing records. [2025-10-20T18:00:24.422263] [INFO ] [MAIN ] CMS: Wiping existing primary data before full refresh. [2025-10-20T18:00:24.422637] [DEBUG] [LOCAL ] CMSC: Loaded 4946 existing records from primary storage. [2025-10-20T18:00:24.422452] [INFO ] [LOCAL ] CMS: Starting deletion of primary data. [2025-10-20T18:00:24.443874] [INFO ] [MAIN ] CMSC: Loaded 4946 existing records. [2025-10-20T18:00:24.463877] [DEBUG] [LOCAL ] CMS: No primary data file found to delete. [2025-10-20T18:00:24.502277] [INFO ] [MAIN ] CMS: Starting full primary refresh. [2025-10-20T18:00:24.557332] [DEBUG] [LOCAL ] CMSD: Loaded 7921 existing records from primary storage. [2025-10-20T18:00:24.640469] [INFO ] [MAIN ] CMSD: Loaded 7921 existing records. [2025-10-20T18:00:25.027032] [WARN ] [MAIN ] CMSC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.079868] [WARN ] [MAIN ] CMSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.117285] [INFO ] [MAIN ] CMSA: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.127827] [INFO ] [MAIN ] CMSC: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.148384] [WARN ] [MAIN ] CMSD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:25.153774] [INFO ] [MAIN ] CMSD: Wiping existing primary data before full refresh. [2025-10-20T18:00:25.142472] [INFO ] [LOCAL ] CMSA: Starting deletion of primary data. [2025-10-20T18:00:25.154047] [INFO ] [LOCAL ] CMSC: Starting deletion of primary data. [2025-10-20T18:00:25.154011] [INFO ] [LOCAL ] CMSD: Starting deletion of primary data. [2025-10-20T18:00:25.178920] [DEBUG] [LOCAL ] CMSA: Deleted primary data file. [2025-10-20T18:00:25.186133] [INFO ] [MAIN ] CMSA: Starting full primary refresh. [2025-10-20T18:00:25.215674] [DEBUG] [LOCAL ] CMSD: Deleted primary data file. [2025-10-20T18:00:25.195611] [DEBUG] [LOCAL ] CMSC: Deleted primary data file. [2025-10-20T18:00:25.225494] [INFO ] [MAIN ] CMSD: Starting full primary refresh. [2025-10-20T18:00:25.239436] [INFO ] [MAIN ] CMSC: Starting full primary refresh. [2025-10-20T18:00:25.887399] [INFO ] [REMOTE ] CMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.045712] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-01. [2025-10-20T18:00:26.372918] [INFO ] [REMOTE ] CMSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.412603] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-01. [2025-10-20T18:00:26.419391] [INFO ] [REMOTE ] CMSD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.425793] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-01. [2025-10-20T18:00:26.512927] [INFO ] [REMOTE ] CMSC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:26.513272] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-01. [2025-10-20T18:00:27.141307] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T18:00:27.143173] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-02. [2025-10-20T18:00:27.218277] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T18:00:27.220077] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-02. [2025-10-20T18:00:27.356239] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:00:27.358958] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-02. [2025-10-20T18:00:27.725966] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:00:27.727538] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-03. [2025-10-20T18:00:27.735058] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:00:27.736378] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-03. [2025-10-20T18:00:27.937184] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:00:27.939110] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-03. [2025-10-20T18:00:28.054202] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:00:28.064293] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-02. [2025-10-20T18:00:28.362788] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:00:28.364543] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-04. [2025-10-20T18:00:28.483626] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:00:28.596828] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T18:00:28.719051] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:00:29.051431] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-05. [2025-10-20T18:00:29.606910] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T18:00:29.790538] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:00:29.851398] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-06. [2025-10-20T18:00:30.190785] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-10-20T18:00:30.651364] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-03. [2025-10-20T18:00:31.451433] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-04. [2025-10-20T18:00:31.863976] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T18:00:32.251428] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-04. [2025-10-20T18:00:32.382847] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:00:32.751877] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T18:00:33.051386] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-04. [2025-10-20T18:00:33.851424] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-07. [2025-10-20T18:00:34.457544] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:00:34.651410] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-05. [2025-10-20T18:00:34.994990] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T18:00:35.285064] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:00:35.451421] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-05. [2025-10-20T18:00:36.112480] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:00:36.251387] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-06. [2025-10-20T18:00:36.604397] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T18:00:37.051435] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-08. [2025-10-20T18:00:37.744605] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:00:37.851393] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-07. [2025-10-20T18:00:38.586162] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:00:38.651353] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-06. [2025-10-20T18:00:39.451354] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-08. [2025-10-20T18:00:39.512638] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T18:00:39.830231] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T18:00:40.251431] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-05. [2025-10-20T18:00:41.051368] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-09. [2025-10-20T18:00:41.613109] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T18:00:41.838431] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:00:41.851340] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-06. [2025-10-20T18:00:42.651348] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-09. [2025-10-20T18:00:43.146457] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T18:00:43.223998] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T18:00:43.451379] [INFO ] [REMOTE ] CMSA: Fetching primary data for 2025-10. [2025-10-20T18:00:44.096952] [INFO ] [REMOTE ] CMSA: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:00:44.098037] [INFO ] [REMOTE ] CMSA: Full fetch complete (4873 records). [2025-10-20T18:00:44.098108] [DEBUG] [VALIDATOR ] CMSA: Validating primary data. [2025-10-20T18:00:44.256516] [INFO ] [REMOTE ] CMSC: Fetching primary data for 2025-10. [2025-10-20T18:00:44.475722] [DEBUG] [VALIDATOR ] CMSA: All expected trading days are present. [2025-10-20T18:00:44.475814] [DEBUG] [VALIDATOR ] CMSA: Primary data validation passed. [2025-10-20T18:00:44.475960] [INFO ] [MAIN ] CMSA: Fetched primary data is valid. [2025-10-20T18:00:44.475999] [DEBUG] [LOCAL ] CMSA: Saving primary data. [2025-10-20T18:00:44.476364] [DEBUG] [LOCAL ] CMSA: Trimmed 4873 records to 3515 (keeping 200 days of data) [2025-10-20T18:00:44.497745] [DEBUG] [LOCAL ] CMSA: Primary data saved (3515 records). [2025-10-20T18:00:44.497882] [INFO ] [MAIN ] CMSA: Full history saved. [2025-10-20T18:00:44.497963] [INFO ] [MAIN ] Update Progress: 481/2367 symbols completed. CMSA took 20.9s. Estimated remaining: 3h 57m 53s. Rate: 7.9 symbols/min [2025-10-20T18:00:44.665057] [INFO ] [MAIN ] CMTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:44.699857] [DEBUG] [LOCAL ] CMTG: Loaded 36246 existing records from primary storage. [2025-10-20T18:00:44.700157] [INFO ] [MAIN ] CMTG: Loaded 36246 existing records. [2025-10-20T18:00:44.877069] [WARN ] [MAIN ] CMTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:44.877296] [INFO ] [MAIN ] CMTG: Wiping existing primary data before full refresh. [2025-10-20T18:00:44.877344] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-10-20T18:00:44.877565] [DEBUG] [LOCAL ] CMTG: Deleted primary data file. [2025-10-20T18:00:44.877621] [INFO ] [MAIN ] CMTG: Starting full primary refresh. [2025-10-20T18:00:45.001035] [INFO ] [REMOTE ] CMSC: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:00:45.036930] [INFO ] [REMOTE ] CMSC: Full fetch complete (6965 records). [2025-10-20T18:00:45.058127] [DEBUG] [VALIDATOR ] CMSC: Validating primary data. [2025-10-20T18:00:45.056126] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-07. [2025-10-20T18:00:45.464046] [INFO ] [REMOTE ] CMTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:45.713794] [DEBUG] [VALIDATOR ] CMSC: All expected trading days are present. [2025-10-20T18:00:45.713922] [DEBUG] [VALIDATOR ] CMSC: Primary data validation passed. [2025-10-20T18:00:45.714301] [INFO ] [MAIN ] CMSC: Fetched primary data is valid. [2025-10-20T18:00:45.714344] [DEBUG] [LOCAL ] CMSC: Saving primary data. [2025-10-20T18:00:45.714914] [DEBUG] [LOCAL ] CMSC: Trimmed 6965 records to 4847 (keeping 200 days of data) [2025-10-20T18:00:45.747013] [DEBUG] [LOCAL ] CMSC: Primary data saved (4847 records). [2025-10-20T18:00:45.747199] [INFO ] [MAIN ] CMSC: Full history saved. [2025-10-20T18:00:45.747305] [INFO ] [MAIN ] Update Progress: 482/2367 symbols completed. CMSC took 22.2s. Estimated remaining: 3h 57m 21s. Rate: 7.9 symbols/min [2025-10-20T18:00:45.853258] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-01. [2025-10-20T18:00:45.927614] [INFO ] [MAIN ] CMU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:45.936897] [DEBUG] [LOCAL ] CMU: Loaded 6381 existing records from primary storage. [2025-10-20T18:00:45.937248] [INFO ] [MAIN ] CMU: Loaded 6381 existing records. [2025-10-20T18:00:46.125745] [WARN ] [MAIN ] CMU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:46.126055] [INFO ] [MAIN ] CMU: Wiping existing primary data before full refresh. [2025-10-20T18:00:46.126112] [INFO ] [LOCAL ] CMU: Starting deletion of primary data. [2025-10-20T18:00:46.126370] [DEBUG] [LOCAL ] CMU: Deleted primary data file. [2025-10-20T18:00:46.126443] [INFO ] [MAIN ] CMU: Starting full primary refresh. [2025-10-20T18:00:46.506636] [INFO ] [REMOTE ] CMU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:46.651453] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-07. [2025-10-20T18:00:46.764177] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T18:00:47.286075] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T18:00:47.452065] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-08. [2025-10-20T18:00:47.454369] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T18:00:47.929668] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:00:48.251355] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-08. [2025-10-20T18:00:49.051420] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-01. [2025-10-20T18:00:49.640802] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:00:49.773350] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:00:49.851366] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-09. [2025-10-20T18:00:50.651393] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-02. [2025-10-20T18:00:51.237137] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T18:00:51.451377] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-02. [2025-10-20T18:00:51.527190] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:00:52.251404] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-03. [2025-10-20T18:00:52.756495] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:00:53.019913] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T18:00:53.051349] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-09. [2025-10-20T18:00:53.685992] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:00:53.851418] [INFO ] [REMOTE ] CMS: Fetching primary data for 2025-10. [2025-10-20T18:00:54.651374] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-03. [2025-10-20T18:00:55.451362] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-04. [2025-10-20T18:00:55.705320] [INFO ] [REMOTE ] CMS: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:00:55.713300] [INFO ] [REMOTE ] CMS: Full fetch complete (76238 records). [2025-10-20T18:00:55.713494] [DEBUG] [VALIDATOR ] CMS: Validating primary data. [2025-10-20T18:00:56.146486] [DEBUG] [VALIDATOR ] CMS: All expected trading days are present. [2025-10-20T18:00:56.146683] [DEBUG] [VALIDATOR ] CMS: Primary data validation passed. [2025-10-20T18:00:56.147714] [INFO ] [MAIN ] CMS: Fetched primary data is valid. [2025-10-20T18:00:56.147807] [DEBUG] [LOCAL ] CMS: Saving primary data. [2025-10-20T18:00:56.149552] [DEBUG] [LOCAL ] CMS: Trimmed 76238 records to 52585 (keeping 200 days of data) [2025-10-20T18:00:56.252819] [INFO ] [REMOTE ] CMSD: Fetching primary data for 2025-10. [2025-10-20T18:00:56.268788] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:00:56.304526] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:00:56.530843] [DEBUG] [LOCAL ] CMS: Primary data saved (52585 records). [2025-10-20T18:00:56.530983] [INFO ] [MAIN ] CMS: Full history saved. [2025-10-20T18:00:56.531053] [INFO ] [MAIN ] Update Progress: 483/2367 symbols completed. CMS took 33.0s. Estimated remaining: 3h 57m 26s. Rate: 7.9 symbols/min [2025-10-20T18:00:56.703018] [INFO ] [MAIN ] CNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:56.742301] [DEBUG] [LOCAL ] CNA: Loaded 33357 existing records from primary storage. [2025-10-20T18:00:56.742734] [INFO ] [MAIN ] CNA: Loaded 33357 existing records. [2025-10-20T18:00:56.939618] [WARN ] [MAIN ] CNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:56.939949] [INFO ] [MAIN ] CNA: Wiping existing primary data before full refresh. [2025-10-20T18:00:56.940012] [INFO ] [LOCAL ] CNA: Starting deletion of primary data. [2025-10-20T18:00:56.940465] [DEBUG] [LOCAL ] CNA: Deleted primary data file. [2025-10-20T18:00:56.940506] [INFO ] [MAIN ] CNA: Starting full primary refresh. [2025-10-20T18:00:57.032417] [INFO ] [REMOTE ] CMSD: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T18:00:57.056944] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-04. [2025-10-20T18:00:57.087935] [INFO ] [REMOTE ] CMSD: Full fetch complete (11510 records). [2025-10-20T18:00:57.150692] [DEBUG] [VALIDATOR ] CMSD: Validating primary data. [2025-10-20T18:00:57.487756] [INFO ] [REMOTE ] CNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:57.708303] [DEBUG] [VALIDATOR ] CMSD: All expected trading days are present. [2025-10-20T18:00:57.708382] [DEBUG] [VALIDATOR ] CMSD: Primary data validation passed. [2025-10-20T18:00:57.708707] [INFO ] [MAIN ] CMSD: Fetched primary data is valid. [2025-10-20T18:00:57.708768] [DEBUG] [LOCAL ] CMSD: Saving primary data. [2025-10-20T18:00:57.709237] [DEBUG] [LOCAL ] CMSD: Trimmed 11510 records to 7682 (keeping 200 days of data) [2025-10-20T18:00:57.754229] [DEBUG] [LOCAL ] CMSD: Primary data saved (7682 records). [2025-10-20T18:00:57.754337] [INFO ] [MAIN ] CMSD: Full history saved. [2025-10-20T18:00:57.754411] [INFO ] [MAIN ] Update Progress: 484/2367 symbols completed. CMSD took 34.2s. Estimated remaining: 3h 56m 53s. Rate: 7.9 symbols/min [2025-10-20T18:00:57.853876] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-05. [2025-10-20T18:00:57.921159] [INFO ] [MAIN ] CNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:00:57.968342] [DEBUG] [LOCAL ] CNC: Loaded 54550 existing records from primary storage. [2025-10-20T18:00:57.968692] [INFO ] [MAIN ] CNC: Loaded 54550 existing records. [2025-10-20T18:00:58.144978] [WARN ] [MAIN ] CNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:00:58.145168] [INFO ] [MAIN ] CNC: Wiping existing primary data before full refresh. [2025-10-20T18:00:58.145213] [INFO ] [LOCAL ] CNC: Starting deletion of primary data. [2025-10-20T18:00:58.145509] [DEBUG] [LOCAL ] CNC: Deleted primary data file. [2025-10-20T18:00:58.145546] [INFO ] [MAIN ] CNC: Starting full primary refresh. [2025-10-20T18:00:58.250931] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T18:00:58.503204] [INFO ] [REMOTE ] CNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:00:58.651342] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-01. [2025-10-20T18:00:58.709909] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:00:59.451376] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-06. [2025-10-20T18:00:59.868316] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T18:00:59.990216] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:01:00.251365] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-05. [2025-10-20T18:01:01.051382] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-07. [2025-10-20T18:01:01.401903] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:01:01.851420] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-06. [2025-10-20T18:01:01.904131] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:01:02.651431] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-01. [2025-10-20T18:01:03.247436] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:01:03.451445] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-07. [2025-10-20T18:01:04.155177] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:01:04.251399] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-02. [2025-10-20T18:01:04.867161] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:01:05.051361] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-02. [2025-10-20T18:01:05.228177] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:01:05.851411] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-08. [2025-10-20T18:01:06.371955] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:01:06.470007] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T18:01:06.651359] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-08. [2025-10-20T18:01:07.451485] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-09. [2025-10-20T18:01:07.917925] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:01:08.078819] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:01:08.251473] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-03. [2025-10-20T18:01:09.051501] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-03. [2025-10-20T18:01:09.724878] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:01:09.851365] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-04. [2025-10-20T18:01:10.589142] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T18:01:10.651417] [INFO ] [REMOTE ] CMU: Fetching primary data for 2025-10. [2025-10-20T18:01:11.410863] [INFO ] [REMOTE ] CMU: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T18:01:11.411983] [INFO ] [REMOTE ] CMU: Full fetch complete (9484 records). [2025-10-20T18:01:11.412052] [DEBUG] [VALIDATOR ] CMU: Validating primary data. [2025-10-20T18:01:11.464666] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-04. [2025-10-20T18:01:11.800766] [DEBUG] [VALIDATOR ] CMU: All expected trading days are present. [2025-10-20T18:01:11.800892] [DEBUG] [VALIDATOR ] CMU: Primary data validation passed. [2025-10-20T18:01:11.801319] [INFO ] [MAIN ] CMU: Fetched primary data is valid. [2025-10-20T18:01:11.801444] [DEBUG] [LOCAL ] CMU: Saving primary data. [2025-10-20T18:01:11.802522] [DEBUG] [LOCAL ] CMU: Trimmed 9484 records to 6357 (keeping 200 days of data) [2025-10-20T18:01:11.877795] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:01:11.890313] [DEBUG] [LOCAL ] CMU: Primary data saved (6357 records). [2025-10-20T18:01:11.890918] [INFO ] [MAIN ] CMU: Full history saved. [2025-10-20T18:01:11.891150] [INFO ] [MAIN ] Update Progress: 485/2367 symbols completed. CMU took 26.1s. Estimated remaining: 3h 57m 11s. Rate: 7.9 symbols/min [2025-10-20T18:01:12.082598] [INFO ] [MAIN ] CNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:12.092953] [DEBUG] [LOCAL ] CNF: Loaded 10666 existing records from primary storage. [2025-10-20T18:01:12.093182] [INFO ] [MAIN ] CNF: Loaded 10666 existing records. [2025-10-20T18:01:12.256927] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-09. [2025-10-20T18:01:12.289287] [WARN ] [MAIN ] CNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:12.289506] [INFO ] [MAIN ] CNF: Wiping existing primary data before full refresh. [2025-10-20T18:01:12.289555] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-10-20T18:01:12.289734] [DEBUG] [LOCAL ] CNF: Deleted primary data file. [2025-10-20T18:01:12.289791] [INFO ] [MAIN ] CNF: Starting full primary refresh. [2025-10-20T18:01:12.671073] [INFO ] [REMOTE ] CNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:12.804209] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:01:13.051418] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-05. [2025-10-20T18:01:13.256386] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T18:01:13.851412] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-01. [2025-10-20T18:01:14.328106] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T18:01:14.494317] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T18:01:14.651411] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-05. [2025-10-20T18:01:15.451432] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-02. [2025-10-20T18:01:15.852427] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-10-20T18:01:16.251428] [INFO ] [REMOTE ] CMTG: Fetching primary data for 2025-10. [2025-10-20T18:01:16.820938] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T18:01:17.051382] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-06. [2025-10-20T18:01:17.515102] [INFO ] [REMOTE ] CMTG: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:01:17.519487] [INFO ] [REMOTE ] CMTG: Full fetch complete (53040 records). [2025-10-20T18:01:17.519582] [DEBUG] [VALIDATOR ] CMTG: Validating primary data. [2025-10-20T18:01:17.520251] [FATAL] [VALIDATOR ] CMTG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:01:17.520337] [FATAL] [MAIN ] CMTG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:01:17.520380] [WARN ] [MAIN ] CMTG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:01:17.520448] [INFO ] [LOCAL ] CMTG: Starting deletion of primary data. [2025-10-20T18:01:17.520516] [DEBUG] [LOCAL ] CMTG: No primary data file found to delete. [2025-10-20T18:01:17.520553] [INFO ] [LOCAL ] CMTG: Starting deletion of derivative data. [2025-10-20T18:01:17.520707] [DEBUG] [LOCAL ] CMTG: Deleted derivative data file. [2025-10-20T18:01:17.520795] [INFO ] [MAIN ] Update Progress: 486/2367 symbols completed. CMTG took 33.0s. Estimated remaining: 3h 56m 56s. Rate: 7.9 symbols/min [2025-10-20T18:01:17.685145] [INFO ] [MAIN ] CNH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:17.736918] [DEBUG] [LOCAL ] CNH: Loaded 54332 existing records from primary storage. [2025-10-20T18:01:17.737310] [INFO ] [MAIN ] CNH: Loaded 54332 existing records. [2025-10-20T18:01:17.856547] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-06. [2025-10-20T18:01:17.913571] [WARN ] [MAIN ] CNH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:17.913961] [INFO ] [MAIN ] CNH: Wiping existing primary data before full refresh. [2025-10-20T18:01:17.914027] [INFO ] [LOCAL ] CNH: Starting deletion of primary data. [2025-10-20T18:01:17.914323] [DEBUG] [LOCAL ] CNH: Deleted primary data file. [2025-10-20T18:01:17.914378] [INFO ] [MAIN ] CNH: Starting full primary refresh. [2025-10-20T18:01:18.265143] [INFO ] [REMOTE ] CNH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:18.357944] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:01:18.651421] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-07. [2025-10-20T18:01:19.460981] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-01. [2025-10-20T18:01:19.465236] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:01:20.251397] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-03. [2025-10-20T18:01:20.807664] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:01:21.028045] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:01:21.051331] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-02. [2025-10-20T18:01:21.060863] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T18:01:21.851398] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-04. [2025-10-20T18:01:22.430736] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:01:22.557495] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:01:22.651393] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-03. [2025-10-20T18:01:23.451381] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-07. [2025-10-20T18:01:24.251365] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-05. [2025-10-20T18:01:24.500575] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:01:24.907728] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:01:24.945482] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:01:25.051413] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-08. [2025-10-20T18:01:25.851383] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-08. [2025-10-20T18:01:26.651391] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-04. [2025-10-20T18:01:26.901386] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:01:26.938058] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:01:27.451397] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-09. [2025-10-20T18:01:28.251408] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-06. [2025-10-20T18:01:28.519671] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:01:28.774578] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:01:29.051425] [INFO ] [REMOTE ] CNA: Fetching primary data for 2025-10. [2025-10-20T18:01:29.178989] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:01:29.851441] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-05. [2025-10-20T18:01:30.169131] [INFO ] [REMOTE ] CNA: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:01:30.175098] [INFO ] [REMOTE ] CNA: Full fetch complete (46129 records). [2025-10-20T18:01:30.175291] [DEBUG] [VALIDATOR ] CNA: Validating primary data. [2025-10-20T18:01:30.560828] [DEBUG] [VALIDATOR ] CNA: All expected trading days are present. [2025-10-20T18:01:30.560995] [DEBUG] [VALIDATOR ] CNA: Primary data validation passed. [2025-10-20T18:01:30.561713] [INFO ] [MAIN ] CNA: Fetched primary data is valid. [2025-10-20T18:01:30.561776] [DEBUG] [LOCAL ] CNA: Saving primary data. [2025-10-20T18:01:30.562883] [DEBUG] [LOCAL ] CNA: Trimmed 46129 records to 32763 (keeping 200 days of data) [2025-10-20T18:01:30.655791] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-09. [2025-10-20T18:01:30.762784] [DEBUG] [LOCAL ] CNA: Primary data saved (32763 records). [2025-10-20T18:01:30.762931] [INFO ] [MAIN ] CNA: Full history saved. [2025-10-20T18:01:30.763013] [INFO ] [MAIN ] Update Progress: 487/2367 symbols completed. CNA took 34.2s. Estimated remaining: 3h 57m 11s. Rate: 7.9 symbols/min [2025-10-20T18:01:30.954815] [INFO ] [MAIN ] CNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:31.005540] [DEBUG] [LOCAL ] CNI: Loaded 52456 existing records from primary storage. [2025-10-20T18:01:31.005989] [INFO ] [MAIN ] CNI: Loaded 52456 existing records. [2025-10-20T18:01:31.196114] [WARN ] [MAIN ] CNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:31.196339] [INFO ] [MAIN ] CNI: Wiping existing primary data before full refresh. [2025-10-20T18:01:31.196411] [INFO ] [LOCAL ] CNI: Starting deletion of primary data. [2025-10-20T18:01:31.196862] [DEBUG] [LOCAL ] CNI: Deleted primary data file. [2025-10-20T18:01:31.196934] [INFO ] [MAIN ] CNI: Starting full primary refresh. [2025-10-20T18:01:31.456148] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-07. [2025-10-20T18:01:31.613080] [INFO ] [REMOTE ] CNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:31.751057] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:01:32.251414] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-06. [2025-10-20T18:01:32.260451] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:01:32.597046] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T18:01:33.051363] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-08. [2025-10-20T18:01:33.655899] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:01:33.851345] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-09. [2025-10-20T18:01:33.930951] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T18:01:34.513065] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T18:01:34.651343] [INFO ] [REMOTE ] CNF: Fetching primary data for 2025-10. [2025-10-20T18:01:35.451344] [INFO ] [REMOTE ] CNC: Fetching primary data for 2025-10. [2025-10-20T18:01:35.635528] [INFO ] [REMOTE ] CNF: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:01:35.637029] [INFO ] [REMOTE ] CNF: Full fetch complete (13022 records). [2025-10-20T18:01:35.637100] [DEBUG] [VALIDATOR ] CNF: Validating primary data. [2025-10-20T18:01:35.637487] [FATAL] [VALIDATOR ] CNF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:01:35.637570] [FATAL] [MAIN ] CNF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:01:35.637620] [WARN ] [MAIN ] CNF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:01:35.637736] [INFO ] [LOCAL ] CNF: Starting deletion of primary data. [2025-10-20T18:01:35.637816] [DEBUG] [LOCAL ] CNF: No primary data file found to delete. [2025-10-20T18:01:35.637863] [INFO ] [LOCAL ] CNF: Starting deletion of derivative data. [2025-10-20T18:01:35.638058] [DEBUG] [LOCAL ] CNF: Deleted derivative data file. [2025-10-20T18:01:35.638158] [INFO ] [MAIN ] Update Progress: 488/2367 symbols completed. CNF took 23.7s. Estimated remaining: 3h 56m 53s. Rate: 7.9 symbols/min [2025-10-20T18:01:35.833313] [INFO ] [MAIN ] CNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:35.879948] [DEBUG] [LOCAL ] CNK: Loaded 53677 existing records from primary storage. [2025-10-20T18:01:35.880266] [INFO ] [MAIN ] CNK: Loaded 53677 existing records. [2025-10-20T18:01:36.086958] [WARN ] [MAIN ] CNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:36.087212] [INFO ] [MAIN ] CNK: Wiping existing primary data before full refresh. [2025-10-20T18:01:36.087269] [INFO ] [LOCAL ] CNK: Starting deletion of primary data. [2025-10-20T18:01:36.087753] [DEBUG] [LOCAL ] CNK: Deleted primary data file. [2025-10-20T18:01:36.087810] [INFO ] [MAIN ] CNK: Starting full primary refresh. [2025-10-20T18:01:36.260541] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-07. [2025-10-20T18:01:36.462361] [INFO ] [REMOTE ] CNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:36.485717] [INFO ] [REMOTE ] CNC: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:01:36.494162] [INFO ] [REMOTE ] CNC: Full fetch complete (77850 records). [2025-10-20T18:01:36.494327] [DEBUG] [VALIDATOR ] CNC: Validating primary data. [2025-10-20T18:01:36.915958] [DEBUG] [VALIDATOR ] CNC: All expected trading days are present. [2025-10-20T18:01:36.916039] [DEBUG] [VALIDATOR ] CNC: Primary data validation passed. [2025-10-20T18:01:36.916869] [INFO ] [MAIN ] CNC: Fetched primary data is valid. [2025-10-20T18:01:36.916931] [DEBUG] [LOCAL ] CNC: Saving primary data. [2025-10-20T18:01:36.918351] [DEBUG] [LOCAL ] CNC: Trimmed 77850 records to 53778 (keeping 200 days of data) [2025-10-20T18:01:37.084229] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-01. [2025-10-20T18:01:37.276566] [DEBUG] [LOCAL ] CNC: Primary data saved (53778 records). [2025-10-20T18:01:37.276813] [INFO ] [MAIN ] CNC: Full history saved. [2025-10-20T18:01:37.276923] [INFO ] [MAIN ] Update Progress: 489/2367 symbols completed. CNC took 39.5s. Estimated remaining: 3h 56m 22s. Rate: 7.9 symbols/min [2025-10-20T18:01:37.468489] [INFO ] [MAIN ] CNM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:37.523978] [DEBUG] [LOCAL ] CNM: Loaded 51797 existing records from primary storage. [2025-10-20T18:01:37.524405] [INFO ] [MAIN ] CNM: Loaded 51797 existing records. [2025-10-20T18:01:37.715216] [WARN ] [MAIN ] CNM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:37.715547] [INFO ] [MAIN ] CNM: Wiping existing primary data before full refresh. [2025-10-20T18:01:37.715613] [INFO ] [LOCAL ] CNM: Starting deletion of primary data. [2025-10-20T18:01:37.715983] [DEBUG] [LOCAL ] CNM: Deleted primary data file. [2025-10-20T18:01:37.716051] [INFO ] [MAIN ] CNM: Starting full primary refresh. [2025-10-20T18:01:37.866756] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-01. [2025-10-20T18:01:38.108296] [INFO ] [REMOTE ] CNM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:38.212521] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T18:01:38.613183] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:01:38.651404] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-08. [2025-10-20T18:01:39.451396] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-02. [2025-10-20T18:01:39.821616] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T18:01:40.251478] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-01. [2025-10-20T18:01:40.382079] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:01:41.058292] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-02. [2025-10-20T18:01:41.065117] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:01:41.851386] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-03. [2025-10-20T18:01:42.004726] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T18:01:42.651493] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-02. [2025-10-20T18:01:42.769294] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T18:01:43.451372] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-03. [2025-10-20T18:01:43.849804] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:01:44.193600] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:01:44.251406] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-04. [2025-10-20T18:01:44.919791] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:01:45.051411] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-04. [2025-10-20T18:01:45.851384] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-03. [2025-10-20T18:01:45.994521] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:01:46.651380] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-09. [2025-10-20T18:01:46.875424] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:01:47.075565] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:01:47.451386] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-04. [2025-10-20T18:01:48.251401] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-05. [2025-10-20T18:01:48.574791] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:01:49.051392] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-05. [2025-10-20T18:01:49.224049] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:01:49.851385] [INFO ] [REMOTE ] CNH: Fetching primary data for 2025-10. [2025-10-20T18:01:50.098098] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:01:50.651386] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-05. [2025-10-20T18:01:50.912518] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:01:51.451348] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-06. [2025-10-20T18:01:51.597469] [INFO ] [REMOTE ] CNH: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:01:51.605340] [INFO ] [REMOTE ] CNH: Full fetch complete (77704 records). [2025-10-20T18:01:51.605472] [DEBUG] [VALIDATOR ] CNH: Validating primary data. [2025-10-20T18:01:52.000491] [DEBUG] [VALIDATOR ] CNH: All expected trading days are present. [2025-10-20T18:01:52.000588] [DEBUG] [VALIDATOR ] CNH: Primary data validation passed. [2025-10-20T18:01:52.001478] [INFO ] [MAIN ] CNH: Fetched primary data is valid. [2025-10-20T18:01:52.001540] [DEBUG] [LOCAL ] CNH: Saving primary data. [2025-10-20T18:01:52.003058] [DEBUG] [LOCAL ] CNH: Trimmed 77704 records to 53548 (keeping 200 days of data) [2025-10-20T18:01:52.256796] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-06. [2025-10-20T18:01:52.380239] [DEBUG] [LOCAL ] CNH: Primary data saved (53548 records). [2025-10-20T18:01:52.384060] [INFO ] [MAIN ] CNH: Full history saved. [2025-10-20T18:01:52.384216] [INFO ] [MAIN ] Update Progress: 490/2367 symbols completed. CNH took 34.9s. Estimated remaining: 3h 56m 44s. Rate: 7.9 symbols/min [2025-10-20T18:01:52.456631] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:01:52.570916] [INFO ] [MAIN ] CNMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:01:52.601273] [DEBUG] [LOCAL ] CNMD: Loaded 32497 existing records from primary storage. [2025-10-20T18:01:52.601577] [INFO ] [MAIN ] CNMD: Loaded 32497 existing records. [2025-10-20T18:01:52.778832] [WARN ] [MAIN ] CNMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:01:52.779098] [INFO ] [MAIN ] CNMD: Wiping existing primary data before full refresh. [2025-10-20T18:01:52.779176] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-10-20T18:01:52.779772] [DEBUG] [LOCAL ] CNMD: Deleted primary data file. [2025-10-20T18:01:52.779855] [INFO ] [MAIN ] CNMD: Starting full primary refresh. [2025-10-20T18:01:53.054249] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:01:53.052486] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-06. [2025-10-20T18:01:53.192617] [INFO ] [REMOTE ] CNMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:01:53.851373] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-07. [2025-10-20T18:01:54.059113] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:01:54.651389] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-01. [2025-10-20T18:01:54.800259] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T18:01:55.451378] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-07. [2025-10-20T18:01:55.677714] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T18:01:56.057907] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T18:01:56.251378] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-07. [2025-10-20T18:01:57.051387] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-02. [2025-10-20T18:01:57.144264] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:01:57.851366] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-08. [2025-10-20T18:01:57.981226] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T18:01:58.154137] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T18:01:58.651435] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-08. [2025-10-20T18:01:59.348417] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:01:59.451380] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-09. [2025-10-20T18:02:00.251410] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-03. [2025-10-20T18:02:00.511107] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:02:01.051352] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-09. [2025-10-20T18:02:01.160536] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:02:01.569487] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:02:01.851362] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-04. [2025-10-20T18:02:02.635952] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:02:02.651351] [INFO ] [REMOTE ] CNM: Fetching primary data for 2025-10. [2025-10-20T18:02:03.104052] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:02:03.451390] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-05. [2025-10-20T18:02:04.251403] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-08. [2025-10-20T18:02:04.537957] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:02:04.611074] [INFO ] [REMOTE ] CNM: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T18:02:04.618632] [INFO ] [REMOTE ] CNM: Full fetch complete (73783 records). [2025-10-20T18:02:04.618803] [DEBUG] [VALIDATOR ] CNM: Validating primary data. [2025-10-20T18:02:04.995359] [DEBUG] [VALIDATOR ] CNM: All expected trading days are present. [2025-10-20T18:02:04.995440] [DEBUG] [VALIDATOR ] CNM: Primary data validation passed. [2025-10-20T18:02:04.996273] [INFO ] [MAIN ] CNM: Fetched primary data is valid. [2025-10-20T18:02:04.996337] [DEBUG] [LOCAL ] CNM: Saving primary data. [2025-10-20T18:02:04.997720] [DEBUG] [LOCAL ] CNM: Trimmed 73783 records to 51031 (keeping 200 days of data) [2025-10-20T18:02:05.053758] [INFO ] [REMOTE ] CNK: Fetching primary data for 2025-10. [2025-10-20T18:02:05.290300] [DEBUG] [LOCAL ] CNM: Primary data saved (51031 records). [2025-10-20T18:02:05.290438] [INFO ] [MAIN ] CNM: Full history saved. [2025-10-20T18:02:05.290515] [INFO ] [MAIN ] Update Progress: 491/2367 symbols completed. CNM took 28.0s. Estimated remaining: 3h 56m 57s. Rate: 7.9 symbols/min [2025-10-20T18:02:05.457142] [INFO ] [MAIN ] CNNE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:05.457299] [DEBUG] [LOCAL ] CNNE: No existing data file found in primary storage. [2025-10-20T18:02:05.457357] [WARN ] [MAIN ] CNNE: No existing data found, performing full refresh instead. [2025-10-20T18:02:05.457465] [INFO ] [MAIN ] CNNE: Wiping existing primary data before full refresh. [2025-10-20T18:02:05.457505] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-10-20T18:02:05.457545] [DEBUG] [LOCAL ] CNNE: No primary data file found to delete. [2025-10-20T18:02:05.457575] [INFO ] [MAIN ] CNNE: Starting full primary refresh. [2025-10-20T18:02:05.800190] [INFO ] [REMOTE ] CNNE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:05.851334] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-01. [2025-10-20T18:02:06.169085] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T18:02:06.651380] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-06. [2025-10-20T18:02:06.962236] [INFO ] [REMOTE ] CNK: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:02:06.970478] [INFO ] [REMOTE ] CNK: Full fetch complete (76221 records). [2025-10-20T18:02:06.970692] [DEBUG] [VALIDATOR ] CNK: Validating primary data. [2025-10-20T18:02:07.289351] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:02:07.391565] [DEBUG] [VALIDATOR ] CNK: All expected trading days are present. [2025-10-20T18:02:07.391697] [DEBUG] [VALIDATOR ] CNK: Primary data validation passed. [2025-10-20T18:02:07.392600] [INFO ] [MAIN ] CNK: Fetched primary data is valid. [2025-10-20T18:02:07.392664] [DEBUG] [LOCAL ] CNK: Saving primary data. [2025-10-20T18:02:07.394224] [DEBUG] [LOCAL ] CNK: Trimmed 76221 records to 52920 (keeping 200 days of data) [2025-10-20T18:02:07.455433] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-02. [2025-10-20T18:02:07.701228] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:02:07.747996] [DEBUG] [LOCAL ] CNK: Primary data saved (52920 records). [2025-10-20T18:02:07.748161] [INFO ] [MAIN ] CNK: Full history saved. [2025-10-20T18:02:07.748474] [INFO ] [MAIN ] Update Progress: 492/2367 symbols completed. CNK took 32.1s. Estimated remaining: 3h 56m 30s. Rate: 7.9 symbols/min [2025-10-20T18:02:07.919576] [INFO ] [MAIN ] CNO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:07.955333] [DEBUG] [LOCAL ] CNO: Loaded 41237 existing records from primary storage. [2025-10-20T18:02:07.955618] [INFO ] [MAIN ] CNO: Loaded 41237 existing records. [2025-10-20T18:02:08.127798] [WARN ] [MAIN ] CNO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:08.128030] [INFO ] [MAIN ] CNO: Wiping existing primary data before full refresh. [2025-10-20T18:02:08.128077] [INFO ] [LOCAL ] CNO: Starting deletion of primary data. [2025-10-20T18:02:08.128334] [DEBUG] [LOCAL ] CNO: Deleted primary data file. [2025-10-20T18:02:08.128372] [INFO ] [MAIN ] CNO: Starting full primary refresh. [2025-10-20T18:02:08.256632] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-07. [2025-10-20T18:02:08.484895] [INFO ] [REMOTE ] CNO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:08.702588] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T18:02:09.051373] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-03. [2025-10-20T18:02:09.819778] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:02:09.851447] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-08. [2025-10-20T18:02:10.230885] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T18:02:10.651507] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-01. [2025-10-20T18:02:10.765907] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T18:02:11.451420] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-09. [2025-10-20T18:02:12.259051] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-04. [2025-10-20T18:02:12.263872] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:02:12.873450] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:02:13.051379] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-09. [2025-10-20T18:02:13.279315] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:02:13.851387] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-05. [2025-10-20T18:02:14.651382] [INFO ] [REMOTE ] CNMD: Fetching primary data for 2025-10. [2025-10-20T18:02:14.787396] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T18:02:14.867032] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:02:15.451486] [INFO ] [REMOTE ] CNI: Fetching primary data for 2025-10. [2025-10-20T18:02:15.664259] [INFO ] [REMOTE ] CNMD: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:02:15.668448] [INFO ] [REMOTE ] CNMD: Full fetch complete (47029 records). [2025-10-20T18:02:15.668592] [DEBUG] [VALIDATOR ] CNMD: Validating primary data. [2025-10-20T18:02:15.669237] [FATAL] [VALIDATOR ] CNMD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:15.669351] [FATAL] [MAIN ] CNMD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:15.669403] [WARN ] [MAIN ] CNMD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:15.669441] [INFO ] [LOCAL ] CNMD: Starting deletion of primary data. [2025-10-20T18:02:15.669495] [DEBUG] [LOCAL ] CNMD: No primary data file found to delete. [2025-10-20T18:02:15.669528] [INFO ] [LOCAL ] CNMD: Starting deletion of derivative data. [2025-10-20T18:02:15.669722] [DEBUG] [LOCAL ] CNMD: Deleted derivative data file. [2025-10-20T18:02:15.669809] [INFO ] [MAIN ] Update Progress: 493/2367 symbols completed. CNMD took 23.3s. Estimated remaining: 3h 56m 23s. Rate: 7.9 symbols/min [2025-10-20T18:02:15.842629] [INFO ] [MAIN ] CNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:15.892946] [DEBUG] [LOCAL ] CNP: Loaded 54258 existing records from primary storage. [2025-10-20T18:02:15.893275] [INFO ] [MAIN ] CNP: Loaded 54258 existing records. [2025-10-20T18:02:16.072245] [WARN ] [MAIN ] CNP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:16.072460] [INFO ] [MAIN ] CNP: Wiping existing primary data before full refresh. [2025-10-20T18:02:16.072547] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-10-20T18:02:16.072781] [DEBUG] [LOCAL ] CNP: Deleted primary data file. [2025-10-20T18:02:16.072835] [INFO ] [MAIN ] CNP: Starting full primary refresh. [2025-10-20T18:02:16.259172] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-02. [2025-10-20T18:02:16.496933] [INFO ] [REMOTE ] CNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:17.051397] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-06. [2025-10-20T18:02:17.555023] [INFO ] [REMOTE ] CNI: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T18:02:17.563160] [INFO ] [REMOTE ] CNI: Full fetch complete (74850 records). [2025-10-20T18:02:17.563407] [DEBUG] [VALIDATOR ] CNI: Validating primary data. [2025-10-20T18:02:17.861492] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-01. [2025-10-20T18:02:17.983550] [DEBUG] [VALIDATOR ] CNI: All expected trading days are present. [2025-10-20T18:02:17.983706] [DEBUG] [VALIDATOR ] CNI: Primary data validation passed. [2025-10-20T18:02:17.984541] [INFO ] [MAIN ] CNI: Fetched primary data is valid. [2025-10-20T18:02:17.984616] [DEBUG] [LOCAL ] CNI: Saving primary data. [2025-10-20T18:02:18.014292] [DEBUG] [LOCAL ] CNI: Trimmed 74850 records to 51678 (keeping 200 days of data) [2025-10-20T18:02:18.015850] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:02:18.274122] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:02:18.390617] [DEBUG] [LOCAL ] CNI: Primary data saved (51678 records). [2025-10-20T18:02:18.390805] [INFO ] [MAIN ] CNI: Full history saved. [2025-10-20T18:02:18.390899] [INFO ] [MAIN ] Update Progress: 494/2367 symbols completed. CNI took 47.6s. Estimated remaining: 3h 55m 57s. Rate: 7.9 symbols/min [2025-10-20T18:02:18.572076] [INFO ] [MAIN ] CNQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:18.619162] [DEBUG] [LOCAL ] CNQ: Loaded 54362 existing records from primary storage. [2025-10-20T18:02:18.619493] [INFO ] [MAIN ] CNQ: Loaded 54362 existing records. [2025-10-20T18:02:18.656571] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-07. [2025-10-20T18:02:18.798468] [WARN ] [MAIN ] CNQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:18.798759] [INFO ] [MAIN ] CNQ: Wiping existing primary data before full refresh. [2025-10-20T18:02:18.798808] [INFO ] [LOCAL ] CNQ: Starting deletion of primary data. [2025-10-20T18:02:18.799248] [DEBUG] [LOCAL ] CNQ: Deleted primary data file. [2025-10-20T18:02:18.799304] [INFO ] [MAIN ] CNQ: Starting full primary refresh. [2025-10-20T18:02:19.153041] [INFO ] [REMOTE ] CNQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:19.451344] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-01. [2025-10-20T18:02:19.752798] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:02:19.811492] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:02:20.251419] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-02. [2025-10-20T18:02:21.051388] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-08. [2025-10-20T18:02:21.363371] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T18:02:21.851378] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-02. [2025-10-20T18:02:21.880879] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:02:22.324480] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:02:22.651403] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-03. [2025-10-20T18:02:23.293239] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T18:02:23.451455] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-03. [2025-10-20T18:02:24.141909] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:02:24.251375] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-03. [2025-10-20T18:02:25.051407] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-04. [2025-10-20T18:02:25.298381] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:02:25.851459] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-04. [2025-10-20T18:02:26.088251] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T18:02:26.651446] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-04. [2025-10-20T18:02:26.750932] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:02:27.451377] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-05. [2025-10-20T18:02:27.809989] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:02:28.015510] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:02:28.251366] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-05. [2025-10-20T18:02:29.051344] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-05. [2025-10-20T18:02:29.097542] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:02:29.749915] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:02:29.851375] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-06. [2025-10-20T18:02:30.651389] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-09. [2025-10-20T18:02:30.858939] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:02:31.216467] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:02:31.451437] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-07. [2025-10-20T18:02:32.233073] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:02:32.251364] [INFO ] [REMOTE ] CNNE: Fetching primary data for 2025-10. [2025-10-20T18:02:33.012413] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:02:33.051455] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-08. [2025-10-20T18:02:33.584568] [INFO ] [REMOTE ] CNNE: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:02:33.592922] [INFO ] [REMOTE ] CNNE: Full fetch complete (50340 records). [2025-10-20T18:02:33.593164] [DEBUG] [VALIDATOR ] CNNE: Validating primary data. [2025-10-20T18:02:33.594453] [FATAL] [VALIDATOR ] CNNE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:33.594566] [FATAL] [MAIN ] CNNE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:33.594618] [WARN ] [MAIN ] CNNE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:33.594715] [INFO ] [LOCAL ] CNNE: Starting deletion of primary data. [2025-10-20T18:02:33.594852] [DEBUG] [LOCAL ] CNNE: No primary data file found to delete. [2025-10-20T18:02:33.594922] [INFO ] [LOCAL ] CNNE: Starting deletion of derivative data. [2025-10-20T18:02:33.594976] [DEBUG] [LOCAL ] CNNE: No derivative data file found to delete. [2025-10-20T18:02:33.595057] [INFO ] [MAIN ] Update Progress: 495/2367 symbols completed. CNNE took 28.3s. Estimated remaining: 3h 56m 19s. Rate: 7.9 symbols/min [2025-10-20T18:02:33.839137] [INFO ] [MAIN ] CNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:33.839320] [DEBUG] [LOCAL ] CNR: No existing data file found in primary storage. [2025-10-20T18:02:33.839410] [WARN ] [MAIN ] CNR: No existing data found, performing full refresh instead. [2025-10-20T18:02:33.839495] [INFO ] [MAIN ] CNR: Wiping existing primary data before full refresh. [2025-10-20T18:02:33.839532] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-10-20T18:02:33.839572] [DEBUG] [LOCAL ] CNR: No primary data file found to delete. [2025-10-20T18:02:33.839603] [INFO ] [MAIN ] CNR: Starting full primary refresh. [2025-10-20T18:02:33.859857] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-06. [2025-10-20T18:02:34.227224] [INFO ] [REMOTE ] CNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:34.280884] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T18:02:34.651383] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-06. [2025-10-20T18:02:35.451411] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-09. [2025-10-20T18:02:35.915258] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T18:02:36.251428] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-07. [2025-10-20T18:02:36.640351] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T18:02:37.051452] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-01. [2025-10-20T18:02:37.272137] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:02:37.860100] [INFO ] [REMOTE ] CNO: Fetching primary data for 2025-10. [2025-10-20T18:02:37.867798] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:02:38.484086] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:02:38.651481] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-08. [2025-10-20T18:02:39.388985] [INFO ] [REMOTE ] CNO: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T18:02:39.395671] [INFO ] [REMOTE ] CNO: Full fetch complete (60822 records). [2025-10-20T18:02:39.395845] [DEBUG] [VALIDATOR ] CNO: Validating primary data. [2025-10-20T18:02:39.463162] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-07. [2025-10-20T18:02:39.814291] [DEBUG] [VALIDATOR ] CNO: All expected trading days are present. [2025-10-20T18:02:39.814380] [DEBUG] [VALIDATOR ] CNO: Primary data validation passed. [2025-10-20T18:02:39.815219] [INFO ] [MAIN ] CNO: Fetched primary data is valid. [2025-10-20T18:02:39.815284] [DEBUG] [LOCAL ] CNO: Saving primary data. [2025-10-20T18:02:39.816484] [DEBUG] [LOCAL ] CNO: Trimmed 60822 records to 40510 (keeping 200 days of data) [2025-10-20T18:02:40.058007] [DEBUG] [LOCAL ] CNO: Primary data saved (40510 records). [2025-10-20T18:02:40.058209] [INFO ] [MAIN ] CNO: Full history saved. [2025-10-20T18:02:40.058673] [INFO ] [MAIN ] Update Progress: 496/2367 symbols completed. CNO took 32.3s. Estimated remaining: 3h 56m 7s. Rate: 7.9 symbols/min [2025-10-20T18:02:40.253495] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-02. [2025-10-20T18:02:40.258295] [INFO ] [MAIN ] CNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:40.258581] [DEBUG] [LOCAL ] CNS: No existing data file found in primary storage. [2025-10-20T18:02:40.258705] [WARN ] [MAIN ] CNS: No existing data found, performing full refresh instead. [2025-10-20T18:02:40.258821] [INFO ] [MAIN ] CNS: Wiping existing primary data before full refresh. [2025-10-20T18:02:40.258862] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-10-20T18:02:40.258925] [DEBUG] [LOCAL ] CNS: No primary data file found to delete. [2025-10-20T18:02:40.258984] [INFO ] [MAIN ] CNS: Starting full primary refresh. [2025-10-20T18:02:40.650249] [INFO ] [REMOTE ] CNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:40.683113] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T18:02:41.039666] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:02:41.051423] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-09. [2025-10-20T18:02:41.861404] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-08. [2025-10-20T18:02:41.868088] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:02:42.528170] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:02:42.651427] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-01. [2025-10-20T18:02:43.451405] [INFO ] [REMOTE ] CNP: Fetching primary data for 2025-10. [2025-10-20T18:02:43.675836] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:02:43.843176] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T18:02:44.251414] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-03. [2025-10-20T18:02:44.953123] [INFO ] [REMOTE ] CNP: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:02:44.961199] [INFO ] [REMOTE ] CNP: Full fetch complete (77161 records). [2025-10-20T18:02:44.961390] [DEBUG] [VALIDATOR ] CNP: Validating primary data. [2025-10-20T18:02:44.962948] [FATAL] [VALIDATOR ] CNP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:02:44.963046] [FATAL] [MAIN ] CNP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:02:44.963091] [WARN ] [MAIN ] CNP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:02:44.963135] [INFO ] [LOCAL ] CNP: Starting deletion of primary data. [2025-10-20T18:02:44.963202] [DEBUG] [LOCAL ] CNP: No primary data file found to delete. [2025-10-20T18:02:44.963242] [INFO ] [LOCAL ] CNP: Starting deletion of derivative data. [2025-10-20T18:02:44.963398] [DEBUG] [LOCAL ] CNP: Deleted derivative data file. [2025-10-20T18:02:44.963487] [INFO ] [MAIN ] Update Progress: 497/2367 symbols completed. CNP took 29.3s. Estimated remaining: 3h 55m 49s. Rate: 7.9 symbols/min [2025-10-20T18:02:45.056481] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-09. [2025-10-20T18:02:45.154900] [INFO ] [MAIN ] CNX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:45.207936] [DEBUG] [LOCAL ] CNX: Loaded 52093 existing records from primary storage. [2025-10-20T18:02:45.208352] [INFO ] [MAIN ] CNX: Loaded 52093 existing records. [2025-10-20T18:02:45.395341] [WARN ] [MAIN ] CNX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:45.395623] [INFO ] [MAIN ] CNX: Wiping existing primary data before full refresh. [2025-10-20T18:02:45.395707] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-10-20T18:02:45.395958] [DEBUG] [LOCAL ] CNX: Deleted primary data file. [2025-10-20T18:02:45.396001] [INFO ] [MAIN ] CNX: Starting full primary refresh. [2025-10-20T18:02:45.798018] [INFO ] [REMOTE ] CNX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:45.851364] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-02. [2025-10-20T18:02:45.956179] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:02:46.651362] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-01. [2025-10-20T18:02:46.926113] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:02:47.032615] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T18:02:47.451424] [INFO ] [REMOTE ] CNQ: Fetching primary data for 2025-10. [2025-10-20T18:02:48.177185] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:02:48.251418] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-03. [2025-10-20T18:02:49.051407] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-02. [2025-10-20T18:02:49.307982] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T18:02:49.415509] [INFO ] [REMOTE ] CNQ: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T18:02:49.424976] [INFO ] [REMOTE ] CNQ: Full fetch complete (77740 records). [2025-10-20T18:02:49.425357] [DEBUG] [VALIDATOR ] CNQ: Validating primary data. [2025-10-20T18:02:49.851996] [DEBUG] [VALIDATOR ] CNQ: All expected trading days are present. [2025-10-20T18:02:49.851547] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-04. [2025-10-20T18:02:49.853096] [DEBUG] [VALIDATOR ] CNQ: Primary data validation passed. [2025-10-20T18:02:49.854080] [INFO ] [MAIN ] CNQ: Fetched primary data is valid. [2025-10-20T18:02:49.854255] [DEBUG] [LOCAL ] CNQ: Saving primary data. [2025-10-20T18:02:49.856210] [DEBUG] [LOCAL ] CNQ: Trimmed 77740 records to 53583 (keeping 200 days of data) [2025-10-20T18:02:50.191222] [DEBUG] [LOCAL ] CNQ: Primary data saved (53583 records). [2025-10-20T18:02:50.191400] [INFO ] [MAIN ] CNQ: Full history saved. [2025-10-20T18:02:50.191490] [INFO ] [MAIN ] Update Progress: 498/2367 symbols completed. CNQ took 31.8s. Estimated remaining: 3h 55m 33s. Rate: 7.9 symbols/min [2025-10-20T18:02:50.371373] [INFO ] [MAIN ] CODI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:02:50.412251] [DEBUG] [LOCAL ] CODI: Loaded 42174 existing records from primary storage. [2025-10-20T18:02:50.412766] [INFO ] [MAIN ] CODI: Loaded 42174 existing records. [2025-10-20T18:02:50.640891] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:02:50.662738] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-04. [2025-10-20T18:02:50.667108] [WARN ] [MAIN ] CODI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:02:50.684802] [INFO ] [MAIN ] CODI: Wiping existing primary data before full refresh. [2025-10-20T18:02:50.685278] [INFO ] [LOCAL ] CODI: Starting deletion of primary data. [2025-10-20T18:02:50.685517] [DEBUG] [LOCAL ] CODI: Deleted primary data file. [2025-10-20T18:02:50.685575] [INFO ] [MAIN ] CODI: Starting full primary refresh. [2025-10-20T18:02:51.088426] [INFO ] [REMOTE ] CODI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:02:51.105746] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:02:51.451377] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-01. [2025-10-20T18:02:52.251390] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-05. [2025-10-20T18:02:52.278954] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T18:02:52.458521] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:02:52.898374] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:02:53.051378] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-03. [2025-10-20T18:02:53.851381] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-02. [2025-10-20T18:02:54.399435] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:02:54.651383] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-06. [2025-10-20T18:02:54.975686] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:02:55.451417] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-04. [2025-10-20T18:02:55.806873] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T18:02:56.251375] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-05. [2025-10-20T18:02:56.818268] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:02:57.051450] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-03. [2025-10-20T18:02:57.460823] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T18:02:57.851413] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-07. [2025-10-20T18:02:58.357829] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:02:58.622228] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T18:02:58.651405] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-08. [2025-10-20T18:02:59.451443] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-04. [2025-10-20T18:02:59.775570] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:03:00.251402] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-06. [2025-10-20T18:03:00.866482] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:03:01.051389] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-05. [2025-10-20T18:03:01.777866] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:03:01.851386] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-05. [2025-10-20T18:03:02.651347] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-07. [2025-10-20T18:03:02.780487] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:03:03.451353] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-09. [2025-10-20T18:03:03.720172] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T18:03:04.230317] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:03:04.251339] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-08. [2025-10-20T18:03:04.599261] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:03:05.051426] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-06. [2025-10-20T18:03:05.667718] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:03:05.851383] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-06. [2025-10-20T18:03:06.651378] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-09. [2025-10-20T18:03:06.765389] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:03:07.451347] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-07. [2025-10-20T18:03:07.673826] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:03:08.246509] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:03:08.255017] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-07. [2025-10-20T18:03:09.051354] [INFO ] [REMOTE ] CNR: Fetching primary data for 2025-10. [2025-10-20T18:03:09.296234] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T18:03:09.585577] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T18:03:09.851376] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-08. [2025-10-20T18:03:10.553047] [INFO ] [REMOTE ] CNR: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:03:10.559544] [INFO ] [REMOTE ] CNR: Full fetch complete (61918 records). [2025-10-20T18:03:10.559703] [DEBUG] [VALIDATOR ] CNR: Validating primary data. [2025-10-20T18:03:10.560382] [FATAL] [VALIDATOR ] CNR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:10.560470] [FATAL] [MAIN ] CNR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:10.560538] [WARN ] [MAIN ] CNR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:10.560612] [INFO ] [LOCAL ] CNR: Starting deletion of primary data. [2025-10-20T18:03:10.560713] [DEBUG] [LOCAL ] CNR: No primary data file found to delete. [2025-10-20T18:03:10.560798] [INFO ] [LOCAL ] CNR: Starting deletion of derivative data. [2025-10-20T18:03:10.560852] [DEBUG] [LOCAL ] CNR: No derivative data file found to delete. [2025-10-20T18:03:10.560910] [INFO ] [MAIN ] Update Progress: 499/2367 symbols completed. CNR took 37.0s. Estimated remaining: 3h 56m 13s. Rate: 7.9 symbols/min [2025-10-20T18:03:10.658678] [INFO ] [REMOTE ] CNS: Fetching primary data for 2025-10. [2025-10-20T18:03:10.744153] [INFO ] [MAIN ] COF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:10.801993] [DEBUG] [LOCAL ] COF: Loaded 54402 existing records from primary storage. [2025-10-20T18:03:10.802329] [INFO ] [MAIN ] COF: Loaded 54402 existing records. [2025-10-20T18:03:10.994183] [WARN ] [MAIN ] COF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:10.994477] [INFO ] [MAIN ] COF: Wiping existing primary data before full refresh. [2025-10-20T18:03:10.994525] [INFO ] [LOCAL ] COF: Starting deletion of primary data. [2025-10-20T18:03:10.994866] [DEBUG] [LOCAL ] COF: Deleted primary data file. [2025-10-20T18:03:10.994922] [INFO ] [MAIN ] COF: Starting full primary refresh. [2025-10-20T18:03:11.377677] [INFO ] [REMOTE ] COF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:11.451366] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-08. [2025-10-20T18:03:11.570341] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:03:11.774185] [INFO ] [REMOTE ] CNS: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:03:11.778231] [INFO ] [REMOTE ] CNS: Full fetch complete (30965 records). [2025-10-20T18:03:11.778306] [DEBUG] [VALIDATOR ] CNS: Validating primary data. [2025-10-20T18:03:11.778822] [FATAL] [VALIDATOR ] CNS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:11.778901] [FATAL] [MAIN ] CNS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:11.778942] [WARN ] [MAIN ] CNS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:11.779009] [INFO ] [LOCAL ] CNS: Starting deletion of primary data. [2025-10-20T18:03:11.779070] [DEBUG] [LOCAL ] CNS: No primary data file found to delete. [2025-10-20T18:03:11.779103] [INFO ] [LOCAL ] CNS: Starting deletion of derivative data. [2025-10-20T18:03:11.779140] [DEBUG] [LOCAL ] CNS: No derivative data file found to delete. [2025-10-20T18:03:11.779190] [INFO ] [MAIN ] Update Progress: 500/2367 symbols completed. CNS took 31.7s. Estimated remaining: 3h 55m 42s. Rate: 7.9 symbols/min [2025-10-20T18:03:11.949310] [INFO ] [MAIN ] COHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:11.949450] [DEBUG] [LOCAL ] COHR: No existing data file found in primary storage. [2025-10-20T18:03:11.949486] [WARN ] [MAIN ] COHR: No existing data found, performing full refresh instead. [2025-10-20T18:03:11.949591] [INFO ] [MAIN ] COHR: Wiping existing primary data before full refresh. [2025-10-20T18:03:11.949631] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-10-20T18:03:11.949715] [DEBUG] [LOCAL ] COHR: No primary data file found to delete. [2025-10-20T18:03:11.949754] [INFO ] [MAIN ] COHR: Starting full primary refresh. [2025-10-20T18:03:12.256514] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-01. [2025-10-20T18:03:12.331939] [INFO ] [REMOTE ] COHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:12.720792] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:03:13.051425] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-01. [2025-10-20T18:03:13.851391] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-09. [2025-10-20T18:03:14.332929] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:03:14.651376] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-09. [2025-10-20T18:03:15.136159] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:03:15.300857] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:03:15.451440] [INFO ] [REMOTE ] CODI: Fetching primary data for 2025-10. [2025-10-20T18:03:16.251364] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-02. [2025-10-20T18:03:16.525301] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:03:16.877086] [INFO ] [REMOTE ] CODI: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T18:03:16.883786] [INFO ] [REMOTE ] CODI: Full fetch complete (53347 records). [2025-10-20T18:03:16.883884] [DEBUG] [VALIDATOR ] CODI: Validating primary data. [2025-10-20T18:03:17.058046] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-02. [2025-10-20T18:03:17.296101] [DEBUG] [VALIDATOR ] CODI: All expected trading days are present. [2025-10-20T18:03:17.296196] [DEBUG] [VALIDATOR ] CODI: Primary data validation passed. [2025-10-20T18:03:17.296945] [INFO ] [MAIN ] CODI: Fetched primary data is valid. [2025-10-20T18:03:17.297017] [DEBUG] [LOCAL ] CODI: Saving primary data. [2025-10-20T18:03:17.298530] [DEBUG] [LOCAL ] CODI: Trimmed 53347 records to 41816 (keeping 200 days of data) [2025-10-20T18:03:17.566356] [DEBUG] [LOCAL ] CODI: Primary data saved (41816 records). [2025-10-20T18:03:17.566725] [INFO ] [MAIN ] CODI: Full history saved. [2025-10-20T18:03:17.566839] [INFO ] [MAIN ] Update Progress: 501/2367 symbols completed. CODI took 27.4s. Estimated remaining: 3h 55m 28s. Rate: 7.9 symbols/min [2025-10-20T18:03:17.771084] [INFO ] [MAIN ] COLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:17.771362] [DEBUG] [LOCAL ] COLD: No existing data file found in primary storage. [2025-10-20T18:03:17.771704] [WARN ] [MAIN ] COLD: No existing data found, performing full refresh instead. [2025-10-20T18:03:17.772109] [INFO ] [MAIN ] COLD: Wiping existing primary data before full refresh. [2025-10-20T18:03:17.772205] [INFO ] [LOCAL ] COLD: Starting deletion of primary data. [2025-10-20T18:03:17.772346] [DEBUG] [LOCAL ] COLD: No primary data file found to delete. [2025-10-20T18:03:17.772421] [INFO ] [MAIN ] COLD: Starting full primary refresh. [2025-10-20T18:03:17.852541] [INFO ] [REMOTE ] CNX: Fetching primary data for 2025-10. [2025-10-20T18:03:18.082730] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:03:18.388229] [INFO ] [REMOTE ] COLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:18.651388] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-03. [2025-10-20T18:03:19.036928] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T18:03:19.451490] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-01. [2025-10-20T18:03:19.802461] [INFO ] [REMOTE ] CNX: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T18:03:19.810018] [INFO ] [REMOTE ] CNX: Full fetch complete (74568 records). [2025-10-20T18:03:19.810163] [DEBUG] [VALIDATOR ] CNX: Validating primary data. [2025-10-20T18:03:19.811041] [FATAL] [VALIDATOR ] CNX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:19.811131] [FATAL] [MAIN ] CNX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:19.811175] [WARN ] [MAIN ] CNX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:19.811238] [INFO ] [LOCAL ] CNX: Starting deletion of primary data. [2025-10-20T18:03:19.811303] [DEBUG] [LOCAL ] CNX: No primary data file found to delete. [2025-10-20T18:03:19.811339] [INFO ] [LOCAL ] CNX: Starting deletion of derivative data. [2025-10-20T18:03:19.811500] [DEBUG] [LOCAL ] CNX: Deleted derivative data file. [2025-10-20T18:03:19.811609] [INFO ] [MAIN ] Update Progress: 502/2367 symbols completed. CNX took 34.8s. Estimated remaining: 3h 55m 0s. Rate: 7.9 symbols/min [2025-10-20T18:03:20.017762] [INFO ] [MAIN ] COMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:20.071078] [DEBUG] [LOCAL ] COMP: Loaded 54186 existing records from primary storage. [2025-10-20T18:03:20.071448] [INFO ] [MAIN ] COMP: Loaded 54186 existing records. [2025-10-20T18:03:20.254867] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-03. [2025-10-20T18:03:20.259720] [WARN ] [MAIN ] COMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:20.260865] [INFO ] [MAIN ] COMP: Wiping existing primary data before full refresh. [2025-10-20T18:03:20.260927] [INFO ] [LOCAL ] COMP: Starting deletion of primary data. [2025-10-20T18:03:20.261280] [DEBUG] [LOCAL ] COMP: Deleted primary data file. [2025-10-20T18:03:20.261320] [INFO ] [MAIN ] COMP: Starting full primary refresh. [2025-10-20T18:03:20.668114] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T18:03:20.673583] [INFO ] [REMOTE ] COMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:20.926766] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:03:21.051423] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-01. [2025-10-20T18:03:21.856569] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-04. [2025-10-20T18:03:21.877676] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:03:22.651399] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-04. [2025-10-20T18:03:22.883766] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:03:23.451386] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-02. [2025-10-20T18:03:23.547151] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:03:24.251446] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-02. [2025-10-20T18:03:24.678553] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:03:24.994223] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:03:25.051407] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-03. [2025-10-20T18:03:25.630555] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:03:25.851423] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-05. [2025-10-20T18:03:26.652534] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-05. [2025-10-20T18:03:26.657508] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:03:27.451387] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-04. [2025-10-20T18:03:27.809610] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:03:28.251461] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-03. [2025-10-20T18:03:28.355996] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:03:28.930527] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:03:29.051473] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-06. [2025-10-20T18:03:29.851394] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-06. [2025-10-20T18:03:30.076000] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T18:03:30.660137] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-05. [2025-10-20T18:03:30.668002] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:03:31.451481] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-04. [2025-10-20T18:03:31.798001] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T18:03:32.106998] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:03:32.251397] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-06. [2025-10-20T18:03:33.051450] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-07. [2025-10-20T18:03:33.326493] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:03:33.851403] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-05. [2025-10-20T18:03:33.917610] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:03:34.651390] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-07. [2025-10-20T18:03:35.246489] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-10-20T18:03:35.451391] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-08. [2025-10-20T18:03:35.619214] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T18:03:36.251424] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-06. [2025-10-20T18:03:36.461764] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:03:37.051398] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-08. [2025-10-20T18:03:37.458851] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:03:37.662838] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:03:37.851397] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-07. [2025-10-20T18:03:38.651357] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-09. [2025-10-20T18:03:38.725061] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:03:39.314795] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:03:39.451386] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-07. [2025-10-20T18:03:40.251348] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-08. [2025-10-20T18:03:40.335997] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:03:41.051363] [INFO ] [REMOTE ] COF: Fetching primary data for 2025-10. [2025-10-20T18:03:41.495596] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:03:41.560752] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T18:03:41.693294] [INFO ] [REMOTE ] COF: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T18:03:41.701627] [INFO ] [REMOTE ] COF: Full fetch complete (77688 records). [2025-10-20T18:03:41.701809] [DEBUG] [VALIDATOR ] COF: Validating primary data. [2025-10-20T18:03:41.856514] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-09. [2025-10-20T18:03:42.106007] [DEBUG] [VALIDATOR ] COF: All expected trading days are present. [2025-10-20T18:03:42.106092] [DEBUG] [VALIDATOR ] COF: Primary data validation passed. [2025-10-20T18:03:42.106987] [INFO ] [MAIN ] COF: Fetched primary data is valid. [2025-10-20T18:03:42.107052] [DEBUG] [LOCAL ] COF: Saving primary data. [2025-10-20T18:03:42.108461] [DEBUG] [LOCAL ] COF: Trimmed 77688 records to 53625 (keeping 200 days of data) [2025-10-20T18:03:42.438205] [DEBUG] [LOCAL ] COF: Primary data saved (53625 records). [2025-10-20T18:03:42.438340] [INFO ] [MAIN ] COF: Full history saved. [2025-10-20T18:03:42.438415] [INFO ] [MAIN ] Update Progress: 503/2367 symbols completed. COF took 31.9s. Estimated remaining: 3h 55m 49s. Rate: 7.9 symbols/min [2025-10-20T18:03:42.605449] [INFO ] [MAIN ] CON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:42.641052] [DEBUG] [LOCAL ] CON: Loaded 39523 existing records from primary storage. [2025-10-20T18:03:42.641344] [INFO ] [MAIN ] CON: Loaded 39523 existing records. [2025-10-20T18:03:42.662044] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-09. [2025-10-20T18:03:42.857377] [WARN ] [MAIN ] CON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:42.857609] [INFO ] [MAIN ] CON: Wiping existing primary data before full refresh. [2025-10-20T18:03:42.857685] [INFO ] [LOCAL ] CON: Starting deletion of primary data. [2025-10-20T18:03:42.858035] [DEBUG] [LOCAL ] CON: Deleted primary data file. [2025-10-20T18:03:42.858092] [INFO ] [MAIN ] CON: Starting full primary refresh. [2025-10-20T18:03:43.214285] [INFO ] [REMOTE ] CON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:43.455508] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-01. [2025-10-20T18:03:43.462074] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:03:44.251338] [INFO ] [REMOTE ] COMP: Fetching primary data for 2025-10. [2025-10-20T18:03:44.337701] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:03:44.714427] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:03:45.051392] [INFO ] [REMOTE ] COLD: Fetching primary data for 2025-10. [2025-10-20T18:03:45.851415] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-08. [2025-10-20T18:03:46.289544] [INFO ] [REMOTE ] COMP: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T18:03:46.297433] [INFO ] [REMOTE ] COMP: Full fetch complete (77396 records). [2025-10-20T18:03:46.297552] [DEBUG] [VALIDATOR ] COMP: Validating primary data. [2025-10-20T18:03:46.687241] [DEBUG] [VALIDATOR ] COMP: All expected trading days are present. [2025-10-20T18:03:46.687580] [DEBUG] [VALIDATOR ] COMP: Primary data validation passed. [2025-10-20T18:03:46.662761] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-02. [2025-10-20T18:03:46.688809] [INFO ] [MAIN ] COMP: Fetched primary data is valid. [2025-10-20T18:03:46.689036] [DEBUG] [LOCAL ] COMP: Saving primary data. [2025-10-20T18:03:46.691786] [DEBUG] [LOCAL ] COMP: Trimmed 77396 records to 53404 (keeping 200 days of data) [2025-10-20T18:03:46.718932] [INFO ] [REMOTE ] COLD: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:03:46.805249] [INFO ] [REMOTE ] COLD: Full fetch complete (74785 records). [2025-10-20T18:03:46.823733] [DEBUG] [VALIDATOR ] COLD: Validating primary data. [2025-10-20T18:03:47.352011] [DEBUG] [VALIDATOR ] COLD: All expected trading days are present. [2025-10-20T18:03:47.352147] [DEBUG] [VALIDATOR ] COLD: Primary data validation passed. [2025-10-20T18:03:47.353090] [INFO ] [MAIN ] COLD: Fetched primary data is valid. [2025-10-20T18:03:47.382432] [DEBUG] [LOCAL ] COLD: Saving primary data. [2025-10-20T18:03:47.408869] [DEBUG] [LOCAL ] COLD: Trimmed 74785 records to 52476 (keeping 200 days of data) [2025-10-20T18:03:47.535541] [DEBUG] [LOCAL ] COMP: Primary data saved (53404 records). [2025-10-20T18:03:47.537515] [INFO ] [MAIN ] COMP: Full history saved. [2025-10-20T18:03:47.537638] [INFO ] [MAIN ] Update Progress: 504/2367 symbols completed. COMP took 27.7s. Estimated remaining: 3h 55m 32s. Rate: 7.9 symbols/min [2025-10-20T18:03:47.849338] [INFO ] [MAIN ] COOK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:47.959503] [DEBUG] [LOCAL ] COOK: Loaded 25238 existing records from primary storage. [2025-10-20T18:03:47.961285] [INFO ] [MAIN ] COOK: Loaded 25238 existing records. [2025-10-20T18:03:48.039021] [DEBUG] [LOCAL ] COLD: Primary data saved (52476 records). [2025-10-20T18:03:48.073629] [INFO ] [MAIN ] COLD: Full history saved. [2025-10-20T18:03:48.090389] [INFO ] [MAIN ] Update Progress: 505/2367 symbols completed. COLD took 30.5s. Estimated remaining: 3h 54m 58s. Rate: 7.9 symbols/min [2025-10-20T18:03:48.223038] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-10-20T18:03:48.270465] [WARN ] [MAIN ] COOK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:48.298490] [INFO ] [MAIN ] COOK: Wiping existing primary data before full refresh. [2025-10-20T18:03:48.302680] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-09. [2025-10-20T18:03:48.302947] [INFO ] [LOCAL ] COOK: Starting deletion of primary data. [2025-10-20T18:03:48.330209] [DEBUG] [LOCAL ] COOK: Deleted primary data file. [2025-10-20T18:03:48.351816] [INFO ] [MAIN ] COOK: Starting full primary refresh. [2025-10-20T18:03:48.393468] [INFO ] [MAIN ] COP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:48.432448] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:03:48.478421] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-03. [2025-10-20T18:03:48.795917] [DEBUG] [LOCAL ] COP: Loaded 54507 existing records from primary storage. [2025-10-20T18:03:48.794945] [INFO ] [REMOTE ] COOK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:48.796417] [INFO ] [MAIN ] COP: Loaded 54507 existing records. [2025-10-20T18:03:48.968364] [WARN ] [MAIN ] COP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:48.968541] [INFO ] [MAIN ] COP: Wiping existing primary data before full refresh. [2025-10-20T18:03:48.968589] [INFO ] [LOCAL ] COP: Starting deletion of primary data. [2025-10-20T18:03:48.969175] [DEBUG] [LOCAL ] COP: Deleted primary data file. [2025-10-20T18:03:48.969231] [INFO ] [MAIN ] COP: Starting full primary refresh. [2025-10-20T18:03:49.059294] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-01. [2025-10-20T18:03:49.340537] [INFO ] [REMOTE ] COP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:49.851367] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-01. [2025-10-20T18:03:50.010504] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:03:50.202059] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:03:50.389555] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T18:03:50.651444] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-02. [2025-10-20T18:03:51.277600] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T18:03:51.451394] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-04. [2025-10-20T18:03:51.719758] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:03:52.251438] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-03. [2025-10-20T18:03:52.894788] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:03:53.051384] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-02. [2025-10-20T18:03:53.082915] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:03:53.851422] [INFO ] [REMOTE ] COHR: Fetching primary data for 2025-10. [2025-10-20T18:03:54.651358] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-05. [2025-10-20T18:03:55.041063] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T18:03:55.451386] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-04. [2025-10-20T18:03:55.643113] [INFO ] [REMOTE ] COHR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T18:03:55.650464] [INFO ] [REMOTE ] COHR: Full fetch complete (77073 records). [2025-10-20T18:03:55.650593] [DEBUG] [VALIDATOR ] COHR: Validating primary data. [2025-10-20T18:03:55.651830] [FATAL] [VALIDATOR ] COHR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:03:55.651915] [FATAL] [MAIN ] COHR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:03:55.651958] [WARN ] [MAIN ] COHR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:03:55.652048] [INFO ] [LOCAL ] COHR: Starting deletion of primary data. [2025-10-20T18:03:55.652114] [DEBUG] [LOCAL ] COHR: No primary data file found to delete. [2025-10-20T18:03:55.652150] [INFO ] [LOCAL ] COHR: Starting deletion of derivative data. [2025-10-20T18:03:55.652190] [DEBUG] [LOCAL ] COHR: No derivative data file found to delete. [2025-10-20T18:03:55.652244] [INFO ] [MAIN ] Update Progress: 506/2367 symbols completed. COHR took 43.9s. Estimated remaining: 3h 54m 51s. Rate: 7.9 symbols/min [2025-10-20T18:03:55.827472] [INFO ] [MAIN ] COR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:03:55.848835] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T18:03:55.901553] [DEBUG] [LOCAL ] COR: Loaded 47244 existing records from primary storage. [2025-10-20T18:03:55.902019] [INFO ] [MAIN ] COR: Loaded 47244 existing records. [2025-10-20T18:03:56.092331] [WARN ] [MAIN ] COR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:03:56.092617] [INFO ] [MAIN ] COR: Wiping existing primary data before full refresh. [2025-10-20T18:03:56.092701] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-10-20T18:03:56.092965] [DEBUG] [LOCAL ] COR: Deleted primary data file. [2025-10-20T18:03:56.093004] [INFO ] [MAIN ] COR: Starting full primary refresh. [2025-10-20T18:03:56.254331] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-03. [2025-10-20T18:03:56.391142] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:03:56.493835] [INFO ] [REMOTE ] COR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:03:57.051370] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-05. [2025-10-20T18:03:57.829084] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:03:57.851333] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-06. [2025-10-20T18:03:58.503559] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-10-20T18:03:58.651362] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-04. [2025-10-20T18:03:59.187758] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:03:59.451371] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-01. [2025-10-20T18:04:00.251426] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-06. [2025-10-20T18:04:00.640149] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T18:04:01.056465] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-05. [2025-10-20T18:04:01.070161] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:04:01.632617] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:04:01.851354] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-07. [2025-10-20T18:04:02.651444] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-07. [2025-10-20T18:04:02.892852] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:04:03.427997] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-05 (2.37 seconds). [2025-10-20T18:04:03.451344] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-06. [2025-10-20T18:04:04.115047] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:04:04.251418] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-08. [2025-10-20T18:04:05.051376] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-08. [2025-10-20T18:04:05.250582] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T18:04:05.549128] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T18:04:05.851378] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-09. [2025-10-20T18:04:06.640967] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:04:06.651345] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-09. [2025-10-20T18:04:07.026801] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:04:07.451395] [INFO ] [REMOTE ] COOK: Fetching primary data for 2025-10. [2025-10-20T18:04:07.921959] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:04:08.251373] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-02. [2025-10-20T18:04:08.436755] [INFO ] [REMOTE ] COOK: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:04:08.441475] [INFO ] [REMOTE ] COOK: Full fetch complete (33115 records). [2025-10-20T18:04:08.441584] [DEBUG] [VALIDATOR ] COOK: Validating primary data. [2025-10-20T18:04:08.860724] [DEBUG] [VALIDATOR ] COOK: All expected trading days are present. [2025-10-20T18:04:08.860823] [DEBUG] [VALIDATOR ] COOK: Primary data validation passed. [2025-10-20T18:04:08.861330] [INFO ] [MAIN ] COOK: Fetched primary data is valid. [2025-10-20T18:04:08.861374] [DEBUG] [LOCAL ] COOK: Saving primary data. [2025-10-20T18:04:08.862336] [DEBUG] [LOCAL ] COOK: Trimmed 33115 records to 24923 (keeping 200 days of data) [2025-10-20T18:04:09.006798] [DEBUG] [LOCAL ] COOK: Primary data saved (24923 records). [2025-10-20T18:04:09.006937] [INFO ] [MAIN ] COOK: Full history saved. [2025-10-20T18:04:09.007027] [INFO ] [MAIN ] Update Progress: 507/2367 symbols completed. COOK took 21.5s. Estimated remaining: 3h 55m 4s. Rate: 7.9 symbols/min [2025-10-20T18:04:09.061218] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-07. [2025-10-20T18:04:09.179866] [INFO ] [MAIN ] COTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:09.180036] [DEBUG] [LOCAL ] COTY: No existing data file found in primary storage. [2025-10-20T18:04:09.180074] [WARN ] [MAIN ] COTY: No existing data found, performing full refresh instead. [2025-10-20T18:04:09.180203] [INFO ] [MAIN ] COTY: Wiping existing primary data before full refresh. [2025-10-20T18:04:09.180243] [INFO ] [LOCAL ] COTY: Starting deletion of primary data. [2025-10-20T18:04:09.180285] [DEBUG] [LOCAL ] COTY: No primary data file found to delete. [2025-10-20T18:04:09.180316] [INFO ] [MAIN ] COTY: Starting full primary refresh. [2025-10-20T18:04:09.535482] [INFO ] [REMOTE ] COTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:09.851356] [INFO ] [REMOTE ] CON: Fetching primary data for 2025-10. [2025-10-20T18:04:10.114575] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:04:10.651360] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-01. [2025-10-20T18:04:10.808051] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:04:11.050469] [INFO ] [REMOTE ] CON: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:04:11.057029] [INFO ] [REMOTE ] CON: Full fetch complete (56813 records). [2025-10-20T18:04:11.057211] [DEBUG] [VALIDATOR ] CON: Validating primary data. [2025-10-20T18:04:11.460829] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-08. [2025-10-20T18:04:11.586080] [DEBUG] [VALIDATOR ] CON: All expected trading days are present. [2025-10-20T18:04:11.586178] [DEBUG] [VALIDATOR ] CON: Primary data validation passed. [2025-10-20T18:04:11.586959] [INFO ] [MAIN ] CON: Fetched primary data is valid. [2025-10-20T18:04:11.587030] [DEBUG] [LOCAL ] CON: Saving primary data. [2025-10-20T18:04:11.588332] [DEBUG] [LOCAL ] CON: Trimmed 56813 records to 38877 (keeping 200 days of data) [2025-10-20T18:04:11.867558] [DEBUG] [LOCAL ] CON: Primary data saved (38877 records). [2025-10-20T18:04:11.867741] [INFO ] [MAIN ] CON: Full history saved. [2025-10-20T18:04:11.867830] [INFO ] [MAIN ] Update Progress: 508/2367 symbols completed. CON took 29.4s. Estimated remaining: 3h 54m 40s. Rate: 7.9 symbols/min [2025-10-20T18:04:12.055209] [INFO ] [MAIN ] COUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:12.101632] [DEBUG] [LOCAL ] COUR: Loaded 50057 existing records from primary storage. [2025-10-20T18:04:12.101971] [INFO ] [MAIN ] COUR: Loaded 50057 existing records. [2025-10-20T18:04:12.252396] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-03. [2025-10-20T18:04:12.285182] [WARN ] [MAIN ] COUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:12.285610] [INFO ] [MAIN ] COUR: Wiping existing primary data before full refresh. [2025-10-20T18:04:12.285808] [INFO ] [LOCAL ] COUR: Starting deletion of primary data. [2025-10-20T18:04:12.286210] [DEBUG] [LOCAL ] COUR: Deleted primary data file. [2025-10-20T18:04:12.286269] [INFO ] [MAIN ] COUR: Starting full primary refresh. [2025-10-20T18:04:12.702283] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T18:04:12.707074] [INFO ] [REMOTE ] COUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:13.051389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-01. [2025-10-20T18:04:13.519545] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T18:04:13.631588] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:04:13.851379] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-02. [2025-10-20T18:04:14.651400] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-04. [2025-10-20T18:04:14.774516] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:04:15.273546] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T18:04:15.451354] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-03. [2025-10-20T18:04:16.251435] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-02. [2025-10-20T18:04:16.412346] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:04:16.784700] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T18:04:17.051388] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-04. [2025-10-20T18:04:17.547459] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T18:04:17.851442] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-09. [2025-10-20T18:04:18.651360] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-03. [2025-10-20T18:04:18.852290] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:04:19.451384] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-05. [2025-10-20T18:04:19.514690] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:04:20.056967] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T18:04:20.251433] [INFO ] [REMOTE ] COP: Fetching primary data for 2025-10. [2025-10-20T18:04:21.051464] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-04. [2025-10-20T18:04:21.121839] [INFO ] [REMOTE ] COP: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:04:21.130725] [INFO ] [REMOTE ] COP: Full fetch complete (77906 records). [2025-10-20T18:04:21.130930] [DEBUG] [VALIDATOR ] COP: Validating primary data. [2025-10-20T18:04:21.605191] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T18:04:21.635204] [DEBUG] [VALIDATOR ] COP: All expected trading days are present. [2025-10-20T18:04:21.635762] [DEBUG] [VALIDATOR ] COP: Primary data validation passed. [2025-10-20T18:04:21.636744] [INFO ] [MAIN ] COP: Fetched primary data is valid. [2025-10-20T18:04:21.636869] [DEBUG] [LOCAL ] COP: Saving primary data. [2025-10-20T18:04:21.638602] [DEBUG] [LOCAL ] COP: Trimmed 77906 records to 53727 (keeping 200 days of data) [2025-10-20T18:04:21.861033] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-05. [2025-10-20T18:04:21.980327] [DEBUG] [LOCAL ] COP: Primary data saved (53727 records). [2025-10-20T18:04:21.980481] [INFO ] [MAIN ] COP: Full history saved. [2025-10-20T18:04:21.980561] [INFO ] [MAIN ] Update Progress: 509/2367 symbols completed. COP took 33.9s. Estimated remaining: 3h 54m 41s. Rate: 7.9 symbols/min [2025-10-20T18:04:22.165572] [INFO ] [MAIN ] CP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:22.229144] [DEBUG] [LOCAL ] CP: Loaded 54017 existing records from primary storage. [2025-10-20T18:04:22.229469] [INFO ] [MAIN ] CP: Loaded 54017 existing records. [2025-10-20T18:04:22.406819] [WARN ] [MAIN ] CP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:22.407005] [INFO ] [MAIN ] CP: Wiping existing primary data before full refresh. [2025-10-20T18:04:22.407051] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-10-20T18:04:22.407227] [DEBUG] [LOCAL ] CP: Deleted primary data file. [2025-10-20T18:04:22.407265] [INFO ] [MAIN ] CP: Starting full primary refresh. [2025-10-20T18:04:22.653092] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-06. [2025-10-20T18:04:22.775018] [INFO ] [REMOTE ] CP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:22.825842] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:04:23.249127] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:04:23.451393] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-01. [2025-10-20T18:04:24.256522] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-05. [2025-10-20T18:04:24.272497] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:04:24.856035] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:04:25.051401] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-02. [2025-10-20T18:04:25.851426] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-06. [2025-10-20T18:04:26.020003] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:04:26.318003] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:04:26.651422] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-03. [2025-10-20T18:04:27.451389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-06. [2025-10-20T18:04:27.723660] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:04:28.251420] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-07. [2025-10-20T18:04:28.366776] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:04:28.893200] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:04:29.051342] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-07. [2025-10-20T18:04:29.424241] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T18:04:29.851386] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-08. [2025-10-20T18:04:30.651396] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-04. [2025-10-20T18:04:30.850371] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:04:31.327841] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:04:31.451383] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-08. [2025-10-20T18:04:31.989541] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:04:32.251383] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-07. [2025-10-20T18:04:33.051467] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-09. [2025-10-20T18:04:33.230372] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:04:33.759606] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T18:04:33.851381] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-08. [2025-10-20T18:04:34.367796] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:04:34.651372] [INFO ] [REMOTE ] COR: Fetching primary data for 2025-10. [2025-10-20T18:04:35.451366] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-05. [2025-10-20T18:04:35.737508] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:04:36.251389] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-09. [2025-10-20T18:04:36.697853] [INFO ] [REMOTE ] COR: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T18:04:36.703955] [INFO ] [REMOTE ] COR: Full fetch complete (68252 records). [2025-10-20T18:04:36.704074] [DEBUG] [VALIDATOR ] COR: Validating primary data. [2025-10-20T18:04:36.704764] [FATAL] [VALIDATOR ] COR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:04:36.704850] [FATAL] [MAIN ] COR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:04:36.704892] [WARN ] [MAIN ] COR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:04:36.704959] [INFO ] [LOCAL ] COR: Starting deletion of primary data. [2025-10-20T18:04:36.705021] [DEBUG] [LOCAL ] COR: No primary data file found to delete. [2025-10-20T18:04:36.705055] [INFO ] [LOCAL ] COR: Starting deletion of derivative data. [2025-10-20T18:04:36.705223] [DEBUG] [LOCAL ] COR: Deleted derivative data file. [2025-10-20T18:04:36.705607] [INFO ] [MAIN ] Update Progress: 510/2367 symbols completed. COR took 41.1s. Estimated remaining: 3h 54m 60s. Rate: 7.9 symbols/min [2025-10-20T18:04:36.882979] [INFO ] [MAIN ] CPA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:36.912853] [DEBUG] [LOCAL ] CPA: Loaded 28704 existing records from primary storage. [2025-10-20T18:04:36.913154] [INFO ] [MAIN ] CPA: Loaded 28704 existing records. [2025-10-20T18:04:37.060750] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-09. [2025-10-20T18:04:37.105243] [WARN ] [MAIN ] CPA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:37.105768] [INFO ] [MAIN ] CPA: Wiping existing primary data before full refresh. [2025-10-20T18:04:37.105918] [INFO ] [LOCAL ] CPA: Starting deletion of primary data. [2025-10-20T18:04:37.106168] [DEBUG] [LOCAL ] CPA: Deleted primary data file. [2025-10-20T18:04:37.106209] [INFO ] [MAIN ] CPA: Starting full primary refresh. [2025-10-20T18:04:37.270969] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T18:04:37.509275] [INFO ] [REMOTE ] CPA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:37.851386] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-01. [2025-10-20T18:04:37.950144] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T18:04:38.644881] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:04:38.655258] [INFO ] [REMOTE ] COUR: Fetching primary data for 2025-10. [2025-10-20T18:04:39.136319] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:04:39.451376] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-02. [2025-10-20T18:04:40.133845] [INFO ] [REMOTE ] COUR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:04:40.141257] [INFO ] [REMOTE ] COUR: Full fetch complete (71100 records). [2025-10-20T18:04:40.141397] [DEBUG] [VALIDATOR ] COUR: Validating primary data. [2025-10-20T18:04:40.256438] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-06. [2025-10-20T18:04:40.540090] [DEBUG] [VALIDATOR ] COUR: All expected trading days are present. [2025-10-20T18:04:40.540169] [DEBUG] [VALIDATOR ] COUR: Primary data validation passed. [2025-10-20T18:04:40.540974] [INFO ] [MAIN ] COUR: Fetched primary data is valid. [2025-10-20T18:04:40.541054] [DEBUG] [LOCAL ] COUR: Saving primary data. [2025-10-20T18:04:40.542368] [DEBUG] [LOCAL ] COUR: Trimmed 71100 records to 49404 (keeping 200 days of data) [2025-10-20T18:04:40.781148] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:04:40.849724] [DEBUG] [LOCAL ] COUR: Primary data saved (49404 records). [2025-10-20T18:04:40.849854] [INFO ] [MAIN ] COUR: Full history saved. [2025-10-20T18:04:40.849928] [INFO ] [MAIN ] Update Progress: 511/2367 symbols completed. COUR took 29.0s. Estimated remaining: 3h 54m 40s. Rate: 7.9 symbols/min [2025-10-20T18:04:41.019758] [INFO ] [MAIN ] CPAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:41.019913] [DEBUG] [LOCAL ] CPAC: No existing data file found in primary storage. [2025-10-20T18:04:41.019948] [WARN ] [MAIN ] CPAC: No existing data found, performing full refresh instead. [2025-10-20T18:04:41.020056] [INFO ] [MAIN ] CPAC: Wiping existing primary data before full refresh. [2025-10-20T18:04:41.020095] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-10-20T18:04:41.020135] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-10-20T18:04:41.020165] [INFO ] [MAIN ] CPAC: Starting full primary refresh. [2025-10-20T18:04:41.056717] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-03. [2025-10-20T18:04:41.422366] [INFO ] [REMOTE ] CPAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:41.537767] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:04:41.851369] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-01. [2025-10-20T18:04:42.146547] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T18:04:42.393748] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T18:04:42.651377] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-02. [2025-10-20T18:04:43.006733] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:04:43.451380] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-07. [2025-10-20T18:04:44.251377] [INFO ] [REMOTE ] COTY: Fetching primary data for 2025-10. [2025-10-20T18:04:45.051380] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-03. [2025-10-20T18:04:45.333451] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:04:45.468130] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T18:04:45.851404] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-04. [2025-10-20T18:04:46.217610] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-04 (0.37 seconds). [2025-10-20T18:04:46.259457] [INFO ] [REMOTE ] COTY: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:04:46.267436] [INFO ] [REMOTE ] COTY: Full fetch complete (77297 records). [2025-10-20T18:04:46.267583] [DEBUG] [VALIDATOR ] COTY: Validating primary data. [2025-10-20T18:04:46.650197] [DEBUG] [VALIDATOR ] COTY: All expected trading days are present. [2025-10-20T18:04:46.650315] [DEBUG] [VALIDATOR ] COTY: Primary data validation passed. [2025-10-20T18:04:46.651205] [INFO ] [MAIN ] COTY: Fetched primary data is valid. [2025-10-20T18:04:46.651273] [DEBUG] [LOCAL ] COTY: Saving primary data. [2025-10-20T18:04:46.651498] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-05. [2025-10-20T18:04:46.653553] [DEBUG] [LOCAL ] COTY: Trimmed 77297 records to 53275 (keeping 200 days of data) [2025-10-20T18:04:46.944626] [DEBUG] [LOCAL ] COTY: Primary data saved (53275 records). [2025-10-20T18:04:46.944795] [INFO ] [MAIN ] COTY: Full history saved. [2025-10-20T18:04:46.944868] [INFO ] [MAIN ] Update Progress: 512/2367 symbols completed. COTY took 37.9s. Estimated remaining: 3h 54m 27s. Rate: 7.9 symbols/min [2025-10-20T18:04:47.031594] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-10-20T18:04:47.121013] [INFO ] [MAIN ] CPAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:47.151271] [DEBUG] [LOCAL ] CPAY: Loaded 30966 existing records from primary storage. [2025-10-20T18:04:47.151570] [INFO ] [MAIN ] CPAY: Loaded 30966 existing records. [2025-10-20T18:04:47.344593] [WARN ] [MAIN ] CPAY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:47.344955] [INFO ] [MAIN ] CPAY: Wiping existing primary data before full refresh. [2025-10-20T18:04:47.345008] [INFO ] [LOCAL ] CPAY: Starting deletion of primary data. [2025-10-20T18:04:47.345237] [DEBUG] [LOCAL ] CPAY: Deleted primary data file. [2025-10-20T18:04:47.345276] [INFO ] [MAIN ] CPAY: Starting full primary refresh. [2025-10-20T18:04:47.456577] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-06. [2025-10-20T18:04:47.737915] [INFO ] [REMOTE ] CPAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:47.904312] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T18:04:48.251402] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-01. [2025-10-20T18:04:49.051374] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-04. [2025-10-20T18:04:49.363634] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:04:49.851366] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-02. [2025-10-20T18:04:50.385681] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:04:50.651364] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-08. [2025-10-20T18:04:50.894369] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:04:51.451422] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-03. [2025-10-20T18:04:52.251385] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-05. [2025-10-20T18:04:52.501227] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T18:04:52.877986] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:04:53.051421] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-07. [2025-10-20T18:04:53.115283] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T18:04:53.567042] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T18:04:53.851487] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-09. [2025-10-20T18:04:54.651429] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-08. [2025-10-20T18:04:55.013858] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:04:55.451389] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-09. [2025-10-20T18:04:55.668790] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:04:55.892393] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:04:56.251347] [INFO ] [REMOTE ] CPAC: Fetching primary data for 2025-10. [2025-10-20T18:04:56.786574] [INFO ] [REMOTE ] CPAC: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T18:04:56.787585] [INFO ] [REMOTE ] CPAC: Full fetch complete (1771 records). [2025-10-20T18:04:56.787679] [DEBUG] [VALIDATOR ] CPAC: Validating primary data. [2025-10-20T18:04:56.788041] [FATAL] [VALIDATOR ] CPAC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:04:56.788122] [FATAL] [MAIN ] CPAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:04:56.788165] [WARN ] [MAIN ] CPAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:04:56.788203] [INFO ] [LOCAL ] CPAC: Starting deletion of primary data. [2025-10-20T18:04:56.788257] [DEBUG] [LOCAL ] CPAC: No primary data file found to delete. [2025-10-20T18:04:56.788291] [INFO ] [LOCAL ] CPAC: Starting deletion of derivative data. [2025-10-20T18:04:56.788330] [DEBUG] [LOCAL ] CPAC: No derivative data file found to delete. [2025-10-20T18:04:56.788383] [INFO ] [MAIN ] Update Progress: 513/2367 symbols completed. CPAC took 15.9s. Estimated remaining: 3h 54m 27s. Rate: 7.9 symbols/min [2025-10-20T18:04:56.962020] [INFO ] [MAIN ] CPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:04:56.976539] [DEBUG] [LOCAL ] CPF: Loaded 15519 existing records from primary storage. [2025-10-20T18:04:56.976842] [INFO ] [MAIN ] CPF: Loaded 15519 existing records. [2025-10-20T18:04:57.056499] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-04. [2025-10-20T18:04:57.172026] [WARN ] [MAIN ] CPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:04:57.172233] [INFO ] [MAIN ] CPF: Wiping existing primary data before full refresh. [2025-10-20T18:04:57.172280] [INFO ] [LOCAL ] CPF: Starting deletion of primary data. [2025-10-20T18:04:57.172437] [DEBUG] [LOCAL ] CPF: Deleted primary data file. [2025-10-20T18:04:57.172475] [INFO ] [MAIN ] CPF: Starting full primary refresh. [2025-10-20T18:04:57.550563] [INFO ] [REMOTE ] CPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:04:57.851393] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-01. [2025-10-20T18:04:58.543556] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:04:58.651391] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-06. [2025-10-20T18:04:58.735860] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T18:04:59.451384] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-05. [2025-10-20T18:04:59.935179] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:05:00.251440] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-02. [2025-10-20T18:05:00.853699] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:05:01.051358] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-07. [2025-10-20T18:05:01.283806] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T18:05:01.851428] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-06. [2025-10-20T18:05:02.651389] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-03. [2025-10-20T18:05:02.702254] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:05:03.202889] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:05:03.451416] [INFO ] [REMOTE ] CP: Fetching primary data for 2025-10. [2025-10-20T18:05:03.766816] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:05:04.251438] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-07. [2025-10-20T18:05:05.051428] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-08. [2025-10-20T18:05:05.572511] [INFO ] [REMOTE ] CP: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T18:05:05.579773] [INFO ] [REMOTE ] CP: Full fetch complete (76919 records). [2025-10-20T18:05:05.579911] [DEBUG] [VALIDATOR ] CP: Validating primary data. [2025-10-20T18:05:05.580556] [FATAL] [VALIDATOR ] CP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:05:05.580684] [FATAL] [MAIN ] CP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:05:05.580757] [WARN ] [MAIN ] CP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:05:05.580850] [INFO ] [LOCAL ] CP: Starting deletion of primary data. [2025-10-20T18:05:05.580938] [DEBUG] [LOCAL ] CP: No primary data file found to delete. [2025-10-20T18:05:05.580978] [INFO ] [LOCAL ] CP: Starting deletion of derivative data. [2025-10-20T18:05:05.581100] [DEBUG] [LOCAL ] CP: Deleted derivative data file. [2025-10-20T18:05:05.581174] [INFO ] [MAIN ] Update Progress: 514/2367 symbols completed. CP took 43.6s. Estimated remaining: 3h 54m 24s. Rate: 7.9 symbols/min [2025-10-20T18:05:05.706539] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:05:05.789614] [INFO ] [MAIN ] CPK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:05.804824] [DEBUG] [LOCAL ] CPK: Loaded 14207 existing records from primary storage. [2025-10-20T18:05:05.805057] [INFO ] [MAIN ] CPK: Loaded 14207 existing records. [2025-10-20T18:05:05.860671] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-04. [2025-10-20T18:05:05.991679] [WARN ] [MAIN ] CPK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:05.991894] [INFO ] [MAIN ] CPK: Wiping existing primary data before full refresh. [2025-10-20T18:05:05.991946] [INFO ] [LOCAL ] CPK: Starting deletion of primary data. [2025-10-20T18:05:05.992112] [DEBUG] [LOCAL ] CPK: Deleted primary data file. [2025-10-20T18:05:05.992148] [INFO ] [MAIN ] CPK: Starting full primary refresh. [2025-10-20T18:05:06.351891] [INFO ] [REMOTE ] CPK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:06.368936] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T18:05:06.651402] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-08. [2025-10-20T18:05:06.802047] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:05:07.451437] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-01. [2025-10-20T18:05:07.749610] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:05:08.024732] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:05:08.251425] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-09. [2025-10-20T18:05:09.051445] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-09. [2025-10-20T18:05:09.417979] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:05:09.851437] [INFO ] [REMOTE ] CPA: Fetching primary data for 2025-10. [2025-10-20T18:05:10.364030] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T18:05:10.651426] [INFO ] [REMOTE ] CPAY: Fetching primary data for 2025-10. [2025-10-20T18:05:11.106380] [INFO ] [REMOTE ] CPA: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T18:05:11.111133] [INFO ] [REMOTE ] CPA: Full fetch complete (41312 records). [2025-10-20T18:05:11.111351] [DEBUG] [VALIDATOR ] CPA: Validating primary data. [2025-10-20T18:05:11.466428] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-02. [2025-10-20T18:05:11.524256] [DEBUG] [VALIDATOR ] CPA: All expected trading days are present. [2025-10-20T18:05:11.524350] [DEBUG] [VALIDATOR ] CPA: Primary data validation passed. [2025-10-20T18:05:11.525087] [INFO ] [MAIN ] CPA: Fetched primary data is valid. [2025-10-20T18:05:11.525158] [DEBUG] [LOCAL ] CPA: Saving primary data. [2025-10-20T18:05:11.526262] [DEBUG] [LOCAL ] CPA: Trimmed 41312 records to 28158 (keeping 200 days of data) [2025-10-20T18:05:11.721107] [DEBUG] [LOCAL ] CPA: Primary data saved (28158 records). [2025-10-20T18:05:11.721266] [INFO ] [MAIN ] CPA: Full history saved. [2025-10-20T18:05:11.721351] [INFO ] [MAIN ] Update Progress: 515/2367 symbols completed. CPA took 35.0s. Estimated remaining: 3h 54m 11s. Rate: 7.9 symbols/min [2025-10-20T18:05:11.923465] [INFO ] [MAIN ] CPNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:11.938120] [INFO ] [REMOTE ] CPAY: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T18:05:11.943175] [INFO ] [REMOTE ] CPAY: Full fetch complete (44916 records). [2025-10-20T18:05:11.943337] [DEBUG] [VALIDATOR ] CPAY: Validating primary data. [2025-10-20T18:05:12.247403] [DEBUG] [LOCAL ] CPNG: Loaded 54459 existing records from primary storage. [2025-10-20T18:05:12.261854] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-05. [2025-10-20T18:05:12.275377] [INFO ] [MAIN ] CPNG: Loaded 54459 existing records. [2025-10-20T18:05:12.556097] [DEBUG] [VALIDATOR ] CPAY: All expected trading days are present. [2025-10-20T18:05:12.550797] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:05:12.586790] [DEBUG] [VALIDATOR ] CPAY: Primary data validation passed. [2025-10-20T18:05:12.599328] [INFO ] [MAIN ] CPAY: Fetched primary data is valid. [2025-10-20T18:05:12.608974] [DEBUG] [LOCAL ] CPAY: Saving primary data. [2025-10-20T18:05:12.634678] [WARN ] [MAIN ] CPNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:12.634921] [INFO ] [MAIN ] CPNG: Wiping existing primary data before full refresh. [2025-10-20T18:05:12.635054] [INFO ] [LOCAL ] CPNG: Starting deletion of primary data. [2025-10-20T18:05:12.635374] [DEBUG] [LOCAL ] CPNG: Deleted primary data file. [2025-10-20T18:05:12.635438] [INFO ] [MAIN ] CPNG: Starting full primary refresh. [2025-10-20T18:05:12.667925] [DEBUG] [LOCAL ] CPAY: Trimmed 44916 records to 30345 (keeping 200 days of data) [2025-10-20T18:05:13.057135] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-03. [2025-10-20T18:05:13.219054] [INFO ] [REMOTE ] CPNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:13.242342] [DEBUG] [LOCAL ] CPAY: Primary data saved (30345 records). [2025-10-20T18:05:13.242576] [INFO ] [MAIN ] CPAY: Full history saved. [2025-10-20T18:05:13.242709] [INFO ] [MAIN ] Update Progress: 516/2367 symbols completed. CPAY took 26.3s. Estimated remaining: 3h 53m 42s. Rate: 7.9 symbols/min [2025-10-20T18:05:13.378893] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:05:13.457451] [INFO ] [MAIN ] CPRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:13.510754] [DEBUG] [LOCAL ] CPRI: Loaded 53228 existing records from primary storage. [2025-10-20T18:05:13.511093] [INFO ] [MAIN ] CPRI: Loaded 53228 existing records. [2025-10-20T18:05:13.702005] [WARN ] [MAIN ] CPRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:13.702274] [INFO ] [MAIN ] CPRI: Wiping existing primary data before full refresh. [2025-10-20T18:05:13.702326] [INFO ] [LOCAL ] CPRI: Starting deletion of primary data. [2025-10-20T18:05:13.702749] [DEBUG] [LOCAL ] CPRI: Deleted primary data file. [2025-10-20T18:05:13.702806] [INFO ] [MAIN ] CPRI: Starting full primary refresh. [2025-10-20T18:05:13.853358] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-06. [2025-10-20T18:05:14.056687] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:05:14.083221] [INFO ] [REMOTE ] CPRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:14.538152] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T18:05:14.651432] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-01. [2025-10-20T18:05:15.451422] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-07. [2025-10-20T18:05:16.251360] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-01. [2025-10-20T18:05:16.452179] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T18:05:16.543043] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:05:17.051436] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-08. [2025-10-20T18:05:17.685945] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T18:05:17.851427] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-09. [2025-10-20T18:05:18.167938] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T18:05:18.625566] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T18:05:18.651393] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-02. [2025-10-20T18:05:19.451450] [INFO ] [REMOTE ] CPF: Fetching primary data for 2025-10. [2025-10-20T18:05:19.956457] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:05:20.251438] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-03. [2025-10-20T18:05:20.494472] [INFO ] [REMOTE ] CPF: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:05:20.497282] [INFO ] [REMOTE ] CPF: Full fetch complete (23557 records). [2025-10-20T18:05:20.497356] [DEBUG] [VALIDATOR ] CPF: Validating primary data. [2025-10-20T18:05:20.905880] [DEBUG] [VALIDATOR ] CPF: All expected trading days are present. [2025-10-20T18:05:20.905966] [DEBUG] [VALIDATOR ] CPF: Primary data validation passed. [2025-10-20T18:05:20.906405] [INFO ] [MAIN ] CPF: Fetched primary data is valid. [2025-10-20T18:05:20.906445] [DEBUG] [LOCAL ] CPF: Saving primary data. [2025-10-20T18:05:20.907165] [DEBUG] [LOCAL ] CPF: Trimmed 23557 records to 15307 (keeping 200 days of data) [2025-10-20T18:05:21.002026] [DEBUG] [LOCAL ] CPF: Primary data saved (15307 records). [2025-10-20T18:05:21.002180] [INFO ] [MAIN ] CPF: Full history saved. [2025-10-20T18:05:21.002262] [INFO ] [MAIN ] Update Progress: 517/2367 symbols completed. CPF took 24.2s. Estimated remaining: 3h 53m 35s. Rate: 7.9 symbols/min [2025-10-20T18:05:21.051422] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-04. [2025-10-20T18:05:21.851457] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-02. [2025-10-20T18:05:22.022098] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:05:22.399564] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T18:05:22.651407] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-05. [2025-10-20T18:05:23.451394] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-04. [2025-10-20T18:05:23.510404] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:05:23.535503] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T18:05:24.251423] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-06. [2025-10-20T18:05:25.051414] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-03. [2025-10-20T18:05:25.082503] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:05:25.290729] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T18:05:25.851368] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-07. [2025-10-20T18:05:26.651438] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-05. [2025-10-20T18:05:26.807935] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T18:05:27.264708] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-10-20T18:05:27.451415] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-04. [2025-10-20T18:05:28.238127] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:05:28.251338] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-06. [2025-10-20T18:05:29.051474] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-08. [2025-10-20T18:05:29.563680] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T18:05:29.600299] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T18:05:29.851351] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-09. [2025-10-20T18:05:29.975178] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:05:30.401233] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T18:05:30.651431] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-05. [2025-10-20T18:05:31.451397] [INFO ] [REMOTE ] CPK: Fetching primary data for 2025-10. [2025-10-20T18:05:32.251419] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-07. [2025-10-20T18:05:32.309989] [INFO ] [REMOTE ] CPK: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T18:05:32.312942] [INFO ] [REMOTE ] CPK: Full fetch complete (18664 records). [2025-10-20T18:05:32.313025] [DEBUG] [VALIDATOR ] CPK: Validating primary data. [2025-10-20T18:05:32.702010] [DEBUG] [VALIDATOR ] CPK: All expected trading days are present. [2025-10-20T18:05:32.702090] [DEBUG] [VALIDATOR ] CPK: Primary data validation passed. [2025-10-20T18:05:32.702477] [INFO ] [MAIN ] CPK: Fetched primary data is valid. [2025-10-20T18:05:32.702516] [DEBUG] [LOCAL ] CPK: Saving primary data. [2025-10-20T18:05:32.703241] [DEBUG] [LOCAL ] CPK: Trimmed 18664 records to 14051 (keeping 200 days of data) [2025-10-20T18:05:32.792414] [DEBUG] [LOCAL ] CPK: Primary data saved (14051 records). [2025-10-20T18:05:32.792572] [INFO ] [MAIN ] CPK: Full history saved. [2025-10-20T18:05:32.792679] [INFO ] [MAIN ] Update Progress: 518/2367 symbols completed. CPK took 27.2s. Estimated remaining: 3h 53m 42s. Rate: 7.9 symbols/min [2025-10-20T18:05:33.017218] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-05 (2.37 seconds). [2025-10-20T18:05:33.051341] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-06. [2025-10-20T18:05:33.971938] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T18:05:33.983496] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-08. [2025-10-20T18:05:35.163678] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T18:05:35.174577] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-07. [2025-10-20T18:05:35.723063] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:05:35.734114] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-09. [2025-10-20T18:05:37.486842] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:05:37.498144] [INFO ] [REMOTE ] CPRI: Fetching primary data for 2025-10. [2025-10-20T18:05:37.717965] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-10-20T18:05:37.730013] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-08. [2025-10-20T18:05:39.345156] [INFO ] [REMOTE ] CPRI: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:05:39.356625] [INFO ] [REMOTE ] CPRI: Full fetch complete (76183 records). [2025-10-20T18:05:39.356898] [DEBUG] [VALIDATOR ] CPRI: Validating primary data. [2025-10-20T18:05:39.784682] [DEBUG] [VALIDATOR ] CPRI: All expected trading days are present. [2025-10-20T18:05:39.784785] [DEBUG] [VALIDATOR ] CPRI: Primary data validation passed. [2025-10-20T18:05:39.785618] [INFO ] [MAIN ] CPRI: Fetched primary data is valid. [2025-10-20T18:05:39.785700] [DEBUG] [LOCAL ] CPRI: Saving primary data. [2025-10-20T18:05:39.787826] [DEBUG] [LOCAL ] CPRI: Trimmed 76183 records to 52442 (keeping 200 days of data) [2025-10-20T18:05:40.102856] [DEBUG] [LOCAL ] CPRI: Primary data saved (52442 records). [2025-10-20T18:05:40.103005] [INFO ] [MAIN ] CPRI: Full history saved. [2025-10-20T18:05:40.103087] [INFO ] [MAIN ] Update Progress: 519/2367 symbols completed. CPRI took 26.9s. Estimated remaining: 3h 53m 34s. Rate: 7.9 symbols/min [2025-10-20T18:05:40.169395] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-08 (2.44 seconds). [2025-10-20T18:05:40.181807] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-09. [2025-10-20T18:05:42.257661] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-10-20T18:05:42.269598] [INFO ] [REMOTE ] CPNG: Fetching primary data for 2025-10. [2025-10-20T18:05:44.694528] [INFO ] [REMOTE ] CPNG: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T18:05:44.702301] [INFO ] [REMOTE ] CPNG: Full fetch complete (77789 records). [2025-10-20T18:05:44.702417] [DEBUG] [VALIDATOR ] CPNG: Validating primary data. [2025-10-20T18:05:45.100664] [DEBUG] [VALIDATOR ] CPNG: All expected trading days are present. [2025-10-20T18:05:45.100749] [DEBUG] [VALIDATOR ] CPNG: Primary data validation passed. [2025-10-20T18:05:45.101571] [INFO ] [MAIN ] CPNG: Fetched primary data is valid. [2025-10-20T18:05:45.101612] [DEBUG] [LOCAL ] CPNG: Saving primary data. [2025-10-20T18:05:45.102988] [DEBUG] [LOCAL ] CPNG: Trimmed 77789 records to 53680 (keeping 200 days of data) [2025-10-20T18:05:45.420313] [DEBUG] [LOCAL ] CPNG: Primary data saved (53680 records). [2025-10-20T18:05:45.420452] [INFO ] [MAIN ] CPNG: Full history saved. [2025-10-20T18:05:45.420744] [INFO ] [MAIN ] Update Progress: 520/2367 symbols completed. CPNG took 33.7s. Estimated remaining: 3h 53m 18s. Rate: 7.9 symbols/min [2025-10-20T18:05:45.951600] [INFO ] [MAIN ] CPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.111016] [INFO ] [MAIN ] CR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.114916] [INFO ] [MAIN ] CQP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.169074] [INFO ] [MAIN ] CPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:05:46.198501] [DEBUG] [LOCAL ] CPS: Loaded 25505 existing records from primary storage. [2025-10-20T18:05:46.199075] [INFO ] [MAIN ] CPS: Loaded 25505 existing records. [2025-10-20T18:05:46.277178] [DEBUG] [LOCAL ] CQP: Loaded 19156 existing records from primary storage. [2025-10-20T18:05:46.293633] [INFO ] [MAIN ] CQP: Loaded 19156 existing records. [2025-10-20T18:05:46.314700] [DEBUG] [LOCAL ] CR: Loaded 24488 existing records from primary storage. [2025-10-20T18:05:46.407871] [INFO ] [MAIN ] CR: Loaded 24488 existing records. [2025-10-20T18:05:46.585637] [WARN ] [MAIN ] CQP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:46.713421] [WARN ] [MAIN ] CPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:46.705203] [INFO ] [MAIN ] CQP: Wiping existing primary data before full refresh. [2025-10-20T18:05:46.728119] [DEBUG] [LOCAL ] CPT: Loaded 45243 existing records from primary storage. [2025-10-20T18:05:46.728303] [INFO ] [MAIN ] CPS: Wiping existing primary data before full refresh. [2025-10-20T18:05:46.728416] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-10-20T18:05:46.731848] [INFO ] [MAIN ] CPT: Loaded 45243 existing records. [2025-10-20T18:05:46.731948] [INFO ] [LOCAL ] CPS: Starting deletion of primary data. [2025-10-20T18:05:46.737370] [DEBUG] [LOCAL ] CQP: Deleted primary data file. [2025-10-20T18:05:46.761601] [DEBUG] [LOCAL ] CPS: Deleted primary data file. [2025-10-20T18:05:46.761561] [INFO ] [MAIN ] CQP: Starting full primary refresh. [2025-10-20T18:05:46.804768] [INFO ] [MAIN ] CPS: Starting full primary refresh. [2025-10-20T18:05:47.007736] [WARN ] [MAIN ] CR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:47.129699] [INFO ] [MAIN ] CR: Wiping existing primary data before full refresh. [2025-10-20T18:05:47.154364] [INFO ] [LOCAL ] CR: Starting deletion of primary data. [2025-10-20T18:05:47.314629] [DEBUG] [LOCAL ] CR: Deleted primary data file. [2025-10-20T18:05:47.342302] [WARN ] [MAIN ] CPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:05:47.350064] [INFO ] [MAIN ] CPT: Wiping existing primary data before full refresh. [2025-10-20T18:05:47.366956] [INFO ] [MAIN ] CR: Starting full primary refresh. [2025-10-20T18:05:47.408039] [INFO ] [LOCAL ] CPT: Starting deletion of primary data. [2025-10-20T18:05:47.618577] [DEBUG] [LOCAL ] CPT: Deleted primary data file. [2025-10-20T18:05:47.628949] [INFO ] [MAIN ] CPT: Starting full primary refresh. [2025-10-20T18:05:48.204753] [INFO ] [REMOTE ] CQP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.234290] [INFO ] [REMOTE ] CPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.248937] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-01. [2025-10-20T18:05:48.266376] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-01. [2025-10-20T18:05:48.604282] [INFO ] [REMOTE ] CR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.611325] [INFO ] [REMOTE ] CPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:05:48.611829] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-01. [2025-10-20T18:05:48.612158] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-01. [2025-10-20T18:05:49.269876] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:05:49.276250] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-02. [2025-10-20T18:05:49.563167] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:05:49.567871] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-02. [2025-10-20T18:05:49.877184] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:05:49.882633] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-02. [2025-10-20T18:05:50.329134] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:05:50.337947] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-02. [2025-10-20T18:05:50.390244] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T18:05:50.395911] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-03. [2025-10-20T18:05:50.473245] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T18:05:50.478591] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-03. [2025-10-20T18:05:50.714547] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T18:05:50.719070] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-03. [2025-10-20T18:05:51.125807] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:05:51.130529] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-04. [2025-10-20T18:05:51.712372] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T18:05:51.721564] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-03. [2025-10-20T18:05:51.759762] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T18:05:51.766242] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-04. [2025-10-20T18:05:51.838199] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:05:51.844014] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-04. [2025-10-20T18:05:52.199197] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:05:52.204051] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-05. [2025-10-20T18:05:52.856371] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T18:05:52.864218] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-05. [2025-10-20T18:05:53.111213] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:05:53.118968] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-05. [2025-10-20T18:05:53.299020] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:05:53.332762] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:05:53.628018] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T18:05:53.851425] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-04. [2025-10-20T18:05:54.455321] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:05:54.651414] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-06. [2025-10-20T18:05:55.308184] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:05:55.392611] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T18:05:55.451391] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-07. [2025-10-20T18:05:56.251683] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-05. [2025-10-20T18:05:56.260477] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:05:57.051389] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-06. [2025-10-20T18:05:57.479091] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:05:57.851364] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-08. [2025-10-20T18:05:58.072844] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T18:05:58.651395] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-06. [2025-10-20T18:05:58.833435] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:05:59.451396] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-06. [2025-10-20T18:06:00.204484] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:06:00.251336] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-07. [2025-10-20T18:06:00.876066] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:06:01.051414] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-09. [2025-10-20T18:06:01.851396] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-07. [2025-10-20T18:06:02.170847] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T18:06:02.554763] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-07 (2.30 seconds). [2025-10-20T18:06:02.651420] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-08. [2025-10-20T18:06:03.022955] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T18:06:03.451421] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-08. [2025-10-20T18:06:04.251402] [INFO ] [REMOTE ] CR: Fetching primary data for 2025-10. [2025-10-20T18:06:04.349042] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:06:04.394037] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T18:06:05.051411] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-09. [2025-10-20T18:06:05.470919] [INFO ] [REMOTE ] CR: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:06:05.474559] [INFO ] [REMOTE ] CR: Full fetch complete (34798 records). [2025-10-20T18:06:05.474699] [DEBUG] [VALIDATOR ] CR: Validating primary data. [2025-10-20T18:06:05.827616] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T18:06:05.874685] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-07. [2025-10-20T18:06:05.902978] [DEBUG] [VALIDATOR ] CR: All expected trading days are present. [2025-10-20T18:06:05.903446] [DEBUG] [VALIDATOR ] CR: Primary data validation passed. [2025-10-20T18:06:05.904101] [INFO ] [MAIN ] CR: Fetched primary data is valid. [2025-10-20T18:06:05.904165] [DEBUG] [LOCAL ] CR: Saving primary data. [2025-10-20T18:06:05.905109] [DEBUG] [LOCAL ] CR: Trimmed 34798 records to 24066 (keeping 200 days of data) [2025-10-20T18:06:06.056104] [DEBUG] [LOCAL ] CR: Primary data saved (24066 records). [2025-10-20T18:06:06.056225] [INFO ] [MAIN ] CR: Full history saved. [2025-10-20T18:06:06.056298] [INFO ] [MAIN ] Update Progress: 521/2367 symbols completed. CR took 20.6s. Estimated remaining: 3h 53m 57s. Rate: 7.9 symbols/min [2025-10-20T18:06:06.231522] [INFO ] [MAIN ] CRBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:06.244733] [DEBUG] [LOCAL ] CRBD: Loaded 9015 existing records from primary storage. [2025-10-20T18:06:06.244918] [INFO ] [MAIN ] CRBD: Loaded 9015 existing records. [2025-10-20T18:06:06.418512] [WARN ] [MAIN ] CRBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:06.418737] [INFO ] [MAIN ] CRBD: Wiping existing primary data before full refresh. [2025-10-20T18:06:06.418821] [INFO ] [LOCAL ] CRBD: Starting deletion of primary data. [2025-10-20T18:06:06.419180] [DEBUG] [LOCAL ] CRBD: Deleted primary data file. [2025-10-20T18:06:06.419265] [INFO ] [MAIN ] CRBD: Starting full primary refresh. [2025-10-20T18:06:06.657890] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-09. [2025-10-20T18:06:06.782854] [INFO ] [REMOTE ] CRBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:06.796311] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T18:06:07.451374] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-08. [2025-10-20T18:06:08.251359] [INFO ] [REMOTE ] CQP: Fetching primary data for 2025-10. [2025-10-20T18:06:08.468332] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:06:08.571500] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:06:08.905307] [INFO ] [REMOTE ] CQP: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:06:08.908420] [INFO ] [REMOTE ] CQP: Full fetch complete (29770 records). [2025-10-20T18:06:08.908511] [DEBUG] [VALIDATOR ] CQP: Validating primary data. [2025-10-20T18:06:08.909114] [FATAL] [VALIDATOR ] CQP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:06:08.909260] [FATAL] [MAIN ] CQP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:06:08.909336] [WARN ] [MAIN ] CQP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:06:08.909427] [INFO ] [LOCAL ] CQP: Starting deletion of primary data. [2025-10-20T18:06:08.909577] [DEBUG] [LOCAL ] CQP: No primary data file found to delete. [2025-10-20T18:06:08.909730] [INFO ] [LOCAL ] CQP: Starting deletion of derivative data. [2025-10-20T18:06:08.910133] [DEBUG] [LOCAL ] CQP: Deleted derivative data file. [2025-10-20T18:06:08.910386] [INFO ] [MAIN ] Update Progress: 522/2367 symbols completed. CQP took 23.5s. Estimated remaining: 3h 53m 32s. Rate: 7.9 symbols/min [2025-10-20T18:06:09.056491] [INFO ] [REMOTE ] CPT: Fetching primary data for 2025-10. [2025-10-20T18:06:09.088095] [INFO ] [MAIN ] CRBG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:09.089100] [DEBUG] [LOCAL ] CRBG: No existing data file found in primary storage. [2025-10-20T18:06:09.089146] [WARN ] [MAIN ] CRBG: No existing data found, performing full refresh instead. [2025-10-20T18:06:09.089257] [INFO ] [MAIN ] CRBG: Wiping existing primary data before full refresh. [2025-10-20T18:06:09.089299] [INFO ] [LOCAL ] CRBG: Starting deletion of primary data. [2025-10-20T18:06:09.089339] [DEBUG] [LOCAL ] CRBG: No primary data file found to delete. [2025-10-20T18:06:09.089370] [INFO ] [MAIN ] CRBG: Starting full primary refresh. [2025-10-20T18:06:09.455456] [INFO ] [REMOTE ] CRBG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:09.851442] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-09. [2025-10-20T18:06:10.651365] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-01. [2025-10-20T18:06:10.962883] [INFO ] [REMOTE ] CPT: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:06:10.985575] [INFO ] [REMOTE ] CPT: Full fetch complete (64633 records). [2025-10-20T18:06:10.986487] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:06:10.991718] [DEBUG] [VALIDATOR ] CPT: Validating primary data. [2025-10-20T18:06:11.420061] [DEBUG] [VALIDATOR ] CPT: All expected trading days are present. [2025-10-20T18:06:11.420218] [DEBUG] [VALIDATOR ] CPT: Primary data validation passed. [2025-10-20T18:06:11.421601] [INFO ] [MAIN ] CPT: Fetched primary data is valid. [2025-10-20T18:06:11.421695] [DEBUG] [LOCAL ] CPT: Saving primary data. [2025-10-20T18:06:11.423816] [DEBUG] [LOCAL ] CPT: Trimmed 64633 records to 44551 (keeping 200 days of data) [2025-10-20T18:06:11.485988] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-01. [2025-10-20T18:06:11.560163] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T18:06:11.745324] [DEBUG] [LOCAL ] CPT: Primary data saved (44551 records). [2025-10-20T18:06:11.745481] [INFO ] [MAIN ] CPT: Full history saved. [2025-10-20T18:06:11.745569] [INFO ] [MAIN ] Update Progress: 523/2367 symbols completed. CPT took 26.3s. Estimated remaining: 3h 53m 8s. Rate: 7.9 symbols/min [2025-10-20T18:06:11.944560] [INFO ] [MAIN ] CRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:11.986737] [DEBUG] [LOCAL ] CRC: Loaded 44717 existing records from primary storage. [2025-10-20T18:06:11.987050] [INFO ] [MAIN ] CRC: Loaded 44717 existing records. [2025-10-20T18:06:12.188872] [WARN ] [MAIN ] CRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:12.189120] [INFO ] [MAIN ] CRC: Wiping existing primary data before full refresh. [2025-10-20T18:06:12.189170] [INFO ] [LOCAL ] CRC: Starting deletion of primary data. [2025-10-20T18:06:12.189724] [DEBUG] [LOCAL ] CRC: Deleted primary data file. [2025-10-20T18:06:12.189811] [INFO ] [MAIN ] CRC: Starting full primary refresh. [2025-10-20T18:06:12.264229] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-02. [2025-10-20T18:06:12.620961] [INFO ] [REMOTE ] CRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:13.019910] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:06:13.051340] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-03. [2025-10-20T18:06:13.306623] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T18:06:13.810965] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T18:06:13.851353] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-04. [2025-10-20T18:06:14.517339] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T18:06:14.651447] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-02. [2025-10-20T18:06:15.451431] [INFO ] [REMOTE ] CPS: Fetching primary data for 2025-10. [2025-10-20T18:06:16.251395] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-01. [2025-10-20T18:06:16.310490] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:06:16.526718] [INFO ] [REMOTE ] CPS: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T18:06:16.530171] [INFO ] [REMOTE ] CPS: Full fetch complete (36419 records). [2025-10-20T18:06:16.530265] [DEBUG] [VALIDATOR ] CPS: Validating primary data. [2025-10-20T18:06:16.933678] [DEBUG] [VALIDATOR ] CPS: All expected trading days are present. [2025-10-20T18:06:16.933772] [DEBUG] [VALIDATOR ] CPS: Primary data validation passed. [2025-10-20T18:06:16.934298] [INFO ] [MAIN ] CPS: Fetched primary data is valid. [2025-10-20T18:06:16.934339] [DEBUG] [LOCAL ] CPS: Saving primary data. [2025-10-20T18:06:16.935147] [DEBUG] [LOCAL ] CPS: Trimmed 36419 records to 25047 (keeping 200 days of data) [2025-10-20T18:06:17.056133] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-05. [2025-10-20T18:06:17.096713] [DEBUG] [LOCAL ] CPS: Primary data saved (25047 records). [2025-10-20T18:06:17.096857] [INFO ] [MAIN ] CPS: Full history saved. [2025-10-20T18:06:17.096944] [INFO ] [MAIN ] Update Progress: 524/2367 symbols completed. CPS took 31.7s. Estimated remaining: 3h 52m 53s. Rate: 7.9 symbols/min [2025-10-20T18:06:17.274330] [INFO ] [MAIN ] CRCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:17.274522] [DEBUG] [LOCAL ] CRCL: No existing data file found in primary storage. [2025-10-20T18:06:17.274562] [WARN ] [MAIN ] CRCL: No existing data found, performing full refresh instead. [2025-10-20T18:06:17.274702] [INFO ] [MAIN ] CRCL: Wiping existing primary data before full refresh. [2025-10-20T18:06:17.274755] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-10-20T18:06:17.274798] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-10-20T18:06:17.274876] [INFO ] [MAIN ] CRCL: Starting full primary refresh. [2025-10-20T18:06:17.623156] [INFO ] [REMOTE ] CRCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:17.856556] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-03. [2025-10-20T18:06:17.878105] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:06:17.887206] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:06:18.651399] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-02. [2025-10-20T18:06:19.224372] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:06:19.451444] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-04. [2025-10-20T18:06:19.707368] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T18:06:20.251400] [INFO ] [REMOTE ] CRCL: Fetching primary data for 2025-01. [2025-10-20T18:06:20.516537] [DEBUG] [REMOTE ] CRCL: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:06:20.517525] [WARN ] [REMOTE ] CRCL: No data returned for month 2025-01 [2025-10-20T18:06:20.517966] [FATAL] [REMOTE ] CRCL: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:06:20.518050] [FATAL] [MAIN ] CRCL: Failed to fetch primary data. [2025-10-20T18:06:20.518097] [WARN ] [MAIN ] CRCL: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:06:20.518170] [INFO ] [LOCAL ] CRCL: Starting deletion of primary data. [2025-10-20T18:06:20.518255] [DEBUG] [LOCAL ] CRCL: No primary data file found to delete. [2025-10-20T18:06:20.518296] [INFO ] [LOCAL ] CRCL: Starting deletion of derivative data. [2025-10-20T18:06:20.518338] [DEBUG] [LOCAL ] CRCL: No derivative data file found to delete. [2025-10-20T18:06:20.518400] [INFO ] [MAIN ] Update Progress: 525/2367 symbols completed. CRCL took 3.4s. Estimated remaining: 3h 52m 30s. Rate: 7.9 symbols/min [2025-10-20T18:06:20.709455] [INFO ] [MAIN ] CRGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:20.766992] [DEBUG] [LOCAL ] CRGY: Loaded 53283 existing records from primary storage. [2025-10-20T18:06:20.767490] [INFO ] [MAIN ] CRGY: Loaded 53283 existing records. [2025-10-20T18:06:20.946148] [WARN ] [MAIN ] CRGY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:20.946497] [INFO ] [MAIN ] CRGY: Wiping existing primary data before full refresh. [2025-10-20T18:06:20.946694] [INFO ] [LOCAL ] CRGY: Starting deletion of primary data. [2025-10-20T18:06:20.947022] [DEBUG] [LOCAL ] CRGY: Deleted primary data file. [2025-10-20T18:06:20.947142] [INFO ] [MAIN ] CRGY: Starting full primary refresh. [2025-10-20T18:06:21.056504] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-03. [2025-10-20T18:06:21.345014] [INFO ] [REMOTE ] CRGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:21.350395] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:06:21.851378] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-01. [2025-10-20T18:06:22.651383] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-05. [2025-10-20T18:06:22.756368] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:06:23.451393] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-06. [2025-10-20T18:06:23.639330] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:06:24.251367] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-02. [2025-10-20T18:06:24.328214] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:06:24.374901] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:06:25.051372] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-07. [2025-10-20T18:06:25.770152] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T18:06:25.809610] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:06:25.851338] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-03. [2025-10-20T18:06:26.651415] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-04. [2025-10-20T18:06:27.451439] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-06. [2025-10-20T18:06:27.655254] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:06:28.256447] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-08. [2025-10-20T18:06:28.269544] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:06:28.813908] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T18:06:29.029995] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:06:29.051348] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-07. [2025-10-20T18:06:29.851369] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-04. [2025-10-20T18:06:30.409411] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:06:30.651419] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-08. [2025-10-20T18:06:31.451374] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-05. [2025-10-20T18:06:31.922618] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T18:06:32.070800] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:06:32.251373] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-09. [2025-10-20T18:06:32.770171] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:06:33.051384] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-05. [2025-10-20T18:06:33.851466] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-06. [2025-10-20T18:06:33.979030] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T18:06:34.651385] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-09. [2025-10-20T18:06:34.949356] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:06:35.013984] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T18:06:35.227803] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T18:06:35.451374] [INFO ] [REMOTE ] CRBD: Fetching primary data for 2025-10. [2025-10-20T18:06:36.191504] [INFO ] [REMOTE ] CRBD: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:06:36.193175] [INFO ] [REMOTE ] CRBD: Full fetch complete (13026 records). [2025-10-20T18:06:36.193245] [DEBUG] [VALIDATOR ] CRBD: Validating primary data. [2025-10-20T18:06:36.260841] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-06. [2025-10-20T18:06:36.595193] [DEBUG] [VALIDATOR ] CRBD: All expected trading days are present. [2025-10-20T18:06:36.595270] [DEBUG] [VALIDATOR ] CRBD: Primary data validation passed. [2025-10-20T18:06:36.595590] [INFO ] [MAIN ] CRBD: Fetched primary data is valid. [2025-10-20T18:06:36.595628] [DEBUG] [LOCAL ] CRBD: Saving primary data. [2025-10-20T18:06:36.596152] [DEBUG] [LOCAL ] CRBD: Trimmed 13026 records to 8858 (keeping 200 days of data) [2025-10-20T18:06:36.649978] [DEBUG] [LOCAL ] CRBD: Primary data saved (8858 records). [2025-10-20T18:06:36.650099] [INFO ] [MAIN ] CRBD: Full history saved. [2025-10-20T18:06:36.650178] [INFO ] [MAIN ] Update Progress: 526/2367 symbols completed. CRBD took 30.6s. Estimated remaining: 3h 52m 53s. Rate: 7.9 symbols/min [2025-10-20T18:06:36.824299] [INFO ] [MAIN ] CRH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:36.876398] [DEBUG] [LOCAL ] CRH: Loaded 54220 existing records from primary storage. [2025-10-20T18:06:36.876757] [INFO ] [MAIN ] CRH: Loaded 54220 existing records. [2025-10-20T18:06:37.063631] [WARN ] [MAIN ] CRH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:37.058077] [INFO ] [REMOTE ] CRBG: Fetching primary data for 2025-10. [2025-10-20T18:06:37.064812] [INFO ] [MAIN ] CRH: Wiping existing primary data before full refresh. [2025-10-20T18:06:37.064871] [INFO ] [LOCAL ] CRH: Starting deletion of primary data. [2025-10-20T18:06:37.065087] [DEBUG] [LOCAL ] CRH: Deleted primary data file. [2025-10-20T18:06:37.065144] [INFO ] [MAIN ] CRH: Starting full primary refresh. [2025-10-20T18:06:37.426410] [INFO ] [REMOTE ] CRH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:37.851404] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-07. [2025-10-20T18:06:38.030327] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:06:38.651395] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-07. [2025-10-20T18:06:39.038333] [INFO ] [REMOTE ] CRBG: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T18:06:39.046916] [INFO ] [REMOTE ] CRBG: Full fetch complete (76047 records). [2025-10-20T18:06:39.047240] [DEBUG] [VALIDATOR ] CRBG: Validating primary data. [2025-10-20T18:06:39.354506] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:06:39.456097] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-08. [2025-10-20T18:06:39.494526] [DEBUG] [VALIDATOR ] CRBG: All expected trading days are present. [2025-10-20T18:06:39.494619] [DEBUG] [VALIDATOR ] CRBG: Primary data validation passed. [2025-10-20T18:06:39.495540] [INFO ] [MAIN ] CRBG: Fetched primary data is valid. [2025-10-20T18:06:39.495598] [DEBUG] [LOCAL ] CRBG: Saving primary data. [2025-10-20T18:06:39.497232] [DEBUG] [LOCAL ] CRBG: Trimmed 76047 records to 52602 (keeping 200 days of data) [2025-10-20T18:06:39.810256] [DEBUG] [LOCAL ] CRBG: Primary data saved (52602 records). [2025-10-20T18:06:39.810401] [INFO ] [MAIN ] CRBG: Full history saved. [2025-10-20T18:06:39.810474] [INFO ] [MAIN ] Update Progress: 527/2367 symbols completed. CRBG took 30.9s. Estimated remaining: 3h 52m 30s. Rate: 7.9 symbols/min [2025-10-20T18:06:39.979372] [INFO ] [MAIN ] CRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:39.979561] [DEBUG] [LOCAL ] CRI: No existing data file found in primary storage. [2025-10-20T18:06:39.979639] [WARN ] [MAIN ] CRI: No existing data found, performing full refresh instead. [2025-10-20T18:06:39.979787] [INFO ] [MAIN ] CRI: Wiping existing primary data before full refresh. [2025-10-20T18:06:39.979831] [INFO ] [LOCAL ] CRI: Starting deletion of primary data. [2025-10-20T18:06:39.979876] [DEBUG] [LOCAL ] CRI: No primary data file found to delete. [2025-10-20T18:06:39.979912] [INFO ] [MAIN ] CRI: Starting full primary refresh. [2025-10-20T18:06:40.258637] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-01. [2025-10-20T18:06:40.406849] [INFO ] [REMOTE ] CRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:41.051349] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-01. [2025-10-20T18:06:41.102475] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-07 (2.45 seconds). [2025-10-20T18:06:41.864266] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-08. [2025-10-20T18:06:41.865956] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-08 (2.41 seconds). [2025-10-20T18:06:42.651433] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-09. [2025-10-20T18:06:43.173156] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-01 (2.91 seconds). [2025-10-20T18:06:43.451421] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-02. [2025-10-20T18:06:43.652956] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:06:43.890374] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T18:06:44.028726] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-01 (2.98 seconds). [2025-10-20T18:06:44.251480] [INFO ] [REMOTE ] CRC: Fetching primary data for 2025-10. [2025-10-20T18:06:44.901012] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T18:06:45.051366] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-03. [2025-10-20T18:06:45.851360] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-09. [2025-10-20T18:06:46.098979] [INFO ] [REMOTE ] CRC: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:06:46.108905] [INFO ] [REMOTE ] CRC: Full fetch complete (62964 records). [2025-10-20T18:06:46.109083] [DEBUG] [VALIDATOR ] CRC: Validating primary data. [2025-10-20T18:06:46.495861] [DEBUG] [VALIDATOR ] CRC: All expected trading days are present. [2025-10-20T18:06:46.495953] [DEBUG] [VALIDATOR ] CRC: Primary data validation passed. [2025-10-20T18:06:46.496711] [INFO ] [MAIN ] CRC: Fetched primary data is valid. [2025-10-20T18:06:46.496775] [DEBUG] [LOCAL ] CRC: Saving primary data. [2025-10-20T18:06:46.498304] [DEBUG] [LOCAL ] CRC: Trimmed 62964 records to 44124 (keeping 200 days of data) [2025-10-20T18:06:46.667291] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-02. [2025-10-20T18:06:46.761985] [DEBUG] [LOCAL ] CRC: Primary data saved (44124 records). [2025-10-20T18:06:46.762130] [INFO ] [MAIN ] CRC: Full history saved. [2025-10-20T18:06:46.762212] [INFO ] [MAIN ] Update Progress: 528/2367 symbols completed. CRC took 35.0s. Estimated remaining: 3h 52m 20s. Rate: 7.9 symbols/min [2025-10-20T18:06:46.938692] [INFO ] [MAIN ] CRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:46.991887] [DEBUG] [LOCAL ] CRK: Loaded 53025 existing records from primary storage. [2025-10-20T18:06:46.992212] [INFO ] [MAIN ] CRK: Loaded 53025 existing records. [2025-10-20T18:06:47.185945] [WARN ] [MAIN ] CRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:47.186292] [INFO ] [MAIN ] CRK: Wiping existing primary data before full refresh. [2025-10-20T18:06:47.186349] [INFO ] [LOCAL ] CRK: Starting deletion of primary data. [2025-10-20T18:06:47.186533] [DEBUG] [LOCAL ] CRK: Deleted primary data file. [2025-10-20T18:06:47.186572] [INFO ] [MAIN ] CRK: Starting full primary refresh. [2025-10-20T18:06:47.354011] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-03 (2.30 seconds). [2025-10-20T18:06:47.461035] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-04. [2025-10-20T18:06:47.610062] [INFO ] [REMOTE ] CRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:47.770269] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:06:47.909767] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T18:06:48.251443] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-03. [2025-10-20T18:06:49.051415] [INFO ] [REMOTE ] CRGY: Fetching primary data for 2025-10. [2025-10-20T18:06:49.499615] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:06:49.851408] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-05. [2025-10-20T18:06:49.956282] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:06:50.590050] [INFO ] [REMOTE ] CRGY: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:06:50.600702] [INFO ] [REMOTE ] CRGY: Full fetch complete (76159 records). [2025-10-20T18:06:50.600852] [DEBUG] [VALIDATOR ] CRGY: Validating primary data. [2025-10-20T18:06:50.656663] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-01. [2025-10-20T18:06:50.992795] [DEBUG] [VALIDATOR ] CRGY: All expected trading days are present. [2025-10-20T18:06:50.992877] [DEBUG] [VALIDATOR ] CRGY: Primary data validation passed. [2025-10-20T18:06:50.993732] [INFO ] [MAIN ] CRGY: Fetched primary data is valid. [2025-10-20T18:06:50.993796] [DEBUG] [LOCAL ] CRGY: Saving primary data. [2025-10-20T18:06:50.995182] [DEBUG] [LOCAL ] CRGY: Trimmed 76159 records to 52514 (keeping 200 days of data) [2025-10-20T18:06:51.299621] [DEBUG] [LOCAL ] CRGY: Primary data saved (52514 records). [2025-10-20T18:06:51.299783] [INFO ] [MAIN ] CRGY: Full history saved. [2025-10-20T18:06:51.299863] [INFO ] [MAIN ] Update Progress: 529/2367 symbols completed. CRGY took 30.8s. Estimated remaining: 3h 52m 2s. Rate: 7.9 symbols/min [2025-10-20T18:06:51.471865] [INFO ] [MAIN ] CRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:06:51.468347] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-04. [2025-10-20T18:06:51.507202] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T18:06:51.550293] [DEBUG] [LOCAL ] CRL: Loaded 43452 existing records from primary storage. [2025-10-20T18:06:51.550605] [INFO ] [MAIN ] CRL: Loaded 43452 existing records. [2025-10-20T18:06:51.738708] [WARN ] [MAIN ] CRL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:06:51.738958] [INFO ] [MAIN ] CRL: Wiping existing primary data before full refresh. [2025-10-20T18:06:51.739013] [INFO ] [LOCAL ] CRL: Starting deletion of primary data. [2025-10-20T18:06:51.739204] [DEBUG] [LOCAL ] CRL: Deleted primary data file. [2025-10-20T18:06:51.739240] [INFO ] [MAIN ] CRL: Starting full primary refresh. [2025-10-20T18:06:52.118521] [INFO ] [REMOTE ] CRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:06:52.251382] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-01. [2025-10-20T18:06:52.556920] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:06:53.051465] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-02. [2025-10-20T18:06:53.294862] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:06:53.387936] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:06:53.851414] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-02. [2025-10-20T18:06:54.661970] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-05. [2025-10-20T18:06:54.667886] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:06:55.431323] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:06:55.451347] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-03. [2025-10-20T18:06:56.251447] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-06. [2025-10-20T18:06:57.051354] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-03. [2025-10-20T18:06:57.555666] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-05 (2.89 seconds). [2025-10-20T18:06:57.834787] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T18:06:57.851323] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-04. [2025-10-20T18:06:58.425272] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:06:58.644256] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:06:58.654784] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-06. [2025-10-20T18:06:59.451406] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-04. [2025-10-20T18:06:59.782860] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:06:59.944620] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:07:00.251354] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-05. [2025-10-20T18:07:01.051381] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-07. [2025-10-20T18:07:01.301859] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:07:01.851409] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-05. [2025-10-20T18:07:02.043180] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:07:02.651396] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-06. [2025-10-20T18:07:02.798339] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:07:03.451404] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-07. [2025-10-20T18:07:03.748765] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T18:07:03.862269] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:07:04.251452] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-07. [2025-10-20T18:07:05.051434] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-06. [2025-10-20T18:07:05.438574] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T18:07:05.638006] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:07:05.851427] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-08. [2025-10-20T18:07:06.651372] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-08. [2025-10-20T18:07:06.692009] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:07:07.443934] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:07:07.458192] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-08. [2025-10-20T18:07:08.015779] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:07:08.251391] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-09. [2025-10-20T18:07:08.958432] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:07:09.051338] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-09. [2025-10-20T18:07:09.851359] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-07. [2025-10-20T18:07:09.928464] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:07:10.250876] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:07:10.651380] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-09. [2025-10-20T18:07:11.451445] [INFO ] [REMOTE ] CRI: Fetching primary data for 2025-10. [2025-10-20T18:07:12.251404] [INFO ] [REMOTE ] CRL: Fetching primary data for 2025-10. [2025-10-20T18:07:12.315663] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-10-20T18:07:12.396759] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:07:13.051495] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-08. [2025-10-20T18:07:13.260982] [INFO ] [REMOTE ] CRI: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:07:13.267965] [INFO ] [REMOTE ] CRI: Full fetch complete (70923 records). [2025-10-20T18:07:13.268134] [DEBUG] [VALIDATOR ] CRI: Validating primary data. [2025-10-20T18:07:13.656932] [DEBUG] [VALIDATOR ] CRI: All expected trading days are present. [2025-10-20T18:07:13.657113] [DEBUG] [VALIDATOR ] CRI: Primary data validation passed. [2025-10-20T18:07:13.658022] [INFO ] [MAIN ] CRI: Fetched primary data is valid. [2025-10-20T18:07:13.658134] [DEBUG] [LOCAL ] CRI: Saving primary data. [2025-10-20T18:07:13.659687] [DEBUG] [LOCAL ] CRI: Trimmed 70923 records to 49604 (keeping 200 days of data) [2025-10-20T18:07:13.853527] [INFO ] [REMOTE ] CRH: Fetching primary data for 2025-10. [2025-10-20T18:07:13.866248] [INFO ] [REMOTE ] CRL: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T18:07:13.922100] [INFO ] [REMOTE ] CRL: Full fetch complete (63531 records). [2025-10-20T18:07:13.933534] [DEBUG] [VALIDATOR ] CRL: Validating primary data. [2025-10-20T18:07:14.094023] [DEBUG] [LOCAL ] CRI: Primary data saved (49604 records). [2025-10-20T18:07:14.104234] [INFO ] [MAIN ] CRI: Full history saved. [2025-10-20T18:07:14.104387] [INFO ] [MAIN ] Update Progress: 530/2367 symbols completed. CRI took 34.3s. Estimated remaining: 3h 52m 47s. Rate: 7.9 symbols/min [2025-10-20T18:07:14.464255] [INFO ] [MAIN ] CRM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:14.625342] [DEBUG] [VALIDATOR ] CRL: All expected trading days are present. [2025-10-20T18:07:14.625478] [DEBUG] [VALIDATOR ] CRL: Primary data validation passed. [2025-10-20T18:07:14.626468] [INFO ] [MAIN ] CRL: Fetched primary data is valid. [2025-10-20T18:07:14.626764] [DEBUG] [LOCAL ] CRL: Saving primary data. [2025-10-20T18:07:14.639076] [DEBUG] [LOCAL ] CRL: Trimmed 63531 records to 42666 (keeping 200 days of data) [2025-10-20T18:07:14.662037] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T18:07:14.774798] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-09. [2025-10-20T18:07:14.800363] [DEBUG] [LOCAL ] CRM: Loaded 54550 existing records from primary storage. [2025-10-20T18:07:14.802090] [INFO ] [MAIN ] CRM: Loaded 54550 existing records. [2025-10-20T18:07:15.006628] [WARN ] [MAIN ] CRM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:15.011871] [INFO ] [MAIN ] CRM: Wiping existing primary data before full refresh. [2025-10-20T18:07:15.012676] [INFO ] [LOCAL ] CRM: Starting deletion of primary data. [2025-10-20T18:07:15.014298] [DEBUG] [LOCAL ] CRM: Deleted primary data file. [2025-10-20T18:07:15.016781] [INFO ] [MAIN ] CRM: Starting full primary refresh. [2025-10-20T18:07:15.298120] [DEBUG] [LOCAL ] CRL: Primary data saved (42666 records). [2025-10-20T18:07:15.329822] [INFO ] [MAIN ] CRL: Full history saved. [2025-10-20T18:07:15.350010] [INFO ] [MAIN ] Update Progress: 531/2367 symbols completed. CRL took 24.1s. Estimated remaining: 3h 52m 18s. Rate: 7.9 symbols/min [2025-10-20T18:07:15.726226] [INFO ] [REMOTE ] CRM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:15.747843] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-01. [2025-10-20T18:07:15.805818] [INFO ] [MAIN ] CRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:15.852070] [DEBUG] [LOCAL ] CRS: Loaded 43714 existing records from primary storage. [2025-10-20T18:07:15.852570] [INFO ] [MAIN ] CRS: Loaded 43714 existing records. [2025-10-20T18:07:16.057240] [WARN ] [MAIN ] CRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:16.057634] [INFO ] [MAIN ] CRS: Wiping existing primary data before full refresh. [2025-10-20T18:07:16.057770] [INFO ] [LOCAL ] CRS: Starting deletion of primary data. [2025-10-20T18:07:16.066102] [DEBUG] [LOCAL ] CRS: Deleted primary data file. [2025-10-20T18:07:16.078487] [INFO ] [MAIN ] CRS: Starting full primary refresh. [2025-10-20T18:07:16.168046] [INFO ] [REMOTE ] CRH: Incremental fetch complete for 2025-10 (2.31 seconds). [2025-10-20T18:07:16.206409] [INFO ] [REMOTE ] CRH: Full fetch complete (77553 records). [2025-10-20T18:07:16.227721] [DEBUG] [VALIDATOR ] CRH: Validating primary data. [2025-10-20T18:07:16.761986] [INFO ] [REMOTE ] CRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:16.800770] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-01. [2025-10-20T18:07:16.916370] [DEBUG] [VALIDATOR ] CRH: All expected trading days are present. [2025-10-20T18:07:16.916480] [DEBUG] [VALIDATOR ] CRH: Primary data validation passed. [2025-10-20T18:07:16.917355] [INFO ] [MAIN ] CRH: Fetched primary data is valid. [2025-10-20T18:07:16.917470] [DEBUG] [LOCAL ] CRH: Saving primary data. [2025-10-20T18:07:16.947857] [DEBUG] [LOCAL ] CRH: Trimmed 77553 records to 53439 (keeping 200 days of data) [2025-10-20T18:07:16.949599] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:07:17.052916] [INFO ] [REMOTE ] CRK: Fetching primary data for 2025-10. [2025-10-20T18:07:17.283459] [DEBUG] [LOCAL ] CRH: Primary data saved (53439 records). [2025-10-20T18:07:17.283590] [INFO ] [MAIN ] CRH: Full history saved. [2025-10-20T18:07:17.283876] [INFO ] [MAIN ] Update Progress: 532/2367 symbols completed. CRH took 40.6s. Estimated remaining: 3h 51m 50s. Rate: 7.9 symbols/min [2025-10-20T18:07:17.452400] [INFO ] [MAIN ] CRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:17.459894] [DEBUG] [LOCAL ] CRT: Loaded 4975 existing records from primary storage. [2025-10-20T18:07:17.460111] [INFO ] [MAIN ] CRT: Loaded 4975 existing records. [2025-10-20T18:07:17.634243] [WARN ] [MAIN ] CRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:17.634825] [INFO ] [MAIN ] CRT: Wiping existing primary data before full refresh. [2025-10-20T18:07:17.634877] [INFO ] [LOCAL ] CRT: Starting deletion of primary data. [2025-10-20T18:07:17.635008] [DEBUG] [LOCAL ] CRT: Deleted primary data file. [2025-10-20T18:07:17.635046] [INFO ] [MAIN ] CRT: Starting full primary refresh. [2025-10-20T18:07:17.994393] [INFO ] [REMOTE ] CRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:17.994787] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-01. [2025-10-20T18:07:18.148727] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:07:18.364240] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:07:18.568041] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:07:18.651353] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-02. [2025-10-20T18:07:18.940999] [INFO ] [REMOTE ] CRK: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:07:18.949250] [INFO ] [REMOTE ] CRK: Full fetch complete (75841 records). [2025-10-20T18:07:18.949421] [DEBUG] [VALIDATOR ] CRK: Validating primary data. [2025-10-20T18:07:19.110112] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:07:19.375374] [DEBUG] [VALIDATOR ] CRK: All expected trading days are present. [2025-10-20T18:07:19.375456] [DEBUG] [VALIDATOR ] CRK: Primary data validation passed. [2025-10-20T18:07:19.376454] [INFO ] [MAIN ] CRK: Fetched primary data is valid. [2025-10-20T18:07:19.376526] [DEBUG] [LOCAL ] CRK: Saving primary data. [2025-10-20T18:07:19.378001] [DEBUG] [LOCAL ] CRK: Trimmed 75841 records to 52250 (keeping 200 days of data) [2025-10-20T18:07:19.459132] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-02. [2025-10-20T18:07:19.676080] [DEBUG] [LOCAL ] CRK: Primary data saved (52250 records). [2025-10-20T18:07:19.676206] [INFO ] [MAIN ] CRK: Full history saved. [2025-10-20T18:07:19.676281] [INFO ] [MAIN ] Update Progress: 533/2367 symbols completed. CRK took 32.9s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:07:19.846237] [INFO ] [MAIN ] CSAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:19.884087] [DEBUG] [LOCAL ] CSAN: Loaded 42773 existing records from primary storage. [2025-10-20T18:07:19.884366] [INFO ] [MAIN ] CSAN: Loaded 42773 existing records. [2025-10-20T18:07:20.058065] [WARN ] [MAIN ] CSAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:20.058273] [INFO ] [MAIN ] CSAN: Wiping existing primary data before full refresh. [2025-10-20T18:07:20.058320] [INFO ] [LOCAL ] CSAN: Starting deletion of primary data. [2025-10-20T18:07:20.058499] [DEBUG] [LOCAL ] CSAN: Deleted primary data file. [2025-10-20T18:07:20.058557] [INFO ] [MAIN ] CSAN: Starting full primary refresh. [2025-10-20T18:07:20.260714] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-03. [2025-10-20T18:07:20.405950] [INFO ] [REMOTE ] CSAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:20.937025] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T18:07:20.952467] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T18:07:21.051398] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-03. [2025-10-20T18:07:21.851403] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-01. [2025-10-20T18:07:22.351790] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:07:22.651400] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-04. [2025-10-20T18:07:23.009545] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:07:23.451395] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-04. [2025-10-20T18:07:24.058869] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:07:24.251378] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-02. [2025-10-20T18:07:24.336534] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:07:25.051369] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-05. [2025-10-20T18:07:25.456835] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T18:07:25.635404] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:07:25.851369] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-06. [2025-10-20T18:07:26.477313] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T18:07:26.651378] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-07. [2025-10-20T18:07:27.266116] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:07:27.451345] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-02. [2025-10-20T18:07:28.251372] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-08. [2025-10-20T18:07:28.966172] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T18:07:29.051392] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-09. [2025-10-20T18:07:29.688091] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-10-20T18:07:29.688381] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T18:07:29.851440] [INFO ] [REMOTE ] CRT: Fetching primary data for 2025-10. [2025-10-20T18:07:30.483332] [INFO ] [REMOTE ] CRT: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T18:07:30.484807] [INFO ] [REMOTE ] CRT: Full fetch complete (7361 records). [2025-10-20T18:07:30.484879] [DEBUG] [VALIDATOR ] CRT: Validating primary data. [2025-10-20T18:07:30.653980] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-03. [2025-10-20T18:07:30.877382] [DEBUG] [VALIDATOR ] CRT: All expected trading days are present. [2025-10-20T18:07:30.877494] [DEBUG] [VALIDATOR ] CRT: Primary data validation passed. [2025-10-20T18:07:30.877854] [INFO ] [MAIN ] CRT: Fetched primary data is valid. [2025-10-20T18:07:30.877919] [DEBUG] [LOCAL ] CRT: Saving primary data. [2025-10-20T18:07:30.878376] [DEBUG] [LOCAL ] CRT: Trimmed 7361 records to 4802 (keeping 200 days of data) [2025-10-20T18:07:30.909760] [DEBUG] [LOCAL ] CRT: Primary data saved (4802 records). [2025-10-20T18:07:30.909972] [INFO ] [MAIN ] CRT: Full history saved. [2025-10-20T18:07:30.910089] [INFO ] [MAIN ] Update Progress: 534/2367 symbols completed. CRT took 13.6s. Estimated remaining: 3h 51m 30s. Rate: 7.9 symbols/min [2025-10-20T18:07:31.100193] [INFO ] [MAIN ] CSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:31.136375] [DEBUG] [LOCAL ] CSL: Loaded 31991 existing records from primary storage. [2025-10-20T18:07:31.136805] [INFO ] [MAIN ] CSL: Loaded 31991 existing records. [2025-10-20T18:07:31.326266] [WARN ] [MAIN ] CSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:31.326596] [INFO ] [MAIN ] CSL: Wiping existing primary data before full refresh. [2025-10-20T18:07:31.326672] [INFO ] [LOCAL ] CSL: Starting deletion of primary data. [2025-10-20T18:07:31.326979] [DEBUG] [LOCAL ] CSL: Deleted primary data file. [2025-10-20T18:07:31.327037] [INFO ] [MAIN ] CSL: Starting full primary refresh. [2025-10-20T18:07:31.456507] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-03. [2025-10-20T18:07:31.725906] [INFO ] [REMOTE ] CSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:32.006276] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:07:32.251403] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-04. [2025-10-20T18:07:33.051374] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-01. [2025-10-20T18:07:33.522165] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T18:07:33.567441] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T18:07:33.851350] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-05. [2025-10-20T18:07:34.366063] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:07:34.651380] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-05. [2025-10-20T18:07:35.186742] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:07:35.451377] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-04. [2025-10-20T18:07:35.672358] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T18:07:36.251380] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-06. [2025-10-20T18:07:37.051371] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-06. [2025-10-20T18:07:37.629408] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T18:07:37.851354] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-05. [2025-10-20T18:07:37.977752] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:07:38.553226] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T18:07:38.651367] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-07. [2025-10-20T18:07:39.451383] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-07. [2025-10-20T18:07:39.834126] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T18:07:40.006047] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T18:07:40.251612] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-06. [2025-10-20T18:07:41.052837] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-08. [2025-10-20T18:07:41.052754] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T18:07:41.851425] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-08. [2025-10-20T18:07:42.615706] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T18:07:42.656554] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-09. [2025-10-20T18:07:42.679792] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-06 (2.43 seconds). [2025-10-20T18:07:42.915737] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T18:07:43.451421] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-09. [2025-10-20T18:07:44.251370] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-02. [2025-10-20T18:07:44.290639] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:07:45.051431] [INFO ] [REMOTE ] CRS: Fetching primary data for 2025-10. [2025-10-20T18:07:45.116714] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:07:45.851354] [INFO ] [REMOTE ] CSAN: Fetching primary data for 2025-10. [2025-10-20T18:07:45.894708] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:07:46.651415] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-03. [2025-10-20T18:07:46.764248] [INFO ] [REMOTE ] CRS: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:07:46.773784] [INFO ] [REMOTE ] CRS: Full fetch complete (62496 records). [2025-10-20T18:07:46.773980] [DEBUG] [VALIDATOR ] CRS: Validating primary data. [2025-10-20T18:07:47.180006] [DEBUG] [VALIDATOR ] CRS: All expected trading days are present. [2025-10-20T18:07:47.180085] [DEBUG] [VALIDATOR ] CRS: Primary data validation passed. [2025-10-20T18:07:47.180833] [INFO ] [MAIN ] CRS: Fetched primary data is valid. [2025-10-20T18:07:47.180895] [DEBUG] [LOCAL ] CRS: Saving primary data. [2025-10-20T18:07:47.182146] [DEBUG] [LOCAL ] CRS: Trimmed 62496 records to 43124 (keeping 200 days of data) [2025-10-20T18:07:47.388415] [INFO ] [REMOTE ] CSAN: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T18:07:47.457936] [INFO ] [REMOTE ] CSAN: Full fetch complete (60284 records). [2025-10-20T18:07:47.457717] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-07. [2025-10-20T18:07:47.476493] [DEBUG] [LOCAL ] CRS: Primary data saved (43124 records). [2025-10-20T18:07:47.467265] [DEBUG] [VALIDATOR ] CSAN: Validating primary data. [2025-10-20T18:07:47.478000] [INFO ] [MAIN ] CRS: Full history saved. [2025-10-20T18:07:47.478110] [INFO ] [MAIN ] Update Progress: 535/2367 symbols completed. CRS took 32.1s. Estimated remaining: 3h 51m 53s. Rate: 7.9 symbols/min [2025-10-20T18:07:47.827116] [INFO ] [MAIN ] CSR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:47.996356] [DEBUG] [LOCAL ] CSR: Loaded 13649 existing records from primary storage. [2025-10-20T18:07:48.001178] [INFO ] [MAIN ] CSR: Loaded 13649 existing records. [2025-10-20T18:07:48.216374] [DEBUG] [VALIDATOR ] CSAN: All expected trading days are present. [2025-10-20T18:07:48.216560] [DEBUG] [VALIDATOR ] CSAN: Primary data validation passed. [2025-10-20T18:07:48.217387] [INFO ] [MAIN ] CSAN: Fetched primary data is valid. [2025-10-20T18:07:48.217499] [DEBUG] [LOCAL ] CSAN: Saving primary data. [2025-10-20T18:07:48.297594] [WARN ] [MAIN ] CSR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:48.297764] [DEBUG] [LOCAL ] CSAN: Trimmed 60284 records to 42287 (keeping 200 days of data) [2025-10-20T18:07:48.298056] [INFO ] [MAIN ] CSR: Wiping existing primary data before full refresh. [2025-10-20T18:07:48.298150] [INFO ] [LOCAL ] CSR: Starting deletion of primary data. [2025-10-20T18:07:48.299006] [DEBUG] [LOCAL ] CSR: Deleted primary data file. [2025-10-20T18:07:48.299130] [INFO ] [MAIN ] CSR: Starting full primary refresh. [2025-10-20T18:07:48.537287] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:07:48.607902] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-04. [2025-10-20T18:07:48.822547] [INFO ] [REMOTE ] CSR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:48.947630] [DEBUG] [LOCAL ] CSAN: Primary data saved (42287 records). [2025-10-20T18:07:48.947794] [INFO ] [MAIN ] CSAN: Full history saved. [2025-10-20T18:07:48.947875] [INFO ] [MAIN ] Update Progress: 536/2367 symbols completed. CSAN took 29.3s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:07:49.056595] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-01. [2025-10-20T18:07:49.118762] [INFO ] [MAIN ] CSTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:07:49.163407] [DEBUG] [LOCAL ] CSTM: Loaded 48273 existing records from primary storage. [2025-10-20T18:07:49.163795] [INFO ] [MAIN ] CSTM: Loaded 48273 existing records. [2025-10-20T18:07:49.345203] [WARN ] [MAIN ] CSTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:07:49.345569] [INFO ] [MAIN ] CSTM: Wiping existing primary data before full refresh. [2025-10-20T18:07:49.345628] [INFO ] [LOCAL ] CSTM: Starting deletion of primary data. [2025-10-20T18:07:49.346206] [DEBUG] [LOCAL ] CSTM: Deleted primary data file. [2025-10-20T18:07:49.346264] [INFO ] [MAIN ] CSTM: Starting full primary refresh. [2025-10-20T18:07:49.700326] [INFO ] [REMOTE ] CSTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:07:49.851393] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-01. [2025-10-20T18:07:50.046242] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-07 (2.59 seconds). [2025-10-20T18:07:50.068092] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T18:07:50.229404] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:07:50.651432] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-02. [2025-10-20T18:07:51.163437] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:07:51.451366] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-03. [2025-10-20T18:07:51.542381] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:07:52.010675] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:07:52.251425] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-05. [2025-10-20T18:07:53.051409] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-04. [2025-10-20T18:07:53.211718] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T18:07:53.739084] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T18:07:53.851384] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-06. [2025-10-20T18:07:54.651394] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-05. [2025-10-20T18:07:55.040062] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:07:55.241267] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:07:55.451399] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-07. [2025-10-20T18:07:56.251380] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-08. [2025-10-20T18:07:56.879246] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:07:57.051454] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-02. [2025-10-20T18:07:57.851434] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-08. [2025-10-20T18:07:58.308968] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T18:07:58.571299] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:07:58.651353] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-03. [2025-10-20T18:07:59.245784] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:07:59.451407] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-09. [2025-10-20T18:07:59.942008] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T18:08:00.251441] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-09. [2025-10-20T18:08:01.051389] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-04. [2025-10-20T18:08:01.673798] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:08:01.851345] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-06. [2025-10-20T18:08:02.295608] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-09 (2.84 seconds). [2025-10-20T18:08:02.651351] [INFO ] [REMOTE ] CRM: Fetching primary data for 2025-10. [2025-10-20T18:08:02.897225] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:08:02.908353] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:08:03.141202] [INFO ] [REMOTE ] CRM: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T18:08:03.149007] [INFO ] [REMOTE ] CRM: Full fetch complete (77946 records). [2025-10-20T18:08:03.149132] [DEBUG] [VALIDATOR ] CRM: Validating primary data. [2025-10-20T18:08:03.451345] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-07. [2025-10-20T18:08:03.548075] [DEBUG] [VALIDATOR ] CRM: All expected trading days are present. [2025-10-20T18:08:03.548154] [DEBUG] [VALIDATOR ] CRM: Primary data validation passed. [2025-10-20T18:08:03.549036] [INFO ] [MAIN ] CRM: Fetched primary data is valid. [2025-10-20T18:08:03.549099] [DEBUG] [LOCAL ] CRM: Saving primary data. [2025-10-20T18:08:03.550473] [DEBUG] [LOCAL ] CRM: Trimmed 77946 records to 53770 (keeping 200 days of data) [2025-10-20T18:08:03.896857] [DEBUG] [LOCAL ] CRM: Primary data saved (53770 records). [2025-10-20T18:08:03.897253] [INFO ] [MAIN ] CRM: Full history saved. [2025-10-20T18:08:03.897355] [INFO ] [MAIN ] Update Progress: 537/2367 symbols completed. CRM took 49.8s. Estimated remaining: 3h 51m 42s. Rate: 7.9 symbols/min [2025-10-20T18:08:04.072633] [INFO ] [MAIN ] CSV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:04.086093] [DEBUG] [LOCAL ] CSV: Loaded 12794 existing records from primary storage. [2025-10-20T18:08:04.086301] [INFO ] [MAIN ] CSV: Loaded 12794 existing records. [2025-10-20T18:08:04.262887] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-05. [2025-10-20T18:08:04.268743] [WARN ] [MAIN ] CSV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:04.269672] [INFO ] [MAIN ] CSV: Wiping existing primary data before full refresh. [2025-10-20T18:08:04.269743] [INFO ] [LOCAL ] CSV: Starting deletion of primary data. [2025-10-20T18:08:04.270121] [DEBUG] [LOCAL ] CSV: Deleted primary data file. [2025-10-20T18:08:04.270201] [INFO ] [MAIN ] CSV: Starting full primary refresh. [2025-10-20T18:08:04.554006] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:08:04.647479] [INFO ] [REMOTE ] CSV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:05.051364] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-01. [2025-10-20T18:08:05.802095] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:08:05.821001] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:08:05.851331] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-02. [2025-10-20T18:08:06.651361] [INFO ] [REMOTE ] CSL: Fetching primary data for 2025-10. [2025-10-20T18:08:06.719037] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:08:07.451419] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-03. [2025-10-20T18:08:08.147290] [INFO ] [REMOTE ] CSL: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:08:08.152204] [INFO ] [REMOTE ] CSL: Full fetch complete (45617 records). [2025-10-20T18:08:08.152313] [DEBUG] [VALIDATOR ] CSL: Validating primary data. [2025-10-20T18:08:08.252057] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-08. [2025-10-20T18:08:08.542251] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T18:08:08.579357] [DEBUG] [VALIDATOR ] CSL: All expected trading days are present. [2025-10-20T18:08:08.579436] [DEBUG] [VALIDATOR ] CSL: Primary data validation passed. [2025-10-20T18:08:08.580058] [INFO ] [MAIN ] CSL: Fetched primary data is valid. [2025-10-20T18:08:08.580122] [DEBUG] [LOCAL ] CSL: Saving primary data. [2025-10-20T18:08:08.581063] [DEBUG] [LOCAL ] CSL: Trimmed 45617 records to 31310 (keeping 200 days of data) [2025-10-20T18:08:08.773621] [DEBUG] [LOCAL ] CSL: Primary data saved (31310 records). [2025-10-20T18:08:08.773803] [INFO ] [MAIN ] CSL: Full history saved. [2025-10-20T18:08:08.774316] [INFO ] [MAIN ] Update Progress: 538/2367 symbols completed. CSL took 37.9s. Estimated remaining: 3h 51m 25s. Rate: 7.9 symbols/min [2025-10-20T18:08:08.948218] [INFO ] [MAIN ] CSW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:08.948468] [DEBUG] [LOCAL ] CSW: No existing data file found in primary storage. [2025-10-20T18:08:08.948604] [WARN ] [MAIN ] CSW: No existing data found, performing full refresh instead. [2025-10-20T18:08:08.948751] [INFO ] [MAIN ] CSW: Wiping existing primary data before full refresh. [2025-10-20T18:08:08.948792] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-10-20T18:08:08.948831] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-10-20T18:08:08.948864] [INFO ] [MAIN ] CSW: Starting full primary refresh. [2025-10-20T18:08:09.056772] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-04. [2025-10-20T18:08:09.186205] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:08:09.401379] [INFO ] [REMOTE ] CSW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:09.851368] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-09. [2025-10-20T18:08:10.154601] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T18:08:10.651372] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-05. [2025-10-20T18:08:10.812512] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T18:08:11.451394] [INFO ] [REMOTE ] CSW: Fetching primary data for 2025-01. [2025-10-20T18:08:11.487774] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:08:11.660046] [DEBUG] [REMOTE ] CSW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:08:11.660361] [WARN ] [REMOTE ] CSW: No data returned for month 2025-01 [2025-10-20T18:08:11.660465] [FATAL] [REMOTE ] CSW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:08:11.660567] [FATAL] [MAIN ] CSW: Failed to fetch primary data. [2025-10-20T18:08:11.660678] [WARN ] [MAIN ] CSW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:08:11.660775] [INFO ] [LOCAL ] CSW: Starting deletion of primary data. [2025-10-20T18:08:11.660858] [DEBUG] [LOCAL ] CSW: No primary data file found to delete. [2025-10-20T18:08:11.660897] [INFO ] [LOCAL ] CSW: Starting deletion of derivative data. [2025-10-20T18:08:11.660938] [DEBUG] [LOCAL ] CSW: No derivative data file found to delete. [2025-10-20T18:08:11.660985] [INFO ] [MAIN ] Update Progress: 539/2367 symbols completed. CSW took 2.9s. Estimated remaining: 3h 51m 2s. Rate: 7.9 symbols/min [2025-10-20T18:08:11.843276] [INFO ] [MAIN ] CTBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:11.843487] [DEBUG] [LOCAL ] CTBB: No existing data file found in primary storage. [2025-10-20T18:08:11.843539] [WARN ] [MAIN ] CTBB: No existing data found, performing full refresh instead. [2025-10-20T18:08:11.843732] [INFO ] [MAIN ] CTBB: Wiping existing primary data before full refresh. [2025-10-20T18:08:11.843787] [INFO ] [LOCAL ] CTBB: Starting deletion of primary data. [2025-10-20T18:08:11.843830] [DEBUG] [LOCAL ] CTBB: No primary data file found to delete. [2025-10-20T18:08:11.843863] [INFO ] [MAIN ] CTBB: Starting full primary refresh. [2025-10-20T18:08:12.204804] [INFO ] [REMOTE ] CTBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:12.251369] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-01. [2025-10-20T18:08:13.021882] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:08:13.051378] [INFO ] [REMOTE ] CSR: Fetching primary data for 2025-10. [2025-10-20T18:08:13.851401] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-02. [2025-10-20T18:08:14.085266] [INFO ] [REMOTE ] CSR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:08:14.088081] [INFO ] [REMOTE ] CSR: Full fetch complete (17973 records). [2025-10-20T18:08:14.088154] [DEBUG] [VALIDATOR ] CSR: Validating primary data. [2025-10-20T18:08:14.419812] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T18:08:14.482701] [DEBUG] [VALIDATOR ] CSR: All expected trading days are present. [2025-10-20T18:08:14.482789] [DEBUG] [VALIDATOR ] CSR: Primary data validation passed. [2025-10-20T18:08:14.483196] [INFO ] [MAIN ] CSR: Fetched primary data is valid. [2025-10-20T18:08:14.483237] [DEBUG] [LOCAL ] CSR: Saving primary data. [2025-10-20T18:08:14.483801] [DEBUG] [LOCAL ] CSR: Trimmed 17973 records to 13476 (keeping 200 days of data) [2025-10-20T18:08:14.563637] [DEBUG] [LOCAL ] CSR: Primary data saved (13476 records). [2025-10-20T18:08:14.563802] [INFO ] [MAIN ] CSR: Full history saved. [2025-10-20T18:08:14.563879] [INFO ] [MAIN ] Update Progress: 540/2367 symbols completed. CSR took 27.1s. Estimated remaining: 3h 50m 38s. Rate: 7.9 symbols/min [2025-10-20T18:08:14.656604] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-06. [2025-10-20T18:08:14.743444] [INFO ] [MAIN ] CTDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:14.750128] [DEBUG] [LOCAL ] CTDD: Loaded 5501 existing records from primary storage. [2025-10-20T18:08:14.750335] [INFO ] [MAIN ] CTDD: Loaded 5501 existing records. [2025-10-20T18:08:14.930988] [WARN ] [MAIN ] CTDD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:14.931205] [INFO ] [MAIN ] CTDD: Wiping existing primary data before full refresh. [2025-10-20T18:08:14.931251] [INFO ] [LOCAL ] CTDD: Starting deletion of primary data. [2025-10-20T18:08:14.931419] [DEBUG] [LOCAL ] CTDD: Deleted primary data file. [2025-10-20T18:08:14.931474] [INFO ] [MAIN ] CTDD: Starting full primary refresh. [2025-10-20T18:08:15.375272] [INFO ] [REMOTE ] CTDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:15.451419] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-01. [2025-10-20T18:08:16.212630] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:08:16.251399] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-06. [2025-10-20T18:08:16.566078] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T18:08:17.051437] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-02. [2025-10-20T18:08:17.412657] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:08:17.451967] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T18:08:17.851402] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-07. [2025-10-20T18:08:18.497898] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:08:18.651383] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-08. [2025-10-20T18:08:19.451427] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-07. [2025-10-20T18:08:19.540002] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:08:20.251367] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-09. [2025-10-20T18:08:20.946515] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:08:21.051359] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-08. [2025-10-20T18:08:21.064075] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:08:21.851416] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-03. [2025-10-20T18:08:22.289181] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:08:22.493587] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:08:22.651418] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-04. [2025-10-20T18:08:23.373115] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:08:23.451416] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-09. [2025-10-20T18:08:24.251435] [INFO ] [REMOTE ] CSV: Fetching primary data for 2025-10. [2025-10-20T18:08:25.044417] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:08:25.053788] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-03. [2025-10-20T18:08:25.237550] [INFO ] [REMOTE ] CSV: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:08:25.239565] [INFO ] [REMOTE ] CSV: Full fetch complete (19121 records). [2025-10-20T18:08:25.239690] [DEBUG] [VALIDATOR ] CSV: Validating primary data. [2025-10-20T18:08:25.596764] [DEBUG] [VALIDATOR ] CSV: All expected trading days are present. [2025-10-20T18:08:25.596844] [DEBUG] [VALIDATOR ] CSV: Primary data validation passed. [2025-10-20T18:08:25.597232] [INFO ] [MAIN ] CSV: Fetched primary data is valid. [2025-10-20T18:08:25.597294] [DEBUG] [LOCAL ] CSV: Saving primary data. [2025-10-20T18:08:25.597881] [DEBUG] [LOCAL ] CSV: Trimmed 19121 records to 12576 (keeping 200 days of data) [2025-10-20T18:08:25.672491] [DEBUG] [LOCAL ] CSV: Primary data saved (12576 records). [2025-10-20T18:08:25.672670] [INFO ] [MAIN ] CSV: Full history saved. [2025-10-20T18:08:25.672776] [INFO ] [MAIN ] Update Progress: 541/2367 symbols completed. CSV took 21.8s. Estimated remaining: 3h 50m 43s. Rate: 7.9 symbols/min [2025-10-20T18:08:25.846380] [INFO ] [MAIN ] CTEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:25.846544] [DEBUG] [LOCAL ] CTEV: No existing data file found in primary storage. [2025-10-20T18:08:25.846581] [WARN ] [MAIN ] CTEV: No existing data found, performing full refresh instead. [2025-10-20T18:08:25.846717] [INFO ] [MAIN ] CTEV: Wiping existing primary data before full refresh. [2025-10-20T18:08:25.846766] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-10-20T18:08:25.846807] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-10-20T18:08:25.846839] [INFO ] [MAIN ] CTEV: Starting full primary refresh. [2025-10-20T18:08:25.857913] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-05. [2025-10-20T18:08:26.138490] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T18:08:26.216313] [INFO ] [REMOTE ] CTEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:26.651449] [INFO ] [REMOTE ] CTEV: Fetching primary data for 2025-01. [2025-10-20T18:08:26.664502] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:08:26.865381] [DEBUG] [REMOTE ] CTEV: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:08:26.865752] [WARN ] [REMOTE ] CTEV: No data returned for month 2025-01 [2025-10-20T18:08:26.865853] [FATAL] [REMOTE ] CTEV: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:08:26.865903] [FATAL] [MAIN ] CTEV: Failed to fetch primary data. [2025-10-20T18:08:26.865936] [WARN ] [MAIN ] CTEV: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:08:26.865974] [INFO ] [LOCAL ] CTEV: Starting deletion of primary data. [2025-10-20T18:08:26.866048] [DEBUG] [LOCAL ] CTEV: No primary data file found to delete. [2025-10-20T18:08:26.866094] [INFO ] [LOCAL ] CTEV: Starting deletion of derivative data. [2025-10-20T18:08:26.866150] [DEBUG] [LOCAL ] CTEV: No derivative data file found to delete. [2025-10-20T18:08:26.866212] [INFO ] [MAIN ] Update Progress: 542/2367 symbols completed. CTEV took 1.2s. Estimated remaining: 3h 50m 14s. Rate: 7.9 symbols/min [2025-10-20T18:08:27.034708] [INFO ] [MAIN ] CTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:27.068299] [DEBUG] [LOCAL ] CTO: Loaded 28050 existing records from primary storage. [2025-10-20T18:08:27.068689] [INFO ] [MAIN ] CTO: Loaded 28050 existing records. [2025-10-20T18:08:27.284079] [WARN ] [MAIN ] CTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:27.284407] [INFO ] [MAIN ] CTO: Wiping existing primary data before full refresh. [2025-10-20T18:08:27.284469] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-10-20T18:08:27.284971] [DEBUG] [LOCAL ] CTO: Deleted primary data file. [2025-10-20T18:08:27.285029] [INFO ] [MAIN ] CTO: Starting full primary refresh. [2025-10-20T18:08:27.459543] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-04. [2025-10-20T18:08:27.754147] [INFO ] [REMOTE ] CTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:28.136260] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T18:08:28.251378] [INFO ] [REMOTE ] CSTM: Fetching primary data for 2025-10. [2025-10-20T18:08:29.051381] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-01. [2025-10-20T18:08:29.851344] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-06. [2025-10-20T18:08:30.163807] [INFO ] [REMOTE ] CSTM: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:08:30.170630] [INFO ] [REMOTE ] CSTM: Full fetch complete (69526 records). [2025-10-20T18:08:30.170787] [DEBUG] [VALIDATOR ] CSTM: Validating primary data. [2025-10-20T18:08:30.406496] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:08:30.487254] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T18:08:30.588950] [DEBUG] [VALIDATOR ] CSTM: All expected trading days are present. [2025-10-20T18:08:30.589033] [DEBUG] [VALIDATOR ] CSTM: Primary data validation passed. [2025-10-20T18:08:30.589826] [INFO ] [MAIN ] CSTM: Fetched primary data is valid. [2025-10-20T18:08:30.589887] [DEBUG] [LOCAL ] CSTM: Saving primary data. [2025-10-20T18:08:30.591337] [DEBUG] [LOCAL ] CSTM: Trimmed 69526 records to 47473 (keeping 200 days of data) [2025-10-20T18:08:30.671447] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-07. [2025-10-20T18:08:30.860034] [DEBUG] [LOCAL ] CSTM: Primary data saved (47473 records). [2025-10-20T18:08:30.860138] [INFO ] [MAIN ] CSTM: Full history saved. [2025-10-20T18:08:30.860215] [INFO ] [MAIN ] Update Progress: 543/2367 symbols completed. CSTM took 41.9s. Estimated remaining: 3h 49m 54s. Rate: 7.9 symbols/min [2025-10-20T18:08:31.023841] [INFO ] [MAIN ] CTOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:31.056991] [DEBUG] [LOCAL ] CTOS: Loaded 37827 existing records from primary storage. [2025-10-20T18:08:31.057295] [INFO ] [MAIN ] CTOS: Loaded 37827 existing records. [2025-10-20T18:08:31.177044] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:08:31.243407] [WARN ] [MAIN ] CTOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:31.243614] [INFO ] [MAIN ] CTOS: Wiping existing primary data before full refresh. [2025-10-20T18:08:31.243693] [INFO ] [LOCAL ] CTOS: Starting deletion of primary data. [2025-10-20T18:08:31.244026] [DEBUG] [LOCAL ] CTOS: Deleted primary data file. [2025-10-20T18:08:31.244065] [INFO ] [MAIN ] CTOS: Starting full primary refresh. [2025-10-20T18:08:31.456520] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-02. [2025-10-20T18:08:31.594137] [INFO ] [REMOTE ] CTOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:32.251368] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-01. [2025-10-20T18:08:32.690814] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:08:33.051376] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-05. [2025-10-20T18:08:33.294073] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:08:33.417910] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T18:08:33.851364] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-03. [2025-10-20T18:08:34.651377] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-06. [2025-10-20T18:08:35.335832] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:08:35.380070] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:08:35.451369] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-08. [2025-10-20T18:08:35.888614] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T18:08:36.251364] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-02. [2025-10-20T18:08:37.051618] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-09. [2025-10-20T18:08:37.388379] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:08:37.717485] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:08:37.851425] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-03. [2025-10-20T18:08:38.651406] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-07. [2025-10-20T18:08:38.889588] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T18:08:39.333965] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T18:08:39.451373] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-08. [2025-10-20T18:08:40.193242] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:08:40.251372] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-04. [2025-10-20T18:08:41.051392] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-09. [2025-10-20T18:08:41.538728] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T18:08:41.640703] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T18:08:41.851374] [INFO ] [REMOTE ] CTBB: Fetching primary data for 2025-10. [2025-10-20T18:08:42.560085] [INFO ] [REMOTE ] CTBB: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:08:42.561502] [INFO ] [REMOTE ] CTBB: Full fetch complete (10059 records). [2025-10-20T18:08:42.561579] [DEBUG] [VALIDATOR ] CTBB: Validating primary data. [2025-10-20T18:08:42.656567] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-04. [2025-10-20T18:08:42.929567] [DEBUG] [VALIDATOR ] CTBB: All expected trading days are present. [2025-10-20T18:08:42.929634] [DEBUG] [VALIDATOR ] CTBB: Primary data validation passed. [2025-10-20T18:08:42.929965] [INFO ] [MAIN ] CTBB: Fetched primary data is valid. [2025-10-20T18:08:42.930011] [DEBUG] [LOCAL ] CTBB: Saving primary data. [2025-10-20T18:08:42.930440] [DEBUG] [LOCAL ] CTBB: Trimmed 10059 records to 6573 (keeping 200 days of data) [2025-10-20T18:08:42.969691] [DEBUG] [LOCAL ] CTBB: Primary data saved (6573 records). [2025-10-20T18:08:42.969823] [INFO ] [MAIN ] CTBB: Full history saved. [2025-10-20T18:08:42.969895] [INFO ] [MAIN ] Update Progress: 544/2367 symbols completed. CTBB took 31.3s. Estimated remaining: 3h 50m 2s. Rate: 7.9 symbols/min [2025-10-20T18:08:43.137097] [INFO ] [MAIN ] CTRA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:43.186471] [DEBUG] [LOCAL ] CTRA: Loaded 54427 existing records from primary storage. [2025-10-20T18:08:43.186822] [INFO ] [MAIN ] CTRA: Loaded 54427 existing records. [2025-10-20T18:08:43.368080] [WARN ] [MAIN ] CTRA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:08:43.368300] [INFO ] [MAIN ] CTRA: Wiping existing primary data before full refresh. [2025-10-20T18:08:43.368359] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-10-20T18:08:43.368638] [DEBUG] [LOCAL ] CTRA: Deleted primary data file. [2025-10-20T18:08:43.368729] [INFO ] [MAIN ] CTRA: Starting full primary refresh. [2025-10-20T18:08:43.456545] [INFO ] [REMOTE ] CTDD: Fetching primary data for 2025-10. [2025-10-20T18:08:43.753424] [INFO ] [REMOTE ] CTRA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:44.251423] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-05. [2025-10-20T18:08:44.666013] [INFO ] [REMOTE ] CTDD: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T18:08:44.667439] [INFO ] [REMOTE ] CTDD: Full fetch complete (7921 records). [2025-10-20T18:08:44.667518] [DEBUG] [VALIDATOR ] CTDD: Validating primary data. [2025-10-20T18:08:45.087246] [DEBUG] [VALIDATOR ] CTDD: All expected trading days are present. [2025-10-20T18:08:45.071061] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-01. [2025-10-20T18:08:45.087678] [DEBUG] [VALIDATOR ] CTDD: Primary data validation passed. [2025-10-20T18:08:45.091383] [INFO ] [MAIN ] CTDD: Fetched primary data is valid. [2025-10-20T18:08:45.092841] [DEBUG] [LOCAL ] CTDD: Saving primary data. [2025-10-20T18:08:45.094107] [DEBUG] [LOCAL ] CTDD: Trimmed 7921 records to 5284 (keeping 200 days of data) [2025-10-20T18:08:45.113402] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-04 (2.46 seconds). [2025-10-20T18:08:45.145304] [DEBUG] [LOCAL ] CTDD: Primary data saved (5284 records). [2025-10-20T18:08:45.145895] [INFO ] [MAIN ] CTDD: Full history saved. [2025-10-20T18:08:45.146006] [INFO ] [MAIN ] Update Progress: 545/2367 symbols completed. CTDD took 30.6s. Estimated remaining: 3h 49m 36s. Rate: 7.9 symbols/min [2025-10-20T18:08:45.326930] [INFO ] [MAIN ] CTRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:08:45.327370] [DEBUG] [LOCAL ] CTRE: No existing data file found in primary storage. [2025-10-20T18:08:45.327495] [WARN ] [MAIN ] CTRE: No existing data found, performing full refresh instead. [2025-10-20T18:08:45.327770] [INFO ] [MAIN ] CTRE: Wiping existing primary data before full refresh. [2025-10-20T18:08:45.342809] [INFO ] [LOCAL ] CTRE: Starting deletion of primary data. [2025-10-20T18:08:45.343708] [DEBUG] [LOCAL ] CTRE: No primary data file found to delete. [2025-10-20T18:08:45.344226] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:08:45.344389] [INFO ] [MAIN ] CTRE: Starting full primary refresh. [2025-10-20T18:08:45.704129] [INFO ] [REMOTE ] CTRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:08:45.851402] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-05. [2025-10-20T18:08:46.651359] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-01. [2025-10-20T18:08:46.957192] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:08:47.082122] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:08:47.451378] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-06. [2025-10-20T18:08:47.937149] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:08:48.251376] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-02. [2025-10-20T18:08:48.345131] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:08:49.051386] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-02. [2025-10-20T18:08:49.834133] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:08:49.851330] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-03. [2025-10-20T18:08:50.626070] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:08:50.651331] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-03. [2025-10-20T18:08:51.217439] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:08:51.451397] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-06. [2025-10-20T18:08:52.017805] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T18:08:52.251414] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-04. [2025-10-20T18:08:52.795579] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:08:53.051444] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-07. [2025-10-20T18:08:53.654992] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:08:53.851381] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-07. [2025-10-20T18:08:54.376293] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:08:54.651427] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-08. [2025-10-20T18:08:55.451430] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-05. [2025-10-20T18:08:55.511629] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:08:55.572449] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T18:08:56.251426] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-08. [2025-10-20T18:08:57.051425] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-04. [2025-10-20T18:08:57.498564] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T18:08:57.817208] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:08:57.851374] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-09. [2025-10-20T18:08:58.651350] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-09. [2025-10-20T18:08:59.209519] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:08:59.376807] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T18:08:59.451351] [INFO ] [REMOTE ] CTOS: Fetching primary data for 2025-10. [2025-10-20T18:08:59.968059] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:09:00.251378] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-05. [2025-10-20T18:09:00.968455] [INFO ] [REMOTE ] CTOS: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:09:00.974833] [INFO ] [REMOTE ] CTOS: Full fetch complete (54095 records). [2025-10-20T18:09:00.974955] [DEBUG] [VALIDATOR ] CTOS: Validating primary data. [2025-10-20T18:09:01.056468] [INFO ] [REMOTE ] CTO: Fetching primary data for 2025-10. [2025-10-20T18:09:01.377930] [DEBUG] [VALIDATOR ] CTOS: All expected trading days are present. [2025-10-20T18:09:01.378021] [DEBUG] [VALIDATOR ] CTOS: Primary data validation passed. [2025-10-20T18:09:01.378761] [INFO ] [MAIN ] CTOS: Fetched primary data is valid. [2025-10-20T18:09:01.378829] [DEBUG] [LOCAL ] CTOS: Saving primary data. [2025-10-20T18:09:01.380147] [DEBUG] [LOCAL ] CTOS: Trimmed 54095 records to 37326 (keeping 200 days of data) [2025-10-20T18:09:01.594985] [DEBUG] [LOCAL ] CTOS: Primary data saved (37326 records). [2025-10-20T18:09:01.595136] [INFO ] [MAIN ] CTOS: Full history saved. [2025-10-20T18:09:01.595214] [INFO ] [MAIN ] Update Progress: 546/2367 symbols completed. CTOS took 30.7s. Estimated remaining: 3h 49m 58s. Rate: 7.9 symbols/min [2025-10-20T18:09:01.770606] [INFO ] [MAIN ] CTRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:01.809400] [DEBUG] [LOCAL ] CTRI: Loaded 40805 existing records from primary storage. [2025-10-20T18:09:01.809938] [INFO ] [MAIN ] CTRI: Loaded 40805 existing records. [2025-10-20T18:09:01.865186] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-06. [2025-10-20T18:09:02.015849] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:09:02.070671] [WARN ] [MAIN ] CTRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:02.071195] [INFO ] [MAIN ] CTRI: Wiping existing primary data before full refresh. [2025-10-20T18:09:02.071251] [INFO ] [LOCAL ] CTRI: Starting deletion of primary data. [2025-10-20T18:09:02.072044] [DEBUG] [LOCAL ] CTRI: Deleted primary data file. [2025-10-20T18:09:02.072097] [INFO ] [MAIN ] CTRI: Starting full primary refresh. [2025-10-20T18:09:02.337795] [INFO ] [REMOTE ] CTO: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:09:02.387776] [INFO ] [REMOTE ] CTO: Full fetch complete (41242 records). [2025-10-20T18:09:02.397086] [DEBUG] [VALIDATOR ] CTO: Validating primary data. [2025-10-20T18:09:02.403419] [FATAL] [VALIDATOR ] CTO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:02.440517] [FATAL] [MAIN ] CTO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:02.465846] [WARN ] [MAIN ] CTO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:02.488776] [INFO ] [LOCAL ] CTO: Starting deletion of primary data. [2025-10-20T18:09:02.488668] [INFO ] [REMOTE ] CTRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:02.489052] [DEBUG] [LOCAL ] CTO: No primary data file found to delete. [2025-10-20T18:09:02.489411] [INFO ] [LOCAL ] CTO: Starting deletion of derivative data. [2025-10-20T18:09:02.489633] [DEBUG] [LOCAL ] CTO: Deleted derivative data file. [2025-10-20T18:09:02.489805] [INFO ] [MAIN ] Update Progress: 547/2367 symbols completed. CTO took 35.6s. Estimated remaining: 3h 49m 28s. Rate: 7.9 symbols/min [2025-10-20T18:09:02.659085] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-06. [2025-10-20T18:09:02.662775] [INFO ] [MAIN ] CTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:02.684291] [DEBUG] [LOCAL ] CTS: Loaded 20047 existing records from primary storage. [2025-10-20T18:09:02.684564] [INFO ] [MAIN ] CTS: Loaded 20047 existing records. [2025-10-20T18:09:02.859893] [WARN ] [MAIN ] CTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:02.860152] [INFO ] [MAIN ] CTS: Wiping existing primary data before full refresh. [2025-10-20T18:09:02.860199] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-10-20T18:09:02.860378] [DEBUG] [LOCAL ] CTS: Deleted primary data file. [2025-10-20T18:09:02.860434] [INFO ] [MAIN ] CTS: Starting full primary refresh. [2025-10-20T18:09:03.221507] [INFO ] [REMOTE ] CTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:03.451401] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-01. [2025-10-20T18:09:03.919071] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-06 (2.05 seconds). [2025-10-20T18:09:04.251340] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-01. [2025-10-20T18:09:04.432180] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:09:04.475356] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:09:05.051455] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-07. [2025-10-20T18:09:05.590987] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T18:09:05.851404] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-02. [2025-10-20T18:09:06.657164] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-07. [2025-10-20T18:09:06.686456] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:09:06.812538] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:09:07.451390] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-08. [2025-10-20T18:09:08.251366] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-03. [2025-10-20T18:09:08.471764] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:09:08.988116] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T18:09:09.051344] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-02. [2025-10-20T18:09:09.461724] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:09:09.851397] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-04. [2025-10-20T18:09:10.276361] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:09:10.651420] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-09. [2025-10-20T18:09:10.729029] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T18:09:11.451424] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-05. [2025-10-20T18:09:12.155628] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T18:09:12.251362] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-06. [2025-10-20T18:09:12.322111] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:09:12.911536] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:09:13.051380] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-07. [2025-10-20T18:09:13.851478] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-03. [2025-10-20T18:09:14.236260] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:09:14.651366] [INFO ] [REMOTE ] CTRA: Fetching primary data for 2025-10. [2025-10-20T18:09:14.684841] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:09:15.451411] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-08. [2025-10-20T18:09:16.251373] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-04. [2025-10-20T18:09:16.644318] [INFO ] [REMOTE ] CTRA: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T18:09:16.652394] [INFO ] [REMOTE ] CTRA: Full fetch complete (77430 records). [2025-10-20T18:09:16.652625] [DEBUG] [VALIDATOR ] CTRA: Validating primary data. [2025-10-20T18:09:16.653544] [FATAL] [VALIDATOR ] CTRA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:16.653671] [FATAL] [MAIN ] CTRA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:16.653735] [WARN ] [MAIN ] CTRA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:16.653785] [INFO ] [LOCAL ] CTRA: Starting deletion of primary data. [2025-10-20T18:09:16.653856] [DEBUG] [LOCAL ] CTRA: No primary data file found to delete. [2025-10-20T18:09:16.653895] [INFO ] [LOCAL ] CTRA: Starting deletion of derivative data. [2025-10-20T18:09:16.654102] [DEBUG] [LOCAL ] CTRA: Deleted derivative data file. [2025-10-20T18:09:16.654212] [INFO ] [MAIN ] Update Progress: 548/2367 symbols completed. CTRA took 33.7s. Estimated remaining: 3h 49m 43s. Rate: 7.9 symbols/min [2025-10-20T18:09:16.850476] [INFO ] [MAIN ] CTVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:16.909214] [DEBUG] [LOCAL ] CTVA: Loaded 54075 existing records from primary storage. [2025-10-20T18:09:16.909697] [INFO ] [MAIN ] CTVA: Loaded 54075 existing records. [2025-10-20T18:09:17.064456] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-08. [2025-10-20T18:09:17.112280] [WARN ] [MAIN ] CTVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:17.113830] [INFO ] [MAIN ] CTVA: Wiping existing primary data before full refresh. [2025-10-20T18:09:17.114002] [INFO ] [LOCAL ] CTVA: Starting deletion of primary data. [2025-10-20T18:09:17.114493] [DEBUG] [LOCAL ] CTVA: Deleted primary data file. [2025-10-20T18:09:17.114566] [INFO ] [MAIN ] CTVA: Starting full primary refresh. [2025-10-20T18:09:17.550161] [INFO ] [REMOTE ] CTVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:17.574174] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-08 (2.12 seconds). [2025-10-20T18:09:17.585890] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:09:17.851466] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-05. [2025-10-20T18:09:17.902789] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:09:18.651360] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-09. [2025-10-20T18:09:19.335196] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:09:19.451385] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-09. [2025-10-20T18:09:19.720147] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:09:20.251383] [INFO ] [REMOTE ] CTS: Fetching primary data for 2025-10. [2025-10-20T18:09:21.051391] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-06. [2025-10-20T18:09:21.201900] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:09:21.236325] [INFO ] [REMOTE ] CTS: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:09:21.239957] [INFO ] [REMOTE ] CTS: Full fetch complete (27126 records). [2025-10-20T18:09:21.240072] [DEBUG] [VALIDATOR ] CTS: Validating primary data. [2025-10-20T18:09:21.240544] [FATAL] [VALIDATOR ] CTS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:09:21.240629] [FATAL] [MAIN ] CTS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:21.240711] [WARN ] [MAIN ] CTS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:21.240784] [INFO ] [LOCAL ] CTS: Starting deletion of primary data. [2025-10-20T18:09:21.240873] [DEBUG] [LOCAL ] CTS: No primary data file found to delete. [2025-10-20T18:09:21.240911] [INFO ] [LOCAL ] CTS: Starting deletion of derivative data. [2025-10-20T18:09:21.241041] [DEBUG] [LOCAL ] CTS: Deleted derivative data file. [2025-10-20T18:09:21.241111] [INFO ] [MAIN ] Update Progress: 549/2367 symbols completed. CTS took 18.8s. Estimated remaining: 3h 49m 25s. Rate: 7.9 symbols/min [2025-10-20T18:09:21.424729] [INFO ] [MAIN ] CUBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:21.424943] [DEBUG] [LOCAL ] CUBB: No existing data file found in primary storage. [2025-10-20T18:09:21.425000] [WARN ] [MAIN ] CUBB: No existing data found, performing full refresh instead. [2025-10-20T18:09:21.425117] [INFO ] [MAIN ] CUBB: Wiping existing primary data before full refresh. [2025-10-20T18:09:21.425159] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-10-20T18:09:21.425200] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-10-20T18:09:21.425232] [INFO ] [MAIN ] CUBB: Starting full primary refresh. [2025-10-20T18:09:21.830309] [INFO ] [REMOTE ] CUBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:21.851441] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-01. [2025-10-20T18:09:22.094604] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:09:22.287394] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-10-20T18:09:22.651409] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-07. [2025-10-20T18:09:23.451415] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-02. [2025-10-20T18:09:23.741799] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T18:09:24.084825] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:09:24.251407] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-03. [2025-10-20T18:09:24.604550] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:09:25.051410] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-01. [2025-10-20T18:09:25.851446] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-04. [2025-10-20T18:09:26.130074] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:09:26.651391] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-08. [2025-10-20T18:09:26.919538] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:09:27.451370] [INFO ] [REMOTE ] CTRE: Fetching primary data for 2025-10. [2025-10-20T18:09:27.847896] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:09:28.251405] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-02. [2025-10-20T18:09:28.785914] [INFO ] [REMOTE ] CTRE: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:09:28.793302] [INFO ] [REMOTE ] CTRE: Full fetch complete (71386 records). [2025-10-20T18:09:28.793460] [DEBUG] [VALIDATOR ] CTRE: Validating primary data. [2025-10-20T18:09:29.059136] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-05. [2025-10-20T18:09:29.178083] [DEBUG] [VALIDATOR ] CTRE: All expected trading days are present. [2025-10-20T18:09:29.178252] [DEBUG] [VALIDATOR ] CTRE: Primary data validation passed. [2025-10-20T18:09:29.179123] [INFO ] [MAIN ] CTRE: Fetched primary data is valid. [2025-10-20T18:09:29.179187] [DEBUG] [LOCAL ] CTRE: Saving primary data. [2025-10-20T18:09:29.180597] [DEBUG] [LOCAL ] CTRE: Trimmed 71386 records to 49507 (keeping 200 days of data) [2025-10-20T18:09:29.391464] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-05 (0.33 seconds). [2025-10-20T18:09:29.485439] [DEBUG] [LOCAL ] CTRE: Primary data saved (49507 records). [2025-10-20T18:09:29.485603] [INFO ] [MAIN ] CTRE: Full history saved. [2025-10-20T18:09:29.485701] [INFO ] [MAIN ] Update Progress: 550/2367 symbols completed. CTRE took 44.3s. Estimated remaining: 3h 49m 20s. Rate: 7.9 symbols/min [2025-10-20T18:09:29.660032] [INFO ] [MAIN ] CUBE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:29.714017] [DEBUG] [LOCAL ] CUBE: Loaded 51666 existing records from primary storage. [2025-10-20T18:09:29.714362] [INFO ] [MAIN ] CUBE: Loaded 51666 existing records. [2025-10-20T18:09:29.856273] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-06. [2025-10-20T18:09:29.921592] [WARN ] [MAIN ] CUBE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:29.922178] [INFO ] [MAIN ] CUBE: Wiping existing primary data before full refresh. [2025-10-20T18:09:29.922320] [INFO ] [LOCAL ] CUBE: Starting deletion of primary data. [2025-10-20T18:09:29.922579] [DEBUG] [LOCAL ] CUBE: Deleted primary data file. [2025-10-20T18:09:29.922636] [INFO ] [MAIN ] CUBE: Starting full primary refresh. [2025-10-20T18:09:30.113013] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:09:30.213003] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-06 (0.36 seconds). [2025-10-20T18:09:30.329718] [INFO ] [REMOTE ] CUBE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:30.651393] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-01. [2025-10-20T18:09:31.451429] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-09. [2025-10-20T18:09:32.251408] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-03. [2025-10-20T18:09:32.363841] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:09:33.051401] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-07. [2025-10-20T18:09:33.110074] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:09:33.480312] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:09:33.566241] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:09:33.851380] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-04. [2025-10-20T18:09:34.651389] [INFO ] [REMOTE ] CTRI: Fetching primary data for 2025-10. [2025-10-20T18:09:35.451442] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-08. [2025-10-20T18:09:35.684664] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:09:35.792746] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:09:36.235753] [INFO ] [REMOTE ] CTRI: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T18:09:36.243069] [INFO ] [REMOTE ] CTRI: Full fetch complete (53957 records). [2025-10-20T18:09:36.243205] [DEBUG] [VALIDATOR ] CTRI: Validating primary data. [2025-10-20T18:09:36.256565] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-02. [2025-10-20T18:09:36.701298] [DEBUG] [VALIDATOR ] CTRI: All expected trading days are present. [2025-10-20T18:09:36.701433] [DEBUG] [VALIDATOR ] CTRI: Primary data validation passed. [2025-10-20T18:09:36.702890] [INFO ] [MAIN ] CTRI: Fetched primary data is valid. [2025-10-20T18:09:36.702965] [DEBUG] [LOCAL ] CTRI: Saving primary data. [2025-10-20T18:09:36.704583] [DEBUG] [LOCAL ] CTRI: Trimmed 53957 records to 40518 (keeping 200 days of data) [2025-10-20T18:09:36.961530] [DEBUG] [LOCAL ] CTRI: Primary data saved (40518 records). [2025-10-20T18:09:36.961777] [INFO ] [MAIN ] CTRI: Full history saved. [2025-10-20T18:09:36.961884] [INFO ] [MAIN ] Update Progress: 551/2367 symbols completed. CTRI took 35.4s. Estimated remaining: 3h 49m 12s. Rate: 7.9 symbols/min [2025-10-20T18:09:37.056565] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-09. [2025-10-20T18:09:37.166585] [INFO ] [MAIN ] CUBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:37.166881] [DEBUG] [LOCAL ] CUBI: No existing data file found in primary storage. [2025-10-20T18:09:37.166929] [WARN ] [MAIN ] CUBI: No existing data found, performing full refresh instead. [2025-10-20T18:09:37.167060] [INFO ] [MAIN ] CUBI: Wiping existing primary data before full refresh. [2025-10-20T18:09:37.167104] [INFO ] [LOCAL ] CUBI: Starting deletion of primary data. [2025-10-20T18:09:37.167146] [DEBUG] [LOCAL ] CUBI: No primary data file found to delete. [2025-10-20T18:09:37.167181] [INFO ] [MAIN ] CUBI: Starting full primary refresh. [2025-10-20T18:09:37.464452] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T18:09:37.582814] [INFO ] [REMOTE ] CUBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:37.851361] [INFO ] [REMOTE ] CUBB: Fetching primary data for 2025-10. [2025-10-20T18:09:37.957858] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T18:09:38.289480] [INFO ] [REMOTE ] CUBB: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T18:09:38.290225] [INFO ] [REMOTE ] CUBB: Full fetch complete (954 records). [2025-10-20T18:09:38.290291] [DEBUG] [VALIDATOR ] CUBB: Validating primary data. [2025-10-20T18:09:38.464626] [WARN ] [VALIDATOR ] CUBB: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T18:09:38.464815] [FATAL] [MAIN ] CUBB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:09:38.464882] [WARN ] [MAIN ] CUBB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:09:38.464952] [INFO ] [LOCAL ] CUBB: Starting deletion of primary data. [2025-10-20T18:09:38.465015] [DEBUG] [LOCAL ] CUBB: No primary data file found to delete. [2025-10-20T18:09:38.465048] [INFO ] [LOCAL ] CUBB: Starting deletion of derivative data. [2025-10-20T18:09:38.465086] [DEBUG] [LOCAL ] CUBB: No derivative data file found to delete. [2025-10-20T18:09:38.465137] [INFO ] [MAIN ] Update Progress: 552/2367 symbols completed. CUBB took 17.2s. Estimated remaining: 3h 48m 44s. Rate: 7.9 symbols/min [2025-10-20T18:09:38.637834] [INFO ] [MAIN ] CUK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:09:38.663711] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-03. [2025-10-20T18:09:38.687179] [DEBUG] [LOCAL ] CUK: Loaded 51814 existing records from primary storage. [2025-10-20T18:09:38.687525] [INFO ] [MAIN ] CUK: Loaded 51814 existing records. [2025-10-20T18:09:38.865892] [WARN ] [MAIN ] CUK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:09:38.866123] [INFO ] [MAIN ] CUK: Wiping existing primary data before full refresh. [2025-10-20T18:09:38.866170] [INFO ] [LOCAL ] CUK: Starting deletion of primary data. [2025-10-20T18:09:38.866575] [DEBUG] [LOCAL ] CUK: Deleted primary data file. [2025-10-20T18:09:38.866634] [INFO ] [MAIN ] CUK: Starting full primary refresh. [2025-10-20T18:09:39.223864] [INFO ] [REMOTE ] CUK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:09:39.451392] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-01. [2025-10-20T18:09:40.163589] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:09:40.251407] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-05. [2025-10-20T18:09:41.051441] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-01. [2025-10-20T18:09:41.347320] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:09:41.851396] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-02. [2025-10-20T18:09:42.086514] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T18:09:42.447969] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:09:42.651366] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-06. [2025-10-20T18:09:43.344168] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:09:43.451350] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-03. [2025-10-20T18:09:44.251404] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-02. [2025-10-20T18:09:44.385175] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:09:45.051354] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-07. [2025-10-20T18:09:45.294146] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:09:45.330571] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:09:45.851422] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-04. [2025-10-20T18:09:46.651375] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-03. [2025-10-20T18:09:46.876441] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T18:09:47.451387] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-04. [2025-10-20T18:09:47.813065] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:09:47.903348] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:09:48.251439] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-05. [2025-10-20T18:09:48.896067] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:09:49.051461] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-05. [2025-10-20T18:09:49.688353] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:09:49.851403] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-08. [2025-10-20T18:09:50.506056] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:09:50.651345] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-06. [2025-10-20T18:09:51.451585] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-04. [2025-10-20T18:09:51.747820] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:09:52.034040] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:09:52.251383] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-09. [2025-10-20T18:09:52.826036] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:09:53.051449] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-07. [2025-10-20T18:09:53.851422] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-06. [2025-10-20T18:09:54.021797] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:09:54.464678] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:09:54.651445] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-08. [2025-10-20T18:09:55.451392] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-05. [2025-10-20T18:09:55.616856] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:09:56.251345] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-07. [2025-10-20T18:09:56.342022] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:09:56.497896] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T18:09:57.051430] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-09. [2025-10-20T18:09:57.690022] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:09:57.851392] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-08. [2025-10-20T18:09:58.376753] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:09:58.651373] [INFO ] [REMOTE ] CUBE: Fetching primary data for 2025-10. [2025-10-20T18:09:59.451443] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-06. [2025-10-20T18:09:59.587867] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:10:00.251388] [INFO ] [REMOTE ] CTVA: Fetching primary data for 2025-10. [2025-10-20T18:10:00.557545] [INFO ] [REMOTE ] CUBE: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:10:00.577420] [INFO ] [REMOTE ] CUBE: Full fetch complete (73307 records). [2025-10-20T18:10:00.579375] [DEBUG] [VALIDATOR ] CUBE: Validating primary data. [2025-10-20T18:10:00.580912] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:10:00.984594] [DEBUG] [VALIDATOR ] CUBE: All expected trading days are present. [2025-10-20T18:10:00.984743] [DEBUG] [VALIDATOR ] CUBE: Primary data validation passed. [2025-10-20T18:10:00.985688] [INFO ] [MAIN ] CUBE: Fetched primary data is valid. [2025-10-20T18:10:00.985755] [DEBUG] [LOCAL ] CUBE: Saving primary data. [2025-10-20T18:10:00.987194] [DEBUG] [LOCAL ] CUBE: Trimmed 73307 records to 50888 (keeping 200 days of data) [2025-10-20T18:10:01.054967] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-07. [2025-10-20T18:10:01.304128] [DEBUG] [LOCAL ] CUBE: Primary data saved (50888 records). [2025-10-20T18:10:01.304277] [INFO ] [MAIN ] CUBE: Full history saved. [2025-10-20T18:10:01.304355] [INFO ] [MAIN ] Update Progress: 553/2367 symbols completed. CUBE took 31.8s. Estimated remaining: 3h 49m 27s. Rate: 7.9 symbols/min [2025-10-20T18:10:01.472313] [INFO ] [MAIN ] CULP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:01.476807] [DEBUG] [LOCAL ] CULP: Loaded 2947 existing records from primary storage. [2025-10-20T18:10:01.477014] [INFO ] [MAIN ] CULP: Loaded 2947 existing records. [2025-10-20T18:10:01.674115] [WARN ] [MAIN ] CULP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:01.674363] [INFO ] [MAIN ] CULP: Wiping existing primary data before full refresh. [2025-10-20T18:10:01.674410] [INFO ] [LOCAL ] CULP: Starting deletion of primary data. [2025-10-20T18:10:01.674577] [DEBUG] [LOCAL ] CULP: Deleted primary data file. [2025-10-20T18:10:01.674633] [INFO ] [MAIN ] CULP: Starting full primary refresh. [2025-10-20T18:10:01.856560] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-09. [2025-10-20T18:10:02.218622] [INFO ] [REMOTE ] CULP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:02.265755] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T18:10:02.312231] [INFO ] [REMOTE ] CTVA: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T18:10:02.321923] [INFO ] [REMOTE ] CTVA: Full fetch complete (77220 records). [2025-10-20T18:10:02.322144] [DEBUG] [VALIDATOR ] CTVA: Validating primary data. [2025-10-20T18:10:02.659268] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-01. [2025-10-20T18:10:02.950766] [DEBUG] [VALIDATOR ] CTVA: All expected trading days are present. [2025-10-20T18:10:02.950891] [DEBUG] [VALIDATOR ] CTVA: Primary data validation passed. [2025-10-20T18:10:02.952253] [INFO ] [MAIN ] CTVA: Fetched primary data is valid. [2025-10-20T18:10:02.952341] [DEBUG] [LOCAL ] CTVA: Saving primary data. [2025-10-20T18:10:02.954480] [DEBUG] [LOCAL ] CTVA: Trimmed 77220 records to 53302 (keeping 200 days of data) [2025-10-20T18:10:03.288355] [DEBUG] [LOCAL ] CTVA: Primary data saved (53302 records). [2025-10-20T18:10:03.288489] [INFO ] [MAIN ] CTVA: Full history saved. [2025-10-20T18:10:03.289006] [INFO ] [MAIN ] Update Progress: 554/2367 symbols completed. CTVA took 46.6s. Estimated remaining: 3h 49m 1s. Rate: 7.9 symbols/min [2025-10-20T18:10:03.457563] [INFO ] [MAIN ] CURB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:03.457628] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-08. [2025-10-20T18:10:03.490173] [DEBUG] [LOCAL ] CURB: Loaded 33392 existing records from primary storage. [2025-10-20T18:10:03.490445] [INFO ] [MAIN ] CURB: Loaded 33392 existing records. [2025-10-20T18:10:03.601782] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:10:03.688130] [WARN ] [MAIN ] CURB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:03.688508] [INFO ] [MAIN ] CURB: Wiping existing primary data before full refresh. [2025-10-20T18:10:03.688580] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-10-20T18:10:03.689077] [DEBUG] [LOCAL ] CURB: Deleted primary data file. [2025-10-20T18:10:03.689145] [INFO ] [MAIN ] CURB: Starting full primary refresh. [2025-10-20T18:10:04.057283] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T18:10:04.076515] [INFO ] [REMOTE ] CURB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:04.251349] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-01. [2025-10-20T18:10:04.484971] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:10:05.051398] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-09. [2025-10-20T18:10:05.851383] [INFO ] [REMOTE ] CUK: Fetching primary data for 2025-10. [2025-10-20T18:10:05.984183] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:10:06.234400] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:10:06.651481] [INFO ] [REMOTE ] CUBI: Fetching primary data for 2025-10. [2025-10-20T18:10:07.451450] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-02. [2025-10-20T18:10:08.032195] [INFO ] [REMOTE ] CUK: Incremental fetch complete for 2025-10 (2.18 seconds). [2025-10-20T18:10:08.042229] [INFO ] [REMOTE ] CUK: Full fetch complete (74358 records). [2025-10-20T18:10:08.042386] [DEBUG] [VALIDATOR ] CUK: Validating primary data. [2025-10-20T18:10:08.155133] [INFO ] [REMOTE ] CUBI: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T18:10:08.180097] [INFO ] [REMOTE ] CUBI: Full fetch complete (39177 records). [2025-10-20T18:10:08.198465] [DEBUG] [VALIDATOR ] CUBI: Validating primary data. [2025-10-20T18:10:08.272576] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-02. [2025-10-20T18:10:08.750145] [DEBUG] [VALIDATOR ] CUK: All expected trading days are present. [2025-10-20T18:10:08.765217] [DEBUG] [VALIDATOR ] CUK: Primary data validation passed. [2025-10-20T18:10:08.787724] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T18:10:08.803550] [INFO ] [MAIN ] CUK: Fetched primary data is valid. [2025-10-20T18:10:08.837074] [DEBUG] [LOCAL ] CUK: Saving primary data. [2025-10-20T18:10:08.969308] [DEBUG] [LOCAL ] CUK: Trimmed 74358 records to 51072 (keeping 200 days of data) [2025-10-20T18:10:08.971683] [DEBUG] [VALIDATOR ] CUBI: All expected trading days are present. [2025-10-20T18:10:08.971083] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:10:08.977495] [DEBUG] [VALIDATOR ] CUBI: Primary data validation passed. [2025-10-20T18:10:08.979048] [INFO ] [MAIN ] CUBI: Fetched primary data is valid. [2025-10-20T18:10:09.026352] [DEBUG] [LOCAL ] CUBI: Saving primary data. [2025-10-20T18:10:09.061135] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-03. [2025-10-20T18:10:09.069049] [DEBUG] [LOCAL ] CUBI: Trimmed 39177 records to 27276 (keeping 200 days of data) [2025-10-20T18:10:09.313415] [DEBUG] [LOCAL ] CUBI: Primary data saved (27276 records). [2025-10-20T18:10:09.323210] [INFO ] [MAIN ] CUBI: Full history saved. [2025-10-20T18:10:09.351184] [INFO ] [MAIN ] Update Progress: 555/2367 symbols completed. CUBI took 32.4s. Estimated remaining: 3h 48m 49s. Rate: 7.9 symbols/min [2025-10-20T18:10:09.545157] [INFO ] [MAIN ] CURV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:09.589808] [DEBUG] [LOCAL ] CURV: Loaded 27541 existing records from primary storage. [2025-10-20T18:10:09.595583] [INFO ] [MAIN ] CURV: Loaded 27541 existing records. [2025-10-20T18:10:09.838760] [WARN ] [MAIN ] CURV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:09.840258] [INFO ] [MAIN ] CURV: Wiping existing primary data before full refresh. [2025-10-20T18:10:09.843237] [INFO ] [LOCAL ] CURV: Starting deletion of primary data. [2025-10-20T18:10:09.845838] [DEBUG] [LOCAL ] CURV: Deleted primary data file. [2025-10-20T18:10:09.846251] [INFO ] [MAIN ] CURV: Starting full primary refresh. [2025-10-20T18:10:09.851728] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-03. [2025-10-20T18:10:09.979012] [DEBUG] [LOCAL ] CUK: Primary data saved (51072 records). [2025-10-20T18:10:10.014070] [INFO ] [MAIN ] CUK: Full history saved. [2025-10-20T18:10:10.025047] [INFO ] [MAIN ] Update Progress: 556/2367 symbols completed. CUK took 31.6s. Estimated remaining: 3h 48m 19s. Rate: 7.9 symbols/min [2025-10-20T18:10:10.382476] [INFO ] [MAIN ] CUZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:10.425407] [INFO ] [REMOTE ] CURV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:10.430803] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T18:10:10.464892] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:10:10.510688] [DEBUG] [LOCAL ] CUZ: Loaded 50363 existing records from primary storage. [2025-10-20T18:10:10.510999] [INFO ] [MAIN ] CUZ: Loaded 50363 existing records. [2025-10-20T18:10:10.658902] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-04. [2025-10-20T18:10:10.690990] [WARN ] [MAIN ] CUZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:10.691338] [INFO ] [MAIN ] CUZ: Wiping existing primary data before full refresh. [2025-10-20T18:10:10.691385] [INFO ] [LOCAL ] CUZ: Starting deletion of primary data. [2025-10-20T18:10:10.691683] [DEBUG] [LOCAL ] CUZ: Deleted primary data file. [2025-10-20T18:10:10.691739] [INFO ] [MAIN ] CUZ: Starting full primary refresh. [2025-10-20T18:10:11.042760] [INFO ] [REMOTE ] CUZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:11.374141] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:10:11.451451] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-04. [2025-10-20T18:10:12.251406] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-05. [2025-10-20T18:10:12.720084] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:10:12.922966] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:10:13.051401] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-05. [2025-10-20T18:10:13.851423] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-06. [2025-10-20T18:10:14.235103] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:10:14.393281] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:10:14.651479] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-01. [2025-10-20T18:10:15.451442] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-01. [2025-10-20T18:10:16.210437] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:10:16.251398] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-02. [2025-10-20T18:10:16.688270] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:10:17.051363] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-06. [2025-10-20T18:10:17.557694] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:10:17.851390] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-03. [2025-10-20T18:10:18.516808] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:10:18.651357] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-07. [2025-10-20T18:10:19.377206] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:10:19.451351] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-07. [2025-10-20T18:10:19.935948] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:10:20.015994] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:10:20.251375] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-02. [2025-10-20T18:10:21.051403] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-04. [2025-10-20T18:10:21.227525] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:10:21.851403] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-08. [2025-10-20T18:10:22.227515] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T18:10:22.651410] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-09. [2025-10-20T18:10:22.743151] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:10:23.302612] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:10:23.451365] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-03. [2025-10-20T18:10:24.251369] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-05. [2025-10-20T18:10:24.650374] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T18:10:25.051363] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-04. [2025-10-20T18:10:25.851370] [INFO ] [REMOTE ] CULP: Fetching primary data for 2025-10. [2025-10-20T18:10:25.962805] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:10:26.076116] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:10:26.468165] [INFO ] [REMOTE ] CULP: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:10:26.469335] [INFO ] [REMOTE ] CULP: Full fetch complete (4535 records). [2025-10-20T18:10:26.469407] [DEBUG] [VALIDATOR ] CULP: Validating primary data. [2025-10-20T18:10:26.658015] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-06. [2025-10-20T18:10:26.837065] [DEBUG] [VALIDATOR ] CULP: All expected trading days are present. [2025-10-20T18:10:26.837149] [DEBUG] [VALIDATOR ] CULP: Primary data validation passed. [2025-10-20T18:10:26.837310] [INFO ] [MAIN ] CULP: Fetched primary data is valid. [2025-10-20T18:10:26.837351] [DEBUG] [LOCAL ] CULP: Saving primary data. [2025-10-20T18:10:26.837831] [DEBUG] [LOCAL ] CULP: Trimmed 4535 records to 2882 (keeping 200 days of data) [2025-10-20T18:10:26.856743] [DEBUG] [LOCAL ] CULP: Primary data saved (2882 records). [2025-10-20T18:10:26.856920] [INFO ] [MAIN ] CULP: Full history saved. [2025-10-20T18:10:26.857055] [INFO ] [MAIN ] Update Progress: 557/2367 symbols completed. CULP took 25.6s. Estimated remaining: 3h 48m 41s. Rate: 7.9 symbols/min [2025-10-20T18:10:27.451434] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-08. [2025-10-20T18:10:28.251369] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-05. [2025-10-20T18:10:28.329826] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:10:28.895879] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:10:29.051357] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-09. [2025-10-20T18:10:29.316285] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:10:29.851449] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-06. [2025-10-20T18:10:30.329797] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:10:30.651430] [INFO ] [REMOTE ] CURB: Fetching primary data for 2025-10. [2025-10-20T18:10:30.851020] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:10:31.451483] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-07. [2025-10-20T18:10:31.549301] [INFO ] [REMOTE ] CURB: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T18:10:31.555202] [INFO ] [REMOTE ] CURB: Full fetch complete (47218 records). [2025-10-20T18:10:31.555453] [DEBUG] [VALIDATOR ] CURB: Validating primary data. [2025-10-20T18:10:31.556381] [FATAL] [VALIDATOR ] CURB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:10:31.556490] [FATAL] [MAIN ] CURB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:10:31.556566] [WARN ] [MAIN ] CURB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:10:31.556699] [INFO ] [LOCAL ] CURB: Starting deletion of primary data. [2025-10-20T18:10:31.556788] [DEBUG] [LOCAL ] CURB: No primary data file found to delete. [2025-10-20T18:10:31.556828] [INFO ] [LOCAL ] CURB: Starting deletion of derivative data. [2025-10-20T18:10:31.557290] [DEBUG] [LOCAL ] CURB: Deleted derivative data file. [2025-10-20T18:10:31.557969] [INFO ] [MAIN ] Update Progress: 558/2367 symbols completed. CURB took 28.3s. Estimated remaining: 3h 48m 24s. Rate: 7.9 symbols/min [2025-10-20T18:10:32.251395] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-07. [2025-10-20T18:10:32.598981] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:10:33.051384] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-08. [2025-10-20T18:10:33.563266] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:10:33.851574] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-08. [2025-10-20T18:10:34.500025] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:10:34.651480] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-09. [2025-10-20T18:10:35.438623] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:10:35.451332] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-09. [2025-10-20T18:10:35.725131] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:10:36.251388] [INFO ] [REMOTE ] CURV: Fetching primary data for 2025-10. [2025-10-20T18:10:37.204424] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T18:10:37.215445] [INFO ] [REMOTE ] CUZ: Fetching primary data for 2025-10. [2025-10-20T18:10:37.364368] [INFO ] [REMOTE ] CURV: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:10:37.369639] [INFO ] [REMOTE ] CURV: Full fetch complete (38962 records). [2025-10-20T18:10:37.369817] [DEBUG] [VALIDATOR ] CURV: Validating primary data. [2025-10-20T18:10:37.776756] [DEBUG] [VALIDATOR ] CURV: All expected trading days are present. [2025-10-20T18:10:37.776863] [DEBUG] [VALIDATOR ] CURV: Primary data validation passed. [2025-10-20T18:10:37.777431] [INFO ] [MAIN ] CURV: Fetched primary data is valid. [2025-10-20T18:10:37.777469] [DEBUG] [LOCAL ] CURV: Saving primary data. [2025-10-20T18:10:37.778465] [DEBUG] [LOCAL ] CURV: Trimmed 38962 records to 27263 (keeping 200 days of data) [2025-10-20T18:10:37.936405] [DEBUG] [LOCAL ] CURV: Primary data saved (27263 records). [2025-10-20T18:10:37.936569] [INFO ] [MAIN ] CURV: Full history saved. [2025-10-20T18:10:37.936688] [INFO ] [MAIN ] Update Progress: 559/2367 symbols completed. CURV took 28.6s. Estimated remaining: 3h 48m 13s. Rate: 7.9 symbols/min [2025-10-20T18:10:38.676680] [INFO ] [REMOTE ] CUZ: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:10:38.683875] [INFO ] [REMOTE ] CUZ: Full fetch complete (70899 records). [2025-10-20T18:10:38.684029] [DEBUG] [VALIDATOR ] CUZ: Validating primary data. [2025-10-20T18:10:39.076086] [DEBUG] [VALIDATOR ] CUZ: All expected trading days are present. [2025-10-20T18:10:39.076205] [DEBUG] [VALIDATOR ] CUZ: Primary data validation passed. [2025-10-20T18:10:39.077089] [INFO ] [MAIN ] CUZ: Fetched primary data is valid. [2025-10-20T18:10:39.077157] [DEBUG] [LOCAL ] CUZ: Saving primary data. [2025-10-20T18:10:39.078628] [DEBUG] [LOCAL ] CUZ: Trimmed 70899 records to 49575 (keeping 200 days of data) [2025-10-20T18:10:39.383751] [DEBUG] [LOCAL ] CUZ: Primary data saved (49575 records). [2025-10-20T18:10:39.383898] [INFO ] [MAIN ] CUZ: Full history saved. [2025-10-20T18:10:39.383978] [INFO ] [MAIN ] Update Progress: 560/2367 symbols completed. CUZ took 29.4s. Estimated remaining: 3h 47m 45s. Rate: 7.9 symbols/min [2025-10-20T18:10:40.108312] [INFO ] [MAIN ] CVLG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.183464] [INFO ] [MAIN ] CVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.230499] [INFO ] [MAIN ] CVE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.253168] [INFO ] [MAIN ] CVEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:40.266217] [DEBUG] [LOCAL ] CVLG: Loaded 15760 existing records from primary storage. [2025-10-20T18:10:40.266966] [INFO ] [MAIN ] CVLG: Loaded 15760 existing records. [2025-10-20T18:10:40.395901] [DEBUG] [LOCAL ] CVEO: Loaded 15568 existing records from primary storage. [2025-10-20T18:10:40.449583] [INFO ] [MAIN ] CVEO: Loaded 15568 existing records. [2025-10-20T18:10:40.491925] [WARN ] [MAIN ] CVLG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:40.505357] [INFO ] [MAIN ] CVLG: Wiping existing primary data before full refresh. [2025-10-20T18:10:40.540743] [INFO ] [LOCAL ] CVLG: Starting deletion of primary data. [2025-10-20T18:10:40.612015] [DEBUG] [LOCAL ] CVLG: Deleted primary data file. [2025-10-20T18:10:40.611917] [DEBUG] [LOCAL ] CVI: Loaded 47330 existing records from primary storage. [2025-10-20T18:10:40.647349] [INFO ] [MAIN ] CVLG: Starting full primary refresh. [2025-10-20T18:10:40.673194] [INFO ] [MAIN ] CVI: Loaded 47330 existing records. [2025-10-20T18:10:40.672894] [DEBUG] [LOCAL ] CVE: Loaded 54342 existing records from primary storage. [2025-10-20T18:10:40.898175] [INFO ] [MAIN ] CVE: Loaded 54342 existing records. [2025-10-20T18:10:40.994342] [WARN ] [MAIN ] CVEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.020831] [INFO ] [MAIN ] CVEO: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.057218] [INFO ] [LOCAL ] CVEO: Starting deletion of primary data. [2025-10-20T18:10:41.124004] [DEBUG] [LOCAL ] CVEO: Deleted primary data file. [2025-10-20T18:10:41.206507] [INFO ] [MAIN ] CVEO: Starting full primary refresh. [2025-10-20T18:10:41.338263] [WARN ] [MAIN ] CVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.459624] [INFO ] [MAIN ] CVI: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.479803] [WARN ] [MAIN ] CVE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:41.528767] [INFO ] [LOCAL ] CVI: Starting deletion of primary data. [2025-10-20T18:10:41.555046] [INFO ] [MAIN ] CVE: Wiping existing primary data before full refresh. [2025-10-20T18:10:41.623941] [DEBUG] [LOCAL ] CVI: Deleted primary data file. [2025-10-20T18:10:41.602794] [INFO ] [LOCAL ] CVE: Starting deletion of primary data. [2025-10-20T18:10:41.671586] [INFO ] [MAIN ] CVI: Starting full primary refresh. [2025-10-20T18:10:41.778334] [DEBUG] [LOCAL ] CVE: Deleted primary data file. [2025-10-20T18:10:41.823840] [INFO ] [REMOTE ] CVLG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:41.829195] [INFO ] [MAIN ] CVE: Starting full primary refresh. [2025-10-20T18:10:41.848615] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-01. [2025-10-20T18:10:42.257685] [INFO ] [REMOTE ] CVEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.268677] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-01. [2025-10-20T18:10:42.795931] [INFO ] [REMOTE ] CVE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.796426] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-01. [2025-10-20T18:10:42.791725] [INFO ] [REMOTE ] CVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:10:42.797599] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-01. [2025-10-20T18:10:43.141092] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:10:43.145064] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-02. [2025-10-20T18:10:43.211580] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:10:43.214015] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-02. [2025-10-20T18:10:43.895411] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:10:43.898028] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-03. [2025-10-20T18:10:44.074326] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T18:10:44.077943] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-03. [2025-10-20T18:10:44.223623] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:10:44.251343] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-02. [2025-10-20T18:10:44.339356] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:10:44.445636] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T18:10:44.702386] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:10:45.051420] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-04. [2025-10-20T18:10:45.334528] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:10:45.851394] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-03. [2025-10-20T18:10:45.949919] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T18:10:46.651387] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-02. [2025-10-20T18:10:47.365840] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:10:47.451331] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-04. [2025-10-20T18:10:48.080402] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:10:48.251365] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-04. [2025-10-20T18:10:48.946556] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:10:49.051366] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-05. [2025-10-20T18:10:49.182629] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T18:10:49.771755] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:10:49.851347] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-06. [2025-10-20T18:10:50.651407] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-05. [2025-10-20T18:10:50.900635] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:10:51.451366] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-07. [2025-10-20T18:10:51.684276] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T18:10:52.184565] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:10:52.251461] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-05. [2025-10-20T18:10:53.051435] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-03. [2025-10-20T18:10:53.848337] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:10:53.857378] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-08. [2025-10-20T18:10:54.553792] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:10:54.651379] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-06. [2025-10-20T18:10:54.756399] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:10:55.376416] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:10:55.451438] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-09. [2025-10-20T18:10:56.195974] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:10:56.251367] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-04. [2025-10-20T18:10:57.051407] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-07. [2025-10-20T18:10:57.851397] [INFO ] [REMOTE ] CVLG: Fetching primary data for 2025-10. [2025-10-20T18:10:58.002667] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:10:58.148090] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:10:58.651405] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-05. [2025-10-20T18:10:58.674395] [INFO ] [REMOTE ] CVLG: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:10:58.676777] [INFO ] [REMOTE ] CVLG: Full fetch complete (21409 records). [2025-10-20T18:10:58.676872] [DEBUG] [VALIDATOR ] CVLG: Validating primary data. [2025-10-20T18:10:59.066437] [DEBUG] [VALIDATOR ] CVLG: All expected trading days are present. [2025-10-20T18:10:59.066516] [DEBUG] [VALIDATOR ] CVLG: Primary data validation passed. [2025-10-20T18:10:59.066953] [INFO ] [MAIN ] CVLG: Fetched primary data is valid. [2025-10-20T18:10:59.067014] [DEBUG] [LOCAL ] CVLG: Saving primary data. [2025-10-20T18:10:59.067567] [DEBUG] [LOCAL ] CVLG: Trimmed 21409 records to 15478 (keeping 200 days of data) [2025-10-20T18:10:59.157616] [DEBUG] [LOCAL ] CVLG: Primary data saved (15478 records). [2025-10-20T18:10:59.157769] [INFO ] [MAIN ] CVLG: Full history saved. [2025-10-20T18:10:59.157848] [INFO ] [MAIN ] Update Progress: 561/2367 symbols completed. CVLG took 19.8s. Estimated remaining: 3h 48m 17s. Rate: 7.9 symbols/min [2025-10-20T18:10:59.326607] [INFO ] [MAIN ] CVNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:10:59.375936] [DEBUG] [LOCAL ] CVNA: Loaded 53628 existing records from primary storage. [2025-10-20T18:10:59.376239] [INFO ] [MAIN ] CVNA: Loaded 53628 existing records. [2025-10-20T18:10:59.456683] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-08. [2025-10-20T18:10:59.567168] [WARN ] [MAIN ] CVNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:10:59.567576] [INFO ] [MAIN ] CVNA: Wiping existing primary data before full refresh. [2025-10-20T18:10:59.567665] [INFO ] [LOCAL ] CVNA: Starting deletion of primary data. [2025-10-20T18:10:59.568046] [DEBUG] [LOCAL ] CVNA: Deleted primary data file. [2025-10-20T18:10:59.568106] [INFO ] [MAIN ] CVNA: Starting full primary refresh. [2025-10-20T18:10:59.954414] [INFO ] [REMOTE ] CVNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:00.251419] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-06. [2025-10-20T18:11:00.365880] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:11:00.546675] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:11:01.051392] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-01. [2025-10-20T18:11:01.851454] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-09. [2025-10-20T18:11:02.213116] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-10-20T18:11:02.651432] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-06. [2025-10-20T18:11:02.759878] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T18:11:02.951899] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:11:03.451370] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-07. [2025-10-20T18:11:04.251444] [INFO ] [REMOTE ] CVEO: Fetching primary data for 2025-10. [2025-10-20T18:11:04.311156] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T18:11:05.051380] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-02. [2025-10-20T18:11:05.113152] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:11:05.156778] [INFO ] [REMOTE ] CVEO: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T18:11:05.159791] [INFO ] [REMOTE ] CVEO: Full fetch complete (20853 records). [2025-10-20T18:11:05.159982] [DEBUG] [VALIDATOR ] CVEO: Validating primary data. [2025-10-20T18:11:05.562953] [DEBUG] [VALIDATOR ] CVEO: All expected trading days are present. [2025-10-20T18:11:05.563031] [DEBUG] [VALIDATOR ] CVEO: Primary data validation passed. [2025-10-20T18:11:05.563494] [INFO ] [MAIN ] CVEO: Fetched primary data is valid. [2025-10-20T18:11:05.563533] [DEBUG] [LOCAL ] CVEO: Saving primary data. [2025-10-20T18:11:05.564171] [DEBUG] [LOCAL ] CVEO: Trimmed 20853 records to 15328 (keeping 200 days of data) [2025-10-20T18:11:05.652587] [DEBUG] [LOCAL ] CVEO: Primary data saved (15328 records). [2025-10-20T18:11:05.652792] [INFO ] [MAIN ] CVEO: Full history saved. [2025-10-20T18:11:05.652884] [INFO ] [MAIN ] Update Progress: 562/2367 symbols completed. CVEO took 26.3s. Estimated remaining: 3h 48m 6s. Rate: 7.9 symbols/min [2025-10-20T18:11:05.822519] [INFO ] [MAIN ] CVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:05.851484] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-08. [2025-10-20T18:11:05.873222] [DEBUG] [LOCAL ] CVS: Loaded 54474 existing records from primary storage. [2025-10-20T18:11:05.873576] [INFO ] [MAIN ] CVS: Loaded 54474 existing records. [2025-10-20T18:11:06.053750] [WARN ] [MAIN ] CVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:06.053967] [INFO ] [MAIN ] CVS: Wiping existing primary data before full refresh. [2025-10-20T18:11:06.054013] [INFO ] [LOCAL ] CVS: Starting deletion of primary data. [2025-10-20T18:11:06.054309] [DEBUG] [LOCAL ] CVS: Deleted primary data file. [2025-10-20T18:11:06.054346] [INFO ] [MAIN ] CVS: Starting full primary refresh. [2025-10-20T18:11:06.406949] [INFO ] [REMOTE ] CVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:06.651380] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-01. [2025-10-20T18:11:06.947518] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T18:11:07.429487] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:11:07.451368] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-09. [2025-10-20T18:11:08.251446] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-03. [2025-10-20T18:11:08.812747] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-01 (2.16 seconds). [2025-10-20T18:11:09.051397] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-07. [2025-10-20T18:11:09.084078] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:11:09.851454] [INFO ] [REMOTE ] CVI: Fetching primary data for 2025-10. [2025-10-20T18:11:10.277463] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T18:11:10.651413] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-04. [2025-10-20T18:11:10.695915] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:11:11.451357] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-02. [2025-10-20T18:11:11.681403] [INFO ] [REMOTE ] CVI: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:11:11.688457] [INFO ] [REMOTE ] CVI: Full fetch complete (67205 records). [2025-10-20T18:11:11.688569] [DEBUG] [VALIDATOR ] CVI: Validating primary data. [2025-10-20T18:11:12.069113] [DEBUG] [VALIDATOR ] CVI: All expected trading days are present. [2025-10-20T18:11:12.069193] [DEBUG] [VALIDATOR ] CVI: Primary data validation passed. [2025-10-20T18:11:12.069967] [INFO ] [MAIN ] CVI: Fetched primary data is valid. [2025-10-20T18:11:12.070027] [DEBUG] [LOCAL ] CVI: Saving primary data. [2025-10-20T18:11:12.071310] [DEBUG] [LOCAL ] CVI: Trimmed 67205 records to 46745 (keeping 200 days of data) [2025-10-20T18:11:12.267806] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-08. [2025-10-20T18:11:12.342789] [DEBUG] [LOCAL ] CVI: Primary data saved (46745 records). [2025-10-20T18:11:12.342950] [INFO ] [MAIN ] CVI: Full history saved. [2025-10-20T18:11:12.343052] [INFO ] [MAIN ] Update Progress: 563/2367 symbols completed. CVI took 32.9s. Estimated remaining: 3h 47m 56s. Rate: 7.9 symbols/min [2025-10-20T18:11:12.509737] [INFO ] [MAIN ] CVX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:12.509920] [DEBUG] [LOCAL ] CVX: No existing data file found in primary storage. [2025-10-20T18:11:12.509958] [WARN ] [MAIN ] CVX: No existing data found, performing full refresh instead. [2025-10-20T18:11:12.510073] [INFO ] [MAIN ] CVX: Wiping existing primary data before full refresh. [2025-10-20T18:11:12.510119] [INFO ] [LOCAL ] CVX: Starting deletion of primary data. [2025-10-20T18:11:12.510161] [DEBUG] [LOCAL ] CVX: No primary data file found to delete. [2025-10-20T18:11:12.510192] [INFO ] [MAIN ] CVX: Starting full primary refresh. [2025-10-20T18:11:12.867961] [INFO ] [REMOTE ] CVX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:12.965236] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T18:11:13.051373] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-05. [2025-10-20T18:11:13.343118] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:11:13.851420] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-03. [2025-10-20T18:11:14.107419] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T18:11:14.651382] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-01. [2025-10-20T18:11:15.062142] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-10-20T18:11:15.451409] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-09. [2025-10-20T18:11:15.713179] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:11:16.251432] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-06. [2025-10-20T18:11:16.928972] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:11:16.972006] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-01 (2.32 seconds). [2025-10-20T18:11:17.051576] [INFO ] [REMOTE ] CVE: Fetching primary data for 2025-10. [2025-10-20T18:11:17.851434] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-04. [2025-10-20T18:11:18.429209] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-06 (2.18 seconds). [2025-10-20T18:11:18.651380] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-02. [2025-10-20T18:11:18.960787] [INFO ] [REMOTE ] CVE: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:11:18.968523] [INFO ] [REMOTE ] CVE: Full fetch complete (77716 records). [2025-10-20T18:11:18.968675] [DEBUG] [VALIDATOR ] CVE: Validating primary data. [2025-10-20T18:11:19.354554] [DEBUG] [VALIDATOR ] CVE: All expected trading days are present. [2025-10-20T18:11:19.354638] [DEBUG] [VALIDATOR ] CVE: Primary data validation passed. [2025-10-20T18:11:19.355511] [INFO ] [MAIN ] CVE: Fetched primary data is valid. [2025-10-20T18:11:19.355554] [DEBUG] [LOCAL ] CVE: Saving primary data. [2025-10-20T18:11:19.357005] [DEBUG] [LOCAL ] CVE: Trimmed 77716 records to 53567 (keeping 200 days of data) [2025-10-20T18:11:19.451527] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-07. [2025-10-20T18:11:19.686481] [DEBUG] [LOCAL ] CVE: Primary data saved (53567 records). [2025-10-20T18:11:19.686625] [INFO ] [MAIN ] CVE: Full history saved. [2025-10-20T18:11:19.686737] [INFO ] [MAIN ] Update Progress: 564/2367 symbols completed. CVE took 40.3s. Estimated remaining: 3h 47m 47s. Rate: 7.9 symbols/min [2025-10-20T18:11:19.859062] [INFO ] [MAIN ] CW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:19.893989] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:11:19.922625] [DEBUG] [LOCAL ] CW: Loaded 22372 existing records from primary storage. [2025-10-20T18:11:19.922982] [INFO ] [MAIN ] CW: Loaded 22372 existing records. [2025-10-20T18:11:20.098801] [WARN ] [MAIN ] CW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:20.098983] [INFO ] [MAIN ] CW: Wiping existing primary data before full refresh. [2025-10-20T18:11:20.099047] [INFO ] [LOCAL ] CW: Starting deletion of primary data. [2025-10-20T18:11:20.099420] [DEBUG] [LOCAL ] CW: Deleted primary data file. [2025-10-20T18:11:20.099476] [INFO ] [MAIN ] CW: Starting full primary refresh. [2025-10-20T18:11:20.253750] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-05. [2025-10-20T18:11:20.460090] [INFO ] [REMOTE ] CW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:20.781717] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T18:11:21.051439] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-01. [2025-10-20T18:11:21.557498] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:11:21.851396] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-08. [2025-10-20T18:11:22.226134] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:11:22.505776] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T18:11:22.651380] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-06. [2025-10-20T18:11:23.451356] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-02. [2025-10-20T18:11:23.818693] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T18:11:24.012831] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:11:24.251382] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-09. [2025-10-20T18:11:24.526454] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:11:25.051371] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-03. [2025-10-20T18:11:25.680573] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:11:25.851372] [INFO ] [REMOTE ] CVNA: Fetching primary data for 2025-10. [2025-10-20T18:11:26.651445] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-07. [2025-10-20T18:11:27.216843] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T18:11:27.451394] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-04. [2025-10-20T18:11:27.950285] [INFO ] [REMOTE ] CVNA: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T18:11:27.958142] [INFO ] [REMOTE ] CVNA: Full fetch complete (76747 records). [2025-10-20T18:11:27.958290] [DEBUG] [VALIDATOR ] CVNA: Validating primary data. [2025-10-20T18:11:28.258257] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-03. [2025-10-20T18:11:28.379635] [DEBUG] [VALIDATOR ] CVNA: All expected trading days are present. [2025-10-20T18:11:28.379777] [DEBUG] [VALIDATOR ] CVNA: Primary data validation passed. [2025-10-20T18:11:28.380776] [INFO ] [MAIN ] CVNA: Fetched primary data is valid. [2025-10-20T18:11:28.380857] [DEBUG] [LOCAL ] CVNA: Saving primary data. [2025-10-20T18:11:28.382395] [DEBUG] [LOCAL ] CVNA: Trimmed 76747 records to 52842 (keeping 200 days of data) [2025-10-20T18:11:28.725887] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:11:28.756085] [DEBUG] [LOCAL ] CVNA: Primary data saved (52842 records). [2025-10-20T18:11:28.756619] [INFO ] [MAIN ] CVNA: Full history saved. [2025-10-20T18:11:28.756757] [INFO ] [MAIN ] Update Progress: 565/2367 symbols completed. CVNA took 29.6s. Estimated remaining: 3h 47m 44s. Rate: 7.9 symbols/min [2025-10-20T18:11:28.925630] [INFO ] [MAIN ] CWAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:28.974615] [DEBUG] [LOCAL ] CWAN: Loaded 53387 existing records from primary storage. [2025-10-20T18:11:28.974955] [INFO ] [MAIN ] CWAN: Loaded 53387 existing records. [2025-10-20T18:11:29.056119] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-08. [2025-10-20T18:11:29.159143] [WARN ] [MAIN ] CWAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:29.159474] [INFO ] [MAIN ] CWAN: Wiping existing primary data before full refresh. [2025-10-20T18:11:29.159607] [INFO ] [LOCAL ] CWAN: Starting deletion of primary data. [2025-10-20T18:11:29.159847] [DEBUG] [LOCAL ] CWAN: Deleted primary data file. [2025-10-20T18:11:29.159898] [INFO ] [MAIN ] CWAN: Starting full primary refresh. [2025-10-20T18:11:29.428073] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:11:29.559105] [INFO ] [REMOTE ] CWAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:29.591324] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T18:11:29.851391] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-04. [2025-10-20T18:11:30.651372] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-01. [2025-10-20T18:11:30.682493] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:11:31.257813] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T18:11:31.451406] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-09. [2025-10-20T18:11:32.251450] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-05. [2025-10-20T18:11:32.429095] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:11:33.051427] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-02. [2025-10-20T18:11:33.261793] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T18:11:33.430230] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T18:11:33.851400] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-06. [2025-10-20T18:11:34.255285] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:11:34.651427] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-03. [2025-10-20T18:11:35.214338] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:11:35.451406] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-07. [2025-10-20T18:11:36.251376] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-05. [2025-10-20T18:11:36.295573] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:11:36.495459] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:11:37.051374] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-08. [2025-10-20T18:11:37.851425] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-04. [2025-10-20T18:11:38.043458] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:11:38.651430] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-09. [2025-10-20T18:11:38.757955] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-05 (2.51 seconds). [2025-10-20T18:11:39.152580] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:11:39.451435] [INFO ] [REMOTE ] CVS: Fetching primary data for 2025-10. [2025-10-20T18:11:39.675926] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:11:40.251440] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-05. [2025-10-20T18:11:40.291097] [INFO ] [REMOTE ] CVS: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:11:40.299138] [INFO ] [REMOTE ] CVS: Full fetch complete (77874 records). [2025-10-20T18:11:40.299263] [DEBUG] [VALIDATOR ] CVS: Validating primary data. [2025-10-20T18:11:40.714621] [DEBUG] [VALIDATOR ] CVS: All expected trading days are present. [2025-10-20T18:11:40.714739] [DEBUG] [VALIDATOR ] CVS: Primary data validation passed. [2025-10-20T18:11:40.715678] [INFO ] [MAIN ] CVS: Fetched primary data is valid. [2025-10-20T18:11:40.715752] [DEBUG] [LOCAL ] CVS: Saving primary data. [2025-10-20T18:11:40.717270] [DEBUG] [LOCAL ] CVS: Trimmed 77874 records to 53694 (keeping 200 days of data) [2025-10-20T18:11:41.045743] [DEBUG] [LOCAL ] CVS: Primary data saved (53694 records). [2025-10-20T18:11:41.045884] [INFO ] [MAIN ] CVS: Full history saved. [2025-10-20T18:11:41.045958] [INFO ] [MAIN ] Update Progress: 566/2367 symbols completed. CVS took 35.4s. Estimated remaining: 3h 47m 52s. Rate: 7.9 symbols/min [2025-10-20T18:11:41.057289] [INFO ] [REMOTE ] CW: Fetching primary data for 2025-10. [2025-10-20T18:11:41.222072] [INFO ] [MAIN ] CWEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:41.264200] [DEBUG] [LOCAL ] CWEN: Loaded 45984 existing records from primary storage. [2025-10-20T18:11:41.264545] [INFO ] [MAIN ] CWEN: Loaded 45984 existing records. [2025-10-20T18:11:41.442746] [WARN ] [MAIN ] CWEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:41.442944] [INFO ] [MAIN ] CWEN: Wiping existing primary data before full refresh. [2025-10-20T18:11:41.442990] [INFO ] [LOCAL ] CWEN: Starting deletion of primary data. [2025-10-20T18:11:41.443202] [DEBUG] [LOCAL ] CWEN: Deleted primary data file. [2025-10-20T18:11:41.443259] [INFO ] [MAIN ] CWEN: Starting full primary refresh. [2025-10-20T18:11:41.808724] [INFO ] [REMOTE ] CWEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:41.851339] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-01. [2025-10-20T18:11:42.004362] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:11:42.352674] [INFO ] [REMOTE ] CW: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:11:42.356184] [INFO ] [REMOTE ] CW: Full fetch complete (33141 records). [2025-10-20T18:11:42.356299] [DEBUG] [VALIDATOR ] CW: Validating primary data. [2025-10-20T18:11:42.658733] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-06. [2025-10-20T18:11:42.816078] [DEBUG] [VALIDATOR ] CW: All expected trading days are present. [2025-10-20T18:11:42.816160] [DEBUG] [VALIDATOR ] CW: Primary data validation passed. [2025-10-20T18:11:42.816828] [INFO ] [MAIN ] CW: Fetched primary data is valid. [2025-10-20T18:11:42.816908] [DEBUG] [LOCAL ] CW: Saving primary data. [2025-10-20T18:11:42.818017] [DEBUG] [LOCAL ] CW: Trimmed 33141 records to 22034 (keeping 200 days of data) [2025-10-20T18:11:42.969568] [DEBUG] [LOCAL ] CW: Primary data saved (22034 records). [2025-10-20T18:11:42.969734] [INFO ] [MAIN ] CW: Full history saved. [2025-10-20T18:11:42.969827] [INFO ] [MAIN ] Update Progress: 567/2367 symbols completed. CW took 23.3s. Estimated remaining: 3h 47m 26s. Rate: 7.9 symbols/min [2025-10-20T18:11:43.144638] [INFO ] [MAIN ] CWH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:43.192962] [DEBUG] [LOCAL ] CWH: Loaded 52055 existing records from primary storage. [2025-10-20T18:11:43.193270] [INFO ] [MAIN ] CWH: Loaded 52055 existing records. [2025-10-20T18:11:43.371494] [WARN ] [MAIN ] CWH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:43.371787] [INFO ] [MAIN ] CWH: Wiping existing primary data before full refresh. [2025-10-20T18:11:43.371994] [INFO ] [LOCAL ] CWH: Starting deletion of primary data. [2025-10-20T18:11:43.372327] [DEBUG] [LOCAL ] CWH: Deleted primary data file. [2025-10-20T18:11:43.372382] [INFO ] [MAIN ] CWH: Starting full primary refresh. [2025-10-20T18:11:43.456583] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-06. [2025-10-20T18:11:43.612941] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:11:43.766637] [INFO ] [REMOTE ] CWH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:44.251340] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-02. [2025-10-20T18:11:45.051341] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-01. [2025-10-20T18:11:45.097627] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T18:11:45.462791] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T18:11:45.856562] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-07. [2025-10-20T18:11:45.866851] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:11:46.651399] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-07. [2025-10-20T18:11:46.699714] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:11:47.451426] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-03. [2025-10-20T18:11:48.251446] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-02. [2025-10-20T18:11:48.313208] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-10-20T18:11:48.376780] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T18:11:49.051416] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-08. [2025-10-20T18:11:49.120176] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T18:11:49.853372] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-08. [2025-10-20T18:11:49.861470] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:11:50.651383] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-04. [2025-10-20T18:11:51.257741] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T18:11:51.311990] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:11:51.451437] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-09. [2025-10-20T18:11:52.244385] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:11:52.253607] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-03. [2025-10-20T18:11:52.968228] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:11:53.051419] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-05. [2025-10-20T18:11:53.738081] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:11:53.851509] [INFO ] [REMOTE ] CWAN: Fetching primary data for 2025-10. [2025-10-20T18:11:54.651365] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-04. [2025-10-20T18:11:54.729109] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:11:55.456604] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-09. [2025-10-20T18:11:55.473009] [INFO ] [REMOTE ] CWAN: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:11:55.482412] [INFO ] [REMOTE ] CWAN: Full fetch complete (75556 records). [2025-10-20T18:11:55.482568] [DEBUG] [VALIDATOR ] CWAN: Validating primary data. [2025-10-20T18:11:55.883212] [DEBUG] [VALIDATOR ] CWAN: All expected trading days are present. [2025-10-20T18:11:55.883311] [DEBUG] [VALIDATOR ] CWAN: Primary data validation passed. [2025-10-20T18:11:55.884368] [INFO ] [MAIN ] CWAN: Fetched primary data is valid. [2025-10-20T18:11:55.884457] [DEBUG] [LOCAL ] CWAN: Saving primary data. [2025-10-20T18:11:55.886558] [DEBUG] [LOCAL ] CWAN: Trimmed 75556 records to 52668 (keeping 200 days of data) [2025-10-20T18:11:56.196959] [DEBUG] [LOCAL ] CWAN: Primary data saved (52668 records). [2025-10-20T18:11:56.197085] [INFO ] [MAIN ] CWAN: Full history saved. [2025-10-20T18:11:56.197165] [INFO ] [MAIN ] Update Progress: 568/2367 symbols completed. CWAN took 27.4s. Estimated remaining: 3h 47m 37s. Rate: 7.9 symbols/min [2025-10-20T18:11:56.260267] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-06. [2025-10-20T18:11:56.378186] [INFO ] [MAIN ] CWK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:11:56.421837] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:11:56.478670] [DEBUG] [LOCAL ] CWK: Loaded 50364 existing records from primary storage. [2025-10-20T18:11:56.478989] [INFO ] [MAIN ] CWK: Loaded 50364 existing records. [2025-10-20T18:11:56.657694] [WARN ] [MAIN ] CWK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:11:56.657918] [INFO ] [MAIN ] CWK: Wiping existing primary data before full refresh. [2025-10-20T18:11:56.657986] [INFO ] [LOCAL ] CWK: Starting deletion of primary data. [2025-10-20T18:11:56.658196] [DEBUG] [LOCAL ] CWK: Deleted primary data file. [2025-10-20T18:11:56.658255] [INFO ] [MAIN ] CWK: Starting full primary refresh. [2025-10-20T18:11:57.019987] [INFO ] [REMOTE ] CWK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:11:57.051381] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-01. [2025-10-20T18:11:57.677400] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T18:11:57.851374] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-05. [2025-10-20T18:11:57.953908] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T18:11:58.159595] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:11:58.651372] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-02. [2025-10-20T18:11:59.297881] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:11:59.451387] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-07. [2025-10-20T18:11:59.877406] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:12:00.251439] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-03. [2025-10-20T18:12:01.051431] [INFO ] [REMOTE ] CVX: Fetching primary data for 2025-10. [2025-10-20T18:12:01.428266] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T18:12:01.851364] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-08. [2025-10-20T18:12:02.101312] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T18:12:02.651365] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-04. [2025-10-20T18:12:03.329631] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:12:03.446111] [INFO ] [REMOTE ] CVX: Incremental fetch complete for 2025-10 (2.39 seconds). [2025-10-20T18:12:03.453284] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-06. [2025-10-20T18:12:03.454945] [INFO ] [REMOTE ] CVX: Full fetch complete (77940 records). [2025-10-20T18:12:03.455036] [DEBUG] [VALIDATOR ] CVX: Validating primary data. [2025-10-20T18:12:03.844850] [DEBUG] [VALIDATOR ] CVX: All expected trading days are present. [2025-10-20T18:12:03.844964] [DEBUG] [VALIDATOR ] CVX: Primary data validation passed. [2025-10-20T18:12:03.845907] [INFO ] [MAIN ] CVX: Fetched primary data is valid. [2025-10-20T18:12:03.846054] [DEBUG] [LOCAL ] CVX: Saving primary data. [2025-10-20T18:12:03.847964] [DEBUG] [LOCAL ] CVX: Trimmed 77940 records to 53760 (keeping 200 days of data) [2025-10-20T18:12:03.982688] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T18:12:04.222658] [DEBUG] [LOCAL ] CVX: Primary data saved (53760 records). [2025-10-20T18:12:04.222799] [INFO ] [MAIN ] CVX: Full history saved. [2025-10-20T18:12:04.222871] [INFO ] [MAIN ] Update Progress: 569/2367 symbols completed. CVX took 51.9s. Estimated remaining: 3h 47m 30s. Rate: 7.9 symbols/min [2025-10-20T18:12:04.256477] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-09. [2025-10-20T18:12:04.390500] [INFO ] [MAIN ] CWT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:04.417831] [DEBUG] [LOCAL ] CWT: Loaded 28510 existing records from primary storage. [2025-10-20T18:12:04.418090] [INFO ] [MAIN ] CWT: Loaded 28510 existing records. [2025-10-20T18:12:04.593610] [WARN ] [MAIN ] CWT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:04.593957] [INFO ] [MAIN ] CWT: Wiping existing primary data before full refresh. [2025-10-20T18:12:04.594017] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-10-20T18:12:04.594264] [DEBUG] [LOCAL ] CWT: Deleted primary data file. [2025-10-20T18:12:04.594320] [INFO ] [MAIN ] CWT: Starting full primary refresh. [2025-10-20T18:12:04.958029] [INFO ] [REMOTE ] CWT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:05.025239] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:12:05.051347] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-05. [2025-10-20T18:12:05.851347] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-01. [2025-10-20T18:12:06.025632] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:12:06.492034] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:12:06.651342] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-07. [2025-10-20T18:12:07.222488] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T18:12:07.451388] [INFO ] [REMOTE ] CWEN: Fetching primary data for 2025-10. [2025-10-20T18:12:08.251409] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-06. [2025-10-20T18:12:08.400385] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:12:09.051370] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-08. [2025-10-20T18:12:09.222727] [INFO ] [REMOTE ] CWEN: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:12:09.230436] [INFO ] [REMOTE ] CWEN: Full fetch complete (65523 records). [2025-10-20T18:12:09.230603] [DEBUG] [VALIDATOR ] CWEN: Validating primary data. [2025-10-20T18:12:09.637197] [DEBUG] [VALIDATOR ] CWEN: All expected trading days are present. [2025-10-20T18:12:09.637301] [DEBUG] [VALIDATOR ] CWEN: Primary data validation passed. [2025-10-20T18:12:09.638126] [INFO ] [MAIN ] CWEN: Fetched primary data is valid. [2025-10-20T18:12:09.638210] [DEBUG] [LOCAL ] CWEN: Saving primary data. [2025-10-20T18:12:09.639884] [DEBUG] [LOCAL ] CWEN: Trimmed 65523 records to 45314 (keeping 200 days of data) [2025-10-20T18:12:09.852072] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-02. [2025-10-20T18:12:09.885412] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:12:09.953293] [DEBUG] [LOCAL ] CWEN: Primary data saved (45314 records). [2025-10-20T18:12:09.953449] [INFO ] [MAIN ] CWEN: Full history saved. [2025-10-20T18:12:09.953970] [INFO ] [MAIN ] Update Progress: 570/2367 symbols completed. CWEN took 28.9s. Estimated remaining: 3h 47m 17s. Rate: 7.9 symbols/min [2025-10-20T18:12:10.126676] [INFO ] [MAIN ] CX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:10.175870] [DEBUG] [LOCAL ] CX: Loaded 53605 existing records from primary storage. [2025-10-20T18:12:10.176198] [INFO ] [MAIN ] CX: Loaded 53605 existing records. [2025-10-20T18:12:10.357789] [WARN ] [MAIN ] CX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:10.358030] [INFO ] [MAIN ] CX: Wiping existing primary data before full refresh. [2025-10-20T18:12:10.358075] [INFO ] [LOCAL ] CX: Starting deletion of primary data. [2025-10-20T18:12:10.358461] [DEBUG] [LOCAL ] CX: Deleted primary data file. [2025-10-20T18:12:10.358516] [INFO ] [MAIN ] CX: Starting full primary refresh. [2025-10-20T18:12:10.655231] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-07. [2025-10-20T18:12:10.777119] [INFO ] [REMOTE ] CX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:10.810706] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:12:11.110770] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T18:12:11.451354] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-09. [2025-10-20T18:12:12.066192] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:12:12.251408] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-08. [2025-10-20T18:12:13.051391] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-03. [2025-10-20T18:12:13.087560] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:12:13.851400] [INFO ] [REMOTE ] CWH: Fetching primary data for 2025-10. [2025-10-20T18:12:13.991709] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:12:14.372772] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:12:14.651411] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-04. [2025-10-20T18:12:15.265499] [INFO ] [REMOTE ] CWH: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T18:12:15.272945] [INFO ] [REMOTE ] CWH: Full fetch complete (73174 records). [2025-10-20T18:12:15.273101] [DEBUG] [VALIDATOR ] CWH: Validating primary data. [2025-10-20T18:12:15.456442] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-09. [2025-10-20T18:12:15.682292] [DEBUG] [VALIDATOR ] CWH: All expected trading days are present. [2025-10-20T18:12:15.682399] [DEBUG] [VALIDATOR ] CWH: Primary data validation passed. [2025-10-20T18:12:15.683697] [INFO ] [MAIN ] CWH: Fetched primary data is valid. [2025-10-20T18:12:15.683781] [DEBUG] [LOCAL ] CWH: Saving primary data. [2025-10-20T18:12:15.685798] [DEBUG] [LOCAL ] CWH: Trimmed 73174 records to 51314 (keeping 200 days of data) [2025-10-20T18:12:16.001110] [DEBUG] [LOCAL ] CWH: Primary data saved (51314 records). [2025-10-20T18:12:16.001246] [INFO ] [MAIN ] CWH: Full history saved. [2025-10-20T18:12:16.001322] [INFO ] [MAIN ] Update Progress: 571/2367 symbols completed. CWH took 33.0s. Estimated remaining: 3h 47m 4s. Rate: 7.9 symbols/min [2025-10-20T18:12:16.171768] [INFO ] [MAIN ] CXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:16.192734] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T18:12:16.204584] [DEBUG] [LOCAL ] CXE: Loaded 6233 existing records from primary storage. [2025-10-20T18:12:16.204800] [INFO ] [MAIN ] CXE: Loaded 6233 existing records. [2025-10-20T18:12:16.259348] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-01. [2025-10-20T18:12:16.385184] [WARN ] [MAIN ] CXE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:16.385391] [INFO ] [MAIN ] CXE: Wiping existing primary data before full refresh. [2025-10-20T18:12:16.385439] [INFO ] [LOCAL ] CXE: Starting deletion of primary data. [2025-10-20T18:12:16.385600] [DEBUG] [LOCAL ] CXE: Deleted primary data file. [2025-10-20T18:12:16.385693] [INFO ] [MAIN ] CXE: Starting full primary refresh. [2025-10-20T18:12:16.741398] [INFO ] [REMOTE ] CXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:16.954146] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:12:17.051375] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-01. [2025-10-20T18:12:17.851356] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-05. [2025-10-20T18:12:17.948533] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:12:18.000614] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T18:12:18.651439] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-02. [2025-10-20T18:12:18.778084] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:12:19.451446] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-02. [2025-10-20T18:12:20.125043] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T18:12:20.228421] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:12:20.251356] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-03. [2025-10-20T18:12:21.051386] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-03. [2025-10-20T18:12:21.703292] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T18:12:21.851365] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-04. [2025-10-20T18:12:21.904576] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:12:22.465452] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:12:22.651422] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-04. [2025-10-20T18:12:23.451378] [INFO ] [REMOTE ] CWK: Fetching primary data for 2025-10. [2025-10-20T18:12:24.251425] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-06. [2025-10-20T18:12:24.428450] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T18:12:25.040350] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:12:25.051348] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-05. [2025-10-20T18:12:25.320018] [INFO ] [REMOTE ] CWK: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:12:25.327818] [INFO ] [REMOTE ] CWK: Full fetch complete (71322 records). [2025-10-20T18:12:25.327966] [DEBUG] [VALIDATOR ] CWK: Validating primary data. [2025-10-20T18:12:25.727596] [DEBUG] [VALIDATOR ] CWK: All expected trading days are present. [2025-10-20T18:12:25.727709] [DEBUG] [VALIDATOR ] CWK: Primary data validation passed. [2025-10-20T18:12:25.728530] [INFO ] [MAIN ] CWK: Fetched primary data is valid. [2025-10-20T18:12:25.728574] [DEBUG] [LOCAL ] CWK: Saving primary data. [2025-10-20T18:12:25.730153] [DEBUG] [LOCAL ] CWK: Trimmed 71322 records to 49668 (keeping 200 days of data) [2025-10-20T18:12:25.822555] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T18:12:25.851739] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-05. [2025-10-20T18:12:26.030012] [DEBUG] [LOCAL ] CWK: Primary data saved (49668 records). [2025-10-20T18:12:26.030153] [INFO ] [MAIN ] CWK: Full history saved. [2025-10-20T18:12:26.030230] [INFO ] [MAIN ] Update Progress: 572/2367 symbols completed. CWK took 29.8s. Estimated remaining: 3h 47m 5s. Rate: 7.9 symbols/min [2025-10-20T18:12:26.196583] [INFO ] [MAIN ] CXH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:26.203993] [DEBUG] [LOCAL ] CXH: Loaded 4462 existing records from primary storage. [2025-10-20T18:12:26.204205] [INFO ] [MAIN ] CXH: Loaded 4462 existing records. [2025-10-20T18:12:26.378736] [WARN ] [MAIN ] CXH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:26.378971] [INFO ] [MAIN ] CXH: Wiping existing primary data before full refresh. [2025-10-20T18:12:26.379026] [INFO ] [LOCAL ] CXH: Starting deletion of primary data. [2025-10-20T18:12:26.379209] [DEBUG] [LOCAL ] CXH: Deleted primary data file. [2025-10-20T18:12:26.379265] [INFO ] [MAIN ] CXH: Starting full primary refresh. [2025-10-20T18:12:26.656625] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-06. [2025-10-20T18:12:26.749773] [INFO ] [REMOTE ] CXH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:27.330153] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:12:27.451338] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-01. [2025-10-20T18:12:27.686195] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:12:27.824628] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T18:12:28.251394] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-02. [2025-10-20T18:12:28.702171] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T18:12:29.051392] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-03. [2025-10-20T18:12:29.411753] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T18:12:29.851414] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-04. [2025-10-20T18:12:30.342823] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T18:12:30.651388] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-05. [2025-10-20T18:12:31.234977] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T18:12:31.451372] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-07. [2025-10-20T18:12:32.251380] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-06. [2025-10-20T18:12:32.652946] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:12:32.915248] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:12:33.051402] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-06. [2025-10-20T18:12:33.851433] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-07. [2025-10-20T18:12:34.312018] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:12:34.650291] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:12:34.659756] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-07. [2025-10-20T18:12:35.372819] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T18:12:35.451402] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-07. [2025-10-20T18:12:36.251436] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-08. [2025-10-20T18:12:37.051456] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-08. [2025-10-20T18:12:37.112757] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:12:37.395108] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T18:12:37.461304] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T18:12:37.851489] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-08. [2025-10-20T18:12:38.651369] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-09. [2025-10-20T18:12:39.451413] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-09. [2025-10-20T18:12:39.585208] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:12:39.640016] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T18:12:40.139575] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:12:40.251436] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-09. [2025-10-20T18:12:41.051458] [INFO ] [REMOTE ] CWT: Fetching primary data for 2025-10. [2025-10-20T18:12:41.851412] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-08. [2025-10-20T18:12:42.085358] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:12:42.118571] [INFO ] [REMOTE ] CWT: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:12:42.124078] [INFO ] [REMOTE ] CWT: Full fetch complete (40973 records). [2025-10-20T18:12:42.124301] [DEBUG] [VALIDATOR ] CWT: Validating primary data. [2025-10-20T18:12:42.125364] [FATAL] [VALIDATOR ] CWT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:12:42.125479] [FATAL] [MAIN ] CWT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:12:42.125537] [WARN ] [MAIN ] CWT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:12:42.125619] [INFO ] [LOCAL ] CWT: Starting deletion of primary data. [2025-10-20T18:12:42.125937] [DEBUG] [LOCAL ] CWT: No primary data file found to delete. [2025-10-20T18:12:42.125994] [INFO ] [LOCAL ] CWT: Starting deletion of derivative data. [2025-10-20T18:12:42.126184] [DEBUG] [LOCAL ] CWT: Deleted derivative data file. [2025-10-20T18:12:42.126312] [INFO ] [MAIN ] Update Progress: 573/2367 symbols completed. CWT took 37.9s. Estimated remaining: 3h 47m 24s. Rate: 7.9 symbols/min [2025-10-20T18:12:42.351169] [INFO ] [MAIN ] CXM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:42.402245] [DEBUG] [LOCAL ] CXM: Loaded 48245 existing records from primary storage. [2025-10-20T18:12:42.402630] [INFO ] [MAIN ] CXM: Loaded 48245 existing records. [2025-10-20T18:12:42.521812] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:12:42.632233] [WARN ] [MAIN ] CXM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:42.632460] [INFO ] [MAIN ] CXM: Wiping existing primary data before full refresh. [2025-10-20T18:12:42.632509] [INFO ] [LOCAL ] CXM: Starting deletion of primary data. [2025-10-20T18:12:42.632766] [DEBUG] [LOCAL ] CXM: Deleted primary data file. [2025-10-20T18:12:42.632823] [INFO ] [MAIN ] CXM: Starting full primary refresh. [2025-10-20T18:12:42.656492] [INFO ] [REMOTE ] CXH: Fetching primary data for 2025-10. [2025-10-20T18:12:43.013928] [INFO ] [REMOTE ] CXM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:43.174107] [INFO ] [REMOTE ] CXH: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T18:12:43.175004] [INFO ] [REMOTE ] CXH: Full fetch complete (5302 records). [2025-10-20T18:12:43.175074] [DEBUG] [VALIDATOR ] CXH: Validating primary data. [2025-10-20T18:12:43.456486] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-09. [2025-10-20T18:12:43.550698] [DEBUG] [VALIDATOR ] CXH: All expected trading days are present. [2025-10-20T18:12:43.550784] [DEBUG] [VALIDATOR ] CXH: Primary data validation passed. [2025-10-20T18:12:43.550942] [INFO ] [MAIN ] CXH: Fetched primary data is valid. [2025-10-20T18:12:43.550981] [DEBUG] [LOCAL ] CXH: Saving primary data. [2025-10-20T18:12:43.551382] [DEBUG] [LOCAL ] CXH: Trimmed 5302 records to 4421 (keeping 200 days of data) [2025-10-20T18:12:43.577539] [DEBUG] [LOCAL ] CXH: Primary data saved (4421 records). [2025-10-20T18:12:43.577712] [INFO ] [MAIN ] CXH: Full history saved. [2025-10-20T18:12:43.577823] [INFO ] [MAIN ] Update Progress: 574/2367 symbols completed. CXH took 17.5s. Estimated remaining: 3h 46m 57s. Rate: 7.9 symbols/min [2025-10-20T18:12:43.752119] [INFO ] [MAIN ] CXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:43.783045] [DEBUG] [LOCAL ] CXT: Loaded 31488 existing records from primary storage. [2025-10-20T18:12:43.783334] [INFO ] [MAIN ] CXT: Loaded 31488 existing records. [2025-10-20T18:12:43.984266] [WARN ] [MAIN ] CXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:43.984465] [INFO ] [MAIN ] CXT: Wiping existing primary data before full refresh. [2025-10-20T18:12:43.984513] [INFO ] [LOCAL ] CXT: Starting deletion of primary data. [2025-10-20T18:12:43.984737] [DEBUG] [LOCAL ] CXT: Deleted primary data file. [2025-10-20T18:12:43.984793] [INFO ] [MAIN ] CXT: Starting full primary refresh. [2025-10-20T18:12:44.184443] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T18:12:44.256457] [INFO ] [REMOTE ] CX: Fetching primary data for 2025-10. [2025-10-20T18:12:44.363615] [INFO ] [REMOTE ] CXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:45.051350] [INFO ] [REMOTE ] CXE: Fetching primary data for 2025-10. [2025-10-20T18:12:45.851441] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-01. [2025-10-20T18:12:45.876001] [INFO ] [REMOTE ] CXE: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:12:45.877488] [INFO ] [REMOTE ] CXE: Full fetch complete (11510 records). [2025-10-20T18:12:45.877557] [DEBUG] [VALIDATOR ] CXE: Validating primary data. [2025-10-20T18:12:46.177487] [INFO ] [REMOTE ] CX: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T18:12:46.230232] [INFO ] [REMOTE ] CX: Full fetch complete (76578 records). [2025-10-20T18:12:46.259038] [DEBUG] [VALIDATOR ] CX: Validating primary data. [2025-10-20T18:12:46.387891] [DEBUG] [VALIDATOR ] CXE: All expected trading days are present. [2025-10-20T18:12:46.414145] [DEBUG] [VALIDATOR ] CXE: Primary data validation passed. [2025-10-20T18:12:46.418135] [INFO ] [MAIN ] CXE: Fetched primary data is valid. [2025-10-20T18:12:46.418488] [DEBUG] [LOCAL ] CXE: Saving primary data. [2025-10-20T18:12:46.445100] [DEBUG] [LOCAL ] CXE: Trimmed 11510 records to 6342 (keeping 200 days of data) [2025-10-20T18:12:46.654910] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-01. [2025-10-20T18:12:46.678035] [DEBUG] [LOCAL ] CXE: Primary data saved (6342 records). [2025-10-20T18:12:46.684186] [INFO ] [MAIN ] CXE: Full history saved. [2025-10-20T18:12:46.684320] [INFO ] [MAIN ] Update Progress: 575/2367 symbols completed. CXE took 30.7s. Estimated remaining: 3h 46m 35s. Rate: 7.9 symbols/min [2025-10-20T18:12:46.897795] [DEBUG] [VALIDATOR ] CX: All expected trading days are present. [2025-10-20T18:12:46.898242] [DEBUG] [VALIDATOR ] CX: Primary data validation passed. [2025-10-20T18:12:46.910297] [INFO ] [MAIN ] CX: Fetched primary data is valid. [2025-10-20T18:12:46.915555] [DEBUG] [LOCAL ] CX: Saving primary data. [2025-10-20T18:12:46.979527] [INFO ] [MAIN ] CXW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:46.980732] [DEBUG] [LOCAL ] CX: Trimmed 76578 records to 52835 (keeping 200 days of data) [2025-10-20T18:12:47.235856] [DEBUG] [LOCAL ] CXW: Loaded 44487 existing records from primary storage. [2025-10-20T18:12:47.258355] [INFO ] [MAIN ] CXW: Loaded 44487 existing records. [2025-10-20T18:12:47.281408] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:12:47.485515] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-02. [2025-10-20T18:12:47.509440] [WARN ] [MAIN ] CXW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:47.527151] [INFO ] [MAIN ] CXW: Wiping existing primary data before full refresh. [2025-10-20T18:12:47.528219] [INFO ] [LOCAL ] CXW: Starting deletion of primary data. [2025-10-20T18:12:47.531803] [DEBUG] [LOCAL ] CXW: Deleted primary data file. [2025-10-20T18:12:47.532428] [INFO ] [MAIN ] CXW: Starting full primary refresh. [2025-10-20T18:12:47.751932] [DEBUG] [LOCAL ] CX: Primary data saved (52835 records). [2025-10-20T18:12:47.790732] [INFO ] [MAIN ] CX: Full history saved. [2025-10-20T18:12:47.824121] [INFO ] [MAIN ] Update Progress: 576/2367 symbols completed. CX took 37.9s. Estimated remaining: 3h 46m 8s. Rate: 7.9 symbols/min [2025-10-20T18:12:48.050825] [INFO ] [REMOTE ] CXW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:48.171735] [INFO ] [MAIN ] CYD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:12:48.201819] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:12:48.237103] [DEBUG] [LOCAL ] CYD: Loaded 27356 existing records from primary storage. [2025-10-20T18:12:48.237422] [INFO ] [MAIN ] CYD: Loaded 27356 existing records. [2025-10-20T18:12:48.258410] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-02. [2025-10-20T18:12:48.432166] [WARN ] [MAIN ] CYD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:12:48.432396] [INFO ] [MAIN ] CYD: Wiping existing primary data before full refresh. [2025-10-20T18:12:48.432447] [INFO ] [LOCAL ] CYD: Starting deletion of primary data. [2025-10-20T18:12:48.432704] [DEBUG] [LOCAL ] CYD: Deleted primary data file. [2025-10-20T18:12:48.432759] [INFO ] [MAIN ] CYD: Starting full primary refresh. [2025-10-20T18:12:48.811432] [INFO ] [REMOTE ] CYD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:12:48.834731] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:12:49.051399] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-01. [2025-10-20T18:12:49.759929] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T18:12:49.851386] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-03. [2025-10-20T18:12:50.651361] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-03. [2025-10-20T18:12:50.697807] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:12:51.451508] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-01. [2025-10-20T18:12:51.457088] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:12:52.031296] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:12:52.031562] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:12:52.251395] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-02. [2025-10-20T18:12:53.051363] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-04. [2025-10-20T18:12:53.218300] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T18:12:53.851401] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-04. [2025-10-20T18:12:54.484593] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:12:54.651386] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-03. [2025-10-20T18:12:55.262278] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T18:12:55.451409] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-05. [2025-10-20T18:12:56.037868] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:12:56.251378] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-02. [2025-10-20T18:12:56.872937] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T18:12:57.051385] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-06. [2025-10-20T18:12:57.844940] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:12:57.853403] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-04. [2025-10-20T18:12:58.651366] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-03. [2025-10-20T18:12:58.700403] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:12:59.115019] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:12:59.451388] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-05. [2025-10-20T18:12:59.910043] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T18:13:00.251415] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-07. [2025-10-20T18:13:00.563612] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T18:13:01.051450] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-05. [2025-10-20T18:13:01.851356] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-06. [2025-10-20T18:13:02.056698] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:13:02.601827] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:13:02.651358] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-04. [2025-10-20T18:13:02.794174] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:13:03.451429] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-06. [2025-10-20T18:13:04.054886] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:13:04.251411] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-05. [2025-10-20T18:13:04.683091] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:13:05.051474] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-07. [2025-10-20T18:13:05.777801] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:13:05.851353] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-06. [2025-10-20T18:13:06.540063] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:13:06.651373] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-08. [2025-10-20T18:13:07.234136] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:13:07.451380] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-08. [2025-10-20T18:13:07.644793] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:13:08.251426] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-09. [2025-10-20T18:13:09.051367] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-07. [2025-10-20T18:13:09.130660] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:13:09.207462] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T18:13:09.851350] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-09. [2025-10-20T18:13:10.315737] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T18:13:10.651370] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-07. [2025-10-20T18:13:11.424227] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:13:11.451349] [INFO ] [REMOTE ] CXM: Fetching primary data for 2025-10. [2025-10-20T18:13:12.251350] [INFO ] [REMOTE ] CYD: Fetching primary data for 2025-10. [2025-10-20T18:13:12.340508] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:13:12.594822] [INFO ] [REMOTE ] CXM: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:13:12.603787] [INFO ] [REMOTE ] CXM: Full fetch complete (68728 records). [2025-10-20T18:13:12.603953] [DEBUG] [VALIDATOR ] CXM: Validating primary data. [2025-10-20T18:13:13.009925] [DEBUG] [VALIDATOR ] CXM: All expected trading days are present. [2025-10-20T18:13:13.010010] [DEBUG] [VALIDATOR ] CXM: Primary data validation passed. [2025-10-20T18:13:13.010817] [INFO ] [MAIN ] CXM: Fetched primary data is valid. [2025-10-20T18:13:13.010879] [DEBUG] [LOCAL ] CXM: Saving primary data. [2025-10-20T18:13:13.012208] [DEBUG] [LOCAL ] CXM: Trimmed 68728 records to 47532 (keeping 200 days of data) [2025-10-20T18:13:13.056926] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-08. [2025-10-20T18:13:13.284442] [DEBUG] [LOCAL ] CXM: Primary data saved (47532 records). [2025-10-20T18:13:13.284608] [INFO ] [MAIN ] CXM: Full history saved. [2025-10-20T18:13:13.284738] [INFO ] [MAIN ] Update Progress: 577/2367 symbols completed. CXM took 31.2s. Estimated remaining: 3h 46m 55s. Rate: 7.9 symbols/min [2025-10-20T18:13:13.477144] [INFO ] [MAIN ] CYH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:13.490848] [INFO ] [REMOTE ] CYD: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T18:13:13.496009] [INFO ] [REMOTE ] CYD: Full fetch complete (35612 records). [2025-10-20T18:13:13.496089] [DEBUG] [VALIDATOR ] CYD: Validating primary data. [2025-10-20T18:13:13.811711] [DEBUG] [LOCAL ] CYH: Loaded 48440 existing records from primary storage. [2025-10-20T18:13:13.822696] [INFO ] [MAIN ] CYH: Loaded 48440 existing records. [2025-10-20T18:13:13.868778] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-08. [2025-10-20T18:13:13.992276] [DEBUG] [VALIDATOR ] CYD: All expected trading days are present. [2025-10-20T18:13:14.001206] [DEBUG] [VALIDATOR ] CYD: Primary data validation passed. [2025-10-20T18:13:14.004424] [INFO ] [MAIN ] CYD: Fetched primary data is valid. [2025-10-20T18:13:14.004521] [DEBUG] [LOCAL ] CYD: Saving primary data. [2025-10-20T18:13:14.058176] [DEBUG] [LOCAL ] CYD: Trimmed 35612 records to 27167 (keeping 200 days of data) [2025-10-20T18:13:14.087083] [WARN ] [MAIN ] CYH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:14.147394] [INFO ] [MAIN ] CYH: Wiping existing primary data before full refresh. [2025-10-20T18:13:14.147976] [INFO ] [LOCAL ] CYH: Starting deletion of primary data. [2025-10-20T18:13:14.175479] [DEBUG] [LOCAL ] CYH: Deleted primary data file. [2025-10-20T18:13:14.175854] [INFO ] [MAIN ] CYH: Starting full primary refresh. [2025-10-20T18:13:14.581143] [DEBUG] [LOCAL ] CYD: Primary data saved (27167 records). [2025-10-20T18:13:14.582594] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T18:13:14.588322] [INFO ] [MAIN ] CYD: Full history saved. [2025-10-20T18:13:14.600173] [INFO ] [MAIN ] Update Progress: 578/2367 symbols completed. CYD took 26.8s. Estimated remaining: 3h 46m 28s. Rate: 7.9 symbols/min [2025-10-20T18:13:14.659956] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-09. [2025-10-20T18:13:14.709414] [INFO ] [REMOTE ] CYH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:14.847363] [INFO ] [MAIN ] D: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:14.847611] [DEBUG] [LOCAL ] D: No existing data file found in primary storage. [2025-10-20T18:13:14.847680] [WARN ] [MAIN ] D: No existing data found, performing full refresh instead. [2025-10-20T18:13:14.847796] [INFO ] [MAIN ] D: Wiping existing primary data before full refresh. [2025-10-20T18:13:14.847836] [INFO ] [LOCAL ] D: Starting deletion of primary data. [2025-10-20T18:13:14.847876] [DEBUG] [LOCAL ] D: No primary data file found to delete. [2025-10-20T18:13:14.847907] [INFO ] [MAIN ] D: Starting full primary refresh. [2025-10-20T18:13:15.193210] [INFO ] [REMOTE ] D: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:15.451385] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-01. [2025-10-20T18:13:15.589029] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:13:16.182980] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:13:16.251343] [INFO ] [REMOTE ] CXT: Fetching primary data for 2025-10. [2025-10-20T18:13:17.051354] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-09. [2025-10-20T18:13:17.159365] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:13:17.431042] [INFO ] [REMOTE ] CXT: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:13:17.435992] [INFO ] [REMOTE ] CXT: Full fetch complete (45706 records). [2025-10-20T18:13:17.436088] [DEBUG] [VALIDATOR ] CXT: Validating primary data. [2025-10-20T18:13:17.800440] [DEBUG] [VALIDATOR ] CXT: All expected trading days are present. [2025-10-20T18:13:17.800522] [DEBUG] [VALIDATOR ] CXT: Primary data validation passed. [2025-10-20T18:13:17.801206] [INFO ] [MAIN ] CXT: Fetched primary data is valid. [2025-10-20T18:13:17.801268] [DEBUG] [LOCAL ] CXT: Saving primary data. [2025-10-20T18:13:17.802205] [DEBUG] [LOCAL ] CXT: Trimmed 45706 records to 30937 (keeping 200 days of data) [2025-10-20T18:13:17.853421] [INFO ] [REMOTE ] D: Fetching primary data for 2025-01. [2025-10-20T18:13:17.990876] [DEBUG] [LOCAL ] CXT: Primary data saved (30937 records). [2025-10-20T18:13:17.990999] [INFO ] [MAIN ] CXT: Full history saved. [2025-10-20T18:13:17.991074] [INFO ] [MAIN ] Update Progress: 579/2367 symbols completed. CXT took 34.4s. Estimated remaining: 3h 46m 8s. Rate: 7.9 symbols/min [2025-10-20T18:13:18.160206] [INFO ] [MAIN ] DAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:18.173838] [DEBUG] [LOCAL ] DAC: Loaded 12751 existing records from primary storage. [2025-10-20T18:13:18.174109] [INFO ] [MAIN ] DAC: Loaded 12751 existing records. [2025-10-20T18:13:18.328352] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:13:18.394239] [WARN ] [MAIN ] DAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:18.394638] [INFO ] [MAIN ] DAC: Wiping existing primary data before full refresh. [2025-10-20T18:13:18.394729] [INFO ] [LOCAL ] DAC: Starting deletion of primary data. [2025-10-20T18:13:18.394983] [DEBUG] [LOCAL ] DAC: Deleted primary data file. [2025-10-20T18:13:18.395044] [INFO ] [MAIN ] DAC: Starting full primary refresh. [2025-10-20T18:13:18.656588] [INFO ] [REMOTE ] CXW: Fetching primary data for 2025-10. [2025-10-20T18:13:18.779552] [INFO ] [REMOTE ] DAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:19.451409] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-01. [2025-10-20T18:13:19.969671] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T18:13:20.215022] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:13:20.251334] [INFO ] [REMOTE ] D: Fetching primary data for 2025-02. [2025-10-20T18:13:20.328235] [INFO ] [REMOTE ] CXW: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:13:20.334503] [INFO ] [REMOTE ] CXW: Full fetch complete (64874 records). [2025-10-20T18:13:20.334619] [DEBUG] [VALIDATOR ] CXW: Validating primary data. [2025-10-20T18:13:20.720251] [DEBUG] [VALIDATOR ] CXW: All expected trading days are present. [2025-10-20T18:13:20.720368] [DEBUG] [VALIDATOR ] CXW: Primary data validation passed. [2025-10-20T18:13:20.721143] [INFO ] [MAIN ] CXW: Fetched primary data is valid. [2025-10-20T18:13:20.721206] [DEBUG] [LOCAL ] CXW: Saving primary data. [2025-10-20T18:13:20.722454] [DEBUG] [LOCAL ] CXW: Trimmed 64874 records to 43775 (keeping 200 days of data) [2025-10-20T18:13:20.989221] [DEBUG] [LOCAL ] CXW: Primary data saved (43775 records). [2025-10-20T18:13:20.989358] [INFO ] [MAIN ] CXW: Full history saved. [2025-10-20T18:13:20.989436] [INFO ] [MAIN ] Update Progress: 580/2367 symbols completed. CXW took 34.3s. Estimated remaining: 3h 45m 46s. Rate: 7.9 symbols/min [2025-10-20T18:13:21.056556] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-02. [2025-10-20T18:13:21.157182] [INFO ] [MAIN ] DAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:21.205141] [DEBUG] [LOCAL ] DAL: Loaded 54533 existing records from primary storage. [2025-10-20T18:13:21.205505] [INFO ] [MAIN ] DAL: Loaded 54533 existing records. [2025-10-20T18:13:21.383035] [WARN ] [MAIN ] DAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:21.383222] [INFO ] [MAIN ] DAL: Wiping existing primary data before full refresh. [2025-10-20T18:13:21.383342] [INFO ] [LOCAL ] DAL: Starting deletion of primary data. [2025-10-20T18:13:21.383981] [DEBUG] [LOCAL ] DAL: Deleted primary data file. [2025-10-20T18:13:21.384037] [INFO ] [MAIN ] DAL: Starting full primary refresh. [2025-10-20T18:13:21.732621] [INFO ] [REMOTE ] DAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:21.851331] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-01. [2025-10-20T18:13:21.998378] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:13:22.651377] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-02. [2025-10-20T18:13:22.701825] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:13:23.451403] [INFO ] [REMOTE ] D: Fetching primary data for 2025-03. [2025-10-20T18:13:23.703574] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:13:23.926939] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T18:13:24.251427] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-02. [2025-10-20T18:13:25.051431] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-03. [2025-10-20T18:13:25.314339] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:13:25.851358] [INFO ] [REMOTE ] D: Fetching primary data for 2025-04. [2025-10-20T18:13:25.947597] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T18:13:26.267449] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T18:13:26.651409] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-04. [2025-10-20T18:13:27.451369] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-03. [2025-10-20T18:13:27.549205] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T18:13:27.754055] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:13:28.251431] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-05. [2025-10-20T18:13:29.051410] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-03. [2025-10-20T18:13:29.157699] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:13:29.317482] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:13:29.851420] [INFO ] [REMOTE ] D: Fetching primary data for 2025-05. [2025-10-20T18:13:30.579158] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:13:30.651365] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-04. [2025-10-20T18:13:31.451367] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-04. [2025-10-20T18:13:31.788431] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T18:13:32.251358] [INFO ] [REMOTE ] D: Fetching primary data for 2025-06. [2025-10-20T18:13:32.448027] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:13:33.051372] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-05. [2025-10-20T18:13:33.446674] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T18:13:33.851399] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-05. [2025-10-20T18:13:34.122544] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:13:34.651345] [INFO ] [REMOTE ] D: Fetching primary data for 2025-07. [2025-10-20T18:13:34.900195] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:13:35.451402] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-06. [2025-10-20T18:13:36.110413] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-10-20T18:13:36.251384] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-06. [2025-10-20T18:13:36.396147] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:13:36.491759] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:13:37.051415] [INFO ] [REMOTE ] D: Fetching primary data for 2025-08. [2025-10-20T18:13:37.851372] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-06. [2025-10-20T18:13:38.151468] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T18:13:38.651350] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-07. [2025-10-20T18:13:38.973054] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T18:13:39.074253] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T18:13:39.451342] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-07. [2025-10-20T18:13:40.251332] [INFO ] [REMOTE ] D: Fetching primary data for 2025-09. [2025-10-20T18:13:40.809084] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:13:41.051359] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-08. [2025-10-20T18:13:41.369925] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T18:13:41.851391] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-08. [2025-10-20T18:13:42.249917] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T18:13:42.651345] [INFO ] [REMOTE ] D: Fetching primary data for 2025-10. [2025-10-20T18:13:42.736861] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:13:43.451456] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-07. [2025-10-20T18:13:43.513741] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:13:43.773294] [INFO ] [REMOTE ] D: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:13:43.782791] [INFO ] [REMOTE ] D: Full fetch complete (77812 records). [2025-10-20T18:13:43.783272] [DEBUG] [VALIDATOR ] D: Validating primary data. [2025-10-20T18:13:44.207759] [DEBUG] [VALIDATOR ] D: All expected trading days are present. [2025-10-20T18:13:44.207864] [DEBUG] [VALIDATOR ] D: Primary data validation passed. [2025-10-20T18:13:44.208782] [INFO ] [MAIN ] D: Fetched primary data is valid. [2025-10-20T18:13:44.208849] [DEBUG] [LOCAL ] D: Saving primary data. [2025-10-20T18:13:44.210428] [DEBUG] [LOCAL ] D: Trimmed 77812 records to 53649 (keeping 200 days of data) [2025-10-20T18:13:44.264728] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-09. [2025-10-20T18:13:44.352477] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T18:13:44.560749] [DEBUG] [LOCAL ] D: Primary data saved (53649 records). [2025-10-20T18:13:44.560890] [INFO ] [MAIN ] D: Full history saved. [2025-10-20T18:13:44.560975] [INFO ] [MAIN ] Update Progress: 581/2367 symbols completed. D took 30.0s. Estimated remaining: 3h 46m 28s. Rate: 7.9 symbols/min [2025-10-20T18:13:44.726828] [INFO ] [MAIN ] DAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:44.772135] [DEBUG] [LOCAL ] DAN: Loaded 50069 existing records from primary storage. [2025-10-20T18:13:44.772439] [INFO ] [MAIN ] DAN: Loaded 50069 existing records. [2025-10-20T18:13:44.946758] [WARN ] [MAIN ] DAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:44.946982] [INFO ] [MAIN ] DAN: Wiping existing primary data before full refresh. [2025-10-20T18:13:44.947029] [INFO ] [LOCAL ] DAN: Starting deletion of primary data. [2025-10-20T18:13:44.947219] [DEBUG] [LOCAL ] DAN: Deleted primary data file. [2025-10-20T18:13:44.947257] [INFO ] [MAIN ] DAN: Starting full primary refresh. [2025-10-20T18:13:45.056478] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-08. [2025-10-20T18:13:45.327663] [INFO ] [REMOTE ] DAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:45.636143] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:13:45.851396] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-01. [2025-10-20T18:13:45.977010] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:13:46.651366] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-09. [2025-10-20T18:13:47.271984] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T18:13:47.451422] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-02. [2025-10-20T18:13:48.251434] [INFO ] [REMOTE ] CYH: Fetching primary data for 2025-10. [2025-10-20T18:13:48.482853] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:13:49.051793] [INFO ] [REMOTE ] DAL: Fetching primary data for 2025-10. [2025-10-20T18:13:49.057205] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:13:49.851398] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-03. [2025-10-20T18:13:49.886814] [INFO ] [REMOTE ] CYH: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:13:49.896619] [INFO ] [REMOTE ] CYH: Full fetch complete (69897 records). [2025-10-20T18:13:49.896821] [DEBUG] [VALIDATOR ] CYH: Validating primary data. [2025-10-20T18:13:50.081483] [INFO ] [REMOTE ] DAL: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:13:50.138959] [INFO ] [REMOTE ] DAL: Full fetch complete (77931 records). [2025-10-20T18:13:50.151298] [DEBUG] [VALIDATOR ] DAL: Validating primary data. [2025-10-20T18:13:50.661044] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-09. [2025-10-20T18:13:50.700921] [DEBUG] [VALIDATOR ] CYH: All expected trading days are present. [2025-10-20T18:13:50.719562] [DEBUG] [VALIDATOR ] CYH: Primary data validation passed. [2025-10-20T18:13:50.729899] [INFO ] [MAIN ] CYH: Fetched primary data is valid. [2025-10-20T18:13:50.774895] [DEBUG] [LOCAL ] CYH: Saving primary data. [2025-10-20T18:13:50.854168] [DEBUG] [LOCAL ] CYH: Trimmed 69897 records to 47742 (keeping 200 days of data) [2025-10-20T18:13:51.101960] [DEBUG] [VALIDATOR ] DAL: All expected trading days are present. [2025-10-20T18:13:51.102032] [DEBUG] [VALIDATOR ] DAL: Primary data validation passed. [2025-10-20T18:13:51.102848] [INFO ] [MAIN ] DAL: Fetched primary data is valid. [2025-10-20T18:13:51.107768] [DEBUG] [LOCAL ] DAL: Saving primary data. [2025-10-20T18:13:51.117546] [DEBUG] [LOCAL ] DAL: Trimmed 77931 records to 53753 (keeping 200 days of data) [2025-10-20T18:13:51.411256] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:13:51.453348] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-04. [2025-10-20T18:13:51.615233] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T18:13:51.637352] [DEBUG] [LOCAL ] CYH: Primary data saved (47742 records). [2025-10-20T18:13:51.638399] [INFO ] [MAIN ] CYH: Full history saved. [2025-10-20T18:13:51.639510] [INFO ] [MAIN ] Update Progress: 582/2367 symbols completed. CYH took 38.4s. Estimated remaining: 3h 46m 18s. Rate: 7.9 symbols/min [2025-10-20T18:13:51.828177] [INFO ] [MAIN ] DAO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:51.919837] [DEBUG] [LOCAL ] DAO: Loaded 20206 existing records from primary storage. [2025-10-20T18:13:51.921473] [INFO ] [MAIN ] DAO: Loaded 20206 existing records. [2025-10-20T18:13:52.109882] [WARN ] [MAIN ] DAO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:52.111745] [INFO ] [MAIN ] DAO: Wiping existing primary data before full refresh. [2025-10-20T18:13:52.112847] [INFO ] [LOCAL ] DAO: Starting deletion of primary data. [2025-10-20T18:13:52.113718] [DEBUG] [LOCAL ] DAO: Deleted primary data file. [2025-10-20T18:13:52.117978] [DEBUG] [LOCAL ] DAL: Primary data saved (53753 records). [2025-10-20T18:13:52.118129] [INFO ] [MAIN ] DAO: Starting full primary refresh. [2025-10-20T18:13:52.118339] [INFO ] [MAIN ] DAL: Full history saved. [2025-10-20T18:13:52.118487] [INFO ] [MAIN ] Update Progress: 583/2367 symbols completed. DAL took 31.1s. Estimated remaining: 3h 45m 49s. Rate: 7.9 symbols/min [2025-10-20T18:13:52.261856] [INFO ] [REMOTE ] DAC: Fetching primary data for 2025-10. [2025-10-20T18:13:52.516940] [INFO ] [MAIN ] DAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:52.677125] [INFO ] [REMOTE ] DAO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:52.712730] [DEBUG] [LOCAL ] DAR: Loaded 52762 existing records from primary storage. [2025-10-20T18:13:52.713141] [INFO ] [MAIN ] DAR: Loaded 52762 existing records. [2025-10-20T18:13:52.891095] [WARN ] [MAIN ] DAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:52.891370] [INFO ] [MAIN ] DAR: Wiping existing primary data before full refresh. [2025-10-20T18:13:52.891448] [INFO ] [LOCAL ] DAR: Starting deletion of primary data. [2025-10-20T18:13:52.891733] [DEBUG] [LOCAL ] DAR: Deleted primary data file. [2025-10-20T18:13:52.891832] [INFO ] [MAIN ] DAR: Starting full primary refresh. [2025-10-20T18:13:53.056673] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-01. [2025-10-20T18:13:53.221105] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:13:53.296638] [INFO ] [REMOTE ] DAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:53.319803] [INFO ] [REMOTE ] DAC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:13:53.322314] [INFO ] [REMOTE ] DAC: Full fetch complete (19260 records). [2025-10-20T18:13:53.322453] [DEBUG] [VALIDATOR ] DAC: Validating primary data. [2025-10-20T18:13:53.711348] [DEBUG] [VALIDATOR ] DAC: All expected trading days are present. [2025-10-20T18:13:53.711430] [DEBUG] [VALIDATOR ] DAC: Primary data validation passed. [2025-10-20T18:13:53.711926] [INFO ] [MAIN ] DAC: Fetched primary data is valid. [2025-10-20T18:13:53.711987] [DEBUG] [LOCAL ] DAC: Saving primary data. [2025-10-20T18:13:53.712708] [DEBUG] [LOCAL ] DAC: Trimmed 19260 records to 12590 (keeping 200 days of data) [2025-10-20T18:13:53.793127] [DEBUG] [LOCAL ] DAC: Primary data saved (12590 records). [2025-10-20T18:13:53.793261] [INFO ] [MAIN ] DAC: Full history saved. [2025-10-20T18:13:53.793334] [INFO ] [MAIN ] Update Progress: 584/2367 symbols completed. DAC took 35.8s. Estimated remaining: 3h 45m 23s. Rate: 7.9 symbols/min [2025-10-20T18:13:53.857235] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-05. [2025-10-20T18:13:53.973278] [INFO ] [MAIN ] DAVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:13:54.019901] [DEBUG] [LOCAL ] DAVA: Loaded 42431 existing records from primary storage. [2025-10-20T18:13:54.020412] [INFO ] [MAIN ] DAVA: Loaded 42431 existing records. [2025-10-20T18:13:54.212597] [WARN ] [MAIN ] DAVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:13:54.213018] [INFO ] [MAIN ] DAVA: Wiping existing primary data before full refresh. [2025-10-20T18:13:54.213084] [INFO ] [LOCAL ] DAVA: Starting deletion of primary data. [2025-10-20T18:13:54.213330] [DEBUG] [LOCAL ] DAVA: Deleted primary data file. [2025-10-20T18:13:54.213386] [INFO ] [MAIN ] DAVA: Starting full primary refresh. [2025-10-20T18:13:54.355060] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:13:54.586618] [INFO ] [REMOTE ] DAVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:13:54.651367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-02. [2025-10-20T18:13:55.445442] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T18:13:55.455126] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-01. [2025-10-20T18:13:55.983182] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:13:56.251429] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-01. [2025-10-20T18:13:56.882453] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:13:57.051422] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-03. [2025-10-20T18:13:57.842845] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T18:13:57.852669] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-02. [2025-10-20T18:13:58.112600] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T18:13:58.651410] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-04. [2025-10-20T18:13:59.040816] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T18:13:59.426932] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T18:13:59.451340] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-05. [2025-10-20T18:14:00.251328] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-06. [2025-10-20T18:14:00.577106] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T18:14:01.051367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-06. [2025-10-20T18:14:01.625721] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:14:01.862620] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-02. [2025-10-20T18:14:01.868499] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:14:02.651373] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-07. [2025-10-20T18:14:03.451353] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-07. [2025-10-20T18:14:03.542241] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:14:03.555332] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T18:14:04.251367] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-08. [2025-10-20T18:14:05.051357] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-03. [2025-10-20T18:14:05.236558] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:14:05.369761] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T18:14:05.851429] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-09. [2025-10-20T18:14:06.572372] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T18:14:06.658013] [INFO ] [REMOTE ] DAO: Fetching primary data for 2025-10. [2025-10-20T18:14:06.663261] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:14:07.451368] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-04. [2025-10-20T18:14:07.802603] [INFO ] [REMOTE ] DAO: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:14:07.805823] [INFO ] [REMOTE ] DAO: Full fetch complete (32419 records). [2025-10-20T18:14:07.805906] [DEBUG] [VALIDATOR ] DAO: Validating primary data. [2025-10-20T18:14:08.179119] [DEBUG] [VALIDATOR ] DAO: All expected trading days are present. [2025-10-20T18:14:08.179271] [DEBUG] [VALIDATOR ] DAO: Primary data validation passed. [2025-10-20T18:14:08.179861] [INFO ] [MAIN ] DAO: Fetched primary data is valid. [2025-10-20T18:14:08.179947] [DEBUG] [LOCAL ] DAO: Saving primary data. [2025-10-20T18:14:08.180705] [DEBUG] [LOCAL ] DAO: Trimmed 32419 records to 19694 (keeping 200 days of data) [2025-10-20T18:14:08.260545] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-03. [2025-10-20T18:14:08.296510] [DEBUG] [LOCAL ] DAO: Primary data saved (19694 records). [2025-10-20T18:14:08.296732] [INFO ] [MAIN ] DAO: Full history saved. [2025-10-20T18:14:08.296944] [INFO ] [MAIN ] Update Progress: 585/2367 symbols completed. DAO took 16.7s. Estimated remaining: 3h 45m 37s. Rate: 7.9 symbols/min [2025-10-20T18:14:08.477169] [INFO ] [MAIN ] DAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:08.477327] [DEBUG] [LOCAL ] DAY: No existing data file found in primary storage. [2025-10-20T18:14:08.477372] [WARN ] [MAIN ] DAY: No existing data found, performing full refresh instead. [2025-10-20T18:14:08.477502] [INFO ] [MAIN ] DAY: Wiping existing primary data before full refresh. [2025-10-20T18:14:08.477543] [INFO ] [LOCAL ] DAY: Starting deletion of primary data. [2025-10-20T18:14:08.477583] [DEBUG] [LOCAL ] DAY: No primary data file found to delete. [2025-10-20T18:14:08.477614] [INFO ] [MAIN ] DAY: Starting full primary refresh. [2025-10-20T18:14:08.835304] [INFO ] [REMOTE ] DAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:09.051339] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-08. [2025-10-20T18:14:09.190017] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:14:09.833225] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:14:09.851346] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-05. [2025-10-20T18:14:10.449914] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:14:10.651364] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-09. [2025-10-20T18:14:11.388272] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:14:11.451369] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-06. [2025-10-20T18:14:11.904096] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:14:12.251355] [INFO ] [REMOTE ] DAN: Fetching primary data for 2025-10. [2025-10-20T18:14:13.056527] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-04. [2025-10-20T18:14:13.069616] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:14:13.636084] [INFO ] [REMOTE ] DAN: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:14:13.643632] [INFO ] [REMOTE ] DAN: Full fetch complete (71321 records). [2025-10-20T18:14:13.643815] [DEBUG] [VALIDATOR ] DAN: Validating primary data. [2025-10-20T18:14:13.856475] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-01. [2025-10-20T18:14:14.050284] [DEBUG] [VALIDATOR ] DAN: All expected trading days are present. [2025-10-20T18:14:14.050390] [DEBUG] [VALIDATOR ] DAN: Primary data validation passed. [2025-10-20T18:14:14.051213] [INFO ] [MAIN ] DAN: Fetched primary data is valid. [2025-10-20T18:14:14.051275] [DEBUG] [LOCAL ] DAN: Saving primary data. [2025-10-20T18:14:14.052750] [DEBUG] [LOCAL ] DAN: Trimmed 71321 records to 49407 (keeping 200 days of data) [2025-10-20T18:14:14.291916] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T18:14:14.379064] [DEBUG] [LOCAL ] DAN: Primary data saved (49407 records). [2025-10-20T18:14:14.379221] [INFO ] [MAIN ] DAN: Full history saved. [2025-10-20T18:14:14.379540] [INFO ] [MAIN ] Update Progress: 586/2367 symbols completed. DAN took 29.8s. Estimated remaining: 3h 45m 25s. Rate: 7.9 symbols/min [2025-10-20T18:14:14.552986] [INFO ] [MAIN ] DB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:14.605157] [DEBUG] [LOCAL ] DB: Loaded 53281 existing records from primary storage. [2025-10-20T18:14:14.605595] [INFO ] [MAIN ] DB: Loaded 53281 existing records. [2025-10-20T18:14:14.656499] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-07. [2025-10-20T18:14:14.793181] [WARN ] [MAIN ] DB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:14.793389] [INFO ] [MAIN ] DB: Wiping existing primary data before full refresh. [2025-10-20T18:14:14.793435] [INFO ] [LOCAL ] DB: Starting deletion of primary data. [2025-10-20T18:14:14.793944] [DEBUG] [LOCAL ] DB: Deleted primary data file. [2025-10-20T18:14:14.794002] [INFO ] [MAIN ] DB: Starting full primary refresh. [2025-10-20T18:14:15.151713] [INFO ] [REMOTE ] DB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:15.451346] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-01. [2025-10-20T18:14:15.685046] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:14:16.193897] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:14:16.251334] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-08. [2025-10-20T18:14:17.051348] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-02. [2025-10-20T18:14:17.304219] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:14:17.526233] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:14:17.851367] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-09. [2025-10-20T18:14:18.596043] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:14:18.651419] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-05. [2025-10-20T18:14:19.420003] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:14:19.451379] [INFO ] [REMOTE ] DAR: Fetching primary data for 2025-10. [2025-10-20T18:14:20.102355] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T18:14:20.251403] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-03. [2025-10-20T18:14:21.051402] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-06. [2025-10-20T18:14:21.262828] [INFO ] [REMOTE ] DAR: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:14:21.270531] [INFO ] [REMOTE ] DAR: Full fetch complete (74980 records). [2025-10-20T18:14:21.270729] [DEBUG] [VALIDATOR ] DAR: Validating primary data. [2025-10-20T18:14:21.691028] [DEBUG] [VALIDATOR ] DAR: All expected trading days are present. [2025-10-20T18:14:21.691237] [DEBUG] [VALIDATOR ] DAR: Primary data validation passed. [2025-10-20T18:14:21.692189] [INFO ] [MAIN ] DAR: Fetched primary data is valid. [2025-10-20T18:14:21.692343] [DEBUG] [LOCAL ] DAR: Saving primary data. [2025-10-20T18:14:21.693860] [DEBUG] [LOCAL ] DAR: Trimmed 74980 records to 51998 (keeping 200 days of data) [2025-10-20T18:14:21.859108] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-02. [2025-10-20T18:14:22.019056] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:14:22.041327] [DEBUG] [LOCAL ] DAR: Primary data saved (51998 records). [2025-10-20T18:14:22.041443] [INFO ] [MAIN ] DAR: Full history saved. [2025-10-20T18:14:22.041514] [INFO ] [MAIN ] Update Progress: 587/2367 symbols completed. DAR took 29.9s. Estimated remaining: 3h 45m 17s. Rate: 7.9 symbols/min [2025-10-20T18:14:22.216223] [INFO ] [MAIN ] DBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:22.216452] [DEBUG] [LOCAL ] DBD: No existing data file found in primary storage. [2025-10-20T18:14:22.216554] [WARN ] [MAIN ] DBD: No existing data found, performing full refresh instead. [2025-10-20T18:14:22.216815] [INFO ] [MAIN ] DBD: Wiping existing primary data before full refresh. [2025-10-20T18:14:22.216883] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-10-20T18:14:22.216996] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-10-20T18:14:22.217133] [INFO ] [MAIN ] DBD: Starting full primary refresh. [2025-10-20T18:14:22.331177] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:14:22.600007] [INFO ] [REMOTE ] DBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:22.651357] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-01. [2025-10-20T18:14:23.424299] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:14:23.451411] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-04. [2025-10-20T18:14:23.629518] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:14:24.251491] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-03. [2025-10-20T18:14:25.051363] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-07. [2025-10-20T18:14:25.124403] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:14:25.851389] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-02. [2025-10-20T18:14:26.341950] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T18:14:26.418105] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T18:14:26.651429] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-04. [2025-10-20T18:14:26.832586] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:14:27.451382] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-03. [2025-10-20T18:14:28.109943] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:14:28.251437] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-08. [2025-10-20T18:14:28.453812] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:14:29.051469] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-04. [2025-10-20T18:14:29.841334] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:14:29.854275] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-05. [2025-10-20T18:14:30.134266] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T18:14:30.651378] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-05. [2025-10-20T18:14:31.451393] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-09. [2025-10-20T18:14:31.709602] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:14:32.251353] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-06. [2025-10-20T18:14:32.541603] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:14:33.051440] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-06. [2025-10-20T18:14:33.343749] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T18:14:33.655129] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:14:33.851367] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-05. [2025-10-20T18:14:34.651391] [INFO ] [REMOTE ] DAVA: Fetching primary data for 2025-10. [2025-10-20T18:14:34.842906] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T18:14:34.903135] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T18:14:35.451385] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-07. [2025-10-20T18:14:36.251381] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-07. [2025-10-20T18:14:36.548803] [INFO ] [REMOTE ] DAVA: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:14:36.558031] [INFO ] [REMOTE ] DAVA: Full fetch complete (57179 records). [2025-10-20T18:14:36.558184] [DEBUG] [VALIDATOR ] DAVA: Validating primary data. [2025-10-20T18:14:37.035438] [DEBUG] [VALIDATOR ] DAVA: All expected trading days are present. [2025-10-20T18:14:37.035528] [DEBUG] [VALIDATOR ] DAVA: Primary data validation passed. [2025-10-20T18:14:37.036477] [INFO ] [MAIN ] DAVA: Fetched primary data is valid. [2025-10-20T18:14:37.036541] [DEBUG] [LOCAL ] DAVA: Saving primary data. [2025-10-20T18:14:37.037944] [DEBUG] [LOCAL ] DAVA: Trimmed 57179 records to 41996 (keeping 200 days of data) [2025-10-20T18:14:37.087202] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-06. [2025-10-20T18:14:37.283178] [DEBUG] [LOCAL ] DAVA: Primary data saved (41996 records). [2025-10-20T18:14:37.283309] [INFO ] [MAIN ] DAVA: Full history saved. [2025-10-20T18:14:37.283385] [INFO ] [MAIN ] Update Progress: 588/2367 symbols completed. DAVA took 43.5s. Estimated remaining: 3h 45m 33s. Rate: 7.9 symbols/min [2025-10-20T18:14:37.458991] [INFO ] [MAIN ] DBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:37.492011] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:14:37.536633] [DEBUG] [LOCAL ] DBI: Loaded 44390 existing records from primary storage. [2025-10-20T18:14:37.536957] [INFO ] [MAIN ] DBI: Loaded 44390 existing records. [2025-10-20T18:14:37.716192] [WARN ] [MAIN ] DBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:37.716444] [INFO ] [MAIN ] DBI: Wiping existing primary data before full refresh. [2025-10-20T18:14:37.716517] [INFO ] [LOCAL ] DBI: Starting deletion of primary data. [2025-10-20T18:14:37.716910] [DEBUG] [LOCAL ] DBI: Deleted primary data file. [2025-10-20T18:14:37.716967] [INFO ] [MAIN ] DBI: Starting full primary refresh. [2025-10-20T18:14:37.855248] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-08. [2025-10-20T18:14:38.115113] [INFO ] [REMOTE ] DBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:38.117316] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:14:38.142272] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:14:38.651361] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-07. [2025-10-20T18:14:39.451432] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-01. [2025-10-20T18:14:39.735403] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T18:14:39.981402] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T18:14:40.251411] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-08. [2025-10-20T18:14:40.731272] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:14:41.051356] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-02. [2025-10-20T18:14:41.384515] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T18:14:41.851463] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-09. [2025-10-20T18:14:42.631157] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:14:42.651347] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-03. [2025-10-20T18:14:42.683550] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:14:43.451423] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-09. [2025-10-20T18:14:44.251899] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-08. [2025-10-20T18:14:44.251757] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:14:45.051397] [INFO ] [REMOTE ] DBD: Fetching primary data for 2025-10. [2025-10-20T18:14:45.325670] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:14:45.851411] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-04. [2025-10-20T18:14:46.077778] [INFO ] [REMOTE ] DBD: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:14:46.081763] [INFO ] [REMOTE ] DBD: Full fetch complete (31685 records). [2025-10-20T18:14:46.081876] [DEBUG] [VALIDATOR ] DBD: Validating primary data. [2025-10-20T18:14:46.082436] [FATAL] [VALIDATOR ] DBD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:14:46.082523] [FATAL] [MAIN ] DBD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:14:46.082565] [WARN ] [MAIN ] DBD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:14:46.082627] [INFO ] [LOCAL ] DBD: Starting deletion of primary data. [2025-10-20T18:14:46.082734] [DEBUG] [LOCAL ] DBD: No primary data file found to delete. [2025-10-20T18:14:46.082776] [INFO ] [LOCAL ] DBD: Starting deletion of derivative data. [2025-10-20T18:14:46.082816] [DEBUG] [LOCAL ] DBD: No derivative data file found to delete. [2025-10-20T18:14:46.082870] [INFO ] [MAIN ] Update Progress: 589/2367 symbols completed. DBD took 24.0s. Estimated remaining: 3h 45m 29s. Rate: 7.9 symbols/min [2025-10-20T18:14:46.254282] [INFO ] [MAIN ] DBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:46.254476] [DEBUG] [LOCAL ] DBL: No existing data file found in primary storage. [2025-10-20T18:14:46.254547] [WARN ] [MAIN ] DBL: No existing data found, performing full refresh instead. [2025-10-20T18:14:46.254823] [INFO ] [MAIN ] DBL: Wiping existing primary data before full refresh. [2025-10-20T18:14:46.254889] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-10-20T18:14:46.254977] [DEBUG] [LOCAL ] DBL: No primary data file found to delete. [2025-10-20T18:14:46.255027] [INFO ] [MAIN ] DBL: Starting full primary refresh. [2025-10-20T18:14:46.394257] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T18:14:46.656574] [INFO ] [REMOTE ] DB: Fetching primary data for 2025-10. [2025-10-20T18:14:46.660922] [INFO ] [REMOTE ] DBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:47.315992] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:14:47.451400] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-05. [2025-10-20T18:14:48.236876] [INFO ] [REMOTE ] DB: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T18:14:48.244517] [INFO ] [REMOTE ] DB: Full fetch complete (75563 records). [2025-10-20T18:14:48.244630] [DEBUG] [VALIDATOR ] DB: Validating primary data. [2025-10-20T18:14:48.256593] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-01. [2025-10-20T18:14:48.659513] [DEBUG] [VALIDATOR ] DB: All expected trading days are present. [2025-10-20T18:14:48.659582] [DEBUG] [VALIDATOR ] DB: Primary data validation passed. [2025-10-20T18:14:48.660442] [INFO ] [MAIN ] DB: Fetched primary data is valid. [2025-10-20T18:14:48.660505] [DEBUG] [LOCAL ] DB: Saving primary data. [2025-10-20T18:14:48.661924] [DEBUG] [LOCAL ] DB: Trimmed 75563 records to 52484 (keeping 200 days of data) [2025-10-20T18:14:48.975272] [DEBUG] [LOCAL ] DB: Primary data saved (52484 records). [2025-10-20T18:14:48.975377] [INFO ] [MAIN ] DB: Full history saved. [2025-10-20T18:14:48.975674] [INFO ] [MAIN ] Update Progress: 590/2367 symbols completed. DB took 34.6s. Estimated remaining: 3h 45m 7s. Rate: 7.9 symbols/min [2025-10-20T18:14:49.067488] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-09. [2025-10-20T18:14:49.115460] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T18:14:49.116251] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T18:14:49.196801] [INFO ] [MAIN ] DBRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:49.244838] [DEBUG] [LOCAL ] DBRG: Loaded 50682 existing records from primary storage. [2025-10-20T18:14:49.245123] [INFO ] [MAIN ] DBRG: Loaded 50682 existing records. [2025-10-20T18:14:49.423119] [WARN ] [MAIN ] DBRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:49.423348] [INFO ] [MAIN ] DBRG: Wiping existing primary data before full refresh. [2025-10-20T18:14:49.423401] [INFO ] [LOCAL ] DBRG: Starting deletion of primary data. [2025-10-20T18:14:49.423612] [DEBUG] [LOCAL ] DBRG: Deleted primary data file. [2025-10-20T18:14:49.423714] [INFO ] [MAIN ] DBRG: Starting full primary refresh. [2025-10-20T18:14:49.773521] [INFO ] [REMOTE ] DBRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:49.851346] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-01. [2025-10-20T18:14:50.651359] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-02. [2025-10-20T18:14:50.745356] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:14:51.413684] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:14:51.451329] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-03. [2025-10-20T18:14:51.549432] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:14:52.217489] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T18:14:52.251401] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-02. [2025-10-20T18:14:53.051401] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-06. [2025-10-20T18:14:53.815977] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:14:53.851339] [INFO ] [REMOTE ] DAY: Fetching primary data for 2025-10. [2025-10-20T18:14:54.651375] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-03. [2025-10-20T18:14:54.766065] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:14:55.451423] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-07. [2025-10-20T18:14:55.495075] [INFO ] [REMOTE ] DAY: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:14:55.501922] [INFO ] [REMOTE ] DAY: Full fetch complete (71787 records). [2025-10-20T18:14:55.502040] [DEBUG] [VALIDATOR ] DAY: Validating primary data. [2025-10-20T18:14:55.896544] [DEBUG] [VALIDATOR ] DAY: All expected trading days are present. [2025-10-20T18:14:55.896627] [DEBUG] [VALIDATOR ] DAY: Primary data validation passed. [2025-10-20T18:14:55.897458] [INFO ] [MAIN ] DAY: Fetched primary data is valid. [2025-10-20T18:14:55.897511] [DEBUG] [LOCAL ] DAY: Saving primary data. [2025-10-20T18:14:55.898922] [DEBUG] [LOCAL ] DAY: Trimmed 71787 records to 49167 (keeping 200 days of data) [2025-10-20T18:14:56.183777] [DEBUG] [LOCAL ] DAY: Primary data saved (49167 records). [2025-10-20T18:14:56.183904] [INFO ] [MAIN ] DAY: Full history saved. [2025-10-20T18:14:56.183972] [INFO ] [MAIN ] Update Progress: 591/2367 symbols completed. DAY took 47.9s. Estimated remaining: 3h 44m 58s. Rate: 7.9 symbols/min [2025-10-20T18:14:56.255837] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-04. [2025-10-20T18:14:56.365898] [INFO ] [MAIN ] DCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:14:56.396947] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:14:56.436858] [DEBUG] [LOCAL ] DCI: Loaded 37560 existing records from primary storage. [2025-10-20T18:14:56.437157] [INFO ] [MAIN ] DCI: Loaded 37560 existing records. [2025-10-20T18:14:56.615612] [WARN ] [MAIN ] DCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:14:56.615853] [INFO ] [MAIN ] DCI: Wiping existing primary data before full refresh. [2025-10-20T18:14:56.615905] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-10-20T18:14:56.616134] [DEBUG] [LOCAL ] DCI: Deleted primary data file. [2025-10-20T18:14:56.616189] [INFO ] [MAIN ] DCI: Starting full primary refresh. [2025-10-20T18:14:57.014533] [INFO ] [REMOTE ] DCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:14:57.015381] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:14:57.046265] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T18:14:57.055715] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-01. [2025-10-20T18:14:57.851366] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-08. [2025-10-20T18:14:58.491873] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:14:58.651377] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-02. [2025-10-20T18:14:59.311758] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:14:59.451375] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-05. [2025-10-20T18:14:59.906691] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T18:15:00.045102] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:15:00.251422] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-04. [2025-10-20T18:15:01.051342] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-03. [2025-10-20T18:15:01.851417] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-09. [2025-10-20T18:15:02.268707] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:15:02.389996] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:15:02.651405] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-05. [2025-10-20T18:15:03.451778] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-04. [2025-10-20T18:15:03.457049] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:15:04.251366] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-06. [2025-10-20T18:15:04.481090] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:15:05.051349] [INFO ] [REMOTE ] DBI: Fetching primary data for 2025-10. [2025-10-20T18:15:05.078735] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:15:05.124881] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T18:15:05.851415] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-05. [2025-10-20T18:15:06.307446] [INFO ] [REMOTE ] DBI: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:15:06.316343] [INFO ] [REMOTE ] DBI: Full fetch complete (64952 records). [2025-10-20T18:15:06.316557] [DEBUG] [VALIDATOR ] DBI: Validating primary data. [2025-10-20T18:15:06.656514] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-06. [2025-10-20T18:15:06.814424] [DEBUG] [VALIDATOR ] DBI: All expected trading days are present. [2025-10-20T18:15:06.814512] [DEBUG] [VALIDATOR ] DBI: Primary data validation passed. [2025-10-20T18:15:06.815319] [INFO ] [MAIN ] DBI: Fetched primary data is valid. [2025-10-20T18:15:06.815386] [DEBUG] [LOCAL ] DBI: Saving primary data. [2025-10-20T18:15:06.816784] [DEBUG] [LOCAL ] DBI: Trimmed 64952 records to 43623 (keeping 200 days of data) [2025-10-20T18:15:07.069745] [DEBUG] [LOCAL ] DBI: Primary data saved (43623 records). [2025-10-20T18:15:07.069874] [INFO ] [MAIN ] DBI: Full history saved. [2025-10-20T18:15:07.069971] [INFO ] [MAIN ] Update Progress: 592/2367 symbols completed. DBI took 29.8s. Estimated remaining: 3h 45m 0s. Rate: 7.9 symbols/min [2025-10-20T18:15:07.245408] [INFO ] [MAIN ] DCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:07.260947] [DEBUG] [LOCAL ] DCO: Loaded 14832 existing records from primary storage. [2025-10-20T18:15:07.261165] [INFO ] [MAIN ] DCO: Loaded 14832 existing records. [2025-10-20T18:15:07.402298] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:15:07.466342] [WARN ] [MAIN ] DCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:07.466940] [INFO ] [MAIN ] DCO: Wiping existing primary data before full refresh. [2025-10-20T18:15:07.461465] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-07. [2025-10-20T18:15:07.467765] [INFO ] [LOCAL ] DCO: Starting deletion of primary data. [2025-10-20T18:15:07.467945] [DEBUG] [LOCAL ] DCO: Deleted primary data file. [2025-10-20T18:15:07.467997] [INFO ] [MAIN ] DCO: Starting full primary refresh. [2025-10-20T18:15:07.860631] [INFO ] [REMOTE ] DCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:07.918458] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:15:08.251358] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-07. [2025-10-20T18:15:08.393978] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:15:09.051377] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-01. [2025-10-20T18:15:09.733918] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:15:09.851377] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-02. [2025-10-20T18:15:10.251013] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T18:15:10.346435] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T18:15:10.651389] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-03. [2025-10-20T18:15:11.451360] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-06. [2025-10-20T18:15:11.481008] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:15:12.251364] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-08. [2025-10-20T18:15:12.624140] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T18:15:13.051398] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-04. [2025-10-20T18:15:13.746583] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T18:15:13.851357] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-05. [2025-10-20T18:15:13.965986] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T18:15:14.651382] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-09. [2025-10-20T18:15:14.662500] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:15:15.451364] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-06. [2025-10-20T18:15:16.251372] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-08. [2025-10-20T18:15:16.393914] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T18:15:16.507281] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:15:16.978558] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T18:15:17.051502] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-07. [2025-10-20T18:15:17.851386] [INFO ] [REMOTE ] DBRG: Fetching primary data for 2025-10. [2025-10-20T18:15:18.458022] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:15:18.651437] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-07. [2025-10-20T18:15:19.451426] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-09. [2025-10-20T18:15:19.506807] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T18:15:19.751186] [INFO ] [REMOTE ] DBRG: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:15:19.759305] [INFO ] [REMOTE ] DBRG: Full fetch complete (72495 records). [2025-10-20T18:15:19.759470] [DEBUG] [VALIDATOR ] DBRG: Validating primary data. [2025-10-20T18:15:20.218414] [DEBUG] [VALIDATOR ] DBRG: All expected trading days are present. [2025-10-20T18:15:20.218508] [DEBUG] [VALIDATOR ] DBRG: Primary data validation passed. [2025-10-20T18:15:20.219493] [INFO ] [MAIN ] DBRG: Fetched primary data is valid. [2025-10-20T18:15:20.219577] [DEBUG] [LOCAL ] DBRG: Saving primary data. [2025-10-20T18:15:20.221242] [DEBUG] [LOCAL ] DBRG: Trimmed 72495 records to 49985 (keeping 200 days of data) [2025-10-20T18:15:20.295500] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-08. [2025-10-20T18:15:20.349900] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:15:20.563378] [DEBUG] [LOCAL ] DBRG: Primary data saved (49985 records). [2025-10-20T18:15:20.563529] [INFO ] [MAIN ] DBRG: Full history saved. [2025-10-20T18:15:20.563890] [INFO ] [MAIN ] Update Progress: 593/2367 symbols completed. DBRG took 31.6s. Estimated remaining: 3h 45m 10s. Rate: 7.9 symbols/min [2025-10-20T18:15:20.733680] [INFO ] [MAIN ] DD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:20.785967] [DEBUG] [LOCAL ] DD: Loaded 53704 existing records from primary storage. [2025-10-20T18:15:20.786296] [INFO ] [MAIN ] DD: Loaded 53704 existing records. [2025-10-20T18:15:20.973709] [WARN ] [MAIN ] DD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:20.973942] [INFO ] [MAIN ] DD: Wiping existing primary data before full refresh. [2025-10-20T18:15:20.974011] [INFO ] [LOCAL ] DD: Starting deletion of primary data. [2025-10-20T18:15:20.974383] [DEBUG] [LOCAL ] DD: Deleted primary data file. [2025-10-20T18:15:20.974452] [INFO ] [MAIN ] DD: Starting full primary refresh. [2025-10-20T18:15:21.056473] [INFO ] [REMOTE ] DBL: Fetching primary data for 2025-10. [2025-10-20T18:15:21.329268] [INFO ] [REMOTE ] DD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:21.668527] [INFO ] [REMOTE ] DBL: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:15:21.670213] [INFO ] [REMOTE ] DBL: Full fetch complete (12640 records). [2025-10-20T18:15:21.670287] [DEBUG] [VALIDATOR ] DBL: Validating primary data. [2025-10-20T18:15:21.670703] [FATAL] [VALIDATOR ] DBL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:15:21.670797] [FATAL] [MAIN ] DBL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:15:21.670876] [WARN ] [MAIN ] DBL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:15:21.670972] [INFO ] [LOCAL ] DBL: Starting deletion of primary data. [2025-10-20T18:15:21.671052] [DEBUG] [LOCAL ] DBL: No primary data file found to delete. [2025-10-20T18:15:21.671089] [INFO ] [LOCAL ] DBL: Starting deletion of derivative data. [2025-10-20T18:15:21.671134] [DEBUG] [LOCAL ] DBL: No derivative data file found to delete. [2025-10-20T18:15:21.671189] [INFO ] [MAIN ] Update Progress: 594/2367 symbols completed. DBL took 35.6s. Estimated remaining: 3h 44m 43s. Rate: 7.9 symbols/min [2025-10-20T18:15:21.852710] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-01. [2025-10-20T18:15:21.880032] [INFO ] [MAIN ] DDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:21.882500] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:15:21.937833] [DEBUG] [LOCAL ] DDD: Loaded 51691 existing records from primary storage. [2025-10-20T18:15:21.938156] [INFO ] [MAIN ] DDD: Loaded 51691 existing records. [2025-10-20T18:15:22.110761] [WARN ] [MAIN ] DDD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:22.110963] [INFO ] [MAIN ] DDD: Wiping existing primary data before full refresh. [2025-10-20T18:15:22.111010] [INFO ] [LOCAL ] DDD: Starting deletion of primary data. [2025-10-20T18:15:22.111184] [DEBUG] [LOCAL ] DDD: Deleted primary data file. [2025-10-20T18:15:22.111221] [INFO ] [MAIN ] DDD: Starting full primary refresh. [2025-10-20T18:15:22.458303] [INFO ] [REMOTE ] DDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:22.651366] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-08. [2025-10-20T18:15:23.451425] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-01. [2025-10-20T18:15:23.746333] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:15:23.781023] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T18:15:24.251397] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-09. [2025-10-20T18:15:25.051393] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-09. [2025-10-20T18:15:25.342417] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:15:25.851392] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-02. [2025-10-20T18:15:25.879001] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:15:25.995160] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:15:26.651370] [INFO ] [REMOTE ] DCI: Fetching primary data for 2025-10. [2025-10-20T18:15:27.451395] [INFO ] [REMOTE ] DCO: Fetching primary data for 2025-10. [2025-10-20T18:15:27.660668] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T18:15:28.040156] [INFO ] [REMOTE ] DCI: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T18:15:28.045339] [INFO ] [REMOTE ] DCI: Full fetch complete (53421 records). [2025-10-20T18:15:28.045489] [DEBUG] [VALIDATOR ] DCI: Validating primary data. [2025-10-20T18:15:28.046143] [FATAL] [VALIDATOR ] DCI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:15:28.046236] [FATAL] [MAIN ] DCI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:15:28.046288] [WARN ] [MAIN ] DCI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:15:28.046366] [INFO ] [LOCAL ] DCI: Starting deletion of primary data. [2025-10-20T18:15:28.046443] [DEBUG] [LOCAL ] DCI: No primary data file found to delete. [2025-10-20T18:15:28.046490] [INFO ] [LOCAL ] DCI: Starting deletion of derivative data. [2025-10-20T18:15:28.046704] [DEBUG] [LOCAL ] DCI: Deleted derivative data file. [2025-10-20T18:15:28.046812] [INFO ] [MAIN ] Update Progress: 595/2367 symbols completed. DCI took 31.9s. Estimated remaining: 3h 44m 32s. Rate: 7.9 symbols/min [2025-10-20T18:15:28.232099] [INFO ] [MAIN ] DDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:28.251827] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-02. [2025-10-20T18:15:28.269765] [DEBUG] [LOCAL ] DDL: Loaded 37618 existing records from primary storage. [2025-10-20T18:15:28.270121] [INFO ] [MAIN ] DDL: Loaded 37618 existing records. [2025-10-20T18:15:28.464086] [WARN ] [MAIN ] DDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:28.464449] [INFO ] [MAIN ] DDL: Wiping existing primary data before full refresh. [2025-10-20T18:15:28.464559] [INFO ] [LOCAL ] DDL: Starting deletion of primary data. [2025-10-20T18:15:28.472338] [DEBUG] [LOCAL ] DDL: Deleted primary data file. [2025-10-20T18:15:28.472480] [INFO ] [MAIN ] DDL: Starting full primary refresh. [2025-10-20T18:15:28.493783] [INFO ] [REMOTE ] DCO: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:15:28.561884] [INFO ] [REMOTE ] DCO: Full fetch complete (19547 records). [2025-10-20T18:15:28.589487] [DEBUG] [VALIDATOR ] DCO: Validating primary data. [2025-10-20T18:15:29.067856] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-03. [2025-10-20T18:15:29.158037] [INFO ] [REMOTE ] DDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:29.257940] [DEBUG] [VALIDATOR ] DCO: All expected trading days are present. [2025-10-20T18:15:29.258034] [DEBUG] [VALIDATOR ] DCO: Primary data validation passed. [2025-10-20T18:15:29.258454] [INFO ] [MAIN ] DCO: Fetched primary data is valid. [2025-10-20T18:15:29.258494] [DEBUG] [LOCAL ] DCO: Saving primary data. [2025-10-20T18:15:29.259352] [DEBUG] [LOCAL ] DCO: Trimmed 19547 records to 14765 (keeping 200 days of data) [2025-10-20T18:15:29.346300] [DEBUG] [LOCAL ] DCO: Primary data saved (14765 records). [2025-10-20T18:15:29.346451] [INFO ] [MAIN ] DCO: Full history saved. [2025-10-20T18:15:29.346530] [INFO ] [MAIN ] Update Progress: 596/2367 symbols completed. DCO took 22.3s. Estimated remaining: 3h 44m 6s. Rate: 7.9 symbols/min [2025-10-20T18:15:29.527054] [INFO ] [MAIN ] DDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:29.541622] [DEBUG] [LOCAL ] DDS: Loaded 12846 existing records from primary storage. [2025-10-20T18:15:29.541874] [INFO ] [MAIN ] DDS: Loaded 12846 existing records. [2025-10-20T18:15:29.732594] [WARN ] [MAIN ] DDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:29.733293] [INFO ] [MAIN ] DDS: Wiping existing primary data before full refresh. [2025-10-20T18:15:29.733347] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-10-20T18:15:29.733493] [DEBUG] [LOCAL ] DDS: Deleted primary data file. [2025-10-20T18:15:29.733530] [INFO ] [MAIN ] DDS: Starting full primary refresh. [2025-10-20T18:15:29.854622] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-01. [2025-10-20T18:15:30.136506] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T18:15:30.141829] [INFO ] [REMOTE ] DDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:30.651388] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-03. [2025-10-20T18:15:30.955567] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:15:31.319084] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:15:31.451445] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-02. [2025-10-20T18:15:32.251348] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-04. [2025-10-20T18:15:32.376326] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:15:32.709490] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T18:15:33.051407] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-01. [2025-10-20T18:15:33.851384] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-03. [2025-10-20T18:15:34.632524] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-10-20T18:15:34.651332] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-05. [2025-10-20T18:15:35.451375] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-04. [2025-10-20T18:15:36.063594] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-01 (3.01 seconds). [2025-10-20T18:15:36.251426] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-02. [2025-10-20T18:15:36.648853] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-03 (2.80 seconds). [2025-10-20T18:15:37.006542] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:15:37.051435] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-03. [2025-10-20T18:15:37.285378] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-05 (2.63 seconds). [2025-10-20T18:15:37.785015] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:15:37.858697] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-04. [2025-10-20T18:15:37.878271] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-04 (2.43 seconds). [2025-10-20T18:15:38.651372] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-05. [2025-10-20T18:15:39.451350] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-04. [2025-10-20T18:15:39.484929] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:15:40.149081] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:15:40.251339] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-06. [2025-10-20T18:15:40.299724] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:15:41.051399] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-05. [2025-10-20T18:15:41.851406] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-06. [2025-10-20T18:15:41.871259] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T18:15:41.987808] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:15:42.651365] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-06. [2025-10-20T18:15:43.309457] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:15:43.457555] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-07. [2025-10-20T18:15:43.478506] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:15:44.108794] [INFO ] [REMOTE ] DDS: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:15:44.251392] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-07. [2025-10-20T18:15:45.051376] [INFO ] [REMOTE ] DDS: Fetching primary data for 2025-08. [2025-10-20T18:15:45.157206] [DEBUG] [REMOTE ] DDS: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-10-20T18:15:45.157464] [WARN ] [REMOTE ] DDS: No data returned for month 2025-08 [2025-10-20T18:15:45.157550] [FATAL] [REMOTE ] DDS: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:15:45.157629] [FATAL] [MAIN ] DDS: Failed to fetch primary data. [2025-10-20T18:15:45.157712] [WARN ] [MAIN ] DDS: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:15:45.157785] [INFO ] [LOCAL ] DDS: Starting deletion of primary data. [2025-10-20T18:15:45.157846] [DEBUG] [LOCAL ] DDS: No primary data file found to delete. [2025-10-20T18:15:45.157882] [INFO ] [LOCAL ] DDS: Starting deletion of derivative data. [2025-10-20T18:15:45.158027] [DEBUG] [LOCAL ] DDS: Deleted derivative data file. [2025-10-20T18:15:45.158107] [INFO ] [MAIN ] Update Progress: 597/2367 symbols completed. DDS took 15.8s. Estimated remaining: 3h 44m 23s. Rate: 7.9 symbols/min [2025-10-20T18:15:45.851372] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-05. [2025-10-20T18:15:46.289053] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:15:46.651386] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-08. [2025-10-20T18:15:47.451443] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-07. [2025-10-20T18:15:47.975423] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:15:48.251409] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-06. [2025-10-20T18:15:49.093433] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-08 (2.44 seconds). [2025-10-20T18:15:49.104836] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-09. [2025-10-20T18:15:49.744254] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T18:15:49.851364] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-07. [2025-10-20T18:15:50.381721] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-07 (2.93 seconds). [2025-10-20T18:15:50.651362] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-08. [2025-10-20T18:15:50.829450] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:15:51.078295] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:15:51.451436] [INFO ] [REMOTE ] DD: Fetching primary data for 2025-10. [2025-10-20T18:15:52.262605] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-08. [2025-10-20T18:15:52.265218] [INFO ] [REMOTE ] DD: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:15:52.274096] [INFO ] [REMOTE ] DD: Full fetch complete (76619 records). [2025-10-20T18:15:52.274255] [DEBUG] [VALIDATOR ] DD: Validating primary data. [2025-10-20T18:15:52.607192] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T18:15:52.758517] [DEBUG] [VALIDATOR ] DD: All expected trading days are present. [2025-10-20T18:15:52.758668] [DEBUG] [VALIDATOR ] DD: Primary data validation passed. [2025-10-20T18:15:52.759717] [INFO ] [MAIN ] DD: Fetched primary data is valid. [2025-10-20T18:15:52.759785] [DEBUG] [LOCAL ] DD: Saving primary data. [2025-10-20T18:15:52.761354] [DEBUG] [LOCAL ] DD: Trimmed 76619 records to 52934 (keeping 200 days of data) [2025-10-20T18:15:53.061988] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-09. [2025-10-20T18:15:53.112980] [DEBUG] [LOCAL ] DD: Primary data saved (52934 records). [2025-10-20T18:15:53.113135] [INFO ] [MAIN ] DD: Full history saved. [2025-10-20T18:15:53.113512] [INFO ] [MAIN ] Update Progress: 598/2367 symbols completed. DD took 32.5s. Estimated remaining: 3h 44m 16s. Rate: 7.9 symbols/min [2025-10-20T18:15:53.535850] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:15:53.851418] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-09. [2025-10-20T18:15:54.781400] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:15:54.792636] [INFO ] [REMOTE ] DDD: Fetching primary data for 2025-10. [2025-10-20T18:15:55.364626] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:15:55.451343] [INFO ] [REMOTE ] DDL: Fetching primary data for 2025-10. [2025-10-20T18:15:56.806553] [INFO ] [REMOTE ] DDD: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:15:56.814411] [INFO ] [REMOTE ] DDD: Full fetch complete (73946 records). [2025-10-20T18:15:56.814557] [DEBUG] [VALIDATOR ] DDD: Validating primary data. [2025-10-20T18:15:57.131192] [INFO ] [REMOTE ] DDL: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:15:57.177936] [INFO ] [REMOTE ] DDL: Full fetch complete (53607 records). [2025-10-20T18:15:57.193553] [DEBUG] [VALIDATOR ] DDL: Validating primary data. [2025-10-20T18:15:57.381392] [DEBUG] [VALIDATOR ] DDD: All expected trading days are present. [2025-10-20T18:15:57.384016] [DEBUG] [VALIDATOR ] DDD: Primary data validation passed. [2025-10-20T18:15:57.385201] [INFO ] [MAIN ] DDD: Fetched primary data is valid. [2025-10-20T18:15:57.395850] [DEBUG] [LOCAL ] DDD: Saving primary data. [2025-10-20T18:15:57.451921] [DEBUG] [LOCAL ] DDD: Trimmed 73946 records to 50939 (keeping 200 days of data) [2025-10-20T18:15:57.772605] [DEBUG] [VALIDATOR ] DDL: All expected trading days are present. [2025-10-20T18:15:57.774337] [DEBUG] [VALIDATOR ] DDL: Primary data validation passed. [2025-10-20T18:15:57.775702] [INFO ] [MAIN ] DDL: Fetched primary data is valid. [2025-10-20T18:15:57.775773] [DEBUG] [LOCAL ] DDL: Saving primary data. [2025-10-20T18:15:57.783049] [DEBUG] [LOCAL ] DDL: Trimmed 53607 records to 37211 (keeping 200 days of data) [2025-10-20T18:15:58.165867] [DEBUG] [LOCAL ] DDL: Primary data saved (37211 records). [2025-10-20T18:15:58.167735] [INFO ] [MAIN ] DDL: Full history saved. [2025-10-20T18:15:58.173486] [INFO ] [MAIN ] Update Progress: 599/2367 symbols completed. DDL took 30.1s. Estimated remaining: 3h 44m 1s. Rate: 7.9 symbols/min [2025-10-20T18:15:58.186967] [DEBUG] [LOCAL ] DDD: Primary data saved (50939 records). [2025-10-20T18:15:58.187151] [INFO ] [MAIN ] DDD: Full history saved. [2025-10-20T18:15:58.187306] [INFO ] [MAIN ] Update Progress: 600/2367 symbols completed. DDD took 36.5s. Estimated remaining: 3h 43m 31s. Rate: 7.9 symbols/min [2025-10-20T18:15:58.814283] [INFO ] [MAIN ] DE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.891750] [DEBUG] [LOCAL ] DE: No existing data file found in primary storage. [2025-10-20T18:15:58.861716] [INFO ] [MAIN ] DEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.912330] [WARN ] [MAIN ] DE: No existing data found, performing full refresh instead. [2025-10-20T18:15:58.929965] [INFO ] [MAIN ] DDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.933747] [INFO ] [MAIN ] DEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:15:58.930609] [INFO ] [MAIN ] DE: Wiping existing primary data before full refresh. [2025-10-20T18:15:58.934088] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-10-20T18:15:58.934767] [DEBUG] [LOCAL ] DE: No primary data file found to delete. [2025-10-20T18:15:58.935050] [INFO ] [MAIN ] DE: Starting full primary refresh. [2025-10-20T18:15:59.044397] [DEBUG] [LOCAL ] DDT: Loaded 2106 existing records from primary storage. [2025-10-20T18:15:59.057336] [INFO ] [MAIN ] DDT: Loaded 2106 existing records. [2025-10-20T18:15:59.286962] [DEBUG] [LOCAL ] DEC: Loaded 30778 existing records from primary storage. [2025-10-20T18:15:59.389162] [DEBUG] [LOCAL ] DEA: Loaded 41023 existing records from primary storage. [2025-10-20T18:15:59.364996] [INFO ] [MAIN ] DEC: Loaded 30778 existing records. [2025-10-20T18:15:59.415054] [INFO ] [MAIN ] DEA: Loaded 41023 existing records. [2025-10-20T18:15:59.497723] [WARN ] [MAIN ] DDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:59.621681] [INFO ] [MAIN ] DDT: Wiping existing primary data before full refresh. [2025-10-20T18:15:59.726230] [INFO ] [LOCAL ] DDT: Starting deletion of primary data. [2025-10-20T18:15:59.808380] [WARN ] [MAIN ] DEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:15:59.813746] [DEBUG] [LOCAL ] DDT: Deleted primary data file. [2025-10-20T18:15:59.845817] [INFO ] [MAIN ] DEC: Wiping existing primary data before full refresh. [2025-10-20T18:15:59.853845] [INFO ] [REMOTE ] DE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:15:59.854016] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-10-20T18:15:59.859483] [INFO ] [MAIN ] DDT: Starting full primary refresh. [2025-10-20T18:15:59.859407] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-01. [2025-10-20T18:15:59.901832] [DEBUG] [LOCAL ] DEC: Deleted primary data file. [2025-10-20T18:15:59.923393] [INFO ] [MAIN ] DEC: Starting full primary refresh. [2025-10-20T18:16:00.026243] [WARN ] [MAIN ] DEA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:00.092504] [INFO ] [MAIN ] DEA: Wiping existing primary data before full refresh. [2025-10-20T18:16:00.096320] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-10-20T18:16:00.127044] [DEBUG] [LOCAL ] DEA: Deleted primary data file. [2025-10-20T18:16:00.136375] [INFO ] [MAIN ] DEA: Starting full primary refresh. [2025-10-20T18:16:00.815509] [INFO ] [REMOTE ] DDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:00.835693] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-01. [2025-10-20T18:16:00.963412] [INFO ] [REMOTE ] DEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:00.992615] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-01. [2025-10-20T18:16:01.033956] [INFO ] [REMOTE ] DEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:01.034635] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-01. [2025-10-20T18:16:01.637075] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T18:16:01.638156] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-02. [2025-10-20T18:16:01.936882] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-02 (0.30 seconds). [2025-10-20T18:16:01.938086] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-03. [2025-10-20T18:16:01.974617] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T18:16:01.984944] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-02. [2025-10-20T18:16:02.123996] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:16:02.129126] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-02. [2025-10-20T18:16:02.433951] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T18:16:02.467597] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:16:02.651408] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-04. [2025-10-20T18:16:03.257421] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:16:03.285387] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:16:03.451400] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-05. [2025-10-20T18:16:03.904973] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T18:16:03.994494] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T18:16:04.251391] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-03. [2025-10-20T18:16:05.051559] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-03. [2025-10-20T18:16:05.763536] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:16:05.851381] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-04. [2025-10-20T18:16:06.651373] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-02. [2025-10-20T18:16:06.981454] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:16:07.143626] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T18:16:07.451386] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-06. [2025-10-20T18:16:08.238705] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:16:08.251344] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-07. [2025-10-20T18:16:08.822096] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-02 (2.17 seconds). [2025-10-20T18:16:08.891018] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:16:09.051377] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-08. [2025-10-20T18:16:09.393966] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:16:09.851409] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-03. [2025-10-20T18:16:10.651388] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-04. [2025-10-20T18:16:11.451412] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-09. [2025-10-20T18:16:11.515422] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T18:16:11.978195] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:16:12.251399] [INFO ] [REMOTE ] DDT: Fetching primary data for 2025-10. [2025-10-20T18:16:12.488758] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T18:16:12.692241] [INFO ] [REMOTE ] DDT: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T18:16:12.693360] [INFO ] [REMOTE ] DDT: Full fetch complete (2816 records). [2025-10-20T18:16:12.693438] [DEBUG] [VALIDATOR ] DDT: Validating primary data. [2025-10-20T18:16:13.054227] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-04. [2025-10-20T18:16:13.060700] [DEBUG] [VALIDATOR ] DDT: All expected trading days are present. [2025-10-20T18:16:13.060811] [DEBUG] [VALIDATOR ] DDT: Primary data validation passed. [2025-10-20T18:16:13.060942] [INFO ] [MAIN ] DDT: Fetched primary data is valid. [2025-10-20T18:16:13.060984] [DEBUG] [LOCAL ] DDT: Saving primary data. [2025-10-20T18:16:13.061585] [DEBUG] [LOCAL ] DDT: Trimmed 2816 records to 1978 (keeping 200 days of data) [2025-10-20T18:16:13.074667] [DEBUG] [LOCAL ] DDT: Primary data saved (1978 records). [2025-10-20T18:16:13.074823] [INFO ] [MAIN ] DDT: Full history saved. [2025-10-20T18:16:13.074919] [INFO ] [MAIN ] Update Progress: 601/2367 symbols completed. DDT took 14.9s. Estimated remaining: 3h 43m 45s. Rate: 7.9 symbols/min [2025-10-20T18:16:13.246589] [INFO ] [MAIN ] DECK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:13.301355] [DEBUG] [LOCAL ] DECK: Loaded 54077 existing records from primary storage. [2025-10-20T18:16:13.301752] [INFO ] [MAIN ] DECK: Loaded 54077 existing records. [2025-10-20T18:16:13.479957] [WARN ] [MAIN ] DECK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:13.480175] [INFO ] [MAIN ] DECK: Wiping existing primary data before full refresh. [2025-10-20T18:16:13.480223] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-10-20T18:16:13.480781] [DEBUG] [LOCAL ] DECK: Deleted primary data file. [2025-10-20T18:16:13.480845] [INFO ] [MAIN ] DECK: Starting full primary refresh. [2025-10-20T18:16:13.836768] [INFO ] [REMOTE ] DECK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:13.851338] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-01. [2025-10-20T18:16:14.651462] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-05. [2025-10-20T18:16:14.900737] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:16:15.205507] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T18:16:15.451475] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-05. [2025-10-20T18:16:15.924512] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:16:16.251503] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-06. [2025-10-20T18:16:17.051389] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-05. [2025-10-20T18:16:17.245578] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:16:17.303870] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:16:17.851405] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-06. [2025-10-20T18:16:18.558040] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T18:16:18.651454] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-06. [2025-10-20T18:16:19.451366] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-07. [2025-10-20T18:16:19.567089] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:16:19.876006] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:16:20.251468] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-07. [2025-10-20T18:16:20.740935] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T18:16:21.051472] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-07. [2025-10-20T18:16:21.746967] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:16:21.851465] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-08. [2025-10-20T18:16:22.618512] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:16:22.651367] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-02. [2025-10-20T18:16:23.012198] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T18:16:23.451475] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-09. [2025-10-20T18:16:24.251467] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-08. [2025-10-20T18:16:24.874272] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T18:16:24.973697] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:16:25.051399] [INFO ] [REMOTE ] DEA: Fetching primary data for 2025-10. [2025-10-20T18:16:25.851436] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-03. [2025-10-20T18:16:25.969931] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:16:26.469803] [INFO ] [REMOTE ] DEA: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:16:26.476434] [INFO ] [REMOTE ] DEA: Full fetch complete (61600 records). [2025-10-20T18:16:26.476615] [DEBUG] [VALIDATOR ] DEA: Validating primary data. [2025-10-20T18:16:26.477761] [FATAL] [VALIDATOR ] DEA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:26.477869] [FATAL] [MAIN ] DEA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:26.477952] [WARN ] [MAIN ] DEA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:26.478044] [INFO ] [LOCAL ] DEA: Starting deletion of primary data. [2025-10-20T18:16:26.478141] [DEBUG] [LOCAL ] DEA: No primary data file found to delete. [2025-10-20T18:16:26.478200] [INFO ] [LOCAL ] DEA: Starting deletion of derivative data. [2025-10-20T18:16:26.478410] [DEBUG] [LOCAL ] DEA: Deleted derivative data file. [2025-10-20T18:16:26.478539] [INFO ] [MAIN ] Update Progress: 602/2367 symbols completed. DEA took 28.3s. Estimated remaining: 3h 43m 54s. Rate: 7.9 symbols/min [2025-10-20T18:16:26.658089] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-09. [2025-10-20T18:16:26.677158] [INFO ] [MAIN ] DEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:26.736800] [DEBUG] [LOCAL ] DEI: Loaded 49027 existing records from primary storage. [2025-10-20T18:16:26.737461] [INFO ] [MAIN ] DEI: Loaded 49027 existing records. [2025-10-20T18:16:26.936490] [WARN ] [MAIN ] DEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:26.936853] [INFO ] [MAIN ] DEI: Wiping existing primary data before full refresh. [2025-10-20T18:16:26.936913] [INFO ] [LOCAL ] DEI: Starting deletion of primary data. [2025-10-20T18:16:26.937165] [DEBUG] [LOCAL ] DEI: Deleted primary data file. [2025-10-20T18:16:26.937223] [INFO ] [MAIN ] DEI: Starting full primary refresh. [2025-10-20T18:16:27.315081] [INFO ] [REMOTE ] DEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:27.451368] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-08. [2025-10-20T18:16:27.626723] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:16:28.256600] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-04. [2025-10-20T18:16:28.275702] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:16:28.677583] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T18:16:29.051481] [INFO ] [REMOTE ] DE: Fetching primary data for 2025-10. [2025-10-20T18:16:29.851505] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-09. [2025-10-20T18:16:30.432957] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T18:16:30.651428] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-05. [2025-10-20T18:16:30.805859] [INFO ] [REMOTE ] DE: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:16:30.814801] [INFO ] [REMOTE ] DE: Full fetch complete (69566 records). [2025-10-20T18:16:30.815175] [DEBUG] [VALIDATOR ] DE: Validating primary data. [2025-10-20T18:16:30.816356] [FATAL] [VALIDATOR ] DE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:30.816476] [FATAL] [MAIN ] DE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:30.816537] [WARN ] [MAIN ] DE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:30.816613] [INFO ] [LOCAL ] DE: Starting deletion of primary data. [2025-10-20T18:16:30.816724] [DEBUG] [LOCAL ] DE: No primary data file found to delete. [2025-10-20T18:16:30.816774] [INFO ] [LOCAL ] DE: Starting deletion of derivative data. [2025-10-20T18:16:30.816818] [DEBUG] [LOCAL ] DE: No derivative data file found to delete. [2025-10-20T18:16:30.816890] [INFO ] [MAIN ] Update Progress: 603/2367 symbols completed. DE took 32.6s. Estimated remaining: 3h 43m 37s. Rate: 7.9 symbols/min [2025-10-20T18:16:31.020077] [INFO ] [MAIN ] DELL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:31.023594] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:16:31.093139] [DEBUG] [LOCAL ] DELL: Loaded 54516 existing records from primary storage. [2025-10-20T18:16:31.093568] [INFO ] [MAIN ] DELL: Loaded 54516 existing records. [2025-10-20T18:16:31.282566] [WARN ] [MAIN ] DELL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:31.282928] [INFO ] [MAIN ] DELL: Wiping existing primary data before full refresh. [2025-10-20T18:16:31.282982] [INFO ] [LOCAL ] DELL: Starting deletion of primary data. [2025-10-20T18:16:31.283827] [DEBUG] [LOCAL ] DELL: Deleted primary data file. [2025-10-20T18:16:31.283887] [INFO ] [MAIN ] DELL: Starting full primary refresh. [2025-10-20T18:16:31.465546] [INFO ] [REMOTE ] DEC: Fetching primary data for 2025-10. [2025-10-20T18:16:31.684687] [INFO ] [REMOTE ] DELL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:32.251417] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-01. [2025-10-20T18:16:32.320824] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:16:32.845441] [INFO ] [REMOTE ] DEC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:16:32.851027] [INFO ] [REMOTE ] DEC: Full fetch complete (44243 records). [2025-10-20T18:16:32.851177] [DEBUG] [VALIDATOR ] DEC: Validating primary data. [2025-10-20T18:16:32.852096] [FATAL] [VALIDATOR ] DEC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:32.852224] [FATAL] [MAIN ] DEC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:32.852296] [WARN ] [MAIN ] DEC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:32.852359] [INFO ] [LOCAL ] DEC: Starting deletion of primary data. [2025-10-20T18:16:32.852464] [DEBUG] [LOCAL ] DEC: No primary data file found to delete. [2025-10-20T18:16:32.852536] [INFO ] [LOCAL ] DEC: Starting deletion of derivative data. [2025-10-20T18:16:32.852739] [DEBUG] [LOCAL ] DEC: Deleted derivative data file. [2025-10-20T18:16:32.852893] [INFO ] [MAIN ] Update Progress: 604/2367 symbols completed. DEC took 34.7s. Estimated remaining: 3h 43m 13s. Rate: 7.9 symbols/min [2025-10-20T18:16:33.046495] [INFO ] [MAIN ] DEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:33.051426] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-06. [2025-10-20T18:16:33.102059] [DEBUG] [LOCAL ] DEO: Loaded 49969 existing records from primary storage. [2025-10-20T18:16:33.102507] [INFO ] [MAIN ] DEO: Loaded 49969 existing records. [2025-10-20T18:16:33.288308] [WARN ] [MAIN ] DEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:33.288616] [INFO ] [MAIN ] DEO: Wiping existing primary data before full refresh. [2025-10-20T18:16:33.288702] [INFO ] [LOCAL ] DEO: Starting deletion of primary data. [2025-10-20T18:16:33.289320] [DEBUG] [LOCAL ] DEO: Deleted primary data file. [2025-10-20T18:16:33.289377] [INFO ] [MAIN ] DEO: Starting full primary refresh. [2025-10-20T18:16:33.674395] [INFO ] [REMOTE ] DEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:33.851484] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-01. [2025-10-20T18:16:34.651410] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-01. [2025-10-20T18:16:34.767984] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-01 (2.52 seconds). [2025-10-20T18:16:34.817174] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:16:35.451357] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-07. [2025-10-20T18:16:35.689943] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:16:36.251485] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-02. [2025-10-20T18:16:36.615366] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:16:37.051503] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-02. [2025-10-20T18:16:37.558134] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:16:37.784899] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:16:37.851391] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-08. [2025-10-20T18:16:38.560207] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:16:38.651461] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-03. [2025-10-20T18:16:39.305274] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:16:39.451450] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-03. [2025-10-20T18:16:40.142718] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:16:40.251419] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-09. [2025-10-20T18:16:40.944187] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:16:41.051431] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-02. [2025-10-20T18:16:41.851391] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-04. [2025-10-20T18:16:42.043902] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:16:42.651372] [INFO ] [REMOTE ] DECK: Fetching primary data for 2025-10. [2025-10-20T18:16:42.983264] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T18:16:43.451350] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-03. [2025-10-20T18:16:43.709392] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T18:16:44.251386] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-04. [2025-10-20T18:16:44.842206] [INFO ] [REMOTE ] DECK: Incremental fetch complete for 2025-10 (2.19 seconds). [2025-10-20T18:16:44.849779] [INFO ] [REMOTE ] DECK: Full fetch complete (76733 records). [2025-10-20T18:16:44.849956] [DEBUG] [VALIDATOR ] DECK: Validating primary data. [2025-10-20T18:16:44.850813] [FATAL] [VALIDATOR ] DECK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:16:44.850905] [FATAL] [MAIN ] DECK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:16:44.850949] [WARN ] [MAIN ] DECK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:16:44.851009] [INFO ] [LOCAL ] DECK: Starting deletion of primary data. [2025-10-20T18:16:44.851106] [DEBUG] [LOCAL ] DECK: No primary data file found to delete. [2025-10-20T18:16:44.851162] [INFO ] [LOCAL ] DECK: Starting deletion of derivative data. [2025-10-20T18:16:44.851323] [DEBUG] [LOCAL ] DECK: Deleted derivative data file. [2025-10-20T18:16:44.851422] [INFO ] [MAIN ] Update Progress: 605/2367 symbols completed. DECK took 31.8s. Estimated remaining: 3h 43m 18s. Rate: 7.9 symbols/min [2025-10-20T18:16:45.021531] [INFO ] [MAIN ] DFH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:16:45.052007] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-05. [2025-10-20T18:16:45.052605] [DEBUG] [LOCAL ] DFH: Loaded 32567 existing records from primary storage. [2025-10-20T18:16:45.053003] [INFO ] [MAIN ] DFH: Loaded 32567 existing records. [2025-10-20T18:16:45.234893] [WARN ] [MAIN ] DFH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:16:45.235175] [INFO ] [MAIN ] DFH: Wiping existing primary data before full refresh. [2025-10-20T18:16:45.235227] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-10-20T18:16:45.235612] [DEBUG] [LOCAL ] DFH: Deleted primary data file. [2025-10-20T18:16:45.235686] [INFO ] [MAIN ] DFH: Starting full primary refresh. [2025-10-20T18:16:45.610245] [INFO ] [REMOTE ] DFH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:16:45.719508] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-10-20T18:16:45.851424] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-04. [2025-10-20T18:16:46.062858] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:16:46.651470] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-01. [2025-10-20T18:16:46.853324] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T18:16:47.451409] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-06. [2025-10-20T18:16:47.660502] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:16:48.095913] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:16:48.251393] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-05. [2025-10-20T18:16:48.900956] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:16:49.051403] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-02. [2025-10-20T18:16:49.851378] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-07. [2025-10-20T18:16:49.999622] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:16:50.220466] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T18:16:50.651375] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-06. [2025-10-20T18:16:51.451382] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-05. [2025-10-20T18:16:51.652796] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:16:52.251433] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-08. [2025-10-20T18:16:52.955704] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-10-20T18:16:53.028042] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:16:53.051354] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-06. [2025-10-20T18:16:53.851375] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-07. [2025-10-20T18:16:54.065900] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:16:54.256840] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:16:54.651378] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-03. [2025-10-20T18:16:55.451380] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-09. [2025-10-20T18:16:56.117559] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:16:56.228384] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T18:16:56.251334] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-08. [2025-10-20T18:16:57.051346] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-04. [2025-10-20T18:16:57.083604] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:16:57.851384] [INFO ] [REMOTE ] DEO: Fetching primary data for 2025-10. [2025-10-20T18:16:58.152276] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T18:16:58.556615] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-08 (2.31 seconds). [2025-10-20T18:16:58.651385] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-05. [2025-10-20T18:16:59.451380] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-07. [2025-10-20T18:16:59.771879] [INFO ] [REMOTE ] DEO: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T18:16:59.779715] [INFO ] [REMOTE ] DEO: Full fetch complete (71759 records). [2025-10-20T18:16:59.779861] [DEBUG] [VALIDATOR ] DEO: Validating primary data. [2025-10-20T18:17:00.121419] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T18:17:00.224966] [DEBUG] [VALIDATOR ] DEO: All expected trading days are present. [2025-10-20T18:17:00.225055] [DEBUG] [VALIDATOR ] DEO: Primary data validation passed. [2025-10-20T18:17:00.225879] [INFO ] [MAIN ] DEO: Fetched primary data is valid. [2025-10-20T18:17:00.225941] [DEBUG] [LOCAL ] DEO: Saving primary data. [2025-10-20T18:17:00.227294] [DEBUG] [LOCAL ] DEO: Trimmed 71759 records to 49256 (keeping 200 days of data) [2025-10-20T18:17:00.283416] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-06. [2025-10-20T18:17:00.534310] [DEBUG] [LOCAL ] DEO: Primary data saved (49256 records). [2025-10-20T18:17:00.534438] [INFO ] [MAIN ] DEO: Full history saved. [2025-10-20T18:17:00.534510] [INFO ] [MAIN ] Update Progress: 606/2367 symbols completed. DEO took 27.7s. Estimated remaining: 3h 43m 34s. Rate: 7.9 symbols/min [2025-10-20T18:17:00.706109] [INFO ] [MAIN ] DFIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:00.728293] [DEBUG] [LOCAL ] DFIN: Loaded 22974 existing records from primary storage. [2025-10-20T18:17:00.728551] [INFO ] [MAIN ] DFIN: Loaded 22974 existing records. [2025-10-20T18:17:00.906276] [WARN ] [MAIN ] DFIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:00.906463] [INFO ] [MAIN ] DFIN: Wiping existing primary data before full refresh. [2025-10-20T18:17:00.906511] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-10-20T18:17:00.906719] [DEBUG] [LOCAL ] DFIN: Deleted primary data file. [2025-10-20T18:17:00.906779] [INFO ] [MAIN ] DFIN: Starting full primary refresh. [2025-10-20T18:17:01.056506] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-09. [2025-10-20T18:17:01.291831] [INFO ] [REMOTE ] DFIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:01.319390] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:17:01.851419] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:17:01.851494] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-08. [2025-10-20T18:17:02.651409] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-07. [2025-10-20T18:17:03.224920] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:17:03.445604] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:17:03.455348] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-01. [2025-10-20T18:17:03.707990] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:17:04.251362] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-09. [2025-10-20T18:17:04.493296] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:17:05.051416] [INFO ] [REMOTE ] DELL: Fetching primary data for 2025-10. [2025-10-20T18:17:05.851380] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-08. [2025-10-20T18:17:05.914113] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:17:06.651467] [INFO ] [REMOTE ] DEI: Fetching primary data for 2025-10. [2025-10-20T18:17:07.300395] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:17:07.451363] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-09. [2025-10-20T18:17:07.635090] [INFO ] [REMOTE ] DELL: Incremental fetch complete for 2025-10 (2.58 seconds). [2025-10-20T18:17:07.643283] [INFO ] [REMOTE ] DELL: Full fetch complete (77915 records). [2025-10-20T18:17:07.643438] [DEBUG] [VALIDATOR ] DELL: Validating primary data. [2025-10-20T18:17:08.048999] [DEBUG] [VALIDATOR ] DELL: All expected trading days are present. [2025-10-20T18:17:08.049135] [DEBUG] [VALIDATOR ] DELL: Primary data validation passed. [2025-10-20T18:17:08.050277] [INFO ] [MAIN ] DELL: Fetched primary data is valid. [2025-10-20T18:17:08.050464] [DEBUG] [LOCAL ] DELL: Saving primary data. [2025-10-20T18:17:08.052088] [DEBUG] [LOCAL ] DELL: Trimmed 77915 records to 53736 (keeping 200 days of data) [2025-10-20T18:17:08.210786] [INFO ] [REMOTE ] DEI: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T18:17:08.220957] [INFO ] [REMOTE ] DEI: Full fetch complete (69906 records). [2025-10-20T18:17:08.226479] [DEBUG] [VALIDATOR ] DEI: Validating primary data. [2025-10-20T18:17:08.332844] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-02. [2025-10-20T18:17:08.838888] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:17:08.867405] [DEBUG] [LOCAL ] DELL: Primary data saved (53736 records). [2025-10-20T18:17:08.872970] [INFO ] [MAIN ] DELL: Full history saved. [2025-10-20T18:17:08.888305] [INFO ] [MAIN ] Update Progress: 607/2367 symbols completed. DELL took 38.1s. Estimated remaining: 3h 43m 29s. Rate: 7.9 symbols/min [2025-10-20T18:17:09.055759] [INFO ] [REMOTE ] DFH: Fetching primary data for 2025-10. [2025-10-20T18:17:09.056032] [DEBUG] [VALIDATOR ] DEI: All expected trading days are present. [2025-10-20T18:17:09.059615] [DEBUG] [VALIDATOR ] DEI: Primary data validation passed. [2025-10-20T18:17:09.066434] [INFO ] [MAIN ] DEI: Fetched primary data is valid. [2025-10-20T18:17:09.077300] [DEBUG] [LOCAL ] DEI: Saving primary data. [2025-10-20T18:17:09.135563] [INFO ] [MAIN ] DFP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:09.136236] [DEBUG] [LOCAL ] DFP: No existing data file found in primary storage. [2025-10-20T18:17:09.136404] [WARN ] [MAIN ] DFP: No existing data found, performing full refresh instead. [2025-10-20T18:17:09.136710] [INFO ] [MAIN ] DFP: Wiping existing primary data before full refresh. [2025-10-20T18:17:09.136936] [INFO ] [LOCAL ] DFP: Starting deletion of primary data. [2025-10-20T18:17:09.136998] [DEBUG] [LOCAL ] DFP: No primary data file found to delete. [2025-10-20T18:17:09.137033] [INFO ] [MAIN ] DFP: Starting full primary refresh. [2025-10-20T18:17:09.168690] [DEBUG] [LOCAL ] DEI: Trimmed 69906 records to 48313 (keeping 200 days of data) [2025-10-20T18:17:09.624947] [INFO ] [REMOTE ] DFP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:09.741828] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T18:17:09.821529] [DEBUG] [LOCAL ] DEI: Primary data saved (48313 records). [2025-10-20T18:17:09.821721] [INFO ] [MAIN ] DEI: Full history saved. [2025-10-20T18:17:09.821816] [INFO ] [MAIN ] Update Progress: 608/2367 symbols completed. DEI took 43.3s. Estimated remaining: 3h 43m 2s. Rate: 7.9 symbols/min [2025-10-20T18:17:09.852889] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-03. [2025-10-20T18:17:09.991838] [INFO ] [MAIN ] DG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:10.047300] [DEBUG] [LOCAL ] DG: Loaded 54049 existing records from primary storage. [2025-10-20T18:17:10.047701] [INFO ] [MAIN ] DG: Loaded 54049 existing records. [2025-10-20T18:17:10.226294] [WARN ] [MAIN ] DG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:10.226557] [INFO ] [MAIN ] DG: Wiping existing primary data before full refresh. [2025-10-20T18:17:10.226828] [INFO ] [LOCAL ] DG: Starting deletion of primary data. [2025-10-20T18:17:10.227694] [DEBUG] [LOCAL ] DG: Deleted primary data file. [2025-10-20T18:17:10.227750] [INFO ] [MAIN ] DG: Starting full primary refresh. [2025-10-20T18:17:10.340179] [INFO ] [REMOTE ] DFH: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:17:10.365565] [INFO ] [REMOTE ] DFH: Full fetch complete (47556 records). [2025-10-20T18:17:10.382751] [DEBUG] [VALIDATOR ] DFH: Validating primary data. [2025-10-20T18:17:10.419260] [FATAL] [VALIDATOR ] DFH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:10.486449] [FATAL] [MAIN ] DFH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:10.521143] [WARN ] [MAIN ] DFH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:10.523591] [INFO ] [LOCAL ] DFH: Starting deletion of primary data. [2025-10-20T18:17:10.534456] [DEBUG] [LOCAL ] DFH: No primary data file found to delete. [2025-10-20T18:17:10.544909] [INFO ] [LOCAL ] DFH: Starting deletion of derivative data. [2025-10-20T18:17:10.575457] [DEBUG] [LOCAL ] DFH: Deleted derivative data file. [2025-10-20T18:17:10.581679] [INFO ] [MAIN ] Update Progress: 609/2367 symbols completed. DFH took 25.7s. Estimated remaining: 3h 42m 34s. Rate: 7.9 symbols/min [2025-10-20T18:17:10.607817] [INFO ] [REMOTE ] DG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:10.658009] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-01. [2025-10-20T18:17:10.781584] [INFO ] [MAIN ] DGX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:10.833686] [DEBUG] [LOCAL ] DGX: Loaded 45306 existing records from primary storage. [2025-10-20T18:17:10.834349] [INFO ] [MAIN ] DGX: Loaded 45306 existing records. [2025-10-20T18:17:11.017752] [WARN ] [MAIN ] DGX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:11.017954] [INFO ] [MAIN ] DGX: Wiping existing primary data before full refresh. [2025-10-20T18:17:11.018029] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-10-20T18:17:11.018340] [DEBUG] [LOCAL ] DGX: Deleted primary data file. [2025-10-20T18:17:11.018397] [INFO ] [MAIN ] DGX: Starting full primary refresh. [2025-10-20T18:17:11.339934] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:17:11.409904] [INFO ] [REMOTE ] DGX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:11.451342] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-04. [2025-10-20T18:17:11.602973] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:17:12.251461] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-01. [2025-10-20T18:17:12.632803] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:17:13.051397] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-02. [2025-10-20T18:17:13.418841] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:17:13.579337] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:17:13.851424] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-05. [2025-10-20T18:17:14.651423] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-02. [2025-10-20T18:17:15.211461] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T18:17:15.451485] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-06. [2025-10-20T18:17:16.251485] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-01. [2025-10-20T18:17:16.679997] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:17:16.717272] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T18:17:17.051443] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-03. [2025-10-20T18:17:17.851374] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-03. [2025-10-20T18:17:17.884835] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:17:18.110931] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T18:17:18.651453] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-04. [2025-10-20T18:17:19.203762] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:17:19.451404] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-02. [2025-10-20T18:17:19.593746] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:17:20.251454] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-05. [2025-10-20T18:17:20.778045] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:17:21.051418] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-04. [2025-10-20T18:17:21.202098] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:17:21.851391] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-07. [2025-10-20T18:17:22.651353] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-03. [2025-10-20T18:17:22.844498] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T18:17:22.950208] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:17:23.451418] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-05. [2025-10-20T18:17:24.251385] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-08. [2025-10-20T18:17:24.509744] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T18:17:25.051346] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-04. [2025-10-20T18:17:25.088606] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:17:25.284954] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:17:25.851415] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-06. [2025-10-20T18:17:26.554071] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T18:17:26.651358] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-07. [2025-10-20T18:17:26.926020] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:17:27.451410] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-06. [2025-10-20T18:17:27.467543] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:17:28.251372] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-09. [2025-10-20T18:17:29.051786] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-05. [2025-10-20T18:17:29.068604] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:17:29.227365] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T18:17:29.851396] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-07. [2025-10-20T18:17:30.651601] [INFO ] [REMOTE ] DFIN: Fetching primary data for 2025-10. [2025-10-20T18:17:31.009879] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:17:31.442627] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T18:17:31.452209] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-06. [2025-10-20T18:17:31.781167] [INFO ] [REMOTE ] DFIN: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T18:17:31.785305] [INFO ] [REMOTE ] DFIN: Full fetch complete (33166 records). [2025-10-20T18:17:31.785481] [DEBUG] [VALIDATOR ] DFIN: Validating primary data. [2025-10-20T18:17:31.786229] [FATAL] [VALIDATOR ] DFIN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:31.786322] [FATAL] [MAIN ] DFIN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:31.786365] [WARN ] [MAIN ] DFIN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:31.786402] [INFO ] [LOCAL ] DFIN: Starting deletion of primary data. [2025-10-20T18:17:31.786458] [DEBUG] [LOCAL ] DFIN: No primary data file found to delete. [2025-10-20T18:17:31.786493] [INFO ] [LOCAL ] DFIN: Starting deletion of derivative data. [2025-10-20T18:17:31.786664] [DEBUG] [LOCAL ] DFIN: Deleted derivative data file. [2025-10-20T18:17:31.786757] [INFO ] [MAIN ] Update Progress: 610/2367 symbols completed. DFIN took 31.3s. Estimated remaining: 3h 43m 6s. Rate: 7.9 symbols/min [2025-10-20T18:17:31.960985] [INFO ] [MAIN ] DHF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:31.982977] [DEBUG] [LOCAL ] DHF: Loaded 21251 existing records from primary storage. [2025-10-20T18:17:31.983301] [INFO ] [MAIN ] DHF: Loaded 21251 existing records. [2025-10-20T18:17:32.160814] [WARN ] [MAIN ] DHF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:32.160956] [INFO ] [MAIN ] DHF: Wiping existing primary data before full refresh. [2025-10-20T18:17:32.160998] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-10-20T18:17:32.161146] [DEBUG] [LOCAL ] DHF: Deleted primary data file. [2025-10-20T18:17:32.161185] [INFO ] [MAIN ] DHF: Starting full primary refresh. [2025-10-20T18:17:32.257854] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-08. [2025-10-20T18:17:32.518154] [INFO ] [REMOTE ] DHF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:33.051357] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-08. [2025-10-20T18:17:33.142844] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T18:17:33.222303] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:17:33.851424] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-09. [2025-10-20T18:17:34.494131] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:17:34.651379] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-09. [2025-10-20T18:17:34.676630] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:17:35.451399] [INFO ] [REMOTE ] DFP: Fetching primary data for 2025-10. [2025-10-20T18:17:36.005545] [INFO ] [REMOTE ] DFP: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T18:17:36.007581] [INFO ] [REMOTE ] DFP: Full fetch complete (15719 records). [2025-10-20T18:17:36.007724] [DEBUG] [VALIDATOR ] DFP: Validating primary data. [2025-10-20T18:17:36.157004] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:17:36.256576] [INFO ] [REMOTE ] DGX: Fetching primary data for 2025-10. [2025-10-20T18:17:36.446583] [DEBUG] [VALIDATOR ] DFP: All expected trading days are present. [2025-10-20T18:17:36.446688] [DEBUG] [VALIDATOR ] DFP: Primary data validation passed. [2025-10-20T18:17:36.447065] [INFO ] [MAIN ] DFP: Fetched primary data is valid. [2025-10-20T18:17:36.447108] [DEBUG] [LOCAL ] DFP: Saving primary data. [2025-10-20T18:17:36.447607] [DEBUG] [LOCAL ] DFP: Trimmed 15719 records to 11121 (keeping 200 days of data) [2025-10-20T18:17:36.515918] [DEBUG] [LOCAL ] DFP: Primary data saved (11121 records). [2025-10-20T18:17:36.516050] [INFO ] [MAIN ] DFP: Full history saved. [2025-10-20T18:17:36.516119] [INFO ] [MAIN ] Update Progress: 611/2367 symbols completed. DFP took 27.6s. Estimated remaining: 3h 42m 50s. Rate: 7.9 symbols/min [2025-10-20T18:17:36.698265] [INFO ] [MAIN ] DHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:36.698459] [DEBUG] [LOCAL ] DHI: No existing data file found in primary storage. [2025-10-20T18:17:36.698497] [WARN ] [MAIN ] DHI: No existing data found, performing full refresh instead. [2025-10-20T18:17:36.698607] [INFO ] [MAIN ] DHI: Wiping existing primary data before full refresh. [2025-10-20T18:17:36.698674] [INFO ] [LOCAL ] DHI: Starting deletion of primary data. [2025-10-20T18:17:36.698739] [DEBUG] [LOCAL ] DHI: No primary data file found to delete. [2025-10-20T18:17:36.698773] [INFO ] [MAIN ] DHI: Starting full primary refresh. [2025-10-20T18:17:37.059734] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-07. [2025-10-20T18:17:37.064627] [INFO ] [REMOTE ] DHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:37.851347] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-01. [2025-10-20T18:17:37.950148] [INFO ] [REMOTE ] DGX: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T18:17:37.957130] [INFO ] [REMOTE ] DGX: Full fetch complete (65083 records). [2025-10-20T18:17:37.957342] [DEBUG] [VALIDATOR ] DGX: Validating primary data. [2025-10-20T18:17:37.958174] [FATAL] [VALIDATOR ] DGX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:17:37.958261] [FATAL] [MAIN ] DGX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:17:37.958303] [WARN ] [MAIN ] DGX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:17:37.958342] [INFO ] [LOCAL ] DGX: Starting deletion of primary data. [2025-10-20T18:17:37.958398] [DEBUG] [LOCAL ] DGX: No primary data file found to delete. [2025-10-20T18:17:37.958432] [INFO ] [LOCAL ] DGX: Starting deletion of derivative data. [2025-10-20T18:17:37.958583] [DEBUG] [LOCAL ] DGX: Deleted derivative data file. [2025-10-20T18:17:37.959404] [INFO ] [MAIN ] Update Progress: 612/2367 symbols completed. DGX took 27.4s. Estimated remaining: 3h 42m 25s. Rate: 7.9 symbols/min [2025-10-20T18:17:38.134697] [INFO ] [MAIN ] DHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:38.134865] [DEBUG] [LOCAL ] DHR: No existing data file found in primary storage. [2025-10-20T18:17:38.134905] [WARN ] [MAIN ] DHR: No existing data found, performing full refresh instead. [2025-10-20T18:17:38.135013] [INFO ] [MAIN ] DHR: Wiping existing primary data before full refresh. [2025-10-20T18:17:38.135056] [INFO ] [LOCAL ] DHR: Starting deletion of primary data. [2025-10-20T18:17:38.135100] [DEBUG] [LOCAL ] DHR: No primary data file found to delete. [2025-10-20T18:17:38.135134] [INFO ] [MAIN ] DHR: Starting full primary refresh. [2025-10-20T18:17:38.531526] [INFO ] [REMOTE ] DHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:38.596477] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:17:38.651384] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-08. [2025-10-20T18:17:39.451432] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-01. [2025-10-20T18:17:39.532462] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:17:40.251450] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-02. [2025-10-20T18:17:40.476094] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:17:40.518102] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:17:41.051407] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-01. [2025-10-20T18:17:41.851354] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-09. [2025-10-20T18:17:42.129134] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T18:17:42.651400] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-02. [2025-10-20T18:17:42.784896] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:17:43.451349] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-03. [2025-10-20T18:17:43.690131] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:17:43.776698] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T18:17:44.251348] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-03. [2025-10-20T18:17:44.973167] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:17:45.051353] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-02. [2025-10-20T18:17:45.096732] [INFO ] [REMOTE ] DHF: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:17:45.851454] [INFO ] [REMOTE ] DG: Fetching primary data for 2025-10. [2025-10-20T18:17:46.651365] [INFO ] [REMOTE ] DHF: Fetching primary data for 2025-04. [2025-10-20T18:17:46.714023] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:17:46.751591] [INFO ] [REMOTE ] DG: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T18:17:46.759836] [DEBUG] [REMOTE ] DHF: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T18:17:46.760985] [WARN ] [REMOTE ] DHF: No data returned for month 2025-04 [2025-10-20T18:17:46.761186] [INFO ] [REMOTE ] DG: Full fetch complete (77343 records). [2025-10-20T18:17:46.761263] [FATAL] [REMOTE ] DHF: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:17:46.761595] [DEBUG] [VALIDATOR ] DG: Validating primary data. [2025-10-20T18:17:46.761754] [FATAL] [MAIN ] DHF: Failed to fetch primary data. [2025-10-20T18:17:46.761938] [WARN ] [MAIN ] DHF: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:17:46.762184] [INFO ] [LOCAL ] DHF: Starting deletion of primary data. [2025-10-20T18:17:46.762335] [DEBUG] [LOCAL ] DHF: No primary data file found to delete. [2025-10-20T18:17:46.762383] [INFO ] [LOCAL ] DHF: Starting deletion of derivative data. [2025-10-20T18:17:46.794882] [DEBUG] [LOCAL ] DHF: Deleted derivative data file. [2025-10-20T18:17:46.839275] [INFO ] [MAIN ] Update Progress: 613/2367 symbols completed. DHF took 15.1s. Estimated remaining: 3h 42m 21s. Rate: 7.9 symbols/min [2025-10-20T18:17:47.263367] [INFO ] [MAIN ] DHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:47.424837] [DEBUG] [VALIDATOR ] DG: All expected trading days are present. [2025-10-20T18:17:47.424960] [DEBUG] [VALIDATOR ] DG: Primary data validation passed. [2025-10-20T18:17:47.425898] [INFO ] [MAIN ] DG: Fetched primary data is valid. [2025-10-20T18:17:47.426047] [DEBUG] [LOCAL ] DG: Saving primary data. [2025-10-20T18:17:47.438449] [DEBUG] [LOCAL ] DG: Trimmed 77343 records to 53269 (keeping 200 days of data) [2025-10-20T18:17:47.499695] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-04. [2025-10-20T18:17:47.583771] [DEBUG] [LOCAL ] DHT: Loaded 48665 existing records from primary storage. [2025-10-20T18:17:47.601801] [INFO ] [MAIN ] DHT: Loaded 48665 existing records. [2025-10-20T18:17:47.931309] [WARN ] [MAIN ] DHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:17:47.936313] [INFO ] [MAIN ] DHT: Wiping existing primary data before full refresh. [2025-10-20T18:17:47.936398] [INFO ] [LOCAL ] DHT: Starting deletion of primary data. [2025-10-20T18:17:47.943822] [DEBUG] [LOCAL ] DHT: Deleted primary data file. [2025-10-20T18:17:47.943895] [INFO ] [MAIN ] DHT: Starting full primary refresh. [2025-10-20T18:17:48.304148] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-03. [2025-10-20T18:17:48.487730] [INFO ] [REMOTE ] DHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:48.487857] [DEBUG] [LOCAL ] DG: Primary data saved (53269 records). [2025-10-20T18:17:48.488476] [INFO ] [MAIN ] DG: Full history saved. [2025-10-20T18:17:48.488616] [INFO ] [MAIN ] Update Progress: 614/2367 symbols completed. DG took 38.7s. Estimated remaining: 3h 41m 56s. Rate: 7.9 symbols/min [2025-10-20T18:17:48.674504] [INFO ] [MAIN ] DHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:17:48.674759] [DEBUG] [LOCAL ] DHX: No existing data file found in primary storage. [2025-10-20T18:17:48.674810] [WARN ] [MAIN ] DHX: No existing data found, performing full refresh instead. [2025-10-20T18:17:48.674932] [INFO ] [MAIN ] DHX: Wiping existing primary data before full refresh. [2025-10-20T18:17:48.674975] [INFO ] [LOCAL ] DHX: Starting deletion of primary data. [2025-10-20T18:17:48.675017] [DEBUG] [LOCAL ] DHX: No primary data file found to delete. [2025-10-20T18:17:48.675050] [INFO ] [MAIN ] DHX: Starting full primary refresh. [2025-10-20T18:17:49.052897] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-01. [2025-10-20T18:17:49.052760] [INFO ] [REMOTE ] DHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:17:49.827532] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-04 (2.33 seconds). [2025-10-20T18:17:49.851377] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-05. [2025-10-20T18:17:49.922433] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:17:50.555308] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:17:50.651336] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-02. [2025-10-20T18:17:51.451378] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-01. [2025-10-20T18:17:51.526393] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:17:52.251554] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-06. [2025-10-20T18:17:52.284660] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:17:52.336338] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T18:17:53.051368] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-03. [2025-10-20T18:17:53.851346] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-04. [2025-10-20T18:17:53.992410] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:17:54.651349] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-07. [2025-10-20T18:17:54.694522] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:17:55.451438] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-04. [2025-10-20T18:17:55.950281] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T18:17:56.251362] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-02. [2025-10-20T18:17:56.667552] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T18:17:57.051423] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-05. [2025-10-20T18:17:57.123181] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:17:57.284924] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T18:17:57.851369] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-03. [2025-10-20T18:17:58.607358] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T18:17:58.651342] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-06. [2025-10-20T18:17:58.797010] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:17:59.451408] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-08. [2025-10-20T18:18:00.013391] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T18:18:00.251391] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-07. [2025-10-20T18:18:01.051385] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-04. [2025-10-20T18:18:01.851355] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-05. [2025-10-20T18:18:01.949935] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-08 (2.50 seconds). [2025-10-20T18:18:02.122916] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:18:02.145930] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T18:18:02.651394] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-05. [2025-10-20T18:18:03.456638] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-08. [2025-10-20T18:18:03.475385] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:18:03.826430] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:18:04.251385] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-06. [2025-10-20T18:18:04.809421] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:18:05.051374] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-07. [2025-10-20T18:18:05.334618] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:18:05.851363] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-09. [2025-10-20T18:18:05.886288] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T18:18:06.651426] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-06. [2025-10-20T18:18:07.451365] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-09. [2025-10-20T18:18:07.974208] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-09 (2.12 seconds). [2025-10-20T18:18:08.128614] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:18:08.251379] [INFO ] [REMOTE ] DHR: Fetching primary data for 2025-10. [2025-10-20T18:18:09.051362] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-07. [2025-10-20T18:18:09.137580] [INFO ] [REMOTE ] DHR: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T18:18:09.146050] [INFO ] [REMOTE ] DHR: Full fetch complete (77441 records). [2025-10-20T18:18:09.146244] [DEBUG] [VALIDATOR ] DHR: Validating primary data. [2025-10-20T18:18:09.558175] [DEBUG] [VALIDATOR ] DHR: All expected trading days are present. [2025-10-20T18:18:09.558384] [DEBUG] [VALIDATOR ] DHR: Primary data validation passed. [2025-10-20T18:18:09.559424] [INFO ] [MAIN ] DHR: Fetched primary data is valid. [2025-10-20T18:18:09.559536] [DEBUG] [LOCAL ] DHR: Saving primary data. [2025-10-20T18:18:09.561312] [DEBUG] [LOCAL ] DHR: Trimmed 77441 records to 53381 (keeping 200 days of data) [2025-10-20T18:18:09.685600] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T18:18:09.851428] [INFO ] [REMOTE ] DHI: Fetching primary data for 2025-10. [2025-10-20T18:18:09.937620] [DEBUG] [LOCAL ] DHR: Primary data saved (53381 records). [2025-10-20T18:18:09.937782] [INFO ] [MAIN ] DHR: Full history saved. [2025-10-20T18:18:09.937855] [INFO ] [MAIN ] Update Progress: 615/2367 symbols completed. DHR took 32.0s. Estimated remaining: 3h 42m 28s. Rate: 7.9 symbols/min [2025-10-20T18:18:10.108498] [INFO ] [MAIN ] DIAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:10.123285] [DEBUG] [LOCAL ] DIAX: Loaded 13192 existing records from primary storage. [2025-10-20T18:18:10.123609] [INFO ] [MAIN ] DIAX: Loaded 13192 existing records. [2025-10-20T18:18:10.317841] [WARN ] [MAIN ] DIAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:10.318174] [INFO ] [MAIN ] DIAX: Wiping existing primary data before full refresh. [2025-10-20T18:18:10.318251] [INFO ] [LOCAL ] DIAX: Starting deletion of primary data. [2025-10-20T18:18:10.318591] [DEBUG] [LOCAL ] DIAX: Deleted primary data file. [2025-10-20T18:18:10.318865] [INFO ] [MAIN ] DIAX: Starting full primary refresh. [2025-10-20T18:18:10.507545] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:18:10.656522] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-08. [2025-10-20T18:18:10.725362] [INFO ] [REMOTE ] DIAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:11.267702] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T18:18:11.451369] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-08. [2025-10-20T18:18:12.040738] [INFO ] [REMOTE ] DHI: Incremental fetch complete for 2025-10 (2.19 seconds). [2025-10-20T18:18:12.049110] [INFO ] [REMOTE ] DHI: Full fetch complete (77394 records). [2025-10-20T18:18:12.049330] [DEBUG] [VALIDATOR ] DHI: Validating primary data. [2025-10-20T18:18:12.262342] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-09. [2025-10-20T18:18:12.447871] [DEBUG] [VALIDATOR ] DHI: All expected trading days are present. [2025-10-20T18:18:12.447961] [DEBUG] [VALIDATOR ] DHI: Primary data validation passed. [2025-10-20T18:18:12.448912] [INFO ] [MAIN ] DHI: Fetched primary data is valid. [2025-10-20T18:18:12.448984] [DEBUG] [LOCAL ] DHI: Saving primary data. [2025-10-20T18:18:12.450518] [DEBUG] [LOCAL ] DHI: Trimmed 77394 records to 53365 (keeping 200 days of data) [2025-10-20T18:18:12.774869] [DEBUG] [LOCAL ] DHI: Primary data saved (53365 records). [2025-10-20T18:18:12.775000] [INFO ] [MAIN ] DHI: Full history saved. [2025-10-20T18:18:12.775067] [INFO ] [MAIN ] Update Progress: 616/2367 symbols completed. DHI took 36.3s. Estimated remaining: 3h 42m 7s. Rate: 7.9 symbols/min [2025-10-20T18:18:12.942961] [INFO ] [MAIN ] DIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:12.977974] [DEBUG] [LOCAL ] DIN: Loaded 37698 existing records from primary storage. [2025-10-20T18:18:12.978246] [INFO ] [MAIN ] DIN: Loaded 37698 existing records. [2025-10-20T18:18:13.053595] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-01. [2025-10-20T18:18:13.188037] [WARN ] [MAIN ] DIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:13.200687] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:18:13.201112] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:18:13.210133] [INFO ] [MAIN ] DIN: Wiping existing primary data before full refresh. [2025-10-20T18:18:13.210666] [INFO ] [LOCAL ] DIN: Starting deletion of primary data. [2025-10-20T18:18:13.214610] [DEBUG] [LOCAL ] DIN: Deleted primary data file. [2025-10-20T18:18:13.214900] [INFO ] [MAIN ] DIN: Starting full primary refresh. [2025-10-20T18:18:13.564413] [INFO ] [REMOTE ] DIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:13.851362] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-01. [2025-10-20T18:18:13.985433] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:18:14.651396] [INFO ] [REMOTE ] DHX: Fetching primary data for 2025-10. [2025-10-20T18:18:15.451439] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-09. [2025-10-20T18:18:15.484413] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:18:15.759411] [INFO ] [REMOTE ] DHX: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:18:15.762555] [INFO ] [REMOTE ] DHX: Full fetch complete (21449 records). [2025-10-20T18:18:15.762704] [DEBUG] [VALIDATOR ] DHX: Validating primary data. [2025-10-20T18:18:16.145361] [DEBUG] [VALIDATOR ] DHX: All expected trading days are present. [2025-10-20T18:18:16.145460] [DEBUG] [VALIDATOR ] DHX: Primary data validation passed. [2025-10-20T18:18:16.145912] [INFO ] [MAIN ] DHX: Fetched primary data is valid. [2025-10-20T18:18:16.145974] [DEBUG] [LOCAL ] DHX: Saving primary data. [2025-10-20T18:18:16.146585] [DEBUG] [LOCAL ] DHX: Trimmed 21449 records to 13199 (keeping 200 days of data) [2025-10-20T18:18:16.218371] [DEBUG] [LOCAL ] DHX: Primary data saved (13199 records). [2025-10-20T18:18:16.218515] [INFO ] [MAIN ] DHX: Full history saved. [2025-10-20T18:18:16.218589] [INFO ] [MAIN ] Update Progress: 617/2367 symbols completed. DHX took 27.7s. Estimated remaining: 3h 41m 47s. Rate: 7.9 symbols/min [2025-10-20T18:18:16.260611] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-02. [2025-10-20T18:18:16.387381] [INFO ] [MAIN ] DINO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:16.433979] [DEBUG] [LOCAL ] DINO: Loaded 53587 existing records from primary storage. [2025-10-20T18:18:16.434352] [INFO ] [MAIN ] DINO: Loaded 53587 existing records. [2025-10-20T18:18:16.607174] [WARN ] [MAIN ] DINO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:16.607377] [INFO ] [MAIN ] DINO: Wiping existing primary data before full refresh. [2025-10-20T18:18:16.607423] [INFO ] [LOCAL ] DINO: Starting deletion of primary data. [2025-10-20T18:18:16.607991] [DEBUG] [LOCAL ] DINO: Deleted primary data file. [2025-10-20T18:18:16.608047] [INFO ] [MAIN ] DINO: Starting full primary refresh. [2025-10-20T18:18:16.965768] [INFO ] [REMOTE ] DINO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:17.051403] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-01. [2025-10-20T18:18:17.076459] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T18:18:17.268362] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:18:17.851377] [INFO ] [REMOTE ] DHT: Fetching primary data for 2025-10. [2025-10-20T18:18:18.651368] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-03. [2025-10-20T18:18:19.033921] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T18:18:19.146454] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T18:18:19.451397] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-02. [2025-10-20T18:18:19.664852] [INFO ] [REMOTE ] DHT: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T18:18:19.672505] [INFO ] [REMOTE ] DHT: Full fetch complete (70233 records). [2025-10-20T18:18:19.672694] [DEBUG] [VALIDATOR ] DHT: Validating primary data. [2025-10-20T18:18:20.075702] [DEBUG] [VALIDATOR ] DHT: All expected trading days are present. [2025-10-20T18:18:20.075796] [DEBUG] [VALIDATOR ] DHT: Primary data validation passed. [2025-10-20T18:18:20.076609] [INFO ] [MAIN ] DHT: Fetched primary data is valid. [2025-10-20T18:18:20.076685] [DEBUG] [LOCAL ] DHT: Saving primary data. [2025-10-20T18:18:20.078069] [DEBUG] [LOCAL ] DHT: Trimmed 70233 records to 47915 (keeping 200 days of data) [2025-10-20T18:18:20.269621] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-02. [2025-10-20T18:18:20.391522] [DEBUG] [LOCAL ] DHT: Primary data saved (47915 records). [2025-10-20T18:18:20.391696] [INFO ] [MAIN ] DHT: Full history saved. [2025-10-20T18:18:20.391789] [INFO ] [MAIN ] Update Progress: 618/2367 symbols completed. DHT took 33.5s. Estimated remaining: 3h 41m 30s. Rate: 7.9 symbols/min [2025-10-20T18:18:20.577060] [INFO ] [MAIN ] DIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:20.631005] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T18:18:20.659801] [DEBUG] [LOCAL ] DIS: Loaded 54568 existing records from primary storage. [2025-10-20T18:18:20.660157] [INFO ] [MAIN ] DIS: Loaded 54568 existing records. [2025-10-20T18:18:20.859876] [WARN ] [MAIN ] DIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:20.860160] [INFO ] [MAIN ] DIS: Wiping existing primary data before full refresh. [2025-10-20T18:18:20.860219] [INFO ] [LOCAL ] DIS: Starting deletion of primary data. [2025-10-20T18:18:20.860715] [DEBUG] [LOCAL ] DIS: Deleted primary data file. [2025-10-20T18:18:20.860772] [INFO ] [MAIN ] DIS: Starting full primary refresh. [2025-10-20T18:18:21.056516] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-03. [2025-10-20T18:18:21.257769] [INFO ] [REMOTE ] DIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:21.668530] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:18:21.851371] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-03. [2025-10-20T18:18:22.651352] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-01. [2025-10-20T18:18:22.984301] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:18:23.060005] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:18:23.451431] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-04. [2025-10-20T18:18:24.251375] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-04. [2025-10-20T18:18:24.708507] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T18:18:25.040627] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:18:25.051325] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-05. [2025-10-20T18:18:25.851410] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-02. [2025-10-20T18:18:25.920094] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:18:26.509250] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:18:26.651386] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-04. [2025-10-20T18:18:27.451433] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-06. [2025-10-20T18:18:27.630114] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:18:27.879809] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T18:18:28.251402] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-05. [2025-10-20T18:18:28.645438] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:18:29.051394] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-07. [2025-10-20T18:18:29.851358] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-05. [2025-10-20T18:18:30.033322] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T18:18:30.490144] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:18:30.602172] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T18:18:30.651426] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-06. [2025-10-20T18:18:31.101055] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T18:18:31.451382] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-06. [2025-10-20T18:18:32.251406] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-08. [2025-10-20T18:18:33.051462] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-03. [2025-10-20T18:18:33.252372] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:18:33.733634] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:18:33.851369] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-09. [2025-10-20T18:18:34.651401] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-07. [2025-10-20T18:18:35.009979] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T18:18:35.456573] [INFO ] [REMOTE ] DIN: Fetching primary data for 2025-10. [2025-10-20T18:18:35.477594] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-03 (2.43 seconds). [2025-10-20T18:18:35.576978] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:18:36.251427] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-07. [2025-10-20T18:18:37.007949] [INFO ] [REMOTE ] DIN: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T18:18:37.013893] [INFO ] [REMOTE ] DIN: Full fetch complete (54925 records). [2025-10-20T18:18:37.014024] [DEBUG] [VALIDATOR ] DIN: Validating primary data. [2025-10-20T18:18:37.056597] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-08. [2025-10-20T18:18:37.443350] [DEBUG] [VALIDATOR ] DIN: All expected trading days are present. [2025-10-20T18:18:37.443449] [DEBUG] [VALIDATOR ] DIN: Primary data validation passed. [2025-10-20T18:18:37.444226] [INFO ] [MAIN ] DIN: Fetched primary data is valid. [2025-10-20T18:18:37.444290] [DEBUG] [LOCAL ] DIN: Saving primary data. [2025-10-20T18:18:37.445421] [DEBUG] [LOCAL ] DIN: Trimmed 54925 records to 37006 (keeping 200 days of data) [2025-10-20T18:18:37.674622] [DEBUG] [LOCAL ] DIN: Primary data saved (37006 records). [2025-10-20T18:18:37.674799] [INFO ] [MAIN ] DIN: Full history saved. [2025-10-20T18:18:37.675136] [INFO ] [MAIN ] Update Progress: 619/2367 symbols completed. DIN took 24.9s. Estimated remaining: 3h 41m 50s. Rate: 7.9 symbols/min [2025-10-20T18:18:37.854743] [INFO ] [MAIN ] DIVY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:37.853707] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-04. [2025-10-20T18:18:37.855119] [DEBUG] [LOCAL ] DIVY: No existing data file found in primary storage. [2025-10-20T18:18:37.863161] [WARN ] [MAIN ] DIVY: No existing data found, performing full refresh instead. [2025-10-20T18:18:37.893408] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:18:37.892809] [INFO ] [MAIN ] DIVY: Wiping existing primary data before full refresh. [2025-10-20T18:18:37.896900] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-10-20T18:18:37.900498] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-10-20T18:18:37.902948] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:18:37.901426] [INFO ] [MAIN ] DIVY: Starting full primary refresh. [2025-10-20T18:18:38.309393] [INFO ] [REMOTE ] DIVY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:38.651351] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-08. [2025-10-20T18:18:39.451413] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-09. [2025-10-20T18:18:40.251362] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-01. [2025-10-20T18:18:40.281923] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:18:40.407179] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:18:40.546394] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-04 (2.69 seconds). [2025-10-20T18:18:40.623751] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T18:18:41.051391] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-09. [2025-10-20T18:18:41.851400] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-02. [2025-10-20T18:18:42.207101] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:18:42.651353] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-05. [2025-10-20T18:18:42.900348] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:18:43.451420] [INFO ] [REMOTE ] DINO: Fetching primary data for 2025-10. [2025-10-20T18:18:44.251430] [INFO ] [REMOTE ] DIAX: Fetching primary data for 2025-10. [2025-10-20T18:18:45.051392] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-03. [2025-10-20T18:18:45.195234] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-10-20T18:18:45.229268] [INFO ] [REMOTE ] DINO: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T18:18:45.238018] [INFO ] [REMOTE ] DINO: Full fetch complete (76708 records). [2025-10-20T18:18:45.238137] [DEBUG] [VALIDATOR ] DINO: Validating primary data. [2025-10-20T18:18:45.345699] [INFO ] [REMOTE ] DIAX: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T18:18:45.375128] [INFO ] [REMOTE ] DIAX: Full fetch complete (17679 records). [2025-10-20T18:18:45.422912] [DEBUG] [VALIDATOR ] DIAX: Validating primary data. [2025-10-20T18:18:45.558720] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-10-20T18:18:45.857085] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-06. [2025-10-20T18:18:45.861245] [DEBUG] [VALIDATOR ] DINO: All expected trading days are present. [2025-10-20T18:18:45.899797] [DEBUG] [VALIDATOR ] DINO: Primary data validation passed. [2025-10-20T18:18:45.933420] [INFO ] [MAIN ] DINO: Fetched primary data is valid. [2025-10-20T18:18:45.944866] [DEBUG] [LOCAL ] DINO: Saving primary data. [2025-10-20T18:18:46.007569] [DEBUG] [LOCAL ] DINO: Trimmed 76708 records to 52842 (keeping 200 days of data) [2025-10-20T18:18:46.129458] [DEBUG] [VALIDATOR ] DIAX: All expected trading days are present. [2025-10-20T18:18:46.129560] [DEBUG] [VALIDATOR ] DIAX: Primary data validation passed. [2025-10-20T18:18:46.157363] [INFO ] [MAIN ] DIAX: Fetched primary data is valid. [2025-10-20T18:18:46.160066] [DEBUG] [LOCAL ] DIAX: Saving primary data. [2025-10-20T18:18:46.176755] [DEBUG] [LOCAL ] DIAX: Trimmed 17679 records to 12903 (keeping 200 days of data) [2025-10-20T18:18:46.426408] [DEBUG] [LOCAL ] DIAX: Primary data saved (12903 records). [2025-10-20T18:18:46.432214] [INFO ] [MAIN ] DIAX: Full history saved. [2025-10-20T18:18:46.432349] [INFO ] [MAIN ] Update Progress: 620/2367 symbols completed. DIAX took 36.5s. Estimated remaining: 3h 41m 45s. Rate: 7.9 symbols/min [2025-10-20T18:18:46.548907] [DEBUG] [LOCAL ] DINO: Primary data saved (52842 records). [2025-10-20T18:18:46.565788] [INFO ] [MAIN ] DINO: Full history saved. [2025-10-20T18:18:46.573749] [INFO ] [MAIN ] Update Progress: 621/2367 symbols completed. DINO took 30.4s. Estimated remaining: 3h 41m 17s. Rate: 7.9 symbols/min [2025-10-20T18:18:46.656502] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-04. [2025-10-20T18:18:46.791686] [INFO ] [MAIN ] DK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:46.804505] [DEBUG] [LOCAL ] DK: No existing data file found in primary storage. [2025-10-20T18:18:46.804792] [WARN ] [MAIN ] DK: No existing data found, performing full refresh instead. [2025-10-20T18:18:46.806218] [INFO ] [MAIN ] DK: Wiping existing primary data before full refresh. [2025-10-20T18:18:46.806608] [INFO ] [LOCAL ] DK: Starting deletion of primary data. [2025-10-20T18:18:46.827383] [DEBUG] [LOCAL ] DK: No primary data file found to delete. [2025-10-20T18:18:46.829587] [INFO ] [MAIN ] DKL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:46.829820] [INFO ] [MAIN ] DK: Starting full primary refresh. [2025-10-20T18:18:47.023490] [DEBUG] [LOCAL ] DKL: Loaded 17652 existing records from primary storage. [2025-10-20T18:18:47.081501] [INFO ] [MAIN ] DKL: Loaded 17652 existing records. [2025-10-20T18:18:47.237624] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:18:47.397563] [INFO ] [REMOTE ] DK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:47.417353] [WARN ] [MAIN ] DKL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:47.418007] [INFO ] [MAIN ] DKL: Wiping existing primary data before full refresh. [2025-10-20T18:18:47.418056] [INFO ] [LOCAL ] DKL: Starting deletion of primary data. [2025-10-20T18:18:47.418361] [DEBUG] [LOCAL ] DKL: Deleted primary data file. [2025-10-20T18:18:47.418417] [INFO ] [MAIN ] DKL: Starting full primary refresh. [2025-10-20T18:18:47.460322] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-01. [2025-10-20T18:18:47.776948] [INFO ] [REMOTE ] DKL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:48.172468] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-10-20T18:18:48.251338] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-07. [2025-10-20T18:18:48.897052] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:18:49.051402] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-05. [2025-10-20T18:18:49.401287] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:18:49.851392] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-06. [2025-10-20T18:18:50.162993] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-10-20T18:18:50.641825] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-07 (2.39 seconds). [2025-10-20T18:18:50.653259] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-02. [2025-10-20T18:18:51.451398] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-07. [2025-10-20T18:18:51.908748] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:18:52.042224] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:18:52.251382] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-03. [2025-10-20T18:18:53.051396] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-01. [2025-10-20T18:18:53.851569] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-08. [2025-10-20T18:18:53.982313] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:18:53.993895] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:18:54.114423] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-10-20T18:18:54.651367] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-08. [2025-10-20T18:18:55.451384] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-04. [2025-10-20T18:18:56.251367] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-09. [2025-10-20T18:18:56.967628] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T18:18:56.996989] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-10-20T18:18:57.039412] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:18:57.051322] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-05. [2025-10-20T18:18:57.851408] [INFO ] [REMOTE ] DIVY: Fetching primary data for 2025-10. [2025-10-20T18:18:58.198142] [INFO ] [REMOTE ] DIVY: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T18:18:58.198965] [INFO ] [REMOTE ] DIVY: Full fetch complete (1701 records). [2025-10-20T18:18:58.199033] [DEBUG] [VALIDATOR ] DIVY: Validating primary data. [2025-10-20T18:18:58.407496] [WARN ] [VALIDATOR ] DIVY: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T18:18:58.407801] [FATAL] [MAIN ] DIVY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:18:58.408067] [WARN ] [MAIN ] DIVY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:18:58.408142] [INFO ] [LOCAL ] DIVY: Starting deletion of primary data. [2025-10-20T18:18:58.408384] [DEBUG] [LOCAL ] DIVY: No primary data file found to delete. [2025-10-20T18:18:58.408450] [INFO ] [LOCAL ] DIVY: Starting deletion of derivative data. [2025-10-20T18:18:58.408582] [DEBUG] [LOCAL ] DIVY: No derivative data file found to delete. [2025-10-20T18:18:58.408743] [INFO ] [MAIN ] Update Progress: 622/2367 symbols completed. DIVY took 20.7s. Estimated remaining: 3h 41m 21s. Rate: 7.9 symbols/min [2025-10-20T18:18:58.604075] [INFO ] [MAIN ] DKS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:18:58.634823] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:18:58.651366] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-06. [2025-10-20T18:18:58.697709] [DEBUG] [LOCAL ] DKS: Loaded 50372 existing records from primary storage. [2025-10-20T18:18:58.698114] [INFO ] [MAIN ] DKS: Loaded 50372 existing records. [2025-10-20T18:18:58.883108] [WARN ] [MAIN ] DKS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:18:58.883343] [INFO ] [MAIN ] DKS: Wiping existing primary data before full refresh. [2025-10-20T18:18:58.883398] [INFO ] [LOCAL ] DKS: Starting deletion of primary data. [2025-10-20T18:18:58.883582] [DEBUG] [LOCAL ] DKS: Deleted primary data file. [2025-10-20T18:18:58.883621] [INFO ] [MAIN ] DKS: Starting full primary refresh. [2025-10-20T18:18:59.253134] [INFO ] [REMOTE ] DKS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:18:59.451368] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-01. [2025-10-20T18:19:00.251353] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-09. [2025-10-20T18:19:00.515858] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T18:19:01.051355] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-02. [2025-10-20T18:19:01.407534] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:19:01.851354] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-07. [2025-10-20T18:19:02.180920] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:19:02.651346] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-03. [2025-10-20T18:19:02.688750] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-09 (2.44 seconds). [2025-10-20T18:19:03.322960] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T18:19:03.451357] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-02. [2025-10-20T18:19:04.167706] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:19:04.251370] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-08. [2025-10-20T18:19:04.926879] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T18:19:05.051375] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-04. [2025-10-20T18:19:05.851348] [INFO ] [REMOTE ] DIS: Fetching primary data for 2025-10. [2025-10-20T18:19:06.093974] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T18:19:06.563166] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:19:06.651352] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-09. [2025-10-20T18:19:06.675147] [INFO ] [REMOTE ] DIS: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T18:19:06.683951] [INFO ] [REMOTE ] DIS: Full fetch complete (77967 records). [2025-10-20T18:19:06.684100] [DEBUG] [VALIDATOR ] DIS: Validating primary data. [2025-10-20T18:19:07.076932] [DEBUG] [VALIDATOR ] DIS: All expected trading days are present. [2025-10-20T18:19:07.077028] [DEBUG] [VALIDATOR ] DIS: Primary data validation passed. [2025-10-20T18:19:07.078008] [INFO ] [MAIN ] DIS: Fetched primary data is valid. [2025-10-20T18:19:07.078070] [DEBUG] [LOCAL ] DIS: Saving primary data. [2025-10-20T18:19:07.079490] [DEBUG] [LOCAL ] DIS: Trimmed 77967 records to 53788 (keeping 200 days of data) [2025-10-20T18:19:07.403102] [DEBUG] [LOCAL ] DIS: Primary data saved (53788 records). [2025-10-20T18:19:07.403233] [INFO ] [MAIN ] DIS: Full history saved. [2025-10-20T18:19:07.403311] [INFO ] [MAIN ] Update Progress: 623/2367 symbols completed. DIS took 47.0s. Estimated remaining: 3h 41m 17s. Rate: 7.9 symbols/min [2025-10-20T18:19:07.458023] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-05. [2025-10-20T18:19:07.573173] [INFO ] [MAIN ] DLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:07.603314] [DEBUG] [LOCAL ] DLB: Loaded 32746 existing records from primary storage. [2025-10-20T18:19:07.603586] [INFO ] [MAIN ] DLB: Loaded 32746 existing records. [2025-10-20T18:19:07.784515] [WARN ] [MAIN ] DLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:07.784745] [INFO ] [MAIN ] DLB: Wiping existing primary data before full refresh. [2025-10-20T18:19:07.784797] [INFO ] [LOCAL ] DLB: Starting deletion of primary data. [2025-10-20T18:19:07.785042] [DEBUG] [LOCAL ] DLB: Deleted primary data file. [2025-10-20T18:19:07.785080] [INFO ] [MAIN ] DLB: Starting full primary refresh. [2025-10-20T18:19:08.140789] [INFO ] [REMOTE ] DLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:08.251389] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-01. [2025-10-20T18:19:08.610916] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T18:19:08.635001] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:19:09.051373] [INFO ] [REMOTE ] DK: Fetching primary data for 2025-10. [2025-10-20T18:19:09.851368] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-03. [2025-10-20T18:19:10.027171] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:19:10.651383] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-06. [2025-10-20T18:19:11.207522] [INFO ] [REMOTE ] DK: Incremental fetch complete for 2025-10 (2.16 seconds). [2025-10-20T18:19:11.214724] [INFO ] [REMOTE ] DK: Full fetch complete (70837 records). [2025-10-20T18:19:11.214887] [DEBUG] [VALIDATOR ] DK: Validating primary data. [2025-10-20T18:19:11.460906] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-02. [2025-10-20T18:19:11.617427] [DEBUG] [VALIDATOR ] DK: All expected trading days are present. [2025-10-20T18:19:11.625326] [DEBUG] [VALIDATOR ] DK: Primary data validation passed. [2025-10-20T18:19:11.627566] [INFO ] [MAIN ] DK: Fetched primary data is valid. [2025-10-20T18:19:11.627698] [DEBUG] [LOCAL ] DK: Saving primary data. [2025-10-20T18:19:11.628992] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T18:19:11.634058] [DEBUG] [LOCAL ] DK: Trimmed 70837 records to 49346 (keeping 200 days of data) [2025-10-20T18:19:11.963144] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T18:19:11.980219] [DEBUG] [LOCAL ] DK: Primary data saved (49346 records). [2025-10-20T18:19:11.980743] [INFO ] [MAIN ] DK: Full history saved. [2025-10-20T18:19:11.980829] [INFO ] [MAIN ] Update Progress: 624/2367 symbols completed. DK took 25.5s. Estimated remaining: 3h 41m 1s. Rate: 7.9 symbols/min [2025-10-20T18:19:12.146080] [INFO ] [MAIN ] DLNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:12.153469] [DEBUG] [LOCAL ] DLNG: Loaded 6256 existing records from primary storage. [2025-10-20T18:19:12.153728] [INFO ] [MAIN ] DLNG: Loaded 6256 existing records. [2025-10-20T18:19:12.251428] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-07. [2025-10-20T18:19:12.330613] [WARN ] [MAIN ] DLNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:12.330835] [INFO ] [MAIN ] DLNG: Wiping existing primary data before full refresh. [2025-10-20T18:19:12.330883] [INFO ] [LOCAL ] DLNG: Starting deletion of primary data. [2025-10-20T18:19:12.331037] [DEBUG] [LOCAL ] DLNG: Deleted primary data file. [2025-10-20T18:19:12.331092] [INFO ] [MAIN ] DLNG: Starting full primary refresh. [2025-10-20T18:19:12.689549] [INFO ] [REMOTE ] DLNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:12.707490] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T18:19:13.020166] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T18:19:13.051375] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-01. [2025-10-20T18:19:13.851386] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-08. [2025-10-20T18:19:14.071688] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:19:14.651384] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-03. [2025-10-20T18:19:14.893887] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:19:15.451408] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-02. [2025-10-20T18:19:15.930139] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T18:19:16.251372] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-09. [2025-10-20T18:19:16.288251] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:19:17.051396] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-03. [2025-10-20T18:19:17.168974] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T18:19:17.708533] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:19:17.851409] [INFO ] [REMOTE ] DKL: Fetching primary data for 2025-10. [2025-10-20T18:19:18.651356] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-04. [2025-10-20T18:19:18.802792] [INFO ] [REMOTE ] DKL: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T18:19:18.804960] [INFO ] [REMOTE ] DKL: Full fetch complete (27662 records). [2025-10-20T18:19:18.805035] [DEBUG] [VALIDATOR ] DKL: Validating primary data. [2025-10-20T18:19:19.164268] [DEBUG] [VALIDATOR ] DKL: All expected trading days are present. [2025-10-20T18:19:19.164343] [DEBUG] [VALIDATOR ] DKL: Primary data validation passed. [2025-10-20T18:19:19.164853] [INFO ] [MAIN ] DKL: Fetched primary data is valid. [2025-10-20T18:19:19.164916] [DEBUG] [LOCAL ] DKL: Saving primary data. [2025-10-20T18:19:19.165534] [DEBUG] [LOCAL ] DKL: Trimmed 27662 records to 17203 (keeping 200 days of data) [2025-10-20T18:19:19.268973] [DEBUG] [LOCAL ] DKL: Primary data saved (17203 records). [2025-10-20T18:19:19.269104] [INFO ] [MAIN ] DKL: Full history saved. [2025-10-20T18:19:19.269179] [INFO ] [MAIN ] Update Progress: 625/2367 symbols completed. DKL took 32.7s. Estimated remaining: 3h 40m 53s. Rate: 7.9 symbols/min [2025-10-20T18:19:19.439404] [INFO ] [MAIN ] DLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:19.451369] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-04. [2025-10-20T18:19:19.490429] [DEBUG] [LOCAL ] DLR: Loaded 51423 existing records from primary storage. [2025-10-20T18:19:19.490775] [INFO ] [MAIN ] DLR: Loaded 51423 existing records. [2025-10-20T18:19:19.673506] [WARN ] [MAIN ] DLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:19.673819] [INFO ] [MAIN ] DLR: Wiping existing primary data before full refresh. [2025-10-20T18:19:19.673873] [INFO ] [LOCAL ] DLR: Starting deletion of primary data. [2025-10-20T18:19:19.674139] [DEBUG] [LOCAL ] DLR: Deleted primary data file. [2025-10-20T18:19:19.674207] [INFO ] [MAIN ] DLR: Starting full primary refresh. [2025-10-20T18:19:20.047711] [INFO ] [REMOTE ] DLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:20.066872] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T18:19:20.101976] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T18:19:20.251344] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-05. [2025-10-20T18:19:21.051369] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-04. [2025-10-20T18:19:21.551428] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T18:19:21.851377] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-01. [2025-10-20T18:19:22.651355] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-06. [2025-10-20T18:19:22.985153] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:19:23.451369] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-05. [2025-10-20T18:19:23.655680] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:19:24.073277] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:19:24.198881] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T18:19:24.251395] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-06. [2025-10-20T18:19:25.051443] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-02. [2025-10-20T18:19:25.104375] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:19:25.851448] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-07. [2025-10-20T18:19:26.304665] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T18:19:26.651392] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-08. [2025-10-20T18:19:26.961434] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T18:19:27.324926] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:19:27.451382] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-07. [2025-10-20T18:19:28.251428] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-03. [2025-10-20T18:19:28.827600] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:19:29.051375] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-05. [2025-10-20T18:19:29.796663] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:19:29.851421] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-04. [2025-10-20T18:19:30.651354] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-09. [2025-10-20T18:19:30.942124] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:19:31.248334] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:19:31.451352] [INFO ] [REMOTE ] DLNG: Fetching primary data for 2025-10. [2025-10-20T18:19:31.751392] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:19:32.040718] [INFO ] [REMOTE ] DLNG: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T18:19:32.042131] [INFO ] [REMOTE ] DLNG: Full fetch complete (11617 records). [2025-10-20T18:19:32.042202] [DEBUG] [VALIDATOR ] DLNG: Validating primary data. [2025-10-20T18:19:32.256591] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-05. [2025-10-20T18:19:32.412143] [DEBUG] [VALIDATOR ] DLNG: All expected trading days are present. [2025-10-20T18:19:32.412428] [DEBUG] [VALIDATOR ] DLNG: Primary data validation passed. [2025-10-20T18:19:32.412743] [INFO ] [MAIN ] DLNG: Fetched primary data is valid. [2025-10-20T18:19:32.412802] [DEBUG] [LOCAL ] DLNG: Saving primary data. [2025-10-20T18:19:32.413290] [DEBUG] [LOCAL ] DLNG: Trimmed 11617 records to 6143 (keeping 200 days of data) [2025-10-20T18:19:32.448825] [DEBUG] [LOCAL ] DLNG: Primary data saved (6143 records). [2025-10-20T18:19:32.448925] [INFO ] [MAIN ] DLNG: Full history saved. [2025-10-20T18:19:32.449001] [INFO ] [MAIN ] Update Progress: 626/2367 symbols completed. DLNG took 20.5s. Estimated remaining: 3h 41m 1s. Rate: 7.9 symbols/min [2025-10-20T18:19:32.618123] [INFO ] [MAIN ] DLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:32.646465] [DEBUG] [LOCAL ] DLX: Loaded 28685 existing records from primary storage. [2025-10-20T18:19:32.646746] [INFO ] [MAIN ] DLX: Loaded 28685 existing records. [2025-10-20T18:19:32.822224] [WARN ] [MAIN ] DLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:32.822436] [INFO ] [MAIN ] DLX: Wiping existing primary data before full refresh. [2025-10-20T18:19:32.822484] [INFO ] [LOCAL ] DLX: Starting deletion of primary data. [2025-10-20T18:19:32.822755] [DEBUG] [LOCAL ] DLX: Deleted primary data file. [2025-10-20T18:19:32.822814] [INFO ] [MAIN ] DLX: Starting full primary refresh. [2025-10-20T18:19:33.058625] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-06. [2025-10-20T18:19:33.184372] [INFO ] [REMOTE ] DLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:33.851594] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-08. [2025-10-20T18:19:34.107138] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:19:34.651364] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-06. [2025-10-20T18:19:34.688445] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T18:19:35.026253] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T18:19:35.451352] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-09. [2025-10-20T18:19:36.251440] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-01. [2025-10-20T18:19:36.326009] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:19:36.798317] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T18:19:37.051374] [INFO ] [REMOTE ] DLB: Fetching primary data for 2025-10. [2025-10-20T18:19:37.414834] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:19:37.851450] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-07. [2025-10-20T18:19:38.327912] [INFO ] [REMOTE ] DLB: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:19:38.335836] [INFO ] [REMOTE ] DLB: Full fetch complete (47353 records). [2025-10-20T18:19:38.336008] [DEBUG] [VALIDATOR ] DLB: Validating primary data. [2025-10-20T18:19:38.656687] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-07. [2025-10-20T18:19:38.721439] [DEBUG] [VALIDATOR ] DLB: All expected trading days are present. [2025-10-20T18:19:38.721522] [DEBUG] [VALIDATOR ] DLB: Primary data validation passed. [2025-10-20T18:19:38.722231] [INFO ] [MAIN ] DLB: Fetched primary data is valid. [2025-10-20T18:19:38.722295] [DEBUG] [LOCAL ] DLB: Saving primary data. [2025-10-20T18:19:38.723342] [DEBUG] [LOCAL ] DLB: Trimmed 47353 records to 32311 (keeping 200 days of data) [2025-10-20T18:19:38.924529] [DEBUG] [LOCAL ] DLB: Primary data saved (32311 records). [2025-10-20T18:19:38.924920] [INFO ] [MAIN ] DLB: Full history saved. [2025-10-20T18:19:38.925017] [INFO ] [MAIN ] Update Progress: 627/2367 symbols completed. DLB took 31.5s. Estimated remaining: 3h 40m 50s. Rate: 7.9 symbols/min [2025-10-20T18:19:39.096670] [INFO ] [MAIN ] DLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:39.116087] [DEBUG] [LOCAL ] DLY: Loaded 20947 existing records from primary storage. [2025-10-20T18:19:39.116360] [INFO ] [MAIN ] DLY: Loaded 20947 existing records. [2025-10-20T18:19:39.298409] [WARN ] [MAIN ] DLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:19:39.298758] [INFO ] [MAIN ] DLY: Wiping existing primary data before full refresh. [2025-10-20T18:19:39.298816] [INFO ] [LOCAL ] DLY: Starting deletion of primary data. [2025-10-20T18:19:39.299047] [DEBUG] [LOCAL ] DLY: Deleted primary data file. [2025-10-20T18:19:39.299102] [INFO ] [MAIN ] DLY: Starting full primary refresh. [2025-10-20T18:19:39.466700] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-02. [2025-10-20T18:19:39.669151] [INFO ] [REMOTE ] DLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:39.699552] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:19:40.251404] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-08. [2025-10-20T18:19:40.501693] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:19:40.820408] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:19:41.051378] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-08. [2025-10-20T18:19:41.851410] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-03. [2025-10-20T18:19:41.953618] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:19:42.463872] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T18:19:42.651408] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-01. [2025-10-20T18:19:43.018343] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:19:43.442869] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:19:43.451333] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-02. [2025-10-20T18:19:44.062306] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:19:44.251435] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-04. [2025-10-20T18:19:45.051430] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-03. [2025-10-20T18:19:45.593367] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:19:45.851409] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-05. [2025-10-20T18:19:46.037185] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T18:19:46.651432] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-04. [2025-10-20T18:19:46.910769] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:19:47.451382] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-09. [2025-10-20T18:19:47.721118] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:19:48.251429] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-05. [2025-10-20T18:19:49.051376] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-09. [2025-10-20T18:19:49.136932] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:19:49.179383] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:19:49.851436] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-06. [2025-10-20T18:19:50.651398] [INFO ] [REMOTE ] DLR: Fetching primary data for 2025-10. [2025-10-20T18:19:50.826910] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T18:19:50.878457] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T18:19:51.356770] [INFO ] [REMOTE ] DLR: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:19:51.371149] [INFO ] [REMOTE ] DLR: Full fetch complete (74165 records). [2025-10-20T18:19:51.371323] [DEBUG] [VALIDATOR ] DLR: Validating primary data. [2025-10-20T18:19:51.458964] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-07. [2025-10-20T18:19:51.815094] [DEBUG] [VALIDATOR ] DLR: All expected trading days are present. [2025-10-20T18:19:51.815172] [DEBUG] [VALIDATOR ] DLR: Primary data validation passed. [2025-10-20T18:19:51.816234] [INFO ] [MAIN ] DLR: Fetched primary data is valid. [2025-10-20T18:19:51.816316] [DEBUG] [LOCAL ] DLR: Saving primary data. [2025-10-20T18:19:51.817698] [DEBUG] [LOCAL ] DLR: Trimmed 74165 records to 50645 (keeping 200 days of data) [2025-10-20T18:19:52.123102] [DEBUG] [LOCAL ] DLR: Primary data saved (50645 records). [2025-10-20T18:19:52.123233] [INFO ] [MAIN ] DLR: Full history saved. [2025-10-20T18:19:52.123310] [INFO ] [MAIN ] Update Progress: 628/2367 symbols completed. DLR took 32.9s. Estimated remaining: 3h 40m 58s. Rate: 7.9 symbols/min [2025-10-20T18:19:52.263466] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-06. [2025-10-20T18:19:52.293934] [INFO ] [MAIN ] DMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:52.294939] [DEBUG] [LOCAL ] DMA: No existing data file found in primary storage. [2025-10-20T18:19:52.294996] [WARN ] [MAIN ] DMA: No existing data found, performing full refresh instead. [2025-10-20T18:19:52.295104] [INFO ] [MAIN ] DMA: Wiping existing primary data before full refresh. [2025-10-20T18:19:52.295145] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-10-20T18:19:52.295186] [DEBUG] [LOCAL ] DMA: No primary data file found to delete. [2025-10-20T18:19:52.295217] [INFO ] [MAIN ] DMA: Starting full primary refresh. [2025-10-20T18:19:52.683358] [INFO ] [REMOTE ] DMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:52.685799] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:19:53.051415] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-01. [2025-10-20T18:19:53.562421] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:19:53.742838] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:19:53.851390] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-07. [2025-10-20T18:19:54.651411] [INFO ] [REMOTE ] DKS: Fetching primary data for 2025-10. [2025-10-20T18:19:55.161943] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:19:55.451384] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-08. [2025-10-20T18:19:56.251369] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-08. [2025-10-20T18:19:56.316281] [INFO ] [REMOTE ] DKS: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:19:56.323537] [INFO ] [REMOTE ] DKS: Full fetch complete (71728 records). [2025-10-20T18:19:56.323716] [DEBUG] [VALIDATOR ] DKS: Validating primary data. [2025-10-20T18:19:56.649681] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:19:56.751119] [DEBUG] [VALIDATOR ] DKS: All expected trading days are present. [2025-10-20T18:19:56.751198] [DEBUG] [VALIDATOR ] DKS: Primary data validation passed. [2025-10-20T18:19:56.752095] [INFO ] [MAIN ] DKS: Fetched primary data is valid. [2025-10-20T18:19:56.752160] [DEBUG] [LOCAL ] DKS: Saving primary data. [2025-10-20T18:19:56.753525] [DEBUG] [LOCAL ] DKS: Trimmed 71728 records to 49551 (keeping 200 days of data) [2025-10-20T18:19:57.053290] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-09. [2025-10-20T18:19:57.072281] [DEBUG] [LOCAL ] DKS: Primary data saved (49551 records). [2025-10-20T18:19:57.072398] [INFO ] [MAIN ] DKS: Full history saved. [2025-10-20T18:19:57.072694] [INFO ] [MAIN ] Update Progress: 629/2367 symbols completed. DKS took 58.7s. Estimated remaining: 3h 40m 43s. Rate: 7.9 symbols/min [2025-10-20T18:19:57.242421] [INFO ] [MAIN ] DMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:19:57.242684] [DEBUG] [LOCAL ] DMB: No existing data file found in primary storage. [2025-10-20T18:19:57.242742] [WARN ] [MAIN ] DMB: No existing data found, performing full refresh instead. [2025-10-20T18:19:57.242851] [INFO ] [MAIN ] DMB: Wiping existing primary data before full refresh. [2025-10-20T18:19:57.242914] [INFO ] [LOCAL ] DMB: Starting deletion of primary data. [2025-10-20T18:19:57.242959] [DEBUG] [LOCAL ] DMB: No primary data file found to delete. [2025-10-20T18:19:57.242993] [INFO ] [MAIN ] DMB: Starting full primary refresh. [2025-10-20T18:19:57.604367] [INFO ] [REMOTE ] DMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:19:57.621450] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:19:57.851379] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-01. [2025-10-20T18:19:58.150942] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:19:58.651345] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-02. [2025-10-20T18:19:58.842385] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T18:19:59.132952] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:19:59.451385] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-02. [2025-10-20T18:20:00.163485] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T18:20:00.251344] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-03. [2025-10-20T18:20:00.907255] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:20:01.051376] [INFO ] [REMOTE ] DLY: Fetching primary data for 2025-10. [2025-10-20T18:20:01.851358] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-03. [2025-10-20T18:20:02.397922] [INFO ] [REMOTE ] DLY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:20:02.407695] [INFO ] [REMOTE ] DLY: Full fetch complete (28491 records). [2025-10-20T18:20:02.407907] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:20:02.408093] [DEBUG] [VALIDATOR ] DLY: Validating primary data. [2025-10-20T18:20:02.660370] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-09. [2025-10-20T18:20:02.795830] [DEBUG] [VALIDATOR ] DLY: All expected trading days are present. [2025-10-20T18:20:02.795910] [DEBUG] [VALIDATOR ] DLY: Primary data validation passed. [2025-10-20T18:20:02.796396] [INFO ] [MAIN ] DLY: Fetched primary data is valid. [2025-10-20T18:20:02.796434] [DEBUG] [LOCAL ] DLY: Saving primary data. [2025-10-20T18:20:02.797216] [DEBUG] [LOCAL ] DLY: Trimmed 28491 records to 20770 (keeping 200 days of data) [2025-10-20T18:20:02.920887] [DEBUG] [LOCAL ] DLY: Primary data saved (20770 records). [2025-10-20T18:20:02.921013] [INFO ] [MAIN ] DLY: Full history saved. [2025-10-20T18:20:02.921090] [INFO ] [MAIN ] Update Progress: 630/2367 symbols completed. DLY took 24.0s. Estimated remaining: 3h 40m 30s. Rate: 7.9 symbols/min [2025-10-20T18:20:03.094460] [INFO ] [MAIN ] DMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:03.094630] [DEBUG] [LOCAL ] DMO: No existing data file found in primary storage. [2025-10-20T18:20:03.094707] [WARN ] [MAIN ] DMO: No existing data found, performing full refresh instead. [2025-10-20T18:20:03.094835] [INFO ] [MAIN ] DMO: Wiping existing primary data before full refresh. [2025-10-20T18:20:03.094878] [INFO ] [LOCAL ] DMO: Starting deletion of primary data. [2025-10-20T18:20:03.094920] [DEBUG] [LOCAL ] DMO: No primary data file found to delete. [2025-10-20T18:20:03.094955] [INFO ] [MAIN ] DMO: Starting full primary refresh. [2025-10-20T18:20:03.457047] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-04. [2025-10-20T18:20:03.463270] [INFO ] [REMOTE ] DMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:04.102329] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:20:04.218491] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:20:04.251334] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-01. [2025-10-20T18:20:05.051376] [INFO ] [REMOTE ] DLX: Fetching primary data for 2025-10. [2025-10-20T18:20:05.154351] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T18:20:05.851369] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-05. [2025-10-20T18:20:06.419402] [INFO ] [REMOTE ] DLX: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:20:06.423228] [INFO ] [REMOTE ] DLX: Full fetch complete (41309 records). [2025-10-20T18:20:06.423328] [DEBUG] [VALIDATOR ] DLX: Validating primary data. [2025-10-20T18:20:06.540213] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:20:06.660598] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-02. [2025-10-20T18:20:06.823385] [DEBUG] [VALIDATOR ] DLX: All expected trading days are present. [2025-10-20T18:20:06.823470] [DEBUG] [VALIDATOR ] DLX: Primary data validation passed. [2025-10-20T18:20:06.824088] [INFO ] [MAIN ] DLX: Fetched primary data is valid. [2025-10-20T18:20:06.824151] [DEBUG] [LOCAL ] DLX: Saving primary data. [2025-10-20T18:20:06.825159] [DEBUG] [LOCAL ] DLX: Trimmed 41309 records to 28318 (keeping 200 days of data) [2025-10-20T18:20:07.000751] [DEBUG] [LOCAL ] DLX: Primary data saved (28318 records). [2025-10-20T18:20:07.000891] [INFO ] [MAIN ] DLX: Full history saved. [2025-10-20T18:20:07.000970] [INFO ] [MAIN ] Update Progress: 631/2367 symbols completed. DLX took 34.6s. Estimated remaining: 3h 40m 13s. Rate: 7.9 symbols/min [2025-10-20T18:20:07.188437] [INFO ] [MAIN ] DNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:07.231542] [DEBUG] [LOCAL ] DNA: Loaded 49343 existing records from primary storage. [2025-10-20T18:20:07.231871] [INFO ] [MAIN ] DNA: Loaded 49343 existing records. [2025-10-20T18:20:07.413788] [WARN ] [MAIN ] DNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:07.414200] [INFO ] [MAIN ] DNA: Wiping existing primary data before full refresh. [2025-10-20T18:20:07.414262] [INFO ] [LOCAL ] DNA: Starting deletion of primary data. [2025-10-20T18:20:07.414560] [DEBUG] [LOCAL ] DNA: Deleted primary data file. [2025-10-20T18:20:07.414619] [INFO ] [MAIN ] DNA: Starting full primary refresh. [2025-10-20T18:20:07.457486] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-04. [2025-10-20T18:20:07.536173] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T18:20:07.799435] [INFO ] [REMOTE ] DNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:08.251442] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-06. [2025-10-20T18:20:08.311132] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:20:08.787972] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:20:09.051393] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-05. [2025-10-20T18:20:09.851388] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-03. [2025-10-20T18:20:09.978399] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:20:10.651379] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-01. [2025-10-20T18:20:10.678340] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:20:11.451432] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-07. [2025-10-20T18:20:12.026100] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:20:12.251410] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-08. [2025-10-20T18:20:12.613470] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T18:20:12.913496] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T18:20:13.051453] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-04. [2025-10-20T18:20:13.824223] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T18:20:13.851335] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-05. [2025-10-20T18:20:14.544104] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:20:14.651411] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-02. [2025-10-20T18:20:15.451398] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-09. [2025-10-20T18:20:16.225601] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T18:20:16.251349] [INFO ] [REMOTE ] DMA: Fetching primary data for 2025-10. [2025-10-20T18:20:16.655514] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T18:20:16.967979] [INFO ] [REMOTE ] DMA: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:20:16.969249] [INFO ] [REMOTE ] DMA: Full fetch complete (6602 records). [2025-10-20T18:20:16.969318] [DEBUG] [VALIDATOR ] DMA: Validating primary data. [2025-10-20T18:20:16.969699] [FATAL] [VALIDATOR ] DMA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:20:16.969795] [FATAL] [MAIN ] DMA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:20:16.969871] [WARN ] [MAIN ] DMA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:20:16.969969] [INFO ] [LOCAL ] DMA: Starting deletion of primary data. [2025-10-20T18:20:16.970067] [DEBUG] [LOCAL ] DMA: No primary data file found to delete. [2025-10-20T18:20:16.970125] [INFO ] [LOCAL ] DMA: Starting deletion of derivative data. [2025-10-20T18:20:16.970171] [DEBUG] [LOCAL ] DMA: No derivative data file found to delete. [2025-10-20T18:20:16.970224] [INFO ] [MAIN ] Update Progress: 632/2367 symbols completed. DMA took 24.8s. Estimated remaining: 3h 40m 12s. Rate: 7.9 symbols/min [2025-10-20T18:20:17.056456] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-03. [2025-10-20T18:20:17.165689] [INFO ] [MAIN ] DNOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:17.206410] [DEBUG] [LOCAL ] DNOW: Loaded 44485 existing records from primary storage. [2025-10-20T18:20:17.206718] [INFO ] [MAIN ] DNOW: Loaded 44485 existing records. [2025-10-20T18:20:17.385112] [WARN ] [MAIN ] DNOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:17.385386] [INFO ] [MAIN ] DNOW: Wiping existing primary data before full refresh. [2025-10-20T18:20:17.385438] [INFO ] [LOCAL ] DNOW: Starting deletion of primary data. [2025-10-20T18:20:17.385714] [DEBUG] [LOCAL ] DNOW: Deleted primary data file. [2025-10-20T18:20:17.385773] [INFO ] [MAIN ] DNOW: Starting full primary refresh. [2025-10-20T18:20:17.749400] [INFO ] [REMOTE ] DNOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:17.851380] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-01. [2025-10-20T18:20:18.651360] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-06. [2025-10-20T18:20:19.038169] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T18:20:19.385492] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:20:19.420200] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:20:19.451438] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-07. [2025-10-20T18:20:20.251428] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-06. [2025-10-20T18:20:20.367346] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T18:20:20.944270] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T18:20:21.051409] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-08. [2025-10-20T18:20:21.811599] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T18:20:21.851382] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-09. [2025-10-20T18:20:22.640861] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:20:22.651344] [INFO ] [REMOTE ] DMB: Fetching primary data for 2025-10. [2025-10-20T18:20:23.441823] [INFO ] [REMOTE ] DMB: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:20:23.443628] [INFO ] [REMOTE ] DMB: Full fetch complete (14670 records). [2025-10-20T18:20:23.443846] [DEBUG] [VALIDATOR ] DMB: Validating primary data. [2025-10-20T18:20:23.456492] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-02. [2025-10-20T18:20:23.874466] [DEBUG] [VALIDATOR ] DMB: All expected trading days are present. [2025-10-20T18:20:23.874558] [DEBUG] [VALIDATOR ] DMB: Primary data validation passed. [2025-10-20T18:20:23.874937] [INFO ] [MAIN ] DMB: Fetched primary data is valid. [2025-10-20T18:20:23.874998] [DEBUG] [LOCAL ] DMB: Saving primary data. [2025-10-20T18:20:23.875688] [DEBUG] [LOCAL ] DMB: Trimmed 14670 records to 10378 (keeping 200 days of data) [2025-10-20T18:20:23.939295] [DEBUG] [LOCAL ] DMB: Primary data saved (10378 records). [2025-10-20T18:20:23.939448] [INFO ] [MAIN ] DMB: Full history saved. [2025-10-20T18:20:23.939521] [INFO ] [MAIN ] Update Progress: 633/2367 symbols completed. DMB took 26.9s. Estimated remaining: 3h 40m 2s. Rate: 7.9 symbols/min [2025-10-20T18:20:24.116335] [INFO ] [MAIN ] DNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:24.117002] [DEBUG] [LOCAL ] DNP: No existing data file found in primary storage. [2025-10-20T18:20:24.117056] [WARN ] [MAIN ] DNP: No existing data found, performing full refresh instead. [2025-10-20T18:20:24.117240] [INFO ] [MAIN ] DNP: Wiping existing primary data before full refresh. [2025-10-20T18:20:24.117294] [INFO ] [LOCAL ] DNP: Starting deletion of primary data. [2025-10-20T18:20:24.117339] [DEBUG] [LOCAL ] DNP: No primary data file found to delete. [2025-10-20T18:20:24.117392] [INFO ] [MAIN ] DNP: Starting full primary refresh. [2025-10-20T18:20:24.258890] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-04. [2025-10-20T18:20:24.544010] [INFO ] [REMOTE ] DNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:25.022064] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:20:25.051405] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-03. [2025-10-20T18:20:25.851399] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-01. [2025-10-20T18:20:26.307183] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T18:20:26.651395] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-07. [2025-10-20T18:20:26.704621] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:20:27.452210] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-04. [2025-10-20T18:20:27.454992] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:20:27.497804] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:20:28.251396] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-05. [2025-10-20T18:20:29.035024] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:20:29.051347] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-05. [2025-10-20T18:20:29.851367] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-08. [2025-10-20T18:20:29.980521] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:20:30.579029] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:20:30.651355] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-02. [2025-10-20T18:20:30.742959] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:20:31.451421] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-09. [2025-10-20T18:20:31.785129] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:20:32.251365] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-06. [2025-10-20T18:20:32.390519] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:20:33.051439] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-03. [2025-10-20T18:20:33.712121] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:20:33.851331] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-07. [2025-10-20T18:20:34.455344] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:20:34.651399] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-06. [2025-10-20T18:20:35.435547] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T18:20:35.451359] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-08. [2025-10-20T18:20:36.245513] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:20:36.255155] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-04. [2025-10-20T18:20:37.057831] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-07. [2025-10-20T18:20:37.078005] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T18:20:37.851430] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-09. [2025-10-20T18:20:37.949962] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:20:38.651418] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-05. [2025-10-20T18:20:39.090839] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:20:39.153481] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T18:20:39.451436] [INFO ] [REMOTE ] DNOW: Fetching primary data for 2025-10. [2025-10-20T18:20:39.565066] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:20:40.251390] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-06. [2025-10-20T18:20:41.051398] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-08. [2025-10-20T18:20:41.194142] [INFO ] [REMOTE ] DNOW: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:20:41.195676] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:20:41.214350] [INFO ] [REMOTE ] DNOW: Full fetch complete (63496 records). [2025-10-20T18:20:41.214572] [DEBUG] [VALIDATOR ] DNOW: Validating primary data. [2025-10-20T18:20:41.607664] [DEBUG] [VALIDATOR ] DNOW: All expected trading days are present. [2025-10-20T18:20:41.607756] [DEBUG] [VALIDATOR ] DNOW: Primary data validation passed. [2025-10-20T18:20:41.608581] [INFO ] [MAIN ] DNOW: Fetched primary data is valid. [2025-10-20T18:20:41.608632] [DEBUG] [LOCAL ] DNOW: Saving primary data. [2025-10-20T18:20:41.610132] [DEBUG] [LOCAL ] DNOW: Trimmed 63496 records to 43858 (keeping 200 days of data) [2025-10-20T18:20:41.861705] [DEBUG] [LOCAL ] DNOW: Primary data saved (43858 records). [2025-10-20T18:20:41.862060] [INFO ] [MAIN ] DNOW: Full history saved. [2025-10-20T18:20:41.861815] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-07. [2025-10-20T18:20:41.862259] [INFO ] [MAIN ] Update Progress: 634/2367 symbols completed. DNOW took 24.9s. Estimated remaining: 3h 40m 23s. Rate: 7.9 symbols/min [2025-10-20T18:20:42.042903] [INFO ] [MAIN ] DOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:42.096714] [DEBUG] [LOCAL ] DOC: Loaded 54334 existing records from primary storage. [2025-10-20T18:20:42.097053] [INFO ] [MAIN ] DOC: Loaded 54334 existing records. [2025-10-20T18:20:42.278816] [WARN ] [MAIN ] DOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:42.279049] [INFO ] [MAIN ] DOC: Wiping existing primary data before full refresh. [2025-10-20T18:20:42.279097] [INFO ] [LOCAL ] DOC: Starting deletion of primary data. [2025-10-20T18:20:42.279531] [DEBUG] [LOCAL ] DOC: Deleted primary data file. [2025-10-20T18:20:42.279588] [INFO ] [MAIN ] DOC: Starting full primary refresh. [2025-10-20T18:20:42.635771] [INFO ] [REMOTE ] DOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:42.651330] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-01. [2025-10-20T18:20:42.917009] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:20:43.356518] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:20:43.451453] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-09. [2025-10-20T18:20:44.251393] [INFO ] [REMOTE ] DMO: Fetching primary data for 2025-10. [2025-10-20T18:20:44.450693] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:20:44.939959] [INFO ] [REMOTE ] DMO: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T18:20:44.941807] [INFO ] [REMOTE ] DMO: Full fetch complete (14559 records). [2025-10-20T18:20:44.941878] [DEBUG] [VALIDATOR ] DMO: Validating primary data. [2025-10-20T18:20:45.054877] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-08. [2025-10-20T18:20:45.338004] [DEBUG] [VALIDATOR ] DMO: All expected trading days are present. [2025-10-20T18:20:45.338130] [DEBUG] [VALIDATOR ] DMO: Primary data validation passed. [2025-10-20T18:20:45.338709] [INFO ] [MAIN ] DMO: Fetched primary data is valid. [2025-10-20T18:20:45.338830] [DEBUG] [LOCAL ] DMO: Saving primary data. [2025-10-20T18:20:45.339956] [DEBUG] [LOCAL ] DMO: Trimmed 14559 records to 9868 (keeping 200 days of data) [2025-10-20T18:20:45.424461] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T18:20:45.440999] [DEBUG] [LOCAL ] DMO: Primary data saved (9868 records). [2025-10-20T18:20:45.441127] [INFO ] [MAIN ] DMO: Full history saved. [2025-10-20T18:20:45.441194] [INFO ] [MAIN ] Update Progress: 635/2367 symbols completed. DMO took 42.5s. Estimated remaining: 3h 40m 4s. Rate: 7.9 symbols/min [2025-10-20T18:20:45.620724] [INFO ] [MAIN ] DOCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:45.666794] [DEBUG] [LOCAL ] DOCN: Loaded 51798 existing records from primary storage. [2025-10-20T18:20:45.667104] [INFO ] [MAIN ] DOCN: Loaded 51798 existing records. [2025-10-20T18:20:45.841845] [WARN ] [MAIN ] DOCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:20:45.842065] [INFO ] [MAIN ] DOCN: Wiping existing primary data before full refresh. [2025-10-20T18:20:45.842110] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-10-20T18:20:45.842415] [DEBUG] [LOCAL ] DOCN: Deleted primary data file. [2025-10-20T18:20:45.842452] [INFO ] [MAIN ] DOCN: Starting full primary refresh. [2025-10-20T18:20:45.856458] [INFO ] [REMOTE ] DNA: Fetching primary data for 2025-10. [2025-10-20T18:20:46.197816] [INFO ] [REMOTE ] DOCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:46.436608] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:20:46.651373] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-01. [2025-10-20T18:20:47.451394] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-02. [2025-10-20T18:20:48.144591] [INFO ] [REMOTE ] DNA: Incremental fetch complete for 2025-10 (2.29 seconds). [2025-10-20T18:20:48.152546] [INFO ] [REMOTE ] DNA: Full fetch complete (71299 records). [2025-10-20T18:20:48.152762] [DEBUG] [VALIDATOR ] DNA: Validating primary data. [2025-10-20T18:20:48.259052] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-09. [2025-10-20T18:20:48.566407] [DEBUG] [VALIDATOR ] DNA: All expected trading days are present. [2025-10-20T18:20:48.566528] [DEBUG] [VALIDATOR ] DNA: Primary data validation passed. [2025-10-20T18:20:48.567581] [INFO ] [MAIN ] DNA: Fetched primary data is valid. [2025-10-20T18:20:48.567760] [DEBUG] [LOCAL ] DNA: Saving primary data. [2025-10-20T18:20:48.569511] [DEBUG] [LOCAL ] DNA: Trimmed 71299 records to 48575 (keeping 200 days of data) [2025-10-20T18:20:48.686110] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:20:48.938188] [DEBUG] [LOCAL ] DNA: Primary data saved (48575 records). [2025-10-20T18:20:48.938484] [INFO ] [MAIN ] DNA: Full history saved. [2025-10-20T18:20:48.938699] [INFO ] [MAIN ] Update Progress: 636/2367 symbols completed. DNA took 41.9s. Estimated remaining: 3h 39m 45s. Rate: 7.9 symbols/min [2025-10-20T18:20:49.059840] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-02. [2025-10-20T18:20:49.157188] [INFO ] [MAIN ] DOCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:20:49.163116] [DEBUG] [LOCAL ] DOCS: No existing data file found in primary storage. [2025-10-20T18:20:49.170378] [WARN ] [MAIN ] DOCS: No existing data found, performing full refresh instead. [2025-10-20T18:20:49.173516] [INFO ] [MAIN ] DOCS: Wiping existing primary data before full refresh. [2025-10-20T18:20:49.173990] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-10-20T18:20:49.175021] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:20:49.175229] [DEBUG] [LOCAL ] DOCS: No primary data file found to delete. [2025-10-20T18:20:49.185163] [INFO ] [MAIN ] DOCS: Starting full primary refresh. [2025-10-20T18:20:49.607313] [INFO ] [REMOTE ] DOCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:20:49.628310] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:20:49.851388] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-01. [2025-10-20T18:20:50.651363] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-03. [2025-10-20T18:20:50.738746] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:20:51.451369] [INFO ] [REMOTE ] DNP: Fetching primary data for 2025-10. [2025-10-20T18:20:51.770140] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T18:20:52.251444] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-02. [2025-10-20T18:20:52.350245] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:20:53.051353] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-04. [2025-10-20T18:20:53.147314] [INFO ] [REMOTE ] DNP: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T18:20:53.154751] [INFO ] [REMOTE ] DNP: Full fetch complete (48751 records). [2025-10-20T18:20:53.154892] [DEBUG] [VALIDATOR ] DNP: Validating primary data. [2025-10-20T18:20:53.586023] [DEBUG] [VALIDATOR ] DNP: All expected trading days are present. [2025-10-20T18:20:53.586110] [DEBUG] [VALIDATOR ] DNP: Primary data validation passed. [2025-10-20T18:20:53.586778] [INFO ] [MAIN ] DNP: Fetched primary data is valid. [2025-10-20T18:20:53.586838] [DEBUG] [LOCAL ] DNP: Saving primary data. [2025-10-20T18:20:53.587863] [DEBUG] [LOCAL ] DNP: Trimmed 48751 records to 33154 (keeping 200 days of data) [2025-10-20T18:20:53.782777] [DEBUG] [LOCAL ] DNP: Primary data saved (33154 records). [2025-10-20T18:20:53.782936] [INFO ] [MAIN ] DNP: Full history saved. [2025-10-20T18:20:53.783020] [INFO ] [MAIN ] Update Progress: 637/2367 symbols completed. DNP took 29.8s. Estimated remaining: 3h 39m 30s. Rate: 7.9 symbols/min [2025-10-20T18:20:53.851454] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-03. [2025-10-20T18:20:54.808982] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-02 (2.56 seconds). [2025-10-20T18:20:54.818892] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-03. [2025-10-20T18:20:55.341032] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T18:20:55.451336] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-05. [2025-10-20T18:20:55.818008] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T18:20:56.251415] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-04. [2025-10-20T18:20:56.622671] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:20:57.051347] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-04. [2025-10-20T18:20:57.189810] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:20:57.835589] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:20:57.851335] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-05. [2025-10-20T18:20:58.657814] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-06. [2025-10-20T18:20:58.683877] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:20:59.278595] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T18:20:59.451435] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-05. [2025-10-20T18:21:00.251377] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-06. [2025-10-20T18:21:00.408285] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T18:21:01.051367] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-07. [2025-10-20T18:21:01.557158] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T18:21:01.851409] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-06. [2025-10-20T18:21:02.016179] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T18:21:02.651380] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-07. [2025-10-20T18:21:03.079352] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:21:03.451371] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-08. [2025-10-20T18:21:03.582301] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:21:04.251432] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-07. [2025-10-20T18:21:04.414096] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:21:04.994063] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:21:05.051455] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-08. [2025-10-20T18:21:05.795872] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:21:05.851371] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-08. [2025-10-20T18:21:06.651419] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-09. [2025-10-20T18:21:07.014015] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T18:21:07.451345] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-09. [2025-10-20T18:21:07.641590] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:21:08.251415] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-09. [2025-10-20T18:21:09.771739] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-09 (3.12 seconds). [2025-10-20T18:21:09.783362] [INFO ] [REMOTE ] DOC: Fetching primary data for 2025-10. [2025-10-20T18:21:09.841132] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-09 (2.39 seconds). [2025-10-20T18:21:09.852589] [INFO ] [REMOTE ] DOCN: Fetching primary data for 2025-10. [2025-10-20T18:21:10.430395] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T18:21:10.651395] [INFO ] [REMOTE ] DOCS: Fetching primary data for 2025-10. [2025-10-20T18:21:11.762529] [INFO ] [REMOTE ] DOCN: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:21:11.770134] [INFO ] [REMOTE ] DOCN: Full fetch complete (72423 records). [2025-10-20T18:21:11.770304] [DEBUG] [VALIDATOR ] DOCN: Validating primary data. [2025-10-20T18:21:11.771179] [FATAL] [VALIDATOR ] DOCN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:21:11.771267] [FATAL] [MAIN ] DOCN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:21:11.771310] [WARN ] [MAIN ] DOCN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:21:11.771347] [INFO ] [LOCAL ] DOCN: Starting deletion of primary data. [2025-10-20T18:21:11.771409] [DEBUG] [LOCAL ] DOCN: No primary data file found to delete. [2025-10-20T18:21:11.771452] [INFO ] [LOCAL ] DOCN: Starting deletion of derivative data. [2025-10-20T18:21:11.771628] [DEBUG] [LOCAL ] DOCN: Deleted derivative data file. [2025-10-20T18:21:11.771773] [INFO ] [MAIN ] Update Progress: 638/2367 symbols completed. DOCN took 26.3s. Estimated remaining: 3h 39m 51s. Rate: 7.9 symbols/min [2025-10-20T18:21:11.834579] [INFO ] [REMOTE ] DOC: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T18:21:11.842895] [INFO ] [REMOTE ] DOC: Full fetch complete (77570 records). [2025-10-20T18:21:11.843014] [DEBUG] [VALIDATOR ] DOC: Validating primary data. [2025-10-20T18:21:12.233556] [DEBUG] [VALIDATOR ] DOC: All expected trading days are present. [2025-10-20T18:21:12.233701] [DEBUG] [VALIDATOR ] DOC: Primary data validation passed. [2025-10-20T18:21:12.234730] [INFO ] [MAIN ] DOC: Fetched primary data is valid. [2025-10-20T18:21:12.234910] [DEBUG] [LOCAL ] DOC: Saving primary data. [2025-10-20T18:21:12.261106] [DEBUG] [LOCAL ] DOC: Trimmed 77570 records to 53556 (keeping 200 days of data) [2025-10-20T18:21:12.262168] [INFO ] [REMOTE ] DOCS: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T18:21:12.270083] [INFO ] [REMOTE ] DOCS: Full fetch complete (73598 records). [2025-10-20T18:21:12.270280] [DEBUG] [VALIDATOR ] DOCS: Validating primary data. [2025-10-20T18:21:12.321759] [FATAL] [VALIDATOR ] DOCS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:21:12.330820] [FATAL] [MAIN ] DOCS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:21:12.357203] [WARN ] [MAIN ] DOCS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:21:12.368535] [INFO ] [LOCAL ] DOCS: Starting deletion of primary data. [2025-10-20T18:21:12.391760] [DEBUG] [LOCAL ] DOCS: No primary data file found to delete. [2025-10-20T18:21:12.391852] [INFO ] [LOCAL ] DOCS: Starting deletion of derivative data. [2025-10-20T18:21:12.391896] [DEBUG] [LOCAL ] DOCS: No derivative data file found to delete. [2025-10-20T18:21:12.391984] [INFO ] [MAIN ] Update Progress: 639/2367 symbols completed. DOCS took 23.5s. Estimated remaining: 3h 39m 24s. Rate: 7.9 symbols/min [2025-10-20T18:21:12.588146] [DEBUG] [LOCAL ] DOC: Primary data saved (53556 records). [2025-10-20T18:21:12.588274] [INFO ] [MAIN ] DOC: Full history saved. [2025-10-20T18:21:12.588357] [INFO ] [MAIN ] Update Progress: 640/2367 symbols completed. DOC took 30.7s. Estimated remaining: 3h 38m 56s. Rate: 7.9 symbols/min [2025-10-20T18:21:13.162009] [INFO ] [MAIN ] DOLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.299181] [INFO ] [MAIN ] DOUG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.323279] [INFO ] [MAIN ] DOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.340619] [INFO ] [MAIN ] DOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:13.411821] [DEBUG] [LOCAL ] DOUG: Loaded 34582 existing records from primary storage. [2025-10-20T18:21:13.412330] [INFO ] [MAIN ] DOUG: Loaded 34582 existing records. [2025-10-20T18:21:13.558775] [DEBUG] [LOCAL ] DOLE: Loaded 39732 existing records from primary storage. [2025-10-20T18:21:13.559720] [DEBUG] [LOCAL ] DOW: Loaded 54513 existing records from primary storage. [2025-10-20T18:21:13.562418] [DEBUG] [LOCAL ] DOV: Loaded 46315 existing records from primary storage. [2025-10-20T18:21:13.573131] [INFO ] [MAIN ] DOLE: Loaded 39732 existing records. [2025-10-20T18:21:13.573041] [INFO ] [MAIN ] DOV: Loaded 46315 existing records. [2025-10-20T18:21:13.573330] [INFO ] [MAIN ] DOW: Loaded 54513 existing records. [2025-10-20T18:21:13.943143] [WARN ] [MAIN ] DOUG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:13.978672] [INFO ] [MAIN ] DOUG: Wiping existing primary data before full refresh. [2025-10-20T18:21:13.990150] [INFO ] [LOCAL ] DOUG: Starting deletion of primary data. [2025-10-20T18:21:14.109459] [DEBUG] [LOCAL ] DOUG: Deleted primary data file. [2025-10-20T18:21:14.150891] [WARN ] [MAIN ] DOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.159900] [INFO ] [MAIN ] DOUG: Starting full primary refresh. [2025-10-20T18:21:14.202411] [INFO ] [MAIN ] DOV: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.244156] [INFO ] [LOCAL ] DOV: Starting deletion of primary data. [2025-10-20T18:21:14.287198] [WARN ] [MAIN ] DOLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.301056] [WARN ] [MAIN ] DOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:14.329014] [INFO ] [MAIN ] DOW: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.329257] [INFO ] [MAIN ] DOLE: Wiping existing primary data before full refresh. [2025-10-20T18:21:14.315683] [DEBUG] [LOCAL ] DOV: Deleted primary data file. [2025-10-20T18:21:14.340031] [INFO ] [LOCAL ] DOW: Starting deletion of primary data. [2025-10-20T18:21:14.345241] [INFO ] [LOCAL ] DOLE: Starting deletion of primary data. [2025-10-20T18:21:14.350464] [INFO ] [MAIN ] DOV: Starting full primary refresh. [2025-10-20T18:21:14.371884] [DEBUG] [LOCAL ] DOW: Deleted primary data file. [2025-10-20T18:21:14.380533] [INFO ] [MAIN ] DOW: Starting full primary refresh. [2025-10-20T18:21:14.386006] [DEBUG] [LOCAL ] DOLE: Deleted primary data file. [2025-10-20T18:21:14.425280] [INFO ] [MAIN ] DOLE: Starting full primary refresh. [2025-10-20T18:21:15.714173] [INFO ] [REMOTE ] DOUG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.745573] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-01. [2025-10-20T18:21:15.786175] [INFO ] [REMOTE ] DOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.815535] [INFO ] [REMOTE ] DOLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.835240] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-01. [2025-10-20T18:21:15.834941] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-01. [2025-10-20T18:21:15.858954] [INFO ] [REMOTE ] DOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:15.859305] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-01. [2025-10-20T18:21:17.291105] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:21:17.298959] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-02. [2025-10-20T18:21:17.371152] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:21:17.378384] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-02. [2025-10-20T18:21:17.419785] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:21:17.428500] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-02. [2025-10-20T18:21:17.908101] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T18:21:17.919117] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-02. [2025-10-20T18:21:18.243263] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:21:18.250450] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-03. [2025-10-20T18:21:18.472817] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T18:21:18.651365] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-03. [2025-10-20T18:21:18.959312] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:21:19.204609] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:21:19.451415] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-03. [2025-10-20T18:21:19.659295] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:21:19.861879] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:21:20.251376] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-04. [2025-10-20T18:21:20.832470] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:21:21.051388] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-03. [2025-10-20T18:21:21.851391] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-04. [2025-10-20T18:21:22.557590] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T18:21:22.651395] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-05. [2025-10-20T18:21:23.451428] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-04. [2025-10-20T18:21:23.911301] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-03 (2.86 seconds). [2025-10-20T18:21:24.251383] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-04. [2025-10-20T18:21:24.318953] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:21:24.434533] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-04 (2.58 seconds). [2025-10-20T18:21:24.742973] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T18:21:25.051476] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-06. [2025-10-20T18:21:25.851480] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-05. [2025-10-20T18:21:26.357494] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:21:26.570772] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-10-20T18:21:26.651335] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-05. [2025-10-20T18:21:27.456528] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-07. [2025-10-20T18:21:27.470284] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:21:28.251429] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-06. [2025-10-20T18:21:28.876407] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:21:28.896520] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T18:21:29.051419] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-05. [2025-10-20T18:21:29.593348] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T18:21:29.851462] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-08. [2025-10-20T18:21:30.651399] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-07. [2025-10-20T18:21:30.742993] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:21:30.840821] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:21:31.451436] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-06. [2025-10-20T18:21:32.251366] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-06. [2025-10-20T18:21:32.685920] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T18:21:32.917925] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:21:33.051422] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-08. [2025-10-20T18:21:33.851422] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-09. [2025-10-20T18:21:34.393791] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-06 (2.14 seconds). [2025-10-20T18:21:34.574615] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:21:34.651410] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-07. [2025-10-20T18:21:35.264749] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:21:35.451431] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-09. [2025-10-20T18:21:36.038873] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:21:36.251436] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-08. [2025-10-20T18:21:36.667144] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T18:21:37.051382] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-07. [2025-10-20T18:21:37.343892] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:21:37.851380] [INFO ] [REMOTE ] DOV: Fetching primary data for 2025-10. [2025-10-20T18:21:38.651563] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-09. [2025-10-20T18:21:39.232971] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-10-20T18:21:39.269135] [INFO ] [REMOTE ] DOV: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:21:39.276155] [INFO ] [REMOTE ] DOV: Full fetch complete (65938 records). [2025-10-20T18:21:39.276261] [DEBUG] [VALIDATOR ] DOV: Validating primary data. [2025-10-20T18:21:39.458559] [INFO ] [REMOTE ] DOLE: Fetching primary data for 2025-10. [2025-10-20T18:21:39.676168] [DEBUG] [VALIDATOR ] DOV: All expected trading days are present. [2025-10-20T18:21:39.676262] [DEBUG] [VALIDATOR ] DOV: Primary data validation passed. [2025-10-20T18:21:39.677160] [INFO ] [MAIN ] DOV: Fetched primary data is valid. [2025-10-20T18:21:39.677226] [DEBUG] [LOCAL ] DOV: Saving primary data. [2025-10-20T18:21:39.678632] [DEBUG] [LOCAL ] DOV: Trimmed 65938 records to 45575 (keeping 200 days of data) [2025-10-20T18:21:39.956437] [DEBUG] [LOCAL ] DOV: Primary data saved (45575 records). [2025-10-20T18:21:39.956541] [INFO ] [MAIN ] DOV: Full history saved. [2025-10-20T18:21:39.956617] [INFO ] [MAIN ] Update Progress: 641/2367 symbols completed. DOV took 27.4s. Estimated remaining: 3h 39m 42s. Rate: 7.9 symbols/min [2025-10-20T18:21:40.126596] [INFO ] [MAIN ] DPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:40.126850] [DEBUG] [LOCAL ] DPG: No existing data file found in primary storage. [2025-10-20T18:21:40.126956] [WARN ] [MAIN ] DPG: No existing data found, performing full refresh instead. [2025-10-20T18:21:40.145109] [INFO ] [MAIN ] DPG: Wiping existing primary data before full refresh. [2025-10-20T18:21:40.145282] [INFO ] [LOCAL ] DPG: Starting deletion of primary data. [2025-10-20T18:21:40.145558] [DEBUG] [LOCAL ] DPG: No primary data file found to delete. [2025-10-20T18:21:40.146008] [INFO ] [MAIN ] DPG: Starting full primary refresh. [2025-10-20T18:21:40.146813] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:21:40.256756] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-08. [2025-10-20T18:21:40.508506] [INFO ] [REMOTE ] DPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:40.928073] [INFO ] [REMOTE ] DOLE: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T18:21:40.934583] [INFO ] [REMOTE ] DOLE: Full fetch complete (54624 records). [2025-10-20T18:21:40.934762] [DEBUG] [VALIDATOR ] DOLE: Validating primary data. [2025-10-20T18:21:41.056518] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-01. [2025-10-20T18:21:41.340679] [DEBUG] [VALIDATOR ] DOLE: All expected trading days are present. [2025-10-20T18:21:41.340780] [DEBUG] [VALIDATOR ] DOLE: Primary data validation passed. [2025-10-20T18:21:41.341475] [INFO ] [MAIN ] DOLE: Fetched primary data is valid. [2025-10-20T18:21:41.341515] [DEBUG] [LOCAL ] DOLE: Saving primary data. [2025-10-20T18:21:41.342782] [DEBUG] [LOCAL ] DOLE: Trimmed 54624 records to 39134 (keeping 200 days of data) [2025-10-20T18:21:41.572890] [DEBUG] [LOCAL ] DOLE: Primary data saved (39134 records). [2025-10-20T18:21:41.573025] [INFO ] [MAIN ] DOLE: Full history saved. [2025-10-20T18:21:41.573281] [INFO ] [MAIN ] Update Progress: 642/2367 symbols completed. DOLE took 29.0s. Estimated remaining: 3h 39m 18s. Rate: 7.9 symbols/min [2025-10-20T18:21:41.742664] [INFO ] [MAIN ] DQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:41.786185] [DEBUG] [LOCAL ] DQ: Loaded 44434 existing records from primary storage. [2025-10-20T18:21:41.786528] [INFO ] [MAIN ] DQ: Loaded 44434 existing records. [2025-10-20T18:21:41.851510] [INFO ] [REMOTE ] DOUG: Fetching primary data for 2025-10. [2025-10-20T18:21:41.968837] [WARN ] [MAIN ] DQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:41.969051] [INFO ] [MAIN ] DQ: Wiping existing primary data before full refresh. [2025-10-20T18:21:41.969120] [INFO ] [LOCAL ] DQ: Starting deletion of primary data. [2025-10-20T18:21:41.969507] [DEBUG] [LOCAL ] DQ: Deleted primary data file. [2025-10-20T18:21:41.969572] [INFO ] [MAIN ] DQ: Starting full primary refresh. [2025-10-20T18:21:42.233969] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:21:42.347760] [INFO ] [REMOTE ] DQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:42.453842] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T18:21:42.651383] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-01. [2025-10-20T18:21:43.310973] [INFO ] [REMOTE ] DOUG: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:21:43.315629] [INFO ] [REMOTE ] DOUG: Full fetch complete (50225 records). [2025-10-20T18:21:43.315799] [DEBUG] [VALIDATOR ] DOUG: Validating primary data. [2025-10-20T18:21:43.466458] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-02. [2025-10-20T18:21:43.708818] [DEBUG] [VALIDATOR ] DOUG: All expected trading days are present. [2025-10-20T18:21:43.708945] [DEBUG] [VALIDATOR ] DOUG: Primary data validation passed. [2025-10-20T18:21:43.709640] [INFO ] [MAIN ] DOUG: Fetched primary data is valid. [2025-10-20T18:21:43.709722] [DEBUG] [LOCAL ] DOUG: Saving primary data. [2025-10-20T18:21:43.710745] [DEBUG] [LOCAL ] DOUG: Trimmed 50225 records to 33965 (keeping 200 days of data) [2025-10-20T18:21:43.903281] [DEBUG] [LOCAL ] DOUG: Primary data saved (33965 records). [2025-10-20T18:21:43.903479] [INFO ] [MAIN ] DOUG: Full history saved. [2025-10-20T18:21:43.903580] [INFO ] [MAIN ] Update Progress: 643/2367 symbols completed. DOUG took 31.3s. Estimated remaining: 3h 38m 56s. Rate: 7.9 symbols/min [2025-10-20T18:21:44.072122] [INFO ] [MAIN ] DRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:44.108153] [DEBUG] [LOCAL ] DRD: Loaded 38427 existing records from primary storage. [2025-10-20T18:21:44.108432] [INFO ] [MAIN ] DRD: Loaded 38427 existing records. [2025-10-20T18:21:44.252487] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-09. [2025-10-20T18:21:44.289768] [WARN ] [MAIN ] DRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:44.291080] [INFO ] [MAIN ] DRD: Wiping existing primary data before full refresh. [2025-10-20T18:21:44.291183] [INFO ] [LOCAL ] DRD: Starting deletion of primary data. [2025-10-20T18:21:44.291631] [DEBUG] [LOCAL ] DRD: Deleted primary data file. [2025-10-20T18:21:44.291809] [INFO ] [MAIN ] DRD: Starting full primary refresh. [2025-10-20T18:21:44.411782] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:21:44.457583] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T18:21:44.702515] [INFO ] [REMOTE ] DRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:45.051372] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-01. [2025-10-20T18:21:45.851353] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-02. [2025-10-20T18:21:46.297852] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T18:21:46.550921] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:21:46.651372] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-02. [2025-10-20T18:21:47.275875] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T18:21:47.451387] [INFO ] [REMOTE ] DOW: Fetching primary data for 2025-10. [2025-10-20T18:21:48.033999] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:21:48.251391] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-03. [2025-10-20T18:21:49.051369] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-03. [2025-10-20T18:21:49.523799] [INFO ] [REMOTE ] DOW: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T18:21:49.532201] [INFO ] [REMOTE ] DOW: Full fetch complete (77910 records). [2025-10-20T18:21:49.532340] [DEBUG] [VALIDATOR ] DOW: Validating primary data. [2025-10-20T18:21:49.855119] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-03. [2025-10-20T18:21:49.983751] [DEBUG] [VALIDATOR ] DOW: All expected trading days are present. [2025-10-20T18:21:49.977156] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T18:21:49.984050] [DEBUG] [VALIDATOR ] DOW: Primary data validation passed. [2025-10-20T18:21:49.991889] [INFO ] [MAIN ] DOW: Fetched primary data is valid. [2025-10-20T18:21:49.990126] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:21:50.000980] [DEBUG] [LOCAL ] DOW: Saving primary data. [2025-10-20T18:21:50.002917] [DEBUG] [LOCAL ] DOW: Trimmed 77910 records to 53733 (keeping 200 days of data) [2025-10-20T18:21:50.324500] [DEBUG] [LOCAL ] DOW: Primary data saved (53733 records). [2025-10-20T18:21:50.324631] [INFO ] [MAIN ] DOW: Full history saved. [2025-10-20T18:21:50.324749] [INFO ] [MAIN ] Update Progress: 644/2367 symbols completed. DOW took 37.7s. Estimated remaining: 3h 38m 46s. Rate: 7.9 symbols/min [2025-10-20T18:21:50.492423] [INFO ] [MAIN ] DRH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:21:50.538389] [DEBUG] [LOCAL ] DRH: Loaded 51122 existing records from primary storage. [2025-10-20T18:21:50.538722] [INFO ] [MAIN ] DRH: Loaded 51122 existing records. [2025-10-20T18:21:50.656538] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-04. [2025-10-20T18:21:50.717739] [WARN ] [MAIN ] DRH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:21:50.717934] [INFO ] [MAIN ] DRH: Wiping existing primary data before full refresh. [2025-10-20T18:21:50.717980] [INFO ] [LOCAL ] DRH: Starting deletion of primary data. [2025-10-20T18:21:50.718472] [DEBUG] [LOCAL ] DRH: Deleted primary data file. [2025-10-20T18:21:50.718528] [INFO ] [MAIN ] DRH: Starting full primary refresh. [2025-10-20T18:21:51.079010] [INFO ] [REMOTE ] DRH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:21:51.100520] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:21:51.413268] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T18:21:51.451356] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-04. [2025-10-20T18:21:52.251394] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-04. [2025-10-20T18:21:52.932370] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:21:53.051391] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-05. [2025-10-20T18:21:53.851376] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-01. [2025-10-20T18:21:53.879074] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:21:54.132486] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:21:54.651362] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-06. [2025-10-20T18:21:55.352056] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:21:55.451368] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-05. [2025-10-20T18:21:56.070057] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:21:56.251383] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-07. [2025-10-20T18:21:56.930890] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:21:57.051364] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-06. [2025-10-20T18:21:57.804464] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:21:57.851378] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-08. [2025-10-20T18:21:57.991674] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:21:58.651417] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-07. [2025-10-20T18:21:59.392253] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:21:59.451406] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-09. [2025-10-20T18:22:00.195284] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T18:22:00.251349] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-08. [2025-10-20T18:22:01.051368] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-05. [2025-10-20T18:22:01.549053] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T18:22:01.851383] [INFO ] [REMOTE ] DRD: Fetching primary data for 2025-10. [2025-10-20T18:22:02.070888] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T18:22:02.205387] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:22:02.651419] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-06. [2025-10-20T18:22:03.451402] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-02. [2025-10-20T18:22:03.484220] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:22:03.751993] [INFO ] [REMOTE ] DRD: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T18:22:03.758923] [INFO ] [REMOTE ] DRD: Full fetch complete (51831 records). [2025-10-20T18:22:03.759078] [DEBUG] [VALIDATOR ] DRD: Validating primary data. [2025-10-20T18:22:04.143938] [DEBUG] [VALIDATOR ] DRD: All expected trading days are present. [2025-10-20T18:22:04.144024] [DEBUG] [VALIDATOR ] DRD: Primary data validation passed. [2025-10-20T18:22:04.144785] [INFO ] [MAIN ] DRD: Fetched primary data is valid. [2025-10-20T18:22:04.144848] [DEBUG] [LOCAL ] DRD: Saving primary data. [2025-10-20T18:22:04.146068] [DEBUG] [LOCAL ] DRD: Trimmed 51831 records to 37805 (keeping 200 days of data) [2025-10-20T18:22:04.251366] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-07. [2025-10-20T18:22:04.374911] [DEBUG] [LOCAL ] DRD: Primary data saved (37805 records). [2025-10-20T18:22:04.375039] [INFO ] [MAIN ] DRD: Full history saved. [2025-10-20T18:22:04.375114] [INFO ] [MAIN ] Update Progress: 645/2367 symbols completed. DRD took 20.5s. Estimated remaining: 3h 38m 55s. Rate: 7.9 symbols/min [2025-10-20T18:22:04.539895] [INFO ] [MAIN ] DRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:04.582770] [DEBUG] [LOCAL ] DRI: Loaded 48386 existing records from primary storage. [2025-10-20T18:22:04.583061] [INFO ] [MAIN ] DRI: Loaded 48386 existing records. [2025-10-20T18:22:04.761045] [WARN ] [MAIN ] DRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:04.761353] [INFO ] [MAIN ] DRI: Wiping existing primary data before full refresh. [2025-10-20T18:22:04.761411] [INFO ] [LOCAL ] DRI: Starting deletion of primary data. [2025-10-20T18:22:04.769783] [DEBUG] [LOCAL ] DRI: Deleted primary data file. [2025-10-20T18:22:04.783271] [INFO ] [MAIN ] DRI: Starting full primary refresh. [2025-10-20T18:22:04.816232] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:22:05.059024] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-03. [2025-10-20T18:22:05.148499] [INFO ] [REMOTE ] DRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:05.245321] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:22:05.851412] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-08. [2025-10-20T18:22:06.621480] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:22:06.642913] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T18:22:06.651318] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-09. [2025-10-20T18:22:07.451377] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-01. [2025-10-20T18:22:07.577935] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T18:22:08.251487] [INFO ] [REMOTE ] DPG: Fetching primary data for 2025-10. [2025-10-20T18:22:09.051355] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-09. [2025-10-20T18:22:09.132206] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:22:09.169376] [INFO ] [REMOTE ] DPG: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:22:09.171826] [INFO ] [REMOTE ] DPG: Full fetch complete (19068 records). [2025-10-20T18:22:09.171930] [DEBUG] [VALIDATOR ] DPG: Validating primary data. [2025-10-20T18:22:09.533573] [DEBUG] [VALIDATOR ] DPG: All expected trading days are present. [2025-10-20T18:22:09.533667] [DEBUG] [VALIDATOR ] DPG: Primary data validation passed. [2025-10-20T18:22:09.534074] [INFO ] [MAIN ] DPG: Fetched primary data is valid. [2025-10-20T18:22:09.534115] [DEBUG] [LOCAL ] DPG: Saving primary data. [2025-10-20T18:22:09.534619] [DEBUG] [LOCAL ] DPG: Trimmed 19068 records to 13502 (keeping 200 days of data) [2025-10-20T18:22:09.613336] [DEBUG] [LOCAL ] DPG: Primary data saved (13502 records). [2025-10-20T18:22:09.613456] [INFO ] [MAIN ] DPG: Full history saved. [2025-10-20T18:22:09.613524] [INFO ] [MAIN ] Update Progress: 646/2367 symbols completed. DPG took 29.7s. Estimated remaining: 3h 38m 41s. Rate: 7.9 symbols/min [2025-10-20T18:22:09.777381] [INFO ] [MAIN ] DRLL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:09.786639] [DEBUG] [LOCAL ] DRLL: Loaded 9219 existing records from primary storage. [2025-10-20T18:22:09.786852] [INFO ] [MAIN ] DRLL: Loaded 9219 existing records. [2025-10-20T18:22:09.859786] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-04. [2025-10-20T18:22:09.964522] [WARN ] [MAIN ] DRLL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:09.964788] [INFO ] [MAIN ] DRLL: Wiping existing primary data before full refresh. [2025-10-20T18:22:09.964841] [INFO ] [LOCAL ] DRLL: Starting deletion of primary data. [2025-10-20T18:22:09.964999] [DEBUG] [LOCAL ] DRLL: Deleted primary data file. [2025-10-20T18:22:09.965037] [INFO ] [MAIN ] DRLL: Starting full primary refresh. [2025-10-20T18:22:10.332291] [INFO ] [REMOTE ] DRLL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:10.651427] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-02. [2025-10-20T18:22:10.870081] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:22:11.451393] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-01. [2025-10-20T18:22:11.632767] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:22:12.034833] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:22:12.085558] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:22:12.251420] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-02. [2025-10-20T18:22:12.875639] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T18:22:13.051378] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-05. [2025-10-20T18:22:13.851435] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-03. [2025-10-20T18:22:14.651428] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-03. [2025-10-20T18:22:14.722330] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:22:15.451365] [INFO ] [REMOTE ] DQ: Fetching primary data for 2025-10. [2025-10-20T18:22:15.505433] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:22:15.609554] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T18:22:16.251360] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-04. [2025-10-20T18:22:17.051389] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-06. [2025-10-20T18:22:17.104015] [INFO ] [REMOTE ] DQ: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T18:22:17.111085] [INFO ] [REMOTE ] DQ: Full fetch complete (63642 records). [2025-10-20T18:22:17.111217] [DEBUG] [VALIDATOR ] DQ: Validating primary data. [2025-10-20T18:22:17.228086] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:22:17.519978] [DEBUG] [VALIDATOR ] DQ: All expected trading days are present. [2025-10-20T18:22:17.520066] [DEBUG] [VALIDATOR ] DQ: Primary data validation passed. [2025-10-20T18:22:17.520843] [INFO ] [MAIN ] DQ: Fetched primary data is valid. [2025-10-20T18:22:17.520907] [DEBUG] [LOCAL ] DQ: Saving primary data. [2025-10-20T18:22:17.522180] [DEBUG] [LOCAL ] DQ: Trimmed 63642 records to 43901 (keeping 200 days of data) [2025-10-20T18:22:17.771836] [DEBUG] [LOCAL ] DQ: Primary data saved (43901 records). [2025-10-20T18:22:17.771970] [INFO ] [MAIN ] DQ: Full history saved. [2025-10-20T18:22:17.772305] [INFO ] [MAIN ] Update Progress: 647/2367 symbols completed. DQ took 36.2s. Estimated remaining: 3h 38m 35s. Rate: 7.9 symbols/min [2025-10-20T18:22:17.860467] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-04. [2025-10-20T18:22:17.940978] [INFO ] [MAIN ] DSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:17.973770] [DEBUG] [LOCAL ] DSL: Loaded 29410 existing records from primary storage. [2025-10-20T18:22:17.974055] [INFO ] [MAIN ] DSL: Loaded 29410 existing records. [2025-10-20T18:22:18.151142] [WARN ] [MAIN ] DSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:18.151411] [INFO ] [MAIN ] DSL: Wiping existing primary data before full refresh. [2025-10-20T18:22:18.151467] [INFO ] [LOCAL ] DSL: Starting deletion of primary data. [2025-10-20T18:22:18.151743] [DEBUG] [LOCAL ] DSL: Deleted primary data file. [2025-10-20T18:22:18.151816] [INFO ] [MAIN ] DSL: Starting full primary refresh. [2025-10-20T18:22:18.515216] [INFO ] [REMOTE ] DSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:18.651395] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-01. [2025-10-20T18:22:18.704358] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:22:19.451393] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-05. [2025-10-20T18:22:19.711014] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:22:19.819276] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:22:20.236709] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:22:20.251336] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-06. [2025-10-20T18:22:20.810905] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:22:21.051374] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-07. [2025-10-20T18:22:21.851397] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-07. [2025-10-20T18:22:22.651379] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-05. [2025-10-20T18:22:22.673806] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:22:22.725724] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T18:22:23.451416] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-02. [2025-10-20T18:22:24.251495] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-08. [2025-10-20T18:22:24.341548] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:22:24.652148] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:22:24.712354] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:22:25.051407] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-06. [2025-10-20T18:22:25.851419] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-08. [2025-10-20T18:22:26.651403] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-09. [2025-10-20T18:22:26.851086] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T18:22:27.237035] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T18:22:27.256946] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T18:22:27.451416] [INFO ] [REMOTE ] DRLL: Fetching primary data for 2025-10. [2025-10-20T18:22:28.237736] [INFO ] [REMOTE ] DRLL: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:22:28.239370] [INFO ] [REMOTE ] DRLL: Full fetch complete (14595 records). [2025-10-20T18:22:28.239448] [DEBUG] [VALIDATOR ] DRLL: Validating primary data. [2025-10-20T18:22:28.260060] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-07. [2025-10-20T18:22:28.633035] [DEBUG] [VALIDATOR ] DRLL: All expected trading days are present. [2025-10-20T18:22:28.633118] [DEBUG] [VALIDATOR ] DRLL: Primary data validation passed. [2025-10-20T18:22:28.633465] [INFO ] [MAIN ] DRLL: Fetched primary data is valid. [2025-10-20T18:22:28.633505] [DEBUG] [LOCAL ] DRLL: Saving primary data. [2025-10-20T18:22:28.634167] [DEBUG] [LOCAL ] DRLL: Trimmed 14595 records to 8856 (keeping 200 days of data) [2025-10-20T18:22:28.688513] [DEBUG] [LOCAL ] DRLL: Primary data saved (8856 records). [2025-10-20T18:22:28.688667] [INFO ] [MAIN ] DRLL: Full history saved. [2025-10-20T18:22:28.688765] [INFO ] [MAIN ] Update Progress: 648/2367 symbols completed. DRLL took 19.1s. Estimated remaining: 3h 38m 36s. Rate: 7.9 symbols/min [2025-10-20T18:22:28.868480] [INFO ] [MAIN ] DSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:28.868727] [DEBUG] [LOCAL ] DSM: No existing data file found in primary storage. [2025-10-20T18:22:28.868790] [WARN ] [MAIN ] DSM: No existing data found, performing full refresh instead. [2025-10-20T18:22:28.868924] [INFO ] [MAIN ] DSM: Wiping existing primary data before full refresh. [2025-10-20T18:22:28.868976] [INFO ] [LOCAL ] DSM: Starting deletion of primary data. [2025-10-20T18:22:28.869029] [DEBUG] [LOCAL ] DSM: No primary data file found to delete. [2025-10-20T18:22:28.869070] [INFO ] [MAIN ] DSM: Starting full primary refresh. [2025-10-20T18:22:29.061779] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-09. [2025-10-20T18:22:29.260085] [INFO ] [REMOTE ] DSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:29.851435] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-01. [2025-10-20T18:22:30.009159] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:22:30.651400] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-08. [2025-10-20T18:22:30.977858] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:22:31.040589] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T18:22:31.451368] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-03. [2025-10-20T18:22:32.229545] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:22:32.251333] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-09. [2025-10-20T18:22:32.296793] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:22:33.051380] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-02. [2025-10-20T18:22:33.851397] [INFO ] [REMOTE ] DRH: Fetching primary data for 2025-10. [2025-10-20T18:22:34.212167] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:22:34.279052] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T18:22:34.651399] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-03. [2025-10-20T18:22:35.274015] [INFO ] [REMOTE ] DRH: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:22:35.281466] [INFO ] [REMOTE ] DRH: Full fetch complete (72007 records). [2025-10-20T18:22:35.281623] [DEBUG] [VALIDATOR ] DRH: Validating primary data. [2025-10-20T18:22:35.456481] [INFO ] [REMOTE ] DRI: Fetching primary data for 2025-10. [2025-10-20T18:22:35.672295] [DEBUG] [VALIDATOR ] DRH: All expected trading days are present. [2025-10-20T18:22:35.672374] [DEBUG] [VALIDATOR ] DRH: Primary data validation passed. [2025-10-20T18:22:35.673362] [INFO ] [MAIN ] DRH: Fetched primary data is valid. [2025-10-20T18:22:35.673498] [DEBUG] [LOCAL ] DRH: Saving primary data. [2025-10-20T18:22:35.675038] [DEBUG] [LOCAL ] DRH: Trimmed 72007 records to 50315 (keeping 200 days of data) [2025-10-20T18:22:35.792042] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:22:35.993111] [DEBUG] [LOCAL ] DRH: Primary data saved (50315 records). [2025-10-20T18:22:35.993293] [INFO ] [MAIN ] DRH: Full history saved. [2025-10-20T18:22:35.993402] [INFO ] [MAIN ] Update Progress: 649/2367 symbols completed. DRH took 45.7s. Estimated remaining: 3h 38m 28s. Rate: 7.9 symbols/min [2025-10-20T18:22:36.163889] [INFO ] [MAIN ] DSMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:36.184115] [INFO ] [REMOTE ] DRI: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T18:22:36.193384] [DEBUG] [LOCAL ] DSMC: Loaded 1866 existing records from primary storage. [2025-10-20T18:22:36.193738] [INFO ] [MAIN ] DSMC: Loaded 1866 existing records. [2025-10-20T18:22:36.251526] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-04. [2025-10-20T18:22:36.238370] [INFO ] [REMOTE ] DRI: Full fetch complete (69569 records). [2025-10-20T18:22:36.273313] [DEBUG] [VALIDATOR ] DRI: Validating primary data. [2025-10-20T18:22:36.531059] [WARN ] [MAIN ] DSMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:36.550041] [INFO ] [MAIN ] DSMC: Wiping existing primary data before full refresh. [2025-10-20T18:22:36.550664] [INFO ] [LOCAL ] DSMC: Starting deletion of primary data. [2025-10-20T18:22:36.567301] [DEBUG] [LOCAL ] DSMC: Deleted primary data file. [2025-10-20T18:22:36.587182] [INFO ] [MAIN ] DSMC: Starting full primary refresh. [2025-10-20T18:22:37.057506] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-04. [2025-10-20T18:22:37.071846] [DEBUG] [VALIDATOR ] DRI: All expected trading days are present. [2025-10-20T18:22:37.076062] [DEBUG] [VALIDATOR ] DRI: Primary data validation passed. [2025-10-20T18:22:37.077135] [INFO ] [MAIN ] DRI: Fetched primary data is valid. [2025-10-20T18:22:37.077231] [DEBUG] [LOCAL ] DRI: Saving primary data. [2025-10-20T18:22:37.139859] [DEBUG] [LOCAL ] DRI: Trimmed 69569 records to 47643 (keeping 200 days of data) [2025-10-20T18:22:37.160602] [INFO ] [REMOTE ] DSMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:37.458538] [DEBUG] [LOCAL ] DRI: Primary data saved (47643 records). [2025-10-20T18:22:37.458695] [INFO ] [MAIN ] DRI: Full history saved. [2025-10-20T18:22:37.458796] [INFO ] [MAIN ] Update Progress: 650/2367 symbols completed. DRI took 33.1s. Estimated remaining: 3h 38m 4s. Rate: 7.9 symbols/min [2025-10-20T18:22:37.626797] [INFO ] [MAIN ] DSU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:22:37.650306] [DEBUG] [LOCAL ] DSU: Loaded 23392 existing records from primary storage. [2025-10-20T18:22:37.650553] [INFO ] [MAIN ] DSU: Loaded 23392 existing records. [2025-10-20T18:22:37.830962] [WARN ] [MAIN ] DSU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:22:37.832153] [INFO ] [MAIN ] DSU: Wiping existing primary data before full refresh. [2025-10-20T18:22:37.832301] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-10-20T18:22:37.832740] [DEBUG] [LOCAL ] DSU: Deleted primary data file. [2025-10-20T18:22:37.839543] [INFO ] [MAIN ] DSU: Starting full primary refresh. [2025-10-20T18:22:37.864627] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-01. [2025-10-20T18:22:37.887569] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:22:38.043813] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:22:38.273922] [INFO ] [REMOTE ] DSU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:22:38.291997] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:22:38.651378] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-05. [2025-10-20T18:22:39.451371] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-02. [2025-10-20T18:22:39.518057] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T18:22:39.943997] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T18:22:40.251369] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-03. [2025-10-20T18:22:40.722351] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T18:22:41.051366] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-04. [2025-10-20T18:22:41.617550] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T18:22:41.851398] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-05. [2025-10-20T18:22:42.320094] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:22:42.651385] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-05. [2025-10-20T18:22:43.451351] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-01. [2025-10-20T18:22:43.710144] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:22:44.251380] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-06. [2025-10-20T18:22:44.503995] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T18:22:44.522260] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-10-20T18:22:45.051384] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-02. [2025-10-20T18:22:45.851380] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-06. [2025-10-20T18:22:46.098014] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:22:46.651351] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-03. [2025-10-20T18:22:46.727317] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T18:22:47.451362] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-07. [2025-10-20T18:22:47.657262] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:22:48.251378] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-07. [2025-10-20T18:22:48.365084] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T18:22:48.905276] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T18:22:49.051366] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-08. [2025-10-20T18:22:49.851390] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-04. [2025-10-20T18:22:49.915886] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T18:22:50.651378] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-06. [2025-10-20T18:22:51.013161] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:22:51.451379] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-05. [2025-10-20T18:22:51.860526] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T18:22:52.251378] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-08. [2025-10-20T18:22:53.051372] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-09. [2025-10-20T18:22:53.851371] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-07. [2025-10-20T18:23:04.324247] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-08 (12.07 seconds). [2025-10-20T18:23:04.325992] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-09. [2025-10-20T18:23:05.688476] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-05 (14.24 seconds). [2025-10-20T18:23:05.692890] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-06. [2025-10-20T18:23:05.833279] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-09 (12.78 seconds). [2025-10-20T18:23:05.836876] [INFO ] [REMOTE ] DSM: Fetching primary data for 2025-10. [2025-10-20T18:23:06.400805] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T18:23:06.402073] [INFO ] [REMOTE ] DSMC: Fetching primary data for 2025-10. [2025-10-20T18:23:06.510179] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-07 (12.66 seconds). [2025-10-20T18:23:06.518209] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-08. [2025-10-20T18:23:07.330632] [INFO ] [REMOTE ] DSMC: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:23:07.331482] [INFO ] [REMOTE ] DSMC: Full fetch complete (2870 records). [2025-10-20T18:23:07.331549] [DEBUG] [VALIDATOR ] DSMC: Validating primary data. [2025-10-20T18:23:07.628417] [INFO ] [REMOTE ] DSM: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T18:23:07.652078] [INFO ] [REMOTE ] DSM: Full fetch complete (23854 records). [2025-10-20T18:23:07.682932] [DEBUG] [VALIDATOR ] DSM: Validating primary data. [2025-10-20T18:23:07.715922] [DEBUG] [VALIDATOR ] DSMC: All expected trading days are present. [2025-10-20T18:23:07.759131] [DEBUG] [VALIDATOR ] DSMC: Primary data validation passed. [2025-10-20T18:23:07.772544] [INFO ] [MAIN ] DSMC: Fetched primary data is valid. [2025-10-20T18:23:07.795966] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T18:23:07.795052] [DEBUG] [LOCAL ] DSMC: Saving primary data. [2025-10-20T18:23:07.828474] [DEBUG] [LOCAL ] DSMC: Trimmed 2870 records to 1838 (keeping 200 days of data) [2025-10-20T18:23:07.817948] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-07. [2025-10-20T18:23:07.891886] [DEBUG] [LOCAL ] DSMC: Primary data saved (1838 records). [2025-10-20T18:23:07.908939] [INFO ] [MAIN ] DSMC: Full history saved. [2025-10-20T18:23:07.914190] [INFO ] [MAIN ] Update Progress: 651/2367 symbols completed. DSMC took 31.9s. Estimated remaining: 3h 38m 56s. Rate: 7.8 symbols/min [2025-10-20T18:23:08.293362] [INFO ] [MAIN ] DSX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:08.380589] [DEBUG] [VALIDATOR ] DSM: All expected trading days are present. [2025-10-20T18:23:08.380704] [DEBUG] [VALIDATOR ] DSM: Primary data validation passed. [2025-10-20T18:23:08.381173] [INFO ] [MAIN ] DSM: Fetched primary data is valid. [2025-10-20T18:23:08.386729] [DEBUG] [LOCAL ] DSM: Saving primary data. [2025-10-20T18:23:08.387777] [DEBUG] [LOCAL ] DSM: Trimmed 23854 records to 14073 (keeping 200 days of data) [2025-10-20T18:23:08.460694] [DEBUG] [LOCAL ] DSX: Loaded 24294 existing records from primary storage. [2025-10-20T18:23:08.466388] [INFO ] [MAIN ] DSX: Loaded 24294 existing records. [2025-10-20T18:23:08.584773] [DEBUG] [LOCAL ] DSM: Primary data saved (14073 records). [2025-10-20T18:23:08.590418] [INFO ] [MAIN ] DSM: Full history saved. [2025-10-20T18:23:08.606091] [INFO ] [MAIN ] Update Progress: 652/2367 symbols completed. DSM took 39.9s. Estimated remaining: 3h 38m 30s. Rate: 7.8 symbols/min [2025-10-20T18:23:08.679677] [WARN ] [MAIN ] DSX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:08.699833] [INFO ] [MAIN ] DSX: Wiping existing primary data before full refresh. [2025-10-20T18:23:08.731683] [INFO ] [LOCAL ] DSX: Starting deletion of primary data. [2025-10-20T18:23:08.774453] [DEBUG] [LOCAL ] DSX: Deleted primary data file. [2025-10-20T18:23:08.774854] [INFO ] [MAIN ] DSX: Starting full primary refresh. [2025-10-20T18:23:08.850238] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-08 (2.33 seconds). [2025-10-20T18:23:08.982210] [INFO ] [MAIN ] DT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:08.985986] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-09. [2025-10-20T18:23:08.986253] [DEBUG] [LOCAL ] DT: No existing data file found in primary storage. [2025-10-20T18:23:08.993142] [WARN ] [MAIN ] DT: No existing data found, performing full refresh instead. [2025-10-20T18:23:09.081664] [INFO ] [MAIN ] DT: Wiping existing primary data before full refresh. [2025-10-20T18:23:09.106856] [INFO ] [LOCAL ] DT: Starting deletion of primary data. [2025-10-20T18:23:09.135006] [DEBUG] [LOCAL ] DT: No primary data file found to delete. [2025-10-20T18:23:09.146678] [INFO ] [MAIN ] DT: Starting full primary refresh. [2025-10-20T18:23:09.229326] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:23:09.314019] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-08. [2025-10-20T18:23:09.388797] [INFO ] [REMOTE ] DSX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:09.410668] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-01. [2025-10-20T18:23:09.669839] [INFO ] [REMOTE ] DT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:09.670248] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-01. [2025-10-20T18:23:10.321391] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:23:10.328606] [INFO ] [REMOTE ] DSL: Fetching primary data for 2025-10. [2025-10-20T18:23:10.499910] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T18:23:10.505018] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-09. [2025-10-20T18:23:10.580452] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:23:10.585935] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-02. [2025-10-20T18:23:11.375963] [INFO ] [REMOTE ] DSL: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:23:11.382023] [INFO ] [REMOTE ] DSL: Full fetch complete (41732 records). [2025-10-20T18:23:11.382177] [DEBUG] [VALIDATOR ] DSL: Validating primary data. [2025-10-20T18:23:11.576704] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:23:11.637164] [INFO ] [REMOTE ] DSU: Fetching primary data for 2025-10. [2025-10-20T18:23:11.725619] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:23:11.791520] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-03. [2025-10-20T18:23:11.917635] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-01 (2.25 seconds). [2025-10-20T18:23:11.946203] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-02. [2025-10-20T18:23:11.947153] [DEBUG] [VALIDATOR ] DSL: All expected trading days are present. [2025-10-20T18:23:11.947215] [DEBUG] [VALIDATOR ] DSL: Primary data validation passed. [2025-10-20T18:23:11.947881] [INFO ] [MAIN ] DSL: Fetched primary data is valid. [2025-10-20T18:23:11.948020] [DEBUG] [LOCAL ] DSL: Saving primary data. [2025-10-20T18:23:11.949142] [DEBUG] [LOCAL ] DSL: Trimmed 41732 records to 29017 (keeping 200 days of data) [2025-10-20T18:23:12.135051] [DEBUG] [LOCAL ] DSL: Primary data saved (29017 records). [2025-10-20T18:23:12.135195] [INFO ] [MAIN ] DSL: Full history saved. [2025-10-20T18:23:12.135628] [INFO ] [MAIN ] Update Progress: 653/2367 symbols completed. DSL took 54.4s. Estimated remaining: 3h 38m 12s. Rate: 7.9 symbols/min [2025-10-20T18:23:12.340374] [INFO ] [MAIN ] DTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:12.349544] [DEBUG] [LOCAL ] DTB: Loaded 5028 existing records from primary storage. [2025-10-20T18:23:12.349799] [INFO ] [MAIN ] DTB: Loaded 5028 existing records. [2025-10-20T18:23:12.613256] [WARN ] [MAIN ] DTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:12.613476] [INFO ] [MAIN ] DTB: Wiping existing primary data before full refresh. [2025-10-20T18:23:12.613545] [INFO ] [LOCAL ] DTB: Starting deletion of primary data. [2025-10-20T18:23:12.613741] [DEBUG] [LOCAL ] DTB: Deleted primary data file. [2025-10-20T18:23:12.613798] [INFO ] [MAIN ] DTB: Starting full primary refresh. [2025-10-20T18:23:13.014918] [INFO ] [REMOTE ] DSU: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:23:13.028967] [INFO ] [REMOTE ] DTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:13.029341] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-01. [2025-10-20T18:23:13.034416] [INFO ] [REMOTE ] DSU: Full fetch complete (32975 records). [2025-10-20T18:23:13.034569] [DEBUG] [VALIDATOR ] DSU: Validating primary data. [2025-10-20T18:23:13.035189] [FATAL] [VALIDATOR ] DSU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:23:13.035275] [FATAL] [MAIN ] DSU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:23:13.035340] [WARN ] [MAIN ] DSU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:23:13.035410] [INFO ] [LOCAL ] DSU: Starting deletion of primary data. [2025-10-20T18:23:13.035475] [DEBUG] [LOCAL ] DSU: No primary data file found to delete. [2025-10-20T18:23:13.035511] [INFO ] [LOCAL ] DSU: Starting deletion of derivative data. [2025-10-20T18:23:13.035691] [DEBUG] [LOCAL ] DSU: Deleted derivative data file. [2025-10-20T18:23:13.035784] [INFO ] [MAIN ] Update Progress: 654/2367 symbols completed. DSU took 35.6s. Estimated remaining: 3h 37m 47s. Rate: 7.9 symbols/min [2025-10-20T18:23:13.217026] [INFO ] [MAIN ] DTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:13.236363] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:23:13.243151] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-04. [2025-10-20T18:23:13.288747] [DEBUG] [LOCAL ] DTE: Loaded 48020 existing records from primary storage. [2025-10-20T18:23:13.289111] [INFO ] [MAIN ] DTE: Loaded 48020 existing records. [2025-10-20T18:23:13.486049] [WARN ] [MAIN ] DTE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:13.486374] [INFO ] [MAIN ] DTE: Wiping existing primary data before full refresh. [2025-10-20T18:23:13.486423] [INFO ] [LOCAL ] DTE: Starting deletion of primary data. [2025-10-20T18:23:13.513084] [DEBUG] [LOCAL ] DTE: Deleted primary data file. [2025-10-20T18:23:13.513314] [INFO ] [MAIN ] DTE: Starting full primary refresh. [2025-10-20T18:23:13.563000] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T18:23:13.642835] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-03. [2025-10-20T18:23:13.796263] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:23:13.827025] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-02. [2025-10-20T18:23:13.921945] [INFO ] [REMOTE ] DTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:13.922295] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-01. [2025-10-20T18:23:14.300371] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:23:14.306457] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-05. [2025-10-20T18:23:14.330863] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T18:23:14.332515] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-03. [2025-10-20T18:23:14.869272] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:23:14.871408] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-04. [2025-10-20T18:23:15.164528] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:23:15.176731] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-04. [2025-10-20T18:23:15.380448] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T18:23:15.390118] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-06. [2025-10-20T18:23:15.504100] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T18:23:15.506601] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-05. [2025-10-20T18:23:15.718478] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:23:16.114326] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:23:16.251346] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-06. [2025-10-20T18:23:16.329338] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:23:16.731464] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T18:23:16.983433] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T18:23:17.051420] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-05. [2025-10-20T18:23:17.851396] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-02. [2025-10-20T18:23:18.597438] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:23:18.651395] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-06. [2025-10-20T18:23:19.244394] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:23:19.451422] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-07. [2025-10-20T18:23:20.056281] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T18:23:20.251392] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-08. [2025-10-20T18:23:20.287640] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:23:20.900529] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:23:21.051410] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-03. [2025-10-20T18:23:21.851398] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-09. [2025-10-20T18:23:22.557034] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:23:22.559168] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:23:22.651377] [INFO ] [REMOTE ] DTB: Fetching primary data for 2025-10. [2025-10-20T18:23:23.258112] [INFO ] [REMOTE ] DTB: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:23:23.259406] [INFO ] [REMOTE ] DTB: Full fetch complete (6567 records). [2025-10-20T18:23:23.259476] [DEBUG] [VALIDATOR ] DTB: Validating primary data. [2025-10-20T18:23:23.456679] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-04. [2025-10-20T18:23:23.638509] [DEBUG] [VALIDATOR ] DTB: All expected trading days are present. [2025-10-20T18:23:23.638610] [DEBUG] [VALIDATOR ] DTB: Primary data validation passed. [2025-10-20T18:23:23.638843] [INFO ] [MAIN ] DTB: Fetched primary data is valid. [2025-10-20T18:23:23.638898] [DEBUG] [LOCAL ] DTB: Saving primary data. [2025-10-20T18:23:23.639364] [DEBUG] [LOCAL ] DTB: Trimmed 6567 records to 4937 (keeping 200 days of data) [2025-10-20T18:23:23.669637] [DEBUG] [LOCAL ] DTB: Primary data saved (4937 records). [2025-10-20T18:23:23.669798] [INFO ] [MAIN ] DTB: Full history saved. [2025-10-20T18:23:23.669880] [INFO ] [MAIN ] Update Progress: 655/2367 symbols completed. DTB took 11.5s. Estimated remaining: 3h 37m 47s. Rate: 7.9 symbols/min [2025-10-20T18:23:23.840441] [INFO ] [MAIN ] DTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:23.840694] [DEBUG] [LOCAL ] DTF: No existing data file found in primary storage. [2025-10-20T18:23:23.840751] [WARN ] [MAIN ] DTF: No existing data found, performing full refresh instead. [2025-10-20T18:23:23.840832] [INFO ] [MAIN ] DTF: Wiping existing primary data before full refresh. [2025-10-20T18:23:23.840867] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-10-20T18:23:23.840906] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-10-20T18:23:23.840936] [INFO ] [MAIN ] DTF: Starting full primary refresh. [2025-10-20T18:23:24.238369] [INFO ] [REMOTE ] DTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:24.251362] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-01. [2025-10-20T18:23:24.773898] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T18:23:25.051400] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-02. [2025-10-20T18:23:25.307334] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T18:23:25.359582] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T18:23:25.851364] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-03. [2025-10-20T18:23:26.278477] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:23:26.651417] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-05. [2025-10-20T18:23:27.451429] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-07. [2025-10-20T18:23:28.251352] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-07. [2025-10-20T18:23:28.349795] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:23:28.633370] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:23:29.051440] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-04. [2025-10-20T18:23:29.368988] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-04 (0.32 seconds). [2025-10-20T18:23:29.851406] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-08. [2025-10-20T18:23:30.221290] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:23:30.651424] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-05. [2025-10-20T18:23:30.956247] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-10-20T18:23:30.959312] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T18:23:31.451488] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-06. [2025-10-20T18:23:32.251426] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-06. [2025-10-20T18:23:32.655157] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:23:33.061446] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-09. [2025-10-20T18:23:33.067475] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:23:33.851407] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-07. [2025-10-20T18:23:33.980764] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T18:23:34.195247] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:23:34.651417] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-07. [2025-10-20T18:23:35.451423] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-08. [2025-10-20T18:23:36.224466] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:23:36.251352] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-08. [2025-10-20T18:23:36.907939] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:23:37.051400] [INFO ] [REMOTE ] DSX: Fetching primary data for 2025-10. [2025-10-20T18:23:37.663717] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T18:23:37.851396] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-09. [2025-10-20T18:23:38.332518] [INFO ] [REMOTE ] DSX: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:23:38.336538] [INFO ] [REMOTE ] DSX: Full fetch complete (35765 records). [2025-10-20T18:23:38.336624] [DEBUG] [VALIDATOR ] DSX: Validating primary data. [2025-10-20T18:23:38.656513] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-08. [2025-10-20T18:23:38.703163] [DEBUG] [VALIDATOR ] DSX: All expected trading days are present. [2025-10-20T18:23:38.703236] [DEBUG] [VALIDATOR ] DSX: Primary data validation passed. [2025-10-20T18:23:38.703887] [INFO ] [MAIN ] DSX: Fetched primary data is valid. [2025-10-20T18:23:38.703951] [DEBUG] [LOCAL ] DSX: Saving primary data. [2025-10-20T18:23:38.704731] [DEBUG] [LOCAL ] DSX: Trimmed 35765 records to 23744 (keeping 200 days of data) [2025-10-20T18:23:38.836451] [DEBUG] [LOCAL ] DSX: Primary data saved (23744 records). [2025-10-20T18:23:38.836572] [INFO ] [MAIN ] DSX: Full history saved. [2025-10-20T18:23:38.836666] [INFO ] [MAIN ] Update Progress: 656/2367 symbols completed. DSX took 30.9s. Estimated remaining: 3h 37m 59s. Rate: 7.8 symbols/min [2025-10-20T18:23:39.006550] [INFO ] [MAIN ] DTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:39.014561] [DEBUG] [LOCAL ] DTG: Loaded 5859 existing records from primary storage. [2025-10-20T18:23:39.014756] [INFO ] [MAIN ] DTG: Loaded 5859 existing records. [2025-10-20T18:23:39.195516] [WARN ] [MAIN ] DTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:39.195801] [INFO ] [MAIN ] DTG: Wiping existing primary data before full refresh. [2025-10-20T18:23:39.195890] [INFO ] [LOCAL ] DTG: Starting deletion of primary data. [2025-10-20T18:23:39.196130] [DEBUG] [LOCAL ] DTG: Deleted primary data file. [2025-10-20T18:23:39.196200] [INFO ] [MAIN ] DTG: Starting full primary refresh. [2025-10-20T18:23:39.295990] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T18:23:39.466560] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-09. [2025-10-20T18:23:39.585554] [INFO ] [REMOTE ] DTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:39.615527] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:23:40.251364] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-01. [2025-10-20T18:23:40.941988] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T18:23:41.051358] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-02. [2025-10-20T18:23:41.268672] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T18:23:41.663749] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:23:41.851378] [INFO ] [REMOTE ] DT: Fetching primary data for 2025-10. [2025-10-20T18:23:42.651514] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-09. [2025-10-20T18:23:43.055016] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T18:23:43.454513] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-03. [2025-10-20T18:23:43.470246] [INFO ] [REMOTE ] DT: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:23:43.481896] [INFO ] [REMOTE ] DT: Full fetch complete (76903 records). [2025-10-20T18:23:43.482061] [DEBUG] [VALIDATOR ] DT: Validating primary data. [2025-10-20T18:23:43.881731] [DEBUG] [VALIDATOR ] DT: All expected trading days are present. [2025-10-20T18:23:43.881825] [DEBUG] [VALIDATOR ] DT: Primary data validation passed. [2025-10-20T18:23:43.882767] [INFO ] [MAIN ] DT: Fetched primary data is valid. [2025-10-20T18:23:43.882836] [DEBUG] [LOCAL ] DT: Saving primary data. [2025-10-20T18:23:43.884291] [DEBUG] [LOCAL ] DT: Trimmed 76903 records to 52952 (keeping 200 days of data) [2025-10-20T18:23:44.188348] [DEBUG] [LOCAL ] DT: Primary data saved (52952 records). [2025-10-20T18:23:44.188498] [INFO ] [MAIN ] DT: Full history saved. [2025-10-20T18:23:44.188583] [INFO ] [MAIN ] Update Progress: 657/2367 symbols completed. DT took 35.6s. Estimated remaining: 3h 37m 45s. Rate: 7.9 symbols/min [2025-10-20T18:23:44.252287] [INFO ] [REMOTE ] DTF: Fetching primary data for 2025-10. [2025-10-20T18:23:44.302138] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T18:23:44.379079] [INFO ] [MAIN ] DTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:44.379264] [DEBUG] [LOCAL ] DTK: No existing data file found in primary storage. [2025-10-20T18:23:44.379314] [WARN ] [MAIN ] DTK: No existing data found, performing full refresh instead. [2025-10-20T18:23:44.379423] [INFO ] [MAIN ] DTK: Wiping existing primary data before full refresh. [2025-10-20T18:23:44.379466] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-10-20T18:23:44.379507] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-10-20T18:23:44.379540] [INFO ] [MAIN ] DTK: Starting full primary refresh. [2025-10-20T18:23:44.729061] [INFO ] [REMOTE ] DTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:44.752737] [INFO ] [REMOTE ] DTF: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T18:23:44.753854] [INFO ] [REMOTE ] DTF: Full fetch complete (1750 records). [2025-10-20T18:23:44.753947] [DEBUG] [VALIDATOR ] DTF: Validating primary data. [2025-10-20T18:23:44.948352] [WARN ] [VALIDATOR ] DTF: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-10-20T18:23:44.948533] [FATAL] [MAIN ] DTF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:23:44.948599] [WARN ] [MAIN ] DTF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:23:44.948691] [INFO ] [LOCAL ] DTF: Starting deletion of primary data. [2025-10-20T18:23:44.948757] [DEBUG] [LOCAL ] DTF: No primary data file found to delete. [2025-10-20T18:23:44.948792] [INFO ] [LOCAL ] DTF: Starting deletion of derivative data. [2025-10-20T18:23:44.948831] [DEBUG] [LOCAL ] DTF: No derivative data file found to delete. [2025-10-20T18:23:44.948883] [INFO ] [MAIN ] Update Progress: 658/2367 symbols completed. DTF took 21.3s. Estimated remaining: 3h 37m 20s. Rate: 7.9 symbols/min [2025-10-20T18:23:45.056533] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-04. [2025-10-20T18:23:45.121587] [INFO ] [MAIN ] DTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:45.162416] [DEBUG] [LOCAL ] DTM: Loaded 44538 existing records from primary storage. [2025-10-20T18:23:45.162764] [INFO ] [MAIN ] DTM: Loaded 44538 existing records. [2025-10-20T18:23:45.345636] [WARN ] [MAIN ] DTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:45.345818] [INFO ] [MAIN ] DTM: Wiping existing primary data before full refresh. [2025-10-20T18:23:45.345866] [INFO ] [LOCAL ] DTM: Starting deletion of primary data. [2025-10-20T18:23:45.346331] [DEBUG] [LOCAL ] DTM: Deleted primary data file. [2025-10-20T18:23:45.346387] [INFO ] [MAIN ] DTM: Starting full primary refresh. [2025-10-20T18:23:45.715100] [INFO ] [REMOTE ] DTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:45.721045] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T18:23:45.851487] [INFO ] [REMOTE ] DTE: Fetching primary data for 2025-10. [2025-10-20T18:23:46.348819] [INFO ] [REMOTE ] DTE: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T18:23:46.355537] [INFO ] [REMOTE ] DTE: Full fetch complete (68927 records). [2025-10-20T18:23:46.355723] [DEBUG] [VALIDATOR ] DTE: Validating primary data. [2025-10-20T18:23:46.653100] [INFO ] [REMOTE ] DTK: Fetching primary data for 2025-01. [2025-10-20T18:23:46.746708] [DEBUG] [VALIDATOR ] DTE: All expected trading days are present. [2025-10-20T18:23:46.746804] [DEBUG] [VALIDATOR ] DTE: Primary data validation passed. [2025-10-20T18:23:46.747603] [INFO ] [MAIN ] DTE: Fetched primary data is valid. [2025-10-20T18:23:46.747665] [DEBUG] [LOCAL ] DTE: Saving primary data. [2025-10-20T18:23:46.749068] [DEBUG] [LOCAL ] DTE: Trimmed 68927 records to 47244 (keeping 200 days of data) [2025-10-20T18:23:46.927089] [DEBUG] [REMOTE ] DTK: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:23:46.936309] [WARN ] [REMOTE ] DTK: No data returned for month 2025-01 [2025-10-20T18:23:46.936449] [FATAL] [REMOTE ] DTK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:23:46.936500] [FATAL] [MAIN ] DTK: Failed to fetch primary data. [2025-10-20T18:23:46.965733] [WARN ] [MAIN ] DTK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:23:46.969379] [INFO ] [LOCAL ] DTK: Starting deletion of primary data. [2025-10-20T18:23:46.969741] [DEBUG] [LOCAL ] DTK: No primary data file found to delete. [2025-10-20T18:23:46.970496] [INFO ] [LOCAL ] DTK: Starting deletion of derivative data. [2025-10-20T18:23:46.973333] [DEBUG] [LOCAL ] DTK: No derivative data file found to delete. [2025-10-20T18:23:46.975772] [INFO ] [MAIN ] Update Progress: 659/2367 symbols completed. DTK took 2.8s. Estimated remaining: 3h 36m 58s. Rate: 7.9 symbols/min [2025-10-20T18:23:47.172239] [INFO ] [MAIN ] DTW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:47.228500] [DEBUG] [LOCAL ] DTW: Loaded 4872 existing records from primary storage. [2025-10-20T18:23:47.229853] [INFO ] [MAIN ] DTW: Loaded 4872 existing records. [2025-10-20T18:23:47.318371] [DEBUG] [LOCAL ] DTE: Primary data saved (47244 records). [2025-10-20T18:23:47.340398] [INFO ] [MAIN ] DTE: Full history saved. [2025-10-20T18:23:47.340576] [INFO ] [MAIN ] Update Progress: 660/2367 symbols completed. DTE took 34.3s. Estimated remaining: 3h 36m 31s. Rate: 7.9 symbols/min [2025-10-20T18:23:47.479144] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-01. [2025-10-20T18:23:47.529746] [WARN ] [MAIN ] DTW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:47.530775] [INFO ] [MAIN ] DTW: Wiping existing primary data before full refresh. [2025-10-20T18:23:47.530841] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-10-20T18:23:47.541925] [DEBUG] [LOCAL ] DTW: Deleted primary data file. [2025-10-20T18:23:47.547211] [INFO ] [MAIN ] DTW: Starting full primary refresh. [2025-10-20T18:23:47.718288] [INFO ] [MAIN ] DUK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:48.000516] [INFO ] [REMOTE ] DTW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:48.017856] [DEBUG] [LOCAL ] DUK: Loaded 54217 existing records from primary storage. [2025-10-20T18:23:48.018131] [INFO ] [MAIN ] DUK: Loaded 54217 existing records. [2025-10-20T18:23:48.197807] [WARN ] [MAIN ] DUK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:48.198021] [INFO ] [MAIN ] DUK: Wiping existing primary data before full refresh. [2025-10-20T18:23:48.198077] [INFO ] [LOCAL ] DUK: Starting deletion of primary data. [2025-10-20T18:23:48.198274] [DEBUG] [LOCAL ] DUK: Deleted primary data file. [2025-10-20T18:23:48.198329] [INFO ] [MAIN ] DUK: Starting full primary refresh. [2025-10-20T18:23:48.263025] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-01. [2025-10-20T18:23:48.556442] [INFO ] [REMOTE ] DUK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:49.051387] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-05. [2025-10-20T18:23:49.075697] [INFO ] [REMOTE ] DTW: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T18:23:49.263579] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T18:23:49.470398] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T18:23:49.851476] [INFO ] [REMOTE ] DTW: Fetching primary data for 2025-02. [2025-10-20T18:23:49.955722] [DEBUG] [REMOTE ] DTW: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-10-20T18:23:49.956027] [WARN ] [REMOTE ] DTW: No data returned for month 2025-02 [2025-10-20T18:23:49.956144] [FATAL] [REMOTE ] DTW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:23:49.956262] [FATAL] [MAIN ] DTW: Failed to fetch primary data. [2025-10-20T18:23:49.956347] [WARN ] [MAIN ] DTW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:23:49.956434] [INFO ] [LOCAL ] DTW: Starting deletion of primary data. [2025-10-20T18:23:49.956517] [DEBUG] [LOCAL ] DTW: No primary data file found to delete. [2025-10-20T18:23:49.956572] [INFO ] [LOCAL ] DTW: Starting deletion of derivative data. [2025-10-20T18:23:49.956772] [DEBUG] [LOCAL ] DTW: Deleted derivative data file. [2025-10-20T18:23:49.956856] [INFO ] [MAIN ] Update Progress: 661/2367 symbols completed. DTW took 3.0s. Estimated remaining: 3h 36m 11s. Rate: 7.9 symbols/min [2025-10-20T18:23:50.136076] [INFO ] [MAIN ] DUKB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:23:50.143595] [DEBUG] [LOCAL ] DUKB: Loaded 6333 existing records from primary storage. [2025-10-20T18:23:50.143851] [INFO ] [MAIN ] DUKB: Loaded 6333 existing records. [2025-10-20T18:23:50.323385] [WARN ] [MAIN ] DUKB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:23:50.323591] [INFO ] [MAIN ] DUKB: Wiping existing primary data before full refresh. [2025-10-20T18:23:50.323684] [INFO ] [LOCAL ] DUKB: Starting deletion of primary data. [2025-10-20T18:23:50.323863] [DEBUG] [LOCAL ] DUKB: Deleted primary data file. [2025-10-20T18:23:50.323922] [INFO ] [MAIN ] DUKB: Starting full primary refresh. [2025-10-20T18:23:50.661667] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-01. [2025-10-20T18:23:50.682371] [INFO ] [REMOTE ] DUKB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:23:51.451428] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-02. [2025-10-20T18:23:52.251408] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-06. [2025-10-20T18:23:52.505098] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:23:52.590039] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:23:52.723630] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:23:53.051364] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-02. [2025-10-20T18:23:53.851428] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-01. [2025-10-20T18:23:54.651391] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-07. [2025-10-20T18:23:54.831880] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T18:23:54.847179] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T18:23:55.003912] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T18:23:55.451416] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-03. [2025-10-20T18:23:56.251384] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-02. [2025-10-20T18:23:56.851912] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:23:57.051430] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-03. [2025-10-20T18:23:57.454367] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:23:57.851415] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-04. [2025-10-20T18:23:58.651376] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-03. [2025-10-20T18:23:58.745903] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T18:23:59.225198] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T18:23:59.451377] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-04. [2025-10-20T18:23:59.937343] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-10-20T18:24:00.251364] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-05. [2025-10-20T18:24:01.051438] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-04. [2025-10-20T18:24:01.581304] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T18:24:01.851366] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-08. [2025-10-20T18:24:01.892274] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T18:24:02.145372] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:24:02.497810] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:24:02.651421] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-05. [2025-10-20T18:24:03.202079] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:24:03.451381] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-09. [2025-10-20T18:24:04.117029] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:24:04.251382] [INFO ] [REMOTE ] DTG: Fetching primary data for 2025-10. [2025-10-20T18:24:04.966445] [INFO ] [REMOTE ] DTG: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:24:04.968463] [INFO ] [REMOTE ] DTG: Full fetch complete (8608 records). [2025-10-20T18:24:04.968551] [DEBUG] [VALIDATOR ] DTG: Validating primary data. [2025-10-20T18:24:05.056464] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-06. [2025-10-20T18:24:05.357870] [DEBUG] [VALIDATOR ] DTG: All expected trading days are present. [2025-10-20T18:24:05.357946] [DEBUG] [VALIDATOR ] DTG: Primary data validation passed. [2025-10-20T18:24:05.358147] [INFO ] [MAIN ] DTG: Fetched primary data is valid. [2025-10-20T18:24:05.358186] [DEBUG] [LOCAL ] DTG: Saving primary data. [2025-10-20T18:24:05.358575] [DEBUG] [LOCAL ] DTG: Trimmed 8608 records to 5735 (keeping 200 days of data) [2025-10-20T18:24:05.392281] [DEBUG] [LOCAL ] DTG: Primary data saved (5735 records). [2025-10-20T18:24:05.392399] [INFO ] [MAIN ] DTG: Full history saved. [2025-10-20T18:24:05.392479] [INFO ] [MAIN ] Update Progress: 662/2367 symbols completed. DTG took 26.6s. Estimated remaining: 3h 36m 23s. Rate: 7.9 symbols/min [2025-10-20T18:24:05.562809] [INFO ] [MAIN ] DV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:05.613597] [DEBUG] [LOCAL ] DV: Loaded 52038 existing records from primary storage. [2025-10-20T18:24:05.613928] [INFO ] [MAIN ] DV: Loaded 52038 existing records. [2025-10-20T18:24:05.793450] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T18:24:05.797785] [WARN ] [MAIN ] DV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:05.800361] [INFO ] [MAIN ] DV: Wiping existing primary data before full refresh. [2025-10-20T18:24:05.800706] [INFO ] [LOCAL ] DV: Starting deletion of primary data. [2025-10-20T18:24:05.800910] [DEBUG] [LOCAL ] DV: Deleted primary data file. [2025-10-20T18:24:05.800966] [INFO ] [MAIN ] DV: Starting full primary refresh. [2025-10-20T18:24:05.856580] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-07. [2025-10-20T18:24:06.170499] [INFO ] [REMOTE ] DV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:06.390501] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T18:24:06.651419] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-08. [2025-10-20T18:24:07.332454] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T18:24:07.451378] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-05. [2025-10-20T18:24:08.251443] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-06. [2025-10-20T18:24:09.051455] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-01. [2025-10-20T18:24:09.120435] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:24:09.851350] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-09. [2025-10-20T18:24:09.967592] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:24:10.452776] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:24:10.651357] [INFO ] [REMOTE ] DUKB: Fetching primary data for 2025-10. [2025-10-20T18:24:10.752472] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:24:11.121575] [INFO ] [REMOTE ] DUKB: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T18:24:11.123057] [INFO ] [REMOTE ] DUKB: Full fetch complete (9060 records). [2025-10-20T18:24:11.123143] [DEBUG] [VALIDATOR ] DUKB: Validating primary data. [2025-10-20T18:24:11.456522] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-07. [2025-10-20T18:24:11.532788] [DEBUG] [VALIDATOR ] DUKB: All expected trading days are present. [2025-10-20T18:24:11.532875] [DEBUG] [VALIDATOR ] DUKB: Primary data validation passed. [2025-10-20T18:24:11.533125] [INFO ] [MAIN ] DUKB: Fetched primary data is valid. [2025-10-20T18:24:11.533168] [DEBUG] [LOCAL ] DUKB: Saving primary data. [2025-10-20T18:24:11.533725] [DEBUG] [LOCAL ] DUKB: Trimmed 9060 records to 6196 (keeping 200 days of data) [2025-10-20T18:24:11.571669] [DEBUG] [LOCAL ] DUKB: Primary data saved (6196 records). [2025-10-20T18:24:11.571816] [INFO ] [MAIN ] DUKB: Full history saved. [2025-10-20T18:24:11.571897] [INFO ] [MAIN ] Update Progress: 663/2367 symbols completed. DUKB took 21.6s. Estimated remaining: 3h 36m 12s. Rate: 7.9 symbols/min [2025-10-20T18:24:11.738999] [INFO ] [MAIN ] DVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:11.781411] [DEBUG] [LOCAL ] DVA: Loaded 41554 existing records from primary storage. [2025-10-20T18:24:11.781757] [INFO ] [MAIN ] DVA: Loaded 41554 existing records. [2025-10-20T18:24:11.961202] [WARN ] [MAIN ] DVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:11.961495] [INFO ] [MAIN ] DVA: Wiping existing primary data before full refresh. [2025-10-20T18:24:11.961557] [INFO ] [LOCAL ] DVA: Starting deletion of primary data. [2025-10-20T18:24:11.961860] [DEBUG] [LOCAL ] DVA: Deleted primary data file. [2025-10-20T18:24:11.961931] [INFO ] [MAIN ] DVA: Starting full primary refresh. [2025-10-20T18:24:12.259466] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-06. [2025-10-20T18:24:12.391101] [INFO ] [REMOTE ] DVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:13.051425] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-02. [2025-10-20T18:24:13.591504] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T18:24:13.631020] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:24:13.851401] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-07. [2025-10-20T18:24:14.599708] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:24:14.651379] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-03. [2025-10-20T18:24:15.367146] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:24:15.451374] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-08. [2025-10-20T18:24:16.250984] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:24:16.261903] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-08. [2025-10-20T18:24:16.912704] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:24:17.051438] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-01. [2025-10-20T18:24:17.851412] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-09. [2025-10-20T18:24:18.169136] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:24:18.434370] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T18:24:18.651452] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-04. [2025-10-20T18:24:19.443119] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:24:19.451672] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-02. [2025-10-20T18:24:20.253375] [INFO ] [REMOTE ] DTM: Fetching primary data for 2025-10. [2025-10-20T18:24:20.259735] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T18:24:21.032180] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:24:21.051338] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-03. [2025-10-20T18:24:21.851341] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-09. [2025-10-20T18:24:22.009674] [INFO ] [REMOTE ] DTM: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:24:22.016624] [INFO ] [REMOTE ] DTM: Full fetch complete (65167 records). [2025-10-20T18:24:22.016798] [DEBUG] [VALIDATOR ] DTM: Validating primary data. [2025-10-20T18:24:22.404813] [DEBUG] [VALIDATOR ] DTM: All expected trading days are present. [2025-10-20T18:24:22.404941] [DEBUG] [VALIDATOR ] DTM: Primary data validation passed. [2025-10-20T18:24:22.405746] [INFO ] [MAIN ] DTM: Fetched primary data is valid. [2025-10-20T18:24:22.405805] [DEBUG] [LOCAL ] DTM: Saving primary data. [2025-10-20T18:24:22.407025] [DEBUG] [LOCAL ] DTM: Trimmed 65167 records to 43821 (keeping 200 days of data) [2025-10-20T18:24:22.536824] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T18:24:22.657842] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-04. [2025-10-20T18:24:22.715418] [DEBUG] [LOCAL ] DTM: Primary data saved (43821 records). [2025-10-20T18:24:22.715554] [INFO ] [MAIN ] DTM: Full history saved. [2025-10-20T18:24:22.715924] [INFO ] [MAIN ] Update Progress: 664/2367 symbols completed. DTM took 37.8s. Estimated remaining: 3h 36m 13s. Rate: 7.9 symbols/min [2025-10-20T18:24:22.882392] [INFO ] [MAIN ] DVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:22.882555] [DEBUG] [LOCAL ] DVN: No existing data file found in primary storage. [2025-10-20T18:24:22.882591] [WARN ] [MAIN ] DVN: No existing data found, performing full refresh instead. [2025-10-20T18:24:22.882721] [INFO ] [MAIN ] DVN: Wiping existing primary data before full refresh. [2025-10-20T18:24:22.882766] [INFO ] [LOCAL ] DVN: Starting deletion of primary data. [2025-10-20T18:24:22.882808] [DEBUG] [LOCAL ] DVN: No primary data file found to delete. [2025-10-20T18:24:22.882879] [INFO ] [MAIN ] DVN: Starting full primary refresh. [2025-10-20T18:24:23.260536] [INFO ] [REMOTE ] DVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:23.422467] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T18:24:23.451330] [INFO ] [REMOTE ] DUK: Fetching primary data for 2025-10. [2025-10-20T18:24:24.251342] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-05. [2025-10-20T18:24:24.298281] [INFO ] [REMOTE ] DUK: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T18:24:24.306043] [INFO ] [REMOTE ] DUK: Full fetch complete (77567 records). [2025-10-20T18:24:24.306163] [DEBUG] [VALIDATOR ] DUK: Validating primary data. [2025-10-20T18:24:24.595538] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T18:24:24.727726] [DEBUG] [VALIDATOR ] DUK: All expected trading days are present. [2025-10-20T18:24:24.727803] [DEBUG] [VALIDATOR ] DUK: Primary data validation passed. [2025-10-20T18:24:24.728668] [INFO ] [MAIN ] DUK: Fetched primary data is valid. [2025-10-20T18:24:24.728729] [DEBUG] [LOCAL ] DUK: Saving primary data. [2025-10-20T18:24:24.730121] [DEBUG] [LOCAL ] DUK: Trimmed 77567 records to 53437 (keeping 200 days of data) [2025-10-20T18:24:25.051214] [DEBUG] [LOCAL ] DUK: Primary data saved (53437 records). [2025-10-20T18:24:25.051341] [INFO ] [MAIN ] DUK: Full history saved. [2025-10-20T18:24:25.051417] [INFO ] [MAIN ] Update Progress: 665/2367 symbols completed. DUK took 37.7s. Estimated remaining: 3h 35m 52s. Rate: 7.9 symbols/min [2025-10-20T18:24:25.051491] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-05. [2025-10-20T18:24:25.228635] [INFO ] [MAIN ] DX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:25.278393] [DEBUG] [LOCAL ] DX: Loaded 53271 existing records from primary storage. [2025-10-20T18:24:25.278762] [INFO ] [MAIN ] DX: Loaded 53271 existing records. [2025-10-20T18:24:25.458001] [WARN ] [MAIN ] DX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:25.458189] [INFO ] [MAIN ] DX: Wiping existing primary data before full refresh. [2025-10-20T18:24:25.458234] [INFO ] [LOCAL ] DX: Starting deletion of primary data. [2025-10-20T18:24:25.458517] [DEBUG] [LOCAL ] DX: Deleted primary data file. [2025-10-20T18:24:25.458553] [INFO ] [MAIN ] DX: Starting full primary refresh. [2025-10-20T18:24:25.821938] [INFO ] [REMOTE ] DX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:25.858196] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-01. [2025-10-20T18:24:25.863526] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:24:26.651407] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-06. [2025-10-20T18:24:26.677562] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T18:24:27.451390] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-01. [2025-10-20T18:24:27.744452] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:24:28.251380] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-02. [2025-10-20T18:24:28.296189] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:24:29.051399] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-07. [2025-10-20T18:24:29.677239] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-01 (2.23 seconds). [2025-10-20T18:24:29.851346] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-06. [2025-10-20T18:24:30.033319] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T18:24:30.651353] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-03. [2025-10-20T18:24:30.789176] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:24:31.281812] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:24:31.451406] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-07. [2025-10-20T18:24:32.251411] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-08. [2025-10-20T18:24:32.639569] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:24:33.012010] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:24:33.051384] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-08. [2025-10-20T18:24:33.851413] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-04. [2025-10-20T18:24:33.950170] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:24:34.450234] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:24:34.651370] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-09. [2025-10-20T18:24:35.451348] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-02. [2025-10-20T18:24:35.833478] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T18:24:35.886095] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T18:24:36.251376] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-05. [2025-10-20T18:24:37.051373] [INFO ] [REMOTE ] DVA: Fetching primary data for 2025-10. [2025-10-20T18:24:37.159030] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T18:24:37.851355] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-03. [2025-10-20T18:24:38.316479] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T18:24:38.371638] [INFO ] [REMOTE ] DVA: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T18:24:38.378136] [INFO ] [REMOTE ] DVA: Full fetch complete (59689 records). [2025-10-20T18:24:38.378271] [DEBUG] [VALIDATOR ] DVA: Validating primary data. [2025-10-20T18:24:38.662268] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-06. [2025-10-20T18:24:38.806085] [DEBUG] [VALIDATOR ] DVA: All expected trading days are present. [2025-10-20T18:24:38.806166] [DEBUG] [VALIDATOR ] DVA: Primary data validation passed. [2025-10-20T18:24:38.806916] [INFO ] [MAIN ] DVA: Fetched primary data is valid. [2025-10-20T18:24:38.806977] [DEBUG] [LOCAL ] DVA: Saving primary data. [2025-10-20T18:24:38.808126] [DEBUG] [LOCAL ] DVA: Trimmed 59689 records to 40887 (keeping 200 days of data) [2025-10-20T18:24:39.045926] [DEBUG] [LOCAL ] DVA: Primary data saved (40887 records). [2025-10-20T18:24:39.046050] [INFO ] [MAIN ] DVA: Full history saved. [2025-10-20T18:24:39.046125] [INFO ] [MAIN ] Update Progress: 666/2367 symbols completed. DVA took 27.5s. Estimated remaining: 3h 36m 1s. Rate: 7.9 symbols/min [2025-10-20T18:24:39.211698] [INFO ] [MAIN ] DXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:39.261014] [DEBUG] [LOCAL ] DXC: Loaded 50621 existing records from primary storage. [2025-10-20T18:24:39.261383] [INFO ] [MAIN ] DXC: Loaded 50621 existing records. [2025-10-20T18:24:39.437707] [WARN ] [MAIN ] DXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:39.437939] [INFO ] [MAIN ] DXC: Wiping existing primary data before full refresh. [2025-10-20T18:24:39.437989] [INFO ] [LOCAL ] DXC: Starting deletion of primary data. [2025-10-20T18:24:39.438175] [DEBUG] [LOCAL ] DXC: Deleted primary data file. [2025-10-20T18:24:39.438255] [INFO ] [MAIN ] DXC: Starting full primary refresh. [2025-10-20T18:24:39.472302] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-09. [2025-10-20T18:24:39.794728] [INFO ] [REMOTE ] DXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:40.251359] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-01. [2025-10-20T18:24:40.495442] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T18:24:40.739805] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-03 (2.89 seconds). [2025-10-20T18:24:41.051377] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-07. [2025-10-20T18:24:41.473630] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T18:24:41.510393] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:24:41.851368] [INFO ] [REMOTE ] DV: Fetching primary data for 2025-10. [2025-10-20T18:24:42.651385] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-02. [2025-10-20T18:24:43.124074] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:24:43.451370] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-08. [2025-10-20T18:24:43.704920] [INFO ] [REMOTE ] DV: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:24:43.712713] [INFO ] [REMOTE ] DV: Full fetch complete (73860 records). [2025-10-20T18:24:43.712840] [DEBUG] [VALIDATOR ] DV: Validating primary data. [2025-10-20T18:24:44.109397] [DEBUG] [VALIDATOR ] DV: All expected trading days are present. [2025-10-20T18:24:44.109707] [DEBUG] [VALIDATOR ] DV: Primary data validation passed. [2025-10-20T18:24:44.110720] [INFO ] [MAIN ] DV: Fetched primary data is valid. [2025-10-20T18:24:44.110821] [DEBUG] [LOCAL ] DV: Saving primary data. [2025-10-20T18:24:44.121073] [DEBUG] [LOCAL ] DV: Trimmed 73860 records to 51250 (keeping 200 days of data) [2025-10-20T18:24:44.139401] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:24:44.254911] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-04. [2025-10-20T18:24:44.457725] [DEBUG] [LOCAL ] DV: Primary data saved (51250 records). [2025-10-20T18:24:44.457955] [INFO ] [MAIN ] DV: Full history saved. [2025-10-20T18:24:44.458036] [INFO ] [MAIN ] Update Progress: 667/2367 symbols completed. DV took 39.1s. Estimated remaining: 3h 35m 48s. Rate: 7.9 symbols/min [2025-10-20T18:24:44.627448] [INFO ] [MAIN ] DXYZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:44.662404] [DEBUG] [LOCAL ] DXYZ: Loaded 37760 existing records from primary storage. [2025-10-20T18:24:44.662717] [INFO ] [MAIN ] DXYZ: Loaded 37760 existing records. [2025-10-20T18:24:44.845197] [WARN ] [MAIN ] DXYZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:44.845409] [INFO ] [MAIN ] DXYZ: Wiping existing primary data before full refresh. [2025-10-20T18:24:44.845475] [INFO ] [LOCAL ] DXYZ: Starting deletion of primary data. [2025-10-20T18:24:44.845840] [DEBUG] [LOCAL ] DXYZ: Deleted primary data file. [2025-10-20T18:24:44.845914] [INFO ] [MAIN ] DXYZ: Starting full primary refresh. [2025-10-20T18:24:45.057383] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-03. [2025-10-20T18:24:45.209777] [INFO ] [REMOTE ] DXYZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:45.321709] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T18:24:45.851396] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-09. [2025-10-20T18:24:46.019303] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:24:46.651350] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-05. [2025-10-20T18:24:46.830948] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T18:24:47.451349] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-01. [2025-10-20T18:24:47.738770] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T18:24:48.251385] [INFO ] [REMOTE ] DX: Fetching primary data for 2025-10. [2025-10-20T18:24:48.557964] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:24:49.051399] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-04. [2025-10-20T18:24:49.652812] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-10-20T18:24:49.851422] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-06. [2025-10-20T18:24:50.398110] [INFO ] [REMOTE ] DX: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T18:24:50.406339] [INFO ] [REMOTE ] DX: Full fetch complete (75864 records). [2025-10-20T18:24:50.406475] [DEBUG] [VALIDATOR ] DX: Validating primary data. [2025-10-20T18:24:50.670172] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-02. [2025-10-20T18:24:50.749740] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:24:50.847137] [DEBUG] [VALIDATOR ] DX: All expected trading days are present. [2025-10-20T18:24:50.847220] [DEBUG] [VALIDATOR ] DX: Primary data validation passed. [2025-10-20T18:24:50.848077] [INFO ] [MAIN ] DX: Fetched primary data is valid. [2025-10-20T18:24:50.848141] [DEBUG] [LOCAL ] DX: Saving primary data. [2025-10-20T18:24:50.849729] [DEBUG] [LOCAL ] DX: Trimmed 75864 records to 52526 (keeping 200 days of data) [2025-10-20T18:24:51.173395] [DEBUG] [LOCAL ] DX: Primary data saved (52526 records). [2025-10-20T18:24:51.173803] [INFO ] [MAIN ] DX: Full history saved. [2025-10-20T18:24:51.173900] [INFO ] [MAIN ] Update Progress: 668/2367 symbols completed. DX took 26.1s. Estimated remaining: 3h 35m 38s. Rate: 7.9 symbols/min [2025-10-20T18:24:51.342550] [INFO ] [MAIN ] DY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:24:51.369236] [DEBUG] [LOCAL ] DY: Loaded 28800 existing records from primary storage. [2025-10-20T18:24:51.369484] [INFO ] [MAIN ] DY: Loaded 28800 existing records. [2025-10-20T18:24:51.458481] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-05. [2025-10-20T18:24:51.551632] [WARN ] [MAIN ] DY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:24:51.551903] [INFO ] [MAIN ] DY: Wiping existing primary data before full refresh. [2025-10-20T18:24:51.551961] [INFO ] [LOCAL ] DY: Starting deletion of primary data. [2025-10-20T18:24:51.552488] [DEBUG] [LOCAL ] DY: Deleted primary data file. [2025-10-20T18:24:51.552546] [INFO ] [MAIN ] DY: Starting full primary refresh. [2025-10-20T18:24:51.920825] [INFO ] [REMOTE ] DY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:24:51.962950] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T18:24:52.251368] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-07. [2025-10-20T18:24:52.621770] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-10-20T18:24:53.051431] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-03. [2025-10-20T18:24:53.264021] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:24:53.851370] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-01. [2025-10-20T18:24:54.101816] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:24:54.651354] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-08. [2025-10-20T18:24:54.848209] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:24:55.286605] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:24:55.451404] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-04. [2025-10-20T18:24:56.251490] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-06. [2025-10-20T18:24:56.362901] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T18:24:57.051402] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-09. [2025-10-20T18:24:57.472718] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:24:57.565185] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:24:57.851437] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-07. [2025-10-20T18:24:58.651401] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-02. [2025-10-20T18:24:59.096352] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T18:24:59.452296] [INFO ] [REMOTE ] DVN: Fetching primary data for 2025-10. [2025-10-20T18:24:59.529949] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:24:59.607818] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:25:00.251385] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-03. [2025-10-20T18:25:00.295926] [INFO ] [REMOTE ] DVN: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:25:00.303886] [INFO ] [REMOTE ] DVN: Full fetch complete (77879 records). [2025-10-20T18:25:00.304037] [DEBUG] [VALIDATOR ] DVN: Validating primary data. [2025-10-20T18:25:00.704394] [DEBUG] [VALIDATOR ] DVN: All expected trading days are present. [2025-10-20T18:25:00.704515] [DEBUG] [VALIDATOR ] DVN: Primary data validation passed. [2025-10-20T18:25:00.705637] [INFO ] [MAIN ] DVN: Fetched primary data is valid. [2025-10-20T18:25:00.705727] [DEBUG] [LOCAL ] DVN: Saving primary data. [2025-10-20T18:25:00.707496] [DEBUG] [LOCAL ] DVN: Trimmed 77879 records to 53699 (keeping 200 days of data) [2025-10-20T18:25:01.030591] [DEBUG] [LOCAL ] DVN: Primary data saved (53699 records). [2025-10-20T18:25:01.030761] [INFO ] [MAIN ] DVN: Full history saved. [2025-10-20T18:25:01.030838] [INFO ] [MAIN ] Update Progress: 669/2367 symbols completed. DVN took 38.3s. Estimated remaining: 3h 35m 36s. Rate: 7.9 symbols/min [2025-10-20T18:25:01.060201] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-08. [2025-10-20T18:25:01.214516] [INFO ] [MAIN ] E: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:01.252757] [DEBUG] [LOCAL ] E: Loaded 33728 existing records from primary storage. [2025-10-20T18:25:01.253107] [INFO ] [MAIN ] E: Loaded 33728 existing records. [2025-10-20T18:25:01.445502] [WARN ] [MAIN ] E: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:01.445781] [INFO ] [MAIN ] E: Wiping existing primary data before full refresh. [2025-10-20T18:25:01.445833] [INFO ] [LOCAL ] E: Starting deletion of primary data. [2025-10-20T18:25:01.446086] [DEBUG] [LOCAL ] E: Deleted primary data file. [2025-10-20T18:25:01.446145] [INFO ] [MAIN ] E: Starting full primary refresh. [2025-10-20T18:25:01.853791] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-05. [2025-10-20T18:25:01.859401] [INFO ] [REMOTE ] E: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:02.333534] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T18:25:02.651379] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-04. [2025-10-20T18:25:03.282420] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T18:25:03.451428] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-09. [2025-10-20T18:25:04.029985] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T18:25:04.112104] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:25:04.251370] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-05. [2025-10-20T18:25:04.788780] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:25:05.051385] [INFO ] [REMOTE ] E: Fetching primary data for 2025-01. [2025-10-20T18:25:05.760696] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T18:25:05.851415] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-06. [2025-10-20T18:25:06.475617] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T18:25:06.651431] [INFO ] [REMOTE ] E: Fetching primary data for 2025-02. [2025-10-20T18:25:07.451399] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-06. [2025-10-20T18:25:07.713321] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T18:25:07.884171] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:25:08.251450] [INFO ] [REMOTE ] E: Fetching primary data for 2025-03. [2025-10-20T18:25:08.752360] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T18:25:09.051397] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-07. [2025-10-20T18:25:09.218776] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T18:25:09.851387] [INFO ] [REMOTE ] DXC: Fetching primary data for 2025-10. [2025-10-20T18:25:10.186437] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T18:25:10.651373] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-07. [2025-10-20T18:25:11.451421] [INFO ] [REMOTE ] E: Fetching primary data for 2025-04. [2025-10-20T18:25:11.669823] [INFO ] [REMOTE ] DXC: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:25:11.677300] [INFO ] [REMOTE ] DXC: Full fetch complete (70224 records). [2025-10-20T18:25:11.677445] [DEBUG] [VALIDATOR ] DXC: Validating primary data. [2025-10-20T18:25:12.068522] [DEBUG] [VALIDATOR ] DXC: All expected trading days are present. [2025-10-20T18:25:12.068626] [DEBUG] [VALIDATOR ] DXC: Primary data validation passed. [2025-10-20T18:25:12.069503] [INFO ] [MAIN ] DXC: Fetched primary data is valid. [2025-10-20T18:25:12.069584] [DEBUG] [LOCAL ] DXC: Saving primary data. [2025-10-20T18:25:12.071172] [DEBUG] [LOCAL ] DXC: Trimmed 70224 records to 49946 (keeping 200 days of data) [2025-10-20T18:25:12.200344] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:25:12.263744] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-08. [2025-10-20T18:25:12.385894] [DEBUG] [LOCAL ] DXC: Primary data saved (49946 records). [2025-10-20T18:25:12.386022] [INFO ] [MAIN ] DXC: Full history saved. [2025-10-20T18:25:12.386100] [INFO ] [MAIN ] Update Progress: 670/2367 symbols completed. DXC took 33.3s. Estimated remaining: 3h 35m 38s. Rate: 7.9 symbols/min [2025-10-20T18:25:12.557870] [INFO ] [MAIN ] EAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:12.558107] [DEBUG] [LOCAL ] EAF: No existing data file found in primary storage. [2025-10-20T18:25:12.558166] [WARN ] [MAIN ] EAF: No existing data found, performing full refresh instead. [2025-10-20T18:25:12.558240] [INFO ] [MAIN ] EAF: Wiping existing primary data before full refresh. [2025-10-20T18:25:12.558276] [INFO ] [LOCAL ] EAF: Starting deletion of primary data. [2025-10-20T18:25:12.558315] [DEBUG] [LOCAL ] EAF: No primary data file found to delete. [2025-10-20T18:25:12.558346] [INFO ] [MAIN ] EAF: Starting full primary refresh. [2025-10-20T18:25:12.942169] [INFO ] [REMOTE ] EAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:12.966073] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:25:13.051390] [INFO ] [REMOTE ] E: Fetching primary data for 2025-05. [2025-10-20T18:25:13.851371] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-01. [2025-10-20T18:25:14.202858] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:25:14.442178] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:25:14.651373] [INFO ] [REMOTE ] E: Fetching primary data for 2025-06. [2025-10-20T18:25:15.426473] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:25:15.451325] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-02. [2025-10-20T18:25:16.016409] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:25:16.251394] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-08. [2025-10-20T18:25:17.034062] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:25:17.051326] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-03. [2025-10-20T18:25:17.351790] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:25:17.851365] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-09. [2025-10-20T18:25:18.651463] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-09. [2025-10-20T18:25:18.813058] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T18:25:19.451406] [INFO ] [REMOTE ] E: Fetching primary data for 2025-07. [2025-10-20T18:25:19.867700] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T18:25:20.024072] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:25:20.251386] [INFO ] [REMOTE ] DY: Fetching primary data for 2025-10. [2025-10-20T18:25:20.878303] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:25:21.051389] [INFO ] [REMOTE ] E: Fetching primary data for 2025-08. [2025-10-20T18:25:21.320550] [INFO ] [REMOTE ] DY: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:25:21.324055] [INFO ] [REMOTE ] DY: Full fetch complete (43351 records). [2025-10-20T18:25:21.324130] [DEBUG] [VALIDATOR ] DY: Validating primary data. [2025-10-20T18:25:21.694218] [DEBUG] [VALIDATOR ] DY: All expected trading days are present. [2025-10-20T18:25:21.694321] [DEBUG] [VALIDATOR ] DY: Primary data validation passed. [2025-10-20T18:25:21.694960] [INFO ] [MAIN ] DY: Fetched primary data is valid. [2025-10-20T18:25:21.695022] [DEBUG] [LOCAL ] DY: Saving primary data. [2025-10-20T18:25:21.695939] [DEBUG] [LOCAL ] DY: Trimmed 43351 records to 28264 (keeping 200 days of data) [2025-10-20T18:25:21.854609] [INFO ] [REMOTE ] DXYZ: Fetching primary data for 2025-10. [2025-10-20T18:25:21.861351] [DEBUG] [LOCAL ] DY: Primary data saved (28264 records). [2025-10-20T18:25:21.861426] [INFO ] [MAIN ] DY: Full history saved. [2025-10-20T18:25:21.861499] [INFO ] [MAIN ] Update Progress: 671/2367 symbols completed. DY took 30.7s. Estimated remaining: 3h 35m 35s. Rate: 7.9 symbols/min [2025-10-20T18:25:22.028732] [INFO ] [MAIN ] EAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:22.034515] [DEBUG] [LOCAL ] EAI: Loaded 4014 existing records from primary storage. [2025-10-20T18:25:22.034779] [INFO ] [MAIN ] EAI: Loaded 4014 existing records. [2025-10-20T18:25:22.210375] [WARN ] [MAIN ] EAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:22.210662] [INFO ] [MAIN ] EAI: Wiping existing primary data before full refresh. [2025-10-20T18:25:22.210722] [INFO ] [LOCAL ] EAI: Starting deletion of primary data. [2025-10-20T18:25:22.210854] [DEBUG] [LOCAL ] EAI: Deleted primary data file. [2025-10-20T18:25:22.210892] [INFO ] [MAIN ] EAI: Starting full primary refresh. [2025-10-20T18:25:22.536848] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T18:25:22.625262] [INFO ] [REMOTE ] EAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:22.651333] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-01. [2025-10-20T18:25:23.152235] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T18:25:23.451368] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-02. [2025-10-20T18:25:23.792709] [INFO ] [REMOTE ] DXYZ: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:25:23.802101] [INFO ] [REMOTE ] DXYZ: Full fetch complete (57034 records). [2025-10-20T18:25:23.802252] [DEBUG] [VALIDATOR ] DXYZ: Validating primary data. [2025-10-20T18:25:24.047623] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:25:24.219140] [DEBUG] [VALIDATOR ] DXYZ: All expected trading days are present. [2025-10-20T18:25:24.219266] [DEBUG] [VALIDATOR ] DXYZ: Primary data validation passed. [2025-10-20T18:25:24.220163] [INFO ] [MAIN ] DXYZ: Fetched primary data is valid. [2025-10-20T18:25:24.220230] [DEBUG] [LOCAL ] DXYZ: Saving primary data. [2025-10-20T18:25:24.221566] [DEBUG] [LOCAL ] DXYZ: Trimmed 57034 records to 37229 (keeping 200 days of data) [2025-10-20T18:25:24.273693] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-03. [2025-10-20T18:25:24.454347] [DEBUG] [LOCAL ] DXYZ: Primary data saved (37229 records). [2025-10-20T18:25:24.454502] [INFO ] [MAIN ] DXYZ: Full history saved. [2025-10-20T18:25:24.454594] [INFO ] [MAIN ] Update Progress: 672/2367 symbols completed. DXYZ took 40.0s. Estimated remaining: 3h 35m 14s. Rate: 7.9 symbols/min [2025-10-20T18:25:24.647826] [INFO ] [MAIN ] EARN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:24.652719] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-10-20T18:25:24.680577] [DEBUG] [LOCAL ] EARN: Loaded 28956 existing records from primary storage. [2025-10-20T18:25:24.680883] [INFO ] [MAIN ] EARN: Loaded 28956 existing records. [2025-10-20T18:25:24.887400] [WARN ] [MAIN ] EARN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:24.887692] [INFO ] [MAIN ] EARN: Wiping existing primary data before full refresh. [2025-10-20T18:25:24.887752] [INFO ] [LOCAL ] EARN: Starting deletion of primary data. [2025-10-20T18:25:24.887936] [DEBUG] [LOCAL ] EARN: Deleted primary data file. [2025-10-20T18:25:24.887973] [INFO ] [MAIN ] EARN: Starting full primary refresh. [2025-10-20T18:25:25.058758] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-04. [2025-10-20T18:25:25.292532] [INFO ] [REMOTE ] EARN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:25.714401] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T18:25:25.851379] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-05. [2025-10-20T18:25:26.277589] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:25:26.651348] [INFO ] [REMOTE ] E: Fetching primary data for 2025-09. [2025-10-20T18:25:27.451376] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-06. [2025-10-20T18:25:27.942870] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T18:25:27.968716] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T18:25:28.251371] [INFO ] [REMOTE ] E: Fetching primary data for 2025-10. [2025-10-20T18:25:29.051381] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-07. [2025-10-20T18:25:29.622831] [INFO ] [REMOTE ] E: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:25:29.627947] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T18:25:29.633028] [INFO ] [REMOTE ] E: Full fetch complete (47718 records). [2025-10-20T18:25:29.633120] [DEBUG] [VALIDATOR ] E: Validating primary data. [2025-10-20T18:25:29.861017] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-04. [2025-10-20T18:25:30.028299] [DEBUG] [VALIDATOR ] E: All expected trading days are present. [2025-10-20T18:25:30.028385] [DEBUG] [VALIDATOR ] E: Primary data validation passed. [2025-10-20T18:25:30.029046] [INFO ] [MAIN ] E: Fetched primary data is valid. [2025-10-20T18:25:30.029107] [DEBUG] [LOCAL ] E: Saving primary data. [2025-10-20T18:25:30.030203] [DEBUG] [LOCAL ] E: Trimmed 47718 records to 33355 (keeping 200 days of data) [2025-10-20T18:25:30.231147] [DEBUG] [LOCAL ] E: Primary data saved (33355 records). [2025-10-20T18:25:30.231294] [INFO ] [MAIN ] E: Full history saved. [2025-10-20T18:25:30.231669] [INFO ] [MAIN ] Update Progress: 673/2367 symbols completed. E took 29.2s. Estimated remaining: 3h 35m 2s. Rate: 7.9 symbols/min [2025-10-20T18:25:30.407784] [INFO ] [MAIN ] EAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:30.460348] [DEBUG] [LOCAL ] EAT: Loaded 50582 existing records from primary storage. [2025-10-20T18:25:30.460689] [INFO ] [MAIN ] EAT: Loaded 50582 existing records. [2025-10-20T18:25:30.648881] [WARN ] [MAIN ] EAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:30.649093] [INFO ] [MAIN ] EAT: Wiping existing primary data before full refresh. [2025-10-20T18:25:30.649156] [INFO ] [LOCAL ] EAT: Starting deletion of primary data. [2025-10-20T18:25:30.649448] [DEBUG] [LOCAL ] EAT: Deleted primary data file. [2025-10-20T18:25:30.649519] [INFO ] [MAIN ] EAT: Starting full primary refresh. [2025-10-20T18:25:30.661489] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-08. [2025-10-20T18:25:31.038599] [INFO ] [REMOTE ] EAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:31.293328] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T18:25:31.438917] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:25:31.451329] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-05. [2025-10-20T18:25:32.251398] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-01. [2025-10-20T18:25:33.004812] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:25:33.051333] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-06. [2025-10-20T18:25:33.851382] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-01. [2025-10-20T18:25:33.939844] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:25:34.457578] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:25:34.651358] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-07. [2025-10-20T18:25:35.309769] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:25:35.451355] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-02. [2025-10-20T18:25:36.251360] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-02. [2025-10-20T18:25:36.455388] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:25:36.784332] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:25:37.051443] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-08. [2025-10-20T18:25:37.851351] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-09. [2025-10-20T18:25:38.095565] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T18:25:38.323924] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T18:25:38.593520] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:25:38.651377] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-09. [2025-10-20T18:25:39.451404] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-03. [2025-10-20T18:25:39.754789] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:25:40.251403] [INFO ] [REMOTE ] EAF: Fetching primary data for 2025-10. [2025-10-20T18:25:41.051383] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-03. [2025-10-20T18:25:41.157263] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:25:41.361380] [INFO ] [REMOTE ] EAF: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:25:41.366517] [INFO ] [REMOTE ] EAF: Full fetch complete (62351 records). [2025-10-20T18:25:41.366768] [DEBUG] [VALIDATOR ] EAF: Validating primary data. [2025-10-20T18:25:41.770726] [DEBUG] [VALIDATOR ] EAF: All expected trading days are present. [2025-10-20T18:25:41.770819] [DEBUG] [VALIDATOR ] EAF: Primary data validation passed. [2025-10-20T18:25:41.771583] [INFO ] [MAIN ] EAF: Fetched primary data is valid. [2025-10-20T18:25:41.771623] [DEBUG] [LOCAL ] EAF: Saving primary data. [2025-10-20T18:25:41.772852] [DEBUG] [LOCAL ] EAF: Trimmed 62351 records to 41341 (keeping 200 days of data) [2025-10-20T18:25:41.853504] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-04. [2025-10-20T18:25:42.002671] [DEBUG] [LOCAL ] EAF: Primary data saved (41341 records). [2025-10-20T18:25:42.002806] [INFO ] [MAIN ] EAF: Full history saved. [2025-10-20T18:25:42.002873] [INFO ] [MAIN ] Update Progress: 674/2367 symbols completed. EAF took 29.6s. Estimated remaining: 3h 35m 5s. Rate: 7.9 symbols/min [2025-10-20T18:25:42.171355] [INFO ] [MAIN ] EB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:42.205549] [DEBUG] [LOCAL ] EB: Loaded 36946 existing records from primary storage. [2025-10-20T18:25:42.205878] [INFO ] [MAIN ] EB: Loaded 36946 existing records. [2025-10-20T18:25:42.386425] [WARN ] [MAIN ] EB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:42.386733] [INFO ] [MAIN ] EB: Wiping existing primary data before full refresh. [2025-10-20T18:25:42.386786] [INFO ] [LOCAL ] EB: Starting deletion of primary data. [2025-10-20T18:25:42.386966] [DEBUG] [LOCAL ] EB: Deleted primary data file. [2025-10-20T18:25:42.387003] [INFO ] [MAIN ] EB: Starting full primary refresh. [2025-10-20T18:25:42.656979] [INFO ] [REMOTE ] EAI: Fetching primary data for 2025-10. [2025-10-20T18:25:42.749169] [INFO ] [REMOTE ] EB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:42.772473] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:25:43.281432] [INFO ] [REMOTE ] EAI: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:25:43.282529] [INFO ] [REMOTE ] EAI: Full fetch complete (5878 records). [2025-10-20T18:25:43.282617] [DEBUG] [VALIDATOR ] EAI: Validating primary data. [2025-10-20T18:25:43.454364] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-01. [2025-10-20T18:25:43.705363] [DEBUG] [VALIDATOR ] EAI: All expected trading days are present. [2025-10-20T18:25:43.705586] [DEBUG] [VALIDATOR ] EAI: Primary data validation passed. [2025-10-20T18:25:43.705862] [INFO ] [MAIN ] EAI: Fetched primary data is valid. [2025-10-20T18:25:43.705947] [DEBUG] [LOCAL ] EAI: Saving primary data. [2025-10-20T18:25:43.706679] [DEBUG] [LOCAL ] EAI: Trimmed 5878 records to 3940 (keeping 200 days of data) [2025-10-20T18:25:43.762064] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T18:25:43.777143] [DEBUG] [LOCAL ] EAI: Primary data saved (3940 records). [2025-10-20T18:25:43.777263] [INFO ] [MAIN ] EAI: Full history saved. [2025-10-20T18:25:43.777335] [INFO ] [MAIN ] Update Progress: 675/2367 symbols completed. EAI took 21.9s. Estimated remaining: 3h 34m 43s. Rate: 7.9 symbols/min [2025-10-20T18:25:43.948388] [INFO ] [MAIN ] EBF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:25:43.972534] [DEBUG] [LOCAL ] EBF: Loaded 19324 existing records from primary storage. [2025-10-20T18:25:43.972898] [INFO ] [MAIN ] EBF: Loaded 19324 existing records. [2025-10-20T18:25:44.180731] [WARN ] [MAIN ] EBF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:25:44.180972] [INFO ] [MAIN ] EBF: Wiping existing primary data before full refresh. [2025-10-20T18:25:44.181018] [INFO ] [LOCAL ] EBF: Starting deletion of primary data. [2025-10-20T18:25:44.181205] [DEBUG] [LOCAL ] EBF: Deleted primary data file. [2025-10-20T18:25:44.181260] [INFO ] [MAIN ] EBF: Starting full primary refresh. [2025-10-20T18:25:44.256487] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-05. [2025-10-20T18:25:44.538453] [INFO ] [REMOTE ] EBF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:25:44.716416] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:25:45.051419] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-02. [2025-10-20T18:25:45.851337] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-04. [2025-10-20T18:25:45.934050] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:25:45.990575] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:25:46.651393] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-01. [2025-10-20T18:25:47.451366] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-06. [2025-10-20T18:25:47.503404] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:25:47.717004] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T18:25:48.251398] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-05. [2025-10-20T18:25:49.051356] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-02. [2025-10-20T18:25:49.117142] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:25:49.776767] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:25:49.851404] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-03. [2025-10-20T18:25:50.200212] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:25:50.651406] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-03. [2025-10-20T18:25:51.215426] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T18:25:51.356342] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T18:25:51.451366] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-04. [2025-10-20T18:25:52.251347] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-07. [2025-10-20T18:25:52.631288] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:25:53.051374] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-04. [2025-10-20T18:25:53.851366] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-06. [2025-10-20T18:25:53.957091] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T18:25:54.513118] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:25:54.603031] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T18:25:54.651350] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-05. [2025-10-20T18:25:55.451404] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-07. [2025-10-20T18:25:56.073334] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T18:25:56.251364] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-08. [2025-10-20T18:25:56.284262] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T18:25:57.051384] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-06. [2025-10-20T18:25:57.851359] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-08. [2025-10-20T18:25:57.929164] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:25:58.396447] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:25:58.651395] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-09. [2025-10-20T18:25:58.775597] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T18:25:59.451407] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-07. [2025-10-20T18:26:00.251413] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-09. [2025-10-20T18:26:00.359857] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:26:01.051412] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-05. [2025-10-20T18:26:01.489094] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:26:01.856592] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-08. [2025-10-20T18:26:01.860315] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:26:02.651381] [INFO ] [REMOTE ] EARN: Fetching primary data for 2025-10. [2025-10-20T18:26:03.053705] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T18:26:03.280527] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T18:26:03.451430] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-09. [2025-10-20T18:26:03.934226] [INFO ] [REMOTE ] EARN: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T18:26:03.938906] [INFO ] [REMOTE ] EARN: Full fetch complete (45360 records). [2025-10-20T18:26:03.939050] [DEBUG] [VALIDATOR ] EARN: Validating primary data. [2025-10-20T18:26:04.256499] [INFO ] [REMOTE ] EAT: Fetching primary data for 2025-10. [2025-10-20T18:26:04.335690] [DEBUG] [VALIDATOR ] EARN: All expected trading days are present. [2025-10-20T18:26:04.335796] [DEBUG] [VALIDATOR ] EARN: Primary data validation passed. [2025-10-20T18:26:04.336530] [INFO ] [MAIN ] EARN: Fetched primary data is valid. [2025-10-20T18:26:04.336598] [DEBUG] [LOCAL ] EARN: Saving primary data. [2025-10-20T18:26:04.338010] [DEBUG] [LOCAL ] EARN: Trimmed 45360 records to 28002 (keeping 200 days of data) [2025-10-20T18:26:04.500337] [DEBUG] [LOCAL ] EARN: Primary data saved (28002 records). [2025-10-20T18:26:04.500484] [INFO ] [MAIN ] EARN: Full history saved. [2025-10-20T18:26:04.500565] [INFO ] [MAIN ] Update Progress: 676/2367 symbols completed. EARN took 40.0s. Estimated remaining: 3h 35m 8s. Rate: 7.9 symbols/min [2025-10-20T18:26:04.676459] [INFO ] [MAIN ] EBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:04.696352] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T18:26:04.738524] [DEBUG] [LOCAL ] EBR: Loaded 43964 existing records from primary storage. [2025-10-20T18:26:04.738867] [INFO ] [MAIN ] EBR: Loaded 43964 existing records. [2025-10-20T18:26:04.928868] [WARN ] [MAIN ] EBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:04.929086] [INFO ] [MAIN ] EBR: Wiping existing primary data before full refresh. [2025-10-20T18:26:04.929135] [INFO ] [LOCAL ] EBR: Starting deletion of primary data. [2025-10-20T18:26:04.929605] [DEBUG] [LOCAL ] EBR: Deleted primary data file. [2025-10-20T18:26:04.929706] [INFO ] [MAIN ] EBR: Starting full primary refresh. [2025-10-20T18:26:05.056559] [INFO ] [REMOTE ] EB: Fetching primary data for 2025-10. [2025-10-20T18:26:05.320833] [INFO ] [REMOTE ] EBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:05.851420] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-01. [2025-10-20T18:26:06.146417] [INFO ] [REMOTE ] EAT: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:26:06.154226] [INFO ] [REMOTE ] EAT: Full fetch complete (72389 records). [2025-10-20T18:26:06.154399] [DEBUG] [VALIDATOR ] EAT: Validating primary data. [2025-10-20T18:26:06.543478] [DEBUG] [VALIDATOR ] EAT: All expected trading days are present. [2025-10-20T18:26:06.543630] [DEBUG] [VALIDATOR ] EAT: Primary data validation passed. [2025-10-20T18:26:06.544555] [INFO ] [MAIN ] EAT: Fetched primary data is valid. [2025-10-20T18:26:06.544605] [DEBUG] [LOCAL ] EAT: Saving primary data. [2025-10-20T18:26:06.546194] [DEBUG] [LOCAL ] EAT: Trimmed 72389 records to 49806 (keeping 200 days of data) [2025-10-20T18:26:06.656805] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-06. [2025-10-20T18:26:06.781851] [INFO ] [REMOTE ] EB: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:26:06.791844] [INFO ] [REMOTE ] EB: Full fetch complete (55115 records). [2025-10-20T18:26:06.793888] [DEBUG] [VALIDATOR ] EB: Validating primary data. [2025-10-20T18:26:07.239912] [DEBUG] [LOCAL ] EAT: Primary data saved (49806 records). [2025-10-20T18:26:07.254950] [INFO ] [MAIN ] EAT: Full history saved. [2025-10-20T18:26:07.255751] [INFO ] [MAIN ] Update Progress: 677/2367 symbols completed. EAT took 37.0s. Estimated remaining: 3h 34m 48s. Rate: 7.9 symbols/min [2025-10-20T18:26:07.377552] [DEBUG] [VALIDATOR ] EB: All expected trading days are present. [2025-10-20T18:26:07.377717] [DEBUG] [VALIDATOR ] EB: Primary data validation passed. [2025-10-20T18:26:07.378728] [INFO ] [MAIN ] EB: Fetched primary data is valid. [2025-10-20T18:26:07.378889] [DEBUG] [LOCAL ] EB: Saving primary data. [2025-10-20T18:26:07.380409] [DEBUG] [LOCAL ] EB: Trimmed 55115 records to 36273 (keeping 200 days of data) [2025-10-20T18:26:07.507329] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:26:07.518833] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-02. [2025-10-20T18:26:07.644410] [DEBUG] [LOCAL ] EB: Primary data saved (36273 records). [2025-10-20T18:26:07.645542] [INFO ] [MAIN ] EB: Full history saved. [2025-10-20T18:26:07.645721] [INFO ] [MAIN ] Update Progress: 678/2367 symbols completed. EB took 25.6s. Estimated remaining: 3h 34m 22s. Rate: 7.9 symbols/min [2025-10-20T18:26:07.646787] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T18:26:08.251505] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-07. [2025-10-20T18:26:08.953018] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:26:09.051430] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-03. [2025-10-20T18:26:09.234577] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T18:26:09.851380] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-08. [2025-10-20T18:26:10.445934] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:26:10.651482] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-04. [2025-10-20T18:26:10.727286] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:26:11.451429] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-09. [2025-10-20T18:26:12.363715] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T18:26:12.375732] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-05. [2025-10-20T18:26:12.466534] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:26:13.051445] [INFO ] [REMOTE ] EBF: Fetching primary data for 2025-10. [2025-10-20T18:26:13.955999] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:26:13.967007] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-06. [2025-10-20T18:26:14.256452] [INFO ] [REMOTE ] EBF: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T18:26:14.260586] [INFO ] [REMOTE ] EBF: Full fetch complete (28078 records). [2025-10-20T18:26:14.260725] [DEBUG] [VALIDATOR ] EBF: Validating primary data. [2025-10-20T18:26:14.634055] [DEBUG] [VALIDATOR ] EBF: All expected trading days are present. [2025-10-20T18:26:14.634140] [DEBUG] [VALIDATOR ] EBF: Primary data validation passed. [2025-10-20T18:26:14.634708] [INFO ] [MAIN ] EBF: Fetched primary data is valid. [2025-10-20T18:26:14.634771] [DEBUG] [LOCAL ] EBF: Saving primary data. [2025-10-20T18:26:14.635510] [DEBUG] [LOCAL ] EBF: Trimmed 28078 records to 19140 (keeping 200 days of data) [2025-10-20T18:26:14.750001] [DEBUG] [LOCAL ] EBF: Primary data saved (19140 records). [2025-10-20T18:26:14.750148] [INFO ] [MAIN ] EBF: Full history saved. [2025-10-20T18:26:14.750227] [INFO ] [MAIN ] Update Progress: 679/2367 symbols completed. EBF took 31.0s. Estimated remaining: 3h 34m 14s. Rate: 7.9 symbols/min [2025-10-20T18:26:15.473613] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T18:26:15.482200] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-07. [2025-10-20T18:26:16.885463] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T18:26:16.894973] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-08. [2025-10-20T18:26:18.156906] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T18:26:18.171356] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-09. [2025-10-20T18:26:19.728017] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T18:26:19.737813] [INFO ] [REMOTE ] EBR: Fetching primary data for 2025-10. [2025-10-20T18:26:21.532820] [INFO ] [REMOTE ] EBR: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T18:26:21.539822] [INFO ] [REMOTE ] EBR: Full fetch complete (62557 records). [2025-10-20T18:26:21.539971] [DEBUG] [VALIDATOR ] EBR: Validating primary data. [2025-10-20T18:26:21.945175] [DEBUG] [VALIDATOR ] EBR: All expected trading days are present. [2025-10-20T18:26:21.945271] [DEBUG] [VALIDATOR ] EBR: Primary data validation passed. [2025-10-20T18:26:21.946237] [INFO ] [MAIN ] EBR: Fetched primary data is valid. [2025-10-20T18:26:21.946304] [DEBUG] [LOCAL ] EBR: Saving primary data. [2025-10-20T18:26:21.947753] [DEBUG] [LOCAL ] EBR: Trimmed 62557 records to 43386 (keeping 200 days of data) [2025-10-20T18:26:22.201806] [DEBUG] [LOCAL ] EBR: Primary data saved (43386 records). [2025-10-20T18:26:22.201939] [INFO ] [MAIN ] EBR: Full history saved. [2025-10-20T18:26:22.202018] [INFO ] [MAIN ] Update Progress: 680/2367 symbols completed. EBR took 17.7s. Estimated remaining: 3h 34m 6s. Rate: 7.9 symbols/min [2025-10-20T18:26:22.835440] [INFO ] [MAIN ] ECC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.863472] [INFO ] [MAIN ] EBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.933237] [INFO ] [MAIN ] ECAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.934807] [INFO ] [MAIN ] EC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:22.997944] [DEBUG] [LOCAL ] ECAT: Loaded 29841 existing records from primary storage. [2025-10-20T18:26:22.998430] [INFO ] [MAIN ] ECAT: Loaded 29841 existing records. [2025-10-20T18:26:23.048827] [DEBUG] [LOCAL ] ECC: Loaded 43973 existing records from primary storage. [2025-10-20T18:26:23.093148] [INFO ] [MAIN ] ECC: Loaded 43973 existing records. [2025-10-20T18:26:23.151833] [DEBUG] [LOCAL ] EC: Loaded 49674 existing records from primary storage. [2025-10-20T18:26:23.145872] [DEBUG] [LOCAL ] EBS: Loaded 47428 existing records from primary storage. [2025-10-20T18:26:23.152591] [INFO ] [MAIN ] EBS: Loaded 47428 existing records. [2025-10-20T18:26:23.152701] [INFO ] [MAIN ] EC: Loaded 49674 existing records. [2025-10-20T18:26:23.460948] [WARN ] [MAIN ] ECAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.557859] [INFO ] [MAIN ] ECAT: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.558157] [INFO ] [LOCAL ] ECAT: Starting deletion of primary data. [2025-10-20T18:26:23.585439] [DEBUG] [LOCAL ] ECAT: Deleted primary data file. [2025-10-20T18:26:23.692822] [INFO ] [MAIN ] ECAT: Starting full primary refresh. [2025-10-20T18:26:23.775951] [WARN ] [MAIN ] EC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.816333] [WARN ] [MAIN ] ECC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.830455] [WARN ] [MAIN ] EBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:23.856768] [INFO ] [MAIN ] ECC: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.866820] [INFO ] [LOCAL ] ECC: Starting deletion of primary data. [2025-10-20T18:26:23.856442] [INFO ] [MAIN ] EC: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.856597] [INFO ] [MAIN ] EBS: Wiping existing primary data before full refresh. [2025-10-20T18:26:23.889532] [INFO ] [LOCAL ] EC: Starting deletion of primary data. [2025-10-20T18:26:23.889463] [DEBUG] [LOCAL ] ECC: Deleted primary data file. [2025-10-20T18:26:23.910262] [INFO ] [LOCAL ] EBS: Starting deletion of primary data. [2025-10-20T18:26:23.915592] [INFO ] [MAIN ] ECC: Starting full primary refresh. [2025-10-20T18:26:23.919680] [DEBUG] [LOCAL ] EC: Deleted primary data file. [2025-10-20T18:26:23.930795] [INFO ] [MAIN ] EC: Starting full primary refresh. [2025-10-20T18:26:23.930716] [DEBUG] [LOCAL ] EBS: Deleted primary data file. [2025-10-20T18:26:23.930964] [INFO ] [MAIN ] EBS: Starting full primary refresh. [2025-10-20T18:26:24.972988] [INFO ] [REMOTE ] ECAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.022844] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-01. [2025-10-20T18:26:25.181251] [INFO ] [REMOTE ] EBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.272344] [INFO ] [REMOTE ] EC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.298374] [INFO ] [REMOTE ] ECC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:25.298987] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-01. [2025-10-20T18:26:25.298636] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-01. [2025-10-20T18:26:25.300704] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-01. [2025-10-20T18:26:26.654167] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:26:26.660289] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-02. [2025-10-20T18:26:26.884892] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:26:26.892911] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-02. [2025-10-20T18:26:26.994029] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:26:27.003978] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-02. [2025-10-20T18:26:27.086908] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:26:27.096391] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-02. [2025-10-20T18:26:27.642525] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:26:27.648582] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-03. [2025-10-20T18:26:28.059087] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T18:26:28.067235] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-03. [2025-10-20T18:26:28.571612] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:26:28.582375] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-03. [2025-10-20T18:26:28.608087] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T18:26:28.615762] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-04. [2025-10-20T18:26:28.655846] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:26:28.667269] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-03. [2025-10-20T18:26:29.775633] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:26:29.793218] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-04. [2025-10-20T18:26:29.912913] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:26:29.919785] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-05. [2025-10-20T18:26:30.089097] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:26:30.099165] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-04. [2025-10-20T18:26:30.450999] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:26:30.651457] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-04. [2025-10-20T18:26:30.833466] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:26:31.451448] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-06. [2025-10-20T18:26:31.511812] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:26:31.602900] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:26:32.079638] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:26:32.251394] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-05. [2025-10-20T18:26:32.709305] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:26:33.051396] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-07. [2025-10-20T18:26:33.834763] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:26:33.851347] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-05. [2025-10-20T18:26:34.395929] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:26:34.651407] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-06. [2025-10-20T18:26:35.451366] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-08. [2025-10-20T18:26:35.541295] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T18:26:36.117829] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:26:36.251372] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-07. [2025-10-20T18:26:36.524800] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:26:37.051381] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-05. [2025-10-20T18:26:37.851349] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-06. [2025-10-20T18:26:38.119430] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:26:38.651376] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-09. [2025-10-20T18:26:38.915715] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:26:39.463500] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-06. [2025-10-20T18:26:39.465874] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:26:39.916692] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:26:40.251367] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-08. [2025-10-20T18:26:41.051429] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-07. [2025-10-20T18:26:41.111554] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:26:41.851354] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-07. [2025-10-20T18:26:41.916005] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:26:42.651411] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-09. [2025-10-20T18:26:42.842348] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:26:43.154126] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T18:26:43.451388] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-08. [2025-10-20T18:26:44.251423] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-08. [2025-10-20T18:26:44.416572] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T18:26:45.051432] [INFO ] [REMOTE ] EC: Fetching primary data for 2025-10. [2025-10-20T18:26:45.143123] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T18:26:45.851393] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-09. [2025-10-20T18:26:45.903488] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:26:46.651442] [INFO ] [REMOTE ] ECAT: Fetching primary data for 2025-10. [2025-10-20T18:26:46.807816] [INFO ] [REMOTE ] EC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:26:46.814908] [INFO ] [REMOTE ] EC: Full fetch complete (71887 records). [2025-10-20T18:26:46.815066] [DEBUG] [VALIDATOR ] EC: Validating primary data. [2025-10-20T18:26:47.210591] [DEBUG] [VALIDATOR ] EC: All expected trading days are present. [2025-10-20T18:26:47.210731] [DEBUG] [VALIDATOR ] EC: Primary data validation passed. [2025-10-20T18:26:47.211547] [INFO ] [MAIN ] EC: Fetched primary data is valid. [2025-10-20T18:26:47.211591] [DEBUG] [LOCAL ] EC: Saving primary data. [2025-10-20T18:26:47.212904] [DEBUG] [LOCAL ] EC: Trimmed 71887 records to 48929 (keeping 200 days of data) [2025-10-20T18:26:47.465706] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-09. [2025-10-20T18:26:47.497227] [DEBUG] [LOCAL ] EC: Primary data saved (48929 records). [2025-10-20T18:26:47.497766] [INFO ] [MAIN ] EC: Full history saved. [2025-10-20T18:26:47.497892] [INFO ] [MAIN ] Update Progress: 681/2367 symbols completed. EC took 25.3s. Estimated remaining: 3h 34m 42s. Rate: 7.9 symbols/min [2025-10-20T18:26:47.636488] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:26:47.713974] [INFO ] [MAIN ] ECCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:47.714188] [DEBUG] [LOCAL ] ECCC: No existing data file found in primary storage. [2025-10-20T18:26:47.714233] [WARN ] [MAIN ] ECCC: No existing data found, performing full refresh instead. [2025-10-20T18:26:47.714348] [INFO ] [MAIN ] ECCC: Wiping existing primary data before full refresh. [2025-10-20T18:26:47.714391] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-10-20T18:26:47.714433] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-10-20T18:26:47.714466] [INFO ] [MAIN ] ECCC: Starting full primary refresh. [2025-10-20T18:26:48.009922] [INFO ] [REMOTE ] ECAT: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:26:48.049563] [INFO ] [REMOTE ] ECAT: Full fetch complete (42976 records). [2025-10-20T18:26:48.051985] [DEBUG] [VALIDATOR ] ECAT: Validating primary data. [2025-10-20T18:26:48.239231] [INFO ] [REMOTE ] ECCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:48.254746] [INFO ] [REMOTE ] ECC: Fetching primary data for 2025-10. [2025-10-20T18:26:48.507049] [DEBUG] [VALIDATOR ] ECAT: All expected trading days are present. [2025-10-20T18:26:48.507139] [DEBUG] [VALIDATOR ] ECAT: Primary data validation passed. [2025-10-20T18:26:48.507894] [INFO ] [MAIN ] ECAT: Fetched primary data is valid. [2025-10-20T18:26:48.507962] [DEBUG] [LOCAL ] ECAT: Saving primary data. [2025-10-20T18:26:48.509078] [DEBUG] [LOCAL ] ECAT: Trimmed 42976 records to 29346 (keeping 200 days of data) [2025-10-20T18:26:48.686471] [DEBUG] [LOCAL ] ECAT: Primary data saved (29346 records). [2025-10-20T18:26:48.686606] [INFO ] [MAIN ] ECAT: Full history saved. [2025-10-20T18:26:48.686712] [INFO ] [MAIN ] Update Progress: 682/2367 symbols completed. ECAT took 26.5s. Estimated remaining: 3h 34m 18s. Rate: 7.9 symbols/min [2025-10-20T18:26:48.864681] [INFO ] [MAIN ] ECCU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:48.896110] [DEBUG] [LOCAL ] ECCU: Loaded 2234 existing records from primary storage. [2025-10-20T18:26:48.893941] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:26:48.905527] [INFO ] [MAIN ] ECCU: Loaded 2234 existing records. [2025-10-20T18:26:49.056553] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-01. [2025-10-20T18:26:49.087556] [WARN ] [MAIN ] ECCU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:26:49.087841] [INFO ] [MAIN ] ECCU: Wiping existing primary data before full refresh. [2025-10-20T18:26:49.087911] [INFO ] [LOCAL ] ECCU: Starting deletion of primary data. [2025-10-20T18:26:49.088117] [DEBUG] [LOCAL ] ECCU: Deleted primary data file. [2025-10-20T18:26:49.088182] [INFO ] [MAIN ] ECCU: Starting full primary refresh. [2025-10-20T18:26:49.443080] [INFO ] [REMOTE ] ECCU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:49.565145] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:26:49.851387] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-02. [2025-10-20T18:26:50.148472] [INFO ] [REMOTE ] ECC: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T18:26:50.156076] [INFO ] [REMOTE ] ECC: Full fetch complete (61538 records). [2025-10-20T18:26:50.156215] [DEBUG] [VALIDATOR ] ECC: Validating primary data. [2025-10-20T18:26:50.246021] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T18:26:50.625154] [DEBUG] [VALIDATOR ] ECC: All expected trading days are present. [2025-10-20T18:26:50.625247] [DEBUG] [VALIDATOR ] ECC: Primary data validation passed. [2025-10-20T18:26:50.626399] [INFO ] [MAIN ] ECC: Fetched primary data is valid. [2025-10-20T18:26:50.626502] [DEBUG] [LOCAL ] ECC: Saving primary data. [2025-10-20T18:26:50.628278] [DEBUG] [LOCAL ] ECC: Trimmed 61538 records to 43475 (keeping 200 days of data) [2025-10-20T18:26:50.690053] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-03. [2025-10-20T18:26:50.912046] [DEBUG] [LOCAL ] ECC: Primary data saved (43475 records). [2025-10-20T18:26:50.912158] [INFO ] [MAIN ] ECC: Full history saved. [2025-10-20T18:26:50.912274] [INFO ] [MAIN ] Update Progress: 683/2367 symbols completed. ECC took 28.7s. Estimated remaining: 3h 33m 57s. Rate: 7.9 symbols/min [2025-10-20T18:26:51.086634] [INFO ] [MAIN ] ECCW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:26:51.088890] [DEBUG] [LOCAL ] ECCW: No existing data file found in primary storage. [2025-10-20T18:26:51.088993] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T18:26:51.090142] [WARN ] [MAIN ] ECCW: No existing data found, performing full refresh instead. [2025-10-20T18:26:51.090254] [INFO ] [MAIN ] ECCW: Wiping existing primary data before full refresh. [2025-10-20T18:26:51.090296] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-10-20T18:26:51.090337] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-10-20T18:26:51.090367] [INFO ] [MAIN ] ECCW: Starting full primary refresh. [2025-10-20T18:26:51.437926] [INFO ] [REMOTE ] ECCW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:26:51.451332] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-01. [2025-10-20T18:26:51.897069] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T18:26:52.251379] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-04. [2025-10-20T18:26:52.591950] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T18:26:53.051412] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-01. [2025-10-20T18:26:53.630895] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:26:53.851365] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-05. [2025-10-20T18:26:54.202183] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:26:54.651428] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-02. [2025-10-20T18:26:55.252862] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:26:55.451386] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-03. [2025-10-20T18:26:56.060208] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T18:26:56.251371] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-04. [2025-10-20T18:26:56.765373] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T18:26:57.051412] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-02. [2025-10-20T18:26:57.293319] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-10-20T18:26:57.851422] [INFO ] [REMOTE ] EBS: Fetching primary data for 2025-10. [2025-10-20T18:26:58.651373] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-03. [2025-10-20T18:26:58.988793] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-10-20T18:26:59.451433] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-04. [2025-10-20T18:26:59.615390] [INFO ] [REMOTE ] EBS: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T18:26:59.622252] [INFO ] [REMOTE ] EBS: Full fetch complete (67400 records). [2025-10-20T18:26:59.622403] [DEBUG] [VALIDATOR ] EBS: Validating primary data. [2025-10-20T18:26:59.728235] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:27:00.018195] [DEBUG] [VALIDATOR ] EBS: All expected trading days are present. [2025-10-20T18:27:00.018284] [DEBUG] [VALIDATOR ] EBS: Primary data validation passed. [2025-10-20T18:27:00.019086] [INFO ] [MAIN ] EBS: Fetched primary data is valid. [2025-10-20T18:27:00.019151] [DEBUG] [LOCAL ] EBS: Saving primary data. [2025-10-20T18:27:00.020513] [DEBUG] [LOCAL ] EBS: Trimmed 67400 records to 46671 (keeping 200 days of data) [2025-10-20T18:27:00.257189] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-06. [2025-10-20T18:27:00.282090] [DEBUG] [LOCAL ] EBS: Primary data saved (46671 records). [2025-10-20T18:27:00.282248] [INFO ] [MAIN ] EBS: Full history saved. [2025-10-20T18:27:00.282850] [INFO ] [MAIN ] Update Progress: 684/2367 symbols completed. EBS took 38.1s. Estimated remaining: 3h 33m 54s. Rate: 7.9 symbols/min [2025-10-20T18:27:00.455954] [INFO ] [MAIN ] ECCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:00.456152] [DEBUG] [LOCAL ] ECCX: No existing data file found in primary storage. [2025-10-20T18:27:00.456189] [WARN ] [MAIN ] ECCX: No existing data found, performing full refresh instead. [2025-10-20T18:27:00.456298] [INFO ] [MAIN ] ECCX: Wiping existing primary data before full refresh. [2025-10-20T18:27:00.456339] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-10-20T18:27:00.456379] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-10-20T18:27:00.456409] [INFO ] [MAIN ] ECCX: Starting full primary refresh. [2025-10-20T18:27:00.807338] [INFO ] [REMOTE ] ECCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:01.051376] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-01. [2025-10-20T18:27:01.093947] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:27:01.559932] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:27:01.851381] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-05. [2025-10-20T18:27:02.440683] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:27:02.651379] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-07. [2025-10-20T18:27:03.018999] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T18:27:03.451362] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-02. [2025-10-20T18:27:03.785253] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T18:27:04.251368] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-05. [2025-10-20T18:27:04.808328] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T18:27:05.051432] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-06. [2025-10-20T18:27:05.454308] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T18:27:05.851456] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-07. [2025-10-20T18:27:06.192344] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:27:06.651387] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-06. [2025-10-20T18:27:07.176125] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T18:27:07.451408] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-08. [2025-10-20T18:27:07.801868] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-10-20T18:27:08.251359] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-09. [2025-10-20T18:27:08.571047] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T18:27:09.051381] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-03. [2025-10-20T18:27:09.510173] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T18:27:09.851376] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-04. [2025-10-20T18:27:10.126549] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:27:10.651387] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-05. [2025-10-20T18:27:11.003731] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:27:11.451434] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-07. [2025-10-20T18:27:11.961561] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:27:12.251396] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-08. [2025-10-20T18:27:12.680423] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T18:27:13.051415] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-08. [2025-10-20T18:27:13.481357] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T18:27:13.851426] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-09. [2025-10-20T18:27:14.194562] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-10-20T18:27:14.651378] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-06. [2025-10-20T18:27:15.124310] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:27:15.451378] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-09. [2025-10-20T18:27:15.862679] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T18:27:16.251379] [INFO ] [REMOTE ] ECCU: Fetching primary data for 2025-10. [2025-10-20T18:27:17.007716] [INFO ] [REMOTE ] ECCU: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T18:27:17.008703] [INFO ] [REMOTE ] ECCU: Full fetch complete (4310 records). [2025-10-20T18:27:17.008773] [DEBUG] [VALIDATOR ] ECCU: Validating primary data. [2025-10-20T18:27:17.056595] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-07. [2025-10-20T18:27:17.384733] [DEBUG] [VALIDATOR ] ECCU: All expected trading days are present. [2025-10-20T18:27:17.384819] [DEBUG] [VALIDATOR ] ECCU: Primary data validation passed. [2025-10-20T18:27:17.384963] [INFO ] [MAIN ] ECCU: Fetched primary data is valid. [2025-10-20T18:27:17.385002] [DEBUG] [LOCAL ] ECCU: Saving primary data. [2025-10-20T18:27:17.385420] [DEBUG] [LOCAL ] ECCU: Trimmed 4310 records to 2179 (keeping 200 days of data) [2025-10-20T18:27:17.400109] [DEBUG] [LOCAL ] ECCU: Primary data saved (2179 records). [2025-10-20T18:27:17.400243] [INFO ] [MAIN ] ECCU: Full history saved. [2025-10-20T18:27:17.400326] [INFO ] [MAIN ] Update Progress: 685/2367 symbols completed. ECCU took 28.7s. Estimated remaining: 3h 34m 9s. Rate: 7.9 symbols/min [2025-10-20T18:27:17.582520] [INFO ] [MAIN ] ECG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:17.585690] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T18:27:17.622741] [DEBUG] [LOCAL ] ECG: Loaded 33657 existing records from primary storage. [2025-10-20T18:27:17.623032] [INFO ] [MAIN ] ECG: Loaded 33657 existing records. [2025-10-20T18:27:17.804806] [WARN ] [MAIN ] ECG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:17.805036] [INFO ] [MAIN ] ECG: Wiping existing primary data before full refresh. [2025-10-20T18:27:17.805084] [INFO ] [LOCAL ] ECG: Starting deletion of primary data. [2025-10-20T18:27:17.805286] [DEBUG] [LOCAL ] ECG: Deleted primary data file. [2025-10-20T18:27:17.805325] [INFO ] [MAIN ] ECG: Starting full primary refresh. [2025-10-20T18:27:17.858599] [INFO ] [REMOTE ] ECCC: Fetching primary data for 2025-10. [2025-10-20T18:27:18.167605] [INFO ] [REMOTE ] ECG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:18.319838] [INFO ] [REMOTE ] ECCC: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:27:18.320693] [INFO ] [REMOTE ] ECCC: Full fetch complete (1299 records). [2025-10-20T18:27:18.320765] [DEBUG] [VALIDATOR ] ECCC: Validating primary data. [2025-10-20T18:27:18.321145] [FATAL] [VALIDATOR ] ECCC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:27:18.321228] [FATAL] [MAIN ] ECCC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:18.321271] [WARN ] [MAIN ] ECCC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:18.321338] [INFO ] [LOCAL ] ECCC: Starting deletion of primary data. [2025-10-20T18:27:18.321401] [DEBUG] [LOCAL ] ECCC: No primary data file found to delete. [2025-10-20T18:27:18.321438] [INFO ] [LOCAL ] ECCC: Starting deletion of derivative data. [2025-10-20T18:27:18.321478] [DEBUG] [LOCAL ] ECCC: No derivative data file found to delete. [2025-10-20T18:27:18.321531] [INFO ] [MAIN ] Update Progress: 686/2367 symbols completed. ECCC took 30.8s. Estimated remaining: 3h 33m 45s. Rate: 7.9 symbols/min [2025-10-20T18:27:18.529819] [INFO ] [MAIN ] ECL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:18.575791] [DEBUG] [LOCAL ] ECL: Loaded 47419 existing records from primary storage. [2025-10-20T18:27:18.576156] [INFO ] [MAIN ] ECL: Loaded 47419 existing records. [2025-10-20T18:27:18.656630] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-01. [2025-10-20T18:27:18.777553] [WARN ] [MAIN ] ECL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:18.777792] [INFO ] [MAIN ] ECL: Wiping existing primary data before full refresh. [2025-10-20T18:27:18.777843] [INFO ] [LOCAL ] ECL: Starting deletion of primary data. [2025-10-20T18:27:18.778466] [DEBUG] [LOCAL ] ECL: Deleted primary data file. [2025-10-20T18:27:18.778523] [INFO ] [MAIN ] ECL: Starting full primary refresh. [2025-10-20T18:27:19.122061] [INFO ] [REMOTE ] ECL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:19.451380] [INFO ] [REMOTE ] ECCW: Fetching primary data for 2025-10. [2025-10-20T18:27:20.149338] [INFO ] [REMOTE ] ECCW: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T18:27:20.151464] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:27:20.150675] [INFO ] [REMOTE ] ECCW: Full fetch complete (1424 records). [2025-10-20T18:27:20.162958] [DEBUG] [VALIDATOR ] ECCW: Validating primary data. [2025-10-20T18:27:20.256549] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-08. [2025-10-20T18:27:20.376316] [WARN ] [VALIDATOR ] ECCW: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T18:27:20.376530] [FATAL] [MAIN ] ECCW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:20.376599] [WARN ] [MAIN ] ECCW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:20.376698] [INFO ] [LOCAL ] ECCW: Starting deletion of primary data. [2025-10-20T18:27:20.376766] [DEBUG] [LOCAL ] ECCW: No primary data file found to delete. [2025-10-20T18:27:20.376803] [INFO ] [LOCAL ] ECCW: Starting deletion of derivative data. [2025-10-20T18:27:20.376845] [DEBUG] [LOCAL ] ECCW: No derivative data file found to delete. [2025-10-20T18:27:20.376900] [INFO ] [MAIN ] Update Progress: 687/2367 symbols completed. ECCW took 29.5s. Estimated remaining: 3h 33m 24s. Rate: 7.9 symbols/min [2025-10-20T18:27:20.547714] [INFO ] [MAIN ] ECO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:20.565783] [DEBUG] [LOCAL ] ECO: Loaded 18402 existing records from primary storage. [2025-10-20T18:27:20.566037] [INFO ] [MAIN ] ECO: Loaded 18402 existing records. [2025-10-20T18:27:20.668406] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T18:27:20.746801] [WARN ] [MAIN ] ECO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:20.747031] [INFO ] [MAIN ] ECO: Wiping existing primary data before full refresh. [2025-10-20T18:27:20.747079] [INFO ] [LOCAL ] ECO: Starting deletion of primary data. [2025-10-20T18:27:20.747275] [DEBUG] [LOCAL ] ECO: Deleted primary data file. [2025-10-20T18:27:20.747338] [INFO ] [MAIN ] ECO: Starting full primary refresh. [2025-10-20T18:27:21.062119] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-09. [2025-10-20T18:27:21.143185] [INFO ] [REMOTE ] ECO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:21.590577] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:27:21.851363] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-01. [2025-10-20T18:27:22.651402] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-01. [2025-10-20T18:27:23.175468] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:27:23.451371] [INFO ] [REMOTE ] ECCX: Fetching primary data for 2025-10. [2025-10-20T18:27:23.917092] [INFO ] [REMOTE ] ECCX: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T18:27:23.917848] [INFO ] [REMOTE ] ECCX: Full fetch complete (1450 records). [2025-10-20T18:27:23.917917] [DEBUG] [VALIDATOR ] ECCX: Validating primary data. [2025-10-20T18:27:24.096601] [WARN ] [VALIDATOR ] ECCX: Validation failed - insufficient historical data. Have 195 unique days, need 200 days [2025-10-20T18:27:24.096837] [FATAL] [MAIN ] ECCX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:24.096922] [WARN ] [MAIN ] ECCX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:24.097015] [INFO ] [LOCAL ] ECCX: Starting deletion of primary data. [2025-10-20T18:27:24.097097] [DEBUG] [LOCAL ] ECCX: No primary data file found to delete. [2025-10-20T18:27:24.097134] [INFO ] [LOCAL ] ECCX: Starting deletion of derivative data. [2025-10-20T18:27:24.097172] [DEBUG] [LOCAL ] ECCX: No derivative data file found to delete. [2025-10-20T18:27:24.097226] [INFO ] [MAIN ] Update Progress: 688/2367 symbols completed. ECCX took 23.8s. Estimated remaining: 3h 33m 7s. Rate: 7.9 symbols/min [2025-10-20T18:27:24.256413] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-02. [2025-10-20T18:27:24.285565] [INFO ] [MAIN ] ECVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:24.330963] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:27:24.377105] [DEBUG] [LOCAL ] ECVT: Loaded 43948 existing records from primary storage. [2025-10-20T18:27:24.377421] [INFO ] [MAIN ] ECVT: Loaded 43948 existing records. [2025-10-20T18:27:24.564355] [WARN ] [MAIN ] ECVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:24.564522] [INFO ] [MAIN ] ECVT: Wiping existing primary data before full refresh. [2025-10-20T18:27:24.564567] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-10-20T18:27:24.564925] [DEBUG] [LOCAL ] ECVT: Deleted primary data file. [2025-10-20T18:27:24.564980] [INFO ] [MAIN ] ECVT: Starting full primary refresh. [2025-10-20T18:27:24.909318] [INFO ] [REMOTE ] ECVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:25.051332] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-01. [2025-10-20T18:27:25.722799] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:27:25.851360] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-03. [2025-10-20T18:27:26.452674] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:27:26.651392] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-02. [2025-10-20T18:27:27.277761] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:27:27.451420] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-04. [2025-10-20T18:27:27.805893] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:27:28.251393] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-02. [2025-10-20T18:27:28.612413] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:27:29.051417] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-02. [2025-10-20T18:27:29.851361] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-03. [2025-10-20T18:27:29.916448] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:27:30.428970] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:27:30.651364] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-03. [2025-10-20T18:27:31.037746] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:27:31.451378] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-03. [2025-10-20T18:27:32.003294] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T18:27:32.251381] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-04. [2025-10-20T18:27:33.050981] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T18:27:33.060509] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-05. [2025-10-20T18:27:33.686189] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:27:33.851373] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-04. [2025-10-20T18:27:34.577602] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:27:34.651374] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-05. [2025-10-20T18:27:34.760347] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:27:35.451369] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-04. [2025-10-20T18:27:35.818190] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:27:36.251379] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-05. [2025-10-20T18:27:37.059780] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-06. [2025-10-20T18:27:37.085008] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:27:37.292181] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:27:37.851369] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-05. [2025-10-20T18:27:38.651380] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-06. [2025-10-20T18:27:38.755593] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T18:27:39.388184] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:27:39.451371] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-07. [2025-10-20T18:27:39.527607] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T18:27:40.251400] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-06. [2025-10-20T18:27:40.652499] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T18:27:41.051427] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-07. [2025-10-20T18:27:41.213164] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T18:27:41.851450] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-06. [2025-10-20T18:27:42.116440] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:27:42.651341] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-08. [2025-10-20T18:27:43.232058] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:27:43.451445] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-08. [2025-10-20T18:27:43.672433] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T18:27:44.251416] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-09. [2025-10-20T18:27:44.947271] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:27:45.051402] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-07. [2025-10-20T18:27:45.660392] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:27:45.851391] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-07. [2025-10-20T18:27:46.651423] [INFO ] [REMOTE ] ECG: Fetching primary data for 2025-10. [2025-10-20T18:27:46.854408] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:27:46.905965] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:27:47.451397] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-08. [2025-10-20T18:27:47.666327] [INFO ] [REMOTE ] ECG: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:27:47.671270] [INFO ] [REMOTE ] ECG: Full fetch complete (50864 records). [2025-10-20T18:27:47.671357] [DEBUG] [VALIDATOR ] ECG: Validating primary data. [2025-10-20T18:27:48.043947] [DEBUG] [VALIDATOR ] ECG: All expected trading days are present. [2025-10-20T18:27:48.044054] [DEBUG] [VALIDATOR ] ECG: Primary data validation passed. [2025-10-20T18:27:48.044751] [INFO ] [MAIN ] ECG: Fetched primary data is valid. [2025-10-20T18:27:48.044813] [DEBUG] [LOCAL ] ECG: Saving primary data. [2025-10-20T18:27:48.045958] [DEBUG] [LOCAL ] ECG: Trimmed 50864 records to 32902 (keeping 200 days of data) [2025-10-20T18:27:48.232232] [DEBUG] [LOCAL ] ECG: Primary data saved (32902 records). [2025-10-20T18:27:48.232363] [INFO ] [MAIN ] ECG: Full history saved. [2025-10-20T18:27:48.232440] [INFO ] [MAIN ] Update Progress: 689/2367 symbols completed. ECG took 30.8s. Estimated remaining: 3h 33m 40s. Rate: 7.9 symbols/min [2025-10-20T18:27:48.263114] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-09. [2025-10-20T18:27:48.402713] [INFO ] [MAIN ] ED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:48.452315] [DEBUG] [LOCAL ] ED: Loaded 53411 existing records from primary storage. [2025-10-20T18:27:48.452686] [INFO ] [MAIN ] ED: Loaded 53411 existing records. [2025-10-20T18:27:48.628661] [WARN ] [MAIN ] ED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:48.628945] [INFO ] [MAIN ] ED: Wiping existing primary data before full refresh. [2025-10-20T18:27:48.629081] [INFO ] [LOCAL ] ED: Starting deletion of primary data. [2025-10-20T18:27:48.629759] [DEBUG] [LOCAL ] ED: Deleted primary data file. [2025-10-20T18:27:48.629828] [INFO ] [MAIN ] ED: Starting full primary refresh. [2025-10-20T18:27:48.891041] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T18:27:49.036981] [INFO ] [REMOTE ] ED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:49.051334] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-01. [2025-10-20T18:27:49.851344] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-08. [2025-10-20T18:27:49.980225] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:27:50.423850] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T18:27:50.651354] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-09. [2025-10-20T18:27:50.946740] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:27:51.451384] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-02. [2025-10-20T18:27:51.941167] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T18:27:52.251428] [INFO ] [REMOTE ] ECVT: Fetching primary data for 2025-10. [2025-10-20T18:27:53.051407] [INFO ] [REMOTE ] ECL: Fetching primary data for 2025-10. [2025-10-20T18:27:53.096445] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:27:53.763468] [INFO ] [REMOTE ] ECVT: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T18:27:53.770215] [INFO ] [REMOTE ] ECVT: Full fetch complete (60056 records). [2025-10-20T18:27:53.770321] [DEBUG] [VALIDATOR ] ECVT: Validating primary data. [2025-10-20T18:27:53.771107] [FATAL] [VALIDATOR ] ECVT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:27:53.771194] [FATAL] [MAIN ] ECVT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:27:53.771237] [WARN ] [MAIN ] ECVT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:27:53.771275] [INFO ] [LOCAL ] ECVT: Starting deletion of primary data. [2025-10-20T18:27:53.771333] [DEBUG] [LOCAL ] ECVT: No primary data file found to delete. [2025-10-20T18:27:53.771368] [INFO ] [LOCAL ] ECVT: Starting deletion of derivative data. [2025-10-20T18:27:53.771552] [DEBUG] [LOCAL ] ECVT: Deleted derivative data file. [2025-10-20T18:27:53.772274] [INFO ] [MAIN ] Update Progress: 690/2367 symbols completed. ECVT took 29.7s. Estimated remaining: 3h 33m 27s. Rate: 7.9 symbols/min [2025-10-20T18:27:53.860983] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-03. [2025-10-20T18:27:53.940583] [INFO ] [MAIN ] EDD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:53.940791] [DEBUG] [LOCAL ] EDD: No existing data file found in primary storage. [2025-10-20T18:27:53.940835] [WARN ] [MAIN ] EDD: No existing data found, performing full refresh instead. [2025-10-20T18:27:53.940949] [INFO ] [MAIN ] EDD: Wiping existing primary data before full refresh. [2025-10-20T18:27:53.940991] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-10-20T18:27:53.941046] [DEBUG] [LOCAL ] EDD: No primary data file found to delete. [2025-10-20T18:27:53.941077] [INFO ] [MAIN ] EDD: Starting full primary refresh. [2025-10-20T18:27:54.294395] [INFO ] [REMOTE ] EDD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:54.651351] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-01. [2025-10-20T18:27:54.739688] [INFO ] [REMOTE ] ECL: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T18:27:54.747625] [INFO ] [REMOTE ] ECL: Full fetch complete (68357 records). [2025-10-20T18:27:54.747792] [DEBUG] [VALIDATOR ] ECL: Validating primary data. [2025-10-20T18:27:55.139262] [DEBUG] [VALIDATOR ] ECL: All expected trading days are present. [2025-10-20T18:27:55.139333] [DEBUG] [VALIDATOR ] ECL: Primary data validation passed. [2025-10-20T18:27:55.140175] [INFO ] [MAIN ] ECL: Fetched primary data is valid. [2025-10-20T18:27:55.140236] [DEBUG] [LOCAL ] ECL: Saving primary data. [2025-10-20T18:27:55.141419] [DEBUG] [LOCAL ] ECL: Trimmed 68357 records to 46730 (keeping 200 days of data) [2025-10-20T18:27:55.427880] [DEBUG] [LOCAL ] ECL: Primary data saved (46730 records). [2025-10-20T18:27:55.428057] [INFO ] [MAIN ] ECL: Full history saved. [2025-10-20T18:27:55.428951] [INFO ] [MAIN ] Update Progress: 691/2367 symbols completed. ECL took 37.1s. Estimated remaining: 3h 33m 5s. Rate: 7.9 symbols/min [2025-10-20T18:27:55.459742] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-09. [2025-10-20T18:27:55.602725] [INFO ] [MAIN ] EDF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:55.617335] [DEBUG] [LOCAL ] EDF: Loaded 13279 existing records from primary storage. [2025-10-20T18:27:55.617552] [INFO ] [MAIN ] EDF: Loaded 13279 existing records. [2025-10-20T18:27:55.800151] [WARN ] [MAIN ] EDF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:55.800482] [INFO ] [MAIN ] EDF: Wiping existing primary data before full refresh. [2025-10-20T18:27:55.800716] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-10-20T18:27:55.814802] [DEBUG] [LOCAL ] EDF: Deleted primary data file. [2025-10-20T18:27:55.815690] [INFO ] [MAIN ] EDF: Starting full primary refresh. [2025-10-20T18:27:55.816407] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:27:55.960314] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T18:27:56.220490] [INFO ] [REMOTE ] EDF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:27:56.251327] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-01. [2025-10-20T18:27:56.588403] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:27:57.016015] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:27:57.051328] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-02. [2025-10-20T18:27:57.851425] [INFO ] [REMOTE ] ECO: Fetching primary data for 2025-10. [2025-10-20T18:27:58.178891] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:27:58.651421] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-04. [2025-10-20T18:27:58.925411] [INFO ] [REMOTE ] ECO: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:27:58.928528] [INFO ] [REMOTE ] ECO: Full fetch complete (28402 records). [2025-10-20T18:27:58.928627] [DEBUG] [VALIDATOR ] ECO: Validating primary data. [2025-10-20T18:27:59.310518] [DEBUG] [VALIDATOR ] ECO: All expected trading days are present. [2025-10-20T18:27:59.310598] [DEBUG] [VALIDATOR ] ECO: Primary data validation passed. [2025-10-20T18:27:59.311179] [INFO ] [MAIN ] ECO: Fetched primary data is valid. [2025-10-20T18:27:59.311243] [DEBUG] [LOCAL ] ECO: Saving primary data. [2025-10-20T18:27:59.312005] [DEBUG] [LOCAL ] ECO: Trimmed 28402 records to 18204 (keeping 200 days of data) [2025-10-20T18:27:59.419079] [DEBUG] [LOCAL ] ECO: Primary data saved (18204 records). [2025-10-20T18:27:59.419248] [INFO ] [MAIN ] ECO: Full history saved. [2025-10-20T18:27:59.419341] [INFO ] [MAIN ] Update Progress: 692/2367 symbols completed. ECO took 39.0s. Estimated remaining: 3h 32m 48s. Rate: 7.9 symbols/min [2025-10-20T18:27:59.456473] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-03. [2025-10-20T18:27:59.594093] [INFO ] [MAIN ] EDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:27:59.613054] [DEBUG] [LOCAL ] EDN: Loaded 19338 existing records from primary storage. [2025-10-20T18:27:59.613280] [INFO ] [MAIN ] EDN: Loaded 19338 existing records. [2025-10-20T18:27:59.803289] [WARN ] [MAIN ] EDN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:27:59.803548] [INFO ] [MAIN ] EDN: Wiping existing primary data before full refresh. [2025-10-20T18:27:59.803597] [INFO ] [LOCAL ] EDN: Starting deletion of primary data. [2025-10-20T18:27:59.803801] [DEBUG] [LOCAL ] EDN: Deleted primary data file. [2025-10-20T18:27:59.803854] [INFO ] [MAIN ] EDN: Starting full primary refresh. [2025-10-20T18:28:00.195110] [INFO ] [REMOTE ] EDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:00.251360] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-02. [2025-10-20T18:28:00.531554] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:28:00.561953] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T18:28:01.051396] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-01. [2025-10-20T18:28:01.579522] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:28:01.851409] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-04. [2025-10-20T18:28:02.264157] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T18:28:02.651352] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-03. [2025-10-20T18:28:02.801278] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T18:28:03.451431] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-05. [2025-10-20T18:28:03.886317] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T18:28:04.251376] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-02. [2025-10-20T18:28:05.051358] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-04. [2025-10-20T18:28:05.357241] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:28:05.386547] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:28:05.851409] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-03. [2025-10-20T18:28:06.031286] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:28:06.651369] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-05. [2025-10-20T18:28:06.817120] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T18:28:07.451443] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-04. [2025-10-20T18:28:07.890697] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T18:28:08.251405] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-06. [2025-10-20T18:28:08.472259] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:28:09.051363] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-06. [2025-10-20T18:28:09.851343] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-05. [2025-10-20T18:28:09.897269] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:28:10.124504] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T18:28:10.651418] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-07. [2025-10-20T18:28:10.778411] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:28:11.451390] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-07. [2025-10-20T18:28:12.251398] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-05. [2025-10-20T18:28:12.586494] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T18:28:12.838349] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:28:12.969500] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:28:13.051342] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-08. [2025-10-20T18:28:13.851415] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-06. [2025-10-20T18:28:14.651393] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-06. [2025-10-20T18:28:14.782884] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:28:14.864329] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:28:15.451361] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T18:28:15.451433] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-09. [2025-10-20T18:28:16.251372] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-07. [2025-10-20T18:28:16.912019] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T18:28:16.948410] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:28:17.051391] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-08. [2025-10-20T18:28:17.851419] [INFO ] [REMOTE ] ED: Fetching primary data for 2025-10. [2025-10-20T18:28:18.125558] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:28:18.651392] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-07. [2025-10-20T18:28:19.451364] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-08. [2025-10-20T18:28:19.641780] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:28:19.678554] [INFO ] [REMOTE ] ED: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:28:19.686439] [INFO ] [REMOTE ] ED: Full fetch complete (76405 records). [2025-10-20T18:28:19.686606] [DEBUG] [VALIDATOR ] ED: Validating primary data. [2025-10-20T18:28:20.086457] [DEBUG] [VALIDATOR ] ED: All expected trading days are present. [2025-10-20T18:28:20.086570] [DEBUG] [VALIDATOR ] ED: Primary data validation passed. [2025-10-20T18:28:20.087721] [INFO ] [MAIN ] ED: Fetched primary data is valid. [2025-10-20T18:28:20.087824] [DEBUG] [LOCAL ] ED: Saving primary data. [2025-10-20T18:28:20.097004] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T18:28:20.101372] [DEBUG] [LOCAL ] ED: Trimmed 76405 records to 52649 (keeping 200 days of data) [2025-10-20T18:28:20.295303] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-09. [2025-10-20T18:28:20.454201] [DEBUG] [LOCAL ] ED: Primary data saved (52649 records). [2025-10-20T18:28:20.454356] [INFO ] [MAIN ] ED: Full history saved. [2025-10-20T18:28:20.454436] [INFO ] [MAIN ] Update Progress: 693/2367 symbols completed. ED took 32.2s. Estimated remaining: 3h 33m 13s. Rate: 7.9 symbols/min [2025-10-20T18:28:20.672235] [INFO ] [MAIN ] EDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:20.719245] [DEBUG] [LOCAL ] EDU: Loaded 45507 existing records from primary storage. [2025-10-20T18:28:20.719621] [INFO ] [MAIN ] EDU: Loaded 45507 existing records. [2025-10-20T18:28:20.913333] [WARN ] [MAIN ] EDU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:20.913579] [INFO ] [MAIN ] EDU: Wiping existing primary data before full refresh. [2025-10-20T18:28:20.913630] [INFO ] [LOCAL ] EDU: Starting deletion of primary data. [2025-10-20T18:28:20.914107] [DEBUG] [LOCAL ] EDU: Deleted primary data file. [2025-10-20T18:28:20.914151] [INFO ] [MAIN ] EDU: Starting full primary refresh. [2025-10-20T18:28:21.061815] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-08. [2025-10-20T18:28:21.286576] [INFO ] [REMOTE ] EDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:21.488589] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T18:28:21.851355] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-01. [2025-10-20T18:28:22.077194] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T18:28:22.651386] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-09. [2025-10-20T18:28:23.174282] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:28:23.451398] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-09. [2025-10-20T18:28:23.906233] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:28:24.103459] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:28:24.251373] [INFO ] [REMOTE ] EDD: Fetching primary data for 2025-10. [2025-10-20T18:28:25.051408] [INFO ] [REMOTE ] EDN: Fetching primary data for 2025-10. [2025-10-20T18:28:25.525744] [INFO ] [REMOTE ] EDD: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:28:25.530024] [INFO ] [REMOTE ] EDD: Full fetch complete (38510 records). [2025-10-20T18:28:25.530119] [DEBUG] [VALIDATOR ] EDD: Validating primary data. [2025-10-20T18:28:25.530685] [FATAL] [VALIDATOR ] EDD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:28:25.530767] [FATAL] [MAIN ] EDD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:25.530808] [WARN ] [MAIN ] EDD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:25.530845] [INFO ] [LOCAL ] EDD: Starting deletion of primary data. [2025-10-20T18:28:25.530914] [DEBUG] [LOCAL ] EDD: No primary data file found to delete. [2025-10-20T18:28:25.530969] [INFO ] [LOCAL ] EDD: Starting deletion of derivative data. [2025-10-20T18:28:25.531012] [DEBUG] [LOCAL ] EDD: No derivative data file found to delete. [2025-10-20T18:28:25.531064] [INFO ] [MAIN ] Update Progress: 694/2367 symbols completed. EDD took 31.8s. Estimated remaining: 3h 32m 59s. Rate: 7.9 symbols/min [2025-10-20T18:28:25.699202] [INFO ] [MAIN ] EE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:25.732456] [DEBUG] [LOCAL ] EE: Loaded 33450 existing records from primary storage. [2025-10-20T18:28:25.732764] [INFO ] [MAIN ] EE: Loaded 33450 existing records. [2025-10-20T18:28:25.856578] [INFO ] [REMOTE ] EDF: Fetching primary data for 2025-10. [2025-10-20T18:28:25.918231] [WARN ] [MAIN ] EE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:25.918430] [INFO ] [MAIN ] EE: Wiping existing primary data before full refresh. [2025-10-20T18:28:25.918478] [INFO ] [LOCAL ] EE: Starting deletion of primary data. [2025-10-20T18:28:25.918727] [DEBUG] [LOCAL ] EE: Deleted primary data file. [2025-10-20T18:28:25.918784] [INFO ] [MAIN ] EE: Starting full primary refresh. [2025-10-20T18:28:26.269224] [INFO ] [REMOTE ] EDN: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:28:26.296851] [INFO ] [REMOTE ] EE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:26.297900] [INFO ] [REMOTE ] EDN: Full fetch complete (30037 records). [2025-10-20T18:28:26.298036] [DEBUG] [VALIDATOR ] EDN: Validating primary data. [2025-10-20T18:28:26.656515] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-01. [2025-10-20T18:28:26.702008] [DEBUG] [VALIDATOR ] EDN: All expected trading days are present. [2025-10-20T18:28:26.702085] [DEBUG] [VALIDATOR ] EDN: Primary data validation passed. [2025-10-20T18:28:26.702777] [INFO ] [MAIN ] EDN: Fetched primary data is valid. [2025-10-20T18:28:26.702993] [DEBUG] [LOCAL ] EDN: Saving primary data. [2025-10-20T18:28:26.703797] [DEBUG] [LOCAL ] EDN: Trimmed 30037 records to 19210 (keeping 200 days of data) [2025-10-20T18:28:26.814149] [DEBUG] [LOCAL ] EDN: Primary data saved (19210 records). [2025-10-20T18:28:26.814284] [INFO ] [MAIN ] EDN: Full history saved. [2025-10-20T18:28:26.814361] [INFO ] [MAIN ] Update Progress: 695/2367 symbols completed. EDN took 27.4s. Estimated remaining: 3h 32m 36s. Rate: 7.9 symbols/min [2025-10-20T18:28:26.981266] [INFO ] [MAIN ] EEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:26.981541] [DEBUG] [LOCAL ] EEA: No existing data file found in primary storage. [2025-10-20T18:28:26.981592] [WARN ] [MAIN ] EEA: No existing data found, performing full refresh instead. [2025-10-20T18:28:26.981730] [INFO ] [MAIN ] EEA: Wiping existing primary data before full refresh. [2025-10-20T18:28:26.981775] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-10-20T18:28:26.981817] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-10-20T18:28:26.981848] [INFO ] [MAIN ] EEA: Starting full primary refresh. [2025-10-20T18:28:27.077214] [INFO ] [REMOTE ] EDF: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T18:28:27.098787] [INFO ] [REMOTE ] EDF: Full fetch complete (20306 records). [2025-10-20T18:28:27.109195] [DEBUG] [VALIDATOR ] EDF: Validating primary data. [2025-10-20T18:28:27.132443] [FATAL] [VALIDATOR ] EDF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:28:27.151668] [FATAL] [MAIN ] EDF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:27.168214] [WARN ] [MAIN ] EDF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:27.214775] [INFO ] [LOCAL ] EDF: Starting deletion of primary data. [2025-10-20T18:28:27.252297] [DEBUG] [LOCAL ] EDF: No primary data file found to delete. [2025-10-20T18:28:27.270781] [INFO ] [LOCAL ] EDF: Starting deletion of derivative data. [2025-10-20T18:28:27.273403] [DEBUG] [LOCAL ] EDF: Deleted derivative data file. [2025-10-20T18:28:27.278946] [INFO ] [MAIN ] Update Progress: 696/2367 symbols completed. EDF took 31.8s. Estimated remaining: 3h 32m 11s. Rate: 7.9 symbols/min [2025-10-20T18:28:27.413074] [INFO ] [REMOTE ] EEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:27.458727] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-02. [2025-10-20T18:28:27.564613] [INFO ] [MAIN ] EEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:27.575699] [DEBUG] [LOCAL ] EEX: Loaded 10348 existing records from primary storage. [2025-10-20T18:28:27.575940] [INFO ] [MAIN ] EEX: Loaded 10348 existing records. [2025-10-20T18:28:27.768932] [WARN ] [MAIN ] EEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:27.769225] [INFO ] [MAIN ] EEX: Wiping existing primary data before full refresh. [2025-10-20T18:28:27.769302] [INFO ] [LOCAL ] EEX: Starting deletion of primary data. [2025-10-20T18:28:27.784050] [DEBUG] [LOCAL ] EEX: Deleted primary data file. [2025-10-20T18:28:27.784193] [INFO ] [MAIN ] EEX: Starting full primary refresh. [2025-10-20T18:28:27.817592] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:28:28.170127] [INFO ] [REMOTE ] EEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:28.251429] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-02. [2025-10-20T18:28:29.051382] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-01. [2025-10-20T18:28:29.227345] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T18:28:29.372768] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:28:29.483867] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:28:29.851374] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-03. [2025-10-20T18:28:30.651424] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-02. [2025-10-20T18:28:31.196078] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:28:31.444638] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:28:31.456675] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-03. [2025-10-20T18:28:32.075232] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:28:32.251384] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-04. [2025-10-20T18:28:33.051405] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-04. [2025-10-20T18:28:33.621176] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T18:28:33.851347] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-03. [2025-10-20T18:28:33.901968] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:28:34.651385] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-05. [2025-10-20T18:28:34.918449] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T18:28:35.451367] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-05. [2025-10-20T18:28:35.886216] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T18:28:35.943476] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:28:36.251407] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-06. [2025-10-20T18:28:36.553014] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-06 (0.30 seconds). [2025-10-20T18:28:37.051344] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-01. [2025-10-20T18:28:37.851436] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-07. [2025-10-20T18:28:38.011583] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T18:28:38.181248] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-10-20T18:28:38.651377] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-08. [2025-10-20T18:28:39.133463] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:28:39.451381] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-04. [2025-10-20T18:28:40.251397] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-06. [2025-10-20T18:28:40.959810] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:28:41.051382] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-05. [2025-10-20T18:28:41.845091] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:28:41.853592] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-09. [2025-10-20T18:28:42.228857] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T18:28:42.321509] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:28:42.651375] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-07. [2025-10-20T18:28:43.451432] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-02. [2025-10-20T18:28:44.243702] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T18:28:44.251318] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-03. [2025-10-20T18:28:44.464940] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:28:45.051409] [INFO ] [REMOTE ] EEA: Fetching primary data for 2025-10. [2025-10-20T18:28:45.316711] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T18:28:45.624229] [INFO ] [REMOTE ] EEA: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T18:28:45.625013] [INFO ] [REMOTE ] EEA: Full fetch complete (3748 records). [2025-10-20T18:28:45.625079] [DEBUG] [VALIDATOR ] EEA: Validating primary data. [2025-10-20T18:28:45.806907] [WARN ] [VALIDATOR ] EEA: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T18:28:45.807149] [FATAL] [MAIN ] EEA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:28:45.807247] [WARN ] [MAIN ] EEA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:28:45.807319] [INFO ] [LOCAL ] EEA: Starting deletion of primary data. [2025-10-20T18:28:45.807385] [DEBUG] [LOCAL ] EEA: No primary data file found to delete. [2025-10-20T18:28:45.807422] [INFO ] [LOCAL ] EEA: Starting deletion of derivative data. [2025-10-20T18:28:45.807463] [DEBUG] [LOCAL ] EEA: No derivative data file found to delete. [2025-10-20T18:28:45.807521] [INFO ] [MAIN ] Update Progress: 697/2367 symbols completed. EEA took 19.0s. Estimated remaining: 3h 32m 30s. Rate: 7.9 symbols/min [2025-10-20T18:28:45.860996] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-08. [2025-10-20T18:28:45.990780] [INFO ] [MAIN ] EFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:46.035013] [DEBUG] [LOCAL ] EFC: Loaded 47042 existing records from primary storage. [2025-10-20T18:28:46.035332] [INFO ] [MAIN ] EFC: Loaded 47042 existing records. [2025-10-20T18:28:46.219137] [WARN ] [MAIN ] EFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:46.219365] [INFO ] [MAIN ] EFC: Wiping existing primary data before full refresh. [2025-10-20T18:28:46.219412] [INFO ] [LOCAL ] EFC: Starting deletion of primary data. [2025-10-20T18:28:46.219785] [DEBUG] [LOCAL ] EFC: Deleted primary data file. [2025-10-20T18:28:46.219843] [INFO ] [MAIN ] EFC: Starting full primary refresh. [2025-10-20T18:28:46.574111] [INFO ] [REMOTE ] EFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:46.651384] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-01. [2025-10-20T18:28:47.397428] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:28:47.451336] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-09. [2025-10-20T18:28:48.260520] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-04. [2025-10-20T18:28:48.263423] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:28:48.670231] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T18:28:48.975101] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:28:49.051348] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-05. [2025-10-20T18:28:49.855399] [INFO ] [REMOTE ] EDU: Fetching primary data for 2025-10. [2025-10-20T18:28:49.858284] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:28:50.651437] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-06. [2025-10-20T18:28:51.440724] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:28:51.451329] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-07. [2025-10-20T18:28:51.494275] [INFO ] [REMOTE ] EDU: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:28:51.501276] [INFO ] [REMOTE ] EDU: Full fetch complete (67148 records). [2025-10-20T18:28:51.501391] [DEBUG] [VALIDATOR ] EDU: Validating primary data. [2025-10-20T18:28:51.882229] [DEBUG] [VALIDATOR ] EDU: All expected trading days are present. [2025-10-20T18:28:51.882312] [DEBUG] [VALIDATOR ] EDU: Primary data validation passed. [2025-10-20T18:28:51.883112] [INFO ] [MAIN ] EDU: Fetched primary data is valid. [2025-10-20T18:28:51.883174] [DEBUG] [LOCAL ] EDU: Saving primary data. [2025-10-20T18:28:51.884476] [DEBUG] [LOCAL ] EDU: Trimmed 67148 records to 44811 (keeping 200 days of data) [2025-10-20T18:28:52.088857] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:28:52.149723] [DEBUG] [LOCAL ] EDU: Primary data saved (44811 records). [2025-10-20T18:28:52.149853] [INFO ] [MAIN ] EDU: Full history saved. [2025-10-20T18:28:52.149952] [INFO ] [MAIN ] Update Progress: 698/2367 symbols completed. EDU took 31.7s. Estimated remaining: 3h 32m 19s. Rate: 7.9 symbols/min [2025-10-20T18:28:52.256484] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-06. [2025-10-20T18:28:52.320269] [INFO ] [MAIN ] EFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:28:52.336332] [DEBUG] [LOCAL ] EFR: Loaded 15309 existing records from primary storage. [2025-10-20T18:28:52.336543] [INFO ] [MAIN ] EFR: Loaded 15309 existing records. [2025-10-20T18:28:52.509564] [WARN ] [MAIN ] EFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:28:52.509799] [INFO ] [MAIN ] EFR: Wiping existing primary data before full refresh. [2025-10-20T18:28:52.509846] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-10-20T18:28:52.510128] [DEBUG] [LOCAL ] EFR: Deleted primary data file. [2025-10-20T18:28:52.510183] [INFO ] [MAIN ] EFR: Starting full primary refresh. [2025-10-20T18:28:52.864480] [INFO ] [REMOTE ] EFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:28:53.051361] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-01. [2025-10-20T18:28:53.683580] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:28:53.698491] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T18:28:53.851405] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-07. [2025-10-20T18:28:54.651435] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-08. [2025-10-20T18:28:55.244104] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:28:55.244795] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T18:28:55.451397] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-02. [2025-10-20T18:28:56.251427] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-08. [2025-10-20T18:28:57.053033] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-02. [2025-10-20T18:28:57.055404] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T18:28:57.238071] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:28:57.804920] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:28:57.851370] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-03. [2025-10-20T18:28:58.651376] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-09. [2025-10-20T18:28:59.451338] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-03. [2025-10-20T18:28:59.826588] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T18:28:59.852895] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:29:00.251362] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-09. [2025-10-20T18:29:00.277324] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T18:29:01.051354] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-04. [2025-10-20T18:29:01.129535] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:29:01.851354] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-04. [2025-10-20T18:29:02.651364] [INFO ] [REMOTE ] EE: Fetching primary data for 2025-10. [2025-10-20T18:29:02.718697] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:29:02.849492] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:29:03.451394] [INFO ] [REMOTE ] EEX: Fetching primary data for 2025-10. [2025-10-20T18:29:03.945565] [INFO ] [REMOTE ] EE: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T18:29:03.951741] [INFO ] [REMOTE ] EE: Full fetch complete (44238 records). [2025-10-20T18:29:03.951877] [DEBUG] [VALIDATOR ] EE: Validating primary data. [2025-10-20T18:29:04.253967] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-05. [2025-10-20T18:29:04.253900] [INFO ] [REMOTE ] EEX: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T18:29:04.268571] [INFO ] [REMOTE ] EEX: Full fetch complete (18458 records). [2025-10-20T18:29:04.269499] [DEBUG] [VALIDATOR ] EEX: Validating primary data. [2025-10-20T18:29:04.491009] [DEBUG] [VALIDATOR ] EE: All expected trading days are present. [2025-10-20T18:29:04.491440] [DEBUG] [VALIDATOR ] EE: Primary data validation passed. [2025-10-20T18:29:04.502689] [INFO ] [MAIN ] EE: Fetched primary data is valid. [2025-10-20T18:29:04.507908] [DEBUG] [LOCAL ] EE: Saving primary data. [2025-10-20T18:29:04.574583] [DEBUG] [LOCAL ] EE: Trimmed 44238 records to 32694 (keeping 200 days of data) [2025-10-20T18:29:04.929747] [DEBUG] [VALIDATOR ] EEX: All expected trading days are present. [2025-10-20T18:29:04.930171] [DEBUG] [VALIDATOR ] EEX: Primary data validation passed. [2025-10-20T18:29:04.931685] [INFO ] [MAIN ] EEX: Fetched primary data is valid. [2025-10-20T18:29:04.931765] [DEBUG] [LOCAL ] EEX: Saving primary data. [2025-10-20T18:29:04.938689] [DEBUG] [LOCAL ] EEX: Trimmed 18458 records to 10029 (keeping 200 days of data) [2025-10-20T18:29:05.053802] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-05. [2025-10-20T18:29:05.074202] [DEBUG] [LOCAL ] EEX: Primary data saved (10029 records). [2025-10-20T18:29:05.077271] [INFO ] [MAIN ] EEX: Full history saved. [2025-10-20T18:29:05.077697] [INFO ] [MAIN ] Update Progress: 699/2367 symbols completed. EEX took 37.8s. Estimated remaining: 3h 32m 24s. Rate: 7.9 symbols/min [2025-10-20T18:29:05.116794] [DEBUG] [LOCAL ] EE: Primary data saved (32694 records). [2025-10-20T18:29:05.122119] [INFO ] [MAIN ] EE: Full history saved. [2025-10-20T18:29:05.136492] [INFO ] [MAIN ] Update Progress: 700/2367 symbols completed. EE took 39.6s. Estimated remaining: 3h 31m 59s. Rate: 7.9 symbols/min [2025-10-20T18:29:05.322302] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T18:29:05.466262] [INFO ] [MAIN ] EFT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:05.486343] [INFO ] [MAIN ] EFX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:05.502457] [DEBUG] [LOCAL ] EFT: Loaded 13215 existing records from primary storage. [2025-10-20T18:29:05.502795] [INFO ] [MAIN ] EFT: Loaded 13215 existing records. [2025-10-20T18:29:05.710103] [WARN ] [MAIN ] EFT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:05.723804] [INFO ] [MAIN ] EFT: Wiping existing primary data before full refresh. [2025-10-20T18:29:05.723891] [INFO ] [LOCAL ] EFT: Starting deletion of primary data. [2025-10-20T18:29:05.724539] [DEBUG] [LOCAL ] EFX: Loaded 45309 existing records from primary storage. [2025-10-20T18:29:05.724864] [DEBUG] [LOCAL ] EFT: Deleted primary data file. [2025-10-20T18:29:05.724841] [INFO ] [MAIN ] EFX: Loaded 45309 existing records. [2025-10-20T18:29:05.730071] [INFO ] [MAIN ] EFT: Starting full primary refresh. [2025-10-20T18:29:05.856495] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-06. [2025-10-20T18:29:06.108268] [WARN ] [MAIN ] EFX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:06.192619] [INFO ] [MAIN ] EFX: Wiping existing primary data before full refresh. [2025-10-20T18:29:06.195094] [INFO ] [LOCAL ] EFX: Starting deletion of primary data. [2025-10-20T18:29:06.195445] [DEBUG] [LOCAL ] EFX: Deleted primary data file. [2025-10-20T18:29:06.206000] [INFO ] [MAIN ] EFX: Starting full primary refresh. [2025-10-20T18:29:06.363004] [INFO ] [REMOTE ] EFT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:06.642632] [INFO ] [REMOTE ] EFX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:06.651330] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-01. [2025-10-20T18:29:06.763837] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T18:29:06.903396] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:29:07.451438] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-06. [2025-10-20T18:29:08.251403] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-01. [2025-10-20T18:29:08.323332] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:29:08.728142] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T18:29:09.031480] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T18:29:09.051369] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-02. [2025-10-20T18:29:09.800524] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:29:09.851383] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-07. [2025-10-20T18:29:10.651386] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-02. [2025-10-20T18:29:11.451402] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-03. [2025-10-20T18:29:11.662029] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:29:12.088820] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:29:12.201395] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:29:12.251341] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-03. [2025-10-20T18:29:13.051432] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-04. [2025-10-20T18:29:13.852755] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-08. [2025-10-20T18:29:13.858092] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:29:14.083594] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:29:14.651413] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-07. [2025-10-20T18:29:15.451377] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-04. [2025-10-20T18:29:15.616279] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:29:15.663462] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:29:16.251374] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-09. [2025-10-20T18:29:17.051371] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-05. [2025-10-20T18:29:17.613355] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:29:17.642161] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:29:17.851343] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-08. [2025-10-20T18:29:18.208897] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T18:29:18.651421] [INFO ] [REMOTE ] EFC: Fetching primary data for 2025-10. [2025-10-20T18:29:18.744180] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:29:19.451359] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-06. [2025-10-20T18:29:20.251395] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-09. [2025-10-20T18:29:20.424858] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T18:29:20.520412] [INFO ] [REMOTE ] EFC: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:29:20.527453] [INFO ] [REMOTE ] EFC: Full fetch complete (66710 records). [2025-10-20T18:29:20.527611] [DEBUG] [VALIDATOR ] EFC: Validating primary data. [2025-10-20T18:29:20.917578] [DEBUG] [VALIDATOR ] EFC: All expected trading days are present. [2025-10-20T18:29:20.917689] [DEBUG] [VALIDATOR ] EFC: Primary data validation passed. [2025-10-20T18:29:20.918471] [INFO ] [MAIN ] EFC: Fetched primary data is valid. [2025-10-20T18:29:20.918513] [DEBUG] [LOCAL ] EFC: Saving primary data. [2025-10-20T18:29:20.919890] [DEBUG] [LOCAL ] EFC: Trimmed 66710 records to 46426 (keeping 200 days of data) [2025-10-20T18:29:21.097409] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-07. [2025-10-20T18:29:21.207061] [DEBUG] [LOCAL ] EFC: Primary data saved (46426 records). [2025-10-20T18:29:21.207198] [INFO ] [MAIN ] EFC: Full history saved. [2025-10-20T18:29:21.207274] [INFO ] [MAIN ] Update Progress: 701/2367 symbols completed. EFC took 35.4s. Estimated remaining: 3h 32m 11s. Rate: 7.9 symbols/min [2025-10-20T18:29:21.319772] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:29:21.395567] [INFO ] [MAIN ] EFXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:21.425498] [DEBUG] [LOCAL ] EFXT: Loaded 30200 existing records from primary storage. [2025-10-20T18:29:21.425836] [INFO ] [MAIN ] EFXT: Loaded 30200 existing records. [2025-10-20T18:29:21.603043] [WARN ] [MAIN ] EFXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:21.603246] [INFO ] [MAIN ] EFXT: Wiping existing primary data before full refresh. [2025-10-20T18:29:21.603292] [INFO ] [LOCAL ] EFXT: Starting deletion of primary data. [2025-10-20T18:29:21.603497] [DEBUG] [LOCAL ] EFXT: Deleted primary data file. [2025-10-20T18:29:21.603552] [INFO ] [MAIN ] EFXT: Starting full primary refresh. [2025-10-20T18:29:21.852376] [INFO ] [REMOTE ] EFR: Fetching primary data for 2025-10. [2025-10-20T18:29:21.947122] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:29:22.022793] [INFO ] [REMOTE ] EFXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:22.651402] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-05. [2025-10-20T18:29:22.828888] [INFO ] [REMOTE ] EFR: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:29:22.833277] [INFO ] [REMOTE ] EFR: Full fetch complete (19397 records). [2025-10-20T18:29:22.833444] [DEBUG] [VALIDATOR ] EFR: Validating primary data. [2025-10-20T18:29:22.834099] [FATAL] [VALIDATOR ] EFR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:29:22.834201] [FATAL] [MAIN ] EFR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:29:22.834256] [WARN ] [MAIN ] EFR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:29:22.834307] [INFO ] [LOCAL ] EFR: Starting deletion of primary data. [2025-10-20T18:29:22.834379] [DEBUG] [LOCAL ] EFR: No primary data file found to delete. [2025-10-20T18:29:22.834424] [INFO ] [LOCAL ] EFR: Starting deletion of derivative data. [2025-10-20T18:29:22.834684] [DEBUG] [LOCAL ] EFR: Deleted derivative data file. [2025-10-20T18:29:22.834793] [INFO ] [MAIN ] Update Progress: 702/2367 symbols completed. EFR took 30.7s. Estimated remaining: 3h 31m 49s. Rate: 7.9 symbols/min [2025-10-20T18:29:23.013121] [INFO ] [MAIN ] EG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:23.042154] [DEBUG] [LOCAL ] EG: Loaded 28182 existing records from primary storage. [2025-10-20T18:29:23.042460] [INFO ] [MAIN ] EG: Loaded 28182 existing records. [2025-10-20T18:29:23.222695] [WARN ] [MAIN ] EG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:23.222982] [INFO ] [MAIN ] EG: Wiping existing primary data before full refresh. [2025-10-20T18:29:23.223036] [INFO ] [LOCAL ] EG: Starting deletion of primary data. [2025-10-20T18:29:23.223218] [DEBUG] [LOCAL ] EG: Deleted primary data file. [2025-10-20T18:29:23.223273] [INFO ] [MAIN ] EG: Starting full primary refresh. [2025-10-20T18:29:23.460303] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-01. [2025-10-20T18:29:23.579984] [INFO ] [REMOTE ] EG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:24.251412] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-01. [2025-10-20T18:29:24.321515] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:29:25.051377] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-08. [2025-10-20T18:29:25.075523] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:29:25.371464] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T18:29:25.835676] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:29:25.851347] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-09. [2025-10-20T18:29:26.651375] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-02. [2025-10-20T18:29:26.684282] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:29:27.451380] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-02. [2025-10-20T18:29:27.850730] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:29:28.251377] [INFO ] [REMOTE ] EFT: Fetching primary data for 2025-10. [2025-10-20T18:29:28.786614] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:29:29.051401] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-06. [2025-10-20T18:29:29.296785] [INFO ] [REMOTE ] EFT: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:29:29.299884] [INFO ] [REMOTE ] EFT: Full fetch complete (17740 records). [2025-10-20T18:29:29.300014] [DEBUG] [VALIDATOR ] EFT: Validating primary data. [2025-10-20T18:29:29.764907] [DEBUG] [VALIDATOR ] EFT: All expected trading days are present. [2025-10-20T18:29:29.764994] [DEBUG] [VALIDATOR ] EFT: Primary data validation passed. [2025-10-20T18:29:29.765432] [INFO ] [MAIN ] EFT: Fetched primary data is valid. [2025-10-20T18:29:29.765496] [DEBUG] [LOCAL ] EFT: Saving primary data. [2025-10-20T18:29:29.766212] [DEBUG] [LOCAL ] EFT: Trimmed 17740 records to 12980 (keeping 200 days of data) [2025-10-20T18:29:29.851438] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-03. [2025-10-20T18:29:29.868000] [DEBUG] [LOCAL ] EFT: Primary data saved (12980 records). [2025-10-20T18:29:29.868154] [INFO ] [MAIN ] EFT: Full history saved. [2025-10-20T18:29:29.868238] [INFO ] [MAIN ] Update Progress: 703/2367 symbols completed. EFT took 24.8s. Estimated remaining: 3h 31m 40s. Rate: 7.9 symbols/min [2025-10-20T18:29:30.038147] [INFO ] [MAIN ] EGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:30.089866] [DEBUG] [LOCAL ] EGO: Loaded 52505 existing records from primary storage. [2025-10-20T18:29:30.090206] [INFO ] [MAIN ] EGO: Loaded 52505 existing records. [2025-10-20T18:29:30.274868] [WARN ] [MAIN ] EGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:30.275161] [INFO ] [MAIN ] EGO: Wiping existing primary data before full refresh. [2025-10-20T18:29:30.275221] [INFO ] [LOCAL ] EGO: Starting deletion of primary data. [2025-10-20T18:29:30.275420] [DEBUG] [LOCAL ] EGO: Deleted primary data file. [2025-10-20T18:29:30.275475] [INFO ] [MAIN ] EGO: Starting full primary refresh. [2025-10-20T18:29:30.658992] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-03. [2025-10-20T18:29:30.663752] [INFO ] [REMOTE ] EGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:30.690984] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:29:31.315063] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:29:31.451346] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-01. [2025-10-20T18:29:31.901841] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T18:29:32.251368] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-04. [2025-10-20T18:29:33.051381] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-07. [2025-10-20T18:29:33.250076] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T18:29:33.723501] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:29:33.851380] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-05. [2025-10-20T18:29:34.651354] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-04. [2025-10-20T18:29:34.835325] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:29:35.451401] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-08. [2025-10-20T18:29:35.556298] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:29:36.161104] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:29:36.251439] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-02. [2025-10-20T18:29:37.051363] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-06. [2025-10-20T18:29:37.851336] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-05. [2025-10-20T18:29:37.943855] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T18:29:38.021155] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-08 (2.57 seconds). [2025-10-20T18:29:38.091401] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:29:38.651352] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-03. [2025-10-20T18:29:39.018872] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:29:39.451423] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-06. [2025-10-20T18:29:39.772866] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:29:40.251363] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-04. [2025-10-20T18:29:40.597263] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:29:41.051385] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-07. [2025-10-20T18:29:41.851353] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-09. [2025-10-20T18:29:42.014360] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:29:42.393508] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:29:42.651358] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-05. [2025-10-20T18:29:43.451350] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-08. [2025-10-20T18:29:43.518332] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:29:44.251367] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-07. [2025-10-20T18:29:44.510287] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T18:29:44.535236] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:29:45.051382] [INFO ] [REMOTE ] EFX: Fetching primary data for 2025-10. [2025-10-20T18:29:45.324765] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:29:45.851377] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-09. [2025-10-20T18:29:46.651418] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-06. [2025-10-20T18:29:46.918223] [INFO ] [REMOTE ] EFX: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:29:46.925596] [INFO ] [REMOTE ] EFX: Full fetch complete (64974 records). [2025-10-20T18:29:46.925747] [DEBUG] [VALIDATOR ] EFX: Validating primary data. [2025-10-20T18:29:47.131852] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:29:47.357213] [DEBUG] [VALIDATOR ] EFX: All expected trading days are present. [2025-10-20T18:29:47.357292] [DEBUG] [VALIDATOR ] EFX: Primary data validation passed. [2025-10-20T18:29:47.358076] [INFO ] [MAIN ] EFX: Fetched primary data is valid. [2025-10-20T18:29:47.358138] [DEBUG] [LOCAL ] EFX: Saving primary data. [2025-10-20T18:29:47.359393] [DEBUG] [LOCAL ] EFX: Trimmed 64974 records to 44781 (keeping 200 days of data) [2025-10-20T18:29:47.451737] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-08. [2025-10-20T18:29:47.638496] [DEBUG] [LOCAL ] EFX: Primary data saved (44781 records). [2025-10-20T18:29:47.638631] [INFO ] [MAIN ] EFX: Full history saved. [2025-10-20T18:29:47.639244] [INFO ] [MAIN ] Update Progress: 704/2367 symbols completed. EFX took 42.5s. Estimated remaining: 3h 31m 56s. Rate: 7.8 symbols/min [2025-10-20T18:29:47.822437] [INFO ] [MAIN ] EGP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:47.862355] [DEBUG] [LOCAL ] EGP: Loaded 30899 existing records from primary storage. [2025-10-20T18:29:47.862686] [INFO ] [MAIN ] EGP: Loaded 30899 existing records. [2025-10-20T18:29:48.049677] [WARN ] [MAIN ] EGP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:48.049921] [INFO ] [MAIN ] EGP: Wiping existing primary data before full refresh. [2025-10-20T18:29:48.049988] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-10-20T18:29:48.050326] [DEBUG] [LOCAL ] EGP: Deleted primary data file. [2025-10-20T18:29:48.050401] [INFO ] [MAIN ] EGP: Starting full primary refresh. [2025-10-20T18:29:48.262690] [INFO ] [REMOTE ] EG: Fetching primary data for 2025-10. [2025-10-20T18:29:48.415398] [INFO ] [REMOTE ] EGP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:48.442966] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T18:29:48.879764] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:29:49.051372] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-07. [2025-10-20T18:29:49.693026] [INFO ] [REMOTE ] EG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T18:29:49.697031] [INFO ] [REMOTE ] EG: Full fetch complete (43121 records). [2025-10-20T18:29:49.697149] [DEBUG] [VALIDATOR ] EG: Validating primary data. [2025-10-20T18:29:49.871245] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-09. [2025-10-20T18:29:50.069447] [DEBUG] [VALIDATOR ] EG: All expected trading days are present. [2025-10-20T18:29:50.069520] [DEBUG] [VALIDATOR ] EG: Primary data validation passed. [2025-10-20T18:29:50.070138] [INFO ] [MAIN ] EG: Fetched primary data is valid. [2025-10-20T18:29:50.070198] [DEBUG] [LOCAL ] EG: Saving primary data. [2025-10-20T18:29:50.071036] [DEBUG] [LOCAL ] EG: Trimmed 43121 records to 27781 (keeping 200 days of data) [2025-10-20T18:29:50.238719] [DEBUG] [LOCAL ] EG: Primary data saved (27781 records). [2025-10-20T18:29:50.238836] [INFO ] [MAIN ] EG: Full history saved. [2025-10-20T18:29:50.238913] [INFO ] [MAIN ] Update Progress: 705/2367 symbols completed. EG took 27.4s. Estimated remaining: 3h 31m 37s. Rate: 7.9 symbols/min [2025-10-20T18:29:50.410186] [INFO ] [MAIN ] EGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:50.442010] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:29:50.484295] [DEBUG] [LOCAL ] EGY: Loaded 42160 existing records from primary storage. [2025-10-20T18:29:50.484566] [INFO ] [MAIN ] EGY: Loaded 42160 existing records. [2025-10-20T18:29:50.656788] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-08. [2025-10-20T18:29:50.661570] [WARN ] [MAIN ] EGY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:50.662669] [INFO ] [MAIN ] EGY: Wiping existing primary data before full refresh. [2025-10-20T18:29:50.662732] [INFO ] [LOCAL ] EGY: Starting deletion of primary data. [2025-10-20T18:29:50.662902] [DEBUG] [LOCAL ] EGY: Deleted primary data file. [2025-10-20T18:29:50.662943] [INFO ] [MAIN ] EGY: Starting full primary refresh. [2025-10-20T18:29:51.037486] [INFO ] [REMOTE ] EGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:51.054484] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:29:51.451384] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-01. [2025-10-20T18:29:52.251349] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-01. [2025-10-20T18:29:52.445963] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:29:53.051403] [INFO ] [REMOTE ] EFXT: Fetching primary data for 2025-10. [2025-10-20T18:29:53.106904] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:29:53.221903] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T18:29:53.851372] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-02. [2025-10-20T18:29:54.299795] [INFO ] [REMOTE ] EFXT: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T18:29:54.307697] [INFO ] [REMOTE ] EFXT: Full fetch complete (44320 records). [2025-10-20T18:29:54.307846] [DEBUG] [VALIDATOR ] EFXT: Validating primary data. [2025-10-20T18:29:54.661860] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-02. [2025-10-20T18:29:54.702086] [DEBUG] [VALIDATOR ] EFXT: All expected trading days are present. [2025-10-20T18:29:54.702159] [DEBUG] [VALIDATOR ] EFXT: Primary data validation passed. [2025-10-20T18:29:54.702794] [INFO ] [MAIN ] EFXT: Fetched primary data is valid. [2025-10-20T18:29:54.702854] [DEBUG] [LOCAL ] EFXT: Saving primary data. [2025-10-20T18:29:54.703771] [DEBUG] [LOCAL ] EFXT: Trimmed 44320 records to 29637 (keeping 200 days of data) [2025-10-20T18:29:54.876461] [DEBUG] [LOCAL ] EFXT: Primary data saved (29637 records). [2025-10-20T18:29:54.876600] [INFO ] [MAIN ] EFXT: Full history saved. [2025-10-20T18:29:54.876710] [INFO ] [MAIN ] Update Progress: 706/2367 symbols completed. EFXT took 33.7s. Estimated remaining: 3h 31m 22s. Rate: 7.9 symbols/min [2025-10-20T18:29:55.056245] [INFO ] [MAIN ] EHAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:29:55.084261] [DEBUG] [LOCAL ] EHAB: Loaded 29656 existing records from primary storage. [2025-10-20T18:29:55.084529] [INFO ] [MAIN ] EHAB: Loaded 29656 existing records. [2025-10-20T18:29:55.235708] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:29:55.302466] [WARN ] [MAIN ] EHAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:29:55.302948] [INFO ] [MAIN ] EHAB: Wiping existing primary data before full refresh. [2025-10-20T18:29:55.303002] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-10-20T18:29:55.303183] [DEBUG] [LOCAL ] EHAB: Deleted primary data file. [2025-10-20T18:29:55.303239] [INFO ] [MAIN ] EHAB: Starting full primary refresh. [2025-10-20T18:29:55.455062] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-09. [2025-10-20T18:29:55.658804] [INFO ] [REMOTE ] EHAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:29:56.070901] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T18:29:56.251382] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-03. [2025-10-20T18:29:57.051388] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-03. [2025-10-20T18:29:57.274565] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:29:57.815749] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T18:29:57.851368] [INFO ] [REMOTE ] EGO: Fetching primary data for 2025-10. [2025-10-20T18:29:58.173695] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:29:58.651362] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-04. [2025-10-20T18:29:59.451408] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-04. [2025-10-20T18:29:59.731810] [INFO ] [REMOTE ] EGO: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T18:29:59.739541] [INFO ] [REMOTE ] EGO: Full fetch complete (74449 records). [2025-10-20T18:29:59.739695] [DEBUG] [VALIDATOR ] EGO: Validating primary data. [2025-10-20T18:30:00.130843] [DEBUG] [VALIDATOR ] EGO: All expected trading days are present. [2025-10-20T18:30:00.131022] [DEBUG] [VALIDATOR ] EGO: Primary data validation passed. [2025-10-20T18:30:00.131934] [INFO ] [MAIN ] EGO: Fetched primary data is valid. [2025-10-20T18:30:00.132014] [DEBUG] [LOCAL ] EGO: Saving primary data. [2025-10-20T18:30:00.133509] [DEBUG] [LOCAL ] EGO: Trimmed 74449 records to 51745 (keeping 200 days of data) [2025-10-20T18:30:00.252837] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-01. [2025-10-20T18:30:00.288261] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:30:00.494672] [DEBUG] [LOCAL ] EGO: Primary data saved (51745 records). [2025-10-20T18:30:00.494869] [INFO ] [MAIN ] EGO: Full history saved. [2025-10-20T18:30:00.495018] [INFO ] [MAIN ] Update Progress: 707/2367 symbols completed. EGO took 30.6s. Estimated remaining: 3h 31m 10s. Rate: 7.9 symbols/min [2025-10-20T18:30:00.745359] [INFO ] [MAIN ] EHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:00.793547] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T18:30:00.844672] [DEBUG] [LOCAL ] EHC: Loaded 41587 existing records from primary storage. [2025-10-20T18:30:00.845147] [INFO ] [MAIN ] EHC: Loaded 41587 existing records. [2025-10-20T18:30:01.065693] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-05. [2025-10-20T18:30:01.093596] [WARN ] [MAIN ] EHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:01.094957] [INFO ] [MAIN ] EHC: Wiping existing primary data before full refresh. [2025-10-20T18:30:01.095030] [INFO ] [LOCAL ] EHC: Starting deletion of primary data. [2025-10-20T18:30:01.095739] [DEBUG] [LOCAL ] EHC: Deleted primary data file. [2025-10-20T18:30:01.095843] [INFO ] [MAIN ] EHC: Starting full primary refresh. [2025-10-20T18:30:01.592115] [INFO ] [REMOTE ] EHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:01.615879] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:30:01.851434] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-05. [2025-10-20T18:30:02.373574] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:30:02.651443] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-01. [2025-10-20T18:30:03.423308] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T18:30:03.451343] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-06. [2025-10-20T18:30:04.251422] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-06. [2025-10-20T18:30:04.311334] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:30:05.040545] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T18:30:05.051333] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-07. [2025-10-20T18:30:05.401221] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:30:05.851394] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-02. [2025-10-20T18:30:06.615839] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T18:30:06.651350] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-08. [2025-10-20T18:30:07.459325] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-02. [2025-10-20T18:30:07.465950] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T18:30:08.214175] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T18:30:08.251364] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-09. [2025-10-20T18:30:08.294766] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:30:09.051432] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-07. [2025-10-20T18:30:09.837774] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:30:09.851340] [INFO ] [REMOTE ] EGY: Fetching primary data for 2025-10. [2025-10-20T18:30:10.510502] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:30:10.651417] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-08. [2025-10-20T18:30:11.451615] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-03. [2025-10-20T18:30:11.481501] [INFO ] [REMOTE ] EGY: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T18:30:11.489068] [INFO ] [REMOTE ] EGY: Full fetch complete (60896 records). [2025-10-20T18:30:11.489329] [DEBUG] [VALIDATOR ] EGY: Validating primary data. [2025-10-20T18:30:11.930903] [DEBUG] [VALIDATOR ] EGY: All expected trading days are present. [2025-10-20T18:30:11.931467] [DEBUG] [VALIDATOR ] EGY: Primary data validation passed. [2025-10-20T18:30:11.933120] [INFO ] [MAIN ] EGY: Fetched primary data is valid. [2025-10-20T18:30:11.935959] [DEBUG] [LOCAL ] EGY: Saving primary data. [2025-10-20T18:30:11.935842] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T18:30:11.944777] [DEBUG] [LOCAL ] EGY: Trimmed 60896 records to 41464 (keeping 200 days of data) [2025-10-20T18:30:12.179344] [DEBUG] [LOCAL ] EGY: Primary data saved (41464 records). [2025-10-20T18:30:12.179480] [INFO ] [MAIN ] EGY: Full history saved. [2025-10-20T18:30:12.179557] [INFO ] [MAIN ] Update Progress: 708/2367 symbols completed. EGY took 21.9s. Estimated remaining: 3h 31m 12s. Rate: 7.9 symbols/min [2025-10-20T18:30:12.256543] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-03. [2025-10-20T18:30:12.352937] [INFO ] [MAIN ] EHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:12.364564] [DEBUG] [LOCAL ] EHI: Loaded 10782 existing records from primary storage. [2025-10-20T18:30:12.364835] [INFO ] [MAIN ] EHI: Loaded 10782 existing records. [2025-10-20T18:30:12.551343] [WARN ] [MAIN ] EHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:12.551581] [INFO ] [MAIN ] EHI: Wiping existing primary data before full refresh. [2025-10-20T18:30:12.551629] [INFO ] [LOCAL ] EHI: Starting deletion of primary data. [2025-10-20T18:30:12.551888] [DEBUG] [LOCAL ] EHI: Deleted primary data file. [2025-10-20T18:30:12.551960] [INFO ] [MAIN ] EHI: Starting full primary refresh. [2025-10-20T18:30:12.929371] [INFO ] [REMOTE ] EHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:12.959720] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T18:30:13.051389] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-09. [2025-10-20T18:30:13.653615] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:30:13.851413] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-04. [2025-10-20T18:30:14.182088] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:30:14.651445] [INFO ] [REMOTE ] EGP: Fetching primary data for 2025-10. [2025-10-20T18:30:14.837769] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:30:15.451416] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-04. [2025-10-20T18:30:15.853296] [INFO ] [REMOTE ] EGP: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:30:15.857813] [INFO ] [REMOTE ] EGP: Full fetch complete (45389 records). [2025-10-20T18:30:15.857967] [DEBUG] [VALIDATOR ] EGP: Validating primary data. [2025-10-20T18:30:15.858667] [FATAL] [VALIDATOR ] EGP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:30:15.858774] [FATAL] [MAIN ] EGP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:15.858845] [WARN ] [MAIN ] EGP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:15.858942] [INFO ] [LOCAL ] EGP: Starting deletion of primary data. [2025-10-20T18:30:15.859024] [DEBUG] [LOCAL ] EGP: No primary data file found to delete. [2025-10-20T18:30:15.859062] [INFO ] [LOCAL ] EGP: Starting deletion of derivative data. [2025-10-20T18:30:15.859291] [DEBUG] [LOCAL ] EGP: Deleted derivative data file. [2025-10-20T18:30:15.859894] [INFO ] [MAIN ] Update Progress: 709/2367 symbols completed. EGP took 28.2s. Estimated remaining: 3h 30m 55s. Rate: 7.9 symbols/min [2025-10-20T18:30:16.036188] [INFO ] [MAIN ] EIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:16.063691] [DEBUG] [LOCAL ] EIC: Loaded 23126 existing records from primary storage. [2025-10-20T18:30:16.064006] [INFO ] [MAIN ] EIC: Loaded 23126 existing records. [2025-10-20T18:30:16.258412] [WARN ] [MAIN ] EIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:16.253896] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-05. [2025-10-20T18:30:16.258961] [INFO ] [MAIN ] EIC: Wiping existing primary data before full refresh. [2025-10-20T18:30:16.259056] [INFO ] [LOCAL ] EIC: Starting deletion of primary data. [2025-10-20T18:30:16.260366] [DEBUG] [LOCAL ] EIC: Deleted primary data file. [2025-10-20T18:30:16.260447] [INFO ] [MAIN ] EIC: Starting full primary refresh. [2025-10-20T18:30:16.637149] [INFO ] [REMOTE ] EIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:17.051423] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-01. [2025-10-20T18:30:17.144865] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:30:17.579275] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:30:17.851461] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-01. [2025-10-20T18:30:18.651370] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-06. [2025-10-20T18:30:18.738120] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:30:18.902492] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:30:19.451354] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-02. [2025-10-20T18:30:19.557737] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T18:30:20.251383] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-07. [2025-10-20T18:30:20.320347] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:30:21.051414] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-05. [2025-10-20T18:30:21.744605] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:30:21.851386] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-02. [2025-10-20T18:30:22.225054] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:30:22.651404] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-08. [2025-10-20T18:30:22.999365] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:30:23.451388] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-03. [2025-10-20T18:30:24.025761] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:30:24.251394] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-09. [2025-10-20T18:30:24.363411] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:30:25.051361] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-06. [2025-10-20T18:30:25.343908] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:30:25.851343] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-04. [2025-10-20T18:30:26.589440] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T18:30:26.626171] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:30:26.651334] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-07. [2025-10-20T18:30:27.451367] [INFO ] [REMOTE ] EHAB: Fetching primary data for 2025-10. [2025-10-20T18:30:28.257288] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-03. [2025-10-20T18:30:28.269762] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:30:28.680254] [INFO ] [REMOTE ] EHAB: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T18:30:28.683951] [INFO ] [REMOTE ] EHAB: Full fetch complete (40774 records). [2025-10-20T18:30:28.684070] [DEBUG] [VALIDATOR ] EHAB: Validating primary data. [2025-10-20T18:30:28.684581] [FATAL] [VALIDATOR ] EHAB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:30:28.684705] [FATAL] [MAIN ] EHAB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:28.684756] [WARN ] [MAIN ] EHAB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:28.684820] [INFO ] [LOCAL ] EHAB: Starting deletion of primary data. [2025-10-20T18:30:28.684883] [DEBUG] [LOCAL ] EHAB: No primary data file found to delete. [2025-10-20T18:30:28.684917] [INFO ] [LOCAL ] EHAB: Starting deletion of derivative data. [2025-10-20T18:30:28.685043] [DEBUG] [LOCAL ] EHAB: Deleted derivative data file. [2025-10-20T18:30:28.685110] [INFO ] [MAIN ] Update Progress: 710/2367 symbols completed. EHAB took 33.8s. Estimated remaining: 3h 30m 59s. Rate: 7.9 symbols/min [2025-10-20T18:30:28.851773] [INFO ] [MAIN ] EICA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:28.851888] [DEBUG] [LOCAL ] EICA: No existing data file found in primary storage. [2025-10-20T18:30:28.851924] [WARN ] [MAIN ] EICA: No existing data found, performing full refresh instead. [2025-10-20T18:30:28.852030] [INFO ] [MAIN ] EICA: Wiping existing primary data before full refresh. [2025-10-20T18:30:28.852070] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-10-20T18:30:28.852122] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-10-20T18:30:28.852155] [INFO ] [MAIN ] EICA: Starting full primary refresh. [2025-10-20T18:30:29.056550] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-08. [2025-10-20T18:30:29.203877] [INFO ] [REMOTE ] EICA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:29.415337] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T18:30:29.851432] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-01. [2025-10-20T18:30:30.651365] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-05. [2025-10-20T18:30:31.451374] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-04. [2025-10-20T18:30:31.535995] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:30:31.578384] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:30:31.604930] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-08 (2.55 seconds). [2025-10-20T18:30:32.251371] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-02. [2025-10-20T18:30:32.518303] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:30:32.539395] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-02 (0.29 seconds). [2025-10-20T18:30:33.051378] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-03. [2025-10-20T18:30:33.404608] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:30:33.851385] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-06. [2025-10-20T18:30:34.584078] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:30:34.651376] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-09. [2025-10-20T18:30:35.451456] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-05. [2025-10-20T18:30:36.048494] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:30:36.251410] [INFO ] [REMOTE ] EHC: Fetching primary data for 2025-10. [2025-10-20T18:30:36.535075] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:30:37.051448] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-04. [2025-10-20T18:30:37.404592] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:30:37.661986] [INFO ] [REMOTE ] EHC: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T18:30:37.668164] [INFO ] [REMOTE ] EHC: Full fetch complete (58779 records). [2025-10-20T18:30:37.668305] [DEBUG] [VALIDATOR ] EHC: Validating primary data. [2025-10-20T18:30:37.861455] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-05. [2025-10-20T18:30:38.056482] [DEBUG] [VALIDATOR ] EHC: All expected trading days are present. [2025-10-20T18:30:38.056568] [DEBUG] [VALIDATOR ] EHC: Primary data validation passed. [2025-10-20T18:30:38.057313] [INFO ] [MAIN ] EHC: Fetched primary data is valid. [2025-10-20T18:30:38.057378] [DEBUG] [LOCAL ] EHC: Saving primary data. [2025-10-20T18:30:38.058675] [DEBUG] [LOCAL ] EHC: Trimmed 58779 records to 41026 (keeping 200 days of data) [2025-10-20T18:30:38.317317] [DEBUG] [LOCAL ] EHC: Primary data saved (41026 records). [2025-10-20T18:30:38.317524] [INFO ] [MAIN ] EHC: Full history saved. [2025-10-20T18:30:38.318601] [INFO ] [MAIN ] Update Progress: 711/2367 symbols completed. EHC took 37.8s. Estimated remaining: 3h 30m 56s. Rate: 7.9 symbols/min [2025-10-20T18:30:38.396140] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:30:38.492782] [INFO ] [MAIN ] EICB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:38.492972] [DEBUG] [LOCAL ] EICB: No existing data file found in primary storage. [2025-10-20T18:30:38.493054] [WARN ] [MAIN ] EICB: No existing data found, performing full refresh instead. [2025-10-20T18:30:38.493194] [INFO ] [MAIN ] EICB: Wiping existing primary data before full refresh. [2025-10-20T18:30:38.493234] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-10-20T18:30:38.493273] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-10-20T18:30:38.493305] [INFO ] [MAIN ] EICB: Starting full primary refresh. [2025-10-20T18:30:38.657821] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-06. [2025-10-20T18:30:38.878896] [INFO ] [REMOTE ] EICB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:39.142865] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T18:30:39.451380] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-01. [2025-10-20T18:30:39.886717] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-10-20T18:30:40.251442] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-07. [2025-10-20T18:30:41.051420] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-07. [2025-10-20T18:30:41.069765] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:30:41.661269] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:30:41.851400] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-02. [2025-10-20T18:30:42.261903] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:30:42.651371] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-08. [2025-10-20T18:30:43.007407] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:30:43.451408] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-08. [2025-10-20T18:30:44.251369] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-06. [2025-10-20T18:30:44.335663] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:30:45.051404] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-09. [2025-10-20T18:30:45.403288] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-10-20T18:30:45.601695] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:30:45.851378] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-07. [2025-10-20T18:30:46.651419] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-03. [2025-10-20T18:30:47.083977] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:30:47.234458] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:30:47.451382] [INFO ] [REMOTE ] EICA: Fetching primary data for 2025-10. [2025-10-20T18:30:47.909616] [INFO ] [REMOTE ] EICA: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:30:47.910548] [INFO ] [REMOTE ] EICA: Full fetch complete (1256 records). [2025-10-20T18:30:47.910632] [DEBUG] [VALIDATOR ] EICA: Validating primary data. [2025-10-20T18:30:48.104139] [WARN ] [VALIDATOR ] EICA: Validation failed - insufficient historical data. Have 193 unique days, need 200 days [2025-10-20T18:30:48.104321] [FATAL] [MAIN ] EICA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:30:48.104390] [WARN ] [MAIN ] EICA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:30:48.104461] [INFO ] [LOCAL ] EICA: Starting deletion of primary data. [2025-10-20T18:30:48.104527] [DEBUG] [LOCAL ] EICA: No primary data file found to delete. [2025-10-20T18:30:48.104563] [INFO ] [LOCAL ] EICA: Starting deletion of derivative data. [2025-10-20T18:30:48.104602] [DEBUG] [LOCAL ] EICA: No derivative data file found to delete. [2025-10-20T18:30:48.104684] [INFO ] [MAIN ] Update Progress: 712/2367 symbols completed. EICA took 19.4s. Estimated remaining: 3h 30m 54s. Rate: 7.8 symbols/min [2025-10-20T18:30:48.255592] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-08. [2025-10-20T18:30:48.277130] [INFO ] [MAIN ] EICC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:48.278021] [DEBUG] [LOCAL ] EICC: No existing data file found in primary storage. [2025-10-20T18:30:48.278079] [WARN ] [MAIN ] EICC: No existing data found, performing full refresh instead. [2025-10-20T18:30:48.278207] [INFO ] [MAIN ] EICC: Wiping existing primary data before full refresh. [2025-10-20T18:30:48.278261] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-10-20T18:30:48.278314] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-10-20T18:30:48.278358] [INFO ] [MAIN ] EICC: Starting full primary refresh. [2025-10-20T18:30:48.653585] [INFO ] [REMOTE ] EICC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:49.051418] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-04. [2025-10-20T18:30:49.156246] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:30:49.412469] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T18:30:49.851361] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-01. [2025-10-20T18:30:50.453962] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T18:30:50.651376] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-09. [2025-10-20T18:30:51.444535] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:30:51.451324] [INFO ] [REMOTE ] EHI: Fetching primary data for 2025-10. [2025-10-20T18:30:52.251395] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-09. [2025-10-20T18:30:52.261142] [INFO ] [REMOTE ] EHI: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:30:52.263440] [INFO ] [REMOTE ] EHI: Full fetch complete (15828 records). [2025-10-20T18:30:52.263541] [DEBUG] [VALIDATOR ] EHI: Validating primary data. [2025-10-20T18:30:52.632048] [DEBUG] [VALIDATOR ] EHI: All expected trading days are present. [2025-10-20T18:30:52.632353] [DEBUG] [VALIDATOR ] EHI: Primary data validation passed. [2025-10-20T18:30:52.632755] [INFO ] [MAIN ] EHI: Fetched primary data is valid. [2025-10-20T18:30:52.632900] [DEBUG] [LOCAL ] EHI: Saving primary data. [2025-10-20T18:30:52.633700] [DEBUG] [LOCAL ] EHI: Trimmed 15828 records to 10543 (keeping 200 days of data) [2025-10-20T18:30:52.695593] [DEBUG] [LOCAL ] EHI: Primary data saved (10543 records). [2025-10-20T18:30:52.695744] [INFO ] [MAIN ] EHI: Full history saved. [2025-10-20T18:30:52.695833] [INFO ] [MAIN ] Update Progress: 713/2367 symbols completed. EHI took 40.5s. Estimated remaining: 3h 30m 39s. Rate: 7.9 symbols/min [2025-10-20T18:30:52.870605] [INFO ] [MAIN ] EIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:52.891390] [DEBUG] [LOCAL ] EIG: Loaded 20485 existing records from primary storage. [2025-10-20T18:30:52.891692] [INFO ] [MAIN ] EIG: Loaded 20485 existing records. [2025-10-20T18:30:53.053532] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-05. [2025-10-20T18:30:53.102158] [WARN ] [MAIN ] EIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:30:53.102438] [INFO ] [MAIN ] EIG: Wiping existing primary data before full refresh. [2025-10-20T18:30:53.102490] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-10-20T18:30:53.102926] [DEBUG] [LOCAL ] EIG: Deleted primary data file. [2025-10-20T18:30:53.102983] [INFO ] [MAIN ] EIG: Starting full primary refresh. [2025-10-20T18:30:53.321457] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:30:53.585781] [INFO ] [REMOTE ] EIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:53.586071] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T18:30:53.851452] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-01. [2025-10-20T18:30:54.651597] [INFO ] [REMOTE ] EIC: Fetching primary data for 2025-10. [2025-10-20T18:30:54.947753] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T18:30:55.451392] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-02. [2025-10-20T18:30:55.797667] [INFO ] [REMOTE ] EIC: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T18:30:55.800817] [INFO ] [REMOTE ] EIC: Full fetch complete (34300 records). [2025-10-20T18:30:55.800895] [DEBUG] [VALIDATOR ] EIC: Validating primary data. [2025-10-20T18:30:55.985368] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:30:56.195012] [DEBUG] [VALIDATOR ] EIC: All expected trading days are present. [2025-10-20T18:30:56.195093] [DEBUG] [VALIDATOR ] EIC: Primary data validation passed. [2025-10-20T18:30:56.195706] [INFO ] [MAIN ] EIC: Fetched primary data is valid. [2025-10-20T18:30:56.195765] [DEBUG] [LOCAL ] EIC: Saving primary data. [2025-10-20T18:30:56.196491] [DEBUG] [LOCAL ] EIC: Trimmed 34300 records to 22699 (keeping 200 days of data) [2025-10-20T18:30:56.256328] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-03. [2025-10-20T18:30:56.336848] [DEBUG] [LOCAL ] EIC: Primary data saved (22699 records). [2025-10-20T18:30:56.337028] [INFO ] [MAIN ] EIC: Full history saved. [2025-10-20T18:30:56.337114] [INFO ] [MAIN ] Update Progress: 714/2367 symbols completed. EIC took 40.5s. Estimated remaining: 3h 30m 22s. Rate: 7.9 symbols/min [2025-10-20T18:30:56.502784] [INFO ] [MAIN ] EIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:30:56.502948] [DEBUG] [LOCAL ] EIX: No existing data file found in primary storage. [2025-10-20T18:30:56.502983] [WARN ] [MAIN ] EIX: No existing data found, performing full refresh instead. [2025-10-20T18:30:56.503095] [INFO ] [MAIN ] EIX: Wiping existing primary data before full refresh. [2025-10-20T18:30:56.503134] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-10-20T18:30:56.503173] [DEBUG] [LOCAL ] EIX: No primary data file found to delete. [2025-10-20T18:30:56.503203] [INFO ] [MAIN ] EIX: Starting full primary refresh. [2025-10-20T18:30:56.853899] [INFO ] [REMOTE ] EIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:30:56.876215] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:30:57.051448] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-01. [2025-10-20T18:30:57.851379] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-04. [2025-10-20T18:30:58.519214] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T18:30:58.651451] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-05. [2025-10-20T18:30:58.991808] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T18:30:59.302964] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:30:59.451403] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-06. [2025-10-20T18:31:00.061524] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T18:31:00.251436] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-02. [2025-10-20T18:31:01.051395] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-07. [2025-10-20T18:31:01.739598] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:31:01.849868] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:31:01.851993] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-06. [2025-10-20T18:31:02.651350] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-02. [2025-10-20T18:31:02.680965] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T18:31:03.451384] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-03. [2025-10-20T18:31:04.251358] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-08. [2025-10-20T18:31:04.398977] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:31:04.486046] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:31:04.644550] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T18:31:05.051431] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-09. [2025-10-20T18:31:05.803738] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T18:31:05.851384] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-03. [2025-10-20T18:31:06.651388] [INFO ] [REMOTE ] EICC: Fetching primary data for 2025-10. [2025-10-20T18:31:07.335182] [INFO ] [REMOTE ] EICC: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:31:07.336805] [INFO ] [REMOTE ] EICC: Full fetch complete (8439 records). [2025-10-20T18:31:07.336890] [DEBUG] [VALIDATOR ] EICC: Validating primary data. [2025-10-20T18:31:07.337297] [FATAL] [VALIDATOR ] EICC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:07.337379] [FATAL] [MAIN ] EICC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:07.337422] [WARN ] [MAIN ] EICC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:07.337459] [INFO ] [LOCAL ] EICC: Starting deletion of primary data. [2025-10-20T18:31:07.337513] [DEBUG] [LOCAL ] EICC: No primary data file found to delete. [2025-10-20T18:31:07.337546] [INFO ] [LOCAL ] EICC: Starting deletion of derivative data. [2025-10-20T18:31:07.337583] [DEBUG] [LOCAL ] EICC: No derivative data file found to delete. [2025-10-20T18:31:07.337634] [INFO ] [MAIN ] Update Progress: 715/2367 symbols completed. EICC took 19.2s. Estimated remaining: 3h 30m 22s. Rate: 7.9 symbols/min [2025-10-20T18:31:07.456612] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-04. [2025-10-20T18:31:07.522326] [INFO ] [MAIN ] EL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:07.586733] [DEBUG] [LOCAL ] EL: Loaded 54155 existing records from primary storage. [2025-10-20T18:31:07.587105] [INFO ] [MAIN ] EL: Loaded 54155 existing records. [2025-10-20T18:31:07.772214] [WARN ] [MAIN ] EL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:07.772566] [INFO ] [MAIN ] EL: Wiping existing primary data before full refresh. [2025-10-20T18:31:07.772728] [INFO ] [LOCAL ] EL: Starting deletion of primary data. [2025-10-20T18:31:07.772953] [DEBUG] [LOCAL ] EL: Deleted primary data file. [2025-10-20T18:31:07.773009] [INFO ] [MAIN ] EL: Starting full primary refresh. [2025-10-20T18:31:08.192572] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T18:31:08.190943] [INFO ] [REMOTE ] EL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:08.251371] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-01. [2025-10-20T18:31:08.286243] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:31:09.051426] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-04. [2025-10-20T18:31:09.851371] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-07. [2025-10-20T18:31:10.014213] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:31:10.396089] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:31:10.651383] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-05. [2025-10-20T18:31:10.882780] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:31:11.451395] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-02. [2025-10-20T18:31:11.781476] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T18:31:12.251422] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-08. [2025-10-20T18:31:12.848432] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:31:13.051413] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-05. [2025-10-20T18:31:13.509313] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:31:13.851353] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-03. [2025-10-20T18:31:14.651346] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-06. [2025-10-20T18:31:14.733162] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:31:15.451349] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-06. [2025-10-20T18:31:15.712788] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:31:15.853236] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T18:31:16.251429] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-07. [2025-10-20T18:31:17.051439] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-09. [2025-10-20T18:31:17.332798] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T18:31:17.380526] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:31:17.490332] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:31:17.851367] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-07. [2025-10-20T18:31:18.651421] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-08. [2025-10-20T18:31:19.451407] [INFO ] [REMOTE ] EICB: Fetching primary data for 2025-10. [2025-10-20T18:31:19.893320] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:31:19.926056] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:31:19.977298] [INFO ] [REMOTE ] EICB: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T18:31:19.978229] [INFO ] [REMOTE ] EICB: Full fetch complete (2692 records). [2025-10-20T18:31:19.978297] [DEBUG] [VALIDATOR ] EICB: Validating primary data. [2025-10-20T18:31:19.978637] [FATAL] [VALIDATOR ] EICB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:19.978740] [FATAL] [MAIN ] EICB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:19.978783] [WARN ] [MAIN ] EICB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:19.978825] [INFO ] [LOCAL ] EICB: Starting deletion of primary data. [2025-10-20T18:31:19.978881] [DEBUG] [LOCAL ] EICB: No primary data file found to delete. [2025-10-20T18:31:19.978915] [INFO ] [LOCAL ] EICB: Starting deletion of derivative data. [2025-10-20T18:31:19.978954] [DEBUG] [LOCAL ] EICB: No derivative data file found to delete. [2025-10-20T18:31:19.979004] [INFO ] [MAIN ] Update Progress: 716/2367 symbols completed. EICB took 41.7s. Estimated remaining: 3h 30m 26s. Rate: 7.8 symbols/min [2025-10-20T18:31:20.147960] [INFO ] [MAIN ] ELAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:20.200697] [DEBUG] [LOCAL ] ELAN: Loaded 54080 existing records from primary storage. [2025-10-20T18:31:20.201033] [INFO ] [MAIN ] ELAN: Loaded 54080 existing records. [2025-10-20T18:31:20.259421] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-08. [2025-10-20T18:31:20.386512] [WARN ] [MAIN ] ELAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:20.386743] [INFO ] [MAIN ] ELAN: Wiping existing primary data before full refresh. [2025-10-20T18:31:20.386807] [INFO ] [LOCAL ] ELAN: Starting deletion of primary data. [2025-10-20T18:31:20.387010] [DEBUG] [LOCAL ] ELAN: Deleted primary data file. [2025-10-20T18:31:20.387047] [INFO ] [MAIN ] ELAN: Starting full primary refresh. [2025-10-20T18:31:20.734914] [INFO ] [REMOTE ] ELAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:21.051373] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-09. [2025-10-20T18:31:21.851371] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-01. [2025-10-20T18:31:22.079608] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:31:22.355937] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-10-20T18:31:22.651374] [INFO ] [REMOTE ] EIG: Fetching primary data for 2025-10. [2025-10-20T18:31:23.121875] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:31:23.451387] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-04. [2025-10-20T18:31:23.716386] [INFO ] [REMOTE ] EIG: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T18:31:23.718964] [INFO ] [REMOTE ] EIG: Full fetch complete (27136 records). [2025-10-20T18:31:23.719034] [DEBUG] [VALIDATOR ] EIG: Validating primary data. [2025-10-20T18:31:23.719609] [FATAL] [VALIDATOR ] EIG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:23.719711] [FATAL] [MAIN ] EIG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:23.719754] [WARN ] [MAIN ] EIG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:23.719790] [INFO ] [LOCAL ] EIG: Starting deletion of primary data. [2025-10-20T18:31:23.719843] [DEBUG] [LOCAL ] EIG: No primary data file found to delete. [2025-10-20T18:31:23.719876] [INFO ] [LOCAL ] EIG: Starting deletion of derivative data. [2025-10-20T18:31:23.720029] [DEBUG] [LOCAL ] EIG: Deleted derivative data file. [2025-10-20T18:31:23.720111] [INFO ] [MAIN ] Update Progress: 717/2367 symbols completed. EIG took 31.0s. Estimated remaining: 3h 30m 9s. Rate: 7.9 symbols/min [2025-10-20T18:31:24.251396] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-09. [2025-10-20T18:31:25.051445] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-02. [2025-10-20T18:31:25.539937] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-10-20T18:31:25.757236] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:31:25.851353] [INFO ] [REMOTE ] EIX: Fetching primary data for 2025-10. [2025-10-20T18:31:26.651397] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-05. [2025-10-20T18:31:26.678364] [INFO ] [REMOTE ] EIX: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T18:31:26.685752] [INFO ] [REMOTE ] EIX: Full fetch complete (77063 records). [2025-10-20T18:31:26.685879] [DEBUG] [VALIDATOR ] EIX: Validating primary data. [2025-10-20T18:31:26.686572] [FATAL] [VALIDATOR ] EIX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:31:26.686682] [FATAL] [MAIN ] EIX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:31:26.686734] [WARN ] [MAIN ] EIX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:31:26.686793] [INFO ] [LOCAL ] EIX: Starting deletion of primary data. [2025-10-20T18:31:26.686849] [DEBUG] [LOCAL ] EIX: No primary data file found to delete. [2025-10-20T18:31:26.686883] [INFO ] [LOCAL ] EIX: Starting deletion of derivative data. [2025-10-20T18:31:26.686920] [DEBUG] [LOCAL ] EIX: No derivative data file found to delete. [2025-10-20T18:31:26.686974] [INFO ] [MAIN ] Update Progress: 718/2367 symbols completed. EIX took 30.3s. Estimated remaining: 3h 29m 51s. Rate: 7.9 symbols/min [2025-10-20T18:31:26.906486] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T18:31:27.451371] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-03. [2025-10-20T18:31:28.325766] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T18:31:28.337183] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-06. [2025-10-20T18:31:28.795150] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:31:29.051381] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-04. [2025-10-20T18:31:30.465207] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T18:31:30.477190] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-07. [2025-10-20T18:31:30.822801] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:31:30.836329] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-05. [2025-10-20T18:31:32.968976] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T18:31:32.981220] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-06. [2025-10-20T18:31:33.499688] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-07 (3.02 seconds). [2025-10-20T18:31:33.512347] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-08. [2025-10-20T18:31:35.201158] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-10-20T18:31:35.211961] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-07. [2025-10-20T18:31:36.183424] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-08 (2.67 seconds). [2025-10-20T18:31:36.195469] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-09. [2025-10-20T18:31:36.700941] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:31:36.712622] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-08. [2025-10-20T18:31:37.711161] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T18:31:37.722215] [INFO ] [REMOTE ] EL: Fetching primary data for 2025-10. [2025-10-20T18:31:38.266464] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T18:31:38.278498] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-09. [2025-10-20T18:31:38.398324] [INFO ] [REMOTE ] EL: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:31:38.406363] [INFO ] [REMOTE ] EL: Full fetch complete (77440 records). [2025-10-20T18:31:38.406512] [DEBUG] [VALIDATOR ] EL: Validating primary data. [2025-10-20T18:31:38.864473] [DEBUG] [VALIDATOR ] EL: All expected trading days are present. [2025-10-20T18:31:38.864550] [DEBUG] [VALIDATOR ] EL: Primary data validation passed. [2025-10-20T18:31:38.865421] [INFO ] [MAIN ] EL: Fetched primary data is valid. [2025-10-20T18:31:38.865484] [DEBUG] [LOCAL ] EL: Saving primary data. [2025-10-20T18:31:38.866906] [DEBUG] [LOCAL ] EL: Trimmed 77440 records to 53387 (keeping 200 days of data) [2025-10-20T18:31:39.191699] [DEBUG] [LOCAL ] EL: Primary data saved (53387 records). [2025-10-20T18:31:39.191849] [INFO ] [MAIN ] EL: Full history saved. [2025-10-20T18:31:39.191930] [INFO ] [MAIN ] Update Progress: 719/2367 symbols completed. EL took 31.9s. Estimated remaining: 3h 29m 54s. Rate: 7.9 symbols/min [2025-10-20T18:31:39.905363] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:31:39.918183] [INFO ] [REMOTE ] ELAN: Fetching primary data for 2025-10. [2025-10-20T18:31:41.490422] [INFO ] [REMOTE ] ELAN: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:31:41.498373] [INFO ] [REMOTE ] ELAN: Full fetch complete (77093 records). [2025-10-20T18:31:41.498529] [DEBUG] [VALIDATOR ] ELAN: Validating primary data. [2025-10-20T18:31:41.883912] [DEBUG] [VALIDATOR ] ELAN: All expected trading days are present. [2025-10-20T18:31:41.883998] [DEBUG] [VALIDATOR ] ELAN: Primary data validation passed. [2025-10-20T18:31:41.884874] [INFO ] [MAIN ] ELAN: Fetched primary data is valid. [2025-10-20T18:31:41.884937] [DEBUG] [LOCAL ] ELAN: Saving primary data. [2025-10-20T18:31:41.886349] [DEBUG] [LOCAL ] ELAN: Trimmed 77093 records to 53315 (keeping 200 days of data) [2025-10-20T18:31:42.189693] [DEBUG] [LOCAL ] ELAN: Primary data saved (53315 records). [2025-10-20T18:31:42.189828] [INFO ] [MAIN ] ELAN: Full history saved. [2025-10-20T18:31:42.189903] [INFO ] [MAIN ] Update Progress: 720/2367 symbols completed. ELAN took 22.2s. Estimated remaining: 3h 29m 36s. Rate: 7.9 symbols/min [2025-10-20T18:31:42.791239] [INFO ] [MAIN ] ELME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.916572] [INFO ] [MAIN ] ELP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.953427] [INFO ] [MAIN ] ELF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.956488] [INFO ] [MAIN ] ELC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:42.965264] [DEBUG] [LOCAL ] ELC: Loaded 3542 existing records from primary storage. [2025-10-20T18:31:42.965728] [INFO ] [MAIN ] ELC: Loaded 3542 existing records. [2025-10-20T18:31:43.124974] [DEBUG] [LOCAL ] ELP: Loaded 26040 existing records from primary storage. [2025-10-20T18:31:43.169951] [DEBUG] [LOCAL ] ELME: Loaded 39953 existing records from primary storage. [2025-10-20T18:31:43.166891] [INFO ] [MAIN ] ELP: Loaded 26040 existing records. [2025-10-20T18:31:43.198489] [WARN ] [MAIN ] ELC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.219964] [INFO ] [MAIN ] ELME: Loaded 39953 existing records. [2025-10-20T18:31:43.220422] [INFO ] [MAIN ] ELC: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.273796] [INFO ] [LOCAL ] ELC: Starting deletion of primary data. [2025-10-20T18:31:43.347152] [DEBUG] [LOCAL ] ELC: Deleted primary data file. [2025-10-20T18:31:43.404553] [INFO ] [MAIN ] ELC: Starting full primary refresh. [2025-10-20T18:31:43.438371] [DEBUG] [LOCAL ] ELF: Loaded 52711 existing records from primary storage. [2025-10-20T18:31:43.583187] [INFO ] [MAIN ] ELF: Loaded 52711 existing records. [2025-10-20T18:31:43.619976] [WARN ] [MAIN ] ELP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.696846] [INFO ] [MAIN ] ELP: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.742131] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-10-20T18:31:43.728565] [WARN ] [MAIN ] ELME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:43.778538] [DEBUG] [LOCAL ] ELP: Deleted primary data file. [2025-10-20T18:31:43.799598] [INFO ] [MAIN ] ELME: Wiping existing primary data before full refresh. [2025-10-20T18:31:43.810024] [INFO ] [MAIN ] ELP: Starting full primary refresh. [2025-10-20T18:31:43.815350] [INFO ] [LOCAL ] ELME: Starting deletion of primary data. [2025-10-20T18:31:43.996733] [DEBUG] [LOCAL ] ELME: Deleted primary data file. [2025-10-20T18:31:44.021227] [INFO ] [MAIN ] ELME: Starting full primary refresh. [2025-10-20T18:31:44.265913] [WARN ] [MAIN ] ELF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:31:44.376808] [INFO ] [MAIN ] ELF: Wiping existing primary data before full refresh. [2025-10-20T18:31:44.402782] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-10-20T18:31:44.493615] [DEBUG] [LOCAL ] ELF: Deleted primary data file. [2025-10-20T18:31:44.579897] [INFO ] [MAIN ] ELF: Starting full primary refresh. [2025-10-20T18:31:44.609284] [INFO ] [REMOTE ] ELC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:44.622344] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-01. [2025-10-20T18:31:44.982530] [INFO ] [REMOTE ] ELP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.048332] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-01. [2025-10-20T18:31:45.267347] [INFO ] [REMOTE ] ELME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.283605] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-01. [2025-10-20T18:31:45.349301] [INFO ] [REMOTE ] ELF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:45.349606] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-01. [2025-10-20T18:31:45.487153] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T18:31:45.488826] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-02. [2025-10-20T18:31:45.812161] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:31:45.813467] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-03. [2025-10-20T18:31:46.365872] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:31:46.372806] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-02. [2025-10-20T18:31:46.378372] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T18:31:46.380124] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-04. [2025-10-20T18:31:46.738023] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:31:46.748923] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-02. [2025-10-20T18:31:47.034710] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T18:31:47.036819] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-05. [2025-10-20T18:31:47.121923] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T18:31:47.134345] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-02. [2025-10-20T18:31:47.310286] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:31:47.315627] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-03. [2025-10-20T18:31:47.622076] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:31:47.624005] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-06. [2025-10-20T18:31:47.691217] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:31:48.004023] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T18:31:48.194382] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T18:31:48.251366] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-07. [2025-10-20T18:31:48.583179] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-10-20T18:31:49.051338] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-04. [2025-10-20T18:31:49.260576] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-02 (2.13 seconds). [2025-10-20T18:31:49.851373] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-03. [2025-10-20T18:31:50.651429] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-03. [2025-10-20T18:31:51.097821] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T18:31:51.451379] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-05. [2025-10-20T18:31:52.251383] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-08. [2025-10-20T18:31:52.601880] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T18:31:52.781096] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-03 (2.13 seconds). [2025-10-20T18:31:52.791607] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T18:31:53.051410] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-06. [2025-10-20T18:31:53.197477] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-03 (3.35 seconds). [2025-10-20T18:31:53.851490] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-04. [2025-10-20T18:31:54.082452] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T18:31:54.651488] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-04. [2025-10-20T18:31:55.451410] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-09. [2025-10-20T18:31:55.847122] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T18:31:56.114564] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T18:31:56.191447] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T18:31:56.251399] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-05. [2025-10-20T18:31:57.051419] [INFO ] [REMOTE ] ELC: Fetching primary data for 2025-10. [2025-10-20T18:31:57.636610] [INFO ] [REMOTE ] ELC: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T18:31:57.638310] [INFO ] [REMOTE ] ELC: Full fetch complete (4927 records). [2025-10-20T18:31:57.638386] [DEBUG] [VALIDATOR ] ELC: Validating primary data. [2025-10-20T18:31:57.859786] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-05. [2025-10-20T18:31:58.040517] [DEBUG] [VALIDATOR ] ELC: All expected trading days are present. [2025-10-20T18:31:58.040628] [DEBUG] [VALIDATOR ] ELC: Primary data validation passed. [2025-10-20T18:31:58.040876] [INFO ] [MAIN ] ELC: Fetched primary data is valid. [2025-10-20T18:31:58.041048] [DEBUG] [LOCAL ] ELC: Saving primary data. [2025-10-20T18:31:58.041636] [DEBUG] [LOCAL ] ELC: Trimmed 4927 records to 3476 (keeping 200 days of data) [2025-10-20T18:31:58.066354] [DEBUG] [LOCAL ] ELC: Primary data saved (3476 records). [2025-10-20T18:31:58.066555] [INFO ] [MAIN ] ELC: Full history saved. [2025-10-20T18:31:58.066683] [INFO ] [MAIN ] Update Progress: 721/2367 symbols completed. ELC took 15.9s. Estimated remaining: 3h 29m 47s. Rate: 7.8 symbols/min [2025-10-20T18:31:58.253407] [INFO ] [MAIN ] ELPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:31:58.253778] [DEBUG] [LOCAL ] ELPC: No existing data file found in primary storage. [2025-10-20T18:31:58.253847] [WARN ] [MAIN ] ELPC: No existing data found, performing full refresh instead. [2025-10-20T18:31:58.253956] [INFO ] [MAIN ] ELPC: Wiping existing primary data before full refresh. [2025-10-20T18:31:58.254095] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-10-20T18:31:58.254201] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-10-20T18:31:58.254247] [INFO ] [MAIN ] ELPC: Starting full primary refresh. [2025-10-20T18:31:58.465431] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T18:31:58.656703] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-07. [2025-10-20T18:31:58.708148] [INFO ] [REMOTE ] ELPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:31:59.178079] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:31:59.451431] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-01. [2025-10-20T18:31:59.852634] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T18:31:59.982395] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T18:32:00.251417] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-06. [2025-10-20T18:32:01.051431] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-08. [2025-10-20T18:32:01.851403] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-02. [2025-10-20T18:32:02.186301] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T18:32:02.280929] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T18:32:02.397833] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:32:02.651450] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-06. [2025-10-20T18:32:03.451455] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-09. [2025-10-20T18:32:04.251368] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-07. [2025-10-20T18:32:04.310596] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T18:32:04.647424] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:32:05.051414] [INFO ] [REMOTE ] ELP: Fetching primary data for 2025-10. [2025-10-20T18:32:05.851401] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-07. [2025-10-20T18:32:06.225992] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:32:06.249448] [INFO ] [REMOTE ] ELP: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:32:06.253378] [INFO ] [REMOTE ] ELP: Full fetch complete (36762 records). [2025-10-20T18:32:06.253472] [DEBUG] [VALIDATOR ] ELP: Validating primary data. [2025-10-20T18:32:06.254235] [FATAL] [VALIDATOR ] ELP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:06.254325] [FATAL] [MAIN ] ELP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:06.254370] [WARN ] [MAIN ] ELP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:06.254411] [INFO ] [LOCAL ] ELP: Starting deletion of primary data. [2025-10-20T18:32:06.254476] [DEBUG] [LOCAL ] ELP: No primary data file found to delete. [2025-10-20T18:32:06.254511] [INFO ] [LOCAL ] ELP: Starting deletion of derivative data. [2025-10-20T18:32:06.254698] [DEBUG] [LOCAL ] ELP: Deleted derivative data file. [2025-10-20T18:32:06.254798] [INFO ] [MAIN ] Update Progress: 722/2367 symbols completed. ELP took 24.1s. Estimated remaining: 3h 29m 41s. Rate: 7.8 symbols/min [2025-10-20T18:32:06.429748] [INFO ] [MAIN ] ELS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:06.487612] [DEBUG] [LOCAL ] ELS: Loaded 50599 existing records from primary storage. [2025-10-20T18:32:06.488291] [INFO ] [MAIN ] ELS: Loaded 50599 existing records. [2025-10-20T18:32:06.656542] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-08. [2025-10-20T18:32:06.692350] [WARN ] [MAIN ] ELS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:06.692971] [INFO ] [MAIN ] ELS: Wiping existing primary data before full refresh. [2025-10-20T18:32:06.693041] [INFO ] [LOCAL ] ELS: Starting deletion of primary data. [2025-10-20T18:32:06.693863] [DEBUG] [LOCAL ] ELS: Deleted primary data file. [2025-10-20T18:32:06.693935] [INFO ] [MAIN ] ELS: Starting full primary refresh. [2025-10-20T18:32:07.098543] [INFO ] [REMOTE ] ELS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:07.398009] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T18:32:07.451395] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-08. [2025-10-20T18:32:08.251402] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-03. [2025-10-20T18:32:08.545371] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T18:32:08.593704] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:32:08.653873] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:32:09.051411] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-09. [2025-10-20T18:32:09.851416] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-01. [2025-10-20T18:32:10.651444] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-09. [2025-10-20T18:32:10.734403] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:32:11.413096] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T18:32:11.451382] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-04. [2025-10-20T18:32:11.996478] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:32:12.251348] [INFO ] [REMOTE ] ELME: Fetching primary data for 2025-10. [2025-10-20T18:32:12.504683] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:32:13.051364] [INFO ] [REMOTE ] ELF: Fetching primary data for 2025-10. [2025-10-20T18:32:13.772331] [INFO ] [REMOTE ] ELME: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:32:13.778264] [INFO ] [REMOTE ] ELME: Full fetch complete (55971 records). [2025-10-20T18:32:13.778361] [DEBUG] [VALIDATOR ] ELME: Validating primary data. [2025-10-20T18:32:13.856699] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-05. [2025-10-20T18:32:14.156710] [DEBUG] [VALIDATOR ] ELME: All expected trading days are present. [2025-10-20T18:32:14.156816] [DEBUG] [VALIDATOR ] ELME: Primary data validation passed. [2025-10-20T18:32:14.157525] [INFO ] [MAIN ] ELME: Fetched primary data is valid. [2025-10-20T18:32:14.157572] [DEBUG] [LOCAL ] ELME: Saving primary data. [2025-10-20T18:32:14.158790] [DEBUG] [LOCAL ] ELME: Trimmed 55971 records to 39361 (keeping 200 days of data) [2025-10-20T18:32:14.254843] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T18:32:14.402715] [DEBUG] [LOCAL ] ELME: Primary data saved (39361 records). [2025-10-20T18:32:14.402860] [INFO ] [MAIN ] ELME: Full history saved. [2025-10-20T18:32:14.402940] [INFO ] [MAIN ] Update Progress: 723/2367 symbols completed. ELME took 32.2s. Estimated remaining: 3h 29m 34s. Rate: 7.8 symbols/min [2025-10-20T18:32:14.576636] [INFO ] [MAIN ] ELV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:14.626745] [DEBUG] [LOCAL ] ELV: Loaded 51393 existing records from primary storage. [2025-10-20T18:32:14.627221] [INFO ] [MAIN ] ELV: Loaded 51393 existing records. [2025-10-20T18:32:14.657825] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-06. [2025-10-20T18:32:14.714440] [INFO ] [REMOTE ] ELF: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:32:14.766185] [INFO ] [REMOTE ] ELF: Full fetch complete (75113 records). [2025-10-20T18:32:14.776669] [DEBUG] [VALIDATOR ] ELF: Validating primary data. [2025-10-20T18:32:14.800856] [FATAL] [VALIDATOR ] ELF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:14.836441] [FATAL] [MAIN ] ELF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:14.841194] [WARN ] [MAIN ] ELV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:14.841327] [WARN ] [MAIN ] ELF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:14.841590] [INFO ] [MAIN ] ELV: Wiping existing primary data before full refresh. [2025-10-20T18:32:14.841724] [INFO ] [LOCAL ] ELF: Starting deletion of primary data. [2025-10-20T18:32:14.841831] [DEBUG] [LOCAL ] ELF: No primary data file found to delete. [2025-10-20T18:32:14.841927] [INFO ] [LOCAL ] ELV: Starting deletion of primary data. [2025-10-20T18:32:14.842028] [INFO ] [LOCAL ] ELF: Starting deletion of derivative data. [2025-10-20T18:32:14.842341] [DEBUG] [LOCAL ] ELV: Deleted primary data file. [2025-10-20T18:32:14.842318] [DEBUG] [LOCAL ] ELF: Deleted derivative data file. [2025-10-20T18:32:14.842895] [INFO ] [MAIN ] Update Progress: 724/2367 symbols completed. ELF took 32.6s. Estimated remaining: 3h 29m 10s. Rate: 7.9 symbols/min [2025-10-20T18:32:14.843001] [INFO ] [MAIN ] ELV: Starting full primary refresh. [2025-10-20T18:32:15.064512] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T18:32:15.204582] [INFO ] [MAIN ] EMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:15.240184] [DEBUG] [LOCAL ] EMA: No existing data file found in primary storage. [2025-10-20T18:32:15.272372] [WARN ] [MAIN ] EMA: No existing data found, performing full refresh instead. [2025-10-20T18:32:15.284057] [INFO ] [MAIN ] EMA: Wiping existing primary data before full refresh. [2025-10-20T18:32:15.289265] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-10-20T18:32:15.300046] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-10-20T18:32:15.310692] [INFO ] [MAIN ] EMA: Starting full primary refresh. [2025-10-20T18:32:15.456822] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-07. [2025-10-20T18:32:15.536850] [INFO ] [REMOTE ] ELV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:15.754722] [INFO ] [REMOTE ] EMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:15.966285] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:32:16.251371] [INFO ] [REMOTE ] EMA: Fetching primary data for 2025-01. [2025-10-20T18:32:16.461264] [DEBUG] [REMOTE ] EMA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:32:16.461601] [WARN ] [REMOTE ] EMA: No data returned for month 2025-01 [2025-10-20T18:32:16.461711] [FATAL] [REMOTE ] EMA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:32:16.461762] [FATAL] [MAIN ] EMA: Failed to fetch primary data. [2025-10-20T18:32:16.461823] [WARN ] [MAIN ] EMA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:32:16.461895] [INFO ] [LOCAL ] EMA: Starting deletion of primary data. [2025-10-20T18:32:16.461958] [DEBUG] [LOCAL ] EMA: No primary data file found to delete. [2025-10-20T18:32:16.461993] [INFO ] [LOCAL ] EMA: Starting deletion of derivative data. [2025-10-20T18:32:16.462032] [DEBUG] [LOCAL ] EMA: No derivative data file found to delete. [2025-10-20T18:32:16.462075] [INFO ] [MAIN ] Update Progress: 725/2367 symbols completed. EMA took 1.6s. Estimated remaining: 3h 28m 49s. Rate: 7.9 symbols/min [2025-10-20T18:32:16.633189] [INFO ] [MAIN ] EMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:16.654716] [DEBUG] [LOCAL ] EMD: Loaded 21209 existing records from primary storage. [2025-10-20T18:32:16.654967] [INFO ] [MAIN ] EMD: Loaded 21209 existing records. [2025-10-20T18:32:16.830199] [WARN ] [MAIN ] EMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:16.830481] [INFO ] [MAIN ] EMD: Wiping existing primary data before full refresh. [2025-10-20T18:32:16.830528] [INFO ] [LOCAL ] EMD: Starting deletion of primary data. [2025-10-20T18:32:16.830783] [DEBUG] [LOCAL ] EMD: Deleted primary data file. [2025-10-20T18:32:16.830839] [INFO ] [MAIN ] EMD: Starting full primary refresh. [2025-10-20T18:32:17.051455] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-01. [2025-10-20T18:32:17.238613] [INFO ] [REMOTE ] EMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:17.851384] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-01. [2025-10-20T18:32:18.651422] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-08. [2025-10-20T18:32:18.777190] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:32:18.986612] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:32:19.035477] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:32:19.451377] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-02. [2025-10-20T18:32:20.251404] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-02. [2025-10-20T18:32:20.391890] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T18:32:21.051425] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-03. [2025-10-20T18:32:21.851353] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-02. [2025-10-20T18:32:21.884725] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:32:22.192868] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:32:22.651366] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-04. [2025-10-20T18:32:23.451358] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-09. [2025-10-20T18:32:23.530625] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:32:24.082584] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:32:24.111925] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:32:24.251383] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-05. [2025-10-20T18:32:25.051379] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-03. [2025-10-20T18:32:25.410846] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T18:32:25.851410] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-06. [2025-10-20T18:32:26.651373] [INFO ] [REMOTE ] ELPC: Fetching primary data for 2025-10. [2025-10-20T18:32:26.801600] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:32:26.865823] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T18:32:27.010101] [INFO ] [REMOTE ] ELPC: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T18:32:27.011280] [INFO ] [REMOTE ] ELPC: Full fetch complete (1395 records). [2025-10-20T18:32:27.011390] [DEBUG] [VALIDATOR ] ELPC: Validating primary data. [2025-10-20T18:32:27.011968] [FATAL] [VALIDATOR ] ELPC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:32:27.012076] [FATAL] [MAIN ] ELPC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:32:27.012139] [WARN ] [MAIN ] ELPC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:32:27.012208] [INFO ] [LOCAL ] ELPC: Starting deletion of primary data. [2025-10-20T18:32:27.012337] [DEBUG] [LOCAL ] ELPC: No primary data file found to delete. [2025-10-20T18:32:27.012409] [INFO ] [LOCAL ] ELPC: Starting deletion of derivative data. [2025-10-20T18:32:27.012504] [DEBUG] [LOCAL ] ELPC: No derivative data file found to delete. [2025-10-20T18:32:27.012770] [INFO ] [MAIN ] Update Progress: 726/2367 symbols completed. ELPC took 28.9s. Estimated remaining: 3h 28m 48s. Rate: 7.9 symbols/min [2025-10-20T18:32:27.209614] [INFO ] [MAIN ] EME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:27.209882] [DEBUG] [LOCAL ] EME: No existing data file found in primary storage. [2025-10-20T18:32:27.209926] [WARN ] [MAIN ] EME: No existing data found, performing full refresh instead. [2025-10-20T18:32:27.210058] [INFO ] [MAIN ] EME: Wiping existing primary data before full refresh. [2025-10-20T18:32:27.210101] [INFO ] [LOCAL ] EME: Starting deletion of primary data. [2025-10-20T18:32:27.210144] [DEBUG] [LOCAL ] EME: No primary data file found to delete. [2025-10-20T18:32:27.210207] [INFO ] [MAIN ] EME: Starting full primary refresh. [2025-10-20T18:32:27.456635] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-07. [2025-10-20T18:32:27.652986] [INFO ] [REMOTE ] EME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:28.251363] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-01. [2025-10-20T18:32:28.610225] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T18:32:29.051435] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-04. [2025-10-20T18:32:29.448566] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:32:29.851423] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-03. [2025-10-20T18:32:30.559996] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:32:30.651420] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-02. [2025-10-20T18:32:31.347371] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:32:31.451336] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-04. [2025-10-20T18:32:32.159799] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:32:32.251418] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-08. [2025-10-20T18:32:33.051370] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-05. [2025-10-20T18:32:33.217858] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T18:32:33.268621] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:32:33.851417] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-03. [2025-10-20T18:32:34.655307] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-09. [2025-10-20T18:32:34.671441] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:32:35.460632] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-06. [2025-10-20T18:32:35.465488] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T18:32:35.821901] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:32:36.251427] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-05. [2025-10-20T18:32:36.980371] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T18:32:37.051359] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-07. [2025-10-20T18:32:37.851481] [INFO ] [REMOTE ] EMD: Fetching primary data for 2025-10. [2025-10-20T18:32:38.238685] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T18:32:38.651449] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-04. [2025-10-20T18:32:38.751195] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T18:32:39.055473] [INFO ] [REMOTE ] EMD: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:32:39.059697] [INFO ] [REMOTE ] EMD: Full fetch complete (31975 records). [2025-10-20T18:32:39.059815] [DEBUG] [VALIDATOR ] EMD: Validating primary data. [2025-10-20T18:32:39.451798] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-06. [2025-10-20T18:32:39.452338] [DEBUG] [VALIDATOR ] EMD: All expected trading days are present. [2025-10-20T18:32:39.453474] [DEBUG] [VALIDATOR ] EMD: Primary data validation passed. [2025-10-20T18:32:39.454541] [INFO ] [MAIN ] EMD: Fetched primary data is valid. [2025-10-20T18:32:39.454753] [DEBUG] [LOCAL ] EMD: Saving primary data. [2025-10-20T18:32:39.455944] [DEBUG] [LOCAL ] EMD: Trimmed 31975 records to 20926 (keeping 200 days of data) [2025-10-20T18:32:39.589335] [DEBUG] [LOCAL ] EMD: Primary data saved (20926 records). [2025-10-20T18:32:39.589528] [INFO ] [MAIN ] EMD: Full history saved. [2025-10-20T18:32:39.589682] [INFO ] [MAIN ] Update Progress: 727/2367 symbols completed. EMD took 23.1s. Estimated remaining: 3h 28m 52s. Rate: 7.9 symbols/min [2025-10-20T18:32:39.781557] [INFO ] [MAIN ] EMF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:39.788866] [DEBUG] [LOCAL ] EMF: Loaded 3246 existing records from primary storage. [2025-10-20T18:32:39.789181] [INFO ] [MAIN ] EMF: Loaded 3246 existing records. [2025-10-20T18:32:39.982008] [WARN ] [MAIN ] EMF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:39.982381] [INFO ] [MAIN ] EMF: Wiping existing primary data before full refresh. [2025-10-20T18:32:39.982439] [INFO ] [LOCAL ] EMF: Starting deletion of primary data. [2025-10-20T18:32:39.982601] [DEBUG] [LOCAL ] EMF: Deleted primary data file. [2025-10-20T18:32:39.982693] [INFO ] [MAIN ] EMF: Starting full primary refresh. [2025-10-20T18:32:40.256499] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-08. [2025-10-20T18:32:40.400233] [INFO ] [REMOTE ] EMF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:40.423174] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:32:40.996515] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:32:41.051408] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-07. [2025-10-20T18:32:41.851432] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-05. [2025-10-20T18:32:42.019034] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:32:42.651364] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-09. [2025-10-20T18:32:42.980808] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T18:32:43.288900] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:32:43.451399] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-08. [2025-10-20T18:32:44.204991] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T18:32:44.251345] [INFO ] [REMOTE ] ELS: Fetching primary data for 2025-10. [2025-10-20T18:32:45.051359] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-06. [2025-10-20T18:32:45.391382] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T18:32:45.851388] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-01. [2025-10-20T18:32:46.161237] [INFO ] [REMOTE ] ELS: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T18:32:46.168858] [INFO ] [REMOTE ] ELS: Full fetch complete (71263 records). [2025-10-20T18:32:46.169005] [DEBUG] [VALIDATOR ] ELS: Validating primary data. [2025-10-20T18:32:46.483544] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:32:46.574762] [DEBUG] [VALIDATOR ] ELS: All expected trading days are present. [2025-10-20T18:32:46.574978] [DEBUG] [VALIDATOR ] ELS: Primary data validation passed. [2025-10-20T18:32:46.576020] [INFO ] [MAIN ] ELS: Fetched primary data is valid. [2025-10-20T18:32:46.576179] [DEBUG] [LOCAL ] ELS: Saving primary data. [2025-10-20T18:32:46.577712] [DEBUG] [LOCAL ] ELS: Trimmed 71263 records to 49903 (keeping 200 days of data) [2025-10-20T18:32:46.667288] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-02. [2025-10-20T18:32:46.698281] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:32:46.914246] [DEBUG] [LOCAL ] ELS: Primary data saved (49903 records). [2025-10-20T18:32:46.914383] [INFO ] [MAIN ] ELS: Full history saved. [2025-10-20T18:32:46.914462] [INFO ] [MAIN ] Update Progress: 728/2367 symbols completed. ELS took 40.7s. Estimated remaining: 3h 28m 43s. Rate: 7.9 symbols/min [2025-10-20T18:32:47.084519] [INFO ] [MAIN ] EMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:47.133047] [DEBUG] [LOCAL ] EMN: Loaded 51488 existing records from primary storage. [2025-10-20T18:32:47.133355] [INFO ] [MAIN ] EMN: Loaded 51488 existing records. [2025-10-20T18:32:47.309406] [WARN ] [MAIN ] EMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:47.309679] [INFO ] [MAIN ] EMN: Wiping existing primary data before full refresh. [2025-10-20T18:32:47.309738] [INFO ] [LOCAL ] EMN: Starting deletion of primary data. [2025-10-20T18:32:47.309920] [DEBUG] [LOCAL ] EMN: Deleted primary data file. [2025-10-20T18:32:47.309958] [INFO ] [MAIN ] EMN: Starting full primary refresh. [2025-10-20T18:32:47.455481] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-09. [2025-10-20T18:32:47.672281] [INFO ] [REMOTE ] EMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:47.868134] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:32:48.251361] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-01. [2025-10-20T18:32:49.051430] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-07. [2025-10-20T18:32:49.851431] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-03. [2025-10-20T18:32:50.056554] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T18:32:50.472460] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:32:50.537291] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-09 (3.08 seconds). [2025-10-20T18:32:50.651376] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-04. [2025-10-20T18:32:51.210629] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T18:32:51.342772] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T18:32:51.451390] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-08. [2025-10-20T18:32:52.251371] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-05. [2025-10-20T18:32:52.746441] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-10-20T18:32:53.051360] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-02. [2025-10-20T18:32:53.211472] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:32:53.851414] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-09. [2025-10-20T18:32:54.566393] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:32:54.651393] [INFO ] [REMOTE ] ELV: Fetching primary data for 2025-10. [2025-10-20T18:32:55.451392] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-03. [2025-10-20T18:32:55.659816] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:32:56.251459] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-06. [2025-10-20T18:32:56.487512] [INFO ] [REMOTE ] ELV: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:32:56.495489] [INFO ] [REMOTE ] ELV: Full fetch complete (73297 records). [2025-10-20T18:32:56.495741] [DEBUG] [VALIDATOR ] ELV: Validating primary data. [2025-10-20T18:32:56.916473] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:32:57.020947] [DEBUG] [VALIDATOR ] ELV: All expected trading days are present. [2025-10-20T18:32:57.021022] [DEBUG] [VALIDATOR ] ELV: Primary data validation passed. [2025-10-20T18:32:57.021882] [INFO ] [MAIN ] ELV: Fetched primary data is valid. [2025-10-20T18:32:57.021944] [DEBUG] [LOCAL ] ELV: Saving primary data. [2025-10-20T18:32:57.023252] [DEBUG] [LOCAL ] ELV: Trimmed 73297 records to 50756 (keeping 200 days of data) [2025-10-20T18:32:57.078115] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-07. [2025-10-20T18:32:57.359634] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:32:57.381722] [DEBUG] [LOCAL ] ELV: Primary data saved (50756 records). [2025-10-20T18:32:57.381858] [INFO ] [MAIN ] ELV: Full history saved. [2025-10-20T18:32:57.381938] [INFO ] [MAIN ] Update Progress: 729/2367 symbols completed. ELV took 43.0s. Estimated remaining: 3h 28m 42s. Rate: 7.8 symbols/min [2025-10-20T18:32:57.491928] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-10-20T18:32:57.557532] [INFO ] [MAIN ] EMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:32:57.571887] [DEBUG] [LOCAL ] EMO: Loaded 13445 existing records from primary storage. [2025-10-20T18:32:57.572118] [INFO ] [MAIN ] EMO: Loaded 13445 existing records. [2025-10-20T18:32:57.749298] [WARN ] [MAIN ] EMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:32:57.749583] [INFO ] [MAIN ] EMO: Wiping existing primary data before full refresh. [2025-10-20T18:32:57.749638] [INFO ] [LOCAL ] EMO: Starting deletion of primary data. [2025-10-20T18:32:57.749880] [DEBUG] [LOCAL ] EMO: Deleted primary data file. [2025-10-20T18:32:57.749939] [INFO ] [MAIN ] EMO: Starting full primary refresh. [2025-10-20T18:32:57.856689] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-04. [2025-10-20T18:32:58.099787] [INFO ] [REMOTE ] EMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:32:58.651388] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-08. [2025-10-20T18:32:59.130018] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:32:59.451394] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-09. [2025-10-20T18:32:59.610841] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:33:00.081818] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T18:33:00.251411] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-01. [2025-10-20T18:33:01.051392] [INFO ] [REMOTE ] EME: Fetching primary data for 2025-10. [2025-10-20T18:33:01.331909] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T18:33:01.851461] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-05. [2025-10-20T18:33:02.651362] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-02. [2025-10-20T18:33:02.790782] [INFO ] [REMOTE ] EME: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:33:02.797127] [INFO ] [REMOTE ] EME: Full fetch complete (47081 records). [2025-10-20T18:33:02.797372] [DEBUG] [VALIDATOR ] EME: Validating primary data. [2025-10-20T18:33:03.227962] [DEBUG] [VALIDATOR ] EME: All expected trading days are present. [2025-10-20T18:33:03.228047] [DEBUG] [VALIDATOR ] EME: Primary data validation passed. [2025-10-20T18:33:03.228755] [INFO ] [MAIN ] EME: Fetched primary data is valid. [2025-10-20T18:33:03.228876] [DEBUG] [LOCAL ] EME: Saving primary data. [2025-10-20T18:33:03.229996] [DEBUG] [LOCAL ] EME: Trimmed 47081 records to 30809 (keeping 200 days of data) [2025-10-20T18:33:03.460027] [INFO ] [REMOTE ] EMF: Fetching primary data for 2025-10. [2025-10-20T18:33:03.488844] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T18:33:03.492772] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:33:03.491013] [DEBUG] [LOCAL ] EME: Primary data saved (30809 records). [2025-10-20T18:33:03.505129] [INFO ] [MAIN ] EME: Full history saved. [2025-10-20T18:33:03.506389] [INFO ] [MAIN ] Update Progress: 730/2367 symbols completed. EME took 36.5s. Estimated remaining: 3h 28m 31s. Rate: 7.9 symbols/min [2025-10-20T18:33:03.676164] [INFO ] [MAIN ] EMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:03.681544] [DEBUG] [LOCAL ] EMP: Loaded 3057 existing records from primary storage. [2025-10-20T18:33:03.681713] [INFO ] [MAIN ] EMP: Loaded 3057 existing records. [2025-10-20T18:33:03.857231] [WARN ] [MAIN ] EMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:03.857439] [INFO ] [MAIN ] EMP: Wiping existing primary data before full refresh. [2025-10-20T18:33:03.857495] [INFO ] [LOCAL ] EMP: Starting deletion of primary data. [2025-10-20T18:33:03.857806] [DEBUG] [LOCAL ] EMP: Deleted primary data file. [2025-10-20T18:33:03.857863] [INFO ] [MAIN ] EMP: Starting full primary refresh. [2025-10-20T18:33:04.203881] [INFO ] [REMOTE ] EMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:04.251351] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-01. [2025-10-20T18:33:04.256754] [INFO ] [REMOTE ] EMF: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T18:33:04.258153] [INFO ] [REMOTE ] EMF: Full fetch complete (6225 records). [2025-10-20T18:33:04.258225] [DEBUG] [VALIDATOR ] EMF: Validating primary data. [2025-10-20T18:33:04.634723] [DEBUG] [VALIDATOR ] EMF: All expected trading days are present. [2025-10-20T18:33:04.634845] [DEBUG] [VALIDATOR ] EMF: Primary data validation passed. [2025-10-20T18:33:04.635069] [INFO ] [MAIN ] EMF: Fetched primary data is valid. [2025-10-20T18:33:04.635109] [DEBUG] [LOCAL ] EMF: Saving primary data. [2025-10-20T18:33:04.635493] [DEBUG] [LOCAL ] EMF: Trimmed 6225 records to 3172 (keeping 200 days of data) [2025-10-20T18:33:04.654380] [DEBUG] [LOCAL ] EMF: Primary data saved (3172 records). [2025-10-20T18:33:04.654441] [INFO ] [MAIN ] EMF: Full history saved. [2025-10-20T18:33:04.654509] [INFO ] [MAIN ] Update Progress: 731/2367 symbols completed. EMF took 25.1s. Estimated remaining: 3h 28m 9s. Rate: 7.9 symbols/min [2025-10-20T18:33:04.821571] [INFO ] [MAIN ] EMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:04.873878] [DEBUG] [LOCAL ] EMR: Loaded 53972 existing records from primary storage. [2025-10-20T18:33:04.874186] [INFO ] [MAIN ] EMR: Loaded 53972 existing records. [2025-10-20T18:33:04.966423] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:33:05.051658] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-06. [2025-10-20T18:33:05.051564] [WARN ] [MAIN ] EMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:05.051881] [INFO ] [MAIN ] EMR: Wiping existing primary data before full refresh. [2025-10-20T18:33:05.052628] [INFO ] [LOCAL ] EMR: Starting deletion of primary data. [2025-10-20T18:33:05.052892] [DEBUG] [LOCAL ] EMR: Deleted primary data file. [2025-10-20T18:33:05.052948] [INFO ] [MAIN ] EMR: Starting full primary refresh. [2025-10-20T18:33:05.399874] [INFO ] [REMOTE ] EMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:05.851383] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-02. [2025-10-20T18:33:06.387107] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:33:06.656454] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-01. [2025-10-20T18:33:06.675828] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:33:07.451430] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-03. [2025-10-20T18:33:08.240257] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T18:33:08.251326] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-04. [2025-10-20T18:33:08.496272] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:33:09.051345] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-02. [2025-10-20T18:33:09.222070] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:33:09.851386] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-07. [2025-10-20T18:33:10.651411] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-03. [2025-10-20T18:33:10.942530] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:33:10.974834] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-03 (0.32 seconds). [2025-10-20T18:33:11.422562] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:33:11.451337] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-08. [2025-10-20T18:33:12.251379] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-04. [2025-10-20T18:33:12.645718] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-10-20T18:33:13.051367] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-05. [2025-10-20T18:33:13.219251] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:33:13.520025] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T18:33:13.851409] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-06. [2025-10-20T18:33:14.324193] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T18:33:14.651379] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-05. [2025-10-20T18:33:15.451380] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-07. [2025-10-20T18:33:15.571753] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T18:33:15.915710] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:33:16.251382] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-03. [2025-10-20T18:33:17.051376] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-09. [2025-10-20T18:33:17.851367] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-06. [2025-10-20T18:33:18.042539] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T18:33:18.651364] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-08. [2025-10-20T18:33:18.711167] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T18:33:18.781578] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T18:33:19.167012] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T18:33:19.451369] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-07. [2025-10-20T18:33:20.251370] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-04. [2025-10-20T18:33:20.437955] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:33:21.051377] [INFO ] [REMOTE ] EMN: Fetching primary data for 2025-10. [2025-10-20T18:33:21.851368] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-09. [2025-10-20T18:33:22.212266] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-09 (0.36 seconds). [2025-10-20T18:33:22.212114] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T18:33:22.651394] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-08. [2025-10-20T18:33:23.076084] [INFO ] [REMOTE ] EMN: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T18:33:23.083905] [INFO ] [REMOTE ] EMN: Full fetch complete (72393 records). [2025-10-20T18:33:23.084037] [DEBUG] [VALIDATOR ] EMN: Validating primary data. [2025-10-20T18:33:23.428512] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:33:23.463899] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-05. [2025-10-20T18:33:23.525327] [DEBUG] [VALIDATOR ] EMN: All expected trading days are present. [2025-10-20T18:33:23.525417] [DEBUG] [VALIDATOR ] EMN: Primary data validation passed. [2025-10-20T18:33:23.526489] [INFO ] [MAIN ] EMN: Fetched primary data is valid. [2025-10-20T18:33:23.526557] [DEBUG] [LOCAL ] EMN: Saving primary data. [2025-10-20T18:33:23.528284] [DEBUG] [LOCAL ] EMN: Trimmed 72393 records to 50838 (keeping 200 days of data) [2025-10-20T18:33:23.841845] [DEBUG] [LOCAL ] EMN: Primary data saved (50838 records). [2025-10-20T18:33:23.841989] [INFO ] [MAIN ] EMN: Full history saved. [2025-10-20T18:33:23.842070] [INFO ] [MAIN ] Update Progress: 732/2367 symbols completed. EMN took 36.9s. Estimated remaining: 3h 28m 27s. Rate: 7.8 symbols/min [2025-10-20T18:33:24.019617] [INFO ] [MAIN ] ENB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:24.075525] [DEBUG] [LOCAL ] ENB: Loaded 54292 existing records from primary storage. [2025-10-20T18:33:24.075917] [INFO ] [MAIN ] ENB: Loaded 54292 existing records. [2025-10-20T18:33:24.253702] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-09. [2025-10-20T18:33:24.253542] [WARN ] [MAIN ] ENB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:24.254215] [INFO ] [MAIN ] ENB: Wiping existing primary data before full refresh. [2025-10-20T18:33:24.255269] [INFO ] [LOCAL ] ENB: Starting deletion of primary data. [2025-10-20T18:33:24.255476] [DEBUG] [LOCAL ] ENB: Deleted primary data file. [2025-10-20T18:33:24.255517] [INFO ] [MAIN ] ENB: Starting full primary refresh. [2025-10-20T18:33:24.602768] [INFO ] [REMOTE ] ENB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:25.051372] [INFO ] [REMOTE ] EMP: Fetching primary data for 2025-10. [2025-10-20T18:33:25.292797] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:33:25.582376] [INFO ] [REMOTE ] EMP: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T18:33:25.585576] [INFO ] [REMOTE ] EMP: Full fetch complete (4089 records). [2025-10-20T18:33:25.584423] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:33:25.585794] [DEBUG] [VALIDATOR ] EMP: Validating primary data. [2025-10-20T18:33:25.856597] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-06. [2025-10-20T18:33:25.957903] [DEBUG] [VALIDATOR ] EMP: All expected trading days are present. [2025-10-20T18:33:25.957983] [DEBUG] [VALIDATOR ] EMP: Primary data validation passed. [2025-10-20T18:33:25.958124] [INFO ] [MAIN ] EMP: Fetched primary data is valid. [2025-10-20T18:33:25.958162] [DEBUG] [LOCAL ] EMP: Saving primary data. [2025-10-20T18:33:25.958534] [DEBUG] [LOCAL ] EMP: Trimmed 4089 records to 2945 (keeping 200 days of data) [2025-10-20T18:33:25.977617] [DEBUG] [LOCAL ] EMP: Primary data saved (2945 records). [2025-10-20T18:33:25.977773] [INFO ] [MAIN ] EMP: Full history saved. [2025-10-20T18:33:25.977856] [INFO ] [MAIN ] Update Progress: 733/2367 symbols completed. EMP took 22.5s. Estimated remaining: 3h 28m 7s. Rate: 7.9 symbols/min [2025-10-20T18:33:26.148207] [INFO ] [MAIN ] ENIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:26.184818] [DEBUG] [LOCAL ] ENIC: Loaded 37209 existing records from primary storage. [2025-10-20T18:33:26.185175] [INFO ] [MAIN ] ENIC: Loaded 37209 existing records. [2025-10-20T18:33:26.384389] [WARN ] [MAIN ] ENIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:26.384619] [INFO ] [MAIN ] ENIC: Wiping existing primary data before full refresh. [2025-10-20T18:33:26.384701] [INFO ] [LOCAL ] ENIC: Starting deletion of primary data. [2025-10-20T18:33:26.384932] [DEBUG] [LOCAL ] ENIC: Deleted primary data file. [2025-10-20T18:33:26.384988] [INFO ] [MAIN ] ENIC: Starting full primary refresh. [2025-10-20T18:33:26.657098] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-01. [2025-10-20T18:33:26.784142] [INFO ] [REMOTE ] ENIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:27.456742] [INFO ] [REMOTE ] EMO: Fetching primary data for 2025-10. [2025-10-20T18:33:27.472416] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:33:28.251409] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-01. [2025-10-20T18:33:28.476716] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T18:33:28.508922] [INFO ] [REMOTE ] EMO: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:33:28.511846] [INFO ] [REMOTE ] EMO: Full fetch complete (17669 records). [2025-10-20T18:33:28.511935] [DEBUG] [VALIDATOR ] EMO: Validating primary data. [2025-10-20T18:33:28.882431] [DEBUG] [VALIDATOR ] EMO: All expected trading days are present. [2025-10-20T18:33:28.882539] [DEBUG] [VALIDATOR ] EMO: Primary data validation passed. [2025-10-20T18:33:28.882947] [INFO ] [MAIN ] EMO: Fetched primary data is valid. [2025-10-20T18:33:28.883007] [DEBUG] [LOCAL ] EMO: Saving primary data. [2025-10-20T18:33:28.883621] [DEBUG] [LOCAL ] EMO: Trimmed 17669 records to 13342 (keeping 200 days of data) [2025-10-20T18:33:28.964849] [DEBUG] [LOCAL ] EMO: Primary data saved (13342 records). [2025-10-20T18:33:28.964998] [INFO ] [MAIN ] EMO: Full history saved. [2025-10-20T18:33:28.965079] [INFO ] [MAIN ] Update Progress: 734/2367 symbols completed. EMO took 31.6s. Estimated remaining: 3h 27m 49s. Rate: 7.9 symbols/min [2025-10-20T18:33:29.061275] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-07. [2025-10-20T18:33:29.135030] [INFO ] [MAIN ] ENO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:29.139191] [DEBUG] [LOCAL ] ENO: Loaded 1872 existing records from primary storage. [2025-10-20T18:33:29.139346] [INFO ] [MAIN ] ENO: Loaded 1872 existing records. [2025-10-20T18:33:29.313596] [WARN ] [MAIN ] ENO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:29.313817] [INFO ] [MAIN ] ENO: Wiping existing primary data before full refresh. [2025-10-20T18:33:29.313863] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-10-20T18:33:29.314046] [DEBUG] [LOCAL ] ENO: Deleted primary data file. [2025-10-20T18:33:29.314089] [INFO ] [MAIN ] ENO: Starting full primary refresh. [2025-10-20T18:33:29.703091] [INFO ] [REMOTE ] ENO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:29.726508] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:33:29.851359] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-01. [2025-10-20T18:33:30.367871] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T18:33:30.651413] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-02. [2025-10-20T18:33:31.016523] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T18:33:31.451409] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-02. [2025-10-20T18:33:31.920333] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:33:31.932614] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:33:32.251417] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-03. [2025-10-20T18:33:32.657323] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T18:33:33.051373] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-02. [2025-10-20T18:33:33.851388] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-03. [2025-10-20T18:33:34.651407] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-04. [2025-10-20T18:33:34.772802] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:33:35.187715] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:33:35.294955] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:33:35.451346] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-04. [2025-10-20T18:33:36.251357] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-08. [2025-10-20T18:33:36.882824] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:33:37.051590] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-05. [2025-10-20T18:33:37.488633] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T18:33:37.851378] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-05. [2025-10-20T18:33:38.152508] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:33:38.651352] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-06. [2025-10-20T18:33:39.079427] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T18:33:39.314146] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:33:39.451374] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-07. [2025-10-20T18:33:39.883057] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:33:40.251378] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-03. [2025-10-20T18:33:41.051410] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-08. [2025-10-20T18:33:41.512345] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:33:41.851376] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-09. [2025-10-20T18:33:42.328519] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T18:33:42.651448] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-06. [2025-10-20T18:33:43.451442] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-09. [2025-10-20T18:33:43.733215] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T18:33:43.935699] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T18:33:44.202040] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:33:44.251369] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-07. [2025-10-20T18:33:45.051397] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-04. [2025-10-20T18:33:45.851398] [INFO ] [REMOTE ] ENO: Fetching primary data for 2025-10. [2025-10-20T18:33:45.892154] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:33:46.520353] [INFO ] [REMOTE ] ENO: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T18:33:46.521177] [INFO ] [REMOTE ] ENO: Full fetch complete (2989 records). [2025-10-20T18:33:46.521287] [DEBUG] [VALIDATOR ] ENO: Validating primary data. [2025-10-20T18:33:46.521621] [FATAL] [VALIDATOR ] ENO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:33:46.521723] [FATAL] [MAIN ] ENO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:33:46.521790] [WARN ] [MAIN ] ENO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:33:46.521862] [INFO ] [LOCAL ] ENO: Starting deletion of primary data. [2025-10-20T18:33:46.521926] [DEBUG] [LOCAL ] ENO: No primary data file found to delete. [2025-10-20T18:33:46.521962] [INFO ] [LOCAL ] ENO: Starting deletion of derivative data. [2025-10-20T18:33:46.522090] [DEBUG] [LOCAL ] ENO: Deleted derivative data file. [2025-10-20T18:33:46.522157] [INFO ] [MAIN ] Update Progress: 735/2367 symbols completed. ENO took 17.6s. Estimated remaining: 3h 28m 3s. Rate: 7.8 symbols/min [2025-10-20T18:33:46.661438] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-08. [2025-10-20T18:33:46.696797] [INFO ] [MAIN ] ENOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:46.737998] [DEBUG] [LOCAL ] ENOV: Loaded 45385 existing records from primary storage. [2025-10-20T18:33:46.738337] [INFO ] [MAIN ] ENOV: Loaded 45385 existing records. [2025-10-20T18:33:46.919777] [WARN ] [MAIN ] ENOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:46.919984] [INFO ] [MAIN ] ENOV: Wiping existing primary data before full refresh. [2025-10-20T18:33:46.920029] [INFO ] [LOCAL ] ENOV: Starting deletion of primary data. [2025-10-20T18:33:46.920240] [DEBUG] [LOCAL ] ENOV: Deleted primary data file. [2025-10-20T18:33:46.920296] [INFO ] [MAIN ] ENOV: Starting full primary refresh. [2025-10-20T18:33:47.349247] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-10-20T18:33:47.355153] [INFO ] [REMOTE ] ENOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:47.451421] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-05. [2025-10-20T18:33:48.108083] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:33:48.251390] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-09. [2025-10-20T18:33:49.051513] [INFO ] [REMOTE ] EMR: Fetching primary data for 2025-10. [2025-10-20T18:33:49.227500] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T18:33:49.750249] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:33:49.851348] [INFO ] [REMOTE ] ENIC: Fetching primary data for 2025-10. [2025-10-20T18:33:50.651424] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-01. [2025-10-20T18:33:51.327020] [INFO ] [REMOTE ] ENIC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:33:51.332163] [INFO ] [REMOTE ] ENIC: Full fetch complete (51383 records). [2025-10-20T18:33:51.332291] [DEBUG] [VALIDATOR ] ENIC: Validating primary data. [2025-10-20T18:33:51.455842] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-06. [2025-10-20T18:33:51.741204] [INFO ] [REMOTE ] EMR: Incremental fetch complete for 2025-10 (2.69 seconds). [2025-10-20T18:33:51.766903] [DEBUG] [VALIDATOR ] ENIC: All expected trading days are present. [2025-10-20T18:33:51.767179] [DEBUG] [VALIDATOR ] ENIC: Primary data validation passed. [2025-10-20T18:33:51.768029] [INFO ] [MAIN ] ENIC: Fetched primary data is valid. [2025-10-20T18:33:51.768116] [DEBUG] [LOCAL ] ENIC: Saving primary data. [2025-10-20T18:33:51.768441] [INFO ] [REMOTE ] EMR: Full fetch complete (77223 records). [2025-10-20T18:33:51.769190] [DEBUG] [VALIDATOR ] EMR: Validating primary data. [2025-10-20T18:33:51.769987] [DEBUG] [LOCAL ] ENIC: Trimmed 51383 records to 36571 (keeping 200 days of data) [2025-10-20T18:33:52.367366] [DEBUG] [LOCAL ] ENIC: Primary data saved (36571 records). [2025-10-20T18:33:52.390419] [INFO ] [MAIN ] ENIC: Full history saved. [2025-10-20T18:33:52.430687] [DEBUG] [VALIDATOR ] EMR: All expected trading days are present. [2025-10-20T18:33:52.430919] [INFO ] [MAIN ] Update Progress: 736/2367 symbols completed. ENIC took 26.5s. Estimated remaining: 3h 27m 52s. Rate: 7.8 symbols/min [2025-10-20T18:33:52.431149] [DEBUG] [VALIDATOR ] EMR: Primary data validation passed. [2025-10-20T18:33:52.437571] [INFO ] [MAIN ] EMR: Fetched primary data is valid. [2025-10-20T18:33:52.463944] [DEBUG] [LOCAL ] EMR: Saving primary data. [2025-10-20T18:33:52.545732] [DEBUG] [LOCAL ] EMR: Trimmed 77223 records to 53176 (keeping 200 days of data) [2025-10-20T18:33:52.637692] [INFO ] [MAIN ] ENR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:52.640556] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T18:33:52.707061] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-02. [2025-10-20T18:33:52.813324] [DEBUG] [LOCAL ] ENR: Loaded 44323 existing records from primary storage. [2025-10-20T18:33:52.816118] [INFO ] [MAIN ] ENR: Loaded 44323 existing records. [2025-10-20T18:33:53.106477] [WARN ] [MAIN ] ENR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:33:53.109119] [INFO ] [MAIN ] ENR: Wiping existing primary data before full refresh. [2025-10-20T18:33:53.109468] [INFO ] [LOCAL ] ENR: Starting deletion of primary data. [2025-10-20T18:33:53.110349] [DEBUG] [LOCAL ] ENR: Deleted primary data file. [2025-10-20T18:33:53.111214] [INFO ] [MAIN ] ENR: Starting full primary refresh. [2025-10-20T18:33:53.572463] [INFO ] [REMOTE ] ENR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:53.573398] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-01. [2025-10-20T18:33:53.611282] [DEBUG] [LOCAL ] EMR: Primary data saved (53176 records). [2025-10-20T18:33:53.612144] [INFO ] [MAIN ] EMR: Full history saved. [2025-10-20T18:33:53.613326] [INFO ] [MAIN ] Update Progress: 737/2367 symbols completed. EMR took 49.0s. Estimated remaining: 3h 27m 30s. Rate: 7.9 symbols/min [2025-10-20T18:33:53.727173] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-06 (2.27 seconds). [2025-10-20T18:33:53.868866] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-07. [2025-10-20T18:33:53.870871] [INFO ] [MAIN ] ENS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:33:53.872154] [DEBUG] [LOCAL ] ENS: No existing data file found in primary storage. [2025-10-20T18:33:53.872217] [WARN ] [MAIN ] ENS: No existing data found, performing full refresh instead. [2025-10-20T18:33:53.872347] [INFO ] [MAIN ] ENS: Wiping existing primary data before full refresh. [2025-10-20T18:33:53.872391] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-10-20T18:33:53.872434] [DEBUG] [LOCAL ] ENS: No primary data file found to delete. [2025-10-20T18:33:53.872469] [INFO ] [MAIN ] ENS: Starting full primary refresh. [2025-10-20T18:33:54.094185] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T18:33:54.284896] [INFO ] [REMOTE ] ENS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:33:54.651410] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-01. [2025-10-20T18:33:55.082058] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T18:33:55.451380] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-03. [2025-10-20T18:33:55.986211] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:33:56.198881] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:33:56.251441] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-08. [2025-10-20T18:33:57.051381] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-02. [2025-10-20T18:33:57.119129] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T18:33:57.851410] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-04. [2025-10-20T18:33:58.139396] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:33:58.477249] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:33:58.651356] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-03. [2025-10-20T18:33:59.109104] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:33:59.451443] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-09. [2025-10-20T18:34:00.075792] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:34:00.251412] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-04. [2025-10-20T18:34:01.051362] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-05. [2025-10-20T18:34:01.397299] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T18:34:01.808556] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T18:34:01.851367] [INFO ] [REMOTE ] ENB: Fetching primary data for 2025-10. [2025-10-20T18:34:02.656787] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-05. [2025-10-20T18:34:02.673994] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:34:03.451341] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-02. [2025-10-20T18:34:04.059543] [INFO ] [REMOTE ] ENB: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T18:34:04.067820] [INFO ] [REMOTE ] ENB: Full fetch complete (77656 records). [2025-10-20T18:34:04.067972] [DEBUG] [VALIDATOR ] ENB: Validating primary data. [2025-10-20T18:34:04.270844] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-06. [2025-10-20T18:34:04.478926] [DEBUG] [VALIDATOR ] ENB: All expected trading days are present. [2025-10-20T18:34:04.479066] [DEBUG] [VALIDATOR ] ENB: Primary data validation passed. [2025-10-20T18:34:04.480469] [INFO ] [MAIN ] ENB: Fetched primary data is valid. [2025-10-20T18:34:04.480681] [DEBUG] [LOCAL ] ENB: Saving primary data. [2025-10-20T18:34:04.508990] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:34:04.518610] [DEBUG] [LOCAL ] ENB: Trimmed 77656 records to 53511 (keeping 200 days of data) [2025-10-20T18:34:04.639064] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T18:34:04.892225] [DEBUG] [LOCAL ] ENB: Primary data saved (53511 records). [2025-10-20T18:34:04.892425] [INFO ] [MAIN ] ENB: Full history saved. [2025-10-20T18:34:04.892531] [INFO ] [MAIN ] Update Progress: 738/2367 symbols completed. ENB took 41.1s. Estimated remaining: 3h 27m 30s. Rate: 7.9 symbols/min [2025-10-20T18:34:05.056570] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-06. [2025-10-20T18:34:05.092458] [INFO ] [MAIN ] ENVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:05.121076] [DEBUG] [LOCAL ] ENVA: Loaded 25790 existing records from primary storage. [2025-10-20T18:34:05.121413] [INFO ] [MAIN ] ENVA: Loaded 25790 existing records. [2025-10-20T18:34:05.339423] [WARN ] [MAIN ] ENVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:05.339672] [INFO ] [MAIN ] ENVA: Wiping existing primary data before full refresh. [2025-10-20T18:34:05.339739] [INFO ] [LOCAL ] ENVA: Starting deletion of primary data. [2025-10-20T18:34:05.339996] [DEBUG] [LOCAL ] ENVA: Deleted primary data file. [2025-10-20T18:34:05.340054] [INFO ] [MAIN ] ENVA: Starting full primary refresh. [2025-10-20T18:34:05.756554] [INFO ] [REMOTE ] ENVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:05.851384] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-01. [2025-10-20T18:34:05.894511] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:34:06.651376] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-07. [2025-10-20T18:34:06.950254] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T18:34:07.055712] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:34:07.451441] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-03. [2025-10-20T18:34:08.251378] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-02. [2025-10-20T18:34:08.408158] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:34:08.849053] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:34:09.051406] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-07. [2025-10-20T18:34:09.538168] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:34:09.851403] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-03. [2025-10-20T18:34:10.572577] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:34:10.651365] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-08. [2025-10-20T18:34:10.981934] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:34:11.451372] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-04. [2025-10-20T18:34:12.251357] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-08. [2025-10-20T18:34:12.298224] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:34:12.766170] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T18:34:13.051385] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-09. [2025-10-20T18:34:13.851383] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-05. [2025-10-20T18:34:13.978681] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:34:14.651369] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-09. [2025-10-20T18:34:14.867201] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:34:15.371754] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:34:15.451380] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-06. [2025-10-20T18:34:16.108420] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:34:16.251413] [INFO ] [REMOTE ] ENR: Fetching primary data for 2025-10. [2025-10-20T18:34:17.035878] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:34:17.051347] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-07. [2025-10-20T18:34:17.851364] [INFO ] [REMOTE ] ENOV: Fetching primary data for 2025-10. [2025-10-20T18:34:18.079989] [INFO ] [REMOTE ] ENR: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:34:18.089430] [INFO ] [REMOTE ] ENR: Full fetch complete (59540 records). [2025-10-20T18:34:18.089620] [DEBUG] [VALIDATOR ] ENR: Validating primary data. [2025-10-20T18:34:18.485077] [DEBUG] [VALIDATOR ] ENR: All expected trading days are present. [2025-10-20T18:34:18.491417] [DEBUG] [VALIDATOR ] ENR: Primary data validation passed. [2025-10-20T18:34:18.503562] [INFO ] [MAIN ] ENR: Fetched primary data is valid. [2025-10-20T18:34:18.503703] [DEBUG] [LOCAL ] ENR: Saving primary data. [2025-10-20T18:34:18.505559] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:34:18.513147] [DEBUG] [LOCAL ] ENR: Trimmed 59540 records to 43825 (keeping 200 days of data) [2025-10-20T18:34:18.690936] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-04. [2025-10-20T18:34:18.796027] [DEBUG] [LOCAL ] ENR: Primary data saved (43825 records). [2025-10-20T18:34:18.796202] [INFO ] [MAIN ] ENR: Full history saved. [2025-10-20T18:34:18.796310] [INFO ] [MAIN ] Update Progress: 739/2367 symbols completed. ENR took 26.4s. Estimated remaining: 3h 27m 36s. Rate: 7.8 symbols/min [2025-10-20T18:34:18.976555] [INFO ] [MAIN ] EOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:18.992747] [DEBUG] [LOCAL ] EOD: Loaded 15256 existing records from primary storage. [2025-10-20T18:34:18.992990] [INFO ] [MAIN ] EOD: Loaded 15256 existing records. [2025-10-20T18:34:19.172148] [WARN ] [MAIN ] EOD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:19.172359] [INFO ] [MAIN ] EOD: Wiping existing primary data before full refresh. [2025-10-20T18:34:19.172405] [INFO ] [LOCAL ] EOD: Starting deletion of primary data. [2025-10-20T18:34:19.172573] [DEBUG] [LOCAL ] EOD: Deleted primary data file. [2025-10-20T18:34:19.172608] [INFO ] [MAIN ] EOD: Starting full primary refresh. [2025-10-20T18:34:19.453580] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-08. [2025-10-20T18:34:19.486905] [INFO ] [REMOTE ] ENOV: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T18:34:19.530524] [INFO ] [REMOTE ] ENOV: Full fetch complete (63455 records). [2025-10-20T18:34:19.537670] [DEBUG] [VALIDATOR ] ENOV: Validating primary data. [2025-10-20T18:34:19.557725] [INFO ] [REMOTE ] EOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:19.958406] [DEBUG] [VALIDATOR ] ENOV: All expected trading days are present. [2025-10-20T18:34:19.958484] [DEBUG] [VALIDATOR ] ENOV: Primary data validation passed. [2025-10-20T18:34:19.959290] [INFO ] [MAIN ] ENOV: Fetched primary data is valid. [2025-10-20T18:34:19.959352] [DEBUG] [LOCAL ] ENOV: Saving primary data. [2025-10-20T18:34:19.960552] [DEBUG] [LOCAL ] ENOV: Trimmed 63455 records to 44779 (keeping 200 days of data) [2025-10-20T18:34:20.215035] [DEBUG] [LOCAL ] ENOV: Primary data saved (44779 records). [2025-10-20T18:34:20.215165] [INFO ] [MAIN ] ENOV: Full history saved. [2025-10-20T18:34:20.215244] [INFO ] [MAIN ] Update Progress: 740/2367 symbols completed. ENOV took 33.7s. Estimated remaining: 3h 27m 15s. Rate: 7.9 symbols/min [2025-10-20T18:34:20.283478] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-01. [2025-10-20T18:34:20.338769] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:34:20.415131] [INFO ] [MAIN ] EOG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:20.469399] [DEBUG] [LOCAL ] EOG: Loaded 54152 existing records from primary storage. [2025-10-20T18:34:20.469857] [INFO ] [MAIN ] EOG: Loaded 54152 existing records. [2025-10-20T18:34:20.714108] [WARN ] [MAIN ] EOG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:20.714314] [INFO ] [MAIN ] EOG: Wiping existing primary data before full refresh. [2025-10-20T18:34:20.714359] [INFO ] [LOCAL ] EOG: Starting deletion of primary data. [2025-10-20T18:34:20.714542] [DEBUG] [LOCAL ] EOG: Deleted primary data file. [2025-10-20T18:34:20.714580] [INFO ] [MAIN ] EOG: Starting full primary refresh. [2025-10-20T18:34:21.020124] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:34:21.066796] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-09. [2025-10-20T18:34:21.113776] [INFO ] [REMOTE ] EOG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:21.518947] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:34:21.851373] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-01. [2025-10-20T18:34:22.534207] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T18:34:22.651432] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-05. [2025-10-20T18:34:23.451372] [INFO ] [REMOTE ] ENS: Fetching primary data for 2025-10. [2025-10-20T18:34:23.727812] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T18:34:23.729357] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:34:24.251438] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-02. [2025-10-20T18:34:24.557212] [INFO ] [REMOTE ] ENS: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:34:24.562966] [INFO ] [REMOTE ] ENS: Full fetch complete (42567 records). [2025-10-20T18:34:24.563106] [DEBUG] [VALIDATOR ] ENS: Validating primary data. [2025-10-20T18:34:24.564671] [FATAL] [VALIDATOR ] ENS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:34:24.564887] [FATAL] [MAIN ] ENS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:34:24.564956] [WARN ] [MAIN ] ENS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:34:24.565031] [INFO ] [LOCAL ] ENS: Starting deletion of primary data. [2025-10-20T18:34:24.565103] [DEBUG] [LOCAL ] ENS: No primary data file found to delete. [2025-10-20T18:34:24.565141] [INFO ] [LOCAL ] ENS: Starting deletion of derivative data. [2025-10-20T18:34:24.565180] [DEBUG] [LOCAL ] ENS: No derivative data file found to delete. [2025-10-20T18:34:24.565249] [INFO ] [MAIN ] Update Progress: 741/2367 symbols completed. ENS took 30.9s. Estimated remaining: 3h 27m 0s. Rate: 7.9 symbols/min [2025-10-20T18:34:24.740530] [INFO ] [MAIN ] EOI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:24.752393] [DEBUG] [LOCAL ] EOI: Loaded 10809 existing records from primary storage. [2025-10-20T18:34:24.752632] [INFO ] [MAIN ] EOI: Loaded 10809 existing records. [2025-10-20T18:34:24.950687] [WARN ] [MAIN ] EOI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:24.951112] [INFO ] [MAIN ] EOI: Wiping existing primary data before full refresh. [2025-10-20T18:34:24.951236] [INFO ] [LOCAL ] EOI: Starting deletion of primary data. [2025-10-20T18:34:24.951505] [DEBUG] [LOCAL ] EOI: Deleted primary data file. [2025-10-20T18:34:24.951563] [INFO ] [MAIN ] EOI: Starting full primary refresh. [2025-10-20T18:34:25.052433] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-06. [2025-10-20T18:34:25.204766] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:34:25.333572] [INFO ] [REMOTE ] EOI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:25.851401] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-03. [2025-10-20T18:34:26.234090] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T18:34:26.373118] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T18:34:26.651464] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-04. [2025-10-20T18:34:27.392623] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T18:34:27.451365] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-05. [2025-10-20T18:34:28.252363] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-07. [2025-10-20T18:34:28.255626] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T18:34:29.051407] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-02. [2025-10-20T18:34:29.281166] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:34:29.851380] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-06. [2025-10-20T18:34:30.651357] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-08. [2025-10-20T18:34:30.777739] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T18:34:30.864536] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T18:34:31.451413] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-01. [2025-10-20T18:34:31.627748] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:34:32.251402] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-09. [2025-10-20T18:34:32.391166] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T18:34:33.051350] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-03. [2025-10-20T18:34:33.421583] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:34:33.851382] [INFO ] [REMOTE ] ENVA: Fetching primary data for 2025-10. [2025-10-20T18:34:34.651349] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-07. [2025-10-20T18:34:34.944456] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:34:34.977499] [INFO ] [REMOTE ] ENVA: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T18:34:34.981249] [INFO ] [REMOTE ] ENVA: Full fetch complete (36606 records). [2025-10-20T18:34:34.981324] [DEBUG] [VALIDATOR ] ENVA: Validating primary data. [2025-10-20T18:34:35.349361] [DEBUG] [VALIDATOR ] ENVA: All expected trading days are present. [2025-10-20T18:34:35.349437] [DEBUG] [VALIDATOR ] ENVA: Primary data validation passed. [2025-10-20T18:34:35.350008] [INFO ] [MAIN ] ENVA: Fetched primary data is valid. [2025-10-20T18:34:35.350070] [DEBUG] [LOCAL ] ENVA: Saving primary data. [2025-10-20T18:34:35.350893] [DEBUG] [LOCAL ] ENVA: Trimmed 36606 records to 25413 (keeping 200 days of data) [2025-10-20T18:34:35.454719] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-02. [2025-10-20T18:34:35.501570] [DEBUG] [LOCAL ] ENVA: Primary data saved (25413 records). [2025-10-20T18:34:35.501734] [INFO ] [MAIN ] ENVA: Full history saved. [2025-10-20T18:34:35.501819] [INFO ] [MAIN ] Update Progress: 742/2367 symbols completed. ENVA took 30.6s. Estimated remaining: 3h 26m 60s. Rate: 7.9 symbols/min [2025-10-20T18:34:35.686613] [INFO ] [MAIN ] EOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:35.701335] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:34:35.719585] [DEBUG] [LOCAL ] EOS: Loaded 15294 existing records from primary storage. [2025-10-20T18:34:35.719870] [INFO ] [MAIN ] EOS: Loaded 15294 existing records. [2025-10-20T18:34:35.899017] [WARN ] [MAIN ] EOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:35.899267] [INFO ] [MAIN ] EOS: Wiping existing primary data before full refresh. [2025-10-20T18:34:35.899316] [INFO ] [LOCAL ] EOS: Starting deletion of primary data. [2025-10-20T18:34:35.899504] [DEBUG] [LOCAL ] EOS: Deleted primary data file. [2025-10-20T18:34:35.899559] [INFO ] [MAIN ] EOS: Starting full primary refresh. [2025-10-20T18:34:36.266929] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-04. [2025-10-20T18:34:36.271537] [INFO ] [REMOTE ] EOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:36.316142] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T18:34:37.051394] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-03. [2025-10-20T18:34:37.851348] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-08. [2025-10-20T18:34:37.937037] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:34:38.135489] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:34:38.651400] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-04. [2025-10-20T18:34:38.779874] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:34:39.451335] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-05. [2025-10-20T18:34:39.499102] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:34:40.251419] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-05. [2025-10-20T18:34:41.051409] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-09. [2025-10-20T18:34:41.081993] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:34:41.399471] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T18:34:41.851415] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-01. [2025-10-20T18:34:41.951862] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:34:42.651389] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-06. [2025-10-20T18:34:43.189607] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T18:34:43.451386] [INFO ] [REMOTE ] EOD: Fetching primary data for 2025-10. [2025-10-20T18:34:44.259815] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-02. [2025-10-20T18:34:44.286399] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:34:44.288801] [INFO ] [REMOTE ] EOD: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:34:44.303029] [INFO ] [REMOTE ] EOD: Full fetch complete (20984 records). [2025-10-20T18:34:44.303144] [DEBUG] [VALIDATOR ] EOD: Validating primary data. [2025-10-20T18:34:44.659571] [DEBUG] [VALIDATOR ] EOD: All expected trading days are present. [2025-10-20T18:34:44.659664] [DEBUG] [VALIDATOR ] EOD: Primary data validation passed. [2025-10-20T18:34:44.660099] [INFO ] [MAIN ] EOD: Fetched primary data is valid. [2025-10-20T18:34:44.660138] [DEBUG] [LOCAL ] EOD: Saving primary data. [2025-10-20T18:34:44.660695] [DEBUG] [LOCAL ] EOD: Trimmed 20984 records to 15056 (keeping 200 days of data) [2025-10-20T18:34:44.742510] [DEBUG] [LOCAL ] EOD: Primary data saved (15056 records). [2025-10-20T18:34:44.742631] [INFO ] [MAIN ] EOD: Full history saved. [2025-10-20T18:34:44.742746] [INFO ] [MAIN ] Update Progress: 743/2367 symbols completed. EOD took 25.9s. Estimated remaining: 3h 26m 56s. Rate: 7.8 symbols/min [2025-10-20T18:34:44.912776] [INFO ] [MAIN ] EOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:44.921755] [DEBUG] [LOCAL ] EOT: Loaded 7100 existing records from primary storage. [2025-10-20T18:34:44.921936] [INFO ] [MAIN ] EOT: Loaded 7100 existing records. [2025-10-20T18:34:45.060777] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-07. [2025-10-20T18:34:45.121218] [WARN ] [MAIN ] EOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:45.121481] [INFO ] [MAIN ] EOT: Wiping existing primary data before full refresh. [2025-10-20T18:34:45.121535] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-10-20T18:34:45.121721] [DEBUG] [LOCAL ] EOT: Deleted primary data file. [2025-10-20T18:34:45.121784] [INFO ] [MAIN ] EOT: Starting full primary refresh. [2025-10-20T18:34:45.282821] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:34:45.490482] [INFO ] [REMOTE ] EOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:45.851380] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-01. [2025-10-20T18:34:46.651398] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-03. [2025-10-20T18:34:46.907920] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:34:47.181498] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:34:47.451402] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-02. [2025-10-20T18:34:47.514079] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T18:34:48.114631] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T18:34:48.251382] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-04. [2025-10-20T18:34:49.051352] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-06. [2025-10-20T18:34:49.229726] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T18:34:49.851444] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-05. [2025-10-20T18:34:49.986723] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:34:50.651425] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-08. [2025-10-20T18:34:50.853310] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T18:34:51.451462] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-06. [2025-10-20T18:34:52.251414] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-03. [2025-10-20T18:34:52.345228] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:34:52.457172] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:34:52.655496] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T18:34:53.051416] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-09. [2025-10-20T18:34:53.851394] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-04. [2025-10-20T18:34:54.651363] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-07. [2025-10-20T18:34:54.681823] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T18:34:54.974234] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:34:55.208858] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T18:34:55.451382] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-07. [2025-10-20T18:34:56.251372] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-05. [2025-10-20T18:34:56.553074] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:34:57.011107] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T18:34:57.051432] [INFO ] [REMOTE ] EOG: Fetching primary data for 2025-10. [2025-10-20T18:34:57.851376] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-08. [2025-10-20T18:34:57.886487] [INFO ] [REMOTE ] EOG: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:34:57.894460] [INFO ] [REMOTE ] EOG: Full fetch complete (77466 records). [2025-10-20T18:34:57.894621] [DEBUG] [VALIDATOR ] EOG: Validating primary data. [2025-10-20T18:34:58.288627] [DEBUG] [VALIDATOR ] EOG: All expected trading days are present. [2025-10-20T18:34:58.288782] [DEBUG] [VALIDATOR ] EOG: Primary data validation passed. [2025-10-20T18:34:58.290041] [INFO ] [MAIN ] EOG: Fetched primary data is valid. [2025-10-20T18:34:58.290129] [DEBUG] [LOCAL ] EOG: Saving primary data. [2025-10-20T18:34:58.291724] [DEBUG] [LOCAL ] EOG: Trimmed 77466 records to 53391 (keeping 200 days of data) [2025-10-20T18:34:58.633256] [DEBUG] [LOCAL ] EOG: Primary data saved (53391 records). [2025-10-20T18:34:58.633420] [INFO ] [MAIN ] EOG: Full history saved. [2025-10-20T18:34:58.633505] [INFO ] [MAIN ] Update Progress: 744/2367 symbols completed. EOG took 38.4s. Estimated remaining: 3h 27m 2s. Rate: 7.8 symbols/min [2025-10-20T18:34:58.656598] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-06. [2025-10-20T18:34:58.812316] [INFO ] [MAIN ] EPAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:34:58.838818] [DEBUG] [LOCAL ] EPAC: Loaded 27001 existing records from primary storage. [2025-10-20T18:34:58.839090] [INFO ] [MAIN ] EPAC: Loaded 27001 existing records. [2025-10-20T18:34:58.954688] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:34:59.031526] [WARN ] [MAIN ] EPAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:34:59.031782] [INFO ] [MAIN ] EPAC: Wiping existing primary data before full refresh. [2025-10-20T18:34:59.031862] [INFO ] [LOCAL ] EPAC: Starting deletion of primary data. [2025-10-20T18:34:59.032123] [DEBUG] [LOCAL ] EPAC: Deleted primary data file. [2025-10-20T18:34:59.032182] [INFO ] [MAIN ] EPAC: Starting full primary refresh. [2025-10-20T18:34:59.335773] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T18:34:59.408222] [INFO ] [REMOTE ] EPAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:34:59.451339] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-07. [2025-10-20T18:35:00.178545] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:35:00.251379] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-09. [2025-10-20T18:35:01.051375] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-01. [2025-10-20T18:35:01.617044] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T18:35:01.851388] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-08. [2025-10-20T18:35:02.651387] [INFO ] [REMOTE ] EOI: Fetching primary data for 2025-10. [2025-10-20T18:35:03.037719] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T18:35:03.117937] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:35:03.451424] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-02. [2025-10-20T18:35:04.220769] [INFO ] [REMOTE ] EOI: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:35:04.223116] [INFO ] [REMOTE ] EOI: Full fetch complete (16173 records). [2025-10-20T18:35:04.223185] [DEBUG] [VALIDATOR ] EOI: Validating primary data. [2025-10-20T18:35:04.256500] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-09. [2025-10-20T18:35:04.586843] [DEBUG] [VALIDATOR ] EOI: All expected trading days are present. [2025-10-20T18:35:04.586927] [DEBUG] [VALIDATOR ] EOI: Primary data validation passed. [2025-10-20T18:35:04.587242] [INFO ] [MAIN ] EOI: Fetched primary data is valid. [2025-10-20T18:35:04.587281] [DEBUG] [LOCAL ] EOI: Saving primary data. [2025-10-20T18:35:04.587810] [DEBUG] [LOCAL ] EOI: Trimmed 16173 records to 10601 (keeping 200 days of data) [2025-10-20T18:35:04.650602] [DEBUG] [LOCAL ] EOI: Primary data saved (10601 records). [2025-10-20T18:35:04.650740] [INFO ] [MAIN ] EOI: Full history saved. [2025-10-20T18:35:04.650819] [INFO ] [MAIN ] Update Progress: 745/2367 symbols completed. EOI took 40.1s. Estimated remaining: 3h 26m 50s. Rate: 7.8 symbols/min [2025-10-20T18:35:04.817283] [INFO ] [MAIN ] EPAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:04.858050] [DEBUG] [LOCAL ] EPAM: Loaded 39978 existing records from primary storage. [2025-10-20T18:35:04.858334] [INFO ] [MAIN ] EPAM: Loaded 39978 existing records. [2025-10-20T18:35:05.038304] [WARN ] [MAIN ] EPAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:05.038860] [INFO ] [MAIN ] EPAM: Wiping existing primary data before full refresh. [2025-10-20T18:35:05.039022] [INFO ] [LOCAL ] EPAM: Starting deletion of primary data. [2025-10-20T18:35:05.052070] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-08. [2025-10-20T18:35:05.044416] [DEBUG] [LOCAL ] EPAM: Deleted primary data file. [2025-10-20T18:35:05.052561] [INFO ] [MAIN ] EPAM: Starting full primary refresh. [2025-10-20T18:35:05.113824] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:35:05.160512] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T18:35:05.415755] [INFO ] [REMOTE ] EPAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:05.851348] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-01. [2025-10-20T18:35:05.940381] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T18:35:06.651391] [INFO ] [REMOTE ] EOT: Fetching primary data for 2025-10. [2025-10-20T18:35:07.451993] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-03. [2025-10-20T18:35:07.456342] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T18:35:07.654752] [INFO ] [REMOTE ] EOT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:35:07.656504] [INFO ] [REMOTE ] EOT: Full fetch complete (10627 records). [2025-10-20T18:35:07.656695] [DEBUG] [VALIDATOR ] EOT: Validating primary data. [2025-10-20T18:35:07.657087] [FATAL] [VALIDATOR ] EOT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:35:07.657167] [FATAL] [MAIN ] EOT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:35:07.657209] [WARN ] [MAIN ] EOT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:35:07.657272] [INFO ] [LOCAL ] EOT: Starting deletion of primary data. [2025-10-20T18:35:07.657333] [DEBUG] [LOCAL ] EOT: No primary data file found to delete. [2025-10-20T18:35:07.657368] [INFO ] [LOCAL ] EOT: Starting deletion of derivative data. [2025-10-20T18:35:07.657518] [DEBUG] [LOCAL ] EOT: Deleted derivative data file. [2025-10-20T18:35:07.657599] [INFO ] [MAIN ] Update Progress: 746/2367 symbols completed. EOT took 22.9s. Estimated remaining: 3h 26m 33s. Rate: 7.8 symbols/min [2025-10-20T18:35:07.829520] [INFO ] [MAIN ] EPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:07.868690] [DEBUG] [LOCAL ] EPC: Loaded 40316 existing records from primary storage. [2025-10-20T18:35:07.869094] [INFO ] [MAIN ] EPC: Loaded 40316 existing records. [2025-10-20T18:35:08.042314] [WARN ] [MAIN ] EPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:08.042601] [INFO ] [MAIN ] EPC: Wiping existing primary data before full refresh. [2025-10-20T18:35:08.042684] [INFO ] [LOCAL ] EPC: Starting deletion of primary data. [2025-10-20T18:35:08.043156] [DEBUG] [LOCAL ] EPC: Deleted primary data file. [2025-10-20T18:35:08.043214] [INFO ] [MAIN ] EPC: Starting full primary refresh. [2025-10-20T18:35:08.259821] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-02. [2025-10-20T18:35:08.397928] [INFO ] [REMOTE ] EPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:08.714402] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T18:35:09.051365] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-04. [2025-10-20T18:35:09.851397] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-09. [2025-10-20T18:35:09.901556] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:35:10.447083] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T18:35:10.651366] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-05. [2025-10-20T18:35:10.726619] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:35:11.451338] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-01. [2025-10-20T18:35:11.776273] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:35:12.251436] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-03. [2025-10-20T18:35:13.051435] [INFO ] [REMOTE ] EOS: Fetching primary data for 2025-10. [2025-10-20T18:35:13.322695] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T18:35:13.851374] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-02. [2025-10-20T18:35:14.005288] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:35:14.251599] [INFO ] [REMOTE ] EOS: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:35:14.254340] [INFO ] [REMOTE ] EOS: Full fetch complete (22914 records). [2025-10-20T18:35:14.254412] [DEBUG] [VALIDATOR ] EOS: Validating primary data. [2025-10-20T18:35:14.620631] [DEBUG] [VALIDATOR ] EOS: All expected trading days are present. [2025-10-20T18:35:14.620738] [DEBUG] [VALIDATOR ] EOS: Primary data validation passed. [2025-10-20T18:35:14.621210] [INFO ] [MAIN ] EOS: Fetched primary data is valid. [2025-10-20T18:35:14.621248] [DEBUG] [LOCAL ] EOS: Saving primary data. [2025-10-20T18:35:14.621871] [DEBUG] [LOCAL ] EOS: Trimmed 22914 records to 14852 (keeping 200 days of data) [2025-10-20T18:35:14.659744] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-06. [2025-10-20T18:35:14.714399] [DEBUG] [LOCAL ] EOS: Primary data saved (14852 records). [2025-10-20T18:35:14.714526] [INFO ] [MAIN ] EOS: Full history saved. [2025-10-20T18:35:14.714604] [INFO ] [MAIN ] Update Progress: 747/2367 symbols completed. EOS took 39.2s. Estimated remaining: 3h 26m 24s. Rate: 7.8 symbols/min [2025-10-20T18:35:14.881744] [INFO ] [MAIN ] EPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:14.930803] [DEBUG] [LOCAL ] EPD: Loaded 54391 existing records from primary storage. [2025-10-20T18:35:14.931165] [INFO ] [MAIN ] EPD: Loaded 54391 existing records. [2025-10-20T18:35:15.111635] [WARN ] [MAIN ] EPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:15.112154] [INFO ] [MAIN ] EPD: Wiping existing primary data before full refresh. [2025-10-20T18:35:15.112214] [INFO ] [LOCAL ] EPD: Starting deletion of primary data. [2025-10-20T18:35:15.112542] [DEBUG] [LOCAL ] EPD: Deleted primary data file. [2025-10-20T18:35:15.112581] [INFO ] [MAIN ] EPD: Starting full primary refresh. [2025-10-20T18:35:15.458402] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-04. [2025-10-20T18:35:15.497052] [INFO ] [REMOTE ] EPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:15.519347] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:35:16.027268] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:35:16.251356] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-03. [2025-10-20T18:35:17.051352] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-01. [2025-10-20T18:35:17.143174] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T18:35:17.548227] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:35:17.851333] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-05. [2025-10-20T18:35:18.651363] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-04. [2025-10-20T18:35:18.957583] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T18:35:19.232631] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T18:35:19.451380] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-07. [2025-10-20T18:35:20.142774] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:35:20.251381] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-02. [2025-10-20T18:35:20.795010] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:35:21.051365] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-06. [2025-10-20T18:35:21.851386] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-08. [2025-10-20T18:35:21.900662] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:35:22.532282] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:35:22.651339] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-07. [2025-10-20T18:35:22.678055] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T18:35:23.451395] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-09. [2025-10-20T18:35:24.180586] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:35:24.251358] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-08. [2025-10-20T18:35:24.488806] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:35:25.051427] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-03. [2025-10-20T18:35:25.775000] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:35:25.851434] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-09. [2025-10-20T18:35:26.651398] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-05. [2025-10-20T18:35:26.983531] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T18:35:27.307294] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:35:27.451402] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-04. [2025-10-20T18:35:27.821813] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:35:28.251420] [INFO ] [REMOTE ] EPAM: Fetching primary data for 2025-10. [2025-10-20T18:35:29.051587] [INFO ] [REMOTE ] EPAC: Fetching primary data for 2025-10. [2025-10-20T18:35:29.613243] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:35:29.696558] [INFO ] [REMOTE ] EPAM: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T18:35:29.702781] [INFO ] [REMOTE ] EPAM: Full fetch complete (56431 records). [2025-10-20T18:35:29.702947] [DEBUG] [VALIDATOR ] EPAM: Validating primary data. [2025-10-20T18:35:29.859144] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-05. [2025-10-20T18:35:30.124049] [DEBUG] [VALIDATOR ] EPAM: All expected trading days are present. [2025-10-20T18:35:30.124192] [DEBUG] [VALIDATOR ] EPAM: Primary data validation passed. [2025-10-20T18:35:30.125383] [INFO ] [MAIN ] EPAM: Fetched primary data is valid. [2025-10-20T18:35:30.125474] [DEBUG] [LOCAL ] EPAM: Saving primary data. [2025-10-20T18:35:30.127784] [DEBUG] [LOCAL ] EPAM: Trimmed 56431 records to 39399 (keeping 200 days of data) [2025-10-20T18:35:30.374787] [DEBUG] [LOCAL ] EPAM: Primary data saved (39399 records). [2025-10-20T18:35:30.375128] [INFO ] [MAIN ] EPAM: Full history saved. [2025-10-20T18:35:30.375723] [INFO ] [MAIN ] Update Progress: 748/2367 symbols completed. EPAM took 25.7s. Estimated remaining: 3h 26m 34s. Rate: 7.8 symbols/min [2025-10-20T18:35:30.514515] [INFO ] [REMOTE ] EPAC: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:35:30.552002] [INFO ] [REMOTE ] EPAC: Full fetch complete (38888 records). [2025-10-20T18:35:30.554560] [DEBUG] [VALIDATOR ] EPAC: Validating primary data. [2025-10-20T18:35:30.604873] [INFO ] [MAIN ] EPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:30.654385] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-06. [2025-10-20T18:35:30.974661] [DEBUG] [LOCAL ] EPR: Loaded 44473 existing records from primary storage. [2025-10-20T18:35:30.988499] [DEBUG] [VALIDATOR ] EPAC: All expected trading days are present. [2025-10-20T18:35:30.988943] [INFO ] [MAIN ] EPR: Loaded 44473 existing records. [2025-10-20T18:35:30.988790] [DEBUG] [VALIDATOR ] EPAC: Primary data validation passed. [2025-10-20T18:35:30.994934] [INFO ] [MAIN ] EPAC: Fetched primary data is valid. [2025-10-20T18:35:31.008895] [DEBUG] [LOCAL ] EPAC: Saving primary data. [2025-10-20T18:35:31.093743] [DEBUG] [LOCAL ] EPAC: Trimmed 38888 records to 26516 (keeping 200 days of data) [2025-10-20T18:35:31.296851] [WARN ] [MAIN ] EPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:31.299980] [INFO ] [MAIN ] EPR: Wiping existing primary data before full refresh. [2025-10-20T18:35:31.300035] [INFO ] [LOCAL ] EPR: Starting deletion of primary data. [2025-10-20T18:35:31.315906] [DEBUG] [LOCAL ] EPR: Deleted primary data file. [2025-10-20T18:35:31.327281] [INFO ] [MAIN ] EPR: Starting full primary refresh. [2025-10-20T18:35:31.338275] [DEBUG] [LOCAL ] EPAC: Primary data saved (26516 records). [2025-10-20T18:35:31.371588] [INFO ] [MAIN ] EPAC: Full history saved. [2025-10-20T18:35:31.388757] [INFO ] [MAIN ] Update Progress: 749/2367 symbols completed. EPAC took 32.8s. Estimated remaining: 3h 26m 12s. Rate: 7.8 symbols/min [2025-10-20T18:35:31.731869] [INFO ] [MAIN ] EPRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:31.907109] [INFO ] [REMOTE ] EPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:31.907549] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-01. [2025-10-20T18:35:31.974709] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:35:31.986937] [DEBUG] [LOCAL ] EPRT: Loaded 49233 existing records from primary storage. [2025-10-20T18:35:31.988406] [INFO ] [MAIN ] EPRT: Loaded 49233 existing records. [2025-10-20T18:35:32.205854] [WARN ] [MAIN ] EPRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:32.206175] [INFO ] [MAIN ] EPRT: Wiping existing primary data before full refresh. [2025-10-20T18:35:32.206240] [INFO ] [LOCAL ] EPRT: Starting deletion of primary data. [2025-10-20T18:35:32.215205] [DEBUG] [LOCAL ] EPRT: Deleted primary data file. [2025-10-20T18:35:32.230371] [INFO ] [MAIN ] EPRT: Starting full primary refresh. [2025-10-20T18:35:32.258162] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-06. [2025-10-20T18:35:32.272279] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:35:32.616412] [INFO ] [REMOTE ] EPRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:33.051417] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-07. [2025-10-20T18:35:33.851362] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-01. [2025-10-20T18:35:34.132986] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T18:35:34.346804] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-01 (2.44 seconds). [2025-10-20T18:35:34.651433] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-07. [2025-10-20T18:35:34.951948] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:35:35.451507] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-02. [2025-10-20T18:35:35.682857] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:35:36.251399] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-02. [2025-10-20T18:35:36.665336] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T18:35:37.000294] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T18:35:37.051439] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-03. [2025-10-20T18:35:37.793366] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:35:37.851356] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-03. [2025-10-20T18:35:38.380964] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T18:35:38.651422] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-08. [2025-10-20T18:35:39.236689] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T18:35:39.451406] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-08. [2025-10-20T18:35:40.251479] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-04. [2025-10-20T18:35:40.281201] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T18:35:41.051413] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-09. [2025-10-20T18:35:41.422319] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T18:35:41.851421] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-09. [2025-10-20T18:35:41.878547] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:35:42.565896] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:35:42.651446] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-04. [2025-10-20T18:35:43.451367] [INFO ] [REMOTE ] EPC: Fetching primary data for 2025-10. [2025-10-20T18:35:43.844004] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T18:35:44.251421] [INFO ] [REMOTE ] EPD: Fetching primary data for 2025-10. [2025-10-20T18:35:44.336040] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T18:35:44.927235] [INFO ] [REMOTE ] EPC: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T18:35:44.937144] [INFO ] [REMOTE ] EPC: Full fetch complete (57300 records). [2025-10-20T18:35:44.937342] [DEBUG] [VALIDATOR ] EPC: Validating primary data. [2025-10-20T18:35:45.056586] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-05. [2025-10-20T18:35:45.347288] [DEBUG] [VALIDATOR ] EPC: All expected trading days are present. [2025-10-20T18:35:45.347372] [DEBUG] [VALIDATOR ] EPC: Primary data validation passed. [2025-10-20T18:35:45.348152] [INFO ] [MAIN ] EPC: Fetched primary data is valid. [2025-10-20T18:35:45.348214] [DEBUG] [LOCAL ] EPC: Saving primary data. [2025-10-20T18:35:45.349318] [DEBUG] [LOCAL ] EPC: Trimmed 57300 records to 39805 (keeping 200 days of data) [2025-10-20T18:35:45.592418] [DEBUG] [LOCAL ] EPC: Primary data saved (39805 records). [2025-10-20T18:35:45.592599] [INFO ] [MAIN ] EPC: Full history saved. [2025-10-20T18:35:45.592762] [INFO ] [MAIN ] Update Progress: 750/2367 symbols completed. EPC took 37.9s. Estimated remaining: 3h 26m 18s. Rate: 7.8 symbols/min [2025-10-20T18:35:45.776952] [INFO ] [MAIN ] EQBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:45.788990] [DEBUG] [LOCAL ] EQBK: Loaded 11534 existing records from primary storage. [2025-10-20T18:35:45.789250] [INFO ] [MAIN ] EQBK: Loaded 11534 existing records. [2025-10-20T18:35:45.860339] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-05. [2025-10-20T18:35:45.976572] [WARN ] [MAIN ] EQBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:45.976899] [INFO ] [MAIN ] EQBK: Wiping existing primary data before full refresh. [2025-10-20T18:35:45.976978] [INFO ] [LOCAL ] EQBK: Starting deletion of primary data. [2025-10-20T18:35:45.977181] [DEBUG] [LOCAL ] EQBK: Deleted primary data file. [2025-10-20T18:35:45.977257] [INFO ] [MAIN ] EQBK: Starting full primary refresh. [2025-10-20T18:35:46.340126] [INFO ] [REMOTE ] EQBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:46.472385] [INFO ] [REMOTE ] EPD: Incremental fetch complete for 2025-10 (2.22 seconds). [2025-10-20T18:35:46.480358] [INFO ] [REMOTE ] EPD: Full fetch complete (77780 records). [2025-10-20T18:35:46.480499] [DEBUG] [VALIDATOR ] EPD: Validating primary data. [2025-10-20T18:35:46.658384] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-01. [2025-10-20T18:35:46.893529] [DEBUG] [VALIDATOR ] EPD: All expected trading days are present. [2025-10-20T18:35:46.893682] [DEBUG] [VALIDATOR ] EPD: Primary data validation passed. [2025-10-20T18:35:46.894668] [INFO ] [MAIN ] EPD: Fetched primary data is valid. [2025-10-20T18:35:46.894893] [DEBUG] [LOCAL ] EPD: Saving primary data. [2025-10-20T18:35:46.922617] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T18:35:46.932675] [DEBUG] [LOCAL ] EPD: Trimmed 77780 records to 53612 (keeping 200 days of data) [2025-10-20T18:35:47.256447] [DEBUG] [LOCAL ] EPD: Primary data saved (53612 records). [2025-10-20T18:35:47.256611] [INFO ] [MAIN ] EPD: Full history saved. [2025-10-20T18:35:47.256760] [INFO ] [MAIN ] Update Progress: 751/2367 symbols completed. EPD took 32.5s. Estimated remaining: 3h 25m 57s. Rate: 7.8 symbols/min [2025-10-20T18:35:47.431453] [INFO ] [MAIN ] EQH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:35:47.457834] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-06. [2025-10-20T18:35:47.481238] [DEBUG] [LOCAL ] EQH: Loaded 53475 existing records from primary storage. [2025-10-20T18:35:47.481572] [INFO ] [MAIN ] EQH: Loaded 53475 existing records. [2025-10-20T18:35:47.661403] [WARN ] [MAIN ] EQH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:35:47.662058] [INFO ] [MAIN ] EQH: Wiping existing primary data before full refresh. [2025-10-20T18:35:47.662319] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-10-20T18:35:47.662998] [DEBUG] [LOCAL ] EQH: Deleted primary data file. [2025-10-20T18:35:47.663062] [INFO ] [MAIN ] EQH: Starting full primary refresh. [2025-10-20T18:35:47.800364] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T18:35:47.818248] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:35:48.088201] [INFO ] [REMOTE ] EQH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:35:48.251383] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-06. [2025-10-20T18:35:48.854740] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:35:49.051381] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-07. [2025-10-20T18:35:49.852706] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-02. [2025-10-20T18:35:49.854950] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T18:35:50.435420] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T18:35:50.651412] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-01. [2025-10-20T18:35:50.683164] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T18:35:51.451393] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-07. [2025-10-20T18:35:52.251391] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-03. [2025-10-20T18:35:52.597608] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T18:35:52.682313] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T18:35:53.051360] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-04. [2025-10-20T18:35:53.209401] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T18:35:53.851394] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-02. [2025-10-20T18:35:54.116462] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:35:54.651377] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-08. [2025-10-20T18:35:55.451350] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-05. [2025-10-20T18:35:55.503243] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:35:56.005869] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:35:56.053814] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:35:56.251399] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-08. [2025-10-20T18:35:57.051388] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-09. [2025-10-20T18:35:57.851386] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-06. [2025-10-20T18:35:57.967154] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:35:58.463733] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T18:35:58.651384] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-09. [2025-10-20T18:35:58.733378] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:35:59.451389] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-07. [2025-10-20T18:36:00.043476] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:36:00.251373] [INFO ] [REMOTE ] EPRT: Fetching primary data for 2025-10. [2025-10-20T18:36:00.389086] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T18:36:01.051382] [INFO ] [REMOTE ] EPR: Fetching primary data for 2025-10. [2025-10-20T18:36:01.851414] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-03. [2025-10-20T18:36:02.108952] [INFO ] [REMOTE ] EPRT: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:36:02.116324] [INFO ] [REMOTE ] EPRT: Full fetch complete (69434 records). [2025-10-20T18:36:02.116435] [DEBUG] [VALIDATOR ] EPRT: Validating primary data. [2025-10-20T18:36:02.511205] [DEBUG] [VALIDATOR ] EPRT: All expected trading days are present. [2025-10-20T18:36:02.511292] [DEBUG] [VALIDATOR ] EPRT: Primary data validation passed. [2025-10-20T18:36:02.512124] [INFO ] [MAIN ] EPRT: Fetched primary data is valid. [2025-10-20T18:36:02.512190] [DEBUG] [LOCAL ] EPRT: Saving primary data. [2025-10-20T18:36:02.513509] [DEBUG] [LOCAL ] EPRT: Trimmed 69434 records to 48488 (keeping 200 days of data) [2025-10-20T18:36:02.693074] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-08. [2025-10-20T18:36:02.819010] [DEBUG] [LOCAL ] EPRT: Primary data saved (48488 records). [2025-10-20T18:36:02.819182] [INFO ] [MAIN ] EPRT: Full history saved. [2025-10-20T18:36:02.819285] [INFO ] [MAIN ] Update Progress: 752/2367 symbols completed. EPRT took 31.4s. Estimated remaining: 3h 26m 7s. Rate: 7.8 symbols/min [2025-10-20T18:36:02.995559] [INFO ] [MAIN ] EQNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:03.045509] [DEBUG] [LOCAL ] EQNR: Loaded 53821 existing records from primary storage. [2025-10-20T18:36:03.045883] [INFO ] [MAIN ] EQNR: Loaded 53821 existing records. [2025-10-20T18:36:03.254671] [INFO ] [REMOTE ] EPR: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T18:36:03.252093] [WARN ] [MAIN ] EQNR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:03.262842] [INFO ] [MAIN ] EQNR: Wiping existing primary data before full refresh. [2025-10-20T18:36:03.263140] [INFO ] [LOCAL ] EQNR: Starting deletion of primary data. [2025-10-20T18:36:03.263585] [INFO ] [REMOTE ] EPR: Full fetch complete (63249 records). [2025-10-20T18:36:03.263831] [DEBUG] [VALIDATOR ] EPR: Validating primary data. [2025-10-20T18:36:03.263912] [DEBUG] [LOCAL ] EQNR: Deleted primary data file. [2025-10-20T18:36:03.264218] [INFO ] [MAIN ] EQNR: Starting full primary refresh. [2025-10-20T18:36:03.876394] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T18:36:03.945450] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-09. [2025-10-20T18:36:04.075133] [INFO ] [REMOTE ] EQNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:04.071194] [DEBUG] [VALIDATOR ] EPR: All expected trading days are present. [2025-10-20T18:36:04.075711] [DEBUG] [VALIDATOR ] EPR: Primary data validation passed. [2025-10-20T18:36:04.076610] [INFO ] [MAIN ] EPR: Fetched primary data is valid. [2025-10-20T18:36:04.076710] [DEBUG] [LOCAL ] EPR: Saving primary data. [2025-10-20T18:36:04.077929] [DEBUG] [LOCAL ] EPR: Trimmed 63249 records to 43859 (keeping 200 days of data) [2025-10-20T18:36:04.227440] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T18:36:04.300752] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-04. [2025-10-20T18:36:04.401155] [DEBUG] [LOCAL ] EPR: Primary data saved (43859 records). [2025-10-20T18:36:04.401289] [INFO ] [MAIN ] EPR: Full history saved. [2025-10-20T18:36:04.401364] [INFO ] [MAIN ] Update Progress: 753/2367 symbols completed. EPR took 34.0s. Estimated remaining: 3h 25m 46s. Rate: 7.8 symbols/min [2025-10-20T18:36:04.571835] [INFO ] [MAIN ] EQR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:04.617438] [DEBUG] [LOCAL ] EQR: Loaded 52498 existing records from primary storage. [2025-10-20T18:36:04.617831] [INFO ] [MAIN ] EQR: Loaded 52498 existing records. [2025-10-20T18:36:04.802669] [WARN ] [MAIN ] EQR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:04.803861] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T18:36:04.804088] [INFO ] [MAIN ] EQR: Wiping existing primary data before full refresh. [2025-10-20T18:36:04.807269] [INFO ] [LOCAL ] EQR: Starting deletion of primary data. [2025-10-20T18:36:04.807937] [DEBUG] [LOCAL ] EQR: Deleted primary data file. [2025-10-20T18:36:04.807994] [INFO ] [MAIN ] EQR: Starting full primary refresh. [2025-10-20T18:36:05.056545] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-01. [2025-10-20T18:36:05.157088] [INFO ] [REMOTE ] EQR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:05.784574] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:36:05.851345] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-01. [2025-10-20T18:36:06.651375] [INFO ] [REMOTE ] EQBK: Fetching primary data for 2025-10. [2025-10-20T18:36:06.865309] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T18:36:07.451354] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-02. [2025-10-20T18:36:07.635569] [INFO ] [REMOTE ] EQBK: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T18:36:07.642691] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:36:07.641513] [INFO ] [REMOTE ] EQBK: Full fetch complete (15732 records). [2025-10-20T18:36:07.653813] [DEBUG] [VALIDATOR ] EQBK: Validating primary data. [2025-10-20T18:36:08.023809] [DEBUG] [VALIDATOR ] EQBK: All expected trading days are present. [2025-10-20T18:36:08.023886] [DEBUG] [VALIDATOR ] EQBK: Primary data validation passed. [2025-10-20T18:36:08.024251] [INFO ] [MAIN ] EQBK: Fetched primary data is valid. [2025-10-20T18:36:08.024291] [DEBUG] [LOCAL ] EQBK: Saving primary data. [2025-10-20T18:36:08.024890] [DEBUG] [LOCAL ] EQBK: Trimmed 15732 records to 11425 (keeping 200 days of data) [2025-10-20T18:36:08.093314] [DEBUG] [LOCAL ] EQBK: Primary data saved (11425 records). [2025-10-20T18:36:08.093451] [INFO ] [MAIN ] EQBK: Full history saved. [2025-10-20T18:36:08.093530] [INFO ] [MAIN ] Update Progress: 754/2367 symbols completed. EQBK took 22.5s. Estimated remaining: 3h 25m 30s. Rate: 7.8 symbols/min [2025-10-20T18:36:08.252138] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-02. [2025-10-20T18:36:08.272590] [INFO ] [MAIN ] EQS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:08.272854] [DEBUG] [LOCAL ] EQS: No existing data file found in primary storage. [2025-10-20T18:36:08.272901] [WARN ] [MAIN ] EQS: No existing data found, performing full refresh instead. [2025-10-20T18:36:08.273006] [INFO ] [MAIN ] EQS: Wiping existing primary data before full refresh. [2025-10-20T18:36:08.273047] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-10-20T18:36:08.273089] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-10-20T18:36:08.273159] [INFO ] [MAIN ] EQS: Starting full primary refresh. [2025-10-20T18:36:08.636973] [INFO ] [REMOTE ] EQS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:09.034868] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T18:36:09.051343] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-05. [2025-10-20T18:36:09.851378] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-01. [2025-10-20T18:36:10.053685] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:36:10.578926] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T18:36:10.651382] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-03. [2025-10-20T18:36:10.777357] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:36:11.451379] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-02. [2025-10-20T18:36:11.818402] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T18:36:12.251373] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-03. [2025-10-20T18:36:12.644363] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:36:13.051364] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-06. [2025-10-20T18:36:13.851366] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-03. [2025-10-20T18:36:13.886921] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:36:14.162445] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-03 (0.31 seconds). [2025-10-20T18:36:14.655379] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-04. [2025-10-20T18:36:14.660893] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:36:15.451372] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-04. [2025-10-20T18:36:15.731689] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T18:36:16.251368] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-04. [2025-10-20T18:36:16.675606] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:36:17.051373] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-07. [2025-10-20T18:36:17.851400] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-05. [2025-10-20T18:36:18.108480] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T18:36:18.221621] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T18:36:18.651388] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-05. [2025-10-20T18:36:18.824068] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:36:19.451388] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-06. [2025-10-20T18:36:19.828863] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T18:36:20.251424] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-08. [2025-10-20T18:36:20.694142] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T18:36:21.051437] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-06. [2025-10-20T18:36:21.851384] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-07. [2025-10-20T18:36:22.031982] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:36:22.430763] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:36:22.445388] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:36:22.651383] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-07. [2025-10-20T18:36:23.451446] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-09. [2025-10-20T18:36:24.258693] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-05. [2025-10-20T18:36:24.295351] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:36:25.051351] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-08. [2025-10-20T18:36:25.169564] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:36:25.575908] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T18:36:25.851388] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-08. [2025-10-20T18:36:25.994853] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:36:26.651435] [INFO ] [REMOTE ] EQH: Fetching primary data for 2025-10. [2025-10-20T18:36:27.451397] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-09. [2025-10-20T18:36:27.739503] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T18:36:27.956533] [INFO ] [REMOTE ] EQH: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T18:36:27.965074] [INFO ] [REMOTE ] EQH: Full fetch complete (76265 records). [2025-10-20T18:36:27.965309] [DEBUG] [VALIDATOR ] EQH: Validating primary data. [2025-10-20T18:36:27.966927] [FATAL] [VALIDATOR ] EQH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:36:27.967213] [FATAL] [MAIN ] EQH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:36:27.967537] [WARN ] [MAIN ] EQH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:36:27.967597] [INFO ] [LOCAL ] EQH: Starting deletion of primary data. [2025-10-20T18:36:27.967700] [DEBUG] [LOCAL ] EQH: No primary data file found to delete. [2025-10-20T18:36:27.967759] [INFO ] [LOCAL ] EQH: Starting deletion of derivative data. [2025-10-20T18:36:27.968001] [DEBUG] [LOCAL ] EQH: Deleted derivative data file. [2025-10-20T18:36:27.968110] [INFO ] [MAIN ] Update Progress: 755/2367 symbols completed. EQH took 40.7s. Estimated remaining: 3h 25m 48s. Rate: 7.8 symbols/min [2025-10-20T18:36:28.160339] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:36:28.165130] [INFO ] [MAIN ] EQT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:28.224927] [DEBUG] [LOCAL ] EQT: Loaded 54465 existing records from primary storage. [2025-10-20T18:36:28.225364] [INFO ] [MAIN ] EQT: Loaded 54465 existing records. [2025-10-20T18:36:28.257322] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-06. [2025-10-20T18:36:28.427769] [WARN ] [MAIN ] EQT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:36:28.428008] [INFO ] [MAIN ] EQT: Wiping existing primary data before full refresh. [2025-10-20T18:36:28.428058] [INFO ] [LOCAL ] EQT: Starting deletion of primary data. [2025-10-20T18:36:28.428359] [DEBUG] [LOCAL ] EQT: Deleted primary data file. [2025-10-20T18:36:28.428400] [INFO ] [MAIN ] EQT: Starting full primary refresh. [2025-10-20T18:36:28.775546] [INFO ] [REMOTE ] EQT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:29.051425] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-09. [2025-10-20T18:36:29.851366] [INFO ] [REMOTE ] EQS: Fetching primary data for 2025-10. [2025-10-20T18:36:29.959605] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T18:36:30.275703] [INFO ] [REMOTE ] EQS: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T18:36:30.276711] [INFO ] [REMOTE ] EQS: Full fetch complete (2387 records). [2025-10-20T18:36:30.276781] [DEBUG] [VALIDATOR ] EQS: Validating primary data. [2025-10-20T18:36:30.475325] [WARN ] [VALIDATOR ] EQS: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:36:30.475670] [FATAL] [MAIN ] EQS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:36:30.475756] [WARN ] [MAIN ] EQS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:36:30.475845] [INFO ] [LOCAL ] EQS: Starting deletion of primary data. [2025-10-20T18:36:30.475940] [DEBUG] [LOCAL ] EQS: No primary data file found to delete. [2025-10-20T18:36:30.476010] [INFO ] [LOCAL ] EQS: Starting deletion of derivative data. [2025-10-20T18:36:30.476123] [DEBUG] [LOCAL ] EQS: No derivative data file found to delete. [2025-10-20T18:36:30.476237] [INFO ] [MAIN ] Update Progress: 756/2367 symbols completed. EQS took 22.4s. Estimated remaining: 3h 25m 30s. Rate: 7.8 symbols/min [2025-10-20T18:36:30.662059] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-01. [2025-10-20T18:36:30.666255] [INFO ] [MAIN ] EQV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:36:30.667765] [DEBUG] [LOCAL ] EQV: No existing data file found in primary storage. [2025-10-20T18:36:30.667821] [WARN ] [MAIN ] EQV: No existing data found, performing full refresh instead. [2025-10-20T18:36:30.667943] [INFO ] [MAIN ] EQV: Wiping existing primary data before full refresh. [2025-10-20T18:36:30.667986] [INFO ] [LOCAL ] EQV: Starting deletion of primary data. [2025-10-20T18:36:30.668028] [DEBUG] [LOCAL ] EQV: No primary data file found to delete. [2025-10-20T18:36:30.668059] [INFO ] [MAIN ] EQV: Starting full primary refresh. [2025-10-20T18:36:31.031334] [INFO ] [REMOTE ] EQV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:36:31.062257] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T18:36:31.451440] [INFO ] [REMOTE ] EQNR: Fetching primary data for 2025-10. [2025-10-20T18:36:32.251434] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-01. [2025-10-20T18:36:33.003199] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T18:36:33.054347] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-07. [2025-10-20T18:36:33.060273] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:36:33.182708] [INFO ] [REMOTE ] EQNR: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:36:33.190410] [INFO ] [REMOTE ] EQNR: Full fetch complete (77011 records). [2025-10-20T18:36:33.190537] [DEBUG] [VALIDATOR ] EQNR: Validating primary data. [2025-10-20T18:36:33.588450] [DEBUG] [VALIDATOR ] EQNR: All expected trading days are present. [2025-10-20T18:36:33.588557] [DEBUG] [VALIDATOR ] EQNR: Primary data validation passed. [2025-10-20T18:36:33.589487] [INFO ] [MAIN ] EQNR: Fetched primary data is valid. [2025-10-20T18:36:33.589550] [DEBUG] [LOCAL ] EQNR: Saving primary data. [2025-10-20T18:36:33.590994] [DEBUG] [LOCAL ] EQNR: Trimmed 77011 records to 53058 (keeping 200 days of data) [2025-10-20T18:36:33.872398] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-02. [2025-10-20T18:36:33.918536] [DEBUG] [LOCAL ] EQNR: Primary data saved (53058 records). [2025-10-20T18:36:33.919200] [INFO ] [MAIN ] EQNR: Full history saved. [2025-10-20T18:36:33.919469] [INFO ] [MAIN ] Update Progress: 757/2367 symbols completed. EQNR took 31.1s. Estimated remaining: 3h 25m 13s. Rate: 7.8 symbols/min [2025-10-20T18:36:34.470159] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T18:36:34.506037] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:36:34.651335] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-08. [2025-10-20T18:36:35.451439] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-03. [2025-10-20T18:36:35.973600] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T18:36:36.256939] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-02. [2025-10-20T18:36:36.262497] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T18:36:37.051446] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-04. [2025-10-20T18:36:37.535592] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T18:36:37.851350] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-05. [2025-10-20T18:36:38.003305] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:36:38.260382] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T18:36:38.651426] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-06. [2025-10-20T18:36:39.218161] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:36:39.451360] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-03. [2025-10-20T18:36:40.251435] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-09. [2025-10-20T18:36:41.051368] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-07. [2025-10-20T18:36:41.607746] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-10-20T18:36:41.851350] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-04. [2025-10-20T18:36:42.104507] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T18:36:42.651387] [INFO ] [REMOTE ] EQR: Fetching primary data for 2025-10. [2025-10-20T18:36:47.162867] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-07 (6.11 seconds). [2025-10-20T18:36:47.164336] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-08. [2025-10-20T18:36:49.657251] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T18:36:49.659590] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-09. [2025-10-20T18:36:50.497109] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T18:36:50.498785] [INFO ] [REMOTE ] EQV: Fetching primary data for 2025-10. [2025-10-20T18:36:51.556944] [INFO ] [REMOTE ] EQV: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:36:51.558085] [INFO ] [REMOTE ] EQV: Full fetch complete (5061 records). [2025-10-20T18:36:51.558186] [DEBUG] [VALIDATOR ] EQV: Validating primary data. [2025-10-20T18:36:51.968895] [DEBUG] [VALIDATOR ] EQV: All expected trading days are present. [2025-10-20T18:36:51.969002] [DEBUG] [VALIDATOR ] EQV: Primary data validation passed. [2025-10-20T18:36:51.969367] [INFO ] [MAIN ] EQV: Fetched primary data is valid. [2025-10-20T18:36:51.969447] [DEBUG] [LOCAL ] EQV: Saving primary data. [2025-10-20T18:36:51.969916] [DEBUG] [LOCAL ] EQV: Trimmed 5061 records to 2745 (keeping 200 days of data) [2025-10-20T18:36:51.987136] [DEBUG] [LOCAL ] EQV: Primary data saved (2745 records). [2025-10-20T18:36:51.987242] [INFO ] [MAIN ] EQV: Full history saved. [2025-10-20T18:36:51.987325] [INFO ] [MAIN ] Update Progress: 758/2367 symbols completed. EQV took 21.5s. Estimated remaining: 3h 25m 28s. Rate: 7.8 symbols/min [2025-10-20T18:36:52.461453] [INFO ] [REMOTE ] EQR: Incremental fetch complete for 2025-10 (9.81 seconds). [2025-10-20T18:36:52.470307] [INFO ] [REMOTE ] EQR: Full fetch complete (74535 records). [2025-10-20T18:36:52.470475] [DEBUG] [VALIDATOR ] EQR: Validating primary data. [2025-10-20T18:36:52.647669] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-04 (10.80 seconds). [2025-10-20T18:36:52.695414] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-05. [2025-10-20T18:36:52.960263] [DEBUG] [VALIDATOR ] EQR: All expected trading days are present. [2025-10-20T18:36:52.960377] [DEBUG] [VALIDATOR ] EQR: Primary data validation passed. [2025-10-20T18:36:52.961354] [INFO ] [MAIN ] EQR: Fetched primary data is valid. [2025-10-20T18:36:52.961419] [DEBUG] [LOCAL ] EQR: Saving primary data. [2025-10-20T18:36:52.962923] [DEBUG] [LOCAL ] EQR: Trimmed 74535 records to 51795 (keeping 200 days of data) [2025-10-20T18:36:53.276771] [DEBUG] [LOCAL ] EQR: Primary data saved (51795 records). [2025-10-20T18:36:53.276909] [INFO ] [MAIN ] EQR: Full history saved. [2025-10-20T18:36:53.277432] [INFO ] [MAIN ] Update Progress: 759/2367 symbols completed. EQR took 48.9s. Estimated remaining: 3h 25m 7s. Rate: 7.8 symbols/min [2025-10-20T18:36:55.322152] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-05 (2.63 seconds). [2025-10-20T18:36:55.334934] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-06. [2025-10-20T18:36:58.203331] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-06 (2.87 seconds). [2025-10-20T18:36:58.214260] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-07. [2025-10-20T18:37:00.313908] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-10-20T18:37:00.327337] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-08. [2025-10-20T18:37:02.705345] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T18:37:02.716905] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-09. [2025-10-20T18:37:04.890196] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:37:04.903110] [INFO ] [REMOTE ] EQT: Fetching primary data for 2025-10. [2025-10-20T18:37:07.253687] [INFO ] [REMOTE ] EQT: Incremental fetch complete for 2025-10 (2.35 seconds). [2025-10-20T18:37:07.261590] [INFO ] [REMOTE ] EQT: Full fetch complete (77860 records). [2025-10-20T18:37:07.261757] [DEBUG] [VALIDATOR ] EQT: Validating primary data. [2025-10-20T18:37:07.654793] [DEBUG] [VALIDATOR ] EQT: All expected trading days are present. [2025-10-20T18:37:07.654874] [DEBUG] [VALIDATOR ] EQT: Primary data validation passed. [2025-10-20T18:37:07.655776] [INFO ] [MAIN ] EQT: Fetched primary data is valid. [2025-10-20T18:37:07.655839] [DEBUG] [LOCAL ] EQT: Saving primary data. [2025-10-20T18:37:07.657296] [DEBUG] [LOCAL ] EQT: Trimmed 77860 records to 53685 (keeping 200 days of data) [2025-10-20T18:37:07.972444] [DEBUG] [LOCAL ] EQT: Primary data saved (53685 records). [2025-10-20T18:37:07.972571] [INFO ] [MAIN ] EQT: Full history saved. [2025-10-20T18:37:07.972669] [INFO ] [MAIN ] Update Progress: 760/2367 symbols completed. EQT took 40.0s. Estimated remaining: 3h 25m 14s. Rate: 7.8 symbols/min [2025-10-20T18:37:08.508480] [INFO ] [MAIN ] ES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.678458] [INFO ] [MAIN ] ESAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.682756] [INFO ] [MAIN ] ERJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.632289] [INFO ] [MAIN ] ERO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:08.722073] [DEBUG] [LOCAL ] ESAB: Loaded 26444 existing records from primary storage. [2025-10-20T18:37:08.722379] [INFO ] [MAIN ] ESAB: Loaded 26444 existing records. [2025-10-20T18:37:08.861877] [DEBUG] [LOCAL ] ERO: Loaded 42618 existing records from primary storage. [2025-10-20T18:37:08.873854] [INFO ] [MAIN ] ERO: Loaded 42618 existing records. [2025-10-20T18:37:09.073052] [DEBUG] [LOCAL ] ES: Loaded 53462 existing records from primary storage. [2025-10-20T18:37:09.078695] [WARN ] [MAIN ] ESAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.084977] [DEBUG] [LOCAL ] ERJ: Loaded 51052 existing records from primary storage. [2025-10-20T18:37:09.095780] [INFO ] [MAIN ] ES: Loaded 53462 existing records. [2025-10-20T18:37:09.085223] [INFO ] [MAIN ] ESAB: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.110043] [INFO ] [MAIN ] ERJ: Loaded 51052 existing records. [2025-10-20T18:37:09.114784] [INFO ] [LOCAL ] ESAB: Starting deletion of primary data. [2025-10-20T18:37:09.226850] [DEBUG] [LOCAL ] ESAB: Deleted primary data file. [2025-10-20T18:37:09.292670] [INFO ] [MAIN ] ESAB: Starting full primary refresh. [2025-10-20T18:37:09.435032] [WARN ] [MAIN ] ERO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.466916] [INFO ] [MAIN ] ERO: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.492824] [INFO ] [LOCAL ] ERO: Starting deletion of primary data. [2025-10-20T18:37:09.615698] [DEBUG] [LOCAL ] ERO: Deleted primary data file. [2025-10-20T18:37:09.641239] [WARN ] [MAIN ] ERJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.649893] [INFO ] [MAIN ] ERO: Starting full primary refresh. [2025-10-20T18:37:09.650220] [INFO ] [MAIN ] ERJ: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.662588] [INFO ] [LOCAL ] ERJ: Starting deletion of primary data. [2025-10-20T18:37:09.718154] [DEBUG] [LOCAL ] ERJ: Deleted primary data file. [2025-10-20T18:37:09.732782] [WARN ] [MAIN ] ES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:09.718353] [INFO ] [MAIN ] ERJ: Starting full primary refresh. [2025-10-20T18:37:09.754111] [INFO ] [MAIN ] ES: Wiping existing primary data before full refresh. [2025-10-20T18:37:09.764704] [INFO ] [LOCAL ] ES: Starting deletion of primary data. [2025-10-20T18:37:09.880335] [DEBUG] [LOCAL ] ES: Deleted primary data file. [2025-10-20T18:37:09.914199] [INFO ] [MAIN ] ES: Starting full primary refresh. [2025-10-20T18:37:10.541050] [INFO ] [REMOTE ] ESAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:10.618590] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-01. [2025-10-20T18:37:11.037213] [INFO ] [REMOTE ] ERO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.061178] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-01. [2025-10-20T18:37:11.115390] [INFO ] [REMOTE ] ERJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.129543] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-01. [2025-10-20T18:37:11.157392] [INFO ] [REMOTE ] ES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:11.157823] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-01. [2025-10-20T18:37:11.952515] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:37:11.957192] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-02. [2025-10-20T18:37:12.856433] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:37:12.868166] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-02. [2025-10-20T18:37:13.025824] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T18:37:13.036445] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-02. [2025-10-20T18:37:13.078995] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:37:13.085420] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-03. [2025-10-20T18:37:13.126983] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T18:37:13.136050] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-02. [2025-10-20T18:37:14.231115] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T18:37:14.237140] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-04. [2025-10-20T18:37:14.401541] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:37:14.411733] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-03. [2025-10-20T18:37:14.479365] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:37:14.487761] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-03. [2025-10-20T18:37:14.772450] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:37:14.781992] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-03. [2025-10-20T18:37:15.520132] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T18:37:15.526770] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-05. [2025-10-20T18:37:15.872426] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:37:15.882626] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-04. [2025-10-20T18:37:16.221168] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T18:37:16.232553] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-04. [2025-10-20T18:37:16.302533] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:37:16.313362] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-04. [2025-10-20T18:37:16.362894] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:37:16.368362] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-06. [2025-10-20T18:37:17.025056] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T18:37:17.036048] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-05. [2025-10-20T18:37:17.320006] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:37:17.325460] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-07. [2025-10-20T18:37:18.231565] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T18:37:18.240920] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-06. [2025-10-20T18:37:18.459381] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:37:18.470936] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-05. [2025-10-20T18:37:18.546906] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:37:18.558728] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-05. [2025-10-20T18:37:18.609190] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:37:18.614850] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-08. [2025-10-20T18:37:19.386249] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:37:19.394538] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-07. [2025-10-20T18:37:19.880763] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T18:37:19.890171] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-09. [2025-10-20T18:37:20.279063] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T18:37:20.290950] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-06. [2025-10-20T18:37:20.322553] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T18:37:20.333967] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-06. [2025-10-20T18:37:20.647215] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T18:37:20.662037] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-08. [2025-10-20T18:37:21.181263] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T18:37:21.188236] [INFO ] [REMOTE ] ESAB: Fetching primary data for 2025-10. [2025-10-20T18:37:21.556564] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T18:37:21.566894] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-07. [2025-10-20T18:37:22.075247] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T18:37:22.086459] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-07. [2025-10-20T18:37:22.110685] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T18:37:22.119771] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-09. [2025-10-20T18:37:22.491941] [INFO ] [REMOTE ] ESAB: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:37:22.496239] [INFO ] [REMOTE ] ESAB: Full fetch complete (36350 records). [2025-10-20T18:37:22.496346] [DEBUG] [VALIDATOR ] ESAB: Validating primary data. [2025-10-20T18:37:22.872906] [DEBUG] [VALIDATOR ] ESAB: All expected trading days are present. [2025-10-20T18:37:22.872984] [DEBUG] [VALIDATOR ] ESAB: Primary data validation passed. [2025-10-20T18:37:22.873551] [INFO ] [MAIN ] ESAB: Fetched primary data is valid. [2025-10-20T18:37:22.873590] [DEBUG] [LOCAL ] ESAB: Saving primary data. [2025-10-20T18:37:22.874436] [DEBUG] [LOCAL ] ESAB: Trimmed 36350 records to 26063 (keeping 200 days of data) [2025-10-20T18:37:23.034332] [DEBUG] [LOCAL ] ESAB: Primary data saved (26063 records). [2025-10-20T18:37:23.034461] [INFO ] [MAIN ] ESAB: Full history saved. [2025-10-20T18:37:23.034540] [INFO ] [MAIN ] Update Progress: 761/2367 symbols completed. ESAB took 15.1s. Estimated remaining: 3h 25m 22s. Rate: 7.8 symbols/min [2025-10-20T18:37:23.207113] [INFO ] [MAIN ] ESE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:23.244185] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:37:23.262111] [DEBUG] [LOCAL ] ESE: Loaded 16777 existing records from primary storage. [2025-10-20T18:37:23.261775] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-08. [2025-10-20T18:37:23.263106] [INFO ] [MAIN ] ESE: Loaded 16777 existing records. [2025-10-20T18:37:23.448857] [WARN ] [MAIN ] ESE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:23.449176] [INFO ] [MAIN ] ESE: Wiping existing primary data before full refresh. [2025-10-20T18:37:23.449257] [INFO ] [LOCAL ] ESE: Starting deletion of primary data. [2025-10-20T18:37:23.449461] [DEBUG] [LOCAL ] ESE: Deleted primary data file. [2025-10-20T18:37:23.449516] [INFO ] [MAIN ] ESE: Starting full primary refresh. [2025-10-20T18:37:23.823332] [INFO ] [REMOTE ] ESE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:23.823971] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-01. [2025-10-20T18:37:23.902416] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T18:37:23.912535] [INFO ] [REMOTE ] ERO: Fetching primary data for 2025-10. [2025-10-20T18:37:24.029511] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T18:37:24.040898] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-08. [2025-10-20T18:37:24.749457] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:37:24.753189] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-02. [2025-10-20T18:37:25.003207] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:37:25.014345] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-09. [2025-10-20T18:37:25.374116] [INFO ] [REMOTE ] ERO: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:37:25.381965] [INFO ] [REMOTE ] ERO: Full fetch complete (60563 records). [2025-10-20T18:37:25.382158] [DEBUG] [VALIDATOR ] ERO: Validating primary data. [2025-10-20T18:37:25.707388] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:37:25.749568] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-03. [2025-10-20T18:37:25.808619] [DEBUG] [VALIDATOR ] ERO: All expected trading days are present. [2025-10-20T18:37:25.808743] [DEBUG] [VALIDATOR ] ERO: Primary data validation passed. [2025-10-20T18:37:25.809565] [INFO ] [MAIN ] ERO: Fetched primary data is valid. [2025-10-20T18:37:25.809666] [DEBUG] [LOCAL ] ERO: Saving primary data. [2025-10-20T18:37:25.811134] [DEBUG] [LOCAL ] ERO: Trimmed 60563 records to 42144 (keeping 200 days of data) [2025-10-20T18:37:25.946066] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:37:25.964876] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-09. [2025-10-20T18:37:26.099506] [DEBUG] [LOCAL ] ERO: Primary data saved (42144 records). [2025-10-20T18:37:26.099712] [INFO ] [MAIN ] ERO: Full history saved. [2025-10-20T18:37:26.099810] [INFO ] [MAIN ] Update Progress: 762/2367 symbols completed. ERO took 18.1s. Estimated remaining: 3h 25m 4s. Rate: 7.8 symbols/min [2025-10-20T18:37:26.271419] [INFO ] [MAIN ] ESI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:26.320521] [DEBUG] [LOCAL ] ESI: Loaded 51407 existing records from primary storage. [2025-10-20T18:37:26.320882] [INFO ] [MAIN ] ESI: Loaded 51407 existing records. [2025-10-20T18:37:26.506232] [WARN ] [MAIN ] ESI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:26.506721] [INFO ] [MAIN ] ESI: Wiping existing primary data before full refresh. [2025-10-20T18:37:26.506909] [INFO ] [LOCAL ] ESI: Starting deletion of primary data. [2025-10-20T18:37:26.507585] [DEBUG] [LOCAL ] ESI: Deleted primary data file. [2025-10-20T18:37:26.507671] [INFO ] [MAIN ] ESI: Starting full primary refresh. [2025-10-20T18:37:26.679294] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T18:37:26.713058] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-04. [2025-10-20T18:37:26.904541] [INFO ] [REMOTE ] ESI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:26.904889] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-01. [2025-10-20T18:37:26.932847] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T18:37:26.944050] [INFO ] [REMOTE ] ERJ: Fetching primary data for 2025-10. [2025-10-20T18:37:27.781513] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:37:27.792931] [INFO ] [REMOTE ] ES: Fetching primary data for 2025-10. [2025-10-20T18:37:27.914146] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T18:37:27.918109] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-05. [2025-10-20T18:37:28.447930] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:37:28.457424] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-02. [2025-10-20T18:37:28.812858] [INFO ] [REMOTE ] ERJ: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:37:28.820626] [INFO ] [REMOTE ] ERJ: Full fetch complete (72962 records). [2025-10-20T18:37:28.820988] [DEBUG] [VALIDATOR ] ERJ: Validating primary data. [2025-10-20T18:37:28.911148] [INFO ] [REMOTE ] ES: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T18:37:28.970746] [INFO ] [REMOTE ] ES: Full fetch complete (76475 records). [2025-10-20T18:37:28.976380] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:37:28.976048] [DEBUG] [VALIDATOR ] ES: Validating primary data. [2025-10-20T18:37:28.991446] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-06. [2025-10-20T18:37:29.595979] [DEBUG] [VALIDATOR ] ERJ: All expected trading days are present. [2025-10-20T18:37:29.619385] [DEBUG] [VALIDATOR ] ERJ: Primary data validation passed. [2025-10-20T18:37:29.676288] [DEBUG] [VALIDATOR ] ES: All expected trading days are present. [2025-10-20T18:37:29.677480] [DEBUG] [VALIDATOR ] ES: Primary data validation passed. [2025-10-20T18:37:29.677398] [INFO ] [MAIN ] ERJ: Fetched primary data is valid. [2025-10-20T18:37:29.678233] [INFO ] [MAIN ] ES: Fetched primary data is valid. [2025-10-20T18:37:29.678406] [DEBUG] [LOCAL ] ES: Saving primary data. [2025-10-20T18:37:29.678608] [DEBUG] [LOCAL ] ERJ: Saving primary data. [2025-10-20T18:37:29.680838] [DEBUG] [LOCAL ] ES: Trimmed 76475 records to 52707 (keeping 200 days of data) [2025-10-20T18:37:29.681670] [DEBUG] [LOCAL ] ERJ: Trimmed 72962 records to 50301 (keeping 200 days of data) [2025-10-20T18:37:29.905938] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T18:37:29.939254] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-03. [2025-10-20T18:37:30.042860] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T18:37:30.078591] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-07. [2025-10-20T18:37:30.272293] [DEBUG] [LOCAL ] ERJ: Primary data saved (50301 records). [2025-10-20T18:37:30.278496] [INFO ] [MAIN ] ERJ: Full history saved. [2025-10-20T18:37:30.283407] [INFO ] [MAIN ] Update Progress: 763/2367 symbols completed. ERJ took 22.3s. Estimated remaining: 3h 24m 49s. Rate: 7.8 symbols/min [2025-10-20T18:37:30.534178] [INFO ] [MAIN ] ESNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:30.541233] [DEBUG] [LOCAL ] ES: Primary data saved (52707 records). [2025-10-20T18:37:30.541700] [INFO ] [MAIN ] ES: Full history saved. [2025-10-20T18:37:30.542260] [INFO ] [MAIN ] Update Progress: 764/2367 symbols completed. ES took 22.5s. Estimated remaining: 3h 24m 26s. Rate: 7.8 symbols/min [2025-10-20T18:37:30.712065] [INFO ] [MAIN ] ESRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:30.741573] [DEBUG] [LOCAL ] ESNT: Loaded 38819 existing records from primary storage. [2025-10-20T18:37:30.741965] [INFO ] [MAIN ] ESNT: Loaded 38819 existing records. [2025-10-20T18:37:30.948955] [WARN ] [MAIN ] ESNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:30.965115] [DEBUG] [LOCAL ] ESRT: Loaded 47197 existing records from primary storage. [2025-10-20T18:37:30.965315] [INFO ] [MAIN ] ESNT: Wiping existing primary data before full refresh. [2025-10-20T18:37:30.965623] [INFO ] [MAIN ] ESRT: Loaded 47197 existing records. [2025-10-20T18:37:30.965734] [INFO ] [LOCAL ] ESNT: Starting deletion of primary data. [2025-10-20T18:37:31.010238] [DEBUG] [LOCAL ] ESNT: Deleted primary data file. [2025-10-20T18:37:31.010334] [INFO ] [MAIN ] ESNT: Starting full primary refresh. [2025-10-20T18:37:31.311288] [WARN ] [MAIN ] ESRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:31.322913] [INFO ] [MAIN ] ESRT: Wiping existing primary data before full refresh. [2025-10-20T18:37:31.324380] [INFO ] [LOCAL ] ESRT: Starting deletion of primary data. [2025-10-20T18:37:31.346150] [DEBUG] [LOCAL ] ESRT: Deleted primary data file. [2025-10-20T18:37:31.393283] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:37:31.387710] [INFO ] [MAIN ] ESRT: Starting full primary refresh. [2025-10-20T18:37:31.485006] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-08. [2025-10-20T18:37:31.651161] [INFO ] [REMOTE ] ESNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:31.658040] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-01. [2025-10-20T18:37:31.940294] [INFO ] [REMOTE ] ESRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:32.251353] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-01. [2025-10-20T18:37:32.409457] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-03 (2.47 seconds). [2025-10-20T18:37:33.051372] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-04. [2025-10-20T18:37:33.130382] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:37:33.672354] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:37:33.753749] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:37:33.851347] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-02. [2025-10-20T18:37:34.651433] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-02. [2025-10-20T18:37:34.926559] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T18:37:35.319424] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:37:35.451373] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-03. [2025-10-20T18:37:35.846684] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T18:37:36.251400] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-05. [2025-10-20T18:37:36.650180] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T18:37:37.051438] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-03. [2025-10-20T18:37:37.851432] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-04. [2025-10-20T18:37:37.900031] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:37:38.391543] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:37:38.651395] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-09. [2025-10-20T18:37:39.358630] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:37:39.451423] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-06. [2025-10-20T18:37:39.781776] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T18:37:40.251408] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-04. [2025-10-20T18:37:40.915217] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:37:41.051382] [INFO ] [REMOTE ] ESE: Fetching primary data for 2025-10. [2025-10-20T18:37:41.757501] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:37:41.851404] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-05. [2025-10-20T18:37:42.039292] [INFO ] [REMOTE ] ESE: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:37:42.042050] [INFO ] [REMOTE ] ESE: Full fetch complete (22811 records). [2025-10-20T18:37:42.042131] [DEBUG] [VALIDATOR ] ESE: Validating primary data. [2025-10-20T18:37:42.414964] [DEBUG] [VALIDATOR ] ESE: All expected trading days are present. [2025-10-20T18:37:42.415057] [DEBUG] [VALIDATOR ] ESE: Primary data validation passed. [2025-10-20T18:37:42.415599] [INFO ] [MAIN ] ESE: Fetched primary data is valid. [2025-10-20T18:37:42.415686] [DEBUG] [LOCAL ] ESE: Saving primary data. [2025-10-20T18:37:42.416379] [DEBUG] [LOCAL ] ESE: Trimmed 22811 records to 16467 (keeping 200 days of data) [2025-10-20T18:37:42.526725] [DEBUG] [LOCAL ] ESE: Primary data saved (16467 records). [2025-10-20T18:37:42.526867] [INFO ] [MAIN ] ESE: Full history saved. [2025-10-20T18:37:42.526947] [INFO ] [MAIN ] Update Progress: 765/2367 symbols completed. ESE took 19.5s. Estimated remaining: 3h 24m 28s. Rate: 7.8 symbols/min [2025-10-20T18:37:42.656539] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-07. [2025-10-20T18:37:42.715236] [INFO ] [MAIN ] ESS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:42.747350] [DEBUG] [LOCAL ] ESS: Loaded 30619 existing records from primary storage. [2025-10-20T18:37:42.747721] [INFO ] [MAIN ] ESS: Loaded 30619 existing records. [2025-10-20T18:37:42.935984] [WARN ] [MAIN ] ESS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:42.936252] [INFO ] [MAIN ] ESS: Wiping existing primary data before full refresh. [2025-10-20T18:37:42.936400] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-10-20T18:37:42.936787] [DEBUG] [LOCAL ] ESS: Deleted primary data file. [2025-10-20T18:37:42.936845] [INFO ] [MAIN ] ESS: Starting full primary refresh. [2025-10-20T18:37:43.329552] [INFO ] [REMOTE ] ESS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:43.331426] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:37:43.451338] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-06. [2025-10-20T18:37:44.251429] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-05. [2025-10-20T18:37:44.408611] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:37:44.983137] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:37:45.051367] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-01. [2025-10-20T18:37:45.851497] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-07. [2025-10-20T18:37:45.889220] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T18:37:46.585266] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T18:37:46.651393] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-06. [2025-10-20T18:37:47.377389] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T18:37:47.451410] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-08. [2025-10-20T18:37:48.251394] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-08. [2025-10-20T18:37:48.426993] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:37:49.025630] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:37:49.051344] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-09. [2025-10-20T18:37:49.629812] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:37:49.851366] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-09. [2025-10-20T18:37:50.330276] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:37:50.651383] [INFO ] [REMOTE ] ESI: Fetching primary data for 2025-10. [2025-10-20T18:37:51.345532] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:37:51.451360] [INFO ] [REMOTE ] ESNT: Fetching primary data for 2025-10. [2025-10-20T18:37:52.251363] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-02. [2025-10-20T18:37:52.466668] [INFO ] [REMOTE ] ESI: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:37:52.474256] [INFO ] [REMOTE ] ESI: Full fetch complete (73231 records). [2025-10-20T18:37:52.474371] [DEBUG] [VALIDATOR ] ESI: Validating primary data. [2025-10-20T18:37:52.852965] [DEBUG] [VALIDATOR ] ESI: All expected trading days are present. [2025-10-20T18:37:52.853045] [DEBUG] [VALIDATOR ] ESI: Primary data validation passed. [2025-10-20T18:37:52.853890] [INFO ] [MAIN ] ESI: Fetched primary data is valid. [2025-10-20T18:37:52.853953] [DEBUG] [LOCAL ] ESI: Saving primary data. [2025-10-20T18:37:52.855245] [DEBUG] [LOCAL ] ESI: Trimmed 73231 records to 50643 (keeping 200 days of data) [2025-10-20T18:37:53.062422] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-07. [2025-10-20T18:37:53.158952] [DEBUG] [LOCAL ] ESI: Primary data saved (50643 records). [2025-10-20T18:37:53.159100] [INFO ] [MAIN ] ESI: Full history saved. [2025-10-20T18:37:53.159237] [INFO ] [MAIN ] Update Progress: 766/2367 symbols completed. ESI took 27.1s. Estimated remaining: 3h 24m 26s. Rate: 7.8 symbols/min [2025-10-20T18:37:53.330843] [INFO ] [MAIN ] ESTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:53.392531] [DEBUG] [LOCAL ] ESTC: Loaded 51091 existing records from primary storage. [2025-10-20T18:37:53.393974] [INFO ] [MAIN ] ESTC: Loaded 51091 existing records. [2025-10-20T18:37:53.393821] [INFO ] [REMOTE ] ESNT: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:37:53.498624] [INFO ] [REMOTE ] ESNT: Full fetch complete (57227 records). [2025-10-20T18:37:53.498898] [DEBUG] [VALIDATOR ] ESNT: Validating primary data. [2025-10-20T18:37:53.673537] [WARN ] [MAIN ] ESTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:53.691973] [INFO ] [MAIN ] ESTC: Wiping existing primary data before full refresh. [2025-10-20T18:37:53.746011] [INFO ] [LOCAL ] ESTC: Starting deletion of primary data. [2025-10-20T18:37:53.769050] [DEBUG] [LOCAL ] ESTC: Deleted primary data file. [2025-10-20T18:37:53.766540] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:37:53.769498] [INFO ] [MAIN ] ESTC: Starting full primary refresh. [2025-10-20T18:37:53.925690] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-03. [2025-10-20T18:37:54.239299] [DEBUG] [VALIDATOR ] ESNT: All expected trading days are present. [2025-10-20T18:37:54.276993] [DEBUG] [VALIDATOR ] ESNT: Primary data validation passed. [2025-10-20T18:37:54.302957] [INFO ] [MAIN ] ESNT: Fetched primary data is valid. [2025-10-20T18:37:54.313709] [DEBUG] [LOCAL ] ESNT: Saving primary data. [2025-10-20T18:37:54.382802] [DEBUG] [LOCAL ] ESNT: Trimmed 57227 records to 38177 (keeping 200 days of data) [2025-10-20T18:37:54.414210] [INFO ] [REMOTE ] ESTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:54.537237] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T18:37:54.654066] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-08. [2025-10-20T18:37:54.683153] [DEBUG] [LOCAL ] ESNT: Primary data saved (38177 records). [2025-10-20T18:37:54.683265] [INFO ] [MAIN ] ESNT: Full history saved. [2025-10-20T18:37:54.683338] [INFO ] [MAIN ] Update Progress: 767/2367 symbols completed. ESNT took 24.4s. Estimated remaining: 3h 24m 6s. Rate: 7.8 symbols/min [2025-10-20T18:37:54.848696] [INFO ] [MAIN ] ET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:37:54.900450] [DEBUG] [LOCAL ] ET: Loaded 54569 existing records from primary storage. [2025-10-20T18:37:54.900785] [INFO ] [MAIN ] ET: Loaded 54569 existing records. [2025-10-20T18:37:55.082497] [WARN ] [MAIN ] ET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:37:55.082756] [INFO ] [MAIN ] ET: Wiping existing primary data before full refresh. [2025-10-20T18:37:55.082856] [INFO ] [LOCAL ] ET: Starting deletion of primary data. [2025-10-20T18:37:55.083081] [DEBUG] [LOCAL ] ET: Deleted primary data file. [2025-10-20T18:37:55.083150] [INFO ] [MAIN ] ET: Starting full primary refresh. [2025-10-20T18:37:55.399317] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:37:55.456532] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-01. [2025-10-20T18:37:55.497570] [INFO ] [REMOTE ] ET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:37:56.251391] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-04. [2025-10-20T18:37:56.337021] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T18:37:57.051380] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-09. [2025-10-20T18:37:57.350917] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T18:37:57.735665] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:37:57.851389] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-02. [2025-10-20T18:37:58.646178] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:37:58.655468] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-05. [2025-10-20T18:37:59.451389] [INFO ] [REMOTE ] ESRT: Fetching primary data for 2025-10. [2025-10-20T18:37:59.488510] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T18:37:59.836273] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:38:00.251408] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-01. [2025-10-20T18:38:01.051419] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-06. [2025-10-20T18:38:01.156464] [INFO ] [REMOTE ] ESRT: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:38:01.163755] [INFO ] [REMOTE ] ESRT: Full fetch complete (66652 records). [2025-10-20T18:38:01.163917] [DEBUG] [VALIDATOR ] ESRT: Validating primary data. [2025-10-20T18:38:01.559248] [DEBUG] [VALIDATOR ] ESRT: All expected trading days are present. [2025-10-20T18:38:01.559338] [DEBUG] [VALIDATOR ] ESRT: Primary data validation passed. [2025-10-20T18:38:01.560158] [INFO ] [MAIN ] ESRT: Fetched primary data is valid. [2025-10-20T18:38:01.560219] [DEBUG] [LOCAL ] ESRT: Saving primary data. [2025-10-20T18:38:01.561489] [DEBUG] [LOCAL ] ESRT: Trimmed 66652 records to 46564 (keeping 200 days of data) [2025-10-20T18:38:01.827211] [DEBUG] [LOCAL ] ESRT: Primary data saved (46564 records). [2025-10-20T18:38:01.827331] [INFO ] [MAIN ] ESRT: Full history saved. [2025-10-20T18:38:01.827946] [INFO ] [MAIN ] Update Progress: 768/2367 symbols completed. ESRT took 31.3s. Estimated remaining: 3h 23m 57s. Rate: 7.8 symbols/min [2025-10-20T18:38:01.860034] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-03. [2025-10-20T18:38:02.009303] [INFO ] [MAIN ] ETB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:02.035385] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T18:38:02.053365] [DEBUG] [LOCAL ] ETB: Loaded 12085 existing records from primary storage. [2025-10-20T18:38:02.053637] [INFO ] [MAIN ] ETB: Loaded 12085 existing records. [2025-10-20T18:38:02.248340] [WARN ] [MAIN ] ETB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:02.248873] [INFO ] [MAIN ] ETB: Wiping existing primary data before full refresh. [2025-10-20T18:38:02.248961] [INFO ] [LOCAL ] ETB: Starting deletion of primary data. [2025-10-20T18:38:02.249339] [DEBUG] [LOCAL ] ETB: Deleted primary data file. [2025-10-20T18:38:02.249426] [INFO ] [MAIN ] ETB: Starting full primary refresh. [2025-10-20T18:38:02.607962] [INFO ] [REMOTE ] ETB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:02.649495] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-01 (2.40 seconds). [2025-10-20T18:38:02.661254] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-01. [2025-10-20T18:38:03.451441] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-02. [2025-10-20T18:38:03.494885] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T18:38:03.842004] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T18:38:04.251415] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-02. [2025-10-20T18:38:05.051395] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-07. [2025-10-20T18:38:05.298273] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:38:05.507979] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:38:05.851376] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-03. [2025-10-20T18:38:06.369543] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:38:06.482367] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T18:38:06.651355] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-08. [2025-10-20T18:38:07.451378] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-03. [2025-10-20T18:38:08.017625] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T18:38:08.251351] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-09. [2025-10-20T18:38:09.051381] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-04. [2025-10-20T18:38:09.349185] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:38:09.467752] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T18:38:09.763269] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:38:09.851359] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-05. [2025-10-20T18:38:10.651345] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-04. [2025-10-20T18:38:10.728703] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T18:38:11.451381] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-06. [2025-10-20T18:38:12.235516] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T18:38:12.251320] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-07. [2025-10-20T18:38:12.350286] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:38:13.057827] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-04. [2025-10-20T18:38:13.058897] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:38:13.851364] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-05. [2025-10-20T18:38:14.651435] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-08. [2025-10-20T18:38:15.226625] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T18:38:15.451415] [INFO ] [REMOTE ] ESS: Fetching primary data for 2025-10. [2025-10-20T18:38:15.536053] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T18:38:15.555751] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T18:38:16.251383] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-05. [2025-10-20T18:38:16.907065] [INFO ] [REMOTE ] ESS: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:38:16.911523] [INFO ] [REMOTE ] ESS: Full fetch complete (43527 records). [2025-10-20T18:38:16.911610] [DEBUG] [VALIDATOR ] ESS: Validating primary data. [2025-10-20T18:38:16.912213] [FATAL] [VALIDATOR ] ESS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:38:16.912295] [FATAL] [MAIN ] ESS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:38:16.912338] [WARN ] [MAIN ] ESS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:38:16.912374] [INFO ] [LOCAL ] ESS: Starting deletion of primary data. [2025-10-20T18:38:16.912450] [DEBUG] [LOCAL ] ESS: No primary data file found to delete. [2025-10-20T18:38:16.912519] [INFO ] [LOCAL ] ESS: Starting deletion of derivative data. [2025-10-20T18:38:16.912784] [DEBUG] [LOCAL ] ESS: Deleted derivative data file. [2025-10-20T18:38:16.912882] [INFO ] [MAIN ] Update Progress: 769/2367 symbols completed. ESS took 34.4s. Estimated remaining: 3h 24m 5s. Rate: 7.8 symbols/min [2025-10-20T18:38:17.056520] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-09. [2025-10-20T18:38:17.081085] [INFO ] [MAIN ] ETD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:17.106523] [DEBUG] [LOCAL ] ETD: Loaded 27663 existing records from primary storage. [2025-10-20T18:38:17.106809] [INFO ] [MAIN ] ETD: Loaded 27663 existing records. [2025-10-20T18:38:17.280517] [WARN ] [MAIN ] ETD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:17.280708] [INFO ] [MAIN ] ETD: Wiping existing primary data before full refresh. [2025-10-20T18:38:17.280762] [INFO ] [LOCAL ] ETD: Starting deletion of primary data. [2025-10-20T18:38:17.280948] [DEBUG] [LOCAL ] ETD: Deleted primary data file. [2025-10-20T18:38:17.280985] [INFO ] [MAIN ] ETD: Starting full primary refresh. [2025-10-20T18:38:17.630304] [INFO ] [REMOTE ] ETD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:17.834942] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T18:38:17.851332] [INFO ] [REMOTE ] ETB: Fetching primary data for 2025-10. [2025-10-20T18:38:18.310516] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T18:38:18.651454] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-06. [2025-10-20T18:38:18.767088] [INFO ] [REMOTE ] ETB: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:38:18.769784] [INFO ] [REMOTE ] ETB: Full fetch complete (18466 records). [2025-10-20T18:38:18.769863] [DEBUG] [VALIDATOR ] ETB: Validating primary data. [2025-10-20T18:38:19.145250] [DEBUG] [VALIDATOR ] ETB: All expected trading days are present. [2025-10-20T18:38:19.145336] [DEBUG] [VALIDATOR ] ETB: Primary data validation passed. [2025-10-20T18:38:19.145897] [INFO ] [MAIN ] ETB: Fetched primary data is valid. [2025-10-20T18:38:19.145960] [DEBUG] [LOCAL ] ETB: Saving primary data. [2025-10-20T18:38:19.146690] [DEBUG] [LOCAL ] ETB: Trimmed 18466 records to 11975 (keeping 200 days of data) [2025-10-20T18:38:19.220153] [DEBUG] [LOCAL ] ETB: Primary data saved (11975 records). [2025-10-20T18:38:19.220290] [INFO ] [MAIN ] ETB: Full history saved. [2025-10-20T18:38:19.220369] [INFO ] [MAIN ] Update Progress: 770/2367 symbols completed. ETB took 17.4s. Estimated remaining: 3h 23m 46s. Rate: 7.8 symbols/min [2025-10-20T18:38:19.390762] [INFO ] [MAIN ] ETG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:19.408897] [DEBUG] [LOCAL ] ETG: Loaded 16892 existing records from primary storage. [2025-10-20T18:38:19.409132] [INFO ] [MAIN ] ETG: Loaded 16892 existing records. [2025-10-20T18:38:19.456616] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-01. [2025-10-20T18:38:19.595460] [WARN ] [MAIN ] ETG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:19.595699] [INFO ] [MAIN ] ETG: Wiping existing primary data before full refresh. [2025-10-20T18:38:19.595800] [INFO ] [LOCAL ] ETG: Starting deletion of primary data. [2025-10-20T18:38:19.595968] [DEBUG] [LOCAL ] ETG: Deleted primary data file. [2025-10-20T18:38:19.596008] [INFO ] [MAIN ] ETG: Starting full primary refresh. [2025-10-20T18:38:19.951334] [INFO ] [REMOTE ] ETG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:20.032271] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T18:38:20.251377] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-01. [2025-10-20T18:38:20.518590] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:38:21.051394] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-02. [2025-10-20T18:38:21.415128] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:38:21.851397] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-02. [2025-10-20T18:38:22.134140] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T18:38:22.597146] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:38:22.651350] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-03. [2025-10-20T18:38:23.451405] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-06. [2025-10-20T18:38:23.748617] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T18:38:24.251433] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-04. [2025-10-20T18:38:25.051365] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-07. [2025-10-20T18:38:25.622975] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:38:25.693594] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:38:25.851392] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-05. [2025-10-20T18:38:26.625202] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T18:38:26.651331] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-08. [2025-10-20T18:38:27.109548] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T18:38:27.451440] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-03. [2025-10-20T18:38:28.251346] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-07. [2025-10-20T18:38:28.482471] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:38:28.532037] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:38:29.051425] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-04. [2025-10-20T18:38:29.851444] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-09. [2025-10-20T18:38:30.055027] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:38:30.656582] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-05. [2025-10-20T18:38:30.664977] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-07 (2.41 seconds). [2025-10-20T18:38:31.451338] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-08. [2025-10-20T18:38:31.484785] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:38:31.695571] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:38:32.251376] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-06. [2025-10-20T18:38:33.051360] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-06. [2025-10-20T18:38:33.090156] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:38:33.800538] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-08 (2.35 seconds). [2025-10-20T18:38:33.851380] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-09. [2025-10-20T18:38:34.107035] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:38:34.651449] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-07. [2025-10-20T18:38:35.451421] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-07. [2025-10-20T18:38:35.875347] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T18:38:36.059228] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T18:38:36.112304] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T18:38:36.251379] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-08. [2025-10-20T18:38:37.051419] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-08. [2025-10-20T18:38:37.722602] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:38:37.851334] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-09. [2025-10-20T18:38:38.229383] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T18:38:38.651415] [INFO ] [REMOTE ] ESTC: Fetching primary data for 2025-10. [2025-10-20T18:38:39.031002] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:38:39.451384] [INFO ] [REMOTE ] ETD: Fetching primary data for 2025-10. [2025-10-20T18:38:40.251409] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-09. [2025-10-20T18:38:40.687243] [INFO ] [REMOTE ] ESTC: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T18:38:40.694690] [INFO ] [REMOTE ] ESTC: Full fetch complete (72485 records). [2025-10-20T18:38:40.694803] [DEBUG] [VALIDATOR ] ESTC: Validating primary data. [2025-10-20T18:38:40.811308] [INFO ] [REMOTE ] ETD: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:38:40.854522] [INFO ] [REMOTE ] ETD: Full fetch complete (38642 records). [2025-10-20T18:38:40.886790] [DEBUG] [VALIDATOR ] ETD: Validating primary data. [2025-10-20T18:38:41.062364] [INFO ] [REMOTE ] ET: Fetching primary data for 2025-10. [2025-10-20T18:38:41.078252] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:38:41.399754] [DEBUG] [VALIDATOR ] ESTC: All expected trading days are present. [2025-10-20T18:38:41.405776] [DEBUG] [VALIDATOR ] ESTC: Primary data validation passed. [2025-10-20T18:38:41.406814] [INFO ] [MAIN ] ESTC: Fetched primary data is valid. [2025-10-20T18:38:41.417824] [DEBUG] [LOCAL ] ESTC: Saving primary data. [2025-10-20T18:38:41.478777] [DEBUG] [LOCAL ] ESTC: Trimmed 72485 records to 50306 (keeping 200 days of data) [2025-10-20T18:38:41.586624] [DEBUG] [VALIDATOR ] ETD: All expected trading days are present. [2025-10-20T18:38:41.586829] [DEBUG] [VALIDATOR ] ETD: Primary data validation passed. [2025-10-20T18:38:41.612311] [INFO ] [MAIN ] ETD: Fetched primary data is valid. [2025-10-20T18:38:41.612726] [DEBUG] [LOCAL ] ETD: Saving primary data. [2025-10-20T18:38:41.631718] [DEBUG] [LOCAL ] ETD: Trimmed 38642 records to 27370 (keeping 200 days of data) [2025-10-20T18:38:41.858682] [INFO ] [REMOTE ] ETG: Fetching primary data for 2025-10. [2025-10-20T18:38:41.951597] [DEBUG] [LOCAL ] ETD: Primary data saved (27370 records). [2025-10-20T18:38:41.955922] [INFO ] [MAIN ] ETD: Full history saved. [2025-10-20T18:38:41.971845] [INFO ] [MAIN ] Update Progress: 771/2367 symbols completed. ETD took 25.1s. Estimated remaining: 3h 24m 10s. Rate: 7.8 symbols/min [2025-10-20T18:38:42.024558] [DEBUG] [LOCAL ] ESTC: Primary data saved (50306 records). [2025-10-20T18:38:42.036903] [INFO ] [MAIN ] ESTC: Full history saved. [2025-10-20T18:38:42.072200] [INFO ] [MAIN ] Update Progress: 772/2367 symbols completed. ESTC took 48.9s. Estimated remaining: 3h 23m 46s. Rate: 7.8 symbols/min [2025-10-20T18:38:42.318686] [INFO ] [MAIN ] ETJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:42.371913] [INFO ] [MAIN ] ETN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:42.401159] [DEBUG] [LOCAL ] ETJ: Loaded 17031 existing records from primary storage. [2025-10-20T18:38:42.401447] [INFO ] [MAIN ] ETJ: Loaded 17031 existing records. [2025-10-20T18:38:42.622082] [WARN ] [MAIN ] ETJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:42.623822] [INFO ] [MAIN ] ETJ: Wiping existing primary data before full refresh. [2025-10-20T18:38:42.623903] [INFO ] [LOCAL ] ETJ: Starting deletion of primary data. [2025-10-20T18:38:42.624939] [DEBUG] [LOCAL ] ETJ: Deleted primary data file. [2025-10-20T18:38:42.624869] [DEBUG] [LOCAL ] ETN: Loaded 53392 existing records from primary storage. [2025-10-20T18:38:42.625106] [INFO ] [MAIN ] ETJ: Starting full primary refresh. [2025-10-20T18:38:42.630630] [INFO ] [MAIN ] ETN: Loaded 53392 existing records. [2025-10-20T18:38:43.073755] [WARN ] [MAIN ] ETN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:43.084884] [INFO ] [MAIN ] ETN: Wiping existing primary data before full refresh. [2025-10-20T18:38:43.102832] [INFO ] [LOCAL ] ETN: Starting deletion of primary data. [2025-10-20T18:38:43.132045] [DEBUG] [LOCAL ] ETN: Deleted primary data file. [2025-10-20T18:38:43.154857] [INFO ] [MAIN ] ETN: Starting full primary refresh. [2025-10-20T18:38:43.155638] [INFO ] [REMOTE ] ETG: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T18:38:43.218025] [INFO ] [REMOTE ] ETG: Full fetch complete (24799 records). [2025-10-20T18:38:43.274484] [DEBUG] [VALIDATOR ] ETG: Validating primary data. [2025-10-20T18:38:43.357823] [INFO ] [REMOTE ] ETJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:43.379926] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-01. [2025-10-20T18:38:44.032770] [INFO ] [REMOTE ] ET: Incremental fetch complete for 2025-10 (2.97 seconds). [2025-10-20T18:38:44.104901] [INFO ] [REMOTE ] ETN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:44.106993] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-01. [2025-10-20T18:38:44.113410] [INFO ] [REMOTE ] ET: Full fetch complete (77969 records). [2025-10-20T18:38:44.120133] [DEBUG] [VALIDATOR ] ET: Validating primary data. [2025-10-20T18:38:44.284965] [DEBUG] [VALIDATOR ] ETG: All expected trading days are present. [2025-10-20T18:38:44.299200] [DEBUG] [VALIDATOR ] ETG: Primary data validation passed. [2025-10-20T18:38:44.319474] [INFO ] [MAIN ] ETG: Fetched primary data is valid. [2025-10-20T18:38:44.327227] [DEBUG] [LOCAL ] ETG: Saving primary data. [2025-10-20T18:38:44.349913] [DEBUG] [LOCAL ] ETG: Trimmed 24799 records to 16667 (keeping 200 days of data) [2025-10-20T18:38:44.680384] [DEBUG] [LOCAL ] ETG: Primary data saved (16667 records). [2025-10-20T18:38:44.709890] [INFO ] [MAIN ] ETG: Full history saved. [2025-10-20T18:38:44.719130] [INFO ] [MAIN ] Update Progress: 773/2367 symbols completed. ETG took 25.5s. Estimated remaining: 3h 23m 28s. Rate: 7.8 symbols/min [2025-10-20T18:38:44.711357] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:38:44.798049] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-02. [2025-10-20T18:38:44.946858] [DEBUG] [VALIDATOR ] ET: All expected trading days are present. [2025-10-20T18:38:44.947353] [DEBUG] [VALIDATOR ] ET: Primary data validation passed. [2025-10-20T18:38:44.964409] [INFO ] [MAIN ] ET: Fetched primary data is valid. [2025-10-20T18:38:44.991580] [DEBUG] [LOCAL ] ET: Saving primary data. [2025-10-20T18:38:45.026103] [INFO ] [MAIN ] ETO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:45.027798] [DEBUG] [LOCAL ] ET: Trimmed 77969 records to 53789 (keeping 200 days of data) [2025-10-20T18:38:45.167495] [DEBUG] [LOCAL ] ETO: Loaded 7052 existing records from primary storage. [2025-10-20T18:38:45.180375] [INFO ] [MAIN ] ETO: Loaded 7052 existing records. [2025-10-20T18:38:45.629082] [WARN ] [MAIN ] ETO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:45.642762] [INFO ] [MAIN ] ETO: Wiping existing primary data before full refresh. [2025-10-20T18:38:45.642910] [INFO ] [LOCAL ] ETO: Starting deletion of primary data. [2025-10-20T18:38:45.643145] [DEBUG] [LOCAL ] ETO: Deleted primary data file. [2025-10-20T18:38:45.643246] [INFO ] [MAIN ] ETO: Starting full primary refresh. [2025-10-20T18:38:45.643070] [DEBUG] [LOCAL ] ET: Primary data saved (53789 records). [2025-10-20T18:38:45.710862] [INFO ] [MAIN ] ET: Full history saved. [2025-10-20T18:38:45.724060] [INFO ] [MAIN ] Update Progress: 774/2367 symbols completed. ET took 51.0s. Estimated remaining: 3h 23m 7s. Rate: 7.8 symbols/min [2025-10-20T18:38:46.081432] [INFO ] [MAIN ] ETR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:38:46.176405] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:38:46.232597] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-03. [2025-10-20T18:38:46.255933] [INFO ] [REMOTE ] ETO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:46.257564] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-01. [2025-10-20T18:38:46.300184] [DEBUG] [LOCAL ] ETR: Loaded 53984 existing records from primary storage. [2025-10-20T18:38:46.300693] [INFO ] [MAIN ] ETR: Loaded 53984 existing records. [2025-10-20T18:38:46.444445] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-01 (2.34 seconds). [2025-10-20T18:38:46.530543] [WARN ] [MAIN ] ETR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:38:46.530791] [INFO ] [MAIN ] ETR: Wiping existing primary data before full refresh. [2025-10-20T18:38:46.530843] [INFO ] [LOCAL ] ETR: Starting deletion of primary data. [2025-10-20T18:38:46.531050] [DEBUG] [LOCAL ] ETR: Deleted primary data file. [2025-10-20T18:38:46.531088] [INFO ] [MAIN ] ETR: Starting full primary refresh. [2025-10-20T18:38:46.666476] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-02. [2025-10-20T18:38:46.882347] [INFO ] [REMOTE ] ETR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:38:47.143691] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:38:47.317405] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T18:38:47.451357] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-04. [2025-10-20T18:38:48.251354] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-02. [2025-10-20T18:38:48.558535] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T18:38:48.743671] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-02 (2.08 seconds). [2025-10-20T18:38:48.761574] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T18:38:49.051394] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-05. [2025-10-20T18:38:49.851395] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-01. [2025-10-20T18:38:49.885166] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:38:50.651375] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-03. [2025-10-20T18:38:51.451363] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-03. [2025-10-20T18:38:51.691169] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:38:52.090541] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:38:52.251368] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-02. [2025-10-20T18:38:52.686311] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T18:38:53.051420] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-04. [2025-10-20T18:38:53.600056] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:38:53.851411] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-04. [2025-10-20T18:38:53.979490] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:38:54.651412] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-06. [2025-10-20T18:38:55.451396] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-03. [2025-10-20T18:38:55.600820] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:38:55.874024] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T18:38:56.251392] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-07. [2025-10-20T18:38:57.051356] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-05. [2025-10-20T18:38:57.137917] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:38:57.320609] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T18:38:57.851383] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-04. [2025-10-20T18:38:57.866324] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T18:38:58.651359] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-08. [2025-10-20T18:38:59.451389] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-05. [2025-10-20T18:38:59.635017] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:38:59.670180] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:39:00.251377] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-05. [2025-10-20T18:39:01.051424] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-09. [2025-10-20T18:39:01.707582] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-10-20T18:39:01.851339] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-06. [2025-10-20T18:39:02.133483] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T18:39:02.376546] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T18:39:02.651363] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-06. [2025-10-20T18:39:03.451368] [INFO ] [REMOTE ] ETJ: Fetching primary data for 2025-10. [2025-10-20T18:39:03.576960] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T18:39:04.251350] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-06. [2025-10-20T18:39:04.426832] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:39:04.473768] [INFO ] [REMOTE ] ETJ: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:39:04.476660] [INFO ] [REMOTE ] ETJ: Full fetch complete (25632 records). [2025-10-20T18:39:04.476736] [DEBUG] [VALIDATOR ] ETJ: Validating primary data. [2025-10-20T18:39:04.818541] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T18:39:04.865353] [DEBUG] [VALIDATOR ] ETJ: All expected trading days are present. [2025-10-20T18:39:04.865431] [DEBUG] [VALIDATOR ] ETJ: Primary data validation passed. [2025-10-20T18:39:04.866026] [INFO ] [MAIN ] ETJ: Fetched primary data is valid. [2025-10-20T18:39:04.866089] [DEBUG] [LOCAL ] ETJ: Saving primary data. [2025-10-20T18:39:04.866732] [DEBUG] [LOCAL ] ETJ: Trimmed 25632 records to 16676 (keeping 200 days of data) [2025-10-20T18:39:04.982468] [DEBUG] [LOCAL ] ETJ: Primary data saved (16676 records). [2025-10-20T18:39:04.982613] [INFO ] [MAIN ] ETJ: Full history saved. [2025-10-20T18:39:04.982721] [INFO ] [MAIN ] Update Progress: 775/2367 symbols completed. ETJ took 23.0s. Estimated remaining: 3h 23m 23s. Rate: 7.8 symbols/min [2025-10-20T18:39:05.056465] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-07. [2025-10-20T18:39:05.151203] [INFO ] [MAIN ] ETSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:05.199771] [DEBUG] [LOCAL ] ETSY: Loaded 54353 existing records from primary storage. [2025-10-20T18:39:05.200107] [INFO ] [MAIN ] ETSY: Loaded 54353 existing records. [2025-10-20T18:39:05.370745] [WARN ] [MAIN ] ETSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:05.370919] [INFO ] [MAIN ] ETSY: Wiping existing primary data before full refresh. [2025-10-20T18:39:05.370966] [INFO ] [LOCAL ] ETSY: Starting deletion of primary data. [2025-10-20T18:39:05.371262] [DEBUG] [LOCAL ] ETSY: Deleted primary data file. [2025-10-20T18:39:05.371318] [INFO ] [MAIN ] ETSY: Starting full primary refresh. [2025-10-20T18:39:05.722124] [INFO ] [REMOTE ] ETSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:05.821303] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T18:39:05.851335] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-08. [2025-10-20T18:39:06.518113] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T18:39:06.651343] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-07. [2025-10-20T18:39:07.451393] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-01. [2025-10-20T18:39:08.251356] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-09. [2025-10-20T18:39:08.500295] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:39:09.051337] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-08. [2025-10-20T18:39:09.057583] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:39:09.245903] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:39:09.851377] [INFO ] [REMOTE ] ETO: Fetching primary data for 2025-10. [2025-10-20T18:39:10.651389] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-02. [2025-10-20T18:39:10.660250] [INFO ] [REMOTE ] ETO: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:39:10.662112] [INFO ] [REMOTE ] ETO: Full fetch complete (9731 records). [2025-10-20T18:39:10.662182] [DEBUG] [VALIDATOR ] ETO: Validating primary data. [2025-10-20T18:39:11.055181] [DEBUG] [VALIDATOR ] ETO: All expected trading days are present. [2025-10-20T18:39:11.055346] [DEBUG] [VALIDATOR ] ETO: Primary data validation passed. [2025-10-20T18:39:11.055630] [INFO ] [MAIN ] ETO: Fetched primary data is valid. [2025-10-20T18:39:11.055765] [DEBUG] [LOCAL ] ETO: Saving primary data. [2025-10-20T18:39:11.056207] [DEBUG] [LOCAL ] ETO: Trimmed 9731 records to 7007 (keeping 200 days of data) [2025-10-20T18:39:11.098376] [DEBUG] [LOCAL ] ETO: Primary data saved (7007 records). [2025-10-20T18:39:11.098466] [INFO ] [MAIN ] ETO: Full history saved. [2025-10-20T18:39:11.098539] [INFO ] [MAIN ] Update Progress: 776/2367 symbols completed. ETO took 26.4s. Estimated remaining: 3h 23m 12s. Rate: 7.8 symbols/min [2025-10-20T18:39:11.267208] [INFO ] [MAIN ] ETV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:11.289746] [DEBUG] [LOCAL ] ETV: Loaded 22515 existing records from primary storage. [2025-10-20T18:39:11.290000] [INFO ] [MAIN ] ETV: Loaded 22515 existing records. [2025-10-20T18:39:11.464297] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-07. [2025-10-20T18:39:11.469158] [WARN ] [MAIN ] ETV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:11.469437] [INFO ] [MAIN ] ETV: Wiping existing primary data before full refresh. [2025-10-20T18:39:11.470329] [INFO ] [LOCAL ] ETV: Starting deletion of primary data. [2025-10-20T18:39:11.470589] [DEBUG] [LOCAL ] ETV: Deleted primary data file. [2025-10-20T18:39:11.470693] [INFO ] [MAIN ] ETV: Starting full primary refresh. [2025-10-20T18:39:11.654030] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-08 (2.60 seconds). [2025-10-20T18:39:11.888275] [INFO ] [REMOTE ] ETV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:12.251375] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-01. [2025-10-20T18:39:12.404978] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:39:13.051385] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-09. [2025-10-20T18:39:13.113749] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:39:13.528497] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:39:13.851379] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-08. [2025-10-20T18:39:14.652304] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-02. [2025-10-20T18:39:15.106775] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T18:39:15.341749] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T18:39:15.451356] [INFO ] [REMOTE ] ETN: Fetching primary data for 2025-10. [2025-10-20T18:39:15.631307] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:39:16.251384] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-09. [2025-10-20T18:39:17.051398] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-03. [2025-10-20T18:39:17.650094] [INFO ] [REMOTE ] ETN: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T18:39:17.657915] [INFO ] [REMOTE ] ETN: Full fetch complete (76644 records). [2025-10-20T18:39:17.658105] [DEBUG] [VALIDATOR ] ETN: Validating primary data. [2025-10-20T18:39:17.867898] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-03. [2025-10-20T18:39:18.060907] [DEBUG] [VALIDATOR ] ETN: All expected trading days are present. [2025-10-20T18:39:18.061166] [DEBUG] [VALIDATOR ] ETN: Primary data validation passed. [2025-10-20T18:39:18.062117] [INFO ] [MAIN ] ETN: Fetched primary data is valid. [2025-10-20T18:39:18.062206] [DEBUG] [LOCAL ] ETN: Saving primary data. [2025-10-20T18:39:18.063824] [DEBUG] [LOCAL ] ETN: Trimmed 76644 records to 52584 (keeping 200 days of data) [2025-10-20T18:39:18.188359] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T18:39:18.227299] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T18:39:18.459511] [DEBUG] [LOCAL ] ETN: Primary data saved (52584 records). [2025-10-20T18:39:18.459680] [INFO ] [MAIN ] ETN: Full history saved. [2025-10-20T18:39:18.459777] [INFO ] [MAIN ] Update Progress: 777/2367 symbols completed. ETN took 36.4s. Estimated remaining: 3h 23m 4s. Rate: 7.8 symbols/min [2025-10-20T18:39:18.628574] [INFO ] [MAIN ] ETW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:18.653438] [INFO ] [REMOTE ] ETR: Fetching primary data for 2025-10. [2025-10-20T18:39:18.654062] [DEBUG] [LOCAL ] ETW: Loaded 25887 existing records from primary storage. [2025-10-20T18:39:18.655181] [INFO ] [MAIN ] ETW: Loaded 25887 existing records. [2025-10-20T18:39:18.839547] [WARN ] [MAIN ] ETW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:18.839846] [INFO ] [MAIN ] ETW: Wiping existing primary data before full refresh. [2025-10-20T18:39:18.839895] [INFO ] [LOCAL ] ETW: Starting deletion of primary data. [2025-10-20T18:39:18.840491] [DEBUG] [LOCAL ] ETW: Deleted primary data file. [2025-10-20T18:39:18.840551] [INFO ] [MAIN ] ETW: Starting full primary refresh. [2025-10-20T18:39:19.202545] [INFO ] [REMOTE ] ETW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:19.451360] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-01. [2025-10-20T18:39:20.020846] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T18:39:20.106818] [INFO ] [REMOTE ] ETR: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T18:39:20.114839] [INFO ] [REMOTE ] ETR: Full fetch complete (77276 records). [2025-10-20T18:39:20.114997] [DEBUG] [VALIDATOR ] ETR: Validating primary data. [2025-10-20T18:39:20.252880] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-04. [2025-10-20T18:39:20.297127] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T18:39:20.549831] [DEBUG] [VALIDATOR ] ETR: All expected trading days are present. [2025-10-20T18:39:20.549918] [DEBUG] [VALIDATOR ] ETR: Primary data validation passed. [2025-10-20T18:39:20.550839] [INFO ] [MAIN ] ETR: Fetched primary data is valid. [2025-10-20T18:39:20.550902] [DEBUG] [LOCAL ] ETR: Saving primary data. [2025-10-20T18:39:20.552328] [DEBUG] [LOCAL ] ETR: Trimmed 77276 records to 53196 (keeping 200 days of data) [2025-10-20T18:39:20.884098] [DEBUG] [LOCAL ] ETR: Primary data saved (53196 records). [2025-10-20T18:39:20.884243] [INFO ] [MAIN ] ETR: Full history saved. [2025-10-20T18:39:20.884323] [INFO ] [MAIN ] Update Progress: 778/2367 symbols completed. ETR took 35.2s. Estimated remaining: 3h 22m 46s. Rate: 7.8 symbols/min [2025-10-20T18:39:21.068715] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-04. [2025-10-20T18:39:21.072293] [INFO ] [MAIN ] ETX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:21.079235] [DEBUG] [LOCAL ] ETX: Loaded 5300 existing records from primary storage. [2025-10-20T18:39:21.079397] [INFO ] [MAIN ] ETX: Loaded 5300 existing records. [2025-10-20T18:39:21.254408] [WARN ] [MAIN ] ETX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:21.254617] [INFO ] [MAIN ] ETX: Wiping existing primary data before full refresh. [2025-10-20T18:39:21.254693] [INFO ] [LOCAL ] ETX: Starting deletion of primary data. [2025-10-20T18:39:21.254889] [DEBUG] [LOCAL ] ETX: Deleted primary data file. [2025-10-20T18:39:21.254944] [INFO ] [MAIN ] ETX: Starting full primary refresh. [2025-10-20T18:39:21.426842] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T18:39:21.667245] [INFO ] [REMOTE ] ETX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:21.851366] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-05. [2025-10-20T18:39:22.476415] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:39:22.651377] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-01. [2025-10-20T18:39:23.051018] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T18:39:23.424307] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:39:23.451439] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-02. [2025-10-20T18:39:24.028280] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T18:39:24.251409] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-05. [2025-10-20T18:39:25.051385] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-06. [2025-10-20T18:39:25.851376] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-02. [2025-10-20T18:39:26.000062] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:39:26.199551] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T18:39:26.651358] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-03. [2025-10-20T18:39:26.900983] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:39:27.199401] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T18:39:27.451371] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-07. [2025-10-20T18:39:28.251420] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-03. [2025-10-20T18:39:28.519322] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:39:29.051382] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-04. [2025-10-20T18:39:29.355436] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T18:39:29.593300] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:39:29.851370] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-04. [2025-10-20T18:39:30.651399] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-08. [2025-10-20T18:39:31.168514] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T18:39:31.451378] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-05. [2025-10-20T18:39:31.730029] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:39:32.251362] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-09. [2025-10-20T18:39:32.571119] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T18:39:33.051360] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-06. [2025-10-20T18:39:33.417271] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T18:39:33.851394] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-06. [2025-10-20T18:39:34.131057] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T18:39:34.651367] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-05. [2025-10-20T18:39:35.300732] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T18:39:35.451359] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-06. [2025-10-20T18:39:35.625634] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:39:36.180027] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:39:36.251396] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-07. [2025-10-20T18:39:37.051392] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-07. [2025-10-20T18:39:37.640197] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:39:37.851418] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-08. [2025-10-20T18:39:38.176937] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T18:39:38.547404] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T18:39:38.651394] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-07. [2025-10-20T18:39:39.451349] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-09. [2025-10-20T18:39:39.723139] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:39:40.028908] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T18:39:40.251392] [INFO ] [REMOTE ] ETX: Fetching primary data for 2025-10. [2025-10-20T18:39:40.959463] [INFO ] [REMOTE ] ETX: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:39:40.960708] [INFO ] [REMOTE ] ETX: Full fetch complete (6789 records). [2025-10-20T18:39:40.960776] [DEBUG] [VALIDATOR ] ETX: Validating primary data. [2025-10-20T18:39:41.059504] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-08. [2025-10-20T18:39:41.316004] [DEBUG] [VALIDATOR ] ETX: All expected trading days are present. [2025-10-20T18:39:41.316084] [DEBUG] [VALIDATOR ] ETX: Primary data validation passed. [2025-10-20T18:39:41.316299] [INFO ] [MAIN ] ETX: Fetched primary data is valid. [2025-10-20T18:39:41.316336] [DEBUG] [LOCAL ] ETX: Saving primary data. [2025-10-20T18:39:41.316737] [DEBUG] [LOCAL ] ETX: Trimmed 6789 records to 5277 (keeping 200 days of data) [2025-10-20T18:39:41.348258] [DEBUG] [LOCAL ] ETX: Primary data saved (5277 records). [2025-10-20T18:39:41.348361] [INFO ] [MAIN ] ETX: Full history saved. [2025-10-20T18:39:41.348432] [INFO ] [MAIN ] Update Progress: 779/2367 symbols completed. ETX took 20.5s. Estimated remaining: 3h 23m 4s. Rate: 7.8 symbols/min [2025-10-20T18:39:41.523314] [INFO ] [MAIN ] ETY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:41.545979] [DEBUG] [LOCAL ] ETY: Loaded 23210 existing records from primary storage. [2025-10-20T18:39:41.546227] [INFO ] [MAIN ] ETY: Loaded 23210 existing records. [2025-10-20T18:39:41.719785] [WARN ] [MAIN ] ETY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:41.720015] [INFO ] [MAIN ] ETY: Wiping existing primary data before full refresh. [2025-10-20T18:39:41.720062] [INFO ] [LOCAL ] ETY: Starting deletion of primary data. [2025-10-20T18:39:41.720218] [DEBUG] [LOCAL ] ETY: Deleted primary data file. [2025-10-20T18:39:41.720255] [INFO ] [MAIN ] ETY: Starting full primary refresh. [2025-10-20T18:39:41.862082] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-08. [2025-10-20T18:39:42.100011] [INFO ] [REMOTE ] ETY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:42.651418] [INFO ] [REMOTE ] ETV: Fetching primary data for 2025-10. [2025-10-20T18:39:43.224871] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:39:43.268374] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T18:39:43.451396] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-01. [2025-10-20T18:39:43.620517] [INFO ] [REMOTE ] ETV: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T18:39:43.624882] [INFO ] [REMOTE ] ETV: Full fetch complete (32456 records). [2025-10-20T18:39:43.624978] [DEBUG] [VALIDATOR ] ETV: Validating primary data. [2025-10-20T18:39:44.013902] [DEBUG] [VALIDATOR ] ETV: All expected trading days are present. [2025-10-20T18:39:44.013987] [DEBUG] [VALIDATOR ] ETV: Primary data validation passed. [2025-10-20T18:39:44.014538] [INFO ] [MAIN ] ETV: Fetched primary data is valid. [2025-10-20T18:39:44.014598] [DEBUG] [LOCAL ] ETV: Saving primary data. [2025-10-20T18:39:44.015454] [DEBUG] [LOCAL ] ETV: Trimmed 32456 records to 22127 (keeping 200 days of data) [2025-10-20T18:39:44.146496] [DEBUG] [LOCAL ] ETV: Primary data saved (22127 records). [2025-10-20T18:39:44.146617] [INFO ] [MAIN ] ETV: Full history saved. [2025-10-20T18:39:44.146722] [INFO ] [MAIN ] Update Progress: 780/2367 symbols completed. ETV took 33.0s. Estimated remaining: 3h 22m 46s. Rate: 7.8 symbols/min [2025-10-20T18:39:44.256450] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-09. [2025-10-20T18:39:44.318663] [INFO ] [MAIN ] EVC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:44.337214] [DEBUG] [LOCAL ] EVC: Loaded 18307 existing records from primary storage. [2025-10-20T18:39:44.337421] [INFO ] [MAIN ] EVC: Loaded 18307 existing records. [2025-10-20T18:39:44.521993] [WARN ] [MAIN ] EVC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:44.522234] [INFO ] [MAIN ] EVC: Wiping existing primary data before full refresh. [2025-10-20T18:39:44.522283] [INFO ] [LOCAL ] EVC: Starting deletion of primary data. [2025-10-20T18:39:44.522450] [DEBUG] [LOCAL ] EVC: Deleted primary data file. [2025-10-20T18:39:44.522489] [INFO ] [MAIN ] EVC: Starting full primary refresh. [2025-10-20T18:39:44.890442] [INFO ] [REMOTE ] EVC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:44.904747] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:39:45.051459] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-01. [2025-10-20T18:39:45.851418] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-09. [2025-10-20T18:39:46.071826] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T18:39:46.542620] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:39:46.651443] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-02. [2025-10-20T18:39:46.933061] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T18:39:47.451413] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-02. [2025-10-20T18:39:47.820878] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T18:39:48.251396] [INFO ] [REMOTE ] ETW: Fetching primary data for 2025-10. [2025-10-20T18:39:48.451174] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T18:39:49.051396] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-03. [2025-10-20T18:39:49.619018] [INFO ] [REMOTE ] ETW: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:39:49.623954] [INFO ] [REMOTE ] ETW: Full fetch complete (36669 records). [2025-10-20T18:39:49.624081] [DEBUG] [VALIDATOR ] ETW: Validating primary data. [2025-10-20T18:39:49.854224] [INFO ] [REMOTE ] ETSY: Fetching primary data for 2025-10. [2025-10-20T18:39:49.999694] [DEBUG] [VALIDATOR ] ETW: All expected trading days are present. [2025-10-20T18:39:49.999794] [DEBUG] [VALIDATOR ] ETW: Primary data validation passed. [2025-10-20T18:39:50.000459] [INFO ] [MAIN ] ETW: Fetched primary data is valid. [2025-10-20T18:39:50.000500] [DEBUG] [LOCAL ] ETW: Saving primary data. [2025-10-20T18:39:50.001449] [DEBUG] [LOCAL ] ETW: Trimmed 36669 records to 25419 (keeping 200 days of data) [2025-10-20T18:39:50.149667] [DEBUG] [LOCAL ] ETW: Primary data saved (25419 records). [2025-10-20T18:39:50.149979] [INFO ] [MAIN ] ETW: Full history saved. [2025-10-20T18:39:50.150095] [INFO ] [MAIN ] Update Progress: 781/2367 symbols completed. ETW took 31.7s. Estimated remaining: 3h 22m 35s. Rate: 7.8 symbols/min [2025-10-20T18:39:50.268127] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:39:50.345007] [INFO ] [MAIN ] EVEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:50.375684] [DEBUG] [LOCAL ] EVEX: Loaded 33091 existing records from primary storage. [2025-10-20T18:39:50.375956] [INFO ] [MAIN ] EVEX: Loaded 33091 existing records. [2025-10-20T18:39:50.555800] [WARN ] [MAIN ] EVEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:50.556013] [INFO ] [MAIN ] EVEX: Wiping existing primary data before full refresh. [2025-10-20T18:39:50.556060] [INFO ] [LOCAL ] EVEX: Starting deletion of primary data. [2025-10-20T18:39:50.556232] [DEBUG] [LOCAL ] EVEX: Deleted primary data file. [2025-10-20T18:39:50.556269] [INFO ] [MAIN ] EVEX: Starting full primary refresh. [2025-10-20T18:39:50.654483] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-03. [2025-10-20T18:39:50.910201] [INFO ] [REMOTE ] EVEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:51.451352] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-04. [2025-10-20T18:39:51.931439] [INFO ] [REMOTE ] ETSY: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T18:39:51.939687] [INFO ] [REMOTE ] ETSY: Full fetch complete (77662 records). [2025-10-20T18:39:51.939861] [DEBUG] [VALIDATOR ] ETSY: Validating primary data. [2025-10-20T18:39:52.070283] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:39:52.256492] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-01. [2025-10-20T18:39:52.367293] [DEBUG] [VALIDATOR ] ETSY: All expected trading days are present. [2025-10-20T18:39:52.367409] [DEBUG] [VALIDATOR ] ETSY: Primary data validation passed. [2025-10-20T18:39:52.368405] [INFO ] [MAIN ] ETSY: Fetched primary data is valid. [2025-10-20T18:39:52.368584] [DEBUG] [LOCAL ] ETSY: Saving primary data. [2025-10-20T18:39:52.370413] [DEBUG] [LOCAL ] ETSY: Trimmed 77662 records to 53569 (keeping 200 days of data) [2025-10-20T18:39:52.501478] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:39:52.698451] [DEBUG] [LOCAL ] ETSY: Primary data saved (53569 records). [2025-10-20T18:39:52.698582] [INFO ] [MAIN ] ETSY: Full history saved. [2025-10-20T18:39:52.698678] [INFO ] [MAIN ] Update Progress: 782/2367 symbols completed. ETSY took 47.7s. Estimated remaining: 3h 22m 17s. Rate: 7.8 symbols/min [2025-10-20T18:39:52.864338] [INFO ] [MAIN ] EVF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:39:52.874430] [DEBUG] [LOCAL ] EVF: Loaded 9913 existing records from primary storage. [2025-10-20T18:39:52.874629] [INFO ] [MAIN ] EVF: Loaded 9913 existing records. [2025-10-20T18:39:53.052351] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-04. [2025-10-20T18:39:53.052238] [WARN ] [MAIN ] EVF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:39:53.052613] [INFO ] [MAIN ] EVF: Wiping existing primary data before full refresh. [2025-10-20T18:39:53.052703] [INFO ] [LOCAL ] EVF: Starting deletion of primary data. [2025-10-20T18:39:53.053587] [DEBUG] [LOCAL ] EVF: Deleted primary data file. [2025-10-20T18:39:53.053670] [INFO ] [MAIN ] EVF: Starting full primary refresh. [2025-10-20T18:39:53.429967] [INFO ] [REMOTE ] EVF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:39:53.683514] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:39:53.851431] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-01. [2025-10-20T18:39:54.314134] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:39:54.607180] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T18:39:54.651383] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-02. [2025-10-20T18:39:55.451356] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-05. [2025-10-20T18:39:55.813777] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:39:56.252846] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-03. [2025-10-20T18:39:56.322590] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T18:39:57.051422] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-02. [2025-10-20T18:39:57.422174] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T18:39:57.476286] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T18:39:57.851437] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-05. [2025-10-20T18:39:58.651389] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-06. [2025-10-20T18:39:59.189731] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:39:59.451387] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-04. [2025-10-20T18:39:59.926124] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T18:40:00.251359] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-06. [2025-10-20T18:40:00.479228] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:40:01.051474] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-05. [2025-10-20T18:40:01.316175] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:40:01.851362] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-07. [2025-10-20T18:40:01.879601] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:40:02.651365] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-07. [2025-10-20T18:40:02.971768] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:40:03.451348] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-08. [2025-10-20T18:40:03.766831] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:40:04.251423] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-03. [2025-10-20T18:40:04.487392] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:40:05.051389] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-08. [2025-10-20T18:40:05.281464] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T18:40:05.851370] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-04. [2025-10-20T18:40:06.146722] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T18:40:06.651379] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-09. [2025-10-20T18:40:06.817792] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:40:07.319096] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T18:40:07.451376] [INFO ] [REMOTE ] EVC: Fetching primary data for 2025-10. [2025-10-20T18:40:08.251401] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-09. [2025-10-20T18:40:08.418407] [INFO ] [REMOTE ] EVC: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T18:40:08.421561] [INFO ] [REMOTE ] EVC: Full fetch complete (28839 records). [2025-10-20T18:40:08.421674] [DEBUG] [VALIDATOR ] EVC: Validating primary data. [2025-10-20T18:40:08.814852] [DEBUG] [VALIDATOR ] EVC: All expected trading days are present. [2025-10-20T18:40:08.814935] [DEBUG] [VALIDATOR ] EVC: Primary data validation passed. [2025-10-20T18:40:08.815491] [INFO ] [MAIN ] EVC: Fetched primary data is valid. [2025-10-20T18:40:08.815530] [DEBUG] [LOCAL ] EVC: Saving primary data. [2025-10-20T18:40:08.816269] [DEBUG] [LOCAL ] EVC: Trimmed 28839 records to 18043 (keeping 200 days of data) [2025-10-20T18:40:08.919792] [DEBUG] [LOCAL ] EVC: Primary data saved (18043 records). [2025-10-20T18:40:08.919982] [INFO ] [MAIN ] EVC: Full history saved. [2025-10-20T18:40:08.920077] [INFO ] [MAIN ] Update Progress: 783/2367 symbols completed. EVC took 24.8s. Estimated remaining: 3h 22m 27s. Rate: 7.8 symbols/min [2025-10-20T18:40:09.056488] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-05. [2025-10-20T18:40:09.089581] [INFO ] [MAIN ] EVG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:09.097505] [DEBUG] [LOCAL ] EVG: Loaded 6021 existing records from primary storage. [2025-10-20T18:40:09.097826] [INFO ] [MAIN ] EVG: Loaded 6021 existing records. [2025-10-20T18:40:09.283264] [WARN ] [MAIN ] EVG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:09.283549] [INFO ] [MAIN ] EVG: Wiping existing primary data before full refresh. [2025-10-20T18:40:09.283599] [INFO ] [LOCAL ] EVG: Starting deletion of primary data. [2025-10-20T18:40:09.283811] [DEBUG] [LOCAL ] EVG: Deleted primary data file. [2025-10-20T18:40:09.283868] [INFO ] [MAIN ] EVG: Starting full primary refresh. [2025-10-20T18:40:09.584562] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:40:09.660838] [INFO ] [REMOTE ] EVG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:09.667199] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:40:09.851363] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-01. [2025-10-20T18:40:10.533689] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:40:10.651342] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-02. [2025-10-20T18:40:11.301049] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T18:40:11.451358] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-06. [2025-10-20T18:40:12.165660] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:40:12.251423] [INFO ] [REMOTE ] ETY: Fetching primary data for 2025-10. [2025-10-20T18:40:13.051410] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-07. [2025-10-20T18:40:13.851361] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-03. [2025-10-20T18:40:13.875901] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T18:40:14.024998] [INFO ] [REMOTE ] ETY: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:40:14.028809] [INFO ] [REMOTE ] ETY: Full fetch complete (34178 records). [2025-10-20T18:40:14.028892] [DEBUG] [VALIDATOR ] ETY: Validating primary data. [2025-10-20T18:40:14.394356] [DEBUG] [VALIDATOR ] ETY: All expected trading days are present. [2025-10-20T18:40:14.394474] [DEBUG] [VALIDATOR ] ETY: Primary data validation passed. [2025-10-20T18:40:14.395231] [INFO ] [MAIN ] ETY: Fetched primary data is valid. [2025-10-20T18:40:14.395313] [DEBUG] [LOCAL ] ETY: Saving primary data. [2025-10-20T18:40:14.396406] [DEBUG] [LOCAL ] ETY: Trimmed 34178 records to 22685 (keeping 200 days of data) [2025-10-20T18:40:14.495482] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:40:14.540403] [DEBUG] [LOCAL ] ETY: Primary data saved (22685 records). [2025-10-20T18:40:14.540529] [INFO ] [MAIN ] ETY: Full history saved. [2025-10-20T18:40:14.540602] [INFO ] [MAIN ] Update Progress: 784/2367 symbols completed. ETY took 33.2s. Estimated remaining: 3h 22m 15s. Rate: 7.8 symbols/min [2025-10-20T18:40:14.654076] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-08. [2025-10-20T18:40:14.722496] [INFO ] [MAIN ] EVH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:14.768909] [DEBUG] [LOCAL ] EVH: Loaded 50335 existing records from primary storage. [2025-10-20T18:40:14.769288] [INFO ] [MAIN ] EVH: Loaded 50335 existing records. [2025-10-20T18:40:14.953430] [WARN ] [MAIN ] EVH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:14.953662] [INFO ] [MAIN ] EVH: Wiping existing primary data before full refresh. [2025-10-20T18:40:14.953723] [INFO ] [LOCAL ] EVH: Starting deletion of primary data. [2025-10-20T18:40:14.953926] [DEBUG] [LOCAL ] EVH: Deleted primary data file. [2025-10-20T18:40:14.953983] [INFO ] [MAIN ] EVH: Starting full primary refresh. [2025-10-20T18:40:15.314552] [INFO ] [REMOTE ] EVH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:15.451338] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-06. [2025-10-20T18:40:15.581906] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:40:16.251382] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-01. [2025-10-20T18:40:16.736589] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:40:17.051358] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-07. [2025-10-20T18:40:17.851364] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-09. [2025-10-20T18:40:17.997284] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T18:40:18.450262] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:40:18.472585] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T18:40:18.651397] [INFO ] [REMOTE ] EVF: Fetching primary data for 2025-10. [2025-10-20T18:40:19.451397] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-04. [2025-10-20T18:40:19.656487] [INFO ] [REMOTE ] EVF: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T18:40:19.658907] [INFO ] [REMOTE ] EVF: Full fetch complete (12846 records). [2025-10-20T18:40:19.658989] [DEBUG] [VALIDATOR ] EVF: Validating primary data. [2025-10-20T18:40:20.036516] [DEBUG] [VALIDATOR ] EVF: All expected trading days are present. [2025-10-20T18:40:20.036598] [DEBUG] [VALIDATOR ] EVF: Primary data validation passed. [2025-10-20T18:40:20.036911] [INFO ] [MAIN ] EVF: Fetched primary data is valid. [2025-10-20T18:40:20.036971] [DEBUG] [LOCAL ] EVF: Saving primary data. [2025-10-20T18:40:20.037444] [DEBUG] [LOCAL ] EVF: Trimmed 12846 records to 9820 (keeping 200 days of data) [2025-10-20T18:40:20.094921] [DEBUG] [LOCAL ] EVF: Primary data saved (9820 records). [2025-10-20T18:40:20.095049] [INFO ] [MAIN ] EVF: Full history saved. [2025-10-20T18:40:20.095128] [INFO ] [MAIN ] Update Progress: 785/2367 symbols completed. EVF took 27.4s. Estimated remaining: 3h 22m 3s. Rate: 7.8 symbols/min [2025-10-20T18:40:20.267305] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-08. [2025-10-20T18:40:20.271337] [INFO ] [MAIN ] EVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:20.285238] [DEBUG] [LOCAL ] EVN: Loaded 10887 existing records from primary storage. [2025-10-20T18:40:20.285462] [INFO ] [MAIN ] EVN: Loaded 10887 existing records. [2025-10-20T18:40:20.392858] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T18:40:20.475493] [WARN ] [MAIN ] EVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:20.475820] [INFO ] [MAIN ] EVN: Wiping existing primary data before full refresh. [2025-10-20T18:40:20.475875] [INFO ] [LOCAL ] EVN: Starting deletion of primary data. [2025-10-20T18:40:20.476111] [DEBUG] [LOCAL ] EVN: Deleted primary data file. [2025-10-20T18:40:20.476196] [INFO ] [MAIN ] EVN: Starting full primary refresh. [2025-10-20T18:40:20.826104] [INFO ] [REMOTE ] EVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:21.051347] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-01. [2025-10-20T18:40:21.857176] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-02. [2025-10-20T18:40:21.868239] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:40:22.161502] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:40:22.651367] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-05. [2025-10-20T18:40:23.245155] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:40:23.430010] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:40:23.451323] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-03. [2025-10-20T18:40:24.251413] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-09. [2025-10-20T18:40:25.051361] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-06. [2025-10-20T18:40:25.365150] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:40:25.648154] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T18:40:25.851374] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-04. [2025-10-20T18:40:25.940535] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:40:26.651406] [INFO ] [REMOTE ] EVEX: Fetching primary data for 2025-10. [2025-10-20T18:40:27.352083] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:40:27.451372] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-05. [2025-10-20T18:40:28.251364] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-02. [2025-10-20T18:40:28.325505] [INFO ] [REMOTE ] EVEX: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:40:28.332524] [INFO ] [REMOTE ] EVEX: Full fetch complete (43835 records). [2025-10-20T18:40:28.332633] [DEBUG] [VALIDATOR ] EVEX: Validating primary data. [2025-10-20T18:40:28.698843] [DEBUG] [VALIDATOR ] EVEX: All expected trading days are present. [2025-10-20T18:40:28.698978] [DEBUG] [VALIDATOR ] EVEX: Primary data validation passed. [2025-10-20T18:40:28.699736] [INFO ] [MAIN ] EVEX: Fetched primary data is valid. [2025-10-20T18:40:28.699804] [DEBUG] [LOCAL ] EVEX: Saving primary data. [2025-10-20T18:40:28.700782] [DEBUG] [LOCAL ] EVEX: Trimmed 43835 records to 32891 (keeping 200 days of data) [2025-10-20T18:40:28.879922] [DEBUG] [LOCAL ] EVEX: Primary data saved (32891 records). [2025-10-20T18:40:28.880090] [INFO ] [MAIN ] EVEX: Full history saved. [2025-10-20T18:40:28.880250] [INFO ] [MAIN ] Update Progress: 786/2367 symbols completed. EVEX took 38.7s. Estimated remaining: 3h 21m 58s. Rate: 7.8 symbols/min [2025-10-20T18:40:29.081486] [INFO ] [MAIN ] EVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:29.079118] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-07. [2025-10-20T18:40:29.085179] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T18:40:29.103759] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T18:40:29.136838] [DEBUG] [LOCAL ] EVR: Loaded 34821 existing records from primary storage. [2025-10-20T18:40:29.137139] [INFO ] [MAIN ] EVR: Loaded 34821 existing records. [2025-10-20T18:40:29.315848] [WARN ] [MAIN ] EVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:29.316097] [INFO ] [MAIN ] EVR: Wiping existing primary data before full refresh. [2025-10-20T18:40:29.316146] [INFO ] [LOCAL ] EVR: Starting deletion of primary data. [2025-10-20T18:40:29.316351] [DEBUG] [LOCAL ] EVR: Deleted primary data file. [2025-10-20T18:40:29.316389] [INFO ] [MAIN ] EVR: Starting full primary refresh. [2025-10-20T18:40:29.679551] [INFO ] [REMOTE ] EVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:29.691633] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:40:29.851370] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-01. [2025-10-20T18:40:30.651382] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-06. [2025-10-20T18:40:31.134084] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T18:40:31.451431] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-03. [2025-10-20T18:40:32.263737] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-08. [2025-10-20T18:40:32.270820] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:40:32.466076] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:40:32.999273] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:40:33.051429] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-07. [2025-10-20T18:40:33.851481] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-02. [2025-10-20T18:40:34.511755] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:40:34.651412] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-08. [2025-10-20T18:40:35.395972] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:40:35.451400] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-03. [2025-10-20T18:40:35.859195] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:40:36.251407] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-04. [2025-10-20T18:40:36.562997] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T18:40:37.051425] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-04. [2025-10-20T18:40:37.142539] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T18:40:37.851435] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-05. [2025-10-20T18:40:38.437004] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T18:40:38.651371] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-09. [2025-10-20T18:40:38.773870] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:40:39.451394] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-06. [2025-10-20T18:40:40.256823] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-09. [2025-10-20T18:40:40.276702] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:40:40.305456] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:40:40.950402] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:40:41.051372] [INFO ] [REMOTE ] EVH: Fetching primary data for 2025-10. [2025-10-20T18:40:41.851387] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-05. [2025-10-20T18:40:42.651434] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-07. [2025-10-20T18:40:42.877513] [INFO ] [REMOTE ] EVH: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:40:42.886084] [INFO ] [REMOTE ] EVH: Full fetch complete (71841 records). [2025-10-20T18:40:42.886346] [DEBUG] [VALIDATOR ] EVH: Validating primary data. [2025-10-20T18:40:43.341023] [DEBUG] [VALIDATOR ] EVH: All expected trading days are present. [2025-10-20T18:40:43.341258] [DEBUG] [VALIDATOR ] EVH: Primary data validation passed. [2025-10-20T18:40:43.343019] [INFO ] [MAIN ] EVH: Fetched primary data is valid. [2025-10-20T18:40:43.343292] [DEBUG] [LOCAL ] EVH: Saving primary data. [2025-10-20T18:40:43.346178] [DEBUG] [LOCAL ] EVH: Trimmed 71841 records to 49646 (keeping 200 days of data) [2025-10-20T18:40:43.452135] [INFO ] [REMOTE ] EVG: Fetching primary data for 2025-10. [2025-10-20T18:40:43.547276] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:40:43.625973] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T18:40:43.687714] [DEBUG] [LOCAL ] EVH: Primary data saved (49646 records). [2025-10-20T18:40:43.687885] [INFO ] [MAIN ] EVH: Full history saved. [2025-10-20T18:40:43.687981] [INFO ] [MAIN ] Update Progress: 787/2367 symbols completed. EVH took 29.1s. Estimated remaining: 3h 22m 5s. Rate: 7.8 symbols/min [2025-10-20T18:40:43.853083] [INFO ] [MAIN ] EVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:43.870258] [DEBUG] [LOCAL ] EVT: Loaded 17008 existing records from primary storage. [2025-10-20T18:40:43.870477] [INFO ] [MAIN ] EVT: Loaded 17008 existing records. [2025-10-20T18:40:44.042788] [WARN ] [MAIN ] EVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:44.043016] [INFO ] [MAIN ] EVT: Wiping existing primary data before full refresh. [2025-10-20T18:40:44.043061] [INFO ] [LOCAL ] EVT: Starting deletion of primary data. [2025-10-20T18:40:44.043255] [DEBUG] [LOCAL ] EVT: Deleted primary data file. [2025-10-20T18:40:44.043310] [INFO ] [MAIN ] EVT: Starting full primary refresh. [2025-10-20T18:40:44.256627] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-08. [2025-10-20T18:40:44.332469] [INFO ] [REMOTE ] EVG: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:40:44.345526] [INFO ] [REMOTE ] EVG: Full fetch complete (8146 records). [2025-10-20T18:40:44.361146] [DEBUG] [VALIDATOR ] EVG: Validating primary data. [2025-10-20T18:40:44.519704] [INFO ] [REMOTE ] EVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:44.803577] [DEBUG] [VALIDATOR ] EVG: All expected trading days are present. [2025-10-20T18:40:44.803677] [DEBUG] [VALIDATOR ] EVG: Primary data validation passed. [2025-10-20T18:40:44.803889] [INFO ] [MAIN ] EVG: Fetched primary data is valid. [2025-10-20T18:40:44.803930] [DEBUG] [LOCAL ] EVG: Saving primary data. [2025-10-20T18:40:44.804384] [DEBUG] [LOCAL ] EVG: Trimmed 8146 records to 5867 (keeping 200 days of data) [2025-10-20T18:40:44.840223] [DEBUG] [LOCAL ] EVG: Primary data saved (5867 records). [2025-10-20T18:40:44.840321] [INFO ] [MAIN ] EVG: Full history saved. [2025-10-20T18:40:44.840399] [INFO ] [MAIN ] Update Progress: 788/2367 symbols completed. EVG took 35.9s. Estimated remaining: 3h 21m 44s. Rate: 7.8 symbols/min [2025-10-20T18:40:45.006106] [INFO ] [MAIN ] EVTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:45.032076] [DEBUG] [LOCAL ] EVTC: Loaded 27241 existing records from primary storage. [2025-10-20T18:40:45.032368] [INFO ] [MAIN ] EVTC: Loaded 27241 existing records. [2025-10-20T18:40:45.061280] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-01. [2025-10-20T18:40:45.217367] [WARN ] [MAIN ] EVTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:45.217622] [INFO ] [MAIN ] EVTC: Wiping existing primary data before full refresh. [2025-10-20T18:40:45.217709] [INFO ] [LOCAL ] EVTC: Starting deletion of primary data. [2025-10-20T18:40:45.217891] [DEBUG] [LOCAL ] EVTC: Deleted primary data file. [2025-10-20T18:40:45.217930] [INFO ] [MAIN ] EVTC: Starting full primary refresh. [2025-10-20T18:40:45.498098] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T18:40:45.587805] [INFO ] [REMOTE ] EVTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:45.851380] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-01. [2025-10-20T18:40:46.505196] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:40:46.651381] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-02. [2025-10-20T18:40:47.110022] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:40:47.451398] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-02. [2025-10-20T18:40:47.630921] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:40:48.251378] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-03. [2025-10-20T18:40:48.490302] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:40:49.051389] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-03. [2025-10-20T18:40:49.229184] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T18:40:49.851395] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-06. [2025-10-20T18:40:50.369457] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:40:50.651367] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-04. [2025-10-20T18:40:51.451342] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-04. [2025-10-20T18:40:51.778735] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T18:40:52.251390] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-09. [2025-10-20T18:40:52.370887] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:40:52.675185] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T18:40:53.051390] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-05. [2025-10-20T18:40:53.723602] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T18:40:53.851396] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-05. [2025-10-20T18:40:54.105909] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T18:40:54.651412] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-07. [2025-10-20T18:40:54.784938] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T18:40:55.451369] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-06. [2025-10-20T18:40:56.040361] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T18:40:56.251401] [INFO ] [REMOTE ] EVN: Fetching primary data for 2025-10. [2025-10-20T18:40:56.373136] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:40:57.051377] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-06. [2025-10-20T18:40:57.092734] [INFO ] [REMOTE ] EVN: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:40:57.094782] [INFO ] [REMOTE ] EVN: Full fetch complete (17068 records). [2025-10-20T18:40:57.094856] [DEBUG] [VALIDATOR ] EVN: Validating primary data. [2025-10-20T18:40:57.464112] [DEBUG] [VALIDATOR ] EVN: All expected trading days are present. [2025-10-20T18:40:57.464188] [DEBUG] [VALIDATOR ] EVN: Primary data validation passed. [2025-10-20T18:40:57.464540] [INFO ] [MAIN ] EVN: Fetched primary data is valid. [2025-10-20T18:40:57.464579] [DEBUG] [LOCAL ] EVN: Saving primary data. [2025-10-20T18:40:57.465114] [DEBUG] [LOCAL ] EVN: Trimmed 17068 records to 10724 (keeping 200 days of data) [2025-10-20T18:40:57.528001] [DEBUG] [LOCAL ] EVN: Primary data saved (10724 records). [2025-10-20T18:40:57.528128] [INFO ] [MAIN ] EVN: Full history saved. [2025-10-20T18:40:57.528205] [INFO ] [MAIN ] Update Progress: 789/2367 symbols completed. EVN took 37.4s. Estimated remaining: 3h 21m 46s. Rate: 7.8 symbols/min [2025-10-20T18:40:57.697836] [INFO ] [MAIN ] EVTL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:40:57.735485] [DEBUG] [LOCAL ] EVTL: Loaded 41058 existing records from primary storage. [2025-10-20T18:40:57.735796] [INFO ] [MAIN ] EVTL: Loaded 41058 existing records. [2025-10-20T18:40:57.851704] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-07. [2025-10-20T18:40:57.921033] [WARN ] [MAIN ] EVTL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:40:57.921255] [INFO ] [MAIN ] EVTL: Wiping existing primary data before full refresh. [2025-10-20T18:40:57.921311] [INFO ] [LOCAL ] EVTL: Starting deletion of primary data. [2025-10-20T18:40:57.921699] [DEBUG] [LOCAL ] EVTL: Deleted primary data file. [2025-10-20T18:40:57.921755] [INFO ] [MAIN ] EVTL: Starting full primary refresh. [2025-10-20T18:40:58.245341] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:40:58.319706] [INFO ] [REMOTE ] EVTL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:40:58.651341] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-01. [2025-10-20T18:40:58.954334] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:40:59.451381] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-08. [2025-10-20T18:41:00.128773] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:41:00.251408] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-08. [2025-10-20T18:41:00.638181] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T18:41:00.836057] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:41:01.051375] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-07. [2025-10-20T18:41:01.851407] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-09. [2025-10-20T18:41:02.337800] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T18:41:02.651389] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-09. [2025-10-20T18:41:03.112024] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:41:03.442528] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:41:03.451328] [INFO ] [REMOTE ] EVT: Fetching primary data for 2025-10. [2025-10-20T18:41:04.251400] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-02. [2025-10-20T18:41:04.475112] [INFO ] [REMOTE ] EVT: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:41:04.477694] [INFO ] [REMOTE ] EVT: Full fetch complete (25197 records). [2025-10-20T18:41:04.477763] [DEBUG] [VALIDATOR ] EVT: Validating primary data. [2025-10-20T18:41:04.847079] [DEBUG] [VALIDATOR ] EVT: All expected trading days are present. [2025-10-20T18:41:04.847216] [DEBUG] [VALIDATOR ] EVT: Primary data validation passed. [2025-10-20T18:41:04.847760] [INFO ] [MAIN ] EVT: Fetched primary data is valid. [2025-10-20T18:41:04.847822] [DEBUG] [LOCAL ] EVT: Saving primary data. [2025-10-20T18:41:04.848806] [DEBUG] [LOCAL ] EVT: Trimmed 25197 records to 16762 (keeping 200 days of data) [2025-10-20T18:41:04.948855] [DEBUG] [LOCAL ] EVT: Primary data saved (16762 records). [2025-10-20T18:41:04.949004] [INFO ] [MAIN ] EVT: Full history saved. [2025-10-20T18:41:04.949082] [INFO ] [MAIN ] Update Progress: 790/2367 symbols completed. EVT took 21.3s. Estimated remaining: 3h 21m 38s. Rate: 7.8 symbols/min [2025-10-20T18:41:05.056571] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-08. [2025-10-20T18:41:05.116951] [INFO ] [MAIN ] EVTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:05.154680] [DEBUG] [LOCAL ] EVTR: Loaded 25284 existing records from primary storage. [2025-10-20T18:41:05.154956] [INFO ] [MAIN ] EVTR: Loaded 25284 existing records. [2025-10-20T18:41:05.331764] [WARN ] [MAIN ] EVTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:05.332029] [INFO ] [MAIN ] EVTR: Wiping existing primary data before full refresh. [2025-10-20T18:41:05.332075] [INFO ] [LOCAL ] EVTR: Starting deletion of primary data. [2025-10-20T18:41:05.332320] [DEBUG] [LOCAL ] EVTR: Deleted primary data file. [2025-10-20T18:41:05.332375] [INFO ] [MAIN ] EVTR: Starting full primary refresh. [2025-10-20T18:41:05.696287] [INFO ] [REMOTE ] EVTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:05.717307] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T18:41:05.827976] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T18:41:05.851338] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-01. [2025-10-20T18:41:06.651376] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-03. [2025-10-20T18:41:07.055742] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:41:07.451378] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-02. [2025-10-20T18:41:07.968318] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:41:08.251343] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-04. [2025-10-20T18:41:08.410348] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:41:09.051358] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-09. [2025-10-20T18:41:09.302300] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:41:09.851340] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-05. [2025-10-20T18:41:10.309625] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:41:10.651359] [INFO ] [REMOTE ] EVTC: Fetching primary data for 2025-10. [2025-10-20T18:41:11.314884] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:41:11.451429] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-03. [2025-10-20T18:41:11.886346] [INFO ] [REMOTE ] EVTC: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T18:41:11.891108] [INFO ] [REMOTE ] EVTC: Full fetch complete (38420 records). [2025-10-20T18:41:11.891194] [DEBUG] [VALIDATOR ] EVTC: Validating primary data. [2025-10-20T18:41:12.252896] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-06. [2025-10-20T18:41:12.286156] [DEBUG] [VALIDATOR ] EVTC: All expected trading days are present. [2025-10-20T18:41:12.286301] [DEBUG] [VALIDATOR ] EVTC: Primary data validation passed. [2025-10-20T18:41:12.287066] [INFO ] [MAIN ] EVTC: Fetched primary data is valid. [2025-10-20T18:41:12.287129] [DEBUG] [LOCAL ] EVTC: Saving primary data. [2025-10-20T18:41:12.287992] [DEBUG] [LOCAL ] EVTC: Trimmed 38420 records to 26895 (keeping 200 days of data) [2025-10-20T18:41:12.472011] [DEBUG] [LOCAL ] EVTC: Primary data saved (26895 records). [2025-10-20T18:41:12.472754] [INFO ] [MAIN ] EVTC: Full history saved. [2025-10-20T18:41:12.473002] [INFO ] [MAIN ] Update Progress: 791/2367 symbols completed. EVTC took 27.6s. Estimated remaining: 3h 21m 30s. Rate: 7.8 symbols/min [2025-10-20T18:41:12.473510] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T18:41:12.694587] [INFO ] [MAIN ] EW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:12.747958] [DEBUG] [LOCAL ] EW: Loaded 54179 existing records from primary storage. [2025-10-20T18:41:12.748276] [INFO ] [MAIN ] EW: Loaded 54179 existing records. [2025-10-20T18:41:12.920769] [WARN ] [MAIN ] EW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:12.920964] [INFO ] [MAIN ] EW: Wiping existing primary data before full refresh. [2025-10-20T18:41:12.921008] [INFO ] [LOCAL ] EW: Starting deletion of primary data. [2025-10-20T18:41:12.921211] [DEBUG] [LOCAL ] EW: Deleted primary data file. [2025-10-20T18:41:12.921267] [INFO ] [MAIN ] EW: Starting full primary refresh. [2025-10-20T18:41:13.056614] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-04. [2025-10-20T18:41:13.268363] [INFO ] [REMOTE ] EW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:13.803542] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:41:13.851358] [INFO ] [REMOTE ] EVR: Fetching primary data for 2025-10. [2025-10-20T18:41:14.071466] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T18:41:14.651386] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-07. [2025-10-20T18:41:15.345583] [INFO ] [REMOTE ] EVR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T18:41:15.350333] [INFO ] [REMOTE ] EVR: Full fetch complete (50835 records). [2025-10-20T18:41:15.350456] [DEBUG] [VALIDATOR ] EVR: Validating primary data. [2025-10-20T18:41:15.456580] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-01. [2025-10-20T18:41:15.733606] [DEBUG] [VALIDATOR ] EVR: All expected trading days are present. [2025-10-20T18:41:15.733952] [DEBUG] [VALIDATOR ] EVR: Primary data validation passed. [2025-10-20T18:41:15.734692] [INFO ] [MAIN ] EVR: Fetched primary data is valid. [2025-10-20T18:41:15.734755] [DEBUG] [LOCAL ] EVR: Saving primary data. [2025-10-20T18:41:15.735922] [DEBUG] [LOCAL ] EVR: Trimmed 50835 records to 34068 (keeping 200 days of data) [2025-10-20T18:41:15.941668] [DEBUG] [LOCAL ] EVR: Primary data saved (34068 records). [2025-10-20T18:41:15.941800] [INFO ] [MAIN ] EVR: Full history saved. [2025-10-20T18:41:15.941874] [INFO ] [MAIN ] Update Progress: 792/2367 symbols completed. EVR took 47.1s. Estimated remaining: 3h 21m 14s. Rate: 7.8 symbols/min [2025-10-20T18:41:16.106363] [INFO ] [MAIN ] EXG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:16.138192] [DEBUG] [LOCAL ] EXG: Loaded 34518 existing records from primary storage. [2025-10-20T18:41:16.138509] [INFO ] [MAIN ] EXG: Loaded 34518 existing records. [2025-10-20T18:41:16.257305] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-05. [2025-10-20T18:41:16.316431] [WARN ] [MAIN ] EXG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:16.316623] [INFO ] [MAIN ] EXG: Wiping existing primary data before full refresh. [2025-10-20T18:41:16.316700] [INFO ] [LOCAL ] EXG: Starting deletion of primary data. [2025-10-20T18:41:16.316979] [DEBUG] [LOCAL ] EXG: Deleted primary data file. [2025-10-20T18:41:16.317036] [INFO ] [MAIN ] EXG: Starting full primary refresh. [2025-10-20T18:41:16.688735] [INFO ] [REMOTE ] EXG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:16.758129] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T18:41:17.051343] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-08. [2025-10-20T18:41:17.217143] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T18:41:17.413014] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T18:41:17.851389] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-02. [2025-10-20T18:41:18.651528] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-06. [2025-10-20T18:41:18.819753] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:41:19.451341] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-09. [2025-10-20T18:41:19.569010] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T18:41:19.603223] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T18:41:20.251396] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-07. [2025-10-20T18:41:21.051406] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-01. [2025-10-20T18:41:21.243412] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:41:21.277234] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:41:21.851405] [INFO ] [REMOTE ] EVTL: Fetching primary data for 2025-10. [2025-10-20T18:41:22.067347] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T18:41:22.651440] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-03. [2025-10-20T18:41:23.451421] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-08. [2025-10-20T18:41:24.078584] [INFO ] [REMOTE ] EVTL: Incremental fetch complete for 2025-10 (2.23 seconds). [2025-10-20T18:41:24.090538] [INFO ] [REMOTE ] EVTL: Full fetch complete (54651 records). [2025-10-20T18:41:24.090718] [DEBUG] [VALIDATOR ] EVTL: Validating primary data. [2025-10-20T18:41:24.256040] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-02. [2025-10-20T18:41:24.529088] [DEBUG] [VALIDATOR ] EVTL: All expected trading days are present. [2025-10-20T18:41:24.529676] [DEBUG] [VALIDATOR ] EVTL: Primary data validation passed. [2025-10-20T18:41:24.532114] [INFO ] [MAIN ] EVTL: Fetched primary data is valid. [2025-10-20T18:41:24.531144] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:41:24.532358] [DEBUG] [LOCAL ] EVTL: Saving primary data. [2025-10-20T18:41:24.561117] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T18:41:24.568089] [DEBUG] [LOCAL ] EVTL: Trimmed 54651 records to 41005 (keeping 200 days of data) [2025-10-20T18:41:24.802930] [DEBUG] [LOCAL ] EVTL: Primary data saved (41005 records). [2025-10-20T18:41:24.803050] [INFO ] [MAIN ] EVTL: Full history saved. [2025-10-20T18:41:24.803121] [INFO ] [MAIN ] Update Progress: 793/2367 symbols completed. EVTL took 27.3s. Estimated remaining: 3h 21m 9s. Rate: 7.8 symbols/min [2025-10-20T18:41:24.975572] [INFO ] [MAIN ] EXK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:25.026700] [DEBUG] [LOCAL ] EXK: Loaded 53951 existing records from primary storage. [2025-10-20T18:41:25.027021] [INFO ] [MAIN ] EXK: Loaded 53951 existing records. [2025-10-20T18:41:25.057317] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-04. [2025-10-20T18:41:25.221313] [WARN ] [MAIN ] EXK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:25.221540] [INFO ] [MAIN ] EXK: Wiping existing primary data before full refresh. [2025-10-20T18:41:25.221590] [INFO ] [LOCAL ] EXK: Starting deletion of primary data. [2025-10-20T18:41:25.222137] [DEBUG] [LOCAL ] EXK: Deleted primary data file. [2025-10-20T18:41:25.222195] [INFO ] [MAIN ] EXK: Starting full primary refresh. [2025-10-20T18:41:25.735840] [INFO ] [REMOTE ] EXK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:25.817935] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:41:25.851420] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-03. [2025-10-20T18:41:26.651400] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-09. [2025-10-20T18:41:26.962775] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T18:41:27.042083] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:41:27.451413] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-05. [2025-10-20T18:41:28.052155] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:41:28.251423] [INFO ] [REMOTE ] EVTR: Fetching primary data for 2025-10. [2025-10-20T18:41:29.051427] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-04. [2025-10-20T18:41:29.284723] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:41:29.649334] [INFO ] [REMOTE ] EVTR: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T18:41:29.654627] [INFO ] [REMOTE ] EVTR: Full fetch complete (33630 records). [2025-10-20T18:41:29.654763] [DEBUG] [VALIDATOR ] EVTR: Validating primary data. [2025-10-20T18:41:29.851510] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-01. [2025-10-20T18:41:30.049888] [DEBUG] [VALIDATOR ] EVTR: All expected trading days are present. [2025-10-20T18:41:30.049969] [DEBUG] [VALIDATOR ] EVTR: Primary data validation passed. [2025-10-20T18:41:30.050534] [INFO ] [MAIN ] EVTR: Fetched primary data is valid. [2025-10-20T18:41:30.050572] [DEBUG] [LOCAL ] EVTR: Saving primary data. [2025-10-20T18:41:30.051501] [DEBUG] [LOCAL ] EVTR: Trimmed 33630 records to 25188 (keeping 200 days of data) [2025-10-20T18:41:30.202963] [DEBUG] [LOCAL ] EVTR: Primary data saved (25188 records). [2025-10-20T18:41:30.203102] [INFO ] [MAIN ] EVTR: Full history saved. [2025-10-20T18:41:30.203182] [INFO ] [MAIN ] Update Progress: 794/2367 symbols completed. EVTR took 25.3s. Estimated remaining: 3h 20m 57s. Rate: 7.8 symbols/min [2025-10-20T18:41:30.380423] [INFO ] [MAIN ] EXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:30.405472] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T18:41:30.439343] [DEBUG] [LOCAL ] EXP: Loaded 29985 existing records from primary storage. [2025-10-20T18:41:30.439603] [INFO ] [MAIN ] EXP: Loaded 29985 existing records. [2025-10-20T18:41:30.620238] [WARN ] [MAIN ] EXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:30.620410] [INFO ] [MAIN ] EXP: Wiping existing primary data before full refresh. [2025-10-20T18:41:30.620459] [INFO ] [LOCAL ] EXP: Starting deletion of primary data. [2025-10-20T18:41:30.620628] [DEBUG] [LOCAL ] EXP: Deleted primary data file. [2025-10-20T18:41:30.620713] [INFO ] [MAIN ] EXP: Starting full primary refresh. [2025-10-20T18:41:30.656529] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-05. [2025-10-20T18:41:30.973884] [INFO ] [REMOTE ] EXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:31.451372] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-06. [2025-10-20T18:41:31.824989] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T18:41:32.067882] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T18:41:32.251384] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-06. [2025-10-20T18:41:33.051382] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-02. [2025-10-20T18:41:33.287568] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T18:41:33.795450] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:41:33.851380] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-07. [2025-10-20T18:41:34.651363] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-07. [2025-10-20T18:41:34.896778] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T18:41:34.972980] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:41:35.451383] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-08. [2025-10-20T18:41:36.251396] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-03. [2025-10-20T18:41:36.520053] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:41:36.701408] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T18:41:37.051364] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-09. [2025-10-20T18:41:37.851365] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-08. [2025-10-20T18:41:38.142061] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:41:38.207507] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T18:41:38.651384] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-04. [2025-10-20T18:41:39.451358] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-01. [2025-10-20T18:41:39.734098] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T18:41:40.251404] [INFO ] [REMOTE ] EXG: Fetching primary data for 2025-10. [2025-10-20T18:41:40.476277] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T18:41:40.990426] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:41:41.051339] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-02. [2025-10-20T18:41:41.613428] [INFO ] [REMOTE ] EXG: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:41:41.618290] [INFO ] [REMOTE ] EXG: Full fetch complete (49749 records). [2025-10-20T18:41:41.618383] [DEBUG] [VALIDATOR ] EXG: Validating primary data. [2025-10-20T18:41:41.858017] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-05. [2025-10-20T18:41:41.994118] [DEBUG] [VALIDATOR ] EXG: All expected trading days are present. [2025-10-20T18:41:41.994196] [DEBUG] [VALIDATOR ] EXG: Primary data validation passed. [2025-10-20T18:41:41.994974] [INFO ] [MAIN ] EXG: Fetched primary data is valid. [2025-10-20T18:41:41.995038] [DEBUG] [LOCAL ] EXG: Saving primary data. [2025-10-20T18:41:41.996103] [DEBUG] [LOCAL ] EXG: Trimmed 49749 records to 33887 (keeping 200 days of data) [2025-10-20T18:41:42.199298] [DEBUG] [LOCAL ] EXG: Primary data saved (33887 records). [2025-10-20T18:41:42.199453] [INFO ] [MAIN ] EXG: Full history saved. [2025-10-20T18:41:42.199538] [INFO ] [MAIN ] Update Progress: 795/2367 symbols completed. EXG took 26.3s. Estimated remaining: 3h 20m 58s. Rate: 7.8 symbols/min [2025-10-20T18:41:42.377664] [INFO ] [MAIN ] EXPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:42.421773] [DEBUG] [LOCAL ] EXPD: Loaded 48595 existing records from primary storage. [2025-10-20T18:41:42.422077] [INFO ] [MAIN ] EXPD: Loaded 48595 existing records. [2025-10-20T18:41:42.602784] [WARN ] [MAIN ] EXPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:42.609218] [INFO ] [MAIN ] EXPD: Wiping existing primary data before full refresh. [2025-10-20T18:41:42.618217] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-10-20T18:41:42.618954] [DEBUG] [LOCAL ] EXPD: Deleted primary data file. [2025-10-20T18:41:42.619695] [INFO ] [MAIN ] EXPD: Starting full primary refresh. [2025-10-20T18:41:42.619602] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:41:42.653694] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-09. [2025-10-20T18:41:42.985732] [INFO ] [REMOTE ] EXPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:43.451399] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-01. [2025-10-20T18:41:43.768455] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T18:41:44.251359] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-06. [2025-10-20T18:41:44.297782] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:41:45.051373] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-03. [2025-10-20T18:41:45.113581] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T18:41:45.851362] [INFO ] [REMOTE ] EW: Fetching primary data for 2025-10. [2025-10-20T18:41:46.070491] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:41:46.492344] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T18:41:46.587127] [INFO ] [REMOTE ] EW: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T18:41:46.598503] [INFO ] [REMOTE ] EW: Full fetch complete (77515 records). [2025-10-20T18:41:46.598715] [DEBUG] [VALIDATOR ] EW: Validating primary data. [2025-10-20T18:41:46.656446] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-04. [2025-10-20T18:41:46.998439] [DEBUG] [VALIDATOR ] EW: All expected trading days are present. [2025-10-20T18:41:46.998521] [DEBUG] [VALIDATOR ] EW: Primary data validation passed. [2025-10-20T18:41:46.999451] [INFO ] [MAIN ] EW: Fetched primary data is valid. [2025-10-20T18:41:46.999522] [DEBUG] [LOCAL ] EW: Saving primary data. [2025-10-20T18:41:47.000986] [DEBUG] [LOCAL ] EW: Trimmed 77515 records to 53400 (keeping 200 days of data) [2025-10-20T18:41:47.313103] [DEBUG] [LOCAL ] EW: Primary data saved (53400 records). [2025-10-20T18:41:47.313250] [INFO ] [MAIN ] EW: Full history saved. [2025-10-20T18:41:47.313603] [INFO ] [MAIN ] Update Progress: 796/2367 symbols completed. EW took 34.8s. Estimated remaining: 3h 20m 45s. Rate: 7.8 symbols/min [2025-10-20T18:41:47.456466] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-07. [2025-10-20T18:41:47.511962] [INFO ] [MAIN ] EXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:41:47.556572] [DEBUG] [LOCAL ] EXR: Loaded 46249 existing records from primary storage. [2025-10-20T18:41:47.556906] [INFO ] [MAIN ] EXR: Loaded 46249 existing records. [2025-10-20T18:41:47.737327] [WARN ] [MAIN ] EXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:41:47.737672] [INFO ] [MAIN ] EXR: Wiping existing primary data before full refresh. [2025-10-20T18:41:47.737779] [INFO ] [LOCAL ] EXR: Starting deletion of primary data. [2025-10-20T18:41:47.738063] [DEBUG] [LOCAL ] EXR: Deleted primary data file. [2025-10-20T18:41:47.738117] [INFO ] [MAIN ] EXR: Starting full primary refresh. [2025-10-20T18:41:47.933902] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T18:41:48.119210] [INFO ] [REMOTE ] EXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:41:48.251369] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-01. [2025-10-20T18:41:49.051408] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-02. [2025-10-20T18:41:49.526358] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T18:41:49.851419] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-08. [2025-10-20T18:41:49.967318] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:41:50.545126] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:41:50.651356] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-02. [2025-10-20T18:41:51.451428] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-03. [2025-10-20T18:41:51.708410] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:41:52.186278] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:41:52.251339] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-03. [2025-10-20T18:41:53.051404] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-09. [2025-10-20T18:41:53.359976] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T18:41:53.851378] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-04. [2025-10-20T18:41:53.904257] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T18:41:54.651425] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-05. [2025-10-20T18:41:54.728169] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T18:41:55.296962] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T18:41:55.451380] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-04. [2025-10-20T18:41:56.090156] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T18:41:56.251421] [INFO ] [REMOTE ] EXK: Fetching primary data for 2025-10. [2025-10-20T18:41:57.051399] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-05. [2025-10-20T18:41:57.152605] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:41:57.851351] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-06. [2025-10-20T18:41:58.651421] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-05. [2025-10-20T18:41:58.677430] [INFO ] [REMOTE ] EXK: Incremental fetch complete for 2025-10 (2.43 seconds). [2025-10-20T18:41:58.685423] [INFO ] [REMOTE ] EXK: Full fetch complete (76809 records). [2025-10-20T18:41:58.685626] [DEBUG] [VALIDATOR ] EXK: Validating primary data. [2025-10-20T18:41:59.043158] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T18:41:59.128929] [DEBUG] [VALIDATOR ] EXK: All expected trading days are present. [2025-10-20T18:41:59.129075] [DEBUG] [VALIDATOR ] EXK: Primary data validation passed. [2025-10-20T18:41:59.130083] [INFO ] [MAIN ] EXK: Fetched primary data is valid. [2025-10-20T18:41:59.130148] [DEBUG] [LOCAL ] EXK: Saving primary data. [2025-10-20T18:41:59.131635] [DEBUG] [LOCAL ] EXK: Trimmed 76809 records to 53185 (keeping 200 days of data) [2025-10-20T18:41:59.342282] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T18:41:59.451924] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-07. [2025-10-20T18:41:59.464582] [DEBUG] [LOCAL ] EXK: Primary data saved (53185 records). [2025-10-20T18:41:59.464790] [INFO ] [MAIN ] EXK: Full history saved. [2025-10-20T18:41:59.465885] [INFO ] [MAIN ] Update Progress: 797/2367 symbols completed. EXK took 34.7s. Estimated remaining: 3h 20m 46s. Rate: 7.8 symbols/min [2025-10-20T18:42:00.251356] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-06. [2025-10-20T18:42:00.390685] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:42:00.729605] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:42:01.051366] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-08. [2025-10-20T18:42:01.851505] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-06. [2025-10-20T18:42:01.967769] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:42:02.034485] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:42:02.651377] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-09. [2025-10-20T18:42:03.406974] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:42:03.451366] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-07. [2025-10-20T18:42:04.102875] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:42:04.251367] [INFO ] [REMOTE ] EXP: Fetching primary data for 2025-10. [2025-10-20T18:42:05.051418] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-07. [2025-10-20T18:42:05.298549] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:42:05.525267] [INFO ] [REMOTE ] EXP: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:42:05.529704] [INFO ] [REMOTE ] EXP: Full fetch complete (42951 records). [2025-10-20T18:42:05.529849] [DEBUG] [VALIDATOR ] EXP: Validating primary data. [2025-10-20T18:42:05.856519] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-08. [2025-10-20T18:42:05.928209] [DEBUG] [VALIDATOR ] EXP: All expected trading days are present. [2025-10-20T18:42:05.928284] [DEBUG] [VALIDATOR ] EXP: Primary data validation passed. [2025-10-20T18:42:05.928942] [INFO ] [MAIN ] EXP: Fetched primary data is valid. [2025-10-20T18:42:05.929025] [DEBUG] [LOCAL ] EXP: Saving primary data. [2025-10-20T18:42:05.930075] [DEBUG] [LOCAL ] EXP: Trimmed 42951 records to 29414 (keeping 200 days of data) [2025-10-20T18:42:06.107468] [DEBUG] [LOCAL ] EXP: Primary data saved (29414 records). [2025-10-20T18:42:06.107633] [INFO ] [MAIN ] EXP: Full history saved. [2025-10-20T18:42:06.107752] [INFO ] [MAIN ] Update Progress: 798/2367 symbols completed. EXP took 35.9s. Estimated remaining: 3h 20m 36s. Rate: 7.8 symbols/min [2025-10-20T18:42:06.795808] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T18:42:06.806037] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-08. [2025-10-20T18:42:07.625091] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:42:07.639895] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-09. [2025-10-20T18:42:08.240497] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:42:08.252541] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-09. [2025-10-20T18:42:09.354459] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T18:42:09.364601] [INFO ] [REMOTE ] EXPD: Fetching primary data for 2025-10. [2025-10-20T18:42:09.742164] [INFO ] [REMOTE ] EXPD: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-10-20T18:42:09.748801] [INFO ] [REMOTE ] EXPD: Full fetch complete (69216 records). [2025-10-20T18:42:09.748946] [DEBUG] [VALIDATOR ] EXPD: Validating primary data. [2025-10-20T18:42:09.749775] [FATAL] [VALIDATOR ] EXPD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:42:09.749872] [FATAL] [MAIN ] EXPD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:42:09.749916] [WARN ] [MAIN ] EXPD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:42:09.749985] [INFO ] [LOCAL ] EXPD: Starting deletion of primary data. [2025-10-20T18:42:09.750051] [DEBUG] [LOCAL ] EXPD: No primary data file found to delete. [2025-10-20T18:42:09.750088] [INFO ] [LOCAL ] EXPD: Starting deletion of derivative data. [2025-10-20T18:42:09.750302] [DEBUG] [LOCAL ] EXPD: Deleted derivative data file. [2025-10-20T18:42:09.750396] [INFO ] [MAIN ] Update Progress: 799/2367 symbols completed. EXPD took 27.6s. Estimated remaining: 3h 20m 21s. Rate: 7.8 symbols/min [2025-10-20T18:42:09.920166] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T18:42:09.930290] [INFO ] [REMOTE ] EXR: Fetching primary data for 2025-10. [2025-10-20T18:42:10.798070] [INFO ] [REMOTE ] EXR: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:42:10.804893] [INFO ] [REMOTE ] EXR: Full fetch complete (66237 records). [2025-10-20T18:42:10.805009] [DEBUG] [VALIDATOR ] EXR: Validating primary data. [2025-10-20T18:42:11.189297] [DEBUG] [VALIDATOR ] EXR: All expected trading days are present. [2025-10-20T18:42:11.189404] [DEBUG] [VALIDATOR ] EXR: Primary data validation passed. [2025-10-20T18:42:11.190327] [INFO ] [MAIN ] EXR: Fetched primary data is valid. [2025-10-20T18:42:11.190414] [DEBUG] [LOCAL ] EXR: Saving primary data. [2025-10-20T18:42:11.191787] [DEBUG] [LOCAL ] EXR: Trimmed 66237 records to 45611 (keeping 200 days of data) [2025-10-20T18:42:11.471045] [DEBUG] [LOCAL ] EXR: Primary data saved (45611 records). [2025-10-20T18:42:11.471169] [INFO ] [MAIN ] EXR: Full history saved. [2025-10-20T18:42:11.471614] [INFO ] [MAIN ] Update Progress: 800/2367 symbols completed. EXR took 24.2s. Estimated remaining: 3h 20m 1s. Rate: 7.8 symbols/min [2025-10-20T18:42:12.052555] [INFO ] [MAIN ] FBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.131533] [INFO ] [MAIN ] F: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.167480] [INFO ] [MAIN ] FBIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.185491] [INFO ] [MAIN ] FAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:12.213989] [DEBUG] [LOCAL ] FBK: Loaded 27962 existing records from primary storage. [2025-10-20T18:42:12.214701] [INFO ] [MAIN ] FBK: Loaded 27962 existing records. [2025-10-20T18:42:12.459709] [WARN ] [MAIN ] FBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:12.491313] [INFO ] [MAIN ] FBK: Wiping existing primary data before full refresh. [2025-10-20T18:42:12.491483] [INFO ] [LOCAL ] FBK: Starting deletion of primary data. [2025-10-20T18:42:12.491855] [DEBUG] [LOCAL ] FBK: Deleted primary data file. [2025-10-20T18:42:12.491980] [INFO ] [MAIN ] FBK: Starting full primary refresh. [2025-10-20T18:42:12.527323] [DEBUG] [LOCAL ] F: Loaded 54600 existing records from primary storage. [2025-10-20T18:42:12.526683] [DEBUG] [LOCAL ] FBIN: Loaded 51068 existing records from primary storage. [2025-10-20T18:42:12.549199] [DEBUG] [LOCAL ] FAF: Loaded 42095 existing records from primary storage. [2025-10-20T18:42:12.575156] [INFO ] [MAIN ] FAF: Loaded 42095 existing records. [2025-10-20T18:42:12.575030] [INFO ] [MAIN ] F: Loaded 54600 existing records. [2025-10-20T18:42:12.569720] [INFO ] [MAIN ] FBIN: Loaded 51068 existing records. [2025-10-20T18:42:13.261409] [WARN ] [MAIN ] F: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.299284] [WARN ] [MAIN ] FAF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.362213] [WARN ] [MAIN ] FBIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:13.324864] [INFO ] [MAIN ] F: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.377782] [INFO ] [MAIN ] FAF: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.393407] [INFO ] [MAIN ] FBIN: Wiping existing primary data before full refresh. [2025-10-20T18:42:13.393574] [INFO ] [LOCAL ] FAF: Starting deletion of primary data. [2025-10-20T18:42:13.398793] [INFO ] [LOCAL ] FBIN: Starting deletion of primary data. [2025-10-20T18:42:13.415743] [INFO ] [LOCAL ] F: Starting deletion of primary data. [2025-10-20T18:42:13.452219] [DEBUG] [LOCAL ] FAF: Deleted primary data file. [2025-10-20T18:42:13.452256] [DEBUG] [LOCAL ] FBIN: Deleted primary data file. [2025-10-20T18:42:13.455331] [INFO ] [MAIN ] FAF: Starting full primary refresh. [2025-10-20T18:42:13.458256] [INFO ] [REMOTE ] FBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:13.469219] [INFO ] [MAIN ] FBIN: Starting full primary refresh. [2025-10-20T18:42:13.463562] [DEBUG] [LOCAL ] F: Deleted primary data file. [2025-10-20T18:42:13.469603] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-01. [2025-10-20T18:42:13.469705] [INFO ] [MAIN ] F: Starting full primary refresh. [2025-10-20T18:42:14.413074] [INFO ] [REMOTE ] FAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.440322] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-01. [2025-10-20T18:42:14.548907] [INFO ] [REMOTE ] F: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.572408] [INFO ] [REMOTE ] F: Fetching primary data for 2025-01. [2025-10-20T18:42:14.577901] [INFO ] [REMOTE ] FBIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:14.578968] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-01. [2025-10-20T18:42:14.777230] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:42:14.781372] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-02. [2025-10-20T18:42:15.503982] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T18:42:15.508180] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-03. [2025-10-20T18:42:16.106882] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:42:16.114832] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-02. [2025-10-20T18:42:16.209083] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:42:16.244495] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T18:42:16.251326] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-04. [2025-10-20T18:42:17.051365] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-02. [2025-10-20T18:42:17.175599] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-01 (2.60 seconds). [2025-10-20T18:42:17.426456] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T18:42:17.612354] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:42:17.851427] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-03. [2025-10-20T18:42:18.202041] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T18:42:18.651432] [INFO ] [REMOTE ] F: Fetching primary data for 2025-02. [2025-10-20T18:42:19.230537] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:42:19.451387] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-04. [2025-10-20T18:42:20.251434] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-03. [2025-10-20T18:42:20.982003] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T18:42:21.046629] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-02 (2.40 seconds). [2025-10-20T18:42:21.056333] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-05. [2025-10-20T18:42:21.413353] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T18:42:21.721253] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T18:42:21.851380] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-05. [2025-10-20T18:42:22.651391] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-04. [2025-10-20T18:42:23.252284] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:42:23.451369] [INFO ] [REMOTE ] F: Fetching primary data for 2025-03. [2025-10-20T18:42:24.205739] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T18:42:24.251385] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-05. [2025-10-20T18:42:25.051360] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-06. [2025-10-20T18:42:25.644444] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-10-20T18:42:25.851387] [INFO ] [REMOTE ] F: Fetching primary data for 2025-04. [2025-10-20T18:42:26.059980] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T18:42:26.628492] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T18:42:26.651345] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-07. [2025-10-20T18:42:27.451387] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-06. [2025-10-20T18:42:28.057072] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:42:28.251379] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-08. [2025-10-20T18:42:28.348188] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-04 (2.50 seconds). [2025-10-20T18:42:28.995473] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:42:29.051383] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-07. [2025-10-20T18:42:29.408945] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T18:42:29.851385] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-06. [2025-10-20T18:42:30.651342] [INFO ] [REMOTE ] F: Fetching primary data for 2025-05. [2025-10-20T18:42:30.741394] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T18:42:30.913843] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:42:31.451437] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-08. [2025-10-20T18:42:32.251385] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-09. [2025-10-20T18:42:33.047826] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:42:33.057882] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-07. [2025-10-20T18:42:33.229688] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-05 (2.58 seconds). [2025-10-20T18:42:33.654825] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T18:42:33.851383] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-09. [2025-10-20T18:42:34.083444] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T18:42:34.651427] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-08. [2025-10-20T18:42:35.451369] [INFO ] [REMOTE ] FAF: Fetching primary data for 2025-10. [2025-10-20T18:42:35.511307] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:42:35.690384] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:42:36.251379] [INFO ] [REMOTE ] F: Fetching primary data for 2025-06. [2025-10-20T18:42:36.777079] [INFO ] [REMOTE ] FAF: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:42:36.783399] [INFO ] [REMOTE ] FAF: Full fetch complete (58180 records). [2025-10-20T18:42:36.783605] [DEBUG] [VALIDATOR ] FAF: Validating primary data. [2025-10-20T18:42:37.057399] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-09. [2025-10-20T18:42:37.213257] [DEBUG] [VALIDATOR ] FAF: All expected trading days are present. [2025-10-20T18:42:37.213376] [DEBUG] [VALIDATOR ] FAF: Primary data validation passed. [2025-10-20T18:42:37.214783] [INFO ] [MAIN ] FAF: Fetched primary data is valid. [2025-10-20T18:42:37.214851] [DEBUG] [LOCAL ] FAF: Saving primary data. [2025-10-20T18:42:37.216673] [DEBUG] [LOCAL ] FAF: Trimmed 58180 records to 41567 (keeping 200 days of data) [2025-10-20T18:42:37.489767] [DEBUG] [LOCAL ] FAF: Primary data saved (41567 records). [2025-10-20T18:42:37.489955] [INFO ] [MAIN ] FAF: Full history saved. [2025-10-20T18:42:37.490055] [INFO ] [MAIN ] Update Progress: 801/2367 symbols completed. FAF took 26.0s. Estimated remaining: 3h 20m 30s. Rate: 7.8 symbols/min [2025-10-20T18:42:37.672342] [INFO ] [MAIN ] FBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:37.723015] [DEBUG] [LOCAL ] FBP: Loaded 46466 existing records from primary storage. [2025-10-20T18:42:37.723458] [INFO ] [MAIN ] FBP: Loaded 46466 existing records. [2025-10-20T18:42:37.856461] [INFO ] [REMOTE ] FBIN: Fetching primary data for 2025-10. [2025-10-20T18:42:37.920069] [WARN ] [MAIN ] FBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:37.920381] [INFO ] [MAIN ] FBP: Wiping existing primary data before full refresh. [2025-10-20T18:42:37.920436] [INFO ] [LOCAL ] FBP: Starting deletion of primary data. [2025-10-20T18:42:37.920706] [DEBUG] [LOCAL ] FBP: Deleted primary data file. [2025-10-20T18:42:37.920765] [INFO ] [MAIN ] FBP: Starting full primary refresh. [2025-10-20T18:42:38.310282] [INFO ] [REMOTE ] FBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:38.326797] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:42:38.418323] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T18:42:38.651392] [INFO ] [REMOTE ] FBK: Fetching primary data for 2025-10. [2025-10-20T18:42:39.451402] [INFO ] [REMOTE ] F: Fetching primary data for 2025-07. [2025-10-20T18:42:39.727319] [INFO ] [REMOTE ] FBIN: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:42:39.735303] [INFO ] [REMOTE ] FBIN: Full fetch complete (73015 records). [2025-10-20T18:42:39.735444] [DEBUG] [VALIDATOR ] FBIN: Validating primary data. [2025-10-20T18:42:40.146110] [DEBUG] [VALIDATOR ] FBIN: All expected trading days are present. [2025-10-20T18:42:40.153034] [DEBUG] [VALIDATOR ] FBIN: Primary data validation passed. [2025-10-20T18:42:40.166374] [INFO ] [MAIN ] FBIN: Fetched primary data is valid. [2025-10-20T18:42:40.166532] [DEBUG] [LOCAL ] FBIN: Saving primary data. [2025-10-20T18:42:40.168528] [INFO ] [REMOTE ] FBK: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T18:42:40.175728] [INFO ] [REMOTE ] FBK: Full fetch complete (35423 records). [2025-10-20T18:42:40.175928] [DEBUG] [LOCAL ] FBIN: Trimmed 73015 records to 50414 (keeping 200 days of data) [2025-10-20T18:42:40.176001] [DEBUG] [VALIDATOR ] FBK: Validating primary data. [2025-10-20T18:42:40.266674] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-01. [2025-10-20T18:42:40.680937] [DEBUG] [VALIDATOR ] FBK: All expected trading days are present. [2025-10-20T18:42:40.681026] [DEBUG] [VALIDATOR ] FBK: Primary data validation passed. [2025-10-20T18:42:40.682917] [INFO ] [MAIN ] FBK: Fetched primary data is valid. [2025-10-20T18:42:40.683235] [DEBUG] [LOCAL ] FBK: Saving primary data. [2025-10-20T18:42:40.688875] [DEBUG] [LOCAL ] FBK: Trimmed 35423 records to 27519 (keeping 200 days of data) [2025-10-20T18:42:41.031885] [DEBUG] [LOCAL ] FBK: Primary data saved (27519 records). [2025-10-20T18:42:41.035971] [INFO ] [MAIN ] FBK: Full history saved. [2025-10-20T18:42:41.036334] [INFO ] [MAIN ] Update Progress: 802/2367 symbols completed. FBK took 29.6s. Estimated remaining: 3h 20m 14s. Rate: 7.8 symbols/min [2025-10-20T18:42:41.037254] [DEBUG] [LOCAL ] FBIN: Primary data saved (50414 records). [2025-10-20T18:42:41.037365] [INFO ] [MAIN ] FBIN: Full history saved. [2025-10-20T18:42:41.070061] [INFO ] [MAIN ] Update Progress: 803/2367 symbols completed. FBIN took 29.5s. Estimated remaining: 3h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T18:42:41.369313] [INFO ] [MAIN ] FBRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:41.386073] [INFO ] [MAIN ] FC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:41.402825] [DEBUG] [LOCAL ] FC: Loaded 15052 existing records from primary storage. [2025-10-20T18:42:41.403091] [INFO ] [MAIN ] FC: Loaded 15052 existing records. [2025-10-20T18:42:41.542603] [DEBUG] [LOCAL ] FBRT: Loaded 35936 existing records from primary storage. [2025-10-20T18:42:41.566454] [INFO ] [MAIN ] FBRT: Loaded 35936 existing records. [2025-10-20T18:42:41.660910] [WARN ] [MAIN ] FC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:41.694828] [INFO ] [MAIN ] FC: Wiping existing primary data before full refresh. [2025-10-20T18:42:41.700114] [INFO ] [LOCAL ] FC: Starting deletion of primary data. [2025-10-20T18:42:41.708443] [DEBUG] [LOCAL ] FC: Deleted primary data file. [2025-10-20T18:42:41.708501] [INFO ] [MAIN ] FC: Starting full primary refresh. [2025-10-20T18:42:41.912317] [WARN ] [MAIN ] FBRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:41.912838] [INFO ] [MAIN ] FBRT: Wiping existing primary data before full refresh. [2025-10-20T18:42:41.918819] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-10-20T18:42:41.939786] [DEBUG] [LOCAL ] FBRT: Deleted primary data file. [2025-10-20T18:42:41.952814] [INFO ] [MAIN ] FBRT: Starting full primary refresh. [2025-10-20T18:42:42.351915] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-10-20T18:42:42.361595] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-02. [2025-10-20T18:42:42.340217] [INFO ] [REMOTE ] FC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:42.366457] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-01. [2025-10-20T18:42:42.587280] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-07 (3.14 seconds). [2025-10-20T18:42:42.645075] [INFO ] [REMOTE ] FBRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:42.651326] [INFO ] [REMOTE ] F: Fetching primary data for 2025-08. [2025-10-20T18:42:43.316244] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T18:42:43.451345] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-02. [2025-10-20T18:42:43.862687] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T18:42:44.013540] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T18:42:44.251386] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-03. [2025-10-20T18:42:44.799639] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-08 (2.15 seconds). [2025-10-20T18:42:45.051389] [INFO ] [REMOTE ] F: Fetching primary data for 2025-09. [2025-10-20T18:42:45.825394] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:42:45.851339] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-01. [2025-10-20T18:42:46.651416] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-03. [2025-10-20T18:42:47.092445] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:42:47.458966] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-04. [2025-10-20T18:42:47.464850] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-09 (2.41 seconds). [2025-10-20T18:42:47.870486] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T18:42:48.251385] [INFO ] [REMOTE ] F: Fetching primary data for 2025-10. [2025-10-20T18:42:49.051408] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-02. [2025-10-20T18:42:49.089038] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:42:49.119860] [INFO ] [REMOTE ] F: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:42:49.127564] [INFO ] [REMOTE ] F: Full fetch complete (78000 records). [2025-10-20T18:42:49.127714] [DEBUG] [VALIDATOR ] F: Validating primary data. [2025-10-20T18:42:49.519635] [DEBUG] [VALIDATOR ] F: All expected trading days are present. [2025-10-20T18:42:49.519750] [DEBUG] [VALIDATOR ] F: Primary data validation passed. [2025-10-20T18:42:49.520692] [INFO ] [MAIN ] F: Fetched primary data is valid. [2025-10-20T18:42:49.520768] [DEBUG] [LOCAL ] F: Saving primary data. [2025-10-20T18:42:49.522232] [DEBUG] [LOCAL ] F: Trimmed 78000 records to 53820 (keeping 200 days of data) [2025-10-20T18:42:49.848499] [DEBUG] [LOCAL ] F: Primary data saved (53820 records). [2025-10-20T18:42:49.848668] [INFO ] [MAIN ] F: Full history saved. [2025-10-20T18:42:49.848764] [INFO ] [MAIN ] Update Progress: 804/2367 symbols completed. F took 38.4s. Estimated remaining: 3h 19m 46s. Rate: 7.8 symbols/min [2025-10-20T18:42:49.860011] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-04. [2025-10-20T18:42:50.030317] [INFO ] [MAIN ] FCF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:42:50.070553] [DEBUG] [LOCAL ] FCF: Loaded 39969 existing records from primary storage. [2025-10-20T18:42:50.070883] [INFO ] [MAIN ] FCF: Loaded 39969 existing records. [2025-10-20T18:42:50.250093] [WARN ] [MAIN ] FCF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:42:50.250394] [INFO ] [MAIN ] FCF: Wiping existing primary data before full refresh. [2025-10-20T18:42:50.250463] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-10-20T18:42:50.250853] [DEBUG] [LOCAL ] FCF: Deleted primary data file. [2025-10-20T18:42:50.250954] [INFO ] [MAIN ] FCF: Starting full primary refresh. [2025-10-20T18:42:50.413478] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:42:50.634405] [INFO ] [REMOTE ] FCF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:42:50.651378] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-03. [2025-10-20T18:42:50.985800] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T18:42:51.451392] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-01. [2025-10-20T18:42:51.864874] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T18:42:52.251357] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-05. [2025-10-20T18:42:52.813141] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:42:52.892487] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T18:42:53.051382] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-06. [2025-10-20T18:42:53.851349] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-05. [2025-10-20T18:42:53.993703] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:42:54.651373] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-07. [2025-10-20T18:42:55.452810] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:42:55.451452] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-02. [2025-10-20T18:42:55.724397] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T18:42:56.251404] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-08. [2025-10-20T18:42:56.741680] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:42:57.051417] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-06. [2025-10-20T18:42:57.302636] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:42:57.851378] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-04. [2025-10-20T18:42:58.521356] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T18:42:58.651365] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-03. [2025-10-20T18:42:59.375284] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T18:42:59.451370] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-05. [2025-10-20T18:43:00.077295] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:43:00.251371] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-04. [2025-10-20T18:43:00.800501] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T18:43:01.051422] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-06. [2025-10-20T18:43:01.680752] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T18:43:01.851401] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-09. [2025-10-20T18:43:02.344846] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:43:02.401409] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T18:43:02.651400] [INFO ] [REMOTE ] FC: Fetching primary data for 2025-10. [2025-10-20T18:43:03.451364] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-05. [2025-10-20T18:43:03.546331] [INFO ] [REMOTE ] FC: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T18:43:03.548637] [INFO ] [REMOTE ] FC: Full fetch complete (21857 records). [2025-10-20T18:43:03.548731] [DEBUG] [VALIDATOR ] FC: Validating primary data. [2025-10-20T18:43:03.925771] [DEBUG] [VALIDATOR ] FC: All expected trading days are present. [2025-10-20T18:43:03.925847] [DEBUG] [VALIDATOR ] FC: Primary data validation passed. [2025-10-20T18:43:03.926277] [INFO ] [MAIN ] FC: Fetched primary data is valid. [2025-10-20T18:43:03.926315] [DEBUG] [LOCAL ] FC: Saving primary data. [2025-10-20T18:43:03.926897] [DEBUG] [LOCAL ] FC: Trimmed 21857 records to 14647 (keeping 200 days of data) [2025-10-20T18:43:04.011396] [DEBUG] [LOCAL ] FC: Primary data saved (14647 records). [2025-10-20T18:43:04.011535] [INFO ] [MAIN ] FC: Full history saved. [2025-10-20T18:43:04.011614] [INFO ] [MAIN ] Update Progress: 805/2367 symbols completed. FC took 22.9s. Estimated remaining: 3h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T18:43:04.193313] [INFO ] [MAIN ] FCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:04.221882] [DEBUG] [LOCAL ] FCN: Loaded 26402 existing records from primary storage. [2025-10-20T18:43:04.222156] [INFO ] [MAIN ] FCN: Loaded 26402 existing records. [2025-10-20T18:43:04.256479] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-07. [2025-10-20T18:43:04.408997] [WARN ] [MAIN ] FCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:04.409253] [INFO ] [MAIN ] FCN: Wiping existing primary data before full refresh. [2025-10-20T18:43:04.409311] [INFO ] [LOCAL ] FCN: Starting deletion of primary data. [2025-10-20T18:43:04.409790] [DEBUG] [LOCAL ] FCN: Deleted primary data file. [2025-10-20T18:43:04.409855] [INFO ] [MAIN ] FCN: Starting full primary refresh. [2025-10-20T18:43:04.776681] [INFO ] [REMOTE ] FCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:05.047134] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:43:05.054675] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-01. [2025-10-20T18:43:05.760249] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T18:43:05.778791] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T18:43:05.851366] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-02. [2025-10-20T18:43:06.651354] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-07. [2025-10-20T18:43:06.868720] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:43:07.451364] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-06. [2025-10-20T18:43:08.251346] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-08. [2025-10-20T18:43:08.448290] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T18:43:08.879190] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:43:09.051396] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-07. [2025-10-20T18:43:09.613184] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T18:43:09.851370] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-09. [2025-10-20T18:43:10.487197] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:43:10.651361] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-03. [2025-10-20T18:43:11.056566] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T18:43:11.451397] [INFO ] [REMOTE ] FBRT: Fetching primary data for 2025-10. [2025-10-20T18:43:11.961746] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T18:43:12.251392] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-04. [2025-10-20T18:43:13.012662] [INFO ] [REMOTE ] FBRT: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T18:43:13.019467] [INFO ] [REMOTE ] FBRT: Full fetch complete (49226 records). [2025-10-20T18:43:13.019705] [DEBUG] [VALIDATOR ] FBRT: Validating primary data. [2025-10-20T18:43:13.020896] [FATAL] [VALIDATOR ] FBRT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:43:13.020992] [FATAL] [MAIN ] FBRT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:13.021036] [WARN ] [MAIN ] FBRT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:13.021078] [INFO ] [LOCAL ] FBRT: Starting deletion of primary data. [2025-10-20T18:43:13.021164] [DEBUG] [LOCAL ] FBRT: No primary data file found to delete. [2025-10-20T18:43:13.021220] [INFO ] [LOCAL ] FBRT: Starting deletion of derivative data. [2025-10-20T18:43:13.021438] [DEBUG] [LOCAL ] FBRT: Deleted derivative data file. [2025-10-20T18:43:13.021582] [INFO ] [MAIN ] Update Progress: 806/2367 symbols completed. FBRT took 32.0s. Estimated remaining: 3h 19m 46s. Rate: 7.8 symbols/min [2025-10-20T18:43:13.056696] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-08. [2025-10-20T18:43:13.217242] [INFO ] [MAIN ] FCPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:13.263174] [DEBUG] [LOCAL ] FCPT: Loaded 37885 existing records from primary storage. [2025-10-20T18:43:13.263617] [INFO ] [MAIN ] FCPT: Loaded 37885 existing records. [2025-10-20T18:43:13.462034] [WARN ] [MAIN ] FCPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:13.462359] [INFO ] [MAIN ] FCPT: Wiping existing primary data before full refresh. [2025-10-20T18:43:13.462413] [INFO ] [LOCAL ] FCPT: Starting deletion of primary data. [2025-10-20T18:43:13.462709] [DEBUG] [LOCAL ] FCPT: Deleted primary data file. [2025-10-20T18:43:13.462771] [INFO ] [MAIN ] FCPT: Starting full primary refresh. [2025-10-20T18:43:13.859733] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-08. [2025-10-20T18:43:13.864311] [INFO ] [REMOTE ] FCPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:13.868864] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:43:14.633199] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T18:43:14.651367] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-01. [2025-10-20T18:43:15.451394] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-05. [2025-10-20T18:43:15.506221] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T18:43:16.102815] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:43:16.251420] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-02. [2025-10-20T18:43:16.450169] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T18:43:17.051426] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-09. [2025-10-20T18:43:17.768388] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:43:17.851382] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-03. [2025-10-20T18:43:18.296928] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:43:18.651376] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-09. [2025-10-20T18:43:19.401472] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:43:19.451383] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-04. [2025-10-20T18:43:20.063918] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:43:20.251470] [INFO ] [REMOTE ] FCF: Fetching primary data for 2025-10. [2025-10-20T18:43:20.679291] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T18:43:21.051406] [INFO ] [REMOTE ] FBP: Fetching primary data for 2025-10. [2025-10-20T18:43:21.714975] [INFO ] [REMOTE ] FCF: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:43:21.720588] [INFO ] [REMOTE ] FCF: Full fetch complete (57955 records). [2025-10-20T18:43:21.720713] [DEBUG] [VALIDATOR ] FCF: Validating primary data. [2025-10-20T18:43:21.721481] [FATAL] [VALIDATOR ] FCF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:43:21.721567] [FATAL] [MAIN ] FCF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:21.721610] [WARN ] [MAIN ] FCF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:21.721674] [INFO ] [LOCAL ] FCF: Starting deletion of primary data. [2025-10-20T18:43:21.721752] [DEBUG] [LOCAL ] FCF: No primary data file found to delete. [2025-10-20T18:43:21.721789] [INFO ] [LOCAL ] FCF: Starting deletion of derivative data. [2025-10-20T18:43:21.721961] [DEBUG] [LOCAL ] FCF: Deleted derivative data file. [2025-10-20T18:43:21.722047] [INFO ] [MAIN ] Update Progress: 807/2367 symbols completed. FCF took 31.9s. Estimated remaining: 3h 19m 40s. Rate: 7.8 symbols/min [2025-10-20T18:43:21.858424] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-05. [2025-10-20T18:43:21.890238] [INFO ] [MAIN ] FCRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:21.890425] [DEBUG] [LOCAL ] FCRX: No existing data file found in primary storage. [2025-10-20T18:43:21.890472] [WARN ] [MAIN ] FCRX: No existing data found, performing full refresh instead. [2025-10-20T18:43:21.890574] [INFO ] [MAIN ] FCRX: Wiping existing primary data before full refresh. [2025-10-20T18:43:21.890614] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-10-20T18:43:21.890690] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-10-20T18:43:21.890741] [INFO ] [MAIN ] FCRX: Starting full primary refresh. [2025-10-20T18:43:22.245513] [INFO ] [REMOTE ] FCRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:22.651342] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-06. [2025-10-20T18:43:22.797564] [INFO ] [REMOTE ] FBP: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:43:22.804611] [INFO ] [REMOTE ] FBP: Full fetch complete (66777 records). [2025-10-20T18:43:22.804754] [DEBUG] [VALIDATOR ] FBP: Validating primary data. [2025-10-20T18:43:23.187372] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:43:23.231187] [DEBUG] [VALIDATOR ] FBP: All expected trading days are present. [2025-10-20T18:43:23.231590] [DEBUG] [VALIDATOR ] FBP: Primary data validation passed. [2025-10-20T18:43:23.232432] [INFO ] [MAIN ] FBP: Fetched primary data is valid. [2025-10-20T18:43:23.232491] [DEBUG] [LOCAL ] FBP: Saving primary data. [2025-10-20T18:43:23.233626] [DEBUG] [LOCAL ] FBP: Trimmed 66777 records to 45752 (keeping 200 days of data) [2025-10-20T18:43:23.454545] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-06. [2025-10-20T18:43:23.505457] [DEBUG] [LOCAL ] FBP: Primary data saved (45752 records). [2025-10-20T18:43:23.505571] [INFO ] [MAIN ] FBP: Full history saved. [2025-10-20T18:43:23.505664] [INFO ] [MAIN ] Update Progress: 808/2367 symbols completed. FBP took 46.0s. Estimated remaining: 3h 19m 21s. Rate: 7.8 symbols/min [2025-10-20T18:43:23.677821] [INFO ] [MAIN ] FCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:23.692476] [DEBUG] [LOCAL ] FCT: Loaded 14906 existing records from primary storage. [2025-10-20T18:43:23.692696] [INFO ] [MAIN ] FCT: Loaded 14906 existing records. [2025-10-20T18:43:23.873665] [WARN ] [MAIN ] FCT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:23.887805] [INFO ] [MAIN ] FCT: Wiping existing primary data before full refresh. [2025-10-20T18:43:23.888810] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T18:43:23.888338] [INFO ] [LOCAL ] FCT: Starting deletion of primary data. [2025-10-20T18:43:23.894847] [DEBUG] [LOCAL ] FCT: Deleted primary data file. [2025-10-20T18:43:23.894905] [INFO ] [MAIN ] FCT: Starting full primary refresh. [2025-10-20T18:43:24.239926] [INFO ] [REMOTE ] FCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:24.251335] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-01. [2025-10-20T18:43:24.878784] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:43:25.051343] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-07. [2025-10-20T18:43:25.361088] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:43:25.851364] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-02. [2025-10-20T18:43:26.308389] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T18:43:26.630321] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T18:43:26.651332] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-03. [2025-10-20T18:43:27.451456] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-07. [2025-10-20T18:43:27.561194] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:43:28.251358] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-04. [2025-10-20T18:43:28.731784] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:43:29.051388] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-08. [2025-10-20T18:43:29.162429] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:43:29.851429] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-08. [2025-10-20T18:43:30.188338] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T18:43:30.651360] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-01. [2025-10-20T18:43:31.225355] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:43:31.246462] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T18:43:31.451373] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-05. [2025-10-20T18:43:32.174468] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T18:43:32.251392] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-09. [2025-10-20T18:43:33.051373] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-02. [2025-10-20T18:43:33.301778] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T18:43:33.464678] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:43:33.851394] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-06. [2025-10-20T18:43:34.651383] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-03. [2025-10-20T18:43:34.817247] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T18:43:34.978158] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-10-20T18:43:35.451377] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-07. [2025-10-20T18:43:36.251344] [INFO ] [REMOTE ] FCN: Fetching primary data for 2025-10. [2025-10-20T18:43:36.615724] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T18:43:37.051364] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-04. [2025-10-20T18:43:37.397346] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T18:43:37.597460] [INFO ] [REMOTE ] FCN: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:43:37.603739] [INFO ] [REMOTE ] FCN: Full fetch complete (36076 records). [2025-10-20T18:43:37.603927] [DEBUG] [VALIDATOR ] FCN: Validating primary data. [2025-10-20T18:43:37.857352] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-08. [2025-10-20T18:43:37.988946] [DEBUG] [VALIDATOR ] FCN: All expected trading days are present. [2025-10-20T18:43:37.989027] [DEBUG] [VALIDATOR ] FCN: Primary data validation passed. [2025-10-20T18:43:37.989712] [INFO ] [MAIN ] FCN: Fetched primary data is valid. [2025-10-20T18:43:37.989774] [DEBUG] [LOCAL ] FCN: Saving primary data. [2025-10-20T18:43:37.990675] [DEBUG] [LOCAL ] FCN: Trimmed 36076 records to 25974 (keeping 200 days of data) [2025-10-20T18:43:38.154630] [DEBUG] [LOCAL ] FCN: Primary data saved (25974 records). [2025-10-20T18:43:38.154809] [INFO ] [MAIN ] FCN: Full history saved. [2025-10-20T18:43:38.154909] [INFO ] [MAIN ] Update Progress: 809/2367 symbols completed. FCN took 34.1s. Estimated remaining: 3h 19m 27s. Rate: 7.8 symbols/min [2025-10-20T18:43:38.330737] [INFO ] [MAIN ] FCX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:38.378965] [DEBUG] [LOCAL ] FCX: Loaded 54481 existing records from primary storage. [2025-10-20T18:43:38.379297] [INFO ] [MAIN ] FCX: Loaded 54481 existing records. [2025-10-20T18:43:38.558396] [WARN ] [MAIN ] FCX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:38.558829] [INFO ] [MAIN ] FCX: Wiping existing primary data before full refresh. [2025-10-20T18:43:38.558939] [INFO ] [LOCAL ] FCX: Starting deletion of primary data. [2025-10-20T18:43:38.559264] [DEBUG] [LOCAL ] FCX: Deleted primary data file. [2025-10-20T18:43:38.559319] [INFO ] [MAIN ] FCX: Starting full primary refresh. [2025-10-20T18:43:38.656035] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-05. [2025-10-20T18:43:38.862171] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T18:43:38.932976] [INFO ] [REMOTE ] FCX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:38.935741] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-10-20T18:43:39.451368] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-09. [2025-10-20T18:43:40.251353] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-06. [2025-10-20T18:43:40.492978] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T18:43:40.769418] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T18:43:41.051366] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-07. [2025-10-20T18:43:41.509190] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T18:43:41.851352] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-08. [2025-10-20T18:43:42.189559] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:43:42.651387] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-09. [2025-10-20T18:43:43.451373] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-09. [2025-10-20T18:43:43.875710] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-10-20T18:43:44.039573] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:43:44.251375] [INFO ] [REMOTE ] FCRX: Fetching primary data for 2025-10. [2025-10-20T18:43:44.621279] [INFO ] [REMOTE ] FCRX: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T18:43:44.622103] [INFO ] [REMOTE ] FCRX: Full fetch complete (1642 records). [2025-10-20T18:43:44.622189] [DEBUG] [VALIDATOR ] FCRX: Validating primary data. [2025-10-20T18:43:44.799629] [WARN ] [VALIDATOR ] FCRX: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:43:44.799860] [FATAL] [MAIN ] FCRX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:43:44.799929] [WARN ] [MAIN ] FCRX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:43:44.799999] [INFO ] [LOCAL ] FCRX: Starting deletion of primary data. [2025-10-20T18:43:44.800060] [DEBUG] [LOCAL ] FCRX: No primary data file found to delete. [2025-10-20T18:43:44.800096] [INFO ] [LOCAL ] FCRX: Starting deletion of derivative data. [2025-10-20T18:43:44.800133] [DEBUG] [LOCAL ] FCRX: No derivative data file found to delete. [2025-10-20T18:43:44.800185] [INFO ] [MAIN ] Update Progress: 810/2367 symbols completed. FCRX took 23.1s. Estimated remaining: 3h 19m 17s. Rate: 7.8 symbols/min [2025-10-20T18:43:44.965010] [INFO ] [MAIN ] FDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:44.995749] [DEBUG] [LOCAL ] FDP: Loaded 30199 existing records from primary storage. [2025-10-20T18:43:44.996007] [INFO ] [MAIN ] FDP: Loaded 30199 existing records. [2025-10-20T18:43:45.056502] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-01. [2025-10-20T18:43:45.173542] [WARN ] [MAIN ] FDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:45.173783] [INFO ] [MAIN ] FDP: Wiping existing primary data before full refresh. [2025-10-20T18:43:45.173831] [INFO ] [LOCAL ] FDP: Starting deletion of primary data. [2025-10-20T18:43:45.174130] [DEBUG] [LOCAL ] FDP: Deleted primary data file. [2025-10-20T18:43:45.174193] [INFO ] [MAIN ] FDP: Starting full primary refresh. [2025-10-20T18:43:45.545141] [INFO ] [REMOTE ] FDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:45.851377] [INFO ] [REMOTE ] FCPT: Fetching primary data for 2025-10. [2025-10-20T18:43:46.651356] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-01. [2025-10-20T18:43:47.271327] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-01 (2.21 seconds). [2025-10-20T18:43:47.312287] [INFO ] [REMOTE ] FCPT: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:43:47.318232] [INFO ] [REMOTE ] FCPT: Full fetch complete (53882 records). [2025-10-20T18:43:47.318367] [DEBUG] [VALIDATOR ] FCPT: Validating primary data. [2025-10-20T18:43:47.457415] [INFO ] [REMOTE ] FCT: Fetching primary data for 2025-10. [2025-10-20T18:43:47.703378] [DEBUG] [VALIDATOR ] FCPT: All expected trading days are present. [2025-10-20T18:43:47.703473] [DEBUG] [VALIDATOR ] FCPT: Primary data validation passed. [2025-10-20T18:43:47.704219] [INFO ] [MAIN ] FCPT: Fetched primary data is valid. [2025-10-20T18:43:47.704283] [DEBUG] [LOCAL ] FCPT: Saving primary data. [2025-10-20T18:43:47.705483] [DEBUG] [LOCAL ] FCPT: Trimmed 53882 records to 37210 (keeping 200 days of data) [2025-10-20T18:43:47.890964] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:43:47.948300] [DEBUG] [LOCAL ] FCPT: Primary data saved (37210 records). [2025-10-20T18:43:47.948440] [INFO ] [MAIN ] FCPT: Full history saved. [2025-10-20T18:43:47.948925] [INFO ] [MAIN ] Update Progress: 811/2367 symbols completed. FCPT took 34.9s. Estimated remaining: 3h 19m 1s. Rate: 7.8 symbols/min [2025-10-20T18:43:48.115560] [INFO ] [MAIN ] FDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:48.143006] [DEBUG] [LOCAL ] FDS: Loaded 28183 existing records from primary storage. [2025-10-20T18:43:48.143256] [INFO ] [MAIN ] FDS: Loaded 28183 existing records. [2025-10-20T18:43:48.256538] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-02. [2025-10-20T18:43:48.335419] [WARN ] [MAIN ] FDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:48.335668] [INFO ] [MAIN ] FDS: Wiping existing primary data before full refresh. [2025-10-20T18:43:48.335729] [INFO ] [LOCAL ] FDS: Starting deletion of primary data. [2025-10-20T18:43:48.335953] [DEBUG] [LOCAL ] FDS: Deleted primary data file. [2025-10-20T18:43:48.336009] [INFO ] [MAIN ] FDS: Starting full primary refresh. [2025-10-20T18:43:48.713239] [INFO ] [REMOTE ] FDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:48.785548] [INFO ] [REMOTE ] FCT: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T18:43:48.788425] [INFO ] [REMOTE ] FCT: Full fetch complete (21124 records). [2025-10-20T18:43:48.788503] [DEBUG] [VALIDATOR ] FCT: Validating primary data. [2025-10-20T18:43:49.056595] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-01. [2025-10-20T18:43:49.164210] [DEBUG] [VALIDATOR ] FCT: All expected trading days are present. [2025-10-20T18:43:49.164287] [DEBUG] [VALIDATOR ] FCT: Primary data validation passed. [2025-10-20T18:43:49.164774] [INFO ] [MAIN ] FCT: Fetched primary data is valid. [2025-10-20T18:43:49.164835] [DEBUG] [LOCAL ] FCT: Saving primary data. [2025-10-20T18:43:49.165499] [DEBUG] [LOCAL ] FCT: Trimmed 21124 records to 14667 (keeping 200 days of data) [2025-10-20T18:43:49.248795] [DEBUG] [LOCAL ] FCT: Primary data saved (14667 records). [2025-10-20T18:43:49.248933] [INFO ] [MAIN ] FCT: Full history saved. [2025-10-20T18:43:49.249009] [INFO ] [MAIN ] Update Progress: 812/2367 symbols completed. FCT took 25.7s. Estimated remaining: 3h 18m 41s. Rate: 7.8 symbols/min [2025-10-20T18:43:49.432758] [INFO ] [MAIN ] FDX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:43:49.481061] [DEBUG] [LOCAL ] FDX: Loaded 52099 existing records from primary storage. [2025-10-20T18:43:49.481379] [INFO ] [MAIN ] FDX: Loaded 52099 existing records. [2025-10-20T18:43:49.657552] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:43:49.685048] [WARN ] [MAIN ] FDX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:43:49.685241] [INFO ] [MAIN ] FDX: Wiping existing primary data before full refresh. [2025-10-20T18:43:49.685289] [INFO ] [LOCAL ] FDX: Starting deletion of primary data. [2025-10-20T18:43:49.685484] [DEBUG] [LOCAL ] FDX: Deleted primary data file. [2025-10-20T18:43:49.685528] [INFO ] [MAIN ] FDX: Starting full primary refresh. [2025-10-20T18:43:49.855923] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-03. [2025-10-20T18:43:50.040272] [INFO ] [REMOTE ] FDX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:43:50.363554] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T18:43:50.651375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-02. [2025-10-20T18:43:51.451434] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-02. [2025-10-20T18:43:51.481927] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T18:43:52.251427] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-04. [2025-10-20T18:43:52.553126] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T18:43:52.711685] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T18:43:53.051362] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-03. [2025-10-20T18:43:53.160291] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T18:43:53.851426] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-01. [2025-10-20T18:43:54.651496] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-05. [2025-10-20T18:43:55.281173] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T18:43:55.451370] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-04. [2025-10-20T18:43:55.567808] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T18:43:55.964397] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:43:56.251433] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-06. [2025-10-20T18:43:57.051427] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-02. [2025-10-20T18:43:57.573879] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T18:43:57.730152] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-04 (2.28 seconds). [2025-10-20T18:43:57.851375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-05. [2025-10-20T18:43:58.651463] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-03. [2025-10-20T18:43:58.804536] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T18:43:59.451399] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-07. [2025-10-20T18:43:59.878731] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T18:43:59.972912] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:44:00.251416] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-06. [2025-10-20T18:44:00.514156] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:44:01.051389] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-03. [2025-10-20T18:44:01.851364] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-08. [2025-10-20T18:44:02.082531] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T18:44:02.651375] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-07. [2025-10-20T18:44:03.317168] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-10-20T18:44:03.441457] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:44:03.451330] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-04. [2025-10-20T18:44:04.251370] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-09. [2025-10-20T18:44:05.051375] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-04. [2025-10-20T18:44:05.166034] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T18:44:05.240843] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T18:44:05.503607] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:44:05.851362] [INFO ] [REMOTE ] FDP: Fetching primary data for 2025-10. [2025-10-20T18:44:06.171172] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T18:44:06.651363] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-05. [2025-10-20T18:44:07.121451] [INFO ] [REMOTE ] FDP: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:44:07.125854] [INFO ] [REMOTE ] FDP: Full fetch complete (42759 records). [2025-10-20T18:44:07.125999] [DEBUG] [VALIDATOR ] FDP: Validating primary data. [2025-10-20T18:44:07.456550] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-05. [2025-10-20T18:44:07.521217] [DEBUG] [VALIDATOR ] FDP: All expected trading days are present. [2025-10-20T18:44:07.521423] [DEBUG] [VALIDATOR ] FDP: Primary data validation passed. [2025-10-20T18:44:07.522168] [INFO ] [MAIN ] FDP: Fetched primary data is valid. [2025-10-20T18:44:07.522232] [DEBUG] [LOCAL ] FDP: Saving primary data. [2025-10-20T18:44:07.523451] [DEBUG] [LOCAL ] FDP: Trimmed 42759 records to 29728 (keeping 200 days of data) [2025-10-20T18:44:07.707429] [DEBUG] [LOCAL ] FDP: Primary data saved (29728 records). [2025-10-20T18:44:07.707707] [INFO ] [MAIN ] FDP: Full history saved. [2025-10-20T18:44:07.707851] [INFO ] [MAIN ] Update Progress: 813/2367 symbols completed. FDP took 22.9s. Estimated remaining: 3h 18m 54s. Rate: 7.8 symbols/min [2025-10-20T18:44:07.836085] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:44:07.912999] [INFO ] [MAIN ] FE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:07.963409] [DEBUG] [LOCAL ] FE: Loaded 54212 existing records from primary storage. [2025-10-20T18:44:07.963772] [INFO ] [MAIN ] FE: Loaded 54212 existing records. [2025-10-20T18:44:08.146660] [WARN ] [MAIN ] FE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:08.146898] [INFO ] [MAIN ] FE: Wiping existing primary data before full refresh. [2025-10-20T18:44:08.146946] [INFO ] [LOCAL ] FE: Starting deletion of primary data. [2025-10-20T18:44:08.147160] [DEBUG] [LOCAL ] FE: Deleted primary data file. [2025-10-20T18:44:08.147218] [INFO ] [MAIN ] FE: Starting full primary refresh. [2025-10-20T18:44:08.256733] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-06. [2025-10-20T18:44:08.513759] [INFO ] [REMOTE ] FE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:09.051438] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-01. [2025-10-20T18:44:09.569333] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:44:09.606065] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T18:44:09.851457] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-07. [2025-10-20T18:44:10.651369] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-06. [2025-10-20T18:44:10.752690] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:44:10.781189] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T18:44:11.451384] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-02. [2025-10-20T18:44:12.251412] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-08. [2025-10-20T18:44:12.588126] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T18:44:12.674557] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T18:44:13.051450] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-03. [2025-10-20T18:44:13.449266] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T18:44:13.851375] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-09. [2025-10-20T18:44:14.651452] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-08. [2025-10-20T18:44:14.777050] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T18:44:15.336757] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:44:15.451455] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-04. [2025-10-20T18:44:16.251428] [INFO ] [REMOTE ] FDS: Fetching primary data for 2025-10. [2025-10-20T18:44:16.795054] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T18:44:17.051421] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-07. [2025-10-20T18:44:17.254793] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:44:17.851494] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-05. [2025-10-20T18:44:18.075120] [INFO ] [REMOTE ] FDS: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T18:44:18.081221] [INFO ] [REMOTE ] FDS: Full fetch complete (39049 records). [2025-10-20T18:44:18.081344] [DEBUG] [VALIDATOR ] FDS: Validating primary data. [2025-10-20T18:44:18.459999] [DEBUG] [VALIDATOR ] FDS: All expected trading days are present. [2025-10-20T18:44:18.460089] [DEBUG] [VALIDATOR ] FDS: Primary data validation passed. [2025-10-20T18:44:18.460796] [INFO ] [MAIN ] FDS: Fetched primary data is valid. [2025-10-20T18:44:18.460857] [DEBUG] [LOCAL ] FDS: Saving primary data. [2025-10-20T18:44:18.461711] [DEBUG] [LOCAL ] FDS: Trimmed 39049 records to 27869 (keeping 200 days of data) [2025-10-20T18:44:18.632215] [DEBUG] [LOCAL ] FDS: Primary data saved (27869 records). [2025-10-20T18:44:18.632353] [INFO ] [MAIN ] FDS: Full history saved. [2025-10-20T18:44:18.632433] [INFO ] [MAIN ] Update Progress: 814/2367 symbols completed. FDS took 30.7s. Estimated remaining: 3h 18m 52s. Rate: 7.8 symbols/min [2025-10-20T18:44:18.662680] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-09. [2025-10-20T18:44:18.805273] [INFO ] [MAIN ] FENG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:18.805435] [DEBUG] [LOCAL ] FENG: No existing data file found in primary storage. [2025-10-20T18:44:18.805472] [WARN ] [MAIN ] FENG: No existing data found, performing full refresh instead. [2025-10-20T18:44:18.805580] [INFO ] [MAIN ] FENG: Wiping existing primary data before full refresh. [2025-10-20T18:44:18.805621] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-10-20T18:44:18.805698] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-10-20T18:44:18.805744] [INFO ] [MAIN ] FENG: Starting full primary refresh. [2025-10-20T18:44:19.171290] [INFO ] [REMOTE ] FENG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:19.202115] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T18:44:19.451381] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-01. [2025-10-20T18:44:19.528712] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:44:20.063357] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T18:44:20.251404] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-06. [2025-10-20T18:44:20.866078] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T18:44:21.051380] [INFO ] [REMOTE ] FCX: Fetching primary data for 2025-10. [2025-10-20T18:44:21.851418] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-08. [2025-10-20T18:44:21.934909] [INFO ] [REMOTE ] FCX: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:44:21.953365] [INFO ] [REMOTE ] FCX: Full fetch complete (77878 records). [2025-10-20T18:44:21.964535] [DEBUG] [VALIDATOR ] FCX: Validating primary data. [2025-10-20T18:44:22.014545] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T18:44:22.380089] [DEBUG] [VALIDATOR ] FCX: All expected trading days are present. [2025-10-20T18:44:22.380179] [DEBUG] [VALIDATOR ] FCX: Primary data validation passed. [2025-10-20T18:44:22.381168] [INFO ] [MAIN ] FCX: Fetched primary data is valid. [2025-10-20T18:44:22.381236] [DEBUG] [LOCAL ] FCX: Saving primary data. [2025-10-20T18:44:22.382982] [DEBUG] [LOCAL ] FCX: Trimmed 77878 records to 53701 (keeping 200 days of data) [2025-10-20T18:44:22.651425] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-07. [2025-10-20T18:44:22.756463] [DEBUG] [LOCAL ] FCX: Primary data saved (53701 records). [2025-10-20T18:44:22.756625] [INFO ] [MAIN ] FCX: Full history saved. [2025-10-20T18:44:22.756748] [INFO ] [MAIN ] Update Progress: 815/2367 symbols completed. FCX took 44.6s. Estimated remaining: 3h 18m 38s. Rate: 7.8 symbols/min [2025-10-20T18:44:23.001665] [INFO ] [MAIN ] FERG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:23.067785] [DEBUG] [LOCAL ] FERG: Loaded 49492 existing records from primary storage. [2025-10-20T18:44:23.068235] [INFO ] [MAIN ] FERG: Loaded 49492 existing records. [2025-10-20T18:44:23.334638] [WARN ] [MAIN ] FERG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:23.334933] [INFO ] [MAIN ] FERG: Wiping existing primary data before full refresh. [2025-10-20T18:44:23.334983] [INFO ] [LOCAL ] FERG: Starting deletion of primary data. [2025-10-20T18:44:23.335468] [DEBUG] [LOCAL ] FERG: Deleted primary data file. [2025-10-20T18:44:23.335507] [INFO ] [MAIN ] FERG: Starting full primary refresh. [2025-10-20T18:44:23.455400] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-02. [2025-10-20T18:44:23.815213] [INFO ] [REMOTE ] FERG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:23.850494] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T18:44:23.918494] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:44:24.251368] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-01. [2025-10-20T18:44:24.766556] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T18:44:25.051367] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-03. [2025-10-20T18:44:25.355748] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-03 (0.30 seconds). [2025-10-20T18:44:25.798119] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T18:44:25.851346] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-04. [2025-10-20T18:44:26.339780] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T18:44:26.651389] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-02. [2025-10-20T18:44:27.451419] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-08. [2025-10-20T18:44:28.156399] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T18:44:28.251365] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-03. [2025-10-20T18:44:29.051423] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-05. [2025-10-20T18:44:29.168899] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:44:29.485960] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:44:29.851368] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-09. [2025-10-20T18:44:30.081014] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T18:44:30.651355] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-04. [2025-10-20T18:44:31.451389] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-06. [2025-10-20T18:44:31.664725] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:44:31.870912] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T18:44:32.251389] [INFO ] [REMOTE ] FE: Fetching primary data for 2025-10. [2025-10-20T18:44:32.408245] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:44:33.051369] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-05. [2025-10-20T18:44:33.180045] [INFO ] [REMOTE ] FE: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:44:33.188134] [INFO ] [REMOTE ] FE: Full fetch complete (77475 records). [2025-10-20T18:44:33.188270] [DEBUG] [VALIDATOR ] FE: Validating primary data. [2025-10-20T18:44:33.578658] [DEBUG] [VALIDATOR ] FE: All expected trading days are present. [2025-10-20T18:44:33.578751] [DEBUG] [VALIDATOR ] FE: Primary data validation passed. [2025-10-20T18:44:33.579768] [INFO ] [MAIN ] FE: Fetched primary data is valid. [2025-10-20T18:44:33.579831] [DEBUG] [LOCAL ] FE: Saving primary data. [2025-10-20T18:44:33.581293] [DEBUG] [LOCAL ] FE: Trimmed 77475 records to 53436 (keeping 200 days of data) [2025-10-20T18:44:33.858903] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-09. [2025-10-20T18:44:33.902008] [DEBUG] [LOCAL ] FE: Primary data saved (53436 records). [2025-10-20T18:44:33.902141] [INFO ] [MAIN ] FE: Full history saved. [2025-10-20T18:44:33.902913] [INFO ] [MAIN ] Update Progress: 816/2367 symbols completed. FE took 26.2s. Estimated remaining: 3h 18m 37s. Rate: 7.8 symbols/min [2025-10-20T18:44:34.077044] [INFO ] [MAIN ] FET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:34.092735] [DEBUG] [LOCAL ] FET: Loaded 13043 existing records from primary storage. [2025-10-20T18:44:34.092979] [INFO ] [MAIN ] FET: Loaded 13043 existing records. [2025-10-20T18:44:34.269868] [WARN ] [MAIN ] FET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:34.270074] [INFO ] [MAIN ] FET: Wiping existing primary data before full refresh. [2025-10-20T18:44:34.270123] [INFO ] [LOCAL ] FET: Starting deletion of primary data. [2025-10-20T18:44:34.270282] [DEBUG] [LOCAL ] FET: Deleted primary data file. [2025-10-20T18:44:34.270323] [INFO ] [MAIN ] FET: Starting full primary refresh. [2025-10-20T18:44:34.629055] [INFO ] [REMOTE ] FET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:34.651338] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-01. [2025-10-20T18:44:34.838443] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T18:44:35.321097] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T18:44:35.451362] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-06. [2025-10-20T18:44:35.684815] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:44:36.251407] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-07. [2025-10-20T18:44:36.631232] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-10-20T18:44:37.056519] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-08. [2025-10-20T18:44:37.072763] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T18:44:37.374952] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-10-20T18:44:37.851405] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-09. [2025-10-20T18:44:38.314860] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:44:38.651373] [INFO ] [REMOTE ] FENG: Fetching primary data for 2025-10. [2025-10-20T18:44:39.109251] [INFO ] [REMOTE ] FENG: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:44:39.110076] [INFO ] [REMOTE ] FENG: Full fetch complete (2278 records). [2025-10-20T18:44:39.110143] [DEBUG] [VALIDATOR ] FENG: Validating primary data. [2025-10-20T18:44:39.285985] [WARN ] [VALIDATOR ] FENG: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T18:44:39.286195] [FATAL] [MAIN ] FENG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:44:39.286265] [WARN ] [MAIN ] FENG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:44:39.286336] [INFO ] [LOCAL ] FENG: Starting deletion of primary data. [2025-10-20T18:44:39.286399] [DEBUG] [LOCAL ] FENG: No primary data file found to delete. [2025-10-20T18:44:39.286434] [INFO ] [LOCAL ] FENG: Starting deletion of derivative data. [2025-10-20T18:44:39.286474] [DEBUG] [LOCAL ] FENG: No derivative data file found to delete. [2025-10-20T18:44:39.286526] [INFO ] [MAIN ] Update Progress: 817/2367 symbols completed. FENG took 20.7s. Estimated remaining: 3h 18m 25s. Rate: 7.8 symbols/min [2025-10-20T18:44:39.455598] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-02. [2025-10-20T18:44:39.457298] [INFO ] [MAIN ] FF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:39.481985] [DEBUG] [LOCAL ] FF: Loaded 24686 existing records from primary storage. [2025-10-20T18:44:39.482224] [INFO ] [MAIN ] FF: Loaded 24686 existing records. [2025-10-20T18:44:39.656364] [WARN ] [MAIN ] FF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:39.656574] [INFO ] [MAIN ] FF: Wiping existing primary data before full refresh. [2025-10-20T18:44:39.656620] [INFO ] [LOCAL ] FF: Starting deletion of primary data. [2025-10-20T18:44:39.656857] [DEBUG] [LOCAL ] FF: Deleted primary data file. [2025-10-20T18:44:39.656913] [INFO ] [MAIN ] FF: Starting full primary refresh. [2025-10-20T18:44:40.032736] [INFO ] [REMOTE ] FF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:40.200009] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T18:44:40.251408] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-01. [2025-10-20T18:44:41.051383] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-07. [2025-10-20T18:44:41.616155] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T18:44:41.851439] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-02. [2025-10-20T18:44:42.651417] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-03. [2025-10-20T18:44:42.864690] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:44:43.071413] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T18:44:43.451386] [INFO ] [REMOTE ] FDX: Fetching primary data for 2025-10. [2025-10-20T18:44:43.492196] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T18:44:44.251412] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-04. [2025-10-20T18:44:45.051414] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-08. [2025-10-20T18:44:45.166988] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T18:44:45.841256] [INFO ] [REMOTE ] FDX: Incremental fetch complete for 2025-10 (2.39 seconds). [2025-10-20T18:44:45.851525] [INFO ] [REMOTE ] FDX: Full fetch complete (74329 records). [2025-10-20T18:44:45.851353] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-05. [2025-10-20T18:44:45.852392] [DEBUG] [VALIDATOR ] FDX: Validating primary data. [2025-10-20T18:44:46.246365] [DEBUG] [VALIDATOR ] FDX: All expected trading days are present. [2025-10-20T18:44:46.246453] [DEBUG] [VALIDATOR ] FDX: Primary data validation passed. [2025-10-20T18:44:46.247341] [INFO ] [MAIN ] FDX: Fetched primary data is valid. [2025-10-20T18:44:46.247405] [DEBUG] [LOCAL ] FDX: Saving primary data. [2025-10-20T18:44:46.248892] [DEBUG] [LOCAL ] FDX: Trimmed 74329 records to 51326 (keeping 200 days of data) [2025-10-20T18:44:46.564036] [DEBUG] [LOCAL ] FDX: Primary data saved (51326 records). [2025-10-20T18:44:46.564157] [INFO ] [MAIN ] FDX: Full history saved. [2025-10-20T18:44:46.564274] [INFO ] [MAIN ] Update Progress: 818/2367 symbols completed. FDX took 57.3s. Estimated remaining: 3h 18m 16s. Rate: 7.8 symbols/min [2025-10-20T18:44:46.654690] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-03. [2025-10-20T18:44:46.749471] [INFO ] [MAIN ] FFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:46.759032] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T18:44:46.791882] [DEBUG] [LOCAL ] FFA: Loaded 5984 existing records from primary storage. [2025-10-20T18:44:46.793627] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T18:44:46.793779] [INFO ] [MAIN ] FFA: Loaded 5984 existing records. [2025-10-20T18:44:46.975924] [WARN ] [MAIN ] FFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:46.976181] [INFO ] [MAIN ] FFA: Wiping existing primary data before full refresh. [2025-10-20T18:44:46.976237] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-10-20T18:44:46.976378] [DEBUG] [LOCAL ] FFA: Deleted primary data file. [2025-10-20T18:44:46.976415] [INFO ] [MAIN ] FFA: Starting full primary refresh. [2025-10-20T18:44:47.323894] [INFO ] [REMOTE ] FFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:47.451417] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-09. [2025-10-20T18:44:47.975578] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T18:44:48.251395] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-06. [2025-10-20T18:44:49.049012] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T18:44:49.051791] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-04. [2025-10-20T18:44:49.139284] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:44:49.851377] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-07. [2025-10-20T18:44:50.297375] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T18:44:50.651401] [INFO ] [REMOTE ] FERG: Fetching primary data for 2025-10. [2025-10-20T18:44:50.698637] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:44:51.451372] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-01. [2025-10-20T18:44:52.022677] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T18:44:52.251380] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-05. [2025-10-20T18:44:52.387437] [INFO ] [REMOTE ] FERG: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T18:44:52.395193] [INFO ] [REMOTE ] FERG: Full fetch complete (71210 records). [2025-10-20T18:44:52.395388] [DEBUG] [VALIDATOR ] FERG: Validating primary data. [2025-10-20T18:44:52.793497] [DEBUG] [VALIDATOR ] FERG: All expected trading days are present. [2025-10-20T18:44:52.793585] [DEBUG] [VALIDATOR ] FERG: Primary data validation passed. [2025-10-20T18:44:52.794438] [INFO ] [MAIN ] FERG: Fetched primary data is valid. [2025-10-20T18:44:52.794502] [DEBUG] [LOCAL ] FERG: Saving primary data. [2025-10-20T18:44:52.795875] [DEBUG] [LOCAL ] FERG: Trimmed 71210 records to 48769 (keeping 200 days of data) [2025-10-20T18:44:53.067880] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-02. [2025-10-20T18:44:53.100529] [DEBUG] [LOCAL ] FERG: Primary data saved (48769 records). [2025-10-20T18:44:53.100676] [INFO ] [MAIN ] FERG: Full history saved. [2025-10-20T18:44:53.100795] [INFO ] [MAIN ] Update Progress: 819/2367 symbols completed. FERG took 30.3s. Estimated remaining: 3h 18m 6s. Rate: 7.8 symbols/min [2025-10-20T18:44:53.283445] [INFO ] [MAIN ] FFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:44:53.301547] [DEBUG] [LOCAL ] FFC: Loaded 16896 existing records from primary storage. [2025-10-20T18:44:53.301821] [INFO ] [MAIN ] FFC: Loaded 16896 existing records. [2025-10-20T18:44:53.486767] [WARN ] [MAIN ] FFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:44:53.487068] [INFO ] [MAIN ] FFC: Wiping existing primary data before full refresh. [2025-10-20T18:44:53.487148] [INFO ] [LOCAL ] FFC: Starting deletion of primary data. [2025-10-20T18:44:53.487475] [DEBUG] [LOCAL ] FFC: Deleted primary data file. [2025-10-20T18:44:53.487538] [INFO ] [MAIN ] FFC: Starting full primary refresh. [2025-10-20T18:44:53.623212] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T18:44:53.708950] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T18:44:53.857066] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-03. [2025-10-20T18:44:53.897554] [INFO ] [REMOTE ] FFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:44:54.580911] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T18:44:54.651373] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-04. [2025-10-20T18:44:55.444520] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T18:44:55.451331] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-08. [2025-10-20T18:44:56.251370] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-05. [2025-10-20T18:44:56.494736] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:44:56.880516] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:44:57.051384] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-06. [2025-10-20T18:44:57.851395] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-01. [2025-10-20T18:44:58.086675] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:44:58.632443] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T18:44:58.651374] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-09. [2025-10-20T18:44:59.451352] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-02. [2025-10-20T18:44:59.750242] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T18:45:00.148902] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T18:45:00.251361] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-07. [2025-10-20T18:45:01.051355] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-03. [2025-10-20T18:45:01.376069] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:45:01.851375] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-08. [2025-10-20T18:45:02.186590] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:45:02.651392] [INFO ] [REMOTE ] FET: Fetching primary data for 2025-10. [2025-10-20T18:45:03.001816] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T18:45:03.451359] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-04. [2025-10-20T18:45:03.703072] [INFO ] [REMOTE ] FET: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T18:45:03.705569] [INFO ] [REMOTE ] FET: Full fetch complete (18434 records). [2025-10-20T18:45:03.705638] [DEBUG] [VALIDATOR ] FET: Validating primary data. [2025-10-20T18:45:04.075859] [DEBUG] [VALIDATOR ] FET: All expected trading days are present. [2025-10-20T18:45:04.076284] [DEBUG] [VALIDATOR ] FET: Primary data validation passed. [2025-10-20T18:45:04.076713] [INFO ] [MAIN ] FET: Fetched primary data is valid. [2025-10-20T18:45:04.076777] [DEBUG] [LOCAL ] FET: Saving primary data. [2025-10-20T18:45:04.077339] [DEBUG] [LOCAL ] FET: Trimmed 18434 records to 12965 (keeping 200 days of data) [2025-10-20T18:45:04.153221] [DEBUG] [LOCAL ] FET: Primary data saved (12965 records). [2025-10-20T18:45:04.153362] [INFO ] [MAIN ] FET: Full history saved. [2025-10-20T18:45:04.153451] [INFO ] [MAIN ] Update Progress: 820/2367 symbols completed. FET took 30.3s. Estimated remaining: 3h 18m 5s. Rate: 7.8 symbols/min [2025-10-20T18:45:04.256602] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-09. [2025-10-20T18:45:04.331124] [INFO ] [MAIN ] FFWM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:04.369951] [DEBUG] [LOCAL ] FFWM: Loaded 36515 existing records from primary storage. [2025-10-20T18:45:04.370211] [INFO ] [MAIN ] FFWM: Loaded 36515 existing records. [2025-10-20T18:45:04.560252] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T18:45:04.563879] [WARN ] [MAIN ] FFWM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:04.567574] [INFO ] [MAIN ] FFWM: Wiping existing primary data before full refresh. [2025-10-20T18:45:04.568020] [INFO ] [LOCAL ] FFWM: Starting deletion of primary data. [2025-10-20T18:45:04.568620] [DEBUG] [LOCAL ] FFWM: Deleted primary data file. [2025-10-20T18:45:04.568715] [INFO ] [MAIN ] FFWM: Starting full primary refresh. [2025-10-20T18:45:04.919736] [INFO ] [REMOTE ] FFWM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:05.051342] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-01. [2025-10-20T18:45:05.669387] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T18:45:05.851398] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-05. [2025-10-20T18:45:06.633336] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T18:45:06.651364] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-02. [2025-10-20T18:45:06.964552] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T18:45:07.451421] [INFO ] [REMOTE ] FF: Fetching primary data for 2025-10. [2025-10-20T18:45:07.975959] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:45:08.251358] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-06. [2025-10-20T18:45:08.655074] [INFO ] [REMOTE ] FF: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:45:08.658662] [INFO ] [REMOTE ] FF: Full fetch complete (36688 records). [2025-10-20T18:45:08.658744] [DEBUG] [VALIDATOR ] FF: Validating primary data. [2025-10-20T18:45:09.034902] [DEBUG] [VALIDATOR ] FF: All expected trading days are present. [2025-10-20T18:45:09.034998] [DEBUG] [VALIDATOR ] FF: Primary data validation passed. [2025-10-20T18:45:09.035755] [INFO ] [MAIN ] FF: Fetched primary data is valid. [2025-10-20T18:45:09.035818] [DEBUG] [LOCAL ] FF: Saving primary data. [2025-10-20T18:45:09.036618] [DEBUG] [LOCAL ] FF: Trimmed 36688 records to 24052 (keeping 200 days of data) [2025-10-20T18:45:09.082922] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-03. [2025-10-20T18:45:09.143804] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:45:09.181532] [DEBUG] [LOCAL ] FF: Primary data saved (24052 records). [2025-10-20T18:45:09.181673] [INFO ] [MAIN ] FF: Full history saved. [2025-10-20T18:45:09.181767] [INFO ] [MAIN ] Update Progress: 821/2367 symbols completed. FF took 29.9s. Estimated remaining: 3h 17m 52s. Rate: 7.8 symbols/min [2025-10-20T18:45:09.353599] [INFO ] [MAIN ] FG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:09.374477] [DEBUG] [LOCAL ] FG: Loaded 22484 existing records from primary storage. [2025-10-20T18:45:09.374723] [INFO ] [MAIN ] FG: Loaded 22484 existing records. [2025-10-20T18:45:09.550207] [WARN ] [MAIN ] FG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:09.550436] [INFO ] [MAIN ] FG: Wiping existing primary data before full refresh. [2025-10-20T18:45:09.550482] [INFO ] [LOCAL ] FG: Starting deletion of primary data. [2025-10-20T18:45:09.550697] [DEBUG] [LOCAL ] FG: Deleted primary data file. [2025-10-20T18:45:09.550762] [INFO ] [MAIN ] FG: Starting full primary refresh. [2025-10-20T18:45:09.863783] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-07. [2025-10-20T18:45:09.908456] [INFO ] [REMOTE ] FG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:10.500673] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T18:45:10.499869] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T18:45:10.651359] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-04. [2025-10-20T18:45:11.451393] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-08. [2025-10-20T18:45:11.973419] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T18:45:12.157846] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T18:45:12.251366] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-01. [2025-10-20T18:45:12.836177] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:45:13.051385] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-09. [2025-10-20T18:45:13.764320] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T18:45:13.851356] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-05. [2025-10-20T18:45:14.651398] [INFO ] [REMOTE ] FFA: Fetching primary data for 2025-10. [2025-10-20T18:45:15.312024] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T18:45:15.330782] [INFO ] [REMOTE ] FFA: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:45:15.333099] [INFO ] [REMOTE ] FFA: Full fetch complete (8628 records). [2025-10-20T18:45:15.333195] [DEBUG] [VALIDATOR ] FFA: Validating primary data. [2025-10-20T18:45:15.333703] [FATAL] [VALIDATOR ] FFA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T18:45:15.333803] [FATAL] [MAIN ] FFA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:45:15.333860] [WARN ] [MAIN ] FFA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:45:15.333939] [INFO ] [LOCAL ] FFA: Starting deletion of primary data. [2025-10-20T18:45:15.334036] [DEBUG] [LOCAL ] FFA: No primary data file found to delete. [2025-10-20T18:45:15.334086] [INFO ] [LOCAL ] FFA: Starting deletion of derivative data. [2025-10-20T18:45:15.334292] [DEBUG] [LOCAL ] FFA: Deleted derivative data file. [2025-10-20T18:45:15.334505] [INFO ] [MAIN ] Update Progress: 822/2367 symbols completed. FFA took 28.8s. Estimated remaining: 3h 17m 42s. Rate: 7.8 symbols/min [2025-10-20T18:45:15.456515] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-02. [2025-10-20T18:45:15.549220] [INFO ] [MAIN ] FGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:15.556627] [DEBUG] [LOCAL ] FGN: Loaded 3750 existing records from primary storage. [2025-10-20T18:45:15.556945] [INFO ] [MAIN ] FGN: Loaded 3750 existing records. [2025-10-20T18:45:15.796914] [WARN ] [MAIN ] FGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:15.797144] [INFO ] [MAIN ] FGN: Wiping existing primary data before full refresh. [2025-10-20T18:45:15.797194] [INFO ] [LOCAL ] FGN: Starting deletion of primary data. [2025-10-20T18:45:15.797379] [DEBUG] [LOCAL ] FGN: Deleted primary data file. [2025-10-20T18:45:15.797423] [INFO ] [MAIN ] FGN: Starting full primary refresh. [2025-10-20T18:45:16.256551] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-06. [2025-10-20T18:45:16.267240] [INFO ] [REMOTE ] FGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:16.323709] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T18:45:17.051369] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-03. [2025-10-20T18:45:17.254032] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:45:17.851399] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-07. [2025-10-20T18:45:18.178134] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:45:18.651412] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-06. [2025-10-20T18:45:18.913496] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T18:45:19.451452] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-01. [2025-10-20T18:45:20.095843] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:45:20.122631] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T18:45:20.251396] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-07. [2025-10-20T18:45:21.051457] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-04. [2025-10-20T18:45:21.588893] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T18:45:21.851423] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-02. [2025-10-20T18:45:22.333711] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T18:45:22.355869] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T18:45:22.651399] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-03. [2025-10-20T18:45:23.246427] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T18:45:23.451429] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-08. [2025-10-20T18:45:24.251422] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-05. [2025-10-20T18:45:24.927377] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:45:25.051420] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-09. [2025-10-20T18:45:25.421199] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:45:25.851377] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-08. [2025-10-20T18:45:26.246591] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:45:26.651361] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-04. [2025-10-20T18:45:26.803723] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T18:45:27.252441] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:45:27.451383] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-06. [2025-10-20T18:45:28.251355] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-09. [2025-10-20T18:45:28.395489] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:45:29.051403] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-05. [2025-10-20T18:45:29.512161] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T18:45:29.542175] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:45:29.851438] [INFO ] [REMOTE ] FFC: Fetching primary data for 2025-10. [2025-10-20T18:45:30.651382] [INFO ] [REMOTE ] FFWM: Fetching primary data for 2025-10. [2025-10-20T18:45:30.780993] [INFO ] [REMOTE ] FFC: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T18:45:30.783510] [INFO ] [REMOTE ] FFC: Full fetch complete (24406 records). [2025-10-20T18:45:30.783582] [DEBUG] [VALIDATOR ] FFC: Validating primary data. [2025-10-20T18:45:31.153279] [DEBUG] [VALIDATOR ] FFC: All expected trading days are present. [2025-10-20T18:45:31.153354] [DEBUG] [VALIDATOR ] FFC: Primary data validation passed. [2025-10-20T18:45:31.153858] [INFO ] [MAIN ] FFC: Fetched primary data is valid. [2025-10-20T18:45:31.153918] [DEBUG] [LOCAL ] FFC: Saving primary data. [2025-10-20T18:45:31.154521] [DEBUG] [LOCAL ] FFC: Trimmed 24406 records to 16622 (keeping 200 days of data) [2025-10-20T18:45:31.251993] [DEBUG] [LOCAL ] FFC: Primary data saved (16622 records). [2025-10-20T18:45:31.252136] [INFO ] [MAIN ] FFC: Full history saved. [2025-10-20T18:45:31.252212] [INFO ] [MAIN ] Update Progress: 823/2367 symbols completed. FFC took 38.2s. Estimated remaining: 3h 17m 50s. Rate: 7.8 symbols/min [2025-10-20T18:45:31.419849] [INFO ] [MAIN ] FGSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:31.420018] [DEBUG] [LOCAL ] FGSN: No existing data file found in primary storage. [2025-10-20T18:45:31.420062] [WARN ] [MAIN ] FGSN: No existing data found, performing full refresh instead. [2025-10-20T18:45:31.420172] [INFO ] [MAIN ] FGSN: Wiping existing primary data before full refresh. [2025-10-20T18:45:31.420219] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-10-20T18:45:31.420258] [DEBUG] [LOCAL ] FGSN: No primary data file found to delete. [2025-10-20T18:45:31.420290] [INFO ] [MAIN ] FGSN: Starting full primary refresh. [2025-10-20T18:45:31.456553] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-07. [2025-10-20T18:45:31.781099] [INFO ] [REMOTE ] FGSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:32.226021] [INFO ] [REMOTE ] FFWM: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T18:45:32.231572] [INFO ] [REMOTE ] FFWM: Full fetch complete (51973 records). [2025-10-20T18:45:32.231728] [DEBUG] [VALIDATOR ] FFWM: Validating primary data. [2025-10-20T18:45:32.263145] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-01. [2025-10-20T18:45:32.616534] [DEBUG] [VALIDATOR ] FFWM: All expected trading days are present. [2025-10-20T18:45:32.616771] [DEBUG] [VALIDATOR ] FFWM: Primary data validation passed. [2025-10-20T18:45:32.617556] [INFO ] [MAIN ] FFWM: Fetched primary data is valid. [2025-10-20T18:45:32.617636] [DEBUG] [LOCAL ] FFWM: Saving primary data. [2025-10-20T18:45:32.618719] [DEBUG] [LOCAL ] FFWM: Trimmed 51973 records to 35833 (keeping 200 days of data) [2025-10-20T18:45:32.736222] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:45:32.829452] [DEBUG] [LOCAL ] FFWM: Primary data saved (35833 records). [2025-10-20T18:45:32.829587] [INFO ] [MAIN ] FFWM: Full history saved. [2025-10-20T18:45:32.829688] [INFO ] [MAIN ] Update Progress: 824/2367 symbols completed. FFWM took 28.7s. Estimated remaining: 3h 17m 30s. Rate: 7.8 symbols/min [2025-10-20T18:45:33.000539] [INFO ] [MAIN ] FHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:33.038704] [DEBUG] [LOCAL ] FHI: Loaded 40432 existing records from primary storage. [2025-10-20T18:45:33.039010] [INFO ] [MAIN ] FHI: Loaded 40432 existing records. [2025-10-20T18:45:33.060685] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-06. [2025-10-20T18:45:33.229822] [WARN ] [MAIN ] FHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:33.230808] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T18:45:33.233378] [INFO ] [MAIN ] FHI: Wiping existing primary data before full refresh. [2025-10-20T18:45:33.234098] [INFO ] [LOCAL ] FHI: Starting deletion of primary data. [2025-10-20T18:45:33.234419] [DEBUG] [LOCAL ] FHI: Deleted primary data file. [2025-10-20T18:45:33.234477] [INFO ] [MAIN ] FHI: Starting full primary refresh. [2025-10-20T18:45:33.578838] [INFO ] [REMOTE ] FHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:33.851366] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-02. [2025-10-20T18:45:34.003631] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T18:45:34.378217] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:45:34.651389] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-03. [2025-10-20T18:45:35.336892] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T18:45:35.451360] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-04. [2025-10-20T18:45:36.201739] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T18:45:36.251342] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-05. [2025-10-20T18:45:36.757879] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T18:45:37.051440] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-06. [2025-10-20T18:45:37.654830] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T18:45:37.851412] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-07. [2025-10-20T18:45:38.331453] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:45:38.651407] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-01. [2025-10-20T18:45:39.451415] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-07. [2025-10-20T18:45:40.019603] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:45:40.251358] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-08. [2025-10-20T18:45:40.298885] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T18:45:40.857185] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T18:45:41.051373] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-09. [2025-10-20T18:45:41.623845] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T18:45:41.851402] [INFO ] [REMOTE ] FGSN: Fetching primary data for 2025-10. [2025-10-20T18:45:42.496437] [INFO ] [REMOTE ] FGSN: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T18:45:42.497545] [INFO ] [REMOTE ] FGSN: Full fetch complete (8036 records). [2025-10-20T18:45:42.497611] [DEBUG] [VALIDATOR ] FGSN: Validating primary data. [2025-10-20T18:45:42.652006] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-08. [2025-10-20T18:45:42.677824] [WARN ] [VALIDATOR ] FGSN: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T18:45:42.679279] [FATAL] [MAIN ] FGSN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:45:42.679354] [WARN ] [MAIN ] FGSN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:45:42.679417] [INFO ] [LOCAL ] FGSN: Starting deletion of primary data. [2025-10-20T18:45:42.679473] [DEBUG] [LOCAL ] FGSN: No primary data file found to delete. [2025-10-20T18:45:42.679508] [INFO ] [LOCAL ] FGSN: Starting deletion of derivative data. [2025-10-20T18:45:42.679547] [DEBUG] [LOCAL ] FGSN: No derivative data file found to delete. [2025-10-20T18:45:42.679604] [INFO ] [MAIN ] Update Progress: 825/2367 symbols completed. FGSN took 11.4s. Estimated remaining: 3h 17m 27s. Rate: 7.8 symbols/min [2025-10-20T18:45:42.850419] [INFO ] [MAIN ] FHN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:42.900914] [DEBUG] [LOCAL ] FHN: Loaded 54172 existing records from primary storage. [2025-10-20T18:45:42.901280] [INFO ] [MAIN ] FHN: Loaded 54172 existing records. [2025-10-20T18:45:43.078335] [WARN ] [MAIN ] FHN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:43.078550] [INFO ] [MAIN ] FHN: Wiping existing primary data before full refresh. [2025-10-20T18:45:43.078595] [INFO ] [LOCAL ] FHN: Starting deletion of primary data. [2025-10-20T18:45:43.078806] [DEBUG] [LOCAL ] FHN: Deleted primary data file. [2025-10-20T18:45:43.078861] [INFO ] [MAIN ] FHN: Starting full primary refresh. [2025-10-20T18:45:43.456566] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-02. [2025-10-20T18:45:43.462243] [INFO ] [REMOTE ] FHN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:43.860493] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:45:44.251384] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-08. [2025-10-20T18:45:44.475718] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:45:44.712143] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:45:45.051390] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-03. [2025-10-20T18:45:45.851386] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-09. [2025-10-20T18:45:46.512469] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:45:46.651353] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-04. [2025-10-20T18:45:46.865526] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:45:47.451394] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-01. [2025-10-20T18:45:48.251356] [INFO ] [REMOTE ] FG: Fetching primary data for 2025-10. [2025-10-20T18:45:48.308633] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:45:49.051365] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-05. [2025-10-20T18:45:49.238600] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T18:45:49.449904] [INFO ] [REMOTE ] FG: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T18:45:49.453450] [INFO ] [REMOTE ] FG: Full fetch complete (28853 records). [2025-10-20T18:45:49.453539] [DEBUG] [VALIDATOR ] FG: Validating primary data. [2025-10-20T18:45:49.827604] [DEBUG] [VALIDATOR ] FG: All expected trading days are present. [2025-10-20T18:45:49.827709] [DEBUG] [VALIDATOR ] FG: Primary data validation passed. [2025-10-20T18:45:49.828299] [INFO ] [MAIN ] FG: Fetched primary data is valid. [2025-10-20T18:45:49.828339] [DEBUG] [LOCAL ] FG: Saving primary data. [2025-10-20T18:45:49.829086] [DEBUG] [LOCAL ] FG: Trimmed 28853 records to 21737 (keeping 200 days of data) [2025-10-20T18:45:49.879877] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-09. [2025-10-20T18:45:49.959818] [DEBUG] [LOCAL ] FG: Primary data saved (21737 records). [2025-10-20T18:45:49.959946] [INFO ] [MAIN ] FG: Full history saved. [2025-10-20T18:45:49.960023] [INFO ] [MAIN ] Update Progress: 826/2367 symbols completed. FG took 40.8s. Estimated remaining: 3h 17m 18s. Rate: 7.8 symbols/min [2025-10-20T18:45:50.133795] [INFO ] [MAIN ] FI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:50.186369] [DEBUG] [LOCAL ] FI: Loaded 54357 existing records from primary storage. [2025-10-20T18:45:50.186817] [INFO ] [MAIN ] FI: Loaded 54357 existing records. [2025-10-20T18:45:50.381512] [WARN ] [MAIN ] FI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:50.382042] [INFO ] [MAIN ] FI: Wiping existing primary data before full refresh. [2025-10-20T18:45:50.382152] [INFO ] [LOCAL ] FI: Starting deletion of primary data. [2025-10-20T18:45:50.382344] [DEBUG] [LOCAL ] FI: Deleted primary data file. [2025-10-20T18:45:50.382388] [INFO ] [MAIN ] FI: Starting full primary refresh. [2025-10-20T18:45:50.473151] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T18:45:50.658785] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-02. [2025-10-20T18:45:50.775083] [INFO ] [REMOTE ] FI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:50.802702] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:45:51.451373] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-01. [2025-10-20T18:45:52.251351] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-06. [2025-10-20T18:45:52.305833] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:45:52.910059] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:45:53.051328] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-02. [2025-10-20T18:45:53.729967] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:45:53.851553] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-07. [2025-10-20T18:45:54.651360] [INFO ] [REMOTE ] FGN: Fetching primary data for 2025-10. [2025-10-20T18:45:54.846533] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:45:55.344912] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T18:45:55.442675] [INFO ] [REMOTE ] FGN: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:45:55.443553] [INFO ] [REMOTE ] FGN: Full fetch complete (5552 records). [2025-10-20T18:45:55.443617] [DEBUG] [VALIDATOR ] FGN: Validating primary data. [2025-10-20T18:45:55.456507] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-08. [2025-10-20T18:45:55.794821] [DEBUG] [VALIDATOR ] FGN: All expected trading days are present. [2025-10-20T18:45:55.794898] [DEBUG] [VALIDATOR ] FGN: Primary data validation passed. [2025-10-20T18:45:55.795050] [INFO ] [MAIN ] FGN: Fetched primary data is valid. [2025-10-20T18:45:55.795088] [DEBUG] [LOCAL ] FGN: Saving primary data. [2025-10-20T18:45:55.795447] [DEBUG] [LOCAL ] FGN: Trimmed 5552 records to 3636 (keeping 200 days of data) [2025-10-20T18:45:55.817332] [DEBUG] [LOCAL ] FGN: Primary data saved (3636 records). [2025-10-20T18:45:55.817440] [INFO ] [MAIN ] FGN: Full history saved. [2025-10-20T18:45:55.817512] [INFO ] [MAIN ] Update Progress: 827/2367 symbols completed. FGN took 40.5s. Estimated remaining: 3h 17m 7s. Rate: 7.8 symbols/min [2025-10-20T18:45:55.982149] [INFO ] [MAIN ] FICO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:45:56.004424] [DEBUG] [LOCAL ] FICO: Loaded 22042 existing records from primary storage. [2025-10-20T18:45:56.004663] [INFO ] [MAIN ] FICO: Loaded 22042 existing records. [2025-10-20T18:45:56.179706] [WARN ] [MAIN ] FICO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:45:56.179908] [INFO ] [MAIN ] FICO: Wiping existing primary data before full refresh. [2025-10-20T18:45:56.179953] [INFO ] [LOCAL ] FICO: Starting deletion of primary data. [2025-10-20T18:45:56.180111] [DEBUG] [LOCAL ] FICO: Deleted primary data file. [2025-10-20T18:45:56.180147] [INFO ] [MAIN ] FICO: Starting full primary refresh. [2025-10-20T18:45:56.256435] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-03. [2025-10-20T18:45:56.528000] [INFO ] [REMOTE ] FICO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:45:57.022165] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:45:57.051333] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-09. [2025-10-20T18:45:57.851400] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-01. [2025-10-20T18:45:58.249099] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T18:45:58.513831] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T18:45:58.651401] [INFO ] [REMOTE ] FHI: Fetching primary data for 2025-10. [2025-10-20T18:45:58.939445] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T18:45:59.451366] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-02. [2025-10-20T18:46:00.069842] [INFO ] [REMOTE ] FHI: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:46:00.075256] [INFO ] [REMOTE ] FHI: Full fetch complete (57678 records). [2025-10-20T18:46:00.075403] [DEBUG] [VALIDATOR ] FHI: Validating primary data. [2025-10-20T18:46:00.261959] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-03. [2025-10-20T18:46:00.475467] [DEBUG] [VALIDATOR ] FHI: All expected trading days are present. [2025-10-20T18:46:00.475679] [DEBUG] [VALIDATOR ] FHI: Primary data validation passed. [2025-10-20T18:46:00.476433] [INFO ] [MAIN ] FHI: Fetched primary data is valid. [2025-10-20T18:46:00.476477] [DEBUG] [LOCAL ] FHI: Saving primary data. [2025-10-20T18:46:00.477736] [DEBUG] [LOCAL ] FHI: Trimmed 57678 records to 39793 (keeping 200 days of data) [2025-10-20T18:46:00.591372] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:46:00.728400] [DEBUG] [LOCAL ] FHI: Primary data saved (39793 records). [2025-10-20T18:46:00.728540] [INFO ] [MAIN ] FHI: Full history saved. [2025-10-20T18:46:00.728621] [INFO ] [MAIN ] Update Progress: 828/2367 symbols completed. FHI took 27.9s. Estimated remaining: 3h 16m 54s. Rate: 7.8 symbols/min [2025-10-20T18:46:00.894832] [INFO ] [MAIN ] FIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:00.894977] [DEBUG] [LOCAL ] FIG: No existing data file found in primary storage. [2025-10-20T18:46:00.895014] [WARN ] [MAIN ] FIG: No existing data found, performing full refresh instead. [2025-10-20T18:46:00.895125] [INFO ] [MAIN ] FIG: Wiping existing primary data before full refresh. [2025-10-20T18:46:00.895171] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-10-20T18:46:00.895248] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-10-20T18:46:00.895294] [INFO ] [MAIN ] FIG: Starting full primary refresh. [2025-10-20T18:46:01.068991] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-04. [2025-10-20T18:46:01.254933] [INFO ] [REMOTE ] FIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:01.851360] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-01. [2025-10-20T18:46:02.106779] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-01 (0.26 seconds). [2025-10-20T18:46:02.541796] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-03 (2.28 seconds). [2025-10-20T18:46:02.651379] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-03. [2025-10-20T18:46:03.080486] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T18:46:03.451419] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-02. [2025-10-20T18:46:03.771902] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:46:03.787907] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:46:04.251421] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-04. [2025-10-20T18:46:05.051403] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-05. [2025-10-20T18:46:05.422809] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T18:46:05.851389] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-03. [2025-10-20T18:46:06.098343] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-10-20T18:46:06.651351] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-05. [2025-10-20T18:46:06.790560] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:46:07.451417] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-04. [2025-10-20T18:46:07.920296] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:46:08.251399] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-04. [2025-10-20T18:46:08.560539] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-10-20T18:46:09.051435] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-06. [2025-10-20T18:46:09.485488] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:46:09.851403] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-06. [2025-10-20T18:46:10.218685] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T18:46:10.651416] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-05. [2025-10-20T18:46:11.353227] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T18:46:11.451410] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-07. [2025-10-20T18:46:12.251376] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-07. [2025-10-20T18:46:12.601063] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T18:46:12.863772] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T18:46:13.051366] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-08. [2025-10-20T18:46:13.699890] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:46:13.851396] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-05. [2025-10-20T18:46:14.187342] [INFO ] [REMOTE ] FIG: Incremental fetch complete for 2025-05 (0.34 seconds). [2025-10-20T18:46:14.651368] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-06. [2025-10-20T18:46:14.818222] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:46:15.451391] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-09. [2025-10-20T18:46:16.251372] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-08. [2025-10-20T18:46:16.573232] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T18:46:17.051429] [INFO ] [REMOTE ] FIG: Fetching primary data for 2025-06. [2025-10-20T18:46:17.192068] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T18:46:17.254512] [DEBUG] [REMOTE ] FIG: Error fetching primary data for 2025-06: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:46:17.254908] [WARN ] [REMOTE ] FIG: No data returned for month 2025-06 [2025-10-20T18:46:17.255004] [FATAL] [REMOTE ] FIG: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:46:17.255083] [FATAL] [MAIN ] FIG: Failed to fetch primary data. [2025-10-20T18:46:17.255163] [WARN ] [MAIN ] FIG: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:46:17.255239] [INFO ] [LOCAL ] FIG: Starting deletion of primary data. [2025-10-20T18:46:17.255305] [DEBUG] [LOCAL ] FIG: No primary data file found to delete. [2025-10-20T18:46:17.255341] [INFO ] [LOCAL ] FIG: Starting deletion of derivative data. [2025-10-20T18:46:17.255384] [DEBUG] [LOCAL ] FIG: No derivative data file found to delete. [2025-10-20T18:46:17.255431] [INFO ] [MAIN ] Update Progress: 829/2367 symbols completed. FIG took 16.5s. Estimated remaining: 3h 17m 3s. Rate: 7.8 symbols/min [2025-10-20T18:46:17.437605] [INFO ] [MAIN ] FIGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:17.461253] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T18:46:17.509199] [DEBUG] [LOCAL ] FIGS: Loaded 49320 existing records from primary storage. [2025-10-20T18:46:17.509635] [INFO ] [MAIN ] FIGS: Loaded 49320 existing records. [2025-10-20T18:46:17.690382] [WARN ] [MAIN ] FIGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:17.690597] [INFO ] [MAIN ] FIGS: Wiping existing primary data before full refresh. [2025-10-20T18:46:17.690666] [INFO ] [LOCAL ] FIGS: Starting deletion of primary data. [2025-10-20T18:46:17.691096] [DEBUG] [LOCAL ] FIGS: Deleted primary data file. [2025-10-20T18:46:17.691153] [INFO ] [MAIN ] FIGS: Starting full primary refresh. [2025-10-20T18:46:17.852045] [INFO ] [REMOTE ] FHN: Fetching primary data for 2025-10. [2025-10-20T18:46:18.052955] [INFO ] [REMOTE ] FIGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:18.651377] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-07. [2025-10-20T18:46:19.451416] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-01. [2025-10-20T18:46:19.713165] [INFO ] [REMOTE ] FHN: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:46:19.721214] [INFO ] [REMOTE ] FHN: Full fetch complete (77474 records). [2025-10-20T18:46:19.721364] [DEBUG] [VALIDATOR ] FHN: Validating primary data. [2025-10-20T18:46:20.111593] [DEBUG] [VALIDATOR ] FHN: All expected trading days are present. [2025-10-20T18:46:20.111701] [DEBUG] [VALIDATOR ] FHN: Primary data validation passed. [2025-10-20T18:46:20.112603] [INFO ] [MAIN ] FHN: Fetched primary data is valid. [2025-10-20T18:46:20.112665] [DEBUG] [LOCAL ] FHN: Saving primary data. [2025-10-20T18:46:20.114370] [DEBUG] [LOCAL ] FHN: Trimmed 77474 records to 53394 (keeping 200 days of data) [2025-10-20T18:46:20.289461] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-09. [2025-10-20T18:46:20.445593] [DEBUG] [LOCAL ] FHN: Primary data saved (53394 records). [2025-10-20T18:46:20.445738] [INFO ] [MAIN ] FHN: Full history saved. [2025-10-20T18:46:20.445821] [INFO ] [MAIN ] Update Progress: 830/2367 symbols completed. FHN took 37.8s. Estimated remaining: 3h 16m 47s. Rate: 7.8 symbols/min [2025-10-20T18:46:20.619814] [INFO ] [MAIN ] FIHL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:20.674254] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T18:46:20.700341] [DEBUG] [LOCAL ] FIHL: Loaded 39982 existing records from primary storage. [2025-10-20T18:46:20.700675] [INFO ] [MAIN ] FIHL: Loaded 39982 existing records. [2025-10-20T18:46:20.880004] [WARN ] [MAIN ] FIHL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:20.880245] [INFO ] [MAIN ] FIHL: Wiping existing primary data before full refresh. [2025-10-20T18:46:20.880299] [INFO ] [LOCAL ] FIHL: Starting deletion of primary data. [2025-10-20T18:46:20.880483] [DEBUG] [LOCAL ] FIHL: Deleted primary data file. [2025-10-20T18:46:20.880537] [INFO ] [MAIN ] FIHL: Starting full primary refresh. [2025-10-20T18:46:21.063802] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-08. [2025-10-20T18:46:21.291409] [INFO ] [REMOTE ] FIHL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:21.285430] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:46:21.314515] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:46:21.851403] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-01. [2025-10-20T18:46:22.651404] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-02. [2025-10-20T18:46:22.996676] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T18:46:23.331074] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:46:23.451373] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-09. [2025-10-20T18:46:24.139574] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T18:46:24.251362] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-03. [2025-10-20T18:46:25.051424] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-02. [2025-10-20T18:46:25.554892] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-09 (2.10 seconds). [2025-10-20T18:46:25.627830] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T18:46:25.851417] [INFO ] [REMOTE ] FICO: Fetching primary data for 2025-10. [2025-10-20T18:46:26.344346] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T18:46:26.651686] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-04. [2025-10-20T18:46:27.308863] [INFO ] [REMOTE ] FICO: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:46:27.313416] [INFO ] [REMOTE ] FICO: Full fetch complete (29935 records). [2025-10-20T18:46:27.313567] [DEBUG] [VALIDATOR ] FICO: Validating primary data. [2025-10-20T18:46:27.458168] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-03. [2025-10-20T18:46:27.702188] [DEBUG] [VALIDATOR ] FICO: All expected trading days are present. [2025-10-20T18:46:27.702265] [DEBUG] [VALIDATOR ] FICO: Primary data validation passed. [2025-10-20T18:46:27.702832] [INFO ] [MAIN ] FICO: Fetched primary data is valid. [2025-10-20T18:46:27.702893] [DEBUG] [LOCAL ] FICO: Saving primary data. [2025-10-20T18:46:27.703780] [DEBUG] [LOCAL ] FICO: Trimmed 29935 records to 21862 (keeping 200 days of data) [2025-10-20T18:46:27.836777] [DEBUG] [LOCAL ] FICO: Primary data saved (21862 records). [2025-10-20T18:46:27.836901] [INFO ] [MAIN ] FICO: Full history saved. [2025-10-20T18:46:27.836972] [INFO ] [MAIN ] Update Progress: 831/2367 symbols completed. FICO took 32.0s. Estimated remaining: 3h 16m 39s. Rate: 7.8 symbols/min [2025-10-20T18:46:28.002596] [INFO ] [MAIN ] FINS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:28.012145] [DEBUG] [LOCAL ] FINS: Loaded 8404 existing records from primary storage. [2025-10-20T18:46:28.012347] [INFO ] [MAIN ] FINS: Loaded 8404 existing records. [2025-10-20T18:46:28.190276] [WARN ] [MAIN ] FINS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:28.190453] [INFO ] [MAIN ] FINS: Wiping existing primary data before full refresh. [2025-10-20T18:46:28.190498] [INFO ] [LOCAL ] FINS: Starting deletion of primary data. [2025-10-20T18:46:28.190675] [DEBUG] [LOCAL ] FINS: Deleted primary data file. [2025-10-20T18:46:28.190730] [INFO ] [MAIN ] FINS: Starting full primary refresh. [2025-10-20T18:46:28.252973] [INFO ] [REMOTE ] FI: Fetching primary data for 2025-10. [2025-10-20T18:46:28.546771] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T18:46:28.586942] [INFO ] [REMOTE ] FINS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:29.051379] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-01. [2025-10-20T18:46:29.158318] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:46:29.192490] [INFO ] [REMOTE ] FI: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T18:46:29.200710] [INFO ] [REMOTE ] FI: Full fetch complete (77477 records). [2025-10-20T18:46:29.200900] [DEBUG] [VALIDATOR ] FI: Validating primary data. [2025-10-20T18:46:29.586485] [DEBUG] [VALIDATOR ] FI: All expected trading days are present. [2025-10-20T18:46:29.586579] [DEBUG] [VALIDATOR ] FI: Primary data validation passed. [2025-10-20T18:46:29.587516] [INFO ] [MAIN ] FI: Fetched primary data is valid. [2025-10-20T18:46:29.587588] [DEBUG] [LOCAL ] FI: Saving primary data. [2025-10-20T18:46:29.589091] [DEBUG] [LOCAL ] FI: Trimmed 77477 records to 53585 (keeping 200 days of data) [2025-10-20T18:46:29.804712] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T18:46:29.877750] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-05. [2025-10-20T18:46:29.931264] [DEBUG] [LOCAL ] FI: Primary data saved (53585 records). [2025-10-20T18:46:29.931432] [INFO ] [MAIN ] FI: Full history saved. [2025-10-20T18:46:29.931990] [INFO ] [MAIN ] Update Progress: 832/2367 symbols completed. FI took 40.0s. Estimated remaining: 3h 16m 21s. Rate: 7.8 symbols/min [2025-10-20T18:46:30.113278] [INFO ] [MAIN ] FINV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:30.165152] [DEBUG] [LOCAL ] FINV: Loaded 45943 existing records from primary storage. [2025-10-20T18:46:30.165469] [INFO ] [MAIN ] FINV: Loaded 45943 existing records. [2025-10-20T18:46:30.340460] [WARN ] [MAIN ] FINV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:30.340703] [INFO ] [MAIN ] FINV: Wiping existing primary data before full refresh. [2025-10-20T18:46:30.340762] [INFO ] [LOCAL ] FINV: Starting deletion of primary data. [2025-10-20T18:46:30.340955] [DEBUG] [LOCAL ] FINV: Deleted primary data file. [2025-10-20T18:46:30.341009] [INFO ] [MAIN ] FINV: Starting full primary refresh. [2025-10-20T18:46:30.652734] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-04. [2025-10-20T18:46:30.693168] [INFO ] [REMOTE ] FINV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:31.382508] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:46:31.451412] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-06. [2025-10-20T18:46:31.836321] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:46:32.251413] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-05. [2025-10-20T18:46:33.051369] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-01. [2025-10-20T18:46:33.092418] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:46:33.521337] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:46:33.851378] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-07. [2025-10-20T18:46:34.462027] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T18:46:34.651363] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-02. [2025-10-20T18:46:34.962118] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T18:46:35.451403] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-08. [2025-10-20T18:46:35.998672] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:46:36.251387] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-02. [2025-10-20T18:46:36.934865] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T18:46:37.051366] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-03. [2025-10-20T18:46:37.119242] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:46:37.851379] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-06. [2025-10-20T18:46:38.645634] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:46:38.654708] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-03. [2025-10-20T18:46:39.259052] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:46:39.272331] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:46:39.451358] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-07. [2025-10-20T18:46:40.251378] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-04. [2025-10-20T18:46:40.694155] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T18:46:41.051399] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-09. [2025-10-20T18:46:41.066797] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:46:41.851362] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-05. [2025-10-20T18:46:42.559483] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:46:42.643478] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:46:42.651319] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-06. [2025-10-20T18:46:43.410669] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T18:46:43.451408] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-08. [2025-10-20T18:46:44.251437] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-04. [2025-10-20T18:46:44.921184] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:46:45.051346] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-09. [2025-10-20T18:46:45.824182] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T18:46:45.851365] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-05. [2025-10-20T18:46:46.264734] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T18:46:46.651423] [INFO ] [REMOTE ] FIHL: Fetching primary data for 2025-10. [2025-10-20T18:46:47.338172] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T18:46:47.451346] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-06. [2025-10-20T18:46:47.911300] [INFO ] [REMOTE ] FIHL: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:46:47.917972] [INFO ] [REMOTE ] FIHL: Full fetch complete (56277 records). [2025-10-20T18:46:47.918119] [DEBUG] [VALIDATOR ] FIHL: Validating primary data. [2025-10-20T18:46:48.260148] [INFO ] [REMOTE ] FIGS: Fetching primary data for 2025-10. [2025-10-20T18:46:48.379741] [DEBUG] [VALIDATOR ] FIHL: All expected trading days are present. [2025-10-20T18:46:48.379871] [DEBUG] [VALIDATOR ] FIHL: Primary data validation passed. [2025-10-20T18:46:48.381234] [INFO ] [MAIN ] FIHL: Fetched primary data is valid. [2025-10-20T18:46:48.381304] [DEBUG] [LOCAL ] FIHL: Saving primary data. [2025-10-20T18:46:48.383235] [DEBUG] [LOCAL ] FIHL: Trimmed 56277 records to 39199 (keeping 200 days of data) [2025-10-20T18:46:48.642279] [DEBUG] [LOCAL ] FIHL: Primary data saved (39199 records). [2025-10-20T18:46:48.642698] [INFO ] [MAIN ] FIHL: Full history saved. [2025-10-20T18:46:48.642823] [INFO ] [MAIN ] Update Progress: 833/2367 symbols completed. FIHL took 28.2s. Estimated remaining: 3h 16m 34s. Rate: 7.8 symbols/min [2025-10-20T18:46:48.839363] [INFO ] [MAIN ] FIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:48.897215] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:46:48.928602] [DEBUG] [LOCAL ] FIS: Loaded 54114 existing records from primary storage. [2025-10-20T18:46:48.929151] [INFO ] [MAIN ] FIS: Loaded 54114 existing records. [2025-10-20T18:46:49.052947] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-07. [2025-10-20T18:46:49.140797] [WARN ] [MAIN ] FIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:49.141113] [INFO ] [MAIN ] FIS: Wiping existing primary data before full refresh. [2025-10-20T18:46:49.141164] [INFO ] [LOCAL ] FIS: Starting deletion of primary data. [2025-10-20T18:46:49.141829] [DEBUG] [LOCAL ] FIS: Deleted primary data file. [2025-10-20T18:46:49.141889] [INFO ] [MAIN ] FIS: Starting full primary refresh. [2025-10-20T18:46:49.508612] [INFO ] [REMOTE ] FIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:49.851398] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-07. [2025-10-20T18:46:50.042808] [INFO ] [REMOTE ] FIGS: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T18:46:50.051136] [INFO ] [REMOTE ] FIGS: Full fetch complete (70868 records). [2025-10-20T18:46:50.051425] [DEBUG] [VALIDATOR ] FIGS: Validating primary data. [2025-10-20T18:46:50.433354] [DEBUG] [VALIDATOR ] FIGS: All expected trading days are present. [2025-10-20T18:46:50.433438] [DEBUG] [VALIDATOR ] FIGS: Primary data validation passed. [2025-10-20T18:46:50.434351] [INFO ] [MAIN ] FIGS: Fetched primary data is valid. [2025-10-20T18:46:50.434413] [DEBUG] [LOCAL ] FIGS: Saving primary data. [2025-10-20T18:46:50.435770] [DEBUG] [LOCAL ] FIGS: Trimmed 70868 records to 48620 (keeping 200 days of data) [2025-10-20T18:46:50.652076] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-01. [2025-10-20T18:46:50.656057] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:46:50.719421] [DEBUG] [LOCAL ] FIGS: Primary data saved (48620 records). [2025-10-20T18:46:50.719745] [INFO ] [MAIN ] FIGS: Full history saved. [2025-10-20T18:46:50.720611] [INFO ] [MAIN ] Update Progress: 834/2367 symbols completed. FIGS took 33.5s. Estimated remaining: 3h 16m 16s. Rate: 7.8 symbols/min [2025-10-20T18:46:50.891977] [INFO ] [MAIN ] FIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:46:50.923089] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:46:50.959419] [DEBUG] [LOCAL ] FIX: Loaded 29256 existing records from primary storage. [2025-10-20T18:46:50.959730] [INFO ] [MAIN ] FIX: Loaded 29256 existing records. [2025-10-20T18:46:51.132730] [WARN ] [MAIN ] FIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:46:51.132978] [INFO ] [MAIN ] FIX: Wiping existing primary data before full refresh. [2025-10-20T18:46:51.133026] [INFO ] [LOCAL ] FIX: Starting deletion of primary data. [2025-10-20T18:46:51.133216] [DEBUG] [LOCAL ] FIX: Deleted primary data file. [2025-10-20T18:46:51.133253] [INFO ] [MAIN ] FIX: Starting full primary refresh. [2025-10-20T18:46:51.456536] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-08. [2025-10-20T18:46:51.486676] [INFO ] [REMOTE ] FIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:46:52.148964] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:46:52.217772] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:46:52.251341] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-02. [2025-10-20T18:46:53.051369] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-09. [2025-10-20T18:46:53.748919] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T18:46:53.851346] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-01. [2025-10-20T18:46:54.036573] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:46:54.651366] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-03. [2025-10-20T18:46:55.281810] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:46:55.451376] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-08. [2025-10-20T18:46:56.083958] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T18:46:56.251364] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-04. [2025-10-20T18:46:56.995622] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T18:46:57.051358] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-02. [2025-10-20T18:46:57.752913] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T18:46:57.851373] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-05. [2025-10-20T18:46:58.636370] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T18:46:58.651329] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-03. [2025-10-20T18:46:59.181341] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:46:59.451370] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-06. [2025-10-20T18:47:00.196531] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T18:47:00.251332] [INFO ] [REMOTE ] FINS: Fetching primary data for 2025-10. [2025-10-20T18:47:01.028274] [INFO ] [REMOTE ] FINS: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T18:47:01.035017] [INFO ] [REMOTE ] FINS: Full fetch complete (11593 records). [2025-10-20T18:47:01.035205] [DEBUG] [VALIDATOR ] FINS: Validating primary data. [2025-10-20T18:47:01.056497] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-04. [2025-10-20T18:47:01.551602] [DEBUG] [VALIDATOR ] FINS: All expected trading days are present. [2025-10-20T18:47:01.551788] [DEBUG] [VALIDATOR ] FINS: Primary data validation passed. [2025-10-20T18:47:01.552478] [INFO ] [MAIN ] FINS: Fetched primary data is valid. [2025-10-20T18:47:01.552611] [DEBUG] [LOCAL ] FINS: Saving primary data. [2025-10-20T18:47:01.553910] [DEBUG] [LOCAL ] FINS: Trimmed 11593 records to 8283 (keeping 200 days of data) [2025-10-20T18:47:01.657397] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T18:47:01.697221] [DEBUG] [LOCAL ] FINS: Primary data saved (8283 records). [2025-10-20T18:47:01.697413] [INFO ] [MAIN ] FINS: Full history saved. [2025-10-20T18:47:01.697527] [INFO ] [MAIN ] Update Progress: 835/2367 symbols completed. FINS took 33.9s. Estimated remaining: 3h 16m 14s. Rate: 7.8 symbols/min [2025-10-20T18:47:01.852310] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-07. [2025-10-20T18:47:01.943919] [INFO ] [MAIN ] FIXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:01.944177] [DEBUG] [LOCAL ] FIXT: No existing data file found in primary storage. [2025-10-20T18:47:01.944226] [WARN ] [MAIN ] FIXT: No existing data found, performing full refresh instead. [2025-10-20T18:47:01.944431] [INFO ] [MAIN ] FIXT: Wiping existing primary data before full refresh. [2025-10-20T18:47:01.944487] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-10-20T18:47:01.944532] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-10-20T18:47:01.944564] [INFO ] [MAIN ] FIXT: Starting full primary refresh. [2025-10-20T18:47:02.383149] [INFO ] [REMOTE ] FIXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:02.651362] [INFO ] [REMOTE ] FIXT: Fetching primary data for 2025-01. [2025-10-20T18:47:02.715941] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:47:02.847408] [DEBUG] [REMOTE ] FIXT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T18:47:02.847801] [WARN ] [REMOTE ] FIXT: No data returned for month 2025-01 [2025-10-20T18:47:02.847903] [FATAL] [REMOTE ] FIXT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:47:02.847954] [FATAL] [MAIN ] FIXT: Failed to fetch primary data. [2025-10-20T18:47:02.848014] [WARN ] [MAIN ] FIXT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:47:02.848085] [INFO ] [LOCAL ] FIXT: Starting deletion of primary data. [2025-10-20T18:47:02.848146] [DEBUG] [LOCAL ] FIXT: No primary data file found to delete. [2025-10-20T18:47:02.848181] [INFO ] [LOCAL ] FIXT: Starting deletion of derivative data. [2025-10-20T18:47:02.848219] [DEBUG] [LOCAL ] FIXT: No derivative data file found to delete. [2025-10-20T18:47:02.848264] [INFO ] [MAIN ] Update Progress: 836/2367 symbols completed. FIXT took 1.2s. Estimated remaining: 3h 15m 54s. Rate: 7.8 symbols/min [2025-10-20T18:47:03.015006] [INFO ] [MAIN ] FLC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:03.022939] [DEBUG] [LOCAL ] FLC: Loaded 5454 existing records from primary storage. [2025-10-20T18:47:03.023148] [INFO ] [MAIN ] FLC: Loaded 5454 existing records. [2025-10-20T18:47:03.210513] [WARN ] [MAIN ] FLC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:03.210750] [INFO ] [MAIN ] FLC: Wiping existing primary data before full refresh. [2025-10-20T18:47:03.210802] [INFO ] [LOCAL ] FLC: Starting deletion of primary data. [2025-10-20T18:47:03.211211] [DEBUG] [LOCAL ] FLC: Deleted primary data file. [2025-10-20T18:47:03.211267] [INFO ] [MAIN ] FLC: Starting full primary refresh. [2025-10-20T18:47:03.461096] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-09. [2025-10-20T18:47:03.578137] [INFO ] [REMOTE ] FLC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:03.734543] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:47:04.251443] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-05. [2025-10-20T18:47:04.974548] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T18:47:05.051337] [INFO ] [REMOTE ] FINV: Fetching primary data for 2025-10. [2025-10-20T18:47:05.564039] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T18:47:05.851433] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-06. [2025-10-20T18:47:06.651406] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-08. [2025-10-20T18:47:06.761715] [INFO ] [REMOTE ] FINV: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:47:06.768773] [INFO ] [REMOTE ] FINV: Full fetch complete (64815 records). [2025-10-20T18:47:06.768885] [DEBUG] [VALIDATOR ] FINV: Validating primary data. [2025-10-20T18:47:07.165287] [DEBUG] [VALIDATOR ] FINV: All expected trading days are present. [2025-10-20T18:47:07.165455] [DEBUG] [VALIDATOR ] FINV: Primary data validation passed. [2025-10-20T18:47:07.166343] [INFO ] [MAIN ] FINV: Fetched primary data is valid. [2025-10-20T18:47:07.166424] [DEBUG] [LOCAL ] FINV: Saving primary data. [2025-10-20T18:47:07.182797] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T18:47:07.188988] [DEBUG] [LOCAL ] FINV: Trimmed 64815 records to 45271 (keeping 200 days of data) [2025-10-20T18:47:07.435754] [DEBUG] [LOCAL ] FINV: Primary data saved (45271 records). [2025-10-20T18:47:07.435879] [INFO ] [MAIN ] FINV: Full history saved. [2025-10-20T18:47:07.436422] [INFO ] [MAIN ] Update Progress: 837/2367 symbols completed. FINV took 37.5s. Estimated remaining: 3h 15m 41s. Rate: 7.8 symbols/min [2025-10-20T18:47:07.451376] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-07. [2025-10-20T18:47:08.251439] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-01. [2025-10-20T18:47:08.351030] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T18:47:08.811942] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:47:08.870062] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T18:47:09.051427] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-09. [2025-10-20T18:47:09.851391] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-02. [2025-10-20T18:47:10.378912] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:47:10.663424] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-03. [2025-10-20T18:47:10.693987] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T18:47:11.325373] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T18:47:11.451415] [INFO ] [REMOTE ] FIS: Fetching primary data for 2025-10. [2025-10-20T18:47:12.251458] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-04. [2025-10-20T18:47:12.304088] [INFO ] [REMOTE ] FIS: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T18:47:12.312348] [INFO ] [REMOTE ] FIS: Full fetch complete (77386 records). [2025-10-20T18:47:12.312517] [DEBUG] [VALIDATOR ] FIS: Validating primary data. [2025-10-20T18:47:12.716073] [DEBUG] [VALIDATOR ] FIS: All expected trading days are present. [2025-10-20T18:47:12.716160] [DEBUG] [VALIDATOR ] FIS: Primary data validation passed. [2025-10-20T18:47:12.717055] [INFO ] [MAIN ] FIS: Fetched primary data is valid. [2025-10-20T18:47:12.717117] [DEBUG] [LOCAL ] FIS: Saving primary data. [2025-10-20T18:47:12.718550] [DEBUG] [LOCAL ] FIS: Trimmed 77386 records to 53327 (keeping 200 days of data) [2025-10-20T18:47:12.967208] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T18:47:13.054423] [DEBUG] [LOCAL ] FIS: Primary data saved (53327 records). [2025-10-20T18:47:13.051811] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-05. [2025-10-20T18:47:13.054571] [INFO ] [MAIN ] FIS: Full history saved. [2025-10-20T18:47:13.054708] [INFO ] [MAIN ] Update Progress: 838/2367 symbols completed. FIS took 24.4s. Estimated remaining: 3h 15m 30s. Rate: 7.8 symbols/min [2025-10-20T18:47:13.664757] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T18:47:13.851408] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-06. [2025-10-20T18:47:14.415680] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T18:47:14.651382] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-07. [2025-10-20T18:47:15.239243] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T18:47:15.451401] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-08. [2025-10-20T18:47:16.251470] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-08. [2025-10-20T18:47:16.735950] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T18:47:16.987639] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:47:17.051362] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-09. [2025-10-20T18:47:17.851459] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-09. [2025-10-20T18:47:18.377417] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:47:18.466147] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T18:47:18.651430] [INFO ] [REMOTE ] FLC: Fetching primary data for 2025-10. [2025-10-20T18:47:19.229998] [INFO ] [REMOTE ] FLC: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T18:47:19.231444] [INFO ] [REMOTE ] FLC: Full fetch complete (7769 records). [2025-10-20T18:47:19.231534] [DEBUG] [VALIDATOR ] FLC: Validating primary data. [2025-10-20T18:47:19.458378] [INFO ] [REMOTE ] FIX: Fetching primary data for 2025-10. [2025-10-20T18:47:19.694455] [DEBUG] [VALIDATOR ] FLC: All expected trading days are present. [2025-10-20T18:47:19.694565] [DEBUG] [VALIDATOR ] FLC: Primary data validation passed. [2025-10-20T18:47:19.695044] [INFO ] [MAIN ] FLC: Fetched primary data is valid. [2025-10-20T18:47:19.695103] [DEBUG] [LOCAL ] FLC: Saving primary data. [2025-10-20T18:47:19.695621] [DEBUG] [LOCAL ] FLC: Trimmed 7769 records to 5417 (keeping 200 days of data) [2025-10-20T18:47:19.738686] [DEBUG] [LOCAL ] FLC: Primary data saved (5417 records). [2025-10-20T18:47:19.738830] [INFO ] [MAIN ] FLC: Full history saved. [2025-10-20T18:47:19.738910] [INFO ] [MAIN ] Update Progress: 839/2367 symbols completed. FLC took 16.9s. Estimated remaining: 3h 15m 20s. Rate: 7.8 symbols/min [2025-10-20T18:47:20.718745] [INFO ] [REMOTE ] FIX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:47:20.725264] [INFO ] [REMOTE ] FIX: Full fetch complete (45598 records). [2025-10-20T18:47:20.725459] [DEBUG] [VALIDATOR ] FIX: Validating primary data. [2025-10-20T18:47:21.128590] [DEBUG] [VALIDATOR ] FIX: All expected trading days are present. [2025-10-20T18:47:21.128690] [DEBUG] [VALIDATOR ] FIX: Primary data validation passed. [2025-10-20T18:47:21.129409] [INFO ] [MAIN ] FIX: Fetched primary data is valid. [2025-10-20T18:47:21.129471] [DEBUG] [LOCAL ] FIX: Saving primary data. [2025-10-20T18:47:21.130414] [DEBUG] [LOCAL ] FIX: Trimmed 45598 records to 28617 (keeping 200 days of data) [2025-10-20T18:47:21.305693] [DEBUG] [LOCAL ] FIX: Primary data saved (28617 records). [2025-10-20T18:47:21.305845] [INFO ] [MAIN ] FIX: Full history saved. [2025-10-20T18:47:21.305924] [INFO ] [MAIN ] Update Progress: 840/2367 symbols completed. FIX took 30.6s. Estimated remaining: 3h 15m 1s. Rate: 7.8 symbols/min [2025-10-20T18:47:21.957187] [INFO ] [MAIN ] FLOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:21.959078] [INFO ] [MAIN ] FLG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.057965] [DEBUG] [LOCAL ] FLOC: No existing data file found in primary storage. [2025-10-20T18:47:22.083295] [WARN ] [MAIN ] FLOC: No existing data found, performing full refresh instead. [2025-10-20T18:47:22.057824] [INFO ] [MAIN ] FLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.087160] [INFO ] [MAIN ] FLNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:22.088008] [INFO ] [MAIN ] FLOC: Wiping existing primary data before full refresh. [2025-10-20T18:47:22.088750] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-10-20T18:47:22.091793] [DEBUG] [LOCAL ] FLOC: No primary data file found to delete. [2025-10-20T18:47:22.091855] [INFO ] [MAIN ] FLOC: Starting full primary refresh. [2025-10-20T18:47:22.395045] [DEBUG] [LOCAL ] FLG: Loaded 54107 existing records from primary storage. [2025-10-20T18:47:22.403288] [DEBUG] [LOCAL ] FLNG: Loaded 34189 existing records from primary storage. [2025-10-20T18:47:22.417585] [INFO ] [MAIN ] FLG: Loaded 54107 existing records. [2025-10-20T18:47:22.439236] [INFO ] [MAIN ] FLNG: Loaded 34189 existing records. [2025-10-20T18:47:22.601902] [DEBUG] [LOCAL ] FLO: Loaded 52831 existing records from primary storage. [2025-10-20T18:47:22.685968] [INFO ] [MAIN ] FLO: Loaded 52831 existing records. [2025-10-20T18:47:22.782323] [INFO ] [REMOTE ] FLOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:22.862617] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-01. [2025-10-20T18:47:22.945181] [WARN ] [MAIN ] FLNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:22.969373] [INFO ] [MAIN ] FLNG: Wiping existing primary data before full refresh. [2025-10-20T18:47:22.997889] [INFO ] [LOCAL ] FLNG: Starting deletion of primary data. [2025-10-20T18:47:23.003330] [DEBUG] [LOCAL ] FLNG: Deleted primary data file. [2025-10-20T18:47:23.027176] [INFO ] [MAIN ] FLNG: Starting full primary refresh. [2025-10-20T18:47:23.047781] [WARN ] [MAIN ] FLG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:23.115084] [INFO ] [MAIN ] FLG: Wiping existing primary data before full refresh. [2025-10-20T18:47:23.130527] [INFO ] [LOCAL ] FLG: Starting deletion of primary data. [2025-10-20T18:47:23.130428] [WARN ] [MAIN ] FLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:23.142865] [DEBUG] [LOCAL ] FLG: Deleted primary data file. [2025-10-20T18:47:23.154757] [INFO ] [MAIN ] FLG: Starting full primary refresh. [2025-10-20T18:47:23.183530] [INFO ] [MAIN ] FLO: Wiping existing primary data before full refresh. [2025-10-20T18:47:23.183694] [INFO ] [LOCAL ] FLO: Starting deletion of primary data. [2025-10-20T18:47:23.271486] [DEBUG] [LOCAL ] FLO: Deleted primary data file. [2025-10-20T18:47:23.297921] [INFO ] [MAIN ] FLO: Starting full primary refresh. [2025-10-20T18:47:24.227123] [INFO ] [REMOTE ] FLNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.268042] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-01. [2025-10-20T18:47:24.298026] [INFO ] [REMOTE ] FLG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.336013] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-01. [2025-10-20T18:47:24.344609] [INFO ] [REMOTE ] FLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:24.336902] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T18:47:24.345618] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-01. [2025-10-20T18:47:24.351286] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-02. [2025-10-20T18:47:25.647119] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T18:47:25.653439] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-03. [2025-10-20T18:47:25.713741] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:47:25.721197] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-02. [2025-10-20T18:47:25.830064] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T18:47:25.851346] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-02. [2025-10-20T18:47:26.022956] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:47:26.651490] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-02. [2025-10-20T18:47:26.770771] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T18:47:27.253419] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:47:27.451391] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-03. [2025-10-20T18:47:27.553008] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T18:47:27.998219] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T18:47:28.251395] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-03. [2025-10-20T18:47:28.901279] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T18:47:29.051395] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-03. [2025-10-20T18:47:29.439086] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T18:47:29.851376] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-04. [2025-10-20T18:47:30.651350] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-04. [2025-10-20T18:47:30.830131] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:47:31.118264] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T18:47:31.451353] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-04. [2025-10-20T18:47:32.025174] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T18:47:32.251386] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-05. [2025-10-20T18:47:33.001405] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T18:47:33.051405] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-04. [2025-10-20T18:47:33.526065] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T18:47:33.851373] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-05. [2025-10-20T18:47:34.651371] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-05. [2025-10-20T18:47:34.690728] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:47:35.173006] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T18:47:35.451399] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-06. [2025-10-20T18:47:36.269816] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-05. [2025-10-20T18:47:36.272730] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T18:47:36.676890] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:47:37.051360] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-06. [2025-10-20T18:47:37.565075] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T18:47:37.851378] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-06. [2025-10-20T18:47:38.651349] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-06. [2025-10-20T18:47:38.866315] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:47:39.038348] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T18:47:39.451371] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-07. [2025-10-20T18:47:40.006289] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T18:47:40.251356] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-07. [2025-10-20T18:47:40.577753] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:47:41.051368] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-07. [2025-10-20T18:47:41.851454] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-08. [2025-10-20T18:47:42.117996] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T18:47:42.167841] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:47:42.656456] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-08. [2025-10-20T18:47:42.668236] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:47:43.451383] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-08. [2025-10-20T18:47:44.251350] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-07. [2025-10-20T18:47:44.459205] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T18:47:44.976579] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T18:47:45.051356] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-09. [2025-10-20T18:47:45.851366] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-09. [2025-10-20T18:47:46.032000] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:47:46.160676] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T18:47:46.605745] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T18:47:46.651333] [INFO ] [REMOTE ] FLOC: Fetching primary data for 2025-10. [2025-10-20T18:47:47.451375] [INFO ] [REMOTE ] FLNG: Fetching primary data for 2025-10. [2025-10-20T18:47:47.918480] [INFO ] [REMOTE ] FLOC: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T18:47:47.922930] [INFO ] [REMOTE ] FLOC: Full fetch complete (40525 records). [2025-10-20T18:47:47.923058] [DEBUG] [VALIDATOR ] FLOC: Validating primary data. [2025-10-20T18:47:48.126701] [WARN ] [VALIDATOR ] FLOC: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T18:47:48.127156] [FATAL] [MAIN ] FLOC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:47:48.127225] [WARN ] [MAIN ] FLOC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:47:48.127294] [INFO ] [LOCAL ] FLOC: Starting deletion of primary data. [2025-10-20T18:47:48.127356] [DEBUG] [LOCAL ] FLOC: No primary data file found to delete. [2025-10-20T18:47:48.127393] [INFO ] [LOCAL ] FLOC: Starting deletion of derivative data. [2025-10-20T18:47:48.127430] [DEBUG] [LOCAL ] FLOC: No derivative data file found to delete. [2025-10-20T18:47:48.127486] [INFO ] [MAIN ] Update Progress: 841/2367 symbols completed. FLOC took 26.8s. Estimated remaining: 3h 15m 28s. Rate: 7.8 symbols/min [2025-10-20T18:47:48.266601] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-08. [2025-10-20T18:47:48.299681] [INFO ] [MAIN ] FLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:48.348797] [DEBUG] [LOCAL ] FLR: Loaded 54019 existing records from primary storage. [2025-10-20T18:47:48.349126] [INFO ] [MAIN ] FLR: Loaded 54019 existing records. [2025-10-20T18:47:48.524790] [WARN ] [MAIN ] FLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:48.525043] [INFO ] [MAIN ] FLR: Wiping existing primary data before full refresh. [2025-10-20T18:47:48.525114] [INFO ] [LOCAL ] FLR: Starting deletion of primary data. [2025-10-20T18:47:48.525828] [DEBUG] [LOCAL ] FLR: Deleted primary data file. [2025-10-20T18:47:48.525885] [INFO ] [MAIN ] FLR: Starting full primary refresh. [2025-10-20T18:47:48.850920] [INFO ] [REMOTE ] FLNG: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T18:47:48.903706] [INFO ] [REMOTE ] FLNG: Full fetch complete (50059 records). [2025-10-20T18:47:48.910793] [DEBUG] [VALIDATOR ] FLNG: Validating primary data. [2025-10-20T18:47:48.914522] [INFO ] [REMOTE ] FLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:49.056475] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-01. [2025-10-20T18:47:49.350940] [DEBUG] [VALIDATOR ] FLNG: All expected trading days are present. [2025-10-20T18:47:49.351018] [DEBUG] [VALIDATOR ] FLNG: Primary data validation passed. [2025-10-20T18:47:49.351813] [INFO ] [MAIN ] FLNG: Fetched primary data is valid. [2025-10-20T18:47:49.351877] [DEBUG] [LOCAL ] FLNG: Saving primary data. [2025-10-20T18:47:49.353022] [DEBUG] [LOCAL ] FLNG: Trimmed 50059 records to 33591 (keeping 200 days of data) [2025-10-20T18:47:49.565148] [DEBUG] [LOCAL ] FLNG: Primary data saved (33591 records). [2025-10-20T18:47:49.565284] [INFO ] [MAIN ] FLNG: Full history saved. [2025-10-20T18:47:49.565363] [INFO ] [MAIN ] Update Progress: 842/2367 symbols completed. FLNG took 28.3s. Estimated remaining: 3h 15m 9s. Rate: 7.8 symbols/min [2025-10-20T18:47:49.746859] [INFO ] [MAIN ] FLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:49.796868] [DEBUG] [LOCAL ] FLS: Loaded 51134 existing records from primary storage. [2025-10-20T18:47:49.797262] [INFO ] [MAIN ] FLS: Loaded 51134 existing records. [2025-10-20T18:47:49.857929] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-09. [2025-10-20T18:47:50.009866] [WARN ] [MAIN ] FLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:50.010203] [INFO ] [MAIN ] FLS: Wiping existing primary data before full refresh. [2025-10-20T18:47:50.010300] [INFO ] [LOCAL ] FLS: Starting deletion of primary data. [2025-10-20T18:47:50.010754] [DEBUG] [LOCAL ] FLS: Deleted primary data file. [2025-10-20T18:47:50.010845] [INFO ] [MAIN ] FLS: Starting full primary refresh. [2025-10-20T18:47:50.281223] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T18:47:50.459675] [INFO ] [REMOTE ] FLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:50.651377] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-09. [2025-10-20T18:47:50.766408] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:47:51.451391] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-02. [2025-10-20T18:47:51.550510] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:47:52.251365] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-01. [2025-10-20T18:47:52.301186] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T18:47:52.772853] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:47:53.051362] [INFO ] [REMOTE ] FLG: Fetching primary data for 2025-10. [2025-10-20T18:47:53.539502] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T18:47:53.851432] [INFO ] [REMOTE ] FLO: Fetching primary data for 2025-10. [2025-10-20T18:47:54.651409] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-03. [2025-10-20T18:47:54.783037] [INFO ] [REMOTE ] FLG: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:47:54.791188] [INFO ] [REMOTE ] FLG: Full fetch complete (77425 records). [2025-10-20T18:47:54.791333] [DEBUG] [VALIDATOR ] FLG: Validating primary data. [2025-10-20T18:47:55.186825] [DEBUG] [VALIDATOR ] FLG: All expected trading days are present. [2025-10-20T18:47:55.187026] [DEBUG] [VALIDATOR ] FLG: Primary data validation passed. [2025-10-20T18:47:55.187993] [INFO ] [MAIN ] FLG: Fetched primary data is valid. [2025-10-20T18:47:55.188065] [DEBUG] [LOCAL ] FLG: Saving primary data. [2025-10-20T18:47:55.189518] [DEBUG] [LOCAL ] FLG: Trimmed 77425 records to 53322 (keeping 200 days of data) [2025-10-20T18:47:55.465711] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-02. [2025-10-20T18:47:55.517428] [DEBUG] [LOCAL ] FLG: Primary data saved (53322 records). [2025-10-20T18:47:55.517791] [INFO ] [MAIN ] FLG: Full history saved. [2025-10-20T18:47:55.518457] [INFO ] [MAIN ] Update Progress: 843/2367 symbols completed. FLG took 34.2s. Estimated remaining: 3h 14m 59s. Rate: 7.8 symbols/min [2025-10-20T18:47:55.713365] [INFO ] [MAIN ] FLUT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:55.718403] [INFO ] [REMOTE ] FLO: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:47:55.727776] [INFO ] [REMOTE ] FLO: Full fetch complete (75303 records). [2025-10-20T18:47:55.728095] [DEBUG] [VALIDATOR ] FLO: Validating primary data. [2025-10-20T18:47:56.063780] [DEBUG] [LOCAL ] FLUT: Loaded 51478 existing records from primary storage. [2025-10-20T18:47:56.085396] [INFO ] [MAIN ] FLUT: Loaded 51478 existing records. [2025-10-20T18:47:56.391534] [DEBUG] [VALIDATOR ] FLO: All expected trading days are present. [2025-10-20T18:47:56.401539] [DEBUG] [VALIDATOR ] FLO: Primary data validation passed. [2025-10-20T18:47:56.406831] [INFO ] [MAIN ] FLO: Fetched primary data is valid. [2025-10-20T18:47:56.406983] [DEBUG] [LOCAL ] FLO: Saving primary data. [2025-10-20T18:47:56.440340] [WARN ] [MAIN ] FLUT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:56.441237] [INFO ] [MAIN ] FLUT: Wiping existing primary data before full refresh. [2025-10-20T18:47:56.441373] [INFO ] [LOCAL ] FLUT: Starting deletion of primary data. [2025-10-20T18:47:56.442129] [DEBUG] [LOCAL ] FLO: Trimmed 75303 records to 52083 (keeping 200 days of data) [2025-10-20T18:47:56.442219] [DEBUG] [LOCAL ] FLUT: Deleted primary data file. [2025-10-20T18:47:56.442444] [INFO ] [MAIN ] FLUT: Starting full primary refresh. [2025-10-20T18:47:56.722743] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T18:47:56.950501] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-04. [2025-10-20T18:47:57.140926] [INFO ] [REMOTE ] FLUT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:57.149338] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-01. [2025-10-20T18:47:57.256918] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:47:57.263379] [DEBUG] [LOCAL ] FLO: Primary data saved (52083 records). [2025-10-20T18:47:57.272734] [INFO ] [MAIN ] FLO: Full history saved. [2025-10-20T18:47:57.273064] [INFO ] [MAIN ] Update Progress: 844/2367 symbols completed. FLO took 35.9s. Estimated remaining: 3h 14m 40s. Rate: 7.8 symbols/min [2025-10-20T18:47:57.461201] [INFO ] [MAIN ] FLXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:47:57.485431] [DEBUG] [LOCAL ] FLXR: Loaded 21398 existing records from primary storage. [2025-10-20T18:47:57.485795] [INFO ] [MAIN ] FLXR: Loaded 21398 existing records. [2025-10-20T18:47:57.674316] [WARN ] [MAIN ] FLXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:47:57.674631] [INFO ] [MAIN ] FLXR: Wiping existing primary data before full refresh. [2025-10-20T18:47:57.674717] [INFO ] [LOCAL ] FLXR: Starting deletion of primary data. [2025-10-20T18:47:57.674959] [DEBUG] [LOCAL ] FLXR: Deleted primary data file. [2025-10-20T18:47:57.674997] [INFO ] [MAIN ] FLXR: Starting full primary refresh. [2025-10-20T18:47:57.855309] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-03. [2025-10-20T18:47:58.046593] [INFO ] [REMOTE ] FLXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:47:58.651374] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-01. [2025-10-20T18:47:58.779628] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:47:58.840874] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T18:47:59.236204] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T18:47:59.451361] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-05. [2025-10-20T18:47:59.739411] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T18:48:00.251470] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-04. [2025-10-20T18:48:01.051360] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-02. [2025-10-20T18:48:01.207369] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T18:48:01.738408] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:48:01.851393] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-06. [2025-10-20T18:48:02.651400] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-05. [2025-10-20T18:48:02.852253] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:48:02.984489] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:48:03.451434] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-02. [2025-10-20T18:48:04.215617] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T18:48:04.246851] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T18:48:04.256406] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-03. [2025-10-20T18:48:05.051396] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-07. [2025-10-20T18:48:05.193075] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T18:48:05.851414] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-03. [2025-10-20T18:48:06.651415] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-04. [2025-10-20T18:48:06.976013] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T18:48:07.451406] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-08. [2025-10-20T18:48:07.581774] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T18:48:07.738069] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:48:08.251403] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-04. [2025-10-20T18:48:09.051391] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-06. [2025-10-20T18:48:09.240272] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T18:48:09.851358] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-09. [2025-10-20T18:48:10.123867] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T18:48:10.651428] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-05. [2025-10-20T18:48:10.816597] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T18:48:11.240541] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T18:48:11.451411] [INFO ] [REMOTE ] FLR: Fetching primary data for 2025-10. [2025-10-20T18:48:11.696378] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:48:12.251448] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-05. [2025-10-20T18:48:13.051393] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-07. [2025-10-20T18:48:13.292629] [INFO ] [REMOTE ] FLR: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:48:13.300902] [INFO ] [REMOTE ] FLR: Full fetch complete (76837 records). [2025-10-20T18:48:13.301065] [DEBUG] [VALIDATOR ] FLR: Validating primary data. [2025-10-20T18:48:13.695989] [DEBUG] [VALIDATOR ] FLR: All expected trading days are present. [2025-10-20T18:48:13.696127] [DEBUG] [VALIDATOR ] FLR: Primary data validation passed. [2025-10-20T18:48:13.697079] [INFO ] [MAIN ] FLR: Fetched primary data is valid. [2025-10-20T18:48:13.697219] [DEBUG] [LOCAL ] FLR: Saving primary data. [2025-10-20T18:48:13.698762] [DEBUG] [LOCAL ] FLR: Trimmed 76837 records to 53258 (keeping 200 days of data) [2025-10-20T18:48:13.867320] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-06. [2025-10-20T18:48:14.027730] [DEBUG] [LOCAL ] FLR: Primary data saved (53258 records). [2025-10-20T18:48:14.027907] [INFO ] [MAIN ] FLR: Full history saved. [2025-10-20T18:48:14.028543] [INFO ] [MAIN ] Update Progress: 845/2367 symbols completed. FLR took 25.9s. Estimated remaining: 3h 14m 49s. Rate: 7.8 symbols/min [2025-10-20T18:48:14.208089] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T18:48:14.231542] [INFO ] [MAIN ] FMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:14.293272] [DEBUG] [LOCAL ] FMC: Loaded 52624 existing records from primary storage. [2025-10-20T18:48:14.293589] [INFO ] [MAIN ] FMC: Loaded 52624 existing records. [2025-10-20T18:48:14.473397] [WARN ] [MAIN ] FMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:14.473600] [INFO ] [MAIN ] FMC: Wiping existing primary data before full refresh. [2025-10-20T18:48:14.473671] [INFO ] [LOCAL ] FMC: Starting deletion of primary data. [2025-10-20T18:48:14.474247] [DEBUG] [LOCAL ] FMC: Deleted primary data file. [2025-10-20T18:48:14.474286] [INFO ] [MAIN ] FMC: Starting full primary refresh. [2025-10-20T18:48:14.654832] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-06. [2025-10-20T18:48:14.854424] [INFO ] [REMOTE ] FMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:14.887793] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T18:48:14.922553] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T18:48:15.451368] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-07. [2025-10-20T18:48:16.251437] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-01. [2025-10-20T18:48:16.436884] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T18:48:16.582961] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:48:17.051576] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-08. [2025-10-20T18:48:17.851391] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-07. [2025-10-20T18:48:17.991074] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T18:48:18.133376] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:48:18.651379] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-09. [2025-10-20T18:48:19.451343] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-02. [2025-10-20T18:48:19.549342] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T18:48:19.832485] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:48:20.251372] [INFO ] [REMOTE ] FLXR: Fetching primary data for 2025-10. [2025-10-20T18:48:21.051374] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-08. [2025-10-20T18:48:21.237484] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T18:48:21.624512] [INFO ] [REMOTE ] FLXR: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:48:21.628963] [INFO ] [REMOTE ] FLXR: Full fetch complete (27223 records). [2025-10-20T18:48:21.629070] [DEBUG] [VALIDATOR ] FLXR: Validating primary data. [2025-10-20T18:48:21.857706] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-03. [2025-10-20T18:48:22.066721] [DEBUG] [VALIDATOR ] FLXR: All expected trading days are present. [2025-10-20T18:48:22.066808] [DEBUG] [VALIDATOR ] FLXR: Primary data validation passed. [2025-10-20T18:48:22.067333] [INFO ] [MAIN ] FLXR: Fetched primary data is valid. [2025-10-20T18:48:22.067371] [DEBUG] [LOCAL ] FLXR: Saving primary data. [2025-10-20T18:48:22.068075] [DEBUG] [LOCAL ] FLXR: Trimmed 27223 records to 21242 (keeping 200 days of data) [2025-10-20T18:48:22.199949] [DEBUG] [LOCAL ] FLXR: Primary data saved (21242 records). [2025-10-20T18:48:22.200079] [INFO ] [MAIN ] FLXR: Full history saved. [2025-10-20T18:48:22.200154] [INFO ] [MAIN ] Update Progress: 846/2367 symbols completed. FLXR took 24.9s. Estimated remaining: 3h 14m 42s. Rate: 7.8 symbols/min [2025-10-20T18:48:22.373486] [INFO ] [MAIN ] FMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:22.381516] [DEBUG] [LOCAL ] FMN: Loaded 5571 existing records from primary storage. [2025-10-20T18:48:22.381867] [INFO ] [MAIN ] FMN: Loaded 5571 existing records. [2025-10-20T18:48:22.563719] [WARN ] [MAIN ] FMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:22.563967] [INFO ] [MAIN ] FMN: Wiping existing primary data before full refresh. [2025-10-20T18:48:22.564043] [INFO ] [LOCAL ] FMN: Starting deletion of primary data. [2025-10-20T18:48:22.564399] [DEBUG] [LOCAL ] FMN: Deleted primary data file. [2025-10-20T18:48:22.564522] [INFO ] [MAIN ] FMN: Starting full primary refresh. [2025-10-20T18:48:22.665113] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-08. [2025-10-20T18:48:22.946881] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T18:48:22.979612] [INFO ] [REMOTE ] FMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:23.451382] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-01. [2025-10-20T18:48:23.574222] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T18:48:24.251440] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-04. [2025-10-20T18:48:24.303134] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T18:48:24.418118] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:48:25.051384] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-09. [2025-10-20T18:48:25.856568] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-02. [2025-10-20T18:48:25.875807] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:48:26.607947] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T18:48:26.651386] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-03. [2025-10-20T18:48:26.742159] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T18:48:27.408888] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T18:48:27.451374] [INFO ] [REMOTE ] FLS: Fetching primary data for 2025-10. [2025-10-20T18:48:28.251417] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-04. [2025-10-20T18:48:28.955156] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T18:48:29.051365] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-09. [2025-10-20T18:48:29.317509] [INFO ] [REMOTE ] FLS: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:48:29.325217] [INFO ] [REMOTE ] FLS: Full fetch complete (72489 records). [2025-10-20T18:48:29.325361] [DEBUG] [VALIDATOR ] FLS: Validating primary data. [2025-10-20T18:48:29.709723] [DEBUG] [VALIDATOR ] FLS: All expected trading days are present. [2025-10-20T18:48:29.709809] [DEBUG] [VALIDATOR ] FLS: Primary data validation passed. [2025-10-20T18:48:29.710701] [INFO ] [MAIN ] FLS: Fetched primary data is valid. [2025-10-20T18:48:29.710764] [DEBUG] [LOCAL ] FLS: Saving primary data. [2025-10-20T18:48:29.712088] [DEBUG] [LOCAL ] FLS: Trimmed 72489 records to 50452 (keeping 200 days of data) [2025-10-20T18:48:29.882885] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-05. [2025-10-20T18:48:30.025014] [DEBUG] [LOCAL ] FLS: Primary data saved (50452 records). [2025-10-20T18:48:30.025149] [INFO ] [MAIN ] FLS: Full history saved. [2025-10-20T18:48:30.025238] [INFO ] [MAIN ] Update Progress: 847/2367 symbols completed. FLS took 40.5s. Estimated remaining: 3h 14m 35s. Rate: 7.8 symbols/min [2025-10-20T18:48:30.208465] [INFO ] [MAIN ] FMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:30.241243] [DEBUG] [LOCAL ] FMS: Loaded 30733 existing records from primary storage. [2025-10-20T18:48:30.241505] [INFO ] [MAIN ] FMS: Loaded 30733 existing records. [2025-10-20T18:48:30.418747] [WARN ] [MAIN ] FMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:30.418965] [INFO ] [MAIN ] FMS: Wiping existing primary data before full refresh. [2025-10-20T18:48:30.419123] [INFO ] [LOCAL ] FMS: Starting deletion of primary data. [2025-10-20T18:48:30.419449] [DEBUG] [LOCAL ] FMS: Deleted primary data file. [2025-10-20T18:48:30.419509] [INFO ] [MAIN ] FMS: Starting full primary refresh. [2025-10-20T18:48:30.508138] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T18:48:30.656696] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-06. [2025-10-20T18:48:30.810816] [INFO ] [REMOTE ] FMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:30.812708] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:48:31.367903] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:48:31.451342] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-01. [2025-10-20T18:48:32.251349] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-07. [2025-10-20T18:48:32.648215] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T18:48:32.814266] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T18:48:33.051378] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-02. [2025-10-20T18:48:33.851350] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-05. [2025-10-20T18:48:34.194803] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T18:48:34.651361] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-03. [2025-10-20T18:48:35.451401] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-08. [2025-10-20T18:48:35.585425] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T18:48:35.806333] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T18:48:36.035683] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T18:48:36.251385] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-09. [2025-10-20T18:48:36.903837] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:48:37.051381] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-06. [2025-10-20T18:48:37.851403] [INFO ] [REMOTE ] FMN: Fetching primary data for 2025-10. [2025-10-20T18:48:38.564540] [INFO ] [REMOTE ] FMN: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T18:48:38.566240] [INFO ] [REMOTE ] FMN: Full fetch complete (10091 records). [2025-10-20T18:48:38.566308] [DEBUG] [VALIDATOR ] FMN: Validating primary data. [2025-10-20T18:48:38.654949] [INFO ] [REMOTE ] FLUT: Fetching primary data for 2025-10. [2025-10-20T18:48:38.935162] [DEBUG] [VALIDATOR ] FMN: All expected trading days are present. [2025-10-20T18:48:38.945330] [DEBUG] [VALIDATOR ] FMN: Primary data validation passed. [2025-10-20T18:48:38.962886] [INFO ] [MAIN ] FMN: Fetched primary data is valid. [2025-10-20T18:48:38.962970] [DEBUG] [LOCAL ] FMN: Saving primary data. [2025-10-20T18:48:38.964997] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T18:48:38.965396] [DEBUG] [LOCAL ] FMN: Trimmed 10091 records to 5517 (keeping 200 days of data) [2025-10-20T18:48:39.008420] [DEBUG] [LOCAL ] FMN: Primary data saved (5517 records). [2025-10-20T18:48:39.008510] [INFO ] [MAIN ] FMN: Full history saved. [2025-10-20T18:48:39.008586] [INFO ] [MAIN ] Update Progress: 848/2367 symbols completed. FMN took 16.8s. Estimated remaining: 3h 14m 29s. Rate: 7.8 symbols/min [2025-10-20T18:48:39.196823] [INFO ] [MAIN ] FMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:39.234565] [DEBUG] [LOCAL ] FMX: Loaded 41172 existing records from primary storage. [2025-10-20T18:48:39.234868] [INFO ] [MAIN ] FMX: Loaded 41172 existing records. [2025-10-20T18:48:39.408700] [WARN ] [MAIN ] FMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:39.408902] [INFO ] [MAIN ] FMX: Wiping existing primary data before full refresh. [2025-10-20T18:48:39.408948] [INFO ] [LOCAL ] FMX: Starting deletion of primary data. [2025-10-20T18:48:39.409276] [DEBUG] [LOCAL ] FMX: Deleted primary data file. [2025-10-20T18:48:39.409332] [INFO ] [MAIN ] FMX: Starting full primary refresh. [2025-10-20T18:48:39.462497] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-07. [2025-10-20T18:48:39.759437] [INFO ] [REMOTE ] FMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:40.251385] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-01. [2025-10-20T18:48:40.620214] [INFO ] [REMOTE ] FLUT: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T18:48:40.627990] [INFO ] [REMOTE ] FLUT: Full fetch complete (73367 records). [2025-10-20T18:48:40.628146] [DEBUG] [VALIDATOR ] FLUT: Validating primary data. [2025-10-20T18:48:41.033023] [DEBUG] [VALIDATOR ] FLUT: All expected trading days are present. [2025-10-20T18:48:41.033104] [DEBUG] [VALIDATOR ] FLUT: Primary data validation passed. [2025-10-20T18:48:41.034061] [INFO ] [MAIN ] FLUT: Fetched primary data is valid. [2025-10-20T18:48:41.034125] [DEBUG] [LOCAL ] FLUT: Saving primary data. [2025-10-20T18:48:41.035518] [DEBUG] [LOCAL ] FLUT: Trimmed 73367 records to 50697 (keeping 200 days of data) [2025-10-20T18:48:41.086773] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-04. [2025-10-20T18:48:41.367369] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:48:41.392201] [DEBUG] [LOCAL ] FLUT: Primary data saved (50697 records). [2025-10-20T18:48:41.392703] [INFO ] [MAIN ] FLUT: Full history saved. [2025-10-20T18:48:41.393340] [INFO ] [MAIN ] Update Progress: 849/2367 symbols completed. FLUT took 45.9s. Estimated remaining: 3h 14m 12s. Rate: 7.8 symbols/min [2025-10-20T18:48:41.563249] [INFO ] [MAIN ] FMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:41.566940] [DEBUG] [LOCAL ] FMY: Loaded 1397 existing records from primary storage. [2025-10-20T18:48:41.567101] [INFO ] [MAIN ] FMY: Loaded 1397 existing records. [2025-10-20T18:48:41.750701] [WARN ] [MAIN ] FMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:41.751085] [INFO ] [MAIN ] FMY: Wiping existing primary data before full refresh. [2025-10-20T18:48:41.751162] [INFO ] [LOCAL ] FMY: Starting deletion of primary data. [2025-10-20T18:48:41.751528] [DEBUG] [LOCAL ] FMY: Deleted primary data file. [2025-10-20T18:48:41.751590] [INFO ] [MAIN ] FMY: Starting full primary refresh. [2025-10-20T18:48:41.853918] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-08. [2025-10-20T18:48:41.861963] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:48:42.146616] [INFO ] [REMOTE ] FMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:48:42.570204] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T18:48:42.651356] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-01. [2025-10-20T18:48:42.932534] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-01 (0.28 seconds). [2025-10-20T18:48:43.451364] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-05. [2025-10-20T18:48:43.617597] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T18:48:44.251400] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-02. [2025-10-20T18:48:44.799268] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T18:48:45.051360] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-06. [2025-10-20T18:48:45.584212] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:48:45.851397] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-03. [2025-10-20T18:48:46.336388] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:48:46.651370] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-02. [2025-10-20T18:48:46.933831] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-02 (0.28 seconds). [2025-10-20T18:48:47.334625] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T18:48:47.451342] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-04. [2025-10-20T18:48:48.251382] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-07. [2025-10-20T18:48:49.056579] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-03. [2025-10-20T18:48:49.068320] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T18:48:49.401806] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:48:49.569182] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T18:48:49.851400] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-04. [2025-10-20T18:48:50.387841] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:48:50.651384] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-05. [2025-10-20T18:48:50.954074] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-10-20T18:48:51.451376] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-05. [2025-10-20T18:48:52.251375] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-06. [2025-10-20T18:48:52.524716] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-06 (0.27 seconds). [2025-10-20T18:48:52.938234] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T18:48:53.051344] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-09. [2025-10-20T18:48:53.851426] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-07. [2025-10-20T18:48:54.276617] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T18:48:54.654712] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-08. [2025-10-20T18:48:54.658084] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:48:55.017573] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T18:48:55.451394] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-06. [2025-10-20T18:48:56.251382] [INFO ] [REMOTE ] FMC: Fetching primary data for 2025-10. [2025-10-20T18:48:56.686872] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T18:48:57.051357] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-07. [2025-10-20T18:48:57.851345] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-08. [2025-10-20T18:48:58.252477] [INFO ] [REMOTE ] FMC: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T18:48:58.260326] [INFO ] [REMOTE ] FMC: Full fetch complete (75384 records). [2025-10-20T18:48:58.260463] [DEBUG] [VALIDATOR ] FMC: Validating primary data. [2025-10-20T18:48:58.652036] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-09. [2025-10-20T18:48:58.652325] [DEBUG] [VALIDATOR ] FMC: All expected trading days are present. [2025-10-20T18:48:58.653382] [DEBUG] [VALIDATOR ] FMC: Primary data validation passed. [2025-10-20T18:48:58.654492] [INFO ] [MAIN ] FMC: Fetched primary data is valid. [2025-10-20T18:48:58.654624] [DEBUG] [LOCAL ] FMC: Saving primary data. [2025-10-20T18:48:58.681840] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T18:48:58.691181] [DEBUG] [LOCAL ] FMC: Trimmed 75384 records to 51892 (keeping 200 days of data) [2025-10-20T18:48:59.020211] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T18:48:59.048557] [DEBUG] [LOCAL ] FMC: Primary data saved (51892 records). [2025-10-20T18:48:59.048739] [INFO ] [MAIN ] FMC: Full history saved. [2025-10-20T18:48:59.049686] [INFO ] [MAIN ] Update Progress: 850/2367 symbols completed. FMC took 45.0s. Estimated remaining: 3h 14m 22s. Rate: 7.8 symbols/min [2025-10-20T18:48:59.149392] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T18:48:59.219516] [INFO ] [MAIN ] FN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:48:59.258786] [DEBUG] [LOCAL ] FN: Loaded 38299 existing records from primary storage. [2025-10-20T18:48:59.259059] [INFO ] [MAIN ] FN: Loaded 38299 existing records. [2025-10-20T18:48:59.434663] [WARN ] [MAIN ] FN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:48:59.434937] [INFO ] [MAIN ] FN: Wiping existing primary data before full refresh. [2025-10-20T18:48:59.434984] [INFO ] [LOCAL ] FN: Starting deletion of primary data. [2025-10-20T18:48:59.435871] [DEBUG] [LOCAL ] FN: Deleted primary data file. [2025-10-20T18:48:59.435943] [INFO ] [MAIN ] FN: Starting full primary refresh. [2025-10-20T18:48:59.456562] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-09. [2025-10-20T18:48:59.843704] [INFO ] [REMOTE ] FN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:00.251393] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-01. [2025-10-20T18:49:01.051449] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-08. [2025-10-20T18:49:01.087625] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T18:49:01.851410] [INFO ] [REMOTE ] FMS: Fetching primary data for 2025-10. [2025-10-20T18:49:01.954535] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:49:02.651375] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-02. [2025-10-20T18:49:02.777137] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:49:03.451056] [INFO ] [REMOTE ] FMS: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T18:49:03.455463] [INFO ] [REMOTE ] FMY: Fetching primary data for 2025-10. [2025-10-20T18:49:03.457029] [INFO ] [REMOTE ] FMS: Full fetch complete (42998 records). [2025-10-20T18:49:03.457101] [DEBUG] [VALIDATOR ] FMS: Validating primary data. [2025-10-20T18:49:03.887161] [DEBUG] [VALIDATOR ] FMS: All expected trading days are present. [2025-10-20T18:49:03.887380] [DEBUG] [VALIDATOR ] FMS: Primary data validation passed. [2025-10-20T18:49:03.888343] [INFO ] [MAIN ] FMS: Fetched primary data is valid. [2025-10-20T18:49:03.888486] [DEBUG] [LOCAL ] FMS: Saving primary data. [2025-10-20T18:49:03.889926] [DEBUG] [LOCAL ] FMS: Trimmed 42998 records to 30419 (keeping 200 days of data) [2025-10-20T18:49:04.025916] [INFO ] [REMOTE ] FMY: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T18:49:04.031619] [INFO ] [REMOTE ] FMY: Full fetch complete (1955 records). [2025-10-20T18:49:04.045688] [DEBUG] [VALIDATOR ] FMY: Validating primary data. [2025-10-20T18:49:04.257577] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-09. [2025-10-20T18:49:04.316392] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:49:04.605435] [DEBUG] [LOCAL ] FMS: Primary data saved (30419 records). [2025-10-20T18:49:04.616041] [INFO ] [MAIN ] FMS: Full history saved. [2025-10-20T18:49:04.626565] [INFO ] [MAIN ] Update Progress: 851/2367 symbols completed. FMS took 34.6s. Estimated remaining: 3h 14m 11s. Rate: 7.8 symbols/min [2025-10-20T18:49:04.805328] [DEBUG] [VALIDATOR ] FMY: All expected trading days are present. [2025-10-20T18:49:04.816142] [DEBUG] [VALIDATOR ] FMY: Primary data validation passed. [2025-10-20T18:49:04.832003] [INFO ] [MAIN ] FMY: Fetched primary data is valid. [2025-10-20T18:49:04.868579] [DEBUG] [LOCAL ] FMY: Saving primary data. [2025-10-20T18:49:04.886308] [DEBUG] [LOCAL ] FMY: Trimmed 1955 records to 1381 (keeping 200 days of data) [2025-10-20T18:49:04.887917] [INFO ] [MAIN ] FNB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:04.899792] [DEBUG] [LOCAL ] FMY: Primary data saved (1381 records). [2025-10-20T18:49:04.899983] [INFO ] [MAIN ] FMY: Full history saved. [2025-10-20T18:49:04.900083] [INFO ] [MAIN ] Update Progress: 852/2367 symbols completed. FMY took 23.5s. Estimated remaining: 3h 13m 50s. Rate: 7.8 symbols/min [2025-10-20T18:49:05.079795] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-03. [2025-10-20T18:49:05.097917] [INFO ] [MAIN ] FND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:05.127865] [DEBUG] [LOCAL ] FNB: Loaded 53376 existing records from primary storage. [2025-10-20T18:49:05.128324] [INFO ] [MAIN ] FNB: Loaded 53376 existing records. [2025-10-20T18:49:05.343429] [WARN ] [MAIN ] FNB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:05.343768] [INFO ] [MAIN ] FNB: Wiping existing primary data before full refresh. [2025-10-20T18:49:05.343851] [INFO ] [LOCAL ] FNB: Starting deletion of primary data. [2025-10-20T18:49:05.344774] [DEBUG] [LOCAL ] FND: Loaded 52865 existing records from primary storage. [2025-10-20T18:49:05.345108] [INFO ] [MAIN ] FND: Loaded 52865 existing records. [2025-10-20T18:49:05.377802] [DEBUG] [LOCAL ] FNB: Deleted primary data file. [2025-10-20T18:49:05.377906] [INFO ] [MAIN ] FNB: Starting full primary refresh. [2025-10-20T18:49:05.698608] [WARN ] [MAIN ] FND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:05.713872] [INFO ] [MAIN ] FND: Wiping existing primary data before full refresh. [2025-10-20T18:49:05.721727] [INFO ] [LOCAL ] FND: Starting deletion of primary data. [2025-10-20T18:49:05.757048] [DEBUG] [LOCAL ] FND: Deleted primary data file. [2025-10-20T18:49:05.767851] [INFO ] [MAIN ] FND: Starting full primary refresh. [2025-10-20T18:49:06.048908] [INFO ] [REMOTE ] FNB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:06.060448] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-01. [2025-10-20T18:49:06.315076] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T18:49:06.370043] [INFO ] [REMOTE ] FND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:06.651374] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-01. [2025-10-20T18:49:06.842771] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T18:49:07.451388] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-04. [2025-10-20T18:49:07.486706] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T18:49:07.966837] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T18:49:08.251371] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-02. [2025-10-20T18:49:09.051351] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-02. [2025-10-20T18:49:09.115909] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:49:09.851382] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-05. [2025-10-20T18:49:09.913500] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:49:10.593195] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:49:10.651338] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-03. [2025-10-20T18:49:11.095592] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T18:49:11.451393] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-06. [2025-10-20T18:49:12.251365] [INFO ] [REMOTE ] FMX: Fetching primary data for 2025-10. [2025-10-20T18:49:12.432152] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:49:12.670082] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T18:49:13.051373] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-07. [2025-10-20T18:49:13.851411] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-04. [2025-10-20T18:49:13.913798] [INFO ] [REMOTE ] FMX: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T18:49:13.919479] [INFO ] [REMOTE ] FMX: Full fetch complete (57933 records). [2025-10-20T18:49:13.919614] [DEBUG] [VALIDATOR ] FMX: Validating primary data. [2025-10-20T18:49:14.315809] [DEBUG] [VALIDATOR ] FMX: All expected trading days are present. [2025-10-20T18:49:14.315921] [DEBUG] [VALIDATOR ] FMX: Primary data validation passed. [2025-10-20T18:49:14.316732] [INFO ] [MAIN ] FMX: Fetched primary data is valid. [2025-10-20T18:49:14.316864] [DEBUG] [LOCAL ] FMX: Saving primary data. [2025-10-20T18:49:14.318236] [DEBUG] [LOCAL ] FMX: Trimmed 57933 records to 40649 (keeping 200 days of data) [2025-10-20T18:49:14.481333] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:49:14.624128] [DEBUG] [LOCAL ] FMX: Primary data saved (40649 records). [2025-10-20T18:49:14.624249] [INFO ] [MAIN ] FMX: Full history saved. [2025-10-20T18:49:14.624324] [INFO ] [MAIN ] Update Progress: 853/2367 symbols completed. FMX took 35.6s. Estimated remaining: 3h 13m 46s. Rate: 7.8 symbols/min [2025-10-20T18:49:14.656473] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-08. [2025-10-20T18:49:14.794286] [INFO ] [MAIN ] FNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:14.836121] [DEBUG] [LOCAL ] FNF: Loaded 49639 existing records from primary storage. [2025-10-20T18:49:14.836456] [INFO ] [MAIN ] FNF: Loaded 49639 existing records. [2025-10-20T18:49:15.014362] [WARN ] [MAIN ] FNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:15.014564] [INFO ] [MAIN ] FNF: Wiping existing primary data before full refresh. [2025-10-20T18:49:15.014610] [INFO ] [LOCAL ] FNF: Starting deletion of primary data. [2025-10-20T18:49:15.015155] [DEBUG] [LOCAL ] FNF: Deleted primary data file. [2025-10-20T18:49:15.015209] [INFO ] [MAIN ] FNF: Starting full primary refresh. [2025-10-20T18:49:15.370317] [INFO ] [REMOTE ] FNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:15.451364] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-03. [2025-10-20T18:49:15.682569] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T18:49:16.251432] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-05. [2025-10-20T18:49:16.405255] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T18:49:17.051413] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-09. [2025-10-20T18:49:17.089918] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T18:49:17.802370] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T18:49:17.851357] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-06. [2025-10-20T18:49:18.195344] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:49:18.651355] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-01. [2025-10-20T18:49:19.451444] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-04. [2025-10-20T18:49:19.564413] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T18:49:20.033210] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T18:49:20.251389] [INFO ] [REMOTE ] FN: Fetching primary data for 2025-10. [2025-10-20T18:49:21.051356] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-02. [2025-10-20T18:49:21.155871] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T18:49:21.608330] [INFO ] [REMOTE ] FN: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T18:49:21.613285] [INFO ] [REMOTE ] FN: Full fetch complete (56372 records). [2025-10-20T18:49:21.613403] [DEBUG] [VALIDATOR ] FN: Validating primary data. [2025-10-20T18:49:21.856637] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-05. [2025-10-20T18:49:22.012100] [DEBUG] [VALIDATOR ] FN: All expected trading days are present. [2025-10-20T18:49:22.012192] [DEBUG] [VALIDATOR ] FN: Primary data validation passed. [2025-10-20T18:49:22.012996] [INFO ] [MAIN ] FN: Fetched primary data is valid. [2025-10-20T18:49:22.013060] [DEBUG] [LOCAL ] FN: Saving primary data. [2025-10-20T18:49:22.014364] [DEBUG] [LOCAL ] FN: Trimmed 56372 records to 37620 (keeping 200 days of data) [2025-10-20T18:49:22.245935] [DEBUG] [LOCAL ] FN: Primary data saved (37620 records). [2025-10-20T18:49:22.246096] [INFO ] [MAIN ] FN: Full history saved. [2025-10-20T18:49:22.246279] [INFO ] [MAIN ] Update Progress: 854/2367 symbols completed. FN took 23.2s. Estimated remaining: 3h 13m 38s. Rate: 7.8 symbols/min [2025-10-20T18:49:22.394079] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:49:22.487475] [INFO ] [MAIN ] FNV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:22.535178] [DEBUG] [LOCAL ] FNV: Loaded 46890 existing records from primary storage. [2025-10-20T18:49:22.535554] [INFO ] [MAIN ] FNV: Loaded 46890 existing records. [2025-10-20T18:49:22.656552] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-03. [2025-10-20T18:49:22.715939] [WARN ] [MAIN ] FNV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:22.716126] [INFO ] [MAIN ] FNV: Wiping existing primary data before full refresh. [2025-10-20T18:49:22.716172] [INFO ] [LOCAL ] FNV: Starting deletion of primary data. [2025-10-20T18:49:22.716753] [DEBUG] [LOCAL ] FNV: Deleted primary data file. [2025-10-20T18:49:22.716808] [INFO ] [MAIN ] FNV: Starting full primary refresh. [2025-10-20T18:49:23.065828] [INFO ] [REMOTE ] FNV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:23.451417] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-01. [2025-10-20T18:49:23.750877] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:49:24.251492] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-06. [2025-10-20T18:49:24.485583] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T18:49:24.913044] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T18:49:25.051404] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-04. [2025-10-20T18:49:25.856539] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-02. [2025-10-20T18:49:25.878138] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T18:49:26.651365] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-07. [2025-10-20T18:49:26.810007] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T18:49:27.385921] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:49:27.451336] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-03. [2025-10-20T18:49:28.251412] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-05. [2025-10-20T18:49:28.439899] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:49:29.056501] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-07. [2025-10-20T18:49:29.072508] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:49:29.851432] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-04. [2025-10-20T18:49:29.950547] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T18:49:30.242966] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T18:49:30.651424] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-08. [2025-10-20T18:49:31.451363] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-08. [2025-10-20T18:49:31.647419] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:49:32.165084] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T18:49:32.251356] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-09. [2025-10-20T18:49:33.051420] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-06. [2025-10-20T18:49:33.230548] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:49:33.856518] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-09. [2025-10-20T18:49:33.874874] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:49:34.612123] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:49:34.651331] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-07. [2025-10-20T18:49:35.451407] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-05. [2025-10-20T18:49:35.580734] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T18:49:36.052024] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T18:49:36.251381] [INFO ] [REMOTE ] FNB: Fetching primary data for 2025-10. [2025-10-20T18:49:37.051360] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-08. [2025-10-20T18:49:37.128222] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:49:37.851400] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-06. [2025-10-20T18:49:38.101380] [INFO ] [REMOTE ] FNB: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T18:49:38.109048] [INFO ] [REMOTE ] FNB: Full fetch complete (76209 records). [2025-10-20T18:49:38.109197] [DEBUG] [VALIDATOR ] FNB: Validating primary data. [2025-10-20T18:49:38.521672] [DEBUG] [VALIDATOR ] FNB: All expected trading days are present. [2025-10-20T18:49:38.521816] [DEBUG] [VALIDATOR ] FNB: Primary data validation passed. [2025-10-20T18:49:38.522747] [INFO ] [MAIN ] FNB: Fetched primary data is valid. [2025-10-20T18:49:38.522826] [DEBUG] [LOCAL ] FNB: Saving primary data. [2025-10-20T18:49:38.524337] [DEBUG] [LOCAL ] FNB: Trimmed 76209 records to 52605 (keeping 200 days of data) [2025-10-20T18:49:38.701948] [INFO ] [REMOTE ] FND: Fetching primary data for 2025-10. [2025-10-20T18:49:38.780113] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:49:38.897422] [DEBUG] [LOCAL ] FNB: Primary data saved (52605 records). [2025-10-20T18:49:38.897569] [INFO ] [MAIN ] FNB: Full history saved. [2025-10-20T18:49:38.897683] [INFO ] [MAIN ] Update Progress: 855/2367 symbols completed. FNB took 34.3s. Estimated remaining: 3h 13m 46s. Rate: 7.8 symbols/min [2025-10-20T18:49:39.084999] [INFO ] [MAIN ] FOA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:39.098590] [DEBUG] [LOCAL ] FOA: Loaded 13224 existing records from primary storage. [2025-10-20T18:49:39.098808] [INFO ] [MAIN ] FOA: Loaded 13224 existing records. [2025-10-20T18:49:39.288356] [WARN ] [MAIN ] FOA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:39.288783] [INFO ] [MAIN ] FOA: Wiping existing primary data before full refresh. [2025-10-20T18:49:39.288907] [INFO ] [LOCAL ] FOA: Starting deletion of primary data. [2025-10-20T18:49:39.289317] [DEBUG] [LOCAL ] FOA: Deleted primary data file. [2025-10-20T18:49:39.289389] [INFO ] [MAIN ] FOA: Starting full primary refresh. [2025-10-20T18:49:39.460094] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-09. [2025-10-20T18:49:39.461073] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:49:39.758668] [INFO ] [REMOTE ] FOA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:40.251416] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-07. [2025-10-20T18:49:40.526965] [INFO ] [REMOTE ] FND: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T18:49:40.534483] [INFO ] [REMOTE ] FND: Full fetch complete (75042 records). [2025-10-20T18:49:40.534597] [DEBUG] [VALIDATOR ] FND: Validating primary data. [2025-10-20T18:49:40.927926] [DEBUG] [VALIDATOR ] FND: All expected trading days are present. [2025-10-20T18:49:40.928066] [DEBUG] [VALIDATOR ] FND: Primary data validation passed. [2025-10-20T18:49:40.929109] [INFO ] [MAIN ] FND: Fetched primary data is valid. [2025-10-20T18:49:40.929298] [DEBUG] [LOCAL ] FND: Saving primary data. [2025-10-20T18:49:40.930762] [DEBUG] [LOCAL ] FND: Trimmed 75042 records to 52091 (keeping 200 days of data) [2025-10-20T18:49:41.041377] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:49:41.052547] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-01. [2025-10-20T18:49:41.268990] [DEBUG] [LOCAL ] FND: Primary data saved (52091 records). [2025-10-20T18:49:41.269166] [INFO ] [MAIN ] FND: Full history saved. [2025-10-20T18:49:41.269682] [INFO ] [MAIN ] Update Progress: 856/2367 symbols completed. FND took 36.4s. Estimated remaining: 3h 13m 29s. Rate: 7.8 symbols/min [2025-10-20T18:49:41.436381] [INFO ] [MAIN ] FOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:41.448510] [DEBUG] [LOCAL ] FOF: Loaded 8707 existing records from primary storage. [2025-10-20T18:49:41.448755] [INFO ] [MAIN ] FOF: Loaded 8707 existing records. [2025-10-20T18:49:41.627366] [WARN ] [MAIN ] FOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:41.627794] [INFO ] [MAIN ] FOF: Wiping existing primary data before full refresh. [2025-10-20T18:49:41.627851] [INFO ] [LOCAL ] FOF: Starting deletion of primary data. [2025-10-20T18:49:41.628060] [DEBUG] [LOCAL ] FOF: Deleted primary data file. [2025-10-20T18:49:41.628116] [INFO ] [MAIN ] FOF: Starting full primary refresh. [2025-10-20T18:49:41.856830] [INFO ] [REMOTE ] FNF: Fetching primary data for 2025-10. [2025-10-20T18:49:41.991337] [INFO ] [REMOTE ] FOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:42.017367] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:49:42.088212] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:49:42.651369] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-02. [2025-10-20T18:49:43.451413] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-08. [2025-10-20T18:49:43.478834] [INFO ] [REMOTE ] FNF: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:49:43.485887] [INFO ] [REMOTE ] FNF: Full fetch complete (70031 records). [2025-10-20T18:49:43.485988] [DEBUG] [VALIDATOR ] FNF: Validating primary data. [2025-10-20T18:49:43.809946] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T18:49:43.901001] [DEBUG] [VALIDATOR ] FNF: All expected trading days are present. [2025-10-20T18:49:43.901082] [DEBUG] [VALIDATOR ] FNF: Primary data validation passed. [2025-10-20T18:49:43.901927] [INFO ] [MAIN ] FNF: Fetched primary data is valid. [2025-10-20T18:49:43.901989] [DEBUG] [LOCAL ] FNF: Saving primary data. [2025-10-20T18:49:43.903395] [DEBUG] [LOCAL ] FNF: Trimmed 70031 records to 48842 (keeping 200 days of data) [2025-10-20T18:49:44.193038] [DEBUG] [LOCAL ] FNF: Primary data saved (48842 records). [2025-10-20T18:49:44.193219] [INFO ] [MAIN ] FNF: Full history saved. [2025-10-20T18:49:44.193312] [INFO ] [MAIN ] Update Progress: 857/2367 symbols completed. FNF took 29.6s. Estimated remaining: 3h 13m 13s. Rate: 7.8 symbols/min [2025-10-20T18:49:44.257687] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-01. [2025-10-20T18:49:44.368357] [INFO ] [MAIN ] FOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:44.396614] [DEBUG] [LOCAL ] FOR: Loaded 20025 existing records from primary storage. [2025-10-20T18:49:44.396901] [INFO ] [MAIN ] FOR: Loaded 20025 existing records. [2025-10-20T18:49:44.579162] [WARN ] [MAIN ] FOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:44.579400] [INFO ] [MAIN ] FOR: Wiping existing primary data before full refresh. [2025-10-20T18:49:44.579447] [INFO ] [LOCAL ] FOR: Starting deletion of primary data. [2025-10-20T18:49:44.579620] [DEBUG] [LOCAL ] FOR: Deleted primary data file. [2025-10-20T18:49:44.579727] [INFO ] [MAIN ] FOR: Starting full primary refresh. [2025-10-20T18:49:44.952964] [INFO ] [REMOTE ] FOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:45.045250] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:49:45.051318] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-02. [2025-10-20T18:49:45.089130] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T18:49:45.512847] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T18:49:45.851349] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-03. [2025-10-20T18:49:46.599999] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T18:49:46.651341] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-04. [2025-10-20T18:49:47.245915] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T18:49:47.451365] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-09. [2025-10-20T18:49:48.251396] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-05. [2025-10-20T18:49:48.714460] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T18:49:49.051339] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-03. [2025-10-20T18:49:49.101245] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T18:49:49.851345] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-06. [2025-10-20T18:49:50.005751] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T18:49:50.576349] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T18:49:50.651373] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-07. [2025-10-20T18:49:51.154802] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T18:49:51.451425] [INFO ] [REMOTE ] FNV: Fetching primary data for 2025-10. [2025-10-20T18:49:52.251415] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-04. [2025-10-20T18:49:53.056447] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-01. [2025-10-20T18:49:53.068711] [INFO ] [REMOTE ] FNV: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T18:49:53.076230] [INFO ] [REMOTE ] FNV: Full fetch complete (66861 records). [2025-10-20T18:49:53.076354] [DEBUG] [VALIDATOR ] FNV: Validating primary data. [2025-10-20T18:49:53.249071] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T18:49:53.480240] [DEBUG] [VALIDATOR ] FNV: All expected trading days are present. [2025-10-20T18:49:53.480322] [DEBUG] [VALIDATOR ] FNV: Primary data validation passed. [2025-10-20T18:49:53.481197] [INFO ] [MAIN ] FNV: Fetched primary data is valid. [2025-10-20T18:49:53.481260] [DEBUG] [LOCAL ] FNV: Saving primary data. [2025-10-20T18:49:53.482531] [DEBUG] [LOCAL ] FNV: Trimmed 66861 records to 46152 (keeping 200 days of data) [2025-10-20T18:49:53.762598] [DEBUG] [LOCAL ] FNV: Primary data saved (46152 records). [2025-10-20T18:49:53.762790] [INFO ] [MAIN ] FNV: Full history saved. [2025-10-20T18:49:53.762892] [INFO ] [MAIN ] Update Progress: 858/2367 symbols completed. FNV took 31.5s. Estimated remaining: 3h 13m 9s. Rate: 7.8 symbols/min [2025-10-20T18:49:53.859693] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-08. [2025-10-20T18:49:53.931569] [INFO ] [MAIN ] FOUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:49:53.978047] [DEBUG] [LOCAL ] FOUR: Loaded 50475 existing records from primary storage. [2025-10-20T18:49:53.978341] [INFO ] [MAIN ] FOUR: Loaded 50475 existing records. [2025-10-20T18:49:54.154959] [WARN ] [MAIN ] FOUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:49:54.155208] [INFO ] [MAIN ] FOUR: Wiping existing primary data before full refresh. [2025-10-20T18:49:54.155273] [INFO ] [LOCAL ] FOUR: Starting deletion of primary data. [2025-10-20T18:49:54.155858] [DEBUG] [LOCAL ] FOUR: Deleted primary data file. [2025-10-20T18:49:54.155923] [INFO ] [MAIN ] FOUR: Starting full primary refresh. [2025-10-20T18:49:54.325696] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:49:54.539071] [INFO ] [REMOTE ] FOUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:49:54.624271] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T18:49:54.651347] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-01. [2025-10-20T18:49:55.451377] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-09. [2025-10-20T18:49:56.251358] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-05. [2025-10-20T18:49:56.262426] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T18:49:56.331261] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T18:49:57.051376] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-02. [2025-10-20T18:49:57.146765] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T18:49:57.851388] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-06. [2025-10-20T18:49:58.635616] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T18:49:58.651352] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-02. [2025-10-20T18:49:58.847468] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T18:49:59.451371] [INFO ] [REMOTE ] FOF: Fetching primary data for 2025-10. [2025-10-20T18:49:59.675181] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:49:59.987161] [INFO ] [REMOTE ] FOF: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T18:49:59.989793] [INFO ] [REMOTE ] FOF: Full fetch complete (11895 records). [2025-10-20T18:49:59.989890] [DEBUG] [VALIDATOR ] FOF: Validating primary data. [2025-10-20T18:50:00.256559] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-07. [2025-10-20T18:50:00.371439] [DEBUG] [VALIDATOR ] FOF: All expected trading days are present. [2025-10-20T18:50:00.371515] [DEBUG] [VALIDATOR ] FOF: Primary data validation passed. [2025-10-20T18:50:00.371799] [INFO ] [MAIN ] FOF: Fetched primary data is valid. [2025-10-20T18:50:00.371858] [DEBUG] [LOCAL ] FOF: Saving primary data. [2025-10-20T18:50:00.372344] [DEBUG] [LOCAL ] FOF: Trimmed 11895 records to 8615 (keeping 200 days of data) [2025-10-20T18:50:00.431725] [DEBUG] [LOCAL ] FOF: Primary data saved (8615 records). [2025-10-20T18:50:00.431914] [INFO ] [MAIN ] FOF: Full history saved. [2025-10-20T18:50:00.432007] [INFO ] [MAIN ] Update Progress: 859/2367 symbols completed. FOF took 19.2s. Estimated remaining: 3h 12m 60s. Rate: 7.8 symbols/min [2025-10-20T18:50:00.640661] [INFO ] [MAIN ] FPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:00.662393] [DEBUG] [LOCAL ] FPF: Loaded 19175 existing records from primary storage. [2025-10-20T18:50:00.662680] [INFO ] [MAIN ] FPF: Loaded 19175 existing records. [2025-10-20T18:50:00.867125] [WARN ] [MAIN ] FPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:00.867342] [INFO ] [MAIN ] FPF: Wiping existing primary data before full refresh. [2025-10-20T18:50:00.867391] [INFO ] [LOCAL ] FPF: Starting deletion of primary data. [2025-10-20T18:50:00.867721] [DEBUG] [LOCAL ] FPF: Deleted primary data file. [2025-10-20T18:50:00.867778] [INFO ] [MAIN ] FPF: Starting full primary refresh. [2025-10-20T18:50:01.051771] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-03. [2025-10-20T18:50:01.254618] [INFO ] [REMOTE ] FPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:01.375085] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:50:01.851360] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-01. [2025-10-20T18:50:02.651338] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-08. [2025-10-20T18:50:02.835267] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T18:50:03.246475] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T18:50:03.451448] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-03. [2025-10-20T18:50:03.629621] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:50:04.251408] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-04. [2025-10-20T18:50:04.684795] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T18:50:05.051395] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-04. [2025-10-20T18:50:05.851349] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-09. [2025-10-20T18:50:06.258441] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T18:50:06.293749] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:50:06.651411] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-05. [2025-10-20T18:50:06.859609] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:50:07.451355] [INFO ] [REMOTE ] FOA: Fetching primary data for 2025-10. [2025-10-20T18:50:07.824320] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T18:50:08.251367] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-05. [2025-10-20T18:50:08.374940] [INFO ] [REMOTE ] FOA: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T18:50:08.377846] [INFO ] [REMOTE ] FOA: Full fetch complete (21457 records). [2025-10-20T18:50:08.378016] [DEBUG] [VALIDATOR ] FOA: Validating primary data. [2025-10-20T18:50:08.756574] [DEBUG] [VALIDATOR ] FOA: All expected trading days are present. [2025-10-20T18:50:08.756685] [DEBUG] [VALIDATOR ] FOA: Primary data validation passed. [2025-10-20T18:50:08.757163] [INFO ] [MAIN ] FOA: Fetched primary data is valid. [2025-10-20T18:50:08.757206] [DEBUG] [LOCAL ] FOA: Saving primary data. [2025-10-20T18:50:08.757886] [DEBUG] [LOCAL ] FOA: Trimmed 21457 records to 13064 (keeping 200 days of data) [2025-10-20T18:50:08.833079] [DEBUG] [LOCAL ] FOA: Primary data saved (13064 records). [2025-10-20T18:50:08.833223] [INFO ] [MAIN ] FOA: Full history saved. [2025-10-20T18:50:08.833302] [INFO ] [MAIN ] Update Progress: 860/2367 symbols completed. FOA took 29.9s. Estimated remaining: 3h 12m 53s. Rate: 7.8 symbols/min [2025-10-20T18:50:09.002472] [INFO ] [MAIN ] FPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:09.022432] [DEBUG] [LOCAL ] FPH: Loaded 19603 existing records from primary storage. [2025-10-20T18:50:09.022760] [INFO ] [MAIN ] FPH: Loaded 19603 existing records. [2025-10-20T18:50:09.056577] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-02. [2025-10-20T18:50:09.205192] [WARN ] [MAIN ] FPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:09.205419] [INFO ] [MAIN ] FPH: Wiping existing primary data before full refresh. [2025-10-20T18:50:09.205469] [INFO ] [LOCAL ] FPH: Starting deletion of primary data. [2025-10-20T18:50:09.205757] [DEBUG] [LOCAL ] FPH: Deleted primary data file. [2025-10-20T18:50:09.205814] [INFO ] [MAIN ] FPH: Starting full primary refresh. [2025-10-20T18:50:09.561084] [INFO ] [REMOTE ] FPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:09.851364] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-01. [2025-10-20T18:50:10.025634] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T18:50:10.075839] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T18:50:10.651378] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-06. [2025-10-20T18:50:11.152970] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T18:50:11.451381] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-03. [2025-10-20T18:50:11.646051] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T18:50:12.251385] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-07. [2025-10-20T18:50:12.443412] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T18:50:13.051343] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-06. [2025-10-20T18:50:13.188853] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T18:50:13.851385] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-08. [2025-10-20T18:50:14.568606] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T18:50:14.651355] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-07. [2025-10-20T18:50:14.884037] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T18:50:15.451387] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-02. [2025-10-20T18:50:16.251345] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-04. [2025-10-20T18:50:16.292074] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:50:16.579124] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T18:50:17.051356] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-03. [2025-10-20T18:50:17.374294] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T18:50:17.780378] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:50:17.851389] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-04. [2025-10-20T18:50:18.651349] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-08. [2025-10-20T18:50:18.883295] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T18:50:19.451400] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-05. [2025-10-20T18:50:20.234506] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:50:20.251322] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-06. [2025-10-20T18:50:20.312880] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T18:50:21.051385] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-09. [2025-10-20T18:50:21.144126] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T18:50:21.851381] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-07. [2025-10-20T18:50:22.656487] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-05. [2025-10-20T18:50:22.675625] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:50:22.982406] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T18:50:23.204357] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:50:23.451416] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-06. [2025-10-20T18:50:24.251396] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-08. [2025-10-20T18:50:24.470540] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T18:50:25.051407] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-07. [2025-10-20T18:50:25.239522] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T18:50:25.851348] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-09. [2025-10-20T18:50:26.061904] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:50:26.651352] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-09. [2025-10-20T18:50:26.915753] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T18:50:27.451381] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-08. [2025-10-20T18:50:27.630976] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T18:50:28.251345] [INFO ] [REMOTE ] FPH: Fetching primary data for 2025-10. [2025-10-20T18:50:28.412001] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:50:29.051366] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-09. [2025-10-20T18:50:29.272993] [INFO ] [REMOTE ] FPH: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T18:50:29.276193] [INFO ] [REMOTE ] FPH: Full fetch complete (30906 records). [2025-10-20T18:50:29.276319] [DEBUG] [VALIDATOR ] FPH: Validating primary data. [2025-10-20T18:50:29.642558] [DEBUG] [VALIDATOR ] FPH: All expected trading days are present. [2025-10-20T18:50:29.642666] [DEBUG] [VALIDATOR ] FPH: Primary data validation passed. [2025-10-20T18:50:29.643245] [INFO ] [MAIN ] FPH: Fetched primary data is valid. [2025-10-20T18:50:29.643300] [DEBUG] [LOCAL ] FPH: Saving primary data. [2025-10-20T18:50:29.643983] [DEBUG] [LOCAL ] FPH: Trimmed 30906 records to 19328 (keeping 200 days of data) [2025-10-20T18:50:29.746993] [DEBUG] [LOCAL ] FPH: Primary data saved (19328 records). [2025-10-20T18:50:29.747118] [INFO ] [MAIN ] FPH: Full history saved. [2025-10-20T18:50:29.747193] [INFO ] [MAIN ] Update Progress: 861/2367 symbols completed. FPH took 20.9s. Estimated remaining: 3h 13m 9s. Rate: 7.8 symbols/min [2025-10-20T18:50:29.856502] [INFO ] [REMOTE ] FOUR: Fetching primary data for 2025-10. [2025-10-20T18:50:29.919015] [INFO ] [MAIN ] FPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:29.950531] [DEBUG] [LOCAL ] FPI: Loaded 31833 existing records from primary storage. [2025-10-20T18:50:29.950844] [INFO ] [MAIN ] FPI: Loaded 31833 existing records. [2025-10-20T18:50:30.086363] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T18:50:30.150551] [WARN ] [MAIN ] FPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:30.150782] [INFO ] [MAIN ] FPI: Wiping existing primary data before full refresh. [2025-10-20T18:50:30.150832] [INFO ] [LOCAL ] FPI: Starting deletion of primary data. [2025-10-20T18:50:30.151016] [DEBUG] [LOCAL ] FPI: Deleted primary data file. [2025-10-20T18:50:30.151053] [INFO ] [MAIN ] FPI: Starting full primary refresh. [2025-10-20T18:50:30.502710] [INFO ] [REMOTE ] FPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:30.651351] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-01. [2025-10-20T18:50:31.451358] [INFO ] [REMOTE ] FOR: Fetching primary data for 2025-10. [2025-10-20T18:50:31.844017] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T18:50:32.132364] [INFO ] [REMOTE ] FOUR: Incremental fetch complete for 2025-10 (2.28 seconds). [2025-10-20T18:50:32.140406] [INFO ] [REMOTE ] FOUR: Full fetch complete (72267 records). [2025-10-20T18:50:32.140592] [DEBUG] [VALIDATOR ] FOUR: Validating primary data. [2025-10-20T18:50:32.256740] [INFO ] [REMOTE ] FPF: Fetching primary data for 2025-10. [2025-10-20T18:50:32.587735] [DEBUG] [VALIDATOR ] FOUR: All expected trading days are present. [2025-10-20T18:50:32.587841] [DEBUG] [VALIDATOR ] FOUR: Primary data validation passed. [2025-10-20T18:50:32.588828] [INFO ] [MAIN ] FOUR: Fetched primary data is valid. [2025-10-20T18:50:32.588893] [DEBUG] [LOCAL ] FOUR: Saving primary data. [2025-10-20T18:50:32.590537] [DEBUG] [LOCAL ] FOUR: Trimmed 72267 records to 49761 (keeping 200 days of data) [2025-10-20T18:50:32.712891] [INFO ] [REMOTE ] FOR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:50:32.787080] [INFO ] [REMOTE ] FOR: Full fetch complete (29773 records). [2025-10-20T18:50:32.787531] [DEBUG] [VALIDATOR ] FOR: Validating primary data. [2025-10-20T18:50:33.056080] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-02. [2025-10-20T18:50:33.285415] [DEBUG] [LOCAL ] FOUR: Primary data saved (49761 records). [2025-10-20T18:50:33.314459] [DEBUG] [VALIDATOR ] FOR: All expected trading days are present. [2025-10-20T18:50:33.312237] [INFO ] [MAIN ] FOUR: Full history saved. [2025-10-20T18:50:33.314745] [DEBUG] [VALIDATOR ] FOR: Primary data validation passed. [2025-10-20T18:50:33.315700] [INFO ] [MAIN ] FOR: Fetched primary data is valid. [2025-10-20T18:50:33.316567] [INFO ] [MAIN ] Update Progress: 862/2367 symbols completed. FOUR took 39.6s. Estimated remaining: 3h 12m 54s. Rate: 7.8 symbols/min [2025-10-20T18:50:33.316701] [DEBUG] [LOCAL ] FOR: Saving primary data. [2025-10-20T18:50:33.434089] [INFO ] [REMOTE ] FPF: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:50:33.434791] [DEBUG] [LOCAL ] FOR: Trimmed 29773 records to 19738 (keeping 200 days of data) [2025-10-20T18:50:33.478708] [INFO ] [REMOTE ] FPF: Full fetch complete (26543 records). [2025-10-20T18:50:33.541876] [INFO ] [MAIN ] FR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:33.542018] [DEBUG] [VALIDATOR ] FPF: Validating primary data. [2025-10-20T18:50:33.854720] [DEBUG] [LOCAL ] FOR: Primary data saved (19738 records). [2025-10-20T18:50:33.898453] [INFO ] [MAIN ] FOR: Full history saved. [2025-10-20T18:50:33.931077] [INFO ] [MAIN ] Update Progress: 863/2367 symbols completed. FOR took 49.7s. Estimated remaining: 3h 12m 34s. Rate: 7.8 symbols/min [2025-10-20T18:50:34.183891] [DEBUG] [LOCAL ] FR: Loaded 47301 existing records from primary storage. [2025-10-20T18:50:34.222035] [INFO ] [MAIN ] FR: Loaded 47301 existing records. [2025-10-20T18:50:34.227475] [DEBUG] [VALIDATOR ] FPF: All expected trading days are present. [2025-10-20T18:50:34.319190] [DEBUG] [VALIDATOR ] FPF: Primary data validation passed. [2025-10-20T18:50:34.347400] [INFO ] [MAIN ] FRA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:34.353747] [INFO ] [MAIN ] FPF: Fetched primary data is valid. [2025-10-20T18:50:34.390036] [DEBUG] [LOCAL ] FPF: Saving primary data. [2025-10-20T18:50:34.487872] [WARN ] [MAIN ] FR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:34.489142] [INFO ] [MAIN ] FR: Wiping existing primary data before full refresh. [2025-10-20T18:50:34.490890] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T18:50:34.496171] [INFO ] [LOCAL ] FR: Starting deletion of primary data. [2025-10-20T18:50:34.489287] [DEBUG] [LOCAL ] FPF: Trimmed 26543 records to 18844 (keeping 200 days of data) [2025-10-20T18:50:34.498242] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-03. [2025-10-20T18:50:34.497924] [DEBUG] [LOCAL ] FR: Deleted primary data file. [2025-10-20T18:50:34.505446] [INFO ] [MAIN ] FR: Starting full primary refresh. [2025-10-20T18:50:34.505378] [DEBUG] [LOCAL ] FRA: Loaded 14476 existing records from primary storage. [2025-10-20T18:50:34.560181] [INFO ] [MAIN ] FRA: Loaded 14476 existing records. [2025-10-20T18:50:34.849035] [DEBUG] [LOCAL ] FPF: Primary data saved (18844 records). [2025-10-20T18:50:34.859512] [INFO ] [MAIN ] FPF: Full history saved. [2025-10-20T18:50:34.859822] [INFO ] [MAIN ] Update Progress: 864/2367 symbols completed. FPF took 34.4s. Estimated remaining: 3h 12m 14s. Rate: 7.8 symbols/min [2025-10-20T18:50:35.039328] [WARN ] [MAIN ] FRA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:35.074310] [INFO ] [MAIN ] FRA: Wiping existing primary data before full refresh. [2025-10-20T18:50:35.080688] [INFO ] [LOCAL ] FRA: Starting deletion of primary data. [2025-10-20T18:50:35.123539] [DEBUG] [LOCAL ] FRA: Deleted primary data file. [2025-10-20T18:50:35.147972] [INFO ] [MAIN ] FRA: Starting full primary refresh. [2025-10-20T18:50:35.357428] [INFO ] [MAIN ] FRGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:35.489454] [INFO ] [REMOTE ] FR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:35.562344] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-01. [2025-10-20T18:50:35.656621] [DEBUG] [LOCAL ] FRGE: Loaded 16315 existing records from primary storage. [2025-10-20T18:50:35.676604] [INFO ] [MAIN ] FRGE: Loaded 16315 existing records. [2025-10-20T18:50:35.857131] [INFO ] [REMOTE ] FRA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:35.862964] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-01. [2025-10-20T18:50:35.951878] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T18:50:35.955633] [WARN ] [MAIN ] FRGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:35.962430] [INFO ] [MAIN ] FRGE: Wiping existing primary data before full refresh. [2025-10-20T18:50:35.962793] [INFO ] [LOCAL ] FRGE: Starting deletion of primary data. [2025-10-20T18:50:35.963035] [DEBUG] [LOCAL ] FRGE: Deleted primary data file. [2025-10-20T18:50:35.963092] [INFO ] [MAIN ] FRGE: Starting full primary refresh. [2025-10-20T18:50:36.256475] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-04. [2025-10-20T18:50:36.306760] [INFO ] [REMOTE ] FRGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:37.031486] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T18:50:37.051328] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-02. [2025-10-20T18:50:37.133208] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:50:37.245993] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T18:50:37.851404] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-05. [2025-10-20T18:50:38.004214] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T18:50:38.651392] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-01. [2025-10-20T18:50:39.243227] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:50:39.451372] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-02. [2025-10-20T18:50:40.091038] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T18:50:40.251358] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-06. [2025-10-20T18:50:40.970583] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:50:41.051370] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-03. [2025-10-20T18:50:41.579375] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T18:50:41.851431] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-03. [2025-10-20T18:50:42.507408] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T18:50:42.651340] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-04. [2025-10-20T18:50:42.827517] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T18:50:43.451389] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-07. [2025-10-20T18:50:44.251391] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-04. [2025-10-20T18:50:44.369464] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T18:50:44.901976] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T18:50:45.051346] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-08. [2025-10-20T18:50:45.219868] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T18:50:45.851351] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-05. [2025-10-20T18:50:45.980993] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:50:46.651357] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-09. [2025-10-20T18:50:46.755390] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T18:50:47.451345] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-02. [2025-10-20T18:50:47.672724] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T18:50:48.251385] [INFO ] [REMOTE ] FPI: Fetching primary data for 2025-10. [2025-10-20T18:50:48.719182] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:50:49.051369] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-03. [2025-10-20T18:50:49.309673] [INFO ] [REMOTE ] FPI: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:50:49.315138] [INFO ] [REMOTE ] FPI: Full fetch complete (43901 records). [2025-10-20T18:50:49.315264] [DEBUG] [VALIDATOR ] FPI: Validating primary data. [2025-10-20T18:50:49.731508] [DEBUG] [VALIDATOR ] FPI: All expected trading days are present. [2025-10-20T18:50:49.731821] [DEBUG] [VALIDATOR ] FPI: Primary data validation passed. [2025-10-20T18:50:49.732753] [INFO ] [MAIN ] FPI: Fetched primary data is valid. [2025-10-20T18:50:49.732825] [DEBUG] [LOCAL ] FPI: Saving primary data. [2025-10-20T18:50:49.734150] [DEBUG] [LOCAL ] FPI: Trimmed 43901 records to 31464 (keeping 200 days of data) [2025-10-20T18:50:49.855062] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-05. [2025-10-20T18:50:49.961320] [DEBUG] [LOCAL ] FPI: Primary data saved (31464 records). [2025-10-20T18:50:49.961476] [INFO ] [MAIN ] FPI: Full history saved. [2025-10-20T18:50:49.961561] [INFO ] [MAIN ] Update Progress: 865/2367 symbols completed. FPI took 20.2s. Estimated remaining: 3h 12m 19s. Rate: 7.8 symbols/min [2025-10-20T18:50:50.177499] [INFO ] [MAIN ] FRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:50:50.236934] [DEBUG] [LOCAL ] FRO: Loaded 52460 existing records from primary storage. [2025-10-20T18:50:50.237525] [INFO ] [MAIN ] FRO: Loaded 52460 existing records. [2025-10-20T18:50:50.502132] [WARN ] [MAIN ] FRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:50:50.502513] [INFO ] [MAIN ] FRO: Wiping existing primary data before full refresh. [2025-10-20T18:50:50.502678] [INFO ] [LOCAL ] FRO: Starting deletion of primary data. [2025-10-20T18:50:50.512754] [DEBUG] [LOCAL ] FRO: Deleted primary data file. [2025-10-20T18:50:50.527692] [INFO ] [MAIN ] FRO: Starting full primary refresh. [2025-10-20T18:50:50.571873] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:50:50.656520] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-06. [2025-10-20T18:50:51.102065] [INFO ] [REMOTE ] FRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:50:51.451380] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-01. [2025-10-20T18:50:51.604880] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T18:50:51.700949] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T18:50:52.251408] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-04. [2025-10-20T18:50:53.051388] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-06. [2025-10-20T18:50:53.485049] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:50:53.512220] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T18:50:53.851376] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-05. [2025-10-20T18:50:54.421292] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:50:54.651364] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-07. [2025-10-20T18:50:54.890134] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:50:55.451374] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-07. [2025-10-20T18:50:56.251437] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-02. [2025-10-20T18:50:56.303431] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:50:56.418037] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T18:50:57.051424] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-08. [2025-10-20T18:50:57.851459] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-06. [2025-10-20T18:50:57.870516] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:50:58.117477] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T18:50:58.651431] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-08. [2025-10-20T18:50:58.719139] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T18:50:59.451382] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-07. [2025-10-20T18:51:00.116381] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T18:51:00.251393] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-09. [2025-10-20T18:51:00.340148] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:51:01.051426] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-03. [2025-10-20T18:51:01.692824] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T18:51:01.851417] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-09. [2025-10-20T18:51:02.651414] [INFO ] [REMOTE ] FRA: Fetching primary data for 2025-10. [2025-10-20T18:51:03.155448] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T18:51:03.313581] [INFO ] [REMOTE ] FRA: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T18:51:03.317090] [INFO ] [REMOTE ] FRA: Full fetch complete (23561 records). [2025-10-20T18:51:03.317320] [DEBUG] [VALIDATOR ] FRA: Validating primary data. [2025-10-20T18:51:03.437958] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:51:03.456637] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-08. [2025-10-20T18:51:03.719140] [DEBUG] [VALIDATOR ] FRA: All expected trading days are present. [2025-10-20T18:51:03.719221] [DEBUG] [VALIDATOR ] FRA: Primary data validation passed. [2025-10-20T18:51:03.719702] [INFO ] [MAIN ] FRA: Fetched primary data is valid. [2025-10-20T18:51:03.719761] [DEBUG] [LOCAL ] FRA: Saving primary data. [2025-10-20T18:51:03.720328] [DEBUG] [LOCAL ] FRA: Trimmed 23561 records to 14261 (keeping 200 days of data) [2025-10-20T18:51:03.809446] [DEBUG] [LOCAL ] FRA: Primary data saved (14261 records). [2025-10-20T18:51:03.809592] [INFO ] [MAIN ] FRA: Full history saved. [2025-10-20T18:51:03.809695] [INFO ] [MAIN ] Update Progress: 866/2367 symbols completed. FRA took 29.9s. Estimated remaining: 3h 12m 22s. Rate: 7.8 symbols/min [2025-10-20T18:51:03.978963] [INFO ] [MAIN ] FRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:04.016313] [DEBUG] [LOCAL ] FRT: Loaded 42290 existing records from primary storage. [2025-10-20T18:51:04.016615] [INFO ] [MAIN ] FRT: Loaded 42290 existing records. [2025-10-20T18:51:04.198667] [WARN ] [MAIN ] FRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:04.198936] [INFO ] [MAIN ] FRT: Wiping existing primary data before full refresh. [2025-10-20T18:51:04.198990] [INFO ] [LOCAL ] FRT: Starting deletion of primary data. [2025-10-20T18:51:04.199289] [DEBUG] [LOCAL ] FRT: Deleted primary data file. [2025-10-20T18:51:04.199347] [INFO ] [MAIN ] FRT: Starting full primary refresh. [2025-10-20T18:51:04.260669] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-04. [2025-10-20T18:51:04.522584] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T18:51:04.608469] [INFO ] [REMOTE ] FRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:05.051426] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-01. [2025-10-20T18:51:05.851452] [INFO ] [REMOTE ] FR: Fetching primary data for 2025-10. [2025-10-20T18:51:06.193254] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T18:51:06.521004] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T18:51:06.651389] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-05. [2025-10-20T18:51:07.449312] [INFO ] [REMOTE ] FR: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T18:51:07.455756] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-02. [2025-10-20T18:51:07.457506] [INFO ] [REMOTE ] FR: Full fetch complete (67109 records). [2025-10-20T18:51:07.457589] [DEBUG] [VALIDATOR ] FR: Validating primary data. [2025-10-20T18:51:07.858271] [DEBUG] [VALIDATOR ] FR: All expected trading days are present. [2025-10-20T18:51:07.858364] [DEBUG] [VALIDATOR ] FR: Primary data validation passed. [2025-10-20T18:51:07.859297] [INFO ] [MAIN ] FR: Fetched primary data is valid. [2025-10-20T18:51:07.859377] [DEBUG] [LOCAL ] FR: Saving primary data. [2025-10-20T18:51:07.860898] [DEBUG] [LOCAL ] FR: Trimmed 67109 records to 46684 (keeping 200 days of data) [2025-10-20T18:51:07.985566] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:51:08.190089] [DEBUG] [LOCAL ] FR: Primary data saved (46684 records). [2025-10-20T18:51:08.190236] [INFO ] [MAIN ] FR: Full history saved. [2025-10-20T18:51:08.190319] [INFO ] [MAIN ] Update Progress: 867/2367 symbols completed. FR took 34.9s. Estimated remaining: 3h 12m 9s. Rate: 7.8 symbols/min [2025-10-20T18:51:08.256503] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-09. [2025-10-20T18:51:08.365153] [INFO ] [MAIN ] FSCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:08.403072] [DEBUG] [LOCAL ] FSCO: Loaded 39152 existing records from primary storage. [2025-10-20T18:51:08.403407] [INFO ] [MAIN ] FSCO: Loaded 39152 existing records. [2025-10-20T18:51:08.587935] [WARN ] [MAIN ] FSCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:08.588152] [INFO ] [MAIN ] FSCO: Wiping existing primary data before full refresh. [2025-10-20T18:51:08.588200] [INFO ] [LOCAL ] FSCO: Starting deletion of primary data. [2025-10-20T18:51:08.588705] [DEBUG] [LOCAL ] FSCO: Deleted primary data file. [2025-10-20T18:51:08.588761] [INFO ] [MAIN ] FSCO: Starting full primary refresh. [2025-10-20T18:51:08.961811] [INFO ] [REMOTE ] FSCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:09.028394] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T18:51:09.051353] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-06. [2025-10-20T18:51:09.136382] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:51:09.851374] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-03. [2025-10-20T18:51:10.651415] [INFO ] [REMOTE ] FRGE: Fetching primary data for 2025-10. [2025-10-20T18:51:10.718184] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T18:51:11.427599] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:51:11.451378] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-04. [2025-10-20T18:51:11.533789] [INFO ] [REMOTE ] FRGE: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T18:51:11.536485] [INFO ] [REMOTE ] FRGE: Full fetch complete (30455 records). [2025-10-20T18:51:11.536598] [DEBUG] [VALIDATOR ] FRGE: Validating primary data. [2025-10-20T18:51:11.959552] [DEBUG] [VALIDATOR ] FRGE: All expected trading days are present. [2025-10-20T18:51:11.959662] [DEBUG] [VALIDATOR ] FRGE: Primary data validation passed. [2025-10-20T18:51:11.960272] [INFO ] [MAIN ] FRGE: Fetched primary data is valid. [2025-10-20T18:51:11.960315] [DEBUG] [LOCAL ] FRGE: Saving primary data. [2025-10-20T18:51:11.961249] [DEBUG] [LOCAL ] FRGE: Trimmed 30455 records to 15767 (keeping 200 days of data) [2025-10-20T18:51:12.066421] [DEBUG] [LOCAL ] FRGE: Primary data saved (15767 records). [2025-10-20T18:51:12.066586] [INFO ] [MAIN ] FRGE: Full history saved. [2025-10-20T18:51:12.066709] [INFO ] [MAIN ] Update Progress: 868/2367 symbols completed. FRGE took 37.2s. Estimated remaining: 3h 11m 55s. Rate: 7.8 symbols/min [2025-10-20T18:51:12.251731] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-07. [2025-10-20T18:51:12.269941] [INFO ] [MAIN ] FSK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:12.319014] [DEBUG] [LOCAL ] FSK: Loaded 50273 existing records from primary storage. [2025-10-20T18:51:12.319367] [INFO ] [MAIN ] FSK: Loaded 50273 existing records. [2025-10-20T18:51:12.495969] [WARN ] [MAIN ] FSK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:12.496187] [INFO ] [MAIN ] FSK: Wiping existing primary data before full refresh. [2025-10-20T18:51:12.496234] [INFO ] [LOCAL ] FSK: Starting deletion of primary data. [2025-10-20T18:51:12.496424] [DEBUG] [LOCAL ] FSK: Deleted primary data file. [2025-10-20T18:51:12.496461] [INFO ] [MAIN ] FSK: Starting full primary refresh. [2025-10-20T18:51:12.851005] [INFO ] [REMOTE ] FSK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:13.051403] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-01. [2025-10-20T18:51:13.099152] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:51:13.851378] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-05. [2025-10-20T18:51:14.023174] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T18:51:14.206674] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:51:14.651384] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-08. [2025-10-20T18:51:15.390262] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:51:15.451411] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-06. [2025-10-20T18:51:16.261376] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-02. [2025-10-20T18:51:16.267048] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T18:51:16.412162] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T18:51:17.051449] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-01. [2025-10-20T18:51:17.534206] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T18:51:17.851377] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-03. [2025-10-20T18:51:18.651397] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-07. [2025-10-20T18:51:18.723753] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:51:19.451445] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-02. [2025-10-20T18:51:19.601358] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:51:19.928283] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T18:51:20.251393] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-04. [2025-10-20T18:51:21.051404] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-08. [2025-10-20T18:51:21.180346] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T18:51:21.736882] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:51:21.851349] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-05. [2025-10-20T18:51:22.546772] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T18:51:22.651377] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-03. [2025-10-20T18:51:23.110036] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T18:51:23.451400] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-09. [2025-10-20T18:51:24.257537] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-09. [2025-10-20T18:51:24.272983] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:51:25.063855] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-06. [2025-10-20T18:51:25.066338] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T18:51:25.457732] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T18:51:25.851389] [INFO ] [REMOTE ] FRO: Fetching primary data for 2025-10. [2025-10-20T18:51:26.465495] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:51:26.651377] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-04. [2025-10-20T18:51:27.451437] [INFO ] [REMOTE ] FRT: Fetching primary data for 2025-10. [2025-10-20T18:51:27.713433] [INFO ] [REMOTE ] FRO: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T18:51:27.721058] [INFO ] [REMOTE ] FRO: Full fetch complete (75378 records). [2025-10-20T18:51:27.721186] [DEBUG] [VALIDATOR ] FRO: Validating primary data. [2025-10-20T18:51:28.116328] [DEBUG] [VALIDATOR ] FRO: All expected trading days are present. [2025-10-20T18:51:28.116494] [DEBUG] [VALIDATOR ] FRO: Primary data validation passed. [2025-10-20T18:51:28.117469] [INFO ] [MAIN ] FRO: Fetched primary data is valid. [2025-10-20T18:51:28.117575] [DEBUG] [LOCAL ] FRO: Saving primary data. [2025-10-20T18:51:28.119327] [DEBUG] [LOCAL ] FRO: Trimmed 75378 records to 51691 (keeping 200 days of data) [2025-10-20T18:51:28.242944] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:51:28.256973] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-07. [2025-10-20T18:51:28.463726] [DEBUG] [LOCAL ] FRO: Primary data saved (51691 records). [2025-10-20T18:51:28.463874] [INFO ] [MAIN ] FRO: Full history saved. [2025-10-20T18:51:28.463953] [INFO ] [MAIN ] Update Progress: 869/2367 symbols completed. FRO took 38.5s. Estimated remaining: 3h 12m 2s. Rate: 7.8 symbols/min [2025-10-20T18:51:28.634962] [INFO ] [MAIN ] FSLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:28.680989] [DEBUG] [LOCAL ] FSLY: Loaded 52088 existing records from primary storage. [2025-10-20T18:51:28.681385] [INFO ] [MAIN ] FSLY: Loaded 52088 existing records. [2025-10-20T18:51:28.870228] [INFO ] [REMOTE ] FRT: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T18:51:28.873792] [WARN ] [MAIN ] FSLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:28.879151] [INFO ] [MAIN ] FSLY: Wiping existing primary data before full refresh. [2025-10-20T18:51:28.879400] [INFO ] [LOCAL ] FSLY: Starting deletion of primary data. [2025-10-20T18:51:28.879740] [DEBUG] [LOCAL ] FSLY: Deleted primary data file. [2025-10-20T18:51:28.879806] [INFO ] [MAIN ] FSLY: Starting full primary refresh. [2025-10-20T18:51:28.879881] [INFO ] [REMOTE ] FRT: Full fetch complete (60504 records). [2025-10-20T18:51:28.941939] [DEBUG] [VALIDATOR ] FRT: Validating primary data. [2025-10-20T18:51:29.056456] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-05. [2025-10-20T18:51:29.593062] [INFO ] [REMOTE ] FSLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:29.669487] [DEBUG] [VALIDATOR ] FRT: All expected trading days are present. [2025-10-20T18:51:29.669732] [DEBUG] [VALIDATOR ] FRT: Primary data validation passed. [2025-10-20T18:51:29.670573] [INFO ] [MAIN ] FRT: Fetched primary data is valid. [2025-10-20T18:51:29.670692] [DEBUG] [LOCAL ] FRT: Saving primary data. [2025-10-20T18:51:29.695280] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:51:29.694190] [DEBUG] [LOCAL ] FRT: Trimmed 60504 records to 41648 (keeping 200 days of data) [2025-10-20T18:51:29.876592] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-01. [2025-10-20T18:51:29.955841] [DEBUG] [LOCAL ] FRT: Primary data saved (41648 records). [2025-10-20T18:51:29.955948] [INFO ] [MAIN ] FRT: Full history saved. [2025-10-20T18:51:29.956019] [INFO ] [MAIN ] Update Progress: 870/2367 symbols completed. FRT took 26.1s. Estimated remaining: 3h 11m 44s. Rate: 7.8 symbols/min [2025-10-20T18:51:30.123523] [INFO ] [MAIN ] FSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:30.173183] [DEBUG] [LOCAL ] FSM: Loaded 54081 existing records from primary storage. [2025-10-20T18:51:30.173631] [INFO ] [MAIN ] FSM: Loaded 54081 existing records. [2025-10-20T18:51:30.352970] [WARN ] [MAIN ] FSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:30.353188] [INFO ] [MAIN ] FSM: Wiping existing primary data before full refresh. [2025-10-20T18:51:30.353242] [INFO ] [LOCAL ] FSM: Starting deletion of primary data. [2025-10-20T18:51:30.353437] [DEBUG] [LOCAL ] FSM: Deleted primary data file. [2025-10-20T18:51:30.353492] [INFO ] [MAIN ] FSM: Starting full primary refresh. [2025-10-20T18:51:30.654353] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-08. [2025-10-20T18:51:30.709613] [INFO ] [REMOTE ] FSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:30.738185] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:51:31.451345] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-01. [2025-10-20T18:51:31.706362] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T18:51:31.735310] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:51:32.251424] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-02. [2025-10-20T18:51:33.051374] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-06. [2025-10-20T18:51:33.186846] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T18:51:33.851390] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-02. [2025-10-20T18:51:33.943623] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T18:51:34.143122] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T18:51:34.651415] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-03. [2025-10-20T18:51:35.451369] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-07. [2025-10-20T18:51:35.607336] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:51:36.251434] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-09. [2025-10-20T18:51:36.388473] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T18:51:37.051354] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-03. [2025-10-20T18:51:37.135351] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T18:51:37.731350] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T18:51:37.851360] [INFO ] [REMOTE ] FSCO: Fetching primary data for 2025-10. [2025-10-20T18:51:38.651418] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-08. [2025-10-20T18:51:38.841504] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T18:51:39.451408] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-04. [2025-10-20T18:51:39.530233] [INFO ] [REMOTE ] FSCO: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:51:39.537152] [INFO ] [REMOTE ] FSCO: Full fetch complete (55405 records). [2025-10-20T18:51:39.537254] [DEBUG] [VALIDATOR ] FSCO: Validating primary data. [2025-10-20T18:51:39.927993] [DEBUG] [VALIDATOR ] FSCO: All expected trading days are present. [2025-10-20T18:51:39.928071] [DEBUG] [VALIDATOR ] FSCO: Primary data validation passed. [2025-10-20T18:51:39.928818] [INFO ] [MAIN ] FSCO: Fetched primary data is valid. [2025-10-20T18:51:39.928878] [DEBUG] [LOCAL ] FSCO: Saving primary data. [2025-10-20T18:51:39.930012] [DEBUG] [LOCAL ] FSCO: Trimmed 55405 records to 38696 (keeping 200 days of data) [2025-10-20T18:51:40.147917] [DEBUG] [LOCAL ] FSCO: Primary data saved (38696 records). [2025-10-20T18:51:40.148046] [INFO ] [MAIN ] FSCO: Full history saved. [2025-10-20T18:51:40.148123] [INFO ] [MAIN ] Update Progress: 871/2367 symbols completed. FSCO took 32.0s. Estimated remaining: 3h 11m 40s. Rate: 7.8 symbols/min [2025-10-20T18:51:40.253402] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-04. [2025-10-20T18:51:40.318293] [INFO ] [MAIN ] FSS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:40.376527] [DEBUG] [LOCAL ] FSS: Loaded 34103 existing records from primary storage. [2025-10-20T18:51:40.378095] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:51:40.388723] [INFO ] [MAIN ] FSS: Loaded 34103 existing records. [2025-10-20T18:51:40.567502] [WARN ] [MAIN ] FSS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:40.567713] [INFO ] [MAIN ] FSS: Wiping existing primary data before full refresh. [2025-10-20T18:51:40.567769] [INFO ] [LOCAL ] FSS: Starting deletion of primary data. [2025-10-20T18:51:40.567953] [DEBUG] [LOCAL ] FSS: Deleted primary data file. [2025-10-20T18:51:40.568009] [INFO ] [MAIN ] FSS: Starting full primary refresh. [2025-10-20T18:51:40.917291] [INFO ] [REMOTE ] FSS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:41.060252] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-09. [2025-10-20T18:51:41.065580] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T18:51:41.851351] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-01. [2025-10-20T18:51:42.049318] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T18:51:42.656859] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-05. [2025-10-20T18:51:42.677908] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:51:43.109736] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T18:51:43.451370] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-02. [2025-10-20T18:51:44.251371] [INFO ] [REMOTE ] FSK: Fetching primary data for 2025-10. [2025-10-20T18:51:44.310851] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T18:51:44.784704] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T18:51:45.051512] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-06. [2025-10-20T18:51:45.851437] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-05. [2025-10-20T18:51:46.197920] [INFO ] [REMOTE ] FSK: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T18:51:46.205579] [INFO ] [REMOTE ] FSK: Full fetch complete (72132 records). [2025-10-20T18:51:46.205726] [DEBUG] [VALIDATOR ] FSK: Validating primary data. [2025-10-20T18:51:46.598747] [DEBUG] [VALIDATOR ] FSK: All expected trading days are present. [2025-10-20T18:51:46.598878] [DEBUG] [VALIDATOR ] FSK: Primary data validation passed. [2025-10-20T18:51:46.599888] [INFO ] [MAIN ] FSK: Fetched primary data is valid. [2025-10-20T18:51:46.599953] [DEBUG] [LOCAL ] FSK: Saving primary data. [2025-10-20T18:51:46.601455] [DEBUG] [LOCAL ] FSK: Trimmed 72132 records to 49506 (keeping 200 days of data) [2025-10-20T18:51:46.655376] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-03. [2025-10-20T18:51:46.870329] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T18:51:46.965965] [DEBUG] [LOCAL ] FSK: Primary data saved (49506 records). [2025-10-20T18:51:46.966099] [INFO ] [MAIN ] FSK: Full history saved. [2025-10-20T18:51:46.966531] [INFO ] [MAIN ] Update Progress: 872/2367 symbols completed. FSK took 34.9s. Estimated remaining: 3h 11m 31s. Rate: 7.8 symbols/min [2025-10-20T18:51:47.133035] [INFO ] [MAIN ] FT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:47.142596] [DEBUG] [LOCAL ] FT: Loaded 6884 existing records from primary storage. [2025-10-20T18:51:47.142978] [INFO ] [MAIN ] FT: Loaded 6884 existing records. [2025-10-20T18:51:47.317982] [WARN ] [MAIN ] FT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:47.318169] [INFO ] [MAIN ] FT: Wiping existing primary data before full refresh. [2025-10-20T18:51:47.318214] [INFO ] [LOCAL ] FT: Starting deletion of primary data. [2025-10-20T18:51:47.318360] [DEBUG] [LOCAL ] FT: Deleted primary data file. [2025-10-20T18:51:47.318396] [INFO ] [MAIN ] FT: Starting full primary refresh. [2025-10-20T18:51:47.455343] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-07. [2025-10-20T18:51:47.702920] [INFO ] [REMOTE ] FT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:47.737350] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T18:51:48.123419] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:51:48.251346] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-01. [2025-10-20T18:51:48.816048] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T18:51:49.057294] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-02. [2025-10-20T18:51:49.064432] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T18:51:49.587661] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:51:49.851368] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-08. [2025-10-20T18:51:50.651371] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-06. [2025-10-20T18:51:51.451364] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-03. [2025-10-20T18:51:51.662269] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T18:51:52.044286] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T18:51:52.073666] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T18:51:52.251369] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-09. [2025-10-20T18:51:53.051363] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-04. [2025-10-20T18:51:53.851428] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-04. [2025-10-20T18:51:54.117441] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T18:51:54.202068] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T18:51:54.561103] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T18:51:54.651425] [INFO ] [REMOTE ] FSM: Fetching primary data for 2025-10. [2025-10-20T18:51:55.451376] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-05. [2025-10-20T18:51:56.143979] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T18:51:56.251434] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-06. [2025-10-20T18:51:56.623579] [INFO ] [REMOTE ] FSM: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T18:51:56.632997] [INFO ] [REMOTE ] FSM: Full fetch complete (77119 records). [2025-10-20T18:51:56.633165] [DEBUG] [VALIDATOR ] FSM: Validating primary data. [2025-10-20T18:51:56.917407] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:51:57.043291] [DEBUG] [VALIDATOR ] FSM: All expected trading days are present. [2025-10-20T18:51:57.043585] [DEBUG] [VALIDATOR ] FSM: Primary data validation passed. [2025-10-20T18:51:57.044536] [INFO ] [MAIN ] FSM: Fetched primary data is valid. [2025-10-20T18:51:57.044613] [DEBUG] [LOCAL ] FSM: Saving primary data. [2025-10-20T18:51:57.046214] [DEBUG] [LOCAL ] FSM: Trimmed 77119 records to 53303 (keeping 200 days of data) [2025-10-20T18:51:57.092198] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-05. [2025-10-20T18:51:57.354339] [DEBUG] [LOCAL ] FSM: Primary data saved (53303 records). [2025-10-20T18:51:57.354480] [INFO ] [MAIN ] FSM: Full history saved. [2025-10-20T18:51:57.354559] [INFO ] [MAIN ] Update Progress: 873/2367 symbols completed. FSM took 27.4s. Estimated remaining: 3h 11m 28s. Rate: 7.8 symbols/min [2025-10-20T18:51:57.523985] [INFO ] [MAIN ] FTHY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:51:57.537945] [DEBUG] [LOCAL ] FTHY: Loaded 12522 existing records from primary storage. [2025-10-20T18:51:57.538193] [INFO ] [MAIN ] FTHY: Loaded 12522 existing records. [2025-10-20T18:51:57.711891] [WARN ] [MAIN ] FTHY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:51:57.712169] [INFO ] [MAIN ] FTHY: Wiping existing primary data before full refresh. [2025-10-20T18:51:57.712217] [INFO ] [LOCAL ] FTHY: Starting deletion of primary data. [2025-10-20T18:51:57.712381] [DEBUG] [LOCAL ] FTHY: Deleted primary data file. [2025-10-20T18:51:57.712419] [INFO ] [MAIN ] FTHY: Starting full primary refresh. [2025-10-20T18:51:57.858814] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-07. [2025-10-20T18:51:58.074075] [INFO ] [REMOTE ] FTHY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:51:58.453707] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T18:51:58.651383] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-06. [2025-10-20T18:51:59.451367] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-01. [2025-10-20T18:51:59.667465] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:51:59.731177] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T18:52:00.251442] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-07. [2025-10-20T18:52:00.364877] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T18:52:01.051546] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-08. [2025-10-20T18:52:01.856990] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-02. [2025-10-20T18:52:01.879201] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T18:52:02.651432] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-08. [2025-10-20T18:52:02.674224] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T18:52:02.855842] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T18:52:03.451440] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-07. [2025-10-20T18:52:03.635105] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T18:52:04.183866] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:52:04.251366] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-08. [2025-10-20T18:52:05.044615] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T18:52:05.051330] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-09. [2025-10-20T18:52:05.793313] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:52:05.851464] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-03. [2025-10-20T18:52:06.599112] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T18:52:06.651424] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-09. [2025-10-20T18:52:07.451427] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-09. [2025-10-20T18:52:08.068970] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:52:08.251341] [INFO ] [REMOTE ] FT: Fetching primary data for 2025-10. [2025-10-20T18:52:09.051361] [INFO ] [REMOTE ] FSS: Fetching primary data for 2025-10. [2025-10-20T18:52:09.277486] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:52:09.293514] [INFO ] [REMOTE ] FT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T18:52:09.295565] [INFO ] [REMOTE ] FT: Full fetch complete (9291 records). [2025-10-20T18:52:09.295638] [DEBUG] [VALIDATOR ] FT: Validating primary data. [2025-10-20T18:52:09.684042] [DEBUG] [VALIDATOR ] FT: All expected trading days are present. [2025-10-20T18:52:09.684119] [DEBUG] [VALIDATOR ] FT: Primary data validation passed. [2025-10-20T18:52:09.684354] [INFO ] [MAIN ] FT: Fetched primary data is valid. [2025-10-20T18:52:09.684392] [DEBUG] [LOCAL ] FT: Saving primary data. [2025-10-20T18:52:09.684857] [DEBUG] [LOCAL ] FT: Trimmed 9291 records to 6734 (keeping 200 days of data) [2025-10-20T18:52:09.722807] [DEBUG] [LOCAL ] FT: Primary data saved (6734 records). [2025-10-20T18:52:09.722910] [INFO ] [MAIN ] FT: Full history saved. [2025-10-20T18:52:09.722982] [INFO ] [MAIN ] Update Progress: 874/2367 symbols completed. FT took 22.8s. Estimated remaining: 3h 11m 28s. Rate: 7.8 symbols/min [2025-10-20T18:52:09.861671] [INFO ] [REMOTE ] FSLY: Fetching primary data for 2025-10. [2025-10-20T18:52:09.894226] [INFO ] [MAIN ] FTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:09.944560] [DEBUG] [LOCAL ] FTI: Loaded 54070 existing records from primary storage. [2025-10-20T18:52:09.944921] [INFO ] [MAIN ] FTI: Loaded 54070 existing records. [2025-10-20T18:52:10.132505] [WARN ] [MAIN ] FTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:10.132794] [INFO ] [MAIN ] FTI: Wiping existing primary data before full refresh. [2025-10-20T18:52:10.132861] [INFO ] [LOCAL ] FTI: Starting deletion of primary data. [2025-10-20T18:52:10.133215] [DEBUG] [LOCAL ] FTI: Deleted primary data file. [2025-10-20T18:52:10.133284] [INFO ] [MAIN ] FTI: Starting full primary refresh. [2025-10-20T18:52:10.500438] [INFO ] [REMOTE ] FTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:10.509513] [INFO ] [REMOTE ] FSS: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:52:10.514287] [INFO ] [REMOTE ] FSS: Full fetch complete (48636 records). [2025-10-20T18:52:10.514404] [DEBUG] [VALIDATOR ] FSS: Validating primary data. [2025-10-20T18:52:10.656488] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-01. [2025-10-20T18:52:10.903445] [DEBUG] [VALIDATOR ] FSS: All expected trading days are present. [2025-10-20T18:52:10.903518] [DEBUG] [VALIDATOR ] FSS: Primary data validation passed. [2025-10-20T18:52:10.904227] [INFO ] [MAIN ] FSS: Fetched primary data is valid. [2025-10-20T18:52:10.904289] [DEBUG] [LOCAL ] FSS: Saving primary data. [2025-10-20T18:52:10.905223] [DEBUG] [LOCAL ] FSS: Trimmed 48636 records to 33463 (keeping 200 days of data) [2025-10-20T18:52:11.104335] [DEBUG] [LOCAL ] FSS: Primary data saved (33463 records). [2025-10-20T18:52:11.104459] [INFO ] [MAIN ] FSS: Full history saved. [2025-10-20T18:52:11.104533] [INFO ] [MAIN ] Update Progress: 875/2367 symbols completed. FSS took 31.0s. Estimated remaining: 3h 11m 10s. Rate: 7.8 symbols/min [2025-10-20T18:52:11.270919] [INFO ] [MAIN ] FTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:11.303232] [DEBUG] [LOCAL ] FTK: Loaded 33277 existing records from primary storage. [2025-10-20T18:52:11.303495] [INFO ] [MAIN ] FTK: Loaded 33277 existing records. [2025-10-20T18:52:11.456782] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-04. [2025-10-20T18:52:11.479841] [WARN ] [MAIN ] FTK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:11.480877] [INFO ] [MAIN ] FTK: Wiping existing primary data before full refresh. [2025-10-20T18:52:11.480927] [INFO ] [LOCAL ] FTK: Starting deletion of primary data. [2025-10-20T18:52:11.481086] [DEBUG] [LOCAL ] FTK: Deleted primary data file. [2025-10-20T18:52:11.481125] [INFO ] [MAIN ] FTK: Starting full primary refresh. [2025-10-20T18:52:11.850274] [INFO ] [REMOTE ] FTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:11.871345] [INFO ] [REMOTE ] FSLY: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T18:52:11.879404] [INFO ] [REMOTE ] FSLY: Full fetch complete (74955 records). [2025-10-20T18:52:11.879671] [DEBUG] [VALIDATOR ] FSLY: Validating primary data. [2025-10-20T18:52:12.265453] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T18:52:12.284275] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-01. [2025-10-20T18:52:12.319018] [DEBUG] [VALIDATOR ] FSLY: All expected trading days are present. [2025-10-20T18:52:12.319123] [DEBUG] [VALIDATOR ] FSLY: Primary data validation passed. [2025-10-20T18:52:12.320067] [INFO ] [MAIN ] FSLY: Fetched primary data is valid. [2025-10-20T18:52:12.320131] [DEBUG] [LOCAL ] FSLY: Saving primary data. [2025-10-20T18:52:12.321512] [DEBUG] [LOCAL ] FSLY: Trimmed 74955 records to 51345 (keeping 200 days of data) [2025-10-20T18:52:12.527344] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T18:52:12.625193] [DEBUG] [LOCAL ] FSLY: Primary data saved (51345 records). [2025-10-20T18:52:12.625326] [INFO ] [MAIN ] FSLY: Full history saved. [2025-10-20T18:52:12.625401] [INFO ] [MAIN ] Update Progress: 876/2367 symbols completed. FSLY took 44.2s. Estimated remaining: 3h 10m 52s. Rate: 7.8 symbols/min [2025-10-20T18:52:12.791287] [INFO ] [MAIN ] FTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:12.828872] [DEBUG] [LOCAL ] FTS: Loaded 43161 existing records from primary storage. [2025-10-20T18:52:12.829222] [INFO ] [MAIN ] FTS: Loaded 43161 existing records. [2025-10-20T18:52:13.005026] [WARN ] [MAIN ] FTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:13.005241] [INFO ] [MAIN ] FTS: Wiping existing primary data before full refresh. [2025-10-20T18:52:13.005286] [INFO ] [LOCAL ] FTS: Starting deletion of primary data. [2025-10-20T18:52:13.005711] [DEBUG] [LOCAL ] FTS: Deleted primary data file. [2025-10-20T18:52:13.005767] [INFO ] [MAIN ] FTS: Starting full primary refresh. [2025-10-20T18:52:13.058825] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-02. [2025-10-20T18:52:13.363203] [INFO ] [REMOTE ] FTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:13.426113] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:52:13.851408] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-01. [2025-10-20T18:52:14.651399] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-02. [2025-10-20T18:52:14.688182] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:52:15.345173] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T18:52:15.451380] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-02. [2025-10-20T18:52:15.706292] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:52:16.251426] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-03. [2025-10-20T18:52:16.995399] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:52:17.051449] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-05. [2025-10-20T18:52:17.851353] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-03. [2025-10-20T18:52:17.884444] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:52:17.958967] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:52:18.651358] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-04. [2025-10-20T18:52:19.254805] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T18:52:19.451365] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-06. [2025-10-20T18:52:20.240171] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T18:52:20.251318] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-07. [2025-10-20T18:52:20.423531] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T18:52:21.039825] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T18:52:21.051331] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-08. [2025-10-20T18:52:21.832287] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:52:21.851343] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-04. [2025-10-20T18:52:22.651388] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-03. [2025-10-20T18:52:23.368981] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T18:52:23.451334] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-05. [2025-10-20T18:52:23.560257] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T18:52:24.251406] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-04. [2025-10-20T18:52:24.926132] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T18:52:25.051363] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-09. [2025-10-20T18:52:25.432171] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T18:52:25.508543] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:52:25.851353] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-05. [2025-10-20T18:52:26.651429] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-06. [2025-10-20T18:52:26.954511] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T18:52:27.451375] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-06. [2025-10-20T18:52:28.224947] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T18:52:28.251341] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-07. [2025-10-20T18:52:28.928284] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:52:29.051337] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-07. [2025-10-20T18:52:29.851345] [INFO ] [REMOTE ] FTHY: Fetching primary data for 2025-10. [2025-10-20T18:52:29.889987] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T18:52:30.413072] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T18:52:30.530169] [INFO ] [REMOTE ] FTHY: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T18:52:30.532713] [INFO ] [REMOTE ] FTHY: Full fetch complete (17339 records). [2025-10-20T18:52:30.532784] [DEBUG] [VALIDATOR ] FTHY: Validating primary data. [2025-10-20T18:52:30.656454] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-08. [2025-10-20T18:52:30.897209] [DEBUG] [VALIDATOR ] FTHY: All expected trading days are present. [2025-10-20T18:52:30.897284] [DEBUG] [VALIDATOR ] FTHY: Primary data validation passed. [2025-10-20T18:52:30.897705] [INFO ] [MAIN ] FTHY: Fetched primary data is valid. [2025-10-20T18:52:30.897765] [DEBUG] [LOCAL ] FTHY: Saving primary data. [2025-10-20T18:52:30.898284] [DEBUG] [LOCAL ] FTHY: Trimmed 17339 records to 12520 (keeping 200 days of data) [2025-10-20T18:52:30.973804] [DEBUG] [LOCAL ] FTHY: Primary data saved (12520 records). [2025-10-20T18:52:30.973942] [INFO ] [MAIN ] FTHY: Full history saved. [2025-10-20T18:52:30.974022] [INFO ] [MAIN ] Update Progress: 877/2367 symbols completed. FTHY took 33.6s. Estimated remaining: 3h 11m 2s. Rate: 7.8 symbols/min [2025-10-20T18:52:31.451428] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-08. [2025-10-20T18:52:32.247841] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:52:32.256134] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-05. [2025-10-20T18:52:32.799307] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T18:52:33.051420] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-09. [2025-10-20T18:52:33.851375] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-09. [2025-10-20T18:52:34.095716] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T18:52:34.325381] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T18:52:34.651424] [INFO ] [REMOTE ] FTK: Fetching primary data for 2025-10. [2025-10-20T18:52:35.341284] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T18:52:35.451356] [INFO ] [REMOTE ] FTS: Fetching primary data for 2025-10. [2025-10-20T18:52:35.761559] [INFO ] [REMOTE ] FTK: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T18:52:35.768697] [INFO ] [REMOTE ] FTK: Full fetch complete (43913 records). [2025-10-20T18:52:35.768843] [DEBUG] [VALIDATOR ] FTK: Validating primary data. [2025-10-20T18:52:36.149439] [DEBUG] [VALIDATOR ] FTK: All expected trading days are present. [2025-10-20T18:52:36.149517] [DEBUG] [VALIDATOR ] FTK: Primary data validation passed. [2025-10-20T18:52:36.150202] [INFO ] [MAIN ] FTK: Fetched primary data is valid. [2025-10-20T18:52:36.150264] [DEBUG] [LOCAL ] FTK: Saving primary data. [2025-10-20T18:52:36.151194] [DEBUG] [LOCAL ] FTK: Trimmed 43913 records to 32881 (keeping 200 days of data) [2025-10-20T18:52:36.253692] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-06. [2025-10-20T18:52:36.338080] [DEBUG] [LOCAL ] FTK: Primary data saved (32881 records). [2025-10-20T18:52:36.338219] [INFO ] [MAIN ] FTK: Full history saved. [2025-10-20T18:52:36.338297] [INFO ] [MAIN ] Update Progress: 878/2367 symbols completed. FTK took 25.2s. Estimated remaining: 3h 10m 51s. Rate: 7.8 symbols/min [2025-10-20T18:52:37.122948] [INFO ] [REMOTE ] FTS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T18:52:37.129346] [INFO ] [REMOTE ] FTS: Full fetch complete (60709 records). [2025-10-20T18:52:37.129462] [DEBUG] [VALIDATOR ] FTS: Validating primary data. [2025-10-20T18:52:37.511841] [DEBUG] [VALIDATOR ] FTS: All expected trading days are present. [2025-10-20T18:52:37.511943] [DEBUG] [VALIDATOR ] FTS: Primary data validation passed. [2025-10-20T18:52:37.512747] [INFO ] [MAIN ] FTS: Fetched primary data is valid. [2025-10-20T18:52:37.512808] [DEBUG] [LOCAL ] FTS: Saving primary data. [2025-10-20T18:52:37.513967] [DEBUG] [LOCAL ] FTS: Trimmed 60709 records to 42558 (keeping 200 days of data) [2025-10-20T18:52:37.775336] [DEBUG] [LOCAL ] FTS: Primary data saved (42558 records). [2025-10-20T18:52:37.775567] [INFO ] [MAIN ] FTS: Full history saved. [2025-10-20T18:52:37.775672] [INFO ] [MAIN ] Update Progress: 879/2367 symbols completed. FTS took 25.2s. Estimated remaining: 3h 10m 32s. Rate: 7.8 symbols/min [2025-10-20T18:52:38.068429] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:52:38.079463] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-07. [2025-10-20T18:52:40.035459] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T18:52:40.049349] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-08. [2025-10-20T18:52:41.908492] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T18:52:41.919946] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-09. [2025-10-20T18:52:43.517759] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:52:43.529189] [INFO ] [REMOTE ] FTI: Fetching primary data for 2025-10. [2025-10-20T18:52:45.240402] [INFO ] [REMOTE ] FTI: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T18:52:45.248250] [INFO ] [REMOTE ] FTI: Full fetch complete (77329 records). [2025-10-20T18:52:45.248398] [DEBUG] [VALIDATOR ] FTI: Validating primary data. [2025-10-20T18:52:45.640420] [DEBUG] [VALIDATOR ] FTI: All expected trading days are present. [2025-10-20T18:52:45.640525] [DEBUG] [VALIDATOR ] FTI: Primary data validation passed. [2025-10-20T18:52:45.641576] [INFO ] [MAIN ] FTI: Fetched primary data is valid. [2025-10-20T18:52:45.641668] [DEBUG] [LOCAL ] FTI: Saving primary data. [2025-10-20T18:52:45.643236] [DEBUG] [LOCAL ] FTI: Trimmed 77329 records to 53290 (keeping 200 days of data) [2025-10-20T18:52:45.959686] [DEBUG] [LOCAL ] FTI: Primary data saved (53290 records). [2025-10-20T18:52:45.959836] [INFO ] [MAIN ] FTI: Full history saved. [2025-10-20T18:52:45.960349] [INFO ] [MAIN ] Update Progress: 880/2367 symbols completed. FTI took 36.2s. Estimated remaining: 3h 10m 25s. Rate: 7.8 symbols/min [2025-10-20T18:52:46.629431] [INFO ] [MAIN ] FTWO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.693560] [INFO ] [MAIN ] FUL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.709087] [INFO ] [MAIN ] FUBO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.735321] [INFO ] [MAIN ] FTV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:46.739925] [DEBUG] [LOCAL ] FTWO: Loaded 1965 existing records from primary storage. [2025-10-20T18:52:46.740397] [INFO ] [MAIN ] FTWO: Loaded 1965 existing records. [2025-10-20T18:52:46.983423] [WARN ] [MAIN ] FTWO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:46.984072] [INFO ] [MAIN ] FTWO: Wiping existing primary data before full refresh. [2025-10-20T18:52:46.984124] [INFO ] [LOCAL ] FTWO: Starting deletion of primary data. [2025-10-20T18:52:46.984287] [DEBUG] [LOCAL ] FTWO: Deleted primary data file. [2025-10-20T18:52:46.984347] [INFO ] [MAIN ] FTWO: Starting full primary refresh. [2025-10-20T18:52:47.121028] [DEBUG] [LOCAL ] FUL: Loaded 32752 existing records from primary storage. [2025-10-20T18:52:47.126771] [INFO ] [MAIN ] FUL: Loaded 32752 existing records. [2025-10-20T18:52:47.127503] [DEBUG] [LOCAL ] FUBO: Loaded 54497 existing records from primary storage. [2025-10-20T18:52:47.167967] [DEBUG] [LOCAL ] FTV: Loaded 53827 existing records from primary storage. [2025-10-20T18:52:47.173335] [INFO ] [MAIN ] FUBO: Loaded 54497 existing records. [2025-10-20T18:52:47.179149] [INFO ] [MAIN ] FTV: Loaded 53827 existing records. [2025-10-20T18:52:47.807783] [WARN ] [MAIN ] FUL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:47.864625] [INFO ] [MAIN ] FUL: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.025211] [INFO ] [LOCAL ] FUL: Starting deletion of primary data. [2025-10-20T18:52:48.008280] [WARN ] [MAIN ] FUBO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:48.004435] [WARN ] [MAIN ] FTV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:48.072021] [INFO ] [MAIN ] FUBO: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.071868] [INFO ] [MAIN ] FTV: Wiping existing primary data before full refresh. [2025-10-20T18:52:48.072185] [INFO ] [LOCAL ] FTV: Starting deletion of primary data. [2025-10-20T18:52:48.071546] [INFO ] [REMOTE ] FTWO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:48.072287] [INFO ] [LOCAL ] FUBO: Starting deletion of primary data. [2025-10-20T18:52:48.072488] [DEBUG] [LOCAL ] FUL: Deleted primary data file. [2025-10-20T18:52:48.073137] [DEBUG] [LOCAL ] FUBO: Deleted primary data file. [2025-10-20T18:52:48.073005] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-01. [2025-10-20T18:52:48.073376] [INFO ] [MAIN ] FUL: Starting full primary refresh. [2025-10-20T18:52:48.073421] [INFO ] [MAIN ] FUBO: Starting full primary refresh. [2025-10-20T18:52:48.073469] [DEBUG] [LOCAL ] FTV: Deleted primary data file. [2025-10-20T18:52:48.104194] [INFO ] [MAIN ] FTV: Starting full primary refresh. [2025-10-20T18:52:48.961287] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T18:52:48.996707] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-02. [2025-10-20T18:52:49.170854] [INFO ] [REMOTE ] FUBO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.190861] [INFO ] [REMOTE ] FUL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.201566] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-01. [2025-10-20T18:52:49.202208] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-01. [2025-10-20T18:52:49.288296] [INFO ] [REMOTE ] FTV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:49.288698] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-01. [2025-10-20T18:52:49.648628] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T18:52:49.650275] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-03. [2025-10-20T18:52:50.092071] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-10-20T18:52:50.093309] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-04. [2025-10-20T18:52:50.526904] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T18:52:50.528875] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:52:50.531147] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-05. [2025-10-20T18:52:50.539963] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-02. [2025-10-20T18:52:50.922729] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T18:52:50.933246] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-02. [2025-10-20T18:52:50.947236] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T18:52:50.948393] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-06. [2025-10-20T18:52:51.456352] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:52:51.466492] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-03. [2025-10-20T18:52:51.636460] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-10-20T18:52:51.649770] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T18:52:51.651264] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-02. [2025-10-20T18:52:51.651806] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-07. [2025-10-20T18:52:52.090455] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T18:52:52.091883] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-08. [2025-10-20T18:52:52.555441] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T18:52:52.556966] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-09. [2025-10-20T18:52:52.607569] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:52:52.868734] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-09 (0.31 seconds). [2025-10-20T18:52:52.988717] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:52:53.051456] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-03. [2025-10-20T18:52:53.412292] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T18:52:53.851366] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-03. [2025-10-20T18:52:54.631210] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:52:54.651351] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-04. [2025-10-20T18:52:55.451403] [INFO ] [REMOTE ] FTWO: Fetching primary data for 2025-10. [2025-10-20T18:52:56.021672] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T18:52:56.185927] [INFO ] [REMOTE ] FTWO: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T18:52:56.187299] [INFO ] [REMOTE ] FTWO: Full fetch complete (3324 records). [2025-10-20T18:52:56.187539] [DEBUG] [VALIDATOR ] FTWO: Validating primary data. [2025-10-20T18:52:56.258320] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-04. [2025-10-20T18:52:56.323731] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:52:56.626300] [DEBUG] [VALIDATOR ] FTWO: All expected trading days are present. [2025-10-20T18:52:56.626408] [DEBUG] [VALIDATOR ] FTWO: Primary data validation passed. [2025-10-20T18:52:56.626590] [INFO ] [MAIN ] FTWO: Fetched primary data is valid. [2025-10-20T18:52:56.626680] [DEBUG] [LOCAL ] FTWO: Saving primary data. [2025-10-20T18:52:56.627080] [DEBUG] [LOCAL ] FTWO: Trimmed 3324 records to 1955 (keeping 200 days of data) [2025-10-20T18:52:56.639787] [DEBUG] [LOCAL ] FTWO: Primary data saved (1955 records). [2025-10-20T18:52:56.639849] [INFO ] [MAIN ] FTWO: Full history saved. [2025-10-20T18:52:56.639918] [INFO ] [MAIN ] Update Progress: 881/2367 symbols completed. FTWO took 10.7s. Estimated remaining: 3h 10m 23s. Rate: 7.8 symbols/min [2025-10-20T18:52:56.822013] [INFO ] [MAIN ] FUN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:52:56.870906] [DEBUG] [LOCAL ] FUN: Loaded 52679 existing records from primary storage. [2025-10-20T18:52:56.871239] [INFO ] [MAIN ] FUN: Loaded 52679 existing records. [2025-10-20T18:52:57.043476] [WARN ] [MAIN ] FUN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:52:57.043768] [INFO ] [MAIN ] FUN: Wiping existing primary data before full refresh. [2025-10-20T18:52:57.043821] [INFO ] [LOCAL ] FUN: Starting deletion of primary data. [2025-10-20T18:52:57.044015] [DEBUG] [LOCAL ] FUN: Deleted primary data file. [2025-10-20T18:52:57.044079] [INFO ] [MAIN ] FUN: Starting full primary refresh. [2025-10-20T18:52:57.056611] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-04. [2025-10-20T18:52:57.396218] [INFO ] [REMOTE ] FUN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:52:57.851375] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-01. [2025-10-20T18:52:58.493078] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T18:52:58.568581] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T18:52:58.651373] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-05. [2025-10-20T18:52:58.989751] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T18:52:59.451382] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-05. [2025-10-20T18:53:00.152267] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T18:53:00.251416] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-02. [2025-10-20T18:53:01.051445] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-06. [2025-10-20T18:53:01.852791] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-05. [2025-10-20T18:53:01.859124] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T18:53:01.905027] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T18:53:02.606129] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T18:53:02.651369] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-07. [2025-10-20T18:53:03.240381] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T18:53:03.451435] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-06. [2025-10-20T18:53:04.251348] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-03. [2025-10-20T18:53:04.440377] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T18:53:05.051354] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-08. [2025-10-20T18:53:05.536155] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-10-20T18:53:05.779930] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T18:53:05.851353] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-06. [2025-10-20T18:53:06.651346] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-07. [2025-10-20T18:53:06.868071] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T18:53:07.113573] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:53:07.451389] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-07. [2025-10-20T18:53:08.251341] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-04. [2025-10-20T18:53:08.691842] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T18:53:08.724102] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T18:53:09.051362] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-09. [2025-10-20T18:53:09.851358] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-08. [2025-10-20T18:53:09.924297] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T18:53:10.651349] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-08. [2025-10-20T18:53:10.814973] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T18:53:10.937423] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T18:53:11.451399] [INFO ] [REMOTE ] FTV: Fetching primary data for 2025-10. [2025-10-20T18:53:12.251374] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-09. [2025-10-20T18:53:12.981289] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-08 (2.33 seconds). [2025-10-20T18:53:13.051421] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-09. [2025-10-20T18:53:13.182180] [INFO ] [REMOTE ] FTV: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:53:13.190304] [INFO ] [REMOTE ] FTV: Full fetch complete (76640 records). [2025-10-20T18:53:13.190505] [DEBUG] [VALIDATOR ] FTV: Validating primary data. [2025-10-20T18:53:13.506023] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T18:53:13.623765] [DEBUG] [VALIDATOR ] FTV: All expected trading days are present. [2025-10-20T18:53:13.623859] [DEBUG] [VALIDATOR ] FTV: Primary data validation passed. [2025-10-20T18:53:13.624774] [INFO ] [MAIN ] FTV: Fetched primary data is valid. [2025-10-20T18:53:13.624848] [DEBUG] [LOCAL ] FTV: Saving primary data. [2025-10-20T18:53:13.626303] [DEBUG] [LOCAL ] FTV: Trimmed 76640 records to 53059 (keeping 200 days of data) [2025-10-20T18:53:13.857869] [INFO ] [REMOTE ] FUL: Fetching primary data for 2025-10. [2025-10-20T18:53:13.947034] [DEBUG] [LOCAL ] FTV: Primary data saved (53059 records). [2025-10-20T18:53:13.947177] [INFO ] [MAIN ] FTV: Full history saved. [2025-10-20T18:53:13.947257] [INFO ] [MAIN ] Update Progress: 882/2367 symbols completed. FTV took 28.0s. Estimated remaining: 3h 10m 31s. Rate: 7.8 symbols/min [2025-10-20T18:53:14.142829] [INFO ] [MAIN ] FVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:14.168682] [DEBUG] [LOCAL ] FVR: Loaded 21415 existing records from primary storage. [2025-10-20T18:53:14.168941] [INFO ] [MAIN ] FVR: Loaded 21415 existing records. [2025-10-20T18:53:14.352546] [WARN ] [MAIN ] FVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:14.352799] [INFO ] [MAIN ] FVR: Wiping existing primary data before full refresh. [2025-10-20T18:53:14.352850] [INFO ] [LOCAL ] FVR: Starting deletion of primary data. [2025-10-20T18:53:14.353053] [DEBUG] [LOCAL ] FVR: Deleted primary data file. [2025-10-20T18:53:14.353095] [INFO ] [MAIN ] FVR: Starting full primary refresh. [2025-10-20T18:53:14.656574] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-05. [2025-10-20T18:53:14.722139] [INFO ] [REMOTE ] FVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:15.076308] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T18:53:15.170739] [INFO ] [REMOTE ] FUL: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T18:53:15.175986] [INFO ] [REMOTE ] FUL: Full fetch complete (47786 records). [2025-10-20T18:53:15.176162] [DEBUG] [VALIDATOR ] FUL: Validating primary data. [2025-10-20T18:53:15.456577] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-01. [2025-10-20T18:53:15.605192] [DEBUG] [VALIDATOR ] FUL: All expected trading days are present. [2025-10-20T18:53:15.605287] [DEBUG] [VALIDATOR ] FUL: Primary data validation passed. [2025-10-20T18:53:15.606033] [INFO ] [MAIN ] FUL: Fetched primary data is valid. [2025-10-20T18:53:15.606101] [DEBUG] [LOCAL ] FUL: Saving primary data. [2025-10-20T18:53:15.608122] [DEBUG] [LOCAL ] FUL: Trimmed 47786 records to 32097 (keeping 200 days of data) [2025-10-20T18:53:15.827297] [DEBUG] [LOCAL ] FUL: Primary data saved (32097 records). [2025-10-20T18:53:15.827516] [INFO ] [MAIN ] FUL: Full history saved. [2025-10-20T18:53:15.827623] [INFO ] [MAIN ] Update Progress: 883/2367 symbols completed. FUL took 29.9s. Estimated remaining: 3h 10m 14s. Rate: 7.8 symbols/min [2025-10-20T18:53:16.019499] [INFO ] [MAIN ] FVRR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:16.067208] [DEBUG] [LOCAL ] FVRR: Loaded 41187 existing records from primary storage. [2025-10-20T18:53:16.067696] [INFO ] [MAIN ] FVRR: Loaded 41187 existing records. [2025-10-20T18:53:16.256877] [INFO ] [REMOTE ] FUBO: Fetching primary data for 2025-10. [2025-10-20T18:53:16.261372] [WARN ] [MAIN ] FVRR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:16.262823] [INFO ] [MAIN ] FVRR: Wiping existing primary data before full refresh. [2025-10-20T18:53:16.262880] [INFO ] [LOCAL ] FVRR: Starting deletion of primary data. [2025-10-20T18:53:16.263399] [DEBUG] [LOCAL ] FVRR: Deleted primary data file. [2025-10-20T18:53:16.263552] [INFO ] [MAIN ] FVRR: Starting full primary refresh. [2025-10-20T18:53:16.693189] [INFO ] [REMOTE ] FVRR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:16.694592] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T18:53:16.732173] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T18:53:17.051372] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-02. [2025-10-20T18:53:17.851433] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-06. [2025-10-20T18:53:17.967143] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:53:18.400719] [INFO ] [REMOTE ] FUBO: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T18:53:18.408541] [INFO ] [REMOTE ] FUBO: Full fetch complete (77867 records). [2025-10-20T18:53:18.408720] [DEBUG] [VALIDATOR ] FUBO: Validating primary data. [2025-10-20T18:53:18.656535] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-03. [2025-10-20T18:53:18.805294] [DEBUG] [VALIDATOR ] FUBO: All expected trading days are present. [2025-10-20T18:53:18.805376] [DEBUG] [VALIDATOR ] FUBO: Primary data validation passed. [2025-10-20T18:53:18.806301] [INFO ] [MAIN ] FUBO: Fetched primary data is valid. [2025-10-20T18:53:18.806366] [DEBUG] [LOCAL ] FUBO: Saving primary data. [2025-10-20T18:53:18.807814] [DEBUG] [LOCAL ] FUBO: Trimmed 77867 records to 53718 (keeping 200 days of data) [2025-10-20T18:53:19.100944] [DEBUG] [LOCAL ] FUBO: Primary data saved (53718 records). [2025-10-20T18:53:19.101065] [INFO ] [MAIN ] FUBO: Full history saved. [2025-10-20T18:53:19.101141] [INFO ] [MAIN ] Update Progress: 884/2367 symbols completed. FUBO took 33.1s. Estimated remaining: 3h 9m 59s. Rate: 7.8 symbols/min [2025-10-20T18:53:19.272724] [INFO ] [MAIN ] FXED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:19.275455] [DEBUG] [LOCAL ] FXED: Loaded 1259 existing records from primary storage. [2025-10-20T18:53:19.275592] [INFO ] [MAIN ] FXED: Loaded 1259 existing records. [2025-10-20T18:53:19.453159] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-01. [2025-10-20T18:53:19.476738] [WARN ] [MAIN ] FXED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:19.478011] [INFO ] [MAIN ] FXED: Wiping existing primary data before full refresh. [2025-10-20T18:53:19.478463] [INFO ] [LOCAL ] FXED: Starting deletion of primary data. [2025-10-20T18:53:19.478860] [DEBUG] [LOCAL ] FXED: Deleted primary data file. [2025-10-20T18:53:19.479093] [INFO ] [MAIN ] FXED: Starting full primary refresh. [2025-10-20T18:53:19.791607] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T18:53:19.818435] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T18:53:19.920675] [INFO ] [REMOTE ] FXED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:20.251382] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-07. [2025-10-20T18:53:21.051413] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-01. [2025-10-20T18:53:21.091305] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T18:53:21.482486] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-01 (0.43 seconds). [2025-10-20T18:53:21.851386] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-02. [2025-10-20T18:53:22.032461] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T18:53:22.105320] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-02 (0.25 seconds). [2025-10-20T18:53:22.651369] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-03. [2025-10-20T18:53:22.998057] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T18:53:23.451373] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-04. [2025-10-20T18:53:24.251413] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-02. [2025-10-20T18:53:24.505193] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T18:53:25.051423] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-05. [2025-10-20T18:53:25.813264] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T18:53:25.851339] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-04. [2025-10-20T18:53:26.107208] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T18:53:26.268029] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T18:53:26.651386] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-03. [2025-10-20T18:53:27.451372] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-08. [2025-10-20T18:53:28.148074] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:53:28.251382] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-05. [2025-10-20T18:53:28.528778] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-10-20T18:53:29.051384] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-06. [2025-10-20T18:53:29.182626] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:53:29.401342] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T18:53:29.851406] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-04. [2025-10-20T18:53:30.651379] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-07. [2025-10-20T18:53:31.129591] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T18:53:31.456542] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-09. [2025-10-20T18:53:31.477178] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:53:32.251417] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-05. [2025-10-20T18:53:33.038925] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T18:53:33.051338] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-08. [2025-10-20T18:53:33.489689] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T18:53:33.727925] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:53:33.851371] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-06. [2025-10-20T18:53:34.651394] [INFO ] [REMOTE ] FUN: Fetching primary data for 2025-10. [2025-10-20T18:53:35.224340] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T18:53:35.451395] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-06. [2025-10-20T18:53:36.236461] [INFO ] [REMOTE ] FUN: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T18:53:36.244379] [INFO ] [REMOTE ] FUN: Full fetch complete (73439 records). [2025-10-20T18:53:36.244545] [DEBUG] [VALIDATOR ] FUN: Validating primary data. [2025-10-20T18:53:36.256590] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-09. [2025-10-20T18:53:36.676583] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T18:53:36.680380] [DEBUG] [VALIDATOR ] FUN: All expected trading days are present. [2025-10-20T18:53:36.686803] [DEBUG] [VALIDATOR ] FUN: Primary data validation passed. [2025-10-20T18:53:36.688008] [INFO ] [MAIN ] FUN: Fetched primary data is valid. [2025-10-20T18:53:36.688173] [DEBUG] [LOCAL ] FUN: Saving primary data. [2025-10-20T18:53:36.690435] [DEBUG] [LOCAL ] FUN: Trimmed 73439 records to 52012 (keeping 200 days of data) [2025-10-20T18:53:36.781939] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:53:36.997147] [DEBUG] [LOCAL ] FUN: Primary data saved (52012 records). [2025-10-20T18:53:36.997331] [INFO ] [MAIN ] FUN: Full history saved. [2025-10-20T18:53:36.998013] [INFO ] [MAIN ] Update Progress: 885/2367 symbols completed. FUN took 40.4s. Estimated remaining: 3h 10m 8s. Rate: 7.8 symbols/min [2025-10-20T18:53:37.060895] [INFO ] [REMOTE ] FXED: Fetching primary data for 2025-10. [2025-10-20T18:53:37.172056] [INFO ] [MAIN ] G: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:37.221112] [DEBUG] [LOCAL ] G: Loaded 50816 existing records from primary storage. [2025-10-20T18:53:37.221442] [INFO ] [MAIN ] G: Loaded 50816 existing records. [2025-10-20T18:53:37.398909] [WARN ] [MAIN ] G: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:37.399141] [INFO ] [MAIN ] G: Wiping existing primary data before full refresh. [2025-10-20T18:53:37.399191] [INFO ] [LOCAL ] G: Starting deletion of primary data. [2025-10-20T18:53:37.399400] [DEBUG] [LOCAL ] G: Deleted primary data file. [2025-10-20T18:53:37.399461] [INFO ] [MAIN ] G: Starting full primary refresh. [2025-10-20T18:53:37.578355] [INFO ] [REMOTE ] FXED: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T18:53:37.615291] [INFO ] [REMOTE ] FXED: Full fetch complete (1785 records). [2025-10-20T18:53:37.647271] [DEBUG] [VALIDATOR ] FXED: Validating primary data. [2025-10-20T18:53:37.857976] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-07. [2025-10-20T18:53:37.937816] [INFO ] [REMOTE ] G: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:38.177520] [DEBUG] [VALIDATOR ] FXED: All expected trading days are present. [2025-10-20T18:53:38.177601] [DEBUG] [VALIDATOR ] FXED: Primary data validation passed. [2025-10-20T18:53:38.177742] [INFO ] [MAIN ] FXED: Fetched primary data is valid. [2025-10-20T18:53:38.177817] [DEBUG] [LOCAL ] FXED: Saving primary data. [2025-10-20T18:53:38.178171] [DEBUG] [LOCAL ] FXED: Trimmed 1785 records to 1237 (keeping 200 days of data) [2025-10-20T18:53:38.186329] [DEBUG] [LOCAL ] FXED: Primary data saved (1237 records). [2025-10-20T18:53:38.186387] [INFO ] [MAIN ] FXED: Full history saved. [2025-10-20T18:53:38.186454] [INFO ] [MAIN ] Update Progress: 886/2367 symbols completed. FXED took 19.1s. Estimated remaining: 3h 9m 50s. Rate: 7.8 symbols/min [2025-10-20T18:53:38.381181] [INFO ] [MAIN ] GAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:38.409853] [DEBUG] [LOCAL ] GAB: Loaded 28162 existing records from primary storage. [2025-10-20T18:53:38.410148] [INFO ] [MAIN ] GAB: Loaded 28162 existing records. [2025-10-20T18:53:38.613447] [WARN ] [MAIN ] GAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:38.613759] [INFO ] [MAIN ] GAB: Wiping existing primary data before full refresh. [2025-10-20T18:53:38.613814] [INFO ] [LOCAL ] GAB: Starting deletion of primary data. [2025-10-20T18:53:38.614027] [DEBUG] [LOCAL ] GAB: Deleted primary data file. [2025-10-20T18:53:38.614083] [INFO ] [MAIN ] GAB: Starting full primary refresh. [2025-10-20T18:53:38.656489] [INFO ] [REMOTE ] G: Fetching primary data for 2025-01. [2025-10-20T18:53:39.034777] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T18:53:39.055133] [INFO ] [REMOTE ] GAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:39.451391] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-08. [2025-10-20T18:53:40.257249] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-01. [2025-10-20T18:53:40.274795] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T18:53:40.459330] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T18:53:41.051384] [INFO ] [REMOTE ] G: Fetching primary data for 2025-02. [2025-10-20T18:53:41.470961] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T18:53:41.851437] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-07. [2025-10-20T18:53:42.651415] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-02. [2025-10-20T18:53:42.715758] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T18:53:43.448544] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T18:53:43.457135] [INFO ] [REMOTE ] G: Fetching primary data for 2025-03. [2025-10-20T18:53:43.609475] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:53:44.251377] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-03. [2025-10-20T18:53:44.958491] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T18:53:45.051371] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-08. [2025-10-20T18:53:45.093626] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T18:53:45.851409] [INFO ] [REMOTE ] G: Fetching primary data for 2025-04. [2025-10-20T18:53:46.651392] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-09. [2025-10-20T18:53:46.778729] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T18:53:47.451384] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-09. [2025-10-20T18:53:47.513912] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T18:53:47.658689] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T18:53:48.251394] [INFO ] [REMOTE ] G: Fetching primary data for 2025-05. [2025-10-20T18:53:48.875602] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T18:53:49.051367] [INFO ] [REMOTE ] FVRR: Fetching primary data for 2025-10. [2025-10-20T18:53:49.651331] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:53:49.851337] [INFO ] [REMOTE ] G: Fetching primary data for 2025-06. [2025-10-20T18:53:50.546191] [INFO ] [REMOTE ] FVRR: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T18:53:50.555559] [INFO ] [REMOTE ] FVRR: Full fetch complete (59298 records). [2025-10-20T18:53:50.555770] [DEBUG] [VALIDATOR ] FVRR: Validating primary data. [2025-10-20T18:53:50.656634] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-04. [2025-10-20T18:53:50.972985] [DEBUG] [VALIDATOR ] FVRR: All expected trading days are present. [2025-10-20T18:53:50.973083] [DEBUG] [VALIDATOR ] FVRR: Primary data validation passed. [2025-10-20T18:53:50.973932] [INFO ] [MAIN ] FVRR: Fetched primary data is valid. [2025-10-20T18:53:50.973999] [DEBUG] [LOCAL ] FVRR: Saving primary data. [2025-10-20T18:53:50.975288] [DEBUG] [LOCAL ] FVRR: Trimmed 59298 records to 40712 (keeping 200 days of data) [2025-10-20T18:53:51.219273] [DEBUG] [LOCAL ] FVRR: Primary data saved (40712 records). [2025-10-20T18:53:51.219427] [INFO ] [MAIN ] FVRR: Full history saved. [2025-10-20T18:53:51.219941] [INFO ] [MAIN ] Update Progress: 887/2367 symbols completed. FVRR took 35.4s. Estimated remaining: 3h 9m 51s. Rate: 7.8 symbols/min [2025-10-20T18:53:51.400521] [INFO ] [MAIN ] GAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:51.407752] [DEBUG] [LOCAL ] GAM: Loaded 3971 existing records from primary storage. [2025-10-20T18:53:51.407966] [INFO ] [MAIN ] GAM: Loaded 3971 existing records. [2025-10-20T18:53:51.470122] [INFO ] [REMOTE ] FVR: Fetching primary data for 2025-10. [2025-10-20T18:53:51.630255] [WARN ] [MAIN ] GAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:51.648463] [INFO ] [MAIN ] GAM: Wiping existing primary data before full refresh. [2025-10-20T18:53:51.648621] [INFO ] [LOCAL ] GAM: Starting deletion of primary data. [2025-10-20T18:53:51.651880] [DEBUG] [LOCAL ] GAM: Deleted primary data file. [2025-10-20T18:53:51.652179] [INFO ] [MAIN ] GAM: Starting full primary refresh. [2025-10-20T18:53:51.659512] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T18:53:52.043395] [INFO ] [REMOTE ] GAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:52.063590] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T18:53:52.251435] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-01. [2025-10-20T18:53:52.611469] [INFO ] [REMOTE ] FVR: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:53:52.614391] [INFO ] [REMOTE ] FVR: Full fetch complete (29325 records). [2025-10-20T18:53:52.614469] [DEBUG] [VALIDATOR ] FVR: Validating primary data. [2025-10-20T18:53:52.749156] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T18:53:52.985192] [DEBUG] [VALIDATOR ] FVR: All expected trading days are present. [2025-10-20T18:53:52.985264] [DEBUG] [VALIDATOR ] FVR: Primary data validation passed. [2025-10-20T18:53:52.985815] [INFO ] [MAIN ] FVR: Fetched primary data is valid. [2025-10-20T18:53:52.985876] [DEBUG] [LOCAL ] FVR: Saving primary data. [2025-10-20T18:53:52.986510] [DEBUG] [LOCAL ] FVR: Trimmed 29325 records to 20901 (keeping 200 days of data) [2025-10-20T18:53:53.071593] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-05. [2025-10-20T18:53:53.113882] [DEBUG] [LOCAL ] FVR: Primary data saved (20901 records). [2025-10-20T18:53:53.113995] [INFO ] [MAIN ] FVR: Full history saved. [2025-10-20T18:53:53.114066] [INFO ] [MAIN ] Update Progress: 888/2367 symbols completed. FVR took 39.2s. Estimated remaining: 3h 9m 33s. Rate: 7.8 symbols/min [2025-10-20T18:53:53.278851] [INFO ] [MAIN ] GAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:53:53.329178] [DEBUG] [LOCAL ] GAP: Loaded 54433 existing records from primary storage. [2025-10-20T18:53:53.329501] [INFO ] [MAIN ] GAP: Loaded 54433 existing records. [2025-10-20T18:53:53.507018] [WARN ] [MAIN ] GAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:53:53.507320] [INFO ] [MAIN ] GAP: Wiping existing primary data before full refresh. [2025-10-20T18:53:53.507372] [INFO ] [LOCAL ] GAP: Starting deletion of primary data. [2025-10-20T18:53:53.507732] [DEBUG] [LOCAL ] GAP: Deleted primary data file. [2025-10-20T18:53:53.507795] [INFO ] [MAIN ] GAP: Starting full primary refresh. [2025-10-20T18:53:53.853316] [INFO ] [REMOTE ] G: Fetching primary data for 2025-07. [2025-10-20T18:53:53.888951] [INFO ] [REMOTE ] GAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:53:54.102018] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T18:53:54.651386] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-02. [2025-10-20T18:53:55.083818] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:53:55.181226] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T18:53:55.451421] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-06. [2025-10-20T18:53:56.251375] [INFO ] [REMOTE ] G: Fetching primary data for 2025-08. [2025-10-20T18:53:56.763159] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T18:53:57.051375] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-01. [2025-10-20T18:53:57.851384] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-07. [2025-10-20T18:53:57.923291] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T18:53:58.651462] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-03. [2025-10-20T18:53:58.762313] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T18:53:59.083506] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T18:53:59.357510] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T18:53:59.451396] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-08. [2025-10-20T18:54:00.251421] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-04. [2025-10-20T18:54:00.526445] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:54:01.037247] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T18:54:01.051369] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-05. [2025-10-20T18:54:01.624438] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T18:54:01.851384] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-06. [2025-10-20T18:54:02.467619] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T18:54:02.651371] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-07. [2025-10-20T18:54:03.195307] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:54:03.451427] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-02. [2025-10-20T18:54:04.251360] [INFO ] [REMOTE ] G: Fetching primary data for 2025-09. [2025-10-20T18:54:05.051432] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-08. [2025-10-20T18:54:05.083583] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:54:05.851405] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-09. [2025-10-20T18:54:05.908778] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T18:54:05.932523] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T18:54:06.651391] [INFO ] [REMOTE ] G: Fetching primary data for 2025-10. [2025-10-20T18:54:07.155354] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T18:54:07.451389] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-03. [2025-10-20T18:54:08.251457] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-09. [2025-10-20T18:54:08.491886] [INFO ] [REMOTE ] G: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:54:08.500230] [INFO ] [REMOTE ] G: Full fetch complete (72246 records). [2025-10-20T18:54:08.500404] [DEBUG] [VALIDATOR ] G: Validating primary data. [2025-10-20T18:54:08.786963] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-10-20T18:54:08.922597] [DEBUG] [VALIDATOR ] G: All expected trading days are present. [2025-10-20T18:54:08.922730] [DEBUG] [VALIDATOR ] G: Primary data validation passed. [2025-10-20T18:54:08.923638] [INFO ] [MAIN ] G: Fetched primary data is valid. [2025-10-20T18:54:08.923732] [DEBUG] [LOCAL ] G: Saving primary data. [2025-10-20T18:54:08.925156] [DEBUG] [LOCAL ] G: Trimmed 72246 records to 50114 (keeping 200 days of data) [2025-10-20T18:54:09.095985] [INFO ] [REMOTE ] GAB: Fetching primary data for 2025-10. [2025-10-20T18:54:09.226099] [DEBUG] [LOCAL ] G: Primary data saved (50114 records). [2025-10-20T18:54:09.226262] [INFO ] [MAIN ] G: Full history saved. [2025-10-20T18:54:09.227179] [INFO ] [MAIN ] Update Progress: 889/2367 symbols completed. G took 32.2s. Estimated remaining: 3h 9m 40s. Rate: 7.8 symbols/min [2025-10-20T18:54:09.400269] [INFO ] [MAIN ] GATX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:09.420343] [DEBUG] [LOCAL ] GATX: Loaded 18269 existing records from primary storage. [2025-10-20T18:54:09.420674] [INFO ] [MAIN ] GATX: Loaded 18269 existing records. [2025-10-20T18:54:09.588681] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T18:54:09.653461] [WARN ] [MAIN ] GATX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:09.653725] [INFO ] [MAIN ] GATX: Wiping existing primary data before full refresh. [2025-10-20T18:54:09.653798] [INFO ] [LOCAL ] GATX: Starting deletion of primary data. [2025-10-20T18:54:09.654296] [DEBUG] [LOCAL ] GATX: Deleted primary data file. [2025-10-20T18:54:09.654370] [INFO ] [MAIN ] GATX: Starting full primary refresh. [2025-10-20T18:54:09.863084] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-04. [2025-10-20T18:54:10.010844] [INFO ] [REMOTE ] GATX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:10.275758] [INFO ] [REMOTE ] GAB: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:54:10.280049] [INFO ] [REMOTE ] GAB: Full fetch complete (39400 records). [2025-10-20T18:54:10.280135] [DEBUG] [VALIDATOR ] GAB: Validating primary data. [2025-10-20T18:54:10.643049] [DEBUG] [VALIDATOR ] GAB: All expected trading days are present. [2025-10-20T18:54:10.643128] [DEBUG] [VALIDATOR ] GAB: Primary data validation passed. [2025-10-20T18:54:10.643768] [INFO ] [MAIN ] GAB: Fetched primary data is valid. [2025-10-20T18:54:10.643829] [DEBUG] [LOCAL ] GAB: Saving primary data. [2025-10-20T18:54:10.644631] [DEBUG] [LOCAL ] GAB: Trimmed 39400 records to 27744 (keeping 200 days of data) [2025-10-20T18:54:10.695576] [INFO ] [REMOTE ] GAM: Fetching primary data for 2025-10. [2025-10-20T18:54:10.811149] [DEBUG] [LOCAL ] GAB: Primary data saved (27744 records). [2025-10-20T18:54:10.811316] [INFO ] [MAIN ] GAB: Full history saved. [2025-10-20T18:54:10.811412] [INFO ] [MAIN ] Update Progress: 890/2367 symbols completed. GAB took 32.6s. Estimated remaining: 3h 9m 22s. Rate: 7.8 symbols/min [2025-10-20T18:54:10.983109] [INFO ] [MAIN ] GBAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:10.997839] [DEBUG] [LOCAL ] GBAB: Loaded 12974 existing records from primary storage. [2025-10-20T18:54:10.998067] [INFO ] [MAIN ] GBAB: Loaded 12974 existing records. [2025-10-20T18:54:11.173804] [WARN ] [MAIN ] GBAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:11.174002] [INFO ] [MAIN ] GBAB: Wiping existing primary data before full refresh. [2025-10-20T18:54:11.174047] [INFO ] [LOCAL ] GBAB: Starting deletion of primary data. [2025-10-20T18:54:11.174202] [DEBUG] [LOCAL ] GBAB: Deleted primary data file. [2025-10-20T18:54:11.174240] [INFO ] [MAIN ] GBAB: Starting full primary refresh. [2025-10-20T18:54:11.457372] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-01. [2025-10-20T18:54:11.450140] [INFO ] [REMOTE ] GAM: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T18:54:11.466097] [INFO ] [REMOTE ] GAM: Full fetch complete (6524 records). [2025-10-20T18:54:11.466189] [DEBUG] [VALIDATOR ] GAM: Validating primary data. [2025-10-20T18:54:11.652297] [INFO ] [REMOTE ] GBAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:11.926371] [DEBUG] [VALIDATOR ] GAM: All expected trading days are present. [2025-10-20T18:54:11.926538] [DEBUG] [VALIDATOR ] GAM: Primary data validation passed. [2025-10-20T18:54:11.926754] [INFO ] [MAIN ] GAM: Fetched primary data is valid. [2025-10-20T18:54:11.926828] [DEBUG] [LOCAL ] GAM: Saving primary data. [2025-10-20T18:54:11.927398] [DEBUG] [LOCAL ] GAM: Trimmed 6524 records to 3901 (keeping 200 days of data) [2025-10-20T18:54:11.979792] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T18:54:11.978462] [DEBUG] [LOCAL ] GAM: Primary data saved (3901 records). [2025-10-20T18:54:11.990346] [INFO ] [MAIN ] GAM: Full history saved. [2025-10-20T18:54:11.990827] [INFO ] [MAIN ] Update Progress: 891/2367 symbols completed. GAM took 20.8s. Estimated remaining: 3h 9m 3s. Rate: 7.8 symbols/min [2025-10-20T18:54:12.163572] [INFO ] [MAIN ] GBCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:12.199634] [DEBUG] [LOCAL ] GBCI: Loaded 41621 existing records from primary storage. [2025-10-20T18:54:12.199972] [INFO ] [MAIN ] GBCI: Loaded 41621 existing records. [2025-10-20T18:54:12.262024] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-01. [2025-10-20T18:54:12.379796] [WARN ] [MAIN ] GBCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:12.380062] [INFO ] [MAIN ] GBCI: Wiping existing primary data before full refresh. [2025-10-20T18:54:12.380184] [INFO ] [LOCAL ] GBCI: Starting deletion of primary data. [2025-10-20T18:54:12.389394] [DEBUG] [LOCAL ] GBCI: Deleted primary data file. [2025-10-20T18:54:12.389458] [INFO ] [MAIN ] GBCI: Starting full primary refresh. [2025-10-20T18:54:12.389764] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T18:54:12.747014] [INFO ] [REMOTE ] GBCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:13.051398] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-01. [2025-10-20T18:54:13.102365] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T18:54:13.851397] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-02. [2025-10-20T18:54:14.547144] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T18:54:14.651452] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-02. [2025-10-20T18:54:14.758890] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T18:54:15.451380] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-02. [2025-10-20T18:54:15.932790] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T18:54:16.251404] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-03. [2025-10-20T18:54:16.351863] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T18:54:17.051430] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-05. [2025-10-20T18:54:17.717759] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:54:17.851386] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-03. [2025-10-20T18:54:18.651394] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-04. [2025-10-20T18:54:18.860506] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:54:18.986280] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T18:54:19.451416] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-03. [2025-10-20T18:54:20.092458] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T18:54:20.120598] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T18:54:20.251416] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-06. [2025-10-20T18:54:21.051405] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-04. [2025-10-20T18:54:21.851370] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-05. [2025-10-20T18:54:22.227618] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T18:54:22.289684] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T18:54:22.651408] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-07. [2025-10-20T18:54:23.248759] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T18:54:23.451427] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-05. [2025-10-20T18:54:24.251351] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-04. [2025-10-20T18:54:24.459736] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T18:54:24.493225] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T18:54:24.865231] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:54:25.051422] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-05. [2025-10-20T18:54:25.839385] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T18:54:25.851369] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-06. [2025-10-20T18:54:26.651415] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-08. [2025-10-20T18:54:26.851515] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T18:54:27.451425] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-06. [2025-10-20T18:54:28.251403] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-06. [2025-10-20T18:54:28.559105] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T18:54:28.958286] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T18:54:29.051377] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-09. [2025-10-20T18:54:29.380958] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T18:54:29.851378] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-07. [2025-10-20T18:54:30.651374] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-07. [2025-10-20T18:54:30.836868] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T18:54:31.456525] [INFO ] [REMOTE ] GAP: Fetching primary data for 2025-10. [2025-10-20T18:54:31.471219] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:54:31.642888] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T18:54:32.251383] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-08. [2025-10-20T18:54:33.051372] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-07. [2025-10-20T18:54:33.291616] [INFO ] [REMOTE ] GAP: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:54:33.302739] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T18:54:33.307274] [INFO ] [REMOTE ] GAP: Full fetch complete (77812 records). [2025-10-20T18:54:33.307401] [DEBUG] [VALIDATOR ] GAP: Validating primary data. [2025-10-20T18:54:33.743276] [DEBUG] [VALIDATOR ] GAP: All expected trading days are present. [2025-10-20T18:54:33.743364] [DEBUG] [VALIDATOR ] GAP: Primary data validation passed. [2025-10-20T18:54:33.744364] [INFO ] [MAIN ] GAP: Fetched primary data is valid. [2025-10-20T18:54:33.744428] [DEBUG] [LOCAL ] GAP: Saving primary data. [2025-10-20T18:54:33.745933] [DEBUG] [LOCAL ] GAP: Trimmed 77812 records to 53653 (keeping 200 days of data) [2025-10-20T18:54:33.872169] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-08. [2025-10-20T18:54:34.096213] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T18:54:34.105399] [DEBUG] [LOCAL ] GAP: Primary data saved (53653 records). [2025-10-20T18:54:34.109285] [INFO ] [MAIN ] GAP: Full history saved. [2025-10-20T18:54:34.109952] [INFO ] [MAIN ] Update Progress: 892/2367 symbols completed. GAP took 41.0s. Estimated remaining: 3h 9m 20s. Rate: 7.8 symbols/min [2025-10-20T18:54:34.280183] [INFO ] [MAIN ] GBLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:34.280408] [DEBUG] [LOCAL ] GBLI: No existing data file found in primary storage. [2025-10-20T18:54:34.280449] [WARN ] [MAIN ] GBLI: No existing data found, performing full refresh instead. [2025-10-20T18:54:34.280523] [INFO ] [MAIN ] GBLI: Wiping existing primary data before full refresh. [2025-10-20T18:54:34.280558] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-10-20T18:54:34.280597] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-10-20T18:54:34.280627] [INFO ] [MAIN ] GBLI: Starting full primary refresh. [2025-10-20T18:54:34.654977] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-08. [2025-10-20T18:54:34.658864] [INFO ] [REMOTE ] GBLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:35.456587] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-09. [2025-10-20T18:54:35.486482] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T18:54:35.493608] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T18:54:36.251388] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-01. [2025-10-20T18:54:36.523861] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-01 (0.27 seconds). [2025-10-20T18:54:36.542797] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:54:37.051395] [INFO ] [REMOTE ] GATX: Fetching primary data for 2025-10. [2025-10-20T18:54:37.720714] [INFO ] [REMOTE ] GATX: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T18:54:37.723105] [INFO ] [REMOTE ] GATX: Full fetch complete (24864 records). [2025-10-20T18:54:37.723179] [DEBUG] [VALIDATOR ] GATX: Validating primary data. [2025-10-20T18:54:37.865803] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-02. [2025-10-20T18:54:38.093966] [DEBUG] [VALIDATOR ] GATX: All expected trading days are present. [2025-10-20T18:54:38.094045] [DEBUG] [VALIDATOR ] GATX: Primary data validation passed. [2025-10-20T18:54:38.094549] [INFO ] [MAIN ] GATX: Fetched primary data is valid. [2025-10-20T18:54:38.094588] [DEBUG] [LOCAL ] GATX: Saving primary data. [2025-10-20T18:54:38.095246] [DEBUG] [LOCAL ] GATX: Trimmed 24864 records to 18106 (keeping 200 days of data) [2025-10-20T18:54:38.183254] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T18:54:38.233384] [DEBUG] [LOCAL ] GATX: Primary data saved (18106 records). [2025-10-20T18:54:38.233555] [INFO ] [MAIN ] GATX: Full history saved. [2025-10-20T18:54:38.233679] [INFO ] [MAIN ] Update Progress: 893/2367 symbols completed. GATX took 29.0s. Estimated remaining: 3h 9m 6s. Rate: 7.8 symbols/min [2025-10-20T18:54:38.442932] [INFO ] [MAIN ] GBTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:38.488496] [DEBUG] [LOCAL ] GBTG: Loaded 40850 existing records from primary storage. [2025-10-20T18:54:38.489003] [INFO ] [MAIN ] GBTG: Loaded 40850 existing records. [2025-10-20T18:54:38.656516] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-03. [2025-10-20T18:54:38.723249] [WARN ] [MAIN ] GBTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:38.723716] [INFO ] [MAIN ] GBTG: Wiping existing primary data before full refresh. [2025-10-20T18:54:38.723791] [INFO ] [LOCAL ] GBTG: Starting deletion of primary data. [2025-10-20T18:54:38.724114] [DEBUG] [LOCAL ] GBTG: Deleted primary data file. [2025-10-20T18:54:38.724172] [INFO ] [MAIN ] GBTG: Starting full primary refresh. [2025-10-20T18:54:39.125472] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T18:54:39.253001] [INFO ] [REMOTE ] GBTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:39.451382] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-04. [2025-10-20T18:54:39.791939] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T18:54:40.251428] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-05. [2025-10-20T18:54:40.596465] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T18:54:41.051372] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-09. [2025-10-20T18:54:41.851394] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-09. [2025-10-20T18:54:42.395810] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T18:54:42.651346] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-01. [2025-10-20T18:54:42.685523] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T18:54:43.451347] [INFO ] [REMOTE ] GBCI: Fetching primary data for 2025-10. [2025-10-20T18:54:43.906104] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T18:54:44.251373] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-02. [2025-10-20T18:54:45.051368] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-06. [2025-10-20T18:54:45.205132] [INFO ] [REMOTE ] GBCI: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T18:54:45.212963] [INFO ] [REMOTE ] GBCI: Full fetch complete (59309 records). [2025-10-20T18:54:45.213119] [DEBUG] [VALIDATOR ] GBCI: Validating primary data. [2025-10-20T18:54:45.609334] [DEBUG] [VALIDATOR ] GBCI: All expected trading days are present. [2025-10-20T18:54:45.588996] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:54:45.624275] [DEBUG] [VALIDATOR ] GBCI: Primary data validation passed. [2025-10-20T18:54:45.628525] [INFO ] [MAIN ] GBCI: Fetched primary data is valid. [2025-10-20T18:54:45.628612] [DEBUG] [LOCAL ] GBCI: Saving primary data. [2025-10-20T18:54:45.630215] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T18:54:45.637612] [DEBUG] [LOCAL ] GBCI: Trimmed 59309 records to 41215 (keeping 200 days of data) [2025-10-20T18:54:45.851523] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-03. [2025-10-20T18:54:45.890935] [DEBUG] [LOCAL ] GBCI: Primary data saved (41215 records). [2025-10-20T18:54:45.891389] [INFO ] [MAIN ] GBCI: Full history saved. [2025-10-20T18:54:45.892122] [INFO ] [MAIN ] Update Progress: 894/2367 symbols completed. GBCI took 33.9s. Estimated remaining: 3h 8m 58s. Rate: 7.8 symbols/min [2025-10-20T18:54:46.059972] [INFO ] [MAIN ] GBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:46.091210] [DEBUG] [LOCAL ] GBX: Loaded 29290 existing records from primary storage. [2025-10-20T18:54:46.091453] [INFO ] [MAIN ] GBX: Loaded 29290 existing records. [2025-10-20T18:54:46.293279] [WARN ] [MAIN ] GBX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:46.293595] [INFO ] [MAIN ] GBX: Wiping existing primary data before full refresh. [2025-10-20T18:54:46.293661] [INFO ] [LOCAL ] GBX: Starting deletion of primary data. [2025-10-20T18:54:46.294177] [DEBUG] [LOCAL ] GBX: Deleted primary data file. [2025-10-20T18:54:46.294233] [INFO ] [MAIN ] GBX: Starting full primary refresh. [2025-10-20T18:54:46.649580] [INFO ] [REMOTE ] GBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:46.651327] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-01. [2025-10-20T18:54:47.187183] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T18:54:47.451372] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-04. [2025-10-20T18:54:47.609186] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T18:54:48.251363] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-07. [2025-10-20T18:54:48.590355] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T18:54:48.906262] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T18:54:49.051357] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-08. [2025-10-20T18:54:49.390226] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T18:54:49.851366] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-09. [2025-10-20T18:54:50.169461] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-09 (0.32 seconds). [2025-10-20T18:54:50.651369] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-02. [2025-10-20T18:54:51.451381] [INFO ] [REMOTE ] GBLI: Fetching primary data for 2025-10. [2025-10-20T18:54:51.568318] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T18:54:51.910318] [INFO ] [REMOTE ] GBLI: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T18:54:51.911050] [INFO ] [REMOTE ] GBLI: Full fetch complete (1276 records). [2025-10-20T18:54:51.911117] [DEBUG] [VALIDATOR ] GBLI: Validating primary data. [2025-10-20T18:54:52.104726] [WARN ] [VALIDATOR ] GBLI: Validation failed - insufficient historical data. Have 173 unique days, need 200 days [2025-10-20T18:54:52.105020] [FATAL] [MAIN ] GBLI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T18:54:52.105094] [WARN ] [MAIN ] GBLI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T18:54:52.105168] [INFO ] [LOCAL ] GBLI: Starting deletion of primary data. [2025-10-20T18:54:52.105264] [DEBUG] [LOCAL ] GBLI: No primary data file found to delete. [2025-10-20T18:54:52.105321] [INFO ] [LOCAL ] GBLI: Starting deletion of derivative data. [2025-10-20T18:54:52.105365] [DEBUG] [LOCAL ] GBLI: No derivative data file found to delete. [2025-10-20T18:54:52.105421] [INFO ] [MAIN ] Update Progress: 895/2367 symbols completed. GBLI took 18.0s. Estimated remaining: 3h 8m 48s. Rate: 7.8 symbols/min [2025-10-20T18:54:52.256534] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-03. [2025-10-20T18:54:52.303050] [INFO ] [MAIN ] GCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:52.345193] [DEBUG] [LOCAL ] GCI: Loaded 44504 existing records from primary storage. [2025-10-20T18:54:52.345507] [INFO ] [MAIN ] GCI: Loaded 44504 existing records. [2025-10-20T18:54:52.535801] [WARN ] [MAIN ] GCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:52.536039] [INFO ] [MAIN ] GCI: Wiping existing primary data before full refresh. [2025-10-20T18:54:52.536087] [INFO ] [LOCAL ] GCI: Starting deletion of primary data. [2025-10-20T18:54:52.536328] [DEBUG] [LOCAL ] GCI: Deleted primary data file. [2025-10-20T18:54:52.536405] [INFO ] [MAIN ] GCI: Starting full primary refresh. [2025-10-20T18:54:52.887916] [INFO ] [REMOTE ] GCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:53.051355] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-05. [2025-10-20T18:54:53.521638] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T18:54:53.851404] [INFO ] [REMOTE ] GBAB: Fetching primary data for 2025-10. [2025-10-20T18:54:54.585003] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T18:54:54.651400] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-04. [2025-10-20T18:54:54.690030] [INFO ] [REMOTE ] GBAB: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T18:54:54.691992] [INFO ] [REMOTE ] GBAB: Full fetch complete (18819 records). [2025-10-20T18:54:54.692060] [DEBUG] [VALIDATOR ] GBAB: Validating primary data. [2025-10-20T18:54:55.053591] [DEBUG] [VALIDATOR ] GBAB: All expected trading days are present. [2025-10-20T18:54:55.053690] [DEBUG] [VALIDATOR ] GBAB: Primary data validation passed. [2025-10-20T18:54:55.054114] [INFO ] [MAIN ] GBAB: Fetched primary data is valid. [2025-10-20T18:54:55.054154] [DEBUG] [LOCAL ] GBAB: Saving primary data. [2025-10-20T18:54:55.054695] [DEBUG] [LOCAL ] GBAB: Trimmed 18819 records to 12847 (keeping 200 days of data) [2025-10-20T18:54:55.131626] [DEBUG] [LOCAL ] GBAB: Primary data saved (12847 records). [2025-10-20T18:54:55.131785] [INFO ] [MAIN ] GBAB: Full history saved. [2025-10-20T18:54:55.131861] [INFO ] [MAIN ] Update Progress: 896/2367 symbols completed. GBAB took 44.3s. Estimated remaining: 3h 8m 33s. Rate: 7.8 symbols/min [2025-10-20T18:54:55.302510] [INFO ] [MAIN ] GCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:54:55.325245] [DEBUG] [LOCAL ] GCO: Loaded 23845 existing records from primary storage. [2025-10-20T18:54:55.325506] [INFO ] [MAIN ] GCO: Loaded 23845 existing records. [2025-10-20T18:54:55.456479] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-01. [2025-10-20T18:54:55.508227] [WARN ] [MAIN ] GCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:54:55.508429] [INFO ] [MAIN ] GCO: Wiping existing primary data before full refresh. [2025-10-20T18:54:55.508476] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-10-20T18:54:55.508637] [DEBUG] [LOCAL ] GCO: Deleted primary data file. [2025-10-20T18:54:55.508717] [INFO ] [MAIN ] GCO: Starting full primary refresh. [2025-10-20T18:54:55.875438] [INFO ] [REMOTE ] GCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:54:56.006962] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T18:54:56.251381] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-01. [2025-10-20T18:54:56.905465] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T18:54:57.051396] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-05. [2025-10-20T18:54:57.313458] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T18:54:57.851431] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-06. [2025-10-20T18:54:58.228931] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T18:54:58.651393] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-06. [2025-10-20T18:54:59.283864] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T18:54:59.343326] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T18:54:59.451445] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-07. [2025-10-20T18:55:00.251429] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-02. [2025-10-20T18:55:01.051415] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-07. [2025-10-20T18:55:01.417382] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T18:55:01.851403] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-08. [2025-10-20T18:55:02.142114] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T18:55:02.658465] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-02. [2025-10-20T18:55:02.676062] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T18:55:03.320168] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:55:03.451375] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-09. [2025-10-20T18:55:03.781991] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T18:55:04.251450] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-03. [2025-10-20T18:55:04.732455] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T18:55:05.051410] [INFO ] [REMOTE ] GBTG: Fetching primary data for 2025-10. [2025-10-20T18:55:05.547431] [INFO ] [REMOTE ] GCO: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T18:55:05.851449] [INFO ] [REMOTE ] GCO: Fetching primary data for 2025-04. [2025-10-20T18:55:05.973021] [DEBUG] [REMOTE ] GCO: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T18:55:05.973331] [WARN ] [REMOTE ] GCO: No data returned for month 2025-04 [2025-10-20T18:55:05.973420] [FATAL] [REMOTE ] GCO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T18:55:05.973495] [FATAL] [MAIN ] GCO: Failed to fetch primary data. [2025-10-20T18:55:05.973556] [WARN ] [MAIN ] GCO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T18:55:05.973627] [INFO ] [LOCAL ] GCO: Starting deletion of primary data. [2025-10-20T18:55:05.973726] [DEBUG] [LOCAL ] GCO: No primary data file found to delete. [2025-10-20T18:55:05.973770] [INFO ] [LOCAL ] GCO: Starting deletion of derivative data. [2025-10-20T18:55:05.973917] [DEBUG] [LOCAL ] GCO: Deleted derivative data file. [2025-10-20T18:55:05.973999] [INFO ] [MAIN ] Update Progress: 897/2367 symbols completed. GCO took 10.8s. Estimated remaining: 3h 8m 30s. Rate: 7.8 symbols/min [2025-10-20T18:55:06.168110] [INFO ] [MAIN ] GCTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:06.191231] [DEBUG] [LOCAL ] GCTS: Loaded 20642 existing records from primary storage. [2025-10-20T18:55:06.191562] [INFO ] [MAIN ] GCTS: Loaded 20642 existing records. [2025-10-20T18:55:06.434813] [INFO ] [REMOTE ] GBTG: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T18:55:06.439431] [WARN ] [MAIN ] GCTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:06.445596] [INFO ] [MAIN ] GCTS: Wiping existing primary data before full refresh. [2025-10-20T18:55:06.445998] [INFO ] [LOCAL ] GCTS: Starting deletion of primary data. [2025-10-20T18:55:06.446598] [DEBUG] [LOCAL ] GCTS: Deleted primary data file. [2025-10-20T18:55:06.446685] [INFO ] [MAIN ] GCTS: Starting full primary refresh. [2025-10-20T18:55:06.489476] [INFO ] [REMOTE ] GBTG: Full fetch complete (56818 records). [2025-10-20T18:55:06.510244] [DEBUG] [VALIDATOR ] GBTG: Validating primary data. [2025-10-20T18:55:06.661145] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-03. [2025-10-20T18:55:07.118363] [INFO ] [REMOTE ] GCTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:07.207489] [DEBUG] [VALIDATOR ] GBTG: All expected trading days are present. [2025-10-20T18:55:07.207579] [DEBUG] [VALIDATOR ] GBTG: Primary data validation passed. [2025-10-20T18:55:07.208400] [INFO ] [MAIN ] GBTG: Fetched primary data is valid. [2025-10-20T18:55:07.208463] [DEBUG] [LOCAL ] GBTG: Saving primary data. [2025-10-20T18:55:07.209826] [DEBUG] [LOCAL ] GBTG: Trimmed 56818 records to 40310 (keeping 200 days of data) [2025-10-20T18:55:07.435194] [DEBUG] [LOCAL ] GBTG: Primary data saved (40310 records). [2025-10-20T18:55:07.435336] [INFO ] [MAIN ] GBTG: Full history saved. [2025-10-20T18:55:07.435427] [INFO ] [MAIN ] Update Progress: 898/2367 symbols completed. GBTG took 29.2s. Estimated remaining: 3h 8m 12s. Rate: 7.8 symbols/min [2025-10-20T18:55:07.456560] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-01. [2025-10-20T18:55:07.606242] [INFO ] [MAIN ] GCV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:07.616313] [DEBUG] [LOCAL ] GCV: Loaded 7903 existing records from primary storage. [2025-10-20T18:55:07.616536] [INFO ] [MAIN ] GCV: Loaded 7903 existing records. [2025-10-20T18:55:07.792807] [WARN ] [MAIN ] GCV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:07.793013] [INFO ] [MAIN ] GCV: Wiping existing primary data before full refresh. [2025-10-20T18:55:07.793062] [INFO ] [LOCAL ] GCV: Starting deletion of primary data. [2025-10-20T18:55:07.793353] [DEBUG] [LOCAL ] GCV: Deleted primary data file. [2025-10-20T18:55:07.793409] [INFO ] [MAIN ] GCV: Starting full primary refresh. [2025-10-20T18:55:08.153060] [INFO ] [REMOTE ] GCV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:08.245839] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:55:08.255521] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-08. [2025-10-20T18:55:08.497730] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:55:09.051408] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-04. [2025-10-20T18:55:09.474835] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T18:55:09.851400] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-09. [2025-10-20T18:55:10.659836] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-01. [2025-10-20T18:55:10.678811] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T18:55:10.940072] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T18:55:11.260873] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T18:55:11.451386] [INFO ] [REMOTE ] GBX: Fetching primary data for 2025-10. [2025-10-20T18:55:12.251407] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-05. [2025-10-20T18:55:12.710174] [INFO ] [REMOTE ] GBX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:55:12.714450] [INFO ] [REMOTE ] GBX: Full fetch complete (42830 records). [2025-10-20T18:55:12.714593] [DEBUG] [VALIDATOR ] GBX: Validating primary data. [2025-10-20T18:55:13.056538] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-02. [2025-10-20T18:55:13.169469] [DEBUG] [VALIDATOR ] GBX: All expected trading days are present. [2025-10-20T18:55:13.169566] [DEBUG] [VALIDATOR ] GBX: Primary data validation passed. [2025-10-20T18:55:13.170310] [INFO ] [MAIN ] GBX: Fetched primary data is valid. [2025-10-20T18:55:13.170378] [DEBUG] [LOCAL ] GBX: Saving primary data. [2025-10-20T18:55:13.171634] [DEBUG] [LOCAL ] GBX: Trimmed 42830 records to 28567 (keeping 200 days of data) [2025-10-20T18:55:13.381395] [DEBUG] [LOCAL ] GBX: Primary data saved (28567 records). [2025-10-20T18:55:13.381541] [INFO ] [MAIN ] GBX: Full history saved. [2025-10-20T18:55:13.382141] [INFO ] [MAIN ] Update Progress: 899/2367 symbols completed. GBX took 27.5s. Estimated remaining: 3h 8m 2s. Rate: 7.8 symbols/min [2025-10-20T18:55:13.574373] [INFO ] [MAIN ] GD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:13.623418] [DEBUG] [LOCAL ] GD: Loaded 47354 existing records from primary storage. [2025-10-20T18:55:13.623753] [INFO ] [MAIN ] GD: Loaded 47354 existing records. [2025-10-20T18:55:13.716795] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T18:55:13.852316] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-02. [2025-10-20T18:55:13.857203] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:55:13.890417] [WARN ] [MAIN ] GD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:13.890888] [INFO ] [MAIN ] GD: Wiping existing primary data before full refresh. [2025-10-20T18:55:13.891721] [INFO ] [LOCAL ] GD: Starting deletion of primary data. [2025-10-20T18:55:13.892103] [DEBUG] [LOCAL ] GD: Deleted primary data file. [2025-10-20T18:55:13.892175] [INFO ] [MAIN ] GD: Starting full primary refresh. [2025-10-20T18:55:14.282203] [INFO ] [REMOTE ] GD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:14.651426] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-06. [2025-10-20T18:55:14.787264] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T18:55:15.451459] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-01. [2025-10-20T18:55:16.261144] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-03. [2025-10-20T18:55:16.264305] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T18:55:17.051446] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-07. [2025-10-20T18:55:17.151067] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:55:17.295788] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T18:55:17.851417] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-03. [2025-10-20T18:55:18.360520] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T18:55:18.458155] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T18:55:18.651366] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-02. [2025-10-20T18:55:19.451378] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-04. [2025-10-20T18:55:20.052465] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:55:20.251396] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-04. [2025-10-20T18:55:20.388273] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T18:55:20.833940] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T18:55:21.051416] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-05. [2025-10-20T18:55:21.827574] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T18:55:21.851365] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-03. [2025-10-20T18:55:22.651422] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-06. [2025-10-20T18:55:23.243969] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T18:55:23.451346] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-07. [2025-10-20T18:55:23.646759] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T18:55:24.185574] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T18:55:24.251414] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-04. [2025-10-20T18:55:25.051423] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-08. [2025-10-20T18:55:25.851400] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-05. [2025-10-20T18:55:25.993776] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T18:55:26.651366] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-08. [2025-10-20T18:55:26.776136] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T18:55:27.059902] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T18:55:27.345465] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T18:55:27.451377] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-09. [2025-10-20T18:55:28.251433] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-09. [2025-10-20T18:55:28.998951] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T18:55:29.040395] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:55:29.051396] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-06. [2025-10-20T18:55:29.851377] [INFO ] [REMOTE ] GCI: Fetching primary data for 2025-10. [2025-10-20T18:55:30.447952] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T18:55:30.651388] [INFO ] [REMOTE ] GCV: Fetching primary data for 2025-10. [2025-10-20T18:55:31.221931] [INFO ] [REMOTE ] GCI: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T18:55:31.228304] [INFO ] [REMOTE ] GCI: Full fetch complete (63421 records). [2025-10-20T18:55:31.228435] [DEBUG] [VALIDATOR ] GCI: Validating primary data. [2025-10-20T18:55:31.457262] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-05. [2025-10-20T18:55:31.613612] [DEBUG] [VALIDATOR ] GCI: All expected trading days are present. [2025-10-20T18:55:31.613733] [DEBUG] [VALIDATOR ] GCI: Primary data validation passed. [2025-10-20T18:55:31.614550] [INFO ] [MAIN ] GCI: Fetched primary data is valid. [2025-10-20T18:55:31.614688] [DEBUG] [LOCAL ] GCI: Saving primary data. [2025-10-20T18:55:31.615928] [DEBUG] [LOCAL ] GCI: Trimmed 63421 records to 43764 (keeping 200 days of data) [2025-10-20T18:55:31.708690] [INFO ] [REMOTE ] GCV: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T18:55:31.713970] [INFO ] [REMOTE ] GCV: Full fetch complete (9865 records). [2025-10-20T18:55:31.715085] [DEBUG] [VALIDATOR ] GCV: Validating primary data. [2025-10-20T18:55:32.185156] [DEBUG] [LOCAL ] GCI: Primary data saved (43764 records). [2025-10-20T18:55:32.202398] [INFO ] [MAIN ] GCI: Full history saved. [2025-10-20T18:55:32.203824] [INFO ] [MAIN ] Update Progress: 900/2367 symbols completed. GCI took 40.1s. Estimated remaining: 3h 8m 12s. Rate: 7.8 symbols/min [2025-10-20T18:55:32.268476] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-07. [2025-10-20T18:55:32.289221] [DEBUG] [VALIDATOR ] GCV: All expected trading days are present. [2025-10-20T18:55:32.311275] [DEBUG] [VALIDATOR ] GCV: Primary data validation passed. [2025-10-20T18:55:32.325795] [INFO ] [MAIN ] GCV: Fetched primary data is valid. [2025-10-20T18:55:32.325892] [DEBUG] [LOCAL ] GCV: Saving primary data. [2025-10-20T18:55:32.330153] [DEBUG] [LOCAL ] GCV: Trimmed 9865 records to 7831 (keeping 200 days of data) [2025-10-20T18:55:32.445277] [INFO ] [MAIN ] GDDY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:32.451724] [DEBUG] [LOCAL ] GCV: Primary data saved (7831 records). [2025-10-20T18:55:32.452171] [INFO ] [MAIN ] GCV: Full history saved. [2025-10-20T18:55:32.452292] [INFO ] [MAIN ] Update Progress: 901/2367 symbols completed. GCV took 25.0s. Estimated remaining: 3h 7m 52s. Rate: 7.8 symbols/min [2025-10-20T18:55:32.662792] [INFO ] [MAIN ] GDIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:32.685187] [DEBUG] [LOCAL ] GDIV: Loaded 3630 existing records from primary storage. [2025-10-20T18:55:32.682256] [DEBUG] [LOCAL ] GDDY: Loaded 50516 existing records from primary storage. [2025-10-20T18:55:32.685509] [INFO ] [MAIN ] GDIV: Loaded 3630 existing records. [2025-10-20T18:55:32.699691] [INFO ] [MAIN ] GDDY: Loaded 50516 existing records. [2025-10-20T18:55:33.069994] [WARN ] [MAIN ] GDDY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:33.085000] [INFO ] [MAIN ] GDDY: Wiping existing primary data before full refresh. [2025-10-20T18:55:33.107543] [INFO ] [LOCAL ] GDDY: Starting deletion of primary data. [2025-10-20T18:55:33.111922] [WARN ] [MAIN ] GDIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:33.112540] [INFO ] [MAIN ] GDIV: Wiping existing primary data before full refresh. [2025-10-20T18:55:33.112612] [DEBUG] [LOCAL ] GDDY: Deleted primary data file. [2025-10-20T18:55:33.113030] [INFO ] [LOCAL ] GDIV: Starting deletion of primary data. [2025-10-20T18:55:33.113301] [DEBUG] [LOCAL ] GDIV: Deleted primary data file. [2025-10-20T18:55:33.113361] [INFO ] [MAIN ] GDIV: Starting full primary refresh. [2025-10-20T18:55:33.112916] [INFO ] [MAIN ] GDDY: Starting full primary refresh. [2025-10-20T18:55:33.727681] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T18:55:33.759317] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-08. [2025-10-20T18:55:33.895639] [INFO ] [REMOTE ] GDIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:33.900503] [INFO ] [REMOTE ] GDDY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:33.901089] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-01. [2025-10-20T18:55:33.928886] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-05 (2.47 seconds). [2025-10-20T18:55:34.651378] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-01. [2025-10-20T18:55:34.794448] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:55:35.279225] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T18:55:35.446072] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T18:55:35.455230] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-06. [2025-10-20T18:55:36.251390] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-09. [2025-10-20T18:55:37.051369] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-02. [2025-10-20T18:55:37.170809] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T18:55:37.399924] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:55:37.851387] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-02. [2025-10-20T18:55:38.396013] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:55:38.651361] [INFO ] [REMOTE ] GCTS: Fetching primary data for 2025-10. [2025-10-20T18:55:38.727523] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T18:55:39.451387] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-03. [2025-10-20T18:55:39.512247] [INFO ] [REMOTE ] GCTS: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T18:55:39.515314] [INFO ] [REMOTE ] GCTS: Full fetch complete (26915 records). [2025-10-20T18:55:39.515398] [DEBUG] [VALIDATOR ] GCTS: Validating primary data. [2025-10-20T18:55:39.909585] [DEBUG] [VALIDATOR ] GCTS: All expected trading days are present. [2025-10-20T18:55:39.909688] [DEBUG] [VALIDATOR ] GCTS: Primary data validation passed. [2025-10-20T18:55:39.910213] [INFO ] [MAIN ] GCTS: Fetched primary data is valid. [2025-10-20T18:55:39.910253] [DEBUG] [LOCAL ] GCTS: Saving primary data. [2025-10-20T18:55:39.911023] [DEBUG] [LOCAL ] GCTS: Trimmed 26915 records to 20245 (keeping 200 days of data) [2025-10-20T18:55:40.021279] [DEBUG] [LOCAL ] GCTS: Primary data saved (20245 records). [2025-10-20T18:55:40.021408] [INFO ] [MAIN ] GCTS: Full history saved. [2025-10-20T18:55:40.021482] [INFO ] [MAIN ] Update Progress: 902/2367 symbols completed. GCTS took 34.0s. Estimated remaining: 3h 7m 45s. Rate: 7.8 symbols/min [2025-10-20T18:55:40.187767] [INFO ] [MAIN ] GDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:40.191447] [DEBUG] [LOCAL ] GDL: Loaded 2550 existing records from primary storage. [2025-10-20T18:55:40.191578] [INFO ] [MAIN ] GDL: Loaded 2550 existing records. [2025-10-20T18:55:40.256614] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-03. [2025-10-20T18:55:40.368516] [WARN ] [MAIN ] GDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:40.368753] [INFO ] [MAIN ] GDL: Wiping existing primary data before full refresh. [2025-10-20T18:55:40.368808] [INFO ] [LOCAL ] GDL: Starting deletion of primary data. [2025-10-20T18:55:40.368952] [DEBUG] [LOCAL ] GDL: Deleted primary data file. [2025-10-20T18:55:40.369007] [INFO ] [MAIN ] GDL: Starting full primary refresh. [2025-10-20T18:55:40.729014] [INFO ] [REMOTE ] GDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:40.732271] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T18:55:41.051395] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-04. [2025-10-20T18:55:41.161800] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T18:55:41.590564] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T18:55:41.851420] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-05. [2025-10-20T18:55:42.451613] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:55:42.651389] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-04. [2025-10-20T18:55:43.451388] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-06. [2025-10-20T18:55:43.993708] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T18:55:44.235482] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T18:55:44.251355] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-05. [2025-10-20T18:55:45.051379] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-07. [2025-10-20T18:55:45.504626] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T18:55:45.595862] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-07 (0.54 seconds). [2025-10-20T18:55:45.851369] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-06. [2025-10-20T18:55:46.651370] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-01. [2025-10-20T18:55:47.163329] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T18:55:47.298155] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T18:55:47.451429] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-07. [2025-10-20T18:55:48.251482] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-07. [2025-10-20T18:55:49.051421] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-08. [2025-10-20T18:55:49.112441] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T18:55:49.370396] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-08 (0.32 seconds). [2025-10-20T18:55:49.374069] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:55:49.851398] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-02. [2025-10-20T18:55:50.265195] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T18:55:50.651368] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-03. [2025-10-20T18:55:51.097696] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T18:55:51.451397] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-08. [2025-10-20T18:55:52.251367] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-09. [2025-10-20T18:55:52.707807] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T18:55:53.051364] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-04. [2025-10-20T18:55:53.227097] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:55:53.359187] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-04 (0.31 seconds). [2025-10-20T18:55:53.851381] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-09. [2025-10-20T18:55:54.651408] [INFO ] [REMOTE ] GDIV: Fetching primary data for 2025-10. [2025-10-20T18:55:55.235118] [INFO ] [REMOTE ] GDIV: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T18:55:55.236375] [INFO ] [REMOTE ] GDIV: Full fetch complete (5413 records). [2025-10-20T18:55:55.236474] [DEBUG] [VALIDATOR ] GDIV: Validating primary data. [2025-10-20T18:55:55.459060] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-08. [2025-10-20T18:55:55.662078] [DEBUG] [VALIDATOR ] GDIV: All expected trading days are present. [2025-10-20T18:55:55.662313] [DEBUG] [VALIDATOR ] GDIV: Primary data validation passed. [2025-10-20T18:55:55.664108] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T18:55:55.662911] [INFO ] [MAIN ] GDIV: Fetched primary data is valid. [2025-10-20T18:55:55.674036] [DEBUG] [LOCAL ] GDIV: Saving primary data. [2025-10-20T18:55:55.675036] [DEBUG] [LOCAL ] GDIV: Trimmed 5413 records to 3580 (keeping 200 days of data) [2025-10-20T18:55:55.699114] [DEBUG] [LOCAL ] GDIV: Primary data saved (3580 records). [2025-10-20T18:55:55.699245] [INFO ] [MAIN ] GDIV: Full history saved. [2025-10-20T18:55:55.699327] [INFO ] [MAIN ] Update Progress: 903/2367 symbols completed. GDIV took 23.2s. Estimated remaining: 3h 7m 50s. Rate: 7.8 symbols/min [2025-10-20T18:55:55.868964] [INFO ] [MAIN ] GDO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:55.876287] [DEBUG] [LOCAL ] GDO: Loaded 5892 existing records from primary storage. [2025-10-20T18:55:55.876495] [INFO ] [MAIN ] GDO: Loaded 5892 existing records. [2025-10-20T18:55:56.069398] [WARN ] [MAIN ] GDO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:56.069719] [INFO ] [MAIN ] GDO: Wiping existing primary data before full refresh. [2025-10-20T18:55:56.069778] [INFO ] [LOCAL ] GDO: Starting deletion of primary data. [2025-10-20T18:55:56.070179] [DEBUG] [LOCAL ] GDO: Deleted primary data file. [2025-10-20T18:55:56.070242] [INFO ] [MAIN ] GDO: Starting full primary refresh. [2025-10-20T18:55:56.264270] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-05. [2025-10-20T18:55:56.474151] [INFO ] [REMOTE ] GDO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:55:56.814070] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T18:55:57.051381] [INFO ] [REMOTE ] GDDY: Fetching primary data for 2025-10. [2025-10-20T18:55:57.225344] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T18:55:57.851451] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-01. [2025-10-20T18:55:58.533761] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T18:55:58.651357] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-09. [2025-10-20T18:55:58.819014] [INFO ] [REMOTE ] GDDY: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T18:55:58.826561] [INFO ] [REMOTE ] GDDY: Full fetch complete (72158 records). [2025-10-20T18:55:58.826744] [DEBUG] [VALIDATOR ] GDDY: Validating primary data. [2025-10-20T18:55:59.206135] [DEBUG] [VALIDATOR ] GDDY: All expected trading days are present. [2025-10-20T18:55:59.206223] [DEBUG] [VALIDATOR ] GDDY: Primary data validation passed. [2025-10-20T18:55:59.207122] [INFO ] [MAIN ] GDDY: Fetched primary data is valid. [2025-10-20T18:55:59.207188] [DEBUG] [LOCAL ] GDDY: Saving primary data. [2025-10-20T18:55:59.208685] [DEBUG] [LOCAL ] GDDY: Trimmed 72158 records to 49758 (keeping 200 days of data) [2025-10-20T18:55:59.451724] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-06. [2025-10-20T18:55:59.544347] [DEBUG] [LOCAL ] GDDY: Primary data saved (49758 records). [2025-10-20T18:55:59.544504] [INFO ] [MAIN ] GDDY: Full history saved. [2025-10-20T18:55:59.544593] [INFO ] [MAIN ] Update Progress: 904/2367 symbols completed. GDDY took 27.3s. Estimated remaining: 3h 7m 36s. Rate: 7.8 symbols/min [2025-10-20T18:55:59.739041] [INFO ] [MAIN ] GDOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:55:59.774572] [DEBUG] [LOCAL ] GDOT: Loaded 38316 existing records from primary storage. [2025-10-20T18:55:59.774881] [INFO ] [MAIN ] GDOT: Loaded 38316 existing records. [2025-10-20T18:55:59.879828] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T18:55:59.959947] [WARN ] [MAIN ] GDOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:55:59.960298] [INFO ] [MAIN ] GDOT: Wiping existing primary data before full refresh. [2025-10-20T18:55:59.960439] [INFO ] [LOCAL ] GDOT: Starting deletion of primary data. [2025-10-20T18:55:59.960757] [DEBUG] [LOCAL ] GDOT: Deleted primary data file. [2025-10-20T18:55:59.960971] [INFO ] [MAIN ] GDOT: Starting full primary refresh. [2025-10-20T18:56:00.255044] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-02. [2025-10-20T18:56:00.267449] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T18:56:00.354595] [INFO ] [REMOTE ] GDOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:00.823258] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T18:56:01.051393] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-07. [2025-10-20T18:56:01.773360] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T18:56:01.851366] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-01. [2025-10-20T18:56:02.651380] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-03. [2025-10-20T18:56:03.126297] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T18:56:03.275166] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T18:56:03.451345] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-04. [2025-10-20T18:56:04.128382] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-04 (0.68 seconds). [2025-10-20T18:56:04.251411] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-02. [2025-10-20T18:56:05.051412] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-08. [2025-10-20T18:56:05.465758] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T18:56:05.551104] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T18:56:05.851381] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-05. [2025-10-20T18:56:06.418623] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T18:56:06.651413] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-06. [2025-10-20T18:56:07.240515] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T18:56:07.451382] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-03. [2025-10-20T18:56:08.251483] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-07. [2025-10-20T18:56:08.866327] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T18:56:08.922569] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:56:09.051420] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-08. [2025-10-20T18:56:09.835357] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T18:56:09.851396] [INFO ] [REMOTE ] GD: Fetching primary data for 2025-10. [2025-10-20T18:56:10.651373] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-04. [2025-10-20T18:56:11.451437] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-09. [2025-10-20T18:56:11.687253] [INFO ] [REMOTE ] GD: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T18:56:11.695001] [INFO ] [REMOTE ] GD: Full fetch complete (69721 records). [2025-10-20T18:56:11.695174] [DEBUG] [VALIDATOR ] GD: Validating primary data. [2025-10-20T18:56:12.045075] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T18:56:12.176858] [DEBUG] [VALIDATOR ] GD: All expected trading days are present. [2025-10-20T18:56:12.179386] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T18:56:12.179631] [DEBUG] [VALIDATOR ] GD: Primary data validation passed. [2025-10-20T18:56:12.180860] [INFO ] [MAIN ] GD: Fetched primary data is valid. [2025-10-20T18:56:12.189227] [DEBUG] [LOCAL ] GD: Saving primary data. [2025-10-20T18:56:12.191325] [DEBUG] [LOCAL ] GD: Trimmed 69721 records to 46577 (keeping 200 days of data) [2025-10-20T18:56:12.256496] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-05. [2025-10-20T18:56:12.488442] [DEBUG] [LOCAL ] GD: Primary data saved (46577 records). [2025-10-20T18:56:12.488608] [INFO ] [MAIN ] GD: Full history saved. [2025-10-20T18:56:12.488947] [INFO ] [MAIN ] Update Progress: 905/2367 symbols completed. GD took 59.1s. Estimated remaining: 3h 7m 37s. Rate: 7.8 symbols/min [2025-10-20T18:56:12.654795] [INFO ] [MAIN ] GDV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:12.675674] [DEBUG] [LOCAL ] GDV: Loaded 19698 existing records from primary storage. [2025-10-20T18:56:12.675923] [INFO ] [MAIN ] GDV: Loaded 19698 existing records. [2025-10-20T18:56:12.846507] [WARN ] [MAIN ] GDV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:12.846746] [INFO ] [MAIN ] GDV: Wiping existing primary data before full refresh. [2025-10-20T18:56:12.846796] [INFO ] [LOCAL ] GDV: Starting deletion of primary data. [2025-10-20T18:56:12.847183] [DEBUG] [LOCAL ] GDV: Deleted primary data file. [2025-10-20T18:56:12.847238] [INFO ] [MAIN ] GDV: Starting full primary refresh. [2025-10-20T18:56:13.056691] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-09. [2025-10-20T18:56:13.232768] [INFO ] [REMOTE ] GDV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:13.732967] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T18:56:13.800565] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T18:56:13.851412] [INFO ] [REMOTE ] GDL: Fetching primary data for 2025-10. [2025-10-20T18:56:14.276187] [INFO ] [REMOTE ] GDL: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T18:56:14.277324] [INFO ] [REMOTE ] GDL: Full fetch complete (3499 records). [2025-10-20T18:56:14.277401] [DEBUG] [VALIDATOR ] GDL: Validating primary data. [2025-10-20T18:56:14.656550] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-06. [2025-10-20T18:56:14.687270] [DEBUG] [VALIDATOR ] GDL: All expected trading days are present. [2025-10-20T18:56:14.687349] [DEBUG] [VALIDATOR ] GDL: Primary data validation passed. [2025-10-20T18:56:14.687469] [INFO ] [MAIN ] GDL: Fetched primary data is valid. [2025-10-20T18:56:14.687506] [DEBUG] [LOCAL ] GDL: Saving primary data. [2025-10-20T18:56:14.687890] [DEBUG] [LOCAL ] GDL: Trimmed 3499 records to 2525 (keeping 200 days of data) [2025-10-20T18:56:14.703558] [DEBUG] [LOCAL ] GDL: Primary data saved (2525 records). [2025-10-20T18:56:14.703628] [INFO ] [MAIN ] GDL: Full history saved. [2025-10-20T18:56:14.703734] [INFO ] [MAIN ] Update Progress: 906/2367 symbols completed. GDL took 34.7s. Estimated remaining: 3h 7m 20s. Rate: 7.8 symbols/min [2025-10-20T18:56:14.867514] [INFO ] [MAIN ] GE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:14.917841] [DEBUG] [LOCAL ] GE: Loaded 54478 existing records from primary storage. [2025-10-20T18:56:14.918158] [INFO ] [MAIN ] GE: Loaded 54478 existing records. [2025-10-20T18:56:15.121227] [WARN ] [MAIN ] GE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:15.121492] [INFO ] [MAIN ] GE: Wiping existing primary data before full refresh. [2025-10-20T18:56:15.121549] [INFO ] [LOCAL ] GE: Starting deletion of primary data. [2025-10-20T18:56:15.121837] [DEBUG] [LOCAL ] GE: Deleted primary data file. [2025-10-20T18:56:15.121905] [INFO ] [MAIN ] GE: Starting full primary refresh. [2025-10-20T18:56:15.451556] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-01. [2025-10-20T18:56:15.471952] [INFO ] [REMOTE ] GE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:16.181991] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T18:56:16.240311] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T18:56:16.251329] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-02. [2025-10-20T18:56:17.051432] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-01. [2025-10-20T18:56:17.237570] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T18:56:17.851371] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-03. [2025-10-20T18:56:18.651398] [INFO ] [REMOTE ] GDO: Fetching primary data for 2025-10. [2025-10-20T18:56:18.851621] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:56:19.162007] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T18:56:19.265704] [INFO ] [REMOTE ] GDO: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T18:56:19.266891] [INFO ] [REMOTE ] GDO: Full fetch complete (8492 records). [2025-10-20T18:56:19.266960] [DEBUG] [VALIDATOR ] GDO: Validating primary data. [2025-10-20T18:56:19.456591] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-02. [2025-10-20T18:56:19.622968] [DEBUG] [VALIDATOR ] GDO: All expected trading days are present. [2025-10-20T18:56:19.623041] [DEBUG] [VALIDATOR ] GDO: Primary data validation passed. [2025-10-20T18:56:19.623245] [INFO ] [MAIN ] GDO: Fetched primary data is valid. [2025-10-20T18:56:19.623283] [DEBUG] [LOCAL ] GDO: Saving primary data. [2025-10-20T18:56:19.623689] [DEBUG] [LOCAL ] GDO: Trimmed 8492 records to 5840 (keeping 200 days of data) [2025-10-20T18:56:19.657432] [DEBUG] [LOCAL ] GDO: Primary data saved (5840 records). [2025-10-20T18:56:19.657517] [INFO ] [MAIN ] GDO: Full history saved. [2025-10-20T18:56:19.657589] [INFO ] [MAIN ] Update Progress: 907/2367 symbols completed. GDO took 24.0s. Estimated remaining: 3h 7m 8s. Rate: 7.8 symbols/min [2025-10-20T18:56:19.823254] [INFO ] [MAIN ] GEF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:19.843075] [DEBUG] [LOCAL ] GEF: Loaded 21098 existing records from primary storage. [2025-10-20T18:56:19.843319] [INFO ] [MAIN ] GEF: Loaded 21098 existing records. [2025-10-20T18:56:20.019125] [WARN ] [MAIN ] GEF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:20.019339] [INFO ] [MAIN ] GEF: Wiping existing primary data before full refresh. [2025-10-20T18:56:20.019385] [INFO ] [LOCAL ] GEF: Starting deletion of primary data. [2025-10-20T18:56:20.019716] [DEBUG] [LOCAL ] GEF: Deleted primary data file. [2025-10-20T18:56:20.019771] [INFO ] [MAIN ] GEF: Starting full primary refresh. [2025-10-20T18:56:20.256456] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-07. [2025-10-20T18:56:20.368747] [INFO ] [REMOTE ] GEF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:21.051369] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-04. [2025-10-20T18:56:21.498291] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T18:56:21.772148] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T18:56:21.851333] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-08. [2025-10-20T18:56:22.213508] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:56:22.651375] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-03. [2025-10-20T18:56:23.278579] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T18:56:23.451382] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-05. [2025-10-20T18:56:24.251350] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-01. [2025-10-20T18:56:24.490163] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T18:56:24.539480] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T18:56:25.051341] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-04. [2025-10-20T18:56:25.278223] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T18:56:25.851405] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-02. [2025-10-20T18:56:26.651386] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-09. [2025-10-20T18:56:26.704880] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T18:56:27.266005] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-04 (2.21 seconds). [2025-10-20T18:56:27.451351] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-03. [2025-10-20T18:56:28.156024] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T18:56:28.251342] [INFO ] [REMOTE ] GDOT: Fetching primary data for 2025-10. [2025-10-20T18:56:28.459236] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T18:56:29.051373] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-06. [2025-10-20T18:56:29.720053] [INFO ] [REMOTE ] GDOT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T18:56:29.725941] [INFO ] [REMOTE ] GDOT: Full fetch complete (53599 records). [2025-10-20T18:56:29.726075] [DEBUG] [VALIDATOR ] GDOT: Validating primary data. [2025-10-20T18:56:29.856631] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-04. [2025-10-20T18:56:30.121803] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T18:56:30.144887] [DEBUG] [VALIDATOR ] GDOT: All expected trading days are present. [2025-10-20T18:56:30.144972] [DEBUG] [VALIDATOR ] GDOT: Primary data validation passed. [2025-10-20T18:56:30.145802] [INFO ] [MAIN ] GDOT: Fetched primary data is valid. [2025-10-20T18:56:30.145868] [DEBUG] [LOCAL ] GDOT: Saving primary data. [2025-10-20T18:56:30.147047] [DEBUG] [LOCAL ] GDOT: Trimmed 53599 records to 37755 (keeping 200 days of data) [2025-10-20T18:56:30.359907] [DEBUG] [LOCAL ] GDOT: Primary data saved (37755 records). [2025-10-20T18:56:30.360056] [INFO ] [MAIN ] GDOT: Full history saved. [2025-10-20T18:56:30.360133] [INFO ] [MAIN ] Update Progress: 908/2367 symbols completed. GDOT took 30.8s. Estimated remaining: 3h 7m 5s. Rate: 7.8 symbols/min [2025-10-20T18:56:30.535022] [INFO ] [MAIN ] GEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:30.568224] [DEBUG] [LOCAL ] GEL: Loaded 32124 existing records from primary storage. [2025-10-20T18:56:30.568508] [INFO ] [MAIN ] GEL: Loaded 32124 existing records. [2025-10-20T18:56:30.656733] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-07. [2025-10-20T18:56:30.753246] [WARN ] [MAIN ] GEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:30.753533] [INFO ] [MAIN ] GEL: Wiping existing primary data before full refresh. [2025-10-20T18:56:30.753580] [INFO ] [LOCAL ] GEL: Starting deletion of primary data. [2025-10-20T18:56:30.753869] [DEBUG] [LOCAL ] GEL: Deleted primary data file. [2025-10-20T18:56:30.753936] [INFO ] [MAIN ] GEL: Starting full primary refresh. [2025-10-20T18:56:31.015324] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T18:56:31.130711] [INFO ] [REMOTE ] GEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:31.451344] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-05. [2025-10-20T18:56:31.670696] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T18:56:32.251390] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-01. [2025-10-20T18:56:32.277225] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:56:33.051370] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-06. [2025-10-20T18:56:33.383740] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T18:56:33.851393] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-02. [2025-10-20T18:56:33.888630] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T18:56:34.651421] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-07. [2025-10-20T18:56:35.168438] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:56:35.451345] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-03. [2025-10-20T18:56:35.701354] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T18:56:36.251424] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-08. [2025-10-20T18:56:36.970752] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T18:56:37.051362] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-04. [2025-10-20T18:56:37.195471] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T18:56:37.851390] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-09. [2025-10-20T18:56:38.075814] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T18:56:38.651395] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-08. [2025-10-20T18:56:38.787723] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T18:56:39.451410] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-05. [2025-10-20T18:56:39.467567] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T18:56:40.251379] [INFO ] [REMOTE ] GDV: Fetching primary data for 2025-10. [2025-10-20T18:56:41.051395] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-09. [2025-10-20T18:56:41.279346] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T18:56:41.391330] [INFO ] [REMOTE ] GDV: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T18:56:41.394766] [INFO ] [REMOTE ] GDV: Full fetch complete (28419 records). [2025-10-20T18:56:41.394889] [DEBUG] [VALIDATOR ] GDV: Validating primary data. [2025-10-20T18:56:41.782917] [DEBUG] [VALIDATOR ] GDV: All expected trading days are present. [2025-10-20T18:56:41.783000] [DEBUG] [VALIDATOR ] GDV: Primary data validation passed. [2025-10-20T18:56:41.783603] [INFO ] [MAIN ] GDV: Fetched primary data is valid. [2025-10-20T18:56:41.783667] [DEBUG] [LOCAL ] GDV: Saving primary data. [2025-10-20T18:56:41.784521] [DEBUG] [LOCAL ] GDV: Trimmed 28419 records to 19418 (keeping 200 days of data) [2025-10-20T18:56:41.868888] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-05. [2025-10-20T18:56:41.906407] [DEBUG] [LOCAL ] GDV: Primary data saved (19418 records). [2025-10-20T18:56:41.906563] [INFO ] [MAIN ] GDV: Full history saved. [2025-10-20T18:56:41.906676] [INFO ] [MAIN ] Update Progress: 909/2367 symbols completed. GDV took 29.4s. Estimated remaining: 3h 7m 4s. Rate: 7.8 symbols/min [2025-10-20T18:56:42.080355] [INFO ] [MAIN ] GENI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:42.130522] [DEBUG] [LOCAL ] GENI: Loaded 53717 existing records from primary storage. [2025-10-20T18:56:42.130977] [INFO ] [MAIN ] GENI: Loaded 53717 existing records. [2025-10-20T18:56:42.235032] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T18:56:42.345818] [WARN ] [MAIN ] GENI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:42.345975] [INFO ] [MAIN ] GENI: Wiping existing primary data before full refresh. [2025-10-20T18:56:42.346021] [INFO ] [LOCAL ] GENI: Starting deletion of primary data. [2025-10-20T18:56:42.346320] [DEBUG] [LOCAL ] GENI: Deleted primary data file. [2025-10-20T18:56:42.346376] [INFO ] [MAIN ] GENI: Starting full primary refresh. [2025-10-20T18:56:42.658885] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-06. [2025-10-20T18:56:42.699522] [INFO ] [REMOTE ] GENI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:43.451392] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-01. [2025-10-20T18:56:43.483828] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T18:56:44.251358] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-06. [2025-10-20T18:56:44.895078] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-06 (2.24 seconds). [2025-10-20T18:56:44.974842] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:56:45.051334] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-07. [2025-10-20T18:56:45.658376] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T18:56:45.851362] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-07. [2025-10-20T18:56:46.651359] [INFO ] [REMOTE ] GEF: Fetching primary data for 2025-10. [2025-10-20T18:56:46.901841] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T18:56:47.282799] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:56:47.451397] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-02. [2025-10-20T18:56:47.678582] [INFO ] [REMOTE ] GEF: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T18:56:47.682155] [INFO ] [REMOTE ] GEF: Full fetch complete (29381 records). [2025-10-20T18:56:47.682252] [DEBUG] [VALIDATOR ] GEF: Validating primary data. [2025-10-20T18:56:48.046797] [DEBUG] [VALIDATOR ] GEF: All expected trading days are present. [2025-10-20T18:56:48.046877] [DEBUG] [VALIDATOR ] GEF: Primary data validation passed. [2025-10-20T18:56:48.047435] [INFO ] [MAIN ] GEF: Fetched primary data is valid. [2025-10-20T18:56:48.047473] [DEBUG] [LOCAL ] GEF: Saving primary data. [2025-10-20T18:56:48.048155] [DEBUG] [LOCAL ] GEF: Trimmed 29381 records to 20852 (keeping 200 days of data) [2025-10-20T18:56:48.171352] [DEBUG] [LOCAL ] GEF: Primary data saved (20852 records). [2025-10-20T18:56:48.171472] [INFO ] [MAIN ] GEF: Full history saved. [2025-10-20T18:56:48.171547] [INFO ] [MAIN ] Update Progress: 910/2367 symbols completed. GEF took 28.5s. Estimated remaining: 3h 6m 54s. Rate: 7.8 symbols/min [2025-10-20T18:56:48.260739] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-08. [2025-10-20T18:56:48.343502] [INFO ] [MAIN ] GEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:48.396438] [DEBUG] [LOCAL ] GEO: Loaded 52550 existing records from primary storage. [2025-10-20T18:56:48.396780] [INFO ] [MAIN ] GEO: Loaded 52550 existing records. [2025-10-20T18:56:48.577177] [WARN ] [MAIN ] GEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:48.577406] [INFO ] [MAIN ] GEO: Wiping existing primary data before full refresh. [2025-10-20T18:56:48.577482] [INFO ] [LOCAL ] GEO: Starting deletion of primary data. [2025-10-20T18:56:48.577759] [DEBUG] [LOCAL ] GEO: Deleted primary data file. [2025-10-20T18:56:48.577834] [INFO ] [MAIN ] GEO: Starting full primary refresh. [2025-10-20T18:56:48.951960] [INFO ] [REMOTE ] GEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:48.983762] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T18:56:49.051416] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-08. [2025-10-20T18:56:49.851378] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-03. [2025-10-20T18:56:50.428991] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T18:56:50.576118] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-10-20T18:56:50.651361] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-09. [2025-10-20T18:56:51.435069] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T18:56:51.451337] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-04. [2025-10-20T18:56:52.251358] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-01. [2025-10-20T18:56:52.819304] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-09 (2.17 seconds). [2025-10-20T18:56:53.040766] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T18:56:53.051507] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-09. [2025-10-20T18:56:53.837356] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T18:56:53.851331] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-05. [2025-10-20T18:56:53.951438] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T18:56:54.651448] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-02. [2025-10-20T18:56:55.395411] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T18:56:55.451381] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-06. [2025-10-20T18:56:56.251383] [INFO ] [REMOTE ] GEL: Fetching primary data for 2025-10. [2025-10-20T18:56:56.323426] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T18:56:57.007907] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T18:56:57.051377] [INFO ] [REMOTE ] GE: Fetching primary data for 2025-10. [2025-10-20T18:56:57.426402] [INFO ] [REMOTE ] GEL: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T18:56:57.430094] [INFO ] [REMOTE ] GEL: Full fetch complete (48079 records). [2025-10-20T18:56:57.430184] [DEBUG] [VALIDATOR ] GEL: Validating primary data. [2025-10-20T18:56:57.832276] [DEBUG] [VALIDATOR ] GEL: All expected trading days are present. [2025-10-20T18:56:57.832363] [DEBUG] [VALIDATOR ] GEL: Primary data validation passed. [2025-10-20T18:56:57.833091] [INFO ] [MAIN ] GEL: Fetched primary data is valid. [2025-10-20T18:56:57.833178] [DEBUG] [LOCAL ] GEL: Saving primary data. [2025-10-20T18:56:57.834248] [DEBUG] [LOCAL ] GEL: Trimmed 48079 records to 31500 (keeping 200 days of data) [2025-10-20T18:56:57.886129] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-03. [2025-10-20T18:56:58.053403] [DEBUG] [LOCAL ] GEL: Primary data saved (31500 records). [2025-10-20T18:56:58.054149] [INFO ] [MAIN ] GEL: Full history saved. [2025-10-20T18:56:58.054952] [INFO ] [REMOTE ] GE: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:56:58.062588] [INFO ] [MAIN ] Update Progress: 911/2367 symbols completed. GEL took 27.7s. Estimated remaining: 3h 6m 50s. Rate: 7.8 symbols/min [2025-10-20T18:56:58.121148] [INFO ] [REMOTE ] GE: Full fetch complete (77865 records). [2025-10-20T18:56:58.136478] [DEBUG] [VALIDATOR ] GE: Validating primary data. [2025-10-20T18:56:58.404755] [INFO ] [MAIN ] GES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:58.653242] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-07. [2025-10-20T18:56:58.657104] [DEBUG] [VALIDATOR ] GE: All expected trading days are present. [2025-10-20T18:56:58.658207] [DEBUG] [VALIDATOR ] GE: Primary data validation passed. [2025-10-20T18:56:58.659962] [INFO ] [MAIN ] GE: Fetched primary data is valid. [2025-10-20T18:56:58.672516] [DEBUG] [LOCAL ] GE: Saving primary data. [2025-10-20T18:56:58.673746] [DEBUG] [LOCAL ] GES: Loaded 38876 existing records from primary storage. [2025-10-20T18:56:58.674351] [INFO ] [MAIN ] GES: Loaded 38876 existing records. [2025-10-20T18:56:58.745387] [DEBUG] [LOCAL ] GE: Trimmed 77865 records to 53698 (keeping 200 days of data) [2025-10-20T18:56:58.949203] [WARN ] [MAIN ] GES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:58.952125] [INFO ] [MAIN ] GES: Wiping existing primary data before full refresh. [2025-10-20T18:56:58.959457] [INFO ] [LOCAL ] GES: Starting deletion of primary data. [2025-10-20T18:56:58.961394] [DEBUG] [LOCAL ] GES: Deleted primary data file. [2025-10-20T18:56:58.961987] [INFO ] [MAIN ] GES: Starting full primary refresh. [2025-10-20T18:56:59.353494] [INFO ] [REMOTE ] GES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:56:59.451993] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-01. [2025-10-20T18:56:59.567202] [DEBUG] [LOCAL ] GE: Primary data saved (53698 records). [2025-10-20T18:56:59.567351] [INFO ] [MAIN ] GE: Full history saved. [2025-10-20T18:56:59.567430] [INFO ] [MAIN ] Update Progress: 912/2367 symbols completed. GE took 44.9s. Estimated remaining: 3h 6m 32s. Rate: 7.8 symbols/min [2025-10-20T18:56:59.739982] [INFO ] [MAIN ] GETY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:56:59.772500] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T18:56:59.814959] [DEBUG] [LOCAL ] GETY: Loaded 35677 existing records from primary storage. [2025-10-20T18:56:59.815291] [INFO ] [MAIN ] GETY: Loaded 35677 existing records. [2025-10-20T18:56:59.995422] [WARN ] [MAIN ] GETY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:56:59.995633] [INFO ] [MAIN ] GETY: Wiping existing primary data before full refresh. [2025-10-20T18:56:59.995729] [INFO ] [LOCAL ] GETY: Starting deletion of primary data. [2025-10-20T18:56:59.995970] [DEBUG] [LOCAL ] GETY: Deleted primary data file. [2025-10-20T18:56:59.996025] [INFO ] [MAIN ] GETY: Starting full primary refresh. [2025-10-20T18:57:00.253223] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-04. [2025-10-20T18:57:00.371441] [INFO ] [REMOTE ] GETY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:00.403750] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T18:57:01.051348] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-08. [2025-10-20T18:57:01.485367] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T18:57:01.851380] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-01. [2025-10-20T18:57:02.414304] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T18:57:02.651356] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-05. [2025-10-20T18:57:02.833969] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T18:57:03.451376] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-02. [2025-10-20T18:57:03.576412] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T18:57:04.251348] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-02. [2025-10-20T18:57:04.393117] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T18:57:04.850939] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T18:57:05.051386] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-09. [2025-10-20T18:57:05.787843] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T18:57:05.851377] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-03. [2025-10-20T18:57:06.651376] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-06. [2025-10-20T18:57:06.886074] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T18:57:07.440480] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:57:07.451337] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-04. [2025-10-20T18:57:08.251354] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-03. [2025-10-20T18:57:08.290835] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T18:57:08.913267] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T18:57:09.051385] [INFO ] [REMOTE ] GENI: Fetching primary data for 2025-10. [2025-10-20T18:57:09.851350] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-05. [2025-10-20T18:57:09.949100] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T18:57:10.651417] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-07. [2025-10-20T18:57:10.780307] [INFO ] [REMOTE ] GENI: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T18:57:10.788201] [INFO ] [REMOTE ] GENI: Full fetch complete (76088 records). [2025-10-20T18:57:10.788355] [DEBUG] [VALIDATOR ] GENI: Validating primary data. [2025-10-20T18:57:11.191284] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:57:11.227906] [DEBUG] [VALIDATOR ] GENI: All expected trading days are present. [2025-10-20T18:57:11.228027] [DEBUG] [VALIDATOR ] GENI: Primary data validation passed. [2025-10-20T18:57:11.229448] [INFO ] [MAIN ] GENI: Fetched primary data is valid. [2025-10-20T18:57:11.229516] [DEBUG] [LOCAL ] GENI: Saving primary data. [2025-10-20T18:57:11.231022] [DEBUG] [LOCAL ] GENI: Trimmed 76088 records to 52943 (keeping 200 days of data) [2025-10-20T18:57:11.452010] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-04. [2025-10-20T18:57:11.545381] [DEBUG] [LOCAL ] GENI: Primary data saved (52943 records). [2025-10-20T18:57:11.545537] [INFO ] [MAIN ] GENI: Full history saved. [2025-10-20T18:57:11.546119] [INFO ] [MAIN ] Update Progress: 913/2367 symbols completed. GENI took 29.6s. Estimated remaining: 3h 6m 31s. Rate: 7.8 symbols/min [2025-10-20T18:57:11.711484] [INFO ] [MAIN ] GEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:11.763104] [DEBUG] [LOCAL ] GEV: Loaded 53772 existing records from primary storage. [2025-10-20T18:57:11.763405] [INFO ] [MAIN ] GEV: Loaded 53772 existing records. [2025-10-20T18:57:11.939820] [WARN ] [MAIN ] GEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:11.940030] [INFO ] [MAIN ] GEV: Wiping existing primary data before full refresh. [2025-10-20T18:57:11.940077] [INFO ] [LOCAL ] GEV: Starting deletion of primary data. [2025-10-20T18:57:11.940264] [DEBUG] [LOCAL ] GEV: Deleted primary data file. [2025-10-20T18:57:11.940301] [INFO ] [MAIN ] GEV: Starting full primary refresh. [2025-10-20T18:57:12.262328] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-06. [2025-10-20T18:57:12.303681] [INFO ] [REMOTE ] GEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:12.552836] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T18:57:13.051347] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-01. [2025-10-20T18:57:13.140204] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T18:57:13.705327] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T18:57:13.851360] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-08. [2025-10-20T18:57:14.651379] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-05. [2025-10-20T18:57:15.252899] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-10-20T18:57:15.331614] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:57:15.451396] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-09. [2025-10-20T18:57:16.064932] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T18:57:16.251378] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-06. [2025-10-20T18:57:17.051012] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T18:57:17.060955] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-07. [2025-10-20T18:57:17.794774] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T18:57:17.851338] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-07. [2025-10-20T18:57:18.488906] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T18:57:18.651398] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-08. [2025-10-20T18:57:19.333032] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T18:57:19.451367] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-02. [2025-10-20T18:57:20.117216] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T18:57:20.251383] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-08. [2025-10-20T18:57:21.051392] [INFO ] [REMOTE ] GEO: Fetching primary data for 2025-10. [2025-10-20T18:57:21.657074] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T18:57:21.775382] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T18:57:21.851386] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-09. [2025-10-20T18:57:22.651345] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-09. [2025-10-20T18:57:22.995041] [INFO ] [REMOTE ] GEO: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T18:57:23.004039] [INFO ] [REMOTE ] GEO: Full fetch complete (75543 records). [2025-10-20T18:57:23.004244] [DEBUG] [VALIDATOR ] GEO: Validating primary data. [2025-10-20T18:57:23.418819] [DEBUG] [VALIDATOR ] GEO: All expected trading days are present. [2025-10-20T18:57:23.433451] [DEBUG] [VALIDATOR ] GEO: Primary data validation passed. [2025-10-20T18:57:23.435437] [INFO ] [MAIN ] GEO: Fetched primary data is valid. [2025-10-20T18:57:23.436559] [DEBUG] [LOCAL ] GEO: Saving primary data. [2025-10-20T18:57:23.436451] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T18:57:23.445938] [DEBUG] [LOCAL ] GEO: Trimmed 75543 records to 51753 (keeping 200 days of data) [2025-10-20T18:57:23.493566] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-03. [2025-10-20T18:57:23.754670] [DEBUG] [LOCAL ] GEO: Primary data saved (51753 records). [2025-10-20T18:57:23.754866] [INFO ] [MAIN ] GEO: Full history saved. [2025-10-20T18:57:23.755521] [INFO ] [MAIN ] Update Progress: 914/2367 symbols completed. GEO took 35.6s. Estimated remaining: 3h 6m 31s. Rate: 7.8 symbols/min [2025-10-20T18:57:23.929961] [INFO ] [MAIN ] GF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:23.930170] [DEBUG] [LOCAL ] GF: No existing data file found in primary storage. [2025-10-20T18:57:23.930357] [WARN ] [MAIN ] GF: No existing data found, performing full refresh instead. [2025-10-20T18:57:23.930587] [INFO ] [MAIN ] GF: Wiping existing primary data before full refresh. [2025-10-20T18:57:23.930762] [INFO ] [LOCAL ] GF: Starting deletion of primary data. [2025-10-20T18:57:23.930838] [DEBUG] [LOCAL ] GF: No primary data file found to delete. [2025-10-20T18:57:23.930888] [INFO ] [MAIN ] GF: Starting full primary refresh. [2025-10-20T18:57:24.104387] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T18:57:24.256466] [INFO ] [REMOTE ] GES: Fetching primary data for 2025-10. [2025-10-20T18:57:24.325578] [INFO ] [REMOTE ] GF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:25.051434] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-01. [2025-10-20T18:57:25.540717] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T18:57:25.602510] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T18:57:25.646319] [INFO ] [REMOTE ] GES: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T18:57:25.654457] [INFO ] [REMOTE ] GES: Full fetch complete (58555 records). [2025-10-20T18:57:25.654702] [DEBUG] [VALIDATOR ] GES: Validating primary data. [2025-10-20T18:57:25.858540] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-02. [2025-10-20T18:57:26.039841] [DEBUG] [VALIDATOR ] GES: All expected trading days are present. [2025-10-20T18:57:26.039933] [DEBUG] [VALIDATOR ] GES: Primary data validation passed. [2025-10-20T18:57:26.040780] [INFO ] [MAIN ] GES: Fetched primary data is valid. [2025-10-20T18:57:26.040852] [DEBUG] [LOCAL ] GES: Saving primary data. [2025-10-20T18:57:26.042119] [DEBUG] [LOCAL ] GES: Trimmed 58555 records to 38154 (keeping 200 days of data) [2025-10-20T18:57:26.270841] [DEBUG] [LOCAL ] GES: Primary data saved (38154 records). [2025-10-20T18:57:26.270997] [INFO ] [MAIN ] GES: Full history saved. [2025-10-20T18:57:26.271077] [INFO ] [MAIN ] Update Progress: 915/2367 symbols completed. GES took 28.2s. Estimated remaining: 3h 6m 15s. Rate: 7.8 symbols/min [2025-10-20T18:57:26.443784] [INFO ] [MAIN ] GFF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:26.473277] [DEBUG] [LOCAL ] GFF: Loaded 30283 existing records from primary storage. [2025-10-20T18:57:26.473553] [INFO ] [MAIN ] GFF: Loaded 30283 existing records. [2025-10-20T18:57:26.582090] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T18:57:26.657203] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-03. [2025-10-20T18:57:26.663922] [WARN ] [MAIN ] GFF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:26.664982] [INFO ] [MAIN ] GFF: Wiping existing primary data before full refresh. [2025-10-20T18:57:26.665034] [INFO ] [LOCAL ] GFF: Starting deletion of primary data. [2025-10-20T18:57:26.665244] [DEBUG] [LOCAL ] GFF: Deleted primary data file. [2025-10-20T18:57:26.665300] [INFO ] [MAIN ] GFF: Starting full primary refresh. [2025-10-20T18:57:27.017897] [INFO ] [REMOTE ] GFF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:27.451397] [INFO ] [REMOTE ] GETY: Fetching primary data for 2025-10. [2025-10-20T18:57:27.541999] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T18:57:28.251433] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-01. [2025-10-20T18:57:28.800696] [INFO ] [REMOTE ] GETY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T18:57:28.806348] [INFO ] [REMOTE ] GETY: Full fetch complete (55671 records). [2025-10-20T18:57:28.806491] [DEBUG] [VALIDATOR ] GETY: Validating primary data. [2025-10-20T18:57:29.056708] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-04. [2025-10-20T18:57:29.187823] [DEBUG] [VALIDATOR ] GETY: All expected trading days are present. [2025-10-20T18:57:29.188088] [DEBUG] [VALIDATOR ] GETY: Primary data validation passed. [2025-10-20T18:57:29.189022] [INFO ] [MAIN ] GETY: Fetched primary data is valid. [2025-10-20T18:57:29.189115] [DEBUG] [LOCAL ] GETY: Saving primary data. [2025-10-20T18:57:29.191022] [DEBUG] [LOCAL ] GETY: Trimmed 55671 records to 35268 (keeping 200 days of data) [2025-10-20T18:57:29.291948] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:57:29.417068] [DEBUG] [LOCAL ] GETY: Primary data saved (35268 records). [2025-10-20T18:57:29.417209] [INFO ] [MAIN ] GETY: Full history saved. [2025-10-20T18:57:29.417287] [INFO ] [MAIN ] Update Progress: 916/2367 symbols completed. GETY took 29.8s. Estimated remaining: 3h 5m 60s. Rate: 7.8 symbols/min [2025-10-20T18:57:29.588693] [INFO ] [MAIN ] GFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:29.637430] [DEBUG] [LOCAL ] GFI: Loaded 53895 existing records from primary storage. [2025-10-20T18:57:29.637787] [INFO ] [MAIN ] GFI: Loaded 53895 existing records. [2025-10-20T18:57:29.813085] [WARN ] [MAIN ] GFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:29.813307] [INFO ] [MAIN ] GFI: Wiping existing primary data before full refresh. [2025-10-20T18:57:29.813353] [INFO ] [LOCAL ] GFI: Starting deletion of primary data. [2025-10-20T18:57:29.813853] [DEBUG] [LOCAL ] GFI: Deleted primary data file. [2025-10-20T18:57:29.813909] [INFO ] [MAIN ] GFI: Starting full primary refresh. [2025-10-20T18:57:29.856610] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-02. [2025-10-20T18:57:30.166922] [INFO ] [REMOTE ] GFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:57:30.651382] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-01. [2025-10-20T18:57:31.219288] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T18:57:31.349032] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T18:57:31.451348] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-05. [2025-10-20T18:57:32.251364] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-03. [2025-10-20T18:57:32.514159] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T18:57:33.051380] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-02. [2025-10-20T18:57:33.249627] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:57:33.845860] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-05 (2.39 seconds). [2025-10-20T18:57:33.857783] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-04. [2025-10-20T18:57:34.651541] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-06. [2025-10-20T18:57:34.884455] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:57:35.298244] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T18:57:35.451369] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-03. [2025-10-20T18:57:36.251366] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-04. [2025-10-20T18:57:36.528462] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T18:57:36.867846] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T18:57:37.051356] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-05. [2025-10-20T18:57:37.204623] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:57:37.671705] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T18:57:37.851373] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-06. [2025-10-20T18:57:38.498054] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T18:57:38.651371] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-04. [2025-10-20T18:57:39.451383] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-07. [2025-10-20T18:57:39.899426] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T18:57:40.251365] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-08. [2025-10-20T18:57:40.688578] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T18:57:40.736751] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T18:57:41.051380] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-05. [2025-10-20T18:57:41.851438] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-09. [2025-10-20T18:57:42.379903] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T18:57:42.385006] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T18:57:42.651399] [INFO ] [REMOTE ] GF: Fetching primary data for 2025-10. [2025-10-20T18:57:43.053178] [INFO ] [REMOTE ] GF: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T18:57:43.055559] [INFO ] [REMOTE ] GF: Full fetch complete (4810 records). [2025-10-20T18:57:43.055776] [DEBUG] [VALIDATOR ] GF: Validating primary data. [2025-10-20T18:57:43.424762] [DEBUG] [VALIDATOR ] GF: All expected trading days are present. [2025-10-20T18:57:43.424841] [DEBUG] [VALIDATOR ] GF: Primary data validation passed. [2025-10-20T18:57:43.424985] [INFO ] [MAIN ] GF: Fetched primary data is valid. [2025-10-20T18:57:43.425022] [DEBUG] [LOCAL ] GF: Saving primary data. [2025-10-20T18:57:43.425373] [DEBUG] [LOCAL ] GF: Trimmed 4810 records to 2662 (keeping 200 days of data) [2025-10-20T18:57:43.440823] [DEBUG] [LOCAL ] GF: Primary data saved (2662 records). [2025-10-20T18:57:43.440888] [INFO ] [MAIN ] GF: Full history saved. [2025-10-20T18:57:43.440948] [INFO ] [MAIN ] Update Progress: 917/2367 symbols completed. GF took 19.7s. Estimated remaining: 3h 6m 2s. Rate: 7.8 symbols/min [2025-10-20T18:57:43.451369] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-05. [2025-10-20T18:57:44.251382] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-06. [2025-10-20T18:57:45.051438] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-07. [2025-10-20T18:57:45.538432] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T18:57:45.572221] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T18:57:45.851375] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-07. [2025-10-20T18:57:46.651386] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-06. [2025-10-20T18:57:46.966380] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T18:57:47.451345] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-08. [2025-10-20T18:57:47.593489] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-07 (2.54 seconds). [2025-10-20T18:57:48.251393] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-08. [2025-10-20T18:57:48.568025] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T18:57:48.737901] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T18:57:49.051406] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-09. [2025-10-20T18:57:49.851442] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-07. [2025-10-20T18:57:50.197184] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T18:57:50.651354] [INFO ] [REMOTE ] GFF: Fetching primary data for 2025-10. [2025-10-20T18:57:50.715877] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-08 (2.46 seconds). [2025-10-20T18:57:51.451345] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-09. [2025-10-20T18:57:51.519547] [INFO ] [REMOTE ] GFF: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T18:57:51.523621] [INFO ] [REMOTE ] GFF: Full fetch complete (43805 records). [2025-10-20T18:57:51.523757] [DEBUG] [VALIDATOR ] GFF: Validating primary data. [2025-10-20T18:57:51.938316] [DEBUG] [VALIDATOR ] GFF: All expected trading days are present. [2025-10-20T18:57:51.941436] [DEBUG] [VALIDATOR ] GFF: Primary data validation passed. [2025-10-20T18:57:51.942136] [INFO ] [MAIN ] GFF: Fetched primary data is valid. [2025-10-20T18:57:51.942383] [DEBUG] [LOCAL ] GFF: Saving primary data. [2025-10-20T18:57:51.944556] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T18:57:51.959304] [DEBUG] [LOCAL ] GFF: Trimmed 43805 records to 29664 (keeping 200 days of data) [2025-10-20T18:57:52.148975] [DEBUG] [LOCAL ] GFF: Primary data saved (29664 records). [2025-10-20T18:57:52.149131] [INFO ] [MAIN ] GFF: Full history saved. [2025-10-20T18:57:52.149210] [INFO ] [MAIN ] Update Progress: 918/2367 symbols completed. GFF took 25.9s. Estimated remaining: 3h 5m 56s. Rate: 7.8 symbols/min [2025-10-20T18:57:52.251460] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-08. [2025-10-20T18:57:53.848087] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T18:57:53.861300] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-09. [2025-10-20T18:57:53.942608] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T18:57:53.954086] [INFO ] [REMOTE ] GEV: Fetching primary data for 2025-10. [2025-10-20T18:57:55.850839] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T18:57:55.863448] [INFO ] [REMOTE ] GFI: Fetching primary data for 2025-10. [2025-10-20T18:57:56.607944] [INFO ] [REMOTE ] GEV: Incremental fetch complete for 2025-10 (2.65 seconds). [2025-10-20T18:57:56.615848] [INFO ] [REMOTE ] GEV: Full fetch complete (77040 records). [2025-10-20T18:57:56.616011] [DEBUG] [VALIDATOR ] GEV: Validating primary data. [2025-10-20T18:57:57.004562] [DEBUG] [VALIDATOR ] GEV: All expected trading days are present. [2025-10-20T18:57:57.004663] [DEBUG] [VALIDATOR ] GEV: Primary data validation passed. [2025-10-20T18:57:57.005667] [INFO ] [MAIN ] GEV: Fetched primary data is valid. [2025-10-20T18:57:57.005734] [DEBUG] [LOCAL ] GEV: Saving primary data. [2025-10-20T18:57:57.007240] [DEBUG] [LOCAL ] GEV: Trimmed 77040 records to 52992 (keeping 200 days of data) [2025-10-20T18:57:57.338852] [DEBUG] [LOCAL ] GEV: Primary data saved (52992 records). [2025-10-20T18:57:57.338986] [INFO ] [MAIN ] GEV: Full history saved. [2025-10-20T18:57:57.339490] [INFO ] [MAIN ] Update Progress: 919/2367 symbols completed. GEV took 45.8s. Estimated remaining: 3h 5m 44s. Rate: 7.8 symbols/min [2025-10-20T18:57:58.074132] [INFO ] [REMOTE ] GFI: Incremental fetch complete for 2025-10 (2.21 seconds). [2025-10-20T18:57:58.081898] [INFO ] [REMOTE ] GFI: Full fetch complete (76940 records). [2025-10-20T18:57:58.082013] [DEBUG] [VALIDATOR ] GFI: Validating primary data. [2025-10-20T18:57:58.472705] [DEBUG] [VALIDATOR ] GFI: All expected trading days are present. [2025-10-20T18:57:58.472789] [DEBUG] [VALIDATOR ] GFI: Primary data validation passed. [2025-10-20T18:57:58.473709] [INFO ] [MAIN ] GFI: Fetched primary data is valid. [2025-10-20T18:57:58.473773] [DEBUG] [LOCAL ] GFI: Saving primary data. [2025-10-20T18:57:58.475162] [DEBUG] [LOCAL ] GFI: Trimmed 76940 records to 53116 (keeping 200 days of data) [2025-10-20T18:57:58.795886] [DEBUG] [LOCAL ] GFI: Primary data saved (53116 records). [2025-10-20T18:57:58.796016] [INFO ] [MAIN ] GFI: Full history saved. [2025-10-20T18:57:58.796094] [INFO ] [MAIN ] Update Progress: 920/2367 symbols completed. GFI took 29.4s. Estimated remaining: 3h 5m 27s. Rate: 7.8 symbols/min [2025-10-20T18:57:59.371060] [INFO ] [MAIN ] GGG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.439494] [INFO ] [MAIN ] GFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.476927] [INFO ] [MAIN ] GFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.503479] [INFO ] [MAIN ] GGB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:57:59.515460] [DEBUG] [LOCAL ] GFR: Loaded 7837 existing records from primary storage. [2025-10-20T18:57:59.515912] [INFO ] [MAIN ] GFR: Loaded 7837 existing records. [2025-10-20T18:57:59.752473] [DEBUG] [LOCAL ] GGG: Loaded 41252 existing records from primary storage. [2025-10-20T18:57:59.755668] [WARN ] [MAIN ] GFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:57:59.756235] [INFO ] [MAIN ] GGG: Loaded 41252 existing records. [2025-10-20T18:57:59.756139] [INFO ] [MAIN ] GFR: Wiping existing primary data before full refresh. [2025-10-20T18:57:59.756374] [INFO ] [LOCAL ] GFR: Starting deletion of primary data. [2025-10-20T18:57:59.841978] [DEBUG] [LOCAL ] GFR: Deleted primary data file. [2025-10-20T18:57:59.832688] [DEBUG] [LOCAL ] GGB: Loaded 52241 existing records from primary storage. [2025-10-20T18:57:59.860177] [INFO ] [MAIN ] GFR: Starting full primary refresh. [2025-10-20T18:57:59.917260] [DEBUG] [LOCAL ] GFL: Loaded 48336 existing records from primary storage. [2025-10-20T18:57:59.917501] [INFO ] [MAIN ] GGB: Loaded 52241 existing records. [2025-10-20T18:57:59.973190] [INFO ] [MAIN ] GFL: Loaded 48336 existing records. [2025-10-20T18:58:00.383246] [WARN ] [MAIN ] GGG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.414142] [INFO ] [MAIN ] GGG: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.421115] [INFO ] [LOCAL ] GGG: Starting deletion of primary data. [2025-10-20T18:58:00.513517] [DEBUG] [LOCAL ] GGG: Deleted primary data file. [2025-10-20T18:58:00.561153] [INFO ] [MAIN ] GGG: Starting full primary refresh. [2025-10-20T18:58:00.683774] [WARN ] [MAIN ] GFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.655554] [WARN ] [MAIN ] GGB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:00.771509] [INFO ] [MAIN ] GFL: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.805400] [INFO ] [MAIN ] GGB: Wiping existing primary data before full refresh. [2025-10-20T18:58:00.819677] [INFO ] [LOCAL ] GGB: Starting deletion of primary data. [2025-10-20T18:58:00.824916] [INFO ] [LOCAL ] GFL: Starting deletion of primary data. [2025-10-20T18:58:00.836922] [DEBUG] [LOCAL ] GGB: Deleted primary data file. [2025-10-20T18:58:00.847704] [INFO ] [MAIN ] GGB: Starting full primary refresh. [2025-10-20T18:58:00.874073] [DEBUG] [LOCAL ] GFL: Deleted primary data file. [2025-10-20T18:58:00.884452] [INFO ] [MAIN ] GFL: Starting full primary refresh. [2025-10-20T18:58:01.025199] [INFO ] [REMOTE ] GFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.054254] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-01. [2025-10-20T18:58:01.711090] [INFO ] [REMOTE ] GGG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.734822] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-01. [2025-10-20T18:58:01.968376] [INFO ] [REMOTE ] GGB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.978605] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-01. [2025-10-20T18:58:01.998772] [INFO ] [REMOTE ] GFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:01.999090] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-01. [2025-10-20T18:58:02.160173] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T18:58:02.162600] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-02. [2025-10-20T18:58:02.719805] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T18:58:02.722184] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-03. [2025-10-20T18:58:03.517761] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:58:03.526952] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-02. [2025-10-20T18:58:03.589729] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T18:58:03.599142] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-02. [2025-10-20T18:58:03.608436] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T18:58:03.612215] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-04. [2025-10-20T18:58:03.644715] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T18:58:03.655048] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-02. [2025-10-20T18:58:04.171207] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T18:58:04.174475] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-05. [2025-10-20T18:58:04.649421] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T18:58:04.657695] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-03. [2025-10-20T18:58:04.755888] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T18:58:04.945509] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T18:58:05.051341] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-06. [2025-10-20T18:58:05.288072] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T18:58:05.756831] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T18:58:05.851384] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-07. [2025-10-20T18:58:06.245097] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T18:58:06.317344] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T18:58:06.651422] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-03. [2025-10-20T18:58:07.451397] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-08. [2025-10-20T18:58:08.048025] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T18:58:08.262192] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-03. [2025-10-20T18:58:08.268614] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T18:58:09.051422] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-04. [2025-10-20T18:58:09.851383] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-09. [2025-10-20T18:58:10.012742] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T18:58:10.545941] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T18:58:10.651416] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-04. [2025-10-20T18:58:10.798239] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:58:11.451374] [INFO ] [REMOTE ] GFR: Fetching primary data for 2025-10. [2025-10-20T18:58:12.251379] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-04. [2025-10-20T18:58:12.259720] [INFO ] [REMOTE ] GFR: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T18:58:12.261293] [INFO ] [REMOTE ] GFR: Full fetch complete (12050 records). [2025-10-20T18:58:12.261373] [DEBUG] [VALIDATOR ] GFR: Validating primary data. [2025-10-20T18:58:12.604957] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T18:58:12.696111] [DEBUG] [VALIDATOR ] GFR: All expected trading days are present. [2025-10-20T18:58:12.696194] [DEBUG] [VALIDATOR ] GFR: Primary data validation passed. [2025-10-20T18:58:12.696565] [INFO ] [MAIN ] GFR: Fetched primary data is valid. [2025-10-20T18:58:12.696603] [DEBUG] [LOCAL ] GFR: Saving primary data. [2025-10-20T18:58:12.697054] [DEBUG] [LOCAL ] GFR: Trimmed 12050 records to 7510 (keeping 200 days of data) [2025-10-20T18:58:12.746470] [DEBUG] [LOCAL ] GFR: Primary data saved (7510 records). [2025-10-20T18:58:12.746719] [INFO ] [MAIN ] GFR: Full history saved. [2025-10-20T18:58:12.746833] [INFO ] [MAIN ] Update Progress: 921/2367 symbols completed. GFR took 14.0s. Estimated remaining: 3h 5m 29s. Rate: 7.8 symbols/min [2025-10-20T18:58:12.927471] [INFO ] [MAIN ] GGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:12.927674] [DEBUG] [LOCAL ] GGT: No existing data file found in primary storage. [2025-10-20T18:58:12.927728] [WARN ] [MAIN ] GGT: No existing data found, performing full refresh instead. [2025-10-20T18:58:12.927836] [INFO ] [MAIN ] GGT: Wiping existing primary data before full refresh. [2025-10-20T18:58:12.927877] [INFO ] [LOCAL ] GGT: Starting deletion of primary data. [2025-10-20T18:58:12.927917] [DEBUG] [LOCAL ] GGT: No primary data file found to delete. [2025-10-20T18:58:12.927948] [INFO ] [MAIN ] GGT: Starting full primary refresh. [2025-10-20T18:58:13.066367] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-05. [2025-10-20T18:58:13.285620] [INFO ] [REMOTE ] GGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:13.851386] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-01. [2025-10-20T18:58:13.997403] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T18:58:14.644492] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T18:58:14.652859] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-05. [2025-10-20T18:58:14.659556] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T18:58:15.451411] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-06. [2025-10-20T18:58:16.251419] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-02. [2025-10-20T18:58:16.369387] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T18:58:16.915334] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T18:58:17.047674] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T18:58:17.051324] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-07. [2025-10-20T18:58:17.851437] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-06. [2025-10-20T18:58:18.651396] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-05. [2025-10-20T18:58:18.701872] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T18:58:19.451364] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-03. [2025-10-20T18:58:19.505155] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T18:58:20.019550] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T18:58:20.251368] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-07. [2025-10-20T18:58:20.335536] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T18:58:21.051376] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-04. [2025-10-20T18:58:21.851448] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-06. [2025-10-20T18:58:21.874329] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T18:58:22.129391] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T18:58:22.651454] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-08. [2025-10-20T18:58:23.331059] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T18:58:23.451356] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-07. [2025-10-20T18:58:23.519410] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T18:58:24.251417] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-08. [2025-10-20T18:58:24.699202] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T18:58:25.051364] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-08. [2025-10-20T18:58:25.846066] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T18:58:25.856400] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-05. [2025-10-20T18:58:26.625465] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T18:58:26.651444] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-09. [2025-10-20T18:58:26.692733] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T18:58:27.451400] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-06. [2025-10-20T18:58:28.077889] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T18:58:28.251371] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-09. [2025-10-20T18:58:28.358793] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T18:58:29.051362] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-07. [2025-10-20T18:58:29.851963] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T18:58:29.851627] [INFO ] [REMOTE ] GGG: Fetching primary data for 2025-10. [2025-10-20T18:58:29.956213] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T18:58:30.651403] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-08. [2025-10-20T18:58:31.439674] [INFO ] [REMOTE ] GGG: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T18:58:31.445501] [INFO ] [REMOTE ] GGG: Full fetch complete (59462 records). [2025-10-20T18:58:31.445691] [DEBUG] [VALIDATOR ] GGG: Validating primary data. [2025-10-20T18:58:31.457355] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-09. [2025-10-20T18:58:31.577151] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T18:58:31.852318] [DEBUG] [VALIDATOR ] GGG: All expected trading days are present. [2025-10-20T18:58:31.852404] [DEBUG] [VALIDATOR ] GGG: Primary data validation passed. [2025-10-20T18:58:31.853203] [INFO ] [MAIN ] GGG: Fetched primary data is valid. [2025-10-20T18:58:31.853264] [DEBUG] [LOCAL ] GGG: Saving primary data. [2025-10-20T18:58:31.854476] [DEBUG] [LOCAL ] GGG: Trimmed 59462 records to 40585 (keeping 200 days of data) [2025-10-20T18:58:32.105568] [DEBUG] [LOCAL ] GGG: Primary data saved (40585 records). [2025-10-20T18:58:32.105718] [INFO ] [MAIN ] GGG: Full history saved. [2025-10-20T18:58:32.105802] [INFO ] [MAIN ] Update Progress: 922/2367 symbols completed. GGG took 33.3s. Estimated remaining: 3h 5m 40s. Rate: 7.8 symbols/min [2025-10-20T18:58:32.257245] [INFO ] [REMOTE ] GGB: Fetching primary data for 2025-10. [2025-10-20T18:58:32.276980] [INFO ] [MAIN ] GGZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:32.281032] [DEBUG] [LOCAL ] GGZ: Loaded 2448 existing records from primary storage. [2025-10-20T18:58:32.281210] [INFO ] [MAIN ] GGZ: Loaded 2448 existing records. [2025-10-20T18:58:32.460281] [WARN ] [MAIN ] GGZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:32.460523] [INFO ] [MAIN ] GGZ: Wiping existing primary data before full refresh. [2025-10-20T18:58:32.460581] [INFO ] [LOCAL ] GGZ: Starting deletion of primary data. [2025-10-20T18:58:32.460736] [DEBUG] [LOCAL ] GGZ: Deleted primary data file. [2025-10-20T18:58:32.460794] [INFO ] [MAIN ] GGZ: Starting full primary refresh. [2025-10-20T18:58:32.813773] [INFO ] [REMOTE ] GGZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:33.051344] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-01. [2025-10-20T18:58:33.262034] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T18:58:33.688715] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T18:58:33.851352] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-02. [2025-10-20T18:58:34.131975] [INFO ] [REMOTE ] GGB: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T18:58:34.139506] [INFO ] [REMOTE ] GGB: Full fetch complete (74358 records). [2025-10-20T18:58:34.139664] [DEBUG] [VALIDATOR ] GGB: Validating primary data. [2025-10-20T18:58:34.243060] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T18:58:34.528396] [DEBUG] [VALIDATOR ] GGB: All expected trading days are present. [2025-10-20T18:58:34.528468] [DEBUG] [VALIDATOR ] GGB: Primary data validation passed. [2025-10-20T18:58:34.529380] [INFO ] [MAIN ] GGB: Fetched primary data is valid. [2025-10-20T18:58:34.529442] [DEBUG] [LOCAL ] GGB: Saving primary data. [2025-10-20T18:58:34.530808] [DEBUG] [LOCAL ] GGB: Trimmed 74358 records to 51536 (keeping 200 days of data) [2025-10-20T18:58:34.690236] [INFO ] [REMOTE ] GFL: Fetching primary data for 2025-10. [2025-10-20T18:58:34.821869] [DEBUG] [LOCAL ] GGB: Primary data saved (51536 records). [2025-10-20T18:58:34.821982] [INFO ] [MAIN ] GGB: Full history saved. [2025-10-20T18:58:34.822484] [INFO ] [MAIN ] Update Progress: 923/2367 symbols completed. GGB took 36.0s. Estimated remaining: 3h 5m 24s. Rate: 7.8 symbols/min [2025-10-20T18:58:34.991554] [INFO ] [MAIN ] GHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:34.997262] [DEBUG] [LOCAL ] GHC: Loaded 2977 existing records from primary storage. [2025-10-20T18:58:34.997449] [INFO ] [MAIN ] GHC: Loaded 2977 existing records. [2025-10-20T18:58:35.175528] [WARN ] [MAIN ] GHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:35.175951] [INFO ] [MAIN ] GHC: Wiping existing primary data before full refresh. [2025-10-20T18:58:35.176003] [INFO ] [LOCAL ] GHC: Starting deletion of primary data. [2025-10-20T18:58:35.176234] [DEBUG] [LOCAL ] GHC: Deleted primary data file. [2025-10-20T18:58:35.176289] [INFO ] [MAIN ] GHC: Starting full primary refresh. [2025-10-20T18:58:35.456572] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-09. [2025-10-20T18:58:35.538764] [INFO ] [REMOTE ] GHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:36.251373] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-01. [2025-10-20T18:58:36.368006] [INFO ] [REMOTE ] GFL: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T18:58:36.377512] [INFO ] [REMOTE ] GFL: Full fetch complete (69606 records). [2025-10-20T18:58:36.377746] [DEBUG] [VALIDATOR ] GFL: Validating primary data. [2025-10-20T18:58:36.512991] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T18:58:36.802480] [DEBUG] [VALIDATOR ] GFL: All expected trading days are present. [2025-10-20T18:58:36.805969] [DEBUG] [VALIDATOR ] GFL: Primary data validation passed. [2025-10-20T18:58:36.806084] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T18:58:36.807153] [INFO ] [MAIN ] GFL: Fetched primary data is valid. [2025-10-20T18:58:36.808364] [DEBUG] [LOCAL ] GFL: Saving primary data. [2025-10-20T18:58:36.809833] [DEBUG] [LOCAL ] GFL: Trimmed 69606 records to 47569 (keeping 200 days of data) [2025-10-20T18:58:37.064528] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-03. [2025-10-20T18:58:37.090444] [DEBUG] [LOCAL ] GFL: Primary data saved (47569 records). [2025-10-20T18:58:37.090752] [INFO ] [MAIN ] GFL: Full history saved. [2025-10-20T18:58:37.090842] [INFO ] [MAIN ] Update Progress: 924/2367 symbols completed. GFL took 38.3s. Estimated remaining: 3h 5m 8s. Rate: 7.8 symbols/min [2025-10-20T18:58:37.255274] [INFO ] [MAIN ] GHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:37.261090] [DEBUG] [LOCAL ] GHG: Loaded 3317 existing records from primary storage. [2025-10-20T18:58:37.261268] [INFO ] [MAIN ] GHG: Loaded 3317 existing records. [2025-10-20T18:58:37.437148] [WARN ] [MAIN ] GHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:37.437332] [INFO ] [MAIN ] GHG: Wiping existing primary data before full refresh. [2025-10-20T18:58:37.437379] [INFO ] [LOCAL ] GHG: Starting deletion of primary data. [2025-10-20T18:58:37.437570] [DEBUG] [LOCAL ] GHG: Deleted primary data file. [2025-10-20T18:58:37.437625] [INFO ] [MAIN ] GHG: Starting full primary refresh. [2025-10-20T18:58:37.713546] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T18:58:37.789623] [INFO ] [REMOTE ] GHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:37.851345] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-01. [2025-10-20T18:58:38.182946] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T18:58:38.651365] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-04. [2025-10-20T18:58:39.261864] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T18:58:39.451382] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-02. [2025-10-20T18:58:39.813028] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:58:40.251369] [INFO ] [REMOTE ] GGT: Fetching primary data for 2025-10. [2025-10-20T18:58:41.051355] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-03. [2025-10-20T18:58:41.238123] [INFO ] [REMOTE ] GGT: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T18:58:41.241570] [INFO ] [REMOTE ] GGT: Full fetch complete (18982 records). [2025-10-20T18:58:41.241682] [DEBUG] [VALIDATOR ] GGT: Validating primary data. [2025-10-20T18:58:41.533026] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T18:58:41.618164] [DEBUG] [VALIDATOR ] GGT: All expected trading days are present. [2025-10-20T18:58:41.618249] [DEBUG] [VALIDATOR ] GGT: Primary data validation passed. [2025-10-20T18:58:41.618779] [INFO ] [MAIN ] GGT: Fetched primary data is valid. [2025-10-20T18:58:41.618841] [DEBUG] [LOCAL ] GGT: Saving primary data. [2025-10-20T18:58:41.619524] [DEBUG] [LOCAL ] GGT: Trimmed 18982 records to 14586 (keeping 200 days of data) [2025-10-20T18:58:41.711835] [DEBUG] [LOCAL ] GGT: Primary data saved (14586 records). [2025-10-20T18:58:41.711996] [INFO ] [MAIN ] GGT: Full history saved. [2025-10-20T18:58:41.712083] [INFO ] [MAIN ] Update Progress: 925/2367 symbols completed. GGT took 29.0s. Estimated remaining: 3h 4m 55s. Rate: 7.8 symbols/min [2025-10-20T18:58:41.854256] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-05. [2025-10-20T18:58:41.932014] [INFO ] [MAIN ] GHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:41.942209] [DEBUG] [LOCAL ] GHI: Loaded 7591 existing records from primary storage. [2025-10-20T18:58:41.942442] [INFO ] [MAIN ] GHI: Loaded 7591 existing records. [2025-10-20T18:58:42.129986] [WARN ] [MAIN ] GHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:42.131096] [INFO ] [MAIN ] GHI: Wiping existing primary data before full refresh. [2025-10-20T18:58:42.131158] [INFO ] [LOCAL ] GHI: Starting deletion of primary data. [2025-10-20T18:58:42.131911] [DEBUG] [LOCAL ] GHI: Deleted primary data file. [2025-10-20T18:58:42.131969] [INFO ] [MAIN ] GHI: Starting full primary refresh. [2025-10-20T18:58:42.457915] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T18:58:42.551075] [INFO ] [REMOTE ] GHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:42.651377] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-06. [2025-10-20T18:58:43.163302] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T18:58:43.451378] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-01. [2025-10-20T18:58:44.222832] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T18:58:44.251374] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-02. [2025-10-20T18:58:44.857082] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T18:58:45.051388] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-07. [2025-10-20T18:58:45.416422] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T18:58:45.851364] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-08. [2025-10-20T18:58:46.272442] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T18:58:46.651368] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-02. [2025-10-20T18:58:47.190366] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T18:58:47.451370] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-09. [2025-10-20T18:58:47.896146] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T18:58:48.251385] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-03. [2025-10-20T18:58:48.976832] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T18:58:49.051396] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-04. [2025-10-20T18:58:49.667502] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T18:58:49.851357] [INFO ] [REMOTE ] GGZ: Fetching primary data for 2025-10. [2025-10-20T18:58:50.468366] [INFO ] [REMOTE ] GGZ: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:58:50.469344] [INFO ] [REMOTE ] GGZ: Full fetch complete (3895 records). [2025-10-20T18:58:50.469413] [DEBUG] [VALIDATOR ] GGZ: Validating primary data. [2025-10-20T18:58:50.659156] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-04. [2025-10-20T18:58:50.831546] [DEBUG] [VALIDATOR ] GGZ: All expected trading days are present. [2025-10-20T18:58:50.831623] [DEBUG] [VALIDATOR ] GGZ: Primary data validation passed. [2025-10-20T18:58:50.831782] [INFO ] [MAIN ] GGZ: Fetched primary data is valid. [2025-10-20T18:58:50.831838] [DEBUG] [LOCAL ] GGZ: Saving primary data. [2025-10-20T18:58:50.832182] [DEBUG] [LOCAL ] GGZ: Trimmed 3895 records to 2412 (keeping 200 days of data) [2025-10-20T18:58:50.846787] [DEBUG] [LOCAL ] GGZ: Primary data saved (2412 records). [2025-10-20T18:58:50.846850] [INFO ] [MAIN ] GGZ: Full history saved. [2025-10-20T18:58:50.846914] [INFO ] [MAIN ] Update Progress: 926/2367 symbols completed. GGZ took 18.7s. Estimated remaining: 3h 4m 50s. Rate: 7.8 symbols/min [2025-10-20T18:58:51.012981] [INFO ] [MAIN ] GHLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:58:51.018673] [DEBUG] [LOCAL ] GHLD: Loaded 4348 existing records from primary storage. [2025-10-20T18:58:51.018855] [INFO ] [MAIN ] GHLD: Loaded 4348 existing records. [2025-10-20T18:58:51.192999] [WARN ] [MAIN ] GHLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:58:51.193218] [INFO ] [MAIN ] GHLD: Wiping existing primary data before full refresh. [2025-10-20T18:58:51.193263] [INFO ] [LOCAL ] GHLD: Starting deletion of primary data. [2025-10-20T18:58:51.193417] [DEBUG] [LOCAL ] GHLD: Deleted primary data file. [2025-10-20T18:58:51.193471] [INFO ] [MAIN ] GHLD: Starting full primary refresh. [2025-10-20T18:58:51.455757] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-03. [2025-10-20T18:58:51.534982] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T18:58:51.564805] [INFO ] [REMOTE ] GHLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:58:51.995527] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T18:58:52.251421] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-01. [2025-10-20T18:58:52.701853] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T18:58:53.051412] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-04. [2025-10-20T18:58:53.605423] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T18:58:53.851429] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-05. [2025-10-20T18:58:54.475736] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T18:58:54.651432] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-05. [2025-10-20T18:58:55.167538] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T18:58:55.451369] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-06. [2025-10-20T18:58:56.122376] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T18:58:56.251371] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-05. [2025-10-20T18:58:56.684977] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T18:58:57.051378] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-06. [2025-10-20T18:58:57.492694] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T18:58:57.851367] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-07. [2025-10-20T18:58:58.472471] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T18:58:58.651359] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-08. [2025-10-20T18:58:59.134129] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T18:58:59.451366] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-02. [2025-10-20T18:58:59.810529] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T18:59:00.251374] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-06. [2025-10-20T18:59:00.891692] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T18:59:01.051414] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-07. [2025-10-20T18:59:01.851410] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-03. [2025-10-20T18:59:01.902134] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T18:59:02.207562] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-03 (0.36 seconds). [2025-10-20T18:59:02.651405] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-07. [2025-10-20T18:59:03.161775] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T18:59:03.451409] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-09. [2025-10-20T18:59:03.883714] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T18:59:04.251404] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-04. [2025-10-20T18:59:04.852687] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T18:59:05.051377] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-08. [2025-10-20T18:59:05.789554] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T18:59:05.851382] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-09. [2025-10-20T18:59:06.651350] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-05. [2025-10-20T18:59:06.718280] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T18:59:07.140535] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T18:59:07.451387] [INFO ] [REMOTE ] GHG: Fetching primary data for 2025-10. [2025-10-20T18:59:08.070256] [INFO ] [REMOTE ] GHG: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T18:59:08.071328] [INFO ] [REMOTE ] GHG: Full fetch complete (4168 records). [2025-10-20T18:59:08.071396] [DEBUG] [VALIDATOR ] GHG: Validating primary data. [2025-10-20T18:59:08.257008] [INFO ] [REMOTE ] GHI: Fetching primary data for 2025-10. [2025-10-20T18:59:08.437166] [DEBUG] [VALIDATOR ] GHG: All expected trading days are present. [2025-10-20T18:59:08.437251] [DEBUG] [VALIDATOR ] GHG: Primary data validation passed. [2025-10-20T18:59:08.437393] [INFO ] [MAIN ] GHG: Fetched primary data is valid. [2025-10-20T18:59:08.437436] [DEBUG] [LOCAL ] GHG: Saving primary data. [2025-10-20T18:59:08.437849] [DEBUG] [LOCAL ] GHG: Trimmed 4168 records to 3282 (keeping 200 days of data) [2025-10-20T18:59:08.455772] [DEBUG] [LOCAL ] GHG: Primary data saved (3282 records). [2025-10-20T18:59:08.455842] [INFO ] [MAIN ] GHG: Full history saved. [2025-10-20T18:59:08.455911] [INFO ] [MAIN ] Update Progress: 927/2367 symbols completed. GHG took 31.4s. Estimated remaining: 3h 4m 58s. Rate: 7.8 symbols/min [2025-10-20T18:59:08.626460] [INFO ] [MAIN ] GHM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:08.641889] [DEBUG] [LOCAL ] GHM: Loaded 14580 existing records from primary storage. [2025-10-20T18:59:08.642116] [INFO ] [MAIN ] GHM: Loaded 14580 existing records. [2025-10-20T18:59:08.820118] [WARN ] [MAIN ] GHM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:08.820348] [INFO ] [MAIN ] GHM: Wiping existing primary data before full refresh. [2025-10-20T18:59:08.820398] [INFO ] [LOCAL ] GHM: Starting deletion of primary data. [2025-10-20T18:59:08.821241] [DEBUG] [LOCAL ] GHM: Deleted primary data file. [2025-10-20T18:59:08.821301] [INFO ] [MAIN ] GHM: Starting full primary refresh. [2025-10-20T18:59:09.059108] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-06. [2025-10-20T18:59:09.176020] [INFO ] [REMOTE ] GHM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:09.337827] [INFO ] [REMOTE ] GHI: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T18:59:09.340019] [INFO ] [REMOTE ] GHI: Full fetch complete (11152 records). [2025-10-20T18:59:09.340089] [DEBUG] [VALIDATOR ] GHI: Validating primary data. [2025-10-20T18:59:09.718865] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T18:59:09.719260] [DEBUG] [VALIDATOR ] GHI: All expected trading days are present. [2025-10-20T18:59:09.721618] [DEBUG] [VALIDATOR ] GHI: Primary data validation passed. [2025-10-20T18:59:09.722013] [INFO ] [MAIN ] GHI: Fetched primary data is valid. [2025-10-20T18:59:09.722063] [DEBUG] [LOCAL ] GHI: Saving primary data. [2025-10-20T18:59:09.722553] [DEBUG] [LOCAL ] GHI: Trimmed 11152 records to 7558 (keeping 200 days of data) [2025-10-20T18:59:09.767005] [DEBUG] [LOCAL ] GHI: Primary data saved (7558 records). [2025-10-20T18:59:09.767119] [INFO ] [MAIN ] GHI: Full history saved. [2025-10-20T18:59:09.767193] [INFO ] [MAIN ] Update Progress: 928/2367 symbols completed. GHI took 28.1s. Estimated remaining: 3h 4m 40s. Rate: 7.8 symbols/min [2025-10-20T18:59:09.856549] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-08. [2025-10-20T18:59:09.937753] [INFO ] [MAIN ] GHY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:09.959958] [DEBUG] [LOCAL ] GHY: Loaded 21330 existing records from primary storage. [2025-10-20T18:59:09.960206] [INFO ] [MAIN ] GHY: Loaded 21330 existing records. [2025-10-20T18:59:10.139109] [WARN ] [MAIN ] GHY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:10.139402] [INFO ] [MAIN ] GHY: Wiping existing primary data before full refresh. [2025-10-20T18:59:10.139454] [INFO ] [LOCAL ] GHY: Starting deletion of primary data. [2025-10-20T18:59:10.139691] [DEBUG] [LOCAL ] GHY: Deleted primary data file. [2025-10-20T18:59:10.139747] [INFO ] [MAIN ] GHY: Starting full primary refresh. [2025-10-20T18:59:10.407183] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T18:59:10.539336] [INFO ] [REMOTE ] GHY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:10.651409] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-01. [2025-10-20T18:59:11.451373] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-09. [2025-10-20T18:59:11.813481] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T18:59:12.047030] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T18:59:12.251365] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-07. [2025-10-20T18:59:12.822594] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T18:59:13.051388] [INFO ] [REMOTE ] GHC: Fetching primary data for 2025-10. [2025-10-20T18:59:13.839267] [INFO ] [REMOTE ] GHC: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T18:59:13.840455] [INFO ] [REMOTE ] GHC: Full fetch complete (3659 records). [2025-10-20T18:59:13.840523] [DEBUG] [VALIDATOR ] GHC: Validating primary data. [2025-10-20T18:59:13.861155] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-08. [2025-10-20T18:59:14.204821] [DEBUG] [VALIDATOR ] GHC: All expected trading days are present. [2025-10-20T18:59:14.204892] [DEBUG] [VALIDATOR ] GHC: Primary data validation passed. [2025-10-20T18:59:14.205018] [INFO ] [MAIN ] GHC: Fetched primary data is valid. [2025-10-20T18:59:14.205056] [DEBUG] [LOCAL ] GHC: Saving primary data. [2025-10-20T18:59:14.205402] [DEBUG] [LOCAL ] GHC: Trimmed 3659 records to 2956 (keeping 200 days of data) [2025-10-20T18:59:14.224686] [DEBUG] [LOCAL ] GHC: Primary data saved (2956 records). [2025-10-20T18:59:14.224757] [INFO ] [MAIN ] GHC: Full history saved. [2025-10-20T18:59:14.224824] [INFO ] [MAIN ] Update Progress: 929/2367 symbols completed. GHC took 39.4s. Estimated remaining: 3h 4m 27s. Rate: 7.8 symbols/min [2025-10-20T18:59:14.401012] [INFO ] [MAIN ] GIB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:14.433201] [DEBUG] [LOCAL ] GIB: Loaded 32143 existing records from primary storage. [2025-10-20T18:59:14.433496] [INFO ] [MAIN ] GIB: Loaded 32143 existing records. [2025-10-20T18:59:14.633354] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T18:59:14.627605] [WARN ] [MAIN ] GIB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:14.633687] [INFO ] [MAIN ] GIB: Wiping existing primary data before full refresh. [2025-10-20T18:59:14.636080] [INFO ] [LOCAL ] GIB: Starting deletion of primary data. [2025-10-20T18:59:14.636303] [DEBUG] [LOCAL ] GIB: Deleted primary data file. [2025-10-20T18:59:14.636359] [INFO ] [MAIN ] GIB: Starting full primary refresh. [2025-10-20T18:59:14.656503] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-09. [2025-10-20T18:59:15.008224] [INFO ] [REMOTE ] GIB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:15.307551] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T18:59:15.451372] [INFO ] [REMOTE ] GHLD: Fetching primary data for 2025-10. [2025-10-20T18:59:16.172151] [INFO ] [REMOTE ] GHLD: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T18:59:16.173344] [INFO ] [REMOTE ] GHLD: Full fetch complete (5084 records). [2025-10-20T18:59:16.173409] [DEBUG] [VALIDATOR ] GHLD: Validating primary data. [2025-10-20T18:59:16.256610] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-01. [2025-10-20T18:59:16.533314] [DEBUG] [VALIDATOR ] GHLD: All expected trading days are present. [2025-10-20T18:59:16.533391] [DEBUG] [VALIDATOR ] GHLD: Primary data validation passed. [2025-10-20T18:59:16.533567] [INFO ] [MAIN ] GHLD: Fetched primary data is valid. [2025-10-20T18:59:16.533605] [DEBUG] [LOCAL ] GHLD: Saving primary data. [2025-10-20T18:59:16.533993] [DEBUG] [LOCAL ] GHLD: Trimmed 5084 records to 4301 (keeping 200 days of data) [2025-10-20T18:59:16.558844] [DEBUG] [LOCAL ] GHLD: Primary data saved (4301 records). [2025-10-20T18:59:16.558918] [INFO ] [MAIN ] GHLD: Full history saved. [2025-10-20T18:59:16.559009] [INFO ] [MAIN ] Update Progress: 930/2367 symbols completed. GHLD took 25.7s. Estimated remaining: 3h 4m 11s. Rate: 7.8 symbols/min [2025-10-20T18:59:16.725507] [INFO ] [MAIN ] GIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:16.739085] [DEBUG] [LOCAL ] GIC: Loaded 13587 existing records from primary storage. [2025-10-20T18:59:16.739277] [INFO ] [MAIN ] GIC: Loaded 13587 existing records. [2025-10-20T18:59:16.917763] [WARN ] [MAIN ] GIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:16.918001] [INFO ] [MAIN ] GIC: Wiping existing primary data before full refresh. [2025-10-20T18:59:16.918047] [INFO ] [LOCAL ] GIC: Starting deletion of primary data. [2025-10-20T18:59:16.918247] [DEBUG] [LOCAL ] GIC: Deleted primary data file. [2025-10-20T18:59:16.918283] [INFO ] [MAIN ] GIC: Starting full primary refresh. [2025-10-20T18:59:17.058727] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-01. [2025-10-20T18:59:17.292168] [INFO ] [REMOTE ] GIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:17.292850] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T18:59:17.851387] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-01. [2025-10-20T18:59:18.385826] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T18:59:18.651380] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-02. [2025-10-20T18:59:18.842187] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T18:59:19.451348] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-02. [2025-10-20T18:59:19.695190] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T18:59:20.251356] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-02. [2025-10-20T18:59:20.415756] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T18:59:21.051375] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-03. [2025-10-20T18:59:21.231035] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T18:59:21.851359] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-03. [2025-10-20T18:59:22.049678] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T18:59:22.651400] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-02. [2025-10-20T18:59:22.982492] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T18:59:23.451365] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-04. [2025-10-20T18:59:23.975784] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T18:59:24.251413] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-03. [2025-10-20T18:59:24.515772] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:59:25.051384] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-03. [2025-10-20T18:59:25.724314] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T18:59:25.771634] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T18:59:25.851391] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-05. [2025-10-20T18:59:26.651401] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-04. [2025-10-20T18:59:26.934717] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T18:59:27.451375] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-06. [2025-10-20T18:59:27.715715] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T18:59:28.251428] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-04. [2025-10-20T18:59:28.368943] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T18:59:29.051402] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-04. [2025-10-20T18:59:29.736884] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T18:59:29.851346] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-05. [2025-10-20T18:59:29.905366] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T18:59:30.651442] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-07. [2025-10-20T18:59:31.188337] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T18:59:31.451356] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-05. [2025-10-20T18:59:31.749560] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T18:59:32.251487] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-05. [2025-10-20T18:59:32.277700] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T18:59:33.050873] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T18:59:33.053661] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-08. [2025-10-20T18:59:33.851379] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-06. [2025-10-20T18:59:34.095393] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T18:59:34.651353] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-09. [2025-10-20T18:59:34.991074] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T18:59:35.451401] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-06. [2025-10-20T18:59:35.718804] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T18:59:36.251440] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-07. [2025-10-20T18:59:36.711506] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T18:59:37.051409] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-06. [2025-10-20T18:59:37.146425] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T18:59:37.851379] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-08. [2025-10-20T18:59:37.898024] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T18:59:38.651347] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-07. [2025-10-20T18:59:38.809939] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T18:59:39.451346] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-09. [2025-10-20T18:59:39.463573] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T18:59:40.251355] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-08. [2025-10-20T18:59:40.333022] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T18:59:41.051338] [INFO ] [REMOTE ] GHM: Fetching primary data for 2025-10. [2025-10-20T18:59:41.335883] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T18:59:41.851371] [INFO ] [REMOTE ] GHY: Fetching primary data for 2025-10. [2025-10-20T18:59:42.052698] [INFO ] [REMOTE ] GHM: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T18:59:42.055705] [INFO ] [REMOTE ] GHM: Full fetch complete (22030 records). [2025-10-20T18:59:42.055788] [DEBUG] [VALIDATOR ] GHM: Validating primary data. [2025-10-20T18:59:42.456816] [DEBUG] [VALIDATOR ] GHM: All expected trading days are present. [2025-10-20T18:59:42.456886] [DEBUG] [VALIDATOR ] GHM: Primary data validation passed. [2025-10-20T18:59:42.457449] [INFO ] [MAIN ] GHM: Fetched primary data is valid. [2025-10-20T18:59:42.457529] [DEBUG] [LOCAL ] GHM: Saving primary data. [2025-10-20T18:59:42.458218] [DEBUG] [LOCAL ] GHM: Trimmed 22030 records to 14369 (keeping 200 days of data) [2025-10-20T18:59:42.540365] [DEBUG] [LOCAL ] GHM: Primary data saved (14369 records). [2025-10-20T18:59:42.540494] [INFO ] [MAIN ] GHM: Full history saved. [2025-10-20T18:59:42.540569] [INFO ] [MAIN ] Update Progress: 931/2367 symbols completed. GHM took 34.1s. Estimated remaining: 3h 4m 32s. Rate: 7.8 symbols/min [2025-10-20T18:59:42.656570] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-09. [2025-10-20T18:59:42.708589] [INFO ] [MAIN ] GIL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:42.749140] [DEBUG] [LOCAL ] GIL: Loaded 44482 existing records from primary storage. [2025-10-20T18:59:42.749450] [INFO ] [MAIN ] GIL: Loaded 44482 existing records. [2025-10-20T18:59:42.926323] [WARN ] [MAIN ] GIL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:42.926505] [INFO ] [MAIN ] GIL: Wiping existing primary data before full refresh. [2025-10-20T18:59:42.926552] [INFO ] [LOCAL ] GIL: Starting deletion of primary data. [2025-10-20T18:59:42.926753] [DEBUG] [LOCAL ] GIL: Deleted primary data file. [2025-10-20T18:59:42.926814] [INFO ] [MAIN ] GIL: Starting full primary refresh. [2025-10-20T18:59:43.112591] [INFO ] [REMOTE ] GHY: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T18:59:43.196546] [INFO ] [REMOTE ] GHY: Full fetch complete (32455 records). [2025-10-20T18:59:43.212368] [DEBUG] [VALIDATOR ] GHY: Validating primary data. [2025-10-20T18:59:43.423165] [INFO ] [REMOTE ] GIL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:43.463816] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-07. [2025-10-20T18:59:43.700170] [DEBUG] [VALIDATOR ] GHY: All expected trading days are present. [2025-10-20T18:59:43.700251] [DEBUG] [VALIDATOR ] GHY: Primary data validation passed. [2025-10-20T18:59:43.701004] [INFO ] [MAIN ] GHY: Fetched primary data is valid. [2025-10-20T18:59:43.701066] [DEBUG] [LOCAL ] GHY: Saving primary data. [2025-10-20T18:59:43.702014] [DEBUG] [LOCAL ] GHY: Trimmed 32455 records to 20931 (keeping 200 days of data) [2025-10-20T18:59:43.795399] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T18:59:43.839921] [DEBUG] [LOCAL ] GHY: Primary data saved (20931 records). [2025-10-20T18:59:43.840060] [INFO ] [MAIN ] GHY: Full history saved. [2025-10-20T18:59:43.840137] [INFO ] [MAIN ] Update Progress: 932/2367 symbols completed. GHY took 34.1s. Estimated remaining: 3h 4m 14s. Rate: 7.8 symbols/min [2025-10-20T18:59:44.016866] [INFO ] [MAIN ] GIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:44.067447] [DEBUG] [LOCAL ] GIS: Loaded 54452 existing records from primary storage. [2025-10-20T18:59:44.067809] [INFO ] [MAIN ] GIS: Loaded 54452 existing records. [2025-10-20T18:59:44.241693] [WARN ] [MAIN ] GIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:44.241900] [INFO ] [MAIN ] GIS: Wiping existing primary data before full refresh. [2025-10-20T18:59:44.241946] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-10-20T18:59:44.242137] [DEBUG] [LOCAL ] GIS: Deleted primary data file. [2025-10-20T18:59:44.242174] [INFO ] [MAIN ] GIS: Starting full primary refresh. [2025-10-20T18:59:44.256614] [INFO ] [REMOTE ] GIC: Fetching primary data for 2025-10. [2025-10-20T18:59:44.611608] [INFO ] [REMOTE ] GIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:44.904383] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T18:59:45.051357] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-01. [2025-10-20T18:59:45.167459] [INFO ] [REMOTE ] GIC: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T18:59:45.169607] [INFO ] [REMOTE ] GIC: Full fetch complete (19871 records). [2025-10-20T18:59:45.169733] [DEBUG] [VALIDATOR ] GIC: Validating primary data. [2025-10-20T18:59:45.527903] [DEBUG] [VALIDATOR ] GIC: All expected trading days are present. [2025-10-20T18:59:45.527978] [DEBUG] [VALIDATOR ] GIC: Primary data validation passed. [2025-10-20T18:59:45.528389] [INFO ] [MAIN ] GIC: Fetched primary data is valid. [2025-10-20T18:59:45.528427] [DEBUG] [LOCAL ] GIC: Saving primary data. [2025-10-20T18:59:45.529008] [DEBUG] [LOCAL ] GIC: Trimmed 19871 records to 13388 (keeping 200 days of data) [2025-10-20T18:59:45.609443] [DEBUG] [LOCAL ] GIC: Primary data saved (13388 records). [2025-10-20T18:59:45.609589] [INFO ] [MAIN ] GIC: Full history saved. [2025-10-20T18:59:45.609697] [INFO ] [MAIN ] Update Progress: 933/2367 symbols completed. GIC took 29.1s. Estimated remaining: 3h 3m 57s. Rate: 7.8 symbols/min [2025-10-20T18:59:45.782177] [INFO ] [MAIN ] GKOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:45.822749] [DEBUG] [LOCAL ] GKOS: Loaded 43111 existing records from primary storage. [2025-10-20T18:59:45.823081] [INFO ] [MAIN ] GKOS: Loaded 43111 existing records. [2025-10-20T18:59:45.858050] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-08. [2025-10-20T18:59:46.017264] [WARN ] [MAIN ] GKOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:46.017502] [INFO ] [MAIN ] GKOS: Wiping existing primary data before full refresh. [2025-10-20T18:59:46.017549] [INFO ] [LOCAL ] GKOS: Starting deletion of primary data. [2025-10-20T18:59:46.017971] [DEBUG] [LOCAL ] GKOS: Deleted primary data file. [2025-10-20T18:59:46.018026] [INFO ] [MAIN ] GKOS: Starting full primary refresh. [2025-10-20T18:59:46.366703] [INFO ] [REMOTE ] GKOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:46.651375] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-01. [2025-10-20T18:59:47.057691] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T18:59:47.339761] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T18:59:47.451372] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-09. [2025-10-20T18:59:48.168935] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T18:59:48.251373] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-02. [2025-10-20T18:59:48.778195] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T18:59:49.051416] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-01. [2025-10-20T18:59:49.851367] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-02. [2025-10-20T18:59:50.079525] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T18:59:50.279403] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T18:59:50.651387] [INFO ] [REMOTE ] GIB: Fetching primary data for 2025-10. [2025-10-20T18:59:51.192817] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T18:59:51.451379] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-03. [2025-10-20T18:59:52.106848] [INFO ] [REMOTE ] GIB: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T18:59:52.111557] [INFO ] [REMOTE ] GIB: Full fetch complete (46256 records). [2025-10-20T18:59:52.111687] [DEBUG] [VALIDATOR ] GIB: Validating primary data. [2025-10-20T18:59:52.256496] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-03. [2025-10-20T18:59:52.492918] [DEBUG] [VALIDATOR ] GIB: All expected trading days are present. [2025-10-20T18:59:52.492999] [DEBUG] [VALIDATOR ] GIB: Primary data validation passed. [2025-10-20T18:59:52.493727] [INFO ] [MAIN ] GIB: Fetched primary data is valid. [2025-10-20T18:59:52.493787] [DEBUG] [LOCAL ] GIB: Saving primary data. [2025-10-20T18:59:52.494746] [DEBUG] [LOCAL ] GIB: Trimmed 46256 records to 31496 (keeping 200 days of data) [2025-10-20T18:59:52.683731] [DEBUG] [LOCAL ] GIB: Primary data saved (31496 records). [2025-10-20T18:59:52.683848] [INFO ] [MAIN ] GIB: Full history saved. [2025-10-20T18:59:52.684425] [INFO ] [MAIN ] Update Progress: 934/2367 symbols completed. GIB took 38.5s. Estimated remaining: 3h 3m 49s. Rate: 7.8 symbols/min [2025-10-20T18:59:52.854438] [INFO ] [MAIN ] GL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T18:59:52.892826] [DEBUG] [LOCAL ] GL: Loaded 38100 existing records from primary storage. [2025-10-20T18:59:52.893185] [INFO ] [MAIN ] GL: Loaded 38100 existing records. [2025-10-20T18:59:53.065814] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-02. [2025-10-20T18:59:53.070564] [WARN ] [MAIN ] GL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T18:59:53.071951] [INFO ] [MAIN ] GL: Wiping existing primary data before full refresh. [2025-10-20T18:59:53.072109] [INFO ] [LOCAL ] GL: Starting deletion of primary data. [2025-10-20T18:59:53.072472] [DEBUG] [LOCAL ] GL: Deleted primary data file. [2025-10-20T18:59:53.072546] [INFO ] [MAIN ] GL: Starting full primary refresh. [2025-10-20T18:59:53.391997] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T18:59:53.484602] [INFO ] [REMOTE ] GL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T18:59:53.509313] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T18:59:53.851414] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-04. [2025-10-20T18:59:54.586004] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T18:59:54.651384] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-03. [2025-10-20T18:59:55.451372] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-04. [2025-10-20T18:59:55.506211] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T18:59:56.220317] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T18:59:56.251353] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-04. [2025-10-20T18:59:57.051388] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-01. [2025-10-20T18:59:57.480090] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T18:59:57.851437] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-05. [2025-10-20T18:59:57.895065] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T18:59:58.621863] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T18:59:58.651329] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-02. [2025-10-20T18:59:59.366941] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T18:59:59.451372] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-06. [2025-10-20T19:00:00.064408] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:00:00.251379] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-05. [2025-10-20T19:00:01.032018] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:00:01.051335] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-05. [2025-10-20T19:00:01.851362] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-03. [2025-10-20T19:00:02.147308] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:00:02.651393] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-07. [2025-10-20T19:00:03.447971] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:00:03.456231] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-06. [2025-10-20T19:00:04.251394] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-04. [2025-10-20T19:00:04.342888] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-05 (3.29 seconds). [2025-10-20T19:00:04.548930] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T19:00:05.051348] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-06. [2025-10-20T19:00:05.514979] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T19:00:05.851464] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-07. [2025-10-20T19:00:06.651495] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-08. [2025-10-20T19:00:07.314433] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-06 (2.26 seconds). [2025-10-20T19:00:07.351936] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-04 (3.10 seconds). [2025-10-20T19:00:07.451352] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-05. [2025-10-20T19:00:07.625988] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:00:08.251415] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-07. [2025-10-20T19:00:08.427828] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:00:08.872008] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:00:09.051443] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-06. [2025-10-20T19:00:09.851370] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-09. [2025-10-20T19:00:10.305073] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:00:10.476666] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T19:00:10.651364] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-08. [2025-10-20T19:00:11.436337] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:00:11.451335] [INFO ] [REMOTE ] GIL: Fetching primary data for 2025-10. [2025-10-20T19:00:11.966952] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T19:00:12.251354] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-09. [2025-10-20T19:00:12.665197] [INFO ] [REMOTE ] GIL: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:00:12.672044] [INFO ] [REMOTE ] GIL: Full fetch complete (62443 records). [2025-10-20T19:00:12.672164] [DEBUG] [VALIDATOR ] GIL: Validating primary data. [2025-10-20T19:00:13.054172] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-08. [2025-10-20T19:00:13.055378] [DEBUG] [VALIDATOR ] GIL: All expected trading days are present. [2025-10-20T19:00:13.055460] [DEBUG] [VALIDATOR ] GIL: Primary data validation passed. [2025-10-20T19:00:13.056293] [INFO ] [MAIN ] GIL: Fetched primary data is valid. [2025-10-20T19:00:13.056355] [DEBUG] [LOCAL ] GIL: Saving primary data. [2025-10-20T19:00:13.057573] [DEBUG] [LOCAL ] GIL: Trimmed 62443 records to 43989 (keeping 200 days of data) [2025-10-20T19:00:13.358815] [DEBUG] [LOCAL ] GIL: Primary data saved (43989 records). [2025-10-20T19:00:13.359118] [INFO ] [MAIN ] GIL: Full history saved. [2025-10-20T19:00:13.367611] [INFO ] [MAIN ] Update Progress: 935/2367 symbols completed. GIL took 30.8s. Estimated remaining: 3h 4m 1s. Rate: 7.8 symbols/min [2025-10-20T19:00:13.401484] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:00:13.573175] [INFO ] [MAIN ] GLOB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:13.619885] [DEBUG] [LOCAL ] GLOB: Loaded 48655 existing records from primary storage. [2025-10-20T19:00:13.620196] [INFO ] [MAIN ] GLOB: Loaded 48655 existing records. [2025-10-20T19:00:13.796476] [WARN ] [MAIN ] GLOB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:13.796639] [INFO ] [MAIN ] GLOB: Wiping existing primary data before full refresh. [2025-10-20T19:00:13.796732] [INFO ] [LOCAL ] GLOB: Starting deletion of primary data. [2025-10-20T19:00:13.796960] [DEBUG] [LOCAL ] GLOB: Deleted primary data file. [2025-10-20T19:00:13.797017] [INFO ] [MAIN ] GLOB: Starting full primary refresh. [2025-10-20T19:00:13.865738] [INFO ] [REMOTE ] GKOS: Fetching primary data for 2025-10. [2025-10-20T19:00:14.187432] [INFO ] [REMOTE ] GLOB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:14.651349] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-07. [2025-10-20T19:00:15.186882] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T19:00:15.451362] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-09. [2025-10-20T19:00:16.251369] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-01. [2025-10-20T19:00:16.366467] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:00:17.051358] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-08. [2025-10-20T19:00:17.127573] [INFO ] [REMOTE ] GKOS: Incremental fetch complete for 2025-10 (3.26 seconds). [2025-10-20T19:00:17.133799] [INFO ] [REMOTE ] GKOS: Full fetch complete (60424 records). [2025-10-20T19:00:17.133904] [DEBUG] [VALIDATOR ] GKOS: Validating primary data. [2025-10-20T19:00:17.530867] [DEBUG] [VALIDATOR ] GKOS: All expected trading days are present. [2025-10-20T19:00:17.531111] [DEBUG] [VALIDATOR ] GKOS: Primary data validation passed. [2025-10-20T19:00:17.532189] [INFO ] [MAIN ] GKOS: Fetched primary data is valid. [2025-10-20T19:00:17.532397] [DEBUG] [LOCAL ] GKOS: Saving primary data. [2025-10-20T19:00:17.534043] [DEBUG] [LOCAL ] GKOS: Trimmed 60424 records to 42502 (keeping 200 days of data) [2025-10-20T19:00:17.656150] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T19:00:17.806418] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:00:17.852844] [INFO ] [REMOTE ] GIS: Fetching primary data for 2025-10. [2025-10-20T19:00:17.858183] [DEBUG] [LOCAL ] GKOS: Primary data saved (42502 records). [2025-10-20T19:00:17.858324] [INFO ] [MAIN ] GKOS: Full history saved. [2025-10-20T19:00:17.858403] [INFO ] [MAIN ] Update Progress: 936/2367 symbols completed. GKOS took 32.2s. Estimated remaining: 3h 3m 48s. Rate: 7.8 symbols/min [2025-10-20T19:00:18.030694] [INFO ] [MAIN ] GLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:18.041097] [DEBUG] [LOCAL ] GLP: Loaded 9431 existing records from primary storage. [2025-10-20T19:00:18.041366] [INFO ] [MAIN ] GLP: Loaded 9431 existing records. [2025-10-20T19:00:18.226047] [WARN ] [MAIN ] GLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:18.226495] [INFO ] [MAIN ] GLP: Wiping existing primary data before full refresh. [2025-10-20T19:00:18.226600] [INFO ] [LOCAL ] GLP: Starting deletion of primary data. [2025-10-20T19:00:18.226957] [DEBUG] [LOCAL ] GLP: Deleted primary data file. [2025-10-20T19:00:18.227062] [INFO ] [MAIN ] GLP: Starting full primary refresh. [2025-10-20T19:00:18.358862] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:00:18.614533] [INFO ] [REMOTE ] GLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:18.651378] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-01. [2025-10-20T19:00:19.439067] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:00:19.451319] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-02. [2025-10-20T19:00:20.229628] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:00:20.251346] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-03. [2025-10-20T19:00:20.796282] [INFO ] [REMOTE ] GIS: Incremental fetch complete for 2025-10 (2.94 seconds). [2025-10-20T19:00:20.803816] [INFO ] [REMOTE ] GIS: Full fetch complete (77451 records). [2025-10-20T19:00:20.803939] [DEBUG] [VALIDATOR ] GIS: Validating primary data. [2025-10-20T19:00:20.804793] [FATAL] [VALIDATOR ] GIS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T19:00:20.804881] [FATAL] [MAIN ] GIS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:00:20.804925] [WARN ] [MAIN ] GIS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:00:20.805005] [INFO ] [LOCAL ] GIS: Starting deletion of primary data. [2025-10-20T19:00:20.805079] [DEBUG] [LOCAL ] GIS: No primary data file found to delete. [2025-10-20T19:00:20.805158] [INFO ] [LOCAL ] GIS: Starting deletion of derivative data. [2025-10-20T19:00:20.805341] [DEBUG] [LOCAL ] GIS: Deleted derivative data file. [2025-10-20T19:00:20.805430] [INFO ] [MAIN ] Update Progress: 937/2367 symbols completed. GIS took 37.0s. Estimated remaining: 3h 3m 33s. Rate: 7.8 symbols/min [2025-10-20T19:00:20.986458] [INFO ] [MAIN ] GLW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:21.035313] [DEBUG] [LOCAL ] GLW: Loaded 54416 existing records from primary storage. [2025-10-20T19:00:21.035623] [INFO ] [MAIN ] GLW: Loaded 54416 existing records. [2025-10-20T19:00:21.056604] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-09. [2025-10-20T19:00:21.158543] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T19:00:21.226919] [WARN ] [MAIN ] GLW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:21.227118] [INFO ] [MAIN ] GLW: Wiping existing primary data before full refresh. [2025-10-20T19:00:21.227164] [INFO ] [LOCAL ] GLW: Starting deletion of primary data. [2025-10-20T19:00:21.227354] [DEBUG] [LOCAL ] GLW: Deleted primary data file. [2025-10-20T19:00:21.227391] [INFO ] [MAIN ] GLW: Starting full primary refresh. [2025-10-20T19:00:21.573636] [INFO ] [REMOTE ] GLW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:21.851390] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-04. [2025-10-20T19:00:22.587771] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T19:00:22.616959] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T19:00:22.651341] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-05. [2025-10-20T19:00:23.451365] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-02. [2025-10-20T19:00:23.566085] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T19:00:24.251493] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-06. [2025-10-20T19:00:24.800222] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T19:00:25.025716] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T19:00:25.051333] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-07. [2025-10-20T19:00:25.750664] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:00:25.851352] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-08. [2025-10-20T19:00:26.651406] [INFO ] [REMOTE ] GL: Fetching primary data for 2025-10. [2025-10-20T19:00:26.707200] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T19:00:27.451404] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-03. [2025-10-20T19:00:28.099272] [INFO ] [REMOTE ] GL: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T19:00:28.104823] [INFO ] [REMOTE ] GL: Full fetch complete (55054 records). [2025-10-20T19:00:28.104917] [DEBUG] [VALIDATOR ] GL: Validating primary data. [2025-10-20T19:00:28.277178] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-09. [2025-10-20T19:00:28.486712] [DEBUG] [VALIDATOR ] GL: All expected trading days are present. [2025-10-20T19:00:28.486833] [DEBUG] [VALIDATOR ] GL: Primary data validation passed. [2025-10-20T19:00:28.487588] [INFO ] [MAIN ] GL: Fetched primary data is valid. [2025-10-20T19:00:28.487629] [DEBUG] [LOCAL ] GL: Saving primary data. [2025-10-20T19:00:28.488825] [DEBUG] [LOCAL ] GL: Trimmed 55054 records to 37483 (keeping 200 days of data) [2025-10-20T19:00:28.763966] [DEBUG] [LOCAL ] GL: Primary data saved (37483 records). [2025-10-20T19:00:28.764123] [INFO ] [MAIN ] GL: Full history saved. [2025-10-20T19:00:28.764749] [INFO ] [MAIN ] Update Progress: 938/2367 symbols completed. GL took 36.1s. Estimated remaining: 3h 3m 26s. Rate: 7.8 symbols/min [2025-10-20T19:00:29.008697] [INFO ] [MAIN ] GM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:29.065818] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-01. [2025-10-20T19:00:29.072164] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:00:29.135898] [DEBUG] [LOCAL ] GM: Loaded 54509 existing records from primary storage. [2025-10-20T19:00:29.136294] [INFO ] [MAIN ] GM: Loaded 54509 existing records. [2025-10-20T19:00:29.314724] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T19:00:29.429177] [WARN ] [MAIN ] GM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:29.429429] [INFO ] [MAIN ] GM: Wiping existing primary data before full refresh. [2025-10-20T19:00:29.429479] [INFO ] [LOCAL ] GM: Starting deletion of primary data. [2025-10-20T19:00:29.429753] [DEBUG] [LOCAL ] GM: Deleted primary data file. [2025-10-20T19:00:29.429811] [INFO ] [MAIN ] GM: Starting full primary refresh. [2025-10-20T19:00:29.856623] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-04. [2025-10-20T19:00:29.967188] [INFO ] [REMOTE ] GM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:30.651360] [INFO ] [REMOTE ] GLP: Fetching primary data for 2025-10. [2025-10-20T19:00:30.950569] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T19:00:31.069085] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:00:31.445319] [INFO ] [REMOTE ] GLP: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:00:31.447873] [INFO ] [REMOTE ] GLP: Full fetch complete (14304 records). [2025-10-20T19:00:31.448141] [DEBUG] [VALIDATOR ] GLP: Validating primary data. [2025-10-20T19:00:31.460495] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-01. [2025-10-20T19:00:31.885980] [DEBUG] [VALIDATOR ] GLP: All expected trading days are present. [2025-10-20T19:00:31.886345] [DEBUG] [VALIDATOR ] GLP: Primary data validation passed. [2025-10-20T19:00:31.887235] [INFO ] [MAIN ] GLP: Fetched primary data is valid. [2025-10-20T19:00:31.887307] [DEBUG] [LOCAL ] GLP: Saving primary data. [2025-10-20T19:00:31.888204] [DEBUG] [LOCAL ] GLP: Trimmed 14304 records to 9318 (keeping 200 days of data) [2025-10-20T19:00:31.954964] [DEBUG] [LOCAL ] GLP: Primary data saved (9318 records). [2025-10-20T19:00:31.955124] [INFO ] [MAIN ] GLP: Full history saved. [2025-10-20T19:00:31.955209] [INFO ] [MAIN ] Update Progress: 939/2367 symbols completed. GLP took 14.1s. Estimated remaining: 3h 3m 11s. Rate: 7.8 symbols/min [2025-10-20T19:00:32.124520] [INFO ] [MAIN ] GME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:32.172610] [DEBUG] [LOCAL ] GME: Loaded 54561 existing records from primary storage. [2025-10-20T19:00:32.172947] [INFO ] [MAIN ] GME: Loaded 54561 existing records. [2025-10-20T19:00:32.259799] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-02. [2025-10-20T19:00:32.346023] [WARN ] [MAIN ] GME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:32.346205] [INFO ] [MAIN ] GME: Wiping existing primary data before full refresh. [2025-10-20T19:00:32.346252] [INFO ] [LOCAL ] GME: Starting deletion of primary data. [2025-10-20T19:00:32.346954] [DEBUG] [LOCAL ] GME: Deleted primary data file. [2025-10-20T19:00:32.347013] [INFO ] [MAIN ] GME: Starting full primary refresh. [2025-10-20T19:00:32.686122] [INFO ] [REMOTE ] GME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:33.051445] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-01. [2025-10-20T19:00:33.761197] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-01 (2.30 seconds). [2025-10-20T19:00:33.833507] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:00:33.851343] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-02. [2025-10-20T19:00:34.651432] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-03. [2025-10-20T19:00:35.444245] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-01 (2.39 seconds). [2025-10-20T19:00:35.455709] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-05. [2025-10-20T19:00:35.773170] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T19:00:36.251357] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-02. [2025-10-20T19:00:36.380632] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:00:37.051371] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-04. [2025-10-20T19:00:37.279338] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:00:37.851427] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-03. [2025-10-20T19:00:38.651380] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-06. [2025-10-20T19:00:38.782207] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-02 (2.53 seconds). [2025-10-20T19:00:39.090686] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T19:00:39.451338] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-05. [2025-10-20T19:00:40.015664] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-03 (2.16 seconds). [2025-10-20T19:00:40.116577] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:00:40.251376] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-04. [2025-10-20T19:00:41.051448] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-03. [2025-10-20T19:00:41.390056] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:00:41.851356] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-06. [2025-10-20T19:00:42.538819] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T19:00:42.651360] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-05. [2025-10-20T19:00:43.451347] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-07. [2025-10-20T19:00:43.593304] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-03 (2.54 seconds). [2025-10-20T19:00:43.637546] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:00:44.251349] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-07. [2025-10-20T19:00:44.724718] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T19:00:45.051369] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-06. [2025-10-20T19:00:45.177423] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T19:00:45.851376] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-08. [2025-10-20T19:00:46.046893] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:00:46.651417] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-08. [2025-10-20T19:00:46.890510] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T19:00:47.451347] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-04. [2025-10-20T19:00:47.646434] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T19:00:48.251334] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-09. [2025-10-20T19:00:48.645357] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T19:00:49.051451] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-07. [2025-10-20T19:00:49.851400] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-09. [2025-10-20T19:00:50.010321] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:00:50.080535] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:00:50.651376] [INFO ] [REMOTE ] GLOB: Fetching primary data for 2025-10. [2025-10-20T19:00:50.875772] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T19:00:51.451403] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-05. [2025-10-20T19:00:52.057024] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:00:52.090539] [INFO ] [REMOTE ] GLOB: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:00:52.098044] [INFO ] [REMOTE ] GLOB: Full fetch complete (65972 records). [2025-10-20T19:00:52.098198] [DEBUG] [VALIDATOR ] GLOB: Validating primary data. [2025-10-20T19:00:52.261671] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-08. [2025-10-20T19:00:52.501175] [DEBUG] [VALIDATOR ] GLOB: All expected trading days are present. [2025-10-20T19:00:52.501257] [DEBUG] [VALIDATOR ] GLOB: Primary data validation passed. [2025-10-20T19:00:52.502173] [INFO ] [MAIN ] GLOB: Fetched primary data is valid. [2025-10-20T19:00:52.502237] [DEBUG] [LOCAL ] GLOB: Saving primary data. [2025-10-20T19:00:52.503517] [DEBUG] [LOCAL ] GLOB: Trimmed 65972 records to 48058 (keeping 200 days of data) [2025-10-20T19:00:52.785496] [DEBUG] [LOCAL ] GLOB: Primary data saved (48058 records). [2025-10-20T19:00:52.785674] [INFO ] [MAIN ] GLOB: Full history saved. [2025-10-20T19:00:52.786534] [INFO ] [MAIN ] Update Progress: 940/2367 symbols completed. GLOB took 39.4s. Estimated remaining: 3h 3m 24s. Rate: 7.8 symbols/min [2025-10-20T19:00:52.961381] [INFO ] [MAIN ] GMED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:53.007446] [DEBUG] [LOCAL ] GMED: Loaded 48880 existing records from primary storage. [2025-10-20T19:00:53.007796] [INFO ] [MAIN ] GMED: Loaded 48880 existing records. [2025-10-20T19:00:53.071675] [INFO ] [REMOTE ] GLW: Fetching primary data for 2025-10. [2025-10-20T19:00:53.189851] [WARN ] [MAIN ] GMED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:53.190089] [INFO ] [MAIN ] GMED: Wiping existing primary data before full refresh. [2025-10-20T19:00:53.190136] [INFO ] [LOCAL ] GMED: Starting deletion of primary data. [2025-10-20T19:00:53.190347] [DEBUG] [LOCAL ] GMED: Deleted primary data file. [2025-10-20T19:00:53.190403] [INFO ] [MAIN ] GMED: Starting full primary refresh. [2025-10-20T19:00:53.553768] [INFO ] [REMOTE ] GMED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:53.781974] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-05 (2.33 seconds). [2025-10-20T19:00:53.851411] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-06. [2025-10-20T19:00:53.957760] [INFO ] [REMOTE ] GLW: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:00:53.968516] [INFO ] [REMOTE ] GLW: Full fetch complete (77794 records). [2025-10-20T19:00:53.968713] [DEBUG] [VALIDATOR ] GLW: Validating primary data. [2025-10-20T19:00:54.431505] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T19:00:54.483144] [DEBUG] [VALIDATOR ] GLW: All expected trading days are present. [2025-10-20T19:00:54.483269] [DEBUG] [VALIDATOR ] GLW: Primary data validation passed. [2025-10-20T19:00:54.484263] [INFO ] [MAIN ] GLW: Fetched primary data is valid. [2025-10-20T19:00:54.484330] [DEBUG] [LOCAL ] GLW: Saving primary data. [2025-10-20T19:00:54.485903] [DEBUG] [LOCAL ] GLW: Trimmed 77794 records to 53635 (keeping 200 days of data) [2025-10-20T19:00:54.660326] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-01. [2025-10-20T19:00:54.833055] [DEBUG] [LOCAL ] GLW: Primary data saved (53635 records). [2025-10-20T19:00:54.833194] [INFO ] [MAIN ] GLW: Full history saved. [2025-10-20T19:00:54.833271] [INFO ] [MAIN ] Update Progress: 941/2367 symbols completed. GLW took 34.0s. Estimated remaining: 3h 3m 7s. Rate: 7.8 symbols/min [2025-10-20T19:00:55.000382] [INFO ] [MAIN ] GMRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:55.034914] [DEBUG] [LOCAL ] GMRE: Loaded 37414 existing records from primary storage. [2025-10-20T19:00:55.035197] [INFO ] [MAIN ] GMRE: Loaded 37414 existing records. [2025-10-20T19:00:55.208796] [WARN ] [MAIN ] GMRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:55.209054] [INFO ] [MAIN ] GMRE: Wiping existing primary data before full refresh. [2025-10-20T19:00:55.209111] [INFO ] [LOCAL ] GMRE: Starting deletion of primary data. [2025-10-20T19:00:55.209320] [DEBUG] [LOCAL ] GMRE: Deleted primary data file. [2025-10-20T19:00:55.209377] [INFO ] [MAIN ] GMRE: Starting full primary refresh. [2025-10-20T19:00:55.458452] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-09. [2025-10-20T19:00:55.585845] [INFO ] [REMOTE ] GMRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:00:56.251346] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-01. [2025-10-20T19:00:56.349437] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:00:56.393751] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-06 (2.54 seconds). [2025-10-20T19:00:57.051377] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-07. [2025-10-20T19:00:57.494738] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T19:00:57.534415] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:00:57.851417] [INFO ] [REMOTE ] GM: Fetching primary data for 2025-10. [2025-10-20T19:00:58.651379] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-02. [2025-10-20T19:00:58.764897] [INFO ] [REMOTE ] GM: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:00:58.773231] [INFO ] [REMOTE ] GM: Full fetch complete (77907 records). [2025-10-20T19:00:58.773368] [DEBUG] [VALIDATOR ] GM: Validating primary data. [2025-10-20T19:00:59.176742] [DEBUG] [VALIDATOR ] GM: All expected trading days are present. [2025-10-20T19:00:59.176819] [DEBUG] [VALIDATOR ] GM: Primary data validation passed. [2025-10-20T19:00:59.177717] [INFO ] [MAIN ] GM: Fetched primary data is valid. [2025-10-20T19:00:59.177780] [DEBUG] [LOCAL ] GM: Saving primary data. [2025-10-20T19:00:59.179212] [DEBUG] [LOCAL ] GM: Trimmed 77907 records to 53729 (keeping 200 days of data) [2025-10-20T19:00:59.455814] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-02. [2025-10-20T19:00:59.510905] [DEBUG] [LOCAL ] GM: Primary data saved (53729 records). [2025-10-20T19:00:59.511050] [INFO ] [MAIN ] GM: Full history saved. [2025-10-20T19:00:59.511528] [INFO ] [MAIN ] Update Progress: 942/2367 symbols completed. GM took 30.7s. Estimated remaining: 3h 2m 55s. Rate: 7.8 symbols/min [2025-10-20T19:00:59.680618] [INFO ] [MAIN ] GNE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:00:59.702160] [DEBUG] [LOCAL ] GNE: Loaded 18212 existing records from primary storage. [2025-10-20T19:00:59.702411] [INFO ] [MAIN ] GNE: Loaded 18212 existing records. [2025-10-20T19:00:59.860182] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-07 (2.81 seconds). [2025-10-20T19:00:59.941165] [WARN ] [MAIN ] GNE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:00:59.941944] [INFO ] [MAIN ] GNE: Wiping existing primary data before full refresh. [2025-10-20T19:00:59.942037] [INFO ] [LOCAL ] GNE: Starting deletion of primary data. [2025-10-20T19:00:59.942233] [DEBUG] [LOCAL ] GNE: Deleted primary data file. [2025-10-20T19:00:59.942339] [INFO ] [MAIN ] GNE: Starting full primary refresh. [2025-10-20T19:01:00.061034] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:01:00.256585] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-08. [2025-10-20T19:01:00.340902] [INFO ] [REMOTE ] GNE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:01.051412] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-03. [2025-10-20T19:01:01.383422] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T19:01:01.851387] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-03. [2025-10-20T19:01:02.651427] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-01. [2025-10-20T19:01:03.127600] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-08 (2.87 seconds). [2025-10-20T19:01:03.304264] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-03 (2.25 seconds). [2025-10-20T19:01:03.356316] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T19:01:03.451383] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-09. [2025-10-20T19:01:03.678506] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T19:01:04.251424] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-04. [2025-10-20T19:01:05.051393] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-04. [2025-10-20T19:01:05.851380] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-02. [2025-10-20T19:01:06.003339] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:01:06.134268] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-09 (2.68 seconds). [2025-10-20T19:01:06.385768] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T19:01:06.497692] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T19:01:06.651388] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-05. [2025-10-20T19:01:07.451383] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-03. [2025-10-20T19:01:08.211703] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:01:08.251341] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-06. [2025-10-20T19:01:08.320228] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:01:09.051344] [INFO ] [REMOTE ] GME: Fetching primary data for 2025-10. [2025-10-20T19:01:09.572421] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:01:09.851429] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-07. [2025-10-20T19:01:10.651372] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-04. [2025-10-20T19:01:11.366670] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:01:11.451338] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-08. [2025-10-20T19:01:11.514587] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T19:01:11.641873] [INFO ] [REMOTE ] GME: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T19:01:11.656554] [INFO ] [REMOTE ] GME: Full fetch complete (77923 records). [2025-10-20T19:01:11.656797] [DEBUG] [VALIDATOR ] GME: Validating primary data. [2025-10-20T19:01:12.049923] [DEBUG] [VALIDATOR ] GME: All expected trading days are present. [2025-10-20T19:01:12.049998] [DEBUG] [VALIDATOR ] GME: Primary data validation passed. [2025-10-20T19:01:12.050956] [INFO ] [MAIN ] GME: Fetched primary data is valid. [2025-10-20T19:01:12.051020] [DEBUG] [LOCAL ] GME: Saving primary data. [2025-10-20T19:01:12.052476] [DEBUG] [LOCAL ] GME: Trimmed 77923 records to 53781 (keeping 200 days of data) [2025-10-20T19:01:12.256942] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-05. [2025-10-20T19:01:12.371293] [DEBUG] [LOCAL ] GME: Primary data saved (53781 records). [2025-10-20T19:01:12.371418] [INFO ] [MAIN ] GME: Full history saved. [2025-10-20T19:01:12.371493] [INFO ] [MAIN ] Update Progress: 943/2367 symbols completed. GME took 40.4s. Estimated remaining: 3h 2m 55s. Rate: 7.8 symbols/min [2025-10-20T19:01:12.550154] [INFO ] [MAIN ] GNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:12.586055] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:01:12.630295] [DEBUG] [LOCAL ] GNK: Loaded 40529 existing records from primary storage. [2025-10-20T19:01:12.630583] [INFO ] [MAIN ] GNK: Loaded 40529 existing records. [2025-10-20T19:01:12.812257] [WARN ] [MAIN ] GNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:12.812477] [INFO ] [MAIN ] GNK: Wiping existing primary data before full refresh. [2025-10-20T19:01:12.812525] [INFO ] [LOCAL ] GNK: Starting deletion of primary data. [2025-10-20T19:01:12.812768] [DEBUG] [LOCAL ] GNK: Deleted primary data file. [2025-10-20T19:01:12.812826] [INFO ] [MAIN ] GNK: Starting full primary refresh. [2025-10-20T19:01:13.053302] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-05. [2025-10-20T19:01:13.169615] [INFO ] [REMOTE ] GNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:13.259272] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:01:13.851435] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-01. [2025-10-20T19:01:14.651427] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-06. [2025-10-20T19:01:14.821532] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T19:01:15.283180] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:01:15.451359] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-06. [2025-10-20T19:01:15.772316] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:01:16.251378] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-07. [2025-10-20T19:01:17.062052] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-02. [2025-10-20T19:01:17.067350] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T19:01:17.384593] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:01:17.851394] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-07. [2025-10-20T19:01:18.132302] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:01:18.651418] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-03. [2025-10-20T19:01:19.418628] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:01:19.451435] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-08. [2025-10-20T19:01:20.232481] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:01:20.251330] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-04. [2025-10-20T19:01:20.523919] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:01:21.051383] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-08. [2025-10-20T19:01:21.851452] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-09. [2025-10-20T19:01:21.969440] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:01:22.527600] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:01:22.651449] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-05. [2025-10-20T19:01:23.247717] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:01:23.451384] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-09. [2025-10-20T19:01:24.213790] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:01:24.251363] [INFO ] [REMOTE ] GMRE: Fetching primary data for 2025-10. [2025-10-20T19:01:24.281595] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:01:25.051448] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-09. [2025-10-20T19:01:25.715058] [INFO ] [REMOTE ] GMRE: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:01:25.719253] [INFO ] [REMOTE ] GMRE: Full fetch complete (52255 records). [2025-10-20T19:01:25.719341] [DEBUG] [VALIDATOR ] GMRE: Validating primary data. [2025-10-20T19:01:25.861772] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-06. [2025-10-20T19:01:26.109888] [DEBUG] [VALIDATOR ] GMRE: All expected trading days are present. [2025-10-20T19:01:26.109977] [DEBUG] [VALIDATOR ] GMRE: Primary data validation passed. [2025-10-20T19:01:26.110748] [INFO ] [MAIN ] GMRE: Fetched primary data is valid. [2025-10-20T19:01:26.110813] [DEBUG] [LOCAL ] GMRE: Saving primary data. [2025-10-20T19:01:26.111970] [DEBUG] [LOCAL ] GMRE: Trimmed 52255 records to 36823 (keeping 200 days of data) [2025-10-20T19:01:26.333504] [DEBUG] [LOCAL ] GMRE: Primary data saved (36823 records). [2025-10-20T19:01:26.333669] [INFO ] [MAIN ] GMRE: Full history saved. [2025-10-20T19:01:26.333769] [INFO ] [MAIN ] Update Progress: 944/2367 symbols completed. GMRE took 31.5s. Estimated remaining: 3h 2m 57s. Rate: 7.8 symbols/min [2025-10-20T19:01:26.506680] [INFO ] [MAIN ] GNL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:26.548241] [DEBUG] [LOCAL ] GNL: Loaded 47399 existing records from primary storage. [2025-10-20T19:01:26.548571] [INFO ] [MAIN ] GNL: Loaded 47399 existing records. [2025-10-20T19:01:26.656594] [INFO ] [REMOTE ] GNE: Fetching primary data for 2025-10. [2025-10-20T19:01:26.767818] [WARN ] [MAIN ] GNL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:26.768426] [INFO ] [MAIN ] GNL: Wiping existing primary data before full refresh. [2025-10-20T19:01:26.770705] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:01:26.768837] [INFO ] [LOCAL ] GNL: Starting deletion of primary data. [2025-10-20T19:01:26.781572] [DEBUG] [LOCAL ] GNL: Deleted primary data file. [2025-10-20T19:01:26.782096] [INFO ] [MAIN ] GNL: Starting full primary refresh. [2025-10-20T19:01:27.142533] [INFO ] [REMOTE ] GNL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:27.259674] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T19:01:27.444721] [INFO ] [REMOTE ] GNE: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:01:27.447211] [INFO ] [REMOTE ] GNE: Full fetch complete (22560 records). [2025-10-20T19:01:27.447300] [DEBUG] [VALIDATOR ] GNE: Validating primary data. [2025-10-20T19:01:27.458949] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-07. [2025-10-20T19:01:27.816682] [DEBUG] [VALIDATOR ] GNE: All expected trading days are present. [2025-10-20T19:01:27.816798] [DEBUG] [VALIDATOR ] GNE: Primary data validation passed. [2025-10-20T19:01:27.817269] [INFO ] [MAIN ] GNE: Fetched primary data is valid. [2025-10-20T19:01:27.817308] [DEBUG] [LOCAL ] GNE: Saving primary data. [2025-10-20T19:01:27.817942] [DEBUG] [LOCAL ] GNE: Trimmed 22560 records to 18091 (keeping 200 days of data) [2025-10-20T19:01:27.922296] [DEBUG] [LOCAL ] GNE: Primary data saved (18091 records). [2025-10-20T19:01:27.922433] [INFO ] [MAIN ] GNE: Full history saved. [2025-10-20T19:01:27.922513] [INFO ] [MAIN ] Update Progress: 945/2367 symbols completed. GNE took 28.4s. Estimated remaining: 3h 2m 40s. Rate: 7.8 symbols/min [2025-10-20T19:01:28.087385] [INFO ] [MAIN ] GNRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:28.130867] [DEBUG] [LOCAL ] GNRC: Loaded 46194 existing records from primary storage. [2025-10-20T19:01:28.131188] [INFO ] [MAIN ] GNRC: Loaded 46194 existing records. [2025-10-20T19:01:28.256489] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-01. [2025-10-20T19:01:28.306886] [WARN ] [MAIN ] GNRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:28.307193] [INFO ] [MAIN ] GNRC: Wiping existing primary data before full refresh. [2025-10-20T19:01:28.307238] [INFO ] [LOCAL ] GNRC: Starting deletion of primary data. [2025-10-20T19:01:28.307442] [DEBUG] [LOCAL ] GNRC: Deleted primary data file. [2025-10-20T19:01:28.307498] [INFO ] [MAIN ] GNRC: Starting full primary refresh. [2025-10-20T19:01:28.659974] [INFO ] [REMOTE ] GNRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:28.905019] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T19:01:29.051415] [INFO ] [REMOTE ] GMED: Fetching primary data for 2025-10. [2025-10-20T19:01:29.851414] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-01. [2025-10-20T19:01:29.939831] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:01:30.565707] [INFO ] [REMOTE ] GMED: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T19:01:30.574573] [INFO ] [REMOTE ] GMED: Full fetch complete (70065 records). [2025-10-20T19:01:30.575007] [DEBUG] [VALIDATOR ] GMED: Validating primary data. [2025-10-20T19:01:30.656595] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-08. [2025-10-20T19:01:31.006677] [DEBUG] [VALIDATOR ] GMED: All expected trading days are present. [2025-10-20T19:01:31.006789] [DEBUG] [VALIDATOR ] GMED: Primary data validation passed. [2025-10-20T19:01:31.007734] [INFO ] [MAIN ] GMED: Fetched primary data is valid. [2025-10-20T19:01:31.007800] [DEBUG] [LOCAL ] GMED: Saving primary data. [2025-10-20T19:01:31.009364] [DEBUG] [LOCAL ] GMED: Trimmed 70065 records to 48170 (keeping 200 days of data) [2025-10-20T19:01:31.295797] [DEBUG] [LOCAL ] GMED: Primary data saved (48170 records). [2025-10-20T19:01:31.295949] [INFO ] [MAIN ] GMED: Full history saved. [2025-10-20T19:01:31.296856] [INFO ] [MAIN ] Update Progress: 946/2367 symbols completed. GMED took 38.5s. Estimated remaining: 3h 2m 26s. Rate: 7.8 symbols/min [2025-10-20T19:01:31.454718] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-02. [2025-10-20T19:01:31.475258] [INFO ] [MAIN ] GNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:31.506745] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:01:31.520391] [DEBUG] [LOCAL ] GNT: Loaded 8665 existing records from primary storage. [2025-10-20T19:01:31.520703] [INFO ] [MAIN ] GNT: Loaded 8665 existing records. [2025-10-20T19:01:31.711021] [WARN ] [MAIN ] GNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:31.711239] [INFO ] [MAIN ] GNT: Wiping existing primary data before full refresh. [2025-10-20T19:01:31.711287] [INFO ] [LOCAL ] GNT: Starting deletion of primary data. [2025-10-20T19:01:31.711461] [DEBUG] [LOCAL ] GNT: Deleted primary data file. [2025-10-20T19:01:31.711501] [INFO ] [MAIN ] GNT: Starting full primary refresh. [2025-10-20T19:01:32.081733] [INFO ] [REMOTE ] GNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:32.145246] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:01:32.251366] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-09. [2025-10-20T19:01:32.681736] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:01:33.051389] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-02. [2025-10-20T19:01:33.218790] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:01:33.851383] [INFO ] [REMOTE ] GNK: Fetching primary data for 2025-10. [2025-10-20T19:01:34.651433] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-01. [2025-10-20T19:01:34.687921] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:01:35.255413] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T19:01:35.312294] [INFO ] [REMOTE ] GNK: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:01:35.317982] [INFO ] [REMOTE ] GNK: Full fetch complete (57317 records). [2025-10-20T19:01:35.318076] [DEBUG] [VALIDATOR ] GNK: Validating primary data. [2025-10-20T19:01:35.456556] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-03. [2025-10-20T19:01:35.706724] [DEBUG] [VALIDATOR ] GNK: All expected trading days are present. [2025-10-20T19:01:35.706793] [DEBUG] [VALIDATOR ] GNK: Primary data validation passed. [2025-10-20T19:01:35.707543] [INFO ] [MAIN ] GNK: Fetched primary data is valid. [2025-10-20T19:01:35.707583] [DEBUG] [LOCAL ] GNK: Saving primary data. [2025-10-20T19:01:35.708609] [DEBUG] [LOCAL ] GNK: Trimmed 57317 records to 39773 (keeping 200 days of data) [2025-10-20T19:01:35.942567] [DEBUG] [LOCAL ] GNK: Primary data saved (39773 records). [2025-10-20T19:01:35.942725] [INFO ] [MAIN ] GNK: Full history saved. [2025-10-20T19:01:35.942811] [INFO ] [MAIN ] Update Progress: 947/2367 symbols completed. GNK took 23.6s. Estimated remaining: 3h 2m 13s. Rate: 7.8 symbols/min [2025-10-20T19:01:36.109473] [INFO ] [MAIN ] GNW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:01:36.155608] [DEBUG] [LOCAL ] GNW: Loaded 52153 existing records from primary storage. [2025-10-20T19:01:36.155927] [INFO ] [MAIN ] GNW: Loaded 52153 existing records. [2025-10-20T19:01:36.256968] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-02. [2025-10-20T19:01:36.337629] [WARN ] [MAIN ] GNW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:01:36.337829] [INFO ] [MAIN ] GNW: Wiping existing primary data before full refresh. [2025-10-20T19:01:36.337900] [INFO ] [LOCAL ] GNW: Starting deletion of primary data. [2025-10-20T19:01:36.338414] [DEBUG] [LOCAL ] GNW: Deleted primary data file. [2025-10-20T19:01:36.338500] [INFO ] [MAIN ] GNW: Starting full primary refresh. [2025-10-20T19:01:36.693259] [INFO ] [REMOTE ] GNW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:01:36.908292] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:01:37.051368] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-03. [2025-10-20T19:01:37.198542] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:01:37.851429] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-01. [2025-10-20T19:01:38.495771] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:01:38.651375] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-04. [2025-10-20T19:01:39.451372] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-04. [2025-10-20T19:01:39.536149] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:01:40.164988] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:01:40.251369] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-03. [2025-10-20T19:01:40.937586] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T19:01:41.051355] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-05. [2025-10-20T19:01:41.124249] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:01:41.851363] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-02. [2025-10-20T19:01:42.656584] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-05. [2025-10-20T19:01:42.668487] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:01:43.312816] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:01:43.451537] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-03. [2025-10-20T19:01:43.876907] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T19:01:44.251378] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-06. [2025-10-20T19:01:44.964429] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T19:01:45.051384] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-04. [2025-10-20T19:01:45.851368] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-06. [2025-10-20T19:01:45.988209] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:01:46.651378] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-07. [2025-10-20T19:01:47.126185] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T19:01:47.388859] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:01:47.451327] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-07. [2025-10-20T19:01:48.251371] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-05. [2025-10-20T19:01:48.306819] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T19:01:48.984251] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T19:01:49.051347] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-08. [2025-10-20T19:01:49.851439] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-08. [2025-10-20T19:01:49.904795] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:01:50.657941] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-06. [2025-10-20T19:01:50.661165] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T19:01:51.096014] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T19:01:51.451401] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-04. [2025-10-20T19:01:52.173193] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:01:52.251432] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-05. [2025-10-20T19:01:52.406184] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:01:52.662542] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T19:01:53.051428] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-06. [2025-10-20T19:01:53.851377] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-07. [2025-10-20T19:01:53.914793] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:01:54.651376] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-09. [2025-10-20T19:01:55.422765] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:01:55.451334] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-08. [2025-10-20T19:01:56.145234] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:01:56.251390] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-07. [2025-10-20T19:01:56.766307] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T19:01:57.024005] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:01:57.051336] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-09. [2025-10-20T19:01:57.851366] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-08. [2025-10-20T19:01:58.506956] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T19:01:58.651372] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-09. [2025-10-20T19:01:58.803750] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:01:59.451420] [INFO ] [REMOTE ] GNRC: Fetching primary data for 2025-10. [2025-10-20T19:02:00.251448] [INFO ] [REMOTE ] GNW: Fetching primary data for 2025-10. [2025-10-20T19:02:00.813401] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T19:02:01.051428] [INFO ] [REMOTE ] GNL: Fetching primary data for 2025-10. [2025-10-20T19:02:01.851369] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-09. [2025-10-20T19:02:02.042840] [INFO ] [REMOTE ] GNRC: Incremental fetch complete for 2025-10 (2.59 seconds). [2025-10-20T19:02:02.049623] [INFO ] [REMOTE ] GNRC: Full fetch complete (65644 records). [2025-10-20T19:02:02.049768] [DEBUG] [VALIDATOR ] GNRC: Validating primary data. [2025-10-20T19:02:02.431388] [DEBUG] [VALIDATOR ] GNRC: All expected trading days are present. [2025-10-20T19:02:02.431467] [DEBUG] [VALIDATOR ] GNRC: Primary data validation passed. [2025-10-20T19:02:02.432322] [INFO ] [MAIN ] GNRC: Fetched primary data is valid. [2025-10-20T19:02:02.432384] [DEBUG] [LOCAL ] GNRC: Saving primary data. [2025-10-20T19:02:02.433741] [DEBUG] [LOCAL ] GNRC: Trimmed 65644 records to 45450 (keeping 200 days of data) [2025-10-20T19:02:02.703203] [DEBUG] [LOCAL ] GNRC: Primary data saved (45450 records). [2025-10-20T19:02:02.703340] [INFO ] [MAIN ] GNRC: Full history saved. [2025-10-20T19:02:02.703902] [INFO ] [MAIN ] Update Progress: 948/2367 symbols completed. GNRC took 34.8s. Estimated remaining: 3h 2m 34s. Rate: 7.8 symbols/min [2025-10-20T19:02:02.874481] [INFO ] [MAIN ] GOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:02.918992] [DEBUG] [LOCAL ] GOF: Loaded 43353 existing records from primary storage. [2025-10-20T19:02:02.919348] [INFO ] [MAIN ] GOF: Loaded 43353 existing records. [2025-10-20T19:02:03.119513] [WARN ] [MAIN ] GOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:03.119990] [INFO ] [MAIN ] GOF: Wiping existing primary data before full refresh. [2025-10-20T19:02:03.120282] [INFO ] [LOCAL ] GOF: Starting deletion of primary data. [2025-10-20T19:02:03.139593] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T19:02:03.138901] [DEBUG] [LOCAL ] GOF: Deleted primary data file. [2025-10-20T19:02:03.160249] [INFO ] [MAIN ] GOF: Starting full primary refresh. [2025-10-20T19:02:03.161224] [INFO ] [REMOTE ] GNT: Fetching primary data for 2025-10. [2025-10-20T19:02:03.168496] [INFO ] [REMOTE ] GNW: Incremental fetch complete for 2025-10 (2.92 seconds). [2025-10-20T19:02:03.289862] [INFO ] [REMOTE ] GNW: Full fetch complete (73842 records). [2025-10-20T19:02:03.290098] [DEBUG] [VALIDATOR ] GNW: Validating primary data. [2025-10-20T19:02:03.836734] [INFO ] [REMOTE ] GOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:03.858517] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-01. [2025-10-20T19:02:04.028909] [INFO ] [REMOTE ] GNL: Incremental fetch complete for 2025-10 (2.98 seconds). [2025-10-20T19:02:04.058891] [DEBUG] [VALIDATOR ] GNW: All expected trading days are present. [2025-10-20T19:02:04.059536] [DEBUG] [VALIDATOR ] GNW: Primary data validation passed. [2025-10-20T19:02:04.061066] [INFO ] [MAIN ] GNW: Fetched primary data is valid. [2025-10-20T19:02:04.061255] [DEBUG] [LOCAL ] GNW: Saving primary data. [2025-10-20T19:02:04.062274] [INFO ] [REMOTE ] GNL: Full fetch complete (68211 records). [2025-10-20T19:02:04.062723] [DEBUG] [VALIDATOR ] GNL: Validating primary data. [2025-10-20T19:02:04.069186] [DEBUG] [LOCAL ] GNW: Trimmed 73842 records to 51392 (keeping 200 days of data) [2025-10-20T19:02:04.292168] [INFO ] [REMOTE ] GNT: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:02:04.301665] [INFO ] [REMOTE ] GNT: Full fetch complete (11345 records). [2025-10-20T19:02:04.319309] [DEBUG] [VALIDATOR ] GNT: Validating primary data. [2025-10-20T19:02:05.062870] [DEBUG] [VALIDATOR ] GNL: All expected trading days are present. [2025-10-20T19:02:05.127890] [DEBUG] [VALIDATOR ] GNL: Primary data validation passed. [2025-10-20T19:02:05.157716] [INFO ] [MAIN ] GNL: Fetched primary data is valid. [2025-10-20T19:02:05.172716] [DEBUG] [LOCAL ] GNL: Saving primary data. [2025-10-20T19:02:05.253630] [DEBUG] [VALIDATOR ] GNT: All expected trading days are present. [2025-10-20T19:02:05.255514] [DEBUG] [VALIDATOR ] GNT: Primary data validation passed. [2025-10-20T19:02:05.256181] [INFO ] [MAIN ] GNT: Fetched primary data is valid. [2025-10-20T19:02:05.256617] [DEBUG] [LOCAL ] GNT: Saving primary data. [2025-10-20T19:02:05.261102] [DEBUG] [LOCAL ] GNW: Primary data saved (51392 records). [2025-10-20T19:02:05.261539] [INFO ] [MAIN ] GNW: Full history saved. [2025-10-20T19:02:05.261307] [DEBUG] [LOCAL ] GNL: Trimmed 68211 records to 46630 (keeping 200 days of data) [2025-10-20T19:02:05.261853] [INFO ] [MAIN ] Update Progress: 949/2367 symbols completed. GNW took 29.3s. Estimated remaining: 3h 2m 19s. Rate: 7.8 symbols/min [2025-10-20T19:02:05.269324] [DEBUG] [LOCAL ] GNT: Trimmed 11345 records to 8645 (keeping 200 days of data) [2025-10-20T19:02:05.652041] [DEBUG] [LOCAL ] GNT: Primary data saved (8645 records). [2025-10-20T19:02:05.661595] [INFO ] [MAIN ] GNT: Full history saved. [2025-10-20T19:02:05.668835] [INFO ] [MAIN ] Update Progress: 950/2367 symbols completed. GNT took 34.4s. Estimated remaining: 3h 2m 0s. Rate: 7.8 symbols/min [2025-10-20T19:02:05.862422] [INFO ] [MAIN ] GOLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:05.953327] [DEBUG] [LOCAL ] GNL: Primary data saved (46630 records). [2025-10-20T19:02:05.958848] [INFO ] [MAIN ] GNL: Full history saved. [2025-10-20T19:02:05.967382] [INFO ] [MAIN ] Update Progress: 951/2367 symbols completed. GNL took 39.6s. Estimated remaining: 3h 1m 42s. Rate: 7.8 symbols/min [2025-10-20T19:02:06.176778] [INFO ] [MAIN ] GOOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:06.179115] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-01 (2.32 seconds). [2025-10-20T19:02:06.237027] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-02. [2025-10-20T19:02:06.274417] [DEBUG] [LOCAL ] GOLF: Loaded 31259 existing records from primary storage. [2025-10-20T19:02:06.312955] [INFO ] [MAIN ] GOLF: Loaded 31259 existing records. [2025-10-20T19:02:06.312624] [INFO ] [MAIN ] GOTU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:06.472582] [DEBUG] [LOCAL ] GOOS: Loaded 45143 existing records from primary storage. [2025-10-20T19:02:06.518623] [INFO ] [MAIN ] GOOS: Loaded 45143 existing records. [2025-10-20T19:02:06.570199] [DEBUG] [LOCAL ] GOTU: Loaded 37702 existing records from primary storage. [2025-10-20T19:02:06.609859] [INFO ] [MAIN ] GOTU: Loaded 37702 existing records. [2025-10-20T19:02:06.604110] [WARN ] [MAIN ] GOLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:06.688554] [INFO ] [MAIN ] GOLF: Wiping existing primary data before full refresh. [2025-10-20T19:02:06.694612] [INFO ] [LOCAL ] GOLF: Starting deletion of primary data. [2025-10-20T19:02:06.725354] [DEBUG] [LOCAL ] GOLF: Deleted primary data file. [2025-10-20T19:02:06.736206] [INFO ] [MAIN ] GOLF: Starting full primary refresh. [2025-10-20T19:02:07.046564] [WARN ] [MAIN ] GOTU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:07.079758] [INFO ] [MAIN ] GOTU: Wiping existing primary data before full refresh. [2025-10-20T19:02:07.079427] [WARN ] [MAIN ] GOOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:07.080273] [INFO ] [LOCAL ] GOTU: Starting deletion of primary data. [2025-10-20T19:02:07.080157] [INFO ] [MAIN ] GOOS: Wiping existing primary data before full refresh. [2025-10-20T19:02:07.091526] [INFO ] [LOCAL ] GOOS: Starting deletion of primary data. [2025-10-20T19:02:07.091405] [DEBUG] [LOCAL ] GOTU: Deleted primary data file. [2025-10-20T19:02:07.102051] [INFO ] [MAIN ] GOTU: Starting full primary refresh. [2025-10-20T19:02:07.177812] [DEBUG] [LOCAL ] GOOS: Deleted primary data file. [2025-10-20T19:02:07.240973] [INFO ] [MAIN ] GOOS: Starting full primary refresh. [2025-10-20T19:02:07.875955] [INFO ] [REMOTE ] GOLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:07.983735] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-01. [2025-10-20T19:02:08.104498] [INFO ] [REMOTE ] GOTU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:08.118203] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-01. [2025-10-20T19:02:08.225485] [INFO ] [REMOTE ] GOOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:08.226559] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-01. [2025-10-20T19:02:08.227422] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T19:02:08.237253] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-03. [2025-10-20T19:02:09.277699] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T19:02:09.284297] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-02. [2025-10-20T19:02:09.741680] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:02:09.751921] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-02. [2025-10-20T19:02:09.829236] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:02:09.851332] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-02. [2025-10-20T19:02:09.939287] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:02:10.241966] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:02:10.651393] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-04. [2025-10-20T19:02:10.983940] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:02:11.067315] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T19:02:11.451390] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-03. [2025-10-20T19:02:12.251367] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-03. [2025-10-20T19:02:12.327414] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:02:13.051397] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-05. [2025-10-20T19:02:13.129459] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:02:13.562895] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T19:02:13.851378] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-04. [2025-10-20T19:02:14.651384] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-04. [2025-10-20T19:02:14.710271] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T19:02:15.264438] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:02:15.451417] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-03. [2025-10-20T19:02:16.112733] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:02:16.251394] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-05. [2025-10-20T19:02:16.707577] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T19:02:17.051432] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-06. [2025-10-20T19:02:17.737930] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:02:17.851386] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-05. [2025-10-20T19:02:18.597889] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:02:18.651359] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-04. [2025-10-20T19:02:19.451416] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-06. [2025-10-20T19:02:19.504500] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:02:20.137414] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:02:20.251368] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-05. [2025-10-20T19:02:20.686769] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T19:02:21.051354] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-07. [2025-10-20T19:02:21.851359] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-07. [2025-10-20T19:02:21.942876] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:02:22.651351] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-06. [2025-10-20T19:02:22.806553] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:02:23.227069] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:02:23.451398] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-06. [2025-10-20T19:02:24.180575] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:02:24.251395] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-07. [2025-10-20T19:02:24.860555] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:02:25.051413] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-07. [2025-10-20T19:02:25.758907] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:02:25.851393] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-08. [2025-10-20T19:02:26.362383] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:02:26.651434] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-08. [2025-10-20T19:02:27.432764] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:02:27.451345] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-09. [2025-10-20T19:02:28.043581] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:02:28.251368] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-08. [2025-10-20T19:02:29.041537] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:02:29.051337] [INFO ] [REMOTE ] GOF: Fetching primary data for 2025-10. [2025-10-20T19:02:29.267716] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:02:29.851383] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-09. [2025-10-20T19:02:30.651469] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-09. [2025-10-20T19:02:30.949062] [INFO ] [REMOTE ] GOF: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T19:02:30.955903] [INFO ] [REMOTE ] GOF: Full fetch complete (62523 records). [2025-10-20T19:02:30.956060] [DEBUG] [VALIDATOR ] GOF: Validating primary data. [2025-10-20T19:02:31.347810] [DEBUG] [VALIDATOR ] GOF: All expected trading days are present. [2025-10-20T19:02:31.348077] [DEBUG] [VALIDATOR ] GOF: Primary data validation passed. [2025-10-20T19:02:31.361619] [INFO ] [MAIN ] GOF: Fetched primary data is valid. [2025-10-20T19:02:31.367732] [DEBUG] [LOCAL ] GOF: Saving primary data. [2025-10-20T19:02:31.374031] [DEBUG] [LOCAL ] GOF: Trimmed 62523 records to 42707 (keeping 200 days of data) [2025-10-20T19:02:31.373879] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:02:31.468285] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-08. [2025-10-20T19:02:31.635967] [DEBUG] [LOCAL ] GOF: Primary data saved (42707 records). [2025-10-20T19:02:31.636088] [INFO ] [MAIN ] GOF: Full history saved. [2025-10-20T19:02:31.636618] [INFO ] [MAIN ] Update Progress: 952/2367 symbols completed. GOF took 28.9s. Estimated remaining: 3h 2m 1s. Rate: 7.8 symbols/min [2025-10-20T19:02:31.806092] [INFO ] [MAIN ] GPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:31.853135] [DEBUG] [LOCAL ] GPC: Loaded 46707 existing records from primary storage. [2025-10-20T19:02:31.853409] [INFO ] [MAIN ] GPC: Loaded 46707 existing records. [2025-10-20T19:02:32.035154] [WARN ] [MAIN ] GPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:32.035485] [INFO ] [MAIN ] GPC: Wiping existing primary data before full refresh. [2025-10-20T19:02:32.035671] [INFO ] [LOCAL ] GPC: Starting deletion of primary data. [2025-10-20T19:02:32.036262] [DEBUG] [LOCAL ] GPC: Deleted primary data file. [2025-10-20T19:02:32.036320] [INFO ] [MAIN ] GPC: Starting full primary refresh. [2025-10-20T19:02:32.256290] [INFO ] [REMOTE ] GOTU: Fetching primary data for 2025-10. [2025-10-20T19:02:32.397854] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:02:32.427786] [INFO ] [REMOTE ] GPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:32.847816] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T19:02:33.051341] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-01. [2025-10-20T19:02:33.396201] [INFO ] [REMOTE ] GOTU: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T19:02:33.401257] [INFO ] [REMOTE ] GOTU: Full fetch complete (58405 records). [2025-10-20T19:02:33.401365] [DEBUG] [VALIDATOR ] GOTU: Validating primary data. [2025-10-20T19:02:33.783792] [DEBUG] [VALIDATOR ] GOTU: All expected trading days are present. [2025-10-20T19:02:33.783867] [DEBUG] [VALIDATOR ] GOTU: Primary data validation passed. [2025-10-20T19:02:33.784690] [INFO ] [MAIN ] GOTU: Fetched primary data is valid. [2025-10-20T19:02:33.784751] [DEBUG] [LOCAL ] GOTU: Saving primary data. [2025-10-20T19:02:33.785845] [DEBUG] [LOCAL ] GOTU: Trimmed 58405 records to 36945 (keeping 200 days of data) [2025-10-20T19:02:33.863461] [INFO ] [REMOTE ] GOOS: Fetching primary data for 2025-10. [2025-10-20T19:02:34.012295] [DEBUG] [LOCAL ] GOTU: Primary data saved (36945 records). [2025-10-20T19:02:34.012538] [INFO ] [MAIN ] GOTU: Full history saved. [2025-10-20T19:02:34.012685] [INFO ] [MAIN ] Update Progress: 953/2367 symbols completed. GOTU took 28.0s. Estimated remaining: 3h 1m 45s. Rate: 7.8 symbols/min [2025-10-20T19:02:34.183395] [INFO ] [MAIN ] GPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:34.200410] [DEBUG] [LOCAL ] GPI: Loaded 15097 existing records from primary storage. [2025-10-20T19:02:34.200667] [INFO ] [MAIN ] GPI: Loaded 15097 existing records. [2025-10-20T19:02:34.383011] [WARN ] [MAIN ] GPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:34.383197] [INFO ] [MAIN ] GPI: Wiping existing primary data before full refresh. [2025-10-20T19:02:34.383244] [INFO ] [LOCAL ] GPI: Starting deletion of primary data. [2025-10-20T19:02:34.383585] [DEBUG] [LOCAL ] GPI: Deleted primary data file. [2025-10-20T19:02:34.383624] [INFO ] [MAIN ] GPI: Starting full primary refresh. [2025-10-20T19:02:34.656796] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-09. [2025-10-20T19:02:34.775587] [INFO ] [REMOTE ] GPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:34.808284] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:02:35.316097] [INFO ] [REMOTE ] GOOS: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T19:02:35.324813] [INFO ] [REMOTE ] GOOS: Full fetch complete (64765 records). [2025-10-20T19:02:35.324995] [DEBUG] [VALIDATOR ] GOOS: Validating primary data. [2025-10-20T19:02:35.453238] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-01. [2025-10-20T19:02:35.752884] [DEBUG] [VALIDATOR ] GOOS: All expected trading days are present. [2025-10-20T19:02:35.753145] [DEBUG] [VALIDATOR ] GOOS: Primary data validation passed. [2025-10-20T19:02:35.754358] [INFO ] [MAIN ] GOOS: Fetched primary data is valid. [2025-10-20T19:02:35.754508] [DEBUG] [LOCAL ] GOOS: Saving primary data. [2025-10-20T19:02:35.755932] [DEBUG] [LOCAL ] GOOS: Trimmed 64765 records to 44284 (keeping 200 days of data) [2025-10-20T19:02:35.871454] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:02:36.057475] [DEBUG] [LOCAL ] GOOS: Primary data saved (44284 records). [2025-10-20T19:02:36.057619] [INFO ] [MAIN ] GOOS: Full history saved. [2025-10-20T19:02:36.057739] [INFO ] [MAIN ] Update Progress: 954/2367 symbols completed. GOOS took 30.4s. Estimated remaining: 3h 1m 29s. Rate: 7.8 symbols/min [2025-10-20T19:02:36.253767] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-02. [2025-10-20T19:02:36.281832] [INFO ] [MAIN ] GPJA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:36.289389] [DEBUG] [LOCAL ] GPJA: Loaded 4230 existing records from primary storage. [2025-10-20T19:02:36.289739] [INFO ] [MAIN ] GPJA: Loaded 4230 existing records. [2025-10-20T19:02:36.478107] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:02:36.558498] [WARN ] [MAIN ] GPJA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:36.558750] [INFO ] [MAIN ] GPJA: Wiping existing primary data before full refresh. [2025-10-20T19:02:36.558806] [INFO ] [LOCAL ] GPJA: Starting deletion of primary data. [2025-10-20T19:02:36.559028] [DEBUG] [LOCAL ] GPJA: Deleted primary data file. [2025-10-20T19:02:36.559085] [INFO ] [MAIN ] GPJA: Starting full primary refresh. [2025-10-20T19:02:36.944950] [INFO ] [REMOTE ] GPJA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:37.051378] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-02. [2025-10-20T19:02:37.851571] [INFO ] [REMOTE ] GOLF: Fetching primary data for 2025-10. [2025-10-20T19:02:37.857602] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T19:02:37.983225] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:02:38.651390] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-03. [2025-10-20T19:02:39.132921] [INFO ] [REMOTE ] GOLF: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T19:02:39.137103] [INFO ] [REMOTE ] GOLF: Full fetch complete (45348 records). [2025-10-20T19:02:39.137181] [DEBUG] [VALIDATOR ] GOLF: Validating primary data. [2025-10-20T19:02:39.456515] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-03. [2025-10-20T19:02:39.520602] [DEBUG] [VALIDATOR ] GOLF: All expected trading days are present. [2025-10-20T19:02:39.520717] [DEBUG] [VALIDATOR ] GOLF: Primary data validation passed. [2025-10-20T19:02:39.521502] [INFO ] [MAIN ] GOLF: Fetched primary data is valid. [2025-10-20T19:02:39.521543] [DEBUG] [LOCAL ] GOLF: Saving primary data. [2025-10-20T19:02:39.522548] [DEBUG] [LOCAL ] GOLF: Trimmed 45348 records to 30669 (keeping 200 days of data) [2025-10-20T19:02:39.709856] [DEBUG] [LOCAL ] GOLF: Primary data saved (30669 records). [2025-10-20T19:02:39.709954] [INFO ] [MAIN ] GOLF: Full history saved. [2025-10-20T19:02:39.710032] [INFO ] [MAIN ] Update Progress: 955/2367 symbols completed. GOLF took 34.4s. Estimated remaining: 3h 1m 15s. Rate: 7.8 symbols/min [2025-10-20T19:02:39.899844] [INFO ] [MAIN ] GPK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:39.901122] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T19:02:39.954846] [DEBUG] [LOCAL ] GPK: Loaded 53548 existing records from primary storage. [2025-10-20T19:02:39.955165] [INFO ] [MAIN ] GPK: Loaded 53548 existing records. [2025-10-20T19:02:40.137895] [WARN ] [MAIN ] GPK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:40.138126] [INFO ] [MAIN ] GPK: Wiping existing primary data before full refresh. [2025-10-20T19:02:40.138181] [INFO ] [LOCAL ] GPK: Starting deletion of primary data. [2025-10-20T19:02:40.138816] [DEBUG] [LOCAL ] GPK: Deleted primary data file. [2025-10-20T19:02:40.138872] [INFO ] [MAIN ] GPK: Starting full primary refresh. [2025-10-20T19:02:40.256463] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-04. [2025-10-20T19:02:40.493873] [INFO ] [REMOTE ] GPK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:02:41.051355] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-01. [2025-10-20T19:02:41.089828] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:02:41.468784] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:02:41.851361] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-05. [2025-10-20T19:02:42.656467] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-01. [2025-10-20T19:02:42.673291] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:02:42.691537] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T19:02:43.451363] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-02. [2025-10-20T19:02:43.493257] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T19:02:44.251371] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-02. [2025-10-20T19:02:44.912038] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T19:02:44.918359] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:02:45.051374] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-03. [2025-10-20T19:02:45.594456] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T19:02:45.851376] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-04. [2025-10-20T19:02:46.419626] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:02:46.651364] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-05. [2025-10-20T19:02:47.099595] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T19:02:47.451424] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-04. [2025-10-20T19:02:48.251377] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-06. [2025-10-20T19:02:48.849768] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T19:02:49.051384] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-06. [2025-10-20T19:02:49.147630] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:02:49.851443] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-05. [2025-10-20T19:02:50.042010] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T19:02:50.651432] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-07. [2025-10-20T19:02:51.451352] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-07. [2025-10-20T19:02:51.451256] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:02:51.636109] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:02:51.802703] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T19:02:52.251411] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-08. [2025-10-20T19:02:53.051371] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-03. [2025-10-20T19:02:53.193990] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T19:02:53.851463] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-06. [2025-10-20T19:02:54.651395] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-08. [2025-10-20T19:02:54.811088] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:02:55.231769] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:02:55.380079] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:02:55.451400] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-09. [2025-10-20T19:02:56.119681] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T19:02:56.251389] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-04. [2025-10-20T19:02:57.051394] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-07. [2025-10-20T19:02:57.851390] [INFO ] [REMOTE ] GPJA: Fetching primary data for 2025-10. [2025-10-20T19:02:57.915629] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:02:58.405391] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:02:58.427333] [INFO ] [REMOTE ] GPJA: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:02:58.428263] [INFO ] [REMOTE ] GPJA: Full fetch complete (6894 records). [2025-10-20T19:02:58.428330] [DEBUG] [VALIDATOR ] GPJA: Validating primary data. [2025-10-20T19:02:58.656546] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-08. [2025-10-20T19:02:58.780349] [DEBUG] [VALIDATOR ] GPJA: All expected trading days are present. [2025-10-20T19:02:58.780421] [DEBUG] [VALIDATOR ] GPJA: Primary data validation passed. [2025-10-20T19:02:58.780591] [INFO ] [MAIN ] GPJA: Fetched primary data is valid. [2025-10-20T19:02:58.780629] [DEBUG] [LOCAL ] GPJA: Saving primary data. [2025-10-20T19:02:58.781027] [DEBUG] [LOCAL ] GPJA: Trimmed 6894 records to 4118 (keeping 200 days of data) [2025-10-20T19:02:58.805689] [DEBUG] [LOCAL ] GPJA: Primary data saved (4118 records). [2025-10-20T19:02:58.805749] [INFO ] [MAIN ] GPJA: Full history saved. [2025-10-20T19:02:58.805813] [INFO ] [MAIN ] Update Progress: 956/2367 symbols completed. GPJA took 22.7s. Estimated remaining: 3h 1m 24s. Rate: 7.8 symbols/min [2025-10-20T19:02:58.973137] [INFO ] [MAIN ] GPMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:02:58.998265] [DEBUG] [LOCAL ] GPMT: Loaded 26215 existing records from primary storage. [2025-10-20T19:02:58.998509] [INFO ] [MAIN ] GPMT: Loaded 26215 existing records. [2025-10-20T19:02:59.176156] [WARN ] [MAIN ] GPMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:02:59.176378] [INFO ] [MAIN ] GPMT: Wiping existing primary data before full refresh. [2025-10-20T19:02:59.176424] [INFO ] [LOCAL ] GPMT: Starting deletion of primary data. [2025-10-20T19:02:59.176625] [DEBUG] [LOCAL ] GPMT: Deleted primary data file. [2025-10-20T19:02:59.176708] [INFO ] [MAIN ] GPMT: Starting full primary refresh. [2025-10-20T19:02:59.451485] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-09. [2025-10-20T19:02:59.532576] [INFO ] [REMOTE ] GPMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:00.141451] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:03:00.191051] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:03:00.251360] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-09. [2025-10-20T19:03:01.051409] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-05. [2025-10-20T19:03:01.851371] [INFO ] [REMOTE ] GPI: Fetching primary data for 2025-10. [2025-10-20T19:03:02.025446] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:03:02.651407] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-01. [2025-10-20T19:03:02.970988] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T19:03:03.014089] [INFO ] [REMOTE ] GPI: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:03:03.017263] [INFO ] [REMOTE ] GPI: Full fetch complete (21779 records). [2025-10-20T19:03:03.017339] [DEBUG] [VALIDATOR ] GPI: Validating primary data. [2025-10-20T19:03:03.396265] [DEBUG] [VALIDATOR ] GPI: All expected trading days are present. [2025-10-20T19:03:03.396338] [DEBUG] [VALIDATOR ] GPI: Primary data validation passed. [2025-10-20T19:03:03.396867] [INFO ] [MAIN ] GPI: Fetched primary data is valid. [2025-10-20T19:03:03.396927] [DEBUG] [LOCAL ] GPI: Saving primary data. [2025-10-20T19:03:03.397673] [DEBUG] [LOCAL ] GPI: Trimmed 21779 records to 14844 (keeping 200 days of data) [2025-10-20T19:03:03.464982] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-06. [2025-10-20T19:03:03.495661] [DEBUG] [LOCAL ] GPI: Primary data saved (14844 records). [2025-10-20T19:03:03.495822] [INFO ] [MAIN ] GPI: Full history saved. [2025-10-20T19:03:03.495915] [INFO ] [MAIN ] Update Progress: 957/2367 symbols completed. GPI took 29.5s. Estimated remaining: 3h 1m 12s. Rate: 7.8 symbols/min [2025-10-20T19:03:03.742715] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T19:03:04.251436] [INFO ] [REMOTE ] GPC: Fetching primary data for 2025-10. [2025-10-20T19:03:05.034916] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:03:05.051347] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-07. [2025-10-20T19:03:05.860552] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-02. [2025-10-20T19:03:05.866252] [INFO ] [REMOTE ] GPC: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T19:03:05.874505] [INFO ] [REMOTE ] GPC: Full fetch complete (68672 records). [2025-10-20T19:03:05.874666] [DEBUG] [VALIDATOR ] GPC: Validating primary data. [2025-10-20T19:03:06.275512] [DEBUG] [VALIDATOR ] GPC: All expected trading days are present. [2025-10-20T19:03:06.275605] [DEBUG] [VALIDATOR ] GPC: Primary data validation passed. [2025-10-20T19:03:06.277415] [INFO ] [MAIN ] GPC: Fetched primary data is valid. [2025-10-20T19:03:06.277497] [DEBUG] [LOCAL ] GPC: Saving primary data. [2025-10-20T19:03:06.279287] [DEBUG] [LOCAL ] GPC: Trimmed 68672 records to 45947 (keeping 200 days of data) [2025-10-20T19:03:06.571947] [DEBUG] [LOCAL ] GPC: Primary data saved (45947 records). [2025-10-20T19:03:06.572096] [INFO ] [MAIN ] GPC: Full history saved. [2025-10-20T19:03:06.572551] [INFO ] [MAIN ] Update Progress: 958/2367 symbols completed. GPC took 34.9s. Estimated remaining: 3h 0m 57s. Rate: 7.8 symbols/min [2025-10-20T19:03:06.839843] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:03:06.852832] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-08. [2025-10-20T19:03:06.935408] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:03:07.451404] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-03. [2025-10-20T19:03:08.660524] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:03:08.675037] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-09. [2025-10-20T19:03:08.695335] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T19:03:09.051445] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-04. [2025-10-20T19:03:10.101270] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:03:10.109517] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-05. [2025-10-20T19:03:10.546907] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T19:03:10.651443] [INFO ] [REMOTE ] GPK: Fetching primary data for 2025-10. [2025-10-20T19:03:11.105667] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:03:11.451459] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-06. [2025-10-20T19:03:12.144863] [INFO ] [REMOTE ] GPK: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:03:12.157565] [INFO ] [REMOTE ] GPK: Full fetch complete (76056 records). [2025-10-20T19:03:12.157809] [DEBUG] [VALIDATOR ] GPK: Validating primary data. [2025-10-20T19:03:12.623461] [DEBUG] [VALIDATOR ] GPK: All expected trading days are present. [2025-10-20T19:03:12.623704] [DEBUG] [VALIDATOR ] GPK: Primary data validation passed. [2025-10-20T19:03:12.625170] [INFO ] [MAIN ] GPK: Fetched primary data is valid. [2025-10-20T19:03:12.625254] [DEBUG] [LOCAL ] GPK: Saving primary data. [2025-10-20T19:03:12.627721] [DEBUG] [LOCAL ] GPK: Trimmed 76056 records to 52784 (keeping 200 days of data) [2025-10-20T19:03:13.026720] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:03:13.049946] [DEBUG] [LOCAL ] GPK: Primary data saved (52784 records). [2025-10-20T19:03:13.050322] [INFO ] [MAIN ] GPK: Full history saved. [2025-10-20T19:03:13.050521] [INFO ] [MAIN ] Update Progress: 959/2367 symbols completed. GPK took 33.3s. Estimated remaining: 3h 0m 48s. Rate: 7.8 symbols/min [2025-10-20T19:03:13.051187] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-07. [2025-10-20T19:03:14.132403] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T19:03:14.140325] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-08. [2025-10-20T19:03:15.182840] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:03:15.187938] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-09. [2025-10-20T19:03:16.145308] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:03:16.149956] [INFO ] [REMOTE ] GPMT: Fetching primary data for 2025-10. [2025-10-20T19:03:17.261756] [INFO ] [REMOTE ] GPMT: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T19:03:17.265854] [INFO ] [REMOTE ] GPMT: Full fetch complete (36538 records). [2025-10-20T19:03:17.266097] [DEBUG] [VALIDATOR ] GPMT: Validating primary data. [2025-10-20T19:03:17.697948] [DEBUG] [VALIDATOR ] GPMT: All expected trading days are present. [2025-10-20T19:03:17.698084] [DEBUG] [VALIDATOR ] GPMT: Primary data validation passed. [2025-10-20T19:03:17.699697] [INFO ] [MAIN ] GPMT: Fetched primary data is valid. [2025-10-20T19:03:17.699769] [DEBUG] [LOCAL ] GPMT: Saving primary data. [2025-10-20T19:03:17.701364] [DEBUG] [LOCAL ] GPMT: Trimmed 36538 records to 25982 (keeping 200 days of data) [2025-10-20T19:03:17.866094] [DEBUG] [LOCAL ] GPMT: Primary data saved (25982 records). [2025-10-20T19:03:17.866261] [INFO ] [MAIN ] GPMT: Full history saved. [2025-10-20T19:03:17.866362] [INFO ] [MAIN ] Update Progress: 960/2367 symbols completed. GPMT took 19.1s. Estimated remaining: 3h 0m 36s. Rate: 7.8 symbols/min [2025-10-20T19:03:18.572590] [INFO ] [MAIN ] GRBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.626295] [INFO ] [MAIN ] GPOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.648990] [INFO ] [MAIN ] GPN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.650696] [INFO ] [MAIN ] GPRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:18.692042] [DEBUG] [LOCAL ] GRBK: Loaded 27388 existing records from primary storage. [2025-10-20T19:03:18.693667] [INFO ] [MAIN ] GRBK: Loaded 27388 existing records. [2025-10-20T19:03:18.744005] [DEBUG] [LOCAL ] GPOR: Loaded 23691 existing records from primary storage. [2025-10-20T19:03:18.822030] [INFO ] [MAIN ] GPOR: Loaded 23691 existing records. [2025-10-20T19:03:19.036354] [DEBUG] [LOCAL ] GPRK: Loaded 40348 existing records from primary storage. [2025-10-20T19:03:19.058504] [WARN ] [MAIN ] GRBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.083694] [INFO ] [MAIN ] GPRK: Loaded 40348 existing records. [2025-10-20T19:03:19.093370] [INFO ] [MAIN ] GRBK: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.121235] [DEBUG] [LOCAL ] GPN: Loaded 52716 existing records from primary storage. [2025-10-20T19:03:19.098791] [INFO ] [LOCAL ] GRBK: Starting deletion of primary data. [2025-10-20T19:03:19.173360] [INFO ] [MAIN ] GPN: Loaded 52716 existing records. [2025-10-20T19:03:19.238217] [DEBUG] [LOCAL ] GRBK: Deleted primary data file. [2025-10-20T19:03:19.238464] [INFO ] [MAIN ] GRBK: Starting full primary refresh. [2025-10-20T19:03:19.313492] [WARN ] [MAIN ] GPOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.421493] [INFO ] [MAIN ] GPOR: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.542864] [INFO ] [LOCAL ] GPOR: Starting deletion of primary data. [2025-10-20T19:03:19.609937] [DEBUG] [LOCAL ] GPOR: Deleted primary data file. [2025-10-20T19:03:19.650313] [WARN ] [MAIN ] GPRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.657758] [INFO ] [MAIN ] GPOR: Starting full primary refresh. [2025-10-20T19:03:19.766274] [INFO ] [MAIN ] GPRK: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.760913] [WARN ] [MAIN ] GPN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:19.861214] [INFO ] [LOCAL ] GPRK: Starting deletion of primary data. [2025-10-20T19:03:19.915479] [INFO ] [MAIN ] GPN: Wiping existing primary data before full refresh. [2025-10-20T19:03:19.944008] [INFO ] [LOCAL ] GPN: Starting deletion of primary data. [2025-10-20T19:03:19.963300] [DEBUG] [LOCAL ] GPRK: Deleted primary data file. [2025-10-20T19:03:19.983096] [DEBUG] [LOCAL ] GPN: Deleted primary data file. [2025-10-20T19:03:19.983009] [INFO ] [MAIN ] GPRK: Starting full primary refresh. [2025-10-20T19:03:20.062828] [INFO ] [MAIN ] GPN: Starting full primary refresh. [2025-10-20T19:03:20.712892] [INFO ] [REMOTE ] GRBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:20.862151] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-01. [2025-10-20T19:03:21.209170] [INFO ] [REMOTE ] GPRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.239282] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-01. [2025-10-20T19:03:21.270353] [INFO ] [REMOTE ] GPOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.293572] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-01. [2025-10-20T19:03:21.351436] [INFO ] [REMOTE ] GPN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:21.352506] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-01. [2025-10-20T19:03:22.334753] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T19:03:22.342026] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-02. [2025-10-20T19:03:22.549047] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:03:22.560500] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-02. [2025-10-20T19:03:22.669168] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:03:22.675164] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-02. [2025-10-20T19:03:22.797279] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:03:22.806923] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-02. [2025-10-20T19:03:23.247620] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T19:03:23.254173] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-03. [2025-10-20T19:03:23.812588] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:03:23.820077] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-03. [2025-10-20T19:03:24.101185] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:03:24.128166] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:03:24.137837] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-03. [2025-10-20T19:03:24.217541] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:03:24.251568] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-03. [2025-10-20T19:03:24.714971] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T19:03:25.051408] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-04. [2025-10-20T19:03:25.298804] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:03:25.851464] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-04. [2025-10-20T19:03:25.926774] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:03:26.182427] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:03:26.651410] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-04. [2025-10-20T19:03:27.243848] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:03:27.451418] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-05. [2025-10-20T19:03:28.251384] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-04. [2025-10-20T19:03:28.401065] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:03:29.001672] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:03:29.051379] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-06. [2025-10-20T19:03:29.546544] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:03:29.851422] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-05. [2025-10-20T19:03:30.532455] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:03:30.651349] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-05. [2025-10-20T19:03:31.198489] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:03:31.451395] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-05. [2025-10-20T19:03:31.829524] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T19:03:32.251348] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-06. [2025-10-20T19:03:32.303072] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T19:03:33.051380] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-06. [2025-10-20T19:03:33.851606] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-06. [2025-10-20T19:03:33.889761] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:03:34.183984] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:03:34.651399] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-07. [2025-10-20T19:03:34.847613] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:03:35.451405] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-07. [2025-10-20T19:03:36.086480] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:03:36.251371] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-07. [2025-10-20T19:03:36.748120] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T19:03:37.051428] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-07. [2025-10-20T19:03:37.851367] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-08. [2025-10-20T19:03:38.174863] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:03:38.207356] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T19:03:38.651372] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-08. [2025-10-20T19:03:38.701186] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:03:39.451375] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-08. [2025-10-20T19:03:40.105805] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:03:40.251378] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-09. [2025-10-20T19:03:40.635995] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:03:41.051413] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-09. [2025-10-20T19:03:41.382357] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T19:03:41.851348] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-08. [2025-10-20T19:03:42.240490] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T19:03:42.651376] [INFO ] [REMOTE ] GPOR: Fetching primary data for 2025-10. [2025-10-20T19:03:43.451350] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-09. [2025-10-20T19:03:43.507419] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:03:43.881406] [INFO ] [REMOTE ] GPOR: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:03:43.885518] [INFO ] [REMOTE ] GPOR: Full fetch complete (34908 records). [2025-10-20T19:03:43.885607] [DEBUG] [VALIDATOR ] GPOR: Validating primary data. [2025-10-20T19:03:44.251637] [INFO ] [REMOTE ] GRBK: Fetching primary data for 2025-10. [2025-10-20T19:03:44.264961] [DEBUG] [VALIDATOR ] GPOR: All expected trading days are present. [2025-10-20T19:03:44.265023] [DEBUG] [VALIDATOR ] GPOR: Primary data validation passed. [2025-10-20T19:03:44.265767] [INFO ] [MAIN ] GPOR: Fetched primary data is valid. [2025-10-20T19:03:44.265931] [DEBUG] [LOCAL ] GPOR: Saving primary data. [2025-10-20T19:03:44.266877] [DEBUG] [LOCAL ] GPOR: Trimmed 34908 records to 23397 (keeping 200 days of data) [2025-10-20T19:03:44.412850] [DEBUG] [LOCAL ] GPOR: Primary data saved (23397 records). [2025-10-20T19:03:44.412998] [INFO ] [MAIN ] GPOR: Full history saved. [2025-10-20T19:03:44.413082] [INFO ] [MAIN ] Update Progress: 961/2367 symbols completed. GPOR took 26.5s. Estimated remaining: 3h 0m 56s. Rate: 7.8 symbols/min [2025-10-20T19:03:44.598748] [INFO ] [MAIN ] GRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:44.609255] [DEBUG] [LOCAL ] GRC: Loaded 9440 existing records from primary storage. [2025-10-20T19:03:44.609483] [INFO ] [MAIN ] GRC: Loaded 9440 existing records. [2025-10-20T19:03:44.802520] [WARN ] [MAIN ] GRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:44.802850] [INFO ] [MAIN ] GRC: Wiping existing primary data before full refresh. [2025-10-20T19:03:44.802931] [INFO ] [LOCAL ] GRC: Starting deletion of primary data. [2025-10-20T19:03:44.803197] [DEBUG] [LOCAL ] GRC: Deleted primary data file. [2025-10-20T19:03:44.803271] [INFO ] [MAIN ] GRC: Starting full primary refresh. [2025-10-20T19:03:44.977722] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T19:03:45.057249] [INFO ] [REMOTE ] GPRK: Fetching primary data for 2025-10. [2025-10-20T19:03:45.197260] [INFO ] [REMOTE ] GRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:45.613362] [INFO ] [REMOTE ] GRBK: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T19:03:45.617697] [INFO ] [REMOTE ] GRBK: Full fetch complete (40133 records). [2025-10-20T19:03:45.617815] [DEBUG] [VALIDATOR ] GRBK: Validating primary data. [2025-10-20T19:03:45.858093] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-01. [2025-10-20T19:03:45.994692] [DEBUG] [VALIDATOR ] GRBK: All expected trading days are present. [2025-10-20T19:03:45.994782] [DEBUG] [VALIDATOR ] GRBK: Primary data validation passed. [2025-10-20T19:03:45.995429] [INFO ] [MAIN ] GRBK: Fetched primary data is valid. [2025-10-20T19:03:45.995469] [DEBUG] [LOCAL ] GRBK: Saving primary data. [2025-10-20T19:03:45.996408] [DEBUG] [LOCAL ] GRBK: Trimmed 40133 records to 26937 (keeping 200 days of data) [2025-10-20T19:03:46.155322] [DEBUG] [LOCAL ] GRBK: Primary data saved (26937 records). [2025-10-20T19:03:46.155463] [INFO ] [MAIN ] GRBK: Full history saved. [2025-10-20T19:03:46.155546] [INFO ] [MAIN ] Update Progress: 962/2367 symbols completed. GRBK took 28.3s. Estimated remaining: 3h 0m 39s. Rate: 7.8 symbols/min [2025-10-20T19:03:46.327189] [INFO ] [MAIN ] GRDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:46.346486] [DEBUG] [LOCAL ] GRDN: Loaded 21216 existing records from primary storage. [2025-10-20T19:03:46.346754] [INFO ] [MAIN ] GRDN: Loaded 21216 existing records. [2025-10-20T19:03:46.522770] [WARN ] [MAIN ] GRDN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:46.523008] [INFO ] [MAIN ] GRDN: Wiping existing primary data before full refresh. [2025-10-20T19:03:46.523058] [INFO ] [LOCAL ] GRDN: Starting deletion of primary data. [2025-10-20T19:03:46.523235] [DEBUG] [LOCAL ] GRDN: Deleted primary data file. [2025-10-20T19:03:46.523271] [INFO ] [MAIN ] GRDN: Starting full primary refresh. [2025-10-20T19:03:46.662016] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-09. [2025-10-20T19:03:46.886425] [INFO ] [REMOTE ] GRDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:46.893132] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:03:46.942709] [INFO ] [REMOTE ] GPRK: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:03:46.948842] [INFO ] [REMOTE ] GPRK: Full fetch complete (58328 records). [2025-10-20T19:03:46.948947] [DEBUG] [VALIDATOR ] GPRK: Validating primary data. [2025-10-20T19:03:47.326831] [DEBUG] [VALIDATOR ] GPRK: All expected trading days are present. [2025-10-20T19:03:47.326908] [DEBUG] [VALIDATOR ] GPRK: Primary data validation passed. [2025-10-20T19:03:47.327733] [INFO ] [MAIN ] GPRK: Fetched primary data is valid. [2025-10-20T19:03:47.327793] [DEBUG] [LOCAL ] GPRK: Saving primary data. [2025-10-20T19:03:47.329096] [DEBUG] [LOCAL ] GPRK: Trimmed 58328 records to 39821 (keeping 200 days of data) [2025-10-20T19:03:47.484289] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-01. [2025-10-20T19:03:47.550856] [DEBUG] [LOCAL ] GPRK: Primary data saved (39821 records). [2025-10-20T19:03:47.550974] [INFO ] [MAIN ] GPRK: Full history saved. [2025-10-20T19:03:47.551048] [INFO ] [MAIN ] Update Progress: 963/2367 symbols completed. GPRK took 29.7s. Estimated remaining: 3h 0m 23s. Rate: 7.8 symbols/min [2025-10-20T19:03:47.715685] [INFO ] [MAIN ] GRMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:47.753181] [DEBUG] [LOCAL ] GRMN: Loaded 40933 existing records from primary storage. [2025-10-20T19:03:47.753450] [INFO ] [MAIN ] GRMN: Loaded 40933 existing records. [2025-10-20T19:03:47.925602] [WARN ] [MAIN ] GRMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:47.925843] [INFO ] [MAIN ] GRMN: Wiping existing primary data before full refresh. [2025-10-20T19:03:47.925888] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-10-20T19:03:47.926069] [DEBUG] [LOCAL ] GRMN: Deleted primary data file. [2025-10-20T19:03:47.926106] [INFO ] [MAIN ] GRMN: Starting full primary refresh. [2025-10-20T19:03:48.253474] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:03:48.252714] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-02. [2025-10-20T19:03:48.327382] [INFO ] [REMOTE ] GRMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:48.473920] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:03:49.051362] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-02. [2025-10-20T19:03:49.103090] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T19:03:49.790953] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:03:49.851442] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-03. [2025-10-20T19:03:50.629210] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:03:50.651342] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-04. [2025-10-20T19:03:51.451387] [INFO ] [REMOTE ] GPN: Fetching primary data for 2025-10. [2025-10-20T19:03:51.475768] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T19:03:52.251358] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-01. [2025-10-20T19:03:53.051394] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-03. [2025-10-20T19:03:53.226841] [INFO ] [REMOTE ] GPN: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:03:53.234365] [INFO ] [REMOTE ] GPN: Full fetch complete (75071 records). [2025-10-20T19:03:53.234511] [DEBUG] [VALIDATOR ] GPN: Validating primary data. [2025-10-20T19:03:53.621372] [DEBUG] [VALIDATOR ] GPN: All expected trading days are present. [2025-10-20T19:03:53.621459] [DEBUG] [VALIDATOR ] GPN: Primary data validation passed. [2025-10-20T19:03:53.622398] [INFO ] [MAIN ] GPN: Fetched primary data is valid. [2025-10-20T19:03:53.622460] [DEBUG] [LOCAL ] GPN: Saving primary data. [2025-10-20T19:03:53.623939] [DEBUG] [LOCAL ] GPN: Trimmed 75071 records to 51943 (keeping 200 days of data) [2025-10-20T19:03:53.855002] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-05. [2025-10-20T19:03:53.926191] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:03:54.001800] [DEBUG] [LOCAL ] GPN: Primary data saved (51943 records). [2025-10-20T19:03:54.003087] [INFO ] [MAIN ] GPN: Full history saved. [2025-10-20T19:03:54.004823] [INFO ] [MAIN ] Update Progress: 964/2367 symbols completed. GPN took 36.1s. Estimated remaining: 3h 0m 13s. Rate: 7.8 symbols/min [2025-10-20T19:03:54.004627] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:03:54.184596] [INFO ] [MAIN ] GRND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:03:54.233776] [DEBUG] [LOCAL ] GRND: Loaded 50893 existing records from primary storage. [2025-10-20T19:03:54.234104] [INFO ] [MAIN ] GRND: Loaded 50893 existing records. [2025-10-20T19:03:54.421906] [WARN ] [MAIN ] GRND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:03:54.422115] [INFO ] [MAIN ] GRND: Wiping existing primary data before full refresh. [2025-10-20T19:03:54.422161] [INFO ] [LOCAL ] GRND: Starting deletion of primary data. [2025-10-20T19:03:54.422675] [DEBUG] [LOCAL ] GRND: Deleted primary data file. [2025-10-20T19:03:54.422742] [INFO ] [MAIN ] GRND: Starting full primary refresh. [2025-10-20T19:03:54.651796] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-02. [2025-10-20T19:03:54.728687] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T19:03:54.808909] [INFO ] [REMOTE ] GRND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:03:55.451376] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-01. [2025-10-20T19:03:56.251416] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-04. [2025-10-20T19:03:56.669434] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T19:03:56.949507] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:03:57.051383] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-02. [2025-10-20T19:03:57.270868] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:03:57.851378] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-05. [2025-10-20T19:03:58.651401] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-03. [2025-10-20T19:03:58.780360] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T19:03:59.091590] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:03:59.451376] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-03. [2025-10-20T19:04:00.200190] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:04:00.251402] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-04. [2025-10-20T19:04:01.051475] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-06. [2025-10-20T19:04:01.406877] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:04:01.481474] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:04:01.851344] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-04. [2025-10-20T19:04:02.133287] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:04:02.651379] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-06. [2025-10-20T19:04:03.365842] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:04:03.451446] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-05. [2025-10-20T19:04:03.647011] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:04:04.251416] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-05. [2025-10-20T19:04:05.051408] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-07. [2025-10-20T19:04:05.133925] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:04:05.851398] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-07. [2025-10-20T19:04:06.141418] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T19:04:06.416627] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:04:06.651386] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-06. [2025-10-20T19:04:07.091530] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:04:07.451392] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-08. [2025-10-20T19:04:08.116403] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:04:08.251351] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-07. [2025-10-20T19:04:08.489703] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:04:09.051371] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-09. [2025-10-20T19:04:09.851399] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-08. [2025-10-20T19:04:09.962519] [INFO ] [REMOTE ] GRMN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:04:10.416446] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T19:04:10.619126] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T19:04:10.651358] [INFO ] [REMOTE ] GRDN: Fetching primary data for 2025-10. [2025-10-20T19:04:11.451368] [INFO ] [REMOTE ] GRMN: Fetching primary data for 2025-08. [2025-10-20T19:04:11.556729] [DEBUG] [REMOTE ] GRMN: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-10-20T19:04:11.557011] [WARN ] [REMOTE ] GRMN: No data returned for month 2025-08 [2025-10-20T19:04:11.557100] [FATAL] [REMOTE ] GRMN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:04:11.557204] [FATAL] [MAIN ] GRMN: Failed to fetch primary data. [2025-10-20T19:04:11.557275] [WARN ] [MAIN ] GRMN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:04:11.557358] [INFO ] [LOCAL ] GRMN: Starting deletion of primary data. [2025-10-20T19:04:11.557428] [DEBUG] [LOCAL ] GRMN: No primary data file found to delete. [2025-10-20T19:04:11.557467] [INFO ] [LOCAL ] GRMN: Starting deletion of derivative data. [2025-10-20T19:04:11.557728] [DEBUG] [LOCAL ] GRMN: Deleted derivative data file. [2025-10-20T19:04:11.557816] [INFO ] [MAIN ] Update Progress: 965/2367 symbols completed. GRMN took 24.0s. Estimated remaining: 3h 0m 20s. Rate: 7.8 symbols/min [2025-10-20T19:04:11.736594] [INFO ] [MAIN ] GRNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:11.768767] [DEBUG] [LOCAL ] GRNT: Loaded 34443 existing records from primary storage. [2025-10-20T19:04:11.769068] [INFO ] [MAIN ] GRNT: Loaded 34443 existing records. [2025-10-20T19:04:11.953485] [WARN ] [MAIN ] GRNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:11.963085] [INFO ] [REMOTE ] GRDN: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:04:11.962785] [INFO ] [MAIN ] GRNT: Wiping existing primary data before full refresh. [2025-10-20T19:04:11.966825] [INFO ] [LOCAL ] GRNT: Starting deletion of primary data. [2025-10-20T19:04:11.966930] [INFO ] [REMOTE ] GRDN: Full fetch complete (26339 records). [2025-10-20T19:04:11.967047] [DEBUG] [VALIDATOR ] GRDN: Validating primary data. [2025-10-20T19:04:11.992029] [DEBUG] [LOCAL ] GRNT: Deleted primary data file. [2025-10-20T19:04:12.012945] [INFO ] [MAIN ] GRNT: Starting full primary refresh. [2025-10-20T19:04:12.251464] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-06. [2025-10-20T19:04:12.738629] [INFO ] [REMOTE ] GRNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:12.738881] [DEBUG] [VALIDATOR ] GRDN: All expected trading days are present. [2025-10-20T19:04:12.739233] [DEBUG] [VALIDATOR ] GRDN: Primary data validation passed. [2025-10-20T19:04:12.739755] [INFO ] [MAIN ] GRDN: Fetched primary data is valid. [2025-10-20T19:04:12.739817] [DEBUG] [LOCAL ] GRDN: Saving primary data. [2025-10-20T19:04:12.740423] [DEBUG] [LOCAL ] GRDN: Trimmed 26339 records to 21063 (keeping 200 days of data) [2025-10-20T19:04:12.860387] [DEBUG] [LOCAL ] GRDN: Primary data saved (21063 records). [2025-10-20T19:04:12.860514] [INFO ] [MAIN ] GRDN: Full history saved. [2025-10-20T19:04:12.860589] [INFO ] [MAIN ] Update Progress: 966/2367 symbols completed. GRDN took 26.7s. Estimated remaining: 3h 0m 3s. Rate: 7.8 symbols/min [2025-10-20T19:04:13.037228] [INFO ] [MAIN ] GROV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:13.047458] [DEBUG] [LOCAL ] GROV: Loaded 8805 existing records from primary storage. [2025-10-20T19:04:13.047697] [INFO ] [MAIN ] GROV: Loaded 8805 existing records. [2025-10-20T19:04:13.058715] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-01. [2025-10-20T19:04:13.235583] [WARN ] [MAIN ] GROV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:13.235837] [INFO ] [MAIN ] GROV: Wiping existing primary data before full refresh. [2025-10-20T19:04:13.235885] [INFO ] [LOCAL ] GROV: Starting deletion of primary data. [2025-10-20T19:04:13.236076] [DEBUG] [LOCAL ] GROV: Deleted primary data file. [2025-10-20T19:04:13.236130] [INFO ] [MAIN ] GROV: Starting full primary refresh. [2025-10-20T19:04:13.585599] [INFO ] [REMOTE ] GROV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:13.851381] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-09. [2025-10-20T19:04:13.951265] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:04:14.465356] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:04:14.606274] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T19:04:14.651409] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-02. [2025-10-20T19:04:15.451401] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-01. [2025-10-20T19:04:15.677339] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:04:16.245391] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:04:16.251335] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-02. [2025-10-20T19:04:17.051402] [INFO ] [REMOTE ] GRC: Fetching primary data for 2025-10. [2025-10-20T19:04:17.061018] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T19:04:17.851447] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-03. [2025-10-20T19:04:18.111755] [INFO ] [REMOTE ] GRC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:04:18.115857] [INFO ] [REMOTE ] GRC: Full fetch complete (13431 records). [2025-10-20T19:04:18.116018] [DEBUG] [VALIDATOR ] GRC: Validating primary data. [2025-10-20T19:04:18.657191] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-03. [2025-10-20T19:04:18.691555] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T19:04:18.698904] [DEBUG] [VALIDATOR ] GRC: All expected trading days are present. [2025-10-20T19:04:18.699400] [DEBUG] [VALIDATOR ] GRC: Primary data validation passed. [2025-10-20T19:04:18.701839] [INFO ] [MAIN ] GRC: Fetched primary data is valid. [2025-10-20T19:04:18.702069] [DEBUG] [LOCAL ] GRC: Saving primary data. [2025-10-20T19:04:18.704152] [DEBUG] [LOCAL ] GRC: Trimmed 13431 records to 9306 (keeping 200 days of data) [2025-10-20T19:04:18.772214] [DEBUG] [LOCAL ] GRC: Primary data saved (9306 records). [2025-10-20T19:04:18.772368] [INFO ] [MAIN ] GRC: Full history saved. [2025-10-20T19:04:18.772454] [INFO ] [MAIN ] Update Progress: 967/2367 symbols completed. GRC took 34.4s. Estimated remaining: 2h 59m 52s. Rate: 7.8 symbols/min [2025-10-20T19:04:18.994357] [INFO ] [MAIN ] GRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:19.002085] [DEBUG] [LOCAL ] GRX: Loaded 4862 existing records from primary storage. [2025-10-20T19:04:19.002347] [INFO ] [MAIN ] GRX: Loaded 4862 existing records. [2025-10-20T19:04:19.211697] [WARN ] [MAIN ] GRX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:19.211943] [INFO ] [MAIN ] GRX: Wiping existing primary data before full refresh. [2025-10-20T19:04:19.211990] [INFO ] [LOCAL ] GRX: Starting deletion of primary data. [2025-10-20T19:04:19.212194] [DEBUG] [LOCAL ] GRX: Deleted primary data file. [2025-10-20T19:04:19.212254] [INFO ] [MAIN ] GRX: Starting full primary refresh. [2025-10-20T19:04:19.459530] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-07. [2025-10-20T19:04:19.640338] [INFO ] [REMOTE ] GRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:20.065514] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:04:20.251377] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-01. [2025-10-20T19:04:20.819913] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:04:21.051355] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-04. [2025-10-20T19:04:21.231056] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:04:21.698550] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T19:04:21.851354] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-04. [2025-10-20T19:04:22.651378] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-02. [2025-10-20T19:04:23.214863] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T19:04:23.239909] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:04:23.451404] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-03. [2025-10-20T19:04:24.036827] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:04:24.251430] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-05. [2025-10-20T19:04:24.825311] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T19:04:25.051385] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-06. [2025-10-20T19:04:25.768752] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:04:25.851434] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-05. [2025-10-20T19:04:26.651415] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-07. [2025-10-20T19:04:27.086130] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T19:04:27.451388] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-06. [2025-10-20T19:04:27.707248] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T19:04:28.251404] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-08. [2025-10-20T19:04:28.856203] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T19:04:28.960223] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T19:04:29.051369] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-09. [2025-10-20T19:04:29.426984] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-09 (0.38 seconds). [2025-10-20T19:04:29.851403] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-08. [2025-10-20T19:04:30.651383] [INFO ] [REMOTE ] GROV: Fetching primary data for 2025-10. [2025-10-20T19:04:31.451374] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-07. [2025-10-20T19:04:31.477250] [INFO ] [REMOTE ] GROV: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:04:31.478856] [INFO ] [REMOTE ] GROV: Full fetch complete (13699 records). [2025-10-20T19:04:31.478925] [DEBUG] [VALIDATOR ] GROV: Validating primary data. [2025-10-20T19:04:31.867369] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T19:04:31.876319] [DEBUG] [VALIDATOR ] GROV: All expected trading days are present. [2025-10-20T19:04:31.876401] [DEBUG] [VALIDATOR ] GROV: Primary data validation passed. [2025-10-20T19:04:31.886266] [INFO ] [MAIN ] GROV: Fetched primary data is valid. [2025-10-20T19:04:31.886343] [DEBUG] [LOCAL ] GROV: Saving primary data. [2025-10-20T19:04:31.887341] [DEBUG] [LOCAL ] GROV: Trimmed 13699 records to 8747 (keeping 200 days of data) [2025-10-20T19:04:31.936032] [DEBUG] [LOCAL ] GROV: Primary data saved (8747 records). [2025-10-20T19:04:31.936165] [INFO ] [MAIN ] GROV: Full history saved. [2025-10-20T19:04:31.936243] [INFO ] [MAIN ] Update Progress: 968/2367 symbols completed. GROV took 19.1s. Estimated remaining: 2h 59m 52s. Rate: 7.8 symbols/min [2025-10-20T19:04:32.102148] [INFO ] [MAIN ] GS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:32.154171] [DEBUG] [LOCAL ] GS: Loaded 52558 existing records from primary storage. [2025-10-20T19:04:32.154509] [INFO ] [MAIN ] GS: Loaded 52558 existing records. [2025-10-20T19:04:32.253063] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-04. [2025-10-20T19:04:32.336422] [WARN ] [MAIN ] GS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:32.336682] [INFO ] [MAIN ] GS: Wiping existing primary data before full refresh. [2025-10-20T19:04:32.336742] [INFO ] [LOCAL ] GS: Starting deletion of primary data. [2025-10-20T19:04:32.337428] [DEBUG] [LOCAL ] GS: Deleted primary data file. [2025-10-20T19:04:32.337486] [INFO ] [MAIN ] GS: Starting full primary refresh. [2025-10-20T19:04:32.690699] [INFO ] [REMOTE ] GS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:32.800936] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:04:33.046830] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:04:33.051334] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-05. [2025-10-20T19:04:33.638182] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:04:33.851368] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-09. [2025-10-20T19:04:34.651360] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-06. [2025-10-20T19:04:35.217398] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T19:04:35.451360] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-01. [2025-10-20T19:04:35.492297] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T19:04:36.251385] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-07. [2025-10-20T19:04:36.842406] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:04:37.051459] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-08. [2025-10-20T19:04:37.355893] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T19:04:37.633222] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:04:37.851361] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-02. [2025-10-20T19:04:38.651382] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-08. [2025-10-20T19:04:39.451455] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-09. [2025-10-20T19:04:39.955890] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:04:39.994078] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-10-20T19:04:40.145328] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T19:04:40.251390] [INFO ] [REMOTE ] GRND: Fetching primary data for 2025-10. [2025-10-20T19:04:41.051372] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-03. [2025-10-20T19:04:41.851367] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-09. [2025-10-20T19:04:42.163189] [INFO ] [REMOTE ] GRND: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T19:04:42.172600] [INFO ] [REMOTE ] GRND: Full fetch complete (71704 records). [2025-10-20T19:04:42.172792] [DEBUG] [VALIDATOR ] GRND: Validating primary data. [2025-10-20T19:04:42.564874] [DEBUG] [VALIDATOR ] GRND: All expected trading days are present. [2025-10-20T19:04:42.564981] [DEBUG] [VALIDATOR ] GRND: Primary data validation passed. [2025-10-20T19:04:42.565949] [INFO ] [MAIN ] GRND: Fetched primary data is valid. [2025-10-20T19:04:42.566015] [DEBUG] [LOCAL ] GRND: Saving primary data. [2025-10-20T19:04:42.567439] [DEBUG] [LOCAL ] GRND: Trimmed 71704 records to 50199 (keeping 200 days of data) [2025-10-20T19:04:42.656813] [INFO ] [REMOTE ] GRX: Fetching primary data for 2025-10. [2025-10-20T19:04:42.852901] [DEBUG] [LOCAL ] GRND: Primary data saved (50199 records). [2025-10-20T19:04:42.853030] [INFO ] [MAIN ] GRND: Full history saved. [2025-10-20T19:04:42.853108] [INFO ] [MAIN ] Update Progress: 969/2367 symbols completed. GRND took 48.8s. Estimated remaining: 2h 59m 49s. Rate: 7.8 symbols/min [2025-10-20T19:04:43.023028] [INFO ] [MAIN ] GSBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:43.058015] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:04:43.095988] [DEBUG] [LOCAL ] GSBD: Loaded 40293 existing records from primary storage. [2025-10-20T19:04:43.096266] [INFO ] [MAIN ] GSBD: Loaded 40293 existing records. [2025-10-20T19:04:43.269482] [INFO ] [REMOTE ] GRX: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T19:04:43.269305] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:04:43.302951] [WARN ] [MAIN ] GSBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:43.296614] [INFO ] [REMOTE ] GRX: Full fetch complete (6538 records). [2025-10-20T19:04:43.309569] [INFO ] [MAIN ] GSBD: Wiping existing primary data before full refresh. [2025-10-20T19:04:43.309827] [INFO ] [LOCAL ] GSBD: Starting deletion of primary data. [2025-10-20T19:04:43.309725] [DEBUG] [VALIDATOR ] GRX: Validating primary data. [2025-10-20T19:04:43.315707] [DEBUG] [LOCAL ] GSBD: Deleted primary data file. [2025-10-20T19:04:43.316208] [INFO ] [MAIN ] GSBD: Starting full primary refresh. [2025-10-20T19:04:43.452086] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-04. [2025-10-20T19:04:44.253893] [INFO ] [REMOTE ] GRNT: Fetching primary data for 2025-10. [2025-10-20T19:04:44.254441] [DEBUG] [VALIDATOR ] GRX: All expected trading days are present. [2025-10-20T19:04:44.253794] [INFO ] [REMOTE ] GSBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:44.256207] [DEBUG] [VALIDATOR ] GRX: Primary data validation passed. [2025-10-20T19:04:44.256599] [INFO ] [MAIN ] GRX: Fetched primary data is valid. [2025-10-20T19:04:44.256790] [DEBUG] [LOCAL ] GRX: Saving primary data. [2025-10-20T19:04:44.258048] [DEBUG] [LOCAL ] GRX: Trimmed 6538 records to 4840 (keeping 200 days of data) [2025-10-20T19:04:44.295701] [DEBUG] [LOCAL ] GRX: Primary data saved (4840 records). [2025-10-20T19:04:44.295959] [INFO ] [MAIN ] GRX: Full history saved. [2025-10-20T19:04:44.296078] [INFO ] [MAIN ] Update Progress: 970/2367 symbols completed. GRX took 25.5s. Estimated remaining: 2h 59m 33s. Rate: 7.8 symbols/min [2025-10-20T19:04:44.486718] [INFO ] [MAIN ] GSK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:44.547583] [DEBUG] [LOCAL ] GSK: Loaded 54285 existing records from primary storage. [2025-10-20T19:04:44.548185] [INFO ] [MAIN ] GSK: Loaded 54285 existing records. [2025-10-20T19:04:44.743804] [WARN ] [MAIN ] GSK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:44.744087] [INFO ] [MAIN ] GSK: Wiping existing primary data before full refresh. [2025-10-20T19:04:44.744136] [INFO ] [LOCAL ] GSK: Starting deletion of primary data. [2025-10-20T19:04:44.744366] [DEBUG] [LOCAL ] GSK: Deleted primary data file. [2025-10-20T19:04:44.744406] [INFO ] [MAIN ] GSK: Starting full primary refresh. [2025-10-20T19:04:45.055496] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-01. [2025-10-20T19:04:45.143497] [INFO ] [REMOTE ] GSK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:45.559534] [INFO ] [REMOTE ] GRNT: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:04:45.564740] [INFO ] [REMOTE ] GRNT: Full fetch complete (48348 records). [2025-10-20T19:04:45.564829] [DEBUG] [VALIDATOR ] GRNT: Validating primary data. [2025-10-20T19:04:45.852164] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-01. [2025-10-20T19:04:45.956760] [DEBUG] [VALIDATOR ] GRNT: All expected trading days are present. [2025-10-20T19:04:45.956941] [DEBUG] [VALIDATOR ] GRNT: Primary data validation passed. [2025-10-20T19:04:45.957721] [INFO ] [MAIN ] GRNT: Fetched primary data is valid. [2025-10-20T19:04:45.957806] [DEBUG] [LOCAL ] GRNT: Saving primary data. [2025-10-20T19:04:45.959100] [DEBUG] [LOCAL ] GRNT: Trimmed 48348 records to 33992 (keeping 200 days of data) [2025-10-20T19:04:46.079963] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:04:46.196862] [DEBUG] [LOCAL ] GRNT: Primary data saved (33992 records). [2025-10-20T19:04:46.196986] [INFO ] [MAIN ] GRNT: Full history saved. [2025-10-20T19:04:46.197060] [INFO ] [MAIN ] Update Progress: 971/2367 symbols completed. GRNT took 34.6s. Estimated remaining: 2h 59m 17s. Rate: 7.8 symbols/min [2025-10-20T19:04:46.365395] [INFO ] [MAIN ] GSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:04:46.396466] [DEBUG] [LOCAL ] GSL: Loaded 33725 existing records from primary storage. [2025-10-20T19:04:46.396731] [INFO ] [MAIN ] GSL: Loaded 33725 existing records. [2025-10-20T19:04:46.578885] [WARN ] [MAIN ] GSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:04:46.579231] [INFO ] [MAIN ] GSL: Wiping existing primary data before full refresh. [2025-10-20T19:04:46.592476] [INFO ] [LOCAL ] GSL: Starting deletion of primary data. [2025-10-20T19:04:46.601030] [DEBUG] [LOCAL ] GSL: Deleted primary data file. [2025-10-20T19:04:46.601822] [INFO ] [MAIN ] GSL: Starting full primary refresh. [2025-10-20T19:04:46.601768] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:04:46.652901] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-05. [2025-10-20T19:04:46.958161] [INFO ] [REMOTE ] GSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:04:47.407089] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:04:47.451409] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-01. [2025-10-20T19:04:48.251467] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-02. [2025-10-20T19:04:48.752368] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:04:48.918341] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-05 (2.27 seconds). [2025-10-20T19:04:49.051391] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-06. [2025-10-20T19:04:49.551174] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:04:49.851378] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-02. [2025-10-20T19:04:50.651365] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-02. [2025-10-20T19:04:51.013382] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-10-20T19:04:51.105955] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:04:51.451386] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-03. [2025-10-20T19:04:52.126824] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T19:04:52.251457] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-03. [2025-10-20T19:04:52.896747] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:04:53.051389] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-04. [2025-10-20T19:04:53.851385] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-03. [2025-10-20T19:04:53.902732] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:04:54.525142] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:04:54.651404] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-05. [2025-10-20T19:04:55.271479] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:04:55.451440] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-04. [2025-10-20T19:04:56.102820] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:04:56.251359] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-06. [2025-10-20T19:04:57.051389] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-07. [2025-10-20T19:04:57.263414] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:04:57.446458] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T19:04:57.851427] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-07. [2025-10-20T19:04:58.651407] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-05. [2025-10-20T19:04:59.123908] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T19:04:59.278903] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T19:04:59.451363] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-08. [2025-10-20T19:05:00.251346] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-08. [2025-10-20T19:05:00.505376] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T19:05:01.051444] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-06. [2025-10-20T19:05:01.258338] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:05:01.305639] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:05:01.851352] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-09. [2025-10-20T19:05:02.651368] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-09. [2025-10-20T19:05:02.933309] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T19:05:03.003221] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:05:03.451400] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-07. [2025-10-20T19:05:04.251340] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-04. [2025-10-20T19:05:04.760597] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-10-20T19:05:05.051552] [INFO ] [REMOTE ] GS: Fetching primary data for 2025-10. [2025-10-20T19:05:05.051480] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:05:05.853996] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-08. [2025-10-20T19:05:05.859780] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:05:05.944104] [INFO ] [REMOTE ] GS: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:05:05.951753] [INFO ] [REMOTE ] GS: Full fetch complete (75845 records). [2025-10-20T19:05:05.951880] [DEBUG] [VALIDATOR ] GS: Validating primary data. [2025-10-20T19:05:06.359513] [DEBUG] [VALIDATOR ] GS: All expected trading days are present. [2025-10-20T19:05:06.359603] [DEBUG] [VALIDATOR ] GS: Primary data validation passed. [2025-10-20T19:05:06.360538] [INFO ] [MAIN ] GS: Fetched primary data is valid. [2025-10-20T19:05:06.360603] [DEBUG] [LOCAL ] GS: Saving primary data. [2025-10-20T19:05:06.362041] [DEBUG] [LOCAL ] GS: Trimmed 75845 records to 51784 (keeping 200 days of data) [2025-10-20T19:05:06.664413] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-05. [2025-10-20T19:05:06.686436] [DEBUG] [LOCAL ] GS: Primary data saved (51784 records). [2025-10-20T19:05:06.686541] [INFO ] [MAIN ] GS: Full history saved. [2025-10-20T19:05:06.687259] [INFO ] [MAIN ] Update Progress: 972/2367 symbols completed. GS took 34.8s. Estimated remaining: 2h 59m 27s. Rate: 7.8 symbols/min [2025-10-20T19:05:06.860029] [INFO ] [MAIN ] GTES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:06.909858] [DEBUG] [LOCAL ] GTES: Loaded 51134 existing records from primary storage. [2025-10-20T19:05:06.910159] [INFO ] [MAIN ] GTES: Loaded 51134 existing records. [2025-10-20T19:05:07.091563] [WARN ] [MAIN ] GTES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:07.091756] [INFO ] [MAIN ] GTES: Wiping existing primary data before full refresh. [2025-10-20T19:05:07.091807] [INFO ] [LOCAL ] GTES: Starting deletion of primary data. [2025-10-20T19:05:07.092078] [DEBUG] [LOCAL ] GTES: Deleted primary data file. [2025-10-20T19:05:07.092119] [INFO ] [MAIN ] GTES: Starting full primary refresh. [2025-10-20T19:05:07.454450] [INFO ] [REMOTE ] GSL: Fetching primary data for 2025-10. [2025-10-20T19:05:07.454392] [INFO ] [REMOTE ] GTES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:07.723764] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:05:08.107055] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:05:08.251361] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-01. [2025-10-20T19:05:08.778991] [INFO ] [REMOTE ] GSL: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T19:05:08.786482] [INFO ] [REMOTE ] GSL: Full fetch complete (48304 records). [2025-10-20T19:05:08.786634] [DEBUG] [VALIDATOR ] GSL: Validating primary data. [2025-10-20T19:05:09.056556] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-06. [2025-10-20T19:05:09.168967] [DEBUG] [VALIDATOR ] GSL: All expected trading days are present. [2025-10-20T19:05:09.169061] [DEBUG] [VALIDATOR ] GSL: Primary data validation passed. [2025-10-20T19:05:09.169791] [INFO ] [MAIN ] GSL: Fetched primary data is valid. [2025-10-20T19:05:09.169852] [DEBUG] [LOCAL ] GSL: Saving primary data. [2025-10-20T19:05:09.170900] [DEBUG] [LOCAL ] GSL: Trimmed 48304 records to 33197 (keeping 200 days of data) [2025-10-20T19:05:09.366493] [DEBUG] [LOCAL ] GSL: Primary data saved (33197 records). [2025-10-20T19:05:09.366624] [INFO ] [MAIN ] GSL: Full history saved. [2025-10-20T19:05:09.366735] [INFO ] [MAIN ] Update Progress: 973/2367 symbols completed. GSL took 23.2s. Estimated remaining: 2h 59m 12s. Rate: 7.8 symbols/min [2025-10-20T19:05:09.538107] [INFO ] [MAIN ] GTLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:09.579604] [DEBUG] [LOCAL ] GTLS: Loaded 43892 existing records from primary storage. [2025-10-20T19:05:09.579921] [INFO ] [MAIN ] GTLS: Loaded 43892 existing records. [2025-10-20T19:05:09.790935] [WARN ] [MAIN ] GTLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:09.791213] [INFO ] [MAIN ] GTLS: Wiping existing primary data before full refresh. [2025-10-20T19:05:09.791291] [INFO ] [LOCAL ] GTLS: Starting deletion of primary data. [2025-10-20T19:05:09.792086] [DEBUG] [LOCAL ] GTLS: Deleted primary data file. [2025-10-20T19:05:09.793960] [INFO ] [MAIN ] GTLS: Starting full primary refresh. [2025-10-20T19:05:09.793886] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T19:05:09.857811] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-02. [2025-10-20T19:05:10.163734] [INFO ] [REMOTE ] GTLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:10.197587] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:05:10.651360] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-01. [2025-10-20T19:05:11.451448] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-07. [2025-10-20T19:05:11.709997] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T19:05:12.074548] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:05:12.251409] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-03. [2025-10-20T19:05:13.006904] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:05:13.051375] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-08. [2025-10-20T19:05:13.851351] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-02. [2025-10-20T19:05:13.883710] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:05:14.560614] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:05:14.651397] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-09. [2025-10-20T19:05:14.986048] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:05:15.451429] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-03. [2025-10-20T19:05:16.251386] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-09. [2025-10-20T19:05:16.566473] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T19:05:17.033100] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:05:17.051389] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-04. [2025-10-20T19:05:17.852020] [INFO ] [REMOTE ] GSK: Fetching primary data for 2025-10. [2025-10-20T19:05:17.854490] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T19:05:18.251290] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:05:18.651368] [INFO ] [REMOTE ] GSBD: Fetching primary data for 2025-10. [2025-10-20T19:05:19.451359] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-05. [2025-10-20T19:05:19.949064] [INFO ] [REMOTE ] GSK: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:05:19.957141] [INFO ] [REMOTE ] GSK: Full fetch complete (77635 records). [2025-10-20T19:05:19.957276] [DEBUG] [VALIDATOR ] GSK: Validating primary data. [2025-10-20T19:05:20.255150] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-04. [2025-10-20T19:05:20.365260] [DEBUG] [VALIDATOR ] GSK: All expected trading days are present. [2025-10-20T19:05:20.365338] [DEBUG] [VALIDATOR ] GSK: Primary data validation passed. [2025-10-20T19:05:20.366293] [INFO ] [MAIN ] GSK: Fetched primary data is valid. [2025-10-20T19:05:20.366355] [DEBUG] [LOCAL ] GSK: Saving primary data. [2025-10-20T19:05:20.367773] [DEBUG] [LOCAL ] GSK: Trimmed 77635 records to 53512 (keeping 200 days of data) [2025-10-20T19:05:20.620761] [INFO ] [REMOTE ] GSBD: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T19:05:20.646937] [INFO ] [REMOTE ] GSBD: Full fetch complete (58595 records). [2025-10-20T19:05:20.650164] [DEBUG] [VALIDATOR ] GSBD: Validating primary data. [2025-10-20T19:05:20.833587] [DEBUG] [LOCAL ] GSK: Primary data saved (53512 records). [2025-10-20T19:05:20.833851] [INFO ] [MAIN ] GSK: Full history saved. [2025-10-20T19:05:20.834533] [INFO ] [MAIN ] Update Progress: 974/2367 symbols completed. GSK took 36.5s. Estimated remaining: 2h 59m 10s. Rate: 7.8 symbols/min [2025-10-20T19:05:21.275239] [INFO ] [MAIN ] GTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:21.309888] [DEBUG] [VALIDATOR ] GSBD: All expected trading days are present. [2025-10-20T19:05:21.310331] [DEBUG] [VALIDATOR ] GSBD: Primary data validation passed. [2025-10-20T19:05:21.311207] [INFO ] [MAIN ] GSBD: Fetched primary data is valid. [2025-10-20T19:05:21.311381] [DEBUG] [LOCAL ] GSBD: Saving primary data. [2025-10-20T19:05:21.335037] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:05:21.343444] [DEBUG] [LOCAL ] GSBD: Trimmed 58595 records to 39645 (keeping 200 days of data) [2025-10-20T19:05:21.344961] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-06. [2025-10-20T19:05:21.494963] [DEBUG] [LOCAL ] GTN: Loaded 45765 existing records from primary storage. [2025-10-20T19:05:21.496551] [INFO ] [MAIN ] GTN: Loaded 45765 existing records. [2025-10-20T19:05:21.812358] [WARN ] [MAIN ] GTN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:21.813311] [INFO ] [MAIN ] GTN: Wiping existing primary data before full refresh. [2025-10-20T19:05:21.817364] [INFO ] [LOCAL ] GTN: Starting deletion of primary data. [2025-10-20T19:05:21.825842] [DEBUG] [LOCAL ] GTN: Deleted primary data file. [2025-10-20T19:05:21.836695] [INFO ] [MAIN ] GTN: Starting full primary refresh. [2025-10-20T19:05:21.854693] [DEBUG] [LOCAL ] GSBD: Primary data saved (39645 records). [2025-10-20T19:05:21.870043] [INFO ] [MAIN ] GSBD: Full history saved. [2025-10-20T19:05:21.884516] [INFO ] [MAIN ] Update Progress: 975/2367 symbols completed. GSBD took 39.0s. Estimated remaining: 2h 58m 53s. Rate: 7.8 symbols/min [2025-10-20T19:05:22.233524] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T19:05:22.349956] [INFO ] [MAIN ] GTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:22.376823] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-05. [2025-10-20T19:05:22.512897] [INFO ] [REMOTE ] GTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:22.529109] [DEBUG] [LOCAL ] GTY: Loaded 30765 existing records from primary storage. [2025-10-20T19:05:22.529559] [INFO ] [MAIN ] GTY: Loaded 30765 existing records. [2025-10-20T19:05:22.653395] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-01. [2025-10-20T19:05:22.722097] [WARN ] [MAIN ] GTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:22.722382] [INFO ] [MAIN ] GTY: Wiping existing primary data before full refresh. [2025-10-20T19:05:22.722434] [INFO ] [LOCAL ] GTY: Starting deletion of primary data. [2025-10-20T19:05:22.722683] [DEBUG] [LOCAL ] GTY: Deleted primary data file. [2025-10-20T19:05:22.722743] [INFO ] [MAIN ] GTY: Starting full primary refresh. [2025-10-20T19:05:23.119876] [INFO ] [REMOTE ] GTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:23.148585] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:05:23.451379] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-07. [2025-10-20T19:05:23.785735] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T19:05:23.841820] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:05:24.251395] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-06. [2025-10-20T19:05:24.990607] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:05:25.051361] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-08. [2025-10-20T19:05:25.770810] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T19:05:25.851374] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-02. [2025-10-20T19:05:26.532735] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:05:26.651382] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-09. [2025-10-20T19:05:27.126891] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T19:05:27.451369] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-07. [2025-10-20T19:05:28.232947] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:05:28.251335] [INFO ] [REMOTE ] GTLS: Fetching primary data for 2025-10. [2025-10-20T19:05:28.804212] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T19:05:29.051375] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-03. [2025-10-20T19:05:29.825159] [INFO ] [REMOTE ] GTLS: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T19:05:29.831758] [INFO ] [REMOTE ] GTLS: Full fetch complete (61617 records). [2025-10-20T19:05:29.831909] [DEBUG] [VALIDATOR ] GTLS: Validating primary data. [2025-10-20T19:05:29.863215] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-01. [2025-10-20T19:05:30.220506] [DEBUG] [VALIDATOR ] GTLS: All expected trading days are present. [2025-10-20T19:05:30.220617] [DEBUG] [VALIDATOR ] GTLS: Primary data validation passed. [2025-10-20T19:05:30.221507] [INFO ] [MAIN ] GTLS: Fetched primary data is valid. [2025-10-20T19:05:30.221559] [DEBUG] [LOCAL ] GTLS: Saving primary data. [2025-10-20T19:05:30.222844] [DEBUG] [LOCAL ] GTLS: Trimmed 61617 records to 43326 (keeping 200 days of data) [2025-10-20T19:05:30.474391] [DEBUG] [LOCAL ] GTLS: Primary data saved (43326 records). [2025-10-20T19:05:30.474517] [INFO ] [MAIN ] GTLS: Full history saved. [2025-10-20T19:05:30.474592] [INFO ] [MAIN ] Update Progress: 976/2367 symbols completed. GTLS took 21.1s. Estimated remaining: 2h 58m 46s. Rate: 7.8 symbols/min [2025-10-20T19:05:30.666686] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-08. [2025-10-20T19:05:30.669589] [INFO ] [MAIN ] GUG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:30.684051] [DEBUG] [LOCAL ] GUG: Loaded 12286 existing records from primary storage. [2025-10-20T19:05:30.684361] [INFO ] [MAIN ] GUG: Loaded 12286 existing records. [2025-10-20T19:05:30.897052] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:05:30.903280] [WARN ] [MAIN ] GUG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:30.913930] [INFO ] [MAIN ] GUG: Wiping existing primary data before full refresh. [2025-10-20T19:05:30.914370] [INFO ] [LOCAL ] GUG: Starting deletion of primary data. [2025-10-20T19:05:30.914683] [DEBUG] [LOCAL ] GUG: Deleted primary data file. [2025-10-20T19:05:30.914745] [INFO ] [MAIN ] GUG: Starting full primary refresh. [2025-10-20T19:05:31.291202] [INFO ] [REMOTE ] GUG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:31.308625] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:05:31.451385] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-01. [2025-10-20T19:05:32.251363] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-02. [2025-10-20T19:05:32.525686] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T19:05:32.555798] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:05:33.051364] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-02. [2025-10-20T19:05:33.431438] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T19:05:33.851387] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-04. [2025-10-20T19:05:33.973416] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T19:05:34.651426] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-03. [2025-10-20T19:05:35.451394] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-09. [2025-10-20T19:05:35.560698] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:05:36.027988] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:05:36.251391] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-03. [2025-10-20T19:05:37.051397] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-04. [2025-10-20T19:05:37.098976] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T19:05:37.104278] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:05:37.851366] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-04. [2025-10-20T19:05:38.409899] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:05:38.651315] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-05. [2025-10-20T19:05:38.752256] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T19:05:39.451420] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-05. [2025-10-20T19:05:39.977368] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T19:05:40.163407] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:05:40.251336] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-06. [2025-10-20T19:05:41.051372] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-06. [2025-10-20T19:05:41.106297] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T19:05:41.851363] [INFO ] [REMOTE ] GTES: Fetching primary data for 2025-10. [2025-10-20T19:05:42.284530] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T19:05:42.651380] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-07. [2025-10-20T19:05:43.451375] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-05. [2025-10-20T19:05:43.624723] [INFO ] [REMOTE ] GTES: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:05:43.632111] [INFO ] [REMOTE ] GTES: Full fetch complete (73921 records). [2025-10-20T19:05:43.632257] [DEBUG] [VALIDATOR ] GTES: Validating primary data. [2025-10-20T19:05:44.046326] [DEBUG] [VALIDATOR ] GTES: All expected trading days are present. [2025-10-20T19:05:44.046497] [DEBUG] [VALIDATOR ] GTES: Primary data validation passed. [2025-10-20T19:05:44.047761] [INFO ] [MAIN ] GTES: Fetched primary data is valid. [2025-10-20T19:05:44.047971] [DEBUG] [LOCAL ] GTES: Saving primary data. [2025-10-20T19:05:44.068538] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:05:44.076396] [DEBUG] [LOCAL ] GTES: Trimmed 73921 records to 50321 (keeping 200 days of data) [2025-10-20T19:05:44.276859] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-07. [2025-10-20T19:05:44.373000] [DEBUG] [LOCAL ] GTES: Primary data saved (50321 records). [2025-10-20T19:05:44.373134] [INFO ] [MAIN ] GTES: Full history saved. [2025-10-20T19:05:44.373854] [INFO ] [MAIN ] Update Progress: 977/2367 symbols completed. GTES took 37.7s. Estimated remaining: 2h 58m 47s. Rate: 7.8 symbols/min [2025-10-20T19:05:44.555887] [INFO ] [MAIN ] GUT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:44.572013] [DEBUG] [LOCAL ] GUT: Loaded 15859 existing records from primary storage. [2025-10-20T19:05:44.572245] [INFO ] [MAIN ] GUT: Loaded 15859 existing records. [2025-10-20T19:05:44.758299] [WARN ] [MAIN ] GUT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:44.758527] [INFO ] [MAIN ] GUT: Wiping existing primary data before full refresh. [2025-10-20T19:05:44.758573] [INFO ] [LOCAL ] GUT: Starting deletion of primary data. [2025-10-20T19:05:44.758844] [DEBUG] [LOCAL ] GUT: Deleted primary data file. [2025-10-20T19:05:44.758902] [INFO ] [MAIN ] GUT: Starting full primary refresh. [2025-10-20T19:05:45.040311] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:05:45.056748] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-08. [2025-10-20T19:05:45.161952] [INFO ] [REMOTE ] GUT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:45.344752] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:05:45.851436] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-06. [2025-10-20T19:05:46.497113] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:05:46.651368] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-08. [2025-10-20T19:05:47.183836] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:05:47.408552] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:05:47.451380] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-07. [2025-10-20T19:05:48.251366] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-01. [2025-10-20T19:05:48.897092] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T19:05:49.051382] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-09. [2025-10-20T19:05:49.260994] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T19:05:49.851370] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-08. [2025-10-20T19:05:50.291535] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:05:50.651366] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-09. [2025-10-20T19:05:51.439475] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T19:05:51.441063] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:05:51.453312] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-02. [2025-10-20T19:05:52.251392] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-09. [2025-10-20T19:05:52.538764] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:05:53.051423] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-03. [2025-10-20T19:05:53.729952] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:05:53.851364] [INFO ] [REMOTE ] GTY: Fetching primary data for 2025-10. [2025-10-20T19:05:53.997936] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:05:54.651379] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-04. [2025-10-20T19:05:55.314014] [INFO ] [REMOTE ] GTY: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:05:55.318876] [INFO ] [REMOTE ] GTY: Full fetch complete (43839 records). [2025-10-20T19:05:55.318988] [DEBUG] [VALIDATOR ] GTY: Validating primary data. [2025-10-20T19:05:55.459951] [INFO ] [REMOTE ] GUG: Fetching primary data for 2025-10. [2025-10-20T19:05:55.666463] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:05:55.718806] [DEBUG] [VALIDATOR ] GTY: All expected trading days are present. [2025-10-20T19:05:55.718872] [DEBUG] [VALIDATOR ] GTY: Primary data validation passed. [2025-10-20T19:05:55.719600] [INFO ] [MAIN ] GTY: Fetched primary data is valid. [2025-10-20T19:05:55.719684] [DEBUG] [LOCAL ] GTY: Saving primary data. [2025-10-20T19:05:55.720559] [DEBUG] [LOCAL ] GTY: Trimmed 43839 records to 30157 (keeping 200 days of data) [2025-10-20T19:05:55.898883] [DEBUG] [LOCAL ] GTY: Primary data saved (30157 records). [2025-10-20T19:05:55.899037] [INFO ] [MAIN ] GTY: Full history saved. [2025-10-20T19:05:55.899120] [INFO ] [MAIN ] Update Progress: 978/2367 symbols completed. GTY took 34.0s. Estimated remaining: 2h 58m 45s. Rate: 7.8 symbols/min [2025-10-20T19:05:56.067001] [INFO ] [MAIN ] GVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:56.100605] [DEBUG] [LOCAL ] GVA: Loaded 33145 existing records from primary storage. [2025-10-20T19:05:56.100890] [INFO ] [MAIN ] GVA: Loaded 33145 existing records. [2025-10-20T19:05:56.260361] [INFO ] [REMOTE ] GTN: Fetching primary data for 2025-10. [2025-10-20T19:05:56.280671] [WARN ] [MAIN ] GVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:56.281224] [INFO ] [MAIN ] GVA: Wiping existing primary data before full refresh. [2025-10-20T19:05:56.281437] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-10-20T19:05:56.282341] [DEBUG] [LOCAL ] GVA: Deleted primary data file. [2025-10-20T19:05:56.282408] [INFO ] [MAIN ] GVA: Starting full primary refresh. [2025-10-20T19:05:56.399996] [INFO ] [REMOTE ] GUG: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:05:56.418333] [INFO ] [REMOTE ] GUG: Full fetch complete (18904 records). [2025-10-20T19:05:56.441741] [DEBUG] [VALIDATOR ] GUG: Validating primary data. [2025-10-20T19:05:56.929547] [INFO ] [REMOTE ] GVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:57.030537] [DEBUG] [VALIDATOR ] GUG: All expected trading days are present. [2025-10-20T19:05:57.030605] [DEBUG] [VALIDATOR ] GUG: Primary data validation passed. [2025-10-20T19:05:57.031003] [INFO ] [MAIN ] GUG: Fetched primary data is valid. [2025-10-20T19:05:57.031062] [DEBUG] [LOCAL ] GUG: Saving primary data. [2025-10-20T19:05:57.031566] [DEBUG] [LOCAL ] GUG: Trimmed 18904 records to 12194 (keeping 200 days of data) [2025-10-20T19:05:57.062918] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-01. [2025-10-20T19:05:57.108011] [DEBUG] [LOCAL ] GUG: Primary data saved (12194 records). [2025-10-20T19:05:57.108110] [INFO ] [MAIN ] GUG: Full history saved. [2025-10-20T19:05:57.108183] [INFO ] [MAIN ] Update Progress: 979/2367 symbols completed. GUG took 26.6s. Estimated remaining: 2h 58m 28s. Rate: 7.8 symbols/min [2025-10-20T19:05:57.277775] [INFO ] [MAIN ] GWH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:57.296000] [DEBUG] [LOCAL ] GWH: Loaded 19414 existing records from primary storage. [2025-10-20T19:05:57.296231] [INFO ] [MAIN ] GWH: Loaded 19414 existing records. [2025-10-20T19:05:57.478776] [WARN ] [MAIN ] GWH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:57.479020] [INFO ] [MAIN ] GWH: Wiping existing primary data before full refresh. [2025-10-20T19:05:57.479077] [INFO ] [LOCAL ] GWH: Starting deletion of primary data. [2025-10-20T19:05:57.479312] [DEBUG] [LOCAL ] GWH: Deleted primary data file. [2025-10-20T19:05:57.479351] [INFO ] [MAIN ] GWH: Starting full primary refresh. [2025-10-20T19:05:57.847157] [INFO ] [REMOTE ] GWH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:57.851321] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-01. [2025-10-20T19:05:57.940154] [INFO ] [REMOTE ] GTN: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:05:57.947688] [INFO ] [REMOTE ] GTN: Full fetch complete (65985 records). [2025-10-20T19:05:57.947945] [DEBUG] [VALIDATOR ] GTN: Validating primary data. [2025-10-20T19:05:58.349578] [DEBUG] [VALIDATOR ] GTN: All expected trading days are present. [2025-10-20T19:05:58.349745] [DEBUG] [VALIDATOR ] GTN: Primary data validation passed. [2025-10-20T19:05:58.350748] [INFO ] [MAIN ] GTN: Fetched primary data is valid. [2025-10-20T19:05:58.350836] [DEBUG] [LOCAL ] GTN: Saving primary data. [2025-10-20T19:05:58.352466] [DEBUG] [LOCAL ] GTN: Trimmed 65985 records to 44989 (keeping 200 days of data) [2025-10-20T19:05:58.481675] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:05:58.645729] [DEBUG] [LOCAL ] GTN: Primary data saved (44989 records). [2025-10-20T19:05:58.645879] [INFO ] [MAIN ] GTN: Full history saved. [2025-10-20T19:05:58.645967] [INFO ] [MAIN ] Update Progress: 980/2367 symbols completed. GTN took 37.8s. Estimated remaining: 2h 58m 12s. Rate: 7.8 symbols/min [2025-10-20T19:05:58.657349] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-05. [2025-10-20T19:05:58.772097] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T19:05:58.857304] [INFO ] [MAIN ] GWRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:05:58.901275] [DEBUG] [LOCAL ] GWRE: Loaded 40437 existing records from primary storage. [2025-10-20T19:05:58.901748] [INFO ] [MAIN ] GWRE: Loaded 40437 existing records. [2025-10-20T19:05:59.102799] [WARN ] [MAIN ] GWRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:05:59.103051] [INFO ] [MAIN ] GWRE: Wiping existing primary data before full refresh. [2025-10-20T19:05:59.103102] [INFO ] [LOCAL ] GWRE: Starting deletion of primary data. [2025-10-20T19:05:59.103302] [DEBUG] [LOCAL ] GWRE: Deleted primary data file. [2025-10-20T19:05:59.103340] [INFO ] [MAIN ] GWRE: Starting full primary refresh. [2025-10-20T19:05:59.453215] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-02. [2025-10-20T19:05:59.453115] [INFO ] [REMOTE ] GWRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:05:59.472361] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:06:00.251381] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-06. [2025-10-20T19:06:00.333304] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T19:06:01.051360] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-03. [2025-10-20T19:06:01.395054] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:06:01.851335] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-07. [2025-10-20T19:06:02.166460] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:06:02.651368] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-04. [2025-10-20T19:06:03.032868] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T19:06:03.451419] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-01. [2025-10-20T19:06:03.537253] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T19:06:04.251419] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-05. [2025-10-20T19:06:05.051346] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-02. [2025-10-20T19:06:05.208942] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:06:05.293703] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T19:06:05.851363] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-08. [2025-10-20T19:06:06.368476] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T19:06:06.651377] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-06. [2025-10-20T19:06:06.899222] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:06:07.451362] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-02. [2025-10-20T19:06:07.902195] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:06:08.251351] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-09. [2025-10-20T19:06:08.738541] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:06:09.051377] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-03. [2025-10-20T19:06:09.157450] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T19:06:09.851372] [INFO ] [REMOTE ] GUT: Fetching primary data for 2025-10. [2025-10-20T19:06:10.651365] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-03. [2025-10-20T19:06:10.846838] [INFO ] [REMOTE ] GUT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:06:10.846523] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T19:06:10.859862] [INFO ] [REMOTE ] GUT: Full fetch complete (22524 records). [2025-10-20T19:06:10.860009] [DEBUG] [VALIDATOR ] GUT: Validating primary data. [2025-10-20T19:06:11.234618] [DEBUG] [VALIDATOR ] GUT: All expected trading days are present. [2025-10-20T19:06:11.234757] [DEBUG] [VALIDATOR ] GUT: Primary data validation passed. [2025-10-20T19:06:11.235291] [INFO ] [MAIN ] GUT: Fetched primary data is valid. [2025-10-20T19:06:11.235334] [DEBUG] [LOCAL ] GUT: Saving primary data. [2025-10-20T19:06:11.235964] [DEBUG] [LOCAL ] GUT: Trimmed 22524 records to 15737 (keeping 200 days of data) [2025-10-20T19:06:11.326742] [DEBUG] [LOCAL ] GUT: Primary data saved (15737 records). [2025-10-20T19:06:11.326880] [INFO ] [MAIN ] GUT: Full history saved. [2025-10-20T19:06:11.326958] [INFO ] [MAIN ] Update Progress: 981/2367 symbols completed. GUT took 27.0s. Estimated remaining: 2h 58m 11s. Rate: 7.8 symbols/min [2025-10-20T19:06:11.456541] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-04. [2025-10-20T19:06:11.511008] [INFO ] [MAIN ] GWW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:11.532482] [DEBUG] [LOCAL ] GWW: Loaded 19822 existing records from primary storage. [2025-10-20T19:06:11.532862] [INFO ] [MAIN ] GWW: Loaded 19822 existing records. [2025-10-20T19:06:11.732351] [WARN ] [MAIN ] GWW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:11.732597] [INFO ] [MAIN ] GWW: Wiping existing primary data before full refresh. [2025-10-20T19:06:11.732667] [INFO ] [LOCAL ] GWW: Starting deletion of primary data. [2025-10-20T19:06:11.732904] [DEBUG] [LOCAL ] GWW: Deleted primary data file. [2025-10-20T19:06:11.732963] [INFO ] [MAIN ] GWW: Starting full primary refresh. [2025-10-20T19:06:12.129838] [INFO ] [REMOTE ] GWW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:12.251389] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-01. [2025-10-20T19:06:12.284846] [INFO ] [REMOTE ] GVA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:06:12.768800] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:06:13.051638] [INFO ] [REMOTE ] GVA: Fetching primary data for 2025-04. [2025-10-20T19:06:13.155252] [DEBUG] [REMOTE ] GVA: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T19:06:13.155572] [WARN ] [REMOTE ] GVA: No data returned for month 2025-04 [2025-10-20T19:06:13.155700] [FATAL] [REMOTE ] GVA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:06:13.155785] [FATAL] [MAIN ] GVA: Failed to fetch primary data. [2025-10-20T19:06:13.155850] [WARN ] [MAIN ] GVA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:06:13.155924] [INFO ] [LOCAL ] GVA: Starting deletion of primary data. [2025-10-20T19:06:13.155987] [DEBUG] [LOCAL ] GVA: No primary data file found to delete. [2025-10-20T19:06:13.156023] [INFO ] [LOCAL ] GVA: Starting deletion of derivative data. [2025-10-20T19:06:13.156223] [DEBUG] [LOCAL ] GVA: Deleted derivative data file. [2025-10-20T19:06:13.156305] [INFO ] [MAIN ] Update Progress: 982/2367 symbols completed. GVA took 17.3s. Estimated remaining: 2h 57m 55s. Rate: 7.8 symbols/min [2025-10-20T19:06:13.334036] [INFO ] [MAIN ] GXO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:13.380811] [DEBUG] [LOCAL ] GXO: Loaded 47418 existing records from primary storage. [2025-10-20T19:06:13.381152] [INFO ] [MAIN ] GXO: Loaded 47418 existing records. [2025-10-20T19:06:13.579389] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:06:13.584902] [WARN ] [MAIN ] GXO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:13.585202] [INFO ] [MAIN ] GXO: Wiping existing primary data before full refresh. [2025-10-20T19:06:13.589991] [INFO ] [LOCAL ] GXO: Starting deletion of primary data. [2025-10-20T19:06:13.590214] [DEBUG] [LOCAL ] GXO: Deleted primary data file. [2025-10-20T19:06:13.590259] [INFO ] [MAIN ] GXO: Starting full primary refresh. [2025-10-20T19:06:13.861186] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-02. [2025-10-20T19:06:13.994203] [INFO ] [REMOTE ] GXO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:14.651402] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-05. [2025-10-20T19:06:15.101859] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T19:06:15.451388] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-03. [2025-10-20T19:06:16.251468] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-07. [2025-10-20T19:06:16.455995] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:06:16.667271] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:06:17.051401] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-04. [2025-10-20T19:06:17.170670] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:06:17.851376] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-06. [2025-10-20T19:06:18.338105] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T19:06:18.651382] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-08. [2025-10-20T19:06:19.337708] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:06:19.451379] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-05. [2025-10-20T19:06:19.696041] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:06:20.251389] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-07. [2025-10-20T19:06:20.453484] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:06:21.051474] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-01. [2025-10-20T19:06:21.840776] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T19:06:21.851343] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-09. [2025-10-20T19:06:22.572186] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T19:06:22.651343] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-02. [2025-10-20T19:06:22.990830] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T19:06:23.451371] [INFO ] [REMOTE ] GWH: Fetching primary data for 2025-10. [2025-10-20T19:06:24.256469] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-06. [2025-10-20T19:06:24.271570] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T19:06:25.051362] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-03. [2025-10-20T19:06:25.100379] [INFO ] [REMOTE ] GWH: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T19:06:25.106548] [INFO ] [REMOTE ] GWH: Full fetch complete (24087 records). [2025-10-20T19:06:25.106702] [DEBUG] [VALIDATOR ] GWH: Validating primary data. [2025-10-20T19:06:25.410859] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:06:25.497964] [DEBUG] [VALIDATOR ] GWH: All expected trading days are present. [2025-10-20T19:06:25.498048] [DEBUG] [VALIDATOR ] GWH: Primary data validation passed. [2025-10-20T19:06:25.498565] [INFO ] [MAIN ] GWH: Fetched primary data is valid. [2025-10-20T19:06:25.498604] [DEBUG] [LOCAL ] GWH: Saving primary data. [2025-10-20T19:06:25.499348] [DEBUG] [LOCAL ] GWH: Trimmed 24087 records to 19537 (keeping 200 days of data) [2025-10-20T19:06:25.608193] [DEBUG] [LOCAL ] GWH: Primary data saved (19537 records). [2025-10-20T19:06:25.608344] [INFO ] [MAIN ] GWH: Full history saved. [2025-10-20T19:06:25.608426] [INFO ] [MAIN ] Update Progress: 983/2367 symbols completed. GWH took 28.5s. Estimated remaining: 2h 57m 54s. Rate: 7.8 symbols/min [2025-10-20T19:06:25.776396] [INFO ] [MAIN ] GYLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:25.780312] [DEBUG] [LOCAL ] GYLD: Loaded 1562 existing records from primary storage. [2025-10-20T19:06:25.780442] [INFO ] [MAIN ] GYLD: Loaded 1562 existing records. [2025-10-20T19:06:25.856518] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-07. [2025-10-20T19:06:25.956490] [WARN ] [MAIN ] GYLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:25.956682] [INFO ] [MAIN ] GYLD: Wiping existing primary data before full refresh. [2025-10-20T19:06:25.956738] [INFO ] [LOCAL ] GYLD: Starting deletion of primary data. [2025-10-20T19:06:25.956908] [DEBUG] [LOCAL ] GYLD: Deleted primary data file. [2025-10-20T19:06:25.956965] [INFO ] [MAIN ] GYLD: Starting full primary refresh. [2025-10-20T19:06:26.328921] [INFO ] [REMOTE ] GYLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:26.651408] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-08. [2025-10-20T19:06:26.779605] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:06:26.825209] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:06:27.451382] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-01. [2025-10-20T19:06:27.818474] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-01 (0.37 seconds). [2025-10-20T19:06:28.130526] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:06:28.251377] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-04. [2025-10-20T19:06:29.051400] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-08. [2025-10-20T19:06:29.851416] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-09. [2025-10-20T19:06:29.919606] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:06:30.170967] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:06:30.651401] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-02. [2025-10-20T19:06:31.026091] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T19:06:31.161040] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T19:06:31.451383] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-03. [2025-10-20T19:06:31.887337] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-10-20T19:06:32.251394] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-04. [2025-10-20T19:06:32.687505] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T19:06:33.051376] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-05. [2025-10-20T19:06:33.851369] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-05. [2025-10-20T19:06:34.300845] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T19:06:34.657312] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-06. [2025-10-20T19:06:34.677195] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:06:35.114731] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T19:06:35.451351] [INFO ] [REMOTE ] GWRE: Fetching primary data for 2025-10. [2025-10-20T19:06:36.251357] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-09. [2025-10-20T19:06:36.942606] [INFO ] [REMOTE ] GWRE: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:06:36.948181] [INFO ] [REMOTE ] GWRE: Full fetch complete (60257 records). [2025-10-20T19:06:36.948304] [DEBUG] [VALIDATOR ] GWRE: Validating primary data. [2025-10-20T19:06:37.056503] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-06. [2025-10-20T19:06:37.336786] [DEBUG] [VALIDATOR ] GWRE: All expected trading days are present. [2025-10-20T19:06:37.336903] [DEBUG] [VALIDATOR ] GWRE: Primary data validation passed. [2025-10-20T19:06:37.337787] [INFO ] [MAIN ] GWRE: Fetched primary data is valid. [2025-10-20T19:06:37.337954] [DEBUG] [LOCAL ] GWRE: Saving primary data. [2025-10-20T19:06:37.339389] [DEBUG] [LOCAL ] GWRE: Trimmed 60257 records to 39801 (keeping 200 days of data) [2025-10-20T19:06:37.454341] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T19:06:37.601345] [DEBUG] [LOCAL ] GWRE: Primary data saved (39801 records). [2025-10-20T19:06:37.601492] [INFO ] [MAIN ] GWRE: Full history saved. [2025-10-20T19:06:37.601570] [INFO ] [MAIN ] Update Progress: 984/2367 symbols completed. GWRE took 39.0s. Estimated remaining: 2h 57m 52s. Rate: 7.8 symbols/min [2025-10-20T19:06:37.767682] [INFO ] [MAIN ] H: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:37.811183] [DEBUG] [LOCAL ] H: Loaded 45047 existing records from primary storage. [2025-10-20T19:06:37.811471] [INFO ] [MAIN ] H: Loaded 45047 existing records. [2025-10-20T19:06:37.856687] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-07. [2025-10-20T19:06:37.990085] [WARN ] [MAIN ] H: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:37.990299] [INFO ] [MAIN ] H: Wiping existing primary data before full refresh. [2025-10-20T19:06:37.990346] [INFO ] [LOCAL ] H: Starting deletion of primary data. [2025-10-20T19:06:37.990560] [DEBUG] [LOCAL ] H: Deleted primary data file. [2025-10-20T19:06:37.990614] [INFO ] [MAIN ] H: Starting full primary refresh. [2025-10-20T19:06:38.395942] [INFO ] [REMOTE ] H: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:38.427615] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T19:06:38.442289] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:06:38.651443] [INFO ] [REMOTE ] H: Fetching primary data for 2025-01. [2025-10-20T19:06:39.451416] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-07. [2025-10-20T19:06:40.142446] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:06:40.251363] [INFO ] [REMOTE ] H: Fetching primary data for 2025-02. [2025-10-20T19:06:40.584625] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:06:41.051362] [INFO ] [REMOTE ] GWW: Fetching primary data for 2025-10. [2025-10-20T19:06:41.821611] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:06:41.851328] [INFO ] [REMOTE ] H: Fetching primary data for 2025-03. [2025-10-20T19:06:42.116065] [INFO ] [REMOTE ] GWW: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:06:42.119472] [INFO ] [REMOTE ] GWW: Full fetch complete (29083 records). [2025-10-20T19:06:42.119553] [DEBUG] [VALIDATOR ] GWW: Validating primary data. [2025-10-20T19:06:42.535245] [DEBUG] [VALIDATOR ] GWW: All expected trading days are present. [2025-10-20T19:06:42.535334] [DEBUG] [VALIDATOR ] GWW: Primary data validation passed. [2025-10-20T19:06:42.535913] [INFO ] [MAIN ] GWW: Fetched primary data is valid. [2025-10-20T19:06:42.535975] [DEBUG] [LOCAL ] GWW: Saving primary data. [2025-10-20T19:06:42.536842] [DEBUG] [LOCAL ] GWW: Trimmed 29083 records to 19418 (keeping 200 days of data) [2025-10-20T19:06:42.655606] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-08. [2025-10-20T19:06:42.658700] [DEBUG] [LOCAL ] GWW: Primary data saved (19418 records). [2025-10-20T19:06:42.659693] [INFO ] [MAIN ] GWW: Full history saved. [2025-10-20T19:06:42.659802] [INFO ] [MAIN ] Update Progress: 985/2367 symbols completed. GWW took 31.3s. Estimated remaining: 2h 57m 41s. Rate: 7.8 symbols/min [2025-10-20T19:06:42.836544] [INFO ] [MAIN ] HAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:42.874929] [DEBUG] [LOCAL ] HAE: Loaded 40001 existing records from primary storage. [2025-10-20T19:06:42.875207] [INFO ] [MAIN ] HAE: Loaded 40001 existing records. [2025-10-20T19:06:43.056279] [WARN ] [MAIN ] HAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:43.056458] [INFO ] [MAIN ] HAE: Wiping existing primary data before full refresh. [2025-10-20T19:06:43.056505] [INFO ] [LOCAL ] HAE: Starting deletion of primary data. [2025-10-20T19:06:43.056733] [DEBUG] [LOCAL ] HAE: Deleted primary data file. [2025-10-20T19:06:43.056800] [INFO ] [MAIN ] HAE: Starting full primary refresh. [2025-10-20T19:06:43.222343] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T19:06:43.426536] [INFO ] [REMOTE ] HAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:43.451331] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-01. [2025-10-20T19:06:43.555450] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:06:44.251391] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-09. [2025-10-20T19:06:44.722745] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T19:06:44.860719] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:06:45.051414] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-08. [2025-10-20T19:06:45.851368] [INFO ] [REMOTE ] H: Fetching primary data for 2025-04. [2025-10-20T19:06:46.651364] [INFO ] [REMOTE ] GYLD: Fetching primary data for 2025-10. [2025-10-20T19:06:46.678033] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:06:47.451422] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-09. [2025-10-20T19:06:47.508388] [INFO ] [REMOTE ] GYLD: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:06:47.509247] [INFO ] [REMOTE ] GYLD: Full fetch complete (2078 records). [2025-10-20T19:06:47.509315] [DEBUG] [VALIDATOR ] GYLD: Validating primary data. [2025-10-20T19:06:47.820215] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:06:47.911815] [DEBUG] [VALIDATOR ] GYLD: All expected trading days are present. [2025-10-20T19:06:47.911895] [DEBUG] [VALIDATOR ] GYLD: Primary data validation passed. [2025-10-20T19:06:47.912005] [INFO ] [MAIN ] GYLD: Fetched primary data is valid. [2025-10-20T19:06:47.912071] [DEBUG] [LOCAL ] GYLD: Saving primary data. [2025-10-20T19:06:47.912406] [DEBUG] [LOCAL ] GYLD: Trimmed 2078 records to 1544 (keeping 200 days of data) [2025-10-20T19:06:47.922863] [DEBUG] [LOCAL ] GYLD: Primary data saved (1544 records). [2025-10-20T19:06:47.922992] [INFO ] [MAIN ] GYLD: Full history saved. [2025-10-20T19:06:47.923073] [INFO ] [MAIN ] Update Progress: 986/2367 symbols completed. GYLD took 22.3s. Estimated remaining: 2h 57m 30s. Rate: 7.8 symbols/min [2025-10-20T19:06:48.088612] [INFO ] [MAIN ] HAFN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:48.125295] [DEBUG] [LOCAL ] HAFN: Loaded 43553 existing records from primary storage. [2025-10-20T19:06:48.125549] [INFO ] [MAIN ] HAFN: Loaded 43553 existing records. [2025-10-20T19:06:48.256623] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-02. [2025-10-20T19:06:48.311138] [WARN ] [MAIN ] HAFN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:48.311589] [INFO ] [MAIN ] HAFN: Wiping existing primary data before full refresh. [2025-10-20T19:06:48.311638] [INFO ] [LOCAL ] HAFN: Starting deletion of primary data. [2025-10-20T19:06:48.311922] [DEBUG] [LOCAL ] HAFN: Deleted primary data file. [2025-10-20T19:06:48.311964] [INFO ] [MAIN ] HAFN: Starting full primary refresh. [2025-10-20T19:06:48.660196] [INFO ] [REMOTE ] HAFN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:49.008054] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T19:06:49.051326] [INFO ] [REMOTE ] GXO: Fetching primary data for 2025-10. [2025-10-20T19:06:49.851369] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-01. [2025-10-20T19:06:49.973058] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:06:50.651404] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-03. [2025-10-20T19:06:50.709123] [INFO ] [REMOTE ] GXO: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T19:06:50.715533] [INFO ] [REMOTE ] GXO: Full fetch complete (69503 records). [2025-10-20T19:06:50.715628] [DEBUG] [VALIDATOR ] GXO: Validating primary data. [2025-10-20T19:06:51.096013] [DEBUG] [VALIDATOR ] GXO: All expected trading days are present. [2025-10-20T19:06:51.096112] [DEBUG] [VALIDATOR ] GXO: Primary data validation passed. [2025-10-20T19:06:51.097773] [INFO ] [MAIN ] GXO: Fetched primary data is valid. [2025-10-20T19:06:51.097844] [DEBUG] [LOCAL ] GXO: Saving primary data. [2025-10-20T19:06:51.099510] [DEBUG] [LOCAL ] GXO: Trimmed 69503 records to 46617 (keeping 200 days of data) [2025-10-20T19:06:51.368871] [DEBUG] [LOCAL ] GXO: Primary data saved (46617 records). [2025-10-20T19:06:51.369026] [INFO ] [MAIN ] GXO: Full history saved. [2025-10-20T19:06:51.369111] [INFO ] [MAIN ] Update Progress: 987/2367 symbols completed. GXO took 38.2s. Estimated remaining: 2h 57m 16s. Rate: 7.8 symbols/min [2025-10-20T19:06:51.467505] [INFO ] [REMOTE ] H: Fetching primary data for 2025-05. [2025-10-20T19:06:51.540379] [INFO ] [MAIN ] HAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:06:51.591497] [DEBUG] [LOCAL ] HAL: Loaded 54498 existing records from primary storage. [2025-10-20T19:06:51.591867] [INFO ] [MAIN ] HAL: Loaded 54498 existing records. [2025-10-20T19:06:51.796996] [WARN ] [MAIN ] HAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:06:51.797507] [INFO ] [MAIN ] HAL: Wiping existing primary data before full refresh. [2025-10-20T19:06:51.799600] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T19:06:51.799724] [INFO ] [LOCAL ] HAL: Starting deletion of primary data. [2025-10-20T19:06:51.809345] [DEBUG] [LOCAL ] HAL: Deleted primary data file. [2025-10-20T19:06:51.810125] [INFO ] [MAIN ] HAL: Starting full primary refresh. [2025-10-20T19:06:52.192050] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:06:52.196516] [INFO ] [REMOTE ] HAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:06:52.251340] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-04. [2025-10-20T19:06:52.773099] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:06:53.051355] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-01. [2025-10-20T19:06:53.851356] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-02. [2025-10-20T19:06:53.980798] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:06:54.651419] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-05. [2025-10-20T19:06:55.018562] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:06:55.451347] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-02. [2025-10-20T19:06:55.669013] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:06:55.748883] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:06:56.251359] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-03. [2025-10-20T19:06:56.997323] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T19:06:57.051354] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-03. [2025-10-20T19:06:57.851349] [INFO ] [REMOTE ] H: Fetching primary data for 2025-06. [2025-10-20T19:06:58.021207] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:06:58.651359] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-06. [2025-10-20T19:06:59.078152] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:06:59.120715] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T19:06:59.451362] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-04. [2025-10-20T19:06:59.821793] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:07:00.251401] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-04. [2025-10-20T19:07:01.051445] [INFO ] [REMOTE ] H: Fetching primary data for 2025-07. [2025-10-20T19:07:01.482243] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T19:07:01.851402] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-05. [2025-10-20T19:07:02.297402] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T19:07:02.376924] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T19:07:02.651363] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-05. [2025-10-20T19:07:03.451385] [INFO ] [REMOTE ] H: Fetching primary data for 2025-08. [2025-10-20T19:07:03.568788] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:07:04.251356] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-06. [2025-10-20T19:07:04.718171] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T19:07:05.051428] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-06. [2025-10-20T19:07:05.197507] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:07:05.786133] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:07:05.851396] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-07. [2025-10-20T19:07:06.651367] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-07. [2025-10-20T19:07:07.016927] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T19:07:07.132469] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:07:07.451384] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-07. [2025-10-20T19:07:08.032998] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:07:08.251366] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-08. [2025-10-20T19:07:09.051388] [INFO ] [REMOTE ] H: Fetching primary data for 2025-09. [2025-10-20T19:07:09.486275] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:07:09.857005] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-08. [2025-10-20T19:07:09.867870] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T19:07:10.655728] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-09. [2025-10-20T19:07:10.657385] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:07:11.429416] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:07:11.451345] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-09. [2025-10-20T19:07:12.251383] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-08. [2025-10-20T19:07:12.305191] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:07:13.051463] [INFO ] [REMOTE ] H: Fetching primary data for 2025-10. [2025-10-20T19:07:13.134267] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:07:13.851497] [INFO ] [REMOTE ] HAE: Fetching primary data for 2025-10. [2025-10-20T19:07:14.062495] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:07:14.651376] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-09. [2025-10-20T19:07:14.777179] [INFO ] [REMOTE ] H: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T19:07:14.784032] [INFO ] [REMOTE ] H: Full fetch complete (63179 records). [2025-10-20T19:07:14.784221] [DEBUG] [VALIDATOR ] H: Validating primary data. [2025-10-20T19:07:15.176708] [DEBUG] [VALIDATOR ] H: All expected trading days are present. [2025-10-20T19:07:15.176896] [DEBUG] [VALIDATOR ] H: Primary data validation passed. [2025-10-20T19:07:15.177787] [INFO ] [MAIN ] H: Fetched primary data is valid. [2025-10-20T19:07:15.177851] [DEBUG] [LOCAL ] H: Saving primary data. [2025-10-20T19:07:15.179224] [DEBUG] [LOCAL ] H: Trimmed 63179 records to 44459 (keeping 200 days of data) [2025-10-20T19:07:15.451882] [INFO ] [REMOTE ] HAFN: Fetching primary data for 2025-10. [2025-10-20T19:07:15.480851] [INFO ] [REMOTE ] HAE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:07:15.491263] [DEBUG] [LOCAL ] H: Primary data saved (44459 records). [2025-10-20T19:07:15.491427] [INFO ] [MAIN ] H: Full history saved. [2025-10-20T19:07:15.491982] [INFO ] [MAIN ] Update Progress: 988/2367 symbols completed. H took 37.9s. Estimated remaining: 2h 57m 31s. Rate: 7.8 symbols/min [2025-10-20T19:07:15.492146] [INFO ] [REMOTE ] HAE: Full fetch complete (59160 records). [2025-10-20T19:07:15.539784] [DEBUG] [VALIDATOR ] HAE: Validating primary data. [2025-10-20T19:07:15.817339] [INFO ] [MAIN ] HASI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:16.158722] [DEBUG] [VALIDATOR ] HAE: All expected trading days are present. [2025-10-20T19:07:16.160945] [DEBUG] [VALIDATOR ] HAE: Primary data validation passed. [2025-10-20T19:07:16.162216] [INFO ] [MAIN ] HAE: Fetched primary data is valid. [2025-10-20T19:07:16.162318] [DEBUG] [LOCAL ] HAE: Saving primary data. [2025-10-20T19:07:16.164492] [DEBUG] [LOCAL ] HASI: Loaded 48399 existing records from primary storage. [2025-10-20T19:07:16.165554] [INFO ] [MAIN ] HASI: Loaded 48399 existing records. [2025-10-20T19:07:16.199278] [DEBUG] [LOCAL ] HAE: Trimmed 59160 records to 39456 (keeping 200 days of data) [2025-10-20T19:07:16.600995] [WARN ] [MAIN ] HASI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:16.608699] [INFO ] [MAIN ] HASI: Wiping existing primary data before full refresh. [2025-10-20T19:07:16.612560] [INFO ] [LOCAL ] HASI: Starting deletion of primary data. [2025-10-20T19:07:16.613212] [DEBUG] [LOCAL ] HASI: Deleted primary data file. [2025-10-20T19:07:16.617165] [INFO ] [MAIN ] HASI: Starting full primary refresh. [2025-10-20T19:07:16.666588] [DEBUG] [LOCAL ] HAE: Primary data saved (39456 records). [2025-10-20T19:07:16.666766] [INFO ] [MAIN ] HAE: Full history saved. [2025-10-20T19:07:16.666982] [INFO ] [MAIN ] Update Progress: 989/2367 symbols completed. HAE took 34.0s. Estimated remaining: 2h 57m 14s. Rate: 7.8 symbols/min [2025-10-20T19:07:17.097585] [INFO ] [MAIN ] HAYW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:17.100934] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T19:07:17.176395] [INFO ] [REMOTE ] HAL: Fetching primary data for 2025-10. [2025-10-20T19:07:17.282101] [INFO ] [REMOTE ] HASI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:17.284713] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-01. [2025-10-20T19:07:17.284327] [INFO ] [REMOTE ] HAFN: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:07:17.292498] [INFO ] [REMOTE ] HAFN: Full fetch complete (64787 records). [2025-10-20T19:07:17.292704] [DEBUG] [VALIDATOR ] HAFN: Validating primary data. [2025-10-20T19:07:17.416106] [DEBUG] [LOCAL ] HAYW: Loaded 49622 existing records from primary storage. [2025-10-20T19:07:17.427808] [INFO ] [MAIN ] HAYW: Loaded 49622 existing records. [2025-10-20T19:07:17.829061] [WARN ] [MAIN ] HAYW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:17.858279] [INFO ] [MAIN ] HAYW: Wiping existing primary data before full refresh. [2025-10-20T19:07:17.858407] [INFO ] [LOCAL ] HAYW: Starting deletion of primary data. [2025-10-20T19:07:17.869249] [DEBUG] [LOCAL ] HAYW: Deleted primary data file. [2025-10-20T19:07:17.886094] [INFO ] [MAIN ] HAYW: Starting full primary refresh. [2025-10-20T19:07:17.947382] [DEBUG] [VALIDATOR ] HAFN: All expected trading days are present. [2025-10-20T19:07:17.968729] [DEBUG] [VALIDATOR ] HAFN: Primary data validation passed. [2025-10-20T19:07:17.981254] [INFO ] [MAIN ] HAFN: Fetched primary data is valid. [2025-10-20T19:07:18.004248] [DEBUG] [LOCAL ] HAFN: Saving primary data. [2025-10-20T19:07:18.058443] [DEBUG] [LOCAL ] HAFN: Trimmed 64787 records to 42843 (keeping 200 days of data) [2025-10-20T19:07:18.412239] [INFO ] [REMOTE ] HAYW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:18.422270] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-01. [2025-10-20T19:07:18.428289] [INFO ] [REMOTE ] HAL: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T19:07:18.551455] [INFO ] [REMOTE ] HAL: Full fetch complete (77897 records). [2025-10-20T19:07:18.580779] [DEBUG] [VALIDATOR ] HAL: Validating primary data. [2025-10-20T19:07:18.585533] [DEBUG] [LOCAL ] HAFN: Primary data saved (42843 records). [2025-10-20T19:07:18.585836] [INFO ] [MAIN ] HAFN: Full history saved. [2025-10-20T19:07:18.585940] [INFO ] [MAIN ] Update Progress: 990/2367 symbols completed. HAFN took 30.7s. Estimated remaining: 2h 56m 58s. Rate: 7.8 symbols/min [2025-10-20T19:07:19.013552] [INFO ] [MAIN ] HBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:19.146520] [DEBUG] [LOCAL ] HBB: Loaded 6540 existing records from primary storage. [2025-10-20T19:07:19.185724] [INFO ] [MAIN ] HBB: Loaded 6540 existing records. [2025-10-20T19:07:19.239631] [DEBUG] [VALIDATOR ] HAL: All expected trading days are present. [2025-10-20T19:07:19.244989] [DEBUG] [VALIDATOR ] HAL: Primary data validation passed. [2025-10-20T19:07:19.273395] [INFO ] [MAIN ] HAL: Fetched primary data is valid. [2025-10-20T19:07:19.305173] [DEBUG] [LOCAL ] HAL: Saving primary data. [2025-10-20T19:07:19.366125] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T19:07:19.371849] [DEBUG] [LOCAL ] HAL: Trimmed 77897 records to 53718 (keeping 200 days of data) [2025-10-20T19:07:19.435428] [WARN ] [MAIN ] HBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:19.435885] [INFO ] [MAIN ] HBB: Wiping existing primary data before full refresh. [2025-10-20T19:07:19.435969] [INFO ] [LOCAL ] HBB: Starting deletion of primary data. [2025-10-20T19:07:19.431516] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-02. [2025-10-20T19:07:19.437373] [DEBUG] [LOCAL ] HBB: Deleted primary data file. [2025-10-20T19:07:19.489577] [INFO ] [MAIN ] HBB: Starting full primary refresh. [2025-10-20T19:07:20.057148] [INFO ] [REMOTE ] HBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:20.061404] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-01. [2025-10-20T19:07:20.161025] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:07:20.191821] [DEBUG] [LOCAL ] HAL: Primary data saved (53718 records). [2025-10-20T19:07:20.191934] [INFO ] [MAIN ] HAL: Full history saved. [2025-10-20T19:07:20.192004] [INFO ] [MAIN ] Update Progress: 991/2367 symbols completed. HAL took 28.8s. Estimated remaining: 2h 56m 42s. Rate: 7.8 symbols/min [2025-10-20T19:07:20.261133] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-02. [2025-10-20T19:07:20.363939] [INFO ] [MAIN ] HBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:20.412590] [DEBUG] [LOCAL ] HBI: Loaded 53738 existing records from primary storage. [2025-10-20T19:07:20.412942] [INFO ] [MAIN ] HBI: Loaded 53738 existing records. [2025-10-20T19:07:20.587283] [WARN ] [MAIN ] HBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:20.587501] [INFO ] [MAIN ] HBI: Wiping existing primary data before full refresh. [2025-10-20T19:07:20.587547] [INFO ] [LOCAL ] HBI: Starting deletion of primary data. [2025-10-20T19:07:20.587758] [DEBUG] [LOCAL ] HBI: Deleted primary data file. [2025-10-20T19:07:20.587813] [INFO ] [MAIN ] HBI: Starting full primary refresh. [2025-10-20T19:07:20.910136] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:07:20.976534] [INFO ] [REMOTE ] HBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:21.051372] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-01. [2025-10-20T19:07:21.120439] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:07:21.800109] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:07:21.851373] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-03. [2025-10-20T19:07:22.651429] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-02. [2025-10-20T19:07:22.786377] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:07:22.970445] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:07:23.077950] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-10-20T19:07:23.451400] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-04. [2025-10-20T19:07:24.251440] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-03. [2025-10-20T19:07:25.051857] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-03. [2025-10-20T19:07:25.051752] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T19:07:25.056110] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:07:25.851431] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-02. [2025-10-20T19:07:26.651437] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-04. [2025-10-20T19:07:26.745810] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:07:27.451995] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-05. [2025-10-20T19:07:27.453513] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:07:27.498485] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:07:28.251420] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-04. [2025-10-20T19:07:28.979405] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:07:29.051335] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-06. [2025-10-20T19:07:29.851367] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-05. [2025-10-20T19:07:29.910435] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:07:30.650385] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:07:30.659160] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-03. [2025-10-20T19:07:30.691424] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T19:07:31.451373] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-05. [2025-10-20T19:07:32.251371] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-06. [2025-10-20T19:07:32.394206] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:07:32.582817] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:07:32.982116] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:07:33.051364] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-06. [2025-10-20T19:07:33.851394] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-04. [2025-10-20T19:07:34.651391] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-07. [2025-10-20T19:07:34.715914] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:07:35.282387] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T19:07:35.451419] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-08. [2025-10-20T19:07:35.525765] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:07:36.175481] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T19:07:36.251346] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-05. [2025-10-20T19:07:37.051371] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-09. [2025-10-20T19:07:37.613765] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T19:07:37.851343] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-07. [2025-10-20T19:07:37.942603] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:07:38.651367] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-06. [2025-10-20T19:07:39.451436] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-07. [2025-10-20T19:07:39.549332] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:07:40.251350] [INFO ] [REMOTE ] HBB: Fetching primary data for 2025-10. [2025-10-20T19:07:40.353765] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:07:40.746203] [INFO ] [REMOTE ] HBB: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:07:40.747383] [INFO ] [REMOTE ] HBB: Full fetch complete (9281 records). [2025-10-20T19:07:40.747451] [DEBUG] [VALIDATOR ] HBB: Validating primary data. [2025-10-20T19:07:41.051556] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-08. [2025-10-20T19:07:41.130487] [DEBUG] [VALIDATOR ] HBB: All expected trading days are present. [2025-10-20T19:07:41.130635] [DEBUG] [VALIDATOR ] HBB: Primary data validation passed. [2025-10-20T19:07:41.131033] [INFO ] [MAIN ] HBB: Fetched primary data is valid. [2025-10-20T19:07:41.131117] [DEBUG] [LOCAL ] HBB: Saving primary data. [2025-10-20T19:07:41.131673] [DEBUG] [LOCAL ] HBB: Trimmed 9281 records to 6459 (keeping 200 days of data) [2025-10-20T19:07:41.177028] [DEBUG] [LOCAL ] HBB: Primary data saved (6459 records). [2025-10-20T19:07:41.191382] [INFO ] [MAIN ] HBB: Full history saved. [2025-10-20T19:07:41.202955] [INFO ] [MAIN ] Update Progress: 992/2367 symbols completed. HBB took 22.6s. Estimated remaining: 2h 56m 53s. Rate: 7.8 symbols/min [2025-10-20T19:07:41.274323] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T19:07:41.391836] [INFO ] [MAIN ] HBM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:41.439754] [DEBUG] [LOCAL ] HBM: Loaded 53738 existing records from primary storage. [2025-10-20T19:07:41.440122] [INFO ] [MAIN ] HBM: Loaded 53738 existing records. [2025-10-20T19:07:41.614536] [WARN ] [MAIN ] HBM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:41.614775] [INFO ] [MAIN ] HBM: Wiping existing primary data before full refresh. [2025-10-20T19:07:41.614825] [INFO ] [LOCAL ] HBM: Starting deletion of primary data. [2025-10-20T19:07:41.615121] [DEBUG] [LOCAL ] HBM: Deleted primary data file. [2025-10-20T19:07:41.615177] [INFO ] [MAIN ] HBM: Starting full primary refresh. [2025-10-20T19:07:41.856506] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-08. [2025-10-20T19:07:41.966606] [INFO ] [REMOTE ] HBM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:42.651377] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-07. [2025-10-20T19:07:42.766968] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:07:43.451429] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-01. [2025-10-20T19:07:43.662622] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:07:44.251346] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-09. [2025-10-20T19:07:44.335039] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T19:07:44.831505] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:07:45.051378] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-02. [2025-10-20T19:07:45.565605] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-09 (1.31 seconds). [2025-10-20T19:07:45.851384] [INFO ] [REMOTE ] HAYW: Fetching primary data for 2025-10. [2025-10-20T19:07:46.575534] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:07:46.651361] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-03. [2025-10-20T19:07:47.451395] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-08. [2025-10-20T19:07:47.877439] [INFO ] [REMOTE ] HAYW: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-10-20T19:07:47.884901] [INFO ] [REMOTE ] HAYW: Full fetch complete (70924 records). [2025-10-20T19:07:47.885036] [DEBUG] [VALIDATOR ] HAYW: Validating primary data. [2025-10-20T19:07:48.263823] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-09. [2025-10-20T19:07:48.289610] [DEBUG] [VALIDATOR ] HAYW: All expected trading days are present. [2025-10-20T19:07:48.289746] [DEBUG] [VALIDATOR ] HAYW: Primary data validation passed. [2025-10-20T19:07:48.290995] [INFO ] [MAIN ] HAYW: Fetched primary data is valid. [2025-10-20T19:07:48.291121] [DEBUG] [LOCAL ] HAYW: Saving primary data. [2025-10-20T19:07:48.292757] [DEBUG] [LOCAL ] HAYW: Trimmed 70924 records to 48833 (keeping 200 days of data) [2025-10-20T19:07:48.538809] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:07:48.627821] [DEBUG] [LOCAL ] HAYW: Primary data saved (48833 records). [2025-10-20T19:07:48.627965] [INFO ] [MAIN ] HAYW: Full history saved. [2025-10-20T19:07:48.628048] [INFO ] [MAIN ] Update Progress: 993/2367 symbols completed. HAYW took 32.0s. Estimated remaining: 2h 56m 45s. Rate: 7.8 symbols/min [2025-10-20T19:07:48.799225] [INFO ] [MAIN ] HCA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:48.842967] [DEBUG] [LOCAL ] HCA: Loaded 47654 existing records from primary storage. [2025-10-20T19:07:48.843289] [INFO ] [MAIN ] HCA: Loaded 47654 existing records. [2025-10-20T19:07:49.027856] [WARN ] [MAIN ] HCA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:49.028131] [INFO ] [MAIN ] HCA: Wiping existing primary data before full refresh. [2025-10-20T19:07:49.028277] [INFO ] [LOCAL ] HCA: Starting deletion of primary data. [2025-10-20T19:07:49.028560] [DEBUG] [LOCAL ] HCA: Deleted primary data file. [2025-10-20T19:07:49.028711] [INFO ] [MAIN ] HCA: Starting full primary refresh. [2025-10-20T19:07:49.054248] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-04. [2025-10-20T19:07:49.336760] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T19:07:49.456114] [INFO ] [REMOTE ] HCA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:49.785344] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:07:49.851334] [INFO ] [REMOTE ] HASI: Fetching primary data for 2025-10. [2025-10-20T19:07:50.651359] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-09. [2025-10-20T19:07:50.908757] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:07:51.451439] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-01. [2025-10-20T19:07:51.906508] [INFO ] [REMOTE ] HASI: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T19:07:51.913823] [INFO ] [REMOTE ] HASI: Full fetch complete (68394 records). [2025-10-20T19:07:51.913936] [DEBUG] [VALIDATOR ] HASI: Validating primary data. [2025-10-20T19:07:52.258841] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-05. [2025-10-20T19:07:52.315733] [DEBUG] [VALIDATOR ] HASI: All expected trading days are present. [2025-10-20T19:07:52.315826] [DEBUG] [VALIDATOR ] HASI: Primary data validation passed. [2025-10-20T19:07:52.316792] [INFO ] [MAIN ] HASI: Fetched primary data is valid. [2025-10-20T19:07:52.316857] [DEBUG] [LOCAL ] HASI: Saving primary data. [2025-10-20T19:07:52.318239] [DEBUG] [LOCAL ] HASI: Trimmed 68394 records to 47810 (keeping 200 days of data) [2025-10-20T19:07:52.662291] [DEBUG] [LOCAL ] HASI: Primary data saved (47810 records). [2025-10-20T19:07:52.664823] [INFO ] [MAIN ] HASI: Full history saved. [2025-10-20T19:07:52.664989] [INFO ] [MAIN ] Update Progress: 994/2367 symbols completed. HASI took 37.2s. Estimated remaining: 2h 56m 32s. Rate: 7.8 symbols/min [2025-10-20T19:07:52.664549] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T19:07:52.851893] [INFO ] [MAIN ] HCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:52.884843] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:07:52.932366] [DEBUG] [LOCAL ] HCC: Loaded 45791 existing records from primary storage. [2025-10-20T19:07:52.932684] [INFO ] [MAIN ] HCC: Loaded 45791 existing records. [2025-10-20T19:07:53.056525] [INFO ] [REMOTE ] HBI: Fetching primary data for 2025-10. [2025-10-20T19:07:53.134304] [WARN ] [MAIN ] HCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:53.134497] [INFO ] [MAIN ] HCC: Wiping existing primary data before full refresh. [2025-10-20T19:07:53.134545] [INFO ] [LOCAL ] HCC: Starting deletion of primary data. [2025-10-20T19:07:53.134766] [DEBUG] [LOCAL ] HCC: Deleted primary data file. [2025-10-20T19:07:53.134822] [INFO ] [MAIN ] HCC: Starting full primary refresh. [2025-10-20T19:07:53.491217] [INFO ] [REMOTE ] HCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:53.851418] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-02. [2025-10-20T19:07:54.141177] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:07:54.651344] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-06. [2025-10-20T19:07:54.902208] [INFO ] [REMOTE ] HBI: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:07:54.911520] [INFO ] [REMOTE ] HBI: Full fetch complete (77005 records). [2025-10-20T19:07:54.911717] [DEBUG] [VALIDATOR ] HBI: Validating primary data. [2025-10-20T19:07:55.314435] [DEBUG] [VALIDATOR ] HBI: All expected trading days are present. [2025-10-20T19:07:55.314525] [DEBUG] [VALIDATOR ] HBI: Primary data validation passed. [2025-10-20T19:07:55.315481] [INFO ] [MAIN ] HBI: Fetched primary data is valid. [2025-10-20T19:07:55.315545] [DEBUG] [LOCAL ] HBI: Saving primary data. [2025-10-20T19:07:55.317205] [DEBUG] [LOCAL ] HBI: Trimmed 77005 records to 52948 (keeping 200 days of data) [2025-10-20T19:07:55.477989] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-01. [2025-10-20T19:07:55.643180] [DEBUG] [LOCAL ] HBI: Primary data saved (52948 records). [2025-10-20T19:07:55.659800] [INFO ] [MAIN ] HBI: Full history saved. [2025-10-20T19:07:55.659962] [INFO ] [MAIN ] Update Progress: 995/2367 symbols completed. HBI took 35.5s. Estimated remaining: 2h 56m 18s. Rate: 7.8 symbols/min [2025-10-20T19:07:55.749724] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:07:55.867454] [INFO ] [MAIN ] HCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:07:55.886552] [DEBUG] [LOCAL ] HCI: Loaded 14354 existing records from primary storage. [2025-10-20T19:07:55.886856] [INFO ] [MAIN ] HCI: Loaded 14354 existing records. [2025-10-20T19:07:56.092888] [WARN ] [MAIN ] HCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:07:56.093099] [INFO ] [MAIN ] HCI: Wiping existing primary data before full refresh. [2025-10-20T19:07:56.093172] [INFO ] [LOCAL ] HCI: Starting deletion of primary data. [2025-10-20T19:07:56.093398] [DEBUG] [LOCAL ] HCI: Deleted primary data file. [2025-10-20T19:07:56.093455] [INFO ] [MAIN ] HCI: Starting full primary refresh. [2025-10-20T19:07:56.256494] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-03. [2025-10-20T19:07:56.497150] [INFO ] [REMOTE ] HCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:07:56.528099] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T19:07:57.039474] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:07:57.051341] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-07. [2025-10-20T19:07:57.851378] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-02. [2025-10-20T19:07:58.272357] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T19:07:58.651375] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-04. [2025-10-20T19:07:58.906934] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T19:07:59.121253] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:07:59.451378] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-08. [2025-10-20T19:08:00.251396] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-03. [2025-10-20T19:08:00.425445] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:08:01.051371] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-05. [2025-10-20T19:08:01.263069] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:08:01.338886] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T19:08:01.851357] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-01. [2025-10-20T19:08:02.308113] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T19:08:02.355300] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T19:08:02.651354] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-02. [2025-10-20T19:08:03.451404] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-04. [2025-10-20T19:08:03.485932] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:08:04.251441] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-09. [2025-10-20T19:08:05.051375] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-03. [2025-10-20T19:08:05.170027] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:08:05.851391] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-05. [2025-10-20T19:08:05.981016] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T19:08:06.223366] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:08:06.651445] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-06. [2025-10-20T19:08:07.052595] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T19:08:07.451442] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-06. [2025-10-20T19:08:08.251389] [INFO ] [REMOTE ] HBM: Fetching primary data for 2025-10. [2025-10-20T19:08:08.303898] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:08:08.912984] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:08:09.051365] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-07. [2025-10-20T19:08:09.851405] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-04. [2025-10-20T19:08:10.240411] [INFO ] [REMOTE ] HBM: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T19:08:10.248986] [INFO ] [REMOTE ] HBM: Full fetch complete (76730 records). [2025-10-20T19:08:10.249166] [DEBUG] [VALIDATOR ] HBM: Validating primary data. [2025-10-20T19:08:10.694464] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-07. [2025-10-20T19:08:10.703811] [DEBUG] [VALIDATOR ] HBM: All expected trading days are present. [2025-10-20T19:08:10.705301] [DEBUG] [VALIDATOR ] HBM: Primary data validation passed. [2025-10-20T19:08:10.706800] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T19:08:10.706346] [INFO ] [MAIN ] HBM: Fetched primary data is valid. [2025-10-20T19:08:10.707151] [DEBUG] [LOCAL ] HBM: Saving primary data. [2025-10-20T19:08:10.740229] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T19:08:10.738980] [DEBUG] [LOCAL ] HBM: Trimmed 76730 records to 52976 (keeping 200 days of data) [2025-10-20T19:08:11.053418] [DEBUG] [LOCAL ] HBM: Primary data saved (52976 records). [2025-10-20T19:08:11.053545] [INFO ] [MAIN ] HBM: Full history saved. [2025-10-20T19:08:11.053633] [INFO ] [MAIN ] Update Progress: 996/2367 symbols completed. HBM took 29.9s. Estimated remaining: 2h 56m 21s. Rate: 7.8 symbols/min [2025-10-20T19:08:11.219026] [INFO ] [MAIN ] HCXY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:11.219155] [DEBUG] [LOCAL ] HCXY: No existing data file found in primary storage. [2025-10-20T19:08:11.219192] [WARN ] [MAIN ] HCXY: No existing data found, performing full refresh instead. [2025-10-20T19:08:11.219305] [INFO ] [MAIN ] HCXY: Wiping existing primary data before full refresh. [2025-10-20T19:08:11.219345] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-10-20T19:08:11.219384] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-10-20T19:08:11.219415] [INFO ] [MAIN ] HCXY: Starting full primary refresh. [2025-10-20T19:08:11.458504] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-05. [2025-10-20T19:08:11.578255] [INFO ] [REMOTE ] HCXY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:12.251399] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-08. [2025-10-20T19:08:12.316721] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T19:08:12.488730] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:08:13.051379] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-08. [2025-10-20T19:08:13.759326] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:08:13.851382] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-09. [2025-10-20T19:08:14.638248] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:08:14.651349] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-01. [2025-10-20T19:08:15.318725] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T19:08:15.350285] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T19:08:15.451376] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-06. [2025-10-20T19:08:16.251375] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-09. [2025-10-20T19:08:16.421944] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T19:08:17.051385] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-07. [2025-10-20T19:08:17.837556] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:08:17.851557] [INFO ] [REMOTE ] HCA: Fetching primary data for 2025-10. [2025-10-20T19:08:18.017929] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:08:18.651370] [INFO ] [REMOTE ] HCC: Fetching primary data for 2025-10. [2025-10-20T19:08:19.451443] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-02. [2025-10-20T19:08:19.646034] [INFO ] [REMOTE ] HCA: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:08:19.653319] [INFO ] [REMOTE ] HCA: Full fetch complete (68946 records). [2025-10-20T19:08:19.653436] [DEBUG] [VALIDATOR ] HCA: Validating primary data. [2025-10-20T19:08:19.728626] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-02 (0.28 seconds). [2025-10-20T19:08:20.046437] [DEBUG] [VALIDATOR ] HCA: All expected trading days are present. [2025-10-20T19:08:20.046570] [DEBUG] [VALIDATOR ] HCA: Primary data validation passed. [2025-10-20T19:08:20.047504] [INFO ] [MAIN ] HCA: Fetched primary data is valid. [2025-10-20T19:08:20.047569] [DEBUG] [LOCAL ] HCA: Saving primary data. [2025-10-20T19:08:20.048979] [DEBUG] [LOCAL ] HCA: Trimmed 68946 records to 46957 (keeping 200 days of data) [2025-10-20T19:08:20.259743] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-03. [2025-10-20T19:08:20.339404] [DEBUG] [LOCAL ] HCA: Primary data saved (46957 records). [2025-10-20T19:08:20.339725] [INFO ] [MAIN ] HCA: Full history saved. [2025-10-20T19:08:20.340624] [INFO ] [MAIN ] Update Progress: 997/2367 symbols completed. HCA took 31.7s. Estimated remaining: 2h 56m 15s. Rate: 7.8 symbols/min [2025-10-20T19:08:20.419760] [INFO ] [REMOTE ] HCC: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:08:20.426249] [INFO ] [REMOTE ] HCC: Full fetch complete (64916 records). [2025-10-20T19:08:20.426404] [DEBUG] [VALIDATOR ] HCC: Validating primary data. [2025-10-20T19:08:20.608366] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T19:08:20.842266] [DEBUG] [VALIDATOR ] HCC: All expected trading days are present. [2025-10-20T19:08:20.842347] [DEBUG] [VALIDATOR ] HCC: Primary data validation passed. [2025-10-20T19:08:20.843194] [INFO ] [MAIN ] HCC: Fetched primary data is valid. [2025-10-20T19:08:20.843258] [DEBUG] [LOCAL ] HCC: Saving primary data. [2025-10-20T19:08:20.844528] [DEBUG] [LOCAL ] HCC: Trimmed 64916 records to 45162 (keeping 200 days of data) [2025-10-20T19:08:21.063238] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-04. [2025-10-20T19:08:21.130835] [DEBUG] [LOCAL ] HCC: Primary data saved (45162 records). [2025-10-20T19:08:21.130954] [INFO ] [MAIN ] HCC: Full history saved. [2025-10-20T19:08:21.131414] [INFO ] [MAIN ] Update Progress: 998/2367 symbols completed. HCC took 28.5s. Estimated remaining: 2h 55m 58s. Rate: 7.8 symbols/min [2025-10-20T19:08:21.306282] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-04 (0.24 seconds). [2025-10-20T19:08:21.851422] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-05. [2025-10-20T19:08:22.093188] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T19:08:22.651353] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-06. [2025-10-20T19:08:23.041595] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T19:08:23.451388] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-08. [2025-10-20T19:08:24.251373] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-07. [2025-10-20T19:08:24.393056] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T19:08:24.569131] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T19:08:25.051370] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-09. [2025-10-20T19:08:25.851402] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-08. [2025-10-20T19:08:26.025808] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:08:26.166138] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-10-20T19:08:26.651421] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-09. [2025-10-20T19:08:26.883210] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-09 (0.23 seconds). [2025-10-20T19:08:27.451367] [INFO ] [REMOTE ] HCXY: Fetching primary data for 2025-10. [2025-10-20T19:08:27.808177] [INFO ] [REMOTE ] HCXY: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T19:08:27.808865] [INFO ] [REMOTE ] HCXY: Full fetch complete (510 records). [2025-10-20T19:08:27.808966] [DEBUG] [VALIDATOR ] HCXY: Validating primary data. [2025-10-20T19:08:27.809363] [FATAL] [VALIDATOR ] HCXY: Data is stale - most recent date (2025-10-16) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T19:08:27.809451] [FATAL] [MAIN ] HCXY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:08:27.809497] [WARN ] [MAIN ] HCXY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:08:27.809537] [INFO ] [LOCAL ] HCXY: Starting deletion of primary data. [2025-10-20T19:08:27.809595] [DEBUG] [LOCAL ] HCXY: No primary data file found to delete. [2025-10-20T19:08:27.809633] [INFO ] [LOCAL ] HCXY: Starting deletion of derivative data. [2025-10-20T19:08:27.809748] [DEBUG] [LOCAL ] HCXY: No derivative data file found to delete. [2025-10-20T19:08:27.809829] [INFO ] [MAIN ] Update Progress: 999/2367 symbols completed. HCXY took 16.8s. Estimated remaining: 2h 55m 49s. Rate: 7.8 symbols/min [2025-10-20T19:08:28.251376] [INFO ] [REMOTE ] HCI: Fetching primary data for 2025-10. [2025-10-20T19:08:29.288593] [INFO ] [REMOTE ] HCI: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:08:29.291281] [INFO ] [REMOTE ] HCI: Full fetch complete (19655 records). [2025-10-20T19:08:29.291360] [DEBUG] [VALIDATOR ] HCI: Validating primary data. [2025-10-20T19:08:29.663092] [DEBUG] [VALIDATOR ] HCI: All expected trading days are present. [2025-10-20T19:08:29.663200] [DEBUG] [VALIDATOR ] HCI: Primary data validation passed. [2025-10-20T19:08:29.663722] [INFO ] [MAIN ] HCI: Fetched primary data is valid. [2025-10-20T19:08:29.663788] [DEBUG] [LOCAL ] HCI: Saving primary data. [2025-10-20T19:08:29.664929] [DEBUG] [LOCAL ] HCI: Trimmed 19655 records to 14113 (keeping 200 days of data) [2025-10-20T19:08:29.757018] [DEBUG] [LOCAL ] HCI: Primary data saved (14113 records). [2025-10-20T19:08:29.757166] [INFO ] [MAIN ] HCI: Full history saved. [2025-10-20T19:08:29.757258] [INFO ] [MAIN ] Update Progress: 1000/2367 symbols completed. HCI took 34.1s. Estimated remaining: 2h 55m 33s. Rate: 7.8 symbols/min [2025-10-20T19:08:30.351524] [INFO ] [MAIN ] HDB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.354952] [INFO ] [MAIN ] HE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.482994] [INFO ] [MAIN ] HEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.503078] [INFO ] [MAIN ] HD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:30.503773] [DEBUG] [LOCAL ] HEI: No existing data file found in primary storage. [2025-10-20T19:08:30.503973] [WARN ] [MAIN ] HEI: No existing data found, performing full refresh instead. [2025-10-20T19:08:30.504202] [INFO ] [MAIN ] HEI: Wiping existing primary data before full refresh. [2025-10-20T19:08:30.504270] [INFO ] [LOCAL ] HEI: Starting deletion of primary data. [2025-10-20T19:08:30.504431] [DEBUG] [LOCAL ] HEI: No primary data file found to delete. [2025-10-20T19:08:30.504497] [INFO ] [MAIN ] HEI: Starting full primary refresh. [2025-10-20T19:08:30.869924] [DEBUG] [LOCAL ] HE: Loaded 49612 existing records from primary storage. [2025-10-20T19:08:30.930251] [DEBUG] [LOCAL ] HDB: Loaded 52710 existing records from primary storage. [2025-10-20T19:08:30.953211] [DEBUG] [LOCAL ] HD: Loaded 54264 existing records from primary storage. [2025-10-20T19:08:30.930381] [INFO ] [MAIN ] HE: Loaded 49612 existing records. [2025-10-20T19:08:30.978342] [INFO ] [MAIN ] HD: Loaded 54264 existing records. [2025-10-20T19:08:30.986702] [INFO ] [REMOTE ] HEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:30.996351] [INFO ] [MAIN ] HDB: Loaded 52710 existing records. [2025-10-20T19:08:31.077361] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-01. [2025-10-20T19:08:31.442034] [WARN ] [MAIN ] HE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.486471] [INFO ] [MAIN ] HE: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.537107] [INFO ] [LOCAL ] HE: Starting deletion of primary data. [2025-10-20T19:08:31.519469] [WARN ] [MAIN ] HD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.541149] [WARN ] [MAIN ] HDB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:31.541806] [INFO ] [MAIN ] HD: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.541972] [INFO ] [LOCAL ] HD: Starting deletion of primary data. [2025-10-20T19:08:31.542063] [DEBUG] [LOCAL ] HE: Deleted primary data file. [2025-10-20T19:08:31.542326] [INFO ] [MAIN ] HE: Starting full primary refresh. [2025-10-20T19:08:31.542392] [INFO ] [MAIN ] HDB: Wiping existing primary data before full refresh. [2025-10-20T19:08:31.542635] [INFO ] [LOCAL ] HDB: Starting deletion of primary data. [2025-10-20T19:08:31.542590] [DEBUG] [LOCAL ] HD: Deleted primary data file. [2025-10-20T19:08:31.542897] [INFO ] [MAIN ] HD: Starting full primary refresh. [2025-10-20T19:08:31.548192] [DEBUG] [LOCAL ] HDB: Deleted primary data file. [2025-10-20T19:08:31.548291] [INFO ] [MAIN ] HDB: Starting full primary refresh. [2025-10-20T19:08:32.520864] [INFO ] [REMOTE ] HE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.594181] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-01. [2025-10-20T19:08:32.597051] [INFO ] [REMOTE ] HDB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.598044] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-01. [2025-10-20T19:08:32.660853] [INFO ] [REMOTE ] HD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:32.661308] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-01. [2025-10-20T19:08:32.704383] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:08:32.712016] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-02. [2025-10-20T19:08:34.224783] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T19:08:34.232261] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-03. [2025-10-20T19:08:34.468013] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:08:34.478750] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-02. [2025-10-20T19:08:34.511327] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:08:34.550143] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T19:08:34.651363] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-02. [2025-10-20T19:08:35.451421] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-02. [2025-10-20T19:08:35.793248] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:08:35.881510] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:08:36.173936] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:08:36.251348] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-03. [2025-10-20T19:08:37.051374] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-03. [2025-10-20T19:08:37.278634] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T19:08:37.858683] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-04. [2025-10-20T19:08:37.879406] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:08:38.507803] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:08:38.651380] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-04. [2025-10-20T19:08:39.456491] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-04. [2025-10-20T19:08:39.470778] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:08:40.251400] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-05. [2025-10-20T19:08:40.399793] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:08:40.666943] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T19:08:41.051411] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-05. [2025-10-20T19:08:41.738608] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:08:41.851353] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-03. [2025-10-20T19:08:42.651433] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-06. [2025-10-20T19:08:42.761462] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:08:43.451355] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-05. [2025-10-20T19:08:43.729677] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:08:43.758074] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T19:08:44.251366] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-04. [2025-10-20T19:08:44.913022] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:08:45.051381] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-06. [2025-10-20T19:08:45.851464] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-06. [2025-10-20T19:08:46.450520] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-10-20T19:08:46.508408] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:08:46.651402] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-07. [2025-10-20T19:08:47.388805] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:08:47.451402] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-07. [2025-10-20T19:08:47.957167] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:08:48.251437] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-05. [2025-10-20T19:08:48.826246] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T19:08:49.051387] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-08. [2025-10-20T19:08:49.851373] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-07. [2025-10-20T19:08:50.435334] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T19:08:50.651340] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-06. [2025-10-20T19:08:50.730549] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T19:08:51.275025] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:08:51.451441] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-08. [2025-10-20T19:08:52.251427] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-09. [2025-10-20T19:08:52.438465] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:08:52.885847] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:08:53.051386] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-09. [2025-10-20T19:08:53.489135] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:08:53.846801] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T19:08:53.852928] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-08. [2025-10-20T19:08:54.651439] [INFO ] [REMOTE ] HEI: Fetching primary data for 2025-10. [2025-10-20T19:08:55.406055] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:08:55.451347] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-09. [2025-10-20T19:08:55.577334] [INFO ] [REMOTE ] HEI: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T19:08:55.581075] [INFO ] [REMOTE ] HEI: Full fetch complete (47512 records). [2025-10-20T19:08:55.581164] [DEBUG] [VALIDATOR ] HEI: Validating primary data. [2025-10-20T19:08:55.952957] [DEBUG] [VALIDATOR ] HEI: All expected trading days are present. [2025-10-20T19:08:55.953059] [DEBUG] [VALIDATOR ] HEI: Primary data validation passed. [2025-10-20T19:08:55.953786] [INFO ] [MAIN ] HEI: Fetched primary data is valid. [2025-10-20T19:08:55.953848] [DEBUG] [LOCAL ] HEI: Saving primary data. [2025-10-20T19:08:55.954888] [DEBUG] [LOCAL ] HEI: Trimmed 47512 records to 30890 (keeping 200 days of data) [2025-10-20T19:08:56.141585] [DEBUG] [LOCAL ] HEI: Primary data saved (30890 records). [2025-10-20T19:08:56.141742] [INFO ] [MAIN ] HEI: Full history saved. [2025-10-20T19:08:56.141821] [INFO ] [MAIN ] Update Progress: 1001/2367 symbols completed. HEI took 26.4s. Estimated remaining: 2h 55m 51s. Rate: 7.8 symbols/min [2025-10-20T19:08:56.256567] [INFO ] [REMOTE ] HE: Fetching primary data for 2025-10. [2025-10-20T19:08:56.308550] [INFO ] [MAIN ] HEQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:56.318148] [DEBUG] [LOCAL ] HEQ: Loaded 6898 existing records from primary storage. [2025-10-20T19:08:56.318374] [INFO ] [MAIN ] HEQ: Loaded 6898 existing records. [2025-10-20T19:08:56.492477] [WARN ] [MAIN ] HEQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:56.492696] [INFO ] [MAIN ] HEQ: Wiping existing primary data before full refresh. [2025-10-20T19:08:56.492753] [INFO ] [LOCAL ] HEQ: Starting deletion of primary data. [2025-10-20T19:08:56.492901] [DEBUG] [LOCAL ] HEQ: Deleted primary data file. [2025-10-20T19:08:56.492937] [INFO ] [MAIN ] HEQ: Starting full primary refresh. [2025-10-20T19:08:56.838292] [INFO ] [REMOTE ] HEQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:57.051359] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-07. [2025-10-20T19:08:57.160768] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:08:57.851471] [INFO ] [REMOTE ] HDB: Fetching primary data for 2025-10. [2025-10-20T19:08:58.040239] [INFO ] [REMOTE ] HE: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:08:58.047437] [INFO ] [REMOTE ] HE: Full fetch complete (72162 records). [2025-10-20T19:08:58.047580] [DEBUG] [VALIDATOR ] HE: Validating primary data. [2025-10-20T19:08:58.451621] [DEBUG] [VALIDATOR ] HE: All expected trading days are present. [2025-10-20T19:08:58.451832] [DEBUG] [VALIDATOR ] HE: Primary data validation passed. [2025-10-20T19:08:58.452795] [INFO ] [MAIN ] HE: Fetched primary data is valid. [2025-10-20T19:08:58.452858] [DEBUG] [LOCAL ] HE: Saving primary data. [2025-10-20T19:08:58.454250] [DEBUG] [LOCAL ] HE: Trimmed 72162 records to 48865 (keeping 200 days of data) [2025-10-20T19:08:58.652931] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-01. [2025-10-20T19:08:58.739847] [DEBUG] [LOCAL ] HE: Primary data saved (48865 records). [2025-10-20T19:08:58.739968] [INFO ] [MAIN ] HE: Full history saved. [2025-10-20T19:08:58.740045] [INFO ] [MAIN ] Update Progress: 1002/2367 symbols completed. HE took 29.0s. Estimated remaining: 2h 55m 36s. Rate: 7.8 symbols/min [2025-10-20T19:08:58.916080] [INFO ] [MAIN ] HESM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:08:58.959751] [DEBUG] [LOCAL ] HESM: Loaded 49235 existing records from primary storage. [2025-10-20T19:08:58.960201] [INFO ] [MAIN ] HESM: Loaded 49235 existing records. [2025-10-20T19:08:59.146589] [WARN ] [MAIN ] HESM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:08:59.147070] [INFO ] [MAIN ] HESM: Wiping existing primary data before full refresh. [2025-10-20T19:08:59.147214] [INFO ] [LOCAL ] HESM: Starting deletion of primary data. [2025-10-20T19:08:59.147748] [DEBUG] [LOCAL ] HESM: Deleted primary data file. [2025-10-20T19:08:59.147805] [INFO ] [MAIN ] HESM: Starting full primary refresh. [2025-10-20T19:08:59.257311] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T19:08:59.456962] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-02. [2025-10-20T19:08:59.562093] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-07 (2.51 seconds). [2025-10-20T19:08:59.617723] [INFO ] [REMOTE ] HESM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:08:59.702965] [INFO ] [REMOTE ] HDB: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:08:59.712576] [INFO ] [REMOTE ] HDB: Full fetch complete (75662 records). [2025-10-20T19:08:59.712839] [DEBUG] [VALIDATOR ] HDB: Validating primary data. [2025-10-20T19:09:00.232517] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:09:00.257027] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-01. [2025-10-20T19:09:00.261482] [DEBUG] [VALIDATOR ] HDB: All expected trading days are present. [2025-10-20T19:09:00.262018] [DEBUG] [VALIDATOR ] HDB: Primary data validation passed. [2025-10-20T19:09:00.267620] [INFO ] [MAIN ] HDB: Fetched primary data is valid. [2025-10-20T19:09:00.267895] [DEBUG] [LOCAL ] HDB: Saving primary data. [2025-10-20T19:09:00.269684] [DEBUG] [LOCAL ] HDB: Trimmed 75662 records to 51936 (keeping 200 days of data) [2025-10-20T19:09:00.625474] [DEBUG] [LOCAL ] HDB: Primary data saved (51936 records). [2025-10-20T19:09:00.625631] [INFO ] [MAIN ] HDB: Full history saved. [2025-10-20T19:09:00.626193] [INFO ] [MAIN ] Update Progress: 1003/2367 symbols completed. HDB took 30.9s. Estimated remaining: 2h 55m 21s. Rate: 7.8 symbols/min [2025-10-20T19:09:00.824235] [INFO ] [MAIN ] HF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:00.824448] [DEBUG] [LOCAL ] HF: No existing data file found in primary storage. [2025-10-20T19:09:00.824493] [WARN ] [MAIN ] HF: No existing data found, performing full refresh instead. [2025-10-20T19:09:00.824601] [INFO ] [MAIN ] HF: Wiping existing primary data before full refresh. [2025-10-20T19:09:00.824664] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-10-20T19:09:00.824759] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-10-20T19:09:00.824797] [INFO ] [MAIN ] HF: Starting full primary refresh. [2025-10-20T19:09:01.056592] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-03. [2025-10-20T19:09:01.247881] [INFO ] [REMOTE ] HF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:01.816795] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T19:09:01.854506] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-08. [2025-10-20T19:09:01.859209] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:09:02.651437] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-04. [2025-10-20T19:09:03.368157] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:09:03.451341] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-05. [2025-10-20T19:09:03.941869] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T19:09:04.111119] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T19:09:04.251385] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-06. [2025-10-20T19:09:04.833993] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:09:05.051435] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-09. [2025-10-20T19:09:05.851413] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-02. [2025-10-20T19:09:06.651377] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-07. [2025-10-20T19:09:07.381851] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T19:09:07.411075] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:09:07.430293] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:09:07.451330] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-08. [2025-10-20T19:09:08.100060] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T19:09:08.251360] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-09. [2025-10-20T19:09:08.960672] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:09:09.051403] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-01. [2025-10-20T19:09:09.387659] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-10-20T19:09:09.851368] [INFO ] [REMOTE ] HEQ: Fetching primary data for 2025-10. [2025-10-20T19:09:10.651455] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-03. [2025-10-20T19:09:10.665167] [INFO ] [REMOTE ] HEQ: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:09:10.667147] [INFO ] [REMOTE ] HEQ: Full fetch complete (9527 records). [2025-10-20T19:09:10.667230] [DEBUG] [VALIDATOR ] HEQ: Validating primary data. [2025-10-20T19:09:11.020046] [DEBUG] [VALIDATOR ] HEQ: All expected trading days are present. [2025-10-20T19:09:11.020123] [DEBUG] [VALIDATOR ] HEQ: Primary data validation passed. [2025-10-20T19:09:11.020345] [INFO ] [MAIN ] HEQ: Fetched primary data is valid. [2025-10-20T19:09:11.020382] [DEBUG] [LOCAL ] HEQ: Saving primary data. [2025-10-20T19:09:11.020819] [DEBUG] [LOCAL ] HEQ: Trimmed 9527 records to 6822 (keeping 200 days of data) [2025-10-20T19:09:11.060539] [DEBUG] [LOCAL ] HEQ: Primary data saved (6822 records). [2025-10-20T19:09:11.060675] [INFO ] [MAIN ] HEQ: Full history saved. [2025-10-20T19:09:11.060770] [INFO ] [MAIN ] Update Progress: 1004/2367 symbols completed. HEQ took 14.9s. Estimated remaining: 2h 55m 17s. Rate: 7.8 symbols/min [2025-10-20T19:09:11.227025] [INFO ] [MAIN ] HFND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:11.227211] [DEBUG] [LOCAL ] HFND: No existing data file found in primary storage. [2025-10-20T19:09:11.227247] [WARN ] [MAIN ] HFND: No existing data found, performing full refresh instead. [2025-10-20T19:09:11.227356] [INFO ] [MAIN ] HFND: Wiping existing primary data before full refresh. [2025-10-20T19:09:11.227398] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-10-20T19:09:11.227437] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-10-20T19:09:11.227468] [INFO ] [MAIN ] HFND: Starting full primary refresh. [2025-10-20T19:09:11.463744] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-02. [2025-10-20T19:09:11.578104] [INFO ] [REMOTE ] HFND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:12.132736] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:09:12.175596] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T19:09:12.251381] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-01. [2025-10-20T19:09:12.600205] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-01 (0.35 seconds). [2025-10-20T19:09:13.051388] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-03. [2025-10-20T19:09:13.298202] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-03 (0.25 seconds). [2025-10-20T19:09:13.851384] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-02. [2025-10-20T19:09:14.265779] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T19:09:14.651383] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-04. [2025-10-20T19:09:14.901308] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-04 (0.25 seconds). [2025-10-20T19:09:15.451413] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-05. [2025-10-20T19:09:15.694274] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-05 (0.24 seconds). [2025-10-20T19:09:16.251396] [INFO ] [REMOTE ] HD: Fetching primary data for 2025-10. [2025-10-20T19:09:17.051396] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-04. [2025-10-20T19:09:17.186518] [INFO ] [REMOTE ] HD: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:09:17.195023] [INFO ] [REMOTE ] HD: Full fetch complete (77564 records). [2025-10-20T19:09:17.195183] [DEBUG] [VALIDATOR ] HD: Validating primary data. [2025-10-20T19:09:17.605132] [DEBUG] [VALIDATOR ] HD: All expected trading days are present. [2025-10-20T19:09:17.605229] [DEBUG] [VALIDATOR ] HD: Primary data validation passed. [2025-10-20T19:09:17.606411] [INFO ] [MAIN ] HD: Fetched primary data is valid. [2025-10-20T19:09:17.606500] [DEBUG] [LOCAL ] HD: Saving primary data. [2025-10-20T19:09:17.608178] [DEBUG] [LOCAL ] HD: Trimmed 77564 records to 53482 (keeping 200 days of data) [2025-10-20T19:09:17.856868] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-03. [2025-10-20T19:09:17.963476] [DEBUG] [LOCAL ] HD: Primary data saved (53482 records). [2025-10-20T19:09:17.963619] [INFO ] [MAIN ] HD: Full history saved. [2025-10-20T19:09:17.963732] [INFO ] [MAIN ] Update Progress: 1005/2367 symbols completed. HD took 48.2s. Estimated remaining: 2h 55m 8s. Rate: 7.8 symbols/min [2025-10-20T19:09:18.161166] [INFO ] [MAIN ] HFRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:18.181813] [DEBUG] [LOCAL ] HFRO: Loaded 17015 existing records from primary storage. [2025-10-20T19:09:18.182093] [INFO ] [MAIN ] HFRO: Loaded 17015 existing records. [2025-10-20T19:09:18.352613] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T19:09:18.396574] [WARN ] [MAIN ] HFRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:18.397029] [INFO ] [MAIN ] HFRO: Wiping existing primary data before full refresh. [2025-10-20T19:09:18.397173] [INFO ] [LOCAL ] HFRO: Starting deletion of primary data. [2025-10-20T19:09:18.397426] [DEBUG] [LOCAL ] HFRO: Deleted primary data file. [2025-10-20T19:09:18.397604] [INFO ] [MAIN ] HFRO: Starting full primary refresh. [2025-10-20T19:09:18.662011] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-06. [2025-10-20T19:09:18.664005] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:09:18.815088] [INFO ] [REMOTE ] HFRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:18.971515] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-10-20T19:09:19.451377] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-01. [2025-10-20T19:09:20.251392] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-04. [2025-10-20T19:09:20.614770] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:09:20.666100] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T19:09:21.051384] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-05. [2025-10-20T19:09:21.402306] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T19:09:21.851379] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-05. [2025-10-20T19:09:22.651396] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-06. [2025-10-20T19:09:22.979667] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:09:23.283102] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:09:23.451358] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-06. [2025-10-20T19:09:24.251382] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-07. [2025-10-20T19:09:24.557612] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-07 (0.31 seconds). [2025-10-20T19:09:24.987867] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:09:25.051363] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-07. [2025-10-20T19:09:25.851366] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-07. [2025-10-20T19:09:26.270391] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T19:09:26.621320] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:09:26.651333] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-08. [2025-10-20T19:09:27.451366] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-08. [2025-10-20T19:09:27.706759] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-08 (0.26 seconds). [2025-10-20T19:09:28.151189] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:09:28.251338] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-09. [2025-10-20T19:09:29.051370] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-02. [2025-10-20T19:09:29.858270] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-09. [2025-10-20T19:09:29.863547] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:09:30.008403] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:09:30.136141] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-10-20T19:09:30.651346] [INFO ] [REMOTE ] HF: Fetching primary data for 2025-10. [2025-10-20T19:09:30.995702] [INFO ] [REMOTE ] HF: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-10-20T19:09:30.996547] [INFO ] [REMOTE ] HF: Full fetch complete (434 records). [2025-10-20T19:09:30.996626] [DEBUG] [VALIDATOR ] HF: Validating primary data. [2025-10-20T19:09:31.187338] [WARN ] [VALIDATOR ] HF: Validation failed - insufficient historical data. Have 143 unique days, need 200 days [2025-10-20T19:09:31.187531] [FATAL] [MAIN ] HF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:09:31.187601] [WARN ] [MAIN ] HF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:09:31.187696] [INFO ] [LOCAL ] HF: Starting deletion of primary data. [2025-10-20T19:09:31.187764] [DEBUG] [LOCAL ] HF: No primary data file found to delete. [2025-10-20T19:09:31.187801] [INFO ] [LOCAL ] HF: Starting deletion of derivative data. [2025-10-20T19:09:31.187841] [DEBUG] [LOCAL ] HF: No derivative data file found to delete. [2025-10-20T19:09:31.187896] [INFO ] [MAIN ] Update Progress: 1006/2367 symbols completed. HF took 30.6s. Estimated remaining: 2h 55m 8s. Rate: 7.8 symbols/min [2025-10-20T19:09:31.363783] [INFO ] [MAIN ] HGER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:31.381378] [DEBUG] [LOCAL ] HGER: Loaded 15286 existing records from primary storage. [2025-10-20T19:09:31.381603] [INFO ] [MAIN ] HGER: Loaded 15286 existing records. [2025-10-20T19:09:31.456560] [INFO ] [REMOTE ] HESM: Fetching primary data for 2025-10. [2025-10-20T19:09:31.565266] [WARN ] [MAIN ] HGER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:31.565480] [INFO ] [MAIN ] HGER: Wiping existing primary data before full refresh. [2025-10-20T19:09:31.565526] [INFO ] [LOCAL ] HGER: Starting deletion of primary data. [2025-10-20T19:09:31.565711] [DEBUG] [LOCAL ] HGER: Deleted primary data file. [2025-10-20T19:09:31.565766] [INFO ] [MAIN ] HGER: Starting full primary refresh. [2025-10-20T19:09:31.922247] [INFO ] [REMOTE ] HGER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:32.251365] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-08. [2025-10-20T19:09:32.673577] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T19:09:33.051413] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-03. [2025-10-20T19:09:33.410580] [INFO ] [REMOTE ] HESM: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T19:09:33.417861] [INFO ] [REMOTE ] HESM: Full fetch complete (69699 records). [2025-10-20T19:09:33.417980] [DEBUG] [VALIDATOR ] HESM: Validating primary data. [2025-10-20T19:09:33.829077] [DEBUG] [VALIDATOR ] HESM: All expected trading days are present. [2025-10-20T19:09:33.829194] [DEBUG] [VALIDATOR ] HESM: Primary data validation passed. [2025-10-20T19:09:33.830288] [INFO ] [MAIN ] HESM: Fetched primary data is valid. [2025-10-20T19:09:33.830365] [DEBUG] [LOCAL ] HESM: Saving primary data. [2025-10-20T19:09:33.831842] [DEBUG] [LOCAL ] HESM: Trimmed 69699 records to 48651 (keeping 200 days of data) [2025-10-20T19:09:33.887894] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-01. [2025-10-20T19:09:34.138376] [DEBUG] [LOCAL ] HESM: Primary data saved (48651 records). [2025-10-20T19:09:34.138631] [INFO ] [MAIN ] HESM: Full history saved. [2025-10-20T19:09:34.138806] [INFO ] [MAIN ] Update Progress: 1007/2367 symbols completed. HESM took 35.4s. Estimated remaining: 2h 54m 53s. Rate: 7.8 symbols/min [2025-10-20T19:09:34.221982] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T19:09:34.326821] [INFO ] [MAIN ] HGLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:34.337638] [DEBUG] [LOCAL ] HGLB: Loaded 9785 existing records from primary storage. [2025-10-20T19:09:34.337883] [INFO ] [MAIN ] HGLB: Loaded 9785 existing records. [2025-10-20T19:09:34.513988] [WARN ] [MAIN ] HGLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:34.514215] [INFO ] [MAIN ] HGLB: Wiping existing primary data before full refresh. [2025-10-20T19:09:34.514281] [INFO ] [LOCAL ] HGLB: Starting deletion of primary data. [2025-10-20T19:09:34.514455] [DEBUG] [LOCAL ] HGLB: Deleted primary data file. [2025-10-20T19:09:34.514517] [INFO ] [MAIN ] HGLB: Starting full primary refresh. [2025-10-20T19:09:34.620539] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:09:34.663989] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-02. [2025-10-20T19:09:34.882460] [INFO ] [REMOTE ] HGLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:35.099200] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:09:35.451415] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-03. [2025-10-20T19:09:36.152247] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T19:09:36.251424] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-04. [2025-10-20T19:09:37.051351] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-09. [2025-10-20T19:09:37.313667] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:09:37.472583] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-10-20T19:09:37.851403] [INFO ] [REMOTE ] HFND: Fetching primary data for 2025-10. [2025-10-20T19:09:38.345979] [INFO ] [REMOTE ] HFND: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:09:38.346956] [INFO ] [REMOTE ] HFND: Full fetch complete (1757 records). [2025-10-20T19:09:38.347022] [DEBUG] [VALIDATOR ] HFND: Validating primary data. [2025-10-20T19:09:38.521473] [WARN ] [VALIDATOR ] HFND: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T19:09:38.521672] [FATAL] [MAIN ] HFND: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:09:38.521731] [WARN ] [MAIN ] HFND: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:09:38.521789] [INFO ] [LOCAL ] HFND: Starting deletion of primary data. [2025-10-20T19:09:38.521841] [DEBUG] [LOCAL ] HFND: No primary data file found to delete. [2025-10-20T19:09:38.521876] [INFO ] [LOCAL ] HFND: Starting deletion of derivative data. [2025-10-20T19:09:38.521915] [DEBUG] [LOCAL ] HFND: No derivative data file found to delete. [2025-10-20T19:09:38.521966] [INFO ] [MAIN ] Update Progress: 1008/2367 symbols completed. HFND took 27.5s. Estimated remaining: 2h 54m 41s. Rate: 7.8 symbols/min [2025-10-20T19:09:38.657937] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-04. [2025-10-20T19:09:38.689458] [INFO ] [MAIN ] HGTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:38.705842] [DEBUG] [LOCAL ] HGTY: Loaded 13543 existing records from primary storage. [2025-10-20T19:09:38.706182] [INFO ] [MAIN ] HGTY: Loaded 13543 existing records. [2025-10-20T19:09:38.884921] [WARN ] [MAIN ] HGTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:38.885118] [INFO ] [MAIN ] HGTY: Wiping existing primary data before full refresh. [2025-10-20T19:09:38.885164] [INFO ] [LOCAL ] HGTY: Starting deletion of primary data. [2025-10-20T19:09:38.885359] [DEBUG] [LOCAL ] HGTY: Deleted primary data file. [2025-10-20T19:09:38.885413] [INFO ] [MAIN ] HGTY: Starting full primary refresh. [2025-10-20T19:09:39.230303] [INFO ] [REMOTE ] HGTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:39.451354] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-05. [2025-10-20T19:09:39.502407] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T19:09:40.224020] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T19:09:40.251381] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-01. [2025-10-20T19:09:40.963331] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:09:41.051363] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-05. [2025-10-20T19:09:41.851402] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-01. [2025-10-20T19:09:41.863862] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:09:42.603107] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T19:09:42.651354] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-06. [2025-10-20T19:09:43.451375] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-06. [2025-10-20T19:09:43.479402] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:09:44.236699] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:09:44.251336] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-07. [2025-10-20T19:09:45.051353] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-02. [2025-10-20T19:09:45.178272] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T19:09:45.677896] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T19:09:45.851372] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-03. [2025-10-20T19:09:46.585243] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:09:46.651359] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-02. [2025-10-20T19:09:47.415560] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:09:47.451382] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-07. [2025-10-20T19:09:48.251382] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-04. [2025-10-20T19:09:48.363038] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:09:49.030169] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T19:09:49.051354] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-05. [2025-10-20T19:09:49.851363] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-03. [2025-10-20T19:09:49.958806] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T19:09:50.422484] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:09:50.651386] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-08. [2025-10-20T19:09:51.451355] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-08. [2025-10-20T19:09:51.813357] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T19:09:52.251435] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-09. [2025-10-20T19:09:52.427029] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T19:09:52.947600] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T19:09:53.051401] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-06. [2025-10-20T19:09:53.816134] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T19:09:53.851349] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-07. [2025-10-20T19:09:54.607859] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:09:54.651408] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-08. [2025-10-20T19:09:55.451393] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-09. [2025-10-20T19:09:55.583611] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T19:09:56.251368] [INFO ] [REMOTE ] HFRO: Fetching primary data for 2025-10. [2025-10-20T19:09:56.418426] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:09:57.051378] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-09. [2025-10-20T19:09:57.125101] [INFO ] [REMOTE ] HFRO: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:09:57.127392] [INFO ] [REMOTE ] HFRO: Full fetch complete (27267 records). [2025-10-20T19:09:57.127465] [DEBUG] [VALIDATOR ] HFRO: Validating primary data. [2025-10-20T19:09:57.496363] [DEBUG] [VALIDATOR ] HFRO: All expected trading days are present. [2025-10-20T19:09:57.496462] [DEBUG] [VALIDATOR ] HFRO: Primary data validation passed. [2025-10-20T19:09:57.497255] [INFO ] [MAIN ] HFRO: Fetched primary data is valid. [2025-10-20T19:09:57.497321] [DEBUG] [LOCAL ] HFRO: Saving primary data. [2025-10-20T19:09:57.498065] [DEBUG] [LOCAL ] HFRO: Trimmed 27267 records to 16506 (keeping 200 days of data) [2025-10-20T19:09:57.598446] [DEBUG] [LOCAL ] HFRO: Primary data saved (16506 records). [2025-10-20T19:09:57.598593] [INFO ] [MAIN ] HFRO: Full history saved. [2025-10-20T19:09:57.598698] [INFO ] [MAIN ] Update Progress: 1009/2367 symbols completed. HFRO took 39.6s. Estimated remaining: 2h 54m 49s. Rate: 7.8 symbols/min [2025-10-20T19:09:57.787598] [INFO ] [MAIN ] HGV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:57.833948] [DEBUG] [LOCAL ] HGV: Loaded 46933 existing records from primary storage. [2025-10-20T19:09:57.834295] [INFO ] [MAIN ] HGV: Loaded 46933 existing records. [2025-10-20T19:09:57.856773] [INFO ] [REMOTE ] HGER: Fetching primary data for 2025-10. [2025-10-20T19:09:58.051596] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:09:58.055738] [WARN ] [MAIN ] HGV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:09:58.056155] [INFO ] [MAIN ] HGV: Wiping existing primary data before full refresh. [2025-10-20T19:09:58.060819] [INFO ] [LOCAL ] HGV: Starting deletion of primary data. [2025-10-20T19:09:58.061856] [DEBUG] [LOCAL ] HGV: Deleted primary data file. [2025-10-20T19:09:58.061927] [INFO ] [MAIN ] HGV: Starting full primary refresh. [2025-10-20T19:09:58.417111] [INFO ] [REMOTE ] HGV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:09:58.651390] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-01. [2025-10-20T19:09:59.184152] [INFO ] [REMOTE ] HGER: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T19:09:59.189001] [INFO ] [REMOTE ] HGER: Full fetch complete (18593 records). [2025-10-20T19:09:59.189120] [DEBUG] [VALIDATOR ] HGER: Validating primary data. [2025-10-20T19:09:59.456502] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-04. [2025-10-20T19:09:59.563932] [DEBUG] [VALIDATOR ] HGER: All expected trading days are present. [2025-10-20T19:09:59.564016] [DEBUG] [VALIDATOR ] HGER: Primary data validation passed. [2025-10-20T19:09:59.564398] [INFO ] [MAIN ] HGER: Fetched primary data is valid. [2025-10-20T19:09:59.564435] [DEBUG] [LOCAL ] HGER: Saving primary data. [2025-10-20T19:09:59.565100] [DEBUG] [LOCAL ] HGER: Trimmed 18593 records to 15250 (keeping 200 days of data) [2025-10-20T19:09:59.653451] [DEBUG] [LOCAL ] HGER: Primary data saved (15250 records). [2025-10-20T19:09:59.653595] [INFO ] [MAIN ] HGER: Full history saved. [2025-10-20T19:09:59.653701] [INFO ] [MAIN ] Update Progress: 1010/2367 symbols completed. HGER took 28.5s. Estimated remaining: 2h 54m 33s. Rate: 7.8 symbols/min [2025-10-20T19:09:59.821097] [INFO ] [MAIN ] HHH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:09:59.852973] [DEBUG] [LOCAL ] HHH: Loaded 33589 existing records from primary storage. [2025-10-20T19:09:59.853401] [INFO ] [MAIN ] HHH: Loaded 33589 existing records. [2025-10-20T19:10:00.031615] [WARN ] [MAIN ] HHH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:00.032173] [INFO ] [MAIN ] HHH: Wiping existing primary data before full refresh. [2025-10-20T19:10:00.032224] [INFO ] [LOCAL ] HHH: Starting deletion of primary data. [2025-10-20T19:10:00.032432] [DEBUG] [LOCAL ] HHH: Deleted primary data file. [2025-10-20T19:10:00.032488] [INFO ] [MAIN ] HHH: Starting full primary refresh. [2025-10-20T19:10:00.277975] [INFO ] [REMOTE ] HGTY: Fetching primary data for 2025-10. [2025-10-20T19:10:00.429572] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:10:00.432717] [INFO ] [REMOTE ] HHH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:00.449473] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T19:10:01.051379] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-02. [2025-10-20T19:10:01.256111] [INFO ] [REMOTE ] HGTY: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:10:01.258557] [INFO ] [REMOTE ] HGTY: Full fetch complete (17575 records). [2025-10-20T19:10:01.258627] [DEBUG] [VALIDATOR ] HGTY: Validating primary data. [2025-10-20T19:10:01.646594] [DEBUG] [VALIDATOR ] HGTY: All expected trading days are present. [2025-10-20T19:10:01.646698] [DEBUG] [VALIDATOR ] HGTY: Primary data validation passed. [2025-10-20T19:10:01.647089] [INFO ] [MAIN ] HGTY: Fetched primary data is valid. [2025-10-20T19:10:01.647129] [DEBUG] [LOCAL ] HGTY: Saving primary data. [2025-10-20T19:10:01.647722] [DEBUG] [LOCAL ] HGTY: Trimmed 17575 records to 13382 (keeping 200 days of data) [2025-10-20T19:10:01.733629] [DEBUG] [LOCAL ] HGTY: Primary data saved (13382 records). [2025-10-20T19:10:01.733841] [INFO ] [MAIN ] HGTY: Full history saved. [2025-10-20T19:10:01.733943] [INFO ] [MAIN ] Update Progress: 1011/2367 symbols completed. HGTY took 23.2s. Estimated remaining: 2h 54m 18s. Rate: 7.8 symbols/min [2025-10-20T19:10:01.860746] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-01. [2025-10-20T19:10:01.941377] [INFO ] [MAIN ] HI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:01.982098] [DEBUG] [LOCAL ] HI: Loaded 36524 existing records from primary storage. [2025-10-20T19:10:01.982432] [INFO ] [MAIN ] HI: Loaded 36524 existing records. [2025-10-20T19:10:02.168242] [WARN ] [MAIN ] HI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:02.168707] [INFO ] [MAIN ] HI: Wiping existing primary data before full refresh. [2025-10-20T19:10:02.168802] [INFO ] [LOCAL ] HI: Starting deletion of primary data. [2025-10-20T19:10:02.169206] [DEBUG] [LOCAL ] HI: Deleted primary data file. [2025-10-20T19:10:02.169271] [INFO ] [MAIN ] HI: Starting full primary refresh. [2025-10-20T19:10:02.522837] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:10:02.634480] [INFO ] [REMOTE ] HI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:02.651439] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-01. [2025-10-20T19:10:03.176587] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:10:03.451416] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-02. [2025-10-20T19:10:04.143311] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:10:04.251388] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-02. [2025-10-20T19:10:04.835214] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:10:05.051426] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-05. [2025-10-20T19:10:05.588715] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:10:05.629638] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:10:05.851379] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-06. [2025-10-20T19:10:06.547634] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T19:10:06.651384] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-03. [2025-10-20T19:10:07.451434] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-03. [2025-10-20T19:10:07.949505] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:10:08.251418] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-03. [2025-10-20T19:10:09.052699] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-07. [2025-10-20T19:10:09.060419] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:10:09.672571] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:10:09.837725] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T19:10:09.851391] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-08. [2025-10-20T19:10:10.651475] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-04. [2025-10-20T19:10:10.699058] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:10:11.451474] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-04. [2025-10-20T19:10:12.256587] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-04. [2025-10-20T19:10:12.299522] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T19:10:13.051381] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-05. [2025-10-20T19:10:13.098588] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T19:10:13.450271] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:10:13.851414] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-05. [2025-10-20T19:10:14.547459] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:10:14.651452] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-05. [2025-10-20T19:10:15.455437] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-09. [2025-10-20T19:10:15.462733] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:10:16.022896] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:10:16.218193] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T19:10:16.251348] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-06. [2025-10-20T19:10:17.051406] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-06. [2025-10-20T19:10:17.820066] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:10:17.851459] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-07. [2025-10-20T19:10:18.461717] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:10:18.651410] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-06. [2025-10-20T19:10:19.451431] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-07. [2025-10-20T19:10:19.563610] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:10:20.010614] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:10:20.251376] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-07. [2025-10-20T19:10:20.943860] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:10:21.051407] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-08. [2025-10-20T19:10:21.452936] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:10:21.851391] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-08. [2025-10-20T19:10:22.238837] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:10:22.651448] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-09. [2025-10-20T19:10:23.316162] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:10:23.451397] [INFO ] [REMOTE ] HGLB: Fetching primary data for 2025-10. [2025-10-20T19:10:23.756447] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:10:24.251405] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-09. [2025-10-20T19:10:24.383975] [INFO ] [REMOTE ] HGLB: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T19:10:24.386256] [INFO ] [REMOTE ] HGLB: Full fetch complete (14413 records). [2025-10-20T19:10:24.386325] [DEBUG] [VALIDATOR ] HGLB: Validating primary data. [2025-10-20T19:10:24.757064] [DEBUG] [VALIDATOR ] HGLB: All expected trading days are present. [2025-10-20T19:10:24.757186] [DEBUG] [VALIDATOR ] HGLB: Primary data validation passed. [2025-10-20T19:10:24.757553] [INFO ] [MAIN ] HGLB: Fetched primary data is valid. [2025-10-20T19:10:24.757600] [DEBUG] [LOCAL ] HGLB: Saving primary data. [2025-10-20T19:10:24.758286] [DEBUG] [LOCAL ] HGLB: Trimmed 14413 records to 9696 (keeping 200 days of data) [2025-10-20T19:10:24.816031] [DEBUG] [LOCAL ] HGLB: Primary data saved (9696 records). [2025-10-20T19:10:24.816154] [INFO ] [MAIN ] HGLB: Full history saved. [2025-10-20T19:10:24.816232] [INFO ] [MAIN ] Update Progress: 1012/2367 symbols completed. HGLB took 50.7s. Estimated remaining: 2h 54m 31s. Rate: 7.8 symbols/min [2025-10-20T19:10:24.988368] [INFO ] [MAIN ] HIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:25.040091] [DEBUG] [LOCAL ] HIG: Loaded 51943 existing records from primary storage. [2025-10-20T19:10:25.040444] [INFO ] [MAIN ] HIG: Loaded 51943 existing records. [2025-10-20T19:10:25.061014] [INFO ] [REMOTE ] HGV: Fetching primary data for 2025-10. [2025-10-20T19:10:25.242895] [WARN ] [MAIN ] HIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:25.243172] [INFO ] [MAIN ] HIG: Wiping existing primary data before full refresh. [2025-10-20T19:10:25.243268] [INFO ] [LOCAL ] HIG: Starting deletion of primary data. [2025-10-20T19:10:25.243542] [DEBUG] [LOCAL ] HIG: Deleted primary data file. [2025-10-20T19:10:25.243599] [INFO ] [MAIN ] HIG: Starting full primary refresh. [2025-10-20T19:10:25.614204] [INFO ] [REMOTE ] HIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:25.632400] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:10:25.851339] [INFO ] [REMOTE ] HI: Fetching primary data for 2025-10. [2025-10-20T19:10:26.651394] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-01. [2025-10-20T19:10:26.687252] [INFO ] [REMOTE ] HGV: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:10:26.693888] [INFO ] [REMOTE ] HGV: Full fetch complete (65908 records). [2025-10-20T19:10:26.694000] [DEBUG] [VALIDATOR ] HGV: Validating primary data. [2025-10-20T19:10:27.080994] [DEBUG] [VALIDATOR ] HGV: All expected trading days are present. [2025-10-20T19:10:27.081151] [DEBUG] [VALIDATOR ] HGV: Primary data validation passed. [2025-10-20T19:10:27.082035] [INFO ] [MAIN ] HGV: Fetched primary data is valid. [2025-10-20T19:10:27.082096] [DEBUG] [LOCAL ] HGV: Saving primary data. [2025-10-20T19:10:27.083441] [DEBUG] [LOCAL ] HGV: Trimmed 65908 records to 46172 (keeping 200 days of data) [2025-10-20T19:10:27.343746] [DEBUG] [LOCAL ] HGV: Primary data saved (46172 records). [2025-10-20T19:10:27.343871] [INFO ] [MAIN ] HGV: Full history saved. [2025-10-20T19:10:27.344721] [INFO ] [MAIN ] Update Progress: 1013/2367 symbols completed. HGV took 29.7s. Estimated remaining: 2h 54m 16s. Rate: 7.8 symbols/min [2025-10-20T19:10:27.461174] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-08. [2025-10-20T19:10:27.514256] [INFO ] [MAIN ] HII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:27.533441] [INFO ] [REMOTE ] HI: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:10:27.539846] [INFO ] [REMOTE ] HI: Full fetch complete (50955 records). [2025-10-20T19:10:27.539975] [DEBUG] [VALIDATOR ] HI: Validating primary data. [2025-10-20T19:10:27.748164] [DEBUG] [LOCAL ] HII: Loaded 34269 existing records from primary storage. [2025-10-20T19:10:27.785750] [INFO ] [MAIN ] HII: Loaded 34269 existing records. [2025-10-20T19:10:28.124330] [WARN ] [MAIN ] HII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:28.147039] [INFO ] [MAIN ] HII: Wiping existing primary data before full refresh. [2025-10-20T19:10:28.147172] [DEBUG] [VALIDATOR ] HI: All expected trading days are present. [2025-10-20T19:10:28.147280] [INFO ] [LOCAL ] HII: Starting deletion of primary data. [2025-10-20T19:10:28.147341] [DEBUG] [VALIDATOR ] HI: Primary data validation passed. [2025-10-20T19:10:28.148173] [INFO ] [MAIN ] HI: Fetched primary data is valid. [2025-10-20T19:10:28.148283] [DEBUG] [LOCAL ] HI: Saving primary data. [2025-10-20T19:10:28.148478] [DEBUG] [LOCAL ] HII: Deleted primary data file. [2025-10-20T19:10:28.148540] [INFO ] [MAIN ] HII: Starting full primary refresh. [2025-10-20T19:10:28.241507] [DEBUG] [LOCAL ] HI: Trimmed 50955 records to 36061 (keeping 200 days of data) [2025-10-20T19:10:28.766443] [INFO ] [REMOTE ] HII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:28.773577] [DEBUG] [LOCAL ] HI: Primary data saved (36061 records). [2025-10-20T19:10:28.773870] [INFO ] [MAIN ] HI: Full history saved. [2025-10-20T19:10:28.773453] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-01. [2025-10-20T19:10:28.774266] [INFO ] [MAIN ] Update Progress: 1014/2367 symbols completed. HI took 27.0s. Estimated remaining: 2h 54m 0s. Rate: 7.8 symbols/min [2025-10-20T19:10:28.875506] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-01 (2.22 seconds). [2025-10-20T19:10:29.029072] [INFO ] [MAIN ] HIMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:29.031673] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:10:29.064692] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-09. [2025-10-20T19:10:29.089105] [DEBUG] [LOCAL ] HIMS: Loaded 54597 existing records from primary storage. [2025-10-20T19:10:29.089477] [INFO ] [MAIN ] HIMS: Loaded 54597 existing records. [2025-10-20T19:10:29.261159] [WARN ] [MAIN ] HIMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:29.261360] [INFO ] [MAIN ] HIMS: Wiping existing primary data before full refresh. [2025-10-20T19:10:29.261406] [INFO ] [LOCAL ] HIMS: Starting deletion of primary data. [2025-10-20T19:10:29.261628] [DEBUG] [LOCAL ] HIMS: Deleted primary data file. [2025-10-20T19:10:29.261715] [INFO ] [MAIN ] HIMS: Starting full primary refresh. [2025-10-20T19:10:29.609479] [INFO ] [REMOTE ] HIMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:29.851361] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-01. [2025-10-20T19:10:30.118839] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:10:30.189689] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T19:10:30.651434] [INFO ] [REMOTE ] HHH: Fetching primary data for 2025-10. [2025-10-20T19:10:31.451407] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-02. [2025-10-20T19:10:32.106910] [INFO ] [REMOTE ] HHH: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:10:32.111398] [INFO ] [REMOTE ] HHH: Full fetch complete (46048 records). [2025-10-20T19:10:32.111480] [DEBUG] [VALIDATOR ] HHH: Validating primary data. [2025-10-20T19:10:32.255319] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-02. [2025-10-20T19:10:32.519383] [DEBUG] [VALIDATOR ] HHH: All expected trading days are present. [2025-10-20T19:10:32.534279] [DEBUG] [VALIDATOR ] HHH: Primary data validation passed. [2025-10-20T19:10:32.535282] [INFO ] [MAIN ] HHH: Fetched primary data is valid. [2025-10-20T19:10:32.535570] [DEBUG] [LOCAL ] HHH: Saving primary data. [2025-10-20T19:10:32.537329] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-01 (2.69 seconds). [2025-10-20T19:10:32.549000] [DEBUG] [LOCAL ] HHH: Trimmed 46048 records to 33185 (keeping 200 days of data) [2025-10-20T19:10:32.754812] [DEBUG] [LOCAL ] HHH: Primary data saved (33185 records). [2025-10-20T19:10:32.755070] [INFO ] [MAIN ] HHH: Full history saved. [2025-10-20T19:10:32.755760] [INFO ] [MAIN ] Update Progress: 1015/2367 symbols completed. HHH took 33.1s. Estimated remaining: 2h 53m 48s. Rate: 7.8 symbols/min [2025-10-20T19:10:32.936044] [INFO ] [MAIN ] HIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:32.961426] [DEBUG] [LOCAL ] HIO: Loaded 24778 existing records from primary storage. [2025-10-20T19:10:32.961703] [INFO ] [MAIN ] HIO: Loaded 24778 existing records. [2025-10-20T19:10:33.056031] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-02. [2025-10-20T19:10:33.173621] [WARN ] [MAIN ] HIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:33.174188] [INFO ] [MAIN ] HIO: Wiping existing primary data before full refresh. [2025-10-20T19:10:33.174780] [INFO ] [LOCAL ] HIO: Starting deletion of primary data. [2025-10-20T19:10:33.176200] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:10:33.186344] [DEBUG] [LOCAL ] HIO: Deleted primary data file. [2025-10-20T19:10:33.186462] [INFO ] [MAIN ] HIO: Starting full primary refresh. [2025-10-20T19:10:33.539108] [INFO ] [REMOTE ] HIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:33.851351] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-01. [2025-10-20T19:10:34.014520] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:10:34.651424] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-03. [2025-10-20T19:10:35.039502] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:10:35.451395] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-02. [2025-10-20T19:10:35.509107] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-02 (2.45 seconds). [2025-10-20T19:10:36.065448] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:10:36.251413] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-04. [2025-10-20T19:10:36.735800] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T19:10:37.051432] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-03. [2025-10-20T19:10:37.851407] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-03. [2025-10-20T19:10:37.916565] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T19:10:38.651412] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-03. [2025-10-20T19:10:39.451404] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-05. [2025-10-20T19:10:39.555518] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:10:39.662210] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-03 (2.61 seconds). [2025-10-20T19:10:39.691222] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:10:40.251376] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-04. [2025-10-20T19:10:41.051382] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-04. [2025-10-20T19:10:41.191933] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T19:10:41.851376] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-04. [2025-10-20T19:10:42.651353] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-06. [2025-10-20T19:10:42.715749] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:10:42.876600] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:10:42.970185] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:10:43.451363] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-05. [2025-10-20T19:10:44.192415] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:10:44.251349] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-07. [2025-10-20T19:10:45.051360] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-05. [2025-10-20T19:10:45.851365] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-05. [2025-10-20T19:10:45.923908] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T19:10:46.211936] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-05 (2.76 seconds). [2025-10-20T19:10:46.556384] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:10:46.651348] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-06. [2025-10-20T19:10:46.878716] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:10:47.451376] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-06. [2025-10-20T19:10:48.251398] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-08. [2025-10-20T19:10:48.524003] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:10:48.994348] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-06 (2.34 seconds). [2025-10-20T19:10:49.051333] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-07. [2025-10-20T19:10:49.685415] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:10:49.851358] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-09. [2025-10-20T19:10:50.651440] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-07. [2025-10-20T19:10:51.460952] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-06. [2025-10-20T19:10:51.466758] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T19:10:51.812724] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-07 (2.76 seconds). [2025-10-20T19:10:51.845477] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:10:52.187049] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:10:52.251362] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-08. [2025-10-20T19:10:53.051440] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-08. [2025-10-20T19:10:53.851408] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-07. [2025-10-20T19:10:54.592897] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T19:10:54.651381] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-09. [2025-10-20T19:10:55.024874] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T19:10:55.080024] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-08 (2.83 seconds). [2025-10-20T19:10:55.451372] [INFO ] [REMOTE ] HIG: Fetching primary data for 2025-10. [2025-10-20T19:10:55.975609] [INFO ] [REMOTE ] HIG: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T19:10:55.983070] [INFO ] [REMOTE ] HIG: Full fetch complete (73639 records). [2025-10-20T19:10:55.983177] [DEBUG] [VALIDATOR ] HIG: Validating primary data. [2025-10-20T19:10:56.126220] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:10:56.261549] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-08. [2025-10-20T19:10:56.404747] [DEBUG] [VALIDATOR ] HIG: All expected trading days are present. [2025-10-20T19:10:56.404834] [DEBUG] [VALIDATOR ] HIG: Primary data validation passed. [2025-10-20T19:10:56.405767] [INFO ] [MAIN ] HIG: Fetched primary data is valid. [2025-10-20T19:10:56.405829] [DEBUG] [LOCAL ] HIG: Saving primary data. [2025-10-20T19:10:56.407173] [DEBUG] [LOCAL ] HIG: Trimmed 73639 records to 51188 (keeping 200 days of data) [2025-10-20T19:10:56.727602] [DEBUG] [LOCAL ] HIG: Primary data saved (51188 records). [2025-10-20T19:10:56.727772] [INFO ] [MAIN ] HIG: Full history saved. [2025-10-20T19:10:56.727857] [INFO ] [MAIN ] Update Progress: 1016/2367 symbols completed. HIG took 31.9s. Estimated remaining: 2h 54m 1s. Rate: 7.8 symbols/min [2025-10-20T19:10:56.897974] [INFO ] [MAIN ] HIPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:10:56.918606] [DEBUG] [LOCAL ] HIPO: Loaded 22131 existing records from primary storage. [2025-10-20T19:10:56.918948] [INFO ] [MAIN ] HIPO: Loaded 22131 existing records. [2025-10-20T19:10:57.058085] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-09. [2025-10-20T19:10:57.100994] [WARN ] [MAIN ] HIPO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:10:57.101193] [INFO ] [MAIN ] HIPO: Wiping existing primary data before full refresh. [2025-10-20T19:10:57.101241] [INFO ] [LOCAL ] HIPO: Starting deletion of primary data. [2025-10-20T19:10:57.101461] [DEBUG] [LOCAL ] HIPO: Deleted primary data file. [2025-10-20T19:10:57.101500] [INFO ] [MAIN ] HIPO: Starting full primary refresh. [2025-10-20T19:10:57.457046] [INFO ] [REMOTE ] HIPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:10:57.745884] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:10:57.851356] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-01. [2025-10-20T19:10:58.651358] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-09. [2025-10-20T19:10:58.955234] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:10:59.451379] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-02. [2025-10-20T19:10:59.546610] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-09 (2.49 seconds). [2025-10-20T19:10:59.911184] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:11:00.251374] [INFO ] [REMOTE ] HIO: Fetching primary data for 2025-10. [2025-10-20T19:11:00.428787] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:11:01.051379] [INFO ] [REMOTE ] HIMS: Fetching primary data for 2025-10. [2025-10-20T19:11:01.851440] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-03. [2025-10-20T19:11:02.005163] [INFO ] [REMOTE ] HIO: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T19:11:02.013920] [INFO ] [REMOTE ] HIO: Full fetch complete (37954 records). [2025-10-20T19:11:02.014131] [DEBUG] [VALIDATOR ] HIO: Validating primary data. [2025-10-20T19:11:02.559690] [DEBUG] [VALIDATOR ] HIO: All expected trading days are present. [2025-10-20T19:11:02.559814] [DEBUG] [VALIDATOR ] HIO: Primary data validation passed. [2025-10-20T19:11:02.560809] [INFO ] [MAIN ] HIO: Fetched primary data is valid. [2025-10-20T19:11:02.560888] [DEBUG] [LOCAL ] HIO: Saving primary data. [2025-10-20T19:11:02.562239] [DEBUG] [LOCAL ] HIO: Trimmed 37954 records to 24124 (keeping 200 days of data) [2025-10-20T19:11:02.690261] [INFO ] [REMOTE ] HII: Fetching primary data for 2025-10. [2025-10-20T19:11:02.790678] [DEBUG] [LOCAL ] HIO: Primary data saved (24124 records). [2025-10-20T19:11:02.790882] [INFO ] [MAIN ] HIO: Full history saved. [2025-10-20T19:11:02.791010] [INFO ] [MAIN ] Update Progress: 1017/2367 symbols completed. HIO took 30.0s. Estimated remaining: 2h 53m 51s. Rate: 7.8 symbols/min [2025-10-20T19:11:03.004440] [INFO ] [MAIN ] HIW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:03.020408] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T19:11:03.075197] [DEBUG] [LOCAL ] HIW: Loaded 45201 existing records from primary storage. [2025-10-20T19:11:03.075534] [INFO ] [MAIN ] HIW: Loaded 45201 existing records. [2025-10-20T19:11:03.251774] [WARN ] [MAIN ] HIW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:03.252005] [INFO ] [MAIN ] HIW: Wiping existing primary data before full refresh. [2025-10-20T19:11:03.252052] [INFO ] [LOCAL ] HIW: Starting deletion of primary data. [2025-10-20T19:11:03.252504] [DEBUG] [LOCAL ] HIW: Deleted primary data file. [2025-10-20T19:11:03.252558] [INFO ] [MAIN ] HIW: Starting full primary refresh. [2025-10-20T19:11:03.473718] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-04. [2025-10-20T19:11:03.613805] [INFO ] [REMOTE ] HIW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:03.706050] [INFO ] [REMOTE ] HIMS: Incremental fetch complete for 2025-10 (2.65 seconds). [2025-10-20T19:11:03.716492] [INFO ] [REMOTE ] HIMS: Full fetch complete (77996 records). [2025-10-20T19:11:03.716795] [DEBUG] [VALIDATOR ] HIMS: Validating primary data. [2025-10-20T19:11:04.130856] [DEBUG] [VALIDATOR ] HIMS: All expected trading days are present. [2025-10-20T19:11:04.131002] [DEBUG] [VALIDATOR ] HIMS: Primary data validation passed. [2025-10-20T19:11:04.132154] [INFO ] [MAIN ] HIMS: Fetched primary data is valid. [2025-10-20T19:11:04.132218] [DEBUG] [LOCAL ] HIMS: Saving primary data. [2025-10-20T19:11:04.133718] [DEBUG] [LOCAL ] HIMS: Trimmed 77996 records to 53817 (keeping 200 days of data) [2025-10-20T19:11:04.251356] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-01. [2025-10-20T19:11:04.360895] [INFO ] [REMOTE ] HII: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:11:04.381273] [INFO ] [REMOTE ] HII: Full fetch complete (52464 records). [2025-10-20T19:11:04.386074] [DEBUG] [VALIDATOR ] HII: Validating primary data. [2025-10-20T19:11:04.767058] [DEBUG] [LOCAL ] HIMS: Primary data saved (53817 records). [2025-10-20T19:11:04.788346] [INFO ] [MAIN ] HIMS: Full history saved. [2025-10-20T19:11:04.790182] [INFO ] [MAIN ] Update Progress: 1018/2367 symbols completed. HIMS took 36.0s. Estimated remaining: 2h 53m 36s. Rate: 7.8 symbols/min [2025-10-20T19:11:04.824706] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T19:11:05.029722] [DEBUG] [VALIDATOR ] HII: All expected trading days are present. [2025-10-20T19:11:05.041728] [DEBUG] [VALIDATOR ] HII: Primary data validation passed. [2025-10-20T19:11:05.071569] [INFO ] [MAIN ] HIX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:05.069174] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-05. [2025-10-20T19:11:05.072761] [INFO ] [MAIN ] HII: Fetched primary data is valid. [2025-10-20T19:11:05.072869] [DEBUG] [LOCAL ] HII: Saving primary data. [2025-10-20T19:11:05.074667] [DEBUG] [LOCAL ] HII: Trimmed 52464 records to 33610 (keeping 200 days of data) [2025-10-20T19:11:05.198933] [DEBUG] [LOCAL ] HIX: Loaded 20696 existing records from primary storage. [2025-10-20T19:11:05.213196] [INFO ] [MAIN ] HIX: Loaded 20696 existing records. [2025-10-20T19:11:05.456628] [DEBUG] [LOCAL ] HII: Primary data saved (33610 records). [2025-10-20T19:11:05.472963] [INFO ] [MAIN ] HII: Full history saved. [2025-10-20T19:11:05.473256] [INFO ] [MAIN ] Update Progress: 1019/2367 symbols completed. HII took 38.1s. Estimated remaining: 2h 53m 19s. Rate: 7.8 symbols/min [2025-10-20T19:11:05.477756] [WARN ] [MAIN ] HIX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:05.477918] [INFO ] [MAIN ] HIX: Wiping existing primary data before full refresh. [2025-10-20T19:11:05.477986] [INFO ] [LOCAL ] HIX: Starting deletion of primary data. [2025-10-20T19:11:05.509945] [DEBUG] [LOCAL ] HIX: Deleted primary data file. [2025-10-20T19:11:05.521451] [INFO ] [MAIN ] HIX: Starting full primary refresh. [2025-10-20T19:11:05.776086] [INFO ] [MAIN ] HKD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:05.959483] [DEBUG] [LOCAL ] HKD: Loaded 18952 existing records from primary storage. [2025-10-20T19:11:06.004238] [INFO ] [MAIN ] HKD: Loaded 18952 existing records. [2025-10-20T19:11:06.033071] [INFO ] [REMOTE ] HIX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:06.033433] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-01. [2025-10-20T19:11:06.183237] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:11:06.190194] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T19:11:06.260097] [WARN ] [MAIN ] HKD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:06.260273] [INFO ] [MAIN ] HKD: Wiping existing primary data before full refresh. [2025-10-20T19:11:06.260320] [INFO ] [LOCAL ] HKD: Starting deletion of primary data. [2025-10-20T19:11:06.260477] [DEBUG] [LOCAL ] HKD: Deleted primary data file. [2025-10-20T19:11:06.260513] [INFO ] [MAIN ] HKD: Starting full primary refresh. [2025-10-20T19:11:06.607323] [INFO ] [REMOTE ] HKD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:06.651364] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-02. [2025-10-20T19:11:07.224870] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:11:07.451374] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-01. [2025-10-20T19:11:08.218224] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:11:08.251328] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-03. [2025-10-20T19:11:08.659108] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:11:09.051392] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-02. [2025-10-20T19:11:09.749203] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:11:09.851377] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-04. [2025-10-20T19:11:09.991977] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T19:11:10.651384] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-06. [2025-10-20T19:11:11.460263] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-02. [2025-10-20T19:11:11.463056] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:11:11.848094] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T19:11:12.251342] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-03. [2025-10-20T19:11:12.344078] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T19:11:13.051391] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-05. [2025-10-20T19:11:13.276171] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:11:13.851387] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-04. [2025-10-20T19:11:14.490557] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:11:14.651351] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-07. [2025-10-20T19:11:15.050953] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:11:15.451384] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-05. [2025-10-20T19:11:16.154869] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:11:16.251385] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-03. [2025-10-20T19:11:17.051414] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-08. [2025-10-20T19:11:17.851430] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-06. [2025-10-20T19:11:18.023126] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-05 (2.57 seconds). [2025-10-20T19:11:18.172180] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:11:18.452545] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-10-20T19:11:18.651377] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-04. [2025-10-20T19:11:19.321899] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:11:19.374676] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T19:11:19.451379] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-06. [2025-10-20T19:11:20.251382] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-07. [2025-10-20T19:11:20.429123] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:11:21.051397] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-05. [2025-10-20T19:11:21.772954] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:11:21.809702] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:11:21.851343] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-06. [2025-10-20T19:11:22.651458] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-09. [2025-10-20T19:11:23.140291] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:11:23.451366] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-07. [2025-10-20T19:11:23.670628] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:11:24.251402] [INFO ] [REMOTE ] HIPO: Fetching primary data for 2025-10. [2025-10-20T19:11:24.314451] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T19:11:25.051369] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-08. [2025-10-20T19:11:25.291716] [INFO ] [REMOTE ] HIPO: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:11:25.294487] [INFO ] [REMOTE ] HIPO: Full fetch complete (31089 records). [2025-10-20T19:11:25.294560] [DEBUG] [VALIDATOR ] HIPO: Validating primary data. [2025-10-20T19:11:25.689882] [DEBUG] [VALIDATOR ] HIPO: All expected trading days are present. [2025-10-20T19:11:25.689986] [DEBUG] [VALIDATOR ] HIPO: Primary data validation passed. [2025-10-20T19:11:25.690570] [INFO ] [MAIN ] HIPO: Fetched primary data is valid. [2025-10-20T19:11:25.690610] [DEBUG] [LOCAL ] HIPO: Saving primary data. [2025-10-20T19:11:25.691299] [DEBUG] [LOCAL ] HIPO: Trimmed 31089 records to 21623 (keeping 200 days of data) [2025-10-20T19:11:25.811969] [DEBUG] [LOCAL ] HIPO: Primary data saved (21623 records). [2025-10-20T19:11:25.812090] [INFO ] [MAIN ] HIPO: Full history saved. [2025-10-20T19:11:25.812164] [INFO ] [MAIN ] Update Progress: 1020/2367 symbols completed. HIPO took 29.1s. Estimated remaining: 2h 53m 28s. Rate: 7.8 symbols/min [2025-10-20T19:11:25.856612] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-08. [2025-10-20T19:11:26.002788] [INFO ] [MAIN ] HL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:26.051986] [DEBUG] [LOCAL ] HL: Loaded 54455 existing records from primary storage. [2025-10-20T19:11:26.052349] [INFO ] [MAIN ] HL: Loaded 54455 existing records. [2025-10-20T19:11:26.228902] [WARN ] [MAIN ] HL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:26.229110] [INFO ] [MAIN ] HL: Wiping existing primary data before full refresh. [2025-10-20T19:11:26.229156] [INFO ] [LOCAL ] HL: Starting deletion of primary data. [2025-10-20T19:11:26.229376] [DEBUG] [LOCAL ] HL: Deleted primary data file. [2025-10-20T19:11:26.229431] [INFO ] [MAIN ] HL: Starting full primary refresh. [2025-10-20T19:11:26.592308] [INFO ] [REMOTE ] HL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:26.651372] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-07. [2025-10-20T19:11:26.725685] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:11:26.876098] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:11:27.356149] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:11:27.451370] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-09. [2025-10-20T19:11:28.251342] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-01. [2025-10-20T19:11:28.778636] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T19:11:29.051364] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-09. [2025-10-20T19:11:29.851370] [INFO ] [REMOTE ] HKD: Fetching primary data for 2025-10. [2025-10-20T19:11:30.001810] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:11:30.651403] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-02. [2025-10-20T19:11:30.856140] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:11:30.954686] [INFO ] [REMOTE ] HKD: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T19:11:30.958187] [INFO ] [REMOTE ] HKD: Full fetch complete (28019 records). [2025-10-20T19:11:30.958269] [DEBUG] [VALIDATOR ] HKD: Validating primary data. [2025-10-20T19:11:31.344348] [DEBUG] [VALIDATOR ] HKD: All expected trading days are present. [2025-10-20T19:11:31.344447] [DEBUG] [VALIDATOR ] HKD: Primary data validation passed. [2025-10-20T19:11:31.345096] [INFO ] [MAIN ] HKD: Fetched primary data is valid. [2025-10-20T19:11:31.345158] [DEBUG] [LOCAL ] HKD: Saving primary data. [2025-10-20T19:11:31.345922] [DEBUG] [LOCAL ] HKD: Trimmed 28019 records to 18743 (keeping 200 days of data) [2025-10-20T19:11:31.448780] [DEBUG] [LOCAL ] HKD: Primary data saved (18743 records). [2025-10-20T19:11:31.448940] [INFO ] [MAIN ] HKD: Full history saved. [2025-10-20T19:11:31.449028] [INFO ] [MAIN ] Update Progress: 1021/2367 symbols completed. HKD took 26.0s. Estimated remaining: 2h 53m 18s. Rate: 7.8 symbols/min [2025-10-20T19:11:31.460452] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-08. [2025-10-20T19:11:31.631821] [INFO ] [MAIN ] HLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:31.676522] [DEBUG] [LOCAL ] HLF: Loaded 49226 existing records from primary storage. [2025-10-20T19:11:31.676853] [INFO ] [MAIN ] HLF: Loaded 49226 existing records. [2025-10-20T19:11:31.855129] [WARN ] [MAIN ] HLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:31.855368] [INFO ] [MAIN ] HLF: Wiping existing primary data before full refresh. [2025-10-20T19:11:31.855415] [INFO ] [LOCAL ] HLF: Starting deletion of primary data. [2025-10-20T19:11:31.855610] [DEBUG] [LOCAL ] HLF: Deleted primary data file. [2025-10-20T19:11:31.855696] [INFO ] [MAIN ] HLF: Starting full primary refresh. [2025-10-20T19:11:32.231501] [INFO ] [REMOTE ] HLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:32.251341] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-01. [2025-10-20T19:11:33.051387] [INFO ] [REMOTE ] HIW: Fetching primary data for 2025-10. [2025-10-20T19:11:33.444406] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-02 (2.79 seconds). [2025-10-20T19:11:33.669241] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T19:11:33.851372] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-03. [2025-10-20T19:11:34.033048] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:11:34.651349] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-09. [2025-10-20T19:11:34.734788] [INFO ] [REMOTE ] HIW: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:11:34.742834] [INFO ] [REMOTE ] HIW: Full fetch complete (64864 records). [2025-10-20T19:11:34.743014] [DEBUG] [VALIDATOR ] HIW: Validating primary data. [2025-10-20T19:11:35.158799] [DEBUG] [VALIDATOR ] HIW: All expected trading days are present. [2025-10-20T19:11:35.158900] [DEBUG] [VALIDATOR ] HIW: Primary data validation passed. [2025-10-20T19:11:35.159769] [INFO ] [MAIN ] HIW: Fetched primary data is valid. [2025-10-20T19:11:35.159831] [DEBUG] [LOCAL ] HIW: Saving primary data. [2025-10-20T19:11:35.161061] [DEBUG] [LOCAL ] HIW: Trimmed 64864 records to 44503 (keeping 200 days of data) [2025-10-20T19:11:35.426153] [DEBUG] [LOCAL ] HIW: Primary data saved (44503 records). [2025-10-20T19:11:35.426321] [INFO ] [MAIN ] HIW: Full history saved. [2025-10-20T19:11:35.426800] [INFO ] [MAIN ] Update Progress: 1022/2367 symbols completed. HIW took 32.6s. Estimated remaining: 2h 53m 5s. Rate: 7.8 symbols/min [2025-10-20T19:11:35.458595] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-02. [2025-10-20T19:11:35.606748] [INFO ] [MAIN ] HLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:35.641846] [DEBUG] [LOCAL ] HLI: Loaded 31316 existing records from primary storage. [2025-10-20T19:11:35.642111] [INFO ] [MAIN ] HLI: Loaded 31316 existing records. [2025-10-20T19:11:35.848172] [WARN ] [MAIN ] HLI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:35.834581] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:11:35.853022] [INFO ] [MAIN ] HLI: Wiping existing primary data before full refresh. [2025-10-20T19:11:35.853247] [INFO ] [LOCAL ] HLI: Starting deletion of primary data. [2025-10-20T19:11:35.853899] [DEBUG] [LOCAL ] HLI: Deleted primary data file. [2025-10-20T19:11:35.854074] [INFO ] [MAIN ] HLI: Starting full primary refresh. [2025-10-20T19:11:36.186362] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-03 (2.33 seconds). [2025-10-20T19:11:36.252252] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-04. [2025-10-20T19:11:36.266160] [INFO ] [REMOTE ] HLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:37.051365] [INFO ] [REMOTE ] HIX: Fetching primary data for 2025-10. [2025-10-20T19:11:37.162339] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:11:37.851376] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-03. [2025-10-20T19:11:37.956921] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:11:38.211449] [INFO ] [REMOTE ] HIX: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:11:38.214500] [INFO ] [REMOTE ] HIX: Full fetch complete (29871 records). [2025-10-20T19:11:38.214577] [DEBUG] [VALIDATOR ] HIX: Validating primary data. [2025-10-20T19:11:38.582692] [DEBUG] [VALIDATOR ] HIX: All expected trading days are present. [2025-10-20T19:11:38.582772] [DEBUG] [VALIDATOR ] HIX: Primary data validation passed. [2025-10-20T19:11:38.583341] [INFO ] [MAIN ] HIX: Fetched primary data is valid. [2025-10-20T19:11:38.583382] [DEBUG] [LOCAL ] HIX: Saving primary data. [2025-10-20T19:11:38.584144] [DEBUG] [LOCAL ] HIX: Trimmed 29871 records to 20398 (keeping 200 days of data) [2025-10-20T19:11:38.664395] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-01. [2025-10-20T19:11:38.703542] [DEBUG] [LOCAL ] HIX: Primary data saved (20398 records). [2025-10-20T19:11:38.703716] [INFO ] [MAIN ] HIX: Full history saved. [2025-10-20T19:11:38.703805] [INFO ] [MAIN ] Update Progress: 1023/2367 symbols completed. HIX took 33.9s. Estimated remaining: 2h 52m 51s. Rate: 7.8 symbols/min [2025-10-20T19:11:38.876281] [INFO ] [MAIN ] HLIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:11:38.904700] [DEBUG] [LOCAL ] HLIO: Loaded 28680 existing records from primary storage. [2025-10-20T19:11:38.904954] [INFO ] [MAIN ] HLIO: Loaded 28680 existing records. [2025-10-20T19:11:39.083578] [WARN ] [MAIN ] HLIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:11:39.083925] [INFO ] [MAIN ] HLIO: Wiping existing primary data before full refresh. [2025-10-20T19:11:39.083984] [INFO ] [LOCAL ] HLIO: Starting deletion of primary data. [2025-10-20T19:11:39.084151] [DEBUG] [LOCAL ] HLIO: Deleted primary data file. [2025-10-20T19:11:39.084190] [INFO ] [MAIN ] HLIO: Starting full primary refresh. [2025-10-20T19:11:39.450801] [INFO ] [REMOTE ] HLIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:11:39.451338] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-01. [2025-10-20T19:11:39.482157] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:11:40.034745] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:11:40.251425] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-05. [2025-10-20T19:11:40.454899] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:11:41.051385] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-04. [2025-10-20T19:11:41.851383] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-02. [2025-10-20T19:11:42.162663] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:11:42.651435] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-02. [2025-10-20T19:11:42.739099] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:11:42.804545] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T19:11:43.451360] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-05. [2025-10-20T19:11:43.624854] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T19:11:44.251424] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-03. [2025-10-20T19:11:45.051404] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-03. [2025-10-20T19:11:45.144628] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:11:45.662778] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T19:11:45.784686] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:11:45.851418] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-04. [2025-10-20T19:11:46.651395] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-06. [2025-10-20T19:11:47.079130] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:11:47.451379] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-05. [2025-10-20T19:11:48.251407] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-06. [2025-10-20T19:11:48.296621] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:11:48.790805] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:11:49.051409] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-04. [2025-10-20T19:11:49.851352] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-06. [2025-10-20T19:11:49.966853] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T19:11:50.354710] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:11:50.651427] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-07. [2025-10-20T19:11:50.710062] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:11:51.451510] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-07. [2025-10-20T19:11:52.251422] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-07. [2025-10-20T19:11:52.656339] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T19:11:52.868033] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:11:53.051369] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-08. [2025-10-20T19:11:53.851352] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-05. [2025-10-20T19:11:53.948745] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:11:54.651453] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-08. [2025-10-20T19:11:55.209905] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-10-20T19:11:55.211609] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T19:11:55.451384] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-06. [2025-10-20T19:11:56.005148] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:11:56.251406] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-09. [2025-10-20T19:11:56.792680] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T19:11:57.051486] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-09. [2025-10-20T19:11:57.851381] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-08. [2025-10-20T19:11:58.514435] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T19:11:58.651361] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-07. [2025-10-20T19:11:58.817096] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:11:59.202304] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:11:59.451425] [INFO ] [REMOTE ] HLF: Fetching primary data for 2025-10. [2025-10-20T19:11:59.993313] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:12:00.251422] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-08. [2025-10-20T19:12:01.051359] [INFO ] [REMOTE ] HL: Fetching primary data for 2025-10. [2025-10-20T19:12:01.394411] [INFO ] [REMOTE ] HLF: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:12:01.401592] [INFO ] [REMOTE ] HLF: Full fetch complete (70919 records). [2025-10-20T19:12:01.401873] [DEBUG] [VALIDATOR ] HLF: Validating primary data. [2025-10-20T19:12:01.552187] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:12:01.848260] [DEBUG] [VALIDATOR ] HLF: All expected trading days are present. [2025-10-20T19:12:01.848347] [DEBUG] [VALIDATOR ] HLF: Primary data validation passed. [2025-10-20T19:12:01.849345] [INFO ] [MAIN ] HLF: Fetched primary data is valid. [2025-10-20T19:12:01.849411] [DEBUG] [LOCAL ] HLF: Saving primary data. [2025-10-20T19:12:01.850895] [DEBUG] [LOCAL ] HLF: Trimmed 70919 records to 48540 (keeping 200 days of data) [2025-10-20T19:12:01.851465] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-09. [2025-10-20T19:12:02.163250] [DEBUG] [LOCAL ] HLF: Primary data saved (48540 records). [2025-10-20T19:12:02.163484] [INFO ] [MAIN ] HLF: Full history saved. [2025-10-20T19:12:02.164303] [INFO ] [MAIN ] Update Progress: 1024/2367 symbols completed. HLF took 30.7s. Estimated remaining: 2h 53m 4s. Rate: 7.8 symbols/min [2025-10-20T19:12:02.331634] [INFO ] [MAIN ] HLLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:02.364879] [DEBUG] [LOCAL ] HLLY: Loaded 33825 existing records from primary storage. [2025-10-20T19:12:02.365143] [INFO ] [MAIN ] HLLY: Loaded 33825 existing records. [2025-10-20T19:12:02.539386] [WARN ] [MAIN ] HLLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:02.539639] [INFO ] [MAIN ] HLLY: Wiping existing primary data before full refresh. [2025-10-20T19:12:02.539720] [INFO ] [LOCAL ] HLLY: Starting deletion of primary data. [2025-10-20T19:12:02.539929] [DEBUG] [LOCAL ] HLLY: Deleted primary data file. [2025-10-20T19:12:02.539986] [INFO ] [MAIN ] HLLY: Starting full primary refresh. [2025-10-20T19:12:02.656519] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-09. [2025-10-20T19:12:02.908860] [INFO ] [REMOTE ] HLLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:03.001504] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:12:03.451411] [INFO ] [REMOTE ] HLIO: Fetching primary data for 2025-10. [2025-10-20T19:12:03.476164] [INFO ] [REMOTE ] HL: Incremental fetch complete for 2025-10 (2.42 seconds). [2025-10-20T19:12:03.484342] [INFO ] [REMOTE ] HL: Full fetch complete (77825 records). [2025-10-20T19:12:03.484473] [DEBUG] [VALIDATOR ] HL: Validating primary data. [2025-10-20T19:12:03.878575] [DEBUG] [VALIDATOR ] HL: All expected trading days are present. [2025-10-20T19:12:03.878709] [DEBUG] [VALIDATOR ] HL: Primary data validation passed. [2025-10-20T19:12:03.879932] [INFO ] [MAIN ] HL: Fetched primary data is valid. [2025-10-20T19:12:03.879997] [DEBUG] [LOCAL ] HL: Saving primary data. [2025-10-20T19:12:03.881443] [DEBUG] [LOCAL ] HL: Trimmed 77825 records to 53676 (keeping 200 days of data) [2025-10-20T19:12:04.103598] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:12:04.230411] [DEBUG] [LOCAL ] HL: Primary data saved (53676 records). [2025-10-20T19:12:04.230566] [INFO ] [MAIN ] HL: Full history saved. [2025-10-20T19:12:04.230671] [INFO ] [MAIN ] Update Progress: 1025/2367 symbols completed. HL took 38.4s. Estimated remaining: 2h 52m 49s. Rate: 7.8 symbols/min [2025-10-20T19:12:04.259965] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-01. [2025-10-20T19:12:04.401713] [INFO ] [MAIN ] HLN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:04.451677] [DEBUG] [LOCAL ] HLN: Loaded 53902 existing records from primary storage. [2025-10-20T19:12:04.451994] [INFO ] [MAIN ] HLN: Loaded 53902 existing records. [2025-10-20T19:12:04.636334] [WARN ] [MAIN ] HLN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:04.636589] [INFO ] [MAIN ] HLN: Wiping existing primary data before full refresh. [2025-10-20T19:12:04.636768] [INFO ] [LOCAL ] HLN: Starting deletion of primary data. [2025-10-20T19:12:04.637374] [DEBUG] [LOCAL ] HLN: Deleted primary data file. [2025-10-20T19:12:04.637427] [INFO ] [MAIN ] HLN: Starting full primary refresh. [2025-10-20T19:12:04.762000] [INFO ] [REMOTE ] HLIO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:12:04.794373] [INFO ] [REMOTE ] HLIO: Full fetch complete (37110 records). [2025-10-20T19:12:04.805227] [DEBUG] [VALIDATOR ] HLIO: Validating primary data. [2025-10-20T19:12:05.052079] [INFO ] [REMOTE ] HLI: Fetching primary data for 2025-10. [2025-10-20T19:12:05.260767] [INFO ] [REMOTE ] HLN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:05.420805] [DEBUG] [VALIDATOR ] HLIO: All expected trading days are present. [2025-10-20T19:12:05.420925] [DEBUG] [VALIDATOR ] HLIO: Primary data validation passed. [2025-10-20T19:12:05.421769] [INFO ] [MAIN ] HLIO: Fetched primary data is valid. [2025-10-20T19:12:05.421841] [DEBUG] [LOCAL ] HLIO: Saving primary data. [2025-10-20T19:12:05.423022] [DEBUG] [LOCAL ] HLIO: Trimmed 37110 records to 28259 (keeping 200 days of data) [2025-10-20T19:12:05.605604] [DEBUG] [LOCAL ] HLIO: Primary data saved (28259 records). [2025-10-20T19:12:05.605800] [INFO ] [MAIN ] HLIO: Full history saved. [2025-10-20T19:12:05.605987] [INFO ] [MAIN ] Update Progress: 1026/2367 symbols completed. HLIO took 26.9s. Estimated remaining: 2h 52m 33s. Rate: 7.8 symbols/min [2025-10-20T19:12:05.769667] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T19:12:05.819723] [INFO ] [MAIN ] HLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:05.851340] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-02. [2025-10-20T19:12:05.874992] [DEBUG] [LOCAL ] HLT: Loaded 51246 existing records from primary storage. [2025-10-20T19:12:05.875356] [INFO ] [MAIN ] HLT: Loaded 51246 existing records. [2025-10-20T19:12:06.061367] [WARN ] [MAIN ] HLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:06.061576] [INFO ] [MAIN ] HLT: Wiping existing primary data before full refresh. [2025-10-20T19:12:06.061623] [INFO ] [LOCAL ] HLT: Starting deletion of primary data. [2025-10-20T19:12:06.061887] [DEBUG] [LOCAL ] HLT: Deleted primary data file. [2025-10-20T19:12:06.061941] [INFO ] [MAIN ] HLT: Starting full primary refresh. [2025-10-20T19:12:06.407074] [INFO ] [REMOTE ] HLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:06.651360] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-01. [2025-10-20T19:12:06.697241] [INFO ] [REMOTE ] HLI: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T19:12:06.701839] [INFO ] [REMOTE ] HLI: Full fetch complete (45378 records). [2025-10-20T19:12:06.701927] [DEBUG] [VALIDATOR ] HLI: Validating primary data. [2025-10-20T19:12:07.104693] [DEBUG] [VALIDATOR ] HLI: All expected trading days are present. [2025-10-20T19:12:07.112191] [DEBUG] [VALIDATOR ] HLI: Primary data validation passed. [2025-10-20T19:12:07.113033] [INFO ] [MAIN ] HLI: Fetched primary data is valid. [2025-10-20T19:12:07.113114] [DEBUG] [LOCAL ] HLI: Saving primary data. [2025-10-20T19:12:07.114307] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T19:12:07.120372] [DEBUG] [LOCAL ] HLI: Trimmed 45378 records to 30785 (keeping 200 days of data) [2025-10-20T19:12:07.307367] [DEBUG] [LOCAL ] HLI: Primary data saved (30785 records). [2025-10-20T19:12:07.307492] [INFO ] [MAIN ] HLI: Full history saved. [2025-10-20T19:12:07.307829] [INFO ] [MAIN ] Update Progress: 1027/2367 symbols completed. HLI took 31.9s. Estimated remaining: 2h 52m 18s. Rate: 7.8 symbols/min [2025-10-20T19:12:07.459758] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-03. [2025-10-20T19:12:07.484283] [INFO ] [MAIN ] HLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:07.528978] [DEBUG] [LOCAL ] HLX: Loaded 48172 existing records from primary storage. [2025-10-20T19:12:07.529312] [INFO ] [MAIN ] HLX: Loaded 48172 existing records. [2025-10-20T19:12:07.710269] [WARN ] [MAIN ] HLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:07.710496] [INFO ] [MAIN ] HLX: Wiping existing primary data before full refresh. [2025-10-20T19:12:07.710546] [INFO ] [LOCAL ] HLX: Starting deletion of primary data. [2025-10-20T19:12:07.710776] [DEBUG] [LOCAL ] HLX: Deleted primary data file. [2025-10-20T19:12:07.710832] [INFO ] [MAIN ] HLX: Starting full primary refresh. [2025-10-20T19:12:08.069005] [INFO ] [REMOTE ] HLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:08.251346] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-01. [2025-10-20T19:12:08.431334] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:12:08.796932] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T19:12:09.051416] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-02. [2025-10-20T19:12:09.637464] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T19:12:09.851377] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-02. [2025-10-20T19:12:10.651382] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-04. [2025-10-20T19:12:10.707622] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:12:11.374104] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:12:11.451380] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-03. [2025-10-20T19:12:12.138964] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:12:12.251387] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-03. [2025-10-20T19:12:13.051436] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-05. [2025-10-20T19:12:13.190556] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:12:13.851378] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-01. [2025-10-20T19:12:14.187133] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T19:12:14.438951] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:12:14.651358] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-04. [2025-10-20T19:12:15.451420] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-04. [2025-10-20T19:12:15.559468] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:12:16.251415] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-02. [2025-10-20T19:12:16.539930] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T19:12:17.051361] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-05. [2025-10-20T19:12:17.168932] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:12:17.851375] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-05. [2025-10-20T19:12:18.040119] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:12:18.651346] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-06. [2025-10-20T19:12:18.765222] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:12:19.409474] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:12:19.451346] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-06. [2025-10-20T19:12:19.517124] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T19:12:20.251389] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-06. [2025-10-20T19:12:21.019300] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:12:21.051341] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-07. [2025-10-20T19:12:21.851368] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-07. [2025-10-20T19:12:21.956380] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:12:22.651349] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-07. [2025-10-20T19:12:22.808935] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:12:23.193717] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:12:23.451389] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-08. [2025-10-20T19:12:24.251350] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-08. [2025-10-20T19:12:24.437598] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:12:25.051418] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-08. [2025-10-20T19:12:25.182780] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T19:12:25.682227] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:12:25.851363] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-09. [2025-10-20T19:12:26.644632] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:12:26.656435] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-09. [2025-10-20T19:12:27.008167] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T19:12:27.451427] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-09. [2025-10-20T19:12:28.111309] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:12:28.251388] [INFO ] [REMOTE ] HLLY: Fetching primary data for 2025-10. [2025-10-20T19:12:29.051460] [INFO ] [REMOTE ] HLX: Fetching primary data for 2025-10. [2025-10-20T19:12:29.261874] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:12:29.606246] [INFO ] [REMOTE ] HLLY: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:12:29.611356] [INFO ] [REMOTE ] HLLY: Full fetch complete (46082 records). [2025-10-20T19:12:29.611515] [DEBUG] [VALIDATOR ] HLLY: Validating primary data. [2025-10-20T19:12:29.861218] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-03. [2025-10-20T19:12:30.011413] [DEBUG] [VALIDATOR ] HLLY: All expected trading days are present. [2025-10-20T19:12:30.011525] [DEBUG] [VALIDATOR ] HLLY: Primary data validation passed. [2025-10-20T19:12:30.012249] [INFO ] [MAIN ] HLLY: Fetched primary data is valid. [2025-10-20T19:12:30.012310] [DEBUG] [LOCAL ] HLLY: Saving primary data. [2025-10-20T19:12:30.013268] [DEBUG] [LOCAL ] HLLY: Trimmed 46082 records to 33339 (keeping 200 days of data) [2025-10-20T19:12:30.190729] [DEBUG] [LOCAL ] HLLY: Primary data saved (33339 records). [2025-10-20T19:12:30.190858] [INFO ] [MAIN ] HLLY: Full history saved. [2025-10-20T19:12:30.190934] [INFO ] [MAIN ] Update Progress: 1028/2367 symbols completed. HLLY took 28.0s. Estimated remaining: 2h 52m 30s. Rate: 7.8 symbols/min [2025-10-20T19:12:30.362981] [INFO ] [MAIN ] HMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:30.405978] [DEBUG] [LOCAL ] HMC: Loaded 47137 existing records from primary storage. [2025-10-20T19:12:30.406304] [INFO ] [MAIN ] HMC: Loaded 47137 existing records. [2025-10-20T19:12:30.583681] [WARN ] [MAIN ] HMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:30.584014] [INFO ] [MAIN ] HMC: Wiping existing primary data before full refresh. [2025-10-20T19:12:30.584191] [INFO ] [LOCAL ] HMC: Starting deletion of primary data. [2025-10-20T19:12:30.584523] [DEBUG] [LOCAL ] HMC: Deleted primary data file. [2025-10-20T19:12:30.584582] [INFO ] [MAIN ] HMC: Starting full primary refresh. [2025-10-20T19:12:30.657775] [INFO ] [REMOTE ] HLN: Fetching primary data for 2025-10. [2025-10-20T19:12:30.852456] [INFO ] [REMOTE ] HLX: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:12:30.898557] [INFO ] [REMOTE ] HLX: Full fetch complete (69053 records). [2025-10-20T19:12:30.904299] [DEBUG] [VALIDATOR ] HLX: Validating primary data. [2025-10-20T19:12:31.099849] [INFO ] [REMOTE ] HMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:31.438432] [DEBUG] [VALIDATOR ] HLX: All expected trading days are present. [2025-10-20T19:12:31.438533] [DEBUG] [VALIDATOR ] HLX: Primary data validation passed. [2025-10-20T19:12:31.439554] [INFO ] [MAIN ] HLX: Fetched primary data is valid. [2025-10-20T19:12:31.439625] [DEBUG] [LOCAL ] HLX: Saving primary data. [2025-10-20T19:12:31.441127] [DEBUG] [LOCAL ] HLX: Trimmed 69053 records to 47458 (keeping 200 days of data) [2025-10-20T19:12:31.485760] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-01. [2025-10-20T19:12:31.735329] [DEBUG] [LOCAL ] HLX: Primary data saved (47458 records). [2025-10-20T19:12:31.735518] [INFO ] [MAIN ] HLX: Full history saved. [2025-10-20T19:12:31.736186] [INFO ] [MAIN ] Update Progress: 1029/2367 symbols completed. HLX took 24.4s. Estimated remaining: 2h 52m 14s. Rate: 7.8 symbols/min [2025-10-20T19:12:31.987260] [INFO ] [MAIN ] HMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:31.985627] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-10-20T19:12:32.022220] [DEBUG] [LOCAL ] HMN: Loaded 23749 existing records from primary storage. [2025-10-20T19:12:32.022496] [INFO ] [MAIN ] HMN: Loaded 23749 existing records. [2025-10-20T19:12:32.196255] [WARN ] [MAIN ] HMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:32.196481] [INFO ] [MAIN ] HMN: Wiping existing primary data before full refresh. [2025-10-20T19:12:32.196527] [INFO ] [LOCAL ] HMN: Starting deletion of primary data. [2025-10-20T19:12:32.196787] [DEBUG] [LOCAL ] HMN: Deleted primary data file. [2025-10-20T19:12:32.196844] [INFO ] [MAIN ] HMN: Starting full primary refresh. [2025-10-20T19:12:32.258967] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-04. [2025-10-20T19:12:32.600293] [INFO ] [REMOTE ] HMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:32.601157] [INFO ] [REMOTE ] HLN: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:12:32.610039] [INFO ] [REMOTE ] HLN: Full fetch complete (77035 records). [2025-10-20T19:12:32.610192] [DEBUG] [VALIDATOR ] HLN: Validating primary data. [2025-10-20T19:12:33.012343] [DEBUG] [VALIDATOR ] HLN: All expected trading days are present. [2025-10-20T19:12:33.012477] [DEBUG] [VALIDATOR ] HLN: Primary data validation passed. [2025-10-20T19:12:33.013438] [INFO ] [MAIN ] HLN: Fetched primary data is valid. [2025-10-20T19:12:33.013585] [DEBUG] [LOCAL ] HLN: Saving primary data. [2025-10-20T19:12:33.015302] [DEBUG] [LOCAL ] HLN: Trimmed 77035 records to 53136 (keeping 200 days of data) [2025-10-20T19:12:33.066739] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-01. [2025-10-20T19:12:33.165198] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:12:33.365795] [DEBUG] [LOCAL ] HLN: Primary data saved (53136 records). [2025-10-20T19:12:33.365923] [INFO ] [MAIN ] HLN: Full history saved. [2025-10-20T19:12:33.366000] [INFO ] [MAIN ] Update Progress: 1030/2367 symbols completed. HLN took 29.1s. Estimated remaining: 2h 51m 58s. Rate: 7.8 symbols/min [2025-10-20T19:12:33.531017] [INFO ] [MAIN ] HMY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:33.580127] [DEBUG] [LOCAL ] HMY: Loaded 53946 existing records from primary storage. [2025-10-20T19:12:33.580432] [INFO ] [MAIN ] HMY: Loaded 53946 existing records. [2025-10-20T19:12:33.753088] [WARN ] [MAIN ] HMY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:33.753302] [INFO ] [MAIN ] HMY: Wiping existing primary data before full refresh. [2025-10-20T19:12:33.753347] [INFO ] [LOCAL ] HMY: Starting deletion of primary data. [2025-10-20T19:12:33.753742] [DEBUG] [LOCAL ] HMY: Deleted primary data file. [2025-10-20T19:12:33.753797] [INFO ] [MAIN ] HMY: Starting full primary refresh. [2025-10-20T19:12:33.853119] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-02. [2025-10-20T19:12:34.114243] [INFO ] [REMOTE ] HMY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:34.245727] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:12:34.267629] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:12:34.651346] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-05. [2025-10-20T19:12:35.451357] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-01. [2025-10-20T19:12:35.593595] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:12:36.084822] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:12:36.251383] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-02. [2025-10-20T19:12:37.051382] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-06. [2025-10-20T19:12:37.277223] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:12:37.453690] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:12:37.851425] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-03. [2025-10-20T19:12:38.651430] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-03. [2025-10-20T19:12:38.723917] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:12:38.886729] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:12:39.451377] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-04. [2025-10-20T19:12:40.202328] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:12:40.251344] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-07. [2025-10-20T19:12:40.653125] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:12:41.051370] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-05. [2025-10-20T19:12:41.851389] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-04. [2025-10-20T19:12:42.080890] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T19:12:42.109923] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T19:12:42.651400] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-08. [2025-10-20T19:12:43.451365] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-06. [2025-10-20T19:12:43.736133] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:12:44.061725] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:12:44.251366] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-09. [2025-10-20T19:12:44.580488] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:12:45.051457] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-05. [2025-10-20T19:12:45.852875] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-02. [2025-10-20T19:12:45.855514] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T19:12:46.651348] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-07. [2025-10-20T19:12:46.749815] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:12:47.451355] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-06. [2025-10-20T19:12:47.748904] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:12:47.935908] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:12:48.251438] [INFO ] [REMOTE ] HLT: Fetching primary data for 2025-10. [2025-10-20T19:12:49.068530] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-08. [2025-10-20T19:12:49.099159] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:12:49.141813] [INFO ] [REMOTE ] HLT: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:12:49.150024] [INFO ] [REMOTE ] HLT: Full fetch complete (73617 records). [2025-10-20T19:12:49.150198] [DEBUG] [VALIDATOR ] HLT: Validating primary data. [2025-10-20T19:12:49.543106] [DEBUG] [VALIDATOR ] HLT: All expected trading days are present. [2025-10-20T19:12:49.543197] [DEBUG] [VALIDATOR ] HLT: Primary data validation passed. [2025-10-20T19:12:49.544125] [INFO ] [MAIN ] HLT: Fetched primary data is valid. [2025-10-20T19:12:49.544193] [DEBUG] [LOCAL ] HLT: Saving primary data. [2025-10-20T19:12:49.545713] [DEBUG] [LOCAL ] HLT: Trimmed 73617 records to 50463 (keeping 200 days of data) [2025-10-20T19:12:49.858961] [DEBUG] [LOCAL ] HLT: Primary data saved (50463 records). [2025-10-20T19:12:49.859119] [INFO ] [MAIN ] HLT: Full history saved. [2025-10-20T19:12:49.855031] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-03. [2025-10-20T19:12:49.859236] [INFO ] [MAIN ] Update Progress: 1031/2367 symbols completed. HLT took 44.3s. Estimated remaining: 2h 52m 2s. Rate: 7.8 symbols/min [2025-10-20T19:12:50.030145] [INFO ] [MAIN ] HNGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:50.030380] [DEBUG] [LOCAL ] HNGE: No existing data file found in primary storage. [2025-10-20T19:12:50.030424] [WARN ] [MAIN ] HNGE: No existing data found, performing full refresh instead. [2025-10-20T19:12:50.030528] [INFO ] [MAIN ] HNGE: Wiping existing primary data before full refresh. [2025-10-20T19:12:50.030569] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-10-20T19:12:50.030609] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-10-20T19:12:50.030682] [INFO ] [MAIN ] HNGE: Starting full primary refresh. [2025-10-20T19:12:50.322542] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T19:12:50.418760] [INFO ] [REMOTE ] HNGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:50.651394] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-09. [2025-10-20T19:12:51.451358] [INFO ] [REMOTE ] HNGE: Fetching primary data for 2025-01. [2025-10-20T19:12:51.642050] [DEBUG] [REMOTE ] HNGE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:12:51.642416] [WARN ] [REMOTE ] HNGE: No data returned for month 2025-01 [2025-10-20T19:12:51.642517] [FATAL] [REMOTE ] HNGE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:12:51.642568] [FATAL] [MAIN ] HNGE: Failed to fetch primary data. [2025-10-20T19:12:51.642632] [WARN ] [MAIN ] HNGE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:12:51.642725] [INFO ] [LOCAL ] HNGE: Starting deletion of primary data. [2025-10-20T19:12:51.642788] [DEBUG] [LOCAL ] HNGE: No primary data file found to delete. [2025-10-20T19:12:51.642823] [INFO ] [LOCAL ] HNGE: Starting deletion of derivative data. [2025-10-20T19:12:51.642862] [DEBUG] [LOCAL ] HNGE: No derivative data file found to delete. [2025-10-20T19:12:51.642907] [INFO ] [MAIN ] Update Progress: 1032/2367 symbols completed. HNGE took 1.8s. Estimated remaining: 2h 51m 46s. Rate: 7.8 symbols/min [2025-10-20T19:12:51.764830] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:12:51.846124] [INFO ] [MAIN ] HNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:51.875761] [DEBUG] [LOCAL ] HNI: Loaded 30085 existing records from primary storage. [2025-10-20T19:12:51.876037] [INFO ] [MAIN ] HNI: Loaded 30085 existing records. [2025-10-20T19:12:52.055109] [WARN ] [MAIN ] HNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:52.055415] [INFO ] [MAIN ] HNI: Wiping existing primary data before full refresh. [2025-10-20T19:12:52.055473] [INFO ] [LOCAL ] HNI: Starting deletion of primary data. [2025-10-20T19:12:52.055874] [DEBUG] [LOCAL ] HNI: Deleted primary data file. [2025-10-20T19:12:52.066955] [INFO ] [MAIN ] HNI: Starting full primary refresh. [2025-10-20T19:12:52.179490] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-03 (2.32 seconds). [2025-10-20T19:12:52.258789] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-04. [2025-10-20T19:12:52.466394] [INFO ] [REMOTE ] HNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:53.051350] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-01. [2025-10-20T19:12:53.851358] [INFO ] [REMOTE ] HMN: Fetching primary data for 2025-10. [2025-10-20T19:12:54.216429] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:12:54.411952] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T19:12:54.654776] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-05. [2025-10-20T19:12:54.657625] [INFO ] [REMOTE ] HMN: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:12:54.660856] [INFO ] [REMOTE ] HMN: Full fetch complete (31583 records). [2025-10-20T19:12:54.660929] [DEBUG] [VALIDATOR ] HMN: Validating primary data. [2025-10-20T19:12:55.030255] [DEBUG] [VALIDATOR ] HMN: All expected trading days are present. [2025-10-20T19:12:55.030344] [DEBUG] [VALIDATOR ] HMN: Primary data validation passed. [2025-10-20T19:12:55.030944] [INFO ] [MAIN ] HMN: Fetched primary data is valid. [2025-10-20T19:12:55.031027] [DEBUG] [LOCAL ] HMN: Saving primary data. [2025-10-20T19:12:55.031884] [DEBUG] [LOCAL ] HMN: Trimmed 31583 records to 23459 (keeping 200 days of data) [2025-10-20T19:12:55.167333] [DEBUG] [LOCAL ] HMN: Primary data saved (23459 records). [2025-10-20T19:12:55.167449] [INFO ] [MAIN ] HMN: Full history saved. [2025-10-20T19:12:55.167521] [INFO ] [MAIN ] Update Progress: 1033/2367 symbols completed. HMN took 23.4s. Estimated remaining: 2h 51m 33s. Rate: 7.8 symbols/min [2025-10-20T19:12:55.336689] [INFO ] [MAIN ] HOG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:12:55.383959] [DEBUG] [LOCAL ] HOG: Loaded 51493 existing records from primary storage. [2025-10-20T19:12:55.384292] [INFO ] [MAIN ] HOG: Loaded 51493 existing records. [2025-10-20T19:12:55.457132] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-07. [2025-10-20T19:12:55.560305] [WARN ] [MAIN ] HOG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:12:55.560498] [INFO ] [MAIN ] HOG: Wiping existing primary data before full refresh. [2025-10-20T19:12:55.560543] [INFO ] [LOCAL ] HOG: Starting deletion of primary data. [2025-10-20T19:12:55.560782] [DEBUG] [LOCAL ] HOG: Deleted primary data file. [2025-10-20T19:12:55.560847] [INFO ] [MAIN ] HOG: Starting full primary refresh. [2025-10-20T19:12:55.902359] [INFO ] [REMOTE ] HOG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:12:56.251346] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-01. [2025-10-20T19:12:56.745472] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-05 (2.09 seconds). [2025-10-20T19:12:57.051442] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-06. [2025-10-20T19:12:57.216797] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:12:57.847200] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:12:57.856676] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-08. [2025-10-20T19:12:58.651409] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-02. [2025-10-20T19:12:58.817219] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:12:59.334317] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T19:12:59.451394] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-09. [2025-10-20T19:12:59.948885] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:13:00.251363] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-03. [2025-10-20T19:13:01.051391] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-07. [2025-10-20T19:13:01.479741] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T19:13:01.851428] [INFO ] [REMOTE ] HMC: Fetching primary data for 2025-10. [2025-10-20T19:13:02.286317] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:13:02.563859] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:13:02.651429] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-08. [2025-10-20T19:13:03.451353] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-04. [2025-10-20T19:13:03.685007] [INFO ] [REMOTE ] HMC: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:13:03.691814] [INFO ] [REMOTE ] HMC: Full fetch complete (68739 records). [2025-10-20T19:13:03.691959] [DEBUG] [VALIDATOR ] HMC: Validating primary data. [2025-10-20T19:13:04.077313] [DEBUG] [VALIDATOR ] HMC: All expected trading days are present. [2025-10-20T19:13:04.077395] [DEBUG] [VALIDATOR ] HMC: Primary data validation passed. [2025-10-20T19:13:04.078288] [INFO ] [MAIN ] HMC: Fetched primary data is valid. [2025-10-20T19:13:04.078387] [DEBUG] [LOCAL ] HMC: Saving primary data. [2025-10-20T19:13:04.079722] [DEBUG] [LOCAL ] HMC: Trimmed 68739 records to 46338 (keeping 200 days of data) [2025-10-20T19:13:04.255184] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-02. [2025-10-20T19:13:04.354143] [DEBUG] [LOCAL ] HMC: Primary data saved (46338 records). [2025-10-20T19:13:04.354297] [INFO ] [MAIN ] HMC: Full history saved. [2025-10-20T19:13:04.355047] [INFO ] [MAIN ] Update Progress: 1034/2367 symbols completed. HMC took 34.2s. Estimated remaining: 2h 51m 27s. Rate: 7.8 symbols/min [2025-10-20T19:13:04.533281] [INFO ] [MAIN ] HOMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:04.577031] [DEBUG] [LOCAL ] HOMB: Loaded 44387 existing records from primary storage. [2025-10-20T19:13:04.577323] [INFO ] [MAIN ] HOMB: Loaded 44387 existing records. [2025-10-20T19:13:04.762932] [WARN ] [MAIN ] HOMB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:04.763192] [INFO ] [MAIN ] HOMB: Wiping existing primary data before full refresh. [2025-10-20T19:13:04.763247] [INFO ] [LOCAL ] HOMB: Starting deletion of primary data. [2025-10-20T19:13:04.763452] [DEBUG] [LOCAL ] HOMB: Deleted primary data file. [2025-10-20T19:13:04.763509] [INFO ] [MAIN ] HOMB: Starting full primary refresh. [2025-10-20T19:13:04.902480] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:13:05.059917] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-09. [2025-10-20T19:13:05.188676] [INFO ] [REMOTE ] HOMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:05.220078] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:13:05.523711] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:13:05.851431] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-05. [2025-10-20T19:13:06.651427] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-03. [2025-10-20T19:13:07.241808] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T19:13:07.451400] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-01. [2025-10-20T19:13:07.671546] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:13:07.929208] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T19:13:08.251433] [INFO ] [REMOTE ] HMY: Fetching primary data for 2025-10. [2025-10-20T19:13:09.051416] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-04. [2025-10-20T19:13:09.096813] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:13:09.851422] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-02. [2025-10-20T19:13:10.214770] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:13:10.507611] [INFO ] [REMOTE ] HMY: Incremental fetch complete for 2025-10 (2.26 seconds). [2025-10-20T19:13:10.516132] [INFO ] [REMOTE ] HMY: Full fetch complete (77086 records). [2025-10-20T19:13:10.516300] [DEBUG] [VALIDATOR ] HMY: Validating primary data. [2025-10-20T19:13:10.657011] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-05. [2025-10-20T19:13:10.914219] [DEBUG] [VALIDATOR ] HMY: All expected trading days are present. [2025-10-20T19:13:10.914399] [DEBUG] [VALIDATOR ] HMY: Primary data validation passed. [2025-10-20T19:13:10.915387] [INFO ] [MAIN ] HMY: Fetched primary data is valid. [2025-10-20T19:13:10.915448] [DEBUG] [LOCAL ] HMY: Saving primary data. [2025-10-20T19:13:10.916922] [DEBUG] [LOCAL ] HMY: Trimmed 77086 records to 53166 (keeping 200 days of data) [2025-10-20T19:13:11.164702] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:13:11.286940] [DEBUG] [LOCAL ] HMY: Primary data saved (53166 records). [2025-10-20T19:13:11.287087] [INFO ] [MAIN ] HMY: Full history saved. [2025-10-20T19:13:11.287170] [INFO ] [MAIN ] Update Progress: 1035/2367 symbols completed. HMY took 37.9s. Estimated remaining: 2h 51m 19s. Rate: 7.8 symbols/min [2025-10-20T19:13:11.456594] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-03. [2025-10-20T19:13:11.486103] [INFO ] [MAIN ] HOUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:11.528555] [DEBUG] [LOCAL ] HOUS: Loaded 46109 existing records from primary storage. [2025-10-20T19:13:11.528921] [INFO ] [MAIN ] HOUS: Loaded 46109 existing records. [2025-10-20T19:13:11.725882] [WARN ] [MAIN ] HOUS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:11.726247] [INFO ] [MAIN ] HOUS: Wiping existing primary data before full refresh. [2025-10-20T19:13:11.726332] [INFO ] [LOCAL ] HOUS: Starting deletion of primary data. [2025-10-20T19:13:11.726708] [DEBUG] [LOCAL ] HOUS: Deleted primary data file. [2025-10-20T19:13:11.726776] [INFO ] [MAIN ] HOUS: Starting full primary refresh. [2025-10-20T19:13:12.026471] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:13:12.142909] [INFO ] [REMOTE ] HOUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:12.251435] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-01. [2025-10-20T19:13:12.993443] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:13:13.051332] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-04. [2025-10-20T19:13:13.624358] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:13:13.851430] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-06. [2025-10-20T19:13:14.362518] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:13:14.651346] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-05. [2025-10-20T19:13:15.002118] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:13:15.451358] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-07. [2025-10-20T19:13:16.123915] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T19:13:16.251342] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-06. [2025-10-20T19:13:16.564976] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:13:17.051366] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-08. [2025-10-20T19:13:17.712329] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:13:17.851451] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-06. [2025-10-20T19:13:18.553068] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:13:18.651364] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-09. [2025-10-20T19:13:19.451432] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-07. [2025-10-20T19:13:19.538607] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T19:13:20.158072] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T19:13:20.251364] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-07. [2025-10-20T19:13:21.008336] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:13:21.051333] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-08. [2025-10-20T19:13:21.851381] [INFO ] [REMOTE ] HNI: Fetching primary data for 2025-10. [2025-10-20T19:13:22.016566] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:13:22.618889] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:13:22.651341] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-09. [2025-10-20T19:13:23.385543] [INFO ] [REMOTE ] HNI: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:13:23.390706] [INFO ] [REMOTE ] HNI: Full fetch complete (42110 records). [2025-10-20T19:13:23.390856] [DEBUG] [VALIDATOR ] HNI: Validating primary data. [2025-10-20T19:13:23.456829] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-08. [2025-10-20T19:13:23.766389] [DEBUG] [VALIDATOR ] HNI: All expected trading days are present. [2025-10-20T19:13:23.766461] [DEBUG] [VALIDATOR ] HNI: Primary data validation passed. [2025-10-20T19:13:23.767118] [INFO ] [MAIN ] HNI: Fetched primary data is valid. [2025-10-20T19:13:23.767180] [DEBUG] [LOCAL ] HNI: Saving primary data. [2025-10-20T19:13:23.768036] [DEBUG] [LOCAL ] HNI: Trimmed 42110 records to 29783 (keeping 200 days of data) [2025-10-20T19:13:23.945411] [DEBUG] [LOCAL ] HNI: Primary data saved (29783 records). [2025-10-20T19:13:23.945610] [INFO ] [MAIN ] HNI: Full history saved. [2025-10-20T19:13:23.945812] [INFO ] [MAIN ] Update Progress: 1036/2367 symbols completed. HNI took 32.3s. Estimated remaining: 2h 51m 17s. Rate: 7.8 symbols/min [2025-10-20T19:13:24.114672] [INFO ] [MAIN ] HOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:24.130453] [DEBUG] [LOCAL ] HOV: Loaded 14171 existing records from primary storage. [2025-10-20T19:13:24.130696] [INFO ] [MAIN ] HOV: Loaded 14171 existing records. [2025-10-20T19:13:24.251615] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-02. [2025-10-20T19:13:24.316966] [WARN ] [MAIN ] HOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:24.341612] [INFO ] [MAIN ] HOV: Wiping existing primary data before full refresh. [2025-10-20T19:13:24.341778] [INFO ] [LOCAL ] HOV: Starting deletion of primary data. [2025-10-20T19:13:24.343194] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:13:24.342305] [DEBUG] [LOCAL ] HOV: Deleted primary data file. [2025-10-20T19:13:24.346516] [INFO ] [MAIN ] HOV: Starting full primary refresh. [2025-10-20T19:13:24.702074] [INFO ] [REMOTE ] HOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:25.051374] [INFO ] [REMOTE ] HOMB: Fetching primary data for 2025-10. [2025-10-20T19:13:25.197575] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T19:13:25.709420] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:13:25.851335] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-03. [2025-10-20T19:13:26.437053] [INFO ] [REMOTE ] HOMB: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T19:13:26.443915] [INFO ] [REMOTE ] HOMB: Full fetch complete (64431 records). [2025-10-20T19:13:26.444032] [DEBUG] [VALIDATOR ] HOMB: Validating primary data. [2025-10-20T19:13:26.663351] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-01. [2025-10-20T19:13:26.826745] [DEBUG] [VALIDATOR ] HOMB: All expected trading days are present. [2025-10-20T19:13:26.826823] [DEBUG] [VALIDATOR ] HOMB: Primary data validation passed. [2025-10-20T19:13:26.827671] [INFO ] [MAIN ] HOMB: Fetched primary data is valid. [2025-10-20T19:13:26.827732] [DEBUG] [LOCAL ] HOMB: Saving primary data. [2025-10-20T19:13:26.829147] [DEBUG] [LOCAL ] HOMB: Trimmed 64431 records to 43754 (keeping 200 days of data) [2025-10-20T19:13:27.083730] [DEBUG] [LOCAL ] HOMB: Primary data saved (43754 records). [2025-10-20T19:13:27.083843] [INFO ] [MAIN ] HOMB: Full history saved. [2025-10-20T19:13:27.083914] [INFO ] [MAIN ] Update Progress: 1037/2367 symbols completed. HOMB took 22.7s. Estimated remaining: 2h 51m 4s. Rate: 7.8 symbols/min [2025-10-20T19:13:27.291762] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:13:27.308272] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T19:13:27.451331] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-02. [2025-10-20T19:13:28.251349] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-09. [2025-10-20T19:13:28.339348] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T19:13:29.051349] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-03. [2025-10-20T19:13:29.851446] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-04. [2025-10-20T19:13:29.868021] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T19:13:29.929099] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:13:30.651432] [INFO ] [REMOTE ] HOG: Fetching primary data for 2025-10. [2025-10-20T19:13:31.374370] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:13:31.451401] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-05. [2025-10-20T19:13:32.251434] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-04. [2025-10-20T19:13:32.414683] [INFO ] [REMOTE ] HOG: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:13:32.423382] [INFO ] [REMOTE ] HOG: Full fetch complete (74308 records). [2025-10-20T19:13:32.423550] [DEBUG] [VALIDATOR ] HOG: Validating primary data. [2025-10-20T19:13:32.850516] [DEBUG] [VALIDATOR ] HOG: All expected trading days are present. [2025-10-20T19:13:32.852532] [DEBUG] [VALIDATOR ] HOG: Primary data validation passed. [2025-10-20T19:13:32.852297] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T19:13:32.853848] [INFO ] [MAIN ] HOG: Fetched primary data is valid. [2025-10-20T19:13:32.863121] [DEBUG] [LOCAL ] HOG: Saving primary data. [2025-10-20T19:13:32.874419] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T19:13:32.874560] [DEBUG] [LOCAL ] HOG: Trimmed 74308 records to 50715 (keeping 200 days of data) [2025-10-20T19:13:33.059528] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-06. [2025-10-20T19:13:33.187582] [DEBUG] [LOCAL ] HOG: Primary data saved (50715 records). [2025-10-20T19:13:33.187757] [INFO ] [MAIN ] HOG: Full history saved. [2025-10-20T19:13:33.188184] [INFO ] [MAIN ] Update Progress: 1038/2367 symbols completed. HOG took 38.0s. Estimated remaining: 2h 50m 54s. Rate: 7.8 symbols/min [2025-10-20T19:13:33.851428] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-05. [2025-10-20T19:13:34.637483] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:13:34.651324] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-07. [2025-10-20T19:13:34.666580] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:13:35.451385] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-06. [2025-10-20T19:13:36.142063] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:13:36.251444] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-08. [2025-10-20T19:13:36.493037] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T19:13:37.051450] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-07. [2025-10-20T19:13:37.777041] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:13:37.851331] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-09. [2025-10-20T19:13:38.018821] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:13:38.651456] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-08. [2025-10-20T19:13:39.576877] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:13:39.596421] [INFO ] [REMOTE ] HOUS: Fetching primary data for 2025-10. [2025-10-20T19:13:39.597533] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T19:13:40.251363] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-09. [2025-10-20T19:13:41.306437] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:13:41.311879] [INFO ] [REMOTE ] HOV: Fetching primary data for 2025-10. [2025-10-20T19:13:41.548775] [INFO ] [REMOTE ] HOUS: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T19:13:41.556686] [INFO ] [REMOTE ] HOUS: Full fetch complete (64599 records). [2025-10-20T19:13:41.556844] [DEBUG] [VALIDATOR ] HOUS: Validating primary data. [2025-10-20T19:13:41.953815] [DEBUG] [VALIDATOR ] HOUS: All expected trading days are present. [2025-10-20T19:13:41.953892] [DEBUG] [VALIDATOR ] HOUS: Primary data validation passed. [2025-10-20T19:13:41.954699] [INFO ] [MAIN ] HOUS: Fetched primary data is valid. [2025-10-20T19:13:41.954761] [DEBUG] [LOCAL ] HOUS: Saving primary data. [2025-10-20T19:13:41.955955] [DEBUG] [LOCAL ] HOUS: Trimmed 64599 records to 45656 (keeping 200 days of data) [2025-10-20T19:13:42.207522] [INFO ] [REMOTE ] HOV: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T19:13:42.207608] [DEBUG] [LOCAL ] HOUS: Primary data saved (45656 records). [2025-10-20T19:13:42.210166] [INFO ] [MAIN ] HOUS: Full history saved. [2025-10-20T19:13:42.210287] [INFO ] [REMOTE ] HOV: Full fetch complete (19014 records). [2025-10-20T19:13:42.210546] [DEBUG] [VALIDATOR ] HOV: Validating primary data. [2025-10-20T19:13:42.210457] [INFO ] [MAIN ] Update Progress: 1039/2367 symbols completed. HOUS took 30.9s. Estimated remaining: 2h 50m 48s. Rate: 7.8 symbols/min [2025-10-20T19:13:42.579730] [DEBUG] [VALIDATOR ] HOV: All expected trading days are present. [2025-10-20T19:13:42.579824] [DEBUG] [VALIDATOR ] HOV: Primary data validation passed. [2025-10-20T19:13:42.580260] [INFO ] [MAIN ] HOV: Fetched primary data is valid. [2025-10-20T19:13:42.580299] [DEBUG] [LOCAL ] HOV: Saving primary data. [2025-10-20T19:13:42.580938] [DEBUG] [LOCAL ] HOV: Trimmed 19014 records to 14023 (keeping 200 days of data) [2025-10-20T19:13:42.665980] [DEBUG] [LOCAL ] HOV: Primary data saved (14023 records). [2025-10-20T19:13:42.666134] [INFO ] [MAIN ] HOV: Full history saved. [2025-10-20T19:13:42.666216] [INFO ] [MAIN ] Update Progress: 1040/2367 symbols completed. HOV took 18.7s. Estimated remaining: 2h 50m 31s. Rate: 7.8 symbols/min [2025-10-20T19:13:43.351976] [INFO ] [MAIN ] HPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.352846] [INFO ] [MAIN ] HPE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.398167] [INFO ] [MAIN ] HPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.403188] [INFO ] [MAIN ] HP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:13:43.414872] [DEBUG] [LOCAL ] HPF: Loaded 7566 existing records from primary storage. [2025-10-20T19:13:43.415514] [INFO ] [MAIN ] HPF: Loaded 7566 existing records. [2025-10-20T19:13:43.473932] [DEBUG] [LOCAL ] HPI: Loaded 10785 existing records from primary storage. [2025-10-20T19:13:43.503798] [INFO ] [MAIN ] HPI: Loaded 10785 existing records. [2025-10-20T19:13:43.767063] [WARN ] [MAIN ] HPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:43.803165] [INFO ] [MAIN ] HPF: Wiping existing primary data before full refresh. [2025-10-20T19:13:43.803503] [INFO ] [LOCAL ] HPF: Starting deletion of primary data. [2025-10-20T19:13:43.809796] [DEBUG] [LOCAL ] HPE: Loaded 54477 existing records from primary storage. [2025-10-20T19:13:43.817032] [DEBUG] [LOCAL ] HPF: Deleted primary data file. [2025-10-20T19:13:43.827501] [INFO ] [MAIN ] HPF: Starting full primary refresh. [2025-10-20T19:13:43.816911] [DEBUG] [LOCAL ] HP: Loaded 52245 existing records from primary storage. [2025-10-20T19:13:43.833800] [INFO ] [MAIN ] HP: Loaded 52245 existing records. [2025-10-20T19:13:43.827370] [INFO ] [MAIN ] HPE: Loaded 54477 existing records. [2025-10-20T19:13:44.029186] [WARN ] [MAIN ] HPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.164128] [INFO ] [MAIN ] HPI: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.199963] [INFO ] [LOCAL ] HPI: Starting deletion of primary data. [2025-10-20T19:13:44.320487] [DEBUG] [LOCAL ] HPI: Deleted primary data file. [2025-10-20T19:13:44.341151] [WARN ] [MAIN ] HPE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.360853] [INFO ] [MAIN ] HPI: Starting full primary refresh. [2025-10-20T19:13:44.371293] [INFO ] [MAIN ] HPE: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.411077] [INFO ] [LOCAL ] HPE: Starting deletion of primary data. [2025-10-20T19:13:44.435028] [DEBUG] [LOCAL ] HPE: Deleted primary data file. [2025-10-20T19:13:44.443579] [INFO ] [MAIN ] HPE: Starting full primary refresh. [2025-10-20T19:13:44.675755] [WARN ] [MAIN ] HP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:13:44.860181] [INFO ] [MAIN ] HP: Wiping existing primary data before full refresh. [2025-10-20T19:13:44.934421] [INFO ] [LOCAL ] HP: Starting deletion of primary data. [2025-10-20T19:13:45.064804] [DEBUG] [LOCAL ] HP: Deleted primary data file. [2025-10-20T19:13:45.098828] [INFO ] [MAIN ] HP: Starting full primary refresh. [2025-10-20T19:13:45.190434] [INFO ] [REMOTE ] HPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.235867] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-01. [2025-10-20T19:13:45.650893] [INFO ] [REMOTE ] HPE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.696900] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-01. [2025-10-20T19:13:45.823016] [INFO ] [REMOTE ] HPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.856322] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-01. [2025-10-20T19:13:45.960391] [INFO ] [REMOTE ] HP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:13:45.960731] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-01. [2025-10-20T19:13:46.143455] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T19:13:46.145789] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-02. [2025-10-20T19:13:46.639319] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:13:46.641477] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-02. [2025-10-20T19:13:46.799539] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:13:46.801458] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-03. [2025-10-20T19:13:47.029114] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T19:13:47.410852] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T19:13:47.451365] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-03. [2025-10-20T19:13:47.545467] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:13:47.700624] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:13:48.217465] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:13:48.251358] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-02. [2025-10-20T19:13:49.051436] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-02. [2025-10-20T19:13:49.851415] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-04. [2025-10-20T19:13:50.225295] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-10-20T19:13:50.344245] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:13:50.622494] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:13:50.651345] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-03. [2025-10-20T19:13:51.451409] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-03. [2025-10-20T19:13:52.251349] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-05. [2025-10-20T19:13:52.644932] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T19:13:52.956140] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T19:13:53.051345] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-04. [2025-10-20T19:13:53.151305] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T19:13:53.851412] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-06. [2025-10-20T19:13:54.638819] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:13:54.651321] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-07. [2025-10-20T19:13:54.903905] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:13:55.451389] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-04. [2025-10-20T19:13:55.499898] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:13:56.178721] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T19:13:56.251400] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-08. [2025-10-20T19:13:56.861177] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:13:57.051356] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-04. [2025-10-20T19:13:57.851385] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-09. [2025-10-20T19:13:58.651442] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-05. [2025-10-20T19:13:58.795536] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:13:58.850769] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:13:59.380631] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:13:59.451377] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-05. [2025-10-20T19:14:00.251368] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-05. [2025-10-20T19:14:01.051380] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-06. [2025-10-20T19:14:01.619868] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:14:01.836804] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:14:01.851393] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-07. [2025-10-20T19:14:02.308762] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T19:14:02.651349] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-06. [2025-10-20T19:14:02.704226] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:14:03.451398] [INFO ] [REMOTE ] HPI: Fetching primary data for 2025-10. [2025-10-20T19:14:04.251411] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-08. [2025-10-20T19:14:04.460763] [INFO ] [REMOTE ] HPI: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:14:04.462872] [INFO ] [REMOTE ] HPI: Full fetch complete (13918 records). [2025-10-20T19:14:04.462940] [DEBUG] [VALIDATOR ] HPI: Validating primary data. [2025-10-20T19:14:04.860169] [DEBUG] [VALIDATOR ] HPI: All expected trading days are present. [2025-10-20T19:14:04.860324] [DEBUG] [VALIDATOR ] HPI: Primary data validation passed. [2025-10-20T19:14:04.860844] [INFO ] [MAIN ] HPI: Fetched primary data is valid. [2025-10-20T19:14:04.860935] [DEBUG] [LOCAL ] HPI: Saving primary data. [2025-10-20T19:14:04.863322] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T19:14:04.873946] [DEBUG] [LOCAL ] HPI: Trimmed 13918 records to 10790 (keeping 200 days of data) [2025-10-20T19:14:04.939018] [DEBUG] [LOCAL ] HPI: Primary data saved (10790 records). [2025-10-20T19:14:04.939140] [INFO ] [MAIN ] HPI: Full history saved. [2025-10-20T19:14:04.939217] [INFO ] [MAIN ] Update Progress: 1041/2367 symbols completed. HPI took 22.3s. Estimated remaining: 2h 50m 42s. Rate: 7.8 symbols/min [2025-10-20T19:14:05.044244] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T19:14:05.057987] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-06. [2025-10-20T19:14:05.123359] [INFO ] [MAIN ] HPP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:05.170538] [DEBUG] [LOCAL ] HPP: Loaded 51937 existing records from primary storage. [2025-10-20T19:14:05.170923] [INFO ] [MAIN ] HPP: Loaded 51937 existing records. [2025-10-20T19:14:05.345934] [WARN ] [MAIN ] HPP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:05.346149] [INFO ] [MAIN ] HPP: Wiping existing primary data before full refresh. [2025-10-20T19:14:05.346196] [INFO ] [LOCAL ] HPP: Starting deletion of primary data. [2025-10-20T19:14:05.346471] [DEBUG] [LOCAL ] HPP: Deleted primary data file. [2025-10-20T19:14:05.346527] [INFO ] [MAIN ] HPP: Starting full primary refresh. [2025-10-20T19:14:05.700000] [INFO ] [REMOTE ] HPP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:05.851338] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-01. [2025-10-20T19:14:06.651413] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-09. [2025-10-20T19:14:06.826764] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:14:07.443947] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:14:07.454169] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-07. [2025-10-20T19:14:07.477687] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:14:08.251426] [INFO ] [REMOTE ] HPF: Fetching primary data for 2025-10. [2025-10-20T19:14:08.991773] [INFO ] [REMOTE ] HPF: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:14:08.993257] [INFO ] [REMOTE ] HPF: Full fetch complete (10569 records). [2025-10-20T19:14:08.993345] [DEBUG] [VALIDATOR ] HPF: Validating primary data. [2025-10-20T19:14:09.056618] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-02. [2025-10-20T19:14:09.353316] [DEBUG] [VALIDATOR ] HPF: All expected trading days are present. [2025-10-20T19:14:09.353389] [DEBUG] [VALIDATOR ] HPF: Primary data validation passed. [2025-10-20T19:14:09.353630] [INFO ] [MAIN ] HPF: Fetched primary data is valid. [2025-10-20T19:14:09.353703] [DEBUG] [LOCAL ] HPF: Saving primary data. [2025-10-20T19:14:09.354121] [DEBUG] [LOCAL ] HPF: Trimmed 10569 records to 7516 (keeping 200 days of data) [2025-10-20T19:14:09.397525] [DEBUG] [LOCAL ] HPF: Primary data saved (7516 records). [2025-10-20T19:14:09.397615] [INFO ] [MAIN ] HPF: Full history saved. [2025-10-20T19:14:09.397722] [INFO ] [MAIN ] Update Progress: 1042/2367 symbols completed. HPF took 26.7s. Estimated remaining: 2h 50m 30s. Rate: 7.8 symbols/min [2025-10-20T19:14:09.569127] [INFO ] [MAIN ] HPQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:09.617255] [DEBUG] [LOCAL ] HPQ: Loaded 54449 existing records from primary storage. [2025-10-20T19:14:09.617573] [INFO ] [MAIN ] HPQ: Loaded 54449 existing records. [2025-10-20T19:14:09.789930] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:14:09.840147] [WARN ] [MAIN ] HPQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:09.840353] [INFO ] [MAIN ] HPQ: Wiping existing primary data before full refresh. [2025-10-20T19:14:09.840404] [INFO ] [LOCAL ] HPQ: Starting deletion of primary data. [2025-10-20T19:14:09.840616] [DEBUG] [LOCAL ] HPQ: Deleted primary data file. [2025-10-20T19:14:09.840699] [INFO ] [MAIN ] HPQ: Starting full primary refresh. [2025-10-20T19:14:09.861127] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-08. [2025-10-20T19:14:10.198205] [INFO ] [REMOTE ] HPQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:10.651359] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-07. [2025-10-20T19:14:10.868007] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:14:11.451417] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-03. [2025-10-20T19:14:11.942474] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:14:12.251349] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-01. [2025-10-20T19:14:12.407054] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:14:13.018895] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:14:13.051362] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-04. [2025-10-20T19:14:13.851393] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-08. [2025-10-20T19:14:14.026157] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:14:14.652155] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-02. [2025-10-20T19:14:14.656457] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:14:15.440261] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:14:15.451348] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-09. [2025-10-20T19:14:16.251355] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-05. [2025-10-20T19:14:16.445470] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:14:17.043676] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:14:17.054971] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-03. [2025-10-20T19:14:17.857095] [INFO ] [REMOTE ] HPE: Fetching primary data for 2025-10. [2025-10-20T19:14:17.871624] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:14:18.651436] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-06. [2025-10-20T19:14:18.967117] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:14:19.451381] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-04. [2025-10-20T19:14:20.035707] [INFO ] [REMOTE ] HPE: Incremental fetch complete for 2025-10 (2.18 seconds). [2025-10-20T19:14:20.043561] [INFO ] [REMOTE ] HPE: Full fetch complete (77876 records). [2025-10-20T19:14:20.043722] [DEBUG] [VALIDATOR ] HPE: Validating primary data. [2025-10-20T19:14:20.256792] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-09. [2025-10-20T19:14:20.481505] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T19:14:20.512075] [DEBUG] [VALIDATOR ] HPE: All expected trading days are present. [2025-10-20T19:14:20.512400] [DEBUG] [VALIDATOR ] HPE: Primary data validation passed. [2025-10-20T19:14:20.513961] [INFO ] [MAIN ] HPE: Fetched primary data is valid. [2025-10-20T19:14:20.514550] [DEBUG] [LOCAL ] HPE: Saving primary data. [2025-10-20T19:14:20.516929] [DEBUG] [LOCAL ] HPE: Trimmed 77876 records to 53697 (keeping 200 days of data) [2025-10-20T19:14:20.875372] [DEBUG] [LOCAL ] HPE: Primary data saved (53697 records). [2025-10-20T19:14:20.875572] [INFO ] [MAIN ] HPE: Full history saved. [2025-10-20T19:14:20.875760] [INFO ] [MAIN ] Update Progress: 1043/2367 symbols completed. HPE took 38.2s. Estimated remaining: 2h 50m 27s. Rate: 7.8 symbols/min [2025-10-20T19:14:21.056958] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-07. [2025-10-20T19:14:21.066880] [INFO ] [MAIN ] HPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:21.083056] [DEBUG] [LOCAL ] HPS: Loaded 10447 existing records from primary storage. [2025-10-20T19:14:21.083928] [INFO ] [MAIN ] HPS: Loaded 10447 existing records. [2025-10-20T19:14:21.283824] [WARN ] [MAIN ] HPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:21.284131] [INFO ] [MAIN ] HPS: Wiping existing primary data before full refresh. [2025-10-20T19:14:21.284184] [INFO ] [LOCAL ] HPS: Starting deletion of primary data. [2025-10-20T19:14:21.284452] [DEBUG] [LOCAL ] HPS: Deleted primary data file. [2025-10-20T19:14:21.284512] [INFO ] [MAIN ] HPS: Starting full primary refresh. [2025-10-20T19:14:21.656909] [INFO ] [REMOTE ] HPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:21.696041] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-10-20T19:14:21.751704] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:14:21.851368] [INFO ] [REMOTE ] HP: Fetching primary data for 2025-10. [2025-10-20T19:14:22.651377] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-01. [2025-10-20T19:14:22.819902] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:14:23.444823] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:14:23.451318] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-02. [2025-10-20T19:14:23.633216] [INFO ] [REMOTE ] HP: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:14:23.640468] [INFO ] [REMOTE ] HP: Full fetch complete (74058 records). [2025-10-20T19:14:23.640587] [DEBUG] [VALIDATOR ] HP: Validating primary data. [2025-10-20T19:14:24.026235] [DEBUG] [VALIDATOR ] HP: All expected trading days are present. [2025-10-20T19:14:24.026386] [DEBUG] [VALIDATOR ] HP: Primary data validation passed. [2025-10-20T19:14:24.027672] [INFO ] [MAIN ] HP: Fetched primary data is valid. [2025-10-20T19:14:24.027780] [DEBUG] [LOCAL ] HP: Saving primary data. [2025-10-20T19:14:24.029430] [DEBUG] [LOCAL ] HP: Trimmed 74058 records to 51484 (keeping 200 days of data) [2025-10-20T19:14:24.127884] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T19:14:24.253065] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-08. [2025-10-20T19:14:24.353885] [DEBUG] [LOCAL ] HP: Primary data saved (51484 records). [2025-10-20T19:14:24.354006] [INFO ] [MAIN ] HP: Full history saved. [2025-10-20T19:14:24.354080] [INFO ] [MAIN ] Update Progress: 1044/2367 symbols completed. HP took 41.7s. Estimated remaining: 2h 50m 14s. Rate: 7.8 symbols/min [2025-10-20T19:14:24.523867] [INFO ] [MAIN ] HQH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:24.550300] [DEBUG] [LOCAL ] HQH: Loaded 26187 existing records from primary storage. [2025-10-20T19:14:24.550545] [INFO ] [MAIN ] HQH: Loaded 26187 existing records. [2025-10-20T19:14:24.722252] [WARN ] [MAIN ] HQH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:24.722443] [INFO ] [MAIN ] HQH: Wiping existing primary data before full refresh. [2025-10-20T19:14:24.722489] [INFO ] [LOCAL ] HQH: Starting deletion of primary data. [2025-10-20T19:14:24.722696] [DEBUG] [LOCAL ] HQH: Deleted primary data file. [2025-10-20T19:14:24.722752] [INFO ] [MAIN ] HQH: Starting full primary refresh. [2025-10-20T19:14:25.060989] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-05. [2025-10-20T19:14:25.065633] [INFO ] [REMOTE ] HQH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:25.851356] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-01. [2025-10-20T19:14:25.990813] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T19:14:26.651362] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-03. [2025-10-20T19:14:27.061630] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:14:27.117857] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T19:14:27.451450] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-06. [2025-10-20T19:14:27.495611] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T19:14:28.251438] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-04. [2025-10-20T19:14:28.999332] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T19:14:29.051341] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-05. [2025-10-20T19:14:29.323312] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:14:29.851390] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-09. [2025-10-20T19:14:29.866346] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T19:14:30.651363] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-02. [2025-10-20T19:14:31.451411] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-06. [2025-10-20T19:14:31.649452] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:14:31.754194] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:14:32.251432] [INFO ] [REMOTE ] HPP: Fetching primary data for 2025-10. [2025-10-20T19:14:32.269476] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:14:33.051410] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-07. [2025-10-20T19:14:33.851416] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-03. [2025-10-20T19:14:33.993857] [INFO ] [REMOTE ] HPP: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:14:34.001355] [INFO ] [REMOTE ] HPP: Full fetch complete (73402 records). [2025-10-20T19:14:34.001491] [DEBUG] [VALIDATOR ] HPP: Validating primary data. [2025-10-20T19:14:34.411933] [DEBUG] [VALIDATOR ] HPP: All expected trading days are present. [2025-10-20T19:14:34.412017] [DEBUG] [VALIDATOR ] HPP: Primary data validation passed. [2025-10-20T19:14:34.413018] [INFO ] [MAIN ] HPP: Fetched primary data is valid. [2025-10-20T19:14:34.413087] [DEBUG] [LOCAL ] HPP: Saving primary data. [2025-10-20T19:14:34.414484] [DEBUG] [LOCAL ] HPP: Trimmed 73402 records to 51262 (keeping 200 days of data) [2025-10-20T19:14:34.664686] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-07. [2025-10-20T19:14:34.720634] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:14:34.719906] [DEBUG] [LOCAL ] HPP: Primary data saved (51262 records). [2025-10-20T19:14:34.726149] [INFO ] [MAIN ] HPP: Full history saved. [2025-10-20T19:14:34.726236] [INFO ] [MAIN ] Update Progress: 1045/2367 symbols completed. HPP took 29.8s. Estimated remaining: 2h 50m 10s. Rate: 7.8 symbols/min [2025-10-20T19:14:34.896732] [INFO ] [MAIN ] HQL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:34.914952] [DEBUG] [LOCAL ] HQL: Loaded 17218 existing records from primary storage. [2025-10-20T19:14:34.915182] [INFO ] [MAIN ] HQL: Loaded 17218 existing records. [2025-10-20T19:14:35.098159] [WARN ] [MAIN ] HQL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:35.098669] [INFO ] [MAIN ] HQL: Wiping existing primary data before full refresh. [2025-10-20T19:14:35.098808] [INFO ] [LOCAL ] HQL: Starting deletion of primary data. [2025-10-20T19:14:35.099100] [DEBUG] [LOCAL ] HQL: Deleted primary data file. [2025-10-20T19:14:35.099161] [INFO ] [MAIN ] HQL: Starting full primary refresh. [2025-10-20T19:14:35.396279] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:14:35.452909] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-08. [2025-10-20T19:14:35.515109] [INFO ] [REMOTE ] HQL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:35.654758] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:14:36.251376] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-04. [2025-10-20T19:14:37.051548] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-08. [2025-10-20T19:14:37.438286] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:14:37.477958] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-08 (2.03 seconds). [2025-10-20T19:14:37.846847] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T19:14:37.851317] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-09. [2025-10-20T19:14:38.383526] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T19:14:38.651384] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-05. [2025-10-20T19:14:39.451343] [INFO ] [REMOTE ] HPS: Fetching primary data for 2025-10. [2025-10-20T19:14:39.825276] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T19:14:40.236225] [INFO ] [REMOTE ] HPS: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T19:14:40.238295] [INFO ] [REMOTE ] HPS: Full fetch complete (15123 records). [2025-10-20T19:14:40.238384] [DEBUG] [VALIDATOR ] HPS: Validating primary data. [2025-10-20T19:14:40.256498] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-06. [2025-10-20T19:14:40.618498] [DEBUG] [VALIDATOR ] HPS: All expected trading days are present. [2025-10-20T19:14:40.618575] [DEBUG] [VALIDATOR ] HPS: Primary data validation passed. [2025-10-20T19:14:40.618953] [INFO ] [MAIN ] HPS: Fetched primary data is valid. [2025-10-20T19:14:40.619014] [DEBUG] [LOCAL ] HPS: Saving primary data. [2025-10-20T19:14:40.619529] [DEBUG] [LOCAL ] HPS: Trimmed 15123 records to 10377 (keeping 200 days of data) [2025-10-20T19:14:40.685325] [DEBUG] [LOCAL ] HPS: Primary data saved (10377 records). [2025-10-20T19:14:40.685672] [INFO ] [MAIN ] HPS: Full history saved. [2025-10-20T19:14:40.685769] [INFO ] [MAIN ] Update Progress: 1046/2367 symbols completed. HPS took 19.8s. Estimated remaining: 2h 49m 60s. Rate: 7.8 symbols/min [2025-10-20T19:14:40.862181] [INFO ] [MAIN ] HR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:40.913229] [DEBUG] [LOCAL ] HR: Loaded 52870 existing records from primary storage. [2025-10-20T19:14:40.913557] [INFO ] [MAIN ] HR: Loaded 52870 existing records. [2025-10-20T19:14:41.057748] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-01. [2025-10-20T19:14:41.098771] [WARN ] [MAIN ] HR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:41.099061] [INFO ] [MAIN ] HR: Wiping existing primary data before full refresh. [2025-10-20T19:14:41.099107] [INFO ] [LOCAL ] HR: Starting deletion of primary data. [2025-10-20T19:14:41.099315] [DEBUG] [LOCAL ] HR: Deleted primary data file. [2025-10-20T19:14:41.099369] [INFO ] [MAIN ] HR: Starting full primary refresh. [2025-10-20T19:14:41.454456] [INFO ] [REMOTE ] HR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:41.467693] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:14:41.851360] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-01. [2025-10-20T19:14:42.118205] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T19:14:42.651398] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-02. [2025-10-20T19:14:43.451389] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-09. [2025-10-20T19:14:43.494452] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:14:43.541836] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T19:14:44.251383] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-03. [2025-10-20T19:14:45.051347] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-07. [2025-10-20T19:14:45.296588] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:14:45.330774] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:14:45.851401] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-02. [2025-10-20T19:14:46.066282] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T19:14:46.651376] [INFO ] [REMOTE ] HPQ: Fetching primary data for 2025-10. [2025-10-20T19:14:47.164634] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:14:47.461618] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-04. [2025-10-20T19:14:47.466930] [INFO ] [REMOTE ] HPQ: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:14:47.475779] [INFO ] [REMOTE ] HPQ: Full fetch complete (77842 records). [2025-10-20T19:14:47.475976] [DEBUG] [VALIDATOR ] HPQ: Validating primary data. [2025-10-20T19:14:47.925451] [DEBUG] [VALIDATOR ] HPQ: All expected trading days are present. [2025-10-20T19:14:47.925535] [DEBUG] [VALIDATOR ] HPQ: Primary data validation passed. [2025-10-20T19:14:47.926529] [INFO ] [MAIN ] HPQ: Fetched primary data is valid. [2025-10-20T19:14:47.926595] [DEBUG] [LOCAL ] HPQ: Saving primary data. [2025-10-20T19:14:47.928040] [DEBUG] [LOCAL ] HPQ: Trimmed 77842 records to 53669 (keeping 200 days of data) [2025-10-20T19:14:48.254039] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-08. [2025-10-20T19:14:48.253941] [DEBUG] [LOCAL ] HPQ: Primary data saved (53669 records). [2025-10-20T19:14:48.254500] [INFO ] [MAIN ] HPQ: Full history saved. [2025-10-20T19:14:48.255247] [INFO ] [MAIN ] Update Progress: 1047/2367 symbols completed. HPQ took 38.9s. Estimated remaining: 2h 49m 52s. Rate: 7.8 symbols/min [2025-10-20T19:14:48.431704] [INFO ] [MAIN ] HRB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:48.464469] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T19:14:48.486170] [DEBUG] [LOCAL ] HRB: Loaded 49064 existing records from primary storage. [2025-10-20T19:14:48.486467] [INFO ] [MAIN ] HRB: Loaded 49064 existing records. [2025-10-20T19:14:48.661408] [WARN ] [MAIN ] HRB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:48.661685] [INFO ] [MAIN ] HRB: Wiping existing primary data before full refresh. [2025-10-20T19:14:48.661763] [INFO ] [LOCAL ] HRB: Starting deletion of primary data. [2025-10-20T19:14:48.662006] [DEBUG] [LOCAL ] HRB: Deleted primary data file. [2025-10-20T19:14:48.662079] [INFO ] [MAIN ] HRB: Starting full primary refresh. [2025-10-20T19:14:49.008684] [INFO ] [REMOTE ] HRB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:49.051335] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-05. [2025-10-20T19:14:49.448936] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:14:49.851401] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-03. [2025-10-20T19:14:50.087234] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T19:14:50.651402] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-06. [2025-10-20T19:14:51.434686] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:14:51.451330] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-04. [2025-10-20T19:14:51.479611] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:14:52.251390] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-09. [2025-10-20T19:14:53.051361] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-07. [2025-10-20T19:14:53.250092] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:14:53.461288] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:14:53.697151] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T19:14:53.851354] [INFO ] [REMOTE ] HQH: Fetching primary data for 2025-10. [2025-10-20T19:14:54.651407] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-08. [2025-10-20T19:14:55.057117] [INFO ] [REMOTE ] HQH: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:14:55.061695] [INFO ] [REMOTE ] HQH: Full fetch complete (37134 records). [2025-10-20T19:14:55.061774] [DEBUG] [VALIDATOR ] HQH: Validating primary data. [2025-10-20T19:14:55.433181] [DEBUG] [VALIDATOR ] HQH: All expected trading days are present. [2025-10-20T19:14:55.433367] [DEBUG] [VALIDATOR ] HQH: Primary data validation passed. [2025-10-20T19:14:55.434074] [INFO ] [MAIN ] HQH: Fetched primary data is valid. [2025-10-20T19:14:55.434146] [DEBUG] [LOCAL ] HQH: Saving primary data. [2025-10-20T19:14:55.435177] [DEBUG] [LOCAL ] HQH: Trimmed 37134 records to 25838 (keeping 200 days of data) [2025-10-20T19:14:55.485511] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-01. [2025-10-20T19:14:55.587846] [DEBUG] [LOCAL ] HQH: Primary data saved (25838 records). [2025-10-20T19:14:55.593241] [INFO ] [MAIN ] HQH: Full history saved. [2025-10-20T19:14:55.593370] [INFO ] [MAIN ] Update Progress: 1048/2367 symbols completed. HQH took 31.2s. Estimated remaining: 2h 49m 43s. Rate: 7.8 symbols/min [2025-10-20T19:14:55.641501] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:14:55.773263] [INFO ] [MAIN ] HRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:14:55.805954] [DEBUG] [LOCAL ] HRI: Loaded 32839 existing records from primary storage. [2025-10-20T19:14:55.806226] [INFO ] [MAIN ] HRI: Loaded 32839 existing records. [2025-10-20T19:14:55.981383] [WARN ] [MAIN ] HRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:14:55.981601] [INFO ] [MAIN ] HRI: Wiping existing primary data before full refresh. [2025-10-20T19:14:55.981673] [INFO ] [LOCAL ] HRI: Starting deletion of primary data. [2025-10-20T19:14:55.981884] [DEBUG] [LOCAL ] HRI: Deleted primary data file. [2025-10-20T19:14:55.981938] [INFO ] [MAIN ] HRI: Starting full primary refresh. [2025-10-20T19:14:56.256603] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-09. [2025-10-20T19:14:56.352142] [INFO ] [REMOTE ] HRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:14:56.650232] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:14:57.051371] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-05. [2025-10-20T19:14:57.256996] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:14:57.851389] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-02. [2025-10-20T19:14:58.592285] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T19:14:58.651342] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-06. [2025-10-20T19:14:59.427559] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:14:59.451362] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-03. [2025-10-20T19:15:00.251417] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-01. [2025-10-20T19:15:00.393484] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:15:01.051434] [INFO ] [REMOTE ] HQL: Fetching primary data for 2025-10. [2025-10-20T19:15:01.218703] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:15:01.661073] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:15:01.851387] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-07. [2025-10-20T19:15:02.432434] [INFO ] [REMOTE ] HQL: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:15:02.436500] [INFO ] [REMOTE ] HQL: Full fetch complete (25348 records). [2025-10-20T19:15:02.436578] [DEBUG] [VALIDATOR ] HQL: Validating primary data. [2025-10-20T19:15:02.660244] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-04. [2025-10-20T19:15:02.798911] [DEBUG] [VALIDATOR ] HQL: All expected trading days are present. [2025-10-20T19:15:02.798989] [DEBUG] [VALIDATOR ] HQL: Primary data validation passed. [2025-10-20T19:15:02.799285] [INFO ] [MAIN ] HQL: Fetched primary data is valid. [2025-10-20T19:15:02.799323] [DEBUG] [LOCAL ] HQL: Saving primary data. [2025-10-20T19:15:02.799939] [DEBUG] [LOCAL ] HQL: Trimmed 25348 records to 16958 (keeping 200 days of data) [2025-10-20T19:15:02.897587] [DEBUG] [LOCAL ] HQL: Primary data saved (16958 records). [2025-10-20T19:15:02.897748] [INFO ] [MAIN ] HQL: Full history saved. [2025-10-20T19:15:02.897831] [INFO ] [MAIN ] Update Progress: 1049/2367 symbols completed. HQL took 28.2s. Estimated remaining: 2h 49m 35s. Rate: 7.8 symbols/min [2025-10-20T19:15:03.061144] [INFO ] [MAIN ] HRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:03.109487] [DEBUG] [LOCAL ] HRL: Loaded 53904 existing records from primary storage. [2025-10-20T19:15:03.109820] [INFO ] [MAIN ] HRL: Loaded 53904 existing records. [2025-10-20T19:15:03.286019] [WARN ] [MAIN ] HRL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:03.286243] [INFO ] [MAIN ] HRL: Wiping existing primary data before full refresh. [2025-10-20T19:15:03.286296] [INFO ] [LOCAL ] HRL: Starting deletion of primary data. [2025-10-20T19:15:03.286727] [DEBUG] [LOCAL ] HRL: Deleted primary data file. [2025-10-20T19:15:03.286783] [INFO ] [MAIN ] HRL: Starting full primary refresh. [2025-10-20T19:15:03.454967] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-02. [2025-10-20T19:15:03.637502] [INFO ] [REMOTE ] HRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:03.762502] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T19:15:04.251433] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-01. [2025-10-20T19:15:04.641821] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T19:15:04.970628] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:15:05.051377] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-03. [2025-10-20T19:15:05.851381] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-05. [2025-10-20T19:15:06.051497] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:15:06.649058] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:15:06.651844] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-08. [2025-10-20T19:15:07.456609] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-04. [2025-10-20T19:15:07.472412] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T19:15:08.251415] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-06. [2025-10-20T19:15:08.433049] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:15:08.643817] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:15:09.051375] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-02. [2025-10-20T19:15:09.536491] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:15:09.851380] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-07. [2025-10-20T19:15:10.656987] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-09. [2025-10-20T19:15:10.675967] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T19:15:11.451378] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-03. [2025-10-20T19:15:11.547228] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:15:12.251350] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-05. [2025-10-20T19:15:12.356588] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T19:15:13.051445] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-08. [2025-10-20T19:15:13.359319] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:15:13.706708] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:15:13.851381] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-06. [2025-10-20T19:15:14.651405] [INFO ] [REMOTE ] HR: Fetching primary data for 2025-10. [2025-10-20T19:15:14.775598] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:15:15.134548] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:15:15.451376] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-04. [2025-10-20T19:15:16.251385] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-09. [2025-10-20T19:15:16.505096] [INFO ] [REMOTE ] HR: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:15:16.513015] [INFO ] [REMOTE ] HR: Full fetch complete (75068 records). [2025-10-20T19:15:16.513180] [DEBUG] [VALIDATOR ] HR: Validating primary data. [2025-10-20T19:15:16.914506] [DEBUG] [VALIDATOR ] HR: All expected trading days are present. [2025-10-20T19:15:16.914586] [DEBUG] [VALIDATOR ] HR: Primary data validation passed. [2025-10-20T19:15:16.915514] [INFO ] [MAIN ] HR: Fetched primary data is valid. [2025-10-20T19:15:16.915578] [DEBUG] [LOCAL ] HR: Saving primary data. [2025-10-20T19:15:16.917046] [DEBUG] [LOCAL ] HR: Trimmed 75068 records to 52115 (keeping 200 days of data) [2025-10-20T19:15:17.092342] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-07. [2025-10-20T19:15:17.238174] [DEBUG] [LOCAL ] HR: Primary data saved (52115 records). [2025-10-20T19:15:17.238309] [INFO ] [MAIN ] HR: Full history saved. [2025-10-20T19:15:17.238395] [INFO ] [MAIN ] Update Progress: 1050/2367 symbols completed. HR took 36.6s. Estimated remaining: 2h 49m 36s. Rate: 7.8 symbols/min [2025-10-20T19:15:17.418785] [INFO ] [MAIN ] HRTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:17.449955] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:15:17.491040] [DEBUG] [LOCAL ] HRTG: Loaded 34930 existing records from primary storage. [2025-10-20T19:15:17.491469] [INFO ] [MAIN ] HRTG: Loaded 34930 existing records. [2025-10-20T19:15:17.677946] [WARN ] [MAIN ] HRTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:17.678220] [INFO ] [MAIN ] HRTG: Wiping existing primary data before full refresh. [2025-10-20T19:15:17.678361] [INFO ] [LOCAL ] HRTG: Starting deletion of primary data. [2025-10-20T19:15:17.678705] [DEBUG] [LOCAL ] HRTG: Deleted primary data file. [2025-10-20T19:15:17.678763] [INFO ] [MAIN ] HRTG: Starting full primary refresh. [2025-10-20T19:15:17.900345] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-05. [2025-10-20T19:15:17.988774] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T19:15:18.116511] [INFO ] [REMOTE ] HRTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:18.632078] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:15:18.651395] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-01. [2025-10-20T19:15:19.451391] [INFO ] [REMOTE ] HRB: Fetching primary data for 2025-10. [2025-10-20T19:15:19.735343] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T19:15:20.067130] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:15:20.251392] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-08. [2025-10-20T19:15:21.051339] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-02. [2025-10-20T19:15:21.128374] [INFO ] [REMOTE ] HRB: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:15:21.135980] [INFO ] [REMOTE ] HRB: Full fetch complete (70025 records). [2025-10-20T19:15:21.136158] [DEBUG] [VALIDATOR ] HRB: Validating primary data. [2025-10-20T19:15:21.328446] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:15:21.577511] [DEBUG] [VALIDATOR ] HRB: All expected trading days are present. [2025-10-20T19:15:21.577585] [DEBUG] [VALIDATOR ] HRB: Primary data validation passed. [2025-10-20T19:15:21.578459] [INFO ] [MAIN ] HRB: Fetched primary data is valid. [2025-10-20T19:15:21.578520] [DEBUG] [LOCAL ] HRB: Saving primary data. [2025-10-20T19:15:21.579803] [DEBUG] [LOCAL ] HRB: Trimmed 70025 records to 48355 (keeping 200 days of data) [2025-10-20T19:15:21.860124] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-09. [2025-10-20T19:15:21.875873] [DEBUG] [LOCAL ] HRB: Primary data saved (48355 records). [2025-10-20T19:15:21.875970] [INFO ] [MAIN ] HRB: Full history saved. [2025-10-20T19:15:21.876047] [INFO ] [MAIN ] Update Progress: 1051/2367 symbols completed. HRB took 33.6s. Estimated remaining: 2h 49m 24s. Rate: 7.8 symbols/min [2025-10-20T19:15:22.055157] [INFO ] [MAIN ] HSBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:22.068765] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:15:22.116031] [DEBUG] [LOCAL ] HSBC: Loaded 51777 existing records from primary storage. [2025-10-20T19:15:22.116370] [INFO ] [MAIN ] HSBC: Loaded 51777 existing records. [2025-10-20T19:15:22.292881] [WARN ] [MAIN ] HSBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:22.293129] [INFO ] [MAIN ] HSBC: Wiping existing primary data before full refresh. [2025-10-20T19:15:22.293180] [INFO ] [LOCAL ] HSBC: Starting deletion of primary data. [2025-10-20T19:15:22.293443] [DEBUG] [LOCAL ] HSBC: Deleted primary data file. [2025-10-20T19:15:22.293483] [INFO ] [MAIN ] HSBC: Starting full primary refresh. [2025-10-20T19:15:22.638699] [INFO ] [REMOTE ] HSBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:22.651330] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-01. [2025-10-20T19:15:23.254435] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:15:23.451374] [INFO ] [REMOTE ] HRI: Fetching primary data for 2025-10. [2025-10-20T19:15:24.251365] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-06. [2025-10-20T19:15:24.422119] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:15:24.471575] [INFO ] [REMOTE ] HRI: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:15:24.476079] [INFO ] [REMOTE ] HRI: Full fetch complete (46630 records). [2025-10-20T19:15:24.476168] [DEBUG] [VALIDATOR ] HRI: Validating primary data. [2025-10-20T19:15:24.881761] [DEBUG] [VALIDATOR ] HRI: All expected trading days are present. [2025-10-20T19:15:24.881839] [DEBUG] [VALIDATOR ] HRI: Primary data validation passed. [2025-10-20T19:15:24.882543] [INFO ] [MAIN ] HRI: Fetched primary data is valid. [2025-10-20T19:15:24.882582] [DEBUG] [LOCAL ] HRI: Saving primary data. [2025-10-20T19:15:24.883540] [DEBUG] [LOCAL ] HRI: Trimmed 46630 records to 32387 (keeping 200 days of data) [2025-10-20T19:15:25.068017] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-03. [2025-10-20T19:15:25.097166] [DEBUG] [LOCAL ] HRI: Primary data saved (32387 records). [2025-10-20T19:15:25.097309] [INFO ] [MAIN ] HRI: Full history saved. [2025-10-20T19:15:25.097399] [INFO ] [MAIN ] Update Progress: 1052/2367 symbols completed. HRI took 29.5s. Estimated remaining: 2h 49m 11s. Rate: 7.8 symbols/min [2025-10-20T19:15:25.300206] [INFO ] [MAIN ] HSHP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:25.316903] [DEBUG] [LOCAL ] HSHP: Loaded 15028 existing records from primary storage. [2025-10-20T19:15:25.317144] [INFO ] [MAIN ] HSHP: Loaded 15028 existing records. [2025-10-20T19:15:25.524447] [WARN ] [MAIN ] HSHP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:25.524688] [INFO ] [MAIN ] HSHP: Wiping existing primary data before full refresh. [2025-10-20T19:15:25.524746] [INFO ] [LOCAL ] HSHP: Starting deletion of primary data. [2025-10-20T19:15:25.524964] [DEBUG] [LOCAL ] HSHP: Deleted primary data file. [2025-10-20T19:15:25.525019] [INFO ] [MAIN ] HSHP: Starting full primary refresh. [2025-10-20T19:15:25.852962] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-02. [2025-10-20T19:15:25.898627] [INFO ] [REMOTE ] HSHP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:26.014890] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:15:26.250514] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T19:15:26.651425] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-01. [2025-10-20T19:15:27.451345] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-04. [2025-10-20T19:15:27.688589] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:15:27.733389] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T19:15:28.251373] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-02. [2025-10-20T19:15:28.884728] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T19:15:29.051350] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-07. [2025-10-20T19:15:29.281994] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:15:29.851390] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-05. [2025-10-20T19:15:30.651359] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-03. [2025-10-20T19:15:31.069519] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T19:15:31.355431] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:15:31.451349] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-06. [2025-10-20T19:15:31.741348] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T19:15:32.251443] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-03. [2025-10-20T19:15:32.815694] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:15:33.051381] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-07. [2025-10-20T19:15:33.851422] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-08. [2025-10-20T19:15:34.264603] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:15:34.331599] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T19:15:34.651377] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-04. [2025-10-20T19:15:35.451375] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-04. [2025-10-20T19:15:35.626267] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T19:15:35.716582] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:15:36.251392] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-08. [2025-10-20T19:15:37.051465] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-09. [2025-10-20T19:15:37.458362] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T19:15:37.669248] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:15:37.851410] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-05. [2025-10-20T19:15:38.651453] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-09. [2025-10-20T19:15:39.017834] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:15:39.451408] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-05. [2025-10-20T19:15:39.801395] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T19:15:40.108282] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:15:40.251369] [INFO ] [REMOTE ] HRL: Fetching primary data for 2025-10. [2025-10-20T19:15:40.315159] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T19:15:41.051379] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-06. [2025-10-20T19:15:41.851382] [INFO ] [REMOTE ] HRTG: Fetching primary data for 2025-10. [2025-10-20T19:15:42.302207] [INFO ] [REMOTE ] HRL: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T19:15:42.311186] [INFO ] [REMOTE ] HRL: Full fetch complete (76999 records). [2025-10-20T19:15:42.311372] [DEBUG] [VALIDATOR ] HRL: Validating primary data. [2025-10-20T19:15:42.652473] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-06. [2025-10-20T19:15:42.909000] [DEBUG] [VALIDATOR ] HRL: All expected trading days are present. [2025-10-20T19:15:42.909148] [DEBUG] [VALIDATOR ] HRL: Primary data validation passed. [2025-10-20T19:15:42.911095] [INFO ] [MAIN ] HRL: Fetched primary data is valid. [2025-10-20T19:15:42.911344] [DEBUG] [LOCAL ] HRL: Saving primary data. [2025-10-20T19:15:42.935548] [DEBUG] [LOCAL ] HRL: Trimmed 76999 records to 53128 (keeping 200 days of data) [2025-10-20T19:15:42.965355] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T19:15:43.269022] [INFO ] [REMOTE ] HRTG: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T19:15:43.294676] [INFO ] [REMOTE ] HRTG: Full fetch complete (44350 records). [2025-10-20T19:15:43.312419] [DEBUG] [VALIDATOR ] HRTG: Validating primary data. [2025-10-20T19:15:43.455804] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-07. [2025-10-20T19:15:43.537767] [DEBUG] [LOCAL ] HRL: Primary data saved (53128 records). [2025-10-20T19:15:43.558898] [INFO ] [MAIN ] HRL: Full history saved. [2025-10-20T19:15:43.565536] [INFO ] [MAIN ] Update Progress: 1053/2367 symbols completed. HRL took 40.7s. Estimated remaining: 2h 49m 17s. Rate: 7.8 symbols/min [2025-10-20T19:15:43.964955] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T19:15:43.957792] [INFO ] [MAIN ] HSY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:44.074757] [DEBUG] [VALIDATOR ] HRTG: All expected trading days are present. [2025-10-20T19:15:44.074864] [DEBUG] [VALIDATOR ] HRTG: Primary data validation passed. [2025-10-20T19:15:44.075836] [INFO ] [MAIN ] HRTG: Fetched primary data is valid. [2025-10-20T19:15:44.076021] [DEBUG] [LOCAL ] HRTG: Saving primary data. [2025-10-20T19:15:44.077233] [DEBUG] [LOCAL ] HRTG: Trimmed 44350 records to 34587 (keeping 200 days of data) [2025-10-20T19:15:44.268127] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-07. [2025-10-20T19:15:44.309597] [DEBUG] [LOCAL ] HSY: Loaded 51192 existing records from primary storage. [2025-10-20T19:15:44.312548] [DEBUG] [LOCAL ] HRTG: Primary data saved (34587 records). [2025-10-20T19:15:44.313248] [INFO ] [MAIN ] HRTG: Full history saved. [2025-10-20T19:15:44.313938] [INFO ] [MAIN ] Update Progress: 1054/2367 symbols completed. HRTG took 27.1s. Estimated remaining: 2h 49m 0s. Rate: 7.8 symbols/min [2025-10-20T19:15:44.314098] [INFO ] [MAIN ] HSY: Loaded 51192 existing records. [2025-10-20T19:15:44.652801] [INFO ] [MAIN ] HTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:44.668198] [DEBUG] [LOCAL ] HTB: No existing data file found in primary storage. [2025-10-20T19:15:44.668285] [WARN ] [MAIN ] HTB: No existing data found, performing full refresh instead. [2025-10-20T19:15:44.668434] [INFO ] [MAIN ] HTB: Wiping existing primary data before full refresh. [2025-10-20T19:15:44.672866] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-10-20T19:15:44.672766] [WARN ] [MAIN ] HSY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:44.673008] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-10-20T19:15:44.673166] [INFO ] [MAIN ] HSY: Wiping existing primary data before full refresh. [2025-10-20T19:15:44.673250] [INFO ] [LOCAL ] HSY: Starting deletion of primary data. [2025-10-20T19:15:44.673333] [INFO ] [MAIN ] HTB: Starting full primary refresh. [2025-10-20T19:15:44.704594] [DEBUG] [LOCAL ] HSY: Deleted primary data file. [2025-10-20T19:15:44.736282] [INFO ] [MAIN ] HSY: Starting full primary refresh. [2025-10-20T19:15:45.417541] [INFO ] [REMOTE ] HTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:45.422826] [INFO ] [REMOTE ] HTB: Fetching primary data for 2025-01. [2025-10-20T19:15:45.467516] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:15:45.465944] [INFO ] [REMOTE ] HSY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:45.510960] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:15:45.652410] [DEBUG] [REMOTE ] HTB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:15:45.652961] [WARN ] [REMOTE ] HTB: No data returned for month 2025-01 [2025-10-20T19:15:45.653064] [FATAL] [REMOTE ] HTB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:15:45.653117] [FATAL] [MAIN ] HTB: Failed to fetch primary data. [2025-10-20T19:15:45.653179] [WARN ] [MAIN ] HTB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:15:45.653258] [INFO ] [LOCAL ] HTB: Starting deletion of primary data. [2025-10-20T19:15:45.653333] [DEBUG] [LOCAL ] HTB: No primary data file found to delete. [2025-10-20T19:15:45.653372] [INFO ] [LOCAL ] HTB: Starting deletion of derivative data. [2025-10-20T19:15:45.653413] [DEBUG] [LOCAL ] HTB: No derivative data file found to delete. [2025-10-20T19:15:45.653466] [INFO ] [MAIN ] Update Progress: 1055/2367 symbols completed. HTB took 1.3s. Estimated remaining: 2h 48m 44s. Rate: 7.8 symbols/min [2025-10-20T19:15:45.837494] [INFO ] [MAIN ] HTD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:45.853257] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-01. [2025-10-20T19:15:45.853673] [DEBUG] [LOCAL ] HTD: Loaded 14424 existing records from primary storage. [2025-10-20T19:15:45.855041] [INFO ] [MAIN ] HTD: Loaded 14424 existing records. [2025-10-20T19:15:46.059305] [WARN ] [MAIN ] HTD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:15:46.059556] [INFO ] [MAIN ] HTD: Wiping existing primary data before full refresh. [2025-10-20T19:15:46.059604] [INFO ] [LOCAL ] HTD: Starting deletion of primary data. [2025-10-20T19:15:46.059895] [DEBUG] [LOCAL ] HTD: Deleted primary data file. [2025-10-20T19:15:46.059952] [INFO ] [MAIN ] HTD: Starting full primary refresh. [2025-10-20T19:15:46.411097] [INFO ] [REMOTE ] HTD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:46.651380] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-08. [2025-10-20T19:15:47.451372] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-08. [2025-10-20T19:15:47.823251] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:15:47.992840] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T19:15:48.251356] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-01. [2025-10-20T19:15:48.305771] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:15:49.051431] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-09. [2025-10-20T19:15:49.093714] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T19:15:49.851400] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-09. [2025-10-20T19:15:50.083736] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T19:15:50.651413] [INFO ] [REMOTE ] HSHP: Fetching primary data for 2025-10. [2025-10-20T19:15:51.451379] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-02. [2025-10-20T19:15:51.617316] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:15:51.733288] [INFO ] [REMOTE ] HSHP: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:15:51.735602] [INFO ] [REMOTE ] HSHP: Full fetch complete (23285 records). [2025-10-20T19:15:51.735705] [DEBUG] [VALIDATOR ] HSHP: Validating primary data. [2025-10-20T19:15:52.107430] [DEBUG] [VALIDATOR ] HSHP: All expected trading days are present. [2025-10-20T19:15:52.107512] [DEBUG] [VALIDATOR ] HSHP: Primary data validation passed. [2025-10-20T19:15:52.108108] [INFO ] [MAIN ] HSHP: Fetched primary data is valid. [2025-10-20T19:15:52.108205] [DEBUG] [LOCAL ] HSHP: Saving primary data. [2025-10-20T19:15:52.108875] [DEBUG] [LOCAL ] HSHP: Trimmed 23285 records to 14817 (keeping 200 days of data) [2025-10-20T19:15:52.197573] [DEBUG] [LOCAL ] HSHP: Primary data saved (14817 records). [2025-10-20T19:15:52.197986] [INFO ] [MAIN ] HSHP: Full history saved. [2025-10-20T19:15:52.198089] [INFO ] [MAIN ] Update Progress: 1056/2367 symbols completed. HSHP took 27.1s. Estimated remaining: 2h 48m 35s. Rate: 7.8 symbols/min [2025-10-20T19:15:52.260811] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-02. [2025-10-20T19:15:52.311568] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T19:15:52.387675] [INFO ] [MAIN ] HTFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:52.387901] [DEBUG] [LOCAL ] HTFC: No existing data file found in primary storage. [2025-10-20T19:15:52.387946] [WARN ] [MAIN ] HTFC: No existing data found, performing full refresh instead. [2025-10-20T19:15:52.388089] [INFO ] [MAIN ] HTFC: Wiping existing primary data before full refresh. [2025-10-20T19:15:52.388130] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-10-20T19:15:52.388172] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-10-20T19:15:52.388249] [INFO ] [MAIN ] HTFC: Starting full primary refresh. [2025-10-20T19:15:52.751612] [INFO ] [REMOTE ] HTFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:53.051423] [INFO ] [REMOTE ] HSBC: Fetching primary data for 2025-10. [2025-10-20T19:15:53.851348] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-03. [2025-10-20T19:15:54.261077] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T19:15:54.651340] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-01. [2025-10-20T19:15:54.733186] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:15:55.034153] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-01 (0.38 seconds). [2025-10-20T19:15:55.057585] [INFO ] [REMOTE ] HSBC: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:15:55.065111] [INFO ] [REMOTE ] HSBC: Full fetch complete (74147 records). [2025-10-20T19:15:55.065241] [DEBUG] [VALIDATOR ] HSBC: Validating primary data. [2025-10-20T19:15:55.456484] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-03. [2025-10-20T19:15:55.456987] [DEBUG] [VALIDATOR ] HSBC: All expected trading days are present. [2025-10-20T19:15:55.457327] [DEBUG] [VALIDATOR ] HSBC: Primary data validation passed. [2025-10-20T19:15:55.459160] [INFO ] [MAIN ] HSBC: Fetched primary data is valid. [2025-10-20T19:15:55.459333] [DEBUG] [LOCAL ] HSBC: Saving primary data. [2025-10-20T19:15:55.460714] [DEBUG] [LOCAL ] HSBC: Trimmed 74147 records to 50998 (keeping 200 days of data) [2025-10-20T19:15:55.776550] [DEBUG] [LOCAL ] HSBC: Primary data saved (50998 records). [2025-10-20T19:15:55.776837] [INFO ] [MAIN ] HSBC: Full history saved. [2025-10-20T19:15:55.776965] [INFO ] [MAIN ] Update Progress: 1057/2367 symbols completed. HSBC took 33.9s. Estimated remaining: 2h 48m 22s. Rate: 7.8 symbols/min [2025-10-20T19:15:55.949951] [INFO ] [MAIN ] HTGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:15:55.950147] [DEBUG] [LOCAL ] HTGC: No existing data file found in primary storage. [2025-10-20T19:15:55.950186] [WARN ] [MAIN ] HTGC: No existing data found, performing full refresh instead. [2025-10-20T19:15:55.950297] [INFO ] [MAIN ] HTGC: Wiping existing primary data before full refresh. [2025-10-20T19:15:55.950338] [INFO ] [LOCAL ] HTGC: Starting deletion of primary data. [2025-10-20T19:15:55.950379] [DEBUG] [LOCAL ] HTGC: No primary data file found to delete. [2025-10-20T19:15:55.950412] [INFO ] [MAIN ] HTGC: Starting full primary refresh. [2025-10-20T19:15:56.266675] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-04. [2025-10-20T19:15:56.308714] [INFO ] [REMOTE ] HTGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:15:57.051343] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-01. [2025-10-20T19:15:57.214283] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T19:15:57.428318] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T19:15:57.851397] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-05. [2025-10-20T19:15:58.645437] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:15:58.651348] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-04. [2025-10-20T19:15:58.732331] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T19:15:59.451384] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-06. [2025-10-20T19:16:00.162497] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:16:00.251339] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-02. [2025-10-20T19:16:00.533482] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:16:01.051450] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-02. [2025-10-20T19:16:01.701159] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:16:01.851387] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-05. [2025-10-20T19:16:01.991481] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:16:02.651371] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-03. [2025-10-20T19:16:02.980811] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-03 (0.33 seconds). [2025-10-20T19:16:03.451347] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-04. [2025-10-20T19:16:03.648841] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:16:03.858499] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T19:16:04.251534] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-05. [2025-10-20T19:16:04.621949] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-05 (0.37 seconds). [2025-10-20T19:16:05.051411] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-07. [2025-10-20T19:16:05.851347] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-06. [2025-10-20T19:16:05.996489] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T19:16:06.184847] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:16:06.651371] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-08. [2025-10-20T19:16:07.451425] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-06. [2025-10-20T19:16:07.497629] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T19:16:08.251413] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-07. [2025-10-20T19:16:08.502971] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-07 (0.25 seconds). [2025-10-20T19:16:09.051391] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-08. [2025-10-20T19:16:09.265773] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T19:16:09.352320] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-08 (0.30 seconds). [2025-10-20T19:16:09.851385] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-09. [2025-10-20T19:16:10.651362] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-03. [2025-10-20T19:16:10.801535] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T19:16:11.451404] [INFO ] [REMOTE ] HTD: Fetching primary data for 2025-10. [2025-10-20T19:16:12.251389] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-09. [2025-10-20T19:16:12.479186] [INFO ] [REMOTE ] HTD: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T19:16:12.483047] [INFO ] [REMOTE ] HTD: Full fetch complete (19664 records). [2025-10-20T19:16:12.492633] [DEBUG] [VALIDATOR ] HTD: Validating primary data. [2025-10-20T19:16:12.599204] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T19:16:12.620527] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-09 (0.37 seconds). [2025-10-20T19:16:12.899447] [DEBUG] [VALIDATOR ] HTD: All expected trading days are present. [2025-10-20T19:16:12.899533] [DEBUG] [VALIDATOR ] HTD: Primary data validation passed. [2025-10-20T19:16:12.899949] [INFO ] [MAIN ] HTD: Fetched primary data is valid. [2025-10-20T19:16:12.900047] [DEBUG] [LOCAL ] HTD: Saving primary data. [2025-10-20T19:16:12.900846] [DEBUG] [LOCAL ] HTD: Trimmed 19664 records to 14305 (keeping 200 days of data) [2025-10-20T19:16:12.987115] [DEBUG] [LOCAL ] HTD: Primary data saved (14305 records). [2025-10-20T19:16:12.987259] [INFO ] [MAIN ] HTD: Full history saved. [2025-10-20T19:16:12.987340] [INFO ] [MAIN ] Update Progress: 1058/2367 symbols completed. HTD took 27.3s. Estimated remaining: 2h 48m 26s. Rate: 7.8 symbols/min [2025-10-20T19:16:13.056469] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-04. [2025-10-20T19:16:13.166311] [INFO ] [MAIN ] HTH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:13.198496] [DEBUG] [LOCAL ] HTH: Loaded 33038 existing records from primary storage. [2025-10-20T19:16:13.198788] [INFO ] [MAIN ] HTH: Loaded 33038 existing records. [2025-10-20T19:16:13.376898] [WARN ] [MAIN ] HTH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:13.377128] [INFO ] [MAIN ] HTH: Wiping existing primary data before full refresh. [2025-10-20T19:16:13.377176] [INFO ] [LOCAL ] HTH: Starting deletion of primary data. [2025-10-20T19:16:13.377411] [DEBUG] [LOCAL ] HTH: Deleted primary data file. [2025-10-20T19:16:13.377468] [INFO ] [MAIN ] HTH: Starting full primary refresh. [2025-10-20T19:16:13.730482] [INFO ] [REMOTE ] HTH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:13.851341] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-07. [2025-10-20T19:16:14.651354] [INFO ] [REMOTE ] HTFC: Fetching primary data for 2025-10. [2025-10-20T19:16:14.826527] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:16:15.051616] [INFO ] [REMOTE ] HTFC: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T19:16:15.052302] [INFO ] [REMOTE ] HTFC: Full fetch complete (839 records). [2025-10-20T19:16:15.052371] [DEBUG] [VALIDATOR ] HTFC: Validating primary data. [2025-10-20T19:16:15.233539] [WARN ] [VALIDATOR ] HTFC: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T19:16:15.233746] [FATAL] [MAIN ] HTFC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:16:15.233821] [WARN ] [MAIN ] HTFC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:16:15.233904] [INFO ] [LOCAL ] HTFC: Starting deletion of primary data. [2025-10-20T19:16:15.233993] [DEBUG] [LOCAL ] HTFC: No primary data file found to delete. [2025-10-20T19:16:15.234037] [INFO ] [LOCAL ] HTFC: Starting deletion of derivative data. [2025-10-20T19:16:15.234079] [DEBUG] [LOCAL ] HTFC: No derivative data file found to delete. [2025-10-20T19:16:15.234135] [INFO ] [MAIN ] Update Progress: 1059/2367 symbols completed. HTFC took 23.0s. Estimated remaining: 2h 48m 12s. Rate: 7.8 symbols/min [2025-10-20T19:16:15.429270] [INFO ] [MAIN ] HUBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:15.451378] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-01. [2025-10-20T19:16:15.466628] [DEBUG] [LOCAL ] HUBB: Loaded 36909 existing records from primary storage. [2025-10-20T19:16:15.466946] [INFO ] [MAIN ] HUBB: Loaded 36909 existing records. [2025-10-20T19:16:15.665975] [WARN ] [MAIN ] HUBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:15.666299] [INFO ] [MAIN ] HUBB: Wiping existing primary data before full refresh. [2025-10-20T19:16:15.666423] [INFO ] [LOCAL ] HUBB: Starting deletion of primary data. [2025-10-20T19:16:15.667016] [DEBUG] [LOCAL ] HUBB: Deleted primary data file. [2025-10-20T19:16:15.667075] [INFO ] [MAIN ] HUBB: Starting full primary refresh. [2025-10-20T19:16:15.990061] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T19:16:16.078261] [INFO ] [REMOTE ] HUBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:16.251376] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-08. [2025-10-20T19:16:16.674496] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T19:16:17.051386] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-01. [2025-10-20T19:16:17.851383] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-05. [2025-10-20T19:16:18.070269] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:16:18.521291] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T19:16:18.651371] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-02. [2025-10-20T19:16:19.451357] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-09. [2025-10-20T19:16:19.666896] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T19:16:20.101538] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:16:20.251361] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-03. [2025-10-20T19:16:21.051396] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-02. [2025-10-20T19:16:21.233231] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:16:21.301067] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:16:21.851392] [INFO ] [REMOTE ] HSY: Fetching primary data for 2025-10. [2025-10-20T19:16:22.658869] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-04. [2025-10-20T19:16:22.664287] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:16:22.873301] [INFO ] [REMOTE ] HSY: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:16:22.881114] [INFO ] [REMOTE ] HSY: Full fetch complete (73654 records). [2025-10-20T19:16:22.881260] [DEBUG] [VALIDATOR ] HSY: Validating primary data. [2025-10-20T19:16:23.275097] [DEBUG] [VALIDATOR ] HSY: All expected trading days are present. [2025-10-20T19:16:23.275218] [DEBUG] [VALIDATOR ] HSY: Primary data validation passed. [2025-10-20T19:16:23.276295] [INFO ] [MAIN ] HSY: Fetched primary data is valid. [2025-10-20T19:16:23.276357] [DEBUG] [LOCAL ] HSY: Saving primary data. [2025-10-20T19:16:23.277807] [DEBUG] [LOCAL ] HSY: Trimmed 73654 records to 50468 (keeping 200 days of data) [2025-10-20T19:16:23.472828] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-06. [2025-10-20T19:16:23.607867] [DEBUG] [LOCAL ] HSY: Primary data saved (50468 records). [2025-10-20T19:16:23.607995] [INFO ] [MAIN ] HSY: Full history saved. [2025-10-20T19:16:23.608073] [INFO ] [MAIN ] Update Progress: 1060/2367 symbols completed. HSY took 40.0s. Estimated remaining: 2h 48m 5s. Rate: 7.8 symbols/min [2025-10-20T19:16:23.777601] [INFO ] [MAIN ] HUBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:23.815365] [DEBUG] [LOCAL ] HUBS: Loaded 40651 existing records from primary storage. [2025-10-20T19:16:23.815681] [INFO ] [MAIN ] HUBS: Loaded 40651 existing records. [2025-10-20T19:16:23.997500] [WARN ] [MAIN ] HUBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:23.997861] [INFO ] [MAIN ] HUBS: Wiping existing primary data before full refresh. [2025-10-20T19:16:23.997931] [INFO ] [LOCAL ] HUBS: Starting deletion of primary data. [2025-10-20T19:16:23.999128] [DEBUG] [LOCAL ] HUBS: Deleted primary data file. [2025-10-20T19:16:23.999185] [INFO ] [MAIN ] HUBS: Starting full primary refresh. [2025-10-20T19:16:24.161760] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T19:16:24.257007] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-05. [2025-10-20T19:16:24.380925] [INFO ] [REMOTE ] HUBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:25.051402] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-03. [2025-10-20T19:16:25.134314] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:16:25.562478] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:16:25.851391] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-01. [2025-10-20T19:16:26.651376] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-07. [2025-10-20T19:16:26.781724] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:16:26.874541] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:16:27.451370] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-04. [2025-10-20T19:16:28.251391] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-06. [2025-10-20T19:16:28.373181] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:16:29.051338] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-08. [2025-10-20T19:16:29.133341] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:16:29.485732] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T19:16:29.851391] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-05. [2025-10-20T19:16:30.651379] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-02. [2025-10-20T19:16:30.709982] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:16:31.366919] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:16:31.451359] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-09. [2025-10-20T19:16:32.101021] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:16:32.251386] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-06. [2025-10-20T19:16:33.051349] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-03. [2025-10-20T19:16:33.236345] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:16:33.599584] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:16:33.851419] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-07. [2025-10-20T19:16:34.651364] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-07. [2025-10-20T19:16:34.678067] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:16:34.751400] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:16:35.451455] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-08. [2025-10-20T19:16:36.226608] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:16:36.251405] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-04. [2025-10-20T19:16:36.866842] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:16:37.051417] [INFO ] [REMOTE ] HTGC: Fetching primary data for 2025-10. [2025-10-20T19:16:37.612055] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:16:37.851363] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-05. [2025-10-20T19:16:38.651370] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-08. [2025-10-20T19:16:38.822248] [INFO ] [REMOTE ] HTGC: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:16:38.832953] [INFO ] [REMOTE ] HTGC: Full fetch complete (68896 records). [2025-10-20T19:16:38.833135] [DEBUG] [VALIDATOR ] HTGC: Validating primary data. [2025-10-20T19:16:39.261982] [DEBUG] [VALIDATOR ] HTGC: All expected trading days are present. [2025-10-20T19:16:39.262096] [DEBUG] [VALIDATOR ] HTGC: Primary data validation passed. [2025-10-20T19:16:39.263048] [INFO ] [MAIN ] HTGC: Fetched primary data is valid. [2025-10-20T19:16:39.263129] [DEBUG] [LOCAL ] HTGC: Saving primary data. [2025-10-20T19:16:39.265005] [DEBUG] [LOCAL ] HTGC: Trimmed 68896 records to 47761 (keeping 200 days of data) [2025-10-20T19:16:39.290491] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:16:39.470731] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-09. [2025-10-20T19:16:39.620845] [DEBUG] [LOCAL ] HTGC: Primary data saved (47761 records). [2025-10-20T19:16:39.622170] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T19:16:39.628981] [INFO ] [MAIN ] HTGC: Full history saved. [2025-10-20T19:16:39.629409] [INFO ] [MAIN ] Update Progress: 1061/2367 symbols completed. HTGC took 43.9s. Estimated remaining: 2h 48m 8s. Rate: 7.8 symbols/min [2025-10-20T19:16:39.797292] [INFO ] [MAIN ] HUM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:39.840362] [DEBUG] [LOCAL ] HUM: Loaded 50015 existing records from primary storage. [2025-10-20T19:16:39.840697] [INFO ] [MAIN ] HUM: Loaded 50015 existing records. [2025-10-20T19:16:40.015584] [WARN ] [MAIN ] HUM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:40.015799] [INFO ] [MAIN ] HUM: Wiping existing primary data before full refresh. [2025-10-20T19:16:40.015847] [INFO ] [LOCAL ] HUM: Starting deletion of primary data. [2025-10-20T19:16:40.016409] [DEBUG] [LOCAL ] HUM: Deleted primary data file. [2025-10-20T19:16:40.016449] [INFO ] [MAIN ] HUM: Starting full primary refresh. [2025-10-20T19:16:40.256611] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-06. [2025-10-20T19:16:40.368703] [INFO ] [REMOTE ] HUM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:40.770635] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T19:16:41.051385] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-01. [2025-10-20T19:16:41.785893] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:16:41.851363] [INFO ] [REMOTE ] HTH: Fetching primary data for 2025-10. [2025-10-20T19:16:42.651370] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-07. [2025-10-20T19:16:42.708192] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:16:43.227612] [INFO ] [REMOTE ] HTH: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:16:43.232018] [INFO ] [REMOTE ] HTH: Full fetch complete (46847 records). [2025-10-20T19:16:43.232101] [DEBUG] [VALIDATOR ] HTH: Validating primary data. [2025-10-20T19:16:43.458084] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-09. [2025-10-20T19:16:43.604731] [DEBUG] [VALIDATOR ] HTH: All expected trading days are present. [2025-10-20T19:16:43.604832] [DEBUG] [VALIDATOR ] HTH: Primary data validation passed. [2025-10-20T19:16:43.605530] [INFO ] [MAIN ] HTH: Fetched primary data is valid. [2025-10-20T19:16:43.605569] [DEBUG] [LOCAL ] HTH: Saving primary data. [2025-10-20T19:16:43.606539] [DEBUG] [LOCAL ] HTH: Trimmed 46847 records to 32664 (keeping 200 days of data) [2025-10-20T19:16:43.800421] [DEBUG] [LOCAL ] HTH: Primary data saved (32664 records). [2025-10-20T19:16:43.800559] [INFO ] [MAIN ] HTH: Full history saved. [2025-10-20T19:16:43.800635] [INFO ] [MAIN ] Update Progress: 1062/2367 symbols completed. HTH took 30.8s. Estimated remaining: 2h 47m 56s. Rate: 7.8 symbols/min [2025-10-20T19:16:43.969445] [INFO ] [MAIN ] HUN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:44.019197] [DEBUG] [LOCAL ] HUN: Loaded 53992 existing records from primary storage. [2025-10-20T19:16:44.019507] [INFO ] [MAIN ] HUN: Loaded 53992 existing records. [2025-10-20T19:16:44.200512] [WARN ] [MAIN ] HUN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:44.200830] [INFO ] [MAIN ] HUN: Wiping existing primary data before full refresh. [2025-10-20T19:16:44.200894] [INFO ] [LOCAL ] HUN: Starting deletion of primary data. [2025-10-20T19:16:44.208603] [DEBUG] [LOCAL ] HUN: Deleted primary data file. [2025-10-20T19:16:44.221583] [INFO ] [MAIN ] HUN: Starting full primary refresh. [2025-10-20T19:16:44.253105] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-02. [2025-10-20T19:16:44.287613] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:16:44.604197] [INFO ] [REMOTE ] HUN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:44.896806] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T19:16:45.051427] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-08. [2025-10-20T19:16:45.851352] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-01. [2025-10-20T19:16:46.325961] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T19:16:46.651432] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-03. [2025-10-20T19:16:46.878763] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:16:47.457732] [INFO ] [REMOTE ] HUBB: Fetching primary data for 2025-10. [2025-10-20T19:16:47.464027] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T19:16:48.206621] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:16:48.251419] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-09. [2025-10-20T19:16:48.992174] [INFO ] [REMOTE ] HUBB: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:16:48.997204] [INFO ] [REMOTE ] HUBB: Full fetch complete (54278 records). [2025-10-20T19:16:48.997323] [DEBUG] [VALIDATOR ] HUBB: Validating primary data. [2025-10-20T19:16:49.060137] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-04. [2025-10-20T19:16:49.376469] [DEBUG] [VALIDATOR ] HUBB: All expected trading days are present. [2025-10-20T19:16:49.376555] [DEBUG] [VALIDATOR ] HUBB: Primary data validation passed. [2025-10-20T19:16:49.377329] [INFO ] [MAIN ] HUBB: Fetched primary data is valid. [2025-10-20T19:16:49.377392] [DEBUG] [LOCAL ] HUBB: Saving primary data. [2025-10-20T19:16:49.378543] [DEBUG] [LOCAL ] HUBB: Trimmed 54278 records to 36261 (keeping 200 days of data) [2025-10-20T19:16:49.604875] [DEBUG] [LOCAL ] HUBB: Primary data saved (36261 records). [2025-10-20T19:16:49.605083] [INFO ] [MAIN ] HUBB: Full history saved. [2025-10-20T19:16:49.605182] [INFO ] [MAIN ] Update Progress: 1063/2367 symbols completed. HUBB took 34.4s. Estimated remaining: 2h 47m 45s. Rate: 7.8 symbols/min [2025-10-20T19:16:49.774121] [INFO ] [MAIN ] HUYA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:49.816418] [DEBUG] [LOCAL ] HUYA: Loaded 44415 existing records from primary storage. [2025-10-20T19:16:49.816777] [INFO ] [MAIN ] HUYA: Loaded 44415 existing records. [2025-10-20T19:16:49.859731] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-02. [2025-10-20T19:16:50.004478] [WARN ] [MAIN ] HUYA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:50.004822] [INFO ] [MAIN ] HUYA: Wiping existing primary data before full refresh. [2025-10-20T19:16:50.004881] [INFO ] [LOCAL ] HUYA: Starting deletion of primary data. [2025-10-20T19:16:50.005419] [DEBUG] [LOCAL ] HUYA: Deleted primary data file. [2025-10-20T19:16:50.005477] [INFO ] [MAIN ] HUYA: Starting full primary refresh. [2025-10-20T19:16:50.313765] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T19:16:50.404721] [INFO ] [REMOTE ] HUYA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:50.651376] [INFO ] [REMOTE ] HUBS: Fetching primary data for 2025-10. [2025-10-20T19:16:50.917931] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:16:51.451356] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-05. [2025-10-20T19:16:51.582980] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:16:52.251407] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-03. [2025-10-20T19:16:52.384900] [INFO ] [REMOTE ] HUBS: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T19:16:52.391496] [INFO ] [REMOTE ] HUBS: Full fetch complete (56978 records). [2025-10-20T19:16:52.391636] [DEBUG] [VALIDATOR ] HUBS: Validating primary data. [2025-10-20T19:16:52.775184] [DEBUG] [VALIDATOR ] HUBS: All expected trading days are present. [2025-10-20T19:16:52.775285] [DEBUG] [VALIDATOR ] HUBS: Primary data validation passed. [2025-10-20T19:16:52.776100] [INFO ] [MAIN ] HUBS: Fetched primary data is valid. [2025-10-20T19:16:52.776161] [DEBUG] [LOCAL ] HUBS: Saving primary data. [2025-10-20T19:16:52.777243] [DEBUG] [LOCAL ] HUBS: Trimmed 56978 records to 40086 (keeping 200 days of data) [2025-10-20T19:16:53.013456] [DEBUG] [LOCAL ] HUBS: Primary data saved (40086 records). [2025-10-20T19:16:53.013572] [INFO ] [MAIN ] HUBS: Full history saved. [2025-10-20T19:16:53.013671] [INFO ] [MAIN ] Update Progress: 1064/2367 symbols completed. HUBS took 29.4s. Estimated remaining: 2h 47m 32s. Rate: 7.8 symbols/min [2025-10-20T19:16:53.078456] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-01. [2025-10-20T19:16:53.202446] [INFO ] [MAIN ] HVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:16:53.244482] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:16:53.264165] [DEBUG] [LOCAL ] HVT: Loaded 15340 existing records from primary storage. [2025-10-20T19:16:53.265212] [INFO ] [MAIN ] HVT: Loaded 15340 existing records. [2025-10-20T19:16:53.450044] [WARN ] [MAIN ] HVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:16:53.450282] [INFO ] [MAIN ] HVT: Wiping existing primary data before full refresh. [2025-10-20T19:16:53.450336] [INFO ] [LOCAL ] HVT: Starting deletion of primary data. [2025-10-20T19:16:53.450525] [DEBUG] [LOCAL ] HVT: Deleted primary data file. [2025-10-20T19:16:53.450581] [INFO ] [MAIN ] HVT: Starting full primary refresh. [2025-10-20T19:16:53.834914] [INFO ] [REMOTE ] HVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:16:53.851371] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-06. [2025-10-20T19:16:53.940330] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:16:54.651360] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-01. [2025-10-20T19:16:54.718864] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:16:55.451415] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-04. [2025-10-20T19:16:55.525748] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:16:55.670746] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:16:56.251373] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-02. [2025-10-20T19:16:57.009999] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:16:57.058881] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-03. [2025-10-20T19:16:57.080767] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:16:57.851406] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-05. [2025-10-20T19:16:58.181852] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:16:58.651429] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-04. [2025-10-20T19:16:59.451387] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-02. [2025-10-20T19:16:59.704339] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T19:16:59.735575] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T19:17:00.251395] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-06. [2025-10-20T19:17:01.051383] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-05. [2025-10-20T19:17:01.276350] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:17:01.851444] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-03. [2025-10-20T19:17:02.116791] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T19:17:02.337406] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-06 (2.09 seconds). [2025-10-20T19:17:02.651379] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-07. [2025-10-20T19:17:03.451386] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-06. [2025-10-20T19:17:03.536369] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:17:04.251362] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-07. [2025-10-20T19:17:04.373895] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:17:04.613173] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:17:05.051399] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-04. [2025-10-20T19:17:05.851358] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-07. [2025-10-20T19:17:06.127412] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:17:06.651346] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-08. [2025-10-20T19:17:07.289786] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-04 (2.24 seconds). [2025-10-20T19:17:07.451384] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-05. [2025-10-20T19:17:08.251376] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-08. [2025-10-20T19:17:08.291701] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-07 (2.44 seconds). [2025-10-20T19:17:09.051420] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-08. [2025-10-20T19:17:09.444047] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T19:17:09.572212] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-08 (2.92 seconds). [2025-10-20T19:17:09.681621] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T19:17:09.851340] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-09. [2025-10-20T19:17:10.203635] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T19:17:10.651369] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-09. [2025-10-20T19:17:11.451403] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-06. [2025-10-20T19:17:11.546851] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T19:17:11.578704] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:17:12.251386] [INFO ] [REMOTE ] HVT: Fetching primary data for 2025-10. [2025-10-20T19:17:12.915978] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:17:13.051377] [INFO ] [REMOTE ] HUM: Fetching primary data for 2025-10. [2025-10-20T19:17:13.383839] [INFO ] [REMOTE ] HVT: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:17:13.386354] [INFO ] [REMOTE ] HVT: Full fetch complete (23113 records). [2025-10-20T19:17:13.386439] [DEBUG] [VALIDATOR ] HVT: Validating primary data. [2025-10-20T19:17:13.745697] [DEBUG] [VALIDATOR ] HVT: All expected trading days are present. [2025-10-20T19:17:13.745784] [DEBUG] [VALIDATOR ] HVT: Primary data validation passed. [2025-10-20T19:17:13.746241] [INFO ] [MAIN ] HVT: Fetched primary data is valid. [2025-10-20T19:17:13.746278] [DEBUG] [LOCAL ] HVT: Saving primary data. [2025-10-20T19:17:13.746888] [DEBUG] [LOCAL ] HVT: Trimmed 23113 records to 14991 (keeping 200 days of data) [2025-10-20T19:17:13.833034] [DEBUG] [LOCAL ] HVT: Primary data saved (14991 records). [2025-10-20T19:17:13.833153] [INFO ] [MAIN ] HVT: Full history saved. [2025-10-20T19:17:13.833226] [INFO ] [MAIN ] Update Progress: 1065/2367 symbols completed. HVT took 20.8s. Estimated remaining: 2h 47m 41s. Rate: 7.8 symbols/min [2025-10-20T19:17:13.863789] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-09. [2025-10-20T19:17:14.003426] [INFO ] [MAIN ] HWM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:14.055275] [DEBUG] [LOCAL ] HWM: Loaded 53536 existing records from primary storage. [2025-10-20T19:17:14.055573] [INFO ] [MAIN ] HWM: Loaded 53536 existing records. [2025-10-20T19:17:14.228410] [WARN ] [MAIN ] HWM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:14.228618] [INFO ] [MAIN ] HWM: Wiping existing primary data before full refresh. [2025-10-20T19:17:14.228692] [INFO ] [LOCAL ] HWM: Starting deletion of primary data. [2025-10-20T19:17:14.228930] [DEBUG] [LOCAL ] HWM: Deleted primary data file. [2025-10-20T19:17:14.228986] [INFO ] [MAIN ] HWM: Starting full primary refresh. [2025-10-20T19:17:14.574314] [INFO ] [REMOTE ] HWM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:14.651357] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-01. [2025-10-20T19:17:15.032740] [INFO ] [REMOTE ] HUM: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T19:17:15.039966] [INFO ] [REMOTE ] HUM: Full fetch complete (71139 records). [2025-10-20T19:17:15.040083] [DEBUG] [VALIDATOR ] HUM: Validating primary data. [2025-10-20T19:17:15.423818] [DEBUG] [VALIDATOR ] HUM: All expected trading days are present. [2025-10-20T19:17:15.423899] [DEBUG] [VALIDATOR ] HUM: Primary data validation passed. [2025-10-20T19:17:15.424823] [INFO ] [MAIN ] HUM: Fetched primary data is valid. [2025-10-20T19:17:15.424888] [DEBUG] [LOCAL ] HUM: Saving primary data. [2025-10-20T19:17:15.426207] [DEBUG] [LOCAL ] HUM: Trimmed 71139 records to 49347 (keeping 200 days of data) [2025-10-20T19:17:15.480880] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-07. [2025-10-20T19:17:15.735460] [DEBUG] [LOCAL ] HUM: Primary data saved (49347 records). [2025-10-20T19:17:15.735783] [INFO ] [MAIN ] HUM: Full history saved. [2025-10-20T19:17:15.735906] [INFO ] [MAIN ] Update Progress: 1066/2367 symbols completed. HUM took 36.1s. Estimated remaining: 2h 47m 26s. Rate: 7.8 symbols/min [2025-10-20T19:17:15.915766] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T19:17:15.979764] [INFO ] [MAIN ] HXL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:16.018626] [DEBUG] [LOCAL ] HXL: Loaded 46542 existing records from primary storage. [2025-10-20T19:17:16.019010] [INFO ] [MAIN ] HXL: Loaded 46542 existing records. [2025-10-20T19:17:16.195593] [WARN ] [MAIN ] HXL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:16.195807] [INFO ] [MAIN ] HXL: Wiping existing primary data before full refresh. [2025-10-20T19:17:16.195853] [INFO ] [LOCAL ] HXL: Starting deletion of primary data. [2025-10-20T19:17:16.196101] [DEBUG] [LOCAL ] HXL: Deleted primary data file. [2025-10-20T19:17:16.196138] [INFO ] [MAIN ] HXL: Starting full primary refresh. [2025-10-20T19:17:16.263759] [INFO ] [REMOTE ] HUN: Fetching primary data for 2025-10. [2025-10-20T19:17:16.554386] [INFO ] [REMOTE ] HXL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:16.583354] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T19:17:17.051397] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-02. [2025-10-20T19:17:17.084184] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:17:17.851366] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-08. [2025-10-20T19:17:18.431330] [INFO ] [REMOTE ] HUN: Incremental fetch complete for 2025-10 (2.17 seconds). [2025-10-20T19:17:18.439506] [INFO ] [REMOTE ] HUN: Full fetch complete (76983 records). [2025-10-20T19:17:18.439739] [DEBUG] [VALIDATOR ] HUN: Validating primary data. [2025-10-20T19:17:18.659696] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-01. [2025-10-20T19:17:18.793317] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:17:18.892129] [DEBUG] [VALIDATOR ] HUN: All expected trading days are present. [2025-10-20T19:17:18.892218] [DEBUG] [VALIDATOR ] HUN: Primary data validation passed. [2025-10-20T19:17:18.893175] [INFO ] [MAIN ] HUN: Fetched primary data is valid. [2025-10-20T19:17:18.893240] [DEBUG] [LOCAL ] HUN: Saving primary data. [2025-10-20T19:17:18.894786] [DEBUG] [LOCAL ] HUN: Trimmed 76983 records to 53219 (keeping 200 days of data) [2025-10-20T19:17:19.208259] [DEBUG] [LOCAL ] HUN: Primary data saved (53219 records). [2025-10-20T19:17:19.208383] [INFO ] [MAIN ] HUN: Full history saved. [2025-10-20T19:17:19.208457] [INFO ] [MAIN ] Update Progress: 1067/2367 symbols completed. HUN took 35.4s. Estimated remaining: 2h 47m 13s. Rate: 7.8 symbols/min [2025-10-20T19:17:19.381567] [INFO ] [MAIN ] HY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:19.392862] [DEBUG] [LOCAL ] HY: Loaded 10191 existing records from primary storage. [2025-10-20T19:17:19.393037] [INFO ] [MAIN ] HY: Loaded 10191 existing records. [2025-10-20T19:17:19.456236] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-03. [2025-10-20T19:17:19.605200] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:17:19.609004] [WARN ] [MAIN ] HY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:19.618480] [INFO ] [MAIN ] HY: Wiping existing primary data before full refresh. [2025-10-20T19:17:19.618678] [INFO ] [LOCAL ] HY: Starting deletion of primary data. [2025-10-20T19:17:19.619129] [DEBUG] [LOCAL ] HY: Deleted primary data file. [2025-10-20T19:17:19.619191] [INFO ] [MAIN ] HY: Starting full primary refresh. [2025-10-20T19:17:19.970512] [INFO ] [REMOTE ] HY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:20.251342] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-01. [2025-10-20T19:17:20.297629] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:17:20.837699] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T19:17:21.051398] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-02. [2025-10-20T19:17:21.106711] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:17:21.851365] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-04. [2025-10-20T19:17:21.876577] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:17:22.651415] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-03. [2025-10-20T19:17:23.451392] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-09. [2025-10-20T19:17:23.664298] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:17:23.698400] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:17:24.251370] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-05. [2025-10-20T19:17:25.041018] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:17:25.051342] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-02. [2025-10-20T19:17:25.841220] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T19:17:25.851699] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-04. [2025-10-20T19:17:26.392607] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T19:17:26.656195] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-03. [2025-10-20T19:17:26.658949] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T19:17:27.451376] [INFO ] [REMOTE ] HUYA: Fetching primary data for 2025-10. [2025-10-20T19:17:28.167687] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T19:17:28.251428] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-04. [2025-10-20T19:17:28.596074] [INFO ] [REMOTE ] HUYA: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T19:17:28.608737] [INFO ] [REMOTE ] HUYA: Full fetch complete (63990 records). [2025-10-20T19:17:28.608988] [DEBUG] [VALIDATOR ] HUYA: Validating primary data. [2025-10-20T19:17:29.056551] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-05. [2025-10-20T19:17:29.157809] [DEBUG] [VALIDATOR ] HUYA: All expected trading days are present. [2025-10-20T19:17:29.157921] [DEBUG] [VALIDATOR ] HUYA: Primary data validation passed. [2025-10-20T19:17:29.158973] [INFO ] [MAIN ] HUYA: Fetched primary data is valid. [2025-10-20T19:17:29.159043] [DEBUG] [LOCAL ] HUYA: Saving primary data. [2025-10-20T19:17:29.160611] [DEBUG] [LOCAL ] HUYA: Trimmed 63990 records to 43607 (keeping 200 days of data) [2025-10-20T19:17:29.414277] [DEBUG] [LOCAL ] HUYA: Primary data saved (43607 records). [2025-10-20T19:17:29.414404] [INFO ] [MAIN ] HUYA: Full history saved. [2025-10-20T19:17:29.414481] [INFO ] [MAIN ] Update Progress: 1068/2367 symbols completed. HUYA took 39.8s. Estimated remaining: 2h 47m 8s. Rate: 7.8 symbols/min [2025-10-20T19:17:29.586321] [INFO ] [MAIN ] HYAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:29.592674] [DEBUG] [LOCAL ] HYAC: No existing data file found in primary storage. [2025-10-20T19:17:29.593392] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:17:29.593169] [WARN ] [MAIN ] HYAC: No existing data found, performing full refresh instead. [2025-10-20T19:17:29.596539] [INFO ] [MAIN ] HYAC: Wiping existing primary data before full refresh. [2025-10-20T19:17:29.596686] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-10-20T19:17:29.596754] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-10-20T19:17:29.596791] [INFO ] [MAIN ] HYAC: Starting full primary refresh. [2025-10-20T19:17:29.854939] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-06. [2025-10-20T19:17:29.966820] [INFO ] [REMOTE ] HYAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:30.007440] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T19:17:30.651367] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-05. [2025-10-20T19:17:31.451392] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-06. [2025-10-20T19:17:31.887372] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T19:17:31.923924] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T19:17:32.251414] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-07. [2025-10-20T19:17:32.330368] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:17:33.051376] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-06. [2025-10-20T19:17:33.851365] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-07. [2025-10-20T19:17:34.637272] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T19:17:34.651348] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-08. [2025-10-20T19:17:34.860304] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-07 (2.61 seconds). [2025-10-20T19:17:35.097639] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:17:35.175904] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-06 (2.12 seconds). [2025-10-20T19:17:35.451375] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-09. [2025-10-20T19:17:36.249232] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T19:17:36.253274] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-01. [2025-10-20T19:17:36.586128] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T19:17:37.051384] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-08. [2025-10-20T19:17:37.851379] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-07. [2025-10-20T19:17:38.651396] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-02. [2025-10-20T19:17:38.894339] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:17:38.976805] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-02 (0.33 seconds). [2025-10-20T19:17:39.450834] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:17:39.460205] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-03. [2025-10-20T19:17:39.685118] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-03 (0.22 seconds). [2025-10-20T19:17:40.251407] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-04. [2025-10-20T19:17:40.577615] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-04 (0.33 seconds). [2025-10-20T19:17:41.051373] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-05. [2025-10-20T19:17:41.364530] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T19:17:41.851362] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-09. [2025-10-20T19:17:42.651380] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-08. [2025-10-20T19:17:43.451380] [INFO ] [REMOTE ] HY: Fetching primary data for 2025-10. [2025-10-20T19:17:43.600067] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:17:44.259676] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-06. [2025-10-20T19:17:44.270343] [INFO ] [REMOTE ] HY: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:17:44.272970] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T19:17:44.283127] [INFO ] [REMOTE ] HY: Full fetch complete (14978 records). [2025-10-20T19:17:44.283625] [DEBUG] [VALIDATOR ] HY: Validating primary data. [2025-10-20T19:17:44.640181] [DEBUG] [VALIDATOR ] HY: All expected trading days are present. [2025-10-20T19:17:44.642502] [DEBUG] [VALIDATOR ] HY: Primary data validation passed. [2025-10-20T19:17:44.642590] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T19:17:44.643169] [INFO ] [MAIN ] HY: Fetched primary data is valid. [2025-10-20T19:17:44.644198] [DEBUG] [LOCAL ] HY: Saving primary data. [2025-10-20T19:17:44.644766] [DEBUG] [LOCAL ] HY: Trimmed 14978 records to 10046 (keeping 200 days of data) [2025-10-20T19:17:44.702526] [DEBUG] [LOCAL ] HY: Primary data saved (10046 records). [2025-10-20T19:17:44.702670] [INFO ] [MAIN ] HY: Full history saved. [2025-10-20T19:17:44.702764] [INFO ] [MAIN ] Update Progress: 1069/2367 symbols completed. HY took 25.5s. Estimated remaining: 2h 47m 10s. Rate: 7.8 symbols/min [2025-10-20T19:17:44.868622] [INFO ] [MAIN ] HYBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:44.868841] [DEBUG] [LOCAL ] HYBX: No existing data file found in primary storage. [2025-10-20T19:17:44.868894] [WARN ] [MAIN ] HYBX: No existing data found, performing full refresh instead. [2025-10-20T19:17:44.868999] [INFO ] [MAIN ] HYBX: Wiping existing primary data before full refresh. [2025-10-20T19:17:44.869040] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-10-20T19:17:44.869079] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-10-20T19:17:44.869110] [INFO ] [MAIN ] HYBX: Starting full primary refresh. [2025-10-20T19:17:45.056703] [INFO ] [REMOTE ] HWM: Fetching primary data for 2025-10. [2025-10-20T19:17:45.221876] [INFO ] [REMOTE ] HYBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:45.851366] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-01. [2025-10-20T19:17:46.185347] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T19:17:46.651366] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-09. [2025-10-20T19:17:47.204748] [INFO ] [REMOTE ] HWM: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T19:17:47.212247] [INFO ] [REMOTE ] HWM: Full fetch complete (76649 records). [2025-10-20T19:17:47.212411] [DEBUG] [VALIDATOR ] HWM: Validating primary data. [2025-10-20T19:17:47.462505] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-02. [2025-10-20T19:17:47.603394] [DEBUG] [VALIDATOR ] HWM: All expected trading days are present. [2025-10-20T19:17:47.603477] [DEBUG] [VALIDATOR ] HWM: Primary data validation passed. [2025-10-20T19:17:47.604417] [INFO ] [MAIN ] HWM: Fetched primary data is valid. [2025-10-20T19:17:47.604479] [DEBUG] [LOCAL ] HWM: Saving primary data. [2025-10-20T19:17:47.605885] [DEBUG] [LOCAL ] HWM: Trimmed 76649 records to 52717 (keeping 200 days of data) [2025-10-20T19:17:47.822881] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T19:17:47.934495] [DEBUG] [LOCAL ] HWM: Primary data saved (52717 records). [2025-10-20T19:17:47.934662] [INFO ] [MAIN ] HWM: Full history saved. [2025-10-20T19:17:47.934808] [INFO ] [MAIN ] Update Progress: 1070/2367 symbols completed. HWM took 34.1s. Estimated remaining: 2h 46m 57s. Rate: 7.8 symbols/min [2025-10-20T19:17:48.102601] [INFO ] [MAIN ] HYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:48.114429] [DEBUG] [LOCAL ] HYI: Loaded 9962 existing records from primary storage. [2025-10-20T19:17:48.114700] [INFO ] [MAIN ] HYI: Loaded 9962 existing records. [2025-10-20T19:17:48.255358] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-03. [2025-10-20T19:17:48.295944] [WARN ] [MAIN ] HYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:48.296426] [INFO ] [MAIN ] HYI: Wiping existing primary data before full refresh. [2025-10-20T19:17:48.296633] [INFO ] [LOCAL ] HYI: Starting deletion of primary data. [2025-10-20T19:17:48.297042] [DEBUG] [LOCAL ] HYI: Deleted primary data file. [2025-10-20T19:17:48.297208] [INFO ] [MAIN ] HYI: Starting full primary refresh. [2025-10-20T19:17:48.445342] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:17:48.657997] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T19:17:48.719976] [INFO ] [REMOTE ] HYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:49.051368] [INFO ] [REMOTE ] HXL: Fetching primary data for 2025-10. [2025-10-20T19:17:49.851427] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-01. [2025-10-20T19:17:50.576994] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:17:50.651379] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-04. [2025-10-20T19:17:50.654395] [INFO ] [REMOTE ] HXL: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T19:17:50.664747] [INFO ] [REMOTE ] HXL: Full fetch complete (66634 records). [2025-10-20T19:17:50.664933] [DEBUG] [VALIDATOR ] HXL: Validating primary data. [2025-10-20T19:17:51.041601] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-10-20T19:17:51.059666] [DEBUG] [VALIDATOR ] HXL: All expected trading days are present. [2025-10-20T19:17:51.059726] [DEBUG] [VALIDATOR ] HXL: Primary data validation passed. [2025-10-20T19:17:51.060552] [INFO ] [MAIN ] HXL: Fetched primary data is valid. [2025-10-20T19:17:51.060591] [DEBUG] [LOCAL ] HXL: Saving primary data. [2025-10-20T19:17:51.061799] [DEBUG] [LOCAL ] HXL: Trimmed 66634 records to 45814 (keeping 200 days of data) [2025-10-20T19:17:51.328961] [DEBUG] [LOCAL ] HXL: Primary data saved (45814 records). [2025-10-20T19:17:51.329079] [INFO ] [MAIN ] HXL: Full history saved. [2025-10-20T19:17:51.329153] [INFO ] [MAIN ] Update Progress: 1071/2367 symbols completed. HXL took 35.6s. Estimated remaining: 2h 46m 44s. Rate: 7.8 symbols/min [2025-10-20T19:17:51.466139] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-02. [2025-10-20T19:17:51.496457] [INFO ] [MAIN ] HYT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:17:51.528812] [DEBUG] [LOCAL ] HYT: Loaded 34156 existing records from primary storage. [2025-10-20T19:17:51.529080] [INFO ] [MAIN ] HYT: Loaded 34156 existing records. [2025-10-20T19:17:51.705425] [WARN ] [MAIN ] HYT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:17:51.705621] [INFO ] [MAIN ] HYT: Wiping existing primary data before full refresh. [2025-10-20T19:17:51.705700] [INFO ] [LOCAL ] HYT: Starting deletion of primary data. [2025-10-20T19:17:51.705944] [DEBUG] [LOCAL ] HYT: Deleted primary data file. [2025-10-20T19:17:51.706000] [INFO ] [MAIN ] HYT: Starting full primary refresh. [2025-10-20T19:17:52.087335] [INFO ] [REMOTE ] HYT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:17:52.251376] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-05. [2025-10-20T19:17:52.261594] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T19:17:52.557379] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T19:17:53.051379] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-03. [2025-10-20T19:17:53.851371] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-07. [2025-10-20T19:17:53.869102] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T19:17:54.278833] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T19:17:54.651418] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-06. [2025-10-20T19:17:55.102171] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T19:17:55.451371] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-04. [2025-10-20T19:17:56.251374] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-01. [2025-10-20T19:17:57.051392] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-07. [2025-10-20T19:17:57.399145] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T19:17:57.407763] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T19:17:57.851334] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-05. [2025-10-20T19:17:58.267865] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T19:17:58.609008] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:17:58.651336] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-02. [2025-10-20T19:17:59.451386] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-06. [2025-10-20T19:17:59.898152] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:18:00.137690] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:18:00.251391] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-07. [2025-10-20T19:18:01.051349] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-03. [2025-10-20T19:18:01.209807] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:18:01.851372] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-08. [2025-10-20T19:18:02.095751] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-08 (0.24 seconds). [2025-10-20T19:18:02.468218] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:18:02.651382] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-09. [2025-10-20T19:18:02.887883] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-09 (0.24 seconds). [2025-10-20T19:18:03.451367] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-04. [2025-10-20T19:18:04.251393] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-08. [2025-10-20T19:18:04.961537] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:18:04.977808] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:18:05.051381] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-08. [2025-10-20T19:18:05.365252] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-08 (0.31 seconds). [2025-10-20T19:18:05.851396] [INFO ] [REMOTE ] HYAC: Fetching primary data for 2025-10. [2025-10-20T19:18:06.276589] [INFO ] [REMOTE ] HYAC: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T19:18:06.277568] [INFO ] [REMOTE ] HYAC: Full fetch complete (923 records). [2025-10-20T19:18:06.277637] [DEBUG] [VALIDATOR ] HYAC: Validating primary data. [2025-10-20T19:18:06.453514] [WARN ] [VALIDATOR ] HYAC: Validation failed - insufficient historical data. Have 166 unique days, need 200 days [2025-10-20T19:18:06.453705] [FATAL] [MAIN ] HYAC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:18:06.453759] [WARN ] [MAIN ] HYAC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:18:06.453799] [INFO ] [LOCAL ] HYAC: Starting deletion of primary data. [2025-10-20T19:18:06.453872] [DEBUG] [LOCAL ] HYAC: No primary data file found to delete. [2025-10-20T19:18:06.453907] [INFO ] [LOCAL ] HYAC: Starting deletion of derivative data. [2025-10-20T19:18:06.453946] [DEBUG] [LOCAL ] HYAC: No derivative data file found to delete. [2025-10-20T19:18:06.453998] [INFO ] [MAIN ] Update Progress: 1072/2367 symbols completed. HYAC took 37.0s. Estimated remaining: 2h 46m 45s. Rate: 7.8 symbols/min [2025-10-20T19:18:06.625419] [INFO ] [MAIN ] HYTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:06.630068] [DEBUG] [LOCAL ] HYTR: Loaded 3189 existing records from primary storage. [2025-10-20T19:18:06.630234] [INFO ] [MAIN ] HYTR: Loaded 3189 existing records. [2025-10-20T19:18:06.658796] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-09. [2025-10-20T19:18:06.809829] [WARN ] [MAIN ] HYTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:06.810065] [INFO ] [MAIN ] HYTR: Wiping existing primary data before full refresh. [2025-10-20T19:18:06.810122] [INFO ] [LOCAL ] HYTR: Starting deletion of primary data. [2025-10-20T19:18:06.810411] [DEBUG] [LOCAL ] HYTR: Deleted primary data file. [2025-10-20T19:18:06.810469] [INFO ] [MAIN ] HYTR: Starting full primary refresh. [2025-10-20T19:18:07.097992] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T19:18:07.177553] [INFO ] [REMOTE ] HYTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:07.451449] [INFO ] [REMOTE ] HYBX: Fetching primary data for 2025-10. [2025-10-20T19:18:07.808372] [INFO ] [REMOTE ] HYBX: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T19:18:07.809317] [INFO ] [REMOTE ] HYBX: Full fetch complete (704 records). [2025-10-20T19:18:07.809396] [DEBUG] [VALIDATOR ] HYBX: Validating primary data. [2025-10-20T19:18:08.003456] [WARN ] [VALIDATOR ] HYBX: Validation failed - insufficient historical data. Have 179 unique days, need 200 days [2025-10-20T19:18:08.003732] [FATAL] [MAIN ] HYBX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:18:08.003803] [WARN ] [MAIN ] HYBX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:18:08.003875] [INFO ] [LOCAL ] HYBX: Starting deletion of primary data. [2025-10-20T19:18:08.003972] [DEBUG] [LOCAL ] HYBX: No primary data file found to delete. [2025-10-20T19:18:08.004028] [INFO ] [LOCAL ] HYBX: Starting deletion of derivative data. [2025-10-20T19:18:08.004071] [DEBUG] [LOCAL ] HYBX: No derivative data file found to delete. [2025-10-20T19:18:08.004150] [INFO ] [MAIN ] Update Progress: 1073/2367 symbols completed. HYBX took 23.3s. Estimated remaining: 2h 46m 30s. Rate: 7.8 symbols/min [2025-10-20T19:18:08.170895] [INFO ] [MAIN ] HZO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:08.199281] [DEBUG] [LOCAL ] HZO: Loaded 29442 existing records from primary storage. [2025-10-20T19:18:08.199571] [INFO ] [MAIN ] HZO: Loaded 29442 existing records. [2025-10-20T19:18:08.260414] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-01. [2025-10-20T19:18:08.379839] [WARN ] [MAIN ] HZO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:08.380049] [INFO ] [MAIN ] HZO: Wiping existing primary data before full refresh. [2025-10-20T19:18:08.380099] [INFO ] [LOCAL ] HZO: Starting deletion of primary data. [2025-10-20T19:18:08.380448] [DEBUG] [LOCAL ] HZO: Deleted primary data file. [2025-10-20T19:18:08.380506] [INFO ] [MAIN ] HZO: Starting full primary refresh. [2025-10-20T19:18:08.752268] [INFO ] [REMOTE ] HZO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:08.782966] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T19:18:09.051421] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-05. [2025-10-20T19:18:09.851384] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-01. [2025-10-20T19:18:10.420809] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:18:10.651353] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-02. [2025-10-20T19:18:10.994839] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:18:11.197970] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T19:18:11.451363] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-03. [2025-10-20T19:18:12.005155] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:18:12.251379] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-02. [2025-10-20T19:18:13.051344] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-06. [2025-10-20T19:18:13.365311] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:18:13.851409] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-09. [2025-10-20T19:18:14.171896] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:18:14.600384] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T19:18:14.651401] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-07. [2025-10-20T19:18:15.451384] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-03. [2025-10-20T19:18:16.230167] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:18:16.251335] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-08. [2025-10-20T19:18:16.826017] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T19:18:17.051583] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-04. [2025-10-20T19:18:17.386919] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T19:18:17.755038] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:18:17.851347] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-09. [2025-10-20T19:18:18.651380] [INFO ] [REMOTE ] HYI: Fetching primary data for 2025-10. [2025-10-20T19:18:19.277494] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T19:18:19.451346] [INFO ] [REMOTE ] HYT: Fetching primary data for 2025-10. [2025-10-20T19:18:19.551959] [INFO ] [REMOTE ] HYI: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T19:18:19.554380] [INFO ] [REMOTE ] HYI: Full fetch complete (13967 records). [2025-10-20T19:18:19.554457] [DEBUG] [VALIDATOR ] HYI: Validating primary data. [2025-10-20T19:18:19.911128] [DEBUG] [VALIDATOR ] HYI: All expected trading days are present. [2025-10-20T19:18:19.911203] [DEBUG] [VALIDATOR ] HYI: Primary data validation passed. [2025-10-20T19:18:19.911495] [INFO ] [MAIN ] HYI: Fetched primary data is valid. [2025-10-20T19:18:19.911533] [DEBUG] [LOCAL ] HYI: Saving primary data. [2025-10-20T19:18:19.912039] [DEBUG] [LOCAL ] HYI: Trimmed 13967 records to 9771 (keeping 200 days of data) [2025-10-20T19:18:19.970342] [DEBUG] [LOCAL ] HYI: Primary data saved (9771 records). [2025-10-20T19:18:19.970478] [INFO ] [MAIN ] HYI: Full history saved. [2025-10-20T19:18:19.970553] [INFO ] [MAIN ] Update Progress: 1074/2367 symbols completed. HYI took 32.0s. Estimated remaining: 2h 46m 27s. Rate: 7.8 symbols/min [2025-10-20T19:18:20.140266] [INFO ] [MAIN ] IAE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:20.147884] [DEBUG] [LOCAL ] IAE: Loaded 6362 existing records from primary storage. [2025-10-20T19:18:20.148115] [INFO ] [MAIN ] IAE: Loaded 6362 existing records. [2025-10-20T19:18:20.256521] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-05. [2025-10-20T19:18:20.326245] [WARN ] [MAIN ] IAE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:20.326420] [INFO ] [MAIN ] IAE: Wiping existing primary data before full refresh. [2025-10-20T19:18:20.326465] [INFO ] [LOCAL ] IAE: Starting deletion of primary data. [2025-10-20T19:18:20.326633] [DEBUG] [LOCAL ] IAE: Deleted primary data file. [2025-10-20T19:18:20.326710] [INFO ] [MAIN ] IAE: Starting full primary refresh. [2025-10-20T19:18:20.681358] [INFO ] [REMOTE ] IAE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:20.793007] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T19:18:20.809544] [INFO ] [REMOTE ] HYT: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T19:18:20.814990] [INFO ] [REMOTE ] HYT: Full fetch complete (47953 records). [2025-10-20T19:18:20.815131] [DEBUG] [VALIDATOR ] HYT: Validating primary data. [2025-10-20T19:18:21.056608] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-04. [2025-10-20T19:18:21.208952] [DEBUG] [VALIDATOR ] HYT: All expected trading days are present. [2025-10-20T19:18:21.209042] [DEBUG] [VALIDATOR ] HYT: Primary data validation passed. [2025-10-20T19:18:21.209820] [INFO ] [MAIN ] HYT: Fetched primary data is valid. [2025-10-20T19:18:21.209885] [DEBUG] [LOCAL ] HYT: Saving primary data. [2025-10-20T19:18:21.210995] [DEBUG] [LOCAL ] HYT: Trimmed 47953 records to 33697 (keeping 200 days of data) [2025-10-20T19:18:21.412977] [DEBUG] [LOCAL ] HYT: Primary data saved (33697 records). [2025-10-20T19:18:21.413118] [INFO ] [MAIN ] HYT: Full history saved. [2025-10-20T19:18:21.413198] [INFO ] [MAIN ] Update Progress: 1075/2367 symbols completed. HYT took 30.1s. Estimated remaining: 2h 46m 12s. Rate: 7.8 symbols/min [2025-10-20T19:18:21.583711] [INFO ] [MAIN ] IAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:21.634275] [DEBUG] [LOCAL ] IAG: Loaded 54268 existing records from primary storage. [2025-10-20T19:18:21.634626] [INFO ] [MAIN ] IAG: Loaded 54268 existing records. [2025-10-20T19:18:21.813241] [WARN ] [MAIN ] IAG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:21.813448] [INFO ] [MAIN ] IAG: Wiping existing primary data before full refresh. [2025-10-20T19:18:21.813493] [INFO ] [LOCAL ] IAG: Starting deletion of primary data. [2025-10-20T19:18:21.813738] [DEBUG] [LOCAL ] IAG: Deleted primary data file. [2025-10-20T19:18:21.813794] [INFO ] [MAIN ] IAG: Starting full primary refresh. [2025-10-20T19:18:21.856687] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-01. [2025-10-20T19:18:22.190182] [INFO ] [REMOTE ] IAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:22.536330] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T19:18:22.560200] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T19:18:22.651341] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-05. [2025-10-20T19:18:23.451397] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-01. [2025-10-20T19:18:23.885234] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T19:18:24.251376] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-02. [2025-10-20T19:18:24.868950] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T19:18:25.051403] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-03. [2025-10-20T19:18:25.260681] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:18:25.737287] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T19:18:25.851394] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-06. [2025-10-20T19:18:26.651401] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-06. [2025-10-20T19:18:26.799912] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:18:27.211446] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T19:18:27.451414] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-07. [2025-10-20T19:18:28.251436] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-07. [2025-10-20T19:18:28.766843] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T19:18:28.790685] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:18:29.051464] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-08. [2025-10-20T19:18:29.851406] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-04. [2025-10-20T19:18:30.335880] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:18:30.432407] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T19:18:30.651415] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-02. [2025-10-20T19:18:31.451461] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-08. [2025-10-20T19:18:31.900353] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:18:32.252056] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-09. [2025-10-20T19:18:32.257839] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:18:33.051393] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-03. [2025-10-20T19:18:33.401606] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:18:33.851440] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-05. [2025-10-20T19:18:34.462733] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:18:34.651344] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-09. [2025-10-20T19:18:34.797982] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:18:35.451372] [INFO ] [REMOTE ] HZO: Fetching primary data for 2025-10. [2025-10-20T19:18:36.251397] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-04. [2025-10-20T19:18:36.707610] [INFO ] [REMOTE ] HZO: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:18:36.711969] [INFO ] [REMOTE ] HZO: Full fetch complete (41270 records). [2025-10-20T19:18:36.712099] [DEBUG] [VALIDATOR ] HZO: Validating primary data. [2025-10-20T19:18:37.058603] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-06. [2025-10-20T19:18:37.103555] [DEBUG] [VALIDATOR ] HZO: All expected trading days are present. [2025-10-20T19:18:37.103684] [DEBUG] [VALIDATOR ] HZO: Primary data validation passed. [2025-10-20T19:18:37.104451] [INFO ] [MAIN ] HZO: Fetched primary data is valid. [2025-10-20T19:18:37.104516] [DEBUG] [LOCAL ] HZO: Saving primary data. [2025-10-20T19:18:37.105542] [DEBUG] [LOCAL ] HZO: Trimmed 41270 records to 28914 (keeping 200 days of data) [2025-10-20T19:18:37.288373] [DEBUG] [LOCAL ] HZO: Primary data saved (28914 records). [2025-10-20T19:18:37.288483] [INFO ] [MAIN ] HZO: Full history saved. [2025-10-20T19:18:37.288558] [INFO ] [MAIN ] Update Progress: 1076/2367 symbols completed. HZO took 29.3s. Estimated remaining: 2h 46m 14s. Rate: 7.8 symbols/min [2025-10-20T19:18:37.481161] [INFO ] [MAIN ] IBM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:18:37.535868] [DEBUG] [LOCAL ] IBM: Loaded 54411 existing records from primary storage. [2025-10-20T19:18:37.536269] [INFO ] [MAIN ] IBM: Loaded 54411 existing records. [2025-10-20T19:18:37.762491] [WARN ] [MAIN ] IBM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:18:37.762789] [INFO ] [MAIN ] IBM: Wiping existing primary data before full refresh. [2025-10-20T19:18:37.762864] [INFO ] [LOCAL ] IBM: Starting deletion of primary data. [2025-10-20T19:18:37.763135] [DEBUG] [LOCAL ] IBM: Deleted primary data file. [2025-10-20T19:18:37.763202] [INFO ] [MAIN ] IBM: Starting full primary refresh. [2025-10-20T19:18:37.909318] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T19:18:37.933242] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-07. [2025-10-20T19:18:38.310749] [INFO ] [REMOTE ] IBM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:18:38.366711] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T19:18:38.649898] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T19:18:38.651585] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-01. [2025-10-20T19:18:39.451426] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-08. [2025-10-20T19:18:40.061735] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:18:40.251444] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-05. [2025-10-20T19:18:40.801299] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-01 (2.15 seconds). [2025-10-20T19:18:41.051430] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-09. [2025-10-20T19:18:41.826478] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:18:41.851343] [INFO ] [REMOTE ] IAE: Fetching primary data for 2025-10. [2025-10-20T19:18:41.882983] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-09 (7.23 seconds). [2025-10-20T19:18:42.092771] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T19:18:42.651463] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-06. [2025-10-20T19:18:42.689980] [INFO ] [REMOTE ] IAE: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T19:18:42.692140] [INFO ] [REMOTE ] IAE: Full fetch complete (9029 records). [2025-10-20T19:18:42.692218] [DEBUG] [VALIDATOR ] IAE: Validating primary data. [2025-10-20T19:18:43.086858] [DEBUG] [VALIDATOR ] IAE: All expected trading days are present. [2025-10-20T19:18:43.086970] [DEBUG] [VALIDATOR ] IAE: Primary data validation passed. [2025-10-20T19:18:43.087361] [INFO ] [MAIN ] IAE: Fetched primary data is valid. [2025-10-20T19:18:43.087407] [DEBUG] [LOCAL ] IAE: Saving primary data. [2025-10-20T19:18:43.087942] [DEBUG] [LOCAL ] IAE: Trimmed 9029 records to 6325 (keeping 200 days of data) [2025-10-20T19:18:43.127281] [DEBUG] [LOCAL ] IAE: Primary data saved (6325 records). [2025-10-20T19:18:43.127468] [INFO ] [MAIN ] IAE: Full history saved. [2025-10-20T19:18:43.127566] [INFO ] [MAIN ] Update Progress: 1077/2367 symbols completed. IAE took 23.2s. Estimated remaining: 2h 46m 4s. Rate: 7.8 symbols/min [2025-10-20T19:18:43.451371] [INFO ] [REMOTE ] HYTR: Fetching primary data for 2025-10. [2025-10-20T19:18:44.021529] [INFO ] [REMOTE ] HYTR: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T19:18:44.022588] [INFO ] [REMOTE ] HYTR: Full fetch complete (4665 records). [2025-10-20T19:18:44.022690] [DEBUG] [VALIDATOR ] HYTR: Validating primary data. [2025-10-20T19:18:44.251733] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-02. [2025-10-20T19:18:44.400754] [DEBUG] [VALIDATOR ] HYTR: All expected trading days are present. [2025-10-20T19:18:44.400857] [DEBUG] [VALIDATOR ] HYTR: Primary data validation passed. [2025-10-20T19:18:44.401074] [INFO ] [MAIN ] HYTR: Fetched primary data is valid. [2025-10-20T19:18:44.401184] [DEBUG] [LOCAL ] HYTR: Saving primary data. [2025-10-20T19:18:44.430729] [DEBUG] [LOCAL ] HYTR: Trimmed 4665 records to 3098 (keeping 200 days of data) [2025-10-20T19:18:44.446995] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:18:44.468394] [DEBUG] [LOCAL ] HYTR: Primary data saved (3098 records). [2025-10-20T19:18:44.468516] [INFO ] [MAIN ] HYTR: Full history saved. [2025-10-20T19:18:44.468591] [INFO ] [MAIN ] Update Progress: 1078/2367 symbols completed. HYTR took 38.0s. Estimated remaining: 2h 45m 49s. Rate: 7.8 symbols/min [2025-10-20T19:18:45.051398] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-07. [2025-10-20T19:18:46.071459] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T19:18:46.081496] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-03. [2025-10-20T19:18:47.015622] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:18:47.028100] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-08. [2025-10-20T19:18:48.261625] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T19:18:48.272849] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-04. [2025-10-20T19:18:48.894558] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:18:48.905889] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-09. [2025-10-20T19:18:50.449305] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T19:18:50.460432] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-05. [2025-10-20T19:18:50.908117] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T19:18:50.919624] [INFO ] [REMOTE ] IAG: Fetching primary data for 2025-10. [2025-10-20T19:18:52.370791] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:18:52.382228] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-06. [2025-10-20T19:18:52.545418] [INFO ] [REMOTE ] IAG: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:18:52.553355] [INFO ] [REMOTE ] IAG: Full fetch complete (77503 records). [2025-10-20T19:18:52.553481] [DEBUG] [VALIDATOR ] IAG: Validating primary data. [2025-10-20T19:18:52.945281] [DEBUG] [VALIDATOR ] IAG: All expected trading days are present. [2025-10-20T19:18:52.945357] [DEBUG] [VALIDATOR ] IAG: Primary data validation passed. [2025-10-20T19:18:52.946297] [INFO ] [MAIN ] IAG: Fetched primary data is valid. [2025-10-20T19:18:52.946373] [DEBUG] [LOCAL ] IAG: Saving primary data. [2025-10-20T19:18:52.947744] [DEBUG] [LOCAL ] IAG: Trimmed 77503 records to 53491 (keeping 200 days of data) [2025-10-20T19:18:53.248784] [DEBUG] [LOCAL ] IAG: Primary data saved (53491 records). [2025-10-20T19:18:53.248954] [INFO ] [MAIN ] IAG: Full history saved. [2025-10-20T19:18:53.249046] [INFO ] [MAIN ] Update Progress: 1079/2367 symbols completed. IAG took 31.8s. Estimated remaining: 2h 45m 42s. Rate: 7.8 symbols/min [2025-10-20T19:18:54.552146] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T19:18:54.563167] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-07. [2025-10-20T19:18:57.002978] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-07 (2.44 seconds). [2025-10-20T19:18:57.014957] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-08. [2025-10-20T19:18:59.537836] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T19:18:59.549499] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-09. [2025-10-20T19:19:02.188353] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-09 (2.64 seconds). [2025-10-20T19:19:02.200902] [INFO ] [REMOTE ] IBM: Fetching primary data for 2025-10. [2025-10-20T19:19:03.062445] [INFO ] [REMOTE ] IBM: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:19:03.070501] [INFO ] [REMOTE ] IBM: Full fetch complete (77740 records). [2025-10-20T19:19:03.070636] [DEBUG] [VALIDATOR ] IBM: Validating primary data. [2025-10-20T19:19:03.468721] [DEBUG] [VALIDATOR ] IBM: All expected trading days are present. [2025-10-20T19:19:03.468838] [DEBUG] [VALIDATOR ] IBM: Primary data validation passed. [2025-10-20T19:19:03.469833] [INFO ] [MAIN ] IBM: Fetched primary data is valid. [2025-10-20T19:19:03.469895] [DEBUG] [LOCAL ] IBM: Saving primary data. [2025-10-20T19:19:03.471355] [DEBUG] [LOCAL ] IBM: Trimmed 77740 records to 53629 (keeping 200 days of data) [2025-10-20T19:19:03.801190] [DEBUG] [LOCAL ] IBM: Primary data saved (53629 records). [2025-10-20T19:19:03.801338] [INFO ] [MAIN ] IBM: Full history saved. [2025-10-20T19:19:03.801429] [INFO ] [MAIN ] Update Progress: 1080/2367 symbols completed. IBM took 26.5s. Estimated remaining: 2h 45m 38s. Rate: 7.8 symbols/min [2025-10-20T19:19:04.522816] [INFO ] [MAIN ] ICE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.541357] [INFO ] [MAIN ] IBTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.565581] [INFO ] [MAIN ] IBP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.559585] [INFO ] [MAIN ] IBN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:04.608588] [DEBUG] [LOCAL ] IBTA: Loaded 37129 existing records from primary storage. [2025-10-20T19:19:04.609077] [INFO ] [MAIN ] IBTA: Loaded 37129 existing records. [2025-10-20T19:19:04.700539] [DEBUG] [LOCAL ] IBP: Loaded 29142 existing records from primary storage. [2025-10-20T19:19:04.721945] [INFO ] [MAIN ] IBP: Loaded 29142 existing records. [2025-10-20T19:19:04.871191] [DEBUG] [LOCAL ] ICE: Loaded 53798 existing records from primary storage. [2025-10-20T19:19:04.915282] [INFO ] [MAIN ] ICE: Loaded 53798 existing records. [2025-10-20T19:19:04.944265] [DEBUG] [LOCAL ] IBN: Loaded 53593 existing records from primary storage. [2025-10-20T19:19:04.958403] [WARN ] [MAIN ] IBTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.009861] [INFO ] [MAIN ] IBTA: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.035052] [INFO ] [LOCAL ] IBTA: Starting deletion of primary data. [2025-10-20T19:19:05.004291] [INFO ] [MAIN ] IBN: Loaded 53593 existing records. [2025-10-20T19:19:05.104086] [DEBUG] [LOCAL ] IBTA: Deleted primary data file. [2025-10-20T19:19:05.195459] [INFO ] [MAIN ] IBTA: Starting full primary refresh. [2025-10-20T19:19:05.208956] [WARN ] [MAIN ] IBP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.228599] [INFO ] [MAIN ] IBP: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.260111] [INFO ] [LOCAL ] IBP: Starting deletion of primary data. [2025-10-20T19:19:05.285882] [DEBUG] [LOCAL ] IBP: Deleted primary data file. [2025-10-20T19:19:05.350845] [INFO ] [MAIN ] IBP: Starting full primary refresh. [2025-10-20T19:19:05.448821] [WARN ] [MAIN ] ICE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.518148] [INFO ] [MAIN ] ICE: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.581535] [INFO ] [LOCAL ] ICE: Starting deletion of primary data. [2025-10-20T19:19:05.576138] [WARN ] [MAIN ] IBN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:05.610167] [INFO ] [MAIN ] IBN: Wiping existing primary data before full refresh. [2025-10-20T19:19:05.681705] [INFO ] [LOCAL ] IBN: Starting deletion of primary data. [2025-10-20T19:19:05.686832] [DEBUG] [LOCAL ] ICE: Deleted primary data file. [2025-10-20T19:19:05.699033] [DEBUG] [LOCAL ] IBN: Deleted primary data file. [2025-10-20T19:19:05.699166] [INFO ] [MAIN ] ICE: Starting full primary refresh. [2025-10-20T19:19:05.724823] [INFO ] [MAIN ] IBN: Starting full primary refresh. [2025-10-20T19:19:06.485001] [INFO ] [REMOTE ] IBTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.568711] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-01. [2025-10-20T19:19:06.845040] [INFO ] [REMOTE ] IBP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.885223] [INFO ] [REMOTE ] IBN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.896213] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-01. [2025-10-20T19:19:06.895794] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-01. [2025-10-20T19:19:06.928679] [INFO ] [REMOTE ] ICE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:06.929282] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-01. [2025-10-20T19:19:08.331616] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:19:08.337456] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-02. [2025-10-20T19:19:08.478835] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:19:08.484006] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-02. [2025-10-20T19:19:08.762631] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:19:08.773394] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-02. [2025-10-20T19:19:09.044798] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T19:19:09.055776] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-02. [2025-10-20T19:19:09.146386] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T19:19:09.151309] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-03. [2025-10-20T19:19:09.695569] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:19:09.701544] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-03. [2025-10-20T19:19:10.485343] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T19:19:10.497309] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-03. [2025-10-20T19:19:10.528903] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:19:10.540274] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-03. [2025-10-20T19:19:10.639459] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:19:10.649350] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-04. [2025-10-20T19:19:10.925183] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:19:10.932415] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-04. [2025-10-20T19:19:12.195830] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T19:19:12.206105] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-05. [2025-10-20T19:19:12.267965] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T19:19:12.275039] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-05. [2025-10-20T19:19:12.337186] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T19:19:12.348202] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-04. [2025-10-20T19:19:12.646208] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T19:19:12.657378] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-04. [2025-10-20T19:19:13.613091] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:19:13.650018] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-06. [2025-10-20T19:19:13.651327] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:19:13.662258] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-06. [2025-10-20T19:19:14.342580] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:19:14.354812] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-05. [2025-10-20T19:19:14.430134] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:19:14.441718] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-05. [2025-10-20T19:19:14.663166] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:19:14.669409] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-07. [2025-10-20T19:19:14.913869] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T19:19:14.920159] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-07. [2025-10-20T19:19:15.634603] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T19:19:15.641168] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-08. [2025-10-20T19:19:15.834082] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:19:15.845728] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-06. [2025-10-20T19:19:16.356570] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:19:16.368354] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-06. [2025-10-20T19:19:16.391996] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:16.399045] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-08. [2025-10-20T19:19:16.972014] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T19:19:17.051367] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-09. [2025-10-20T19:19:17.615007] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:19:17.812555] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:19:17.851407] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-07. [2025-10-20T19:19:18.063788] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:19:18.541348] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:19:18.651333] [INFO ] [REMOTE ] IBTA: Fetching primary data for 2025-10. [2025-10-20T19:19:19.317934] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:19.451357] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-08. [2025-10-20T19:19:20.112937] [INFO ] [REMOTE ] IBTA: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:19:20.118132] [INFO ] [REMOTE ] IBTA: Full fetch complete (51313 records). [2025-10-20T19:19:20.118244] [DEBUG] [VALIDATOR ] IBTA: Validating primary data. [2025-10-20T19:19:20.264709] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-09. [2025-10-20T19:19:20.513411] [DEBUG] [VALIDATOR ] IBTA: All expected trading days are present. [2025-10-20T19:19:20.513491] [DEBUG] [VALIDATOR ] IBTA: Primary data validation passed. [2025-10-20T19:19:20.514310] [INFO ] [MAIN ] IBTA: Fetched primary data is valid. [2025-10-20T19:19:20.514372] [DEBUG] [LOCAL ] IBTA: Saving primary data. [2025-10-20T19:19:20.515396] [DEBUG] [LOCAL ] IBTA: Trimmed 51313 records to 36338 (keeping 200 days of data) [2025-10-20T19:19:20.722942] [DEBUG] [LOCAL ] IBTA: Primary data saved (36338 records). [2025-10-20T19:19:20.723133] [INFO ] [MAIN ] IBTA: Full history saved. [2025-10-20T19:19:20.723228] [INFO ] [MAIN ] Update Progress: 1081/2367 symbols completed. IBTA took 16.9s. Estimated remaining: 2h 45m 41s. Rate: 7.8 symbols/min [2025-10-20T19:19:20.901918] [INFO ] [MAIN ] ICL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:20.938964] [DEBUG] [LOCAL ] ICL: Loaded 39044 existing records from primary storage. [2025-10-20T19:19:20.939275] [INFO ] [MAIN ] ICL: Loaded 39044 existing records. [2025-10-20T19:19:21.056542] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-07. [2025-10-20T19:19:21.130509] [WARN ] [MAIN ] ICL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:21.130831] [INFO ] [MAIN ] ICL: Wiping existing primary data before full refresh. [2025-10-20T19:19:21.130880] [INFO ] [LOCAL ] ICL: Starting deletion of primary data. [2025-10-20T19:19:21.131096] [DEBUG] [LOCAL ] ICL: Deleted primary data file. [2025-10-20T19:19:21.131155] [INFO ] [MAIN ] ICL: Starting full primary refresh. [2025-10-20T19:19:21.538792] [INFO ] [REMOTE ] ICL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:21.534064] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:19:21.625900] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:19:21.851367] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-09. [2025-10-20T19:19:22.651344] [INFO ] [REMOTE ] IBP: Fetching primary data for 2025-10. [2025-10-20T19:19:22.946308] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:19:23.451366] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-01. [2025-10-20T19:19:23.643669] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:19:23.877054] [INFO ] [REMOTE ] IBP: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:19:23.881151] [INFO ] [REMOTE ] IBP: Full fetch complete (41089 records). [2025-10-20T19:19:23.881261] [DEBUG] [VALIDATOR ] IBP: Validating primary data. [2025-10-20T19:19:24.265544] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-08. [2025-10-20T19:19:24.315470] [DEBUG] [VALIDATOR ] IBP: All expected trading days are present. [2025-10-20T19:19:24.315552] [DEBUG] [VALIDATOR ] IBP: Primary data validation passed. [2025-10-20T19:19:24.316239] [INFO ] [MAIN ] IBP: Fetched primary data is valid. [2025-10-20T19:19:24.316300] [DEBUG] [LOCAL ] IBP: Saving primary data. [2025-10-20T19:19:24.317144] [DEBUG] [LOCAL ] IBP: Trimmed 41089 records to 28823 (keeping 200 days of data) [2025-10-20T19:19:24.491825] [DEBUG] [LOCAL ] IBP: Primary data saved (28823 records). [2025-10-20T19:19:24.491953] [INFO ] [MAIN ] IBP: Full history saved. [2025-10-20T19:19:24.492071] [INFO ] [MAIN ] Update Progress: 1082/2367 symbols completed. IBP took 20.7s. Estimated remaining: 2h 45m 29s. Rate: 7.8 symbols/min [2025-10-20T19:19:24.670468] [INFO ] [MAIN ] IDA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:24.707617] [DEBUG] [LOCAL ] IDA: Loaded 33149 existing records from primary storage. [2025-10-20T19:19:24.707908] [INFO ] [MAIN ] IDA: Loaded 33149 existing records. [2025-10-20T19:19:24.890759] [WARN ] [MAIN ] IDA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:24.891078] [INFO ] [MAIN ] IDA: Wiping existing primary data before full refresh. [2025-10-20T19:19:24.891293] [INFO ] [LOCAL ] IDA: Starting deletion of primary data. [2025-10-20T19:19:24.911780] [DEBUG] [LOCAL ] IDA: Deleted primary data file. [2025-10-20T19:19:24.911889] [INFO ] [MAIN ] IDA: Starting full primary refresh. [2025-10-20T19:19:24.993350] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T19:19:25.061058] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-02. [2025-10-20T19:19:25.273812] [INFO ] [REMOTE ] IDA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:25.851343] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-01. [2025-10-20T19:19:26.085856] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:19:26.651356] [INFO ] [REMOTE ] IBN: Fetching primary data for 2025-10. [2025-10-20T19:19:26.741253] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:19:27.272870] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:19:27.451360] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-03. [2025-10-20T19:19:28.251407] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-02. [2025-10-20T19:19:28.549013] [INFO ] [REMOTE ] IBN: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T19:19:28.556908] [INFO ] [REMOTE ] IBN: Full fetch complete (76628 records). [2025-10-20T19:19:28.557038] [DEBUG] [VALIDATOR ] IBN: Validating primary data. [2025-10-20T19:19:28.969742] [DEBUG] [VALIDATOR ] IBN: All expected trading days are present. [2025-10-20T19:19:28.970007] [DEBUG] [VALIDATOR ] IBN: Primary data validation passed. [2025-10-20T19:19:28.971009] [INFO ] [MAIN ] IBN: Fetched primary data is valid. [2025-10-20T19:19:28.971093] [DEBUG] [LOCAL ] IBN: Saving primary data. [2025-10-20T19:19:28.994685] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:19:29.002994] [DEBUG] [LOCAL ] IBN: Trimmed 76628 records to 52844 (keeping 200 days of data) [2025-10-20T19:19:29.058615] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-04. [2025-10-20T19:19:29.327239] [DEBUG] [LOCAL ] IBN: Primary data saved (52844 records). [2025-10-20T19:19:29.341703] [INFO ] [MAIN ] IBN: Full history saved. [2025-10-20T19:19:29.342330] [INFO ] [MAIN ] Update Progress: 1083/2367 symbols completed. IBN took 25.5s. Estimated remaining: 2h 45m 17s. Rate: 7.8 symbols/min [2025-10-20T19:19:29.342907] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:19:29.515115] [INFO ] [MAIN ] IDE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:29.525980] [DEBUG] [LOCAL ] IDE: Loaded 9258 existing records from primary storage. [2025-10-20T19:19:29.526246] [INFO ] [MAIN ] IDE: Loaded 9258 existing records. [2025-10-20T19:19:29.700682] [WARN ] [MAIN ] IDE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:29.700954] [INFO ] [MAIN ] IDE: Wiping existing primary data before full refresh. [2025-10-20T19:19:29.701003] [INFO ] [LOCAL ] IDE: Starting deletion of primary data. [2025-10-20T19:19:29.701161] [DEBUG] [LOCAL ] IDE: Deleted primary data file. [2025-10-20T19:19:29.701198] [INFO ] [MAIN ] IDE: Starting full primary refresh. [2025-10-20T19:19:29.856133] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-09. [2025-10-20T19:19:30.063112] [INFO ] [REMOTE ] IDE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:30.616245] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:19:30.651432] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-03. [2025-10-20T19:19:31.451377] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-01. [2025-10-20T19:19:31.775338] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:19:31.806193] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T19:19:32.239273] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:19:32.251334] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-02. [2025-10-20T19:19:32.952550] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T19:19:33.051357] [INFO ] [REMOTE ] ICE: Fetching primary data for 2025-10. [2025-10-20T19:19:33.851424] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-05. [2025-10-20T19:19:34.651355] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-04. [2025-10-20T19:19:35.014631] [INFO ] [REMOTE ] ICE: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T19:19:35.022477] [INFO ] [REMOTE ] ICE: Full fetch complete (76991 records). [2025-10-20T19:19:35.022625] [DEBUG] [VALIDATOR ] ICE: Validating primary data. [2025-10-20T19:19:35.418424] [DEBUG] [VALIDATOR ] ICE: All expected trading days are present. [2025-10-20T19:19:35.418674] [DEBUG] [VALIDATOR ] ICE: Primary data validation passed. [2025-10-20T19:19:35.419702] [INFO ] [MAIN ] ICE: Fetched primary data is valid. [2025-10-20T19:19:35.419806] [DEBUG] [LOCAL ] ICE: Saving primary data. [2025-10-20T19:19:35.451415] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-03. [2025-10-20T19:19:35.452543] [DEBUG] [LOCAL ] ICE: Trimmed 76991 records to 53012 (keeping 200 days of data) [2025-10-20T19:19:35.453885] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:19:35.799951] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:19:35.827927] [DEBUG] [LOCAL ] ICE: Primary data saved (53012 records). [2025-10-20T19:19:35.828049] [INFO ] [MAIN ] ICE: Full history saved. [2025-10-20T19:19:35.828122] [INFO ] [MAIN ] Update Progress: 1084/2367 symbols completed. ICE took 32.0s. Estimated remaining: 2h 45m 8s. Rate: 7.8 symbols/min [2025-10-20T19:19:36.004221] [INFO ] [MAIN ] IDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:36.009967] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:19:36.027945] [DEBUG] [LOCAL ] IDT: Loaded 18788 existing records from primary storage. [2025-10-20T19:19:36.028173] [INFO ] [MAIN ] IDT: Loaded 18788 existing records. [2025-10-20T19:19:36.209343] [WARN ] [MAIN ] IDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:36.209578] [INFO ] [MAIN ] IDT: Wiping existing primary data before full refresh. [2025-10-20T19:19:36.209626] [INFO ] [LOCAL ] IDT: Starting deletion of primary data. [2025-10-20T19:19:36.209823] [DEBUG] [LOCAL ] IDT: Deleted primary data file. [2025-10-20T19:19:36.209869] [INFO ] [MAIN ] IDT: Starting full primary refresh. [2025-10-20T19:19:36.256785] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-06. [2025-10-20T19:19:36.564993] [INFO ] [REMOTE ] IDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:37.051381] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-04. [2025-10-20T19:19:37.763935] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T19:19:37.820352] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:19:37.851336] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-05. [2025-10-20T19:19:38.367299] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T19:19:38.651370] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-06. [2025-10-20T19:19:39.373632] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:19:39.451391] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-07. [2025-10-20T19:19:40.254989] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-07. [2025-10-20T19:19:40.256618] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T19:19:41.051387] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-05. [2025-10-20T19:19:41.722388] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:19:41.851413] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-08. [2025-10-20T19:19:42.607749] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T19:19:42.651436] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-01. [2025-10-20T19:19:43.248045] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T19:19:43.451347] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-06. [2025-10-20T19:19:43.524472] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T19:19:44.251401] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-02. [2025-10-20T19:19:44.628576] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T19:19:44.852875] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:19:45.051385] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-07. [2025-10-20T19:19:45.851383] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-08. [2025-10-20T19:19:46.552677] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:19:46.638340] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T19:19:46.651330] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-09. [2025-10-20T19:19:47.433771] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:19:47.451339] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-09. [2025-10-20T19:19:48.251421] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-03. [2025-10-20T19:19:48.912963] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:19:49.051423] [INFO ] [REMOTE ] ICL: Fetching primary data for 2025-10. [2025-10-20T19:19:49.329484] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:19:49.851387] [INFO ] [REMOTE ] IDE: Fetching primary data for 2025-10. [2025-10-20T19:19:50.450888] [INFO ] [REMOTE ] ICL: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:19:50.456420] [INFO ] [REMOTE ] ICL: Full fetch complete (56131 records). [2025-10-20T19:19:50.456534] [DEBUG] [VALIDATOR ] ICL: Validating primary data. [2025-10-20T19:19:50.656540] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-08. [2025-10-20T19:19:50.837249] [INFO ] [REMOTE ] IDE: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T19:19:50.858427] [DEBUG] [VALIDATOR ] ICL: All expected trading days are present. [2025-10-20T19:19:50.858280] [INFO ] [REMOTE ] IDE: Full fetch complete (12783 records). [2025-10-20T19:19:50.858566] [DEBUG] [VALIDATOR ] ICL: Primary data validation passed. [2025-10-20T19:19:50.858705] [DEBUG] [VALIDATOR ] IDE: Validating primary data. [2025-10-20T19:19:50.859616] [INFO ] [MAIN ] ICL: Fetched primary data is valid. [2025-10-20T19:19:50.859739] [DEBUG] [LOCAL ] ICL: Saving primary data. [2025-10-20T19:19:50.963990] [DEBUG] [LOCAL ] ICL: Trimmed 56131 records to 38359 (keeping 200 days of data) [2025-10-20T19:19:51.431974] [DEBUG] [VALIDATOR ] IDE: All expected trading days are present. [2025-10-20T19:19:51.434306] [DEBUG] [VALIDATOR ] IDE: Primary data validation passed. [2025-10-20T19:19:51.435477] [INFO ] [MAIN ] IDE: Fetched primary data is valid. [2025-10-20T19:19:51.435843] [DEBUG] [LOCAL ] IDE: Saving primary data. [2025-10-20T19:19:51.441407] [DEBUG] [LOCAL ] IDE: Trimmed 12783 records to 9091 (keeping 200 days of data) [2025-10-20T19:19:51.471414] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-04. [2025-10-20T19:19:51.481057] [DEBUG] [LOCAL ] ICL: Primary data saved (38359 records). [2025-10-20T19:19:51.493461] [INFO ] [MAIN ] ICL: Full history saved. [2025-10-20T19:19:51.493943] [INFO ] [MAIN ] Update Progress: 1085/2367 symbols completed. ICL took 30.8s. Estimated remaining: 2h 45m 10s. Rate: 7.8 symbols/min [2025-10-20T19:19:51.621105] [DEBUG] [LOCAL ] IDE: Primary data saved (9091 records). [2025-10-20T19:19:51.632132] [INFO ] [MAIN ] IDE: Full history saved. [2025-10-20T19:19:51.642580] [INFO ] [MAIN ] Update Progress: 1086/2367 symbols completed. IDE took 22.3s. Estimated remaining: 2h 44m 53s. Rate: 7.8 symbols/min [2025-10-20T19:19:51.758965] [INFO ] [MAIN ] IEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:51.869453] [INFO ] [MAIN ] IFF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:51.891367] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:19:51.928528] [DEBUG] [LOCAL ] IEX: Loaded 38977 existing records from primary storage. [2025-10-20T19:19:51.928944] [INFO ] [MAIN ] IEX: Loaded 38977 existing records. [2025-10-20T19:19:52.078777] [DEBUG] [LOCAL ] IFF: Loaded 51919 existing records from primary storage. [2025-10-20T19:19:52.094697] [INFO ] [MAIN ] IFF: Loaded 51919 existing records. [2025-10-20T19:19:52.270299] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-09. [2025-10-20T19:19:52.270235] [WARN ] [MAIN ] IEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:52.271475] [INFO ] [MAIN ] IEX: Wiping existing primary data before full refresh. [2025-10-20T19:19:52.271539] [INFO ] [LOCAL ] IEX: Starting deletion of primary data. [2025-10-20T19:19:52.282122] [DEBUG] [LOCAL ] IEX: Deleted primary data file. [2025-10-20T19:19:52.320696] [INFO ] [MAIN ] IEX: Starting full primary refresh. [2025-10-20T19:19:52.362301] [WARN ] [MAIN ] IFF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:52.374275] [INFO ] [MAIN ] IFF: Wiping existing primary data before full refresh. [2025-10-20T19:19:52.399562] [INFO ] [LOCAL ] IFF: Starting deletion of primary data. [2025-10-20T19:19:52.430602] [DEBUG] [LOCAL ] IFF: Deleted primary data file. [2025-10-20T19:19:52.450105] [INFO ] [MAIN ] IFF: Starting full primary refresh. [2025-10-20T19:19:52.720727] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T19:19:53.091402] [INFO ] [REMOTE ] IEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:53.063752] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-05. [2025-10-20T19:19:53.128283] [INFO ] [REMOTE ] IFF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:53.532272] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:19:53.851427] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-01. [2025-10-20T19:19:54.609886] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:19:54.651397] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-01. [2025-10-20T19:19:55.451390] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-06. [2025-10-20T19:19:56.251354] [INFO ] [REMOTE ] IDA: Fetching primary data for 2025-10. [2025-10-20T19:19:56.571389] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T19:19:56.613734] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-01 (2.76 seconds). [2025-10-20T19:19:56.985551] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:19:57.051425] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-02. [2025-10-20T19:19:57.680456] [INFO ] [REMOTE ] IDA: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:19:57.685313] [INFO ] [REMOTE ] IDA: Full fetch complete (45461 records). [2025-10-20T19:19:57.685440] [DEBUG] [VALIDATOR ] IDA: Validating primary data. [2025-10-20T19:19:57.862241] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-07. [2025-10-20T19:19:58.065719] [DEBUG] [VALIDATOR ] IDA: All expected trading days are present. [2025-10-20T19:19:58.065804] [DEBUG] [VALIDATOR ] IDA: Primary data validation passed. [2025-10-20T19:19:58.066505] [INFO ] [MAIN ] IDA: Fetched primary data is valid. [2025-10-20T19:19:58.066545] [DEBUG] [LOCAL ] IDA: Saving primary data. [2025-10-20T19:19:58.067502] [DEBUG] [LOCAL ] IDA: Trimmed 45461 records to 32795 (keeping 200 days of data) [2025-10-20T19:19:58.270760] [DEBUG] [LOCAL ] IDA: Primary data saved (32795 records). [2025-10-20T19:19:58.270890] [INFO ] [MAIN ] IDA: Full history saved. [2025-10-20T19:19:58.270964] [INFO ] [MAIN ] Update Progress: 1087/2367 symbols completed. IDA took 33.8s. Estimated remaining: 2h 44m 44s. Rate: 7.8 symbols/min [2025-10-20T19:19:58.442853] [INFO ] [MAIN ] IFN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:19:58.462056] [DEBUG] [LOCAL ] IFN: Loaded 18730 existing records from primary storage. [2025-10-20T19:19:58.462291] [INFO ] [MAIN ] IFN: Loaded 18730 existing records. [2025-10-20T19:19:58.641459] [WARN ] [MAIN ] IFN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:19:58.642050] [INFO ] [MAIN ] IFN: Wiping existing primary data before full refresh. [2025-10-20T19:19:58.675424] [INFO ] [LOCAL ] IFN: Starting deletion of primary data. [2025-10-20T19:19:58.675262] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-02. [2025-10-20T19:19:58.689844] [DEBUG] [LOCAL ] IFN: Deleted primary data file. [2025-10-20T19:19:58.689939] [INFO ] [MAIN ] IFN: Starting full primary refresh. [2025-10-20T19:19:58.712777] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:19:59.038380] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T19:19:59.071677] [INFO ] [REMOTE ] IFN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:19:59.451359] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-01. [2025-10-20T19:20:00.251347] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-08. [2025-10-20T19:20:00.338454] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:20:00.641370] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:20:01.051433] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-03. [2025-10-20T19:20:01.534768] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:20:01.851500] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-02. [2025-10-20T19:20:02.651386] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-03. [2025-10-20T19:20:03.125193] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T19:20:03.149715] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:20:03.451477] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-03. [2025-10-20T19:20:04.237380] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:20:04.253730] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-04. [2025-10-20T19:20:04.257163] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T19:20:05.051347] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-09. [2025-10-20T19:20:05.851395] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-04. [2025-10-20T19:20:06.056551] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:20:06.123933] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:20:06.651396] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-04. [2025-10-20T19:20:07.456780] [INFO ] [REMOTE ] IDT: Fetching primary data for 2025-10. [2025-10-20T19:20:07.468485] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:20:07.689343] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:20:08.251371] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-05. [2025-10-20T19:20:08.710071] [INFO ] [REMOTE ] IDT: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T19:20:08.713857] [INFO ] [REMOTE ] IDT: Full fetch complete (23953 records). [2025-10-20T19:20:08.713933] [DEBUG] [VALIDATOR ] IDT: Validating primary data. [2025-10-20T19:20:09.055003] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-05. [2025-10-20T19:20:09.093172] [DEBUG] [VALIDATOR ] IDT: All expected trading days are present. [2025-10-20T19:20:09.093347] [DEBUG] [VALIDATOR ] IDT: Primary data validation passed. [2025-10-20T19:20:09.093967] [INFO ] [MAIN ] IDT: Fetched primary data is valid. [2025-10-20T19:20:09.094033] [DEBUG] [LOCAL ] IDT: Saving primary data. [2025-10-20T19:20:09.094878] [DEBUG] [LOCAL ] IDT: Trimmed 23953 records to 18522 (keeping 200 days of data) [2025-10-20T19:20:09.206954] [DEBUG] [LOCAL ] IDT: Primary data saved (18522 records). [2025-10-20T19:20:09.207077] [INFO ] [MAIN ] IDT: Full history saved. [2025-10-20T19:20:09.207153] [INFO ] [MAIN ] Update Progress: 1088/2367 symbols completed. IDT took 33.4s. Estimated remaining: 2h 44m 40s. Rate: 7.8 symbols/min [2025-10-20T19:20:09.340001] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T19:20:09.401057] [INFO ] [MAIN ] IFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:09.421839] [DEBUG] [LOCAL ] IFS: Loaded 22863 existing records from primary storage. [2025-10-20T19:20:09.422078] [INFO ] [MAIN ] IFS: Loaded 22863 existing records. [2025-10-20T19:20:09.604233] [WARN ] [MAIN ] IFS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:09.604453] [INFO ] [MAIN ] IFS: Wiping existing primary data before full refresh. [2025-10-20T19:20:09.604512] [INFO ] [LOCAL ] IFS: Starting deletion of primary data. [2025-10-20T19:20:09.604856] [DEBUG] [LOCAL ] IFS: Deleted primary data file. [2025-10-20T19:20:09.604915] [INFO ] [MAIN ] IFS: Starting full primary refresh. [2025-10-20T19:20:09.856818] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-05. [2025-10-20T19:20:09.963770] [INFO ] [REMOTE ] IFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:10.651406] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-06. [2025-10-20T19:20:10.961303] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:20:11.407359] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:20:11.451357] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-06. [2025-10-20T19:20:11.514075] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T19:20:12.251413] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-06. [2025-10-20T19:20:13.007627] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:20:13.051344] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-07. [2025-10-20T19:20:13.837008] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T19:20:13.851327] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-07. [2025-10-20T19:20:14.651371] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-07. [2025-10-20T19:20:15.006736] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:20:15.065376] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:20:15.451382] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-08. [2025-10-20T19:20:16.251391] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-01. [2025-10-20T19:20:17.010129] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T19:20:17.047204] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:20:17.051316] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-02. [2025-10-20T19:20:17.428155] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-10-20T19:20:17.851402] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-09. [2025-10-20T19:20:18.132887] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T19:20:18.651431] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-03. [2025-10-20T19:20:19.102633] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T19:20:19.451384] [INFO ] [REMOTE ] IEX: Fetching primary data for 2025-10. [2025-10-20T19:20:19.904809] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T19:20:20.251377] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-08. [2025-10-20T19:20:20.984703] [INFO ] [REMOTE ] IEX: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:20:20.989991] [INFO ] [REMOTE ] IEX: Full fetch complete (53824 records). [2025-10-20T19:20:20.990090] [DEBUG] [VALIDATOR ] IEX: Validating primary data. [2025-10-20T19:20:21.056704] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-04. [2025-10-20T19:20:21.367477] [DEBUG] [VALIDATOR ] IEX: All expected trading days are present. [2025-10-20T19:20:21.367564] [DEBUG] [VALIDATOR ] IEX: Primary data validation passed. [2025-10-20T19:20:21.368355] [INFO ] [MAIN ] IEX: Fetched primary data is valid. [2025-10-20T19:20:21.368418] [DEBUG] [LOCAL ] IEX: Saving primary data. [2025-10-20T19:20:21.369588] [DEBUG] [LOCAL ] IEX: Trimmed 53824 records to 38423 (keeping 200 days of data) [2025-10-20T19:20:21.606777] [DEBUG] [LOCAL ] IEX: Primary data saved (38423 records). [2025-10-20T19:20:21.606923] [INFO ] [MAIN ] IEX: Full history saved. [2025-10-20T19:20:21.607003] [INFO ] [MAIN ] Update Progress: 1089/2367 symbols completed. IEX took 30.1s. Estimated remaining: 2h 44m 38s. Rate: 7.8 symbols/min [2025-10-20T19:20:21.775436] [INFO ] [MAIN ] IGA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:21.786775] [DEBUG] [LOCAL ] IGA: Loaded 10395 existing records from primary storage. [2025-10-20T19:20:21.786997] [INFO ] [MAIN ] IGA: Loaded 10395 existing records. [2025-10-20T19:20:21.855865] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-08. [2025-10-20T19:20:21.970384] [WARN ] [MAIN ] IGA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:21.970741] [INFO ] [MAIN ] IGA: Wiping existing primary data before full refresh. [2025-10-20T19:20:21.970805] [INFO ] [LOCAL ] IGA: Starting deletion of primary data. [2025-10-20T19:20:21.971049] [DEBUG] [LOCAL ] IGA: Deleted primary data file. [2025-10-20T19:20:21.971152] [INFO ] [MAIN ] IGA: Starting full primary refresh. [2025-10-20T19:20:22.322875] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T19:20:22.373739] [INFO ] [REMOTE ] IGA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:22.389821] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T19:20:22.651436] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-09. [2025-10-20T19:20:22.952576] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:20:23.451450] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-05. [2025-10-20T19:20:24.251417] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-09. [2025-10-20T19:20:24.464993] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:20:24.564739] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:20:25.051444] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-01. [2025-10-20T19:20:25.219135] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:20:25.829953] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:20:25.851346] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-02. [2025-10-20T19:20:26.634357] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:20:26.651342] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-03. [2025-10-20T19:20:27.428771] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:20:27.451373] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-04. [2025-10-20T19:20:28.251352] [INFO ] [REMOTE ] IFN: Fetching primary data for 2025-10. [2025-10-20T19:20:28.410403] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T19:20:29.051415] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-05. [2025-10-20T19:20:29.259983] [INFO ] [REMOTE ] IFN: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:20:29.263594] [INFO ] [REMOTE ] IFN: Full fetch complete (28344 records). [2025-10-20T19:20:29.263770] [DEBUG] [VALIDATOR ] IFN: Validating primary data. [2025-10-20T19:20:29.656351] [DEBUG] [VALIDATOR ] IFN: All expected trading days are present. [2025-10-20T19:20:29.656538] [DEBUG] [VALIDATOR ] IFN: Primary data validation passed. [2025-10-20T19:20:29.657432] [INFO ] [MAIN ] IFN: Fetched primary data is valid. [2025-10-20T19:20:29.657497] [DEBUG] [LOCAL ] IFN: Saving primary data. [2025-10-20T19:20:29.658593] [DEBUG] [LOCAL ] IFN: Trimmed 28344 records to 18570 (keeping 200 days of data) [2025-10-20T19:20:29.777268] [DEBUG] [LOCAL ] IFN: Primary data saved (18570 records). [2025-10-20T19:20:29.783301] [INFO ] [MAIN ] IFN: Full history saved. [2025-10-20T19:20:29.784375] [INFO ] [MAIN ] Update Progress: 1090/2367 symbols completed. IFN took 31.5s. Estimated remaining: 2h 44m 31s. Rate: 7.8 symbols/min [2025-10-20T19:20:29.784200] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:20:29.856265] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-06. [2025-10-20T19:20:29.979019] [INFO ] [MAIN ] IGD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:30.000876] [DEBUG] [LOCAL ] IGD: Loaded 19676 existing records from primary storage. [2025-10-20T19:20:30.001220] [INFO ] [MAIN ] IGD: Loaded 19676 existing records. [2025-10-20T19:20:30.185517] [WARN ] [MAIN ] IGD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:30.185858] [INFO ] [MAIN ] IGD: Wiping existing primary data before full refresh. [2025-10-20T19:20:30.185914] [INFO ] [LOCAL ] IGD: Starting deletion of primary data. [2025-10-20T19:20:30.186227] [DEBUG] [LOCAL ] IGD: Deleted primary data file. [2025-10-20T19:20:30.186272] [INFO ] [MAIN ] IGD: Starting full primary refresh. [2025-10-20T19:20:30.555140] [INFO ] [REMOTE ] IGD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:30.651427] [INFO ] [REMOTE ] IFF: Fetching primary data for 2025-10. [2025-10-20T19:20:31.153616] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:20:31.451383] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-01. [2025-10-20T19:20:32.251351] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-07. [2025-10-20T19:20:32.582530] [INFO ] [REMOTE ] IFF: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:20:32.590042] [INFO ] [REMOTE ] IFF: Full fetch complete (73707 records). [2025-10-20T19:20:32.590165] [DEBUG] [VALIDATOR ] IFF: Validating primary data. [2025-10-20T19:20:32.980695] [DEBUG] [VALIDATOR ] IFF: All expected trading days are present. [2025-10-20T19:20:32.980821] [DEBUG] [VALIDATOR ] IFF: Primary data validation passed. [2025-10-20T19:20:32.982570] [INFO ] [MAIN ] IFF: Fetched primary data is valid. [2025-10-20T19:20:32.982666] [DEBUG] [LOCAL ] IFF: Saving primary data. [2025-10-20T19:20:33.000019] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:20:33.007271] [DEBUG] [LOCAL ] IFF: Trimmed 73707 records to 51225 (keeping 200 days of data) [2025-10-20T19:20:33.062485] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-02. [2025-10-20T19:20:33.327328] [DEBUG] [LOCAL ] IFF: Primary data saved (51225 records). [2025-10-20T19:20:33.327467] [INFO ] [MAIN ] IFF: Full history saved. [2025-10-20T19:20:33.327543] [INFO ] [MAIN ] Update Progress: 1091/2367 symbols completed. IFF took 41.7s. Estimated remaining: 2h 44m 18s. Rate: 7.8 symbols/min [2025-10-20T19:20:33.464087] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:20:33.519266] [INFO ] [MAIN ] IGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:33.525123] [DEBUG] [LOCAL ] IGI: Loaded 4092 existing records from primary storage. [2025-10-20T19:20:33.525300] [INFO ] [MAIN ] IGI: Loaded 4092 existing records. [2025-10-20T19:20:33.700746] [WARN ] [MAIN ] IGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:33.700943] [INFO ] [MAIN ] IGI: Wiping existing primary data before full refresh. [2025-10-20T19:20:33.700993] [INFO ] [LOCAL ] IGI: Starting deletion of primary data. [2025-10-20T19:20:33.701121] [DEBUG] [LOCAL ] IGI: Deleted primary data file. [2025-10-20T19:20:33.701158] [INFO ] [MAIN ] IGI: Starting full primary refresh. [2025-10-20T19:20:33.858346] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-06. [2025-10-20T19:20:34.054527] [INFO ] [REMOTE ] IGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:34.202256] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:20:34.651416] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-08. [2025-10-20T19:20:34.674472] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:20:35.451416] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-07. [2025-10-20T19:20:35.742001] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T19:20:36.251413] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-09. [2025-10-20T19:20:36.326778] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T19:20:37.051532] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-03. [2025-10-20T19:20:37.302724] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T19:20:37.828299] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:20:37.851340] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-04. [2025-10-20T19:20:38.651404] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-08. [2025-10-20T19:20:38.875142] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:20:39.386315] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:20:39.451392] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-05. [2025-10-20T19:20:40.251417] [INFO ] [REMOTE ] IFS: Fetching primary data for 2025-10. [2025-10-20T19:20:40.426813] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:20:41.051345] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-01. [2025-10-20T19:20:41.233488] [INFO ] [REMOTE ] IFS: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:20:41.236834] [INFO ] [REMOTE ] IFS: Full fetch complete (32212 records). [2025-10-20T19:20:41.236942] [DEBUG] [VALIDATOR ] IFS: Validating primary data. [2025-10-20T19:20:41.611372] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:20:41.644240] [DEBUG] [VALIDATOR ] IFS: All expected trading days are present. [2025-10-20T19:20:41.644330] [DEBUG] [VALIDATOR ] IFS: Primary data validation passed. [2025-10-20T19:20:41.645040] [INFO ] [MAIN ] IFS: Fetched primary data is valid. [2025-10-20T19:20:41.645108] [DEBUG] [LOCAL ] IFS: Saving primary data. [2025-10-20T19:20:41.645976] [DEBUG] [LOCAL ] IFS: Trimmed 32212 records to 22330 (keeping 200 days of data) [2025-10-20T19:20:41.777713] [DEBUG] [LOCAL ] IFS: Primary data saved (22330 records). [2025-10-20T19:20:41.777857] [INFO ] [MAIN ] IFS: Full history saved. [2025-10-20T19:20:41.777942] [INFO ] [MAIN ] Update Progress: 1092/2367 symbols completed. IFS took 32.6s. Estimated remaining: 2h 44m 11s. Rate: 7.8 symbols/min [2025-10-20T19:20:41.856715] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-02. [2025-10-20T19:20:41.955114] [INFO ] [MAIN ] IGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:41.983838] [DEBUG] [LOCAL ] IGR: Loaded 30813 existing records from primary storage. [2025-10-20T19:20:41.984115] [INFO ] [MAIN ] IGR: Loaded 30813 existing records. [2025-10-20T19:20:42.167441] [WARN ] [MAIN ] IGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:42.167707] [INFO ] [MAIN ] IGR: Wiping existing primary data before full refresh. [2025-10-20T19:20:42.167766] [INFO ] [LOCAL ] IGR: Starting deletion of primary data. [2025-10-20T19:20:42.167977] [DEBUG] [LOCAL ] IGR: Deleted primary data file. [2025-10-20T19:20:42.168017] [INFO ] [MAIN ] IGR: Starting full primary refresh. [2025-10-20T19:20:42.565566] [INFO ] [REMOTE ] IGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:42.651384] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-01. [2025-10-20T19:20:42.692970] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T19:20:43.451409] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-09. [2025-10-20T19:20:44.029864] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:20:44.044088] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:20:44.251385] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-02. [2025-10-20T19:20:45.051357] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-03. [2025-10-20T19:20:45.483045] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:20:45.616399] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:20:45.851372] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-03. [2025-10-20T19:20:46.651407] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-04. [2025-10-20T19:20:47.069484] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:20:47.122225] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T19:20:47.451371] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-06. [2025-10-20T19:20:48.251349] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-05. [2025-10-20T19:20:48.517092] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:20:48.683826] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T19:20:49.051431] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-06. [2025-10-20T19:20:49.601519] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T19:20:49.851392] [INFO ] [REMOTE ] IGA: Fetching primary data for 2025-10. [2025-10-20T19:20:50.651373] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-07. [2025-10-20T19:20:50.732475] [INFO ] [REMOTE ] IGA: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T19:20:50.735429] [INFO ] [REMOTE ] IGA: Full fetch complete (14366 records). [2025-10-20T19:20:50.735586] [DEBUG] [VALIDATOR ] IGA: Validating primary data. [2025-10-20T19:20:51.140424] [DEBUG] [VALIDATOR ] IGA: All expected trading days are present. [2025-10-20T19:20:51.140561] [DEBUG] [VALIDATOR ] IGA: Primary data validation passed. [2025-10-20T19:20:51.141151] [INFO ] [MAIN ] IGA: Fetched primary data is valid. [2025-10-20T19:20:51.141215] [DEBUG] [LOCAL ] IGA: Saving primary data. [2025-10-20T19:20:51.141811] [DEBUG] [LOCAL ] IGA: Trimmed 14366 records to 10268 (keeping 200 days of data) [2025-10-20T19:20:51.208149] [DEBUG] [LOCAL ] IGA: Primary data saved (10268 records). [2025-10-20T19:20:51.208345] [INFO ] [MAIN ] IGA: Full history saved. [2025-10-20T19:20:51.208446] [INFO ] [MAIN ] Update Progress: 1093/2367 symbols completed. IGA took 29.6s. Estimated remaining: 2h 44m 6s. Rate: 7.8 symbols/min [2025-10-20T19:20:51.325130] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T19:20:51.407016] [INFO ] [MAIN ] IH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:51.407261] [DEBUG] [LOCAL ] IH: No existing data file found in primary storage. [2025-10-20T19:20:51.407300] [WARN ] [MAIN ] IH: No existing data found, performing full refresh instead. [2025-10-20T19:20:51.407429] [INFO ] [MAIN ] IH: Wiping existing primary data before full refresh. [2025-10-20T19:20:51.407471] [INFO ] [LOCAL ] IH: Starting deletion of primary data. [2025-10-20T19:20:51.407513] [DEBUG] [LOCAL ] IH: No primary data file found to delete. [2025-10-20T19:20:51.407546] [INFO ] [MAIN ] IH: Starting full primary refresh. [2025-10-20T19:20:51.456574] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-08. [2025-10-20T19:20:51.782499] [INFO ] [REMOTE ] IH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:52.134883] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T19:20:52.251431] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-09. [2025-10-20T19:20:52.830020] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:20:53.051411] [INFO ] [REMOTE ] IGI: Fetching primary data for 2025-10. [2025-10-20T19:20:53.851390] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-01. [2025-10-20T19:20:53.864983] [INFO ] [REMOTE ] IGI: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:20:53.866788] [INFO ] [REMOTE ] IGI: Full fetch complete (5502 records). [2025-10-20T19:20:53.866877] [DEBUG] [VALIDATOR ] IGI: Validating primary data. [2025-10-20T19:20:54.242279] [DEBUG] [VALIDATOR ] IGI: All expected trading days are present. [2025-10-20T19:20:54.242382] [DEBUG] [VALIDATOR ] IGI: Primary data validation passed. [2025-10-20T19:20:54.242564] [INFO ] [MAIN ] IGI: Fetched primary data is valid. [2025-10-20T19:20:54.242604] [DEBUG] [LOCAL ] IGI: Saving primary data. [2025-10-20T19:20:54.242989] [DEBUG] [LOCAL ] IGI: Trimmed 5502 records to 4077 (keeping 200 days of data) [2025-10-20T19:20:54.267376] [DEBUG] [LOCAL ] IGI: Primary data saved (4077 records). [2025-10-20T19:20:54.267505] [INFO ] [MAIN ] IGI: Full history saved. [2025-10-20T19:20:54.267591] [INFO ] [MAIN ] Update Progress: 1094/2367 symbols completed. IGI took 20.9s. Estimated remaining: 2h 43m 52s. Rate: 7.8 symbols/min [2025-10-20T19:20:54.376563] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T19:20:54.448212] [INFO ] [MAIN ] IHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:20:54.458168] [DEBUG] [LOCAL ] IHD: Loaded 7396 existing records from primary storage. [2025-10-20T19:20:54.458398] [INFO ] [MAIN ] IHD: Loaded 7396 existing records. [2025-10-20T19:20:54.630020] [WARN ] [MAIN ] IHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:20:54.630254] [INFO ] [MAIN ] IHD: Wiping existing primary data before full refresh. [2025-10-20T19:20:54.630301] [INFO ] [LOCAL ] IHD: Starting deletion of primary data. [2025-10-20T19:20:54.630461] [DEBUG] [LOCAL ] IHD: Deleted primary data file. [2025-10-20T19:20:54.630526] [INFO ] [MAIN ] IHD: Starting full primary refresh. [2025-10-20T19:20:54.658957] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-02. [2025-10-20T19:20:55.011377] [INFO ] [REMOTE ] IHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:20:55.326176] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:20:55.451444] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-01. [2025-10-20T19:20:56.250214] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:20:56.252824] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-03. [2025-10-20T19:20:56.991943] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T19:20:57.051355] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-04. [2025-10-20T19:20:57.851384] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-04. [2025-10-20T19:20:58.352381] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:20:58.394835] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T19:20:58.651378] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-07. [2025-10-20T19:20:59.451358] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-02. [2025-10-20T19:20:59.748839] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:21:00.176633] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T19:21:00.251417] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-08. [2025-10-20T19:21:01.051371] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-05. [2025-10-20T19:21:01.514478] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T19:21:01.614971] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T19:21:01.851406] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-09. [2025-10-20T19:21:02.651370] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-03. [2025-10-20T19:21:02.870584] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:21:03.401867] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T19:21:03.451337] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-04. [2025-10-20T19:21:04.109938] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T19:21:04.251365] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-05. [2025-10-20T19:21:05.051429] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-05. [2025-10-20T19:21:05.574679] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:21:05.833378] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T19:21:05.851335] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-06. [2025-10-20T19:21:06.584253] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:21:06.651398] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-06. [2025-10-20T19:21:07.091795] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:21:07.451383] [INFO ] [REMOTE ] IGD: Fetching primary data for 2025-10. [2025-10-20T19:21:08.251384] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-06. [2025-10-20T19:21:08.579105] [INFO ] [REMOTE ] IGD: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:21:08.582989] [INFO ] [REMOTE ] IGD: Full fetch complete (30169 records). [2025-10-20T19:21:08.583132] [DEBUG] [VALIDATOR ] IGD: Validating primary data. [2025-10-20T19:21:08.978584] [DEBUG] [VALIDATOR ] IGD: All expected trading days are present. [2025-10-20T19:21:08.978685] [DEBUG] [VALIDATOR ] IGD: Primary data validation passed. [2025-10-20T19:21:08.979339] [INFO ] [MAIN ] IGD: Fetched primary data is valid. [2025-10-20T19:21:08.979400] [DEBUG] [LOCAL ] IGD: Saving primary data. [2025-10-20T19:21:08.980105] [DEBUG] [LOCAL ] IGD: Trimmed 30169 records to 19266 (keeping 200 days of data) [2025-10-20T19:21:09.057688] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-07. [2025-10-20T19:21:09.093160] [DEBUG] [LOCAL ] IGD: Primary data saved (19266 records). [2025-10-20T19:21:09.093307] [INFO ] [MAIN ] IGD: Full history saved. [2025-10-20T19:21:09.093389] [INFO ] [MAIN ] Update Progress: 1095/2367 symbols completed. IGD took 39.3s. Estimated remaining: 2h 43m 53s. Rate: 7.8 symbols/min [2025-10-20T19:21:09.271233] [INFO ] [MAIN ] IHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:09.297089] [DEBUG] [LOCAL ] IHG: Loaded 24671 existing records from primary storage. [2025-10-20T19:21:09.297351] [INFO ] [MAIN ] IHG: Loaded 24671 existing records. [2025-10-20T19:21:09.461197] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:21:09.510058] [WARN ] [MAIN ] IHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:09.510282] [INFO ] [MAIN ] IHG: Wiping existing primary data before full refresh. [2025-10-20T19:21:09.510331] [INFO ] [LOCAL ] IHG: Starting deletion of primary data. [2025-10-20T19:21:09.510631] [DEBUG] [LOCAL ] IHG: Deleted primary data file. [2025-10-20T19:21:09.510725] [INFO ] [MAIN ] IHG: Starting full primary refresh. [2025-10-20T19:21:09.615166] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T19:21:09.856596] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-08. [2025-10-20T19:21:09.909297] [INFO ] [REMOTE ] IHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:10.554054] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T19:21:10.651396] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-07. [2025-10-20T19:21:11.451365] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-01. [2025-10-20T19:21:12.203252] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:21:12.251351] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-08. [2025-10-20T19:21:12.421228] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T19:21:13.051383] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-07. [2025-10-20T19:21:13.358994] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:21:13.713335] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T19:21:13.851416] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-09. [2025-10-20T19:21:14.342730] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T19:21:14.651371] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-08. [2025-10-20T19:21:15.266107] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:21:15.451373] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-09. [2025-10-20T19:21:16.225501] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T19:21:16.251415] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-02. [2025-10-20T19:21:17.051431] [INFO ] [REMOTE ] IH: Fetching primary data for 2025-10. [2025-10-20T19:21:17.456715] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:21:17.688480] [INFO ] [REMOTE ] IH: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T19:21:17.689787] [INFO ] [REMOTE ] IH: Full fetch complete (5035 records). [2025-10-20T19:21:17.689861] [DEBUG] [VALIDATOR ] IH: Validating primary data. [2025-10-20T19:21:17.861449] [INFO ] [REMOTE ] IHD: Fetching primary data for 2025-10. [2025-10-20T19:21:18.102552] [DEBUG] [VALIDATOR ] IH: All expected trading days are present. [2025-10-20T19:21:18.102624] [DEBUG] [VALIDATOR ] IH: Primary data validation passed. [2025-10-20T19:21:18.102799] [INFO ] [MAIN ] IH: Fetched primary data is valid. [2025-10-20T19:21:18.102847] [DEBUG] [LOCAL ] IH: Saving primary data. [2025-10-20T19:21:18.103186] [DEBUG] [LOCAL ] IH: Trimmed 5035 records to 2768 (keeping 200 days of data) [2025-10-20T19:21:18.118253] [DEBUG] [LOCAL ] IH: Primary data saved (2768 records). [2025-10-20T19:21:18.118314] [INFO ] [MAIN ] IH: Full history saved. [2025-10-20T19:21:18.118371] [INFO ] [MAIN ] Update Progress: 1096/2367 symbols completed. IH took 26.9s. Estimated remaining: 2h 43m 47s. Rate: 7.8 symbols/min [2025-10-20T19:21:18.285488] [INFO ] [MAIN ] IHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:18.318935] [DEBUG] [LOCAL ] IHS: Loaded 38062 existing records from primary storage. [2025-10-20T19:21:18.319237] [INFO ] [MAIN ] IHS: Loaded 38062 existing records. [2025-10-20T19:21:18.491949] [WARN ] [MAIN ] IHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:18.492613] [INFO ] [MAIN ] IHS: Wiping existing primary data before full refresh. [2025-10-20T19:21:18.492714] [INFO ] [LOCAL ] IHS: Starting deletion of primary data. [2025-10-20T19:21:18.492936] [DEBUG] [LOCAL ] IHS: Deleted primary data file. [2025-10-20T19:21:18.492992] [INFO ] [MAIN ] IHS: Starting full primary refresh. [2025-10-20T19:21:18.654333] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-09. [2025-10-20T19:21:18.833801] [INFO ] [REMOTE ] IHD: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:21:18.862214] [INFO ] [REMOTE ] IHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:18.864463] [INFO ] [REMOTE ] IHD: Full fetch complete (11763 records). [2025-10-20T19:21:18.864539] [DEBUG] [VALIDATOR ] IHD: Validating primary data. [2025-10-20T19:21:19.211940] [DEBUG] [VALIDATOR ] IHD: All expected trading days are present. [2025-10-20T19:21:19.212011] [DEBUG] [VALIDATOR ] IHD: Primary data validation passed. [2025-10-20T19:21:19.212241] [INFO ] [MAIN ] IHD: Fetched primary data is valid. [2025-10-20T19:21:19.212279] [DEBUG] [LOCAL ] IHD: Saving primary data. [2025-10-20T19:21:19.212695] [DEBUG] [LOCAL ] IHD: Trimmed 11763 records to 7362 (keeping 200 days of data) [2025-10-20T19:21:19.253717] [DEBUG] [LOCAL ] IHD: Primary data saved (7362 records). [2025-10-20T19:21:19.253806] [INFO ] [MAIN ] IHD: Full history saved. [2025-10-20T19:21:19.253894] [INFO ] [MAIN ] Update Progress: 1097/2367 symbols completed. IHD took 25.0s. Estimated remaining: 2h 43m 31s. Rate: 7.8 symbols/min [2025-10-20T19:21:19.417035] [INFO ] [MAIN ] IIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:19.424336] [DEBUG] [LOCAL ] IIF: Loaded 5622 existing records from primary storage. [2025-10-20T19:21:19.424524] [INFO ] [MAIN ] IIF: Loaded 5622 existing records. [2025-10-20T19:21:19.456521] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-01. [2025-10-20T19:21:19.599816] [WARN ] [MAIN ] IIF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:19.600013] [INFO ] [MAIN ] IIF: Wiping existing primary data before full refresh. [2025-10-20T19:21:19.600059] [INFO ] [LOCAL ] IIF: Starting deletion of primary data. [2025-10-20T19:21:19.600255] [DEBUG] [LOCAL ] IIF: Deleted primary data file. [2025-10-20T19:21:19.600311] [INFO ] [MAIN ] IIF: Starting full primary refresh. [2025-10-20T19:21:19.962948] [INFO ] [REMOTE ] IIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:19.979952] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T19:21:20.251372] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-01. [2025-10-20T19:21:20.726342] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T19:21:21.038434] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:21:21.051359] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-02. [2025-10-20T19:21:21.760525] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:21:21.851412] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-03. [2025-10-20T19:21:22.447531] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T19:21:22.651407] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-02. [2025-10-20T19:21:23.451365] [INFO ] [REMOTE ] IGR: Fetching primary data for 2025-10. [2025-10-20T19:21:23.768105] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:21:24.251420] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-03. [2025-10-20T19:21:24.685365] [INFO ] [REMOTE ] IGR: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:21:24.689593] [INFO ] [REMOTE ] IGR: Full fetch complete (44722 records). [2025-10-20T19:21:24.689704] [DEBUG] [VALIDATOR ] IGR: Validating primary data. [2025-10-20T19:21:25.056752] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-04. [2025-10-20T19:21:25.074922] [DEBUG] [VALIDATOR ] IGR: All expected trading days are present. [2025-10-20T19:21:25.074998] [DEBUG] [VALIDATOR ] IGR: Primary data validation passed. [2025-10-20T19:21:25.075766] [INFO ] [MAIN ] IGR: Fetched primary data is valid. [2025-10-20T19:21:25.075912] [DEBUG] [LOCAL ] IGR: Saving primary data. [2025-10-20T19:21:25.076901] [DEBUG] [LOCAL ] IGR: Trimmed 44722 records to 30502 (keeping 200 days of data) [2025-10-20T19:21:25.247097] [DEBUG] [LOCAL ] IGR: Primary data saved (30502 records). [2025-10-20T19:21:25.247214] [INFO ] [MAIN ] IGR: Full history saved. [2025-10-20T19:21:25.247284] [INFO ] [MAIN ] Update Progress: 1098/2367 symbols completed. IGR took 43.5s. Estimated remaining: 2h 43m 22s. Rate: 7.8 symbols/min [2025-10-20T19:21:25.412177] [INFO ] [MAIN ] IIIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:25.428162] [DEBUG] [LOCAL ] IIIN: Loaded 16427 existing records from primary storage. [2025-10-20T19:21:25.428364] [INFO ] [MAIN ] IIIN: Loaded 16427 existing records. [2025-10-20T19:21:25.603042] [WARN ] [MAIN ] IIIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:25.603262] [INFO ] [MAIN ] IIIN: Wiping existing primary data before full refresh. [2025-10-20T19:21:25.603329] [INFO ] [LOCAL ] IIIN: Starting deletion of primary data. [2025-10-20T19:21:25.603481] [DEBUG] [LOCAL ] IIIN: Deleted primary data file. [2025-10-20T19:21:25.603518] [INFO ] [MAIN ] IIIN: Starting full primary refresh. [2025-10-20T19:21:25.870601] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-03. [2025-10-20T19:21:25.918991] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:21:25.999789] [INFO ] [REMOTE ] IIIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:26.006616] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T19:21:26.651409] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-05. [2025-10-20T19:21:27.179704] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T19:21:27.232433] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:21:27.451397] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-01. [2025-10-20T19:21:28.251403] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-06. [2025-10-20T19:21:28.562747] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T19:21:28.629166] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-06 (0.38 seconds). [2025-10-20T19:21:29.051461] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-04. [2025-10-20T19:21:29.851382] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-04. [2025-10-20T19:21:30.437107] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T19:21:30.651423] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-02. [2025-10-20T19:21:30.919514] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:21:31.451416] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-07. [2025-10-20T19:21:31.627844] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:21:32.215709] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T19:21:32.251367] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-08. [2025-10-20T19:21:32.860687] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:21:33.051395] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-05. [2025-10-20T19:21:33.851422] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-09. [2025-10-20T19:21:34.362243] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:21:34.427598] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:21:34.651380] [INFO ] [REMOTE ] IIF: Fetching primary data for 2025-10. [2025-10-20T19:21:35.285531] [INFO ] [REMOTE ] IIF: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T19:21:35.286508] [INFO ] [REMOTE ] IIF: Full fetch complete (8919 records). [2025-10-20T19:21:35.286594] [DEBUG] [VALIDATOR ] IIF: Validating primary data. [2025-10-20T19:21:35.451400] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-03. [2025-10-20T19:21:35.666343] [DEBUG] [VALIDATOR ] IIF: All expected trading days are present. [2025-10-20T19:21:35.666410] [DEBUG] [VALIDATOR ] IIF: Primary data validation passed. [2025-10-20T19:21:35.666677] [INFO ] [MAIN ] IIF: Fetched primary data is valid. [2025-10-20T19:21:35.666742] [DEBUG] [LOCAL ] IIF: Saving primary data. [2025-10-20T19:21:35.667281] [DEBUG] [LOCAL ] IIF: Trimmed 8919 records to 5448 (keeping 200 days of data) [2025-10-20T19:21:35.698105] [DEBUG] [LOCAL ] IIF: Primary data saved (5448 records). [2025-10-20T19:21:35.698210] [INFO ] [MAIN ] IIF: Full history saved. [2025-10-20T19:21:35.698283] [INFO ] [MAIN ] Update Progress: 1099/2367 symbols completed. IIF took 16.4s. Estimated remaining: 2h 43m 17s. Rate: 7.8 symbols/min [2025-10-20T19:21:35.866621] [INFO ] [MAIN ] IIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:35.883362] [DEBUG] [LOCAL ] IIM: Loaded 15903 existing records from primary storage. [2025-10-20T19:21:35.883605] [INFO ] [MAIN ] IIM: Loaded 15903 existing records. [2025-10-20T19:21:36.058516] [WARN ] [MAIN ] IIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:36.058757] [INFO ] [MAIN ] IIM: Wiping existing primary data before full refresh. [2025-10-20T19:21:36.058824] [INFO ] [LOCAL ] IIM: Starting deletion of primary data. [2025-10-20T19:21:36.059055] [DEBUG] [LOCAL ] IIM: Deleted primary data file. [2025-10-20T19:21:36.059095] [INFO ] [MAIN ] IIM: Starting full primary refresh. [2025-10-20T19:21:36.261706] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-06. [2025-10-20T19:21:36.424112] [INFO ] [REMOTE ] IIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:36.561145] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T19:21:37.051384] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-05. [2025-10-20T19:21:37.428506] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:21:37.851366] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-01. [2025-10-20T19:21:38.444690] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:21:38.610150] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:21:38.651333] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-02. [2025-10-20T19:21:39.451372] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-07. [2025-10-20T19:21:39.792890] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:21:40.251340] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-04. [2025-10-20T19:21:40.654521] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:21:41.051373] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-03. [2025-10-20T19:21:41.401327] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:21:41.851365] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-05. [2025-10-20T19:21:42.034691] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T19:21:42.651349] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-06. [2025-10-20T19:21:42.883849] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:21:43.451426] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-04. [2025-10-20T19:21:44.084520] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T19:21:44.138799] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T19:21:44.251380] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-05. [2025-10-20T19:21:44.957951] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:21:45.051406] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-06. [2025-10-20T19:21:45.851351] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-06. [2025-10-20T19:21:46.005072] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:21:46.651340] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-07. [2025-10-20T19:21:46.888570] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T19:21:47.451346] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-08. [2025-10-20T19:21:47.637288] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:21:48.251336] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-07. [2025-10-20T19:21:48.353949] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:21:49.051369] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-09. [2025-10-20T19:21:49.750440] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:21:49.851333] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-08. [2025-10-20T19:21:50.112003] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:21:50.651372] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-08. [2025-10-20T19:21:51.157196] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:21:51.451356] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-07. [2025-10-20T19:21:51.753464] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:21:52.251365] [INFO ] [REMOTE ] IHG: Fetching primary data for 2025-10. [2025-10-20T19:21:52.481535] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T19:21:53.051459] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-09. [2025-10-20T19:21:53.453753] [INFO ] [REMOTE ] IHG: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T19:21:53.458756] [INFO ] [REMOTE ] IHG: Full fetch complete (34224 records). [2025-10-20T19:21:53.458891] [DEBUG] [VALIDATOR ] IHG: Validating primary data. [2025-10-20T19:21:53.833081] [DEBUG] [VALIDATOR ] IHG: All expected trading days are present. [2025-10-20T19:21:53.833172] [DEBUG] [VALIDATOR ] IHG: Primary data validation passed. [2025-10-20T19:21:53.834091] [INFO ] [MAIN ] IHG: Fetched primary data is valid. [2025-10-20T19:21:53.834153] [DEBUG] [LOCAL ] IHG: Saving primary data. [2025-10-20T19:21:53.835271] [DEBUG] [LOCAL ] IHG: Trimmed 34224 records to 24215 (keeping 200 days of data) [2025-10-20T19:21:53.892348] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-08. [2025-10-20T19:21:53.992041] [DEBUG] [LOCAL ] IHG: Primary data saved (24215 records). [2025-10-20T19:21:53.992172] [INFO ] [MAIN ] IHG: Full history saved. [2025-10-20T19:21:53.992252] [INFO ] [MAIN ] Update Progress: 1100/2367 symbols completed. IHG took 44.9s. Estimated remaining: 2h 43m 21s. Rate: 7.8 symbols/min [2025-10-20T19:21:54.162406] [INFO ] [MAIN ] IIPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:54.174576] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T19:21:54.206690] [DEBUG] [LOCAL ] IIPR: Loaded 29862 existing records from primary storage. [2025-10-20T19:21:54.206964] [INFO ] [MAIN ] IIPR: Loaded 29862 existing records. [2025-10-20T19:21:54.386956] [WARN ] [MAIN ] IIPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:54.387161] [INFO ] [MAIN ] IIPR: Wiping existing primary data before full refresh. [2025-10-20T19:21:54.387209] [INFO ] [LOCAL ] IIPR: Starting deletion of primary data. [2025-10-20T19:21:54.387568] [DEBUG] [LOCAL ] IIPR: Deleted primary data file. [2025-10-20T19:21:54.387609] [INFO ] [MAIN ] IIPR: Starting full primary refresh. [2025-10-20T19:21:54.656522] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-09. [2025-10-20T19:21:54.761324] [INFO ] [REMOTE ] IIPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:54.762049] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T19:21:55.451370] [INFO ] [REMOTE ] IIM: Fetching primary data for 2025-10. [2025-10-20T19:21:55.899207] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T19:21:56.127433] [INFO ] [REMOTE ] IIM: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T19:21:56.130300] [INFO ] [REMOTE ] IIM: Full fetch complete (24908 records). [2025-10-20T19:21:56.130382] [DEBUG] [VALIDATOR ] IIM: Validating primary data. [2025-10-20T19:21:56.256537] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-01. [2025-10-20T19:21:56.513274] [DEBUG] [VALIDATOR ] IIM: All expected trading days are present. [2025-10-20T19:21:56.513348] [DEBUG] [VALIDATOR ] IIM: Primary data validation passed. [2025-10-20T19:21:56.513922] [INFO ] [MAIN ] IIM: Fetched primary data is valid. [2025-10-20T19:21:56.513985] [DEBUG] [LOCAL ] IIM: Saving primary data. [2025-10-20T19:21:56.514823] [DEBUG] [LOCAL ] IIM: Trimmed 24908 records to 15541 (keeping 200 days of data) [2025-10-20T19:21:56.609057] [DEBUG] [LOCAL ] IIM: Primary data saved (15541 records). [2025-10-20T19:21:56.609188] [INFO ] [MAIN ] IIM: Full history saved. [2025-10-20T19:21:56.609261] [INFO ] [MAIN ] Update Progress: 1101/2367 symbols completed. IIM took 20.9s. Estimated remaining: 2h 43m 8s. Rate: 7.8 symbols/min [2025-10-20T19:21:56.783863] [INFO ] [MAIN ] IMAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:21:56.824800] [DEBUG] [LOCAL ] IMAX: Loaded 43881 existing records from primary storage. [2025-10-20T19:21:56.825095] [INFO ] [MAIN ] IMAX: Loaded 43881 existing records. [2025-10-20T19:21:56.999053] [WARN ] [MAIN ] IMAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:21:56.999278] [INFO ] [MAIN ] IMAX: Wiping existing primary data before full refresh. [2025-10-20T19:21:56.999324] [INFO ] [LOCAL ] IMAX: Starting deletion of primary data. [2025-10-20T19:21:56.999570] [DEBUG] [LOCAL ] IMAX: Deleted primary data file. [2025-10-20T19:21:56.999607] [INFO ] [MAIN ] IMAX: Starting full primary refresh. [2025-10-20T19:21:57.061738] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-09. [2025-10-20T19:21:57.359459] [INFO ] [REMOTE ] IMAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:21:57.813837] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:21:57.851335] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-01. [2025-10-20T19:21:57.909449] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:21:58.651418] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-02. [2025-10-20T19:21:59.308871] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T19:21:59.451411] [INFO ] [REMOTE ] IIIN: Fetching primary data for 2025-10. [2025-10-20T19:21:59.843267] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T19:22:00.251358] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-03. [2025-10-20T19:22:00.450997] [INFO ] [REMOTE ] IIIN: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:22:00.453848] [INFO ] [REMOTE ] IIIN: Full fetch complete (24865 records). [2025-10-20T19:22:00.453917] [DEBUG] [VALIDATOR ] IIIN: Validating primary data. [2025-10-20T19:22:00.841332] [DEBUG] [VALIDATOR ] IIIN: All expected trading days are present. [2025-10-20T19:22:00.841459] [DEBUG] [VALIDATOR ] IIIN: Primary data validation passed. [2025-10-20T19:22:00.841996] [INFO ] [MAIN ] IIIN: Fetched primary data is valid. [2025-10-20T19:22:00.842056] [DEBUG] [LOCAL ] IIIN: Saving primary data. [2025-10-20T19:22:00.842673] [DEBUG] [LOCAL ] IIIN: Trimmed 24865 records to 16310 (keeping 200 days of data) [2025-10-20T19:22:00.939827] [DEBUG] [LOCAL ] IIIN: Primary data saved (16310 records). [2025-10-20T19:22:00.939952] [INFO ] [MAIN ] IIIN: Full history saved. [2025-10-20T19:22:00.940029] [INFO ] [MAIN ] Update Progress: 1102/2367 symbols completed. IIIN took 35.7s. Estimated remaining: 2h 42m 56s. Rate: 7.8 symbols/min [2025-10-20T19:22:01.056596] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-02. [2025-10-20T19:22:01.112739] [INFO ] [MAIN ] INFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:01.156945] [DEBUG] [LOCAL ] INFA: Loaded 49646 existing records from primary storage. [2025-10-20T19:22:01.157327] [INFO ] [MAIN ] INFA: Loaded 49646 existing records. [2025-10-20T19:22:01.334435] [WARN ] [MAIN ] INFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:01.334879] [INFO ] [MAIN ] INFA: Wiping existing primary data before full refresh. [2025-10-20T19:22:01.334936] [INFO ] [LOCAL ] INFA: Starting deletion of primary data. [2025-10-20T19:22:01.335411] [DEBUG] [LOCAL ] INFA: Deleted primary data file. [2025-10-20T19:22:01.335468] [INFO ] [MAIN ] INFA: Starting full primary refresh. [2025-10-20T19:22:01.709114] [INFO ] [REMOTE ] INFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:01.851736] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-01. [2025-10-20T19:22:01.885404] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:22:02.622914] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:22:02.651363] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-03. [2025-10-20T19:22:03.157569] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T19:22:03.451381] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-02. [2025-10-20T19:22:04.079997] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T19:22:04.251391] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-04. [2025-10-20T19:22:05.051358] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-04. [2025-10-20T19:22:05.134531] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:22:05.840461] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:22:05.851334] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-05. [2025-10-20T19:22:06.658846] [INFO ] [REMOTE ] IHS: Fetching primary data for 2025-10. [2025-10-20T19:22:06.664159] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:22:07.153474] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T19:22:07.451364] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-03. [2025-10-20T19:22:07.959598] [INFO ] [REMOTE ] IHS: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T19:22:07.964134] [INFO ] [REMOTE ] IHS: Full fetch complete (51728 records). [2025-10-20T19:22:07.964225] [DEBUG] [VALIDATOR ] IHS: Validating primary data. [2025-10-20T19:22:08.257099] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-05. [2025-10-20T19:22:08.351059] [DEBUG] [VALIDATOR ] IHS: All expected trading days are present. [2025-10-20T19:22:08.351130] [DEBUG] [VALIDATOR ] IHS: Primary data validation passed. [2025-10-20T19:22:08.351934] [INFO ] [MAIN ] IHS: Fetched primary data is valid. [2025-10-20T19:22:08.351995] [DEBUG] [LOCAL ] IHS: Saving primary data. [2025-10-20T19:22:08.353014] [DEBUG] [LOCAL ] IHS: Trimmed 51728 records to 37267 (keeping 200 days of data) [2025-10-20T19:22:08.556250] [DEBUG] [LOCAL ] IHS: Primary data saved (37267 records). [2025-10-20T19:22:08.556395] [INFO ] [MAIN ] IHS: Full history saved. [2025-10-20T19:22:08.556475] [INFO ] [MAIN ] Update Progress: 1103/2367 symbols completed. IHS took 50.4s. Estimated remaining: 2h 42m 48s. Rate: 7.8 symbols/min [2025-10-20T19:22:08.741178] [INFO ] [MAIN ] INFY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:08.792496] [DEBUG] [LOCAL ] INFY: Loaded 54376 existing records from primary storage. [2025-10-20T19:22:08.792877] [INFO ] [MAIN ] INFY: Loaded 54376 existing records. [2025-10-20T19:22:08.985399] [WARN ] [MAIN ] INFY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:08.985613] [INFO ] [MAIN ] INFY: Wiping existing primary data before full refresh. [2025-10-20T19:22:08.985690] [INFO ] [LOCAL ] INFY: Starting deletion of primary data. [2025-10-20T19:22:08.985910] [DEBUG] [LOCAL ] INFY: Deleted primary data file. [2025-10-20T19:22:08.985964] [INFO ] [MAIN ] INFY: Starting full primary refresh. [2025-10-20T19:22:09.070516] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-06. [2025-10-20T19:22:09.353673] [INFO ] [REMOTE ] INFY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:09.382073] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T19:22:09.723321] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T19:22:09.851349] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-06. [2025-10-20T19:22:10.285545] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T19:22:10.651377] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-04. [2025-10-20T19:22:11.291149] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T19:22:11.451369] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-07. [2025-10-20T19:22:12.070922] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T19:22:12.251405] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-05. [2025-10-20T19:22:12.593586] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:22:13.051433] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-08. [2025-10-20T19:22:13.663760] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T19:22:13.851490] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-01. [2025-10-20T19:22:14.545439] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:22:14.651353] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-09. [2025-10-20T19:22:15.451411] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-06. [2025-10-20T19:22:15.644737] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:22:16.162600] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T19:22:16.251364] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-02. [2025-10-20T19:22:16.897840] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:22:17.051368] [INFO ] [REMOTE ] IMAX: Fetching primary data for 2025-10. [2025-10-20T19:22:17.863808] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-07. [2025-10-20T19:22:17.889368] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:22:18.643957] [INFO ] [REMOTE ] IMAX: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:22:18.650633] [INFO ] [REMOTE ] IMAX: Full fetch complete (60919 records). [2025-10-20T19:22:18.650775] [DEBUG] [VALIDATOR ] IMAX: Validating primary data. [2025-10-20T19:22:18.651360] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-03. [2025-10-20T19:22:19.046889] [DEBUG] [VALIDATOR ] IMAX: All expected trading days are present. [2025-10-20T19:22:19.047092] [DEBUG] [VALIDATOR ] IMAX: Primary data validation passed. [2025-10-20T19:22:19.048059] [INFO ] [MAIN ] IMAX: Fetched primary data is valid. [2025-10-20T19:22:19.048125] [DEBUG] [LOCAL ] IMAX: Saving primary data. [2025-10-20T19:22:19.049473] [DEBUG] [LOCAL ] IMAX: Trimmed 60919 records to 43484 (keeping 200 days of data) [2025-10-20T19:22:19.333931] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:22:19.335074] [DEBUG] [LOCAL ] IMAX: Primary data saved (43484 records). [2025-10-20T19:22:19.341385] [INFO ] [MAIN ] IMAX: Full history saved. [2025-10-20T19:22:19.341726] [INFO ] [MAIN ] Update Progress: 1104/2367 symbols completed. IMAX took 22.7s. Estimated remaining: 2h 42m 44s. Rate: 7.8 symbols/min [2025-10-20T19:22:19.456619] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-07. [2025-10-20T19:22:19.518072] [INFO ] [MAIN ] ING: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:19.567355] [DEBUG] [LOCAL ] ING: Loaded 51494 existing records from primary storage. [2025-10-20T19:22:19.567743] [INFO ] [MAIN ] ING: Loaded 51494 existing records. [2025-10-20T19:22:19.754785] [WARN ] [MAIN ] ING: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:19.755052] [INFO ] [MAIN ] ING: Wiping existing primary data before full refresh. [2025-10-20T19:22:19.755106] [INFO ] [LOCAL ] ING: Starting deletion of primary data. [2025-10-20T19:22:19.755373] [DEBUG] [LOCAL ] ING: Deleted primary data file. [2025-10-20T19:22:19.755429] [INFO ] [MAIN ] ING: Starting full primary refresh. [2025-10-20T19:22:20.135851] [INFO ] [REMOTE ] ING: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:20.251447] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-01. [2025-10-20T19:22:20.643828] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T19:22:21.051375] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-08. [2025-10-20T19:22:21.185977] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T19:22:21.851400] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-04. [2025-10-20T19:22:22.039184] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:22:22.476082] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:22:22.651387] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-09. [2025-10-20T19:22:23.451409] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-02. [2025-10-20T19:22:23.708718] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:22:23.995878] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T19:22:24.251384] [INFO ] [REMOTE ] IIPR: Fetching primary data for 2025-10. [2025-10-20T19:22:24.748080] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:22:25.051377] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-08. [2025-10-20T19:22:25.511374] [INFO ] [REMOTE ] IIPR: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:22:25.517691] [INFO ] [REMOTE ] IIPR: Full fetch complete (43873 records). [2025-10-20T19:22:25.517845] [DEBUG] [VALIDATOR ] IIPR: Validating primary data. [2025-10-20T19:22:25.860514] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-03. [2025-10-20T19:22:25.914699] [DEBUG] [VALIDATOR ] IIPR: All expected trading days are present. [2025-10-20T19:22:25.914785] [DEBUG] [VALIDATOR ] IIPR: Primary data validation passed. [2025-10-20T19:22:25.915469] [INFO ] [MAIN ] IIPR: Fetched primary data is valid. [2025-10-20T19:22:25.915541] [DEBUG] [LOCAL ] IIPR: Saving primary data. [2025-10-20T19:22:25.916696] [DEBUG] [LOCAL ] IIPR: Trimmed 43873 records to 29071 (keeping 200 days of data) [2025-10-20T19:22:26.096455] [DEBUG] [LOCAL ] IIPR: Primary data saved (29071 records). [2025-10-20T19:22:26.096591] [INFO ] [MAIN ] IIPR: Full history saved. [2025-10-20T19:22:26.096689] [INFO ] [MAIN ] Update Progress: 1105/2367 symbols completed. IIPR took 32.1s. Estimated remaining: 2h 42m 35s. Rate: 7.8 symbols/min [2025-10-20T19:22:26.267695] [INFO ] [MAIN ] INGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:26.295685] [DEBUG] [LOCAL ] INGM: Loaded 28706 existing records from primary storage. [2025-10-20T19:22:26.295950] [INFO ] [MAIN ] INGM: Loaded 28706 existing records. [2025-10-20T19:22:26.476566] [WARN ] [MAIN ] INGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:26.476931] [INFO ] [MAIN ] INGM: Wiping existing primary data before full refresh. [2025-10-20T19:22:26.477022] [INFO ] [LOCAL ] INGM: Starting deletion of primary data. [2025-10-20T19:22:26.477257] [DEBUG] [LOCAL ] INGM: Deleted primary data file. [2025-10-20T19:22:26.477308] [INFO ] [MAIN ] INGM: Starting full primary refresh. [2025-10-20T19:22:26.651682] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-05. [2025-10-20T19:22:26.924471] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T19:22:26.994561] [INFO ] [REMOTE ] INGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:27.451406] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-09. [2025-10-20T19:22:27.749232] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:22:28.251383] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-04. [2025-10-20T19:22:28.730012] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T19:22:28.985378] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T19:22:29.051551] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-01. [2025-10-20T19:22:29.715174] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:22:29.825941] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:22:29.851377] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-02. [2025-10-20T19:22:30.651394] [INFO ] [REMOTE ] INFA: Fetching primary data for 2025-10. [2025-10-20T19:22:30.959809] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:22:31.451382] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-06. [2025-10-20T19:22:32.251383] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-03. [2025-10-20T19:22:32.326841] [INFO ] [REMOTE ] INFA: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:22:32.335473] [INFO ] [REMOTE ] INFA: Full fetch complete (70880 records). [2025-10-20T19:22:32.335703] [DEBUG] [VALIDATOR ] INFA: Validating primary data. [2025-10-20T19:22:32.740159] [DEBUG] [VALIDATOR ] INFA: All expected trading days are present. [2025-10-20T19:22:32.740259] [DEBUG] [VALIDATOR ] INFA: Primary data validation passed. [2025-10-20T19:22:32.741428] [INFO ] [MAIN ] INFA: Fetched primary data is valid. [2025-10-20T19:22:32.741576] [DEBUG] [LOCAL ] INFA: Saving primary data. [2025-10-20T19:22:32.743032] [DEBUG] [LOCAL ] INFA: Trimmed 70880 records to 48813 (keeping 200 days of data) [2025-10-20T19:22:32.982827] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:22:33.061260] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-07. [2025-10-20T19:22:33.078324] [DEBUG] [LOCAL ] INFA: Primary data saved (48813 records). [2025-10-20T19:22:33.078464] [INFO ] [MAIN ] INFA: Full history saved. [2025-10-20T19:22:33.078543] [INFO ] [MAIN ] Update Progress: 1106/2367 symbols completed. INFA took 32.1s. Estimated remaining: 2h 42m 27s. Rate: 7.8 symbols/min [2025-10-20T19:22:33.264533] [INFO ] [MAIN ] INGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:33.299518] [DEBUG] [LOCAL ] INGR: Loaded 34266 existing records from primary storage. [2025-10-20T19:22:33.299870] [INFO ] [MAIN ] INGR: Loaded 34266 existing records. [2025-10-20T19:22:33.486353] [WARN ] [MAIN ] INGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:33.486792] [INFO ] [MAIN ] INGR: Wiping existing primary data before full refresh. [2025-10-20T19:22:33.486854] [INFO ] [LOCAL ] INGR: Starting deletion of primary data. [2025-10-20T19:22:33.487208] [DEBUG] [LOCAL ] INGR: Deleted primary data file. [2025-10-20T19:22:33.487269] [INFO ] [MAIN ] INGR: Starting full primary refresh. [2025-10-20T19:22:33.845416] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:22:33.852721] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-05. [2025-10-20T19:22:33.934714] [INFO ] [REMOTE ] INGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:34.651360] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-01. [2025-10-20T19:22:35.023810] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T19:22:35.451367] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-08. [2025-10-20T19:22:35.574297] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:22:35.810190] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:22:36.251419] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-06. [2025-10-20T19:22:37.051402] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-02. [2025-10-20T19:22:37.308667] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T19:22:37.851429] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-04. [2025-10-20T19:22:37.909877] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:22:38.081398] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:22:38.651378] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-07. [2025-10-20T19:22:39.255222] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:22:39.451384] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-05. [2025-10-20T19:22:40.251340] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-09. [2025-10-20T19:22:40.436451] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:22:40.724402] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T19:22:41.051414] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-03. [2025-10-20T19:22:41.851377] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-06. [2025-10-20T19:22:42.036887] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T19:22:42.531827] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:22:42.651408] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-08. [2025-10-20T19:22:42.965702] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T19:22:43.451421] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-07. [2025-10-20T19:22:44.251388] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-04. [2025-10-20T19:22:44.408333] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T19:22:44.701323] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T19:22:45.051370] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-09. [2025-10-20T19:22:45.651019] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:22:45.851349] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-05. [2025-10-20T19:22:46.529291] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:22:46.651411] [INFO ] [REMOTE ] INFY: Fetching primary data for 2025-10. [2025-10-20T19:22:47.205328] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:22:47.451436] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-06. [2025-10-20T19:22:48.251351] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-08. [2025-10-20T19:22:48.477242] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T19:22:48.706326] [INFO ] [REMOTE ] INFY: Incremental fetch complete for 2025-10 (2.05 seconds). [2025-10-20T19:22:48.714591] [INFO ] [REMOTE ] INFY: Full fetch complete (77747 records). [2025-10-20T19:22:48.714768] [DEBUG] [VALIDATOR ] INFY: Validating primary data. [2025-10-20T19:22:49.058694] [INFO ] [REMOTE ] ING: Fetching primary data for 2025-10. [2025-10-20T19:22:49.113383] [DEBUG] [VALIDATOR ] INFY: All expected trading days are present. [2025-10-20T19:22:49.113541] [DEBUG] [VALIDATOR ] INFY: Primary data validation passed. [2025-10-20T19:22:49.114700] [INFO ] [MAIN ] INFY: Fetched primary data is valid. [2025-10-20T19:22:49.114784] [DEBUG] [LOCAL ] INFY: Saving primary data. [2025-10-20T19:22:49.116451] [DEBUG] [LOCAL ] INFY: Trimmed 77747 records to 53597 (keeping 200 days of data) [2025-10-20T19:22:49.237789] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:22:49.463385] [DEBUG] [LOCAL ] INFY: Primary data saved (53597 records). [2025-10-20T19:22:49.463528] [INFO ] [MAIN ] INFY: Full history saved. [2025-10-20T19:22:49.463605] [INFO ] [MAIN ] Update Progress: 1107/2367 symbols completed. INFY took 40.9s. Estimated remaining: 2h 42m 29s. Rate: 7.8 symbols/min [2025-10-20T19:22:49.635293] [INFO ] [MAIN ] INN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:49.673252] [DEBUG] [LOCAL ] INN: Loaded 42375 existing records from primary storage. [2025-10-20T19:22:49.673559] [INFO ] [MAIN ] INN: Loaded 42375 existing records. [2025-10-20T19:22:49.848536] [WARN ] [MAIN ] INN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:22:49.848831] [INFO ] [MAIN ] INN: Wiping existing primary data before full refresh. [2025-10-20T19:22:49.848883] [INFO ] [LOCAL ] INN: Starting deletion of primary data. [2025-10-20T19:22:49.849095] [DEBUG] [LOCAL ] INN: Deleted primary data file. [2025-10-20T19:22:49.849151] [INFO ] [MAIN ] INN: Starting full primary refresh. [2025-10-20T19:22:49.860221] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-07. [2025-10-20T19:22:50.197866] [INFO ] [REMOTE ] INN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:50.651394] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-09. [2025-10-20T19:22:51.003610] [INFO ] [REMOTE ] ING: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:22:51.011159] [INFO ] [REMOTE ] ING: Full fetch complete (73819 records). [2025-10-20T19:22:51.011276] [DEBUG] [VALIDATOR ] ING: Validating primary data. [2025-10-20T19:22:51.406682] [DEBUG] [VALIDATOR ] ING: All expected trading days are present. [2025-10-20T19:22:51.421636] [DEBUG] [VALIDATOR ] ING: Primary data validation passed. [2025-10-20T19:22:51.434130] [INFO ] [MAIN ] ING: Fetched primary data is valid. [2025-10-20T19:22:51.434218] [DEBUG] [LOCAL ] ING: Saving primary data. [2025-10-20T19:22:51.436582] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:22:51.445044] [DEBUG] [LOCAL ] ING: Trimmed 73819 records to 50721 (keeping 200 days of data) [2025-10-20T19:22:51.497295] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-01. [2025-10-20T19:22:51.751527] [DEBUG] [LOCAL ] ING: Primary data saved (50721 records). [2025-10-20T19:22:51.751703] [INFO ] [MAIN ] ING: Full history saved. [2025-10-20T19:22:51.751816] [INFO ] [MAIN ] Update Progress: 1108/2367 symbols completed. ING took 32.4s. Estimated remaining: 2h 42m 15s. Rate: 7.8 symbols/min [2025-10-20T19:22:51.925256] [INFO ] [MAIN ] INR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:22:51.925409] [DEBUG] [LOCAL ] INR: No existing data file found in primary storage. [2025-10-20T19:22:51.925445] [WARN ] [MAIN ] INR: No existing data found, performing full refresh instead. [2025-10-20T19:22:51.925549] [INFO ] [MAIN ] INR: Wiping existing primary data before full refresh. [2025-10-20T19:22:51.925589] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-10-20T19:22:51.925635] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-10-20T19:22:51.925721] [INFO ] [MAIN ] INR: Starting full primary refresh. [2025-10-20T19:22:52.253789] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-08. [2025-10-20T19:22:52.275209] [INFO ] [REMOTE ] INR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:22:53.051376] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-01. [2025-10-20T19:23:05.008921] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-01 (11.96 seconds). [2025-10-20T19:23:05.009970] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-02. [2025-10-20T19:23:05.276470] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-09 (14.63 seconds). [2025-10-20T19:23:05.282858] [INFO ] [REMOTE ] INGM: Fetching primary data for 2025-10. [2025-10-20T19:23:06.574798] [INFO ] [REMOTE ] INGM: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T19:23:06.579415] [INFO ] [REMOTE ] INGM: Full fetch complete (41766 records). [2025-10-20T19:23:06.579523] [DEBUG] [VALIDATOR ] INGM: Validating primary data. [2025-10-20T19:23:06.975221] [DEBUG] [VALIDATOR ] INGM: All expected trading days are present. [2025-10-20T19:23:06.975305] [DEBUG] [VALIDATOR ] INGM: Primary data validation passed. [2025-10-20T19:23:06.976013] [INFO ] [MAIN ] INGM: Fetched primary data is valid. [2025-10-20T19:23:06.976076] [DEBUG] [LOCAL ] INGM: Saving primary data. [2025-10-20T19:23:06.977189] [DEBUG] [LOCAL ] INGM: Trimmed 41766 records to 28268 (keeping 200 days of data) [2025-10-20T19:23:07.156699] [DEBUG] [LOCAL ] INGM: Primary data saved (28268 records). [2025-10-20T19:23:07.156851] [INFO ] [MAIN ] INGM: Full history saved. [2025-10-20T19:23:07.156935] [INFO ] [MAIN ] Update Progress: 1109/2367 symbols completed. INGM took 41.1s. Estimated remaining: 2h 42m 16s. Rate: 7.8 symbols/min [2025-10-20T19:23:07.333933] [INFO ] [MAIN ] INSP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:07.364516] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-01 (15.87 seconds). [2025-10-20T19:23:07.373314] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-02. [2025-10-20T19:23:07.408838] [DEBUG] [LOCAL ] INSP: Loaded 38922 existing records from primary storage. [2025-10-20T19:23:07.409179] [INFO ] [MAIN ] INSP: Loaded 38922 existing records. [2025-10-20T19:23:07.612350] [WARN ] [MAIN ] INSP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:07.612592] [INFO ] [MAIN ] INSP: Wiping existing primary data before full refresh. [2025-10-20T19:23:07.612639] [INFO ] [LOCAL ] INSP: Starting deletion of primary data. [2025-10-20T19:23:07.612867] [DEBUG] [LOCAL ] INSP: Deleted primary data file. [2025-10-20T19:23:07.612915] [INFO ] [MAIN ] INSP: Starting full primary refresh. [2025-10-20T19:23:07.975596] [INFO ] [REMOTE ] INSP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:07.976015] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-01. [2025-10-20T19:23:08.715253] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T19:23:08.726537] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-03. [2025-10-20T19:23:09.352140] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-08 (17.10 seconds). [2025-10-20T19:23:09.359720] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-09. [2025-10-20T19:23:09.394077] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:23:09.401764] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-02. [2025-10-20T19:23:10.201320] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T19:23:10.209481] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-04. [2025-10-20T19:23:10.391036] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T19:23:10.398395] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-03. [2025-10-20T19:23:10.482971] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-02 (5.47 seconds). [2025-10-20T19:23:10.492402] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-03. [2025-10-20T19:23:10.565222] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T19:23:10.574049] [INFO ] [REMOTE ] INGR: Fetching primary data for 2025-10. [2025-10-20T19:23:11.857248] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:23:11.865521] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-04. [2025-10-20T19:23:11.894817] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:23:11.904918] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-05. [2025-10-20T19:23:11.932396] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:23:11.941536] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-04. [2025-10-20T19:23:12.054432] [INFO ] [REMOTE ] INGR: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:23:12.060287] [INFO ] [REMOTE ] INGR: Full fetch complete (50728 records). [2025-10-20T19:23:12.060414] [DEBUG] [VALIDATOR ] INGR: Validating primary data. [2025-10-20T19:23:12.450933] [DEBUG] [VALIDATOR ] INGR: All expected trading days are present. [2025-10-20T19:23:12.451020] [DEBUG] [VALIDATOR ] INGR: Primary data validation passed. [2025-10-20T19:23:12.451794] [INFO ] [MAIN ] INGR: Fetched primary data is valid. [2025-10-20T19:23:12.451857] [DEBUG] [LOCAL ] INGR: Saving primary data. [2025-10-20T19:23:12.453022] [DEBUG] [LOCAL ] INGR: Trimmed 50728 records to 33774 (keeping 200 days of data) [2025-10-20T19:23:12.669164] [DEBUG] [LOCAL ] INGR: Primary data saved (33774 records). [2025-10-20T19:23:12.669293] [INFO ] [MAIN ] INGR: Full history saved. [2025-10-20T19:23:12.669367] [INFO ] [MAIN ] Update Progress: 1110/2367 symbols completed. INGR took 39.6s. Estimated remaining: 2h 42m 6s. Rate: 7.8 symbols/min [2025-10-20T19:23:12.842763] [INFO ] [MAIN ] INSW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:12.881547] [DEBUG] [LOCAL ] INSW: Loaded 40192 existing records from primary storage. [2025-10-20T19:23:12.882008] [INFO ] [MAIN ] INSW: Loaded 40192 existing records. [2025-10-20T19:23:13.088602] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T19:23:13.087558] [WARN ] [MAIN ] INSW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:13.113820] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-05. [2025-10-20T19:23:13.110919] [INFO ] [MAIN ] INSW: Wiping existing primary data before full refresh. [2025-10-20T19:23:13.115022] [INFO ] [LOCAL ] INSW: Starting deletion of primary data. [2025-10-20T19:23:13.115475] [DEBUG] [LOCAL ] INSW: Deleted primary data file. [2025-10-20T19:23:13.115997] [INFO ] [MAIN ] INSW: Starting full primary refresh. [2025-10-20T19:23:13.117044] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T19:23:13.194308] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-05. [2025-10-20T19:23:13.519219] [INFO ] [REMOTE ] INSW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:13.519616] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-01. [2025-10-20T19:23:13.550063] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:23:13.559423] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-06. [2025-10-20T19:23:14.113187] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T19:23:14.119395] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-06. [2025-10-20T19:23:14.507722] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:23:14.515395] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-06. [2025-10-20T19:23:14.797287] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T19:23:14.803518] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-07. [2025-10-20T19:23:15.070376] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T19:23:15.081974] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-07. [2025-10-20T19:23:15.187382] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:23:15.197113] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-02. [2025-10-20T19:23:15.703959] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:23:15.711925] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-08. [2025-10-20T19:23:15.870444] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:23:15.877992] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-07. [2025-10-20T19:23:16.188302] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:23:16.198464] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-08. [2025-10-20T19:23:16.253884] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T19:23:16.262543] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-03. [2025-10-20T19:23:17.063795] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:23:17.070598] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-09. [2025-10-20T19:23:17.181572] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T19:23:17.190436] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-08. [2025-10-20T19:23:17.349877] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T19:23:17.359379] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-09. [2025-10-20T19:23:17.842942] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:23:17.852097] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-04. [2025-10-20T19:23:17.964057] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T19:23:18.651490] [INFO ] [REMOTE ] INR: Fetching primary data for 2025-10. [2025-10-20T19:23:18.710573] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:23:18.745465] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:23:19.451443] [INFO ] [REMOTE ] INN: Fetching primary data for 2025-10. [2025-10-20T19:23:19.658990] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:23:19.989504] [INFO ] [REMOTE ] INR: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:23:19.996244] [INFO ] [REMOTE ] INR: Full fetch complete (36887 records). [2025-10-20T19:23:19.996456] [DEBUG] [VALIDATOR ] INR: Validating primary data. [2025-10-20T19:23:20.245626] [WARN ] [VALIDATOR ] INR: Validation failed - insufficient historical data. Have 181 unique days, need 200 days [2025-10-20T19:23:20.246232] [FATAL] [MAIN ] INR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:23:20.246329] [WARN ] [MAIN ] INR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:23:20.246403] [INFO ] [LOCAL ] INR: Starting deletion of primary data. [2025-10-20T19:23:20.246475] [DEBUG] [LOCAL ] INR: No primary data file found to delete. [2025-10-20T19:23:20.246510] [INFO ] [LOCAL ] INR: Starting deletion of derivative data. [2025-10-20T19:23:20.246549] [DEBUG] [LOCAL ] INR: No derivative data file found to delete. [2025-10-20T19:23:20.246613] [INFO ] [MAIN ] Update Progress: 1111/2367 symbols completed. INR took 28.5s. Estimated remaining: 2h 41m 58s. Rate: 7.8 symbols/min [2025-10-20T19:23:20.252103] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-09. [2025-10-20T19:23:20.453505] [INFO ] [MAIN ] INVH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:20.504587] [DEBUG] [LOCAL ] INVH: Loaded 53973 existing records from primary storage. [2025-10-20T19:23:20.505032] [INFO ] [MAIN ] INVH: Loaded 53973 existing records. [2025-10-20T19:23:20.684700] [WARN ] [MAIN ] INVH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:20.684961] [INFO ] [MAIN ] INVH: Wiping existing primary data before full refresh. [2025-10-20T19:23:20.685078] [INFO ] [LOCAL ] INVH: Starting deletion of primary data. [2025-10-20T19:23:20.685408] [DEBUG] [LOCAL ] INVH: Deleted primary data file. [2025-10-20T19:23:20.685501] [INFO ] [MAIN ] INVH: Starting full primary refresh. [2025-10-20T19:23:20.889265] [INFO ] [REMOTE ] INN: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:23:20.974017] [INFO ] [REMOTE ] INN: Full fetch complete (57765 records). [2025-10-20T19:23:20.995013] [DEBUG] [VALIDATOR ] INN: Validating primary data. [2025-10-20T19:23:21.057836] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-05. [2025-10-20T19:23:21.190293] [INFO ] [REMOTE ] INVH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:21.467706] [DEBUG] [VALIDATOR ] INN: All expected trading days are present. [2025-10-20T19:23:21.467775] [DEBUG] [VALIDATOR ] INN: Primary data validation passed. [2025-10-20T19:23:21.468793] [INFO ] [MAIN ] INN: Fetched primary data is valid. [2025-10-20T19:23:21.468856] [DEBUG] [LOCAL ] INN: Saving primary data. [2025-10-20T19:23:21.470179] [DEBUG] [LOCAL ] INN: Trimmed 57765 records to 41658 (keeping 200 days of data) [2025-10-20T19:23:21.710078] [DEBUG] [LOCAL ] INN: Primary data saved (41658 records). [2025-10-20T19:23:21.710194] [INFO ] [MAIN ] INN: Full history saved. [2025-10-20T19:23:21.710266] [INFO ] [MAIN ] Update Progress: 1112/2367 symbols completed. INN took 32.2s. Estimated remaining: 2h 41m 43s. Rate: 7.8 symbols/min [2025-10-20T19:23:21.856693] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-01. [2025-10-20T19:23:21.883088] [INFO ] [MAIN ] INVX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:21.884210] [DEBUG] [LOCAL ] INVX: No existing data file found in primary storage. [2025-10-20T19:23:21.884387] [WARN ] [MAIN ] INVX: No existing data found, performing full refresh instead. [2025-10-20T19:23:21.884549] [INFO ] [MAIN ] INVX: Wiping existing primary data before full refresh. [2025-10-20T19:23:21.884612] [INFO ] [LOCAL ] INVX: Starting deletion of primary data. [2025-10-20T19:23:21.884815] [DEBUG] [LOCAL ] INVX: No primary data file found to delete. [2025-10-20T19:23:21.884886] [INFO ] [MAIN ] INVX: Starting full primary refresh. [2025-10-20T19:23:22.179284] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T19:23:22.307146] [INFO ] [REMOTE ] INVX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:22.662329] [INFO ] [REMOTE ] INSP: Fetching primary data for 2025-10. [2025-10-20T19:23:22.666196] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:23:23.451373] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-01. [2025-10-20T19:23:23.638436] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:23:24.251370] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-02. [2025-10-20T19:23:24.578740] [INFO ] [REMOTE ] INSP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T19:23:24.586201] [INFO ] [REMOTE ] INSP: Full fetch complete (53858 records). [2025-10-20T19:23:24.586326] [DEBUG] [VALIDATOR ] INSP: Validating primary data. [2025-10-20T19:23:24.852335] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T19:23:25.030016] [DEBUG] [VALIDATOR ] INSP: All expected trading days are present. [2025-10-20T19:23:25.030102] [DEBUG] [VALIDATOR ] INSP: Primary data validation passed. [2025-10-20T19:23:25.030936] [INFO ] [MAIN ] INSP: Fetched primary data is valid. [2025-10-20T19:23:25.031000] [DEBUG] [LOCAL ] INSP: Saving primary data. [2025-10-20T19:23:25.032237] [DEBUG] [LOCAL ] INSP: Trimmed 53858 records to 38552 (keeping 200 days of data) [2025-10-20T19:23:25.085286] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-06. [2025-10-20T19:23:25.268778] [DEBUG] [LOCAL ] INSP: Primary data saved (38552 records). [2025-10-20T19:23:25.268907] [INFO ] [MAIN ] INSP: Full history saved. [2025-10-20T19:23:25.268984] [INFO ] [MAIN ] Update Progress: 1113/2367 symbols completed. INSP took 18.1s. Estimated remaining: 2h 41m 30s. Rate: 7.8 symbols/min [2025-10-20T19:23:25.441231] [INFO ] [MAIN ] IONQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:25.495150] [DEBUG] [LOCAL ] IONQ: Loaded 54568 existing records from primary storage. [2025-10-20T19:23:25.495449] [INFO ] [MAIN ] IONQ: Loaded 54568 existing records. [2025-10-20T19:23:25.689001] [WARN ] [MAIN ] IONQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:25.689214] [INFO ] [MAIN ] IONQ: Wiping existing primary data before full refresh. [2025-10-20T19:23:25.689262] [INFO ] [LOCAL ] IONQ: Starting deletion of primary data. [2025-10-20T19:23:25.689443] [DEBUG] [LOCAL ] IONQ: Deleted primary data file. [2025-10-20T19:23:25.689481] [INFO ] [MAIN ] IONQ: Starting full primary refresh. [2025-10-20T19:23:25.855489] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-02. [2025-10-20T19:23:26.112204] [INFO ] [REMOTE ] IONQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:26.115368] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T19:23:26.650926] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:23:26.658911] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-03. [2025-10-20T19:23:27.226215] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:23:27.451406] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-01. [2025-10-20T19:23:28.251390] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-07. [2025-10-20T19:23:28.407505] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:23:29.051392] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-03. [2025-10-20T19:23:29.666743] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:23:29.851389] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-08. [2025-10-20T19:23:29.972372] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-01 (2.52 seconds). [2025-10-20T19:23:29.999524] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:23:30.651432] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-02. [2025-10-20T19:23:31.351854] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:23:31.451408] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-09. [2025-10-20T19:23:32.251421] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-04. [2025-10-20T19:23:32.868382] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:23:33.051374] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-04. [2025-10-20T19:23:33.119530] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-02 (2.47 seconds). [2025-10-20T19:23:33.662107] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:23:33.851422] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-05. [2025-10-20T19:23:34.535466] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T19:23:34.651432] [INFO ] [REMOTE ] INSW: Fetching primary data for 2025-10. [2025-10-20T19:23:35.159730] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:23:35.451392] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-03. [2025-10-20T19:23:35.957752] [INFO ] [REMOTE ] INSW: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:23:35.963378] [INFO ] [REMOTE ] INSW: Full fetch complete (59762 records). [2025-10-20T19:23:35.963523] [DEBUG] [VALIDATOR ] INSW: Validating primary data. [2025-10-20T19:23:36.260002] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-05. [2025-10-20T19:23:36.356996] [DEBUG] [VALIDATOR ] INSW: All expected trading days are present. [2025-10-20T19:23:36.357083] [DEBUG] [VALIDATOR ] INSW: Primary data validation passed. [2025-10-20T19:23:36.357922] [INFO ] [MAIN ] INSW: Fetched primary data is valid. [2025-10-20T19:23:36.357984] [DEBUG] [LOCAL ] INSW: Saving primary data. [2025-10-20T19:23:36.359406] [DEBUG] [LOCAL ] INSW: Trimmed 59762 records to 39541 (keeping 200 days of data) [2025-10-20T19:23:36.598131] [DEBUG] [LOCAL ] INSW: Primary data saved (39541 records). [2025-10-20T19:23:36.598235] [INFO ] [MAIN ] INSW: Full history saved. [2025-10-20T19:23:36.598313] [INFO ] [MAIN ] Update Progress: 1114/2367 symbols completed. INSW took 23.9s. Estimated remaining: 2h 41m 27s. Rate: 7.8 symbols/min [2025-10-20T19:23:36.767159] [INFO ] [MAIN ] IOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:36.818854] [DEBUG] [LOCAL ] IOT: Loaded 54260 existing records from primary storage. [2025-10-20T19:23:36.819157] [INFO ] [MAIN ] IOT: Loaded 54260 existing records. [2025-10-20T19:23:36.991513] [WARN ] [MAIN ] IOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:36.991742] [INFO ] [MAIN ] IOT: Wiping existing primary data before full refresh. [2025-10-20T19:23:36.991827] [INFO ] [LOCAL ] IOT: Starting deletion of primary data. [2025-10-20T19:23:36.992142] [DEBUG] [LOCAL ] IOT: Deleted primary data file. [2025-10-20T19:23:36.992179] [INFO ] [MAIN ] IOT: Starting full primary refresh. [2025-10-20T19:23:37.060099] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-06. [2025-10-20T19:23:37.350576] [INFO ] [REMOTE ] IOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:37.851417] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-01. [2025-10-20T19:23:38.150197] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:23:38.209822] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-03 (2.76 seconds). [2025-10-20T19:23:38.421729] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T19:23:38.651419] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-06. [2025-10-20T19:23:39.451436] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-04. [2025-10-20T19:23:39.621436] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:23:40.251400] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-02. [2025-10-20T19:23:40.354325] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:23:41.051432] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-07. [2025-10-20T19:23:41.851361] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-07. [2025-10-20T19:23:42.009618] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:23:42.055826] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-04 (2.60 seconds). [2025-10-20T19:23:42.651345] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-05. [2025-10-20T19:23:42.863044] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T19:23:42.895507] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T19:23:43.451368] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-03. [2025-10-20T19:23:44.251431] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-08. [2025-10-20T19:23:45.051372] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-08. [2025-10-20T19:23:45.670787] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-03 (2.22 seconds). [2025-10-20T19:23:45.697246] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:23:45.790300] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-05 (3.14 seconds). [2025-10-20T19:23:45.851339] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-06. [2025-10-20T19:23:46.546383] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:23:46.651368] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-09. [2025-10-20T19:23:47.451365] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-09. [2025-10-20T19:23:48.251347] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-04. [2025-10-20T19:23:48.446922] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:23:48.486684] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-06 (2.64 seconds). [2025-10-20T19:23:48.512117] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:23:49.051385] [INFO ] [REMOTE ] INVH: Fetching primary data for 2025-10. [2025-10-20T19:23:49.851386] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-07. [2025-10-20T19:23:50.044137] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:23:50.651354] [INFO ] [REMOTE ] INVX: Fetching primary data for 2025-10. [2025-10-20T19:23:50.883764] [INFO ] [REMOTE ] INVH: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T19:23:50.891536] [INFO ] [REMOTE ] INVH: Full fetch complete (76963 records). [2025-10-20T19:23:50.891693] [DEBUG] [VALIDATOR ] INVH: Validating primary data. [2025-10-20T19:23:51.277455] [DEBUG] [VALIDATOR ] INVH: All expected trading days are present. [2025-10-20T19:23:51.277533] [DEBUG] [VALIDATOR ] INVH: Primary data validation passed. [2025-10-20T19:23:51.278476] [INFO ] [MAIN ] INVH: Fetched primary data is valid. [2025-10-20T19:23:51.278540] [DEBUG] [LOCAL ] INVH: Saving primary data. [2025-10-20T19:23:51.279899] [DEBUG] [LOCAL ] INVH: Trimmed 76963 records to 53195 (keeping 200 days of data) [2025-10-20T19:23:51.458459] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-05. [2025-10-20T19:23:51.597476] [DEBUG] [LOCAL ] INVH: Primary data saved (53195 records). [2025-10-20T19:23:51.597618] [INFO ] [MAIN ] INVH: Full history saved. [2025-10-20T19:23:51.597870] [INFO ] [MAIN ] Update Progress: 1115/2367 symbols completed. INVH took 31.4s. Estimated remaining: 2h 41m 27s. Rate: 7.8 symbols/min [2025-10-20T19:23:51.664446] [INFO ] [REMOTE ] INVX: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:23:51.720640] [INFO ] [REMOTE ] INVX: Full fetch complete (42936 records). [2025-10-20T19:23:51.731274] [DEBUG] [VALIDATOR ] INVX: Validating primary data. [2025-10-20T19:23:51.898951] [INFO ] [MAIN ] IP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:52.204291] [DEBUG] [VALIDATOR ] INVX: All expected trading days are present. [2025-10-20T19:23:52.204467] [DEBUG] [VALIDATOR ] INVX: Primary data validation passed. [2025-10-20T19:23:52.219729] [INFO ] [MAIN ] INVX: Fetched primary data is valid. [2025-10-20T19:23:52.220681] [DEBUG] [LOCAL ] INVX: Saving primary data. [2025-10-20T19:23:52.222031] [DEBUG] [LOCAL ] IP: Loaded 54087 existing records from primary storage. [2025-10-20T19:23:52.222359] [INFO ] [MAIN ] IP: Loaded 54087 existing records. [2025-10-20T19:23:52.324908] [DEBUG] [LOCAL ] INVX: Trimmed 42936 records to 29241 (keeping 200 days of data) [2025-10-20T19:23:52.533527] [WARN ] [MAIN ] IP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:52.573705] [INFO ] [MAIN ] IP: Wiping existing primary data before full refresh. [2025-10-20T19:23:52.578603] [INFO ] [LOCAL ] IP: Starting deletion of primary data. [2025-10-20T19:23:52.598436] [DEBUG] [LOCAL ] IP: Deleted primary data file. [2025-10-20T19:23:52.598851] [INFO ] [MAIN ] IP: Starting full primary refresh. [2025-10-20T19:23:52.664101] [DEBUG] [LOCAL ] INVX: Primary data saved (29241 records). [2025-10-20T19:23:52.678093] [INFO ] [MAIN ] INVX: Full history saved. [2025-10-20T19:23:52.690311] [INFO ] [MAIN ] Update Progress: 1116/2367 symbols completed. INVX took 31.0s. Estimated remaining: 2h 41m 12s. Rate: 7.8 symbols/min [2025-10-20T19:23:53.050187] [INFO ] [MAIN ] IPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:23:53.177195] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-07 (3.33 seconds). [2025-10-20T19:23:53.203415] [INFO ] [REMOTE ] IP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:53.211551] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-01. [2025-10-20T19:23:53.218388] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-08. [2025-10-20T19:23:53.258014] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:23:53.303130] [DEBUG] [LOCAL ] IPG: Loaded 54111 existing records from primary storage. [2025-10-20T19:23:53.303449] [INFO ] [MAIN ] IPG: Loaded 54111 existing records. [2025-10-20T19:23:53.480929] [WARN ] [MAIN ] IPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:23:53.481172] [INFO ] [MAIN ] IPG: Wiping existing primary data before full refresh. [2025-10-20T19:23:53.481230] [INFO ] [LOCAL ] IPG: Starting deletion of primary data. [2025-10-20T19:23:53.481472] [DEBUG] [LOCAL ] IPG: Deleted primary data file. [2025-10-20T19:23:53.481528] [INFO ] [MAIN ] IPG: Starting full primary refresh. [2025-10-20T19:23:53.831833] [INFO ] [REMOTE ] IPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:23:53.851334] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-01. [2025-10-20T19:23:54.651428] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-06. [2025-10-20T19:23:54.966410] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:23:55.451374] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-02. [2025-10-20T19:23:55.613879] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:23:55.920604] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-08 (2.70 seconds). [2025-10-20T19:23:56.251399] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-09. [2025-10-20T19:23:56.586030] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T19:23:56.743606] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:23:57.051376] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-07. [2025-10-20T19:23:57.851380] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-03. [2025-10-20T19:23:58.651356] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-02. [2025-10-20T19:23:58.886993] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-09 (2.64 seconds). [2025-10-20T19:23:59.019056] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:23:59.451382] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-08. [2025-10-20T19:23:59.644063] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:24:00.237982] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:24:00.251334] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-03. [2025-10-20T19:24:01.051375] [INFO ] [REMOTE ] IONQ: Fetching primary data for 2025-10. [2025-10-20T19:24:01.733337] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-10-20T19:24:01.851355] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-09. [2025-10-20T19:24:02.567876] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-03 (2.32 seconds). [2025-10-20T19:24:02.651383] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-04. [2025-10-20T19:24:03.451383] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-04. [2025-10-20T19:24:03.522448] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:24:03.999780] [INFO ] [REMOTE ] IONQ: Incremental fetch complete for 2025-10 (2.95 seconds). [2025-10-20T19:24:04.007867] [INFO ] [REMOTE ] IONQ: Full fetch complete (77968 records). [2025-10-20T19:24:04.008028] [DEBUG] [VALIDATOR ] IONQ: Validating primary data. [2025-10-20T19:24:04.257877] [INFO ] [REMOTE ] IOT: Fetching primary data for 2025-10. [2025-10-20T19:24:04.423000] [DEBUG] [VALIDATOR ] IONQ: All expected trading days are present. [2025-10-20T19:24:04.423171] [DEBUG] [VALIDATOR ] IONQ: Primary data validation passed. [2025-10-20T19:24:04.424363] [INFO ] [MAIN ] IONQ: Fetched primary data is valid. [2025-10-20T19:24:04.424467] [DEBUG] [LOCAL ] IONQ: Saving primary data. [2025-10-20T19:24:04.426241] [DEBUG] [LOCAL ] IONQ: Trimmed 77968 records to 53788 (keeping 200 days of data) [2025-10-20T19:24:04.548562] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:24:04.784047] [DEBUG] [LOCAL ] IONQ: Primary data saved (53788 records). [2025-10-20T19:24:04.784180] [INFO ] [MAIN ] IONQ: Full history saved. [2025-10-20T19:24:04.785032] [INFO ] [MAIN ] Update Progress: 1117/2367 symbols completed. IONQ took 39.5s. Estimated remaining: 2h 41m 9s. Rate: 7.8 symbols/min [2025-10-20T19:24:05.051417] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-05. [2025-10-20T19:24:06.306224] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-04 (2.85 seconds). [2025-10-20T19:24:06.317208] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-05. [2025-10-20T19:24:07.984742] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-05 (2.93 seconds). [2025-10-20T19:24:07.996103] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-06. [2025-10-20T19:24:08.212622] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:24:08.223821] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-06. [2025-10-20T19:24:08.735256] [INFO ] [REMOTE ] IOT: Incremental fetch complete for 2025-10 (4.48 seconds). [2025-10-20T19:24:08.742941] [INFO ] [REMOTE ] IOT: Full fetch complete (77553 records). [2025-10-20T19:24:08.743056] [DEBUG] [VALIDATOR ] IOT: Validating primary data. [2025-10-20T19:24:09.132562] [DEBUG] [VALIDATOR ] IOT: All expected trading days are present. [2025-10-20T19:24:09.132699] [DEBUG] [VALIDATOR ] IOT: Primary data validation passed. [2025-10-20T19:24:09.133671] [INFO ] [MAIN ] IOT: Fetched primary data is valid. [2025-10-20T19:24:09.133732] [DEBUG] [LOCAL ] IOT: Saving primary data. [2025-10-20T19:24:09.135079] [DEBUG] [LOCAL ] IOT: Trimmed 77553 records to 53478 (keeping 200 days of data) [2025-10-20T19:24:09.443048] [DEBUG] [LOCAL ] IOT: Primary data saved (53478 records). [2025-10-20T19:24:09.443351] [INFO ] [MAIN ] IOT: Full history saved. [2025-10-20T19:24:09.443584] [INFO ] [MAIN ] Update Progress: 1118/2367 symbols completed. IOT took 32.8s. Estimated remaining: 2h 40m 58s. Rate: 7.8 symbols/min [2025-10-20T19:24:09.473826] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:24:09.485458] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-07. [2025-10-20T19:24:09.982574] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:24:09.993177] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-07. [2025-10-20T19:24:11.316683] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T19:24:11.328113] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-08. [2025-10-20T19:24:11.644211] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T19:24:11.657145] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-08. [2025-10-20T19:24:13.044455] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:24:13.060743] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-09. [2025-10-20T19:24:13.489918] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:24:13.501239] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-09. [2025-10-20T19:24:14.778010] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:24:14.790268] [INFO ] [REMOTE ] IPG: Fetching primary data for 2025-10. [2025-10-20T19:24:15.260740] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:24:15.272303] [INFO ] [REMOTE ] IP: Fetching primary data for 2025-10. [2025-10-20T19:24:16.926808] [INFO ] [REMOTE ] IPG: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T19:24:16.938027] [INFO ] [REMOTE ] IPG: Full fetch complete (77434 records). [2025-10-20T19:24:16.938195] [DEBUG] [VALIDATOR ] IPG: Validating primary data. [2025-10-20T19:24:17.390242] [DEBUG] [VALIDATOR ] IPG: All expected trading days are present. [2025-10-20T19:24:17.390348] [DEBUG] [VALIDATOR ] IPG: Primary data validation passed. [2025-10-20T19:24:17.392176] [INFO ] [REMOTE ] IP: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T19:24:17.393488] [INFO ] [MAIN ] IPG: Fetched primary data is valid. [2025-10-20T19:24:17.397376] [DEBUG] [LOCAL ] IPG: Saving primary data. [2025-10-20T19:24:17.405497] [INFO ] [REMOTE ] IP: Full fetch complete (77413 records). [2025-10-20T19:24:17.405845] [DEBUG] [VALIDATOR ] IP: Validating primary data. [2025-10-20T19:24:17.406098] [DEBUG] [LOCAL ] IPG: Trimmed 77434 records to 53330 (keeping 200 days of data) [2025-10-20T19:24:18.100868] [DEBUG] [VALIDATOR ] IP: All expected trading days are present. [2025-10-20T19:24:18.103164] [DEBUG] [VALIDATOR ] IP: Primary data validation passed. [2025-10-20T19:24:18.104518] [INFO ] [MAIN ] IP: Fetched primary data is valid. [2025-10-20T19:24:18.104888] [DEBUG] [LOCAL ] IP: Saving primary data. [2025-10-20T19:24:18.126120] [DEBUG] [LOCAL ] IP: Trimmed 77413 records to 53306 (keeping 200 days of data) [2025-10-20T19:24:18.317710] [DEBUG] [LOCAL ] IPG: Primary data saved (53330 records). [2025-10-20T19:24:18.329010] [INFO ] [MAIN ] IPG: Full history saved. [2025-10-20T19:24:18.334891] [INFO ] [MAIN ] Update Progress: 1119/2367 symbols completed. IPG took 25.6s. Estimated remaining: 2h 40m 52s. Rate: 7.8 symbols/min [2025-10-20T19:24:18.540955] [DEBUG] [LOCAL ] IP: Primary data saved (53306 records). [2025-10-20T19:24:18.541097] [INFO ] [MAIN ] IP: Full history saved. [2025-10-20T19:24:18.541175] [INFO ] [MAIN ] Update Progress: 1120/2367 symbols completed. IP took 26.9s. Estimated remaining: 2h 40m 35s. Rate: 7.8 symbols/min [2025-10-20T19:24:19.157139] [INFO ] [MAIN ] IPI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.243418] [INFO ] [MAIN ] IQI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.265553] [INFO ] [MAIN ] IR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.269629] [INFO ] [MAIN ] IQV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:19.294431] [DEBUG] [LOCAL ] IQI: Loaded 18096 existing records from primary storage. [2025-10-20T19:24:19.294874] [INFO ] [MAIN ] IQI: Loaded 18096 existing records. [2025-10-20T19:24:19.337630] [DEBUG] [LOCAL ] IPI: Loaded 21783 existing records from primary storage. [2025-10-20T19:24:19.379796] [INFO ] [MAIN ] IPI: Loaded 21783 existing records. [2025-10-20T19:24:19.552756] [DEBUG] [LOCAL ] IR: Loaded 53548 existing records from primary storage. [2025-10-20T19:24:19.629298] [INFO ] [MAIN ] IR: Loaded 53548 existing records. [2025-10-20T19:24:19.709728] [DEBUG] [LOCAL ] IQV: Loaded 50750 existing records from primary storage. [2025-10-20T19:24:19.783313] [WARN ] [MAIN ] IQI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:19.821604] [INFO ] [MAIN ] IQV: Loaded 50750 existing records. [2025-10-20T19:24:19.868028] [INFO ] [MAIN ] IQI: Wiping existing primary data before full refresh. [2025-10-20T19:24:19.885267] [WARN ] [MAIN ] IPI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:19.920730] [INFO ] [LOCAL ] IQI: Starting deletion of primary data. [2025-10-20T19:24:19.915490] [INFO ] [MAIN ] IPI: Wiping existing primary data before full refresh. [2025-10-20T19:24:19.982126] [INFO ] [LOCAL ] IPI: Starting deletion of primary data. [2025-10-20T19:24:19.982039] [DEBUG] [LOCAL ] IQI: Deleted primary data file. [2025-10-20T19:24:20.024783] [WARN ] [MAIN ] IR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:20.025236] [DEBUG] [LOCAL ] IPI: Deleted primary data file. [2025-10-20T19:24:20.025068] [INFO ] [MAIN ] IQI: Starting full primary refresh. [2025-10-20T19:24:20.032831] [INFO ] [MAIN ] IPI: Starting full primary refresh. [2025-10-20T19:24:20.041686] [INFO ] [MAIN ] IR: Wiping existing primary data before full refresh. [2025-10-20T19:24:20.112483] [INFO ] [LOCAL ] IR: Starting deletion of primary data. [2025-10-20T19:24:20.234414] [DEBUG] [LOCAL ] IR: Deleted primary data file. [2025-10-20T19:24:20.311636] [INFO ] [MAIN ] IR: Starting full primary refresh. [2025-10-20T19:24:20.531270] [WARN ] [MAIN ] IQV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:20.669499] [INFO ] [MAIN ] IQV: Wiping existing primary data before full refresh. [2025-10-20T19:24:20.697340] [INFO ] [LOCAL ] IQV: Starting deletion of primary data. [2025-10-20T19:24:20.901105] [DEBUG] [LOCAL ] IQV: Deleted primary data file. [2025-10-20T19:24:20.963573] [INFO ] [MAIN ] IQV: Starting full primary refresh. [2025-10-20T19:24:21.426949] [INFO ] [REMOTE ] IQI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.442342] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-01. [2025-10-20T19:24:21.436228] [INFO ] [REMOTE ] IPI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.443829] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-01. [2025-10-20T19:24:21.629470] [INFO ] [REMOTE ] IR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.665771] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-01. [2025-10-20T19:24:21.773434] [INFO ] [REMOTE ] IQV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:21.773773] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-01. [2025-10-20T19:24:22.555803] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T19:24:22.559890] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-02. [2025-10-20T19:24:22.585472] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:24:22.589061] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-02. [2025-10-20T19:24:23.139117] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:24:23.148902] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-02. [2025-10-20T19:24:23.441296] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:24:23.451844] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-02. [2025-10-20T19:24:23.516736] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:24:23.520935] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-03. [2025-10-20T19:24:23.579969] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:24:23.585000] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-03. [2025-10-20T19:24:24.476729] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T19:24:24.486669] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-03. [2025-10-20T19:24:24.532253] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T19:24:24.536134] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-04. [2025-10-20T19:24:24.639847] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:24:24.645492] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-04. [2025-10-20T19:24:25.143135] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:24:25.153332] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-03. [2025-10-20T19:24:25.663628] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:24:25.693801] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:24:25.851383] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-05. [2025-10-20T19:24:25.978826] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:24:26.651348] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-04. [2025-10-20T19:24:26.882638] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:24:26.944215] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T19:24:27.451377] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-06. [2025-10-20T19:24:28.199487] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T19:24:28.251378] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-05. [2025-10-20T19:24:28.435593] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:24:29.051388] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-04. [2025-10-20T19:24:29.200188] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T19:24:29.851427] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-07. [2025-10-20T19:24:30.634922] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T19:24:30.651331] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-06. [2025-10-20T19:24:30.827837] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:24:31.451400] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-05. [2025-10-20T19:24:31.587123] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T19:24:32.251436] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-08. [2025-10-20T19:24:33.051391] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-07. [2025-10-20T19:24:33.151263] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:24:33.390393] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T19:24:33.851433] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-05. [2025-10-20T19:24:34.549910] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:24:34.651403] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-06. [2025-10-20T19:24:35.451346] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-08. [2025-10-20T19:24:35.659355] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T19:24:36.251398] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-06. [2025-10-20T19:24:36.466439] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:24:36.677598] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:24:37.051358] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-07. [2025-10-20T19:24:37.576594] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T19:24:37.851383] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-09. [2025-10-20T19:24:38.513071] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:24:38.651378] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-09. [2025-10-20T19:24:38.884708] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T19:24:39.451347] [INFO ] [REMOTE ] IQI: Fetching primary data for 2025-10. [2025-10-20T19:24:39.691395] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T19:24:40.251371] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-08. [2025-10-20T19:24:40.397306] [INFO ] [REMOTE ] IQI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T19:24:40.400051] [INFO ] [REMOTE ] IQI: Full fetch complete (26402 records). [2025-10-20T19:24:40.400121] [DEBUG] [VALIDATOR ] IQI: Validating primary data. [2025-10-20T19:24:40.811931] [DEBUG] [VALIDATOR ] IQI: All expected trading days are present. [2025-10-20T19:24:40.812024] [DEBUG] [VALIDATOR ] IQI: Primary data validation passed. [2025-10-20T19:24:40.812552] [INFO ] [MAIN ] IQI: Fetched primary data is valid. [2025-10-20T19:24:40.812591] [DEBUG] [LOCAL ] IQI: Saving primary data. [2025-10-20T19:24:40.813229] [DEBUG] [LOCAL ] IQI: Trimmed 26402 records to 17832 (keeping 200 days of data) [2025-10-20T19:24:40.917214] [DEBUG] [LOCAL ] IQI: Primary data saved (17832 records). [2025-10-20T19:24:40.917342] [INFO ] [MAIN ] IQI: Full history saved. [2025-10-20T19:24:40.917444] [INFO ] [MAIN ] Update Progress: 1121/2367 symbols completed. IQI took 22.4s. Estimated remaining: 2h 40m 44s. Rate: 7.8 symbols/min [2025-10-20T19:24:41.057062] [INFO ] [REMOTE ] IPI: Fetching primary data for 2025-10. [2025-10-20T19:24:41.089625] [INFO ] [MAIN ] IRM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:41.134021] [DEBUG] [LOCAL ] IRM: Loaded 49820 existing records from primary storage. [2025-10-20T19:24:41.134348] [INFO ] [MAIN ] IRM: Loaded 49820 existing records. [2025-10-20T19:24:41.312386] [WARN ] [MAIN ] IRM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:41.312629] [INFO ] [MAIN ] IRM: Wiping existing primary data before full refresh. [2025-10-20T19:24:41.312728] [INFO ] [LOCAL ] IRM: Starting deletion of primary data. [2025-10-20T19:24:41.313217] [DEBUG] [LOCAL ] IRM: Deleted primary data file. [2025-10-20T19:24:41.313278] [INFO ] [MAIN ] IRM: Starting full primary refresh. [2025-10-20T19:24:41.672457] [INFO ] [REMOTE ] IRM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:41.851378] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-01. [2025-10-20T19:24:41.909718] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:24:42.288425] [INFO ] [REMOTE ] IPI: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:24:42.297335] [INFO ] [REMOTE ] IPI: Full fetch complete (30170 records). [2025-10-20T19:24:42.297519] [DEBUG] [VALIDATOR ] IPI: Validating primary data. [2025-10-20T19:24:42.659749] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-09. [2025-10-20T19:24:42.674170] [DEBUG] [VALIDATOR ] IPI: All expected trading days are present. [2025-10-20T19:24:42.674262] [DEBUG] [VALIDATOR ] IPI: Primary data validation passed. [2025-10-20T19:24:42.674920] [INFO ] [MAIN ] IPI: Fetched primary data is valid. [2025-10-20T19:24:42.675101] [DEBUG] [LOCAL ] IPI: Saving primary data. [2025-10-20T19:24:42.676065] [DEBUG] [LOCAL ] IPI: Trimmed 30170 records to 21513 (keeping 200 days of data) [2025-10-20T19:24:42.799445] [DEBUG] [LOCAL ] IPI: Primary data saved (21513 records). [2025-10-20T19:24:42.799574] [INFO ] [MAIN ] IPI: Full history saved. [2025-10-20T19:24:42.799669] [INFO ] [MAIN ] Update Progress: 1122/2367 symbols completed. IPI took 24.3s. Estimated remaining: 2h 40m 30s. Rate: 7.8 symbols/min [2025-10-20T19:24:42.964736] [INFO ] [MAIN ] IRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:42.987578] [DEBUG] [LOCAL ] IRS: Loaded 23427 existing records from primary storage. [2025-10-20T19:24:42.987851] [INFO ] [MAIN ] IRS: Loaded 23427 existing records. [2025-10-20T19:24:43.166584] [WARN ] [MAIN ] IRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:43.166833] [INFO ] [MAIN ] IRS: Wiping existing primary data before full refresh. [2025-10-20T19:24:43.166882] [INFO ] [LOCAL ] IRS: Starting deletion of primary data. [2025-10-20T19:24:43.167073] [DEBUG] [LOCAL ] IRS: Deleted primary data file. [2025-10-20T19:24:43.167129] [INFO ] [MAIN ] IRS: Starting full primary refresh. [2025-10-20T19:24:43.452907] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-07. [2025-10-20T19:24:43.588320] [INFO ] [REMOTE ] IRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:43.627164] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T19:24:44.251467] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-02. [2025-10-20T19:24:44.415465] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:24:45.051375] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-01. [2025-10-20T19:24:45.265524] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T19:24:45.851358] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-08. [2025-10-20T19:24:45.967342] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:24:46.222598] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:24:46.651392] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-02. [2025-10-20T19:24:47.451441] [INFO ] [REMOTE ] IQV: Fetching primary data for 2025-10. [2025-10-20T19:24:47.653350] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T19:24:47.677992] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T19:24:47.899610] [INFO ] [REMOTE ] IQV: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T19:24:47.906907] [INFO ] [REMOTE ] IQV: Full fetch complete (72475 records). [2025-10-20T19:24:47.907048] [DEBUG] [VALIDATOR ] IQV: Validating primary data. [2025-10-20T19:24:48.254557] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-03. [2025-10-20T19:24:48.293371] [DEBUG] [VALIDATOR ] IQV: All expected trading days are present. [2025-10-20T19:24:48.293467] [DEBUG] [VALIDATOR ] IQV: Primary data validation passed. [2025-10-20T19:24:48.294502] [INFO ] [MAIN ] IQV: Fetched primary data is valid. [2025-10-20T19:24:48.294683] [DEBUG] [LOCAL ] IQV: Saving primary data. [2025-10-20T19:24:48.296190] [DEBUG] [LOCAL ] IQV: Trimmed 72475 records to 50022 (keeping 200 days of data) [2025-10-20T19:24:48.590515] [DEBUG] [LOCAL ] IQV: Primary data saved (50022 records). [2025-10-20T19:24:48.590671] [INFO ] [MAIN ] IQV: Full history saved. [2025-10-20T19:24:48.590770] [INFO ] [MAIN ] Update Progress: 1123/2367 symbols completed. IQV took 30.0s. Estimated remaining: 2h 40m 20s. Rate: 7.8 symbols/min [2025-10-20T19:24:48.758865] [INFO ] [MAIN ] IRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:48.807120] [DEBUG] [LOCAL ] IRT: Loaded 51273 existing records from primary storage. [2025-10-20T19:24:48.807454] [INFO ] [MAIN ] IRT: Loaded 51273 existing records. [2025-10-20T19:24:48.979716] [WARN ] [MAIN ] IRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:48.979911] [INFO ] [MAIN ] IRT: Wiping existing primary data before full refresh. [2025-10-20T19:24:48.979959] [INFO ] [LOCAL ] IRT: Starting deletion of primary data. [2025-10-20T19:24:48.980163] [DEBUG] [LOCAL ] IRT: Deleted primary data file. [2025-10-20T19:24:48.980219] [INFO ] [MAIN ] IRT: Starting full primary refresh. [2025-10-20T19:24:49.056508] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-03. [2025-10-20T19:24:49.348355] [INFO ] [REMOTE ] IRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:49.361264] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T19:24:49.851415] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-01. [2025-10-20T19:24:50.651395] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-09. [2025-10-20T19:24:50.710773] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:24:51.065120] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:24:51.451390] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-04. [2025-10-20T19:24:52.116401] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:24:52.251365] [INFO ] [REMOTE ] IR: Fetching primary data for 2025-10. [2025-10-20T19:24:52.484084] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T19:24:53.051404] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-05. [2025-10-20T19:24:53.851419] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-04. [2025-10-20T19:24:54.124421] [INFO ] [REMOTE ] IR: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:24:54.139638] [INFO ] [REMOTE ] IR: Full fetch complete (76488 records). [2025-10-20T19:24:54.145515] [DEBUG] [VALIDATOR ] IR: Validating primary data. [2025-10-20T19:24:54.146584] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:24:54.611498] [DEBUG] [VALIDATOR ] IR: All expected trading days are present. [2025-10-20T19:24:54.611574] [DEBUG] [VALIDATOR ] IR: Primary data validation passed. [2025-10-20T19:24:54.612522] [INFO ] [MAIN ] IR: Fetched primary data is valid. [2025-10-20T19:24:54.612598] [DEBUG] [LOCAL ] IR: Saving primary data. [2025-10-20T19:24:54.614002] [DEBUG] [LOCAL ] IR: Trimmed 76488 records to 52765 (keeping 200 days of data) [2025-10-20T19:24:54.668082] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-06. [2025-10-20T19:24:54.934924] [DEBUG] [LOCAL ] IR: Primary data saved (52765 records). [2025-10-20T19:24:54.935055] [INFO ] [MAIN ] IR: Full history saved. [2025-10-20T19:24:54.935130] [INFO ] [MAIN ] Update Progress: 1124/2367 symbols completed. IR took 36.4s. Estimated remaining: 2h 40m 11s. Rate: 7.8 symbols/min [2025-10-20T19:24:55.101196] [INFO ] [MAIN ] ISD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:24:55.117491] [DEBUG] [LOCAL ] ISD: Loaded 16121 existing records from primary storage. [2025-10-20T19:24:55.117777] [INFO ] [MAIN ] ISD: Loaded 16121 existing records. [2025-10-20T19:24:55.297981] [WARN ] [MAIN ] ISD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:24:55.298196] [INFO ] [MAIN ] ISD: Wiping existing primary data before full refresh. [2025-10-20T19:24:55.298244] [INFO ] [LOCAL ] ISD: Starting deletion of primary data. [2025-10-20T19:24:55.298561] [DEBUG] [LOCAL ] ISD: Deleted primary data file. [2025-10-20T19:24:55.298628] [INFO ] [MAIN ] ISD: Starting full primary refresh. [2025-10-20T19:24:55.451979] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-02. [2025-10-20T19:24:55.672753] [INFO ] [REMOTE ] ISD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:24:55.707568] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:24:55.787464] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:24:56.251412] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-01. [2025-10-20T19:24:56.988252] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:24:57.051353] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-03. [2025-10-20T19:24:57.301957] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T19:24:57.851386] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-07. [2025-10-20T19:24:58.655949] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-02. [2025-10-20T19:24:58.661565] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:24:58.809138] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:24:59.451408] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-04. [2025-10-20T19:24:59.580737] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T19:25:00.251426] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-05. [2025-10-20T19:25:01.051395] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-03. [2025-10-20T19:25:01.592954] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T19:25:01.851361] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-05. [2025-10-20T19:25:01.976202] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T19:25:02.187282] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:25:02.651374] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-08. [2025-10-20T19:25:03.451342] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-04. [2025-10-20T19:25:03.723548] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:25:03.782099] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:25:04.251344] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-09. [2025-10-20T19:25:04.638452] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T19:25:05.051386] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-06. [2025-10-20T19:25:05.706536] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:25:05.851437] [INFO ] [REMOTE ] IRS: Fetching primary data for 2025-10. [2025-10-20T19:25:06.651383] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-05. [2025-10-20T19:25:06.871429] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:25:07.451417] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-06. [2025-10-20T19:25:07.591992] [INFO ] [REMOTE ] IRS: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:25:07.596395] [INFO ] [REMOTE ] IRS: Full fetch complete (32708 records). [2025-10-20T19:25:07.596483] [DEBUG] [VALIDATOR ] IRS: Validating primary data. [2025-10-20T19:25:07.728961] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:25:07.987604] [DEBUG] [VALIDATOR ] IRS: All expected trading days are present. [2025-10-20T19:25:07.987716] [DEBUG] [VALIDATOR ] IRS: Primary data validation passed. [2025-10-20T19:25:07.988342] [INFO ] [MAIN ] IRS: Fetched primary data is valid. [2025-10-20T19:25:07.988383] [DEBUG] [LOCAL ] IRS: Saving primary data. [2025-10-20T19:25:07.989147] [DEBUG] [LOCAL ] IRS: Trimmed 32708 records to 23243 (keeping 200 days of data) [2025-10-20T19:25:08.120971] [DEBUG] [LOCAL ] IRS: Primary data saved (23243 records). [2025-10-20T19:25:08.121095] [INFO ] [MAIN ] IRS: Full history saved. [2025-10-20T19:25:08.121169] [INFO ] [MAIN ] Update Progress: 1125/2367 symbols completed. IRS took 25.3s. Estimated remaining: 2h 40m 9s. Rate: 7.8 symbols/min [2025-10-20T19:25:08.262368] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-06. [2025-10-20T19:25:08.293731] [INFO ] [MAIN ] IT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:08.334477] [DEBUG] [LOCAL ] IT: Loaded 42892 existing records from primary storage. [2025-10-20T19:25:08.334787] [INFO ] [MAIN ] IT: Loaded 42892 existing records. [2025-10-20T19:25:08.510701] [WARN ] [MAIN ] IT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:08.510967] [INFO ] [MAIN ] IT: Wiping existing primary data before full refresh. [2025-10-20T19:25:08.511015] [INFO ] [LOCAL ] IT: Starting deletion of primary data. [2025-10-20T19:25:08.511199] [DEBUG] [LOCAL ] IT: Deleted primary data file. [2025-10-20T19:25:08.511254] [INFO ] [MAIN ] IT: Starting full primary refresh. [2025-10-20T19:25:08.870869] [INFO ] [REMOTE ] IT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:09.051368] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-01. [2025-10-20T19:25:09.215042] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:25:09.259059] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:25:09.851387] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-07. [2025-10-20T19:25:10.544537] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:25:10.651349] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-02. [2025-10-20T19:25:10.773214] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:25:11.451357] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-07. [2025-10-20T19:25:11.824863] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:25:12.251396] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-08. [2025-10-20T19:25:13.051379] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-03. [2025-10-20T19:25:13.157907] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T19:25:13.344956] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:25:13.851419] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-08. [2025-10-20T19:25:14.586027] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T19:25:14.651334] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-04. [2025-10-20T19:25:15.451527] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-09. [2025-10-20T19:25:15.521463] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:25:16.164463] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:25:16.251400] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-05. [2025-10-20T19:25:16.375146] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T19:25:17.051428] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-07. [2025-10-20T19:25:17.620972] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:25:17.851380] [INFO ] [REMOTE ] ISD: Fetching primary data for 2025-10. [2025-10-20T19:25:18.651361] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-09. [2025-10-20T19:25:18.814693] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:25:18.852497] [INFO ] [REMOTE ] ISD: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:25:18.856194] [INFO ] [REMOTE ] ISD: Full fetch complete (24181 records). [2025-10-20T19:25:18.856332] [DEBUG] [VALIDATOR ] ISD: Validating primary data. [2025-10-20T19:25:19.232593] [DEBUG] [VALIDATOR ] ISD: All expected trading days are present. [2025-10-20T19:25:19.232692] [DEBUG] [VALIDATOR ] ISD: Primary data validation passed. [2025-10-20T19:25:19.233198] [INFO ] [MAIN ] ISD: Fetched primary data is valid. [2025-10-20T19:25:19.233238] [DEBUG] [LOCAL ] ISD: Saving primary data. [2025-10-20T19:25:19.233878] [DEBUG] [LOCAL ] ISD: Trimmed 24181 records to 15782 (keeping 200 days of data) [2025-10-20T19:25:19.329408] [DEBUG] [LOCAL ] ISD: Primary data saved (15782 records). [2025-10-20T19:25:19.329510] [INFO ] [MAIN ] ISD: Full history saved. [2025-10-20T19:25:19.329590] [INFO ] [MAIN ] Update Progress: 1126/2367 symbols completed. ISD took 24.4s. Estimated remaining: 2h 40m 5s. Rate: 7.8 symbols/min [2025-10-20T19:25:19.456440] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-08. [2025-10-20T19:25:19.503842] [INFO ] [MAIN ] ITGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:19.536485] [DEBUG] [LOCAL ] ITGR: Loaded 31722 existing records from primary storage. [2025-10-20T19:25:19.536772] [INFO ] [MAIN ] ITGR: Loaded 31722 existing records. [2025-10-20T19:25:19.715986] [WARN ] [MAIN ] ITGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:19.716246] [INFO ] [MAIN ] ITGR: Wiping existing primary data before full refresh. [2025-10-20T19:25:19.716301] [INFO ] [LOCAL ] ITGR: Starting deletion of primary data. [2025-10-20T19:25:19.716487] [DEBUG] [LOCAL ] ITGR: Deleted primary data file. [2025-10-20T19:25:19.716526] [INFO ] [MAIN ] ITGR: Starting full primary refresh. [2025-10-20T19:25:20.076324] [INFO ] [REMOTE ] ITGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:20.251411] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-06. [2025-10-20T19:25:20.362145] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:25:21.051343] [INFO ] [REMOTE ] IRT: Fetching primary data for 2025-10. [2025-10-20T19:25:21.164471] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:25:21.668845] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:25:21.851407] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-07. [2025-10-20T19:25:22.573925] [INFO ] [REMOTE ] IRT: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T19:25:22.581399] [INFO ] [REMOTE ] IRT: Full fetch complete (72912 records). [2025-10-20T19:25:22.581547] [DEBUG] [VALIDATOR ] IRT: Validating primary data. [2025-10-20T19:25:22.656568] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-01. [2025-10-20T19:25:22.970970] [DEBUG] [VALIDATOR ] IRT: All expected trading days are present. [2025-10-20T19:25:22.971046] [DEBUG] [VALIDATOR ] IRT: Primary data validation passed. [2025-10-20T19:25:22.971996] [INFO ] [MAIN ] IRT: Fetched primary data is valid. [2025-10-20T19:25:22.972056] [DEBUG] [LOCAL ] IRT: Saving primary data. [2025-10-20T19:25:22.973383] [DEBUG] [LOCAL ] IRT: Trimmed 72912 records to 50527 (keeping 200 days of data) [2025-10-20T19:25:23.273031] [DEBUG] [LOCAL ] IRT: Primary data saved (50527 records). [2025-10-20T19:25:23.273153] [INFO ] [MAIN ] IRT: Full history saved. [2025-10-20T19:25:23.273229] [INFO ] [MAIN ] Update Progress: 1127/2367 symbols completed. IRT took 34.7s. Estimated remaining: 2h 39m 53s. Rate: 7.8 symbols/min [2025-10-20T19:25:23.461365] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-09. [2025-10-20T19:25:23.464606] [INFO ] [MAIN ] ITT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:23.491974] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:25:23.530447] [DEBUG] [LOCAL ] ITT: Loaded 33177 existing records from primary storage. [2025-10-20T19:25:23.530729] [INFO ] [MAIN ] ITT: Loaded 33177 existing records. [2025-10-20T19:25:23.703555] [WARN ] [MAIN ] ITT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:23.703747] [INFO ] [MAIN ] ITT: Wiping existing primary data before full refresh. [2025-10-20T19:25:23.703795] [INFO ] [LOCAL ] ITT: Starting deletion of primary data. [2025-10-20T19:25:23.703981] [DEBUG] [LOCAL ] ITT: Deleted primary data file. [2025-10-20T19:25:23.704035] [INFO ] [MAIN ] ITT: Starting full primary refresh. [2025-10-20T19:25:23.825403] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T19:25:24.071624] [INFO ] [REMOTE ] ITT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:24.251412] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-01. [2025-10-20T19:25:25.051367] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-08. [2025-10-20T19:25:25.187094] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:25:25.740044] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:25:25.851339] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-02. [2025-10-20T19:25:26.651470] [INFO ] [REMOTE ] IRM: Fetching primary data for 2025-10. [2025-10-20T19:25:26.774726] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:25:26.841436] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T19:25:27.451354] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-09. [2025-10-20T19:25:28.251406] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-03. [2025-10-20T19:25:28.445352] [INFO ] [REMOTE ] IRM: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:25:28.452526] [INFO ] [REMOTE ] IRM: Full fetch complete (72238 records). [2025-10-20T19:25:28.452709] [DEBUG] [VALIDATOR ] IRM: Validating primary data. [2025-10-20T19:25:28.836818] [DEBUG] [VALIDATOR ] IRM: All expected trading days are present. [2025-10-20T19:25:28.837003] [DEBUG] [VALIDATOR ] IRM: Primary data validation passed. [2025-10-20T19:25:28.837993] [INFO ] [MAIN ] IRM: Fetched primary data is valid. [2025-10-20T19:25:28.838109] [DEBUG] [LOCAL ] IRM: Saving primary data. [2025-10-20T19:25:28.839584] [DEBUG] [LOCAL ] IRM: Trimmed 72238 records to 49015 (keeping 200 days of data) [2025-10-20T19:25:29.052929] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-02. [2025-10-20T19:25:29.142139] [DEBUG] [LOCAL ] IRM: Primary data saved (49015 records). [2025-10-20T19:25:29.142277] [INFO ] [MAIN ] IRM: Full history saved. [2025-10-20T19:25:29.142356] [INFO ] [MAIN ] Update Progress: 1128/2367 symbols completed. IRM took 48.2s. Estimated remaining: 2h 39m 43s. Rate: 7.8 symbols/min [2025-10-20T19:25:29.346716] [INFO ] [MAIN ] ITUB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:29.354043] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-10-20T19:25:29.415106] [DEBUG] [LOCAL ] ITUB: Loaded 54103 existing records from primary storage. [2025-10-20T19:25:29.415455] [INFO ] [MAIN ] ITUB: Loaded 54103 existing records. [2025-10-20T19:25:29.599435] [WARN ] [MAIN ] ITUB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:29.599752] [INFO ] [MAIN ] ITUB: Wiping existing primary data before full refresh. [2025-10-20T19:25:29.599813] [INFO ] [LOCAL ] ITUB: Starting deletion of primary data. [2025-10-20T19:25:29.600392] [DEBUG] [LOCAL ] ITUB: Deleted primary data file. [2025-10-20T19:25:29.600528] [INFO ] [MAIN ] ITUB: Starting full primary refresh. [2025-10-20T19:25:29.801575] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:25:29.856459] [INFO ] [REMOTE ] IT: Fetching primary data for 2025-10. [2025-10-20T19:25:30.022543] [INFO ] [REMOTE ] ITUB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:30.233916] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T19:25:30.651429] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-03. [2025-10-20T19:25:31.451376] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-04. [2025-10-20T19:25:31.729285] [INFO ] [REMOTE ] IT: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:25:31.736470] [INFO ] [REMOTE ] IT: Full fetch complete (57726 records). [2025-10-20T19:25:31.736613] [DEBUG] [VALIDATOR ] IT: Validating primary data. [2025-10-20T19:25:32.130948] [DEBUG] [VALIDATOR ] IT: All expected trading days are present. [2025-10-20T19:25:32.131143] [DEBUG] [VALIDATOR ] IT: Primary data validation passed. [2025-10-20T19:25:32.132031] [INFO ] [MAIN ] IT: Fetched primary data is valid. [2025-10-20T19:25:32.138604] [DEBUG] [LOCAL ] IT: Saving primary data. [2025-10-20T19:25:32.152585] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:25:32.160630] [DEBUG] [LOCAL ] IT: Trimmed 57726 records to 42199 (keeping 200 days of data) [2025-10-20T19:25:32.252439] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-04. [2025-10-20T19:25:32.409940] [DEBUG] [LOCAL ] IT: Primary data saved (42199 records). [2025-10-20T19:25:32.410066] [INFO ] [MAIN ] IT: Full history saved. [2025-10-20T19:25:32.410138] [INFO ] [MAIN ] Update Progress: 1129/2367 symbols completed. IT took 24.3s. Estimated remaining: 2h 39m 31s. Rate: 7.8 symbols/min [2025-10-20T19:25:32.580555] [INFO ] [MAIN ] ITW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:32.624296] [DEBUG] [LOCAL ] ITW: Loaded 46398 existing records from primary storage. [2025-10-20T19:25:32.624586] [INFO ] [MAIN ] ITW: Loaded 46398 existing records. [2025-10-20T19:25:32.812870] [WARN ] [MAIN ] ITW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:32.813175] [INFO ] [MAIN ] ITW: Wiping existing primary data before full refresh. [2025-10-20T19:25:32.813252] [INFO ] [LOCAL ] ITW: Starting deletion of primary data. [2025-10-20T19:25:32.813499] [DEBUG] [LOCAL ] ITW: Deleted primary data file. [2025-10-20T19:25:32.813573] [INFO ] [MAIN ] ITW: Starting full primary refresh. [2025-10-20T19:25:33.057121] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-01. [2025-10-20T19:25:33.176245] [INFO ] [REMOTE ] ITW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:33.205784] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:25:33.851873] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-01. [2025-10-20T19:25:33.851721] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:25:34.651364] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-05. [2025-10-20T19:25:34.862537] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:25:35.451382] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-02. [2025-10-20T19:25:35.501762] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:25:35.962048] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T19:25:36.251381] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-05. [2025-10-20T19:25:37.051357] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-02. [2025-10-20T19:25:37.131021] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:25:37.782568] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:25:37.851349] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-06. [2025-10-20T19:25:38.651400] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-03. [2025-10-20T19:25:38.711082] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:25:39.129712] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:25:39.451363] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-07. [2025-10-20T19:25:40.251409] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-06. [2025-10-20T19:25:40.466377] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T19:25:40.672997] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T19:25:41.051419] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-04. [2025-10-20T19:25:41.388107] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:25:41.851379] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-03. [2025-10-20T19:25:42.651403] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-07. [2025-10-20T19:25:42.743975] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T19:25:43.136343] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T19:25:43.451377] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-08. [2025-10-20T19:25:44.087716] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:25:44.251511] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-05. [2025-10-20T19:25:44.684277] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:25:45.051454] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-08. [2025-10-20T19:25:45.851363] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-09. [2025-10-20T19:25:46.131350] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:25:46.180751] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:25:46.651379] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-09. [2025-10-20T19:25:47.124714] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:25:47.451344] [INFO ] [REMOTE ] ITT: Fetching primary data for 2025-10. [2025-10-20T19:25:47.613778] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:25:48.251374] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-06. [2025-10-20T19:25:48.509716] [INFO ] [REMOTE ] ITT: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:25:48.514733] [INFO ] [REMOTE ] ITT: Full fetch complete (47495 records). [2025-10-20T19:25:48.514829] [DEBUG] [VALIDATOR ] ITT: Validating primary data. [2025-10-20T19:25:48.946789] [DEBUG] [VALIDATOR ] ITT: All expected trading days are present. [2025-10-20T19:25:48.946888] [DEBUG] [VALIDATOR ] ITT: Primary data validation passed. [2025-10-20T19:25:48.947698] [INFO ] [MAIN ] ITT: Fetched primary data is valid. [2025-10-20T19:25:48.947761] [DEBUG] [LOCAL ] ITT: Saving primary data. [2025-10-20T19:25:48.948747] [DEBUG] [LOCAL ] ITT: Trimmed 47495 records to 32539 (keeping 200 days of data) [2025-10-20T19:25:49.056482] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-04. [2025-10-20T19:25:49.164172] [DEBUG] [LOCAL ] ITT: Primary data saved (32539 records). [2025-10-20T19:25:49.164328] [INFO ] [MAIN ] ITT: Full history saved. [2025-10-20T19:25:49.164417] [INFO ] [MAIN ] Update Progress: 1130/2367 symbols completed. ITT took 25.9s. Estimated remaining: 2h 39m 33s. Rate: 7.8 symbols/min [2025-10-20T19:25:49.353844] [INFO ] [MAIN ] IVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:49.399819] [DEBUG] [LOCAL ] IVR: Loaded 48761 existing records from primary storage. [2025-10-20T19:25:49.400146] [INFO ] [MAIN ] IVR: Loaded 48761 existing records. [2025-10-20T19:25:49.582251] [WARN ] [MAIN ] IVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:49.582446] [INFO ] [MAIN ] IVR: Wiping existing primary data before full refresh. [2025-10-20T19:25:49.582494] [INFO ] [LOCAL ] IVR: Starting deletion of primary data. [2025-10-20T19:25:49.582697] [DEBUG] [LOCAL ] IVR: Deleted primary data file. [2025-10-20T19:25:49.582754] [INFO ] [MAIN ] IVR: Starting full primary refresh. [2025-10-20T19:25:49.855843] [INFO ] [REMOTE ] ITGR: Fetching primary data for 2025-10. [2025-10-20T19:25:49.938050] [INFO ] [REMOTE ] IVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:49.990050] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:25:50.651340] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-07. [2025-10-20T19:25:50.845360] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:25:51.240386] [INFO ] [REMOTE ] ITGR: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:25:51.244990] [INFO ] [REMOTE ] ITGR: Full fetch complete (42955 records). [2025-10-20T19:25:51.245098] [DEBUG] [VALIDATOR ] ITGR: Validating primary data. [2025-10-20T19:25:51.459373] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-05. [2025-10-20T19:25:51.615874] [DEBUG] [VALIDATOR ] ITGR: All expected trading days are present. [2025-10-20T19:25:51.615981] [DEBUG] [VALIDATOR ] ITGR: Primary data validation passed. [2025-10-20T19:25:51.616737] [INFO ] [MAIN ] ITGR: Fetched primary data is valid. [2025-10-20T19:25:51.616797] [DEBUG] [LOCAL ] ITGR: Saving primary data. [2025-10-20T19:25:51.617774] [DEBUG] [LOCAL ] ITGR: Trimmed 42955 records to 31097 (keeping 200 days of data) [2025-10-20T19:25:51.797685] [DEBUG] [LOCAL ] ITGR: Primary data saved (31097 records). [2025-10-20T19:25:51.797821] [INFO ] [MAIN ] ITGR: Full history saved. [2025-10-20T19:25:51.797910] [INFO ] [MAIN ] Update Progress: 1131/2367 symbols completed. ITGR took 32.5s. Estimated remaining: 2h 39m 19s. Rate: 7.8 symbols/min [2025-10-20T19:25:51.964341] [INFO ] [MAIN ] IVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:25:51.993619] [DEBUG] [LOCAL ] IVT: Loaded 31836 existing records from primary storage. [2025-10-20T19:25:51.993902] [INFO ] [MAIN ] IVT: Loaded 31836 existing records. [2025-10-20T19:25:52.165938] [WARN ] [MAIN ] IVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:25:52.166148] [INFO ] [MAIN ] IVT: Wiping existing primary data before full refresh. [2025-10-20T19:25:52.166192] [INFO ] [LOCAL ] IVT: Starting deletion of primary data. [2025-10-20T19:25:52.166407] [DEBUG] [LOCAL ] IVT: Deleted primary data file. [2025-10-20T19:25:52.166461] [INFO ] [MAIN ] IVT: Starting full primary refresh. [2025-10-20T19:25:52.261787] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-01. [2025-10-20T19:25:52.523234] [INFO ] [REMOTE ] IVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:25:52.556387] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T19:25:53.051397] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-01. [2025-10-20T19:25:53.177719] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T19:25:53.851435] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-06. [2025-10-20T19:25:54.025280] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:25:54.089294] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:25:54.651385] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-02. [2025-10-20T19:25:55.459442] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-02. [2025-10-20T19:25:55.482150] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:25:55.753189] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:25:56.251369] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-03. [2025-10-20T19:25:56.778597] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T19:25:57.051403] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-03. [2025-10-20T19:25:57.814531] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:25:57.851385] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-08. [2025-10-20T19:25:58.415046] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:25:58.651419] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-04. [2025-10-20T19:25:59.451352] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-07. [2025-10-20T19:25:59.560575] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:26:00.251355] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-04. [2025-10-20T19:26:00.504334] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:26:01.051367] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-05. [2025-10-20T19:26:01.599009] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-07 (2.15 seconds). [2025-10-20T19:26:01.851388] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-09. [2025-10-20T19:26:01.965084] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:26:02.651423] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-08. [2025-10-20T19:26:02.893340] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T19:26:03.173696] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:26:03.451381] [INFO ] [REMOTE ] ITUB: Fetching primary data for 2025-10. [2025-10-20T19:26:04.251392] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-06. [2025-10-20T19:26:04.295853] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:26:05.051388] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-05. [2025-10-20T19:26:05.452070] [INFO ] [REMOTE ] ITUB: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:26:05.460169] [INFO ] [REMOTE ] ITUB: Full fetch complete (77321 records). [2025-10-20T19:26:05.460308] [DEBUG] [VALIDATOR ] ITUB: Validating primary data. [2025-10-20T19:26:05.859207] [DEBUG] [VALIDATOR ] ITUB: All expected trading days are present. [2025-10-20T19:26:05.859456] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-09. [2025-10-20T19:26:05.859669] [DEBUG] [VALIDATOR ] ITUB: Primary data validation passed. [2025-10-20T19:26:05.861702] [INFO ] [MAIN ] ITUB: Fetched primary data is valid. [2025-10-20T19:26:05.861823] [DEBUG] [LOCAL ] ITUB: Saving primary data. [2025-10-20T19:26:05.886725] [DEBUG] [LOCAL ] ITUB: Trimmed 77321 records to 53327 (keeping 200 days of data) [2025-10-20T19:26:05.889142] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:26:06.000333] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T19:26:06.234379] [DEBUG] [LOCAL ] ITUB: Primary data saved (53327 records). [2025-10-20T19:26:06.234519] [INFO ] [MAIN ] ITUB: Full history saved. [2025-10-20T19:26:06.234597] [INFO ] [MAIN ] Update Progress: 1132/2367 symbols completed. ITUB took 37.1s. Estimated remaining: 2h 39m 19s. Rate: 7.8 symbols/min [2025-10-20T19:26:06.405909] [INFO ] [MAIN ] IVZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:06.455476] [DEBUG] [LOCAL ] IVZ: Loaded 53895 existing records from primary storage. [2025-10-20T19:26:06.455808] [INFO ] [MAIN ] IVZ: Loaded 53895 existing records. [2025-10-20T19:26:06.634395] [WARN ] [MAIN ] IVZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:06.634667] [INFO ] [MAIN ] IVZ: Wiping existing primary data before full refresh. [2025-10-20T19:26:06.634735] [INFO ] [LOCAL ] IVZ: Starting deletion of primary data. [2025-10-20T19:26:06.635106] [DEBUG] [LOCAL ] IVZ: Deleted primary data file. [2025-10-20T19:26:06.635166] [INFO ] [MAIN ] IVZ: Starting full primary refresh. [2025-10-20T19:26:06.657995] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-06. [2025-10-20T19:26:06.986895] [INFO ] [REMOTE ] IVZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:07.451337] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-07. [2025-10-20T19:26:07.636459] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:26:07.958731] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:26:08.251371] [INFO ] [REMOTE ] ITW: Fetching primary data for 2025-10. [2025-10-20T19:26:09.051392] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-01. [2025-10-20T19:26:09.170513] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:26:09.851388] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-07. [2025-10-20T19:26:09.951080] [INFO ] [REMOTE ] ITW: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T19:26:09.957842] [INFO ] [REMOTE ] ITW: Full fetch complete (66742 records). [2025-10-20T19:26:09.958002] [DEBUG] [VALIDATOR ] ITW: Validating primary data. [2025-10-20T19:26:10.344283] [DEBUG] [VALIDATOR ] ITW: All expected trading days are present. [2025-10-20T19:26:10.344400] [DEBUG] [VALIDATOR ] ITW: Primary data validation passed. [2025-10-20T19:26:10.345383] [INFO ] [MAIN ] ITW: Fetched primary data is valid. [2025-10-20T19:26:10.345502] [DEBUG] [LOCAL ] ITW: Saving primary data. [2025-10-20T19:26:10.347041] [DEBUG] [LOCAL ] ITW: Trimmed 66742 records to 45637 (keeping 200 days of data) [2025-10-20T19:26:10.652000] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-08. [2025-10-20T19:26:10.678118] [DEBUG] [LOCAL ] ITW: Primary data saved (45637 records). [2025-10-20T19:26:10.684199] [INFO ] [MAIN ] ITW: Full history saved. [2025-10-20T19:26:10.683987] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:26:10.685582] [INFO ] [MAIN ] Update Progress: 1133/2367 symbols completed. ITW took 38.3s. Estimated remaining: 2h 39m 8s. Rate: 7.8 symbols/min [2025-10-20T19:26:10.865347] [INFO ] [MAIN ] IX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:10.887457] [DEBUG] [LOCAL ] IX: Loaded 22572 existing records from primary storage. [2025-10-20T19:26:10.887727] [INFO ] [MAIN ] IX: Loaded 22572 existing records. [2025-10-20T19:26:11.064549] [WARN ] [MAIN ] IX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:11.064867] [INFO ] [MAIN ] IX: Wiping existing primary data before full refresh. [2025-10-20T19:26:11.064948] [INFO ] [LOCAL ] IX: Starting deletion of primary data. [2025-10-20T19:26:11.065375] [DEBUG] [LOCAL ] IX: Deleted primary data file. [2025-10-20T19:26:11.065447] [INFO ] [MAIN ] IX: Starting full primary refresh. [2025-10-20T19:26:11.380734] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T19:26:11.458998] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-02. [2025-10-20T19:26:11.463486] [INFO ] [REMOTE ] IX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:12.251404] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-08. [2025-10-20T19:26:12.349514] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T19:26:12.606097] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:26:13.051399] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-01. [2025-10-20T19:26:13.461082] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T19:26:13.851419] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-09. [2025-10-20T19:26:13.903068] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:26:14.651444] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-02. [2025-10-20T19:26:15.436950] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T19:26:15.451411] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-03. [2025-10-20T19:26:15.501007] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:26:16.251425] [INFO ] [REMOTE ] IVR: Fetching primary data for 2025-10. [2025-10-20T19:26:16.832855] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:26:17.051421] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-04. [2025-10-20T19:26:17.851347] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-03. [2025-10-20T19:26:18.182403] [INFO ] [REMOTE ] IVR: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:26:18.191072] [INFO ] [REMOTE ] IVR: Full fetch complete (69131 records). [2025-10-20T19:26:18.191317] [DEBUG] [VALIDATOR ] IVR: Validating primary data. [2025-10-20T19:26:18.569439] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:26:18.660376] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-09. [2025-10-20T19:26:18.679565] [DEBUG] [VALIDATOR ] IVR: All expected trading days are present. [2025-10-20T19:26:18.679749] [DEBUG] [VALIDATOR ] IVR: Primary data validation passed. [2025-10-20T19:26:18.681013] [INFO ] [MAIN ] IVR: Fetched primary data is valid. [2025-10-20T19:26:18.681088] [DEBUG] [LOCAL ] IVR: Saving primary data. [2025-10-20T19:26:18.682807] [DEBUG] [LOCAL ] IVR: Trimmed 69131 records to 47946 (keeping 200 days of data) [2025-10-20T19:26:18.982121] [DEBUG] [LOCAL ] IVR: Primary data saved (47946 records). [2025-10-20T19:26:18.982261] [INFO ] [MAIN ] IVR: Full history saved. [2025-10-20T19:26:18.982343] [INFO ] [MAIN ] Update Progress: 1134/2367 symbols completed. IVR took 29.8s. Estimated remaining: 2h 39m 1s. Rate: 7.8 symbols/min [2025-10-20T19:26:19.177718] [INFO ] [MAIN ] J: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:19.221982] [DEBUG] [LOCAL ] J: Loaded 41066 existing records from primary storage. [2025-10-20T19:26:19.222334] [INFO ] [MAIN ] J: Loaded 41066 existing records. [2025-10-20T19:26:19.406479] [WARN ] [MAIN ] J: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:19.406829] [INFO ] [MAIN ] J: Wiping existing primary data before full refresh. [2025-10-20T19:26:19.406937] [INFO ] [LOCAL ] J: Starting deletion of primary data. [2025-10-20T19:26:19.407420] [DEBUG] [LOCAL ] J: Deleted primary data file. [2025-10-20T19:26:19.407549] [INFO ] [MAIN ] J: Starting full primary refresh. [2025-10-20T19:26:19.461575] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-05. [2025-10-20T19:26:19.589044] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T19:26:19.833383] [INFO ] [REMOTE ] J: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:20.023238] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:26:20.251409] [INFO ] [REMOTE ] J: Fetching primary data for 2025-01. [2025-10-20T19:26:20.705578] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:26:21.051422] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-06. [2025-10-20T19:26:21.408968] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:26:21.800971] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T19:26:21.851422] [INFO ] [REMOTE ] J: Fetching primary data for 2025-02. [2025-10-20T19:26:22.651384] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-07. [2025-10-20T19:26:23.415511] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:26:23.451346] [INFO ] [REMOTE ] J: Fetching primary data for 2025-03. [2025-10-20T19:26:23.470727] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T19:26:24.251473] [INFO ] [REMOTE ] IVT: Fetching primary data for 2025-10. [2025-10-20T19:26:25.051574] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-04. [2025-10-20T19:26:25.055341] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:26:25.691135] [INFO ] [REMOTE ] IVT: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:26:25.696753] [INFO ] [REMOTE ] IVT: Full fetch complete (45873 records). [2025-10-20T19:26:25.696852] [DEBUG] [VALIDATOR ] IVT: Validating primary data. [2025-10-20T19:26:25.859579] [INFO ] [REMOTE ] J: Fetching primary data for 2025-04. [2025-10-20T19:26:26.085691] [DEBUG] [VALIDATOR ] IVT: All expected trading days are present. [2025-10-20T19:26:26.085772] [DEBUG] [VALIDATOR ] IVT: Primary data validation passed. [2025-10-20T19:26:26.086491] [INFO ] [MAIN ] IVT: Fetched primary data is valid. [2025-10-20T19:26:26.086531] [DEBUG] [LOCAL ] IVT: Saving primary data. [2025-10-20T19:26:26.087419] [DEBUG] [LOCAL ] IVT: Trimmed 45873 records to 31523 (keeping 200 days of data) [2025-10-20T19:26:26.276483] [DEBUG] [LOCAL ] IVT: Primary data saved (31523 records). [2025-10-20T19:26:26.276632] [INFO ] [MAIN ] IVT: Full history saved. [2025-10-20T19:26:26.276760] [INFO ] [MAIN ] Update Progress: 1135/2367 symbols completed. IVT took 34.5s. Estimated remaining: 2h 38m 52s. Rate: 7.8 symbols/min [2025-10-20T19:26:26.443564] [INFO ] [MAIN ] JBGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:26.482966] [DEBUG] [LOCAL ] JBGS: Loaded 42660 existing records from primary storage. [2025-10-20T19:26:26.483242] [INFO ] [MAIN ] JBGS: Loaded 42660 existing records. [2025-10-20T19:26:26.658256] [WARN ] [MAIN ] JBGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:26.658545] [INFO ] [MAIN ] JBGS: Wiping existing primary data before full refresh. [2025-10-20T19:26:26.654349] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-08. [2025-10-20T19:26:26.658834] [INFO ] [LOCAL ] JBGS: Starting deletion of primary data. [2025-10-20T19:26:26.659105] [DEBUG] [LOCAL ] JBGS: Deleted primary data file. [2025-10-20T19:26:26.659857] [INFO ] [MAIN ] JBGS: Starting full primary refresh. [2025-10-20T19:26:27.019375] [INFO ] [REMOTE ] JBGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:27.048922] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:26:27.451443] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-01. [2025-10-20T19:26:27.570514] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:26:27.717975] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T19:26:28.251446] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-09. [2025-10-20T19:26:28.751788] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:26:29.051349] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-05. [2025-10-20T19:26:29.220134] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:26:29.851434] [INFO ] [REMOTE ] IX: Fetching primary data for 2025-10. [2025-10-20T19:26:30.616897] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T19:26:30.646260] [INFO ] [REMOTE ] IX: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T19:26:30.649914] [INFO ] [REMOTE ] IX: Full fetch complete (28472 records). [2025-10-20T19:26:30.650085] [DEBUG] [VALIDATOR ] IX: Validating primary data. [2025-10-20T19:26:30.656536] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-06. [2025-10-20T19:26:31.085409] [DEBUG] [VALIDATOR ] IX: All expected trading days are present. [2025-10-20T19:26:31.085505] [DEBUG] [VALIDATOR ] IX: Primary data validation passed. [2025-10-20T19:26:31.086179] [INFO ] [MAIN ] IX: Fetched primary data is valid. [2025-10-20T19:26:31.086245] [DEBUG] [LOCAL ] IX: Saving primary data. [2025-10-20T19:26:31.087035] [DEBUG] [LOCAL ] IX: Trimmed 28472 records to 22282 (keeping 200 days of data) [2025-10-20T19:26:31.221837] [DEBUG] [LOCAL ] IX: Primary data saved (22282 records). [2025-10-20T19:26:31.222015] [INFO ] [MAIN ] IX: Full history saved. [2025-10-20T19:26:31.222126] [INFO ] [MAIN ] Update Progress: 1136/2367 symbols completed. IX took 20.5s. Estimated remaining: 2h 38m 42s. Rate: 7.8 symbols/min [2025-10-20T19:26:31.394565] [INFO ] [MAIN ] JBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:31.437699] [DEBUG] [LOCAL ] JBI: Loaded 43558 existing records from primary storage. [2025-10-20T19:26:31.438025] [INFO ] [MAIN ] JBI: Loaded 43558 existing records. [2025-10-20T19:26:31.458395] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-02. [2025-10-20T19:26:31.624821] [WARN ] [MAIN ] JBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:31.625059] [INFO ] [MAIN ] JBI: Wiping existing primary data before full refresh. [2025-10-20T19:26:31.625105] [INFO ] [LOCAL ] JBI: Starting deletion of primary data. [2025-10-20T19:26:31.625321] [DEBUG] [LOCAL ] JBI: Deleted primary data file. [2025-10-20T19:26:31.625376] [INFO ] [MAIN ] JBI: Starting full primary refresh. [2025-10-20T19:26:31.979032] [INFO ] [REMOTE ] JBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:32.010874] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:26:32.251372] [INFO ] [REMOTE ] J: Fetching primary data for 2025-05. [2025-10-20T19:26:32.923080] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:26:33.051383] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-03. [2025-10-20T19:26:33.833871] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T19:26:33.851357] [INFO ] [REMOTE ] J: Fetching primary data for 2025-06. [2025-10-20T19:26:34.349579] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:26:34.651380] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-04. [2025-10-20T19:26:34.858232] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:26:35.451442] [INFO ] [REMOTE ] J: Fetching primary data for 2025-07. [2025-10-20T19:26:35.879692] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:26:36.251444] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-05. [2025-10-20T19:26:36.521119] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T19:26:37.051377] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-01. [2025-10-20T19:26:37.800196] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:26:37.851350] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-06. [2025-10-20T19:26:38.302112] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T19:26:38.651493] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-07. [2025-10-20T19:26:39.308245] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:26:39.451412] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-07. [2025-10-20T19:26:40.251389] [INFO ] [REMOTE ] J: Fetching primary data for 2025-08. [2025-10-20T19:26:40.406222] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T19:26:40.915279] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:26:41.051406] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-08. [2025-10-20T19:26:41.639091] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:26:41.851366] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-02. [2025-10-20T19:26:42.651418] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-08. [2025-10-20T19:26:42.801582] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:26:43.144705] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:26:43.451370] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-03. [2025-10-20T19:26:43.997527] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:26:44.251463] [INFO ] [REMOTE ] J: Fetching primary data for 2025-09. [2025-10-20T19:26:44.949414] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:26:45.051370] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-04. [2025-10-20T19:26:45.634934] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:26:45.851396] [INFO ] [REMOTE ] J: Fetching primary data for 2025-10. [2025-10-20T19:26:46.519305] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:26:46.651338] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-05. [2025-10-20T19:26:47.384514] [INFO ] [REMOTE ] J: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T19:26:47.391243] [INFO ] [REMOTE ] J: Full fetch complete (59687 records). [2025-10-20T19:26:47.391385] [DEBUG] [VALIDATOR ] J: Validating primary data. [2025-10-20T19:26:47.456905] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-09. [2025-10-20T19:26:47.793890] [DEBUG] [VALIDATOR ] J: All expected trading days are present. [2025-10-20T19:26:47.793991] [DEBUG] [VALIDATOR ] J: Primary data validation passed. [2025-10-20T19:26:47.794838] [INFO ] [MAIN ] J: Fetched primary data is valid. [2025-10-20T19:26:47.794989] [DEBUG] [LOCAL ] J: Saving primary data. [2025-10-20T19:26:47.821961] [DEBUG] [LOCAL ] J: Trimmed 59687 records to 40502 (keeping 200 days of data) [2025-10-20T19:26:47.823832] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T19:26:48.081850] [DEBUG] [LOCAL ] J: Primary data saved (40502 records). [2025-10-20T19:26:48.081990] [INFO ] [MAIN ] J: Full history saved. [2025-10-20T19:26:48.082078] [INFO ] [MAIN ] Update Progress: 1137/2367 symbols completed. J took 29.1s. Estimated remaining: 2h 38m 44s. Rate: 7.7 symbols/min [2025-10-20T19:26:48.249747] [INFO ] [MAIN ] JBL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:48.251490] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-09. [2025-10-20T19:26:48.294784] [DEBUG] [LOCAL ] JBL: Loaded 49045 existing records from primary storage. [2025-10-20T19:26:48.295078] [INFO ] [MAIN ] JBL: Loaded 49045 existing records. [2025-10-20T19:26:48.475139] [WARN ] [MAIN ] JBL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:48.475339] [INFO ] [MAIN ] JBL: Wiping existing primary data before full refresh. [2025-10-20T19:26:48.475387] [INFO ] [LOCAL ] JBL: Starting deletion of primary data. [2025-10-20T19:26:48.476437] [DEBUG] [LOCAL ] JBL: Deleted primary data file. [2025-10-20T19:26:48.476497] [INFO ] [MAIN ] JBL: Starting full primary refresh. [2025-10-20T19:26:48.852082] [INFO ] [REMOTE ] JBL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:48.876548] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:26:49.051381] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-01. [2025-10-20T19:26:49.651778] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:26:49.851377] [INFO ] [REMOTE ] JBGS: Fetching primary data for 2025-10. [2025-10-20T19:26:50.651378] [INFO ] [REMOTE ] IVZ: Fetching primary data for 2025-10. [2025-10-20T19:26:50.837028] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T19:26:51.451439] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-06. [2025-10-20T19:26:51.522435] [INFO ] [REMOTE ] JBGS: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:26:51.528034] [INFO ] [REMOTE ] JBGS: Full fetch complete (59219 records). [2025-10-20T19:26:51.528224] [DEBUG] [VALIDATOR ] JBGS: Validating primary data. [2025-10-20T19:26:51.920229] [DEBUG] [VALIDATOR ] JBGS: All expected trading days are present. [2025-10-20T19:26:51.920312] [DEBUG] [VALIDATOR ] JBGS: Primary data validation passed. [2025-10-20T19:26:51.921149] [INFO ] [MAIN ] JBGS: Fetched primary data is valid. [2025-10-20T19:26:51.921211] [DEBUG] [LOCAL ] JBGS: Saving primary data. [2025-10-20T19:26:51.922342] [DEBUG] [LOCAL ] JBGS: Trimmed 59219 records to 41953 (keeping 200 days of data) [2025-10-20T19:26:52.170050] [DEBUG] [LOCAL ] JBGS: Primary data saved (41953 records). [2025-10-20T19:26:52.170175] [INFO ] [MAIN ] JBGS: Full history saved. [2025-10-20T19:26:52.170248] [INFO ] [MAIN ] Update Progress: 1138/2367 symbols completed. JBGS took 25.9s. Estimated remaining: 2h 38m 32s. Rate: 7.8 symbols/min [2025-10-20T19:26:52.258883] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-02. [2025-10-20T19:26:52.348975] [INFO ] [MAIN ] JBND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:52.380757] [DEBUG] [LOCAL ] JBND: Loaded 27442 existing records from primary storage. [2025-10-20T19:26:52.381112] [INFO ] [MAIN ] JBND: Loaded 27442 existing records. [2025-10-20T19:26:52.589285] [WARN ] [MAIN ] JBND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:26:52.591731] [INFO ] [MAIN ] JBND: Wiping existing primary data before full refresh. [2025-10-20T19:26:52.591467] [INFO ] [REMOTE ] IVZ: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:26:52.599392] [INFO ] [LOCAL ] JBND: Starting deletion of primary data. [2025-10-20T19:26:52.600089] [DEBUG] [LOCAL ] JBND: Deleted primary data file. [2025-10-20T19:26:52.600154] [INFO ] [MAIN ] JBND: Starting full primary refresh. [2025-10-20T19:26:52.636827] [INFO ] [REMOTE ] IVZ: Full fetch complete (77029 records). [2025-10-20T19:26:52.669534] [DEBUG] [VALIDATOR ] IVZ: Validating primary data. [2025-10-20T19:26:52.875446] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:26:53.079309] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-07. [2025-10-20T19:26:53.315580] [INFO ] [REMOTE ] JBND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:53.407920] [DEBUG] [VALIDATOR ] IVZ: All expected trading days are present. [2025-10-20T19:26:53.408075] [DEBUG] [VALIDATOR ] IVZ: Primary data validation passed. [2025-10-20T19:26:53.409073] [INFO ] [MAIN ] IVZ: Fetched primary data is valid. [2025-10-20T19:26:53.409134] [DEBUG] [LOCAL ] IVZ: Saving primary data. [2025-10-20T19:26:53.410479] [DEBUG] [LOCAL ] IVZ: Trimmed 77029 records to 53129 (keeping 200 days of data) [2025-10-20T19:26:53.756459] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:26:53.764532] [DEBUG] [LOCAL ] IVZ: Primary data saved (53129 records). [2025-10-20T19:26:53.774018] [INFO ] [MAIN ] IVZ: Full history saved. [2025-10-20T19:26:53.774144] [INFO ] [MAIN ] Update Progress: 1139/2367 symbols completed. IVZ took 47.5s. Estimated remaining: 2h 38m 18s. Rate: 7.8 symbols/min [2025-10-20T19:26:53.856494] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-03. [2025-10-20T19:26:53.945620] [INFO ] [MAIN ] JBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:53.945786] [DEBUG] [LOCAL ] JBS: No existing data file found in primary storage. [2025-10-20T19:26:53.945827] [WARN ] [MAIN ] JBS: No existing data found, performing full refresh instead. [2025-10-20T19:26:53.945967] [INFO ] [MAIN ] JBS: Wiping existing primary data before full refresh. [2025-10-20T19:26:53.946063] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-10-20T19:26:53.946128] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-10-20T19:26:53.946162] [INFO ] [MAIN ] JBS: Starting full primary refresh. [2025-10-20T19:26:54.290200] [INFO ] [REMOTE ] JBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:54.651356] [INFO ] [REMOTE ] JBS: Fetching primary data for 2025-01. [2025-10-20T19:26:54.731581] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T19:26:54.847457] [DEBUG] [REMOTE ] JBS: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:26:54.847956] [WARN ] [REMOTE ] JBS: No data returned for month 2025-01 [2025-10-20T19:26:54.848059] [FATAL] [REMOTE ] JBS: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:26:54.848110] [FATAL] [MAIN ] JBS: Failed to fetch primary data. [2025-10-20T19:26:54.848146] [WARN ] [MAIN ] JBS: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:26:54.848190] [INFO ] [LOCAL ] JBS: Starting deletion of primary data. [2025-10-20T19:26:54.848253] [DEBUG] [LOCAL ] JBS: No primary data file found to delete. [2025-10-20T19:26:54.848290] [INFO ] [LOCAL ] JBS: Starting deletion of derivative data. [2025-10-20T19:26:54.848334] [DEBUG] [LOCAL ] JBS: No derivative data file found to delete. [2025-10-20T19:26:54.848386] [INFO ] [MAIN ] Update Progress: 1140/2367 symbols completed. JBS took 1.1s. Estimated remaining: 2h 38m 3s. Rate: 7.8 symbols/min [2025-10-20T19:26:55.072054] [INFO ] [MAIN ] JBTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:26:55.072319] [DEBUG] [LOCAL ] JBTM: No existing data file found in primary storage. [2025-10-20T19:26:55.072385] [WARN ] [MAIN ] JBTM: No existing data found, performing full refresh instead. [2025-10-20T19:26:55.072526] [INFO ] [MAIN ] JBTM: Wiping existing primary data before full refresh. [2025-10-20T19:26:55.072582] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-10-20T19:26:55.072675] [DEBUG] [LOCAL ] JBTM: No primary data file found to delete. [2025-10-20T19:26:55.072745] [INFO ] [MAIN ] JBTM: Starting full primary refresh. [2025-10-20T19:26:55.430630] [INFO ] [REMOTE ] JBTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:26:55.451334] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-01. [2025-10-20T19:26:55.802266] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T19:26:56.251384] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-01. [2025-10-20T19:26:56.775030] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:26:57.051399] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-02. [2025-10-20T19:26:57.194866] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:26:57.851401] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-04. [2025-10-20T19:26:58.156963] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:26:58.651382] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-03. [2025-10-20T19:26:59.451383] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-02. [2025-10-20T19:26:59.508669] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T19:26:59.871895] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T19:27:00.251427] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-08. [2025-10-20T19:27:00.383911] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:27:01.051385] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-05. [2025-10-20T19:27:01.851416] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-03. [2025-10-20T19:27:02.080477] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:27:02.651384] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-04. [2025-10-20T19:27:02.814952] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T19:27:03.059397] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T19:27:03.451369] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-04. [2025-10-20T19:27:04.172209] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:27:04.251370] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-06. [2025-10-20T19:27:04.526173] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:27:05.051370] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-05. [2025-10-20T19:27:05.858215] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-09. [2025-10-20T19:27:05.898114] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:27:06.052848] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:27:06.651374] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-05. [2025-10-20T19:27:07.092814] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T19:27:07.451396] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-07. [2025-10-20T19:27:07.759982] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:27:08.251368] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-06. [2025-10-20T19:27:09.051383] [INFO ] [REMOTE ] JBI: Fetching primary data for 2025-10. [2025-10-20T19:27:09.329436] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:27:09.394557] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:27:09.851384] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-07. [2025-10-20T19:27:10.455264] [INFO ] [REMOTE ] JBI: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:27:10.461207] [INFO ] [REMOTE ] JBI: Full fetch complete (63764 records). [2025-10-20T19:27:10.461339] [DEBUG] [VALIDATOR ] JBI: Validating primary data. [2025-10-20T19:27:10.661362] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-08. [2025-10-20T19:27:10.870261] [DEBUG] [VALIDATOR ] JBI: All expected trading days are present. [2025-10-20T19:27:10.870349] [DEBUG] [VALIDATOR ] JBI: Primary data validation passed. [2025-10-20T19:27:10.871598] [INFO ] [MAIN ] JBI: Fetched primary data is valid. [2025-10-20T19:27:10.871708] [DEBUG] [LOCAL ] JBI: Saving primary data. [2025-10-20T19:27:10.873079] [DEBUG] [LOCAL ] JBI: Trimmed 63764 records to 42783 (keeping 200 days of data) [2025-10-20T19:27:11.123532] [DEBUG] [LOCAL ] JBI: Primary data saved (42783 records). [2025-10-20T19:27:11.123770] [INFO ] [MAIN ] JBI: Full history saved. [2025-10-20T19:27:11.124807] [INFO ] [MAIN ] Update Progress: 1141/2367 symbols completed. JBI took 39.9s. Estimated remaining: 2h 38m 4s. Rate: 7.8 symbols/min [2025-10-20T19:27:11.314251] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:27:11.316486] [INFO ] [MAIN ] JCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:11.332391] [DEBUG] [LOCAL ] JCE: Loaded 7507 existing records from primary storage. [2025-10-20T19:27:11.332603] [INFO ] [MAIN ] JCE: Loaded 7507 existing records. [2025-10-20T19:27:11.456451] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-06. [2025-10-20T19:27:11.509063] [WARN ] [MAIN ] JCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:11.509259] [INFO ] [MAIN ] JCE: Wiping existing primary data before full refresh. [2025-10-20T19:27:11.509306] [INFO ] [LOCAL ] JCE: Starting deletion of primary data. [2025-10-20T19:27:11.509472] [DEBUG] [LOCAL ] JCE: Deleted primary data file. [2025-10-20T19:27:11.509509] [INFO ] [MAIN ] JCE: Starting full primary refresh. [2025-10-20T19:27:11.868760] [INFO ] [REMOTE ] JCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:12.251356] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-08. [2025-10-20T19:27:12.620630] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T19:27:12.648062] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T19:27:13.051362] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-09. [2025-10-20T19:27:13.129372] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T19:27:13.851391] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-01. [2025-10-20T19:27:14.265258] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-01 (0.41 seconds). [2025-10-20T19:27:14.538591] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:27:14.651348] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-07. [2025-10-20T19:27:15.451390] [INFO ] [REMOTE ] JBL: Fetching primary data for 2025-10. [2025-10-20T19:27:15.889161] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:27:16.251436] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-09. [2025-10-20T19:27:17.051366] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-02. [2025-10-20T19:27:17.435238] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:27:17.523427] [INFO ] [REMOTE ] JBL: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T19:27:17.530972] [INFO ] [REMOTE ] JBL: Full fetch complete (70633 records). [2025-10-20T19:27:17.531140] [DEBUG] [VALIDATOR ] JBL: Validating primary data. [2025-10-20T19:27:17.655079] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:27:17.856491] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-08. [2025-10-20T19:27:17.936865] [DEBUG] [VALIDATOR ] JBL: All expected trading days are present. [2025-10-20T19:27:17.936969] [DEBUG] [VALIDATOR ] JBL: Primary data validation passed. [2025-10-20T19:27:17.937893] [INFO ] [MAIN ] JBL: Fetched primary data is valid. [2025-10-20T19:27:17.937954] [DEBUG] [LOCAL ] JBL: Saving primary data. [2025-10-20T19:27:17.939347] [DEBUG] [LOCAL ] JBL: Trimmed 70633 records to 48383 (keeping 200 days of data) [2025-10-20T19:27:18.244109] [DEBUG] [LOCAL ] JBL: Primary data saved (48383 records). [2025-10-20T19:27:18.244308] [INFO ] [MAIN ] JBL: Full history saved. [2025-10-20T19:27:18.244399] [INFO ] [MAIN ] Update Progress: 1142/2367 symbols completed. JBL took 30.2s. Estimated remaining: 2h 37m 56s. Rate: 7.8 symbols/min [2025-10-20T19:27:18.412217] [INFO ] [MAIN ] JCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:18.468102] [DEBUG] [LOCAL ] JCI: Loaded 54306 existing records from primary storage. [2025-10-20T19:27:18.468484] [INFO ] [MAIN ] JCI: Loaded 54306 existing records. [2025-10-20T19:27:18.643445] [WARN ] [MAIN ] JCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:18.643664] [INFO ] [MAIN ] JCI: Wiping existing primary data before full refresh. [2025-10-20T19:27:18.643724] [INFO ] [LOCAL ] JCI: Starting deletion of primary data. [2025-10-20T19:27:18.643936] [DEBUG] [LOCAL ] JCI: Deleted primary data file. [2025-10-20T19:27:18.644009] [INFO ] [MAIN ] JCI: Starting full primary refresh. [2025-10-20T19:27:18.651376] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-03. [2025-10-20T19:27:19.057106] [INFO ] [REMOTE ] JCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:19.250898] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:27:19.316936] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:27:19.451423] [INFO ] [REMOTE ] JBTM: Fetching primary data for 2025-10. [2025-10-20T19:27:20.251380] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-04. [2025-10-20T19:27:20.940484] [INFO ] [REMOTE ] JBTM: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:27:20.946515] [INFO ] [REMOTE ] JBTM: Full fetch complete (46931 records). [2025-10-20T19:27:20.946600] [DEBUG] [VALIDATOR ] JBTM: Validating primary data. [2025-10-20T19:27:21.036532] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:27:21.056602] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-01. [2025-10-20T19:27:21.150941] [WARN ] [VALIDATOR ] JBTM: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:27:21.151455] [FATAL] [MAIN ] JBTM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:27:21.151525] [WARN ] [MAIN ] JBTM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:27:21.151598] [INFO ] [LOCAL ] JBTM: Starting deletion of primary data. [2025-10-20T19:27:21.151687] [DEBUG] [LOCAL ] JBTM: No primary data file found to delete. [2025-10-20T19:27:21.151770] [INFO ] [LOCAL ] JBTM: Starting deletion of derivative data. [2025-10-20T19:27:21.151839] [DEBUG] [LOCAL ] JBTM: No derivative data file found to delete. [2025-10-20T19:27:21.151898] [INFO ] [MAIN ] Update Progress: 1143/2367 symbols completed. JBTM took 26.3s. Estimated remaining: 2h 37m 43s. Rate: 7.8 symbols/min [2025-10-20T19:27:21.318614] [INFO ] [MAIN ] JEF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:21.364543] [DEBUG] [LOCAL ] JEF: Loaded 50950 existing records from primary storage. [2025-10-20T19:27:21.364858] [INFO ] [MAIN ] JEF: Loaded 50950 existing records. [2025-10-20T19:27:21.539287] [WARN ] [MAIN ] JEF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:21.539461] [INFO ] [MAIN ] JEF: Wiping existing primary data before full refresh. [2025-10-20T19:27:21.539506] [INFO ] [LOCAL ] JEF: Starting deletion of primary data. [2025-10-20T19:27:21.539738] [DEBUG] [LOCAL ] JEF: Deleted primary data file. [2025-10-20T19:27:21.539796] [INFO ] [MAIN ] JEF: Starting full primary refresh. [2025-10-20T19:27:21.858513] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-09. [2025-10-20T19:27:21.886947] [INFO ] [REMOTE ] JEF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:22.651424] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-05. [2025-10-20T19:27:22.806361] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T19:27:23.245133] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:27:23.259824] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:27:23.451402] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-06. [2025-10-20T19:27:24.192943] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T19:27:24.251407] [INFO ] [REMOTE ] JBND: Fetching primary data for 2025-10. [2025-10-20T19:27:25.051461] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-07. [2025-10-20T19:27:25.586738] [INFO ] [REMOTE ] JBND: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:27:25.591986] [INFO ] [REMOTE ] JBND: Full fetch complete (35860 records). [2025-10-20T19:27:25.592068] [DEBUG] [VALIDATOR ] JBND: Validating primary data. [2025-10-20T19:27:25.856515] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-02. [2025-10-20T19:27:25.892075] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T19:27:25.987147] [DEBUG] [VALIDATOR ] JBND: All expected trading days are present. [2025-10-20T19:27:25.987234] [DEBUG] [VALIDATOR ] JBND: Primary data validation passed. [2025-10-20T19:27:25.988060] [INFO ] [MAIN ] JBND: Fetched primary data is valid. [2025-10-20T19:27:25.988127] [DEBUG] [LOCAL ] JBND: Saving primary data. [2025-10-20T19:27:25.992120] [DEBUG] [LOCAL ] JBND: Trimmed 35860 records to 27243 (keeping 200 days of data) [2025-10-20T19:27:26.160104] [DEBUG] [LOCAL ] JBND: Primary data saved (27243 records). [2025-10-20T19:27:26.160258] [INFO ] [MAIN ] JBND: Full history saved. [2025-10-20T19:27:26.160340] [INFO ] [MAIN ] Update Progress: 1144/2367 symbols completed. JBND took 34.0s. Estimated remaining: 2h 37m 32s. Rate: 7.8 symbols/min [2025-10-20T19:27:26.339343] [INFO ] [MAIN ] JELD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:26.380473] [DEBUG] [LOCAL ] JELD: Loaded 46697 existing records from primary storage. [2025-10-20T19:27:26.380793] [INFO ] [MAIN ] JELD: Loaded 46697 existing records. [2025-10-20T19:27:26.555681] [WARN ] [MAIN ] JELD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:26.555895] [INFO ] [MAIN ] JELD: Wiping existing primary data before full refresh. [2025-10-20T19:27:26.555944] [INFO ] [LOCAL ] JELD: Starting deletion of primary data. [2025-10-20T19:27:26.556116] [DEBUG] [LOCAL ] JELD: Deleted primary data file. [2025-10-20T19:27:26.556156] [INFO ] [MAIN ] JELD: Starting full primary refresh. [2025-10-20T19:27:26.656520] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-08. [2025-10-20T19:27:26.914213] [INFO ] [REMOTE ] JELD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:27.302731] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T19:27:27.451376] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-09. [2025-10-20T19:27:27.666323] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:27:28.230601] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:27:28.251351] [INFO ] [REMOTE ] JCE: Fetching primary data for 2025-10. [2025-10-20T19:27:29.051381] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-01. [2025-10-20T19:27:29.163751] [INFO ] [REMOTE ] JCE: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:27:29.166178] [INFO ] [REMOTE ] JCE: Full fetch complete (9739 records). [2025-10-20T19:27:29.166253] [DEBUG] [VALIDATOR ] JCE: Validating primary data. [2025-10-20T19:27:29.529580] [DEBUG] [VALIDATOR ] JCE: All expected trading days are present. [2025-10-20T19:27:29.529687] [DEBUG] [VALIDATOR ] JCE: Primary data validation passed. [2025-10-20T19:27:29.529947] [INFO ] [MAIN ] JCE: Fetched primary data is valid. [2025-10-20T19:27:29.529989] [DEBUG] [LOCAL ] JCE: Saving primary data. [2025-10-20T19:27:29.530403] [DEBUG] [LOCAL ] JCE: Trimmed 9739 records to 7308 (keeping 200 days of data) [2025-10-20T19:27:29.575005] [DEBUG] [LOCAL ] JCE: Primary data saved (7308 records). [2025-10-20T19:27:29.575154] [INFO ] [MAIN ] JCE: Full history saved. [2025-10-20T19:27:29.575236] [INFO ] [MAIN ] Update Progress: 1145/2367 symbols completed. JCE took 18.5s. Estimated remaining: 2h 37m 20s. Rate: 7.8 symbols/min [2025-10-20T19:27:29.750778] [INFO ] [MAIN ] JFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:29.782861] [DEBUG] [LOCAL ] JFR: Loaded 31519 existing records from primary storage. [2025-10-20T19:27:29.783144] [INFO ] [MAIN ] JFR: Loaded 31519 existing records. [2025-10-20T19:27:29.860471] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-03. [2025-10-20T19:27:29.966710] [WARN ] [MAIN ] JFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:29.966887] [INFO ] [MAIN ] JFR: Wiping existing primary data before full refresh. [2025-10-20T19:27:29.966934] [INFO ] [LOCAL ] JFR: Starting deletion of primary data. [2025-10-20T19:27:29.967108] [DEBUG] [LOCAL ] JFR: Deleted primary data file. [2025-10-20T19:27:29.967146] [INFO ] [MAIN ] JFR: Starting full primary refresh. [2025-10-20T19:27:30.333177] [INFO ] [REMOTE ] JFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:30.651378] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-01. [2025-10-20T19:27:30.750072] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T19:27:31.451424] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-01. [2025-10-20T19:27:31.702851] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T19:27:31.768764] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T19:27:32.251368] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-04. [2025-10-20T19:27:32.783348] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:27:33.051446] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-02. [2025-10-20T19:27:33.851434] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-02. [2025-10-20T19:27:34.118026] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:27:34.494334] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:27:34.651393] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-03. [2025-10-20T19:27:35.438397] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:27:35.451340] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-02. [2025-10-20T19:27:35.732206] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:27:36.251375] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-04. [2025-10-20T19:27:36.493048] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T19:27:37.051392] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-03. [2025-10-20T19:27:37.779262] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T19:27:37.851416] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-05. [2025-10-20T19:27:38.651376] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-03. [2025-10-20T19:27:38.758980] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T19:27:39.451379] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-04. [2025-10-20T19:27:39.543499] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:27:39.646164] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T19:27:40.251445] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-06. [2025-10-20T19:27:40.970889] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:27:41.051338] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-05. [2025-10-20T19:27:41.632441] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:27:41.851391] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-05. [2025-10-20T19:27:42.604424] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:27:42.651328] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-06. [2025-10-20T19:27:43.456508] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-07. [2025-10-20T19:27:43.479305] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:27:44.031398] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:27:44.251376] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-06. [2025-10-20T19:27:45.039862] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:27:45.051336] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-08. [2025-10-20T19:27:45.851424] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-04. [2025-10-20T19:27:45.926009] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:27:46.235013] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:27:46.651533] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-07. [2025-10-20T19:27:47.153512] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T19:27:47.451378] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-05. [2025-10-20T19:27:48.251350] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-09. [2025-10-20T19:27:48.348587] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:27:48.639747] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T19:27:49.051440] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-07. [2025-10-20T19:27:49.829712] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T19:27:49.851356] [INFO ] [REMOTE ] JELD: Fetching primary data for 2025-10. [2025-10-20T19:27:50.651369] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-08. [2025-10-20T19:27:50.917010] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T19:27:51.283119] [INFO ] [REMOTE ] JELD: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:27:51.290100] [INFO ] [REMOTE ] JELD: Full fetch complete (64565 records). [2025-10-20T19:27:51.290247] [DEBUG] [VALIDATOR ] JELD: Validating primary data. [2025-10-20T19:27:51.456709] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-08. [2025-10-20T19:27:51.729169] [DEBUG] [VALIDATOR ] JELD: All expected trading days are present. [2025-10-20T19:27:51.729264] [DEBUG] [VALIDATOR ] JELD: Primary data validation passed. [2025-10-20T19:27:51.730300] [INFO ] [MAIN ] JELD: Fetched primary data is valid. [2025-10-20T19:27:51.730373] [DEBUG] [LOCAL ] JELD: Saving primary data. [2025-10-20T19:27:51.731792] [DEBUG] [LOCAL ] JELD: Trimmed 64565 records to 46009 (keeping 200 days of data) [2025-10-20T19:27:51.989974] [DEBUG] [LOCAL ] JELD: Primary data saved (46009 records). [2025-10-20T19:27:51.990128] [INFO ] [MAIN ] JELD: Full history saved. [2025-10-20T19:27:51.990219] [INFO ] [MAIN ] Update Progress: 1146/2367 symbols completed. JELD took 25.8s. Estimated remaining: 2h 37m 28s. Rate: 7.8 symbols/min [2025-10-20T19:27:52.166768] [INFO ] [MAIN ] JGH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:27:52.198609] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:27:52.219573] [DEBUG] [LOCAL ] JGH: Loaded 12828 existing records from primary storage. [2025-10-20T19:27:52.219846] [INFO ] [MAIN ] JGH: Loaded 12828 existing records. [2025-10-20T19:27:52.256524] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-06. [2025-10-20T19:27:52.405175] [WARN ] [MAIN ] JGH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:27:52.405433] [INFO ] [MAIN ] JGH: Wiping existing primary data before full refresh. [2025-10-20T19:27:52.405481] [INFO ] [LOCAL ] JGH: Starting deletion of primary data. [2025-10-20T19:27:52.405697] [DEBUG] [LOCAL ] JGH: Deleted primary data file. [2025-10-20T19:27:52.405758] [INFO ] [MAIN ] JGH: Starting full primary refresh. [2025-10-20T19:27:52.771468] [INFO ] [REMOTE ] JGH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:27:53.051391] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-01. [2025-10-20T19:27:53.350446] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T19:27:53.507800] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:27:53.783510] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:27:53.851396] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-02. [2025-10-20T19:27:54.440495] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T19:27:54.651368] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-03. [2025-10-20T19:27:55.417713] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:27:55.451360] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-04. [2025-10-20T19:27:56.251417] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-09. [2025-10-20T19:27:56.274564] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T19:27:57.051395] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-07. [2025-10-20T19:27:57.851387] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-09. [2025-10-20T19:27:57.899801] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T19:27:58.466800] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:27:58.651412] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-08. [2025-10-20T19:27:59.451363] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-05. [2025-10-20T19:27:59.688053] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T19:27:59.717892] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:28:00.220003] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T19:28:00.251457] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-09. [2025-10-20T19:28:01.051374] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-06. [2025-10-20T19:28:01.836580] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T19:28:01.852603] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-07. [2025-10-20T19:28:02.016813] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:28:02.598569] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T19:28:02.651366] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-08. [2025-10-20T19:28:03.451353] [INFO ] [REMOTE ] JFR: Fetching primary data for 2025-10. [2025-10-20T19:28:03.481518] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T19:28:04.251370] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-09. [2025-10-20T19:28:04.908741] [INFO ] [REMOTE ] JFR: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T19:28:04.914079] [INFO ] [REMOTE ] JFR: Full fetch complete (48310 records). [2025-10-20T19:28:04.914182] [DEBUG] [VALIDATOR ] JFR: Validating primary data. [2025-10-20T19:28:05.056410] [INFO ] [REMOTE ] JEF: Fetching primary data for 2025-10. [2025-10-20T19:28:05.132946] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T19:28:05.318258] [DEBUG] [VALIDATOR ] JFR: All expected trading days are present. [2025-10-20T19:28:05.318387] [DEBUG] [VALIDATOR ] JFR: Primary data validation passed. [2025-10-20T19:28:05.319340] [INFO ] [MAIN ] JFR: Fetched primary data is valid. [2025-10-20T19:28:05.319412] [DEBUG] [LOCAL ] JFR: Saving primary data. [2025-10-20T19:28:05.320487] [DEBUG] [LOCAL ] JFR: Trimmed 48310 records to 30952 (keeping 200 days of data) [2025-10-20T19:28:05.499522] [DEBUG] [LOCAL ] JFR: Primary data saved (30952 records). [2025-10-20T19:28:05.499639] [INFO ] [MAIN ] JFR: Full history saved. [2025-10-20T19:28:05.499754] [INFO ] [MAIN ] Update Progress: 1147/2367 symbols completed. JFR took 35.9s. Estimated remaining: 2h 37m 26s. Rate: 7.7 symbols/min [2025-10-20T19:28:05.669011] [INFO ] [MAIN ] JHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:05.712019] [DEBUG] [LOCAL ] JHG: Loaded 48298 existing records from primary storage. [2025-10-20T19:28:05.712329] [INFO ] [MAIN ] JHG: Loaded 48298 existing records. [2025-10-20T19:28:05.857998] [INFO ] [REMOTE ] JCI: Fetching primary data for 2025-10. [2025-10-20T19:28:05.889053] [WARN ] [MAIN ] JHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:05.889221] [INFO ] [MAIN ] JHG: Wiping existing primary data before full refresh. [2025-10-20T19:28:05.889267] [INFO ] [LOCAL ] JHG: Starting deletion of primary data. [2025-10-20T19:28:05.889516] [DEBUG] [LOCAL ] JHG: Deleted primary data file. [2025-10-20T19:28:05.889575] [INFO ] [MAIN ] JHG: Starting full primary refresh. [2025-10-20T19:28:06.244692] [INFO ] [REMOTE ] JHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:06.651422] [INFO ] [REMOTE ] JGH: Fetching primary data for 2025-10. [2025-10-20T19:28:06.805676] [INFO ] [REMOTE ] JCI: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T19:28:06.813409] [INFO ] [REMOTE ] JCI: Full fetch complete (77644 records). [2025-10-20T19:28:06.813541] [DEBUG] [VALIDATOR ] JCI: Validating primary data. [2025-10-20T19:28:07.176440] [INFO ] [REMOTE ] JEF: Incremental fetch complete for 2025-10 (2.12 seconds). [2025-10-20T19:28:07.221159] [INFO ] [REMOTE ] JEF: Full fetch complete (73337 records). [2025-10-20T19:28:07.240634] [DEBUG] [VALIDATOR ] JEF: Validating primary data. [2025-10-20T19:28:07.244395] [DEBUG] [VALIDATOR ] JCI: All expected trading days are present. [2025-10-20T19:28:07.244498] [DEBUG] [VALIDATOR ] JCI: Primary data validation passed. [2025-10-20T19:28:07.245572] [INFO ] [MAIN ] JCI: Fetched primary data is valid. [2025-10-20T19:28:07.245674] [DEBUG] [LOCAL ] JCI: Saving primary data. [2025-10-20T19:28:07.357303] [DEBUG] [LOCAL ] JCI: Trimmed 77644 records to 53523 (keeping 200 days of data) [2025-10-20T19:28:07.460516] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-01. [2025-10-20T19:28:07.653127] [INFO ] [REMOTE ] JGH: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:28:07.681936] [INFO ] [REMOTE ] JGH: Full fetch complete (16405 records). [2025-10-20T19:28:07.690147] [DEBUG] [VALIDATOR ] JGH: Validating primary data. [2025-10-20T19:28:08.001761] [DEBUG] [VALIDATOR ] JEF: All expected trading days are present. [2025-10-20T19:28:08.018928] [DEBUG] [VALIDATOR ] JEF: Primary data validation passed. [2025-10-20T19:28:08.033176] [INFO ] [MAIN ] JEF: Fetched primary data is valid. [2025-10-20T19:28:08.048753] [DEBUG] [LOCAL ] JEF: Saving primary data. [2025-10-20T19:28:08.193342] [DEBUG] [LOCAL ] JEF: Trimmed 73337 records to 50184 (keeping 200 days of data) [2025-10-20T19:28:08.371638] [DEBUG] [LOCAL ] JCI: Primary data saved (53523 records). [2025-10-20T19:28:08.397893] [INFO ] [MAIN ] JCI: Full history saved. [2025-10-20T19:28:08.410084] [INFO ] [MAIN ] Update Progress: 1148/2367 symbols completed. JCI took 50.2s. Estimated remaining: 2h 37m 13s. Rate: 7.8 symbols/min [2025-10-20T19:28:08.717323] [DEBUG] [VALIDATOR ] JGH: All expected trading days are present. [2025-10-20T19:28:08.759178] [DEBUG] [VALIDATOR ] JGH: Primary data validation passed. [2025-10-20T19:28:08.760180] [INFO ] [MAIN ] JGH: Fetched primary data is valid. [2025-10-20T19:28:08.774709] [DEBUG] [LOCAL ] JGH: Saving primary data. [2025-10-20T19:28:08.825562] [INFO ] [MAIN ] JHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:08.839397] [DEBUG] [LOCAL ] JGH: Trimmed 16405 records to 12697 (keeping 200 days of data) [2025-10-20T19:28:08.911531] [DEBUG] [LOCAL ] JHI: Loaded 4176 existing records from primary storage. [2025-10-20T19:28:08.952043] [INFO ] [MAIN ] JHI: Loaded 4176 existing records. [2025-10-20T19:28:09.018466] [DEBUG] [LOCAL ] JGH: Primary data saved (12697 records). [2025-10-20T19:28:09.051480] [INFO ] [MAIN ] JGH: Full history saved. [2025-10-20T19:28:09.069317] [INFO ] [MAIN ] Update Progress: 1149/2367 symbols completed. JGH took 17.1s. Estimated remaining: 2h 36m 58s. Rate: 7.8 symbols/min [2025-10-20T19:28:09.111458] [DEBUG] [LOCAL ] JEF: Primary data saved (50184 records). [2025-10-20T19:28:09.132284] [INFO ] [MAIN ] JEF: Full history saved. [2025-10-20T19:28:09.193266] [INFO ] [MAIN ] Update Progress: 1150/2367 symbols completed. JEF took 48.0s. Estimated remaining: 2h 36m 42s. Rate: 7.8 symbols/min [2025-10-20T19:28:09.287555] [WARN ] [MAIN ] JHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:09.343435] [INFO ] [MAIN ] JHI: Wiping existing primary data before full refresh. [2025-10-20T19:28:09.391416] [INFO ] [LOCAL ] JHI: Starting deletion of primary data. [2025-10-20T19:28:09.455954] [DEBUG] [LOCAL ] JHI: Deleted primary data file. [2025-10-20T19:28:09.458476] [INFO ] [MAIN ] JHI: Starting full primary refresh. [2025-10-20T19:28:09.480987] [INFO ] [MAIN ] JHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:09.677161] [DEBUG] [LOCAL ] JHS: Loaded 2952 existing records from primary storage. [2025-10-20T19:28:09.669525] [INFO ] [MAIN ] JHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:09.689109] [INFO ] [MAIN ] JHS: Loaded 2952 existing records. [2025-10-20T19:28:09.702108] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-01 (2.24 seconds). [2025-10-20T19:28:09.760485] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-02. [2025-10-20T19:28:09.998996] [WARN ] [MAIN ] JHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:10.038368] [INFO ] [MAIN ] JHS: Wiping existing primary data before full refresh. [2025-10-20T19:28:10.064913] [INFO ] [LOCAL ] JHS: Starting deletion of primary data. [2025-10-20T19:28:10.086071] [DEBUG] [LOCAL ] JHS: Deleted primary data file. [2025-10-20T19:28:10.089498] [INFO ] [MAIN ] JHS: Starting full primary refresh. [2025-10-20T19:28:10.280216] [INFO ] [REMOTE ] JHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:10.291110] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-01. [2025-10-20T19:28:10.294673] [DEBUG] [LOCAL ] JHX: Loaded 53939 existing records from primary storage. [2025-10-20T19:28:10.311200] [INFO ] [MAIN ] JHX: Loaded 53939 existing records. [2025-10-20T19:28:10.781173] [WARN ] [MAIN ] JHX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:10.786526] [INFO ] [MAIN ] JHX: Wiping existing primary data before full refresh. [2025-10-20T19:28:10.792658] [INFO ] [LOCAL ] JHX: Starting deletion of primary data. [2025-10-20T19:28:10.793945] [DEBUG] [LOCAL ] JHX: Deleted primary data file. [2025-10-20T19:28:10.809432] [INFO ] [MAIN ] JHX: Starting full primary refresh. [2025-10-20T19:28:10.813121] [INFO ] [REMOTE ] JHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:10.844402] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-01. [2025-10-20T19:28:10.944909] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:28:10.958532] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-02. [2025-10-20T19:28:11.184120] [INFO ] [REMOTE ] JHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:11.233039] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T19:28:11.261525] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:28:11.451403] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-01. [2025-10-20T19:28:11.471823] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T19:28:12.251402] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-03. [2025-10-20T19:28:12.438168] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T19:28:13.051460] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-02. [2025-10-20T19:28:13.689601] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T19:28:13.851428] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-03. [2025-10-20T19:28:13.919971] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:28:14.400329] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:28:14.651388] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-04. [2025-10-20T19:28:15.254839] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T19:28:15.451379] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-05. [2025-10-20T19:28:15.936382] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T19:28:16.251431] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-02. [2025-10-20T19:28:16.839976] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T19:28:17.051371] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-06. [2025-10-20T19:28:17.583352] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T19:28:17.851377] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-07. [2025-10-20T19:28:18.377144] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T19:28:18.651387] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-08. [2025-10-20T19:28:19.290638] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T19:28:19.451382] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-09. [2025-10-20T19:28:20.027559] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:28:20.251414] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-04. [2025-10-20T19:28:21.051430] [INFO ] [REMOTE ] JHI: Fetching primary data for 2025-10. [2025-10-20T19:28:21.789098] [INFO ] [REMOTE ] JHI: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:28:21.790848] [INFO ] [REMOTE ] JHI: Full fetch complete (5775 records). [2025-10-20T19:28:21.790945] [DEBUG] [VALIDATOR ] JHI: Validating primary data. [2025-10-20T19:28:21.856526] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-03. [2025-10-20T19:28:22.107559] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:28:22.197820] [DEBUG] [VALIDATOR ] JHI: All expected trading days are present. [2025-10-20T19:28:22.197918] [DEBUG] [VALIDATOR ] JHI: Primary data validation passed. [2025-10-20T19:28:22.198131] [INFO ] [MAIN ] JHI: Fetched primary data is valid. [2025-10-20T19:28:22.198187] [DEBUG] [LOCAL ] JHI: Saving primary data. [2025-10-20T19:28:22.198551] [DEBUG] [LOCAL ] JHI: Trimmed 5775 records to 4157 (keeping 200 days of data) [2025-10-20T19:28:22.223409] [DEBUG] [LOCAL ] JHI: Primary data saved (4157 records). [2025-10-20T19:28:22.223503] [INFO ] [MAIN ] JHI: Full history saved. [2025-10-20T19:28:22.223576] [INFO ] [MAIN ] Update Progress: 1151/2367 symbols completed. JHI took 13.8s. Estimated remaining: 2h 36m 40s. Rate: 7.8 symbols/min [2025-10-20T19:28:22.388062] [INFO ] [MAIN ] JILL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:22.400598] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-03 (0.54 seconds). [2025-10-20T19:28:22.407491] [DEBUG] [LOCAL ] JILL: Loaded 16049 existing records from primary storage. [2025-10-20T19:28:22.407682] [INFO ] [MAIN ] JILL: Loaded 16049 existing records. [2025-10-20T19:28:22.579691] [WARN ] [MAIN ] JILL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:22.579912] [INFO ] [MAIN ] JILL: Wiping existing primary data before full refresh. [2025-10-20T19:28:22.579958] [INFO ] [LOCAL ] JILL: Starting deletion of primary data. [2025-10-20T19:28:22.580207] [DEBUG] [LOCAL ] JILL: Deleted primary data file. [2025-10-20T19:28:22.580262] [INFO ] [MAIN ] JILL: Starting full primary refresh. [2025-10-20T19:28:22.656540] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-04. [2025-10-20T19:28:22.943493] [INFO ] [REMOTE ] JILL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:23.160392] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T19:28:23.451411] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-01. [2025-10-20T19:28:24.251389] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-05. [2025-10-20T19:28:24.335129] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T19:28:24.706610] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T19:28:25.051379] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-02. [2025-10-20T19:28:25.837608] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T19:28:25.851337] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-03. [2025-10-20T19:28:26.651349] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-06. [2025-10-20T19:28:26.860892] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:28:27.089334] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:28:27.451378] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-04. [2025-10-20T19:28:28.251746] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-05. [2025-10-20T19:28:28.252730] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:28:29.051409] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-03. [2025-10-20T19:28:29.851358] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-05. [2025-10-20T19:28:30.044724] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:28:30.103291] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:28:30.651347] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-07. [2025-10-20T19:28:30.792483] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T19:28:31.225272] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:28:31.451389] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-08. [2025-10-20T19:28:31.981334] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:28:32.251376] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-04. [2025-10-20T19:28:33.051360] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-06. [2025-10-20T19:28:33.851413] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-06. [2025-10-20T19:28:33.955699] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:28:34.496709] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T19:28:34.630625] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:28:34.651343] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-07. [2025-10-20T19:28:35.451472] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-07. [2025-10-20T19:28:36.215492] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T19:28:36.251363] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-08. [2025-10-20T19:28:36.406899] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T19:28:37.051396] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-05. [2025-10-20T19:28:37.813331] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:28:37.851340] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-09. [2025-10-20T19:28:38.651374] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-08. [2025-10-20T19:28:38.795900] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T19:28:39.370844] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:28:39.451343] [INFO ] [REMOTE ] JHG: Fetching primary data for 2025-10. [2025-10-20T19:28:39.666432] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:28:40.251372] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-06. [2025-10-20T19:28:41.051355] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-09. [2025-10-20T19:28:41.117514] [INFO ] [REMOTE ] JHG: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:28:41.124622] [INFO ] [REMOTE ] JHG: Full fetch complete (68630 records). [2025-10-20T19:28:41.124772] [DEBUG] [VALIDATOR ] JHG: Validating primary data. [2025-10-20T19:28:41.523387] [DEBUG] [VALIDATOR ] JHG: All expected trading days are present. [2025-10-20T19:28:41.523491] [DEBUG] [VALIDATOR ] JHG: Primary data validation passed. [2025-10-20T19:28:41.524527] [INFO ] [MAIN ] JHG: Fetched primary data is valid. [2025-10-20T19:28:41.524599] [DEBUG] [LOCAL ] JHG: Saving primary data. [2025-10-20T19:28:41.526108] [DEBUG] [LOCAL ] JHG: Trimmed 68630 records to 47569 (keeping 200 days of data) [2025-10-20T19:28:41.644322] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:28:41.827517] [DEBUG] [LOCAL ] JHG: Primary data saved (47569 records). [2025-10-20T19:28:41.827712] [INFO ] [MAIN ] JHG: Full history saved. [2025-10-20T19:28:41.828967] [INFO ] [MAIN ] Update Progress: 1152/2367 symbols completed. JHG took 36.3s. Estimated remaining: 2h 36m 45s. Rate: 7.8 symbols/min [2025-10-20T19:28:41.861175] [INFO ] [REMOTE ] JHS: Fetching primary data for 2025-10. [2025-10-20T19:28:42.004278] [INFO ] [MAIN ] JKS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:42.033483] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T19:28:42.079924] [DEBUG] [LOCAL ] JKS: Loaded 40154 existing records from primary storage. [2025-10-20T19:28:42.080223] [INFO ] [MAIN ] JKS: Loaded 40154 existing records. [2025-10-20T19:28:42.257602] [WARN ] [MAIN ] JKS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:42.257841] [INFO ] [MAIN ] JKS: Wiping existing primary data before full refresh. [2025-10-20T19:28:42.257904] [INFO ] [LOCAL ] JKS: Starting deletion of primary data. [2025-10-20T19:28:42.258264] [DEBUG] [LOCAL ] JKS: Deleted primary data file. [2025-10-20T19:28:42.258303] [INFO ] [MAIN ] JKS: Starting full primary refresh. [2025-10-20T19:28:42.613164] [INFO ] [REMOTE ] JKS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:42.651339] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-01. [2025-10-20T19:28:42.681576] [INFO ] [REMOTE ] JHS: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T19:28:42.682460] [INFO ] [REMOTE ] JHS: Full fetch complete (4446 records). [2025-10-20T19:28:42.682525] [DEBUG] [VALIDATOR ] JHS: Validating primary data. [2025-10-20T19:28:43.041780] [DEBUG] [VALIDATOR ] JHS: All expected trading days are present. [2025-10-20T19:28:43.041853] [DEBUG] [VALIDATOR ] JHS: Primary data validation passed. [2025-10-20T19:28:43.041986] [INFO ] [MAIN ] JHS: Fetched primary data is valid. [2025-10-20T19:28:43.042023] [DEBUG] [LOCAL ] JHS: Saving primary data. [2025-10-20T19:28:43.042357] [DEBUG] [LOCAL ] JHS: Trimmed 4446 records to 2931 (keeping 200 days of data) [2025-10-20T19:28:43.059852] [DEBUG] [LOCAL ] JHS: Primary data saved (2931 records). [2025-10-20T19:28:43.059919] [INFO ] [MAIN ] JHS: Full history saved. [2025-10-20T19:28:43.059992] [INFO ] [MAIN ] Update Progress: 1153/2367 symbols completed. JHS took 33.9s. Estimated remaining: 2h 36m 31s. Rate: 7.8 symbols/min [2025-10-20T19:28:43.230180] [INFO ] [MAIN ] JLL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:43.261466] [DEBUG] [LOCAL ] JLL: Loaded 32147 existing records from primary storage. [2025-10-20T19:28:43.261745] [INFO ] [MAIN ] JLL: Loaded 32147 existing records. [2025-10-20T19:28:43.447934] [WARN ] [MAIN ] JLL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:43.448138] [INFO ] [MAIN ] JLL: Wiping existing primary data before full refresh. [2025-10-20T19:28:43.448185] [INFO ] [LOCAL ] JLL: Starting deletion of primary data. [2025-10-20T19:28:43.448534] [DEBUG] [LOCAL ] JLL: Deleted primary data file. [2025-10-20T19:28:43.448573] [INFO ] [MAIN ] JLL: Starting full primary refresh. [2025-10-20T19:28:43.460307] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-09. [2025-10-20T19:28:43.801096] [INFO ] [REMOTE ] JLL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:44.256845] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-01. [2025-10-20T19:28:44.270329] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:28:44.520234] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:28:45.051373] [INFO ] [REMOTE ] JILL: Fetching primary data for 2025-10. [2025-10-20T19:28:45.565019] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T19:28:45.851384] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-02. [2025-10-20T19:28:46.013544] [INFO ] [REMOTE ] JILL: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:28:46.015996] [INFO ] [REMOTE ] JILL: Full fetch complete (22419 records). [2025-10-20T19:28:46.016096] [DEBUG] [VALIDATOR ] JILL: Validating primary data. [2025-10-20T19:28:46.393579] [DEBUG] [VALIDATOR ] JILL: All expected trading days are present. [2025-10-20T19:28:46.393691] [DEBUG] [VALIDATOR ] JILL: Primary data validation passed. [2025-10-20T19:28:46.394241] [INFO ] [MAIN ] JILL: Fetched primary data is valid. [2025-10-20T19:28:46.394283] [DEBUG] [LOCAL ] JILL: Saving primary data. [2025-10-20T19:28:46.394981] [DEBUG] [LOCAL ] JILL: Trimmed 22419 records to 15604 (keeping 200 days of data) [2025-10-20T19:28:46.490054] [DEBUG] [LOCAL ] JILL: Primary data saved (15604 records). [2025-10-20T19:28:46.490187] [INFO ] [MAIN ] JILL: Full history saved. [2025-10-20T19:28:46.490267] [INFO ] [MAIN ] Update Progress: 1154/2367 symbols completed. JILL took 24.3s. Estimated remaining: 2h 36m 18s. Rate: 7.8 symbols/min [2025-10-20T19:28:46.661960] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-02. [2025-10-20T19:28:46.664483] [INFO ] [MAIN ] JLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:28:46.670886] [DEBUG] [LOCAL ] JLS: Loaded 4259 existing records from primary storage. [2025-10-20T19:28:46.671047] [INFO ] [MAIN ] JLS: Loaded 4259 existing records. [2025-10-20T19:28:46.846454] [WARN ] [MAIN ] JLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:28:46.846760] [INFO ] [MAIN ] JLS: Wiping existing primary data before full refresh. [2025-10-20T19:28:46.846811] [INFO ] [LOCAL ] JLS: Starting deletion of primary data. [2025-10-20T19:28:46.846987] [DEBUG] [LOCAL ] JLS: Deleted primary data file. [2025-10-20T19:28:46.847044] [INFO ] [MAIN ] JLS: Starting full primary refresh. [2025-10-20T19:28:47.204847] [INFO ] [REMOTE ] JLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:28:47.237960] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T19:28:47.451374] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-03. [2025-10-20T19:28:48.221031] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:28:48.251342] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-03. [2025-10-20T19:28:48.468946] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:28:49.051377] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-04. [2025-10-20T19:28:49.851402] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-07. [2025-10-20T19:28:49.916953] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:28:50.201888] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:28:50.651380] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-04. [2025-10-20T19:28:51.451385] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-05. [2025-10-20T19:28:51.566479] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T19:28:52.251576] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-01. [2025-10-20T19:28:52.352746] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:28:52.782253] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T19:28:52.815004] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:28:53.051362] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-08. [2025-10-20T19:28:53.851398] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-06. [2025-10-20T19:28:54.651437] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-05. [2025-10-20T19:28:54.972468] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T19:28:55.168006] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:28:55.451400] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-09. [2025-10-20T19:28:56.130160] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T19:28:56.251386] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-07. [2025-10-20T19:28:57.051363] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-06. [2025-10-20T19:28:57.295816] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T19:28:57.567085] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T19:28:57.851396] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-02. [2025-10-20T19:28:58.155991] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T19:28:58.480408] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T19:28:58.651358] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-07. [2025-10-20T19:28:59.451376] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-08. [2025-10-20T19:29:00.216709] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:29:00.251376] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-08. [2025-10-20T19:29:00.947523] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T19:29:01.051415] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-03. [2025-10-20T19:29:01.614402] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:29:01.701543] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T19:29:01.851387] [INFO ] [REMOTE ] JHX: Fetching primary data for 2025-10. [2025-10-20T19:29:02.651383] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-09. [2025-10-20T19:29:03.451348] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-09. [2025-10-20T19:29:03.861337] [INFO ] [REMOTE ] JHX: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:29:03.869911] [INFO ] [REMOTE ] JHX: Full fetch complete (61997 records). [2025-10-20T19:29:03.870141] [DEBUG] [VALIDATOR ] JHX: Validating primary data. [2025-10-20T19:29:04.259894] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-04. [2025-10-20T19:29:04.282963] [DEBUG] [VALIDATOR ] JHX: All expected trading days are present. [2025-10-20T19:29:04.283116] [DEBUG] [VALIDATOR ] JHX: Primary data validation passed. [2025-10-20T19:29:04.284993] [INFO ] [MAIN ] JHX: Fetched primary data is valid. [2025-10-20T19:29:04.285366] [DEBUG] [LOCAL ] JHX: Saving primary data. [2025-10-20T19:29:04.287371] [DEBUG] [LOCAL ] JHX: Trimmed 61997 records to 53195 (keeping 200 days of data) [2025-10-20T19:29:04.538001] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:29:04.678220] [DEBUG] [LOCAL ] JHX: Primary data saved (53195 records). [2025-10-20T19:29:04.678380] [INFO ] [MAIN ] JHX: Full history saved. [2025-10-20T19:29:04.678464] [INFO ] [MAIN ] Update Progress: 1155/2367 symbols completed. JHX took 55.4s. Estimated remaining: 2h 36m 21s. Rate: 7.8 symbols/min [2025-10-20T19:29:04.850758] [INFO ] [MAIN ] JMIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:04.869288] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:29:04.924480] [DEBUG] [LOCAL ] JMIA: Loaded 51831 existing records from primary storage. [2025-10-20T19:29:04.924766] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T19:29:04.926695] [INFO ] [MAIN ] JMIA: Loaded 51831 existing records. [2025-10-20T19:29:05.063865] [INFO ] [REMOTE ] JLL: Fetching primary data for 2025-10. [2025-10-20T19:29:05.104435] [WARN ] [MAIN ] JMIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:05.104865] [INFO ] [MAIN ] JMIA: Wiping existing primary data before full refresh. [2025-10-20T19:29:05.104937] [INFO ] [LOCAL ] JMIA: Starting deletion of primary data. [2025-10-20T19:29:05.105187] [DEBUG] [LOCAL ] JMIA: Deleted primary data file. [2025-10-20T19:29:05.105227] [INFO ] [MAIN ] JMIA: Starting full primary refresh. [2025-10-20T19:29:05.453231] [INFO ] [REMOTE ] JMIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:05.851391] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-01. [2025-10-20T19:29:06.541004] [INFO ] [REMOTE ] JLL: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:29:06.545173] [INFO ] [REMOTE ] JLL: Full fetch complete (45164 records). [2025-10-20T19:29:06.545263] [DEBUG] [VALIDATOR ] JLL: Validating primary data. [2025-10-20T19:29:06.656572] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-05. [2025-10-20T19:29:06.917736] [DEBUG] [VALIDATOR ] JLL: All expected trading days are present. [2025-10-20T19:29:06.917814] [DEBUG] [VALIDATOR ] JLL: Primary data validation passed. [2025-10-20T19:29:06.918520] [INFO ] [MAIN ] JLL: Fetched primary data is valid. [2025-10-20T19:29:06.918559] [DEBUG] [LOCAL ] JLL: Saving primary data. [2025-10-20T19:29:06.919466] [DEBUG] [LOCAL ] JLL: Trimmed 45164 records to 31753 (keeping 200 days of data) [2025-10-20T19:29:07.106732] [DEBUG] [LOCAL ] JLL: Primary data saved (31753 records). [2025-10-20T19:29:07.106851] [INFO ] [MAIN ] JLL: Full history saved. [2025-10-20T19:29:07.106922] [INFO ] [MAIN ] Update Progress: 1156/2367 symbols completed. JLL took 24.0s. Estimated remaining: 2h 36m 8s. Rate: 7.8 symbols/min [2025-10-20T19:29:07.309068] [INFO ] [MAIN ] JMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:07.313321] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T19:29:07.313461] [DEBUG] [LOCAL ] JMM: No existing data file found in primary storage. [2025-10-20T19:29:07.315281] [WARN ] [MAIN ] JMM: No existing data found, performing full refresh instead. [2025-10-20T19:29:07.315423] [INFO ] [MAIN ] JMM: Wiping existing primary data before full refresh. [2025-10-20T19:29:07.315490] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-10-20T19:29:07.315544] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-10-20T19:29:07.315577] [INFO ] [MAIN ] JMM: Starting full primary refresh. [2025-10-20T19:29:07.456500] [INFO ] [REMOTE ] JKS: Fetching primary data for 2025-10. [2025-10-20T19:29:07.710965] [INFO ] [REMOTE ] JMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:07.742825] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T19:29:08.251456] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-02. [2025-10-20T19:29:09.051435] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-01. [2025-10-20T19:29:09.137600] [INFO ] [REMOTE ] JKS: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:29:09.143758] [INFO ] [REMOTE ] JKS: Full fetch complete (58805 records). [2025-10-20T19:29:09.143896] [DEBUG] [VALIDATOR ] JKS: Validating primary data. [2025-10-20T19:29:09.437267] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T19:29:09.540692] [DEBUG] [VALIDATOR ] JKS: All expected trading days are present. [2025-10-20T19:29:09.540789] [DEBUG] [VALIDATOR ] JKS: Primary data validation passed. [2025-10-20T19:29:09.541619] [INFO ] [MAIN ] JKS: Fetched primary data is valid. [2025-10-20T19:29:09.541690] [DEBUG] [LOCAL ] JKS: Saving primary data. [2025-10-20T19:29:09.542910] [DEBUG] [LOCAL ] JKS: Trimmed 58805 records to 39426 (keeping 200 days of data) [2025-10-20T19:29:09.775488] [DEBUG] [LOCAL ] JKS: Primary data saved (39426 records). [2025-10-20T19:29:09.775765] [INFO ] [MAIN ] JKS: Full history saved. [2025-10-20T19:29:09.775865] [INFO ] [MAIN ] Update Progress: 1157/2367 symbols completed. JKS took 27.9s. Estimated remaining: 2h 35m 55s. Rate: 7.8 symbols/min [2025-10-20T19:29:09.851344] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-06. [2025-10-20T19:29:09.961582] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T19:29:10.188974] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-06 (0.34 seconds). [2025-10-20T19:29:10.651388] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-07. [2025-10-20T19:29:11.217163] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:29:11.451425] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-03. [2025-10-20T19:29:12.251402] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-08. [2025-10-20T19:29:12.842351] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T19:29:13.051360] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-02. [2025-10-20T19:29:13.317777] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T19:29:13.387995] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-10-20T19:29:13.851370] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-03. [2025-10-20T19:29:14.198188] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T19:29:14.651385] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-04. [2025-10-20T19:29:15.451419] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-09. [2025-10-20T19:29:16.040159] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:29:16.251379] [INFO ] [REMOTE ] JLS: Fetching primary data for 2025-10. [2025-10-20T19:29:16.283504] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:29:16.930280] [INFO ] [REMOTE ] JLS: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T19:29:16.931838] [INFO ] [REMOTE ] JLS: Full fetch complete (5810 records). [2025-10-20T19:29:16.931928] [DEBUG] [VALIDATOR ] JLS: Validating primary data. [2025-10-20T19:29:17.056763] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-05. [2025-10-20T19:29:17.313888] [DEBUG] [VALIDATOR ] JLS: All expected trading days are present. [2025-10-20T19:29:17.313965] [DEBUG] [VALIDATOR ] JLS: Primary data validation passed. [2025-10-20T19:29:17.314140] [INFO ] [MAIN ] JLS: Fetched primary data is valid. [2025-10-20T19:29:17.314179] [DEBUG] [LOCAL ] JLS: Saving primary data. [2025-10-20T19:29:17.314540] [DEBUG] [LOCAL ] JLS: Trimmed 5810 records to 4066 (keeping 200 days of data) [2025-10-20T19:29:17.338767] [DEBUG] [LOCAL ] JLS: Primary data saved (4066 records). [2025-10-20T19:29:17.338830] [INFO ] [MAIN ] JLS: Full history saved. [2025-10-20T19:29:17.338894] [INFO ] [MAIN ] Update Progress: 1158/2367 symbols completed. JLS took 30.8s. Estimated remaining: 2h 35m 47s. Rate: 7.8 symbols/min [2025-10-20T19:29:17.851447] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-04. [2025-10-20T19:29:18.289981] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T19:29:18.651347] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-05. [2025-10-20T19:29:18.923738] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:29:19.118495] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-05 (0.47 seconds). [2025-10-20T19:29:19.451421] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-06. [2025-10-20T19:29:19.879540] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T19:29:20.251376] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-07. [2025-10-20T19:29:20.717371] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T19:29:21.051365] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-08. [2025-10-20T19:29:21.497848] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:29:21.851422] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-06. [2025-10-20T19:29:22.651440] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-09. [2025-10-20T19:29:23.085051] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T19:29:23.451367] [INFO ] [REMOTE ] JMM: Fetching primary data for 2025-10. [2025-10-20T19:29:23.499759] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T19:29:24.004099] [INFO ] [REMOTE ] JMM: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T19:29:24.004849] [INFO ] [REMOTE ] JMM: Full fetch complete (2298 records). [2025-10-20T19:29:24.004918] [DEBUG] [VALIDATOR ] JMM: Validating primary data. [2025-10-20T19:29:24.184025] [WARN ] [VALIDATOR ] JMM: Validation failed - insufficient historical data. Have 194 unique days, need 200 days [2025-10-20T19:29:24.184207] [FATAL] [MAIN ] JMM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:29:24.184286] [WARN ] [MAIN ] JMM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:29:24.184333] [INFO ] [LOCAL ] JMM: Starting deletion of primary data. [2025-10-20T19:29:24.184386] [DEBUG] [LOCAL ] JMM: No primary data file found to delete. [2025-10-20T19:29:24.184421] [INFO ] [LOCAL ] JMM: Starting deletion of derivative data. [2025-10-20T19:29:24.184458] [DEBUG] [LOCAL ] JMM: No derivative data file found to delete. [2025-10-20T19:29:24.184512] [INFO ] [MAIN ] Update Progress: 1159/2367 symbols completed. JMM took 17.1s. Estimated remaining: 2h 35m 39s. Rate: 7.8 symbols/min [2025-10-20T19:29:24.251394] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-07. [2025-10-20T19:29:26.231737] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T19:29:26.242494] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-08. [2025-10-20T19:29:28.213040] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T19:29:28.224339] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-09. [2025-10-20T19:29:30.215809] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T19:29:30.226923] [INFO ] [REMOTE ] JMIA: Fetching primary data for 2025-10. [2025-10-20T19:29:32.717620] [INFO ] [REMOTE ] JMIA: Incremental fetch complete for 2025-10 (2.49 seconds). [2025-10-20T19:29:32.725226] [INFO ] [REMOTE ] JMIA: Full fetch complete (73466 records). [2025-10-20T19:29:32.725374] [DEBUG] [VALIDATOR ] JMIA: Validating primary data. [2025-10-20T19:29:33.135134] [DEBUG] [VALIDATOR ] JMIA: All expected trading days are present. [2025-10-20T19:29:33.135262] [DEBUG] [VALIDATOR ] JMIA: Primary data validation passed. [2025-10-20T19:29:33.136280] [INFO ] [MAIN ] JMIA: Fetched primary data is valid. [2025-10-20T19:29:33.136343] [DEBUG] [LOCAL ] JMIA: Saving primary data. [2025-10-20T19:29:33.137756] [DEBUG] [LOCAL ] JMIA: Trimmed 73466 records to 51222 (keeping 200 days of data) [2025-10-20T19:29:33.422096] [DEBUG] [LOCAL ] JMIA: Primary data saved (51222 records). [2025-10-20T19:29:33.422251] [INFO ] [MAIN ] JMIA: Full history saved. [2025-10-20T19:29:33.422332] [INFO ] [MAIN ] Update Progress: 1160/2367 symbols completed. JMIA took 28.7s. Estimated remaining: 2h 35m 32s. Rate: 7.8 symbols/min [2025-10-20T19:29:33.997242] [INFO ] [MAIN ] JNJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.093414] [INFO ] [MAIN ] JOE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.136853] [INFO ] [MAIN ] JOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.156569] [INFO ] [MAIN ] JOBY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:34.170185] [DEBUG] [LOCAL ] JOF: Loaded 6969 existing records from primary storage. [2025-10-20T19:29:34.170576] [INFO ] [MAIN ] JOF: Loaded 6969 existing records. [2025-10-20T19:29:34.222275] [DEBUG] [LOCAL ] JOE: Loaded 24168 existing records from primary storage. [2025-10-20T19:29:34.246420] [INFO ] [MAIN ] JOE: Loaded 24168 existing records. [2025-10-20T19:29:34.561513] [WARN ] [MAIN ] JOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:34.603221] [DEBUG] [LOCAL ] JNJ: Loaded 54533 existing records from primary storage. [2025-10-20T19:29:34.601796] [WARN ] [MAIN ] JOE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:34.604397] [INFO ] [MAIN ] JOE: Wiping existing primary data before full refresh. [2025-10-20T19:29:34.603390] [INFO ] [MAIN ] JOF: Wiping existing primary data before full refresh. [2025-10-20T19:29:34.604748] [INFO ] [LOCAL ] JOE: Starting deletion of primary data. [2025-10-20T19:29:34.605078] [INFO ] [MAIN ] JNJ: Loaded 54533 existing records. [2025-10-20T19:29:34.604848] [INFO ] [LOCAL ] JOF: Starting deletion of primary data. [2025-10-20T19:29:34.632329] [DEBUG] [LOCAL ] JOE: Deleted primary data file. [2025-10-20T19:29:34.653665] [INFO ] [MAIN ] JOE: Starting full primary refresh. [2025-10-20T19:29:34.659558] [DEBUG] [LOCAL ] JOF: Deleted primary data file. [2025-10-20T19:29:34.659488] [DEBUG] [LOCAL ] JOBY: Loaded 54588 existing records from primary storage. [2025-10-20T19:29:34.690439] [INFO ] [MAIN ] JOBY: Loaded 54588 existing records. [2025-10-20T19:29:34.681205] [INFO ] [MAIN ] JOF: Starting full primary refresh. [2025-10-20T19:29:35.298156] [WARN ] [MAIN ] JNJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:35.395373] [INFO ] [MAIN ] JNJ: Wiping existing primary data before full refresh. [2025-10-20T19:29:35.391005] [WARN ] [MAIN ] JOBY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:35.411661] [INFO ] [MAIN ] JOBY: Wiping existing primary data before full refresh. [2025-10-20T19:29:35.411405] [INFO ] [LOCAL ] JNJ: Starting deletion of primary data. [2025-10-20T19:29:35.446451] [INFO ] [LOCAL ] JOBY: Starting deletion of primary data. [2025-10-20T19:29:35.460798] [DEBUG] [LOCAL ] JOBY: Deleted primary data file. [2025-10-20T19:29:35.497902] [INFO ] [MAIN ] JOBY: Starting full primary refresh. [2025-10-20T19:29:35.516948] [DEBUG] [LOCAL ] JNJ: Deleted primary data file. [2025-10-20T19:29:35.531599] [INFO ] [MAIN ] JNJ: Starting full primary refresh. [2025-10-20T19:29:36.015235] [INFO ] [REMOTE ] JOE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.122199] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-01. [2025-10-20T19:29:36.205940] [INFO ] [REMOTE ] JOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.241891] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-01. [2025-10-20T19:29:36.524669] [INFO ] [REMOTE ] JNJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.543214] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-01. [2025-10-20T19:29:36.542981] [INFO ] [REMOTE ] JOBY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:36.544312] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-01. [2025-10-20T19:29:36.913892] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T19:29:36.915431] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-02. [2025-10-20T19:29:37.374364] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T19:29:37.380274] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-02. [2025-10-20T19:29:37.522184] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T19:29:37.523744] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-03. [2025-10-20T19:29:38.105202] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T19:29:38.107467] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-04. [2025-10-20T19:29:38.457694] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T19:29:38.462439] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-03. [2025-10-20T19:29:38.760548] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-01 (2.22 seconds). [2025-10-20T19:29:38.771006] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-02. [2025-10-20T19:29:38.868851] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T19:29:38.871773] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-05. [2025-10-20T19:29:39.071330] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-01 (2.53 seconds). [2025-10-20T19:29:39.081775] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-02. [2025-10-20T19:29:39.252993] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-10-20T19:29:39.254776] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-06. [2025-10-20T19:29:39.338064] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:29:39.344024] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-04. [2025-10-20T19:29:39.835614] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:29:39.837560] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-07. [2025-10-20T19:29:40.286139] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T19:29:40.293222] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-05. [2025-10-20T19:29:40.523354] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T19:29:40.526504] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-08. [2025-10-20T19:29:40.610694] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T19:29:41.051414] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-03. [2025-10-20T19:29:41.249389] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T19:29:41.302882] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T19:29:41.414273] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T19:29:41.851395] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-06. [2025-10-20T19:29:42.651405] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-03. [2025-10-20T19:29:42.991069] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:29:43.078032] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T19:29:43.451441] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-07. [2025-10-20T19:29:44.251402] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-09. [2025-10-20T19:29:44.613050] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T19:29:44.956465] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:29:45.051342] [INFO ] [REMOTE ] JOF: Fetching primary data for 2025-10. [2025-10-20T19:29:45.104553] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-10-20T19:29:45.764212] [INFO ] [REMOTE ] JOF: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T19:29:45.765733] [INFO ] [REMOTE ] JOF: Full fetch complete (9416 records). [2025-10-20T19:29:45.765800] [DEBUG] [VALIDATOR ] JOF: Validating primary data. [2025-10-20T19:29:45.856604] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-08. [2025-10-20T19:29:46.140493] [DEBUG] [VALIDATOR ] JOF: All expected trading days are present. [2025-10-20T19:29:46.140575] [DEBUG] [VALIDATOR ] JOF: Primary data validation passed. [2025-10-20T19:29:46.140886] [INFO ] [MAIN ] JOF: Fetched primary data is valid. [2025-10-20T19:29:46.140949] [DEBUG] [LOCAL ] JOF: Saving primary data. [2025-10-20T19:29:46.141400] [DEBUG] [LOCAL ] JOF: Trimmed 9416 records to 6925 (keeping 200 days of data) [2025-10-20T19:29:46.183597] [DEBUG] [LOCAL ] JOF: Primary data saved (6925 records). [2025-10-20T19:29:46.183765] [INFO ] [MAIN ] JOF: Full history saved. [2025-10-20T19:29:46.183852] [INFO ] [MAIN ] Update Progress: 1161/2367 symbols completed. JOF took 12.7s. Estimated remaining: 2h 35m 30s. Rate: 7.8 symbols/min [2025-10-20T19:29:46.367086] [INFO ] [MAIN ] JPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:46.401677] [DEBUG] [LOCAL ] JPC: Loaded 37747 existing records from primary storage. [2025-10-20T19:29:46.401956] [INFO ] [MAIN ] JPC: Loaded 37747 existing records. [2025-10-20T19:29:46.576220] [WARN ] [MAIN ] JPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:46.576443] [INFO ] [MAIN ] JPC: Wiping existing primary data before full refresh. [2025-10-20T19:29:46.576507] [INFO ] [LOCAL ] JPC: Starting deletion of primary data. [2025-10-20T19:29:46.576788] [DEBUG] [LOCAL ] JPC: Deleted primary data file. [2025-10-20T19:29:46.576863] [INFO ] [MAIN ] JPC: Starting full primary refresh. [2025-10-20T19:29:46.656463] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-04. [2025-10-20T19:29:46.965555] [INFO ] [REMOTE ] JPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:46.978342] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T19:29:47.451422] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-09. [2025-10-20T19:29:48.251357] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-01. [2025-10-20T19:29:48.472831] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:29:49.052682] [INFO ] [REMOTE ] JOE: Fetching primary data for 2025-10. [2025-10-20T19:29:49.058565] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-04 (2.40 seconds). [2025-10-20T19:29:49.572306] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:29:49.851355] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-05. [2025-10-20T19:29:50.169158] [INFO ] [REMOTE ] JOE: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T19:29:50.174467] [INFO ] [REMOTE ] JOE: Full fetch complete (35625 records). [2025-10-20T19:29:50.174580] [DEBUG] [VALIDATOR ] JOE: Validating primary data. [2025-10-20T19:29:50.547675] [DEBUG] [VALIDATOR ] JOE: All expected trading days are present. [2025-10-20T19:29:50.547778] [DEBUG] [VALIDATOR ] JOE: Primary data validation passed. [2025-10-20T19:29:50.548450] [INFO ] [MAIN ] JOE: Fetched primary data is valid. [2025-10-20T19:29:50.548499] [DEBUG] [LOCAL ] JOE: Saving primary data. [2025-10-20T19:29:50.549514] [DEBUG] [LOCAL ] JOE: Trimmed 35625 records to 23890 (keeping 200 days of data) [2025-10-20T19:29:50.656016] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-02. [2025-10-20T19:29:50.690893] [DEBUG] [LOCAL ] JOE: Primary data saved (23890 records). [2025-10-20T19:29:50.691006] [INFO ] [MAIN ] JOE: Full history saved. [2025-10-20T19:29:50.691078] [INFO ] [MAIN ] Update Progress: 1162/2367 symbols completed. JOE took 17.3s. Estimated remaining: 2h 35m 19s. Rate: 7.8 symbols/min [2025-10-20T19:29:50.868114] [INFO ] [MAIN ] JPM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:29:50.920491] [DEBUG] [LOCAL ] JPM: Loaded 54574 existing records from primary storage. [2025-10-20T19:29:50.920873] [INFO ] [MAIN ] JPM: Loaded 54574 existing records. [2025-10-20T19:29:51.108318] [WARN ] [MAIN ] JPM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:29:51.108545] [INFO ] [MAIN ] JPM: Wiping existing primary data before full refresh. [2025-10-20T19:29:51.108602] [INFO ] [LOCAL ] JPM: Starting deletion of primary data. [2025-10-20T19:29:51.108921] [DEBUG] [LOCAL ] JPM: Deleted primary data file. [2025-10-20T19:29:51.108978] [INFO ] [MAIN ] JPM: Starting full primary refresh. [2025-10-20T19:29:51.452139] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-04. [2025-10-20T19:29:51.470977] [INFO ] [REMOTE ] JPM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:29:52.030512] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:29:52.239965] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-05 (2.39 seconds). [2025-10-20T19:29:52.251335] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-06. [2025-10-20T19:29:53.051422] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-01. [2025-10-20T19:29:53.774418] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-10-20T19:29:53.851343] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-05. [2025-10-20T19:29:54.314131] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T19:29:54.651369] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-07. [2025-10-20T19:29:55.331980] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-01 (2.28 seconds). [2025-10-20T19:29:55.451351] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-02. [2025-10-20T19:29:56.022267] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-05 (2.17 seconds). [2025-10-20T19:29:56.251380] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-03. [2025-10-20T19:29:56.363600] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:29:57.051372] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-06. [2025-10-20T19:29:57.751019] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:29:57.802065] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-10-20T19:29:57.851375] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-04. [2025-10-20T19:29:58.651432] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-08. [2025-10-20T19:29:59.189285] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T19:29:59.451404] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-05. [2025-10-20T19:29:59.507339] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-06 (2.46 seconds). [2025-10-20T19:30:00.251358] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-07. [2025-10-20T19:30:00.729861] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T19:30:00.897418] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:30:01.051369] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-06. [2025-10-20T19:30:01.851356] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-09. [2025-10-20T19:30:02.352975] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T19:30:02.651379] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-03. [2025-10-20T19:30:03.211989] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-07 (2.96 seconds). [2025-10-20T19:30:03.451447] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-08. [2025-10-20T19:30:04.252408] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-07. [2025-10-20T19:30:04.262255] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-09 (2.41 seconds). [2025-10-20T19:30:05.055055] [INFO ] [REMOTE ] JNJ: Fetching primary data for 2025-10. [2025-10-20T19:30:05.060629] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-03 (2.41 seconds). [2025-10-20T19:30:05.851362] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-04. [2025-10-20T19:30:05.965296] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T19:30:06.288363] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-08 (2.84 seconds). [2025-10-20T19:30:06.651384] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-08. [2025-10-20T19:30:07.451459] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-09. [2025-10-20T19:30:07.508046] [INFO ] [REMOTE ] JNJ: Incremental fetch complete for 2025-10 (2.45 seconds). [2025-10-20T19:30:07.516256] [INFO ] [REMOTE ] JNJ: Full fetch complete (77933 records). [2025-10-20T19:30:07.516426] [DEBUG] [VALIDATOR ] JNJ: Validating primary data. [2025-10-20T19:30:07.931531] [DEBUG] [VALIDATOR ] JNJ: All expected trading days are present. [2025-10-20T19:30:07.931622] [DEBUG] [VALIDATOR ] JNJ: Primary data validation passed. [2025-10-20T19:30:07.932632] [INFO ] [MAIN ] JNJ: Fetched primary data is valid. [2025-10-20T19:30:07.932726] [DEBUG] [LOCAL ] JNJ: Saving primary data. [2025-10-20T19:30:07.934217] [DEBUG] [LOCAL ] JNJ: Trimmed 77933 records to 53753 (keeping 200 days of data) [2025-10-20T19:30:08.299453] [DEBUG] [LOCAL ] JNJ: Primary data saved (53753 records). [2025-10-20T19:30:08.299589] [INFO ] [MAIN ] JNJ: Full history saved. [2025-10-20T19:30:08.299691] [INFO ] [MAIN ] Update Progress: 1163/2367 symbols completed. JNJ took 34.9s. Estimated remaining: 2h 35m 21s. Rate: 7.7 symbols/min [2025-10-20T19:30:08.490985] [INFO ] [MAIN ] JQC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:08.492867] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:30:08.501769] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-09. [2025-10-20T19:30:08.532410] [DEBUG] [LOCAL ] JQC: Loaded 34272 existing records from primary storage. [2025-10-20T19:30:08.532697] [INFO ] [MAIN ] JQC: Loaded 34272 existing records. [2025-10-20T19:30:08.713925] [WARN ] [MAIN ] JQC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:08.714191] [INFO ] [MAIN ] JQC: Wiping existing primary data before full refresh. [2025-10-20T19:30:08.714287] [INFO ] [LOCAL ] JQC: Starting deletion of primary data. [2025-10-20T19:30:08.714564] [DEBUG] [LOCAL ] JQC: Deleted primary data file. [2025-10-20T19:30:08.714668] [INFO ] [MAIN ] JQC: Starting full primary refresh. [2025-10-20T19:30:08.994229] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-04 (3.14 seconds). [2025-10-20T19:30:09.056448] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-05. [2025-10-20T19:30:09.126975] [INFO ] [REMOTE ] JQC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:09.851372] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-01. [2025-10-20T19:30:09.901494] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:30:09.949455] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-09 (2.50 seconds). [2025-10-20T19:30:10.651377] [INFO ] [REMOTE ] JOBY: Fetching primary data for 2025-10. [2025-10-20T19:30:11.307215] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T19:30:11.451391] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-02. [2025-10-20T19:30:11.582430] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-05 (2.53 seconds). [2025-10-20T19:30:12.251356] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-06. [2025-10-20T19:30:12.818359] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T19:30:13.051435] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-03. [2025-10-20T19:30:13.448398] [INFO ] [REMOTE ] JOBY: Incremental fetch complete for 2025-10 (2.80 seconds). [2025-10-20T19:30:13.456585] [INFO ] [REMOTE ] JOBY: Full fetch complete (77974 records). [2025-10-20T19:30:13.456804] [DEBUG] [VALIDATOR ] JOBY: Validating primary data. [2025-10-20T19:30:13.852186] [DEBUG] [VALIDATOR ] JOBY: All expected trading days are present. [2025-10-20T19:30:13.851959] [INFO ] [REMOTE ] JPC: Fetching primary data for 2025-10. [2025-10-20T19:30:13.852387] [DEBUG] [VALIDATOR ] JOBY: Primary data validation passed. [2025-10-20T19:30:13.853677] [INFO ] [MAIN ] JOBY: Fetched primary data is valid. [2025-10-20T19:30:13.854391] [DEBUG] [LOCAL ] JOBY: Saving primary data. [2025-10-20T19:30:13.855995] [DEBUG] [LOCAL ] JOBY: Trimmed 77974 records to 53809 (keeping 200 days of data) [2025-10-20T19:30:14.191548] [DEBUG] [LOCAL ] JOBY: Primary data saved (53809 records). [2025-10-20T19:30:14.191800] [INFO ] [MAIN ] JOBY: Full history saved. [2025-10-20T19:30:14.191955] [INFO ] [MAIN ] Update Progress: 1164/2367 symbols completed. JOBY took 40.7s. Estimated remaining: 2h 35m 12s. Rate: 7.8 symbols/min [2025-10-20T19:30:14.369746] [INFO ] [MAIN ] JRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:14.407735] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-06 (2.16 seconds). [2025-10-20T19:30:14.429075] [DEBUG] [LOCAL ] JRI: Loaded 15593 existing records from primary storage. [2025-10-20T19:30:14.429518] [INFO ] [MAIN ] JRI: Loaded 15593 existing records. [2025-10-20T19:30:14.597524] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:30:14.652854] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-04. [2025-10-20T19:30:14.652276] [WARN ] [MAIN ] JRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:14.654002] [INFO ] [MAIN ] JRI: Wiping existing primary data before full refresh. [2025-10-20T19:30:14.654071] [INFO ] [LOCAL ] JRI: Starting deletion of primary data. [2025-10-20T19:30:14.654320] [DEBUG] [LOCAL ] JRI: Deleted primary data file. [2025-10-20T19:30:14.654387] [INFO ] [MAIN ] JRI: Starting full primary refresh. [2025-10-20T19:30:15.032728] [INFO ] [REMOTE ] JRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:15.054374] [INFO ] [REMOTE ] JPC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T19:30:15.059947] [INFO ] [REMOTE ] JPC: Full fetch complete (55065 records). [2025-10-20T19:30:15.060034] [DEBUG] [VALIDATOR ] JPC: Validating primary data. [2025-10-20T19:30:15.452111] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-01. [2025-10-20T19:30:15.470082] [DEBUG] [VALIDATOR ] JPC: All expected trading days are present. [2025-10-20T19:30:15.470166] [DEBUG] [VALIDATOR ] JPC: Primary data validation passed. [2025-10-20T19:30:15.471114] [INFO ] [MAIN ] JPC: Fetched primary data is valid. [2025-10-20T19:30:15.471321] [DEBUG] [LOCAL ] JPC: Saving primary data. [2025-10-20T19:30:15.472562] [DEBUG] [LOCAL ] JPC: Trimmed 55065 records to 37073 (keeping 200 days of data) [2025-10-20T19:30:15.682369] [DEBUG] [LOCAL ] JPC: Primary data saved (37073 records). [2025-10-20T19:30:15.682509] [INFO ] [MAIN ] JPC: Full history saved. [2025-10-20T19:30:15.682586] [INFO ] [MAIN ] Update Progress: 1165/2367 symbols completed. JPC took 29.5s. Estimated remaining: 2h 34m 57s. Rate: 7.8 symbols/min [2025-10-20T19:30:15.856757] [INFO ] [MAIN ] JRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:15.869542] [DEBUG] [LOCAL ] JRS: Loaded 11802 existing records from primary storage. [2025-10-20T19:30:15.869925] [INFO ] [MAIN ] JRS: Loaded 11802 existing records. [2025-10-20T19:30:16.079488] [WARN ] [MAIN ] JRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:16.079804] [INFO ] [MAIN ] JRS: Wiping existing primary data before full refresh. [2025-10-20T19:30:16.079946] [INFO ] [LOCAL ] JRS: Starting deletion of primary data. [2025-10-20T19:30:16.080115] [DEBUG] [LOCAL ] JRS: Deleted primary data file. [2025-10-20T19:30:16.080156] [INFO ] [MAIN ] JRS: Starting full primary refresh. [2025-10-20T19:30:16.275727] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-07. [2025-10-20T19:30:16.434420] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:30:16.504069] [INFO ] [REMOTE ] JRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:16.592705] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:30:17.051413] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-01. [2025-10-20T19:30:17.851470] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-02. [2025-10-20T19:30:17.865416] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T19:30:18.379097] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T19:30:18.651400] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-02. [2025-10-20T19:30:18.856716] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-07 (2.58 seconds). [2025-10-20T19:30:19.451415] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-08. [2025-10-20T19:30:19.471700] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:30:20.251398] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-03. [2025-10-20T19:30:21.051445] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-05. [2025-10-20T19:30:21.077197] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T19:30:21.702246] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T19:30:21.851423] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-04. [2025-10-20T19:30:22.052913] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T19:30:22.651381] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-09. [2025-10-20T19:30:22.661278] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T19:30:23.451408] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-06. [2025-10-20T19:30:24.251423] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-05. [2025-10-20T19:30:24.570018] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:30:24.714462] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T19:30:25.051361] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-03. [2025-10-20T19:30:25.097131] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T19:30:25.851394] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-06. [2025-10-20T19:30:26.060177] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:30:26.648327] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T19:30:26.652128] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-04. [2025-10-20T19:30:27.451338] [INFO ] [REMOTE ] JPM: Fetching primary data for 2025-10. [2025-10-20T19:30:27.641325] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T19:30:28.251446] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-07. [2025-10-20T19:30:29.051390] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-07. [2025-10-20T19:30:29.235754] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:30:29.851392] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-05. [2025-10-20T19:30:30.056906] [INFO ] [REMOTE ] JPM: Incremental fetch complete for 2025-10 (2.61 seconds). [2025-10-20T19:30:30.065021] [INFO ] [REMOTE ] JPM: Full fetch complete (77973 records). [2025-10-20T19:30:30.065141] [DEBUG] [VALIDATOR ] JPM: Validating primary data. [2025-10-20T19:30:30.481154] [DEBUG] [VALIDATOR ] JPM: All expected trading days are present. [2025-10-20T19:30:30.481225] [DEBUG] [VALIDATOR ] JPM: Primary data validation passed. [2025-10-20T19:30:30.482300] [INFO ] [MAIN ] JPM: Fetched primary data is valid. [2025-10-20T19:30:30.482364] [DEBUG] [LOCAL ] JPM: Saving primary data. [2025-10-20T19:30:30.483864] [DEBUG] [LOCAL ] JPM: Trimmed 77973 records to 53794 (keeping 200 days of data) [2025-10-20T19:30:30.598869] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:30:30.695696] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-08. [2025-10-20T19:30:30.779819] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T19:30:30.868741] [DEBUG] [LOCAL ] JPM: Primary data saved (53794 records). [2025-10-20T19:30:30.868901] [INFO ] [MAIN ] JPM: Full history saved. [2025-10-20T19:30:30.868982] [INFO ] [MAIN ] Update Progress: 1166/2367 symbols completed. JPM took 40.2s. Estimated remaining: 2h 34m 57s. Rate: 7.8 symbols/min [2025-10-20T19:30:31.037102] [INFO ] [MAIN ] JXN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:31.077850] [DEBUG] [LOCAL ] JXN: Loaded 45301 existing records from primary storage. [2025-10-20T19:30:31.078179] [INFO ] [MAIN ] JXN: Loaded 45301 existing records. [2025-10-20T19:30:31.255051] [WARN ] [MAIN ] JXN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:31.255253] [INFO ] [MAIN ] JXN: Wiping existing primary data before full refresh. [2025-10-20T19:30:31.255321] [INFO ] [LOCAL ] JXN: Starting deletion of primary data. [2025-10-20T19:30:31.255537] [DEBUG] [LOCAL ] JXN: Deleted primary data file. [2025-10-20T19:30:31.255594] [INFO ] [MAIN ] JXN: Starting full primary refresh. [2025-10-20T19:30:31.461153] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-06. [2025-10-20T19:30:31.611945] [INFO ] [REMOTE ] JXN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:32.115190] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:30:32.251345] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-09. [2025-10-20T19:30:32.420659] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T19:30:33.051373] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-07. [2025-10-20T19:30:33.571228] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:30:33.851351] [INFO ] [REMOTE ] JQC: Fetching primary data for 2025-10. [2025-10-20T19:30:34.158808] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T19:30:34.651436] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-08. [2025-10-20T19:30:35.150263] [INFO ] [REMOTE ] JQC: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T19:30:35.156379] [INFO ] [REMOTE ] JQC: Full fetch complete (49835 records). [2025-10-20T19:30:35.156524] [DEBUG] [VALIDATOR ] JQC: Validating primary data. [2025-10-20T19:30:35.457097] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-01. [2025-10-20T19:30:35.471181] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:30:35.562021] [DEBUG] [VALIDATOR ] JQC: All expected trading days are present. [2025-10-20T19:30:35.562103] [DEBUG] [VALIDATOR ] JQC: Primary data validation passed. [2025-10-20T19:30:35.562877] [INFO ] [MAIN ] JQC: Fetched primary data is valid. [2025-10-20T19:30:35.562940] [DEBUG] [LOCAL ] JQC: Saving primary data. [2025-10-20T19:30:35.564084] [DEBUG] [LOCAL ] JQC: Trimmed 49835 records to 33824 (keeping 200 days of data) [2025-10-20T19:30:35.760674] [DEBUG] [LOCAL ] JQC: Primary data saved (33824 records). [2025-10-20T19:30:35.760824] [INFO ] [MAIN ] JQC: Full history saved. [2025-10-20T19:30:35.760902] [INFO ] [MAIN ] Update Progress: 1167/2367 symbols completed. JQC took 27.5s. Estimated remaining: 2h 34m 47s. Rate: 7.8 symbols/min [2025-10-20T19:30:35.930724] [INFO ] [MAIN ] K: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:35.979004] [DEBUG] [LOCAL ] K: Loaded 51154 existing records from primary storage. [2025-10-20T19:30:35.979284] [INFO ] [MAIN ] K: Loaded 51154 existing records. [2025-10-20T19:30:36.155405] [WARN ] [MAIN ] K: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:36.155613] [INFO ] [MAIN ] K: Wiping existing primary data before full refresh. [2025-10-20T19:30:36.155704] [INFO ] [LOCAL ] K: Starting deletion of primary data. [2025-10-20T19:30:36.155892] [DEBUG] [LOCAL ] K: Deleted primary data file. [2025-10-20T19:30:36.155931] [INFO ] [MAIN ] K: Starting full primary refresh. [2025-10-20T19:30:36.257316] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-08. [2025-10-20T19:30:36.507753] [INFO ] [REMOTE ] K: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:37.051339] [INFO ] [REMOTE ] K: Fetching primary data for 2025-01. [2025-10-20T19:30:37.113677] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T19:30:37.115896] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:30:37.851423] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-09. [2025-10-20T19:30:38.651345] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-09. [2025-10-20T19:30:38.760290] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:30:38.853226] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:30:39.451351] [INFO ] [REMOTE ] JRI: Fetching primary data for 2025-10. [2025-10-20T19:30:39.504869] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:30:40.251398] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-02. [2025-10-20T19:30:40.389506] [INFO ] [REMOTE ] JRI: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:30:40.392461] [INFO ] [REMOTE ] JRI: Full fetch complete (23130 records). [2025-10-20T19:30:40.392542] [DEBUG] [VALIDATOR ] JRI: Validating primary data. [2025-10-20T19:30:40.770545] [DEBUG] [VALIDATOR ] JRI: All expected trading days are present. [2025-10-20T19:30:40.770622] [DEBUG] [VALIDATOR ] JRI: Primary data validation passed. [2025-10-20T19:30:40.771114] [INFO ] [MAIN ] JRI: Fetched primary data is valid. [2025-10-20T19:30:40.771165] [DEBUG] [LOCAL ] JRI: Saving primary data. [2025-10-20T19:30:40.771752] [DEBUG] [LOCAL ] JRI: Trimmed 23130 records to 15423 (keeping 200 days of data) [2025-10-20T19:30:40.863810] [DEBUG] [LOCAL ] JRI: Primary data saved (15423 records). [2025-10-20T19:30:40.863956] [INFO ] [MAIN ] JRI: Full history saved. [2025-10-20T19:30:40.864034] [INFO ] [MAIN ] Update Progress: 1168/2367 symbols completed. JRI took 26.7s. Estimated remaining: 2h 34m 36s. Rate: 7.8 symbols/min [2025-10-20T19:30:41.032774] [INFO ] [MAIN ] KAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:41.047058] [DEBUG] [LOCAL ] KAI: Loaded 13783 existing records from primary storage. [2025-10-20T19:30:41.047248] [INFO ] [MAIN ] KAI: Loaded 13783 existing records. [2025-10-20T19:30:41.058281] [INFO ] [REMOTE ] K: Fetching primary data for 2025-02. [2025-10-20T19:30:41.224289] [WARN ] [MAIN ] KAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:41.224529] [INFO ] [MAIN ] KAI: Wiping existing primary data before full refresh. [2025-10-20T19:30:41.224583] [INFO ] [LOCAL ] KAI: Starting deletion of primary data. [2025-10-20T19:30:41.224835] [DEBUG] [LOCAL ] KAI: Deleted primary data file. [2025-10-20T19:30:41.224892] [INFO ] [MAIN ] KAI: Starting full primary refresh. [2025-10-20T19:30:41.581547] [INFO ] [REMOTE ] KAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:41.606505] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T19:30:41.851375] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-01. [2025-10-20T19:30:42.625845] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T19:30:42.651341] [INFO ] [REMOTE ] JRS: Fetching primary data for 2025-10. [2025-10-20T19:30:42.703133] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:30:43.451357] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-02. [2025-10-20T19:30:43.524715] [INFO ] [REMOTE ] JRS: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:30:43.527047] [INFO ] [REMOTE ] JRS: Full fetch complete (16227 records). [2025-10-20T19:30:43.527120] [DEBUG] [VALIDATOR ] JRS: Validating primary data. [2025-10-20T19:30:43.877787] [DEBUG] [VALIDATOR ] JRS: All expected trading days are present. [2025-10-20T19:30:43.877907] [DEBUG] [VALIDATOR ] JRS: Primary data validation passed. [2025-10-20T19:30:43.878251] [INFO ] [MAIN ] JRS: Fetched primary data is valid. [2025-10-20T19:30:43.878290] [DEBUG] [LOCAL ] JRS: Saving primary data. [2025-10-20T19:30:43.878877] [DEBUG] [LOCAL ] JRS: Trimmed 16227 records to 11623 (keeping 200 days of data) [2025-10-20T19:30:43.943455] [DEBUG] [LOCAL ] JRS: Primary data saved (11623 records). [2025-10-20T19:30:43.943578] [INFO ] [MAIN ] JRS: Full history saved. [2025-10-20T19:30:43.943673] [INFO ] [MAIN ] Update Progress: 1169/2367 symbols completed. JRS took 28.3s. Estimated remaining: 2h 34m 24s. Rate: 7.8 symbols/min [2025-10-20T19:30:44.112819] [INFO ] [MAIN ] KAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:30:44.149234] [DEBUG] [LOCAL ] KAR: Loaded 40601 existing records from primary storage. [2025-10-20T19:30:44.149508] [INFO ] [MAIN ] KAR: Loaded 40601 existing records. [2025-10-20T19:30:44.256575] [INFO ] [REMOTE ] K: Fetching primary data for 2025-03. [2025-10-20T19:30:44.329777] [WARN ] [MAIN ] KAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:30:44.330011] [INFO ] [MAIN ] KAR: Wiping existing primary data before full refresh. [2025-10-20T19:30:44.330205] [INFO ] [LOCAL ] KAR: Starting deletion of primary data. [2025-10-20T19:30:44.330388] [DEBUG] [LOCAL ] KAR: Deleted primary data file. [2025-10-20T19:30:44.330427] [INFO ] [MAIN ] KAR: Starting full primary refresh. [2025-10-20T19:30:44.513416] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T19:30:44.693637] [INFO ] [REMOTE ] KAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:30:45.051350] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-03. [2025-10-20T19:30:45.851416] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-01. [2025-10-20T19:30:45.910326] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:30:46.354151] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:30:46.651433] [INFO ] [REMOTE ] K: Fetching primary data for 2025-04. [2025-10-20T19:30:46.735415] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T19:30:47.451408] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-04. [2025-10-20T19:30:48.251356] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-02. [2025-10-20T19:30:48.385832] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:30:49.051411] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-03. [2025-10-20T19:30:49.095512] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:30:49.225352] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T19:30:49.851353] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-03. [2025-10-20T19:30:50.029542] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T19:30:50.651383] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-05. [2025-10-20T19:30:51.350891] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:30:51.451365] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-04. [2025-10-20T19:30:52.251448] [INFO ] [REMOTE ] K: Fetching primary data for 2025-05. [2025-10-20T19:30:52.301760] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:30:53.043385] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:30:53.051544] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-06. [2025-10-20T19:30:53.851369] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-05. [2025-10-20T19:30:53.984532] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:30:54.650187] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:30:54.658942] [INFO ] [REMOTE ] K: Fetching primary data for 2025-06. [2025-10-20T19:30:55.380190] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:30:55.451377] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-06. [2025-10-20T19:30:56.140084] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:30:56.251388] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-07. [2025-10-20T19:30:56.833607] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:30:57.051413] [INFO ] [REMOTE ] K: Fetching primary data for 2025-07. [2025-10-20T19:30:57.748051] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:30:57.851379] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-07. [2025-10-20T19:30:58.651441] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-08. [2025-10-20T19:30:58.810208] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:30:59.375912] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:30:59.451368] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-04. [2025-10-20T19:31:00.251348] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-08. [2025-10-20T19:31:00.311422] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:31:00.487894] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:31:01.051407] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-09. [2025-10-20T19:31:01.804819] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:31:01.851340] [INFO ] [REMOTE ] K: Fetching primary data for 2025-08. [2025-10-20T19:31:02.138209] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T19:31:02.651422] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-05. [2025-10-20T19:31:03.435767] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:31:03.451330] [INFO ] [REMOTE ] K: Fetching primary data for 2025-09. [2025-10-20T19:31:03.656899] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T19:31:04.251358] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-09. [2025-10-20T19:31:05.051404] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-06. [2025-10-20T19:31:05.369834] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T19:31:05.851349] [INFO ] [REMOTE ] JXN: Fetching primary data for 2025-10. [2025-10-20T19:31:05.962169] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:31:06.225955] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:31:06.651390] [INFO ] [REMOTE ] KAR: Fetching primary data for 2025-10. [2025-10-20T19:31:07.451381] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-07. [2025-10-20T19:31:07.591822] [INFO ] [REMOTE ] JXN: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:31:07.597825] [INFO ] [REMOTE ] JXN: Full fetch complete (65622 records). [2025-10-20T19:31:07.597971] [DEBUG] [VALIDATOR ] JXN: Validating primary data. [2025-10-20T19:31:08.004160] [DEBUG] [VALIDATOR ] JXN: All expected trading days are present. [2025-10-20T19:31:08.004255] [DEBUG] [VALIDATOR ] JXN: Primary data validation passed. [2025-10-20T19:31:08.005430] [INFO ] [MAIN ] JXN: Fetched primary data is valid. [2025-10-20T19:31:08.005496] [DEBUG] [LOCAL ] JXN: Saving primary data. [2025-10-20T19:31:08.007279] [DEBUG] [LOCAL ] JXN: Trimmed 65622 records to 44551 (keeping 200 days of data) [2025-10-20T19:31:08.242890] [INFO ] [REMOTE ] KAR: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:31:08.262855] [INFO ] [REMOTE ] K: Fetching primary data for 2025-10. [2025-10-20T19:31:08.272793] [INFO ] [REMOTE ] KAR: Full fetch complete (56954 records). [2025-10-20T19:31:08.273980] [DEBUG] [VALIDATOR ] KAR: Validating primary data. [2025-10-20T19:31:08.575113] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:31:08.642670] [DEBUG] [LOCAL ] JXN: Primary data saved (44551 records). [2025-10-20T19:31:08.645590] [INFO ] [MAIN ] JXN: Full history saved. [2025-10-20T19:31:08.645755] [INFO ] [MAIN ] Update Progress: 1170/2367 symbols completed. JXN took 37.8s. Estimated remaining: 2h 34m 33s. Rate: 7.7 symbols/min [2025-10-20T19:31:08.892154] [DEBUG] [VALIDATOR ] KAR: All expected trading days are present. [2025-10-20T19:31:08.897759] [DEBUG] [VALIDATOR ] KAR: Primary data validation passed. [2025-10-20T19:31:08.900030] [INFO ] [MAIN ] KAR: Fetched primary data is valid. [2025-10-20T19:31:08.901153] [DEBUG] [LOCAL ] KAR: Saving primary data. [2025-10-20T19:31:08.918947] [INFO ] [MAIN ] KB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:08.920489] [DEBUG] [LOCAL ] KAR: Trimmed 56954 records to 39864 (keeping 200 days of data) [2025-10-20T19:31:09.051405] [DEBUG] [LOCAL ] KB: Loaded 23613 existing records from primary storage. [2025-10-20T19:31:09.051481] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-08. [2025-10-20T19:31:09.058344] [INFO ] [MAIN ] KB: Loaded 23613 existing records. [2025-10-20T19:31:09.337295] [WARN ] [MAIN ] KB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:09.338293] [INFO ] [MAIN ] KB: Wiping existing primary data before full refresh. [2025-10-20T19:31:09.338685] [INFO ] [LOCAL ] KB: Starting deletion of primary data. [2025-10-20T19:31:09.341616] [DEBUG] [LOCAL ] KB: Deleted primary data file. [2025-10-20T19:31:09.341999] [INFO ] [MAIN ] KB: Starting full primary refresh. [2025-10-20T19:31:09.658703] [DEBUG] [LOCAL ] KAR: Primary data saved (39864 records). [2025-10-20T19:31:09.669135] [INFO ] [MAIN ] KAR: Full history saved. [2025-10-20T19:31:09.692809] [INFO ] [MAIN ] Update Progress: 1171/2367 symbols completed. KAR took 25.7s. Estimated remaining: 2h 34m 19s. Rate: 7.8 symbols/min [2025-10-20T19:31:09.803084] [INFO ] [REMOTE ] KB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:09.856440] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-01. [2025-10-20T19:31:09.916075] [INFO ] [MAIN ] KBDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:09.924791] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T19:31:09.946753] [DEBUG] [LOCAL ] KBDC: Loaded 22794 existing records from primary storage. [2025-10-20T19:31:09.947022] [INFO ] [MAIN ] KBDC: Loaded 22794 existing records. [2025-10-20T19:31:10.157327] [WARN ] [MAIN ] KBDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:10.157582] [INFO ] [MAIN ] KBDC: Wiping existing primary data before full refresh. [2025-10-20T19:31:10.157684] [INFO ] [LOCAL ] KBDC: Starting deletion of primary data. [2025-10-20T19:31:10.158060] [DEBUG] [LOCAL ] KBDC: Deleted primary data file. [2025-10-20T19:31:10.158166] [INFO ] [MAIN ] KBDC: Starting full primary refresh. [2025-10-20T19:31:10.348574] [INFO ] [REMOTE ] K: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T19:31:10.387877] [INFO ] [REMOTE ] K: Full fetch complete (72302 records). [2025-10-20T19:31:10.435210] [DEBUG] [VALIDATOR ] K: Validating primary data. [2025-10-20T19:31:10.655236] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-09. [2025-10-20T19:31:10.708064] [INFO ] [REMOTE ] KBDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:10.986807] [DEBUG] [VALIDATOR ] K: All expected trading days are present. [2025-10-20T19:31:10.986972] [DEBUG] [VALIDATOR ] K: Primary data validation passed. [2025-10-20T19:31:10.988610] [INFO ] [MAIN ] K: Fetched primary data is valid. [2025-10-20T19:31:10.989240] [DEBUG] [LOCAL ] K: Saving primary data. [2025-10-20T19:31:10.989159] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T19:31:10.995065] [DEBUG] [LOCAL ] K: Trimmed 72302 records to 50372 (keeping 200 days of data) [2025-10-20T19:31:11.298428] [DEBUG] [LOCAL ] K: Primary data saved (50372 records). [2025-10-20T19:31:11.298570] [INFO ] [MAIN ] K: Full history saved. [2025-10-20T19:31:11.299298] [INFO ] [MAIN ] Update Progress: 1172/2367 symbols completed. K took 35.5s. Estimated remaining: 2h 34m 5s. Rate: 7.8 symbols/min [2025-10-20T19:31:11.464315] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-02. [2025-10-20T19:31:11.467040] [INFO ] [MAIN ] KBH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:11.513963] [DEBUG] [LOCAL ] KBH: Loaded 49532 existing records from primary storage. [2025-10-20T19:31:11.514263] [INFO ] [MAIN ] KBH: Loaded 49532 existing records. [2025-10-20T19:31:11.628214] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:31:11.701272] [WARN ] [MAIN ] KBH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:11.701478] [INFO ] [MAIN ] KBH: Wiping existing primary data before full refresh. [2025-10-20T19:31:11.701524] [INFO ] [LOCAL ] KBH: Starting deletion of primary data. [2025-10-20T19:31:11.701997] [DEBUG] [LOCAL ] KBH: Deleted primary data file. [2025-10-20T19:31:11.702053] [INFO ] [MAIN ] KBH: Starting full primary refresh. [2025-10-20T19:31:12.046105] [INFO ] [REMOTE ] KBH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:12.251337] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-01. [2025-10-20T19:31:12.512196] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T19:31:13.051395] [INFO ] [REMOTE ] KAI: Fetching primary data for 2025-10. [2025-10-20T19:31:13.851369] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-01. [2025-10-20T19:31:14.071862] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T19:31:14.120607] [INFO ] [REMOTE ] KAI: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T19:31:14.123237] [INFO ] [REMOTE ] KAI: Full fetch complete (19280 records). [2025-10-20T19:31:14.123308] [DEBUG] [VALIDATOR ] KAI: Validating primary data. [2025-10-20T19:31:14.480680] [DEBUG] [VALIDATOR ] KAI: All expected trading days are present. [2025-10-20T19:31:14.480766] [DEBUG] [VALIDATOR ] KAI: Primary data validation passed. [2025-10-20T19:31:14.481190] [INFO ] [MAIN ] KAI: Fetched primary data is valid. [2025-10-20T19:31:14.481234] [DEBUG] [LOCAL ] KAI: Saving primary data. [2025-10-20T19:31:14.481784] [DEBUG] [LOCAL ] KAI: Trimmed 19280 records to 13606 (keeping 200 days of data) [2025-10-20T19:31:14.558917] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:31:14.582864] [DEBUG] [LOCAL ] KAI: Primary data saved (13606 records). [2025-10-20T19:31:14.582999] [INFO ] [MAIN ] KAI: Full history saved. [2025-10-20T19:31:14.583075] [INFO ] [MAIN ] Update Progress: 1173/2367 symbols completed. KAI took 33.7s. Estimated remaining: 2h 33m 53s. Rate: 7.8 symbols/min [2025-10-20T19:31:14.656486] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-02. [2025-10-20T19:31:14.777475] [INFO ] [MAIN ] KBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:14.822257] [DEBUG] [LOCAL ] KBR: Loaded 48737 existing records from primary storage. [2025-10-20T19:31:14.822566] [INFO ] [MAIN ] KBR: Loaded 48737 existing records. [2025-10-20T19:31:15.000570] [WARN ] [MAIN ] KBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:15.000830] [INFO ] [MAIN ] KBR: Wiping existing primary data before full refresh. [2025-10-20T19:31:15.000877] [INFO ] [LOCAL ] KBR: Starting deletion of primary data. [2025-10-20T19:31:15.001064] [DEBUG] [LOCAL ] KBR: Deleted primary data file. [2025-10-20T19:31:15.001102] [INFO ] [MAIN ] KBR: Starting full primary refresh. [2025-10-20T19:31:15.364594] [INFO ] [REMOTE ] KBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:15.451411] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-03. [2025-10-20T19:31:15.657332] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T19:31:16.251339] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-03. [2025-10-20T19:31:16.423797] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T19:31:17.051429] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-02. [2025-10-20T19:31:17.321829] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T19:31:17.851465] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-04. [2025-10-20T19:31:18.506498] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:31:18.651354] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-03. [2025-10-20T19:31:18.932594] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T19:31:19.451422] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-05. [2025-10-20T19:31:20.251390] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-01. [2025-10-20T19:31:20.318959] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:31:20.504137] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T19:31:21.051439] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-04. [2025-10-20T19:31:21.851430] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-06. [2025-10-20T19:31:21.885261] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:31:22.663290] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-02. [2025-10-20T19:31:22.668729] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:31:22.836177] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:31:23.451425] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-04. [2025-10-20T19:31:24.251391] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-05. [2025-10-20T19:31:24.307100] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:31:24.596593] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:31:25.051392] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-07. [2025-10-20T19:31:25.712597] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:31:25.851415] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-06. [2025-10-20T19:31:26.110298] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T19:31:26.651449] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-05. [2025-10-20T19:31:27.408227] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:31:27.451359] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-07. [2025-10-20T19:31:27.638590] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T19:31:28.251408] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-08. [2025-10-20T19:31:29.051406] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-06. [2025-10-20T19:31:29.223965] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:31:29.292286] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T19:31:29.833187] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T19:31:29.851349] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-07. [2025-10-20T19:31:30.651435] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-03. [2025-10-20T19:31:31.180720] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T19:31:31.451374] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-08. [2025-10-20T19:31:32.129797] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:31:32.251370] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-09. [2025-10-20T19:31:33.051364] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-08. [2025-10-20T19:31:33.118285] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:31:33.509451] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T19:31:33.851347] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-04. [2025-10-20T19:31:34.123025] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:31:34.651449] [INFO ] [REMOTE ] KBDC: Fetching primary data for 2025-10. [2025-10-20T19:31:35.316074] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:31:35.451362] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-09. [2025-10-20T19:31:35.961342] [INFO ] [REMOTE ] KBDC: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:31:35.966347] [INFO ] [REMOTE ] KBDC: Full fetch complete (30743 records). [2025-10-20T19:31:35.966430] [DEBUG] [VALIDATOR ] KBDC: Validating primary data. [2025-10-20T19:31:36.256529] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-09. [2025-10-20T19:31:36.335105] [DEBUG] [VALIDATOR ] KBDC: All expected trading days are present. [2025-10-20T19:31:36.335212] [DEBUG] [VALIDATOR ] KBDC: Primary data validation passed. [2025-10-20T19:31:36.335924] [INFO ] [MAIN ] KBDC: Fetched primary data is valid. [2025-10-20T19:31:36.335984] [DEBUG] [LOCAL ] KBDC: Saving primary data. [2025-10-20T19:31:36.336694] [DEBUG] [LOCAL ] KBDC: Trimmed 30743 records to 22651 (keeping 200 days of data) [2025-10-20T19:31:36.472541] [DEBUG] [LOCAL ] KBDC: Primary data saved (22651 records). [2025-10-20T19:31:36.472702] [INFO ] [MAIN ] KBDC: Full history saved. [2025-10-20T19:31:36.472797] [INFO ] [MAIN ] Update Progress: 1174/2367 symbols completed. KBDC took 26.8s. Estimated remaining: 2h 33m 59s. Rate: 7.7 symbols/min [2025-10-20T19:31:36.647235] [INFO ] [MAIN ] KD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:36.695042] [DEBUG] [LOCAL ] KD: Loaded 53125 existing records from primary storage. [2025-10-20T19:31:36.695349] [INFO ] [MAIN ] KD: Loaded 53125 existing records. [2025-10-20T19:31:36.876931] [WARN ] [MAIN ] KD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:36.877147] [INFO ] [MAIN ] KD: Wiping existing primary data before full refresh. [2025-10-20T19:31:36.877206] [INFO ] [LOCAL ] KD: Starting deletion of primary data. [2025-10-20T19:31:36.877381] [DEBUG] [LOCAL ] KD: Deleted primary data file. [2025-10-20T19:31:36.877420] [INFO ] [MAIN ] KD: Starting full primary refresh. [2025-10-20T19:31:37.059090] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-05. [2025-10-20T19:31:37.225584] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:31:37.277633] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:31:37.346550] [INFO ] [REMOTE ] KD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:37.851401] [INFO ] [REMOTE ] KBH: Fetching primary data for 2025-10. [2025-10-20T19:31:38.651399] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-01. [2025-10-20T19:31:38.734785] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:31:39.451357] [INFO ] [REMOTE ] KB: Fetching primary data for 2025-10. [2025-10-20T19:31:39.615923] [INFO ] [REMOTE ] KBH: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:31:39.623134] [INFO ] [REMOTE ] KBH: Full fetch complete (71099 records). [2025-10-20T19:31:39.623265] [DEBUG] [VALIDATOR ] KBH: Validating primary data. [2025-10-20T19:31:40.013937] [DEBUG] [VALIDATOR ] KBH: All expected trading days are present. [2025-10-20T19:31:40.014022] [DEBUG] [VALIDATOR ] KBH: Primary data validation passed. [2025-10-20T19:31:40.014978] [INFO ] [MAIN ] KBH: Fetched primary data is valid. [2025-10-20T19:31:40.015042] [DEBUG] [LOCAL ] KBH: Saving primary data. [2025-10-20T19:31:40.016385] [DEBUG] [LOCAL ] KBH: Trimmed 71099 records to 48759 (keeping 200 days of data) [2025-10-20T19:31:40.253696] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-06. [2025-10-20T19:31:40.315476] [DEBUG] [LOCAL ] KBH: Primary data saved (48759 records). [2025-10-20T19:31:40.315639] [INFO ] [MAIN ] KBH: Full history saved. [2025-10-20T19:31:40.316601] [INFO ] [MAIN ] Update Progress: 1175/2367 symbols completed. KBH took 29.0s. Estimated remaining: 2h 33m 48s. Rate: 7.8 symbols/min [2025-10-20T19:31:40.511029] [INFO ] [MAIN ] KEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:40.543571] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T19:31:40.547923] [DEBUG] [LOCAL ] KEN: Loaded 4731 existing records from primary storage. [2025-10-20T19:31:40.558900] [INFO ] [MAIN ] KEN: Loaded 4731 existing records. [2025-10-20T19:31:40.740857] [INFO ] [REMOTE ] KB: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T19:31:40.780698] [WARN ] [MAIN ] KEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:40.780854] [INFO ] [REMOTE ] KB: Full fetch complete (33610 records). [2025-10-20T19:31:40.781183] [INFO ] [MAIN ] KEN: Wiping existing primary data before full refresh. [2025-10-20T19:31:40.781058] [DEBUG] [VALIDATOR ] KB: Validating primary data. [2025-10-20T19:31:40.781308] [INFO ] [LOCAL ] KEN: Starting deletion of primary data. [2025-10-20T19:31:40.812938] [DEBUG] [LOCAL ] KEN: Deleted primary data file. [2025-10-20T19:31:40.813029] [INFO ] [MAIN ] KEN: Starting full primary refresh. [2025-10-20T19:31:41.052704] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-02. [2025-10-20T19:31:41.533354] [INFO ] [REMOTE ] KEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:41.548036] [DEBUG] [VALIDATOR ] KB: All expected trading days are present. [2025-10-20T19:31:41.548414] [DEBUG] [VALIDATOR ] KB: Primary data validation passed. [2025-10-20T19:31:41.549116] [INFO ] [MAIN ] KB: Fetched primary data is valid. [2025-10-20T19:31:41.549177] [DEBUG] [LOCAL ] KB: Saving primary data. [2025-10-20T19:31:41.550065] [DEBUG] [LOCAL ] KB: Trimmed 33610 records to 23270 (keeping 200 days of data) [2025-10-20T19:31:41.687695] [DEBUG] [LOCAL ] KB: Primary data saved (23270 records). [2025-10-20T19:31:41.687909] [INFO ] [MAIN ] KB: Full history saved. [2025-10-20T19:31:41.688148] [INFO ] [MAIN ] Update Progress: 1176/2367 symbols completed. KB took 33.0s. Estimated remaining: 2h 33m 33s. Rate: 7.8 symbols/min [2025-10-20T19:31:41.856203] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-01. [2025-10-20T19:31:41.858910] [INFO ] [MAIN ] KEP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:41.888634] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:31:41.924599] [DEBUG] [LOCAL ] KEP: Loaded 32548 existing records from primary storage. [2025-10-20T19:31:41.924900] [INFO ] [MAIN ] KEP: Loaded 32548 existing records. [2025-10-20T19:31:42.098172] [WARN ] [MAIN ] KEP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:42.098412] [INFO ] [MAIN ] KEP: Wiping existing primary data before full refresh. [2025-10-20T19:31:42.098459] [INFO ] [LOCAL ] KEP: Starting deletion of primary data. [2025-10-20T19:31:42.098696] [DEBUG] [LOCAL ] KEP: Deleted primary data file. [2025-10-20T19:31:42.098754] [INFO ] [MAIN ] KEP: Starting full primary refresh. [2025-10-20T19:31:42.443069] [INFO ] [REMOTE ] KEP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:42.651353] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-01. [2025-10-20T19:31:42.830274] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T19:31:43.451362] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-02. [2025-10-20T19:31:44.070149] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-02 (3.02 seconds). [2025-10-20T19:31:44.251489] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-03. [2025-10-20T19:31:44.716708] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:31:45.051403] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-03. [2025-10-20T19:31:45.613987] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:31:45.666162] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-01 (3.01 seconds). [2025-10-20T19:31:45.851391] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-07. [2025-10-20T19:31:46.304522] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T19:31:46.651406] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-04. [2025-10-20T19:31:47.218734] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T19:31:47.451372] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-02. [2025-10-20T19:31:47.626943] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T19:31:48.251351] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-08. [2025-10-20T19:31:48.488696] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T19:31:49.051372] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-03. [2025-10-20T19:31:49.856536] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-05. [2025-10-20T19:31:49.862883] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T19:31:50.105101] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:31:50.505380] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T19:31:50.651374] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-04. [2025-10-20T19:31:51.451391] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-09. [2025-10-20T19:31:52.256513] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-06. [2025-10-20T19:31:52.276304] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:31:52.896716] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T19:31:52.938577] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:31:53.051368] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-07. [2025-10-20T19:31:53.699137] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T19:31:53.851369] [INFO ] [REMOTE ] KBR: Fetching primary data for 2025-10. [2025-10-20T19:31:54.651424] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-04. [2025-10-20T19:31:55.451438] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-08. [2025-10-20T19:31:55.729624] [INFO ] [REMOTE ] KBR: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T19:31:55.740831] [INFO ] [REMOTE ] KBR: Full fetch complete (69946 records). [2025-10-20T19:31:55.741015] [DEBUG] [VALIDATOR ] KBR: Validating primary data. [2025-10-20T19:31:56.060355] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:31:56.060497] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:31:56.184344] [DEBUG] [VALIDATOR ] KBR: All expected trading days are present. [2025-10-20T19:31:56.184431] [DEBUG] [VALIDATOR ] KBR: Primary data validation passed. [2025-10-20T19:31:56.185365] [INFO ] [MAIN ] KBR: Fetched primary data is valid. [2025-10-20T19:31:56.185428] [DEBUG] [LOCAL ] KBR: Saving primary data. [2025-10-20T19:31:56.186790] [DEBUG] [LOCAL ] KBR: Trimmed 69946 records to 48051 (keeping 200 days of data) [2025-10-20T19:31:56.275302] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-05. [2025-10-20T19:31:56.486315] [DEBUG] [LOCAL ] KBR: Primary data saved (48051 records). [2025-10-20T19:31:56.486457] [INFO ] [MAIN ] KBR: Full history saved. [2025-10-20T19:31:56.486535] [INFO ] [MAIN ] Update Progress: 1177/2367 symbols completed. KBR took 41.9s. Estimated remaining: 2h 33m 33s. Rate: 7.8 symbols/min [2025-10-20T19:31:56.662274] [INFO ] [MAIN ] KEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:31:56.699480] [DEBUG] [LOCAL ] KEX: Loaded 39285 existing records from primary storage. [2025-10-20T19:31:56.700000] [INFO ] [MAIN ] KEX: Loaded 39285 existing records. [2025-10-20T19:31:56.889660] [WARN ] [MAIN ] KEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:31:56.890993] [INFO ] [MAIN ] KEX: Wiping existing primary data before full refresh. [2025-10-20T19:31:56.891079] [INFO ] [LOCAL ] KEX: Starting deletion of primary data. [2025-10-20T19:31:56.891849] [DEBUG] [LOCAL ] KEX: Deleted primary data file. [2025-10-20T19:31:56.891911] [INFO ] [MAIN ] KEX: Starting full primary refresh. [2025-10-20T19:31:57.052676] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-09. [2025-10-20T19:31:57.264230] [INFO ] [REMOTE ] KEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:31:57.856487] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-01. [2025-10-20T19:31:57.883528] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:31:57.901561] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T19:31:58.651434] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-06. [2025-10-20T19:31:59.451474] [INFO ] [REMOTE ] KEN: Fetching primary data for 2025-10. [2025-10-20T19:31:59.484355] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:32:00.083991] [INFO ] [REMOTE ] KEN: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T19:32:00.085487] [INFO ] [REMOTE ] KEN: Full fetch complete (5938 records). [2025-10-20T19:32:00.085691] [DEBUG] [VALIDATOR ] KEN: Validating primary data. [2025-10-20T19:32:00.258583] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-05. [2025-10-20T19:32:00.477108] [DEBUG] [VALIDATOR ] KEN: All expected trading days are present. [2025-10-20T19:32:00.477225] [DEBUG] [VALIDATOR ] KEN: Primary data validation passed. [2025-10-20T19:32:00.477462] [INFO ] [MAIN ] KEN: Fetched primary data is valid. [2025-10-20T19:32:00.477570] [DEBUG] [LOCAL ] KEN: Saving primary data. [2025-10-20T19:32:00.478108] [DEBUG] [LOCAL ] KEN: Trimmed 5938 records to 4695 (keeping 200 days of data) [2025-10-20T19:32:00.536148] [DEBUG] [LOCAL ] KEN: Primary data saved (4695 records). [2025-10-20T19:32:00.536333] [INFO ] [MAIN ] KEN: Full history saved. [2025-10-20T19:32:00.536471] [INFO ] [MAIN ] Update Progress: 1178/2367 symbols completed. KEN took 20.2s. Estimated remaining: 2h 33m 21s. Rate: 7.8 symbols/min [2025-10-20T19:32:00.597922] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:32:00.749699] [INFO ] [MAIN ] KEY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:00.749886] [DEBUG] [LOCAL ] KEY: No existing data file found in primary storage. [2025-10-20T19:32:00.749923] [WARN ] [MAIN ] KEY: No existing data found, performing full refresh instead. [2025-10-20T19:32:00.750035] [INFO ] [MAIN ] KEY: Wiping existing primary data before full refresh. [2025-10-20T19:32:00.750076] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-10-20T19:32:00.750116] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-10-20T19:32:00.750147] [INFO ] [MAIN ] KEY: Starting full primary refresh. [2025-10-20T19:32:01.056482] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-07. [2025-10-20T19:32:01.114242] [INFO ] [REMOTE ] KEY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:01.574748] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:32:01.851370] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-01. [2025-10-20T19:32:02.651347] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-02. [2025-10-20T19:32:03.076690] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T19:32:03.451350] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-06. [2025-10-20T19:32:03.622673] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T19:32:03.959606] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:32:04.251384] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-03. [2025-10-20T19:32:05.003733] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:32:05.051380] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-07. [2025-10-20T19:32:05.694927] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:32:05.851387] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-04. [2025-10-20T19:32:06.392835] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T19:32:06.651393] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-02. [2025-10-20T19:32:07.339772] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T19:32:07.451441] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-05. [2025-10-20T19:32:08.251425] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-08. [2025-10-20T19:32:08.304885] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:32:08.872205] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:32:09.051372] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-06. [2025-10-20T19:32:09.851430] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-03. [2025-10-20T19:32:10.060458] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:32:10.175413] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T19:32:10.651415] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-07. [2025-10-20T19:32:11.451382] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-09. [2025-10-20T19:32:11.606075] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:32:12.082996] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:32:12.251365] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-08. [2025-10-20T19:32:13.041563] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:32:13.051963] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-08. [2025-10-20T19:32:13.665037] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:32:13.851423] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-09. [2025-10-20T19:32:14.426183] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:32:14.651395] [INFO ] [REMOTE ] KD: Fetching primary data for 2025-10. [2025-10-20T19:32:15.279104] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T19:32:15.451415] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-04. [2025-10-20T19:32:16.251352] [INFO ] [REMOTE ] KEX: Fetching primary data for 2025-10. [2025-10-20T19:32:16.648557] [INFO ] [REMOTE ] KD: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:32:16.657349] [INFO ] [REMOTE ] KD: Full fetch complete (76127 records). [2025-10-20T19:32:16.657545] [DEBUG] [VALIDATOR ] KD: Validating primary data. [2025-10-20T19:32:17.057027] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-09. [2025-10-20T19:32:17.082784] [DEBUG] [VALIDATOR ] KD: All expected trading days are present. [2025-10-20T19:32:17.082893] [DEBUG] [VALIDATOR ] KD: Primary data validation passed. [2025-10-20T19:32:17.084107] [INFO ] [MAIN ] KD: Fetched primary data is valid. [2025-10-20T19:32:17.084419] [DEBUG] [LOCAL ] KD: Saving primary data. [2025-10-20T19:32:17.086231] [DEBUG] [LOCAL ] KD: Trimmed 76127 records to 52354 (keeping 200 days of data) [2025-10-20T19:32:17.236479] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T19:32:17.434876] [DEBUG] [LOCAL ] KD: Primary data saved (52354 records). [2025-10-20T19:32:17.435011] [INFO ] [MAIN ] KD: Full history saved. [2025-10-20T19:32:17.435096] [INFO ] [MAIN ] Update Progress: 1179/2367 symbols completed. KD took 41.0s. Estimated remaining: 2h 33m 23s. Rate: 7.7 symbols/min [2025-10-20T19:32:17.608297] [INFO ] [MAIN ] KEYS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:17.608449] [DEBUG] [LOCAL ] KEYS: No existing data file found in primary storage. [2025-10-20T19:32:17.608502] [WARN ] [MAIN ] KEYS: No existing data found, performing full refresh instead. [2025-10-20T19:32:17.608607] [INFO ] [MAIN ] KEYS: Wiping existing primary data before full refresh. [2025-10-20T19:32:17.608702] [INFO ] [LOCAL ] KEYS: Starting deletion of primary data. [2025-10-20T19:32:17.608761] [DEBUG] [LOCAL ] KEYS: No primary data file found to delete. [2025-10-20T19:32:17.608795] [INFO ] [MAIN ] KEYS: Starting full primary refresh. [2025-10-20T19:32:17.856604] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-05. [2025-10-20T19:32:17.978112] [INFO ] [REMOTE ] KEYS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:18.066021] [INFO ] [REMOTE ] KEX: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T19:32:18.073604] [INFO ] [REMOTE ] KEX: Full fetch complete (56087 records). [2025-10-20T19:32:18.073840] [DEBUG] [VALIDATOR ] KEX: Validating primary data. [2025-10-20T19:32:18.199246] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T19:32:18.494330] [DEBUG] [VALIDATOR ] KEX: All expected trading days are present. [2025-10-20T19:32:18.494411] [DEBUG] [VALIDATOR ] KEX: Primary data validation passed. [2025-10-20T19:32:18.495229] [INFO ] [MAIN ] KEX: Fetched primary data is valid. [2025-10-20T19:32:18.495292] [DEBUG] [LOCAL ] KEX: Saving primary data. [2025-10-20T19:32:18.496377] [DEBUG] [LOCAL ] KEX: Trimmed 56087 records to 38824 (keeping 200 days of data) [2025-10-20T19:32:18.663571] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-01. [2025-10-20T19:32:18.735247] [DEBUG] [LOCAL ] KEX: Primary data saved (38824 records). [2025-10-20T19:32:18.735371] [INFO ] [MAIN ] KEX: Full history saved. [2025-10-20T19:32:18.735444] [INFO ] [MAIN ] Update Progress: 1180/2367 symbols completed. KEX took 22.2s. Estimated remaining: 2h 33m 9s. Rate: 7.8 symbols/min [2025-10-20T19:32:18.913040] [INFO ] [MAIN ] KF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:18.918157] [DEBUG] [LOCAL ] KF: Loaded 2385 existing records from primary storage. [2025-10-20T19:32:18.918501] [INFO ] [MAIN ] KF: Loaded 2385 existing records. [2025-10-20T19:32:19.111749] [WARN ] [MAIN ] KF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:19.111962] [INFO ] [MAIN ] KF: Wiping existing primary data before full refresh. [2025-10-20T19:32:19.112017] [INFO ] [LOCAL ] KF: Starting deletion of primary data. [2025-10-20T19:32:19.112191] [DEBUG] [LOCAL ] KF: Deleted primary data file. [2025-10-20T19:32:19.112247] [INFO ] [MAIN ] KF: Starting full primary refresh. [2025-10-20T19:32:19.462026] [INFO ] [REMOTE ] KEP: Fetching primary data for 2025-10. [2025-10-20T19:32:19.504146] [INFO ] [REMOTE ] KF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:19.742006] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T19:32:20.232119] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:32:20.251382] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-01. [2025-10-20T19:32:20.610720] [INFO ] [REMOTE ] KEP: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T19:32:20.619158] [INFO ] [REMOTE ] KEP: Full fetch complete (42358 records). [2025-10-20T19:32:20.619316] [DEBUG] [VALIDATOR ] KEP: Validating primary data. [2025-10-20T19:32:20.767277] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T19:32:21.008001] [DEBUG] [VALIDATOR ] KEP: All expected trading days are present. [2025-10-20T19:32:21.008110] [DEBUG] [VALIDATOR ] KEP: Primary data validation passed. [2025-10-20T19:32:21.008872] [INFO ] [MAIN ] KEP: Fetched primary data is valid. [2025-10-20T19:32:21.008934] [DEBUG] [LOCAL ] KEP: Saving primary data. [2025-10-20T19:32:21.009942] [DEBUG] [LOCAL ] KEP: Trimmed 42358 records to 32390 (keeping 200 days of data) [2025-10-20T19:32:21.056936] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-02. [2025-10-20T19:32:21.199265] [DEBUG] [LOCAL ] KEP: Primary data saved (32390 records). [2025-10-20T19:32:21.199405] [INFO ] [MAIN ] KEP: Full history saved. [2025-10-20T19:32:21.199484] [INFO ] [MAIN ] Update Progress: 1181/2367 symbols completed. KEP took 39.5s. Estimated remaining: 2h 32m 55s. Rate: 7.8 symbols/min [2025-10-20T19:32:21.367088] [INFO ] [MAIN ] KFRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:21.387816] [DEBUG] [LOCAL ] KFRC: Loaded 20885 existing records from primary storage. [2025-10-20T19:32:21.388056] [INFO ] [MAIN ] KFRC: Loaded 20885 existing records. [2025-10-20T19:32:21.571335] [WARN ] [MAIN ] KFRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:21.571540] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T19:32:21.573228] [INFO ] [MAIN ] KFRC: Wiping existing primary data before full refresh. [2025-10-20T19:32:21.573301] [INFO ] [LOCAL ] KFRC: Starting deletion of primary data. [2025-10-20T19:32:21.573616] [DEBUG] [LOCAL ] KFRC: Deleted primary data file. [2025-10-20T19:32:21.573698] [INFO ] [MAIN ] KFRC: Starting full primary refresh. [2025-10-20T19:32:21.856531] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-03. [2025-10-20T19:32:21.926476] [INFO ] [REMOTE ] KFRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:22.451492] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:32:22.651365] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-02. [2025-10-20T19:32:23.451427] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-04. [2025-10-20T19:32:24.049965] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T19:32:24.251370] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-05. [2025-10-20T19:32:24.302194] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:32:24.801662] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T19:32:25.051358] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-06. [2025-10-20T19:32:25.607327] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T19:32:25.851411] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-06. [2025-10-20T19:32:26.651375] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-07. [2025-10-20T19:32:27.098397] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:32:27.451396] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-08. [2025-10-20T19:32:27.672077] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:32:27.896936] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:32:28.251388] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-01. [2025-10-20T19:32:29.032622] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:32:29.051365] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-02. [2025-10-20T19:32:29.851384] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-03. [2025-10-20T19:32:29.960136] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T19:32:30.651395] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-07. [2025-10-20T19:32:31.451458] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-03. [2025-10-20T19:32:31.580421] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:32:32.251341] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-09. [2025-10-20T19:32:32.326549] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T19:32:32.533992] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-09 (0.28 seconds). [2025-10-20T19:32:32.579874] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T19:32:33.051430] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-04. [2025-10-20T19:32:33.851381] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-08. [2025-10-20T19:32:34.071756] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T19:32:34.651387] [INFO ] [REMOTE ] KF: Fetching primary data for 2025-10. [2025-10-20T19:32:35.229339] [INFO ] [REMOTE ] KF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:32:35.230257] [INFO ] [REMOTE ] KF: Full fetch complete (3519 records). [2025-10-20T19:32:35.230323] [DEBUG] [VALIDATOR ] KF: Validating primary data. [2025-10-20T19:32:35.459294] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-04. [2025-10-20T19:32:35.593180] [DEBUG] [VALIDATOR ] KF: All expected trading days are present. [2025-10-20T19:32:35.593330] [DEBUG] [VALIDATOR ] KF: Primary data validation passed. [2025-10-20T19:32:35.593482] [INFO ] [MAIN ] KF: Fetched primary data is valid. [2025-10-20T19:32:35.593564] [DEBUG] [LOCAL ] KF: Saving primary data. [2025-10-20T19:32:35.594163] [DEBUG] [LOCAL ] KF: Trimmed 3519 records to 2366 (keeping 200 days of data) [2025-10-20T19:32:35.607754] [DEBUG] [LOCAL ] KF: Primary data saved (2366 records). [2025-10-20T19:32:35.607814] [INFO ] [MAIN ] KF: Full history saved. [2025-10-20T19:32:35.607877] [INFO ] [MAIN ] Update Progress: 1182/2367 symbols completed. KF took 16.9s. Estimated remaining: 2h 32m 54s. Rate: 7.7 symbols/min [2025-10-20T19:32:35.814840] [INFO ] [MAIN ] KFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:35.817364] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T19:32:35.836381] [DEBUG] [LOCAL ] KFS: Loaded 8837 existing records from primary storage. [2025-10-20T19:32:35.836576] [INFO ] [MAIN ] KFS: Loaded 8837 existing records. [2025-10-20T19:32:36.011384] [WARN ] [MAIN ] KFS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:36.011580] [INFO ] [MAIN ] KFS: Wiping existing primary data before full refresh. [2025-10-20T19:32:36.011677] [INFO ] [LOCAL ] KFS: Starting deletion of primary data. [2025-10-20T19:32:36.011863] [DEBUG] [LOCAL ] KFS: Deleted primary data file. [2025-10-20T19:32:36.011917] [INFO ] [MAIN ] KFS: Starting full primary refresh. [2025-10-20T19:32:36.256564] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-05. [2025-10-20T19:32:36.376478] [INFO ] [REMOTE ] KFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:37.051415] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-09. [2025-10-20T19:32:37.361415] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:32:37.371377] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:32:37.851420] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-06. [2025-10-20T19:32:38.651402] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-01. [2025-10-20T19:32:38.740914] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:32:38.929491] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:32:39.303609] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:32:39.451375] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-07. [2025-10-20T19:32:40.251377] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-02. [2025-10-20T19:32:40.605780] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T19:32:40.817098] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T19:32:41.051379] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-03. [2025-10-20T19:32:41.640449] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:32:41.851384] [INFO ] [REMOTE ] KEY: Fetching primary data for 2025-10. [2025-10-20T19:32:42.651386] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-05. [2025-10-20T19:32:43.451447] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-04. [2025-10-20T19:32:43.727999] [INFO ] [REMOTE ] KEY: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T19:32:43.737931] [INFO ] [REMOTE ] KEY: Full fetch complete (77427 records). [2025-10-20T19:32:43.738109] [DEBUG] [VALIDATOR ] KEY: Validating primary data. [2025-10-20T19:32:43.973039] [WARN ] [VALIDATOR ] KEY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:32:43.974141] [FATAL] [MAIN ] KEY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:32:43.974225] [WARN ] [MAIN ] KEY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:32:43.974287] [INFO ] [LOCAL ] KEY: Starting deletion of primary data. [2025-10-20T19:32:43.974365] [DEBUG] [LOCAL ] KEY: No primary data file found to delete. [2025-10-20T19:32:43.974440] [INFO ] [LOCAL ] KEY: Starting deletion of derivative data. [2025-10-20T19:32:43.974520] [DEBUG] [LOCAL ] KEY: No derivative data file found to delete. [2025-10-20T19:32:43.974598] [INFO ] [MAIN ] Update Progress: 1183/2367 symbols completed. KEY took 43.4s. Estimated remaining: 2h 32m 47s. Rate: 7.7 symbols/min [2025-10-20T19:32:44.198841] [INFO ] [MAIN ] KFY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:44.239750] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:32:44.241590] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T19:32:44.254014] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-05. [2025-10-20T19:32:44.292851] [DEBUG] [LOCAL ] KFY: Loaded 30339 existing records from primary storage. [2025-10-20T19:32:44.293302] [INFO ] [MAIN ] KFY: Loaded 30339 existing records. [2025-10-20T19:32:44.477687] [WARN ] [MAIN ] KFY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:44.477917] [INFO ] [MAIN ] KFY: Wiping existing primary data before full refresh. [2025-10-20T19:32:44.477964] [INFO ] [LOCAL ] KFY: Starting deletion of primary data. [2025-10-20T19:32:44.478219] [DEBUG] [LOCAL ] KFY: Deleted primary data file. [2025-10-20T19:32:44.478257] [INFO ] [MAIN ] KFY: Starting full primary refresh. [2025-10-20T19:32:44.834817] [INFO ] [REMOTE ] KFY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:45.017544] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T19:32:45.051416] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-01. [2025-10-20T19:32:45.851352] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-06. [2025-10-20T19:32:46.257152] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T19:32:46.651394] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-02. [2025-10-20T19:32:47.264551] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:32:47.451389] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-07. [2025-10-20T19:32:47.937939] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:32:48.251370] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-08. [2025-10-20T19:32:48.950817] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:32:49.051345] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-08. [2025-10-20T19:32:49.363135] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:32:49.851370] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-06. [2025-10-20T19:32:50.586046] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:32:50.619253] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:32:50.651342] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-09. [2025-10-20T19:32:51.451381] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-07. [2025-10-20T19:32:52.252472] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-03. [2025-10-20T19:32:52.269222] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T19:32:52.276992] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T19:32:53.051367] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-08. [2025-10-20T19:32:53.711782] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:32:53.802100] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T19:32:53.851365] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-04. [2025-10-20T19:32:54.651371] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-09. [2025-10-20T19:32:55.012741] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:32:55.431437] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:32:55.451390] [INFO ] [REMOTE ] KFS: Fetching primary data for 2025-10. [2025-10-20T19:32:56.251410] [INFO ] [REMOTE ] KEYS: Fetching primary data for 2025-10. [2025-10-20T19:32:56.412736] [INFO ] [REMOTE ] KFS: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:32:56.414520] [INFO ] [REMOTE ] KFS: Full fetch complete (11131 records). [2025-10-20T19:32:56.414587] [DEBUG] [VALIDATOR ] KFS: Validating primary data. [2025-10-20T19:32:56.782135] [DEBUG] [VALIDATOR ] KFS: All expected trading days are present. [2025-10-20T19:32:56.782211] [DEBUG] [VALIDATOR ] KFS: Primary data validation passed. [2025-10-20T19:32:56.782470] [INFO ] [MAIN ] KFS: Fetched primary data is valid. [2025-10-20T19:32:56.782507] [DEBUG] [LOCAL ] KFS: Saving primary data. [2025-10-20T19:32:56.782974] [DEBUG] [LOCAL ] KFS: Trimmed 11131 records to 8699 (keeping 200 days of data) [2025-10-20T19:32:56.833920] [DEBUG] [LOCAL ] KFS: Primary data saved (8699 records). [2025-10-20T19:32:56.834061] [INFO ] [MAIN ] KFS: Full history saved. [2025-10-20T19:32:56.834139] [INFO ] [MAIN ] Update Progress: 1184/2367 symbols completed. KFS took 21.2s. Estimated remaining: 2h 32m 45s. Rate: 7.7 symbols/min [2025-10-20T19:32:57.013158] [INFO ] [MAIN ] KGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:57.065521] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-05. [2025-10-20T19:32:57.068609] [DEBUG] [LOCAL ] KGC: Loaded 54480 existing records from primary storage. [2025-10-20T19:32:57.069069] [INFO ] [MAIN ] KGC: Loaded 54480 existing records. [2025-10-20T19:32:57.257266] [WARN ] [MAIN ] KGC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:57.257507] [INFO ] [MAIN ] KGC: Wiping existing primary data before full refresh. [2025-10-20T19:32:57.257561] [INFO ] [LOCAL ] KGC: Starting deletion of primary data. [2025-10-20T19:32:57.257770] [DEBUG] [LOCAL ] KGC: Deleted primary data file. [2025-10-20T19:32:57.257827] [INFO ] [MAIN ] KGC: Starting full primary refresh. [2025-10-20T19:32:57.635523] [INFO ] [REMOTE ] KGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:57.851375] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-01. [2025-10-20T19:32:57.926810] [INFO ] [REMOTE ] KEYS: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:32:57.933241] [INFO ] [REMOTE ] KEYS: Full fetch complete (63139 records). [2025-10-20T19:32:57.933364] [DEBUG] [VALIDATOR ] KEYS: Validating primary data. [2025-10-20T19:32:58.331871] [DEBUG] [VALIDATOR ] KEYS: All expected trading days are present. [2025-10-20T19:32:58.331973] [DEBUG] [VALIDATOR ] KEYS: Primary data validation passed. [2025-10-20T19:32:58.333010] [INFO ] [MAIN ] KEYS: Fetched primary data is valid. [2025-10-20T19:32:58.333089] [DEBUG] [LOCAL ] KEYS: Saving primary data. [2025-10-20T19:32:58.334385] [DEBUG] [LOCAL ] KEYS: Trimmed 63139 records to 43295 (keeping 200 days of data) [2025-10-20T19:32:58.453294] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:32:58.612104] [DEBUG] [LOCAL ] KEYS: Primary data saved (43295 records). [2025-10-20T19:32:58.612232] [INFO ] [MAIN ] KEYS: Full history saved. [2025-10-20T19:32:58.612298] [INFO ] [MAIN ] Update Progress: 1185/2367 symbols completed. KEYS took 41.2s. Estimated remaining: 2h 32m 31s. Rate: 7.7 symbols/min [2025-10-20T19:32:58.656589] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-09. [2025-10-20T19:32:58.781853] [INFO ] [MAIN ] KGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:32:58.822859] [DEBUG] [LOCAL ] KGS: Loaded 46744 existing records from primary storage. [2025-10-20T19:32:58.823146] [INFO ] [MAIN ] KGS: Loaded 46744 existing records. [2025-10-20T19:32:58.999803] [WARN ] [MAIN ] KGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:32:59.000058] [INFO ] [MAIN ] KGS: Wiping existing primary data before full refresh. [2025-10-20T19:32:59.000107] [INFO ] [LOCAL ] KGS: Starting deletion of primary data. [2025-10-20T19:32:59.000377] [DEBUG] [LOCAL ] KGS: Deleted primary data file. [2025-10-20T19:32:59.000443] [INFO ] [MAIN ] KGS: Starting full primary refresh. [2025-10-20T19:32:59.381130] [INFO ] [REMOTE ] KGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:32:59.451342] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-01. [2025-10-20T19:32:59.664803] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:32:59.821100] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T19:33:00.251399] [INFO ] [REMOTE ] KFRC: Fetching primary data for 2025-10. [2025-10-20T19:33:01.051362] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-06. [2025-10-20T19:33:01.279018] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:33:01.330516] [INFO ] [REMOTE ] KFRC: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:33:01.334408] [INFO ] [REMOTE ] KFRC: Full fetch complete (26845 records). [2025-10-20T19:33:01.334483] [DEBUG] [VALIDATOR ] KFRC: Validating primary data. [2025-10-20T19:33:01.722501] [DEBUG] [VALIDATOR ] KFRC: All expected trading days are present. [2025-10-20T19:33:01.722619] [DEBUG] [VALIDATOR ] KFRC: Primary data validation passed. [2025-10-20T19:33:01.723385] [INFO ] [MAIN ] KFRC: Fetched primary data is valid. [2025-10-20T19:33:01.723445] [DEBUG] [LOCAL ] KFRC: Saving primary data. [2025-10-20T19:33:01.724205] [DEBUG] [LOCAL ] KFRC: Trimmed 26845 records to 20727 (keeping 200 days of data) [2025-10-20T19:33:01.845985] [DEBUG] [LOCAL ] KFRC: Primary data saved (20727 records). [2025-10-20T19:33:01.846119] [INFO ] [MAIN ] KFRC: Full history saved. [2025-10-20T19:33:01.846198] [INFO ] [MAIN ] Update Progress: 1186/2367 symbols completed. KFRC took 40.6s. Estimated remaining: 2h 32m 19s. Rate: 7.8 symbols/min [2025-10-20T19:33:01.857399] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-02. [2025-10-20T19:33:02.017854] [INFO ] [MAIN ] KIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:02.067376] [DEBUG] [LOCAL ] KIM: Loaded 53783 existing records from primary storage. [2025-10-20T19:33:02.067714] [INFO ] [MAIN ] KIM: Loaded 53783 existing records. [2025-10-20T19:33:02.253469] [WARN ] [MAIN ] KIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:02.253849] [INFO ] [MAIN ] KIM: Wiping existing primary data before full refresh. [2025-10-20T19:33:02.253913] [INFO ] [LOCAL ] KIM: Starting deletion of primary data. [2025-10-20T19:33:02.254092] [DEBUG] [LOCAL ] KIM: Deleted primary data file. [2025-10-20T19:33:02.254131] [INFO ] [MAIN ] KIM: Starting full primary refresh. [2025-10-20T19:33:02.533855] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T19:33:02.645711] [INFO ] [REMOTE ] KIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:02.651472] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-01. [2025-10-20T19:33:03.221890] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T19:33:03.451386] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-07. [2025-10-20T19:33:04.251432] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-03. [2025-10-20T19:33:04.456943] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T19:33:04.757181] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:33:05.051476] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-08. [2025-10-20T19:33:05.851410] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-02. [2025-10-20T19:33:05.909936] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:33:06.159963] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T19:33:06.651383] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-02. [2025-10-20T19:33:07.451354] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-04. [2025-10-20T19:33:07.550342] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T19:33:08.251421] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-09. [2025-10-20T19:33:08.524752] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T19:33:09.051393] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-03. [2025-10-20T19:33:09.095520] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:33:09.609603] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:33:09.851457] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-05. [2025-10-20T19:33:10.651409] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-03. [2025-10-20T19:33:10.926903] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:33:11.448161] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:33:11.451441] [INFO ] [REMOTE ] KFY: Fetching primary data for 2025-10. [2025-10-20T19:33:12.251439] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-04. [2025-10-20T19:33:12.546829] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T19:33:13.051419] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-06. [2025-10-20T19:33:13.084572] [INFO ] [REMOTE ] KFY: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T19:33:13.089073] [INFO ] [REMOTE ] KFY: Full fetch complete (43311 records). [2025-10-20T19:33:13.089153] [DEBUG] [VALIDATOR ] KFY: Validating primary data. [2025-10-20T19:33:13.466183] [DEBUG] [VALIDATOR ] KFY: All expected trading days are present. [2025-10-20T19:33:13.466268] [DEBUG] [VALIDATOR ] KFY: Primary data validation passed. [2025-10-20T19:33:13.467006] [INFO ] [MAIN ] KFY: Fetched primary data is valid. [2025-10-20T19:33:13.467069] [DEBUG] [LOCAL ] KFY: Saving primary data. [2025-10-20T19:33:13.468103] [DEBUG] [LOCAL ] KFY: Trimmed 43311 records to 29744 (keeping 200 days of data) [2025-10-20T19:33:13.648317] [DEBUG] [LOCAL ] KFY: Primary data saved (29744 records). [2025-10-20T19:33:13.648461] [INFO ] [MAIN ] KFY: Full history saved. [2025-10-20T19:33:13.648541] [INFO ] [MAIN ] Update Progress: 1187/2367 symbols completed. KFY took 29.7s. Estimated remaining: 2h 32m 15s. Rate: 7.8 symbols/min [2025-10-20T19:33:13.840358] [INFO ] [MAIN ] KIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:13.854489] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-04. [2025-10-20T19:33:13.865493] [DEBUG] [LOCAL ] KIO: Loaded 22090 existing records from primary storage. [2025-10-20T19:33:13.865738] [INFO ] [MAIN ] KIO: Loaded 22090 existing records. [2025-10-20T19:33:14.042979] [WARN ] [MAIN ] KIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:14.043179] [INFO ] [MAIN ] KIO: Wiping existing primary data before full refresh. [2025-10-20T19:33:14.043232] [INFO ] [LOCAL ] KIO: Starting deletion of primary data. [2025-10-20T19:33:14.043539] [DEBUG] [LOCAL ] KIO: Deleted primary data file. [2025-10-20T19:33:14.043632] [INFO ] [MAIN ] KIO: Starting full primary refresh. [2025-10-20T19:33:14.153357] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:33:14.455110] [INFO ] [REMOTE ] KIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:14.519780] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:33:14.651374] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-05. [2025-10-20T19:33:15.451428] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-01. [2025-10-20T19:33:16.024148] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T19:33:16.251364] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-05. [2025-10-20T19:33:16.442538] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:33:16.549045] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:33:17.051375] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-07. [2025-10-20T19:33:17.851363] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-06. [2025-10-20T19:33:18.392886] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T19:33:18.432464] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T19:33:18.651364] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-08. [2025-10-20T19:33:19.124318] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T19:33:19.451354] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-02. [2025-10-20T19:33:20.251433] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-07. [2025-10-20T19:33:20.296109] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:33:20.662044] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T19:33:21.051421] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-06. [2025-10-20T19:33:21.851388] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-03. [2025-10-20T19:33:21.852633] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:33:22.651358] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-09. [2025-10-20T19:33:22.932169] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:33:23.045788] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T19:33:23.451393] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-04. [2025-10-20T19:33:24.152795] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T19:33:24.251335] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-08. [2025-10-20T19:33:24.491956] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:33:25.051411] [INFO ] [REMOTE ] KGS: Fetching primary data for 2025-10. [2025-10-20T19:33:25.851370] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-07. [2025-10-20T19:33:26.057529] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:33:26.651448] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-09. [2025-10-20T19:33:26.753141] [INFO ] [REMOTE ] KGS: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T19:33:26.762708] [INFO ] [REMOTE ] KGS: Full fetch complete (66516 records). [2025-10-20T19:33:26.762865] [DEBUG] [VALIDATOR ] KGS: Validating primary data. [2025-10-20T19:33:27.175920] [DEBUG] [VALIDATOR ] KGS: All expected trading days are present. [2025-10-20T19:33:27.176001] [DEBUG] [VALIDATOR ] KGS: Primary data validation passed. [2025-10-20T19:33:27.176889] [INFO ] [MAIN ] KGS: Fetched primary data is valid. [2025-10-20T19:33:27.176951] [DEBUG] [LOCAL ] KGS: Saving primary data. [2025-10-20T19:33:27.178221] [DEBUG] [LOCAL ] KGS: Trimmed 66516 records to 46120 (keeping 200 days of data) [2025-10-20T19:33:27.448751] [DEBUG] [LOCAL ] KGS: Primary data saved (46120 records). [2025-10-20T19:33:27.448898] [INFO ] [MAIN ] KGS: Full history saved. [2025-10-20T19:33:27.448974] [INFO ] [MAIN ] Update Progress: 1188/2367 symbols completed. KGS took 28.8s. Estimated remaining: 2h 32m 13s. Rate: 7.7 symbols/min [2025-10-20T19:33:27.460193] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-05. [2025-10-20T19:33:27.623203] [INFO ] [MAIN ] KKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:27.677875] [DEBUG] [LOCAL ] KKR: Loaded 54283 existing records from primary storage. [2025-10-20T19:33:27.678213] [INFO ] [MAIN ] KKR: Loaded 54283 existing records. [2025-10-20T19:33:27.861317] [WARN ] [MAIN ] KKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:27.861573] [INFO ] [MAIN ] KKR: Wiping existing primary data before full refresh. [2025-10-20T19:33:27.861668] [INFO ] [LOCAL ] KKR: Starting deletion of primary data. [2025-10-20T19:33:27.861914] [DEBUG] [LOCAL ] KKR: Deleted primary data file. [2025-10-20T19:33:27.861977] [INFO ] [MAIN ] KKR: Starting full primary refresh. [2025-10-20T19:33:28.264353] [INFO ] [REMOTE ] KKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:28.264836] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-01. [2025-10-20T19:33:28.267836] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-07 (2.42 seconds). [2025-10-20T19:33:28.314441] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:33:28.565240] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:33:29.051370] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-06. [2025-10-20T19:33:29.851355] [INFO ] [REMOTE ] KIM: Fetching primary data for 2025-10. [2025-10-20T19:33:30.072280] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:33:30.116845] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:33:30.651398] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-08. [2025-10-20T19:33:31.454267] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-07. [2025-10-20T19:33:31.457156] [INFO ] [REMOTE ] KIM: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T19:33:31.465851] [INFO ] [REMOTE ] KIM: Full fetch complete (76978 records). [2025-10-20T19:33:31.465992] [DEBUG] [VALIDATOR ] KIM: Validating primary data. [2025-10-20T19:33:31.865672] [DEBUG] [VALIDATOR ] KIM: All expected trading days are present. [2025-10-20T19:33:31.865760] [DEBUG] [VALIDATOR ] KIM: Primary data validation passed. [2025-10-20T19:33:31.866741] [INFO ] [MAIN ] KIM: Fetched primary data is valid. [2025-10-20T19:33:31.866803] [DEBUG] [LOCAL ] KIM: Saving primary data. [2025-10-20T19:33:31.868223] [DEBUG] [LOCAL ] KIM: Trimmed 76978 records to 53012 (keeping 200 days of data) [2025-10-20T19:33:32.196129] [DEBUG] [LOCAL ] KIM: Primary data saved (53012 records). [2025-10-20T19:33:32.196243] [INFO ] [MAIN ] KIM: Full history saved. [2025-10-20T19:33:32.196313] [INFO ] [MAIN ] Update Progress: 1189/2367 symbols completed. KIM took 30.4s. Estimated remaining: 2h 32m 3s. Rate: 7.7 symbols/min [2025-10-20T19:33:32.259827] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-02. [2025-10-20T19:33:32.379941] [INFO ] [MAIN ] KKRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:32.388151] [DEBUG] [LOCAL ] KKRS: Loaded 5669 existing records from primary storage. [2025-10-20T19:33:32.388387] [INFO ] [MAIN ] KKRS: Loaded 5669 existing records. [2025-10-20T19:33:32.584057] [WARN ] [MAIN ] KKRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:32.584978] [INFO ] [MAIN ] KKRS: Wiping existing primary data before full refresh. [2025-10-20T19:33:32.585063] [INFO ] [LOCAL ] KKRS: Starting deletion of primary data. [2025-10-20T19:33:32.585267] [DEBUG] [LOCAL ] KKRS: Deleted primary data file. [2025-10-20T19:33:32.585347] [INFO ] [MAIN ] KKRS: Starting full primary refresh. [2025-10-20T19:33:32.716136] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T19:33:32.981084] [INFO ] [REMOTE ] KKRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:33.012754] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-08 (2.36 seconds). [2025-10-20T19:33:33.051344] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-09. [2025-10-20T19:33:33.851427] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-08. [2025-10-20T19:33:34.049127] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:33:34.651360] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-01. [2025-10-20T19:33:34.928546] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:33:35.218893] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:33:35.275922] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T19:33:35.451367] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-02. [2025-10-20T19:33:36.031179] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T19:33:36.251366] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-09. [2025-10-20T19:33:37.051449] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-03. [2025-10-20T19:33:37.316383] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:33:37.598226] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:33:37.851372] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-03. [2025-10-20T19:33:38.651406] [INFO ] [REMOTE ] KIO: Fetching primary data for 2025-10. [2025-10-20T19:33:39.451385] [INFO ] [REMOTE ] KGC: Fetching primary data for 2025-10. [2025-10-20T19:33:39.834812] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T19:33:39.963963] [INFO ] [REMOTE ] KIO: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:33:39.968819] [INFO ] [REMOTE ] KIO: Full fetch complete (32242 records). [2025-10-20T19:33:39.968993] [DEBUG] [VALIDATOR ] KIO: Validating primary data. [2025-10-20T19:33:40.256949] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-04. [2025-10-20T19:33:40.378137] [DEBUG] [VALIDATOR ] KIO: All expected trading days are present. [2025-10-20T19:33:40.378237] [DEBUG] [VALIDATOR ] KIO: Primary data validation passed. [2025-10-20T19:33:40.379288] [INFO ] [MAIN ] KIO: Fetched primary data is valid. [2025-10-20T19:33:40.379374] [DEBUG] [LOCAL ] KIO: Saving primary data. [2025-10-20T19:33:40.380518] [DEBUG] [LOCAL ] KIO: Trimmed 32242 records to 21766 (keeping 200 days of data) [2025-10-20T19:33:40.512860] [DEBUG] [LOCAL ] KIO: Primary data saved (21766 records). [2025-10-20T19:33:40.512998] [INFO ] [MAIN ] KIO: Full history saved. [2025-10-20T19:33:40.513078] [INFO ] [MAIN ] Update Progress: 1190/2367 symbols completed. KIO took 26.9s. Estimated remaining: 2h 31m 55s. Rate: 7.7 symbols/min [2025-10-20T19:33:40.695545] [INFO ] [MAIN ] KKRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:40.695753] [DEBUG] [LOCAL ] KKRT: No existing data file found in primary storage. [2025-10-20T19:33:40.695800] [WARN ] [MAIN ] KKRT: No existing data found, performing full refresh instead. [2025-10-20T19:33:40.695937] [INFO ] [MAIN ] KKRT: Wiping existing primary data before full refresh. [2025-10-20T19:33:40.695979] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-10-20T19:33:40.696023] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-10-20T19:33:40.696055] [INFO ] [MAIN ] KKRT: Starting full primary refresh. [2025-10-20T19:33:41.072499] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-04. [2025-10-20T19:33:41.077249] [INFO ] [REMOTE ] KKRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:41.108174] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T19:33:41.724022] [INFO ] [REMOTE ] KGC: Incremental fetch complete for 2025-10 (2.27 seconds). [2025-10-20T19:33:41.732457] [INFO ] [REMOTE ] KGC: Full fetch complete (77873 records). [2025-10-20T19:33:41.732615] [DEBUG] [VALIDATOR ] KGC: Validating primary data. [2025-10-20T19:33:41.856524] [INFO ] [REMOTE ] KKRT: Fetching primary data for 2025-01. [2025-10-20T19:33:42.131741] [DEBUG] [REMOTE ] KKRT: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:33:42.135611] [DEBUG] [VALIDATOR ] KGC: All expected trading days are present. [2025-10-20T19:33:42.135891] [DEBUG] [VALIDATOR ] KGC: Primary data validation passed. [2025-10-20T19:33:42.137126] [INFO ] [MAIN ] KGC: Fetched primary data is valid. [2025-10-20T19:33:42.136006] [WARN ] [REMOTE ] KKRT: No data returned for month 2025-01 [2025-10-20T19:33:42.137378] [DEBUG] [LOCAL ] KGC: Saving primary data. [2025-10-20T19:33:42.137316] [FATAL] [REMOTE ] KKRT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:33:42.137755] [FATAL] [MAIN ] KKRT: Failed to fetch primary data. [2025-10-20T19:33:42.137827] [WARN ] [MAIN ] KKRT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:33:42.137894] [INFO ] [LOCAL ] KKRT: Starting deletion of primary data. [2025-10-20T19:33:42.137971] [DEBUG] [LOCAL ] KKRT: No primary data file found to delete. [2025-10-20T19:33:42.138026] [INFO ] [LOCAL ] KKRT: Starting deletion of derivative data. [2025-10-20T19:33:42.138522] [DEBUG] [LOCAL ] KKRT: No derivative data file found to delete. [2025-10-20T19:33:42.138724] [INFO ] [MAIN ] Update Progress: 1191/2367 symbols completed. KKRT took 1.6s. Estimated remaining: 2h 31m 42s. Rate: 7.8 symbols/min [2025-10-20T19:33:42.215513] [DEBUG] [LOCAL ] KGC: Trimmed 77873 records to 53700 (keeping 200 days of data) [2025-10-20T19:33:42.417996] [INFO ] [MAIN ] KLAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:42.422058] [DEBUG] [LOCAL ] KLAR: No existing data file found in primary storage. [2025-10-20T19:33:42.424810] [WARN ] [MAIN ] KLAR: No existing data found, performing full refresh instead. [2025-10-20T19:33:42.427733] [INFO ] [MAIN ] KLAR: Wiping existing primary data before full refresh. [2025-10-20T19:33:42.428986] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-10-20T19:33:42.429787] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-10-20T19:33:42.436438] [INFO ] [MAIN ] KLAR: Starting full primary refresh. [2025-10-20T19:33:42.653039] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-05. [2025-10-20T19:33:42.923149] [INFO ] [REMOTE ] KLAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:43.016508] [DEBUG] [LOCAL ] KGC: Primary data saved (53700 records). [2025-10-20T19:33:43.016765] [INFO ] [MAIN ] KGC: Full history saved. [2025-10-20T19:33:43.016896] [INFO ] [MAIN ] Update Progress: 1192/2367 symbols completed. KGC took 46.2s. Estimated remaining: 2h 31m 27s. Rate: 7.8 symbols/min [2025-10-20T19:33:43.205677] [INFO ] [MAIN ] KLC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:43.220313] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T19:33:43.263564] [DEBUG] [LOCAL ] KLC: Loaded 37016 existing records from primary storage. [2025-10-20T19:33:43.263906] [INFO ] [MAIN ] KLC: Loaded 37016 existing records. [2025-10-20T19:33:43.440955] [WARN ] [MAIN ] KLC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:43.441138] [INFO ] [MAIN ] KLC: Wiping existing primary data before full refresh. [2025-10-20T19:33:43.441183] [INFO ] [LOCAL ] KLC: Starting deletion of primary data. [2025-10-20T19:33:43.441496] [DEBUG] [LOCAL ] KLC: Deleted primary data file. [2025-10-20T19:33:43.441551] [INFO ] [MAIN ] KLC: Starting full primary refresh. [2025-10-20T19:33:43.456607] [INFO ] [REMOTE ] KLAR: Fetching primary data for 2025-01. [2025-10-20T19:33:43.530217] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T19:33:43.813496] [INFO ] [REMOTE ] KLC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:43.913409] [DEBUG] [REMOTE ] KLAR: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:33:43.913683] [WARN ] [REMOTE ] KLAR: No data returned for month 2025-01 [2025-10-20T19:33:43.913768] [FATAL] [REMOTE ] KLAR: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:33:43.913814] [FATAL] [MAIN ] KLAR: Failed to fetch primary data. [2025-10-20T19:33:43.913888] [WARN ] [MAIN ] KLAR: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:33:43.913958] [INFO ] [LOCAL ] KLAR: Starting deletion of primary data. [2025-10-20T19:33:43.914042] [DEBUG] [LOCAL ] KLAR: No primary data file found to delete. [2025-10-20T19:33:43.914096] [INFO ] [LOCAL ] KLAR: Starting deletion of derivative data. [2025-10-20T19:33:43.914138] [DEBUG] [LOCAL ] KLAR: No derivative data file found to delete. [2025-10-20T19:33:43.914181] [INFO ] [MAIN ] Update Progress: 1193/2367 symbols completed. KLAR took 1.8s. Estimated remaining: 2h 31m 13s. Rate: 7.8 symbols/min [2025-10-20T19:33:44.081168] [INFO ] [MAIN ] KMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:44.133159] [DEBUG] [LOCAL ] KMI: Loaded 54473 existing records from primary storage. [2025-10-20T19:33:44.133460] [INFO ] [MAIN ] KMI: Loaded 54473 existing records. [2025-10-20T19:33:44.256555] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-05. [2025-10-20T19:33:44.314098] [WARN ] [MAIN ] KMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:44.314264] [INFO ] [MAIN ] KMI: Wiping existing primary data before full refresh. [2025-10-20T19:33:44.314309] [INFO ] [LOCAL ] KMI: Starting deletion of primary data. [2025-10-20T19:33:44.314808] [DEBUG] [LOCAL ] KMI: Deleted primary data file. [2025-10-20T19:33:44.314862] [INFO ] [MAIN ] KMI: Starting full primary refresh. [2025-10-20T19:33:44.672849] [INFO ] [REMOTE ] KMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:45.051417] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-01. [2025-10-20T19:33:45.851430] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-06. [2025-10-20T19:33:46.370564] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T19:33:46.505074] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T19:33:46.547931] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T19:33:46.651342] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-07. [2025-10-20T19:33:47.248588] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T19:33:47.451401] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-08. [2025-10-20T19:33:48.030616] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:33:48.251388] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-06. [2025-10-20T19:33:49.051391] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-09. [2025-10-20T19:33:49.624117] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:33:49.851345] [INFO ] [REMOTE ] KKRS: Fetching primary data for 2025-10. [2025-10-20T19:33:50.142120] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:33:50.555741] [INFO ] [REMOTE ] KKRS: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T19:33:50.556774] [INFO ] [REMOTE ] KKRS: Full fetch complete (7562 records). [2025-10-20T19:33:50.556839] [DEBUG] [VALIDATOR ] KKRS: Validating primary data. [2025-10-20T19:33:50.656683] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-02. [2025-10-20T19:33:50.911916] [DEBUG] [VALIDATOR ] KKRS: All expected trading days are present. [2025-10-20T19:33:50.911990] [DEBUG] [VALIDATOR ] KKRS: Primary data validation passed. [2025-10-20T19:33:50.912221] [INFO ] [MAIN ] KKRS: Fetched primary data is valid. [2025-10-20T19:33:50.912264] [DEBUG] [LOCAL ] KKRS: Saving primary data. [2025-10-20T19:33:50.912662] [DEBUG] [LOCAL ] KKRS: Trimmed 7562 records to 5590 (keeping 200 days of data) [2025-10-20T19:33:50.945511] [DEBUG] [LOCAL ] KKRS: Primary data saved (5590 records). [2025-10-20T19:33:50.945596] [INFO ] [MAIN ] KKRS: Full history saved. [2025-10-20T19:33:50.945686] [INFO ] [MAIN ] Update Progress: 1194/2367 symbols completed. KKRS took 18.7s. Estimated remaining: 2h 31m 4s. Rate: 7.8 symbols/min [2025-10-20T19:33:51.114757] [INFO ] [MAIN ] KMPB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:33:51.121258] [DEBUG] [LOCAL ] KMPB: Loaded 3680 existing records from primary storage. [2025-10-20T19:33:51.121447] [INFO ] [MAIN ] KMPB: Loaded 3680 existing records. [2025-10-20T19:33:51.298461] [WARN ] [MAIN ] KMPB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:33:51.298744] [INFO ] [MAIN ] KMPB: Wiping existing primary data before full refresh. [2025-10-20T19:33:51.298806] [INFO ] [LOCAL ] KMPB: Starting deletion of primary data. [2025-10-20T19:33:51.298971] [DEBUG] [LOCAL ] KMPB: Deleted primary data file. [2025-10-20T19:33:51.299029] [INFO ] [MAIN ] KMPB: Starting full primary refresh. [2025-10-20T19:33:51.460701] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-01. [2025-10-20T19:33:51.660917] [INFO ] [REMOTE ] KMPB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:33:51.753453] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T19:33:52.251596] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-03. [2025-10-20T19:33:53.051383] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-01. [2025-10-20T19:33:53.807332] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:33:53.809971] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:33:53.845914] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-01 (2.39 seconds). [2025-10-20T19:33:53.855985] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-04. [2025-10-20T19:33:54.651376] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-07. [2025-10-20T19:33:55.377840] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:33:55.451373] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-02. [2025-10-20T19:33:56.251371] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-05. [2025-10-20T19:33:56.663828] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:33:57.051395] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-02. [2025-10-20T19:33:57.413965] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T19:33:57.496194] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:33:57.704841] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:33:57.851367] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-03. [2025-10-20T19:33:58.651377] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-06. [2025-10-20T19:33:59.451365] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-08. [2025-10-20T19:33:59.934317] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:33:59.981414] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T19:34:00.251410] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-03. [2025-10-20T19:34:00.918371] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:34:01.051390] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-04. [2025-10-20T19:34:01.586553] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T19:34:01.635062] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T19:34:01.851366] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-04. [2025-10-20T19:34:02.651419] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-07. [2025-10-20T19:34:03.451415] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-09. [2025-10-20T19:34:03.963986] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-10-20T19:34:04.078269] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:34:04.251403] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-08. [2025-10-20T19:34:05.044634] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:34:05.055107] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-05. [2025-10-20T19:34:05.713544] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:34:05.851369] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-09. [2025-10-20T19:34:06.651372] [INFO ] [REMOTE ] KKR: Fetching primary data for 2025-10. [2025-10-20T19:34:06.694828] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:34:07.242989] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:34:07.451439] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-06. [2025-10-20T19:34:08.251375] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-05. [2025-10-20T19:34:08.540070] [INFO ] [REMOTE ] KKR: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:34:08.548076] [INFO ] [REMOTE ] KKR: Full fetch complete (77648 records). [2025-10-20T19:34:08.548208] [DEBUG] [VALIDATOR ] KKR: Validating primary data. [2025-10-20T19:34:08.842105] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:34:08.966883] [DEBUG] [VALIDATOR ] KKR: All expected trading days are present. [2025-10-20T19:34:08.966960] [DEBUG] [VALIDATOR ] KKR: Primary data validation passed. [2025-10-20T19:34:08.967921] [INFO ] [MAIN ] KKR: Fetched primary data is valid. [2025-10-20T19:34:08.967981] [DEBUG] [LOCAL ] KKR: Saving primary data. [2025-10-20T19:34:08.969323] [DEBUG] [LOCAL ] KKR: Trimmed 77648 records to 53503 (keeping 200 days of data) [2025-10-20T19:34:09.058200] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-06. [2025-10-20T19:34:09.343447] [DEBUG] [LOCAL ] KKR: Primary data saved (53503 records). [2025-10-20T19:34:09.344237] [INFO ] [MAIN ] KKR: Full history saved. [2025-10-20T19:34:09.345456] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:34:09.346130] [INFO ] [MAIN ] Update Progress: 1195/2367 symbols completed. KKR took 41.9s. Estimated remaining: 2h 31m 7s. Rate: 7.8 symbols/min [2025-10-20T19:34:09.528388] [INFO ] [MAIN ] KMPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:09.567132] [DEBUG] [LOCAL ] KMPR: Loaded 40730 existing records from primary storage. [2025-10-20T19:34:09.567398] [INFO ] [MAIN ] KMPR: Loaded 40730 existing records. [2025-10-20T19:34:09.677349] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T19:34:09.750834] [WARN ] [MAIN ] KMPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:09.751034] [INFO ] [MAIN ] KMPR: Wiping existing primary data before full refresh. [2025-10-20T19:34:09.751140] [INFO ] [LOCAL ] KMPR: Starting deletion of primary data. [2025-10-20T19:34:09.751320] [DEBUG] [LOCAL ] KMPR: Deleted primary data file. [2025-10-20T19:34:09.751359] [INFO ] [MAIN ] KMPR: Starting full primary refresh. [2025-10-20T19:34:09.854436] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-07. [2025-10-20T19:34:10.147146] [INFO ] [REMOTE ] KMPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:10.651378] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-01. [2025-10-20T19:34:11.451355] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-07. [2025-10-20T19:34:11.927475] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T19:34:11.998515] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T19:34:12.077835] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T19:34:12.251413] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-02. [2025-10-20T19:34:13.051406] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-08. [2025-10-20T19:34:13.233681] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:34:13.628176] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:34:13.851376] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-08. [2025-10-20T19:34:14.651365] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-09. [2025-10-20T19:34:15.237540] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:34:15.451366] [INFO ] [REMOTE ] KLC: Fetching primary data for 2025-10. [2025-10-20T19:34:15.573144] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:34:16.251435] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-03. [2025-10-20T19:34:16.835506] [INFO ] [REMOTE ] KLC: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T19:34:16.841209] [INFO ] [REMOTE ] KLC: Full fetch complete (50521 records). [2025-10-20T19:34:16.841305] [DEBUG] [VALIDATOR ] KLC: Validating primary data. [2025-10-20T19:34:17.060404] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-09. [2025-10-20T19:34:17.225302] [DEBUG] [VALIDATOR ] KLC: All expected trading days are present. [2025-10-20T19:34:17.225480] [DEBUG] [VALIDATOR ] KLC: Primary data validation passed. [2025-10-20T19:34:17.226462] [INFO ] [MAIN ] KLC: Fetched primary data is valid. [2025-10-20T19:34:17.226523] [DEBUG] [LOCAL ] KLC: Saving primary data. [2025-10-20T19:34:17.227504] [DEBUG] [LOCAL ] KLC: Trimmed 50521 records to 36293 (keeping 200 days of data) [2025-10-20T19:34:17.425932] [DEBUG] [LOCAL ] KLC: Primary data saved (36293 records). [2025-10-20T19:34:17.426062] [INFO ] [MAIN ] KLC: Full history saved. [2025-10-20T19:34:17.426140] [INFO ] [MAIN ] Update Progress: 1196/2367 symbols completed. KLC took 34.4s. Estimated remaining: 2h 30m 60s. Rate: 7.8 symbols/min [2025-10-20T19:34:17.596429] [INFO ] [MAIN ] KMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:17.615241] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:34:17.658308] [DEBUG] [LOCAL ] KMT: Loaded 43532 existing records from primary storage. [2025-10-20T19:34:17.658639] [INFO ] [MAIN ] KMT: Loaded 43532 existing records. [2025-10-20T19:34:17.834033] [WARN ] [MAIN ] KMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:17.834281] [INFO ] [MAIN ] KMT: Wiping existing primary data before full refresh. [2025-10-20T19:34:17.834357] [INFO ] [LOCAL ] KMT: Starting deletion of primary data. [2025-10-20T19:34:17.834569] [DEBUG] [LOCAL ] KMT: Deleted primary data file. [2025-10-20T19:34:17.834629] [INFO ] [MAIN ] KMT: Starting full primary refresh. [2025-10-20T19:34:17.856606] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-04. [2025-10-20T19:34:18.196111] [INFO ] [REMOTE ] KMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:18.651364] [INFO ] [REMOTE ] KMPB: Fetching primary data for 2025-10. [2025-10-20T19:34:18.949105] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:34:19.399032] [INFO ] [REMOTE ] KMPB: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T19:34:19.400183] [INFO ] [REMOTE ] KMPB: Full fetch complete (4969 records). [2025-10-20T19:34:19.400255] [DEBUG] [VALIDATOR ] KMPB: Validating primary data. [2025-10-20T19:34:19.469939] [INFO ] [REMOTE ] KMI: Fetching primary data for 2025-10. [2025-10-20T19:34:19.587054] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:34:19.804600] [DEBUG] [VALIDATOR ] KMPB: All expected trading days are present. [2025-10-20T19:34:19.804701] [DEBUG] [VALIDATOR ] KMPB: Primary data validation passed. [2025-10-20T19:34:19.804858] [INFO ] [MAIN ] KMPB: Fetched primary data is valid. [2025-10-20T19:34:19.804896] [DEBUG] [LOCAL ] KMPB: Saving primary data. [2025-10-20T19:34:19.805269] [DEBUG] [LOCAL ] KMPB: Trimmed 4969 records to 3558 (keeping 200 days of data) [2025-10-20T19:34:19.826448] [DEBUG] [LOCAL ] KMPB: Primary data saved (3558 records). [2025-10-20T19:34:19.826572] [INFO ] [MAIN ] KMPB: Full history saved. [2025-10-20T19:34:19.826671] [INFO ] [MAIN ] Update Progress: 1197/2367 symbols completed. KMPB took 28.9s. Estimated remaining: 2h 30m 47s. Rate: 7.8 symbols/min [2025-10-20T19:34:20.251434] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-05. [2025-10-20T19:34:20.389056] [INFO ] [REMOTE ] KMI: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:34:20.396964] [INFO ] [REMOTE ] KMI: Full fetch complete (77873 records). [2025-10-20T19:34:20.397090] [DEBUG] [VALIDATOR ] KMI: Validating primary data. [2025-10-20T19:34:20.779978] [DEBUG] [VALIDATOR ] KMI: All expected trading days are present. [2025-10-20T19:34:20.780074] [DEBUG] [VALIDATOR ] KMI: Primary data validation passed. [2025-10-20T19:34:20.781053] [INFO ] [MAIN ] KMI: Fetched primary data is valid. [2025-10-20T19:34:20.781114] [DEBUG] [LOCAL ] KMI: Saving primary data. [2025-10-20T19:34:20.782535] [DEBUG] [LOCAL ] KMI: Trimmed 77873 records to 53693 (keeping 200 days of data) [2025-10-20T19:34:21.057518] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-01. [2025-10-20T19:34:21.100371] [DEBUG] [LOCAL ] KMI: Primary data saved (53693 records). [2025-10-20T19:34:21.100476] [INFO ] [MAIN ] KMI: Full history saved. [2025-10-20T19:34:21.101491] [INFO ] [MAIN ] Update Progress: 1198/2367 symbols completed. KMI took 37.2s. Estimated remaining: 2h 30m 33s. Rate: 7.8 symbols/min [2025-10-20T19:34:21.681633] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:34:21.851367] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-06. [2025-10-20T19:34:22.700148] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:34:22.708496] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-02. [2025-10-20T19:34:22.786004] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T19:34:23.451473] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-07. [2025-10-20T19:34:24.194814] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:34:24.251377] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-03. [2025-10-20T19:34:24.853001] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T19:34:25.051359] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-08. [2025-10-20T19:34:25.683729] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T19:34:25.851413] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-04. [2025-10-20T19:34:26.564030] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:34:26.651387] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-09. [2025-10-20T19:34:27.302445] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T19:34:27.451441] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-05. [2025-10-20T19:34:28.022024] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T19:34:28.251458] [INFO ] [REMOTE ] KMPR: Fetching primary data for 2025-10. [2025-10-20T19:34:28.960092] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:34:29.051366] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-06. [2025-10-20T19:34:29.560839] [INFO ] [REMOTE ] KMPR: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T19:34:29.567802] [INFO ] [REMOTE ] KMPR: Full fetch complete (53343 records). [2025-10-20T19:34:29.567904] [DEBUG] [VALIDATOR ] KMPR: Validating primary data. [2025-10-20T19:34:29.953440] [DEBUG] [VALIDATOR ] KMPR: All expected trading days are present. [2025-10-20T19:34:29.953522] [DEBUG] [VALIDATOR ] KMPR: Primary data validation passed. [2025-10-20T19:34:29.954318] [INFO ] [MAIN ] KMPR: Fetched primary data is valid. [2025-10-20T19:34:29.954378] [DEBUG] [LOCAL ] KMPR: Saving primary data. [2025-10-20T19:34:29.955418] [DEBUG] [LOCAL ] KMPR: Trimmed 53343 records to 40452 (keeping 200 days of data) [2025-10-20T19:34:30.191789] [DEBUG] [LOCAL ] KMPR: Primary data saved (40452 records). [2025-10-20T19:34:30.191912] [INFO ] [MAIN ] KMPR: Full history saved. [2025-10-20T19:34:30.192391] [INFO ] [MAIN ] Update Progress: 1199/2367 symbols completed. KMPR took 20.8s. Estimated remaining: 2h 30m 26s. Rate: 7.8 symbols/min [2025-10-20T19:34:30.498779] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:34:30.507303] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-07. [2025-10-20T19:34:32.056753] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:34:32.065970] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-08. [2025-10-20T19:34:33.525343] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T19:34:33.535145] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-09. [2025-10-20T19:34:34.806681] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:34:34.816163] [INFO ] [REMOTE ] KMT: Fetching primary data for 2025-10. [2025-10-20T19:34:36.165250] [INFO ] [REMOTE ] KMT: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:34:36.171117] [INFO ] [REMOTE ] KMT: Full fetch complete (61953 records). [2025-10-20T19:34:36.171213] [DEBUG] [VALIDATOR ] KMT: Validating primary data. [2025-10-20T19:34:36.572386] [DEBUG] [VALIDATOR ] KMT: All expected trading days are present. [2025-10-20T19:34:36.572471] [DEBUG] [VALIDATOR ] KMT: Primary data validation passed. [2025-10-20T19:34:36.573353] [INFO ] [MAIN ] KMT: Fetched primary data is valid. [2025-10-20T19:34:36.573414] [DEBUG] [LOCAL ] KMT: Saving primary data. [2025-10-20T19:34:36.574625] [DEBUG] [LOCAL ] KMT: Trimmed 61953 records to 42948 (keeping 200 days of data) [2025-10-20T19:34:36.836297] [DEBUG] [LOCAL ] KMT: Primary data saved (42948 records). [2025-10-20T19:34:36.836427] [INFO ] [MAIN ] KMT: Full history saved. [2025-10-20T19:34:36.836502] [INFO ] [MAIN ] Update Progress: 1200/2367 symbols completed. KMT took 19.4s. Estimated remaining: 2h 30m 17s. Rate: 7.8 symbols/min [2025-10-20T19:34:37.381623] [INFO ] [MAIN ] KN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.547334] [INFO ] [MAIN ] KMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.549088] [INFO ] [MAIN ] KNOP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.552273] [INFO ] [MAIN ] KNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:37.580869] [DEBUG] [LOCAL ] KNOP: Loaded 10627 existing records from primary storage. [2025-10-20T19:34:37.590333] [INFO ] [MAIN ] KNOP: Loaded 10627 existing records. [2025-10-20T19:34:37.686820] [DEBUG] [LOCAL ] KNF: Loaded 37116 existing records from primary storage. [2025-10-20T19:34:37.707870] [DEBUG] [LOCAL ] KN: Loaded 39111 existing records from primary storage. [2025-10-20T19:34:37.713492] [INFO ] [MAIN ] KN: Loaded 39111 existing records. [2025-10-20T19:34:37.740170] [INFO ] [MAIN ] KNF: Loaded 37116 existing records. [2025-10-20T19:34:38.061058] [DEBUG] [LOCAL ] KMX: Loaded 54057 existing records from primary storage. [2025-10-20T19:34:38.105707] [WARN ] [MAIN ] KNOP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.137779] [INFO ] [MAIN ] KMX: Loaded 54057 existing records. [2025-10-20T19:34:38.152902] [INFO ] [MAIN ] KNOP: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.196594] [WARN ] [MAIN ] KN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.212175] [INFO ] [LOCAL ] KNOP: Starting deletion of primary data. [2025-10-20T19:34:38.264518] [INFO ] [MAIN ] KN: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.289553] [INFO ] [LOCAL ] KN: Starting deletion of primary data. [2025-10-20T19:34:38.300300] [DEBUG] [LOCAL ] KNOP: Deleted primary data file. [2025-10-20T19:34:38.331100] [DEBUG] [LOCAL ] KN: Deleted primary data file. [2025-10-20T19:34:38.324744] [WARN ] [MAIN ] KNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.336496] [INFO ] [MAIN ] KN: Starting full primary refresh. [2025-10-20T19:34:38.357386] [INFO ] [MAIN ] KNF: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.357542] [INFO ] [LOCAL ] KNF: Starting deletion of primary data. [2025-10-20T19:34:38.331027] [INFO ] [MAIN ] KNOP: Starting full primary refresh. [2025-10-20T19:34:38.418695] [DEBUG] [LOCAL ] KNF: Deleted primary data file. [2025-10-20T19:34:38.468584] [INFO ] [MAIN ] KNF: Starting full primary refresh. [2025-10-20T19:34:38.669525] [WARN ] [MAIN ] KMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:38.787002] [INFO ] [MAIN ] KMX: Wiping existing primary data before full refresh. [2025-10-20T19:34:38.855734] [INFO ] [LOCAL ] KMX: Starting deletion of primary data. [2025-10-20T19:34:39.046493] [DEBUG] [LOCAL ] KMX: Deleted primary data file. [2025-10-20T19:34:39.093066] [INFO ] [MAIN ] KMX: Starting full primary refresh. [2025-10-20T19:34:39.484137] [INFO ] [REMOTE ] KNOP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.530369] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-01. [2025-10-20T19:34:39.582335] [INFO ] [REMOTE ] KN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.593244] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-01. [2025-10-20T19:34:39.681540] [INFO ] [REMOTE ] KNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.712062] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-01. [2025-10-20T19:34:39.911479] [INFO ] [REMOTE ] KMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:34:39.911804] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-01. [2025-10-20T19:34:40.670295] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:34:40.672954] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-02. [2025-10-20T19:34:41.093822] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:34:41.101612] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-02. [2025-10-20T19:34:41.229778] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T19:34:41.236214] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-02. [2025-10-20T19:34:41.365458] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T19:34:41.368986] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-03. [2025-10-20T19:34:41.738070] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:34:41.748407] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-02. [2025-10-20T19:34:42.228807] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T19:34:42.231829] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-04. [2025-10-20T19:34:42.369978] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:34:42.376842] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-03. [2025-10-20T19:34:42.591948] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:34:42.600084] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-03. [2025-10-20T19:34:42.764704] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T19:34:42.768075] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-05. [2025-10-20T19:34:43.334789] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T19:34:43.363061] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:34:43.451348] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-03. [2025-10-20T19:34:43.500751] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:34:44.057346] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:34:44.251357] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-04. [2025-10-20T19:34:45.051453] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-04. [2025-10-20T19:34:45.229533] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T19:34:45.851378] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-04. [2025-10-20T19:34:46.096865] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:34:46.309459] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T19:34:46.651371] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-05. [2025-10-20T19:34:47.451374] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-05. [2025-10-20T19:34:47.955106] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T19:34:48.006991] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T19:34:48.251370] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-06. [2025-10-20T19:34:48.711484] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T19:34:49.051379] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-06. [2025-10-20T19:34:49.591284] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T19:34:49.851417] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-06. [2025-10-20T19:34:50.046782] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T19:34:50.482242] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T19:34:50.651367] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-07. [2025-10-20T19:34:51.451370] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-07. [2025-10-20T19:34:52.059993] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T19:34:52.251368] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-05. [2025-10-20T19:34:52.841156] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T19:34:53.051351] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-08. [2025-10-20T19:34:53.851380] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-08. [2025-10-20T19:34:54.115358] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:34:54.486702] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:34:54.651380] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-07. [2025-10-20T19:34:55.251610] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T19:34:55.451434] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-09. [2025-10-20T19:34:55.508777] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T19:34:56.251416] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-09. [2025-10-20T19:34:56.849570] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T19:34:57.051391] [INFO ] [REMOTE ] KNF: Fetching primary data for 2025-10. [2025-10-20T19:34:57.636275] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T19:34:57.851350] [INFO ] [REMOTE ] KN: Fetching primary data for 2025-10. [2025-10-20T19:34:58.633848] [INFO ] [REMOTE ] KNF: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:34:58.639819] [INFO ] [REMOTE ] KNF: Full fetch complete (51421 records). [2025-10-20T19:34:58.639913] [DEBUG] [VALIDATOR ] KNF: Validating primary data. [2025-10-20T19:34:58.661081] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-06. [2025-10-20T19:34:59.020628] [DEBUG] [VALIDATOR ] KNF: All expected trading days are present. [2025-10-20T19:34:59.020748] [DEBUG] [VALIDATOR ] KNF: Primary data validation passed. [2025-10-20T19:34:59.021522] [INFO ] [MAIN ] KNF: Fetched primary data is valid. [2025-10-20T19:34:59.021563] [DEBUG] [LOCAL ] KNF: Saving primary data. [2025-10-20T19:34:59.022580] [DEBUG] [LOCAL ] KNF: Trimmed 51421 records to 36667 (keeping 200 days of data) [2025-10-20T19:34:59.243063] [DEBUG] [LOCAL ] KNF: Primary data saved (36667 records). [2025-10-20T19:34:59.243312] [INFO ] [MAIN ] KNF: Full history saved. [2025-10-20T19:34:59.244101] [INFO ] [MAIN ] Update Progress: 1201/2367 symbols completed. KNF took 22.4s. Estimated remaining: 2h 30m 24s. Rate: 7.8 symbols/min [2025-10-20T19:34:59.430062] [INFO ] [MAIN ] KNSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:34:59.431261] [INFO ] [REMOTE ] KN: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:34:59.437509] [INFO ] [REMOTE ] KN: Full fetch complete (56557 records). [2025-10-20T19:34:59.437723] [DEBUG] [VALIDATOR ] KN: Validating primary data. [2025-10-20T19:34:59.459039] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-08. [2025-10-20T19:34:59.593068] [DEBUG] [LOCAL ] KNSL: Loaded 18750 existing records from primary storage. [2025-10-20T19:34:59.612295] [INFO ] [MAIN ] KNSL: Loaded 18750 existing records. [2025-10-20T19:34:59.954479] [WARN ] [MAIN ] KNSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:34:59.986251] [INFO ] [MAIN ] KNSL: Wiping existing primary data before full refresh. [2025-10-20T19:34:59.986635] [INFO ] [LOCAL ] KNSL: Starting deletion of primary data. [2025-10-20T19:35:00.002200] [DEBUG] [LOCAL ] KNSL: Deleted primary data file. [2025-10-20T19:35:00.007710] [INFO ] [MAIN ] KNSL: Starting full primary refresh. [2025-10-20T19:35:00.057077] [DEBUG] [VALIDATOR ] KN: All expected trading days are present. [2025-10-20T19:35:00.064042] [DEBUG] [VALIDATOR ] KN: Primary data validation passed. [2025-10-20T19:35:00.090933] [INFO ] [MAIN ] KN: Fetched primary data is valid. [2025-10-20T19:35:00.096697] [DEBUG] [LOCAL ] KN: Saving primary data. [2025-10-20T19:35:00.140303] [DEBUG] [LOCAL ] KN: Trimmed 56557 records to 38488 (keeping 200 days of data) [2025-10-20T19:35:00.360397] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:35:00.430040] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-09. [2025-10-20T19:35:00.589037] [INFO ] [REMOTE ] KNSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:00.621015] [DEBUG] [LOCAL ] KN: Primary data saved (38488 records). [2025-10-20T19:35:00.621140] [INFO ] [MAIN ] KN: Full history saved. [2025-10-20T19:35:00.621247] [INFO ] [MAIN ] Update Progress: 1202/2367 symbols completed. KN took 23.8s. Estimated remaining: 2h 30m 10s. Rate: 7.8 symbols/min [2025-10-20T19:35:00.824679] [INFO ] [MAIN ] KNTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:00.826344] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T19:35:00.881167] [DEBUG] [LOCAL ] KNTK: Loaded 47788 existing records from primary storage. [2025-10-20T19:35:00.881555] [INFO ] [MAIN ] KNTK: Loaded 47788 existing records. [2025-10-20T19:35:01.066234] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-01. [2025-10-20T19:35:01.072452] [WARN ] [MAIN ] KNTK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:01.073613] [INFO ] [MAIN ] KNTK: Wiping existing primary data before full refresh. [2025-10-20T19:35:01.073712] [INFO ] [LOCAL ] KNTK: Starting deletion of primary data. [2025-10-20T19:35:01.074456] [DEBUG] [LOCAL ] KNTK: Deleted primary data file. [2025-10-20T19:35:01.074515] [INFO ] [MAIN ] KNTK: Starting full primary refresh. [2025-10-20T19:35:01.423341] [INFO ] [REMOTE ] KNTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:01.617414] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T19:35:01.851443] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-01. [2025-10-20T19:35:02.166970] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:35:02.651418] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-02. [2025-10-20T19:35:03.456668] [INFO ] [REMOTE ] KNOP: Fetching primary data for 2025-10. [2025-10-20T19:35:03.475912] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:35:04.178102] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:35:04.251329] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-03. [2025-10-20T19:35:05.051435] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-02. [2025-10-20T19:35:05.851405] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-07. [2025-10-20T19:35:07.177727] [INFO ] [REMOTE ] KNOP: Incremental fetch complete for 2025-10 (3.72 seconds). [2025-10-20T19:35:07.180024] [INFO ] [REMOTE ] KNOP: Full fetch complete (14665 records). [2025-10-20T19:35:07.180106] [DEBUG] [VALIDATOR ] KNOP: Validating primary data. [2025-10-20T19:35:07.546440] [DEBUG] [VALIDATOR ] KNOP: All expected trading days are present. [2025-10-20T19:35:07.546515] [DEBUG] [VALIDATOR ] KNOP: Primary data validation passed. [2025-10-20T19:35:07.546860] [INFO ] [MAIN ] KNOP: Fetched primary data is valid. [2025-10-20T19:35:07.546918] [DEBUG] [LOCAL ] KNOP: Saving primary data. [2025-10-20T19:35:07.547392] [DEBUG] [LOCAL ] KNOP: Trimmed 14665 records to 10181 (keeping 200 days of data) [2025-10-20T19:35:07.606315] [DEBUG] [LOCAL ] KNOP: Primary data saved (10181 records). [2025-10-20T19:35:07.606440] [INFO ] [MAIN ] KNOP: Full history saved. [2025-10-20T19:35:07.606518] [INFO ] [MAIN ] Update Progress: 1203/2367 symbols completed. KNOP took 30.7s. Estimated remaining: 2h 30m 2s. Rate: 7.8 symbols/min [2025-10-20T19:35:07.716846] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-03 (3.47 seconds). [2025-10-20T19:35:07.741954] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-04. [2025-10-20T19:35:07.791525] [INFO ] [MAIN ] KNX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:07.840317] [DEBUG] [LOCAL ] KNX: Loaded 53395 existing records from primary storage. [2025-10-20T19:35:07.840716] [INFO ] [MAIN ] KNX: Loaded 53395 existing records. [2025-10-20T19:35:08.015588] [WARN ] [MAIN ] KNX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:08.015877] [INFO ] [MAIN ] KNX: Wiping existing primary data before full refresh. [2025-10-20T19:35:08.015928] [INFO ] [LOCAL ] KNX: Starting deletion of primary data. [2025-10-20T19:35:08.016277] [DEBUG] [LOCAL ] KNX: Deleted primary data file. [2025-10-20T19:35:08.016335] [INFO ] [MAIN ] KNX: Starting full primary refresh. [2025-10-20T19:35:08.379603] [INFO ] [REMOTE ] KNX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:08.379915] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-01. [2025-10-20T19:35:08.726200] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-02 (3.67 seconds). [2025-10-20T19:35:08.734339] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-03. [2025-10-20T19:35:08.897341] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:35:09.051438] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-05. [2025-10-20T19:35:09.175877] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-07 (3.32 seconds). [2025-10-20T19:35:09.713707] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T19:35:09.843003] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:35:09.851351] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-08. [2025-10-20T19:35:10.321822] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T19:35:10.651365] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-02. [2025-10-20T19:35:11.451350] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-06. [2025-10-20T19:35:11.494074] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:35:12.141535] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T19:35:12.181716] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:35:12.251346] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-04. [2025-10-20T19:35:13.051460] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-09. [2025-10-20T19:35:13.724504] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:35:13.851332] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-05. [2025-10-20T19:35:14.651430] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-07. [2025-10-20T19:35:15.024445] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:35:15.120245] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T19:35:15.451334] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-06. [2025-10-20T19:35:15.769579] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:35:16.251439] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-03. [2025-10-20T19:35:17.051433] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-08. [2025-10-20T19:35:17.148150] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:35:17.851401] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-07. [2025-10-20T19:35:18.081345] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T19:35:18.102265] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:35:18.651468] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-04. [2025-10-20T19:35:19.451415] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-09. [2025-10-20T19:35:20.251340] [INFO ] [REMOTE ] KMX: Fetching primary data for 2025-10. [2025-10-20T19:35:20.744482] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-07 (2.89 seconds). [2025-10-20T19:35:21.024765] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T19:35:21.051332] [INFO ] [REMOTE ] KNSL: Fetching primary data for 2025-10. [2025-10-20T19:35:21.339192] [INFO ] [REMOTE ] KMX: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T19:35:21.346916] [INFO ] [REMOTE ] KMX: Full fetch complete (76383 records). [2025-10-20T19:35:21.347037] [DEBUG] [VALIDATOR ] KMX: Validating primary data. [2025-10-20T19:35:21.735334] [DEBUG] [VALIDATOR ] KMX: All expected trading days are present. [2025-10-20T19:35:21.735479] [DEBUG] [VALIDATOR ] KMX: Primary data validation passed. [2025-10-20T19:35:21.736477] [INFO ] [MAIN ] KMX: Fetched primary data is valid. [2025-10-20T19:35:21.736537] [DEBUG] [LOCAL ] KMX: Saving primary data. [2025-10-20T19:35:21.737961] [DEBUG] [LOCAL ] KMX: Trimmed 76383 records to 53284 (keeping 200 days of data) [2025-10-20T19:35:21.851377] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-08. [2025-10-20T19:35:22.046720] [DEBUG] [LOCAL ] KMX: Primary data saved (53284 records). [2025-10-20T19:35:22.046864] [INFO ] [MAIN ] KMX: Full history saved. [2025-10-20T19:35:22.046970] [INFO ] [MAIN ] Update Progress: 1204/2367 symbols completed. KMX took 45.2s. Estimated remaining: 2h 30m 0s. Rate: 7.8 symbols/min [2025-10-20T19:35:22.214076] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-04 (3.56 seconds). [2025-10-20T19:35:22.278286] [INFO ] [REMOTE ] KNSL: Incremental fetch complete for 2025-10 (1.23 seconds). [2025-10-20T19:35:22.267833] [INFO ] [MAIN ] KO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:22.282865] [DEBUG] [LOCAL ] KO: No existing data file found in primary storage. [2025-10-20T19:35:22.283215] [WARN ] [MAIN ] KO: No existing data found, performing full refresh instead. [2025-10-20T19:35:22.282770] [INFO ] [REMOTE ] KNSL: Full fetch complete (27226 records). [2025-10-20T19:35:22.283394] [INFO ] [MAIN ] KO: Wiping existing primary data before full refresh. [2025-10-20T19:35:22.283484] [DEBUG] [VALIDATOR ] KNSL: Validating primary data. [2025-10-20T19:35:22.283565] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-10-20T19:35:22.283769] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-10-20T19:35:22.283881] [INFO ] [MAIN ] KO: Starting full primary refresh. [2025-10-20T19:35:22.656457] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-05. [2025-10-20T19:35:23.016271] [INFO ] [REMOTE ] KO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:23.026174] [DEBUG] [VALIDATOR ] KNSL: All expected trading days are present. [2025-10-20T19:35:23.026224] [DEBUG] [VALIDATOR ] KNSL: Primary data validation passed. [2025-10-20T19:35:23.026757] [INFO ] [MAIN ] KNSL: Fetched primary data is valid. [2025-10-20T19:35:23.026816] [DEBUG] [LOCAL ] KNSL: Saving primary data. [2025-10-20T19:35:23.027416] [DEBUG] [LOCAL ] KNSL: Trimmed 27226 records to 18516 (keeping 200 days of data) [2025-10-20T19:35:23.140047] [DEBUG] [LOCAL ] KNSL: Primary data saved (18516 records). [2025-10-20T19:35:23.140182] [INFO ] [MAIN ] KNSL: Full history saved. [2025-10-20T19:35:23.140262] [INFO ] [MAIN ] Update Progress: 1205/2367 symbols completed. KNSL took 23.9s. Estimated remaining: 2h 29m 46s. Rate: 7.8 symbols/min [2025-10-20T19:35:23.318007] [INFO ] [MAIN ] KODK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:23.358597] [DEBUG] [LOCAL ] KODK: Loaded 44414 existing records from primary storage. [2025-10-20T19:35:23.358949] [INFO ] [MAIN ] KODK: Loaded 44414 existing records. [2025-10-20T19:35:23.456330] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-01. [2025-10-20T19:35:23.541512] [WARN ] [MAIN ] KODK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:23.541877] [INFO ] [MAIN ] KODK: Wiping existing primary data before full refresh. [2025-10-20T19:35:23.541950] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-10-20T19:35:23.550896] [DEBUG] [LOCAL ] KODK: Deleted primary data file. [2025-10-20T19:35:23.566330] [INFO ] [MAIN ] KODK: Starting full primary refresh. [2025-10-20T19:35:23.601180] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T19:35:23.967161] [INFO ] [REMOTE ] KODK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:24.048216] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:35:24.251407] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-06. [2025-10-20T19:35:25.051433] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-09. [2025-10-20T19:35:25.851381] [INFO ] [REMOTE ] KODK: Fetching primary data for 2025-01. [2025-10-20T19:35:25.954627] [DEBUG] [REMOTE ] KODK: Error fetching primary data for 2025-01: 'API call returned an empty response.' [2025-10-20T19:35:25.954966] [WARN ] [REMOTE ] KODK: No data returned for month 2025-01 [2025-10-20T19:35:25.955054] [FATAL] [REMOTE ] KODK: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:35:25.955100] [FATAL] [MAIN ] KODK: Failed to fetch primary data. [2025-10-20T19:35:25.955161] [WARN ] [MAIN ] KODK: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:35:25.955232] [INFO ] [LOCAL ] KODK: Starting deletion of primary data. [2025-10-20T19:35:25.955293] [DEBUG] [LOCAL ] KODK: No primary data file found to delete. [2025-10-20T19:35:25.955329] [INFO ] [LOCAL ] KODK: Starting deletion of derivative data. [2025-10-20T19:35:25.955484] [DEBUG] [LOCAL ] KODK: Deleted derivative data file. [2025-10-20T19:35:25.955882] [INFO ] [MAIN ] Update Progress: 1206/2367 symbols completed. KODK took 2.8s. Estimated remaining: 2h 29m 34s. Rate: 7.8 symbols/min [2025-10-20T19:35:26.133838] [INFO ] [MAIN ] KOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:26.166494] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T19:35:26.199273] [DEBUG] [LOCAL ] KOF: Loaded 26961 existing records from primary storage. [2025-10-20T19:35:26.199724] [INFO ] [MAIN ] KOF: Loaded 26961 existing records. [2025-10-20T19:35:26.395552] [WARN ] [MAIN ] KOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:26.420224] [INFO ] [MAIN ] KOF: Wiping existing primary data before full refresh. [2025-10-20T19:35:26.423270] [INFO ] [LOCAL ] KOF: Starting deletion of primary data. [2025-10-20T19:35:26.448361] [DEBUG] [LOCAL ] KOF: Deleted primary data file. [2025-10-20T19:35:26.448920] [INFO ] [MAIN ] KOF: Starting full primary refresh. [2025-10-20T19:35:26.450924] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-01 (2.99 seconds). [2025-10-20T19:35:26.497009] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T19:35:26.651345] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-07. [2025-10-20T19:35:26.858556] [INFO ] [REMOTE ] KOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:27.451382] [INFO ] [REMOTE ] KNTK: Fetching primary data for 2025-10. [2025-10-20T19:35:28.251393] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-01. [2025-10-20T19:35:28.451222] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:35:29.009508] [INFO ] [REMOTE ] KNTK: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T19:35:29.016841] [INFO ] [REMOTE ] KNTK: Full fetch complete (65669 records). [2025-10-20T19:35:29.016962] [DEBUG] [VALIDATOR ] KNTK: Validating primary data. [2025-10-20T19:35:29.059046] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-02. [2025-10-20T19:35:29.414044] [DEBUG] [VALIDATOR ] KNTK: All expected trading days are present. [2025-10-20T19:35:29.414182] [DEBUG] [VALIDATOR ] KNTK: Primary data validation passed. [2025-10-20T19:35:29.415552] [INFO ] [MAIN ] KNTK: Fetched primary data is valid. [2025-10-20T19:35:29.415619] [DEBUG] [LOCAL ] KNTK: Saving primary data. [2025-10-20T19:35:29.417119] [DEBUG] [LOCAL ] KNTK: Trimmed 65669 records to 47220 (keeping 200 days of data) [2025-10-20T19:35:29.730733] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T19:35:29.749906] [DEBUG] [LOCAL ] KNTK: Primary data saved (47220 records). [2025-10-20T19:35:29.750047] [INFO ] [MAIN ] KNTK: Full history saved. [2025-10-20T19:35:29.750816] [INFO ] [MAIN ] Update Progress: 1207/2367 symbols completed. KNTK took 29.1s. Estimated remaining: 2h 29m 22s. Rate: 7.8 symbols/min [2025-10-20T19:35:29.856522] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-08. [2025-10-20T19:35:29.940529] [INFO ] [MAIN ] KOP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:29.960760] [DEBUG] [LOCAL ] KOP: Loaded 17674 existing records from primary storage. [2025-10-20T19:35:29.961044] [INFO ] [MAIN ] KOP: Loaded 17674 existing records. [2025-10-20T19:35:30.160423] [WARN ] [MAIN ] KOP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:30.160846] [INFO ] [MAIN ] KOP: Wiping existing primary data before full refresh. [2025-10-20T19:35:30.160914] [INFO ] [LOCAL ] KOP: Starting deletion of primary data. [2025-10-20T19:35:30.161179] [DEBUG] [LOCAL ] KOP: Deleted primary data file. [2025-10-20T19:35:30.161245] [INFO ] [MAIN ] KOP: Starting full primary refresh. [2025-10-20T19:35:30.578720] [INFO ] [REMOTE ] KOP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:30.651351] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-01. [2025-10-20T19:35:31.332388] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-02 (2.27 seconds). [2025-10-20T19:35:31.451385] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-02. [2025-10-20T19:35:32.246883] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:35:32.251329] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-02. [2025-10-20T19:35:32.439815] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-08 (2.58 seconds). [2025-10-20T19:35:32.849969] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T19:35:32.921451] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T19:35:33.051382] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-03. [2025-10-20T19:35:33.851417] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-09. [2025-10-20T19:35:34.651407] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-03. [2025-10-20T19:35:35.451382] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-03. [2025-10-20T19:35:35.655164] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:35:35.687883] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:35:35.737994] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-03 (2.69 seconds). [2025-10-20T19:35:36.251425] [INFO ] [REMOTE ] KNX: Fetching primary data for 2025-10. [2025-10-20T19:35:36.442693] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T19:35:37.053120] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-04. [2025-10-20T19:35:37.055606] [INFO ] [REMOTE ] KNX: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T19:35:37.063899] [INFO ] [REMOTE ] KNX: Full fetch complete (76289 records). [2025-10-20T19:35:37.064048] [DEBUG] [VALIDATOR ] KNX: Validating primary data. [2025-10-20T19:35:37.452853] [DEBUG] [VALIDATOR ] KNX: All expected trading days are present. [2025-10-20T19:35:37.452932] [DEBUG] [VALIDATOR ] KNX: Primary data validation passed. [2025-10-20T19:35:37.453990] [INFO ] [MAIN ] KNX: Fetched primary data is valid. [2025-10-20T19:35:37.454054] [DEBUG] [LOCAL ] KNX: Saving primary data. [2025-10-20T19:35:37.455448] [DEBUG] [LOCAL ] KNX: Trimmed 76289 records to 52639 (keeping 200 days of data) [2025-10-20T19:35:37.800033] [DEBUG] [LOCAL ] KNX: Primary data saved (52639 records). [2025-10-20T19:35:37.800201] [INFO ] [MAIN ] KNX: Full history saved. [2025-10-20T19:35:37.800287] [INFO ] [MAIN ] Update Progress: 1208/2367 symbols completed. KNX took 30.2s. Estimated remaining: 2h 29m 15s. Rate: 7.8 symbols/min [2025-10-20T19:35:37.864116] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-04. [2025-10-20T19:35:38.033083] [INFO ] [MAIN ] KORE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:38.039712] [DEBUG] [LOCAL ] KORE: Loaded 2191 existing records from primary storage. [2025-10-20T19:35:38.039997] [INFO ] [MAIN ] KORE: Loaded 2191 existing records. [2025-10-20T19:35:38.231694] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T19:35:38.345638] [WARN ] [MAIN ] KORE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:38.345976] [INFO ] [MAIN ] KORE: Wiping existing primary data before full refresh. [2025-10-20T19:35:38.346067] [INFO ] [LOCAL ] KORE: Starting deletion of primary data. [2025-10-20T19:35:38.346303] [DEBUG] [LOCAL ] KORE: Deleted primary data file. [2025-10-20T19:35:38.346383] [INFO ] [MAIN ] KORE: Starting full primary refresh. [2025-10-20T19:35:38.658424] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-05. [2025-10-20T19:35:38.794986] [INFO ] [REMOTE ] KORE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:39.451383] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-01. [2025-10-20T19:35:39.784929] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T19:35:40.251418] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-04. [2025-10-20T19:35:40.385611] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T19:35:40.491322] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-04 (2.63 seconds). [2025-10-20T19:35:41.051372] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-02. [2025-10-20T19:35:41.357147] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-02 (0.31 seconds). [2025-10-20T19:35:41.384319] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:35:41.851416] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-05. [2025-10-20T19:35:42.651341] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-06. [2025-10-20T19:35:43.004771] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T19:35:43.197759] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T19:35:43.451412] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-03. [2025-10-20T19:35:43.861597] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T19:35:44.251351] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-04. [2025-10-20T19:35:44.700039] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T19:35:45.051383] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-05. [2025-10-20T19:35:45.851374] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-06. [2025-10-20T19:35:46.651359] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-07. [2025-10-20T19:35:47.042963] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T19:35:47.288407] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-10-20T19:35:47.451357] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-07. [2025-10-20T19:35:47.552933] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:35:48.251369] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-08. [2025-10-20T19:35:48.656984] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T19:35:49.051355] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-06. [2025-10-20T19:35:49.282845] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T19:35:49.851363] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-09. [2025-10-20T19:35:50.651432] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-05. [2025-10-20T19:35:50.827475] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:35:51.074567] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T19:35:51.456537] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-06. [2025-10-20T19:35:51.469475] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T19:35:51.887815] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T19:35:52.251351] [INFO ] [REMOTE ] KOP: Fetching primary data for 2025-10. [2025-10-20T19:35:53.051358] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-07. [2025-10-20T19:35:53.230957] [INFO ] [REMOTE ] KOP: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:35:53.234029] [INFO ] [REMOTE ] KOP: Full fetch complete (25442 records). [2025-10-20T19:35:53.234121] [DEBUG] [VALIDATOR ] KOP: Validating primary data. [2025-10-20T19:35:53.620036] [DEBUG] [VALIDATOR ] KOP: All expected trading days are present. [2025-10-20T19:35:53.620440] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:35:53.620525] [DEBUG] [VALIDATOR ] KOP: Primary data validation passed. [2025-10-20T19:35:53.622709] [INFO ] [MAIN ] KOP: Fetched primary data is valid. [2025-10-20T19:35:53.623057] [DEBUG] [LOCAL ] KOP: Saving primary data. [2025-10-20T19:35:53.623750] [DEBUG] [LOCAL ] KOP: Trimmed 25442 records to 17454 (keeping 200 days of data) [2025-10-20T19:35:53.732192] [DEBUG] [LOCAL ] KOP: Primary data saved (17454 records). [2025-10-20T19:35:53.732326] [INFO ] [MAIN ] KOP: Full history saved. [2025-10-20T19:35:53.732418] [INFO ] [MAIN ] Update Progress: 1209/2367 symbols completed. KOP took 24.0s. Estimated remaining: 2h 29m 15s. Rate: 7.8 symbols/min [2025-10-20T19:35:53.856602] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-08. [2025-10-20T19:35:53.902409] [INFO ] [MAIN ] KOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:35:53.954552] [DEBUG] [LOCAL ] KOS: Loaded 53875 existing records from primary storage. [2025-10-20T19:35:53.954899] [INFO ] [MAIN ] KOS: Loaded 53875 existing records. [2025-10-20T19:35:54.128092] [WARN ] [MAIN ] KOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:35:54.128353] [INFO ] [MAIN ] KOS: Wiping existing primary data before full refresh. [2025-10-20T19:35:54.128400] [INFO ] [LOCAL ] KOS: Starting deletion of primary data. [2025-10-20T19:35:54.128693] [DEBUG] [LOCAL ] KOS: Deleted primary data file. [2025-10-20T19:35:54.128750] [INFO ] [MAIN ] KOS: Starting full primary refresh. [2025-10-20T19:35:54.491897] [INFO ] [REMOTE ] KOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:35:54.651354] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-01. [2025-10-20T19:35:55.005715] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T19:35:55.451396] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-07. [2025-10-20T19:35:56.251355] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-08. [2025-10-20T19:35:56.539817] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T19:35:56.695550] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-01 (2.04 seconds). [2025-10-20T19:35:57.051433] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-02. [2025-10-20T19:35:57.851369] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-09. [2025-10-20T19:35:57.922314] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-10-20T19:35:58.657278] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-08. [2025-10-20T19:35:58.682397] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:35:59.078202] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T19:35:59.451447] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-03. [2025-10-20T19:36:00.251423] [INFO ] [REMOTE ] KOF: Fetching primary data for 2025-10. [2025-10-20T19:36:01.051348] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-09. [2025-10-20T19:36:01.620608] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-03 (2.17 seconds). [2025-10-20T19:36:01.655414] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T19:36:01.659340] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-08 (3.00 seconds). [2025-10-20T19:36:01.851631] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-04. [2025-10-20T19:36:02.118600] [INFO ] [REMOTE ] KOF: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:36:02.122974] [INFO ] [REMOTE ] KOF: Full fetch complete (38180 records). [2025-10-20T19:36:02.123047] [DEBUG] [VALIDATOR ] KOF: Validating primary data. [2025-10-20T19:36:02.497876] [DEBUG] [VALIDATOR ] KOF: All expected trading days are present. [2025-10-20T19:36:02.497965] [DEBUG] [VALIDATOR ] KOF: Primary data validation passed. [2025-10-20T19:36:02.498741] [INFO ] [MAIN ] KOF: Fetched primary data is valid. [2025-10-20T19:36:02.498803] [DEBUG] [LOCAL ] KOF: Saving primary data. [2025-10-20T19:36:02.499789] [DEBUG] [LOCAL ] KOF: Trimmed 38180 records to 26742 (keeping 200 days of data) [2025-10-20T19:36:02.653013] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-09. [2025-10-20T19:36:02.664348] [DEBUG] [LOCAL ] KOF: Primary data saved (26742 records). [2025-10-20T19:36:02.665371] [INFO ] [MAIN ] KOF: Full history saved. [2025-10-20T19:36:02.665463] [INFO ] [MAIN ] Update Progress: 1210/2367 symbols completed. KOF took 36.7s. Estimated remaining: 2h 29m 8s. Rate: 7.8 symbols/min [2025-10-20T19:36:02.840254] [INFO ] [MAIN ] KR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:02.889274] [DEBUG] [LOCAL ] KR: Loaded 54460 existing records from primary storage. [2025-10-20T19:36:02.889580] [INFO ] [MAIN ] KR: Loaded 54460 existing records. [2025-10-20T19:36:03.069051] [WARN ] [MAIN ] KR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:03.069257] [INFO ] [MAIN ] KR: Wiping existing primary data before full refresh. [2025-10-20T19:36:03.069313] [INFO ] [LOCAL ] KR: Starting deletion of primary data. [2025-10-20T19:36:03.069508] [DEBUG] [LOCAL ] KR: Deleted primary data file. [2025-10-20T19:36:03.069545] [INFO ] [MAIN ] KR: Starting full primary refresh. [2025-10-20T19:36:03.439881] [INFO ] [REMOTE ] KR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:03.451379] [INFO ] [REMOTE ] KORE: Fetching primary data for 2025-10. [2025-10-20T19:36:03.530917] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:36:03.945838] [INFO ] [REMOTE ] KORE: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:36:03.947217] [INFO ] [REMOTE ] KORE: Full fetch complete (4423 records). [2025-10-20T19:36:03.947301] [DEBUG] [VALIDATOR ] KORE: Validating primary data. [2025-10-20T19:36:04.256624] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-01. [2025-10-20T19:36:04.314365] [DEBUG] [VALIDATOR ] KORE: All expected trading days are present. [2025-10-20T19:36:04.314683] [DEBUG] [VALIDATOR ] KORE: Primary data validation passed. [2025-10-20T19:36:04.314917] [INFO ] [MAIN ] KORE: Fetched primary data is valid. [2025-10-20T19:36:04.314964] [DEBUG] [LOCAL ] KORE: Saving primary data. [2025-10-20T19:36:04.315453] [DEBUG] [LOCAL ] KORE: Trimmed 4423 records to 2176 (keeping 200 days of data) [2025-10-20T19:36:04.329119] [DEBUG] [LOCAL ] KORE: Primary data saved (2176 records). [2025-10-20T19:36:04.329255] [INFO ] [MAIN ] KORE: Full history saved. [2025-10-20T19:36:04.329337] [INFO ] [MAIN ] Update Progress: 1211/2367 symbols completed. KORE took 26.5s. Estimated remaining: 2h 28m 55s. Rate: 7.8 symbols/min [2025-10-20T19:36:04.496838] [INFO ] [MAIN ] KRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:04.540787] [DEBUG] [LOCAL ] KRC: Loaded 48982 existing records from primary storage. [2025-10-20T19:36:04.541119] [INFO ] [MAIN ] KRC: Loaded 48982 existing records. [2025-10-20T19:36:04.720105] [WARN ] [MAIN ] KRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:04.721064] [INFO ] [MAIN ] KRC: Wiping existing primary data before full refresh. [2025-10-20T19:36:04.721240] [INFO ] [LOCAL ] KRC: Starting deletion of primary data. [2025-10-20T19:36:04.721474] [DEBUG] [LOCAL ] KRC: Deleted primary data file. [2025-10-20T19:36:04.721530] [INFO ] [MAIN ] KRC: Starting full primary refresh. [2025-10-20T19:36:05.074145] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-05. [2025-10-20T19:36:05.087578] [INFO ] [REMOTE ] KRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:05.224357] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-09 (2.57 seconds). [2025-10-20T19:36:05.851349] [INFO ] [REMOTE ] KO: Fetching primary data for 2025-10. [2025-10-20T19:36:06.165831] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T19:36:06.651393] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-02. [2025-10-20T19:36:06.763157] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T19:36:06.794014] [INFO ] [REMOTE ] KO: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:36:06.801808] [INFO ] [REMOTE ] KO: Full fetch complete (77218 records). [2025-10-20T19:36:06.801927] [DEBUG] [VALIDATOR ] KO: Validating primary data. [2025-10-20T19:36:06.995988] [WARN ] [VALIDATOR ] KO: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T19:36:06.996791] [FATAL] [MAIN ] KO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:36:06.996896] [WARN ] [MAIN ] KO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:36:06.996987] [INFO ] [LOCAL ] KO: Starting deletion of primary data. [2025-10-20T19:36:06.997068] [DEBUG] [LOCAL ] KO: No primary data file found to delete. [2025-10-20T19:36:06.997106] [INFO ] [LOCAL ] KO: Starting deletion of derivative data. [2025-10-20T19:36:06.997145] [DEBUG] [LOCAL ] KO: No derivative data file found to delete. [2025-10-20T19:36:06.997202] [INFO ] [MAIN ] Update Progress: 1212/2367 symbols completed. KO took 45.0s. Estimated remaining: 2h 28m 42s. Rate: 7.8 symbols/min [2025-10-20T19:36:07.163432] [INFO ] [MAIN ] KREF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:07.198225] [DEBUG] [LOCAL ] KREF: Loaded 38479 existing records from primary storage. [2025-10-20T19:36:07.198527] [INFO ] [MAIN ] KREF: Loaded 38479 existing records. [2025-10-20T19:36:07.373055] [WARN ] [MAIN ] KREF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:07.373266] [INFO ] [MAIN ] KREF: Wiping existing primary data before full refresh. [2025-10-20T19:36:07.373323] [INFO ] [LOCAL ] KREF: Starting deletion of primary data. [2025-10-20T19:36:07.373521] [DEBUG] [LOCAL ] KREF: Deleted primary data file. [2025-10-20T19:36:07.373585] [INFO ] [MAIN ] KREF: Starting full primary refresh. [2025-10-20T19:36:07.456686] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-01. [2025-10-20T19:36:07.727301] [INFO ] [REMOTE ] KREF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:08.251395] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-06. [2025-10-20T19:36:08.438986] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:36:09.051404] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-03. [2025-10-20T19:36:09.094580] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:36:09.861768] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-01. [2025-10-20T19:36:09.864628] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T19:36:10.651345] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-07. [2025-10-20T19:36:10.848914] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T19:36:11.144763] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:36:11.451414] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-02. [2025-10-20T19:36:12.251351] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-04. [2025-10-20T19:36:12.570973] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:36:12.981408] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:36:13.051364] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-03. [2025-10-20T19:36:13.851365] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-08. [2025-10-20T19:36:14.287005] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T19:36:14.651410] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-05. [2025-10-20T19:36:14.807885] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:36:15.451411] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-04. [2025-10-20T19:36:16.024996] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T19:36:16.251388] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-09. [2025-10-20T19:36:17.051438] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-02. [2025-10-20T19:36:17.567054] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-05 (2.92 seconds). [2025-10-20T19:36:17.611833] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T19:36:17.851363] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-06. [2025-10-20T19:36:18.320604] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:36:18.431638] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-09 (2.18 seconds). [2025-10-20T19:36:18.651353] [INFO ] [REMOTE ] KOS: Fetching primary data for 2025-10. [2025-10-20T19:36:19.451358] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-03. [2025-10-20T19:36:19.744542] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:36:20.232675] [INFO ] [REMOTE ] KOS: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T19:36:20.240512] [INFO ] [REMOTE ] KOS: Full fetch complete (77025 records). [2025-10-20T19:36:20.240715] [DEBUG] [VALIDATOR ] KOS: Validating primary data. [2025-10-20T19:36:20.256564] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-05. [2025-10-20T19:36:20.662061] [DEBUG] [VALIDATOR ] KOS: All expected trading days are present. [2025-10-20T19:36:20.662188] [DEBUG] [VALIDATOR ] KOS: Primary data validation passed. [2025-10-20T19:36:20.663427] [INFO ] [MAIN ] KOS: Fetched primary data is valid. [2025-10-20T19:36:20.663578] [DEBUG] [LOCAL ] KOS: Saving primary data. [2025-10-20T19:36:20.665297] [DEBUG] [LOCAL ] KOS: Trimmed 77025 records to 53095 (keeping 200 days of data) [2025-10-20T19:36:20.916200] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T19:36:21.022125] [DEBUG] [LOCAL ] KOS: Primary data saved (53095 records). [2025-10-20T19:36:21.022277] [INFO ] [MAIN ] KOS: Full history saved. [2025-10-20T19:36:21.023102] [INFO ] [MAIN ] Update Progress: 1213/2367 symbols completed. KOS took 27.3s. Estimated remaining: 2h 28m 41s. Rate: 7.8 symbols/min [2025-10-20T19:36:21.054980] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-07. [2025-10-20T19:36:21.195790] [INFO ] [MAIN ] KRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:21.242704] [DEBUG] [LOCAL ] KRG: Loaded 50023 existing records from primary storage. [2025-10-20T19:36:21.243062] [INFO ] [MAIN ] KRG: Loaded 50023 existing records. [2025-10-20T19:36:21.447095] [WARN ] [MAIN ] KRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:21.447368] [INFO ] [MAIN ] KRG: Wiping existing primary data before full refresh. [2025-10-20T19:36:21.447415] [INFO ] [LOCAL ] KRG: Starting deletion of primary data. [2025-10-20T19:36:21.447632] [DEBUG] [LOCAL ] KRG: Deleted primary data file. [2025-10-20T19:36:21.447726] [INFO ] [MAIN ] KRG: Starting full primary refresh. [2025-10-20T19:36:21.803983] [INFO ] [REMOTE ] KRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:21.839731] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T19:36:21.851336] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-01. [2025-10-20T19:36:22.651363] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-06. [2025-10-20T19:36:23.125771] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T19:36:23.451412] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-04. [2025-10-20T19:36:23.539067] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:36:24.235303] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:36:24.251349] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-08. [2025-10-20T19:36:24.971216] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T19:36:25.051391] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-02. [2025-10-20T19:36:25.851360] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-05. [2025-10-20T19:36:26.156406] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T19:36:26.557808] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T19:36:26.651413] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-03. [2025-10-20T19:36:27.343590] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:36:27.451379] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-07. [2025-10-20T19:36:28.251352] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-09. [2025-10-20T19:36:28.384120] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:36:29.051472] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-06. [2025-10-20T19:36:29.137679] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T19:36:29.851344] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-08. [2025-10-20T19:36:30.185555] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T19:36:30.517526] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:36:30.651390] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-07. [2025-10-20T19:36:31.451457] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-04. [2025-10-20T19:36:31.515482] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:36:32.191744] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:36:32.251375] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-08. [2025-10-20T19:36:33.051398] [INFO ] [REMOTE ] KR: Fetching primary data for 2025-10. [2025-10-20T19:36:33.273245] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T19:36:33.510965] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T19:36:33.851355] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-09. [2025-10-20T19:36:33.923187] [INFO ] [REMOTE ] KR: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:36:33.931181] [INFO ] [REMOTE ] KR: Full fetch complete (77843 records). [2025-10-20T19:36:33.931353] [DEBUG] [VALIDATOR ] KR: Validating primary data. [2025-10-20T19:36:34.350361] [DEBUG] [VALIDATOR ] KR: All expected trading days are present. [2025-10-20T19:36:34.350481] [DEBUG] [VALIDATOR ] KR: Primary data validation passed. [2025-10-20T19:36:34.351920] [INFO ] [MAIN ] KR: Fetched primary data is valid. [2025-10-20T19:36:34.351988] [DEBUG] [LOCAL ] KR: Saving primary data. [2025-10-20T19:36:34.354121] [DEBUG] [LOCAL ] KR: Trimmed 77843 records to 53681 (keeping 200 days of data) [2025-10-20T19:36:34.661584] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-09. [2025-10-20T19:36:34.717816] [DEBUG] [LOCAL ] KR: Primary data saved (53681 records). [2025-10-20T19:36:34.718021] [INFO ] [MAIN ] KR: Full history saved. [2025-10-20T19:36:34.718375] [INFO ] [MAIN ] Update Progress: 1214/2367 symbols completed. KR took 32.1s. Estimated remaining: 2h 28m 38s. Rate: 7.8 symbols/min [2025-10-20T19:36:34.898970] [INFO ] [MAIN ] KRMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:34.899196] [DEBUG] [LOCAL ] KRMN: No existing data file found in primary storage. [2025-10-20T19:36:34.899233] [WARN ] [MAIN ] KRMN: No existing data found, performing full refresh instead. [2025-10-20T19:36:34.899359] [INFO ] [MAIN ] KRMN: Wiping existing primary data before full refresh. [2025-10-20T19:36:34.899399] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-10-20T19:36:34.899439] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-10-20T19:36:34.899472] [INFO ] [MAIN ] KRMN: Starting full primary refresh. [2025-10-20T19:36:35.280291] [INFO ] [REMOTE ] KRMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:35.324017] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:36:35.451388] [INFO ] [REMOTE ] KREF: Fetching primary data for 2025-10. [2025-10-20T19:36:36.251348] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-05. [2025-10-20T19:36:36.368773] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:36:36.932906] [INFO ] [REMOTE ] KREF: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:36:36.939188] [INFO ] [REMOTE ] KREF: Full fetch complete (52830 records). [2025-10-20T19:36:36.939330] [DEBUG] [VALIDATOR ] KREF: Validating primary data. [2025-10-20T19:36:37.056481] [INFO ] [REMOTE ] KRMN: Fetching primary data for 2025-01. [2025-10-20T19:36:37.336201] [DEBUG] [REMOTE ] KRMN: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:36:37.349115] [DEBUG] [VALIDATOR ] KREF: All expected trading days are present. [2025-10-20T19:36:37.349189] [DEBUG] [VALIDATOR ] KREF: Primary data validation passed. [2025-10-20T19:36:37.350130] [INFO ] [MAIN ] KREF: Fetched primary data is valid. [2025-10-20T19:36:37.350207] [WARN ] [REMOTE ] KRMN: No data returned for month 2025-01 [2025-10-20T19:36:37.350387] [DEBUG] [LOCAL ] KREF: Saving primary data. [2025-10-20T19:36:37.350340] [FATAL] [REMOTE ] KRMN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:36:37.350610] [FATAL] [MAIN ] KRMN: Failed to fetch primary data. [2025-10-20T19:36:37.350705] [WARN ] [MAIN ] KRMN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:36:37.350782] [INFO ] [LOCAL ] KRMN: Starting deletion of primary data. [2025-10-20T19:36:37.351027] [DEBUG] [LOCAL ] KRMN: No primary data file found to delete. [2025-10-20T19:36:37.351089] [INFO ] [LOCAL ] KRMN: Starting deletion of derivative data. [2025-10-20T19:36:37.351216] [DEBUG] [LOCAL ] KRMN: No derivative data file found to delete. [2025-10-20T19:36:37.351368] [INFO ] [MAIN ] Update Progress: 1215/2367 symbols completed. KRMN took 2.6s. Estimated remaining: 2h 28m 26s. Rate: 7.8 symbols/min [2025-10-20T19:36:37.427464] [DEBUG] [LOCAL ] KREF: Trimmed 52830 records to 38036 (keeping 200 days of data) [2025-10-20T19:36:37.629658] [INFO ] [MAIN ] KRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:37.782308] [DEBUG] [LOCAL ] KREF: Primary data saved (38036 records). [2025-10-20T19:36:37.782876] [DEBUG] [LOCAL ] KRO: Loaded 26887 existing records from primary storage. [2025-10-20T19:36:37.783147] [INFO ] [MAIN ] KREF: Full history saved. [2025-10-20T19:36:37.783487] [INFO ] [MAIN ] Update Progress: 1216/2367 symbols completed. KREF took 30.8s. Estimated remaining: 2h 28m 11s. Rate: 7.8 symbols/min [2025-10-20T19:36:37.783694] [INFO ] [MAIN ] KRO: Loaded 26887 existing records. [2025-10-20T19:36:37.852827] [INFO ] [REMOTE ] KRC: Fetching primary data for 2025-10. [2025-10-20T19:36:38.120234] [WARN ] [MAIN ] KRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:38.132522] [INFO ] [MAIN ] KRO: Wiping existing primary data before full refresh. [2025-10-20T19:36:38.167550] [INFO ] [LOCAL ] KRO: Starting deletion of primary data. [2025-10-20T19:36:38.152963] [INFO ] [MAIN ] KRP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:38.188504] [DEBUG] [LOCAL ] KRO: Deleted primary data file. [2025-10-20T19:36:38.192471] [INFO ] [MAIN ] KRO: Starting full primary refresh. [2025-10-20T19:36:38.224507] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T19:36:38.459740] [DEBUG] [LOCAL ] KRP: Loaded 36833 existing records from primary storage. [2025-10-20T19:36:38.498862] [INFO ] [MAIN ] KRP: Loaded 36833 existing records. [2025-10-20T19:36:38.663939] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-06. [2025-10-20T19:36:38.733013] [INFO ] [REMOTE ] KRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:38.810008] [WARN ] [MAIN ] KRP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:38.810249] [INFO ] [MAIN ] KRP: Wiping existing primary data before full refresh. [2025-10-20T19:36:38.810297] [INFO ] [LOCAL ] KRP: Starting deletion of primary data. [2025-10-20T19:36:38.810476] [DEBUG] [LOCAL ] KRP: Deleted primary data file. [2025-10-20T19:36:38.810512] [INFO ] [MAIN ] KRP: Starting full primary refresh. [2025-10-20T19:36:39.175477] [INFO ] [REMOTE ] KRP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:39.451370] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-01. [2025-10-20T19:36:39.520585] [INFO ] [REMOTE ] KRC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:36:39.529621] [INFO ] [REMOTE ] KRC: Full fetch complete (70047 records). [2025-10-20T19:36:39.529843] [DEBUG] [VALIDATOR ] KRC: Validating primary data. [2025-10-20T19:36:39.963571] [DEBUG] [VALIDATOR ] KRC: All expected trading days are present. [2025-10-20T19:36:39.963695] [DEBUG] [VALIDATOR ] KRC: Primary data validation passed. [2025-10-20T19:36:39.964726] [INFO ] [MAIN ] KRC: Fetched primary data is valid. [2025-10-20T19:36:39.964788] [DEBUG] [LOCAL ] KRC: Saving primary data. [2025-10-20T19:36:39.966286] [DEBUG] [LOCAL ] KRC: Trimmed 70047 records to 48217 (keeping 200 days of data) [2025-10-20T19:36:40.253229] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-01. [2025-10-20T19:36:40.268771] [DEBUG] [LOCAL ] KRC: Primary data saved (48217 records). [2025-10-20T19:36:40.268917] [INFO ] [MAIN ] KRC: Full history saved. [2025-10-20T19:36:40.269053] [INFO ] [MAIN ] Update Progress: 1217/2367 symbols completed. KRC took 35.9s. Estimated remaining: 2h 27m 59s. Rate: 7.8 symbols/min [2025-10-20T19:36:40.436805] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T19:36:40.481047] [INFO ] [MAIN ] KSS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:40.532562] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T19:36:40.556744] [DEBUG] [LOCAL ] KSS: Loaded 54372 existing records from primary storage. [2025-10-20T19:36:40.557097] [INFO ] [MAIN ] KSS: Loaded 54372 existing records. [2025-10-20T19:36:40.731382] [WARN ] [MAIN ] KSS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:40.731597] [INFO ] [MAIN ] KSS: Wiping existing primary data before full refresh. [2025-10-20T19:36:40.731669] [INFO ] [LOCAL ] KSS: Starting deletion of primary data. [2025-10-20T19:36:40.731903] [DEBUG] [LOCAL ] KSS: Deleted primary data file. [2025-10-20T19:36:40.731960] [INFO ] [MAIN ] KSS: Starting full primary refresh. [2025-10-20T19:36:41.062972] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-02. [2025-10-20T19:36:41.082713] [INFO ] [REMOTE ] KSS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:41.844510] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:36:41.852529] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-07. [2025-10-20T19:36:41.894552] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T19:36:42.651382] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-02. [2025-10-20T19:36:43.451411] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-03. [2025-10-20T19:36:43.495189] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:36:44.118378] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:36:44.251362] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-03. [2025-10-20T19:36:44.568724] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:36:45.051384] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-04. [2025-10-20T19:36:45.851408] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-08. [2025-10-20T19:36:46.097339] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:36:46.255188] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:36:46.651348] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-01. [2025-10-20T19:36:47.451373] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-05. [2025-10-20T19:36:47.486048] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:36:48.251393] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-09. [2025-10-20T19:36:48.611606] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T19:36:48.651504] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T19:36:49.051381] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-06. [2025-10-20T19:36:49.599600] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:36:49.851377] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-02. [2025-10-20T19:36:50.071940] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:36:50.651375] [INFO ] [REMOTE ] KRG: Fetching primary data for 2025-10. [2025-10-20T19:36:51.451394] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-07. [2025-10-20T19:36:51.749819] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:36:52.245217] [INFO ] [REMOTE ] KRG: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T19:36:52.252691] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-03. [2025-10-20T19:36:52.255084] [INFO ] [REMOTE ] KRG: Full fetch complete (71323 records). [2025-10-20T19:36:52.255365] [DEBUG] [VALIDATOR ] KRG: Validating primary data. [2025-10-20T19:36:52.674039] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T19:36:52.674393] [DEBUG] [VALIDATOR ] KRG: All expected trading days are present. [2025-10-20T19:36:52.680091] [DEBUG] [VALIDATOR ] KRG: Primary data validation passed. [2025-10-20T19:36:52.681701] [INFO ] [MAIN ] KRG: Fetched primary data is valid. [2025-10-20T19:36:52.681789] [DEBUG] [LOCAL ] KRG: Saving primary data. [2025-10-20T19:36:52.683420] [DEBUG] [LOCAL ] KRG: Trimmed 71323 records to 49298 (keeping 200 days of data) [2025-10-20T19:36:52.977464] [DEBUG] [LOCAL ] KRG: Primary data saved (49298 records). [2025-10-20T19:36:52.977606] [INFO ] [MAIN ] KRG: Full history saved. [2025-10-20T19:36:52.978396] [INFO ] [MAIN ] Update Progress: 1218/2367 symbols completed. KRG took 32.0s. Estimated remaining: 2h 27m 56s. Rate: 7.8 symbols/min [2025-10-20T19:36:53.056716] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-04. [2025-10-20T19:36:53.152293] [INFO ] [MAIN ] KT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:36:53.197344] [DEBUG] [LOCAL ] KT: Loaded 46205 existing records from primary storage. [2025-10-20T19:36:53.197673] [INFO ] [MAIN ] KT: Loaded 46205 existing records. [2025-10-20T19:36:53.378544] [WARN ] [MAIN ] KT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:36:53.378811] [INFO ] [MAIN ] KT: Wiping existing primary data before full refresh. [2025-10-20T19:36:53.378861] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-10-20T19:36:53.379054] [DEBUG] [LOCAL ] KT: Deleted primary data file. [2025-10-20T19:36:53.379110] [INFO ] [MAIN ] KT: Starting full primary refresh. [2025-10-20T19:36:53.742564] [INFO ] [REMOTE ] KT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:36:53.851381] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-01. [2025-10-20T19:36:54.394567] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T19:36:54.651398] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-08. [2025-10-20T19:36:54.700816] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:36:55.432013] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:36:55.451338] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-05. [2025-10-20T19:36:55.966354] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:36:56.251432] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-09. [2025-10-20T19:36:56.967812] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:36:57.051403] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-02. [2025-10-20T19:36:57.071302] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T19:36:57.851402] [INFO ] [REMOTE ] KRO: Fetching primary data for 2025-10. [2025-10-20T19:36:58.626282] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:36:58.651334] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-03. [2025-10-20T19:36:59.276799] [INFO ] [REMOTE ] KRO: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:36:59.281007] [INFO ] [REMOTE ] KRO: Full fetch complete (35287 records). [2025-10-20T19:36:59.281093] [DEBUG] [VALIDATOR ] KRO: Validating primary data. [2025-10-20T19:36:59.454445] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-06. [2025-10-20T19:36:59.659630] [DEBUG] [VALIDATOR ] KRO: All expected trading days are present. [2025-10-20T19:36:59.659765] [DEBUG] [VALIDATOR ] KRO: Primary data validation passed. [2025-10-20T19:36:59.660516] [INFO ] [MAIN ] KRO: Fetched primary data is valid. [2025-10-20T19:36:59.660558] [DEBUG] [LOCAL ] KRO: Saving primary data. [2025-10-20T19:36:59.661366] [DEBUG] [LOCAL ] KRO: Trimmed 35287 records to 26644 (keeping 200 days of data) [2025-10-20T19:36:59.812659] [DEBUG] [LOCAL ] KRO: Primary data saved (26644 records). [2025-10-20T19:36:59.812807] [INFO ] [MAIN ] KRO: Full history saved. [2025-10-20T19:36:59.812887] [INFO ] [MAIN ] Update Progress: 1219/2367 symbols completed. KRO took 22.5s. Estimated remaining: 2h 27m 47s. Rate: 7.8 symbols/min [2025-10-20T19:36:59.988183] [INFO ] [MAIN ] KTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:00.028697] [DEBUG] [LOCAL ] KTB: Loaded 41064 existing records from primary storage. [2025-10-20T19:37:00.029015] [INFO ] [MAIN ] KTB: Loaded 41064 existing records. [2025-10-20T19:37:00.215611] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:37:00.235022] [WARN ] [MAIN ] KTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:00.244355] [INFO ] [MAIN ] KTB: Wiping existing primary data before full refresh. [2025-10-20T19:37:00.244753] [INFO ] [LOCAL ] KTB: Starting deletion of primary data. [2025-10-20T19:37:00.245066] [DEBUG] [LOCAL ] KTB: Deleted primary data file. [2025-10-20T19:37:00.245123] [INFO ] [MAIN ] KTB: Starting full primary refresh. [2025-10-20T19:37:00.257677] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-04. [2025-10-20T19:37:00.599858] [INFO ] [REMOTE ] KTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:00.623128] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T19:37:01.051380] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-01. [2025-10-20T19:37:01.851425] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-07. [2025-10-20T19:37:02.417228] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:37:02.568388] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-04 (2.31 seconds). [2025-10-20T19:37:02.651373] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-05. [2025-10-20T19:37:03.451390] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-02. [2025-10-20T19:37:03.741998] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:37:04.251443] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-08. [2025-10-20T19:37:04.735533] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T19:37:04.828753] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:37:05.051411] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-03. [2025-10-20T19:37:05.236880] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:37:05.851373] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-06. [2025-10-20T19:37:06.362923] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T19:37:06.651411] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-04. [2025-10-20T19:37:07.451347] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-09. [2025-10-20T19:37:07.788198] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T19:37:08.224386] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T19:37:08.251359] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-05. [2025-10-20T19:37:08.522023] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T19:37:09.051409] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-07. [2025-10-20T19:37:09.785299] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:37:09.851340] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-06. [2025-10-20T19:37:10.651388] [INFO ] [REMOTE ] KRP: Fetching primary data for 2025-10. [2025-10-20T19:37:10.984639] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T19:37:11.022840] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:37:11.451426] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-04. [2025-10-20T19:37:12.189597] [INFO ] [REMOTE ] KRP: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T19:37:12.195402] [INFO ] [REMOTE ] KRP: Full fetch complete (55037 records). [2025-10-20T19:37:12.195599] [DEBUG] [VALIDATOR ] KRP: Validating primary data. [2025-10-20T19:37:12.262991] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-08. [2025-10-20T19:37:12.617432] [DEBUG] [VALIDATOR ] KRP: All expected trading days are present. [2025-10-20T19:37:12.617553] [DEBUG] [VALIDATOR ] KRP: Primary data validation passed. [2025-10-20T19:37:12.619153] [INFO ] [MAIN ] KRP: Fetched primary data is valid. [2025-10-20T19:37:12.619258] [DEBUG] [LOCAL ] KRP: Saving primary data. [2025-10-20T19:37:12.621010] [DEBUG] [LOCAL ] KRP: Trimmed 55037 records to 36203 (keeping 200 days of data) [2025-10-20T19:37:12.864404] [DEBUG] [LOCAL ] KRP: Primary data saved (36203 records). [2025-10-20T19:37:12.864615] [INFO ] [MAIN ] KRP: Full history saved. [2025-10-20T19:37:12.864754] [INFO ] [MAIN ] Update Progress: 1220/2367 symbols completed. KRP took 35.1s. Estimated remaining: 2h 27m 44s. Rate: 7.8 symbols/min [2025-10-20T19:37:13.050613] [INFO ] [MAIN ] KTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:13.051359] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-07. [2025-10-20T19:37:13.068036] [DEBUG] [LOCAL ] KTF: Loaded 11900 existing records from primary storage. [2025-10-20T19:37:13.068547] [INFO ] [MAIN ] KTF: Loaded 11900 existing records. [2025-10-20T19:37:13.264362] [WARN ] [MAIN ] KTF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:13.281309] [INFO ] [MAIN ] KTF: Wiping existing primary data before full refresh. [2025-10-20T19:37:13.293176] [INFO ] [LOCAL ] KTF: Starting deletion of primary data. [2025-10-20T19:37:13.293482] [DEBUG] [LOCAL ] KTF: Deleted primary data file. [2025-10-20T19:37:13.293737] [INFO ] [MAIN ] KTF: Starting full primary refresh. [2025-10-20T19:37:13.300831] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T19:37:13.689951] [INFO ] [REMOTE ] KTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:13.851475] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-01. [2025-10-20T19:37:14.485521] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T19:37:14.525334] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:37:14.651408] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-08. [2025-10-20T19:37:14.711049] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T19:37:15.451467] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-09. [2025-10-20T19:37:16.042469] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T19:37:16.251430] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-02. [2025-10-20T19:37:17.022856] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T19:37:17.051420] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-03. [2025-10-20T19:37:17.280329] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T19:37:17.845042] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T19:37:17.851331] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-04. [2025-10-20T19:37:18.651381] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-05. [2025-10-20T19:37:18.893275] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T19:37:19.451443] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-05. [2025-10-20T19:37:20.114154] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T19:37:20.189150] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T19:37:20.251337] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-06. [2025-10-20T19:37:21.051347] [INFO ] [REMOTE ] KSS: Fetching primary data for 2025-10. [2025-10-20T19:37:21.725562] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:37:21.851563] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-07. [2025-10-20T19:37:22.651368] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-06. [2025-10-20T19:37:22.900998] [INFO ] [REMOTE ] KSS: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:37:22.909363] [INFO ] [REMOTE ] KSS: Full fetch complete (77753 records). [2025-10-20T19:37:22.909544] [DEBUG] [VALIDATOR ] KSS: Validating primary data. [2025-10-20T19:37:23.339826] [DEBUG] [VALIDATOR ] KSS: All expected trading days are present. [2025-10-20T19:37:23.341240] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:37:23.340426] [DEBUG] [VALIDATOR ] KSS: Primary data validation passed. [2025-10-20T19:37:23.351073] [INFO ] [MAIN ] KSS: Fetched primary data is valid. [2025-10-20T19:37:23.351590] [DEBUG] [LOCAL ] KSS: Saving primary data. [2025-10-20T19:37:23.353176] [DEBUG] [LOCAL ] KSS: Trimmed 77753 records to 53591 (keeping 200 days of data) [2025-10-20T19:37:23.456314] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-08. [2025-10-20T19:37:23.475868] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:37:23.691886] [DEBUG] [LOCAL ] KSS: Primary data saved (53591 records). [2025-10-20T19:37:23.692038] [INFO ] [MAIN ] KSS: Full history saved. [2025-10-20T19:37:23.692128] [INFO ] [MAIN ] Update Progress: 1221/2367 symbols completed. KSS took 43.4s. Estimated remaining: 2h 27m 39s. Rate: 7.8 symbols/min [2025-10-20T19:37:23.857544] [INFO ] [MAIN ] KUKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:23.857727] [DEBUG] [LOCAL ] KUKE: No existing data file found in primary storage. [2025-10-20T19:37:23.857773] [WARN ] [MAIN ] KUKE: No existing data found, performing full refresh instead. [2025-10-20T19:37:23.857878] [INFO ] [MAIN ] KUKE: Wiping existing primary data before full refresh. [2025-10-20T19:37:23.857919] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-10-20T19:37:23.857958] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-10-20T19:37:23.857990] [INFO ] [MAIN ] KUKE: Starting full primary refresh. [2025-10-20T19:37:24.201601] [INFO ] [REMOTE ] KUKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:24.251368] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-01. [2025-10-20T19:37:25.034123] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:37:25.051338] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-09. [2025-10-20T19:37:25.195871] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:37:25.851373] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-02. [2025-10-20T19:37:26.596210] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:37:26.651386] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-03. [2025-10-20T19:37:26.725858] [INFO ] [REMOTE ] KT: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:37:27.451368] [INFO ] [REMOTE ] KT: Fetching primary data for 2025-10. [2025-10-20T19:37:27.557443] [DEBUG] [REMOTE ] KT: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:37:27.557765] [WARN ] [REMOTE ] KT: No data returned for month 2025-10 [2025-10-20T19:37:27.557857] [FATAL] [REMOTE ] KT: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:37:27.557971] [FATAL] [MAIN ] KT: Failed to fetch primary data. [2025-10-20T19:37:27.558048] [WARN ] [MAIN ] KT: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:37:27.558134] [INFO ] [LOCAL ] KT: Starting deletion of primary data. [2025-10-20T19:37:27.558211] [DEBUG] [LOCAL ] KT: No primary data file found to delete. [2025-10-20T19:37:27.558261] [INFO ] [LOCAL ] KT: Starting deletion of derivative data. [2025-10-20T19:37:27.558440] [DEBUG] [LOCAL ] KT: Deleted derivative data file. [2025-10-20T19:37:27.558986] [INFO ] [MAIN ] Update Progress: 1222/2367 symbols completed. KT took 34.6s. Estimated remaining: 2h 27m 28s. Rate: 7.8 symbols/min [2025-10-20T19:37:27.670015] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T19:37:27.742665] [INFO ] [MAIN ] KVUE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:27.794790] [DEBUG] [LOCAL ] KVUE: Loaded 54447 existing records from primary storage. [2025-10-20T19:37:27.795141] [INFO ] [MAIN ] KVUE: Loaded 54447 existing records. [2025-10-20T19:37:27.977116] [WARN ] [MAIN ] KVUE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:27.977375] [INFO ] [MAIN ] KVUE: Wiping existing primary data before full refresh. [2025-10-20T19:37:27.977466] [INFO ] [LOCAL ] KVUE: Starting deletion of primary data. [2025-10-20T19:37:27.977713] [DEBUG] [LOCAL ] KVUE: Deleted primary data file. [2025-10-20T19:37:27.977771] [INFO ] [MAIN ] KVUE: Starting full primary refresh. [2025-10-20T19:37:28.254332] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-04. [2025-10-20T19:37:28.333385] [INFO ] [REMOTE ] KVUE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:28.898544] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T19:37:29.051371] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-07. [2025-10-20T19:37:29.831028] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T19:37:29.851363] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-08. [2025-10-20T19:37:30.633035] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T19:37:30.651324] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-09. [2025-10-20T19:37:31.451392] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-01. [2025-10-20T19:37:31.557938] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T19:37:32.251376] [INFO ] [REMOTE ] KTF: Fetching primary data for 2025-10. [2025-10-20T19:37:33.009072] [INFO ] [REMOTE ] KTF: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T19:37:33.010823] [INFO ] [REMOTE ] KTF: Full fetch complete (17266 records). [2025-10-20T19:37:33.010891] [DEBUG] [VALIDATOR ] KTF: Validating primary data. [2025-10-20T19:37:33.056626] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-05. [2025-10-20T19:37:33.380322] [DEBUG] [VALIDATOR ] KTF: All expected trading days are present. [2025-10-20T19:37:33.380628] [DEBUG] [VALIDATOR ] KTF: Primary data validation passed. [2025-10-20T19:37:33.381052] [INFO ] [MAIN ] KTF: Fetched primary data is valid. [2025-10-20T19:37:33.392926] [DEBUG] [LOCAL ] KTF: Saving primary data. [2025-10-20T19:37:33.414846] [DEBUG] [LOCAL ] KTF: Trimmed 17266 records to 11669 (keeping 200 days of data) [2025-10-20T19:37:33.417213] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T19:37:33.499138] [DEBUG] [LOCAL ] KTF: Primary data saved (11669 records). [2025-10-20T19:37:33.499273] [INFO ] [MAIN ] KTF: Full history saved. [2025-10-20T19:37:33.499352] [INFO ] [MAIN ] Update Progress: 1223/2367 symbols completed. KTF took 20.6s. Estimated remaining: 2h 27m 19s. Rate: 7.8 symbols/min [2025-10-20T19:37:33.577774] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T19:37:33.672536] [INFO ] [MAIN ] KVYO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:33.720634] [DEBUG] [LOCAL ] KVYO: Loaded 51549 existing records from primary storage. [2025-10-20T19:37:33.721013] [INFO ] [MAIN ] KVYO: Loaded 51549 existing records. [2025-10-20T19:37:33.858242] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-02. [2025-10-20T19:37:33.897825] [WARN ] [MAIN ] KVYO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:33.898196] [INFO ] [MAIN ] KVYO: Wiping existing primary data before full refresh. [2025-10-20T19:37:33.898249] [INFO ] [LOCAL ] KVYO: Starting deletion of primary data. [2025-10-20T19:37:33.898467] [DEBUG] [LOCAL ] KVYO: Deleted primary data file. [2025-10-20T19:37:33.898523] [INFO ] [MAIN ] KVYO: Starting full primary refresh. [2025-10-20T19:37:34.245302] [INFO ] [REMOTE ] KVYO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:34.651430] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-01. [2025-10-20T19:37:35.451455] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-06. [2025-10-20T19:37:35.656247] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:37:35.931800] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T19:37:36.179630] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T19:37:36.251345] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-07. [2025-10-20T19:37:36.705815] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:37:37.051415] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-08. [2025-10-20T19:37:37.679955] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T19:37:37.851408] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-02. [2025-10-20T19:37:38.651371] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-09. [2025-10-20T19:37:39.417074] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:37:39.451337] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-03. [2025-10-20T19:37:40.124319] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:37:40.251384] [INFO ] [REMOTE ] KTB: Fetching primary data for 2025-10. [2025-10-20T19:37:40.933619] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:37:41.051443] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-03. [2025-10-20T19:37:41.851360] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-04. [2025-10-20T19:37:41.873208] [INFO ] [REMOTE ] KTB: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:37:41.878812] [INFO ] [REMOTE ] KTB: Full fetch complete (58258 records). [2025-10-20T19:37:41.878944] [DEBUG] [VALIDATOR ] KTB: Validating primary data. [2025-10-20T19:37:42.263958] [DEBUG] [VALIDATOR ] KTB: All expected trading days are present. [2025-10-20T19:37:42.264040] [DEBUG] [VALIDATOR ] KTB: Primary data validation passed. [2025-10-20T19:37:42.264898] [INFO ] [MAIN ] KTB: Fetched primary data is valid. [2025-10-20T19:37:42.264961] [DEBUG] [LOCAL ] KTB: Saving primary data. [2025-10-20T19:37:42.266208] [DEBUG] [LOCAL ] KTB: Trimmed 58258 records to 40517 (keeping 200 days of data) [2025-10-20T19:37:42.508341] [DEBUG] [LOCAL ] KTB: Primary data saved (40517 records). [2025-10-20T19:37:42.508455] [INFO ] [MAIN ] KTB: Full history saved. [2025-10-20T19:37:42.508524] [INFO ] [MAIN ] Update Progress: 1224/2367 symbols completed. KTB took 42.7s. Estimated remaining: 2h 27m 12s. Rate: 7.8 symbols/min [2025-10-20T19:37:42.679605] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-09. [2025-10-20T19:37:42.683229] [INFO ] [MAIN ] KW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:42.719625] [DEBUG] [LOCAL ] KW: Loaded 41091 existing records from primary storage. [2025-10-20T19:37:42.719928] [INFO ] [MAIN ] KW: Loaded 41091 existing records. [2025-10-20T19:37:42.925370] [WARN ] [MAIN ] KW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:42.925783] [INFO ] [MAIN ] KW: Wiping existing primary data before full refresh. [2025-10-20T19:37:42.926009] [INFO ] [LOCAL ] KW: Starting deletion of primary data. [2025-10-20T19:37:42.926413] [DEBUG] [LOCAL ] KW: Deleted primary data file. [2025-10-20T19:37:42.926484] [INFO ] [MAIN ] KW: Starting full primary refresh. [2025-10-20T19:37:43.062760] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T19:37:43.331606] [INFO ] [REMOTE ] KW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:43.422861] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:37:43.451327] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-04. [2025-10-20T19:37:43.546958] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:37:44.251441] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-05. [2025-10-20T19:37:45.051357] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-01. [2025-10-20T19:37:45.318152] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:37:45.621698] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:37:45.851375] [INFO ] [REMOTE ] KUKE: Fetching primary data for 2025-10. [2025-10-20T19:37:46.457695] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:37:46.651365] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-02. [2025-10-20T19:37:47.007308] [INFO ] [REMOTE ] KUKE: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:37:47.009692] [INFO ] [REMOTE ] KUKE: Full fetch complete (9164 records). [2025-10-20T19:37:47.009766] [DEBUG] [VALIDATOR ] KUKE: Validating primary data. [2025-10-20T19:37:47.195509] [WARN ] [VALIDATOR ] KUKE: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:37:47.195826] [FATAL] [MAIN ] KUKE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:37:47.195912] [WARN ] [MAIN ] KUKE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:37:47.195985] [INFO ] [LOCAL ] KUKE: Starting deletion of primary data. [2025-10-20T19:37:47.196050] [DEBUG] [LOCAL ] KUKE: No primary data file found to delete. [2025-10-20T19:37:47.196126] [INFO ] [LOCAL ] KUKE: Starting deletion of derivative data. [2025-10-20T19:37:47.196179] [DEBUG] [LOCAL ] KUKE: No derivative data file found to delete. [2025-10-20T19:37:47.196238] [INFO ] [MAIN ] Update Progress: 1225/2367 symbols completed. KUKE took 23.5s. Estimated remaining: 2h 27m 2s. Rate: 7.8 symbols/min [2025-10-20T19:37:47.371206] [INFO ] [MAIN ] KWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:47.388857] [DEBUG] [LOCAL ] KWR: Loaded 16242 existing records from primary storage. [2025-10-20T19:37:47.389085] [INFO ] [MAIN ] KWR: Loaded 16242 existing records. [2025-10-20T19:37:47.456570] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-06. [2025-10-20T19:37:47.576979] [WARN ] [MAIN ] KWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:47.577289] [INFO ] [MAIN ] KWR: Wiping existing primary data before full refresh. [2025-10-20T19:37:47.577387] [INFO ] [LOCAL ] KWR: Starting deletion of primary data. [2025-10-20T19:37:47.577628] [DEBUG] [LOCAL ] KWR: Deleted primary data file. [2025-10-20T19:37:47.577727] [INFO ] [MAIN ] KWR: Starting full primary refresh. [2025-10-20T19:37:47.886053] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:37:47.988209] [INFO ] [REMOTE ] KWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:37:48.251349] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-03. [2025-10-20T19:37:48.950198] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:37:49.051380] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-07. [2025-10-20T19:37:49.789362] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:37:49.851369] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-05. [2025-10-20T19:37:50.651372] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-01. [2025-10-20T19:37:50.735944] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T19:37:51.451379] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-08. [2025-10-20T19:37:51.678108] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T19:37:51.783623] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T19:37:52.251397] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-06. [2025-10-20T19:37:53.051351] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-02. [2025-10-20T19:37:53.136259] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T19:37:53.848623] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T19:37:53.851991] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-09. [2025-10-20T19:37:54.068167] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T19:37:54.651412] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-03. [2025-10-20T19:37:55.451418] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-07. [2025-10-20T19:37:55.548907] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T19:37:55.733591] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T19:37:56.251425] [INFO ] [REMOTE ] KVYO: Fetching primary data for 2025-10. [2025-10-20T19:37:57.051364] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-04. [2025-10-20T19:37:57.303903] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T19:37:57.851422] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-04. [2025-10-20T19:37:58.048152] [INFO ] [REMOTE ] KVYO: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:37:58.056009] [INFO ] [REMOTE ] KVYO: Full fetch complete (72146 records). [2025-10-20T19:37:58.056163] [DEBUG] [VALIDATOR ] KVYO: Validating primary data. [2025-10-20T19:37:58.452664] [DEBUG] [VALIDATOR ] KVYO: All expected trading days are present. [2025-10-20T19:37:58.452767] [DEBUG] [VALIDATOR ] KVYO: Primary data validation passed. [2025-10-20T19:37:58.453730] [INFO ] [MAIN ] KVYO: Fetched primary data is valid. [2025-10-20T19:37:58.453811] [DEBUG] [LOCAL ] KVYO: Saving primary data. [2025-10-20T19:37:58.479828] [DEBUG] [LOCAL ] KVYO: Trimmed 72146 records to 50869 (keeping 200 days of data) [2025-10-20T19:37:58.481465] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T19:37:58.658849] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-08. [2025-10-20T19:37:58.780122] [DEBUG] [LOCAL ] KVYO: Primary data saved (50869 records). [2025-10-20T19:37:58.780255] [INFO ] [MAIN ] KVYO: Full history saved. [2025-10-20T19:37:58.780331] [INFO ] [MAIN ] Update Progress: 1226/2367 symbols completed. KVYO took 25.3s. Estimated remaining: 2h 26m 58s. Rate: 7.8 symbols/min [2025-10-20T19:37:58.947985] [INFO ] [MAIN ] KYN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:37:58.959814] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:37:58.992929] [DEBUG] [LOCAL ] KYN: Loaded 30763 existing records from primary storage. [2025-10-20T19:37:58.993214] [INFO ] [MAIN ] KYN: Loaded 30763 existing records. [2025-10-20T19:37:59.163378] [WARN ] [MAIN ] KYN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:37:59.163597] [INFO ] [MAIN ] KYN: Wiping existing primary data before full refresh. [2025-10-20T19:37:59.163682] [INFO ] [LOCAL ] KYN: Starting deletion of primary data. [2025-10-20T19:37:59.163952] [DEBUG] [LOCAL ] KYN: Deleted primary data file. [2025-10-20T19:37:59.164023] [INFO ] [MAIN ] KYN: Starting full primary refresh. [2025-10-20T19:37:59.456477] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-05. [2025-10-20T19:37:59.513379] [INFO ] [REMOTE ] KYN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:00.251358] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-05. [2025-10-20T19:38:00.306386] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T19:38:00.390732] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T19:38:01.051384] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-09. [2025-10-20T19:38:01.851348] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-01. [2025-10-20T19:38:02.155413] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:38:02.651403] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-06. [2025-10-20T19:38:03.260348] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:38:03.272210] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:38:03.380955] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T19:38:03.451348] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-06. [2025-10-20T19:38:04.251381] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-02. [2025-10-20T19:38:04.695799] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T19:38:05.051377] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-07. [2025-10-20T19:38:05.385485] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:38:05.851422] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-07. [2025-10-20T19:38:06.243394] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:38:06.651406] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-08. [2025-10-20T19:38:06.921180] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T19:38:07.451438] [INFO ] [REMOTE ] KVUE: Fetching primary data for 2025-10. [2025-10-20T19:38:08.177601] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:38:08.251358] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-09. [2025-10-20T19:38:08.316103] [INFO ] [REMOTE ] KVUE: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:38:08.324161] [INFO ] [REMOTE ] KVUE: Full fetch complete (77842 records). [2025-10-20T19:38:08.324327] [DEBUG] [VALIDATOR ] KVUE: Validating primary data. [2025-10-20T19:38:08.710728] [DEBUG] [VALIDATOR ] KVUE: All expected trading days are present. [2025-10-20T19:38:08.710812] [DEBUG] [VALIDATOR ] KVUE: Primary data validation passed. [2025-10-20T19:38:08.711788] [INFO ] [MAIN ] KVUE: Fetched primary data is valid. [2025-10-20T19:38:08.711850] [DEBUG] [LOCAL ] KVUE: Saving primary data. [2025-10-20T19:38:08.713233] [DEBUG] [LOCAL ] KVUE: Trimmed 77842 records to 53667 (keeping 200 days of data) [2025-10-20T19:38:09.038316] [DEBUG] [LOCAL ] KVUE: Primary data saved (53667 records). [2025-10-20T19:38:09.038452] [INFO ] [MAIN ] KVUE: Full history saved. [2025-10-20T19:38:09.038527] [INFO ] [MAIN ] Update Progress: 1227/2367 symbols completed. KVUE took 41.5s. Estimated remaining: 2h 26m 52s. Rate: 7.8 symbols/min [2025-10-20T19:38:09.058928] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-03. [2025-10-20T19:38:09.207169] [INFO ] [MAIN ] L: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:09.245436] [DEBUG] [LOCAL ] L: Loaded 41844 existing records from primary storage. [2025-10-20T19:38:09.245741] [INFO ] [MAIN ] L: Loaded 41844 existing records. [2025-10-20T19:38:09.422502] [WARN ] [MAIN ] L: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:09.422715] [INFO ] [MAIN ] L: Wiping existing primary data before full refresh. [2025-10-20T19:38:09.422769] [INFO ] [LOCAL ] L: Starting deletion of primary data. [2025-10-20T19:38:09.422932] [DEBUG] [LOCAL ] L: Deleted primary data file. [2025-10-20T19:38:09.422969] [INFO ] [MAIN ] L: Starting full primary refresh. [2025-10-20T19:38:09.784167] [INFO ] [REMOTE ] L: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:09.804950] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T19:38:09.851333] [INFO ] [REMOTE ] L: Fetching primary data for 2025-01. [2025-10-20T19:38:10.416817] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:38:10.651374] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-08. [2025-10-20T19:38:11.396673] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T19:38:11.451431] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-04. [2025-10-20T19:38:11.622718] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T19:38:12.251485] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-09. [2025-10-20T19:38:12.856076] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:38:12.871602] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T19:38:13.051421] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-05. [2025-10-20T19:38:13.851370] [INFO ] [REMOTE ] KWR: Fetching primary data for 2025-10. [2025-10-20T19:38:14.339196] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T19:38:14.651489] [INFO ] [REMOTE ] KW: Fetching primary data for 2025-10. [2025-10-20T19:38:14.854082] [INFO ] [REMOTE ] KWR: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:38:14.856315] [INFO ] [REMOTE ] KWR: Full fetch complete (23302 records). [2025-10-20T19:38:14.856427] [DEBUG] [VALIDATOR ] KWR: Validating primary data. [2025-10-20T19:38:15.234082] [DEBUG] [VALIDATOR ] KWR: All expected trading days are present. [2025-10-20T19:38:15.234167] [DEBUG] [VALIDATOR ] KWR: Primary data validation passed. [2025-10-20T19:38:15.234721] [INFO ] [MAIN ] KWR: Fetched primary data is valid. [2025-10-20T19:38:15.234784] [DEBUG] [LOCAL ] KWR: Saving primary data. [2025-10-20T19:38:15.235565] [DEBUG] [LOCAL ] KWR: Trimmed 23302 records to 15952 (keeping 200 days of data) [2025-10-20T19:38:15.336094] [DEBUG] [LOCAL ] KWR: Primary data saved (15952 records). [2025-10-20T19:38:15.336233] [INFO ] [MAIN ] KWR: Full history saved. [2025-10-20T19:38:15.336311] [INFO ] [MAIN ] Update Progress: 1228/2367 symbols completed. KWR took 28.1s. Estimated remaining: 2h 26m 43s. Rate: 7.8 symbols/min [2025-10-20T19:38:15.456459] [INFO ] [REMOTE ] L: Fetching primary data for 2025-02. [2025-10-20T19:38:15.510019] [INFO ] [MAIN ] LAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:15.557572] [DEBUG] [LOCAL ] LAC: Loaded 53290 existing records from primary storage. [2025-10-20T19:38:15.557936] [INFO ] [MAIN ] LAC: Loaded 53290 existing records. [2025-10-20T19:38:15.734118] [WARN ] [MAIN ] LAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:15.734311] [INFO ] [MAIN ] LAC: Wiping existing primary data before full refresh. [2025-10-20T19:38:15.734357] [INFO ] [LOCAL ] LAC: Starting deletion of primary data. [2025-10-20T19:38:15.734618] [DEBUG] [LOCAL ] LAC: Deleted primary data file. [2025-10-20T19:38:15.734693] [INFO ] [MAIN ] LAC: Starting full primary refresh. [2025-10-20T19:38:16.101477] [INFO ] [REMOTE ] LAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:16.139147] [INFO ] [REMOTE ] KW: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:38:16.145751] [INFO ] [REMOTE ] KW: Full fetch complete (57310 records). [2025-10-20T19:38:16.145863] [DEBUG] [VALIDATOR ] KW: Validating primary data. [2025-10-20T19:38:16.256566] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-06. [2025-10-20T19:38:16.574206] [DEBUG] [VALIDATOR ] KW: All expected trading days are present. [2025-10-20T19:38:16.574283] [DEBUG] [VALIDATOR ] KW: Primary data validation passed. [2025-10-20T19:38:16.576003] [INFO ] [MAIN ] KW: Fetched primary data is valid. [2025-10-20T19:38:16.576065] [DEBUG] [LOCAL ] KW: Saving primary data. [2025-10-20T19:38:16.577199] [DEBUG] [LOCAL ] KW: Trimmed 57310 records to 40515 (keeping 200 days of data) [2025-10-20T19:38:16.807720] [DEBUG] [LOCAL ] KW: Primary data saved (40515 records). [2025-10-20T19:38:16.807928] [INFO ] [MAIN ] KW: Full history saved. [2025-10-20T19:38:16.808274] [INFO ] [MAIN ] Update Progress: 1229/2367 symbols completed. KW took 34.3s. Estimated remaining: 2h 26m 30s. Rate: 7.8 symbols/min [2025-10-20T19:38:16.981775] [INFO ] [MAIN ] LAD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:17.005631] [DEBUG] [LOCAL ] LAD: Loaded 23663 existing records from primary storage. [2025-10-20T19:38:17.005905] [INFO ] [MAIN ] LAD: Loaded 23663 existing records. [2025-10-20T19:38:17.060082] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-01. [2025-10-20T19:38:17.175498] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:38:17.227361] [WARN ] [MAIN ] LAD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:17.228775] [INFO ] [MAIN ] LAD: Wiping existing primary data before full refresh. [2025-10-20T19:38:17.228836] [INFO ] [LOCAL ] LAD: Starting deletion of primary data. [2025-10-20T19:38:17.229026] [DEBUG] [LOCAL ] LAD: Deleted primary data file. [2025-10-20T19:38:17.229063] [INFO ] [MAIN ] LAD: Starting full primary refresh. [2025-10-20T19:38:17.583144] [INFO ] [REMOTE ] LAD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:17.604151] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:38:17.851381] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-01. [2025-10-20T19:38:18.651379] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-07. [2025-10-20T19:38:19.153411] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:38:19.152612] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-10-20T19:38:19.451442] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-02. [2025-10-20T19:38:19.800587] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T19:38:20.251430] [INFO ] [REMOTE ] L: Fetching primary data for 2025-03. [2025-10-20T19:38:21.051388] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-08. [2025-10-20T19:38:21.299483] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T19:38:21.856470] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-02. [2025-10-20T19:38:21.870170] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:38:22.408848] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T19:38:22.651378] [INFO ] [REMOTE ] L: Fetching primary data for 2025-04. [2025-10-20T19:38:23.010417] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:38:23.451393] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-03. [2025-10-20T19:38:24.229908] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T19:38:24.251365] [INFO ] [REMOTE ] L: Fetching primary data for 2025-05. [2025-10-20T19:38:24.844214] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T19:38:25.051391] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-03. [2025-10-20T19:38:25.753176] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T19:38:25.851385] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-09. [2025-10-20T19:38:26.651380] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-04. [2025-10-20T19:38:27.023476] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T19:38:27.149850] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T19:38:27.451392] [INFO ] [REMOTE ] L: Fetching primary data for 2025-06. [2025-10-20T19:38:27.560417] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T19:38:28.251417] [INFO ] [REMOTE ] KYN: Fetching primary data for 2025-10. [2025-10-20T19:38:28.994216] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:38:29.051338] [INFO ] [REMOTE ] L: Fetching primary data for 2025-07. [2025-10-20T19:38:29.566988] [INFO ] [REMOTE ] KYN: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T19:38:29.572054] [INFO ] [REMOTE ] KYN: Full fetch complete (44449 records). [2025-10-20T19:38:29.572165] [DEBUG] [VALIDATOR ] KYN: Validating primary data. [2025-10-20T19:38:29.856613] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-05. [2025-10-20T19:38:29.978015] [DEBUG] [VALIDATOR ] KYN: All expected trading days are present. [2025-10-20T19:38:29.978126] [DEBUG] [VALIDATOR ] KYN: Primary data validation passed. [2025-10-20T19:38:29.979572] [INFO ] [MAIN ] KYN: Fetched primary data is valid. [2025-10-20T19:38:29.979661] [DEBUG] [LOCAL ] KYN: Saving primary data. [2025-10-20T19:38:29.981024] [DEBUG] [LOCAL ] KYN: Trimmed 44449 records to 30326 (keeping 200 days of data) [2025-10-20T19:38:30.178550] [DEBUG] [LOCAL ] KYN: Primary data saved (30326 records). [2025-10-20T19:38:30.178801] [INFO ] [MAIN ] KYN: Full history saved. [2025-10-20T19:38:30.178915] [INFO ] [MAIN ] Update Progress: 1230/2367 symbols completed. KYN took 31.4s. Estimated remaining: 2h 26m 27s. Rate: 7.8 symbols/min [2025-10-20T19:38:30.357194] [INFO ] [MAIN ] LADR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:30.402391] [DEBUG] [LOCAL ] LADR: Loaded 39947 existing records from primary storage. [2025-10-20T19:38:30.402866] [INFO ] [MAIN ] LADR: Loaded 39947 existing records. [2025-10-20T19:38:30.615329] [WARN ] [MAIN ] LADR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:30.644493] [INFO ] [MAIN ] LADR: Wiping existing primary data before full refresh. [2025-10-20T19:38:30.644678] [INFO ] [LOCAL ] LADR: Starting deletion of primary data. [2025-10-20T19:38:30.645066] [DEBUG] [LOCAL ] LADR: Deleted primary data file. [2025-10-20T19:38:30.645353] [INFO ] [MAIN ] LADR: Starting full primary refresh. [2025-10-20T19:38:30.659993] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-04. [2025-10-20T19:38:30.653476] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:38:31.079711] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T19:38:31.086981] [INFO ] [REMOTE ] LADR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:31.451448] [INFO ] [REMOTE ] L: Fetching primary data for 2025-08. [2025-10-20T19:38:32.251426] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-06. [2025-10-20T19:38:32.660130] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T19:38:32.958537] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:38:33.051435] [INFO ] [REMOTE ] L: Fetching primary data for 2025-09. [2025-10-20T19:38:33.713080] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T19:38:33.851476] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-01. [2025-10-20T19:38:34.651372] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-07. [2025-10-20T19:38:35.451384] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-05. [2025-10-20T19:38:35.732282] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-09 (2.68 seconds). [2025-10-20T19:38:36.251365] [INFO ] [REMOTE ] L: Fetching primary data for 2025-10. [2025-10-20T19:38:36.494196] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-01 (2.64 seconds). [2025-10-20T19:38:36.709739] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:38:37.051357] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-02. [2025-10-20T19:38:37.482144] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T19:38:37.851094] [INFO ] [REMOTE ] L: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T19:38:37.856792] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-08. [2025-10-20T19:38:37.858429] [INFO ] [REMOTE ] L: Full fetch complete (59436 records). [2025-10-20T19:38:37.858521] [DEBUG] [VALIDATOR ] L: Validating primary data. [2025-10-20T19:38:38.244143] [DEBUG] [VALIDATOR ] L: All expected trading days are present. [2025-10-20T19:38:38.244276] [DEBUG] [VALIDATOR ] L: Primary data validation passed. [2025-10-20T19:38:38.245166] [INFO ] [MAIN ] L: Fetched primary data is valid. [2025-10-20T19:38:38.245312] [DEBUG] [LOCAL ] L: Saving primary data. [2025-10-20T19:38:38.246567] [DEBUG] [LOCAL ] L: Trimmed 59436 records to 41243 (keeping 200 days of data) [2025-10-20T19:38:38.482954] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T19:38:38.517604] [DEBUG] [LOCAL ] L: Primary data saved (41243 records). [2025-10-20T19:38:38.517736] [INFO ] [MAIN ] L: Full history saved. [2025-10-20T19:38:38.517813] [INFO ] [MAIN ] Update Progress: 1231/2367 symbols completed. L took 29.5s. Estimated remaining: 2h 26m 20s. Rate: 7.8 symbols/min [2025-10-20T19:38:38.657770] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-03. [2025-10-20T19:38:38.691010] [INFO ] [MAIN ] LANV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:38.695934] [DEBUG] [LOCAL ] LANV: Loaded 2807 existing records from primary storage. [2025-10-20T19:38:38.696090] [INFO ] [MAIN ] LANV: Loaded 2807 existing records. [2025-10-20T19:38:38.875747] [WARN ] [MAIN ] LANV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:38.875940] [INFO ] [MAIN ] LANV: Wiping existing primary data before full refresh. [2025-10-20T19:38:38.875990] [INFO ] [LOCAL ] LANV: Starting deletion of primary data. [2025-10-20T19:38:38.876118] [DEBUG] [LOCAL ] LANV: Deleted primary data file. [2025-10-20T19:38:38.876158] [INFO ] [MAIN ] LANV: Starting full primary refresh. [2025-10-20T19:38:39.209918] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:38:39.252749] [INFO ] [REMOTE ] LANV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:39.451372] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-01. [2025-10-20T19:38:39.848020] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-01 (0.40 seconds). [2025-10-20T19:38:40.107818] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:38:40.251415] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-02. [2025-10-20T19:38:40.731937] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T19:38:41.051375] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-06. [2025-10-20T19:38:41.851421] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-09. [2025-10-20T19:38:42.651379] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-04. [2025-10-20T19:38:42.909395] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:38:42.943077] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:38:43.451376] [INFO ] [REMOTE ] LAD: Fetching primary data for 2025-10. [2025-10-20T19:38:44.254765] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-03. [2025-10-20T19:38:44.270780] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:38:44.315135] [INFO ] [REMOTE ] LAD: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:38:44.318977] [INFO ] [REMOTE ] LAD: Full fetch complete (35396 records). [2025-10-20T19:38:44.319113] [DEBUG] [VALIDATOR ] LAD: Validating primary data. [2025-10-20T19:38:44.685551] [DEBUG] [VALIDATOR ] LAD: All expected trading days are present. [2025-10-20T19:38:44.685688] [DEBUG] [VALIDATOR ] LAD: Primary data validation passed. [2025-10-20T19:38:44.687162] [INFO ] [MAIN ] LAD: Fetched primary data is valid. [2025-10-20T19:38:44.687258] [DEBUG] [LOCAL ] LAD: Saving primary data. [2025-10-20T19:38:44.688138] [DEBUG] [LOCAL ] LAD: Trimmed 35396 records to 23301 (keeping 200 days of data) [2025-10-20T19:38:44.778421] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T19:38:44.832792] [DEBUG] [LOCAL ] LAD: Primary data saved (23301 records). [2025-10-20T19:38:44.832908] [INFO ] [MAIN ] LAD: Full history saved. [2025-10-20T19:38:44.832983] [INFO ] [MAIN ] Update Progress: 1232/2367 symbols completed. LAD took 28.0s. Estimated remaining: 2h 26m 11s. Rate: 7.8 symbols/min [2025-10-20T19:38:45.003569] [INFO ] [MAIN ] LAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:45.003783] [DEBUG] [LOCAL ] LAR: No existing data file found in primary storage. [2025-10-20T19:38:45.003833] [WARN ] [MAIN ] LAR: No existing data found, performing full refresh instead. [2025-10-20T19:38:45.003963] [INFO ] [MAIN ] LAR: Wiping existing primary data before full refresh. [2025-10-20T19:38:45.004031] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-10-20T19:38:45.004087] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-10-20T19:38:45.004120] [INFO ] [MAIN ] LAR: Starting full primary refresh. [2025-10-20T19:38:45.056571] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-05. [2025-10-20T19:38:45.358626] [INFO ] [REMOTE ] LAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:38:45.851364] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-04. [2025-10-20T19:38:46.298493] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T19:38:46.571363] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:38:46.651347] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-01. [2025-10-20T19:38:47.277482] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T19:38:47.451370] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-07. [2025-10-20T19:38:48.251377] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-05. [2025-10-20T19:38:49.051365] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-02. [2025-10-20T19:38:49.414579] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T19:38:49.851432] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-06. [2025-10-20T19:38:50.534985] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-07 (3.08 seconds). [2025-10-20T19:38:50.651360] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-08. [2025-10-20T19:38:51.400786] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-10-20T19:38:51.451336] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-03. [2025-10-20T19:38:51.525197] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:38:52.251454] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-06. [2025-10-20T19:38:52.668940] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T19:38:52.729758] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:38:53.032252] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T19:38:53.051335] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-04. [2025-10-20T19:38:53.851362] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-07. [2025-10-20T19:38:54.651440] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-07. [2025-10-20T19:38:54.728076] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:38:55.336966] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T19:38:55.376360] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T19:38:55.451399] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-05. [2025-10-20T19:38:56.251396] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-08. [2025-10-20T19:38:56.800038] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T19:38:57.051459] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-09. [2025-10-20T19:38:57.094172] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:38:57.851416] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-09. [2025-10-20T19:38:58.500242] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T19:38:58.651400] [INFO ] [REMOTE ] LANV: Fetching primary data for 2025-10. [2025-10-20T19:38:59.196231] [INFO ] [REMOTE ] LANV: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T19:38:59.197277] [INFO ] [REMOTE ] LANV: Full fetch complete (4246 records). [2025-10-20T19:38:59.197350] [DEBUG] [VALIDATOR ] LANV: Validating primary data. [2025-10-20T19:38:59.462938] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-08. [2025-10-20T19:38:59.516439] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-09 (2.46 seconds). [2025-10-20T19:38:59.606373] [DEBUG] [VALIDATOR ] LANV: All expected trading days are present. [2025-10-20T19:38:59.606448] [DEBUG] [VALIDATOR ] LANV: Primary data validation passed. [2025-10-20T19:38:59.606571] [INFO ] [MAIN ] LANV: Fetched primary data is valid. [2025-10-20T19:38:59.606608] [DEBUG] [LOCAL ] LANV: Saving primary data. [2025-10-20T19:38:59.606973] [DEBUG] [LOCAL ] LANV: Trimmed 4246 records to 2795 (keeping 200 days of data) [2025-10-20T19:38:59.621702] [DEBUG] [LOCAL ] LANV: Primary data saved (2795 records). [2025-10-20T19:38:59.621761] [INFO ] [MAIN ] LANV: Full history saved. [2025-10-20T19:38:59.621825] [INFO ] [MAIN ] Update Progress: 1233/2367 symbols completed. LANV took 21.1s. Estimated remaining: 2h 26m 10s. Rate: 7.8 symbols/min [2025-10-20T19:38:59.786164] [INFO ] [MAIN ] LAW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:38:59.801742] [DEBUG] [LOCAL ] LAW: Loaded 14898 existing records from primary storage. [2025-10-20T19:38:59.801966] [INFO ] [MAIN ] LAW: Loaded 14898 existing records. [2025-10-20T19:38:59.976151] [WARN ] [MAIN ] LAW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:38:59.976397] [INFO ] [MAIN ] LAW: Wiping existing primary data before full refresh. [2025-10-20T19:38:59.976445] [INFO ] [LOCAL ] LAW: Starting deletion of primary data. [2025-10-20T19:38:59.976580] [DEBUG] [LOCAL ] LAW: Deleted primary data file. [2025-10-20T19:38:59.976619] [INFO ] [MAIN ] LAW: Starting full primary refresh. [2025-10-20T19:39:00.256479] [INFO ] [REMOTE ] LAC: Fetching primary data for 2025-10. [2025-10-20T19:39:00.333695] [INFO ] [REMOTE ] LAW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:00.952893] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T19:39:01.051367] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-09. [2025-10-20T19:39:01.851386] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-06. [2025-10-20T19:39:02.573050] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:39:02.651421] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-01. [2025-10-20T19:39:03.175687] [INFO ] [REMOTE ] LAC: Incremental fetch complete for 2025-10 (2.92 seconds). [2025-10-20T19:39:03.184729] [INFO ] [REMOTE ] LAC: Full fetch complete (76140 records). [2025-10-20T19:39:03.184892] [DEBUG] [VALIDATOR ] LAC: Validating primary data. [2025-10-20T19:39:03.472273] [INFO ] [REMOTE ] LADR: Fetching primary data for 2025-10. [2025-10-20T19:39:03.604545] [DEBUG] [VALIDATOR ] LAC: All expected trading days are present. [2025-10-20T19:39:03.605198] [DEBUG] [VALIDATOR ] LAC: Primary data validation passed. [2025-10-20T19:39:03.606612] [INFO ] [MAIN ] LAC: Fetched primary data is valid. [2025-10-20T19:39:03.605533] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T19:39:03.606850] [DEBUG] [LOCAL ] LAC: Saving primary data. [2025-10-20T19:39:03.649558] [DEBUG] [LOCAL ] LAC: Trimmed 76140 records to 52554 (keeping 200 days of data) [2025-10-20T19:39:03.653148] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:39:03.972971] [DEBUG] [LOCAL ] LAC: Primary data saved (52554 records). [2025-10-20T19:39:03.973144] [INFO ] [MAIN ] LAC: Full history saved. [2025-10-20T19:39:03.973237] [INFO ] [MAIN ] Update Progress: 1234/2367 symbols completed. LAC took 48.6s. Estimated remaining: 2h 25m 59s. Rate: 7.8 symbols/min [2025-10-20T19:39:04.162177] [INFO ] [MAIN ] LAZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:04.203381] [DEBUG] [LOCAL ] LAZ: Loaded 45283 existing records from primary storage. [2025-10-20T19:39:04.203746] [INFO ] [MAIN ] LAZ: Loaded 45283 existing records. [2025-10-20T19:39:04.257599] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-07. [2025-10-20T19:39:04.391104] [WARN ] [MAIN ] LAZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:04.391323] [INFO ] [MAIN ] LAZ: Wiping existing primary data before full refresh. [2025-10-20T19:39:04.391386] [INFO ] [LOCAL ] LAZ: Starting deletion of primary data. [2025-10-20T19:39:04.391630] [DEBUG] [LOCAL ] LAZ: Deleted primary data file. [2025-10-20T19:39:04.391744] [INFO ] [MAIN ] LAZ: Starting full primary refresh. [2025-10-20T19:39:04.745530] [INFO ] [REMOTE ] LAZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:05.051389] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-02. [2025-10-20T19:39:05.214928] [INFO ] [REMOTE ] LADR: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:39:05.221290] [INFO ] [REMOTE ] LADR: Full fetch complete (55626 records). [2025-10-20T19:39:05.221414] [DEBUG] [VALIDATOR ] LADR: Validating primary data. [2025-10-20T19:39:05.601202] [DEBUG] [VALIDATOR ] LADR: All expected trading days are present. [2025-10-20T19:39:05.601302] [DEBUG] [VALIDATOR ] LADR: Primary data validation passed. [2025-10-20T19:39:05.602119] [INFO ] [MAIN ] LADR: Fetched primary data is valid. [2025-10-20T19:39:05.602179] [DEBUG] [LOCAL ] LADR: Saving primary data. [2025-10-20T19:39:05.603258] [DEBUG] [LOCAL ] LADR: Trimmed 55626 records to 39435 (keeping 200 days of data) [2025-10-20T19:39:05.838094] [DEBUG] [LOCAL ] LADR: Primary data saved (39435 records). [2025-10-20T19:39:05.838226] [INFO ] [MAIN ] LADR: Full history saved. [2025-10-20T19:39:05.838299] [INFO ] [MAIN ] Update Progress: 1235/2367 symbols completed. LADR took 35.7s. Estimated remaining: 2h 25m 46s. Rate: 7.8 symbols/min [2025-10-20T19:39:05.859754] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-01. [2025-10-20T19:39:06.017780] [INFO ] [MAIN ] LB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:06.026277] [DEBUG] [LOCAL ] LB: No existing data file found in primary storage. [2025-10-20T19:39:06.027945] [WARN ] [MAIN ] LB: No existing data found, performing full refresh instead. [2025-10-20T19:39:06.028295] [INFO ] [MAIN ] LB: Wiping existing primary data before full refresh. [2025-10-20T19:39:06.029410] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T19:39:06.029469] [INFO ] [LOCAL ] LB: Starting deletion of primary data. [2025-10-20T19:39:06.033739] [DEBUG] [LOCAL ] LB: No primary data file found to delete. [2025-10-20T19:39:06.033798] [INFO ] [MAIN ] LB: Starting full primary refresh. [2025-10-20T19:39:06.345128] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-07 (2.09 seconds). [2025-10-20T19:39:06.432789] [INFO ] [REMOTE ] LB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:06.651363] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-03. [2025-10-20T19:39:07.430370] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T19:39:07.451345] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-08. [2025-10-20T19:39:07.546179] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:39:08.251383] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-04. [2025-10-20T19:39:09.052896] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-02. [2025-10-20T19:39:09.053718] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:39:09.232707] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:39:09.851406] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-01. [2025-10-20T19:39:10.320729] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:39:10.651378] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-05. [2025-10-20T19:39:11.408934] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:39:11.451365] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-03. [2025-10-20T19:39:11.586737] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T19:39:12.251428] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-06. [2025-10-20T19:39:13.051401] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-09. [2025-10-20T19:39:13.187877] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T19:39:13.219898] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T19:39:13.851355] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-07. [2025-10-20T19:39:14.651477] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-04. [2025-10-20T19:39:14.747602] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:39:14.781849] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:39:15.451425] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-08. [2025-10-20T19:39:16.256206] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-02. [2025-10-20T19:39:16.262276] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:39:16.427857] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T19:39:17.051419] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-09. [2025-10-20T19:39:17.756936] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:39:17.829009] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:39:17.851398] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-05. [2025-10-20T19:39:18.651402] [INFO ] [REMOTE ] LAW: Fetching primary data for 2025-10. [2025-10-20T19:39:19.465878] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-03. [2025-10-20T19:39:19.499844] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T19:39:19.564394] [INFO ] [REMOTE ] LAW: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:39:19.567493] [INFO ] [REMOTE ] LAW: Full fetch complete (20914 records). [2025-10-20T19:39:19.567624] [DEBUG] [VALIDATOR ] LAW: Validating primary data. [2025-10-20T19:39:19.933754] [DEBUG] [VALIDATOR ] LAW: All expected trading days are present. [2025-10-20T19:39:19.933840] [DEBUG] [VALIDATOR ] LAW: Primary data validation passed. [2025-10-20T19:39:19.934294] [INFO ] [MAIN ] LAW: Fetched primary data is valid. [2025-10-20T19:39:19.934332] [DEBUG] [LOCAL ] LAW: Saving primary data. [2025-10-20T19:39:19.934979] [DEBUG] [LOCAL ] LAW: Trimmed 20914 records to 14803 (keeping 200 days of data) [2025-10-20T19:39:20.021210] [DEBUG] [LOCAL ] LAW: Primary data saved (14803 records). [2025-10-20T19:39:20.021374] [INFO ] [MAIN ] LAW: Full history saved. [2025-10-20T19:39:20.021461] [INFO ] [MAIN ] Update Progress: 1236/2367 symbols completed. LAW took 20.4s. Estimated remaining: 2h 25m 44s. Rate: 7.8 symbols/min [2025-10-20T19:39:20.220466] [INFO ] [MAIN ] LBRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:20.251386] [INFO ] [REMOTE ] LAR: Fetching primary data for 2025-10. [2025-10-20T19:39:20.275879] [DEBUG] [LOCAL ] LBRT: Loaded 53738 existing records from primary storage. [2025-10-20T19:39:20.276223] [INFO ] [MAIN ] LBRT: Loaded 53738 existing records. [2025-10-20T19:39:20.491126] [WARN ] [MAIN ] LBRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:20.491338] [INFO ] [MAIN ] LBRT: Wiping existing primary data before full refresh. [2025-10-20T19:39:20.491384] [INFO ] [LOCAL ] LBRT: Starting deletion of primary data. [2025-10-20T19:39:20.491939] [DEBUG] [LOCAL ] LBRT: Deleted primary data file. [2025-10-20T19:39:20.491994] [INFO ] [MAIN ] LBRT: Starting full primary refresh. [2025-10-20T19:39:20.842736] [INFO ] [REMOTE ] LBRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:21.059009] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-06. [2025-10-20T19:39:21.081695] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:39:21.851359] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-04. [2025-10-20T19:39:22.325755] [INFO ] [REMOTE ] LAR: Incremental fetch complete for 2025-10 (2.07 seconds). [2025-10-20T19:39:22.333872] [INFO ] [REMOTE ] LAR: Full fetch complete (61701 records). [2025-10-20T19:39:22.333998] [DEBUG] [VALIDATOR ] LAR: Validating primary data. [2025-10-20T19:39:22.555904] [WARN ] [VALIDATOR ] LAR: Validation failed - insufficient historical data. Have 185 unique days, need 200 days [2025-10-20T19:39:22.556587] [FATAL] [MAIN ] LAR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:39:22.556759] [WARN ] [MAIN ] LAR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:39:22.556841] [INFO ] [LOCAL ] LAR: Starting deletion of primary data. [2025-10-20T19:39:22.556965] [DEBUG] [LOCAL ] LAR: No primary data file found to delete. [2025-10-20T19:39:22.557036] [INFO ] [LOCAL ] LAR: Starting deletion of derivative data. [2025-10-20T19:39:22.557091] [DEBUG] [LOCAL ] LAR: No derivative data file found to delete. [2025-10-20T19:39:22.557210] [INFO ] [MAIN ] Update Progress: 1237/2367 symbols completed. LAR took 37.7s. Estimated remaining: 2h 25m 31s. Rate: 7.8 symbols/min [2025-10-20T19:39:22.581996] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T19:39:22.651344] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-07. [2025-10-20T19:39:23.320288] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:39:23.451364] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-05. [2025-10-20T19:39:23.787963] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:39:24.251394] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-08. [2025-10-20T19:39:24.606768] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T19:39:25.051422] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-01. [2025-10-20T19:39:25.485518] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T19:39:25.851390] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-06. [2025-10-20T19:39:26.651459] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-09. [2025-10-20T19:39:26.900168] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:39:27.232908] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:39:27.451630] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-02. [2025-10-20T19:39:28.251364] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-07. [2025-10-20T19:39:28.320004] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T19:39:29.051395] [INFO ] [REMOTE ] LAZ: Fetching primary data for 2025-10. [2025-10-20T19:39:29.090399] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:39:29.851396] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-03. [2025-10-20T19:39:29.885436] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T19:39:30.565450] [INFO ] [REMOTE ] LAZ: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T19:39:30.571495] [INFO ] [REMOTE ] LAZ: Full fetch complete (65448 records). [2025-10-20T19:39:30.571609] [DEBUG] [VALIDATOR ] LAZ: Validating primary data. [2025-10-20T19:39:30.661383] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-08. [2025-10-20T19:39:30.956810] [DEBUG] [VALIDATOR ] LAZ: All expected trading days are present. [2025-10-20T19:39:30.956907] [DEBUG] [VALIDATOR ] LAZ: Primary data validation passed. [2025-10-20T19:39:30.957832] [INFO ] [MAIN ] LAZ: Fetched primary data is valid. [2025-10-20T19:39:30.957897] [DEBUG] [LOCAL ] LAZ: Saving primary data. [2025-10-20T19:39:30.959224] [DEBUG] [LOCAL ] LAZ: Trimmed 65448 records to 44543 (keeping 200 days of data) [2025-10-20T19:39:31.227449] [DEBUG] [LOCAL ] LAZ: Primary data saved (44543 records). [2025-10-20T19:39:31.227558] [INFO ] [MAIN ] LAZ: Full history saved. [2025-10-20T19:39:31.227636] [INFO ] [MAIN ] Update Progress: 1238/2367 symbols completed. LAZ took 27.3s. Estimated remaining: 2h 25m 25s. Rate: 7.8 symbols/min [2025-10-20T19:39:31.546060] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:39:31.557066] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-04. [2025-10-20T19:39:32.207302] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T19:39:32.251345] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-09. [2025-10-20T19:39:33.283941] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T19:39:33.296194] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-05. [2025-10-20T19:39:33.357751] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T19:39:33.851430] [INFO ] [REMOTE ] LB: Fetching primary data for 2025-10. [2025-10-20T19:39:35.030166] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:39:35.041185] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-06. [2025-10-20T19:39:35.320737] [INFO ] [REMOTE ] LB: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:39:35.326332] [INFO ] [REMOTE ] LB: Full fetch complete (55925 records). [2025-10-20T19:39:35.326471] [DEBUG] [VALIDATOR ] LB: Validating primary data. [2025-10-20T19:39:35.700131] [DEBUG] [VALIDATOR ] LB: All expected trading days are present. [2025-10-20T19:39:35.700206] [DEBUG] [VALIDATOR ] LB: Primary data validation passed. [2025-10-20T19:39:35.701013] [INFO ] [MAIN ] LB: Fetched primary data is valid. [2025-10-20T19:39:35.701074] [DEBUG] [LOCAL ] LB: Saving primary data. [2025-10-20T19:39:35.702115] [DEBUG] [LOCAL ] LB: Trimmed 55925 records to 37212 (keeping 200 days of data) [2025-10-20T19:39:35.956785] [DEBUG] [LOCAL ] LB: Primary data saved (37212 records). [2025-10-20T19:39:35.956994] [INFO ] [MAIN ] LB: Full history saved. [2025-10-20T19:39:35.957109] [INFO ] [MAIN ] Update Progress: 1239/2367 symbols completed. LB took 30.1s. Estimated remaining: 2h 25m 14s. Rate: 7.8 symbols/min [2025-10-20T19:39:36.613980] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T19:39:36.629153] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-07. [2025-10-20T19:39:38.392273] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T19:39:38.403783] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-08. [2025-10-20T19:39:40.034400] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:39:40.046493] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-09. [2025-10-20T19:39:41.906329] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T19:39:41.917775] [INFO ] [REMOTE ] LBRT: Fetching primary data for 2025-10. [2025-10-20T19:39:43.835533] [INFO ] [REMOTE ] LBRT: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T19:39:43.844171] [INFO ] [REMOTE ] LBRT: Full fetch complete (76910 records). [2025-10-20T19:39:43.844321] [DEBUG] [VALIDATOR ] LBRT: Validating primary data. [2025-10-20T19:39:44.240147] [DEBUG] [VALIDATOR ] LBRT: All expected trading days are present. [2025-10-20T19:39:44.240254] [DEBUG] [VALIDATOR ] LBRT: Primary data validation passed. [2025-10-20T19:39:44.241256] [INFO ] [MAIN ] LBRT: Fetched primary data is valid. [2025-10-20T19:39:44.241323] [DEBUG] [LOCAL ] LBRT: Saving primary data. [2025-10-20T19:39:44.242764] [DEBUG] [LOCAL ] LBRT: Trimmed 76910 records to 53003 (keeping 200 days of data) [2025-10-20T19:39:44.568221] [DEBUG] [LOCAL ] LBRT: Primary data saved (53003 records). [2025-10-20T19:39:44.568380] [INFO ] [MAIN ] LBRT: Full history saved. [2025-10-20T19:39:44.569191] [INFO ] [MAIN ] Update Progress: 1240/2367 symbols completed. LBRT took 24.5s. Estimated remaining: 2h 25m 7s. Rate: 7.8 symbols/min [2025-10-20T19:39:45.226481] [INFO ] [MAIN ] LDI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.255891] [INFO ] [MAIN ] LCII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.303552] [INFO ] [MAIN ] LDOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.308807] [INFO ] [MAIN ] LC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:39:45.348353] [DEBUG] [LOCAL ] LCII: Loaded 27730 existing records from primary storage. [2025-10-20T19:39:45.358860] [INFO ] [MAIN ] LCII: Loaded 27730 existing records. [2025-10-20T19:39:45.487461] [DEBUG] [LOCAL ] LDI: Loaded 45582 existing records from primary storage. [2025-10-20T19:39:45.500948] [INFO ] [MAIN ] LDI: Loaded 45582 existing records. [2025-10-20T19:39:45.553234] [DEBUG] [LOCAL ] LDOS: Loaded 45546 existing records from primary storage. [2025-10-20T19:39:45.638424] [INFO ] [MAIN ] LDOS: Loaded 45546 existing records. [2025-10-20T19:39:45.656989] [DEBUG] [LOCAL ] LC: Loaded 48927 existing records from primary storage. [2025-10-20T19:39:45.763429] [WARN ] [MAIN ] LCII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:45.797195] [INFO ] [MAIN ] LCII: Wiping existing primary data before full refresh. [2025-10-20T19:39:45.730317] [INFO ] [MAIN ] LC: Loaded 48927 existing records. [2025-10-20T19:39:45.813235] [INFO ] [LOCAL ] LCII: Starting deletion of primary data. [2025-10-20T19:39:45.967453] [DEBUG] [LOCAL ] LCII: Deleted primary data file. [2025-10-20T19:39:46.005725] [WARN ] [MAIN ] LDI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.031705] [INFO ] [MAIN ] LCII: Starting full primary refresh. [2025-10-20T19:39:46.031664] [INFO ] [MAIN ] LDI: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.140213] [WARN ] [MAIN ] LDOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.113005] [INFO ] [LOCAL ] LDI: Starting deletion of primary data. [2025-10-20T19:39:46.198090] [DEBUG] [LOCAL ] LDI: Deleted primary data file. [2025-10-20T19:39:46.192878] [INFO ] [MAIN ] LDOS: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.256827] [INFO ] [LOCAL ] LDOS: Starting deletion of primary data. [2025-10-20T19:39:46.287284] [INFO ] [MAIN ] LDI: Starting full primary refresh. [2025-10-20T19:39:46.287176] [DEBUG] [LOCAL ] LDOS: Deleted primary data file. [2025-10-20T19:39:46.290753] [WARN ] [MAIN ] LC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:39:46.301461] [INFO ] [MAIN ] LC: Wiping existing primary data before full refresh. [2025-10-20T19:39:46.307421] [INFO ] [MAIN ] LDOS: Starting full primary refresh. [2025-10-20T19:39:46.312758] [INFO ] [LOCAL ] LC: Starting deletion of primary data. [2025-10-20T19:39:46.456588] [DEBUG] [LOCAL ] LC: Deleted primary data file. [2025-10-20T19:39:46.528133] [INFO ] [MAIN ] LC: Starting full primary refresh. [2025-10-20T19:39:47.147579] [INFO ] [REMOTE ] LCII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.281904] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-01. [2025-10-20T19:39:47.527636] [INFO ] [REMOTE ] LDI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.614389] [INFO ] [REMOTE ] LDOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.603869] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-01. [2025-10-20T19:39:47.625796] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-01. [2025-10-20T19:39:47.706860] [INFO ] [REMOTE ] LC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:39:47.707157] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-01. [2025-10-20T19:39:48.559343] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:39:48.564877] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-02. [2025-10-20T19:39:48.961563] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:39:48.968409] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-02. [2025-10-20T19:39:49.076460] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:39:49.086314] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-02. [2025-10-20T19:39:49.398554] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:39:49.409799] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-02. [2025-10-20T19:39:49.733443] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:39:49.738867] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-03. [2025-10-20T19:39:50.428538] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T19:39:50.435766] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-03. [2025-10-20T19:39:50.822863] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:39:50.833184] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-03. [2025-10-20T19:39:50.992712] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:39:51.002736] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-03. [2025-10-20T19:39:51.026546] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T19:39:51.033472] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-04. [2025-10-20T19:39:51.970995] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:39:51.979897] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-04. [2025-10-20T19:39:52.453036] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:39:52.463696] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-04. [2025-10-20T19:39:52.600786] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T19:39:52.609037] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-05. [2025-10-20T19:39:52.679869] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:39:52.689943] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-04. [2025-10-20T19:39:53.104162] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:39:53.113437] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-05. [2025-10-20T19:39:53.886030] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T19:39:53.893741] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-06. [2025-10-20T19:39:54.183467] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T19:39:54.194170] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-05. [2025-10-20T19:39:54.724952] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:39:54.733249] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-06. [2025-10-20T19:39:54.854255] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T19:39:54.881892] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T19:39:55.451345] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-05. [2025-10-20T19:39:55.892703] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:39:55.985597] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T19:39:56.251400] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-06. [2025-10-20T19:39:57.051358] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-07. [2025-10-20T19:39:57.150299] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:39:57.851392] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-06. [2025-10-20T19:39:57.933327] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T19:39:58.286780] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:39:58.651412] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-07. [2025-10-20T19:39:59.346627] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T19:39:59.451401] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-08. [2025-10-20T19:40:00.251707] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-07. [2025-10-20T19:40:00.254227] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:40:00.816909] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:40:01.051368] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-08. [2025-10-20T19:40:01.851387] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-07. [2025-10-20T19:40:02.258802] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:40:02.651377] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-09. [2025-10-20T19:40:03.178014] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T19:40:03.451363] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-08. [2025-10-20T19:40:03.622409] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:40:03.734164] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T19:40:04.251400] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-09. [2025-10-20T19:40:05.051369] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-08. [2025-10-20T19:40:05.283353] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T19:40:05.851452] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-09. [2025-10-20T19:40:06.514625] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T19:40:06.564800] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:40:06.651383] [INFO ] [REMOTE ] LDI: Fetching primary data for 2025-10. [2025-10-20T19:40:07.451409] [INFO ] [REMOTE ] LCII: Fetching primary data for 2025-10. [2025-10-20T19:40:07.515536] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:40:08.251408] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-09. [2025-10-20T19:40:08.623042] [INFO ] [REMOTE ] LCII: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T19:40:08.627698] [INFO ] [REMOTE ] LCII: Full fetch complete (39104 records). [2025-10-20T19:40:08.627963] [DEBUG] [VALIDATOR ] LCII: Validating primary data. [2025-10-20T19:40:09.013263] [INFO ] [REMOTE ] LDI: Incremental fetch complete for 2025-10 (2.36 seconds). [2025-10-20T19:40:09.051310] [DEBUG] [VALIDATOR ] LCII: All expected trading days are present. [2025-10-20T19:40:09.051385] [INFO ] [REMOTE ] LC: Fetching primary data for 2025-10. [2025-10-20T19:40:09.051525] [DEBUG] [VALIDATOR ] LCII: Primary data validation passed. [2025-10-20T19:40:09.052359] [INFO ] [MAIN ] LCII: Fetched primary data is valid. [2025-10-20T19:40:09.053358] [DEBUG] [LOCAL ] LCII: Saving primary data. [2025-10-20T19:40:09.052464] [INFO ] [REMOTE ] LDI: Full fetch complete (61986 records). [2025-10-20T19:40:09.053460] [DEBUG] [VALIDATOR ] LDI: Validating primary data. [2025-10-20T19:40:09.097917] [DEBUG] [LOCAL ] LCII: Trimmed 39104 records to 27217 (keeping 200 days of data) [2025-10-20T19:40:09.550719] [DEBUG] [LOCAL ] LCII: Primary data saved (27217 records). [2025-10-20T19:40:09.617110] [INFO ] [MAIN ] LCII: Full history saved. [2025-10-20T19:40:09.617485] [DEBUG] [VALIDATOR ] LDI: All expected trading days are present. [2025-10-20T19:40:09.617788] [INFO ] [MAIN ] Update Progress: 1241/2367 symbols completed. LCII took 25.0s. Estimated remaining: 2h 25m 15s. Rate: 7.8 symbols/min [2025-10-20T19:40:09.623232] [DEBUG] [VALIDATOR ] LDI: Primary data validation passed. [2025-10-20T19:40:09.632226] [INFO ] [MAIN ] LDI: Fetched primary data is valid. [2025-10-20T19:40:09.632320] [DEBUG] [LOCAL ] LDI: Saving primary data. [2025-10-20T19:40:09.705451] [DEBUG] [LOCAL ] LDI: Trimmed 61986 records to 44983 (keeping 200 days of data) [2025-10-20T19:40:09.926963] [INFO ] [MAIN ] LDP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:10.063498] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T19:40:10.085080] [INFO ] [REMOTE ] LDOS: Fetching primary data for 2025-10. [2025-10-20T19:40:10.095488] [DEBUG] [LOCAL ] LDP: Loaded 12949 existing records from primary storage. [2025-10-20T19:40:10.099155] [INFO ] [MAIN ] LDP: Loaded 12949 existing records. [2025-10-20T19:40:10.252224] [DEBUG] [LOCAL ] LDI: Primary data saved (44983 records). [2025-10-20T19:40:10.269352] [INFO ] [MAIN ] LDI: Full history saved. [2025-10-20T19:40:10.269481] [INFO ] [MAIN ] Update Progress: 1242/2367 symbols completed. LDI took 25.7s. Estimated remaining: 2h 25m 1s. Rate: 7.8 symbols/min [2025-10-20T19:40:10.491383] [WARN ] [MAIN ] LDP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:10.525745] [INFO ] [MAIN ] LDP: Wiping existing primary data before full refresh. [2025-10-20T19:40:10.545808] [INFO ] [MAIN ] LEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:10.546159] [DEBUG] [LOCAL ] LEA: No existing data file found in primary storage. [2025-10-20T19:40:10.546253] [WARN ] [MAIN ] LEA: No existing data found, performing full refresh instead. [2025-10-20T19:40:10.546018] [INFO ] [LOCAL ] LDP: Starting deletion of primary data. [2025-10-20T19:40:10.546740] [INFO ] [MAIN ] LEA: Wiping existing primary data before full refresh. [2025-10-20T19:40:10.546808] [INFO ] [LOCAL ] LEA: Starting deletion of primary data. [2025-10-20T19:40:10.546854] [DEBUG] [LOCAL ] LEA: No primary data file found to delete. [2025-10-20T19:40:10.546885] [INFO ] [MAIN ] LEA: Starting full primary refresh. [2025-10-20T19:40:10.578125] [DEBUG] [LOCAL ] LDP: Deleted primary data file. [2025-10-20T19:40:10.627232] [INFO ] [MAIN ] LDP: Starting full primary refresh. [2025-10-20T19:40:11.250784] [INFO ] [REMOTE ] LC: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T19:40:11.302956] [INFO ] [REMOTE ] LEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:11.329478] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-01. [2025-10-20T19:40:11.341585] [INFO ] [REMOTE ] LC: Full fetch complete (70701 records). [2025-10-20T19:40:11.362615] [DEBUG] [VALIDATOR ] LC: Validating primary data. [2025-10-20T19:40:11.369834] [INFO ] [REMOTE ] LDP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:11.456556] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-01. [2025-10-20T19:40:11.833049] [DEBUG] [VALIDATOR ] LC: All expected trading days are present. [2025-10-20T19:40:11.833197] [DEBUG] [VALIDATOR ] LC: Primary data validation passed. [2025-10-20T19:40:11.835137] [INFO ] [MAIN ] LC: Fetched primary data is valid. [2025-10-20T19:40:11.835346] [DEBUG] [LOCAL ] LC: Saving primary data. [2025-10-20T19:40:11.854094] [INFO ] [REMOTE ] LDOS: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T19:40:11.860855] [INFO ] [REMOTE ] LDOS: Full fetch complete (66980 records). [2025-10-20T19:40:11.861199] [DEBUG] [VALIDATOR ] LDOS: Validating primary data. [2025-10-20T19:40:11.861102] [DEBUG] [LOCAL ] LC: Trimmed 70701 records to 48258 (keeping 200 days of data) [2025-10-20T19:40:12.556761] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:40:12.585596] [DEBUG] [VALIDATOR ] LDOS: All expected trading days are present. [2025-10-20T19:40:12.605095] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-02. [2025-10-20T19:40:12.600997] [DEBUG] [VALIDATOR ] LDOS: Primary data validation passed. [2025-10-20T19:40:12.606797] [INFO ] [MAIN ] LDOS: Fetched primary data is valid. [2025-10-20T19:40:12.641735] [DEBUG] [LOCAL ] LDOS: Saving primary data. [2025-10-20T19:40:12.659206] [DEBUG] [LOCAL ] LC: Primary data saved (48258 records). [2025-10-20T19:40:12.659526] [INFO ] [MAIN ] LC: Full history saved. [2025-10-20T19:40:12.659891] [INFO ] [MAIN ] Update Progress: 1243/2367 symbols completed. LC took 28.1s. Estimated remaining: 2h 24m 48s. Rate: 7.8 symbols/min [2025-10-20T19:40:12.660114] [DEBUG] [LOCAL ] LDOS: Trimmed 66980 records to 44754 (keeping 200 days of data) [2025-10-20T19:40:12.972618] [INFO ] [MAIN ] LEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:13.127686] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:40:13.137999] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-02. [2025-10-20T19:40:13.191987] [DEBUG] [LOCAL ] LEG: Loaded 50873 existing records from primary storage. [2025-10-20T19:40:13.206099] [INFO ] [MAIN ] LEG: Loaded 50873 existing records. [2025-10-20T19:40:13.319305] [DEBUG] [LOCAL ] LDOS: Primary data saved (44754 records). [2025-10-20T19:40:13.322213] [INFO ] [MAIN ] LDOS: Full history saved. [2025-10-20T19:40:13.322555] [INFO ] [MAIN ] Update Progress: 1244/2367 symbols completed. LDOS took 28.7s. Estimated remaining: 2h 24m 34s. Rate: 7.8 symbols/min [2025-10-20T19:40:13.540843] [WARN ] [MAIN ] LEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:13.551464] [INFO ] [MAIN ] LEG: Wiping existing primary data before full refresh. [2025-10-20T19:40:13.584229] [INFO ] [LOCAL ] LEG: Starting deletion of primary data. [2025-10-20T19:40:13.605693] [DEBUG] [LOCAL ] LEG: Deleted primary data file. [2025-10-20T19:40:13.605855] [INFO ] [MAIN ] LEG: Starting full primary refresh. [2025-10-20T19:40:13.605602] [INFO ] [MAIN ] LEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:13.721864] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:40:13.858126] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-03. [2025-10-20T19:40:14.019280] [INFO ] [REMOTE ] LEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:14.020915] [DEBUG] [LOCAL ] LEN: Loaded 53973 existing records from primary storage. [2025-10-20T19:40:14.021324] [INFO ] [MAIN ] LEN: Loaded 53973 existing records. [2025-10-20T19:40:14.201508] [WARN ] [MAIN ] LEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:14.201771] [INFO ] [MAIN ] LEN: Wiping existing primary data before full refresh. [2025-10-20T19:40:14.201820] [INFO ] [LOCAL ] LEN: Starting deletion of primary data. [2025-10-20T19:40:14.202035] [DEBUG] [LOCAL ] LEN: Deleted primary data file. [2025-10-20T19:40:14.202089] [INFO ] [MAIN ] LEN: Starting full primary refresh. [2025-10-20T19:40:14.573510] [INFO ] [REMOTE ] LEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:14.651354] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-01. [2025-10-20T19:40:14.723583] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T19:40:14.883465] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T19:40:15.451375] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-04. [2025-10-20T19:40:16.219810] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:40:16.251564] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T19:40:16.251687] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-05. [2025-10-20T19:40:17.051372] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-01. [2025-10-20T19:40:17.230359] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:40:17.851392] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-02. [2025-10-20T19:40:18.651432] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-06. [2025-10-20T19:40:18.870897] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T19:40:19.451419] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-02. [2025-10-20T19:40:19.540167] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T19:40:19.567865] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:40:20.251428] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-07. [2025-10-20T19:40:21.051345] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-03. [2025-10-20T19:40:21.131310] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T19:40:21.249305] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T19:40:21.851406] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-03. [2025-10-20T19:40:22.651351] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-03. [2025-10-20T19:40:22.727320] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T19:40:23.451373] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-04. [2025-10-20T19:40:23.482186] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:40:24.251361] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-04. [2025-10-20T19:40:24.312553] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:40:24.809427] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T19:40:25.051376] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-08. [2025-10-20T19:40:25.789372] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:40:25.851341] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-05. [2025-10-20T19:40:25.888399] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T19:40:26.651384] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-05. [2025-10-20T19:40:27.290540] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T19:40:27.451410] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-06. [2025-10-20T19:40:28.260733] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-04. [2025-10-20T19:40:28.265793] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:40:29.007926] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:40:29.051334] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-07. [2025-10-20T19:40:29.851352] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-06. [2025-10-20T19:40:30.245782] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:40:30.548522] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T19:40:30.651339] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-08. [2025-10-20T19:40:31.461330] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-05. [2025-10-20T19:40:31.485919] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:40:32.084042] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T19:40:32.251408] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-09. [2025-10-20T19:40:33.051413] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-07. [2025-10-20T19:40:33.404455] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:40:33.852288] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-06. [2025-10-20T19:40:33.859829] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T19:40:34.651372] [INFO ] [REMOTE ] LEA: Fetching primary data for 2025-10. [2025-10-20T19:40:34.753568] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:40:35.451382] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-08. [2025-10-20T19:40:35.695535] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T19:40:35.829756] [INFO ] [REMOTE ] LEA: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T19:40:35.835761] [INFO ] [REMOTE ] LEA: Full fetch complete (60683 records). [2025-10-20T19:40:35.835877] [DEBUG] [VALIDATOR ] LEA: Validating primary data. [2025-10-20T19:40:36.207857] [DEBUG] [VALIDATOR ] LEA: All expected trading days are present. [2025-10-20T19:40:36.207938] [DEBUG] [VALIDATOR ] LEA: Primary data validation passed. [2025-10-20T19:40:36.208809] [INFO ] [MAIN ] LEA: Fetched primary data is valid. [2025-10-20T19:40:36.208871] [DEBUG] [LOCAL ] LEA: Saving primary data. [2025-10-20T19:40:36.210069] [DEBUG] [LOCAL ] LEA: Trimmed 60683 records to 41313 (keeping 200 days of data) [2025-10-20T19:40:36.262476] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-07. [2025-10-20T19:40:36.462046] [DEBUG] [LOCAL ] LEA: Primary data saved (41313 records). [2025-10-20T19:40:36.462170] [INFO ] [MAIN ] LEA: Full history saved. [2025-10-20T19:40:36.462235] [INFO ] [MAIN ] Update Progress: 1245/2367 symbols completed. LEA took 26.2s. Estimated remaining: 2h 24m 40s. Rate: 7.8 symbols/min [2025-10-20T19:40:36.629284] [INFO ] [MAIN ] LEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:36.646747] [DEBUG] [LOCAL ] LEO: Loaded 16814 existing records from primary storage. [2025-10-20T19:40:36.646986] [INFO ] [MAIN ] LEO: Loaded 16814 existing records. [2025-10-20T19:40:36.850298] [WARN ] [MAIN ] LEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:36.850621] [INFO ] [MAIN ] LEO: Wiping existing primary data before full refresh. [2025-10-20T19:40:36.850773] [INFO ] [LOCAL ] LEO: Starting deletion of primary data. [2025-10-20T19:40:36.851073] [DEBUG] [LOCAL ] LEO: Deleted primary data file. [2025-10-20T19:40:36.851135] [INFO ] [MAIN ] LEO: Starting full primary refresh. [2025-10-20T19:40:37.040160] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:40:37.076405] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-09. [2025-10-20T19:40:37.266909] [INFO ] [REMOTE ] LEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:37.851367] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-01. [2025-10-20T19:40:37.963150] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T19:40:38.290218] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:40:38.651349] [INFO ] [REMOTE ] LDP: Fetching primary data for 2025-10. [2025-10-20T19:40:38.895198] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:40:39.451384] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-08. [2025-10-20T19:40:39.534054] [INFO ] [REMOTE ] LDP: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T19:40:39.536966] [INFO ] [REMOTE ] LDP: Full fetch complete (18779 records). [2025-10-20T19:40:39.537067] [DEBUG] [VALIDATOR ] LDP: Validating primary data. [2025-10-20T19:40:39.916991] [DEBUG] [VALIDATOR ] LDP: All expected trading days are present. [2025-10-20T19:40:39.917080] [DEBUG] [VALIDATOR ] LDP: Primary data validation passed. [2025-10-20T19:40:39.917532] [INFO ] [MAIN ] LDP: Fetched primary data is valid. [2025-10-20T19:40:39.917577] [DEBUG] [LOCAL ] LDP: Saving primary data. [2025-10-20T19:40:39.918261] [DEBUG] [LOCAL ] LDP: Trimmed 18779 records to 12697 (keeping 200 days of data) [2025-10-20T19:40:40.002051] [DEBUG] [LOCAL ] LDP: Primary data saved (12697 records). [2025-10-20T19:40:40.002195] [INFO ] [MAIN ] LDP: Full history saved. [2025-10-20T19:40:40.002277] [INFO ] [MAIN ] Update Progress: 1246/2367 symbols completed. LDP took 30.4s. Estimated remaining: 2h 24m 29s. Rate: 7.8 symbols/min [2025-10-20T19:40:40.184287] [INFO ] [MAIN ] LEVI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:40.238444] [DEBUG] [LOCAL ] LEVI: Loaded 52559 existing records from primary storage. [2025-10-20T19:40:40.238854] [INFO ] [MAIN ] LEVI: Loaded 52559 existing records. [2025-10-20T19:40:40.259941] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-02. [2025-10-20T19:40:40.427271] [WARN ] [MAIN ] LEVI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:40.427524] [INFO ] [MAIN ] LEVI: Wiping existing primary data before full refresh. [2025-10-20T19:40:40.427572] [INFO ] [LOCAL ] LEVI: Starting deletion of primary data. [2025-10-20T19:40:40.428086] [DEBUG] [LOCAL ] LEVI: Deleted primary data file. [2025-10-20T19:40:40.428143] [INFO ] [MAIN ] LEVI: Starting full primary refresh. [2025-10-20T19:40:40.782306] [INFO ] [REMOTE ] LEVI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:41.051364] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-09. [2025-10-20T19:40:41.352398] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T19:40:41.533541] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T19:40:41.851372] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-03. [2025-10-20T19:40:42.651389] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-09. [2025-10-20T19:40:42.833914] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:40:42.863340] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:40:43.451359] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-04. [2025-10-20T19:40:44.251406] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-01. [2025-10-20T19:40:44.478181] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T19:40:44.954947] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-09 (2.30 seconds). [2025-10-20T19:40:45.051366] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-05. [2025-10-20T19:40:45.851410] [INFO ] [REMOTE ] LEG: Fetching primary data for 2025-10. [2025-10-20T19:40:45.976080] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T19:40:45.975516] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:40:46.651377] [INFO ] [REMOTE ] LEN: Fetching primary data for 2025-10. [2025-10-20T19:40:47.451388] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-06. [2025-10-20T19:40:47.654881] [INFO ] [REMOTE ] LEG: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:40:47.662316] [INFO ] [REMOTE ] LEG: Full fetch complete (72913 records). [2025-10-20T19:40:47.662466] [DEBUG] [VALIDATOR ] LEG: Validating primary data. [2025-10-20T19:40:48.053439] [DEBUG] [VALIDATOR ] LEG: All expected trading days are present. [2025-10-20T19:40:48.053529] [DEBUG] [VALIDATOR ] LEG: Primary data validation passed. [2025-10-20T19:40:48.054475] [INFO ] [MAIN ] LEG: Fetched primary data is valid. [2025-10-20T19:40:48.054537] [DEBUG] [LOCAL ] LEG: Saving primary data. [2025-10-20T19:40:48.055966] [DEBUG] [LOCAL ] LEG: Trimmed 72913 records to 50094 (keeping 200 days of data) [2025-10-20T19:40:48.260296] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-02. [2025-10-20T19:40:48.383256] [DEBUG] [LOCAL ] LEG: Primary data saved (50094 records). [2025-10-20T19:40:48.383437] [INFO ] [MAIN ] LEG: Full history saved. [2025-10-20T19:40:48.383814] [INFO ] [MAIN ] Update Progress: 1247/2367 symbols completed. LEG took 35.7s. Estimated remaining: 2h 24m 22s. Rate: 7.8 symbols/min [2025-10-20T19:40:48.500894] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T19:40:48.577827] [INFO ] [MAIN ] LFT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:48.596984] [DEBUG] [LOCAL ] LFT: Loaded 18518 existing records from primary storage. [2025-10-20T19:40:48.597247] [INFO ] [MAIN ] LFT: Loaded 18518 existing records. [2025-10-20T19:40:48.776476] [WARN ] [MAIN ] LFT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:48.776742] [INFO ] [MAIN ] LFT: Wiping existing primary data before full refresh. [2025-10-20T19:40:48.776794] [INFO ] [LOCAL ] LFT: Starting deletion of primary data. [2025-10-20T19:40:48.777046] [DEBUG] [LOCAL ] LFT: Deleted primary data file. [2025-10-20T19:40:48.777086] [INFO ] [MAIN ] LFT: Starting full primary refresh. [2025-10-20T19:40:49.054773] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-07. [2025-10-20T19:40:49.148691] [INFO ] [REMOTE ] LFT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:49.170120] [INFO ] [REMOTE ] LEN: Incremental fetch complete for 2025-10 (2.52 seconds). [2025-10-20T19:40:49.177941] [INFO ] [REMOTE ] LEN: Full fetch complete (77214 records). [2025-10-20T19:40:49.178089] [DEBUG] [VALIDATOR ] LEN: Validating primary data. [2025-10-20T19:40:49.560612] [DEBUG] [VALIDATOR ] LEN: All expected trading days are present. [2025-10-20T19:40:49.560864] [DEBUG] [VALIDATOR ] LEN: Primary data validation passed. [2025-10-20T19:40:49.561890] [INFO ] [MAIN ] LEN: Fetched primary data is valid. [2025-10-20T19:40:49.561953] [DEBUG] [LOCAL ] LEN: Saving primary data. [2025-10-20T19:40:49.563422] [DEBUG] [LOCAL ] LEN: Trimmed 77214 records to 53202 (keeping 200 days of data) [2025-10-20T19:40:49.857486] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-01. [2025-10-20T19:40:49.914976] [DEBUG] [LOCAL ] LEN: Primary data saved (53202 records). [2025-10-20T19:40:49.915299] [INFO ] [MAIN ] LEN: Full history saved. [2025-10-20T19:40:49.915489] [INFO ] [MAIN ] Update Progress: 1248/2367 symbols completed. LEN took 36.6s. Estimated remaining: 2h 24m 8s. Rate: 7.8 symbols/min [2025-10-20T19:40:50.138251] [INFO ] [MAIN ] LGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:40:50.147959] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T19:40:50.148470] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T19:40:50.174572] [DEBUG] [LOCAL ] LGI: Loaded 10171 existing records from primary storage. [2025-10-20T19:40:50.174859] [INFO ] [MAIN ] LGI: Loaded 10171 existing records. [2025-10-20T19:40:50.382895] [WARN ] [MAIN ] LGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:40:50.383154] [INFO ] [MAIN ] LGI: Wiping existing primary data before full refresh. [2025-10-20T19:40:50.383229] [INFO ] [LOCAL ] LGI: Starting deletion of primary data. [2025-10-20T19:40:50.383484] [DEBUG] [LOCAL ] LGI: Deleted primary data file. [2025-10-20T19:40:50.383552] [INFO ] [MAIN ] LGI: Starting full primary refresh. [2025-10-20T19:40:50.652911] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-03. [2025-10-20T19:40:50.798437] [INFO ] [REMOTE ] LGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:40:50.858531] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:40:51.451403] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-02. [2025-10-20T19:40:52.251365] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-01. [2025-10-20T19:40:52.273847] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:40:52.469669] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:40:53.011983] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T19:40:53.051340] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-02. [2025-10-20T19:40:53.796096] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:40:53.851438] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-03. [2025-10-20T19:40:54.624892] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T19:40:54.651379] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-08. [2025-10-20T19:40:55.451360] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-04. [2025-10-20T19:40:55.562480] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T19:40:56.240882] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:40:56.251354] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-05. [2025-10-20T19:40:57.041824] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:40:57.051339] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-06. [2025-10-20T19:40:57.771675] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T19:40:57.851376] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-03. [2025-10-20T19:40:58.651362] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-04. [2025-10-20T19:40:58.739573] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T19:40:59.451377] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-09. [2025-10-20T19:41:00.251376] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-07. [2025-10-20T19:41:00.423851] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T19:41:00.526078] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T19:41:01.051417] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-05. [2025-10-20T19:41:01.249959] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T19:41:01.851395] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-08. [2025-10-20T19:41:02.651362] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-04. [2025-10-20T19:41:02.871558] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T19:41:03.083419] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T19:41:03.451372] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-06. [2025-10-20T19:41:03.779346] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T19:41:04.251422] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-09. [2025-10-20T19:41:04.988167] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:41:05.059150] [INFO ] [REMOTE ] LGI: Fetching primary data for 2025-10. [2025-10-20T19:41:05.077568] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T19:41:05.851353] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-05. [2025-10-20T19:41:05.982412] [INFO ] [REMOTE ] LGI: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:41:05.984500] [INFO ] [REMOTE ] LGI: Full fetch complete (14465 records). [2025-10-20T19:41:05.984571] [DEBUG] [VALIDATOR ] LGI: Validating primary data. [2025-10-20T19:41:06.358991] [DEBUG] [VALIDATOR ] LGI: All expected trading days are present. [2025-10-20T19:41:06.359094] [DEBUG] [VALIDATOR ] LGI: Primary data validation passed. [2025-10-20T19:41:06.359458] [INFO ] [MAIN ] LGI: Fetched primary data is valid. [2025-10-20T19:41:06.359499] [DEBUG] [LOCAL ] LGI: Saving primary data. [2025-10-20T19:41:06.360201] [DEBUG] [LOCAL ] LGI: Trimmed 14465 records to 10042 (keeping 200 days of data) [2025-10-20T19:41:06.422061] [DEBUG] [LOCAL ] LGI: Primary data saved (10042 records). [2025-10-20T19:41:06.422196] [INFO ] [MAIN ] LGI: Full history saved. [2025-10-20T19:41:06.422274] [INFO ] [MAIN ] Update Progress: 1249/2367 symbols completed. LGI took 16.5s. Estimated remaining: 2h 24m 9s. Rate: 7.8 symbols/min [2025-10-20T19:41:06.601955] [INFO ] [MAIN ] LH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:06.649388] [DEBUG] [LOCAL ] LH: Loaded 38017 existing records from primary storage. [2025-10-20T19:41:06.649685] [INFO ] [MAIN ] LH: Loaded 38017 existing records. [2025-10-20T19:41:06.660826] [INFO ] [REMOTE ] LEO: Fetching primary data for 2025-10. [2025-10-20T19:41:06.833273] [WARN ] [MAIN ] LH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:06.833596] [INFO ] [MAIN ] LH: Wiping existing primary data before full refresh. [2025-10-20T19:41:06.833735] [INFO ] [LOCAL ] LH: Starting deletion of primary data. [2025-10-20T19:41:06.833999] [DEBUG] [LOCAL ] LH: Deleted primary data file. [2025-10-20T19:41:06.834043] [INFO ] [MAIN ] LH: Starting full primary refresh. [2025-10-20T19:41:06.953574] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T19:41:07.208836] [INFO ] [REMOTE ] LH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:07.451390] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-01. [2025-10-20T19:41:07.631409] [INFO ] [REMOTE ] LEO: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:41:07.634348] [INFO ] [REMOTE ] LEO: Full fetch complete (26292 records). [2025-10-20T19:41:07.634423] [DEBUG] [VALIDATOR ] LEO: Validating primary data. [2025-10-20T19:41:08.025331] [DEBUG] [VALIDATOR ] LEO: All expected trading days are present. [2025-10-20T19:41:08.025436] [DEBUG] [VALIDATOR ] LEO: Primary data validation passed. [2025-10-20T19:41:08.026017] [INFO ] [MAIN ] LEO: Fetched primary data is valid. [2025-10-20T19:41:08.026079] [DEBUG] [LOCAL ] LEO: Saving primary data. [2025-10-20T19:41:08.026860] [DEBUG] [LOCAL ] LEO: Trimmed 26292 records to 16617 (keeping 200 days of data) [2025-10-20T19:41:08.122337] [DEBUG] [LOCAL ] LEO: Primary data saved (16617 records). [2025-10-20T19:41:08.122484] [INFO ] [MAIN ] LEO: Full history saved. [2025-10-20T19:41:08.122563] [INFO ] [MAIN ] Update Progress: 1250/2367 symbols completed. LEO took 31.7s. Estimated remaining: 2h 23m 55s. Rate: 7.8 symbols/min [2025-10-20T19:41:08.259123] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-07. [2025-10-20T19:41:08.291429] [INFO ] [MAIN ] LHX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:08.336112] [DEBUG] [LOCAL ] LHX: Loaded 49301 existing records from primary storage. [2025-10-20T19:41:08.336420] [INFO ] [MAIN ] LHX: Loaded 49301 existing records. [2025-10-20T19:41:08.509437] [WARN ] [MAIN ] LHX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:08.509686] [INFO ] [MAIN ] LHX: Wiping existing primary data before full refresh. [2025-10-20T19:41:08.509752] [INFO ] [LOCAL ] LHX: Starting deletion of primary data. [2025-10-20T19:41:08.510421] [DEBUG] [LOCAL ] LHX: Deleted primary data file. [2025-10-20T19:41:08.510478] [INFO ] [MAIN ] LHX: Starting full primary refresh. [2025-10-20T19:41:08.872586] [INFO ] [REMOTE ] LHX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:09.040332] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T19:41:09.051336] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-02. [2025-10-20T19:41:09.851495] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-01. [2025-10-20T19:41:10.143913] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:41:10.240804] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T19:41:10.651394] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-03. [2025-10-20T19:41:11.451368] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-08. [2025-10-20T19:41:11.550677] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T19:41:12.256905] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-06. [2025-10-20T19:41:12.281950] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T19:41:13.051346] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-02. [2025-10-20T19:41:13.113046] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:41:13.439394] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T19:41:13.851434] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-04. [2025-10-20T19:41:14.651387] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-07. [2025-10-20T19:41:14.845682] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T19:41:15.451386] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-09. [2025-10-20T19:41:15.502269] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T19:41:15.556243] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:41:16.251428] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-05. [2025-10-20T19:41:17.051421] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-03. [2025-10-20T19:41:17.111087] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:41:17.851377] [INFO ] [REMOTE ] LEVI: Fetching primary data for 2025-10. [2025-10-20T19:41:17.878629] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:41:18.651362] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-06. [2025-10-20T19:41:18.938737] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T19:41:19.451398] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-04. [2025-10-20T19:41:19.931518] [INFO ] [REMOTE ] LEVI: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T19:41:19.939742] [INFO ] [REMOTE ] LEVI: Full fetch complete (74962 records). [2025-10-20T19:41:19.939913] [DEBUG] [VALIDATOR ] LEVI: Validating primary data. [2025-10-20T19:41:20.262385] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-08. [2025-10-20T19:41:20.435952] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T19:41:20.467126] [DEBUG] [VALIDATOR ] LEVI: All expected trading days are present. [2025-10-20T19:41:20.467542] [DEBUG] [VALIDATOR ] LEVI: Primary data validation passed. [2025-10-20T19:41:20.468586] [INFO ] [MAIN ] LEVI: Fetched primary data is valid. [2025-10-20T19:41:20.468672] [DEBUG] [LOCAL ] LEVI: Saving primary data. [2025-10-20T19:41:20.470017] [DEBUG] [LOCAL ] LEVI: Trimmed 74962 records to 51821 (keeping 200 days of data) [2025-10-20T19:41:20.780232] [DEBUG] [LOCAL ] LEVI: Primary data saved (51821 records). [2025-10-20T19:41:20.780374] [INFO ] [MAIN ] LEVI: Full history saved. [2025-10-20T19:41:20.780454] [INFO ] [MAIN ] Update Progress: 1251/2367 symbols completed. LEVI took 40.8s. Estimated remaining: 2h 23m 52s. Rate: 7.8 symbols/min [2025-10-20T19:41:20.951200] [INFO ] [MAIN ] LII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:20.992617] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:41:21.013960] [DEBUG] [LOCAL ] LII: Loaded 28209 existing records from primary storage. [2025-10-20T19:41:21.014256] [INFO ] [MAIN ] LII: Loaded 28209 existing records. [2025-10-20T19:41:21.056478] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-07. [2025-10-20T19:41:21.203473] [WARN ] [MAIN ] LII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:21.203705] [INFO ] [MAIN ] LII: Wiping existing primary data before full refresh. [2025-10-20T19:41:21.203763] [INFO ] [LOCAL ] LII: Starting deletion of primary data. [2025-10-20T19:41:21.203935] [DEBUG] [LOCAL ] LII: Deleted primary data file. [2025-10-20T19:41:21.203974] [INFO ] [MAIN ] LII: Starting full primary refresh. [2025-10-20T19:41:21.441902] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T19:41:21.581250] [INFO ] [REMOTE ] LII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:21.851443] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-05. [2025-10-20T19:41:22.595563] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:41:22.651357] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-01. [2025-10-20T19:41:23.451419] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-08. [2025-10-20T19:41:23.716374] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:41:24.026109] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T19:41:24.251437] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-06. [2025-10-20T19:41:24.822342] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:41:25.051387] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-09. [2025-10-20T19:41:25.851389] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-09. [2025-10-20T19:41:26.138561] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T19:41:26.305954] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T19:41:26.651403] [INFO ] [REMOTE ] LH: Fetching primary data for 2025-10. [2025-10-20T19:41:26.828122] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:41:27.451406] [INFO ] [REMOTE ] LFT: Fetching primary data for 2025-10. [2025-10-20T19:41:28.093374] [INFO ] [REMOTE ] LH: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T19:41:28.098441] [INFO ] [REMOTE ] LH: Full fetch complete (54915 records). [2025-10-20T19:41:28.098544] [DEBUG] [VALIDATOR ] LH: Validating primary data. [2025-10-20T19:41:28.272292] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-07. [2025-10-20T19:41:28.488523] [INFO ] [REMOTE ] LFT: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:41:28.508485] [DEBUG] [VALIDATOR ] LH: All expected trading days are present. [2025-10-20T19:41:28.508594] [INFO ] [REMOTE ] LFT: Full fetch complete (23879 records). [2025-10-20T19:41:28.508714] [DEBUG] [VALIDATOR ] LH: Primary data validation passed. [2025-10-20T19:41:28.508801] [DEBUG] [VALIDATOR ] LFT: Validating primary data. [2025-10-20T19:41:28.509801] [INFO ] [MAIN ] LH: Fetched primary data is valid. [2025-10-20T19:41:28.509933] [DEBUG] [LOCAL ] LH: Saving primary data. [2025-10-20T19:41:28.611819] [DEBUG] [LOCAL ] LH: Trimmed 54915 records to 37275 (keeping 200 days of data) [2025-10-20T19:41:29.014422] [DEBUG] [VALIDATOR ] LFT: All expected trading days are present. [2025-10-20T19:41:29.014510] [DEBUG] [VALIDATOR ] LFT: Primary data validation passed. [2025-10-20T19:41:29.061398] [INFO ] [MAIN ] LFT: Fetched primary data is valid. [2025-10-20T19:41:29.060762] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-02. [2025-10-20T19:41:29.062486] [DEBUG] [LOCAL ] LFT: Saving primary data. [2025-10-20T19:41:29.071954] [DEBUG] [LOCAL ] LFT: Trimmed 23879 records to 18230 (keeping 200 days of data) [2025-10-20T19:41:29.202515] [DEBUG] [LOCAL ] LH: Primary data saved (37275 records). [2025-10-20T19:41:29.203316] [INFO ] [MAIN ] LH: Full history saved. [2025-10-20T19:41:29.215725] [INFO ] [MAIN ] Update Progress: 1252/2367 symbols completed. LH took 22.8s. Estimated remaining: 2h 23m 45s. Rate: 7.8 symbols/min [2025-10-20T19:41:29.254233] [DEBUG] [LOCAL ] LFT: Primary data saved (18230 records). [2025-10-20T19:41:29.298143] [INFO ] [MAIN ] LFT: Full history saved. [2025-10-20T19:41:29.309836] [INFO ] [MAIN ] Update Progress: 1253/2367 symbols completed. LFT took 40.9s. Estimated remaining: 2h 23m 31s. Rate: 7.8 symbols/min [2025-10-20T19:41:29.550938] [INFO ] [MAIN ] LION: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:29.597598] [INFO ] [MAIN ] LITB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:29.605778] [DEBUG] [LOCAL ] LITB: Loaded 4239 existing records from primary storage. [2025-10-20T19:41:29.606027] [INFO ] [MAIN ] LITB: Loaded 4239 existing records. [2025-10-20T19:41:29.835426] [WARN ] [MAIN ] LITB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:29.836549] [DEBUG] [LOCAL ] LION: Loaded 47066 existing records from primary storage. [2025-10-20T19:41:29.836672] [INFO ] [MAIN ] LITB: Wiping existing primary data before full refresh. [2025-10-20T19:41:29.836909] [INFO ] [MAIN ] LION: Loaded 47066 existing records. [2025-10-20T19:41:29.836832] [INFO ] [LOCAL ] LITB: Starting deletion of primary data. [2025-10-20T19:41:29.860847] [DEBUG] [LOCAL ] LITB: Deleted primary data file. [2025-10-20T19:41:29.904739] [INFO ] [MAIN ] LITB: Starting full primary refresh. [2025-10-20T19:41:30.157375] [WARN ] [MAIN ] LION: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:30.181798] [INFO ] [MAIN ] LION: Wiping existing primary data before full refresh. [2025-10-20T19:41:30.192160] [INFO ] [LOCAL ] LION: Starting deletion of primary data. [2025-10-20T19:41:30.266137] [DEBUG] [LOCAL ] LION: Deleted primary data file. [2025-10-20T19:41:30.306444] [INFO ] [MAIN ] LION: Starting full primary refresh. [2025-10-20T19:41:30.546741] [INFO ] [REMOTE ] LITB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:30.557994] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-01. [2025-10-20T19:41:30.796047] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:41:30.840093] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-07 (2.57 seconds). [2025-10-20T19:41:30.846544] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-03. [2025-10-20T19:41:31.034347] [INFO ] [REMOTE ] LION: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:31.116215] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:41:31.451427] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-02. [2025-10-20T19:41:31.889865] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T19:41:32.256449] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-01. [2025-10-20T19:41:32.266914] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T19:41:33.041997] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:41:33.051339] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-03. [2025-10-20T19:41:33.671367] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T19:41:33.851405] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-04. [2025-10-20T19:41:34.587435] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T19:41:34.651407] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-02. [2025-10-20T19:41:35.451348] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-05. [2025-10-20T19:41:35.767288] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:41:35.927771] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T19:41:36.251367] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-03. [2025-10-20T19:41:37.051350] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-06. [2025-10-20T19:41:37.301809] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T19:41:37.489845] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:41:37.851362] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-04. [2025-10-20T19:41:38.651411] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-08. [2025-10-20T19:41:39.393107] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T19:41:39.451353] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-05. [2025-10-20T19:41:40.251351] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-04. [2025-10-20T19:41:40.434055] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T19:41:40.774733] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T19:41:41.051403] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-06. [2025-10-20T19:41:41.134738] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T19:41:41.851377] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-07. [2025-10-20T19:41:42.192540] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T19:41:42.299682] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T19:41:42.651382] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-09. [2025-10-20T19:41:43.451373] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-08. [2025-10-20T19:41:43.916141] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T19:41:44.251382] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-07. [2025-10-20T19:41:44.415575] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T19:41:45.051359] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-09. [2025-10-20T19:41:45.556672] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:41:45.569297] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T19:41:45.851373] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-08. [2025-10-20T19:41:46.651341] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-05. [2025-10-20T19:41:47.039312] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T19:41:47.451361] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-09. [2025-10-20T19:41:48.205418] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:41:48.251391] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-06. [2025-10-20T19:41:48.598141] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T19:41:49.051395] [INFO ] [REMOTE ] LITB: Fetching primary data for 2025-10. [2025-10-20T19:41:49.627744] [INFO ] [REMOTE ] LITB: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T19:41:49.628902] [INFO ] [REMOTE ] LITB: Full fetch complete (5746 records). [2025-10-20T19:41:49.628969] [DEBUG] [VALIDATOR ] LITB: Validating primary data. [2025-10-20T19:41:49.856684] [INFO ] [REMOTE ] LII: Fetching primary data for 2025-10. [2025-10-20T19:41:50.055368] [DEBUG] [VALIDATOR ] LITB: All expected trading days are present. [2025-10-20T19:41:50.055615] [DEBUG] [VALIDATOR ] LITB: Primary data validation passed. [2025-10-20T19:41:50.057845] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T19:41:50.056238] [INFO ] [MAIN ] LITB: Fetched primary data is valid. [2025-10-20T19:41:50.057973] [DEBUG] [LOCAL ] LITB: Saving primary data. [2025-10-20T19:41:50.068714] [DEBUG] [LOCAL ] LITB: Trimmed 5746 records to 3871 (keeping 200 days of data) [2025-10-20T19:41:50.090303] [DEBUG] [LOCAL ] LITB: Primary data saved (3871 records). [2025-10-20T19:41:50.090431] [INFO ] [MAIN ] LITB: Full history saved. [2025-10-20T19:41:50.090511] [INFO ] [MAIN ] Update Progress: 1254/2367 symbols completed. LITB took 20.8s. Estimated remaining: 2h 23m 34s. Rate: 7.8 symbols/min [2025-10-20T19:41:50.273078] [INFO ] [MAIN ] LLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:50.328534] [DEBUG] [LOCAL ] LLY: Loaded 54022 existing records from primary storage. [2025-10-20T19:41:50.329071] [INFO ] [MAIN ] LLY: Loaded 54022 existing records. [2025-10-20T19:41:50.526580] [WARN ] [MAIN ] LLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:50.526879] [INFO ] [MAIN ] LLY: Wiping existing primary data before full refresh. [2025-10-20T19:41:50.526931] [INFO ] [LOCAL ] LLY: Starting deletion of primary data. [2025-10-20T19:41:50.527491] [DEBUG] [LOCAL ] LLY: Deleted primary data file. [2025-10-20T19:41:50.527553] [INFO ] [MAIN ] LLY: Starting full primary refresh. [2025-10-20T19:41:50.656723] [INFO ] [REMOTE ] LHX: Fetching primary data for 2025-10. [2025-10-20T19:41:50.902627] [INFO ] [REMOTE ] LLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:51.242543] [INFO ] [REMOTE ] LII: Incremental fetch complete for 2025-10 (1.39 seconds). [2025-10-20T19:41:51.247686] [INFO ] [REMOTE ] LII: Full fetch complete (41554 records). [2025-10-20T19:41:51.247840] [DEBUG] [VALIDATOR ] LII: Validating primary data. [2025-10-20T19:41:51.454500] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-07. [2025-10-20T19:41:51.694794] [DEBUG] [VALIDATOR ] LII: All expected trading days are present. [2025-10-20T19:41:51.694882] [DEBUG] [VALIDATOR ] LII: Primary data validation passed. [2025-10-20T19:41:51.695751] [INFO ] [MAIN ] LII: Fetched primary data is valid. [2025-10-20T19:41:51.695817] [DEBUG] [LOCAL ] LII: Saving primary data. [2025-10-20T19:41:51.697013] [DEBUG] [LOCAL ] LII: Trimmed 41554 records to 27796 (keeping 200 days of data) [2025-10-20T19:41:51.893821] [DEBUG] [LOCAL ] LII: Primary data saved (27796 records). [2025-10-20T19:41:51.893967] [INFO ] [MAIN ] LII: Full history saved. [2025-10-20T19:41:51.894056] [INFO ] [MAIN ] Update Progress: 1255/2367 symbols completed. LII took 31.1s. Estimated remaining: 2h 23m 21s. Rate: 7.8 symbols/min [2025-10-20T19:41:52.069310] [INFO ] [MAIN ] LMND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:52.117678] [DEBUG] [LOCAL ] LMND: Loaded 52841 existing records from primary storage. [2025-10-20T19:41:52.117994] [INFO ] [MAIN ] LMND: Loaded 52841 existing records. [2025-10-20T19:41:52.258365] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-01. [2025-10-20T19:41:52.301951] [WARN ] [MAIN ] LMND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:41:52.302497] [INFO ] [MAIN ] LMND: Wiping existing primary data before full refresh. [2025-10-20T19:41:52.302854] [INFO ] [LOCAL ] LMND: Starting deletion of primary data. [2025-10-20T19:41:52.303211] [DEBUG] [LOCAL ] LMND: Deleted primary data file. [2025-10-20T19:41:52.303274] [INFO ] [MAIN ] LMND: Starting full primary refresh. [2025-10-20T19:41:52.612943] [INFO ] [REMOTE ] LHX: Incremental fetch complete for 2025-10 (1.96 seconds). [2025-10-20T19:41:52.641794] [INFO ] [REMOTE ] LHX: Full fetch complete (70784 records). [2025-10-20T19:41:52.652245] [DEBUG] [VALIDATOR ] LHX: Validating primary data. [2025-10-20T19:41:52.817092] [INFO ] [REMOTE ] LMND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:53.060113] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-01. [2025-10-20T19:41:53.118445] [DEBUG] [VALIDATOR ] LHX: All expected trading days are present. [2025-10-20T19:41:53.118553] [DEBUG] [VALIDATOR ] LHX: Primary data validation passed. [2025-10-20T19:41:53.119562] [INFO ] [MAIN ] LHX: Fetched primary data is valid. [2025-10-20T19:41:53.119735] [DEBUG] [LOCAL ] LHX: Saving primary data. [2025-10-20T19:41:53.121543] [DEBUG] [LOCAL ] LHX: Trimmed 70784 records to 48528 (keeping 200 days of data) [2025-10-20T19:41:53.249754] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:41:53.475815] [DEBUG] [LOCAL ] LHX: Primary data saved (48528 records). [2025-10-20T19:41:53.475956] [INFO ] [MAIN ] LHX: Full history saved. [2025-10-20T19:41:53.476037] [INFO ] [MAIN ] Update Progress: 1256/2367 symbols completed. LHX took 45.4s. Estimated remaining: 2h 23m 8s. Rate: 7.8 symbols/min [2025-10-20T19:41:53.664183] [INFO ] [MAIN ] LMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:41:53.664381] [DEBUG] [LOCAL ] LMT: No existing data file found in primary storage. [2025-10-20T19:41:53.664421] [WARN ] [MAIN ] LMT: No existing data found, performing full refresh instead. [2025-10-20T19:41:53.664534] [INFO ] [MAIN ] LMT: Wiping existing primary data before full refresh. [2025-10-20T19:41:53.664576] [INFO ] [LOCAL ] LMT: Starting deletion of primary data. [2025-10-20T19:41:53.664619] [DEBUG] [LOCAL ] LMT: No primary data file found to delete. [2025-10-20T19:41:53.664683] [INFO ] [MAIN ] LMT: Starting full primary refresh. [2025-10-20T19:41:53.854534] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-08. [2025-10-20T19:41:54.062748] [INFO ] [REMOTE ] LMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:41:54.651405] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-01. [2025-10-20T19:41:54.753397] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-01 (2.50 seconds). [2025-10-20T19:41:54.930634] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:41:55.446032] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T19:41:55.455773] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-02. [2025-10-20T19:41:56.251346] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-02. [2025-10-20T19:41:56.576470] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T19:41:57.051347] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-02. [2025-10-20T19:41:57.369803] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T19:41:57.851398] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-09. [2025-10-20T19:41:58.651383] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-03. [2025-10-20T19:41:58.788236] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-02 (2.54 seconds). [2025-10-20T19:41:59.059142] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T19:41:59.451346] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-03. [2025-10-20T19:41:59.538761] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:42:00.251357] [INFO ] [REMOTE ] LION: Fetching primary data for 2025-10. [2025-10-20T19:42:00.526666] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:42:01.051449] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-04. [2025-10-20T19:42:01.851511] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-03. [2025-10-20T19:42:02.192084] [INFO ] [REMOTE ] LION: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:42:02.217950] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-03 (2.77 seconds). [2025-10-20T19:42:02.215766] [INFO ] [REMOTE ] LION: Full fetch complete (57669 records). [2025-10-20T19:42:02.228636] [DEBUG] [VALIDATOR ] LION: Validating primary data. [2025-10-20T19:42:02.600974] [DEBUG] [VALIDATOR ] LION: All expected trading days are present. [2025-10-20T19:42:02.601061] [DEBUG] [VALIDATOR ] LION: Primary data validation passed. [2025-10-20T19:42:02.601943] [INFO ] [MAIN ] LION: Fetched primary data is valid. [2025-10-20T19:42:02.602621] [DEBUG] [LOCAL ] LION: Saving primary data. [2025-10-20T19:42:02.604315] [DEBUG] [LOCAL ] LION: Trimmed 57669 records to 46794 (keeping 200 days of data) [2025-10-20T19:42:02.656476] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-04. [2025-10-20T19:42:02.829053] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:42:02.911464] [DEBUG] [LOCAL ] LION: Primary data saved (46794 records). [2025-10-20T19:42:02.911630] [INFO ] [MAIN ] LION: Full history saved. [2025-10-20T19:42:02.912541] [INFO ] [MAIN ] Update Progress: 1257/2367 symbols completed. LION took 33.7s. Estimated remaining: 2h 23m 2s. Rate: 7.8 symbols/min [2025-10-20T19:42:03.085967] [INFO ] [MAIN ] LNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:03.139365] [DEBUG] [LOCAL ] LNC: Loaded 52301 existing records from primary storage. [2025-10-20T19:42:03.139714] [INFO ] [MAIN ] LNC: Loaded 52301 existing records. [2025-10-20T19:42:03.329027] [WARN ] [MAIN ] LNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:03.329280] [INFO ] [MAIN ] LNC: Wiping existing primary data before full refresh. [2025-10-20T19:42:03.329341] [INFO ] [LOCAL ] LNC: Starting deletion of primary data. [2025-10-20T19:42:03.329572] [DEBUG] [LOCAL ] LNC: Deleted primary data file. [2025-10-20T19:42:03.329630] [INFO ] [MAIN ] LNC: Starting full primary refresh. [2025-10-20T19:42:03.456601] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-05. [2025-10-20T19:42:03.687428] [INFO ] [REMOTE ] LNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:04.063927] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-03 (2.21 seconds). [2025-10-20T19:42:04.251428] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-01. [2025-10-20T19:42:05.051356] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-04. [2025-10-20T19:42:05.209628] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T19:42:05.358327] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:42:05.825672] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:42:05.851369] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-02. [2025-10-20T19:42:06.651392] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-05. [2025-10-20T19:42:07.173460] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T19:42:07.451364] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-06. [2025-10-20T19:42:07.494367] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:42:08.251397] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-05. [2025-10-20T19:42:09.051365] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-03. [2025-10-20T19:42:09.250196] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-05 (2.60 seconds). [2025-10-20T19:42:09.544213] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-06 (2.09 seconds). [2025-10-20T19:42:09.851376] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-07. [2025-10-20T19:42:10.037794] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:42:10.653010] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:42:10.651970] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-06. [2025-10-20T19:42:11.451377] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-04. [2025-10-20T19:42:12.009664] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T19:42:12.251412] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-06. [2025-10-20T19:42:13.056551] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-08. [2025-10-20T19:42:13.069895] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T19:42:13.195932] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:42:13.851369] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-05. [2025-10-20T19:42:14.651372] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-07. [2025-10-20T19:42:14.694683] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-06 (2.44 seconds). [2025-10-20T19:42:15.252522] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-08 (2.20 seconds). [2025-10-20T19:42:15.300830] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:42:15.451370] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-07. [2025-10-20T19:42:16.251394] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-06. [2025-10-20T19:42:16.678781] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:42:17.051417] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-08. [2025-10-20T19:42:17.850581] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:42:17.854472] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-09. [2025-10-20T19:42:18.067936] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-07 (2.62 seconds). [2025-10-20T19:42:18.651435] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-08. [2025-10-20T19:42:19.261733] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T19:42:19.451459] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-07. [2025-10-20T19:42:19.965017] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-10-20T19:42:20.251391] [INFO ] [REMOTE ] LMND: Fetching primary data for 2025-10. [2025-10-20T19:42:21.051422] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-09. [2025-10-20T19:42:21.247884] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T19:42:21.325453] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-08 (2.67 seconds). [2025-10-20T19:42:21.851423] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-08. [2025-10-20T19:42:22.502087] [INFO ] [REMOTE ] LMND: Incremental fetch complete for 2025-10 (2.25 seconds). [2025-10-20T19:42:22.509848] [INFO ] [REMOTE ] LMND: Full fetch complete (75472 records). [2025-10-20T19:42:22.510006] [DEBUG] [VALIDATOR ] LMND: Validating primary data. [2025-10-20T19:42:22.657688] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-09. [2025-10-20T19:42:22.966082] [DEBUG] [VALIDATOR ] LMND: All expected trading days are present. [2025-10-20T19:42:22.966176] [DEBUG] [VALIDATOR ] LMND: Primary data validation passed. [2025-10-20T19:42:22.967377] [INFO ] [MAIN ] LMND: Fetched primary data is valid. [2025-10-20T19:42:22.967449] [DEBUG] [LOCAL ] LMND: Saving primary data. [2025-10-20T19:42:22.969210] [DEBUG] [LOCAL ] LMND: Trimmed 75472 records to 52135 (keeping 200 days of data) [2025-10-20T19:42:23.314990] [DEBUG] [LOCAL ] LMND: Primary data saved (52135 records). [2025-10-20T19:42:23.315517] [INFO ] [MAIN ] LMND: Full history saved. [2025-10-20T19:42:23.315878] [INFO ] [MAIN ] Update Progress: 1258/2367 symbols completed. LMND took 31.4s. Estimated remaining: 2h 23m 5s. Rate: 7.8 symbols/min [2025-10-20T19:42:23.560925] [INFO ] [MAIN ] LND: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:23.568160] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-09 (2.52 seconds). [2025-10-20T19:42:23.592078] [DEBUG] [LOCAL ] LND: Loaded 5027 existing records from primary storage. [2025-10-20T19:42:23.592384] [INFO ] [MAIN ] LND: Loaded 5027 existing records. [2025-10-20T19:42:23.593013] [INFO ] [REMOTE ] LMT: Fetching primary data for 2025-10. [2025-10-20T19:42:23.900824] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T19:42:23.933362] [WARN ] [MAIN ] LND: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:23.948133] [INFO ] [MAIN ] LND: Wiping existing primary data before full refresh. [2025-10-20T19:42:23.948249] [INFO ] [LOCAL ] LND: Starting deletion of primary data. [2025-10-20T19:42:23.948931] [DEBUG] [LOCAL ] LND: Deleted primary data file. [2025-10-20T19:42:23.949000] [INFO ] [MAIN ] LND: Starting full primary refresh. [2025-10-20T19:42:24.262952] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-09. [2025-10-20T19:42:24.456163] [INFO ] [REMOTE ] LND: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:25.051403] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-01. [2025-10-20T19:42:25.375968] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-09 (2.72 seconds). [2025-10-20T19:42:25.611454] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:42:25.851421] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-02. [2025-10-20T19:42:26.232636] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:42:26.331236] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T19:42:26.355635] [INFO ] [REMOTE ] LMT: Incremental fetch complete for 2025-10 (2.76 seconds). [2025-10-20T19:42:26.362927] [INFO ] [REMOTE ] LMT: Full fetch complete (72494 records). [2025-10-20T19:42:26.363047] [DEBUG] [VALIDATOR ] LMT: Validating primary data. [2025-10-20T19:42:26.659122] [INFO ] [REMOTE ] LLY: Fetching primary data for 2025-10. [2025-10-20T19:42:26.757860] [DEBUG] [VALIDATOR ] LMT: All expected trading days are present. [2025-10-20T19:42:26.757942] [DEBUG] [VALIDATOR ] LMT: Primary data validation passed. [2025-10-20T19:42:26.758911] [INFO ] [MAIN ] LMT: Fetched primary data is valid. [2025-10-20T19:42:26.759017] [DEBUG] [LOCAL ] LMT: Saving primary data. [2025-10-20T19:42:26.760380] [DEBUG] [LOCAL ] LMT: Trimmed 72494 records to 49622 (keeping 200 days of data) [2025-10-20T19:42:27.057217] [DEBUG] [LOCAL ] LMT: Primary data saved (49622 records). [2025-10-20T19:42:27.057347] [INFO ] [MAIN ] LMT: Full history saved. [2025-10-20T19:42:27.057414] [INFO ] [MAIN ] Update Progress: 1259/2367 symbols completed. LMT took 33.6s. Estimated remaining: 2h 22m 54s. Rate: 7.8 symbols/min [2025-10-20T19:42:27.227403] [INFO ] [MAIN ] LNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:27.274639] [DEBUG] [LOCAL ] LNG: Loaded 51953 existing records from primary storage. [2025-10-20T19:42:27.274997] [INFO ] [MAIN ] LNG: Loaded 51953 existing records. [2025-10-20T19:42:27.455314] [INFO ] [REMOTE ] LNC: Fetching primary data for 2025-10. [2025-10-20T19:42:27.455207] [WARN ] [MAIN ] LNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:27.456625] [INFO ] [MAIN ] LNG: Wiping existing primary data before full refresh. [2025-10-20T19:42:27.456901] [INFO ] [LOCAL ] LNG: Starting deletion of primary data. [2025-10-20T19:42:27.457213] [DEBUG] [LOCAL ] LNG: Deleted primary data file. [2025-10-20T19:42:27.457270] [INFO ] [MAIN ] LNG: Starting full primary refresh. [2025-10-20T19:42:27.832460] [INFO ] [REMOTE ] LLY: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T19:42:27.837819] [INFO ] [REMOTE ] LNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:27.845684] [INFO ] [REMOTE ] LLY: Full fetch complete (77237 records). [2025-10-20T19:42:27.845808] [DEBUG] [VALIDATOR ] LLY: Validating primary data. [2025-10-20T19:42:28.236332] [DEBUG] [VALIDATOR ] LLY: All expected trading days are present. [2025-10-20T19:42:28.236419] [DEBUG] [VALIDATOR ] LLY: Primary data validation passed. [2025-10-20T19:42:28.237396] [INFO ] [MAIN ] LLY: Fetched primary data is valid. [2025-10-20T19:42:28.237459] [DEBUG] [LOCAL ] LLY: Saving primary data. [2025-10-20T19:42:28.238932] [DEBUG] [LOCAL ] LLY: Trimmed 77237 records to 53228 (keeping 200 days of data) [2025-10-20T19:42:28.295347] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-01. [2025-10-20T19:42:28.574877] [DEBUG] [LOCAL ] LLY: Primary data saved (53228 records). [2025-10-20T19:42:28.575020] [INFO ] [MAIN ] LLY: Full history saved. [2025-10-20T19:42:28.575564] [INFO ] [MAIN ] Update Progress: 1260/2367 symbols completed. LLY took 38.5s. Estimated remaining: 2h 22m 41s. Rate: 7.8 symbols/min [2025-10-20T19:42:28.746596] [INFO ] [MAIN ] LNN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:28.760075] [DEBUG] [LOCAL ] LNN: Loaded 11624 existing records from primary storage. [2025-10-20T19:42:28.760295] [INFO ] [MAIN ] LNN: Loaded 11624 existing records. [2025-10-20T19:42:28.944308] [WARN ] [MAIN ] LNN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:28.944579] [INFO ] [MAIN ] LNN: Wiping existing primary data before full refresh. [2025-10-20T19:42:28.944818] [INFO ] [LOCAL ] LNN: Starting deletion of primary data. [2025-10-20T19:42:28.945246] [DEBUG] [LOCAL ] LNN: Deleted primary data file. [2025-10-20T19:42:28.945315] [INFO ] [MAIN ] LNN: Starting full primary refresh. [2025-10-20T19:42:29.051674] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-03. [2025-10-20T19:42:29.254815] [INFO ] [REMOTE ] LNC: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T19:42:29.295177] [INFO ] [REMOTE ] LNC: Full fetch complete (74559 records). [2025-10-20T19:42:29.342742] [DEBUG] [VALIDATOR ] LNC: Validating primary data. [2025-10-20T19:42:29.363991] [INFO ] [REMOTE ] LNN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:29.677617] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T19:42:29.774512] [DEBUG] [VALIDATOR ] LNC: All expected trading days are present. [2025-10-20T19:42:29.774601] [DEBUG] [VALIDATOR ] LNC: Primary data validation passed. [2025-10-20T19:42:29.775551] [INFO ] [MAIN ] LNC: Fetched primary data is valid. [2025-10-20T19:42:29.775616] [DEBUG] [LOCAL ] LNC: Saving primary data. [2025-10-20T19:42:29.777081] [DEBUG] [LOCAL ] LNC: Trimmed 74559 records to 51511 (keeping 200 days of data) [2025-10-20T19:42:29.864197] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-04. [2025-10-20T19:42:30.088311] [DEBUG] [LOCAL ] LNC: Primary data saved (51511 records). [2025-10-20T19:42:30.088520] [INFO ] [MAIN ] LNC: Full history saved. [2025-10-20T19:42:30.089303] [INFO ] [MAIN ] Update Progress: 1261/2367 symbols completed. LNC took 27.2s. Estimated remaining: 2h 22m 28s. Rate: 7.8 symbols/min [2025-10-20T19:42:30.259864] [INFO ] [MAIN ] LOAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:30.291566] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-01 (2.00 seconds). [2025-10-20T19:42:30.334290] [DEBUG] [LOCAL ] LOAR: Loaded 42593 existing records from primary storage. [2025-10-20T19:42:30.334587] [INFO ] [MAIN ] LOAR: Loaded 42593 existing records. [2025-10-20T19:42:30.514113] [WARN ] [MAIN ] LOAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:30.514309] [INFO ] [MAIN ] LOAR: Wiping existing primary data before full refresh. [2025-10-20T19:42:30.514354] [INFO ] [LOCAL ] LOAR: Starting deletion of primary data. [2025-10-20T19:42:30.514769] [DEBUG] [LOCAL ] LOAR: Deleted primary data file. [2025-10-20T19:42:30.514849] [INFO ] [MAIN ] LOAR: Starting full primary refresh. [2025-10-20T19:42:30.620447] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T19:42:30.661442] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-02. [2025-10-20T19:42:30.875436] [INFO ] [REMOTE ] LOAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:31.451349] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-01. [2025-10-20T19:42:32.251358] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-05. [2025-10-20T19:42:32.279954] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T19:42:32.663239] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T19:42:32.857715] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:42:33.051430] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-02. [2025-10-20T19:42:33.812121] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:42:33.851450] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-03. [2025-10-20T19:42:34.651389] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-06. [2025-10-20T19:42:34.700412] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T19:42:35.268272] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T19:42:35.451371] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-04. [2025-10-20T19:42:36.251391] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-07. [2025-10-20T19:42:36.389051] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T19:42:36.843701] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T19:42:37.051384] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-08. [2025-10-20T19:42:37.625472] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T19:42:37.851363] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-09. [2025-10-20T19:42:38.489269] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T19:42:38.651378] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-01. [2025-10-20T19:42:39.451366] [INFO ] [REMOTE ] LND: Fetching primary data for 2025-10. [2025-10-20T19:42:39.867747] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T19:42:40.251346] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-03. [2025-10-20T19:42:40.393131] [INFO ] [REMOTE ] LND: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:42:40.395262] [INFO ] [REMOTE ] LND: Full fetch complete (7834 records). [2025-10-20T19:42:40.395384] [DEBUG] [VALIDATOR ] LND: Validating primary data. [2025-10-20T19:42:40.773713] [DEBUG] [VALIDATOR ] LND: All expected trading days are present. [2025-10-20T19:42:40.773796] [DEBUG] [VALIDATOR ] LND: Primary data validation passed. [2025-10-20T19:42:40.774021] [INFO ] [MAIN ] LND: Fetched primary data is valid. [2025-10-20T19:42:40.774060] [DEBUG] [LOCAL ] LND: Saving primary data. [2025-10-20T19:42:40.774447] [DEBUG] [LOCAL ] LND: Trimmed 7834 records to 4966 (keeping 200 days of data) [2025-10-20T19:42:40.800869] [DEBUG] [LOCAL ] LND: Primary data saved (4966 records). [2025-10-20T19:42:40.800945] [INFO ] [MAIN ] LND: Full history saved. [2025-10-20T19:42:40.801016] [INFO ] [MAIN ] Update Progress: 1262/2367 symbols completed. LND took 17.5s. Estimated remaining: 2h 22m 23s. Rate: 7.8 symbols/min [2025-10-20T19:42:40.965750] [INFO ] [MAIN ] LOB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:40.991414] [DEBUG] [LOCAL ] LOB: Loaded 27202 existing records from primary storage. [2025-10-20T19:42:40.991720] [INFO ] [MAIN ] LOB: Loaded 27202 existing records. [2025-10-20T19:42:41.056596] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-05. [2025-10-20T19:42:41.168188] [WARN ] [MAIN ] LOB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:41.168411] [INFO ] [MAIN ] LOB: Wiping existing primary data before full refresh. [2025-10-20T19:42:41.168458] [INFO ] [LOCAL ] LOB: Starting deletion of primary data. [2025-10-20T19:42:41.168812] [DEBUG] [LOCAL ] LOB: Deleted primary data file. [2025-10-20T19:42:41.168868] [INFO ] [MAIN ] LOB: Starting full primary refresh. [2025-10-20T19:42:41.516355] [INFO ] [REMOTE ] LOB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:41.851469] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-01. [2025-10-20T19:42:41.854532] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T19:42:42.189801] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T19:42:42.514854] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-10-20T19:42:42.651387] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-06. [2025-10-20T19:42:43.459034] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-02. [2025-10-20T19:42:43.460058] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T19:42:44.251394] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-04. [2025-10-20T19:42:44.729415] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:42:45.051501] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-07. [2025-10-20T19:42:45.851346] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-02. [2025-10-20T19:42:46.040737] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:42:46.239540] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T19:42:46.651401] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-05. [2025-10-20T19:42:46.962237] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:42:47.451370] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-03. [2025-10-20T19:42:48.251363] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-03. [2025-10-20T19:42:48.506711] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T19:42:48.787094] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T19:42:49.051355] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-04. [2025-10-20T19:42:49.212724] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T19:42:49.851388] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-04. [2025-10-20T19:42:50.453894] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:42:50.651348] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-08. [2025-10-20T19:42:51.357818] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:42:51.397778] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T19:42:51.451417] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-05. [2025-10-20T19:42:52.251440] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-09. [2025-10-20T19:42:53.051365] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-06. [2025-10-20T19:42:53.093870] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T19:42:53.157815] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T19:42:53.851369] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-06. [2025-10-20T19:42:54.651350] [INFO ] [REMOTE ] LNN: Fetching primary data for 2025-10. [2025-10-20T19:42:54.841352] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T19:42:55.387221] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:42:55.451389] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-07. [2025-10-20T19:42:55.693316] [INFO ] [REMOTE ] LNN: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T19:42:55.696615] [INFO ] [REMOTE ] LNN: Full fetch complete (15911 records). [2025-10-20T19:42:55.696787] [DEBUG] [VALIDATOR ] LNN: Validating primary data. [2025-10-20T19:42:56.067614] [DEBUG] [VALIDATOR ] LNN: All expected trading days are present. [2025-10-20T19:42:56.067715] [DEBUG] [VALIDATOR ] LNN: Primary data validation passed. [2025-10-20T19:42:56.068037] [INFO ] [MAIN ] LNN: Fetched primary data is valid. [2025-10-20T19:42:56.068074] [DEBUG] [LOCAL ] LNN: Saving primary data. [2025-10-20T19:42:56.068570] [DEBUG] [LOCAL ] LNN: Trimmed 15911 records to 11478 (keeping 200 days of data) [2025-10-20T19:42:56.140361] [DEBUG] [LOCAL ] LNN: Primary data saved (11478 records). [2025-10-20T19:42:56.140510] [INFO ] [MAIN ] LNN: Full history saved. [2025-10-20T19:42:56.140599] [INFO ] [MAIN ] Update Progress: 1263/2367 symbols completed. LNN took 27.6s. Estimated remaining: 2h 22m 22s. Rate: 7.8 symbols/min [2025-10-20T19:42:56.256606] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-05. [2025-10-20T19:42:56.309782] [INFO ] [MAIN ] LOCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:42:56.314793] [DEBUG] [LOCAL ] LOCL: Loaded 3843 existing records from primary storage. [2025-10-20T19:42:56.314947] [INFO ] [MAIN ] LOCL: Loaded 3843 existing records. [2025-10-20T19:42:56.494095] [WARN ] [MAIN ] LOCL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:42:56.494332] [INFO ] [MAIN ] LOCL: Wiping existing primary data before full refresh. [2025-10-20T19:42:56.494400] [INFO ] [LOCAL ] LOCL: Starting deletion of primary data. [2025-10-20T19:42:56.494540] [DEBUG] [LOCAL ] LOCL: Deleted primary data file. [2025-10-20T19:42:56.494595] [INFO ] [MAIN ] LOCL: Starting full primary refresh. [2025-10-20T19:42:56.855430] [INFO ] [REMOTE ] LOCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:42:57.005086] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:42:57.051352] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-08. [2025-10-20T19:42:57.363313] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:42:57.851422] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-01. [2025-10-20T19:42:58.583773] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:42:58.618960] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:42:58.651340] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-09. [2025-10-20T19:42:59.451355] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-02. [2025-10-20T19:42:59.790255] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-02 (0.34 seconds). [2025-10-20T19:43:00.129693] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T19:43:00.251335] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-07. [2025-10-20T19:43:01.051374] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-03. [2025-10-20T19:43:01.680464] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T19:43:01.851387] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-04. [2025-10-20T19:43:02.294828] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T19:43:02.642075] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T19:43:02.651324] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-05. [2025-10-20T19:43:03.093469] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T19:43:03.451450] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-08. [2025-10-20T19:43:04.251417] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-06. [2025-10-20T19:43:04.717723] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T19:43:05.051371] [INFO ] [REMOTE ] LOAR: Fetching primary data for 2025-10. [2025-10-20T19:43:05.511613] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T19:43:05.851415] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-09. [2025-10-20T19:43:06.447765] [INFO ] [REMOTE ] LOAR: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:43:06.453444] [INFO ] [REMOTE ] LOAR: Full fetch complete (55973 records). [2025-10-20T19:43:06.453565] [DEBUG] [VALIDATOR ] LOAR: Validating primary data. [2025-10-20T19:43:06.656746] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-07. [2025-10-20T19:43:06.839086] [DEBUG] [VALIDATOR ] LOAR: All expected trading days are present. [2025-10-20T19:43:06.839182] [DEBUG] [VALIDATOR ] LOAR: Primary data validation passed. [2025-10-20T19:43:06.840155] [INFO ] [MAIN ] LOAR: Fetched primary data is valid. [2025-10-20T19:43:06.840223] [DEBUG] [LOCAL ] LOAR: Saving primary data. [2025-10-20T19:43:06.841559] [DEBUG] [LOCAL ] LOAR: Trimmed 55973 records to 42010 (keeping 200 days of data) [2025-10-20T19:43:07.081835] [DEBUG] [LOCAL ] LOAR: Primary data saved (42010 records). [2025-10-20T19:43:07.081966] [INFO ] [MAIN ] LOAR: Full history saved. [2025-10-20T19:43:07.082045] [INFO ] [MAIN ] Update Progress: 1264/2367 symbols completed. LOAR took 37.0s. Estimated remaining: 2h 22m 17s. Rate: 7.8 symbols/min [2025-10-20T19:43:07.250317] [INFO ] [MAIN ] LOMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:07.283285] [DEBUG] [LOCAL ] LOMA: Loaded 33150 existing records from primary storage. [2025-10-20T19:43:07.283546] [INFO ] [MAIN ] LOMA: Loaded 33150 existing records. [2025-10-20T19:43:07.351946] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T19:43:07.464818] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-06. [2025-10-20T19:43:07.471017] [WARN ] [MAIN ] LOMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:07.471224] [INFO ] [MAIN ] LOMA: Wiping existing primary data before full refresh. [2025-10-20T19:43:07.472009] [INFO ] [LOCAL ] LOMA: Starting deletion of primary data. [2025-10-20T19:43:07.472228] [DEBUG] [LOCAL ] LOMA: Deleted primary data file. [2025-10-20T19:43:07.472287] [INFO ] [MAIN ] LOMA: Starting full primary refresh. [2025-10-20T19:43:07.835424] [INFO ] [REMOTE ] LOMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:07.864363] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T19:43:08.251379] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-08. [2025-10-20T19:43:08.619405] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:43:08.726901] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T19:43:09.051409] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-09. [2025-10-20T19:43:09.653517] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T19:43:09.851451] [INFO ] [REMOTE ] LOCL: Fetching primary data for 2025-10. [2025-10-20T19:43:10.651397] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-01. [2025-10-20T19:43:10.709870] [INFO ] [REMOTE ] LOCL: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:43:10.710946] [INFO ] [REMOTE ] LOCL: Full fetch complete (4539 records). [2025-10-20T19:43:10.711013] [DEBUG] [VALIDATOR ] LOCL: Validating primary data. [2025-10-20T19:43:11.065694] [DEBUG] [VALIDATOR ] LOCL: All expected trading days are present. [2025-10-20T19:43:11.065778] [DEBUG] [VALIDATOR ] LOCL: Primary data validation passed. [2025-10-20T19:43:11.065915] [INFO ] [MAIN ] LOCL: Fetched primary data is valid. [2025-10-20T19:43:11.065952] [DEBUG] [LOCAL ] LOCL: Saving primary data. [2025-10-20T19:43:11.066311] [DEBUG] [LOCAL ] LOCL: Trimmed 4539 records to 3814 (keeping 200 days of data) [2025-10-20T19:43:11.087229] [DEBUG] [LOCAL ] LOCL: Primary data saved (3814 records). [2025-10-20T19:43:11.087359] [INFO ] [MAIN ] LOCL: Full history saved. [2025-10-20T19:43:11.087436] [INFO ] [MAIN ] Update Progress: 1265/2367 symbols completed. LOCL took 14.9s. Estimated remaining: 2h 22m 6s. Rate: 7.8 symbols/min [2025-10-20T19:43:11.258690] [INFO ] [MAIN ] LOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:11.309571] [DEBUG] [LOCAL ] LOW: Loaded 53762 existing records from primary storage. [2025-10-20T19:43:11.309942] [INFO ] [MAIN ] LOW: Loaded 53762 existing records. [2025-10-20T19:43:11.465219] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-07. [2025-10-20T19:43:11.505758] [WARN ] [MAIN ] LOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:11.505914] [INFO ] [MAIN ] LOW: Wiping existing primary data before full refresh. [2025-10-20T19:43:11.505960] [INFO ] [LOCAL ] LOW: Starting deletion of primary data. [2025-10-20T19:43:11.506132] [DEBUG] [LOCAL ] LOW: Deleted primary data file. [2025-10-20T19:43:11.506168] [INFO ] [MAIN ] LOW: Starting full primary refresh. [2025-10-20T19:43:11.854908] [INFO ] [REMOTE ] LOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:12.015921] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:43:12.251396] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-02. [2025-10-20T19:43:12.447962] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:43:13.051349] [INFO ] [REMOTE ] LNG: Fetching primary data for 2025-10. [2025-10-20T19:43:13.489528] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T19:43:13.564862] [INFO ] [REMOTE ] LNG: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T19:43:13.572113] [INFO ] [REMOTE ] LNG: Full fetch complete (74771 records). [2025-10-20T19:43:13.572252] [DEBUG] [VALIDATOR ] LNG: Validating primary data. [2025-10-20T19:43:13.856579] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-03. [2025-10-20T19:43:13.964706] [DEBUG] [VALIDATOR ] LNG: All expected trading days are present. [2025-10-20T19:43:13.964808] [DEBUG] [VALIDATOR ] LNG: Primary data validation passed. [2025-10-20T19:43:13.965783] [INFO ] [MAIN ] LNG: Fetched primary data is valid. [2025-10-20T19:43:13.965849] [DEBUG] [LOCAL ] LNG: Saving primary data. [2025-10-20T19:43:13.967297] [DEBUG] [LOCAL ] LNG: Trimmed 74771 records to 51168 (keeping 200 days of data) [2025-10-20T19:43:14.271742] [DEBUG] [LOCAL ] LNG: Primary data saved (51168 records). [2025-10-20T19:43:14.271873] [INFO ] [MAIN ] LNG: Full history saved. [2025-10-20T19:43:14.271947] [INFO ] [MAIN ] Update Progress: 1266/2367 symbols completed. LNG took 47.2s. Estimated remaining: 2h 21m 54s. Rate: 7.8 symbols/min [2025-10-20T19:43:14.436783] [INFO ] [MAIN ] LPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:14.476617] [DEBUG] [LOCAL ] LPG: Loaded 40105 existing records from primary storage. [2025-10-20T19:43:14.476915] [INFO ] [MAIN ] LPG: Loaded 40105 existing records. [2025-10-20T19:43:14.649879] [WARN ] [MAIN ] LPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:14.650088] [INFO ] [MAIN ] LPG: Wiping existing primary data before full refresh. [2025-10-20T19:43:14.650133] [INFO ] [LOCAL ] LPG: Starting deletion of primary data. [2025-10-20T19:43:14.650489] [DEBUG] [LOCAL ] LPG: Deleted primary data file. [2025-10-20T19:43:14.650543] [INFO ] [MAIN ] LPG: Starting full primary refresh. [2025-10-20T19:43:14.661478] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-01. [2025-10-20T19:43:15.000427] [INFO ] [REMOTE ] LPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:15.217513] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T19:43:15.451373] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-01. [2025-10-20T19:43:16.251402] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-08. [2025-10-20T19:43:16.672607] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T19:43:17.051402] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-02. [2025-10-20T19:43:17.179676] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T19:43:17.384084] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T19:43:17.851360] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-04. [2025-10-20T19:43:18.651430] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-09. [2025-10-20T19:43:18.708756] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:43:19.122152] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T19:43:19.451397] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-05. [2025-10-20T19:43:19.491045] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T19:43:20.251379] [INFO ] [REMOTE ] LOB: Fetching primary data for 2025-10. [2025-10-20T19:43:20.695724] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:43:21.051413] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-03. [2025-10-20T19:43:21.438780] [INFO ] [REMOTE ] LOB: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T19:43:21.442895] [INFO ] [REMOTE ] LOB: Full fetch complete (38479 records). [2025-10-20T19:43:21.443017] [DEBUG] [VALIDATOR ] LOB: Validating primary data. [2025-10-20T19:43:21.809252] [DEBUG] [VALIDATOR ] LOB: All expected trading days are present. [2025-10-20T19:43:21.809328] [DEBUG] [VALIDATOR ] LOB: Primary data validation passed. [2025-10-20T19:43:21.809982] [INFO ] [MAIN ] LOB: Fetched primary data is valid. [2025-10-20T19:43:21.810042] [DEBUG] [LOCAL ] LOB: Saving primary data. [2025-10-20T19:43:21.810879] [DEBUG] [LOCAL ] LOB: Trimmed 38479 records to 26844 (keeping 200 days of data) [2025-10-20T19:43:21.862279] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-06. [2025-10-20T19:43:21.974266] [DEBUG] [LOCAL ] LOB: Primary data saved (26844 records). [2025-10-20T19:43:21.974400] [INFO ] [MAIN ] LOB: Full history saved. [2025-10-20T19:43:21.974479] [INFO ] [MAIN ] Update Progress: 1267/2367 symbols completed. LOB took 41.2s. Estimated remaining: 2h 21m 46s. Rate: 7.8 symbols/min [2025-10-20T19:43:22.138184] [INFO ] [MAIN ] LPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:22.165075] [DEBUG] [LOCAL ] LPL: Loaded 28863 existing records from primary storage. [2025-10-20T19:43:22.165324] [INFO ] [MAIN ] LPL: Loaded 28863 existing records. [2025-10-20T19:43:22.337292] [WARN ] [MAIN ] LPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:22.337497] [INFO ] [MAIN ] LPL: Wiping existing primary data before full refresh. [2025-10-20T19:43:22.337543] [INFO ] [LOCAL ] LPL: Starting deletion of primary data. [2025-10-20T19:43:22.337763] [DEBUG] [LOCAL ] LPL: Deleted primary data file. [2025-10-20T19:43:22.337819] [INFO ] [MAIN ] LPL: Starting full primary refresh. [2025-10-20T19:43:22.660996] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-02. [2025-10-20T19:43:22.692711] [INFO ] [REMOTE ] LPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:22.956945] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:43:23.140619] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:43:23.451417] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-07. [2025-10-20T19:43:24.200044] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T19:43:24.251424] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-01. [2025-10-20T19:43:24.760480] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T19:43:25.051406] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-03. [2025-10-20T19:43:25.272722] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T19:43:25.851390] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-02. [2025-10-20T19:43:26.594951] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T19:43:26.651365] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-04. [2025-10-20T19:43:26.815928] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:43:27.451369] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-03. [2025-10-20T19:43:28.215231] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:43:28.251422] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-05. [2025-10-20T19:43:28.514235] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T19:43:29.051394] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-08. [2025-10-20T19:43:29.594985] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:43:29.851412] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-06. [2025-10-20T19:43:30.333145] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T19:43:30.651378] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-04. [2025-10-20T19:43:31.416159] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T19:43:31.451351] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-04. [2025-10-20T19:43:31.760169] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:43:32.251373] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-07. [2025-10-20T19:43:33.051387] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-05. [2025-10-20T19:43:33.335787] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:43:33.710382] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:43:33.851389] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-05. [2025-10-20T19:43:34.128088] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:43:34.651365] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-06. [2025-10-20T19:43:35.451357] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-08. [2025-10-20T19:43:35.661747] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T19:43:35.783683] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T19:43:36.251429] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-07. [2025-10-20T19:43:36.873026] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:43:37.051370] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-09. [2025-10-20T19:43:37.419972] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T19:43:37.851391] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-09. [2025-10-20T19:43:38.541548] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:43:38.651364] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-08. [2025-10-20T19:43:39.318359] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:43:39.451380] [INFO ] [REMOTE ] LPG: Fetching primary data for 2025-10. [2025-10-20T19:43:40.071031] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:43:40.251367] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-09. [2025-10-20T19:43:40.879585] [INFO ] [REMOTE ] LPG: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T19:43:40.884981] [INFO ] [REMOTE ] LPG: Full fetch complete (59041 records). [2025-10-20T19:43:40.885114] [DEBUG] [VALIDATOR ] LPG: Validating primary data. [2025-10-20T19:43:41.062765] [INFO ] [REMOTE ] LOMA: Fetching primary data for 2025-10. [2025-10-20T19:43:41.284687] [DEBUG] [VALIDATOR ] LPG: All expected trading days are present. [2025-10-20T19:43:41.284794] [DEBUG] [VALIDATOR ] LPG: Primary data validation passed. [2025-10-20T19:43:41.285754] [INFO ] [MAIN ] LPG: Fetched primary data is valid. [2025-10-20T19:43:41.285820] [DEBUG] [LOCAL ] LPG: Saving primary data. [2025-10-20T19:43:41.287070] [DEBUG] [LOCAL ] LPG: Trimmed 59041 records to 39541 (keeping 200 days of data) [2025-10-20T19:43:41.520960] [DEBUG] [LOCAL ] LPG: Primary data saved (39541 records). [2025-10-20T19:43:41.521122] [INFO ] [MAIN ] LPG: Full history saved. [2025-10-20T19:43:41.521201] [INFO ] [MAIN ] Update Progress: 1268/2367 symbols completed. LPG took 27.2s. Estimated remaining: 2h 21m 49s. Rate: 7.7 symbols/min [2025-10-20T19:43:41.688050] [INFO ] [MAIN ] LPX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:41.725152] [DEBUG] [LOCAL ] LPX: Loaded 40820 existing records from primary storage. [2025-10-20T19:43:41.725428] [INFO ] [MAIN ] LPX: Loaded 40820 existing records. [2025-10-20T19:43:41.856379] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-06. [2025-10-20T19:43:41.908761] [WARN ] [MAIN ] LPX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:41.909073] [INFO ] [MAIN ] LPX: Wiping existing primary data before full refresh. [2025-10-20T19:43:41.909140] [INFO ] [LOCAL ] LPX: Starting deletion of primary data. [2025-10-20T19:43:41.909371] [DEBUG] [LOCAL ] LPX: Deleted primary data file. [2025-10-20T19:43:41.909427] [INFO ] [MAIN ] LPX: Starting full primary refresh. [2025-10-20T19:43:41.995785] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T19:43:42.286125] [INFO ] [REMOTE ] LPX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:42.408281] [INFO ] [REMOTE ] LOMA: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:43:42.413708] [INFO ] [REMOTE ] LOMA: Full fetch complete (47886 records). [2025-10-20T19:43:42.413793] [DEBUG] [VALIDATOR ] LOMA: Validating primary data. [2025-10-20T19:43:42.652478] [INFO ] [REMOTE ] LPL: Fetching primary data for 2025-10. [2025-10-20T19:43:42.788986] [DEBUG] [VALIDATOR ] LOMA: All expected trading days are present. [2025-10-20T19:43:42.789314] [DEBUG] [VALIDATOR ] LOMA: Primary data validation passed. [2025-10-20T19:43:42.790084] [INFO ] [MAIN ] LOMA: Fetched primary data is valid. [2025-10-20T19:43:42.790155] [DEBUG] [LOCAL ] LOMA: Saving primary data. [2025-10-20T19:43:42.791265] [DEBUG] [LOCAL ] LOMA: Trimmed 47886 records to 32769 (keeping 200 days of data) [2025-10-20T19:43:42.970740] [DEBUG] [LOCAL ] LOMA: Primary data saved (32769 records). [2025-10-20T19:43:42.970860] [INFO ] [MAIN ] LOMA: Full history saved. [2025-10-20T19:43:42.970934] [INFO ] [MAIN ] Update Progress: 1269/2367 symbols completed. LOMA took 35.9s. Estimated remaining: 2h 21m 35s. Rate: 7.8 symbols/min [2025-10-20T19:43:43.136813] [INFO ] [MAIN ] LRN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:43.177581] [DEBUG] [LOCAL ] LRN: Loaded 40127 existing records from primary storage. [2025-10-20T19:43:43.177875] [INFO ] [MAIN ] LRN: Loaded 40127 existing records. [2025-10-20T19:43:43.355397] [WARN ] [MAIN ] LRN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:43.355725] [INFO ] [MAIN ] LRN: Wiping existing primary data before full refresh. [2025-10-20T19:43:43.355793] [INFO ] [LOCAL ] LRN: Starting deletion of primary data. [2025-10-20T19:43:43.356347] [DEBUG] [LOCAL ] LRN: Deleted primary data file. [2025-10-20T19:43:43.356403] [INFO ] [MAIN ] LRN: Starting full primary refresh. [2025-10-20T19:43:43.453946] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-01. [2025-10-20T19:43:43.761773] [INFO ] [REMOTE ] LRN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:43.766107] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T19:43:43.843762] [INFO ] [REMOTE ] LPL: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T19:43:43.849590] [INFO ] [REMOTE ] LPL: Full fetch complete (37230 records). [2025-10-20T19:43:43.849692] [DEBUG] [VALIDATOR ] LPL: Validating primary data. [2025-10-20T19:43:44.210491] [DEBUG] [VALIDATOR ] LPL: All expected trading days are present. [2025-10-20T19:43:44.210562] [DEBUG] [VALIDATOR ] LPL: Primary data validation passed. [2025-10-20T19:43:44.211151] [INFO ] [MAIN ] LPL: Fetched primary data is valid. [2025-10-20T19:43:44.211210] [DEBUG] [LOCAL ] LPL: Saving primary data. [2025-10-20T19:43:44.211996] [DEBUG] [LOCAL ] LPL: Trimmed 37230 records to 28753 (keeping 200 days of data) [2025-10-20T19:43:44.254914] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-01. [2025-10-20T19:43:44.370864] [DEBUG] [LOCAL ] LPL: Primary data saved (28753 records). [2025-10-20T19:43:44.370986] [INFO ] [MAIN ] LPL: Full history saved. [2025-10-20T19:43:44.371059] [INFO ] [MAIN ] Update Progress: 1270/2367 symbols completed. LPL took 22.4s. Estimated remaining: 2h 21m 22s. Rate: 7.8 symbols/min [2025-10-20T19:43:44.534467] [INFO ] [MAIN ] LSPD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:43:44.572980] [DEBUG] [LOCAL ] LSPD: Loaded 43208 existing records from primary storage. [2025-10-20T19:43:44.573254] [INFO ] [MAIN ] LSPD: Loaded 43208 existing records. [2025-10-20T19:43:44.747963] [WARN ] [MAIN ] LSPD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:43:44.748239] [INFO ] [MAIN ] LSPD: Wiping existing primary data before full refresh. [2025-10-20T19:43:44.748305] [INFO ] [LOCAL ] LSPD: Starting deletion of primary data. [2025-10-20T19:43:44.748602] [DEBUG] [LOCAL ] LSPD: Deleted primary data file. [2025-10-20T19:43:44.748683] [INFO ] [MAIN ] LSPD: Starting full primary refresh. [2025-10-20T19:43:45.058617] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-07. [2025-10-20T19:43:45.111459] [INFO ] [REMOTE ] LSPD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:43:45.137263] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T19:43:45.859672] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-01. [2025-10-20T19:43:45.864326] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T19:43:46.651349] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-02. [2025-10-20T19:43:47.120863] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T19:43:47.159043] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T19:43:47.451403] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-02. [2025-10-20T19:43:48.102307] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:43:48.251384] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-02. [2025-10-20T19:43:48.969430] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T19:43:49.051345] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-03. [2025-10-20T19:43:49.700255] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T19:43:49.851365] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-03. [2025-10-20T19:43:50.662133] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-03. [2025-10-20T19:43:50.667785] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T19:43:51.418239] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:43:51.451345] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-04. [2025-10-20T19:43:52.129242] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T19:43:52.251401] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-04. [2025-10-20T19:43:53.043032] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T19:43:53.053856] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-04. [2025-10-20T19:43:53.783176] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T19:43:53.851358] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-05. [2025-10-20T19:43:54.651452] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-08. [2025-10-20T19:43:54.692425] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:43:55.357351] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:43:55.451382] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-06. [2025-10-20T19:43:56.251416] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-05. [2025-10-20T19:43:56.744147] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T19:43:56.823361] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T19:43:57.051431] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-09. [2025-10-20T19:43:57.800141] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T19:43:57.851402] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-06. [2025-10-20T19:43:58.651389] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-05. [2025-10-20T19:43:58.912776] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T19:43:59.430279] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:43:59.451346] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-07. [2025-10-20T19:43:59.734190] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:44:00.251393] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-07. [2025-10-20T19:44:00.929144] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T19:44:01.051415] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-08. [2025-10-20T19:44:01.742197] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T19:44:01.851385] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-08. [2025-10-20T19:44:02.627285] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:02.651366] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-09. [2025-10-20T19:44:03.428637] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:03.451349] [INFO ] [REMOTE ] LOW: Fetching primary data for 2025-10. [2025-10-20T19:44:04.251392] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-06. [2025-10-20T19:44:04.312426] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T19:44:04.356914] [INFO ] [REMOTE ] LOW: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T19:44:04.364727] [INFO ] [REMOTE ] LOW: Full fetch complete (76732 records). [2025-10-20T19:44:04.364883] [DEBUG] [VALIDATOR ] LOW: Validating primary data. [2025-10-20T19:44:04.751246] [DEBUG] [VALIDATOR ] LOW: All expected trading days are present. [2025-10-20T19:44:04.751337] [DEBUG] [VALIDATOR ] LOW: Primary data validation passed. [2025-10-20T19:44:04.752328] [INFO ] [MAIN ] LOW: Fetched primary data is valid. [2025-10-20T19:44:04.752395] [DEBUG] [LOCAL ] LOW: Saving primary data. [2025-10-20T19:44:04.754030] [DEBUG] [LOCAL ] LOW: Trimmed 76732 records to 52993 (keeping 200 days of data) [2025-10-20T19:44:05.061951] [INFO ] [REMOTE ] LRN: Fetching primary data for 2025-10. [2025-10-20T19:44:05.080742] [DEBUG] [LOCAL ] LOW: Primary data saved (52993 records). [2025-10-20T19:44:05.080864] [INFO ] [MAIN ] LOW: Full history saved. [2025-10-20T19:44:05.081162] [INFO ] [MAIN ] Update Progress: 1271/2367 symbols completed. LOW took 54.0s. Estimated remaining: 2h 21m 26s. Rate: 7.7 symbols/min [2025-10-20T19:44:05.248925] [INFO ] [MAIN ] LTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:05.280619] [DEBUG] [LOCAL ] LTC: Loaded 31063 existing records from primary storage. [2025-10-20T19:44:05.280976] [INFO ] [MAIN ] LTC: Loaded 31063 existing records. [2025-10-20T19:44:05.456250] [WARN ] [MAIN ] LTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:05.456450] [INFO ] [MAIN ] LTC: Wiping existing primary data before full refresh. [2025-10-20T19:44:05.456496] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-10-20T19:44:05.456696] [DEBUG] [LOCAL ] LTC: Deleted primary data file. [2025-10-20T19:44:05.456752] [INFO ] [MAIN ] LTC: Starting full primary refresh. [2025-10-20T19:44:05.814808] [INFO ] [REMOTE ] LTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:05.851342] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-01. [2025-10-20T19:44:05.980476] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T19:44:06.651346] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-09. [2025-10-20T19:44:06.817043] [INFO ] [REMOTE ] LRN: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:44:06.822696] [INFO ] [REMOTE ] LRN: Full fetch complete (58438 records). [2025-10-20T19:44:06.823021] [DEBUG] [VALIDATOR ] LRN: Validating primary data. [2025-10-20T19:44:07.213549] [DEBUG] [VALIDATOR ] LRN: All expected trading days are present. [2025-10-20T19:44:07.213628] [DEBUG] [VALIDATOR ] LRN: Primary data validation passed. [2025-10-20T19:44:07.214552] [INFO ] [MAIN ] LRN: Fetched primary data is valid. [2025-10-20T19:44:07.214616] [DEBUG] [LOCAL ] LRN: Saving primary data. [2025-10-20T19:44:07.215797] [DEBUG] [LOCAL ] LRN: Trimmed 58438 records to 39505 (keeping 200 days of data) [2025-10-20T19:44:07.455425] [DEBUG] [LOCAL ] LRN: Primary data saved (39505 records). [2025-10-20T19:44:07.452580] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-07. [2025-10-20T19:44:07.455617] [INFO ] [MAIN ] LRN: Full history saved. [2025-10-20T19:44:07.456842] [INFO ] [MAIN ] Update Progress: 1272/2367 symbols completed. LRN took 24.5s. Estimated remaining: 2h 21m 13s. Rate: 7.8 symbols/min [2025-10-20T19:44:07.633077] [INFO ] [MAIN ] LTH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:07.650763] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T19:44:07.703116] [DEBUG] [LOCAL ] LTH: Loaded 52497 existing records from primary storage. [2025-10-20T19:44:07.703428] [INFO ] [MAIN ] LTH: Loaded 52497 existing records. [2025-10-20T19:44:07.878518] [WARN ] [MAIN ] LTH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:07.878910] [INFO ] [MAIN ] LTH: Wiping existing primary data before full refresh. [2025-10-20T19:44:07.879058] [INFO ] [LOCAL ] LTH: Starting deletion of primary data. [2025-10-20T19:44:07.879400] [DEBUG] [LOCAL ] LTH: Deleted primary data file. [2025-10-20T19:44:07.879612] [INFO ] [MAIN ] LTH: Starting full primary refresh. [2025-10-20T19:44:08.124268] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T19:44:08.266472] [INFO ] [REMOTE ] LSPD: Fetching primary data for 2025-10. [2025-10-20T19:44:08.271110] [INFO ] [REMOTE ] LTH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:08.679769] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T19:44:09.051438] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-08. [2025-10-20T19:44:09.851445] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-01. [2025-10-20T19:44:10.634361] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T19:44:10.651333] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-09. [2025-10-20T19:44:11.410827] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T19:44:11.451353] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-02. [2025-10-20T19:44:12.066355] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:44:12.251360] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-02. [2025-10-20T19:44:13.051453] [INFO ] [REMOTE ] LPX: Fetching primary data for 2025-10. [2025-10-20T19:44:13.099736] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:44:13.273573] [INFO ] [REMOTE ] LSPD: Incremental fetch complete for 2025-10 (5.01 seconds). [2025-10-20T19:44:13.280498] [INFO ] [REMOTE ] LSPD: Full fetch complete (62183 records). [2025-10-20T19:44:13.280734] [DEBUG] [VALIDATOR ] LSPD: Validating primary data. [2025-10-20T19:44:13.724177] [DEBUG] [VALIDATOR ] LSPD: All expected trading days are present. [2025-10-20T19:44:13.720808] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:44:13.724474] [DEBUG] [VALIDATOR ] LSPD: Primary data validation passed. [2025-10-20T19:44:13.734577] [INFO ] [MAIN ] LSPD: Fetched primary data is valid. [2025-10-20T19:44:13.734736] [DEBUG] [LOCAL ] LSPD: Saving primary data. [2025-10-20T19:44:13.737597] [DEBUG] [LOCAL ] LSPD: Trimmed 62183 records to 42440 (keeping 200 days of data) [2025-10-20T19:44:13.851450] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-03. [2025-10-20T19:44:13.981440] [DEBUG] [LOCAL ] LSPD: Primary data saved (42440 records). [2025-10-20T19:44:13.981571] [INFO ] [MAIN ] LSPD: Full history saved. [2025-10-20T19:44:13.981670] [INFO ] [MAIN ] Update Progress: 1273/2367 symbols completed. LSPD took 29.6s. Estimated remaining: 2h 21m 5s. Rate: 7.8 symbols/min [2025-10-20T19:44:14.149903] [INFO ] [MAIN ] LTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:14.186150] [DEBUG] [LOCAL ] LTM: Loaded 39389 existing records from primary storage. [2025-10-20T19:44:14.186462] [INFO ] [MAIN ] LTM: Loaded 39389 existing records. [2025-10-20T19:44:14.368207] [WARN ] [MAIN ] LTM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:14.368437] [INFO ] [MAIN ] LTM: Wiping existing primary data before full refresh. [2025-10-20T19:44:14.368483] [INFO ] [LOCAL ] LTM: Starting deletion of primary data. [2025-10-20T19:44:14.368771] [DEBUG] [LOCAL ] LTM: Deleted primary data file. [2025-10-20T19:44:14.368827] [INFO ] [MAIN ] LTM: Starting full primary refresh. [2025-10-20T19:44:14.662441] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-03. [2025-10-20T19:44:14.733443] [INFO ] [REMOTE ] LTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:14.806085] [INFO ] [REMOTE ] LPX: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T19:44:14.812347] [INFO ] [REMOTE ] LPX: Full fetch complete (58861 records). [2025-10-20T19:44:14.812482] [DEBUG] [VALIDATOR ] LPX: Validating primary data. [2025-10-20T19:44:15.250963] [DEBUG] [VALIDATOR ] LPX: All expected trading days are present. [2025-10-20T19:44:15.251060] [DEBUG] [VALIDATOR ] LPX: Primary data validation passed. [2025-10-20T19:44:15.251990] [INFO ] [MAIN ] LPX: Fetched primary data is valid. [2025-10-20T19:44:15.252056] [DEBUG] [LOCAL ] LPX: Saving primary data. [2025-10-20T19:44:15.253338] [DEBUG] [LOCAL ] LPX: Trimmed 58861 records to 40152 (keeping 200 days of data) [2025-10-20T19:44:15.460740] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-01. [2025-10-20T19:44:15.499608] [DEBUG] [LOCAL ] LPX: Primary data saved (40152 records). [2025-10-20T19:44:15.499775] [INFO ] [MAIN ] LPX: Full history saved. [2025-10-20T19:44:15.499856] [INFO ] [MAIN ] Update Progress: 1274/2367 symbols completed. LPX took 34.0s. Estimated remaining: 2h 20m 52s. Rate: 7.8 symbols/min [2025-10-20T19:44:15.670342] [INFO ] [MAIN ] LU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:15.703985] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T19:44:15.751227] [DEBUG] [LOCAL ] LU: Loaded 49804 existing records from primary storage. [2025-10-20T19:44:15.751555] [INFO ] [MAIN ] LU: Loaded 49804 existing records. [2025-10-20T19:44:15.932578] [WARN ] [MAIN ] LU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:15.932972] [INFO ] [MAIN ] LU: Wiping existing primary data before full refresh. [2025-10-20T19:44:15.933060] [INFO ] [LOCAL ] LU: Starting deletion of primary data. [2025-10-20T19:44:15.933555] [DEBUG] [LOCAL ] LU: Deleted primary data file. [2025-10-20T19:44:15.933616] [INFO ] [MAIN ] LU: Starting full primary refresh. [2025-10-20T19:44:16.271680] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-04. [2025-10-20T19:44:16.321352] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T19:44:16.325577] [INFO ] [REMOTE ] LU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:16.810399] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:44:17.051439] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-04. [2025-10-20T19:44:17.851421] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-01. [2025-10-20T19:44:18.075335] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:44:18.453361] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T19:44:18.651377] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-05. [2025-10-20T19:44:19.451385] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-02. [2025-10-20T19:44:19.522030] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:44:20.071437] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T19:44:20.251405] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-02. [2025-10-20T19:44:20.697013] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:44:21.051358] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-03. [2025-10-20T19:44:21.851432] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-06. [2025-10-20T19:44:21.884534] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:44:21.952404] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T19:44:22.651429] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-03. [2025-10-20T19:44:22.771615] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:44:23.451374] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-07. [2025-10-20T19:44:24.086581] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T19:44:24.251377] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-05. [2025-10-20T19:44:24.911789] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T19:44:25.051391] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-04. [2025-10-20T19:44:25.851410] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-04. [2025-10-20T19:44:25.921810] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T19:44:26.648119] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:44:26.657569] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-06. [2025-10-20T19:44:27.081160] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T19:44:27.451370] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-08. [2025-10-20T19:44:28.251413] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-05. [2025-10-20T19:44:28.295825] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:44:28.871554] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T19:44:29.051425] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-05. [2025-10-20T19:44:29.851365] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-09. [2025-10-20T19:44:29.978105] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T19:44:30.417256] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T19:44:30.651383] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-06. [2025-10-20T19:44:31.196909] [INFO ] [REMOTE ] LTC: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:44:31.451401] [INFO ] [REMOTE ] LTC: Fetching primary data for 2025-10. [2025-10-20T19:44:31.557503] [DEBUG] [REMOTE ] LTC: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:44:31.557856] [WARN ] [REMOTE ] LTC: No data returned for month 2025-10 [2025-10-20T19:44:31.557944] [FATAL] [REMOTE ] LTC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:44:31.558045] [FATAL] [MAIN ] LTC: Failed to fetch primary data. [2025-10-20T19:44:31.558124] [WARN ] [MAIN ] LTC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:44:31.558208] [INFO ] [LOCAL ] LTC: Starting deletion of primary data. [2025-10-20T19:44:31.558290] [DEBUG] [LOCAL ] LTC: No primary data file found to delete. [2025-10-20T19:44:31.558339] [INFO ] [LOCAL ] LTC: Starting deletion of derivative data. [2025-10-20T19:44:31.558518] [DEBUG] [LOCAL ] LTC: Deleted derivative data file. [2025-10-20T19:44:31.558922] [INFO ] [MAIN ] Update Progress: 1275/2367 symbols completed. LTC took 26.5s. Estimated remaining: 2h 20m 51s. Rate: 7.8 symbols/min [2025-10-20T19:44:31.742696] [INFO ] [MAIN ] LUCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:31.770180] [DEBUG] [LOCAL ] LUCK: Loaded 24521 existing records from primary storage. [2025-10-20T19:44:31.770483] [INFO ] [MAIN ] LUCK: Loaded 24521 existing records. [2025-10-20T19:44:31.960474] [WARN ] [MAIN ] LUCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:31.960861] [INFO ] [MAIN ] LUCK: Wiping existing primary data before full refresh. [2025-10-20T19:44:31.960936] [INFO ] [LOCAL ] LUCK: Starting deletion of primary data. [2025-10-20T19:44:31.961166] [DEBUG] [LOCAL ] LUCK: Deleted primary data file. [2025-10-20T19:44:31.961232] [INFO ] [MAIN ] LUCK: Starting full primary refresh. [2025-10-20T19:44:32.253885] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-06. [2025-10-20T19:44:32.368231] [INFO ] [REMOTE ] LUCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:32.396387] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:44:33.051376] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-01. [2025-10-20T19:44:33.641581] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T19:44:33.851405] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-07. [2025-10-20T19:44:34.499752] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T19:44:34.651432] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-07. [2025-10-20T19:44:35.451444] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-07. [2025-10-20T19:44:35.625259] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:44:36.092456] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:44:36.251408] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-08. [2025-10-20T19:44:37.051356] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-08. [2025-10-20T19:44:37.120571] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T19:44:37.817496] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T19:44:37.851379] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-09. [2025-10-20T19:44:38.651374] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-08. [2025-10-20T19:44:38.754024] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T19:44:39.442561] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:44:39.452819] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-02. [2025-10-20T19:44:40.157078] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:44:40.251343] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-09. [2025-10-20T19:44:40.415809] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T19:44:41.051375] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-09. [2025-10-20T19:44:41.846243] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:44:41.856357] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-03. [2025-10-20T19:44:42.651338] [INFO ] [REMOTE ] LTM: Fetching primary data for 2025-10. [2025-10-20T19:44:42.852485] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T19:44:43.082436] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T19:44:43.451394] [INFO ] [REMOTE ] LTH: Fetching primary data for 2025-10. [2025-10-20T19:44:44.210772] [INFO ] [REMOTE ] LTM: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T19:44:44.217811] [INFO ] [REMOTE ] LTM: Full fetch complete (50815 records). [2025-10-20T19:44:44.217900] [DEBUG] [VALIDATOR ] LTM: Validating primary data. [2025-10-20T19:44:44.260399] [INFO ] [REMOTE ] LU: Fetching primary data for 2025-10. [2025-10-20T19:44:44.598892] [DEBUG] [VALIDATOR ] LTM: All expected trading days are present. [2025-10-20T19:44:44.598991] [DEBUG] [VALIDATOR ] LTM: Primary data validation passed. [2025-10-20T19:44:44.599807] [INFO ] [MAIN ] LTM: Fetched primary data is valid. [2025-10-20T19:44:44.599868] [DEBUG] [LOCAL ] LTM: Saving primary data. [2025-10-20T19:44:44.600978] [DEBUG] [LOCAL ] LTM: Trimmed 50815 records to 39040 (keeping 200 days of data) [2025-10-20T19:44:44.822712] [DEBUG] [LOCAL ] LTM: Primary data saved (39040 records). [2025-10-20T19:44:44.822844] [INFO ] [MAIN ] LTM: Full history saved. [2025-10-20T19:44:44.822921] [INFO ] [MAIN ] Update Progress: 1276/2367 symbols completed. LTM took 30.8s. Estimated remaining: 2h 20m 48s. Rate: 7.7 symbols/min [2025-10-20T19:44:44.991059] [INFO ] [MAIN ] LUMN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:44:45.040153] [DEBUG] [LOCAL ] LUMN: Loaded 54426 existing records from primary storage. [2025-10-20T19:44:45.040451] [INFO ] [MAIN ] LUMN: Loaded 54426 existing records. [2025-10-20T19:44:45.071221] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-04. [2025-10-20T19:44:45.242764] [INFO ] [REMOTE ] LTH: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T19:44:45.246483] [WARN ] [MAIN ] LUMN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:44:45.254010] [INFO ] [MAIN ] LUMN: Wiping existing primary data before full refresh. [2025-10-20T19:44:45.254131] [INFO ] [LOCAL ] LUMN: Starting deletion of primary data. [2025-10-20T19:44:45.254296] [INFO ] [REMOTE ] LTH: Full fetch complete (74768 records). [2025-10-20T19:44:45.254449] [DEBUG] [LOCAL ] LUMN: Deleted primary data file. [2025-10-20T19:44:45.254531] [INFO ] [MAIN ] LUMN: Starting full primary refresh. [2025-10-20T19:44:45.254413] [DEBUG] [VALIDATOR ] LTH: Validating primary data. [2025-10-20T19:44:45.947217] [INFO ] [REMOTE ] LUMN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:44:45.950085] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-01. [2025-10-20T19:44:46.049662] [DEBUG] [VALIDATOR ] LTH: All expected trading days are present. [2025-10-20T19:44:46.049823] [DEBUG] [VALIDATOR ] LTH: Primary data validation passed. [2025-10-20T19:44:46.051133] [INFO ] [MAIN ] LTH: Fetched primary data is valid. [2025-10-20T19:44:46.051213] [DEBUG] [LOCAL ] LTH: Saving primary data. [2025-10-20T19:44:46.052786] [DEBUG] [LOCAL ] LTH: Trimmed 74768 records to 51686 (keeping 200 days of data) [2025-10-20T19:44:46.273094] [INFO ] [REMOTE ] LU: Incremental fetch complete for 2025-10 (2.01 seconds). [2025-10-20T19:44:46.302618] [INFO ] [REMOTE ] LU: Full fetch complete (70916 records). [2025-10-20T19:44:46.314741] [DEBUG] [VALIDATOR ] LU: Validating primary data. [2025-10-20T19:44:46.687909] [DEBUG] [LOCAL ] LTH: Primary data saved (51686 records). [2025-10-20T19:44:46.695731] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:44:46.701580] [INFO ] [MAIN ] LTH: Full history saved. [2025-10-20T19:44:46.712560] [INFO ] [MAIN ] Update Progress: 1277/2367 symbols completed. LTH took 39.3s. Estimated remaining: 2h 20m 35s. Rate: 7.8 symbols/min [2025-10-20T19:44:46.729394] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-05. [2025-10-20T19:44:46.821407] [DEBUG] [VALIDATOR ] LU: All expected trading days are present. [2025-10-20T19:44:46.821482] [DEBUG] [VALIDATOR ] LU: Primary data validation passed. [2025-10-20T19:44:46.822398] [INFO ] [MAIN ] LU: Fetched primary data is valid. [2025-10-20T19:44:46.822459] [DEBUG] [LOCAL ] LU: Saving primary data. [2025-10-20T19:44:46.823777] [DEBUG] [LOCAL ] LU: Trimmed 70916 records to 49163 (keeping 200 days of data) [2025-10-20T19:44:47.088908] [DEBUG] [LOCAL ] LU: Primary data saved (49163 records). [2025-10-20T19:44:47.089047] [INFO ] [MAIN ] LU: Full history saved. [2025-10-20T19:44:47.089126] [INFO ] [MAIN ] Update Progress: 1278/2367 symbols completed. LU took 31.6s. Estimated remaining: 2h 20m 21s. Rate: 7.8 symbols/min [2025-10-20T19:44:47.662559] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:44:47.679522] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-02. [2025-10-20T19:44:47.706080] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:44:48.251387] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-06. [2025-10-20T19:44:49.456777] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T19:44:49.463402] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-07. [2025-10-20T19:44:49.583582] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T19:44:49.851434] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-03. [2025-10-20T19:44:50.404928] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T19:44:50.651425] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-08. [2025-10-20T19:44:51.553348] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T19:44:51.559526] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-09. [2025-10-20T19:44:51.606818] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:44:52.251377] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-04. [2025-10-20T19:44:52.340949] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T19:44:53.051488] [INFO ] [REMOTE ] LUCK: Fetching primary data for 2025-10. [2025-10-20T19:44:54.070882] [INFO ] [REMOTE ] LUCK: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T19:44:54.074027] [INFO ] [REMOTE ] LUCK: Full fetch complete (39606 records). [2025-10-20T19:44:54.074116] [DEBUG] [VALIDATOR ] LUCK: Validating primary data. [2025-10-20T19:44:54.431306] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T19:44:54.489439] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-05. [2025-10-20T19:44:54.529965] [DEBUG] [VALIDATOR ] LUCK: All expected trading days are present. [2025-10-20T19:44:54.530041] [DEBUG] [VALIDATOR ] LUCK: Primary data validation passed. [2025-10-20T19:44:54.530830] [INFO ] [MAIN ] LUCK: Fetched primary data is valid. [2025-10-20T19:44:54.530892] [DEBUG] [LOCAL ] LUCK: Saving primary data. [2025-10-20T19:44:54.531776] [DEBUG] [LOCAL ] LUCK: Trimmed 39606 records to 24157 (keeping 200 days of data) [2025-10-20T19:44:54.672372] [DEBUG] [LOCAL ] LUCK: Primary data saved (24157 records). [2025-10-20T19:44:54.672493] [INFO ] [MAIN ] LUCK: Full history saved. [2025-10-20T19:44:54.672569] [INFO ] [MAIN ] Update Progress: 1279/2367 symbols completed. LUCK took 23.1s. Estimated remaining: 2h 20m 13s. Rate: 7.8 symbols/min [2025-10-20T19:44:56.469937] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T19:44:56.481153] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-06. [2025-10-20T19:44:58.349674] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:44:58.360712] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-07. [2025-10-20T19:45:00.327006] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T19:45:00.338567] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-08. [2025-10-20T19:45:02.452003] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T19:45:02.464015] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-09. [2025-10-20T19:45:04.678701] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T19:45:04.690234] [INFO ] [REMOTE ] LUMN: Fetching primary data for 2025-10. [2025-10-20T19:45:07.101028] [INFO ] [REMOTE ] LUMN: Incremental fetch complete for 2025-10 (2.41 seconds). [2025-10-20T19:45:07.112927] [INFO ] [REMOTE ] LUMN: Full fetch complete (77778 records). [2025-10-20T19:45:07.113112] [DEBUG] [VALIDATOR ] LUMN: Validating primary data. [2025-10-20T19:45:07.507367] [DEBUG] [VALIDATOR ] LUMN: All expected trading days are present. [2025-10-20T19:45:07.507447] [DEBUG] [VALIDATOR ] LUMN: Primary data validation passed. [2025-10-20T19:45:07.508444] [INFO ] [MAIN ] LUMN: Fetched primary data is valid. [2025-10-20T19:45:07.508507] [DEBUG] [LOCAL ] LUMN: Saving primary data. [2025-10-20T19:45:07.509967] [DEBUG] [LOCAL ] LUMN: Trimmed 77778 records to 53645 (keeping 200 days of data) [2025-10-20T19:45:07.803950] [DEBUG] [LOCAL ] LUMN: Primary data saved (53645 records). [2025-10-20T19:45:07.804085] [INFO ] [MAIN ] LUMN: Full history saved. [2025-10-20T19:45:07.804161] [INFO ] [MAIN ] Update Progress: 1280/2367 symbols completed. LUMN took 23.0s. Estimated remaining: 2h 20m 10s. Rate: 7.8 symbols/min [2025-10-20T19:45:08.409601] [INFO ] [MAIN ] LUXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.434347] [INFO ] [MAIN ] LUV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.491562] [DEBUG] [LOCAL ] LUXE: No existing data file found in primary storage. [2025-10-20T19:45:08.511423] [INFO ] [MAIN ] LVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.494436] [WARN ] [MAIN ] LUXE: No existing data found, performing full refresh instead. [2025-10-20T19:45:08.516475] [INFO ] [MAIN ] LUXE: Wiping existing primary data before full refresh. [2025-10-20T19:45:08.516564] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-10-20T19:45:08.516636] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-10-20T19:45:08.517220] [INFO ] [MAIN ] LUXE: Starting full primary refresh. [2025-10-20T19:45:08.516150] [INFO ] [MAIN ] LVWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:08.672919] [DEBUG] [LOCAL ] LVWR: Loaded 18573 existing records from primary storage. [2025-10-20T19:45:08.686885] [INFO ] [MAIN ] LVWR: Loaded 18573 existing records. [2025-10-20T19:45:09.035343] [DEBUG] [LOCAL ] LVS: Loaded 54350 existing records from primary storage. [2025-10-20T19:45:09.090432] [INFO ] [MAIN ] LVS: Loaded 54350 existing records. [2025-10-20T19:45:09.090287] [WARN ] [MAIN ] LVWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.107257] [DEBUG] [LOCAL ] LUV: Loaded 54469 existing records from primary storage. [2025-10-20T19:45:09.124071] [INFO ] [MAIN ] LVWR: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.181326] [INFO ] [MAIN ] LUV: Loaded 54469 existing records. [2025-10-20T19:45:09.238112] [INFO ] [REMOTE ] LUXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:09.199783] [INFO ] [LOCAL ] LVWR: Starting deletion of primary data. [2025-10-20T19:45:09.259263] [INFO ] [REMOTE ] LUXE: Fetching primary data for 2025-01. [2025-10-20T19:45:09.274892] [DEBUG] [LOCAL ] LVWR: Deleted primary data file. [2025-10-20T19:45:09.280460] [INFO ] [MAIN ] LVWR: Starting full primary refresh. [2025-10-20T19:45:09.579916] [WARN ] [MAIN ] LVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.653502] [WARN ] [MAIN ] LUV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:09.649823] [INFO ] [MAIN ] LVS: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.658888] [INFO ] [LOCAL ] LVS: Starting deletion of primary data. [2025-10-20T19:45:09.664130] [INFO ] [MAIN ] LUV: Wiping existing primary data before full refresh. [2025-10-20T19:45:09.688496] [DEBUG] [LOCAL ] LVS: Deleted primary data file. [2025-10-20T19:45:09.683333] [INFO ] [LOCAL ] LUV: Starting deletion of primary data. [2025-10-20T19:45:09.697671] [INFO ] [MAIN ] LVS: Starting full primary refresh. [2025-10-20T19:45:09.697730] [DEBUG] [LOCAL ] LUV: Deleted primary data file. [2025-10-20T19:45:09.768178] [INFO ] [MAIN ] LUV: Starting full primary refresh. [2025-10-20T19:45:09.768116] [DEBUG] [REMOTE ] LUXE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:45:09.861723] [WARN ] [REMOTE ] LUXE: No data returned for month 2025-01 [2025-10-20T19:45:09.997808] [FATAL] [REMOTE ] LUXE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:45:10.036853] [FATAL] [MAIN ] LUXE: Failed to fetch primary data. [2025-10-20T19:45:10.060358] [WARN ] [MAIN ] LUXE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:45:10.103590] [INFO ] [LOCAL ] LUXE: Starting deletion of primary data. [2025-10-20T19:45:10.209737] [DEBUG] [LOCAL ] LUXE: No primary data file found to delete. [2025-10-20T19:45:10.246955] [INFO ] [LOCAL ] LUXE: Starting deletion of derivative data. [2025-10-20T19:45:10.270006] [INFO ] [REMOTE ] LVWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.302055] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-01. [2025-10-20T19:45:10.358040] [DEBUG] [LOCAL ] LUXE: No derivative data file found to delete. [2025-10-20T19:45:10.426776] [INFO ] [MAIN ] Update Progress: 1281/2367 symbols completed. LUXE took 2.6s. Estimated remaining: 2h 19m 58s. Rate: 7.8 symbols/min [2025-10-20T19:45:10.877758] [INFO ] [REMOTE ] LUV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.859430] [INFO ] [REMOTE ] LVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:10.885121] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-01. [2025-10-20T19:45:10.884889] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-01. [2025-10-20T19:45:10.916292] [INFO ] [MAIN ] LW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:10.967409] [DEBUG] [LOCAL ] LW: Loaded 51966 existing records from primary storage. [2025-10-20T19:45:10.967793] [INFO ] [MAIN ] LW: Loaded 51966 existing records. [2025-10-20T19:45:11.092395] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:45:11.137258] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-02. [2025-10-20T19:45:11.159761] [WARN ] [MAIN ] LW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:11.160463] [INFO ] [MAIN ] LW: Wiping existing primary data before full refresh. [2025-10-20T19:45:11.160521] [INFO ] [LOCAL ] LW: Starting deletion of primary data. [2025-10-20T19:45:11.160751] [DEBUG] [LOCAL ] LW: Deleted primary data file. [2025-10-20T19:45:11.160808] [INFO ] [MAIN ] LW: Starting full primary refresh. [2025-10-20T19:45:11.533473] [INFO ] [REMOTE ] LW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:11.533853] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-01. [2025-10-20T19:45:11.739847] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T19:45:11.741536] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-03. [2025-10-20T19:45:12.208566] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T19:45:12.210201] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-04. [2025-10-20T19:45:12.714149] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T19:45:12.724974] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-02. [2025-10-20T19:45:12.990105] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T19:45:13.004996] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-02. [2025-10-20T19:45:13.014439] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T19:45:13.016276] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-05. [2025-10-20T19:45:13.385536] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T19:45:13.395988] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-02. [2025-10-20T19:45:13.916306] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T19:45:13.922322] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-06. [2025-10-20T19:45:14.481205] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:45:14.491472] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-03. [2025-10-20T19:45:14.815161] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T19:45:14.826422] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-03. [2025-10-20T19:45:14.978200] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T19:45:14.988290] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-03. [2025-10-20T19:45:15.391609] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:45:15.400616] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-07. [2025-10-20T19:45:16.322618] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T19:45:16.327486] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-08. [2025-10-20T19:45:16.455265] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:45:16.466325] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-04. [2025-10-20T19:45:16.637797] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T19:45:16.648924] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-04. [2025-10-20T19:45:16.739217] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:45:16.750344] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-04. [2025-10-20T19:45:16.855026] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T19:45:16.858173] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-09. [2025-10-20T19:45:17.841114] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:45:17.845628] [INFO ] [REMOTE ] LVWR: Fetching primary data for 2025-10. [2025-10-20T19:45:18.350756] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:45:18.363283] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-05. [2025-10-20T19:45:18.434001] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:45:18.446721] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-05. [2025-10-20T19:45:18.562071] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:45:18.651490] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-05. [2025-10-20T19:45:18.766309] [INFO ] [REMOTE ] LVWR: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T19:45:18.769169] [INFO ] [REMOTE ] LVWR: Full fetch complete (20660 records). [2025-10-20T19:45:18.769327] [DEBUG] [VALIDATOR ] LVWR: Validating primary data. [2025-10-20T19:45:19.212958] [DEBUG] [VALIDATOR ] LVWR: All expected trading days are present. [2025-10-20T19:45:19.213052] [DEBUG] [VALIDATOR ] LVWR: Primary data validation passed. [2025-10-20T19:45:19.213597] [INFO ] [MAIN ] LVWR: Fetched primary data is valid. [2025-10-20T19:45:19.213702] [DEBUG] [LOCAL ] LVWR: Saving primary data. [2025-10-20T19:45:19.214736] [DEBUG] [LOCAL ] LVWR: Trimmed 20660 records to 18569 (keeping 200 days of data) [2025-10-20T19:45:19.341353] [DEBUG] [LOCAL ] LVWR: Primary data saved (18569 records). [2025-10-20T19:45:19.341505] [INFO ] [MAIN ] LVWR: Full history saved. [2025-10-20T19:45:19.341590] [INFO ] [MAIN ] Update Progress: 1282/2367 symbols completed. LVWR took 11.5s. Estimated remaining: 2h 19m 51s. Rate: 7.8 symbols/min [2025-10-20T19:45:19.526921] [INFO ] [MAIN ] LXFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:19.546007] [DEBUG] [LOCAL ] LXFR: Loaded 16810 existing records from primary storage. [2025-10-20T19:45:19.546309] [INFO ] [MAIN ] LXFR: Loaded 16810 existing records. [2025-10-20T19:45:19.723796] [WARN ] [MAIN ] LXFR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:19.723994] [INFO ] [MAIN ] LXFR: Wiping existing primary data before full refresh. [2025-10-20T19:45:19.724042] [INFO ] [LOCAL ] LXFR: Starting deletion of primary data. [2025-10-20T19:45:19.724201] [DEBUG] [LOCAL ] LXFR: Deleted primary data file. [2025-10-20T19:45:19.724252] [INFO ] [MAIN ] LXFR: Starting full primary refresh. [2025-10-20T19:45:20.081156] [INFO ] [REMOTE ] LXFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:20.081816] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-01. [2025-10-20T19:45:20.121604] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T19:45:20.179098] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T19:45:20.251336] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-06. [2025-10-20T19:45:20.404000] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T19:45:21.051392] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-06. [2025-10-20T19:45:21.076952] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T19:45:21.804364] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T19:45:21.851411] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-06. [2025-10-20T19:45:22.651356] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-07. [2025-10-20T19:45:22.812338] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:45:23.447934] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:45:23.457221] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-02. [2025-10-20T19:45:24.251364] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-07. [2025-10-20T19:45:24.475459] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T19:45:24.666071] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T19:45:25.051367] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-08. [2025-10-20T19:45:25.851355] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-03. [2025-10-20T19:45:26.173149] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:45:26.651414] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-07. [2025-10-20T19:45:26.860400] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T19:45:27.011358] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T19:45:27.451398] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-09. [2025-10-20T19:45:28.251374] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-08. [2025-10-20T19:45:28.536630] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:45:29.051434] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-04. [2025-10-20T19:45:29.337212] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T19:45:29.851361] [INFO ] [REMOTE ] LUV: Fetching primary data for 2025-10. [2025-10-20T19:45:29.891160] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T19:45:29.941566] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T19:45:30.651398] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-09. [2025-10-20T19:45:30.715781] [INFO ] [REMOTE ] LUV: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:45:30.727996] [INFO ] [REMOTE ] LUV: Full fetch complete (77865 records). [2025-10-20T19:45:30.728187] [DEBUG] [VALIDATOR ] LUV: Validating primary data. [2025-10-20T19:45:31.153358] [DEBUG] [VALIDATOR ] LUV: All expected trading days are present. [2025-10-20T19:45:31.153483] [DEBUG] [VALIDATOR ] LUV: Primary data validation passed. [2025-10-20T19:45:31.154495] [INFO ] [MAIN ] LUV: Fetched primary data is valid. [2025-10-20T19:45:31.154612] [DEBUG] [LOCAL ] LUV: Saving primary data. [2025-10-20T19:45:31.156603] [DEBUG] [LOCAL ] LUV: Trimmed 77865 records to 53689 (keeping 200 days of data) [2025-10-20T19:45:31.462215] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-05. [2025-10-20T19:45:31.493184] [DEBUG] [LOCAL ] LUV: Primary data saved (53689 records). [2025-10-20T19:45:31.493342] [INFO ] [MAIN ] LUV: Full history saved. [2025-10-20T19:45:31.493435] [INFO ] [MAIN ] Update Progress: 1283/2367 symbols completed. LUV took 23.7s. Estimated remaining: 2h 19m 47s. Rate: 7.8 symbols/min [2025-10-20T19:45:31.663411] [INFO ] [MAIN ] LXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:31.712509] [DEBUG] [LOCAL ] LXP: Loaded 51460 existing records from primary storage. [2025-10-20T19:45:31.712925] [INFO ] [MAIN ] LXP: Loaded 51460 existing records. [2025-10-20T19:45:31.889126] [WARN ] [MAIN ] LXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:31.889369] [INFO ] [MAIN ] LXP: Wiping existing primary data before full refresh. [2025-10-20T19:45:31.889417] [INFO ] [LOCAL ] LXP: Starting deletion of primary data. [2025-10-20T19:45:31.889639] [DEBUG] [LOCAL ] LXP: Deleted primary data file. [2025-10-20T19:45:31.889757] [INFO ] [MAIN ] LXP: Starting full primary refresh. [2025-10-20T19:45:32.254218] [INFO ] [REMOTE ] LXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:32.254302] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-08. [2025-10-20T19:45:32.401438] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:45:32.436788] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T19:45:33.051374] [INFO ] [REMOTE ] LW: Fetching primary data for 2025-10. [2025-10-20T19:45:33.851364] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-06. [2025-10-20T19:45:34.090160] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:45:34.651351] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-01. [2025-10-20T19:45:34.817743] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T19:45:34.960878] [INFO ] [REMOTE ] LW: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T19:45:34.968609] [INFO ] [REMOTE ] LW: Full fetch complete (74766 records). [2025-10-20T19:45:34.968792] [DEBUG] [VALIDATOR ] LW: Validating primary data. [2025-10-20T19:45:35.368785] [DEBUG] [VALIDATOR ] LW: All expected trading days are present. [2025-10-20T19:45:35.368879] [DEBUG] [VALIDATOR ] LW: Primary data validation passed. [2025-10-20T19:45:35.370194] [INFO ] [MAIN ] LW: Fetched primary data is valid. [2025-10-20T19:45:35.370258] [DEBUG] [LOCAL ] LW: Saving primary data. [2025-10-20T19:45:35.371737] [DEBUG] [LOCAL ] LW: Trimmed 74766 records to 51191 (keeping 200 days of data) [2025-10-20T19:45:35.462066] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-07. [2025-10-20T19:45:35.683507] [DEBUG] [LOCAL ] LW: Primary data saved (51191 records). [2025-10-20T19:45:35.683672] [INFO ] [MAIN ] LW: Full history saved. [2025-10-20T19:45:35.683785] [INFO ] [MAIN ] Update Progress: 1284/2367 symbols completed. LW took 25.2s. Estimated remaining: 2h 19m 37s. Rate: 7.8 symbols/min [2025-10-20T19:45:35.854601] [INFO ] [MAIN ] LXU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:35.886092] [DEBUG] [LOCAL ] LXU: Loaded 32174 existing records from primary storage. [2025-10-20T19:45:35.886345] [INFO ] [MAIN ] LXU: Loaded 32174 existing records. [2025-10-20T19:45:36.077337] [WARN ] [MAIN ] LXU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:36.077553] [INFO ] [MAIN ] LXU: Wiping existing primary data before full refresh. [2025-10-20T19:45:36.077602] [INFO ] [LOCAL ] LXU: Starting deletion of primary data. [2025-10-20T19:45:36.077850] [DEBUG] [LOCAL ] LXU: Deleted primary data file. [2025-10-20T19:45:36.077907] [INFO ] [MAIN ] LXU: Starting full primary refresh. [2025-10-20T19:45:36.259312] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-09. [2025-10-20T19:45:36.485227] [INFO ] [REMOTE ] LXU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:36.486839] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T19:45:36.510004] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T19:45:37.051366] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-01. [2025-10-20T19:45:37.851439] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-08. [2025-10-20T19:45:38.287676] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T19:45:38.388985] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T19:45:38.437620] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T19:45:38.651369] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-02. [2025-10-20T19:45:39.451321] [INFO ] [REMOTE ] LVS: Fetching primary data for 2025-10. [2025-10-20T19:45:40.148325] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T19:45:40.251346] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-03. [2025-10-20T19:45:41.051373] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-09. [2025-10-20T19:45:41.546929] [INFO ] [REMOTE ] LVS: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:45:41.558277] [INFO ] [REMOTE ] LVS: Full fetch complete (77722 records). [2025-10-20T19:45:41.558446] [DEBUG] [VALIDATOR ] LVS: Validating primary data. [2025-10-20T19:45:41.863375] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-02. [2025-10-20T19:45:41.967910] [DEBUG] [VALIDATOR ] LVS: All expected trading days are present. [2025-10-20T19:45:41.968073] [DEBUG] [VALIDATOR ] LVS: Primary data validation passed. [2025-10-20T19:45:41.969226] [INFO ] [MAIN ] LVS: Fetched primary data is valid. [2025-10-20T19:45:41.969405] [DEBUG] [LOCAL ] LVS: Saving primary data. [2025-10-20T19:45:42.009383] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T19:45:42.008940] [DEBUG] [LOCAL ] LVS: Trimmed 77722 records to 53570 (keeping 200 days of data) [2025-10-20T19:45:42.011909] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T19:45:42.347324] [DEBUG] [LOCAL ] LVS: Primary data saved (53570 records). [2025-10-20T19:45:42.347458] [INFO ] [MAIN ] LVS: Full history saved. [2025-10-20T19:45:42.347878] [INFO ] [MAIN ] Update Progress: 1285/2367 symbols completed. LVS took 34.5s. Estimated remaining: 2h 19m 28s. Rate: 7.8 symbols/min [2025-10-20T19:45:42.514013] [INFO ] [MAIN ] LYB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:42.564384] [DEBUG] [LOCAL ] LYB: Loaded 54398 existing records from primary storage. [2025-10-20T19:45:42.564737] [INFO ] [MAIN ] LYB: Loaded 54398 existing records. [2025-10-20T19:45:42.656532] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-04. [2025-10-20T19:45:42.743684] [WARN ] [MAIN ] LYB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:42.743923] [INFO ] [MAIN ] LYB: Wiping existing primary data before full refresh. [2025-10-20T19:45:42.743972] [INFO ] [LOCAL ] LYB: Starting deletion of primary data. [2025-10-20T19:45:42.744306] [DEBUG] [LOCAL ] LYB: Deleted primary data file. [2025-10-20T19:45:42.744364] [INFO ] [MAIN ] LYB: Starting full primary refresh. [2025-10-20T19:45:43.129037] [INFO ] [REMOTE ] LYB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:43.130500] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T19:45:43.451385] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-01. [2025-10-20T19:45:44.251378] [INFO ] [REMOTE ] LXFR: Fetching primary data for 2025-10. [2025-10-20T19:45:44.430781] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:45:44.808175] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:45:45.051378] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-03. [2025-10-20T19:45:45.282108] [INFO ] [REMOTE ] LXFR: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T19:45:45.284549] [INFO ] [REMOTE ] LXFR: Full fetch complete (24745 records). [2025-10-20T19:45:45.284621] [DEBUG] [VALIDATOR ] LXFR: Validating primary data. [2025-10-20T19:45:45.657967] [DEBUG] [VALIDATOR ] LXFR: All expected trading days are present. [2025-10-20T19:45:45.658054] [DEBUG] [VALIDATOR ] LXFR: Primary data validation passed. [2025-10-20T19:45:45.658584] [INFO ] [MAIN ] LXFR: Fetched primary data is valid. [2025-10-20T19:45:45.658667] [DEBUG] [LOCAL ] LXFR: Saving primary data. [2025-10-20T19:45:45.659287] [DEBUG] [LOCAL ] LXFR: Trimmed 24745 records to 16472 (keeping 200 days of data) [2025-10-20T19:45:45.756421] [DEBUG] [LOCAL ] LXFR: Primary data saved (16472 records). [2025-10-20T19:45:45.756553] [INFO ] [MAIN ] LXFR: Full history saved. [2025-10-20T19:45:45.756632] [INFO ] [MAIN ] Update Progress: 1286/2367 symbols completed. LXFR took 26.4s. Estimated remaining: 2h 19m 17s. Rate: 7.8 symbols/min [2025-10-20T19:45:45.856557] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-02. [2025-10-20T19:45:45.930814] [INFO ] [MAIN ] LYG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:45:45.979978] [DEBUG] [LOCAL ] LYG: Loaded 53435 existing records from primary storage. [2025-10-20T19:45:45.980312] [INFO ] [MAIN ] LYG: Loaded 53435 existing records. [2025-10-20T19:45:46.174012] [WARN ] [MAIN ] LYG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:45:46.174357] [INFO ] [MAIN ] LYG: Wiping existing primary data before full refresh. [2025-10-20T19:45:46.174491] [INFO ] [LOCAL ] LYG: Starting deletion of primary data. [2025-10-20T19:45:46.175026] [DEBUG] [LOCAL ] LYG: Deleted primary data file. [2025-10-20T19:45:46.175081] [INFO ] [MAIN ] LYG: Starting full primary refresh. [2025-10-20T19:45:46.434860] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T19:45:46.569379] [INFO ] [REMOTE ] LYG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:45:46.651400] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-04. [2025-10-20T19:45:47.451439] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-05. [2025-10-20T19:45:47.589356] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T19:45:48.212192] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:45:48.251336] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-05. [2025-10-20T19:45:49.051383] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-03. [2025-10-20T19:45:49.059091] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:45:49.213901] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T19:45:49.851360] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-06. [2025-10-20T19:45:50.651347] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-06. [2025-10-20T19:45:50.964830] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:45:51.326248] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T19:45:51.451334] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-07. [2025-10-20T19:45:52.251383] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-01. [2025-10-20T19:45:52.323463] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:45:52.387984] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T19:45:53.051429] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-04. [2025-10-20T19:45:53.851415] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-07. [2025-10-20T19:45:54.009184] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:45:54.651411] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-08. [2025-10-20T19:45:54.952897] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T19:45:55.451405] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-05. [2025-10-20T19:45:55.493594] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T19:45:55.609685] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T19:45:56.251418] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-08. [2025-10-20T19:45:57.051350] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-09. [2025-10-20T19:45:57.363401] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:45:57.851344] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-06. [2025-10-20T19:45:57.898475] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T19:45:58.324414] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T19:45:58.651420] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-02. [2025-10-20T19:45:59.451401] [INFO ] [REMOTE ] LXU: Fetching primary data for 2025-10. [2025-10-20T19:45:59.550555] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T19:46:00.251477] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-07. [2025-10-20T19:46:00.392323] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T19:46:00.429732] [INFO ] [REMOTE ] LXU: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:46:00.434191] [INFO ] [REMOTE ] LXU: Full fetch complete (45306 records). [2025-10-20T19:46:00.434298] [DEBUG] [VALIDATOR ] LXU: Validating primary data. [2025-10-20T19:46:00.824924] [DEBUG] [VALIDATOR ] LXU: All expected trading days are present. [2025-10-20T19:46:00.825004] [DEBUG] [VALIDATOR ] LXU: Primary data validation passed. [2025-10-20T19:46:00.825832] [INFO ] [MAIN ] LXU: Fetched primary data is valid. [2025-10-20T19:46:00.825894] [DEBUG] [LOCAL ] LXU: Saving primary data. [2025-10-20T19:46:00.826915] [DEBUG] [LOCAL ] LXU: Trimmed 45306 records to 31669 (keeping 200 days of data) [2025-10-20T19:46:01.013945] [DEBUG] [LOCAL ] LXU: Primary data saved (31669 records). [2025-10-20T19:46:01.014121] [INFO ] [MAIN ] LXU: Full history saved. [2025-10-20T19:46:01.014240] [INFO ] [MAIN ] Update Progress: 1287/2367 symbols completed. LXU took 25.3s. Estimated remaining: 2h 19m 15s. Rate: 7.8 symbols/min [2025-10-20T19:46:01.056472] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-03. [2025-10-20T19:46:01.193782] [INFO ] [MAIN ] LYV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:01.244565] [DEBUG] [LOCAL ] LYV: Loaded 51878 existing records from primary storage. [2025-10-20T19:46:01.244913] [INFO ] [MAIN ] LYV: Loaded 51878 existing records. [2025-10-20T19:46:01.419923] [WARN ] [MAIN ] LYV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:01.420121] [INFO ] [MAIN ] LYV: Wiping existing primary data before full refresh. [2025-10-20T19:46:01.420188] [INFO ] [LOCAL ] LYV: Starting deletion of primary data. [2025-10-20T19:46:01.420365] [DEBUG] [LOCAL ] LYV: Deleted primary data file. [2025-10-20T19:46:01.420402] [INFO ] [MAIN ] LYV: Starting full primary refresh. [2025-10-20T19:46:01.766004] [INFO ] [REMOTE ] LYV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:01.851333] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-01. [2025-10-20T19:46:02.109579] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T19:46:02.651400] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-08. [2025-10-20T19:46:03.124261] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T19:46:03.451472] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-04. [2025-10-20T19:46:03.499384] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:46:04.251438] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-02. [2025-10-20T19:46:05.051354] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-09. [2025-10-20T19:46:05.302740] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-08 (2.65 seconds). [2025-10-20T19:46:05.626336] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T19:46:05.851362] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-05. [2025-10-20T19:46:06.051312] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:46:06.651402] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-03. [2025-10-20T19:46:06.742144] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T19:46:07.451350] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-09. [2025-10-20T19:46:07.644563] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T19:46:08.251443] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-06. [2025-10-20T19:46:08.475176] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:46:09.051348] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-04. [2025-10-20T19:46:09.453310] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T19:46:09.851437] [INFO ] [REMOTE ] LXP: Fetching primary data for 2025-10. [2025-10-20T19:46:10.007540] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:46:10.651391] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-07. [2025-10-20T19:46:10.794816] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:46:11.451470] [INFO ] [REMOTE ] LYB: Fetching primary data for 2025-10. [2025-10-20T19:46:11.607130] [INFO ] [REMOTE ] LXP: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T19:46:11.620083] [INFO ] [REMOTE ] LXP: Full fetch complete (72854 records). [2025-10-20T19:46:11.620421] [DEBUG] [VALIDATOR ] LXP: Validating primary data. [2025-10-20T19:46:12.262951] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-05. [2025-10-20T19:46:12.271547] [DEBUG] [VALIDATOR ] LXP: All expected trading days are present. [2025-10-20T19:46:12.271746] [DEBUG] [VALIDATOR ] LXP: Primary data validation passed. [2025-10-20T19:46:12.273904] [INFO ] [MAIN ] LXP: Fetched primary data is valid. [2025-10-20T19:46:12.274240] [DEBUG] [LOCAL ] LXP: Saving primary data. [2025-10-20T19:46:12.276177] [DEBUG] [LOCAL ] LXP: Trimmed 72854 records to 50793 (keeping 200 days of data) [2025-10-20T19:46:12.534224] [INFO ] [REMOTE ] LYB: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T19:46:12.576484] [INFO ] [REMOTE ] LYB: Full fetch complete (77632 records). [2025-10-20T19:46:12.576726] [DEBUG] [VALIDATOR ] LYB: Validating primary data. [2025-10-20T19:46:12.701439] [DEBUG] [LOCAL ] LXP: Primary data saved (50793 records). [2025-10-20T19:46:12.727929] [INFO ] [MAIN ] LXP: Full history saved. [2025-10-20T19:46:12.747379] [INFO ] [MAIN ] Update Progress: 1288/2367 symbols completed. LXP took 41.3s. Estimated remaining: 2h 19m 11s. Rate: 7.8 symbols/min [2025-10-20T19:46:13.003834] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T19:46:13.187722] [INFO ] [MAIN ] LZB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:13.200423] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-08. [2025-10-20T19:46:13.306555] [DEBUG] [VALIDATOR ] LYB: All expected trading days are present. [2025-10-20T19:46:13.306771] [DEBUG] [VALIDATOR ] LYB: Primary data validation passed. [2025-10-20T19:46:13.308020] [INFO ] [MAIN ] LYB: Fetched primary data is valid. [2025-10-20T19:46:13.308153] [DEBUG] [LOCAL ] LYB: Saving primary data. [2025-10-20T19:46:13.310468] [DEBUG] [LOCAL ] LYB: Trimmed 77632 records to 53621 (keeping 200 days of data) [2025-10-20T19:46:13.447812] [DEBUG] [LOCAL ] LZB: Loaded 35693 existing records from primary storage. [2025-10-20T19:46:13.452797] [INFO ] [MAIN ] LZB: Loaded 35693 existing records. [2025-10-20T19:46:13.751232] [WARN ] [MAIN ] LZB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:13.763594] [INFO ] [MAIN ] LZB: Wiping existing primary data before full refresh. [2025-10-20T19:46:13.765760] [INFO ] [LOCAL ] LZB: Starting deletion of primary data. [2025-10-20T19:46:13.769042] [DEBUG] [LOCAL ] LZB: Deleted primary data file. [2025-10-20T19:46:13.769910] [INFO ] [MAIN ] LZB: Starting full primary refresh. [2025-10-20T19:46:14.158275] [DEBUG] [LOCAL ] LYB: Primary data saved (53621 records). [2025-10-20T19:46:14.158466] [INFO ] [MAIN ] LYB: Full history saved. [2025-10-20T19:46:14.158980] [INFO ] [MAIN ] Update Progress: 1289/2367 symbols completed. LYB took 31.8s. Estimated remaining: 2h 18m 58s. Rate: 7.8 symbols/min [2025-10-20T19:46:14.287378] [INFO ] [REMOTE ] LZB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:14.332941] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-01. [2025-10-20T19:46:14.434991] [INFO ] [MAIN ] LZM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:14.463309] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-10-20T19:46:14.486938] [DEBUG] [LOCAL ] LZM: Loaded 14886 existing records from primary storage. [2025-10-20T19:46:14.487220] [INFO ] [MAIN ] LZM: Loaded 14886 existing records. [2025-10-20T19:46:14.663271] [WARN ] [MAIN ] LZM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:14.658686] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-06. [2025-10-20T19:46:14.664230] [INFO ] [MAIN ] LZM: Wiping existing primary data before full refresh. [2025-10-20T19:46:14.664284] [INFO ] [LOCAL ] LZM: Starting deletion of primary data. [2025-10-20T19:46:14.664432] [DEBUG] [LOCAL ] LZM: Deleted primary data file. [2025-10-20T19:46:14.664508] [INFO ] [MAIN ] LZM: Starting full primary refresh. [2025-10-20T19:46:15.051119] [INFO ] [REMOTE ] LZM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:15.046183] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T19:46:15.451331] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-09. [2025-10-20T19:46:15.686848] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T19:46:16.004578] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T19:46:16.251364] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-01. [2025-10-20T19:46:17.050751] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T19:46:17.052923] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-02. [2025-10-20T19:46:17.360407] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T19:46:17.851369] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-07. [2025-10-20T19:46:18.493137] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:46:18.651373] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-02. [2025-10-20T19:46:19.375597] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T19:46:19.451334] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-03. [2025-10-20T19:46:19.619728] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T19:46:20.251390] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-03. [2025-10-20T19:46:21.011812] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T19:46:21.051341] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-04. [2025-10-20T19:46:21.151737] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T19:46:21.851397] [INFO ] [REMOTE ] LYG: Fetching primary data for 2025-10. [2025-10-20T19:46:22.511827] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T19:46:22.651374] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-04. [2025-10-20T19:46:23.451355] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-05. [2025-10-20T19:46:23.702152] [INFO ] [REMOTE ] LYG: Incremental fetch complete for 2025-10 (1.85 seconds). [2025-10-20T19:46:23.715964] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:46:23.725678] [INFO ] [REMOTE ] LYG: Full fetch complete (76212 records). [2025-10-20T19:46:23.725803] [DEBUG] [VALIDATOR ] LYG: Validating primary data. [2025-10-20T19:46:24.109049] [DEBUG] [VALIDATOR ] LYG: All expected trading days are present. [2025-10-20T19:46:24.109142] [DEBUG] [VALIDATOR ] LYG: Primary data validation passed. [2025-10-20T19:46:24.110126] [INFO ] [MAIN ] LYG: Fetched primary data is valid. [2025-10-20T19:46:24.110189] [DEBUG] [LOCAL ] LYG: Saving primary data. [2025-10-20T19:46:24.111594] [DEBUG] [LOCAL ] LYG: Trimmed 76212 records to 52666 (keeping 200 days of data) [2025-10-20T19:46:24.274693] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-05. [2025-10-20T19:46:24.414824] [DEBUG] [LOCAL ] LYG: Primary data saved (52666 records). [2025-10-20T19:46:24.414954] [INFO ] [MAIN ] LYG: Full history saved. [2025-10-20T19:46:24.415027] [INFO ] [MAIN ] Update Progress: 1290/2367 symbols completed. LYG took 38.7s. Estimated remaining: 2h 18m 52s. Rate: 7.8 symbols/min [2025-10-20T19:46:24.580810] [INFO ] [MAIN ] M: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:24.634380] [DEBUG] [LOCAL ] M: Loaded 54325 existing records from primary storage. [2025-10-20T19:46:24.634714] [INFO ] [MAIN ] M: Loaded 54325 existing records. [2025-10-20T19:46:24.827811] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T19:46:24.832484] [WARN ] [MAIN ] M: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:24.839673] [INFO ] [MAIN ] M: Wiping existing primary data before full refresh. [2025-10-20T19:46:24.839766] [INFO ] [LOCAL ] M: Starting deletion of primary data. [2025-10-20T19:46:24.840337] [DEBUG] [LOCAL ] M: Deleted primary data file. [2025-10-20T19:46:24.840397] [INFO ] [MAIN ] M: Starting full primary refresh. [2025-10-20T19:46:25.056878] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-06. [2025-10-20T19:46:25.167317] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T19:46:25.210073] [INFO ] [REMOTE ] M: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:25.851438] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-06. [2025-10-20T19:46:26.597188] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:46:26.612960] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T19:46:26.651404] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-07. [2025-10-20T19:46:27.451382] [INFO ] [REMOTE ] M: Fetching primary data for 2025-01. [2025-10-20T19:46:27.564870] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:46:28.251349] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-08. [2025-10-20T19:46:29.051428] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-08. [2025-10-20T19:46:29.354697] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T19:46:29.812885] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T19:46:29.851359] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-09. [2025-10-20T19:46:29.884901] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T19:46:30.651396] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-07. [2025-10-20T19:46:30.681221] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:46:31.451426] [INFO ] [REMOTE ] M: Fetching primary data for 2025-02. [2025-10-20T19:46:32.050987] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T19:46:32.251441] [INFO ] [REMOTE ] LZM: Fetching primary data for 2025-10. [2025-10-20T19:46:33.051373] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-09. [2025-10-20T19:46:33.248582] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:46:33.380384] [INFO ] [REMOTE ] LZM: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T19:46:33.384099] [INFO ] [REMOTE ] LZM: Full fetch complete (19736 records). [2025-10-20T19:46:33.384194] [DEBUG] [VALIDATOR ] LZM: Validating primary data. [2025-10-20T19:46:33.744092] [DEBUG] [VALIDATOR ] LZM: All expected trading days are present. [2025-10-20T19:46:33.744172] [DEBUG] [VALIDATOR ] LZM: Primary data validation passed. [2025-10-20T19:46:33.744566] [INFO ] [MAIN ] LZM: Fetched primary data is valid. [2025-10-20T19:46:33.744604] [DEBUG] [LOCAL ] LZM: Saving primary data. [2025-10-20T19:46:33.745213] [DEBUG] [LOCAL ] LZM: Trimmed 19736 records to 14708 (keeping 200 days of data) [2025-10-20T19:46:33.825057] [DEBUG] [LOCAL ] LZM: Primary data saved (14708 records). [2025-10-20T19:46:33.825231] [INFO ] [MAIN ] LZM: Full history saved. [2025-10-20T19:46:33.825315] [INFO ] [MAIN ] Update Progress: 1291/2367 symbols completed. LZM took 19.6s. Estimated remaining: 2h 18m 46s. Rate: 7.8 symbols/min [2025-10-20T19:46:33.856574] [INFO ] [REMOTE ] M: Fetching primary data for 2025-03. [2025-10-20T19:46:33.995965] [INFO ] [MAIN ] MA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:34.045365] [DEBUG] [LOCAL ] MA: Loaded 53252 existing records from primary storage. [2025-10-20T19:46:34.045692] [INFO ] [MAIN ] MA: Loaded 53252 existing records. [2025-10-20T19:46:34.221928] [WARN ] [MAIN ] MA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:34.222169] [INFO ] [MAIN ] MA: Wiping existing primary data before full refresh. [2025-10-20T19:46:34.222224] [INFO ] [LOCAL ] MA: Starting deletion of primary data. [2025-10-20T19:46:34.222811] [DEBUG] [LOCAL ] MA: Deleted primary data file. [2025-10-20T19:46:34.222879] [INFO ] [MAIN ] MA: Starting full primary refresh. [2025-10-20T19:46:34.584874] [INFO ] [REMOTE ] MA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:34.651328] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-01. [2025-10-20T19:46:34.826281] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T19:46:35.451397] [INFO ] [REMOTE ] LYV: Fetching primary data for 2025-10. [2025-10-20T19:46:35.755206] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T19:46:36.251361] [INFO ] [REMOTE ] M: Fetching primary data for 2025-04. [2025-10-20T19:46:36.286933] [INFO ] [REMOTE ] LYV: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T19:46:36.294832] [INFO ] [REMOTE ] LYV: Full fetch complete (74181 records). [2025-10-20T19:46:36.294943] [DEBUG] [VALIDATOR ] LYV: Validating primary data. [2025-10-20T19:46:36.689728] [DEBUG] [VALIDATOR ] LYV: All expected trading days are present. [2025-10-20T19:46:36.689835] [DEBUG] [VALIDATOR ] LYV: Primary data validation passed. [2025-10-20T19:46:36.690841] [INFO ] [MAIN ] LYV: Fetched primary data is valid. [2025-10-20T19:46:36.690901] [DEBUG] [LOCAL ] LYV: Saving primary data. [2025-10-20T19:46:36.692423] [DEBUG] [LOCAL ] LYV: Trimmed 74181 records to 51086 (keeping 200 days of data) [2025-10-20T19:46:36.810483] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-01 (2.16 seconds). [2025-10-20T19:46:37.026841] [DEBUG] [LOCAL ] LYV: Primary data saved (51086 records). [2025-10-20T19:46:37.026967] [INFO ] [MAIN ] LYV: Full history saved. [2025-10-20T19:46:37.027047] [INFO ] [MAIN ] Update Progress: 1292/2367 symbols completed. LYV took 36.0s. Estimated remaining: 2h 18m 34s. Rate: 7.8 symbols/min [2025-10-20T19:46:37.057519] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-08. [2025-10-20T19:46:37.198344] [INFO ] [MAIN ] MAA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:37.239685] [DEBUG] [LOCAL ] MAA: Loaded 43519 existing records from primary storage. [2025-10-20T19:46:37.239991] [INFO ] [MAIN ] MAA: Loaded 43519 existing records. [2025-10-20T19:46:37.428688] [WARN ] [MAIN ] MAA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:37.428915] [INFO ] [MAIN ] MAA: Wiping existing primary data before full refresh. [2025-10-20T19:46:37.428974] [INFO ] [LOCAL ] MAA: Starting deletion of primary data. [2025-10-20T19:46:37.429175] [DEBUG] [LOCAL ] MAA: Deleted primary data file. [2025-10-20T19:46:37.429246] [INFO ] [MAIN ] MAA: Starting full primary refresh. [2025-10-20T19:46:37.785802] [INFO ] [REMOTE ] MAA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:37.851335] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-01. [2025-10-20T19:46:38.219263] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T19:46:38.595977] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T19:46:38.651388] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-02. [2025-10-20T19:46:39.352093] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T19:46:39.451370] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-09. [2025-10-20T19:46:40.251361] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-02. [2025-10-20T19:46:40.487448] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T19:46:40.869989] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T19:46:41.051412] [INFO ] [REMOTE ] LZB: Fetching primary data for 2025-10. [2025-10-20T19:46:41.725026] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:46:41.851405] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-03. [2025-10-20T19:46:42.619748] [INFO ] [REMOTE ] LZB: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T19:46:42.624615] [INFO ] [REMOTE ] LZB: Full fetch complete (50965 records). [2025-10-20T19:46:42.624725] [DEBUG] [VALIDATOR ] LZB: Validating primary data. [2025-10-20T19:46:42.656494] [INFO ] [REMOTE ] M: Fetching primary data for 2025-05. [2025-10-20T19:46:42.996022] [DEBUG] [VALIDATOR ] LZB: All expected trading days are present. [2025-10-20T19:46:42.996099] [DEBUG] [VALIDATOR ] LZB: Primary data validation passed. [2025-10-20T19:46:42.996869] [INFO ] [MAIN ] LZB: Fetched primary data is valid. [2025-10-20T19:46:42.996931] [DEBUG] [LOCAL ] LZB: Saving primary data. [2025-10-20T19:46:42.997882] [DEBUG] [LOCAL ] LZB: Trimmed 50965 records to 35168 (keeping 200 days of data) [2025-10-20T19:46:43.202898] [DEBUG] [LOCAL ] LZB: Primary data saved (35168 records). [2025-10-20T19:46:43.203015] [INFO ] [MAIN ] LZB: Full history saved. [2025-10-20T19:46:43.203087] [INFO ] [MAIN ] Update Progress: 1293/2367 symbols completed. LZB took 30.5s. Estimated remaining: 2h 18m 25s. Rate: 7.8 symbols/min [2025-10-20T19:46:43.374099] [INFO ] [MAIN ] MAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:46:43.424563] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T19:46:43.449309] [DEBUG] [LOCAL ] MAC: Loaded 50453 existing records from primary storage. [2025-10-20T19:46:43.449591] [INFO ] [MAIN ] MAC: Loaded 50453 existing records. [2025-10-20T19:46:43.460711] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-04. [2025-10-20T19:46:43.625280] [WARN ] [MAIN ] MAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:46:43.625487] [INFO ] [MAIN ] MAC: Wiping existing primary data before full refresh. [2025-10-20T19:46:43.625534] [INFO ] [LOCAL ] MAC: Starting deletion of primary data. [2025-10-20T19:46:43.625760] [DEBUG] [LOCAL ] MAC: Deleted primary data file. [2025-10-20T19:46:43.625817] [INFO ] [MAIN ] MAC: Starting full primary refresh. [2025-10-20T19:46:43.972755] [INFO ] [REMOTE ] MAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:46:44.251372] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-01. [2025-10-20T19:46:44.641304] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T19:46:45.051411] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-03. [2025-10-20T19:46:45.136297] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T19:46:45.416054] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T19:46:45.851368] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-05. [2025-10-20T19:46:46.651347] [INFO ] [REMOTE ] M: Fetching primary data for 2025-06. [2025-10-20T19:46:46.874066] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T19:46:47.343058] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T19:46:47.451340] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-06. [2025-10-20T19:46:48.251346] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-02. [2025-10-20T19:46:48.516879] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T19:46:49.051407] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-04. [2025-10-20T19:46:49.113228] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:46:49.395079] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:46:49.851465] [INFO ] [REMOTE ] M: Fetching primary data for 2025-07. [2025-10-20T19:46:50.651400] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-07. [2025-10-20T19:46:51.011909] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T19:46:51.451346] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-03. [2025-10-20T19:46:51.741050] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T19:46:51.907032] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T19:46:52.251413] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-08. [2025-10-20T19:46:53.051459] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-05. [2025-10-20T19:46:53.197480] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:46:53.784639] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T19:46:53.851383] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-09. [2025-10-20T19:46:54.651429] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-04. [2025-10-20T19:46:55.451354] [INFO ] [REMOTE ] M: Fetching primary data for 2025-08. [2025-10-20T19:46:55.661313] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-05 (2.61 seconds). [2025-10-20T19:46:55.736158] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:46:56.251352] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-06. [2025-10-20T19:46:56.358520] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:46:57.051436] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-05. [2025-10-20T19:46:57.174158] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:46:57.851352] [INFO ] [REMOTE ] M: Fetching primary data for 2025-09. [2025-10-20T19:46:58.200509] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:46:58.388488] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T19:46:58.651448] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-06. [2025-10-20T19:46:59.451408] [INFO ] [REMOTE ] MAA: Fetching primary data for 2025-10. [2025-10-20T19:46:59.801994] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T19:47:00.251410] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-07. [2025-10-20T19:47:00.286826] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:47:01.051373] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-07. [2025-10-20T19:47:01.454103] [INFO ] [REMOTE ] MAA: Incremental fetch complete for 2025-10 (2.00 seconds). [2025-10-20T19:47:01.460308] [INFO ] [REMOTE ] MAA: Full fetch complete (61664 records). [2025-10-20T19:47:01.460427] [DEBUG] [VALIDATOR ] MAA: Validating primary data. [2025-10-20T19:47:01.836488] [DEBUG] [VALIDATOR ] MAA: All expected trading days are present. [2025-10-20T19:47:01.836574] [DEBUG] [VALIDATOR ] MAA: Primary data validation passed. [2025-10-20T19:47:01.837454] [INFO ] [MAIN ] MAA: Fetched primary data is valid. [2025-10-20T19:47:01.837517] [DEBUG] [LOCAL ] MAA: Saving primary data. [2025-10-20T19:47:01.838822] [DEBUG] [LOCAL ] MAA: Trimmed 61664 records to 42908 (keeping 200 days of data) [2025-10-20T19:47:01.892616] [INFO ] [REMOTE ] M: Fetching primary data for 2025-10. [2025-10-20T19:47:02.106785] [DEBUG] [LOCAL ] MAA: Primary data saved (42908 records). [2025-10-20T19:47:02.106902] [INFO ] [MAIN ] MAA: Full history saved. [2025-10-20T19:47:02.106973] [INFO ] [MAIN ] Update Progress: 1294/2367 symbols completed. MAA took 25.1s. Estimated remaining: 2h 18m 27s. Rate: 7.8 symbols/min [2025-10-20T19:47:02.281365] [INFO ] [MAIN ] MAGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:02.316777] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T19:47:02.354105] [DEBUG] [LOCAL ] MAGN: Loaded 36703 existing records from primary storage. [2025-10-20T19:47:02.354386] [INFO ] [MAIN ] MAGN: Loaded 36703 existing records. [2025-10-20T19:47:02.550973] [WARN ] [MAIN ] MAGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:02.551163] [INFO ] [MAIN ] MAGN: Wiping existing primary data before full refresh. [2025-10-20T19:47:02.551209] [INFO ] [LOCAL ] MAGN: Starting deletion of primary data. [2025-10-20T19:47:02.551724] [DEBUG] [LOCAL ] MAGN: Deleted primary data file. [2025-10-20T19:47:02.551818] [INFO ] [MAIN ] MAGN: Starting full primary refresh. [2025-10-20T19:47:02.651484] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-08. [2025-10-20T19:47:02.931547] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T19:47:02.935744] [INFO ] [REMOTE ] MAGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:03.451335] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-08. [2025-10-20T19:47:03.933781] [INFO ] [REMOTE ] M: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T19:47:03.941553] [INFO ] [REMOTE ] M: Full fetch complete (77658 records). [2025-10-20T19:47:03.941713] [DEBUG] [VALIDATOR ] M: Validating primary data. [2025-10-20T19:47:04.256262] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-01. [2025-10-20T19:47:04.406070] [DEBUG] [VALIDATOR ] M: All expected trading days are present. [2025-10-20T19:47:04.406266] [DEBUG] [VALIDATOR ] M: Primary data validation passed. [2025-10-20T19:47:04.407834] [INFO ] [MAIN ] M: Fetched primary data is valid. [2025-10-20T19:47:04.408072] [DEBUG] [LOCAL ] M: Saving primary data. [2025-10-20T19:47:04.410516] [DEBUG] [LOCAL ] M: Trimmed 77658 records to 53544 (keeping 200 days of data) [2025-10-20T19:47:04.565757] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T19:47:04.877094] [DEBUG] [LOCAL ] M: Primary data saved (53544 records). [2025-10-20T19:47:04.877388] [INFO ] [MAIN ] M: Full history saved. [2025-10-20T19:47:04.877535] [INFO ] [MAIN ] Update Progress: 1295/2367 symbols completed. M took 40.5s. Estimated remaining: 2h 18m 15s. Rate: 7.8 symbols/min [2025-10-20T19:47:05.055157] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-09. [2025-10-20T19:47:05.128280] [INFO ] [MAIN ] MAIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:05.166260] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T19:47:05.219638] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T19:47:05.243886] [DEBUG] [LOCAL ] MAIN: Loaded 42467 existing records from primary storage. [2025-10-20T19:47:05.244389] [INFO ] [MAIN ] MAIN: Loaded 42467 existing records. [2025-10-20T19:47:05.437612] [WARN ] [MAIN ] MAIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:05.437885] [INFO ] [MAIN ] MAIN: Wiping existing primary data before full refresh. [2025-10-20T19:47:05.437952] [INFO ] [LOCAL ] MAIN: Starting deletion of primary data. [2025-10-20T19:47:05.438199] [DEBUG] [LOCAL ] MAIN: Deleted primary data file. [2025-10-20T19:47:05.438260] [INFO ] [MAIN ] MAIN: Starting full primary refresh. [2025-10-20T19:47:05.784559] [INFO ] [REMOTE ] MAIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:05.851331] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-01. [2025-10-20T19:47:06.651404] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-02. [2025-10-20T19:47:07.024671] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:47:07.451385] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-09. [2025-10-20T19:47:07.569078] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:47:07.983029] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T19:47:08.251425] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-03. [2025-10-20T19:47:09.051383] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-02. [2025-10-20T19:47:09.331771] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T19:47:09.851338] [INFO ] [REMOTE ] MAC: Fetching primary data for 2025-10. [2025-10-20T19:47:09.980339] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:47:10.651374] [INFO ] [REMOTE ] MA: Fetching primary data for 2025-10. [2025-10-20T19:47:11.458292] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-04. [2025-10-20T19:47:11.460962] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-02 (2.41 seconds). [2025-10-20T19:47:11.831226] [INFO ] [REMOTE ] MAC: Incremental fetch complete for 2025-10 (1.98 seconds). [2025-10-20T19:47:11.839298] [INFO ] [REMOTE ] MAC: Full fetch complete (71608 records). [2025-10-20T19:47:11.839463] [DEBUG] [VALIDATOR ] MAC: Validating primary data. [2025-10-20T19:47:12.253462] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-03. [2025-10-20T19:47:12.275903] [DEBUG] [VALIDATOR ] MAC: All expected trading days are present. [2025-10-20T19:47:12.275985] [DEBUG] [VALIDATOR ] MAC: Primary data validation passed. [2025-10-20T19:47:12.277053] [INFO ] [MAIN ] MAC: Fetched primary data is valid. [2025-10-20T19:47:12.277121] [DEBUG] [LOCAL ] MAC: Saving primary data. [2025-10-20T19:47:12.278788] [DEBUG] [LOCAL ] MAC: Trimmed 71608 records to 49690 (keeping 200 days of data) [2025-10-20T19:47:12.587684] [DEBUG] [LOCAL ] MAC: Primary data saved (49690 records). [2025-10-20T19:47:12.587954] [INFO ] [MAIN ] MAC: Full history saved. [2025-10-20T19:47:12.588101] [INFO ] [MAIN ] Update Progress: 1296/2367 symbols completed. MAC took 29.4s. Estimated remaining: 2h 18m 7s. Rate: 7.8 symbols/min [2025-10-20T19:47:12.750714] [INFO ] [REMOTE ] MA: Incremental fetch complete for 2025-10 (2.10 seconds). [2025-10-20T19:47:12.790717] [INFO ] [MAIN ] MAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:12.791088] [INFO ] [REMOTE ] MA: Full fetch complete (76410 records). [2025-10-20T19:47:12.791749] [DEBUG] [VALIDATOR ] MA: Validating primary data. [2025-10-20T19:47:13.163206] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T19:47:13.169256] [DEBUG] [LOCAL ] MAN: Loaded 44786 existing records from primary storage. [2025-10-20T19:47:13.210034] [INFO ] [MAIN ] MAN: Loaded 44786 existing records. [2025-10-20T19:47:13.264960] [DEBUG] [VALIDATOR ] MA: All expected trading days are present. [2025-10-20T19:47:13.278116] [DEBUG] [VALIDATOR ] MA: Primary data validation passed. [2025-10-20T19:47:13.298040] [INFO ] [MAIN ] MA: Fetched primary data is valid. [2025-10-20T19:47:13.278444] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-05. [2025-10-20T19:47:13.317599] [DEBUG] [LOCAL ] MA: Saving primary data. [2025-10-20T19:47:13.393957] [DEBUG] [LOCAL ] MA: Trimmed 76410 records to 52467 (keeping 200 days of data) [2025-10-20T19:47:13.541824] [WARN ] [MAIN ] MAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:13.568533] [INFO ] [MAIN ] MAN: Wiping existing primary data before full refresh. [2025-10-20T19:47:13.586575] [INFO ] [LOCAL ] MAN: Starting deletion of primary data. [2025-10-20T19:47:13.598008] [DEBUG] [LOCAL ] MAN: Deleted primary data file. [2025-10-20T19:47:13.600031] [INFO ] [MAIN ] MAN: Starting full primary refresh. [2025-10-20T19:47:14.186071] [INFO ] [REMOTE ] MAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:14.188757] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-01. [2025-10-20T19:47:14.203258] [DEBUG] [LOCAL ] MA: Primary data saved (52467 records). [2025-10-20T19:47:14.203669] [INFO ] [MAIN ] MA: Full history saved. [2025-10-20T19:47:14.204216] [INFO ] [MAIN ] Update Progress: 1297/2367 symbols completed. MA took 40.4s. Estimated remaining: 2h 17m 54s. Rate: 7.8 symbols/min [2025-10-20T19:47:14.331675] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:47:14.408341] [INFO ] [MAIN ] MANU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:14.436530] [DEBUG] [LOCAL ] MANU: Loaded 27565 existing records from primary storage. [2025-10-20T19:47:14.436809] [INFO ] [MAIN ] MANU: Loaded 27565 existing records. [2025-10-20T19:47:14.612738] [WARN ] [MAIN ] MANU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:14.612920] [INFO ] [MAIN ] MANU: Wiping existing primary data before full refresh. [2025-10-20T19:47:14.612966] [INFO ] [LOCAL ] MANU: Starting deletion of primary data. [2025-10-20T19:47:14.613138] [DEBUG] [LOCAL ] MANU: Deleted primary data file. [2025-10-20T19:47:14.613175] [INFO ] [MAIN ] MANU: Starting full primary refresh. [2025-10-20T19:47:14.655375] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-04. [2025-10-20T19:47:14.916173] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:47:14.999949] [INFO ] [REMOTE ] MANU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:15.451403] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-01. [2025-10-20T19:47:15.759604] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:47:16.251412] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-06. [2025-10-20T19:47:16.606625] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T19:47:16.643429] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T19:47:17.051390] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-02. [2025-10-20T19:47:17.540438] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T19:47:17.851389] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-05. [2025-10-20T19:47:18.449911] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T19:47:18.651444] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-03. [2025-10-20T19:47:19.448812] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T19:47:19.457426] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-02. [2025-10-20T19:47:20.251426] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-06. [2025-10-20T19:47:20.319052] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T19:47:20.627824] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T19:47:21.051396] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-07. [2025-10-20T19:47:21.851387] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-03. [2025-10-20T19:47:21.906439] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T19:47:22.523134] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T19:47:22.651671] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-07. [2025-10-20T19:47:23.125759] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T19:47:23.451382] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-04. [2025-10-20T19:47:24.248311] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T19:47:24.256993] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-04. [2025-10-20T19:47:24.606981] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T19:47:25.051450] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-08. [2025-10-20T19:47:25.859605] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-05. [2025-10-20T19:47:25.879955] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:47:26.350696] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:47:26.651481] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-09. [2025-10-20T19:47:27.003142] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T19:47:27.451412] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-08. [2025-10-20T19:47:28.251398] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-05. [2025-10-20T19:47:28.286588] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T19:47:29.051378] [INFO ] [REMOTE ] MAGN: Fetching primary data for 2025-10. [2025-10-20T19:47:29.335292] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T19:47:29.851389] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-06. [2025-10-20T19:47:30.117326] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T19:47:30.651409] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-09. [2025-10-20T19:47:30.993154] [INFO ] [REMOTE ] MAGN: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:47:31.000388] [INFO ] [REMOTE ] MAGN: Full fetch complete (51892 records). [2025-10-20T19:47:31.000540] [DEBUG] [VALIDATOR ] MAGN: Validating primary data. [2025-10-20T19:47:31.440008] [DEBUG] [VALIDATOR ] MAGN: All expected trading days are present. [2025-10-20T19:47:31.442673] [DEBUG] [VALIDATOR ] MAGN: Primary data validation passed. [2025-10-20T19:47:31.444944] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T19:47:31.443843] [INFO ] [MAIN ] MAGN: Fetched primary data is valid. [2025-10-20T19:47:31.451832] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-06. [2025-10-20T19:47:31.453772] [DEBUG] [LOCAL ] MAGN: Saving primary data. [2025-10-20T19:47:31.454983] [DEBUG] [LOCAL ] MAGN: Trimmed 51892 records to 36341 (keeping 200 days of data) [2025-10-20T19:47:31.667616] [DEBUG] [LOCAL ] MAGN: Primary data saved (36341 records). [2025-10-20T19:47:31.667800] [INFO ] [MAIN ] MAGN: Full history saved. [2025-10-20T19:47:31.667887] [INFO ] [MAIN ] Update Progress: 1298/2367 symbols completed. MAGN took 29.6s. Estimated remaining: 2h 17m 55s. Rate: 7.8 symbols/min [2025-10-20T19:47:31.843154] [INFO ] [MAIN ] MAS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:31.894680] [DEBUG] [LOCAL ] MAS: Loaded 53044 existing records from primary storage. [2025-10-20T19:47:31.895011] [INFO ] [MAIN ] MAS: Loaded 53044 existing records. [2025-10-20T19:47:32.077467] [WARN ] [MAIN ] MAS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:32.077743] [INFO ] [MAIN ] MAS: Wiping existing primary data before full refresh. [2025-10-20T19:47:32.077798] [INFO ] [LOCAL ] MAS: Starting deletion of primary data. [2025-10-20T19:47:32.077978] [DEBUG] [LOCAL ] MAS: Deleted primary data file. [2025-10-20T19:47:32.078016] [INFO ] [MAIN ] MAS: Starting full primary refresh. [2025-10-20T19:47:32.261424] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-07. [2025-10-20T19:47:32.443700] [INFO ] [REMOTE ] MAS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:32.469744] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T19:47:32.979011] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T19:47:33.051379] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-07. [2025-10-20T19:47:33.649835] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T19:47:33.851389] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-01. [2025-10-20T19:47:34.592587] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T19:47:34.651349] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-08. [2025-10-20T19:47:35.451351] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-08. [2025-10-20T19:47:35.590523] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T19:47:35.723701] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T19:47:36.251365] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-02. [2025-10-20T19:47:36.498386] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:47:37.051408] [INFO ] [REMOTE ] MAIN: Fetching primary data for 2025-10. [2025-10-20T19:47:37.851468] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-09. [2025-10-20T19:47:37.907378] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:47:38.651477] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-03. [2025-10-20T19:47:38.938413] [INFO ] [REMOTE ] MAIN: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T19:47:38.945293] [INFO ] [REMOTE ] MAIN: Full fetch complete (60425 records). [2025-10-20T19:47:38.945430] [DEBUG] [VALIDATOR ] MAIN: Validating primary data. [2025-10-20T19:47:39.340964] [DEBUG] [VALIDATOR ] MAIN: All expected trading days are present. [2025-10-20T19:47:39.341105] [DEBUG] [VALIDATOR ] MAIN: Primary data validation passed. [2025-10-20T19:47:39.341993] [INFO ] [MAIN ] MAIN: Fetched primary data is valid. [2025-10-20T19:47:39.342099] [DEBUG] [LOCAL ] MAIN: Saving primary data. [2025-10-20T19:47:39.343621] [DEBUG] [LOCAL ] MAIN: Trimmed 60425 records to 41956 (keeping 200 days of data) [2025-10-20T19:47:39.370535] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:47:39.465305] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-09. [2025-10-20T19:47:39.637787] [DEBUG] [LOCAL ] MAIN: Primary data saved (41956 records). [2025-10-20T19:47:39.637939] [INFO ] [MAIN ] MAIN: Full history saved. [2025-10-20T19:47:39.638020] [INFO ] [MAIN ] Update Progress: 1299/2367 symbols completed. MAIN took 34.8s. Estimated remaining: 2h 17m 47s. Rate: 7.8 symbols/min [2025-10-20T19:47:39.806272] [INFO ] [MAIN ] MATV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:39.839711] [DEBUG] [LOCAL ] MATV: Loaded 36069 existing records from primary storage. [2025-10-20T19:47:39.839980] [INFO ] [MAIN ] MATV: Loaded 36069 existing records. [2025-10-20T19:47:40.019475] [WARN ] [MAIN ] MATV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:40.019706] [INFO ] [MAIN ] MATV: Wiping existing primary data before full refresh. [2025-10-20T19:47:40.019765] [INFO ] [LOCAL ] MATV: Starting deletion of primary data. [2025-10-20T19:47:40.020011] [DEBUG] [LOCAL ] MATV: Deleted primary data file. [2025-10-20T19:47:40.020067] [INFO ] [MAIN ] MATV: Starting full primary refresh. [2025-10-20T19:47:40.252221] [INFO ] [REMOTE ] MAN: Fetching primary data for 2025-10. [2025-10-20T19:47:40.377543] [INFO ] [REMOTE ] MATV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:40.633955] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T19:47:40.927132] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:47:41.051398] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-04. [2025-10-20T19:47:41.851385] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-01. [2025-10-20T19:47:41.946486] [INFO ] [REMOTE ] MAN: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T19:47:41.953394] [INFO ] [REMOTE ] MAN: Full fetch complete (62781 records). [2025-10-20T19:47:41.953548] [DEBUG] [VALIDATOR ] MAN: Validating primary data. [2025-10-20T19:47:42.344049] [DEBUG] [VALIDATOR ] MAN: All expected trading days are present. [2025-10-20T19:47:42.344323] [DEBUG] [VALIDATOR ] MAN: Primary data validation passed. [2025-10-20T19:47:42.345221] [INFO ] [MAIN ] MAN: Fetched primary data is valid. [2025-10-20T19:47:42.345282] [DEBUG] [LOCAL ] MAN: Saving primary data. [2025-10-20T19:47:42.346479] [DEBUG] [LOCAL ] MAN: Trimmed 62781 records to 44330 (keeping 200 days of data) [2025-10-20T19:47:42.600635] [DEBUG] [LOCAL ] MAN: Primary data saved (44330 records). [2025-10-20T19:47:42.600790] [INFO ] [MAIN ] MAN: Full history saved. [2025-10-20T19:47:42.600877] [INFO ] [MAIN ] Update Progress: 1300/2367 symbols completed. MAN took 30.0s. Estimated remaining: 2h 17m 35s. Rate: 7.8 symbols/min [2025-10-20T19:47:42.663860] [INFO ] [REMOTE ] MANU: Fetching primary data for 2025-10. [2025-10-20T19:47:42.773914] [INFO ] [MAIN ] MATX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:42.807242] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T19:47:42.843819] [DEBUG] [LOCAL ] MATX: Loaded 32418 existing records from primary storage. [2025-10-20T19:47:42.844085] [INFO ] [MAIN ] MATX: Loaded 32418 existing records. [2025-10-20T19:47:43.018562] [WARN ] [MAIN ] MATX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:43.018792] [INFO ] [MAIN ] MATX: Wiping existing primary data before full refresh. [2025-10-20T19:47:43.018839] [INFO ] [LOCAL ] MATX: Starting deletion of primary data. [2025-10-20T19:47:43.019216] [DEBUG] [LOCAL ] MATX: Deleted primary data file. [2025-10-20T19:47:43.019272] [INFO ] [MAIN ] MATX: Starting full primary refresh. [2025-10-20T19:47:43.258866] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T19:47:43.396744] [INFO ] [REMOTE ] MATX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:43.451328] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-01. [2025-10-20T19:47:44.132599] [INFO ] [REMOTE ] MANU: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:47:44.137373] [INFO ] [REMOTE ] MANU: Full fetch complete (39239 records). [2025-10-20T19:47:44.137448] [DEBUG] [VALIDATOR ] MANU: Validating primary data. [2025-10-20T19:47:44.255670] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-05. [2025-10-20T19:47:44.517226] [DEBUG] [VALIDATOR ] MANU: All expected trading days are present. [2025-10-20T19:47:44.517360] [DEBUG] [VALIDATOR ] MANU: Primary data validation passed. [2025-10-20T19:47:44.518202] [INFO ] [MAIN ] MANU: Fetched primary data is valid. [2025-10-20T19:47:44.518334] [DEBUG] [LOCAL ] MANU: Saving primary data. [2025-10-20T19:47:44.553153] [DEBUG] [LOCAL ] MANU: Trimmed 39239 records to 27281 (keeping 200 days of data) [2025-10-20T19:47:44.553048] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:47:44.708622] [DEBUG] [LOCAL ] MANU: Primary data saved (27281 records). [2025-10-20T19:47:44.708766] [INFO ] [MAIN ] MANU: Full history saved. [2025-10-20T19:47:44.708846] [INFO ] [MAIN ] Update Progress: 1301/2367 symbols completed. MANU took 30.5s. Estimated remaining: 2h 17m 23s. Rate: 7.8 symbols/min [2025-10-20T19:47:44.875177] [INFO ] [MAIN ] MAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:47:44.905881] [DEBUG] [LOCAL ] MAX: Loaded 31450 existing records from primary storage. [2025-10-20T19:47:44.906141] [INFO ] [MAIN ] MAX: Loaded 31450 existing records. [2025-10-20T19:47:45.061759] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-02. [2025-10-20T19:47:45.082684] [WARN ] [MAIN ] MAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:47:45.082933] [INFO ] [MAIN ] MAX: Wiping existing primary data before full refresh. [2025-10-20T19:47:45.082982] [INFO ] [LOCAL ] MAX: Starting deletion of primary data. [2025-10-20T19:47:45.083159] [DEBUG] [LOCAL ] MAX: Deleted primary data file. [2025-10-20T19:47:45.083214] [INFO ] [MAIN ] MAX: Starting full primary refresh. [2025-10-20T19:47:45.436811] [INFO ] [REMOTE ] MAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:47:45.851352] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-01. [2025-10-20T19:47:46.009072] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T19:47:46.651400] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-06. [2025-10-20T19:47:46.863892] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T19:47:47.364132] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T19:47:47.451381] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-03. [2025-10-20T19:47:48.028692] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T19:47:48.251443] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-02. [2025-10-20T19:47:49.051448] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-07. [2025-10-20T19:47:49.163879] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T19:47:49.475346] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T19:47:49.851398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-03. [2025-10-20T19:47:50.651379] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-02. [2025-10-20T19:47:50.845975] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T19:47:51.150134] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:47:51.451427] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-04. [2025-10-20T19:47:51.885709] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T19:47:52.251398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-04. [2025-10-20T19:47:52.776100] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T19:47:53.051396] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-03. [2025-10-20T19:47:53.373821] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:47:53.851474] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-05. [2025-10-20T19:47:54.536486] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:47:54.651398] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-05. [2025-10-20T19:47:55.283322] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T19:47:55.451396] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-06. [2025-10-20T19:47:56.135428] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T19:47:56.251397] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-06. [2025-10-20T19:47:56.874006] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T19:47:57.051525] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-07. [2025-10-20T19:47:57.085374] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:47:57.851436] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-08. [2025-10-20T19:47:58.479376] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:47:58.651400] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-08. [2025-10-20T19:47:59.451380] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-04. [2025-10-20T19:48:00.251366] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-07. [2025-10-20T19:48:00.432795] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-08 (2.58 seconds). [2025-10-20T19:48:00.988971] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-08 (2.34 seconds). [2025-10-20T19:48:01.051362] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-09. [2025-10-20T19:48:01.644906] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T19:48:01.851347] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-05. [2025-10-20T19:48:02.651350] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-09. [2025-10-20T19:48:02.800915] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-07 (2.55 seconds). [2025-10-20T19:48:03.089243] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:48:03.181221] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-09 (2.13 seconds). [2025-10-20T19:48:03.451369] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-08. [2025-10-20T19:48:04.251400] [INFO ] [REMOTE ] MATV: Fetching primary data for 2025-10. [2025-10-20T19:48:04.642823] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T19:48:04.762561] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:48:05.051374] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-06. [2025-10-20T19:48:05.651724] [INFO ] [REMOTE ] MATV: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:48:05.656966] [INFO ] [REMOTE ] MATV: Full fetch complete (52266 records). [2025-10-20T19:48:05.657076] [DEBUG] [VALIDATOR ] MATV: Validating primary data. [2025-10-20T19:48:05.856538] [INFO ] [REMOTE ] MAS: Fetching primary data for 2025-10. [2025-10-20T19:48:06.097305] [DEBUG] [VALIDATOR ] MATV: All expected trading days are present. [2025-10-20T19:48:06.097386] [DEBUG] [VALIDATOR ] MATV: Primary data validation passed. [2025-10-20T19:48:06.098262] [INFO ] [MAIN ] MATV: Fetched primary data is valid. [2025-10-20T19:48:06.098326] [DEBUG] [LOCAL ] MATV: Saving primary data. [2025-10-20T19:48:06.099484] [DEBUG] [LOCAL ] MATV: Trimmed 52266 records to 35365 (keeping 200 days of data) [2025-10-20T19:48:06.205770] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T19:48:06.332216] [DEBUG] [LOCAL ] MATV: Primary data saved (35365 records). [2025-10-20T19:48:06.332348] [INFO ] [MAIN ] MATV: Full history saved. [2025-10-20T19:48:06.332424] [INFO ] [MAIN ] Update Progress: 1302/2367 symbols completed. MATV took 26.7s. Estimated remaining: 2h 17m 27s. Rate: 7.7 symbols/min [2025-10-20T19:48:06.499601] [INFO ] [MAIN ] MBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:06.540691] [DEBUG] [LOCAL ] MBC: Loaded 44551 existing records from primary storage. [2025-10-20T19:48:06.540986] [INFO ] [MAIN ] MBC: Loaded 44551 existing records. [2025-10-20T19:48:06.656442] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-09. [2025-10-20T19:48:06.716463] [WARN ] [MAIN ] MBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:06.716667] [INFO ] [MAIN ] MBC: Wiping existing primary data before full refresh. [2025-10-20T19:48:06.716727] [INFO ] [LOCAL ] MBC: Starting deletion of primary data. [2025-10-20T19:48:06.716911] [DEBUG] [LOCAL ] MBC: Deleted primary data file. [2025-10-20T19:48:06.716948] [INFO ] [MAIN ] MBC: Starting full primary refresh. [2025-10-20T19:48:07.063205] [INFO ] [REMOTE ] MBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:07.451377] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-01. [2025-10-20T19:48:07.783827] [INFO ] [REMOTE ] MAS: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T19:48:07.791620] [INFO ] [REMOTE ] MAS: Full fetch complete (75351 records). [2025-10-20T19:48:07.791781] [DEBUG] [VALIDATOR ] MAS: Validating primary data. [2025-10-20T19:48:08.012140] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:48:08.261044] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-07. [2025-10-20T19:48:08.289751] [DEBUG] [VALIDATOR ] MAS: All expected trading days are present. [2025-10-20T19:48:08.289883] [DEBUG] [VALIDATOR ] MAS: Primary data validation passed. [2025-10-20T19:48:08.291952] [INFO ] [MAIN ] MAS: Fetched primary data is valid. [2025-10-20T19:48:08.292058] [DEBUG] [LOCAL ] MAS: Saving primary data. [2025-10-20T19:48:08.294625] [DEBUG] [LOCAL ] MAS: Trimmed 75351 records to 52281 (keeping 200 days of data) [2025-10-20T19:48:08.653268] [DEBUG] [LOCAL ] MAS: Primary data saved (52281 records). [2025-10-20T19:48:08.653473] [INFO ] [MAIN ] MAS: Full history saved. [2025-10-20T19:48:08.653588] [INFO ] [MAIN ] Update Progress: 1303/2367 symbols completed. MAS took 37.0s. Estimated remaining: 2h 17m 15s. Rate: 7.8 symbols/min [2025-10-20T19:48:08.844827] [INFO ] [MAIN ] MBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:08.869223] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:48:08.901309] [DEBUG] [LOCAL ] MBI: Loaded 24689 existing records from primary storage. [2025-10-20T19:48:08.901708] [INFO ] [MAIN ] MBI: Loaded 24689 existing records. [2025-10-20T19:48:09.054571] [INFO ] [REMOTE ] MATX: Fetching primary data for 2025-10. [2025-10-20T19:48:09.147727] [WARN ] [MAIN ] MBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:09.148080] [INFO ] [MAIN ] MBI: Wiping existing primary data before full refresh. [2025-10-20T19:48:09.148136] [INFO ] [LOCAL ] MBI: Starting deletion of primary data. [2025-10-20T19:48:09.148415] [DEBUG] [LOCAL ] MBI: Deleted primary data file. [2025-10-20T19:48:09.148471] [INFO ] [MAIN ] MBI: Starting full primary refresh. [2025-10-20T19:48:09.534444] [INFO ] [REMOTE ] MBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:09.551724] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T19:48:09.851359] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-01. [2025-10-20T19:48:10.318055] [INFO ] [REMOTE ] MATX: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T19:48:10.322906] [INFO ] [REMOTE ] MATX: Full fetch complete (43895 records). [2025-10-20T19:48:10.322980] [DEBUG] [VALIDATOR ] MATX: Validating primary data. [2025-10-20T19:48:10.659586] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-08. [2025-10-20T19:48:10.692282] [DEBUG] [VALIDATOR ] MATX: All expected trading days are present. [2025-10-20T19:48:10.692416] [DEBUG] [VALIDATOR ] MATX: Primary data validation passed. [2025-10-20T19:48:10.693190] [INFO ] [MAIN ] MATX: Fetched primary data is valid. [2025-10-20T19:48:10.693263] [DEBUG] [LOCAL ] MATX: Saving primary data. [2025-10-20T19:48:10.694582] [DEBUG] [LOCAL ] MATX: Trimmed 43895 records to 32025 (keeping 200 days of data) [2025-10-20T19:48:10.891903] [DEBUG] [LOCAL ] MATX: Primary data saved (32025 records). [2025-10-20T19:48:10.892055] [INFO ] [MAIN ] MATX: Full history saved. [2025-10-20T19:48:10.892172] [INFO ] [MAIN ] Update Progress: 1304/2367 symbols completed. MATX took 28.3s. Estimated remaining: 2h 17m 2s. Rate: 7.8 symbols/min [2025-10-20T19:48:11.034151] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T19:48:11.081302] [INFO ] [MAIN ] MC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:11.120368] [DEBUG] [LOCAL ] MC: Loaded 44544 existing records from primary storage. [2025-10-20T19:48:11.120675] [INFO ] [MAIN ] MC: Loaded 44544 existing records. [2025-10-20T19:48:11.291975] [WARN ] [MAIN ] MC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:11.292181] [INFO ] [MAIN ] MC: Wiping existing primary data before full refresh. [2025-10-20T19:48:11.292227] [INFO ] [LOCAL ] MC: Starting deletion of primary data. [2025-10-20T19:48:11.292425] [DEBUG] [LOCAL ] MC: Deleted primary data file. [2025-10-20T19:48:11.292481] [INFO ] [MAIN ] MC: Starting full primary refresh. [2025-10-20T19:48:11.459574] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-02. [2025-10-20T19:48:11.643661] [INFO ] [REMOTE ] MC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:12.041957] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T19:48:12.251380] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-09. [2025-10-20T19:48:13.051398] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-02. [2025-10-20T19:48:13.114423] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T19:48:13.710879] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T19:48:13.851383] [INFO ] [REMOTE ] MAX: Fetching primary data for 2025-10. [2025-10-20T19:48:14.174533] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T19:48:14.651438] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-01. [2025-10-20T19:48:15.195582] [INFO ] [REMOTE ] MAX: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:48:15.201154] [INFO ] [REMOTE ] MAX: Full fetch complete (45363 records). [2025-10-20T19:48:15.201301] [DEBUG] [VALIDATOR ] MAX: Validating primary data. [2025-10-20T19:48:15.456624] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-03. [2025-10-20T19:48:15.602054] [DEBUG] [VALIDATOR ] MAX: All expected trading days are present. [2025-10-20T19:48:15.602134] [DEBUG] [VALIDATOR ] MAX: Primary data validation passed. [2025-10-20T19:48:15.602876] [INFO ] [MAIN ] MAX: Fetched primary data is valid. [2025-10-20T19:48:15.602939] [DEBUG] [LOCAL ] MAX: Saving primary data. [2025-10-20T19:48:15.603897] [DEBUG] [LOCAL ] MAX: Trimmed 45363 records to 30945 (keeping 200 days of data) [2025-10-20T19:48:15.781003] [DEBUG] [LOCAL ] MAX: Primary data saved (30945 records). [2025-10-20T19:48:15.781227] [INFO ] [MAIN ] MAX: Full history saved. [2025-10-20T19:48:15.781627] [INFO ] [MAIN ] Update Progress: 1305/2367 symbols completed. MAX took 31.1s. Estimated remaining: 2h 16m 52s. Rate: 7.8 symbols/min [2025-10-20T19:48:15.957494] [INFO ] [MAIN ] MCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:15.971074] [DEBUG] [LOCAL ] MCB: Loaded 11549 existing records from primary storage. [2025-10-20T19:48:15.971315] [INFO ] [MAIN ] MCB: Loaded 11549 existing records. [2025-10-20T19:48:16.164445] [WARN ] [MAIN ] MCB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:16.164700] [INFO ] [MAIN ] MCB: Wiping existing primary data before full refresh. [2025-10-20T19:48:16.164771] [INFO ] [LOCAL ] MCB: Starting deletion of primary data. [2025-10-20T19:48:16.172830] [DEBUG] [LOCAL ] MCB: Deleted primary data file. [2025-10-20T19:48:16.185732] [INFO ] [MAIN ] MCB: Starting full primary refresh. [2025-10-20T19:48:16.220224] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T19:48:16.253160] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-02. [2025-10-20T19:48:16.569227] [INFO ] [REMOTE ] MCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:16.808960] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T19:48:17.051399] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-04. [2025-10-20T19:48:17.857696] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-01. [2025-10-20T19:48:17.862600] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T19:48:18.204195] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:48:18.651397] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-03. [2025-10-20T19:48:18.703141] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T19:48:19.451408] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-02. [2025-10-20T19:48:20.231539] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:48:20.254202] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-03. [2025-10-20T19:48:20.265984] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T19:48:21.051400] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-05. [2025-10-20T19:48:21.746527] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:48:21.856448] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-03. [2025-10-20T19:48:21.865686] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:48:22.651354] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-06. [2025-10-20T19:48:22.730839] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:48:23.451400] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-04. [2025-10-20T19:48:23.733562] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:48:24.251402] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-07. [2025-10-20T19:48:24.965204] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:48:25.051370] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-04. [2025-10-20T19:48:25.370613] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T19:48:25.851395] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-08. [2025-10-20T19:48:26.657961] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-05. [2025-10-20T19:48:26.663994] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:48:27.213208] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T19:48:27.451403] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-09. [2025-10-20T19:48:28.108952] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T19:48:28.251348] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-04. [2025-10-20T19:48:28.544112] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T19:48:29.051343] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-06. [2025-10-20T19:48:29.314683] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T19:48:29.851382] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-05. [2025-10-20T19:48:30.445615] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T19:48:30.653981] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-05. [2025-10-20T19:48:30.655076] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T19:48:31.451419] [INFO ] [REMOTE ] MBI: Fetching primary data for 2025-10. [2025-10-20T19:48:32.254280] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-07. [2025-10-20T19:48:32.259972] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T19:48:32.576382] [INFO ] [REMOTE ] MBI: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T19:48:32.580210] [INFO ] [REMOTE ] MBI: Full fetch complete (36199 records). [2025-10-20T19:48:32.580322] [DEBUG] [VALIDATOR ] MBI: Validating primary data. [2025-10-20T19:48:32.953185] [DEBUG] [VALIDATOR ] MBI: All expected trading days are present. [2025-10-20T19:48:32.953269] [DEBUG] [VALIDATOR ] MBI: Primary data validation passed. [2025-10-20T19:48:32.953959] [INFO ] [MAIN ] MBI: Fetched primary data is valid. [2025-10-20T19:48:32.954023] [DEBUG] [LOCAL ] MBI: Saving primary data. [2025-10-20T19:48:32.955059] [DEBUG] [LOCAL ] MBI: Trimmed 36199 records to 24304 (keeping 200 days of data) [2025-10-20T19:48:33.052277] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-06. [2025-10-20T19:48:33.085953] [DEBUG] [LOCAL ] MBI: Primary data saved (24304 records). [2025-10-20T19:48:33.086057] [INFO ] [MAIN ] MBI: Full history saved. [2025-10-20T19:48:33.086128] [INFO ] [MAIN ] Update Progress: 1306/2367 symbols completed. MBI took 24.4s. Estimated remaining: 2h 16m 52s. Rate: 7.8 symbols/min [2025-10-20T19:48:33.251253] [INFO ] [MAIN ] MCD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:33.251385] [DEBUG] [LOCAL ] MCD: No existing data file found in primary storage. [2025-10-20T19:48:33.251423] [WARN ] [MAIN ] MCD: No existing data found, performing full refresh instead. [2025-10-20T19:48:33.251529] [INFO ] [MAIN ] MCD: Wiping existing primary data before full refresh. [2025-10-20T19:48:33.251573] [INFO ] [LOCAL ] MCD: Starting deletion of primary data. [2025-10-20T19:48:33.251682] [DEBUG] [LOCAL ] MCD: No primary data file found to delete. [2025-10-20T19:48:33.251747] [INFO ] [MAIN ] MCD: Starting full primary refresh. [2025-10-20T19:48:33.607763] [INFO ] [REMOTE ] MCD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:33.692197] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T19:48:33.851343] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-01. [2025-10-20T19:48:34.546204] [INFO ] [REMOTE ] MC: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T19:48:34.651380] [INFO ] [REMOTE ] MC: Fetching primary data for 2025-07. [2025-10-20T19:48:34.754716] [DEBUG] [REMOTE ] MC: Error fetching primary data for 2025-07: 'API call returned an empty response.' [2025-10-20T19:48:34.755037] [WARN ] [REMOTE ] MC: No data returned for month 2025-07 [2025-10-20T19:48:34.755128] [FATAL] [REMOTE ] MC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:48:34.755204] [FATAL] [MAIN ] MC: Failed to fetch primary data. [2025-10-20T19:48:34.755240] [WARN ] [MAIN ] MC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:48:34.755277] [INFO ] [LOCAL ] MC: Starting deletion of primary data. [2025-10-20T19:48:34.755330] [DEBUG] [LOCAL ] MC: No primary data file found to delete. [2025-10-20T19:48:34.755364] [INFO ] [LOCAL ] MC: Starting deletion of derivative data. [2025-10-20T19:48:34.755528] [DEBUG] [LOCAL ] MC: Deleted derivative data file. [2025-10-20T19:48:34.755607] [INFO ] [MAIN ] Update Progress: 1307/2367 symbols completed. MC took 23.9s. Estimated remaining: 2h 16m 40s. Rate: 7.8 symbols/min [2025-10-20T19:48:34.925968] [INFO ] [MAIN ] MCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:34.934508] [DEBUG] [LOCAL ] MCI: Loaded 6457 existing records from primary storage. [2025-10-20T19:48:34.934713] [INFO ] [MAIN ] MCI: Loaded 6457 existing records. [2025-10-20T19:48:35.108378] [WARN ] [MAIN ] MCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:35.108561] [INFO ] [MAIN ] MCI: Wiping existing primary data before full refresh. [2025-10-20T19:48:35.108614] [INFO ] [LOCAL ] MCI: Starting deletion of primary data. [2025-10-20T19:48:35.108771] [DEBUG] [LOCAL ] MCI: Deleted primary data file. [2025-10-20T19:48:35.108822] [INFO ] [MAIN ] MCI: Starting full primary refresh. [2025-10-20T19:48:35.453783] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-06. [2025-10-20T19:48:35.453706] [INFO ] [REMOTE ] MCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:35.897854] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T19:48:36.167398] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:48:36.251467] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-02. [2025-10-20T19:48:37.051374] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-07. [2025-10-20T19:48:37.851354] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-08. [2025-10-20T19:48:37.923241] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T19:48:38.109785] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T19:48:38.651391] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-01. [2025-10-20T19:48:39.149771] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T19:48:39.386178] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T19:48:39.451337] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-02. [2025-10-20T19:48:40.251392] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-08. [2025-10-20T19:48:40.271774] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:48:40.957984] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T19:48:41.051397] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-03. [2025-10-20T19:48:41.784571] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:48:41.851342] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-09. [2025-10-20T19:48:42.651374] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-03. [2025-10-20T19:48:43.371856] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T19:48:43.451415] [INFO ] [REMOTE ] MBC: Fetching primary data for 2025-10. [2025-10-20T19:48:44.251345] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-09. [2025-10-20T19:48:44.776017] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-10-20T19:48:44.843996] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:48:45.051380] [INFO ] [REMOTE ] MCB: Fetching primary data for 2025-10. [2025-10-20T19:48:45.070904] [INFO ] [REMOTE ] MBC: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:48:45.078056] [INFO ] [REMOTE ] MBC: Full fetch complete (63127 records). [2025-10-20T19:48:45.078160] [DEBUG] [VALIDATOR ] MBC: Validating primary data. [2025-10-20T19:48:45.457330] [DEBUG] [VALIDATOR ] MBC: All expected trading days are present. [2025-10-20T19:48:45.457417] [DEBUG] [VALIDATOR ] MBC: Primary data validation passed. [2025-10-20T19:48:45.458315] [INFO ] [MAIN ] MBC: Fetched primary data is valid. [2025-10-20T19:48:45.458380] [DEBUG] [LOCAL ] MBC: Saving primary data. [2025-10-20T19:48:45.462925] [DEBUG] [LOCAL ] MBC: Trimmed 63127 records to 43967 (keeping 200 days of data) [2025-10-20T19:48:45.713605] [DEBUG] [LOCAL ] MBC: Primary data saved (43967 records). [2025-10-20T19:48:45.713749] [INFO ] [MAIN ] MBC: Full history saved. [2025-10-20T19:48:45.713828] [INFO ] [MAIN ] Update Progress: 1308/2367 symbols completed. MBC took 39.4s. Estimated remaining: 2h 16m 35s. Rate: 7.8 symbols/min [2025-10-20T19:48:45.860966] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-04. [2025-10-20T19:48:45.888405] [INFO ] [MAIN ] MCK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:45.926242] [DEBUG] [LOCAL ] MCK: Loaded 39064 existing records from primary storage. [2025-10-20T19:48:45.926548] [INFO ] [MAIN ] MCK: Loaded 39064 existing records. [2025-10-20T19:48:46.058383] [INFO ] [REMOTE ] MCB: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:48:46.101245] [INFO ] [REMOTE ] MCB: Full fetch complete (16641 records). [2025-10-20T19:48:46.117444] [DEBUG] [VALIDATOR ] MCB: Validating primary data. [2025-10-20T19:48:46.121214] [WARN ] [MAIN ] MCK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:46.121545] [INFO ] [MAIN ] MCK: Wiping existing primary data before full refresh. [2025-10-20T19:48:46.121683] [INFO ] [LOCAL ] MCK: Starting deletion of primary data. [2025-10-20T19:48:46.206561] [DEBUG] [LOCAL ] MCK: Deleted primary data file. [2025-10-20T19:48:46.229930] [INFO ] [MAIN ] MCK: Starting full primary refresh. [2025-10-20T19:48:46.669356] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-04. [2025-10-20T19:48:46.839079] [DEBUG] [VALIDATOR ] MCB: All expected trading days are present. [2025-10-20T19:48:46.869994] [DEBUG] [VALIDATOR ] MCB: Primary data validation passed. [2025-10-20T19:48:46.881216] [INFO ] [MAIN ] MCB: Fetched primary data is valid. [2025-10-20T19:48:46.894703] [DEBUG] [LOCAL ] MCB: Saving primary data. [2025-10-20T19:48:46.894615] [INFO ] [REMOTE ] MCK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:46.895472] [DEBUG] [LOCAL ] MCB: Trimmed 16641 records to 11359 (keeping 200 days of data) [2025-10-20T19:48:46.963736] [DEBUG] [LOCAL ] MCB: Primary data saved (11359 records). [2025-10-20T19:48:46.963882] [INFO ] [MAIN ] MCB: Full history saved. [2025-10-20T19:48:46.963963] [INFO ] [MAIN ] Update Progress: 1309/2367 symbols completed. MCB took 31.2s. Estimated remaining: 2h 16m 22s. Rate: 7.8 symbols/min [2025-10-20T19:48:47.140249] [INFO ] [MAIN ] MCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:48:47.150419] [DEBUG] [LOCAL ] MCN: Loaded 8247 existing records from primary storage. [2025-10-20T19:48:47.150634] [INFO ] [MAIN ] MCN: Loaded 8247 existing records. [2025-10-20T19:48:47.338281] [WARN ] [MAIN ] MCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:48:47.338584] [INFO ] [MAIN ] MCN: Wiping existing primary data before full refresh. [2025-10-20T19:48:47.338637] [INFO ] [LOCAL ] MCN: Starting deletion of primary data. [2025-10-20T19:48:47.338839] [DEBUG] [LOCAL ] MCN: Deleted primary data file. [2025-10-20T19:48:47.338896] [INFO ] [MAIN ] MCN: Starting full primary refresh. [2025-10-20T19:48:47.444115] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T19:48:47.454672] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-01. [2025-10-20T19:48:47.701477] [INFO ] [REMOTE ] MCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:48:48.047996] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T19:48:48.251421] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-01. [2025-10-20T19:48:49.030152] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T19:48:49.051352] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-02. [2025-10-20T19:48:49.127318] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T19:48:49.760406] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:48:49.851429] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-05. [2025-10-20T19:48:50.651426] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-02. [2025-10-20T19:48:51.451351] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-05. [2025-10-20T19:48:51.845844] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T19:48:52.090757] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T19:48:52.163631] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:48:52.251334] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-03. [2025-10-20T19:48:52.973986] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T19:48:53.051396] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-06. [2025-10-20T19:48:53.851363] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-04. [2025-10-20T19:48:54.651416] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-03. [2025-10-20T19:48:54.678632] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T19:48:55.001834] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T19:48:55.451371] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-07. [2025-10-20T19:48:56.096959] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T19:48:56.251346] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-04. [2025-10-20T19:48:57.051409] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-05. [2025-10-20T19:48:57.794285] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T19:48:57.848067] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T19:48:57.854178] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-06. [2025-10-20T19:48:57.865203] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:48:58.651405] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-06. [2025-10-20T19:48:58.669938] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T19:48:59.450564] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T19:48:59.453813] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-07. [2025-10-20T19:49:00.115319] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T19:49:00.251364] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-05. [2025-10-20T19:49:01.051439] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-07. [2025-10-20T19:49:01.851396] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-08. [2025-10-20T19:49:01.953541] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T19:49:02.358913] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T19:49:02.658449] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-09. [2025-10-20T19:49:02.665438] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T19:49:03.374377] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T19:49:03.451412] [INFO ] [REMOTE ] MCI: Fetching primary data for 2025-10. [2025-10-20T19:49:04.183205] [INFO ] [REMOTE ] MCI: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T19:49:04.184661] [INFO ] [REMOTE ] MCI: Full fetch complete (9896 records). [2025-10-20T19:49:04.184752] [DEBUG] [VALIDATOR ] MCI: Validating primary data. [2025-10-20T19:49:04.256613] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-08. [2025-10-20T19:49:04.557794] [DEBUG] [VALIDATOR ] MCI: All expected trading days are present. [2025-10-20T19:49:04.557867] [DEBUG] [VALIDATOR ] MCI: Primary data validation passed. [2025-10-20T19:49:04.558116] [INFO ] [MAIN ] MCI: Fetched primary data is valid. [2025-10-20T19:49:04.558155] [DEBUG] [LOCAL ] MCI: Saving primary data. [2025-10-20T19:49:04.558593] [DEBUG] [LOCAL ] MCI: Trimmed 9896 records to 6235 (keeping 200 days of data) [2025-10-20T19:49:04.594836] [DEBUG] [LOCAL ] MCI: Primary data saved (6235 records). [2025-10-20T19:49:04.594921] [INFO ] [MAIN ] MCI: Full history saved. [2025-10-20T19:49:04.594993] [INFO ] [MAIN ] Update Progress: 1310/2367 symbols completed. MCI took 29.8s. Estimated remaining: 2h 16m 22s. Rate: 7.8 symbols/min [2025-10-20T19:49:04.764012] [INFO ] [MAIN ] MCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:04.801308] [DEBUG] [LOCAL ] MCO: Loaded 40089 existing records from primary storage. [2025-10-20T19:49:04.801608] [INFO ] [MAIN ] MCO: Loaded 40089 existing records. [2025-10-20T19:49:04.977151] [WARN ] [MAIN ] MCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:04.977365] [INFO ] [MAIN ] MCO: Wiping existing primary data before full refresh. [2025-10-20T19:49:04.977415] [INFO ] [LOCAL ] MCO: Starting deletion of primary data. [2025-10-20T19:49:04.977608] [DEBUG] [LOCAL ] MCO: Deleted primary data file. [2025-10-20T19:49:04.977702] [INFO ] [MAIN ] MCO: Starting full primary refresh. [2025-10-20T19:49:05.056561] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-08. [2025-10-20T19:49:05.357536] [INFO ] [REMOTE ] MCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:05.851375] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-06. [2025-10-20T19:49:05.880829] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:49:06.446488] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T19:49:06.651611] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-01. [2025-10-20T19:49:07.304841] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T19:49:07.451405] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-09. [2025-10-20T19:49:08.251374] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-07. [2025-10-20T19:49:08.338765] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T19:49:09.051474] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-02. [2025-10-20T19:49:09.475091] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T19:49:09.488549] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T19:49:09.851416] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-09. [2025-10-20T19:49:10.305097] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T19:49:10.587805] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:49:10.651382] [INFO ] [REMOTE ] MCD: Fetching primary data for 2025-10. [2025-10-20T19:49:11.451339] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-03. [2025-10-20T19:49:11.621740] [INFO ] [REMOTE ] MCD: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:49:11.629447] [INFO ] [REMOTE ] MCD: Full fetch complete (77148 records). [2025-10-20T19:49:11.629565] [DEBUG] [VALIDATOR ] MCD: Validating primary data. [2025-10-20T19:49:11.821187] [WARN ] [VALIDATOR ] MCD: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:49:11.821950] [FATAL] [MAIN ] MCD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:49:11.822031] [WARN ] [MAIN ] MCD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:49:11.822100] [INFO ] [LOCAL ] MCD: Starting deletion of primary data. [2025-10-20T19:49:11.822162] [DEBUG] [LOCAL ] MCD: No primary data file found to delete. [2025-10-20T19:49:11.822196] [INFO ] [LOCAL ] MCD: Starting deletion of derivative data. [2025-10-20T19:49:11.822234] [DEBUG] [LOCAL ] MCD: No derivative data file found to delete. [2025-10-20T19:49:11.822289] [INFO ] [MAIN ] Update Progress: 1311/2367 symbols completed. MCD took 38.7s. Estimated remaining: 2h 16m 14s. Rate: 7.8 symbols/min [2025-10-20T19:49:11.989147] [INFO ] [MAIN ] MCR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:11.999344] [DEBUG] [LOCAL ] MCR: Loaded 9061 existing records from primary storage. [2025-10-20T19:49:11.999544] [INFO ] [MAIN ] MCR: Loaded 9061 existing records. [2025-10-20T19:49:12.177296] [WARN ] [MAIN ] MCR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:12.177480] [INFO ] [MAIN ] MCR: Wiping existing primary data before full refresh. [2025-10-20T19:49:12.177527] [INFO ] [LOCAL ] MCR: Starting deletion of primary data. [2025-10-20T19:49:12.177753] [DEBUG] [LOCAL ] MCR: Deleted primary data file. [2025-10-20T19:49:12.177812] [INFO ] [MAIN ] MCR: Starting full primary refresh. [2025-10-20T19:49:12.256631] [INFO ] [REMOTE ] MCN: Fetching primary data for 2025-10. [2025-10-20T19:49:12.547229] [INFO ] [REMOTE ] MCR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:13.051397] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-08. [2025-10-20T19:49:13.053880] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:49:13.068554] [INFO ] [REMOTE ] MCN: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:49:13.070078] [INFO ] [REMOTE ] MCN: Full fetch complete (11580 records). [2025-10-20T19:49:13.070149] [DEBUG] [VALIDATOR ] MCN: Validating primary data. [2025-10-20T19:49:13.430942] [DEBUG] [VALIDATOR ] MCN: All expected trading days are present. [2025-10-20T19:49:13.431017] [DEBUG] [VALIDATOR ] MCN: Primary data validation passed. [2025-10-20T19:49:13.431273] [INFO ] [MAIN ] MCN: Fetched primary data is valid. [2025-10-20T19:49:13.431310] [DEBUG] [LOCAL ] MCN: Saving primary data. [2025-10-20T19:49:13.431874] [DEBUG] [LOCAL ] MCN: Trimmed 11580 records to 8069 (keeping 200 days of data) [2025-10-20T19:49:13.478188] [DEBUG] [LOCAL ] MCN: Primary data saved (8069 records). [2025-10-20T19:49:13.478320] [INFO ] [MAIN ] MCN: Full history saved. [2025-10-20T19:49:13.478398] [INFO ] [MAIN ] Update Progress: 1312/2367 symbols completed. MCN took 26.5s. Estimated remaining: 2h 16m 1s. Rate: 7.8 symbols/min [2025-10-20T19:49:13.646319] [INFO ] [MAIN ] MCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:13.667398] [DEBUG] [LOCAL ] MCS: Loaded 21467 existing records from primary storage. [2025-10-20T19:49:13.667633] [INFO ] [MAIN ] MCS: Loaded 21467 existing records. [2025-10-20T19:49:13.846162] [WARN ] [MAIN ] MCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:13.846366] [INFO ] [MAIN ] MCS: Wiping existing primary data before full refresh. [2025-10-20T19:49:13.846411] [INFO ] [LOCAL ] MCS: Starting deletion of primary data. [2025-10-20T19:49:13.846607] [DEBUG] [LOCAL ] MCS: Deleted primary data file. [2025-10-20T19:49:13.846693] [INFO ] [MAIN ] MCS: Starting full primary refresh. [2025-10-20T19:49:13.857872] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-01. [2025-10-20T19:49:14.215700] [INFO ] [REMOTE ] MCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:14.651355] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-04. [2025-10-20T19:49:14.742079] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T19:49:14.802793] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T19:49:15.451440] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-02. [2025-10-20T19:49:16.192453] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T19:49:16.251344] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-03. [2025-10-20T19:49:16.428794] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:49:16.709593] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T19:49:17.051410] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-05. [2025-10-20T19:49:17.851367] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-09. [2025-10-20T19:49:18.591094] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T19:49:18.651377] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-01. [2025-10-20T19:49:19.396457] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T19:49:19.451353] [INFO ] [REMOTE ] MCK: Fetching primary data for 2025-10. [2025-10-20T19:49:19.713607] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T19:49:20.251394] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-06. [2025-10-20T19:49:20.496978] [INFO ] [REMOTE ] MCK: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T19:49:20.502189] [INFO ] [REMOTE ] MCK: Full fetch complete (56728 records). [2025-10-20T19:49:20.502335] [DEBUG] [VALIDATOR ] MCK: Validating primary data. [2025-10-20T19:49:20.894421] [DEBUG] [VALIDATOR ] MCK: All expected trading days are present. [2025-10-20T19:49:20.894512] [DEBUG] [VALIDATOR ] MCK: Primary data validation passed. [2025-10-20T19:49:20.895370] [INFO ] [MAIN ] MCK: Fetched primary data is valid. [2025-10-20T19:49:20.895435] [DEBUG] [LOCAL ] MCK: Saving primary data. [2025-10-20T19:49:20.896706] [DEBUG] [LOCAL ] MCK: Trimmed 56728 records to 38322 (keeping 200 days of data) [2025-10-20T19:49:21.071939] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-02. [2025-10-20T19:49:21.140196] [DEBUG] [LOCAL ] MCK: Primary data saved (38322 records). [2025-10-20T19:49:21.140343] [INFO ] [MAIN ] MCK: Full history saved. [2025-10-20T19:49:21.140426] [INFO ] [MAIN ] Update Progress: 1313/2367 symbols completed. MCK took 35.4s. Estimated remaining: 2h 15m 53s. Rate: 7.8 symbols/min [2025-10-20T19:49:21.307105] [INFO ] [MAIN ] MCY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:21.329549] [DEBUG] [LOCAL ] MCY: Loaded 24248 existing records from primary storage. [2025-10-20T19:49:21.329821] [INFO ] [MAIN ] MCY: Loaded 24248 existing records. [2025-10-20T19:49:21.511846] [WARN ] [MAIN ] MCY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:21.519349] [INFO ] [MAIN ] MCY: Wiping existing primary data before full refresh. [2025-10-20T19:49:21.519501] [INFO ] [LOCAL ] MCY: Starting deletion of primary data. [2025-10-20T19:49:21.532462] [DEBUG] [LOCAL ] MCY: Deleted primary data file. [2025-10-20T19:49:21.532926] [INFO ] [MAIN ] MCY: Starting full primary refresh. [2025-10-20T19:49:21.533538] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T19:49:21.854318] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-07. [2025-10-20T19:49:21.894884] [INFO ] [REMOTE ] MCY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:22.075540] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T19:49:22.651369] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-01. [2025-10-20T19:49:23.451358] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-03. [2025-10-20T19:49:23.478091] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T19:49:24.255323] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-04. [2025-10-20T19:49:24.278148] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:49:24.573850] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T19:49:24.777312] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T19:49:25.051371] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-02. [2025-10-20T19:49:25.851385] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-08. [2025-10-20T19:49:26.353041] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:49:26.651360] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-04. [2025-10-20T19:49:27.264379] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T19:49:27.451427] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-05. [2025-10-20T19:49:27.759099] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:49:28.155396] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T19:49:28.251369] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-06. [2025-10-20T19:49:28.938414] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:49:29.051373] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-07. [2025-10-20T19:49:29.768759] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-07 (0.72 seconds). [2025-10-20T19:49:29.851399] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-03. [2025-10-20T19:49:30.651377] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-05. [2025-10-20T19:49:31.155510] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T19:49:31.451377] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-04. [2025-10-20T19:49:31.731048] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:49:32.251394] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-09. [2025-10-20T19:49:32.601201] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T19:49:33.051368] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-06. [2025-10-20T19:49:33.470584] [INFO ] [REMOTE ] MCO: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T19:49:33.851379] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-08. [2025-10-20T19:49:34.004587] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T19:49:34.651393] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-07. [2025-10-20T19:49:34.668995] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T19:49:35.451389] [INFO ] [REMOTE ] MCO: Fetching primary data for 2025-10. [2025-10-20T19:49:35.555751] [DEBUG] [REMOTE ] MCO: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T19:49:35.556041] [WARN ] [REMOTE ] MCO: No data returned for month 2025-10 [2025-10-20T19:49:35.556162] [FATAL] [REMOTE ] MCO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:49:35.556257] [FATAL] [MAIN ] MCO: Failed to fetch primary data. [2025-10-20T19:49:35.556334] [WARN ] [MAIN ] MCO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:49:35.556420] [INFO ] [LOCAL ] MCO: Starting deletion of primary data. [2025-10-20T19:49:35.556533] [DEBUG] [LOCAL ] MCO: No primary data file found to delete. [2025-10-20T19:49:35.556590] [INFO ] [LOCAL ] MCO: Starting deletion of derivative data. [2025-10-20T19:49:35.556842] [DEBUG] [LOCAL ] MCO: Deleted derivative data file. [2025-10-20T19:49:35.556941] [INFO ] [MAIN ] Update Progress: 1314/2367 symbols completed. MCO took 31.0s. Estimated remaining: 2h 15m 51s. Rate: 7.8 symbols/min [2025-10-20T19:49:35.748227] [INFO ] [MAIN ] MD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:35.782384] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T19:49:35.801229] [DEBUG] [LOCAL ] MD: Loaded 39102 existing records from primary storage. [2025-10-20T19:49:35.801527] [INFO ] [MAIN ] MD: Loaded 39102 existing records. [2025-10-20T19:49:35.974971] [WARN ] [MAIN ] MD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:35.975161] [INFO ] [MAIN ] MD: Wiping existing primary data before full refresh. [2025-10-20T19:49:35.975208] [INFO ] [LOCAL ] MD: Starting deletion of primary data. [2025-10-20T19:49:35.975430] [DEBUG] [LOCAL ] MD: Deleted primary data file. [2025-10-20T19:49:35.975485] [INFO ] [MAIN ] MD: Starting full primary refresh. [2025-10-20T19:49:36.256618] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-08. [2025-10-20T19:49:36.321665] [INFO ] [REMOTE ] MD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:37.051360] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-09. [2025-10-20T19:49:37.306207] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T19:49:37.637914] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:49:37.851385] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-01. [2025-10-20T19:49:38.651371] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-05. [2025-10-20T19:49:39.186858] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T19:49:39.451377] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-02. [2025-10-20T19:49:39.892499] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:49:40.251432] [INFO ] [REMOTE ] MCR: Fetching primary data for 2025-10. [2025-10-20T19:49:40.758970] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T19:49:40.943084] [INFO ] [REMOTE ] MCR: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T19:49:40.944935] [INFO ] [REMOTE ] MCR: Full fetch complete (13535 records). [2025-10-20T19:49:40.945005] [DEBUG] [VALIDATOR ] MCR: Validating primary data. [2025-10-20T19:49:41.056472] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-06. [2025-10-20T19:49:41.304068] [DEBUG] [VALIDATOR ] MCR: All expected trading days are present. [2025-10-20T19:49:41.304143] [DEBUG] [VALIDATOR ] MCR: Primary data validation passed. [2025-10-20T19:49:41.304462] [INFO ] [MAIN ] MCR: Fetched primary data is valid. [2025-10-20T19:49:41.304501] [DEBUG] [LOCAL ] MCR: Saving primary data. [2025-10-20T19:49:41.305018] [DEBUG] [LOCAL ] MCR: Trimmed 13535 records to 8879 (keeping 200 days of data) [2025-10-20T19:49:41.354993] [DEBUG] [LOCAL ] MCR: Primary data saved (8879 records). [2025-10-20T19:49:41.355120] [INFO ] [MAIN ] MCR: Full history saved. [2025-10-20T19:49:41.355192] [INFO ] [MAIN ] Update Progress: 1315/2367 symbols completed. MCR took 29.5s. Estimated remaining: 2h 15m 42s. Rate: 7.8 symbols/min [2025-10-20T19:49:41.520119] [INFO ] [MAIN ] MDST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:41.529021] [DEBUG] [LOCAL ] MDST: Loaded 6124 existing records from primary storage. [2025-10-20T19:49:41.529224] [INFO ] [MAIN ] MDST: Loaded 6124 existing records. [2025-10-20T19:49:41.703372] [WARN ] [MAIN ] MDST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:41.703595] [INFO ] [MAIN ] MDST: Wiping existing primary data before full refresh. [2025-10-20T19:49:41.703678] [INFO ] [LOCAL ] MDST: Starting deletion of primary data. [2025-10-20T19:49:41.703884] [DEBUG] [LOCAL ] MDST: Deleted primary data file. [2025-10-20T19:49:41.703939] [INFO ] [MAIN ] MDST: Starting full primary refresh. [2025-10-20T19:49:41.856457] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-03. [2025-10-20T19:49:42.072753] [INFO ] [REMOTE ] MDST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:42.115890] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T19:49:42.651361] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-01. [2025-10-20T19:49:43.048391] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T19:49:43.301049] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:49:43.451514] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-02. [2025-10-20T19:49:44.066239] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T19:49:44.251423] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-07. [2025-10-20T19:49:45.051406] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-04. [2025-10-20T19:49:45.389236] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:49:45.851393] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-03. [2025-10-20T19:49:46.472186] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T19:49:46.521715] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T19:49:46.651376] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-04. [2025-10-20T19:49:47.218829] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:49:47.451405] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-05. [2025-10-20T19:49:47.983136] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T19:49:48.251388] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-05. [2025-10-20T19:49:49.051417] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-09. [2025-10-20T19:49:49.766610] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T19:49:49.851383] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-06. [2025-10-20T19:49:50.102117] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T19:49:50.651352] [INFO ] [REMOTE ] MCS: Fetching primary data for 2025-10. [2025-10-20T19:49:51.287813] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T19:49:51.451384] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-07. [2025-10-20T19:49:51.865240] [INFO ] [REMOTE ] MCS: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T19:49:51.869077] [INFO ] [REMOTE ] MCS: Full fetch complete (30751 records). [2025-10-20T19:49:51.869155] [DEBUG] [VALIDATOR ] MCS: Validating primary data. [2025-10-20T19:49:52.231080] [DEBUG] [VALIDATOR ] MCS: All expected trading days are present. [2025-10-20T19:49:52.231157] [DEBUG] [VALIDATOR ] MCS: Primary data validation passed. [2025-10-20T19:49:52.231786] [INFO ] [MAIN ] MCS: Fetched primary data is valid. [2025-10-20T19:49:52.231847] [DEBUG] [LOCAL ] MCS: Saving primary data. [2025-10-20T19:49:52.232518] [DEBUG] [LOCAL ] MCS: Trimmed 30751 records to 21228 (keeping 200 days of data) [2025-10-20T19:49:52.282238] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-06. [2025-10-20T19:49:52.361020] [DEBUG] [LOCAL ] MCS: Primary data saved (21228 records). [2025-10-20T19:49:52.361136] [INFO ] [MAIN ] MCS: Full history saved. [2025-10-20T19:49:52.361209] [INFO ] [MAIN ] Update Progress: 1316/2367 symbols completed. MCS took 38.9s. Estimated remaining: 2h 15m 36s. Rate: 7.8 symbols/min [2025-10-20T19:49:52.532941] [INFO ] [MAIN ] MDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:49:52.556095] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:49:52.610848] [DEBUG] [LOCAL ] MDT: Loaded 54458 existing records from primary storage. [2025-10-20T19:49:52.611162] [INFO ] [MAIN ] MDT: Loaded 54458 existing records. [2025-10-20T19:49:52.794067] [WARN ] [MAIN ] MDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:49:52.794386] [INFO ] [MAIN ] MDT: Wiping existing primary data before full refresh. [2025-10-20T19:49:52.794444] [INFO ] [LOCAL ] MDT: Starting deletion of primary data. [2025-10-20T19:49:52.794731] [DEBUG] [LOCAL ] MDT: Deleted primary data file. [2025-10-20T19:49:52.794788] [INFO ] [MAIN ] MDT: Starting full primary refresh. [2025-10-20T19:49:52.975984] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T19:49:53.056569] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-08. [2025-10-20T19:49:53.167710] [INFO ] [REMOTE ] MDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:49:53.851353] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-07. [2025-10-20T19:49:54.159225] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T19:49:54.651407] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-08. [2025-10-20T19:49:54.716687] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T19:49:55.451386] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-08. [2025-10-20T19:49:56.088824] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T19:49:56.089253] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T19:49:56.251406] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-09. [2025-10-20T19:49:56.965306] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:49:57.051467] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-09. [2025-10-20T19:49:57.851350] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-01. [2025-10-20T19:49:58.038942] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T19:49:58.651377] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-09. [2025-10-20T19:49:59.451381] [INFO ] [REMOTE ] MCY: Fetching primary data for 2025-10. [2025-10-20T19:49:59.814896] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T19:50:00.146019] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T19:50:00.251492] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-02. [2025-10-20T19:50:00.668693] [INFO ] [REMOTE ] MCY: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T19:50:00.674117] [INFO ] [REMOTE ] MCY: Full fetch complete (40951 records). [2025-10-20T19:50:00.674292] [DEBUG] [VALIDATOR ] MCY: Validating primary data. [2025-10-20T19:50:01.056489] [INFO ] [REMOTE ] MDST: Fetching primary data for 2025-10. [2025-10-20T19:50:01.122518] [DEBUG] [VALIDATOR ] MCY: All expected trading days are present. [2025-10-20T19:50:01.122616] [DEBUG] [VALIDATOR ] MCY: Primary data validation passed. [2025-10-20T19:50:01.123393] [INFO ] [MAIN ] MCY: Fetched primary data is valid. [2025-10-20T19:50:01.123455] [DEBUG] [LOCAL ] MCY: Saving primary data. [2025-10-20T19:50:01.124427] [DEBUG] [LOCAL ] MCY: Trimmed 40951 records to 23809 (keeping 200 days of data) [2025-10-20T19:50:01.264054] [DEBUG] [LOCAL ] MCY: Primary data saved (23809 records). [2025-10-20T19:50:01.264177] [INFO ] [MAIN ] MCY: Full history saved. [2025-10-20T19:50:01.264251] [INFO ] [MAIN ] Update Progress: 1317/2367 symbols completed. MCY took 40.1s. Estimated remaining: 2h 15m 30s. Rate: 7.7 symbols/min [2025-10-20T19:50:01.851359] [INFO ] [REMOTE ] MD: Fetching primary data for 2025-10. [2025-10-20T19:50:02.019781] [INFO ] [REMOTE ] MDST: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T19:50:02.021532] [INFO ] [REMOTE ] MDST: Full fetch complete (8828 records). [2025-10-20T19:50:02.021634] [DEBUG] [VALIDATOR ] MDST: Validating primary data. [2025-10-20T19:50:02.397322] [DEBUG] [VALIDATOR ] MDST: All expected trading days are present. [2025-10-20T19:50:02.397516] [DEBUG] [VALIDATOR ] MDST: Primary data validation passed. [2025-10-20T19:50:02.397776] [INFO ] [MAIN ] MDST: Fetched primary data is valid. [2025-10-20T19:50:02.397851] [DEBUG] [LOCAL ] MDST: Saving primary data. [2025-10-20T19:50:02.398384] [DEBUG] [LOCAL ] MDST: Trimmed 8828 records to 6125 (keeping 200 days of data) [2025-10-20T19:50:02.439441] [DEBUG] [LOCAL ] MDST: Primary data saved (6125 records). [2025-10-20T19:50:02.439563] [INFO ] [MAIN ] MDST: Full history saved. [2025-10-20T19:50:02.439638] [INFO ] [MAIN ] Update Progress: 1318/2367 symbols completed. MDST took 21.1s. Estimated remaining: 2h 15m 17s. Rate: 7.8 symbols/min [2025-10-20T19:50:02.557834] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-02 (2.31 seconds). [2025-10-20T19:50:02.651369] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-03. [2025-10-20T19:50:03.317290] [INFO ] [REMOTE ] MD: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:50:03.323262] [INFO ] [REMOTE ] MD: Full fetch complete (55218 records). [2025-10-20T19:50:03.323362] [DEBUG] [VALIDATOR ] MD: Validating primary data. [2025-10-20T19:50:03.730792] [DEBUG] [VALIDATOR ] MD: All expected trading days are present. [2025-10-20T19:50:03.730888] [DEBUG] [VALIDATOR ] MD: Primary data validation passed. [2025-10-20T19:50:03.731862] [INFO ] [MAIN ] MD: Fetched primary data is valid. [2025-10-20T19:50:03.731929] [DEBUG] [LOCAL ] MD: Saving primary data. [2025-10-20T19:50:03.733221] [DEBUG] [LOCAL ] MD: Trimmed 55218 records to 38589 (keeping 200 days of data) [2025-10-20T19:50:03.959963] [DEBUG] [LOCAL ] MD: Primary data saved (38589 records). [2025-10-20T19:50:03.960113] [INFO ] [MAIN ] MD: Full history saved. [2025-10-20T19:50:03.960194] [INFO ] [MAIN ] Update Progress: 1319/2367 symbols completed. MD took 28.4s. Estimated remaining: 2h 15m 4s. Rate: 7.8 symbols/min [2025-10-20T19:50:04.735224] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T19:50:04.747021] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-04. [2025-10-20T19:50:06.657926] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T19:50:06.669108] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-05. [2025-10-20T19:50:08.553364] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T19:50:08.564563] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-06. [2025-10-20T19:50:10.813582] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-06 (2.25 seconds). [2025-10-20T19:50:10.824463] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-07. [2025-10-20T19:50:12.931222] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T19:50:12.942926] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-08. [2025-10-20T19:50:14.988769] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T19:50:15.000876] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-09. [2025-10-20T19:50:16.968022] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:50:16.979416] [INFO ] [REMOTE ] MDT: Fetching primary data for 2025-10. [2025-10-20T19:50:17.824985] [INFO ] [REMOTE ] MDT: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T19:50:17.832777] [INFO ] [REMOTE ] MDT: Full fetch complete (77857 records). [2025-10-20T19:50:17.832896] [DEBUG] [VALIDATOR ] MDT: Validating primary data. [2025-10-20T19:50:18.230938] [DEBUG] [VALIDATOR ] MDT: All expected trading days are present. [2025-10-20T19:50:18.231019] [DEBUG] [VALIDATOR ] MDT: Primary data validation passed. [2025-10-20T19:50:18.232000] [INFO ] [MAIN ] MDT: Fetched primary data is valid. [2025-10-20T19:50:18.232060] [DEBUG] [LOCAL ] MDT: Saving primary data. [2025-10-20T19:50:18.233414] [DEBUG] [LOCAL ] MDT: Trimmed 77857 records to 53678 (keeping 200 days of data) [2025-10-20T19:50:18.556695] [DEBUG] [LOCAL ] MDT: Primary data saved (53678 records). [2025-10-20T19:50:18.556844] [INFO ] [MAIN ] MDT: Full history saved. [2025-10-20T19:50:18.557139] [INFO ] [MAIN ] Update Progress: 1320/2367 symbols completed. MDT took 26.2s. Estimated remaining: 2h 15m 2s. Rate: 7.8 symbols/min [2025-10-20T19:50:19.129119] [INFO ] [MAIN ] MED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.258027] [INFO ] [MAIN ] MEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.303252] [INFO ] [MAIN ] MDV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.311295] [INFO ] [MAIN ] MDU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:19.328484] [DEBUG] [LOCAL ] MDV: Loaded 8524 existing records from primary storage. [2025-10-20T19:50:19.328926] [INFO ] [MAIN ] MDV: Loaded 8524 existing records. [2025-10-20T19:50:19.381275] [DEBUG] [LOCAL ] MEC: Loaded 16672 existing records from primary storage. [2025-10-20T19:50:19.449714] [INFO ] [MAIN ] MEC: Loaded 16672 existing records. [2025-10-20T19:50:19.444098] [DEBUG] [LOCAL ] MED: Loaded 21503 existing records from primary storage. [2025-10-20T19:50:19.503898] [INFO ] [MAIN ] MED: Loaded 21503 existing records. [2025-10-20T19:50:19.762105] [WARN ] [MAIN ] MDV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:19.849011] [INFO ] [MAIN ] MDV: Wiping existing primary data before full refresh. [2025-10-20T19:50:19.850748] [DEBUG] [LOCAL ] MDU: Loaded 48794 existing records from primary storage. [2025-10-20T19:50:19.884166] [INFO ] [MAIN ] MDU: Loaded 48794 existing records. [2025-10-20T19:50:19.884027] [INFO ] [LOCAL ] MDV: Starting deletion of primary data. [2025-10-20T19:50:19.900721] [WARN ] [MAIN ] MEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:19.916478] [INFO ] [MAIN ] MEC: Wiping existing primary data before full refresh. [2025-10-20T19:50:19.988548] [INFO ] [LOCAL ] MEC: Starting deletion of primary data. [2025-10-20T19:50:20.060987] [WARN ] [MAIN ] MED: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:20.053230] [DEBUG] [LOCAL ] MEC: Deleted primary data file. [2025-10-20T19:50:20.011941] [DEBUG] [LOCAL ] MDV: Deleted primary data file. [2025-10-20T19:50:20.076827] [INFO ] [MAIN ] MDV: Starting full primary refresh. [2025-10-20T19:50:20.066451] [INFO ] [MAIN ] MEC: Starting full primary refresh. [2025-10-20T19:50:20.087293] [INFO ] [MAIN ] MED: Wiping existing primary data before full refresh. [2025-10-20T19:50:20.127538] [INFO ] [LOCAL ] MED: Starting deletion of primary data. [2025-10-20T19:50:20.166561] [DEBUG] [LOCAL ] MED: Deleted primary data file. [2025-10-20T19:50:20.185512] [INFO ] [MAIN ] MED: Starting full primary refresh. [2025-10-20T19:50:20.472256] [WARN ] [MAIN ] MDU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:20.586909] [INFO ] [MAIN ] MDU: Wiping existing primary data before full refresh. [2025-10-20T19:50:20.688929] [INFO ] [LOCAL ] MDU: Starting deletion of primary data. [2025-10-20T19:50:20.770226] [DEBUG] [LOCAL ] MDU: Deleted primary data file. [2025-10-20T19:50:20.841607] [INFO ] [MAIN ] MDU: Starting full primary refresh. [2025-10-20T19:50:21.233857] [INFO ] [REMOTE ] MEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.255356] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-01. [2025-10-20T19:50:21.397583] [INFO ] [REMOTE ] MDV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.455109] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-01. [2025-10-20T19:50:21.454748] [INFO ] [REMOTE ] MED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.465242] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-01. [2025-10-20T19:50:21.665962] [INFO ] [REMOTE ] MDU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:21.666300] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-01. [2025-10-20T19:50:22.130665] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T19:50:22.132839] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-02. [2025-10-20T19:50:22.283202] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T19:50:22.286860] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-02. [2025-10-20T19:50:22.669134] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T19:50:22.674222] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-02. [2025-10-20T19:50:22.839465] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:50:22.841829] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-03. [2025-10-20T19:50:22.944612] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T19:50:22.954361] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-02. [2025-10-20T19:50:23.054364] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T19:50:23.056769] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-03. [2025-10-20T19:50:23.365464] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T19:50:23.368091] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-04. [2025-10-20T19:50:23.807321] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:50:23.812511] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-03. [2025-10-20T19:50:23.978127] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T19:50:23.983394] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-04. [2025-10-20T19:50:24.099284] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:50:24.108928] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-03. [2025-10-20T19:50:24.288032] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T19:50:24.290627] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-05. [2025-10-20T19:50:24.948804] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T19:50:24.954121] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-04. [2025-10-20T19:50:25.035531] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T19:50:25.041080] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-05. [2025-10-20T19:50:25.081887] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:50:25.084583] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-06. [2025-10-20T19:50:25.769113] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T19:50:25.774801] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-06. [2025-10-20T19:50:25.862537] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T19:50:25.868866] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T19:50:25.871776] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-05. [2025-10-20T19:50:25.873133] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-07. [2025-10-20T19:50:25.985468] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T19:50:25.995825] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-04. [2025-10-20T19:50:26.360205] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T19:50:26.427361] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-10-20T19:50:26.659178] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-07. [2025-10-20T19:50:26.660309] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T19:50:27.302364] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T19:50:27.451363] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-05. [2025-10-20T19:50:27.564416] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:50:28.251371] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-06. [2025-10-20T19:50:29.051366] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-08. [2025-10-20T19:50:29.155711] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:50:29.331528] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T19:50:29.851498] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-08. [2025-10-20T19:50:30.041531] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:50:30.586277] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T19:50:30.651432] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-07. [2025-10-20T19:50:31.451411] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-09. [2025-10-20T19:50:31.793590] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:50:32.020512] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:50:32.251407] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-06. [2025-10-20T19:50:33.051406] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-08. [2025-10-20T19:50:33.829282] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T19:50:33.851329] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-07. [2025-10-20T19:50:34.126625] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:50:34.651369] [INFO ] [REMOTE ] MEC: Fetching primary data for 2025-10. [2025-10-20T19:50:35.430964] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T19:50:35.451399] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-08. [2025-10-20T19:50:35.711835] [INFO ] [REMOTE ] MEC: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T19:50:35.714525] [INFO ] [REMOTE ] MEC: Full fetch complete (22759 records). [2025-10-20T19:50:35.714594] [DEBUG] [VALIDATOR ] MEC: Validating primary data. [2025-10-20T19:50:36.082037] [DEBUG] [VALIDATOR ] MEC: All expected trading days are present. [2025-10-20T19:50:36.082128] [DEBUG] [VALIDATOR ] MEC: Primary data validation passed. [2025-10-20T19:50:36.082576] [INFO ] [MAIN ] MEC: Fetched primary data is valid. [2025-10-20T19:50:36.082680] [DEBUG] [LOCAL ] MEC: Saving primary data. [2025-10-20T19:50:36.083300] [DEBUG] [LOCAL ] MEC: Trimmed 22759 records to 16516 (keeping 200 days of data) [2025-10-20T19:50:36.178802] [DEBUG] [LOCAL ] MEC: Primary data saved (16516 records). [2025-10-20T19:50:36.178953] [INFO ] [MAIN ] MEC: Full history saved. [2025-10-20T19:50:36.179038] [INFO ] [MAIN ] Update Progress: 1321/2367 symbols completed. MEC took 17.6s. Estimated remaining: 2h 15m 2s. Rate: 7.7 symbols/min [2025-10-20T19:50:36.256610] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-09. [2025-10-20T19:50:36.353613] [INFO ] [MAIN ] MEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:36.384736] [DEBUG] [LOCAL ] MEG: Loaded 32094 existing records from primary storage. [2025-10-20T19:50:36.385000] [INFO ] [MAIN ] MEG: Loaded 32094 existing records. [2025-10-20T19:50:36.562416] [WARN ] [MAIN ] MEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:36.562667] [INFO ] [MAIN ] MEG: Wiping existing primary data before full refresh. [2025-10-20T19:50:36.562729] [INFO ] [LOCAL ] MEG: Starting deletion of primary data. [2025-10-20T19:50:36.562912] [DEBUG] [LOCAL ] MEG: Deleted primary data file. [2025-10-20T19:50:36.562951] [INFO ] [MAIN ] MEG: Starting full primary refresh. [2025-10-20T19:50:36.927731] [INFO ] [REMOTE ] MEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:36.970990] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T19:50:37.014330] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T19:50:37.051339] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-09. [2025-10-20T19:50:37.851415] [INFO ] [REMOTE ] MDV: Fetching primary data for 2025-10. [2025-10-20T19:50:38.651373] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-09. [2025-10-20T19:50:38.679535] [INFO ] [REMOTE ] MDV: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:50:38.681185] [INFO ] [REMOTE ] MDV: Full fetch complete (11685 records). [2025-10-20T19:50:38.681256] [DEBUG] [VALIDATOR ] MDV: Validating primary data. [2025-10-20T19:50:39.028762] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T19:50:39.118309] [DEBUG] [VALIDATOR ] MDV: All expected trading days are present. [2025-10-20T19:50:39.118383] [DEBUG] [VALIDATOR ] MDV: Primary data validation passed. [2025-10-20T19:50:39.118674] [INFO ] [MAIN ] MDV: Fetched primary data is valid. [2025-10-20T19:50:39.118732] [DEBUG] [LOCAL ] MDV: Saving primary data. [2025-10-20T19:50:39.119154] [DEBUG] [LOCAL ] MDV: Trimmed 11685 records to 8358 (keeping 200 days of data) [2025-10-20T19:50:39.168879] [DEBUG] [LOCAL ] MDV: Primary data saved (8358 records). [2025-10-20T19:50:39.169009] [INFO ] [MAIN ] MDV: Full history saved. [2025-10-20T19:50:39.169085] [INFO ] [MAIN ] Update Progress: 1322/2367 symbols completed. MDV took 20.6s. Estimated remaining: 2h 14m 50s. Rate: 7.8 symbols/min [2025-10-20T19:50:39.340779] [INFO ] [MAIN ] MEGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:39.361596] [DEBUG] [LOCAL ] MEGI: Loaded 20099 existing records from primary storage. [2025-10-20T19:50:39.361870] [INFO ] [MAIN ] MEGI: Loaded 20099 existing records. [2025-10-20T19:50:39.460512] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-01. [2025-10-20T19:50:39.545008] [WARN ] [MAIN ] MEGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:39.545283] [INFO ] [MAIN ] MEGI: Wiping existing primary data before full refresh. [2025-10-20T19:50:39.545348] [INFO ] [LOCAL ] MEGI: Starting deletion of primary data. [2025-10-20T19:50:39.545535] [DEBUG] [LOCAL ] MEGI: Deleted primary data file. [2025-10-20T19:50:39.545592] [INFO ] [MAIN ] MEGI: Starting full primary refresh. [2025-10-20T19:50:39.835871] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T19:50:39.919217] [INFO ] [REMOTE ] MEGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:40.251412] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-01. [2025-10-20T19:50:40.951431] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T19:50:41.051342] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-02. [2025-10-20T19:50:41.347208] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T19:50:41.851398] [INFO ] [REMOTE ] MDU: Fetching primary data for 2025-10. [2025-10-20T19:50:42.308519] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T19:50:42.651377] [INFO ] [REMOTE ] MED: Fetching primary data for 2025-10. [2025-10-20T19:50:43.451414] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-02. [2025-10-20T19:50:43.529343] [INFO ] [REMOTE ] MDU: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T19:50:43.544102] [INFO ] [REMOTE ] MED: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T19:50:43.547219] [INFO ] [REMOTE ] MDU: Full fetch complete (70681 records). [2025-10-20T19:50:43.547401] [DEBUG] [VALIDATOR ] MDU: Validating primary data. [2025-10-20T19:50:43.547575] [INFO ] [REMOTE ] MED: Full fetch complete (31931 records). [2025-10-20T19:50:43.547679] [DEBUG] [VALIDATOR ] MED: Validating primary data. [2025-10-20T19:50:44.253948] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-03. [2025-10-20T19:50:44.369134] [DEBUG] [VALIDATOR ] MDU: All expected trading days are present. [2025-10-20T19:50:44.369718] [DEBUG] [VALIDATOR ] MDU: Primary data validation passed. [2025-10-20T19:50:44.370600] [INFO ] [MAIN ] MDU: Fetched primary data is valid. [2025-10-20T19:50:44.370895] [DEBUG] [VALIDATOR ] MED: All expected trading days are present. [2025-10-20T19:50:44.371066] [DEBUG] [LOCAL ] MDU: Saving primary data. [2025-10-20T19:50:44.371166] [DEBUG] [VALIDATOR ] MED: Primary data validation passed. [2025-10-20T19:50:44.372155] [INFO ] [MAIN ] MED: Fetched primary data is valid. [2025-10-20T19:50:44.372495] [DEBUG] [LOCAL ] MED: Saving primary data. [2025-10-20T19:50:44.374147] [DEBUG] [LOCAL ] MED: Trimmed 31931 records to 21123 (keeping 200 days of data) [2025-10-20T19:50:44.374037] [DEBUG] [LOCAL ] MDU: Trimmed 70681 records to 48046 (keeping 200 days of data) [2025-10-20T19:50:44.599073] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T19:50:44.739741] [DEBUG] [LOCAL ] MED: Primary data saved (21123 records). [2025-10-20T19:50:44.741895] [INFO ] [MAIN ] MED: Full history saved. [2025-10-20T19:50:44.742761] [INFO ] [MAIN ] Update Progress: 1323/2367 symbols completed. MED took 26.2s. Estimated remaining: 2h 14m 41s. Rate: 7.8 symbols/min [2025-10-20T19:50:44.985619] [DEBUG] [LOCAL ] MDU: Primary data saved (48046 records). [2025-10-20T19:50:44.985811] [INFO ] [MAIN ] MDU: Full history saved. [2025-10-20T19:50:44.989330] [INFO ] [MAIN ] Update Progress: 1324/2367 symbols completed. MDU took 26.4s. Estimated remaining: 2h 14m 27s. Rate: 7.8 symbols/min [2025-10-20T19:50:44.989118] [INFO ] [MAIN ] MEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:45.054951] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-03. [2025-10-20T19:50:45.165480] [INFO ] [MAIN ] MET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:50:45.187607] [DEBUG] [LOCAL ] MEI: Loaded 32083 existing records from primary storage. [2025-10-20T19:50:45.188012] [INFO ] [MAIN ] MEI: Loaded 32083 existing records. [2025-10-20T19:50:45.385234] [WARN ] [MAIN ] MEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:45.385467] [INFO ] [MAIN ] MEI: Wiping existing primary data before full refresh. [2025-10-20T19:50:45.385539] [INFO ] [LOCAL ] MEI: Starting deletion of primary data. [2025-10-20T19:50:45.385767] [DEBUG] [LOCAL ] MEI: Deleted primary data file. [2025-10-20T19:50:45.385825] [INFO ] [MAIN ] MEI: Starting full primary refresh. [2025-10-20T19:50:45.515896] [DEBUG] [LOCAL ] MET: Loaded 54096 existing records from primary storage. [2025-10-20T19:50:45.537203] [INFO ] [MAIN ] MET: Loaded 54096 existing records. [2025-10-20T19:50:45.964475] [INFO ] [REMOTE ] MEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:45.997850] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-01. [2025-10-20T19:50:46.009535] [WARN ] [MAIN ] MET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:50:46.012974] [INFO ] [MAIN ] MET: Wiping existing primary data before full refresh. [2025-10-20T19:50:46.013417] [INFO ] [LOCAL ] MET: Starting deletion of primary data. [2025-10-20T19:50:46.015442] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T19:50:46.020870] [DEBUG] [LOCAL ] MET: Deleted primary data file. [2025-10-20T19:50:46.041059] [INFO ] [MAIN ] MET: Starting full primary refresh. [2025-10-20T19:50:46.042403] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:50:46.410299] [INFO ] [REMOTE ] MET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:50:46.651373] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-01. [2025-10-20T19:50:47.290712] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T19:50:47.451371] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-04. [2025-10-20T19:50:48.251369] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-04. [2025-10-20T19:50:48.375981] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T19:50:49.015191] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T19:50:49.051431] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-05. [2025-10-20T19:50:49.265748] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T19:50:49.851425] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-05. [2025-10-20T19:50:49.979797] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T19:50:50.651449] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-02. [2025-10-20T19:50:50.876559] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T19:50:51.451380] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-06. [2025-10-20T19:50:51.717468] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:50:52.251374] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-06. [2025-10-20T19:50:52.467778] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T19:50:53.051355] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-07. [2025-10-20T19:50:53.569478] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:50:53.851461] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-02. [2025-10-20T19:50:54.237760] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T19:50:54.651431] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-07. [2025-10-20T19:50:55.456528] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-03. [2025-10-20T19:50:55.486320] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:50:55.754799] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T19:50:56.251452] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-08. [2025-10-20T19:50:56.695434] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T19:50:57.051351] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-04. [2025-10-20T19:50:57.276950] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T19:50:57.851394] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-08. [2025-10-20T19:50:58.462713] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:50:58.651402] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-09. [2025-10-20T19:50:58.988816] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T19:50:59.451422] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-09. [2025-10-20T19:50:59.975822] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:51:00.251382] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-03. [2025-10-20T19:51:00.427715] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T19:51:01.051419] [INFO ] [REMOTE ] MEGI: Fetching primary data for 2025-10. [2025-10-20T19:51:01.851433] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-05. [2025-10-20T19:51:02.227607] [INFO ] [REMOTE ] MEGI: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T19:51:02.230539] [INFO ] [REMOTE ] MEGI: Full fetch complete (29178 records). [2025-10-20T19:51:02.230690] [DEBUG] [VALIDATOR ] MEGI: Validating primary data. [2025-10-20T19:51:02.612942] [DEBUG] [VALIDATOR ] MEGI: All expected trading days are present. [2025-10-20T19:51:02.613115] [DEBUG] [VALIDATOR ] MEGI: Primary data validation passed. [2025-10-20T19:51:02.613821] [INFO ] [MAIN ] MEGI: Fetched primary data is valid. [2025-10-20T19:51:02.613884] [DEBUG] [LOCAL ] MEGI: Saving primary data. [2025-10-20T19:51:02.614758] [DEBUG] [LOCAL ] MEGI: Trimmed 29178 records to 19796 (keeping 200 days of data) [2025-10-20T19:51:02.664551] [INFO ] [REMOTE ] MEG: Fetching primary data for 2025-10. [2025-10-20T19:51:02.738423] [DEBUG] [LOCAL ] MEGI: Primary data saved (19796 records). [2025-10-20T19:51:02.738588] [INFO ] [MAIN ] MEGI: Full history saved. [2025-10-20T19:51:02.738691] [INFO ] [MAIN ] Update Progress: 1325/2367 symbols completed. MEGI took 23.6s. Estimated remaining: 2h 14m 27s. Rate: 7.7 symbols/min [2025-10-20T19:51:02.916587] [INFO ] [MAIN ] MFA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:02.952270] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-03 (2.70 seconds). [2025-10-20T19:51:03.001691] [DEBUG] [LOCAL ] MFA: Loaded 46378 existing records from primary storage. [2025-10-20T19:51:03.002062] [INFO ] [MAIN ] MFA: Loaded 46378 existing records. [2025-10-20T19:51:03.190083] [WARN ] [MAIN ] MFA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:03.205473] [INFO ] [MAIN ] MFA: Wiping existing primary data before full refresh. [2025-10-20T19:51:03.206285] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-05 (1.35 seconds). [2025-10-20T19:51:03.206376] [INFO ] [LOCAL ] MFA: Starting deletion of primary data. [2025-10-20T19:51:03.211761] [DEBUG] [LOCAL ] MFA: Deleted primary data file. [2025-10-20T19:51:03.212385] [INFO ] [MAIN ] MFA: Starting full primary refresh. [2025-10-20T19:51:03.457610] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-04. [2025-10-20T19:51:03.564399] [INFO ] [REMOTE ] MFA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:04.251389] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-06. [2025-10-20T19:51:04.282508] [INFO ] [REMOTE ] MEG: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T19:51:04.288236] [INFO ] [REMOTE ] MEG: Full fetch complete (47500 records). [2025-10-20T19:51:04.288355] [DEBUG] [VALIDATOR ] MEG: Validating primary data. [2025-10-20T19:51:04.692098] [DEBUG] [VALIDATOR ] MEG: All expected trading days are present. [2025-10-20T19:51:04.692186] [DEBUG] [VALIDATOR ] MEG: Primary data validation passed. [2025-10-20T19:51:04.692987] [INFO ] [MAIN ] MEG: Fetched primary data is valid. [2025-10-20T19:51:04.693053] [DEBUG] [LOCAL ] MEG: Saving primary data. [2025-10-20T19:51:04.694171] [DEBUG] [LOCAL ] MEG: Trimmed 47500 records to 31609 (keeping 200 days of data) [2025-10-20T19:51:04.876405] [DEBUG] [LOCAL ] MEG: Primary data saved (31609 records). [2025-10-20T19:51:04.876621] [INFO ] [MAIN ] MEG: Full history saved. [2025-10-20T19:51:04.876848] [INFO ] [MAIN ] Update Progress: 1326/2367 symbols completed. MEG took 28.7s. Estimated remaining: 2h 14m 15s. Rate: 7.8 symbols/min [2025-10-20T19:51:05.047592] [INFO ] [MAIN ] MFAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:05.059268] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-01. [2025-10-20T19:51:05.081296] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T19:51:05.096154] [DEBUG] [LOCAL ] MFAN: Loaded 4421 existing records from primary storage. [2025-10-20T19:51:05.096754] [INFO ] [MAIN ] MFAN: Loaded 4421 existing records. [2025-10-20T19:51:05.298586] [WARN ] [MAIN ] MFAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:05.298943] [INFO ] [MAIN ] MFAN: Wiping existing primary data before full refresh. [2025-10-20T19:51:05.299053] [INFO ] [LOCAL ] MFAN: Starting deletion of primary data. [2025-10-20T19:51:05.304512] [DEBUG] [LOCAL ] MFAN: Deleted primary data file. [2025-10-20T19:51:05.313613] [INFO ] [MAIN ] MFAN: Starting full primary refresh. [2025-10-20T19:51:05.320138] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:51:05.671058] [INFO ] [REMOTE ] MFAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:05.851377] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-01. [2025-10-20T19:51:06.401692] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T19:51:06.643921] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:51:06.652055] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-02. [2025-10-20T19:51:07.123263] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T19:51:07.451369] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-03. [2025-10-20T19:51:08.046292] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T19:51:08.251384] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-04. [2025-10-20T19:51:08.862301] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T19:51:09.051384] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-07. [2025-10-20T19:51:09.851416] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-02. [2025-10-20T19:51:10.557700] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T19:51:10.651395] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-08. [2025-10-20T19:51:11.422180] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T19:51:11.451337] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-03. [2025-10-20T19:51:11.959215] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:51:12.251371] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-09. [2025-10-20T19:51:13.051365] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-05. [2025-10-20T19:51:13.141417] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:51:13.602401] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:51:13.851378] [INFO ] [REMOTE ] MEI: Fetching primary data for 2025-10. [2025-10-20T19:51:14.651359] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-05. [2025-10-20T19:51:15.048275] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T19:51:15.131197] [INFO ] [REMOTE ] MEI: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T19:51:15.135804] [INFO ] [REMOTE ] MEI: Full fetch complete (45365 records). [2025-10-20T19:51:15.135900] [DEBUG] [VALIDATOR ] MEI: Validating primary data. [2025-10-20T19:51:15.334412] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T19:51:15.456608] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-06. [2025-10-20T19:51:15.541365] [DEBUG] [VALIDATOR ] MEI: All expected trading days are present. [2025-10-20T19:51:15.541443] [DEBUG] [VALIDATOR ] MEI: Primary data validation passed. [2025-10-20T19:51:15.542328] [INFO ] [MAIN ] MEI: Fetched primary data is valid. [2025-10-20T19:51:15.542389] [DEBUG] [LOCAL ] MEI: Saving primary data. [2025-10-20T19:51:15.543376] [DEBUG] [LOCAL ] MEI: Trimmed 45365 records to 31501 (keeping 200 days of data) [2025-10-20T19:51:15.717760] [DEBUG] [LOCAL ] MEI: Primary data saved (31501 records). [2025-10-20T19:51:15.717889] [INFO ] [MAIN ] MEI: Full history saved. [2025-10-20T19:51:15.718243] [INFO ] [MAIN ] Update Progress: 1327/2367 symbols completed. MEI took 31.0s. Estimated remaining: 2h 14m 10s. Rate: 7.8 symbols/min [2025-10-20T19:51:15.882089] [INFO ] [MAIN ] MFAO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:15.889687] [DEBUG] [LOCAL ] MFAO: Loaded 4021 existing records from primary storage. [2025-10-20T19:51:15.889907] [INFO ] [MAIN ] MFAO: Loaded 4021 existing records. [2025-10-20T19:51:16.067360] [WARN ] [MAIN ] MFAO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:16.067633] [INFO ] [MAIN ] MFAO: Wiping existing primary data before full refresh. [2025-10-20T19:51:16.067720] [INFO ] [LOCAL ] MFAO: Starting deletion of primary data. [2025-10-20T19:51:16.067886] [DEBUG] [LOCAL ] MFAO: Deleted primary data file. [2025-10-20T19:51:16.067942] [INFO ] [MAIN ] MFAO: Starting full primary refresh. [2025-10-20T19:51:16.171321] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T19:51:16.259325] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-04. [2025-10-20T19:51:16.421739] [INFO ] [REMOTE ] MFAO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:17.051369] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-01. [2025-10-20T19:51:17.612563] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-01 (0.56 seconds). [2025-10-20T19:51:17.851351] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-07. [2025-10-20T19:51:17.886467] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T19:51:18.427505] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T19:51:18.651384] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-08. [2025-10-20T19:51:19.204109] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T19:51:19.451411] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-09. [2025-10-20T19:51:20.043032] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:51:20.251375] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-05. [2025-10-20T19:51:21.051369] [INFO ] [REMOTE ] MFAN: Fetching primary data for 2025-10. [2025-10-20T19:51:21.730398] [INFO ] [REMOTE ] MFAN: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T19:51:21.731697] [INFO ] [REMOTE ] MFAN: Full fetch complete (5925 records). [2025-10-20T19:51:21.731769] [DEBUG] [VALIDATOR ] MFAN: Validating primary data. [2025-10-20T19:51:21.852343] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-02. [2025-10-20T19:51:22.133401] [DEBUG] [VALIDATOR ] MFAN: All expected trading days are present. [2025-10-20T19:51:22.133509] [DEBUG] [VALIDATOR ] MFAN: Primary data validation passed. [2025-10-20T19:51:22.133776] [INFO ] [MAIN ] MFAN: Fetched primary data is valid. [2025-10-20T19:51:22.133865] [DEBUG] [LOCAL ] MFAN: Saving primary data. [2025-10-20T19:51:22.143701] [DEBUG] [LOCAL ] MFAN: Trimmed 5925 records to 4316 (keeping 200 days of data) [2025-10-20T19:51:22.172758] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T19:51:22.195258] [DEBUG] [LOCAL ] MFAN: Primary data saved (4316 records). [2025-10-20T19:51:22.195381] [INFO ] [MAIN ] MFAN: Full history saved. [2025-10-20T19:51:22.195466] [INFO ] [MAIN ] Update Progress: 1328/2367 symbols completed. MFAN took 17.3s. Estimated remaining: 2h 14m 1s. Rate: 7.8 symbols/min [2025-10-20T19:51:22.375068] [INFO ] [MAIN ] MFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:22.424174] [DEBUG] [LOCAL ] MFC: Loaded 51852 existing records from primary storage. [2025-10-20T19:51:22.424544] [INFO ] [MAIN ] MFC: Loaded 51852 existing records. [2025-10-20T19:51:22.507809] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T19:51:22.617409] [WARN ] [MAIN ] MFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:22.617635] [INFO ] [MAIN ] MFC: Wiping existing primary data before full refresh. [2025-10-20T19:51:22.617717] [INFO ] [LOCAL ] MFC: Starting deletion of primary data. [2025-10-20T19:51:22.618125] [DEBUG] [LOCAL ] MFC: Deleted primary data file. [2025-10-20T19:51:22.618182] [INFO ] [MAIN ] MFC: Starting full primary refresh. [2025-10-20T19:51:22.657602] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-03. [2025-10-20T19:51:22.983809] [INFO ] [REMOTE ] MFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:23.301525] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T19:51:23.451343] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-06. [2025-10-20T19:51:24.251373] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-04. [2025-10-20T19:51:24.869227] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T19:51:25.051352] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-06. [2025-10-20T19:51:25.200001] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:51:25.851367] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-01. [2025-10-20T19:51:26.659225] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-05. [2025-10-20T19:51:26.666579] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T19:51:27.139291] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T19:51:27.257741] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T19:51:27.451354] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-07. [2025-10-20T19:51:28.251382] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-07. [2025-10-20T19:51:29.051375] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-06. [2025-10-20T19:51:29.283510] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T19:51:29.371964] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-06 (0.32 seconds). [2025-10-20T19:51:29.536476] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T19:51:29.851416] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-08. [2025-10-20T19:51:30.651418] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-08. [2025-10-20T19:51:31.451352] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-07. [2025-10-20T19:51:31.688690] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T19:51:31.787959] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T19:51:32.000299] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-07 (0.55 seconds). [2025-10-20T19:51:32.251392] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-08. [2025-10-20T19:51:32.788260] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T19:51:33.051408] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-02. [2025-10-20T19:51:33.851452] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-09. [2025-10-20T19:51:34.651351] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-09. [2025-10-20T19:51:34.682166] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T19:51:35.182660] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T19:51:35.451360] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-03. [2025-10-20T19:51:35.575638] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:51:36.251422] [INFO ] [REMOTE ] MFAO: Fetching primary data for 2025-10. [2025-10-20T19:51:36.822822] [INFO ] [REMOTE ] MFAO: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T19:51:36.823976] [INFO ] [REMOTE ] MFAO: Full fetch complete (5476 records). [2025-10-20T19:51:36.824042] [DEBUG] [VALIDATOR ] MFAO: Validating primary data. [2025-10-20T19:51:37.056585] [INFO ] [REMOTE ] MET: Fetching primary data for 2025-10. [2025-10-20T19:51:37.182431] [DEBUG] [VALIDATOR ] MFAO: All expected trading days are present. [2025-10-20T19:51:37.182577] [DEBUG] [VALIDATOR ] MFAO: Primary data validation passed. [2025-10-20T19:51:37.182787] [INFO ] [MAIN ] MFAO: Fetched primary data is valid. [2025-10-20T19:51:37.182863] [DEBUG] [LOCAL ] MFAO: Saving primary data. [2025-10-20T19:51:37.183287] [DEBUG] [LOCAL ] MFAO: Trimmed 5476 records to 3969 (keeping 200 days of data) [2025-10-20T19:51:37.207000] [DEBUG] [LOCAL ] MFAO: Primary data saved (3969 records). [2025-10-20T19:51:37.207186] [INFO ] [MAIN ] MFAO: Full history saved. [2025-10-20T19:51:37.207323] [INFO ] [MAIN ] Update Progress: 1329/2367 symbols completed. MFAO took 21.5s. Estimated remaining: 2h 13m 59s. Rate: 7.7 symbols/min [2025-10-20T19:51:37.365518] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:51:37.426858] [INFO ] [MAIN ] MFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:37.472021] [DEBUG] [LOCAL ] MFG: Loaded 50230 existing records from primary storage. [2025-10-20T19:51:37.472357] [INFO ] [MAIN ] MFG: Loaded 50230 existing records. [2025-10-20T19:51:37.646467] [WARN ] [MAIN ] MFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:37.646761] [INFO ] [MAIN ] MFG: Wiping existing primary data before full refresh. [2025-10-20T19:51:37.646830] [INFO ] [LOCAL ] MFG: Starting deletion of primary data. [2025-10-20T19:51:37.647007] [DEBUG] [LOCAL ] MFG: Deleted primary data file. [2025-10-20T19:51:37.647044] [INFO ] [MAIN ] MFG: Starting full primary refresh. [2025-10-20T19:51:37.855700] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-04. [2025-10-20T19:51:38.027509] [INFO ] [REMOTE ] MFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:38.029857] [INFO ] [REMOTE ] MET: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T19:51:38.038005] [INFO ] [REMOTE ] MET: Full fetch complete (77368 records). [2025-10-20T19:51:38.038179] [DEBUG] [VALIDATOR ] MET: Validating primary data. [2025-10-20T19:51:38.426589] [DEBUG] [VALIDATOR ] MET: All expected trading days are present. [2025-10-20T19:51:38.426687] [DEBUG] [VALIDATOR ] MET: Primary data validation passed. [2025-10-20T19:51:38.427663] [INFO ] [MAIN ] MET: Fetched primary data is valid. [2025-10-20T19:51:38.427724] [DEBUG] [LOCAL ] MET: Saving primary data. [2025-10-20T19:51:38.429156] [DEBUG] [LOCAL ] MET: Trimmed 77368 records to 53307 (keeping 200 days of data) [2025-10-20T19:51:38.652282] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-09. [2025-10-20T19:51:38.757591] [DEBUG] [LOCAL ] MET: Primary data saved (53307 records). [2025-10-20T19:51:38.757764] [INFO ] [MAIN ] MET: Full history saved. [2025-10-20T19:51:38.757851] [INFO ] [MAIN ] Update Progress: 1330/2367 symbols completed. MET took 53.8s. Estimated remaining: 2h 13m 46s. Rate: 7.8 symbols/min [2025-10-20T19:51:38.927016] [INFO ] [MAIN ] MFM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:38.942489] [DEBUG] [LOCAL ] MFM: Loaded 14700 existing records from primary storage. [2025-10-20T19:51:38.942749] [INFO ] [MAIN ] MFM: Loaded 14700 existing records. [2025-10-20T19:51:39.118480] [WARN ] [MAIN ] MFM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:51:39.118722] [INFO ] [MAIN ] MFM: Wiping existing primary data before full refresh. [2025-10-20T19:51:39.118776] [INFO ] [LOCAL ] MFM: Starting deletion of primary data. [2025-10-20T19:51:39.118936] [DEBUG] [LOCAL ] MFM: Deleted primary data file. [2025-10-20T19:51:39.118973] [INFO ] [MAIN ] MFM: Starting full primary refresh. [2025-10-20T19:51:39.453521] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-01. [2025-10-20T19:51:39.484549] [INFO ] [REMOTE ] MFM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:39.733262] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T19:51:40.251413] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-01. [2025-10-20T19:51:40.362970] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T19:51:40.994935] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T19:51:41.051345] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-02. [2025-10-20T19:51:41.106218] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:51:41.833380] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:51:41.851330] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-05. [2025-10-20T19:51:42.651474] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-03. [2025-10-20T19:51:43.451349] [INFO ] [REMOTE ] MFA: Fetching primary data for 2025-10. [2025-10-20T19:51:43.567061] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-03 (0.92 seconds). [2025-10-20T19:51:43.650139] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T19:51:44.251386] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-04. [2025-10-20T19:51:44.851732] [INFO ] [REMOTE ] MFA: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T19:51:44.858629] [INFO ] [REMOTE ] MFA: Full fetch complete (65358 records). [2025-10-20T19:51:44.858820] [DEBUG] [VALIDATOR ] MFA: Validating primary data. [2025-10-20T19:51:45.061010] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-06. [2025-10-20T19:51:45.251838] [DEBUG] [VALIDATOR ] MFA: All expected trading days are present. [2025-10-20T19:51:45.252088] [DEBUG] [VALIDATOR ] MFA: Primary data validation passed. [2025-10-20T19:51:45.253140] [INFO ] [MAIN ] MFA: Fetched primary data is valid. [2025-10-20T19:51:45.253224] [DEBUG] [LOCAL ] MFA: Saving primary data. [2025-10-20T19:51:45.254680] [DEBUG] [LOCAL ] MFA: Trimmed 65358 records to 45743 (keeping 200 days of data) [2025-10-20T19:51:45.357924] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:51:45.546481] [DEBUG] [LOCAL ] MFA: Primary data saved (45743 records). [2025-10-20T19:51:45.546625] [INFO ] [MAIN ] MFA: Full history saved. [2025-10-20T19:51:45.546741] [INFO ] [MAIN ] Update Progress: 1331/2367 symbols completed. MFA took 42.8s. Estimated remaining: 2h 13m 38s. Rate: 7.8 symbols/min [2025-10-20T19:51:45.712071] [INFO ] [MAIN ] MFSB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:51:45.712229] [DEBUG] [LOCAL ] MFSB: No existing data file found in primary storage. [2025-10-20T19:51:45.712265] [WARN ] [MAIN ] MFSB: No existing data found, performing full refresh instead. [2025-10-20T19:51:45.712389] [INFO ] [MAIN ] MFSB: Wiping existing primary data before full refresh. [2025-10-20T19:51:45.712428] [INFO ] [LOCAL ] MFSB: Starting deletion of primary data. [2025-10-20T19:51:45.712467] [DEBUG] [LOCAL ] MFSB: No primary data file found to delete. [2025-10-20T19:51:45.712499] [INFO ] [MAIN ] MFSB: Starting full primary refresh. [2025-10-20T19:51:45.852470] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-02. [2025-10-20T19:51:46.064675] [INFO ] [REMOTE ] MFSB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:51:46.651447] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-05. [2025-10-20T19:51:46.804609] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:51:47.413629] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:51:47.454609] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-01. [2025-10-20T19:51:47.458314] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T19:51:47.789187] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-01 (0.33 seconds). [2025-10-20T19:51:48.251368] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-03. [2025-10-20T19:51:49.051393] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-07. [2025-10-20T19:51:49.852611] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-06. [2025-10-20T19:51:49.855695] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T19:51:50.496112] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T19:51:50.651349] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-02. [2025-10-20T19:51:50.778631] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T19:51:50.893397] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-02 (0.24 seconds). [2025-10-20T19:51:51.451364] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-07. [2025-10-20T19:51:52.251359] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-08. [2025-10-20T19:51:52.396512] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T19:51:53.051480] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-04. [2025-10-20T19:51:53.851349] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-08. [2025-10-20T19:51:53.969729] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:51:54.373134] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T19:51:54.651431] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-09. [2025-10-20T19:51:54.822183] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:51:55.451389] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-03. [2025-10-20T19:51:55.788225] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-03 (0.34 seconds). [2025-10-20T19:51:56.251376] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-05. [2025-10-20T19:51:56.327826] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:51:57.051385] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-04. [2025-10-20T19:51:57.388072] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-04 (0.34 seconds). [2025-10-20T19:51:57.851420] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-09. [2025-10-20T19:51:57.895186] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T19:51:58.651423] [INFO ] [REMOTE ] MFC: Fetching primary data for 2025-10. [2025-10-20T19:51:58.671194] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T19:51:59.451378] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-06. [2025-10-20T19:52:00.251380] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-05. [2025-10-20T19:52:00.371459] [INFO ] [REMOTE ] MFC: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T19:52:00.380090] [INFO ] [REMOTE ] MFC: Full fetch complete (74391 records). [2025-10-20T19:52:00.380241] [DEBUG] [VALIDATOR ] MFC: Validating primary data. [2025-10-20T19:52:00.560331] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T19:52:00.776010] [DEBUG] [VALIDATOR ] MFC: All expected trading days are present. [2025-10-20T19:52:00.776098] [DEBUG] [VALIDATOR ] MFC: Primary data validation passed. [2025-10-20T19:52:00.777103] [INFO ] [MAIN ] MFC: Fetched primary data is valid. [2025-10-20T19:52:00.777167] [DEBUG] [LOCAL ] MFC: Saving primary data. [2025-10-20T19:52:00.778524] [DEBUG] [LOCAL ] MFC: Trimmed 74391 records to 51048 (keeping 200 days of data) [2025-10-20T19:52:01.053260] [INFO ] [REMOTE ] MFM: Fetching primary data for 2025-10. [2025-10-20T19:52:01.089768] [DEBUG] [LOCAL ] MFC: Primary data saved (51048 records). [2025-10-20T19:52:01.089926] [INFO ] [MAIN ] MFC: Full history saved. [2025-10-20T19:52:01.090023] [INFO ] [MAIN ] Update Progress: 1332/2367 symbols completed. MFC took 38.9s. Estimated remaining: 2h 13m 36s. Rate: 7.7 symbols/min [2025-10-20T19:52:01.266465] [INFO ] [MAIN ] MFSG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:01.295019] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T19:52:01.307470] [DEBUG] [LOCAL ] MFSG: Loaded 5881 existing records from primary storage. [2025-10-20T19:52:01.307949] [INFO ] [MAIN ] MFSG: Loaded 5881 existing records. [2025-10-20T19:52:01.491377] [WARN ] [MAIN ] MFSG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:01.491671] [INFO ] [MAIN ] MFSG: Wiping existing primary data before full refresh. [2025-10-20T19:52:01.491742] [INFO ] [LOCAL ] MFSG: Starting deletion of primary data. [2025-10-20T19:52:01.492060] [DEBUG] [LOCAL ] MFSG: Deleted primary data file. [2025-10-20T19:52:01.492131] [INFO ] [MAIN ] MFSG: Starting full primary refresh. [2025-10-20T19:52:01.866823] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-06. [2025-10-20T19:52:01.878752] [INFO ] [REMOTE ] MFSG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:01.881037] [INFO ] [REMOTE ] MFM: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:52:01.883329] [INFO ] [REMOTE ] MFM: Full fetch complete (19349 records). [2025-10-20T19:52:01.883409] [DEBUG] [VALIDATOR ] MFM: Validating primary data. [2025-10-20T19:52:02.247190] [DEBUG] [VALIDATOR ] MFM: All expected trading days are present. [2025-10-20T19:52:02.247311] [DEBUG] [VALIDATOR ] MFM: Primary data validation passed. [2025-10-20T19:52:02.247810] [INFO ] [MAIN ] MFM: Fetched primary data is valid. [2025-10-20T19:52:02.247871] [DEBUG] [LOCAL ] MFM: Saving primary data. [2025-10-20T19:52:02.248412] [DEBUG] [LOCAL ] MFM: Trimmed 19349 records to 14274 (keeping 200 days of data) [2025-10-20T19:52:02.308243] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T19:52:02.332764] [DEBUG] [LOCAL ] MFM: Primary data saved (14274 records). [2025-10-20T19:52:02.332856] [INFO ] [MAIN ] MFM: Full history saved. [2025-10-20T19:52:02.332925] [INFO ] [MAIN ] Update Progress: 1333/2367 symbols completed. MFM took 23.6s. Estimated remaining: 2h 13m 24s. Rate: 7.8 symbols/min [2025-10-20T19:52:02.500998] [INFO ] [MAIN ] MFSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:02.514201] [DEBUG] [LOCAL ] MFSI: Loaded 12436 existing records from primary storage. [2025-10-20T19:52:02.514395] [INFO ] [MAIN ] MFSI: Loaded 12436 existing records. [2025-10-20T19:52:02.656440] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-01. [2025-10-20T19:52:02.699592] [WARN ] [MAIN ] MFSI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:02.700496] [INFO ] [MAIN ] MFSI: Wiping existing primary data before full refresh. [2025-10-20T19:52:02.700543] [INFO ] [LOCAL ] MFSI: Starting deletion of primary data. [2025-10-20T19:52:02.700745] [DEBUG] [LOCAL ] MFSI: Deleted primary data file. [2025-10-20T19:52:02.700801] [INFO ] [MAIN ] MFSI: Starting full primary refresh. [2025-10-20T19:52:03.073560] [INFO ] [REMOTE ] MFSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:03.225944] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:52:03.451373] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-07. [2025-10-20T19:52:04.251435] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-02. [2025-10-20T19:52:04.764730] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T19:52:05.051408] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-01. [2025-10-20T19:52:05.122753] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T19:52:05.712304] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-10-20T19:52:05.851395] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-07. [2025-10-20T19:52:06.170902] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T19:52:06.651387] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-08. [2025-10-20T19:52:07.451386] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-03. [2025-10-20T19:52:07.886099] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T19:52:08.251368] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-08. [2025-10-20T19:52:08.420329] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T19:52:08.720342] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T19:52:09.051378] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-02. [2025-10-20T19:52:09.626873] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T19:52:09.851399] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-03. [2025-10-20T19:52:10.450907] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T19:52:10.651381] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-04. [2025-10-20T19:52:11.346292] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T19:52:11.451391] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-04. [2025-10-20T19:52:11.816142] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-04 (0.36 seconds). [2025-10-20T19:52:12.251391] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-09. [2025-10-20T19:52:13.051378] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-05. [2025-10-20T19:52:13.797234] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T19:52:13.851402] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-06. [2025-10-20T19:52:13.948000] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T19:52:14.604032] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T19:52:14.651406] [INFO ] [REMOTE ] MFG: Fetching primary data for 2025-10. [2025-10-20T19:52:15.451408] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-05. [2025-10-20T19:52:16.002984] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T19:52:16.251409] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-07. [2025-10-20T19:52:16.530746] [INFO ] [REMOTE ] MFG: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T19:52:16.538500] [INFO ] [REMOTE ] MFG: Full fetch complete (69750 records). [2025-10-20T19:52:16.538699] [DEBUG] [VALIDATOR ] MFG: Validating primary data. [2025-10-20T19:52:16.941783] [DEBUG] [VALIDATOR ] MFG: All expected trading days are present. [2025-10-20T19:52:16.941894] [DEBUG] [VALIDATOR ] MFG: Primary data validation passed. [2025-10-20T19:52:16.943300] [INFO ] [MAIN ] MFG: Fetched primary data is valid. [2025-10-20T19:52:16.943375] [DEBUG] [LOCAL ] MFG: Saving primary data. [2025-10-20T19:52:16.944889] [DEBUG] [LOCAL ] MFG: Trimmed 69750 records to 49621 (keeping 200 days of data) [2025-10-20T19:52:17.052373] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-09. [2025-10-20T19:52:17.233089] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T19:52:17.248761] [DEBUG] [LOCAL ] MFG: Primary data saved (49621 records). [2025-10-20T19:52:17.248911] [INFO ] [MAIN ] MFG: Full history saved. [2025-10-20T19:52:17.248997] [INFO ] [MAIN ] Update Progress: 1334/2367 symbols completed. MFG took 40.0s. Estimated remaining: 2h 13m 21s. Rate: 7.7 symbols/min [2025-10-20T19:52:17.455136] [INFO ] [MAIN ] MFSV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:17.460428] [DEBUG] [LOCAL ] MFSV: Loaded 2916 existing records from primary storage. [2025-10-20T19:52:17.460584] [INFO ] [MAIN ] MFSV: Loaded 2916 existing records. [2025-10-20T19:52:17.654982] [WARN ] [MAIN ] MFSV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:17.655214] [INFO ] [MAIN ] MFSV: Wiping existing primary data before full refresh. [2025-10-20T19:52:17.655263] [INFO ] [LOCAL ] MFSV: Starting deletion of primary data. [2025-10-20T19:52:17.655396] [DEBUG] [LOCAL ] MFSV: Deleted primary data file. [2025-10-20T19:52:17.655436] [INFO ] [MAIN ] MFSV: Starting full primary refresh. [2025-10-20T19:52:17.741342] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T19:52:17.858048] [INFO ] [REMOTE ] MFSB: Fetching primary data for 2025-10. [2025-10-20T19:52:18.055364] [INFO ] [REMOTE ] MFSV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:18.601250] [INFO ] [REMOTE ] MFSB: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:52:18.602554] [INFO ] [REMOTE ] MFSB: Full fetch complete (1980 records). [2025-10-20T19:52:18.602625] [DEBUG] [VALIDATOR ] MFSB: Validating primary data. [2025-10-20T19:52:18.661719] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-01. [2025-10-20T19:52:18.785220] [WARN ] [VALIDATOR ] MFSB: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T19:52:18.785413] [FATAL] [MAIN ] MFSB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:52:18.785480] [WARN ] [MAIN ] MFSB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:52:18.785551] [INFO ] [LOCAL ] MFSB: Starting deletion of primary data. [2025-10-20T19:52:18.785612] [DEBUG] [LOCAL ] MFSB: No primary data file found to delete. [2025-10-20T19:52:18.785671] [INFO ] [LOCAL ] MFSB: Starting deletion of derivative data. [2025-10-20T19:52:18.785734] [DEBUG] [LOCAL ] MFSB: No derivative data file found to delete. [2025-10-20T19:52:18.785790] [INFO ] [MAIN ] Update Progress: 1335/2367 symbols completed. MFSB took 33.2s. Estimated remaining: 2h 13m 9s. Rate: 7.8 symbols/min [2025-10-20T19:52:18.947510] [INFO ] [MAIN ] MG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:18.964093] [DEBUG] [LOCAL ] MG: Loaded 16048 existing records from primary storage. [2025-10-20T19:52:18.964331] [INFO ] [MAIN ] MG: Loaded 16048 existing records. [2025-10-20T19:52:19.067800] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-01 (0.41 seconds). [2025-10-20T19:52:19.143580] [WARN ] [MAIN ] MG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:19.143800] [INFO ] [MAIN ] MG: Wiping existing primary data before full refresh. [2025-10-20T19:52:19.143847] [INFO ] [LOCAL ] MG: Starting deletion of primary data. [2025-10-20T19:52:19.144019] [DEBUG] [LOCAL ] MG: Deleted primary data file. [2025-10-20T19:52:19.144073] [INFO ] [MAIN ] MG: Starting full primary refresh. [2025-10-20T19:52:19.460760] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-08. [2025-10-20T19:52:19.491317] [INFO ] [REMOTE ] MG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:20.251390] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-01. [2025-10-20T19:52:20.452636] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T19:52:21.051363] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-09. [2025-10-20T19:52:21.223787] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T19:52:21.851357] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-06. [2025-10-20T19:52:21.985510] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T19:52:22.447309] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T19:52:22.651409] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-07. [2025-10-20T19:52:23.223893] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T19:52:23.451382] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-02. [2025-10-20T19:52:24.251375] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-02. [2025-10-20T19:52:24.321593] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T19:52:24.672605] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T19:52:25.051372] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-03. [2025-10-20T19:52:25.851392] [INFO ] [REMOTE ] MFSI: Fetching primary data for 2025-10. [2025-10-20T19:52:26.160323] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T19:52:26.651445] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-04. [2025-10-20T19:52:26.856105] [INFO ] [REMOTE ] MFSI: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T19:52:26.858939] [INFO ] [REMOTE ] MFSI: Full fetch complete (14796 records). [2025-10-20T19:52:26.859007] [DEBUG] [VALIDATOR ] MFSI: Validating primary data. [2025-10-20T19:52:27.223472] [DEBUG] [VALIDATOR ] MFSI: All expected trading days are present. [2025-10-20T19:52:27.223552] [DEBUG] [VALIDATOR ] MFSI: Primary data validation passed. [2025-10-20T19:52:27.223889] [INFO ] [MAIN ] MFSI: Fetched primary data is valid. [2025-10-20T19:52:27.223949] [DEBUG] [LOCAL ] MFSI: Saving primary data. [2025-10-20T19:52:27.224550] [DEBUG] [LOCAL ] MFSI: Trimmed 14796 records to 12471 (keeping 200 days of data) [2025-10-20T19:52:27.297553] [DEBUG] [LOCAL ] MFSI: Primary data saved (12471 records). [2025-10-20T19:52:27.297710] [INFO ] [MAIN ] MFSI: Full history saved. [2025-10-20T19:52:27.297798] [INFO ] [MAIN ] Update Progress: 1336/2367 symbols completed. MFSI took 25.0s. Estimated remaining: 2h 13m 2s. Rate: 7.8 symbols/min [2025-10-20T19:52:27.472157] [INFO ] [MAIN ] MGA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:27.468674] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-08. [2025-10-20T19:52:27.523531] [DEBUG] [LOCAL ] MGA: Loaded 51244 existing records from primary storage. [2025-10-20T19:52:27.523901] [INFO ] [MAIN ] MGA: Loaded 51244 existing records. [2025-10-20T19:52:27.729187] [WARN ] [MAIN ] MGA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:27.729442] [INFO ] [MAIN ] MGA: Wiping existing primary data before full refresh. [2025-10-20T19:52:27.729674] [INFO ] [LOCAL ] MGA: Starting deletion of primary data. [2025-10-20T19:52:27.740858] [DEBUG] [LOCAL ] MGA: Deleted primary data file. [2025-10-20T19:52:27.740959] [INFO ] [MAIN ] MGA: Starting full primary refresh. [2025-10-20T19:52:27.772157] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:52:28.093088] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T19:52:28.169599] [INFO ] [REMOTE ] MGA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:28.251367] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-03. [2025-10-20T19:52:28.818245] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:52:29.051399] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-05. [2025-10-20T19:52:29.851384] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-01. [2025-10-20T19:52:30.031473] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T19:52:30.651404] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-04. [2025-10-20T19:52:31.085188] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T19:52:31.457718] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-09. [2025-10-20T19:52:31.476389] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T19:52:32.251411] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-05. [2025-10-20T19:52:32.273867] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T19:52:32.740404] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T19:52:33.051408] [INFO ] [REMOTE ] MFSG: Fetching primary data for 2025-10. [2025-10-20T19:52:33.851361] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-02. [2025-10-20T19:52:33.859026] [INFO ] [REMOTE ] MFSG: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:52:33.860821] [INFO ] [REMOTE ] MFSG: Full fetch complete (6717 records). [2025-10-20T19:52:33.860891] [DEBUG] [VALIDATOR ] MFSG: Validating primary data. [2025-10-20T19:52:34.237315] [DEBUG] [VALIDATOR ] MFSG: All expected trading days are present. [2025-10-20T19:52:34.237391] [DEBUG] [VALIDATOR ] MFSG: Primary data validation passed. [2025-10-20T19:52:34.237576] [INFO ] [MAIN ] MFSG: Fetched primary data is valid. [2025-10-20T19:52:34.237614] [DEBUG] [LOCAL ] MFSG: Saving primary data. [2025-10-20T19:52:34.238033] [DEBUG] [LOCAL ] MFSG: Trimmed 6717 records to 5899 (keeping 200 days of data) [2025-10-20T19:52:34.271992] [DEBUG] [LOCAL ] MFSG: Primary data saved (5899 records). [2025-10-20T19:52:34.272068] [INFO ] [MAIN ] MFSG: Full history saved. [2025-10-20T19:52:34.272138] [INFO ] [MAIN ] Update Progress: 1337/2367 symbols completed. MFSG took 33.2s. Estimated remaining: 2h 12m 53s. Rate: 7.8 symbols/min [2025-10-20T19:52:34.443846] [INFO ] [MAIN ] MGF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:34.450734] [DEBUG] [LOCAL ] MGF: Loaded 4196 existing records from primary storage. [2025-10-20T19:52:34.450951] [INFO ] [MAIN ] MGF: Loaded 4196 existing records. [2025-10-20T19:52:34.630904] [WARN ] [MAIN ] MGF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:34.631117] [INFO ] [MAIN ] MGF: Wiping existing primary data before full refresh. [2025-10-20T19:52:34.631167] [INFO ] [LOCAL ] MGF: Starting deletion of primary data. [2025-10-20T19:52:34.631296] [DEBUG] [LOCAL ] MGF: Deleted primary data file. [2025-10-20T19:52:34.631335] [INFO ] [MAIN ] MGF: Starting full primary refresh. [2025-10-20T19:52:34.661548] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-06. [2025-10-20T19:52:34.986938] [INFO ] [REMOTE ] MGF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:35.451381] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-01. [2025-10-20T19:52:35.490497] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:52:35.544202] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T19:52:36.001167] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T19:52:36.251371] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-07. [2025-10-20T19:52:37.051422] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-02. [2025-10-20T19:52:37.157097] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:52:37.609691] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T19:52:37.851405] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-06. [2025-10-20T19:52:38.258869] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T19:52:38.651398] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-08. [2025-10-20T19:52:39.451394] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-03. [2025-10-20T19:52:39.566484] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T19:52:40.030892] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T19:52:40.251373] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-04. [2025-10-20T19:52:40.829507] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T19:52:41.051407] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-03. [2025-10-20T19:52:41.851383] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-07. [2025-10-20T19:52:42.195213] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T19:52:42.651361] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-08. [2025-10-20T19:52:42.780553] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T19:52:43.224537] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T19:52:43.451411] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-05. [2025-10-20T19:52:44.044011] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:52:44.251366] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-04. [2025-10-20T19:52:45.051437] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-09. [2025-10-20T19:52:45.636172] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:52:45.851349] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-06. [2025-10-20T19:52:45.995022] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T19:52:46.431497] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:52:46.651458] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-07. [2025-10-20T19:52:46.988467] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T19:52:47.451389] [INFO ] [REMOTE ] MFSV: Fetching primary data for 2025-10. [2025-10-20T19:52:48.178538] [INFO ] [REMOTE ] MFSV: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T19:52:48.180326] [INFO ] [REMOTE ] MFSV: Full fetch complete (3600 records). [2025-10-20T19:52:48.180406] [DEBUG] [VALIDATOR ] MFSV: Validating primary data. [2025-10-20T19:52:48.259448] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-09. [2025-10-20T19:52:48.574615] [DEBUG] [VALIDATOR ] MFSV: All expected trading days are present. [2025-10-20T19:52:48.574723] [DEBUG] [VALIDATOR ] MFSV: Primary data validation passed. [2025-10-20T19:52:48.574860] [INFO ] [MAIN ] MFSV: Fetched primary data is valid. [2025-10-20T19:52:48.574905] [DEBUG] [LOCAL ] MFSV: Saving primary data. [2025-10-20T19:52:48.575267] [DEBUG] [LOCAL ] MFSV: Trimmed 3600 records to 2924 (keeping 200 days of data) [2025-10-20T19:52:48.593089] [DEBUG] [LOCAL ] MFSV: Primary data saved (2924 records). [2025-10-20T19:52:48.593226] [INFO ] [MAIN ] MFSV: Full history saved. [2025-10-20T19:52:48.593305] [INFO ] [MAIN ] Update Progress: 1338/2367 symbols completed. MFSV took 31.3s. Estimated remaining: 2h 12m 51s. Rate: 7.7 symbols/min [2025-10-20T19:52:48.772541] [INFO ] [MAIN ] MGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:48.822459] [DEBUG] [LOCAL ] MGM: Loaded 54387 existing records from primary storage. [2025-10-20T19:52:48.822817] [INFO ] [MAIN ] MGM: Loaded 54387 existing records. [2025-10-20T19:52:49.000091] [WARN ] [MAIN ] MGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:49.000270] [INFO ] [MAIN ] MGM: Wiping existing primary data before full refresh. [2025-10-20T19:52:49.000315] [INFO ] [LOCAL ] MGM: Starting deletion of primary data. [2025-10-20T19:52:49.000515] [DEBUG] [LOCAL ] MGM: Deleted primary data file. [2025-10-20T19:52:49.000570] [INFO ] [MAIN ] MGM: Starting full primary refresh. [2025-10-20T19:52:49.064136] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-08. [2025-10-20T19:52:49.178624] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T19:52:49.366526] [INFO ] [REMOTE ] MGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:49.525728] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T19:52:49.851403] [INFO ] [REMOTE ] MG: Fetching primary data for 2025-10. [2025-10-20T19:52:50.651425] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-01. [2025-10-20T19:52:50.876558] [INFO ] [REMOTE ] MG: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T19:52:50.879090] [INFO ] [REMOTE ] MG: Full fetch complete (23400 records). [2025-10-20T19:52:50.879174] [DEBUG] [VALIDATOR ] MG: Validating primary data. [2025-10-20T19:52:51.258221] [DEBUG] [VALIDATOR ] MG: All expected trading days are present. [2025-10-20T19:52:51.258295] [DEBUG] [VALIDATOR ] MG: Primary data validation passed. [2025-10-20T19:52:51.258853] [INFO ] [MAIN ] MG: Fetched primary data is valid. [2025-10-20T19:52:51.258912] [DEBUG] [LOCAL ] MG: Saving primary data. [2025-10-20T19:52:51.259511] [DEBUG] [LOCAL ] MG: Trimmed 23400 records to 15611 (keeping 200 days of data) [2025-10-20T19:52:51.343743] [DEBUG] [LOCAL ] MG: Primary data saved (15611 records). [2025-10-20T19:52:51.343869] [INFO ] [MAIN ] MG: Full history saved. [2025-10-20T19:52:51.343943] [INFO ] [MAIN ] Update Progress: 1339/2367 symbols completed. MG took 32.6s. Estimated remaining: 2h 12m 39s. Rate: 7.7 symbols/min [2025-10-20T19:52:51.456559] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-05. [2025-10-20T19:52:51.521533] [INFO ] [MAIN ] MGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:51.529474] [DEBUG] [LOCAL ] MGR: Loaded 5666 existing records from primary storage. [2025-10-20T19:52:51.529707] [INFO ] [MAIN ] MGR: Loaded 5666 existing records. [2025-10-20T19:52:51.704497] [WARN ] [MAIN ] MGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:51.704763] [INFO ] [MAIN ] MGR: Wiping existing primary data before full refresh. [2025-10-20T19:52:51.704820] [INFO ] [LOCAL ] MGR: Starting deletion of primary data. [2025-10-20T19:52:51.704969] [DEBUG] [LOCAL ] MGR: Deleted primary data file. [2025-10-20T19:52:51.705024] [INFO ] [MAIN ] MGR: Starting full primary refresh. [2025-10-20T19:52:52.075058] [INFO ] [REMOTE ] MGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:52.251347] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-09. [2025-10-20T19:52:52.494821] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T19:52:52.804803] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T19:52:53.051420] [INFO ] [REMOTE ] MGF: Fetching primary data for 2025-10. [2025-10-20T19:52:53.195445] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T19:52:53.698112] [INFO ] [REMOTE ] MGF: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T19:52:53.699332] [INFO ] [REMOTE ] MGF: Full fetch complete (5740 records). [2025-10-20T19:52:53.699403] [DEBUG] [VALIDATOR ] MGF: Validating primary data. [2025-10-20T19:52:53.853807] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-01. [2025-10-20T19:52:54.066087] [DEBUG] [VALIDATOR ] MGF: All expected trading days are present. [2025-10-20T19:52:54.066174] [DEBUG] [VALIDATOR ] MGF: Primary data validation passed. [2025-10-20T19:52:54.066345] [INFO ] [MAIN ] MGF: Fetched primary data is valid. [2025-10-20T19:52:54.066386] [DEBUG] [LOCAL ] MGF: Saving primary data. [2025-10-20T19:52:54.066783] [DEBUG] [LOCAL ] MGF: Trimmed 5740 records to 4141 (keeping 200 days of data) [2025-10-20T19:52:54.089791] [DEBUG] [LOCAL ] MGF: Primary data saved (4141 records). [2025-10-20T19:52:54.089862] [INFO ] [MAIN ] MGF: Full history saved. [2025-10-20T19:52:54.089932] [INFO ] [MAIN ] Update Progress: 1340/2367 symbols completed. MGF took 19.8s. Estimated remaining: 2h 12m 28s. Rate: 7.8 symbols/min [2025-10-20T19:52:54.255547] [INFO ] [MAIN ] MGRB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:52:54.262405] [DEBUG] [LOCAL ] MGRB: Loaded 4058 existing records from primary storage. [2025-10-20T19:52:54.262594] [INFO ] [MAIN ] MGRB: Loaded 4058 existing records. [2025-10-20T19:52:54.442069] [WARN ] [MAIN ] MGRB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:52:54.442335] [INFO ] [MAIN ] MGRB: Wiping existing primary data before full refresh. [2025-10-20T19:52:54.442389] [INFO ] [LOCAL ] MGRB: Starting deletion of primary data. [2025-10-20T19:52:54.442511] [DEBUG] [LOCAL ] MGRB: Deleted primary data file. [2025-10-20T19:52:54.442549] [INFO ] [MAIN ] MGRB: Starting full primary refresh. [2025-10-20T19:52:54.604973] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T19:52:54.658396] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-06. [2025-10-20T19:52:54.812928] [INFO ] [REMOTE ] MGRB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:52:55.451523] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-02. [2025-10-20T19:52:56.004585] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T19:52:56.251372] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-02. [2025-10-20T19:52:56.296621] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T19:52:57.051438] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-01. [2025-10-20T19:52:57.703317] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T19:52:57.851354] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-02. [2025-10-20T19:52:58.138985] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T19:52:58.402830] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T19:52:58.651413] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-03. [2025-10-20T19:52:59.451375] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-07. [2025-10-20T19:53:00.251355] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-03. [2025-10-20T19:53:00.629689] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T19:53:01.046259] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T19:53:01.051317] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-04. [2025-10-20T19:53:01.586007] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T19:53:01.851426] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-04. [2025-10-20T19:53:01.867302] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T19:53:02.651438] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-08. [2025-10-20T19:53:03.451395] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-03. [2025-10-20T19:53:03.813734] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T19:53:03.932938] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T19:53:04.251375] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-05. [2025-10-20T19:53:04.307816] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T19:53:04.876144] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T19:53:05.051443] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-05. [2025-10-20T19:53:05.851393] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-09. [2025-10-20T19:53:06.651358] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-04. [2025-10-20T19:53:06.899766] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T19:53:07.292216] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T19:53:07.451363] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-05. [2025-10-20T19:53:07.568253] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:53:08.066707] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T19:53:08.251367] [INFO ] [REMOTE ] MGA: Fetching primary data for 2025-10. [2025-10-20T19:53:09.051424] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-06. [2025-10-20T19:53:09.644399] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T19:53:09.851416] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-07. [2025-10-20T19:53:10.057011] [INFO ] [REMOTE ] MGA: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T19:53:10.064183] [INFO ] [REMOTE ] MGA: Full fetch complete (73664 records). [2025-10-20T19:53:10.064295] [DEBUG] [VALIDATOR ] MGA: Validating primary data. [2025-10-20T19:53:10.451901] [DEBUG] [VALIDATOR ] MGA: All expected trading days are present. [2025-10-20T19:53:10.451979] [DEBUG] [VALIDATOR ] MGA: Primary data validation passed. [2025-10-20T19:53:10.452941] [INFO ] [MAIN ] MGA: Fetched primary data is valid. [2025-10-20T19:53:10.453002] [DEBUG] [LOCAL ] MGA: Saving primary data. [2025-10-20T19:53:10.454323] [DEBUG] [LOCAL ] MGA: Trimmed 73664 records to 50442 (keeping 200 days of data) [2025-10-20T19:53:10.667705] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-06. [2025-10-20T19:53:10.668141] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T19:53:10.776419] [DEBUG] [LOCAL ] MGA: Primary data saved (50442 records). [2025-10-20T19:53:10.776592] [INFO ] [MAIN ] MGA: Full history saved. [2025-10-20T19:53:10.776725] [INFO ] [MAIN ] Update Progress: 1341/2367 symbols completed. MGA took 43.5s. Estimated remaining: 2h 12m 27s. Rate: 7.7 symbols/min [2025-10-20T19:53:10.951691] [INFO ] [MAIN ] MGRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:10.956810] [DEBUG] [LOCAL ] MGRD: Loaded 2248 existing records from primary storage. [2025-10-20T19:53:10.957038] [INFO ] [MAIN ] MGRD: Loaded 2248 existing records. [2025-10-20T19:53:11.151705] [WARN ] [MAIN ] MGRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:11.151911] [INFO ] [MAIN ] MGRD: Wiping existing primary data before full refresh. [2025-10-20T19:53:11.151958] [INFO ] [LOCAL ] MGRD: Starting deletion of primary data. [2025-10-20T19:53:11.152107] [DEBUG] [LOCAL ] MGRD: Deleted primary data file. [2025-10-20T19:53:11.152164] [INFO ] [MAIN ] MGRD: Starting full primary refresh. [2025-10-20T19:53:11.458580] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-06. [2025-10-20T19:53:11.557460] [INFO ] [REMOTE ] MGRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:12.137162] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T19:53:12.251386] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-08. [2025-10-20T19:53:12.418780] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T19:53:12.832213] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:53:13.051371] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-09. [2025-10-20T19:53:13.608375] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T19:53:13.851402] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-07. [2025-10-20T19:53:14.330765] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T19:53:14.651380] [INFO ] [REMOTE ] MGR: Fetching primary data for 2025-10. [2025-10-20T19:53:15.338595] [INFO ] [REMOTE ] MGR: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T19:53:15.339954] [INFO ] [REMOTE ] MGR: Full fetch complete (7844 records). [2025-10-20T19:53:15.340021] [DEBUG] [VALIDATOR ] MGR: Validating primary data. [2025-10-20T19:53:15.456463] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-01. [2025-10-20T19:53:15.711466] [DEBUG] [VALIDATOR ] MGR: All expected trading days are present. [2025-10-20T19:53:15.711549] [DEBUG] [VALIDATOR ] MGR: Primary data validation passed. [2025-10-20T19:53:15.711778] [INFO ] [MAIN ] MGR: Fetched primary data is valid. [2025-10-20T19:53:15.711837] [DEBUG] [LOCAL ] MGR: Saving primary data. [2025-10-20T19:53:15.712215] [DEBUG] [LOCAL ] MGR: Trimmed 7844 records to 5576 (keeping 200 days of data) [2025-10-20T19:53:15.745151] [DEBUG] [LOCAL ] MGR: Primary data saved (5576 records). [2025-10-20T19:53:15.745246] [INFO ] [MAIN ] MGR: Full history saved. [2025-10-20T19:53:15.745315] [INFO ] [MAIN ] Update Progress: 1342/2367 symbols completed. MGR took 24.4s. Estimated remaining: 2h 12m 17s. Rate: 7.7 symbols/min [2025-10-20T19:53:15.911784] [INFO ] [MAIN ] MGRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:15.920890] [DEBUG] [LOCAL ] MGRE: Loaded 6619 existing records from primary storage. [2025-10-20T19:53:15.921196] [INFO ] [MAIN ] MGRE: Loaded 6619 existing records. [2025-10-20T19:53:16.026399] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T19:53:16.103991] [WARN ] [MAIN ] MGRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:16.104184] [INFO ] [MAIN ] MGRE: Wiping existing primary data before full refresh. [2025-10-20T19:53:16.104228] [INFO ] [LOCAL ] MGRE: Starting deletion of primary data. [2025-10-20T19:53:16.104359] [DEBUG] [LOCAL ] MGRE: Deleted primary data file. [2025-10-20T19:53:16.104395] [INFO ] [MAIN ] MGRE: Starting full primary refresh. [2025-10-20T19:53:16.263192] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-07. [2025-10-20T19:53:16.485961] [INFO ] [REMOTE ] MGRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:17.051365] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-01. [2025-10-20T19:53:17.667233] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T19:53:17.851346] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-08. [2025-10-20T19:53:18.305301] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T19:53:18.428139] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:53:18.651414] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-02. [2025-10-20T19:53:19.021157] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T19:53:19.451381] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-03. [2025-10-20T19:53:20.025869] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T19:53:20.251370] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-04. [2025-10-20T19:53:20.952420] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T19:53:21.051339] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-05. [2025-10-20T19:53:21.671039] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T19:53:21.851360] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-09. [2025-10-20T19:53:22.300866] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-09 (0.45 seconds). [2025-10-20T19:53:22.651383] [INFO ] [REMOTE ] MGRB: Fetching primary data for 2025-10. [2025-10-20T19:53:23.192623] [INFO ] [REMOTE ] MGRB: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T19:53:23.193536] [INFO ] [REMOTE ] MGRB: Full fetch complete (5537 records). [2025-10-20T19:53:23.193603] [DEBUG] [VALIDATOR ] MGRB: Validating primary data. [2025-10-20T19:53:23.456461] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-02. [2025-10-20T19:53:23.569724] [DEBUG] [VALIDATOR ] MGRB: All expected trading days are present. [2025-10-20T19:53:23.569822] [DEBUG] [VALIDATOR ] MGRB: Primary data validation passed. [2025-10-20T19:53:23.569978] [INFO ] [MAIN ] MGRB: Fetched primary data is valid. [2025-10-20T19:53:23.570015] [DEBUG] [LOCAL ] MGRB: Saving primary data. [2025-10-20T19:53:23.570394] [DEBUG] [LOCAL ] MGRB: Trimmed 5537 records to 4005 (keeping 200 days of data) [2025-10-20T19:53:23.596314] [DEBUG] [LOCAL ] MGRB: Primary data saved (4005 records). [2025-10-20T19:53:23.596420] [INFO ] [MAIN ] MGRB: Full history saved. [2025-10-20T19:53:23.596496] [INFO ] [MAIN ] Update Progress: 1343/2367 symbols completed. MGRB took 29.5s. Estimated remaining: 2h 12m 9s. Rate: 7.7 symbols/min [2025-10-20T19:53:23.776463] [INFO ] [MAIN ] MGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:23.826433] [DEBUG] [LOCAL ] MGY: Loaded 53185 existing records from primary storage. [2025-10-20T19:53:23.826770] [INFO ] [MAIN ] MGY: Loaded 53185 existing records. [2025-10-20T19:53:23.986172] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T19:53:24.013543] [WARN ] [MAIN ] MGY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:24.015042] [INFO ] [MAIN ] MGY: Wiping existing primary data before full refresh. [2025-10-20T19:53:24.015090] [INFO ] [LOCAL ] MGY: Starting deletion of primary data. [2025-10-20T19:53:24.015261] [DEBUG] [LOCAL ] MGY: Deleted primary data file. [2025-10-20T19:53:24.015298] [INFO ] [MAIN ] MGY: Starting full primary refresh. [2025-10-20T19:53:24.257857] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-08. [2025-10-20T19:53:24.389417] [INFO ] [REMOTE ] MGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:25.051345] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-06. [2025-10-20T19:53:25.801185] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T19:53:25.851360] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-03. [2025-10-20T19:53:26.081967] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:53:26.271522] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T19:53:26.651385] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-01. [2025-10-20T19:53:27.451426] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-09. [2025-10-20T19:53:28.251396] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-07. [2025-10-20T19:53:28.315927] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T19:53:28.932283] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T19:53:29.051438] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-04. [2025-10-20T19:53:29.191450] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T19:53:29.537096] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T19:53:29.851417] [INFO ] [REMOTE ] MGM: Fetching primary data for 2025-10. [2025-10-20T19:53:30.654499] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-02. [2025-10-20T19:53:30.659412] [INFO ] [REMOTE ] MGM: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:53:30.667612] [INFO ] [REMOTE ] MGM: Full fetch complete (77762 records). [2025-10-20T19:53:30.667782] [DEBUG] [VALIDATOR ] MGM: Validating primary data. [2025-10-20T19:53:31.065310] [DEBUG] [VALIDATOR ] MGM: All expected trading days are present. [2025-10-20T19:53:31.065401] [DEBUG] [VALIDATOR ] MGM: Primary data validation passed. [2025-10-20T19:53:31.066419] [INFO ] [MAIN ] MGM: Fetched primary data is valid. [2025-10-20T19:53:31.066483] [DEBUG] [LOCAL ] MGM: Saving primary data. [2025-10-20T19:53:31.068009] [DEBUG] [LOCAL ] MGM: Trimmed 77762 records to 53607 (keeping 200 days of data) [2025-10-20T19:53:31.383228] [DEBUG] [LOCAL ] MGM: Primary data saved (53607 records). [2025-10-20T19:53:31.383361] [INFO ] [MAIN ] MGM: Full history saved. [2025-10-20T19:53:31.383726] [INFO ] [MAIN ] Update Progress: 1344/2367 symbols completed. MGM took 42.8s. Estimated remaining: 2h 12m 1s. Rate: 7.7 symbols/min [2025-10-20T19:53:31.457531] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-08. [2025-10-20T19:53:31.552150] [INFO ] [MAIN ] MH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:31.552315] [DEBUG] [LOCAL ] MH: No existing data file found in primary storage. [2025-10-20T19:53:31.552351] [WARN ] [MAIN ] MH: No existing data found, performing full refresh instead. [2025-10-20T19:53:31.552458] [INFO ] [MAIN ] MH: Wiping existing primary data before full refresh. [2025-10-20T19:53:31.552498] [INFO ] [LOCAL ] MH: Starting deletion of primary data. [2025-10-20T19:53:31.552538] [DEBUG] [LOCAL ] MH: No primary data file found to delete. [2025-10-20T19:53:31.552570] [INFO ] [MAIN ] MH: Starting full primary refresh. [2025-10-20T19:53:31.908019] [INFO ] [REMOTE ] MH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:32.160480] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T19:53:32.208888] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T19:53:32.251337] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-03. [2025-10-20T19:53:33.051379] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-09. [2025-10-20T19:53:33.794752] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T19:53:33.800431] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:53:33.851360] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-04. [2025-10-20T19:53:34.651499] [INFO ] [REMOTE ] MH: Fetching primary data for 2025-01. [2025-10-20T19:53:34.854415] [DEBUG] [REMOTE ] MH: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:53:34.854765] [WARN ] [REMOTE ] MH: No data returned for month 2025-01 [2025-10-20T19:53:34.854869] [FATAL] [REMOTE ] MH: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:53:34.854919] [FATAL] [MAIN ] MH: Failed to fetch primary data. [2025-10-20T19:53:34.854951] [WARN ] [MAIN ] MH: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:53:34.854985] [INFO ] [LOCAL ] MH: Starting deletion of primary data. [2025-10-20T19:53:34.855040] [DEBUG] [LOCAL ] MH: No primary data file found to delete. [2025-10-20T19:53:34.855074] [INFO ] [LOCAL ] MH: Starting deletion of derivative data. [2025-10-20T19:53:34.855120] [DEBUG] [LOCAL ] MH: No derivative data file found to delete. [2025-10-20T19:53:34.855165] [INFO ] [MAIN ] Update Progress: 1345/2367 symbols completed. MH took 3.5s. Estimated remaining: 2h 11m 50s. Rate: 7.8 symbols/min [2025-10-20T19:53:35.024544] [INFO ] [MAIN ] MHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:35.039383] [DEBUG] [LOCAL ] MHD: Loaded 12323 existing records from primary storage. [2025-10-20T19:53:35.039621] [INFO ] [MAIN ] MHD: Loaded 12323 existing records. [2025-10-20T19:53:35.224532] [WARN ] [MAIN ] MHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:35.224805] [INFO ] [MAIN ] MHD: Wiping existing primary data before full refresh. [2025-10-20T19:53:35.224856] [INFO ] [LOCAL ] MHD: Starting deletion of primary data. [2025-10-20T19:53:35.225075] [DEBUG] [LOCAL ] MHD: Deleted primary data file. [2025-10-20T19:53:35.225133] [INFO ] [MAIN ] MHD: Starting full primary refresh. [2025-10-20T19:53:35.460589] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-05. [2025-10-20T19:53:35.592969] [INFO ] [REMOTE ] MHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:35.629580] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T19:53:35.974246] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T19:53:36.251448] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-01. [2025-10-20T19:53:37.051343] [INFO ] [REMOTE ] MGRE: Fetching primary data for 2025-10. [2025-10-20T19:53:37.210561] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T19:53:37.851387] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-02. [2025-10-20T19:53:37.884122] [INFO ] [REMOTE ] MGRE: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T19:53:37.885362] [INFO ] [REMOTE ] MGRE: Full fetch complete (9185 records). [2025-10-20T19:53:37.885431] [DEBUG] [VALIDATOR ] MGRE: Validating primary data. [2025-10-20T19:53:38.242336] [DEBUG] [VALIDATOR ] MGRE: All expected trading days are present. [2025-10-20T19:53:38.242419] [DEBUG] [VALIDATOR ] MGRE: Primary data validation passed. [2025-10-20T19:53:38.242664] [INFO ] [MAIN ] MGRE: Fetched primary data is valid. [2025-10-20T19:53:38.242725] [DEBUG] [LOCAL ] MGRE: Saving primary data. [2025-10-20T19:53:38.243132] [DEBUG] [LOCAL ] MGRE: Trimmed 9185 records to 6471 (keeping 200 days of data) [2025-10-20T19:53:38.280604] [DEBUG] [LOCAL ] MGRE: Primary data saved (6471 records). [2025-10-20T19:53:38.280707] [INFO ] [MAIN ] MGRE: Full history saved. [2025-10-20T19:53:38.280787] [INFO ] [MAIN ] Update Progress: 1346/2367 symbols completed. MGRE took 22.5s. Estimated remaining: 2h 11m 39s. Rate: 7.8 symbols/min [2025-10-20T19:53:38.449619] [INFO ] [MAIN ] MHF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:38.460277] [DEBUG] [LOCAL ] MHF: Loaded 9100 existing records from primary storage. [2025-10-20T19:53:38.460475] [INFO ] [MAIN ] MHF: Loaded 9100 existing records. [2025-10-20T19:53:38.613336] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T19:53:38.650476] [WARN ] [MAIN ] MHF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:38.650630] [INFO ] [MAIN ] MHF: Wiping existing primary data before full refresh. [2025-10-20T19:53:38.650708] [INFO ] [LOCAL ] MHF: Starting deletion of primary data. [2025-10-20T19:53:38.650848] [DEBUG] [LOCAL ] MHF: Deleted primary data file. [2025-10-20T19:53:38.650885] [INFO ] [MAIN ] MHF: Starting full primary refresh. [2025-10-20T19:53:38.656443] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-05. [2025-10-20T19:53:39.002800] [INFO ] [REMOTE ] MHF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:39.451384] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-03. [2025-10-20T19:53:40.251366] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-01. [2025-10-20T19:53:40.322535] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T19:53:40.488249] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T19:53:40.699706] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T19:53:41.051383] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-02. [2025-10-20T19:53:41.851445] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-06. [2025-10-20T19:53:41.873833] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:53:42.651407] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-06. [2025-10-20T19:53:43.100953] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T19:53:43.451413] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-07. [2025-10-20T19:53:43.532871] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T19:53:43.953995] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T19:53:44.251403] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-08. [2025-10-20T19:53:44.685684] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T19:53:45.051417] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-09. [2025-10-20T19:53:45.487170] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-09 (0.44 seconds). [2025-10-20T19:53:45.851387] [INFO ] [REMOTE ] MGRD: Fetching primary data for 2025-10. [2025-10-20T19:53:46.363500] [INFO ] [REMOTE ] MGRD: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T19:53:46.364381] [INFO ] [REMOTE ] MGRD: Full fetch complete (3320 records). [2025-10-20T19:53:46.364452] [DEBUG] [VALIDATOR ] MGRD: Validating primary data. [2025-10-20T19:53:46.656521] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-04. [2025-10-20T19:53:46.735334] [DEBUG] [VALIDATOR ] MGRD: All expected trading days are present. [2025-10-20T19:53:46.735425] [DEBUG] [VALIDATOR ] MGRD: Primary data validation passed. [2025-10-20T19:53:46.735573] [INFO ] [MAIN ] MGRD: Fetched primary data is valid. [2025-10-20T19:53:46.735672] [DEBUG] [LOCAL ] MGRD: Saving primary data. [2025-10-20T19:53:46.736115] [DEBUG] [LOCAL ] MGRD: Trimmed 3320 records to 2218 (keeping 200 days of data) [2025-10-20T19:53:46.751186] [DEBUG] [LOCAL ] MGRD: Primary data saved (2218 records). [2025-10-20T19:53:46.751302] [INFO ] [MAIN ] MGRD: Full history saved. [2025-10-20T19:53:46.751377] [INFO ] [MAIN ] Update Progress: 1347/2367 symbols completed. MGRD took 36.0s. Estimated remaining: 2h 11m 32s. Rate: 7.8 symbols/min [2025-10-20T19:53:46.926359] [INFO ] [MAIN ] MHK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:53:46.970091] [DEBUG] [LOCAL ] MHK: Loaded 42681 existing records from primary storage. [2025-10-20T19:53:46.970711] [INFO ] [MAIN ] MHK: Loaded 42681 existing records. [2025-10-20T19:53:47.160253] [WARN ] [MAIN ] MHK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:53:47.160496] [INFO ] [MAIN ] MHK: Wiping existing primary data before full refresh. [2025-10-20T19:53:47.160555] [INFO ] [LOCAL ] MHK: Starting deletion of primary data. [2025-10-20T19:53:47.161171] [DEBUG] [LOCAL ] MHK: Deleted primary data file. [2025-10-20T19:53:47.161231] [INFO ] [MAIN ] MHK: Starting full primary refresh. [2025-10-20T19:53:47.453114] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-07. [2025-10-20T19:53:47.535056] [INFO ] [REMOTE ] MHK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:53:47.661975] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T19:53:48.251368] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-05. [2025-10-20T19:53:49.051408] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-01. [2025-10-20T19:53:49.358779] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:53:49.366380] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T19:53:49.851418] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-08. [2025-10-20T19:53:50.570390] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T19:53:50.651420] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-02. [2025-10-20T19:53:51.451396] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-06. [2025-10-20T19:53:51.554574] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T19:53:51.785036] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T19:53:52.251405] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-09. [2025-10-20T19:53:52.265486] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T19:53:53.051365] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-03. [2025-10-20T19:53:53.851374] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-03. [2025-10-20T19:53:54.533303] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T19:53:54.547337] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T19:53:54.651343] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-04. [2025-10-20T19:53:55.156025] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-09 (2.90 seconds). [2025-10-20T19:53:55.433201] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T19:53:55.451346] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-07. [2025-10-20T19:53:56.216707] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T19:53:56.251333] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-08. [2025-10-20T19:53:57.028317] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T19:53:57.051382] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-05. [2025-10-20T19:53:57.734299] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T19:53:57.851377] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-09. [2025-10-20T19:53:58.651418] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-06. [2025-10-20T19:53:58.755465] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T19:53:59.267219] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T19:53:59.451423] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-07. [2025-10-20T19:54:00.188848] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T19:54:00.251445] [INFO ] [REMOTE ] MGY: Fetching primary data for 2025-10. [2025-10-20T19:54:01.051419] [INFO ] [REMOTE ] MHD: Fetching primary data for 2025-10. [2025-10-20T19:54:01.851433] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-08. [2025-10-20T19:54:02.194897] [INFO ] [REMOTE ] MGY: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T19:54:02.203716] [INFO ] [REMOTE ] MGY: Full fetch complete (75945 records). [2025-10-20T19:54:02.203945] [DEBUG] [VALIDATOR ] MGY: Validating primary data. [2025-10-20T19:54:02.321962] [INFO ] [REMOTE ] MHD: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T19:54:02.351128] [INFO ] [REMOTE ] MHD: Full fetch complete (18138 records). [2025-10-20T19:54:02.374878] [DEBUG] [VALIDATOR ] MHD: Validating primary data. [2025-10-20T19:54:02.656690] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-04. [2025-10-20T19:54:02.663355] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T19:54:03.042235] [DEBUG] [VALIDATOR ] MGY: All expected trading days are present. [2025-10-20T19:54:03.052681] [DEBUG] [VALIDATOR ] MGY: Primary data validation passed. [2025-10-20T19:54:03.083160] [INFO ] [MAIN ] MGY: Fetched primary data is valid. [2025-10-20T19:54:03.122676] [DEBUG] [LOCAL ] MGY: Saving primary data. [2025-10-20T19:54:03.123483] [DEBUG] [VALIDATOR ] MHD: All expected trading days are present. [2025-10-20T19:54:03.123918] [DEBUG] [VALIDATOR ] MHD: Primary data validation passed. [2025-10-20T19:54:03.124182] [INFO ] [MAIN ] MHD: Fetched primary data is valid. [2025-10-20T19:54:03.124257] [DEBUG] [LOCAL ] MHD: Saving primary data. [2025-10-20T19:54:03.125932] [DEBUG] [LOCAL ] MHD: Trimmed 18138 records to 12053 (keeping 200 days of data) [2025-10-20T19:54:03.127179] [DEBUG] [LOCAL ] MGY: Trimmed 75945 records to 52437 (keeping 200 days of data) [2025-10-20T19:54:03.319719] [DEBUG] [LOCAL ] MHD: Primary data saved (12053 records). [2025-10-20T19:54:03.327789] [INFO ] [MAIN ] MHD: Full history saved. [2025-10-20T19:54:03.327952] [INFO ] [MAIN ] Update Progress: 1348/2367 symbols completed. MHD took 28.5s. Estimated remaining: 2h 11m 31s. Rate: 7.7 symbols/min [2025-10-20T19:54:03.488161] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-09. [2025-10-20T19:54:03.719278] [INFO ] [MAIN ] MHLA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:03.738083] [DEBUG] [LOCAL ] MHLA: No existing data file found in primary storage. [2025-10-20T19:54:03.740945] [WARN ] [MAIN ] MHLA: No existing data found, performing full refresh instead. [2025-10-20T19:54:03.745709] [INFO ] [MAIN ] MHLA: Wiping existing primary data before full refresh. [2025-10-20T19:54:03.752569] [INFO ] [LOCAL ] MHLA: Starting deletion of primary data. [2025-10-20T19:54:03.763707] [DEBUG] [LOCAL ] MHLA: No primary data file found to delete. [2025-10-20T19:54:03.766956] [INFO ] [MAIN ] MHLA: Starting full primary refresh. [2025-10-20T19:54:03.766823] [DEBUG] [LOCAL ] MGY: Primary data saved (52437 records). [2025-10-20T19:54:03.772371] [INFO ] [MAIN ] MGY: Full history saved. [2025-10-20T19:54:03.772573] [INFO ] [MAIN ] Update Progress: 1349/2367 symbols completed. MGY took 40.2s. Estimated remaining: 2h 11m 18s. Rate: 7.8 symbols/min [2025-10-20T19:54:04.155508] [INFO ] [MAIN ] MHN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:04.335583] [DEBUG] [LOCAL ] MHN: Loaded 7574 existing records from primary storage. [2025-10-20T19:54:04.382284] [INFO ] [REMOTE ] MHLA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:04.382945] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-01. [2025-10-20T19:54:04.383129] [INFO ] [MAIN ] MHN: Loaded 7574 existing records. [2025-10-20T19:54:04.484365] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T19:54:04.634158] [WARN ] [MAIN ] MHN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:04.636610] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T19:54:04.646311] [INFO ] [MAIN ] MHN: Wiping existing primary data before full refresh. [2025-10-20T19:54:04.647100] [INFO ] [LOCAL ] MHN: Starting deletion of primary data. [2025-10-20T19:54:04.647870] [DEBUG] [LOCAL ] MHN: Deleted primary data file. [2025-10-20T19:54:04.647939] [INFO ] [MAIN ] MHN: Starting full primary refresh. [2025-10-20T19:54:04.773043] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T19:54:05.055749] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-05. [2025-10-20T19:54:05.055629] [INFO ] [REMOTE ] MHN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:05.851378] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-01. [2025-10-20T19:54:06.566853] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T19:54:06.651428] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-02. [2025-10-20T19:54:06.684668] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T19:54:07.020672] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T19:54:07.451375] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-02. [2025-10-20T19:54:08.170912] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T19:54:08.251388] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-03. [2025-10-20T19:54:08.623894] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T19:54:09.051362] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-06. [2025-10-20T19:54:09.851381] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-04. [2025-10-20T19:54:10.266748] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T19:54:10.651410] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-03. [2025-10-20T19:54:11.168862] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-06 (2.12 seconds). [2025-10-20T19:54:11.374925] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T19:54:11.451336] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-04. [2025-10-20T19:54:12.251408] [INFO ] [REMOTE ] MHF: Fetching primary data for 2025-10. [2025-10-20T19:54:12.277357] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T19:54:13.051359] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-05. [2025-10-20T19:54:13.109199] [INFO ] [REMOTE ] MHF: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:54:13.111154] [INFO ] [REMOTE ] MHF: Full fetch complete (12829 records). [2025-10-20T19:54:13.111250] [DEBUG] [VALIDATOR ] MHF: Validating primary data. [2025-10-20T19:54:13.464804] [DEBUG] [VALIDATOR ] MHF: All expected trading days are present. [2025-10-20T19:54:13.464875] [DEBUG] [VALIDATOR ] MHF: Primary data validation passed. [2025-10-20T19:54:13.465155] [INFO ] [MAIN ] MHF: Fetched primary data is valid. [2025-10-20T19:54:13.465192] [DEBUG] [LOCAL ] MHF: Saving primary data. [2025-10-20T19:54:13.465627] [DEBUG] [LOCAL ] MHF: Trimmed 12829 records to 9023 (keeping 200 days of data) [2025-10-20T19:54:13.518963] [DEBUG] [LOCAL ] MHF: Primary data saved (9023 records). [2025-10-20T19:54:13.519083] [INFO ] [MAIN ] MHF: Full history saved. [2025-10-20T19:54:13.519155] [INFO ] [MAIN ] Update Progress: 1350/2367 symbols completed. MHF took 35.2s. Estimated remaining: 2h 11m 12s. Rate: 7.8 symbols/min [2025-10-20T19:54:13.690543] [INFO ] [MAIN ] MHO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:13.714689] [DEBUG] [LOCAL ] MHO: Loaded 24095 existing records from primary storage. [2025-10-20T19:54:13.714933] [INFO ] [MAIN ] MHO: Loaded 24095 existing records. [2025-10-20T19:54:13.797158] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T19:54:13.864722] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-06. [2025-10-20T19:54:13.906490] [WARN ] [MAIN ] MHO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:13.907210] [INFO ] [MAIN ] MHO: Wiping existing primary data before full refresh. [2025-10-20T19:54:13.907267] [INFO ] [LOCAL ] MHO: Starting deletion of primary data. [2025-10-20T19:54:13.907468] [DEBUG] [LOCAL ] MHO: Deleted primary data file. [2025-10-20T19:54:13.907524] [INFO ] [MAIN ] MHO: Starting full primary refresh. [2025-10-20T19:54:14.262629] [INFO ] [REMOTE ] MHO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:14.508249] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T19:54:14.651431] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-07. [2025-10-20T19:54:15.379312] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T19:54:15.451395] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-08. [2025-10-20T19:54:16.159348] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T19:54:16.251430] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-07. [2025-10-20T19:54:17.051424] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-01. [2025-10-20T19:54:17.851392] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-09. [2025-10-20T19:54:17.906539] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T19:54:18.247613] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T19:54:18.466864] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T19:54:18.651406] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-02. [2025-10-20T19:54:19.451396] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-05. [2025-10-20T19:54:19.938996] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T19:54:19.982780] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T19:54:20.251446] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-08. [2025-10-20T19:54:21.051382] [INFO ] [REMOTE ] MHN: Fetching primary data for 2025-10. [2025-10-20T19:54:21.719905] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T19:54:21.736423] [INFO ] [REMOTE ] MHN: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T19:54:21.737860] [INFO ] [REMOTE ] MHN: Full fetch complete (11438 records). [2025-10-20T19:54:21.737928] [DEBUG] [VALIDATOR ] MHN: Validating primary data. [2025-10-20T19:54:21.856453] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-03. [2025-10-20T19:54:22.100510] [DEBUG] [VALIDATOR ] MHN: All expected trading days are present. [2025-10-20T19:54:22.100599] [DEBUG] [VALIDATOR ] MHN: Primary data validation passed. [2025-10-20T19:54:22.100900] [INFO ] [MAIN ] MHN: Fetched primary data is valid. [2025-10-20T19:54:22.100955] [DEBUG] [LOCAL ] MHN: Saving primary data. [2025-10-20T19:54:22.101381] [DEBUG] [LOCAL ] MHN: Trimmed 11438 records to 7475 (keeping 200 days of data) [2025-10-20T19:54:22.145336] [DEBUG] [LOCAL ] MHN: Primary data saved (7475 records). [2025-10-20T19:54:22.145475] [INFO ] [MAIN ] MHN: Full history saved. [2025-10-20T19:54:22.145551] [INFO ] [MAIN ] Update Progress: 1351/2367 symbols completed. MHN took 18.3s. Estimated remaining: 2h 11m 5s. Rate: 7.8 symbols/min [2025-10-20T19:54:22.313170] [INFO ] [MAIN ] MIAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:22.313340] [DEBUG] [LOCAL ] MIAX: No existing data file found in primary storage. [2025-10-20T19:54:22.313378] [WARN ] [MAIN ] MIAX: No existing data found, performing full refresh instead. [2025-10-20T19:54:22.313486] [INFO ] [MAIN ] MIAX: Wiping existing primary data before full refresh. [2025-10-20T19:54:22.313527] [INFO ] [LOCAL ] MIAX: Starting deletion of primary data. [2025-10-20T19:54:22.313568] [DEBUG] [LOCAL ] MIAX: No primary data file found to delete. [2025-10-20T19:54:22.313602] [INFO ] [MAIN ] MIAX: Starting full primary refresh. [2025-10-20T19:54:22.656681] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-09. [2025-10-20T19:54:22.661508] [INFO ] [REMOTE ] MIAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:23.193893] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T19:54:23.451363] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-04. [2025-10-20T19:54:24.044606] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T19:54:24.251347] [INFO ] [REMOTE ] MIAX: Fetching primary data for 2025-01. [2025-10-20T19:54:24.620532] [DEBUG] [REMOTE ] MIAX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:54:24.621097] [WARN ] [REMOTE ] MIAX: No data returned for month 2025-01 [2025-10-20T19:54:24.621214] [FATAL] [REMOTE ] MIAX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:54:24.621303] [FATAL] [MAIN ] MIAX: Failed to fetch primary data. [2025-10-20T19:54:24.621389] [WARN ] [MAIN ] MIAX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:54:24.621479] [INFO ] [LOCAL ] MIAX: Starting deletion of primary data. [2025-10-20T19:54:24.621565] [DEBUG] [LOCAL ] MIAX: No primary data file found to delete. [2025-10-20T19:54:24.621609] [INFO ] [LOCAL ] MIAX: Starting deletion of derivative data. [2025-10-20T19:54:24.621704] [DEBUG] [LOCAL ] MIAX: No derivative data file found to delete. [2025-10-20T19:54:24.621780] [INFO ] [MAIN ] Update Progress: 1352/2367 symbols completed. MIAX took 2.5s. Estimated remaining: 2h 10m 53s. Rate: 7.8 symbols/min [2025-10-20T19:54:24.816244] [INFO ] [MAIN ] MIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:24.823192] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T19:54:24.841486] [DEBUG] [LOCAL ] MIN: Loaded 14068 existing records from primary storage. [2025-10-20T19:54:24.841749] [INFO ] [MAIN ] MIN: Loaded 14068 existing records. [2025-10-20T19:54:25.022288] [WARN ] [MAIN ] MIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:25.022543] [INFO ] [MAIN ] MIN: Wiping existing primary data before full refresh. [2025-10-20T19:54:25.022593] [INFO ] [LOCAL ] MIN: Starting deletion of primary data. [2025-10-20T19:54:25.022782] [DEBUG] [LOCAL ] MIN: Deleted primary data file. [2025-10-20T19:54:25.022879] [INFO ] [MAIN ] MIN: Starting full primary refresh. [2025-10-20T19:54:25.056439] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-06. [2025-10-20T19:54:25.382926] [INFO ] [REMOTE ] MIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:25.458079] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T19:54:25.851374] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-07. [2025-10-20T19:54:26.082130] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-07 (0.23 seconds). [2025-10-20T19:54:26.651415] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-08. [2025-10-20T19:54:26.982675] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-08 (0.33 seconds). [2025-10-20T19:54:27.451371] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-01. [2025-10-20T19:54:28.200211] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T19:54:28.251367] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-02. [2025-10-20T19:54:29.034711] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T19:54:29.051348] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-03. [2025-10-20T19:54:29.802485] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T19:54:29.851449] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-05. [2025-10-20T19:54:30.651363] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-09. [2025-10-20T19:54:30.924283] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T19:54:31.004661] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-10-20T19:54:31.451385] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-06. [2025-10-20T19:54:32.251380] [INFO ] [REMOTE ] MHK: Fetching primary data for 2025-10. [2025-10-20T19:54:32.710939] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T19:54:33.051441] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-04. [2025-10-20T19:54:33.758822] [INFO ] [REMOTE ] MHK: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T19:54:33.764739] [INFO ] [REMOTE ] MHK: Full fetch complete (61266 records). [2025-10-20T19:54:33.764843] [DEBUG] [VALIDATOR ] MHK: Validating primary data. [2025-10-20T19:54:33.856521] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-07. [2025-10-20T19:54:33.883043] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T19:54:34.185017] [DEBUG] [VALIDATOR ] MHK: All expected trading days are present. [2025-10-20T19:54:34.185151] [DEBUG] [VALIDATOR ] MHK: Primary data validation passed. [2025-10-20T19:54:34.186369] [INFO ] [MAIN ] MHK: Fetched primary data is valid. [2025-10-20T19:54:34.186436] [DEBUG] [LOCAL ] MHK: Saving primary data. [2025-10-20T19:54:34.188139] [DEBUG] [LOCAL ] MHK: Trimmed 61266 records to 42112 (keeping 200 days of data) [2025-10-20T19:54:34.464730] [DEBUG] [LOCAL ] MHK: Primary data saved (42112 records). [2025-10-20T19:54:34.464930] [INFO ] [MAIN ] MHK: Full history saved. [2025-10-20T19:54:34.465037] [INFO ] [MAIN ] Update Progress: 1353/2367 symbols completed. MHK took 47.7s. Estimated remaining: 2h 10m 47s. Rate: 7.8 symbols/min [2025-10-20T19:54:34.642020] [INFO ] [MAIN ] MIR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:34.651397] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-05. [2025-10-20T19:54:34.701757] [DEBUG] [LOCAL ] MIR: Loaded 52051 existing records from primary storage. [2025-10-20T19:54:34.702222] [INFO ] [MAIN ] MIR: Loaded 52051 existing records. [2025-10-20T19:54:34.887392] [WARN ] [MAIN ] MIR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:34.887699] [INFO ] [MAIN ] MIR: Wiping existing primary data before full refresh. [2025-10-20T19:54:34.887762] [INFO ] [LOCAL ] MIR: Starting deletion of primary data. [2025-10-20T19:54:34.887993] [DEBUG] [LOCAL ] MIR: Deleted primary data file. [2025-10-20T19:54:34.888038] [INFO ] [MAIN ] MIR: Starting full primary refresh. [2025-10-20T19:54:35.260948] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:54:35.274571] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:54:35.318458] [INFO ] [REMOTE ] MIR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:35.451366] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-08. [2025-10-20T19:54:36.251363] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-01. [2025-10-20T19:54:36.655294] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T19:54:37.051413] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-09. [2025-10-20T19:54:37.856507] [INFO ] [REMOTE ] MHLA: Fetching primary data for 2025-10. [2025-10-20T19:54:37.869771] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T19:54:38.150934] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T19:54:38.302497] [INFO ] [REMOTE ] MHLA: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T19:54:38.303255] [INFO ] [REMOTE ] MHLA: Full fetch complete (1510 records). [2025-10-20T19:54:38.303328] [DEBUG] [VALIDATOR ] MHLA: Validating primary data. [2025-10-20T19:54:38.491567] [WARN ] [VALIDATOR ] MHLA: Validation failed - insufficient historical data. Have 179 unique days, need 200 days [2025-10-20T19:54:38.491820] [FATAL] [MAIN ] MHLA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:54:38.491906] [WARN ] [MAIN ] MHLA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:54:38.491997] [INFO ] [LOCAL ] MHLA: Starting deletion of primary data. [2025-10-20T19:54:38.492078] [DEBUG] [LOCAL ] MHLA: No primary data file found to delete. [2025-10-20T19:54:38.492132] [INFO ] [LOCAL ] MHLA: Starting deletion of derivative data. [2025-10-20T19:54:38.492173] [DEBUG] [LOCAL ] MHLA: No derivative data file found to delete. [2025-10-20T19:54:38.492228] [INFO ] [MAIN ] Update Progress: 1354/2367 symbols completed. MHLA took 35.2s. Estimated remaining: 2h 10m 36s. Rate: 7.8 symbols/min [2025-10-20T19:54:38.669013] [INFO ] [MAIN ] MITN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:38.664878] [INFO ] [REMOTE ] MHO: Fetching primary data for 2025-10. [2025-10-20T19:54:38.670163] [DEBUG] [LOCAL ] MITN: No existing data file found in primary storage. [2025-10-20T19:54:38.670235] [WARN ] [MAIN ] MITN: No existing data found, performing full refresh instead. [2025-10-20T19:54:38.670368] [INFO ] [MAIN ] MITN: Wiping existing primary data before full refresh. [2025-10-20T19:54:38.670415] [INFO ] [LOCAL ] MITN: Starting deletion of primary data. [2025-10-20T19:54:38.670460] [DEBUG] [LOCAL ] MITN: No primary data file found to delete. [2025-10-20T19:54:38.670493] [INFO ] [MAIN ] MITN: Starting full primary refresh. [2025-10-20T19:54:39.033637] [INFO ] [REMOTE ] MITN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:39.451398] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-02. [2025-10-20T19:54:39.822088] [INFO ] [REMOTE ] MHO: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T19:54:39.825470] [INFO ] [REMOTE ] MHO: Full fetch complete (36564 records). [2025-10-20T19:54:39.825560] [DEBUG] [VALIDATOR ] MHO: Validating primary data. [2025-10-20T19:54:40.201372] [DEBUG] [VALIDATOR ] MHO: All expected trading days are present. [2025-10-20T19:54:40.201462] [DEBUG] [VALIDATOR ] MHO: Primary data validation passed. [2025-10-20T19:54:40.202203] [INFO ] [MAIN ] MHO: Fetched primary data is valid. [2025-10-20T19:54:40.202266] [DEBUG] [LOCAL ] MHO: Saving primary data. [2025-10-20T19:54:40.203220] [DEBUG] [LOCAL ] MHO: Trimmed 36564 records to 23788 (keeping 200 days of data) [2025-10-20T19:54:40.253883] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-01. [2025-10-20T19:54:40.344182] [DEBUG] [LOCAL ] MHO: Primary data saved (23788 records). [2025-10-20T19:54:40.344304] [INFO ] [MAIN ] MHO: Full history saved. [2025-10-20T19:54:40.344376] [INFO ] [MAIN ] Update Progress: 1355/2367 symbols completed. MHO took 26.8s. Estimated remaining: 2h 10m 24s. Rate: 7.8 symbols/min [2025-10-20T19:54:40.512730] [INFO ] [MAIN ] MITP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:40.517186] [DEBUG] [LOCAL ] MITP: Loaded 2336 existing records from primary storage. [2025-10-20T19:54:40.517328] [INFO ] [MAIN ] MITP: Loaded 2336 existing records. [2025-10-20T19:54:40.697230] [WARN ] [MAIN ] MITP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:40.700077] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-01 (0.45 seconds). [2025-10-20T19:54:40.699957] [INFO ] [MAIN ] MITP: Wiping existing primary data before full refresh. [2025-10-20T19:54:40.701236] [INFO ] [LOCAL ] MITP: Starting deletion of primary data. [2025-10-20T19:54:40.701410] [DEBUG] [LOCAL ] MITP: Deleted primary data file. [2025-10-20T19:54:40.701466] [INFO ] [MAIN ] MITP: Starting full primary refresh. [2025-10-20T19:54:41.053089] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-02. [2025-10-20T19:54:41.059442] [INFO ] [REMOTE ] MITP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:41.087494] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T19:54:41.499254] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T19:54:41.851418] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-06. [2025-10-20T19:54:42.651406] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-03. [2025-10-20T19:54:42.840193] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T19:54:43.200736] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T19:54:43.451377] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-01. [2025-10-20T19:54:43.875025] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T19:54:44.251379] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-03. [2025-10-20T19:54:45.051399] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-04. [2025-10-20T19:54:45.596364] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T19:54:45.851438] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-07. [2025-10-20T19:54:45.896226] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T19:54:46.651379] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-05. [2025-10-20T19:54:46.993283] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T19:54:47.047902] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T19:54:47.451407] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-04. [2025-10-20T19:54:48.251398] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-08. [2025-10-20T19:54:49.051404] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-06. [2025-10-20T19:54:49.094472] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T19:54:49.165912] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T19:54:49.386091] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T19:54:49.851366] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-07. [2025-10-20T19:54:50.217741] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T19:54:50.651396] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-09. [2025-10-20T19:54:51.451456] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-08. [2025-10-20T19:54:51.482289] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T19:54:51.889025] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T19:54:52.251482] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-05. [2025-10-20T19:54:53.051466] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-09. [2025-10-20T19:54:53.396007] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-10-20T19:54:53.851436] [INFO ] [REMOTE ] MIN: Fetching primary data for 2025-10. [2025-10-20T19:54:53.932614] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:54:54.651377] [INFO ] [REMOTE ] MITN: Fetching primary data for 2025-10. [2025-10-20T19:54:54.859883] [INFO ] [REMOTE ] MIN: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T19:54:54.862736] [INFO ] [REMOTE ] MIN: Full fetch complete (17684 records). [2025-10-20T19:54:54.862841] [DEBUG] [VALIDATOR ] MIN: Validating primary data. [2025-10-20T19:54:55.144153] [INFO ] [REMOTE ] MITN: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T19:54:55.172406] [INFO ] [REMOTE ] MITN: Full fetch complete (2364 records). [2025-10-20T19:54:55.175277] [DEBUG] [VALIDATOR ] MITN: Validating primary data. [2025-10-20T19:54:55.400204] [DEBUG] [VALIDATOR ] MIN: All expected trading days are present. [2025-10-20T19:54:55.428282] [DEBUG] [VALIDATOR ] MIN: Primary data validation passed. [2025-10-20T19:54:55.454009] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-02. [2025-10-20T19:54:55.453926] [INFO ] [MAIN ] MIN: Fetched primary data is valid. [2025-10-20T19:54:55.462167] [DEBUG] [LOCAL ] MIN: Saving primary data. [2025-10-20T19:54:55.460990] [WARN ] [VALIDATOR ] MITN: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:54:55.462480] [FATAL] [MAIN ] MITN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:54:55.463251] [DEBUG] [LOCAL ] MIN: Trimmed 17684 records to 13919 (keeping 200 days of data) [2025-10-20T19:54:55.463325] [WARN ] [MAIN ] MITN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:54:55.463677] [INFO ] [LOCAL ] MITN: Starting deletion of primary data. [2025-10-20T19:54:55.463796] [DEBUG] [LOCAL ] MITN: No primary data file found to delete. [2025-10-20T19:54:55.463893] [INFO ] [LOCAL ] MITN: Starting deletion of derivative data. [2025-10-20T19:54:55.464200] [DEBUG] [LOCAL ] MITN: No derivative data file found to delete. [2025-10-20T19:54:55.464309] [INFO ] [MAIN ] Update Progress: 1356/2367 symbols completed. MITN took 17.0s. Estimated remaining: 2h 10m 22s. Rate: 7.8 symbols/min [2025-10-20T19:54:55.715178] [DEBUG] [LOCAL ] MIN: Primary data saved (13919 records). [2025-10-20T19:54:55.730568] [INFO ] [MAIN ] MIN: Full history saved. [2025-10-20T19:54:55.733676] [INFO ] [MAIN ] Update Progress: 1357/2367 symbols completed. MIN took 31.1s. Estimated remaining: 2h 10m 9s. Rate: 7.8 symbols/min [2025-10-20T19:54:55.733484] [INFO ] [MAIN ] MITT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:54:55.755075] [DEBUG] [LOCAL ] MITT: Loaded 22934 existing records from primary storage. [2025-10-20T19:54:55.755348] [INFO ] [MAIN ] MITT: Loaded 22934 existing records. [2025-10-20T19:54:55.951130] [WARN ] [MAIN ] MITT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:54:55.954149] [INFO ] [MAIN ] MITT: Wiping existing primary data before full refresh. [2025-10-20T19:54:55.954040] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T19:54:55.955496] [INFO ] [LOCAL ] MITT: Starting deletion of primary data. [2025-10-20T19:54:55.955714] [DEBUG] [LOCAL ] MITT: Deleted primary data file. [2025-10-20T19:54:55.955768] [INFO ] [MAIN ] MITT: Starting full primary refresh. [2025-10-20T19:54:56.256808] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-03. [2025-10-20T19:54:56.323362] [INFO ] [REMOTE ] MITT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:54:56.741883] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T19:54:57.051443] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-06. [2025-10-20T19:54:57.851456] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-04. [2025-10-20T19:54:58.417982] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:54:58.651382] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-05. [2025-10-20T19:54:58.725144] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T19:54:59.300817] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T19:54:59.451478] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-01. [2025-10-20T19:55:00.251394] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-06. [2025-10-20T19:55:00.600636] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T19:55:00.836574] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T19:55:01.051422] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-02. [2025-10-20T19:55:01.851412] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-07. [2025-10-20T19:55:02.187050] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T19:55:02.351333] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T19:55:02.651431] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-08. [2025-10-20T19:55:03.109626] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T19:55:03.451415] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-09. [2025-10-20T19:55:03.876264] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-09 (0.42 seconds). [2025-10-20T19:55:04.251373] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-03. [2025-10-20T19:55:05.051359] [INFO ] [REMOTE ] MITP: Fetching primary data for 2025-10. [2025-10-20T19:55:05.466131] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T19:55:05.501183] [INFO ] [REMOTE ] MITP: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T19:55:05.501899] [INFO ] [REMOTE ] MITP: Full fetch complete (3256 records). [2025-10-20T19:55:05.501966] [DEBUG] [VALIDATOR ] MITP: Validating primary data. [2025-10-20T19:55:05.858952] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-07. [2025-10-20T19:55:05.859981] [DEBUG] [VALIDATOR ] MITP: All expected trading days are present. [2025-10-20T19:55:05.860045] [DEBUG] [VALIDATOR ] MITP: Primary data validation passed. [2025-10-20T19:55:05.860189] [INFO ] [MAIN ] MITP: Fetched primary data is valid. [2025-10-20T19:55:05.860252] [DEBUG] [LOCAL ] MITP: Saving primary data. [2025-10-20T19:55:05.860667] [DEBUG] [LOCAL ] MITP: Trimmed 3256 records to 2277 (keeping 200 days of data) [2025-10-20T19:55:05.874334] [DEBUG] [LOCAL ] MITP: Primary data saved (2277 records). [2025-10-20T19:55:05.874403] [INFO ] [MAIN ] MITP: Full history saved. [2025-10-20T19:55:05.874469] [INFO ] [MAIN ] Update Progress: 1358/2367 symbols completed. MITP took 25.5s. Estimated remaining: 2h 10m 3s. Rate: 7.8 symbols/min [2025-10-20T19:55:06.651373] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-04. [2025-10-20T19:55:07.779207] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T19:55:07.790847] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-08. [2025-10-20T19:55:08.034511] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T19:55:08.251393] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-05. [2025-10-20T19:55:09.302602] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T19:55:09.313511] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-09. [2025-10-20T19:55:09.382343] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T19:55:09.851373] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-06. [2025-10-20T19:55:10.973088] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T19:55:10.978608] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-07. [2025-10-20T19:55:11.098445] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T19:55:11.451403] [INFO ] [REMOTE ] MIR: Fetching primary data for 2025-10. [2025-10-20T19:55:12.177967] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T19:55:12.251333] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-08. [2025-10-20T19:55:13.319477] [INFO ] [REMOTE ] MIR: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T19:55:13.327257] [INFO ] [REMOTE ] MIR: Full fetch complete (73953 records). [2025-10-20T19:55:13.327398] [DEBUG] [VALIDATOR ] MIR: Validating primary data. [2025-10-20T19:55:13.519751] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T19:55:13.622155] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-09. [2025-10-20T19:55:13.772608] [DEBUG] [VALIDATOR ] MIR: All expected trading days are present. [2025-10-20T19:55:13.772755] [DEBUG] [VALIDATOR ] MIR: Primary data validation passed. [2025-10-20T19:55:13.773830] [INFO ] [MAIN ] MIR: Fetched primary data is valid. [2025-10-20T19:55:13.773892] [DEBUG] [LOCAL ] MIR: Saving primary data. [2025-10-20T19:55:13.775283] [DEBUG] [LOCAL ] MIR: Trimmed 73953 records to 51381 (keeping 200 days of data) [2025-10-20T19:55:14.064690] [DEBUG] [LOCAL ] MIR: Primary data saved (51381 records). [2025-10-20T19:55:14.064820] [INFO ] [MAIN ] MIR: Full history saved. [2025-10-20T19:55:14.064897] [INFO ] [MAIN ] Update Progress: 1359/2367 symbols completed. MIR took 39.6s. Estimated remaining: 2h 9m 55s. Rate: 7.8 symbols/min [2025-10-20T19:55:14.941573] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T19:55:14.946585] [INFO ] [REMOTE ] MITT: Fetching primary data for 2025-10. [2025-10-20T19:55:16.034791] [INFO ] [REMOTE ] MITT: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T19:55:16.038217] [INFO ] [REMOTE ] MITT: Full fetch complete (33740 records). [2025-10-20T19:55:16.038291] [DEBUG] [VALIDATOR ] MITT: Validating primary data. [2025-10-20T19:55:16.406555] [DEBUG] [VALIDATOR ] MITT: All expected trading days are present. [2025-10-20T19:55:16.406662] [DEBUG] [VALIDATOR ] MITT: Primary data validation passed. [2025-10-20T19:55:16.407338] [INFO ] [MAIN ] MITT: Fetched primary data is valid. [2025-10-20T19:55:16.407434] [DEBUG] [LOCAL ] MITT: Saving primary data. [2025-10-20T19:55:16.408588] [DEBUG] [LOCAL ] MITT: Trimmed 33740 records to 22548 (keeping 200 days of data) [2025-10-20T19:55:16.536719] [DEBUG] [LOCAL ] MITT: Primary data saved (22548 records). [2025-10-20T19:55:16.536858] [INFO ] [MAIN ] MITT: Full history saved. [2025-10-20T19:55:16.536948] [INFO ] [MAIN ] Update Progress: 1360/2367 symbols completed. MITT took 21.1s. Estimated remaining: 2h 9m 44s. Rate: 7.8 symbols/min [2025-10-20T19:55:17.130778] [INFO ] [MAIN ] MKC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:17.158829] [INFO ] [MAIN ] MKL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:17.261126] [INFO ] [MAIN ] MIY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:17.257477] [INFO ] [MAIN ] MLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:17.272921] [DEBUG] [LOCAL ] MIY: Loaded 6335 existing records from primary storage. [2025-10-20T19:55:17.273368] [INFO ] [MAIN ] MIY: Loaded 6335 existing records. [2025-10-20T19:55:17.441831] [DEBUG] [LOCAL ] MKL: Loaded 34581 existing records from primary storage. [2025-10-20T19:55:17.443890] [INFO ] [MAIN ] MKL: Loaded 34581 existing records. [2025-10-20T19:55:17.449864] [DEBUG] [LOCAL ] MKC: Loaded 53279 existing records from primary storage. [2025-10-20T19:55:17.460717] [INFO ] [MAIN ] MKC: Loaded 53279 existing records. [2025-10-20T19:55:17.612897] [WARN ] [MAIN ] MIY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:17.752116] [DEBUG] [LOCAL ] MLI: Loaded 44088 existing records from primary storage. [2025-10-20T19:55:17.751415] [INFO ] [MAIN ] MIY: Wiping existing primary data before full refresh. [2025-10-20T19:55:17.757147] [INFO ] [LOCAL ] MIY: Starting deletion of primary data. [2025-10-20T19:55:17.757308] [INFO ] [MAIN ] MLI: Loaded 44088 existing records. [2025-10-20T19:55:17.768226] [DEBUG] [LOCAL ] MIY: Deleted primary data file. [2025-10-20T19:55:17.773685] [INFO ] [MAIN ] MIY: Starting full primary refresh. [2025-10-20T19:55:18.029311] [WARN ] [MAIN ] MKC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:18.067321] [INFO ] [MAIN ] MKC: Wiping existing primary data before full refresh. [2025-10-20T19:55:18.080825] [WARN ] [MAIN ] MKL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:18.115690] [INFO ] [LOCAL ] MKC: Starting deletion of primary data. [2025-10-20T19:55:18.115600] [INFO ] [MAIN ] MKL: Wiping existing primary data before full refresh. [2025-10-20T19:55:18.126827] [INFO ] [LOCAL ] MKL: Starting deletion of primary data. [2025-10-20T19:55:18.137290] [DEBUG] [LOCAL ] MKC: Deleted primary data file. [2025-10-20T19:55:18.158463] [INFO ] [MAIN ] MKC: Starting full primary refresh. [2025-10-20T19:55:18.182198] [DEBUG] [LOCAL ] MKL: Deleted primary data file. [2025-10-20T19:55:18.216628] [INFO ] [MAIN ] MKL: Starting full primary refresh. [2025-10-20T19:55:18.275375] [WARN ] [MAIN ] MLI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:18.377390] [INFO ] [MAIN ] MLI: Wiping existing primary data before full refresh. [2025-10-20T19:55:18.453359] [INFO ] [LOCAL ] MLI: Starting deletion of primary data. [2025-10-20T19:55:18.522866] [DEBUG] [LOCAL ] MLI: Deleted primary data file. [2025-10-20T19:55:18.580914] [INFO ] [MAIN ] MLI: Starting full primary refresh. [2025-10-20T19:55:19.195263] [INFO ] [REMOTE ] MIY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:19.267333] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-01. [2025-10-20T19:55:19.495036] [INFO ] [REMOTE ] MKL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:19.504099] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-01. [2025-10-20T19:55:19.498615] [INFO ] [REMOTE ] MKC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:19.505200] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-01. [2025-10-20T19:55:19.670728] [INFO ] [REMOTE ] MLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:19.671048] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-01. [2025-10-20T19:55:19.980430] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:55:19.983482] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-02. [2025-10-20T19:55:20.630844] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T19:55:20.633272] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-03. [2025-10-20T19:55:20.881491] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T19:55:20.888502] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-02. [2025-10-20T19:55:21.152132] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T19:55:21.153947] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-04. [2025-10-20T19:55:21.178257] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T19:55:21.187191] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-02. [2025-10-20T19:55:21.386963] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T19:55:21.627133] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T19:55:21.851330] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-05. [2025-10-20T19:55:22.263971] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T19:55:22.292623] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T19:55:22.483993] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:55:22.651292] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-03. [2025-10-20T19:55:23.451379] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-03. [2025-10-20T19:55:24.038462] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T19:55:24.251362] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-02. [2025-10-20T19:55:25.051386] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-04. [2025-10-20T19:55:25.091106] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T19:55:25.851389] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-06. [2025-10-20T19:55:25.915752] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T19:55:26.174357] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T19:55:26.554907] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T19:55:26.651399] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-03. [2025-10-20T19:55:27.451378] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-07. [2025-10-20T19:55:28.193361] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T19:55:28.251346] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-08. [2025-10-20T19:55:28.446272] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T19:55:28.861699] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:55:29.051377] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-09. [2025-10-20T19:55:29.640889] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:55:29.851440] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-05. [2025-10-20T19:55:30.651389] [INFO ] [REMOTE ] MIY: Fetching primary data for 2025-10. [2025-10-20T19:55:31.334032] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T19:55:31.381117] [INFO ] [REMOTE ] MIY: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T19:55:31.382940] [INFO ] [REMOTE ] MIY: Full fetch complete (8680 records). [2025-10-20T19:55:31.383026] [DEBUG] [VALIDATOR ] MIY: Validating primary data. [2025-10-20T19:55:31.456564] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-04. [2025-10-20T19:55:31.752437] [DEBUG] [VALIDATOR ] MIY: All expected trading days are present. [2025-10-20T19:55:31.752510] [DEBUG] [VALIDATOR ] MIY: Primary data validation passed. [2025-10-20T19:55:31.752772] [INFO ] [MAIN ] MIY: Fetched primary data is valid. [2025-10-20T19:55:31.752834] [DEBUG] [LOCAL ] MIY: Saving primary data. [2025-10-20T19:55:31.753244] [DEBUG] [LOCAL ] MIY: Trimmed 8680 records to 6244 (keeping 200 days of data) [2025-10-20T19:55:31.790957] [DEBUG] [LOCAL ] MIY: Primary data saved (6244 records). [2025-10-20T19:55:31.791116] [INFO ] [MAIN ] MIY: Full history saved. [2025-10-20T19:55:31.791211] [INFO ] [MAIN ] Update Progress: 1361/2367 symbols completed. MIY took 15.3s. Estimated remaining: 2h 9m 41s. Rate: 7.8 symbols/min [2025-10-20T19:55:31.963170] [INFO ] [MAIN ] MLM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:31.992587] [DEBUG] [LOCAL ] MLM: Loaded 29602 existing records from primary storage. [2025-10-20T19:55:31.992941] [INFO ] [MAIN ] MLM: Loaded 29602 existing records. [2025-10-20T19:55:32.183375] [WARN ] [MAIN ] MLM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:32.183585] [INFO ] [MAIN ] MLM: Wiping existing primary data before full refresh. [2025-10-20T19:55:32.183634] [INFO ] [LOCAL ] MLM: Starting deletion of primary data. [2025-10-20T19:55:32.183879] [DEBUG] [LOCAL ] MLM: Deleted primary data file. [2025-10-20T19:55:32.183935] [INFO ] [MAIN ] MLM: Starting full primary refresh. [2025-10-20T19:55:32.256491] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-06. [2025-10-20T19:55:32.569731] [INFO ] [REMOTE ] MLM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:33.051465] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-04. [2025-10-20T19:55:33.255802] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T19:55:33.322538] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T19:55:33.851399] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-01. [2025-10-20T19:55:34.401743] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T19:55:34.651424] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-05. [2025-10-20T19:55:35.207130] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T19:55:35.451451] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-07. [2025-10-20T19:55:35.824199] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T19:55:36.251449] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-05. [2025-10-20T19:55:36.863349] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T19:55:37.051418] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-06. [2025-10-20T19:55:37.851354] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-02. [2025-10-20T19:55:37.927637] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T19:55:38.653772] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-08. [2025-10-20T19:55:38.656365] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T19:55:39.377072] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:55:39.451397] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-06. [2025-10-20T19:55:40.004139] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T19:55:40.251429] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-07. [2025-10-20T19:55:41.051406] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-03. [2025-10-20T19:55:41.191613] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T19:55:41.857227] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-09. [2025-10-20T19:55:41.875307] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T19:55:42.282921] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T19:55:42.651466] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-08. [2025-10-20T19:55:43.202480] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T19:55:43.451455] [INFO ] [REMOTE ] MKL: Fetching primary data for 2025-10. [2025-10-20T19:55:44.174210] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T19:55:44.251354] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-07. [2025-10-20T19:55:44.921876] [INFO ] [REMOTE ] MKL: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T19:55:44.927142] [INFO ] [REMOTE ] MKL: Full fetch complete (49966 records). [2025-10-20T19:55:44.927298] [DEBUG] [VALIDATOR ] MKL: Validating primary data. [2025-10-20T19:55:45.056533] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-09. [2025-10-20T19:55:45.339022] [DEBUG] [VALIDATOR ] MKL: All expected trading days are present. [2025-10-20T19:55:45.339110] [DEBUG] [VALIDATOR ] MKL: Primary data validation passed. [2025-10-20T19:55:45.339913] [INFO ] [MAIN ] MKL: Fetched primary data is valid. [2025-10-20T19:55:45.339977] [DEBUG] [LOCAL ] MKL: Saving primary data. [2025-10-20T19:55:45.341084] [DEBUG] [LOCAL ] MKL: Trimmed 49966 records to 34079 (keeping 200 days of data) [2025-10-20T19:55:45.549390] [DEBUG] [LOCAL ] MKL: Primary data saved (34079 records). [2025-10-20T19:55:45.549520] [INFO ] [MAIN ] MKL: Full history saved. [2025-10-20T19:55:45.549595] [INFO ] [MAIN ] Update Progress: 1362/2367 symbols completed. MKL took 29.0s. Estimated remaining: 2h 9m 38s. Rate: 7.8 symbols/min [2025-10-20T19:55:45.715697] [INFO ] [MAIN ] MLNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:45.741614] [DEBUG] [LOCAL ] MLNK: Loaded 27410 existing records from primary storage. [2025-10-20T19:55:45.741906] [INFO ] [MAIN ] MLNK: Loaded 27410 existing records. [2025-10-20T19:55:45.856556] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-04. [2025-10-20T19:55:45.919347] [WARN ] [MAIN ] MLNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:45.919804] [INFO ] [MAIN ] MLNK: Wiping existing primary data before full refresh. [2025-10-20T19:55:45.919863] [INFO ] [LOCAL ] MLNK: Starting deletion of primary data. [2025-10-20T19:55:45.920020] [DEBUG] [LOCAL ] MLNK: Deleted primary data file. [2025-10-20T19:55:45.920058] [INFO ] [MAIN ] MLNK: Starting full primary refresh. [2025-10-20T19:55:46.270875] [INFO ] [REMOTE ] MLNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:46.651354] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-01. [2025-10-20T19:55:46.736629] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T19:55:46.892291] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-07 (2.64 seconds). [2025-10-20T19:55:47.451436] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-08. [2025-10-20T19:55:47.497602] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:55:47.553911] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T19:55:48.251377] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-02. [2025-10-20T19:55:49.051368] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-05. [2025-10-20T19:55:49.273141] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T19:55:49.323988] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T19:55:49.851365] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-03. [2025-10-20T19:55:50.465428] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T19:55:50.651357] [INFO ] [REMOTE ] MLI: Fetching primary data for 2025-10. [2025-10-20T19:55:50.914490] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T19:55:51.451387] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-06. [2025-10-20T19:55:51.997161] [INFO ] [REMOTE ] MLI: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T19:55:52.003400] [INFO ] [REMOTE ] MLI: Full fetch complete (63174 records). [2025-10-20T19:55:52.003507] [DEBUG] [VALIDATOR ] MLI: Validating primary data. [2025-10-20T19:55:52.259447] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-04. [2025-10-20T19:55:52.395293] [DEBUG] [VALIDATOR ] MLI: All expected trading days are present. [2025-10-20T19:55:52.395368] [DEBUG] [VALIDATOR ] MLI: Primary data validation passed. [2025-10-20T19:55:52.396291] [INFO ] [MAIN ] MLI: Fetched primary data is valid. [2025-10-20T19:55:52.396352] [DEBUG] [LOCAL ] MLI: Saving primary data. [2025-10-20T19:55:52.397581] [DEBUG] [LOCAL ] MLI: Trimmed 63174 records to 43423 (keeping 200 days of data) [2025-10-20T19:55:52.657859] [DEBUG] [LOCAL ] MLI: Primary data saved (43423 records). [2025-10-20T19:55:52.657991] [INFO ] [MAIN ] MLI: Full history saved. [2025-10-20T19:55:52.658401] [INFO ] [MAIN ] Update Progress: 1363/2367 symbols completed. MLI took 36.1s. Estimated remaining: 2h 9m 30s. Rate: 7.8 symbols/min [2025-10-20T19:55:52.858527] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T19:55:52.857775] [INFO ] [MAIN ] MLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:55:52.871122] [DEBUG] [LOCAL ] MLP: Loaded 3109 existing records from primary storage. [2025-10-20T19:55:52.871316] [INFO ] [MAIN ] MLP: Loaded 3109 existing records. [2025-10-20T19:55:53.048912] [WARN ] [MAIN ] MLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:55:53.049147] [INFO ] [MAIN ] MLP: Wiping existing primary data before full refresh. [2025-10-20T19:55:53.049194] [INFO ] [LOCAL ] MLP: Starting deletion of primary data. [2025-10-20T19:55:53.049371] [DEBUG] [LOCAL ] MLP: Deleted primary data file. [2025-10-20T19:55:53.049425] [INFO ] [MAIN ] MLP: Starting full primary refresh. [2025-10-20T19:55:53.060951] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-07. [2025-10-20T19:55:53.440583] [INFO ] [REMOTE ] MLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:55:53.454746] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T19:55:53.851458] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-01. [2025-10-20T19:55:54.424193] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T19:55:54.459823] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T19:55:54.651407] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-08. [2025-10-20T19:55:55.451370] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-09. [2025-10-20T19:55:56.027729] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T19:55:56.253171] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-09. [2025-10-20T19:55:57.051357] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-02. [2025-10-20T19:55:57.173031] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T19:55:57.481758] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T19:55:57.523659] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T19:55:57.851374] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-05. [2025-10-20T19:55:58.651418] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-03. [2025-10-20T19:55:58.956874] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T19:55:59.141807] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T19:55:59.451387] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-04. [2025-10-20T19:56:00.019009] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T19:56:00.251362] [INFO ] [REMOTE ] MLM: Fetching primary data for 2025-10. [2025-10-20T19:56:01.051363] [INFO ] [REMOTE ] MKC: Fetching primary data for 2025-10. [2025-10-20T19:56:01.728248] [INFO ] [REMOTE ] MLM: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T19:56:01.732537] [INFO ] [REMOTE ] MLM: Full fetch complete (45263 records). [2025-10-20T19:56:01.732625] [DEBUG] [VALIDATOR ] MLM: Validating primary data. [2025-10-20T19:56:01.856481] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-06. [2025-10-20T19:56:02.115346] [DEBUG] [VALIDATOR ] MLM: All expected trading days are present. [2025-10-20T19:56:02.115421] [DEBUG] [VALIDATOR ] MLM: Primary data validation passed. [2025-10-20T19:56:02.116253] [INFO ] [MAIN ] MLM: Fetched primary data is valid. [2025-10-20T19:56:02.116325] [DEBUG] [LOCAL ] MLM: Saving primary data. [2025-10-20T19:56:02.117209] [DEBUG] [LOCAL ] MLM: Trimmed 45263 records to 29020 (keeping 200 days of data) [2025-10-20T19:56:02.306267] [DEBUG] [LOCAL ] MLM: Primary data saved (29020 records). [2025-10-20T19:56:02.306384] [INFO ] [MAIN ] MLM: Full history saved. [2025-10-20T19:56:02.306462] [INFO ] [MAIN ] Update Progress: 1364/2367 symbols completed. MLM took 30.5s. Estimated remaining: 2h 9m 24s. Rate: 7.8 symbols/min [2025-10-20T19:56:02.475360] [INFO ] [MAIN ] MLR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:02.489369] [DEBUG] [LOCAL ] MLR: Loaded 12610 existing records from primary storage. [2025-10-20T19:56:02.489610] [INFO ] [MAIN ] MLR: Loaded 12610 existing records. [2025-10-20T19:56:02.659695] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-05. [2025-10-20T19:56:02.664272] [WARN ] [MAIN ] MLR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:02.665228] [INFO ] [MAIN ] MLR: Wiping existing primary data before full refresh. [2025-10-20T19:56:02.665279] [INFO ] [LOCAL ] MLR: Starting deletion of primary data. [2025-10-20T19:56:02.665539] [DEBUG] [LOCAL ] MLR: Deleted primary data file. [2025-10-20T19:56:02.665622] [INFO ] [MAIN ] MLR: Starting full primary refresh. [2025-10-20T19:56:03.028894] [INFO ] [REMOTE ] MLR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:03.048162] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T19:56:03.076290] [INFO ] [REMOTE ] MKC: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T19:56:03.084689] [INFO ] [REMOTE ] MKC: Full fetch complete (75930 records). [2025-10-20T19:56:03.084809] [DEBUG] [VALIDATOR ] MKC: Validating primary data. [2025-10-20T19:56:03.266323] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T19:56:03.458862] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-07. [2025-10-20T19:56:03.478605] [DEBUG] [VALIDATOR ] MKC: All expected trading days are present. [2025-10-20T19:56:03.478696] [DEBUG] [VALIDATOR ] MKC: Primary data validation passed. [2025-10-20T19:56:03.479778] [INFO ] [MAIN ] MKC: Fetched primary data is valid. [2025-10-20T19:56:03.479979] [DEBUG] [LOCAL ] MKC: Saving primary data. [2025-10-20T19:56:03.481404] [DEBUG] [LOCAL ] MKC: Trimmed 75930 records to 52529 (keeping 200 days of data) [2025-10-20T19:56:03.795023] [DEBUG] [LOCAL ] MKC: Primary data saved (52529 records). [2025-10-20T19:56:03.795146] [INFO ] [MAIN ] MKC: Full history saved. [2025-10-20T19:56:03.795221] [INFO ] [MAIN ] Update Progress: 1365/2367 symbols completed. MKC took 47.3s. Estimated remaining: 2h 9m 11s. Rate: 7.8 symbols/min [2025-10-20T19:56:03.961603] [INFO ] [MAIN ] MMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:04.010347] [DEBUG] [LOCAL ] MMC: Loaded 52707 existing records from primary storage. [2025-10-20T19:56:04.010673] [INFO ] [MAIN ] MMC: Loaded 52707 existing records. [2025-10-20T19:56:04.184206] [WARN ] [MAIN ] MMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:04.184378] [INFO ] [MAIN ] MMC: Wiping existing primary data before full refresh. [2025-10-20T19:56:04.184426] [INFO ] [LOCAL ] MMC: Starting deletion of primary data. [2025-10-20T19:56:04.184633] [DEBUG] [LOCAL ] MMC: Deleted primary data file. [2025-10-20T19:56:04.184794] [INFO ] [MAIN ] MMC: Starting full primary refresh. [2025-10-20T19:56:04.259468] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-01. [2025-10-20T19:56:04.549600] [INFO ] [REMOTE ] MMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:04.607715] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T19:56:05.051430] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-08. [2025-10-20T19:56:05.300747] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T19:56:05.851434] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-01. [2025-10-20T19:56:06.651364] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-02. [2025-10-20T19:56:06.838559] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T19:56:07.451362] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-09. [2025-10-20T19:56:07.610499] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:56:07.647746] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T19:56:08.251370] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-02. [2025-10-20T19:56:09.038971] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T19:56:09.051344] [INFO ] [REMOTE ] MLNK: Fetching primary data for 2025-10. [2025-10-20T19:56:09.851385] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-03. [2025-10-20T19:56:09.986006] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T19:56:10.391110] [INFO ] [REMOTE ] MLNK: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T19:56:10.396538] [INFO ] [REMOTE ] MLNK: Full fetch complete (37516 records). [2025-10-20T19:56:10.396615] [DEBUG] [VALIDATOR ] MLNK: Validating primary data. [2025-10-20T19:56:10.656692] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-06. [2025-10-20T19:56:10.771087] [DEBUG] [VALIDATOR ] MLNK: All expected trading days are present. [2025-10-20T19:56:10.771160] [DEBUG] [VALIDATOR ] MLNK: Primary data validation passed. [2025-10-20T19:56:10.771854] [INFO ] [MAIN ] MLNK: Fetched primary data is valid. [2025-10-20T19:56:10.771914] [DEBUG] [LOCAL ] MLNK: Saving primary data. [2025-10-20T19:56:10.772689] [DEBUG] [LOCAL ] MLNK: Trimmed 37516 records to 27135 (keeping 200 days of data) [2025-10-20T19:56:10.931200] [DEBUG] [LOCAL ] MLNK: Primary data saved (27135 records). [2025-10-20T19:56:10.937351] [INFO ] [MAIN ] MLNK: Full history saved. [2025-10-20T19:56:10.937782] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T19:56:10.937921] [INFO ] [MAIN ] Update Progress: 1366/2367 symbols completed. MLNK took 25.4s. Estimated remaining: 2h 9m 3s. Rate: 7.8 symbols/min [2025-10-20T19:56:11.113950] [INFO ] [MAIN ] MMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:11.117872] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T19:56:11.130121] [DEBUG] [LOCAL ] MMD: Loaded 11032 existing records from primary storage. [2025-10-20T19:56:11.130301] [INFO ] [MAIN ] MMD: Loaded 11032 existing records. [2025-10-20T19:56:11.304380] [WARN ] [MAIN ] MMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:11.304603] [INFO ] [MAIN ] MMD: Wiping existing primary data before full refresh. [2025-10-20T19:56:11.304696] [INFO ] [LOCAL ] MMD: Starting deletion of primary data. [2025-10-20T19:56:11.304891] [DEBUG] [LOCAL ] MMD: Deleted primary data file. [2025-10-20T19:56:11.304947] [INFO ] [MAIN ] MMD: Starting full primary refresh. [2025-10-20T19:56:11.462902] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-04. [2025-10-20T19:56:11.676846] [INFO ] [REMOTE ] MMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:12.251405] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-03. [2025-10-20T19:56:12.574568] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:56:13.051376] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-07. [2025-10-20T19:56:13.557877] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T19:56:13.851350] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-05. [2025-10-20T19:56:14.210535] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:56:14.651375] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-08. [2025-10-20T19:56:14.749722] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T19:56:15.111511] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T19:56:15.451395] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-04. [2025-10-20T19:56:16.251399] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-09. [2025-10-20T19:56:16.835750] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T19:56:17.051394] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-01. [2025-10-20T19:56:17.313057] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T19:56:17.576263] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T19:56:17.851369] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-02. [2025-10-20T19:56:18.651380] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-06. [2025-10-20T19:56:18.785415] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T19:56:19.451375] [INFO ] [REMOTE ] MLP: Fetching primary data for 2025-10. [2025-10-20T19:56:19.480294] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T19:56:20.079493] [INFO ] [REMOTE ] MLP: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T19:56:20.080499] [INFO ] [REMOTE ] MLP: Full fetch complete (4406 records). [2025-10-20T19:56:20.080572] [DEBUG] [VALIDATOR ] MLP: Validating primary data. [2025-10-20T19:56:20.259185] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-07. [2025-10-20T19:56:20.470037] [DEBUG] [VALIDATOR ] MLP: All expected trading days are present. [2025-10-20T19:56:20.470110] [DEBUG] [VALIDATOR ] MLP: Primary data validation passed. [2025-10-20T19:56:20.470243] [INFO ] [MAIN ] MLP: Fetched primary data is valid. [2025-10-20T19:56:20.470281] [DEBUG] [LOCAL ] MLP: Saving primary data. [2025-10-20T19:56:20.470677] [DEBUG] [LOCAL ] MLP: Trimmed 4406 records to 3092 (keeping 200 days of data) [2025-10-20T19:56:20.489040] [DEBUG] [LOCAL ] MLP: Primary data saved (3092 records). [2025-10-20T19:56:20.489168] [INFO ] [MAIN ] MLP: Full history saved. [2025-10-20T19:56:20.489245] [INFO ] [MAIN ] Update Progress: 1367/2367 symbols completed. MLP took 27.8s. Estimated remaining: 2h 8m 57s. Rate: 7.8 symbols/min [2025-10-20T19:56:20.665800] [INFO ] [MAIN ] MMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:20.682415] [DEBUG] [LOCAL ] MMI: Loaded 15785 existing records from primary storage. [2025-10-20T19:56:20.682635] [INFO ] [MAIN ] MMI: Loaded 15785 existing records. [2025-10-20T19:56:20.859874] [WARN ] [MAIN ] MMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:20.860085] [INFO ] [MAIN ] MMI: Wiping existing primary data before full refresh. [2025-10-20T19:56:20.860133] [INFO ] [LOCAL ] MMI: Starting deletion of primary data. [2025-10-20T19:56:20.860281] [DEBUG] [LOCAL ] MMI: Deleted primary data file. [2025-10-20T19:56:20.860319] [INFO ] [MAIN ] MMI: Starting full primary refresh. [2025-10-20T19:56:21.058888] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-03. [2025-10-20T19:56:21.236744] [INFO ] [REMOTE ] MMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:21.245188] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:56:21.851370] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-01. [2025-10-20T19:56:21.957413] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T19:56:22.651433] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-05. [2025-10-20T19:56:22.749832] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T19:56:23.451387] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-02. [2025-10-20T19:56:24.251403] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-04. [2025-10-20T19:56:24.610635] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T19:56:25.010989] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-10-20T19:56:25.051336] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-03. [2025-10-20T19:56:25.358588] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T19:56:25.771748] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T19:56:25.851349] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-08. [2025-10-20T19:56:26.651356] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-05. [2025-10-20T19:56:26.808091] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T19:56:27.367859] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T19:56:27.451385] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-06. [2025-10-20T19:56:28.251437] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-06. [2025-10-20T19:56:28.265838] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T19:56:29.051418] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-09. [2025-10-20T19:56:29.851422] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-07. [2025-10-20T19:56:30.049763] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:56:30.074516] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T19:56:30.619557] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T19:56:30.651335] [INFO ] [REMOTE ] MLR: Fetching primary data for 2025-10. [2025-10-20T19:56:31.451361] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-07. [2025-10-20T19:56:31.579857] [INFO ] [REMOTE ] MLR: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T19:56:31.582594] [INFO ] [REMOTE ] MLR: Full fetch complete (18972 records). [2025-10-20T19:56:31.582700] [DEBUG] [VALIDATOR ] MLR: Validating primary data. [2025-10-20T19:56:31.979686] [DEBUG] [VALIDATOR ] MLR: All expected trading days are present. [2025-10-20T19:56:31.979781] [DEBUG] [VALIDATOR ] MLR: Primary data validation passed. [2025-10-20T19:56:31.980172] [INFO ] [MAIN ] MLR: Fetched primary data is valid. [2025-10-20T19:56:31.980212] [DEBUG] [LOCAL ] MLR: Saving primary data. [2025-10-20T19:56:31.980839] [DEBUG] [LOCAL ] MLR: Trimmed 18972 records to 12314 (keeping 200 days of data) [2025-10-20T19:56:32.057064] [DEBUG] [LOCAL ] MLR: Primary data saved (12314 records). [2025-10-20T19:56:32.057237] [INFO ] [MAIN ] MLR: Full history saved. [2025-10-20T19:56:32.057336] [INFO ] [MAIN ] Update Progress: 1368/2367 symbols completed. MLR took 29.8s. Estimated remaining: 2h 8m 52s. Rate: 7.8 symbols/min [2025-10-20T19:56:32.234943] [INFO ] [MAIN ] MMID: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:32.235126] [DEBUG] [LOCAL ] MMID: No existing data file found in primary storage. [2025-10-20T19:56:32.235165] [WARN ] [MAIN ] MMID: No existing data found, performing full refresh instead. [2025-10-20T19:56:32.235292] [INFO ] [MAIN ] MMID: Wiping existing primary data before full refresh. [2025-10-20T19:56:32.235334] [INFO ] [LOCAL ] MMID: Starting deletion of primary data. [2025-10-20T19:56:32.235376] [DEBUG] [LOCAL ] MMID: No primary data file found to delete. [2025-10-20T19:56:32.235445] [INFO ] [MAIN ] MMID: Starting full primary refresh. [2025-10-20T19:56:32.256835] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-04. [2025-10-20T19:56:32.605924] [INFO ] [REMOTE ] MMID: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:33.051346] [INFO ] [REMOTE ] MMID: Fetching primary data for 2025-01. [2025-10-20T19:56:33.303824] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T19:56:33.365204] [DEBUG] [REMOTE ] MMID: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:56:33.365459] [WARN ] [REMOTE ] MMID: No data returned for month 2025-01 [2025-10-20T19:56:33.365544] [FATAL] [REMOTE ] MMID: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:56:33.365612] [FATAL] [MAIN ] MMID: Failed to fetch primary data. [2025-10-20T19:56:33.365697] [WARN ] [MAIN ] MMID: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:56:33.365765] [INFO ] [LOCAL ] MMID: Starting deletion of primary data. [2025-10-20T19:56:33.365826] [DEBUG] [LOCAL ] MMID: No primary data file found to delete. [2025-10-20T19:56:33.365861] [INFO ] [LOCAL ] MMID: Starting deletion of derivative data. [2025-10-20T19:56:33.365898] [DEBUG] [LOCAL ] MMID: No derivative data file found to delete. [2025-10-20T19:56:33.365940] [INFO ] [MAIN ] Update Progress: 1369/2367 symbols completed. MMID took 1.3s. Estimated remaining: 2h 8m 39s. Rate: 7.8 symbols/min [2025-10-20T19:56:33.499028] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T19:56:33.565319] [INFO ] [MAIN ] MMKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:33.565525] [DEBUG] [LOCAL ] MMKT: No existing data file found in primary storage. [2025-10-20T19:56:33.565570] [WARN ] [MAIN ] MMKT: No existing data found, performing full refresh instead. [2025-10-20T19:56:33.565708] [INFO ] [MAIN ] MMKT: Wiping existing primary data before full refresh. [2025-10-20T19:56:33.565760] [INFO ] [LOCAL ] MMKT: Starting deletion of primary data. [2025-10-20T19:56:33.565804] [DEBUG] [LOCAL ] MMKT: No primary data file found to delete. [2025-10-20T19:56:33.565837] [INFO ] [MAIN ] MMKT: Starting full primary refresh. [2025-10-20T19:56:33.856459] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-08. [2025-10-20T19:56:33.922294] [INFO ] [REMOTE ] MMKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:34.651350] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-01. [2025-10-20T19:56:34.807242] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T19:56:34.949483] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-01 (0.30 seconds). [2025-10-20T19:56:35.451438] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-02. [2025-10-20T19:56:35.868910] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T19:56:36.251375] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-03. [2025-10-20T19:56:36.667733] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T19:56:37.051378] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-08. [2025-10-20T19:56:37.851369] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-04. [2025-10-20T19:56:38.353105] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T19:56:38.651364] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-05. [2025-10-20T19:56:38.777096] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T19:56:38.929150] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-05 (0.28 seconds). [2025-10-20T19:56:39.451396] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-09. [2025-10-20T19:56:40.251464] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-09. [2025-10-20T19:56:40.267995] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T19:56:41.051392] [INFO ] [REMOTE ] MMD: Fetching primary data for 2025-10. [2025-10-20T19:56:41.856171] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-06. [2025-10-20T19:56:41.857030] [INFO ] [REMOTE ] MMD: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T19:56:41.860075] [INFO ] [REMOTE ] MMD: Full fetch complete (16991 records). [2025-10-20T19:56:41.860275] [DEBUG] [VALIDATOR ] MMD: Validating primary data. [2025-10-20T19:56:42.207208] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T19:56:42.252965] [DEBUG] [VALIDATOR ] MMD: All expected trading days are present. [2025-10-20T19:56:42.253060] [DEBUG] [VALIDATOR ] MMD: Primary data validation passed. [2025-10-20T19:56:42.253423] [INFO ] [MAIN ] MMD: Fetched primary data is valid. [2025-10-20T19:56:42.253494] [DEBUG] [LOCAL ] MMD: Saving primary data. [2025-10-20T19:56:42.254378] [DEBUG] [LOCAL ] MMD: Trimmed 16991 records to 10770 (keeping 200 days of data) [2025-10-20T19:56:42.335965] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-09 (2.08 seconds). [2025-10-20T19:56:42.361925] [DEBUG] [LOCAL ] MMD: Primary data saved (10770 records). [2025-10-20T19:56:42.362063] [INFO ] [MAIN ] MMD: Full history saved. [2025-10-20T19:56:42.362139] [INFO ] [MAIN ] Update Progress: 1370/2367 symbols completed. MMD took 31.4s. Estimated remaining: 2h 8m 33s. Rate: 7.8 symbols/min [2025-10-20T19:56:42.541511] [INFO ] [MAIN ] MMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:42.596730] [DEBUG] [LOCAL ] MMM: Loaded 53890 existing records from primary storage. [2025-10-20T19:56:42.597053] [INFO ] [MAIN ] MMM: Loaded 53890 existing records. [2025-10-20T19:56:42.656555] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-05. [2025-10-20T19:56:42.785801] [WARN ] [MAIN ] MMM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:42.786017] [INFO ] [MAIN ] MMM: Wiping existing primary data before full refresh. [2025-10-20T19:56:42.786066] [INFO ] [LOCAL ] MMM: Starting deletion of primary data. [2025-10-20T19:56:42.786248] [DEBUG] [LOCAL ] MMM: Deleted primary data file. [2025-10-20T19:56:42.786288] [INFO ] [MAIN ] MMM: Starting full primary refresh. [2025-10-20T19:56:43.153937] [INFO ] [REMOTE ] MMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:43.451410] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-01. [2025-10-20T19:56:43.737423] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T19:56:44.251437] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-07. [2025-10-20T19:56:44.596007] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-07 (0.34 seconds). [2025-10-20T19:56:45.051414] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-06. [2025-10-20T19:56:45.594890] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-01 (2.14 seconds). [2025-10-20T19:56:45.851467] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-08. [2025-10-20T19:56:46.035925] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:56:46.304505] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T19:56:46.651421] [INFO ] [REMOTE ] MMC: Fetching primary data for 2025-10. [2025-10-20T19:56:47.451360] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-07. [2025-10-20T19:56:47.506695] [INFO ] [REMOTE ] MMC: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T19:56:47.517680] [INFO ] [REMOTE ] MMC: Full fetch complete (75187 records). [2025-10-20T19:56:47.517844] [DEBUG] [VALIDATOR ] MMC: Validating primary data. [2025-10-20T19:56:47.926344] [DEBUG] [VALIDATOR ] MMC: All expected trading days are present. [2025-10-20T19:56:47.926436] [DEBUG] [VALIDATOR ] MMC: Primary data validation passed. [2025-10-20T19:56:47.927468] [INFO ] [MAIN ] MMC: Fetched primary data is valid. [2025-10-20T19:56:47.927537] [DEBUG] [LOCAL ] MMC: Saving primary data. [2025-10-20T19:56:47.929162] [DEBUG] [LOCAL ] MMC: Trimmed 75187 records to 51958 (keeping 200 days of data) [2025-10-20T19:56:48.245345] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T19:56:48.251525] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-08. [2025-10-20T19:56:48.269714] [DEBUG] [LOCAL ] MMC: Primary data saved (51958 records). [2025-10-20T19:56:48.269799] [INFO ] [MAIN ] MMC: Full history saved. [2025-10-20T19:56:48.269871] [INFO ] [MAIN ] Update Progress: 1371/2367 symbols completed. MMC took 44.5s. Estimated remaining: 2h 8m 23s. Rate: 7.8 symbols/min [2025-10-20T19:56:48.439094] [INFO ] [MAIN ] MMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:48.471118] [DEBUG] [LOCAL ] MMS: Loaded 34279 existing records from primary storage. [2025-10-20T19:56:48.471380] [INFO ] [MAIN ] MMS: Loaded 34279 existing records. [2025-10-20T19:56:48.657440] [WARN ] [MAIN ] MMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:48.657752] [INFO ] [MAIN ] MMS: Wiping existing primary data before full refresh. [2025-10-20T19:56:48.657814] [INFO ] [LOCAL ] MMS: Starting deletion of primary data. [2025-10-20T19:56:48.658019] [DEBUG] [LOCAL ] MMS: Deleted primary data file. [2025-10-20T19:56:48.658076] [INFO ] [MAIN ] MMS: Starting full primary refresh. [2025-10-20T19:56:49.029147] [INFO ] [REMOTE ] MMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:49.025469] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T19:56:49.051339] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-01. [2025-10-20T19:56:49.851367] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-02. [2025-10-20T19:56:50.536169] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T19:56:50.651366] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-09. [2025-10-20T19:56:50.991952] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-09 (0.34 seconds). [2025-10-20T19:56:51.333885] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T19:56:51.451378] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-03. [2025-10-20T19:56:52.251427] [INFO ] [REMOTE ] MMKT: Fetching primary data for 2025-10. [2025-10-20T19:56:52.844118] [INFO ] [REMOTE ] MMKT: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T19:56:52.845013] [INFO ] [REMOTE ] MMKT: Full fetch complete (2127 records). [2025-10-20T19:56:52.845084] [DEBUG] [VALIDATOR ] MMKT: Validating primary data. [2025-10-20T19:56:53.030402] [WARN ] [VALIDATOR ] MMKT: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T19:56:53.030587] [FATAL] [MAIN ] MMKT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:56:53.030685] [WARN ] [MAIN ] MMKT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:56:53.030760] [INFO ] [LOCAL ] MMKT: Starting deletion of primary data. [2025-10-20T19:56:53.030840] [DEBUG] [LOCAL ] MMKT: No primary data file found to delete. [2025-10-20T19:56:53.030896] [INFO ] [LOCAL ] MMKT: Starting deletion of derivative data. [2025-10-20T19:56:53.030941] [DEBUG] [LOCAL ] MMKT: No derivative data file found to delete. [2025-10-20T19:56:53.031013] [INFO ] [MAIN ] Update Progress: 1372/2367 symbols completed. MMKT took 19.7s. Estimated remaining: 2h 8m 14s. Rate: 7.8 symbols/min [2025-10-20T19:56:53.055492] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-09. [2025-10-20T19:56:53.207269] [INFO ] [MAIN ] MMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:56:53.219329] [DEBUG] [LOCAL ] MMT: Loaded 10420 existing records from primary storage. [2025-10-20T19:56:53.219564] [INFO ] [MAIN ] MMT: Loaded 10420 existing records. [2025-10-20T19:56:53.404199] [WARN ] [MAIN ] MMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:56:53.404775] [INFO ] [MAIN ] MMT: Wiping existing primary data before full refresh. [2025-10-20T19:56:53.404851] [INFO ] [LOCAL ] MMT: Starting deletion of primary data. [2025-10-20T19:56:53.405043] [DEBUG] [LOCAL ] MMT: Deleted primary data file. [2025-10-20T19:56:53.405094] [INFO ] [MAIN ] MMT: Starting full primary refresh. [2025-10-20T19:56:53.712348] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-03 (2.26 seconds). [2025-10-20T19:56:53.829198] [INFO ] [REMOTE ] MMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:56:53.851330] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-01. [2025-10-20T19:56:54.142104] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T19:56:54.651361] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-02. [2025-10-20T19:56:54.685470] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T19:56:55.451373] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-04. [2025-10-20T19:56:56.065276] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T19:56:56.251415] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-02. [2025-10-20T19:56:57.056006] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-03. [2025-10-20T19:56:57.059295] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T19:56:57.456722] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T19:56:57.851379] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-05. [2025-10-20T19:56:58.558992] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T19:56:58.651421] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-04. [2025-10-20T19:56:59.451362] [INFO ] [REMOTE ] MMI: Fetching primary data for 2025-10. [2025-10-20T19:56:59.835680] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T19:57:00.128032] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T19:57:00.251406] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-05. [2025-10-20T19:57:00.433386] [INFO ] [REMOTE ] MMI: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:57:00.436492] [INFO ] [REMOTE ] MMI: Full fetch complete (21745 records). [2025-10-20T19:57:00.436566] [DEBUG] [VALIDATOR ] MMI: Validating primary data. [2025-10-20T19:57:00.803756] [DEBUG] [VALIDATOR ] MMI: All expected trading days are present. [2025-10-20T19:57:00.803846] [DEBUG] [VALIDATOR ] MMI: Primary data validation passed. [2025-10-20T19:57:00.804307] [INFO ] [MAIN ] MMI: Fetched primary data is valid. [2025-10-20T19:57:00.804346] [DEBUG] [LOCAL ] MMI: Saving primary data. [2025-10-20T19:57:00.804977] [DEBUG] [LOCAL ] MMI: Trimmed 21745 records to 15464 (keeping 200 days of data) [2025-10-20T19:57:00.895919] [DEBUG] [LOCAL ] MMI: Primary data saved (15464 records). [2025-10-20T19:57:00.896049] [INFO ] [MAIN ] MMI: Full history saved. [2025-10-20T19:57:00.896124] [INFO ] [MAIN ] Update Progress: 1373/2367 symbols completed. MMI took 40.4s. Estimated remaining: 2h 8m 6s. Rate: 7.8 symbols/min [2025-10-20T19:57:01.066751] [INFO ] [MAIN ] MMU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:01.063458] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-03. [2025-10-20T19:57:01.084247] [DEBUG] [LOCAL ] MMU: Loaded 16787 existing records from primary storage. [2025-10-20T19:57:01.084471] [INFO ] [MAIN ] MMU: Loaded 16787 existing records. [2025-10-20T19:57:01.261105] [WARN ] [MAIN ] MMU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:01.261299] [INFO ] [MAIN ] MMU: Wiping existing primary data before full refresh. [2025-10-20T19:57:01.261346] [INFO ] [LOCAL ] MMU: Starting deletion of primary data. [2025-10-20T19:57:01.261512] [DEBUG] [LOCAL ] MMU: Deleted primary data file. [2025-10-20T19:57:01.261594] [INFO ] [MAIN ] MMU: Starting full primary refresh. [2025-10-20T19:57:01.627111] [INFO ] [REMOTE ] MMU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:01.851384] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-01. [2025-10-20T19:57:01.947064] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T19:57:02.208721] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T19:57:02.559757] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:57:02.651360] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-06. [2025-10-20T19:57:03.451457] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-06. [2025-10-20T19:57:04.251351] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-04. [2025-10-20T19:57:04.503098] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T19:57:04.957958] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T19:57:05.056662] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-02. [2025-10-20T19:57:05.059631] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T19:57:05.851349] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-07. [2025-10-20T19:57:06.113593] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T19:57:06.651369] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-07. [2025-10-20T19:57:07.451413] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-03. [2025-10-20T19:57:07.900154] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T19:57:08.024086] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T19:57:08.251352] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-05. [2025-10-20T19:57:08.521054] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T19:57:09.024921] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T19:57:09.051346] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-08. [2025-10-20T19:57:09.851372] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-08. [2025-10-20T19:57:10.420525] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T19:57:10.651378] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-09. [2025-10-20T19:57:11.451371] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-04. [2025-10-20T19:57:12.093738] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T19:57:12.243602] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-10-20T19:57:12.253935] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-06. [2025-10-20T19:57:12.336308] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T19:57:13.051373] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-05. [2025-10-20T19:57:13.175249] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T19:57:13.687703] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T19:57:13.851371] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-06. [2025-10-20T19:57:14.651355] [INFO ] [REMOTE ] MMS: Fetching primary data for 2025-10. [2025-10-20T19:57:14.738081] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T19:57:15.451396] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-09. [2025-10-20T19:57:16.191415] [INFO ] [REMOTE ] MMS: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T19:57:16.198823] [INFO ] [REMOTE ] MMS: Full fetch complete (51315 records). [2025-10-20T19:57:16.198993] [DEBUG] [VALIDATOR ] MMS: Validating primary data. [2025-10-20T19:57:16.259534] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-07. [2025-10-20T19:57:16.598726] [DEBUG] [VALIDATOR ] MMS: All expected trading days are present. [2025-10-20T19:57:16.598846] [DEBUG] [VALIDATOR ] MMS: Primary data validation passed. [2025-10-20T19:57:16.599973] [INFO ] [MAIN ] MMS: Fetched primary data is valid. [2025-10-20T19:57:16.600034] [DEBUG] [LOCAL ] MMS: Saving primary data. [2025-10-20T19:57:16.601177] [DEBUG] [LOCAL ] MMS: Trimmed 51315 records to 33590 (keeping 200 days of data) [2025-10-20T19:57:16.803444] [DEBUG] [LOCAL ] MMS: Primary data saved (33590 records). [2025-10-20T19:57:16.803585] [INFO ] [MAIN ] MMS: Full history saved. [2025-10-20T19:57:16.803686] [INFO ] [MAIN ] Update Progress: 1374/2367 symbols completed. MMS took 28.5s. Estimated remaining: 2h 8m 4s. Rate: 7.8 symbols/min [2025-10-20T19:57:16.974931] [INFO ] [MAIN ] MNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:17.001719] [DEBUG] [LOCAL ] MNR: Loaded 25302 existing records from primary storage. [2025-10-20T19:57:17.001986] [INFO ] [MAIN ] MNR: Loaded 25302 existing records. [2025-10-20T19:57:17.065159] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-07. [2025-10-20T19:57:17.190032] [WARN ] [MAIN ] MNR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:17.190287] [INFO ] [MAIN ] MNR: Wiping existing primary data before full refresh. [2025-10-20T19:57:17.190341] [INFO ] [LOCAL ] MNR: Starting deletion of primary data. [2025-10-20T19:57:17.190563] [DEBUG] [LOCAL ] MNR: Deleted primary data file. [2025-10-20T19:57:17.190619] [INFO ] [MAIN ] MNR: Starting full primary refresh. [2025-10-20T19:57:17.285867] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T19:57:17.579277] [INFO ] [REMOTE ] MNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:17.612953] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T19:57:17.851347] [INFO ] [REMOTE ] MMM: Fetching primary data for 2025-10. [2025-10-20T19:57:18.123147] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T19:57:18.651360] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-08. [2025-10-20T19:57:18.725534] [INFO ] [REMOTE ] MMM: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T19:57:18.733297] [INFO ] [REMOTE ] MMM: Full fetch complete (77220 records). [2025-10-20T19:57:18.733415] [DEBUG] [VALIDATOR ] MMM: Validating primary data. [2025-10-20T19:57:19.132261] [DEBUG] [VALIDATOR ] MMM: All expected trading days are present. [2025-10-20T19:57:19.132349] [DEBUG] [VALIDATOR ] MMM: Primary data validation passed. [2025-10-20T19:57:19.133394] [INFO ] [MAIN ] MMM: Fetched primary data is valid. [2025-10-20T19:57:19.133477] [DEBUG] [LOCAL ] MMM: Saving primary data. [2025-10-20T19:57:19.134836] [DEBUG] [LOCAL ] MMM: Trimmed 77220 records to 53112 (keeping 200 days of data) [2025-10-20T19:57:19.240792] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T19:57:19.451418] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-01. [2025-10-20T19:57:19.477211] [DEBUG] [LOCAL ] MMM: Primary data saved (53112 records). [2025-10-20T19:57:19.477368] [INFO ] [MAIN ] MMM: Full history saved. [2025-10-20T19:57:19.477497] [INFO ] [MAIN ] Update Progress: 1375/2367 symbols completed. MMM took 37.1s. Estimated remaining: 2h 7m 53s. Rate: 7.8 symbols/min [2025-10-20T19:57:19.646932] [INFO ] [MAIN ] MNSO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:19.689185] [DEBUG] [LOCAL ] MNSO: Loaded 44819 existing records from primary storage. [2025-10-20T19:57:19.689515] [INFO ] [MAIN ] MNSO: Loaded 44819 existing records. [2025-10-20T19:57:19.863118] [WARN ] [MAIN ] MNSO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:19.863317] [INFO ] [MAIN ] MNSO: Wiping existing primary data before full refresh. [2025-10-20T19:57:19.863364] [INFO ] [LOCAL ] MNSO: Starting deletion of primary data. [2025-10-20T19:57:19.863559] [DEBUG] [LOCAL ] MNSO: Deleted primary data file. [2025-10-20T19:57:19.863614] [INFO ] [MAIN ] MNSO: Starting full primary refresh. [2025-10-20T19:57:20.209801] [INFO ] [REMOTE ] MNSO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:20.251342] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-01. [2025-10-20T19:57:20.546377] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T19:57:21.051372] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-08. [2025-10-20T19:57:21.851389] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-09. [2025-10-20T19:57:21.901489] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T19:57:22.139478] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T19:57:22.649266] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T19:57:22.651539] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-02. [2025-10-20T19:57:23.451500] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-02. [2025-10-20T19:57:23.950755] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T19:57:24.251438] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-03. [2025-10-20T19:57:24.983367] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T19:57:25.051353] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-03. [2025-10-20T19:57:25.382004] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T19:57:25.851398] [INFO ] [REMOTE ] MMT: Fetching primary data for 2025-10. [2025-10-20T19:57:26.605684] [INFO ] [REMOTE ] MMT: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T19:57:26.607350] [INFO ] [REMOTE ] MMT: Full fetch complete (15895 records). [2025-10-20T19:57:26.607423] [DEBUG] [VALIDATOR ] MMT: Validating primary data. [2025-10-20T19:57:26.651598] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-04. [2025-10-20T19:57:27.010080] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T19:57:27.045594] [DEBUG] [VALIDATOR ] MMT: All expected trading days are present. [2025-10-20T19:57:27.045713] [DEBUG] [VALIDATOR ] MMT: Primary data validation passed. [2025-10-20T19:57:27.046108] [INFO ] [MAIN ] MMT: Fetched primary data is valid. [2025-10-20T19:57:27.046184] [DEBUG] [LOCAL ] MMT: Saving primary data. [2025-10-20T19:57:27.046876] [DEBUG] [LOCAL ] MMT: Trimmed 15895 records to 10181 (keeping 200 days of data) [2025-10-20T19:57:27.107736] [DEBUG] [LOCAL ] MMT: Primary data saved (10181 records). [2025-10-20T19:57:27.107883] [INFO ] [MAIN ] MMT: Full history saved. [2025-10-20T19:57:27.107961] [INFO ] [MAIN ] Update Progress: 1376/2367 symbols completed. MMT took 34.1s. Estimated remaining: 2h 7m 45s. Rate: 7.8 symbols/min [2025-10-20T19:57:27.280761] [INFO ] [MAIN ] MNTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:27.280915] [DEBUG] [LOCAL ] MNTN: No existing data file found in primary storage. [2025-10-20T19:57:27.280951] [WARN ] [MAIN ] MNTN: No existing data found, performing full refresh instead. [2025-10-20T19:57:27.281056] [INFO ] [MAIN ] MNTN: Wiping existing primary data before full refresh. [2025-10-20T19:57:27.281096] [INFO ] [LOCAL ] MNTN: Starting deletion of primary data. [2025-10-20T19:57:27.281136] [DEBUG] [LOCAL ] MNTN: No primary data file found to delete. [2025-10-20T19:57:27.281167] [INFO ] [MAIN ] MNTN: Starting full primary refresh. [2025-10-20T19:57:27.455332] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-04. [2025-10-20T19:57:27.639306] [INFO ] [REMOTE ] MNTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:27.734278] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T19:57:28.251427] [INFO ] [REMOTE ] MNTN: Fetching primary data for 2025-01. [2025-10-20T19:57:28.468044] [DEBUG] [REMOTE ] MNTN: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:57:28.468466] [WARN ] [REMOTE ] MNTN: No data returned for month 2025-01 [2025-10-20T19:57:28.468553] [FATAL] [REMOTE ] MNTN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:57:28.468628] [FATAL] [MAIN ] MNTN: Failed to fetch primary data. [2025-10-20T19:57:28.468708] [WARN ] [MAIN ] MNTN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:57:28.468779] [INFO ] [LOCAL ] MNTN: Starting deletion of primary data. [2025-10-20T19:57:28.468841] [DEBUG] [LOCAL ] MNTN: No primary data file found to delete. [2025-10-20T19:57:28.468876] [INFO ] [LOCAL ] MNTN: Starting deletion of derivative data. [2025-10-20T19:57:28.468914] [DEBUG] [LOCAL ] MNTN: No derivative data file found to delete. [2025-10-20T19:57:28.468958] [INFO ] [MAIN ] Update Progress: 1377/2367 symbols completed. MNTN took 1.4s. Estimated remaining: 2h 7m 33s. Rate: 7.8 symbols/min [2025-10-20T19:57:28.632943] [INFO ] [MAIN ] MO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:28.633120] [DEBUG] [LOCAL ] MO: No existing data file found in primary storage. [2025-10-20T19:57:28.633157] [WARN ] [MAIN ] MO: No existing data found, performing full refresh instead. [2025-10-20T19:57:28.633263] [INFO ] [MAIN ] MO: Wiping existing primary data before full refresh. [2025-10-20T19:57:28.633302] [INFO ] [LOCAL ] MO: Starting deletion of primary data. [2025-10-20T19:57:28.633342] [DEBUG] [LOCAL ] MO: No primary data file found to delete. [2025-10-20T19:57:28.633372] [INFO ] [MAIN ] MO: Starting full primary refresh. [2025-10-20T19:57:28.984608] [INFO ] [REMOTE ] MO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:29.051332] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-01. [2025-10-20T19:57:29.226804] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:57:29.851360] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-09. [2025-10-20T19:57:30.651478] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-05. [2025-10-20T19:57:30.777146] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T19:57:31.376247] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-01 (2.32 seconds). [2025-10-20T19:57:31.451413] [INFO ] [REMOTE ] MMU: Fetching primary data for 2025-10. [2025-10-20T19:57:31.863239] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T19:57:32.251366] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-05. [2025-10-20T19:57:32.390559] [INFO ] [REMOTE ] MMU: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T19:57:32.393408] [INFO ] [REMOTE ] MMU: Full fetch complete (24994 records). [2025-10-20T19:57:32.393506] [DEBUG] [VALIDATOR ] MMU: Validating primary data. [2025-10-20T19:57:32.756353] [DEBUG] [VALIDATOR ] MMU: All expected trading days are present. [2025-10-20T19:57:32.756426] [DEBUG] [VALIDATOR ] MMU: Primary data validation passed. [2025-10-20T19:57:32.756890] [INFO ] [MAIN ] MMU: Fetched primary data is valid. [2025-10-20T19:57:32.756950] [DEBUG] [LOCAL ] MMU: Saving primary data. [2025-10-20T19:57:32.757529] [DEBUG] [LOCAL ] MMU: Trimmed 24994 records to 16694 (keeping 200 days of data) [2025-10-20T19:57:32.862279] [DEBUG] [LOCAL ] MMU: Primary data saved (16694 records). [2025-10-20T19:57:32.862386] [INFO ] [MAIN ] MMU: Full history saved. [2025-10-20T19:57:32.862464] [INFO ] [MAIN ] Update Progress: 1378/2367 symbols completed. MMU took 32.0s. Estimated remaining: 2h 7m 22s. Rate: 7.8 symbols/min [2025-10-20T19:57:33.036421] [INFO ] [MAIN ] MOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:33.058154] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-06. [2025-10-20T19:57:33.078584] [DEBUG] [LOCAL ] MOD: Loaded 45405 existing records from primary storage. [2025-10-20T19:57:33.078897] [INFO ] [MAIN ] MOD: Loaded 45405 existing records. [2025-10-20T19:57:33.252125] [WARN ] [MAIN ] MOD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:33.252318] [INFO ] [MAIN ] MOD: Wiping existing primary data before full refresh. [2025-10-20T19:57:33.252363] [INFO ] [LOCAL ] MOD: Starting deletion of primary data. [2025-10-20T19:57:33.252537] [DEBUG] [LOCAL ] MOD: Deleted primary data file. [2025-10-20T19:57:33.252573] [INFO ] [MAIN ] MOD: Starting full primary refresh. [2025-10-20T19:57:33.601891] [INFO ] [REMOTE ] MOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:33.643600] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T19:57:33.851384] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-02. [2025-10-20T19:57:34.462503] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T19:57:34.651381] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-01. [2025-10-20T19:57:35.451387] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-06. [2025-10-20T19:57:36.070351] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T19:57:36.228595] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T19:57:36.251355] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-02. [2025-10-20T19:57:36.691286] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T19:57:37.051441] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-07. [2025-10-20T19:57:37.851417] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-07. [2025-10-20T19:57:37.916744] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T19:57:38.207445] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T19:57:38.651442] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-03. [2025-10-20T19:57:39.451424] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-03. [2025-10-20T19:57:39.554804] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T19:57:40.251504] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-08. [2025-10-20T19:57:40.655614] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T19:57:41.051453] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-04. [2025-10-20T19:57:41.196911] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:57:41.328091] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T19:57:41.851398] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-04. [2025-10-20T19:57:42.651367] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-08. [2025-10-20T19:57:43.348272] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-10-20T19:57:43.451351] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-05. [2025-10-20T19:57:43.490822] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T19:57:44.251377] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-05. [2025-10-20T19:57:44.343854] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T19:57:45.051359] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-09. [2025-10-20T19:57:45.347330] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:57:45.487531] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T19:57:45.851408] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-06. [2025-10-20T19:57:46.112482] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T19:57:46.651467] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-06. [2025-10-20T19:57:47.451461] [INFO ] [REMOTE ] MNR: Fetching primary data for 2025-10. [2025-10-20T19:57:48.030678] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-06 (2.18 seconds). [2025-10-20T19:57:48.191249] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T19:57:48.251362] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-07. [2025-10-20T19:57:48.703634] [INFO ] [REMOTE ] MNR: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T19:57:48.707676] [INFO ] [REMOTE ] MNR: Full fetch complete (36913 records). [2025-10-20T19:57:48.707757] [DEBUG] [VALIDATOR ] MNR: Validating primary data. [2025-10-20T19:57:49.056086] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-07. [2025-10-20T19:57:49.069467] [DEBUG] [VALIDATOR ] MNR: All expected trading days are present. [2025-10-20T19:57:49.069522] [DEBUG] [VALIDATOR ] MNR: Primary data validation passed. [2025-10-20T19:57:49.070185] [INFO ] [MAIN ] MNR: Fetched primary data is valid. [2025-10-20T19:57:49.070348] [DEBUG] [LOCAL ] MNR: Saving primary data. [2025-10-20T19:57:49.071187] [DEBUG] [LOCAL ] MNR: Trimmed 36913 records to 25017 (keeping 200 days of data) [2025-10-20T19:57:49.222839] [DEBUG] [LOCAL ] MNR: Primary data saved (25017 records). [2025-10-20T19:57:49.222972] [INFO ] [MAIN ] MNR: Full history saved. [2025-10-20T19:57:49.223048] [INFO ] [MAIN ] Update Progress: 1379/2367 symbols completed. MNR took 32.4s. Estimated remaining: 2h 7m 21s. Rate: 7.8 symbols/min [2025-10-20T19:57:49.387848] [INFO ] [MAIN ] MODG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:49.434231] [DEBUG] [LOCAL ] MODG: Loaded 52312 existing records from primary storage. [2025-10-20T19:57:49.434523] [INFO ] [MAIN ] MODG: Loaded 52312 existing records. [2025-10-20T19:57:49.610443] [WARN ] [MAIN ] MODG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:49.610662] [INFO ] [MAIN ] MODG: Wiping existing primary data before full refresh. [2025-10-20T19:57:49.610742] [INFO ] [LOCAL ] MODG: Starting deletion of primary data. [2025-10-20T19:57:49.610917] [DEBUG] [LOCAL ] MODG: Deleted primary data file. [2025-10-20T19:57:49.610961] [INFO ] [MAIN ] MODG: Starting full primary refresh. [2025-10-20T19:57:49.870022] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-09. [2025-10-20T19:57:49.974488] [INFO ] [REMOTE ] MODG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:50.003092] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T19:57:50.651420] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-01. [2025-10-20T19:57:51.418783] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T19:57:51.451344] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-08. [2025-10-20T19:57:51.595525] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:57:52.251402] [INFO ] [REMOTE ] MNSO: Fetching primary data for 2025-10. [2025-10-20T19:57:52.413751] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T19:57:53.051453] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-02. [2025-10-20T19:57:53.750887] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-08 (2.30 seconds). [2025-10-20T19:57:53.851351] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-09. [2025-10-20T19:57:53.993887] [INFO ] [REMOTE ] MNSO: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T19:57:54.000685] [INFO ] [REMOTE ] MNSO: Full fetch complete (64831 records). [2025-10-20T19:57:54.000799] [DEBUG] [VALIDATOR ] MNSO: Validating primary data. [2025-10-20T19:57:54.394483] [DEBUG] [VALIDATOR ] MNSO: All expected trading days are present. [2025-10-20T19:57:54.394598] [DEBUG] [VALIDATOR ] MNSO: Primary data validation passed. [2025-10-20T19:57:54.396288] [INFO ] [MAIN ] MNSO: Fetched primary data is valid. [2025-10-20T19:57:54.396406] [DEBUG] [LOCAL ] MNSO: Saving primary data. [2025-10-20T19:57:54.398013] [DEBUG] [LOCAL ] MNSO: Trimmed 64831 records to 44180 (keeping 200 days of data) [2025-10-20T19:57:54.520678] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T19:57:54.662676] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-03. [2025-10-20T19:57:54.698354] [DEBUG] [LOCAL ] MNSO: Primary data saved (44180 records). [2025-10-20T19:57:54.698501] [INFO ] [MAIN ] MNSO: Full history saved. [2025-10-20T19:57:54.698584] [INFO ] [MAIN ] Update Progress: 1380/2367 symbols completed. MNSO took 35.2s. Estimated remaining: 2h 7m 12s. Rate: 7.8 symbols/min [2025-10-20T19:57:54.863872] [INFO ] [MAIN ] MOH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:54.904795] [DEBUG] [LOCAL ] MOH: Loaded 43308 existing records from primary storage. [2025-10-20T19:57:54.905098] [INFO ] [MAIN ] MOH: Loaded 43308 existing records. [2025-10-20T19:57:55.081152] [WARN ] [MAIN ] MOH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:55.081579] [INFO ] [MAIN ] MOH: Wiping existing primary data before full refresh. [2025-10-20T19:57:55.081634] [INFO ] [LOCAL ] MOH: Starting deletion of primary data. [2025-10-20T19:57:55.081985] [DEBUG] [LOCAL ] MOH: Deleted primary data file. [2025-10-20T19:57:55.082042] [INFO ] [MAIN ] MOH: Starting full primary refresh. [2025-10-20T19:57:55.423898] [INFO ] [REMOTE ] MOH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:57:55.451347] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-01. [2025-10-20T19:57:55.965509] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-09 (2.11 seconds). [2025-10-20T19:57:56.155543] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T19:57:56.251374] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-04. [2025-10-20T19:57:56.845286] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T19:57:57.051424] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-02. [2025-10-20T19:57:57.851378] [INFO ] [REMOTE ] MO: Fetching primary data for 2025-10. [2025-10-20T19:57:58.082932] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T19:57:58.521981] [INFO ] [REMOTE ] MO: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T19:57:58.530523] [INFO ] [REMOTE ] MO: Full fetch complete (77541 records). [2025-10-20T19:57:58.530739] [DEBUG] [VALIDATOR ] MO: Validating primary data. [2025-10-20T19:57:58.656280] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-05. [2025-10-20T19:57:58.763260] [WARN ] [VALIDATOR ] MO: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T19:57:58.767068] [FATAL] [MAIN ] MO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T19:57:58.767387] [WARN ] [MAIN ] MO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T19:57:58.769033] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T19:57:58.778314] [INFO ] [LOCAL ] MO: Starting deletion of primary data. [2025-10-20T19:57:58.779108] [DEBUG] [LOCAL ] MO: No primary data file found to delete. [2025-10-20T19:57:58.779164] [INFO ] [LOCAL ] MO: Starting deletion of derivative data. [2025-10-20T19:57:58.779207] [DEBUG] [LOCAL ] MO: No derivative data file found to delete. [2025-10-20T19:57:58.779271] [INFO ] [MAIN ] Update Progress: 1381/2367 symbols completed. MO took 30.3s. Estimated remaining: 2h 7m 1s. Rate: 7.8 symbols/min [2025-10-20T19:57:58.947948] [INFO ] [MAIN ] MOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:57:58.996405] [DEBUG] [LOCAL ] MOS: Loaded 54288 existing records from primary storage. [2025-10-20T19:57:58.996730] [INFO ] [MAIN ] MOS: Loaded 54288 existing records. [2025-10-20T19:57:59.172274] [WARN ] [MAIN ] MOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:57:59.172475] [INFO ] [MAIN ] MOS: Wiping existing primary data before full refresh. [2025-10-20T19:57:59.172530] [INFO ] [LOCAL ] MOS: Starting deletion of primary data. [2025-10-20T19:57:59.172736] [DEBUG] [LOCAL ] MOS: Deleted primary data file. [2025-10-20T19:57:59.172799] [INFO ] [MAIN ] MOS: Starting full primary refresh. [2025-10-20T19:57:59.456513] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-08. [2025-10-20T19:57:59.522258] [INFO ] [REMOTE ] MOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:00.251390] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-03. [2025-10-20T19:58:00.565499] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T19:58:01.051370] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-06. [2025-10-20T19:58:01.518025] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T19:58:01.851349] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-09. [2025-10-20T19:58:01.940195] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T19:58:02.642821] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T19:58:02.652599] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-04. [2025-10-20T19:58:03.451357] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-01. [2025-10-20T19:58:03.482538] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T19:58:04.261858] [INFO ] [REMOTE ] MOD: Fetching primary data for 2025-10. [2025-10-20T19:58:04.267071] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T19:58:04.895224] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T19:58:05.051423] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-02. [2025-10-20T19:58:05.851404] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-05. [2025-10-20T19:58:06.037239] [INFO ] [REMOTE ] MOD: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T19:58:06.044107] [INFO ] [REMOTE ] MOD: Full fetch complete (66408 records). [2025-10-20T19:58:06.044411] [DEBUG] [VALIDATOR ] MOD: Validating primary data. [2025-10-20T19:58:06.462501] [DEBUG] [VALIDATOR ] MOD: All expected trading days are present. [2025-10-20T19:58:06.462634] [DEBUG] [VALIDATOR ] MOD: Primary data validation passed. [2025-10-20T19:58:06.464182] [INFO ] [MAIN ] MOD: Fetched primary data is valid. [2025-10-20T19:58:06.464249] [DEBUG] [LOCAL ] MOD: Saving primary data. [2025-10-20T19:58:06.466196] [DEBUG] [LOCAL ] MOD: Trimmed 66408 records to 44610 (keeping 200 days of data) [2025-10-20T19:58:06.658141] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-07. [2025-10-20T19:58:06.760045] [DEBUG] [LOCAL ] MOD: Primary data saved (44610 records). [2025-10-20T19:58:06.760446] [INFO ] [MAIN ] MOD: Full history saved. [2025-10-20T19:58:06.761130] [INFO ] [MAIN ] Update Progress: 1382/2367 symbols completed. MOD took 33.9s. Estimated remaining: 2h 6m 54s. Rate: 7.8 symbols/min [2025-10-20T19:58:06.978069] [INFO ] [MAIN ] MOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:06.981008] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T19:58:07.018478] [DEBUG] [LOCAL ] MOV: Loaded 21884 existing records from primary storage. [2025-10-20T19:58:07.019107] [INFO ] [MAIN ] MOV: Loaded 21884 existing records. [2025-10-20T19:58:07.258419] [WARN ] [MAIN ] MOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:07.258784] [INFO ] [MAIN ] MOV: Wiping existing primary data before full refresh. [2025-10-20T19:58:07.258845] [INFO ] [LOCAL ] MOV: Starting deletion of primary data. [2025-10-20T19:58:07.259145] [DEBUG] [LOCAL ] MOV: Deleted primary data file. [2025-10-20T19:58:07.259209] [INFO ] [MAIN ] MOV: Starting full primary refresh. [2025-10-20T19:58:07.455047] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-03. [2025-10-20T19:58:07.616054] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T19:58:07.680165] [INFO ] [REMOTE ] MOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:08.079818] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T19:58:08.251366] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-08. [2025-10-20T19:58:09.051369] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-01. [2025-10-20T19:58:09.369792] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T19:58:09.851439] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-06. [2025-10-20T19:58:10.064941] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:58:10.082007] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T19:58:10.651394] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-02. [2025-10-20T19:58:10.829434] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:58:11.361673] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:58:11.451385] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-03. [2025-10-20T19:58:12.251343] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-04. [2025-10-20T19:58:12.488561] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T19:58:13.051390] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-07. [2025-10-20T19:58:13.851369] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-04. [2025-10-20T19:58:14.061564] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T19:58:14.651379] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-09. [2025-10-20T19:58:15.168453] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-07 (2.12 seconds). [2025-10-20T19:58:15.317441] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T19:58:15.451379] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-08. [2025-10-20T19:58:16.251376] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-05. [2025-10-20T19:58:16.403476] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T19:58:17.051420] [INFO ] [REMOTE ] MODG: Fetching primary data for 2025-10. [2025-10-20T19:58:17.169218] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T19:58:17.223390] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T19:58:17.851390] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-09. [2025-10-20T19:58:18.651373] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-06. [2025-10-20T19:58:18.955059] [INFO ] [REMOTE ] MODG: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T19:58:18.962460] [INFO ] [REMOTE ] MODG: Full fetch complete (74828 records). [2025-10-20T19:58:18.962589] [DEBUG] [VALIDATOR ] MODG: Validating primary data. [2025-10-20T19:58:19.353265] [DEBUG] [VALIDATOR ] MODG: All expected trading days are present. [2025-10-20T19:58:19.353400] [DEBUG] [VALIDATOR ] MODG: Primary data validation passed. [2025-10-20T19:58:19.354582] [INFO ] [MAIN ] MODG: Fetched primary data is valid. [2025-10-20T19:58:19.354729] [DEBUG] [LOCAL ] MODG: Saving primary data. [2025-10-20T19:58:19.356209] [DEBUG] [LOCAL ] MODG: Trimmed 74828 records to 51551 (keeping 200 days of data) [2025-10-20T19:58:19.451778] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-05. [2025-10-20T19:58:19.586377] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:58:19.692024] [DEBUG] [LOCAL ] MODG: Primary data saved (51551 records). [2025-10-20T19:58:19.692313] [INFO ] [MAIN ] MODG: Full history saved. [2025-10-20T19:58:19.692411] [INFO ] [MAIN ] Update Progress: 1383/2367 symbols completed. MODG took 30.5s. Estimated remaining: 2h 6m 50s. Rate: 7.8 symbols/min [2025-10-20T19:58:19.852764] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T19:58:19.918372] [INFO ] [MAIN ] MP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:19.974618] [DEBUG] [LOCAL ] MP: Loaded 54502 existing records from primary storage. [2025-10-20T19:58:19.975022] [INFO ] [MAIN ] MP: Loaded 54502 existing records. [2025-10-20T19:58:20.161789] [WARN ] [MAIN ] MP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:20.161990] [INFO ] [MAIN ] MP: Wiping existing primary data before full refresh. [2025-10-20T19:58:20.162058] [INFO ] [LOCAL ] MP: Starting deletion of primary data. [2025-10-20T19:58:20.162264] [DEBUG] [LOCAL ] MP: Deleted primary data file. [2025-10-20T19:58:20.162334] [INFO ] [MAIN ] MP: Starting full primary refresh. [2025-10-20T19:58:20.258554] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-07. [2025-10-20T19:58:20.510254] [INFO ] [REMOTE ] MP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:21.051367] [INFO ] [REMOTE ] MOH: Fetching primary data for 2025-10. [2025-10-20T19:58:21.248573] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T19:58:21.396604] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T19:58:21.851393] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-08. [2025-10-20T19:58:22.651419] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-06. [2025-10-20T19:58:22.720005] [INFO ] [REMOTE ] MOH: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T19:58:22.726863] [INFO ] [REMOTE ] MOH: Full fetch complete (59547 records). [2025-10-20T19:58:22.732190] [DEBUG] [VALIDATOR ] MOH: Validating primary data. [2025-10-20T19:58:22.739676] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T19:58:23.123808] [DEBUG] [VALIDATOR ] MOH: All expected trading days are present. [2025-10-20T19:58:23.123885] [DEBUG] [VALIDATOR ] MOH: Primary data validation passed. [2025-10-20T19:58:23.124769] [INFO ] [MAIN ] MOH: Fetched primary data is valid. [2025-10-20T19:58:23.124829] [DEBUG] [LOCAL ] MOH: Saving primary data. [2025-10-20T19:58:23.126011] [DEBUG] [LOCAL ] MOH: Trimmed 59547 records to 43003 (keeping 200 days of data) [2025-10-20T19:58:23.376993] [DEBUG] [LOCAL ] MOH: Primary data saved (43003 records). [2025-10-20T19:58:23.377124] [INFO ] [MAIN ] MOH: Full history saved. [2025-10-20T19:58:23.377202] [INFO ] [MAIN ] Update Progress: 1384/2367 symbols completed. MOH took 28.7s. Estimated remaining: 2h 6m 39s. Rate: 7.8 symbols/min [2025-10-20T19:58:23.456723] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-01. [2025-10-20T19:58:23.551134] [INFO ] [MAIN ] MPA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:23.558733] [DEBUG] [LOCAL ] MPA: Loaded 5405 existing records from primary storage. [2025-10-20T19:58:23.558913] [INFO ] [MAIN ] MPA: Loaded 5405 existing records. [2025-10-20T19:58:23.735279] [WARN ] [MAIN ] MPA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:23.735511] [INFO ] [MAIN ] MPA: Wiping existing primary data before full refresh. [2025-10-20T19:58:23.735557] [INFO ] [LOCAL ] MPA: Starting deletion of primary data. [2025-10-20T19:58:23.735755] [DEBUG] [LOCAL ] MPA: Deleted primary data file. [2025-10-20T19:58:23.735810] [INFO ] [MAIN ] MPA: Starting full primary refresh. [2025-10-20T19:58:24.101375] [INFO ] [REMOTE ] MPA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:24.251387] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-01. [2025-10-20T19:58:24.449621] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T19:58:24.600953] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-01 (0.35 seconds). [2025-10-20T19:58:25.051339] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-07. [2025-10-20T19:58:25.323939] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T19:58:25.851401] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-02. [2025-10-20T19:58:26.432997] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T19:58:26.619959] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T19:58:26.651345] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-02. [2025-10-20T19:58:27.451355] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-03. [2025-10-20T19:58:28.010375] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T19:58:28.251373] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-08. [2025-10-20T19:58:28.623976] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-02 (1.97 seconds). [2025-10-20T19:58:29.051454] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-04. [2025-10-20T19:58:29.747287] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T19:58:29.851445] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-05. [2025-10-20T19:58:30.036840] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T19:58:30.476487] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T19:58:30.651444] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-09. [2025-10-20T19:58:31.451438] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-06. [2025-10-20T19:58:32.131389] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T19:58:32.251416] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-07. [2025-10-20T19:58:32.594114] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-09 (1.94 seconds). [2025-10-20T19:58:32.873245] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T19:58:33.051377] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-08. [2025-10-20T19:58:33.636186] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:58:33.851412] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-09. [2025-10-20T19:58:34.651394] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-03. [2025-10-20T19:58:34.928033] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T19:58:35.451370] [INFO ] [REMOTE ] MOS: Fetching primary data for 2025-10. [2025-10-20T19:58:36.251344] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-09. [2025-10-20T19:58:36.758385] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T19:58:36.845898] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-09 (0.59 seconds). [2025-10-20T19:58:36.949019] [INFO ] [REMOTE ] MOS: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T19:58:36.956973] [INFO ] [REMOTE ] MOS: Full fetch complete (77628 records). [2025-10-20T19:58:36.957136] [DEBUG] [VALIDATOR ] MOS: Validating primary data. [2025-10-20T19:58:37.056480] [INFO ] [REMOTE ] MOV: Fetching primary data for 2025-10. [2025-10-20T19:58:37.349029] [DEBUG] [VALIDATOR ] MOS: All expected trading days are present. [2025-10-20T19:58:37.349105] [DEBUG] [VALIDATOR ] MOS: Primary data validation passed. [2025-10-20T19:58:37.350104] [INFO ] [MAIN ] MOS: Fetched primary data is valid. [2025-10-20T19:58:37.350167] [DEBUG] [LOCAL ] MOS: Saving primary data. [2025-10-20T19:58:37.351529] [DEBUG] [LOCAL ] MOS: Trimmed 77628 records to 53512 (keeping 200 days of data) [2025-10-20T19:58:37.673274] [DEBUG] [LOCAL ] MOS: Primary data saved (53512 records). [2025-10-20T19:58:37.673423] [INFO ] [MAIN ] MOS: Full history saved. [2025-10-20T19:58:37.673509] [INFO ] [MAIN ] Update Progress: 1385/2367 symbols completed. MOS took 38.9s. Estimated remaining: 2h 6m 36s. Rate: 7.8 symbols/min [2025-10-20T19:58:37.839353] [INFO ] [MAIN ] MPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:37.851361] [INFO ] [REMOTE ] MPA: Fetching primary data for 2025-10. [2025-10-20T19:58:37.891210] [DEBUG] [LOCAL ] MPC: Loaded 52777 existing records from primary storage. [2025-10-20T19:58:37.891507] [INFO ] [MAIN ] MPC: Loaded 52777 existing records. [2025-10-20T19:58:38.071323] [WARN ] [MAIN ] MPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:38.071573] [INFO ] [MAIN ] MPC: Wiping existing primary data before full refresh. [2025-10-20T19:58:38.071622] [INFO ] [LOCAL ] MPC: Starting deletion of primary data. [2025-10-20T19:58:38.071887] [DEBUG] [LOCAL ] MPC: Deleted primary data file. [2025-10-20T19:58:38.071946] [INFO ] [MAIN ] MPC: Starting full primary refresh. [2025-10-20T19:58:38.379557] [INFO ] [REMOTE ] MPA: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T19:58:38.381159] [INFO ] [REMOTE ] MPA: Full fetch complete (7030 records). [2025-10-20T19:58:38.381611] [DEBUG] [VALIDATOR ] MPA: Validating primary data. [2025-10-20T19:58:38.372575] [INFO ] [REMOTE ] MOV: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T19:58:38.534805] [INFO ] [REMOTE ] MOV: Full fetch complete (30562 records). [2025-10-20T19:58:38.525156] [INFO ] [REMOTE ] MPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:38.540127] [DEBUG] [VALIDATOR ] MOV: Validating primary data. [2025-10-20T19:58:38.651348] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-01. [2025-10-20T19:58:39.144565] [DEBUG] [VALIDATOR ] MPA: All expected trading days are present. [2025-10-20T19:58:39.150368] [DEBUG] [VALIDATOR ] MPA: Primary data validation passed. [2025-10-20T19:58:39.150692] [INFO ] [MAIN ] MPA: Fetched primary data is valid. [2025-10-20T19:58:39.155896] [DEBUG] [LOCAL ] MPA: Saving primary data. [2025-10-20T19:58:39.184538] [DEBUG] [LOCAL ] MPA: Trimmed 7030 records to 5274 (keeping 200 days of data) [2025-10-20T19:58:39.231348] [DEBUG] [VALIDATOR ] MOV: All expected trading days are present. [2025-10-20T19:58:39.231440] [DEBUG] [VALIDATOR ] MOV: Primary data validation passed. [2025-10-20T19:58:39.232064] [INFO ] [MAIN ] MOV: Fetched primary data is valid. [2025-10-20T19:58:39.232144] [DEBUG] [LOCAL ] MOV: Saving primary data. [2025-10-20T19:58:39.258577] [DEBUG] [LOCAL ] MOV: Trimmed 30562 records to 21500 (keeping 200 days of data) [2025-10-20T19:58:39.266094] [DEBUG] [LOCAL ] MPA: Primary data saved (5274 records). [2025-10-20T19:58:39.266583] [INFO ] [MAIN ] MPA: Full history saved. [2025-10-20T19:58:39.266962] [INFO ] [MAIN ] Update Progress: 1386/2367 symbols completed. MPA took 15.9s. Estimated remaining: 2h 6m 24s. Rate: 7.8 symbols/min [2025-10-20T19:58:39.457792] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-04. [2025-10-20T19:58:39.589590] [INFO ] [MAIN ] MPLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:39.594802] [DEBUG] [LOCAL ] MOV: Primary data saved (21500 records). [2025-10-20T19:58:39.595288] [INFO ] [MAIN ] MOV: Full history saved. [2025-10-20T19:58:39.595382] [INFO ] [MAIN ] Update Progress: 1387/2367 symbols completed. MOV took 32.8s. Estimated remaining: 2h 6m 11s. Rate: 7.8 symbols/min [2025-10-20T19:58:39.812528] [INFO ] [MAIN ] MPV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:39.819783] [DEBUG] [LOCAL ] MPV: Loaded 4696 existing records from primary storage. [2025-10-20T19:58:39.819981] [INFO ] [MAIN ] MPV: Loaded 4696 existing records. [2025-10-20T19:58:39.918197] [DEBUG] [LOCAL ] MPLX: Loaded 51524 existing records from primary storage. [2025-10-20T19:58:39.942366] [INFO ] [MAIN ] MPLX: Loaded 51524 existing records. [2025-10-20T19:58:40.139225] [WARN ] [MAIN ] MPV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:40.149807] [INFO ] [MAIN ] MPV: Wiping existing primary data before full refresh. [2025-10-20T19:58:40.149901] [INFO ] [LOCAL ] MPV: Starting deletion of primary data. [2025-10-20T19:58:40.176554] [DEBUG] [LOCAL ] MPV: Deleted primary data file. [2025-10-20T19:58:40.188473] [INFO ] [MAIN ] MPV: Starting full primary refresh. [2025-10-20T19:58:40.246973] [WARN ] [MAIN ] MPLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:40.264255] [INFO ] [MAIN ] MPLX: Wiping existing primary data before full refresh. [2025-10-20T19:58:40.264370] [INFO ] [LOCAL ] MPLX: Starting deletion of primary data. [2025-10-20T19:58:40.294624] [DEBUG] [LOCAL ] MPLX: Deleted primary data file. [2025-10-20T19:58:40.294884] [INFO ] [MAIN ] MPLX: Starting full primary refresh. [2025-10-20T19:58:40.967925] [INFO ] [REMOTE ] MPV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:40.992138] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-01. [2025-10-20T19:58:41.024203] [INFO ] [REMOTE ] MPLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:58:41.051340] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-01. [2025-10-20T19:58:41.058217] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-01 (2.41 seconds). [2025-10-20T19:58:41.698637] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T19:58:41.735062] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-04 (2.28 seconds). [2025-10-20T19:58:41.851412] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-02. [2025-10-20T19:58:42.651350] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-05. [2025-10-20T19:58:42.892895] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T19:58:43.451414] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-02. [2025-10-20T19:58:43.610719] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T19:58:44.064839] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T19:58:44.251428] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-02. [2025-10-20T19:58:44.860084] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T19:58:45.051418] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-06. [2025-10-20T19:58:45.806753] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T19:58:45.851330] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-03. [2025-10-20T19:58:46.651422] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-03. [2025-10-20T19:58:47.116053] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T19:58:47.451490] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-03. [2025-10-20T19:58:47.602963] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T19:58:47.641530] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T19:58:48.251447] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-04. [2025-10-20T19:58:48.713386] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-10-20T19:58:49.051371] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-05. [2025-10-20T19:58:49.313395] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T19:58:49.644906] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T19:58:49.851381] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-04. [2025-10-20T19:58:50.651421] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-04. [2025-10-20T19:58:51.451410] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-06. [2025-10-20T19:58:51.623534] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T19:58:52.064288] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T19:58:52.082773] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T19:58:52.251381] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-05. [2025-10-20T19:58:53.051393] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-05. [2025-10-20T19:58:53.851364] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-07. [2025-10-20T19:58:54.148059] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T19:58:54.213098] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-10-20T19:58:54.385581] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T19:58:54.651352] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-08. [2025-10-20T19:58:55.231862] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T19:58:55.451370] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-07. [2025-10-20T19:58:56.251386] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-09. [2025-10-20T19:58:56.820073] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:58:57.051426] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-06. [2025-10-20T19:58:57.851436] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-06. [2025-10-20T19:58:58.046156] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-07 (2.59 seconds). [2025-10-20T19:58:58.651420] [INFO ] [REMOTE ] MPV: Fetching primary data for 2025-10. [2025-10-20T19:58:58.862224] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T19:58:59.098500] [INFO ] [REMOTE ] MPV: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T19:58:59.099822] [INFO ] [REMOTE ] MPV: Full fetch complete (7658 records). [2025-10-20T19:58:59.099889] [DEBUG] [VALIDATOR ] MPV: Validating primary data. [2025-10-20T19:58:59.460359] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-07. [2025-10-20T19:58:59.492831] [DEBUG] [VALIDATOR ] MPV: All expected trading days are present. [2025-10-20T19:58:59.492947] [DEBUG] [VALIDATOR ] MPV: Primary data validation passed. [2025-10-20T19:58:59.493537] [INFO ] [MAIN ] MPV: Fetched primary data is valid. [2025-10-20T19:58:59.505247] [DEBUG] [LOCAL ] MPV: Saving primary data. [2025-10-20T19:58:59.526057] [DEBUG] [LOCAL ] MPV: Trimmed 7658 records to 4405 (keeping 200 days of data) [2025-10-20T19:58:59.541218] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T19:58:59.569366] [DEBUG] [LOCAL ] MPV: Primary data saved (4405 records). [2025-10-20T19:58:59.569488] [INFO ] [MAIN ] MPV: Full history saved. [2025-10-20T19:58:59.569582] [INFO ] [MAIN ] Update Progress: 1388/2367 symbols completed. MPV took 20.0s. Estimated remaining: 2h 6m 12s. Rate: 7.8 symbols/min [2025-10-20T19:58:59.753169] [INFO ] [MAIN ] MPW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:58:59.803407] [DEBUG] [LOCAL ] MPW: Loaded 54274 existing records from primary storage. [2025-10-20T19:58:59.803781] [INFO ] [MAIN ] MPW: Loaded 54274 existing records. [2025-10-20T19:58:59.979984] [WARN ] [MAIN ] MPW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:58:59.980188] [INFO ] [MAIN ] MPW: Wiping existing primary data before full refresh. [2025-10-20T19:58:59.980234] [INFO ] [LOCAL ] MPW: Starting deletion of primary data. [2025-10-20T19:58:59.980436] [DEBUG] [LOCAL ] MPW: Deleted primary data file. [2025-10-20T19:58:59.980491] [INFO ] [MAIN ] MPW: Starting full primary refresh. [2025-10-20T19:59:00.255308] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-08. [2025-10-20T19:59:00.333742] [INFO ] [REMOTE ] MPW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:01.051428] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-01. [2025-10-20T19:59:01.731780] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-07 (2.27 seconds). [2025-10-20T19:59:01.851369] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-08. [2025-10-20T19:59:02.651360] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-07. [2025-10-20T19:59:02.693309] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T19:59:02.732781] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-08 (2.48 seconds). [2025-10-20T19:59:03.451362] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-02. [2025-10-20T19:59:03.525985] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T19:59:04.081854] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T19:59:04.251415] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-08. [2025-10-20T19:59:05.051377] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-09. [2025-10-20T19:59:05.332340] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T19:59:05.851346] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-09. [2025-10-20T19:59:06.059154] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T19:59:06.651387] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-03. [2025-10-20T19:59:06.949596] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-10-20T19:59:07.451372] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-09. [2025-10-20T19:59:08.205869] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T19:59:08.251328] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-04. [2025-10-20T19:59:08.321485] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-09 (2.47 seconds). [2025-10-20T19:59:09.051403] [INFO ] [REMOTE ] MPC: Fetching primary data for 2025-10. [2025-10-20T19:59:09.177140] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T19:59:09.851341] [INFO ] [REMOTE ] MPLX: Fetching primary data for 2025-10. [2025-10-20T19:59:09.997926] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T19:59:10.651367] [INFO ] [REMOTE ] MP: Fetching primary data for 2025-10. [2025-10-20T19:59:10.772608] [INFO ] [REMOTE ] MPC: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T19:59:10.780194] [INFO ] [REMOTE ] MPC: Full fetch complete (75785 records). [2025-10-20T19:59:10.780311] [DEBUG] [VALIDATOR ] MPC: Validating primary data. [2025-10-20T19:59:11.165978] [DEBUG] [VALIDATOR ] MPC: All expected trading days are present. [2025-10-20T19:59:11.166086] [DEBUG] [VALIDATOR ] MPC: Primary data validation passed. [2025-10-20T19:59:11.167099] [INFO ] [MAIN ] MPC: Fetched primary data is valid. [2025-10-20T19:59:11.167162] [DEBUG] [LOCAL ] MPC: Saving primary data. [2025-10-20T19:59:11.168627] [DEBUG] [LOCAL ] MPC: Trimmed 75785 records to 51981 (keeping 200 days of data) [2025-10-20T19:59:11.464405] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-05. [2025-10-20T19:59:11.486937] [DEBUG] [LOCAL ] MPC: Primary data saved (51981 records). [2025-10-20T19:59:11.487028] [INFO ] [MAIN ] MPC: Full history saved. [2025-10-20T19:59:11.487100] [INFO ] [MAIN ] Update Progress: 1389/2367 symbols completed. MPC took 33.8s. Estimated remaining: 2h 6m 7s. Rate: 7.8 symbols/min [2025-10-20T19:59:11.659682] [INFO ] [MAIN ] MPX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:11.666434] [DEBUG] [LOCAL ] MPX: Loaded 4090 existing records from primary storage. [2025-10-20T19:59:11.666666] [INFO ] [MAIN ] MPX: Loaded 4090 existing records. [2025-10-20T19:59:11.849141] [WARN ] [MAIN ] MPX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:59:11.868382] [INFO ] [MAIN ] MPX: Wiping existing primary data before full refresh. [2025-10-20T19:59:11.869771] [INFO ] [REMOTE ] MPLX: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T19:59:11.869096] [INFO ] [LOCAL ] MPX: Starting deletion of primary data. [2025-10-20T19:59:11.877368] [DEBUG] [LOCAL ] MPX: Deleted primary data file. [2025-10-20T19:59:11.877442] [INFO ] [MAIN ] MPX: Starting full primary refresh. [2025-10-20T19:59:11.921611] [INFO ] [REMOTE ] MPLX: Full fetch complete (74362 records). [2025-10-20T19:59:11.954809] [DEBUG] [VALIDATOR ] MPLX: Validating primary data. [2025-10-20T19:59:12.583845] [INFO ] [REMOTE ] MPX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:12.607810] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-01. [2025-10-20T19:59:12.667213] [DEBUG] [VALIDATOR ] MPLX: All expected trading days are present. [2025-10-20T19:59:12.667284] [DEBUG] [VALIDATOR ] MPLX: Primary data validation passed. [2025-10-20T19:59:12.668277] [INFO ] [MAIN ] MPLX: Fetched primary data is valid. [2025-10-20T19:59:12.668365] [DEBUG] [LOCAL ] MPLX: Saving primary data. [2025-10-20T19:59:12.669723] [DEBUG] [LOCAL ] MPLX: Trimmed 74362 records to 50822 (keeping 200 days of data) [2025-10-20T19:59:12.991921] [DEBUG] [LOCAL ] MPLX: Primary data saved (50822 records). [2025-10-20T19:59:12.992107] [INFO ] [MAIN ] MPLX: Full history saved. [2025-10-20T19:59:12.992195] [INFO ] [MAIN ] Update Progress: 1390/2367 symbols completed. MPLX took 33.7s. Estimated remaining: 2h 5m 55s. Rate: 7.8 symbols/min [2025-10-20T19:59:13.164911] [INFO ] [MAIN ] MQT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:13.174298] [DEBUG] [LOCAL ] MQT: Loaded 8424 existing records from primary storage. [2025-10-20T19:59:13.174484] [INFO ] [MAIN ] MQT: Loaded 8424 existing records. [2025-10-20T19:59:13.360314] [WARN ] [MAIN ] MQT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:59:13.360742] [INFO ] [MAIN ] MQT: Wiping existing primary data before full refresh. [2025-10-20T19:59:13.360810] [INFO ] [LOCAL ] MQT: Starting deletion of primary data. [2025-10-20T19:59:13.361069] [DEBUG] [LOCAL ] MQT: Deleted primary data file. [2025-10-20T19:59:13.363688] [INFO ] [MAIN ] MQT: Starting full primary refresh. [2025-10-20T19:59:13.397207] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T19:59:13.446417] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-02. [2025-10-20T19:59:13.507456] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T19:59:13.788142] [INFO ] [REMOTE ] MQT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:13.811612] [INFO ] [REMOTE ] MP: Incremental fetch complete for 2025-10 (3.16 seconds). [2025-10-20T19:59:13.820318] [INFO ] [REMOTE ] MP: Full fetch complete (77574 records). [2025-10-20T19:59:13.820512] [DEBUG] [VALIDATOR ] MP: Validating primary data. [2025-10-20T19:59:13.856452] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-01. [2025-10-20T19:59:14.218816] [DEBUG] [VALIDATOR ] MP: All expected trading days are present. [2025-10-20T19:59:14.218902] [DEBUG] [VALIDATOR ] MP: Primary data validation passed. [2025-10-20T19:59:14.219931] [INFO ] [MAIN ] MP: Fetched primary data is valid. [2025-10-20T19:59:14.219993] [DEBUG] [LOCAL ] MP: Saving primary data. [2025-10-20T19:59:14.221453] [DEBUG] [LOCAL ] MP: Trimmed 77574 records to 53744 (keeping 200 days of data) [2025-10-20T19:59:14.316882] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T19:59:14.548199] [DEBUG] [LOCAL ] MP: Primary data saved (53744 records). [2025-10-20T19:59:14.548358] [INFO ] [MAIN ] MP: Full history saved. [2025-10-20T19:59:14.548449] [INFO ] [MAIN ] Update Progress: 1391/2367 symbols completed. MP took 54.9s. Estimated remaining: 2h 5m 43s. Rate: 7.8 symbols/min [2025-10-20T19:59:14.656451] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-03. [2025-10-20T19:59:14.722687] [INFO ] [MAIN ] MQY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:14.731741] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T19:59:14.749758] [DEBUG] [LOCAL ] MQY: Loaded 17099 existing records from primary storage. [2025-10-20T19:59:14.750002] [INFO ] [MAIN ] MQY: Loaded 17099 existing records. [2025-10-20T19:59:14.923988] [WARN ] [MAIN ] MQY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:59:14.924217] [INFO ] [MAIN ] MQY: Wiping existing primary data before full refresh. [2025-10-20T19:59:14.924275] [INFO ] [LOCAL ] MQY: Starting deletion of primary data. [2025-10-20T19:59:14.924458] [DEBUG] [LOCAL ] MQY: Deleted primary data file. [2025-10-20T19:59:14.924512] [INFO ] [MAIN ] MQY: Starting full primary refresh. [2025-10-20T19:59:15.286806] [INFO ] [REMOTE ] MQY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:15.337022] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-10-20T19:59:15.451357] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-01. [2025-10-20T19:59:16.251346] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-02. [2025-10-20T19:59:16.596079] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T19:59:17.051354] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-06. [2025-10-20T19:59:17.073526] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T19:59:17.851357] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-04. [2025-10-20T19:59:18.488685] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T19:59:18.651362] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-05. [2025-10-20T19:59:18.813809] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T19:59:19.228898] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T19:59:19.451462] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-06. [2025-10-20T19:59:20.033338] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T19:59:20.251367] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-07. [2025-10-20T19:59:20.771011] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T19:59:21.051406] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-03. [2025-10-20T19:59:21.781607] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T19:59:21.851380] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-04. [2025-10-20T19:59:22.651352] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-08. [2025-10-20T19:59:22.803538] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T19:59:23.281399] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T19:59:23.451403] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-05. [2025-10-20T19:59:24.160305] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T19:59:24.251445] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-09. [2025-10-20T19:59:24.823110] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T19:59:25.051395] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-02. [2025-10-20T19:59:25.758548] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T19:59:25.851439] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-07. [2025-10-20T19:59:26.651381] [INFO ] [REMOTE ] MPX: Fetching primary data for 2025-10. [2025-10-20T19:59:27.393008] [INFO ] [REMOTE ] MPX: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T19:59:27.394171] [INFO ] [REMOTE ] MPX: Full fetch complete (6504 records). [2025-10-20T19:59:27.394242] [DEBUG] [VALIDATOR ] MPX: Validating primary data. [2025-10-20T19:59:27.457753] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-06. [2025-10-20T19:59:27.776733] [DEBUG] [VALIDATOR ] MPX: All expected trading days are present. [2025-10-20T19:59:27.776893] [DEBUG] [VALIDATOR ] MPX: Primary data validation passed. [2025-10-20T19:59:27.777153] [INFO ] [MAIN ] MPX: Fetched primary data is valid. [2025-10-20T19:59:27.777268] [DEBUG] [LOCAL ] MPX: Saving primary data. [2025-10-20T19:59:27.777966] [DEBUG] [LOCAL ] MPX: Trimmed 6504 records to 4023 (keeping 200 days of data) [2025-10-20T19:59:27.803108] [DEBUG] [LOCAL ] MPX: Primary data saved (4023 records). [2025-10-20T19:59:27.803279] [INFO ] [MAIN ] MPX: Full history saved. [2025-10-20T19:59:27.803385] [INFO ] [MAIN ] Update Progress: 1392/2367 symbols completed. MPX took 16.3s. Estimated remaining: 2h 5m 39s. Rate: 7.8 symbols/min [2025-10-20T19:59:28.035234] [INFO ] [MAIN ] MRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:28.037553] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T19:59:28.054726] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T19:59:28.092715] [DEBUG] [LOCAL ] MRC: Loaded 37839 existing records from primary storage. [2025-10-20T19:59:28.093036] [INFO ] [MAIN ] MRC: Loaded 37839 existing records. [2025-10-20T19:59:28.254944] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-07. [2025-10-20T19:59:28.275782] [WARN ] [MAIN ] MRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:59:28.276022] [INFO ] [MAIN ] MRC: Wiping existing primary data before full refresh. [2025-10-20T19:59:28.276071] [INFO ] [LOCAL ] MRC: Starting deletion of primary data. [2025-10-20T19:59:28.276272] [DEBUG] [LOCAL ] MRC: Deleted primary data file. [2025-10-20T19:59:28.276326] [INFO ] [MAIN ] MRC: Starting full primary refresh. [2025-10-20T19:59:28.632100] [INFO ] [REMOTE ] MRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:29.051412] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-01. [2025-10-20T19:59:29.160439] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T19:59:29.851464] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-03. [2025-10-20T19:59:30.517424] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T19:59:30.651349] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-02. [2025-10-20T19:59:30.886311] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T19:59:31.451427] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-08. [2025-10-20T19:59:31.761168] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T19:59:32.251403] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-03. [2025-10-20T19:59:33.051434] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-04. [2025-10-20T19:59:33.448347] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T19:59:33.520957] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T19:59:33.851348] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-04. [2025-10-20T19:59:34.124354] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T19:59:34.651415] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-09. [2025-10-20T19:59:35.363834] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T19:59:35.451421] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-08. [2025-10-20T19:59:36.065565] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T19:59:36.251486] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-09. [2025-10-20T19:59:36.616843] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T19:59:36.982528] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T19:59:37.051401] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-05. [2025-10-20T19:59:37.851387] [INFO ] [REMOTE ] MQT: Fetching primary data for 2025-10. [2025-10-20T19:59:37.962331] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T19:59:38.580537] [INFO ] [REMOTE ] MQT: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T19:59:38.582170] [INFO ] [REMOTE ] MQT: Full fetch complete (13012 records). [2025-10-20T19:59:38.582244] [DEBUG] [VALIDATOR ] MQT: Validating primary data. [2025-10-20T19:59:38.656639] [INFO ] [REMOTE ] MPW: Fetching primary data for 2025-10. [2025-10-20T19:59:38.949566] [DEBUG] [VALIDATOR ] MQT: All expected trading days are present. [2025-10-20T19:59:38.949670] [DEBUG] [VALIDATOR ] MQT: Primary data validation passed. [2025-10-20T19:59:38.949989] [INFO ] [MAIN ] MQT: Fetched primary data is valid. [2025-10-20T19:59:38.950035] [DEBUG] [LOCAL ] MQT: Saving primary data. [2025-10-20T19:59:38.950544] [DEBUG] [LOCAL ] MQT: Trimmed 13012 records to 8156 (keeping 200 days of data) [2025-10-20T19:59:39.000470] [DEBUG] [LOCAL ] MQT: Primary data saved (8156 records). [2025-10-20T19:59:39.000623] [INFO ] [MAIN ] MQT: Full history saved. [2025-10-20T19:59:39.000930] [INFO ] [MAIN ] Update Progress: 1393/2367 symbols completed. MQT took 26.0s. Estimated remaining: 2h 5m 34s. Rate: 7.8 symbols/min [2025-10-20T19:59:39.171500] [INFO ] [MAIN ] MRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:39.171727] [DEBUG] [LOCAL ] MRK: No existing data file found in primary storage. [2025-10-20T19:59:39.171778] [WARN ] [MAIN ] MRK: No existing data found, performing full refresh instead. [2025-10-20T19:59:39.171891] [INFO ] [MAIN ] MRK: Wiping existing primary data before full refresh. [2025-10-20T19:59:39.171931] [INFO ] [LOCAL ] MRK: Starting deletion of primary data. [2025-10-20T19:59:39.171971] [DEBUG] [LOCAL ] MRK: No primary data file found to delete. [2025-10-20T19:59:39.172001] [INFO ] [MAIN ] MRK: Starting full primary refresh. [2025-10-20T19:59:39.456548] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-06. [2025-10-20T19:59:39.533776] [INFO ] [REMOTE ] MRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:40.251371] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-01. [2025-10-20T19:59:40.438158] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T19:59:40.647427] [INFO ] [REMOTE ] MPW: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T19:59:40.655521] [INFO ] [REMOTE ] MPW: Full fetch complete (77588 records). [2025-10-20T19:59:40.655724] [DEBUG] [VALIDATOR ] MPW: Validating primary data. [2025-10-20T19:59:41.053147] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-05. [2025-10-20T19:59:41.053458] [DEBUG] [VALIDATOR ] MPW: All expected trading days are present. [2025-10-20T19:59:41.053535] [DEBUG] [VALIDATOR ] MPW: Primary data validation passed. [2025-10-20T19:59:41.055391] [INFO ] [MAIN ] MPW: Fetched primary data is valid. [2025-10-20T19:59:41.055552] [DEBUG] [LOCAL ] MPW: Saving primary data. [2025-10-20T19:59:41.057160] [DEBUG] [LOCAL ] MPW: Trimmed 77588 records to 53498 (keeping 200 days of data) [2025-10-20T19:59:41.375400] [DEBUG] [LOCAL ] MPW: Primary data saved (53498 records). [2025-10-20T19:59:41.375526] [INFO ] [MAIN ] MPW: Full history saved. [2025-10-20T19:59:41.375877] [INFO ] [MAIN ] Update Progress: 1394/2367 symbols completed. MPW took 41.8s. Estimated remaining: 2h 5m 22s. Rate: 7.8 symbols/min [2025-10-20T19:59:41.570678] [INFO ] [MAIN ] MRP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:41.570859] [DEBUG] [LOCAL ] MRP: No existing data file found in primary storage. [2025-10-20T19:59:41.570897] [WARN ] [MAIN ] MRP: No existing data found, performing full refresh instead. [2025-10-20T19:59:41.571003] [INFO ] [MAIN ] MRP: Wiping existing primary data before full refresh. [2025-10-20T19:59:41.571043] [INFO ] [LOCAL ] MRP: Starting deletion of primary data. [2025-10-20T19:59:41.571083] [DEBUG] [LOCAL ] MRP: No primary data file found to delete. [2025-10-20T19:59:41.571113] [INFO ] [MAIN ] MRP: Starting full primary refresh. [2025-10-20T19:59:41.856490] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-07. [2025-10-20T19:59:41.921877] [INFO ] [REMOTE ] MRP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:42.539496] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-01 (2.29 seconds). [2025-10-20T19:59:42.651355] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-02. [2025-10-20T19:59:42.684975] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T19:59:42.722742] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T19:59:43.451517] [INFO ] [REMOTE ] MRP: Fetching primary data for 2025-01. [2025-10-20T19:59:43.664245] [DEBUG] [REMOTE ] MRP: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T19:59:43.664547] [WARN ] [REMOTE ] MRP: No data returned for month 2025-01 [2025-10-20T19:59:43.664640] [FATAL] [REMOTE ] MRP: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T19:59:43.664725] [FATAL] [MAIN ] MRP: Failed to fetch primary data. [2025-10-20T19:59:43.664784] [WARN ] [MAIN ] MRP: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T19:59:43.664858] [INFO ] [LOCAL ] MRP: Starting deletion of primary data. [2025-10-20T19:59:43.664921] [DEBUG] [LOCAL ] MRP: No primary data file found to delete. [2025-10-20T19:59:43.664956] [INFO ] [LOCAL ] MRP: Starting deletion of derivative data. [2025-10-20T19:59:43.664994] [DEBUG] [LOCAL ] MRP: No derivative data file found to delete. [2025-10-20T19:59:43.665038] [INFO ] [MAIN ] Update Progress: 1395/2367 symbols completed. MRP took 2.3s. Estimated remaining: 2h 5m 11s. Rate: 7.8 symbols/min [2025-10-20T19:59:43.834746] [INFO ] [MAIN ] MS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:43.834927] [DEBUG] [LOCAL ] MS: No existing data file found in primary storage. [2025-10-20T19:59:43.834983] [WARN ] [MAIN ] MS: No existing data found, performing full refresh instead. [2025-10-20T19:59:43.835096] [INFO ] [MAIN ] MS: Wiping existing primary data before full refresh. [2025-10-20T19:59:43.835137] [INFO ] [LOCAL ] MS: Starting deletion of primary data. [2025-10-20T19:59:43.835177] [DEBUG] [LOCAL ] MS: No primary data file found to delete. [2025-10-20T19:59:43.835208] [INFO ] [MAIN ] MS: Starting full primary refresh. [2025-10-20T19:59:44.187487] [INFO ] [REMOTE ] MS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:44.251341] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-01. [2025-10-20T19:59:44.932862] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-02 (2.28 seconds). [2025-10-20T19:59:45.051374] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-03. [2025-10-20T19:59:45.851359] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-06. [2025-10-20T19:59:45.961270] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T19:59:46.651381] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-02. [2025-10-20T19:59:47.141921] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T19:59:47.171398] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T19:59:47.451375] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-08. [2025-10-20T19:59:48.251377] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-07. [2025-10-20T19:59:48.422506] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T19:59:48.566717] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T19:59:49.051420] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-04. [2025-10-20T19:59:49.803344] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T19:59:49.851407] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-03. [2025-10-20T19:59:50.651369] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-08. [2025-10-20T19:59:51.339703] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T19:59:51.451402] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-05. [2025-10-20T19:59:51.903358] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T19:59:51.964945] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T19:59:52.251369] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-09. [2025-10-20T19:59:53.051459] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-09. [2025-10-20T19:59:53.609284] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T19:59:53.749669] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T19:59:53.799789] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-05 (2.35 seconds). [2025-10-20T19:59:53.851331] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-06. [2025-10-20T19:59:54.651409] [INFO ] [REMOTE ] MQY: Fetching primary data for 2025-10. [2025-10-20T19:59:55.451410] [INFO ] [REMOTE ] MRC: Fetching primary data for 2025-10. [2025-10-20T19:59:55.631990] [INFO ] [REMOTE ] MQY: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T19:59:55.651006] [INFO ] [REMOTE ] MQY: Full fetch complete (26221 records). [2025-10-20T19:59:55.664687] [DEBUG] [VALIDATOR ] MQY: Validating primary data. [2025-10-20T19:59:55.713669] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T19:59:56.068395] [DEBUG] [VALIDATOR ] MQY: All expected trading days are present. [2025-10-20T19:59:56.068490] [DEBUG] [VALIDATOR ] MQY: Primary data validation passed. [2025-10-20T19:59:56.069071] [INFO ] [MAIN ] MQY: Fetched primary data is valid. [2025-10-20T19:59:56.069130] [DEBUG] [LOCAL ] MQY: Saving primary data. [2025-10-20T19:59:56.069904] [DEBUG] [LOCAL ] MQY: Trimmed 26221 records to 16735 (keeping 200 days of data) [2025-10-20T19:59:56.170697] [DEBUG] [LOCAL ] MQY: Primary data saved (16735 records). [2025-10-20T19:59:56.170844] [INFO ] [MAIN ] MQY: Full history saved. [2025-10-20T19:59:56.171005] [INFO ] [MAIN ] Update Progress: 1396/2367 symbols completed. MQY took 41.6s. Estimated remaining: 2h 5m 6s. Rate: 7.8 symbols/min [2025-10-20T19:59:56.259976] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-07. [2025-10-20T19:59:56.340335] [INFO ] [MAIN ] MSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T19:59:56.360518] [DEBUG] [LOCAL ] MSA: Loaded 19108 existing records from primary storage. [2025-10-20T19:59:56.360823] [INFO ] [MAIN ] MSA: Loaded 19108 existing records. [2025-10-20T19:59:56.537055] [WARN ] [MAIN ] MSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T19:59:56.537277] [INFO ] [MAIN ] MSA: Wiping existing primary data before full refresh. [2025-10-20T19:59:56.537325] [INFO ] [LOCAL ] MSA: Starting deletion of primary data. [2025-10-20T19:59:56.537559] [DEBUG] [LOCAL ] MSA: Deleted primary data file. [2025-10-20T19:59:56.537616] [INFO ] [MAIN ] MSA: Starting full primary refresh. [2025-10-20T19:59:56.939083] [INFO ] [REMOTE ] MSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T19:59:56.941309] [INFO ] [REMOTE ] MRC: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T19:59:56.950167] [INFO ] [REMOTE ] MRC: Full fetch complete (55269 records). [2025-10-20T19:59:56.950370] [DEBUG] [VALIDATOR ] MRC: Validating primary data. [2025-10-20T19:59:57.060180] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-01. [2025-10-20T19:59:57.364977] [DEBUG] [VALIDATOR ] MRC: All expected trading days are present. [2025-10-20T19:59:57.365061] [DEBUG] [VALIDATOR ] MRC: Primary data validation passed. [2025-10-20T19:59:57.365958] [INFO ] [MAIN ] MRC: Fetched primary data is valid. [2025-10-20T19:59:57.366022] [DEBUG] [LOCAL ] MRC: Saving primary data. [2025-10-20T19:59:57.367141] [DEBUG] [LOCAL ] MRC: Trimmed 55269 records to 37338 (keeping 200 days of data) [2025-10-20T19:59:57.581854] [DEBUG] [LOCAL ] MRC: Primary data saved (37338 records). [2025-10-20T19:59:57.582018] [INFO ] [MAIN ] MRC: Full history saved. [2025-10-20T19:59:57.582110] [INFO ] [MAIN ] Update Progress: 1397/2367 symbols completed. MRC took 29.8s. Estimated remaining: 2h 4m 54s. Rate: 7.8 symbols/min [2025-10-20T19:59:57.851375] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-04. [2025-10-20T19:59:57.964784] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T19:59:58.288751] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T19:59:58.651440] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-08. [2025-10-20T19:59:59.451464] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-02. [2025-10-20T19:59:59.999352] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T20:00:00.105620] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:00:00.251360] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-03. [2025-10-20T20:00:01.051412] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-05. [2025-10-20T20:00:01.414585] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-08 (2.76 seconds). [2025-10-20T20:00:01.489969] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T20:00:01.851381] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-04. [2025-10-20T20:00:02.651451] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-09. [2025-10-20T20:00:03.859959] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-05 (2.81 seconds). [2025-10-20T20:00:03.871247] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-06. [2025-10-20T20:00:03.943534] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-04 (2.09 seconds). [2025-10-20T20:00:04.251445] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-05. [2025-10-20T20:00:05.428735] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-09 (2.78 seconds). [2025-10-20T20:00:05.440132] [INFO ] [REMOTE ] MRK: Fetching primary data for 2025-10. [2025-10-20T20:00:05.866533] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:00:05.871540] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-06. [2025-10-20T20:00:06.275039] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-06 (2.40 seconds). [2025-10-20T20:00:06.629171] [INFO ] [REMOTE ] MRK: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T20:00:06.637123] [INFO ] [REMOTE ] MRK: Full fetch complete (77968 records). [2025-10-20T20:00:06.637274] [DEBUG] [VALIDATOR ] MRK: Validating primary data. [2025-10-20T20:00:06.669275] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-07. [2025-10-20T20:00:07.051917] [DEBUG] [VALIDATOR ] MRK: All expected trading days are present. [2025-10-20T20:00:07.052053] [DEBUG] [VALIDATOR ] MRK: Primary data validation passed. [2025-10-20T20:00:07.053425] [INFO ] [MAIN ] MRK: Fetched primary data is valid. [2025-10-20T20:00:07.053511] [DEBUG] [LOCAL ] MRK: Saving primary data. [2025-10-20T20:00:07.055522] [DEBUG] [LOCAL ] MRK: Trimmed 77968 records to 53790 (keeping 200 days of data) [2025-10-20T20:00:07.400692] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T20:00:07.412127] [DEBUG] [LOCAL ] MRK: Primary data saved (53790 records). [2025-10-20T20:00:07.412731] [INFO ] [MAIN ] MRK: Full history saved. [2025-10-20T20:00:07.412809] [INFO ] [MAIN ] Update Progress: 1398/2367 symbols completed. MRK took 28.4s. Estimated remaining: 2h 4m 48s. Rate: 7.8 symbols/min [2025-10-20T20:00:07.451382] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-07. [2025-10-20T20:00:08.656052] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T20:00:08.660410] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-08. [2025-10-20T20:00:08.991920] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-10-20T20:00:09.051362] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-08. [2025-10-20T20:00:09.667716] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T20:00:09.851435] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-09. [2025-10-20T20:00:10.954702] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T20:00:10.972360] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T20:00:10.976868] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-09. [2025-10-20T20:00:11.451435] [INFO ] [REMOTE ] MSA: Fetching primary data for 2025-10. [2025-10-20T20:00:12.252992] [INFO ] [REMOTE ] MSA: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:00:12.256019] [INFO ] [REMOTE ] MSA: Full fetch complete (25781 records). [2025-10-20T20:00:12.256107] [DEBUG] [VALIDATOR ] MSA: Validating primary data. [2025-10-20T20:00:12.618829] [DEBUG] [VALIDATOR ] MSA: All expected trading days are present. [2025-10-20T20:00:12.618906] [DEBUG] [VALIDATOR ] MSA: Primary data validation passed. [2025-10-20T20:00:12.619439] [INFO ] [MAIN ] MSA: Fetched primary data is valid. [2025-10-20T20:00:12.619477] [DEBUG] [LOCAL ] MSA: Saving primary data. [2025-10-20T20:00:12.620140] [DEBUG] [LOCAL ] MSA: Trimmed 25781 records to 18915 (keeping 200 days of data) [2025-10-20T20:00:12.734790] [DEBUG] [LOCAL ] MSA: Primary data saved (18915 records). [2025-10-20T20:00:12.735036] [INFO ] [MAIN ] MSA: Full history saved. [2025-10-20T20:00:12.735143] [INFO ] [MAIN ] Update Progress: 1399/2367 symbols completed. MSA took 16.6s. Estimated remaining: 2h 4m 39s. Rate: 7.8 symbols/min [2025-10-20T20:00:12.932311] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T20:00:12.943558] [INFO ] [REMOTE ] MS: Fetching primary data for 2025-10. [2025-10-20T20:00:14.974238] [INFO ] [REMOTE ] MS: Incremental fetch complete for 2025-10 (2.03 seconds). [2025-10-20T20:00:14.982229] [INFO ] [REMOTE ] MS: Full fetch complete (77858 records). [2025-10-20T20:00:14.982421] [DEBUG] [VALIDATOR ] MS: Validating primary data. [2025-10-20T20:00:15.380806] [DEBUG] [VALIDATOR ] MS: All expected trading days are present. [2025-10-20T20:00:15.380902] [DEBUG] [VALIDATOR ] MS: Primary data validation passed. [2025-10-20T20:00:15.382102] [INFO ] [MAIN ] MS: Fetched primary data is valid. [2025-10-20T20:00:15.382189] [DEBUG] [LOCAL ] MS: Saving primary data. [2025-10-20T20:00:15.383739] [DEBUG] [LOCAL ] MS: Trimmed 77858 records to 53688 (keeping 200 days of data) [2025-10-20T20:00:15.720626] [DEBUG] [LOCAL ] MS: Primary data saved (53688 records). [2025-10-20T20:00:15.720884] [INFO ] [MAIN ] MS: Full history saved. [2025-10-20T20:00:15.721012] [INFO ] [MAIN ] Update Progress: 1400/2367 symbols completed. MS took 32.1s. Estimated remaining: 2h 4m 28s. Rate: 7.8 symbols/min [2025-10-20T20:00:16.321633] [INFO ] [MAIN ] MSCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:16.466002] [INFO ] [MAIN ] MSB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:16.501508] [INFO ] [MAIN ] MSD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:16.505193] [INFO ] [MAIN ] MSC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:16.506045] [DEBUG] [LOCAL ] MSC: No existing data file found in primary storage. [2025-10-20T20:00:16.506098] [WARN ] [MAIN ] MSC: No existing data found, performing full refresh instead. [2025-10-20T20:00:16.506220] [INFO ] [MAIN ] MSC: Wiping existing primary data before full refresh. [2025-10-20T20:00:16.506272] [INFO ] [LOCAL ] MSC: Starting deletion of primary data. [2025-10-20T20:00:16.506314] [DEBUG] [LOCAL ] MSC: No primary data file found to delete. [2025-10-20T20:00:16.506345] [INFO ] [MAIN ] MSC: Starting full primary refresh. [2025-10-20T20:00:16.637482] [DEBUG] [LOCAL ] MSB: Loaded 7917 existing records from primary storage. [2025-10-20T20:00:16.685589] [INFO ] [MAIN ] MSB: Loaded 7917 existing records. [2025-10-20T20:00:16.692209] [DEBUG] [LOCAL ] MSD: Loaded 16810 existing records from primary storage. [2025-10-20T20:00:16.732995] [INFO ] [MAIN ] MSD: Loaded 16810 existing records. [2025-10-20T20:00:16.899604] [DEBUG] [LOCAL ] MSCI: Loaded 34965 existing records from primary storage. [2025-10-20T20:00:17.011720] [INFO ] [MAIN ] MSCI: Loaded 34965 existing records. [2025-10-20T20:00:17.306059] [WARN ] [MAIN ] MSB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:17.358969] [INFO ] [MAIN ] MSB: Wiping existing primary data before full refresh. [2025-10-20T20:00:17.367566] [INFO ] [LOCAL ] MSB: Starting deletion of primary data. [2025-10-20T20:00:17.408074] [WARN ] [MAIN ] MSD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:17.419519] [DEBUG] [LOCAL ] MSB: Deleted primary data file. [2025-10-20T20:00:17.468676] [INFO ] [MAIN ] MSB: Starting full primary refresh. [2025-10-20T20:00:17.468604] [INFO ] [REMOTE ] MSC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:17.483282] [INFO ] [MAIN ] MSD: Wiping existing primary data before full refresh. [2025-10-20T20:00:17.577896] [WARN ] [MAIN ] MSCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:17.588611] [INFO ] [MAIN ] MSCI: Wiping existing primary data before full refresh. [2025-10-20T20:00:17.563315] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-01. [2025-10-20T20:00:17.574258] [INFO ] [LOCAL ] MSD: Starting deletion of primary data. [2025-10-20T20:00:17.619696] [INFO ] [LOCAL ] MSCI: Starting deletion of primary data. [2025-10-20T20:00:17.624036] [DEBUG] [LOCAL ] MSD: Deleted primary data file. [2025-10-20T20:00:17.629417] [INFO ] [MAIN ] MSD: Starting full primary refresh. [2025-10-20T20:00:17.623985] [DEBUG] [LOCAL ] MSCI: Deleted primary data file. [2025-10-20T20:00:17.651873] [INFO ] [MAIN ] MSCI: Starting full primary refresh. [2025-10-20T20:00:18.392004] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T20:00:18.441027] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-02. [2025-10-20T20:00:18.518380] [INFO ] [REMOTE ] MSB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:18.524837] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-01. [2025-10-20T20:00:18.661752] [INFO ] [REMOTE ] MSD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:18.662073] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-01. [2025-10-20T20:00:18.740899] [INFO ] [REMOTE ] MSCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:18.741234] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-01. [2025-10-20T20:00:19.038432] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T20:00:19.040038] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-03. [2025-10-20T20:00:19.450899] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T20:00:19.453060] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T20:00:19.452467] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-04. [2025-10-20T20:00:19.457858] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-02. [2025-10-20T20:00:19.579529] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T20:00:19.582904] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-02. [2025-10-20T20:00:19.799337] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T20:00:19.879826] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T20:00:20.251381] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-02. [2025-10-20T20:00:20.271197] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T20:00:20.364917] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:00:21.051387] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-03. [2025-10-20T20:00:21.723837] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T20:00:21.851399] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-05. [2025-10-20T20:00:21.886492] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T20:00:22.651361] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-03. [2025-10-20T20:00:22.886696] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T20:00:23.451358] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-06. [2025-10-20T20:00:23.596624] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T20:00:23.761888] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-06 (0.31 seconds). [2025-10-20T20:00:24.251391] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-04. [2025-10-20T20:00:25.051424] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-04. [2025-10-20T20:00:25.852332] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-07. [2025-10-20T20:00:25.854149] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T20:00:26.006327] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T20:00:26.276762] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T20:00:26.651384] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-05. [2025-10-20T20:00:27.451361] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-03. [2025-10-20T20:00:27.714997] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T20:00:28.251370] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-06. [2025-10-20T20:00:29.024582] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T20:00:29.051399] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-05. [2025-10-20T20:00:29.077636] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T20:00:29.820870] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T20:00:29.851326] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-06. [2025-10-20T20:00:30.651388] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-07. [2025-10-20T20:00:30.679576] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T20:00:31.451374] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-04. [2025-10-20T20:00:31.623209] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T20:00:32.251379] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-07. [2025-10-20T20:00:33.051440] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-08. [2025-10-20T20:00:33.066033] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T20:00:33.126897] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:00:33.851347] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-08. [2025-10-20T20:00:34.184321] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T20:00:34.635023] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T20:00:34.651418] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-09. [2025-10-20T20:00:35.451458] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-05. [2025-10-20T20:00:35.564675] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T20:00:36.251468] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-08. [2025-10-20T20:00:36.635932] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T20:00:36.699039] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T20:00:37.051426] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-09. [2025-10-20T20:00:37.548630] [INFO ] [REMOTE ] MSC: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T20:00:37.851432] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-09. [2025-10-20T20:00:38.651426] [INFO ] [REMOTE ] MSD: Fetching primary data for 2025-10. [2025-10-20T20:00:38.665032] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:00:39.451444] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-06. [2025-10-20T20:00:39.864490] [INFO ] [REMOTE ] MSD: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T20:00:39.868128] [INFO ] [REMOTE ] MSD: Full fetch complete (23094 records). [2025-10-20T20:00:39.868229] [DEBUG] [VALIDATOR ] MSD: Validating primary data. [2025-10-20T20:00:40.251402] [INFO ] [REMOTE ] MSB: Fetching primary data for 2025-10. [2025-10-20T20:00:40.257775] [DEBUG] [VALIDATOR ] MSD: All expected trading days are present. [2025-10-20T20:00:40.258619] [DEBUG] [VALIDATOR ] MSD: Primary data validation passed. [2025-10-20T20:00:40.259161] [INFO ] [MAIN ] MSD: Fetched primary data is valid. [2025-10-20T20:00:40.259213] [DEBUG] [LOCAL ] MSD: Saving primary data. [2025-10-20T20:00:40.260100] [DEBUG] [LOCAL ] MSD: Trimmed 23094 records to 16558 (keeping 200 days of data) [2025-10-20T20:00:40.358438] [DEBUG] [LOCAL ] MSD: Primary data saved (16558 records). [2025-10-20T20:00:40.358577] [INFO ] [MAIN ] MSD: Full history saved. [2025-10-20T20:00:40.358692] [INFO ] [MAIN ] Update Progress: 1401/2367 symbols completed. MSD took 24.6s. Estimated remaining: 2h 4m 31s. Rate: 7.8 symbols/min [2025-10-20T20:00:40.532874] [INFO ] [MAIN ] MSDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:40.572060] [DEBUG] [LOCAL ] MSDL: Loaded 39331 existing records from primary storage. [2025-10-20T20:00:40.572378] [INFO ] [MAIN ] MSDL: Loaded 39331 existing records. [2025-10-20T20:00:40.748684] [WARN ] [MAIN ] MSDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:40.748947] [INFO ] [MAIN ] MSDL: Wiping existing primary data before full refresh. [2025-10-20T20:00:40.749133] [INFO ] [LOCAL ] MSDL: Starting deletion of primary data. [2025-10-20T20:00:40.749397] [DEBUG] [LOCAL ] MSDL: Deleted primary data file. [2025-10-20T20:00:40.749502] [INFO ] [MAIN ] MSDL: Starting full primary refresh. [2025-10-20T20:00:40.912920] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:00:41.060095] [INFO ] [REMOTE ] MSC: Fetching primary data for 2025-10. [2025-10-20T20:00:41.142200] [INFO ] [REMOTE ] MSDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:41.168251] [INFO ] [REMOTE ] MSB: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T20:00:41.169878] [INFO ] [REMOTE ] MSB: Full fetch complete (13127 records). [2025-10-20T20:00:41.169947] [DEBUG] [VALIDATOR ] MSB: Validating primary data. [2025-10-20T20:00:41.202613] [DEBUG] [REMOTE ] MSC: Error fetching primary data for 2025-10: 'API call returned an empty response.' [2025-10-20T20:00:41.240580] [WARN ] [REMOTE ] MSC: No data returned for month 2025-10 [2025-10-20T20:00:41.264174] [FATAL] [REMOTE ] MSC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:00:41.274903] [FATAL] [MAIN ] MSC: Failed to fetch primary data. [2025-10-20T20:00:41.285895] [WARN ] [MAIN ] MSC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:00:41.296232] [INFO ] [LOCAL ] MSC: Starting deletion of primary data. [2025-10-20T20:00:41.313631] [DEBUG] [LOCAL ] MSC: No primary data file found to delete. [2025-10-20T20:00:41.318474] [INFO ] [LOCAL ] MSC: Starting deletion of derivative data. [2025-10-20T20:00:41.323941] [DEBUG] [LOCAL ] MSC: No derivative data file found to delete. [2025-10-20T20:00:41.326093] [INFO ] [MAIN ] Update Progress: 1402/2367 symbols completed. MSC took 25.6s. Estimated remaining: 2h 4m 19s. Rate: 7.8 symbols/min [2025-10-20T20:00:41.710063] [INFO ] [MAIN ] MSGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:41.735490] [DEBUG] [VALIDATOR ] MSB: All expected trading days are present. [2025-10-20T20:00:41.735573] [DEBUG] [VALIDATOR ] MSB: Primary data validation passed. [2025-10-20T20:00:41.736004] [INFO ] [MAIN ] MSB: Fetched primary data is valid. [2025-10-20T20:00:41.736065] [DEBUG] [LOCAL ] MSB: Saving primary data. [2025-10-20T20:00:41.736757] [DEBUG] [LOCAL ] MSB: Trimmed 13127 records to 7950 (keeping 200 days of data) [2025-10-20T20:00:41.791973] [DEBUG] [LOCAL ] MSB: Primary data saved (7950 records). [2025-10-20T20:00:41.792133] [INFO ] [MAIN ] MSB: Full history saved. [2025-10-20T20:00:41.792219] [INFO ] [MAIN ] Update Progress: 1403/2367 symbols completed. MSB took 26.1s. Estimated remaining: 2h 4m 6s. Rate: 7.8 symbols/min [2025-10-20T20:00:41.857082] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-07. [2025-10-20T20:00:41.910849] [DEBUG] [LOCAL ] MSGE: Loaded 27423 existing records from primary storage. [2025-10-20T20:00:41.938242] [INFO ] [MAIN ] MSGE: Loaded 27423 existing records. [2025-10-20T20:00:42.061043] [INFO ] [MAIN ] MSGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:42.172326] [WARN ] [MAIN ] MSGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:42.172620] [INFO ] [MAIN ] MSGE: Wiping existing primary data before full refresh. [2025-10-20T20:00:42.172700] [INFO ] [LOCAL ] MSGE: Starting deletion of primary data. [2025-10-20T20:00:42.172885] [DEBUG] [LOCAL ] MSGE: Deleted primary data file. [2025-10-20T20:00:42.173263] [INFO ] [MAIN ] MSGE: Starting full primary refresh. [2025-10-20T20:00:42.261462] [DEBUG] [LOCAL ] MSGS: Loaded 17627 existing records from primary storage. [2025-10-20T20:00:42.283059] [INFO ] [MAIN ] MSGS: Loaded 17627 existing records. [2025-10-20T20:00:42.692032] [WARN ] [MAIN ] MSGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:00:42.657839] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-01. [2025-10-20T20:00:42.709734] [INFO ] [MAIN ] MSGS: Wiping existing primary data before full refresh. [2025-10-20T20:00:42.740837] [INFO ] [LOCAL ] MSGS: Starting deletion of primary data. [2025-10-20T20:00:42.756633] [DEBUG] [LOCAL ] MSGS: Deleted primary data file. [2025-10-20T20:00:42.759196] [INFO ] [MAIN ] MSGS: Starting full primary refresh. [2025-10-20T20:00:42.782291] [INFO ] [REMOTE ] MSGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:43.161997] [INFO ] [REMOTE ] MSGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:43.451362] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-01. [2025-10-20T20:00:43.564005] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T20:00:44.009363] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:00:44.251405] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-08. [2025-10-20T20:00:44.742006] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T20:00:45.051466] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-02. [2025-10-20T20:00:45.726484] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T20:00:45.851383] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-09. [2025-10-20T20:00:46.133819] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T20:00:46.651449] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-01. [2025-10-20T20:00:47.451410] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-03. [2025-10-20T20:00:47.520097] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T20:00:47.539252] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:00:48.251432] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-02. [2025-10-20T20:00:48.888938] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T20:00:49.051448] [INFO ] [REMOTE ] MSCI: Fetching primary data for 2025-10. [2025-10-20T20:00:49.292918] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T20:00:49.851391] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-02. [2025-10-20T20:00:50.546747] [INFO ] [REMOTE ] MSCI: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T20:00:50.551257] [INFO ] [REMOTE ] MSCI: Full fetch complete (50577 records). [2025-10-20T20:00:50.551344] [DEBUG] [VALIDATOR ] MSCI: Validating primary data. [2025-10-20T20:00:50.652262] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-04. [2025-10-20T20:00:50.941634] [DEBUG] [VALIDATOR ] MSCI: All expected trading days are present. [2025-10-20T20:00:50.941739] [DEBUG] [VALIDATOR ] MSCI: Primary data validation passed. [2025-10-20T20:00:50.942518] [INFO ] [MAIN ] MSCI: Fetched primary data is valid. [2025-10-20T20:00:50.942557] [DEBUG] [LOCAL ] MSCI: Saving primary data. [2025-10-20T20:00:50.943499] [DEBUG] [LOCAL ] MSCI: Trimmed 50577 records to 34427 (keeping 200 days of data) [2025-10-20T20:00:51.052897] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T20:00:51.181855] [DEBUG] [LOCAL ] MSCI: Primary data saved (34427 records). [2025-10-20T20:00:51.182004] [INFO ] [MAIN ] MSCI: Full history saved. [2025-10-20T20:00:51.182115] [INFO ] [MAIN ] Update Progress: 1404/2367 symbols completed. MSCI took 35.4s. Estimated remaining: 2h 3m 60s. Rate: 7.8 symbols/min [2025-10-20T20:00:51.355459] [INFO ] [MAIN ] MSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:00:51.355623] [DEBUG] [LOCAL ] MSI: No existing data file found in primary storage. [2025-10-20T20:00:51.355695] [WARN ] [MAIN ] MSI: No existing data found, performing full refresh instead. [2025-10-20T20:00:51.355810] [INFO ] [MAIN ] MSI: Wiping existing primary data before full refresh. [2025-10-20T20:00:51.355853] [INFO ] [LOCAL ] MSI: Starting deletion of primary data. [2025-10-20T20:00:51.355894] [DEBUG] [LOCAL ] MSI: No primary data file found to delete. [2025-10-20T20:00:51.355927] [INFO ] [MAIN ] MSI: Starting full primary refresh. [2025-10-20T20:00:51.456626] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-03. [2025-10-20T20:00:51.731183] [INFO ] [REMOTE ] MSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:00:52.229850] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T20:00:52.251322] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-05. [2025-10-20T20:00:52.483918] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:00:53.051382] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-04. [2025-10-20T20:00:53.689349] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T20:00:53.851368] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-03. [2025-10-20T20:00:54.020551] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T20:00:54.651343] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-01. [2025-10-20T20:00:55.118847] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-03 (1.27 seconds). [2025-10-20T20:00:55.451359] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-05. [2025-10-20T20:00:56.259934] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-06. [2025-10-20T20:00:56.264412] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T20:00:56.432374] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T20:00:57.051380] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-04. [2025-10-20T20:00:57.742986] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T20:00:57.851393] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-02. [2025-10-20T20:00:57.887677] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T20:00:58.651399] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-05. [2025-10-20T20:00:59.372072] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T20:00:59.451422] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-06. [2025-10-20T20:00:59.796992] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T20:01:00.251392] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-06. [2025-10-20T20:01:00.533919] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T20:01:01.051375] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-03. [2025-10-20T20:01:01.563434] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T20:01:01.851410] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-07. [2025-10-20T20:01:02.651429] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-07. [2025-10-20T20:01:02.985196] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:01:03.064713] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T20:01:03.451374] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-08. [2025-10-20T20:01:03.787159] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T20:01:04.251450] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-08. [2025-10-20T20:01:04.608726] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:01:05.051401] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-04. [2025-10-20T20:01:05.851345] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-07. [2025-10-20T20:01:05.885994] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T20:01:06.651395] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-09. [2025-10-20T20:01:06.796364] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:01:06.895881] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T20:01:07.451421] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-09. [2025-10-20T20:01:08.113974] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T20:01:08.251375] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-05. [2025-10-20T20:01:08.740608] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T20:01:09.051396] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-08. [2025-10-20T20:01:09.796533] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T20:01:09.851337] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-09. [2025-10-20T20:01:09.957281] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T20:01:10.651356] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-06. [2025-10-20T20:01:10.675951] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:01:11.451383] [INFO ] [REMOTE ] MSGE: Fetching primary data for 2025-10. [2025-10-20T20:01:12.178821] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T20:01:12.251434] [INFO ] [REMOTE ] MSGS: Fetching primary data for 2025-10. [2025-10-20T20:01:12.875500] [INFO ] [REMOTE ] MSGE: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T20:01:12.883274] [INFO ] [REMOTE ] MSGE: Full fetch complete (40205 records). [2025-10-20T20:01:12.883507] [DEBUG] [VALIDATOR ] MSGE: Validating primary data. [2025-10-20T20:01:13.058269] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-07. [2025-10-20T20:01:13.342342] [INFO ] [REMOTE ] MSGS: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T20:01:13.344830] [DEBUG] [VALIDATOR ] MSGE: All expected trading days are present. [2025-10-20T20:01:13.348602] [DEBUG] [VALIDATOR ] MSGE: Primary data validation passed. [2025-10-20T20:01:13.349469] [INFO ] [MAIN ] MSGE: Fetched primary data is valid. [2025-10-20T20:01:13.349554] [DEBUG] [LOCAL ] MSGE: Saving primary data. [2025-10-20T20:01:13.350043] [INFO ] [REMOTE ] MSGS: Full fetch complete (23764 records). [2025-10-20T20:01:13.350710] [DEBUG] [VALIDATOR ] MSGS: Validating primary data. [2025-10-20T20:01:13.383342] [DEBUG] [LOCAL ] MSGE: Trimmed 40205 records to 27014 (keeping 200 days of data) [2025-10-20T20:01:13.870521] [INFO ] [REMOTE ] MSDL: Fetching primary data for 2025-10. [2025-10-20T20:01:13.902195] [DEBUG] [VALIDATOR ] MSGS: All expected trading days are present. [2025-10-20T20:01:13.903009] [DEBUG] [VALIDATOR ] MSGS: Primary data validation passed. [2025-10-20T20:01:13.907363] [DEBUG] [LOCAL ] MSGE: Primary data saved (27014 records). [2025-10-20T20:01:13.908371] [INFO ] [MAIN ] MSGS: Fetched primary data is valid. [2025-10-20T20:01:13.908510] [INFO ] [MAIN ] MSGE: Full history saved. [2025-10-20T20:01:13.908624] [INFO ] [MAIN ] Update Progress: 1405/2367 symbols completed. MSGE took 32.6s. Estimated remaining: 2h 4m 2s. Rate: 7.8 symbols/min [2025-10-20T20:01:13.908736] [DEBUG] [LOCAL ] MSGS: Saving primary data. [2025-10-20T20:01:13.954143] [DEBUG] [LOCAL ] MSGS: Trimmed 23764 records to 17291 (keeping 200 days of data) [2025-10-20T20:01:14.174996] [INFO ] [MAIN ] MSIF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:14.184213] [DEBUG] [LOCAL ] MSIF: No existing data file found in primary storage. [2025-10-20T20:01:14.188851] [WARN ] [MAIN ] MSIF: No existing data found, performing full refresh instead. [2025-10-20T20:01:14.192565] [INFO ] [MAIN ] MSIF: Wiping existing primary data before full refresh. [2025-10-20T20:01:14.194286] [INFO ] [LOCAL ] MSIF: Starting deletion of primary data. [2025-10-20T20:01:14.197053] [DEBUG] [LOCAL ] MSGS: Primary data saved (17291 records). [2025-10-20T20:01:14.197161] [INFO ] [MAIN ] MSGS: Full history saved. [2025-10-20T20:01:14.194926] [DEBUG] [LOCAL ] MSIF: No primary data file found to delete. [2025-10-20T20:01:14.197352] [INFO ] [MAIN ] Update Progress: 1406/2367 symbols completed. MSGS took 32.4s. Estimated remaining: 2h 3m 49s. Rate: 7.8 symbols/min [2025-10-20T20:01:14.197258] [INFO ] [MAIN ] MSIF: Starting full primary refresh. [2025-10-20T20:01:14.588400] [INFO ] [MAIN ] MSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:14.747761] [INFO ] [REMOTE ] MSIF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:14.748218] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-01. [2025-10-20T20:01:14.778478] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:01:14.810035] [DEBUG] [LOCAL ] MSM: Loaded 36195 existing records from primary storage. [2025-10-20T20:01:14.810299] [INFO ] [MAIN ] MSM: Loaded 36195 existing records. [2025-10-20T20:01:14.983678] [WARN ] [MAIN ] MSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:14.983872] [INFO ] [MAIN ] MSM: Wiping existing primary data before full refresh. [2025-10-20T20:01:14.983950] [INFO ] [LOCAL ] MSM: Starting deletion of primary data. [2025-10-20T20:01:14.984154] [DEBUG] [LOCAL ] MSM: Deleted primary data file. [2025-10-20T20:01:14.984208] [INFO ] [MAIN ] MSM: Starting full primary refresh. [2025-10-20T20:01:15.265510] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T20:01:15.344971] [INFO ] [REMOTE ] MSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:15.451454] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-01. [2025-10-20T20:01:15.573304] [INFO ] [REMOTE ] MSDL: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T20:01:15.580112] [INFO ] [REMOTE ] MSDL: Full fetch complete (56209 records). [2025-10-20T20:01:15.580245] [DEBUG] [VALIDATOR ] MSDL: Validating primary data. [2025-10-20T20:01:15.966908] [DEBUG] [VALIDATOR ] MSDL: All expected trading days are present. [2025-10-20T20:01:15.966999] [DEBUG] [VALIDATOR ] MSDL: Primary data validation passed. [2025-10-20T20:01:15.968003] [INFO ] [MAIN ] MSDL: Fetched primary data is valid. [2025-10-20T20:01:15.968071] [DEBUG] [LOCAL ] MSDL: Saving primary data. [2025-10-20T20:01:15.969417] [DEBUG] [LOCAL ] MSDL: Trimmed 56209 records to 38684 (keeping 200 days of data) [2025-10-20T20:01:16.203859] [DEBUG] [LOCAL ] MSDL: Primary data saved (38684 records). [2025-10-20T20:01:16.204002] [INFO ] [MAIN ] MSDL: Full history saved. [2025-10-20T20:01:16.204354] [INFO ] [MAIN ] Update Progress: 1407/2367 symbols completed. MSDL took 35.8s. Estimated remaining: 2h 3m 38s. Rate: 7.8 symbols/min [2025-10-20T20:01:16.257777] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-02. [2025-10-20T20:01:16.378741] [INFO ] [MAIN ] MT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:16.428839] [DEBUG] [LOCAL ] MT: Loaded 51241 existing records from primary storage. [2025-10-20T20:01:16.429169] [INFO ] [MAIN ] MT: Loaded 51241 existing records. [2025-10-20T20:01:16.607262] [WARN ] [MAIN ] MT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:16.607537] [INFO ] [MAIN ] MT: Wiping existing primary data before full refresh. [2025-10-20T20:01:16.607614] [INFO ] [LOCAL ] MT: Starting deletion of primary data. [2025-10-20T20:01:16.607888] [DEBUG] [LOCAL ] MT: Deleted primary data file. [2025-10-20T20:01:16.608151] [INFO ] [MAIN ] MT: Starting full primary refresh. [2025-10-20T20:01:16.966232] [INFO ] [REMOTE ] MT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:17.004872] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T20:01:17.051360] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-01. [2025-10-20T20:01:17.145541] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T20:01:17.851372] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-02. [2025-10-20T20:01:18.651356] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-03. [2025-10-20T20:01:18.724890] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T20:01:19.191026] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:01:19.242082] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T20:01:19.451433] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-03. [2025-10-20T20:01:20.251388] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-04. [2025-10-20T20:01:20.879115] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T20:01:20.956794] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T20:01:21.051371] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-05. [2025-10-20T20:01:21.650141] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T20:01:21.851368] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-02. [2025-10-20T20:01:22.651432] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-08. [2025-10-20T20:01:23.451376] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-06. [2025-10-20T20:01:23.618180] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T20:01:24.004803] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T20:01:24.251351] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-09. [2025-10-20T20:01:24.396916] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T20:01:25.051409] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-03. [2025-10-20T20:01:25.784230] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T20:01:25.851347] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-04. [2025-10-20T20:01:26.651413] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-07. [2025-10-20T20:01:27.011482] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T20:01:27.381571] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T20:01:27.451425] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-05. [2025-10-20T20:01:27.503520] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T20:01:28.251471] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-04. [2025-10-20T20:01:28.846195] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T20:01:29.051390] [INFO ] [REMOTE ] MSI: Fetching primary data for 2025-10. [2025-10-20T20:01:29.851430] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-08. [2025-10-20T20:01:30.077775] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:01:30.586890] [INFO ] [REMOTE ] MSI: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T20:01:30.592406] [INFO ] [REMOTE ] MSI: Full fetch complete (58114 records). [2025-10-20T20:01:30.592534] [DEBUG] [VALIDATOR ] MSI: Validating primary data. [2025-10-20T20:01:30.656564] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-06. [2025-10-20T20:01:30.985609] [DEBUG] [VALIDATOR ] MSI: All expected trading days are present. [2025-10-20T20:01:30.985752] [DEBUG] [VALIDATOR ] MSI: Primary data validation passed. [2025-10-20T20:01:30.986818] [INFO ] [MAIN ] MSI: Fetched primary data is valid. [2025-10-20T20:01:30.986999] [DEBUG] [LOCAL ] MSI: Saving primary data. [2025-10-20T20:01:31.003431] [DEBUG] [LOCAL ] MSI: Trimmed 58114 records to 39863 (keeping 200 days of data) [2025-10-20T20:01:31.003291] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T20:01:31.255198] [DEBUG] [LOCAL ] MSI: Primary data saved (39863 records). [2025-10-20T20:01:31.255364] [INFO ] [MAIN ] MSI: Full history saved. [2025-10-20T20:01:31.255444] [INFO ] [MAIN ] Update Progress: 1408/2367 symbols completed. MSI took 40.1s. Estimated remaining: 2h 3m 35s. Rate: 7.8 symbols/min [2025-10-20T20:01:31.422301] [INFO ] [MAIN ] MTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:31.451396] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-09. [2025-10-20T20:01:31.470305] [DEBUG] [LOCAL ] MTB: Loaded 47322 existing records from primary storage. [2025-10-20T20:01:31.470617] [INFO ] [MAIN ] MTB: Loaded 47322 existing records. [2025-10-20T20:01:31.649519] [WARN ] [MAIN ] MTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:31.649752] [INFO ] [MAIN ] MTB: Wiping existing primary data before full refresh. [2025-10-20T20:01:31.649807] [INFO ] [LOCAL ] MTB: Starting deletion of primary data. [2025-10-20T20:01:31.650128] [DEBUG] [LOCAL ] MTB: Deleted primary data file. [2025-10-20T20:01:31.650165] [INFO ] [MAIN ] MTB: Starting full primary refresh. [2025-10-20T20:01:32.019999] [INFO ] [REMOTE ] MTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:32.097144] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T20:01:32.251358] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-07. [2025-10-20T20:01:32.530554] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T20:01:33.051429] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-01. [2025-10-20T20:01:33.761134] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T20:01:33.851334] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-08. [2025-10-20T20:01:34.651342] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-05. [2025-10-20T20:01:34.817877] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:01:35.033191] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-08 (1.18 seconds). [2025-10-20T20:01:35.451383] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-09. [2025-10-20T20:01:36.259580] [INFO ] [REMOTE ] MSIF: Fetching primary data for 2025-10. [2025-10-20T20:01:36.267275] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:01:37.003080] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T20:01:37.051381] [INFO ] [REMOTE ] MSM: Fetching primary data for 2025-10. [2025-10-20T20:01:37.559976] [INFO ] [REMOTE ] MSIF: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T20:01:37.563419] [INFO ] [REMOTE ] MSIF: Full fetch complete (18988 records). [2025-10-20T20:01:37.563494] [DEBUG] [VALIDATOR ] MSIF: Validating primary data. [2025-10-20T20:01:37.759509] [WARN ] [VALIDATOR ] MSIF: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T20:01:37.759860] [FATAL] [MAIN ] MSIF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:01:37.759929] [WARN ] [MAIN ] MSIF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:01:37.759998] [INFO ] [LOCAL ] MSIF: Starting deletion of primary data. [2025-10-20T20:01:37.760061] [DEBUG] [LOCAL ] MSIF: No primary data file found to delete. [2025-10-20T20:01:37.760099] [INFO ] [LOCAL ] MSIF: Starting deletion of derivative data. [2025-10-20T20:01:37.760139] [DEBUG] [LOCAL ] MSIF: No derivative data file found to delete. [2025-10-20T20:01:37.760196] [INFO ] [MAIN ] Update Progress: 1409/2367 symbols completed. MSIF took 23.9s. Estimated remaining: 2h 3m 27s. Rate: 7.8 symbols/min [2025-10-20T20:01:37.856447] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-02. [2025-10-20T20:01:37.947622] [INFO ] [MAIN ] MTD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:37.965480] [DEBUG] [LOCAL ] MTD: Loaded 15618 existing records from primary storage. [2025-10-20T20:01:37.965754] [INFO ] [MAIN ] MTD: Loaded 15618 existing records. [2025-10-20T20:01:38.147053] [WARN ] [MAIN ] MTD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:38.147269] [INFO ] [MAIN ] MTD: Wiping existing primary data before full refresh. [2025-10-20T20:01:38.147316] [INFO ] [LOCAL ] MTD: Starting deletion of primary data. [2025-10-20T20:01:38.147522] [DEBUG] [LOCAL ] MTD: Deleted primary data file. [2025-10-20T20:01:38.147578] [INFO ] [MAIN ] MTD: Starting full primary refresh. [2025-10-20T20:01:38.549129] [INFO ] [REMOTE ] MTD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:38.543417] [INFO ] [REMOTE ] MSM: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T20:01:38.554957] [INFO ] [REMOTE ] MSM: Full fetch complete (52048 records). [2025-10-20T20:01:38.555043] [DEBUG] [VALIDATOR ] MSM: Validating primary data. [2025-10-20T20:01:38.656463] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-01. [2025-10-20T20:01:38.939671] [DEBUG] [VALIDATOR ] MSM: All expected trading days are present. [2025-10-20T20:01:38.939765] [DEBUG] [VALIDATOR ] MSM: Primary data validation passed. [2025-10-20T20:01:38.940568] [INFO ] [MAIN ] MSM: Fetched primary data is valid. [2025-10-20T20:01:38.940608] [DEBUG] [LOCAL ] MSM: Saving primary data. [2025-10-20T20:01:38.941701] [DEBUG] [LOCAL ] MSM: Trimmed 52048 records to 35537 (keeping 200 days of data) [2025-10-20T20:01:39.158884] [DEBUG] [LOCAL ] MSM: Primary data saved (35537 records). [2025-10-20T20:01:39.159099] [INFO ] [MAIN ] MSM: Full history saved. [2025-10-20T20:01:39.159211] [INFO ] [MAIN ] Update Progress: 1410/2367 symbols completed. MSM took 24.9s. Estimated remaining: 2h 3m 15s. Rate: 7.8 symbols/min [2025-10-20T20:01:39.290845] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:01:39.362618] [INFO ] [MAIN ] MTDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:39.412302] [DEBUG] [LOCAL ] MTDR: Loaded 52441 existing records from primary storage. [2025-10-20T20:01:39.412628] [INFO ] [MAIN ] MTDR: Loaded 52441 existing records. [2025-10-20T20:01:39.456053] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-03. [2025-10-20T20:01:39.593235] [WARN ] [MAIN ] MTDR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:39.593504] [INFO ] [MAIN ] MTDR: Wiping existing primary data before full refresh. [2025-10-20T20:01:39.593662] [INFO ] [LOCAL ] MTDR: Starting deletion of primary data. [2025-10-20T20:01:39.601027] [DEBUG] [LOCAL ] MTDR: Deleted primary data file. [2025-10-20T20:01:39.601536] [INFO ] [MAIN ] MTDR: Starting full primary refresh. [2025-10-20T20:01:39.601823] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T20:01:39.954727] [INFO ] [REMOTE ] MTDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:40.251408] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-01. [2025-10-20T20:01:41.051350] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-06. [2025-10-20T20:01:41.190150] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:01:41.851407] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-02. [2025-10-20T20:01:41.988842] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T20:01:42.631477] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T20:01:42.651333] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-07. [2025-10-20T20:01:42.880390] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T20:01:43.451423] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-04. [2025-10-20T20:01:44.251379] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-03. [2025-10-20T20:01:44.298385] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T20:01:45.051368] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-02. [2025-10-20T20:01:45.286391] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:01:45.326216] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T20:01:45.851411] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-04. [2025-10-20T20:01:46.630373] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:01:46.651347] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-03. [2025-10-20T20:01:47.076856] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T20:01:47.451372] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-08. [2025-10-20T20:01:48.251392] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-05. [2025-10-20T20:01:48.418756] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T20:01:48.832430] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T20:01:49.051393] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-09. [2025-10-20T20:01:49.086718] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T20:01:49.851366] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-04. [2025-10-20T20:01:50.606233] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T20:01:50.651392] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-06. [2025-10-20T20:01:51.451349] [INFO ] [REMOTE ] MT: Fetching primary data for 2025-10. [2025-10-20T20:01:51.623094] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T20:01:51.629000] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T20:01:52.251371] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-05. [2025-10-20T20:01:52.921203] [INFO ] [REMOTE ] MT: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T20:01:52.929140] [INFO ] [REMOTE ] MT: Full fetch complete (73785 records). [2025-10-20T20:01:52.929293] [DEBUG] [VALIDATOR ] MT: Validating primary data. [2025-10-20T20:01:53.056434] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-05. [2025-10-20T20:01:53.338153] [DEBUG] [VALIDATOR ] MT: All expected trading days are present. [2025-10-20T20:01:53.338235] [DEBUG] [VALIDATOR ] MT: Primary data validation passed. [2025-10-20T20:01:53.339299] [INFO ] [MAIN ] MT: Fetched primary data is valid. [2025-10-20T20:01:53.339361] [DEBUG] [LOCAL ] MT: Saving primary data. [2025-10-20T20:01:53.340759] [DEBUG] [LOCAL ] MT: Trimmed 73785 records to 50451 (keeping 200 days of data) [2025-10-20T20:01:53.634033] [DEBUG] [LOCAL ] MT: Primary data saved (50451 records). [2025-10-20T20:01:53.634183] [INFO ] [MAIN ] MT: Full history saved. [2025-10-20T20:01:53.634523] [INFO ] [MAIN ] Update Progress: 1411/2367 symbols completed. MT took 37.4s. Estimated remaining: 2h 3m 11s. Rate: 7.8 symbols/min [2025-10-20T20:01:53.811496] [INFO ] [MAIN ] MTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:01:53.863227] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-07. [2025-10-20T20:01:53.867246] [DEBUG] [LOCAL ] MTG: Loaded 51857 existing records from primary storage. [2025-10-20T20:01:53.867811] [INFO ] [MAIN ] MTG: Loaded 51857 existing records. [2025-10-20T20:01:54.058765] [WARN ] [MAIN ] MTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:01:54.059061] [INFO ] [MAIN ] MTG: Wiping existing primary data before full refresh. [2025-10-20T20:01:54.059198] [INFO ] [LOCAL ] MTG: Starting deletion of primary data. [2025-10-20T20:01:54.068418] [DEBUG] [LOCAL ] MTG: Deleted primary data file. [2025-10-20T20:01:54.084886] [INFO ] [MAIN ] MTG: Starting full primary refresh. [2025-10-20T20:01:54.133464] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T20:01:54.477722] [INFO ] [REMOTE ] MTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:01:54.612224] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T20:01:54.651361] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-01. [2025-10-20T20:01:55.041384] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T20:01:55.451370] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-06. [2025-10-20T20:01:56.193201] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T20:01:56.251338] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-02. [2025-10-20T20:01:57.051351] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-06. [2025-10-20T20:01:57.126633] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T20:01:57.849515] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T20:01:57.858388] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-08. [2025-10-20T20:01:58.511934] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:01:58.651359] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-07. [2025-10-20T20:01:58.842381] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T20:01:59.451424] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-09. [2025-10-20T20:02:00.251364] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-07. [2025-10-20T20:02:00.348916] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T20:02:00.379082] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:02:01.051424] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-08. [2025-10-20T20:02:01.851374] [INFO ] [REMOTE ] MTD: Fetching primary data for 2025-10. [2025-10-20T20:02:02.658096] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-03. [2025-10-20T20:02:02.674315] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-07 (2.42 seconds). [2025-10-20T20:02:02.758501] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T20:02:03.451430] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-08. [2025-10-20T20:02:03.476072] [INFO ] [REMOTE ] MTD: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T20:02:03.478667] [INFO ] [REMOTE ] MTD: Full fetch complete (21432 records). [2025-10-20T20:02:03.478743] [DEBUG] [VALIDATOR ] MTD: Validating primary data. [2025-10-20T20:02:03.838638] [DEBUG] [VALIDATOR ] MTD: All expected trading days are present. [2025-10-20T20:02:03.838756] [DEBUG] [VALIDATOR ] MTD: Primary data validation passed. [2025-10-20T20:02:03.839285] [INFO ] [MAIN ] MTD: Fetched primary data is valid. [2025-10-20T20:02:03.839335] [DEBUG] [LOCAL ] MTD: Saving primary data. [2025-10-20T20:02:03.840011] [DEBUG] [LOCAL ] MTD: Trimmed 21432 records to 15324 (keeping 200 days of data) [2025-10-20T20:02:03.940968] [DEBUG] [LOCAL ] MTD: Primary data saved (15324 records). [2025-10-20T20:02:03.941116] [INFO ] [MAIN ] MTD: Full history saved. [2025-10-20T20:02:03.941199] [INFO ] [MAIN ] Update Progress: 1412/2367 symbols completed. MTD took 26.2s. Estimated remaining: 2h 3m 5s. Rate: 7.8 symbols/min [2025-10-20T20:02:04.112958] [INFO ] [MAIN ] MTH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:04.153144] [DEBUG] [LOCAL ] MTH: Loaded 44523 existing records from primary storage. [2025-10-20T20:02:04.153563] [INFO ] [MAIN ] MTH: Loaded 44523 existing records. [2025-10-20T20:02:04.257383] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-09. [2025-10-20T20:02:04.341475] [WARN ] [MAIN ] MTH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:04.341850] [INFO ] [MAIN ] MTH: Wiping existing primary data before full refresh. [2025-10-20T20:02:04.342015] [INFO ] [LOCAL ] MTH: Starting deletion of primary data. [2025-10-20T20:02:04.342318] [DEBUG] [LOCAL ] MTH: Deleted primary data file. [2025-10-20T20:02:04.342379] [INFO ] [MAIN ] MTH: Starting full primary refresh. [2025-10-20T20:02:04.768782] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T20:02:04.775395] [INFO ] [REMOTE ] MTH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:04.879836] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T20:02:05.051351] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-01. [2025-10-20T20:02:05.851360] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-09. [2025-10-20T20:02:05.887824] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T20:02:06.651387] [INFO ] [REMOTE ] MTB: Fetching primary data for 2025-10. [2025-10-20T20:02:06.680471] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T20:02:07.451404] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-02. [2025-10-20T20:02:07.632305] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T20:02:08.251364] [INFO ] [REMOTE ] MTDR: Fetching primary data for 2025-10. [2025-10-20T20:02:08.463674] [INFO ] [REMOTE ] MTB: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T20:02:08.470700] [INFO ] [REMOTE ] MTB: Full fetch complete (68363 records). [2025-10-20T20:02:08.470831] [DEBUG] [VALIDATOR ] MTB: Validating primary data. [2025-10-20T20:02:08.966890] [DEBUG] [VALIDATOR ] MTB: All expected trading days are present. [2025-10-20T20:02:08.967018] [DEBUG] [VALIDATOR ] MTB: Primary data validation passed. [2025-10-20T20:02:08.968027] [INFO ] [MAIN ] MTB: Fetched primary data is valid. [2025-10-20T20:02:08.968107] [DEBUG] [LOCAL ] MTB: Saving primary data. [2025-10-20T20:02:08.992818] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T20:02:09.001468] [DEBUG] [LOCAL ] MTB: Trimmed 68363 records to 46699 (keeping 200 days of data) [2025-10-20T20:02:09.071447] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-04. [2025-10-20T20:02:09.325082] [DEBUG] [LOCAL ] MTB: Primary data saved (46699 records). [2025-10-20T20:02:09.325246] [INFO ] [MAIN ] MTB: Full history saved. [2025-10-20T20:02:09.325352] [INFO ] [MAIN ] Update Progress: 1413/2367 symbols completed. MTB took 38.1s. Estimated remaining: 2h 2m 56s. Rate: 7.8 symbols/min [2025-10-20T20:02:09.495110] [INFO ] [MAIN ] MTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:09.533726] [DEBUG] [LOCAL ] MTN: Loaded 40099 existing records from primary storage. [2025-10-20T20:02:09.534006] [INFO ] [MAIN ] MTN: Loaded 40099 existing records. [2025-10-20T20:02:09.709504] [WARN ] [MAIN ] MTN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:09.709756] [INFO ] [MAIN ] MTN: Wiping existing primary data before full refresh. [2025-10-20T20:02:09.709809] [INFO ] [LOCAL ] MTN: Starting deletion of primary data. [2025-10-20T20:02:09.709999] [DEBUG] [LOCAL ] MTN: Deleted primary data file. [2025-10-20T20:02:09.710045] [INFO ] [MAIN ] MTN: Starting full primary refresh. [2025-10-20T20:02:09.855675] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-03. [2025-10-20T20:02:10.044198] [INFO ] [REMOTE ] MTDR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T20:02:10.101548] [INFO ] [REMOTE ] MTDR: Full fetch complete (74265 records). [2025-10-20T20:02:10.119858] [DEBUG] [VALIDATOR ] MTDR: Validating primary data. [2025-10-20T20:02:10.123848] [INFO ] [REMOTE ] MTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:10.518499] [DEBUG] [VALIDATOR ] MTDR: All expected trading days are present. [2025-10-20T20:02:10.518581] [DEBUG] [VALIDATOR ] MTDR: Primary data validation passed. [2025-10-20T20:02:10.519700] [INFO ] [MAIN ] MTDR: Fetched primary data is valid. [2025-10-20T20:02:10.519766] [DEBUG] [LOCAL ] MTDR: Saving primary data. [2025-10-20T20:02:10.521189] [DEBUG] [LOCAL ] MTDR: Trimmed 74265 records to 51765 (keeping 200 days of data) [2025-10-20T20:02:10.692956] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-01. [2025-10-20T20:02:10.835016] [DEBUG] [LOCAL ] MTDR: Primary data saved (51765 records). [2025-10-20T20:02:10.835253] [INFO ] [MAIN ] MTDR: Full history saved. [2025-10-20T20:02:10.835403] [INFO ] [MAIN ] Update Progress: 1414/2367 symbols completed. MTDR took 31.7s. Estimated remaining: 2h 2m 44s. Rate: 7.8 symbols/min [2025-10-20T20:02:11.012564] [INFO ] [MAIN ] MTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:11.012719] [DEBUG] [LOCAL ] MTR: No existing data file found in primary storage. [2025-10-20T20:02:11.012770] [WARN ] [MAIN ] MTR: No existing data found, performing full refresh instead. [2025-10-20T20:02:11.012893] [INFO ] [MAIN ] MTR: Wiping existing primary data before full refresh. [2025-10-20T20:02:11.012936] [INFO ] [LOCAL ] MTR: Starting deletion of primary data. [2025-10-20T20:02:11.012980] [DEBUG] [LOCAL ] MTR: No primary data file found to delete. [2025-10-20T20:02:11.013013] [INFO ] [MAIN ] MTR: Starting full primary refresh. [2025-10-20T20:02:11.152141] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T20:02:11.429196] [INFO ] [REMOTE ] MTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:11.454416] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-01. [2025-10-20T20:02:11.460208] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T20:02:11.955312] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T20:02:12.256455] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-02. [2025-10-20T20:02:12.272330] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T20:02:12.641909] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T20:02:13.051424] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-02. [2025-10-20T20:02:13.851394] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-04. [2025-10-20T20:02:14.539942] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T20:02:14.651428] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-03. [2025-10-20T20:02:15.451476] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-05. [2025-10-20T20:02:15.708764] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T20:02:15.782842] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T20:02:16.251595] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-04. [2025-10-20T20:02:17.051349] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-03. [2025-10-20T20:02:17.135030] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T20:02:17.468112] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T20:02:17.509224] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T20:02:17.851418] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-05. [2025-10-20T20:02:18.651393] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-05. [2025-10-20T20:02:19.347228] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T20:02:19.451420] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-06. [2025-10-20T20:02:19.926335] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T20:02:20.251402] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-06. [2025-10-20T20:02:21.051414] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-06. [2025-10-20T20:02:21.171193] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T20:02:21.855538] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-04. [2025-10-20T20:02:21.859008] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T20:02:22.370552] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T20:02:22.514796] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:02:22.651382] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-07. [2025-10-20T20:02:23.451390] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-05. [2025-10-20T20:02:23.756143] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-05 (0.30 seconds). [2025-10-20T20:02:24.125993] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T20:02:24.251382] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-08. [2025-10-20T20:02:25.051421] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-06. [2025-10-20T20:02:25.585615] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T20:02:25.747532] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T20:02:25.851361] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-09. [2025-10-20T20:02:26.651383] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-07. [2025-10-20T20:02:27.350537] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T20:02:27.451422] [INFO ] [REMOTE ] MTN: Fetching primary data for 2025-10. [2025-10-20T20:02:28.251451] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-07. [2025-10-20T20:02:28.400545] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T20:02:28.757691] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T20:02:29.051396] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-08. [2025-10-20T20:02:29.144187] [INFO ] [REMOTE ] MTN: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T20:02:29.150475] [INFO ] [REMOTE ] MTN: Full fetch complete (56954 records). [2025-10-20T20:02:29.150617] [DEBUG] [VALIDATOR ] MTN: Validating primary data. [2025-10-20T20:02:29.396052] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-08 (0.34 seconds). [2025-10-20T20:02:29.552411] [DEBUG] [VALIDATOR ] MTN: All expected trading days are present. [2025-10-20T20:02:29.552482] [DEBUG] [VALIDATOR ] MTN: Primary data validation passed. [2025-10-20T20:02:29.553350] [INFO ] [MAIN ] MTN: Fetched primary data is valid. [2025-10-20T20:02:29.553422] [DEBUG] [LOCAL ] MTN: Saving primary data. [2025-10-20T20:02:29.554485] [DEBUG] [LOCAL ] MTN: Trimmed 56954 records to 39520 (keeping 200 days of data) [2025-10-20T20:02:29.821114] [DEBUG] [LOCAL ] MTN: Primary data saved (39520 records). [2025-10-20T20:02:29.821257] [INFO ] [MAIN ] MTN: Full history saved. [2025-10-20T20:02:29.821336] [INFO ] [MAIN ] Update Progress: 1415/2367 symbols completed. MTN took 20.5s. Estimated remaining: 2h 2m 44s. Rate: 7.8 symbols/min [2025-10-20T20:02:29.856518] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-09. [2025-10-20T20:02:29.999731] [INFO ] [MAIN ] MTRN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:30.020071] [DEBUG] [LOCAL ] MTRN: Loaded 18534 existing records from primary storage. [2025-10-20T20:02:30.020294] [INFO ] [MAIN ] MTRN: Loaded 18534 existing records. [2025-10-20T20:02:30.192223] [WARN ] [MAIN ] MTRN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:30.192511] [INFO ] [MAIN ] MTRN: Wiping existing primary data before full refresh. [2025-10-20T20:02:30.192570] [INFO ] [LOCAL ] MTRN: Starting deletion of primary data. [2025-10-20T20:02:30.193202] [DEBUG] [LOCAL ] MTRN: Deleted primary data file. [2025-10-20T20:02:30.193258] [INFO ] [MAIN ] MTRN: Starting full primary refresh. [2025-10-20T20:02:30.366548] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-10-20T20:02:30.544214] [INFO ] [REMOTE ] MTRN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:30.651376] [INFO ] [REMOTE ] MTR: Fetching primary data for 2025-10. [2025-10-20T20:02:31.102562] [INFO ] [REMOTE ] MTR: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T20:02:31.103365] [INFO ] [REMOTE ] MTR: Full fetch complete (2366 records). [2025-10-20T20:02:31.103434] [DEBUG] [VALIDATOR ] MTR: Validating primary data. [2025-10-20T20:02:31.295298] [WARN ] [VALIDATOR ] MTR: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T20:02:31.295500] [FATAL] [MAIN ] MTR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:02:31.295579] [WARN ] [MAIN ] MTR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:02:31.295683] [INFO ] [LOCAL ] MTR: Starting deletion of primary data. [2025-10-20T20:02:31.295768] [DEBUG] [LOCAL ] MTR: No primary data file found to delete. [2025-10-20T20:02:31.295807] [INFO ] [LOCAL ] MTR: Starting deletion of derivative data. [2025-10-20T20:02:31.295847] [DEBUG] [LOCAL ] MTR: No derivative data file found to delete. [2025-10-20T20:02:31.295905] [INFO ] [MAIN ] Update Progress: 1416/2367 symbols completed. MTR took 20.5s. Estimated remaining: 2h 2m 32s. Rate: 7.8 symbols/min [2025-10-20T20:02:31.451887] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-01. [2025-10-20T20:02:31.470533] [INFO ] [MAIN ] MTUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:31.470710] [DEBUG] [LOCAL ] MTUS: No existing data file found in primary storage. [2025-10-20T20:02:31.470761] [WARN ] [MAIN ] MTUS: No existing data found, performing full refresh instead. [2025-10-20T20:02:31.470865] [INFO ] [MAIN ] MTUS: Wiping existing primary data before full refresh. [2025-10-20T20:02:31.470908] [INFO ] [LOCAL ] MTUS: Starting deletion of primary data. [2025-10-20T20:02:31.470949] [DEBUG] [LOCAL ] MTUS: No primary data file found to delete. [2025-10-20T20:02:31.470980] [INFO ] [MAIN ] MTUS: Starting full primary refresh. [2025-10-20T20:02:31.820872] [INFO ] [REMOTE ] MTUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:32.251343] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-01. [2025-10-20T20:02:32.341831] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:02:33.051384] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-02. [2025-10-20T20:02:33.413823] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T20:02:33.664287] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T20:02:33.851424] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-07. [2025-10-20T20:02:34.651440] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-08. [2025-10-20T20:02:35.451436] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-02. [2025-10-20T20:02:35.548440] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T20:02:35.934636] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T20:02:36.251450] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-09. [2025-10-20T20:02:36.696174] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T20:02:37.051379] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-03. [2025-10-20T20:02:37.819250] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T20:02:37.851380] [INFO ] [REMOTE ] MTH: Fetching primary data for 2025-10. [2025-10-20T20:02:37.865522] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T20:02:38.651398] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-04. [2025-10-20T20:02:39.454346] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-03. [2025-10-20T20:02:39.462813] [INFO ] [REMOTE ] MTH: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T20:02:39.480572] [INFO ] [REMOTE ] MTH: Full fetch complete (64065 records). [2025-10-20T20:02:39.482034] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:02:39.485844] [DEBUG] [VALIDATOR ] MTH: Validating primary data. [2025-10-20T20:02:39.864840] [DEBUG] [VALIDATOR ] MTH: All expected trading days are present. [2025-10-20T20:02:39.864917] [DEBUG] [VALIDATOR ] MTH: Primary data validation passed. [2025-10-20T20:02:39.865828] [INFO ] [MAIN ] MTH: Fetched primary data is valid. [2025-10-20T20:02:39.865889] [DEBUG] [LOCAL ] MTH: Saving primary data. [2025-10-20T20:02:39.867151] [DEBUG] [LOCAL ] MTH: Trimmed 64065 records to 43889 (keeping 200 days of data) [2025-10-20T20:02:40.129153] [DEBUG] [LOCAL ] MTH: Primary data saved (43889 records). [2025-10-20T20:02:40.129278] [INFO ] [MAIN ] MTH: Full history saved. [2025-10-20T20:02:40.129355] [INFO ] [MAIN ] Update Progress: 1417/2367 symbols completed. MTH took 36.2s. Estimated remaining: 2h 2m 25s. Rate: 7.8 symbols/min [2025-10-20T20:02:40.266866] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-05. [2025-10-20T20:02:40.300993] [INFO ] [MAIN ] MTW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:40.328952] [DEBUG] [LOCAL ] MTW: Loaded 26834 existing records from primary storage. [2025-10-20T20:02:40.329197] [INFO ] [MAIN ] MTW: Loaded 26834 existing records. [2025-10-20T20:02:40.505964] [WARN ] [MAIN ] MTW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:40.506172] [INFO ] [MAIN ] MTW: Wiping existing primary data before full refresh. [2025-10-20T20:02:40.506219] [INFO ] [LOCAL ] MTW: Starting deletion of primary data. [2025-10-20T20:02:40.506404] [DEBUG] [LOCAL ] MTW: Deleted primary data file. [2025-10-20T20:02:40.506461] [INFO ] [MAIN ] MTW: Starting full primary refresh. [2025-10-20T20:02:40.886095] [INFO ] [REMOTE ] MTW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:40.888089] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T20:02:41.051339] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-01. [2025-10-20T20:02:41.443951] [INFO ] [REMOTE ] MTRN: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T20:02:41.851465] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-04. [2025-10-20T20:02:42.158938] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T20:02:42.651441] [INFO ] [REMOTE ] MTRN: Fetching primary data for 2025-06. [2025-10-20T20:02:42.736523] [DEBUG] [REMOTE ] MTRN: Error fetching primary data for 2025-06: 'API call returned an empty response.' [2025-10-20T20:02:42.736991] [WARN ] [REMOTE ] MTRN: No data returned for month 2025-06 [2025-10-20T20:02:42.737129] [FATAL] [REMOTE ] MTRN: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:02:42.737229] [FATAL] [MAIN ] MTRN: Failed to fetch primary data. [2025-10-20T20:02:42.737313] [WARN ] [MAIN ] MTRN: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:02:42.737407] [INFO ] [LOCAL ] MTRN: Starting deletion of primary data. [2025-10-20T20:02:42.737502] [DEBUG] [LOCAL ] MTRN: No primary data file found to delete. [2025-10-20T20:02:42.737568] [INFO ] [LOCAL ] MTRN: Starting deletion of derivative data. [2025-10-20T20:02:42.737836] [DEBUG] [LOCAL ] MTRN: Deleted derivative data file. [2025-10-20T20:02:42.737941] [INFO ] [MAIN ] Update Progress: 1418/2367 symbols completed. MTRN took 12.9s. Estimated remaining: 2h 2m 14s. Rate: 7.8 symbols/min [2025-10-20T20:02:42.910961] [INFO ] [MAIN ] MTX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:42.932565] [DEBUG] [LOCAL ] MTX: Loaded 22982 existing records from primary storage. [2025-10-20T20:02:42.932874] [INFO ] [MAIN ] MTX: Loaded 22982 existing records. [2025-10-20T20:02:43.118850] [WARN ] [MAIN ] MTX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:43.119066] [INFO ] [MAIN ] MTX: Wiping existing primary data before full refresh. [2025-10-20T20:02:43.119116] [INFO ] [LOCAL ] MTX: Starting deletion of primary data. [2025-10-20T20:02:43.119298] [DEBUG] [LOCAL ] MTX: Deleted primary data file. [2025-10-20T20:02:43.119354] [INFO ] [MAIN ] MTX: Starting full primary refresh. [2025-10-20T20:02:43.455316] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-02. [2025-10-20T20:02:43.493105] [INFO ] [REMOTE ] MTX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:43.512520] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T20:02:44.251374] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-01. [2025-10-20T20:02:44.708575] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T20:02:45.051347] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-08. [2025-10-20T20:02:45.348873] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T20:02:45.851410] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-02. [2025-10-20T20:02:46.657524] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-03. [2025-10-20T20:02:46.675927] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:02:46.706462] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T20:02:47.451408] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-03. [2025-10-20T20:02:47.812335] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T20:02:48.251431] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-04. [2025-10-20T20:02:48.510767] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T20:02:49.051378] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-04. [2025-10-20T20:02:49.523616] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T20:02:49.851350] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-05. [2025-10-20T20:02:50.207125] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T20:02:50.651410] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-05. [2025-10-20T20:02:50.736696] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:02:51.451375] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-09. [2025-10-20T20:02:51.777809] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T20:02:52.251378] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-05. [2025-10-20T20:02:53.056550] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-06. [2025-10-20T20:02:53.085348] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T20:02:53.497846] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T20:02:53.851362] [INFO ] [REMOTE ] MTG: Fetching primary data for 2025-10. [2025-10-20T20:02:53.983485] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T20:02:54.651387] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-06. [2025-10-20T20:02:55.175212] [INFO ] [REMOTE ] MTG: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T20:02:55.182738] [INFO ] [REMOTE ] MTG: Full fetch complete (74201 records). [2025-10-20T20:02:55.182891] [DEBUG] [VALIDATOR ] MTG: Validating primary data. [2025-10-20T20:02:55.456562] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-06. [2025-10-20T20:02:55.593466] [DEBUG] [VALIDATOR ] MTG: All expected trading days are present. [2025-10-20T20:02:55.593637] [DEBUG] [VALIDATOR ] MTG: Primary data validation passed. [2025-10-20T20:02:55.594741] [INFO ] [MAIN ] MTG: Fetched primary data is valid. [2025-10-20T20:02:55.594806] [DEBUG] [LOCAL ] MTG: Saving primary data. [2025-10-20T20:02:55.596255] [DEBUG] [LOCAL ] MTG: Trimmed 74201 records to 51055 (keeping 200 days of data) [2025-10-20T20:02:55.917315] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T20:02:55.940463] [DEBUG] [LOCAL ] MTG: Primary data saved (51055 records). [2025-10-20T20:02:55.940616] [INFO ] [MAIN ] MTG: Full history saved. [2025-10-20T20:02:55.941118] [INFO ] [MAIN ] Update Progress: 1419/2367 symbols completed. MTG took 1m 2s. Estimated remaining: 2h 2m 10s. Rate: 7.8 symbols/min [2025-10-20T20:02:56.153127] [INFO ] [MAIN ] MTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:02:56.207245] [DEBUG] [LOCAL ] MTZ: Loaded 43615 existing records from primary storage. [2025-10-20T20:02:56.207562] [INFO ] [MAIN ] MTZ: Loaded 43615 existing records. [2025-10-20T20:02:56.256506] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-07. [2025-10-20T20:02:56.416559] [WARN ] [MAIN ] MTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:02:56.416782] [INFO ] [MAIN ] MTZ: Wiping existing primary data before full refresh. [2025-10-20T20:02:56.416830] [INFO ] [LOCAL ] MTZ: Starting deletion of primary data. [2025-10-20T20:02:56.417055] [DEBUG] [LOCAL ] MTZ: Deleted primary data file. [2025-10-20T20:02:56.417116] [INFO ] [MAIN ] MTZ: Starting full primary refresh. [2025-10-20T20:02:56.757517] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T20:02:56.805776] [INFO ] [REMOTE ] MTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:02:57.051429] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-07. [2025-10-20T20:02:57.372416] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T20:02:57.851357] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-01. [2025-10-20T20:02:58.327164] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T20:02:58.651433] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-08. [2025-10-20T20:02:59.451364] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-07. [2025-10-20T20:02:59.507060] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:02:59.615535] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T20:03:00.251420] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-08. [2025-10-20T20:03:01.055437] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-09. [2025-10-20T20:03:01.061688] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T20:03:01.587592] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T20:03:01.851408] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-09. [2025-10-20T20:03:02.234330] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:03:02.651405] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-08. [2025-10-20T20:03:02.943622] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:03:03.451369] [INFO ] [REMOTE ] MTW: Fetching primary data for 2025-10. [2025-10-20T20:03:03.737074] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T20:03:04.251371] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-09. [2025-10-20T20:03:04.730713] [INFO ] [REMOTE ] MTW: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T20:03:04.734515] [INFO ] [REMOTE ] MTW: Full fetch complete (38174 records). [2025-10-20T20:03:04.734614] [DEBUG] [VALIDATOR ] MTW: Validating primary data. [2025-10-20T20:03:05.055811] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-02. [2025-10-20T20:03:05.131878] [DEBUG] [VALIDATOR ] MTW: All expected trading days are present. [2025-10-20T20:03:05.131999] [DEBUG] [VALIDATOR ] MTW: Primary data validation passed. [2025-10-20T20:03:05.132805] [INFO ] [MAIN ] MTW: Fetched primary data is valid. [2025-10-20T20:03:05.132865] [DEBUG] [LOCAL ] MTW: Saving primary data. [2025-10-20T20:03:05.133759] [DEBUG] [LOCAL ] MTW: Trimmed 38174 records to 26563 (keeping 200 days of data) [2025-10-20T20:03:05.294860] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T20:03:05.307421] [DEBUG] [LOCAL ] MTW: Primary data saved (26563 records). [2025-10-20T20:03:05.307510] [INFO ] [MAIN ] MTW: Full history saved. [2025-10-20T20:03:05.307577] [INFO ] [MAIN ] Update Progress: 1420/2367 symbols completed. MTW took 25.2s. Estimated remaining: 2h 2m 3s. Rate: 7.8 symbols/min [2025-10-20T20:03:05.488250] [INFO ] [MAIN ] MUA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:05.501166] [DEBUG] [LOCAL ] MUA: Loaded 11702 existing records from primary storage. [2025-10-20T20:03:05.501405] [INFO ] [MAIN ] MUA: Loaded 11702 existing records. [2025-10-20T20:03:05.688818] [WARN ] [MAIN ] MUA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:05.689119] [INFO ] [MAIN ] MUA: Wiping existing primary data before full refresh. [2025-10-20T20:03:05.689976] [INFO ] [LOCAL ] MUA: Starting deletion of primary data. [2025-10-20T20:03:05.690159] [DEBUG] [LOCAL ] MUA: Deleted primary data file. [2025-10-20T20:03:05.690200] [INFO ] [MAIN ] MUA: Starting full primary refresh. [2025-10-20T20:03:05.870435] [INFO ] [REMOTE ] MTX: Fetching primary data for 2025-10. [2025-10-20T20:03:06.063020] [INFO ] [REMOTE ] MUA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:06.396577] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:03:06.651382] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-01. [2025-10-20T20:03:06.907453] [INFO ] [REMOTE ] MTX: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T20:03:06.911000] [INFO ] [REMOTE ] MTX: Full fetch complete (31402 records). [2025-10-20T20:03:06.911142] [DEBUG] [VALIDATOR ] MTX: Validating primary data. [2025-10-20T20:03:07.290716] [DEBUG] [VALIDATOR ] MTX: All expected trading days are present. [2025-10-20T20:03:07.290815] [DEBUG] [VALIDATOR ] MTX: Primary data validation passed. [2025-10-20T20:03:07.291477] [INFO ] [MAIN ] MTX: Fetched primary data is valid. [2025-10-20T20:03:07.291518] [DEBUG] [LOCAL ] MTX: Saving primary data. [2025-10-20T20:03:07.292428] [DEBUG] [LOCAL ] MTX: Trimmed 31402 records to 22705 (keeping 200 days of data) [2025-10-20T20:03:07.429463] [DEBUG] [LOCAL ] MTX: Primary data saved (22705 records). [2025-10-20T20:03:07.429595] [INFO ] [MAIN ] MTX: Full history saved. [2025-10-20T20:03:07.429696] [INFO ] [MAIN ] Update Progress: 1421/2367 symbols completed. MTX took 24.7s. Estimated remaining: 2h 1m 52s. Rate: 7.8 symbols/min [2025-10-20T20:03:07.460260] [INFO ] [REMOTE ] MTUS: Fetching primary data for 2025-10. [2025-10-20T20:03:07.615887] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:03:07.617739] [INFO ] [MAIN ] MUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:07.644283] [DEBUG] [LOCAL ] MUC: Loaded 21055 existing records from primary storage. [2025-10-20T20:03:07.644537] [INFO ] [MAIN ] MUC: Loaded 21055 existing records. [2025-10-20T20:03:07.843383] [WARN ] [MAIN ] MUC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:07.843596] [INFO ] [MAIN ] MUC: Wiping existing primary data before full refresh. [2025-10-20T20:03:07.843665] [INFO ] [LOCAL ] MUC: Starting deletion of primary data. [2025-10-20T20:03:07.843882] [DEBUG] [LOCAL ] MUC: Deleted primary data file. [2025-10-20T20:03:07.843938] [INFO ] [MAIN ] MUC: Starting full primary refresh. [2025-10-20T20:03:08.203916] [INFO ] [REMOTE ] MUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:08.251388] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-02. [2025-10-20T20:03:08.722926] [INFO ] [REMOTE ] MTUS: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T20:03:08.728061] [INFO ] [REMOTE ] MTUS: Full fetch complete (41196 records). [2025-10-20T20:03:08.728205] [DEBUG] [VALIDATOR ] MTUS: Validating primary data. [2025-10-20T20:03:09.056477] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-03. [2025-10-20T20:03:09.128335] [DEBUG] [VALIDATOR ] MTUS: All expected trading days are present. [2025-10-20T20:03:09.128464] [DEBUG] [VALIDATOR ] MTUS: Primary data validation passed. [2025-10-20T20:03:09.141813] [INFO ] [MAIN ] MTUS: Fetched primary data is valid. [2025-10-20T20:03:09.142500] [DEBUG] [LOCAL ] MTUS: Saving primary data. [2025-10-20T20:03:09.143102] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T20:03:09.149272] [DEBUG] [LOCAL ] MTUS: Trimmed 41196 records to 27547 (keeping 200 days of data) [2025-10-20T20:03:09.314665] [DEBUG] [LOCAL ] MTUS: Primary data saved (27547 records). [2025-10-20T20:03:09.314808] [INFO ] [MAIN ] MTUS: Full history saved. [2025-10-20T20:03:09.314880] [INFO ] [MAIN ] Update Progress: 1422/2367 symbols completed. MTUS took 38.0s. Estimated remaining: 2h 1m 40s. Rate: 7.8 symbols/min [2025-10-20T20:03:09.484298] [INFO ] [MAIN ] MUE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:09.497937] [DEBUG] [LOCAL ] MUE: Loaded 12287 existing records from primary storage. [2025-10-20T20:03:09.498174] [INFO ] [MAIN ] MUE: Loaded 12287 existing records. [2025-10-20T20:03:09.678793] [WARN ] [MAIN ] MUE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:09.679015] [INFO ] [MAIN ] MUE: Wiping existing primary data before full refresh. [2025-10-20T20:03:09.679062] [INFO ] [LOCAL ] MUE: Starting deletion of primary data. [2025-10-20T20:03:09.679252] [DEBUG] [LOCAL ] MUE: Deleted primary data file. [2025-10-20T20:03:09.679308] [INFO ] [MAIN ] MUE: Starting full primary refresh. [2025-10-20T20:03:09.851427] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-01. [2025-10-20T20:03:10.041979] [INFO ] [REMOTE ] MUE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:10.651342] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-01. [2025-10-20T20:03:10.763617] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T20:03:10.808565] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:03:11.167688] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-01 (0.52 seconds). [2025-10-20T20:03:11.451375] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-02. [2025-10-20T20:03:12.251355] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-04. [2025-10-20T20:03:12.560335] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T20:03:13.051400] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-02. [2025-10-20T20:03:13.708606] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T20:03:13.851455] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-03. [2025-10-20T20:03:13.996423] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T20:03:14.651420] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-03. [2025-10-20T20:03:14.796740] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T20:03:15.451453] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-03. [2025-10-20T20:03:15.772865] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T20:03:16.213738] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T20:03:16.251366] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-04. [2025-10-20T20:03:17.051478] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-04. [2025-10-20T20:03:17.342777] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:03:17.570825] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T20:03:17.851396] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-04. [2025-10-20T20:03:18.651414] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-05. [2025-10-20T20:03:18.759455] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T20:03:19.451377] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-05. [2025-10-20T20:03:19.583985] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T20:03:20.206635] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T20:03:20.251372] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-06. [2025-10-20T20:03:21.051358] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-05. [2025-10-20T20:03:21.202143] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T20:03:21.851427] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-07. [2025-10-20T20:03:22.414772] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T20:03:22.656444] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-06. [2025-10-20T20:03:22.671204] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:03:23.451391] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-08. [2025-10-20T20:03:23.702819] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T20:03:24.257684] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-07. [2025-10-20T20:03:24.261121] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T20:03:25.051412] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-09. [2025-10-20T20:03:25.389682] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:03:25.658831] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T20:03:25.851436] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-06. [2025-10-20T20:03:26.651347] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-05. [2025-10-20T20:03:27.352679] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T20:03:27.451385] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-07. [2025-10-20T20:03:27.480274] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T20:03:28.251421] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-06. [2025-10-20T20:03:29.052191] [INFO ] [REMOTE ] MUA: Fetching primary data for 2025-10. [2025-10-20T20:03:29.054079] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T20:03:29.211861] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T20:03:29.851402] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-08. [2025-10-20T20:03:29.909593] [INFO ] [REMOTE ] MUA: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:03:29.911719] [INFO ] [REMOTE ] MUA: Full fetch complete (18376 records). [2025-10-20T20:03:29.911803] [DEBUG] [VALIDATOR ] MUA: Validating primary data. [2025-10-20T20:03:30.276749] [DEBUG] [VALIDATOR ] MUA: All expected trading days are present. [2025-10-20T20:03:30.276830] [DEBUG] [VALIDATOR ] MUA: Primary data validation passed. [2025-10-20T20:03:30.277215] [INFO ] [MAIN ] MUA: Fetched primary data is valid. [2025-10-20T20:03:30.277253] [DEBUG] [LOCAL ] MUA: Saving primary data. [2025-10-20T20:03:30.277845] [DEBUG] [LOCAL ] MUA: Trimmed 18376 records to 11500 (keeping 200 days of data) [2025-10-20T20:03:30.347342] [DEBUG] [LOCAL ] MUA: Primary data saved (11500 records). [2025-10-20T20:03:30.347482] [INFO ] [MAIN ] MUA: Full history saved. [2025-10-20T20:03:30.347561] [INFO ] [MAIN ] Update Progress: 1423/2367 symbols completed. MUA took 25.0s. Estimated remaining: 2h 1m 41s. Rate: 7.8 symbols/min [2025-10-20T20:03:30.515706] [INFO ] [MAIN ] MUFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:30.566428] [DEBUG] [LOCAL ] MUFG: Loaded 53237 existing records from primary storage. [2025-10-20T20:03:30.566783] [INFO ] [MAIN ] MUFG: Loaded 53237 existing records. [2025-10-20T20:03:30.656483] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-08. [2025-10-20T20:03:30.742661] [WARN ] [MAIN ] MUFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:30.742848] [INFO ] [MAIN ] MUFG: Wiping existing primary data before full refresh. [2025-10-20T20:03:30.742894] [INFO ] [LOCAL ] MUFG: Starting deletion of primary data. [2025-10-20T20:03:30.743083] [DEBUG] [LOCAL ] MUFG: Deleted primary data file. [2025-10-20T20:03:30.743120] [INFO ] [MAIN ] MUFG: Starting full primary refresh. [2025-10-20T20:03:31.101223] [INFO ] [REMOTE ] MUFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:31.451359] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-07. [2025-10-20T20:03:31.501801] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T20:03:31.809803] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T20:03:32.251440] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-01. [2025-10-20T20:03:32.273354] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T20:03:33.051428] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-09. [2025-10-20T20:03:33.851382] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-09. [2025-10-20T20:03:34.079978] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T20:03:34.229546] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:03:34.651415] [INFO ] [REMOTE ] MUC: Fetching primary data for 2025-10. [2025-10-20T20:03:35.276532] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T20:03:35.451413] [INFO ] [REMOTE ] MTZ: Fetching primary data for 2025-10. [2025-10-20T20:03:35.835811] [INFO ] [REMOTE ] MUC: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T20:03:35.839435] [INFO ] [REMOTE ] MUC: Full fetch complete (29497 records). [2025-10-20T20:03:35.839587] [DEBUG] [VALIDATOR ] MUC: Validating primary data. [2025-10-20T20:03:36.247752] [DEBUG] [VALIDATOR ] MUC: All expected trading days are present. [2025-10-20T20:03:36.247839] [DEBUG] [VALIDATOR ] MUC: Primary data validation passed. [2025-10-20T20:03:36.248500] [INFO ] [MAIN ] MUC: Fetched primary data is valid. [2025-10-20T20:03:36.248563] [DEBUG] [LOCAL ] MUC: Saving primary data. [2025-10-20T20:03:36.249396] [DEBUG] [LOCAL ] MUC: Trimmed 29497 records to 20776 (keeping 200 days of data) [2025-10-20T20:03:36.299633] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-02. [2025-10-20T20:03:36.380483] [DEBUG] [LOCAL ] MUC: Primary data saved (20776 records). [2025-10-20T20:03:36.380609] [INFO ] [MAIN ] MUC: Full history saved. [2025-10-20T20:03:36.380718] [INFO ] [MAIN ] Update Progress: 1424/2367 symbols completed. MUC took 29.0s. Estimated remaining: 2h 1m 32s. Rate: 7.8 symbols/min [2025-10-20T20:03:36.555026] [INFO ] [MAIN ] MUJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:36.571082] [DEBUG] [LOCAL ] MUJ: Loaded 14061 existing records from primary storage. [2025-10-20T20:03:36.571332] [INFO ] [MAIN ] MUJ: Loaded 14061 existing records. [2025-10-20T20:03:36.754612] [WARN ] [MAIN ] MUJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:36.754886] [INFO ] [MAIN ] MUJ: Wiping existing primary data before full refresh. [2025-10-20T20:03:36.754934] [INFO ] [LOCAL ] MUJ: Starting deletion of primary data. [2025-10-20T20:03:36.755119] [DEBUG] [LOCAL ] MUJ: Deleted primary data file. [2025-10-20T20:03:36.755187] [INFO ] [MAIN ] MUJ: Starting full primary refresh. [2025-10-20T20:03:37.054847] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-08. [2025-10-20T20:03:37.131397] [INFO ] [REMOTE ] MUJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:37.149431] [INFO ] [REMOTE ] MTZ: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T20:03:37.155877] [INFO ] [REMOTE ] MTZ: Full fetch complete (63802 records). [2025-10-20T20:03:37.156027] [DEBUG] [VALIDATOR ] MTZ: Validating primary data. [2025-10-20T20:03:37.537158] [DEBUG] [VALIDATOR ] MTZ: All expected trading days are present. [2025-10-20T20:03:37.537339] [DEBUG] [VALIDATOR ] MTZ: Primary data validation passed. [2025-10-20T20:03:37.538432] [INFO ] [MAIN ] MTZ: Fetched primary data is valid. [2025-10-20T20:03:37.538507] [DEBUG] [LOCAL ] MTZ: Saving primary data. [2025-10-20T20:03:37.540056] [DEBUG] [LOCAL ] MTZ: Trimmed 63802 records to 42828 (keeping 200 days of data) [2025-10-20T20:03:37.783433] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T20:03:37.832457] [DEBUG] [LOCAL ] MTZ: Primary data saved (42828 records). [2025-10-20T20:03:37.832585] [INFO ] [MAIN ] MTZ: Full history saved. [2025-10-20T20:03:37.832975] [INFO ] [MAIN ] Update Progress: 1425/2367 symbols completed. MTZ took 41.9s. Estimated remaining: 2h 1m 20s. Rate: 7.8 symbols/min [2025-10-20T20:03:37.861772] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-03. [2025-10-20T20:03:38.008296] [INFO ] [MAIN ] MUR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:38.059500] [DEBUG] [LOCAL ] MUR: Loaded 53689 existing records from primary storage. [2025-10-20T20:03:38.059874] [INFO ] [MAIN ] MUR: Loaded 53689 existing records. [2025-10-20T20:03:38.171556] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:03:38.252825] [WARN ] [MAIN ] MUR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:38.253094] [INFO ] [MAIN ] MUR: Wiping existing primary data before full refresh. [2025-10-20T20:03:38.253141] [INFO ] [LOCAL ] MUR: Starting deletion of primary data. [2025-10-20T20:03:38.253338] [DEBUG] [LOCAL ] MUR: Deleted primary data file. [2025-10-20T20:03:38.253394] [INFO ] [MAIN ] MUR: Starting full primary refresh. [2025-10-20T20:03:38.607881] [INFO ] [REMOTE ] MUR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:38.651335] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-01. [2025-10-20T20:03:39.451393] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-09. [2025-10-20T20:03:39.785416] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T20:03:40.132564] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T20:03:40.236085] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T20:03:40.251344] [INFO ] [REMOTE ] MUE: Fetching primary data for 2025-10. [2025-10-20T20:03:41.051381] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-02. [2025-10-20T20:03:41.133457] [INFO ] [REMOTE ] MUE: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T20:03:41.135875] [INFO ] [REMOTE ] MUE: Full fetch complete (15961 records). [2025-10-20T20:03:41.135946] [DEBUG] [VALIDATOR ] MUE: Validating primary data. [2025-10-20T20:03:41.498728] [DEBUG] [VALIDATOR ] MUE: All expected trading days are present. [2025-10-20T20:03:41.498814] [DEBUG] [VALIDATOR ] MUE: Primary data validation passed. [2025-10-20T20:03:41.499191] [INFO ] [MAIN ] MUE: Fetched primary data is valid. [2025-10-20T20:03:41.499232] [DEBUG] [LOCAL ] MUE: Saving primary data. [2025-10-20T20:03:41.499947] [DEBUG] [LOCAL ] MUE: Trimmed 15961 records to 12237 (keeping 200 days of data) [2025-10-20T20:03:41.572709] [DEBUG] [LOCAL ] MUE: Primary data saved (12237 records). [2025-10-20T20:03:41.572850] [INFO ] [MAIN ] MUE: Full history saved. [2025-10-20T20:03:41.572930] [INFO ] [MAIN ] Update Progress: 1426/2367 symbols completed. MUE took 32.3s. Estimated remaining: 2h 1m 10s. Rate: 7.8 symbols/min [2025-10-20T20:03:41.751115] [INFO ] [MAIN ] MUSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:03:41.775487] [DEBUG] [LOCAL ] MUSA: Loaded 24959 existing records from primary storage. [2025-10-20T20:03:41.775772] [INFO ] [MAIN ] MUSA: Loaded 24959 existing records. [2025-10-20T20:03:41.856492] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-01. [2025-10-20T20:03:41.959279] [WARN ] [MAIN ] MUSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:03:41.959559] [INFO ] [MAIN ] MUSA: Wiping existing primary data before full refresh. [2025-10-20T20:03:41.959613] [INFO ] [LOCAL ] MUSA: Starting deletion of primary data. [2025-10-20T20:03:41.959815] [DEBUG] [LOCAL ] MUSA: Deleted primary data file. [2025-10-20T20:03:41.959866] [INFO ] [MAIN ] MUSA: Starting full primary refresh. [2025-10-20T20:03:42.318473] [INFO ] [REMOTE ] MUSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:03:42.651381] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-04. [2025-10-20T20:03:42.685745] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T20:03:42.850769] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T20:03:43.451442] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-01. [2025-10-20T20:03:44.251364] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-02. [2025-10-20T20:03:44.447560] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T20:03:44.546865] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T20:03:44.726071] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T20:03:45.051449] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-03. [2025-10-20T20:03:45.851485] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-05. [2025-10-20T20:03:46.651344] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-02. [2025-10-20T20:03:46.789389] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T20:03:47.334051] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T20:03:47.451375] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-04. [2025-10-20T20:03:47.876990] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T20:03:48.251440] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-06. [2025-10-20T20:03:48.969253] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T20:03:49.051406] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-03. [2025-10-20T20:03:49.808850] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T20:03:49.851334] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-04. [2025-10-20T20:03:49.992507] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T20:03:50.651406] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-07. [2025-10-20T20:03:50.678414] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:03:51.451409] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-05. [2025-10-20T20:03:52.261268] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-05. [2025-10-20T20:03:52.282198] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T20:03:52.895932] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T20:03:53.051395] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-06. [2025-10-20T20:03:53.338602] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T20:03:53.851351] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-08. [2025-10-20T20:03:53.976374] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T20:03:54.651406] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-06. [2025-10-20T20:03:55.451423] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-03. [2025-10-20T20:03:55.611339] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T20:03:56.062092] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T20:03:56.251387] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-09. [2025-10-20T20:03:56.648856] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T20:03:57.051402] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-07. [2025-10-20T20:03:57.851347] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-04. [2025-10-20T20:03:58.037567] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T20:03:58.102406] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T20:03:58.651402] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-08. [2025-10-20T20:03:58.819231] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T20:03:59.451411] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-05. [2025-10-20T20:03:59.580247] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T20:04:00.251368] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-07. [2025-10-20T20:04:00.848723] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T20:04:01.051382] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-06. [2025-10-20T20:04:01.851367] [INFO ] [REMOTE ] MUFG: Fetching primary data for 2025-10. [2025-10-20T20:04:02.190961] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T20:04:02.285815] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T20:04:02.651383] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-07. [2025-10-20T20:04:03.453816] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-08. [2025-10-20T20:04:03.459252] [INFO ] [REMOTE ] MUFG: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T20:04:03.467728] [INFO ] [REMOTE ] MUFG: Full fetch complete (75825 records). [2025-10-20T20:04:03.467874] [DEBUG] [VALIDATOR ] MUFG: Validating primary data. [2025-10-20T20:04:03.880352] [DEBUG] [VALIDATOR ] MUFG: All expected trading days are present. [2025-10-20T20:04:03.880495] [DEBUG] [VALIDATOR ] MUFG: Primary data validation passed. [2025-10-20T20:04:03.881587] [INFO ] [MAIN ] MUFG: Fetched primary data is valid. [2025-10-20T20:04:03.881788] [DEBUG] [LOCAL ] MUFG: Saving primary data. [2025-10-20T20:04:03.898428] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T20:04:03.904090] [DEBUG] [LOCAL ] MUFG: Trimmed 75825 records to 52417 (keeping 200 days of data) [2025-10-20T20:04:04.207242] [DEBUG] [LOCAL ] MUFG: Primary data saved (52417 records). [2025-10-20T20:04:04.207389] [INFO ] [MAIN ] MUFG: Full history saved. [2025-10-20T20:04:04.207471] [INFO ] [MAIN ] Update Progress: 1427/2367 symbols completed. MUFG took 33.9s. Estimated remaining: 2h 1m 12s. Rate: 7.8 symbols/min [2025-10-20T20:04:04.261793] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-08. [2025-10-20T20:04:04.378739] [INFO ] [MAIN ] MUSE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:04.378902] [DEBUG] [LOCAL ] MUSE: No existing data file found in primary storage. [2025-10-20T20:04:04.378939] [WARN ] [MAIN ] MUSE: No existing data found, performing full refresh instead. [2025-10-20T20:04:04.379045] [INFO ] [MAIN ] MUSE: Wiping existing primary data before full refresh. [2025-10-20T20:04:04.379085] [INFO ] [LOCAL ] MUSE: Starting deletion of primary data. [2025-10-20T20:04:04.379124] [DEBUG] [LOCAL ] MUSE: No primary data file found to delete. [2025-10-20T20:04:04.379155] [INFO ] [MAIN ] MUSE: Starting full primary refresh. [2025-10-20T20:04:04.727978] [INFO ] [REMOTE ] MUSE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:05.051420] [INFO ] [REMOTE ] MUSE: Fetching primary data for 2025-01. [2025-10-20T20:04:05.176018] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T20:04:05.293974] [DEBUG] [REMOTE ] MUSE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:04:05.294277] [WARN ] [REMOTE ] MUSE: No data returned for month 2025-01 [2025-10-20T20:04:05.294362] [FATAL] [REMOTE ] MUSE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:04:05.294411] [FATAL] [MAIN ] MUSE: Failed to fetch primary data. [2025-10-20T20:04:05.294467] [WARN ] [MAIN ] MUSE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:04:05.294537] [INFO ] [LOCAL ] MUSE: Starting deletion of primary data. [2025-10-20T20:04:05.294597] [DEBUG] [LOCAL ] MUSE: No primary data file found to delete. [2025-10-20T20:04:05.294632] [INFO ] [LOCAL ] MUSE: Starting deletion of derivative data. [2025-10-20T20:04:05.294716] [DEBUG] [LOCAL ] MUSE: No derivative data file found to delete. [2025-10-20T20:04:05.294772] [INFO ] [MAIN ] Update Progress: 1428/2367 symbols completed. MUSE took 1.1s. Estimated remaining: 2h 1m 0s. Rate: 7.8 symbols/min [2025-10-20T20:04:05.468970] [INFO ] [MAIN ] MUX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:05.508380] [DEBUG] [LOCAL ] MUX: Loaded 44165 existing records from primary storage. [2025-10-20T20:04:05.508688] [INFO ] [MAIN ] MUX: Loaded 44165 existing records. [2025-10-20T20:04:05.702746] [WARN ] [MAIN ] MUX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:05.704114] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T20:04:05.708904] [INFO ] [MAIN ] MUX: Wiping existing primary data before full refresh. [2025-10-20T20:04:05.709271] [INFO ] [LOCAL ] MUX: Starting deletion of primary data. [2025-10-20T20:04:05.709855] [DEBUG] [LOCAL ] MUX: Deleted primary data file. [2025-10-20T20:04:05.709913] [INFO ] [MAIN ] MUX: Starting full primary refresh. [2025-10-20T20:04:05.857317] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-09. [2025-10-20T20:04:06.065082] [INFO ] [REMOTE ] MUX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:06.651423] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-09. [2025-10-20T20:04:07.360933] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T20:04:07.451353] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-09. [2025-10-20T20:04:07.765060] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T20:04:08.029168] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T20:04:08.251389] [INFO ] [REMOTE ] MUR: Fetching primary data for 2025-10. [2025-10-20T20:04:09.051417] [INFO ] [REMOTE ] MUSA: Fetching primary data for 2025-10. [2025-10-20T20:04:09.851405] [INFO ] [REMOTE ] MUJ: Fetching primary data for 2025-10. [2025-10-20T20:04:10.045773] [INFO ] [REMOTE ] MUR: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T20:04:10.053495] [INFO ] [REMOTE ] MUR: Full fetch complete (76834 records). [2025-10-20T20:04:10.053622] [DEBUG] [VALIDATOR ] MUR: Validating primary data. [2025-10-20T20:04:10.368941] [INFO ] [REMOTE ] MUSA: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T20:04:10.417537] [INFO ] [REMOTE ] MUSA: Full fetch complete (35476 records). [2025-10-20T20:04:10.445520] [DEBUG] [VALIDATOR ] MUSA: Validating primary data. [2025-10-20T20:04:10.463988] [DEBUG] [VALIDATOR ] MUR: All expected trading days are present. [2025-10-20T20:04:10.464050] [DEBUG] [VALIDATOR ] MUR: Primary data validation passed. [2025-10-20T20:04:10.498160] [INFO ] [MAIN ] MUR: Fetched primary data is valid. [2025-10-20T20:04:10.530589] [DEBUG] [LOCAL ] MUR: Saving primary data. [2025-10-20T20:04:10.574709] [DEBUG] [LOCAL ] MUR: Trimmed 76834 records to 52913 (keeping 200 days of data) [2025-10-20T20:04:10.698012] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-01. [2025-10-20T20:04:10.857773] [INFO ] [REMOTE ] MUJ: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T20:04:10.885912] [INFO ] [REMOTE ] MUJ: Full fetch complete (19568 records). [2025-10-20T20:04:10.905700] [DEBUG] [VALIDATOR ] MUJ: Validating primary data. [2025-10-20T20:04:11.157240] [DEBUG] [VALIDATOR ] MUSA: All expected trading days are present. [2025-10-20T20:04:11.162715] [DEBUG] [VALIDATOR ] MUSA: Primary data validation passed. [2025-10-20T20:04:11.168874] [INFO ] [MAIN ] MUSA: Fetched primary data is valid. [2025-10-20T20:04:11.192429] [DEBUG] [LOCAL ] MUSA: Saving primary data. [2025-10-20T20:04:11.287015] [DEBUG] [LOCAL ] MUSA: Trimmed 35476 records to 24721 (keeping 200 days of data) [2025-10-20T20:04:11.604940] [DEBUG] [LOCAL ] MUR: Primary data saved (52913 records). [2025-10-20T20:04:11.631789] [INFO ] [MAIN ] MUR: Full history saved. [2025-10-20T20:04:11.672562] [INFO ] [MAIN ] Update Progress: 1429/2367 symbols completed. MUR took 33.8s. Estimated remaining: 2h 0m 51s. Rate: 7.8 symbols/min [2025-10-20T20:04:11.672740] [DEBUG] [VALIDATOR ] MUJ: All expected trading days are present. [2025-10-20T20:04:11.673619] [DEBUG] [VALIDATOR ] MUJ: Primary data validation passed. [2025-10-20T20:04:11.688768] [INFO ] [MAIN ] MUJ: Fetched primary data is valid. [2025-10-20T20:04:11.720777] [DEBUG] [LOCAL ] MUJ: Saving primary data. [2025-10-20T20:04:11.756384] [DEBUG] [LOCAL ] MUJ: Trimmed 19568 records to 13886 (keeping 200 days of data) [2025-10-20T20:04:11.941616] [INFO ] [MAIN ] MVF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:11.943779] [DEBUG] [LOCAL ] MVF: No existing data file found in primary storage. [2025-10-20T20:04:11.944829] [WARN ] [MAIN ] MVF: No existing data found, performing full refresh instead. [2025-10-20T20:04:11.952421] [INFO ] [MAIN ] MVF: Wiping existing primary data before full refresh. [2025-10-20T20:04:11.966786] [INFO ] [LOCAL ] MVF: Starting deletion of primary data. [2025-10-20T20:04:11.978329] [DEBUG] [LOCAL ] MVF: No primary data file found to delete. [2025-10-20T20:04:11.979502] [INFO ] [MAIN ] MVF: Starting full primary refresh. [2025-10-20T20:04:12.008389] [DEBUG] [LOCAL ] MUSA: Primary data saved (24721 records). [2025-10-20T20:04:12.018099] [INFO ] [MAIN ] MUSA: Full history saved. [2025-10-20T20:04:12.038188] [DEBUG] [LOCAL ] MUJ: Primary data saved (13886 records). [2025-10-20T20:04:12.038334] [INFO ] [MAIN ] Update Progress: 1430/2367 symbols completed. MUSA took 30.5s. Estimated remaining: 2h 0m 39s. Rate: 7.8 symbols/min [2025-10-20T20:04:12.038472] [INFO ] [MAIN ] MUJ: Full history saved. [2025-10-20T20:04:12.063969] [INFO ] [MAIN ] Update Progress: 1431/2367 symbols completed. MUJ took 35.7s. Estimated remaining: 2h 0m 26s. Rate: 7.8 symbols/min [2025-10-20T20:04:12.468470] [INFO ] [MAIN ] MVO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:12.643521] [INFO ] [MAIN ] MVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:12.649398] [DEBUG] [LOCAL ] MVO: Loaded 9002 existing records from primary storage. [2025-10-20T20:04:12.656610] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T20:04:12.674242] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-02. [2025-10-20T20:04:12.673667] [INFO ] [MAIN ] MVO: Loaded 9002 existing records. [2025-10-20T20:04:12.858681] [INFO ] [REMOTE ] MVF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:12.870350] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-01. [2025-10-20T20:04:12.847214] [DEBUG] [LOCAL ] MVT: Loaded 10095 existing records from primary storage. [2025-10-20T20:04:12.871568] [INFO ] [MAIN ] MVT: Loaded 10095 existing records. [2025-10-20T20:04:13.056982] [WARN ] [MAIN ] MVO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:13.067876] [INFO ] [MAIN ] MVO: Wiping existing primary data before full refresh. [2025-10-20T20:04:13.078210] [INFO ] [LOCAL ] MVO: Starting deletion of primary data. [2025-10-20T20:04:13.096501] [DEBUG] [LOCAL ] MVO: Deleted primary data file. [2025-10-20T20:04:13.115915] [INFO ] [MAIN ] MVO: Starting full primary refresh. [2025-10-20T20:04:13.201832] [WARN ] [MAIN ] MVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:13.237408] [INFO ] [MAIN ] MVT: Wiping existing primary data before full refresh. [2025-10-20T20:04:13.240179] [INFO ] [LOCAL ] MVT: Starting deletion of primary data. [2025-10-20T20:04:13.240615] [DEBUG] [LOCAL ] MVT: Deleted primary data file. [2025-10-20T20:04:13.261491] [INFO ] [MAIN ] MVT: Starting full primary refresh. [2025-10-20T20:04:13.788560] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T20:04:13.817559] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-02. [2025-10-20T20:04:13.840449] [INFO ] [REMOTE ] MVO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:13.854502] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-01. [2025-10-20T20:04:13.926101] [INFO ] [REMOTE ] MVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:13.982137] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T20:04:14.651361] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-03. [2025-10-20T20:04:14.668725] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T20:04:14.895920] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T20:04:15.451376] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-01. [2025-10-20T20:04:16.243801] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T20:04:16.252598] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-02. [2025-10-20T20:04:16.276778] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T20:04:17.017816] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:04:17.051333] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-02. [2025-10-20T20:04:17.683768] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T20:04:17.851362] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-03. [2025-10-20T20:04:18.651374] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-04. [2025-10-20T20:04:18.817098] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T20:04:19.451386] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-04. [2025-10-20T20:04:20.134000] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T20:04:20.251347] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-05. [2025-10-20T20:04:20.522275] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T20:04:21.051369] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-03. [2025-10-20T20:04:21.729977] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T20:04:21.851355] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-06. [2025-10-20T20:04:21.998331] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T20:04:22.651394] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-03. [2025-10-20T20:04:23.410994] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T20:04:23.437486] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:04:23.451432] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-05. [2025-10-20T20:04:24.171370] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T20:04:24.251391] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-04. [2025-10-20T20:04:25.051427] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-07. [2025-10-20T20:04:25.324849] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T20:04:25.851370] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-04. [2025-10-20T20:04:26.584967] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T20:04:26.633545] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T20:04:26.651346] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-05. [2025-10-20T20:04:27.405215] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:04:27.451330] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-06. [2025-10-20T20:04:28.251374] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-06. [2025-10-20T20:04:28.401399] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T20:04:28.667091] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T20:04:29.051366] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-07. [2025-10-20T20:04:29.845067] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:04:29.851324] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-08. [2025-10-20T20:04:30.616413] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:04:30.651450] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-07. [2025-10-20T20:04:31.451620] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-05. [2025-10-20T20:04:31.453289] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T20:04:32.251380] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-09. [2025-10-20T20:04:32.279292] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T20:04:33.051358] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-08. [2025-10-20T20:04:33.080439] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T20:04:33.851423] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-06. [2025-10-20T20:04:34.433915] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T20:04:34.562381] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:04:34.651343] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-09. [2025-10-20T20:04:35.451379] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-07. [2025-10-20T20:04:36.065930] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T20:04:36.251344] [INFO ] [REMOTE ] MVT: Fetching primary data for 2025-10. [2025-10-20T20:04:36.365865] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:04:37.000077] [INFO ] [REMOTE ] MVT: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:04:37.001702] [INFO ] [REMOTE ] MVT: Full fetch complete (13766 records). [2025-10-20T20:04:37.001770] [DEBUG] [VALIDATOR ] MVT: Validating primary data. [2025-10-20T20:04:37.061680] [INFO ] [REMOTE ] MUX: Fetching primary data for 2025-10. [2025-10-20T20:04:37.358936] [DEBUG] [VALIDATOR ] MVT: All expected trading days are present. [2025-10-20T20:04:37.359011] [DEBUG] [VALIDATOR ] MVT: Primary data validation passed. [2025-10-20T20:04:37.359321] [INFO ] [MAIN ] MVT: Fetched primary data is valid. [2025-10-20T20:04:37.359358] [DEBUG] [LOCAL ] MVT: Saving primary data. [2025-10-20T20:04:37.359841] [DEBUG] [LOCAL ] MVT: Trimmed 13766 records to 9962 (keeping 200 days of data) [2025-10-20T20:04:37.417259] [DEBUG] [LOCAL ] MVT: Primary data saved (9962 records). [2025-10-20T20:04:37.417381] [INFO ] [MAIN ] MVT: Full history saved. [2025-10-20T20:04:37.417456] [INFO ] [MAIN ] Update Progress: 1432/2367 symbols completed. MVT took 25.3s. Estimated remaining: 2h 0m 30s. Rate: 7.8 symbols/min [2025-10-20T20:04:37.585638] [INFO ] [MAIN ] MWA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:37.626456] [DEBUG] [LOCAL ] MWA: Loaded 46654 existing records from primary storage. [2025-10-20T20:04:37.626777] [INFO ] [MAIN ] MWA: Loaded 46654 existing records. [2025-10-20T20:04:37.803233] [WARN ] [MAIN ] MWA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:37.803447] [INFO ] [MAIN ] MWA: Wiping existing primary data before full refresh. [2025-10-20T20:04:37.803498] [INFO ] [LOCAL ] MWA: Starting deletion of primary data. [2025-10-20T20:04:37.803727] [DEBUG] [LOCAL ] MWA: Deleted primary data file. [2025-10-20T20:04:37.803782] [INFO ] [MAIN ] MWA: Starting full primary refresh. [2025-10-20T20:04:37.861163] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-08. [2025-10-20T20:04:38.160146] [INFO ] [REMOTE ] MWA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:38.440611] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:04:38.651364] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-08. [2025-10-20T20:04:39.014773] [INFO ] [REMOTE ] MUX: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T20:04:39.022237] [INFO ] [REMOTE ] MUX: Full fetch complete (61234 records). [2025-10-20T20:04:39.022326] [DEBUG] [VALIDATOR ] MUX: Validating primary data. [2025-10-20T20:04:39.402136] [DEBUG] [VALIDATOR ] MUX: All expected trading days are present. [2025-10-20T20:04:39.402232] [DEBUG] [VALIDATOR ] MUX: Primary data validation passed. [2025-10-20T20:04:39.403216] [INFO ] [MAIN ] MUX: Fetched primary data is valid. [2025-10-20T20:04:39.403276] [DEBUG] [LOCAL ] MUX: Saving primary data. [2025-10-20T20:04:39.404372] [DEBUG] [LOCAL ] MUX: Trimmed 61234 records to 43712 (keeping 200 days of data) [2025-10-20T20:04:39.458220] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-09. [2025-10-20T20:04:39.514838] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:04:39.664050] [DEBUG] [LOCAL ] MUX: Primary data saved (43712 records). [2025-10-20T20:04:39.664174] [INFO ] [MAIN ] MUX: Full history saved. [2025-10-20T20:04:39.664254] [INFO ] [MAIN ] Update Progress: 1433/2367 symbols completed. MUX took 34.4s. Estimated remaining: 2h 0m 19s. Rate: 7.8 symbols/min [2025-10-20T20:04:39.829126] [INFO ] [MAIN ] MX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:39.849976] [DEBUG] [LOCAL ] MX: Loaded 22627 existing records from primary storage. [2025-10-20T20:04:39.850249] [INFO ] [MAIN ] MX: Loaded 22627 existing records. [2025-10-20T20:04:40.024229] [WARN ] [MAIN ] MX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:40.024396] [INFO ] [MAIN ] MX: Wiping existing primary data before full refresh. [2025-10-20T20:04:40.024440] [INFO ] [LOCAL ] MX: Starting deletion of primary data. [2025-10-20T20:04:40.024683] [DEBUG] [LOCAL ] MX: Deleted primary data file. [2025-10-20T20:04:40.024740] [INFO ] [MAIN ] MX: Starting full primary refresh. [2025-10-20T20:04:40.256557] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-01. [2025-10-20T20:04:40.282636] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:04:40.387287] [INFO ] [REMOTE ] MX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:41.051398] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-01. [2025-10-20T20:04:41.851365] [INFO ] [REMOTE ] MVO: Fetching primary data for 2025-10. [2025-10-20T20:04:41.989056] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T20:04:42.012381] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:04:42.651359] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-02. [2025-10-20T20:04:42.711002] [INFO ] [REMOTE ] MVO: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:04:42.713189] [INFO ] [REMOTE ] MVO: Full fetch complete (14609 records). [2025-10-20T20:04:42.713268] [DEBUG] [VALIDATOR ] MVO: Validating primary data. [2025-10-20T20:04:43.074879] [DEBUG] [VALIDATOR ] MVO: All expected trading days are present. [2025-10-20T20:04:43.074958] [DEBUG] [VALIDATOR ] MVO: Primary data validation passed. [2025-10-20T20:04:43.075306] [INFO ] [MAIN ] MVO: Fetched primary data is valid. [2025-10-20T20:04:43.075348] [DEBUG] [LOCAL ] MVO: Saving primary data. [2025-10-20T20:04:43.075865] [DEBUG] [LOCAL ] MVO: Trimmed 14609 records to 8946 (keeping 200 days of data) [2025-10-20T20:04:43.128254] [DEBUG] [LOCAL ] MVO: Primary data saved (8946 records). [2025-10-20T20:04:43.128414] [INFO ] [MAIN ] MVO: Full history saved. [2025-10-20T20:04:43.128494] [INFO ] [MAIN ] Update Progress: 1434/2367 symbols completed. MVO took 31.1s. Estimated remaining: 2h 0m 8s. Rate: 7.8 symbols/min [2025-10-20T20:04:43.300489] [INFO ] [MAIN ] MXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:43.300722] [DEBUG] [LOCAL ] MXE: No existing data file found in primary storage. [2025-10-20T20:04:43.300774] [WARN ] [MAIN ] MXE: No existing data found, performing full refresh instead. [2025-10-20T20:04:43.300889] [INFO ] [MAIN ] MXE: Wiping existing primary data before full refresh. [2025-10-20T20:04:43.300932] [INFO ] [LOCAL ] MXE: Starting deletion of primary data. [2025-10-20T20:04:43.300974] [DEBUG] [LOCAL ] MXE: No primary data file found to delete. [2025-10-20T20:04:43.301008] [INFO ] [MAIN ] MXE: Starting full primary refresh. [2025-10-20T20:04:43.461292] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-02. [2025-10-20T20:04:43.672196] [INFO ] [REMOTE ] MXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:43.702461] [INFO ] [REMOTE ] MX: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T20:04:44.251465] [INFO ] [REMOTE ] MX: Fetching primary data for 2025-03. [2025-10-20T20:04:44.351927] [DEBUG] [REMOTE ] MX: Error fetching primary data for 2025-03: 'API call returned an empty response.' [2025-10-20T20:04:44.352189] [WARN ] [REMOTE ] MX: No data returned for month 2025-03 [2025-10-20T20:04:44.352302] [FATAL] [REMOTE ] MX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:04:44.352417] [FATAL] [MAIN ] MX: Failed to fetch primary data. [2025-10-20T20:04:44.352494] [WARN ] [MAIN ] MX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:04:44.352577] [INFO ] [LOCAL ] MX: Starting deletion of primary data. [2025-10-20T20:04:44.352690] [DEBUG] [LOCAL ] MX: No primary data file found to delete. [2025-10-20T20:04:44.352760] [INFO ] [LOCAL ] MX: Starting deletion of derivative data. [2025-10-20T20:04:44.352974] [DEBUG] [LOCAL ] MX: Deleted derivative data file. [2025-10-20T20:04:44.353071] [INFO ] [MAIN ] Update Progress: 1435/2367 symbols completed. MX took 4.7s. Estimated remaining: 1h 59m 56s. Rate: 7.8 symbols/min [2025-10-20T20:04:44.523988] [INFO ] [MAIN ] MXF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:44.531445] [DEBUG] [LOCAL ] MXF: Loaded 5717 existing records from primary storage. [2025-10-20T20:04:44.531608] [INFO ] [MAIN ] MXF: Loaded 5717 existing records. [2025-10-20T20:04:44.713302] [WARN ] [MAIN ] MXF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:44.713530] [INFO ] [MAIN ] MXF: Wiping existing primary data before full refresh. [2025-10-20T20:04:44.713598] [INFO ] [LOCAL ] MXF: Starting deletion of primary data. [2025-10-20T20:04:44.713875] [DEBUG] [LOCAL ] MXF: Deleted primary data file. [2025-10-20T20:04:44.713932] [INFO ] [MAIN ] MXF: Starting full primary refresh. [2025-10-20T20:04:45.051360] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-09. [2025-10-20T20:04:45.076763] [INFO ] [REMOTE ] MXF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:45.103586] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T20:04:45.734952] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-10-20T20:04:45.851366] [INFO ] [REMOTE ] MVF: Fetching primary data for 2025-10. [2025-10-20T20:04:46.651367] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-03. [2025-10-20T20:04:46.781457] [INFO ] [REMOTE ] MVF: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T20:04:46.783820] [INFO ] [REMOTE ] MVF: Full fetch complete (18440 records). [2025-10-20T20:04:46.783887] [DEBUG] [VALIDATOR ] MVF: Validating primary data. [2025-10-20T20:04:47.143339] [DEBUG] [VALIDATOR ] MVF: All expected trading days are present. [2025-10-20T20:04:47.143418] [DEBUG] [VALIDATOR ] MVF: Primary data validation passed. [2025-10-20T20:04:47.143812] [INFO ] [MAIN ] MVF: Fetched primary data is valid. [2025-10-20T20:04:47.143871] [DEBUG] [LOCAL ] MVF: Saving primary data. [2025-10-20T20:04:47.144373] [DEBUG] [LOCAL ] MVF: Trimmed 18440 records to 12674 (keeping 200 days of data) [2025-10-20T20:04:47.215630] [DEBUG] [LOCAL ] MVF: Primary data saved (12674 records). [2025-10-20T20:04:47.215779] [INFO ] [MAIN ] MVF: Full history saved. [2025-10-20T20:04:47.215847] [INFO ] [MAIN ] Update Progress: 1436/2367 symbols completed. MVF took 35.5s. Estimated remaining: 1h 59m 45s. Rate: 7.8 symbols/min [2025-10-20T20:04:47.381446] [INFO ] [MAIN ] MYD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:04:47.395840] [DEBUG] [LOCAL ] MYD: Loaded 13587 existing records from primary storage. [2025-10-20T20:04:47.396055] [INFO ] [MAIN ] MYD: Loaded 13587 existing records. [2025-10-20T20:04:47.458968] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-01. [2025-10-20T20:04:47.575964] [WARN ] [MAIN ] MYD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:04:47.576181] [INFO ] [MAIN ] MYD: Wiping existing primary data before full refresh. [2025-10-20T20:04:47.576226] [INFO ] [LOCAL ] MYD: Starting deletion of primary data. [2025-10-20T20:04:47.576384] [DEBUG] [LOCAL ] MYD: Deleted primary data file. [2025-10-20T20:04:47.576421] [INFO ] [MAIN ] MYD: Starting full primary refresh. [2025-10-20T20:04:47.926515] [INFO ] [REMOTE ] MYD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:04:47.959077] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T20:04:48.062401] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T20:04:48.251355] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-04. [2025-10-20T20:04:49.051413] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-01. [2025-10-20T20:04:49.766060] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T20:04:49.851383] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-02. [2025-10-20T20:04:49.952967] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:04:50.348757] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T20:04:50.651371] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-02. [2025-10-20T20:04:51.298697] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:04:51.451371] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-03. [2025-10-20T20:04:52.186949] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T20:04:52.251372] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-04. [2025-10-20T20:04:53.032547] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T20:04:53.051332] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-05. [2025-10-20T20:04:53.829181] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T20:04:53.851351] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-05. [2025-10-20T20:04:54.651365] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-06. [2025-10-20T20:04:55.237313] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T20:04:55.451391] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-01. [2025-10-20T20:04:55.590566] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T20:04:56.161895] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T20:04:56.251328] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-02. [2025-10-20T20:04:57.051354] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-03. [2025-10-20T20:04:57.171179] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:04:57.343319] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T20:04:57.851427] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-04. [2025-10-20T20:04:58.285036] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T20:04:58.651391] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-07. [2025-10-20T20:04:59.246508] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T20:04:59.451420] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-05. [2025-10-20T20:05:00.025197] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T20:05:00.251367] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-08. [2025-10-20T20:05:01.051389] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-06. [2025-10-20T20:05:01.073914] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T20:05:01.851446] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-03. [2025-10-20T20:05:02.593548] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T20:05:02.604435] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T20:05:02.651386] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-04. [2025-10-20T20:05:03.451360] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-09. [2025-10-20T20:05:03.737452] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:05:03.972246] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-09 (0.52 seconds). [2025-10-20T20:05:04.251368] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-06. [2025-10-20T20:05:04.819420] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T20:05:05.051364] [INFO ] [REMOTE ] MXF: Fetching primary data for 2025-10. [2025-10-20T20:05:05.799602] [INFO ] [REMOTE ] MXF: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:05:05.801157] [INFO ] [REMOTE ] MXF: Full fetch complete (9793 records). [2025-10-20T20:05:05.801243] [DEBUG] [VALIDATOR ] MXF: Validating primary data. [2025-10-20T20:05:05.856670] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-07. [2025-10-20T20:05:06.212744] [DEBUG] [VALIDATOR ] MXF: All expected trading days are present. [2025-10-20T20:05:06.212845] [DEBUG] [VALIDATOR ] MXF: Primary data validation passed. [2025-10-20T20:05:06.213092] [INFO ] [MAIN ] MXF: Fetched primary data is valid. [2025-10-20T20:05:06.213130] [DEBUG] [LOCAL ] MXF: Saving primary data. [2025-10-20T20:05:06.213524] [DEBUG] [LOCAL ] MXF: Trimmed 9793 records to 5568 (keeping 200 days of data) [2025-10-20T20:05:06.246682] [DEBUG] [LOCAL ] MXF: Primary data saved (5568 records). [2025-10-20T20:05:06.246802] [INFO ] [MAIN ] MXF: Full history saved. [2025-10-20T20:05:06.246881] [INFO ] [MAIN ] Update Progress: 1437/2367 symbols completed. MXF took 21.9s. Estimated remaining: 1h 59m 45s. Rate: 7.8 symbols/min [2025-10-20T20:05:06.651418] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-05. [2025-10-20T20:05:07.451427] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-07. [2025-10-20T20:05:07.517285] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:05:07.519248] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T20:05:07.928415] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T20:05:08.251369] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-08. [2025-10-20T20:05:09.051438] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-08. [2025-10-20T20:05:09.563759] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-08 (0.51 seconds). [2025-10-20T20:05:09.824790] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T20:05:09.851331] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-09. [2025-10-20T20:05:10.651416] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-06. [2025-10-20T20:05:11.331959] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T20:05:11.451347] [INFO ] [REMOTE ] MWA: Fetching primary data for 2025-10. [2025-10-20T20:05:11.521320] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T20:05:12.251461] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-07. [2025-10-20T20:05:13.051388] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-09. [2025-10-20T20:05:13.099144] [INFO ] [REMOTE ] MWA: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T20:05:13.099787] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T20:05:13.114227] [INFO ] [REMOTE ] MWA: Full fetch complete (68086 records). [2025-10-20T20:05:13.114352] [DEBUG] [VALIDATOR ] MWA: Validating primary data. [2025-10-20T20:05:13.492329] [DEBUG] [VALIDATOR ] MWA: All expected trading days are present. [2025-10-20T20:05:13.492403] [DEBUG] [VALIDATOR ] MWA: Primary data validation passed. [2025-10-20T20:05:13.493334] [INFO ] [MAIN ] MWA: Fetched primary data is valid. [2025-10-20T20:05:13.493394] [DEBUG] [LOCAL ] MWA: Saving primary data. [2025-10-20T20:05:13.494619] [DEBUG] [LOCAL ] MWA: Trimmed 68086 records to 45836 (keeping 200 days of data) [2025-10-20T20:05:13.559071] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-09 (0.51 seconds). [2025-10-20T20:05:13.777767] [DEBUG] [LOCAL ] MWA: Primary data saved (45836 records). [2025-10-20T20:05:13.777955] [INFO ] [MAIN ] MWA: Full history saved. [2025-10-20T20:05:13.778087] [INFO ] [MAIN ] Update Progress: 1438/2367 symbols completed. MWA took 36.4s. Estimated remaining: 1h 59m 37s. Rate: 7.8 symbols/min [2025-10-20T20:05:13.851470] [INFO ] [REMOTE ] MXE: Fetching primary data for 2025-10. [2025-10-20T20:05:14.351837] [INFO ] [REMOTE ] MXE: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T20:05:14.352694] [INFO ] [REMOTE ] MXE: Full fetch complete (2585 records). [2025-10-20T20:05:14.352764] [DEBUG] [VALIDATOR ] MXE: Validating primary data. [2025-10-20T20:05:14.534157] [WARN ] [VALIDATOR ] MXE: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T20:05:14.534389] [FATAL] [MAIN ] MXE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:05:14.534460] [WARN ] [MAIN ] MXE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:05:14.534530] [INFO ] [LOCAL ] MXE: Starting deletion of primary data. [2025-10-20T20:05:14.534593] [DEBUG] [LOCAL ] MXE: No primary data file found to delete. [2025-10-20T20:05:14.534629] [INFO ] [LOCAL ] MXE: Starting deletion of derivative data. [2025-10-20T20:05:14.534723] [DEBUG] [LOCAL ] MXE: No derivative data file found to delete. [2025-10-20T20:05:14.534792] [INFO ] [MAIN ] Update Progress: 1439/2367 symbols completed. MXE took 31.4s. Estimated remaining: 1h 59m 25s. Rate: 7.8 symbols/min [2025-10-20T20:05:14.651407] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-08. [2025-10-20T20:05:15.475474] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T20:05:15.478304] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-09. [2025-10-20T20:05:16.439446] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T20:05:16.443328] [INFO ] [REMOTE ] MYD: Fetching primary data for 2025-10. [2025-10-20T20:05:17.361566] [INFO ] [REMOTE ] MYD: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T20:05:17.364343] [INFO ] [REMOTE ] MYD: Full fetch complete (20135 records). [2025-10-20T20:05:17.364482] [DEBUG] [VALIDATOR ] MYD: Validating primary data. [2025-10-20T20:05:17.756116] [DEBUG] [VALIDATOR ] MYD: All expected trading days are present. [2025-10-20T20:05:17.756195] [DEBUG] [VALIDATOR ] MYD: Primary data validation passed. [2025-10-20T20:05:17.756584] [INFO ] [MAIN ] MYD: Fetched primary data is valid. [2025-10-20T20:05:17.756625] [DEBUG] [LOCAL ] MYD: Saving primary data. [2025-10-20T20:05:17.757204] [DEBUG] [LOCAL ] MYD: Trimmed 20135 records to 13274 (keeping 200 days of data) [2025-10-20T20:05:17.841210] [DEBUG] [LOCAL ] MYD: Primary data saved (13274 records). [2025-10-20T20:05:17.841344] [INFO ] [MAIN ] MYD: Full history saved. [2025-10-20T20:05:17.841419] [INFO ] [MAIN ] Update Progress: 1440/2367 symbols completed. MYD took 30.6s. Estimated remaining: 1h 59m 14s. Rate: 7.8 symbols/min [2025-10-20T20:05:18.557169] [INFO ] [MAIN ] MYE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:18.559787] [INFO ] [MAIN ] MYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:18.558707] [INFO ] [MAIN ] NABL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:18.549421] [INFO ] [MAIN ] MYN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:18.572590] [DEBUG] [LOCAL ] MYN: Loaded 10035 existing records from primary storage. [2025-10-20T20:05:18.573092] [INFO ] [MAIN ] MYN: Loaded 10035 existing records. [2025-10-20T20:05:18.719004] [DEBUG] [LOCAL ] MYI: Loaded 19288 existing records from primary storage. [2025-10-20T20:05:18.717875] [DEBUG] [LOCAL ] MYE: Loaded 24710 existing records from primary storage. [2025-10-20T20:05:18.730891] [INFO ] [MAIN ] MYI: Loaded 19288 existing records. [2025-10-20T20:05:18.767228] [INFO ] [MAIN ] MYE: Loaded 24710 existing records. [2025-10-20T20:05:18.772904] [DEBUG] [LOCAL ] NABL: Loaded 39737 existing records from primary storage. [2025-10-20T20:05:18.815273] [INFO ] [MAIN ] NABL: Loaded 39737 existing records. [2025-10-20T20:05:18.905252] [WARN ] [MAIN ] MYN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:18.981234] [INFO ] [MAIN ] MYN: Wiping existing primary data before full refresh. [2025-10-20T20:05:19.034726] [INFO ] [LOCAL ] MYN: Starting deletion of primary data. [2025-10-20T20:05:19.118804] [DEBUG] [LOCAL ] MYN: Deleted primary data file. [2025-10-20T20:05:19.165504] [INFO ] [MAIN ] MYN: Starting full primary refresh. [2025-10-20T20:05:19.258042] [WARN ] [MAIN ] MYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:19.345101] [INFO ] [MAIN ] MYI: Wiping existing primary data before full refresh. [2025-10-20T20:05:19.350435] [INFO ] [LOCAL ] MYI: Starting deletion of primary data. [2025-10-20T20:05:19.427159] [WARN ] [MAIN ] NABL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:19.449368] [DEBUG] [LOCAL ] MYI: Deleted primary data file. [2025-10-20T20:05:19.455041] [INFO ] [MAIN ] NABL: Wiping existing primary data before full refresh. [2025-10-20T20:05:19.460708] [INFO ] [LOCAL ] NABL: Starting deletion of primary data. [2025-10-20T20:05:19.460594] [INFO ] [MAIN ] MYI: Starting full primary refresh. [2025-10-20T20:05:19.460502] [WARN ] [MAIN ] MYE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:19.466164] [DEBUG] [LOCAL ] NABL: Deleted primary data file. [2025-10-20T20:05:19.506253] [INFO ] [MAIN ] MYE: Wiping existing primary data before full refresh. [2025-10-20T20:05:19.506093] [INFO ] [MAIN ] NABL: Starting full primary refresh. [2025-10-20T20:05:19.511599] [INFO ] [LOCAL ] MYE: Starting deletion of primary data. [2025-10-20T20:05:19.542668] [DEBUG] [LOCAL ] MYE: Deleted primary data file. [2025-10-20T20:05:19.625711] [INFO ] [MAIN ] MYE: Starting full primary refresh. [2025-10-20T20:05:20.354427] [INFO ] [REMOTE ] MYN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:20.405125] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-01. [2025-10-20T20:05:20.684058] [INFO ] [REMOTE ] MYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:20.700790] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-01. [2025-10-20T20:05:20.839134] [INFO ] [REMOTE ] NABL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:20.855350] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-01. [2025-10-20T20:05:20.861187] [INFO ] [REMOTE ] MYE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:20.861513] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-01. [2025-10-20T20:05:21.392597] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T20:05:21.395427] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-02. [2025-10-20T20:05:21.808944] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T20:05:21.813302] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-02. [2025-10-20T20:05:21.850368] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T20:05:21.857795] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-02. [2025-10-20T20:05:22.137306] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T20:05:22.338870] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T20:05:22.651342] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-03. [2025-10-20T20:05:22.914439] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T20:05:23.034086] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T20:05:23.451391] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-03. [2025-10-20T20:05:23.718834] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T20:05:24.251368] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-02. [2025-10-20T20:05:24.279537] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T20:05:25.051369] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-04. [2025-10-20T20:05:25.292424] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T20:05:25.851402] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-04. [2025-10-20T20:05:25.876359] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:05:26.651377] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-03. [2025-10-20T20:05:26.968864] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T20:05:27.451422] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-05. [2025-10-20T20:05:28.122618] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T20:05:28.251405] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-04. [2025-10-20T20:05:28.407903] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T20:05:29.051404] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-06. [2025-10-20T20:05:29.815238] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T20:05:29.851389] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-05. [2025-10-20T20:05:30.184614] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:05:30.651386] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-05. [2025-10-20T20:05:31.380989] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T20:05:31.405444] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:05:31.451370] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-06. [2025-10-20T20:05:32.251385] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-06. [2025-10-20T20:05:32.316383] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T20:05:33.051427] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-03. [2025-10-20T20:05:33.715888] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:05:33.851381] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-07. [2025-10-20T20:05:34.435629] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T20:05:34.651379] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-04. [2025-10-20T20:05:34.893235] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T20:05:35.451401] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-08. [2025-10-20T20:05:35.954267] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T20:05:36.251400] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-07. [2025-10-20T20:05:36.814866] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T20:05:37.041335] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:05:37.051341] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-08. [2025-10-20T20:05:37.847546] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T20:05:37.851336] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-05. [2025-10-20T20:05:38.651388] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-07. [2025-10-20T20:05:38.812452] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T20:05:39.451396] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-09. [2025-10-20T20:05:39.622326] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T20:05:40.251467] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-06. [2025-10-20T20:05:40.600136] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T20:05:41.051369] [INFO ] [REMOTE ] NABL: Fetching primary data for 2025-10. [2025-10-20T20:05:41.367387] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T20:05:41.851371] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-09. [2025-10-20T20:05:42.471371] [INFO ] [REMOTE ] NABL: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T20:05:42.477072] [INFO ] [REMOTE ] NABL: Full fetch complete (56980 records). [2025-10-20T20:05:42.477216] [DEBUG] [VALIDATOR ] NABL: Validating primary data. [2025-10-20T20:05:42.602068] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T20:05:42.663606] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-07. [2025-10-20T20:05:42.875980] [DEBUG] [VALIDATOR ] NABL: All expected trading days are present. [2025-10-20T20:05:42.876070] [DEBUG] [VALIDATOR ] NABL: Primary data validation passed. [2025-10-20T20:05:42.876974] [INFO ] [MAIN ] NABL: Fetched primary data is valid. [2025-10-20T20:05:42.877040] [DEBUG] [LOCAL ] NABL: Saving primary data. [2025-10-20T20:05:42.878243] [DEBUG] [LOCAL ] NABL: Trimmed 56980 records to 39071 (keeping 200 days of data) [2025-10-20T20:05:43.098678] [DEBUG] [LOCAL ] NABL: Primary data saved (39071 records). [2025-10-20T20:05:43.098807] [INFO ] [MAIN ] NABL: Full history saved. [2025-10-20T20:05:43.099433] [INFO ] [MAIN ] Update Progress: 1441/2367 symbols completed. NABL took 25.3s. Estimated remaining: 1h 59m 18s. Rate: 7.8 symbols/min [2025-10-20T20:05:43.265937] [INFO ] [MAIN ] NAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:43.290984] [DEBUG] [LOCAL ] NAC: Loaded 24827 existing records from primary storage. [2025-10-20T20:05:43.291245] [INFO ] [MAIN ] NAC: Loaded 24827 existing records. [2025-10-20T20:05:43.459952] [INFO ] [REMOTE ] MYN: Fetching primary data for 2025-10. [2025-10-20T20:05:43.464432] [WARN ] [MAIN ] NAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:43.465304] [INFO ] [MAIN ] NAC: Wiping existing primary data before full refresh. [2025-10-20T20:05:43.465351] [INFO ] [LOCAL ] NAC: Starting deletion of primary data. [2025-10-20T20:05:43.465515] [DEBUG] [LOCAL ] NAC: Deleted primary data file. [2025-10-20T20:05:43.465562] [INFO ] [MAIN ] NAC: Starting full primary refresh. [2025-10-20T20:05:43.740753] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T20:05:43.841299] [INFO ] [REMOTE ] NAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:44.216700] [INFO ] [REMOTE ] MYN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:05:44.218685] [INFO ] [REMOTE ] MYN: Full fetch complete (15098 records). [2025-10-20T20:05:44.218754] [DEBUG] [VALIDATOR ] MYN: Validating primary data. [2025-10-20T20:05:44.257203] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-08. [2025-10-20T20:05:44.578201] [DEBUG] [VALIDATOR ] MYN: All expected trading days are present. [2025-10-20T20:05:44.578312] [DEBUG] [VALIDATOR ] MYN: Primary data validation passed. [2025-10-20T20:05:44.578661] [INFO ] [MAIN ] MYN: Fetched primary data is valid. [2025-10-20T20:05:44.578721] [DEBUG] [LOCAL ] MYN: Saving primary data. [2025-10-20T20:05:44.579179] [DEBUG] [LOCAL ] MYN: Trimmed 15098 records to 9964 (keeping 200 days of data) [2025-10-20T20:05:44.635848] [DEBUG] [LOCAL ] MYN: Primary data saved (9964 records). [2025-10-20T20:05:44.635949] [INFO ] [MAIN ] MYN: Full history saved. [2025-10-20T20:05:44.636021] [INFO ] [MAIN ] Update Progress: 1442/2367 symbols completed. MYN took 26.8s. Estimated remaining: 1h 59m 6s. Rate: 7.8 symbols/min [2025-10-20T20:05:44.802303] [INFO ] [MAIN ] NAD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:44.835939] [DEBUG] [LOCAL ] NAD: Loaded 34190 existing records from primary storage. [2025-10-20T20:05:44.836199] [INFO ] [MAIN ] NAD: Loaded 34190 existing records. [2025-10-20T20:05:45.010517] [WARN ] [MAIN ] NAD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:45.010755] [INFO ] [MAIN ] NAD: Wiping existing primary data before full refresh. [2025-10-20T20:05:45.010806] [INFO ] [LOCAL ] NAD: Starting deletion of primary data. [2025-10-20T20:05:45.011001] [DEBUG] [LOCAL ] NAD: Deleted primary data file. [2025-10-20T20:05:45.011058] [INFO ] [MAIN ] NAD: Starting full primary refresh. [2025-10-20T20:05:45.056604] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-08. [2025-10-20T20:05:45.372231] [INFO ] [REMOTE ] NAD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:45.465120] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T20:05:45.851414] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-01. [2025-10-20T20:05:46.132633] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T20:05:46.651402] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-01. [2025-10-20T20:05:46.928952] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T20:05:47.451381] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-09. [2025-10-20T20:05:48.057175] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T20:05:48.251354] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-02. [2025-10-20T20:05:48.363701] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T20:05:49.051374] [INFO ] [REMOTE ] MYE: Fetching primary data for 2025-10. [2025-10-20T20:05:49.541327] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:05:49.851357] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-09. [2025-10-20T20:05:50.215943] [INFO ] [REMOTE ] MYE: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T20:05:50.219676] [INFO ] [REMOTE ] MYE: Full fetch complete (36009 records). [2025-10-20T20:05:50.219758] [DEBUG] [VALIDATOR ] MYE: Validating primary data. [2025-10-20T20:05:50.588147] [DEBUG] [VALIDATOR ] MYE: All expected trading days are present. [2025-10-20T20:05:50.588247] [DEBUG] [VALIDATOR ] MYE: Primary data validation passed. [2025-10-20T20:05:50.589035] [INFO ] [MAIN ] MYE: Fetched primary data is valid. [2025-10-20T20:05:50.589095] [DEBUG] [LOCAL ] MYE: Saving primary data. [2025-10-20T20:05:50.589859] [DEBUG] [LOCAL ] MYE: Trimmed 36009 records to 24224 (keeping 200 days of data) [2025-10-20T20:05:50.675944] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-03. [2025-10-20T20:05:50.724735] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:05:50.759793] [DEBUG] [LOCAL ] MYE: Primary data saved (24224 records). [2025-10-20T20:05:50.759920] [INFO ] [MAIN ] MYE: Full history saved. [2025-10-20T20:05:50.759996] [INFO ] [MAIN ] Update Progress: 1443/2367 symbols completed. MYE took 32.9s. Estimated remaining: 1h 58m 57s. Rate: 7.8 symbols/min [2025-10-20T20:05:50.923924] [INFO ] [MAIN ] NAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:50.934438] [DEBUG] [LOCAL ] NAN: Loaded 8601 existing records from primary storage. [2025-10-20T20:05:50.934673] [INFO ] [MAIN ] NAN: Loaded 8601 existing records. [2025-10-20T20:05:51.108051] [WARN ] [MAIN ] NAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:51.108351] [INFO ] [MAIN ] NAN: Wiping existing primary data before full refresh. [2025-10-20T20:05:51.108407] [INFO ] [LOCAL ] NAN: Starting deletion of primary data. [2025-10-20T20:05:51.108563] [DEBUG] [LOCAL ] NAN: Deleted primary data file. [2025-10-20T20:05:51.108599] [INFO ] [MAIN ] NAN: Starting full primary refresh. [2025-10-20T20:05:51.456468] [INFO ] [REMOTE ] NAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:51.452788] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-02. [2025-10-20T20:05:51.743137] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T20:05:52.144053] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T20:05:52.251339] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-03. [2025-10-20T20:05:53.051339] [INFO ] [REMOTE ] MYI: Fetching primary data for 2025-10. [2025-10-20T20:05:53.348811] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T20:05:53.851335] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-04. [2025-10-20T20:05:54.129130] [INFO ] [REMOTE ] MYI: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T20:05:54.132406] [INFO ] [REMOTE ] MYI: Full fetch complete (28414 records). [2025-10-20T20:05:54.132483] [DEBUG] [VALIDATOR ] MYI: Validating primary data. [2025-10-20T20:05:54.508824] [DEBUG] [VALIDATOR ] MYI: All expected trading days are present. [2025-10-20T20:05:54.508901] [DEBUG] [VALIDATOR ] MYI: Primary data validation passed. [2025-10-20T20:05:54.509532] [INFO ] [MAIN ] MYI: Fetched primary data is valid. [2025-10-20T20:05:54.509570] [DEBUG] [LOCAL ] MYI: Saving primary data. [2025-10-20T20:05:54.510243] [DEBUG] [LOCAL ] MYI: Trimmed 28414 records to 18937 (keeping 200 days of data) [2025-10-20T20:05:54.634462] [DEBUG] [LOCAL ] MYI: Primary data saved (18937 records). [2025-10-20T20:05:54.634602] [INFO ] [MAIN ] MYI: Full history saved. [2025-10-20T20:05:54.634704] [INFO ] [MAIN ] Update Progress: 1444/2367 symbols completed. MYI took 36.8s. Estimated remaining: 1h 58m 47s. Rate: 7.8 symbols/min [2025-10-20T20:05:54.656480] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-01. [2025-10-20T20:05:54.830186] [INFO ] [MAIN ] NAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:05:54.873305] [DEBUG] [LOCAL ] NAT: Loaded 49970 existing records from primary storage. [2025-10-20T20:05:54.873639] [INFO ] [MAIN ] NAT: Loaded 49970 existing records. [2025-10-20T20:05:55.055560] [WARN ] [MAIN ] NAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:05:55.055889] [INFO ] [MAIN ] NAT: Wiping existing primary data before full refresh. [2025-10-20T20:05:55.055938] [INFO ] [LOCAL ] NAT: Starting deletion of primary data. [2025-10-20T20:05:55.056143] [DEBUG] [LOCAL ] NAT: Deleted primary data file. [2025-10-20T20:05:55.056203] [INFO ] [MAIN ] NAT: Starting full primary refresh. [2025-10-20T20:05:55.259557] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T20:05:55.454521] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-04. [2025-10-20T20:05:55.457445] [INFO ] [REMOTE ] NAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:05:55.495341] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T20:05:56.251381] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-05. [2025-10-20T20:05:56.963359] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T20:05:57.051349] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-05. [2025-10-20T20:05:57.291089] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T20:05:57.851356] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-06. [2025-10-20T20:05:58.107134] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T20:05:58.651420] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-02. [2025-10-20T20:05:58.915458] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T20:05:59.418831] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:05:59.451393] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-07. [2025-10-20T20:06:00.251393] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-06. [2025-10-20T20:06:00.487744] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T20:06:01.051377] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-08. [2025-10-20T20:06:01.683841] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T20:06:01.816311] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T20:06:01.851361] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-09. [2025-10-20T20:06:02.651375] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-03. [2025-10-20T20:06:03.438184] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T20:06:03.454039] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-01. [2025-10-20T20:06:03.455636] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T20:06:04.251435] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-07. [2025-10-20T20:06:05.051386] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-04. [2025-10-20T20:06:05.225397] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:06:05.293625] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T20:06:05.851448] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-08. [2025-10-20T20:06:05.875032] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:06:06.651426] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-05. [2025-10-20T20:06:06.979558] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T20:06:07.359313] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T20:06:07.451355] [INFO ] [REMOTE ] NAC: Fetching primary data for 2025-10. [2025-10-20T20:06:08.251371] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-06. [2025-10-20T20:06:08.696254] [INFO ] [REMOTE ] NAC: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T20:06:08.699892] [INFO ] [REMOTE ] NAC: Full fetch complete (35393 records). [2025-10-20T20:06:08.699992] [DEBUG] [VALIDATOR ] NAC: Validating primary data. [2025-10-20T20:06:08.900139] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T20:06:09.064940] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-02. [2025-10-20T20:06:09.137928] [DEBUG] [VALIDATOR ] NAC: All expected trading days are present. [2025-10-20T20:06:09.138059] [DEBUG] [VALIDATOR ] NAC: Primary data validation passed. [2025-10-20T20:06:09.139739] [INFO ] [MAIN ] NAC: Fetched primary data is valid. [2025-10-20T20:06:09.139849] [DEBUG] [LOCAL ] NAC: Saving primary data. [2025-10-20T20:06:09.141226] [DEBUG] [LOCAL ] NAC: Trimmed 35393 records to 24479 (keeping 200 days of data) [2025-10-20T20:06:09.348182] [DEBUG] [LOCAL ] NAC: Primary data saved (24479 records). [2025-10-20T20:06:09.348372] [INFO ] [MAIN ] NAC: Full history saved. [2025-10-20T20:06:09.348492] [INFO ] [MAIN ] Update Progress: 1445/2367 symbols completed. NAC took 26.2s. Estimated remaining: 1h 58m 44s. Rate: 7.8 symbols/min [2025-10-20T20:06:09.568469] [INFO ] [MAIN ] NATL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:09.602262] [DEBUG] [LOCAL ] NATL: Loaded 33023 existing records from primary storage. [2025-10-20T20:06:09.602571] [INFO ] [MAIN ] NATL: Loaded 33023 existing records. [2025-10-20T20:06:09.856716] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-07. [2025-10-20T20:06:09.877097] [WARN ] [MAIN ] NATL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:09.878846] [INFO ] [MAIN ] NATL: Wiping existing primary data before full refresh. [2025-10-20T20:06:09.878926] [INFO ] [LOCAL ] NATL: Starting deletion of primary data. [2025-10-20T20:06:09.879200] [DEBUG] [LOCAL ] NATL: Deleted primary data file. [2025-10-20T20:06:09.879276] [INFO ] [MAIN ] NATL: Starting full primary refresh. [2025-10-20T20:06:10.316774] [INFO ] [REMOTE ] NATL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:10.651364] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-09. [2025-10-20T20:06:10.682668] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T20:06:10.915571] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T20:06:11.451412] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-01. [2025-10-20T20:06:11.740380] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:06:12.251336] [INFO ] [REMOTE ] NAD: Fetching primary data for 2025-10. [2025-10-20T20:06:12.628318] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T20:06:13.051385] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-02. [2025-10-20T20:06:13.514417] [INFO ] [REMOTE ] NAD: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T20:06:13.519337] [INFO ] [REMOTE ] NAD: Full fetch complete (48708 records). [2025-10-20T20:06:13.519440] [DEBUG] [VALIDATOR ] NAD: Validating primary data. [2025-10-20T20:06:13.856242] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-08. [2025-10-20T20:06:13.905214] [DEBUG] [VALIDATOR ] NAD: All expected trading days are present. [2025-10-20T20:06:13.905383] [DEBUG] [VALIDATOR ] NAD: Primary data validation passed. [2025-10-20T20:06:13.906460] [INFO ] [MAIN ] NAD: Fetched primary data is valid. [2025-10-20T20:06:13.906681] [DEBUG] [LOCAL ] NAD: Saving primary data. [2025-10-20T20:06:13.907911] [DEBUG] [LOCAL ] NAD: Trimmed 48708 records to 33743 (keeping 200 days of data) [2025-10-20T20:06:14.140155] [DEBUG] [LOCAL ] NAD: Primary data saved (33743 records). [2025-10-20T20:06:14.141296] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T20:06:14.140785] [INFO ] [MAIN ] NAD: Full history saved. [2025-10-20T20:06:14.146669] [INFO ] [MAIN ] Update Progress: 1446/2367 symbols completed. NAD took 29.5s. Estimated remaining: 1h 58m 34s. Rate: 7.8 symbols/min [2025-10-20T20:06:14.311708] [INFO ] [MAIN ] NAZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:14.316582] [DEBUG] [LOCAL ] NAZ: Loaded 2699 existing records from primary storage. [2025-10-20T20:06:14.316760] [INFO ] [MAIN ] NAZ: Loaded 2699 existing records. [2025-10-20T20:06:14.495869] [WARN ] [MAIN ] NAZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:14.496092] [INFO ] [MAIN ] NAZ: Wiping existing primary data before full refresh. [2025-10-20T20:06:14.496159] [INFO ] [LOCAL ] NAZ: Starting deletion of primary data. [2025-10-20T20:06:14.496315] [DEBUG] [LOCAL ] NAZ: Deleted primary data file. [2025-10-20T20:06:14.496373] [INFO ] [MAIN ] NAZ: Starting full primary refresh. [2025-10-20T20:06:14.656883] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-03. [2025-10-20T20:06:14.763095] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T20:06:14.872863] [INFO ] [REMOTE ] NAZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:15.451449] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-01. [2025-10-20T20:06:15.945788] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-10-20T20:06:16.098965] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T20:06:16.251387] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-04. [2025-10-20T20:06:17.051371] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-02. [2025-10-20T20:06:17.679602] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T20:06:17.682529] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T20:06:17.851376] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-03. [2025-10-20T20:06:18.267708] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T20:06:18.651371] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-05. [2025-10-20T20:06:19.451407] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-03. [2025-10-20T20:06:19.875448] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T20:06:20.251406] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-06. [2025-10-20T20:06:21.051366] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-09. [2025-10-20T20:06:21.231794] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T20:06:21.510023] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T20:06:21.851426] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-07. [2025-10-20T20:06:21.876242] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:06:22.651369] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-04. [2025-10-20T20:06:23.140211] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T20:06:23.255572] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:06:23.451374] [INFO ] [REMOTE ] NAN: Fetching primary data for 2025-10. [2025-10-20T20:06:24.210227] [INFO ] [REMOTE ] NAN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:06:24.212048] [INFO ] [REMOTE ] NAN: Full fetch complete (13507 records). [2025-10-20T20:06:24.212143] [DEBUG] [VALIDATOR ] NAN: Validating primary data. [2025-10-20T20:06:24.256516] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-08. [2025-10-20T20:06:24.580578] [DEBUG] [VALIDATOR ] NAN: All expected trading days are present. [2025-10-20T20:06:24.580677] [DEBUG] [VALIDATOR ] NAN: Primary data validation passed. [2025-10-20T20:06:24.581015] [INFO ] [MAIN ] NAN: Fetched primary data is valid. [2025-10-20T20:06:24.581059] [DEBUG] [LOCAL ] NAN: Saving primary data. [2025-10-20T20:06:24.581514] [DEBUG] [LOCAL ] NAN: Trimmed 13507 records to 8464 (keeping 200 days of data) [2025-10-20T20:06:24.631544] [DEBUG] [LOCAL ] NAN: Primary data saved (8464 records). [2025-10-20T20:06:24.631690] [INFO ] [MAIN ] NAN: Full history saved. [2025-10-20T20:06:24.631780] [INFO ] [MAIN ] Update Progress: 1447/2367 symbols completed. NAN took 33.9s. Estimated remaining: 1h 58m 28s. Rate: 7.8 symbols/min [2025-10-20T20:06:24.805192] [INFO ] [MAIN ] NBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:24.815795] [DEBUG] [LOCAL ] NBB: Loaded 9173 existing records from primary storage. [2025-10-20T20:06:24.815966] [INFO ] [MAIN ] NBB: Loaded 9173 existing records. [2025-10-20T20:06:24.991075] [WARN ] [MAIN ] NBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:24.991272] [INFO ] [MAIN ] NBB: Wiping existing primary data before full refresh. [2025-10-20T20:06:24.991319] [INFO ] [LOCAL ] NBB: Starting deletion of primary data. [2025-10-20T20:06:24.991473] [DEBUG] [LOCAL ] NBB: Deleted primary data file. [2025-10-20T20:06:24.991509] [INFO ] [MAIN ] NBB: Starting full primary refresh. [2025-10-20T20:06:25.056559] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-04. [2025-10-20T20:06:25.368171] [INFO ] [REMOTE ] NBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:25.819888] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T20:06:25.851358] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-01. [2025-10-20T20:06:26.607389] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T20:06:26.651364] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-05. [2025-10-20T20:06:26.943627] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-05 (0.29 seconds). [2025-10-20T20:06:26.993912] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T20:06:27.451361] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-09. [2025-10-20T20:06:28.251405] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-05. [2025-10-20T20:06:28.935916] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T20:06:29.051366] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-06. [2025-10-20T20:06:29.565999] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T20:06:29.851374] [INFO ] [REMOTE ] NATL: Fetching primary data for 2025-10. [2025-10-20T20:06:30.059976] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T20:06:30.651367] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-07. [2025-10-20T20:06:31.231838] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T20:06:31.451411] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-02. [2025-10-20T20:06:31.477262] [INFO ] [REMOTE ] NATL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T20:06:31.481753] [INFO ] [REMOTE ] NATL: Full fetch complete (45127 records). [2025-10-20T20:06:31.481850] [DEBUG] [VALIDATOR ] NATL: Validating primary data. [2025-10-20T20:06:31.866014] [DEBUG] [VALIDATOR ] NATL: All expected trading days are present. [2025-10-20T20:06:31.866145] [DEBUG] [VALIDATOR ] NATL: Primary data validation passed. [2025-10-20T20:06:31.867140] [INFO ] [MAIN ] NATL: Fetched primary data is valid. [2025-10-20T20:06:31.867217] [DEBUG] [LOCAL ] NATL: Saving primary data. [2025-10-20T20:06:31.868377] [DEBUG] [LOCAL ] NATL: Trimmed 45127 records to 32454 (keeping 200 days of data) [2025-10-20T20:06:31.966186] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T20:06:32.063166] [DEBUG] [LOCAL ] NATL: Primary data saved (32454 records). [2025-10-20T20:06:32.063295] [INFO ] [MAIN ] NATL: Full history saved. [2025-10-20T20:06:32.063629] [INFO ] [MAIN ] Update Progress: 1448/2367 symbols completed. NATL took 22.7s. Estimated remaining: 1h 58m 20s. Rate: 7.8 symbols/min [2025-10-20T20:06:32.231775] [INFO ] [MAIN ] NBHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:32.255559] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-08. [2025-10-20T20:06:32.256033] [DEBUG] [LOCAL ] NBHC: Loaded 24150 existing records from primary storage. [2025-10-20T20:06:32.257471] [INFO ] [MAIN ] NBHC: Loaded 24150 existing records. [2025-10-20T20:06:32.433575] [WARN ] [MAIN ] NBHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:32.433837] [INFO ] [MAIN ] NBHC: Wiping existing primary data before full refresh. [2025-10-20T20:06:32.433885] [INFO ] [LOCAL ] NBHC: Starting deletion of primary data. [2025-10-20T20:06:32.434100] [DEBUG] [LOCAL ] NBHC: Deleted primary data file. [2025-10-20T20:06:32.434156] [INFO ] [MAIN ] NBHC: Starting full primary refresh. [2025-10-20T20:06:32.792336] [INFO ] [REMOTE ] NBHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:32.812609] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T20:06:33.051378] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-01. [2025-10-20T20:06:33.851365] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-06. [2025-10-20T20:06:34.082630] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:06:34.651394] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-02. [2025-10-20T20:06:35.451354] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-03. [2025-10-20T20:06:35.552991] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T20:06:35.686789] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T20:06:36.251348] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-03. [2025-10-20T20:06:36.328396] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T20:06:37.051355] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-04. [2025-10-20T20:06:37.663862] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T20:06:37.851388] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-09. [2025-10-20T20:06:37.891508] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T20:06:38.425436] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T20:06:38.651425] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-05. [2025-10-20T20:06:39.367481] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T20:06:39.451340] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-06. [2025-10-20T20:06:40.182435] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T20:06:40.251352] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-07. [2025-10-20T20:06:41.012875] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T20:06:41.051366] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-08. [2025-10-20T20:06:41.815553] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T20:06:41.851396] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-09. [2025-10-20T20:06:42.594493] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T20:06:42.651375] [INFO ] [REMOTE ] NBB: Fetching primary data for 2025-10. [2025-10-20T20:06:43.403428] [INFO ] [REMOTE ] NBB: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:06:43.405214] [INFO ] [REMOTE ] NBB: Full fetch complete (13177 records). [2025-10-20T20:06:43.405282] [DEBUG] [VALIDATOR ] NBB: Validating primary data. [2025-10-20T20:06:43.456597] [INFO ] [REMOTE ] NAZ: Fetching primary data for 2025-10. [2025-10-20T20:06:43.812368] [DEBUG] [VALIDATOR ] NBB: All expected trading days are present. [2025-10-20T20:06:43.812443] [DEBUG] [VALIDATOR ] NBB: Primary data validation passed. [2025-10-20T20:06:43.812774] [INFO ] [MAIN ] NBB: Fetched primary data is valid. [2025-10-20T20:06:43.812834] [DEBUG] [LOCAL ] NBB: Saving primary data. [2025-10-20T20:06:43.813288] [DEBUG] [LOCAL ] NBB: Trimmed 13177 records to 9040 (keeping 200 days of data) [2025-10-20T20:06:43.871606] [DEBUG] [LOCAL ] NBB: Primary data saved (9040 records). [2025-10-20T20:06:43.872009] [INFO ] [MAIN ] NBB: Full history saved. [2025-10-20T20:06:43.872117] [INFO ] [MAIN ] Update Progress: 1449/2367 symbols completed. NBB took 19.2s. Estimated remaining: 1h 58m 15s. Rate: 7.8 symbols/min [2025-10-20T20:06:44.046071] [INFO ] [MAIN ] NBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:44.077749] [DEBUG] [LOCAL ] NBR: Loaded 34897 existing records from primary storage. [2025-10-20T20:06:44.078008] [INFO ] [MAIN ] NBR: Loaded 34897 existing records. [2025-10-20T20:06:44.163539] [INFO ] [REMOTE ] NAZ: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:06:44.186895] [INFO ] [REMOTE ] NAZ: Full fetch complete (4108 records). [2025-10-20T20:06:44.187093] [DEBUG] [VALIDATOR ] NAZ: Validating primary data. [2025-10-20T20:06:44.261546] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-07. [2025-10-20T20:06:44.351106] [WARN ] [MAIN ] NBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:44.369930] [INFO ] [MAIN ] NBR: Wiping existing primary data before full refresh. [2025-10-20T20:06:44.370029] [INFO ] [LOCAL ] NBR: Starting deletion of primary data. [2025-10-20T20:06:44.375953] [DEBUG] [LOCAL ] NBR: Deleted primary data file. [2025-10-20T20:06:44.409509] [INFO ] [MAIN ] NBR: Starting full primary refresh. [2025-10-20T20:06:44.820338] [DEBUG] [VALIDATOR ] NAZ: All expected trading days are present. [2025-10-20T20:06:44.825905] [DEBUG] [VALIDATOR ] NAZ: Primary data validation passed. [2025-10-20T20:06:44.829118] [INFO ] [MAIN ] NAZ: Fetched primary data is valid. [2025-10-20T20:06:44.834616] [DEBUG] [LOCAL ] NAZ: Saving primary data. [2025-10-20T20:06:44.836953] [DEBUG] [LOCAL ] NAZ: Trimmed 4108 records to 2677 (keeping 200 days of data) [2025-10-20T20:06:44.979467] [DEBUG] [LOCAL ] NAZ: Primary data saved (2677 records). [2025-10-20T20:06:44.982206] [INFO ] [MAIN ] NAZ: Full history saved. [2025-10-20T20:06:44.982618] [INFO ] [MAIN ] Update Progress: 1450/2367 symbols completed. NAZ took 30.8s. Estimated remaining: 1h 58m 3s. Rate: 7.8 symbols/min [2025-10-20T20:06:45.064905] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-04. [2025-10-20T20:06:45.144880] [INFO ] [REMOTE ] NBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:45.242362] [INFO ] [MAIN ] NBXG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:06:45.264072] [DEBUG] [LOCAL ] NBXG: Loaded 22256 existing records from primary storage. [2025-10-20T20:06:45.264337] [INFO ] [MAIN ] NBXG: Loaded 22256 existing records. [2025-10-20T20:06:45.446057] [WARN ] [MAIN ] NBXG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:06:45.446295] [INFO ] [MAIN ] NBXG: Wiping existing primary data before full refresh. [2025-10-20T20:06:45.446353] [INFO ] [LOCAL ] NBXG: Starting deletion of primary data. [2025-10-20T20:06:45.446542] [DEBUG] [LOCAL ] NBXG: Deleted primary data file. [2025-10-20T20:06:45.446600] [INFO ] [MAIN ] NBXG: Starting full primary refresh. [2025-10-20T20:06:45.809450] [INFO ] [REMOTE ] NBXG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:06:45.851333] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-01. [2025-10-20T20:06:46.297271] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T20:06:46.323850] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T20:06:46.651344] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-05. [2025-10-20T20:06:46.880689] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:06:47.451443] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-02. [2025-10-20T20:06:47.654889] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T20:06:48.251401] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-08. [2025-10-20T20:06:48.458968] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T20:06:49.051357] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-01. [2025-10-20T20:06:49.851384] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-03. [2025-10-20T20:06:49.854118] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:06:50.368985] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T20:06:50.651379] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-02. [2025-10-20T20:06:51.068359] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T20:06:51.451340] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-04. [2025-10-20T20:06:51.919608] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T20:06:52.251355] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-03. [2025-10-20T20:06:52.581575] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T20:06:53.051405] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-05. [2025-10-20T20:06:53.621545] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T20:06:53.851366] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-06. [2025-10-20T20:06:54.124435] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T20:06:54.651351] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-06. [2025-10-20T20:06:54.843049] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T20:06:55.358975] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T20:06:55.451384] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-07. [2025-10-20T20:06:56.251377] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-07. [2025-10-20T20:06:56.612518] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T20:06:57.051367] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-08. [2025-10-20T20:06:57.153346] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T20:06:57.851417] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-09. [2025-10-20T20:06:58.118026] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T20:06:58.651505] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-09. [2025-10-20T20:06:59.451381] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-04. [2025-10-20T20:06:59.557611] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:06:59.663118] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T20:07:00.251384] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-08. [2025-10-20T20:07:00.546160] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:07:01.051340] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-05. [2025-10-20T20:07:01.373077] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:07:01.851412] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-09. [2025-10-20T20:07:02.559326] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T20:07:02.660880] [INFO ] [REMOTE ] NAT: Fetching primary data for 2025-10. [2025-10-20T20:07:02.664354] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:07:03.451412] [INFO ] [REMOTE ] NBXG: Fetching primary data for 2025-10. [2025-10-20T20:07:04.251372] [INFO ] [REMOTE ] NBHC: Fetching primary data for 2025-10. [2025-10-20T20:07:04.472698] [INFO ] [REMOTE ] NBXG: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T20:07:04.475993] [INFO ] [REMOTE ] NBXG: Full fetch complete (32482 records). [2025-10-20T20:07:04.476106] [DEBUG] [VALIDATOR ] NBXG: Validating primary data. [2025-10-20T20:07:04.806269] [INFO ] [REMOTE ] NAT: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T20:07:04.814109] [INFO ] [REMOTE ] NAT: Full fetch complete (72231 records). [2025-10-20T20:07:04.824851] [DEBUG] [VALIDATOR ] NAT: Validating primary data. [2025-10-20T20:07:05.005953] [DEBUG] [VALIDATOR ] NBXG: All expected trading days are present. [2025-10-20T20:07:05.006078] [DEBUG] [VALIDATOR ] NBXG: Primary data validation passed. [2025-10-20T20:07:05.007048] [INFO ] [MAIN ] NBXG: Fetched primary data is valid. [2025-10-20T20:07:05.007129] [DEBUG] [LOCAL ] NBXG: Saving primary data. [2025-10-20T20:07:05.056532] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-06. [2025-10-20T20:07:05.066509] [DEBUG] [LOCAL ] NBXG: Trimmed 32482 records to 21838 (keeping 200 days of data) [2025-10-20T20:07:05.415520] [DEBUG] [VALIDATOR ] NAT: All expected trading days are present. [2025-10-20T20:07:05.422433] [DEBUG] [VALIDATOR ] NAT: Primary data validation passed. [2025-10-20T20:07:05.423544] [INFO ] [MAIN ] NAT: Fetched primary data is valid. [2025-10-20T20:07:05.423626] [DEBUG] [LOCAL ] NAT: Saving primary data. [2025-10-20T20:07:05.450452] [DEBUG] [LOCAL ] NAT: Trimmed 72231 records to 49239 (keeping 200 days of data) [2025-10-20T20:07:05.454767] [DEBUG] [LOCAL ] NBXG: Primary data saved (21838 records). [2025-10-20T20:07:05.455248] [INFO ] [MAIN ] NBXG: Full history saved. [2025-10-20T20:07:05.455537] [INFO ] [MAIN ] Update Progress: 1451/2367 symbols completed. NBXG took 20.5s. Estimated remaining: 1h 58m 4s. Rate: 7.8 symbols/min [2025-10-20T20:07:05.761632] [INFO ] [MAIN ] NC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:05.834151] [INFO ] [REMOTE ] NBHC: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T20:07:05.872857] [INFO ] [REMOTE ] NBHC: Full fetch complete (33212 records). [2025-10-20T20:07:05.885314] [DEBUG] [VALIDATOR ] NBHC: Validating primary data. [2025-10-20T20:07:05.885560] [DEBUG] [LOCAL ] NC: Loaded 2000 existing records from primary storage. [2025-10-20T20:07:05.906148] [INFO ] [MAIN ] NC: Loaded 2000 existing records. [2025-10-20T20:07:06.120511] [DEBUG] [LOCAL ] NAT: Primary data saved (49239 records). [2025-10-20T20:07:06.128291] [INFO ] [MAIN ] NAT: Full history saved. [2025-10-20T20:07:06.137479] [INFO ] [MAIN ] Update Progress: 1452/2367 symbols completed. NAT took 1m 12s. Estimated remaining: 1h 57m 52s. Rate: 7.8 symbols/min [2025-10-20T20:07:06.535902] [WARN ] [MAIN ] NC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:06.560699] [INFO ] [MAIN ] NC: Wiping existing primary data before full refresh. [2025-10-20T20:07:06.599469] [INFO ] [LOCAL ] NC: Starting deletion of primary data. [2025-10-20T20:07:06.622587] [DEBUG] [LOCAL ] NC: Deleted primary data file. [2025-10-20T20:07:06.638373] [INFO ] [MAIN ] NC: Starting full primary refresh. [2025-10-20T20:07:06.848662] [INFO ] [MAIN ] NCA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:06.848788] [DEBUG] [VALIDATOR ] NBHC: All expected trading days are present. [2025-10-20T20:07:06.872097] [DEBUG] [VALIDATOR ] NBHC: Primary data validation passed. [2025-10-20T20:07:06.888441] [INFO ] [MAIN ] NBHC: Fetched primary data is valid. [2025-10-20T20:07:06.910035] [DEBUG] [LOCAL ] NBHC: Saving primary data. [2025-10-20T20:07:07.012113] [DEBUG] [LOCAL ] NBHC: Trimmed 33212 records to 23824 (keeping 200 days of data) [2025-10-20T20:07:07.030145] [DEBUG] [LOCAL ] NCA: Loaded 10059 existing records from primary storage. [2025-10-20T20:07:07.186785] [INFO ] [MAIN ] NCA: Loaded 10059 existing records. [2025-10-20T20:07:07.203161] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-06 (2.15 seconds). [2025-10-20T20:07:07.299979] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-07. [2025-10-20T20:07:07.398786] [INFO ] [REMOTE ] NC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:07.399818] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-01. [2025-10-20T20:07:07.505522] [WARN ] [MAIN ] NCA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:07.521525] [INFO ] [MAIN ] NCA: Wiping existing primary data before full refresh. [2025-10-20T20:07:07.526673] [INFO ] [LOCAL ] NCA: Starting deletion of primary data. [2025-10-20T20:07:07.527358] [DEBUG] [LOCAL ] NBHC: Primary data saved (23824 records). [2025-10-20T20:07:07.527546] [INFO ] [MAIN ] NBHC: Full history saved. [2025-10-20T20:07:07.527736] [DEBUG] [LOCAL ] NCA: Deleted primary data file. [2025-10-20T20:07:07.527691] [INFO ] [MAIN ] Update Progress: 1453/2367 symbols completed. NBHC took 35.5s. Estimated remaining: 1h 57m 40s. Rate: 7.8 symbols/min [2025-10-20T20:07:07.527866] [INFO ] [MAIN ] NCA: Starting full primary refresh. [2025-10-20T20:07:07.864198] [INFO ] [MAIN ] NCDL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:07.982859] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T20:07:07.999878] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-02. [2025-10-20T20:07:08.065564] [DEBUG] [LOCAL ] NCDL: Loaded 22009 existing records from primary storage. [2025-10-20T20:07:08.067991] [INFO ] [MAIN ] NCDL: Loaded 22009 existing records. [2025-10-20T20:07:08.129516] [INFO ] [REMOTE ] NCA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:08.257573] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-01. [2025-10-20T20:07:08.262047] [WARN ] [MAIN ] NCDL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:08.262250] [INFO ] [MAIN ] NCDL: Wiping existing primary data before full refresh. [2025-10-20T20:07:08.262313] [INFO ] [LOCAL ] NCDL: Starting deletion of primary data. [2025-10-20T20:07:08.263249] [DEBUG] [LOCAL ] NCDL: Deleted primary data file. [2025-10-20T20:07:08.263317] [INFO ] [MAIN ] NCDL: Starting full primary refresh. [2025-10-20T20:07:08.619144] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T20:07:08.622887] [INFO ] [REMOTE ] NCDL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:08.807521] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T20:07:09.051407] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-08. [2025-10-20T20:07:09.091505] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T20:07:09.851421] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-01. [2025-10-20T20:07:10.373581] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T20:07:10.651398] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-02. [2025-10-20T20:07:10.934060] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T20:07:11.430050] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:07:11.451339] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-03. [2025-10-20T20:07:12.205135] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T20:07:12.251371] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-04. [2025-10-20T20:07:13.051443] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-02. [2025-10-20T20:07:13.180756] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:07:13.851384] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-03. [2025-10-20T20:07:14.125181] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T20:07:14.312594] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T20:07:14.651396] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-04. [2025-10-20T20:07:15.285124] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T20:07:15.451404] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-05. [2025-10-20T20:07:16.204037] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:07:16.251389] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-06. [2025-10-20T20:07:17.051410] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-09. [2025-10-20T20:07:17.064498] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T20:07:17.851386] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-03. [2025-10-20T20:07:18.588760] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-10-20T20:07:18.651357] [INFO ] [REMOTE ] NBR: Fetching primary data for 2025-10. [2025-10-20T20:07:18.971954] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T20:07:19.451383] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-05. [2025-10-20T20:07:19.910066] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-05 (0.46 seconds). [2025-10-20T20:07:19.991816] [INFO ] [REMOTE ] NBR: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T20:07:19.996551] [INFO ] [REMOTE ] NBR: Full fetch complete (49071 records). [2025-10-20T20:07:19.996715] [DEBUG] [VALIDATOR ] NBR: Validating primary data. [2025-10-20T20:07:20.256698] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-04. [2025-10-20T20:07:20.404438] [DEBUG] [VALIDATOR ] NBR: All expected trading days are present. [2025-10-20T20:07:20.404536] [DEBUG] [VALIDATOR ] NBR: Primary data validation passed. [2025-10-20T20:07:20.405367] [INFO ] [MAIN ] NBR: Fetched primary data is valid. [2025-10-20T20:07:20.405427] [DEBUG] [LOCAL ] NBR: Saving primary data. [2025-10-20T20:07:20.406403] [DEBUG] [LOCAL ] NBR: Trimmed 49071 records to 34623 (keeping 200 days of data) [2025-10-20T20:07:20.600889] [DEBUG] [LOCAL ] NBR: Primary data saved (34623 records). [2025-10-20T20:07:20.601026] [INFO ] [MAIN ] NBR: Full history saved. [2025-10-20T20:07:20.601100] [INFO ] [MAIN ] Update Progress: 1454/2367 symbols completed. NBR took 36.7s. Estimated remaining: 1h 57m 35s. Rate: 7.8 symbols/min [2025-10-20T20:07:20.769443] [INFO ] [MAIN ] NCLH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:20.769622] [DEBUG] [LOCAL ] NCLH: No existing data file found in primary storage. [2025-10-20T20:07:20.769693] [WARN ] [MAIN ] NCLH: No existing data found, performing full refresh instead. [2025-10-20T20:07:20.769822] [INFO ] [MAIN ] NCLH: Wiping existing primary data before full refresh. [2025-10-20T20:07:20.769862] [INFO ] [LOCAL ] NCLH: Starting deletion of primary data. [2025-10-20T20:07:20.769904] [DEBUG] [LOCAL ] NCLH: No primary data file found to delete. [2025-10-20T20:07:20.769956] [INFO ] [MAIN ] NCLH: Starting full primary refresh. [2025-10-20T20:07:21.058291] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-06. [2025-10-20T20:07:21.166436] [INFO ] [REMOTE ] NCLH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:21.182203] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:07:21.536054] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T20:07:21.851437] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-07. [2025-10-20T20:07:22.608882] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T20:07:22.651379] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-08. [2025-10-20T20:07:23.451434] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-05. [2025-10-20T20:07:23.498964] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T20:07:24.251394] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-09. [2025-10-20T20:07:24.477979] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T20:07:25.051406] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-06. [2025-10-20T20:07:25.062968] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:07:25.851386] [INFO ] [REMOTE ] NCA: Fetching primary data for 2025-10. [2025-10-20T20:07:26.114359] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T20:07:26.550309] [INFO ] [REMOTE ] NCA: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:07:26.551757] [INFO ] [REMOTE ] NCA: Full fetch complete (14287 records). [2025-10-20T20:07:26.551836] [DEBUG] [VALIDATOR ] NCA: Validating primary data. [2025-10-20T20:07:26.657865] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-07. [2025-10-20T20:07:26.941891] [DEBUG] [VALIDATOR ] NCA: All expected trading days are present. [2025-10-20T20:07:26.941966] [DEBUG] [VALIDATOR ] NCA: Primary data validation passed. [2025-10-20T20:07:26.942276] [INFO ] [MAIN ] NCA: Fetched primary data is valid. [2025-10-20T20:07:26.942313] [DEBUG] [LOCAL ] NCA: Saving primary data. [2025-10-20T20:07:26.942799] [DEBUG] [LOCAL ] NCA: Trimmed 14287 records to 9920 (keeping 200 days of data) [2025-10-20T20:07:26.998795] [DEBUG] [LOCAL ] NCA: Primary data saved (9920 records). [2025-10-20T20:07:26.998908] [INFO ] [MAIN ] NCA: Full history saved. [2025-10-20T20:07:26.998982] [INFO ] [MAIN ] Update Progress: 1455/2367 symbols completed. NCA took 20.8s. Estimated remaining: 1h 57m 27s. Rate: 7.8 symbols/min [2025-10-20T20:07:27.168165] [INFO ] [MAIN ] NCV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:27.183771] [DEBUG] [LOCAL ] NCV: Loaded 14136 existing records from primary storage. [2025-10-20T20:07:27.183997] [INFO ] [MAIN ] NCV: Loaded 14136 existing records. [2025-10-20T20:07:27.358065] [WARN ] [MAIN ] NCV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:27.358336] [INFO ] [MAIN ] NCV: Wiping existing primary data before full refresh. [2025-10-20T20:07:27.358401] [INFO ] [LOCAL ] NCV: Starting deletion of primary data. [2025-10-20T20:07:27.358590] [DEBUG] [LOCAL ] NCV: Deleted primary data file. [2025-10-20T20:07:27.358697] [INFO ] [MAIN ] NCV: Starting full primary refresh. [2025-10-20T20:07:27.456442] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-01. [2025-10-20T20:07:27.732441] [INFO ] [REMOTE ] NCV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:27.764164] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T20:07:28.251381] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-01. [2025-10-20T20:07:29.051365] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-08. [2025-10-20T20:07:29.504001] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T20:07:29.549920] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T20:07:29.851367] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-02. [2025-10-20T20:07:30.003097] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T20:07:30.651378] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-09. [2025-10-20T20:07:31.451340] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-07. [2025-10-20T20:07:31.577747] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:07:31.807810] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T20:07:31.841968] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T20:07:32.251377] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-02. [2025-10-20T20:07:33.051433] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-08. [2025-10-20T20:07:33.202458] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T20:07:33.428839] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T20:07:33.851437] [INFO ] [REMOTE ] NCDL: Fetching primary data for 2025-10. [2025-10-20T20:07:34.651432] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-03. [2025-10-20T20:07:35.113240] [INFO ] [REMOTE ] NCDL: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T20:07:35.117455] [INFO ] [REMOTE ] NCDL: Full fetch complete (32913 records). [2025-10-20T20:07:35.117577] [DEBUG] [VALIDATOR ] NCDL: Validating primary data. [2025-10-20T20:07:35.464776] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-03. [2025-10-20T20:07:35.513566] [DEBUG] [VALIDATOR ] NCDL: All expected trading days are present. [2025-10-20T20:07:35.513769] [DEBUG] [VALIDATOR ] NCDL: Primary data validation passed. [2025-10-20T20:07:35.514535] [INFO ] [MAIN ] NCDL: Fetched primary data is valid. [2025-10-20T20:07:35.514632] [DEBUG] [LOCAL ] NCDL: Saving primary data. [2025-10-20T20:07:35.515702] [DEBUG] [LOCAL ] NCDL: Trimmed 32913 records to 21728 (keeping 200 days of data) [2025-10-20T20:07:35.611511] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T20:07:35.658347] [DEBUG] [LOCAL ] NCDL: Primary data saved (21728 records). [2025-10-20T20:07:35.658463] [INFO ] [MAIN ] NCDL: Full history saved. [2025-10-20T20:07:35.658536] [INFO ] [MAIN ] Update Progress: 1456/2367 symbols completed. NCDL took 28.1s. Estimated remaining: 1h 57m 20s. Rate: 7.8 symbols/min [2025-10-20T20:07:35.834105] [INFO ] [MAIN ] NCZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:35.847678] [DEBUG] [LOCAL ] NCZ: Loaded 12580 existing records from primary storage. [2025-10-20T20:07:35.847877] [INFO ] [MAIN ] NCZ: Loaded 12580 existing records. [2025-10-20T20:07:36.034356] [WARN ] [MAIN ] NCZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:36.034580] [INFO ] [MAIN ] NCZ: Wiping existing primary data before full refresh. [2025-10-20T20:07:36.034630] [INFO ] [LOCAL ] NCZ: Starting deletion of primary data. [2025-10-20T20:07:36.034852] [DEBUG] [LOCAL ] NCZ: Deleted primary data file. [2025-10-20T20:07:36.034909] [INFO ] [MAIN ] NCZ: Starting full primary refresh. [2025-10-20T20:07:36.258365] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-04. [2025-10-20T20:07:36.393942] [INFO ] [REMOTE ] NCZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:37.051460] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-01. [2025-10-20T20:07:37.489269] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T20:07:37.803835] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T20:07:37.851337] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-05. [2025-10-20T20:07:38.167200] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T20:07:38.651409] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-04. [2025-10-20T20:07:38.718543] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:07:39.451425] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-02. [2025-10-20T20:07:40.251373] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-06. [2025-10-20T20:07:40.377138] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T20:07:40.638543] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T20:07:40.719828] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T20:07:41.051391] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-07. [2025-10-20T20:07:41.851393] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-09. [2025-10-20T20:07:41.993827] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T20:07:42.182012] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-09 (0.33 seconds). [2025-10-20T20:07:42.651372] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-03. [2025-10-20T20:07:43.451380] [INFO ] [REMOTE ] NC: Fetching primary data for 2025-10. [2025-10-20T20:07:43.468220] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:07:44.040274] [INFO ] [REMOTE ] NC: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T20:07:44.041242] [INFO ] [REMOTE ] NC: Full fetch complete (2759 records). [2025-10-20T20:07:44.041308] [DEBUG] [VALIDATOR ] NC: Validating primary data. [2025-10-20T20:07:44.256529] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-04. [2025-10-20T20:07:44.420131] [DEBUG] [VALIDATOR ] NC: All expected trading days are present. [2025-10-20T20:07:44.420210] [DEBUG] [VALIDATOR ] NC: Primary data validation passed. [2025-10-20T20:07:44.420322] [INFO ] [MAIN ] NC: Fetched primary data is valid. [2025-10-20T20:07:44.420362] [DEBUG] [LOCAL ] NC: Saving primary data. [2025-10-20T20:07:44.420727] [DEBUG] [LOCAL ] NC: Trimmed 2759 records to 2000 (keeping 200 days of data) [2025-10-20T20:07:44.433042] [DEBUG] [LOCAL ] NC: Primary data saved (2000 records). [2025-10-20T20:07:44.433105] [INFO ] [MAIN ] NC: Full history saved. [2025-10-20T20:07:44.433173] [INFO ] [MAIN ] Update Progress: 1457/2367 symbols completed. NC took 39.0s. Estimated remaining: 1h 57m 13s. Rate: 7.8 symbols/min [2025-10-20T20:07:44.602465] [INFO ] [MAIN ] NDMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:44.620959] [DEBUG] [LOCAL ] NDMO: Loaded 17181 existing records from primary storage. [2025-10-20T20:07:44.621205] [INFO ] [MAIN ] NDMO: Loaded 17181 existing records. [2025-10-20T20:07:44.801815] [WARN ] [MAIN ] NDMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:44.802046] [INFO ] [MAIN ] NDMO: Wiping existing primary data before full refresh. [2025-10-20T20:07:44.802093] [INFO ] [LOCAL ] NDMO: Starting deletion of primary data. [2025-10-20T20:07:44.802287] [DEBUG] [LOCAL ] NDMO: Deleted primary data file. [2025-10-20T20:07:44.802344] [INFO ] [MAIN ] NDMO: Starting full primary refresh. [2025-10-20T20:07:45.055057] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-05. [2025-10-20T20:07:45.169316] [INFO ] [REMOTE ] NDMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:45.203414] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T20:07:45.851435] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-08. [2025-10-20T20:07:46.651386] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-05. [2025-10-20T20:07:46.750554] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T20:07:46.975115] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T20:07:47.451399] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-09. [2025-10-20T20:07:47.478440] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T20:07:48.251421] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-06. [2025-10-20T20:07:48.370476] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T20:07:48.994853] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T20:07:49.051351] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-07. [2025-10-20T20:07:49.851357] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-01. [2025-10-20T20:07:50.033384] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T20:07:50.458774] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T20:07:50.651371] [INFO ] [REMOTE ] NCV: Fetching primary data for 2025-10. [2025-10-20T20:07:51.451403] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-08. [2025-10-20T20:07:51.716564] [INFO ] [REMOTE ] NCV: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T20:07:51.724835] [INFO ] [REMOTE ] NCV: Full fetch complete (22281 records). [2025-10-20T20:07:51.725077] [DEBUG] [VALIDATOR ] NCV: Validating primary data. [2025-10-20T20:07:52.105471] [DEBUG] [VALIDATOR ] NCV: All expected trading days are present. [2025-10-20T20:07:52.105602] [DEBUG] [VALIDATOR ] NCV: Primary data validation passed. [2025-10-20T20:07:52.106114] [INFO ] [MAIN ] NCV: Fetched primary data is valid. [2025-10-20T20:07:52.106172] [DEBUG] [LOCAL ] NCV: Saving primary data. [2025-10-20T20:07:52.106811] [DEBUG] [LOCAL ] NCV: Trimmed 22281 records to 13877 (keeping 200 days of data) [2025-10-20T20:07:52.190430] [DEBUG] [LOCAL ] NCV: Primary data saved (13877 records). [2025-10-20T20:07:52.190560] [INFO ] [MAIN ] NCV: Full history saved. [2025-10-20T20:07:52.190639] [INFO ] [MAIN ] Update Progress: 1458/2367 symbols completed. NCV took 25.2s. Estimated remaining: 1h 57m 5s. Rate: 7.8 symbols/min [2025-10-20T20:07:52.256607] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-06. [2025-10-20T20:07:52.310242] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:07:52.369896] [INFO ] [MAIN ] NE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:52.418364] [DEBUG] [LOCAL ] NE: Loaded 53135 existing records from primary storage. [2025-10-20T20:07:52.418735] [INFO ] [MAIN ] NE: Loaded 53135 existing records. [2025-10-20T20:07:52.598871] [WARN ] [MAIN ] NE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:52.599026] [INFO ] [MAIN ] NE: Wiping existing primary data before full refresh. [2025-10-20T20:07:52.599074] [INFO ] [LOCAL ] NE: Starting deletion of primary data. [2025-10-20T20:07:52.599242] [DEBUG] [LOCAL ] NE: Deleted primary data file. [2025-10-20T20:07:52.599281] [INFO ] [MAIN ] NE: Starting full primary refresh. [2025-10-20T20:07:52.944486] [INFO ] [REMOTE ] NE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:53.051327] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-01. [2025-10-20T20:07:53.851376] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-02. [2025-10-20T20:07:54.353020] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T20:07:54.661686] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-09. [2025-10-20T20:07:54.670536] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T20:07:54.721914] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T20:07:55.336618] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T20:07:55.451347] [INFO ] [REMOTE ] NCZ: Fetching primary data for 2025-10. [2025-10-20T20:07:56.221149] [INFO ] [REMOTE ] NCZ: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:07:56.224011] [INFO ] [REMOTE ] NCZ: Full fetch complete (20996 records). [2025-10-20T20:07:56.224080] [DEBUG] [VALIDATOR ] NCZ: Validating primary data. [2025-10-20T20:07:56.256599] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-07. [2025-10-20T20:07:56.582569] [DEBUG] [VALIDATOR ] NCZ: All expected trading days are present. [2025-10-20T20:07:56.582675] [DEBUG] [VALIDATOR ] NCZ: Primary data validation passed. [2025-10-20T20:07:56.583110] [INFO ] [MAIN ] NCZ: Fetched primary data is valid. [2025-10-20T20:07:56.583150] [DEBUG] [LOCAL ] NCZ: Saving primary data. [2025-10-20T20:07:56.583704] [DEBUG] [LOCAL ] NCZ: Trimmed 20996 records to 12277 (keeping 200 days of data) [2025-10-20T20:07:56.655927] [DEBUG] [LOCAL ] NCZ: Primary data saved (12277 records). [2025-10-20T20:07:56.656055] [INFO ] [MAIN ] NCZ: Full history saved. [2025-10-20T20:07:56.656130] [INFO ] [MAIN ] Update Progress: 1459/2367 symbols completed. NCZ took 21.0s. Estimated remaining: 1h 56m 55s. Rate: 7.8 symbols/min [2025-10-20T20:07:56.820283] [INFO ] [MAIN ] NEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:07:56.854263] [DEBUG] [LOCAL ] NEA: Loaded 35712 existing records from primary storage. [2025-10-20T20:07:56.854553] [INFO ] [MAIN ] NEA: Loaded 35712 existing records. [2025-10-20T20:07:57.025668] [WARN ] [MAIN ] NEA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:07:57.025880] [INFO ] [MAIN ] NEA: Wiping existing primary data before full refresh. [2025-10-20T20:07:57.025926] [INFO ] [LOCAL ] NEA: Starting deletion of primary data. [2025-10-20T20:07:57.026106] [DEBUG] [LOCAL ] NEA: Deleted primary data file. [2025-10-20T20:07:57.026143] [INFO ] [MAIN ] NEA: Starting full primary refresh. [2025-10-20T20:07:57.056690] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-03. [2025-10-20T20:07:57.378674] [INFO ] [REMOTE ] NEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:07:57.851405] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-02. [2025-10-20T20:07:57.971499] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T20:07:58.232982] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T20:07:58.651366] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-04. [2025-10-20T20:07:59.451346] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-01. [2025-10-20T20:07:59.536927] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T20:07:59.653064] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T20:08:00.251387] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-08. [2025-10-20T20:08:00.606851] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T20:08:01.051430] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-05. [2025-10-20T20:08:01.851363] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-03. [2025-10-20T20:08:01.959469] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T20:08:02.171476] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T20:08:02.651369] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-09. [2025-10-20T20:08:03.451430] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-06. [2025-10-20T20:08:03.735760] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T20:08:04.204291] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T20:08:04.251396] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-04. [2025-10-20T20:08:04.261038] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T20:08:05.051419] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-07. [2025-10-20T20:08:05.851375] [INFO ] [REMOTE ] NCLH: Fetching primary data for 2025-10. [2025-10-20T20:08:06.054840] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T20:08:06.060710] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T20:08:06.592350] [INFO ] [REMOTE ] NCLH: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T20:08:06.600741] [INFO ] [REMOTE ] NCLH: Full fetch complete (77933 records). [2025-10-20T20:08:06.600891] [DEBUG] [VALIDATOR ] NCLH: Validating primary data. [2025-10-20T20:08:06.656564] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-02. [2025-10-20T20:08:07.005239] [DEBUG] [VALIDATOR ] NCLH: All expected trading days are present. [2025-10-20T20:08:07.005327] [DEBUG] [VALIDATOR ] NCLH: Primary data validation passed. [2025-10-20T20:08:07.006380] [INFO ] [MAIN ] NCLH: Fetched primary data is valid. [2025-10-20T20:08:07.006446] [DEBUG] [LOCAL ] NCLH: Saving primary data. [2025-10-20T20:08:07.007910] [DEBUG] [LOCAL ] NCLH: Trimmed 77933 records to 53753 (keeping 200 days of data) [2025-10-20T20:08:07.321802] [DEBUG] [LOCAL ] NCLH: Primary data saved (53753 records). [2025-10-20T20:08:07.321975] [INFO ] [MAIN ] NCLH: Full history saved. [2025-10-20T20:08:07.322060] [INFO ] [MAIN ] Update Progress: 1460/2367 symbols completed. NCLH took 46.7s. Estimated remaining: 1h 56m 49s. Rate: 7.8 symbols/min [2025-10-20T20:08:07.458629] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-05. [2025-10-20T20:08:07.489793] [INFO ] [MAIN ] NEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:07.537407] [DEBUG] [LOCAL ] NEE: Loaded 54487 existing records from primary storage. [2025-10-20T20:08:07.537734] [INFO ] [MAIN ] NEE: Loaded 54487 existing records. [2025-10-20T20:08:07.717062] [WARN ] [MAIN ] NEE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:07.717314] [INFO ] [MAIN ] NEE: Wiping existing primary data before full refresh. [2025-10-20T20:08:07.717390] [INFO ] [LOCAL ] NEE: Starting deletion of primary data. [2025-10-20T20:08:07.717771] [DEBUG] [LOCAL ] NEE: Deleted primary data file. [2025-10-20T20:08:07.717846] [INFO ] [MAIN ] NEE: Starting full primary refresh. [2025-10-20T20:08:07.860412] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T20:08:08.096939] [INFO ] [REMOTE ] NEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:08.251333] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-01. [2025-10-20T20:08:09.051377] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-08. [2025-10-20T20:08:09.283010] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T20:08:09.851405] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-06. [2025-10-20T20:08:10.038325] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T20:08:10.334868] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T20:08:10.651479] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-09. [2025-10-20T20:08:11.451365] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-02. [2025-10-20T20:08:11.672961] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:08:11.674955] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:08:12.251407] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-07. [2025-10-20T20:08:13.051386] [INFO ] [REMOTE ] NDMO: Fetching primary data for 2025-10. [2025-10-20T20:08:13.593886] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-02 (2.14 seconds). [2025-10-20T20:08:13.851343] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-03. [2025-10-20T20:08:14.057715] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T20:08:14.076242] [INFO ] [REMOTE ] NDMO: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T20:08:14.078905] [INFO ] [REMOTE ] NDMO: Full fetch complete (23376 records). [2025-10-20T20:08:14.078975] [DEBUG] [VALIDATOR ] NDMO: Validating primary data. [2025-10-20T20:08:14.449411] [DEBUG] [VALIDATOR ] NDMO: All expected trading days are present. [2025-10-20T20:08:14.449495] [DEBUG] [VALIDATOR ] NDMO: Primary data validation passed. [2025-10-20T20:08:14.450225] [INFO ] [MAIN ] NDMO: Fetched primary data is valid. [2025-10-20T20:08:14.450286] [DEBUG] [LOCAL ] NDMO: Saving primary data. [2025-10-20T20:08:14.451117] [DEBUG] [LOCAL ] NDMO: Trimmed 23376 records to 16985 (keeping 200 days of data) [2025-10-20T20:08:14.551403] [DEBUG] [LOCAL ] NDMO: Primary data saved (16985 records). [2025-10-20T20:08:14.551534] [INFO ] [MAIN ] NDMO: Full history saved. [2025-10-20T20:08:14.551609] [INFO ] [MAIN ] Update Progress: 1461/2367 symbols completed. NDMO took 30.1s. Estimated remaining: 1h 56m 41s. Rate: 7.8 symbols/min [2025-10-20T20:08:14.654684] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-08. [2025-10-20T20:08:14.726446] [INFO ] [MAIN ] NEM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:14.775411] [DEBUG] [LOCAL ] NEM: Loaded 54488 existing records from primary storage. [2025-10-20T20:08:14.775753] [INFO ] [MAIN ] NEM: Loaded 54488 existing records. [2025-10-20T20:08:14.962150] [WARN ] [MAIN ] NEM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:14.962392] [INFO ] [MAIN ] NEM: Wiping existing primary data before full refresh. [2025-10-20T20:08:14.962440] [INFO ] [LOCAL ] NEM: Starting deletion of primary data. [2025-10-20T20:08:14.962694] [DEBUG] [LOCAL ] NEM: Deleted primary data file. [2025-10-20T20:08:14.962752] [INFO ] [MAIN ] NEM: Starting full primary refresh. [2025-10-20T20:08:15.330570] [INFO ] [REMOTE ] NEM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:15.451348] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-01. [2025-10-20T20:08:15.727038] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T20:08:16.251376] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-04. [2025-10-20T20:08:16.478935] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T20:08:17.051372] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-03. [2025-10-20T20:08:17.462152] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T20:08:17.851358] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-09. [2025-10-20T20:08:18.432170] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T20:08:18.465064] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T20:08:18.651391] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-02. [2025-10-20T20:08:19.451482] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-05. [2025-10-20T20:08:19.706951] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T20:08:20.251417] [INFO ] [REMOTE ] NE: Fetching primary data for 2025-10. [2025-10-20T20:08:20.342238] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T20:08:21.051358] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-04. [2025-10-20T20:08:21.580970] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T20:08:21.851407] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-06. [2025-10-20T20:08:21.874171] [INFO ] [REMOTE ] NE: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T20:08:21.882694] [INFO ] [REMOTE ] NE: Full fetch complete (75749 records). [2025-10-20T20:08:21.882833] [DEBUG] [VALIDATOR ] NE: Validating primary data. [2025-10-20T20:08:22.279106] [DEBUG] [VALIDATOR ] NE: All expected trading days are present. [2025-10-20T20:08:22.279241] [DEBUG] [VALIDATOR ] NE: Primary data validation passed. [2025-10-20T20:08:22.280303] [INFO ] [MAIN ] NE: Fetched primary data is valid. [2025-10-20T20:08:22.280366] [DEBUG] [LOCAL ] NE: Saving primary data. [2025-10-20T20:08:22.281817] [DEBUG] [LOCAL ] NE: Trimmed 75749 records to 52353 (keeping 200 days of data) [2025-10-20T20:08:22.614854] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T20:08:22.634527] [DEBUG] [LOCAL ] NE: Primary data saved (52353 records). [2025-10-20T20:08:22.634631] [INFO ] [MAIN ] NE: Full history saved. [2025-10-20T20:08:22.634736] [INFO ] [MAIN ] Update Progress: 1462/2367 symbols completed. NE took 30.4s. Estimated remaining: 1h 56m 34s. Rate: 7.8 symbols/min [2025-10-20T20:08:22.656547] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-05. [2025-10-20T20:08:22.808433] [INFO ] [MAIN ] NET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:22.859036] [DEBUG] [LOCAL ] NET: Loaded 53652 existing records from primary storage. [2025-10-20T20:08:22.859353] [INFO ] [MAIN ] NET: Loaded 53652 existing records. [2025-10-20T20:08:23.039421] [WARN ] [MAIN ] NET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:23.039662] [INFO ] [MAIN ] NET: Wiping existing primary data before full refresh. [2025-10-20T20:08:23.039722] [INFO ] [LOCAL ] NET: Starting deletion of primary data. [2025-10-20T20:08:23.039947] [DEBUG] [LOCAL ] NET: Deleted primary data file. [2025-10-20T20:08:23.040000] [INFO ] [MAIN ] NET: Starting full primary refresh. [2025-10-20T20:08:23.402454] [INFO ] [REMOTE ] NET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:23.451382] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-01. [2025-10-20T20:08:23.823030] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T20:08:23.887712] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T20:08:24.251387] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-07. [2025-10-20T20:08:25.051439] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-03. [2025-10-20T20:08:25.495101] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-01 (2.04 seconds). [2025-10-20T20:08:25.851406] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-06. [2025-10-20T20:08:26.477515] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T20:08:26.651416] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-08. [2025-10-20T20:08:27.166397] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T20:08:27.233026] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T20:08:27.451365] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-04. [2025-10-20T20:08:28.251349] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-02. [2025-10-20T20:08:28.718597] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T20:08:29.051439] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-09. [2025-10-20T20:08:29.837993] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-04 (2.39 seconds). [2025-10-20T20:08:29.851331] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-05. [2025-10-20T20:08:29.891045] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T20:08:30.651368] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-03. [2025-10-20T20:08:30.912149] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T20:08:31.451396] [INFO ] [REMOTE ] NEE: Fetching primary data for 2025-10. [2025-10-20T20:08:32.109377] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-05 (2.26 seconds). [2025-10-20T20:08:32.251371] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-07. [2025-10-20T20:08:32.404971] [INFO ] [REMOTE ] NEE: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T20:08:32.413188] [INFO ] [REMOTE ] NEE: Full fetch complete (77887 records). [2025-10-20T20:08:32.413371] [DEBUG] [VALIDATOR ] NEE: Validating primary data. [2025-10-20T20:08:32.817818] [DEBUG] [VALIDATOR ] NEE: All expected trading days are present. [2025-10-20T20:08:32.817928] [DEBUG] [VALIDATOR ] NEE: Primary data validation passed. [2025-10-20T20:08:32.819262] [INFO ] [MAIN ] NEE: Fetched primary data is valid. [2025-10-20T20:08:32.819375] [DEBUG] [LOCAL ] NEE: Saving primary data. [2025-10-20T20:08:32.851324] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-10-20T20:08:32.849980] [DEBUG] [LOCAL ] NEE: Trimmed 77887 records to 53707 (keeping 200 days of data) [2025-10-20T20:08:33.076283] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-06. [2025-10-20T20:08:33.201688] [DEBUG] [LOCAL ] NEE: Primary data saved (53707 records). [2025-10-20T20:08:33.201834] [INFO ] [MAIN ] NEE: Full history saved. [2025-10-20T20:08:33.201916] [INFO ] [MAIN ] Update Progress: 1463/2367 symbols completed. NEE took 25.9s. Estimated remaining: 1h 56m 28s. Rate: 7.8 symbols/min [2025-10-20T20:08:33.370381] [INFO ] [MAIN ] NEU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:33.380422] [DEBUG] [LOCAL ] NEU: Loaded 8654 existing records from primary storage. [2025-10-20T20:08:33.380617] [INFO ] [MAIN ] NEU: Loaded 8654 existing records. [2025-10-20T20:08:33.568751] [WARN ] [MAIN ] NEU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:33.569263] [INFO ] [MAIN ] NEU: Wiping existing primary data before full refresh. [2025-10-20T20:08:33.569394] [INFO ] [LOCAL ] NEU: Starting deletion of primary data. [2025-10-20T20:08:33.569718] [DEBUG] [LOCAL ] NEU: Deleted primary data file. [2025-10-20T20:08:33.569783] [INFO ] [MAIN ] NEU: Starting full primary refresh. [2025-10-20T20:08:33.720340] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T20:08:33.856478] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-08. [2025-10-20T20:08:34.029001] [INFO ] [REMOTE ] NEU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:34.651370] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-01. [2025-10-20T20:08:35.270493] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-06 (2.19 seconds). [2025-10-20T20:08:35.320100] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T20:08:35.387354] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T20:08:35.451379] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-02. [2025-10-20T20:08:36.100341] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:08:36.251370] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-03. [2025-10-20T20:08:36.910840] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:08:37.051416] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-04. [2025-10-20T20:08:37.721586] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:08:37.851389] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-05. [2025-10-20T20:08:38.443500] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T20:08:38.651427] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-09. [2025-10-20T20:08:39.451390] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-04. [2025-10-20T20:08:40.048831] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T20:08:40.251359] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-07. [2025-10-20T20:08:41.051416] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-06. [2025-10-20T20:08:41.627754] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T20:08:41.692385] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T20:08:41.851359] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-07. [2025-10-20T20:08:42.460468] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-10-20T20:08:42.651379] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-08. [2025-10-20T20:08:42.706847] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T20:08:43.451483] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-05. [2025-10-20T20:08:44.251386] [INFO ] [REMOTE ] NEA: Fetching primary data for 2025-10. [2025-10-20T20:08:44.808504] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-10-20T20:08:45.051436] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-09. [2025-10-20T20:08:45.214887] [INFO ] [REMOTE ] NEA: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T20:08:45.222390] [INFO ] [REMOTE ] NEA: Full fetch complete (50933 records). [2025-10-20T20:08:45.222860] [DEBUG] [VALIDATOR ] NEA: Validating primary data. [2025-10-20T20:08:45.433163] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T20:08:45.654778] [DEBUG] [VALIDATOR ] NEA: All expected trading days are present. [2025-10-20T20:08:45.654867] [DEBUG] [VALIDATOR ] NEA: Primary data validation passed. [2025-10-20T20:08:45.655717] [INFO ] [MAIN ] NEA: Fetched primary data is valid. [2025-10-20T20:08:45.655779] [DEBUG] [LOCAL ] NEA: Saving primary data. [2025-10-20T20:08:45.656893] [DEBUG] [LOCAL ] NEA: Trimmed 50933 records to 35207 (keeping 200 days of data) [2025-10-20T20:08:45.854308] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-06. [2025-10-20T20:08:45.869311] [DEBUG] [LOCAL ] NEA: Primary data saved (35207 records). [2025-10-20T20:08:45.869707] [INFO ] [MAIN ] NEA: Full history saved. [2025-10-20T20:08:45.870094] [INFO ] [MAIN ] Update Progress: 1464/2367 symbols completed. NEA took 49.2s. Estimated remaining: 1h 56m 23s. Rate: 7.8 symbols/min [2025-10-20T20:08:46.038455] [INFO ] [MAIN ] NEXA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:46.046942] [DEBUG] [LOCAL ] NEXA: Loaded 5430 existing records from primary storage. [2025-10-20T20:08:46.047156] [INFO ] [MAIN ] NEXA: Loaded 5430 existing records. [2025-10-20T20:08:46.222773] [WARN ] [MAIN ] NEXA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:46.223037] [INFO ] [MAIN ] NEXA: Wiping existing primary data before full refresh. [2025-10-20T20:08:46.223083] [INFO ] [LOCAL ] NEXA: Starting deletion of primary data. [2025-10-20T20:08:46.223270] [DEBUG] [LOCAL ] NEXA: Deleted primary data file. [2025-10-20T20:08:46.223325] [INFO ] [MAIN ] NEXA: Starting full primary refresh. [2025-10-20T20:08:46.577416] [INFO ] [REMOTE ] NEXA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:46.651359] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-01. [2025-10-20T20:08:47.230848] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T20:08:47.451415] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-08. [2025-10-20T20:08:47.494818] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-09 (2.44 seconds). [2025-10-20T20:08:47.963861] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T20:08:47.992502] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T20:08:48.251434] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-02. [2025-10-20T20:08:48.840914] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T20:08:49.051396] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-03. [2025-10-20T20:08:49.526093] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T20:08:49.851370] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-04. [2025-10-20T20:08:50.327223] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T20:08:50.651389] [INFO ] [REMOTE ] NEM: Fetching primary data for 2025-10. [2025-10-20T20:08:51.344247] [INFO ] [REMOTE ] NEM: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:08:51.352260] [INFO ] [REMOTE ] NEM: Full fetch complete (77887 records). [2025-10-20T20:08:51.352422] [DEBUG] [VALIDATOR ] NEM: Validating primary data. [2025-10-20T20:08:51.454408] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-09. [2025-10-20T20:08:51.755673] [DEBUG] [VALIDATOR ] NEM: All expected trading days are present. [2025-10-20T20:08:51.756010] [DEBUG] [VALIDATOR ] NEM: Primary data validation passed. [2025-10-20T20:08:51.757105] [INFO ] [MAIN ] NEM: Fetched primary data is valid. [2025-10-20T20:08:51.757167] [DEBUG] [LOCAL ] NEM: Saving primary data. [2025-10-20T20:08:51.758827] [DEBUG] [LOCAL ] NEM: Trimmed 77887 records to 53708 (keeping 200 days of data) [2025-10-20T20:08:52.083365] [DEBUG] [LOCAL ] NEM: Primary data saved (53708 records). [2025-10-20T20:08:52.083487] [INFO ] [MAIN ] NEM: Full history saved. [2025-10-20T20:08:52.083561] [INFO ] [MAIN ] Update Progress: 1465/2367 symbols completed. NEM took 37.5s. Estimated remaining: 1h 56m 14s. Rate: 7.8 symbols/min [2025-10-20T20:08:52.254231] [INFO ] [MAIN ] NFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:52.254473] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-05. [2025-10-20T20:08:52.290539] [DEBUG] [LOCAL ] NFG: Loaded 39051 existing records from primary storage. [2025-10-20T20:08:52.290842] [INFO ] [MAIN ] NFG: Loaded 39051 existing records. [2025-10-20T20:08:52.400351] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T20:08:52.480754] [WARN ] [MAIN ] NFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:52.480995] [INFO ] [MAIN ] NFG: Wiping existing primary data before full refresh. [2025-10-20T20:08:52.481044] [INFO ] [LOCAL ] NFG: Starting deletion of primary data. [2025-10-20T20:08:52.481441] [DEBUG] [LOCAL ] NFG: Deleted primary data file. [2025-10-20T20:08:52.481498] [INFO ] [MAIN ] NFG: Starting full primary refresh. [2025-10-20T20:08:52.836366] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T20:08:52.841337] [INFO ] [REMOTE ] NFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:53.051354] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-06. [2025-10-20T20:08:53.669441] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T20:08:53.851388] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-07. [2025-10-20T20:08:54.651424] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-01. [2025-10-20T20:08:55.451425] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-07. [2025-10-20T20:08:56.043330] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T20:08:56.149588] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-07 (2.30 seconds). [2025-10-20T20:08:56.219034] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T20:08:56.251328] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-08. [2025-10-20T20:08:56.683108] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T20:08:57.051360] [INFO ] [REMOTE ] NEU: Fetching primary data for 2025-10. [2025-10-20T20:08:57.736521] [INFO ] [REMOTE ] NEU: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:08:57.738187] [INFO ] [REMOTE ] NEU: Full fetch complete (10805 records). [2025-10-20T20:08:57.741460] [DEBUG] [VALIDATOR ] NEU: Validating primary data. [2025-10-20T20:08:57.856533] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-08. [2025-10-20T20:08:58.108713] [DEBUG] [VALIDATOR ] NEU: All expected trading days are present. [2025-10-20T20:08:58.108796] [DEBUG] [VALIDATOR ] NEU: Primary data validation passed. [2025-10-20T20:08:58.109068] [INFO ] [MAIN ] NEU: Fetched primary data is valid. [2025-10-20T20:08:58.109107] [DEBUG] [LOCAL ] NEU: Saving primary data. [2025-10-20T20:08:58.109530] [DEBUG] [LOCAL ] NEU: Trimmed 10805 records to 8633 (keeping 200 days of data) [2025-10-20T20:08:58.161704] [DEBUG] [LOCAL ] NEU: Primary data saved (8633 records). [2025-10-20T20:08:58.161821] [INFO ] [MAIN ] NEU: Full history saved. [2025-10-20T20:08:58.161894] [INFO ] [MAIN ] Update Progress: 1466/2367 symbols completed. NEU took 25.0s. Estimated remaining: 1h 56m 6s. Rate: 7.8 symbols/min [2025-10-20T20:08:58.327386] [INFO ] [MAIN ] NFJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:08:58.348164] [DEBUG] [LOCAL ] NFJ: Loaded 20526 existing records from primary storage. [2025-10-20T20:08:58.348409] [INFO ] [MAIN ] NFJ: Loaded 20526 existing records. [2025-10-20T20:08:58.522986] [WARN ] [MAIN ] NFJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:08:58.523222] [INFO ] [MAIN ] NFJ: Wiping existing primary data before full refresh. [2025-10-20T20:08:58.523290] [INFO ] [LOCAL ] NFJ: Starting deletion of primary data. [2025-10-20T20:08:58.523508] [DEBUG] [LOCAL ] NFJ: Deleted primary data file. [2025-10-20T20:08:58.523566] [INFO ] [MAIN ] NFJ: Starting full primary refresh. [2025-10-20T20:08:58.662378] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-09. [2025-10-20T20:08:58.879047] [INFO ] [REMOTE ] NFJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:08:59.426339] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T20:08:59.451331] [INFO ] [REMOTE ] NEXA: Fetching primary data for 2025-10. [2025-10-20T20:09:00.181087] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-10-20T20:09:00.251344] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-09. [2025-10-20T20:09:00.578316] [INFO ] [REMOTE ] NEXA: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T20:09:00.580944] [INFO ] [REMOTE ] NEXA: Full fetch complete (7999 records). [2025-10-20T20:09:00.581015] [DEBUG] [VALIDATOR ] NEXA: Validating primary data. [2025-10-20T20:09:00.935444] [DEBUG] [VALIDATOR ] NEXA: All expected trading days are present. [2025-10-20T20:09:00.935521] [DEBUG] [VALIDATOR ] NEXA: Primary data validation passed. [2025-10-20T20:09:00.935735] [INFO ] [MAIN ] NEXA: Fetched primary data is valid. [2025-10-20T20:09:00.935795] [DEBUG] [LOCAL ] NEXA: Saving primary data. [2025-10-20T20:09:00.936300] [DEBUG] [LOCAL ] NEXA: Trimmed 7999 records to 5494 (keeping 200 days of data) [2025-10-20T20:09:00.967448] [DEBUG] [LOCAL ] NEXA: Primary data saved (5494 records). [2025-10-20T20:09:00.967580] [INFO ] [MAIN ] NEXA: Full history saved. [2025-10-20T20:09:00.967678] [INFO ] [MAIN ] Update Progress: 1467/2367 symbols completed. NEXA took 15.1s. Estimated remaining: 1h 55m 55s. Rate: 7.8 symbols/min [2025-10-20T20:09:01.060361] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-02. [2025-10-20T20:09:01.140123] [INFO ] [MAIN ] NGG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:01.187963] [DEBUG] [LOCAL ] NGG: Loaded 47373 existing records from primary storage. [2025-10-20T20:09:01.188275] [INFO ] [MAIN ] NGG: Loaded 47373 existing records. [2025-10-20T20:09:01.362505] [WARN ] [MAIN ] NGG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:01.362752] [INFO ] [MAIN ] NGG: Wiping existing primary data before full refresh. [2025-10-20T20:09:01.362803] [INFO ] [LOCAL ] NGG: Starting deletion of primary data. [2025-10-20T20:09:01.363047] [DEBUG] [LOCAL ] NGG: Deleted primary data file. [2025-10-20T20:09:01.363085] [INFO ] [MAIN ] NGG: Starting full primary refresh. [2025-10-20T20:09:01.722168] [INFO ] [REMOTE ] NGG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:01.851402] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-01. [2025-10-20T20:09:02.302634] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T20:09:02.544753] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T20:09:02.651394] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-03. [2025-10-20T20:09:03.056795] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T20:09:03.451424] [INFO ] [REMOTE ] NET: Fetching primary data for 2025-10. [2025-10-20T20:09:04.154410] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:09:04.251441] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-01. [2025-10-20T20:09:05.051346] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-04. [2025-10-20T20:09:05.690550] [INFO ] [REMOTE ] NET: Incremental fetch complete for 2025-10 (2.24 seconds). [2025-10-20T20:09:05.699153] [INFO ] [REMOTE ] NET: Full fetch complete (76908 records). [2025-10-20T20:09:05.699441] [DEBUG] [VALIDATOR ] NET: Validating primary data. [2025-10-20T20:09:05.852567] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-02. [2025-10-20T20:09:06.089072] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T20:09:06.159267] [DEBUG] [VALIDATOR ] NET: All expected trading days are present. [2025-10-20T20:09:06.159357] [DEBUG] [VALIDATOR ] NET: Primary data validation passed. [2025-10-20T20:09:06.160478] [INFO ] [MAIN ] NET: Fetched primary data is valid. [2025-10-20T20:09:06.160543] [DEBUG] [LOCAL ] NET: Saving primary data. [2025-10-20T20:09:06.161997] [DEBUG] [LOCAL ] NET: Trimmed 76908 records to 52843 (keeping 200 days of data) [2025-10-20T20:09:06.402420] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T20:09:06.518403] [DEBUG] [LOCAL ] NET: Primary data saved (52843 records). [2025-10-20T20:09:06.518582] [INFO ] [MAIN ] NET: Full history saved. [2025-10-20T20:09:06.518689] [INFO ] [MAIN ] Update Progress: 1468/2367 symbols completed. NET took 43.9s. Estimated remaining: 1h 55m 46s. Rate: 7.8 symbols/min [2025-10-20T20:09:06.657446] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-02. [2025-10-20T20:09:06.692899] [INFO ] [MAIN ] NGL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:06.722944] [DEBUG] [LOCAL ] NGL: Loaded 30303 existing records from primary storage. [2025-10-20T20:09:06.723221] [INFO ] [MAIN ] NGL: Loaded 30303 existing records. [2025-10-20T20:09:06.902082] [WARN ] [MAIN ] NGL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:06.902289] [INFO ] [MAIN ] NGL: Wiping existing primary data before full refresh. [2025-10-20T20:09:06.902336] [INFO ] [LOCAL ] NGL: Starting deletion of primary data. [2025-10-20T20:09:06.902516] [DEBUG] [LOCAL ] NGL: Deleted primary data file. [2025-10-20T20:09:06.902553] [INFO ] [MAIN ] NGL: Starting full primary refresh. [2025-10-20T20:09:07.170321] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T20:09:07.285116] [INFO ] [REMOTE ] NGL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:07.451379] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-01. [2025-10-20T20:09:08.251359] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-05. [2025-10-20T20:09:08.306580] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T20:09:08.518963] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:09:09.051436] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-03. [2025-10-20T20:09:09.774372] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T20:09:09.851371] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-02. [2025-10-20T20:09:10.133086] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T20:09:10.651346] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-06. [2025-10-20T20:09:10.974827] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T20:09:11.451393] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-04. [2025-10-20T20:09:11.989593] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T20:09:12.142149] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T20:09:12.251386] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-07. [2025-10-20T20:09:13.051422] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-03. [2025-10-20T20:09:13.800546] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T20:09:13.851430] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-08. [2025-10-20T20:09:14.651372] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-05. [2025-10-20T20:09:14.759807] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T20:09:15.362451] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:09:15.451334] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-09. [2025-10-20T20:09:15.850967] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T20:09:16.251412] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-03. [2025-10-20T20:09:16.835388] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T20:09:17.051384] [INFO ] [REMOTE ] NFG: Fetching primary data for 2025-10. [2025-10-20T20:09:17.352109] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T20:09:17.851433] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-06. [2025-10-20T20:09:18.651447] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-04. [2025-10-20T20:09:18.747195] [INFO ] [REMOTE ] NFG: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T20:09:18.753531] [INFO ] [REMOTE ] NFG: Full fetch complete (57273 records). [2025-10-20T20:09:18.753721] [DEBUG] [VALIDATOR ] NFG: Validating primary data. [2025-10-20T20:09:18.965066] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T20:09:19.155332] [DEBUG] [VALIDATOR ] NFG: All expected trading days are present. [2025-10-20T20:09:19.155426] [DEBUG] [VALIDATOR ] NFG: Primary data validation passed. [2025-10-20T20:09:19.156372] [INFO ] [MAIN ] NFG: Fetched primary data is valid. [2025-10-20T20:09:19.156434] [DEBUG] [LOCAL ] NFG: Saving primary data. [2025-10-20T20:09:19.157567] [DEBUG] [LOCAL ] NFG: Trimmed 57273 records to 38404 (keeping 200 days of data) [2025-10-20T20:09:19.385742] [DEBUG] [LOCAL ] NFG: Primary data saved (38404 records). [2025-10-20T20:09:19.385873] [INFO ] [MAIN ] NFG: Full history saved. [2025-10-20T20:09:19.385948] [INFO ] [MAIN ] Update Progress: 1469/2367 symbols completed. NFG took 27.3s. Estimated remaining: 1h 55m 41s. Rate: 7.8 symbols/min [2025-10-20T20:09:19.460276] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-04. [2025-10-20T20:09:19.560619] [INFO ] [MAIN ] NGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:19.573244] [DEBUG] [LOCAL ] NGS: Loaded 11294 existing records from primary storage. [2025-10-20T20:09:19.573458] [INFO ] [MAIN ] NGS: Loaded 11294 existing records. [2025-10-20T20:09:19.764051] [WARN ] [MAIN ] NGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:19.764258] [INFO ] [MAIN ] NGS: Wiping existing primary data before full refresh. [2025-10-20T20:09:19.764306] [INFO ] [LOCAL ] NGS: Starting deletion of primary data. [2025-10-20T20:09:19.764461] [DEBUG] [LOCAL ] NGS: Deleted primary data file. [2025-10-20T20:09:19.764499] [INFO ] [MAIN ] NGS: Starting full primary refresh. [2025-10-20T20:09:19.960945] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T20:09:20.154778] [INFO ] [REMOTE ] NGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:20.251365] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-01. [2025-10-20T20:09:21.057317] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-07. [2025-10-20T20:09:21.063271] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T20:09:21.193635] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T20:09:21.851368] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-05. [2025-10-20T20:09:22.215204] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T20:09:22.651425] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-02. [2025-10-20T20:09:23.455619] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-08. [2025-10-20T20:09:23.458423] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T20:09:23.590604] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T20:09:24.251424] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-03. [2025-10-20T20:09:24.527251] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T20:09:25.051409] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-05. [2025-10-20T20:09:25.216280] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T20:09:25.851381] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-06. [2025-10-20T20:09:26.093912] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T20:09:26.651391] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-09. [2025-10-20T20:09:27.456538] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-06. [2025-10-20T20:09:27.470849] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T20:09:27.794774] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T20:09:28.251390] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-07. [2025-10-20T20:09:28.782010] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T20:09:29.051458] [INFO ] [REMOTE ] NFJ: Fetching primary data for 2025-10. [2025-10-20T20:09:29.851399] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-07. [2025-10-20T20:09:29.932011] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T20:09:30.067299] [INFO ] [REMOTE ] NFJ: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T20:09:30.070684] [INFO ] [REMOTE ] NFJ: Full fetch complete (29753 records). [2025-10-20T20:09:30.070776] [DEBUG] [VALIDATOR ] NFJ: Validating primary data. [2025-10-20T20:09:30.434521] [DEBUG] [VALIDATOR ] NFJ: All expected trading days are present. [2025-10-20T20:09:30.434606] [DEBUG] [VALIDATOR ] NFJ: Primary data validation passed. [2025-10-20T20:09:30.435210] [INFO ] [MAIN ] NFJ: Fetched primary data is valid. [2025-10-20T20:09:30.435272] [DEBUG] [LOCAL ] NFJ: Saving primary data. [2025-10-20T20:09:30.435966] [DEBUG] [LOCAL ] NFJ: Trimmed 29753 records to 20008 (keeping 200 days of data) [2025-10-20T20:09:30.552442] [DEBUG] [LOCAL ] NFJ: Primary data saved (20008 records). [2025-10-20T20:09:30.552564] [INFO ] [MAIN ] NFJ: Full history saved. [2025-10-20T20:09:30.552635] [INFO ] [MAIN ] Update Progress: 1470/2367 symbols completed. NFJ took 32.4s. Estimated remaining: 1h 55m 36s. Rate: 7.8 symbols/min [2025-10-20T20:09:30.653407] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-08. [2025-10-20T20:09:30.745123] [INFO ] [MAIN ] NGVC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:30.767258] [DEBUG] [LOCAL ] NGVC: Loaded 23195 existing records from primary storage. [2025-10-20T20:09:30.767509] [INFO ] [MAIN ] NGVC: Loaded 23195 existing records. [2025-10-20T20:09:30.952018] [WARN ] [MAIN ] NGVC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:30.952241] [INFO ] [MAIN ] NGVC: Wiping existing primary data before full refresh. [2025-10-20T20:09:30.952297] [INFO ] [LOCAL ] NGVC: Starting deletion of primary data. [2025-10-20T20:09:30.952515] [DEBUG] [LOCAL ] NGVC: Deleted primary data file. [2025-10-20T20:09:30.952577] [INFO ] [MAIN ] NGVC: Starting full primary refresh. [2025-10-20T20:09:31.324620] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T20:09:31.345326] [INFO ] [REMOTE ] NGVC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:31.451409] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-08. [2025-10-20T20:09:32.251355] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-01. [2025-10-20T20:09:32.299832] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T20:09:32.363806] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T20:09:33.051372] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-09. [2025-10-20T20:09:33.110963] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T20:09:33.851372] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-02. [2025-10-20T20:09:34.346824] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T20:09:34.651373] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-09. [2025-10-20T20:09:34.893189] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T20:09:35.451386] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-03. [2025-10-20T20:09:36.209857] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T20:09:36.251369] [INFO ] [REMOTE ] NGG: Fetching primary data for 2025-10. [2025-10-20T20:09:36.526171] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T20:09:37.051430] [INFO ] [REMOTE ] NGL: Fetching primary data for 2025-10. [2025-10-20T20:09:37.851417] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-04. [2025-10-20T20:09:37.965363] [INFO ] [REMOTE ] NGG: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T20:09:37.972625] [INFO ] [REMOTE ] NGG: Full fetch complete (66644 records). [2025-10-20T20:09:37.972809] [DEBUG] [VALIDATOR ] NGG: Validating primary data. [2025-10-20T20:09:38.370053] [DEBUG] [VALIDATOR ] NGG: All expected trading days are present. [2025-10-20T20:09:38.370166] [DEBUG] [VALIDATOR ] NGG: Primary data validation passed. [2025-10-20T20:09:38.371208] [INFO ] [MAIN ] NGG: Fetched primary data is valid. [2025-10-20T20:09:38.371327] [DEBUG] [LOCAL ] NGG: Saving primary data. [2025-10-20T20:09:38.381444] [INFO ] [REMOTE ] NGL: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T20:09:38.385694] [INFO ] [REMOTE ] NGL: Full fetch complete (41560 records). [2025-10-20T20:09:38.385923] [DEBUG] [VALIDATOR ] NGL: Validating primary data. [2025-10-20T20:09:38.386128] [DEBUG] [LOCAL ] NGG: Trimmed 66644 records to 46845 (keeping 200 days of data) [2025-10-20T20:09:38.663810] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-04. [2025-10-20T20:09:38.705134] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T20:09:39.065142] [DEBUG] [VALIDATOR ] NGL: All expected trading days are present. [2025-10-20T20:09:39.081057] [DEBUG] [VALIDATOR ] NGL: Primary data validation passed. [2025-10-20T20:09:39.085133] [INFO ] [MAIN ] NGL: Fetched primary data is valid. [2025-10-20T20:09:39.093966] [DEBUG] [LOCAL ] NGL: Saving primary data. [2025-10-20T20:09:39.105866] [DEBUG] [LOCAL ] NGG: Primary data saved (46845 records). [2025-10-20T20:09:39.106131] [INFO ] [MAIN ] NGG: Full history saved. [2025-10-20T20:09:39.106231] [DEBUG] [LOCAL ] NGL: Trimmed 41560 records to 29965 (keeping 200 days of data) [2025-10-20T20:09:39.106875] [INFO ] [MAIN ] Update Progress: 1471/2367 symbols completed. NGG took 38.1s. Estimated remaining: 1h 55m 28s. Rate: 7.8 symbols/min [2025-10-20T20:09:39.371599] [INFO ] [MAIN ] NGVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:39.449137] [DEBUG] [LOCAL ] NGL: Primary data saved (29965 records). [2025-10-20T20:09:39.449306] [INFO ] [MAIN ] NGL: Full history saved. [2025-10-20T20:09:39.449402] [INFO ] [MAIN ] Update Progress: 1472/2367 symbols completed. NGL took 32.9s. Estimated remaining: 1h 55m 16s. Rate: 7.8 symbols/min [2025-10-20T20:09:39.476157] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-05. [2025-10-20T20:09:39.566919] [DEBUG] [LOCAL ] NGVT: Loaded 24418 existing records from primary storage. [2025-10-20T20:09:39.570157] [INFO ] [MAIN ] NGVT: Loaded 24418 existing records. [2025-10-20T20:09:39.742487] [INFO ] [MAIN ] NHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:39.824240] [WARN ] [MAIN ] NGVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:39.829869] [INFO ] [MAIN ] NGVT: Wiping existing primary data before full refresh. [2025-10-20T20:09:39.829982] [INFO ] [LOCAL ] NGVT: Starting deletion of primary data. [2025-10-20T20:09:39.830187] [DEBUG] [LOCAL ] NGVT: Deleted primary data file. [2025-10-20T20:09:39.830230] [INFO ] [MAIN ] NGVT: Starting full primary refresh. [2025-10-20T20:09:39.958001] [DEBUG] [LOCAL ] NHI: Loaded 22366 existing records from primary storage. [2025-10-20T20:09:40.044262] [INFO ] [MAIN ] NHI: Loaded 22366 existing records. [2025-10-20T20:09:40.077731] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T20:09:40.252754] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-05. [2025-10-20T20:09:40.411071] [WARN ] [MAIN ] NHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:40.406751] [INFO ] [REMOTE ] NGVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:40.411842] [INFO ] [MAIN ] NHI: Wiping existing primary data before full refresh. [2025-10-20T20:09:40.411927] [INFO ] [LOCAL ] NHI: Starting deletion of primary data. [2025-10-20T20:09:40.412235] [DEBUG] [LOCAL ] NHI: Deleted primary data file. [2025-10-20T20:09:40.419619] [INFO ] [MAIN ] NHI: Starting full primary refresh. [2025-10-20T20:09:40.452726] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-05 (0.98 seconds). [2025-10-20T20:09:40.775256] [INFO ] [REMOTE ] NHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:41.051387] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-01. [2025-10-20T20:09:41.437680] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T20:09:41.851390] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-06. [2025-10-20T20:09:42.189303] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T20:09:42.651414] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-01. [2025-10-20T20:09:42.969007] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T20:09:43.451413] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-07. [2025-10-20T20:09:43.721979] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:09:44.251338] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-02. [2025-10-20T20:09:44.610190] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T20:09:44.993958] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T20:09:45.051381] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-03. [2025-10-20T20:09:45.851385] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-08. [2025-10-20T20:09:45.944778] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T20:09:46.651360] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-04. [2025-10-20T20:09:47.009831] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:09:47.451384] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-02. [2025-10-20T20:09:47.933301] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T20:09:48.251406] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-06. [2025-10-20T20:09:48.723628] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T20:09:49.054347] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-05. [2025-10-20T20:09:49.057693] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T20:09:49.851365] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-09. [2025-10-20T20:09:50.147382] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T20:09:50.651371] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-06. [2025-10-20T20:09:50.994010] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T20:09:51.451391] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-07. [2025-10-20T20:09:51.661443] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T20:09:51.970758] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T20:09:52.251412] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-08. [2025-10-20T20:09:53.051357] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-03. [2025-10-20T20:09:53.122452] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:09:53.851449] [INFO ] [REMOTE ] NGVC: Fetching primary data for 2025-10. [2025-10-20T20:09:54.031437] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T20:09:54.631422] [INFO ] [REMOTE ] NGVC: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:09:54.635061] [INFO ] [REMOTE ] NGVC: Full fetch complete (31076 records). [2025-10-20T20:09:54.635167] [DEBUG] [VALIDATOR ] NGVC: Validating primary data. [2025-10-20T20:09:54.656455] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-07. [2025-10-20T20:09:55.011410] [DEBUG] [VALIDATOR ] NGVC: All expected trading days are present. [2025-10-20T20:09:55.011492] [DEBUG] [VALIDATOR ] NGVC: Primary data validation passed. [2025-10-20T20:09:55.012120] [INFO ] [MAIN ] NGVC: Fetched primary data is valid. [2025-10-20T20:09:55.012182] [DEBUG] [LOCAL ] NGVC: Saving primary data. [2025-10-20T20:09:55.013067] [DEBUG] [LOCAL ] NGVC: Trimmed 31076 records to 22829 (keeping 200 days of data) [2025-10-20T20:09:55.152168] [DEBUG] [LOCAL ] NGVC: Primary data saved (22829 records). [2025-10-20T20:09:55.152349] [INFO ] [MAIN ] NGVC: Full history saved. [2025-10-20T20:09:55.152447] [INFO ] [MAIN ] Update Progress: 1473/2367 symbols completed. NGVC took 24.6s. Estimated remaining: 1h 55m 13s. Rate: 7.8 symbols/min [2025-10-20T20:09:55.323167] [INFO ] [MAIN ] NI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:55.374297] [DEBUG] [LOCAL ] NI: Loaded 54184 existing records from primary storage. [2025-10-20T20:09:55.374685] [INFO ] [MAIN ] NI: Loaded 54184 existing records. [2025-10-20T20:09:55.454686] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-09. [2025-10-20T20:09:55.567333] [WARN ] [MAIN ] NI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:55.567541] [INFO ] [MAIN ] NI: Wiping existing primary data before full refresh. [2025-10-20T20:09:55.567588] [INFO ] [LOCAL ] NI: Starting deletion of primary data. [2025-10-20T20:09:55.567807] [DEBUG] [LOCAL ] NI: Deleted primary data file. [2025-10-20T20:09:55.567864] [INFO ] [MAIN ] NI: Starting full primary refresh. [2025-10-20T20:09:55.989579] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T20:09:55.998177] [INFO ] [REMOTE ] NI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:56.218133] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T20:09:56.251328] [INFO ] [REMOTE ] NGS: Fetching primary data for 2025-10. [2025-10-20T20:09:57.051386] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-08. [2025-10-20T20:09:57.087075] [INFO ] [REMOTE ] NGS: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:09:57.088927] [INFO ] [REMOTE ] NGS: Full fetch complete (16896 records). [2025-10-20T20:09:57.088996] [DEBUG] [VALIDATOR ] NGS: Validating primary data. [2025-10-20T20:09:57.460514] [DEBUG] [VALIDATOR ] NGS: All expected trading days are present. [2025-10-20T20:09:57.460619] [DEBUG] [VALIDATOR ] NGS: Primary data validation passed. [2025-10-20T20:09:57.461068] [INFO ] [MAIN ] NGS: Fetched primary data is valid. [2025-10-20T20:09:57.461118] [DEBUG] [LOCAL ] NGS: Saving primary data. [2025-10-20T20:09:57.461692] [DEBUG] [LOCAL ] NGS: Trimmed 16896 records to 11109 (keeping 200 days of data) [2025-10-20T20:09:57.531704] [DEBUG] [LOCAL ] NGS: Primary data saved (11109 records). [2025-10-20T20:09:57.531853] [INFO ] [MAIN ] NGS: Full history saved. [2025-10-20T20:09:57.531938] [INFO ] [MAIN ] Update Progress: 1474/2367 symbols completed. NGS took 38.1s. Estimated remaining: 1h 55m 2s. Rate: 7.8 symbols/min [2025-10-20T20:09:57.712009] [INFO ] [MAIN ] NIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:09:57.723714] [DEBUG] [LOCAL ] NIC: Loaded 9031 existing records from primary storage. [2025-10-20T20:09:57.723957] [INFO ] [MAIN ] NIC: Loaded 9031 existing records. [2025-10-20T20:09:57.856550] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-01. [2025-10-20T20:09:57.924339] [WARN ] [MAIN ] NIC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:09:57.924810] [INFO ] [MAIN ] NIC: Wiping existing primary data before full refresh. [2025-10-20T20:09:57.924918] [INFO ] [LOCAL ] NIC: Starting deletion of primary data. [2025-10-20T20:09:57.925185] [DEBUG] [LOCAL ] NIC: Deleted primary data file. [2025-10-20T20:09:57.925241] [INFO ] [MAIN ] NIC: Starting full primary refresh. [2025-10-20T20:09:58.395337] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T20:09:58.512975] [INFO ] [REMOTE ] NIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:09:58.651437] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-01. [2025-10-20T20:09:59.420687] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T20:09:59.425172] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T20:09:59.451365] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-02. [2025-10-20T20:10:00.251367] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-04. [2025-10-20T20:10:00.320786] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T20:10:01.051442] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-09. [2025-10-20T20:10:01.419110] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T20:10:01.851374] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-05. [2025-10-20T20:10:02.216595] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T20:10:02.651401] [INFO ] [REMOTE ] NGVT: Fetching primary data for 2025-10. [2025-10-20T20:10:03.009357] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:10:03.451435] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-03. [2025-10-20T20:10:03.756136] [INFO ] [REMOTE ] NGVT: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T20:10:03.760420] [INFO ] [REMOTE ] NGVT: Full fetch complete (34437 records). [2025-10-20T20:10:03.760524] [DEBUG] [VALIDATOR ] NGVT: Validating primary data. [2025-10-20T20:10:04.188417] [DEBUG] [VALIDATOR ] NGVT: All expected trading days are present. [2025-10-20T20:10:04.188497] [DEBUG] [VALIDATOR ] NGVT: Primary data validation passed. [2025-10-20T20:10:04.189144] [INFO ] [MAIN ] NGVT: Fetched primary data is valid. [2025-10-20T20:10:04.189207] [DEBUG] [LOCAL ] NGVT: Saving primary data. [2025-10-20T20:10:04.189944] [DEBUG] [LOCAL ] NGVT: Trimmed 34437 records to 24042 (keeping 200 days of data) [2025-10-20T20:10:04.271091] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-06. [2025-10-20T20:10:04.285806] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T20:10:04.337004] [DEBUG] [LOCAL ] NGVT: Primary data saved (24042 records). [2025-10-20T20:10:04.337125] [INFO ] [MAIN ] NGVT: Full history saved. [2025-10-20T20:10:04.337202] [INFO ] [MAIN ] Update Progress: 1475/2367 symbols completed. NGVT took 25.2s. Estimated remaining: 1h 54m 54s. Rate: 7.8 symbols/min [2025-10-20T20:10:04.502108] [INFO ] [MAIN ] NIE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:04.514411] [DEBUG] [LOCAL ] NIE: Loaded 12094 existing records from primary storage. [2025-10-20T20:10:04.514588] [INFO ] [MAIN ] NIE: Loaded 12094 existing records. [2025-10-20T20:10:04.686847] [WARN ] [MAIN ] NIE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:10:04.687058] [INFO ] [MAIN ] NIE: Wiping existing primary data before full refresh. [2025-10-20T20:10:04.687104] [INFO ] [LOCAL ] NIE: Starting deletion of primary data. [2025-10-20T20:10:04.687269] [DEBUG] [LOCAL ] NIE: Deleted primary data file. [2025-10-20T20:10:04.687325] [INFO ] [MAIN ] NIE: Starting full primary refresh. [2025-10-20T20:10:05.035452] [INFO ] [REMOTE ] NIE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:05.051335] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-01. [2025-10-20T20:10:05.526021] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T20:10:05.851465] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-07. [2025-10-20T20:10:05.944421] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:10:06.651359] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-02. [2025-10-20T20:10:06.690414] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:10:07.451392] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-04. [2025-10-20T20:10:07.468929] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:10:08.251448] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-08. [2025-10-20T20:10:08.342541] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T20:10:09.051378] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-05. [2025-10-20T20:10:09.332479] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T20:10:09.795383] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T20:10:09.851444] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-03. [2025-10-20T20:10:10.651370] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-02. [2025-10-20T20:10:10.719355] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:10:11.451448] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-09. [2025-10-20T20:10:12.251378] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-06. [2025-10-20T20:10:12.515028] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T20:10:12.534473] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T20:10:12.991393] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T20:10:13.051339] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-07. [2025-10-20T20:10:13.851457] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-03. [2025-10-20T20:10:13.892519] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:10:14.651426] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-04. [2025-10-20T20:10:15.451357] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-08. [2025-10-20T20:10:15.534907] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T20:10:15.784267] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T20:10:16.159775] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T20:10:16.251416] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-09. [2025-10-20T20:10:16.960242] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T20:10:17.051413] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-05. [2025-10-20T20:10:17.851396] [INFO ] [REMOTE ] NIC: Fetching primary data for 2025-10. [2025-10-20T20:10:17.962111] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T20:10:18.651476] [INFO ] [REMOTE ] NHI: Fetching primary data for 2025-10. [2025-10-20T20:10:19.451415] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-04. [2025-10-20T20:10:19.809446] [INFO ] [REMOTE ] NHI: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T20:10:19.812863] [INFO ] [REMOTE ] NHI: Full fetch complete (31346 records). [2025-10-20T20:10:19.812945] [DEBUG] [VALIDATOR ] NHI: Validating primary data. [2025-10-20T20:10:20.186722] [DEBUG] [VALIDATOR ] NHI: All expected trading days are present. [2025-10-20T20:10:20.186807] [DEBUG] [VALIDATOR ] NHI: Primary data validation passed. [2025-10-20T20:10:20.187497] [INFO ] [MAIN ] NHI: Fetched primary data is valid. [2025-10-20T20:10:20.187562] [DEBUG] [LOCAL ] NHI: Saving primary data. [2025-10-20T20:10:20.188456] [DEBUG] [LOCAL ] NHI: Trimmed 31346 records to 22020 (keeping 200 days of data) [2025-10-20T20:10:20.251461] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-06. [2025-10-20T20:10:20.327312] [DEBUG] [LOCAL ] NHI: Primary data saved (22020 records). [2025-10-20T20:10:20.327460] [INFO ] [MAIN ] NHI: Full history saved. [2025-10-20T20:10:20.327551] [INFO ] [MAIN ] Update Progress: 1476/2367 symbols completed. NHI took 40.9s. Estimated remaining: 1h 54m 51s. Rate: 7.8 symbols/min [2025-10-20T20:10:20.492482] [INFO ] [MAIN ] NIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:20.498172] [DEBUG] [LOCAL ] NIM: Loaded 3590 existing records from primary storage. [2025-10-20T20:10:20.498364] [INFO ] [MAIN ] NIM: Loaded 3590 existing records. [2025-10-20T20:10:20.674566] [INFO ] [REMOTE ] NIC: Incremental fetch complete for 2025-10 (2.82 seconds). [2025-10-20T20:10:20.679516] [WARN ] [MAIN ] NIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:10:20.681183] [INFO ] [MAIN ] NIM: Wiping existing primary data before full refresh. [2025-10-20T20:10:20.681604] [INFO ] [LOCAL ] NIM: Starting deletion of primary data. [2025-10-20T20:10:20.681738] [INFO ] [REMOTE ] NIC: Full fetch complete (12525 records). [2025-10-20T20:10:20.681849] [DEBUG] [VALIDATOR ] NIC: Validating primary data. [2025-10-20T20:10:20.681966] [DEBUG] [LOCAL ] NIM: Deleted primary data file. [2025-10-20T20:10:20.682073] [INFO ] [MAIN ] NIM: Starting full primary refresh. [2025-10-20T20:10:21.264062] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T20:10:21.311623] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-07. [2025-10-20T20:10:21.407622] [DEBUG] [VALIDATOR ] NIC: All expected trading days are present. [2025-10-20T20:10:21.440411] [DEBUG] [VALIDATOR ] NIC: Primary data validation passed. [2025-10-20T20:10:21.440300] [INFO ] [REMOTE ] NIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:21.440877] [INFO ] [MAIN ] NIC: Fetched primary data is valid. [2025-10-20T20:10:21.441485] [DEBUG] [LOCAL ] NIC: Saving primary data. [2025-10-20T20:10:21.442354] [DEBUG] [LOCAL ] NIC: Trimmed 12525 records to 8898 (keeping 200 days of data) [2025-10-20T20:10:21.530042] [DEBUG] [LOCAL ] NIC: Primary data saved (8898 records). [2025-10-20T20:10:21.533151] [INFO ] [MAIN ] NIC: Full history saved. [2025-10-20T20:10:21.533257] [INFO ] [MAIN ] Update Progress: 1477/2367 symbols completed. NIC took 24.0s. Estimated remaining: 1h 54m 40s. Rate: 7.8 symbols/min [2025-10-20T20:10:21.535383] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T20:10:21.851381] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-05. [2025-10-20T20:10:22.443159] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:10:22.651433] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-08. [2025-10-20T20:10:23.265933] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-08 (0.61 seconds). [2025-10-20T20:10:23.447363] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T20:10:23.457425] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-01. [2025-10-20T20:10:24.108273] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T20:10:24.251432] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-06. [2025-10-20T20:10:25.051356] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-02. [2025-10-20T20:10:25.722966] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T20:10:25.851396] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-09. [2025-10-20T20:10:26.038122] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T20:10:26.616675] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T20:10:26.651329] [INFO ] [REMOTE ] NIE: Fetching primary data for 2025-10. [2025-10-20T20:10:27.451359] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-03. [2025-10-20T20:10:27.603837] [INFO ] [REMOTE ] NIE: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T20:10:27.606283] [INFO ] [REMOTE ] NIE: Full fetch complete (16819 records). [2025-10-20T20:10:27.606359] [DEBUG] [VALIDATOR ] NIE: Validating primary data. [2025-10-20T20:10:27.993119] [DEBUG] [VALIDATOR ] NIE: All expected trading days are present. [2025-10-20T20:10:27.993197] [DEBUG] [VALIDATOR ] NIE: Primary data validation passed. [2025-10-20T20:10:27.993549] [INFO ] [MAIN ] NIE: Fetched primary data is valid. [2025-10-20T20:10:27.993587] [DEBUG] [LOCAL ] NIE: Saving primary data. [2025-10-20T20:10:27.994299] [DEBUG] [LOCAL ] NIE: Trimmed 16819 records to 11838 (keeping 200 days of data) [2025-10-20T20:10:28.065423] [DEBUG] [LOCAL ] NIE: Primary data saved (11838 records). [2025-10-20T20:10:28.065546] [INFO ] [MAIN ] NIE: Full history saved. [2025-10-20T20:10:28.065621] [INFO ] [MAIN ] Update Progress: 1478/2367 symbols completed. NIE took 23.7s. Estimated remaining: 1h 54m 31s. Rate: 7.8 symbols/min [2025-10-20T20:10:28.150226] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T20:10:28.251372] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-07. [2025-10-20T20:10:29.051374] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-04. [2025-10-20T20:10:29.652116] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T20:10:29.851388] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-05. [2025-10-20T20:10:30.058245] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T20:10:30.424758] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T20:10:30.651433] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-08. [2025-10-20T20:10:31.451510] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-06. [2025-10-20T20:10:31.997047] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T20:10:32.251414] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-07. [2025-10-20T20:10:32.488845] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T20:10:32.581598] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-07 (0.33 seconds). [2025-10-20T20:10:33.051386] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-08. [2025-10-20T20:10:33.528145] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T20:10:33.851397] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-09. [2025-10-20T20:10:34.651388] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-09. [2025-10-20T20:10:35.136501] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T20:10:35.451349] [INFO ] [REMOTE ] NIM: Fetching primary data for 2025-10. [2025-10-20T20:10:35.652292] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T20:10:36.176546] [INFO ] [REMOTE ] NIM: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T20:10:36.177512] [INFO ] [REMOTE ] NIM: Full fetch complete (6204 records). [2025-10-20T20:10:36.177578] [DEBUG] [VALIDATOR ] NIM: Validating primary data. [2025-10-20T20:10:36.256463] [INFO ] [REMOTE ] NI: Fetching primary data for 2025-10. [2025-10-20T20:10:36.534708] [DEBUG] [VALIDATOR ] NIM: All expected trading days are present. [2025-10-20T20:10:36.534787] [DEBUG] [VALIDATOR ] NIM: Primary data validation passed. [2025-10-20T20:10:36.534945] [INFO ] [MAIN ] NIM: Fetched primary data is valid. [2025-10-20T20:10:36.534981] [DEBUG] [LOCAL ] NIM: Saving primary data. [2025-10-20T20:10:36.535331] [DEBUG] [LOCAL ] NIM: Trimmed 6204 records to 3520 (keeping 200 days of data) [2025-10-20T20:10:36.555895] [DEBUG] [LOCAL ] NIM: Primary data saved (3520 records). [2025-10-20T20:10:36.555978] [INFO ] [MAIN ] NIM: Full history saved. [2025-10-20T20:10:36.556046] [INFO ] [MAIN ] Update Progress: 1479/2367 symbols completed. NIM took 16.2s. Estimated remaining: 1h 54m 24s. Rate: 7.8 symbols/min [2025-10-20T20:10:37.151060] [INFO ] [REMOTE ] NI: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T20:10:37.158821] [INFO ] [REMOTE ] NI: Full fetch complete (77509 records). [2025-10-20T20:10:37.158959] [DEBUG] [VALIDATOR ] NI: Validating primary data. [2025-10-20T20:10:37.541893] [DEBUG] [VALIDATOR ] NI: All expected trading days are present. [2025-10-20T20:10:37.541971] [DEBUG] [VALIDATOR ] NI: Primary data validation passed. [2025-10-20T20:10:37.542988] [INFO ] [MAIN ] NI: Fetched primary data is valid. [2025-10-20T20:10:37.543050] [DEBUG] [LOCAL ] NI: Saving primary data. [2025-10-20T20:10:37.544460] [DEBUG] [LOCAL ] NI: Trimmed 77509 records to 53410 (keeping 200 days of data) [2025-10-20T20:10:37.856425] [DEBUG] [LOCAL ] NI: Primary data saved (53410 records). [2025-10-20T20:10:37.856547] [INFO ] [MAIN ] NI: Full history saved. [2025-10-20T20:10:37.856620] [INFO ] [MAIN ] Update Progress: 1480/2367 symbols completed. NI took 42.7s. Estimated remaining: 1h 54m 12s. Rate: 7.8 symbols/min [2025-10-20T20:10:38.484543] [INFO ] [MAIN ] NIQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:38.512124] [INFO ] [MAIN ] NJR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:38.511294] [INFO ] [MAIN ] NINE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:38.544919] [DEBUG] [LOCAL ] NIQ: No existing data file found in primary storage. [2025-10-20T20:10:38.564012] [WARN ] [MAIN ] NIQ: No existing data found, performing full refresh instead. [2025-10-20T20:10:38.563939] [INFO ] [MAIN ] NIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:38.566735] [INFO ] [MAIN ] NIQ: Wiping existing primary data before full refresh. [2025-10-20T20:10:38.566913] [INFO ] [LOCAL ] NIQ: Starting deletion of primary data. [2025-10-20T20:10:38.566978] [DEBUG] [LOCAL ] NIQ: No primary data file found to delete. [2025-10-20T20:10:38.567094] [INFO ] [MAIN ] NIQ: Starting full primary refresh. [2025-10-20T20:10:38.734001] [DEBUG] [LOCAL ] NINE: Loaded 28875 existing records from primary storage. [2025-10-20T20:10:38.774414] [INFO ] [MAIN ] NINE: Loaded 28875 existing records. [2025-10-20T20:10:38.877930] [DEBUG] [LOCAL ] NJR: Loaded 38726 existing records from primary storage. [2025-10-20T20:10:38.958274] [INFO ] [MAIN ] NJR: Loaded 38726 existing records. [2025-10-20T20:10:39.043593] [DEBUG] [LOCAL ] NIO: Loaded 54582 existing records from primary storage. [2025-10-20T20:10:39.098321] [INFO ] [MAIN ] NIO: Loaded 54582 existing records. [2025-10-20T20:10:39.391094] [INFO ] [REMOTE ] NIQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:39.388288] [WARN ] [MAIN ] NINE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:10:39.406973] [INFO ] [REMOTE ] NIQ: Fetching primary data for 2025-01. [2025-10-20T20:10:39.407126] [INFO ] [MAIN ] NINE: Wiping existing primary data before full refresh. [2025-10-20T20:10:39.447116] [INFO ] [LOCAL ] NINE: Starting deletion of primary data. [2025-10-20T20:10:39.486101] [WARN ] [MAIN ] NJR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:10:39.492963] [INFO ] [MAIN ] NJR: Wiping existing primary data before full refresh. [2025-10-20T20:10:39.486195] [DEBUG] [LOCAL ] NINE: Deleted primary data file. [2025-10-20T20:10:39.504434] [INFO ] [LOCAL ] NJR: Starting deletion of primary data. [2025-10-20T20:10:39.504605] [INFO ] [MAIN ] NINE: Starting full primary refresh. [2025-10-20T20:10:39.530364] [DEBUG] [LOCAL ] NJR: Deleted primary data file. [2025-10-20T20:10:39.567738] [INFO ] [MAIN ] NJR: Starting full primary refresh. [2025-10-20T20:10:39.708355] [WARN ] [MAIN ] NIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:10:39.736586] [INFO ] [MAIN ] NIO: Wiping existing primary data before full refresh. [2025-10-20T20:10:39.666681] [DEBUG] [REMOTE ] NIQ: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:10:39.741898] [INFO ] [LOCAL ] NIO: Starting deletion of primary data. [2025-10-20T20:10:39.803621] [DEBUG] [LOCAL ] NIO: Deleted primary data file. [2025-10-20T20:10:39.803574] [WARN ] [REMOTE ] NIQ: No data returned for month 2025-01 [2025-10-20T20:10:39.818194] [INFO ] [MAIN ] NIO: Starting full primary refresh. [2025-10-20T20:10:39.828783] [FATAL] [REMOTE ] NIQ: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:10:39.850449] [FATAL] [MAIN ] NIQ: Failed to fetch primary data. [2025-10-20T20:10:40.026007] [WARN ] [MAIN ] NIQ: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:10:40.054626] [INFO ] [LOCAL ] NIQ: Starting deletion of primary data. [2025-10-20T20:10:40.119353] [DEBUG] [LOCAL ] NIQ: No primary data file found to delete. [2025-10-20T20:10:40.132071] [INFO ] [LOCAL ] NIQ: Starting deletion of derivative data. [2025-10-20T20:10:40.172598] [DEBUG] [LOCAL ] NIQ: No derivative data file found to delete. [2025-10-20T20:10:40.252027] [INFO ] [MAIN ] Update Progress: 1481/2367 symbols completed. NIQ took 2.4s. Estimated remaining: 1h 54m 1s. Rate: 7.8 symbols/min [2025-10-20T20:10:40.706975] [INFO ] [REMOTE ] NINE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:40.727318] [INFO ] [REMOTE ] NJR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:40.757108] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-01. [2025-10-20T20:10:40.742919] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-01. [2025-10-20T20:10:40.863986] [INFO ] [MAIN ] NKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:10:40.869937] [DEBUG] [LOCAL ] NKE: No existing data file found in primary storage. [2025-10-20T20:10:40.909689] [WARN ] [MAIN ] NKE: No existing data found, performing full refresh instead. [2025-10-20T20:10:40.924000] [INFO ] [MAIN ] NKE: Wiping existing primary data before full refresh. [2025-10-20T20:10:40.926320] [INFO ] [LOCAL ] NKE: Starting deletion of primary data. [2025-10-20T20:10:40.940788] [DEBUG] [LOCAL ] NKE: No primary data file found to delete. [2025-10-20T20:10:40.942468] [INFO ] [MAIN ] NKE: Starting full primary refresh. [2025-10-20T20:10:40.945986] [INFO ] [REMOTE ] NIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:40.988843] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-01. [2025-10-20T20:10:41.301508] [INFO ] [REMOTE ] NKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:10:41.301854] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-01. [2025-10-20T20:10:42.227512] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T20:10:42.235552] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-02. [2025-10-20T20:10:42.562984] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T20:10:42.571358] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-02. [2025-10-20T20:10:43.495820] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-01 (2.51 seconds). [2025-10-20T20:10:43.506853] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-02. [2025-10-20T20:10:43.684513] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T20:10:43.691732] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-03. [2025-10-20T20:10:43.784392] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-01 (2.48 seconds). [2025-10-20T20:10:43.851434] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T20:10:44.251462] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-02. [2025-10-20T20:10:45.051377] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-03. [2025-10-20T20:10:45.251216] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T20:10:45.725616] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-02 (2.22 seconds). [2025-10-20T20:10:45.851365] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-03. [2025-10-20T20:10:45.902417] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:10:46.401905] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T20:10:46.651386] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-03. [2025-10-20T20:10:47.451378] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-04. [2025-10-20T20:10:48.251423] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-04. [2025-10-20T20:10:48.416860] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-03 (2.57 seconds). [2025-10-20T20:10:48.818937] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T20:10:48.961267] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-03 (2.31 seconds). [2025-10-20T20:10:49.051383] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-04. [2025-10-20T20:10:49.772159] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T20:10:49.851378] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-04. [2025-10-20T20:10:50.651382] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-05. [2025-10-20T20:10:51.451393] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-05. [2025-10-20T20:10:51.602260] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T20:10:51.653447] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T20:10:52.251380] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-06. [2025-10-20T20:10:52.371516] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-04 (2.52 seconds). [2025-10-20T20:10:52.483662] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T20:10:53.051378] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-05. [2025-10-20T20:10:53.824521] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T20:10:53.851330] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-07. [2025-10-20T20:10:54.651426] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-06. [2025-10-20T20:10:55.256427] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T20:10:55.407709] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-05 (2.36 seconds). [2025-10-20T20:10:55.451452] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-05. [2025-10-20T20:10:56.076780] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T20:10:56.251420] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-07. [2025-10-20T20:10:57.051461] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-06. [2025-10-20T20:10:57.727250] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T20:10:57.851478] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-08. [2025-10-20T20:10:57.990446] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-10-20T20:10:58.651390] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-06. [2025-10-20T20:10:59.338919] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-06 (2.29 seconds). [2025-10-20T20:10:59.374101] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T20:10:59.451339] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-07. [2025-10-20T20:11:00.251443] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-09. [2025-10-20T20:11:01.051350] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-08. [2025-10-20T20:11:01.529574] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-06 (2.88 seconds). [2025-10-20T20:11:01.851447] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-07. [2025-10-20T20:11:02.042273] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T20:11:02.086701] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-07 (2.64 seconds). [2025-10-20T20:11:02.265928] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T20:11:02.651434] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-08. [2025-10-20T20:11:03.451369] [INFO ] [REMOTE ] NJR: Fetching primary data for 2025-10. [2025-10-20T20:11:04.141431] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-07 (2.29 seconds). [2025-10-20T20:11:04.251375] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-09. [2025-10-20T20:11:04.921457] [INFO ] [REMOTE ] NJR: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T20:11:04.927099] [INFO ] [REMOTE ] NJR: Full fetch complete (54834 records). [2025-10-20T20:11:04.927187] [DEBUG] [VALIDATOR ] NJR: Validating primary data. [2025-10-20T20:11:05.053744] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-08. [2025-10-20T20:11:05.339275] [DEBUG] [VALIDATOR ] NJR: All expected trading days are present. [2025-10-20T20:11:05.340351] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:11:05.345325] [DEBUG] [VALIDATOR ] NJR: Primary data validation passed. [2025-10-20T20:11:05.346781] [INFO ] [MAIN ] NJR: Fetched primary data is valid. [2025-10-20T20:11:05.362973] [DEBUG] [LOCAL ] NJR: Saving primary data. [2025-10-20T20:11:05.376161] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-08 (2.72 seconds). [2025-10-20T20:11:05.386893] [DEBUG] [LOCAL ] NJR: Trimmed 54834 records to 38008 (keeping 200 days of data) [2025-10-20T20:11:05.611095] [DEBUG] [LOCAL ] NJR: Primary data saved (38008 records). [2025-10-20T20:11:05.611218] [INFO ] [MAIN ] NJR: Full history saved. [2025-10-20T20:11:05.611292] [INFO ] [MAIN ] Update Progress: 1482/2367 symbols completed. NJR took 27.7s. Estimated remaining: 1h 54m 4s. Rate: 7.8 symbols/min [2025-10-20T20:11:05.777025] [INFO ] [MAIN ] NKX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:05.791365] [DEBUG] [LOCAL ] NKX: Loaded 13862 existing records from primary storage. [2025-10-20T20:11:05.791553] [INFO ] [MAIN ] NKX: Loaded 13862 existing records. [2025-10-20T20:11:05.856470] [INFO ] [REMOTE ] NINE: Fetching primary data for 2025-10. [2025-10-20T20:11:05.973798] [WARN ] [MAIN ] NKX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:05.974004] [INFO ] [MAIN ] NKX: Wiping existing primary data before full refresh. [2025-10-20T20:11:05.974051] [INFO ] [LOCAL ] NKX: Starting deletion of primary data. [2025-10-20T20:11:05.974318] [DEBUG] [LOCAL ] NKX: Deleted primary data file. [2025-10-20T20:11:05.974376] [INFO ] [MAIN ] NKX: Starting full primary refresh. [2025-10-20T20:11:06.327472] [INFO ] [REMOTE ] NKX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:06.651406] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-09. [2025-10-20T20:11:07.284174] [INFO ] [REMOTE ] NINE: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T20:11:07.289039] [INFO ] [REMOTE ] NINE: Full fetch complete (43306 records). [2025-10-20T20:11:07.289174] [DEBUG] [VALIDATOR ] NINE: Validating primary data. [2025-10-20T20:11:07.455402] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-01. [2025-10-20T20:11:07.678418] [DEBUG] [VALIDATOR ] NINE: All expected trading days are present. [2025-10-20T20:11:07.678526] [DEBUG] [VALIDATOR ] NINE: Primary data validation passed. [2025-10-20T20:11:07.679551] [INFO ] [MAIN ] NINE: Fetched primary data is valid. [2025-10-20T20:11:07.679681] [DEBUG] [LOCAL ] NINE: Saving primary data. [2025-10-20T20:11:07.710284] [DEBUG] [LOCAL ] NINE: Trimmed 43306 records to 28597 (keeping 200 days of data) [2025-10-20T20:11:07.712773] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-08 (2.66 seconds). [2025-10-20T20:11:07.884517] [DEBUG] [LOCAL ] NINE: Primary data saved (28597 records). [2025-10-20T20:11:07.884694] [INFO ] [MAIN ] NINE: Full history saved. [2025-10-20T20:11:07.884788] [INFO ] [MAIN ] Update Progress: 1483/2367 symbols completed. NINE took 30.0s. Estimated remaining: 1h 53m 53s. Rate: 7.8 symbols/min [2025-10-20T20:11:08.051275] [INFO ] [MAIN ] NL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:08.051453] [DEBUG] [LOCAL ] NL: No existing data file found in primary storage. [2025-10-20T20:11:08.051490] [WARN ] [MAIN ] NL: No existing data found, performing full refresh instead. [2025-10-20T20:11:08.051595] [INFO ] [MAIN ] NL: Wiping existing primary data before full refresh. [2025-10-20T20:11:08.051636] [INFO ] [LOCAL ] NL: Starting deletion of primary data. [2025-10-20T20:11:08.051722] [DEBUG] [LOCAL ] NL: No primary data file found to delete. [2025-10-20T20:11:08.051760] [INFO ] [MAIN ] NL: Starting full primary refresh. [2025-10-20T20:11:08.259551] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-09. [2025-10-20T20:11:08.413517] [INFO ] [REMOTE ] NL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:08.422488] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T20:11:09.051375] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-01. [2025-10-20T20:11:09.170458] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-09 (2.52 seconds). [2025-10-20T20:11:09.700135] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T20:11:09.851455] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-02. [2025-10-20T20:11:10.330033] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T20:11:10.651423] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-02. [2025-10-20T20:11:10.712420] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T20:11:11.288538] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T20:11:11.451425] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-03. [2025-10-20T20:11:12.033316] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T20:11:12.251429] [INFO ] [REMOTE ] NKE: Fetching primary data for 2025-10. [2025-10-20T20:11:12.931471] [INFO ] [REMOTE ] NKE: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:11:12.939696] [INFO ] [REMOTE ] NKE: Full fetch complete (77217 records). [2025-10-20T20:11:12.939864] [DEBUG] [VALIDATOR ] NKE: Validating primary data. [2025-10-20T20:11:13.056504] [INFO ] [REMOTE ] NIO: Fetching primary data for 2025-10. [2025-10-20T20:11:13.149406] [WARN ] [VALIDATOR ] NKE: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T20:11:13.150179] [FATAL] [MAIN ] NKE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:11:13.150248] [WARN ] [MAIN ] NKE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:11:13.150318] [INFO ] [LOCAL ] NKE: Starting deletion of primary data. [2025-10-20T20:11:13.150383] [DEBUG] [LOCAL ] NKE: No primary data file found to delete. [2025-10-20T20:11:13.150418] [INFO ] [LOCAL ] NKE: Starting deletion of derivative data. [2025-10-20T20:11:13.150455] [DEBUG] [LOCAL ] NKE: No derivative data file found to delete. [2025-10-20T20:11:13.150511] [INFO ] [MAIN ] Update Progress: 1484/2367 symbols completed. NKE took 32.8s. Estimated remaining: 1h 53m 44s. Rate: 7.8 symbols/min [2025-10-20T20:11:13.321436] [INFO ] [MAIN ] NLOP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:13.333933] [DEBUG] [LOCAL ] NLOP: Loaded 11880 existing records from primary storage. [2025-10-20T20:11:13.334152] [INFO ] [MAIN ] NLOP: Loaded 11880 existing records. [2025-10-20T20:11:13.509551] [WARN ] [MAIN ] NLOP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:13.509812] [INFO ] [MAIN ] NLOP: Wiping existing primary data before full refresh. [2025-10-20T20:11:13.509902] [INFO ] [LOCAL ] NLOP: Starting deletion of primary data. [2025-10-20T20:11:13.510163] [DEBUG] [LOCAL ] NLOP: Deleted primary data file. [2025-10-20T20:11:13.510219] [INFO ] [MAIN ] NLOP: Starting full primary refresh. [2025-10-20T20:11:13.857717] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-04. [2025-10-20T20:11:13.902132] [INFO ] [REMOTE ] NLOP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:14.579768] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T20:11:14.651379] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-05. [2025-10-20T20:11:15.242224] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T20:11:15.451429] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-06. [2025-10-20T20:11:15.568006] [INFO ] [REMOTE ] NIO: Incremental fetch complete for 2025-10 (2.51 seconds). [2025-10-20T20:11:15.576778] [INFO ] [REMOTE ] NIO: Full fetch complete (77982 records). [2025-10-20T20:11:15.576999] [DEBUG] [VALIDATOR ] NIO: Validating primary data. [2025-10-20T20:11:16.004759] [DEBUG] [VALIDATOR ] NIO: All expected trading days are present. [2025-10-20T20:11:16.004907] [DEBUG] [VALIDATOR ] NIO: Primary data validation passed. [2025-10-20T20:11:16.006609] [INFO ] [MAIN ] NIO: Fetched primary data is valid. [2025-10-20T20:11:16.006712] [DEBUG] [LOCAL ] NIO: Saving primary data. [2025-10-20T20:11:16.008869] [DEBUG] [LOCAL ] NIO: Trimmed 77982 records to 53802 (keeping 200 days of data) [2025-10-20T20:11:16.116370] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T20:11:16.251595] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-01. [2025-10-20T20:11:16.366077] [DEBUG] [LOCAL ] NIO: Primary data saved (53802 records). [2025-10-20T20:11:16.366292] [INFO ] [MAIN ] NIO: Full history saved. [2025-10-20T20:11:16.366705] [INFO ] [MAIN ] Update Progress: 1485/2367 symbols completed. NIO took 38.5s. Estimated remaining: 1h 53m 34s. Rate: 7.8 symbols/min [2025-10-20T20:11:16.545882] [INFO ] [MAIN ] NLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:16.546156] [DEBUG] [LOCAL ] NLY: No existing data file found in primary storage. [2025-10-20T20:11:16.546195] [WARN ] [MAIN ] NLY: No existing data found, performing full refresh instead. [2025-10-20T20:11:16.546327] [INFO ] [MAIN ] NLY: Wiping existing primary data before full refresh. [2025-10-20T20:11:16.546369] [INFO ] [LOCAL ] NLY: Starting deletion of primary data. [2025-10-20T20:11:16.546412] [DEBUG] [LOCAL ] NLY: No primary data file found to delete. [2025-10-20T20:11:16.546447] [INFO ] [MAIN ] NLY: Starting full primary refresh. [2025-10-20T20:11:16.930871] [INFO ] [REMOTE ] NLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:17.051378] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-01. [2025-10-20T20:11:17.258263] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T20:11:17.851385] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-03. [2025-10-20T20:11:18.651420] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-07. [2025-10-20T20:11:18.673026] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:11:18.886545] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T20:11:19.359445] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T20:11:19.451416] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-02. [2025-10-20T20:11:20.251367] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-08. [2025-10-20T20:11:20.827331] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:11:21.051354] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-04. [2025-10-20T20:11:21.225316] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T20:11:21.656027] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T20:11:21.851399] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-05. [2025-10-20T20:11:22.651413] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-09. [2025-10-20T20:11:22.664374] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T20:11:23.212674] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T20:11:23.451434] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-06. [2025-10-20T20:11:24.251370] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-03. [2025-10-20T20:11:24.440390] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T20:11:25.051382] [INFO ] [REMOTE ] NL: Fetching primary data for 2025-10. [2025-10-20T20:11:25.851427] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-02. [2025-10-20T20:11:25.868786] [INFO ] [REMOTE ] NL: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T20:11:25.870040] [INFO ] [REMOTE ] NL: Full fetch complete (6916 records). [2025-10-20T20:11:25.870123] [DEBUG] [VALIDATOR ] NL: Validating primary data. [2025-10-20T20:11:26.057216] [WARN ] [VALIDATOR ] NL: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T20:11:26.057597] [FATAL] [MAIN ] NL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:11:26.057755] [WARN ] [MAIN ] NL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:11:26.057829] [INFO ] [LOCAL ] NL: Starting deletion of primary data. [2025-10-20T20:11:26.057895] [DEBUG] [LOCAL ] NL: No primary data file found to delete. [2025-10-20T20:11:26.057933] [INFO ] [LOCAL ] NL: Starting deletion of derivative data. [2025-10-20T20:11:26.057972] [DEBUG] [LOCAL ] NL: No derivative data file found to delete. [2025-10-20T20:11:26.058026] [INFO ] [MAIN ] Update Progress: 1486/2367 symbols completed. NL took 18.2s. Estimated remaining: 1h 53m 27s. Rate: 7.8 symbols/min [2025-10-20T20:11:26.231235] [INFO ] [MAIN ] NMAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:26.291586] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T20:11:26.297116] [DEBUG] [LOCAL ] NMAI: Loaded 13002 existing records from primary storage. [2025-10-20T20:11:26.308934] [INFO ] [MAIN ] NMAI: Loaded 13002 existing records. [2025-10-20T20:11:26.491465] [WARN ] [MAIN ] NMAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:26.491706] [INFO ] [MAIN ] NMAI: Wiping existing primary data before full refresh. [2025-10-20T20:11:26.491767] [INFO ] [LOCAL ] NMAI: Starting deletion of primary data. [2025-10-20T20:11:26.491975] [DEBUG] [LOCAL ] NMAI: Deleted primary data file. [2025-10-20T20:11:26.492033] [INFO ] [MAIN ] NMAI: Starting full primary refresh. [2025-10-20T20:11:26.656569] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-04. [2025-10-20T20:11:26.791944] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T20:11:26.869724] [INFO ] [REMOTE ] NMAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:27.451427] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-03. [2025-10-20T20:11:28.253927] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-01. [2025-10-20T20:11:28.255637] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T20:11:28.734025] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T20:11:29.051411] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-04. [2025-10-20T20:11:29.329248] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T20:11:29.851374] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-07. [2025-10-20T20:11:29.997001] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T20:11:30.522602] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T20:11:30.651351] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-08. [2025-10-20T20:11:31.425842] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:11:31.451328] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-09. [2025-10-20T20:11:32.251407] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-02. [2025-10-20T20:11:32.380307] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:11:33.027417] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:11:33.051349] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-05. [2025-10-20T20:11:33.851412] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-03. [2025-10-20T20:11:34.651351] [INFO ] [REMOTE ] NKX: Fetching primary data for 2025-10. [2025-10-20T20:11:34.895217] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T20:11:34.895708] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T20:11:35.216701] [INFO ] [REMOTE ] NKX: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T20:11:35.219001] [INFO ] [REMOTE ] NKX: Full fetch complete (19098 records). [2025-10-20T20:11:35.219093] [DEBUG] [VALIDATOR ] NKX: Validating primary data. [2025-10-20T20:11:35.461914] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-04. [2025-10-20T20:11:35.590984] [DEBUG] [VALIDATOR ] NKX: All expected trading days are present. [2025-10-20T20:11:35.591075] [DEBUG] [VALIDATOR ] NKX: Primary data validation passed. [2025-10-20T20:11:35.591490] [INFO ] [MAIN ] NKX: Fetched primary data is valid. [2025-10-20T20:11:35.591531] [DEBUG] [LOCAL ] NKX: Saving primary data. [2025-10-20T20:11:35.592061] [DEBUG] [LOCAL ] NKX: Trimmed 19098 records to 13697 (keeping 200 days of data) [2025-10-20T20:11:35.672041] [DEBUG] [LOCAL ] NKX: Primary data saved (13697 records). [2025-10-20T20:11:35.672165] [INFO ] [MAIN ] NKX: Full history saved. [2025-10-20T20:11:35.672240] [INFO ] [MAIN ] Update Progress: 1487/2367 symbols completed. NKX took 30.1s. Estimated remaining: 1h 53m 20s. Rate: 7.8 symbols/min [2025-10-20T20:11:35.838877] [INFO ] [MAIN ] NMAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:35.839023] [DEBUG] [LOCAL ] NMAX: No existing data file found in primary storage. [2025-10-20T20:11:35.839059] [WARN ] [MAIN ] NMAX: No existing data found, performing full refresh instead. [2025-10-20T20:11:35.839164] [INFO ] [MAIN ] NMAX: Wiping existing primary data before full refresh. [2025-10-20T20:11:35.839204] [INFO ] [LOCAL ] NMAX: Starting deletion of primary data. [2025-10-20T20:11:35.839243] [DEBUG] [LOCAL ] NMAX: No primary data file found to delete. [2025-10-20T20:11:35.839276] [INFO ] [MAIN ] NMAX: Starting full primary refresh. [2025-10-20T20:11:36.190842] [INFO ] [REMOTE ] NMAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:36.251398] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-05. [2025-10-20T20:11:36.283845] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:11:37.030051] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T20:11:37.051323] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-06. [2025-10-20T20:11:37.856520] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-05. [2025-10-20T20:11:37.858692] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T20:11:38.627287] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T20:11:38.651378] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-06. [2025-10-20T20:11:39.451403] [INFO ] [REMOTE ] NMAX: Fetching primary data for 2025-01. [2025-10-20T20:11:39.583190] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T20:11:39.663533] [DEBUG] [REMOTE ] NMAX: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:11:39.663879] [WARN ] [REMOTE ] NMAX: No data returned for month 2025-01 [2025-10-20T20:11:39.663982] [FATAL] [REMOTE ] NMAX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:11:39.664056] [FATAL] [MAIN ] NMAX: Failed to fetch primary data. [2025-10-20T20:11:39.664128] [WARN ] [MAIN ] NMAX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:11:39.664196] [INFO ] [LOCAL ] NMAX: Starting deletion of primary data. [2025-10-20T20:11:39.664261] [DEBUG] [LOCAL ] NMAX: No primary data file found to delete. [2025-10-20T20:11:39.664295] [INFO ] [LOCAL ] NMAX: Starting deletion of derivative data. [2025-10-20T20:11:39.664332] [DEBUG] [LOCAL ] NMAX: No derivative data file found to delete. [2025-10-20T20:11:39.664376] [INFO ] [MAIN ] Update Progress: 1488/2367 symbols completed. NMAX took 4.0s. Estimated remaining: 1h 53m 11s. Rate: 7.8 symbols/min [2025-10-20T20:11:39.829712] [INFO ] [MAIN ] NMCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:39.845828] [DEBUG] [LOCAL ] NMCO: Loaded 16861 existing records from primary storage. [2025-10-20T20:11:39.846064] [INFO ] [MAIN ] NMCO: Loaded 16861 existing records. [2025-10-20T20:11:40.030930] [WARN ] [MAIN ] NMCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:40.031158] [INFO ] [MAIN ] NMCO: Wiping existing primary data before full refresh. [2025-10-20T20:11:40.031205] [INFO ] [LOCAL ] NMCO: Starting deletion of primary data. [2025-10-20T20:11:40.031388] [DEBUG] [LOCAL ] NMCO: Deleted primary data file. [2025-10-20T20:11:40.031425] [INFO ] [MAIN ] NMCO: Starting full primary refresh. [2025-10-20T20:11:40.256555] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-07. [2025-10-20T20:11:40.381874] [INFO ] [REMOTE ] NMCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:41.051366] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-06. [2025-10-20T20:11:41.117946] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T20:11:41.851395] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-07. [2025-10-20T20:11:42.651363] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-08. [2025-10-20T20:11:42.745368] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T20:11:42.920608] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T20:11:43.451423] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-01. [2025-10-20T20:11:43.657046] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T20:11:44.251380] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-09. [2025-10-20T20:11:44.381963] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T20:11:45.051373] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-08. [2025-10-20T20:11:45.338699] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:11:45.851369] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-02. [2025-10-20T20:11:45.923353] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:11:46.651350] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-09. [2025-10-20T20:11:46.731728] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T20:11:47.451396] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-03. [2025-10-20T20:11:47.664988] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T20:11:48.251375] [INFO ] [REMOTE ] NMAI: Fetching primary data for 2025-10. [2025-10-20T20:11:48.294578] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T20:11:48.909025] [INFO ] [REMOTE ] NMAI: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:11:48.911469] [INFO ] [REMOTE ] NMAI: Full fetch complete (19530 records). [2025-10-20T20:11:48.911545] [DEBUG] [VALIDATOR ] NMAI: Validating primary data. [2025-10-20T20:11:49.059200] [INFO ] [REMOTE ] NLOP: Fetching primary data for 2025-10. [2025-10-20T20:11:49.279361] [DEBUG] [VALIDATOR ] NMAI: All expected trading days are present. [2025-10-20T20:11:49.279448] [DEBUG] [VALIDATOR ] NMAI: Primary data validation passed. [2025-10-20T20:11:49.279974] [INFO ] [MAIN ] NMAI: Fetched primary data is valid. [2025-10-20T20:11:49.280036] [DEBUG] [LOCAL ] NMAI: Saving primary data. [2025-10-20T20:11:49.280607] [DEBUG] [LOCAL ] NMAI: Trimmed 19530 records to 12634 (keeping 200 days of data) [2025-10-20T20:11:49.357308] [DEBUG] [LOCAL ] NMAI: Primary data saved (12634 records). [2025-10-20T20:11:49.357445] [INFO ] [MAIN ] NMAI: Full history saved. [2025-10-20T20:11:49.357523] [INFO ] [MAIN ] Update Progress: 1489/2367 symbols completed. NMAI took 23.3s. Estimated remaining: 1h 53m 4s. Rate: 7.8 symbols/min [2025-10-20T20:11:49.529346] [INFO ] [MAIN ] NMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:49.542090] [DEBUG] [LOCAL ] NMG: Loaded 11397 existing records from primary storage. [2025-10-20T20:11:49.542310] [INFO ] [MAIN ] NMG: Loaded 11397 existing records. [2025-10-20T20:11:49.717273] [WARN ] [MAIN ] NMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:49.717505] [INFO ] [MAIN ] NMG: Wiping existing primary data before full refresh. [2025-10-20T20:11:49.717553] [INFO ] [LOCAL ] NMG: Starting deletion of primary data. [2025-10-20T20:11:49.717748] [DEBUG] [LOCAL ] NMG: Deleted primary data file. [2025-10-20T20:11:49.717804] [INFO ] [MAIN ] NMG: Starting full primary refresh. [2025-10-20T20:11:49.857522] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-04. [2025-10-20T20:11:50.091972] [INFO ] [REMOTE ] NMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:50.113414] [INFO ] [REMOTE ] NLOP: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T20:11:50.115341] [INFO ] [REMOTE ] NLOP: Full fetch complete (17735 records). [2025-10-20T20:11:50.115406] [DEBUG] [VALIDATOR ] NLOP: Validating primary data. [2025-10-20T20:11:50.478708] [DEBUG] [VALIDATOR ] NLOP: All expected trading days are present. [2025-10-20T20:11:50.478787] [DEBUG] [VALIDATOR ] NLOP: Primary data validation passed. [2025-10-20T20:11:50.479168] [INFO ] [MAIN ] NLOP: Fetched primary data is valid. [2025-10-20T20:11:50.479204] [DEBUG] [LOCAL ] NLOP: Saving primary data. [2025-10-20T20:11:50.479808] [DEBUG] [LOCAL ] NLOP: Trimmed 17735 records to 11699 (keeping 200 days of data) [2025-10-20T20:11:50.563899] [DEBUG] [LOCAL ] NLOP: Primary data saved (11699 records). [2025-10-20T20:11:50.564075] [INFO ] [MAIN ] NLOP: Full history saved. [2025-10-20T20:11:50.564184] [INFO ] [MAIN ] Update Progress: 1490/2367 symbols completed. NLOP took 37.4s. Estimated remaining: 1h 52m 52s. Rate: 7.8 symbols/min [2025-10-20T20:11:50.659369] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-01. [2025-10-20T20:11:50.740319] [INFO ] [MAIN ] NMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:11:50.746103] [DEBUG] [LOCAL ] NMI: Loaded 4080 existing records from primary storage. [2025-10-20T20:11:50.746316] [INFO ] [MAIN ] NMI: Loaded 4080 existing records. [2025-10-20T20:11:50.937343] [WARN ] [MAIN ] NMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:11:50.937562] [INFO ] [MAIN ] NMI: Wiping existing primary data before full refresh. [2025-10-20T20:11:50.937610] [INFO ] [LOCAL ] NMI: Starting deletion of primary data. [2025-10-20T20:11:50.937783] [DEBUG] [LOCAL ] NMI: Deleted primary data file. [2025-10-20T20:11:50.937836] [INFO ] [MAIN ] NMI: Starting full primary refresh. [2025-10-20T20:11:51.108673] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T20:11:51.330187] [INFO ] [REMOTE ] NMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:11:51.364378] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T20:11:51.451445] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-01. [2025-10-20T20:11:52.181991] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T20:11:52.251347] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-02. [2025-10-20T20:11:53.024749] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:11:53.051391] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-03. [2025-10-20T20:11:53.645422] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T20:11:53.851392] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-07. [2025-10-20T20:11:54.651420] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-04. [2025-10-20T20:11:55.441780] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T20:11:55.451323] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-05. [2025-10-20T20:11:55.881969] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T20:11:55.881626] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T20:11:56.251423] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-06. [2025-10-20T20:11:56.870934] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T20:11:57.051413] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-05. [2025-10-20T20:11:57.851513] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-02. [2025-10-20T20:11:57.992717] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T20:11:58.434865] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T20:11:58.651414] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-03. [2025-10-20T20:11:59.116707] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T20:11:59.451391] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-08. [2025-10-20T20:12:00.251482] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-07. [2025-10-20T20:12:01.051394] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-04. [2025-10-20T20:12:01.197311] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T20:12:01.442610] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-04 (0.39 seconds). [2025-10-20T20:12:01.667130] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T20:12:01.851363] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-09. [2025-10-20T20:12:02.651387] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-05. [2025-10-20T20:12:03.218966] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T20:12:03.451381] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-06. [2025-10-20T20:12:03.771403] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T20:12:04.251391] [INFO ] [REMOTE ] NLY: Fetching primary data for 2025-10. [2025-10-20T20:12:04.397090] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T20:12:05.051444] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-07. [2025-10-20T20:12:05.851353] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-06. [2025-10-20T20:12:06.031119] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T20:12:06.149973] [INFO ] [REMOTE ] NLY: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T20:12:06.158511] [INFO ] [REMOTE ] NLY: Full fetch complete (77697 records). [2025-10-20T20:12:06.158707] [DEBUG] [VALIDATOR ] NLY: Validating primary data. [2025-10-20T20:12:06.356878] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T20:12:06.580529] [DEBUG] [VALIDATOR ] NLY: All expected trading days are present. [2025-10-20T20:12:06.580679] [DEBUG] [VALIDATOR ] NLY: Primary data validation passed. [2025-10-20T20:12:06.582238] [INFO ] [MAIN ] NLY: Fetched primary data is valid. [2025-10-20T20:12:06.582307] [DEBUG] [LOCAL ] NLY: Saving primary data. [2025-10-20T20:12:06.584407] [DEBUG] [LOCAL ] NLY: Trimmed 77697 records to 53564 (keeping 200 days of data) [2025-10-20T20:12:06.687134] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-08. [2025-10-20T20:12:06.966360] [DEBUG] [LOCAL ] NLY: Primary data saved (53564 records). [2025-10-20T20:12:06.966584] [INFO ] [MAIN ] NLY: Full history saved. [2025-10-20T20:12:06.966703] [INFO ] [MAIN ] Update Progress: 1491/2367 symbols completed. NLY took 50.6s. Estimated remaining: 1h 52m 50s. Rate: 7.8 symbols/min [2025-10-20T20:12:07.149715] [INFO ] [MAIN ] NMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:07.167885] [DEBUG] [LOCAL ] NMM: Loaded 14552 existing records from primary storage. [2025-10-20T20:12:07.168236] [INFO ] [MAIN ] NMM: Loaded 14552 existing records. [2025-10-20T20:12:07.265394] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:12:07.378767] [WARN ] [MAIN ] NMM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:07.379154] [INFO ] [MAIN ] NMM: Wiping existing primary data before full refresh. [2025-10-20T20:12:07.379210] [INFO ] [LOCAL ] NMM: Starting deletion of primary data. [2025-10-20T20:12:07.379455] [DEBUG] [LOCAL ] NMM: Deleted primary data file. [2025-10-20T20:12:07.379515] [INFO ] [MAIN ] NMM: Starting full primary refresh. [2025-10-20T20:12:07.456755] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-07. [2025-10-20T20:12:07.766320] [INFO ] [REMOTE ] NMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:08.118671] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-07 (0.66 seconds). [2025-10-20T20:12:08.251411] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-09. [2025-10-20T20:12:09.051437] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-08. [2025-10-20T20:12:09.069976] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:12:09.674391] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T20:12:09.851369] [INFO ] [REMOTE ] NMG: Fetching primary data for 2025-10. [2025-10-20T20:12:10.651371] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-08. [2025-10-20T20:12:11.350478] [INFO ] [REMOTE ] NMG: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T20:12:11.359720] [INFO ] [REMOTE ] NMG: Full fetch complete (15796 records). [2025-10-20T20:12:11.359905] [DEBUG] [VALIDATOR ] NMG: Validating primary data. [2025-10-20T20:12:11.454150] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-09. [2025-10-20T20:12:11.786726] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T20:12:11.802618] [DEBUG] [VALIDATOR ] NMG: All expected trading days are present. [2025-10-20T20:12:11.803394] [DEBUG] [VALIDATOR ] NMG: Primary data validation passed. [2025-10-20T20:12:11.804756] [INFO ] [MAIN ] NMG: Fetched primary data is valid. [2025-10-20T20:12:11.804827] [DEBUG] [LOCAL ] NMG: Saving primary data. [2025-10-20T20:12:11.805869] [DEBUG] [LOCAL ] NMG: Trimmed 15796 records to 11574 (keeping 200 days of data) [2025-10-20T20:12:11.881183] [DEBUG] [LOCAL ] NMG: Primary data saved (11574 records). [2025-10-20T20:12:11.881341] [INFO ] [MAIN ] NMG: Full history saved. [2025-10-20T20:12:11.881424] [INFO ] [MAIN ] Update Progress: 1492/2367 symbols completed. NMG took 22.5s. Estimated remaining: 1h 52m 40s. Rate: 7.8 symbols/min [2025-10-20T20:12:12.053382] [INFO ] [MAIN ] NMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:12.087229] [DEBUG] [LOCAL ] NMR: Loaded 36138 existing records from primary storage. [2025-10-20T20:12:12.087521] [INFO ] [MAIN ] NMR: Loaded 36138 existing records. [2025-10-20T20:12:12.179864] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-09 (0.73 seconds). [2025-10-20T20:12:12.261406] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-01. [2025-10-20T20:12:12.297299] [WARN ] [MAIN ] NMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:12.297514] [INFO ] [MAIN ] NMR: Wiping existing primary data before full refresh. [2025-10-20T20:12:12.297561] [INFO ] [LOCAL ] NMR: Starting deletion of primary data. [2025-10-20T20:12:12.297893] [DEBUG] [LOCAL ] NMR: Deleted primary data file. [2025-10-20T20:12:12.297952] [INFO ] [MAIN ] NMR: Starting full primary refresh. [2025-10-20T20:12:12.662403] [INFO ] [REMOTE ] NMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:13.051349] [INFO ] [REMOTE ] NMI: Fetching primary data for 2025-10. [2025-10-20T20:12:13.284871] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T20:12:13.670698] [INFO ] [REMOTE ] NMI: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:12:13.671601] [INFO ] [REMOTE ] NMI: Full fetch complete (5707 records). [2025-10-20T20:12:13.671692] [DEBUG] [VALIDATOR ] NMI: Validating primary data. [2025-10-20T20:12:13.856946] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-09. [2025-10-20T20:12:14.022852] [DEBUG] [VALIDATOR ] NMI: All expected trading days are present. [2025-10-20T20:12:14.022929] [DEBUG] [VALIDATOR ] NMI: Primary data validation passed. [2025-10-20T20:12:14.023090] [INFO ] [MAIN ] NMI: Fetched primary data is valid. [2025-10-20T20:12:14.023128] [DEBUG] [LOCAL ] NMI: Saving primary data. [2025-10-20T20:12:14.023509] [DEBUG] [LOCAL ] NMI: Trimmed 5707 records to 4049 (keeping 200 days of data) [2025-10-20T20:12:14.047188] [DEBUG] [LOCAL ] NMI: Primary data saved (4049 records). [2025-10-20T20:12:14.047259] [INFO ] [MAIN ] NMI: Full history saved. [2025-10-20T20:12:14.047328] [INFO ] [MAIN ] Update Progress: 1493/2367 symbols completed. NMI took 23.5s. Estimated remaining: 1h 52m 29s. Rate: 7.8 symbols/min [2025-10-20T20:12:14.241343] [INFO ] [MAIN ] NMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:14.248530] [DEBUG] [LOCAL ] NMS: Loaded 2936 existing records from primary storage. [2025-10-20T20:12:14.248782] [INFO ] [MAIN ] NMS: Loaded 2936 existing records. [2025-10-20T20:12:14.423256] [WARN ] [MAIN ] NMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:14.423460] [INFO ] [MAIN ] NMS: Wiping existing primary data before full refresh. [2025-10-20T20:12:14.423506] [INFO ] [LOCAL ] NMS: Starting deletion of primary data. [2025-10-20T20:12:14.423679] [DEBUG] [LOCAL ] NMS: Deleted primary data file. [2025-10-20T20:12:14.423737] [INFO ] [MAIN ] NMS: Starting full primary refresh. [2025-10-20T20:12:14.660692] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-01. [2025-10-20T20:12:14.799248] [INFO ] [REMOTE ] NMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:14.890224] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T20:12:15.451404] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-01. [2025-10-20T20:12:15.980074] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-01 (0.53 seconds). [2025-10-20T20:12:16.128538] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T20:12:16.251443] [INFO ] [REMOTE ] NMCO: Fetching primary data for 2025-10. [2025-10-20T20:12:17.051412] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-02. [2025-10-20T20:12:17.184453] [INFO ] [REMOTE ] NMCO: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T20:12:17.187086] [INFO ] [REMOTE ] NMCO: Full fetch complete (22851 records). [2025-10-20T20:12:17.187160] [DEBUG] [VALIDATOR ] NMCO: Validating primary data. [2025-10-20T20:12:17.546808] [DEBUG] [VALIDATOR ] NMCO: All expected trading days are present. [2025-10-20T20:12:17.546891] [DEBUG] [VALIDATOR ] NMCO: Primary data validation passed. [2025-10-20T20:12:17.547399] [INFO ] [MAIN ] NMCO: Fetched primary data is valid. [2025-10-20T20:12:17.547442] [DEBUG] [LOCAL ] NMCO: Saving primary data. [2025-10-20T20:12:17.548062] [DEBUG] [LOCAL ] NMCO: Trimmed 22851 records to 16596 (keeping 200 days of data) [2025-10-20T20:12:17.647910] [DEBUG] [LOCAL ] NMCO: Primary data saved (16596 records). [2025-10-20T20:12:17.648069] [INFO ] [MAIN ] NMCO: Full history saved. [2025-10-20T20:12:17.648155] [INFO ] [MAIN ] Update Progress: 1494/2367 symbols completed. NMCO took 38.0s. Estimated remaining: 1h 52m 19s. Rate: 7.8 symbols/min [2025-10-20T20:12:17.818444] [INFO ] [MAIN ] NMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:17.823839] [DEBUG] [LOCAL ] NMT: Loaded 2867 existing records from primary storage. [2025-10-20T20:12:17.824008] [INFO ] [MAIN ] NMT: Loaded 2867 existing records. [2025-10-20T20:12:17.856539] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-02. [2025-10-20T20:12:18.011090] [WARN ] [MAIN ] NMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:18.011447] [INFO ] [MAIN ] NMT: Wiping existing primary data before full refresh. [2025-10-20T20:12:18.011501] [INFO ] [LOCAL ] NMT: Starting deletion of primary data. [2025-10-20T20:12:18.011724] [DEBUG] [LOCAL ] NMT: Deleted primary data file. [2025-10-20T20:12:18.011782] [INFO ] [MAIN ] NMT: Starting full primary refresh. [2025-10-20T20:12:18.392549] [INFO ] [REMOTE ] NMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:18.396838] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T20:12:18.397816] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T20:12:18.651401] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-03. [2025-10-20T20:12:19.451393] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-03. [2025-10-20T20:12:19.901824] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T20:12:20.177912] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T20:12:20.251348] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-04. [2025-10-20T20:12:21.051381] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-02. [2025-10-20T20:12:21.769297] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T20:12:21.816397] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:12:21.851377] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-03. [2025-10-20T20:12:22.651423] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-04. [2025-10-20T20:12:22.723892] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:12:23.101808] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T20:12:23.451376] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-05. [2025-10-20T20:12:24.251394] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-04. [2025-10-20T20:12:24.826880] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T20:12:25.051402] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-06. [2025-10-20T20:12:25.351264] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T20:12:25.851386] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-01. [2025-10-20T20:12:26.380433] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T20:12:26.441198] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T20:12:26.651386] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-05. [2025-10-20T20:12:27.221833] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T20:12:27.451412] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-06. [2025-10-20T20:12:27.921608] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T20:12:28.251410] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-05. [2025-10-20T20:12:29.051351] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-07. [2025-10-20T20:12:29.075808] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T20:12:29.851409] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-07. [2025-10-20T20:12:30.329742] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T20:12:30.401555] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T20:12:30.651372] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-08. [2025-10-20T20:12:31.251729] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T20:12:31.451412] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-06. [2025-10-20T20:12:32.251355] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-09. [2025-10-20T20:12:32.351780] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T20:12:32.724294] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T20:12:33.051359] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-08. [2025-10-20T20:12:33.851384] [INFO ] [REMOTE ] NMS: Fetching primary data for 2025-10. [2025-10-20T20:12:34.397195] [INFO ] [REMOTE ] NMS: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:12:34.398566] [INFO ] [REMOTE ] NMS: Full fetch complete (4053 records). [2025-10-20T20:12:34.398683] [DEBUG] [VALIDATOR ] NMS: Validating primary data. [2025-10-20T20:12:34.543220] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T20:12:34.656521] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-09. [2025-10-20T20:12:34.834014] [DEBUG] [VALIDATOR ] NMS: All expected trading days are present. [2025-10-20T20:12:34.834149] [DEBUG] [VALIDATOR ] NMS: Primary data validation passed. [2025-10-20T20:12:34.834358] [INFO ] [MAIN ] NMS: Fetched primary data is valid. [2025-10-20T20:12:34.834401] [DEBUG] [LOCAL ] NMS: Saving primary data. [2025-10-20T20:12:34.834854] [DEBUG] [LOCAL ] NMS: Trimmed 4053 records to 2879 (keeping 200 days of data) [2025-10-20T20:12:34.853538] [DEBUG] [LOCAL ] NMS: Primary data saved (2879 records). [2025-10-20T20:12:34.853624] [INFO ] [MAIN ] NMS: Full history saved. [2025-10-20T20:12:34.853742] [INFO ] [MAIN ] Update Progress: 1495/2367 symbols completed. NMS took 20.8s. Estimated remaining: 1h 52m 17s. Rate: 7.8 symbols/min [2025-10-20T20:12:35.043544] [INFO ] [MAIN ] NMZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:35.067467] [DEBUG] [LOCAL ] NMZ: Loaded 21897 existing records from primary storage. [2025-10-20T20:12:35.067766] [INFO ] [MAIN ] NMZ: Loaded 21897 existing records. [2025-10-20T20:12:35.264913] [WARN ] [MAIN ] NMZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:35.265130] [INFO ] [MAIN ] NMZ: Wiping existing primary data before full refresh. [2025-10-20T20:12:35.265194] [INFO ] [LOCAL ] NMZ: Starting deletion of primary data. [2025-10-20T20:12:35.265460] [DEBUG] [LOCAL ] NMZ: Deleted primary data file. [2025-10-20T20:12:35.265533] [INFO ] [MAIN ] NMZ: Starting full primary refresh. [2025-10-20T20:12:35.456709] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-02. [2025-10-20T20:12:35.657254] [INFO ] [REMOTE ] NMZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:36.010678] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T20:12:36.186735] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T20:12:36.251343] [INFO ] [REMOTE ] NMR: Fetching primary data for 2025-10. [2025-10-20T20:12:37.051394] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-07. [2025-10-20T20:12:37.851381] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-01. [2025-10-20T20:12:37.878436] [INFO ] [REMOTE ] NMR: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T20:12:37.884898] [INFO ] [REMOTE ] NMR: Full fetch complete (51263 records). [2025-10-20T20:12:37.889289] [DEBUG] [VALIDATOR ] NMR: Validating primary data. [2025-10-20T20:12:37.894076] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:12:38.267538] [DEBUG] [VALIDATOR ] NMR: All expected trading days are present. [2025-10-20T20:12:38.267614] [DEBUG] [VALIDATOR ] NMR: Primary data validation passed. [2025-10-20T20:12:38.268446] [INFO ] [MAIN ] NMR: Fetched primary data is valid. [2025-10-20T20:12:38.268507] [DEBUG] [LOCAL ] NMR: Saving primary data. [2025-10-20T20:12:38.269505] [DEBUG] [LOCAL ] NMR: Trimmed 51263 records to 35622 (keeping 200 days of data) [2025-10-20T20:12:38.462563] [DEBUG] [LOCAL ] NMR: Primary data saved (35622 records). [2025-10-20T20:12:38.462726] [INFO ] [MAIN ] NMR: Full history saved. [2025-10-20T20:12:38.462815] [INFO ] [MAIN ] Update Progress: 1496/2367 symbols completed. NMR took 26.6s. Estimated remaining: 1h 52m 7s. Rate: 7.8 symbols/min [2025-10-20T20:12:38.625872] [INFO ] [MAIN ] NNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:38.638279] [DEBUG] [LOCAL ] NNI: Loaded 10605 existing records from primary storage. [2025-10-20T20:12:38.638492] [INFO ] [MAIN ] NNI: Loaded 10605 existing records. [2025-10-20T20:12:38.658482] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-03. [2025-10-20T20:12:38.817893] [WARN ] [MAIN ] NNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:38.818378] [INFO ] [MAIN ] NNI: Wiping existing primary data before full refresh. [2025-10-20T20:12:38.818491] [INFO ] [LOCAL ] NNI: Starting deletion of primary data. [2025-10-20T20:12:38.818702] [DEBUG] [LOCAL ] NNI: Deleted primary data file. [2025-10-20T20:12:38.818761] [INFO ] [MAIN ] NNI: Starting full primary refresh. [2025-10-20T20:12:39.163485] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T20:12:39.205227] [INFO ] [REMOTE ] NNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:39.209690] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T20:12:39.451403] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-01. [2025-10-20T20:12:40.251474] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-04. [2025-10-20T20:12:40.306339] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T20:12:40.818760] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T20:12:41.051361] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-02. [2025-10-20T20:12:41.545242] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T20:12:41.851368] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-08. [2025-10-20T20:12:42.651372] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-03. [2025-10-20T20:12:42.848252] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T20:12:43.451343] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-02. [2025-10-20T20:12:43.677304] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:12:44.251355] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-04. [2025-10-20T20:12:45.051452] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-09. [2025-10-20T20:12:45.693698] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T20:12:45.851362] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-05. [2025-10-20T20:12:45.879738] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-02 (2.43 seconds). [2025-10-20T20:12:46.379900] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T20:12:46.466957] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T20:12:46.651373] [INFO ] [REMOTE ] NMM: Fetching primary data for 2025-10. [2025-10-20T20:12:47.357280] [INFO ] [REMOTE ] NMM: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:12:47.359621] [INFO ] [REMOTE ] NMM: Full fetch complete (21624 records). [2025-10-20T20:12:47.359713] [DEBUG] [VALIDATOR ] NMM: Validating primary data. [2025-10-20T20:12:47.456779] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-06. [2025-10-20T20:12:47.740085] [DEBUG] [VALIDATOR ] NMM: All expected trading days are present. [2025-10-20T20:12:47.740163] [DEBUG] [VALIDATOR ] NMM: Primary data validation passed. [2025-10-20T20:12:47.740603] [INFO ] [MAIN ] NMM: Fetched primary data is valid. [2025-10-20T20:12:47.740727] [DEBUG] [LOCAL ] NMM: Saving primary data. [2025-10-20T20:12:47.741280] [DEBUG] [LOCAL ] NMM: Trimmed 21624 records to 14378 (keeping 200 days of data) [2025-10-20T20:12:47.824279] [DEBUG] [LOCAL ] NMM: Primary data saved (14378 records). [2025-10-20T20:12:47.824405] [INFO ] [MAIN ] NMM: Full history saved. [2025-10-20T20:12:47.824481] [INFO ] [MAIN ] Update Progress: 1497/2367 symbols completed. NMM took 40.9s. Estimated remaining: 1h 52m 0s. Rate: 7.8 symbols/min [2025-10-20T20:12:47.992633] [INFO ] [MAIN ] NNN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:48.001435] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T20:12:48.044749] [DEBUG] [LOCAL ] NNN: Loaded 48962 existing records from primary storage. [2025-10-20T20:12:48.045092] [INFO ] [MAIN ] NNN: Loaded 48962 existing records. [2025-10-20T20:12:48.217787] [WARN ] [MAIN ] NNN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:48.218018] [INFO ] [MAIN ] NNN: Wiping existing primary data before full refresh. [2025-10-20T20:12:48.218064] [INFO ] [LOCAL ] NNN: Starting deletion of primary data. [2025-10-20T20:12:48.218324] [DEBUG] [LOCAL ] NNN: Deleted primary data file. [2025-10-20T20:12:48.218378] [INFO ] [MAIN ] NNN: Starting full primary refresh. [2025-10-20T20:12:48.256675] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-05. [2025-10-20T20:12:48.570593] [INFO ] [REMOTE ] NNN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:49.051416] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-07. [2025-10-20T20:12:49.128059] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:12:49.851433] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-01. [2025-10-20T20:12:49.930499] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T20:12:50.651423] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-06. [2025-10-20T20:12:51.091468] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T20:12:51.451399] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-08. [2025-10-20T20:12:51.558495] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T20:12:52.260636] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-07. [2025-10-20T20:12:52.264381] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T20:12:52.729086] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T20:12:53.051376] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-08. [2025-10-20T20:12:53.522677] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T20:12:53.851411] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-09. [2025-10-20T20:12:54.431224] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T20:12:54.651371] [INFO ] [REMOTE ] NMT: Fetching primary data for 2025-10. [2025-10-20T20:12:55.103308] [INFO ] [REMOTE ] NMT: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T20:12:55.104189] [INFO ] [REMOTE ] NMT: Full fetch complete (3889 records). [2025-10-20T20:12:55.104254] [DEBUG] [VALIDATOR ] NMT: Validating primary data. [2025-10-20T20:12:55.445637] [DEBUG] [VALIDATOR ] NMT: All expected trading days are present. [2025-10-20T20:12:55.445742] [DEBUG] [VALIDATOR ] NMT: Primary data validation passed. [2025-10-20T20:12:55.445860] [INFO ] [MAIN ] NMT: Fetched primary data is valid. [2025-10-20T20:12:55.445897] [DEBUG] [LOCAL ] NMT: Saving primary data. [2025-10-20T20:12:55.446222] [DEBUG] [LOCAL ] NMT: Trimmed 3889 records to 2818 (keeping 200 days of data) [2025-10-20T20:12:55.462456] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-03. [2025-10-20T20:12:55.464413] [DEBUG] [LOCAL ] NMT: Primary data saved (2818 records). [2025-10-20T20:12:55.464667] [INFO ] [MAIN ] NMT: Full history saved. [2025-10-20T20:12:55.464793] [INFO ] [MAIN ] Update Progress: 1498/2367 symbols completed. NMT took 37.8s. Estimated remaining: 1h 51m 52s. Rate: 7.8 symbols/min [2025-10-20T20:12:55.633138] [INFO ] [MAIN ] NNY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:55.640915] [DEBUG] [LOCAL ] NNY: Loaded 5778 existing records from primary storage. [2025-10-20T20:12:55.641083] [INFO ] [MAIN ] NNY: Loaded 5778 existing records. [2025-10-20T20:12:55.810614] [WARN ] [MAIN ] NNY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:55.810881] [INFO ] [MAIN ] NNY: Wiping existing primary data before full refresh. [2025-10-20T20:12:55.810946] [INFO ] [LOCAL ] NNY: Starting deletion of primary data. [2025-10-20T20:12:55.811151] [DEBUG] [LOCAL ] NNY: Deleted primary data file. [2025-10-20T20:12:55.811208] [INFO ] [MAIN ] NNY: Starting full primary refresh. [2025-10-20T20:12:56.210840] [INFO ] [REMOTE ] NNY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:12:56.251329] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-09. [2025-10-20T20:12:56.500001] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-03 (1.04 seconds). [2025-10-20T20:12:57.051411] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-04. [2025-10-20T20:12:57.120160] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:12:57.851434] [INFO ] [REMOTE ] NNI: Fetching primary data for 2025-10. [2025-10-20T20:12:58.209448] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T20:12:58.651359] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-02. [2025-10-20T20:12:58.730601] [INFO ] [REMOTE ] NNI: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T20:12:58.732889] [INFO ] [REMOTE ] NNI: Full fetch complete (13105 records). [2025-10-20T20:12:58.733008] [DEBUG] [VALIDATOR ] NNI: Validating primary data. [2025-10-20T20:12:59.123198] [DEBUG] [VALIDATOR ] NNI: All expected trading days are present. [2025-10-20T20:12:59.123278] [DEBUG] [VALIDATOR ] NNI: Primary data validation passed. [2025-10-20T20:12:59.123577] [INFO ] [MAIN ] NNI: Fetched primary data is valid. [2025-10-20T20:12:59.123617] [DEBUG] [LOCAL ] NNI: Saving primary data. [2025-10-20T20:12:59.124248] [DEBUG] [LOCAL ] NNI: Trimmed 13105 records to 10532 (keeping 200 days of data) [2025-10-20T20:12:59.191624] [DEBUG] [LOCAL ] NNI: Primary data saved (10532 records). [2025-10-20T20:12:59.191797] [INFO ] [MAIN ] NNI: Full history saved. [2025-10-20T20:12:59.191880] [INFO ] [MAIN ] Update Progress: 1499/2367 symbols completed. NNI took 20.7s. Estimated remaining: 1h 51m 42s. Rate: 7.8 symbols/min [2025-10-20T20:12:59.362886] [INFO ] [MAIN ] NOA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:12:59.380580] [DEBUG] [LOCAL ] NOA: Loaded 16210 existing records from primary storage. [2025-10-20T20:12:59.380859] [INFO ] [MAIN ] NOA: Loaded 16210 existing records. [2025-10-20T20:12:59.456486] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-05. [2025-10-20T20:12:59.558616] [WARN ] [MAIN ] NOA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:12:59.558857] [INFO ] [MAIN ] NOA: Wiping existing primary data before full refresh. [2025-10-20T20:12:59.558911] [INFO ] [LOCAL ] NOA: Starting deletion of primary data. [2025-10-20T20:12:59.559186] [DEBUG] [LOCAL ] NOA: Deleted primary data file. [2025-10-20T20:12:59.559256] [INFO ] [MAIN ] NOA: Starting full primary refresh. [2025-10-20T20:12:59.908079] [INFO ] [REMOTE ] NOA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:00.251352] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-01. [2025-10-20T20:13:00.320245] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T20:13:00.370722] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T20:13:01.051339] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-03. [2025-10-20T20:13:01.228232] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T20:13:01.851434] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-02. [2025-10-20T20:13:02.651426] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-01. [2025-10-20T20:13:02.676436] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T20:13:02.825592] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T20:13:03.258426] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T20:13:03.451385] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-04. [2025-10-20T20:13:04.251533] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-02. [2025-10-20T20:13:04.932503] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T20:13:05.051415] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-03. [2025-10-20T20:13:05.129996] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:13:05.851384] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-06. [2025-10-20T20:13:06.198412] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T20:13:06.651358] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-05. [2025-10-20T20:13:06.877874] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T20:13:07.451395] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-07. [2025-10-20T20:13:08.249689] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T20:13:08.258858] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-03. [2025-10-20T20:13:08.559908] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T20:13:08.626561] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T20:13:09.051362] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-06. [2025-10-20T20:13:09.851398] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-04. [2025-10-20T20:13:10.662970] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-08. [2025-10-20T20:13:10.689847] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T20:13:10.942151] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:13:11.451403] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-04. [2025-10-20T20:13:11.676490] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T20:13:12.171533] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T20:13:12.251397] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-09. [2025-10-20T20:13:13.051437] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-05. [2025-10-20T20:13:13.185796] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:13:13.701949] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T20:13:13.851371] [INFO ] [REMOTE ] NMZ: Fetching primary data for 2025-10. [2025-10-20T20:13:14.651376] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-07. [2025-10-20T20:13:14.950790] [INFO ] [REMOTE ] NMZ: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T20:13:14.954037] [INFO ] [REMOTE ] NMZ: Full fetch complete (31104 records). [2025-10-20T20:13:14.954133] [DEBUG] [VALIDATOR ] NMZ: Validating primary data. [2025-10-20T20:13:15.368030] [DEBUG] [VALIDATOR ] NMZ: All expected trading days are present. [2025-10-20T20:13:15.368109] [DEBUG] [VALIDATOR ] NMZ: Primary data validation passed. [2025-10-20T20:13:15.368781] [INFO ] [MAIN ] NMZ: Fetched primary data is valid. [2025-10-20T20:13:15.368841] [DEBUG] [LOCAL ] NMZ: Saving primary data. [2025-10-20T20:13:15.369526] [DEBUG] [LOCAL ] NMZ: Trimmed 31104 records to 21584 (keeping 200 days of data) [2025-10-20T20:13:15.454738] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-05. [2025-10-20T20:13:15.500274] [DEBUG] [LOCAL ] NMZ: Primary data saved (21584 records). [2025-10-20T20:13:15.500417] [INFO ] [MAIN ] NMZ: Full history saved. [2025-10-20T20:13:15.500496] [INFO ] [MAIN ] Update Progress: 1500/2367 symbols completed. NMZ took 40.6s. Estimated remaining: 1h 51m 40s. Rate: 7.8 symbols/min [2025-10-20T20:13:15.684217] [INFO ] [MAIN ] NOAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:15.706349] [DEBUG] [LOCAL ] NOAH: Loaded 20519 existing records from primary storage. [2025-10-20T20:13:15.706619] [INFO ] [MAIN ] NOAH: Loaded 20519 existing records. [2025-10-20T20:13:15.901851] [WARN ] [MAIN ] NOAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:13:15.902055] [INFO ] [MAIN ] NOAH: Wiping existing primary data before full refresh. [2025-10-20T20:13:15.902101] [INFO ] [LOCAL ] NOAH: Starting deletion of primary data. [2025-10-20T20:13:15.902270] [DEBUG] [LOCAL ] NOAH: Deleted primary data file. [2025-10-20T20:13:15.902308] [INFO ] [MAIN ] NOAH: Starting full primary refresh. [2025-10-20T20:13:16.254842] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-06. [2025-10-20T20:13:16.254754] [INFO ] [REMOTE ] NOAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:16.554481] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T20:13:16.729034] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T20:13:16.881009] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:13:17.051421] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-06. [2025-10-20T20:13:17.851368] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-08. [2025-10-20T20:13:17.884675] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T20:13:18.651408] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-07. [2025-10-20T20:13:19.451366] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-01. [2025-10-20T20:13:19.524411] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T20:13:19.600768] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T20:13:20.251409] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-09. [2025-10-20T20:13:20.490590] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T20:13:21.051363] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-02. [2025-10-20T20:13:21.730969] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T20:13:21.778122] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T20:13:21.851362] [INFO ] [REMOTE ] NNN: Fetching primary data for 2025-10. [2025-10-20T20:13:22.651417] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-08. [2025-10-20T20:13:23.451357] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-03. [2025-10-20T20:13:23.482592] [INFO ] [REMOTE ] NNN: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T20:13:23.489590] [INFO ] [REMOTE ] NNN: Full fetch complete (70309 records). [2025-10-20T20:13:23.489730] [DEBUG] [VALIDATOR ] NNN: Validating primary data. [2025-10-20T20:13:23.884823] [DEBUG] [VALIDATOR ] NNN: All expected trading days are present. [2025-10-20T20:13:23.884989] [DEBUG] [VALIDATOR ] NNN: Primary data validation passed. [2025-10-20T20:13:23.886133] [INFO ] [MAIN ] NNN: Fetched primary data is valid. [2025-10-20T20:13:23.886216] [DEBUG] [LOCAL ] NNN: Saving primary data. [2025-10-20T20:13:23.906763] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T20:13:23.914129] [DEBUG] [LOCAL ] NNN: Trimmed 70309 records to 48230 (keeping 200 days of data) [2025-10-20T20:13:24.199165] [DEBUG] [LOCAL ] NNN: Primary data saved (48230 records). [2025-10-20T20:13:24.199295] [INFO ] [MAIN ] NNN: Full history saved. [2025-10-20T20:13:24.199599] [INFO ] [MAIN ] Update Progress: 1501/2367 symbols completed. NNN took 36.4s. Estimated remaining: 1h 51m 32s. Rate: 7.8 symbols/min [2025-10-20T20:13:24.261373] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-09. [2025-10-20T20:13:24.372568] [INFO ] [MAIN ] NOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:24.413482] [DEBUG] [LOCAL ] NOC: Loaded 42214 existing records from primary storage. [2025-10-20T20:13:24.413790] [INFO ] [MAIN ] NOC: Loaded 42214 existing records. [2025-10-20T20:13:24.549147] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T20:13:24.608264] [WARN ] [MAIN ] NOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:13:24.608506] [INFO ] [MAIN ] NOC: Wiping existing primary data before full refresh. [2025-10-20T20:13:24.608562] [INFO ] [LOCAL ] NOC: Starting deletion of primary data. [2025-10-20T20:13:24.608778] [DEBUG] [LOCAL ] NOC: Deleted primary data file. [2025-10-20T20:13:24.608851] [INFO ] [MAIN ] NOC: Starting full primary refresh. [2025-10-20T20:13:24.958060] [INFO ] [REMOTE ] NOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:25.051349] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-01. [2025-10-20T20:13:25.244175] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T20:13:25.851381] [INFO ] [REMOTE ] NOA: Fetching primary data for 2025-10. [2025-10-20T20:13:26.651404] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-07. [2025-10-20T20:13:26.747224] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:13:26.882753] [INFO ] [REMOTE ] NOA: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:13:26.885637] [INFO ] [REMOTE ] NOA: Full fetch complete (22586 records). [2025-10-20T20:13:26.885738] [DEBUG] [VALIDATOR ] NOA: Validating primary data. [2025-10-20T20:13:27.246913] [DEBUG] [VALIDATOR ] NOA: All expected trading days are present. [2025-10-20T20:13:27.247006] [DEBUG] [VALIDATOR ] NOA: Primary data validation passed. [2025-10-20T20:13:27.247465] [INFO ] [MAIN ] NOA: Fetched primary data is valid. [2025-10-20T20:13:27.247506] [DEBUG] [LOCAL ] NOA: Saving primary data. [2025-10-20T20:13:27.248078] [DEBUG] [LOCAL ] NOA: Trimmed 22586 records to 15970 (keeping 200 days of data) [2025-10-20T20:13:27.324973] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T20:13:27.349574] [DEBUG] [LOCAL ] NOA: Primary data saved (15970 records). [2025-10-20T20:13:27.349692] [INFO ] [MAIN ] NOA: Full history saved. [2025-10-20T20:13:27.349775] [INFO ] [MAIN ] Update Progress: 1502/2367 symbols completed. NOA took 28.2s. Estimated remaining: 1h 51m 22s. Rate: 7.8 symbols/min [2025-10-20T20:13:27.456497] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-02. [2025-10-20T20:13:27.519816] [INFO ] [MAIN ] NOG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:27.566538] [DEBUG] [LOCAL ] NOG: Loaded 52110 existing records from primary storage. [2025-10-20T20:13:27.566878] [INFO ] [MAIN ] NOG: Loaded 52110 existing records. [2025-10-20T20:13:27.740893] [WARN ] [MAIN ] NOG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:13:27.741142] [INFO ] [MAIN ] NOG: Wiping existing primary data before full refresh. [2025-10-20T20:13:27.741202] [INFO ] [LOCAL ] NOG: Starting deletion of primary data. [2025-10-20T20:13:27.741454] [DEBUG] [LOCAL ] NOG: Deleted primary data file. [2025-10-20T20:13:27.741518] [INFO ] [MAIN ] NOG: Starting full primary refresh. [2025-10-20T20:13:28.100696] [INFO ] [REMOTE ] NOG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:28.251332] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-01. [2025-10-20T20:13:29.051356] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-08. [2025-10-20T20:13:29.217459] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T20:13:29.613218] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T20:13:29.669392] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T20:13:29.851393] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-09. [2025-10-20T20:13:30.545902] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T20:13:30.651367] [INFO ] [REMOTE ] NNY: Fetching primary data for 2025-10. [2025-10-20T20:13:31.325584] [INFO ] [REMOTE ] NNY: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T20:13:31.326983] [INFO ] [REMOTE ] NNY: Full fetch complete (8303 records). [2025-10-20T20:13:31.327059] [DEBUG] [VALIDATOR ] NNY: Validating primary data. [2025-10-20T20:13:31.456497] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-04. [2025-10-20T20:13:31.718062] [DEBUG] [VALIDATOR ] NNY: All expected trading days are present. [2025-10-20T20:13:31.718202] [DEBUG] [VALIDATOR ] NNY: Primary data validation passed. [2025-10-20T20:13:31.718522] [INFO ] [MAIN ] NNY: Fetched primary data is valid. [2025-10-20T20:13:31.718593] [DEBUG] [LOCAL ] NNY: Saving primary data. [2025-10-20T20:13:31.719247] [DEBUG] [LOCAL ] NNY: Trimmed 8303 records to 5657 (keeping 200 days of data) [2025-10-20T20:13:31.755059] [DEBUG] [LOCAL ] NNY: Primary data saved (5657 records). [2025-10-20T20:13:31.755243] [INFO ] [MAIN ] NNY: Full history saved. [2025-10-20T20:13:31.755349] [INFO ] [MAIN ] Update Progress: 1503/2367 symbols completed. NNY took 36.3s. Estimated remaining: 1h 51m 12s. Rate: 7.8 symbols/min [2025-10-20T20:13:31.938074] [INFO ] [MAIN ] NOK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:31.991759] [DEBUG] [LOCAL ] NOK: Loaded 54402 existing records from primary storage. [2025-10-20T20:13:31.992193] [INFO ] [MAIN ] NOK: Loaded 54402 existing records. [2025-10-20T20:13:32.175894] [WARN ] [MAIN ] NOK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:13:32.176206] [INFO ] [MAIN ] NOK: Wiping existing primary data before full refresh. [2025-10-20T20:13:32.176258] [INFO ] [LOCAL ] NOK: Starting deletion of primary data. [2025-10-20T20:13:32.176496] [DEBUG] [LOCAL ] NOK: Deleted primary data file. [2025-10-20T20:13:32.176536] [INFO ] [MAIN ] NOK: Starting full primary refresh. [2025-10-20T20:13:32.256715] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-02. [2025-10-20T20:13:32.550370] [INFO ] [REMOTE ] NOK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:32.706223] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T20:13:33.051398] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-05. [2025-10-20T20:13:33.851415] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-03. [2025-10-20T20:13:33.953003] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:13:34.019839] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T20:13:34.651420] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-06. [2025-10-20T20:13:35.451378] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-03. [2025-10-20T20:13:35.742003] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T20:13:35.768558] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T20:13:36.251441] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-01. [2025-10-20T20:13:37.051559] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-04. [2025-10-20T20:13:37.267102] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T20:13:37.851385] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-04. [2025-10-20T20:13:38.348963] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T20:13:38.651453] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-07. [2025-10-20T20:13:38.934053] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T20:13:39.451356] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-02. [2025-10-20T20:13:39.563709] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T20:13:39.636425] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T20:13:40.251381] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-05. [2025-10-20T20:13:41.051432] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-08. [2025-10-20T20:13:41.132464] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T20:13:41.851365] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-03. [2025-10-20T20:13:42.090259] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T20:13:42.093092] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T20:13:42.651379] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-09. [2025-10-20T20:13:43.451349] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-05. [2025-10-20T20:13:43.740341] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:13:44.241177] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-03 (2.39 seconds). [2025-10-20T20:13:44.252071] [INFO ] [REMOTE ] NOAH: Fetching primary data for 2025-10. [2025-10-20T20:13:45.051386] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-06. [2025-10-20T20:13:45.094298] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T20:13:45.217758] [INFO ] [REMOTE ] NOAH: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T20:13:45.220911] [INFO ] [REMOTE ] NOAH: Full fetch complete (27362 records). [2025-10-20T20:13:45.220987] [DEBUG] [VALIDATOR ] NOAH: Validating primary data. [2025-10-20T20:13:45.580443] [DEBUG] [VALIDATOR ] NOAH: All expected trading days are present. [2025-10-20T20:13:45.580524] [DEBUG] [VALIDATOR ] NOAH: Primary data validation passed. [2025-10-20T20:13:45.581089] [INFO ] [MAIN ] NOAH: Fetched primary data is valid. [2025-10-20T20:13:45.581148] [DEBUG] [LOCAL ] NOAH: Saving primary data. [2025-10-20T20:13:45.581884] [DEBUG] [LOCAL ] NOAH: Trimmed 27362 records to 20177 (keeping 200 days of data) [2025-10-20T20:13:45.697202] [DEBUG] [LOCAL ] NOAH: Primary data saved (20177 records). [2025-10-20T20:13:45.697321] [INFO ] [MAIN ] NOAH: Full history saved. [2025-10-20T20:13:45.697399] [INFO ] [MAIN ] Update Progress: 1504/2367 symbols completed. NOAH took 30.2s. Estimated remaining: 1h 51m 8s. Rate: 7.8 symbols/min [2025-10-20T20:13:45.860354] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-04. [2025-10-20T20:13:45.863549] [INFO ] [MAIN ] NOM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:45.863788] [DEBUG] [LOCAL ] NOM: No existing data file found in primary storage. [2025-10-20T20:13:45.864515] [WARN ] [MAIN ] NOM: No existing data found, performing full refresh instead. [2025-10-20T20:13:45.864627] [INFO ] [MAIN ] NOM: Wiping existing primary data before full refresh. [2025-10-20T20:13:45.864701] [INFO ] [LOCAL ] NOM: Starting deletion of primary data. [2025-10-20T20:13:45.864776] [DEBUG] [LOCAL ] NOM: No primary data file found to delete. [2025-10-20T20:13:45.864810] [INFO ] [MAIN ] NOM: Starting full primary refresh. [2025-10-20T20:13:46.239891] [INFO ] [REMOTE ] NOM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:13:46.591237] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T20:13:46.651374] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-07. [2025-10-20T20:13:47.451406] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-01. [2025-10-20T20:13:48.004419] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T20:13:48.037792] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T20:13:48.251360] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-05. [2025-10-20T20:13:48.433008] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T20:13:49.051378] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-02. [2025-10-20T20:13:49.469473] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T20:13:49.851462] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-03. [2025-10-20T20:13:50.095307] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T20:13:50.360425] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-03 (0.51 seconds). [2025-10-20T20:13:50.651404] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-06. [2025-10-20T20:13:51.451392] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-06. [2025-10-20T20:13:52.251446] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-08. [2025-10-20T20:13:52.415157] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T20:13:53.051357] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-04. [2025-10-20T20:13:53.462790] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T20:13:53.481014] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T20:13:53.807347] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T20:13:53.851360] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-07. [2025-10-20T20:13:54.651417] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-09. [2025-10-20T20:13:55.451394] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-07. [2025-10-20T20:13:55.754028] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T20:13:56.205974] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T20:13:56.251339] [INFO ] [REMOTE ] NOC: Fetching primary data for 2025-10. [2025-10-20T20:13:57.051474] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-08. [2025-10-20T20:13:57.271787] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T20:13:57.851369] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-05. [2025-10-20T20:13:57.995384] [INFO ] [REMOTE ] NOC: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T20:13:58.001042] [INFO ] [REMOTE ] NOC: Full fetch complete (61868 records). [2025-10-20T20:13:58.001198] [DEBUG] [VALIDATOR ] NOC: Validating primary data. [2025-10-20T20:13:58.242163] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-10-20T20:13:58.412305] [DEBUG] [VALIDATOR ] NOC: All expected trading days are present. [2025-10-20T20:13:58.412561] [DEBUG] [VALIDATOR ] NOC: Primary data validation passed. [2025-10-20T20:13:58.414390] [INFO ] [MAIN ] NOC: Fetched primary data is valid. [2025-10-20T20:13:58.414500] [DEBUG] [LOCAL ] NOC: Saving primary data. [2025-10-20T20:13:58.416735] [DEBUG] [LOCAL ] NOC: Trimmed 61868 records to 41627 (keeping 200 days of data) [2025-10-20T20:13:58.572372] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T20:13:58.656558] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-08. [2025-10-20T20:13:58.751658] [DEBUG] [LOCAL ] NOC: Primary data saved (41627 records). [2025-10-20T20:13:58.751850] [INFO ] [MAIN ] NOC: Full history saved. [2025-10-20T20:13:58.752370] [INFO ] [MAIN ] Update Progress: 1505/2367 symbols completed. NOC took 34.6s. Estimated remaining: 1h 51m 4s. Rate: 7.8 symbols/min [2025-10-20T20:13:58.975942] [INFO ] [MAIN ] NOMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:13:59.026800] [DEBUG] [LOCAL ] NOMD: Loaded 43553 existing records from primary storage. [2025-10-20T20:13:59.027208] [INFO ] [MAIN ] NOMD: Loaded 43553 existing records. [2025-10-20T20:13:59.238144] [WARN ] [MAIN ] NOMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:13:59.238531] [INFO ] [MAIN ] NOMD: Wiping existing primary data before full refresh. [2025-10-20T20:13:59.238630] [INFO ] [LOCAL ] NOMD: Starting deletion of primary data. [2025-10-20T20:13:59.238911] [DEBUG] [LOCAL ] NOMD: Deleted primary data file. [2025-10-20T20:13:59.238956] [INFO ] [MAIN ] NOMD: Starting full primary refresh. [2025-10-20T20:13:59.462281] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-09. [2025-10-20T20:13:59.621404] [INFO ] [REMOTE ] NOMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:00.251388] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-06. [2025-10-20T20:14:00.699210] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T20:14:00.794258] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-08 (2.14 seconds). [2025-10-20T20:14:01.051391] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-07. [2025-10-20T20:14:01.172099] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:14:01.573071] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T20:14:01.851373] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-08. [2025-10-20T20:14:02.298408] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T20:14:02.651449] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-09. [2025-10-20T20:14:03.451385] [INFO ] [REMOTE ] NOG: Fetching primary data for 2025-10. [2025-10-20T20:14:04.251379] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-01. [2025-10-20T20:14:04.806001] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-09 (2.15 seconds). [2025-10-20T20:14:05.051421] [INFO ] [REMOTE ] NOK: Fetching primary data for 2025-10. [2025-10-20T20:14:05.311916] [INFO ] [REMOTE ] NOG: Incremental fetch complete for 2025-10 (1.86 seconds). [2025-10-20T20:14:05.319818] [INFO ] [REMOTE ] NOG: Full fetch complete (74267 records). [2025-10-20T20:14:05.319980] [DEBUG] [VALIDATOR ] NOG: Validating primary data. [2025-10-20T20:14:05.719145] [DEBUG] [VALIDATOR ] NOG: All expected trading days are present. [2025-10-20T20:14:05.719236] [DEBUG] [VALIDATOR ] NOG: Primary data validation passed. [2025-10-20T20:14:05.720239] [INFO ] [MAIN ] NOG: Fetched primary data is valid. [2025-10-20T20:14:05.720303] [DEBUG] [LOCAL ] NOG: Saving primary data. [2025-10-20T20:14:05.721736] [DEBUG] [LOCAL ] NOG: Trimmed 74267 records to 51365 (keeping 200 days of data) [2025-10-20T20:14:05.851426] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-09. [2025-10-20T20:14:05.976758] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T20:14:06.076882] [DEBUG] [LOCAL ] NOG: Primary data saved (51365 records). [2025-10-20T20:14:06.077013] [INFO ] [MAIN ] NOG: Full history saved. [2025-10-20T20:14:06.077090] [INFO ] [MAIN ] Update Progress: 1506/2367 symbols completed. NOG took 38.7s. Estimated remaining: 1h 50m 56s. Rate: 7.8 symbols/min [2025-10-20T20:14:06.247315] [INFO ] [MAIN ] NOTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:06.253708] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T20:14:06.286021] [DEBUG] [LOCAL ] NOTE: Loaded 40766 existing records from primary storage. [2025-10-20T20:14:06.286300] [INFO ] [MAIN ] NOTE: Loaded 40766 existing records. [2025-10-20T20:14:06.460702] [WARN ] [MAIN ] NOTE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:06.460901] [INFO ] [MAIN ] NOTE: Wiping existing primary data before full refresh. [2025-10-20T20:14:06.460949] [INFO ] [LOCAL ] NOTE: Starting deletion of primary data. [2025-10-20T20:14:06.461113] [DEBUG] [LOCAL ] NOTE: Deleted primary data file. [2025-10-20T20:14:06.461151] [INFO ] [MAIN ] NOTE: Starting full primary refresh. [2025-10-20T20:14:06.656471] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-02. [2025-10-20T20:14:06.813274] [INFO ] [REMOTE ] NOTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:07.141850] [INFO ] [REMOTE ] NOK: Incremental fetch complete for 2025-10 (2.09 seconds). [2025-10-20T20:14:07.150010] [INFO ] [REMOTE ] NOK: Full fetch complete (77774 records). [2025-10-20T20:14:07.150168] [DEBUG] [VALIDATOR ] NOK: Validating primary data. [2025-10-20T20:14:07.456466] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-01. [2025-10-20T20:14:07.545721] [DEBUG] [VALIDATOR ] NOK: All expected trading days are present. [2025-10-20T20:14:07.545803] [DEBUG] [VALIDATOR ] NOK: Primary data validation passed. [2025-10-20T20:14:07.546930] [INFO ] [MAIN ] NOK: Fetched primary data is valid. [2025-10-20T20:14:07.546994] [DEBUG] [LOCAL ] NOK: Saving primary data. [2025-10-20T20:14:07.548391] [DEBUG] [LOCAL ] NOK: Trimmed 77774 records to 53622 (keeping 200 days of data) [2025-10-20T20:14:07.854991] [DEBUG] [LOCAL ] NOK: Primary data saved (53622 records). [2025-10-20T20:14:07.855121] [INFO ] [MAIN ] NOK: Full history saved. [2025-10-20T20:14:07.855195] [INFO ] [MAIN ] Update Progress: 1507/2367 symbols completed. NOK took 36.1s. Estimated remaining: 1h 50m 45s. Rate: 7.8 symbols/min [2025-10-20T20:14:08.065841] [INFO ] [MAIN ] NOV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:08.087737] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:14:08.144165] [DEBUG] [LOCAL ] NOV: Loaded 53950 existing records from primary storage. [2025-10-20T20:14:08.144594] [INFO ] [MAIN ] NOV: Loaded 53950 existing records. [2025-10-20T20:14:08.256496] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-03. [2025-10-20T20:14:08.324809] [WARN ] [MAIN ] NOV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:08.325035] [INFO ] [MAIN ] NOV: Wiping existing primary data before full refresh. [2025-10-20T20:14:08.325090] [INFO ] [LOCAL ] NOV: Starting deletion of primary data. [2025-10-20T20:14:08.325597] [DEBUG] [LOCAL ] NOV: Deleted primary data file. [2025-10-20T20:14:08.325680] [INFO ] [MAIN ] NOV: Starting full primary refresh. [2025-10-20T20:14:08.734880] [INFO ] [REMOTE ] NOV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:09.051409] [INFO ] [REMOTE ] NOM: Fetching primary data for 2025-10. [2025-10-20T20:14:09.472045] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T20:14:09.529879] [INFO ] [REMOTE ] NOM: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T20:14:09.530606] [INFO ] [REMOTE ] NOM: Full fetch complete (1553 records). [2025-10-20T20:14:09.530718] [DEBUG] [VALIDATOR ] NOM: Validating primary data. [2025-10-20T20:14:09.713530] [WARN ] [VALIDATOR ] NOM: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T20:14:09.713784] [FATAL] [MAIN ] NOM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:14:09.713858] [WARN ] [MAIN ] NOM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:14:09.714096] [INFO ] [LOCAL ] NOM: Starting deletion of primary data. [2025-10-20T20:14:09.714186] [DEBUG] [LOCAL ] NOM: No primary data file found to delete. [2025-10-20T20:14:09.714317] [INFO ] [LOCAL ] NOM: Starting deletion of derivative data. [2025-10-20T20:14:09.714485] [DEBUG] [LOCAL ] NOM: No derivative data file found to delete. [2025-10-20T20:14:09.714561] [INFO ] [MAIN ] Update Progress: 1508/2367 symbols completed. NOM took 24.0s. Estimated remaining: 1h 50m 34s. Rate: 7.8 symbols/min [2025-10-20T20:14:09.860982] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-02. [2025-10-20T20:14:09.911348] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T20:14:09.909867] [INFO ] [MAIN ] NOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:09.965299] [DEBUG] [LOCAL ] NOW: Loaded 48724 existing records from primary storage. [2025-10-20T20:14:09.965633] [INFO ] [MAIN ] NOW: Loaded 48724 existing records. [2025-10-20T20:14:10.140994] [WARN ] [MAIN ] NOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:10.141203] [INFO ] [MAIN ] NOW: Wiping existing primary data before full refresh. [2025-10-20T20:14:10.141249] [INFO ] [LOCAL ] NOW: Starting deletion of primary data. [2025-10-20T20:14:10.141464] [DEBUG] [LOCAL ] NOW: Deleted primary data file. [2025-10-20T20:14:10.141519] [INFO ] [MAIN ] NOW: Starting full primary refresh. [2025-10-20T20:14:10.489989] [INFO ] [REMOTE ] NOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:10.651343] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-01. [2025-10-20T20:14:11.451406] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-01. [2025-10-20T20:14:11.742264] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T20:14:12.251433] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-03. [2025-10-20T20:14:12.717329] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T20:14:13.051389] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-04. [2025-10-20T20:14:13.744854] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-01 (2.29 seconds). [2025-10-20T20:14:13.851359] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-02. [2025-10-20T20:14:14.073539] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T20:14:14.599849] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T20:14:14.651338] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-05. [2025-10-20T20:14:15.448359] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T20:14:15.457424] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-02. [2025-10-20T20:14:16.251372] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-04. [2025-10-20T20:14:16.313266] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T20:14:17.051596] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-06. [2025-10-20T20:14:17.486104] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T20:14:17.852456] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T20:14:17.851689] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-03. [2025-10-20T20:14:18.256373] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T20:14:18.651370] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-05. [2025-10-20T20:14:19.451417] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-07. [2025-10-20T20:14:19.683898] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T20:14:20.251423] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-03. [2025-10-20T20:14:20.375776] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:14:20.858756] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T20:14:21.051501] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-08. [2025-10-20T20:14:21.851356] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-04. [2025-10-20T20:14:22.103324] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T20:14:22.513497] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T20:14:22.651372] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-09. [2025-10-20T20:14:23.451348] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-06. [2025-10-20T20:14:23.772111] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T20:14:24.255217] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-05. [2025-10-20T20:14:24.267431] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T20:14:25.051351] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-04. [2025-10-20T20:14:25.161183] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T20:14:25.851354] [INFO ] [REMOTE ] NOMD: Fetching primary data for 2025-10. [2025-10-20T20:14:25.925221] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T20:14:26.651429] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-06. [2025-10-20T20:14:27.350336] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-10-20T20:14:27.383247] [INFO ] [REMOTE ] NOMD: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T20:14:27.390977] [INFO ] [REMOTE ] NOMD: Full fetch complete (60544 records). [2025-10-20T20:14:27.391090] [DEBUG] [VALIDATOR ] NOMD: Validating primary data. [2025-10-20T20:14:27.456493] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-05. [2025-10-20T20:14:27.777817] [DEBUG] [VALIDATOR ] NOMD: All expected trading days are present. [2025-10-20T20:14:27.777902] [DEBUG] [VALIDATOR ] NOMD: Primary data validation passed. [2025-10-20T20:14:27.778932] [INFO ] [MAIN ] NOMD: Fetched primary data is valid. [2025-10-20T20:14:27.778996] [DEBUG] [LOCAL ] NOMD: Saving primary data. [2025-10-20T20:14:27.780225] [DEBUG] [LOCAL ] NOMD: Trimmed 60544 records to 43107 (keeping 200 days of data) [2025-10-20T20:14:28.037525] [DEBUG] [LOCAL ] NOMD: Primary data saved (43107 records). [2025-10-20T20:14:28.037669] [INFO ] [MAIN ] NOMD: Full history saved. [2025-10-20T20:14:28.038013] [INFO ] [MAIN ] Update Progress: 1509/2367 symbols completed. NOMD took 29.3s. Estimated remaining: 1h 50m 32s. Rate: 7.8 symbols/min [2025-10-20T20:14:28.207381] [INFO ] [MAIN ] NP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:28.207691] [DEBUG] [LOCAL ] NP: No existing data file found in primary storage. [2025-10-20T20:14:28.207748] [WARN ] [MAIN ] NP: No existing data found, performing full refresh instead. [2025-10-20T20:14:28.207960] [INFO ] [MAIN ] NP: Wiping existing primary data before full refresh. [2025-10-20T20:14:28.208118] [INFO ] [LOCAL ] NP: Starting deletion of primary data. [2025-10-20T20:14:28.208261] [DEBUG] [LOCAL ] NP: No primary data file found to delete. [2025-10-20T20:14:28.208352] [INFO ] [MAIN ] NP: Starting full primary refresh. [2025-10-20T20:14:28.271353] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-07. [2025-10-20T20:14:28.550292] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T20:14:28.614336] [INFO ] [REMOTE ] NP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:29.051444] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-07. [2025-10-20T20:14:29.706275] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-05 (2.25 seconds). [2025-10-20T20:14:29.851370] [INFO ] [REMOTE ] NP: Fetching primary data for 2025-01. [2025-10-20T20:14:30.044261] [DEBUG] [REMOTE ] NP: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:14:30.044628] [WARN ] [REMOTE ] NP: No data returned for month 2025-01 [2025-10-20T20:14:30.044749] [FATAL] [REMOTE ] NP: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:14:30.044830] [FATAL] [MAIN ] NP: Failed to fetch primary data. [2025-10-20T20:14:30.044906] [WARN ] [MAIN ] NP: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:14:30.044991] [INFO ] [LOCAL ] NP: Starting deletion of primary data. [2025-10-20T20:14:30.045069] [DEBUG] [LOCAL ] NP: No primary data file found to delete. [2025-10-20T20:14:30.045119] [INFO ] [LOCAL ] NP: Starting deletion of derivative data. [2025-10-20T20:14:30.045176] [DEBUG] [LOCAL ] NP: No derivative data file found to delete. [2025-10-20T20:14:30.045236] [INFO ] [MAIN ] Update Progress: 1510/2367 symbols completed. NP took 2.0s. Estimated remaining: 1h 50m 21s. Rate: 7.8 symbols/min [2025-10-20T20:14:30.234039] [INFO ] [MAIN ] NPB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:30.250231] [DEBUG] [LOCAL ] NPB: No existing data file found in primary storage. [2025-10-20T20:14:30.250311] [WARN ] [MAIN ] NPB: No existing data found, performing full refresh instead. [2025-10-20T20:14:30.263061] [INFO ] [MAIN ] NPB: Wiping existing primary data before full refresh. [2025-10-20T20:14:30.263175] [INFO ] [LOCAL ] NPB: Starting deletion of primary data. [2025-10-20T20:14:30.263235] [DEBUG] [LOCAL ] NPB: No primary data file found to delete. [2025-10-20T20:14:30.263479] [INFO ] [MAIN ] NPB: Starting full primary refresh. [2025-10-20T20:14:30.309388] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T20:14:30.639514] [INFO ] [REMOTE ] NPB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:30.657313] [INFO ] [REMOTE ] NPB: Fetching primary data for 2025-01. [2025-10-20T20:14:30.671131] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T20:14:30.883938] [DEBUG] [REMOTE ] NPB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:14:30.884257] [WARN ] [REMOTE ] NPB: No data returned for month 2025-01 [2025-10-20T20:14:30.884361] [FATAL] [REMOTE ] NPB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:14:30.884454] [FATAL] [MAIN ] NPB: Failed to fetch primary data. [2025-10-20T20:14:30.884534] [WARN ] [MAIN ] NPB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:14:30.884621] [INFO ] [LOCAL ] NPB: Starting deletion of primary data. [2025-10-20T20:14:30.884734] [DEBUG] [LOCAL ] NPB: No primary data file found to delete. [2025-10-20T20:14:30.884781] [INFO ] [LOCAL ] NPB: Starting deletion of derivative data. [2025-10-20T20:14:30.884824] [DEBUG] [LOCAL ] NPB: No derivative data file found to delete. [2025-10-20T20:14:30.884895] [INFO ] [MAIN ] Update Progress: 1511/2367 symbols completed. NPB took 0.8s. Estimated remaining: 1h 50m 9s. Rate: 7.8 symbols/min [2025-10-20T20:14:31.052486] [INFO ] [MAIN ] NPCT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:31.072454] [DEBUG] [LOCAL ] NPCT: Loaded 16917 existing records from primary storage. [2025-10-20T20:14:31.072745] [INFO ] [MAIN ] NPCT: Loaded 16917 existing records. [2025-10-20T20:14:31.251345] [WARN ] [MAIN ] NPCT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:31.251606] [INFO ] [MAIN ] NPCT: Wiping existing primary data before full refresh. [2025-10-20T20:14:31.251684] [INFO ] [LOCAL ] NPCT: Starting deletion of primary data. [2025-10-20T20:14:31.251871] [DEBUG] [LOCAL ] NPCT: Deleted primary data file. [2025-10-20T20:14:31.251910] [INFO ] [MAIN ] NPCT: Starting full primary refresh. [2025-10-20T20:14:31.456506] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-08. [2025-10-20T20:14:31.606876] [INFO ] [REMOTE ] NPCT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:32.251395] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-06. [2025-10-20T20:14:33.001762] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T20:14:33.051335] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-09. [2025-10-20T20:14:33.851371] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-01. [2025-10-20T20:14:34.118456] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T20:14:34.354601] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T20:14:34.651378] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-08. [2025-10-20T20:14:34.953759] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T20:14:35.451399] [INFO ] [REMOTE ] NOTE: Fetching primary data for 2025-10. [2025-10-20T20:14:36.251364] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-02. [2025-10-20T20:14:36.418072] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T20:14:36.759874] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T20:14:36.823260] [INFO ] [REMOTE ] NOTE: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T20:14:36.828399] [INFO ] [REMOTE ] NOTE: Full fetch complete (61491 records). [2025-10-20T20:14:36.828506] [DEBUG] [VALIDATOR ] NOTE: Validating primary data. [2025-10-20T20:14:37.056602] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-07. [2025-10-20T20:14:37.212749] [DEBUG] [VALIDATOR ] NOTE: All expected trading days are present. [2025-10-20T20:14:37.212838] [DEBUG] [VALIDATOR ] NOTE: Primary data validation passed. [2025-10-20T20:14:37.213778] [INFO ] [MAIN ] NOTE: Fetched primary data is valid. [2025-10-20T20:14:37.213841] [DEBUG] [LOCAL ] NOTE: Saving primary data. [2025-10-20T20:14:37.214968] [DEBUG] [LOCAL ] NOTE: Trimmed 61491 records to 39914 (keeping 200 days of data) [2025-10-20T20:14:37.440223] [DEBUG] [LOCAL ] NOTE: Primary data saved (39914 records). [2025-10-20T20:14:37.440355] [INFO ] [MAIN ] NOTE: Full history saved. [2025-10-20T20:14:37.440432] [INFO ] [MAIN ] Update Progress: 1512/2367 symbols completed. NOTE took 31.4s. Estimated remaining: 1h 50m 1s. Rate: 7.8 symbols/min [2025-10-20T20:14:37.605675] [INFO ] [MAIN ] NPFD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:37.617372] [DEBUG] [LOCAL ] NPFD: Loaded 10224 existing records from primary storage. [2025-10-20T20:14:37.617571] [INFO ] [MAIN ] NPFD: Loaded 10224 existing records. [2025-10-20T20:14:37.789829] [WARN ] [MAIN ] NPFD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:37.790083] [INFO ] [MAIN ] NPFD: Wiping existing primary data before full refresh. [2025-10-20T20:14:37.790130] [INFO ] [LOCAL ] NPFD: Starting deletion of primary data. [2025-10-20T20:14:37.790270] [DEBUG] [LOCAL ] NPFD: Deleted primary data file. [2025-10-20T20:14:37.790307] [INFO ] [MAIN ] NPFD: Starting full primary refresh. [2025-10-20T20:14:37.856610] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-03. [2025-10-20T20:14:38.149185] [INFO ] [REMOTE ] NPFD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:38.651369] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-01. [2025-10-20T20:14:38.884406] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:14:39.338604] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-07 (2.28 seconds). [2025-10-20T20:14:39.386332] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T20:14:39.451356] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-02. [2025-10-20T20:14:40.191770] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T20:14:40.251345] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-04. [2025-10-20T20:14:41.051349] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-03. [2025-10-20T20:14:41.209985] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T20:14:41.829108] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T20:14:41.851358] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-09. [2025-10-20T20:14:42.651452] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-04. [2025-10-20T20:14:43.451384] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-05. [2025-10-20T20:14:43.501238] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T20:14:43.643764] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T20:14:44.080241] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T20:14:44.251379] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-06. [2025-10-20T20:14:45.051435] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-08. [2025-10-20T20:14:45.295733] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T20:14:45.851365] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-07. [2025-10-20T20:14:46.651354] [INFO ] [REMOTE ] NOV: Fetching primary data for 2025-10. [2025-10-20T20:14:46.834828] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-07 (0.98 seconds). [2025-10-20T20:14:47.231220] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T20:14:47.451380] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-09. [2025-10-20T20:14:48.251416] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-05. [2025-10-20T20:14:48.453153] [INFO ] [REMOTE ] NOV: Incremental fetch complete for 2025-10 (1.80 seconds). [2025-10-20T20:14:48.461936] [INFO ] [REMOTE ] NOV: Full fetch complete (77164 records). [2025-10-20T20:14:48.462204] [DEBUG] [VALIDATOR ] NOV: Validating primary data. [2025-10-20T20:14:48.852279] [DEBUG] [VALIDATOR ] NOV: All expected trading days are present. [2025-10-20T20:14:48.852364] [DEBUG] [VALIDATOR ] NOV: Primary data validation passed. [2025-10-20T20:14:48.853467] [INFO ] [MAIN ] NOV: Fetched primary data is valid. [2025-10-20T20:14:48.853530] [DEBUG] [LOCAL ] NOV: Saving primary data. [2025-10-20T20:14:48.854984] [DEBUG] [LOCAL ] NOV: Trimmed 77164 records to 53173 (keeping 200 days of data) [2025-10-20T20:14:48.969116] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T20:14:49.059963] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-06. [2025-10-20T20:14:49.220378] [DEBUG] [LOCAL ] NOV: Primary data saved (53173 records). [2025-10-20T20:14:49.220564] [INFO ] [MAIN ] NOV: Full history saved. [2025-10-20T20:14:49.220716] [INFO ] [MAIN ] Update Progress: 1513/2367 symbols completed. NOV took 41.4s. Estimated remaining: 1h 49m 55s. Rate: 7.8 symbols/min [2025-10-20T20:14:49.447883] [INFO ] [MAIN ] NPK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:49.486715] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:14:49.487106] [DEBUG] [LOCAL ] NPK: Loaded 8568 existing records from primary storage. [2025-10-20T20:14:49.487575] [INFO ] [MAIN ] NPK: Loaded 8568 existing records. [2025-10-20T20:14:49.696353] [WARN ] [MAIN ] NPK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:49.696975] [INFO ] [MAIN ] NPK: Wiping existing primary data before full refresh. [2025-10-20T20:14:49.697033] [INFO ] [LOCAL ] NPK: Starting deletion of primary data. [2025-10-20T20:14:49.697318] [DEBUG] [LOCAL ] NPK: Deleted primary data file. [2025-10-20T20:14:49.697375] [INFO ] [MAIN ] NPK: Starting full primary refresh. [2025-10-20T20:14:49.859272] [INFO ] [REMOTE ] NOW: Fetching primary data for 2025-10. [2025-10-20T20:14:49.956207] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T20:14:50.071860] [INFO ] [REMOTE ] NPK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:50.651392] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-08. [2025-10-20T20:14:50.737207] [INFO ] [REMOTE ] NOW: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T20:14:50.744896] [INFO ] [REMOTE ] NOW: Full fetch complete (70762 records). [2025-10-20T20:14:50.744995] [DEBUG] [VALIDATOR ] NOW: Validating primary data. [2025-10-20T20:14:51.130468] [DEBUG] [VALIDATOR ] NOW: All expected trading days are present. [2025-10-20T20:14:51.130545] [DEBUG] [VALIDATOR ] NOW: Primary data validation passed. [2025-10-20T20:14:51.131401] [INFO ] [MAIN ] NOW: Fetched primary data is valid. [2025-10-20T20:14:51.131496] [DEBUG] [LOCAL ] NOW: Saving primary data. [2025-10-20T20:14:51.132833] [DEBUG] [LOCAL ] NOW: Trimmed 70762 records to 47924 (keeping 200 days of data) [2025-10-20T20:14:51.417857] [DEBUG] [LOCAL ] NOW: Primary data saved (47924 records). [2025-10-20T20:14:51.417980] [INFO ] [MAIN ] NOW: Full history saved. [2025-10-20T20:14:51.418054] [INFO ] [MAIN ] Update Progress: 1514/2367 symbols completed. NOW took 41.7s. Estimated remaining: 1h 49m 45s. Rate: 7.8 symbols/min [2025-10-20T20:14:51.460923] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-01. [2025-10-20T20:14:51.599293] [INFO ] [MAIN ] NPKI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:51.597139] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T20:14:51.638565] [DEBUG] [LOCAL ] NPKI: Loaded 40905 existing records from primary storage. [2025-10-20T20:14:51.638865] [INFO ] [MAIN ] NPKI: Loaded 40905 existing records. [2025-10-20T20:14:51.813791] [WARN ] [MAIN ] NPKI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:51.813964] [INFO ] [MAIN ] NPKI: Wiping existing primary data before full refresh. [2025-10-20T20:14:51.814012] [INFO ] [LOCAL ] NPKI: Starting deletion of primary data. [2025-10-20T20:14:51.814216] [DEBUG] [LOCAL ] NPKI: Deleted primary data file. [2025-10-20T20:14:51.814279] [INFO ] [MAIN ] NPKI: Starting full primary refresh. [2025-10-20T20:14:52.160064] [INFO ] [REMOTE ] NPKI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:52.194775] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T20:14:52.251326] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-01. [2025-10-20T20:14:53.051429] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-09. [2025-10-20T20:14:53.494711] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T20:14:53.851397] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-07. [2025-10-20T20:14:54.041132] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T20:14:54.651352] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-02. [2025-10-20T20:14:54.670402] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T20:14:55.451396] [INFO ] [REMOTE ] NPCT: Fetching primary data for 2025-10. [2025-10-20T20:14:56.072626] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T20:14:56.251346] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-03. [2025-10-20T20:14:56.326249] [INFO ] [REMOTE ] NPCT: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T20:14:56.328477] [INFO ] [REMOTE ] NPCT: Full fetch complete (24543 records). [2025-10-20T20:14:56.328548] [DEBUG] [VALIDATOR ] NPCT: Validating primary data. [2025-10-20T20:14:56.681191] [DEBUG] [VALIDATOR ] NPCT: All expected trading days are present. [2025-10-20T20:14:56.681269] [DEBUG] [VALIDATOR ] NPCT: Primary data validation passed. [2025-10-20T20:14:56.681794] [INFO ] [MAIN ] NPCT: Fetched primary data is valid. [2025-10-20T20:14:56.681856] [DEBUG] [LOCAL ] NPCT: Saving primary data. [2025-10-20T20:14:56.682451] [DEBUG] [LOCAL ] NPCT: Trimmed 24543 records to 16647 (keeping 200 days of data) [2025-10-20T20:14:56.780826] [DEBUG] [LOCAL ] NPCT: Primary data saved (16647 records). [2025-10-20T20:14:56.780957] [INFO ] [MAIN ] NPCT: Full history saved. [2025-10-20T20:14:56.781031] [INFO ] [MAIN ] Update Progress: 1515/2367 symbols completed. NPCT took 25.9s. Estimated remaining: 1h 49m 36s. Rate: 7.8 symbols/min [2025-10-20T20:14:56.947619] [INFO ] [MAIN ] NPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:14:56.963015] [DEBUG] [LOCAL ] NPO: Loaded 13896 existing records from primary storage. [2025-10-20T20:14:56.963214] [INFO ] [MAIN ] NPO: Loaded 13896 existing records. [2025-10-20T20:14:57.056717] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-08. [2025-10-20T20:14:57.147417] [WARN ] [MAIN ] NPO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:14:57.147691] [INFO ] [MAIN ] NPO: Wiping existing primary data before full refresh. [2025-10-20T20:14:57.147760] [INFO ] [LOCAL ] NPO: Starting deletion of primary data. [2025-10-20T20:14:57.148373] [DEBUG] [LOCAL ] NPO: Deleted primary data file. [2025-10-20T20:14:57.148430] [INFO ] [MAIN ] NPO: Starting full primary refresh. [2025-10-20T20:14:57.496269] [INFO ] [REMOTE ] NPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:14:57.623180] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T20:14:57.829751] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:14:57.851343] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-01. [2025-10-20T20:14:58.651340] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-02. [2025-10-20T20:14:58.668256] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T20:14:59.277220] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T20:14:59.451349] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-03. [2025-10-20T20:15:00.179574] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T20:15:00.251429] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-02. [2025-10-20T20:15:01.051363] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-04. [2025-10-20T20:15:01.084557] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T20:15:01.851451] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-09. [2025-10-20T20:15:02.651368] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-03. [2025-10-20T20:15:02.729466] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:15:02.876913] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:15:03.451370] [INFO ] [REMOTE ] NPFD: Fetching primary data for 2025-10. [2025-10-20T20:15:03.516478] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:15:04.251453] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-04. [2025-10-20T20:15:04.391931] [INFO ] [REMOTE ] NPFD: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T20:15:04.394052] [INFO ] [REMOTE ] NPFD: Full fetch complete (14563 records). [2025-10-20T20:15:04.394119] [DEBUG] [VALIDATOR ] NPFD: Validating primary data. [2025-10-20T20:15:04.745554] [DEBUG] [VALIDATOR ] NPFD: All expected trading days are present. [2025-10-20T20:15:04.745626] [DEBUG] [VALIDATOR ] NPFD: Primary data validation passed. [2025-10-20T20:15:04.746000] [INFO ] [MAIN ] NPFD: Fetched primary data is valid. [2025-10-20T20:15:04.746050] [DEBUG] [LOCAL ] NPFD: Saving primary data. [2025-10-20T20:15:04.746511] [DEBUG] [LOCAL ] NPFD: Trimmed 14563 records to 10026 (keeping 200 days of data) [2025-10-20T20:15:04.806950] [DEBUG] [LOCAL ] NPFD: Primary data saved (10026 records). [2025-10-20T20:15:04.807087] [INFO ] [MAIN ] NPFD: Full history saved. [2025-10-20T20:15:04.807170] [INFO ] [MAIN ] Update Progress: 1516/2367 symbols completed. NPFD took 27.4s. Estimated remaining: 1h 49m 28s. Rate: 7.8 symbols/min [2025-10-20T20:15:04.976213] [INFO ] [MAIN ] NPV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:04.984128] [DEBUG] [LOCAL ] NPV: Loaded 5437 existing records from primary storage. [2025-10-20T20:15:04.984305] [INFO ] [MAIN ] NPV: Loaded 5437 existing records. [2025-10-20T20:15:05.056535] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-05. [2025-10-20T20:15:05.160170] [WARN ] [MAIN ] NPV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:05.160336] [INFO ] [MAIN ] NPV: Wiping existing primary data before full refresh. [2025-10-20T20:15:05.160381] [INFO ] [LOCAL ] NPV: Starting deletion of primary data. [2025-10-20T20:15:05.160521] [DEBUG] [LOCAL ] NPV: Deleted primary data file. [2025-10-20T20:15:05.160559] [INFO ] [MAIN ] NPV: Starting full primary refresh. [2025-10-20T20:15:05.445902] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T20:15:05.528078] [INFO ] [REMOTE ] NPV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:05.851371] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-05. [2025-10-20T20:15:06.651419] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-04. [2025-10-20T20:15:06.680230] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:15:06.727452] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T20:15:07.205665] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T20:15:07.451418] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-06. [2025-10-20T20:15:08.251372] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-01. [2025-10-20T20:15:08.316333] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T20:15:08.916325] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-10-20T20:15:09.051403] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-05. [2025-10-20T20:15:09.777519] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T20:15:09.851418] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-06. [2025-10-20T20:15:10.651427] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-07. [2025-10-20T20:15:11.336889] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T20:15:11.451364] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-07. [2025-10-20T20:15:11.527669] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T20:15:12.251390] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-06. [2025-10-20T20:15:13.051401] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-02. [2025-10-20T20:15:13.150544] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T20:15:13.166590] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T20:15:13.604415] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T20:15:13.851362] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-07. [2025-10-20T20:15:14.651354] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-08. [2025-10-20T20:15:14.812456] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T20:15:15.451433] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-08. [2025-10-20T20:15:16.159733] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:15:16.226372] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:15:16.251354] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-08. [2025-10-20T20:15:17.019907] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:15:17.051341] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-09. [2025-10-20T20:15:17.752045] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T20:15:17.851351] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-09. [2025-10-20T20:15:18.494092] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T20:15:18.651431] [INFO ] [REMOTE ] NPK: Fetching primary data for 2025-10. [2025-10-20T20:15:19.451385] [INFO ] [REMOTE ] NPO: Fetching primary data for 2025-10. [2025-10-20T20:15:19.523424] [INFO ] [REMOTE ] NPK: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T20:15:19.526130] [INFO ] [REMOTE ] NPK: Full fetch complete (11141 records). [2025-10-20T20:15:19.526254] [DEBUG] [VALIDATOR ] NPK: Validating primary data. [2025-10-20T20:15:19.965989] [DEBUG] [VALIDATOR ] NPK: All expected trading days are present. [2025-10-20T20:15:19.966095] [DEBUG] [VALIDATOR ] NPK: Primary data validation passed. [2025-10-20T20:15:19.966525] [INFO ] [MAIN ] NPK: Fetched primary data is valid. [2025-10-20T20:15:19.966579] [DEBUG] [LOCAL ] NPK: Saving primary data. [2025-10-20T20:15:19.967290] [DEBUG] [LOCAL ] NPK: Trimmed 11141 records to 8500 (keeping 200 days of data) [2025-10-20T20:15:20.026973] [DEBUG] [LOCAL ] NPK: Primary data saved (8500 records). [2025-10-20T20:15:20.027196] [INFO ] [MAIN ] NPK: Full history saved. [2025-10-20T20:15:20.027306] [INFO ] [MAIN ] Update Progress: 1517/2367 symbols completed. NPK took 30.8s. Estimated remaining: 1h 49m 24s. Rate: 7.8 symbols/min [2025-10-20T20:15:20.251375] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-03. [2025-10-20T20:15:20.565176] [INFO ] [REMOTE ] NPO: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T20:15:20.568283] [INFO ] [REMOTE ] NPO: Full fetch complete (18815 records). [2025-10-20T20:15:20.568374] [DEBUG] [VALIDATOR ] NPO: Validating primary data. [2025-10-20T20:15:20.930047] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-10-20T20:15:20.956265] [DEBUG] [VALIDATOR ] NPO: All expected trading days are present. [2025-10-20T20:15:20.956353] [DEBUG] [VALIDATOR ] NPO: Primary data validation passed. [2025-10-20T20:15:20.957420] [INFO ] [MAIN ] NPO: Fetched primary data is valid. [2025-10-20T20:15:20.957473] [DEBUG] [LOCAL ] NPO: Saving primary data. [2025-10-20T20:15:20.958024] [DEBUG] [LOCAL ] NPO: Trimmed 18815 records to 13626 (keeping 200 days of data) [2025-10-20T20:15:21.042810] [DEBUG] [LOCAL ] NPO: Primary data saved (13626 records). [2025-10-20T20:15:21.042939] [INFO ] [MAIN ] NPO: Full history saved. [2025-10-20T20:15:21.043015] [INFO ] [MAIN ] Update Progress: 1518/2367 symbols completed. NPO took 24.3s. Estimated remaining: 1h 49m 13s. Rate: 7.8 symbols/min [2025-10-20T20:15:21.051338] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-09. [2025-10-20T20:15:21.851358] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-04. [2025-10-20T20:15:22.407531] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T20:15:22.465164] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T20:15:22.651390] [INFO ] [REMOTE ] NPKI: Fetching primary data for 2025-10. [2025-10-20T20:15:23.451401] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-05. [2025-10-20T20:15:24.087388] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T20:15:24.156335] [INFO ] [REMOTE ] NPKI: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T20:15:24.162404] [INFO ] [REMOTE ] NPKI: Full fetch complete (56213 records). [2025-10-20T20:15:24.162496] [DEBUG] [VALIDATOR ] NPKI: Validating primary data. [2025-10-20T20:15:24.261204] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-06. [2025-10-20T20:15:24.553984] [DEBUG] [VALIDATOR ] NPKI: All expected trading days are present. [2025-10-20T20:15:24.554075] [DEBUG] [VALIDATOR ] NPKI: Primary data validation passed. [2025-10-20T20:15:24.554960] [INFO ] [MAIN ] NPKI: Fetched primary data is valid. [2025-10-20T20:15:24.555025] [DEBUG] [LOCAL ] NPKI: Saving primary data. [2025-10-20T20:15:24.556231] [DEBUG] [LOCAL ] NPKI: Trimmed 56213 records to 40492 (keeping 200 days of data) [2025-10-20T20:15:24.787565] [DEBUG] [LOCAL ] NPKI: Primary data saved (40492 records). [2025-10-20T20:15:24.787811] [INFO ] [MAIN ] NPKI: Full history saved. [2025-10-20T20:15:24.787907] [INFO ] [MAIN ] Update Progress: 1519/2367 symbols completed. NPKI took 33.4s. Estimated remaining: 1h 49m 3s. Rate: 7.8 symbols/min [2025-10-20T20:15:24.857534] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T20:15:25.051412] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-07. [2025-10-20T20:15:25.700322] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T20:15:25.851355] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-08. [2025-10-20T20:15:26.538929] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T20:15:26.651360] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-09. [2025-10-20T20:15:27.138405] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T20:15:27.451431] [INFO ] [REMOTE ] NPV: Fetching primary data for 2025-10. [2025-10-20T20:15:28.307164] [INFO ] [REMOTE ] NPV: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:15:28.308753] [INFO ] [REMOTE ] NPV: Full fetch complete (7166 records). [2025-10-20T20:15:28.308820] [DEBUG] [VALIDATOR ] NPV: Validating primary data. [2025-10-20T20:15:28.669197] [DEBUG] [VALIDATOR ] NPV: All expected trading days are present. [2025-10-20T20:15:28.669290] [DEBUG] [VALIDATOR ] NPV: Primary data validation passed. [2025-10-20T20:15:28.669489] [INFO ] [MAIN ] NPV: Fetched primary data is valid. [2025-10-20T20:15:28.669527] [DEBUG] [LOCAL ] NPV: Saving primary data. [2025-10-20T20:15:28.669944] [DEBUG] [LOCAL ] NPV: Trimmed 7166 records to 5369 (keeping 200 days of data) [2025-10-20T20:15:28.701568] [DEBUG] [LOCAL ] NPV: Primary data saved (5369 records). [2025-10-20T20:15:28.701694] [INFO ] [MAIN ] NPV: Full history saved. [2025-10-20T20:15:28.701782] [INFO ] [MAIN ] Update Progress: 1520/2367 symbols completed. NPV took 23.9s. Estimated remaining: 1h 48m 53s. Rate: 7.8 symbols/min [2025-10-20T20:15:29.322586] [INFO ] [MAIN ] NQP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:29.386352] [INFO ] [MAIN ] NPWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:29.424361] [INFO ] [MAIN ] NREF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:29.426411] [INFO ] [MAIN ] NRDY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:29.437224] [DEBUG] [LOCAL ] NQP: Loaded 9363 existing records from primary storage. [2025-10-20T20:15:29.439608] [INFO ] [MAIN ] NQP: Loaded 9363 existing records. [2025-10-20T20:15:29.502461] [DEBUG] [LOCAL ] NREF: Loaded 7969 existing records from primary storage. [2025-10-20T20:15:29.551010] [INFO ] [MAIN ] NREF: Loaded 7969 existing records. [2025-10-20T20:15:29.782452] [DEBUG] [LOCAL ] NRDY: Loaded 33086 existing records from primary storage. [2025-10-20T20:15:29.783405] [DEBUG] [LOCAL ] NPWR: Loaded 44112 existing records from primary storage. [2025-10-20T20:15:29.804890] [WARN ] [MAIN ] NQP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:29.805460] [INFO ] [MAIN ] NQP: Wiping existing primary data before full refresh. [2025-10-20T20:15:29.805797] [INFO ] [MAIN ] NPWR: Loaded 44112 existing records. [2025-10-20T20:15:29.805984] [INFO ] [MAIN ] NRDY: Loaded 33086 existing records. [2025-10-20T20:15:29.806127] [INFO ] [LOCAL ] NQP: Starting deletion of primary data. [2025-10-20T20:15:29.829550] [DEBUG] [LOCAL ] NQP: Deleted primary data file. [2025-10-20T20:15:29.868774] [INFO ] [MAIN ] NQP: Starting full primary refresh. [2025-10-20T20:15:30.057323] [WARN ] [MAIN ] NREF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:30.092861] [INFO ] [MAIN ] NREF: Wiping existing primary data before full refresh. [2025-10-20T20:15:30.152240] [INFO ] [LOCAL ] NREF: Starting deletion of primary data. [2025-10-20T20:15:30.304988] [DEBUG] [LOCAL ] NREF: Deleted primary data file. [2025-10-20T20:15:30.345620] [INFO ] [MAIN ] NREF: Starting full primary refresh. [2025-10-20T20:15:30.364249] [WARN ] [MAIN ] NPWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:30.484767] [INFO ] [MAIN ] NPWR: Wiping existing primary data before full refresh. [2025-10-20T20:15:30.522265] [INFO ] [LOCAL ] NPWR: Starting deletion of primary data. [2025-10-20T20:15:30.522138] [WARN ] [MAIN ] NRDY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:30.638104] [INFO ] [MAIN ] NRDY: Wiping existing primary data before full refresh. [2025-10-20T20:15:30.608561] [DEBUG] [LOCAL ] NPWR: Deleted primary data file. [2025-10-20T20:15:30.674426] [INFO ] [LOCAL ] NRDY: Starting deletion of primary data. [2025-10-20T20:15:30.674350] [INFO ] [MAIN ] NPWR: Starting full primary refresh. [2025-10-20T20:15:30.731673] [DEBUG] [LOCAL ] NRDY: Deleted primary data file. [2025-10-20T20:15:30.788906] [INFO ] [MAIN ] NRDY: Starting full primary refresh. [2025-10-20T20:15:31.049339] [INFO ] [REMOTE ] NQP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:31.124538] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-01. [2025-10-20T20:15:31.643988] [INFO ] [REMOTE ] NREF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:31.677284] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-01. [2025-10-20T20:15:31.825554] [INFO ] [REMOTE ] NPWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:31.825894] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-01. [2025-10-20T20:15:31.865545] [INFO ] [REMOTE ] NRDY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:31.865850] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-01. [2025-10-20T20:15:31.971056] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T20:15:31.973211] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-02. [2025-10-20T20:15:32.182911] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T20:15:32.251342] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-02. [2025-10-20T20:15:32.694537] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T20:15:32.704866] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T20:15:33.051368] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-03. [2025-10-20T20:15:33.305273] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T20:15:33.410838] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T20:15:33.805760] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T20:15:33.851336] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-04. [2025-10-20T20:15:34.651367] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-02. [2025-10-20T20:15:34.660065] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T20:15:35.451369] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-05. [2025-10-20T20:15:35.930419] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T20:15:36.050187] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T20:15:36.251386] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-06. [2025-10-20T20:15:37.050513] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T20:15:37.052952] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-02. [2025-10-20T20:15:37.851383] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-07. [2025-10-20T20:15:38.508686] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T20:15:38.651385] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-03. [2025-10-20T20:15:38.746886] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T20:15:39.451367] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-03. [2025-10-20T20:15:40.126803] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T20:15:40.234769] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T20:15:40.251338] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-04. [2025-10-20T20:15:41.034001] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T20:15:41.051330] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-04. [2025-10-20T20:15:41.851371] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-05. [2025-10-20T20:15:42.411903] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T20:15:42.448797] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T20:15:42.651541] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-06. [2025-10-20T20:15:43.252181] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T20:15:43.451374] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-07. [2025-10-20T20:15:44.239097] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:15:44.251329] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-08. [2025-10-20T20:15:44.961061] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T20:15:45.051369] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-09. [2025-10-20T20:15:45.851379] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-08. [2025-10-20T20:15:45.867440] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:15:46.610346] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T20:15:46.651338] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-09. [2025-10-20T20:15:47.425541] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T20:15:47.451330] [INFO ] [REMOTE ] NQP: Fetching primary data for 2025-10. [2025-10-20T20:15:48.186003] [INFO ] [REMOTE ] NQP: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T20:15:48.187558] [INFO ] [REMOTE ] NQP: Full fetch complete (12795 records). [2025-10-20T20:15:48.187626] [DEBUG] [VALIDATOR ] NQP: Validating primary data. [2025-10-20T20:15:48.256589] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-05. [2025-10-20T20:15:48.578914] [DEBUG] [VALIDATOR ] NQP: All expected trading days are present. [2025-10-20T20:15:48.578997] [DEBUG] [VALIDATOR ] NQP: Primary data validation passed. [2025-10-20T20:15:48.579306] [INFO ] [MAIN ] NQP: Fetched primary data is valid. [2025-10-20T20:15:48.579351] [DEBUG] [LOCAL ] NQP: Saving primary data. [2025-10-20T20:15:48.579857] [DEBUG] [LOCAL ] NQP: Trimmed 12795 records to 9208 (keeping 200 days of data) [2025-10-20T20:15:48.635850] [DEBUG] [LOCAL ] NQP: Primary data saved (9208 records). [2025-10-20T20:15:48.635979] [INFO ] [MAIN ] NQP: Full history saved. [2025-10-20T20:15:48.636055] [INFO ] [MAIN ] Update Progress: 1521/2367 symbols completed. NQP took 19.9s. Estimated remaining: 1h 48m 52s. Rate: 7.8 symbols/min [2025-10-20T20:15:48.822907] [INFO ] [MAIN ] NRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:48.876949] [DEBUG] [LOCAL ] NRG: Loaded 53621 existing records from primary storage. [2025-10-20T20:15:48.877264] [INFO ] [MAIN ] NRG: Loaded 53621 existing records. [2025-10-20T20:15:49.059478] [INFO ] [REMOTE ] NREF: Fetching primary data for 2025-10. [2025-10-20T20:15:49.064986] [WARN ] [MAIN ] NRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:49.066196] [INFO ] [MAIN ] NRG: Wiping existing primary data before full refresh. [2025-10-20T20:15:49.066268] [INFO ] [LOCAL ] NRG: Starting deletion of primary data. [2025-10-20T20:15:49.066471] [DEBUG] [LOCAL ] NRG: Deleted primary data file. [2025-10-20T20:15:49.066511] [INFO ] [MAIN ] NRG: Starting full primary refresh. [2025-10-20T20:15:49.427473] [INFO ] [REMOTE ] NRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:49.751023] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T20:15:49.851360] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-06. [2025-10-20T20:15:49.901949] [INFO ] [REMOTE ] NREF: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:15:49.903594] [INFO ] [REMOTE ] NREF: Full fetch complete (10862 records). [2025-10-20T20:15:49.903686] [DEBUG] [VALIDATOR ] NREF: Validating primary data. [2025-10-20T20:15:50.275623] [DEBUG] [VALIDATOR ] NREF: All expected trading days are present. [2025-10-20T20:15:50.275738] [DEBUG] [VALIDATOR ] NREF: Primary data validation passed. [2025-10-20T20:15:50.275990] [INFO ] [MAIN ] NREF: Fetched primary data is valid. [2025-10-20T20:15:50.276027] [DEBUG] [LOCAL ] NREF: Saving primary data. [2025-10-20T20:15:50.276488] [DEBUG] [LOCAL ] NREF: Trimmed 10862 records to 7833 (keeping 200 days of data) [2025-10-20T20:15:50.324186] [DEBUG] [LOCAL ] NREF: Primary data saved (7833 records). [2025-10-20T20:15:50.324329] [INFO ] [MAIN ] NREF: Full history saved. [2025-10-20T20:15:50.324411] [INFO ] [MAIN ] Update Progress: 1522/2367 symbols completed. NREF took 21.6s. Estimated remaining: 1h 48m 41s. Rate: 7.8 symbols/min [2025-10-20T20:15:50.497385] [INFO ] [MAIN ] NRGV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:15:50.538593] [DEBUG] [LOCAL ] NRGV: Loaded 44503 existing records from primary storage. [2025-10-20T20:15:50.538975] [INFO ] [MAIN ] NRGV: Loaded 44503 existing records. [2025-10-20T20:15:50.675265] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-01. [2025-10-20T20:15:50.750006] [WARN ] [MAIN ] NRGV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:15:50.750215] [INFO ] [MAIN ] NRGV: Wiping existing primary data before full refresh. [2025-10-20T20:15:50.750260] [INFO ] [LOCAL ] NRGV: Starting deletion of primary data. [2025-10-20T20:15:50.750443] [DEBUG] [LOCAL ] NRGV: Deleted primary data file. [2025-10-20T20:15:50.750509] [INFO ] [MAIN ] NRGV: Starting full primary refresh. [2025-10-20T20:15:51.120262] [INFO ] [REMOTE ] NRGV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:15:51.264448] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T20:15:51.451420] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-07. [2025-10-20T20:15:52.251451] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-01. [2025-10-20T20:15:52.471417] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T20:15:52.847950] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:15:53.051431] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-08. [2025-10-20T20:15:53.796588] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T20:15:53.851493] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-03. [2025-10-20T20:15:53.900888] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T20:15:54.651457] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-09. [2025-10-20T20:15:55.451410] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-02. [2025-10-20T20:15:55.524830] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T20:15:55.968354] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T20:15:56.251378] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-02. [2025-10-20T20:15:56.894071] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T20:15:57.051379] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-03. [2025-10-20T20:15:57.851407] [INFO ] [REMOTE ] NRDY: Fetching primary data for 2025-10. [2025-10-20T20:15:58.098850] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T20:15:58.641288] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T20:15:58.651325] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-04. [2025-10-20T20:15:59.322868] [INFO ] [REMOTE ] NRDY: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T20:15:59.327832] [INFO ] [REMOTE ] NRDY: Full fetch complete (50292 records). [2025-10-20T20:15:59.327924] [DEBUG] [VALIDATOR ] NRDY: Validating primary data. [2025-10-20T20:15:59.456596] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-03. [2025-10-20T20:15:59.710884] [DEBUG] [VALIDATOR ] NRDY: All expected trading days are present. [2025-10-20T20:15:59.710962] [DEBUG] [VALIDATOR ] NRDY: Primary data validation passed. [2025-10-20T20:15:59.711796] [INFO ] [MAIN ] NRDY: Fetched primary data is valid. [2025-10-20T20:15:59.711863] [DEBUG] [LOCAL ] NRDY: Saving primary data. [2025-10-20T20:15:59.712821] [DEBUG] [LOCAL ] NRDY: Trimmed 50292 records to 32396 (keeping 200 days of data) [2025-10-20T20:15:59.890409] [DEBUG] [LOCAL ] NRDY: Primary data saved (32396 records). [2025-10-20T20:15:59.890755] [INFO ] [MAIN ] NRDY: Full history saved. [2025-10-20T20:15:59.890873] [INFO ] [MAIN ] Update Progress: 1523/2367 symbols completed. NRDY took 31.2s. Estimated remaining: 1h 48m 35s. Rate: 7.8 symbols/min [2025-10-20T20:16:00.102560] [INFO ] [MAIN ] NRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:00.119357] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T20:16:00.148793] [DEBUG] [LOCAL ] NRK: Loaded 20563 existing records from primary storage. [2025-10-20T20:16:00.149066] [INFO ] [MAIN ] NRK: Loaded 20563 existing records. [2025-10-20T20:16:00.254003] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-04. [2025-10-20T20:16:00.333312] [WARN ] [MAIN ] NRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:00.333530] [INFO ] [MAIN ] NRK: Wiping existing primary data before full refresh. [2025-10-20T20:16:00.333580] [INFO ] [LOCAL ] NRK: Starting deletion of primary data. [2025-10-20T20:16:00.333812] [DEBUG] [LOCAL ] NRK: Deleted primary data file. [2025-10-20T20:16:00.333869] [INFO ] [MAIN ] NRK: Starting full primary refresh. [2025-10-20T20:16:00.688817] [INFO ] [REMOTE ] NRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:01.051361] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-01. [2025-10-20T20:16:01.393237] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T20:16:01.851390] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-04. [2025-10-20T20:16:02.092952] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T20:16:02.155373] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T20:16:02.651392] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-05. [2025-10-20T20:16:03.451407] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-05. [2025-10-20T20:16:03.878745] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T20:16:04.205926] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T20:16:04.251385] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-05. [2025-10-20T20:16:04.862986] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T20:16:05.051342] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-06. [2025-10-20T20:16:05.851337] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-06. [2025-10-20T20:16:06.355363] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-05 (2.10 seconds). [2025-10-20T20:16:06.651367] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-06. [2025-10-20T20:16:06.734272] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T20:16:07.451432] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-07. [2025-10-20T20:16:07.545948] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T20:16:08.251381] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-07. [2025-10-20T20:16:08.686045] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-06 (2.03 seconds). [2025-10-20T20:16:09.051392] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-02. [2025-10-20T20:16:09.275942] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T20:16:09.851453] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-07. [2025-10-20T20:16:09.996720] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T20:16:10.062773] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T20:16:10.651371] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-03. [2025-10-20T20:16:11.451363] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-08. [2025-10-20T20:16:11.756503] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T20:16:11.918189] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T20:16:12.251363] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-08. [2025-10-20T20:16:13.012886] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T20:16:13.051335] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-09. [2025-10-20T20:16:13.851383] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-04. [2025-10-20T20:16:14.409459] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-08 (2.16 seconds). [2025-10-20T20:16:14.651428] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-08. [2025-10-20T20:16:14.681802] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:16:14.874813] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T20:16:15.451402] [INFO ] [REMOTE ] NRGV: Fetching primary data for 2025-10. [2025-10-20T20:16:16.214225] [INFO ] [REMOTE ] NRGV: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:16:16.221424] [INFO ] [REMOTE ] NRGV: Full fetch complete (62114 records). [2025-10-20T20:16:16.221583] [DEBUG] [VALIDATOR ] NRGV: Validating primary data. [2025-10-20T20:16:16.222369] [FATAL] [VALIDATOR ] NRGV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:16:16.222459] [FATAL] [MAIN ] NRGV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:16:16.222504] [WARN ] [MAIN ] NRGV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:16:16.222545] [INFO ] [LOCAL ] NRGV: Starting deletion of primary data. [2025-10-20T20:16:16.222605] [DEBUG] [LOCAL ] NRGV: No primary data file found to delete. [2025-10-20T20:16:16.222678] [INFO ] [LOCAL ] NRGV: Starting deletion of derivative data. [2025-10-20T20:16:16.222878] [DEBUG] [LOCAL ] NRGV: Deleted derivative data file. [2025-10-20T20:16:16.222973] [INFO ] [MAIN ] Update Progress: 1524/2367 symbols completed. NRGV took 25.9s. Estimated remaining: 1h 48m 32s. Rate: 7.8 symbols/min [2025-10-20T20:16:16.255431] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-09. [2025-10-20T20:16:16.409388] [INFO ] [MAIN ] NRP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:16.441392] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T20:16:16.444876] [DEBUG] [LOCAL ] NRP: Loaded 4238 existing records from primary storage. [2025-10-20T20:16:16.445383] [INFO ] [MAIN ] NRP: Loaded 4238 existing records. [2025-10-20T20:16:16.651160] [WARN ] [MAIN ] NRP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:16.651395] [INFO ] [MAIN ] NRP: Wiping existing primary data before full refresh. [2025-10-20T20:16:16.651443] [INFO ] [LOCAL ] NRP: Starting deletion of primary data. [2025-10-20T20:16:16.651576] [DEBUG] [LOCAL ] NRP: Deleted primary data file. [2025-10-20T20:16:16.651616] [INFO ] [MAIN ] NRP: Starting full primary refresh. [2025-10-20T20:16:17.025589] [INFO ] [REMOTE ] NRP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:17.051380] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-01. [2025-10-20T20:16:17.787627] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T20:16:17.851400] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-09. [2025-10-20T20:16:18.293669] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:16:18.651388] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-02. [2025-10-20T20:16:19.352437] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T20:16:19.399336] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T20:16:19.451357] [INFO ] [REMOTE ] NPWR: Fetching primary data for 2025-10. [2025-10-20T20:16:20.251370] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-03. [2025-10-20T20:16:20.892824] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T20:16:21.051334] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-05. [2025-10-20T20:16:21.220433] [INFO ] [REMOTE ] NPWR: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T20:16:21.227632] [INFO ] [REMOTE ] NPWR: Full fetch complete (60574 records). [2025-10-20T20:16:21.227816] [DEBUG] [VALIDATOR ] NPWR: Validating primary data. [2025-10-20T20:16:21.607694] [DEBUG] [VALIDATOR ] NPWR: All expected trading days are present. [2025-10-20T20:16:21.607790] [DEBUG] [VALIDATOR ] NPWR: Primary data validation passed. [2025-10-20T20:16:21.608709] [INFO ] [MAIN ] NPWR: Fetched primary data is valid. [2025-10-20T20:16:21.608772] [DEBUG] [LOCAL ] NPWR: Saving primary data. [2025-10-20T20:16:21.610024] [DEBUG] [LOCAL ] NPWR: Trimmed 60574 records to 43398 (keeping 200 days of data) [2025-10-20T20:16:21.854666] [DEBUG] [LOCAL ] NPWR: Primary data saved (43398 records). [2025-10-20T20:16:21.854748] [INFO ] [REMOTE ] NRG: Fetching primary data for 2025-10. [2025-10-20T20:16:21.855685] [INFO ] [MAIN ] NPWR: Full history saved. [2025-10-20T20:16:21.855781] [INFO ] [MAIN ] Update Progress: 1525/2367 symbols completed. NPWR took 53.2s. Estimated remaining: 1h 48m 23s. Rate: 7.8 symbols/min [2025-10-20T20:16:22.029321] [INFO ] [MAIN ] NRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:22.036313] [DEBUG] [LOCAL ] NRT: Loaded 4988 existing records from primary storage. [2025-10-20T20:16:22.036499] [INFO ] [MAIN ] NRT: Loaded 4988 existing records. [2025-10-20T20:16:22.166606] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T20:16:22.242243] [WARN ] [MAIN ] NRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:22.243194] [INFO ] [MAIN ] NRT: Wiping existing primary data before full refresh. [2025-10-20T20:16:22.243305] [INFO ] [LOCAL ] NRT: Starting deletion of primary data. [2025-10-20T20:16:22.243589] [DEBUG] [LOCAL ] NRT: Deleted primary data file. [2025-10-20T20:16:22.243764] [INFO ] [MAIN ] NRT: Starting full primary refresh. [2025-10-20T20:16:22.608458] [INFO ] [REMOTE ] NRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:22.651382] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-01. [2025-10-20T20:16:22.679304] [INFO ] [REMOTE ] NRG: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T20:16:22.687094] [INFO ] [REMOTE ] NRG: Full fetch complete (76447 records). [2025-10-20T20:16:22.687228] [DEBUG] [VALIDATOR ] NRG: Validating primary data. [2025-10-20T20:16:22.688103] [FATAL] [VALIDATOR ] NRG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:16:22.688223] [FATAL] [MAIN ] NRG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:16:22.688272] [WARN ] [MAIN ] NRG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:16:22.688340] [INFO ] [LOCAL ] NRG: Starting deletion of primary data. [2025-10-20T20:16:22.688428] [DEBUG] [LOCAL ] NRG: No primary data file found to delete. [2025-10-20T20:16:22.688486] [INFO ] [LOCAL ] NRG: Starting deletion of derivative data. [2025-10-20T20:16:22.688712] [DEBUG] [LOCAL ] NRG: Deleted derivative data file. [2025-10-20T20:16:22.689161] [INFO ] [MAIN ] Update Progress: 1526/2367 symbols completed. NRG took 34.1s. Estimated remaining: 1h 48m 11s. Rate: 7.8 symbols/min [2025-10-20T20:16:22.863561] [INFO ] [MAIN ] NRUC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:22.870085] [DEBUG] [LOCAL ] NRUC: Loaded 3185 existing records from primary storage. [2025-10-20T20:16:22.870303] [INFO ] [MAIN ] NRUC: Loaded 3185 existing records. [2025-10-20T20:16:23.043058] [WARN ] [MAIN ] NRUC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:23.043316] [INFO ] [MAIN ] NRUC: Wiping existing primary data before full refresh. [2025-10-20T20:16:23.043365] [INFO ] [LOCAL ] NRUC: Starting deletion of primary data. [2025-10-20T20:16:23.043505] [DEBUG] [LOCAL ] NRUC: Deleted primary data file. [2025-10-20T20:16:23.043587] [INFO ] [MAIN ] NRUC: Starting full primary refresh. [2025-10-20T20:16:23.413253] [INFO ] [REMOTE ] NRUC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:23.451410] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-01. [2025-10-20T20:16:23.538143] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:16:24.100226] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T20:16:24.251616] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-04. [2025-10-20T20:16:24.906968] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T20:16:25.051431] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-02. [2025-10-20T20:16:25.778002] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T20:16:25.851345] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-03. [2025-10-20T20:16:26.312709] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-03 (0.46 seconds). [2025-10-20T20:16:26.651381] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-04. [2025-10-20T20:16:27.322851] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:16:27.451379] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-05. [2025-10-20T20:16:28.073555] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T20:16:28.251386] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-05. [2025-10-20T20:16:28.961944] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T20:16:29.051391] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-02. [2025-10-20T20:16:29.735678] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T20:16:29.851401] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-06. [2025-10-20T20:16:30.651347] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-06. [2025-10-20T20:16:30.926215] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T20:16:31.167925] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T20:16:31.451415] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-03. [2025-10-20T20:16:32.238238] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:16:32.251349] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-07. [2025-10-20T20:16:33.051380] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-04. [2025-10-20T20:16:33.494638] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T20:16:33.696403] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T20:16:33.851409] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-05. [2025-10-20T20:16:34.597691] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:16:34.651383] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-06. [2025-10-20T20:16:35.320513] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T20:16:35.451416] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-07. [2025-10-20T20:16:36.123141] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T20:16:36.251388] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-06. [2025-10-20T20:16:36.901048] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T20:16:37.051402] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-07. [2025-10-20T20:16:37.579843] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T20:16:37.851396] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-08. [2025-10-20T20:16:38.651413] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-08. [2025-10-20T20:16:39.017639] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T20:16:39.179665] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T20:16:39.451370] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-09. [2025-10-20T20:16:40.065023] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T20:16:40.251408] [INFO ] [REMOTE ] NRUC: Fetching primary data for 2025-10. [2025-10-20T20:16:40.784237] [INFO ] [REMOTE ] NRUC: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:16:40.785827] [INFO ] [REMOTE ] NRUC: Full fetch complete (5119 records). [2025-10-20T20:16:40.785913] [DEBUG] [VALIDATOR ] NRUC: Validating primary data. [2025-10-20T20:16:41.061721] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-08. [2025-10-20T20:16:41.181068] [DEBUG] [VALIDATOR ] NRUC: All expected trading days are present. [2025-10-20T20:16:41.181144] [DEBUG] [VALIDATOR ] NRUC: Primary data validation passed. [2025-10-20T20:16:41.181307] [INFO ] [MAIN ] NRUC: Fetched primary data is valid. [2025-10-20T20:16:41.181345] [DEBUG] [LOCAL ] NRUC: Saving primary data. [2025-10-20T20:16:41.181718] [DEBUG] [LOCAL ] NRUC: Trimmed 5119 records to 3117 (keeping 200 days of data) [2025-10-20T20:16:41.199784] [DEBUG] [LOCAL ] NRUC: Primary data saved (3117 records). [2025-10-20T20:16:41.199846] [INFO ] [MAIN ] NRUC: Full history saved. [2025-10-20T20:16:41.199914] [INFO ] [MAIN ] Update Progress: 1527/2367 symbols completed. NRUC took 18.5s. Estimated remaining: 1h 48m 9s. Rate: 7.8 symbols/min [2025-10-20T20:16:41.363112] [INFO ] [MAIN ] NSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:41.406308] [DEBUG] [LOCAL ] NSA: Loaded 45821 existing records from primary storage. [2025-10-20T20:16:41.406593] [INFO ] [MAIN ] NSA: Loaded 45821 existing records. [2025-10-20T20:16:41.579456] [WARN ] [MAIN ] NSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:41.579703] [INFO ] [MAIN ] NSA: Wiping existing primary data before full refresh. [2025-10-20T20:16:41.579759] [INFO ] [LOCAL ] NSA: Starting deletion of primary data. [2025-10-20T20:16:41.579945] [DEBUG] [LOCAL ] NSA: Deleted primary data file. [2025-10-20T20:16:41.579983] [INFO ] [MAIN ] NSA: Starting full primary refresh. [2025-10-20T20:16:41.760365] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T20:16:41.856471] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-09. [2025-10-20T20:16:41.943437] [INFO ] [REMOTE ] NSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:42.651377] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-09. [2025-10-20T20:16:42.982546] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T20:16:43.298122] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T20:16:43.451380] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-07. [2025-10-20T20:16:44.154817] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T20:16:44.251353] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-08. [2025-10-20T20:16:45.054112] [INFO ] [REMOTE ] NRK: Fetching primary data for 2025-10. [2025-10-20T20:16:45.055125] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T20:16:45.851352] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-01. [2025-10-20T20:16:46.082073] [INFO ] [REMOTE ] NRK: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:16:46.087424] [INFO ] [REMOTE ] NRK: Full fetch complete (27513 records). [2025-10-20T20:16:46.087611] [DEBUG] [VALIDATOR ] NRK: Validating primary data. [2025-10-20T20:16:46.655740] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-09. [2025-10-20T20:16:46.657272] [DEBUG] [VALIDATOR ] NRK: All expected trading days are present. [2025-10-20T20:16:46.657425] [DEBUG] [VALIDATOR ] NRK: Primary data validation passed. [2025-10-20T20:16:46.658371] [INFO ] [MAIN ] NRK: Fetched primary data is valid. [2025-10-20T20:16:46.658545] [DEBUG] [LOCAL ] NRK: Saving primary data. [2025-10-20T20:16:46.659786] [DEBUG] [LOCAL ] NRK: Trimmed 27513 records to 20401 (keeping 200 days of data) [2025-10-20T20:16:46.832300] [DEBUG] [LOCAL ] NRK: Primary data saved (20401 records). [2025-10-20T20:16:46.832454] [INFO ] [MAIN ] NRK: Full history saved. [2025-10-20T20:16:46.832552] [INFO ] [MAIN ] Update Progress: 1528/2367 symbols completed. NRK took 46.9s. Estimated remaining: 1h 48m 1s. Rate: 7.8 symbols/min [2025-10-20T20:16:47.059005] [INFO ] [MAIN ] NSC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:47.124379] [DEBUG] [LOCAL ] NSC: Loaded 49475 existing records from primary storage. [2025-10-20T20:16:47.124820] [INFO ] [MAIN ] NSC: Loaded 49475 existing records. [2025-10-20T20:16:47.312491] [WARN ] [MAIN ] NSC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:47.312731] [INFO ] [MAIN ] NSC: Wiping existing primary data before full refresh. [2025-10-20T20:16:47.312784] [INFO ] [LOCAL ] NSC: Starting deletion of primary data. [2025-10-20T20:16:47.312997] [DEBUG] [LOCAL ] NSC: Deleted primary data file. [2025-10-20T20:16:47.313052] [INFO ] [MAIN ] NSC: Starting full primary refresh. [2025-10-20T20:16:47.435387] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T20:16:47.452475] [INFO ] [REMOTE ] NRP: Fetching primary data for 2025-10. [2025-10-20T20:16:47.614735] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T20:16:47.720083] [INFO ] [REMOTE ] NSC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:48.151955] [INFO ] [REMOTE ] NRP: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:16:48.153221] [INFO ] [REMOTE ] NRP: Full fetch complete (6307 records). [2025-10-20T20:16:48.153289] [DEBUG] [VALIDATOR ] NRP: Validating primary data. [2025-10-20T20:16:48.257412] [INFO ] [REMOTE ] NRT: Fetching primary data for 2025-10. [2025-10-20T20:16:48.521180] [DEBUG] [VALIDATOR ] NRP: All expected trading days are present. [2025-10-20T20:16:48.521259] [DEBUG] [VALIDATOR ] NRP: Primary data validation passed. [2025-10-20T20:16:48.521435] [INFO ] [MAIN ] NRP: Fetched primary data is valid. [2025-10-20T20:16:48.521475] [DEBUG] [LOCAL ] NRP: Saving primary data. [2025-10-20T20:16:48.521905] [DEBUG] [LOCAL ] NRP: Trimmed 6307 records to 4208 (keeping 200 days of data) [2025-10-20T20:16:48.546859] [DEBUG] [LOCAL ] NRP: Primary data saved (4208 records). [2025-10-20T20:16:48.546934] [INFO ] [MAIN ] NRP: Full history saved. [2025-10-20T20:16:48.547005] [INFO ] [MAIN ] Update Progress: 1529/2367 symbols completed. NRP took 32.3s. Estimated remaining: 1h 47m 50s. Rate: 7.8 symbols/min [2025-10-20T20:16:48.714984] [INFO ] [MAIN ] NSP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:48.748902] [DEBUG] [LOCAL ] NSP: Loaded 36092 existing records from primary storage. [2025-10-20T20:16:48.749168] [INFO ] [MAIN ] NSP: Loaded 36092 existing records. [2025-10-20T20:16:48.929447] [WARN ] [MAIN ] NSP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:48.929849] [INFO ] [MAIN ] NSP: Wiping existing primary data before full refresh. [2025-10-20T20:16:48.929911] [INFO ] [LOCAL ] NSP: Starting deletion of primary data. [2025-10-20T20:16:48.930162] [DEBUG] [LOCAL ] NSP: Deleted primary data file. [2025-10-20T20:16:48.930227] [INFO ] [MAIN ] NSP: Starting full primary refresh. [2025-10-20T20:16:49.035826] [INFO ] [REMOTE ] NRT: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:16:49.053440] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-01. [2025-10-20T20:16:49.058787] [INFO ] [REMOTE ] NRT: Full fetch complete (7856 records). [2025-10-20T20:16:49.058942] [DEBUG] [VALIDATOR ] NRT: Validating primary data. [2025-10-20T20:16:49.593443] [INFO ] [REMOTE ] NSP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:49.669039] [DEBUG] [VALIDATOR ] NRT: All expected trading days are present. [2025-10-20T20:16:49.669105] [DEBUG] [VALIDATOR ] NRT: Primary data validation passed. [2025-10-20T20:16:49.669305] [INFO ] [MAIN ] NRT: Fetched primary data is valid. [2025-10-20T20:16:49.669345] [DEBUG] [LOCAL ] NRT: Saving primary data. [2025-10-20T20:16:49.669754] [DEBUG] [LOCAL ] NRT: Trimmed 7856 records to 4894 (keeping 200 days of data) [2025-10-20T20:16:49.696432] [DEBUG] [LOCAL ] NRT: Primary data saved (4894 records). [2025-10-20T20:16:49.696498] [INFO ] [MAIN ] NRT: Full history saved. [2025-10-20T20:16:49.696565] [INFO ] [MAIN ] Update Progress: 1530/2367 symbols completed. NRT took 27.8s. Estimated remaining: 1h 47m 38s. Rate: 7.8 symbols/min [2025-10-20T20:16:49.863122] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-01. [2025-10-20T20:16:49.866562] [INFO ] [MAIN ] NTB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:16:49.888250] [DEBUG] [LOCAL ] NTB: Loaded 20647 existing records from primary storage. [2025-10-20T20:16:49.888528] [INFO ] [MAIN ] NTB: Loaded 20647 existing records. [2025-10-20T20:16:50.060859] [WARN ] [MAIN ] NTB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:16:50.061101] [INFO ] [MAIN ] NTB: Wiping existing primary data before full refresh. [2025-10-20T20:16:50.061148] [INFO ] [LOCAL ] NTB: Starting deletion of primary data. [2025-10-20T20:16:50.061306] [DEBUG] [LOCAL ] NTB: Deleted primary data file. [2025-10-20T20:16:50.061345] [INFO ] [MAIN ] NTB: Starting full primary refresh. [2025-10-20T20:16:50.411097] [INFO ] [REMOTE ] NTB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:16:50.651383] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-01. [2025-10-20T20:16:50.919671] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T20:16:51.315363] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T20:16:51.438266] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T20:16:51.451337] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-02. [2025-10-20T20:16:52.251377] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-02. [2025-10-20T20:16:52.536436] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T20:16:53.051418] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-02. [2025-10-20T20:16:53.649245] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T20:16:53.851396] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-03. [2025-10-20T20:16:54.155095] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T20:16:54.651398] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-03. [2025-10-20T20:16:54.683310] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T20:16:55.451383] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-02. [2025-10-20T20:16:56.227285] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:16:56.251445] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-03. [2025-10-20T20:16:57.051380] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-04. [2025-10-20T20:16:57.277495] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T20:16:57.369833] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T20:16:57.851385] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-04. [2025-10-20T20:16:58.310222] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T20:16:58.651387] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-04. [2025-10-20T20:16:59.451346] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-05. [2025-10-20T20:16:59.511090] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T20:17:00.206617] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T20:17:00.251347] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-05. [2025-10-20T20:17:00.534830] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T20:17:01.051381] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-03. [2025-10-20T20:17:01.851427] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-06. [2025-10-20T20:17:01.976636] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T20:17:02.651389] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-05. [2025-10-20T20:17:02.962493] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T20:17:02.987855] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T20:17:03.451398] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-07. [2025-10-20T20:17:04.248512] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T20:17:04.255969] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-06. [2025-10-20T20:17:04.550922] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T20:17:05.051383] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-06. [2025-10-20T20:17:05.786517] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T20:17:05.851417] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-07. [2025-10-20T20:17:06.545474] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T20:17:06.651384] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-07. [2025-10-20T20:17:07.451369] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-04. [2025-10-20T20:17:07.525690] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T20:17:08.078348] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T20:17:08.251400] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-08. [2025-10-20T20:17:09.051352] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-08. [2025-10-20T20:17:09.236794] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T20:17:09.851406] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-08. [2025-10-20T20:17:09.896173] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T20:17:10.626004] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T20:17:10.651335] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-09. [2025-10-20T20:17:10.988456] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T20:17:11.451361] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-09. [2025-10-20T20:17:12.096139] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T20:17:12.251349] [INFO ] [REMOTE ] NSP: Fetching primary data for 2025-10. [2025-10-20T20:17:12.485850] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T20:17:12.943597] [INFO ] [REMOTE ] NSP: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:17:12.948936] [INFO ] [REMOTE ] NSP: Full fetch complete (49966 records). [2025-10-20T20:17:12.949037] [DEBUG] [VALIDATOR ] NSP: Validating primary data. [2025-10-20T20:17:12.949610] [FATAL] [VALIDATOR ] NSP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:17:12.949759] [FATAL] [MAIN ] NSP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:17:12.949838] [WARN ] [MAIN ] NSP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:17:12.949944] [INFO ] [LOCAL ] NSP: Starting deletion of primary data. [2025-10-20T20:17:12.950029] [DEBUG] [LOCAL ] NSP: No primary data file found to delete. [2025-10-20T20:17:12.950069] [INFO ] [LOCAL ] NSP: Starting deletion of derivative data. [2025-10-20T20:17:12.950264] [DEBUG] [LOCAL ] NSP: Deleted derivative data file. [2025-10-20T20:17:12.950353] [INFO ] [MAIN ] Update Progress: 1531/2367 symbols completed. NSP took 24.4s. Estimated remaining: 1h 47m 39s. Rate: 7.8 symbols/min [2025-10-20T20:17:13.053404] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-05. [2025-10-20T20:17:13.127256] [INFO ] [MAIN ] NTEST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:13.127419] [DEBUG] [LOCAL ] NTEST: No existing data file found in primary storage. [2025-10-20T20:17:13.127458] [WARN ] [MAIN ] NTEST: No existing data found, performing full refresh instead. [2025-10-20T20:17:13.127594] [INFO ] [MAIN ] NTEST: Wiping existing primary data before full refresh. [2025-10-20T20:17:13.127685] [INFO ] [LOCAL ] NTEST: Starting deletion of primary data. [2025-10-20T20:17:13.127748] [DEBUG] [LOCAL ] NTEST: No primary data file found to delete. [2025-10-20T20:17:13.127805] [INFO ] [MAIN ] NTEST: Starting full primary refresh. [2025-10-20T20:17:13.494297] [INFO ] [REMOTE ] NTEST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:13.851375] [INFO ] [REMOTE ] NTB: Fetching primary data for 2025-10. [2025-10-20T20:17:14.651364] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-01. [2025-10-20T20:17:14.919329] [INFO ] [REMOTE ] NTB: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T20:17:14.921059] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T20:17:14.934196] [INFO ] [REMOTE ] NTB: Full fetch complete (30759 records). [2025-10-20T20:17:14.934793] [DEBUG] [VALIDATOR ] NTB: Validating primary data. [2025-10-20T20:17:15.309956] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-01 (0.66 seconds). [2025-10-20T20:17:15.310342] [DEBUG] [VALIDATOR ] NTB: All expected trading days are present. [2025-10-20T20:17:15.311827] [DEBUG] [VALIDATOR ] NTB: Primary data validation passed. [2025-10-20T20:17:15.312554] [INFO ] [MAIN ] NTB: Fetched primary data is valid. [2025-10-20T20:17:15.312597] [DEBUG] [LOCAL ] NTB: Saving primary data. [2025-10-20T20:17:15.313370] [DEBUG] [LOCAL ] NTB: Trimmed 30759 records to 20386 (keeping 200 days of data) [2025-10-20T20:17:15.434143] [DEBUG] [LOCAL ] NTB: Primary data saved (20386 records). [2025-10-20T20:17:15.434283] [INFO ] [MAIN ] NTB: Full history saved. [2025-10-20T20:17:15.434364] [INFO ] [MAIN ] Update Progress: 1532/2367 symbols completed. NTB took 25.7s. Estimated remaining: 1h 47m 28s. Rate: 7.8 symbols/min [2025-10-20T20:17:15.456556] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-06. [2025-10-20T20:17:15.606276] [INFO ] [MAIN ] NTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:15.654285] [DEBUG] [LOCAL ] NTR: Loaded 53379 existing records from primary storage. [2025-10-20T20:17:15.654664] [INFO ] [MAIN ] NTR: Loaded 53379 existing records. [2025-10-20T20:17:15.830009] [WARN ] [MAIN ] NTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:17:15.830234] [INFO ] [MAIN ] NTR: Wiping existing primary data before full refresh. [2025-10-20T20:17:15.830279] [INFO ] [LOCAL ] NTR: Starting deletion of primary data. [2025-10-20T20:17:15.830508] [DEBUG] [LOCAL ] NTR: Deleted primary data file. [2025-10-20T20:17:15.830563] [INFO ] [MAIN ] NTR: Starting full primary refresh. [2025-10-20T20:17:16.211805] [INFO ] [REMOTE ] NTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:16.251371] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-09. [2025-10-20T20:17:17.051416] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-01. [2025-10-20T20:17:17.169315] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T20:17:17.853119] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-07. [2025-10-20T20:17:17.858276] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T20:17:18.651376] [INFO ] [REMOTE ] NSA: Fetching primary data for 2025-10. [2025-10-20T20:17:18.866230] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T20:17:19.408470] [INFO ] [REMOTE ] NSA: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:17:19.415500] [INFO ] [REMOTE ] NSA: Full fetch complete (62354 records). [2025-10-20T20:17:19.415688] [DEBUG] [VALIDATOR ] NSA: Validating primary data. [2025-10-20T20:17:19.416384] [FATAL] [VALIDATOR ] NSA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:17:19.416478] [FATAL] [MAIN ] NSA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:17:19.416554] [WARN ] [MAIN ] NSA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:17:19.416621] [INFO ] [LOCAL ] NSA: Starting deletion of primary data. [2025-10-20T20:17:19.416730] [DEBUG] [LOCAL ] NSA: No primary data file found to delete. [2025-10-20T20:17:19.416778] [INFO ] [LOCAL ] NSA: Starting deletion of derivative data. [2025-10-20T20:17:19.416952] [DEBUG] [LOCAL ] NSA: Deleted derivative data file. [2025-10-20T20:17:19.417315] [INFO ] [MAIN ] Update Progress: 1533/2367 symbols completed. NSA took 38.2s. Estimated remaining: 1h 47m 19s. Rate: 7.8 symbols/min [2025-10-20T20:17:19.461488] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-02. [2025-10-20T20:17:19.602343] [INFO ] [MAIN ] NTST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:19.647360] [DEBUG] [LOCAL ] NTST: Loaded 44754 existing records from primary storage. [2025-10-20T20:17:19.647945] [INFO ] [MAIN ] NTST: Loaded 44754 existing records. [2025-10-20T20:17:19.848254] [WARN ] [MAIN ] NTST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:17:19.848744] [INFO ] [MAIN ] NTST: Wiping existing primary data before full refresh. [2025-10-20T20:17:19.848909] [INFO ] [LOCAL ] NTST: Starting deletion of primary data. [2025-10-20T20:17:19.849169] [DEBUG] [LOCAL ] NTST: Deleted primary data file. [2025-10-20T20:17:19.849335] [INFO ] [MAIN ] NTST: Starting full primary refresh. [2025-10-20T20:17:19.983453] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T20:17:20.165086] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T20:17:20.258297] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-02. [2025-10-20T20:17:20.291611] [INFO ] [REMOTE ] NTST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:21.051368] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-08. [2025-10-20T20:17:21.851349] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-03. [2025-10-20T20:17:22.110126] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T20:17:22.494058] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T20:17:22.651407] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-01. [2025-10-20T20:17:22.976938] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T20:17:23.451390] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-04. [2025-10-20T20:17:23.933688] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T20:17:24.088762] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:17:24.251406] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-05. [2025-10-20T20:17:25.051351] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-09. [2025-10-20T20:17:25.344879] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T20:17:25.851397] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-02. [2025-10-20T20:17:26.651343] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-03. [2025-10-20T20:17:26.924972] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T20:17:27.266913] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T20:17:27.451364] [INFO ] [REMOTE ] NSC: Fetching primary data for 2025-10. [2025-10-20T20:17:28.154327] [INFO ] [REMOTE ] NSC: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:17:28.161567] [INFO ] [REMOTE ] NSC: Full fetch complete (70617 records). [2025-10-20T20:17:28.161786] [DEBUG] [VALIDATOR ] NSC: Validating primary data. [2025-10-20T20:17:28.162688] [FATAL] [VALIDATOR ] NSC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:17:28.162811] [FATAL] [MAIN ] NSC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:17:28.162865] [WARN ] [MAIN ] NSC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:17:28.162952] [INFO ] [LOCAL ] NSC: Starting deletion of primary data. [2025-10-20T20:17:28.163021] [DEBUG] [LOCAL ] NSC: No primary data file found to delete. [2025-10-20T20:17:28.163057] [INFO ] [LOCAL ] NSC: Starting deletion of derivative data. [2025-10-20T20:17:28.163358] [DEBUG] [LOCAL ] NSC: Deleted derivative data file. [2025-10-20T20:17:28.163449] [INFO ] [MAIN ] Update Progress: 1534/2367 symbols completed. NSC took 41.3s. Estimated remaining: 1h 47m 11s. Rate: 7.8 symbols/min [2025-10-20T20:17:28.256550] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-06. [2025-10-20T20:17:28.380342] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:17:28.381990] [INFO ] [MAIN ] NU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:28.447788] [DEBUG] [LOCAL ] NU: Loaded 54526 existing records from primary storage. [2025-10-20T20:17:28.448223] [INFO ] [MAIN ] NU: Loaded 54526 existing records. [2025-10-20T20:17:28.620638] [WARN ] [MAIN ] NU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:17:28.620931] [INFO ] [MAIN ] NU: Wiping existing primary data before full refresh. [2025-10-20T20:17:28.620991] [INFO ] [LOCAL ] NU: Starting deletion of primary data. [2025-10-20T20:17:28.621228] [DEBUG] [LOCAL ] NU: Deleted primary data file. [2025-10-20T20:17:28.621282] [INFO ] [MAIN ] NU: Starting full primary refresh. [2025-10-20T20:17:28.896337] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T20:17:28.975068] [INFO ] [REMOTE ] NU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:29.051348] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-07. [2025-10-20T20:17:29.733833] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T20:17:29.851373] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-03. [2025-10-20T20:17:30.651379] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-04. [2025-10-20T20:17:31.451361] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-08. [2025-10-20T20:17:31.522254] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T20:17:32.029208] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:17:32.178560] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T20:17:32.251340] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-05. [2025-10-20T20:17:33.051463] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-04. [2025-10-20T20:17:33.851366] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-01. [2025-10-20T20:17:34.023356] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:17:34.651375] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-06. [2025-10-20T20:17:34.730128] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:17:35.451376] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-05. [2025-10-20T20:17:36.223399] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-01 (2.37 seconds). [2025-10-20T20:17:36.251334] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-02. [2025-10-20T20:17:36.408860] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T20:17:37.010983] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T20:17:37.051370] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-06. [2025-10-20T20:17:37.851437] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-07. [2025-10-20T20:17:38.325571] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T20:17:38.547930] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-02 (2.30 seconds). [2025-10-20T20:17:38.651443] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-07. [2025-10-20T20:17:39.451416] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-09. [2025-10-20T20:17:39.766667] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T20:17:40.251354] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-08. [2025-10-20T20:17:40.307088] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T20:17:40.343499] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T20:17:41.051362] [INFO ] [REMOTE ] NTEST: Fetching primary data for 2025-10. [2025-10-20T20:17:41.542771] [INFO ] [REMOTE ] NTEST: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T20:17:41.543945] [INFO ] [REMOTE ] NTEST: Full fetch complete (1975 records). [2025-10-20T20:17:41.544042] [DEBUG] [VALIDATOR ] NTEST: Validating primary data. [2025-10-20T20:17:41.760265] [WARN ] [VALIDATOR ] NTEST: Validation failed - insufficient historical data. Have 177 unique days, need 200 days [2025-10-20T20:17:41.760797] [FATAL] [MAIN ] NTEST: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:17:41.760935] [WARN ] [MAIN ] NTEST: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:17:41.761044] [INFO ] [LOCAL ] NTEST: Starting deletion of primary data. [2025-10-20T20:17:41.761173] [DEBUG] [LOCAL ] NTEST: No primary data file found to delete. [2025-10-20T20:17:41.761232] [INFO ] [LOCAL ] NTEST: Starting deletion of derivative data. [2025-10-20T20:17:41.761279] [DEBUG] [LOCAL ] NTEST: No derivative data file found to delete. [2025-10-20T20:17:41.761350] [INFO ] [MAIN ] Update Progress: 1535/2367 symbols completed. NTEST took 28.8s. Estimated remaining: 1h 47m 7s. Rate: 7.8 symbols/min [2025-10-20T20:17:41.852430] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-08. [2025-10-20T20:17:41.954506] [INFO ] [MAIN ] NUE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:41.954955] [DEBUG] [LOCAL ] NUE: No existing data file found in primary storage. [2025-10-20T20:17:41.955079] [WARN ] [MAIN ] NUE: No existing data found, performing full refresh instead. [2025-10-20T20:17:41.955427] [INFO ] [MAIN ] NUE: Wiping existing primary data before full refresh. [2025-10-20T20:17:41.955506] [INFO ] [LOCAL ] NUE: Starting deletion of primary data. [2025-10-20T20:17:41.955692] [DEBUG] [LOCAL ] NUE: No primary data file found to delete. [2025-10-20T20:17:41.955747] [INFO ] [MAIN ] NUE: Starting full primary refresh. [2025-10-20T20:17:42.275843] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-08 (2.02 seconds). [2025-10-20T20:17:42.412202] [INFO ] [REMOTE ] NUE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:42.651356] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-09. [2025-10-20T20:17:43.451350] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-03. [2025-10-20T20:17:43.507959] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T20:17:44.251347] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-09. [2025-10-20T20:17:44.608554] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T20:17:45.051360] [INFO ] [REMOTE ] NTR: Fetching primary data for 2025-10. [2025-10-20T20:17:45.532945] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T20:17:45.823223] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-03 (2.37 seconds). [2025-10-20T20:17:45.851349] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-04. [2025-10-20T20:17:45.882087] [INFO ] [REMOTE ] NTR: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T20:17:45.889431] [INFO ] [REMOTE ] NTR: Full fetch complete (76126 records). [2025-10-20T20:17:45.889583] [DEBUG] [VALIDATOR ] NTR: Validating primary data. [2025-10-20T20:17:45.890396] [FATAL] [VALIDATOR ] NTR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:17:45.890500] [FATAL] [MAIN ] NTR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:17:45.890569] [WARN ] [MAIN ] NTR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:17:45.890679] [INFO ] [LOCAL ] NTR: Starting deletion of primary data. [2025-10-20T20:17:45.890779] [DEBUG] [LOCAL ] NTR: No primary data file found to delete. [2025-10-20T20:17:45.890831] [INFO ] [LOCAL ] NTR: Starting deletion of derivative data. [2025-10-20T20:17:45.890955] [DEBUG] [LOCAL ] NTR: Deleted derivative data file. [2025-10-20T20:17:45.891029] [INFO ] [MAIN ] Update Progress: 1536/2367 symbols completed. NTR took 30.5s. Estimated remaining: 1h 46m 57s. Rate: 7.8 symbols/min [2025-10-20T20:17:46.061565] [INFO ] [MAIN ] NUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:46.096145] [DEBUG] [LOCAL ] NUS: Loaded 36728 existing records from primary storage. [2025-10-20T20:17:46.096433] [INFO ] [MAIN ] NUS: Loaded 36728 existing records. [2025-10-20T20:17:46.281057] [WARN ] [MAIN ] NUS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:17:46.281295] [INFO ] [MAIN ] NUS: Wiping existing primary data before full refresh. [2025-10-20T20:17:46.281343] [INFO ] [LOCAL ] NUS: Starting deletion of primary data. [2025-10-20T20:17:46.281804] [DEBUG] [LOCAL ] NUS: Deleted primary data file. [2025-10-20T20:17:46.281862] [INFO ] [MAIN ] NUS: Starting full primary refresh. [2025-10-20T20:17:46.643115] [INFO ] [REMOTE ] NUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:46.651328] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-01. [2025-10-20T20:17:47.451393] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-01. [2025-10-20T20:17:48.144912] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T20:17:48.210962] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T20:17:48.251423] [INFO ] [REMOTE ] NTST: Fetching primary data for 2025-10. [2025-10-20T20:17:49.051451] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-02. [2025-10-20T20:17:49.219226] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:17:49.852521] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-05. [2025-10-20T20:17:49.857072] [INFO ] [REMOTE ] NTST: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T20:17:49.864837] [INFO ] [REMOTE ] NTST: Full fetch complete (61842 records). [2025-10-20T20:17:49.864978] [DEBUG] [VALIDATOR ] NTST: Validating primary data. [2025-10-20T20:17:50.309469] [DEBUG] [VALIDATOR ] NTST: All expected trading days are present. [2025-10-20T20:17:50.309608] [DEBUG] [VALIDATOR ] NTST: Primary data validation passed. [2025-10-20T20:17:50.310843] [INFO ] [MAIN ] NTST: Fetched primary data is valid. [2025-10-20T20:17:50.310996] [DEBUG] [LOCAL ] NTST: Saving primary data. [2025-10-20T20:17:50.320399] [DEBUG] [LOCAL ] NTST: Trimmed 61842 records to 44149 (keeping 200 days of data) [2025-10-20T20:17:50.334810] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T20:17:50.610714] [DEBUG] [LOCAL ] NTST: Primary data saved (44149 records). [2025-10-20T20:17:50.610847] [INFO ] [MAIN ] NTST: Full history saved. [2025-10-20T20:17:50.611314] [INFO ] [MAIN ] Update Progress: 1537/2367 symbols completed. NTST took 31.2s. Estimated remaining: 1h 46m 48s. Rate: 7.8 symbols/min [2025-10-20T20:17:50.656553] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-03. [2025-10-20T20:17:50.782296] [INFO ] [MAIN ] NUV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:17:50.807086] [DEBUG] [LOCAL ] NUV: Loaded 24144 existing records from primary storage. [2025-10-20T20:17:50.807345] [INFO ] [MAIN ] NUV: Loaded 24144 existing records. [2025-10-20T20:17:51.000786] [WARN ] [MAIN ] NUV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:17:51.001022] [INFO ] [MAIN ] NUV: Wiping existing primary data before full refresh. [2025-10-20T20:17:51.001070] [INFO ] [LOCAL ] NUV: Starting deletion of primary data. [2025-10-20T20:17:51.001275] [DEBUG] [LOCAL ] NUV: Deleted primary data file. [2025-10-20T20:17:51.001333] [INFO ] [MAIN ] NUV: Starting full primary refresh. [2025-10-20T20:17:51.397392] [INFO ] [REMOTE ] NUV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:17:51.451378] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-02. [2025-10-20T20:17:52.251362] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-01. [2025-10-20T20:17:52.359974] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T20:17:52.369628] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-05 (2.52 seconds). [2025-10-20T20:17:53.051351] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-04. [2025-10-20T20:17:53.443945] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T20:17:53.485689] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T20:17:53.851362] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-02. [2025-10-20T20:17:54.284808] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T20:17:54.651352] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-06. [2025-10-20T20:17:54.971152] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T20:17:55.451448] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-03. [2025-10-20T20:17:56.251410] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-03. [2025-10-20T20:17:56.851594] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-06 (2.20 seconds). [2025-10-20T20:17:57.036443] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:17:57.051361] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-04. [2025-10-20T20:17:57.485152] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T20:17:57.851371] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-05. [2025-10-20T20:17:58.651387] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-07. [2025-10-20T20:17:59.083451] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T20:17:59.137924] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T20:17:59.451387] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-04. [2025-10-20T20:18:00.251450] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-05. [2025-10-20T20:18:00.808220] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T20:18:01.051347] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-05. [2025-10-20T20:18:01.107245] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-07 (2.46 seconds). [2025-10-20T20:18:01.851374] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-08. [2025-10-20T20:18:02.262948] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T20:18:02.373091] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T20:18:02.651398] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-06. [2025-10-20T20:18:03.451416] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-06. [2025-10-20T20:18:03.957885] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T20:18:04.251399] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-09. [2025-10-20T20:18:04.593012] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T20:18:04.616480] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T20:18:05.051383] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-07. [2025-10-20T20:18:05.851392] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-07. [2025-10-20T20:18:06.220854] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T20:18:06.651338] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-08. [2025-10-20T20:18:06.736242] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T20:18:07.451389] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-06. [2025-10-20T20:18:07.510536] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:18:07.900104] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-07 (2.05 seconds). [2025-10-20T20:18:08.251446] [INFO ] [REMOTE ] NU: Fetching primary data for 2025-10. [2025-10-20T20:18:08.601053] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T20:18:08.993432] [INFO ] [REMOTE ] NU: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T20:18:09.000873] [INFO ] [REMOTE ] NU: Full fetch complete (77535 records). [2025-10-20T20:18:09.001052] [DEBUG] [VALIDATOR ] NU: Validating primary data. [2025-10-20T20:18:09.001760] [FATAL] [VALIDATOR ] NU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:18:09.001838] [FATAL] [MAIN ] NU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:18:09.001903] [WARN ] [MAIN ] NU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:18:09.001970] [INFO ] [LOCAL ] NU: Starting deletion of primary data. [2025-10-20T20:18:09.002034] [DEBUG] [LOCAL ] NU: No primary data file found to delete. [2025-10-20T20:18:09.002069] [INFO ] [LOCAL ] NU: Starting deletion of derivative data. [2025-10-20T20:18:09.002204] [DEBUG] [LOCAL ] NU: Deleted derivative data file. [2025-10-20T20:18:09.002279] [INFO ] [MAIN ] Update Progress: 1538/2367 symbols completed. NU took 40.8s. Estimated remaining: 1h 46m 46s. Rate: 7.8 symbols/min [2025-10-20T20:18:09.056514] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-07. [2025-10-20T20:18:09.175803] [INFO ] [MAIN ] NUVB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:09.226346] [DEBUG] [LOCAL ] NUVB: Loaded 52343 existing records from primary storage. [2025-10-20T20:18:09.226710] [INFO ] [MAIN ] NUVB: Loaded 52343 existing records. [2025-10-20T20:18:09.418050] [WARN ] [MAIN ] NUVB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:09.418435] [INFO ] [MAIN ] NUVB: Wiping existing primary data before full refresh. [2025-10-20T20:18:09.418483] [INFO ] [LOCAL ] NUVB: Starting deletion of primary data. [2025-10-20T20:18:09.418736] [DEBUG] [LOCAL ] NUVB: Deleted primary data file. [2025-10-20T20:18:09.418805] [INFO ] [MAIN ] NUVB: Starting full primary refresh. [2025-10-20T20:18:09.783788] [INFO ] [REMOTE ] NUVB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:09.851339] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-01. [2025-10-20T20:18:10.614098] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T20:18:10.651352] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-08. [2025-10-20T20:18:11.244000] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T20:18:11.451372] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-09. [2025-10-20T20:18:12.130126] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T20:18:12.251353] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-02. [2025-10-20T20:18:12.551378] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T20:18:13.051370] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-08. [2025-10-20T20:18:13.851380] [INFO ] [REMOTE ] NUV: Fetching primary data for 2025-10. [2025-10-20T20:18:13.894399] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T20:18:14.651352] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-03. [2025-10-20T20:18:14.969479] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T20:18:15.012413] [INFO ] [REMOTE ] NUV: Incremental fetch complete for 2025-10 (1.16 seconds). [2025-10-20T20:18:15.016779] [INFO ] [REMOTE ] NUV: Full fetch complete (33791 records). [2025-10-20T20:18:15.016877] [DEBUG] [VALIDATOR ] NUV: Validating primary data. [2025-10-20T20:18:15.382141] [DEBUG] [VALIDATOR ] NUV: All expected trading days are present. [2025-10-20T20:18:15.382223] [DEBUG] [VALIDATOR ] NUV: Primary data validation passed. [2025-10-20T20:18:15.382896] [INFO ] [MAIN ] NUV: Fetched primary data is valid. [2025-10-20T20:18:15.382959] [DEBUG] [LOCAL ] NUV: Saving primary data. [2025-10-20T20:18:15.383675] [DEBUG] [LOCAL ] NUV: Trimmed 33791 records to 23604 (keeping 200 days of data) [2025-10-20T20:18:15.468493] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-09. [2025-10-20T20:18:15.524994] [DEBUG] [LOCAL ] NUV: Primary data saved (23604 records). [2025-10-20T20:18:15.525112] [INFO ] [MAIN ] NUV: Full history saved. [2025-10-20T20:18:15.525187] [INFO ] [MAIN ] Update Progress: 1539/2367 symbols completed. NUV took 24.9s. Estimated remaining: 1h 46m 38s. Rate: 7.8 symbols/min [2025-10-20T20:18:15.697283] [INFO ] [MAIN ] NUW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:15.704905] [DEBUG] [LOCAL ] NUW: Loaded 6809 existing records from primary storage. [2025-10-20T20:18:15.705078] [INFO ] [MAIN ] NUW: Loaded 6809 existing records. [2025-10-20T20:18:15.888612] [WARN ] [MAIN ] NUW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:15.888855] [INFO ] [MAIN ] NUW: Wiping existing primary data before full refresh. [2025-10-20T20:18:15.888905] [INFO ] [LOCAL ] NUW: Starting deletion of primary data. [2025-10-20T20:18:15.889049] [DEBUG] [LOCAL ] NUW: Deleted primary data file. [2025-10-20T20:18:15.889087] [INFO ] [MAIN ] NUW: Starting full primary refresh. [2025-10-20T20:18:16.249446] [INFO ] [REMOTE ] NUW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:16.251331] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-01. [2025-10-20T20:18:16.489324] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T20:18:17.051399] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-04. [2025-10-20T20:18:17.063165] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T20:18:17.514537] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T20:18:17.851350] [INFO ] [REMOTE ] NUE: Fetching primary data for 2025-10. [2025-10-20T20:18:18.599030] [INFO ] [REMOTE ] NUE: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:18:18.606380] [INFO ] [REMOTE ] NUE: Full fetch complete (74453 records). [2025-10-20T20:18:18.606528] [DEBUG] [VALIDATOR ] NUE: Validating primary data. [2025-10-20T20:18:18.607320] [FATAL] [VALIDATOR ] NUE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:18:18.607408] [FATAL] [MAIN ] NUE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:18:18.607451] [WARN ] [MAIN ] NUE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:18:18.607489] [INFO ] [LOCAL ] NUE: Starting deletion of primary data. [2025-10-20T20:18:18.607545] [DEBUG] [LOCAL ] NUE: No primary data file found to delete. [2025-10-20T20:18:18.607579] [INFO ] [LOCAL ] NUE: Starting deletion of derivative data. [2025-10-20T20:18:18.607618] [DEBUG] [LOCAL ] NUE: No derivative data file found to delete. [2025-10-20T20:18:18.607706] [INFO ] [MAIN ] Update Progress: 1540/2367 symbols completed. NUE took 36.8s. Estimated remaining: 1h 46m 27s. Rate: 7.8 symbols/min [2025-10-20T20:18:18.662742] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-09. [2025-10-20T20:18:18.787224] [INFO ] [MAIN ] NVG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:18.816571] [DEBUG] [LOCAL ] NVG: Loaded 27878 existing records from primary storage. [2025-10-20T20:18:18.817003] [INFO ] [MAIN ] NVG: Loaded 27878 existing records. [2025-10-20T20:18:19.010771] [WARN ] [MAIN ] NVG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:19.011219] [INFO ] [MAIN ] NVG: Wiping existing primary data before full refresh. [2025-10-20T20:18:19.011354] [INFO ] [LOCAL ] NVG: Starting deletion of primary data. [2025-10-20T20:18:19.011724] [DEBUG] [LOCAL ] NVG: Deleted primary data file. [2025-10-20T20:18:19.011831] [INFO ] [MAIN ] NVG: Starting full primary refresh. [2025-10-20T20:18:19.193522] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T20:18:19.440789] [INFO ] [REMOTE ] NVG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:19.451350] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-01. [2025-10-20T20:18:20.156284] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T20:18:20.251391] [INFO ] [REMOTE ] NUS: Fetching primary data for 2025-10. [2025-10-20T20:18:20.873283] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T20:18:20.938580] [INFO ] [REMOTE ] NUS: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:18:20.943690] [INFO ] [REMOTE ] NUS: Full fetch complete (53326 records). [2025-10-20T20:18:20.943808] [DEBUG] [VALIDATOR ] NUS: Validating primary data. [2025-10-20T20:18:20.944452] [FATAL] [VALIDATOR ] NUS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:18:20.944539] [FATAL] [MAIN ] NUS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:18:20.944583] [WARN ] [MAIN ] NUS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:18:20.944623] [INFO ] [LOCAL ] NUS: Starting deletion of primary data. [2025-10-20T20:18:20.944754] [DEBUG] [LOCAL ] NUS: No primary data file found to delete. [2025-10-20T20:18:20.944804] [INFO ] [LOCAL ] NUS: Starting deletion of derivative data. [2025-10-20T20:18:20.944980] [DEBUG] [LOCAL ] NUS: Deleted derivative data file. [2025-10-20T20:18:20.945064] [INFO ] [MAIN ] Update Progress: 1541/2367 symbols completed. NUS took 35.1s. Estimated remaining: 1h 46m 17s. Rate: 7.8 symbols/min [2025-10-20T20:18:21.053636] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-02. [2025-10-20T20:18:21.124431] [INFO ] [MAIN ] NVGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:21.155009] [DEBUG] [LOCAL ] NVGS: Loaded 30546 existing records from primary storage. [2025-10-20T20:18:21.155287] [INFO ] [MAIN ] NVGS: Loaded 30546 existing records. [2025-10-20T20:18:21.334411] [WARN ] [MAIN ] NVGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:21.334633] [INFO ] [MAIN ] NVGS: Wiping existing primary data before full refresh. [2025-10-20T20:18:21.334713] [INFO ] [LOCAL ] NVGS: Starting deletion of primary data. [2025-10-20T20:18:21.334892] [DEBUG] [LOCAL ] NVGS: Deleted primary data file. [2025-10-20T20:18:21.334930] [INFO ] [MAIN ] NVGS: Starting full primary refresh. [2025-10-20T20:18:21.681885] [INFO ] [REMOTE ] NVGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:21.851343] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-01. [2025-10-20T20:18:22.220234] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T20:18:22.651414] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-03. [2025-10-20T20:18:23.003219] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T20:18:23.451425] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-02. [2025-10-20T20:18:23.882637] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T20:18:24.251420] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-04. [2025-10-20T20:18:24.672871] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T20:18:25.051360] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-02. [2025-10-20T20:18:25.849903] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T20:18:25.850616] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T20:18:25.856986] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-05. [2025-10-20T20:18:26.651458] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-03. [2025-10-20T20:18:27.451404] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-03. [2025-10-20T20:18:27.468993] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:18:27.646505] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T20:18:28.251375] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-06. [2025-10-20T20:18:28.787345] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T20:18:29.051453] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-05. [2025-10-20T20:18:29.851352] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-04. [2025-10-20T20:18:29.956506] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T20:18:30.003042] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T20:18:30.651381] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-06. [2025-10-20T20:18:30.909210] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T20:18:31.451369] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-07. [2025-10-20T20:18:31.903229] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T20:18:32.251455] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-04. [2025-10-20T20:18:33.044172] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T20:18:33.051319] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-05. [2025-10-20T20:18:33.332010] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T20:18:33.570192] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T20:18:33.851434] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-06. [2025-10-20T20:18:34.481687] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:18:34.651387] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-07. [2025-10-20T20:18:35.182118] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-07 (0.53 seconds). [2025-10-20T20:18:35.451380] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-08. [2025-10-20T20:18:36.127511] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T20:18:36.251384] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-09. [2025-10-20T20:18:36.918200] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T20:18:37.051375] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-08. [2025-10-20T20:18:37.851381] [INFO ] [REMOTE ] NUW: Fetching primary data for 2025-10. [2025-10-20T20:18:38.384748] [INFO ] [REMOTE ] NUW: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:18:38.385872] [INFO ] [REMOTE ] NUW: Full fetch complete (10792 records). [2025-10-20T20:18:38.385941] [DEBUG] [VALIDATOR ] NUW: Validating primary data. [2025-10-20T20:18:38.653987] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-07. [2025-10-20T20:18:38.773110] [DEBUG] [VALIDATOR ] NUW: All expected trading days are present. [2025-10-20T20:18:38.773187] [DEBUG] [VALIDATOR ] NUW: Primary data validation passed. [2025-10-20T20:18:38.773432] [INFO ] [MAIN ] NUW: Fetched primary data is valid. [2025-10-20T20:18:38.773469] [DEBUG] [LOCAL ] NUW: Saving primary data. [2025-10-20T20:18:38.773909] [DEBUG] [LOCAL ] NUW: Trimmed 10792 records to 6681 (keeping 200 days of data) [2025-10-20T20:18:38.815042] [DEBUG] [LOCAL ] NUW: Primary data saved (6681 records). [2025-10-20T20:18:38.815164] [INFO ] [MAIN ] NUW: Full history saved. [2025-10-20T20:18:38.815238] [INFO ] [MAIN ] Update Progress: 1542/2367 symbols completed. NUW took 23.3s. Estimated remaining: 1h 46m 14s. Rate: 7.8 symbols/min [2025-10-20T20:18:38.986318] [INFO ] [MAIN ] NVO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:39.044185] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T20:18:39.072382] [DEBUG] [LOCAL ] NVO: Loaded 54564 existing records from primary storage. [2025-10-20T20:18:39.072726] [INFO ] [MAIN ] NVO: Loaded 54564 existing records. [2025-10-20T20:18:39.269037] [WARN ] [MAIN ] NVO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:39.269266] [INFO ] [MAIN ] NVO: Wiping existing primary data before full refresh. [2025-10-20T20:18:39.269317] [INFO ] [LOCAL ] NVO: Starting deletion of primary data. [2025-10-20T20:18:39.269524] [DEBUG] [LOCAL ] NVO: Deleted primary data file. [2025-10-20T20:18:39.269580] [INFO ] [MAIN ] NVO: Starting full primary refresh. [2025-10-20T20:18:39.460518] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-05. [2025-10-20T20:18:39.616738] [INFO ] [REMOTE ] NVO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:40.055438] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:18:40.251372] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-08. [2025-10-20T20:18:40.934797] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T20:18:41.051426] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-06. [2025-10-20T20:18:41.156559] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T20:18:41.851367] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-09. [2025-10-20T20:18:42.502892] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T20:18:42.651399] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-01. [2025-10-20T20:18:42.873904] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:18:43.451371] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-09. [2025-10-20T20:18:44.251376] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-07. [2025-10-20T20:18:45.051345] [INFO ] [REMOTE ] NVG: Fetching primary data for 2025-10. [2025-10-20T20:18:45.150232] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-01 (2.50 seconds). [2025-10-20T20:18:45.253436] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T20:18:45.829251] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T20:18:45.851335] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-08. [2025-10-20T20:18:45.886403] [INFO ] [REMOTE ] NVG: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:18:45.890449] [INFO ] [REMOTE ] NVG: Full fetch complete (39674 records). [2025-10-20T20:18:45.890533] [DEBUG] [VALIDATOR ] NVG: Validating primary data. [2025-10-20T20:18:46.266145] [DEBUG] [VALIDATOR ] NVG: All expected trading days are present. [2025-10-20T20:18:46.266241] [DEBUG] [VALIDATOR ] NVG: Primary data validation passed. [2025-10-20T20:18:46.267000] [INFO ] [MAIN ] NVG: Fetched primary data is valid. [2025-10-20T20:18:46.267068] [DEBUG] [LOCAL ] NVG: Saving primary data. [2025-10-20T20:18:46.268060] [DEBUG] [LOCAL ] NVG: Trimmed 39674 records to 27492 (keeping 200 days of data) [2025-10-20T20:18:46.430485] [DEBUG] [LOCAL ] NVG: Primary data saved (27492 records). [2025-10-20T20:18:46.430615] [INFO ] [MAIN ] NVG: Full history saved. [2025-10-20T20:18:46.430720] [INFO ] [MAIN ] Update Progress: 1543/2367 symbols completed. NVG took 27.8s. Estimated remaining: 1h 46m 7s. Rate: 7.8 symbols/min [2025-10-20T20:18:46.597809] [INFO ] [MAIN ] NVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:46.616894] [DEBUG] [LOCAL ] NVR: Loaded 18842 existing records from primary storage. [2025-10-20T20:18:46.617138] [INFO ] [MAIN ] NVR: Loaded 18842 existing records. [2025-10-20T20:18:46.659453] [INFO ] [REMOTE ] NUVB: Fetching primary data for 2025-10. [2025-10-20T20:18:46.805296] [WARN ] [MAIN ] NVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:46.805503] [INFO ] [MAIN ] NVR: Wiping existing primary data before full refresh. [2025-10-20T20:18:46.805554] [INFO ] [LOCAL ] NVR: Starting deletion of primary data. [2025-10-20T20:18:46.805792] [DEBUG] [LOCAL ] NVR: Deleted primary data file. [2025-10-20T20:18:46.805851] [INFO ] [MAIN ] NVR: Starting full primary refresh. [2025-10-20T20:18:47.174421] [INFO ] [REMOTE ] NVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:47.191310] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T20:18:47.454417] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-02. [2025-10-20T20:18:47.461538] [INFO ] [REMOTE ] NUVB: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:18:47.472209] [INFO ] [REMOTE ] NUVB: Full fetch complete (73016 records). [2025-10-20T20:18:47.472497] [DEBUG] [VALIDATOR ] NUVB: Validating primary data. [2025-10-20T20:18:47.473561] [FATAL] [VALIDATOR ] NUVB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:18:47.473782] [FATAL] [MAIN ] NUVB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:18:47.473857] [WARN ] [MAIN ] NUVB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:18:47.473936] [INFO ] [LOCAL ] NUVB: Starting deletion of primary data. [2025-10-20T20:18:47.474016] [DEBUG] [LOCAL ] NUVB: No primary data file found to delete. [2025-10-20T20:18:47.474055] [INFO ] [LOCAL ] NUVB: Starting deletion of derivative data. [2025-10-20T20:18:47.474263] [DEBUG] [LOCAL ] NUVB: Deleted derivative data file. [2025-10-20T20:18:47.474376] [INFO ] [MAIN ] Update Progress: 1544/2367 symbols completed. NUVB took 38.5s. Estimated remaining: 1h 45m 55s. Rate: 7.8 symbols/min [2025-10-20T20:18:47.650201] [INFO ] [MAIN ] NVRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:47.688895] [DEBUG] [LOCAL ] NVRI: Loaded 39555 existing records from primary storage. [2025-10-20T20:18:47.689332] [INFO ] [MAIN ] NVRI: Loaded 39555 existing records. [2025-10-20T20:18:47.866278] [WARN ] [MAIN ] NVRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:47.866491] [INFO ] [MAIN ] NVRI: Wiping existing primary data before full refresh. [2025-10-20T20:18:47.866540] [INFO ] [LOCAL ] NVRI: Starting deletion of primary data. [2025-10-20T20:18:47.866791] [DEBUG] [LOCAL ] NVRI: Deleted primary data file. [2025-10-20T20:18:47.866849] [INFO ] [MAIN ] NVRI: Starting full primary refresh. [2025-10-20T20:18:48.218786] [INFO ] [REMOTE ] NVRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:48.251348] [INFO ] [REMOTE ] NVRI: Fetching primary data for 2025-01. [2025-10-20T20:18:49.051448] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-01. [2025-10-20T20:18:49.774388] [INFO ] [REMOTE ] NVRI: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T20:18:49.860767] [INFO ] [REMOTE ] NVRI: Fetching primary data for 2025-02. [2025-10-20T20:18:49.882505] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-02 (2.43 seconds). [2025-10-20T20:18:50.250589] [DEBUG] [REMOTE ] NVRI: Error fetching primary data for 2025-02: 'API call returned an empty response.' [2025-10-20T20:18:50.251087] [WARN ] [REMOTE ] NVRI: No data returned for month 2025-02 [2025-10-20T20:18:50.251185] [FATAL] [REMOTE ] NVRI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:18:50.251276] [FATAL] [MAIN ] NVRI: Failed to fetch primary data. [2025-10-20T20:18:50.251340] [WARN ] [MAIN ] NVRI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:18:50.251413] [INFO ] [LOCAL ] NVRI: Starting deletion of primary data. [2025-10-20T20:18:50.251488] [DEBUG] [LOCAL ] NVRI: No primary data file found to delete. [2025-10-20T20:18:50.251525] [INFO ] [LOCAL ] NVRI: Starting deletion of derivative data. [2025-10-20T20:18:50.251716] [DEBUG] [LOCAL ] NVRI: Deleted derivative data file. [2025-10-20T20:18:50.251812] [INFO ] [MAIN ] Update Progress: 1545/2367 symbols completed. NVRI took 2.8s. Estimated remaining: 1h 45m 45s. Rate: 7.8 symbols/min [2025-10-20T20:18:50.429294] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T20:18:50.451825] [INFO ] [MAIN ] NVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:50.503266] [DEBUG] [LOCAL ] NVS: Loaded 52190 existing records from primary storage. [2025-10-20T20:18:50.503742] [INFO ] [MAIN ] NVS: Loaded 52190 existing records. [2025-10-20T20:18:50.653393] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-02. [2025-10-20T20:18:50.680548] [WARN ] [MAIN ] NVS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:50.681001] [INFO ] [MAIN ] NVS: Wiping existing primary data before full refresh. [2025-10-20T20:18:50.681056] [INFO ] [LOCAL ] NVS: Starting deletion of primary data. [2025-10-20T20:18:50.681255] [DEBUG] [LOCAL ] NVS: Deleted primary data file. [2025-10-20T20:18:50.681311] [INFO ] [MAIN ] NVS: Starting full primary refresh. [2025-10-20T20:18:51.026694] [INFO ] [REMOTE ] NVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:51.451365] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-09. [2025-10-20T20:18:51.913825] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T20:18:52.251359] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-03. [2025-10-20T20:18:52.631326] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:18:53.051368] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-01. [2025-10-20T20:18:53.587575] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T20:18:53.851399] [INFO ] [REMOTE ] NVGS: Fetching primary data for 2025-10. [2025-10-20T20:18:54.587682] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T20:18:54.651346] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-04. [2025-10-20T20:18:55.053702] [INFO ] [REMOTE ] NVGS: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T20:18:55.058159] [INFO ] [REMOTE ] NVGS: Full fetch complete (42127 records). [2025-10-20T20:18:55.058320] [DEBUG] [VALIDATOR ] NVGS: Validating primary data. [2025-10-20T20:18:55.452241] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-02. [2025-10-20T20:18:55.466615] [DEBUG] [VALIDATOR ] NVGS: All expected trading days are present. [2025-10-20T20:18:55.466728] [DEBUG] [VALIDATOR ] NVGS: Primary data validation passed. [2025-10-20T20:18:55.467510] [INFO ] [MAIN ] NVGS: Fetched primary data is valid. [2025-10-20T20:18:55.467686] [DEBUG] [LOCAL ] NVGS: Saving primary data. [2025-10-20T20:18:55.468558] [DEBUG] [LOCAL ] NVGS: Trimmed 42127 records to 30187 (keeping 200 days of data) [2025-10-20T20:18:55.643942] [DEBUG] [LOCAL ] NVGS: Primary data saved (30187 records). [2025-10-20T20:18:55.644079] [INFO ] [MAIN ] NVGS: Full history saved. [2025-10-20T20:18:55.644155] [INFO ] [MAIN ] Update Progress: 1546/2367 symbols completed. NVGS took 34.7s. Estimated remaining: 1h 45m 36s. Rate: 7.8 symbols/min [2025-10-20T20:18:55.807246] [INFO ] [MAIN ] NVST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:18:55.854494] [DEBUG] [LOCAL ] NVST: Loaded 52300 existing records from primary storage. [2025-10-20T20:18:55.854821] [INFO ] [MAIN ] NVST: Loaded 52300 existing records. [2025-10-20T20:18:56.029920] [WARN ] [MAIN ] NVST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:18:56.030100] [INFO ] [MAIN ] NVST: Wiping existing primary data before full refresh. [2025-10-20T20:18:56.030146] [INFO ] [LOCAL ] NVST: Starting deletion of primary data. [2025-10-20T20:18:56.030330] [DEBUG] [LOCAL ] NVST: Deleted primary data file. [2025-10-20T20:18:56.030366] [INFO ] [MAIN ] NVST: Starting full primary refresh. [2025-10-20T20:18:56.253333] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-03. [2025-10-20T20:18:56.409628] [INFO ] [REMOTE ] NVST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:18:56.446868] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T20:18:57.051358] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-01. [2025-10-20T20:18:57.380061] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-04 (2.73 seconds). [2025-10-20T20:18:57.601701] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T20:18:57.851426] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-05. [2025-10-20T20:18:58.651371] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-03. [2025-10-20T20:18:58.710778] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T20:18:59.451359] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-04. [2025-10-20T20:18:59.525324] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:19:00.251345] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-02. [2025-10-20T20:19:00.584531] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-05 (2.73 seconds). [2025-10-20T20:19:00.676620] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T20:19:01.051372] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-04. [2025-10-20T20:19:01.678989] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:19:01.851386] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-06. [2025-10-20T20:19:01.915472] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T20:19:02.651369] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-03. [2025-10-20T20:19:03.451375] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-05. [2025-10-20T20:19:04.251370] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-05. [2025-10-20T20:19:04.376889] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T20:19:04.451297] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T20:19:04.530381] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-06 (2.68 seconds). [2025-10-20T20:19:05.051397] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-06. [2025-10-20T20:19:05.101413] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T20:19:05.851365] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-07. [2025-10-20T20:19:05.966037] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T20:19:06.651403] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-06. [2025-10-20T20:19:07.451376] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-04. [2025-10-20T20:19:07.934198] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T20:19:08.251407] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-07. [2025-10-20T20:19:08.730319] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-07 (2.88 seconds). [2025-10-20T20:19:09.051399] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-08. [2025-10-20T20:19:09.374739] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T20:19:09.395726] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T20:19:09.851368] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-07. [2025-10-20T20:19:10.651435] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-05. [2025-10-20T20:19:10.696008] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:19:11.451340] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-08. [2025-10-20T20:19:11.849523] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-08 (2.80 seconds). [2025-10-20T20:19:12.251408] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-09. [2025-10-20T20:19:12.427150] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T20:19:12.473210] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T20:19:13.051358] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-06. [2025-10-20T20:19:13.851487] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-08. [2025-10-20T20:19:14.640417] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T20:19:14.651447] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-09. [2025-10-20T20:19:14.768870] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T20:19:14.995015] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-09 (2.74 seconds). [2025-10-20T20:19:15.451375] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-07. [2025-10-20T20:19:15.763437] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T20:19:16.251451] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-09. [2025-10-20T20:19:17.051354] [INFO ] [REMOTE ] NVR: Fetching primary data for 2025-10. [2025-10-20T20:19:17.115109] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T20:19:17.407063] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T20:19:17.620448] [INFO ] [REMOTE ] NVR: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T20:19:17.623732] [INFO ] [REMOTE ] NVR: Full fetch complete (29031 records). [2025-10-20T20:19:17.623828] [DEBUG] [VALIDATOR ] NVR: Validating primary data. [2025-10-20T20:19:17.624288] [FATAL] [VALIDATOR ] NVR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:19:17.624369] [FATAL] [MAIN ] NVR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:19:17.624410] [WARN ] [MAIN ] NVR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:19:17.624448] [INFO ] [LOCAL ] NVR: Starting deletion of primary data. [2025-10-20T20:19:17.624503] [DEBUG] [LOCAL ] NVR: No primary data file found to delete. [2025-10-20T20:19:17.624537] [INFO ] [LOCAL ] NVR: Starting deletion of derivative data. [2025-10-20T20:19:17.624709] [DEBUG] [LOCAL ] NVR: Deleted derivative data file. [2025-10-20T20:19:17.624797] [INFO ] [MAIN ] Update Progress: 1547/2367 symbols completed. NVR took 31.2s. Estimated remaining: 1h 45m 36s. Rate: 7.8 symbols/min [2025-10-20T20:19:17.792329] [INFO ] [MAIN ] NVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:17.840060] [DEBUG] [LOCAL ] NVT: Loaded 52222 existing records from primary storage. [2025-10-20T20:19:17.840390] [INFO ] [MAIN ] NVT: Loaded 52222 existing records. [2025-10-20T20:19:17.860748] [INFO ] [REMOTE ] NVS: Fetching primary data for 2025-10. [2025-10-20T20:19:18.026875] [WARN ] [MAIN ] NVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:18.027103] [INFO ] [MAIN ] NVT: Wiping existing primary data before full refresh. [2025-10-20T20:19:18.027151] [INFO ] [LOCAL ] NVT: Starting deletion of primary data. [2025-10-20T20:19:18.027352] [DEBUG] [LOCAL ] NVT: Deleted primary data file. [2025-10-20T20:19:18.027391] [INFO ] [MAIN ] NVT: Starting full primary refresh. [2025-10-20T20:19:18.382605] [INFO ] [REMOTE ] NVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:18.651379] [INFO ] [REMOTE ] NVO: Fetching primary data for 2025-10. [2025-10-20T20:19:19.451378] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-01. [2025-10-20T20:19:19.804316] [INFO ] [REMOTE ] NVS: Incremental fetch complete for 2025-10 (1.94 seconds). [2025-10-20T20:19:19.811968] [INFO ] [REMOTE ] NVS: Full fetch complete (75058 records). [2025-10-20T20:19:19.812131] [DEBUG] [VALIDATOR ] NVS: Validating primary data. [2025-10-20T20:19:20.212816] [DEBUG] [VALIDATOR ] NVS: All expected trading days are present. [2025-10-20T20:19:20.212935] [DEBUG] [VALIDATOR ] NVS: Primary data validation passed. [2025-10-20T20:19:20.214080] [INFO ] [MAIN ] NVS: Fetched primary data is valid. [2025-10-20T20:19:20.214201] [DEBUG] [LOCAL ] NVS: Saving primary data. [2025-10-20T20:19:20.215606] [DEBUG] [LOCAL ] NVS: Trimmed 75058 records to 51400 (keeping 200 days of data) [2025-10-20T20:19:20.265186] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-08. [2025-10-20T20:19:20.520713] [DEBUG] [LOCAL ] NVS: Primary data saved (51400 records). [2025-10-20T20:19:20.520866] [INFO ] [MAIN ] NVS: Full history saved. [2025-10-20T20:19:20.520948] [INFO ] [MAIN ] Update Progress: 1548/2367 symbols completed. NVS took 30.3s. Estimated remaining: 1h 45m 26s. Rate: 7.8 symbols/min [2025-10-20T20:19:20.687193] [INFO ] [MAIN ] NWG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:20.736919] [DEBUG] [LOCAL ] NWG: Loaded 52188 existing records from primary storage. [2025-10-20T20:19:20.737224] [INFO ] [MAIN ] NWG: Loaded 52188 existing records. [2025-10-20T20:19:20.925900] [WARN ] [MAIN ] NWG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:20.926196] [INFO ] [MAIN ] NWG: Wiping existing primary data before full refresh. [2025-10-20T20:19:20.926342] [INFO ] [LOCAL ] NWG: Starting deletion of primary data. [2025-10-20T20:19:20.926723] [DEBUG] [LOCAL ] NWG: Deleted primary data file. [2025-10-20T20:19:20.926785] [INFO ] [MAIN ] NWG: Starting full primary refresh. [2025-10-20T20:19:21.337330] [INFO ] [REMOTE ] NWG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:21.341842] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T20:19:21.342565] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-01. [2025-10-20T20:19:21.406970] [INFO ] [REMOTE ] NVO: Incremental fetch complete for 2025-10 (2.76 seconds). [2025-10-20T20:19:21.415183] [INFO ] [REMOTE ] NVO: Full fetch complete (77964 records). [2025-10-20T20:19:21.415333] [DEBUG] [VALIDATOR ] NVO: Validating primary data. [2025-10-20T20:19:21.814158] [DEBUG] [VALIDATOR ] NVO: All expected trading days are present. [2025-10-20T20:19:21.814254] [DEBUG] [VALIDATOR ] NVO: Primary data validation passed. [2025-10-20T20:19:21.815822] [INFO ] [MAIN ] NVO: Fetched primary data is valid. [2025-10-20T20:19:21.815894] [DEBUG] [LOCAL ] NVO: Saving primary data. [2025-10-20T20:19:21.817608] [DEBUG] [LOCAL ] NVO: Trimmed 77964 records to 53784 (keeping 200 days of data) [2025-10-20T20:19:21.871608] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-02. [2025-10-20T20:19:22.158713] [DEBUG] [LOCAL ] NVO: Primary data saved (53784 records). [2025-10-20T20:19:22.158881] [INFO ] [MAIN ] NVO: Full history saved. [2025-10-20T20:19:22.159242] [INFO ] [MAIN ] Update Progress: 1549/2367 symbols completed. NVO took 43.3s. Estimated remaining: 1h 45m 15s. Rate: 7.8 symbols/min [2025-10-20T20:19:22.304320] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T20:19:22.375686] [INFO ] [MAIN ] NWN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:22.404354] [DEBUG] [LOCAL ] NWN: Loaded 27037 existing records from primary storage. [2025-10-20T20:19:22.404628] [INFO ] [MAIN ] NWN: Loaded 27037 existing records. [2025-10-20T20:19:22.593215] [WARN ] [MAIN ] NWN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:22.593459] [INFO ] [MAIN ] NWN: Wiping existing primary data before full refresh. [2025-10-20T20:19:22.593509] [INFO ] [LOCAL ] NWN: Starting deletion of primary data. [2025-10-20T20:19:22.593794] [DEBUG] [LOCAL ] NWN: Deleted primary data file. [2025-10-20T20:19:22.593851] [INFO ] [MAIN ] NWN: Starting full primary refresh. [2025-10-20T20:19:22.652739] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-09. [2025-10-20T20:19:22.974630] [INFO ] [REMOTE ] NWN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:23.128400] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:19:23.451431] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-01. [2025-10-20T20:19:23.670689] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-02 (1.80 seconds). [2025-10-20T20:19:24.251413] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-02. [2025-10-20T20:19:24.530238] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T20:19:24.803136] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:19:25.051374] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-02. [2025-10-20T20:19:25.851404] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-03. [2025-10-20T20:19:25.996560] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T20:19:26.176163] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T20:19:26.651381] [INFO ] [REMOTE ] NVST: Fetching primary data for 2025-10. [2025-10-20T20:19:27.433003] [INFO ] [REMOTE ] NVST: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:19:27.440618] [INFO ] [REMOTE ] NVST: Full fetch complete (74064 records). [2025-10-20T20:19:27.440892] [DEBUG] [VALIDATOR ] NVST: Validating primary data. [2025-10-20T20:19:27.441958] [FATAL] [VALIDATOR ] NVST: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:19:27.442055] [FATAL] [MAIN ] NVST: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:19:27.442106] [WARN ] [MAIN ] NVST: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:19:27.442146] [INFO ] [LOCAL ] NVST: Starting deletion of primary data. [2025-10-20T20:19:27.442200] [DEBUG] [LOCAL ] NVST: No primary data file found to delete. [2025-10-20T20:19:27.442235] [INFO ] [LOCAL ] NVST: Starting deletion of derivative data. [2025-10-20T20:19:27.442382] [DEBUG] [LOCAL ] NVST: Deleted derivative data file. [2025-10-20T20:19:27.442474] [INFO ] [MAIN ] Update Progress: 1550/2367 symbols completed. NVST took 31.8s. Estimated remaining: 1h 45m 6s. Rate: 7.8 symbols/min [2025-10-20T20:19:27.457022] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-03. [2025-10-20T20:19:27.641477] [INFO ] [MAIN ] NX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:27.672564] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T20:19:27.713171] [DEBUG] [LOCAL ] NX: Loaded 33799 existing records from primary storage. [2025-10-20T20:19:27.713456] [INFO ] [MAIN ] NX: Loaded 33799 existing records. [2025-10-20T20:19:27.886932] [WARN ] [MAIN ] NX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:27.887199] [INFO ] [MAIN ] NX: Wiping existing primary data before full refresh. [2025-10-20T20:19:27.887255] [INFO ] [LOCAL ] NX: Starting deletion of primary data. [2025-10-20T20:19:27.887795] [DEBUG] [LOCAL ] NX: Deleted primary data file. [2025-10-20T20:19:27.887853] [INFO ] [MAIN ] NX: Starting full primary refresh. [2025-10-20T20:19:28.244167] [INFO ] [REMOTE ] NX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:28.251329] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-01. [2025-10-20T20:19:29.051368] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-04. [2025-10-20T20:19:29.351562] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T20:19:29.580079] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T20:19:29.851374] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-03. [2025-10-20T20:19:30.651428] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-02. [2025-10-20T20:19:30.876854] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:19:31.000043] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T20:19:31.451380] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-04. [2025-10-20T20:19:32.032697] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T20:19:32.251388] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-03. [2025-10-20T20:19:33.051373] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-05. [2025-10-20T20:19:33.507847] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-04 (2.06 seconds). [2025-10-20T20:19:33.805717] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T20:19:33.851354] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-04. [2025-10-20T20:19:34.651393] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-05. [2025-10-20T20:19:34.822935] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:19:35.292926] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T20:19:35.451364] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-05. [2025-10-20T20:19:36.251372] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-06. [2025-10-20T20:19:36.484582] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T20:19:36.714309] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T20:19:37.051412] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-06. [2025-10-20T20:19:37.761755] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T20:19:37.851385] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-06. [2025-10-20T20:19:38.651360] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-07. [2025-10-20T20:19:38.829378] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T20:19:39.332580] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T20:19:39.451400] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-07. [2025-10-20T20:19:40.234749] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T20:19:40.251339] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-08. [2025-10-20T20:19:41.051361] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-07. [2025-10-20T20:19:41.357482] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T20:19:41.851382] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-08. [2025-10-20T20:19:41.976680] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T20:19:42.583482] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-07 (1.53 seconds). [2025-10-20T20:19:42.651348] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-08. [2025-10-20T20:19:43.451430] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-09. [2025-10-20T20:19:43.550431] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:19:44.084499] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T20:19:44.251377] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-09. [2025-10-20T20:19:45.051371] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-09. [2025-10-20T20:19:45.362977] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T20:19:45.851392] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-04. [2025-10-20T20:19:45.910830] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T20:19:46.553900] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T20:19:46.651432] [INFO ] [REMOTE ] NWG: Fetching primary data for 2025-10. [2025-10-20T20:19:47.238295] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T20:19:47.375382] [INFO ] [REMOTE ] NWG: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T20:19:47.382622] [INFO ] [REMOTE ] NWG: Full fetch complete (73695 records). [2025-10-20T20:19:47.382806] [DEBUG] [VALIDATOR ] NWG: Validating primary data. [2025-10-20T20:19:47.383574] [FATAL] [VALIDATOR ] NWG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:19:47.383716] [FATAL] [MAIN ] NWG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:19:47.383818] [WARN ] [MAIN ] NWG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:19:47.383914] [INFO ] [LOCAL ] NWG: Starting deletion of primary data. [2025-10-20T20:19:47.384016] [DEBUG] [LOCAL ] NWG: No primary data file found to delete. [2025-10-20T20:19:47.384070] [INFO ] [LOCAL ] NWG: Starting deletion of derivative data. [2025-10-20T20:19:47.384188] [DEBUG] [LOCAL ] NWG: Deleted derivative data file. [2025-10-20T20:19:47.384257] [INFO ] [MAIN ] Update Progress: 1551/2367 symbols completed. NWG took 26.9s. Estimated remaining: 1h 45m 4s. Rate: 7.8 symbols/min [2025-10-20T20:19:47.456511] [INFO ] [REMOTE ] NX: Fetching primary data for 2025-10. [2025-10-20T20:19:47.560743] [INFO ] [MAIN ] NXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:47.567243] [DEBUG] [LOCAL ] NXC: Loaded 4164 existing records from primary storage. [2025-10-20T20:19:47.567431] [INFO ] [MAIN ] NXC: Loaded 4164 existing records. [2025-10-20T20:19:47.767775] [WARN ] [MAIN ] NXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:47.767987] [INFO ] [MAIN ] NXC: Wiping existing primary data before full refresh. [2025-10-20T20:19:47.768035] [INFO ] [LOCAL ] NXC: Starting deletion of primary data. [2025-10-20T20:19:47.768214] [DEBUG] [LOCAL ] NXC: Deleted primary data file. [2025-10-20T20:19:47.768271] [INFO ] [MAIN ] NXC: Starting full primary refresh. [2025-10-20T20:19:48.125175] [INFO ] [REMOTE ] NXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:48.138750] [INFO ] [REMOTE ] NX: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:19:48.143639] [INFO ] [REMOTE ] NX: Full fetch complete (47924 records). [2025-10-20T20:19:48.143759] [DEBUG] [VALIDATOR ] NX: Validating primary data. [2025-10-20T20:19:48.144385] [FATAL] [VALIDATOR ] NX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:19:48.144489] [FATAL] [MAIN ] NX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:19:48.144557] [WARN ] [MAIN ] NX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:19:48.144629] [INFO ] [LOCAL ] NX: Starting deletion of primary data. [2025-10-20T20:19:48.144733] [DEBUG] [LOCAL ] NX: No primary data file found to delete. [2025-10-20T20:19:48.144776] [INFO ] [LOCAL ] NX: Starting deletion of derivative data. [2025-10-20T20:19:48.144932] [DEBUG] [LOCAL ] NX: Deleted derivative data file. [2025-10-20T20:19:48.145020] [INFO ] [MAIN ] Update Progress: 1552/2367 symbols completed. NX took 20.7s. Estimated remaining: 1h 44m 53s. Rate: 7.8 symbols/min [2025-10-20T20:19:48.256474] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-01. [2025-10-20T20:19:48.321565] [INFO ] [MAIN ] NXDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:48.321754] [DEBUG] [LOCAL ] NXDR: No existing data file found in primary storage. [2025-10-20T20:19:48.321799] [WARN ] [MAIN ] NXDR: No existing data found, performing full refresh instead. [2025-10-20T20:19:48.321930] [INFO ] [MAIN ] NXDR: Wiping existing primary data before full refresh. [2025-10-20T20:19:48.321986] [INFO ] [LOCAL ] NXDR: Starting deletion of primary data. [2025-10-20T20:19:48.322030] [DEBUG] [LOCAL ] NXDR: No primary data file found to delete. [2025-10-20T20:19:48.322062] [INFO ] [MAIN ] NXDR: Starting full primary refresh. [2025-10-20T20:19:48.691508] [INFO ] [REMOTE ] NXDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:48.941854] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T20:19:49.051385] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-02. [2025-10-20T20:19:49.820023] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:19:49.851436] [INFO ] [REMOTE ] NXDR: Fetching primary data for 2025-01. [2025-10-20T20:19:50.091795] [DEBUG] [REMOTE ] NXDR: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:19:50.092129] [WARN ] [REMOTE ] NXDR: No data returned for month 2025-01 [2025-10-20T20:19:50.092224] [FATAL] [REMOTE ] NXDR: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:19:50.092298] [FATAL] [MAIN ] NXDR: Failed to fetch primary data. [2025-10-20T20:19:50.092364] [WARN ] [MAIN ] NXDR: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:19:50.092437] [INFO ] [LOCAL ] NXDR: Starting deletion of primary data. [2025-10-20T20:19:50.092502] [DEBUG] [LOCAL ] NXDR: No primary data file found to delete. [2025-10-20T20:19:50.092541] [INFO ] [LOCAL ] NXDR: Starting deletion of derivative data. [2025-10-20T20:19:50.092580] [DEBUG] [LOCAL ] NXDR: No derivative data file found to delete. [2025-10-20T20:19:50.092624] [INFO ] [MAIN ] Update Progress: 1553/2367 symbols completed. NXDR took 1.9s. Estimated remaining: 1h 44m 42s. Rate: 7.8 symbols/min [2025-10-20T20:19:50.288703] [INFO ] [MAIN ] NXDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:50.314302] [DEBUG] [LOCAL ] NXDT: Loaded 15223 existing records from primary storage. [2025-10-20T20:19:50.314548] [INFO ] [MAIN ] NXDT: Loaded 15223 existing records. [2025-10-20T20:19:50.497046] [WARN ] [MAIN ] NXDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:50.497270] [INFO ] [MAIN ] NXDT: Wiping existing primary data before full refresh. [2025-10-20T20:19:50.497317] [INFO ] [LOCAL ] NXDT: Starting deletion of primary data. [2025-10-20T20:19:50.497501] [DEBUG] [LOCAL ] NXDT: Deleted primary data file. [2025-10-20T20:19:50.497555] [INFO ] [MAIN ] NXDT: Starting full primary refresh. [2025-10-20T20:19:50.651401] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-05. [2025-10-20T20:19:50.861594] [INFO ] [REMOTE ] NXDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:51.451378] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-03. [2025-10-20T20:19:51.878368] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T20:19:52.251409] [INFO ] [REMOTE ] NVT: Fetching primary data for 2025-10. [2025-10-20T20:19:52.267706] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:19:53.031712] [INFO ] [REMOTE ] NVT: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:19:53.038878] [INFO ] [REMOTE ] NVT: Full fetch complete (74501 records). [2025-10-20T20:19:53.039047] [DEBUG] [VALIDATOR ] NVT: Validating primary data. [2025-10-20T20:19:53.039970] [FATAL] [VALIDATOR ] NVT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:19:53.040070] [FATAL] [MAIN ] NVT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:19:53.040122] [WARN ] [MAIN ] NVT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:19:53.040162] [INFO ] [LOCAL ] NVT: Starting deletion of primary data. [2025-10-20T20:19:53.040218] [DEBUG] [LOCAL ] NVT: No primary data file found to delete. [2025-10-20T20:19:53.040252] [INFO ] [LOCAL ] NVT: Starting deletion of derivative data. [2025-10-20T20:19:53.040408] [DEBUG] [LOCAL ] NVT: Deleted derivative data file. [2025-10-20T20:19:53.040498] [INFO ] [MAIN ] Update Progress: 1554/2367 symbols completed. NVT took 35.4s. Estimated remaining: 1h 44m 32s. Rate: 7.8 symbols/min [2025-10-20T20:19:53.056490] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-06. [2025-10-20T20:19:53.220055] [INFO ] [MAIN ] NXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:19:53.272036] [DEBUG] [LOCAL ] NXE: Loaded 53824 existing records from primary storage. [2025-10-20T20:19:53.272352] [INFO ] [MAIN ] NXE: Loaded 53824 existing records. [2025-10-20T20:19:53.456472] [WARN ] [MAIN ] NXE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:19:53.456718] [INFO ] [MAIN ] NXE: Wiping existing primary data before full refresh. [2025-10-20T20:19:53.456774] [INFO ] [LOCAL ] NXE: Starting deletion of primary data. [2025-10-20T20:19:53.456971] [DEBUG] [LOCAL ] NXE: Deleted primary data file. [2025-10-20T20:19:53.457009] [INFO ] [MAIN ] NXE: Starting full primary refresh. [2025-10-20T20:19:53.838465] [INFO ] [REMOTE ] NXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:19:53.851339] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-01. [2025-10-20T20:19:54.401937] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T20:19:54.651377] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-01. [2025-10-20T20:19:55.451341] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-04. [2025-10-20T20:19:55.485547] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T20:19:55.507838] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T20:19:56.186309] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T20:19:56.251376] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-05. [2025-10-20T20:19:56.989660] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T20:19:57.051376] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-02. [2025-10-20T20:19:57.851404] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-06. [2025-10-20T20:19:58.485923] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:19:58.651365] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-02. [2025-10-20T20:19:58.760173] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T20:19:59.451363] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-03. [2025-10-20T20:19:59.678318] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-02 (1.03 seconds). [2025-10-20T20:20:00.251382] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-07. [2025-10-20T20:20:01.051354] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-03. [2025-10-20T20:20:01.501835] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T20:20:01.851359] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-07. [2025-10-20T20:20:02.042227] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T20:20:02.383351] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T20:20:02.600069] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-07 (0.75 seconds). [2025-10-20T20:20:02.651440] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-04. [2025-10-20T20:20:03.451374] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-04. [2025-10-20T20:20:04.251455] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-08. [2025-10-20T20:20:04.637841] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T20:20:04.693607] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T20:20:05.051406] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-08. [2025-10-20T20:20:05.145727] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T20:20:05.584228] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T20:20:05.851448] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-05. [2025-10-20T20:20:06.651365] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-05. [2025-10-20T20:20:07.451380] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-09. [2025-10-20T20:20:07.715254] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T20:20:07.897284] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T20:20:08.133872] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-09 (0.68 seconds). [2025-10-20T20:20:08.251428] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-09. [2025-10-20T20:20:09.051353] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-06. [2025-10-20T20:20:09.444873] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T20:20:09.851361] [INFO ] [REMOTE ] NXC: Fetching primary data for 2025-10. [2025-10-20T20:20:10.250983] [INFO ] [REMOTE ] NXC: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T20:20:10.252054] [INFO ] [REMOTE ] NXC: Full fetch complete (5613 records). [2025-10-20T20:20:10.252123] [DEBUG] [VALIDATOR ] NXC: Validating primary data. [2025-10-20T20:20:10.610607] [DEBUG] [VALIDATOR ] NXC: All expected trading days are present. [2025-10-20T20:20:10.610716] [DEBUG] [VALIDATOR ] NXC: Primary data validation passed. [2025-10-20T20:20:10.610889] [INFO ] [MAIN ] NXC: Fetched primary data is valid. [2025-10-20T20:20:10.610930] [DEBUG] [LOCAL ] NXC: Saving primary data. [2025-10-20T20:20:10.611293] [DEBUG] [LOCAL ] NXC: Trimmed 5613 records to 4121 (keeping 200 days of data) [2025-10-20T20:20:10.636142] [DEBUG] [LOCAL ] NXC: Primary data saved (4121 records). [2025-10-20T20:20:10.636244] [INFO ] [MAIN ] NXC: Full history saved. [2025-10-20T20:20:10.636320] [INFO ] [MAIN ] Update Progress: 1555/2367 symbols completed. NXC took 23.3s. Estimated remaining: 1h 44m 29s. Rate: 7.8 symbols/min [2025-10-20T20:20:10.667123] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-06. [2025-10-20T20:20:10.812934] [INFO ] [MAIN ] NXG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:10.825168] [DEBUG] [LOCAL ] NXG: Loaded 10631 existing records from primary storage. [2025-10-20T20:20:10.825384] [INFO ] [MAIN ] NXG: Loaded 10631 existing records. [2025-10-20T20:20:11.011565] [WARN ] [MAIN ] NXG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:11.011940] [INFO ] [MAIN ] NXG: Wiping existing primary data before full refresh. [2025-10-20T20:20:11.012041] [INFO ] [LOCAL ] NXG: Starting deletion of primary data. [2025-10-20T20:20:11.012214] [DEBUG] [LOCAL ] NXG: Deleted primary data file. [2025-10-20T20:20:11.012319] [INFO ] [MAIN ] NXG: Starting full primary refresh. [2025-10-20T20:20:11.197127] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-06 (2.15 seconds). [2025-10-20T20:20:11.397344] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T20:20:11.439771] [INFO ] [REMOTE ] NXG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:11.451330] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-07. [2025-10-20T20:20:12.251377] [INFO ] [REMOTE ] NWN: Fetching primary data for 2025-10. [2025-10-20T20:20:12.473380] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T20:20:12.851406] [INFO ] [REMOTE ] NWN: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T20:20:12.854749] [INFO ] [REMOTE ] NWN: Full fetch complete (37157 records). [2025-10-20T20:20:12.854866] [DEBUG] [VALIDATOR ] NWN: Validating primary data. [2025-10-20T20:20:12.855370] [FATAL] [VALIDATOR ] NWN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:20:12.855446] [FATAL] [MAIN ] NWN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:20:12.855511] [WARN ] [MAIN ] NWN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:20:12.855582] [INFO ] [LOCAL ] NWN: Starting deletion of primary data. [2025-10-20T20:20:12.855664] [DEBUG] [LOCAL ] NWN: No primary data file found to delete. [2025-10-20T20:20:12.855721] [INFO ] [LOCAL ] NWN: Starting deletion of derivative data. [2025-10-20T20:20:12.855859] [DEBUG] [LOCAL ] NWN: Deleted derivative data file. [2025-10-20T20:20:12.855942] [INFO ] [MAIN ] Update Progress: 1556/2367 symbols completed. NWN took 50.7s. Estimated remaining: 1h 44m 19s. Rate: 7.8 symbols/min [2025-10-20T20:20:13.032072] [INFO ] [MAIN ] NXJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:13.049194] [DEBUG] [LOCAL ] NXJ: Loaded 15122 existing records from primary storage. [2025-10-20T20:20:13.049399] [INFO ] [MAIN ] NXJ: Loaded 15122 existing records. [2025-10-20T20:20:13.060075] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-07. [2025-10-20T20:20:13.229986] [WARN ] [MAIN ] NXJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:13.230205] [INFO ] [MAIN ] NXJ: Wiping existing primary data before full refresh. [2025-10-20T20:20:13.230254] [INFO ] [LOCAL ] NXJ: Starting deletion of primary data. [2025-10-20T20:20:13.230447] [DEBUG] [LOCAL ] NXJ: Deleted primary data file. [2025-10-20T20:20:13.230503] [INFO ] [MAIN ] NXJ: Starting full primary refresh. [2025-10-20T20:20:13.583678] [INFO ] [REMOTE ] NXJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:13.851364] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-01. [2025-10-20T20:20:14.651356] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-01. [2025-10-20T20:20:14.663187] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T20:20:15.236603] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-07 (2.18 seconds). [2025-10-20T20:20:15.451371] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-08. [2025-10-20T20:20:15.716739] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:20:16.251420] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-02. [2025-10-20T20:20:17.024878] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:20:17.051330] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-03. [2025-10-20T20:20:17.372202] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T20:20:17.851346] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-09. [2025-10-20T20:20:17.903258] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:20:18.651429] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-04. [2025-10-20T20:20:19.451365] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-02. [2025-10-20T20:20:19.573435] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T20:20:19.886270] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T20:20:20.108892] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T20:20:20.251367] [INFO ] [REMOTE ] NXE: Fetching primary data for 2025-10. [2025-10-20T20:20:21.016843] [INFO ] [REMOTE ] NXE: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:20:21.024630] [INFO ] [REMOTE ] NXE: Full fetch complete (76556 records). [2025-10-20T20:20:21.024858] [DEBUG] [VALIDATOR ] NXE: Validating primary data. [2025-10-20T20:20:21.025742] [FATAL] [VALIDATOR ] NXE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:20:21.025842] [FATAL] [MAIN ] NXE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:20:21.025886] [WARN ] [MAIN ] NXE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:20:21.025950] [INFO ] [LOCAL ] NXE: Starting deletion of primary data. [2025-10-20T20:20:21.026016] [DEBUG] [LOCAL ] NXE: No primary data file found to delete. [2025-10-20T20:20:21.026053] [INFO ] [LOCAL ] NXE: Starting deletion of derivative data. [2025-10-20T20:20:21.026213] [DEBUG] [LOCAL ] NXE: Deleted derivative data file. [2025-10-20T20:20:21.026304] [INFO ] [MAIN ] Update Progress: 1557/2367 symbols completed. NXE took 28.0s. Estimated remaining: 1h 44m 11s. Rate: 7.8 symbols/min [2025-10-20T20:20:21.051391] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-05. [2025-10-20T20:20:21.851010] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T20:20:21.852881] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-03. [2025-10-20T20:20:22.651416] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-06. [2025-10-20T20:20:22.701975] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:20:23.451398] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-04. [2025-10-20T20:20:23.479334] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T20:20:24.251370] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-08. [2025-10-20T20:20:24.500195] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T20:20:25.051352] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-07. [2025-10-20T20:20:25.251415] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T20:20:25.851359] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-05. [2025-10-20T20:20:26.050211] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T20:20:26.651347] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-08. [2025-10-20T20:20:26.809698] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T20:20:27.451354] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-06. [2025-10-20T20:20:27.508591] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:20:28.251441] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-09. [2025-10-20T20:20:28.559623] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T20:20:29.051372] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-09. [2025-10-20T20:20:29.171042] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T20:20:29.851398] [INFO ] [REMOTE ] NXDT: Fetching primary data for 2025-10. [2025-10-20T20:20:29.924519] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:20:30.651351] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-07. [2025-10-20T20:20:30.761599] [INFO ] [REMOTE ] NXDT: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T20:20:30.764053] [INFO ] [REMOTE ] NXDT: Full fetch complete (24259 records). [2025-10-20T20:20:30.764122] [DEBUG] [VALIDATOR ] NXDT: Validating primary data. [2025-10-20T20:20:31.121204] [DEBUG] [VALIDATOR ] NXDT: All expected trading days are present. [2025-10-20T20:20:31.121278] [DEBUG] [VALIDATOR ] NXDT: Primary data validation passed. [2025-10-20T20:20:31.121772] [INFO ] [MAIN ] NXDT: Fetched primary data is valid. [2025-10-20T20:20:31.121831] [DEBUG] [LOCAL ] NXDT: Saving primary data. [2025-10-20T20:20:31.122380] [DEBUG] [LOCAL ] NXDT: Trimmed 24259 records to 14961 (keeping 200 days of data) [2025-10-20T20:20:31.207090] [DEBUG] [LOCAL ] NXDT: Primary data saved (14961 records). [2025-10-20T20:20:31.207211] [INFO ] [MAIN ] NXDT: Full history saved. [2025-10-20T20:20:31.207296] [INFO ] [MAIN ] Update Progress: 1558/2367 symbols completed. NXDT took 41.1s. Estimated remaining: 1h 44m 5s. Rate: 7.8 symbols/min [2025-10-20T20:20:31.451341] [INFO ] [REMOTE ] NXG: Fetching primary data for 2025-10. [2025-10-20T20:20:31.749502] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T20:20:32.251405] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-08. [2025-10-20T20:20:32.367907] [INFO ] [REMOTE ] NXG: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T20:20:32.370020] [INFO ] [REMOTE ] NXG: Full fetch complete (14678 records). [2025-10-20T20:20:32.370086] [DEBUG] [VALIDATOR ] NXG: Validating primary data. [2025-10-20T20:20:32.722679] [DEBUG] [VALIDATOR ] NXG: All expected trading days are present. [2025-10-20T20:20:32.722767] [DEBUG] [VALIDATOR ] NXG: Primary data validation passed. [2025-10-20T20:20:32.723094] [INFO ] [MAIN ] NXG: Fetched primary data is valid. [2025-10-20T20:20:32.723131] [DEBUG] [LOCAL ] NXG: Saving primary data. [2025-10-20T20:20:32.723613] [DEBUG] [LOCAL ] NXG: Trimmed 14678 records to 10497 (keeping 200 days of data) [2025-10-20T20:20:32.786493] [DEBUG] [LOCAL ] NXG: Primary data saved (10497 records). [2025-10-20T20:20:32.786625] [INFO ] [MAIN ] NXG: Full history saved. [2025-10-20T20:20:32.786735] [INFO ] [MAIN ] Update Progress: 1559/2367 symbols completed. NXG took 22.2s. Estimated remaining: 1h 43m 54s. Rate: 7.8 symbols/min [2025-10-20T20:20:33.252982] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T20:20:33.257359] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-09. [2025-10-20T20:20:34.262926] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T20:20:34.266794] [INFO ] [REMOTE ] NXJ: Fetching primary data for 2025-10. [2025-10-20T20:20:35.079255] [INFO ] [REMOTE ] NXJ: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T20:20:35.081289] [INFO ] [REMOTE ] NXJ: Full fetch complete (20337 records). [2025-10-20T20:20:35.081360] [DEBUG] [VALIDATOR ] NXJ: Validating primary data. [2025-10-20T20:20:35.436294] [DEBUG] [VALIDATOR ] NXJ: All expected trading days are present. [2025-10-20T20:20:35.436372] [DEBUG] [VALIDATOR ] NXJ: Primary data validation passed. [2025-10-20T20:20:35.436809] [INFO ] [MAIN ] NXJ: Fetched primary data is valid. [2025-10-20T20:20:35.436870] [DEBUG] [LOCAL ] NXJ: Saving primary data. [2025-10-20T20:20:35.437484] [DEBUG] [LOCAL ] NXJ: Trimmed 20337 records to 14894 (keeping 200 days of data) [2025-10-20T20:20:35.531148] [DEBUG] [LOCAL ] NXJ: Primary data saved (14894 records). [2025-10-20T20:20:35.531291] [INFO ] [MAIN ] NXJ: Full history saved. [2025-10-20T20:20:35.531381] [INFO ] [MAIN ] Update Progress: 1560/2367 symbols completed. NXJ took 22.7s. Estimated remaining: 1h 43m 44s. Rate: 7.8 symbols/min [2025-10-20T20:20:36.298677] [INFO ] [MAIN ] NXP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:36.305218] [INFO ] [MAIN ] NXN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:36.308049] [INFO ] [MAIN ] NYC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:36.329954] [DEBUG] [LOCAL ] NYC: No existing data file found in primary storage. [2025-10-20T20:20:36.352726] [INFO ] [MAIN ] NXRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:36.354008] [WARN ] [MAIN ] NYC: No existing data found, performing full refresh instead. [2025-10-20T20:20:36.354858] [INFO ] [MAIN ] NYC: Wiping existing primary data before full refresh. [2025-10-20T20:20:36.354984] [INFO ] [LOCAL ] NYC: Starting deletion of primary data. [2025-10-20T20:20:36.355239] [DEBUG] [LOCAL ] NYC: No primary data file found to delete. [2025-10-20T20:20:36.358009] [INFO ] [MAIN ] NYC: Starting full primary refresh. [2025-10-20T20:20:36.357829] [DEBUG] [LOCAL ] NXN: Loaded 2305 existing records from primary storage. [2025-10-20T20:20:36.382413] [INFO ] [MAIN ] NXN: Loaded 2305 existing records. [2025-10-20T20:20:36.473123] [DEBUG] [LOCAL ] NXP: Loaded 12478 existing records from primary storage. [2025-10-20T20:20:36.501335] [INFO ] [MAIN ] NXP: Loaded 12478 existing records. [2025-10-20T20:20:36.580632] [DEBUG] [LOCAL ] NXRT: Loaded 19213 existing records from primary storage. [2025-10-20T20:20:36.615314] [INFO ] [MAIN ] NXRT: Loaded 19213 existing records. [2025-10-20T20:20:37.222030] [WARN ] [MAIN ] NXN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:37.363174] [WARN ] [MAIN ] NXP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:37.387321] [INFO ] [MAIN ] NXN: Wiping existing primary data before full refresh. [2025-10-20T20:20:37.397953] [INFO ] [MAIN ] NXP: Wiping existing primary data before full refresh. [2025-10-20T20:20:37.423905] [INFO ] [LOCAL ] NXP: Starting deletion of primary data. [2025-10-20T20:20:37.423811] [INFO ] [LOCAL ] NXN: Starting deletion of primary data. [2025-10-20T20:20:37.513833] [WARN ] [MAIN ] NXRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:37.529520] [DEBUG] [LOCAL ] NXN: Deleted primary data file. [2025-10-20T20:20:37.529938] [INFO ] [MAIN ] NXRT: Wiping existing primary data before full refresh. [2025-10-20T20:20:37.530032] [INFO ] [LOCAL ] NXRT: Starting deletion of primary data. [2025-10-20T20:20:37.529870] [INFO ] [MAIN ] NXN: Starting full primary refresh. [2025-10-20T20:20:37.513926] [DEBUG] [LOCAL ] NXP: Deleted primary data file. [2025-10-20T20:20:37.563464] [DEBUG] [LOCAL ] NXRT: Deleted primary data file. [2025-10-20T20:20:37.571609] [INFO ] [MAIN ] NXP: Starting full primary refresh. [2025-10-20T20:20:37.606620] [INFO ] [MAIN ] NXRT: Starting full primary refresh. [2025-10-20T20:20:37.722709] [INFO ] [REMOTE ] NYC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:37.776740] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-01. [2025-10-20T20:20:38.605819] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-01 (0.83 seconds). [2025-10-20T20:20:38.696263] [INFO ] [REMOTE ] NXN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:38.703389] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-02. [2025-10-20T20:20:38.708885] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-01. [2025-10-20T20:20:38.701999] [INFO ] [REMOTE ] NXRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:38.730970] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-01. [2025-10-20T20:20:38.794225] [INFO ] [REMOTE ] NXP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:38.794773] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-01. [2025-10-20T20:20:39.253166] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T20:20:39.254276] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-03. [2025-10-20T20:20:39.276040] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T20:20:39.451371] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-02. [2025-10-20T20:20:39.472451] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T20:20:39.691806] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:20:39.740973] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T20:20:39.977562] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T20:20:40.251385] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-03. [2025-10-20T20:20:40.814810] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-03 (0.56 seconds). [2025-10-20T20:20:41.051400] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-02. [2025-10-20T20:20:41.851346] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-04. [2025-10-20T20:20:41.928681] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T20:20:42.421632] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-04 (0.57 seconds). [2025-10-20T20:20:42.651379] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-05. [2025-10-20T20:20:43.347735] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T20:20:43.451369] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-04. [2025-10-20T20:20:43.960670] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T20:20:44.251368] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-02. [2025-10-20T20:20:44.970582] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T20:20:45.051363] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-03. [2025-10-20T20:20:45.851409] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-03. [2025-10-20T20:20:46.134209] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T20:20:46.651365] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-05. [2025-10-20T20:20:46.811872] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T20:20:47.064965] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T20:20:47.451439] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-06. [2025-10-20T20:20:47.954767] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T20:20:48.251418] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-07. [2025-10-20T20:20:48.662333] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-07 (0.41 seconds). [2025-10-20T20:20:49.051419] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-04. [2025-10-20T20:20:49.851391] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-08. [2025-10-20T20:20:50.124534] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T20:20:50.227938] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-08 (0.38 seconds). [2025-10-20T20:20:50.651417] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-09. [2025-10-20T20:20:51.147368] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T20:20:51.451373] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-05. [2025-10-20T20:20:52.251369] [INFO ] [REMOTE ] NYC: Fetching primary data for 2025-10. [2025-10-20T20:20:52.275156] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T20:20:52.664580] [INFO ] [REMOTE ] NYC: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T20:20:52.665439] [INFO ] [REMOTE ] NYC: Full fetch complete (1857 records). [2025-10-20T20:20:52.665510] [DEBUG] [VALIDATOR ] NYC: Validating primary data. [2025-10-20T20:20:52.847521] [WARN ] [VALIDATOR ] NYC: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T20:20:52.847760] [FATAL] [MAIN ] NYC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:20:52.847833] [WARN ] [MAIN ] NYC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:20:52.847906] [INFO ] [LOCAL ] NYC: Starting deletion of primary data. [2025-10-20T20:20:52.847970] [DEBUG] [LOCAL ] NYC: No primary data file found to delete. [2025-10-20T20:20:52.848009] [INFO ] [LOCAL ] NYC: Starting deletion of derivative data. [2025-10-20T20:20:52.848050] [DEBUG] [LOCAL ] NYC: No derivative data file found to delete. [2025-10-20T20:20:52.848108] [INFO ] [MAIN ] Update Progress: 1561/2367 symbols completed. NYC took 17.3s. Estimated remaining: 1h 43m 41s. Rate: 7.8 symbols/min [2025-10-20T20:20:53.024226] [INFO ] [MAIN ] NYT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:20:53.051431] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-06. [2025-10-20T20:20:53.071571] [DEBUG] [LOCAL ] NYT: Loaded 50375 existing records from primary storage. [2025-10-20T20:20:53.071917] [INFO ] [MAIN ] NYT: Loaded 50375 existing records. [2025-10-20T20:20:53.266309] [WARN ] [MAIN ] NYT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:20:53.266621] [INFO ] [MAIN ] NYT: Wiping existing primary data before full refresh. [2025-10-20T20:20:53.266726] [INFO ] [LOCAL ] NYT: Starting deletion of primary data. [2025-10-20T20:20:53.266983] [DEBUG] [LOCAL ] NYT: Deleted primary data file. [2025-10-20T20:20:53.267042] [INFO ] [MAIN ] NYT: Starting full primary refresh. [2025-10-20T20:20:53.540453] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T20:20:53.650836] [INFO ] [REMOTE ] NYT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:20:53.851421] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-04. [2025-10-20T20:20:54.651464] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-06. [2025-10-20T20:20:54.984758] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T20:20:55.250816] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T20:20:55.451395] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-01. [2025-10-20T20:20:56.251416] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-05. [2025-10-20T20:20:57.051398] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-07. [2025-10-20T20:20:57.110517] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T20:20:57.299881] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T20:20:57.628493] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T20:20:57.851401] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-08. [2025-10-20T20:20:58.374216] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T20:20:58.651419] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-02. [2025-10-20T20:20:59.451383] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-06. [2025-10-20T20:21:00.251362] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-07. [2025-10-20T20:21:00.478292] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T20:21:00.541047] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T20:21:01.051365] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-03. [2025-10-20T20:21:01.288835] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T20:21:01.851378] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-09. [2025-10-20T20:21:02.409550] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T20:21:02.651452] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-08. [2025-10-20T20:21:02.876616] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T20:21:03.451348] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-04. [2025-10-20T20:21:03.533853] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T20:21:04.251429] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-09. [2025-10-20T20:21:05.051412] [INFO ] [REMOTE ] NXN: Fetching primary data for 2025-10. [2025-10-20T20:21:05.144120] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T20:21:05.300316] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T20:21:05.575627] [INFO ] [REMOTE ] NXN: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T20:21:05.576554] [INFO ] [REMOTE ] NXN: Full fetch complete (3460 records). [2025-10-20T20:21:05.576623] [DEBUG] [VALIDATOR ] NXN: Validating primary data. [2025-10-20T20:21:05.856540] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-05. [2025-10-20T20:21:05.949868] [DEBUG] [VALIDATOR ] NXN: All expected trading days are present. [2025-10-20T20:21:05.949947] [DEBUG] [VALIDATOR ] NXN: Primary data validation passed. [2025-10-20T20:21:05.950072] [INFO ] [MAIN ] NXN: Fetched primary data is valid. [2025-10-20T20:21:05.950112] [DEBUG] [LOCAL ] NXN: Saving primary data. [2025-10-20T20:21:05.950465] [DEBUG] [LOCAL ] NXN: Trimmed 3460 records to 2284 (keeping 200 days of data) [2025-10-20T20:21:05.964591] [DEBUG] [LOCAL ] NXN: Primary data saved (2284 records). [2025-10-20T20:21:05.964735] [INFO ] [MAIN ] NXN: Full history saved. [2025-10-20T20:21:05.964824] [INFO ] [MAIN ] Update Progress: 1562/2367 symbols completed. NXN took 30.4s. Estimated remaining: 1h 43m 36s. Rate: 7.8 symbols/min [2025-10-20T20:21:06.140490] [INFO ] [MAIN ] NZF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:06.181971] [DEBUG] [LOCAL ] NZF: Loaded 29261 existing records from primary storage. [2025-10-20T20:21:06.182251] [INFO ] [MAIN ] NZF: Loaded 29261 existing records. [2025-10-20T20:21:06.362767] [WARN ] [MAIN ] NZF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:06.362990] [INFO ] [MAIN ] NZF: Wiping existing primary data before full refresh. [2025-10-20T20:21:06.363038] [INFO ] [LOCAL ] NZF: Starting deletion of primary data. [2025-10-20T20:21:06.363217] [DEBUG] [LOCAL ] NZF: Deleted primary data file. [2025-10-20T20:21:06.363255] [INFO ] [MAIN ] NZF: Starting full primary refresh. [2025-10-20T20:21:06.656477] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-07. [2025-10-20T20:21:06.729525] [INFO ] [REMOTE ] NZF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:07.451380] [INFO ] [REMOTE ] NXP: Fetching primary data for 2025-10. [2025-10-20T20:21:07.767344] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T20:21:07.819629] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T20:21:08.009308] [INFO ] [REMOTE ] NXP: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:21:08.011869] [INFO ] [REMOTE ] NXP: Full fetch complete (16827 records). [2025-10-20T20:21:08.011958] [DEBUG] [VALIDATOR ] NXP: Validating primary data. [2025-10-20T20:21:08.256604] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-01. [2025-10-20T20:21:08.397433] [DEBUG] [VALIDATOR ] NXP: All expected trading days are present. [2025-10-20T20:21:08.397511] [DEBUG] [VALIDATOR ] NXP: Primary data validation passed. [2025-10-20T20:21:08.397916] [INFO ] [MAIN ] NXP: Fetched primary data is valid. [2025-10-20T20:21:08.397977] [DEBUG] [LOCAL ] NXP: Saving primary data. [2025-10-20T20:21:08.398542] [DEBUG] [LOCAL ] NXP: Trimmed 16827 records to 12235 (keeping 200 days of data) [2025-10-20T20:21:08.473666] [DEBUG] [LOCAL ] NXP: Primary data saved (12235 records). [2025-10-20T20:21:08.473819] [INFO ] [MAIN ] NXP: Full history saved. [2025-10-20T20:21:08.473902] [INFO ] [MAIN ] Update Progress: 1563/2367 symbols completed. NXP took 32.9s. Estimated remaining: 1h 43m 26s. Rate: 7.8 symbols/min [2025-10-20T20:21:08.646290] [INFO ] [MAIN ] O: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:08.646464] [DEBUG] [LOCAL ] O: No existing data file found in primary storage. [2025-10-20T20:21:08.646501] [WARN ] [MAIN ] O: No existing data found, performing full refresh instead. [2025-10-20T20:21:08.646614] [INFO ] [MAIN ] O: Wiping existing primary data before full refresh. [2025-10-20T20:21:08.646685] [INFO ] [LOCAL ] O: Starting deletion of primary data. [2025-10-20T20:21:08.646758] [DEBUG] [LOCAL ] O: No primary data file found to delete. [2025-10-20T20:21:08.646799] [INFO ] [MAIN ] O: Starting full primary refresh. [2025-10-20T20:21:09.009224] [INFO ] [REMOTE ] O: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:09.051421] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-08. [2025-10-20T20:21:09.756840] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T20:21:09.851386] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-02. [2025-10-20T20:21:10.178164] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T20:21:10.651407] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-06. [2025-10-20T20:21:10.912400] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T20:21:11.451412] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-03. [2025-10-20T20:21:12.251407] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-09. [2025-10-20T20:21:12.443631] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T20:21:12.838303] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T20:21:13.051400] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-04. [2025-10-20T20:21:13.396587] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T20:21:13.851361] [INFO ] [REMOTE ] O: Fetching primary data for 2025-01. [2025-10-20T20:21:14.278239] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T20:21:14.651419] [INFO ] [REMOTE ] NXRT: Fetching primary data for 2025-10. [2025-10-20T20:21:15.222330] [INFO ] [REMOTE ] NXRT: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T20:21:15.225572] [INFO ] [REMOTE ] NXRT: Full fetch complete (25892 records). [2025-10-20T20:21:15.225727] [DEBUG] [VALIDATOR ] NXRT: Validating primary data. [2025-10-20T20:21:15.226261] [FATAL] [VALIDATOR ] NXRT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:21:15.226362] [FATAL] [MAIN ] NXRT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:21:15.226405] [WARN ] [MAIN ] NXRT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:21:15.226443] [INFO ] [LOCAL ] NXRT: Starting deletion of primary data. [2025-10-20T20:21:15.226499] [DEBUG] [LOCAL ] NXRT: No primary data file found to delete. [2025-10-20T20:21:15.226533] [INFO ] [LOCAL ] NXRT: Starting deletion of derivative data. [2025-10-20T20:21:15.226691] [DEBUG] [LOCAL ] NXRT: Deleted derivative data file. [2025-10-20T20:21:15.226779] [INFO ] [MAIN ] Update Progress: 1564/2367 symbols completed. NXRT took 39.7s. Estimated remaining: 1h 43m 17s. Rate: 7.8 symbols/min [2025-10-20T20:21:15.398703] [INFO ] [MAIN ] OBDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:15.452004] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-05. [2025-10-20T20:21:15.468858] [DEBUG] [LOCAL ] OBDC: Loaded 52457 existing records from primary storage. [2025-10-20T20:21:15.469300] [INFO ] [MAIN ] OBDC: Loaded 52457 existing records. [2025-10-20T20:21:15.650600] [WARN ] [MAIN ] OBDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:15.650825] [INFO ] [MAIN ] OBDC: Wiping existing primary data before full refresh. [2025-10-20T20:21:15.650875] [INFO ] [LOCAL ] OBDC: Starting deletion of primary data. [2025-10-20T20:21:15.651255] [DEBUG] [LOCAL ] OBDC: Deleted primary data file. [2025-10-20T20:21:15.651312] [INFO ] [MAIN ] OBDC: Starting full primary refresh. [2025-10-20T20:21:15.995386] [INFO ] [REMOTE ] OBDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:16.251364] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-07. [2025-10-20T20:21:16.597335] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-01 (2.75 seconds). [2025-10-20T20:21:16.768371] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T20:21:17.051444] [INFO ] [REMOTE ] O: Fetching primary data for 2025-02. [2025-10-20T20:21:17.791408] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T20:21:17.851337] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-08. [2025-10-20T20:21:18.651433] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-06. [2025-10-20T20:21:19.451366] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-01. [2025-10-20T20:21:19.524631] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T20:21:19.567776] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-02 (2.52 seconds). [2025-10-20T20:21:20.024139] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T20:21:20.251549] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-07. [2025-10-20T20:21:21.051428] [INFO ] [REMOTE ] O: Fetching primary data for 2025-03. [2025-10-20T20:21:21.329550] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T20:21:21.558163] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T20:21:21.851397] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-02. [2025-10-20T20:21:22.651414] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-08. [2025-10-20T20:21:23.451366] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-09. [2025-10-20T20:21:23.562508] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T20:21:23.723186] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-03 (2.67 seconds). [2025-10-20T20:21:24.058171] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T20:21:24.251383] [INFO ] [REMOTE ] O: Fetching primary data for 2025-04. [2025-10-20T20:21:25.051391] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-03. [2025-10-20T20:21:25.267228] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T20:21:25.851366] [INFO ] [REMOTE ] NYT: Fetching primary data for 2025-10. [2025-10-20T20:21:26.609835] [INFO ] [REMOTE ] NYT: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:21:26.616725] [INFO ] [REMOTE ] NYT: Full fetch complete (71927 records). [2025-10-20T20:21:26.616858] [DEBUG] [VALIDATOR ] NYT: Validating primary data. [2025-10-20T20:21:26.617591] [FATAL] [VALIDATOR ] NYT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:21:26.617698] [FATAL] [MAIN ] NYT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:21:26.617746] [WARN ] [MAIN ] NYT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:21:26.617810] [INFO ] [LOCAL ] NYT: Starting deletion of primary data. [2025-10-20T20:21:26.617874] [DEBUG] [LOCAL ] NYT: No primary data file found to delete. [2025-10-20T20:21:26.617908] [INFO ] [LOCAL ] NYT: Starting deletion of derivative data. [2025-10-20T20:21:26.618046] [DEBUG] [LOCAL ] NYT: Deleted derivative data file. [2025-10-20T20:21:26.618415] [INFO ] [MAIN ] Update Progress: 1565/2367 symbols completed. NYT took 33.8s. Estimated remaining: 1h 43m 12s. Rate: 7.8 symbols/min [2025-10-20T20:21:26.652795] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-09. [2025-10-20T20:21:26.806905] [INFO ] [MAIN ] OBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:26.856432] [DEBUG] [LOCAL ] OBK: Loaded 16702 existing records from primary storage. [2025-10-20T20:21:26.858445] [INFO ] [MAIN ] OBK: Loaded 16702 existing records. [2025-10-20T20:21:26.858301] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T20:21:27.061938] [WARN ] [MAIN ] OBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:27.062168] [INFO ] [MAIN ] OBK: Wiping existing primary data before full refresh. [2025-10-20T20:21:27.062245] [INFO ] [LOCAL ] OBK: Starting deletion of primary data. [2025-10-20T20:21:27.062589] [DEBUG] [LOCAL ] OBK: Deleted primary data file. [2025-10-20T20:21:27.062756] [INFO ] [MAIN ] OBK: Starting full primary refresh. [2025-10-20T20:21:27.220432] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-04 (2.97 seconds). [2025-10-20T20:21:27.452240] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-04. [2025-10-20T20:21:27.474568] [INFO ] [REMOTE ] OBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:28.016377] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T20:21:28.251352] [INFO ] [REMOTE ] NZF: Fetching primary data for 2025-10. [2025-10-20T20:21:29.051858] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-01. [2025-10-20T20:21:29.416751] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T20:21:29.458270] [INFO ] [REMOTE ] NZF: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T20:21:29.466135] [INFO ] [REMOTE ] NZF: Full fetch complete (42195 records). [2025-10-20T20:21:29.466435] [DEBUG] [VALIDATOR ] NZF: Validating primary data. [2025-10-20T20:21:29.856590] [INFO ] [REMOTE ] O: Fetching primary data for 2025-05. [2025-10-20T20:21:30.011294] [DEBUG] [VALIDATOR ] NZF: All expected trading days are present. [2025-10-20T20:21:30.011382] [DEBUG] [VALIDATOR ] NZF: Primary data validation passed. [2025-10-20T20:21:30.012276] [INFO ] [MAIN ] NZF: Fetched primary data is valid. [2025-10-20T20:21:30.012342] [DEBUG] [LOCAL ] NZF: Saving primary data. [2025-10-20T20:21:30.013665] [DEBUG] [LOCAL ] NZF: Trimmed 42195 records to 28787 (keeping 200 days of data) [2025-10-20T20:21:30.138090] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T20:21:30.218324] [DEBUG] [LOCAL ] NZF: Primary data saved (28787 records). [2025-10-20T20:21:30.218460] [INFO ] [MAIN ] NZF: Full history saved. [2025-10-20T20:21:30.218540] [INFO ] [MAIN ] Update Progress: 1566/2367 symbols completed. NZF took 24.3s. Estimated remaining: 1h 43m 2s. Rate: 7.8 symbols/min [2025-10-20T20:21:30.389789] [INFO ] [MAIN ] OC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:30.434137] [DEBUG] [LOCAL ] OC: Loaded 45614 existing records from primary storage. [2025-10-20T20:21:30.434519] [INFO ] [MAIN ] OC: Loaded 45614 existing records. [2025-10-20T20:21:30.613247] [WARN ] [MAIN ] OC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:30.613434] [INFO ] [MAIN ] OC: Wiping existing primary data before full refresh. [2025-10-20T20:21:30.613480] [INFO ] [LOCAL ] OC: Starting deletion of primary data. [2025-10-20T20:21:30.613983] [DEBUG] [LOCAL ] OC: Deleted primary data file. [2025-10-20T20:21:30.614039] [INFO ] [MAIN ] OC: Starting full primary refresh. [2025-10-20T20:21:30.656977] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-02. [2025-10-20T20:21:30.993141] [INFO ] [REMOTE ] OC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:31.451385] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-05. [2025-10-20T20:21:31.792015] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T20:21:32.251439] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-03. [2025-10-20T20:21:32.560042] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-05 (2.70 seconds). [2025-10-20T20:21:33.051447] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-01. [2025-10-20T20:21:33.104907] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T20:21:33.335587] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T20:21:33.851445] [INFO ] [REMOTE ] O: Fetching primary data for 2025-06. [2025-10-20T20:21:34.443031] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T20:21:34.651378] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-02. [2025-10-20T20:21:35.451394] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-04. [2025-10-20T20:21:36.243805] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-06 (2.39 seconds). [2025-10-20T20:21:36.254405] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-06. [2025-10-20T20:21:36.286206] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T20:21:36.334716] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T20:21:37.051402] [INFO ] [REMOTE ] O: Fetching primary data for 2025-07. [2025-10-20T20:21:37.757209] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T20:21:37.851371] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-05. [2025-10-20T20:21:38.651372] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-07. [2025-10-20T20:21:38.906180] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-05 (1.05 seconds). [2025-10-20T20:21:39.451373] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-06. [2025-10-20T20:21:39.749559] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-07 (2.70 seconds). [2025-10-20T20:21:40.148844] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T20:21:40.251373] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-03. [2025-10-20T20:21:40.588946] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T20:21:41.051385] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-07. [2025-10-20T20:21:41.851384] [INFO ] [REMOTE ] O: Fetching primary data for 2025-08. [2025-10-20T20:21:41.999619] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T20:21:42.097821] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T20:21:42.651361] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-08. [2025-10-20T20:21:43.451364] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-08. [2025-10-20T20:21:44.253063] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-04. [2025-10-20T20:21:44.262126] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-08 (2.41 seconds). [2025-10-20T20:21:44.288304] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T20:21:44.476548] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T20:21:45.051458] [INFO ] [REMOTE ] O: Fetching primary data for 2025-09. [2025-10-20T20:21:45.851375] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-09. [2025-10-20T20:21:45.920226] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:21:46.123412] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T20:21:46.651383] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-09. [2025-10-20T20:21:47.451364] [INFO ] [REMOTE ] O: Fetching primary data for 2025-10. [2025-10-20T20:21:47.719127] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T20:21:47.881164] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T20:21:48.046926] [INFO ] [REMOTE ] O: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T20:21:48.054477] [INFO ] [REMOTE ] O: Full fetch complete (77182 records). [2025-10-20T20:21:48.054635] [DEBUG] [VALIDATOR ] O: Validating primary data. [2025-10-20T20:21:48.055469] [FATAL] [VALIDATOR ] O: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:21:48.055567] [FATAL] [MAIN ] O: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:21:48.055609] [WARN ] [MAIN ] O: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:21:48.055682] [INFO ] [LOCAL ] O: Starting deletion of primary data. [2025-10-20T20:21:48.055780] [DEBUG] [LOCAL ] O: No primary data file found to delete. [2025-10-20T20:21:48.055824] [INFO ] [LOCAL ] O: Starting deletion of derivative data. [2025-10-20T20:21:48.055864] [DEBUG] [LOCAL ] O: No derivative data file found to delete. [2025-10-20T20:21:48.055920] [INFO ] [MAIN ] Update Progress: 1567/2367 symbols completed. O took 39.6s. Estimated remaining: 1h 42m 59s. Rate: 7.8 symbols/min [2025-10-20T20:21:48.222237] [INFO ] [MAIN ] OCFT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:48.231219] [DEBUG] [LOCAL ] OCFT: Loaded 7392 existing records from primary storage. [2025-10-20T20:21:48.231434] [INFO ] [MAIN ] OCFT: Loaded 7392 existing records. [2025-10-20T20:21:48.259990] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-05. [2025-10-20T20:21:48.411548] [WARN ] [MAIN ] OCFT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:48.411776] [INFO ] [MAIN ] OCFT: Wiping existing primary data before full refresh. [2025-10-20T20:21:48.411823] [INFO ] [LOCAL ] OCFT: Starting deletion of primary data. [2025-10-20T20:21:48.411970] [DEBUG] [LOCAL ] OCFT: Deleted primary data file. [2025-10-20T20:21:48.412007] [INFO ] [MAIN ] OCFT: Starting full primary refresh. [2025-10-20T20:21:48.761440] [INFO ] [REMOTE ] OCFT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:49.051385] [INFO ] [REMOTE ] OBDC: Fetching primary data for 2025-10. [2025-10-20T20:21:49.851353] [INFO ] [REMOTE ] OBK: Fetching primary data for 2025-10. [2025-10-20T20:21:50.044529] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T20:21:50.515091] [INFO ] [REMOTE ] OBDC: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T20:21:50.522986] [INFO ] [REMOTE ] OBDC: Full fetch complete (75371 records). [2025-10-20T20:21:50.523110] [DEBUG] [VALIDATOR ] OBDC: Validating primary data. [2025-10-20T20:21:50.661971] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-06. [2025-10-20T20:21:50.881407] [INFO ] [REMOTE ] OBK: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:21:50.920544] [INFO ] [REMOTE ] OBK: Full fetch complete (23998 records). [2025-10-20T20:21:50.941384] [DEBUG] [VALIDATOR ] OBK: Validating primary data. [2025-10-20T20:21:50.945801] [DEBUG] [VALIDATOR ] OBDC: All expected trading days are present. [2025-10-20T20:21:50.946304] [DEBUG] [VALIDATOR ] OBDC: Primary data validation passed. [2025-10-20T20:21:50.947738] [INFO ] [MAIN ] OBDC: Fetched primary data is valid. [2025-10-20T20:21:50.947831] [DEBUG] [LOCAL ] OBDC: Saving primary data. [2025-10-20T20:21:51.063764] [DEBUG] [LOCAL ] OBDC: Trimmed 75371 records to 51730 (keeping 200 days of data) [2025-10-20T20:21:51.460235] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-01. [2025-10-20T20:21:51.618362] [DEBUG] [VALIDATOR ] OBK: All expected trading days are present. [2025-10-20T20:21:51.618515] [DEBUG] [VALIDATOR ] OBK: Primary data validation passed. [2025-10-20T20:21:51.619081] [INFO ] [MAIN ] OBK: Fetched primary data is valid. [2025-10-20T20:21:51.619161] [DEBUG] [LOCAL ] OBK: Saving primary data. [2025-10-20T20:21:51.644389] [DEBUG] [LOCAL ] OBK: Trimmed 23998 records to 16500 (keeping 200 days of data) [2025-10-20T20:21:51.754443] [DEBUG] [LOCAL ] OBDC: Primary data saved (51730 records). [2025-10-20T20:21:51.762790] [DEBUG] [LOCAL ] OBK: Primary data saved (16500 records). [2025-10-20T20:21:51.762967] [INFO ] [MAIN ] OBDC: Full history saved. [2025-10-20T20:21:51.763060] [INFO ] [MAIN ] OBK: Full history saved. [2025-10-20T20:21:51.763169] [INFO ] [MAIN ] Update Progress: 1568/2367 symbols completed. OBDC took 36.5s. Estimated remaining: 1h 42m 50s. Rate: 7.8 symbols/min [2025-10-20T20:21:51.763485] [INFO ] [MAIN ] Update Progress: 1569/2367 symbols completed. OBK took 25.1s. Estimated remaining: 1h 42m 38s. Rate: 7.8 symbols/min [2025-10-20T20:21:52.091675] [INFO ] [MAIN ] ODC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:52.109804] [INFO ] [MAIN ] ODV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:21:52.120664] [DEBUG] [LOCAL ] ODC: Loaded 8687 existing records from primary storage. [2025-10-20T20:21:52.126127] [INFO ] [MAIN ] ODC: Loaded 8687 existing records. [2025-10-20T20:21:52.126339] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T20:21:52.255409] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-02. [2025-10-20T20:21:52.258278] [DEBUG] [LOCAL ] ODV: Loaded 29656 existing records from primary storage. [2025-10-20T20:21:52.299838] [INFO ] [MAIN ] ODV: Loaded 29656 existing records. [2025-10-20T20:21:52.379977] [WARN ] [MAIN ] ODC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:52.380672] [INFO ] [MAIN ] ODC: Wiping existing primary data before full refresh. [2025-10-20T20:21:52.418028] [INFO ] [LOCAL ] ODC: Starting deletion of primary data. [2025-10-20T20:21:52.454072] [DEBUG] [LOCAL ] ODC: Deleted primary data file. [2025-10-20T20:21:52.454355] [INFO ] [MAIN ] ODC: Starting full primary refresh. [2025-10-20T20:21:52.630780] [WARN ] [MAIN ] ODV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:21:52.642423] [INFO ] [MAIN ] ODV: Wiping existing primary data before full refresh. [2025-10-20T20:21:52.652954] [INFO ] [LOCAL ] ODV: Starting deletion of primary data. [2025-10-20T20:21:52.658338] [DEBUG] [LOCAL ] ODV: Deleted primary data file. [2025-10-20T20:21:52.691929] [INFO ] [MAIN ] ODV: Starting full primary refresh. [2025-10-20T20:21:52.986215] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-06 (2.32 seconds). [2025-10-20T20:21:53.079725] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:21:53.135154] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-03. [2025-10-20T20:21:53.134518] [INFO ] [REMOTE ] ODC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:53.315628] [INFO ] [REMOTE ] ODV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:21:53.851399] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-01. [2025-10-20T20:21:54.278740] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T20:21:54.458623] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T20:21:54.651394] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-04. [2025-10-20T20:21:55.451364] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-01. [2025-10-20T20:21:55.471571] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:21:56.152300] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T20:21:56.251436] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-02. [2025-10-20T20:21:56.897398] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:21:57.051378] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-03. [2025-10-20T20:21:57.851402] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-05. [2025-10-20T20:21:58.078461] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:21:58.651463] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-07. [2025-10-20T20:21:58.713157] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T20:21:59.451403] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-04. [2025-10-20T20:22:00.251404] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-06. [2025-10-20T20:22:00.655866] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T20:22:00.688961] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T20:22:01.051417] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-08. [2025-10-20T20:22:01.385016] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:22:01.851380] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-07. [2025-10-20T20:22:02.637484] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T20:22:02.651358] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-09. [2025-10-20T20:22:02.686764] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:22:03.451409] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-05. [2025-10-20T20:22:04.251347] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-08. [2025-10-20T20:22:04.343296] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T20:22:04.539788] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T20:22:04.986465] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T20:22:05.051344] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-09. [2025-10-20T20:22:05.684333] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T20:22:05.851394] [INFO ] [REMOTE ] OCFT: Fetching primary data for 2025-10. [2025-10-20T20:22:06.467001] [INFO ] [REMOTE ] OCFT: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:22:06.468291] [INFO ] [REMOTE ] OCFT: Full fetch complete (11648 records). [2025-10-20T20:22:06.468363] [DEBUG] [VALIDATOR ] OCFT: Validating primary data. [2025-10-20T20:22:06.664242] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-06. [2025-10-20T20:22:06.860385] [DEBUG] [VALIDATOR ] OCFT: All expected trading days are present. [2025-10-20T20:22:06.860464] [DEBUG] [VALIDATOR ] OCFT: Primary data validation passed. [2025-10-20T20:22:06.860761] [INFO ] [MAIN ] OCFT: Fetched primary data is valid. [2025-10-20T20:22:06.860822] [DEBUG] [LOCAL ] OCFT: Saving primary data. [2025-10-20T20:22:06.861247] [DEBUG] [LOCAL ] OCFT: Trimmed 11648 records to 7276 (keeping 200 days of data) [2025-10-20T20:22:06.900121] [DEBUG] [LOCAL ] OCFT: Primary data saved (7276 records). [2025-10-20T20:22:06.900241] [INFO ] [MAIN ] OCFT: Full history saved. [2025-10-20T20:22:06.900326] [INFO ] [MAIN ] Update Progress: 1570/2367 symbols completed. OCFT took 18.8s. Estimated remaining: 1h 42m 34s. Rate: 7.8 symbols/min [2025-10-20T20:22:07.071190] [INFO ] [MAIN ] OEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:07.106713] [DEBUG] [LOCAL ] OEC: Loaded 37875 existing records from primary storage. [2025-10-20T20:22:07.106994] [INFO ] [MAIN ] OEC: Loaded 37875 existing records. [2025-10-20T20:22:07.296120] [WARN ] [MAIN ] OEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:07.296336] [INFO ] [MAIN ] OEC: Wiping existing primary data before full refresh. [2025-10-20T20:22:07.296383] [INFO ] [LOCAL ] OEC: Starting deletion of primary data. [2025-10-20T20:22:07.296836] [DEBUG] [LOCAL ] OEC: Deleted primary data file. [2025-10-20T20:22:07.296892] [INFO ] [MAIN ] OEC: Starting full primary refresh. [2025-10-20T20:22:07.460270] [INFO ] [REMOTE ] OC: Fetching primary data for 2025-10. [2025-10-20T20:22:07.681980] [INFO ] [REMOTE ] OEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:07.868261] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T20:22:08.251411] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-02. [2025-10-20T20:22:08.300333] [INFO ] [REMOTE ] OC: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:22:08.307024] [INFO ] [REMOTE ] OC: Full fetch complete (64403 records). [2025-10-20T20:22:08.307168] [DEBUG] [VALIDATOR ] OC: Validating primary data. [2025-10-20T20:22:08.307816] [FATAL] [VALIDATOR ] OC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:22:08.307902] [FATAL] [MAIN ] OC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:22:08.307945] [WARN ] [MAIN ] OC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:22:08.308006] [INFO ] [LOCAL ] OC: Starting deletion of primary data. [2025-10-20T20:22:08.308073] [DEBUG] [LOCAL ] OC: No primary data file found to delete. [2025-10-20T20:22:08.308108] [INFO ] [LOCAL ] OC: Starting deletion of derivative data. [2025-10-20T20:22:08.308379] [DEBUG] [LOCAL ] OC: Deleted derivative data file. [2025-10-20T20:22:08.308471] [INFO ] [MAIN ] Update Progress: 1571/2367 symbols completed. OC took 38.1s. Estimated remaining: 1h 42m 23s. Rate: 7.8 symbols/min [2025-10-20T20:22:08.480698] [INFO ] [MAIN ] OFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:08.501687] [DEBUG] [LOCAL ] OFG: Loaded 21171 existing records from primary storage. [2025-10-20T20:22:08.502017] [INFO ] [MAIN ] OFG: Loaded 21171 existing records. [2025-10-20T20:22:08.700059] [WARN ] [MAIN ] OFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:08.700791] [INFO ] [MAIN ] OFG: Wiping existing primary data before full refresh. [2025-10-20T20:22:08.700852] [INFO ] [LOCAL ] OFG: Starting deletion of primary data. [2025-10-20T20:22:08.701586] [DEBUG] [LOCAL ] OFG: Deleted primary data file. [2025-10-20T20:22:08.701704] [INFO ] [MAIN ] OFG: Starting full primary refresh. [2025-10-20T20:22:08.968893] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T20:22:09.063777] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-07. [2025-10-20T20:22:09.072559] [INFO ] [REMOTE ] OFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:09.851400] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-01. [2025-10-20T20:22:10.359757] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T20:22:10.651399] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-03. [2025-10-20T20:22:11.199878] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:22:11.451348] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-01. [2025-10-20T20:22:11.504960] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:22:12.251391] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-08. [2025-10-20T20:22:12.725549] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T20:22:13.051457] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-04. [2025-10-20T20:22:13.710844] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T20:22:13.851341] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-02. [2025-10-20T20:22:13.939075] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T20:22:14.651374] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-09. [2025-10-20T20:22:15.256934] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T20:22:15.451410] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-03. [2025-10-20T20:22:16.260948] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-02. [2025-10-20T20:22:16.265621] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T20:22:17.009949] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T20:22:17.051330] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-04. [2025-10-20T20:22:17.243259] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T20:22:17.851431] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-05. [2025-10-20T20:22:18.569332] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T20:22:18.592726] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T20:22:18.651329] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-05. [2025-10-20T20:22:19.451404] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-06. [2025-10-20T20:22:20.251394] [INFO ] [REMOTE ] ODV: Fetching primary data for 2025-10. [2025-10-20T20:22:20.352447] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T20:22:20.373084] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T20:22:20.959035] [INFO ] [REMOTE ] ODV: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:22:20.966131] [INFO ] [REMOTE ] ODV: Full fetch complete (35264 records). [2025-10-20T20:22:20.966289] [DEBUG] [VALIDATOR ] ODV: Validating primary data. [2025-10-20T20:22:20.967020] [FATAL] [VALIDATOR ] ODV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:22:20.967107] [FATAL] [MAIN ] ODV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:22:20.967151] [WARN ] [MAIN ] ODV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:22:20.967216] [INFO ] [LOCAL ] ODV: Starting deletion of primary data. [2025-10-20T20:22:20.967285] [DEBUG] [LOCAL ] ODV: No primary data file found to delete. [2025-10-20T20:22:20.967351] [INFO ] [LOCAL ] ODV: Starting deletion of derivative data. [2025-10-20T20:22:20.967485] [DEBUG] [LOCAL ] ODV: Deleted derivative data file. [2025-10-20T20:22:20.967560] [INFO ] [MAIN ] Update Progress: 1572/2367 symbols completed. ODV took 29.2s. Estimated remaining: 1h 42m 18s. Rate: 7.8 symbols/min [2025-10-20T20:22:21.056700] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-07. [2025-10-20T20:22:21.144734] [INFO ] [MAIN ] OGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:21.185788] [DEBUG] [LOCAL ] OGE: Loaded 46620 existing records from primary storage. [2025-10-20T20:22:21.186093] [INFO ] [MAIN ] OGE: Loaded 46620 existing records. [2025-10-20T20:22:21.358922] [WARN ] [MAIN ] OGE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:21.359142] [INFO ] [MAIN ] OGE: Wiping existing primary data before full refresh. [2025-10-20T20:22:21.359188] [INFO ] [LOCAL ] OGE: Starting deletion of primary data. [2025-10-20T20:22:21.359500] [DEBUG] [LOCAL ] OGE: Deleted primary data file. [2025-10-20T20:22:21.359537] [INFO ] [MAIN ] OGE: Starting full primary refresh. [2025-10-20T20:22:21.717752] [INFO ] [REMOTE ] OGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:21.851432] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-01. [2025-10-20T20:22:21.969580] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T20:22:22.651477] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-03. [2025-10-20T20:22:23.451379] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-08. [2025-10-20T20:22:23.637065] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:22:23.971580] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T20:22:24.075143] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T20:22:24.251395] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-09. [2025-10-20T20:22:25.052420] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-04. [2025-10-20T20:22:25.057201] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:22:25.851438] [INFO ] [REMOTE ] ODC: Fetching primary data for 2025-10. [2025-10-20T20:22:26.461524] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T20:22:26.651377] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-05. [2025-10-20T20:22:26.668069] [INFO ] [REMOTE ] ODC: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T20:22:26.670249] [INFO ] [REMOTE ] ODC: Full fetch complete (11360 records). [2025-10-20T20:22:26.670330] [DEBUG] [VALIDATOR ] ODC: Validating primary data. [2025-10-20T20:22:27.024070] [DEBUG] [VALIDATOR ] ODC: All expected trading days are present. [2025-10-20T20:22:27.024177] [DEBUG] [VALIDATOR ] ODC: Primary data validation passed. [2025-10-20T20:22:27.024446] [INFO ] [MAIN ] ODC: Fetched primary data is valid. [2025-10-20T20:22:27.024520] [DEBUG] [LOCAL ] ODC: Saving primary data. [2025-10-20T20:22:27.025003] [DEBUG] [LOCAL ] ODC: Trimmed 11360 records to 8628 (keeping 200 days of data) [2025-10-20T20:22:27.078730] [DEBUG] [LOCAL ] ODC: Primary data saved (8628 records). [2025-10-20T20:22:27.078869] [INFO ] [MAIN ] ODC: Full history saved. [2025-10-20T20:22:27.078946] [INFO ] [MAIN ] Update Progress: 1573/2367 symbols completed. ODC took 35.3s. Estimated remaining: 1h 42m 9s. Rate: 7.8 symbols/min [2025-10-20T20:22:27.242987] [INFO ] [MAIN ] OGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:27.294594] [DEBUG] [LOCAL ] OGN: Loaded 53908 existing records from primary storage. [2025-10-20T20:22:27.295035] [INFO ] [MAIN ] OGN: Loaded 53908 existing records. [2025-10-20T20:22:27.453353] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-02. [2025-10-20T20:22:27.480280] [WARN ] [MAIN ] OGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:27.481210] [INFO ] [MAIN ] OGN: Wiping existing primary data before full refresh. [2025-10-20T20:22:27.481260] [INFO ] [LOCAL ] OGN: Starting deletion of primary data. [2025-10-20T20:22:27.481818] [DEBUG] [LOCAL ] OGN: Deleted primary data file. [2025-10-20T20:22:27.481958] [INFO ] [MAIN ] OGN: Starting full primary refresh. [2025-10-20T20:22:27.881638] [INFO ] [REMOTE ] OGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:27.896045] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T20:22:28.251408] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-01. [2025-10-20T20:22:29.051352] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-06. [2025-10-20T20:22:29.321434] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T20:22:29.851369] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-06. [2025-10-20T20:22:30.052854] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T20:22:30.149318] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T20:22:30.651366] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-02. [2025-10-20T20:22:30.934077] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T20:22:31.451369] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-07. [2025-10-20T20:22:32.251356] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-03. [2025-10-20T20:22:32.575405] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-02 (1.92 seconds). [2025-10-20T20:22:32.912342] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T20:22:33.051376] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-08. [2025-10-20T20:22:33.779689] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T20:22:33.851417] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-03. [2025-10-20T20:22:34.596711] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T20:22:34.651377] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-04. [2025-10-20T20:22:35.451423] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-07. [2025-10-20T20:22:35.806107] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T20:22:36.251422] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-04. [2025-10-20T20:22:36.297228] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T20:22:36.601365] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T20:22:37.051450] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-08. [2025-10-20T20:22:37.851343] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-09. [2025-10-20T20:22:38.307557] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-04 (2.06 seconds). [2025-10-20T20:22:38.331278] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T20:22:38.651435] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-09. [2025-10-20T20:22:39.251430] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T20:22:39.451379] [INFO ] [REMOTE ] OEC: Fetching primary data for 2025-10. [2025-10-20T20:22:39.787540] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T20:22:40.251376] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-05. [2025-10-20T20:22:40.794276] [INFO ] [REMOTE ] OEC: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T20:22:40.800215] [INFO ] [REMOTE ] OEC: Full fetch complete (53453 records). [2025-10-20T20:22:40.800324] [DEBUG] [VALIDATOR ] OEC: Validating primary data. [2025-10-20T20:22:41.061061] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-05. [2025-10-20T20:22:41.196896] [DEBUG] [VALIDATOR ] OEC: All expected trading days are present. [2025-10-20T20:22:41.196981] [DEBUG] [VALIDATOR ] OEC: Primary data validation passed. [2025-10-20T20:22:41.197832] [INFO ] [MAIN ] OEC: Fetched primary data is valid. [2025-10-20T20:22:41.197896] [DEBUG] [LOCAL ] OEC: Saving primary data. [2025-10-20T20:22:41.199470] [DEBUG] [LOCAL ] OEC: Trimmed 53453 records to 37290 (keeping 200 days of data) [2025-10-20T20:22:41.425378] [DEBUG] [LOCAL ] OEC: Primary data saved (37290 records). [2025-10-20T20:22:41.425523] [INFO ] [MAIN ] OEC: Full history saved. [2025-10-20T20:22:41.425604] [INFO ] [MAIN ] Update Progress: 1574/2367 symbols completed. OEC took 34.5s. Estimated remaining: 1h 42m 5s. Rate: 7.8 symbols/min [2025-10-20T20:22:41.605311] [INFO ] [MAIN ] OGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:41.637416] [DEBUG] [LOCAL ] OGS: Loaded 34532 existing records from primary storage. [2025-10-20T20:22:41.637729] [INFO ] [MAIN ] OGS: Loaded 34532 existing records. [2025-10-20T20:22:41.819080] [WARN ] [MAIN ] OGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:41.819340] [INFO ] [MAIN ] OGS: Wiping existing primary data before full refresh. [2025-10-20T20:22:41.819393] [INFO ] [LOCAL ] OGS: Starting deletion of primary data. [2025-10-20T20:22:41.819828] [DEBUG] [LOCAL ] OGS: Deleted primary data file. [2025-10-20T20:22:41.819886] [INFO ] [MAIN ] OGS: Starting full primary refresh. [2025-10-20T20:22:41.862530] [INFO ] [REMOTE ] OFG: Fetching primary data for 2025-10. [2025-10-20T20:22:42.187482] [INFO ] [REMOTE ] OGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:42.213185] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-05 (1.96 seconds). [2025-10-20T20:22:42.479046] [INFO ] [REMOTE ] OFG: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:22:42.482027] [INFO ] [REMOTE ] OFG: Full fetch complete (33618 records). [2025-10-20T20:22:42.482101] [DEBUG] [VALIDATOR ] OFG: Validating primary data. [2025-10-20T20:22:42.482525] [FATAL] [VALIDATOR ] OFG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:22:42.482605] [FATAL] [MAIN ] OFG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:22:42.482674] [WARN ] [MAIN ] OFG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:22:42.482748] [INFO ] [LOCAL ] OFG: Starting deletion of primary data. [2025-10-20T20:22:42.482809] [DEBUG] [LOCAL ] OFG: No primary data file found to delete. [2025-10-20T20:22:42.482843] [INFO ] [LOCAL ] OFG: Starting deletion of derivative data. [2025-10-20T20:22:42.483039] [DEBUG] [LOCAL ] OFG: Deleted derivative data file. [2025-10-20T20:22:42.483123] [INFO ] [MAIN ] Update Progress: 1575/2367 symbols completed. OFG took 34.2s. Estimated remaining: 1h 41m 54s. Rate: 7.8 symbols/min [2025-10-20T20:22:42.652197] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-06. [2025-10-20T20:22:42.652136] [INFO ] [MAIN ] OHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:22:42.700114] [DEBUG] [LOCAL ] OHI: Loaded 52674 existing records from primary storage. [2025-10-20T20:22:42.700416] [INFO ] [MAIN ] OHI: Loaded 52674 existing records. [2025-10-20T20:22:42.881012] [WARN ] [MAIN ] OHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:22:42.881344] [INFO ] [MAIN ] OHI: Wiping existing primary data before full refresh. [2025-10-20T20:22:42.881410] [INFO ] [LOCAL ] OHI: Starting deletion of primary data. [2025-10-20T20:22:42.882050] [DEBUG] [LOCAL ] OHI: Deleted primary data file. [2025-10-20T20:22:42.882106] [INFO ] [MAIN ] OHI: Starting full primary refresh. [2025-10-20T20:22:43.266605] [INFO ] [REMOTE ] OHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:22:43.271548] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T20:22:43.451361] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-06. [2025-10-20T20:22:44.256495] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-01. [2025-10-20T20:22:44.274402] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T20:22:45.051423] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-07. [2025-10-20T20:22:45.202258] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T20:22:45.851436] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-01. [2025-10-20T20:22:45.982446] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T20:22:46.651419] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-02. [2025-10-20T20:22:46.746423] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T20:22:47.003350] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T20:22:47.451375] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-08. [2025-10-20T20:22:48.251463] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-07. [2025-10-20T20:22:48.413168] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T20:22:49.051363] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-03. [2025-10-20T20:22:49.154190] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:22:49.851362] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-09. [2025-10-20T20:22:50.213068] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T20:22:50.548033] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:22:50.651391] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-08. [2025-10-20T20:22:51.433124] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T20:22:51.451339] [INFO ] [REMOTE ] OGE: Fetching primary data for 2025-10. [2025-10-20T20:22:52.251393] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-02. [2025-10-20T20:22:52.500635] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T20:22:53.051435] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-09. [2025-10-20T20:22:53.851442] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-04. [2025-10-20T20:23:05.316528] [INFO ] [REMOTE ] OGE: Incremental fetch complete for 2025-10 (13.87 seconds). [2025-10-20T20:23:05.322757] [INFO ] [REMOTE ] OGE: Full fetch complete (66483 records). [2025-10-20T20:23:05.322873] [DEBUG] [VALIDATOR ] OGE: Validating primary data. [2025-10-20T20:23:05.323572] [FATAL] [VALIDATOR ] OGE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:05.323683] [FATAL] [MAIN ] OGE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:05.323734] [WARN ] [MAIN ] OGE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:05.323773] [INFO ] [LOCAL ] OGE: Starting deletion of primary data. [2025-10-20T20:23:05.323827] [DEBUG] [LOCAL ] OGE: No primary data file found to delete. [2025-10-20T20:23:05.323860] [INFO ] [LOCAL ] OGE: Starting deletion of derivative data. [2025-10-20T20:23:05.323984] [DEBUG] [LOCAL ] OGE: Deleted derivative data file. [2025-10-20T20:23:05.324599] [INFO ] [MAIN ] Update Progress: 1576/2367 symbols completed. OGE took 44.4s. Estimated remaining: 1h 41m 54s. Rate: 7.8 symbols/min [2025-10-20T20:23:05.513183] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-04 (11.66 seconds). [2025-10-20T20:23:05.539636] [INFO ] [MAIN ] OI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:05.552331] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-05. [2025-10-20T20:23:05.573255] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-02 (13.32 seconds). [2025-10-20T20:23:05.581284] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-03. [2025-10-20T20:23:05.616926] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-09 (12.57 seconds). [2025-10-20T20:23:05.628993] [INFO ] [REMOTE ] OGN: Fetching primary data for 2025-10. [2025-10-20T20:23:05.668676] [DEBUG] [LOCAL ] OI: Loaded 47958 existing records from primary storage. [2025-10-20T20:23:05.669171] [INFO ] [MAIN ] OI: Loaded 47958 existing records. [2025-10-20T20:23:05.851388] [WARN ] [MAIN ] OI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:05.851742] [INFO ] [MAIN ] OI: Wiping existing primary data before full refresh. [2025-10-20T20:23:05.851803] [INFO ] [LOCAL ] OI: Starting deletion of primary data. [2025-10-20T20:23:05.852243] [DEBUG] [LOCAL ] OI: Deleted primary data file. [2025-10-20T20:23:05.852463] [INFO ] [MAIN ] OI: Starting full primary refresh. [2025-10-20T20:23:06.212428] [INFO ] [REMOTE ] OI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:06.212823] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-01. [2025-10-20T20:23:06.598852] [INFO ] [REMOTE ] OGN: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T20:23:06.606206] [INFO ] [REMOTE ] OGN: Full fetch complete (76573 records). [2025-10-20T20:23:06.606355] [DEBUG] [VALIDATOR ] OGN: Validating primary data. [2025-10-20T20:23:06.607187] [FATAL] [VALIDATOR ] OGN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:06.607287] [FATAL] [MAIN ] OGN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:06.607358] [WARN ] [MAIN ] OGN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:06.607456] [INFO ] [LOCAL ] OGN: Starting deletion of primary data. [2025-10-20T20:23:06.607537] [DEBUG] [LOCAL ] OGN: No primary data file found to delete. [2025-10-20T20:23:06.607577] [INFO ] [LOCAL ] OGN: Starting deletion of derivative data. [2025-10-20T20:23:06.607738] [DEBUG] [LOCAL ] OGN: Deleted derivative data file. [2025-10-20T20:23:06.607829] [INFO ] [MAIN ] Update Progress: 1577/2367 symbols completed. OGN took 39.5s. Estimated remaining: 1h 41m 43s. Rate: 7.8 symbols/min [2025-10-20T20:23:06.778025] [INFO ] [MAIN ] OIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:06.792600] [DEBUG] [LOCAL ] OIA: Loaded 13271 existing records from primary storage. [2025-10-20T20:23:06.792858] [INFO ] [MAIN ] OIA: Loaded 13271 existing records. [2025-10-20T20:23:06.974234] [WARN ] [MAIN ] OIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:06.974509] [INFO ] [MAIN ] OIA: Wiping existing primary data before full refresh. [2025-10-20T20:23:06.974561] [INFO ] [LOCAL ] OIA: Starting deletion of primary data. [2025-10-20T20:23:06.974776] [DEBUG] [LOCAL ] OIA: Deleted primary data file. [2025-10-20T20:23:06.974831] [INFO ] [MAIN ] OIA: Starting full primary refresh. [2025-10-20T20:23:07.283127] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T20:23:07.305682] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-04. [2025-10-20T20:23:07.362582] [INFO ] [REMOTE ] OIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:07.362883] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-01. [2025-10-20T20:23:07.769971] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-05 (2.22 seconds). [2025-10-20T20:23:07.781086] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-06. [2025-10-20T20:23:08.051516] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T20:23:08.060340] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-02. [2025-10-20T20:23:08.338546] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T20:23:08.341979] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-02. [2025-10-20T20:23:08.865838] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T20:23:08.873422] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-05. [2025-10-20T20:23:09.137698] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T20:23:09.142344] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-03. [2025-10-20T20:23:09.606995] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T20:23:09.622961] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-07. [2025-10-20T20:23:09.740114] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T20:23:09.750072] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-03. [2025-10-20T20:23:10.136449] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T20:23:10.140877] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-04. [2025-10-20T20:23:10.301208] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T20:23:10.310178] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-06. [2025-10-20T20:23:11.110671] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-04 (0.97 seconds). [2025-10-20T20:23:11.114200] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-05. [2025-10-20T20:23:11.530588] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T20:23:11.542897] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-08. [2025-10-20T20:23:11.609540] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T20:23:11.620620] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-04. [2025-10-20T20:23:11.735021] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T20:23:11.742323] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-07. [2025-10-20T20:23:11.984163] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:23:11.987000] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-06. [2025-10-20T20:23:12.869109] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T20:23:12.872523] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-07. [2025-10-20T20:23:13.053115] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T20:23:13.061496] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-08. [2025-10-20T20:23:13.152831] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T20:23:13.163865] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-09. [2025-10-20T20:23:13.298501] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:23:13.308889] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-05. [2025-10-20T20:23:13.641091] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T20:23:13.851361] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-08. [2025-10-20T20:23:14.569588] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:23:14.651384] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-09. [2025-10-20T20:23:14.892435] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T20:23:14.935023] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T20:23:14.965217] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T20:23:15.451379] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-06. [2025-10-20T20:23:15.528696] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:23:16.251379] [INFO ] [REMOTE ] OGS: Fetching primary data for 2025-10. [2025-10-20T20:23:16.919715] [INFO ] [REMOTE ] OGS: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T20:23:16.924083] [INFO ] [REMOTE ] OGS: Full fetch complete (47658 records). [2025-10-20T20:23:16.924177] [DEBUG] [VALIDATOR ] OGS: Validating primary data. [2025-10-20T20:23:16.924820] [FATAL] [VALIDATOR ] OGS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:16.924902] [FATAL] [MAIN ] OGS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:16.924949] [WARN ] [MAIN ] OGS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:16.925015] [INFO ] [LOCAL ] OGS: Starting deletion of primary data. [2025-10-20T20:23:16.925082] [DEBUG] [LOCAL ] OGS: No primary data file found to delete. [2025-10-20T20:23:16.925126] [INFO ] [LOCAL ] OGS: Starting deletion of derivative data. [2025-10-20T20:23:16.925268] [DEBUG] [LOCAL ] OGS: Deleted derivative data file. [2025-10-20T20:23:16.925349] [INFO ] [MAIN ] Update Progress: 1578/2367 symbols completed. OGS took 35.5s. Estimated remaining: 1h 41m 36s. Rate: 7.8 symbols/min [2025-10-20T20:23:17.061576] [INFO ] [REMOTE ] OHI: Fetching primary data for 2025-10. [2025-10-20T20:23:17.112894] [INFO ] [MAIN ] OII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:17.139431] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T20:23:17.181054] [DEBUG] [LOCAL ] OII: Loaded 42618 existing records from primary storage. [2025-10-20T20:23:17.181357] [INFO ] [MAIN ] OII: Loaded 42618 existing records. [2025-10-20T20:23:17.353144] [WARN ] [MAIN ] OII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:17.353369] [INFO ] [MAIN ] OII: Wiping existing primary data before full refresh. [2025-10-20T20:23:17.353417] [INFO ] [LOCAL ] OII: Starting deletion of primary data. [2025-10-20T20:23:17.353638] [DEBUG] [LOCAL ] OII: Deleted primary data file. [2025-10-20T20:23:17.353728] [INFO ] [MAIN ] OII: Starting full primary refresh. [2025-10-20T20:23:17.705838] [INFO ] [REMOTE ] OII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:17.851346] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-01. [2025-10-20T20:23:17.897847] [INFO ] [REMOTE ] OHI: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:23:17.905071] [INFO ] [REMOTE ] OHI: Full fetch complete (74968 records). [2025-10-20T20:23:17.905221] [DEBUG] [VALIDATOR ] OHI: Validating primary data. [2025-10-20T20:23:17.906113] [FATAL] [VALIDATOR ] OHI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:17.906207] [FATAL] [MAIN ] OHI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:17.906250] [WARN ] [MAIN ] OHI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:17.906289] [INFO ] [LOCAL ] OHI: Starting deletion of primary data. [2025-10-20T20:23:17.906349] [DEBUG] [LOCAL ] OHI: No primary data file found to delete. [2025-10-20T20:23:17.906383] [INFO ] [LOCAL ] OHI: Starting deletion of derivative data. [2025-10-20T20:23:17.906538] [DEBUG] [LOCAL ] OHI: Deleted derivative data file. [2025-10-20T20:23:17.906630] [INFO ] [MAIN ] Update Progress: 1579/2367 symbols completed. OHI took 35.4s. Estimated remaining: 1h 41m 25s. Rate: 7.8 symbols/min [2025-10-20T20:23:18.080310] [INFO ] [MAIN ] OIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:18.116841] [DEBUG] [LOCAL ] OIS: Loaded 40550 existing records from primary storage. [2025-10-20T20:23:18.117135] [INFO ] [MAIN ] OIS: Loaded 40550 existing records. [2025-10-20T20:23:18.293110] [WARN ] [MAIN ] OIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:18.293330] [INFO ] [MAIN ] OIS: Wiping existing primary data before full refresh. [2025-10-20T20:23:18.293375] [INFO ] [LOCAL ] OIS: Starting deletion of primary data. [2025-10-20T20:23:18.293569] [DEBUG] [LOCAL ] OIS: Deleted primary data file. [2025-10-20T20:23:18.293624] [INFO ] [MAIN ] OIS: Starting full primary refresh. [2025-10-20T20:23:18.646759] [INFO ] [REMOTE ] OIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:18.651337] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-01. [2025-10-20T20:23:19.367363] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T20:23:19.451384] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-07. [2025-10-20T20:23:20.104589] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T20:23:20.251376] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-02. [2025-10-20T20:23:21.051380] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-09. [2025-10-20T20:23:21.283858] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T20:23:21.696403] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T20:23:21.851366] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-03. [2025-10-20T20:23:22.142267] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:23:22.651379] [INFO ] [REMOTE ] OIA: Fetching primary data for 2025-10. [2025-10-20T20:23:23.400267] [INFO ] [REMOTE ] OIA: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:23:23.402144] [INFO ] [REMOTE ] OIA: Full fetch complete (20204 records). [2025-10-20T20:23:23.402212] [DEBUG] [VALIDATOR ] OIA: Validating primary data. [2025-10-20T20:23:23.456869] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-02. [2025-10-20T20:23:23.592060] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T20:23:23.805876] [DEBUG] [VALIDATOR ] OIA: All expected trading days are present. [2025-10-20T20:23:23.805955] [DEBUG] [VALIDATOR ] OIA: Primary data validation passed. [2025-10-20T20:23:23.806408] [INFO ] [MAIN ] OIA: Fetched primary data is valid. [2025-10-20T20:23:23.806447] [DEBUG] [LOCAL ] OIA: Saving primary data. [2025-10-20T20:23:23.807024] [DEBUG] [LOCAL ] OIA: Trimmed 20204 records to 13084 (keeping 200 days of data) [2025-10-20T20:23:23.881188] [DEBUG] [LOCAL ] OIA: Primary data saved (13084 records). [2025-10-20T20:23:23.881320] [INFO ] [MAIN ] OIA: Full history saved. [2025-10-20T20:23:23.881398] [INFO ] [MAIN ] Update Progress: 1580/2367 symbols completed. OIA took 17.3s. Estimated remaining: 1h 41m 17s. Rate: 7.8 symbols/min [2025-10-20T20:23:24.054792] [INFO ] [MAIN ] OKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:24.103802] [DEBUG] [LOCAL ] OKE: Loaded 54350 existing records from primary storage. [2025-10-20T20:23:24.104117] [INFO ] [MAIN ] OKE: Loaded 54350 existing records. [2025-10-20T20:23:24.253709] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-08. [2025-10-20T20:23:24.282152] [WARN ] [MAIN ] OKE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:24.282538] [INFO ] [MAIN ] OKE: Wiping existing primary data before full refresh. [2025-10-20T20:23:24.282607] [INFO ] [LOCAL ] OKE: Starting deletion of primary data. [2025-10-20T20:23:24.282922] [DEBUG] [LOCAL ] OKE: Deleted primary data file. [2025-10-20T20:23:24.282977] [INFO ] [MAIN ] OKE: Starting full primary refresh. [2025-10-20T20:23:24.633710] [INFO ] [REMOTE ] OKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:25.051364] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-01. [2025-10-20T20:23:25.099599] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T20:23:25.851413] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-03. [2025-10-20T20:23:26.073168] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T20:23:26.651392] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-04. [2025-10-20T20:23:26.998825] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T20:23:27.286503] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T20:23:27.451386] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-09. [2025-10-20T20:23:28.251399] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-04. [2025-10-20T20:23:28.302468] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T20:23:29.052318] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-02. [2025-10-20T20:23:29.058580] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T20:23:29.682313] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T20:23:29.851384] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-05. [2025-10-20T20:23:30.651401] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-05. [2025-10-20T20:23:30.840088] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T20:23:31.431769] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T20:23:31.451336] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-06. [2025-10-20T20:23:31.913665] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T20:23:32.251424] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-06. [2025-10-20T20:23:32.670681] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T20:23:33.051379] [INFO ] [REMOTE ] OI: Fetching primary data for 2025-10. [2025-10-20T20:23:33.805049] [INFO ] [REMOTE ] OI: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:23:33.811724] [INFO ] [REMOTE ] OI: Full fetch complete (67956 records). [2025-10-20T20:23:33.811834] [DEBUG] [VALIDATOR ] OI: Validating primary data. [2025-10-20T20:23:33.812534] [FATAL] [VALIDATOR ] OI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:33.812614] [FATAL] [MAIN ] OI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:33.812695] [WARN ] [MAIN ] OI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:33.812785] [INFO ] [LOCAL ] OI: Starting deletion of primary data. [2025-10-20T20:23:33.812859] [DEBUG] [LOCAL ] OI: No primary data file found to delete. [2025-10-20T20:23:33.812894] [INFO ] [LOCAL ] OI: Starting deletion of derivative data. [2025-10-20T20:23:33.813029] [DEBUG] [LOCAL ] OI: Deleted derivative data file. [2025-10-20T20:23:33.813097] [INFO ] [MAIN ] Update Progress: 1581/2367 symbols completed. OI took 28.5s. Estimated remaining: 1h 41m 10s. Rate: 7.8 symbols/min [2025-10-20T20:23:33.856515] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-07. [2025-10-20T20:23:34.013682] [INFO ] [MAIN ] OKLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:34.014329] [DEBUG] [LOCAL ] OKLO: No existing data file found in primary storage. [2025-10-20T20:23:34.014457] [WARN ] [MAIN ] OKLO: No existing data found, performing full refresh instead. [2025-10-20T20:23:34.016238] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T20:23:34.028021] [INFO ] [MAIN ] OKLO: Wiping existing primary data before full refresh. [2025-10-20T20:23:34.028269] [INFO ] [LOCAL ] OKLO: Starting deletion of primary data. [2025-10-20T20:23:34.028336] [DEBUG] [LOCAL ] OKLO: No primary data file found to delete. [2025-10-20T20:23:34.028731] [INFO ] [MAIN ] OKLO: Starting full primary refresh. [2025-10-20T20:23:34.395965] [INFO ] [REMOTE ] OKLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:34.651364] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-01. [2025-10-20T20:23:35.451344] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-07. [2025-10-20T20:23:35.499915] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T20:23:36.251431] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-03. [2025-10-20T20:23:37.036396] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T20:23:37.051333] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-08. [2025-10-20T20:23:37.328711] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-01 (2.68 seconds). [2025-10-20T20:23:37.851354] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-08. [2025-10-20T20:23:38.253001] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T20:23:38.600916] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T20:23:38.651346] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-09. [2025-10-20T20:23:39.161348] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T20:23:39.451371] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-04. [2025-10-20T20:23:39.701691] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T20:23:40.251376] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-02. [2025-10-20T20:23:41.051379] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-09. [2025-10-20T20:23:41.489150] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-04 (2.04 seconds). [2025-10-20T20:23:41.851377] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-05. [2025-10-20T20:23:42.534863] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T20:23:42.571472] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-02 (2.32 seconds). [2025-10-20T20:23:42.651401] [INFO ] [REMOTE ] OIS: Fetching primary data for 2025-10. [2025-10-20T20:23:43.451383] [INFO ] [REMOTE ] OII: Fetching primary data for 2025-10. [2025-10-20T20:23:43.534877] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T20:23:43.725850] [INFO ] [REMOTE ] OIS: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T20:23:43.731437] [INFO ] [REMOTE ] OIS: Full fetch complete (57198 records). [2025-10-20T20:23:43.731543] [DEBUG] [VALIDATOR ] OIS: Validating primary data. [2025-10-20T20:23:44.153788] [INFO ] [REMOTE ] OII: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:23:44.152853] [DEBUG] [VALIDATOR ] OIS: All expected trading days are present. [2025-10-20T20:23:44.159165] [DEBUG] [VALIDATOR ] OIS: Primary data validation passed. [2025-10-20T20:23:44.160318] [INFO ] [MAIN ] OIS: Fetched primary data is valid. [2025-10-20T20:23:44.160590] [DEBUG] [LOCAL ] OIS: Saving primary data. [2025-10-20T20:23:44.161121] [INFO ] [REMOTE ] OII: Full fetch complete (61446 records). [2025-10-20T20:23:44.161531] [DEBUG] [VALIDATOR ] OII: Validating primary data. [2025-10-20T20:23:44.162408] [DEBUG] [LOCAL ] OIS: Trimmed 57198 records to 39917 (keeping 200 days of data) [2025-10-20T20:23:44.163037] [FATAL] [VALIDATOR ] OII: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:23:44.163498] [FATAL] [MAIN ] OII: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:23:44.163857] [WARN ] [MAIN ] OII: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:23:44.164009] [INFO ] [LOCAL ] OII: Starting deletion of primary data. [2025-10-20T20:23:44.164106] [DEBUG] [LOCAL ] OII: No primary data file found to delete. [2025-10-20T20:23:44.164176] [INFO ] [LOCAL ] OII: Starting deletion of derivative data. [2025-10-20T20:23:44.164395] [DEBUG] [LOCAL ] OII: Deleted derivative data file. [2025-10-20T20:23:44.164531] [INFO ] [MAIN ] Update Progress: 1582/2367 symbols completed. OII took 27.2s. Estimated remaining: 1h 41m 4s. Rate: 7.8 symbols/min [2025-10-20T20:23:44.256538] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-06. [2025-10-20T20:23:44.521280] [INFO ] [MAIN ] OLN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:44.581372] [DEBUG] [LOCAL ] OIS: Primary data saved (39917 records). [2025-10-20T20:23:44.581600] [INFO ] [MAIN ] OIS: Full history saved. [2025-10-20T20:23:44.581732] [INFO ] [MAIN ] Update Progress: 1583/2367 symbols completed. OIS took 26.7s. Estimated remaining: 1h 40m 52s. Rate: 7.8 symbols/min [2025-10-20T20:23:44.769679] [INFO ] [MAIN ] OLP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:23:44.789283] [DEBUG] [LOCAL ] OLN: Loaded 52723 existing records from primary storage. [2025-10-20T20:23:44.789685] [INFO ] [MAIN ] OLN: Loaded 52723 existing records. [2025-10-20T20:23:44.900730] [DEBUG] [LOCAL ] OLP: Loaded 10256 existing records from primary storage. [2025-10-20T20:23:44.901291] [INFO ] [MAIN ] OLP: Loaded 10256 existing records. [2025-10-20T20:23:45.054830] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-03. [2025-10-20T20:23:45.082015] [WARN ] [MAIN ] OLN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:45.089031] [INFO ] [MAIN ] OLN: Wiping existing primary data before full refresh. [2025-10-20T20:23:45.097217] [INFO ] [LOCAL ] OLN: Starting deletion of primary data. [2025-10-20T20:23:45.103499] [DEBUG] [LOCAL ] OLN: Deleted primary data file. [2025-10-20T20:23:45.103804] [INFO ] [MAIN ] OLN: Starting full primary refresh. [2025-10-20T20:23:45.288504] [WARN ] [MAIN ] OLP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:23:45.288783] [INFO ] [MAIN ] OLP: Wiping existing primary data before full refresh. [2025-10-20T20:23:45.289084] [INFO ] [LOCAL ] OLP: Starting deletion of primary data. [2025-10-20T20:23:45.304843] [DEBUG] [LOCAL ] OLP: Deleted primary data file. [2025-10-20T20:23:45.310033] [INFO ] [MAIN ] OLP: Starting full primary refresh. [2025-10-20T20:23:45.837033] [INFO ] [REMOTE ] OLN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:45.856447] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-01. [2025-10-20T20:23:45.920480] [INFO ] [REMOTE ] OLP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:23:46.384454] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T20:23:46.651425] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-01. [2025-10-20T20:23:47.451424] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-07. [2025-10-20T20:23:47.673876] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T20:23:47.993254] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-01 (2.14 seconds). [2025-10-20T20:23:48.037954] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-03 (2.98 seconds). [2025-10-20T20:23:48.251400] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-02. [2025-10-20T20:23:49.051430] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-04. [2025-10-20T20:23:49.236695] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T20:23:49.851412] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-03. [2025-10-20T20:23:49.998747] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-07 (2.55 seconds). [2025-10-20T20:23:50.651474] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-02. [2025-10-20T20:23:50.947576] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T20:23:51.451402] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-04. [2025-10-20T20:23:51.527507] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T20:23:52.251507] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-05. [2025-10-20T20:23:52.457140] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T20:23:52.467405] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T20:23:53.051375] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-03. [2025-10-20T20:23:53.851387] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-08. [2025-10-20T20:23:54.651356] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-05. [2025-10-20T20:23:54.929693] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T20:23:55.002773] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-05 (2.75 seconds). [2025-10-20T20:23:55.190333] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T20:23:55.451402] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-06. [2025-10-20T20:23:56.000515] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-08 (2.15 seconds). [2025-10-20T20:23:56.013083] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T20:23:56.251381] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-07. [2025-10-20T20:23:57.051343] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-06. [2025-10-20T20:23:57.123460] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T20:23:57.851426] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-04. [2025-10-20T20:23:58.651435] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-09. [2025-10-20T20:23:59.451366] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-08. [2025-10-20T20:23:59.677240] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:23:59.717001] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-06 (2.67 seconds). [2025-10-20T20:23:59.999514] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T20:24:00.251391] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-05. [2025-10-20T20:24:00.691199] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:24:01.051379] [INFO ] [REMOTE ] OKE: Fetching primary data for 2025-10. [2025-10-20T20:24:01.783357] [INFO ] [REMOTE ] OKE: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T20:24:01.791540] [INFO ] [REMOTE ] OKE: Full fetch complete (77283 records). [2025-10-20T20:24:01.791704] [DEBUG] [VALIDATOR ] OKE: Validating primary data. [2025-10-20T20:24:01.792498] [FATAL] [VALIDATOR ] OKE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:24:01.792587] [FATAL] [MAIN ] OKE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:24:01.792631] [WARN ] [MAIN ] OKE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:24:01.792720] [INFO ] [LOCAL ] OKE: Starting deletion of primary data. [2025-10-20T20:24:01.792820] [DEBUG] [LOCAL ] OKE: No primary data file found to delete. [2025-10-20T20:24:01.792874] [INFO ] [LOCAL ] OKE: Starting deletion of derivative data. [2025-10-20T20:24:01.793022] [DEBUG] [LOCAL ] OKE: Deleted derivative data file. [2025-10-20T20:24:01.793113] [INFO ] [MAIN ] Update Progress: 1584/2367 symbols completed. OKE took 37.9s. Estimated remaining: 1h 40m 49s. Rate: 7.8 symbols/min [2025-10-20T20:24:01.857028] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-09. [2025-10-20T20:24:01.984383] [INFO ] [MAIN ] OMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:02.017990] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:24:02.071485] [DEBUG] [LOCAL ] OMC: Loaded 54070 existing records from primary storage. [2025-10-20T20:24:02.071832] [INFO ] [MAIN ] OMC: Loaded 54070 existing records. [2025-10-20T20:24:02.256148] [WARN ] [MAIN ] OMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:02.256363] [INFO ] [MAIN ] OMC: Wiping existing primary data before full refresh. [2025-10-20T20:24:02.256427] [INFO ] [LOCAL ] OMC: Starting deletion of primary data. [2025-10-20T20:24:02.256696] [DEBUG] [LOCAL ] OMC: Deleted primary data file. [2025-10-20T20:24:02.256764] [INFO ] [MAIN ] OMC: Starting full primary refresh. [2025-10-20T20:24:02.629401] [INFO ] [REMOTE ] OMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:02.651331] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-01. [2025-10-20T20:24:02.786324] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:24:03.451380] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-07. [2025-10-20T20:24:04.251353] [INFO ] [REMOTE ] OLP: Fetching primary data for 2025-10. [2025-10-20T20:24:04.438074] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:24:05.051363] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-06. [2025-10-20T20:24:05.125151] [INFO ] [REMOTE ] OLP: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T20:24:05.127698] [INFO ] [REMOTE ] OLP: Full fetch complete (14285 records). [2025-10-20T20:24:05.127834] [DEBUG] [VALIDATOR ] OLP: Validating primary data. [2025-10-20T20:24:05.518609] [DEBUG] [VALIDATOR ] OLP: All expected trading days are present. [2025-10-20T20:24:05.518716] [DEBUG] [VALIDATOR ] OLP: Primary data validation passed. [2025-10-20T20:24:05.519041] [INFO ] [MAIN ] OLP: Fetched primary data is valid. [2025-10-20T20:24:05.519080] [DEBUG] [LOCAL ] OLP: Saving primary data. [2025-10-20T20:24:05.519547] [DEBUG] [LOCAL ] OLP: Trimmed 14285 records to 10116 (keeping 200 days of data) [2025-10-20T20:24:05.579303] [DEBUG] [LOCAL ] OLP: Primary data saved (10116 records). [2025-10-20T20:24:05.579420] [INFO ] [MAIN ] OLP: Full history saved. [2025-10-20T20:24:05.579495] [INFO ] [MAIN ] Update Progress: 1585/2367 symbols completed. OLP took 21.0s. Estimated remaining: 1h 40m 39s. Rate: 7.8 symbols/min [2025-10-20T20:24:05.751457] [INFO ] [MAIN ] OMF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:05.793336] [DEBUG] [LOCAL ] OMF: Loaded 47130 existing records from primary storage. [2025-10-20T20:24:05.793632] [INFO ] [MAIN ] OMF: Loaded 47130 existing records. [2025-10-20T20:24:05.859148] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-02. [2025-10-20T20:24:05.977805] [WARN ] [MAIN ] OMF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:05.978037] [INFO ] [MAIN ] OMF: Wiping existing primary data before full refresh. [2025-10-20T20:24:05.978117] [INFO ] [LOCAL ] OMF: Starting deletion of primary data. [2025-10-20T20:24:05.978341] [DEBUG] [LOCAL ] OMF: Deleted primary data file. [2025-10-20T20:24:05.978397] [INFO ] [MAIN ] OMF: Starting full primary refresh. [2025-10-20T20:24:06.332747] [INFO ] [REMOTE ] OMF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:06.501964] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-07 (3.05 seconds). [2025-10-20T20:24:06.651344] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-08. [2025-10-20T20:24:06.837381] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T20:24:07.451356] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-07. [2025-10-20T20:24:07.642479] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T20:24:08.251448] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-01. [2025-10-20T20:24:09.051720] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-03. [2025-10-20T20:24:09.434790] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T20:24:09.476098] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-08 (2.82 seconds). [2025-10-20T20:24:09.851428] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-09. [2025-10-20T20:24:09.975760] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T20:24:10.651425] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-02. [2025-10-20T20:24:10.949110] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T20:24:11.451391] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-08. [2025-10-20T20:24:12.251495] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-04. [2025-10-20T20:24:12.304078] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T20:24:12.599560] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-09 (2.75 seconds). [2025-10-20T20:24:12.955975] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T20:24:13.051368] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-09. [2025-10-20T20:24:13.851358] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-03. [2025-10-20T20:24:14.271971] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T20:24:14.651379] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-05. [2025-10-20T20:24:14.882541] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T20:24:15.451355] [INFO ] [REMOTE ] OLN: Fetching primary data for 2025-10. [2025-10-20T20:24:15.554189] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T20:24:16.232897] [INFO ] [REMOTE ] OLN: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:24:16.240826] [INFO ] [REMOTE ] OLN: Full fetch complete (74420 records). [2025-10-20T20:24:16.240998] [DEBUG] [VALIDATOR ] OLN: Validating primary data. [2025-10-20T20:24:16.242091] [FATAL] [VALIDATOR ] OLN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:24:16.242186] [FATAL] [MAIN ] OLN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:24:16.242230] [WARN ] [MAIN ] OLN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:24:16.242303] [INFO ] [LOCAL ] OLN: Starting deletion of primary data. [2025-10-20T20:24:16.242383] [DEBUG] [LOCAL ] OLN: No primary data file found to delete. [2025-10-20T20:24:16.242421] [INFO ] [LOCAL ] OLN: Starting deletion of derivative data. [2025-10-20T20:24:16.242595] [DEBUG] [LOCAL ] OLN: Deleted derivative data file. [2025-10-20T20:24:16.242708] [INFO ] [MAIN ] Update Progress: 1586/2367 symbols completed. OLN took 32.1s. Estimated remaining: 1h 40m 33s. Rate: 7.8 symbols/min [2025-10-20T20:24:16.256499] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-04. [2025-10-20T20:24:16.425454] [INFO ] [MAIN ] OMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:16.472634] [DEBUG] [LOCAL ] OMI: Loaded 46830 existing records from primary storage. [2025-10-20T20:24:16.473017] [INFO ] [MAIN ] OMI: Loaded 46830 existing records. [2025-10-20T20:24:16.652305] [WARN ] [MAIN ] OMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:16.652573] [INFO ] [MAIN ] OMI: Wiping existing primary data before full refresh. [2025-10-20T20:24:16.652685] [INFO ] [LOCAL ] OMI: Starting deletion of primary data. [2025-10-20T20:24:16.653151] [DEBUG] [LOCAL ] OMI: Deleted primary data file. [2025-10-20T20:24:16.653221] [INFO ] [MAIN ] OMI: Starting full primary refresh. [2025-10-20T20:24:16.812617] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-05 (2.16 seconds). [2025-10-20T20:24:17.055548] [INFO ] [REMOTE ] OKLO: Fetching primary data for 2025-10. [2025-10-20T20:24:17.078071] [INFO ] [REMOTE ] OMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:17.851351] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-01. [2025-10-20T20:24:17.872843] [INFO ] [REMOTE ] OKLO: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T20:24:17.880961] [INFO ] [REMOTE ] OKLO: Full fetch complete (77999 records). [2025-10-20T20:24:17.881091] [DEBUG] [VALIDATOR ] OKLO: Validating primary data. [2025-10-20T20:24:18.289540] [DEBUG] [VALIDATOR ] OKLO: All expected trading days are present. [2025-10-20T20:24:18.289710] [DEBUG] [VALIDATOR ] OKLO: Primary data validation passed. [2025-10-20T20:24:18.290861] [INFO ] [MAIN ] OKLO: Fetched primary data is valid. [2025-10-20T20:24:18.290939] [DEBUG] [LOCAL ] OKLO: Saving primary data. [2025-10-20T20:24:18.321527] [DEBUG] [LOCAL ] OKLO: Trimmed 77999 records to 53819 (keeping 200 days of data) [2025-10-20T20:24:18.322599] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T20:24:18.648669] [DEBUG] [LOCAL ] OKLO: Primary data saved (53819 records). [2025-10-20T20:24:18.648813] [INFO ] [MAIN ] OKLO: Full history saved. [2025-10-20T20:24:18.648891] [INFO ] [MAIN ] Update Progress: 1587/2367 symbols completed. OKLO took 44.8s. Estimated remaining: 1h 40m 23s. Rate: 7.8 symbols/min [2025-10-20T20:24:18.659932] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-06. [2025-10-20T20:24:18.825000] [INFO ] [MAIN ] ONIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:18.833225] [DEBUG] [LOCAL ] ONIT: Loaded 7728 existing records from primary storage. [2025-10-20T20:24:18.833391] [INFO ] [MAIN ] ONIT: Loaded 7728 existing records. [2025-10-20T20:24:19.011706] [WARN ] [MAIN ] ONIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:19.011907] [INFO ] [MAIN ] ONIT: Wiping existing primary data before full refresh. [2025-10-20T20:24:19.012044] [INFO ] [LOCAL ] ONIT: Starting deletion of primary data. [2025-10-20T20:24:19.018462] [DEBUG] [LOCAL ] ONIT: Deleted primary data file. [2025-10-20T20:24:19.029153] [INFO ] [MAIN ] ONIT: Starting full primary refresh. [2025-10-20T20:24:19.062038] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T20:24:19.386473] [INFO ] [REMOTE ] ONIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:19.451369] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-01. [2025-10-20T20:24:20.251409] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-05. [2025-10-20T20:24:20.316252] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T20:24:20.630089] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T20:24:21.051439] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-02. [2025-10-20T20:24:21.851393] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-02. [2025-10-20T20:24:22.017198] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:24:22.651376] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-06. [2025-10-20T20:24:22.724006] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T20:24:22.752962] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:24:23.451363] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-03. [2025-10-20T20:24:24.251365] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-03. [2025-10-20T20:24:24.353129] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T20:24:24.377737] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T20:24:25.051379] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-07. [2025-10-20T20:24:25.851492] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-07. [2025-10-20T20:24:25.921705] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T20:24:26.651411] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-04. [2025-10-20T20:24:26.759222] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T20:24:27.451419] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-04. [2025-10-20T20:24:27.625916] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T20:24:27.908119] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T20:24:28.251350] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-05. [2025-10-20T20:24:28.279908] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:24:29.051376] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-08. [2025-10-20T20:24:29.851448] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-08. [2025-10-20T20:24:29.924757] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T20:24:30.651392] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-05. [2025-10-20T20:24:30.948730] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T20:24:31.456598] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-06. [2025-10-20T20:24:31.457482] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T20:24:31.540322] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T20:24:32.251375] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-09. [2025-10-20T20:24:33.020437] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T20:24:33.051334] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-07. [2025-10-20T20:24:33.851407] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-09. [2025-10-20T20:24:33.894000] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T20:24:34.555393] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T20:24:34.651346] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-08. [2025-10-20T20:24:35.451350] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-06. [2025-10-20T20:24:35.688699] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T20:24:36.251435] [INFO ] [REMOTE ] OMC: Fetching primary data for 2025-10. [2025-10-20T20:24:36.394848] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T20:24:36.423245] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T20:24:36.998004] [INFO ] [REMOTE ] OMC: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:24:37.005329] [INFO ] [REMOTE ] OMC: Full fetch complete (76855 records). [2025-10-20T20:24:37.005440] [DEBUG] [VALIDATOR ] OMC: Validating primary data. [2025-10-20T20:24:37.006104] [FATAL] [VALIDATOR ] OMC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:24:37.006189] [FATAL] [MAIN ] OMC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:24:37.006233] [WARN ] [MAIN ] OMC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:24:37.006309] [INFO ] [LOCAL ] OMC: Starting deletion of primary data. [2025-10-20T20:24:37.006384] [DEBUG] [LOCAL ] OMC: No primary data file found to delete. [2025-10-20T20:24:37.006433] [INFO ] [LOCAL ] OMC: Starting deletion of derivative data. [2025-10-20T20:24:37.006763] [DEBUG] [LOCAL ] OMC: Deleted derivative data file. [2025-10-20T20:24:37.006881] [INFO ] [MAIN ] Update Progress: 1588/2367 symbols completed. OMC took 35.2s. Estimated remaining: 1h 40m 20s. Rate: 7.8 symbols/min [2025-10-20T20:24:37.060166] [INFO ] [REMOTE ] OMF: Fetching primary data for 2025-10. [2025-10-20T20:24:37.179021] [INFO ] [MAIN ] ONL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:37.207484] [DEBUG] [LOCAL ] ONL: Loaded 29936 existing records from primary storage. [2025-10-20T20:24:37.207760] [INFO ] [MAIN ] ONL: Loaded 29936 existing records. [2025-10-20T20:24:37.379086] [WARN ] [MAIN ] ONL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:37.379267] [INFO ] [MAIN ] ONL: Wiping existing primary data before full refresh. [2025-10-20T20:24:37.379312] [INFO ] [LOCAL ] ONL: Starting deletion of primary data. [2025-10-20T20:24:37.379514] [DEBUG] [LOCAL ] ONL: Deleted primary data file. [2025-10-20T20:24:37.379568] [INFO ] [MAIN ] ONL: Starting full primary refresh. [2025-10-20T20:24:37.723210] [INFO ] [REMOTE ] ONL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:37.851327] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-01. [2025-10-20T20:24:38.651337] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-07. [2025-10-20T20:24:38.691611] [INFO ] [REMOTE ] OMF: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T20:24:38.698862] [INFO ] [REMOTE ] OMF: Full fetch complete (67733 records). [2025-10-20T20:24:38.698966] [DEBUG] [VALIDATOR ] OMF: Validating primary data. [2025-10-20T20:24:39.088493] [DEBUG] [VALIDATOR ] OMF: All expected trading days are present. [2025-10-20T20:24:39.088583] [DEBUG] [VALIDATOR ] OMF: Primary data validation passed. [2025-10-20T20:24:39.089547] [INFO ] [MAIN ] OMF: Fetched primary data is valid. [2025-10-20T20:24:39.089633] [DEBUG] [LOCAL ] OMF: Saving primary data. [2025-10-20T20:24:39.090909] [DEBUG] [LOCAL ] OMF: Trimmed 67733 records to 46485 (keeping 200 days of data) [2025-10-20T20:24:39.204514] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:24:39.388982] [DEBUG] [LOCAL ] OMF: Primary data saved (46485 records). [2025-10-20T20:24:39.389207] [INFO ] [MAIN ] OMF: Full history saved. [2025-10-20T20:24:39.389328] [INFO ] [MAIN ] Update Progress: 1589/2367 symbols completed. OMF took 33.8s. Estimated remaining: 1h 40m 10s. Rate: 7.8 symbols/min [2025-10-20T20:24:39.452248] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-02. [2025-10-20T20:24:39.498732] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T20:24:39.569826] [INFO ] [MAIN ] ONON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:39.618209] [DEBUG] [LOCAL ] ONON: Loaded 54394 existing records from primary storage. [2025-10-20T20:24:39.618538] [INFO ] [MAIN ] ONON: Loaded 54394 existing records. [2025-10-20T20:24:39.790049] [WARN ] [MAIN ] ONON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:39.790246] [INFO ] [MAIN ] ONON: Wiping existing primary data before full refresh. [2025-10-20T20:24:39.790291] [INFO ] [LOCAL ] ONON: Starting deletion of primary data. [2025-10-20T20:24:39.790476] [DEBUG] [LOCAL ] ONON: Deleted primary data file. [2025-10-20T20:24:39.790513] [INFO ] [MAIN ] ONON: Starting full primary refresh. [2025-10-20T20:24:40.137262] [INFO ] [REMOTE ] ONON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:40.251364] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-01. [2025-10-20T20:24:40.662073] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T20:24:41.051370] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-03. [2025-10-20T20:24:41.851383] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-09. [2025-10-20T20:24:42.188286] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T20:24:42.647756] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T20:24:42.655475] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-08. [2025-10-20T20:24:43.239084] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:24:43.451362] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-04. [2025-10-20T20:24:43.505375] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T20:24:44.251427] [INFO ] [REMOTE ] OMI: Fetching primary data for 2025-10. [2025-10-20T20:24:45.033941] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T20:24:45.055737] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-02. [2025-10-20T20:24:45.060535] [INFO ] [REMOTE ] OMI: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T20:24:45.067556] [INFO ] [REMOTE ] OMI: Full fetch complete (65249 records). [2025-10-20T20:24:45.067718] [DEBUG] [VALIDATOR ] OMI: Validating primary data. [2025-10-20T20:24:45.068479] [FATAL] [VALIDATOR ] OMI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:24:45.068565] [FATAL] [MAIN ] OMI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:24:45.068609] [WARN ] [MAIN ] OMI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:24:45.068699] [INFO ] [LOCAL ] OMI: Starting deletion of primary data. [2025-10-20T20:24:45.068768] [DEBUG] [LOCAL ] OMI: No primary data file found to delete. [2025-10-20T20:24:45.068805] [INFO ] [LOCAL ] OMI: Starting deletion of derivative data. [2025-10-20T20:24:45.068954] [DEBUG] [LOCAL ] OMI: Deleted derivative data file. [2025-10-20T20:24:45.069040] [INFO ] [MAIN ] Update Progress: 1590/2367 symbols completed. OMI took 28.8s. Estimated remaining: 1h 40m 1s. Rate: 7.8 symbols/min [2025-10-20T20:24:45.240414] [INFO ] [MAIN ] ONTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:45.261506] [DEBUG] [LOCAL ] ONTF: Loaded 22973 existing records from primary storage. [2025-10-20T20:24:45.261767] [INFO ] [MAIN ] ONTF: Loaded 22973 existing records. [2025-10-20T20:24:45.433936] [WARN ] [MAIN ] ONTF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:45.434144] [INFO ] [MAIN ] ONTF: Wiping existing primary data before full refresh. [2025-10-20T20:24:45.434190] [INFO ] [LOCAL ] ONTF: Starting deletion of primary data. [2025-10-20T20:24:45.434351] [DEBUG] [LOCAL ] ONTF: Deleted primary data file. [2025-10-20T20:24:45.434388] [INFO ] [MAIN ] ONTF: Starting full primary refresh. [2025-10-20T20:24:45.783535] [INFO ] [REMOTE ] ONTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:45.851340] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-01. [2025-10-20T20:24:46.651392] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-05. [2025-10-20T20:24:46.987120] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T20:24:47.043375] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T20:24:47.451384] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-09. [2025-10-20T20:24:47.957606] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-05 (1.31 seconds). [2025-10-20T20:24:47.994986] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-10-20T20:24:48.251373] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-02. [2025-10-20T20:24:49.051395] [INFO ] [REMOTE ] ONIT: Fetching primary data for 2025-10. [2025-10-20T20:24:49.241027] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T20:24:49.698072] [INFO ] [REMOTE ] ONIT: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T20:24:49.699515] [INFO ] [REMOTE ] ONIT: Full fetch complete (11053 records). [2025-10-20T20:24:49.699581] [DEBUG] [VALIDATOR ] ONIT: Validating primary data. [2025-10-20T20:24:49.851410] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-03. [2025-10-20T20:24:50.064957] [DEBUG] [VALIDATOR ] ONIT: All expected trading days are present. [2025-10-20T20:24:50.065036] [DEBUG] [VALIDATOR ] ONIT: Primary data validation passed. [2025-10-20T20:24:50.065318] [INFO ] [MAIN ] ONIT: Fetched primary data is valid. [2025-10-20T20:24:50.065380] [DEBUG] [LOCAL ] ONIT: Saving primary data. [2025-10-20T20:24:50.065892] [DEBUG] [LOCAL ] ONIT: Trimmed 11053 records to 7641 (keeping 200 days of data) [2025-10-20T20:24:50.110905] [DEBUG] [LOCAL ] ONIT: Primary data saved (7641 records). [2025-10-20T20:24:50.111038] [INFO ] [MAIN ] ONIT: Full history saved. [2025-10-20T20:24:50.111122] [INFO ] [MAIN ] Update Progress: 1591/2367 symbols completed. ONIT took 31.5s. Estimated remaining: 1h 39m 52s. Rate: 7.8 symbols/min [2025-10-20T20:24:50.284390] [INFO ] [MAIN ] ONTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:24:50.329483] [DEBUG] [LOCAL ] ONTO: Loaded 47950 existing records from primary storage. [2025-10-20T20:24:50.329851] [INFO ] [MAIN ] ONTO: Loaded 47950 existing records. [2025-10-20T20:24:50.503730] [WARN ] [MAIN ] ONTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:24:50.503942] [INFO ] [MAIN ] ONTO: Wiping existing primary data before full refresh. [2025-10-20T20:24:50.503989] [INFO ] [LOCAL ] ONTO: Starting deletion of primary data. [2025-10-20T20:24:50.504208] [DEBUG] [LOCAL ] ONTO: Deleted primary data file. [2025-10-20T20:24:50.504251] [INFO ] [MAIN ] ONTO: Starting full primary refresh. [2025-10-20T20:24:50.656441] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-06. [2025-10-20T20:24:50.868731] [INFO ] [REMOTE ] ONTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:24:51.051527] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T20:24:51.451396] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-03. [2025-10-20T20:24:52.245426] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T20:24:52.252673] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-01. [2025-10-20T20:24:53.051394] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-04. [2025-10-20T20:24:53.638075] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-03 (2.19 seconds). [2025-10-20T20:24:53.851368] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-04. [2025-10-20T20:24:54.042921] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:24:54.083612] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T20:24:54.651373] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-05. [2025-10-20T20:24:55.451337] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-07. [2025-10-20T20:24:55.812289] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:24:56.004794] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-04 (2.15 seconds). [2025-10-20T20:24:56.251371] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-02. [2025-10-20T20:24:56.451359] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T20:24:57.051446] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-08. [2025-10-20T20:24:57.851421] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-05. [2025-10-20T20:24:57.978195] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:24:58.286876] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T20:24:58.651404] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-03. [2025-10-20T20:24:59.451355] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-06. [2025-10-20T20:24:59.989248] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T20:25:00.132988] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-06 (0.68 seconds). [2025-10-20T20:25:00.251355] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-06. [2025-10-20T20:25:00.434907] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T20:25:01.051389] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-04. [2025-10-20T20:25:01.851435] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-07. [2025-10-20T20:25:02.551701] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-10-20T20:25:02.651365] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-07. [2025-10-20T20:25:02.901344] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T20:25:03.025789] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T20:25:03.451467] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-08. [2025-10-20T20:25:04.251399] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-05. [2025-10-20T20:25:04.825996] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T20:25:04.879606] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T20:25:05.051464] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-08. [2025-10-20T20:25:05.851376] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-09. [2025-10-20T20:25:06.651402] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-09. [2025-10-20T20:25:07.237429] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-05 (2.99 seconds). [2025-10-20T20:25:07.451437] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-06. [2025-10-20T20:25:08.418337] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-08 (3.37 seconds). [2025-10-20T20:25:08.429703] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-09. [2025-10-20T20:25:09.586332] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-09 (3.73 seconds). [2025-10-20T20:25:09.593715] [INFO ] [REMOTE ] ONL: Fetching primary data for 2025-10. [2025-10-20T20:25:10.043345] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-09 (3.39 seconds). [2025-10-20T20:25:10.049416] [INFO ] [REMOTE ] ONTF: Fetching primary data for 2025-10. [2025-10-20T20:25:10.216371] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-06 (2.76 seconds). [2025-10-20T20:25:10.651359] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-07. [2025-10-20T20:25:10.726078] [INFO ] [REMOTE ] ONTF: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:25:10.729564] [INFO ] [REMOTE ] ONTF: Full fetch complete (31584 records). [2025-10-20T20:25:10.729665] [DEBUG] [VALIDATOR ] ONTF: Validating primary data. [2025-10-20T20:25:10.977441] [INFO ] [REMOTE ] ONL: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T20:25:11.024256] [INFO ] [REMOTE ] ONL: Full fetch complete (42632 records). [2025-10-20T20:25:11.029947] [DEBUG] [VALIDATOR ] ONL: Validating primary data. [2025-10-20T20:25:11.258549] [DEBUG] [VALIDATOR ] ONTF: All expected trading days are present. [2025-10-20T20:25:11.258921] [DEBUG] [VALIDATOR ] ONTF: Primary data validation passed. [2025-10-20T20:25:11.261518] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-09 (2.83 seconds). [2025-10-20T20:25:11.260193] [INFO ] [MAIN ] ONTF: Fetched primary data is valid. [2025-10-20T20:25:11.261717] [DEBUG] [LOCAL ] ONTF: Saving primary data. [2025-10-20T20:25:11.331039] [DEBUG] [LOCAL ] ONTF: Trimmed 31584 records to 22672 (keeping 200 days of data) [2025-10-20T20:25:11.466624] [INFO ] [REMOTE ] ONON: Fetching primary data for 2025-10. [2025-10-20T20:25:11.667009] [DEBUG] [VALIDATOR ] ONL: All expected trading days are present. [2025-10-20T20:25:11.673596] [DEBUG] [VALIDATOR ] ONL: Primary data validation passed. [2025-10-20T20:25:11.678374] [INFO ] [MAIN ] ONL: Fetched primary data is valid. [2025-10-20T20:25:11.679673] [DEBUG] [LOCAL ] ONL: Saving primary data. [2025-10-20T20:25:11.682994] [DEBUG] [LOCAL ] ONL: Trimmed 42632 records to 29214 (keeping 200 days of data) [2025-10-20T20:25:11.682898] [DEBUG] [LOCAL ] ONTF: Primary data saved (22672 records). [2025-10-20T20:25:11.683169] [INFO ] [MAIN ] ONTF: Full history saved. [2025-10-20T20:25:11.683489] [INFO ] [MAIN ] Update Progress: 1592/2367 symbols completed. ONTF took 26.6s. Estimated remaining: 1h 39m 51s. Rate: 7.8 symbols/min [2025-10-20T20:25:11.982878] [INFO ] [MAIN ] OOMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:12.051888] [DEBUG] [LOCAL ] ONL: Primary data saved (29214 records). [2025-10-20T20:25:12.052031] [INFO ] [MAIN ] ONL: Full history saved. [2025-10-20T20:25:12.052166] [INFO ] [MAIN ] Update Progress: 1593/2367 symbols completed. ONL took 35.0s. Estimated remaining: 1h 39m 40s. Rate: 7.8 symbols/min [2025-10-20T20:25:12.163732] [DEBUG] [LOCAL ] OOMA: Loaded 18820 existing records from primary storage. [2025-10-20T20:25:12.185436] [INFO ] [MAIN ] OOMA: Loaded 18820 existing records. [2025-10-20T20:25:12.248207] [INFO ] [MAIN ] OPAD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:12.442889] [INFO ] [REMOTE ] ONON: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T20:25:12.465145] [WARN ] [MAIN ] OOMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:12.480111] [INFO ] [MAIN ] OOMA: Wiping existing primary data before full refresh. [2025-10-20T20:25:12.481176] [INFO ] [LOCAL ] OOMA: Starting deletion of primary data. [2025-10-20T20:25:12.482287] [DEBUG] [LOCAL ] OPAD: Loaded 25646 existing records from primary storage. [2025-10-20T20:25:12.481576] [DEBUG] [LOCAL ] OOMA: Deleted primary data file. [2025-10-20T20:25:12.481732] [INFO ] [REMOTE ] ONON: Full fetch complete (77379 records). [2025-10-20T20:25:12.482735] [INFO ] [MAIN ] OPAD: Loaded 25646 existing records. [2025-10-20T20:25:12.482822] [INFO ] [MAIN ] OOMA: Starting full primary refresh. [2025-10-20T20:25:12.508566] [DEBUG] [VALIDATOR ] ONON: Validating primary data. [2025-10-20T20:25:12.540097] [FATAL] [VALIDATOR ] ONON: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:25:12.658154] [FATAL] [MAIN ] ONON: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:25:12.694591] [WARN ] [MAIN ] ONON: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:25:12.722458] [INFO ] [LOCAL ] ONON: Starting deletion of primary data. [2025-10-20T20:25:12.722232] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-07 (2.07 seconds). [2025-10-20T20:25:12.732696] [DEBUG] [LOCAL ] ONON: No primary data file found to delete. [2025-10-20T20:25:12.739129] [INFO ] [LOCAL ] ONON: Starting deletion of derivative data. [2025-10-20T20:25:12.750009] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-08. [2025-10-20T20:25:12.781065] [DEBUG] [LOCAL ] ONON: Deleted derivative data file. [2025-10-20T20:25:12.810613] [INFO ] [MAIN ] Update Progress: 1594/2367 symbols completed. ONON took 33.4s. Estimated remaining: 1h 39m 29s. Rate: 7.8 symbols/min [2025-10-20T20:25:12.915693] [WARN ] [MAIN ] OPAD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:12.947203] [INFO ] [MAIN ] OPAD: Wiping existing primary data before full refresh. [2025-10-20T20:25:12.952483] [INFO ] [LOCAL ] OPAD: Starting deletion of primary data. [2025-10-20T20:25:12.998248] [DEBUG] [LOCAL ] OPAD: Deleted primary data file. [2025-10-20T20:25:13.050803] [INFO ] [MAIN ] OPAD: Starting full primary refresh. [2025-10-20T20:25:13.342057] [INFO ] [MAIN ] OPFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:13.418163] [INFO ] [REMOTE ] OOMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:13.429154] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-01. [2025-10-20T20:25:13.705365] [INFO ] [REMOTE ] OPAD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:13.722895] [DEBUG] [LOCAL ] OPFI: Loaded 46886 existing records from primary storage. [2025-10-20T20:25:13.723336] [INFO ] [MAIN ] OPFI: Loaded 46886 existing records. [2025-10-20T20:25:13.856724] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-01. [2025-10-20T20:25:13.909574] [WARN ] [MAIN ] OPFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:13.909800] [INFO ] [MAIN ] OPFI: Wiping existing primary data before full refresh. [2025-10-20T20:25:13.909852] [INFO ] [LOCAL ] OPFI: Starting deletion of primary data. [2025-10-20T20:25:13.910045] [DEBUG] [LOCAL ] OPFI: Deleted primary data file. [2025-10-20T20:25:13.910101] [INFO ] [MAIN ] OPFI: Starting full primary refresh. [2025-10-20T20:25:14.266103] [INFO ] [REMOTE ] OPFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:14.656452] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-01. [2025-10-20T20:25:14.679413] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T20:25:14.679231] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T20:25:14.712794] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T20:25:15.451415] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-02. [2025-10-20T20:25:16.251337] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-02. [2025-10-20T20:25:16.365176] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T20:25:16.692395] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-01 (2.04 seconds). [2025-10-20T20:25:16.933616] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T20:25:17.051378] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-03. [2025-10-20T20:25:17.711606] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:25:17.851369] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-09. [2025-10-20T20:25:18.651399] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-02. [2025-10-20T20:25:19.451402] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-03. [2025-10-20T20:25:19.732716] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T20:25:20.251440] [INFO ] [REMOTE ] ONTO: Fetching primary data for 2025-10. [2025-10-20T20:25:20.691258] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T20:25:20.729394] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T20:25:21.034593] [INFO ] [REMOTE ] ONTO: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:25:21.041728] [INFO ] [REMOTE ] ONTO: Full fetch complete (67000 records). [2025-10-20T20:25:21.041879] [DEBUG] [VALIDATOR ] ONTO: Validating primary data. [2025-10-20T20:25:21.042887] [FATAL] [VALIDATOR ] ONTO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:25:21.042974] [FATAL] [MAIN ] ONTO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:25:21.043026] [WARN ] [MAIN ] ONTO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:25:21.043112] [INFO ] [LOCAL ] ONTO: Starting deletion of primary data. [2025-10-20T20:25:21.043189] [DEBUG] [LOCAL ] ONTO: No primary data file found to delete. [2025-10-20T20:25:21.043238] [INFO ] [LOCAL ] ONTO: Starting deletion of derivative data. [2025-10-20T20:25:21.043391] [DEBUG] [LOCAL ] ONTO: Deleted derivative data file. [2025-10-20T20:25:21.043797] [INFO ] [MAIN ] Update Progress: 1595/2367 symbols completed. ONTO took 30.9s. Estimated remaining: 1h 39m 21s. Rate: 7.8 symbols/min [2025-10-20T20:25:21.056438] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-04. [2025-10-20T20:25:21.219202] [INFO ] [MAIN ] OPP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:21.233974] [DEBUG] [LOCAL ] OPP: Loaded 12536 existing records from primary storage. [2025-10-20T20:25:21.234209] [INFO ] [MAIN ] OPP: Loaded 12536 existing records. [2025-10-20T20:25:21.411803] [WARN ] [MAIN ] OPP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:21.412000] [INFO ] [MAIN ] OPP: Wiping existing primary data before full refresh. [2025-10-20T20:25:21.412048] [INFO ] [LOCAL ] OPP: Starting deletion of primary data. [2025-10-20T20:25:21.412207] [DEBUG] [LOCAL ] OPP: Deleted primary data file. [2025-10-20T20:25:21.412246] [INFO ] [MAIN ] OPP: Starting full primary refresh. [2025-10-20T20:25:21.761068] [INFO ] [REMOTE ] OPP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:21.851385] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-01. [2025-10-20T20:25:21.956710] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T20:25:22.651415] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-04. [2025-10-20T20:25:22.700931] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T20:25:23.451396] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-02. [2025-10-20T20:25:23.839550] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T20:25:24.251380] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-03. [2025-10-20T20:25:24.293905] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T20:25:25.051382] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-03. [2025-10-20T20:25:25.851355] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-05. [2025-10-20T20:25:25.926339] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:25:26.071757] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T20:25:26.651341] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-05. [2025-10-20T20:25:26.743082] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:25:27.314534] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T20:25:27.451401] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-06. [2025-10-20T20:25:28.251356] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-04. [2025-10-20T20:25:28.548935] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T20:25:29.051371] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-04. [2025-10-20T20:25:29.851372] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-07. [2025-10-20T20:25:30.001507] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T20:25:30.031973] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T20:25:30.651345] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-05. [2025-10-20T20:25:30.971463] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T20:25:31.332804] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T20:25:31.451361] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-08. [2025-10-20T20:25:32.251373] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-06. [2025-10-20T20:25:32.526127] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T20:25:33.051387] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-09. [2025-10-20T20:25:33.104546] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T20:25:33.851357] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-05. [2025-10-20T20:25:34.113609] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T20:25:34.651370] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-06. [2025-10-20T20:25:35.451349] [INFO ] [REMOTE ] OOMA: Fetching primary data for 2025-10. [2025-10-20T20:25:35.478604] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T20:25:35.706168] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T20:25:36.251357] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-07. [2025-10-20T20:25:36.480334] [INFO ] [REMOTE ] OOMA: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:25:36.483139] [INFO ] [REMOTE ] OOMA: Full fetch complete (25841 records). [2025-10-20T20:25:36.483247] [DEBUG] [VALIDATOR ] OOMA: Validating primary data. [2025-10-20T20:25:36.864907] [DEBUG] [VALIDATOR ] OOMA: All expected trading days are present. [2025-10-20T20:25:36.864992] [DEBUG] [VALIDATOR ] OOMA: Primary data validation passed. [2025-10-20T20:25:36.865577] [INFO ] [MAIN ] OOMA: Fetched primary data is valid. [2025-10-20T20:25:36.865639] [DEBUG] [LOCAL ] OOMA: Saving primary data. [2025-10-20T20:25:36.866349] [DEBUG] [LOCAL ] OOMA: Trimmed 25841 records to 18427 (keeping 200 days of data) [2025-10-20T20:25:36.972673] [DEBUG] [LOCAL ] OOMA: Primary data saved (18427 records). [2025-10-20T20:25:36.972814] [INFO ] [MAIN ] OOMA: Full history saved. [2025-10-20T20:25:36.972889] [INFO ] [MAIN ] Update Progress: 1596/2367 symbols completed. OOMA took 25.3s. Estimated remaining: 1h 39m 18s. Rate: 7.8 symbols/min [2025-10-20T20:25:37.056629] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-06. [2025-10-20T20:25:37.159497] [INFO ] [MAIN ] OPY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:37.159716] [DEBUG] [LOCAL ] OPY: No existing data file found in primary storage. [2025-10-20T20:25:37.159776] [WARN ] [MAIN ] OPY: No existing data found, performing full refresh instead. [2025-10-20T20:25:37.159934] [INFO ] [MAIN ] OPY: Wiping existing primary data before full refresh. [2025-10-20T20:25:37.159976] [INFO ] [LOCAL ] OPY: Starting deletion of primary data. [2025-10-20T20:25:37.160019] [DEBUG] [LOCAL ] OPY: No primary data file found to delete. [2025-10-20T20:25:37.160051] [INFO ] [MAIN ] OPY: Starting full primary refresh. [2025-10-20T20:25:37.282411] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T20:25:37.524911] [INFO ] [REMOTE ] OPY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:37.851369] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-01. [2025-10-20T20:25:38.651366] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-08. [2025-10-20T20:25:38.737909] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:25:38.937752] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T20:25:39.451350] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-02. [2025-10-20T20:25:39.520093] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:25:40.124187] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T20:25:40.251371] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-09. [2025-10-20T20:25:41.051348] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-03. [2025-10-20T20:25:41.188541] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T20:25:41.715721] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:25:41.851357] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-07. [2025-10-20T20:25:42.651376] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-04. [2025-10-20T20:25:43.451402] [INFO ] [REMOTE ] OPP: Fetching primary data for 2025-10. [2025-10-20T20:25:43.458821] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T20:25:43.704195] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T20:25:44.251398] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-07. [2025-10-20T20:25:44.398150] [INFO ] [REMOTE ] OPP: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T20:25:44.401102] [INFO ] [REMOTE ] OPP: Full fetch complete (17489 records). [2025-10-20T20:25:44.401174] [DEBUG] [VALIDATOR ] OPP: Validating primary data. [2025-10-20T20:25:44.758242] [DEBUG] [VALIDATOR ] OPP: All expected trading days are present. [2025-10-20T20:25:44.758317] [DEBUG] [VALIDATOR ] OPP: Primary data validation passed. [2025-10-20T20:25:44.758703] [INFO ] [MAIN ] OPP: Fetched primary data is valid. [2025-10-20T20:25:44.758763] [DEBUG] [LOCAL ] OPP: Saving primary data. [2025-10-20T20:25:44.759255] [DEBUG] [LOCAL ] OPP: Trimmed 17489 records to 12377 (keeping 200 days of data) [2025-10-20T20:25:44.829799] [DEBUG] [LOCAL ] OPP: Primary data saved (12377 records). [2025-10-20T20:25:44.829919] [INFO ] [MAIN ] OPP: Full history saved. [2025-10-20T20:25:44.829992] [INFO ] [MAIN ] Update Progress: 1597/2367 symbols completed. OPP took 23.8s. Estimated remaining: 1h 39m 10s. Rate: 7.8 symbols/min [2025-10-20T20:25:45.051368] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-05. [2025-10-20T20:25:45.851414] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-08. [2025-10-20T20:25:45.916294] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T20:25:45.966787] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T20:25:46.651374] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-08. [2025-10-20T20:25:47.451377] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-06. [2025-10-20T20:25:47.619947] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T20:25:48.101382] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T20:25:48.251347] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-09. [2025-10-20T20:25:48.410903] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T20:25:49.051388] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-07. [2025-10-20T20:25:49.749924] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T20:25:49.851372] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-08. [2025-10-20T20:25:50.629942] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-09 (2.38 seconds). [2025-10-20T20:25:50.651513] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-09. [2025-10-20T20:25:50.652426] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T20:25:51.451370] [INFO ] [REMOTE ] OPAD: Fetching primary data for 2025-10. [2025-10-20T20:25:52.251430] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-09. [2025-10-20T20:25:52.367145] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T20:25:53.051431] [INFO ] [REMOTE ] OPFI: Fetching primary data for 2025-10. [2025-10-20T20:25:53.132336] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:25:53.399349] [INFO ] [REMOTE ] OPAD: Incremental fetch complete for 2025-10 (1.95 seconds). [2025-10-20T20:25:53.406177] [INFO ] [REMOTE ] OPAD: Full fetch complete (28416 records). [2025-10-20T20:25:53.406288] [DEBUG] [VALIDATOR ] OPAD: Validating primary data. [2025-10-20T20:25:53.763713] [DEBUG] [VALIDATOR ] OPAD: All expected trading days are present. [2025-10-20T20:25:53.763799] [DEBUG] [VALIDATOR ] OPAD: Primary data validation passed. [2025-10-20T20:25:53.764411] [INFO ] [MAIN ] OPAD: Fetched primary data is valid. [2025-10-20T20:25:53.764448] [DEBUG] [LOCAL ] OPAD: Saving primary data. [2025-10-20T20:25:53.765165] [DEBUG] [LOCAL ] OPAD: Trimmed 28416 records to 25848 (keeping 200 days of data) [2025-10-20T20:25:53.860404] [INFO ] [REMOTE ] OPY: Fetching primary data for 2025-10. [2025-10-20T20:25:53.929893] [DEBUG] [LOCAL ] OPAD: Primary data saved (25848 records). [2025-10-20T20:25:53.930032] [INFO ] [MAIN ] OPAD: Full history saved. [2025-10-20T20:25:53.930113] [INFO ] [MAIN ] Update Progress: 1598/2367 symbols completed. OPAD took 41.9s. Estimated remaining: 1h 39m 3s. Rate: 7.8 symbols/min [2025-10-20T20:25:54.532781] [INFO ] [REMOTE ] OPFI: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T20:25:54.539438] [INFO ] [REMOTE ] OPFI: Full fetch complete (68799 records). [2025-10-20T20:25:54.539552] [DEBUG] [VALIDATOR ] OPFI: Validating primary data. [2025-10-20T20:25:54.634401] [INFO ] [REMOTE ] OPY: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:25:54.646388] [INFO ] [REMOTE ] OPY: Full fetch complete (10202 records). [2025-10-20T20:25:54.662097] [DEBUG] [VALIDATOR ] OPY: Validating primary data. [2025-10-20T20:25:55.058568] [WARN ] [VALIDATOR ] OPY: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T20:25:55.064703] [FATAL] [MAIN ] OPY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:25:55.075005] [WARN ] [MAIN ] OPY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:25:55.080201] [INFO ] [LOCAL ] OPY: Starting deletion of primary data. [2025-10-20T20:25:55.085409] [DEBUG] [LOCAL ] OPY: No primary data file found to delete. [2025-10-20T20:25:55.090614] [INFO ] [LOCAL ] OPY: Starting deletion of derivative data. [2025-10-20T20:25:55.107297] [DEBUG] [LOCAL ] OPY: No derivative data file found to delete. [2025-10-20T20:25:55.107406] [INFO ] [MAIN ] Update Progress: 1599/2367 symbols completed. OPY took 18.1s. Estimated remaining: 1h 38m 52s. Rate: 7.8 symbols/min [2025-10-20T20:25:55.163471] [DEBUG] [VALIDATOR ] OPFI: All expected trading days are present. [2025-10-20T20:25:55.163577] [DEBUG] [VALIDATOR ] OPFI: Primary data validation passed. [2025-10-20T20:25:55.164928] [INFO ] [MAIN ] OPFI: Fetched primary data is valid. [2025-10-20T20:25:55.164994] [DEBUG] [LOCAL ] OPFI: Saving primary data. [2025-10-20T20:25:55.166339] [DEBUG] [LOCAL ] OPFI: Trimmed 68799 records to 46172 (keeping 200 days of data) [2025-10-20T20:25:55.452204] [DEBUG] [LOCAL ] OPFI: Primary data saved (46172 records). [2025-10-20T20:25:55.452346] [INFO ] [MAIN ] OPFI: Full history saved. [2025-10-20T20:25:55.452429] [INFO ] [MAIN ] Update Progress: 1600/2367 symbols completed. OPFI took 42.6s. Estimated remaining: 1h 38m 41s. Rate: 7.8 symbols/min [2025-10-20T20:25:56.032082] [INFO ] [MAIN ] ORA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:56.072482] [INFO ] [MAIN ] ORC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:56.083150] [DEBUG] [LOCAL ] ORC: No existing data file found in primary storage. [2025-10-20T20:25:56.134143] [WARN ] [MAIN ] ORC: No existing data found, performing full refresh instead. [2025-10-20T20:25:56.186464] [INFO ] [MAIN ] OR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:56.185695] [INFO ] [MAIN ] ORCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:25:56.187161] [INFO ] [MAIN ] ORC: Wiping existing primary data before full refresh. [2025-10-20T20:25:56.187227] [DEBUG] [LOCAL ] ORCL: No existing data file found in primary storage. [2025-10-20T20:25:56.187307] [INFO ] [LOCAL ] ORC: Starting deletion of primary data. [2025-10-20T20:25:56.187489] [DEBUG] [LOCAL ] ORC: No primary data file found to delete. [2025-10-20T20:25:56.187352] [WARN ] [MAIN ] ORCL: No existing data found, performing full refresh instead. [2025-10-20T20:25:56.187567] [INFO ] [MAIN ] ORC: Starting full primary refresh. [2025-10-20T20:25:56.187675] [INFO ] [MAIN ] ORCL: Wiping existing primary data before full refresh. [2025-10-20T20:25:56.187850] [INFO ] [LOCAL ] ORCL: Starting deletion of primary data. [2025-10-20T20:25:56.198756] [DEBUG] [LOCAL ] ORCL: No primary data file found to delete. [2025-10-20T20:25:56.198884] [INFO ] [MAIN ] ORCL: Starting full primary refresh. [2025-10-20T20:25:56.750021] [DEBUG] [LOCAL ] ORA: Loaded 33548 existing records from primary storage. [2025-10-20T20:25:56.782167] [INFO ] [MAIN ] ORA: Loaded 33548 existing records. [2025-10-20T20:25:56.808956] [DEBUG] [LOCAL ] OR: Loaded 47473 existing records from primary storage. [2025-10-20T20:25:56.925677] [INFO ] [MAIN ] OR: Loaded 47473 existing records. [2025-10-20T20:25:57.058688] [INFO ] [REMOTE ] ORC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:57.059021] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-01. [2025-10-20T20:25:57.160913] [INFO ] [REMOTE ] ORCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:57.205371] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-01. [2025-10-20T20:25:57.310737] [WARN ] [MAIN ] ORA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:57.349991] [INFO ] [MAIN ] ORA: Wiping existing primary data before full refresh. [2025-10-20T20:25:57.397380] [WARN ] [MAIN ] OR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:25:57.397470] [INFO ] [LOCAL ] ORA: Starting deletion of primary data. [2025-10-20T20:25:57.397697] [INFO ] [MAIN ] OR: Wiping existing primary data before full refresh. [2025-10-20T20:25:57.397870] [INFO ] [LOCAL ] OR: Starting deletion of primary data. [2025-10-20T20:25:57.398248] [DEBUG] [LOCAL ] OR: Deleted primary data file. [2025-10-20T20:25:57.398211] [DEBUG] [LOCAL ] ORA: Deleted primary data file. [2025-10-20T20:25:57.398376] [INFO ] [MAIN ] ORA: Starting full primary refresh. [2025-10-20T20:25:57.398422] [INFO ] [MAIN ] OR: Starting full primary refresh. [2025-10-20T20:25:58.100020] [INFO ] [REMOTE ] ORA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:58.100636] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-01. [2025-10-20T20:25:58.187828] [INFO ] [REMOTE ] OR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:25:58.188303] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-01. [2025-10-20T20:25:58.222272] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T20:25:58.233598] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-02. [2025-10-20T20:25:59.001956] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:25:59.013160] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-03. [2025-10-20T20:25:59.201633] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-01 (2.14 seconds). [2025-10-20T20:25:59.266105] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T20:25:59.451377] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-02. [2025-10-20T20:25:59.752706] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T20:25:59.805694] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:26:00.251402] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-02. [2025-10-20T20:26:01.051347] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-02. [2025-10-20T20:26:01.526144] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T20:26:01.851468] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-04. [2025-10-20T20:26:01.894087] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T20:26:02.630839] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:26:02.651332] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-03. [2025-10-20T20:26:02.706639] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T20:26:03.451424] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-03. [2025-10-20T20:26:04.103429] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T20:26:04.251383] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-03. [2025-10-20T20:26:05.051400] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-05. [2025-10-20T20:26:05.544346] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T20:26:05.834413] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T20:26:05.851329] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-06. [2025-10-20T20:26:05.883608] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T20:26:06.651353] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-04. [2025-10-20T20:26:06.698685] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T20:26:07.451426] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-04. [2025-10-20T20:26:08.124952] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T20:26:08.251351] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-05. [2025-10-20T20:26:09.051342] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-07. [2025-10-20T20:26:09.241265] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T20:26:09.524274] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T20:26:09.617057] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T20:26:09.851421] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-05. [2025-10-20T20:26:10.651427] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-04. [2025-10-20T20:26:11.451365] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-08. [2025-10-20T20:26:11.615210] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T20:26:12.226967] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T20:26:12.251333] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-06. [2025-10-20T20:26:12.580010] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T20:26:13.051428] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-05. [2025-10-20T20:26:13.851422] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-09. [2025-10-20T20:26:13.930208] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T20:26:14.657706] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-06. [2025-10-20T20:26:14.664852] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:26:15.153021] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-05 (2.10 seconds). [2025-10-20T20:26:15.451372] [INFO ] [REMOTE ] ORCL: Fetching primary data for 2025-10. [2025-10-20T20:26:16.025707] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T20:26:16.203436] [INFO ] [REMOTE ] ORCL: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:26:16.212322] [INFO ] [REMOTE ] ORCL: Full fetch complete (77213 records). [2025-10-20T20:26:16.212557] [DEBUG] [VALIDATOR ] ORCL: Validating primary data. [2025-10-20T20:26:16.213485] [FATAL] [VALIDATOR ] ORCL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:26:16.213580] [FATAL] [MAIN ] ORCL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:26:16.213623] [WARN ] [MAIN ] ORCL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:26:16.213704] [INFO ] [LOCAL ] ORCL: Starting deletion of primary data. [2025-10-20T20:26:16.213776] [DEBUG] [LOCAL ] ORCL: No primary data file found to delete. [2025-10-20T20:26:16.213812] [INFO ] [LOCAL ] ORCL: Starting deletion of derivative data. [2025-10-20T20:26:16.213901] [DEBUG] [LOCAL ] ORCL: No derivative data file found to delete. [2025-10-20T20:26:16.213969] [INFO ] [MAIN ] Update Progress: 1601/2367 symbols completed. ORCL took 20.8s. Estimated remaining: 1h 38m 39s. Rate: 7.8 symbols/min [2025-10-20T20:26:16.256500] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-07. [2025-10-20T20:26:16.384599] [INFO ] [MAIN ] ORI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:16.384803] [DEBUG] [LOCAL ] ORI: No existing data file found in primary storage. [2025-10-20T20:26:16.384846] [WARN ] [MAIN ] ORI: No existing data found, performing full refresh instead. [2025-10-20T20:26:16.384957] [INFO ] [MAIN ] ORI: Wiping existing primary data before full refresh. [2025-10-20T20:26:16.384999] [INFO ] [LOCAL ] ORI: Starting deletion of primary data. [2025-10-20T20:26:16.385041] [DEBUG] [LOCAL ] ORI: No primary data file found to delete. [2025-10-20T20:26:16.385072] [INFO ] [MAIN ] ORI: Starting full primary refresh. [2025-10-20T20:26:16.731503] [INFO ] [REMOTE ] ORI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:17.051446] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-06. [2025-10-20T20:26:17.851462] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-01. [2025-10-20T20:26:17.973402] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:26:18.651380] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-08. [2025-10-20T20:26:19.024321] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T20:26:19.451357] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-07. [2025-10-20T20:26:19.510450] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T20:26:20.189582] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T20:26:20.251378] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-07. [2025-10-20T20:26:21.051430] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-09. [2025-10-20T20:26:21.641416] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T20:26:21.851351] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-02. [2025-10-20T20:26:22.021468] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T20:26:22.509882] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T20:26:22.651369] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-08. [2025-10-20T20:26:23.087808] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T20:26:23.451432] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-08. [2025-10-20T20:26:24.251393] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-03. [2025-10-20T20:26:24.346432] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:26:25.051382] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-09. [2025-10-20T20:26:25.537786] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T20:26:25.851400] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-09. [2025-10-20T20:26:25.959711] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T20:26:26.651376] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-04. [2025-10-20T20:26:26.913403] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T20:26:27.451411] [INFO ] [REMOTE ] ORA: Fetching primary data for 2025-10. [2025-10-20T20:26:27.888379] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:26:28.047016] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T20:26:28.143603] [INFO ] [REMOTE ] ORA: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:26:28.151251] [INFO ] [REMOTE ] ORA: Full fetch complete (48767 records). [2025-10-20T20:26:28.151405] [DEBUG] [VALIDATOR ] ORA: Validating primary data. [2025-10-20T20:26:28.152206] [FATAL] [VALIDATOR ] ORA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:26:28.152311] [FATAL] [MAIN ] ORA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:26:28.152365] [WARN ] [MAIN ] ORA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:26:28.152442] [INFO ] [LOCAL ] ORA: Starting deletion of primary data. [2025-10-20T20:26:28.152523] [DEBUG] [LOCAL ] ORA: No primary data file found to delete. [2025-10-20T20:26:28.152594] [INFO ] [LOCAL ] ORA: Starting deletion of derivative data. [2025-10-20T20:26:28.152802] [DEBUG] [LOCAL ] ORA: Deleted derivative data file. [2025-10-20T20:26:28.152909] [INFO ] [MAIN ] Update Progress: 1602/2367 symbols completed. ORA took 32.7s. Estimated remaining: 1h 38m 34s. Rate: 7.8 symbols/min [2025-10-20T20:26:28.255377] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-05. [2025-10-20T20:26:28.326132] [INFO ] [MAIN ] ORN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:28.355404] [DEBUG] [LOCAL ] ORN: Loaded 31321 existing records from primary storage. [2025-10-20T20:26:28.355725] [INFO ] [MAIN ] ORN: Loaded 31321 existing records. [2025-10-20T20:26:28.546884] [WARN ] [MAIN ] ORN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:26:28.547100] [INFO ] [MAIN ] ORN: Wiping existing primary data before full refresh. [2025-10-20T20:26:28.547147] [INFO ] [LOCAL ] ORN: Starting deletion of primary data. [2025-10-20T20:26:28.547349] [DEBUG] [LOCAL ] ORN: Deleted primary data file. [2025-10-20T20:26:28.547406] [INFO ] [MAIN ] ORN: Starting full primary refresh. [2025-10-20T20:26:28.913040] [INFO ] [REMOTE ] ORN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:29.051358] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-01. [2025-10-20T20:26:29.851350] [INFO ] [REMOTE ] OR: Fetching primary data for 2025-10. [2025-10-20T20:26:29.911804] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T20:26:30.017126] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T20:26:30.651373] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-02. [2025-10-20T20:26:31.451367] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-06. [2025-10-20T20:26:31.569759] [INFO ] [REMOTE ] OR: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T20:26:31.577378] [INFO ] [REMOTE ] OR: Full fetch complete (66646 records). [2025-10-20T20:26:31.577527] [DEBUG] [VALIDATOR ] OR: Validating primary data. [2025-10-20T20:26:31.917491] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T20:26:32.000219] [DEBUG] [VALIDATOR ] OR: All expected trading days are present. [2025-10-20T20:26:32.000358] [DEBUG] [VALIDATOR ] OR: Primary data validation passed. [2025-10-20T20:26:32.001348] [INFO ] [MAIN ] OR: Fetched primary data is valid. [2025-10-20T20:26:32.001411] [DEBUG] [LOCAL ] OR: Saving primary data. [2025-10-20T20:26:32.002745] [DEBUG] [LOCAL ] OR: Trimmed 66646 records to 46804 (keeping 200 days of data) [2025-10-20T20:26:32.258664] [INFO ] [REMOTE ] ORC: Fetching primary data for 2025-10. [2025-10-20T20:26:32.282871] [DEBUG] [LOCAL ] OR: Primary data saved (46804 records). [2025-10-20T20:26:32.282983] [INFO ] [MAIN ] OR: Full history saved. [2025-10-20T20:26:32.283393] [INFO ] [MAIN ] Update Progress: 1603/2367 symbols completed. OR took 36.8s. Estimated remaining: 1h 38m 24s. Rate: 7.8 symbols/min [2025-10-20T20:26:32.455279] [INFO ] [MAIN ] OSCR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:32.515023] [DEBUG] [LOCAL ] OSCR: Loaded 54539 existing records from primary storage. [2025-10-20T20:26:32.515379] [INFO ] [MAIN ] OSCR: Loaded 54539 existing records. [2025-10-20T20:26:32.709819] [WARN ] [MAIN ] OSCR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:26:32.710033] [INFO ] [MAIN ] OSCR: Wiping existing primary data before full refresh. [2025-10-20T20:26:32.710116] [INFO ] [LOCAL ] OSCR: Starting deletion of primary data. [2025-10-20T20:26:32.710307] [DEBUG] [LOCAL ] OSCR: Deleted primary data file. [2025-10-20T20:26:32.710347] [INFO ] [MAIN ] OSCR: Starting full primary refresh. [2025-10-20T20:26:33.054696] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-03. [2025-10-20T20:26:33.097818] [INFO ] [REMOTE ] OSCR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:33.118240] [INFO ] [REMOTE ] ORC: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:26:33.153814] [INFO ] [REMOTE ] ORC: Full fetch complete (75180 records). [2025-10-20T20:26:33.154449] [DEBUG] [VALIDATOR ] ORC: Validating primary data. [2025-10-20T20:26:33.155671] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T20:26:33.165756] [FATAL] [VALIDATOR ] ORC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:26:33.166186] [FATAL] [MAIN ] ORC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:26:33.166260] [WARN ] [MAIN ] ORC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:26:33.166611] [INFO ] [LOCAL ] ORC: Starting deletion of primary data. [2025-10-20T20:26:33.166724] [DEBUG] [LOCAL ] ORC: No primary data file found to delete. [2025-10-20T20:26:33.166771] [INFO ] [LOCAL ] ORC: Starting deletion of derivative data. [2025-10-20T20:26:33.166814] [DEBUG] [LOCAL ] ORC: No derivative data file found to delete. [2025-10-20T20:26:33.166870] [INFO ] [MAIN ] Update Progress: 1604/2367 symbols completed. ORC took 37.7s. Estimated remaining: 1h 38m 13s. Rate: 7.8 symbols/min [2025-10-20T20:26:33.335368] [INFO ] [MAIN ] OSK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:33.335543] [DEBUG] [LOCAL ] OSK: No existing data file found in primary storage. [2025-10-20T20:26:33.335580] [WARN ] [MAIN ] OSK: No existing data found, performing full refresh instead. [2025-10-20T20:26:33.335710] [INFO ] [MAIN ] OSK: Wiping existing primary data before full refresh. [2025-10-20T20:26:33.335759] [INFO ] [LOCAL ] OSK: Starting deletion of primary data. [2025-10-20T20:26:33.335799] [DEBUG] [LOCAL ] OSK: No primary data file found to delete. [2025-10-20T20:26:33.335830] [INFO ] [MAIN ] OSK: Starting full primary refresh. [2025-10-20T20:26:33.699177] [INFO ] [REMOTE ] OSK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:33.851391] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-07. [2025-10-20T20:26:33.924778] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:26:34.651374] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-01. [2025-10-20T20:26:35.277273] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T20:26:35.451383] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-08. [2025-10-20T20:26:36.251351] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-04. [2025-10-20T20:26:36.322248] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T20:26:37.051372] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-01. [2025-10-20T20:26:37.143697] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T20:26:37.566865] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T20:26:37.851410] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-05. [2025-10-20T20:26:38.651403] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-09. [2025-10-20T20:26:38.915725] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T20:26:38.988024] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T20:26:39.451350] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-02. [2025-10-20T20:26:40.251397] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-02. [2025-10-20T20:26:40.357138] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:26:41.051376] [INFO ] [REMOTE ] ORI: Fetching primary data for 2025-10. [2025-10-20T20:26:41.208554] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T20:26:41.793424] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T20:26:41.819813] [INFO ] [REMOTE ] ORI: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:26:41.826624] [INFO ] [REMOTE ] ORI: Full fetch complete (70215 records). [2025-10-20T20:26:41.826753] [DEBUG] [VALIDATOR ] ORI: Validating primary data. [2025-10-20T20:26:41.827445] [FATAL] [VALIDATOR ] ORI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:26:41.827526] [FATAL] [MAIN ] ORI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:26:41.827569] [WARN ] [MAIN ] ORI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:26:41.827611] [INFO ] [LOCAL ] ORI: Starting deletion of primary data. [2025-10-20T20:26:41.827690] [DEBUG] [LOCAL ] ORI: No primary data file found to delete. [2025-10-20T20:26:41.827744] [INFO ] [LOCAL ] ORI: Starting deletion of derivative data. [2025-10-20T20:26:41.827785] [DEBUG] [LOCAL ] ORI: No derivative data file found to delete. [2025-10-20T20:26:41.827839] [INFO ] [MAIN ] Update Progress: 1605/2367 symbols completed. ORI took 25.6s. Estimated remaining: 1h 38m 6s. Rate: 7.8 symbols/min [2025-10-20T20:26:41.857716] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-06. [2025-10-20T20:26:42.001556] [INFO ] [MAIN ] OTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:42.001759] [DEBUG] [LOCAL ] OTF: No existing data file found in primary storage. [2025-10-20T20:26:42.001805] [WARN ] [MAIN ] OTF: No existing data found, performing full refresh instead. [2025-10-20T20:26:42.001918] [INFO ] [MAIN ] OTF: Wiping existing primary data before full refresh. [2025-10-20T20:26:42.001960] [INFO ] [LOCAL ] OTF: Starting deletion of primary data. [2025-10-20T20:26:42.002002] [DEBUG] [LOCAL ] OTF: No primary data file found to delete. [2025-10-20T20:26:42.002034] [INFO ] [MAIN ] OTF: Starting full primary refresh. [2025-10-20T20:26:42.356414] [INFO ] [REMOTE ] OTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:42.651417] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-03. [2025-10-20T20:26:43.314423] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:26:43.451384] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-03. [2025-10-20T20:26:44.251363] [INFO ] [REMOTE ] OTF: Fetching primary data for 2025-01. [2025-10-20T20:26:44.401967] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T20:26:44.504294] [DEBUG] [REMOTE ] OTF: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:26:44.504610] [WARN ] [REMOTE ] OTF: No data returned for month 2025-01 [2025-10-20T20:26:44.504739] [FATAL] [REMOTE ] OTF: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:26:44.504793] [FATAL] [MAIN ] OTF: Failed to fetch primary data. [2025-10-20T20:26:44.504865] [WARN ] [MAIN ] OTF: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:26:44.504938] [INFO ] [LOCAL ] OTF: Starting deletion of primary data. [2025-10-20T20:26:44.505002] [DEBUG] [LOCAL ] OTF: No primary data file found to delete. [2025-10-20T20:26:44.505038] [INFO ] [LOCAL ] OTF: Starting deletion of derivative data. [2025-10-20T20:26:44.505078] [DEBUG] [LOCAL ] OTF: No derivative data file found to delete. [2025-10-20T20:26:44.505125] [INFO ] [MAIN ] Update Progress: 1606/2367 symbols completed. OTF took 2.7s. Estimated remaining: 1h 37m 56s. Rate: 7.8 symbols/min [2025-10-20T20:26:44.687310] [INFO ] [MAIN ] OTIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:44.737733] [DEBUG] [LOCAL ] OTIS: Loaded 53507 existing records from primary storage. [2025-10-20T20:26:44.738111] [INFO ] [MAIN ] OTIS: Loaded 53507 existing records. [2025-10-20T20:26:44.919721] [WARN ] [MAIN ] OTIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:26:44.919939] [INFO ] [MAIN ] OTIS: Wiping existing primary data before full refresh. [2025-10-20T20:26:44.919988] [INFO ] [LOCAL ] OTIS: Starting deletion of primary data. [2025-10-20T20:26:44.920228] [DEBUG] [LOCAL ] OTIS: Deleted primary data file. [2025-10-20T20:26:44.920287] [INFO ] [MAIN ] OTIS: Starting full primary refresh. [2025-10-20T20:26:45.063539] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-04. [2025-10-20T20:26:45.289834] [INFO ] [REMOTE ] OTIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:45.535000] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-03 (2.08 seconds). [2025-10-20T20:26:45.851421] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-04. [2025-10-20T20:26:46.651375] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-07. [2025-10-20T20:26:46.801415] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:26:47.451465] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-01. [2025-10-20T20:26:47.975930] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T20:26:48.068282] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T20:26:48.251374] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-05. [2025-10-20T20:26:49.051356] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-08. [2025-10-20T20:26:49.287843] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T20:26:49.851403] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-05. [2025-10-20T20:26:50.406916] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T20:26:50.464232] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-05 (2.21 seconds). [2025-10-20T20:26:50.651388] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-06. [2025-10-20T20:26:51.408905] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T20:26:51.451345] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-06. [2025-10-20T20:26:52.251420] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-09. [2025-10-20T20:26:53.051392] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-02. [2025-10-20T20:26:53.115509] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T20:26:53.225479] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-06 (2.57 seconds). [2025-10-20T20:26:53.603353] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T20:26:53.851389] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-07. [2025-10-20T20:26:54.651344] [INFO ] [REMOTE ] ORN: Fetching primary data for 2025-10. [2025-10-20T20:26:54.747074] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:26:55.451350] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-03. [2025-10-20T20:26:55.493428] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T20:26:55.930096] [INFO ] [REMOTE ] ORN: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T20:26:55.934352] [INFO ] [REMOTE ] ORN: Full fetch complete (45598 records). [2025-10-20T20:26:55.934435] [DEBUG] [VALIDATOR ] ORN: Validating primary data. [2025-10-20T20:26:56.256466] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-07. [2025-10-20T20:26:56.304504] [DEBUG] [VALIDATOR ] ORN: All expected trading days are present. [2025-10-20T20:26:56.304572] [DEBUG] [VALIDATOR ] ORN: Primary data validation passed. [2025-10-20T20:26:56.305377] [INFO ] [MAIN ] ORN: Fetched primary data is valid. [2025-10-20T20:26:56.305436] [DEBUG] [LOCAL ] ORN: Saving primary data. [2025-10-20T20:26:56.306409] [DEBUG] [LOCAL ] ORN: Trimmed 45598 records to 30941 (keeping 200 days of data) [2025-10-20T20:26:56.479069] [DEBUG] [LOCAL ] ORN: Primary data saved (30941 records). [2025-10-20T20:26:56.479205] [INFO ] [MAIN ] ORN: Full history saved. [2025-10-20T20:26:56.479283] [INFO ] [MAIN ] Update Progress: 1607/2367 symbols completed. ORN took 28.3s. Estimated remaining: 1h 37m 50s. Rate: 7.8 symbols/min [2025-10-20T20:26:56.648235] [INFO ] [MAIN ] OUT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:26:56.694145] [DEBUG] [LOCAL ] OUT: Loaded 48459 existing records from primary storage. [2025-10-20T20:26:56.694565] [INFO ] [MAIN ] OUT: Loaded 48459 existing records. [2025-10-20T20:26:56.866786] [WARN ] [MAIN ] OUT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:26:56.867076] [INFO ] [MAIN ] OUT: Wiping existing primary data before full refresh. [2025-10-20T20:26:56.867134] [INFO ] [LOCAL ] OUT: Starting deletion of primary data. [2025-10-20T20:26:56.867839] [DEBUG] [LOCAL ] OUT: Deleted primary data file. [2025-10-20T20:26:56.867897] [INFO ] [MAIN ] OUT: Starting full primary refresh. [2025-10-20T20:26:57.065541] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-08. [2025-10-20T20:26:57.225519] [INFO ] [REMOTE ] OUT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:26:57.415681] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T20:26:57.851422] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-01. [2025-10-20T20:26:58.651351] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-04. [2025-10-20T20:26:58.836165] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T20:26:58.929440] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-07 (2.67 seconds). [2025-10-20T20:26:59.451347] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-08. [2025-10-20T20:26:59.542784] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T20:27:00.251356] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-09. [2025-10-20T20:27:00.433234] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T20:27:01.051411] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-05. [2025-10-20T20:27:01.753940] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T20:27:01.817841] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-10-20T20:27:01.851384] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-09. [2025-10-20T20:27:02.509406] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T20:27:02.651373] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-02. [2025-10-20T20:27:03.451379] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-06. [2025-10-20T20:27:04.251347] [INFO ] [REMOTE ] OSK: Fetching primary data for 2025-10. [2025-10-20T20:27:04.349914] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:27:04.441213] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-09 (2.59 seconds). [2025-10-20T20:27:05.012115] [INFO ] [REMOTE ] OSK: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:27:05.018486] [INFO ] [REMOTE ] OSK: Full fetch complete (61020 records). [2025-10-20T20:27:05.018598] [DEBUG] [VALIDATOR ] OSK: Validating primary data. [2025-10-20T20:27:05.019381] [FATAL] [VALIDATOR ] OSK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:05.019470] [FATAL] [MAIN ] OSK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:05.019515] [WARN ] [MAIN ] OSK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:05.019585] [INFO ] [LOCAL ] OSK: Starting deletion of primary data. [2025-10-20T20:27:05.019690] [DEBUG] [LOCAL ] OSK: No primary data file found to delete. [2025-10-20T20:27:05.019747] [INFO ] [LOCAL ] OSK: Starting deletion of derivative data. [2025-10-20T20:27:05.019792] [DEBUG] [LOCAL ] OSK: No derivative data file found to delete. [2025-10-20T20:27:05.019849] [INFO ] [MAIN ] Update Progress: 1608/2367 symbols completed. OSK took 31.9s. Estimated remaining: 1h 37m 43s. Rate: 7.8 symbols/min [2025-10-20T20:27:05.053724] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-03. [2025-10-20T20:27:05.195012] [INFO ] [MAIN ] OVV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:05.226719] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T20:27:05.282210] [DEBUG] [LOCAL ] OVV: Loaded 54291 existing records from primary storage. [2025-10-20T20:27:05.282518] [INFO ] [MAIN ] OVV: Loaded 54291 existing records. [2025-10-20T20:27:05.459023] [WARN ] [MAIN ] OVV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:05.459257] [INFO ] [MAIN ] OVV: Wiping existing primary data before full refresh. [2025-10-20T20:27:05.459315] [INFO ] [LOCAL ] OVV: Starting deletion of primary data. [2025-10-20T20:27:05.459520] [DEBUG] [LOCAL ] OVV: Deleted primary data file. [2025-10-20T20:27:05.459575] [INFO ] [MAIN ] OVV: Starting full primary refresh. [2025-10-20T20:27:05.826800] [INFO ] [REMOTE ] OVV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:05.851326] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-01. [2025-10-20T20:27:06.651341] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-07. [2025-10-20T20:27:06.826534] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T20:27:07.451415] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-04. [2025-10-20T20:27:07.643532] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:27:08.201554] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T20:27:08.251370] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-08. [2025-10-20T20:27:09.051387] [INFO ] [REMOTE ] OSCR: Fetching primary data for 2025-10. [2025-10-20T20:27:09.197854] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T20:27:09.796895] [INFO ] [REMOTE ] OSCR: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:27:09.804870] [INFO ] [REMOTE ] OSCR: Full fetch complete (77348 records). [2025-10-20T20:27:09.805020] [DEBUG] [VALIDATOR ] OSCR: Validating primary data. [2025-10-20T20:27:09.805852] [FATAL] [VALIDATOR ] OSCR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:09.805936] [FATAL] [MAIN ] OSCR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:09.805990] [WARN ] [MAIN ] OSCR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:09.806075] [INFO ] [LOCAL ] OSCR: Starting deletion of primary data. [2025-10-20T20:27:09.806151] [DEBUG] [LOCAL ] OSCR: No primary data file found to delete. [2025-10-20T20:27:09.806198] [INFO ] [LOCAL ] OSCR: Starting deletion of derivative data. [2025-10-20T20:27:09.806350] [DEBUG] [LOCAL ] OSCR: Deleted derivative data file. [2025-10-20T20:27:09.806870] [INFO ] [MAIN ] Update Progress: 1609/2367 symbols completed. OSCR took 37.5s. Estimated remaining: 1h 37m 34s. Rate: 7.8 symbols/min [2025-10-20T20:27:09.862000] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-05. [2025-10-20T20:27:09.983785] [INFO ] [MAIN ] OWL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:10.038769] [DEBUG] [LOCAL ] OWL: Loaded 54336 existing records from primary storage. [2025-10-20T20:27:10.039122] [INFO ] [MAIN ] OWL: Loaded 54336 existing records. [2025-10-20T20:27:10.221042] [WARN ] [MAIN ] OWL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:10.221343] [INFO ] [MAIN ] OWL: Wiping existing primary data before full refresh. [2025-10-20T20:27:10.221442] [INFO ] [LOCAL ] OWL: Starting deletion of primary data. [2025-10-20T20:27:10.221691] [DEBUG] [LOCAL ] OWL: Deleted primary data file. [2025-10-20T20:27:10.221751] [INFO ] [MAIN ] OWL: Starting full primary refresh. [2025-10-20T20:27:10.619050] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-10-20T20:27:10.631793] [INFO ] [REMOTE ] OWL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:10.651337] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-09. [2025-10-20T20:27:11.451418] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-02. [2025-10-20T20:27:11.632693] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:27:12.078688] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T20:27:12.251380] [INFO ] [REMOTE ] OTIS: Fetching primary data for 2025-10. [2025-10-20T20:27:13.047095] [INFO ] [REMOTE ] OTIS: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:27:13.054054] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-01. [2025-10-20T20:27:13.055867] [INFO ] [REMOTE ] OTIS: Full fetch complete (76018 records). [2025-10-20T20:27:13.056086] [DEBUG] [VALIDATOR ] OTIS: Validating primary data. [2025-10-20T20:27:13.057119] [FATAL] [VALIDATOR ] OTIS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:13.057204] [FATAL] [MAIN ] OTIS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:13.057270] [WARN ] [MAIN ] OTIS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:13.057343] [INFO ] [LOCAL ] OTIS: Starting deletion of primary data. [2025-10-20T20:27:13.057410] [DEBUG] [LOCAL ] OTIS: No primary data file found to delete. [2025-10-20T20:27:13.057445] [INFO ] [LOCAL ] OTIS: Starting deletion of derivative data. [2025-10-20T20:27:13.057589] [DEBUG] [LOCAL ] OTIS: Deleted derivative data file. [2025-10-20T20:27:13.057701] [INFO ] [MAIN ] Update Progress: 1610/2367 symbols completed. OTIS took 28.6s. Estimated remaining: 1h 37m 24s. Rate: 7.8 symbols/min [2025-10-20T20:27:13.230560] [INFO ] [MAIN ] OWLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:13.265946] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T20:27:13.281048] [DEBUG] [LOCAL ] OWLT: Loaded 9544 existing records from primary storage. [2025-10-20T20:27:13.281277] [INFO ] [MAIN ] OWLT: Loaded 9544 existing records. [2025-10-20T20:27:13.459153] [WARN ] [MAIN ] OWLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:13.459358] [INFO ] [MAIN ] OWLT: Wiping existing primary data before full refresh. [2025-10-20T20:27:13.459405] [INFO ] [LOCAL ] OWLT: Starting deletion of primary data. [2025-10-20T20:27:13.459560] [DEBUG] [LOCAL ] OWLT: Deleted primary data file. [2025-10-20T20:27:13.459599] [INFO ] [MAIN ] OWLT: Starting full primary refresh. [2025-10-20T20:27:13.811243] [INFO ] [REMOTE ] OWLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:13.851341] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-01. [2025-10-20T20:27:14.336102] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-10-20T20:27:14.651339] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-06. [2025-10-20T20:27:14.918216] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T20:27:15.451423] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-02. [2025-10-20T20:27:15.916135] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T20:27:16.251393] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-03. [2025-10-20T20:27:16.344378] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T20:27:17.051406] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-07. [2025-10-20T20:27:17.851391] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-02. [2025-10-20T20:27:18.207985] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T20:27:18.651409] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-03. [2025-10-20T20:27:18.775293] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:27:19.449155] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T20:27:19.451723] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-04. [2025-10-20T20:27:19.673952] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T20:27:20.116994] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:27:20.251417] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-05. [2025-10-20T20:27:20.924399] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T20:27:21.051378] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-06. [2025-10-20T20:27:21.812729] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T20:27:21.851382] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-07. [2025-10-20T20:27:22.651445] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-04. [2025-10-20T20:27:22.690214] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:27:23.451460] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-08. [2025-10-20T20:27:24.251386] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-08. [2025-10-20T20:27:24.493616] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T20:27:25.051365] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-05. [2025-10-20T20:27:25.164112] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T20:27:25.205594] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T20:27:25.851373] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-09. [2025-10-20T20:27:26.651340] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-09. [2025-10-20T20:27:26.892278] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T20:27:27.092141] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T20:27:27.455985] [INFO ] [REMOTE ] OUT: Fetching primary data for 2025-10. [2025-10-20T20:27:27.456970] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:27:28.209412] [INFO ] [REMOTE ] OUT: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:27:28.215688] [INFO ] [REMOTE ] OUT: Full fetch complete (68819 records). [2025-10-20T20:27:28.215785] [DEBUG] [VALIDATOR ] OUT: Validating primary data. [2025-10-20T20:27:28.216430] [FATAL] [VALIDATOR ] OUT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:28.216513] [FATAL] [MAIN ] OUT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:28.216555] [WARN ] [MAIN ] OUT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:28.216625] [INFO ] [LOCAL ] OUT: Starting deletion of primary data. [2025-10-20T20:27:28.216732] [DEBUG] [LOCAL ] OUT: No primary data file found to delete. [2025-10-20T20:27:28.216786] [INFO ] [LOCAL ] OUT: Starting deletion of derivative data. [2025-10-20T20:27:28.216924] [DEBUG] [LOCAL ] OUT: Deleted derivative data file. [2025-10-20T20:27:28.216992] [INFO ] [MAIN ] Update Progress: 1611/2367 symbols completed. OUT took 31.7s. Estimated remaining: 1h 37m 20s. Rate: 7.8 symbols/min [2025-10-20T20:27:28.256475] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-06. [2025-10-20T20:27:28.385702] [INFO ] [MAIN ] OXM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:28.417517] [DEBUG] [LOCAL ] OXM: Loaded 33719 existing records from primary storage. [2025-10-20T20:27:28.417830] [INFO ] [MAIN ] OXM: Loaded 33719 existing records. [2025-10-20T20:27:28.593724] [WARN ] [MAIN ] OXM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:28.593940] [INFO ] [MAIN ] OXM: Wiping existing primary data before full refresh. [2025-10-20T20:27:28.593984] [INFO ] [LOCAL ] OXM: Starting deletion of primary data. [2025-10-20T20:27:28.594258] [DEBUG] [LOCAL ] OXM: Deleted primary data file. [2025-10-20T20:27:28.594312] [INFO ] [MAIN ] OXM: Starting full primary refresh. [2025-10-20T20:27:28.937152] [INFO ] [REMOTE ] OXM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:29.051369] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-01. [2025-10-20T20:27:29.851429] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-03. [2025-10-20T20:27:30.073475] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:27:30.352350] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T20:27:30.651390] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-07. [2025-10-20T20:27:31.451372] [INFO ] [REMOTE ] OWLT: Fetching primary data for 2025-10. [2025-10-20T20:27:31.894077] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T20:27:32.251351] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-02. [2025-10-20T20:27:32.316325] [INFO ] [REMOTE ] OWLT: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:27:32.318544] [INFO ] [REMOTE ] OWLT: Full fetch complete (12183 records). [2025-10-20T20:27:32.318612] [DEBUG] [VALIDATOR ] OWLT: Validating primary data. [2025-10-20T20:27:32.679259] [DEBUG] [VALIDATOR ] OWLT: All expected trading days are present. [2025-10-20T20:27:32.679353] [DEBUG] [VALIDATOR ] OWLT: Primary data validation passed. [2025-10-20T20:27:32.679750] [INFO ] [MAIN ] OWLT: Fetched primary data is valid. [2025-10-20T20:27:32.679838] [DEBUG] [LOCAL ] OWLT: Saving primary data. [2025-10-20T20:27:32.690983] [DEBUG] [LOCAL ] OWLT: Trimmed 12183 records to 9504 (keeping 200 days of data) [2025-10-20T20:27:32.731184] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T20:27:32.774402] [DEBUG] [LOCAL ] OWLT: Primary data saved (9504 records). [2025-10-20T20:27:32.774521] [INFO ] [MAIN ] OWLT: Full history saved. [2025-10-20T20:27:32.774593] [INFO ] [MAIN ] Update Progress: 1612/2367 symbols completed. OWLT took 19.7s. Estimated remaining: 1h 37m 10s. Rate: 7.8 symbols/min [2025-10-20T20:27:32.940778] [INFO ] [MAIN ] OXY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:32.940930] [DEBUG] [LOCAL ] OXY: No existing data file found in primary storage. [2025-10-20T20:27:32.940968] [WARN ] [MAIN ] OXY: No existing data found, performing full refresh instead. [2025-10-20T20:27:32.941075] [INFO ] [MAIN ] OXY: Wiping existing primary data before full refresh. [2025-10-20T20:27:32.941116] [INFO ] [LOCAL ] OXY: Starting deletion of primary data. [2025-10-20T20:27:32.941156] [DEBUG] [LOCAL ] OXY: No primary data file found to delete. [2025-10-20T20:27:32.941187] [INFO ] [MAIN ] OXY: Starting full primary refresh. [2025-10-20T20:27:33.056596] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-08. [2025-10-20T20:27:33.298785] [INFO ] [REMOTE ] OXY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:33.451509] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T20:27:33.851367] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-03. [2025-10-20T20:27:34.651350] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-04. [2025-10-20T20:27:35.054716] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T20:27:35.241044] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T20:27:35.451367] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-09. [2025-10-20T20:27:36.251408] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-01. [2025-10-20T20:27:36.585336] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T20:27:37.051426] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-04. [2025-10-20T20:27:37.296554] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T20:27:37.851450] [INFO ] [REMOTE ] OVV: Fetching primary data for 2025-10. [2025-10-20T20:27:38.569255] [INFO ] [REMOTE ] OVV: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T20:27:38.579503] [INFO ] [REMOTE ] OVV: Full fetch complete (77137 records). [2025-10-20T20:27:38.579905] [DEBUG] [VALIDATOR ] OVV: Validating primary data. [2025-10-20T20:27:38.602181] [FATAL] [VALIDATOR ] OVV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:38.603757] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T20:27:38.603844] [FATAL] [MAIN ] OVV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:38.604021] [WARN ] [MAIN ] OVV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:38.612039] [INFO ] [LOCAL ] OVV: Starting deletion of primary data. [2025-10-20T20:27:38.612507] [DEBUG] [LOCAL ] OVV: No primary data file found to delete. [2025-10-20T20:27:38.612589] [INFO ] [LOCAL ] OVV: Starting deletion of derivative data. [2025-10-20T20:27:38.612856] [DEBUG] [LOCAL ] OVV: Deleted derivative data file. [2025-10-20T20:27:38.613003] [INFO ] [MAIN ] Update Progress: 1613/2367 symbols completed. OVV took 33.6s. Estimated remaining: 1h 37m 2s. Rate: 7.8 symbols/min [2025-10-20T20:27:38.654763] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-05. [2025-10-20T20:27:38.803971] [INFO ] [MAIN ] PAAS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:38.843143] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-01 (2.59 seconds). [2025-10-20T20:27:38.899866] [DEBUG] [LOCAL ] PAAS: Loaded 54403 existing records from primary storage. [2025-10-20T20:27:38.900202] [INFO ] [MAIN ] PAAS: Loaded 54403 existing records. [2025-10-20T20:27:39.077980] [WARN ] [MAIN ] PAAS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:39.078223] [INFO ] [MAIN ] PAAS: Wiping existing primary data before full refresh. [2025-10-20T20:27:39.078274] [INFO ] [LOCAL ] PAAS: Starting deletion of primary data. [2025-10-20T20:27:39.078817] [DEBUG] [LOCAL ] PAAS: Deleted primary data file. [2025-10-20T20:27:39.078874] [INFO ] [MAIN ] PAAS: Starting full primary refresh. [2025-10-20T20:27:39.428168] [INFO ] [REMOTE ] PAAS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:39.451330] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-01. [2025-10-20T20:27:40.251377] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-05. [2025-10-20T20:27:40.656393] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T20:27:41.051441] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-06. [2025-10-20T20:27:41.412327] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T20:27:41.661581] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T20:27:41.851371] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-02. [2025-10-20T20:27:42.651364] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-02. [2025-10-20T20:27:42.815370] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T20:27:43.451432] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-07. [2025-10-20T20:27:44.251501] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-06. [2025-10-20T20:27:44.320071] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-02 (2.47 seconds). [2025-10-20T20:27:44.580480] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T20:27:45.051355] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-03. [2025-10-20T20:27:45.367217] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T20:27:45.469700] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T20:27:45.851353] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-03. [2025-10-20T20:27:46.651345] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-07. [2025-10-20T20:27:47.115467] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-10-20T20:27:47.451443] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-08. [2025-10-20T20:27:47.989315] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T20:27:48.207592] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T20:27:48.251332] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-08. [2025-10-20T20:27:49.025841] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T20:27:49.051369] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-04. [2025-10-20T20:27:49.584188] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T20:27:49.851400] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-04. [2025-10-20T20:27:50.651386] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-09. [2025-10-20T20:27:51.451352] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-09. [2025-10-20T20:27:51.605893] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-04 (2.55 seconds). [2025-10-20T20:27:52.043342] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-04 (2.19 seconds). [2025-10-20T20:27:52.088048] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T20:27:52.251433] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-05. [2025-10-20T20:27:53.051351] [INFO ] [REMOTE ] OXM: Fetching primary data for 2025-10. [2025-10-20T20:27:53.469918] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T20:27:53.716271] [INFO ] [REMOTE ] OXM: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:27:53.720902] [INFO ] [REMOTE ] OXM: Full fetch complete (46043 records). [2025-10-20T20:27:53.720988] [DEBUG] [VALIDATOR ] OXM: Validating primary data. [2025-10-20T20:27:53.721557] [FATAL] [VALIDATOR ] OXM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:53.721638] [FATAL] [MAIN ] OXM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:53.721717] [WARN ] [MAIN ] OXM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:53.721758] [INFO ] [LOCAL ] OXM: Starting deletion of primary data. [2025-10-20T20:27:53.721820] [DEBUG] [LOCAL ] OXM: No primary data file found to delete. [2025-10-20T20:27:53.721888] [INFO ] [LOCAL ] OXM: Starting deletion of derivative data. [2025-10-20T20:27:53.722105] [DEBUG] [LOCAL ] OXM: Deleted derivative data file. [2025-10-20T20:27:53.722205] [INFO ] [MAIN ] Update Progress: 1614/2367 symbols completed. OXM took 25.5s. Estimated remaining: 1h 36m 57s. Rate: 7.8 symbols/min [2025-10-20T20:27:53.866238] [INFO ] [REMOTE ] OWL: Fetching primary data for 2025-10. [2025-10-20T20:27:53.899060] [INFO ] [MAIN ] PAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:53.913403] [DEBUG] [LOCAL ] PAC: Loaded 11922 existing records from primary storage. [2025-10-20T20:27:53.913693] [INFO ] [MAIN ] PAC: Loaded 11922 existing records. [2025-10-20T20:27:54.090527] [WARN ] [MAIN ] PAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:54.090770] [INFO ] [MAIN ] PAC: Wiping existing primary data before full refresh. [2025-10-20T20:27:54.090822] [INFO ] [LOCAL ] PAC: Starting deletion of primary data. [2025-10-20T20:27:54.091010] [DEBUG] [LOCAL ] PAC: Deleted primary data file. [2025-10-20T20:27:54.091067] [INFO ] [MAIN ] PAC: Starting full primary refresh. [2025-10-20T20:27:54.472838] [INFO ] [REMOTE ] PAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:54.498980] [INFO ] [REMOTE ] OWL: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:27:54.509033] [INFO ] [REMOTE ] OWL: Full fetch complete (77321 records). [2025-10-20T20:27:54.509322] [DEBUG] [VALIDATOR ] OWL: Validating primary data. [2025-10-20T20:27:54.510483] [FATAL] [VALIDATOR ] OWL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:27:54.510589] [FATAL] [MAIN ] OWL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:27:54.510691] [WARN ] [MAIN ] OWL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:27:54.510791] [INFO ] [LOCAL ] OWL: Starting deletion of primary data. [2025-10-20T20:27:54.510896] [DEBUG] [LOCAL ] OWL: No primary data file found to delete. [2025-10-20T20:27:54.510945] [INFO ] [LOCAL ] OWL: Starting deletion of derivative data. [2025-10-20T20:27:54.511155] [DEBUG] [LOCAL ] OWL: Deleted derivative data file. [2025-10-20T20:27:54.511836] [INFO ] [MAIN ] Update Progress: 1615/2367 symbols completed. OWL took 44.7s. Estimated remaining: 1h 36m 47s. Rate: 7.8 symbols/min [2025-10-20T20:27:54.662580] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-01. [2025-10-20T20:27:54.683177] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-05 (2.43 seconds). [2025-10-20T20:27:54.765003] [INFO ] [MAIN ] PACK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:27:54.802500] [DEBUG] [LOCAL ] PACK: Loaded 32458 existing records from primary storage. [2025-10-20T20:27:54.802930] [INFO ] [MAIN ] PACK: Loaded 32458 existing records. [2025-10-20T20:27:55.002080] [WARN ] [MAIN ] PACK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:27:55.002401] [INFO ] [MAIN ] PACK: Wiping existing primary data before full refresh. [2025-10-20T20:27:55.002458] [INFO ] [LOCAL ] PACK: Starting deletion of primary data. [2025-10-20T20:27:55.002756] [DEBUG] [LOCAL ] PACK: Deleted primary data file. [2025-10-20T20:27:55.002813] [INFO ] [MAIN ] PACK: Starting full primary refresh. [2025-10-20T20:27:55.381191] [INFO ] [REMOTE ] PACK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:27:55.451445] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-01. [2025-10-20T20:27:55.618746] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:27:56.251388] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-05. [2025-10-20T20:27:56.666884] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T20:27:57.051392] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-02. [2025-10-20T20:27:57.851359] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-02. [2025-10-20T20:27:58.019335] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T20:27:58.266849] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-05 (2.02 seconds). [2025-10-20T20:27:58.651408] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-06. [2025-10-20T20:27:59.102086] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-02 (1.25 seconds). [2025-10-20T20:27:59.451409] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-06. [2025-10-20T20:28:00.251424] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-03. [2025-10-20T20:28:01.051381] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-03. [2025-10-20T20:28:01.423218] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-06 (2.77 seconds). [2025-10-20T20:28:01.510972] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T20:28:01.555941] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T20:28:01.851386] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-04. [2025-10-20T20:28:02.551716] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:28:02.651458] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-07. [2025-10-20T20:28:02.899945] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T20:28:03.451440] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-05. [2025-10-20T20:28:04.251349] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-07. [2025-10-20T20:28:04.550565] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T20:28:04.750447] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-10-20T20:28:05.051367] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-06. [2025-10-20T20:28:05.851408] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-08. [2025-10-20T20:28:06.023957] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T20:28:06.651356] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-07. [2025-10-20T20:28:06.798496] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-07 (2.55 seconds). [2025-10-20T20:28:07.451352] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-04. [2025-10-20T20:28:07.670820] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T20:28:07.940538] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T20:28:08.251379] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-09. [2025-10-20T20:28:08.696028] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T20:28:09.051425] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-08. [2025-10-20T20:28:09.851416] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-08. [2025-10-20T20:28:10.460042] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T20:28:10.651387] [INFO ] [REMOTE ] PAAS: Fetching primary data for 2025-10. [2025-10-20T20:28:10.755836] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T20:28:11.308316] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-08 (2.26 seconds). [2025-10-20T20:28:11.424738] [INFO ] [REMOTE ] PAAS: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:28:11.432325] [INFO ] [REMOTE ] PAAS: Full fetch complete (77253 records). [2025-10-20T20:28:11.432475] [DEBUG] [VALIDATOR ] PAAS: Validating primary data. [2025-10-20T20:28:11.433298] [FATAL] [VALIDATOR ] PAAS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:28:11.433401] [FATAL] [MAIN ] PAAS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:28:11.433472] [WARN ] [MAIN ] PAAS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:28:11.433566] [INFO ] [LOCAL ] PAAS: Starting deletion of primary data. [2025-10-20T20:28:11.433675] [DEBUG] [LOCAL ] PAAS: No primary data file found to delete. [2025-10-20T20:28:11.433731] [INFO ] [LOCAL ] PAAS: Starting deletion of derivative data. [2025-10-20T20:28:11.433882] [DEBUG] [LOCAL ] PAAS: Deleted derivative data file. [2025-10-20T20:28:11.433969] [INFO ] [MAIN ] Update Progress: 1616/2367 symbols completed. PAAS took 32.8s. Estimated remaining: 1h 36m 43s. Rate: 7.8 symbols/min [2025-10-20T20:28:11.456514] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-09. [2025-10-20T20:28:11.610522] [INFO ] [MAIN ] PACS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:11.643974] [DEBUG] [LOCAL ] PACS: Loaded 35165 existing records from primary storage. [2025-10-20T20:28:11.644249] [INFO ] [MAIN ] PACS: Loaded 35165 existing records. [2025-10-20T20:28:11.826063] [WARN ] [MAIN ] PACS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:11.826268] [INFO ] [MAIN ] PACS: Wiping existing primary data before full refresh. [2025-10-20T20:28:11.826314] [INFO ] [LOCAL ] PACS: Starting deletion of primary data. [2025-10-20T20:28:11.826721] [DEBUG] [LOCAL ] PACS: Deleted primary data file. [2025-10-20T20:28:11.826779] [INFO ] [MAIN ] PACS: Starting full primary refresh. [2025-10-20T20:28:12.202838] [INFO ] [REMOTE ] PACS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:12.251371] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-09. [2025-10-20T20:28:12.389442] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:28:13.051355] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-05. [2025-10-20T20:28:13.110678] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T20:28:13.851373] [INFO ] [REMOTE ] PAC: Fetching primary data for 2025-10. [2025-10-20T20:28:14.564287] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T20:28:14.651356] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-06. [2025-10-20T20:28:14.807403] [INFO ] [REMOTE ] PAC: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T20:28:14.809772] [INFO ] [REMOTE ] PAC: Full fetch complete (16823 records). [2025-10-20T20:28:14.809861] [DEBUG] [VALIDATOR ] PAC: Validating primary data. [2025-10-20T20:28:15.174575] [DEBUG] [VALIDATOR ] PAC: All expected trading days are present. [2025-10-20T20:28:15.174699] [DEBUG] [VALIDATOR ] PAC: Primary data validation passed. [2025-10-20T20:28:15.175156] [INFO ] [MAIN ] PAC: Fetched primary data is valid. [2025-10-20T20:28:15.175217] [DEBUG] [LOCAL ] PAC: Saving primary data. [2025-10-20T20:28:15.175848] [DEBUG] [LOCAL ] PAC: Trimmed 16823 records to 11741 (keeping 200 days of data) [2025-10-20T20:28:15.250482] [DEBUG] [LOCAL ] PAC: Primary data saved (11741 records). [2025-10-20T20:28:15.250616] [INFO ] [MAIN ] PAC: Full history saved. [2025-10-20T20:28:15.250735] [INFO ] [MAIN ] Update Progress: 1617/2367 symbols completed. PAC took 21.5s. Estimated remaining: 1h 36m 34s. Rate: 7.8 symbols/min [2025-10-20T20:28:15.420369] [INFO ] [MAIN ] PAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:15.443316] [DEBUG] [LOCAL ] PAG: Loaded 21914 existing records from primary storage. [2025-10-20T20:28:15.443558] [INFO ] [MAIN ] PAG: Loaded 21914 existing records. [2025-10-20T20:28:15.456592] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-01. [2025-10-20T20:28:15.624062] [WARN ] [MAIN ] PAG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:15.624298] [INFO ] [MAIN ] PAG: Wiping existing primary data before full refresh. [2025-10-20T20:28:15.624346] [INFO ] [LOCAL ] PAG: Starting deletion of primary data. [2025-10-20T20:28:15.624539] [DEBUG] [LOCAL ] PAG: Deleted primary data file. [2025-10-20T20:28:15.624576] [INFO ] [MAIN ] PAG: Starting full primary refresh. [2025-10-20T20:28:15.910760] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T20:28:16.098587] [INFO ] [REMOTE ] PAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:16.251418] [INFO ] [REMOTE ] OXY: Fetching primary data for 2025-10. [2025-10-20T20:28:16.992669] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T20:28:17.035945] [INFO ] [REMOTE ] OXY: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:28:17.047562] [INFO ] [REMOTE ] OXY: Full fetch complete (77195 records). [2025-10-20T20:28:17.047798] [DEBUG] [VALIDATOR ] OXY: Validating primary data. [2025-10-20T20:28:17.048940] [FATAL] [VALIDATOR ] OXY: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:28:17.049060] [FATAL] [MAIN ] OXY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:28:17.049124] [WARN ] [MAIN ] OXY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:28:17.049213] [INFO ] [LOCAL ] OXY: Starting deletion of primary data. [2025-10-20T20:28:17.049306] [DEBUG] [LOCAL ] OXY: No primary data file found to delete. [2025-10-20T20:28:17.049368] [INFO ] [LOCAL ] OXY: Starting deletion of derivative data. [2025-10-20T20:28:17.049439] [DEBUG] [LOCAL ] OXY: No derivative data file found to delete. [2025-10-20T20:28:17.049529] [INFO ] [MAIN ] Update Progress: 1618/2367 symbols completed. OXY took 44.3s. Estimated remaining: 1h 36m 23s. Rate: 7.8 symbols/min [2025-10-20T20:28:17.056447] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-01. [2025-10-20T20:28:17.245495] [INFO ] [MAIN ] PAGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:17.298210] [DEBUG] [LOCAL ] PAGS: Loaded 53590 existing records from primary storage. [2025-10-20T20:28:17.298705] [INFO ] [MAIN ] PAGS: Loaded 53590 existing records. [2025-10-20T20:28:17.507243] [WARN ] [MAIN ] PAGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:17.507557] [INFO ] [MAIN ] PAGS: Wiping existing primary data before full refresh. [2025-10-20T20:28:17.507610] [INFO ] [LOCAL ] PAGS: Starting deletion of primary data. [2025-10-20T20:28:17.508247] [DEBUG] [LOCAL ] PAGS: Deleted primary data file. [2025-10-20T20:28:17.508304] [INFO ] [MAIN ] PAGS: Starting full primary refresh. [2025-10-20T20:28:17.854035] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-07. [2025-10-20T20:28:17.853965] [INFO ] [REMOTE ] PAGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:18.215993] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T20:28:18.651361] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-01. [2025-10-20T20:28:19.177099] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T20:28:19.451384] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-02. [2025-10-20T20:28:20.251420] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-02. [2025-10-20T20:28:20.467473] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T20:28:20.805314] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T20:28:21.051377] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-02. [2025-10-20T20:28:21.473809] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T20:28:21.851387] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-03. [2025-10-20T20:28:22.651416] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-08. [2025-10-20T20:28:22.918938] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T20:28:23.074781] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T20:28:23.451470] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-03. [2025-10-20T20:28:24.183715] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T20:28:24.251381] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-09. [2025-10-20T20:28:24.834796] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T20:28:25.051379] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-03. [2025-10-20T20:28:25.788489] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-10-20T20:28:25.851358] [INFO ] [REMOTE ] PACK: Fetching primary data for 2025-10. [2025-10-20T20:28:26.651376] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-04. [2025-10-20T20:28:26.803424] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T20:28:27.225701] [INFO ] [REMOTE ] PACK: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T20:28:27.231431] [INFO ] [REMOTE ] PACK: Full fetch complete (45195 records). [2025-10-20T20:28:27.231556] [DEBUG] [VALIDATOR ] PACK: Validating primary data. [2025-10-20T20:28:27.456580] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-04. [2025-10-20T20:28:27.645423] [DEBUG] [VALIDATOR ] PACK: All expected trading days are present. [2025-10-20T20:28:27.645683] [DEBUG] [VALIDATOR ] PACK: Primary data validation passed. [2025-10-20T20:28:27.647754] [INFO ] [MAIN ] PACK: Fetched primary data is valid. [2025-10-20T20:28:27.647845] [DEBUG] [LOCAL ] PACK: Saving primary data. [2025-10-20T20:28:27.649754] [DEBUG] [LOCAL ] PACK: Trimmed 45195 records to 32000 (keeping 200 days of data) [2025-10-20T20:28:27.858323] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T20:28:27.857226] [DEBUG] [LOCAL ] PACK: Primary data saved (32000 records). [2025-10-20T20:28:27.866801] [INFO ] [MAIN ] PACK: Full history saved. [2025-10-20T20:28:27.866935] [INFO ] [MAIN ] Update Progress: 1619/2367 symbols completed. PACK took 33.4s. Estimated remaining: 1h 36m 17s. Rate: 7.8 symbols/min [2025-10-20T20:28:28.051860] [INFO ] [MAIN ] PAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:28.057010] [DEBUG] [LOCAL ] PAI: Loaded 2805 existing records from primary storage. [2025-10-20T20:28:28.057153] [INFO ] [MAIN ] PAI: Loaded 2805 existing records. [2025-10-20T20:28:28.233169] [WARN ] [MAIN ] PAI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:28.233425] [INFO ] [MAIN ] PAI: Wiping existing primary data before full refresh. [2025-10-20T20:28:28.233472] [INFO ] [LOCAL ] PAI: Starting deletion of primary data. [2025-10-20T20:28:28.233624] [DEBUG] [LOCAL ] PAI: Deleted primary data file. [2025-10-20T20:28:28.233713] [INFO ] [MAIN ] PAI: Starting full primary refresh. [2025-10-20T20:28:28.263226] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-04. [2025-10-20T20:28:28.578155] [INFO ] [REMOTE ] PAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:29.051412] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-05. [2025-10-20T20:28:29.371427] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T20:28:29.674989] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T20:28:29.851404] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-05. [2025-10-20T20:28:29.972669] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T20:28:30.651433] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-06. [2025-10-20T20:28:31.451441] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-05. [2025-10-20T20:28:31.715253] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T20:28:32.176164] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T20:28:32.251332] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-07. [2025-10-20T20:28:32.582418] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T20:28:33.051387] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-06. [2025-10-20T20:28:33.728243] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T20:28:33.851380] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-06. [2025-10-20T20:28:34.651365] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-01. [2025-10-20T20:28:34.760455] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T20:28:34.922159] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T20:28:35.256842] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T20:28:35.451381] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-07. [2025-10-20T20:28:36.251370] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-02. [2025-10-20T20:28:36.542809] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T20:28:36.797677] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T20:28:37.051367] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-03. [2025-10-20T20:28:37.435463] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-03 (0.38 seconds). [2025-10-20T20:28:37.851379] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-07. [2025-10-20T20:28:38.651381] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-08. [2025-10-20T20:28:39.451394] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-04. [2025-10-20T20:28:39.869692] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T20:28:40.074617] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T20:28:40.075244] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T20:28:40.251346] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-05. [2025-10-20T20:28:40.760688] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T20:28:41.051390] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-06. [2025-10-20T20:28:41.576935] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T20:28:41.851426] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-08. [2025-10-20T20:28:42.651358] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-09. [2025-10-20T20:28:42.966132] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T20:28:43.451379] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-08. [2025-10-20T20:28:44.228348] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T20:28:44.251342] [INFO ] [REMOTE ] PACS: Fetching primary data for 2025-10. [2025-10-20T20:28:45.051395] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-07. [2025-10-20T20:28:45.430598] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-10-20T20:28:45.606190] [INFO ] [REMOTE ] PACS: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T20:28:45.611625] [INFO ] [REMOTE ] PACS: Full fetch complete (50549 records). [2025-10-20T20:28:45.611779] [DEBUG] [VALIDATOR ] PACS: Validating primary data. [2025-10-20T20:28:45.693024] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T20:28:45.856491] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-09. [2025-10-20T20:28:46.003965] [DEBUG] [VALIDATOR ] PACS: All expected trading days are present. [2025-10-20T20:28:46.004046] [DEBUG] [VALIDATOR ] PACS: Primary data validation passed. [2025-10-20T20:28:46.004892] [INFO ] [MAIN ] PACS: Fetched primary data is valid. [2025-10-20T20:28:46.004953] [DEBUG] [LOCAL ] PACS: Saving primary data. [2025-10-20T20:28:46.006015] [DEBUG] [LOCAL ] PACS: Trimmed 50549 records to 34736 (keeping 200 days of data) [2025-10-20T20:28:46.203961] [DEBUG] [LOCAL ] PACS: Primary data saved (34736 records). [2025-10-20T20:28:46.204104] [INFO ] [MAIN ] PACS: Full history saved. [2025-10-20T20:28:46.204181] [INFO ] [MAIN ] Update Progress: 1620/2367 symbols completed. PACS took 34.8s. Estimated remaining: 1h 36m 14s. Rate: 7.8 symbols/min [2025-10-20T20:28:46.392600] [INFO ] [MAIN ] PAII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:46.392788] [DEBUG] [LOCAL ] PAII: No existing data file found in primary storage. [2025-10-20T20:28:46.392831] [WARN ] [MAIN ] PAII: No existing data found, performing full refresh instead. [2025-10-20T20:28:46.392941] [INFO ] [MAIN ] PAII: Wiping existing primary data before full refresh. [2025-10-20T20:28:46.392984] [INFO ] [LOCAL ] PAII: Starting deletion of primary data. [2025-10-20T20:28:46.393024] [DEBUG] [LOCAL ] PAII: No primary data file found to delete. [2025-10-20T20:28:46.393057] [INFO ] [MAIN ] PAII: Starting full primary refresh. [2025-10-20T20:28:46.656520] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-08. [2025-10-20T20:28:46.753662] [INFO ] [REMOTE ] PAII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:47.241782] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T20:28:47.451366] [INFO ] [REMOTE ] PAII: Fetching primary data for 2025-01. [2025-10-20T20:28:47.770332] [DEBUG] [REMOTE ] PAII: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:28:47.770679] [WARN ] [REMOTE ] PAII: No data returned for month 2025-01 [2025-10-20T20:28:47.770770] [FATAL] [REMOTE ] PAII: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:28:47.770816] [FATAL] [MAIN ] PAII: Failed to fetch primary data. [2025-10-20T20:28:47.770862] [WARN ] [MAIN ] PAII: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:28:47.770946] [INFO ] [LOCAL ] PAII: Starting deletion of primary data. [2025-10-20T20:28:47.771022] [DEBUG] [LOCAL ] PAII: No primary data file found to delete. [2025-10-20T20:28:47.771093] [INFO ] [LOCAL ] PAII: Starting deletion of derivative data. [2025-10-20T20:28:47.771252] [DEBUG] [LOCAL ] PAII: No derivative data file found to delete. [2025-10-20T20:28:47.771463] [INFO ] [MAIN ] Update Progress: 1621/2367 symbols completed. PAII took 1.6s. Estimated remaining: 1h 36m 3s. Rate: 7.8 symbols/min [2025-10-20T20:28:47.887389] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T20:28:48.010917] [INFO ] [MAIN ] PAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:48.039277] [DEBUG] [LOCAL ] PAM: Loaded 29195 existing records from primary storage. [2025-10-20T20:28:48.039533] [INFO ] [MAIN ] PAM: Loaded 29195 existing records. [2025-10-20T20:28:48.237358] [WARN ] [MAIN ] PAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:48.237568] [INFO ] [MAIN ] PAM: Wiping existing primary data before full refresh. [2025-10-20T20:28:48.237613] [INFO ] [LOCAL ] PAM: Starting deletion of primary data. [2025-10-20T20:28:48.237865] [DEBUG] [LOCAL ] PAM: Deleted primary data file. [2025-10-20T20:28:48.237918] [INFO ] [MAIN ] PAM: Starting full primary refresh. [2025-10-20T20:28:48.258223] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-09. [2025-10-20T20:28:48.595449] [INFO ] [REMOTE ] PAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:48.976753] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T20:28:49.051403] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-01. [2025-10-20T20:28:49.851389] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-09. [2025-10-20T20:28:50.253900] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T20:28:50.651444] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-02. [2025-10-20T20:28:50.902244] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-09 (1.05 seconds). [2025-10-20T20:28:51.451397] [INFO ] [REMOTE ] PAGS: Fetching primary data for 2025-10. [2025-10-20T20:28:52.083348] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:28:52.165698] [INFO ] [REMOTE ] PAGS: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:28:52.173207] [INFO ] [REMOTE ] PAGS: Full fetch complete (76142 records). [2025-10-20T20:28:52.173323] [DEBUG] [VALIDATOR ] PAGS: Validating primary data. [2025-10-20T20:28:52.174139] [FATAL] [VALIDATOR ] PAGS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:28:52.174221] [FATAL] [MAIN ] PAGS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:28:52.174295] [WARN ] [MAIN ] PAGS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:28:52.174365] [INFO ] [LOCAL ] PAGS: Starting deletion of primary data. [2025-10-20T20:28:52.174429] [DEBUG] [LOCAL ] PAGS: No primary data file found to delete. [2025-10-20T20:28:52.174464] [INFO ] [LOCAL ] PAGS: Starting deletion of derivative data. [2025-10-20T20:28:52.174613] [DEBUG] [LOCAL ] PAGS: Deleted derivative data file. [2025-10-20T20:28:52.174727] [INFO ] [MAIN ] Update Progress: 1622/2367 symbols completed. PAGS took 35.1s. Estimated remaining: 1h 35m 54s. Rate: 7.8 symbols/min [2025-10-20T20:28:52.256568] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-03. [2025-10-20T20:28:52.351557] [INFO ] [MAIN ] PAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:52.386371] [DEBUG] [LOCAL ] PAR: Loaded 34624 existing records from primary storage. [2025-10-20T20:28:52.386696] [INFO ] [MAIN ] PAR: Loaded 34624 existing records. [2025-10-20T20:28:52.586819] [WARN ] [MAIN ] PAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:52.587047] [INFO ] [MAIN ] PAR: Wiping existing primary data before full refresh. [2025-10-20T20:28:52.587102] [INFO ] [LOCAL ] PAR: Starting deletion of primary data. [2025-10-20T20:28:52.587299] [DEBUG] [LOCAL ] PAR: Deleted primary data file. [2025-10-20T20:28:52.587350] [INFO ] [MAIN ] PAR: Starting full primary refresh. [2025-10-20T20:28:52.992283] [INFO ] [REMOTE ] PAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:53.051382] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-01. [2025-10-20T20:28:53.496951] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T20:28:53.851471] [INFO ] [REMOTE ] PAI: Fetching primary data for 2025-10. [2025-10-20T20:28:54.385848] [INFO ] [REMOTE ] PAI: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:28:54.387348] [INFO ] [REMOTE ] PAI: Full fetch complete (4153 records). [2025-10-20T20:28:54.387694] [DEBUG] [VALIDATOR ] PAI: Validating primary data. [2025-10-20T20:28:54.388986] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T20:28:54.656467] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-04. [2025-10-20T20:28:54.758716] [DEBUG] [VALIDATOR ] PAI: All expected trading days are present. [2025-10-20T20:28:54.758797] [DEBUG] [VALIDATOR ] PAI: Primary data validation passed. [2025-10-20T20:28:54.758927] [INFO ] [MAIN ] PAI: Fetched primary data is valid. [2025-10-20T20:28:54.758963] [DEBUG] [LOCAL ] PAI: Saving primary data. [2025-10-20T20:28:54.759318] [DEBUG] [LOCAL ] PAI: Trimmed 4153 records to 2814 (keeping 200 days of data) [2025-10-20T20:28:54.776200] [DEBUG] [LOCAL ] PAI: Primary data saved (2814 records). [2025-10-20T20:28:54.776259] [INFO ] [MAIN ] PAI: Full history saved. [2025-10-20T20:28:54.776322] [INFO ] [MAIN ] Update Progress: 1623/2367 symbols completed. PAI took 26.9s. Estimated remaining: 1h 35m 44s. Rate: 7.8 symbols/min [2025-10-20T20:28:54.944578] [INFO ] [MAIN ] PARR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:54.987668] [DEBUG] [LOCAL ] PARR: Loaded 48792 existing records from primary storage. [2025-10-20T20:28:54.988089] [INFO ] [MAIN ] PARR: Loaded 48792 existing records. [2025-10-20T20:28:55.170582] [WARN ] [MAIN ] PARR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:55.170852] [INFO ] [MAIN ] PARR: Wiping existing primary data before full refresh. [2025-10-20T20:28:55.170922] [INFO ] [LOCAL ] PARR: Starting deletion of primary data. [2025-10-20T20:28:55.171115] [DEBUG] [LOCAL ] PARR: Deleted primary data file. [2025-10-20T20:28:55.171154] [INFO ] [MAIN ] PARR: Starting full primary refresh. [2025-10-20T20:28:55.460081] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-02. [2025-10-20T20:28:55.546513] [INFO ] [REMOTE ] PARR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:56.020560] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T20:28:56.251404] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-01. [2025-10-20T20:28:56.846191] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T20:28:57.051549] [INFO ] [REMOTE ] PAG: Fetching primary data for 2025-10. [2025-10-20T20:28:57.640689] [INFO ] [REMOTE ] PAG: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T20:28:57.643910] [INFO ] [REMOTE ] PAG: Full fetch complete (30573 records). [2025-10-20T20:28:57.643983] [DEBUG] [VALIDATOR ] PAG: Validating primary data. [2025-10-20T20:28:57.644414] [FATAL] [VALIDATOR ] PAG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:28:57.644501] [FATAL] [MAIN ] PAG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:28:57.644542] [WARN ] [MAIN ] PAG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:28:57.644579] [INFO ] [LOCAL ] PAG: Starting deletion of primary data. [2025-10-20T20:28:57.644632] [DEBUG] [LOCAL ] PAG: No primary data file found to delete. [2025-10-20T20:28:57.644705] [INFO ] [LOCAL ] PAG: Starting deletion of derivative data. [2025-10-20T20:28:57.644877] [DEBUG] [LOCAL ] PAG: Deleted derivative data file. [2025-10-20T20:28:57.644958] [INFO ] [MAIN ] Update Progress: 1624/2367 symbols completed. PAG took 42.4s. Estimated remaining: 1h 35m 34s. Rate: 7.8 symbols/min [2025-10-20T20:28:57.819701] [INFO ] [MAIN ] PATH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:28:57.849737] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:28:57.858603] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-05. [2025-10-20T20:28:57.907337] [DEBUG] [LOCAL ] PATH: Loaded 54460 existing records from primary storage. [2025-10-20T20:28:57.907669] [INFO ] [MAIN ] PATH: Loaded 54460 existing records. [2025-10-20T20:28:58.083863] [WARN ] [MAIN ] PATH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:28:58.084078] [INFO ] [MAIN ] PATH: Wiping existing primary data before full refresh. [2025-10-20T20:28:58.084125] [INFO ] [LOCAL ] PATH: Starting deletion of primary data. [2025-10-20T20:28:58.084378] [DEBUG] [LOCAL ] PATH: Deleted primary data file. [2025-10-20T20:28:58.084434] [INFO ] [MAIN ] PATH: Starting full primary refresh. [2025-10-20T20:28:58.442516] [INFO ] [REMOTE ] PATH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:28:58.651362] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-01. [2025-10-20T20:28:58.825344] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T20:28:59.451411] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-06. [2025-10-20T20:29:00.251452] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-02. [2025-10-20T20:29:00.602161] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T20:29:01.016119] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-01 (2.36 seconds). [2025-10-20T20:29:01.051371] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-02. [2025-10-20T20:29:01.789150] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T20:29:01.851396] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-03. [2025-10-20T20:29:02.651366] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-07. [2025-10-20T20:29:03.400293] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-02 (2.35 seconds). [2025-10-20T20:29:03.451333] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-03. [2025-10-20T20:29:03.608588] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T20:29:03.963121] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T20:29:04.251416] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-03. [2025-10-20T20:29:05.051411] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-04. [2025-10-20T20:29:05.247221] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T20:29:05.851341] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-04. [2025-10-20T20:29:05.924986] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-03 (2.47 seconds). [2025-10-20T20:29:06.651370] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-04. [2025-10-20T20:29:06.759815] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T20:29:07.354282] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T20:29:07.451350] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-05. [2025-10-20T20:29:08.251384] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-05. [2025-10-20T20:29:08.818402] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T20:29:08.884869] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T20:29:09.051414] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-06. [2025-10-20T20:29:09.851399] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-05. [2025-10-20T20:29:10.017388] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:29:10.171463] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T20:29:10.651413] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-07. [2025-10-20T20:29:11.451427] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-06. [2025-10-20T20:29:11.959992] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T20:29:12.073939] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-05 (2.22 seconds). [2025-10-20T20:29:12.251384] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-08. [2025-10-20T20:29:12.823073] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T20:29:13.051385] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-07. [2025-10-20T20:29:13.206800] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T20:29:13.851434] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-09. [2025-10-20T20:29:14.651383] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-08. [2025-10-20T20:29:14.709100] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T20:29:15.364070] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T20:29:15.451370] [INFO ] [REMOTE ] PAM: Fetching primary data for 2025-10. [2025-10-20T20:29:15.814082] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:29:16.251392] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-08. [2025-10-20T20:29:16.772682] [INFO ] [REMOTE ] PAM: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T20:29:16.780437] [INFO ] [REMOTE ] PAM: Full fetch complete (42443 records). [2025-10-20T20:29:16.780575] [DEBUG] [VALIDATOR ] PAM: Validating primary data. [2025-10-20T20:29:17.061536] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-09. [2025-10-20T20:29:17.161195] [DEBUG] [VALIDATOR ] PAM: All expected trading days are present. [2025-10-20T20:29:17.161278] [DEBUG] [VALIDATOR ] PAM: Primary data validation passed. [2025-10-20T20:29:17.162150] [INFO ] [MAIN ] PAM: Fetched primary data is valid. [2025-10-20T20:29:17.162214] [DEBUG] [LOCAL ] PAM: Saving primary data. [2025-10-20T20:29:17.163227] [DEBUG] [LOCAL ] PAM: Trimmed 42443 records to 28940 (keeping 200 days of data) [2025-10-20T20:29:17.331331] [DEBUG] [LOCAL ] PAM: Primary data saved (28940 records). [2025-10-20T20:29:17.331472] [INFO ] [MAIN ] PAM: Full history saved. [2025-10-20T20:29:17.331554] [INFO ] [MAIN ] Update Progress: 1625/2367 symbols completed. PAM took 29.6s. Estimated remaining: 1h 35m 32s. Rate: 7.8 symbols/min [2025-10-20T20:29:17.504579] [INFO ] [MAIN ] PAXS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:17.524482] [DEBUG] [LOCAL ] PAXS: Loaded 21123 existing records from primary storage. [2025-10-20T20:29:17.524740] [INFO ] [MAIN ] PAXS: Loaded 21123 existing records. [2025-10-20T20:29:17.708547] [WARN ] [MAIN ] PAXS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:17.709593] [INFO ] [MAIN ] PAXS: Wiping existing primary data before full refresh. [2025-10-20T20:29:17.709684] [INFO ] [LOCAL ] PAXS: Starting deletion of primary data. [2025-10-20T20:29:17.710378] [DEBUG] [LOCAL ] PAXS: Deleted primary data file. [2025-10-20T20:29:17.710436] [INFO ] [MAIN ] PAXS: Starting full primary refresh. [2025-10-20T20:29:17.855823] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-06. [2025-10-20T20:29:18.074217] [INFO ] [REMOTE ] PAXS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:18.108276] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T20:29:18.535524] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T20:29:18.651406] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-09. [2025-10-20T20:29:19.451424] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-01. [2025-10-20T20:29:20.100682] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-06 (2.24 seconds). [2025-10-20T20:29:20.251427] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-07. [2025-10-20T20:29:20.503607] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T20:29:20.627297] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T20:29:21.051401] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-02. [2025-10-20T20:29:21.851425] [INFO ] [REMOTE ] PAR: Fetching primary data for 2025-10. [2025-10-20T20:29:22.074562] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T20:29:22.594207] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-07 (2.34 seconds). [2025-10-20T20:29:22.651332] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-03. [2025-10-20T20:29:23.369738] [INFO ] [REMOTE ] PAR: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T20:29:23.376013] [INFO ] [REMOTE ] PAR: Full fetch complete (49175 records). [2025-10-20T20:29:23.376126] [DEBUG] [VALIDATOR ] PAR: Validating primary data. [2025-10-20T20:29:23.466944] [INFO ] [REMOTE ] PARR: Fetching primary data for 2025-10. [2025-10-20T20:29:23.526564] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T20:29:23.785287] [DEBUG] [VALIDATOR ] PAR: All expected trading days are present. [2025-10-20T20:29:23.785381] [DEBUG] [VALIDATOR ] PAR: Primary data validation passed. [2025-10-20T20:29:23.786220] [INFO ] [MAIN ] PAR: Fetched primary data is valid. [2025-10-20T20:29:23.786285] [DEBUG] [LOCAL ] PAR: Saving primary data. [2025-10-20T20:29:23.787378] [DEBUG] [LOCAL ] PAR: Trimmed 49175 records to 34290 (keeping 200 days of data) [2025-10-20T20:29:24.005128] [DEBUG] [LOCAL ] PAR: Primary data saved (34290 records). [2025-10-20T20:29:24.005275] [INFO ] [MAIN ] PAR: Full history saved. [2025-10-20T20:29:24.005355] [INFO ] [MAIN ] Update Progress: 1626/2367 symbols completed. PAR took 31.8s. Estimated remaining: 1h 35m 24s. Rate: 7.8 symbols/min [2025-10-20T20:29:24.205724] [INFO ] [MAIN ] PAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:24.247404] [INFO ] [REMOTE ] PARR: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:29:24.253769] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-04. [2025-10-20T20:29:24.255079] [INFO ] [REMOTE ] PARR: Full fetch complete (68772 records). [2025-10-20T20:29:24.255174] [DEBUG] [VALIDATOR ] PARR: Validating primary data. [2025-10-20T20:29:24.256039] [FATAL] [VALIDATOR ] PARR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:29:24.256208] [FATAL] [MAIN ] PARR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:29:24.256271] [WARN ] [MAIN ] PARR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:29:24.256311] [INFO ] [LOCAL ] PARR: Starting deletion of primary data. [2025-10-20T20:29:24.256384] [DEBUG] [LOCAL ] PARR: No primary data file found to delete. [2025-10-20T20:29:24.256439] [INFO ] [LOCAL ] PARR: Starting deletion of derivative data. [2025-10-20T20:29:24.256605] [DEBUG] [LOCAL ] PARR: Deleted derivative data file. [2025-10-20T20:29:24.257072] [INFO ] [MAIN ] Update Progress: 1627/2367 symbols completed. PARR took 29.5s. Estimated remaining: 1h 35m 13s. Rate: 7.8 symbols/min [2025-10-20T20:29:24.352211] [DEBUG] [LOCAL ] PAY: Loaded 39122 existing records from primary storage. [2025-10-20T20:29:24.373268] [INFO ] [MAIN ] PAY: Loaded 39122 existing records. [2025-10-20T20:29:24.548807] [INFO ] [MAIN ] PAYC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:24.623159] [WARN ] [MAIN ] PAY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:24.623470] [INFO ] [MAIN ] PAY: Wiping existing primary data before full refresh. [2025-10-20T20:29:24.623633] [INFO ] [LOCAL ] PAY: Starting deletion of primary data. [2025-10-20T20:29:24.624252] [DEBUG] [LOCAL ] PAY: Deleted primary data file. [2025-10-20T20:29:24.624300] [INFO ] [MAIN ] PAY: Starting full primary refresh. [2025-10-20T20:29:24.897310] [DEBUG] [LOCAL ] PAYC: Loaded 36844 existing records from primary storage. [2025-10-20T20:29:24.919603] [INFO ] [MAIN ] PAYC: Loaded 36844 existing records. [2025-10-20T20:29:25.069720] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-08. [2025-10-20T20:29:25.117369] [INFO ] [REMOTE ] PAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:25.194680] [WARN ] [MAIN ] PAYC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:25.194905] [INFO ] [MAIN ] PAYC: Wiping existing primary data before full refresh. [2025-10-20T20:29:25.194960] [INFO ] [LOCAL ] PAYC: Starting deletion of primary data. [2025-10-20T20:29:25.195363] [DEBUG] [LOCAL ] PAYC: Deleted primary data file. [2025-10-20T20:29:25.195406] [INFO ] [MAIN ] PAYC: Starting full primary refresh. [2025-10-20T20:29:25.578334] [INFO ] [REMOTE ] PAYC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:25.580835] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T20:29:25.851365] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-01. [2025-10-20T20:29:26.651384] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-05. [2025-10-20T20:29:27.359741] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-08 (2.29 seconds). [2025-10-20T20:29:27.414584] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T20:29:27.455500] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-06. [2025-10-20T20:29:27.457427] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T20:29:28.251432] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-01. [2025-10-20T20:29:28.621810] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T20:29:29.051404] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-09. [2025-10-20T20:29:29.696277] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:29:29.851448] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-02. [2025-10-20T20:29:30.651437] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-02. [2025-10-20T20:29:31.262837] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-09 (2.21 seconds). [2025-10-20T20:29:31.431054] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:29:31.451337] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-03. [2025-10-20T20:29:31.963695] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T20:29:32.251412] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-07. [2025-10-20T20:29:33.035520] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:29:33.051317] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-04. [2025-10-20T20:29:33.383578] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:29:33.851430] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-08. [2025-10-20T20:29:34.476442] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T20:29:34.651364] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-05. [2025-10-20T20:29:34.825355] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T20:29:35.451380] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-09. [2025-10-20T20:29:36.230189] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T20:29:36.251387] [INFO ] [REMOTE ] PATH: Fetching primary data for 2025-10. [2025-10-20T20:29:36.283491] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T20:29:36.975016] [INFO ] [REMOTE ] PATH: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T20:29:36.982730] [INFO ] [REMOTE ] PATH: Full fetch complete (77461 records). [2025-10-20T20:29:36.982925] [DEBUG] [VALIDATOR ] PATH: Validating primary data. [2025-10-20T20:29:36.983698] [FATAL] [VALIDATOR ] PATH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:29:36.983800] [FATAL] [MAIN ] PATH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:29:36.983843] [WARN ] [MAIN ] PATH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:29:36.983880] [INFO ] [LOCAL ] PATH: Starting deletion of primary data. [2025-10-20T20:29:36.983936] [DEBUG] [LOCAL ] PATH: No primary data file found to delete. [2025-10-20T20:29:36.983970] [INFO ] [LOCAL ] PATH: Starting deletion of derivative data. [2025-10-20T20:29:36.984118] [DEBUG] [LOCAL ] PATH: Deleted derivative data file. [2025-10-20T20:29:36.984209] [INFO ] [MAIN ] Update Progress: 1628/2367 symbols completed. PATH took 39.3s. Estimated remaining: 1h 35m 7s. Rate: 7.8 symbols/min [2025-10-20T20:29:37.056545] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-06. [2025-10-20T20:29:37.170058] [INFO ] [MAIN ] PB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:37.212361] [DEBUG] [LOCAL ] PB: Loaded 42728 existing records from primary storage. [2025-10-20T20:29:37.212705] [INFO ] [MAIN ] PB: Loaded 42728 existing records. [2025-10-20T20:29:37.394293] [WARN ] [MAIN ] PB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:37.394542] [INFO ] [MAIN ] PB: Wiping existing primary data before full refresh. [2025-10-20T20:29:37.394590] [INFO ] [LOCAL ] PB: Starting deletion of primary data. [2025-10-20T20:29:37.395085] [DEBUG] [LOCAL ] PB: Deleted primary data file. [2025-10-20T20:29:37.395141] [INFO ] [MAIN ] PB: Starting full primary refresh. [2025-10-20T20:29:37.764303] [INFO ] [REMOTE ] PB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:37.851371] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-01. [2025-10-20T20:29:38.651367] [INFO ] [REMOTE ] PAXS: Fetching primary data for 2025-10. [2025-10-20T20:29:38.717062] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T20:29:39.451410] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-03. [2025-10-20T20:29:39.495706] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T20:29:39.665458] [INFO ] [REMOTE ] PAXS: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T20:29:39.668584] [INFO ] [REMOTE ] PAXS: Full fetch complete (29633 records). [2025-10-20T20:29:39.668683] [DEBUG] [VALIDATOR ] PAXS: Validating primary data. [2025-10-20T20:29:40.030599] [DEBUG] [VALIDATOR ] PAXS: All expected trading days are present. [2025-10-20T20:29:40.030698] [DEBUG] [VALIDATOR ] PAXS: Primary data validation passed. [2025-10-20T20:29:40.031301] [INFO ] [MAIN ] PAXS: Fetched primary data is valid. [2025-10-20T20:29:40.031343] [DEBUG] [LOCAL ] PAXS: Saving primary data. [2025-10-20T20:29:40.032006] [DEBUG] [LOCAL ] PAXS: Trimmed 29633 records to 21022 (keeping 200 days of data) [2025-10-20T20:29:40.159126] [DEBUG] [LOCAL ] PAXS: Primary data saved (21022 records). [2025-10-20T20:29:40.159259] [INFO ] [MAIN ] PAXS: Full history saved. [2025-10-20T20:29:40.159349] [INFO ] [MAIN ] Update Progress: 1629/2367 symbols completed. PAXS took 22.8s. Estimated remaining: 1h 34m 57s. Rate: 7.8 symbols/min [2025-10-20T20:29:40.257144] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-02. [2025-10-20T20:29:40.330166] [INFO ] [MAIN ] PBA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:40.373625] [DEBUG] [LOCAL ] PBA: Loaded 49917 existing records from primary storage. [2025-10-20T20:29:40.373967] [INFO ] [MAIN ] PBA: Loaded 49917 existing records. [2025-10-20T20:29:40.556274] [WARN ] [MAIN ] PBA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:40.556464] [INFO ] [MAIN ] PBA: Wiping existing primary data before full refresh. [2025-10-20T20:29:40.556513] [INFO ] [LOCAL ] PBA: Starting deletion of primary data. [2025-10-20T20:29:40.556734] [DEBUG] [LOCAL ] PBA: Deleted primary data file. [2025-10-20T20:29:40.556794] [INFO ] [MAIN ] PBA: Starting full primary refresh. [2025-10-20T20:29:40.916009] [INFO ] [REMOTE ] PBA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:40.937545] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T20:29:41.051350] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-04. [2025-10-20T20:29:41.749917] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T20:29:41.851434] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-03. [2025-10-20T20:29:42.063465] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T20:29:42.651381] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-05. [2025-10-20T20:29:43.389696] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T20:29:43.451342] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-07. [2025-10-20T20:29:43.851208] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T20:29:44.251375] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-06. [2025-10-20T20:29:44.559867] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T20:29:45.051376] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-01. [2025-10-20T20:29:45.851428] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-04. [2025-10-20T20:29:45.884951] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T20:29:46.651372] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-08. [2025-10-20T20:29:46.755331] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:29:47.451437] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-02. [2025-10-20T20:29:47.504397] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T20:29:47.860204] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T20:29:48.251394] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-05. [2025-10-20T20:29:49.051416] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-09. [2025-10-20T20:29:49.180317] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:29:49.809467] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T20:29:49.851371] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-06. [2025-10-20T20:29:50.623967] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T20:29:50.651393] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-03. [2025-10-20T20:29:51.321081] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T20:29:51.451394] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-07. [2025-10-20T20:29:52.251369] [INFO ] [REMOTE ] PAYC: Fetching primary data for 2025-10. [2025-10-20T20:29:52.375685] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T20:29:52.582084] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:29:52.935432] [INFO ] [REMOTE ] PAYC: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:29:52.941040] [INFO ] [REMOTE ] PAYC: Full fetch complete (52997 records). [2025-10-20T20:29:52.941152] [DEBUG] [VALIDATOR ] PAYC: Validating primary data. [2025-10-20T20:29:52.941754] [FATAL] [VALIDATOR ] PAYC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:29:52.941849] [FATAL] [MAIN ] PAYC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:29:52.941922] [WARN ] [MAIN ] PAYC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:29:52.942011] [INFO ] [LOCAL ] PAYC: Starting deletion of primary data. [2025-10-20T20:29:52.942086] [DEBUG] [LOCAL ] PAYC: No primary data file found to delete. [2025-10-20T20:29:52.942123] [INFO ] [LOCAL ] PAYC: Starting deletion of derivative data. [2025-10-20T20:29:52.942236] [DEBUG] [LOCAL ] PAYC: Deleted derivative data file. [2025-10-20T20:29:52.942304] [INFO ] [MAIN ] Update Progress: 1630/2367 symbols completed. PAYC took 28.7s. Estimated remaining: 1h 34m 52s. Rate: 7.8 symbols/min [2025-10-20T20:29:53.056484] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-07. [2025-10-20T20:29:53.116485] [INFO ] [MAIN ] PBF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:53.182088] [DEBUG] [LOCAL ] PBF: Loaded 53909 existing records from primary storage. [2025-10-20T20:29:53.182540] [INFO ] [MAIN ] PBF: Loaded 53909 existing records. [2025-10-20T20:29:53.382536] [WARN ] [MAIN ] PBF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:53.382831] [INFO ] [MAIN ] PBF: Wiping existing primary data before full refresh. [2025-10-20T20:29:53.382888] [INFO ] [LOCAL ] PBF: Starting deletion of primary data. [2025-10-20T20:29:53.383126] [DEBUG] [LOCAL ] PBF: Deleted primary data file. [2025-10-20T20:29:53.383183] [INFO ] [MAIN ] PBF: Starting full primary refresh. [2025-10-20T20:29:53.738165] [INFO ] [REMOTE ] PBF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:29:53.851410] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-01. [2025-10-20T20:29:54.651378] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-04. [2025-10-20T20:29:54.726225] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T20:29:55.451384] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-08. [2025-10-20T20:29:55.524457] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T20:29:56.251413] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-08. [2025-10-20T20:29:56.397832] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T20:29:57.036619] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T20:29:57.051325] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-09. [2025-10-20T20:29:57.574012] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T20:29:57.851368] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-02. [2025-10-20T20:29:58.641602] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T20:29:58.651355] [INFO ] [REMOTE ] PB: Fetching primary data for 2025-10. [2025-10-20T20:29:59.041188] [INFO ] [REMOTE ] PB: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-10-20T20:29:59.049401] [INFO ] [REMOTE ] PB: Full fetch complete (59398 records). [2025-10-20T20:29:59.049544] [DEBUG] [VALIDATOR ] PB: Validating primary data. [2025-10-20T20:29:59.050372] [FATAL] [VALIDATOR ] PB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:29:59.050483] [FATAL] [MAIN ] PB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:29:59.050534] [WARN ] [MAIN ] PB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:29:59.050595] [INFO ] [LOCAL ] PB: Starting deletion of primary data. [2025-10-20T20:29:59.050680] [DEBUG] [LOCAL ] PB: No primary data file found to delete. [2025-10-20T20:29:59.050756] [INFO ] [LOCAL ] PB: Starting deletion of derivative data. [2025-10-20T20:29:59.050934] [DEBUG] [LOCAL ] PB: Deleted derivative data file. [2025-10-20T20:29:59.051043] [INFO ] [MAIN ] Update Progress: 1631/2367 symbols completed. PB took 22.1s. Estimated remaining: 1h 34m 43s. Rate: 7.8 symbols/min [2025-10-20T20:29:59.221462] [INFO ] [MAIN ] PBH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:29:59.252087] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T20:29:59.287801] [DEBUG] [LOCAL ] PBH: Loaded 33130 existing records from primary storage. [2025-10-20T20:29:59.288107] [INFO ] [MAIN ] PBH: Loaded 33130 existing records. [2025-10-20T20:29:59.459098] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-09. [2025-10-20T20:29:59.481214] [WARN ] [MAIN ] PBH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:29:59.482903] [INFO ] [MAIN ] PBH: Wiping existing primary data before full refresh. [2025-10-20T20:29:59.482966] [INFO ] [LOCAL ] PBH: Starting deletion of primary data. [2025-10-20T20:29:59.483160] [DEBUG] [LOCAL ] PBH: Deleted primary data file. [2025-10-20T20:29:59.483200] [INFO ] [MAIN ] PBH: Starting full primary refresh. [2025-10-20T20:29:59.840553] [INFO ] [REMOTE ] PBH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:00.251446] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-03. [2025-10-20T20:30:00.817293] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T20:30:01.051424] [INFO ] [REMOTE ] PAY: Fetching primary data for 2025-10. [2025-10-20T20:30:01.851357] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-01. [2025-10-20T20:30:02.246302] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T20:30:02.536917] [INFO ] [REMOTE ] PAY: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T20:30:02.546950] [INFO ] [REMOTE ] PAY: Full fetch complete (54273 records). [2025-10-20T20:30:02.547154] [DEBUG] [VALIDATOR ] PAY: Validating primary data. [2025-10-20T20:30:02.661695] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-05. [2025-10-20T20:30:03.094527] [DEBUG] [VALIDATOR ] PAY: All expected trading days are present. [2025-10-20T20:30:03.094627] [DEBUG] [VALIDATOR ] PAY: Primary data validation passed. [2025-10-20T20:30:03.095855] [INFO ] [MAIN ] PAY: Fetched primary data is valid. [2025-10-20T20:30:03.095927] [DEBUG] [LOCAL ] PAY: Saving primary data. [2025-10-20T20:30:03.097268] [DEBUG] [LOCAL ] PAY: Trimmed 54273 records to 38632 (keeping 200 days of data) [2025-10-20T20:30:03.242088] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T20:30:03.377048] [DEBUG] [LOCAL ] PAY: Primary data saved (38632 records). [2025-10-20T20:30:03.377211] [INFO ] [MAIN ] PAY: Full history saved. [2025-10-20T20:30:03.377300] [INFO ] [MAIN ] Update Progress: 1632/2367 symbols completed. PAY took 39.4s. Estimated remaining: 1h 34m 34s. Rate: 7.8 symbols/min [2025-10-20T20:30:03.456560] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-04. [2025-10-20T20:30:03.577050] [INFO ] [MAIN ] PBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:30:03.634126] [DEBUG] [LOCAL ] PBI: Loaded 50739 existing records from primary storage. [2025-10-20T20:30:03.634501] [INFO ] [MAIN ] PBI: Loaded 50739 existing records. [2025-10-20T20:30:03.819990] [WARN ] [MAIN ] PBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:30:03.820215] [INFO ] [MAIN ] PBI: Wiping existing primary data before full refresh. [2025-10-20T20:30:03.820264] [INFO ] [LOCAL ] PBI: Starting deletion of primary data. [2025-10-20T20:30:03.820447] [DEBUG] [LOCAL ] PBI: Deleted primary data file. [2025-10-20T20:30:03.820485] [INFO ] [MAIN ] PBI: Starting full primary refresh. [2025-10-20T20:30:04.172774] [INFO ] [REMOTE ] PBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:04.251390] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-01. [2025-10-20T20:30:04.559591] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T20:30:05.051460] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-06. [2025-10-20T20:30:05.469069] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T20:30:05.851395] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-05. [2025-10-20T20:30:06.069603] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T20:30:06.651394] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-02. [2025-10-20T20:30:06.904522] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T20:30:07.451376] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-07. [2025-10-20T20:30:07.630835] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T20:30:08.251435] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-06. [2025-10-20T20:30:08.484429] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T20:30:08.865870] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T20:30:09.051374] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-02. [2025-10-20T20:30:09.851363] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-08. [2025-10-20T20:30:10.183192] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T20:30:10.249354] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T20:30:10.651450] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-03. [2025-10-20T20:30:11.451470] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-07. [2025-10-20T20:30:11.689837] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T20:30:11.890002] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T20:30:12.251391] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-09. [2025-10-20T20:30:13.051393] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-04. [2025-10-20T20:30:13.493898] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T20:30:13.858740] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-03. [2025-10-20T20:30:13.880257] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T20:30:14.193935] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T20:30:14.651404] [INFO ] [REMOTE ] PBA: Fetching primary data for 2025-10. [2025-10-20T20:30:15.451445] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-05. [2025-10-20T20:30:15.701831] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T20:30:16.251426] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-08. [2025-10-20T20:30:16.341802] [INFO ] [REMOTE ] PBA: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T20:30:16.350012] [INFO ] [REMOTE ] PBA: Full fetch complete (71676 records). [2025-10-20T20:30:16.350188] [DEBUG] [VALIDATOR ] PBA: Validating primary data. [2025-10-20T20:30:16.766669] [DEBUG] [VALIDATOR ] PBA: All expected trading days are present. [2025-10-20T20:30:16.766838] [DEBUG] [VALIDATOR ] PBA: Primary data validation passed. [2025-10-20T20:30:16.768109] [INFO ] [MAIN ] PBA: Fetched primary data is valid. [2025-10-20T20:30:16.768204] [DEBUG] [LOCAL ] PBA: Saving primary data. [2025-10-20T20:30:16.770249] [DEBUG] [LOCAL ] PBA: Trimmed 71676 records to 49232 (keeping 200 days of data) [2025-10-20T20:30:16.888915] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T20:30:17.068922] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-06. [2025-10-20T20:30:17.104720] [DEBUG] [LOCAL ] PBA: Primary data saved (49232 records). [2025-10-20T20:30:17.104865] [INFO ] [MAIN ] PBA: Full history saved. [2025-10-20T20:30:17.104945] [INFO ] [MAIN ] Update Progress: 1633/2367 symbols completed. PBA took 36.9s. Estimated remaining: 1h 34m 29s. Rate: 7.8 symbols/min [2025-10-20T20:30:17.272154] [INFO ] [MAIN ] PBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:30:17.322959] [DEBUG] [LOCAL ] PBR: Loaded 54493 existing records from primary storage. [2025-10-20T20:30:17.323281] [INFO ] [MAIN ] PBR: Loaded 54493 existing records. [2025-10-20T20:30:17.495935] [WARN ] [MAIN ] PBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:30:17.496139] [INFO ] [MAIN ] PBR: Wiping existing primary data before full refresh. [2025-10-20T20:30:17.496185] [INFO ] [LOCAL ] PBR: Starting deletion of primary data. [2025-10-20T20:30:17.496567] [DEBUG] [LOCAL ] PBR: Deleted primary data file. [2025-10-20T20:30:17.496624] [INFO ] [MAIN ] PBR: Starting full primary refresh. [2025-10-20T20:30:17.849273] [INFO ] [REMOTE ] PBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:17.851325] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-01. [2025-10-20T20:30:18.124941] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T20:30:18.177144] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T20:30:18.651393] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-09. [2025-10-20T20:30:19.451409] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-07. [2025-10-20T20:30:19.499559] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T20:30:20.251489] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-02. [2025-10-20T20:30:20.640416] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T20:30:20.855606] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:30:21.051372] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-08. [2025-10-20T20:30:21.851369] [INFO ] [REMOTE ] PBF: Fetching primary data for 2025-10. [2025-10-20T20:30:22.296962] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T20:30:22.429275] [INFO ] [REMOTE ] PBF: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T20:30:22.436691] [INFO ] [REMOTE ] PBF: Full fetch complete (76362 records). [2025-10-20T20:30:22.436859] [DEBUG] [VALIDATOR ] PBF: Validating primary data. [2025-10-20T20:30:22.437580] [FATAL] [VALIDATOR ] PBF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:30:22.437703] [FATAL] [MAIN ] PBF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:30:22.437771] [WARN ] [MAIN ] PBF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:30:22.437862] [INFO ] [LOCAL ] PBF: Starting deletion of primary data. [2025-10-20T20:30:22.437973] [DEBUG] [LOCAL ] PBF: No primary data file found to delete. [2025-10-20T20:30:22.438045] [INFO ] [LOCAL ] PBF: Starting deletion of derivative data. [2025-10-20T20:30:22.438256] [DEBUG] [LOCAL ] PBF: Deleted derivative data file. [2025-10-20T20:30:22.438592] [INFO ] [MAIN ] Update Progress: 1634/2367 symbols completed. PBF took 29.5s. Estimated remaining: 1h 34m 20s. Rate: 7.8 symbols/min [2025-10-20T20:30:22.624722] [INFO ] [MAIN ] PBT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:30:22.649128] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:30:22.657005] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-03. [2025-10-20T20:30:22.673235] [DEBUG] [LOCAL ] PBT: Loaded 17711 existing records from primary storage. [2025-10-20T20:30:22.673463] [INFO ] [MAIN ] PBT: Loaded 17711 existing records. [2025-10-20T20:30:22.848497] [WARN ] [MAIN ] PBT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:30:22.848772] [INFO ] [MAIN ] PBT: Wiping existing primary data before full refresh. [2025-10-20T20:30:22.848822] [INFO ] [LOCAL ] PBT: Starting deletion of primary data. [2025-10-20T20:30:22.848988] [DEBUG] [LOCAL ] PBT: Deleted primary data file. [2025-10-20T20:30:22.849025] [INFO ] [MAIN ] PBT: Starting full primary refresh. [2025-10-20T20:30:23.215888] [INFO ] [REMOTE ] PBT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:23.451402] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-01. [2025-10-20T20:30:24.251385] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-09. [2025-10-20T20:30:24.610872] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T20:30:24.852404] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-10-20T20:30:25.051403] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-04. [2025-10-20T20:30:25.762027] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T20:30:25.851364] [INFO ] [REMOTE ] PBH: Fetching primary data for 2025-10. [2025-10-20T20:30:26.197964] [INFO ] [REMOTE ] PBH: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T20:30:26.203279] [INFO ] [REMOTE ] PBH: Full fetch complete (44467 records). [2025-10-20T20:30:26.203379] [DEBUG] [VALIDATOR ] PBH: Validating primary data. [2025-10-20T20:30:26.204023] [FATAL] [VALIDATOR ] PBH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:30:26.204111] [FATAL] [MAIN ] PBH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:30:26.204155] [WARN ] [MAIN ] PBH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:30:26.204219] [INFO ] [LOCAL ] PBH: Starting deletion of primary data. [2025-10-20T20:30:26.204297] [DEBUG] [LOCAL ] PBH: No primary data file found to delete. [2025-10-20T20:30:26.204352] [INFO ] [LOCAL ] PBH: Starting deletion of derivative data. [2025-10-20T20:30:26.204519] [DEBUG] [LOCAL ] PBH: Deleted derivative data file. [2025-10-20T20:30:26.204608] [INFO ] [MAIN ] Update Progress: 1635/2367 symbols completed. PBH took 27.2s. Estimated remaining: 1h 34m 11s. Rate: 7.8 symbols/min [2025-10-20T20:30:26.384950] [INFO ] [MAIN ] PCF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:30:26.395123] [DEBUG] [LOCAL ] PCF: Loaded 9163 existing records from primary storage. [2025-10-20T20:30:26.395336] [INFO ] [MAIN ] PCF: Loaded 9163 existing records. [2025-10-20T20:30:26.567998] [WARN ] [MAIN ] PCF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:30:26.568225] [INFO ] [MAIN ] PCF: Wiping existing primary data before full refresh. [2025-10-20T20:30:26.568273] [INFO ] [LOCAL ] PCF: Starting deletion of primary data. [2025-10-20T20:30:26.568489] [DEBUG] [LOCAL ] PCF: Deleted primary data file. [2025-10-20T20:30:26.568558] [INFO ] [MAIN ] PCF: Starting full primary refresh. [2025-10-20T20:30:26.656639] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-04. [2025-10-20T20:30:26.928937] [INFO ] [REMOTE ] PCF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:27.303221] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-04 (2.25 seconds). [2025-10-20T20:30:27.451376] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-05. [2025-10-20T20:30:28.251422] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-02. [2025-10-20T20:30:29.051375] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-01. [2025-10-20T20:30:29.137681] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T20:30:29.851391] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-05. [2025-10-20T20:30:30.778670] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-05 (3.33 seconds). [2025-10-20T20:30:30.779485] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T20:30:30.792570] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-02. [2025-10-20T20:30:30.969400] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-02 (2.72 seconds). [2025-10-20T20:30:31.451378] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-06. [2025-10-20T20:30:31.710283] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T20:30:31.811688] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T20:30:32.251395] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-06. [2025-10-20T20:30:33.051376] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-03. [2025-10-20T20:30:33.851417] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-03. [2025-10-20T20:30:33.875972] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:30:33.954688] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-06 (2.50 seconds). [2025-10-20T20:30:34.099424] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T20:30:34.557168] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T20:30:34.651347] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-04. [2025-10-20T20:30:35.451375] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-07. [2025-10-20T20:30:35.476872] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:30:36.251416] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-07. [2025-10-20T20:30:37.051442] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-05. [2025-10-20T20:30:37.719131] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-07 (2.27 seconds). [2025-10-20T20:30:37.851392] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-04. [2025-10-20T20:30:37.861238] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T20:30:38.132669] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T20:30:38.651369] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-08. [2025-10-20T20:30:38.888511] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T20:30:39.451376] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-06. [2025-10-20T20:30:40.251374] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-08. [2025-10-20T20:30:40.260798] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T20:30:40.350555] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:30:41.051377] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-07. [2025-10-20T20:30:41.851378] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-05. [2025-10-20T20:30:41.935583] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T20:30:42.504115] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T20:30:42.651382] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-09. [2025-10-20T20:30:42.974738] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T20:30:43.451389] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-08. [2025-10-20T20:30:44.251385] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-09. [2025-10-20T20:30:44.261231] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T20:30:44.700321] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T20:30:45.051402] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-06. [2025-10-20T20:30:45.851362] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-09. [2025-10-20T20:30:46.142026] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T20:30:46.234547] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T20:30:46.483866] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T20:30:46.651399] [INFO ] [REMOTE ] PCF: Fetching primary data for 2025-10. [2025-10-20T20:30:47.391249] [INFO ] [REMOTE ] PCF: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T20:30:47.393057] [INFO ] [REMOTE ] PCF: Full fetch complete (13508 records). [2025-10-20T20:30:47.393123] [DEBUG] [VALIDATOR ] PCF: Validating primary data. [2025-10-20T20:30:47.456712] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-07. [2025-10-20T20:30:47.749068] [DEBUG] [VALIDATOR ] PCF: All expected trading days are present. [2025-10-20T20:30:47.749147] [DEBUG] [VALIDATOR ] PCF: Primary data validation passed. [2025-10-20T20:30:47.749483] [INFO ] [MAIN ] PCF: Fetched primary data is valid. [2025-10-20T20:30:47.749522] [DEBUG] [LOCAL ] PCF: Saving primary data. [2025-10-20T20:30:47.749994] [DEBUG] [LOCAL ] PCF: Trimmed 13508 records to 9084 (keeping 200 days of data) [2025-10-20T20:30:47.801398] [DEBUG] [LOCAL ] PCF: Primary data saved (9084 records). [2025-10-20T20:30:47.801529] [INFO ] [MAIN ] PCF: Full history saved. [2025-10-20T20:30:47.801608] [INFO ] [MAIN ] Update Progress: 1636/2367 symbols completed. PCF took 21.6s. Estimated remaining: 1h 34m 9s. Rate: 7.8 symbols/min [2025-10-20T20:30:47.969056] [INFO ] [MAIN ] PCG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:30:48.020006] [DEBUG] [LOCAL ] PCG: Loaded 54499 existing records from primary storage. [2025-10-20T20:30:48.020323] [INFO ] [MAIN ] PCG: Loaded 54499 existing records. [2025-10-20T20:30:48.195468] [WARN ] [MAIN ] PCG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:30:48.195733] [INFO ] [MAIN ] PCG: Wiping existing primary data before full refresh. [2025-10-20T20:30:48.195788] [INFO ] [LOCAL ] PCG: Starting deletion of primary data. [2025-10-20T20:30:48.196791] [DEBUG] [LOCAL ] PCG: Deleted primary data file. [2025-10-20T20:30:48.196849] [INFO ] [MAIN ] PCG: Starting full primary refresh. [2025-10-20T20:30:48.261512] [INFO ] [REMOTE ] PBI: Fetching primary data for 2025-10. [2025-10-20T20:30:48.595183] [INFO ] [REMOTE ] PCG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:30:48.597583] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T20:30:49.055172] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-01. [2025-10-20T20:30:49.059767] [INFO ] [REMOTE ] PBI: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:30:49.067188] [INFO ] [REMOTE ] PBI: Full fetch complete (72061 records). [2025-10-20T20:30:49.067329] [DEBUG] [VALIDATOR ] PBI: Validating primary data. [2025-10-20T20:30:49.068211] [FATAL] [VALIDATOR ] PBI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:30:49.068299] [FATAL] [MAIN ] PBI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:30:49.068343] [WARN ] [MAIN ] PBI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:30:49.068408] [INFO ] [LOCAL ] PBI: Starting deletion of primary data. [2025-10-20T20:30:49.068473] [DEBUG] [LOCAL ] PBI: No primary data file found to delete. [2025-10-20T20:30:49.068508] [INFO ] [LOCAL ] PBI: Starting deletion of derivative data. [2025-10-20T20:30:49.068669] [DEBUG] [LOCAL ] PBI: Deleted derivative data file. [2025-10-20T20:30:49.068761] [INFO ] [MAIN ] Update Progress: 1637/2367 symbols completed. PBI took 45.7s. Estimated remaining: 1h 33m 59s. Rate: 7.8 symbols/min [2025-10-20T20:30:49.851449] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-08. [2025-10-20T20:30:50.654117] [INFO ] [REMOTE ] PBR: Fetching primary data for 2025-10. [2025-10-20T20:30:50.659395] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:30:50.972815] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:30:51.451510] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-09. [2025-10-20T20:30:52.251449] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-02. [2025-10-20T20:30:52.593153] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T20:30:52.912539] [INFO ] [REMOTE ] PBR: Incremental fetch complete for 2025-10 (2.26 seconds). [2025-10-20T20:30:52.920300] [INFO ] [REMOTE ] PBR: Full fetch complete (77882 records). [2025-10-20T20:30:52.920409] [DEBUG] [VALIDATOR ] PBR: Validating primary data. [2025-10-20T20:30:53.059566] [INFO ] [REMOTE ] PBT: Fetching primary data for 2025-10. [2025-10-20T20:30:53.334636] [DEBUG] [VALIDATOR ] PBR: All expected trading days are present. [2025-10-20T20:30:53.334748] [DEBUG] [VALIDATOR ] PBR: Primary data validation passed. [2025-10-20T20:30:53.335795] [INFO ] [MAIN ] PBR: Fetched primary data is valid. [2025-10-20T20:30:53.335856] [DEBUG] [LOCAL ] PBR: Saving primary data. [2025-10-20T20:30:53.337297] [DEBUG] [LOCAL ] PBR: Trimmed 77882 records to 53713 (keeping 200 days of data) [2025-10-20T20:30:53.657555] [DEBUG] [LOCAL ] PBR: Primary data saved (53713 records). [2025-10-20T20:30:53.657728] [INFO ] [MAIN ] PBR: Full history saved. [2025-10-20T20:30:53.657817] [INFO ] [MAIN ] Update Progress: 1638/2367 symbols completed. PBR took 36.6s. Estimated remaining: 1h 33m 50s. Rate: 7.8 symbols/min [2025-10-20T20:30:54.063728] [INFO ] [REMOTE ] PBT: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T20:30:54.066512] [INFO ] [REMOTE ] PBT: Full fetch complete (24681 records). [2025-10-20T20:30:54.066582] [DEBUG] [VALIDATOR ] PBT: Validating primary data. [2025-10-20T20:30:54.442697] [DEBUG] [VALIDATOR ] PBT: All expected trading days are present. [2025-10-20T20:30:54.442813] [DEBUG] [VALIDATOR ] PBT: Primary data validation passed. [2025-10-20T20:30:54.443387] [INFO ] [MAIN ] PBT: Fetched primary data is valid. [2025-10-20T20:30:54.443501] [DEBUG] [LOCAL ] PBT: Saving primary data. [2025-10-20T20:30:54.444361] [DEBUG] [LOCAL ] PBT: Trimmed 24681 records to 17542 (keeping 200 days of data) [2025-10-20T20:30:54.569924] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-02 (2.32 seconds). [2025-10-20T20:30:54.588509] [DEBUG] [LOCAL ] PBT: Primary data saved (17542 records). [2025-10-20T20:30:54.589249] [INFO ] [MAIN ] PBT: Full history saved. [2025-10-20T20:30:54.589726] [INFO ] [MAIN ] Update Progress: 1639/2367 symbols completed. PBT took 32.2s. Estimated remaining: 1h 33m 39s. Rate: 7.8 symbols/min [2025-10-20T20:30:54.589595] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-03. [2025-10-20T20:30:56.562484] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T20:30:56.573431] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-04. [2025-10-20T20:30:58.736610] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-04 (2.16 seconds). [2025-10-20T20:30:58.747798] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-05. [2025-10-20T20:31:00.625204] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T20:31:00.636233] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-06. [2025-10-20T20:31:02.719721] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-10-20T20:31:02.730971] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-07. [2025-10-20T20:31:04.864803] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-07 (2.13 seconds). [2025-10-20T20:31:04.876428] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-08. [2025-10-20T20:31:06.942092] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T20:31:06.957969] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-09. [2025-10-20T20:31:07.865882] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T20:31:07.877692] [INFO ] [REMOTE ] PCG: Fetching primary data for 2025-10. [2025-10-20T20:31:08.664964] [INFO ] [REMOTE ] PCG: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T20:31:08.672326] [INFO ] [REMOTE ] PCG: Full fetch complete (77508 records). [2025-10-20T20:31:08.672475] [DEBUG] [VALIDATOR ] PCG: Validating primary data. [2025-10-20T20:31:08.673161] [FATAL] [VALIDATOR ] PCG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:31:08.673249] [FATAL] [MAIN ] PCG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:31:08.673292] [WARN ] [MAIN ] PCG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:31:08.673328] [INFO ] [LOCAL ] PCG: Starting deletion of primary data. [2025-10-20T20:31:08.673384] [DEBUG] [LOCAL ] PCG: No primary data file found to delete. [2025-10-20T20:31:08.673419] [INFO ] [LOCAL ] PCG: Starting deletion of derivative data. [2025-10-20T20:31:08.673573] [DEBUG] [LOCAL ] PCG: Deleted derivative data file. [2025-10-20T20:31:08.673689] [INFO ] [MAIN ] Update Progress: 1640/2367 symbols completed. PCG took 20.9s. Estimated remaining: 1h 33m 34s. Rate: 7.8 symbols/min [2025-10-20T20:31:09.389871] [INFO ] [MAIN ] PCQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:09.416789] [INFO ] [MAIN ] PCM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:09.486901] [INFO ] [MAIN ] PCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:09.489856] [INFO ] [MAIN ] PCOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:09.500429] [DEBUG] [LOCAL ] PCM: Loaded 6870 existing records from primary storage. [2025-10-20T20:31:09.500733] [INFO ] [MAIN ] PCM: Loaded 6870 existing records. [2025-10-20T20:31:09.517587] [DEBUG] [LOCAL ] PCQ: Loaded 12357 existing records from primary storage. [2025-10-20T20:31:09.528213] [INFO ] [MAIN ] PCQ: Loaded 12357 existing records. [2025-10-20T20:31:09.887747] [DEBUG] [LOCAL ] PCN: Loaded 25556 existing records from primary storage. [2025-10-20T20:31:09.903772] [INFO ] [MAIN ] PCN: Loaded 25556 existing records. [2025-10-20T20:31:09.907111] [WARN ] [MAIN ] PCQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:09.900874] [WARN ] [MAIN ] PCM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:09.947608] [INFO ] [MAIN ] PCM: Wiping existing primary data before full refresh. [2025-10-20T20:31:09.952921] [INFO ] [LOCAL ] PCM: Starting deletion of primary data. [2025-10-20T20:31:09.947544] [INFO ] [MAIN ] PCQ: Wiping existing primary data before full refresh. [2025-10-20T20:31:09.995615] [DEBUG] [LOCAL ] PCM: Deleted primary data file. [2025-10-20T20:31:10.024287] [INFO ] [MAIN ] PCM: Starting full primary refresh. [2025-10-20T20:31:10.029550] [INFO ] [LOCAL ] PCQ: Starting deletion of primary data. [2025-10-20T20:31:10.057677] [DEBUG] [LOCAL ] PCOR: Loaded 48362 existing records from primary storage. [2025-10-20T20:31:10.073435] [DEBUG] [LOCAL ] PCQ: Deleted primary data file. [2025-10-20T20:31:10.076723] [INFO ] [MAIN ] PCOR: Loaded 48362 existing records. [2025-10-20T20:31:10.101689] [INFO ] [MAIN ] PCQ: Starting full primary refresh. [2025-10-20T20:31:10.287218] [WARN ] [MAIN ] PCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:10.400178] [INFO ] [MAIN ] PCN: Wiping existing primary data before full refresh. [2025-10-20T20:31:10.446721] [INFO ] [LOCAL ] PCN: Starting deletion of primary data. [2025-10-20T20:31:10.576537] [DEBUG] [LOCAL ] PCN: Deleted primary data file. [2025-10-20T20:31:10.631150] [INFO ] [MAIN ] PCN: Starting full primary refresh. [2025-10-20T20:31:10.799316] [WARN ] [MAIN ] PCOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:10.953588] [INFO ] [MAIN ] PCOR: Wiping existing primary data before full refresh. [2025-10-20T20:31:10.980046] [INFO ] [LOCAL ] PCOR: Starting deletion of primary data. [2025-10-20T20:31:11.065352] [DEBUG] [LOCAL ] PCOR: Deleted primary data file. [2025-10-20T20:31:11.166282] [INFO ] [REMOTE ] PCM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:11.149308] [INFO ] [MAIN ] PCOR: Starting full primary refresh. [2025-10-20T20:31:11.235824] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-01. [2025-10-20T20:31:11.294260] [INFO ] [REMOTE ] PCQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:11.309031] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-01. [2025-10-20T20:31:11.616064] [INFO ] [REMOTE ] PCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:11.654418] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-01. [2025-10-20T20:31:11.881632] [INFO ] [REMOTE ] PCOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:11.882512] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-01. [2025-10-20T20:31:12.187722] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T20:31:12.191291] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-02. [2025-10-20T20:31:12.286027] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T20:31:12.290511] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-02. [2025-10-20T20:31:12.850229] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T20:31:12.857297] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T20:31:12.862294] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-02. [2025-10-20T20:31:12.866233] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-03. [2025-10-20T20:31:13.114058] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:31:13.116776] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-03. [2025-10-20T20:31:13.608744] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T20:31:13.617903] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-02. [2025-10-20T20:31:13.653364] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:31:13.655932] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-04. [2025-10-20T20:31:13.670622] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T20:31:13.675402] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-03. [2025-10-20T20:31:13.953429] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T20:31:13.956323] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-04. [2025-10-20T20:31:14.393162] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T20:31:14.395363] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-05. [2025-10-20T20:31:14.733891] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T20:31:14.740860] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-04. [2025-10-20T20:31:14.873600] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T20:31:14.875531] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-06. [2025-10-20T20:31:15.040119] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T20:31:15.044044] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-05. [2025-10-20T20:31:15.196070] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:31:15.206012] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-03. [2025-10-20T20:31:15.390805] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T20:31:15.393064] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-07. [2025-10-20T20:31:15.817752] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T20:31:15.824673] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-05. [2025-10-20T20:31:15.887435] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T20:31:16.071212] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T20:31:16.251401] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-06. [2025-10-20T20:31:17.049551] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T20:31:17.059488] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-08. [2025-10-20T20:31:17.092676] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-05 (1.27 seconds). [2025-10-20T20:31:17.147866] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T20:31:17.778664] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T20:31:17.851372] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-09. [2025-10-20T20:31:18.651413] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-07. [2025-10-20T20:31:18.691731] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T20:31:19.451408] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-04. [2025-10-20T20:31:19.505061] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T20:31:20.251377] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-06. [2025-10-20T20:31:21.051400] [INFO ] [REMOTE ] PCM: Fetching primary data for 2025-10. [2025-10-20T20:31:21.204005] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T20:31:21.419078] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T20:31:21.811250] [INFO ] [REMOTE ] PCM: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:31:21.813176] [INFO ] [REMOTE ] PCM: Full fetch complete (10453 records). [2025-10-20T20:31:21.813475] [DEBUG] [VALIDATOR ] PCM: Validating primary data. [2025-10-20T20:31:21.856564] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-05. [2025-10-20T20:31:22.199033] [DEBUG] [VALIDATOR ] PCM: All expected trading days are present. [2025-10-20T20:31:22.199115] [DEBUG] [VALIDATOR ] PCM: Primary data validation passed. [2025-10-20T20:31:22.199387] [INFO ] [MAIN ] PCM: Fetched primary data is valid. [2025-10-20T20:31:22.199426] [DEBUG] [LOCAL ] PCM: Saving primary data. [2025-10-20T20:31:22.199891] [DEBUG] [LOCAL ] PCM: Trimmed 10453 records to 6851 (keeping 200 days of data) [2025-10-20T20:31:22.237877] [DEBUG] [LOCAL ] PCM: Primary data saved (6851 records). [2025-10-20T20:31:22.237959] [INFO ] [MAIN ] PCM: Full history saved. [2025-10-20T20:31:22.238033] [INFO ] [MAIN ] Update Progress: 1641/2367 symbols completed. PCM took 13.6s. Estimated remaining: 1h 33m 29s. Rate: 7.8 symbols/min [2025-10-20T20:31:22.402896] [INFO ] [MAIN ] PD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:22.447153] [DEBUG] [LOCAL ] PD: Loaded 48923 existing records from primary storage. [2025-10-20T20:31:22.447487] [INFO ] [MAIN ] PD: Loaded 48923 existing records. [2025-10-20T20:31:22.619520] [WARN ] [MAIN ] PD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:22.619785] [INFO ] [MAIN ] PD: Wiping existing primary data before full refresh. [2025-10-20T20:31:22.619833] [INFO ] [LOCAL ] PD: Starting deletion of primary data. [2025-10-20T20:31:22.620051] [DEBUG] [LOCAL ] PD: Deleted primary data file. [2025-10-20T20:31:22.620107] [INFO ] [MAIN ] PD: Starting full primary refresh. [2025-10-20T20:31:22.661140] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-07. [2025-10-20T20:31:22.973095] [INFO ] [REMOTE ] PD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:23.451345] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-01. [2025-10-20T20:31:23.494712] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T20:31:23.991214] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T20:31:24.251433] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-08. [2025-10-20T20:31:25.045113] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T20:31:25.053732] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-08. [2025-10-20T20:31:25.550390] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T20:31:25.851398] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-02. [2025-10-20T20:31:26.088284] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T20:31:26.651431] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-09. [2025-10-20T20:31:27.433152] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:31:27.451341] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-03. [2025-10-20T20:31:27.935942] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T20:31:28.251371] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-09. [2025-10-20T20:31:29.051421] [INFO ] [REMOTE ] PCN: Fetching primary data for 2025-10. [2025-10-20T20:31:29.180909] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:31:29.258535] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T20:31:29.851397] [INFO ] [REMOTE ] PCQ: Fetching primary data for 2025-10. [2025-10-20T20:31:30.050291] [INFO ] [REMOTE ] PCN: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T20:31:30.054966] [INFO ] [REMOTE ] PCN: Full fetch complete (34760 records). [2025-10-20T20:31:30.055068] [DEBUG] [VALIDATOR ] PCN: Validating primary data. [2025-10-20T20:31:30.428988] [DEBUG] [VALIDATOR ] PCN: All expected trading days are present. [2025-10-20T20:31:30.429061] [DEBUG] [VALIDATOR ] PCN: Primary data validation passed. [2025-10-20T20:31:30.429824] [INFO ] [MAIN ] PCN: Fetched primary data is valid. [2025-10-20T20:31:30.429885] [DEBUG] [LOCAL ] PCN: Saving primary data. [2025-10-20T20:31:30.430672] [DEBUG] [LOCAL ] PCN: Trimmed 34760 records to 25417 (keeping 200 days of data) [2025-10-20T20:31:30.584530] [DEBUG] [LOCAL ] PCN: Primary data saved (25417 records). [2025-10-20T20:31:30.584705] [INFO ] [MAIN ] PCN: Full history saved. [2025-10-20T20:31:30.584797] [INFO ] [MAIN ] Update Progress: 1642/2367 symbols completed. PCN took 21.9s. Estimated remaining: 1h 33m 21s. Rate: 7.8 symbols/min [2025-10-20T20:31:30.655735] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-06. [2025-10-20T20:31:30.696992] [INFO ] [REMOTE ] PCQ: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T20:31:30.752387] [INFO ] [REMOTE ] PCQ: Full fetch complete (16879 records). [2025-10-20T20:31:30.753764] [DEBUG] [VALIDATOR ] PCQ: Validating primary data. [2025-10-20T20:31:30.809115] [INFO ] [MAIN ] PDCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:30.848145] [DEBUG] [LOCAL ] PDCC: No existing data file found in primary storage. [2025-10-20T20:31:30.854283] [WARN ] [MAIN ] PDCC: No existing data found, performing full refresh instead. [2025-10-20T20:31:30.871988] [INFO ] [MAIN ] PDCC: Wiping existing primary data before full refresh. [2025-10-20T20:31:30.881298] [INFO ] [LOCAL ] PDCC: Starting deletion of primary data. [2025-10-20T20:31:30.885287] [DEBUG] [LOCAL ] PDCC: No primary data file found to delete. [2025-10-20T20:31:30.885336] [INFO ] [MAIN ] PDCC: Starting full primary refresh. [2025-10-20T20:31:31.457686] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-04. [2025-10-20T20:31:31.510531] [DEBUG] [VALIDATOR ] PCQ: All expected trading days are present. [2025-10-20T20:31:31.524498] [DEBUG] [VALIDATOR ] PCQ: Primary data validation passed. [2025-10-20T20:31:31.554789] [INFO ] [REMOTE ] PDCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:31.555517] [INFO ] [MAIN ] PCQ: Fetched primary data is valid. [2025-10-20T20:31:31.555937] [DEBUG] [LOCAL ] PCQ: Saving primary data. [2025-10-20T20:31:31.556828] [DEBUG] [LOCAL ] PCQ: Trimmed 16879 records to 12261 (keeping 200 days of data) [2025-10-20T20:31:31.630621] [DEBUG] [LOCAL ] PCQ: Primary data saved (12261 records). [2025-10-20T20:31:31.630782] [INFO ] [MAIN ] PCQ: Full history saved. [2025-10-20T20:31:31.630866] [INFO ] [MAIN ] Update Progress: 1643/2367 symbols completed. PCQ took 23.0s. Estimated remaining: 1h 33m 11s. Rate: 7.8 symbols/min [2025-10-20T20:31:31.795485] [INFO ] [MAIN ] PDI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:31.845820] [DEBUG] [LOCAL ] PDI: Loaded 50903 existing records from primary storage. [2025-10-20T20:31:31.846145] [INFO ] [MAIN ] PDI: Loaded 50903 existing records. [2025-10-20T20:31:32.029810] [WARN ] [MAIN ] PDI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:32.030077] [INFO ] [MAIN ] PDI: Wiping existing primary data before full refresh. [2025-10-20T20:31:32.030124] [INFO ] [LOCAL ] PDI: Starting deletion of primary data. [2025-10-20T20:31:32.030370] [DEBUG] [LOCAL ] PDI: Deleted primary data file. [2025-10-20T20:31:32.030427] [INFO ] [MAIN ] PDI: Starting full primary refresh. [2025-10-20T20:31:32.257915] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-01. [2025-10-20T20:31:32.394825] [INFO ] [REMOTE ] PDI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:32.440318] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T20:31:32.597175] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-01 (0.34 seconds). [2025-10-20T20:31:33.051372] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-01. [2025-10-20T20:31:33.219962] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T20:31:33.851374] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-07. [2025-10-20T20:31:34.651351] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-02. [2025-10-20T20:31:35.011636] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T20:31:35.013033] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T20:31:35.451392] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-03. [2025-10-20T20:31:35.577025] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T20:31:35.940074] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T20:31:36.251378] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-08. [2025-10-20T20:31:37.051390] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-02. [2025-10-20T20:31:37.851363] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-04. [2025-10-20T20:31:37.965147] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T20:31:38.344505] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T20:31:38.651417] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-05. [2025-10-20T20:31:38.958553] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-02 (1.91 seconds). [2025-10-20T20:31:39.046308] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-05 (0.39 seconds). [2025-10-20T20:31:39.451394] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-05. [2025-10-20T20:31:40.251373] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-09. [2025-10-20T20:31:41.051390] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-03. [2025-10-20T20:31:41.100821] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T20:31:41.851380] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-06. [2025-10-20T20:31:41.945702] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T20:31:42.270190] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T20:31:42.651404] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-07. [2025-10-20T20:31:42.935004] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-03 (1.88 seconds). [2025-10-20T20:31:43.042331] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T20:31:43.451440] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-08. [2025-10-20T20:31:43.851387] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-08 (0.40 seconds). [2025-10-20T20:31:44.251379] [INFO ] [REMOTE ] PCOR: Fetching primary data for 2025-10. [2025-10-20T20:31:45.051444] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-04. [2025-10-20T20:31:45.841185] [INFO ] [REMOTE ] PCOR: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T20:31:45.848632] [INFO ] [REMOTE ] PCOR: Full fetch complete (69420 records). [2025-10-20T20:31:45.848829] [DEBUG] [VALIDATOR ] PCOR: Validating primary data. [2025-10-20T20:31:45.862086] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-09. [2025-10-20T20:31:46.236980] [DEBUG] [VALIDATOR ] PCOR: All expected trading days are present. [2025-10-20T20:31:46.237070] [DEBUG] [VALIDATOR ] PCOR: Primary data validation passed. [2025-10-20T20:31:46.238099] [INFO ] [MAIN ] PCOR: Fetched primary data is valid. [2025-10-20T20:31:46.238169] [DEBUG] [LOCAL ] PCOR: Saving primary data. [2025-10-20T20:31:46.239548] [DEBUG] [LOCAL ] PCOR: Trimmed 69420 records to 47617 (keeping 200 days of data) [2025-10-20T20:31:46.411083] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T20:31:46.526753] [DEBUG] [LOCAL ] PCOR: Primary data saved (47617 records). [2025-10-20T20:31:46.526894] [INFO ] [MAIN ] PCOR: Full history saved. [2025-10-20T20:31:46.526974] [INFO ] [MAIN ] Update Progress: 1644/2367 symbols completed. PCOR took 37.8s. Estimated remaining: 1h 33m 6s. Rate: 7.8 symbols/min [2025-10-20T20:31:46.651638] [INFO ] [REMOTE ] PDCC: Fetching primary data for 2025-10. [2025-10-20T20:31:46.703373] [INFO ] [MAIN ] PDM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:46.740823] [DEBUG] [LOCAL ] PDM: Loaded 40439 existing records from primary storage. [2025-10-20T20:31:46.741143] [INFO ] [MAIN ] PDM: Loaded 40439 existing records. [2025-10-20T20:31:46.928557] [WARN ] [MAIN ] PDM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:46.928790] [INFO ] [MAIN ] PDM: Wiping existing primary data before full refresh. [2025-10-20T20:31:46.928837] [INFO ] [LOCAL ] PDM: Starting deletion of primary data. [2025-10-20T20:31:46.929049] [DEBUG] [LOCAL ] PDM: Deleted primary data file. [2025-10-20T20:31:46.929107] [INFO ] [MAIN ] PDM: Starting full primary refresh. [2025-10-20T20:31:47.120857] [INFO ] [REMOTE ] PDCC: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T20:31:47.129542] [INFO ] [REMOTE ] PDCC: Full fetch complete (1376 records). [2025-10-20T20:31:47.161259] [DEBUG] [VALIDATOR ] PDCC: Validating primary data. [2025-10-20T20:31:47.459382] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-06. [2025-10-20T20:31:47.506788] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-04 (2.46 seconds). [2025-10-20T20:31:47.499063] [INFO ] [REMOTE ] PDM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:47.582614] [WARN ] [VALIDATOR ] PDCC: Validation failed - insufficient historical data. Have 194 unique days, need 200 days [2025-10-20T20:31:47.582818] [FATAL] [MAIN ] PDCC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:31:47.582886] [WARN ] [MAIN ] PDCC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:31:47.582955] [INFO ] [LOCAL ] PDCC: Starting deletion of primary data. [2025-10-20T20:31:47.583015] [DEBUG] [LOCAL ] PDCC: No primary data file found to delete. [2025-10-20T20:31:47.583049] [INFO ] [LOCAL ] PDCC: Starting deletion of derivative data. [2025-10-20T20:31:47.583087] [DEBUG] [LOCAL ] PDCC: No derivative data file found to delete. [2025-10-20T20:31:47.583140] [INFO ] [MAIN ] Update Progress: 1645/2367 symbols completed. PDCC took 17.0s. Estimated remaining: 1h 32m 55s. Rate: 7.8 symbols/min [2025-10-20T20:31:47.764906] [INFO ] [MAIN ] PDO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:31:47.799419] [DEBUG] [LOCAL ] PDO: Loaded 31991 existing records from primary storage. [2025-10-20T20:31:47.799745] [INFO ] [MAIN ] PDO: Loaded 31991 existing records. [2025-10-20T20:31:48.001730] [WARN ] [MAIN ] PDO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:31:48.001997] [INFO ] [MAIN ] PDO: Wiping existing primary data before full refresh. [2025-10-20T20:31:48.002064] [INFO ] [LOCAL ] PDO: Starting deletion of primary data. [2025-10-20T20:31:48.002280] [DEBUG] [LOCAL ] PDO: Deleted primary data file. [2025-10-20T20:31:48.002335] [INFO ] [MAIN ] PDO: Starting full primary refresh. [2025-10-20T20:31:48.256514] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-05. [2025-10-20T20:31:48.416035] [INFO ] [REMOTE ] PDO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:31:49.056527] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-01. [2025-10-20T20:31:49.070777] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T20:31:49.851408] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-07. [2025-10-20T20:31:50.311075] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T20:31:50.586867] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T20:31:50.651351] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-02. [2025-10-20T20:31:51.394377] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T20:31:51.451339] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-08. [2025-10-20T20:31:52.108714] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T20:31:52.251396] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-01. [2025-10-20T20:31:53.051382] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-06. [2025-10-20T20:31:53.264132] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T20:31:53.810538] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T20:31:53.851346] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-02. [2025-10-20T20:31:54.651368] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-03. [2025-10-20T20:31:54.843523] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T20:31:55.037736] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T20:31:55.451435] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-09. [2025-10-20T20:31:56.227697] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:31:56.251332] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-04. [2025-10-20T20:31:57.051351] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-03. [2025-10-20T20:31:57.161159] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:31:57.596854] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T20:31:57.851429] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-07. [2025-10-20T20:31:58.536391] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T20:31:58.651427] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-05. [2025-10-20T20:31:59.451385] [INFO ] [REMOTE ] PD: Fetching primary data for 2025-10. [2025-10-20T20:31:59.850497] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T20:32:00.251350] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-08. [2025-10-20T20:32:00.298095] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T20:32:01.051380] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-04. [2025-10-20T20:32:01.204498] [INFO ] [REMOTE ] PD: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T20:32:01.211853] [INFO ] [REMOTE ] PD: Full fetch complete (68245 records). [2025-10-20T20:32:01.211968] [DEBUG] [VALIDATOR ] PD: Validating primary data. [2025-10-20T20:32:01.633735] [DEBUG] [VALIDATOR ] PD: All expected trading days are present. [2025-10-20T20:32:01.633828] [DEBUG] [VALIDATOR ] PD: Primary data validation passed. [2025-10-20T20:32:01.634854] [INFO ] [MAIN ] PD: Fetched primary data is valid. [2025-10-20T20:32:01.634916] [DEBUG] [LOCAL ] PD: Saving primary data. [2025-10-20T20:32:01.636256] [DEBUG] [LOCAL ] PD: Trimmed 68245 records to 48336 (keeping 200 days of data) [2025-10-20T20:32:01.851462] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-06. [2025-10-20T20:32:01.929874] [DEBUG] [LOCAL ] PD: Primary data saved (48336 records). [2025-10-20T20:32:01.930053] [INFO ] [MAIN ] PD: Full history saved. [2025-10-20T20:32:01.930171] [INFO ] [MAIN ] Update Progress: 1646/2367 symbols completed. PD took 39.7s. Estimated remaining: 1h 32m 51s. Rate: 7.8 symbols/min [2025-10-20T20:32:02.125196] [INFO ] [MAIN ] PDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:02.143320] [DEBUG] [LOCAL ] PDS: Loaded 17407 existing records from primary storage. [2025-10-20T20:32:02.143705] [INFO ] [MAIN ] PDS: Loaded 17407 existing records. [2025-10-20T20:32:02.322680] [WARN ] [MAIN ] PDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:02.323141] [INFO ] [MAIN ] PDS: Wiping existing primary data before full refresh. [2025-10-20T20:32:02.323244] [INFO ] [LOCAL ] PDS: Starting deletion of primary data. [2025-10-20T20:32:02.338748] [DEBUG] [LOCAL ] PDS: Deleted primary data file. [2025-10-20T20:32:02.349047] [INFO ] [MAIN ] PDS: Starting full primary refresh. [2025-10-20T20:32:02.419415] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T20:32:02.660192] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-09. [2025-10-20T20:32:02.723422] [INFO ] [REMOTE ] PDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:02.757740] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T20:32:03.082557] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T20:32:03.451442] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-05. [2025-10-20T20:32:04.251435] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-07. [2025-10-20T20:32:04.959592] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-09 (2.30 seconds). [2025-10-20T20:32:04.991799] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T20:32:05.051438] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-01. [2025-10-20T20:32:05.851449] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-06. [2025-10-20T20:32:06.241767] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T20:32:06.646418] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-07 (2.39 seconds). [2025-10-20T20:32:06.655591] [INFO ] [REMOTE ] PDI: Fetching primary data for 2025-10. [2025-10-20T20:32:07.070030] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T20:32:07.451455] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-08. [2025-10-20T20:32:08.251410] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-07. [2025-10-20T20:32:08.537035] [INFO ] [REMOTE ] PDI: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T20:32:08.544704] [INFO ] [REMOTE ] PDI: Full fetch complete (73128 records). [2025-10-20T20:32:08.544833] [DEBUG] [VALIDATOR ] PDI: Validating primary data. [2025-10-20T20:32:08.837092] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T20:32:08.961380] [DEBUG] [VALIDATOR ] PDI: All expected trading days are present. [2025-10-20T20:32:08.961454] [DEBUG] [VALIDATOR ] PDI: Primary data validation passed. [2025-10-20T20:32:08.962472] [INFO ] [MAIN ] PDI: Fetched primary data is valid. [2025-10-20T20:32:08.962532] [DEBUG] [LOCAL ] PDI: Saving primary data. [2025-10-20T20:32:08.963861] [DEBUG] [LOCAL ] PDI: Trimmed 73128 records to 50190 (keeping 200 days of data) [2025-10-20T20:32:09.058612] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-02. [2025-10-20T20:32:09.263738] [DEBUG] [LOCAL ] PDI: Primary data saved (50190 records). [2025-10-20T20:32:09.263932] [INFO ] [MAIN ] PDI: Full history saved. [2025-10-20T20:32:09.264043] [INFO ] [MAIN ] Update Progress: 1647/2367 symbols completed. PDI took 37.6s. Estimated remaining: 1h 32m 43s. Rate: 7.8 symbols/min [2025-10-20T20:32:09.456149] [INFO ] [MAIN ] PDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:09.457992] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T20:32:09.481792] [DEBUG] [LOCAL ] PDT: Loaded 17796 existing records from primary storage. [2025-10-20T20:32:09.482051] [INFO ] [MAIN ] PDT: Loaded 17796 existing records. [2025-10-20T20:32:09.652491] [WARN ] [MAIN ] PDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:09.652738] [INFO ] [MAIN ] PDT: Wiping existing primary data before full refresh. [2025-10-20T20:32:09.652789] [INFO ] [LOCAL ] PDT: Starting deletion of primary data. [2025-10-20T20:32:09.652954] [DEBUG] [LOCAL ] PDT: Deleted primary data file. [2025-10-20T20:32:09.652991] [INFO ] [MAIN ] PDT: Starting full primary refresh. [2025-10-20T20:32:09.856531] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-08. [2025-10-20T20:32:10.023118] [INFO ] [REMOTE ] PDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:10.194930] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T20:32:10.651379] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-01. [2025-10-20T20:32:11.302403] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T20:32:11.318687] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T20:32:11.451369] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-03. [2025-10-20T20:32:12.251342] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-09. [2025-10-20T20:32:12.646176] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T20:32:13.051399] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-02. [2025-10-20T20:32:13.625499] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T20:32:13.782152] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-02 (0.73 seconds). [2025-10-20T20:32:13.851405] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-03. [2025-10-20T20:32:14.651402] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-09. [2025-10-20T20:32:14.980136] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T20:32:15.451398] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-04. [2025-10-20T20:32:16.155547] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T20:32:16.251397] [INFO ] [REMOTE ] PDO: Fetching primary data for 2025-10. [2025-10-20T20:32:16.618447] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T20:32:17.051413] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-04. [2025-10-20T20:32:17.324422] [INFO ] [REMOTE ] PDO: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T20:32:17.332288] [INFO ] [REMOTE ] PDO: Full fetch complete (46987 records). [2025-10-20T20:32:17.332463] [DEBUG] [VALIDATOR ] PDO: Validating primary data. [2025-10-20T20:32:17.716071] [DEBUG] [VALIDATOR ] PDO: All expected trading days are present. [2025-10-20T20:32:17.716157] [DEBUG] [VALIDATOR ] PDO: Primary data validation passed. [2025-10-20T20:32:17.716991] [INFO ] [MAIN ] PDO: Fetched primary data is valid. [2025-10-20T20:32:17.717054] [DEBUG] [LOCAL ] PDO: Saving primary data. [2025-10-20T20:32:17.718209] [DEBUG] [LOCAL ] PDO: Trimmed 46987 records to 31542 (keeping 200 days of data) [2025-10-20T20:32:17.876537] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-05. [2025-10-20T20:32:17.922607] [DEBUG] [LOCAL ] PDO: Primary data saved (31542 records). [2025-10-20T20:32:17.922816] [INFO ] [MAIN ] PDO: Full history saved. [2025-10-20T20:32:17.922911] [INFO ] [MAIN ] Update Progress: 1648/2367 symbols completed. PDO took 30.3s. Estimated remaining: 1h 32m 35s. Rate: 7.8 symbols/min [2025-10-20T20:32:18.092427] [INFO ] [MAIN ] PDX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:18.108590] [DEBUG] [LOCAL ] PDX: Loaded 14520 existing records from primary storage. [2025-10-20T20:32:18.108860] [INFO ] [MAIN ] PDX: Loaded 14520 existing records. [2025-10-20T20:32:18.259157] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T20:32:18.304894] [WARN ] [MAIN ] PDX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:18.305118] [INFO ] [MAIN ] PDX: Wiping existing primary data before full refresh. [2025-10-20T20:32:18.305181] [INFO ] [LOCAL ] PDX: Starting deletion of primary data. [2025-10-20T20:32:18.305372] [DEBUG] [LOCAL ] PDX: Deleted primary data file. [2025-10-20T20:32:18.305430] [INFO ] [MAIN ] PDX: Starting full primary refresh. [2025-10-20T20:32:18.656553] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-05. [2025-10-20T20:32:18.662945] [INFO ] [REMOTE ] PDX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:18.899983] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T20:32:19.451339] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-01. [2025-10-20T20:32:19.831429] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T20:32:20.251373] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-06. [2025-10-20T20:32:20.752820] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T20:32:21.051397] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-02. [2025-10-20T20:32:21.320881] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T20:32:21.851345] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-06. [2025-10-20T20:32:22.090867] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T20:32:22.651424] [INFO ] [REMOTE ] PDM: Fetching primary data for 2025-10. [2025-10-20T20:32:22.932236] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T20:32:23.451369] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-07. [2025-10-20T20:32:24.033491] [INFO ] [REMOTE ] PDM: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T20:32:24.039501] [INFO ] [REMOTE ] PDM: Full fetch complete (57359 records). [2025-10-20T20:32:24.039600] [DEBUG] [VALIDATOR ] PDM: Validating primary data. [2025-10-20T20:32:24.257393] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-07. [2025-10-20T20:32:24.422711] [DEBUG] [VALIDATOR ] PDM: All expected trading days are present. [2025-10-20T20:32:24.422788] [DEBUG] [VALIDATOR ] PDM: Primary data validation passed. [2025-10-20T20:32:24.423853] [INFO ] [MAIN ] PDM: Fetched primary data is valid. [2025-10-20T20:32:24.423990] [DEBUG] [LOCAL ] PDM: Saving primary data. [2025-10-20T20:32:24.425397] [DEBUG] [LOCAL ] PDM: Trimmed 57359 records to 39915 (keeping 200 days of data) [2025-10-20T20:32:24.641166] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T20:32:24.665017] [DEBUG] [LOCAL ] PDM: Primary data saved (39915 records). [2025-10-20T20:32:24.665122] [INFO ] [MAIN ] PDM: Full history saved. [2025-10-20T20:32:24.665544] [INFO ] [MAIN ] Update Progress: 1649/2367 symbols completed. PDM took 38.1s. Estimated remaining: 1h 32m 27s. Rate: 7.8 symbols/min [2025-10-20T20:32:24.829404] [INFO ] [MAIN ] PEB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:24.877462] [DEBUG] [LOCAL ] PEB: Loaded 51448 existing records from primary storage. [2025-10-20T20:32:24.877779] [INFO ] [MAIN ] PEB: Loaded 51448 existing records. [2025-10-20T20:32:25.064176] [WARN ] [MAIN ] PEB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:25.058465] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-03. [2025-10-20T20:32:25.065021] [INFO ] [MAIN ] PEB: Wiping existing primary data before full refresh. [2025-10-20T20:32:25.065856] [INFO ] [LOCAL ] PEB: Starting deletion of primary data. [2025-10-20T20:32:25.066141] [DEBUG] [LOCAL ] PEB: Deleted primary data file. [2025-10-20T20:32:25.066200] [INFO ] [MAIN ] PEB: Starting full primary refresh. [2025-10-20T20:32:25.360837] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T20:32:25.443172] [INFO ] [REMOTE ] PEB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:25.851392] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-08. [2025-10-20T20:32:26.088541] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:32:26.651354] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-04. [2025-10-20T20:32:27.009264] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:32:27.451355] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-01. [2025-10-20T20:32:27.737419] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:32:28.251426] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-08. [2025-10-20T20:32:29.051346] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-09. [2025-10-20T20:32:29.091468] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T20:32:29.273678] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T20:32:29.851341] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-09. [2025-10-20T20:32:30.070341] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:32:30.651354] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-02. [2025-10-20T20:32:30.731933] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:32:31.451364] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-05. [2025-10-20T20:32:32.251398] [INFO ] [REMOTE ] PDT: Fetching primary data for 2025-10. [2025-10-20T20:32:32.325952] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T20:32:32.437638] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T20:32:33.051343] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-06. [2025-10-20T20:32:33.285754] [INFO ] [REMOTE ] PDT: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:32:33.288994] [INFO ] [REMOTE ] PDT: Full fetch complete (25168 records). [2025-10-20T20:32:33.289097] [DEBUG] [VALIDATOR ] PDT: Validating primary data. [2025-10-20T20:32:33.653758] [DEBUG] [VALIDATOR ] PDT: All expected trading days are present. [2025-10-20T20:32:33.653845] [DEBUG] [VALIDATOR ] PDT: Primary data validation passed. [2025-10-20T20:32:33.654445] [INFO ] [MAIN ] PDT: Fetched primary data is valid. [2025-10-20T20:32:33.654490] [DEBUG] [LOCAL ] PDT: Saving primary data. [2025-10-20T20:32:33.655293] [DEBUG] [LOCAL ] PDT: Trimmed 25168 records to 17512 (keeping 200 days of data) [2025-10-20T20:32:33.765678] [DEBUG] [LOCAL ] PDT: Primary data saved (17512 records). [2025-10-20T20:32:33.765817] [INFO ] [MAIN ] PDT: Full history saved. [2025-10-20T20:32:33.765896] [INFO ] [MAIN ] Update Progress: 1650/2367 symbols completed. PDT took 24.5s. Estimated remaining: 1h 32m 20s. Rate: 7.8 symbols/min [2025-10-20T20:32:33.855788] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-03. [2025-10-20T20:32:33.937744] [INFO ] [MAIN ] PEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:33.988983] [DEBUG] [LOCAL ] PEG: Loaded 53920 existing records from primary storage. [2025-10-20T20:32:33.989312] [INFO ] [MAIN ] PEG: Loaded 53920 existing records. [2025-10-20T20:32:34.112306] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T20:32:34.196970] [WARN ] [MAIN ] PEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:34.197213] [INFO ] [MAIN ] PEG: Wiping existing primary data before full refresh. [2025-10-20T20:32:34.197260] [INFO ] [LOCAL ] PEG: Starting deletion of primary data. [2025-10-20T20:32:34.197451] [DEBUG] [LOCAL ] PEG: Deleted primary data file. [2025-10-20T20:32:34.197488] [INFO ] [MAIN ] PEG: Starting full primary refresh. [2025-10-20T20:32:34.551677] [INFO ] [REMOTE ] PEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:34.651340] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-01. [2025-10-20T20:32:35.451367] [INFO ] [REMOTE ] PDS: Fetching primary data for 2025-10. [2025-10-20T20:32:35.710876] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T20:32:35.997953] [INFO ] [REMOTE ] PDS: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:32:36.000727] [INFO ] [REMOTE ] PDS: Full fetch complete (25682 records). [2025-10-20T20:32:36.000838] [DEBUG] [VALIDATOR ] PDS: Validating primary data. [2025-10-20T20:32:36.265150] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-04. [2025-10-20T20:32:36.389941] [DEBUG] [VALIDATOR ] PDS: All expected trading days are present. [2025-10-20T20:32:36.390055] [DEBUG] [VALIDATOR ] PDS: Primary data validation passed. [2025-10-20T20:32:36.390699] [INFO ] [MAIN ] PDS: Fetched primary data is valid. [2025-10-20T20:32:36.390844] [DEBUG] [LOCAL ] PDS: Saving primary data. [2025-10-20T20:32:36.391701] [DEBUG] [LOCAL ] PDS: Trimmed 25682 records to 16958 (keeping 200 days of data) [2025-10-20T20:32:36.490946] [DEBUG] [LOCAL ] PDS: Primary data saved (16958 records). [2025-10-20T20:32:36.491187] [INFO ] [MAIN ] PDS: Full history saved. [2025-10-20T20:32:36.491298] [INFO ] [MAIN ] Update Progress: 1651/2367 symbols completed. PDS took 34.6s. Estimated remaining: 1h 32m 10s. Rate: 7.8 symbols/min [2025-10-20T20:32:36.711008] [INFO ] [MAIN ] PEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:36.714161] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T20:32:36.729571] [DEBUG] [LOCAL ] PEN: No existing data file found in primary storage. [2025-10-20T20:32:36.729692] [WARN ] [MAIN ] PEN: No existing data found, performing full refresh instead. [2025-10-20T20:32:36.730829] [INFO ] [MAIN ] PEN: Wiping existing primary data before full refresh. [2025-10-20T20:32:36.730881] [INFO ] [LOCAL ] PEN: Starting deletion of primary data. [2025-10-20T20:32:36.730931] [DEBUG] [LOCAL ] PEN: No primary data file found to delete. [2025-10-20T20:32:36.730970] [INFO ] [MAIN ] PEN: Starting full primary refresh. [2025-10-20T20:32:37.056585] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-02. [2025-10-20T20:32:37.093035] [INFO ] [REMOTE ] PEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:37.851421] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-07. [2025-10-20T20:32:38.180256] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T20:32:38.651385] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-01. [2025-10-20T20:32:38.877539] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T20:32:38.916261] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T20:32:39.451423] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-08. [2025-10-20T20:32:40.133559] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T20:32:40.148004] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T20:32:40.251385] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-02. [2025-10-20T20:32:41.051378] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-03. [2025-10-20T20:32:41.658464] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T20:32:41.851358] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-05. [2025-10-20T20:32:42.651361] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-09. [2025-10-20T20:32:42.965847] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T20:32:43.451347] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-04. [2025-10-20T20:32:43.574187] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:32:43.623056] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T20:32:44.251421] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-03. [2025-10-20T20:32:45.051349] [INFO ] [REMOTE ] PDX: Fetching primary data for 2025-10. [2025-10-20T20:32:45.431082] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T20:32:45.635463] [INFO ] [REMOTE ] PDX: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T20:32:45.637791] [INFO ] [REMOTE ] PDX: Full fetch complete (23736 records). [2025-10-20T20:32:45.637863] [DEBUG] [VALIDATOR ] PDX: Validating primary data. [2025-10-20T20:32:45.886107] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T20:32:45.860353] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-06. [2025-10-20T20:32:46.037679] [DEBUG] [VALIDATOR ] PDX: All expected trading days are present. [2025-10-20T20:32:46.037763] [DEBUG] [VALIDATOR ] PDX: Primary data validation passed. [2025-10-20T20:32:46.038263] [INFO ] [MAIN ] PDX: Fetched primary data is valid. [2025-10-20T20:32:46.038322] [DEBUG] [LOCAL ] PDX: Saving primary data. [2025-10-20T20:32:46.038911] [DEBUG] [LOCAL ] PDX: Trimmed 23736 records to 14168 (keeping 200 days of data) [2025-10-20T20:32:46.122106] [DEBUG] [LOCAL ] PDX: Primary data saved (14168 records). [2025-10-20T20:32:46.122232] [INFO ] [MAIN ] PDX: Full history saved. [2025-10-20T20:32:46.122305] [INFO ] [MAIN ] Update Progress: 1652/2367 symbols completed. PDX took 28.2s. Estimated remaining: 1h 32m 3s. Rate: 7.8 symbols/min [2025-10-20T20:32:46.291133] [INFO ] [MAIN ] PEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:46.303002] [DEBUG] [LOCAL ] PEO: Loaded 11292 existing records from primary storage. [2025-10-20T20:32:46.303179] [INFO ] [MAIN ] PEO: Loaded 11292 existing records. [2025-10-20T20:32:46.478824] [WARN ] [MAIN ] PEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:46.479019] [INFO ] [MAIN ] PEO: Wiping existing primary data before full refresh. [2025-10-20T20:32:46.479064] [INFO ] [LOCAL ] PEO: Starting deletion of primary data. [2025-10-20T20:32:46.479298] [DEBUG] [LOCAL ] PEO: Deleted primary data file. [2025-10-20T20:32:46.479355] [INFO ] [MAIN ] PEO: Starting full primary refresh. [2025-10-20T20:32:46.661490] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-05. [2025-10-20T20:32:46.837492] [INFO ] [REMOTE ] PEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:47.451366] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-04. [2025-10-20T20:32:47.757803] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-06 (1.90 seconds). [2025-10-20T20:32:48.251387] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-01. [2025-10-20T20:32:48.604097] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T20:32:48.645678] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T20:32:49.051350] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-06. [2025-10-20T20:32:49.233346] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T20:32:49.851379] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-07. [2025-10-20T20:32:50.651386] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-02. [2025-10-20T20:32:50.797099] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T20:32:51.451391] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-07. [2025-10-20T20:32:51.472780] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:32:51.579132] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T20:32:52.251399] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-08. [2025-10-20T20:32:53.051357] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-05. [2025-10-20T20:32:53.359129] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T20:32:53.851454] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-03. [2025-10-20T20:32:54.072826] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T20:32:54.456017] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T20:32:54.581191] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T20:32:54.651392] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-06. [2025-10-20T20:32:55.451383] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-09. [2025-10-20T20:32:55.961033] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T20:32:56.251424] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-08. [2025-10-20T20:32:57.010682] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T20:32:57.051331] [INFO ] [REMOTE ] PEB: Fetching primary data for 2025-10. [2025-10-20T20:32:57.828397] [INFO ] [REMOTE ] PEB: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:32:57.835983] [INFO ] [REMOTE ] PEB: Full fetch complete (72830 records). [2025-10-20T20:32:57.836154] [DEBUG] [VALIDATOR ] PEB: Validating primary data. [2025-10-20T20:32:57.837054] [FATAL] [VALIDATOR ] PEB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:32:57.837160] [FATAL] [MAIN ] PEB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:32:57.837233] [WARN ] [MAIN ] PEB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:32:57.837328] [INFO ] [LOCAL ] PEB: Starting deletion of primary data. [2025-10-20T20:32:57.837410] [DEBUG] [LOCAL ] PEB: No primary data file found to delete. [2025-10-20T20:32:57.837450] [INFO ] [LOCAL ] PEB: Starting deletion of derivative data. [2025-10-20T20:32:57.837611] [DEBUG] [LOCAL ] PEB: Deleted derivative data file. [2025-10-20T20:32:57.837976] [INFO ] [MAIN ] Update Progress: 1653/2367 symbols completed. PEB took 33.2s. Estimated remaining: 1h 31m 57s. Rate: 7.8 symbols/min [2025-10-20T20:32:57.861783] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-07. [2025-10-20T20:32:58.066440] [INFO ] [MAIN ] PERF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:32:58.088703] [DEBUG] [LOCAL ] PERF: Loaded 14549 existing records from primary storage. [2025-10-20T20:32:58.089063] [INFO ] [MAIN ] PERF: Loaded 14549 existing records. [2025-10-20T20:32:58.336941] [WARN ] [MAIN ] PERF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:32:58.337293] [INFO ] [MAIN ] PERF: Wiping existing primary data before full refresh. [2025-10-20T20:32:58.337489] [INFO ] [LOCAL ] PERF: Starting deletion of primary data. [2025-10-20T20:32:58.350112] [DEBUG] [LOCAL ] PERF: Deleted primary data file. [2025-10-20T20:32:58.372827] [INFO ] [MAIN ] PERF: Starting full primary refresh. [2025-10-20T20:32:58.386009] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T20:32:58.665493] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-04. [2025-10-20T20:32:58.914874] [INFO ] [REMOTE ] PERF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:32:59.415415] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T20:32:59.434051] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T20:32:59.451342] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-05. [2025-10-20T20:33:00.251376] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-09. [2025-10-20T20:33:00.331523] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T20:33:01.051356] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-06. [2025-10-20T20:33:01.851368] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-08. [2025-10-20T20:33:01.933684] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T20:33:02.119015] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T20:33:02.651377] [INFO ] [REMOTE ] PEG: Fetching primary data for 2025-10. [2025-10-20T20:33:03.160003] [INFO ] [REMOTE ] PEG: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T20:33:03.186533] [INFO ] [REMOTE ] PEG: Full fetch complete (76592 records). [2025-10-20T20:33:03.186838] [DEBUG] [VALIDATOR ] PEG: Validating primary data. [2025-10-20T20:33:03.188797] [FATAL] [VALIDATOR ] PEG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:33:03.190182] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T20:33:03.190394] [FATAL] [MAIN ] PEG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:33:03.196533] [WARN ] [MAIN ] PEG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:33:03.197098] [INFO ] [LOCAL ] PEG: Starting deletion of primary data. [2025-10-20T20:33:03.197344] [DEBUG] [LOCAL ] PEG: No primary data file found to delete. [2025-10-20T20:33:03.197401] [INFO ] [LOCAL ] PEG: Starting deletion of derivative data. [2025-10-20T20:33:03.197534] [DEBUG] [LOCAL ] PEG: Deleted derivative data file. [2025-10-20T20:33:03.197639] [INFO ] [MAIN ] Update Progress: 1654/2367 symbols completed. PEG took 29.4s. Estimated remaining: 1h 31m 49s. Rate: 7.8 symbols/min [2025-10-20T20:33:03.379732] [INFO ] [MAIN ] PEW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:03.379922] [DEBUG] [LOCAL ] PEW: No existing data file found in primary storage. [2025-10-20T20:33:03.379962] [WARN ] [MAIN ] PEW: No existing data found, performing full refresh instead. [2025-10-20T20:33:03.380076] [INFO ] [MAIN ] PEW: Wiping existing primary data before full refresh. [2025-10-20T20:33:03.380119] [INFO ] [LOCAL ] PEW: Starting deletion of primary data. [2025-10-20T20:33:03.380160] [DEBUG] [LOCAL ] PEW: No primary data file found to delete. [2025-10-20T20:33:03.380190] [INFO ] [MAIN ] PEW: Starting full primary refresh. [2025-10-20T20:33:03.456618] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-09. [2025-10-20T20:33:03.742190] [INFO ] [REMOTE ] PEW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:04.251434] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-01. [2025-10-20T20:33:04.662532] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T20:33:05.051349] [INFO ] [REMOTE ] PEW: Fetching primary data for 2025-01. [2025-10-20T20:33:05.132809] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T20:33:05.297894] [DEBUG] [REMOTE ] PEW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:33:05.298239] [WARN ] [REMOTE ] PEW: No data returned for month 2025-01 [2025-10-20T20:33:05.298322] [FATAL] [REMOTE ] PEW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:33:05.298367] [FATAL] [MAIN ] PEW: Failed to fetch primary data. [2025-10-20T20:33:05.298430] [WARN ] [MAIN ] PEW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:33:05.298501] [INFO ] [LOCAL ] PEW: Starting deletion of primary data. [2025-10-20T20:33:05.298570] [DEBUG] [LOCAL ] PEW: No primary data file found to delete. [2025-10-20T20:33:05.298612] [INFO ] [LOCAL ] PEW: Starting deletion of derivative data. [2025-10-20T20:33:05.298699] [DEBUG] [LOCAL ] PEW: No derivative data file found to delete. [2025-10-20T20:33:05.298767] [INFO ] [MAIN ] Update Progress: 1655/2367 symbols completed. PEW took 2.1s. Estimated remaining: 1h 31m 38s. Rate: 7.8 symbols/min [2025-10-20T20:33:05.474481] [INFO ] [MAIN ] PFD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:05.482282] [DEBUG] [LOCAL ] PFD: Loaded 5443 existing records from primary storage. [2025-10-20T20:33:05.482500] [INFO ] [MAIN ] PFD: Loaded 5443 existing records. [2025-10-20T20:33:05.658680] [WARN ] [MAIN ] PFD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:05.658929] [INFO ] [MAIN ] PFD: Wiping existing primary data before full refresh. [2025-10-20T20:33:05.658988] [INFO ] [LOCAL ] PFD: Starting deletion of primary data. [2025-10-20T20:33:05.659165] [DEBUG] [LOCAL ] PFD: Deleted primary data file. [2025-10-20T20:33:05.659220] [INFO ] [MAIN ] PFD: Starting full primary refresh. [2025-10-20T20:33:05.860793] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-07. [2025-10-20T20:33:06.020710] [INFO ] [REMOTE ] PFD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:06.651368] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-02. [2025-10-20T20:33:07.010916] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T20:33:07.451441] [INFO ] [REMOTE ] PFD: Fetching primary data for 2025-01. [2025-10-20T20:33:07.819688] [DEBUG] [REMOTE ] PFD: Error fetching primary data for 2025-01: 'API call returned an empty response.' [2025-10-20T20:33:07.820047] [WARN ] [REMOTE ] PFD: No data returned for month 2025-01 [2025-10-20T20:33:07.820138] [FATAL] [REMOTE ] PFD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:33:07.820183] [FATAL] [MAIN ] PFD: Failed to fetch primary data. [2025-10-20T20:33:07.820241] [WARN ] [MAIN ] PFD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:33:07.820310] [INFO ] [LOCAL ] PFD: Starting deletion of primary data. [2025-10-20T20:33:07.820390] [DEBUG] [LOCAL ] PFD: No primary data file found to delete. [2025-10-20T20:33:07.820446] [INFO ] [LOCAL ] PFD: Starting deletion of derivative data. [2025-10-20T20:33:07.820592] [DEBUG] [LOCAL ] PFD: Deleted derivative data file. [2025-10-20T20:33:07.820706] [INFO ] [MAIN ] Update Progress: 1656/2367 symbols completed. PFD took 2.5s. Estimated remaining: 1h 31m 29s. Rate: 7.8 symbols/min [2025-10-20T20:33:07.942269] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:33:08.002101] [INFO ] [MAIN ] PFE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:08.051820] [DEBUG] [LOCAL ] PFE: Loaded 54593 existing records from primary storage. [2025-10-20T20:33:08.052134] [INFO ] [MAIN ] PFE: Loaded 54593 existing records. [2025-10-20T20:33:08.225032] [WARN ] [MAIN ] PFE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:08.225217] [INFO ] [MAIN ] PFE: Wiping existing primary data before full refresh. [2025-10-20T20:33:08.225262] [INFO ] [LOCAL ] PFE: Starting deletion of primary data. [2025-10-20T20:33:08.225583] [DEBUG] [LOCAL ] PFE: Deleted primary data file. [2025-10-20T20:33:08.225621] [INFO ] [MAIN ] PFE: Starting full primary refresh. [2025-10-20T20:33:08.256578] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-03. [2025-10-20T20:33:08.584871] [INFO ] [REMOTE ] PFE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:09.051346] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-01. [2025-10-20T20:33:09.196255] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T20:33:09.851428] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-08. [2025-10-20T20:33:10.651390] [INFO ] [REMOTE ] PEN: Fetching primary data for 2025-10. [2025-10-20T20:33:10.692929] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T20:33:11.301550] [INFO ] [REMOTE ] PEN: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T20:33:11.305558] [INFO ] [REMOTE ] PEN: Full fetch complete (44063 records). [2025-10-20T20:33:11.305682] [DEBUG] [VALIDATOR ] PEN: Validating primary data. [2025-10-20T20:33:11.306205] [FATAL] [VALIDATOR ] PEN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:33:11.306290] [FATAL] [MAIN ] PEN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:33:11.306333] [WARN ] [MAIN ] PEN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:33:11.306370] [INFO ] [LOCAL ] PEN: Starting deletion of primary data. [2025-10-20T20:33:11.306424] [DEBUG] [LOCAL ] PEN: No primary data file found to delete. [2025-10-20T20:33:11.306458] [INFO ] [LOCAL ] PEN: Starting deletion of derivative data. [2025-10-20T20:33:11.306496] [DEBUG] [LOCAL ] PEN: No derivative data file found to delete. [2025-10-20T20:33:11.306550] [INFO ] [MAIN ] Update Progress: 1657/2367 symbols completed. PEN took 34.8s. Estimated remaining: 1h 31m 19s. Rate: 7.8 symbols/min [2025-10-20T20:33:11.454324] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-09. [2025-10-20T20:33:11.481235] [INFO ] [MAIN ] PFGC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:11.526304] [DEBUG] [LOCAL ] PFGC: Loaded 47796 existing records from primary storage. [2025-10-20T20:33:11.526765] [INFO ] [MAIN ] PFGC: Loaded 47796 existing records. [2025-10-20T20:33:11.711243] [WARN ] [MAIN ] PFGC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:11.711730] [INFO ] [MAIN ] PFGC: Wiping existing primary data before full refresh. [2025-10-20T20:33:11.711856] [INFO ] [LOCAL ] PFGC: Starting deletion of primary data. [2025-10-20T20:33:11.712133] [DEBUG] [LOCAL ] PFGC: Deleted primary data file. [2025-10-20T20:33:11.712242] [INFO ] [MAIN ] PFGC: Starting full primary refresh. [2025-10-20T20:33:11.922205] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-01 (2.87 seconds). [2025-10-20T20:33:12.138020] [INFO ] [REMOTE ] PFGC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:12.251382] [INFO ] [REMOTE ] PFGC: Fetching primary data for 2025-01. [2025-10-20T20:33:12.419872] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T20:33:13.051394] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-04. [2025-10-20T20:33:13.834408] [INFO ] [REMOTE ] PFGC: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T20:33:13.836355] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T20:33:13.851316] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-05. [2025-10-20T20:33:14.404853] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T20:33:14.651404] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-06. [2025-10-20T20:33:15.451401] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-02. [2025-10-20T20:33:15.793373] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T20:33:16.251363] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-07. [2025-10-20T20:33:17.051404] [INFO ] [REMOTE ] PEO: Fetching primary data for 2025-10. [2025-10-20T20:33:17.491425] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T20:33:17.851394] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-08. [2025-10-20T20:33:17.919556] [INFO ] [REMOTE ] PEO: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T20:33:17.921545] [INFO ] [REMOTE ] PEO: Full fetch complete (16252 records). [2025-10-20T20:33:17.921613] [DEBUG] [VALIDATOR ] PEO: Validating primary data. [2025-10-20T20:33:18.326134] [DEBUG] [VALIDATOR ] PEO: All expected trading days are present. [2025-10-20T20:33:18.326019] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-02 (2.87 seconds). [2025-10-20T20:33:18.326315] [DEBUG] [VALIDATOR ] PEO: Primary data validation passed. [2025-10-20T20:33:18.337173] [INFO ] [MAIN ] PEO: Fetched primary data is valid. [2025-10-20T20:33:18.337626] [DEBUG] [LOCAL ] PEO: Saving primary data. [2025-10-20T20:33:18.338556] [DEBUG] [LOCAL ] PEO: Trimmed 16252 records to 11179 (keeping 200 days of data) [2025-10-20T20:33:18.408711] [DEBUG] [LOCAL ] PEO: Primary data saved (11179 records). [2025-10-20T20:33:18.408871] [INFO ] [MAIN ] PEO: Full history saved. [2025-10-20T20:33:18.408963] [INFO ] [MAIN ] Update Progress: 1658/2367 symbols completed. PEO took 32.3s. Estimated remaining: 1h 31m 11s. Rate: 7.8 symbols/min [2025-10-20T20:33:18.577899] [INFO ] [MAIN ] PFH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:18.586675] [DEBUG] [LOCAL ] PFH: Loaded 6643 existing records from primary storage. [2025-10-20T20:33:18.586886] [INFO ] [MAIN ] PFH: Loaded 6643 existing records. [2025-10-20T20:33:18.656794] [INFO ] [REMOTE ] PFGC: Fetching primary data for 2025-02. [2025-10-20T20:33:18.773086] [WARN ] [MAIN ] PFH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:18.773294] [INFO ] [MAIN ] PFH: Wiping existing primary data before full refresh. [2025-10-20T20:33:18.773339] [INFO ] [LOCAL ] PFH: Starting deletion of primary data. [2025-10-20T20:33:18.773482] [DEBUG] [LOCAL ] PFH: Deleted primary data file. [2025-10-20T20:33:18.773519] [INFO ] [MAIN ] PFH: Starting full primary refresh. [2025-10-20T20:33:19.129847] [INFO ] [REMOTE ] PFH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:19.141411] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T20:33:19.451371] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-01. [2025-10-20T20:33:20.251356] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-09. [2025-10-20T20:33:20.324373] [INFO ] [REMOTE ] PFGC: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T20:33:20.475617] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T20:33:21.051362] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-02. [2025-10-20T20:33:21.168487] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T20:33:21.573176] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T20:33:21.851381] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-03. [2025-10-20T20:33:22.633595] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T20:33:22.651329] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-04. [2025-10-20T20:33:23.452356] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-03. [2025-10-20T20:33:23.455162] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T20:33:24.251371] [INFO ] [REMOTE ] PFGC: Fetching primary data for 2025-03. [2025-10-20T20:33:25.051394] [INFO ] [REMOTE ] PERF: Fetching primary data for 2025-10. [2025-10-20T20:33:25.755625] [INFO ] [REMOTE ] PERF: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:33:25.758458] [INFO ] [REMOTE ] PERF: Full fetch complete (21770 records). [2025-10-20T20:33:25.758529] [DEBUG] [VALIDATOR ] PERF: Validating primary data. [2025-10-20T20:33:25.861525] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-05. [2025-10-20T20:33:26.135858] [DEBUG] [VALIDATOR ] PERF: All expected trading days are present. [2025-10-20T20:33:26.145617] [DEBUG] [VALIDATOR ] PERF: Primary data validation passed. [2025-10-20T20:33:26.146270] [INFO ] [MAIN ] PERF: Fetched primary data is valid. [2025-10-20T20:33:26.146432] [DEBUG] [LOCAL ] PERF: Saving primary data. [2025-10-20T20:33:26.165968] [INFO ] [REMOTE ] PFGC: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T20:33:26.166239] [DEBUG] [LOCAL ] PERF: Trimmed 21770 records to 14457 (keeping 200 days of data) [2025-10-20T20:33:26.286789] [DEBUG] [LOCAL ] PERF: Primary data saved (14457 records). [2025-10-20T20:33:26.287808] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-03 (2.84 seconds). [2025-10-20T20:33:26.288060] [INFO ] [MAIN ] PERF: Full history saved. [2025-10-20T20:33:26.298763] [INFO ] [MAIN ] Update Progress: 1659/2367 symbols completed. PERF took 28.5s. Estimated remaining: 1h 31m 3s. Rate: 7.8 symbols/min [2025-10-20T20:33:26.486908] [INFO ] [MAIN ] PFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:26.493925] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T20:33:26.509539] [DEBUG] [LOCAL ] PFL: Loaded 14622 existing records from primary storage. [2025-10-20T20:33:26.509882] [INFO ] [MAIN ] PFL: Loaded 14622 existing records. [2025-10-20T20:33:26.656774] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-04. [2025-10-20T20:33:26.697497] [WARN ] [MAIN ] PFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:26.697842] [INFO ] [MAIN ] PFL: Wiping existing primary data before full refresh. [2025-10-20T20:33:26.697903] [INFO ] [LOCAL ] PFL: Starting deletion of primary data. [2025-10-20T20:33:26.698191] [DEBUG] [LOCAL ] PFL: Deleted primary data file. [2025-10-20T20:33:26.698255] [INFO ] [MAIN ] PFL: Starting full primary refresh. [2025-10-20T20:33:27.046043] [INFO ] [REMOTE ] PFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:27.451385] [INFO ] [REMOTE ] PFGC: Fetching primary data for 2025-04. [2025-10-20T20:33:28.251393] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-06. [2025-10-20T20:33:29.027345] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T20:33:29.051336] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-07. [2025-10-20T20:33:29.061261] [DEBUG] [REMOTE ] PFGC: Error fetching primary data for 2025-04: ("Connection broken: ConnectionResetError(104, 'Connection reset by peer')", ConnectionResetError(104, 'Connection reset by peer')) [2025-10-20T20:33:29.061426] [WARN ] [REMOTE ] PFGC: No data returned for month 2025-04 [2025-10-20T20:33:29.061548] [FATAL] [REMOTE ] PFGC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:33:29.061725] [FATAL] [MAIN ] PFGC: Failed to fetch primary data. [2025-10-20T20:33:29.061848] [WARN ] [MAIN ] PFGC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:33:29.061976] [INFO ] [LOCAL ] PFGC: Starting deletion of primary data. [2025-10-20T20:33:29.062084] [DEBUG] [LOCAL ] PFGC: No primary data file found to delete. [2025-10-20T20:33:29.062155] [INFO ] [LOCAL ] PFGC: Starting deletion of derivative data. [2025-10-20T20:33:29.062313] [DEBUG] [LOCAL ] PFGC: Deleted derivative data file. [2025-10-20T20:33:29.062382] [INFO ] [MAIN ] Update Progress: 1660/2367 symbols completed. PFGC took 17.8s. Estimated remaining: 1h 30m 54s. Rate: 7.8 symbols/min [2025-10-20T20:33:29.249912] [INFO ] [MAIN ] PFLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:29.292978] [DEBUG] [LOCAL ] PFLT: Loaded 44975 existing records from primary storage. [2025-10-20T20:33:29.293300] [INFO ] [MAIN ] PFLT: Loaded 44975 existing records. [2025-10-20T20:33:29.515245] [WARN ] [MAIN ] PFLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:29.515574] [INFO ] [MAIN ] PFLT: Wiping existing primary data before full refresh. [2025-10-20T20:33:29.515781] [INFO ] [LOCAL ] PFLT: Starting deletion of primary data. [2025-10-20T20:33:29.516338] [DEBUG] [LOCAL ] PFLT: Deleted primary data file. [2025-10-20T20:33:29.516396] [INFO ] [MAIN ] PFLT: Starting full primary refresh. [2025-10-20T20:33:29.730012] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-04 (3.07 seconds). [2025-10-20T20:33:29.856366] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-05. [2025-10-20T20:33:29.908368] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T20:33:29.999580] [INFO ] [REMOTE ] PFLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:30.651359] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-01. [2025-10-20T20:33:31.451390] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-08. [2025-10-20T20:33:32.162586] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T20:33:32.251394] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-01. [2025-10-20T20:33:32.565689] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T20:33:32.719121] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-05 (2.86 seconds). [2025-10-20T20:33:33.051437] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-09. [2025-10-20T20:33:33.181032] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T20:33:33.829802] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T20:33:33.851339] [INFO ] [REMOTE ] PFH: Fetching primary data for 2025-10. [2025-10-20T20:33:34.337883] [INFO ] [REMOTE ] PFH: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T20:33:34.339175] [INFO ] [REMOTE ] PFH: Full fetch complete (9657 records). [2025-10-20T20:33:34.339254] [DEBUG] [VALIDATOR ] PFH: Validating primary data. [2025-10-20T20:33:34.656457] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-02. [2025-10-20T20:33:34.717140] [DEBUG] [VALIDATOR ] PFH: All expected trading days are present. [2025-10-20T20:33:34.717228] [DEBUG] [VALIDATOR ] PFH: Primary data validation passed. [2025-10-20T20:33:34.717455] [INFO ] [MAIN ] PFH: Fetched primary data is valid. [2025-10-20T20:33:34.717493] [DEBUG] [LOCAL ] PFH: Saving primary data. [2025-10-20T20:33:34.717918] [DEBUG] [LOCAL ] PFH: Trimmed 9657 records to 6502 (keeping 200 days of data) [2025-10-20T20:33:34.755828] [DEBUG] [LOCAL ] PFH: Primary data saved (6502 records). [2025-10-20T20:33:34.755920] [INFO ] [MAIN ] PFH: Full history saved. [2025-10-20T20:33:34.755991] [INFO ] [MAIN ] Update Progress: 1661/2367 symbols completed. PFH took 16.3s. Estimated remaining: 1h 30m 45s. Rate: 7.8 symbols/min [2025-10-20T20:33:34.920173] [INFO ] [MAIN ] PFN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:34.941375] [DEBUG] [LOCAL ] PFN: Loaded 22752 existing records from primary storage. [2025-10-20T20:33:34.941593] [INFO ] [MAIN ] PFN: Loaded 22752 existing records. [2025-10-20T20:33:35.117841] [WARN ] [MAIN ] PFN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:35.118078] [INFO ] [MAIN ] PFN: Wiping existing primary data before full refresh. [2025-10-20T20:33:35.118136] [INFO ] [LOCAL ] PFN: Starting deletion of primary data. [2025-10-20T20:33:35.118328] [DEBUG] [LOCAL ] PFN: Deleted primary data file. [2025-10-20T20:33:35.118383] [INFO ] [MAIN ] PFN: Starting full primary refresh. [2025-10-20T20:33:35.462932] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-02. [2025-10-20T20:33:35.468226] [INFO ] [REMOTE ] PFN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:35.593187] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T20:33:36.251363] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-06. [2025-10-20T20:33:37.056037] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-01. [2025-10-20T20:33:37.061145] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T20:33:37.851374] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-03. [2025-10-20T20:33:38.100781] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T20:33:38.651364] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-02. [2025-10-20T20:33:38.835138] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-06 (2.58 seconds). [2025-10-20T20:33:38.855294] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T20:33:39.451354] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-04. [2025-10-20T20:33:39.470870] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:33:40.251453] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-03. [2025-10-20T20:33:40.579090] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T20:33:41.051406] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-05. [2025-10-20T20:33:41.425340] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T20:33:41.851347] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-04. [2025-10-20T20:33:41.890456] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T20:33:42.651372] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-07. [2025-10-20T20:33:43.036201] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T20:33:43.451403] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-05. [2025-10-20T20:33:44.251400] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-06. [2025-10-20T20:33:44.532403] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T20:33:45.051392] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-03. [2025-10-20T20:33:45.378063] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:33:45.851535] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-07. [2025-10-20T20:33:45.853888] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-07 (3.20 seconds). [2025-10-20T20:33:46.651406] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-08. [2025-10-20T20:33:46.676312] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T20:33:47.007476] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T20:33:47.451423] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-06. [2025-10-20T20:33:48.251366] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-08. [2025-10-20T20:33:48.873444] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T20:33:49.051345] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-04. [2025-10-20T20:33:49.116823] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:33:49.368484] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-08 (2.72 seconds). [2025-10-20T20:33:49.851388] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-07. [2025-10-20T20:33:50.651386] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-09. [2025-10-20T20:33:50.874900] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T20:33:50.930439] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T20:33:51.451406] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-08. [2025-10-20T20:33:51.725207] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T20:33:52.251501] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-05. [2025-10-20T20:33:52.320286] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:33:53.051407] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-09. [2025-10-20T20:33:53.795224] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T20:33:53.851416] [INFO ] [REMOTE ] PFE: Fetching primary data for 2025-10. [2025-10-20T20:33:53.974981] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:33:54.631685] [INFO ] [REMOTE ] PFE: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:33:54.639593] [INFO ] [REMOTE ] PFE: Full fetch complete (77603 records). [2025-10-20T20:33:54.639842] [DEBUG] [VALIDATOR ] PFE: Validating primary data. [2025-10-20T20:33:54.640704] [FATAL] [VALIDATOR ] PFE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:33:54.640805] [FATAL] [MAIN ] PFE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:33:54.640849] [WARN ] [MAIN ] PFE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:33:54.640921] [INFO ] [LOCAL ] PFE: Starting deletion of primary data. [2025-10-20T20:33:54.641003] [DEBUG] [LOCAL ] PFE: No primary data file found to delete. [2025-10-20T20:33:54.641060] [INFO ] [LOCAL ] PFE: Starting deletion of derivative data. [2025-10-20T20:33:54.641366] [DEBUG] [LOCAL ] PFE: Deleted derivative data file. [2025-10-20T20:33:54.641462] [INFO ] [MAIN ] Update Progress: 1662/2367 symbols completed. PFE took 46.8s. Estimated remaining: 1h 30m 42s. Rate: 7.8 symbols/min [2025-10-20T20:33:54.656482] [INFO ] [REMOTE ] PFL: Fetching primary data for 2025-10. [2025-10-20T20:33:54.820856] [INFO ] [MAIN ] PFO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:54.828499] [DEBUG] [LOCAL ] PFO: Loaded 6140 existing records from primary storage. [2025-10-20T20:33:54.828713] [INFO ] [MAIN ] PFO: Loaded 6140 existing records. [2025-10-20T20:33:54.999893] [WARN ] [MAIN ] PFO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:55.000124] [INFO ] [MAIN ] PFO: Wiping existing primary data before full refresh. [2025-10-20T20:33:55.000171] [INFO ] [LOCAL ] PFO: Starting deletion of primary data. [2025-10-20T20:33:55.000321] [DEBUG] [LOCAL ] PFO: Deleted primary data file. [2025-10-20T20:33:55.000357] [INFO ] [MAIN ] PFO: Starting full primary refresh. [2025-10-20T20:33:55.341371] [INFO ] [REMOTE ] PFO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:55.451393] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-01. [2025-10-20T20:33:55.716754] [INFO ] [REMOTE ] PFL: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T20:33:55.720283] [INFO ] [REMOTE ] PFL: Full fetch complete (20756 records). [2025-10-20T20:33:55.720371] [DEBUG] [VALIDATOR ] PFL: Validating primary data. [2025-10-20T20:33:56.094440] [DEBUG] [VALIDATOR ] PFL: All expected trading days are present. [2025-10-20T20:33:56.094519] [DEBUG] [VALIDATOR ] PFL: Primary data validation passed. [2025-10-20T20:33:56.094980] [INFO ] [MAIN ] PFL: Fetched primary data is valid. [2025-10-20T20:33:56.095040] [DEBUG] [LOCAL ] PFL: Saving primary data. [2025-10-20T20:33:56.095610] [DEBUG] [LOCAL ] PFL: Trimmed 20756 records to 14544 (keeping 200 days of data) [2025-10-20T20:33:56.180099] [DEBUG] [LOCAL ] PFL: Primary data saved (14544 records). [2025-10-20T20:33:56.180237] [INFO ] [MAIN ] PFL: Full history saved. [2025-10-20T20:33:56.180314] [INFO ] [MAIN ] Update Progress: 1663/2367 symbols completed. PFL took 29.9s. Estimated remaining: 1h 30m 32s. Rate: 7.8 symbols/min [2025-10-20T20:33:56.256524] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-09. [2025-10-20T20:33:56.348737] [INFO ] [MAIN ] PFS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:56.384217] [DEBUG] [LOCAL ] PFS: Loaded 36502 existing records from primary storage. [2025-10-20T20:33:56.384508] [INFO ] [MAIN ] PFS: Loaded 36502 existing records. [2025-10-20T20:33:56.492109] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T20:33:56.569258] [WARN ] [MAIN ] PFS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:56.569521] [INFO ] [MAIN ] PFS: Wiping existing primary data before full refresh. [2025-10-20T20:33:56.569579] [INFO ] [LOCAL ] PFS: Starting deletion of primary data. [2025-10-20T20:33:56.569802] [DEBUG] [LOCAL ] PFS: Deleted primary data file. [2025-10-20T20:33:56.569868] [INFO ] [MAIN ] PFS: Starting full primary refresh. [2025-10-20T20:33:56.914257] [INFO ] [REMOTE ] PFS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:33:57.051353] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-01. [2025-10-20T20:33:57.495301] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T20:33:57.851376] [INFO ] [REMOTE ] PFN: Fetching primary data for 2025-10. [2025-10-20T20:33:58.651419] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-02. [2025-10-20T20:33:58.760885] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T20:33:58.947957] [INFO ] [REMOTE ] PFN: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T20:33:58.952163] [INFO ] [REMOTE ] PFN: Full fetch complete (31350 records). [2025-10-20T20:33:58.952247] [DEBUG] [VALIDATOR ] PFN: Validating primary data. [2025-10-20T20:33:59.294497] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T20:33:59.324375] [DEBUG] [VALIDATOR ] PFN: All expected trading days are present. [2025-10-20T20:33:59.324478] [DEBUG] [VALIDATOR ] PFN: Primary data validation passed. [2025-10-20T20:33:59.325197] [INFO ] [MAIN ] PFN: Fetched primary data is valid. [2025-10-20T20:33:59.325259] [DEBUG] [LOCAL ] PFN: Saving primary data. [2025-10-20T20:33:59.326193] [DEBUG] [LOCAL ] PFN: Trimmed 31350 records to 22398 (keeping 200 days of data) [2025-10-20T20:33:59.452337] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-03. [2025-10-20T20:33:59.456345] [DEBUG] [LOCAL ] PFN: Primary data saved (22398 records). [2025-10-20T20:33:59.456677] [INFO ] [MAIN ] PFN: Full history saved. [2025-10-20T20:33:59.456797] [INFO ] [MAIN ] Update Progress: 1664/2367 symbols completed. PFN took 24.7s. Estimated remaining: 1h 30m 23s. Rate: 7.8 symbols/min [2025-10-20T20:33:59.629154] [INFO ] [MAIN ] PFSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:33:59.658118] [DEBUG] [LOCAL ] PFSI: Loaded 29716 existing records from primary storage. [2025-10-20T20:33:59.658396] [INFO ] [MAIN ] PFSI: Loaded 29716 existing records. [2025-10-20T20:33:59.839704] [WARN ] [MAIN ] PFSI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:33:59.839898] [INFO ] [MAIN ] PFSI: Wiping existing primary data before full refresh. [2025-10-20T20:33:59.839945] [INFO ] [LOCAL ] PFSI: Starting deletion of primary data. [2025-10-20T20:33:59.840114] [DEBUG] [LOCAL ] PFSI: Deleted primary data file. [2025-10-20T20:33:59.840153] [INFO ] [MAIN ] PFSI: Starting full primary refresh. [2025-10-20T20:34:00.200848] [INFO ] [REMOTE ] PFSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:00.194936] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T20:34:00.251336] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-01. [2025-10-20T20:34:01.051416] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-02. [2025-10-20T20:34:01.848999] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:34:01.854759] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-06. [2025-10-20T20:34:02.326168] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T20:34:02.651437] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-04. [2025-10-20T20:34:03.325144] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:34:03.352201] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T20:34:03.451361] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-07. [2025-10-20T20:34:04.251480] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-03. [2025-10-20T20:34:05.051398] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-05. [2025-10-20T20:34:05.227297] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T20:34:05.762443] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T20:34:05.817730] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T20:34:05.851404] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-02. [2025-10-20T20:34:06.651379] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-06. [2025-10-20T20:34:07.258891] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T20:34:07.303816] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T20:34:07.451383] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-08. [2025-10-20T20:34:08.251381] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-07. [2025-10-20T20:34:09.014746] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T20:34:09.051350] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-08. [2025-10-20T20:34:09.281761] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T20:34:09.838998] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T20:34:09.851326] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-09. [2025-10-20T20:34:10.651357] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-09. [2025-10-20T20:34:10.701207] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T20:34:11.451375] [INFO ] [REMOTE ] PFO: Fetching primary data for 2025-10. [2025-10-20T20:34:12.254618] [INFO ] [REMOTE ] PFO: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:34:12.254454] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-04. [2025-10-20T20:34:12.256687] [INFO ] [REMOTE ] PFO: Full fetch complete (7822 records). [2025-10-20T20:34:12.256756] [DEBUG] [VALIDATOR ] PFO: Validating primary data. [2025-10-20T20:34:12.650029] [DEBUG] [VALIDATOR ] PFO: All expected trading days are present. [2025-10-20T20:34:12.650174] [DEBUG] [VALIDATOR ] PFO: Primary data validation passed. [2025-10-20T20:34:12.650395] [INFO ] [MAIN ] PFO: Fetched primary data is valid. [2025-10-20T20:34:12.650491] [DEBUG] [LOCAL ] PFO: Saving primary data. [2025-10-20T20:34:12.652178] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T20:34:12.661464] [DEBUG] [LOCAL ] PFO: Trimmed 7822 records to 6099 (keeping 200 days of data) [2025-10-20T20:34:12.703477] [DEBUG] [LOCAL ] PFO: Primary data saved (6099 records). [2025-10-20T20:34:12.703589] [INFO ] [MAIN ] PFO: Full history saved. [2025-10-20T20:34:12.703684] [INFO ] [MAIN ] Update Progress: 1665/2367 symbols completed. PFO took 18.1s. Estimated remaining: 1h 30m 17s. Rate: 7.8 symbols/min [2025-10-20T20:34:12.868421] [INFO ] [MAIN ] PG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:12.919471] [DEBUG] [LOCAL ] PG: Loaded 54533 existing records from primary storage. [2025-10-20T20:34:12.919801] [INFO ] [MAIN ] PG: Loaded 54533 existing records. [2025-10-20T20:34:13.059992] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-03. [2025-10-20T20:34:13.100874] [WARN ] [MAIN ] PG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:34:13.101084] [INFO ] [MAIN ] PG: Wiping existing primary data before full refresh. [2025-10-20T20:34:13.101137] [INFO ] [LOCAL ] PG: Starting deletion of primary data. [2025-10-20T20:34:13.101705] [DEBUG] [LOCAL ] PG: Deleted primary data file. [2025-10-20T20:34:13.101761] [INFO ] [MAIN ] PG: Starting full primary refresh. [2025-10-20T20:34:13.472148] [INFO ] [REMOTE ] PG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:13.851343] [INFO ] [REMOTE ] PFLT: Fetching primary data for 2025-10. [2025-10-20T20:34:14.160061] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T20:34:14.551284] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T20:34:14.651379] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-01. [2025-10-20T20:34:15.451359] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-04. [2025-10-20T20:34:15.504960] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-01 (0.85 seconds). [2025-10-20T20:34:15.765589] [INFO ] [REMOTE ] PFLT: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T20:34:15.773490] [INFO ] [REMOTE ] PFLT: Full fetch complete (64365 records). [2025-10-20T20:34:15.773737] [DEBUG] [VALIDATOR ] PFLT: Validating primary data. [2025-10-20T20:34:16.170956] [DEBUG] [VALIDATOR ] PFLT: All expected trading days are present. [2025-10-20T20:34:16.171043] [DEBUG] [VALIDATOR ] PFLT: Primary data validation passed. [2025-10-20T20:34:16.172018] [INFO ] [MAIN ] PFLT: Fetched primary data is valid. [2025-10-20T20:34:16.172081] [DEBUG] [LOCAL ] PFLT: Saving primary data. [2025-10-20T20:34:16.173388] [DEBUG] [LOCAL ] PFLT: Trimmed 64365 records to 44339 (keeping 200 days of data) [2025-10-20T20:34:16.262237] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-05. [2025-10-20T20:34:16.446782] [DEBUG] [LOCAL ] PFLT: Primary data saved (44339 records). [2025-10-20T20:34:16.446904] [INFO ] [MAIN ] PFLT: Full history saved. [2025-10-20T20:34:16.446978] [INFO ] [MAIN ] Update Progress: 1666/2367 symbols completed. PFLT took 47.4s. Estimated remaining: 1h 30m 8s. Rate: 7.8 symbols/min [2025-10-20T20:34:16.617055] [INFO ] [MAIN ] PGP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:16.624997] [DEBUG] [LOCAL ] PGP: Loaded 5778 existing records from primary storage. [2025-10-20T20:34:16.625224] [INFO ] [MAIN ] PGP: Loaded 5778 existing records. [2025-10-20T20:34:16.806825] [WARN ] [MAIN ] PGP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:34:16.807065] [INFO ] [MAIN ] PGP: Wiping existing primary data before full refresh. [2025-10-20T20:34:16.807114] [INFO ] [LOCAL ] PGP: Starting deletion of primary data. [2025-10-20T20:34:16.807282] [DEBUG] [LOCAL ] PGP: Deleted primary data file. [2025-10-20T20:34:16.807342] [INFO ] [MAIN ] PGP: Starting full primary refresh. [2025-10-20T20:34:17.053943] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-02. [2025-10-20T20:34:17.190146] [INFO ] [REMOTE ] PGP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:17.191631] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:34:17.648251] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T20:34:17.851360] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-06. [2025-10-20T20:34:17.931857] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T20:34:18.651402] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-01. [2025-10-20T20:34:19.089481] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T20:34:19.451419] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-03. [2025-10-20T20:34:19.540553] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:34:20.073791] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T20:34:20.251390] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-05. [2025-10-20T20:34:21.051378] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-02. [2025-10-20T20:34:21.506784] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T20:34:21.732835] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T20:34:21.851367] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-07. [2025-10-20T20:34:22.651413] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-03. [2025-10-20T20:34:23.309196] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:34:23.390149] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T20:34:23.451437] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-04. [2025-10-20T20:34:24.251364] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-08. [2025-10-20T20:34:24.320808] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T20:34:25.051378] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-04. [2025-10-20T20:34:25.761455] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:34:25.851373] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-09. [2025-10-20T20:34:25.925403] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T20:34:26.651380] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-06. [2025-10-20T20:34:27.320394] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T20:34:27.451367] [INFO ] [REMOTE ] PFS: Fetching primary data for 2025-10. [2025-10-20T20:34:27.829115] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T20:34:28.138970] [INFO ] [REMOTE ] PFS: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:34:28.144943] [INFO ] [REMOTE ] PFS: Full fetch complete (52010 records). [2025-10-20T20:34:28.145079] [DEBUG] [VALIDATOR ] PFS: Validating primary data. [2025-10-20T20:34:28.145677] [FATAL] [VALIDATOR ] PFS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:34:28.145788] [FATAL] [MAIN ] PFS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:34:28.145834] [WARN ] [MAIN ] PFS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:34:28.145875] [INFO ] [LOCAL ] PFS: Starting deletion of primary data. [2025-10-20T20:34:28.145930] [DEBUG] [LOCAL ] PFS: No primary data file found to delete. [2025-10-20T20:34:28.145966] [INFO ] [LOCAL ] PFS: Starting deletion of derivative data. [2025-10-20T20:34:28.146102] [DEBUG] [LOCAL ] PFS: Deleted derivative data file. [2025-10-20T20:34:28.146412] [INFO ] [MAIN ] Update Progress: 1667/2367 symbols completed. PFS took 32.0s. Estimated remaining: 1h 30m 2s. Rate: 7.8 symbols/min [2025-10-20T20:34:28.256569] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-05. [2025-10-20T20:34:28.356918] [INFO ] [MAIN ] PGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:28.410278] [DEBUG] [LOCAL ] PGR: Loaded 53647 existing records from primary storage. [2025-10-20T20:34:28.410620] [INFO ] [MAIN ] PGR: Loaded 53647 existing records. [2025-10-20T20:34:28.631738] [WARN ] [MAIN ] PGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:34:28.631939] [INFO ] [MAIN ] PGR: Wiping existing primary data before full refresh. [2025-10-20T20:34:28.631986] [INFO ] [LOCAL ] PGR: Starting deletion of primary data. [2025-10-20T20:34:28.632551] [DEBUG] [LOCAL ] PGR: Deleted primary data file. [2025-10-20T20:34:28.632613] [INFO ] [MAIN ] PGR: Starting full primary refresh. [2025-10-20T20:34:28.980245] [INFO ] [REMOTE ] PGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:28.985566] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T20:34:29.051352] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-06. [2025-10-20T20:34:29.619894] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T20:34:29.851367] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-07. [2025-10-20T20:34:30.651388] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-07. [2025-10-20T20:34:31.271228] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T20:34:31.386498] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T20:34:31.451400] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-05. [2025-10-20T20:34:32.251395] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-08. [2025-10-20T20:34:32.371010] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T20:34:32.713894] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T20:34:33.051426] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-01. [2025-10-20T20:34:33.851412] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-08. [2025-10-20T20:34:34.651415] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-09. [2025-10-20T20:34:35.182451] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T20:34:35.348163] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T20:34:35.445806] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-01 (2.39 seconds). [2025-10-20T20:34:35.456169] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-06. [2025-10-20T20:34:36.251434] [INFO ] [REMOTE ] PGP: Fetching primary data for 2025-10. [2025-10-20T20:34:36.337406] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T20:34:36.901839] [INFO ] [REMOTE ] PGP: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T20:34:36.903327] [INFO ] [REMOTE ] PGP: Full fetch complete (7927 records). [2025-10-20T20:34:36.903393] [DEBUG] [VALIDATOR ] PGP: Validating primary data. [2025-10-20T20:34:37.056689] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-02. [2025-10-20T20:34:37.268750] [DEBUG] [VALIDATOR ] PGP: All expected trading days are present. [2025-10-20T20:34:37.268830] [DEBUG] [VALIDATOR ] PGP: Primary data validation passed. [2025-10-20T20:34:37.269041] [INFO ] [MAIN ] PGP: Fetched primary data is valid. [2025-10-20T20:34:37.269079] [DEBUG] [LOCAL ] PGP: Saving primary data. [2025-10-20T20:34:37.269464] [DEBUG] [LOCAL ] PGP: Trimmed 7927 records to 5690 (keeping 200 days of data) [2025-10-20T20:34:37.301836] [DEBUG] [LOCAL ] PGP: Primary data saved (5690 records). [2025-10-20T20:34:37.301907] [INFO ] [MAIN ] PGP: Full history saved. [2025-10-20T20:34:37.301975] [INFO ] [MAIN ] Update Progress: 1668/2367 symbols completed. PGP took 20.9s. Estimated remaining: 1h 29m 55s. Rate: 7.8 symbols/min [2025-10-20T20:34:37.471910] [INFO ] [MAIN ] PGRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:37.514525] [DEBUG] [LOCAL ] PGRE: Loaded 47694 existing records from primary storage. [2025-10-20T20:34:37.514895] [INFO ] [MAIN ] PGRE: Loaded 47694 existing records. [2025-10-20T20:34:37.694461] [WARN ] [MAIN ] PGRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:34:37.694675] [INFO ] [MAIN ] PGRE: Wiping existing primary data before full refresh. [2025-10-20T20:34:37.694732] [INFO ] [LOCAL ] PGRE: Starting deletion of primary data. [2025-10-20T20:34:37.694922] [DEBUG] [LOCAL ] PGRE: Deleted primary data file. [2025-10-20T20:34:37.694960] [INFO ] [MAIN ] PGRE: Starting full primary refresh. [2025-10-20T20:34:37.870866] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-09. [2025-10-20T20:34:38.058508] [INFO ] [REMOTE ] PGRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:38.651429] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-07. [2025-10-20T20:34:39.038984] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T20:34:39.439173] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T20:34:39.451320] [INFO ] [REMOTE ] PFSI: Fetching primary data for 2025-10. [2025-10-20T20:34:39.545341] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T20:34:40.251429] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-01. [2025-10-20T20:34:40.675343] [INFO ] [REMOTE ] PFSI: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T20:34:40.680220] [INFO ] [REMOTE ] PFSI: Full fetch complete (41592 records). [2025-10-20T20:34:40.680350] [DEBUG] [VALIDATOR ] PFSI: Validating primary data. [2025-10-20T20:34:41.059665] [DEBUG] [VALIDATOR ] PFSI: All expected trading days are present. [2025-10-20T20:34:41.057348] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-08. [2025-10-20T20:34:41.059816] [DEBUG] [VALIDATOR ] PFSI: Primary data validation passed. [2025-10-20T20:34:41.060578] [INFO ] [MAIN ] PFSI: Fetched primary data is valid. [2025-10-20T20:34:41.060668] [DEBUG] [LOCAL ] PFSI: Saving primary data. [2025-10-20T20:34:41.062233] [DEBUG] [LOCAL ] PFSI: Trimmed 41592 records to 29172 (keeping 200 days of data) [2025-10-20T20:34:41.237455] [DEBUG] [LOCAL ] PFSI: Primary data saved (29172 records). [2025-10-20T20:34:41.237577] [INFO ] [MAIN ] PFSI: Full history saved. [2025-10-20T20:34:41.237683] [INFO ] [MAIN ] Update Progress: 1669/2367 symbols completed. PFSI took 41.8s. Estimated remaining: 1h 29m 45s. Rate: 7.8 symbols/min [2025-10-20T20:34:41.410999] [INFO ] [MAIN ] PGZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:41.418282] [DEBUG] [LOCAL ] PGZ: Loaded 5636 existing records from primary storage. [2025-10-20T20:34:41.418437] [INFO ] [MAIN ] PGZ: Loaded 5636 existing records. [2025-10-20T20:34:41.621344] [WARN ] [MAIN ] PGZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:34:41.621610] [INFO ] [MAIN ] PGZ: Wiping existing primary data before full refresh. [2025-10-20T20:34:41.621687] [INFO ] [LOCAL ] PGZ: Starting deletion of primary data. [2025-10-20T20:34:41.621880] [DEBUG] [LOCAL ] PGZ: Deleted primary data file. [2025-10-20T20:34:41.621936] [INFO ] [MAIN ] PGZ: Starting full primary refresh. [2025-10-20T20:34:41.859596] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-03. [2025-10-20T20:34:41.973518] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T20:34:42.002898] [INFO ] [REMOTE ] PGZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:42.044699] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T20:34:42.651403] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-09. [2025-10-20T20:34:43.451404] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-02. [2025-10-20T20:34:43.496356] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T20:34:43.998727] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T20:34:44.251376] [INFO ] [REMOTE ] PG: Fetching primary data for 2025-10. [2025-10-20T20:34:44.809183] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T20:34:45.023111] [INFO ] [REMOTE ] PG: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:34:45.030840] [INFO ] [REMOTE ] PG: Full fetch complete (77538 records). [2025-10-20T20:34:45.031012] [DEBUG] [VALIDATOR ] PG: Validating primary data. [2025-10-20T20:34:45.031724] [FATAL] [VALIDATOR ] PG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:34:45.031812] [FATAL] [MAIN ] PG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:34:45.031855] [WARN ] [MAIN ] PG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:34:45.031894] [INFO ] [LOCAL ] PG: Starting deletion of primary data. [2025-10-20T20:34:45.031951] [DEBUG] [LOCAL ] PG: No primary data file found to delete. [2025-10-20T20:34:45.032003] [INFO ] [LOCAL ] PG: Starting deletion of derivative data. [2025-10-20T20:34:45.032125] [DEBUG] [LOCAL ] PG: Deleted derivative data file. [2025-10-20T20:34:45.032196] [INFO ] [MAIN ] Update Progress: 1670/2367 symbols completed. PG took 32.3s. Estimated remaining: 1h 29m 36s. Rate: 7.8 symbols/min [2025-10-20T20:34:45.061217] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-03. [2025-10-20T20:34:45.209019] [INFO ] [MAIN ] PH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:34:45.209247] [DEBUG] [LOCAL ] PH: No existing data file found in primary storage. [2025-10-20T20:34:45.209287] [WARN ] [MAIN ] PH: No existing data found, performing full refresh instead. [2025-10-20T20:34:45.209398] [INFO ] [MAIN ] PH: Wiping existing primary data before full refresh. [2025-10-20T20:34:45.209441] [INFO ] [LOCAL ] PH: Starting deletion of primary data. [2025-10-20T20:34:45.209483] [DEBUG] [LOCAL ] PH: No primary data file found to delete. [2025-10-20T20:34:45.209516] [INFO ] [MAIN ] PH: Starting full primary refresh. [2025-10-20T20:34:45.573816] [INFO ] [REMOTE ] PH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:34:45.851360] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-01. [2025-10-20T20:34:46.565849] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:34:46.651335] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-04. [2025-10-20T20:34:46.808208] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T20:34:47.451368] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-04. [2025-10-20T20:34:48.251375] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-02. [2025-10-20T20:34:48.827678] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T20:34:48.827840] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-02 (0.58 seconds). [2025-10-20T20:34:49.051361] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-05. [2025-10-20T20:34:49.134936] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:34:49.851393] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-03. [2025-10-20T20:34:50.509516] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:34:50.651386] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-01. [2025-10-20T20:34:51.101191] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T20:34:51.451394] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-04. [2025-10-20T20:34:52.111843] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T20:34:52.251378] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-06. [2025-10-20T20:34:52.476739] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-01 (1.83 seconds). [2025-10-20T20:34:53.051489] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-05. [2025-10-20T20:34:53.669180] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T20:34:53.851402] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-06. [2025-10-20T20:34:53.949887] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T20:34:54.651399] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-07. [2025-10-20T20:34:54.895685] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T20:34:55.451385] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-05. [2025-10-20T20:34:56.251428] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-07. [2025-10-20T20:34:56.280529] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T20:34:56.658060] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T20:34:56.952815] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T20:34:57.051426] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-08. [2025-10-20T20:34:57.730482] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T20:34:57.851387] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-06. [2025-10-20T20:34:58.651449] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-08. [2025-10-20T20:34:59.450630] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T20:34:59.459441] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-09. [2025-10-20T20:35:00.131232] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T20:35:00.251392] [INFO ] [REMOTE ] PGZ: Fetching primary data for 2025-10. [2025-10-20T20:35:00.644341] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T20:35:01.051446] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-02. [2025-10-20T20:35:01.206486] [INFO ] [REMOTE ] PGZ: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T20:35:01.208219] [INFO ] [REMOTE ] PGZ: Full fetch complete (8251 records). [2025-10-20T20:35:01.208367] [DEBUG] [VALIDATOR ] PGZ: Validating primary data. [2025-10-20T20:35:01.669767] [DEBUG] [VALIDATOR ] PGZ: All expected trading days are present. [2025-10-20T20:35:01.669859] [DEBUG] [VALIDATOR ] PGZ: Primary data validation passed. [2025-10-20T20:35:01.670129] [INFO ] [MAIN ] PGZ: Fetched primary data is valid. [2025-10-20T20:35:01.670170] [DEBUG] [LOCAL ] PGZ: Saving primary data. [2025-10-20T20:35:01.670882] [DEBUG] [LOCAL ] PGZ: Trimmed 8251 records to 5586 (keeping 200 days of data) [2025-10-20T20:35:01.709465] [DEBUG] [LOCAL ] PGZ: Primary data saved (5586 records). [2025-10-20T20:35:01.709636] [INFO ] [MAIN ] PGZ: Full history saved. [2025-10-20T20:35:01.709785] [INFO ] [MAIN ] Update Progress: 1671/2367 symbols completed. PGZ took 20.5s. Estimated remaining: 1h 29m 32s. Rate: 7.8 symbols/min [2025-10-20T20:35:01.856711] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-09. [2025-10-20T20:35:01.928700] [INFO ] [MAIN ] PHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:01.928898] [DEBUG] [LOCAL ] PHG: No existing data file found in primary storage. [2025-10-20T20:35:01.928963] [WARN ] [MAIN ] PHG: No existing data found, performing full refresh instead. [2025-10-20T20:35:01.929107] [INFO ] [MAIN ] PHG: Wiping existing primary data before full refresh. [2025-10-20T20:35:01.929157] [INFO ] [LOCAL ] PHG: Starting deletion of primary data. [2025-10-20T20:35:01.929235] [DEBUG] [LOCAL ] PHG: No primary data file found to delete. [2025-10-20T20:35:01.929297] [INFO ] [MAIN ] PHG: Starting full primary refresh. [2025-10-20T20:35:02.340784] [INFO ] [REMOTE ] PHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:02.651416] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-07. [2025-10-20T20:35:02.955111] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T20:35:03.451394] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-01. [2025-10-20T20:35:03.589824] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T20:35:04.251453] [INFO ] [REMOTE ] PGR: Fetching primary data for 2025-10. [2025-10-20T20:35:04.461713] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T20:35:04.651279] [INFO ] [REMOTE ] PGR: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T20:35:04.662108] [INFO ] [REMOTE ] PGR: Full fetch complete (76372 records). [2025-10-20T20:35:04.662308] [DEBUG] [VALIDATOR ] PGR: Validating primary data. [2025-10-20T20:35:04.663354] [FATAL] [VALIDATOR ] PGR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:35:04.663462] [FATAL] [MAIN ] PGR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:35:04.663521] [WARN ] [MAIN ] PGR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:35:04.663582] [INFO ] [LOCAL ] PGR: Starting deletion of primary data. [2025-10-20T20:35:04.663690] [DEBUG] [LOCAL ] PGR: No primary data file found to delete. [2025-10-20T20:35:04.663788] [INFO ] [LOCAL ] PGR: Starting deletion of derivative data. [2025-10-20T20:35:04.664011] [DEBUG] [LOCAL ] PGR: Deleted derivative data file. [2025-10-20T20:35:04.664517] [INFO ] [MAIN ] Update Progress: 1672/2367 symbols completed. PGR took 36.5s. Estimated remaining: 1h 29m 22s. Rate: 7.8 symbols/min [2025-10-20T20:35:04.833327] [INFO ] [MAIN ] PHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:04.833533] [DEBUG] [LOCAL ] PHI: No existing data file found in primary storage. [2025-10-20T20:35:04.833572] [WARN ] [MAIN ] PHI: No existing data found, performing full refresh instead. [2025-10-20T20:35:04.833704] [INFO ] [MAIN ] PHI: Wiping existing primary data before full refresh. [2025-10-20T20:35:04.833754] [INFO ] [LOCAL ] PHI: Starting deletion of primary data. [2025-10-20T20:35:04.833795] [DEBUG] [LOCAL ] PHI: No primary data file found to delete. [2025-10-20T20:35:04.833826] [INFO ] [MAIN ] PHI: Starting full primary refresh. [2025-10-20T20:35:05.081704] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-03. [2025-10-20T20:35:05.223598] [INFO ] [REMOTE ] PHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:05.225687] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:35:05.851425] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-01. [2025-10-20T20:35:06.651467] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-02. [2025-10-20T20:35:06.876550] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:35:07.108220] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T20:35:07.451405] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-02. [2025-10-20T20:35:07.995323] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:35:08.251351] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-08. [2025-10-20T20:35:08.340716] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T20:35:09.051441] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-03. [2025-10-20T20:35:09.851369] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-03. [2025-10-20T20:35:09.983811] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T20:35:10.170030] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T20:35:10.651419] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-04. [2025-10-20T20:35:11.431201] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:35:11.451338] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-04. [2025-10-20T20:35:11.477346] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:35:12.251414] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-05. [2025-10-20T20:35:13.051357] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-09. [2025-10-20T20:35:13.116226] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T20:35:13.180179] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:35:13.851390] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-04. [2025-10-20T20:35:14.651526] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T20:35:14.651729] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-06. [2025-10-20T20:35:15.451352] [INFO ] [REMOTE ] PGRE: Fetching primary data for 2025-10. [2025-10-20T20:35:15.683088] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T20:35:15.721111] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T20:35:15.884396] [INFO ] [REMOTE ] PGRE: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T20:35:15.890740] [INFO ] [REMOTE ] PGRE: Full fetch complete (64267 records). [2025-10-20T20:35:15.890841] [DEBUG] [VALIDATOR ] PGRE: Validating primary data. [2025-10-20T20:35:15.891505] [FATAL] [VALIDATOR ] PGRE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:35:15.891586] [FATAL] [MAIN ] PGRE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:35:15.891628] [WARN ] [MAIN ] PGRE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:35:15.891708] [INFO ] [LOCAL ] PGRE: Starting deletion of primary data. [2025-10-20T20:35:15.891774] [DEBUG] [LOCAL ] PGRE: No primary data file found to delete. [2025-10-20T20:35:15.891808] [INFO ] [LOCAL ] PGRE: Starting deletion of derivative data. [2025-10-20T20:35:15.891950] [DEBUG] [LOCAL ] PGRE: Deleted derivative data file. [2025-10-20T20:35:15.892036] [INFO ] [MAIN ] Update Progress: 1673/2367 symbols completed. PGRE took 38.6s. Estimated remaining: 1h 29m 16s. Rate: 7.8 symbols/min [2025-10-20T20:35:16.058806] [INFO ] [MAIN ] PHIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:16.087705] [DEBUG] [LOCAL ] PHIN: Loaded 32358 existing records from primary storage. [2025-10-20T20:35:16.087939] [INFO ] [MAIN ] PHIN: Loaded 32358 existing records. [2025-10-20T20:35:16.256622] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-07. [2025-10-20T20:35:16.261383] [WARN ] [MAIN ] PHIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:35:16.262300] [INFO ] [MAIN ] PHIN: Wiping existing primary data before full refresh. [2025-10-20T20:35:16.262347] [INFO ] [LOCAL ] PHIN: Starting deletion of primary data. [2025-10-20T20:35:16.262616] [DEBUG] [LOCAL ] PHIN: Deleted primary data file. [2025-10-20T20:35:16.262712] [INFO ] [MAIN ] PHIN: Starting full primary refresh. [2025-10-20T20:35:16.626431] [INFO ] [REMOTE ] PHIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:17.051350] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-01. [2025-10-20T20:35:17.177956] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T20:35:17.851382] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-05. [2025-10-20T20:35:18.334897] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T20:35:18.651394] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-05. [2025-10-20T20:35:19.451390] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-02. [2025-10-20T20:35:19.587305] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T20:35:20.251423] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-08. [2025-10-20T20:35:20.337197] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:35:20.877406] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:35:20.894036] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T20:35:21.051360] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-09. [2025-10-20T20:35:21.851377] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-03. [2025-10-20T20:35:22.298270] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T20:35:22.651422] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-06. [2025-10-20T20:35:23.440903] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T20:35:23.451328] [INFO ] [REMOTE ] PHI: Fetching primary data for 2025-10. [2025-10-20T20:35:24.251346] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-06. [2025-10-20T20:35:24.332722] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T20:35:24.385134] [INFO ] [REMOTE ] PHI: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T20:35:24.388206] [INFO ] [REMOTE ] PHI: Full fetch complete (21248 records). [2025-10-20T20:35:24.388291] [DEBUG] [VALIDATOR ] PHI: Validating primary data. [2025-10-20T20:35:24.751785] [DEBUG] [VALIDATOR ] PHI: All expected trading days are present. [2025-10-20T20:35:24.751858] [DEBUG] [VALIDATOR ] PHI: Primary data validation passed. [2025-10-20T20:35:24.752280] [INFO ] [MAIN ] PHI: Fetched primary data is valid. [2025-10-20T20:35:24.752318] [DEBUG] [LOCAL ] PHI: Saving primary data. [2025-10-20T20:35:24.752912] [DEBUG] [LOCAL ] PHI: Trimmed 21248 records to 15165 (keeping 200 days of data) [2025-10-20T20:35:24.843541] [DEBUG] [LOCAL ] PHI: Primary data saved (15165 records). [2025-10-20T20:35:24.843704] [INFO ] [MAIN ] PHI: Full history saved. [2025-10-20T20:35:24.843785] [INFO ] [MAIN ] Update Progress: 1674/2367 symbols completed. PHI took 20.2s. Estimated remaining: 1h 29m 9s. Rate: 7.8 symbols/min [2025-10-20T20:35:25.020403] [INFO ] [MAIN ] PHK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:25.044522] [DEBUG] [LOCAL ] PHK: Loaded 27189 existing records from primary storage. [2025-10-20T20:35:25.044876] [INFO ] [MAIN ] PHK: Loaded 27189 existing records. [2025-10-20T20:35:25.057471] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-04. [2025-10-20T20:35:25.237144] [WARN ] [MAIN ] PHK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:35:25.237352] [INFO ] [MAIN ] PHK: Wiping existing primary data before full refresh. [2025-10-20T20:35:25.237400] [INFO ] [LOCAL ] PHK: Starting deletion of primary data. [2025-10-20T20:35:25.237768] [DEBUG] [LOCAL ] PHK: Deleted primary data file. [2025-10-20T20:35:25.237825] [INFO ] [MAIN ] PHK: Starting full primary refresh. [2025-10-20T20:35:25.608060] [INFO ] [REMOTE ] PHK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:25.851356] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-01. [2025-10-20T20:35:25.922790] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T20:35:26.656498] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-07. [2025-10-20T20:35:26.671437] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T20:35:27.451405] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-05. [2025-10-20T20:35:27.553854] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:35:28.251341] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-02. [2025-10-20T20:35:28.441745] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T20:35:28.922449] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T20:35:29.051371] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-06. [2025-10-20T20:35:29.169631] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:35:29.851379] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-03. [2025-10-20T20:35:30.449536] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T20:35:30.651407] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-08. [2025-10-20T20:35:30.991720] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T20:35:31.451438] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-04. [2025-10-20T20:35:32.251421] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-07. [2025-10-20T20:35:32.338727] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T20:35:32.854457] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T20:35:33.051444] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-07. [2025-10-20T20:35:33.657585] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T20:35:33.851423] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-09. [2025-10-20T20:35:34.406534] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T20:35:34.651368] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-05. [2025-10-20T20:35:35.436564] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T20:35:35.451333] [INFO ] [REMOTE ] PHG: Fetching primary data for 2025-10. [2025-10-20T20:35:35.539885] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:35:36.251370] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-08. [2025-10-20T20:35:37.047968] [INFO ] [REMOTE ] PHG: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T20:35:37.053785] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-08. [2025-10-20T20:35:37.055087] [INFO ] [REMOTE ] PHG: Full fetch complete (65707 records). [2025-10-20T20:35:37.055161] [DEBUG] [VALIDATOR ] PHG: Validating primary data. [2025-10-20T20:35:37.510886] [DEBUG] [VALIDATOR ] PHG: All expected trading days are present. [2025-10-20T20:35:37.511019] [DEBUG] [VALIDATOR ] PHG: Primary data validation passed. [2025-10-20T20:35:37.512211] [INFO ] [MAIN ] PHG: Fetched primary data is valid. [2025-10-20T20:35:37.512335] [DEBUG] [LOCAL ] PHG: Saving primary data. [2025-10-20T20:35:37.514009] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T20:35:37.521790] [DEBUG] [LOCAL ] PHG: Trimmed 65707 records to 45631 (keeping 200 days of data) [2025-10-20T20:35:37.808865] [DEBUG] [LOCAL ] PHG: Primary data saved (45631 records). [2025-10-20T20:35:37.809014] [INFO ] [MAIN ] PHG: Full history saved. [2025-10-20T20:35:37.809085] [INFO ] [MAIN ] Update Progress: 1675/2367 symbols completed. PHG took 36.1s. Estimated remaining: 1h 29m 3s. Rate: 7.8 symbols/min [2025-10-20T20:35:37.863820] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-06. [2025-10-20T20:35:37.981059] [INFO ] [MAIN ] PHM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:38.030181] [DEBUG] [LOCAL ] PHM: Loaded 52425 existing records from primary storage. [2025-10-20T20:35:38.030483] [INFO ] [MAIN ] PHM: Loaded 52425 existing records. [2025-10-20T20:35:38.202798] [WARN ] [MAIN ] PHM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:35:38.202997] [INFO ] [MAIN ] PHM: Wiping existing primary data before full refresh. [2025-10-20T20:35:38.203042] [INFO ] [LOCAL ] PHM: Starting deletion of primary data. [2025-10-20T20:35:38.203285] [DEBUG] [LOCAL ] PHM: Deleted primary data file. [2025-10-20T20:35:38.203322] [INFO ] [MAIN ] PHM: Starting full primary refresh. [2025-10-20T20:35:38.558471] [INFO ] [REMOTE ] PHM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:38.623488] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T20:35:38.651347] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-01. [2025-10-20T20:35:38.993286] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:35:39.451385] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-09. [2025-10-20T20:35:40.251347] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-09. [2025-10-20T20:35:40.753751] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-01 (2.10 seconds). [2025-10-20T20:35:40.836747] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T20:35:41.051419] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-07. [2025-10-20T20:35:41.615803] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T20:35:41.851428] [INFO ] [REMOTE ] PHIN: Fetching primary data for 2025-10. [2025-10-20T20:35:42.266102] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T20:35:42.506509] [INFO ] [REMOTE ] PHIN: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:35:42.510832] [INFO ] [REMOTE ] PHIN: Full fetch complete (47411 records). [2025-10-20T20:35:42.510909] [DEBUG] [VALIDATOR ] PHIN: Validating primary data. [2025-10-20T20:35:42.511470] [FATAL] [VALIDATOR ] PHIN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:35:42.511554] [FATAL] [MAIN ] PHIN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:35:42.511596] [WARN ] [MAIN ] PHIN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:35:42.511632] [INFO ] [LOCAL ] PHIN: Starting deletion of primary data. [2025-10-20T20:35:42.511724] [DEBUG] [LOCAL ] PHIN: No primary data file found to delete. [2025-10-20T20:35:42.511768] [INFO ] [LOCAL ] PHIN: Starting deletion of derivative data. [2025-10-20T20:35:42.511926] [DEBUG] [LOCAL ] PHIN: Deleted derivative data file. [2025-10-20T20:35:42.512008] [INFO ] [MAIN ] Update Progress: 1676/2367 symbols completed. PHIN took 26.6s. Estimated remaining: 1h 28m 54s. Rate: 7.8 symbols/min [2025-10-20T20:35:42.657800] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-08. [2025-10-20T20:35:42.683540] [INFO ] [MAIN ] PHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:35:42.718678] [DEBUG] [LOCAL ] PHR: Loaded 38391 existing records from primary storage. [2025-10-20T20:35:42.718972] [INFO ] [MAIN ] PHR: Loaded 38391 existing records. [2025-10-20T20:35:42.891680] [WARN ] [MAIN ] PHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:35:42.891848] [INFO ] [MAIN ] PHR: Wiping existing primary data before full refresh. [2025-10-20T20:35:42.891893] [INFO ] [LOCAL ] PHR: Starting deletion of primary data. [2025-10-20T20:35:42.892180] [DEBUG] [LOCAL ] PHR: Deleted primary data file. [2025-10-20T20:35:42.892217] [INFO ] [MAIN ] PHR: Starting full primary refresh. [2025-10-20T20:35:43.238029] [INFO ] [REMOTE ] PHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:35:43.451381] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-02. [2025-10-20T20:35:44.018809] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T20:35:44.251439] [INFO ] [REMOTE ] PH: Fetching primary data for 2025-10. [2025-10-20T20:35:44.913232] [INFO ] [REMOTE ] PH: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:35:44.917751] [INFO ] [REMOTE ] PH: Full fetch complete (55727 records). [2025-10-20T20:35:44.917876] [DEBUG] [VALIDATOR ] PH: Validating primary data. [2025-10-20T20:35:44.918569] [FATAL] [VALIDATOR ] PH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:35:44.918701] [FATAL] [MAIN ] PH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:35:44.918750] [WARN ] [MAIN ] PH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:35:44.918787] [INFO ] [LOCAL ] PH: Starting deletion of primary data. [2025-10-20T20:35:44.918844] [DEBUG] [LOCAL ] PH: No primary data file found to delete. [2025-10-20T20:35:44.918877] [INFO ] [LOCAL ] PH: Starting deletion of derivative data. [2025-10-20T20:35:44.918915] [DEBUG] [LOCAL ] PH: No derivative data file found to delete. [2025-10-20T20:35:44.919008] [INFO ] [MAIN ] Update Progress: 1677/2367 symbols completed. PH took 59.9s. Estimated remaining: 1h 28m 44s. Rate: 7.8 symbols/min [2025-10-20T20:35:45.051416] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-01. [2025-10-20T20:35:45.208259] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T20:35:45.851414] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-03. [2025-10-20T20:35:46.617993] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T20:35:46.651416] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-09. [2025-10-20T20:35:47.451418] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-02. [2025-10-20T20:35:47.724381] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T20:35:47.770494] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T20:35:48.251442] [INFO ] [REMOTE ] PHK: Fetching primary data for 2025-10. [2025-10-20T20:35:48.842210] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T20:35:49.051399] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-03. [2025-10-20T20:35:49.204793] [INFO ] [REMOTE ] PHK: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T20:35:49.209789] [INFO ] [REMOTE ] PHK: Full fetch complete (39500 records). [2025-10-20T20:35:49.209887] [DEBUG] [VALIDATOR ] PHK: Validating primary data. [2025-10-20T20:35:49.604400] [DEBUG] [VALIDATOR ] PHK: All expected trading days are present. [2025-10-20T20:35:49.604475] [DEBUG] [VALIDATOR ] PHK: Primary data validation passed. [2025-10-20T20:35:49.605225] [INFO ] [MAIN ] PHK: Fetched primary data is valid. [2025-10-20T20:35:49.605286] [DEBUG] [LOCAL ] PHK: Saving primary data. [2025-10-20T20:35:49.606126] [DEBUG] [LOCAL ] PHK: Trimmed 39500 records to 26939 (keeping 200 days of data) [2025-10-20T20:35:49.760431] [DEBUG] [LOCAL ] PHK: Primary data saved (26939 records). [2025-10-20T20:35:49.760583] [INFO ] [MAIN ] PHK: Full history saved. [2025-10-20T20:35:49.760694] [INFO ] [MAIN ] Update Progress: 1678/2367 symbols completed. PHK took 24.9s. Estimated remaining: 1h 28m 35s. Rate: 7.8 symbols/min [2025-10-20T20:35:49.851402] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-04. [2025-10-20T20:35:50.673162] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T20:35:50.681449] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-04. [2025-10-20T20:35:51.749883] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T20:35:51.761021] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-05. [2025-10-20T20:35:51.991838] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T20:35:52.251430] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-05. [2025-10-20T20:35:53.380709] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T20:35:53.388848] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-06. [2025-10-20T20:35:53.472131] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T20:35:53.851381] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-06. [2025-10-20T20:35:54.704450] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-06 (1.32 seconds). [2025-10-20T20:35:54.716984] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-07. [2025-10-20T20:35:55.676144] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:35:55.687994] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-07. [2025-10-20T20:35:56.148505] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T20:35:56.251423] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-08. [2025-10-20T20:35:57.798440] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-07 (2.11 seconds). [2025-10-20T20:35:57.810387] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-08. [2025-10-20T20:35:57.950961] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:35:57.959774] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-09. [2025-10-20T20:35:59.526461] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T20:35:59.538121] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-09. [2025-10-20T20:35:59.634633] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T20:35:59.646916] [INFO ] [REMOTE ] PHR: Fetching primary data for 2025-10. [2025-10-20T20:36:01.370479] [INFO ] [REMOTE ] PHR: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T20:36:01.378844] [INFO ] [REMOTE ] PHR: Full fetch complete (53413 records). [2025-10-20T20:36:01.378986] [DEBUG] [VALIDATOR ] PHR: Validating primary data. [2025-10-20T20:36:01.815833] [DEBUG] [VALIDATOR ] PHR: All expected trading days are present. [2025-10-20T20:36:01.816240] [DEBUG] [VALIDATOR ] PHR: Primary data validation passed. [2025-10-20T20:36:01.817982] [INFO ] [MAIN ] PHR: Fetched primary data is valid. [2025-10-20T20:36:01.818130] [DEBUG] [LOCAL ] PHR: Saving primary data. [2025-10-20T20:36:01.836169] [DEBUG] [LOCAL ] PHR: Trimmed 53413 records to 37955 (keeping 200 days of data) [2025-10-20T20:36:01.853876] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-09 (2.32 seconds). [2025-10-20T20:36:01.972154] [INFO ] [REMOTE ] PHM: Fetching primary data for 2025-10. [2025-10-20T20:36:02.114343] [DEBUG] [LOCAL ] PHR: Primary data saved (37955 records). [2025-10-20T20:36:02.114493] [INFO ] [MAIN ] PHR: Full history saved. [2025-10-20T20:36:02.114571] [INFO ] [MAIN ] Update Progress: 1679/2367 symbols completed. PHR took 19.6s. Estimated remaining: 1h 28m 30s. Rate: 7.8 symbols/min [2025-10-20T20:36:02.776581] [INFO ] [REMOTE ] PHM: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:36:02.784988] [INFO ] [REMOTE ] PHM: Full fetch complete (74816 records). [2025-10-20T20:36:02.785241] [DEBUG] [VALIDATOR ] PHM: Validating primary data. [2025-10-20T20:36:02.786128] [FATAL] [VALIDATOR ] PHM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:36:02.786254] [FATAL] [MAIN ] PHM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:36:02.786298] [WARN ] [MAIN ] PHM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:36:02.786334] [INFO ] [LOCAL ] PHM: Starting deletion of primary data. [2025-10-20T20:36:02.786394] [DEBUG] [LOCAL ] PHM: No primary data file found to delete. [2025-10-20T20:36:02.786432] [INFO ] [LOCAL ] PHM: Starting deletion of derivative data. [2025-10-20T20:36:02.786872] [DEBUG] [LOCAL ] PHM: Deleted derivative data file. [2025-10-20T20:36:02.786968] [INFO ] [MAIN ] Update Progress: 1680/2367 symbols completed. PHM took 25.0s. Estimated remaining: 1h 28m 19s. Rate: 7.8 symbols/min [2025-10-20T20:36:03.575238] [INFO ] [MAIN ] PII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:03.562456] [INFO ] [MAIN ] PIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:03.596159] [INFO ] [MAIN ] PINS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:03.577829] [INFO ] [MAIN ] PINE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:03.605390] [DEBUG] [LOCAL ] PIM: Loaded 4880 existing records from primary storage. [2025-10-20T20:36:03.606097] [INFO ] [MAIN ] PIM: Loaded 4880 existing records. [2025-10-20T20:36:03.686980] [DEBUG] [LOCAL ] PINE: Loaded 14582 existing records from primary storage. [2025-10-20T20:36:03.715748] [INFO ] [MAIN ] PINE: Loaded 14582 existing records. [2025-10-20T20:36:03.940050] [WARN ] [MAIN ] PIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:03.941541] [INFO ] [MAIN ] PIM: Wiping existing primary data before full refresh. [2025-10-20T20:36:03.942003] [INFO ] [LOCAL ] PIM: Starting deletion of primary data. [2025-10-20T20:36:03.958098] [DEBUG] [LOCAL ] PIM: Deleted primary data file. [2025-10-20T20:36:03.968992] [INFO ] [MAIN ] PIM: Starting full primary refresh. [2025-10-20T20:36:04.072865] [WARN ] [MAIN ] PINE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:04.100148] [DEBUG] [LOCAL ] PII: Loaded 49613 existing records from primary storage. [2025-10-20T20:36:04.150687] [DEBUG] [LOCAL ] PINS: Loaded 54473 existing records from primary storage. [2025-10-20T20:36:04.161674] [INFO ] [MAIN ] PINE: Wiping existing primary data before full refresh. [2025-10-20T20:36:04.161788] [INFO ] [MAIN ] PII: Loaded 49613 existing records. [2025-10-20T20:36:04.170823] [INFO ] [LOCAL ] PINE: Starting deletion of primary data. [2025-10-20T20:36:04.186095] [DEBUG] [LOCAL ] PINE: Deleted primary data file. [2025-10-20T20:36:04.186243] [INFO ] [MAIN ] PINS: Loaded 54473 existing records. [2025-10-20T20:36:04.191702] [INFO ] [MAIN ] PINE: Starting full primary refresh. [2025-10-20T20:36:04.873767] [WARN ] [MAIN ] PINS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:04.897438] [INFO ] [MAIN ] PINS: Wiping existing primary data before full refresh. [2025-10-20T20:36:04.915609] [INFO ] [LOCAL ] PINS: Starting deletion of primary data. [2025-10-20T20:36:04.956895] [WARN ] [MAIN ] PII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:05.013356] [DEBUG] [LOCAL ] PINS: Deleted primary data file. [2025-10-20T20:36:05.037995] [INFO ] [MAIN ] PINS: Starting full primary refresh. [2025-10-20T20:36:05.037896] [INFO ] [MAIN ] PII: Wiping existing primary data before full refresh. [2025-10-20T20:36:05.090065] [INFO ] [LOCAL ] PII: Starting deletion of primary data. [2025-10-20T20:36:05.084825] [INFO ] [REMOTE ] PIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:05.143259] [DEBUG] [LOCAL ] PII: Deleted primary data file. [2025-10-20T20:36:05.143411] [INFO ] [MAIN ] PII: Starting full primary refresh. [2025-10-20T20:36:05.175792] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-01. [2025-10-20T20:36:05.495563] [INFO ] [REMOTE ] PINE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:05.620451] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-01. [2025-10-20T20:36:06.012830] [INFO ] [REMOTE ] PINS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:06.018544] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-01. [2025-10-20T20:36:06.057827] [INFO ] [REMOTE ] PII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:06.058137] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-01. [2025-10-20T20:36:06.246353] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:36:06.248367] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-02. [2025-10-20T20:36:06.898348] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T20:36:06.905562] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-02. [2025-10-20T20:36:06.902777] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:36:06.907676] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-03. [2025-10-20T20:36:07.656016] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T20:36:07.659455] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-03. [2025-10-20T20:36:07.791544] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T20:36:07.793083] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-04. [2025-10-20T20:36:07.866787] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T20:36:08.131135] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T20:36:08.247612] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T20:36:08.251319] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-04. [2025-10-20T20:36:08.383435] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T20:36:09.051444] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-02. [2025-10-20T20:36:09.308153] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T20:36:09.851450] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-05. [2025-10-20T20:36:10.651348] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-05. [2025-10-20T20:36:10.777191] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T20:36:11.143165] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-10-20T20:36:11.354433] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-05 (0.70 seconds). [2025-10-20T20:36:11.451420] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-06. [2025-10-20T20:36:12.088112] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T20:36:12.251389] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-03. [2025-10-20T20:36:13.051364] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-07. [2025-10-20T20:36:13.765738] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T20:36:13.851389] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-08. [2025-10-20T20:36:14.243315] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T20:36:14.294264] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T20:36:14.651416] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-09. [2025-10-20T20:36:15.455094] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-04. [2025-10-20T20:36:15.457156] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T20:36:16.251366] [INFO ] [REMOTE ] PIM: Fetching primary data for 2025-10. [2025-10-20T20:36:16.842922] [INFO ] [REMOTE ] PIM: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T20:36:16.844085] [INFO ] [REMOTE ] PIM: Full fetch complete (6390 records). [2025-10-20T20:36:16.844159] [DEBUG] [VALIDATOR ] PIM: Validating primary data. [2025-10-20T20:36:17.052890] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-06. [2025-10-20T20:36:17.205710] [DEBUG] [VALIDATOR ] PIM: All expected trading days are present. [2025-10-20T20:36:17.205858] [DEBUG] [VALIDATOR ] PIM: Primary data validation passed. [2025-10-20T20:36:17.206067] [INFO ] [MAIN ] PIM: Fetched primary data is valid. [2025-10-20T20:36:17.206109] [DEBUG] [LOCAL ] PIM: Saving primary data. [2025-10-20T20:36:17.206480] [DEBUG] [LOCAL ] PIM: Trimmed 6390 records to 4832 (keeping 200 days of data) [2025-10-20T20:36:17.236538] [DEBUG] [LOCAL ] PIM: Primary data saved (4832 records). [2025-10-20T20:36:17.236618] [INFO ] [MAIN ] PIM: Full history saved. [2025-10-20T20:36:17.236722] [INFO ] [MAIN ] Update Progress: 1681/2367 symbols completed. PIM took 14.4s. Estimated remaining: 1h 28m 14s. Rate: 7.8 symbols/min [2025-10-20T20:36:17.405167] [INFO ] [MAIN ] PIPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:17.421106] [DEBUG] [LOCAL ] PIPR: Loaded 15399 existing records from primary storage. [2025-10-20T20:36:17.421309] [INFO ] [MAIN ] PIPR: Loaded 15399 existing records. [2025-10-20T20:36:17.598608] [WARN ] [MAIN ] PIPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:17.599056] [INFO ] [MAIN ] PIPR: Wiping existing primary data before full refresh. [2025-10-20T20:36:17.599135] [INFO ] [LOCAL ] PIPR: Starting deletion of primary data. [2025-10-20T20:36:17.599341] [DEBUG] [LOCAL ] PIPR: Deleted primary data file. [2025-10-20T20:36:17.599462] [INFO ] [MAIN ] PIPR: Starting full primary refresh. [2025-10-20T20:36:17.845294] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-04 (2.39 seconds). [2025-10-20T20:36:17.859308] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-02. [2025-10-20T20:36:18.021076] [INFO ] [REMOTE ] PIPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:18.046276] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T20:36:18.651442] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-01. [2025-10-20T20:36:19.451364] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-05. [2025-10-20T20:36:19.603783] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T20:36:19.846799] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T20:36:20.251390] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-02. [2025-10-20T20:36:21.051378] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-03. [2025-10-20T20:36:21.065236] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T20:36:21.753986] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-05 (2.30 seconds). [2025-10-20T20:36:21.851362] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-03. [2025-10-20T20:36:22.651348] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-06. [2025-10-20T20:36:22.791391] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T20:36:22.885612] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T20:36:23.451427] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-07. [2025-10-20T20:36:24.251391] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-04. [2025-10-20T20:36:24.581713] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T20:36:24.621453] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T20:36:25.051394] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-08. [2025-10-20T20:36:25.440587] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T20:36:25.851425] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-04. [2025-10-20T20:36:25.915493] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:36:26.651373] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-07. [2025-10-20T20:36:27.451415] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-05. [2025-10-20T20:36:27.695513] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T20:36:28.251411] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-09. [2025-10-20T20:36:28.339948] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:36:28.825246] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-10-20T20:36:29.051423] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-06. [2025-10-20T20:36:29.121199] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:36:29.851393] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-05. [2025-10-20T20:36:30.058960] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T20:36:30.651376] [INFO ] [REMOTE ] PINE: Fetching primary data for 2025-10. [2025-10-20T20:36:31.409783] [INFO ] [REMOTE ] PINE: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:36:31.412810] [INFO ] [REMOTE ] PINE: Full fetch complete (21046 records). [2025-10-20T20:36:31.412884] [DEBUG] [VALIDATOR ] PINE: Validating primary data. [2025-10-20T20:36:31.456488] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-07. [2025-10-20T20:36:31.798118] [DEBUG] [VALIDATOR ] PINE: All expected trading days are present. [2025-10-20T20:36:31.798304] [DEBUG] [VALIDATOR ] PINE: Primary data validation passed. [2025-10-20T20:36:31.799060] [INFO ] [MAIN ] PINE: Fetched primary data is valid. [2025-10-20T20:36:31.799135] [DEBUG] [LOCAL ] PINE: Saving primary data. [2025-10-20T20:36:31.800147] [DEBUG] [LOCAL ] PINE: Trimmed 21046 records to 14337 (keeping 200 days of data) [2025-10-20T20:36:31.891129] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T20:36:31.926783] [DEBUG] [LOCAL ] PINE: Primary data saved (14337 records). [2025-10-20T20:36:31.927265] [INFO ] [MAIN ] PINE: Full history saved. [2025-10-20T20:36:31.927358] [INFO ] [MAIN ] Update Progress: 1682/2367 symbols completed. PINE took 29.1s. Estimated remaining: 1h 28m 9s. Rate: 7.8 symbols/min [2025-10-20T20:36:32.099543] [INFO ] [MAIN ] PJT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:32.122506] [DEBUG] [LOCAL ] PJT: Loaded 21931 existing records from primary storage. [2025-10-20T20:36:32.122806] [INFO ] [MAIN ] PJT: Loaded 21931 existing records. [2025-10-20T20:36:32.256522] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-08. [2025-10-20T20:36:32.319408] [WARN ] [MAIN ] PJT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:32.319618] [INFO ] [MAIN ] PJT: Wiping existing primary data before full refresh. [2025-10-20T20:36:32.319699] [INFO ] [LOCAL ] PJT: Starting deletion of primary data. [2025-10-20T20:36:32.319895] [DEBUG] [LOCAL ] PJT: Deleted primary data file. [2025-10-20T20:36:32.319935] [INFO ] [MAIN ] PJT: Starting full primary refresh. [2025-10-20T20:36:32.672582] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T20:36:32.707221] [INFO ] [REMOTE ] PJT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:33.051384] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-08. [2025-10-20T20:36:33.851358] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-06. [2025-10-20T20:36:34.056434] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T20:36:34.323859] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T20:36:34.651355] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-09. [2025-10-20T20:36:35.455774] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-01. [2025-10-20T20:36:35.463019] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T20:36:35.560810] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T20:36:36.251385] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-07. [2025-10-20T20:36:36.641924] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T20:36:37.051428] [INFO ] [REMOTE ] PIPR: Fetching primary data for 2025-10. [2025-10-20T20:36:37.371872] [INFO ] [REMOTE ] PIPR: Incremental fetch complete for 2025-10 (0.32 seconds). [2025-10-20T20:36:37.374448] [INFO ] [REMOTE ] PIPR: Full fetch complete (20911 records). [2025-10-20T20:36:37.374540] [DEBUG] [VALIDATOR ] PIPR: Validating primary data. [2025-10-20T20:36:37.375015] [FATAL] [VALIDATOR ] PIPR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:36:37.375119] [FATAL] [MAIN ] PIPR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:36:37.375161] [WARN ] [MAIN ] PIPR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:36:37.375198] [INFO ] [LOCAL ] PIPR: Starting deletion of primary data. [2025-10-20T20:36:37.375252] [DEBUG] [LOCAL ] PIPR: No primary data file found to delete. [2025-10-20T20:36:37.375287] [INFO ] [LOCAL ] PIPR: Starting deletion of derivative data. [2025-10-20T20:36:37.375422] [DEBUG] [LOCAL ] PIPR: Deleted derivative data file. [2025-10-20T20:36:37.375506] [INFO ] [MAIN ] Update Progress: 1683/2367 symbols completed. PIPR took 20.1s. Estimated remaining: 1h 28m 1s. Rate: 7.8 symbols/min [2025-10-20T20:36:37.546775] [INFO ] [MAIN ] PK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:37.597430] [DEBUG] [LOCAL ] PK: Loaded 53536 existing records from primary storage. [2025-10-20T20:36:37.597754] [INFO ] [MAIN ] PK: Loaded 53536 existing records. [2025-10-20T20:36:37.778284] [WARN ] [MAIN ] PK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:37.778707] [INFO ] [MAIN ] PK: Wiping existing primary data before full refresh. [2025-10-20T20:36:37.778783] [INFO ] [LOCAL ] PK: Starting deletion of primary data. [2025-10-20T20:36:37.778982] [DEBUG] [LOCAL ] PK: Deleted primary data file. [2025-10-20T20:36:37.779021] [INFO ] [MAIN ] PK: Starting full primary refresh. [2025-10-20T20:36:37.856532] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-09. [2025-10-20T20:36:38.173316] [INFO ] [REMOTE ] PK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:38.208604] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T20:36:38.651446] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-01. [2025-10-20T20:36:39.451448] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-02. [2025-10-20T20:36:40.074968] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T20:36:40.251359] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-08. [2025-10-20T20:36:40.440809] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:36:40.679545] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T20:36:41.051419] [INFO ] [REMOTE ] PINS: Fetching primary data for 2025-10. [2025-10-20T20:36:41.423801] [INFO ] [REMOTE ] PINS: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T20:36:41.431110] [INFO ] [REMOTE ] PINS: Full fetch complete (77481 records). [2025-10-20T20:36:41.431258] [DEBUG] [VALIDATOR ] PINS: Validating primary data. [2025-10-20T20:36:41.432125] [FATAL] [VALIDATOR ] PINS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:36:41.432225] [FATAL] [MAIN ] PINS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:36:41.432267] [WARN ] [MAIN ] PINS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:36:41.432303] [INFO ] [LOCAL ] PINS: Starting deletion of primary data. [2025-10-20T20:36:41.432358] [DEBUG] [LOCAL ] PINS: No primary data file found to delete. [2025-10-20T20:36:41.432392] [INFO ] [LOCAL ] PINS: Starting deletion of derivative data. [2025-10-20T20:36:41.432578] [DEBUG] [LOCAL ] PINS: Deleted derivative data file. [2025-10-20T20:36:41.432709] [INFO ] [MAIN ] Update Progress: 1684/2367 symbols completed. PINS took 38.6s. Estimated remaining: 1h 27m 51s. Rate: 7.8 symbols/min [2025-10-20T20:36:41.597361] [INFO ] [MAIN ] PKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:41.613054] [DEBUG] [LOCAL ] PKE: Loaded 14335 existing records from primary storage. [2025-10-20T20:36:41.613313] [INFO ] [MAIN ] PKE: Loaded 14335 existing records. [2025-10-20T20:36:41.791629] [WARN ] [MAIN ] PKE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:41.791892] [INFO ] [MAIN ] PKE: Wiping existing primary data before full refresh. [2025-10-20T20:36:41.792065] [INFO ] [LOCAL ] PKE: Starting deletion of primary data. [2025-10-20T20:36:41.792315] [DEBUG] [LOCAL ] PKE: Deleted primary data file. [2025-10-20T20:36:41.792370] [INFO ] [MAIN ] PKE: Starting full primary refresh. [2025-10-20T20:36:41.855795] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-03. [2025-10-20T20:36:42.151178] [INFO ] [REMOTE ] PKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:42.178316] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T20:36:42.651379] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-02. [2025-10-20T20:36:43.323595] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T20:36:43.451374] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-04. [2025-10-20T20:36:44.020253] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T20:36:44.251422] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-03. [2025-10-20T20:36:44.515317] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T20:36:45.051431] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-05. [2025-10-20T20:36:45.832549] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:36:45.851324] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-04. [2025-10-20T20:36:46.327530] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T20:36:46.651362] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-09. [2025-10-20T20:36:47.451464] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-01. [2025-10-20T20:36:47.569549] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T20:36:48.057706] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T20:36:48.250294] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T20:36:48.252621] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-06. [2025-10-20T20:36:49.051385] [INFO ] [REMOTE ] PII: Fetching primary data for 2025-10. [2025-10-20T20:36:49.436559] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T20:36:49.811943] [INFO ] [REMOTE ] PII: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:36:49.818637] [INFO ] [REMOTE ] PII: Full fetch complete (71029 records). [2025-10-20T20:36:49.818829] [DEBUG] [VALIDATOR ] PII: Validating primary data. [2025-10-20T20:36:49.819667] [FATAL] [VALIDATOR ] PII: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:36:49.819761] [FATAL] [MAIN ] PII: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:36:49.819812] [WARN ] [MAIN ] PII: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:36:49.819860] [INFO ] [LOCAL ] PII: Starting deletion of primary data. [2025-10-20T20:36:49.819970] [DEBUG] [LOCAL ] PII: No primary data file found to delete. [2025-10-20T20:36:49.820030] [INFO ] [LOCAL ] PII: Starting deletion of derivative data. [2025-10-20T20:36:49.820192] [DEBUG] [LOCAL ] PII: Deleted derivative data file. [2025-10-20T20:36:49.820303] [INFO ] [MAIN ] Update Progress: 1685/2367 symbols completed. PII took 47.0s. Estimated remaining: 1h 27m 44s. Rate: 7.8 symbols/min [2025-10-20T20:36:49.856495] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-02. [2025-10-20T20:36:50.019865] [INFO ] [MAIN ] PKG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:36:50.060436] [DEBUG] [LOCAL ] PKG: Loaded 39212 existing records from primary storage. [2025-10-20T20:36:50.060835] [INFO ] [MAIN ] PKG: Loaded 39212 existing records. [2025-10-20T20:36:50.261357] [WARN ] [MAIN ] PKG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:36:50.261597] [INFO ] [MAIN ] PKG: Wiping existing primary data before full refresh. [2025-10-20T20:36:50.261696] [INFO ] [LOCAL ] PKG: Starting deletion of primary data. [2025-10-20T20:36:50.261971] [DEBUG] [LOCAL ] PKG: Deleted primary data file. [2025-10-20T20:36:50.262030] [INFO ] [MAIN ] PKG: Starting full primary refresh. [2025-10-20T20:36:50.611859] [INFO ] [REMOTE ] PKG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:36:50.651355] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-07. [2025-10-20T20:36:50.792224] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T20:36:51.451442] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-01. [2025-10-20T20:36:51.805448] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T20:36:52.251386] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-03. [2025-10-20T20:36:53.045632] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:36:53.051324] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-04. [2025-10-20T20:36:53.129102] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T20:36:53.851417] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-05. [2025-10-20T20:36:53.971030] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T20:36:54.651372] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-02. [2025-10-20T20:36:55.451437] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-08. [2025-10-20T20:36:55.817858] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T20:36:55.880813] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T20:36:56.251424] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-06. [2025-10-20T20:36:56.780784] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T20:36:57.051460] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-05. [2025-10-20T20:36:57.605027] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T20:36:57.851415] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-09. [2025-10-20T20:36:58.117474] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T20:36:58.651392] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-07. [2025-10-20T20:36:58.733596] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:36:59.451428] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-06. [2025-10-20T20:37:00.251408] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-03. [2025-10-20T20:37:00.281941] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T20:37:00.378213] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T20:37:01.051446] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-07. [2025-10-20T20:37:01.851387] [INFO ] [REMOTE ] PJT: Fetching primary data for 2025-10. [2025-10-20T20:37:01.984887] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:37:02.004754] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T20:37:02.285241] [INFO ] [REMOTE ] PJT: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T20:37:02.288553] [INFO ] [REMOTE ] PJT: Full fetch complete (32102 records). [2025-10-20T20:37:02.288633] [DEBUG] [VALIDATOR ] PJT: Validating primary data. [2025-10-20T20:37:02.289158] [FATAL] [VALIDATOR ] PJT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:37:02.289239] [FATAL] [MAIN ] PJT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:37:02.289304] [WARN ] [MAIN ] PJT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:37:02.289370] [INFO ] [LOCAL ] PJT: Starting deletion of primary data. [2025-10-20T20:37:02.289431] [DEBUG] [LOCAL ] PJT: No primary data file found to delete. [2025-10-20T20:37:02.289465] [INFO ] [LOCAL ] PJT: Starting deletion of derivative data. [2025-10-20T20:37:02.289578] [DEBUG] [LOCAL ] PJT: Deleted derivative data file. [2025-10-20T20:37:02.289666] [INFO ] [MAIN ] Update Progress: 1686/2367 symbols completed. PJT took 30.4s. Estimated remaining: 1h 27m 38s. Rate: 7.8 symbols/min [2025-10-20T20:37:02.455368] [INFO ] [MAIN ] PKST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:02.475161] [DEBUG] [LOCAL ] PKST: Loaded 18809 existing records from primary storage. [2025-10-20T20:37:02.475385] [INFO ] [MAIN ] PKST: Loaded 18809 existing records. [2025-10-20T20:37:02.650828] [WARN ] [MAIN ] PKST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:02.651021] [INFO ] [MAIN ] PKST: Wiping existing primary data before full refresh. [2025-10-20T20:37:02.651069] [INFO ] [LOCAL ] PKST: Starting deletion of primary data. [2025-10-20T20:37:02.651235] [DEBUG] [LOCAL ] PKST: Deleted primary data file. [2025-10-20T20:37:02.651272] [INFO ] [MAIN ] PKST: Starting full primary refresh. [2025-10-20T20:37:02.656754] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-08. [2025-10-20T20:37:03.000845] [INFO ] [REMOTE ] PKST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:03.451386] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-04. [2025-10-20T20:37:04.169873] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:37:04.251390] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-09. [2025-10-20T20:37:05.051379] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-01. [2025-10-20T20:37:05.300980] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T20:37:05.851394] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-05. [2025-10-20T20:37:05.993874] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T20:37:06.078790] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T20:37:06.651383] [INFO ] [REMOTE ] PK: Fetching primary data for 2025-10. [2025-10-20T20:37:07.147868] [INFO ] [REMOTE ] PK: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T20:37:07.155740] [INFO ] [REMOTE ] PK: Full fetch complete (76165 records). [2025-10-20T20:37:07.156156] [DEBUG] [VALIDATOR ] PK: Validating primary data. [2025-10-20T20:37:07.157115] [FATAL] [VALIDATOR ] PK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:37:07.157229] [FATAL] [MAIN ] PK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:37:07.157275] [WARN ] [MAIN ] PK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:37:07.157337] [INFO ] [LOCAL ] PK: Starting deletion of primary data. [2025-10-20T20:37:07.157401] [DEBUG] [LOCAL ] PK: No primary data file found to delete. [2025-10-20T20:37:07.157435] [INFO ] [LOCAL ] PK: Starting deletion of derivative data. [2025-10-20T20:37:07.157549] [DEBUG] [LOCAL ] PK: Deleted derivative data file. [2025-10-20T20:37:07.157912] [INFO ] [MAIN ] Update Progress: 1687/2367 symbols completed. PK took 29.8s. Estimated remaining: 1h 27m 29s. Rate: 7.8 symbols/min [2025-10-20T20:37:07.354700] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T20:37:07.356588] [INFO ] [MAIN ] PKX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:07.389521] [DEBUG] [LOCAL ] PKX: Loaded 23348 existing records from primary storage. [2025-10-20T20:37:07.389818] [INFO ] [MAIN ] PKX: Loaded 23348 existing records. [2025-10-20T20:37:07.456543] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-06. [2025-10-20T20:37:07.570520] [WARN ] [MAIN ] PKX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:07.570752] [INFO ] [MAIN ] PKX: Wiping existing primary data before full refresh. [2025-10-20T20:37:07.570803] [INFO ] [LOCAL ] PKX: Starting deletion of primary data. [2025-10-20T20:37:07.571041] [DEBUG] [LOCAL ] PKX: Deleted primary data file. [2025-10-20T20:37:07.571081] [INFO ] [MAIN ] PKX: Starting full primary refresh. [2025-10-20T20:37:07.946907] [INFO ] [REMOTE ] PKX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:08.251382] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-01. [2025-10-20T20:37:09.051372] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-08. [2025-10-20T20:37:09.082992] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T20:37:09.710615] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T20:37:09.782474] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T20:37:09.851380] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-02. [2025-10-20T20:37:10.651452] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-02. [2025-10-20T20:37:11.018541] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T20:37:11.451453] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-03. [2025-10-20T20:37:11.672509] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T20:37:12.251418] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-07. [2025-10-20T20:37:12.704118] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-03 (1.25 seconds). [2025-10-20T20:37:13.051408] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-04. [2025-10-20T20:37:13.851366] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-03. [2025-10-20T20:37:13.884524] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T20:37:14.313266] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T20:37:14.651401] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-08. [2025-10-20T20:37:15.455842] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-09. [2025-10-20T20:37:15.458327] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T20:37:16.251385] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-04. [2025-10-20T20:37:16.357969] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T20:37:16.610959] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T20:37:17.051375] [INFO ] [REMOTE ] PKE: Fetching primary data for 2025-10. [2025-10-20T20:37:17.392458] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T20:37:17.851427] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-05. [2025-10-20T20:37:17.906850] [INFO ] [REMOTE ] PKE: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T20:37:17.910436] [INFO ] [REMOTE ] PKE: Full fetch complete (18344 records). [2025-10-20T20:37:17.910547] [DEBUG] [VALIDATOR ] PKE: Validating primary data. [2025-10-20T20:37:18.298946] [DEBUG] [VALIDATOR ] PKE: All expected trading days are present. [2025-10-20T20:37:18.299025] [DEBUG] [VALIDATOR ] PKE: Primary data validation passed. [2025-10-20T20:37:18.299424] [INFO ] [MAIN ] PKE: Fetched primary data is valid. [2025-10-20T20:37:18.299463] [DEBUG] [LOCAL ] PKE: Saving primary data. [2025-10-20T20:37:18.300061] [DEBUG] [LOCAL ] PKE: Trimmed 18344 records to 14258 (keeping 200 days of data) [2025-10-20T20:37:18.383315] [DEBUG] [LOCAL ] PKE: Primary data saved (14258 records). [2025-10-20T20:37:18.383472] [INFO ] [MAIN ] PKE: Full history saved. [2025-10-20T20:37:18.383552] [INFO ] [MAIN ] Update Progress: 1688/2367 symbols completed. PKE took 37.0s. Estimated remaining: 1h 27m 23s. Rate: 7.8 symbols/min [2025-10-20T20:37:18.554696] [INFO ] [MAIN ] PL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:18.604418] [DEBUG] [LOCAL ] PL: Loaded 53962 existing records from primary storage. [2025-10-20T20:37:18.604806] [INFO ] [MAIN ] PL: Loaded 53962 existing records. [2025-10-20T20:37:18.658349] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-09. [2025-10-20T20:37:18.782881] [WARN ] [MAIN ] PL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:18.783087] [INFO ] [MAIN ] PL: Wiping existing primary data before full refresh. [2025-10-20T20:37:18.783132] [INFO ] [LOCAL ] PL: Starting deletion of primary data. [2025-10-20T20:37:18.783433] [DEBUG] [LOCAL ] PL: Deleted primary data file. [2025-10-20T20:37:18.783488] [INFO ] [MAIN ] PL: Starting full primary refresh. [2025-10-20T20:37:19.173167] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T20:37:19.179313] [INFO ] [REMOTE ] PL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:19.451423] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-06. [2025-10-20T20:37:20.217831] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T20:37:20.251333] [INFO ] [REMOTE ] PKG: Fetching primary data for 2025-10. [2025-10-20T20:37:20.621420] [INFO ] [REMOTE ] PKG: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T20:37:20.626489] [INFO ] [REMOTE ] PKG: Full fetch complete (57272 records). [2025-10-20T20:37:20.626600] [DEBUG] [VALIDATOR ] PKG: Validating primary data. [2025-10-20T20:37:20.627243] [FATAL] [VALIDATOR ] PKG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:37:20.627326] [FATAL] [MAIN ] PKG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:37:20.627397] [WARN ] [MAIN ] PKG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:37:20.627469] [INFO ] [LOCAL ] PKG: Starting deletion of primary data. [2025-10-20T20:37:20.627531] [DEBUG] [LOCAL ] PKG: No primary data file found to delete. [2025-10-20T20:37:20.627567] [INFO ] [LOCAL ] PKG: Starting deletion of derivative data. [2025-10-20T20:37:20.627756] [DEBUG] [LOCAL ] PKG: Deleted derivative data file. [2025-10-20T20:37:20.627871] [INFO ] [MAIN ] Update Progress: 1689/2367 symbols completed. PKG took 30.8s. Estimated remaining: 1h 27m 13s. Rate: 7.8 symbols/min [2025-10-20T20:37:20.823245] [INFO ] [MAIN ] PLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:20.819330] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T20:37:20.879148] [DEBUG] [LOCAL ] PLD: Loaded 54196 existing records from primary storage. [2025-10-20T20:37:20.879461] [INFO ] [MAIN ] PLD: Loaded 54196 existing records. [2025-10-20T20:37:21.057987] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-07. [2025-10-20T20:37:21.065114] [WARN ] [MAIN ] PLD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:21.065336] [INFO ] [MAIN ] PLD: Wiping existing primary data before full refresh. [2025-10-20T20:37:21.065388] [INFO ] [LOCAL ] PLD: Starting deletion of primary data. [2025-10-20T20:37:21.065716] [DEBUG] [LOCAL ] PLD: Deleted primary data file. [2025-10-20T20:37:21.065772] [INFO ] [MAIN ] PLD: Starting full primary refresh. [2025-10-20T20:37:21.426990] [INFO ] [REMOTE ] PLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:21.851387] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-01. [2025-10-20T20:37:22.137033] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T20:37:22.651384] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-05. [2025-10-20T20:37:23.451363] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-01. [2025-10-20T20:37:23.694554] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T20:37:23.863119] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T20:37:24.251404] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-02. [2025-10-20T20:37:25.051405] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-06. [2025-10-20T20:37:25.708489] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-01 (2.26 seconds). [2025-10-20T20:37:25.851351] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-02. [2025-10-20T20:37:26.024700] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T20:37:26.073099] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T20:37:26.651387] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-07. [2025-10-20T20:37:27.451361] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-08. [2025-10-20T20:37:27.681207] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T20:37:27.862917] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T20:37:28.251371] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-03. [2025-10-20T20:37:28.565060] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T20:37:29.051428] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-03. [2025-10-20T20:37:29.851429] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-08. [2025-10-20T20:37:30.160531] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T20:37:30.651390] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-04. [2025-10-20T20:37:30.951546] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T20:37:31.007260] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T20:37:31.451406] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-09. [2025-10-20T20:37:32.251390] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-04. [2025-10-20T20:37:32.660800] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T20:37:32.790088] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T20:37:33.051384] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-09. [2025-10-20T20:37:33.851388] [INFO ] [REMOTE ] PKX: Fetching primary data for 2025-10. [2025-10-20T20:37:33.980945] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:37:34.009887] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T20:37:34.651347] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-05. [2025-10-20T20:37:34.892593] [INFO ] [REMOTE ] PKX: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T20:37:34.896274] [INFO ] [REMOTE ] PKX: Full fetch complete (36134 records). [2025-10-20T20:37:34.896358] [DEBUG] [VALIDATOR ] PKX: Validating primary data. [2025-10-20T20:37:35.300853] [DEBUG] [VALIDATOR ] PKX: All expected trading days are present. [2025-10-20T20:37:35.300929] [DEBUG] [VALIDATOR ] PKX: Primary data validation passed. [2025-10-20T20:37:35.301794] [INFO ] [MAIN ] PKX: Fetched primary data is valid. [2025-10-20T20:37:35.301854] [DEBUG] [LOCAL ] PKX: Saving primary data. [2025-10-20T20:37:35.302590] [DEBUG] [LOCAL ] PKX: Trimmed 36134 records to 22849 (keeping 200 days of data) [2025-10-20T20:37:35.437342] [DEBUG] [LOCAL ] PKX: Primary data saved (22849 records). [2025-10-20T20:37:35.437505] [INFO ] [MAIN ] PKX: Full history saved. [2025-10-20T20:37:35.437606] [INFO ] [MAIN ] Update Progress: 1690/2367 symbols completed. PKX took 28.3s. Estimated remaining: 1h 27m 8s. Rate: 7.8 symbols/min [2025-10-20T20:37:35.458700] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-05. [2025-10-20T20:37:35.609432] [INFO ] [MAIN ] PLNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:35.654565] [DEBUG] [LOCAL ] PLNT: Loaded 48907 existing records from primary storage. [2025-10-20T20:37:35.654880] [INFO ] [MAIN ] PLNT: Loaded 48907 existing records. [2025-10-20T20:37:35.829782] [WARN ] [MAIN ] PLNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:35.829976] [INFO ] [MAIN ] PLNT: Wiping existing primary data before full refresh. [2025-10-20T20:37:35.830022] [INFO ] [LOCAL ] PLNT: Starting deletion of primary data. [2025-10-20T20:37:35.830331] [DEBUG] [LOCAL ] PLNT: Deleted primary data file. [2025-10-20T20:37:35.830369] [INFO ] [MAIN ] PLNT: Starting full primary refresh. [2025-10-20T20:37:36.182290] [INFO ] [REMOTE ] PLNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:36.251345] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-01. [2025-10-20T20:37:36.639292] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T20:37:37.051415] [INFO ] [REMOTE ] PKST: Fetching primary data for 2025-10. [2025-10-20T20:37:37.457458] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T20:37:37.851360] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-06. [2025-10-20T20:37:38.060927] [INFO ] [REMOTE ] PKST: Incremental fetch complete for 2025-10 (1.01 seconds). [2025-10-20T20:37:38.064353] [INFO ] [REMOTE ] PKST: Full fetch complete (28921 records). [2025-10-20T20:37:38.064479] [DEBUG] [VALIDATOR ] PKST: Validating primary data. [2025-10-20T20:37:38.478368] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-01 (2.23 seconds). [2025-10-20T20:37:38.500109] [DEBUG] [VALIDATOR ] PKST: All expected trading days are present. [2025-10-20T20:37:38.500186] [DEBUG] [VALIDATOR ] PKST: Primary data validation passed. [2025-10-20T20:37:38.500730] [INFO ] [MAIN ] PKST: Fetched primary data is valid. [2025-10-20T20:37:38.500790] [DEBUG] [LOCAL ] PKST: Saving primary data. [2025-10-20T20:37:38.501467] [DEBUG] [LOCAL ] PKST: Trimmed 28921 records to 18304 (keeping 200 days of data) [2025-10-20T20:37:38.607274] [DEBUG] [LOCAL ] PKST: Primary data saved (18304 records). [2025-10-20T20:37:38.607402] [INFO ] [MAIN ] PKST: Full history saved. [2025-10-20T20:37:38.607476] [INFO ] [MAIN ] Update Progress: 1691/2367 symbols completed. PKST took 36.3s. Estimated remaining: 1h 26m 59s. Rate: 7.8 symbols/min [2025-10-20T20:37:38.661329] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-06. [2025-10-20T20:37:38.780820] [INFO ] [MAIN ] PLOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:38.799908] [DEBUG] [LOCAL ] PLOW: Loaded 20316 existing records from primary storage. [2025-10-20T20:37:38.800151] [INFO ] [MAIN ] PLOW: Loaded 20316 existing records. [2025-10-20T20:37:38.974636] [WARN ] [MAIN ] PLOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:38.974895] [INFO ] [MAIN ] PLOW: Wiping existing primary data before full refresh. [2025-10-20T20:37:38.974955] [INFO ] [LOCAL ] PLOW: Starting deletion of primary data. [2025-10-20T20:37:38.975230] [DEBUG] [LOCAL ] PLOW: Deleted primary data file. [2025-10-20T20:37:38.975287] [INFO ] [MAIN ] PLOW: Starting full primary refresh. [2025-10-20T20:37:39.351039] [INFO ] [REMOTE ] PLOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:39.451345] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-02. [2025-10-20T20:37:39.725317] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T20:37:40.251420] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-01. [2025-10-20T20:37:40.867530] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T20:37:41.051383] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-07. [2025-10-20T20:37:41.175543] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T20:37:41.279972] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:37:41.851585] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-03. [2025-10-20T20:37:42.651376] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-07. [2025-10-20T20:37:43.451428] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-02. [2025-10-20T20:37:43.501895] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-07 (2.45 seconds). [2025-10-20T20:37:43.600759] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T20:37:44.251401] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-04. [2025-10-20T20:37:44.513403] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T20:37:45.056633] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-03. [2025-10-20T20:37:45.062163] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-07 (2.41 seconds). [2025-10-20T20:37:45.691028] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T20:37:45.851363] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-08. [2025-10-20T20:37:46.143969] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T20:37:46.651422] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-05. [2025-10-20T20:37:47.451377] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-08. [2025-10-20T20:37:48.084622] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-08 (2.23 seconds). [2025-10-20T20:37:48.251385] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-09. [2025-10-20T20:37:48.405348] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-05 (1.75 seconds). [2025-10-20T20:37:48.932764] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T20:37:49.051430] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-06. [2025-10-20T20:37:49.851444] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-04. [2025-10-20T20:37:50.651433] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-09. [2025-10-20T20:37:50.724265] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T20:37:50.784572] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:37:50.834537] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-09 (2.58 seconds). [2025-10-20T20:37:51.451417] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-07. [2025-10-20T20:37:52.251366] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-05. [2025-10-20T20:37:52.698680] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T20:37:52.906980] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T20:37:53.051341] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-08. [2025-10-20T20:37:53.092506] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T20:37:53.851429] [INFO ] [REMOTE ] PLD: Fetching primary data for 2025-10. [2025-10-20T20:37:54.581596] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T20:37:54.611047] [INFO ] [REMOTE ] PLD: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:37:54.618397] [INFO ] [REMOTE ] PLD: Full fetch complete (77123 records). [2025-10-20T20:37:54.618514] [DEBUG] [VALIDATOR ] PLD: Validating primary data. [2025-10-20T20:37:54.619191] [FATAL] [VALIDATOR ] PLD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:37:54.619277] [FATAL] [MAIN ] PLD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:37:54.619340] [WARN ] [MAIN ] PLD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:37:54.619423] [INFO ] [LOCAL ] PLD: Starting deletion of primary data. [2025-10-20T20:37:54.619500] [DEBUG] [LOCAL ] PLD: No primary data file found to delete. [2025-10-20T20:37:54.619548] [INFO ] [LOCAL ] PLD: Starting deletion of derivative data. [2025-10-20T20:37:54.619733] [DEBUG] [LOCAL ] PLD: Deleted derivative data file. [2025-10-20T20:37:54.619836] [INFO ] [MAIN ] Update Progress: 1692/2367 symbols completed. PLD took 34.0s. Estimated remaining: 1h 26m 54s. Rate: 7.8 symbols/min [2025-10-20T20:37:54.656464] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-09. [2025-10-20T20:37:54.795487] [INFO ] [MAIN ] PLYM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:54.828840] [DEBUG] [LOCAL ] PLYM: Loaded 34815 existing records from primary storage. [2025-10-20T20:37:54.829111] [INFO ] [MAIN ] PLYM: Loaded 34815 existing records. [2025-10-20T20:37:55.005421] [WARN ] [MAIN ] PLYM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:55.005699] [INFO ] [MAIN ] PLYM: Wiping existing primary data before full refresh. [2025-10-20T20:37:55.005769] [INFO ] [LOCAL ] PLYM: Starting deletion of primary data. [2025-10-20T20:37:55.006083] [DEBUG] [LOCAL ] PLYM: Deleted primary data file. [2025-10-20T20:37:55.006138] [INFO ] [MAIN ] PLYM: Starting full primary refresh. [2025-10-20T20:37:55.352031] [INFO ] [REMOTE ] PLYM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:55.451378] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-06. [2025-10-20T20:37:56.070130] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T20:37:56.226780] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T20:37:56.251329] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-07. [2025-10-20T20:37:57.051480] [INFO ] [REMOTE ] PL: Fetching primary data for 2025-10. [2025-10-20T20:37:57.433609] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T20:37:57.791586] [INFO ] [REMOTE ] PL: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T20:37:57.799177] [INFO ] [REMOTE ] PL: Full fetch complete (76723 records). [2025-10-20T20:37:57.799335] [DEBUG] [VALIDATOR ] PL: Validating primary data. [2025-10-20T20:37:57.800187] [FATAL] [VALIDATOR ] PL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:37:57.800283] [FATAL] [MAIN ] PL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:37:57.800325] [WARN ] [MAIN ] PL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:37:57.800363] [INFO ] [LOCAL ] PL: Starting deletion of primary data. [2025-10-20T20:37:57.800419] [DEBUG] [LOCAL ] PL: No primary data file found to delete. [2025-10-20T20:37:57.800453] [INFO ] [LOCAL ] PL: Starting deletion of derivative data. [2025-10-20T20:37:57.800603] [DEBUG] [LOCAL ] PL: Deleted derivative data file. [2025-10-20T20:37:57.800736] [INFO ] [MAIN ] Update Progress: 1693/2367 symbols completed. PL took 39.4s. Estimated remaining: 1h 26m 45s. Rate: 7.8 symbols/min [2025-10-20T20:37:57.856688] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-08. [2025-10-20T20:37:57.975636] [INFO ] [MAIN ] PM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:37:58.029296] [DEBUG] [LOCAL ] PM: Loaded 54453 existing records from primary storage. [2025-10-20T20:37:58.029677] [INFO ] [MAIN ] PM: Loaded 54453 existing records. [2025-10-20T20:37:58.204978] [WARN ] [MAIN ] PM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:37:58.205214] [INFO ] [MAIN ] PM: Wiping existing primary data before full refresh. [2025-10-20T20:37:58.205260] [INFO ] [LOCAL ] PM: Starting deletion of primary data. [2025-10-20T20:37:58.205554] [DEBUG] [LOCAL ] PM: Deleted primary data file. [2025-10-20T20:37:58.205592] [INFO ] [MAIN ] PM: Starting full primary refresh. [2025-10-20T20:37:58.555361] [INFO ] [REMOTE ] PM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:37:58.651398] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-01. [2025-10-20T20:37:58.823512] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T20:37:59.451398] [INFO ] [REMOTE ] PLNT: Fetching primary data for 2025-10. [2025-10-20T20:38:00.103190] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T20:38:00.171561] [INFO ] [REMOTE ] PLNT: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T20:38:00.178233] [INFO ] [REMOTE ] PLNT: Full fetch complete (70397 records). [2025-10-20T20:38:00.178351] [DEBUG] [VALIDATOR ] PLNT: Validating primary data. [2025-10-20T20:38:00.179119] [FATAL] [VALIDATOR ] PLNT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:38:00.179206] [FATAL] [MAIN ] PLNT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:38:00.179248] [WARN ] [MAIN ] PLNT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:38:00.179285] [INFO ] [LOCAL ] PLNT: Starting deletion of primary data. [2025-10-20T20:38:00.179341] [DEBUG] [LOCAL ] PLNT: No primary data file found to delete. [2025-10-20T20:38:00.179375] [INFO ] [LOCAL ] PLNT: Starting deletion of derivative data. [2025-10-20T20:38:00.179519] [DEBUG] [LOCAL ] PLNT: Deleted derivative data file. [2025-10-20T20:38:00.179886] [INFO ] [MAIN ] Update Progress: 1694/2367 symbols completed. PLNT took 24.7s. Estimated remaining: 1h 26m 35s. Rate: 7.8 symbols/min [2025-10-20T20:38:00.256497] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-02. [2025-10-20T20:38:00.350276] [INFO ] [MAIN ] PML: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:00.372228] [DEBUG] [LOCAL ] PML: Loaded 22320 existing records from primary storage. [2025-10-20T20:38:00.372462] [INFO ] [MAIN ] PML: Loaded 22320 existing records. [2025-10-20T20:38:00.544214] [WARN ] [MAIN ] PML: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:00.544418] [INFO ] [MAIN ] PML: Wiping existing primary data before full refresh. [2025-10-20T20:38:00.544464] [INFO ] [LOCAL ] PML: Starting deletion of primary data. [2025-10-20T20:38:00.544631] [DEBUG] [LOCAL ] PML: Deleted primary data file. [2025-10-20T20:38:00.544711] [INFO ] [MAIN ] PML: Starting full primary refresh. [2025-10-20T20:38:00.898612] [INFO ] [REMOTE ] PML: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:01.051397] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-09. [2025-10-20T20:38:01.663808] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T20:38:01.851353] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-03. [2025-10-20T20:38:02.651396] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-01. [2025-10-20T20:38:03.277214] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-09 (2.23 seconds). [2025-10-20T20:38:03.451464] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-01. [2025-10-20T20:38:04.251395] [INFO ] [REMOTE ] PLOW: Fetching primary data for 2025-10. [2025-10-20T20:38:07.339878] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-03 (5.49 seconds). [2025-10-20T20:38:07.347323] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-04. [2025-10-20T20:38:07.354483] [INFO ] [REMOTE ] PLOW: Incremental fetch complete for 2025-10 (3.10 seconds). [2025-10-20T20:38:07.357257] [INFO ] [REMOTE ] PLOW: Full fetch complete (28400 records). [2025-10-20T20:38:07.357355] [DEBUG] [VALIDATOR ] PLOW: Validating primary data. [2025-10-20T20:38:07.712225] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-01 (4.26 seconds). [2025-10-20T20:38:07.752865] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-02. [2025-10-20T20:38:07.781362] [DEBUG] [VALIDATOR ] PLOW: All expected trading days are present. [2025-10-20T20:38:07.781449] [DEBUG] [VALIDATOR ] PLOW: Primary data validation passed. [2025-10-20T20:38:07.782085] [INFO ] [MAIN ] PLOW: Fetched primary data is valid. [2025-10-20T20:38:07.782287] [DEBUG] [LOCAL ] PLOW: Saving primary data. [2025-10-20T20:38:07.783213] [DEBUG] [LOCAL ] PLOW: Trimmed 28400 records to 19918 (keeping 200 days of data) [2025-10-20T20:38:07.902885] [DEBUG] [LOCAL ] PLOW: Primary data saved (19918 records). [2025-10-20T20:38:07.903112] [INFO ] [MAIN ] PLOW: Full history saved. [2025-10-20T20:38:07.903200] [INFO ] [MAIN ] Update Progress: 1695/2367 symbols completed. PLOW took 29.3s. Estimated remaining: 1h 26m 27s. Rate: 7.8 symbols/min [2025-10-20T20:38:08.075123] [INFO ] [MAIN ] PMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:08.087213] [DEBUG] [LOCAL ] PMM: Loaded 10328 existing records from primary storage. [2025-10-20T20:38:08.087443] [INFO ] [MAIN ] PMM: Loaded 10328 existing records. [2025-10-20T20:38:08.286465] [WARN ] [MAIN ] PMM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:08.287700] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-01 (5.64 seconds). [2025-10-20T20:38:08.287869] [INFO ] [MAIN ] PMM: Wiping existing primary data before full refresh. [2025-10-20T20:38:08.292400] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-02. [2025-10-20T20:38:08.292456] [INFO ] [LOCAL ] PMM: Starting deletion of primary data. [2025-10-20T20:38:08.293374] [DEBUG] [LOCAL ] PMM: Deleted primary data file. [2025-10-20T20:38:08.293430] [INFO ] [MAIN ] PMM: Starting full primary refresh. [2025-10-20T20:38:08.671907] [INFO ] [REMOTE ] PMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:08.672445] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-01. [2025-10-20T20:38:08.693503] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T20:38:08.701348] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-05. [2025-10-20T20:38:08.737577] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T20:38:08.993952] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-02 (0.70 seconds). [2025-10-20T20:38:09.051371] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-03. [2025-10-20T20:38:09.699620] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T20:38:09.728760] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T20:38:09.745166] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:38:09.851374] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-06. [2025-10-20T20:38:10.651422] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-03. [2025-10-20T20:38:11.204287] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T20:38:11.451368] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-02. [2025-10-20T20:38:11.512017] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T20:38:12.251371] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-04. [2025-10-20T20:38:12.527117] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T20:38:13.051368] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-07. [2025-10-20T20:38:13.331060] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T20:38:13.851416] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-05. [2025-10-20T20:38:14.438892] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T20:38:14.464865] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T20:38:14.651370] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-04. [2025-10-20T20:38:15.455232] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-03. [2025-10-20T20:38:15.461766] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T20:38:16.251371] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-05. [2025-10-20T20:38:16.469829] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T20:38:17.051395] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-08. [2025-10-20T20:38:17.213676] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T20:38:17.851391] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-04. [2025-10-20T20:38:18.615214] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-04 (0.76 seconds). [2025-10-20T20:38:18.652260] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-05. [2025-10-20T20:38:18.652152] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:38:19.451373] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-09. [2025-10-20T20:38:19.542269] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:38:20.251366] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-06. [2025-10-20T20:38:21.046808] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T20:38:21.056867] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-06. [2025-10-20T20:38:21.113277] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T20:38:21.678874] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-06 (0.62 seconds). [2025-10-20T20:38:21.851383] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-06. [2025-10-20T20:38:22.651356] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-07. [2025-10-20T20:38:22.961017] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T20:38:23.451428] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-07. [2025-10-20T20:38:23.530194] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T20:38:24.251371] [INFO ] [REMOTE ] PLYM: Fetching primary data for 2025-10. [2025-10-20T20:38:24.532323] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T20:38:25.051454] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-07. [2025-10-20T20:38:25.660779] [INFO ] [REMOTE ] PLYM: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T20:38:25.667212] [INFO ] [REMOTE ] PLYM: Full fetch complete (47919 records). [2025-10-20T20:38:25.667347] [DEBUG] [VALIDATOR ] PLYM: Validating primary data. [2025-10-20T20:38:25.870679] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-08. [2025-10-20T20:38:25.970381] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T20:38:26.090113] [DEBUG] [VALIDATOR ] PLYM: All expected trading days are present. [2025-10-20T20:38:26.090235] [DEBUG] [VALIDATOR ] PLYM: Primary data validation passed. [2025-10-20T20:38:26.091882] [INFO ] [MAIN ] PLYM: Fetched primary data is valid. [2025-10-20T20:38:26.092041] [DEBUG] [LOCAL ] PLYM: Saving primary data. [2025-10-20T20:38:26.093748] [DEBUG] [LOCAL ] PLYM: Trimmed 47919 records to 34451 (keeping 200 days of data) [2025-10-20T20:38:26.326622] [DEBUG] [LOCAL ] PLYM: Primary data saved (34451 records). [2025-10-20T20:38:26.326885] [INFO ] [MAIN ] PLYM: Full history saved. [2025-10-20T20:38:26.327006] [INFO ] [MAIN ] Update Progress: 1696/2367 symbols completed. PLYM took 31.7s. Estimated remaining: 1h 26m 24s. Rate: 7.8 symbols/min [2025-10-20T20:38:26.510020] [INFO ] [MAIN ] PMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:26.627802] [DEBUG] [LOCAL ] PMO: Loaded 9956 existing records from primary storage. [2025-10-20T20:38:26.628183] [INFO ] [MAIN ] PMO: Loaded 9956 existing records. [2025-10-20T20:38:26.663315] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-08. [2025-10-20T20:38:26.846166] [WARN ] [MAIN ] PMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:26.846682] [INFO ] [MAIN ] PMO: Wiping existing primary data before full refresh. [2025-10-20T20:38:26.846852] [INFO ] [LOCAL ] PMO: Starting deletion of primary data. [2025-10-20T20:38:26.847176] [DEBUG] [LOCAL ] PMO: Deleted primary data file. [2025-10-20T20:38:26.847295] [INFO ] [MAIN ] PMO: Starting full primary refresh. [2025-10-20T20:38:27.028579] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:38:27.266749] [INFO ] [REMOTE ] PMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:27.451377] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-01. [2025-10-20T20:38:27.964008] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T20:38:28.251370] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-08. [2025-10-20T20:38:28.494270] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T20:38:29.051438] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-09. [2025-10-20T20:38:29.086128] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T20:38:29.851374] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-09. [2025-10-20T20:38:30.072098] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:38:30.599114] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-09 (0.75 seconds). [2025-10-20T20:38:30.651371] [INFO ] [REMOTE ] PM: Fetching primary data for 2025-10. [2025-10-20T20:38:31.458466] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-02. [2025-10-20T20:38:31.463435] [INFO ] [REMOTE ] PM: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T20:38:31.471430] [INFO ] [REMOTE ] PM: Full fetch complete (77441 records). [2025-10-20T20:38:31.471600] [DEBUG] [VALIDATOR ] PM: Validating primary data. [2025-10-20T20:38:31.472506] [FATAL] [VALIDATOR ] PM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:38:31.472604] [FATAL] [MAIN ] PM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:38:31.472675] [WARN ] [MAIN ] PM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:38:31.472755] [INFO ] [LOCAL ] PM: Starting deletion of primary data. [2025-10-20T20:38:31.472819] [DEBUG] [LOCAL ] PM: No primary data file found to delete. [2025-10-20T20:38:31.472854] [INFO ] [LOCAL ] PM: Starting deletion of derivative data. [2025-10-20T20:38:31.472993] [DEBUG] [LOCAL ] PM: Deleted derivative data file. [2025-10-20T20:38:31.473084] [INFO ] [MAIN ] Update Progress: 1697/2367 symbols completed. PM took 33.7s. Estimated remaining: 1h 26m 15s. Rate: 7.8 symbols/min [2025-10-20T20:38:31.646815] [INFO ] [MAIN ] PMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:31.687541] [DEBUG] [LOCAL ] PMT: Loaded 41335 existing records from primary storage. [2025-10-20T20:38:31.687874] [INFO ] [MAIN ] PMT: Loaded 41335 existing records. [2025-10-20T20:38:31.868219] [WARN ] [MAIN ] PMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:31.868412] [INFO ] [MAIN ] PMT: Wiping existing primary data before full refresh. [2025-10-20T20:38:31.868459] [INFO ] [LOCAL ] PMT: Starting deletion of primary data. [2025-10-20T20:38:31.868635] [DEBUG] [LOCAL ] PMT: Deleted primary data file. [2025-10-20T20:38:31.868722] [INFO ] [MAIN ] PMT: Starting full primary refresh. [2025-10-20T20:38:32.226527] [INFO ] [REMOTE ] PMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:32.251374] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-09. [2025-10-20T20:38:32.353921] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T20:38:33.051433] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-01. [2025-10-20T20:38:33.172413] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T20:38:33.851436] [INFO ] [REMOTE ] PML: Fetching primary data for 2025-10. [2025-10-20T20:38:34.626226] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T20:38:34.651350] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-02. [2025-10-20T20:38:35.034102] [INFO ] [REMOTE ] PML: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T20:38:35.037956] [INFO ] [REMOTE ] PML: Full fetch complete (29800 records). [2025-10-20T20:38:35.038084] [DEBUG] [VALIDATOR ] PML: Validating primary data. [2025-10-20T20:38:35.419113] [DEBUG] [VALIDATOR ] PML: All expected trading days are present. [2025-10-20T20:38:35.419187] [DEBUG] [VALIDATOR ] PML: Primary data validation passed. [2025-10-20T20:38:35.419942] [INFO ] [MAIN ] PML: Fetched primary data is valid. [2025-10-20T20:38:35.420009] [DEBUG] [LOCAL ] PML: Saving primary data. [2025-10-20T20:38:35.421018] [DEBUG] [LOCAL ] PML: Trimmed 29800 records to 22073 (keeping 200 days of data) [2025-10-20T20:38:35.469817] [INFO ] [REMOTE ] PMM: Fetching primary data for 2025-10. [2025-10-20T20:38:35.549017] [DEBUG] [LOCAL ] PML: Primary data saved (22073 records). [2025-10-20T20:38:35.549134] [INFO ] [MAIN ] PML: Full history saved. [2025-10-20T20:38:35.549206] [INFO ] [MAIN ] Update Progress: 1698/2367 symbols completed. PML took 35.4s. Estimated remaining: 1h 26m 6s. Rate: 7.8 symbols/min [2025-10-20T20:38:35.722444] [INFO ] [MAIN ] PMTU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:35.722693] [DEBUG] [LOCAL ] PMTU: No existing data file found in primary storage. [2025-10-20T20:38:35.722749] [WARN ] [MAIN ] PMTU: No existing data found, performing full refresh instead. [2025-10-20T20:38:35.722876] [INFO ] [MAIN ] PMTU: Wiping existing primary data before full refresh. [2025-10-20T20:38:35.722942] [INFO ] [LOCAL ] PMTU: Starting deletion of primary data. [2025-10-20T20:38:35.722999] [DEBUG] [LOCAL ] PMTU: No primary data file found to delete. [2025-10-20T20:38:35.723042] [INFO ] [MAIN ] PMTU: Starting full primary refresh. [2025-10-20T20:38:36.098797] [INFO ] [REMOTE ] PMTU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:36.133378] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T20:38:36.251353] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-03. [2025-10-20T20:38:36.318271] [INFO ] [REMOTE ] PMM: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T20:38:36.320248] [INFO ] [REMOTE ] PMM: Full fetch complete (17725 records). [2025-10-20T20:38:36.320315] [DEBUG] [VALIDATOR ] PMM: Validating primary data. [2025-10-20T20:38:36.705166] [DEBUG] [VALIDATOR ] PMM: All expected trading days are present. [2025-10-20T20:38:36.705246] [DEBUG] [VALIDATOR ] PMM: Primary data validation passed. [2025-10-20T20:38:36.705732] [INFO ] [MAIN ] PMM: Fetched primary data is valid. [2025-10-20T20:38:36.705793] [DEBUG] [LOCAL ] PMM: Saving primary data. [2025-10-20T20:38:36.706314] [DEBUG] [LOCAL ] PMM: Trimmed 17725 records to 9961 (keeping 200 days of data) [2025-10-20T20:38:36.764166] [DEBUG] [LOCAL ] PMM: Primary data saved (9961 records). [2025-10-20T20:38:36.764299] [INFO ] [MAIN ] PMM: Full history saved. [2025-10-20T20:38:36.764375] [INFO ] [MAIN ] Update Progress: 1699/2367 symbols completed. PMM took 28.9s. Estimated remaining: 1h 25m 55s. Rate: 7.8 symbols/min [2025-10-20T20:38:36.935838] [INFO ] [MAIN ] PMTV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:36.936039] [DEBUG] [LOCAL ] PMTV: No existing data file found in primary storage. [2025-10-20T20:38:36.936078] [WARN ] [MAIN ] PMTV: No existing data found, performing full refresh instead. [2025-10-20T20:38:36.936199] [INFO ] [MAIN ] PMTV: Wiping existing primary data before full refresh. [2025-10-20T20:38:36.936257] [INFO ] [LOCAL ] PMTV: Starting deletion of primary data. [2025-10-20T20:38:36.936303] [DEBUG] [LOCAL ] PMTV: No primary data file found to delete. [2025-10-20T20:38:36.936337] [INFO ] [MAIN ] PMTV: Starting full primary refresh. [2025-10-20T20:38:37.056538] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-01. [2025-10-20T20:38:37.309316] [INFO ] [REMOTE ] PMTV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:37.624339] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T20:38:37.851348] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-02. [2025-10-20T20:38:37.927616] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T20:38:38.337905] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T20:38:38.651421] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-03. [2025-10-20T20:38:39.048418] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T20:38:39.451412] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-03. [2025-10-20T20:38:40.190764] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T20:38:40.251404] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-04. [2025-10-20T20:38:41.051353] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-04. [2025-10-20T20:38:41.359989] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T20:38:41.520816] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-04 (0.47 seconds). [2025-10-20T20:38:41.851377] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-05. [2025-10-20T20:38:42.197281] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-05 (0.35 seconds). [2025-10-20T20:38:42.651420] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-06. [2025-10-20T20:38:43.057358] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T20:38:43.451380] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-04. [2025-10-20T20:38:44.251377] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-05. [2025-10-20T20:38:45.001724] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:38:45.051371] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-06. [2025-10-20T20:38:45.187768] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:38:45.851364] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-07. [2025-10-20T20:38:45.871437] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T20:38:46.229461] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-07 (0.38 seconds). [2025-10-20T20:38:46.651414] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-08. [2025-10-20T20:38:47.344567] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T20:38:47.451382] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-09. [2025-10-20T20:38:47.850200] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T20:38:48.251373] [INFO ] [REMOTE ] PMTV: Fetching primary data for 2025-01. [2025-10-20T20:38:48.545353] [DEBUG] [REMOTE ] PMTV: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:38:48.545812] [WARN ] [REMOTE ] PMTV: No data returned for month 2025-01 [2025-10-20T20:38:48.545907] [FATAL] [REMOTE ] PMTV: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:38:48.545961] [FATAL] [MAIN ] PMTV: Failed to fetch primary data. [2025-10-20T20:38:48.545995] [WARN ] [MAIN ] PMTV: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:38:48.546063] [INFO ] [LOCAL ] PMTV: Starting deletion of primary data. [2025-10-20T20:38:48.546135] [DEBUG] [LOCAL ] PMTV: No primary data file found to delete. [2025-10-20T20:38:48.546173] [INFO ] [LOCAL ] PMTV: Starting deletion of derivative data. [2025-10-20T20:38:48.546212] [DEBUG] [LOCAL ] PMTV: No derivative data file found to delete. [2025-10-20T20:38:48.546261] [INFO ] [MAIN ] Update Progress: 1700/2367 symbols completed. PMTV took 11.8s. Estimated remaining: 1h 25m 49s. Rate: 7.8 symbols/min [2025-10-20T20:38:48.725159] [INFO ] [MAIN ] PMTW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:48.725386] [DEBUG] [LOCAL ] PMTW: No existing data file found in primary storage. [2025-10-20T20:38:48.725433] [WARN ] [MAIN ] PMTW: No existing data found, performing full refresh instead. [2025-10-20T20:38:48.725606] [INFO ] [MAIN ] PMTW: Wiping existing primary data before full refresh. [2025-10-20T20:38:48.725678] [INFO ] [LOCAL ] PMTW: Starting deletion of primary data. [2025-10-20T20:38:48.725744] [DEBUG] [LOCAL ] PMTW: No primary data file found to delete. [2025-10-20T20:38:48.725783] [INFO ] [MAIN ] PMTW: Starting full primary refresh. [2025-10-20T20:38:49.056638] [INFO ] [REMOTE ] PMTU: Fetching primary data for 2025-10. [2025-10-20T20:38:49.117503] [INFO ] [REMOTE ] PMTW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:49.486250] [INFO ] [REMOTE ] PMTU: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T20:38:49.486967] [INFO ] [REMOTE ] PMTU: Full fetch complete (1195 records). [2025-10-20T20:38:49.487033] [DEBUG] [VALIDATOR ] PMTU: Validating primary data. [2025-10-20T20:38:49.670544] [WARN ] [VALIDATOR ] PMTU: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T20:38:49.670773] [FATAL] [MAIN ] PMTU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:38:49.670829] [WARN ] [MAIN ] PMTU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:38:49.670896] [INFO ] [LOCAL ] PMTU: Starting deletion of primary data. [2025-10-20T20:38:49.670960] [DEBUG] [LOCAL ] PMTU: No primary data file found to delete. [2025-10-20T20:38:49.670995] [INFO ] [LOCAL ] PMTU: Starting deletion of derivative data. [2025-10-20T20:38:49.671034] [DEBUG] [LOCAL ] PMTU: No derivative data file found to delete. [2025-10-20T20:38:49.671090] [INFO ] [MAIN ] Update Progress: 1701/2367 symbols completed. PMTU took 14.1s. Estimated remaining: 1h 25m 39s. Rate: 7.8 symbols/min [2025-10-20T20:38:49.843783] [INFO ] [MAIN ] PNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:49.851381] [INFO ] [REMOTE ] PMTW: Fetching primary data for 2025-01. [2025-10-20T20:38:49.902458] [DEBUG] [LOCAL ] PNC: Loaded 52874 existing records from primary storage. [2025-10-20T20:38:49.902831] [INFO ] [MAIN ] PNC: Loaded 52874 existing records. [2025-10-20T20:38:50.083953] [WARN ] [MAIN ] PNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:50.084183] [INFO ] [MAIN ] PNC: Wiping existing primary data before full refresh. [2025-10-20T20:38:50.084236] [INFO ] [LOCAL ] PNC: Starting deletion of primary data. [2025-10-20T20:38:50.084798] [DEBUG] [LOCAL ] PNC: Deleted primary data file. [2025-10-20T20:38:50.084885] [INFO ] [MAIN ] PNC: Starting full primary refresh. [2025-10-20T20:38:50.129910] [DEBUG] [REMOTE ] PMTW: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:38:50.167860] [WARN ] [REMOTE ] PMTW: No data returned for month 2025-01 [2025-10-20T20:38:50.178793] [FATAL] [REMOTE ] PMTW: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:38:50.189862] [FATAL] [MAIN ] PMTW: Failed to fetch primary data. [2025-10-20T20:38:50.190011] [WARN ] [MAIN ] PMTW: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:38:50.190381] [INFO ] [LOCAL ] PMTW: Starting deletion of primary data. [2025-10-20T20:38:50.201011] [DEBUG] [LOCAL ] PMTW: No primary data file found to delete. [2025-10-20T20:38:50.206214] [INFO ] [LOCAL ] PMTW: Starting deletion of derivative data. [2025-10-20T20:38:50.232799] [DEBUG] [LOCAL ] PMTW: No derivative data file found to delete. [2025-10-20T20:38:50.267699] [INFO ] [MAIN ] Update Progress: 1702/2367 symbols completed. PMTW took 1.7s. Estimated remaining: 1h 25m 28s. Rate: 7.8 symbols/min [2025-10-20T20:38:50.658720] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-07. [2025-10-20T20:38:50.673074] [INFO ] [REMOTE ] PNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:50.667770] [INFO ] [MAIN ] PNI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:38:50.683365] [DEBUG] [LOCAL ] PNI: Loaded 6950 existing records from primary storage. [2025-10-20T20:38:50.683616] [INFO ] [MAIN ] PNI: Loaded 6950 existing records. [2025-10-20T20:38:50.866597] [WARN ] [MAIN ] PNI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:38:50.866829] [INFO ] [MAIN ] PNI: Wiping existing primary data before full refresh. [2025-10-20T20:38:50.866878] [INFO ] [LOCAL ] PNI: Starting deletion of primary data. [2025-10-20T20:38:50.867030] [DEBUG] [LOCAL ] PNI: Deleted primary data file. [2025-10-20T20:38:50.867069] [INFO ] [MAIN ] PNI: Starting full primary refresh. [2025-10-20T20:38:51.218580] [INFO ] [REMOTE ] PNI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:38:51.451388] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-01. [2025-10-20T20:38:51.524534] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T20:38:52.251418] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-05. [2025-10-20T20:38:52.443857] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T20:38:53.051376] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-02. [2025-10-20T20:38:53.764413] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T20:38:53.825634] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:38:53.851333] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-03. [2025-10-20T20:38:54.494005] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T20:38:54.651398] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-08. [2025-10-20T20:38:55.451379] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-04. [2025-10-20T20:38:55.528545] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T20:38:56.251428] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-01. [2025-10-20T20:38:56.301502] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T20:38:57.051462] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-09. [2025-10-20T20:38:57.851355] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-05. [2025-10-20T20:38:57.936736] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T20:38:58.125461] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T20:38:58.520720] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T20:38:58.651375] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-06. [2025-10-20T20:38:59.451395] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-06. [2025-10-20T20:39:00.088348] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T20:39:00.187703] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T20:39:00.251377] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-07. [2025-10-20T20:39:01.039297] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:39:01.051353] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-08. [2025-10-20T20:39:01.651105] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T20:39:01.851407] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-02. [2025-10-20T20:39:02.651360] [INFO ] [REMOTE ] PMO: Fetching primary data for 2025-10. [2025-10-20T20:39:03.354815] [INFO ] [REMOTE ] PMO: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:39:03.356404] [INFO ] [REMOTE ] PMO: Full fetch complete (16127 records). [2025-10-20T20:39:03.356568] [DEBUG] [VALIDATOR ] PMO: Validating primary data. [2025-10-20T20:39:03.458735] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-09. [2025-10-20T20:39:03.754287] [DEBUG] [VALIDATOR ] PMO: All expected trading days are present. [2025-10-20T20:39:03.754490] [DEBUG] [VALIDATOR ] PMO: Primary data validation passed. [2025-10-20T20:39:03.755021] [INFO ] [MAIN ] PMO: Fetched primary data is valid. [2025-10-20T20:39:03.755242] [DEBUG] [LOCAL ] PMO: Saving primary data. [2025-10-20T20:39:03.756173] [DEBUG] [LOCAL ] PMO: Trimmed 16127 records to 9680 (keeping 200 days of data) [2025-10-20T20:39:03.846450] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-02 (2.00 seconds). [2025-10-20T20:39:03.861846] [DEBUG] [LOCAL ] PMO: Primary data saved (9680 records). [2025-10-20T20:39:03.862316] [INFO ] [MAIN ] PMO: Full history saved. [2025-10-20T20:39:03.862427] [INFO ] [MAIN ] Update Progress: 1703/2367 symbols completed. PMO took 37.5s. Estimated remaining: 1h 25m 23s. Rate: 7.8 symbols/min [2025-10-20T20:39:04.028442] [INFO ] [MAIN ] PNNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:04.058416] [DEBUG] [LOCAL ] PNNT: Loaded 33922 existing records from primary storage. [2025-10-20T20:39:04.058719] [INFO ] [MAIN ] PNNT: Loaded 33922 existing records. [2025-10-20T20:39:04.234764] [WARN ] [MAIN ] PNNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:04.235136] [INFO ] [MAIN ] PNNT: Wiping existing primary data before full refresh. [2025-10-20T20:39:04.235197] [INFO ] [LOCAL ] PNNT: Starting deletion of primary data. [2025-10-20T20:39:04.235431] [DEBUG] [LOCAL ] PNNT: Deleted primary data file. [2025-10-20T20:39:04.235486] [INFO ] [MAIN ] PNNT: Starting full primary refresh. [2025-10-20T20:39:04.256567] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-03. [2025-10-20T20:39:04.358303] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T20:39:04.641251] [INFO ] [REMOTE ] PNNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:05.051405] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-07. [2025-10-20T20:39:05.851374] [INFO ] [REMOTE ] PNI: Fetching primary data for 2025-10. [2025-10-20T20:39:06.242073] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-03 (1.99 seconds). [2025-10-20T20:39:06.508386] [INFO ] [REMOTE ] PNI: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:39:06.509784] [INFO ] [REMOTE ] PNI: Full fetch complete (9722 records). [2025-10-20T20:39:06.509852] [DEBUG] [VALIDATOR ] PNI: Validating primary data. [2025-10-20T20:39:06.659897] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-04. [2025-10-20T20:39:06.870301] [DEBUG] [VALIDATOR ] PNI: All expected trading days are present. [2025-10-20T20:39:06.870387] [DEBUG] [VALIDATOR ] PNI: Primary data validation passed. [2025-10-20T20:39:06.870628] [INFO ] [MAIN ] PNI: Fetched primary data is valid. [2025-10-20T20:39:06.870700] [DEBUG] [LOCAL ] PNI: Saving primary data. [2025-10-20T20:39:06.871114] [DEBUG] [LOCAL ] PNI: Trimmed 9722 records to 6908 (keeping 200 days of data) [2025-10-20T20:39:06.910226] [DEBUG] [LOCAL ] PNI: Primary data saved (6908 records). [2025-10-20T20:39:06.910325] [INFO ] [MAIN ] PNI: Full history saved. [2025-10-20T20:39:06.910397] [INFO ] [MAIN ] Update Progress: 1704/2367 symbols completed. PNI took 16.6s. Estimated remaining: 1h 25m 14s. Rate: 7.8 symbols/min [2025-10-20T20:39:07.126266] [INFO ] [MAIN ] PNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:07.128824] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T20:39:07.183482] [DEBUG] [LOCAL ] PNR: Loaded 48223 existing records from primary storage. [2025-10-20T20:39:07.183842] [INFO ] [MAIN ] PNR: Loaded 48223 existing records. [2025-10-20T20:39:07.368173] [WARN ] [MAIN ] PNR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:07.368402] [INFO ] [MAIN ] PNR: Wiping existing primary data before full refresh. [2025-10-20T20:39:07.368452] [INFO ] [LOCAL ] PNR: Starting deletion of primary data. [2025-10-20T20:39:07.368710] [DEBUG] [LOCAL ] PNR: Deleted primary data file. [2025-10-20T20:39:07.368770] [INFO ] [MAIN ] PNR: Starting full primary refresh. [2025-10-20T20:39:07.456510] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-08. [2025-10-20T20:39:07.787957] [INFO ] [REMOTE ] PNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:08.251391] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-01. [2025-10-20T20:39:08.770715] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-10-20T20:39:09.053451] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-05. [2025-10-20T20:39:09.056539] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:39:09.614185] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T20:39:09.851368] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-02. [2025-10-20T20:39:10.651381] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-09. [2025-10-20T20:39:11.001412] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T20:39:11.451369] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-01. [2025-10-20T20:39:11.583213] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:39:11.835199] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:39:12.251416] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-06. [2025-10-20T20:39:13.051386] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-03. [2025-10-20T20:39:13.189363] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T20:39:13.851375] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-02. [2025-10-20T20:39:14.020102] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T20:39:14.651372] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-07. [2025-10-20T20:39:15.148929] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T20:39:15.451376] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-03. [2025-10-20T20:39:15.815466] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-03 (2.76 seconds). [2025-10-20T20:39:16.251376] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-04. [2025-10-20T20:39:16.485800] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T20:39:16.694876] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T20:39:17.051407] [INFO ] [REMOTE ] PMT: Fetching primary data for 2025-10. [2025-10-20T20:39:17.506782] [INFO ] [REMOTE ] PMT: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T20:39:17.512954] [INFO ] [REMOTE ] PMT: Full fetch complete (58363 records). [2025-10-20T20:39:17.513064] [DEBUG] [VALIDATOR ] PMT: Validating primary data. [2025-10-20T20:39:17.513737] [FATAL] [VALIDATOR ] PMT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:39:17.513819] [FATAL] [MAIN ] PMT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:39:17.513880] [WARN ] [MAIN ] PMT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:39:17.513942] [INFO ] [LOCAL ] PMT: Starting deletion of primary data. [2025-10-20T20:39:17.514004] [DEBUG] [LOCAL ] PMT: No primary data file found to delete. [2025-10-20T20:39:17.514038] [INFO ] [LOCAL ] PMT: Starting deletion of derivative data. [2025-10-20T20:39:17.514149] [DEBUG] [LOCAL ] PMT: Deleted derivative data file. [2025-10-20T20:39:17.514214] [INFO ] [MAIN ] Update Progress: 1705/2367 symbols completed. PMT took 46.0s. Estimated remaining: 1h 25m 7s. Rate: 7.8 symbols/min [2025-10-20T20:39:17.690746] [INFO ] [MAIN ] PNW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:17.731404] [DEBUG] [LOCAL ] PNW: Loaded 46225 existing records from primary storage. [2025-10-20T20:39:17.731740] [INFO ] [MAIN ] PNW: Loaded 46225 existing records. [2025-10-20T20:39:17.852742] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-04. [2025-10-20T20:39:17.942948] [WARN ] [MAIN ] PNW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:17.946900] [INFO ] [MAIN ] PNW: Wiping existing primary data before full refresh. [2025-10-20T20:39:17.947072] [INFO ] [LOCAL ] PNW: Starting deletion of primary data. [2025-10-20T20:39:17.947507] [DEBUG] [LOCAL ] PNW: Deleted primary data file. [2025-10-20T20:39:17.949695] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:39:17.947889] [INFO ] [MAIN ] PNW: Starting full primary refresh. [2025-10-20T20:39:18.312075] [INFO ] [REMOTE ] PNW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:18.651367] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-01. [2025-10-20T20:39:19.451423] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-05. [2025-10-20T20:39:19.615432] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T20:39:20.168617] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T20:39:20.251415] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-02. [2025-10-20T20:39:21.051387] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-08. [2025-10-20T20:39:21.144444] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:39:21.851450] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-05. [2025-10-20T20:39:21.905736] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T20:39:22.651386] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-03. [2025-10-20T20:39:23.011817] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T20:39:23.077477] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T20:39:23.451387] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-09. [2025-10-20T20:39:24.251364] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-06. [2025-10-20T20:39:24.417292] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T20:39:25.051437] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-04. [2025-10-20T20:39:25.377809] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T20:39:25.632990] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T20:39:25.851432] [INFO ] [REMOTE ] PNC: Fetching primary data for 2025-10. [2025-10-20T20:39:26.417255] [INFO ] [REMOTE ] PNC: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T20:39:26.424402] [INFO ] [REMOTE ] PNC: Full fetch complete (75375 records). [2025-10-20T20:39:26.424521] [DEBUG] [VALIDATOR ] PNC: Validating primary data. [2025-10-20T20:39:26.425313] [FATAL] [VALIDATOR ] PNC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:39:26.425400] [FATAL] [MAIN ] PNC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:39:26.425442] [WARN ] [MAIN ] PNC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:39:26.425478] [INFO ] [LOCAL ] PNC: Starting deletion of primary data. [2025-10-20T20:39:26.425535] [DEBUG] [LOCAL ] PNC: No primary data file found to delete. [2025-10-20T20:39:26.425595] [INFO ] [LOCAL ] PNC: Starting deletion of derivative data. [2025-10-20T20:39:26.425793] [DEBUG] [LOCAL ] PNC: Deleted derivative data file. [2025-10-20T20:39:26.426258] [INFO ] [MAIN ] Update Progress: 1706/2367 symbols completed. PNC took 36.8s. Estimated remaining: 1h 24m 60s. Rate: 7.8 symbols/min [2025-10-20T20:39:26.604536] [INFO ] [MAIN ] POR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:26.634841] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T20:39:26.661479] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-06. [2025-10-20T20:39:26.683619] [DEBUG] [LOCAL ] POR: Loaded 47568 existing records from primary storage. [2025-10-20T20:39:26.683921] [INFO ] [MAIN ] POR: Loaded 47568 existing records. [2025-10-20T20:39:26.857498] [WARN ] [MAIN ] POR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:26.857742] [INFO ] [MAIN ] POR: Wiping existing primary data before full refresh. [2025-10-20T20:39:26.857793] [INFO ] [LOCAL ] POR: Starting deletion of primary data. [2025-10-20T20:39:26.858017] [DEBUG] [LOCAL ] POR: Deleted primary data file. [2025-10-20T20:39:26.858072] [INFO ] [MAIN ] POR: Starting full primary refresh. [2025-10-20T20:39:27.203307] [INFO ] [REMOTE ] POR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:27.451373] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-01. [2025-10-20T20:39:28.251367] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-05. [2025-10-20T20:39:28.332069] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T20:39:29.051418] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-07. [2025-10-20T20:39:29.338044] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T20:39:29.851362] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-02. [2025-10-20T20:39:29.892413] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T20:39:30.635070] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T20:39:30.651335] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-08. [2025-10-20T20:39:31.143255] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:39:31.451387] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-07. [2025-10-20T20:39:32.092059] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T20:39:32.251392] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-03. [2025-10-20T20:39:32.926860] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T20:39:33.051355] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-08. [2025-10-20T20:39:33.590387] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T20:39:33.851435] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-06. [2025-10-20T20:39:34.644042] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T20:39:34.651324] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-09. [2025-10-20T20:39:35.229944] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T20:39:35.451389] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-07. [2025-10-20T20:39:36.102634] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T20:39:36.251404] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-09. [2025-10-20T20:39:37.051398] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-04. [2025-10-20T20:39:37.222610] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T20:39:37.851357] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-08. [2025-10-20T20:39:37.960018] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:39:38.651416] [INFO ] [REMOTE ] PNR: Fetching primary data for 2025-10. [2025-10-20T20:39:38.866499] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T20:39:39.358500] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T20:39:39.413426] [INFO ] [REMOTE ] PNR: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:39:39.419883] [INFO ] [REMOTE ] PNR: Full fetch complete (69331 records). [2025-10-20T20:39:39.420046] [DEBUG] [VALIDATOR ] PNR: Validating primary data. [2025-10-20T20:39:39.420737] [FATAL] [VALIDATOR ] PNR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:39:39.420822] [FATAL] [MAIN ] PNR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:39:39.420864] [WARN ] [MAIN ] PNR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:39:39.420902] [INFO ] [LOCAL ] PNR: Starting deletion of primary data. [2025-10-20T20:39:39.420958] [DEBUG] [LOCAL ] PNR: No primary data file found to delete. [2025-10-20T20:39:39.420992] [INFO ] [LOCAL ] PNR: Starting deletion of derivative data. [2025-10-20T20:39:39.421142] [DEBUG] [LOCAL ] PNR: Deleted derivative data file. [2025-10-20T20:39:39.421229] [INFO ] [MAIN ] Update Progress: 1707/2367 symbols completed. PNR took 32.5s. Estimated remaining: 1h 24m 54s. Rate: 7.8 symbols/min [2025-10-20T20:39:39.452824] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-09. [2025-10-20T20:39:39.601391] [INFO ] [MAIN ] POST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:39.645159] [DEBUG] [LOCAL ] POST: Loaded 39267 existing records from primary storage. [2025-10-20T20:39:39.645481] [INFO ] [MAIN ] POST: Loaded 39267 existing records. [2025-10-20T20:39:39.831547] [WARN ] [MAIN ] POST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:39.831780] [INFO ] [MAIN ] POST: Wiping existing primary data before full refresh. [2025-10-20T20:39:39.831829] [INFO ] [LOCAL ] POST: Starting deletion of primary data. [2025-10-20T20:39:39.832021] [DEBUG] [LOCAL ] POST: Deleted primary data file. [2025-10-20T20:39:39.832058] [INFO ] [MAIN ] POST: Starting full primary refresh. [2025-10-20T20:39:40.192377] [INFO ] [REMOTE ] POST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:40.251389] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-01. [2025-10-20T20:39:41.051382] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-05. [2025-10-20T20:39:41.129948] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T20:39:41.528568] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T20:39:41.851401] [INFO ] [REMOTE ] PNW: Fetching primary data for 2025-10. [2025-10-20T20:39:42.235590] [INFO ] [REMOTE ] PNW: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-10-20T20:39:42.241951] [INFO ] [REMOTE ] PNW: Full fetch complete (65713 records). [2025-10-20T20:39:42.242141] [DEBUG] [VALIDATOR ] PNW: Validating primary data. [2025-10-20T20:39:42.242958] [FATAL] [VALIDATOR ] PNW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:39:42.243065] [FATAL] [MAIN ] PNW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:39:42.243112] [WARN ] [MAIN ] PNW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:39:42.243196] [INFO ] [LOCAL ] PNW: Starting deletion of primary data. [2025-10-20T20:39:42.243273] [DEBUG] [LOCAL ] PNW: No primary data file found to delete. [2025-10-20T20:39:42.243321] [INFO ] [LOCAL ] PNW: Starting deletion of derivative data. [2025-10-20T20:39:42.243501] [DEBUG] [LOCAL ] PNW: Deleted derivative data file. [2025-10-20T20:39:42.243610] [INFO ] [MAIN ] Update Progress: 1708/2367 symbols completed. PNW took 24.7s. Estimated remaining: 1h 24m 44s. Rate: 7.8 symbols/min [2025-10-20T20:39:42.440005] [INFO ] [MAIN ] PPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:42.489495] [DEBUG] [LOCAL ] PPG: Loaded 51405 existing records from primary storage. [2025-10-20T20:39:42.489858] [INFO ] [MAIN ] PPG: Loaded 51405 existing records. [2025-10-20T20:39:42.665325] [INFO ] [REMOTE ] PNNT: Fetching primary data for 2025-10. [2025-10-20T20:39:42.668245] [WARN ] [MAIN ] PPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:42.669487] [INFO ] [MAIN ] PPG: Wiping existing primary data before full refresh. [2025-10-20T20:39:42.669573] [INFO ] [LOCAL ] PPG: Starting deletion of primary data. [2025-10-20T20:39:42.669947] [DEBUG] [LOCAL ] PPG: Deleted primary data file. [2025-10-20T20:39:42.692443] [INFO ] [MAIN ] PPG: Starting full primary refresh. [2025-10-20T20:39:42.762736] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T20:39:43.057464] [INFO ] [REMOTE ] PPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:43.451430] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-01. [2025-10-20T20:39:44.142944] [INFO ] [REMOTE ] PNNT: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T20:39:44.151581] [INFO ] [REMOTE ] PNNT: Full fetch complete (47855 records). [2025-10-20T20:39:44.151803] [DEBUG] [VALIDATOR ] PNNT: Validating primary data. [2025-10-20T20:39:44.256602] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-02. [2025-10-20T20:39:44.549020] [DEBUG] [VALIDATOR ] PNNT: All expected trading days are present. [2025-10-20T20:39:44.549108] [DEBUG] [VALIDATOR ] PNNT: Primary data validation passed. [2025-10-20T20:39:44.549971] [INFO ] [MAIN ] PNNT: Fetched primary data is valid. [2025-10-20T20:39:44.550045] [DEBUG] [LOCAL ] PNNT: Saving primary data. [2025-10-20T20:39:44.551147] [DEBUG] [LOCAL ] PNNT: Trimmed 47855 records to 33472 (keeping 200 days of data) [2025-10-20T20:39:44.750526] [DEBUG] [LOCAL ] PNNT: Primary data saved (33472 records). [2025-10-20T20:39:44.750742] [INFO ] [MAIN ] PNNT: Full history saved. [2025-10-20T20:39:44.750838] [INFO ] [MAIN ] Update Progress: 1709/2367 symbols completed. PNNT took 40.9s. Estimated remaining: 1h 24m 35s. Rate: 7.8 symbols/min [2025-10-20T20:39:44.931694] [INFO ] [MAIN ] PPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:39:44.981425] [DEBUG] [LOCAL ] PPL: Loaded 54281 existing records from primary storage. [2025-10-20T20:39:44.981766] [INFO ] [MAIN ] PPL: Loaded 54281 existing records. [2025-10-20T20:39:45.065451] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-06. [2025-10-20T20:39:45.166939] [WARN ] [MAIN ] PPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:39:45.167206] [INFO ] [MAIN ] PPL: Wiping existing primary data before full refresh. [2025-10-20T20:39:45.167281] [INFO ] [LOCAL ] PPL: Starting deletion of primary data. [2025-10-20T20:39:45.167630] [DEBUG] [LOCAL ] PPL: Deleted primary data file. [2025-10-20T20:39:45.167737] [INFO ] [MAIN ] PPL: Starting full primary refresh. [2025-10-20T20:39:45.539835] [INFO ] [REMOTE ] PPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:39:45.569442] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-01 (2.12 seconds). [2025-10-20T20:39:45.600194] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:39:45.851386] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-03. [2025-10-20T20:39:46.651350] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-02. [2025-10-20T20:39:46.806186] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T20:39:47.069037] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T20:39:47.451462] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-04. [2025-10-20T20:39:47.987416] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:39:48.251349] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-03. [2025-10-20T20:39:49.051374] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-01. [2025-10-20T20:39:49.151667] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:39:49.750160] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:39:49.851422] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-07. [2025-10-20T20:39:50.651413] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-04. [2025-10-20T20:39:50.868673] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T20:39:51.375522] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T20:39:51.451350] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-08. [2025-10-20T20:39:52.251343] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-02. [2025-10-20T20:39:52.332278] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:39:52.807341] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T20:39:53.051383] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-05. [2025-10-20T20:39:53.821173] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T20:39:53.851345] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-03. [2025-10-20T20:39:54.656544] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-05. [2025-10-20T20:39:54.670416] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:39:55.275544] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-03 (1.42 seconds). [2025-10-20T20:39:55.451356] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-04. [2025-10-20T20:39:56.144099] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T20:39:56.251356] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-06. [2025-10-20T20:39:57.051355] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-06. [2025-10-20T20:39:57.398082] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T20:39:57.648117] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T20:39:57.851412] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-09. [2025-10-20T20:39:58.651354] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-05. [2025-10-20T20:39:58.737739] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T20:39:59.451356] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-07. [2025-10-20T20:39:59.685147] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T20:40:00.251350] [INFO ] [REMOTE ] POR: Fetching primary data for 2025-10. [2025-10-20T20:40:00.550926] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T20:40:01.056451] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-07. [2025-10-20T20:40:01.066919] [INFO ] [REMOTE ] POR: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T20:40:01.073771] [INFO ] [REMOTE ] POR: Full fetch complete (68117 records). [2025-10-20T20:40:01.073878] [DEBUG] [VALIDATOR ] POR: Validating primary data. [2025-10-20T20:40:01.074622] [FATAL] [VALIDATOR ] POR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:01.074723] [FATAL] [MAIN ] POR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:01.074769] [WARN ] [MAIN ] POR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:01.074837] [INFO ] [LOCAL ] POR: Starting deletion of primary data. [2025-10-20T20:40:01.074905] [DEBUG] [LOCAL ] POR: No primary data file found to delete. [2025-10-20T20:40:01.074941] [INFO ] [LOCAL ] POR: Starting deletion of derivative data. [2025-10-20T20:40:01.075213] [DEBUG] [LOCAL ] POR: Deleted derivative data file. [2025-10-20T20:40:01.075690] [INFO ] [MAIN ] Update Progress: 1710/2367 symbols completed. POR took 34.6s. Estimated remaining: 1h 24m 30s. Rate: 7.8 symbols/min [2025-10-20T20:40:01.273118] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T20:40:01.290730] [INFO ] [MAIN ] PPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:01.320369] [DEBUG] [LOCAL ] PPT: Loaded 20031 existing records from primary storage. [2025-10-20T20:40:01.320617] [INFO ] [MAIN ] PPT: Loaded 20031 existing records. [2025-10-20T20:40:01.519146] [WARN ] [MAIN ] PPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:01.519417] [INFO ] [MAIN ] PPT: Wiping existing primary data before full refresh. [2025-10-20T20:40:01.519466] [INFO ] [LOCAL ] PPT: Starting deletion of primary data. [2025-10-20T20:40:01.519693] [DEBUG] [LOCAL ] PPT: Deleted primary data file. [2025-10-20T20:40:01.519752] [INFO ] [MAIN ] PPT: Starting full primary refresh. [2025-10-20T20:40:01.858860] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-08. [2025-10-20T20:40:01.895258] [INFO ] [REMOTE ] PPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:02.651479] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-01. [2025-10-20T20:40:03.016562] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T20:40:03.451418] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-08. [2025-10-20T20:40:03.534778] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T20:40:03.795254] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T20:40:04.251429] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-09. [2025-10-20T20:40:05.052466] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-02. [2025-10-20T20:40:05.052316] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:40:05.761830] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T20:40:05.851334] [INFO ] [REMOTE ] POST: Fetching primary data for 2025-10. [2025-10-20T20:40:06.037732] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T20:40:06.544502] [INFO ] [REMOTE ] POST: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:40:06.549455] [INFO ] [REMOTE ] POST: Full fetch complete (55424 records). [2025-10-20T20:40:06.549546] [DEBUG] [VALIDATOR ] POST: Validating primary data. [2025-10-20T20:40:06.550111] [FATAL] [VALIDATOR ] POST: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:06.550192] [FATAL] [MAIN ] POST: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:06.550233] [WARN ] [MAIN ] POST: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:06.550268] [INFO ] [LOCAL ] POST: Starting deletion of primary data. [2025-10-20T20:40:06.550324] [DEBUG] [LOCAL ] POST: No primary data file found to delete. [2025-10-20T20:40:06.550358] [INFO ] [LOCAL ] POST: Starting deletion of derivative data. [2025-10-20T20:40:06.550474] [DEBUG] [LOCAL ] POST: Deleted derivative data file. [2025-10-20T20:40:06.550538] [INFO ] [MAIN ] Update Progress: 1711/2367 symbols completed. POST took 27.1s. Estimated remaining: 1h 24m 22s. Rate: 7.8 symbols/min [2025-10-20T20:40:06.652600] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-09. [2025-10-20T20:40:06.721233] [INFO ] [MAIN ] PR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:06.771793] [DEBUG] [LOCAL ] PR: Loaded 54390 existing records from primary storage. [2025-10-20T20:40:06.772121] [INFO ] [MAIN ] PR: Loaded 54390 existing records. [2025-10-20T20:40:06.948607] [WARN ] [MAIN ] PR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:06.948829] [INFO ] [MAIN ] PR: Wiping existing primary data before full refresh. [2025-10-20T20:40:06.948876] [INFO ] [LOCAL ] PR: Starting deletion of primary data. [2025-10-20T20:40:06.949078] [DEBUG] [LOCAL ] PR: Deleted primary data file. [2025-10-20T20:40:06.949132] [INFO ] [MAIN ] PR: Starting full primary refresh. [2025-10-20T20:40:07.295072] [INFO ] [REMOTE ] PR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:07.451364] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-01. [2025-10-20T20:40:08.251352] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-06. [2025-10-20T20:40:08.652801] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T20:40:09.051431] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-03. [2025-10-20T20:40:09.556715] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-01 (2.11 seconds). [2025-10-20T20:40:09.851331] [INFO ] [REMOTE ] PPG: Fetching primary data for 2025-10. [2025-10-20T20:40:09.965320] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T20:40:10.038458] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T20:40:10.282409] [INFO ] [REMOTE ] PPG: Incremental fetch complete for 2025-10 (0.43 seconds). [2025-10-20T20:40:10.289398] [INFO ] [REMOTE ] PPG: Full fetch complete (73684 records). [2025-10-20T20:40:10.289509] [DEBUG] [VALIDATOR ] PPG: Validating primary data. [2025-10-20T20:40:10.290257] [FATAL] [VALIDATOR ] PPG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:10.290345] [FATAL] [MAIN ] PPG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:10.290388] [WARN ] [MAIN ] PPG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:10.290425] [INFO ] [LOCAL ] PPG: Starting deletion of primary data. [2025-10-20T20:40:10.290481] [DEBUG] [LOCAL ] PPG: No primary data file found to delete. [2025-10-20T20:40:10.290515] [INFO ] [LOCAL ] PPG: Starting deletion of derivative data. [2025-10-20T20:40:10.290701] [DEBUG] [LOCAL ] PPG: Deleted derivative data file. [2025-10-20T20:40:10.290787] [INFO ] [MAIN ] Update Progress: 1712/2367 symbols completed. PPG took 28.0s. Estimated remaining: 1h 24m 12s. Rate: 7.8 symbols/min [2025-10-20T20:40:10.459355] [INFO ] [MAIN ] PRA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:10.491453] [DEBUG] [LOCAL ] PRA: Loaded 35319 existing records from primary storage. [2025-10-20T20:40:10.491734] [INFO ] [MAIN ] PRA: Loaded 35319 existing records. [2025-10-20T20:40:10.655289] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-02. [2025-10-20T20:40:10.677178] [WARN ] [MAIN ] PRA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:10.677412] [INFO ] [MAIN ] PRA: Wiping existing primary data before full refresh. [2025-10-20T20:40:10.677498] [INFO ] [LOCAL ] PRA: Starting deletion of primary data. [2025-10-20T20:40:10.677710] [DEBUG] [LOCAL ] PRA: Deleted primary data file. [2025-10-20T20:40:10.677766] [INFO ] [MAIN ] PRA: Starting full primary refresh. [2025-10-20T20:40:11.036262] [INFO ] [REMOTE ] PRA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:11.451375] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-07. [2025-10-20T20:40:12.251367] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-01. [2025-10-20T20:40:12.589419] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T20:40:13.051438] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-04. [2025-10-20T20:40:13.166585] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:40:13.274485] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T20:40:13.851384] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-03. [2025-10-20T20:40:14.031017] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T20:40:14.651372] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-02. [2025-10-20T20:40:15.451410] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-08. [2025-10-20T20:40:15.531325] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T20:40:15.860516] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T20:40:16.251358] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-05. [2025-10-20T20:40:17.012617] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T20:40:17.051330] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-09. [2025-10-20T20:40:17.366802] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T20:40:17.851428] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-04. [2025-10-20T20:40:18.651381] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-03. [2025-10-20T20:40:18.701851] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T20:40:19.451436] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-06. [2025-10-20T20:40:19.646744] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T20:40:20.111274] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T20:40:20.251360] [INFO ] [REMOTE ] PPL: Fetching primary data for 2025-10. [2025-10-20T20:40:20.556155] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T20:40:20.660388] [INFO ] [REMOTE ] PPL: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T20:40:20.668036] [INFO ] [REMOTE ] PPL: Full fetch complete (77223 records). [2025-10-20T20:40:20.668188] [DEBUG] [VALIDATOR ] PPL: Validating primary data. [2025-10-20T20:40:20.668943] [FATAL] [VALIDATOR ] PPL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:20.669029] [FATAL] [MAIN ] PPL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:20.669071] [WARN ] [MAIN ] PPL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:20.669107] [INFO ] [LOCAL ] PPL: Starting deletion of primary data. [2025-10-20T20:40:20.669162] [DEBUG] [LOCAL ] PPL: No primary data file found to delete. [2025-10-20T20:40:20.669196] [INFO ] [LOCAL ] PPL: Starting deletion of derivative data. [2025-10-20T20:40:20.669310] [DEBUG] [LOCAL ] PPL: Deleted derivative data file. [2025-10-20T20:40:20.669378] [INFO ] [MAIN ] Update Progress: 1713/2367 symbols completed. PPL took 35.9s. Estimated remaining: 1h 24m 6s. Rate: 7.8 symbols/min [2025-10-20T20:40:20.844215] [INFO ] [MAIN ] PRCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:20.849285] [DEBUG] [LOCAL ] PRCS: Loaded 2830 existing records from primary storage. [2025-10-20T20:40:20.849465] [INFO ] [MAIN ] PRCS: Loaded 2830 existing records. [2025-10-20T20:40:21.038477] [WARN ] [MAIN ] PRCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:21.038718] [INFO ] [MAIN ] PRCS: Wiping existing primary data before full refresh. [2025-10-20T20:40:21.038775] [INFO ] [LOCAL ] PRCS: Starting deletion of primary data. [2025-10-20T20:40:21.038941] [DEBUG] [LOCAL ] PRCS: Deleted primary data file. [2025-10-20T20:40:21.038999] [INFO ] [MAIN ] PRCS: Starting full primary refresh. [2025-10-20T20:40:21.059999] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-05. [2025-10-20T20:40:21.410528] [INFO ] [REMOTE ] PRCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:21.851366] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-04. [2025-10-20T20:40:22.651397] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-07. [2025-10-20T20:40:22.726327] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T20:40:23.354701] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T20:40:23.451347] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-05. [2025-10-20T20:40:23.707027] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T20:40:24.251373] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-08. [2025-10-20T20:40:25.007724] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T20:40:25.056470] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-01. [2025-10-20T20:40:25.066826] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T20:40:25.691491] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T20:40:25.851365] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-02. [2025-10-20T20:40:26.362624] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-02 (0.51 seconds). [2025-10-20T20:40:26.651365] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-03. [2025-10-20T20:40:27.216984] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T20:40:27.451410] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-06. [2025-10-20T20:40:28.251384] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-06. [2025-10-20T20:40:29.051356] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-09. [2025-10-20T20:40:29.405704] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T20:40:29.598832] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T20:40:29.851392] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-04. [2025-10-20T20:40:30.268811] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T20:40:30.471257] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-04 (0.62 seconds). [2025-10-20T20:40:30.651419] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-07. [2025-10-20T20:40:31.451356] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-05. [2025-10-20T20:40:32.074207] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T20:40:32.158503] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T20:40:32.251379] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-06. [2025-10-20T20:40:32.703231] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T20:40:33.051373] [INFO ] [REMOTE ] PPT: Fetching primary data for 2025-10. [2025-10-20T20:40:33.851365] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-07. [2025-10-20T20:40:33.918052] [INFO ] [REMOTE ] PPT: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T20:40:33.920971] [INFO ] [REMOTE ] PPT: Full fetch complete (28424 records). [2025-10-20T20:40:33.921050] [DEBUG] [VALIDATOR ] PPT: Validating primary data. [2025-10-20T20:40:34.310780] [DEBUG] [VALIDATOR ] PPT: All expected trading days are present. [2025-10-20T20:40:34.310903] [DEBUG] [VALIDATOR ] PPT: Primary data validation passed. [2025-10-20T20:40:34.312021] [INFO ] [MAIN ] PPT: Fetched primary data is valid. [2025-10-20T20:40:34.312089] [DEBUG] [LOCAL ] PPT: Saving primary data. [2025-10-20T20:40:34.313779] [DEBUG] [LOCAL ] PPT: Trimmed 28424 records to 19733 (keeping 200 days of data) [2025-10-20T20:40:34.413979] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T20:40:34.445377] [DEBUG] [LOCAL ] PPT: Primary data saved (19733 records). [2025-10-20T20:40:34.445591] [INFO ] [MAIN ] PPT: Full history saved. [2025-10-20T20:40:34.445718] [INFO ] [MAIN ] Update Progress: 1714/2367 symbols completed. PPT took 33.4s. Estimated remaining: 1h 24m 0s. Rate: 7.8 symbols/min [2025-10-20T20:40:34.625275] [INFO ] [MAIN ] PRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:34.659334] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-07. [2025-10-20T20:40:34.662859] [DEBUG] [LOCAL ] PRG: Loaded 30087 existing records from primary storage. [2025-10-20T20:40:34.663366] [INFO ] [MAIN ] PRG: Loaded 30087 existing records. [2025-10-20T20:40:34.852511] [WARN ] [MAIN ] PRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:34.852836] [INFO ] [MAIN ] PRG: Wiping existing primary data before full refresh. [2025-10-20T20:40:34.852887] [INFO ] [LOCAL ] PRG: Starting deletion of primary data. [2025-10-20T20:40:34.853120] [DEBUG] [LOCAL ] PRG: Deleted primary data file. [2025-10-20T20:40:34.853161] [INFO ] [MAIN ] PRG: Starting full primary refresh. [2025-10-20T20:40:35.218558] [INFO ] [REMOTE ] PRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:35.451444] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-08. [2025-10-20T20:40:36.251378] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-08. [2025-10-20T20:40:36.638028] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T20:40:36.680762] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T20:40:36.770474] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T20:40:37.051370] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-09. [2025-10-20T20:40:37.851387] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-01. [2025-10-20T20:40:38.332944] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-09 (1.28 seconds). [2025-10-20T20:40:38.651464] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-09. [2025-10-20T20:40:39.144942] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T20:40:39.144824] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T20:40:39.451456] [INFO ] [REMOTE ] PRA: Fetching primary data for 2025-10. [2025-10-20T20:40:40.110773] [INFO ] [REMOTE ] PRA: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:40:40.115816] [INFO ] [REMOTE ] PRA: Full fetch complete (44863 records). [2025-10-20T20:40:40.115922] [DEBUG] [VALIDATOR ] PRA: Validating primary data. [2025-10-20T20:40:40.116524] [FATAL] [VALIDATOR ] PRA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:40.116618] [FATAL] [MAIN ] PRA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:40.116723] [WARN ] [MAIN ] PRA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:40.116810] [INFO ] [LOCAL ] PRA: Starting deletion of primary data. [2025-10-20T20:40:40.116884] [DEBUG] [LOCAL ] PRA: No primary data file found to delete. [2025-10-20T20:40:40.116920] [INFO ] [LOCAL ] PRA: Starting deletion of derivative data. [2025-10-20T20:40:40.117036] [DEBUG] [LOCAL ] PRA: Deleted derivative data file. [2025-10-20T20:40:40.117105] [INFO ] [MAIN ] Update Progress: 1715/2367 symbols completed. PRA took 29.8s. Estimated remaining: 1h 23m 52s. Rate: 7.8 symbols/min [2025-10-20T20:40:40.261064] [INFO ] [REMOTE ] PRCS: Fetching primary data for 2025-10. [2025-10-20T20:40:40.294978] [INFO ] [MAIN ] PRGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:40.337504] [DEBUG] [LOCAL ] PRGO: Loaded 47838 existing records from primary storage. [2025-10-20T20:40:40.337823] [INFO ] [MAIN ] PRGO: Loaded 47838 existing records. [2025-10-20T20:40:40.521546] [WARN ] [MAIN ] PRGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:40.521767] [INFO ] [MAIN ] PRGO: Wiping existing primary data before full refresh. [2025-10-20T20:40:40.521824] [INFO ] [LOCAL ] PRGO: Starting deletion of primary data. [2025-10-20T20:40:40.522040] [DEBUG] [LOCAL ] PRGO: Deleted primary data file. [2025-10-20T20:40:40.522096] [INFO ] [MAIN ] PRGO: Starting full primary refresh. [2025-10-20T20:40:40.800447] [INFO ] [REMOTE ] PRCS: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T20:40:40.811919] [INFO ] [REMOTE ] PRCS: Full fetch complete (3896 records). [2025-10-20T20:40:40.812210] [DEBUG] [VALIDATOR ] PRCS: Validating primary data. [2025-10-20T20:40:41.008705] [INFO ] [REMOTE ] PRGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:41.064977] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-02. [2025-10-20T20:40:41.253616] [DEBUG] [VALIDATOR ] PRCS: All expected trading days are present. [2025-10-20T20:40:41.253720] [DEBUG] [VALIDATOR ] PRCS: Primary data validation passed. [2025-10-20T20:40:41.253847] [INFO ] [MAIN ] PRCS: Fetched primary data is valid. [2025-10-20T20:40:41.253885] [DEBUG] [LOCAL ] PRCS: Saving primary data. [2025-10-20T20:40:41.254215] [DEBUG] [LOCAL ] PRCS: Trimmed 3896 records to 2786 (keeping 200 days of data) [2025-10-20T20:40:41.270257] [DEBUG] [LOCAL ] PRCS: Primary data saved (2786 records). [2025-10-20T20:40:41.270321] [INFO ] [MAIN ] PRCS: Full history saved. [2025-10-20T20:40:41.270386] [INFO ] [MAIN ] Update Progress: 1716/2367 symbols completed. PRCS took 20.6s. Estimated remaining: 1h 23m 42s. Rate: 7.8 symbols/min [2025-10-20T20:40:41.446481] [INFO ] [MAIN ] PRH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:40:41.453894] [DEBUG] [LOCAL ] PRH: Loaded 4808 existing records from primary storage. [2025-10-20T20:40:41.454092] [INFO ] [MAIN ] PRH: Loaded 4808 existing records. [2025-10-20T20:40:41.630463] [WARN ] [MAIN ] PRH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:40:41.630764] [INFO ] [MAIN ] PRH: Wiping existing primary data before full refresh. [2025-10-20T20:40:41.630844] [INFO ] [LOCAL ] PRH: Starting deletion of primary data. [2025-10-20T20:40:41.631131] [DEBUG] [LOCAL ] PRH: Deleted primary data file. [2025-10-20T20:40:41.631208] [INFO ] [MAIN ] PRH: Starting full primary refresh. [2025-10-20T20:40:41.861507] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-01. [2025-10-20T20:40:41.990488] [INFO ] [REMOTE ] PRH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:40:42.418960] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T20:40:42.651429] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-08. [2025-10-20T20:40:43.451384] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-03. [2025-10-20T20:40:43.630806] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:40:44.251377] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-01. [2025-10-20T20:40:44.586603] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T20:40:44.748805] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T20:40:45.054526] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-02. [2025-10-20T20:40:45.056388] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T20:40:45.851373] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-02. [2025-10-20T20:40:46.651433] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-09. [2025-10-20T20:40:46.693211] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T20:40:46.723481] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T20:40:47.451378] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-04. [2025-10-20T20:40:48.251357] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-03. [2025-10-20T20:40:48.568489] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T20:40:48.951908] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T20:40:49.051392] [INFO ] [REMOTE ] PR: Fetching primary data for 2025-10. [2025-10-20T20:40:49.755599] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:40:49.800971] [INFO ] [REMOTE ] PR: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:40:49.808310] [INFO ] [REMOTE ] PR: Full fetch complete (77373 records). [2025-10-20T20:40:49.808433] [DEBUG] [VALIDATOR ] PR: Validating primary data. [2025-10-20T20:40:49.809223] [FATAL] [VALIDATOR ] PR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:40:49.809317] [FATAL] [MAIN ] PR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:40:49.809359] [WARN ] [MAIN ] PR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:40:49.809426] [INFO ] [LOCAL ] PR: Starting deletion of primary data. [2025-10-20T20:40:49.809489] [DEBUG] [LOCAL ] PR: No primary data file found to delete. [2025-10-20T20:40:49.809523] [INFO ] [LOCAL ] PR: Starting deletion of derivative data. [2025-10-20T20:40:49.809676] [DEBUG] [LOCAL ] PR: Deleted derivative data file. [2025-10-20T20:40:49.809763] [INFO ] [MAIN ] Update Progress: 1717/2367 symbols completed. PR took 43.3s. Estimated remaining: 1h 23m 34s. Rate: 7.8 symbols/min [2025-10-20T20:40:49.851331] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-04. [2025-10-20T20:40:50.651440] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-03. [2025-10-20T20:40:51.296311] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T20:40:51.451376] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-05. [2025-10-20T20:40:51.610400] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T20:40:52.251437] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-05. [2025-10-20T20:40:52.620735] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T20:40:53.051379] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-06. [2025-10-20T20:40:53.566543] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T20:40:53.851353] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-06. [2025-10-20T20:40:54.335059] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T20:40:54.651375] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-04. [2025-10-20T20:40:55.426071] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T20:40:55.425220] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T20:40:55.451331] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-05. [2025-10-20T20:40:56.209261] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T20:40:56.251345] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-07. [2025-10-20T20:40:57.051382] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-07. [2025-10-20T20:40:57.851423] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-06. [2025-10-20T20:40:57.969540] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:40:58.405702] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T20:40:58.436515] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T20:40:58.651381] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-08. [2025-10-20T20:40:59.451390] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-07. [2025-10-20T20:41:00.092317] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T20:41:00.256603] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-08. [2025-10-20T20:41:00.268891] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T20:41:01.051400] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-08. [2025-10-20T20:41:01.737875] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T20:41:01.793866] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T20:41:01.851345] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-09. [2025-10-20T20:41:02.252304] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T20:41:02.651370] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-09. [2025-10-20T20:41:03.451346] [INFO ] [REMOTE ] PRH: Fetching primary data for 2025-10. [2025-10-20T20:41:03.983735] [INFO ] [REMOTE ] PRH: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:41:03.984866] [INFO ] [REMOTE ] PRH: Full fetch complete (7472 records). [2025-10-20T20:41:03.984932] [DEBUG] [VALIDATOR ] PRH: Validating primary data. [2025-10-20T20:41:04.261477] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-09. [2025-10-20T20:41:04.346966] [DEBUG] [VALIDATOR ] PRH: All expected trading days are present. [2025-10-20T20:41:04.347091] [DEBUG] [VALIDATOR ] PRH: Primary data validation passed. [2025-10-20T20:41:04.347298] [INFO ] [MAIN ] PRH: Fetched primary data is valid. [2025-10-20T20:41:04.347384] [DEBUG] [LOCAL ] PRH: Saving primary data. [2025-10-20T20:41:04.348074] [DEBUG] [LOCAL ] PRH: Trimmed 7472 records to 4740 (keeping 200 days of data) [2025-10-20T20:41:04.400730] [DEBUG] [LOCAL ] PRH: Primary data saved (4740 records). [2025-10-20T20:41:04.403554] [INFO ] [MAIN ] PRH: Full history saved. [2025-10-20T20:41:04.403722] [INFO ] [MAIN ] Update Progress: 1718/2367 symbols completed. PRH took 23.1s. Estimated remaining: 1h 23m 29s. Rate: 7.8 symbols/min [2025-10-20T20:41:04.405402] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T20:41:05.051427] [INFO ] [REMOTE ] PRGO: Fetching primary data for 2025-10. [2025-10-20T20:41:05.460348] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T20:41:05.813613] [INFO ] [REMOTE ] PRGO: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:41:05.821434] [INFO ] [REMOTE ] PRGO: Full fetch complete (68504 records). [2025-10-20T20:41:05.821616] [DEBUG] [VALIDATOR ] PRGO: Validating primary data. [2025-10-20T20:41:05.822371] [FATAL] [VALIDATOR ] PRGO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:41:05.822450] [FATAL] [MAIN ] PRGO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:41:05.822517] [WARN ] [MAIN ] PRGO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:41:05.822587] [INFO ] [LOCAL ] PRGO: Starting deletion of primary data. [2025-10-20T20:41:05.822685] [DEBUG] [LOCAL ] PRGO: No primary data file found to delete. [2025-10-20T20:41:05.822751] [INFO ] [LOCAL ] PRGO: Starting deletion of derivative data. [2025-10-20T20:41:05.822991] [DEBUG] [LOCAL ] PRGO: Deleted derivative data file. [2025-10-20T20:41:05.823085] [INFO ] [MAIN ] Update Progress: 1719/2367 symbols completed. PRGO took 25.7s. Estimated remaining: 1h 23m 19s. Rate: 7.8 symbols/min [2025-10-20T20:41:05.851361] [INFO ] [REMOTE ] PRG: Fetching primary data for 2025-10. [2025-10-20T20:41:06.507423] [INFO ] [REMOTE ] PRG: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:41:06.512020] [INFO ] [REMOTE ] PRG: Full fetch complete (44150 records). [2025-10-20T20:41:06.512103] [DEBUG] [VALIDATOR ] PRG: Validating primary data. [2025-10-20T20:41:06.512710] [FATAL] [VALIDATOR ] PRG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:41:06.512792] [FATAL] [MAIN ] PRG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:41:06.512835] [WARN ] [MAIN ] PRG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:41:06.512873] [INFO ] [LOCAL ] PRG: Starting deletion of primary data. [2025-10-20T20:41:06.512932] [DEBUG] [LOCAL ] PRG: No primary data file found to delete. [2025-10-20T20:41:06.512969] [INFO ] [LOCAL ] PRG: Starting deletion of derivative data. [2025-10-20T20:41:06.513082] [DEBUG] [LOCAL ] PRG: Deleted derivative data file. [2025-10-20T20:41:06.513433] [INFO ] [MAIN ] Update Progress: 1720/2367 symbols completed. PRG took 32.1s. Estimated remaining: 1h 23m 9s. Rate: 7.8 symbols/min [2025-10-20T20:41:07.142489] [INFO ] [MAIN ] PRKS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:07.229962] [INFO ] [MAIN ] PRLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:07.232948] [INFO ] [MAIN ] PRIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:07.234525] [INFO ] [MAIN ] PRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:07.261031] [DEBUG] [LOCAL ] PRLB: Loaded 20632 existing records from primary storage. [2025-10-20T20:41:07.261368] [INFO ] [MAIN ] PRLB: Loaded 20632 existing records. [2025-10-20T20:41:07.374681] [DEBUG] [LOCAL ] PRI: Loaded 18363 existing records from primary storage. [2025-10-20T20:41:07.390412] [INFO ] [MAIN ] PRI: Loaded 18363 existing records. [2025-10-20T20:41:07.580779] [DEBUG] [LOCAL ] PRIM: Loaded 44853 existing records from primary storage. [2025-10-20T20:41:07.569605] [DEBUG] [LOCAL ] PRKS: Loaded 43185 existing records from primary storage. [2025-10-20T20:41:07.608216] [WARN ] [MAIN ] PRLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:07.608617] [INFO ] [MAIN ] PRIM: Loaded 44853 existing records. [2025-10-20T20:41:07.614213] [INFO ] [MAIN ] PRLB: Wiping existing primary data before full refresh. [2025-10-20T20:41:07.639964] [INFO ] [MAIN ] PRKS: Loaded 43185 existing records. [2025-10-20T20:41:07.684677] [INFO ] [LOCAL ] PRLB: Starting deletion of primary data. [2025-10-20T20:41:07.755981] [DEBUG] [LOCAL ] PRLB: Deleted primary data file. [2025-10-20T20:41:07.756584] [INFO ] [MAIN ] PRLB: Starting full primary refresh. [2025-10-20T20:41:08.007021] [WARN ] [MAIN ] PRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:08.043442] [INFO ] [MAIN ] PRI: Wiping existing primary data before full refresh. [2025-10-20T20:41:08.074072] [INFO ] [LOCAL ] PRI: Starting deletion of primary data. [2025-10-20T20:41:08.141778] [DEBUG] [LOCAL ] PRI: Deleted primary data file. [2025-10-20T20:41:08.153127] [INFO ] [MAIN ] PRI: Starting full primary refresh. [2025-10-20T20:41:08.219976] [WARN ] [MAIN ] PRKS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:08.279076] [WARN ] [MAIN ] PRIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:08.282041] [INFO ] [MAIN ] PRIM: Wiping existing primary data before full refresh. [2025-10-20T20:41:08.317896] [INFO ] [LOCAL ] PRIM: Starting deletion of primary data. [2025-10-20T20:41:08.318058] [INFO ] [MAIN ] PRKS: Wiping existing primary data before full refresh. [2025-10-20T20:41:08.369730] [DEBUG] [LOCAL ] PRIM: Deleted primary data file. [2025-10-20T20:41:08.383798] [INFO ] [MAIN ] PRIM: Starting full primary refresh. [2025-10-20T20:41:08.364506] [INFO ] [LOCAL ] PRKS: Starting deletion of primary data. [2025-10-20T20:41:08.556012] [DEBUG] [LOCAL ] PRKS: Deleted primary data file. [2025-10-20T20:41:08.600605] [INFO ] [MAIN ] PRKS: Starting full primary refresh. [2025-10-20T20:41:09.002538] [INFO ] [REMOTE ] PRLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:09.046703] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-01. [2025-10-20T20:41:09.451826] [INFO ] [REMOTE ] PRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:09.506372] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-01. [2025-10-20T20:41:09.511181] [INFO ] [REMOTE ] PRIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:09.525607] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-01. [2025-10-20T20:41:09.645388] [INFO ] [REMOTE ] PRKS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:09.645739] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-01. [2025-10-20T20:41:10.270016] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T20:41:10.274219] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-02. [2025-10-20T20:41:10.900902] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T20:41:10.904468] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-02. [2025-10-20T20:41:11.143680] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T20:41:11.187526] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T20:41:11.252133] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T20:41:11.451371] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-02. [2025-10-20T20:41:12.251375] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-03. [2025-10-20T20:41:12.738060] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:41:13.051435] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-02. [2025-10-20T20:41:13.414000] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T20:41:13.851363] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-03. [2025-10-20T20:41:13.916622] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-02 (3.01 seconds). [2025-10-20T20:41:14.371555] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T20:41:14.651396] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-03. [2025-10-20T20:41:15.451402] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-03. [2025-10-20T20:41:15.841208] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T20:41:16.251361] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-04. [2025-10-20T20:41:17.051340] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-04. [2025-10-20T20:41:17.232772] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T20:41:17.391791] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-04 (1.14 seconds). [2025-10-20T20:41:17.851424] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-05. [2025-10-20T20:41:18.163336] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T20:41:18.651428] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-05. [2025-10-20T20:41:18.838414] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T20:41:19.451404] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-06. [2025-10-20T20:41:19.846189] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T20:41:20.251425] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-06. [2025-10-20T20:41:20.549312] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T20:41:21.051345] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-07. [2025-10-20T20:41:21.129039] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T20:41:21.851355] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-07. [2025-10-20T20:41:22.100825] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T20:41:22.651401] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-04. [2025-10-20T20:41:23.049000] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T20:41:23.451387] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-08. [2025-10-20T20:41:24.251345] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-08. [2025-10-20T20:41:24.282474] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T20:41:24.573926] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:41:25.051361] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-05. [2025-10-20T20:41:25.405877] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-08 (1.15 seconds). [2025-10-20T20:41:25.851399] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-09. [2025-10-20T20:41:26.651387] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-09. [2025-10-20T20:41:26.766489] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:41:26.807604] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T20:41:27.394714] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T20:41:27.451407] [INFO ] [REMOTE ] PRI: Fetching primary data for 2025-10. [2025-10-20T20:41:28.014876] [INFO ] [REMOTE ] PRI: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:41:28.018874] [INFO ] [REMOTE ] PRI: Full fetch complete (25547 records). [2025-10-20T20:41:28.019010] [DEBUG] [VALIDATOR ] PRI: Validating primary data. [2025-10-20T20:41:28.019623] [FATAL] [VALIDATOR ] PRI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:41:28.019743] [FATAL] [MAIN ] PRI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:41:28.019800] [WARN ] [MAIN ] PRI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:41:28.019881] [INFO ] [LOCAL ] PRI: Starting deletion of primary data. [2025-10-20T20:41:28.019964] [DEBUG] [LOCAL ] PRI: No primary data file found to delete. [2025-10-20T20:41:28.020017] [INFO ] [LOCAL ] PRI: Starting deletion of derivative data. [2025-10-20T20:41:28.020220] [DEBUG] [LOCAL ] PRI: Deleted derivative data file. [2025-10-20T20:41:28.020328] [INFO ] [MAIN ] Update Progress: 1721/2367 symbols completed. PRI took 21.5s. Estimated remaining: 1h 23m 6s. Rate: 7.8 symbols/min [2025-10-20T20:41:28.212885] [INFO ] [MAIN ] PRM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:28.254333] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-06. [2025-10-20T20:41:28.256591] [DEBUG] [LOCAL ] PRM: Loaded 45003 existing records from primary storage. [2025-10-20T20:41:28.257078] [INFO ] [MAIN ] PRM: Loaded 45003 existing records. [2025-10-20T20:41:28.433201] [WARN ] [MAIN ] PRM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:28.433422] [INFO ] [MAIN ] PRM: Wiping existing primary data before full refresh. [2025-10-20T20:41:28.433473] [INFO ] [LOCAL ] PRM: Starting deletion of primary data. [2025-10-20T20:41:28.433931] [DEBUG] [LOCAL ] PRM: Deleted primary data file. [2025-10-20T20:41:28.433985] [INFO ] [MAIN ] PRM: Starting full primary refresh. [2025-10-20T20:41:28.782810] [INFO ] [REMOTE ] PRM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:29.051401] [INFO ] [REMOTE ] PRLB: Fetching primary data for 2025-10. [2025-10-20T20:41:29.602761] [INFO ] [REMOTE ] PRLB: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:41:29.606003] [INFO ] [REMOTE ] PRLB: Full fetch complete (30163 records). [2025-10-20T20:41:29.606103] [DEBUG] [VALIDATOR ] PRLB: Validating primary data. [2025-10-20T20:41:29.606799] [FATAL] [VALIDATOR ] PRLB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:41:29.606882] [FATAL] [MAIN ] PRLB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:41:29.606926] [WARN ] [MAIN ] PRLB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:41:29.606991] [INFO ] [LOCAL ] PRLB: Starting deletion of primary data. [2025-10-20T20:41:29.607055] [DEBUG] [LOCAL ] PRLB: No primary data file found to delete. [2025-10-20T20:41:29.607090] [INFO ] [LOCAL ] PRLB: Starting deletion of derivative data. [2025-10-20T20:41:29.607210] [DEBUG] [LOCAL ] PRLB: Deleted derivative data file. [2025-10-20T20:41:29.607277] [INFO ] [MAIN ] Update Progress: 1722/2367 symbols completed. PRLB took 23.1s. Estimated remaining: 1h 22m 56s. Rate: 7.8 symbols/min [2025-10-20T20:41:29.797346] [INFO ] [MAIN ] PRMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:29.824096] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T20:41:29.856337] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-01. [2025-10-20T20:41:29.880001] [DEBUG] [LOCAL ] PRMB: Loaded 53905 existing records from primary storage. [2025-10-20T20:41:29.880307] [INFO ] [MAIN ] PRMB: Loaded 53905 existing records. [2025-10-20T20:41:30.053055] [WARN ] [MAIN ] PRMB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:30.053268] [INFO ] [MAIN ] PRMB: Wiping existing primary data before full refresh. [2025-10-20T20:41:30.053321] [INFO ] [LOCAL ] PRMB: Starting deletion of primary data. [2025-10-20T20:41:30.053515] [DEBUG] [LOCAL ] PRMB: Deleted primary data file. [2025-10-20T20:41:30.053560] [INFO ] [MAIN ] PRMB: Starting full primary refresh. [2025-10-20T20:41:30.401787] [INFO ] [REMOTE ] PRMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:30.651364] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-07. [2025-10-20T20:41:31.451412] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-01. [2025-10-20T20:41:31.552158] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:41:31.989085] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T20:41:32.251423] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-02. [2025-10-20T20:41:33.051368] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-08. [2025-10-20T20:41:33.139156] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T20:41:33.812053] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T20:41:33.851370] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-03. [2025-10-20T20:41:34.533128] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T20:41:34.651403] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-02. [2025-10-20T20:41:35.433563] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:41:35.451365] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-04. [2025-10-20T20:41:36.229799] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:41:36.251355] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-03. [2025-10-20T20:41:37.050997] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T20:41:37.060240] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-09. [2025-10-20T20:41:37.851362] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-05. [2025-10-20T20:41:38.118076] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T20:41:38.454537] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T20:41:38.651372] [INFO ] [REMOTE ] PRIM: Fetching primary data for 2025-10. [2025-10-20T20:41:39.069449] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T20:41:39.451382] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-04. [2025-10-20T20:41:39.957742] [INFO ] [REMOTE ] PRIM: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T20:41:39.963869] [INFO ] [REMOTE ] PRIM: Full fetch complete (63740 records). [2025-10-20T20:41:39.963983] [DEBUG] [VALIDATOR ] PRIM: Validating primary data. [2025-10-20T20:41:40.251346] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-06. [2025-10-20T20:41:40.344232] [DEBUG] [VALIDATOR ] PRIM: All expected trading days are present. [2025-10-20T20:41:40.344336] [DEBUG] [VALIDATOR ] PRIM: Primary data validation passed. [2025-10-20T20:41:40.345285] [INFO ] [MAIN ] PRIM: Fetched primary data is valid. [2025-10-20T20:41:40.345346] [DEBUG] [LOCAL ] PRIM: Saving primary data. [2025-10-20T20:41:40.346516] [DEBUG] [LOCAL ] PRIM: Trimmed 63740 records to 44184 (keeping 200 days of data) [2025-10-20T20:41:40.612089] [DEBUG] [LOCAL ] PRIM: Primary data saved (44184 records). [2025-10-20T20:41:40.612212] [INFO ] [MAIN ] PRIM: Full history saved. [2025-10-20T20:41:40.612286] [INFO ] [MAIN ] Update Progress: 1723/2367 symbols completed. PRIM took 34.1s. Estimated remaining: 1h 22m 50s. Rate: 7.8 symbols/min [2025-10-20T20:41:40.780785] [INFO ] [MAIN ] PRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:40.817847] [DEBUG] [LOCAL ] PRO: Loaded 40543 existing records from primary storage. [2025-10-20T20:41:40.818138] [INFO ] [MAIN ] PRO: Loaded 40543 existing records. [2025-10-20T20:41:40.999324] [WARN ] [MAIN ] PRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:40.999734] [INFO ] [MAIN ] PRO: Wiping existing primary data before full refresh. [2025-10-20T20:41:40.999863] [INFO ] [LOCAL ] PRO: Starting deletion of primary data. [2025-10-20T20:41:41.000173] [DEBUG] [LOCAL ] PRO: Deleted primary data file. [2025-10-20T20:41:41.000231] [INFO ] [MAIN ] PRO: Starting full primary refresh. [2025-10-20T20:41:41.366710] [INFO ] [REMOTE ] PRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:41.367212] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-01. [2025-10-20T20:41:41.397680] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T20:41:41.430950] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T20:41:41.851388] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-07. [2025-10-20T20:41:42.651358] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-05. [2025-10-20T20:41:42.806725] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:41:43.451376] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-02. [2025-10-20T20:41:43.603413] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T20:41:44.251347] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-08. [2025-10-20T20:41:44.589279] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T20:41:44.762135] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T20:41:45.051361] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-06. [2025-10-20T20:41:45.793065] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-03 (31.94 seconds). [2025-10-20T20:41:45.851337] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-04. [2025-10-20T20:41:45.968311] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T20:41:46.560734] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T20:41:46.651327] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-07. [2025-10-20T20:41:47.453448] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-03. [2025-10-20T20:41:47.456712] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T20:41:48.110319] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T20:41:48.251347] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-08. [2025-10-20T20:41:48.892141] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T20:41:49.051383] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-05. [2025-10-20T20:41:49.851416] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-09. [2025-10-20T20:41:50.159482] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T20:41:50.651401] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-09. [2025-10-20T20:41:50.678771] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T20:41:51.306571] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T20:41:51.451390] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-06. [2025-10-20T20:41:52.172614] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T20:41:52.251354] [INFO ] [REMOTE ] PRMB: Fetching primary data for 2025-10. [2025-10-20T20:41:52.917282] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T20:41:53.051352] [INFO ] [REMOTE ] PRM: Fetching primary data for 2025-10. [2025-10-20T20:41:53.851351] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-04. [2025-10-20T20:41:53.938139] [INFO ] [REMOTE ] PRMB: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T20:41:53.945917] [INFO ] [REMOTE ] PRMB: Full fetch complete (76341 records). [2025-10-20T20:41:53.946051] [DEBUG] [VALIDATOR ] PRMB: Validating primary data. [2025-10-20T20:41:54.343393] [DEBUG] [VALIDATOR ] PRMB: All expected trading days are present. [2025-10-20T20:41:54.343542] [DEBUG] [VALIDATOR ] PRMB: Primary data validation passed. [2025-10-20T20:41:54.344636] [INFO ] [MAIN ] PRMB: Fetched primary data is valid. [2025-10-20T20:41:54.344734] [DEBUG] [LOCAL ] PRMB: Saving primary data. [2025-10-20T20:41:54.346502] [DEBUG] [LOCAL ] PRMB: Trimmed 76341 records to 53123 (keeping 200 days of data) [2025-10-20T20:41:54.481283] [INFO ] [REMOTE ] PRM: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T20:41:54.490198] [INFO ] [REMOTE ] PRM: Full fetch complete (64775 records). [2025-10-20T20:41:54.493123] [DEBUG] [VALIDATOR ] PRM: Validating primary data. [2025-10-20T20:41:54.660250] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-07. [2025-10-20T20:41:55.093141] [DEBUG] [VALIDATOR ] PRM: All expected trading days are present. [2025-10-20T20:41:55.107144] [DEBUG] [LOCAL ] PRMB: Primary data saved (53123 records). [2025-10-20T20:41:55.107327] [INFO ] [MAIN ] PRMB: Full history saved. [2025-10-20T20:41:55.107414] [DEBUG] [VALIDATOR ] PRM: Primary data validation passed. [2025-10-20T20:41:55.107927] [INFO ] [MAIN ] Update Progress: 1724/2367 symbols completed. PRMB took 25.5s. Estimated remaining: 1h 22m 44s. Rate: 7.8 symbols/min [2025-10-20T20:41:55.109333] [INFO ] [MAIN ] PRM: Fetched primary data is valid. [2025-10-20T20:41:55.109489] [DEBUG] [LOCAL ] PRM: Saving primary data. [2025-10-20T20:41:55.211860] [DEBUG] [LOCAL ] PRM: Trimmed 64775 records to 44358 (keeping 200 days of data) [2025-10-20T20:41:55.358951] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T20:41:55.360619] [INFO ] [MAIN ] PRSU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:55.408842] [DEBUG] [LOCAL ] PRSU: No existing data file found in primary storage. [2025-10-20T20:41:55.411445] [WARN ] [MAIN ] PRSU: No existing data found, performing full refresh instead. [2025-10-20T20:41:55.411588] [INFO ] [MAIN ] PRSU: Wiping existing primary data before full refresh. [2025-10-20T20:41:55.412028] [INFO ] [LOCAL ] PRSU: Starting deletion of primary data. [2025-10-20T20:41:55.413740] [DEBUG] [LOCAL ] PRSU: No primary data file found to delete. [2025-10-20T20:41:55.421851] [INFO ] [MAIN ] PRSU: Starting full primary refresh. [2025-10-20T20:41:55.453672] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-05. [2025-10-20T20:41:55.905550] [DEBUG] [LOCAL ] PRM: Primary data saved (44358 records). [2025-10-20T20:41:55.911159] [INFO ] [MAIN ] PRM: Full history saved. [2025-10-20T20:41:55.919474] [INFO ] [MAIN ] Update Progress: 1725/2367 symbols completed. PRM took 27.9s. Estimated remaining: 1h 22m 34s. Rate: 7.8 symbols/min [2025-10-20T20:41:55.940139] [INFO ] [REMOTE ] PRSU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:56.105033] [INFO ] [MAIN ] PRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:41:56.112229] [DEBUG] [LOCAL ] PRT: Loaded 5219 existing records from primary storage. [2025-10-20T20:41:56.112404] [INFO ] [MAIN ] PRT: Loaded 5219 existing records. [2025-10-20T20:41:56.253496] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-01. [2025-10-20T20:41:56.293018] [WARN ] [MAIN ] PRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:41:56.293964] [INFO ] [MAIN ] PRT: Wiping existing primary data before full refresh. [2025-10-20T20:41:56.294091] [INFO ] [LOCAL ] PRT: Starting deletion of primary data. [2025-10-20T20:41:56.294275] [DEBUG] [LOCAL ] PRT: Deleted primary data file. [2025-10-20T20:41:56.294346] [INFO ] [MAIN ] PRT: Starting full primary refresh. [2025-10-20T20:41:56.593605] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T20:41:56.679336] [INFO ] [REMOTE ] PRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:41:57.033522] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T20:41:57.051327] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-08. [2025-10-20T20:41:57.441608] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T20:41:57.851358] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-01. [2025-10-20T20:41:58.417134] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T20:41:58.651390] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-02. [2025-10-20T20:41:58.730362] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T20:41:59.291457] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T20:41:59.451397] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-02. [2025-10-20T20:42:00.251421] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-06. [2025-10-20T20:42:00.728710] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T20:42:01.051373] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-03. [2025-10-20T20:42:01.838285] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T20:42:01.851365] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-03. [2025-10-20T20:42:01.870142] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:42:02.651426] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-07. [2025-10-20T20:42:03.163808] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T20:42:03.451411] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-04. [2025-10-20T20:42:03.887798] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-04 (0.44 seconds). [2025-10-20T20:42:03.914550] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T20:42:04.251443] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-04. [2025-10-20T20:42:05.051395] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-09. [2025-10-20T20:42:05.574564] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T20:42:05.851381] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-05. [2025-10-20T20:42:06.651345] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-05. [2025-10-20T20:42:06.685314] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T20:42:07.007013] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:42:07.186268] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-05 (0.53 seconds). [2025-10-20T20:42:07.451398] [INFO ] [REMOTE ] PRKS: Fetching primary data for 2025-10. [2025-10-20T20:42:08.251385] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-06. [2025-10-20T20:42:08.978363] [INFO ] [REMOTE ] PRKS: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T20:42:08.984929] [INFO ] [REMOTE ] PRKS: Full fetch complete (61704 records). [2025-10-20T20:42:08.985112] [DEBUG] [VALIDATOR ] PRKS: Validating primary data. [2025-10-20T20:42:09.056483] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-08. [2025-10-20T20:42:09.093316] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T20:42:09.415508] [DEBUG] [VALIDATOR ] PRKS: All expected trading days are present. [2025-10-20T20:42:09.415586] [DEBUG] [VALIDATOR ] PRKS: Primary data validation passed. [2025-10-20T20:42:09.416513] [INFO ] [MAIN ] PRKS: Fetched primary data is valid. [2025-10-20T20:42:09.416576] [DEBUG] [LOCAL ] PRKS: Saving primary data. [2025-10-20T20:42:09.417750] [DEBUG] [LOCAL ] PRKS: Trimmed 61704 records to 42462 (keeping 200 days of data) [2025-10-20T20:42:09.666036] [DEBUG] [LOCAL ] PRKS: Primary data saved (42462 records). [2025-10-20T20:42:09.666158] [INFO ] [MAIN ] PRKS: Full history saved. [2025-10-20T20:42:09.666233] [INFO ] [MAIN ] Update Progress: 1726/2367 symbols completed. PRKS took 1m 3s. Estimated remaining: 1h 22m 29s. Rate: 7.8 symbols/min [2025-10-20T20:42:09.836919] [INFO ] [MAIN ] PRU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:09.862477] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-06. [2025-10-20T20:42:09.890782] [DEBUG] [LOCAL ] PRU: Loaded 52007 existing records from primary storage. [2025-10-20T20:42:09.891069] [INFO ] [MAIN ] PRU: Loaded 52007 existing records. [2025-10-20T20:42:10.066091] [WARN ] [MAIN ] PRU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:10.066344] [INFO ] [MAIN ] PRU: Wiping existing primary data before full refresh. [2025-10-20T20:42:10.066406] [INFO ] [LOCAL ] PRU: Starting deletion of primary data. [2025-10-20T20:42:10.066678] [DEBUG] [LOCAL ] PRU: Deleted primary data file. [2025-10-20T20:42:10.066738] [INFO ] [MAIN ] PRU: Starting full primary refresh. [2025-10-20T20:42:10.425943] [INFO ] [REMOTE ] PRU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:10.515816] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T20:42:10.651372] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-07. [2025-10-20T20:42:10.781547] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T20:42:11.451427] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-07. [2025-10-20T20:42:11.809696] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T20:42:12.077764] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-07 (0.63 seconds). [2025-10-20T20:42:12.251377] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-08. [2025-10-20T20:42:12.784324] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T20:42:13.051415] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-09. [2025-10-20T20:42:13.542439] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T20:42:13.851368] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-08. [2025-10-20T20:42:14.651381] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-09. [2025-10-20T20:42:14.975825] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:42:15.451368] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-01. [2025-10-20T20:42:15.955721] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T20:42:16.251379] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-09. [2025-10-20T20:42:17.051388] [INFO ] [REMOTE ] PRO: Fetching primary data for 2025-10. [2025-10-20T20:42:17.347046] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T20:42:17.348127] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T20:42:17.851380] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-02. [2025-10-20T20:42:18.651405] [INFO ] [REMOTE ] PRT: Fetching primary data for 2025-10. [2025-10-20T20:42:18.701679] [INFO ] [REMOTE ] PRO: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T20:42:18.710343] [INFO ] [REMOTE ] PRO: Full fetch complete (53782 records). [2025-10-20T20:42:18.710569] [DEBUG] [VALIDATOR ] PRO: Validating primary data. [2025-10-20T20:42:19.082508] [DEBUG] [VALIDATOR ] PRO: All expected trading days are present. [2025-10-20T20:42:19.082596] [DEBUG] [VALIDATOR ] PRO: Primary data validation passed. [2025-10-20T20:42:19.083500] [INFO ] [MAIN ] PRO: Fetched primary data is valid. [2025-10-20T20:42:19.083563] [DEBUG] [LOCAL ] PRO: Saving primary data. [2025-10-20T20:42:19.084688] [DEBUG] [LOCAL ] PRO: Trimmed 53782 records to 40179 (keeping 200 days of data) [2025-10-20T20:42:19.247797] [INFO ] [REMOTE ] PRT: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T20:42:19.286929] [INFO ] [REMOTE ] PRT: Full fetch complete (8628 records). [2025-10-20T20:42:19.288148] [DEBUG] [VALIDATOR ] PRT: Validating primary data. [2025-10-20T20:42:19.456273] [INFO ] [REMOTE ] PRSU: Fetching primary data for 2025-10. [2025-10-20T20:42:19.489064] [DEBUG] [LOCAL ] PRO: Primary data saved (40179 records). [2025-10-20T20:42:19.503753] [INFO ] [MAIN ] PRO: Full history saved. [2025-10-20T20:42:19.513853] [INFO ] [MAIN ] Update Progress: 1727/2367 symbols completed. PRO took 38.9s. Estimated remaining: 1h 22m 22s. Rate: 7.8 symbols/min [2025-10-20T20:42:19.826475] [DEBUG] [VALIDATOR ] PRT: All expected trading days are present. [2025-10-20T20:42:19.844726] [DEBUG] [VALIDATOR ] PRT: Primary data validation passed. [2025-10-20T20:42:19.860151] [INFO ] [MAIN ] PRT: Fetched primary data is valid. [2025-10-20T20:42:19.876748] [DEBUG] [LOCAL ] PRT: Saving primary data. [2025-10-20T20:42:19.879143] [INFO ] [MAIN ] PRVS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:19.879673] [DEBUG] [LOCAL ] PRVS: No existing data file found in primary storage. [2025-10-20T20:42:19.880081] [DEBUG] [LOCAL ] PRT: Trimmed 8628 records to 5153 (keeping 200 days of data) [2025-10-20T20:42:19.880526] [WARN ] [MAIN ] PRVS: No existing data found, performing full refresh instead. [2025-10-20T20:42:19.881103] [INFO ] [MAIN ] PRVS: Wiping existing primary data before full refresh. [2025-10-20T20:42:19.917308] [INFO ] [LOCAL ] PRVS: Starting deletion of primary data. [2025-10-20T20:42:19.926683] [DEBUG] [LOCAL ] PRVS: No primary data file found to delete. [2025-10-20T20:42:19.927118] [INFO ] [MAIN ] PRVS: Starting full primary refresh. [2025-10-20T20:42:19.985414] [DEBUG] [LOCAL ] PRT: Primary data saved (5153 records). [2025-10-20T20:42:20.008916] [INFO ] [MAIN ] PRT: Full history saved. [2025-10-20T20:42:20.024557] [INFO ] [MAIN ] Update Progress: 1728/2367 symbols completed. PRT took 24.1s. Estimated remaining: 1h 22m 11s. Rate: 7.8 symbols/min [2025-10-20T20:42:20.008807] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-02 (2.16 seconds). [2025-10-20T20:42:20.083550] [INFO ] [REMOTE ] PRSU: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:42:20.210029] [INFO ] [REMOTE ] PRSU: Full fetch complete (34388 records). [2025-10-20T20:42:20.225371] [DEBUG] [VALIDATOR ] PRSU: Validating primary data. [2025-10-20T20:42:20.260711] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-03. [2025-10-20T20:42:20.320568] [FATAL] [VALIDATOR ] PRSU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:42:20.363537] [FATAL] [MAIN ] PRSU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:42:20.363718] [WARN ] [MAIN ] PRSU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:42:20.391755] [INFO ] [LOCAL ] PRSU: Starting deletion of primary data. [2025-10-20T20:42:20.420303] [INFO ] [MAIN ] PSA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:20.425816] [DEBUG] [LOCAL ] PRSU: No primary data file found to delete. [2025-10-20T20:42:20.425912] [INFO ] [LOCAL ] PRSU: Starting deletion of derivative data. [2025-10-20T20:42:20.438957] [DEBUG] [LOCAL ] PRSU: No derivative data file found to delete. [2025-10-20T20:42:20.450689] [INFO ] [MAIN ] Update Progress: 1729/2367 symbols completed. PRSU took 25.3s. Estimated remaining: 1h 22m 1s. Rate: 7.8 symbols/min [2025-10-20T20:42:20.648379] [INFO ] [REMOTE ] PRVS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:20.742388] [INFO ] [MAIN ] PSBD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:20.744386] [DEBUG] [LOCAL ] PSA: Loaded 42766 existing records from primary storage. [2025-10-20T20:42:20.744638] [INFO ] [MAIN ] PSA: Loaded 42766 existing records. [2025-10-20T20:42:20.858631] [DEBUG] [LOCAL ] PSBD: Loaded 8292 existing records from primary storage. [2025-10-20T20:42:20.869616] [INFO ] [MAIN ] PSBD: Loaded 8292 existing records. [2025-10-20T20:42:21.033859] [WARN ] [MAIN ] PSA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:21.044780] [INFO ] [MAIN ] PSA: Wiping existing primary data before full refresh. [2025-10-20T20:42:21.044908] [INFO ] [LOCAL ] PSA: Starting deletion of primary data. [2025-10-20T20:42:21.050599] [DEBUG] [LOCAL ] PSA: Deleted primary data file. [2025-10-20T20:42:21.050735] [INFO ] [MAIN ] PSA: Starting full primary refresh. [2025-10-20T20:42:21.055977] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-01. [2025-10-20T20:42:21.199541] [WARN ] [MAIN ] PSBD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:21.211175] [INFO ] [MAIN ] PSBD: Wiping existing primary data before full refresh. [2025-10-20T20:42:21.211274] [INFO ] [LOCAL ] PSBD: Starting deletion of primary data. [2025-10-20T20:42:21.233817] [DEBUG] [LOCAL ] PSBD: Deleted primary data file. [2025-10-20T20:42:21.239552] [INFO ] [MAIN ] PSBD: Starting full primary refresh. [2025-10-20T20:42:21.776123] [INFO ] [REMOTE ] PSA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:21.789963] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T20:42:21.854341] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-01. [2025-10-20T20:42:21.897224] [INFO ] [REMOTE ] PSBD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:22.273888] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T20:42:22.651428] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-01. [2025-10-20T20:42:23.452188] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-04. [2025-10-20T20:42:23.457201] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:42:23.595071] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T20:42:24.251428] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-02. [2025-10-20T20:42:25.051442] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-02. [2025-10-20T20:42:25.767847] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-04 (2.32 seconds). [2025-10-20T20:42:25.822441] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:42:25.845636] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T20:42:25.852483] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-03. [2025-10-20T20:42:26.481314] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T20:42:26.651371] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-05. [2025-10-20T20:42:27.451394] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-03. [2025-10-20T20:42:28.251447] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-02. [2025-10-20T20:42:28.651941] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-10-20T20:42:28.654745] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T20:42:29.019205] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T20:42:29.051394] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-04. [2025-10-20T20:42:29.851397] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-03. [2025-10-20T20:42:30.338830] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T20:42:30.651371] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-04. [2025-10-20T20:42:30.764560] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T20:42:31.361476] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T20:42:31.451439] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-04. [2025-10-20T20:42:31.858027] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T20:42:32.251372] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-05. [2025-10-20T20:42:33.051370] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-06. [2025-10-20T20:42:33.851372] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-05. [2025-10-20T20:42:34.017296] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T20:42:34.535118] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T20:42:34.565762] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T20:42:34.651361] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-07. [2025-10-20T20:42:35.451359] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-06. [2025-10-20T20:42:36.227019] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T20:42:36.251394] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-07. [2025-10-20T20:42:36.649442] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T20:42:37.006898] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T20:42:37.051341] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-05. [2025-10-20T20:42:37.473547] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T20:42:37.851451] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-08. [2025-10-20T20:42:38.651386] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-08. [2025-10-20T20:42:39.447886] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T20:42:39.451322] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-09. [2025-10-20T20:42:39.737836] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T20:42:40.035528] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T20:42:40.251418] [INFO ] [REMOTE ] PSBD: Fetching primary data for 2025-10. [2025-10-20T20:42:41.051486] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-09. [2025-10-20T20:42:41.851682] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-06. [2025-10-20T20:42:42.295989] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-06 (0.44 seconds). [2025-10-20T20:42:42.651362] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-06. [2025-10-20T20:42:42.858622] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T20:42:43.451378] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-07. [2025-10-20T20:42:43.839244] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T20:42:43.991305] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T20:42:44.251448] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-07. [2025-10-20T20:42:45.051438] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-08. [2025-10-20T20:42:45.851378] [INFO ] [REMOTE ] PRU: Fetching primary data for 2025-10. [2025-10-20T20:42:45.975110] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T20:42:46.438700] [INFO ] [REMOTE ] PRU: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T20:42:46.445638] [INFO ] [REMOTE ] PRU: Full fetch complete (74226 records). [2025-10-20T20:42:46.445808] [DEBUG] [VALIDATOR ] PRU: Validating primary data. [2025-10-20T20:42:46.446565] [FATAL] [VALIDATOR ] PRU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:42:46.446704] [FATAL] [MAIN ] PRU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:42:46.446774] [WARN ] [MAIN ] PRU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:42:46.446831] [INFO ] [LOCAL ] PRU: Starting deletion of primary data. [2025-10-20T20:42:46.446932] [DEBUG] [LOCAL ] PRU: No primary data file found to delete. [2025-10-20T20:42:46.446976] [INFO ] [LOCAL ] PRU: Starting deletion of derivative data. [2025-10-20T20:42:46.447133] [DEBUG] [LOCAL ] PRU: Deleted derivative data file. [2025-10-20T20:42:46.447223] [INFO ] [MAIN ] Update Progress: 1730/2367 symbols completed. PRU took 36.8s. Estimated remaining: 1h 22m 0s. Rate: 7.8 symbols/min [2025-10-20T20:42:46.625071] [INFO ] [MAIN ] PSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:46.633978] [DEBUG] [LOCAL ] PSF: Loaded 7069 existing records from primary storage. [2025-10-20T20:42:46.634187] [INFO ] [MAIN ] PSF: Loaded 7069 existing records. [2025-10-20T20:42:46.665506] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-08. [2025-10-20T20:42:46.821519] [WARN ] [MAIN ] PSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:46.821765] [INFO ] [MAIN ] PSF: Wiping existing primary data before full refresh. [2025-10-20T20:42:46.821820] [INFO ] [LOCAL ] PSF: Starting deletion of primary data. [2025-10-20T20:42:46.821960] [DEBUG] [LOCAL ] PSF: Deleted primary data file. [2025-10-20T20:42:46.822002] [INFO ] [MAIN ] PSF: Starting full primary refresh. [2025-10-20T20:42:47.182830] [INFO ] [REMOTE ] PSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:47.451354] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-01. [2025-10-20T20:42:48.340537] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T20:42:48.344397] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-02. [2025-10-20T20:42:48.383044] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T20:42:49.018516] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T20:42:49.051357] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-03. [2025-10-20T20:42:49.713931] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:42:49.851374] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-09. [2025-10-20T20:42:50.651379] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-04. [2025-10-20T20:42:51.403331] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T20:42:51.451414] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-05. [2025-10-20T20:42:51.527707] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T20:42:52.251449] [INFO ] [REMOTE ] PSA: Fetching primary data for 2025-10. [2025-10-20T20:42:52.252318] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T20:42:52.706884] [INFO ] [REMOTE ] PSA: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T20:42:52.712547] [INFO ] [REMOTE ] PSA: Full fetch complete (59330 records). [2025-10-20T20:42:52.712684] [DEBUG] [VALIDATOR ] PSA: Validating primary data. [2025-10-20T20:42:52.713355] [FATAL] [VALIDATOR ] PSA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:42:52.713436] [FATAL] [MAIN ] PSA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:42:52.713505] [WARN ] [MAIN ] PSA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:42:52.713574] [INFO ] [LOCAL ] PSA: Starting deletion of primary data. [2025-10-20T20:42:52.713637] [DEBUG] [LOCAL ] PSA: No primary data file found to delete. [2025-10-20T20:42:52.713723] [INFO ] [LOCAL ] PSA: Starting deletion of derivative data. [2025-10-20T20:42:52.713847] [DEBUG] [LOCAL ] PSA: Deleted derivative data file. [2025-10-20T20:42:52.713913] [INFO ] [MAIN ] Update Progress: 1731/2367 symbols completed. PSA took 32.7s. Estimated remaining: 1h 21m 52s. Rate: 7.8 symbols/min [2025-10-20T20:42:52.882317] [INFO ] [MAIN ] PSFE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:52.912638] [DEBUG] [LOCAL ] PSFE: Loaded 30299 existing records from primary storage. [2025-10-20T20:42:52.912904] [INFO ] [MAIN ] PSFE: Loaded 30299 existing records. [2025-10-20T20:42:53.056536] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-06. [2025-10-20T20:42:53.096222] [WARN ] [MAIN ] PSFE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:53.096522] [INFO ] [MAIN ] PSFE: Wiping existing primary data before full refresh. [2025-10-20T20:42:53.096573] [INFO ] [LOCAL ] PSFE: Starting deletion of primary data. [2025-10-20T20:42:53.096786] [DEBUG] [LOCAL ] PSFE: Deleted primary data file. [2025-10-20T20:42:53.096841] [INFO ] [MAIN ] PSFE: Starting full primary refresh. [2025-10-20T20:42:53.451578] [INFO ] [REMOTE ] PSFE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:53.771695] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T20:42:53.851391] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-07. [2025-10-20T20:42:54.347082] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-08 (9.30 seconds). [2025-10-20T20:42:54.506151] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T20:42:54.651441] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-08. [2025-10-20T20:42:54.758911] [INFO ] [REMOTE ] PSBD: Incremental fetch complete for 2025-10 (14.51 seconds). [2025-10-20T20:42:54.761807] [INFO ] [REMOTE ] PSBD: Full fetch complete (10822 records). [2025-10-20T20:42:54.761894] [DEBUG] [VALIDATOR ] PSBD: Validating primary data. [2025-10-20T20:42:55.110986] [DEBUG] [VALIDATOR ] PSBD: All expected trading days are present. [2025-10-20T20:42:55.111052] [DEBUG] [VALIDATOR ] PSBD: Primary data validation passed. [2025-10-20T20:42:55.111299] [INFO ] [MAIN ] PSBD: Fetched primary data is valid. [2025-10-20T20:42:55.111335] [DEBUG] [LOCAL ] PSBD: Saving primary data. [2025-10-20T20:42:55.111783] [DEBUG] [LOCAL ] PSBD: Trimmed 10822 records to 8153 (keeping 200 days of data) [2025-10-20T20:42:55.159100] [DEBUG] [LOCAL ] PSBD: Primary data saved (8153 records). [2025-10-20T20:42:55.159195] [INFO ] [MAIN ] PSBD: Full history saved. [2025-10-20T20:42:55.159262] [INFO ] [MAIN ] Update Progress: 1732/2367 symbols completed. PSBD took 34.7s. Estimated remaining: 1h 21m 42s. Rate: 7.8 symbols/min [2025-10-20T20:42:55.324382] [INFO ] [MAIN ] PSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:55.366130] [DEBUG] [LOCAL ] PSN: Loaded 46125 existing records from primary storage. [2025-10-20T20:42:55.366395] [INFO ] [MAIN ] PSN: Loaded 46125 existing records. [2025-10-20T20:42:55.472088] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T20:42:55.453374] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-09. [2025-10-20T20:42:55.554448] [WARN ] [MAIN ] PSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:42:55.554689] [INFO ] [MAIN ] PSN: Wiping existing primary data before full refresh. [2025-10-20T20:42:55.554747] [INFO ] [LOCAL ] PSN: Starting deletion of primary data. [2025-10-20T20:42:55.555003] [DEBUG] [LOCAL ] PSN: Deleted primary data file. [2025-10-20T20:42:55.555058] [INFO ] [MAIN ] PSN: Starting full primary refresh. [2025-10-20T20:42:55.884512] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-09 (0.43 seconds). [2025-10-20T20:42:55.903172] [INFO ] [REMOTE ] PSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:56.251399] [INFO ] [REMOTE ] PRVS: Fetching primary data for 2025-10. [2025-10-20T20:42:56.655314] [INFO ] [REMOTE ] PRVS: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T20:42:56.656065] [INFO ] [REMOTE ] PRVS: Full fetch complete (1236 records). [2025-10-20T20:42:56.656166] [DEBUG] [VALIDATOR ] PRVS: Validating primary data. [2025-10-20T20:42:56.835918] [WARN ] [VALIDATOR ] PRVS: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T20:42:56.836101] [FATAL] [MAIN ] PRVS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:42:56.836169] [WARN ] [MAIN ] PRVS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:42:56.836240] [INFO ] [LOCAL ] PRVS: Starting deletion of primary data. [2025-10-20T20:42:56.836299] [DEBUG] [LOCAL ] PRVS: No primary data file found to delete. [2025-10-20T20:42:56.836334] [INFO ] [LOCAL ] PRVS: Starting deletion of derivative data. [2025-10-20T20:42:56.836372] [DEBUG] [LOCAL ] PRVS: No derivative data file found to delete. [2025-10-20T20:42:56.836424] [INFO ] [MAIN ] Update Progress: 1733/2367 symbols completed. PRVS took 37.3s. Estimated remaining: 1h 21m 32s. Rate: 7.8 symbols/min [2025-10-20T20:42:56.999330] [INFO ] [MAIN ] PSO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:42:56.999433] [DEBUG] [LOCAL ] PSO: No existing data file found in primary storage. [2025-10-20T20:42:56.999468] [WARN ] [MAIN ] PSO: No existing data found, performing full refresh instead. [2025-10-20T20:42:56.999565] [INFO ] [MAIN ] PSO: Wiping existing primary data before full refresh. [2025-10-20T20:42:56.999604] [INFO ] [LOCAL ] PSO: Starting deletion of primary data. [2025-10-20T20:42:56.999699] [DEBUG] [LOCAL ] PSO: No primary data file found to delete. [2025-10-20T20:42:56.999748] [INFO ] [MAIN ] PSO: Starting full primary refresh. [2025-10-20T20:42:57.060171] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-09. [2025-10-20T20:42:57.352229] [INFO ] [REMOTE ] PSO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:42:57.851405] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-01. [2025-10-20T20:42:57.986011] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:42:58.651412] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-01. [2025-10-20T20:42:59.355004] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T20:42:59.451374] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-02. [2025-10-20T20:42:59.624806] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T20:43:00.251366] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-01. [2025-10-20T20:43:00.802938] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T20:43:01.051417] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-03. [2025-10-20T20:43:01.851359] [INFO ] [REMOTE ] PSF: Fetching primary data for 2025-10. [2025-10-20T20:43:01.890289] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T20:43:02.466721] [INFO ] [REMOTE ] PSF: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:43:02.468203] [INFO ] [REMOTE ] PSF: Full fetch complete (9712 records). [2025-10-20T20:43:02.468269] [DEBUG] [VALIDATOR ] PSF: Validating primary data. [2025-10-20T20:43:02.657599] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-02. [2025-10-20T20:43:02.850299] [DEBUG] [VALIDATOR ] PSF: All expected trading days are present. [2025-10-20T20:43:02.852635] [DEBUG] [VALIDATOR ] PSF: Primary data validation passed. [2025-10-20T20:43:02.852953] [INFO ] [MAIN ] PSF: Fetched primary data is valid. [2025-10-20T20:43:02.853456] [DEBUG] [LOCAL ] PSF: Saving primary data. [2025-10-20T20:43:02.854271] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T20:43:02.863389] [DEBUG] [LOCAL ] PSF: Trimmed 9712 records to 7015 (keeping 200 days of data) [2025-10-20T20:43:02.904881] [DEBUG] [LOCAL ] PSF: Primary data saved (7015 records). [2025-10-20T20:43:02.904981] [INFO ] [MAIN ] PSF: Full history saved. [2025-10-20T20:43:02.905058] [INFO ] [MAIN ] Update Progress: 1734/2367 symbols completed. PSF took 16.5s. Estimated remaining: 1h 21m 24s. Rate: 7.8 symbols/min [2025-10-20T20:43:03.075359] [INFO ] [MAIN ] PSQH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:03.112177] [DEBUG] [LOCAL ] PSQH: Loaded 36091 existing records from primary storage. [2025-10-20T20:43:03.112467] [INFO ] [MAIN ] PSQH: Loaded 36091 existing records. [2025-10-20T20:43:03.292795] [WARN ] [MAIN ] PSQH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:03.293008] [INFO ] [MAIN ] PSQH: Wiping existing primary data before full refresh. [2025-10-20T20:43:03.293057] [INFO ] [LOCAL ] PSQH: Starting deletion of primary data. [2025-10-20T20:43:03.293247] [DEBUG] [LOCAL ] PSQH: Deleted primary data file. [2025-10-20T20:43:03.293286] [INFO ] [MAIN ] PSQH: Starting full primary refresh. [2025-10-20T20:43:03.453735] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-04. [2025-10-20T20:43:03.658222] [INFO ] [REMOTE ] PSQH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:04.122199] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T20:43:04.251360] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-01. [2025-10-20T20:43:05.051363] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-03. [2025-10-20T20:43:05.162162] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T20:43:05.851408] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-05. [2025-10-20T20:43:06.026974] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T20:43:06.546738] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T20:43:06.651342] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-04. [2025-10-20T20:43:07.321695] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T20:43:07.451392] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-02. [2025-10-20T20:43:08.001587] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T20:43:08.251375] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-06. [2025-10-20T20:43:09.051360] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-02. [2025-10-20T20:43:09.142108] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T20:43:09.492590] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T20:43:09.851424] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-05. [2025-10-20T20:43:10.373862] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T20:43:10.651384] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-07. [2025-10-20T20:43:11.455221] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-03. [2025-10-20T20:43:11.459399] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T20:43:12.252793] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-06. [2025-10-20T20:43:12.255006] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T20:43:13.058812] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-03. [2025-10-20T20:43:13.079176] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T20:43:13.448906] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T20:43:13.851441] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-04. [2025-10-20T20:43:14.651355] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-07. [2025-10-20T20:43:14.728133] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T20:43:15.451352] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-04. [2025-10-20T20:43:15.592917] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:43:15.766254] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T20:43:16.251359] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-08. [2025-10-20T20:43:16.960876] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T20:43:17.051445] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-05. [2025-10-20T20:43:17.368756] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T20:43:17.851368] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-05. [2025-10-20T20:43:18.651357] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-08. [2025-10-20T20:43:18.872495] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T20:43:19.451451] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-06. [2025-10-20T20:43:19.502584] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T20:43:20.146579] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T20:43:20.251334] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-09. [2025-10-20T20:43:21.060346] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-09. [2025-10-20T20:43:21.079069] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T20:43:21.649743] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T20:43:21.851351] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-07. [2025-10-20T20:43:22.264701] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T20:43:22.651442] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-06. [2025-10-20T20:43:23.227342] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T20:43:23.451370] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-08. [2025-10-20T20:43:24.109486] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T20:43:24.251424] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-07. [2025-10-20T20:43:25.012115] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T20:43:25.051335] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-09. [2025-10-20T20:43:25.606203] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T20:43:25.851383] [INFO ] [REMOTE ] PSFE: Fetching primary data for 2025-10. [2025-10-20T20:43:26.657229] [INFO ] [REMOTE ] PSO: Fetching primary data for 2025-10. [2025-10-20T20:43:26.660425] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T20:43:26.797588] [INFO ] [REMOTE ] PSFE: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T20:43:26.801942] [INFO ] [REMOTE ] PSFE: Full fetch complete (43094 records). [2025-10-20T20:43:26.802032] [DEBUG] [VALIDATOR ] PSFE: Validating primary data. [2025-10-20T20:43:27.193565] [DEBUG] [VALIDATOR ] PSFE: All expected trading days are present. [2025-10-20T20:43:27.193636] [DEBUG] [VALIDATOR ] PSFE: Primary data validation passed. [2025-10-20T20:43:27.194417] [INFO ] [MAIN ] PSFE: Fetched primary data is valid. [2025-10-20T20:43:27.194459] [DEBUG] [LOCAL ] PSFE: Saving primary data. [2025-10-20T20:43:27.195296] [DEBUG] [LOCAL ] PSFE: Trimmed 43094 records to 29874 (keeping 200 days of data) [2025-10-20T20:43:27.358881] [DEBUG] [LOCAL ] PSFE: Primary data saved (29874 records). [2025-10-20T20:43:27.359006] [INFO ] [MAIN ] PSFE: Full history saved. [2025-10-20T20:43:27.359093] [INFO ] [MAIN ] Update Progress: 1735/2367 symbols completed. PSFE took 34.6s. Estimated remaining: 1h 21m 22s. Rate: 7.8 symbols/min [2025-10-20T20:43:27.457132] [INFO ] [REMOTE ] PSN: Fetching primary data for 2025-10. [2025-10-20T20:43:27.525204] [INFO ] [MAIN ] PSTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:27.573492] [DEBUG] [LOCAL ] PSTG: Loaded 53630 existing records from primary storage. [2025-10-20T20:43:27.573808] [INFO ] [MAIN ] PSTG: Loaded 53630 existing records. [2025-10-20T20:43:27.759373] [WARN ] [MAIN ] PSTG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:27.781180] [INFO ] [MAIN ] PSTG: Wiping existing primary data before full refresh. [2025-10-20T20:43:27.781257] [INFO ] [LOCAL ] PSTG: Starting deletion of primary data. [2025-10-20T20:43:27.782199] [DEBUG] [LOCAL ] PSTG: Deleted primary data file. [2025-10-20T20:43:27.782152] [INFO ] [REMOTE ] PSO: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T20:43:27.782375] [INFO ] [MAIN ] PSTG: Starting full primary refresh. [2025-10-20T20:43:27.854673] [INFO ] [REMOTE ] PSO: Full fetch complete (52757 records). [2025-10-20T20:43:27.866069] [DEBUG] [VALIDATOR ] PSO: Validating primary data. [2025-10-20T20:43:28.256703] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-08. [2025-10-20T20:43:28.417003] [INFO ] [REMOTE ] PSTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:28.443415] [INFO ] [REMOTE ] PSN: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T20:43:28.508175] [INFO ] [REMOTE ] PSN: Full fetch complete (66761 records). [2025-10-20T20:43:28.523964] [DEBUG] [VALIDATOR ] PSN: Validating primary data. [2025-10-20T20:43:28.553766] [FATAL] [VALIDATOR ] PSN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:43:28.556123] [FATAL] [MAIN ] PSN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:43:28.574931] [WARN ] [MAIN ] PSN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:43:28.587822] [DEBUG] [VALIDATOR ] PSO: All expected trading days are present. [2025-10-20T20:43:28.587888] [DEBUG] [VALIDATOR ] PSO: Primary data validation passed. [2025-10-20T20:43:28.588876] [INFO ] [MAIN ] PSO: Fetched primary data is valid. [2025-10-20T20:43:28.587991] [INFO ] [LOCAL ] PSN: Starting deletion of primary data. [2025-10-20T20:43:28.589058] [DEBUG] [LOCAL ] PSO: Saving primary data. [2025-10-20T20:43:28.589005] [DEBUG] [LOCAL ] PSN: No primary data file found to delete. [2025-10-20T20:43:28.589260] [INFO ] [LOCAL ] PSN: Starting deletion of derivative data. [2025-10-20T20:43:28.589614] [DEBUG] [LOCAL ] PSN: Deleted derivative data file. [2025-10-20T20:43:28.590192] [INFO ] [MAIN ] Update Progress: 1736/2367 symbols completed. PSN took 33.4s. Estimated remaining: 1h 21m 12s. Rate: 7.8 symbols/min [2025-10-20T20:43:28.663826] [DEBUG] [LOCAL ] PSO: Trimmed 52757 records to 36830 (keeping 200 days of data) [2025-10-20T20:43:28.857551] [INFO ] [MAIN ] PSTL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:28.978224] [DEBUG] [LOCAL ] PSTL: Loaded 20382 existing records from primary storage. [2025-10-20T20:43:28.988471] [DEBUG] [LOCAL ] PSO: Primary data saved (36830 records). [2025-10-20T20:43:28.988568] [INFO ] [MAIN ] PSO: Full history saved. [2025-10-20T20:43:28.988685] [INFO ] [MAIN ] Update Progress: 1737/2367 symbols completed. PSO took 32.2s. Estimated remaining: 1h 21m 2s. Rate: 7.8 symbols/min [2025-10-20T20:43:28.999914] [INFO ] [MAIN ] PSTL: Loaded 20382 existing records. [2025-10-20T20:43:29.067315] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-01. [2025-10-20T20:43:29.307683] [INFO ] [MAIN ] PSX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:29.349820] [WARN ] [MAIN ] PSTL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:29.350124] [INFO ] [MAIN ] PSTL: Wiping existing primary data before full refresh. [2025-10-20T20:43:29.350183] [INFO ] [LOCAL ] PSTL: Starting deletion of primary data. [2025-10-20T20:43:29.350393] [DEBUG] [LOCAL ] PSTL: Deleted primary data file. [2025-10-20T20:43:29.350449] [INFO ] [MAIN ] PSTL: Starting full primary refresh. [2025-10-20T20:43:29.680722] [DEBUG] [LOCAL ] PSX: Loaded 53890 existing records from primary storage. [2025-10-20T20:43:29.699967] [INFO ] [MAIN ] PSX: Loaded 53890 existing records. [2025-10-20T20:43:29.778103] [INFO ] [REMOTE ] PSTL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:29.866935] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-01. [2025-10-20T20:43:29.960049] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T20:43:29.979412] [WARN ] [MAIN ] PSX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:29.986579] [INFO ] [MAIN ] PSX: Wiping existing primary data before full refresh. [2025-10-20T20:43:29.986764] [INFO ] [LOCAL ] PSX: Starting deletion of primary data. [2025-10-20T20:43:29.987137] [DEBUG] [LOCAL ] PSX: Deleted primary data file. [2025-10-20T20:43:29.987194] [INFO ] [MAIN ] PSX: Starting full primary refresh. [2025-10-20T20:43:30.338184] [INFO ] [REMOTE ] PSX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:30.651384] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-01. [2025-10-20T20:43:30.802768] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T20:43:31.093012] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T20:43:31.451349] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-02. [2025-10-20T20:43:32.251355] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-09. [2025-10-20T20:43:32.498840] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T20:43:32.535490] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T20:43:33.051420] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-03. [2025-10-20T20:43:33.851367] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-02. [2025-10-20T20:43:34.021969] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T20:43:34.078501] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T20:43:34.651378] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-04. [2025-10-20T20:43:35.451345] [INFO ] [REMOTE ] PSQH: Fetching primary data for 2025-10. [2025-10-20T20:43:35.733695] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T20:43:35.874931] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-04 (1.22 seconds). [2025-10-20T20:43:36.251370] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-02. [2025-10-20T20:43:37.051383] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-03. [2025-10-20T20:43:37.150231] [INFO ] [REMOTE ] PSQH: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T20:43:37.156454] [INFO ] [REMOTE ] PSQH: Full fetch complete (53331 records). [2025-10-20T20:43:37.156555] [DEBUG] [VALIDATOR ] PSQH: Validating primary data. [2025-10-20T20:43:37.543077] [DEBUG] [VALIDATOR ] PSQH: All expected trading days are present. [2025-10-20T20:43:37.543197] [DEBUG] [VALIDATOR ] PSQH: Primary data validation passed. [2025-10-20T20:43:37.544099] [INFO ] [MAIN ] PSQH: Fetched primary data is valid. [2025-10-20T20:43:37.544162] [DEBUG] [LOCAL ] PSQH: Saving primary data. [2025-10-20T20:43:37.545285] [DEBUG] [LOCAL ] PSQH: Trimmed 53331 records to 35389 (keeping 200 days of data) [2025-10-20T20:43:37.756053] [DEBUG] [LOCAL ] PSQH: Primary data saved (35389 records). [2025-10-20T20:43:37.756209] [INFO ] [MAIN ] PSQH: Full history saved. [2025-10-20T20:43:37.756293] [INFO ] [MAIN ] Update Progress: 1738/2367 symbols completed. PSQH took 34.9s. Estimated remaining: 1h 20m 54s. Rate: 7.8 symbols/min [2025-10-20T20:43:37.856940] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-05. [2025-10-20T20:43:37.956358] [INFO ] [MAIN ] PTA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:37.990195] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T20:43:38.014978] [DEBUG] [LOCAL ] PTA: Loaded 15411 existing records from primary storage. [2025-10-20T20:43:38.015286] [INFO ] [MAIN ] PTA: Loaded 15411 existing records. [2025-10-20T20:43:38.237902] [WARN ] [MAIN ] PTA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:38.238202] [INFO ] [MAIN ] PTA: Wiping existing primary data before full refresh. [2025-10-20T20:43:38.238267] [INFO ] [LOCAL ] PTA: Starting deletion of primary data. [2025-10-20T20:43:38.238449] [DEBUG] [LOCAL ] PTA: Deleted primary data file. [2025-10-20T20:43:38.238506] [INFO ] [MAIN ] PTA: Starting full primary refresh. [2025-10-20T20:43:38.656591] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-03. [2025-10-20T20:43:38.761552] [INFO ] [REMOTE ] PTA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:38.967524] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T20:43:39.003371] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T20:43:39.451386] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-04. [2025-10-20T20:43:40.251343] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-01. [2025-10-20T20:43:40.574751] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T20:43:41.051342] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-04. [2025-10-20T20:43:41.094449] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T20:43:41.184994] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T20:43:41.851381] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-02. [2025-10-20T20:43:42.651367] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-05. [2025-10-20T20:43:42.685761] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T20:43:43.051548] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T20:43:43.451378] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-03. [2025-10-20T20:43:44.251348] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-06. [2025-10-20T20:43:44.563776] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T20:43:44.600108] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T20:43:45.007932] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T20:43:45.051400] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-07. [2025-10-20T20:43:45.851426] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-04. [2025-10-20T20:43:46.108616] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T20:43:46.651352] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-06. [2025-10-20T20:43:46.779791] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:43:47.451363] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-05. [2025-10-20T20:43:48.251409] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-08. [2025-10-20T20:43:48.475928] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:43:49.053119] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-07. [2025-10-20T20:43:49.056174] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T20:43:49.313601] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T20:43:49.851391] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-05. [2025-10-20T20:43:50.651452] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-09. [2025-10-20T20:43:50.821774] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T20:43:51.000146] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T20:43:51.451407] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-08. [2025-10-20T20:43:51.727590] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T20:43:52.251408] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-06. [2025-10-20T20:43:53.051361] [INFO ] [REMOTE ] PSTL: Fetching primary data for 2025-10. [2025-10-20T20:43:53.266134] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T20:43:53.325116] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T20:43:53.766115] [INFO ] [REMOTE ] PSTL: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:43:53.769367] [INFO ] [REMOTE ] PSTL: Full fetch complete (29553 records). [2025-10-20T20:43:53.769447] [DEBUG] [VALIDATOR ] PSTL: Validating primary data. [2025-10-20T20:43:53.856632] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-07. [2025-10-20T20:43:54.131951] [DEBUG] [VALIDATOR ] PSTL: All expected trading days are present. [2025-10-20T20:43:54.132027] [DEBUG] [VALIDATOR ] PSTL: Primary data validation passed. [2025-10-20T20:43:54.132599] [INFO ] [MAIN ] PSTL: Fetched primary data is valid. [2025-10-20T20:43:54.132636] [DEBUG] [LOCAL ] PSTL: Saving primary data. [2025-10-20T20:43:54.133380] [DEBUG] [LOCAL ] PSTL: Trimmed 29553 records to 20025 (keeping 200 days of data) [2025-10-20T20:43:54.250751] [DEBUG] [LOCAL ] PSTL: Primary data saved (20025 records). [2025-10-20T20:43:54.250872] [INFO ] [MAIN ] PSTL: Full history saved. [2025-10-20T20:43:54.250944] [INFO ] [MAIN ] Update Progress: 1739/2367 symbols completed. PSTL took 25.7s. Estimated remaining: 1h 20m 50s. Rate: 7.8 symbols/min [2025-10-20T20:43:54.431337] [INFO ] [MAIN ] PTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:43:54.467449] [DEBUG] [LOCAL ] PTY: Loaded 38349 existing records from primary storage. [2025-10-20T20:43:54.467750] [INFO ] [MAIN ] PTY: Loaded 38349 existing records. [2025-10-20T20:43:54.642208] [WARN ] [MAIN ] PTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:43:54.642422] [INFO ] [MAIN ] PTY: Wiping existing primary data before full refresh. [2025-10-20T20:43:54.642468] [INFO ] [LOCAL ] PTY: Starting deletion of primary data. [2025-10-20T20:43:54.642677] [DEBUG] [LOCAL ] PTY: Deleted primary data file. [2025-10-20T20:43:54.642733] [INFO ] [MAIN ] PTY: Starting full primary refresh. [2025-10-20T20:43:54.656462] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-06. [2025-10-20T20:43:54.950609] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T20:43:55.012220] [INFO ] [REMOTE ] PTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:43:55.451436] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-09. [2025-10-20T20:43:56.251595] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-01. [2025-10-20T20:43:56.530535] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T20:43:57.051378] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-08. [2025-10-20T20:43:57.654901] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T20:43:57.692916] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:43:57.712094] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T20:43:57.851354] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-09. [2025-10-20T20:43:58.651360] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-07. [2025-10-20T20:43:58.850332] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T20:43:59.451370] [INFO ] [REMOTE ] PSTG: Fetching primary data for 2025-10. [2025-10-20T20:44:00.220493] [INFO ] [REMOTE ] PSTG: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:44:00.227826] [INFO ] [REMOTE ] PSTG: Full fetch complete (76318 records). [2025-10-20T20:44:00.227939] [DEBUG] [VALIDATOR ] PSTG: Validating primary data. [2025-10-20T20:44:00.228750] [FATAL] [VALIDATOR ] PSTG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:44:00.228856] [FATAL] [MAIN ] PSTG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:44:00.228899] [WARN ] [MAIN ] PSTG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:44:00.228937] [INFO ] [LOCAL ] PSTG: Starting deletion of primary data. [2025-10-20T20:44:00.228994] [DEBUG] [LOCAL ] PSTG: No primary data file found to delete. [2025-10-20T20:44:00.229029] [INFO ] [LOCAL ] PSTG: Starting deletion of derivative data. [2025-10-20T20:44:00.229181] [DEBUG] [LOCAL ] PSTG: Deleted derivative data file. [2025-10-20T20:44:00.229267] [INFO ] [MAIN ] Update Progress: 1740/2367 symbols completed. PSTG took 32.9s. Estimated remaining: 1h 20m 42s. Rate: 7.8 symbols/min [2025-10-20T20:44:00.258409] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-02. [2025-10-20T20:44:00.404915] [INFO ] [MAIN ] PUK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:00.446088] [DEBUG] [LOCAL ] PUK: Loaded 45178 existing records from primary storage. [2025-10-20T20:44:00.446376] [INFO ] [MAIN ] PUK: Loaded 45178 existing records. [2025-10-20T20:44:00.633492] [WARN ] [MAIN ] PUK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:00.633833] [INFO ] [MAIN ] PUK: Wiping existing primary data before full refresh. [2025-10-20T20:44:00.633906] [INFO ] [LOCAL ] PUK: Starting deletion of primary data. [2025-10-20T20:44:00.634277] [DEBUG] [LOCAL ] PUK: Deleted primary data file. [2025-10-20T20:44:00.634348] [INFO ] [MAIN ] PUK: Starting full primary refresh. [2025-10-20T20:44:00.954525] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-07 (2.30 seconds). [2025-10-20T20:44:01.043632] [INFO ] [REMOTE ] PUK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:01.051340] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-01. [2025-10-20T20:44:01.840078] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:44:01.851325] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-03. [2025-10-20T20:44:02.402255] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:44:02.651402] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-02. [2025-10-20T20:44:02.970299] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T20:44:03.451410] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-04. [2025-10-20T20:44:04.241020] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T20:44:04.251327] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-03. [2025-10-20T20:44:05.051421] [INFO ] [REMOTE ] PTA: Fetching primary data for 2025-10. [2025-10-20T20:44:05.179813] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:44:05.851390] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-08. [2025-10-20T20:44:05.900035] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T20:44:06.109033] [INFO ] [REMOTE ] PTA: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T20:44:06.112382] [INFO ] [REMOTE ] PTA: Full fetch complete (22581 records). [2025-10-20T20:44:06.112451] [DEBUG] [VALIDATOR ] PTA: Validating primary data. [2025-10-20T20:44:06.507539] [DEBUG] [VALIDATOR ] PTA: All expected trading days are present. [2025-10-20T20:44:06.507624] [DEBUG] [VALIDATOR ] PTA: Primary data validation passed. [2025-10-20T20:44:06.508201] [INFO ] [MAIN ] PTA: Fetched primary data is valid. [2025-10-20T20:44:06.508263] [DEBUG] [LOCAL ] PTA: Saving primary data. [2025-10-20T20:44:06.508970] [DEBUG] [LOCAL ] PTA: Trimmed 22581 records to 15320 (keeping 200 days of data) [2025-10-20T20:44:06.607971] [DEBUG] [LOCAL ] PTA: Primary data saved (15320 records). [2025-10-20T20:44:06.608125] [INFO ] [MAIN ] PTA: Full history saved. [2025-10-20T20:44:06.608211] [INFO ] [MAIN ] Update Progress: 1741/2367 symbols completed. PTA took 28.9s. Estimated remaining: 1h 20m 33s. Rate: 7.8 symbols/min [2025-10-20T20:44:06.656474] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-05. [2025-10-20T20:44:06.795717] [INFO ] [MAIN ] PUMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:06.795897] [DEBUG] [LOCAL ] PUMP: No existing data file found in primary storage. [2025-10-20T20:44:06.795936] [WARN ] [MAIN ] PUMP: No existing data found, performing full refresh instead. [2025-10-20T20:44:06.796062] [INFO ] [MAIN ] PUMP: Wiping existing primary data before full refresh. [2025-10-20T20:44:06.796114] [INFO ] [LOCAL ] PUMP: Starting deletion of primary data. [2025-10-20T20:44:06.796154] [DEBUG] [LOCAL ] PUMP: No primary data file found to delete. [2025-10-20T20:44:06.796184] [INFO ] [MAIN ] PUMP: Starting full primary refresh. [2025-10-20T20:44:07.190043] [INFO ] [REMOTE ] PUMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:07.451378] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-01. [2025-10-20T20:44:07.853840] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T20:44:07.887082] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T20:44:08.251377] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-06. [2025-10-20T20:44:09.051372] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-04. [2025-10-20T20:44:09.119996] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T20:44:09.827338] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T20:44:09.851329] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-07. [2025-10-20T20:44:10.468929] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T20:44:10.651361] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-05. [2025-10-20T20:44:11.277241] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T20:44:11.451445] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-09. [2025-10-20T20:44:12.251387] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-02. [2025-10-20T20:44:12.336397] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:44:13.051355] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-06. [2025-10-20T20:44:13.471405] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T20:44:13.800059] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T20:44:13.851336] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-03. [2025-10-20T20:44:14.651377] [INFO ] [REMOTE ] PSX: Fetching primary data for 2025-10. [2025-10-20T20:44:14.783553] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T20:44:15.116263] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T20:44:15.436783] [INFO ] [REMOTE ] PSX: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T20:44:15.444498] [INFO ] [REMOTE ] PSX: Full fetch complete (76761 records). [2025-10-20T20:44:15.444688] [DEBUG] [VALIDATOR ] PSX: Validating primary data. [2025-10-20T20:44:15.445523] [FATAL] [VALIDATOR ] PSX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:44:15.445610] [FATAL] [MAIN ] PSX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:44:15.445689] [WARN ] [MAIN ] PSX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:44:15.445761] [INFO ] [LOCAL ] PSX: Starting deletion of primary data. [2025-10-20T20:44:15.445861] [DEBUG] [LOCAL ] PSX: No primary data file found to delete. [2025-10-20T20:44:15.445934] [INFO ] [LOCAL ] PSX: Starting deletion of derivative data. [2025-10-20T20:44:15.446086] [DEBUG] [LOCAL ] PSX: Deleted derivative data file. [2025-10-20T20:44:15.446178] [INFO ] [MAIN ] Update Progress: 1742/2367 symbols completed. PSX took 46.5s. Estimated remaining: 1h 20m 26s. Rate: 7.8 symbols/min [2025-10-20T20:44:15.451370] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-04. [2025-10-20T20:44:15.652135] [INFO ] [MAIN ] PVH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:15.699851] [DEBUG] [LOCAL ] PVH: Loaded 50636 existing records from primary storage. [2025-10-20T20:44:15.700171] [INFO ] [MAIN ] PVH: Loaded 50636 existing records. [2025-10-20T20:44:15.881298] [WARN ] [MAIN ] PVH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:15.881539] [INFO ] [MAIN ] PVH: Wiping existing primary data before full refresh. [2025-10-20T20:44:15.881591] [INFO ] [LOCAL ] PVH: Starting deletion of primary data. [2025-10-20T20:44:15.881813] [DEBUG] [LOCAL ] PVH: Deleted primary data file. [2025-10-20T20:44:15.881869] [INFO ] [MAIN ] PVH: Starting full primary refresh. [2025-10-20T20:44:16.257029] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-07. [2025-10-20T20:44:16.261801] [INFO ] [REMOTE ] PVH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:17.051374] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-08. [2025-10-20T20:44:17.256833] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T20:44:17.851406] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-01. [2025-10-20T20:44:18.045852] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T20:44:18.585590] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T20:44:18.651375] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-09. [2025-10-20T20:44:19.451356] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-05. [2025-10-20T20:44:19.523747] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T20:44:20.251355] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-02. [2025-10-20T20:44:20.284756] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T20:44:20.903139] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T20:44:21.051362] [INFO ] [REMOTE ] PTY: Fetching primary data for 2025-10. [2025-10-20T20:44:21.851367] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-06. [2025-10-20T20:44:21.928498] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T20:44:22.316130] [INFO ] [REMOTE ] PTY: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T20:44:22.323027] [INFO ] [REMOTE ] PTY: Full fetch complete (54407 records). [2025-10-20T20:44:22.323181] [DEBUG] [VALIDATOR ] PTY: Validating primary data. [2025-10-20T20:44:22.651782] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-08. [2025-10-20T20:44:22.752435] [DEBUG] [VALIDATOR ] PTY: All expected trading days are present. [2025-10-20T20:44:22.752512] [DEBUG] [VALIDATOR ] PTY: Primary data validation passed. [2025-10-20T20:44:22.753599] [INFO ] [MAIN ] PTY: Fetched primary data is valid. [2025-10-20T20:44:22.753688] [DEBUG] [LOCAL ] PTY: Saving primary data. [2025-10-20T20:44:22.755061] [DEBUG] [LOCAL ] PTY: Trimmed 54407 records to 37923 (keeping 200 days of data) [2025-10-20T20:44:22.990717] [DEBUG] [LOCAL ] PTY: Primary data saved (37923 records). [2025-10-20T20:44:22.990840] [INFO ] [MAIN ] PTY: Full history saved. [2025-10-20T20:44:22.991180] [INFO ] [MAIN ] Update Progress: 1743/2367 symbols completed. PTY took 28.7s. Estimated remaining: 1h 20m 18s. Rate: 7.8 symbols/min [2025-10-20T20:44:23.158847] [INFO ] [MAIN ] PVL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:23.166362] [DEBUG] [LOCAL ] PVL: Loaded 4769 existing records from primary storage. [2025-10-20T20:44:23.166572] [INFO ] [MAIN ] PVL: Loaded 4769 existing records. [2025-10-20T20:44:23.353452] [WARN ] [MAIN ] PVL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:23.353737] [INFO ] [MAIN ] PVL: Wiping existing primary data before full refresh. [2025-10-20T20:44:23.353812] [INFO ] [LOCAL ] PVL: Starting deletion of primary data. [2025-10-20T20:44:23.363003] [DEBUG] [LOCAL ] PVL: Deleted primary data file. [2025-10-20T20:44:23.375262] [INFO ] [MAIN ] PVL: Starting full primary refresh. [2025-10-20T20:44:23.450348] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T20:44:23.456493] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-03. [2025-10-20T20:44:23.756147] [INFO ] [REMOTE ] PVL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:24.251380] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-01. [2025-10-20T20:44:24.379434] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T20:44:25.033541] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T20:44:25.051321] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-02. [2025-10-20T20:44:25.399267] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T20:44:25.519992] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T20:44:25.851422] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-03. [2025-10-20T20:44:26.517437] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T20:44:26.651420] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-04. [2025-10-20T20:44:27.451455] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-07. [2025-10-20T20:44:28.251426] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-09. [2025-10-20T20:44:28.384557] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:44:29.051447] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-05. [2025-10-20T20:44:29.227509] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T20:44:29.851351] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-08. [2025-10-20T20:44:29.990918] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T20:44:30.651423] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-04. [2025-10-20T20:44:30.912413] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T20:44:31.298186] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T20:44:31.451402] [INFO ] [REMOTE ] PUK: Fetching primary data for 2025-10. [2025-10-20T20:44:31.676681] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T20:44:32.251385] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-09. [2025-10-20T20:44:33.008504] [INFO ] [REMOTE ] PUK: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T20:44:33.015247] [INFO ] [REMOTE ] PUK: Full fetch complete (65358 records). [2025-10-20T20:44:33.015389] [DEBUG] [VALIDATOR ] PUK: Validating primary data. [2025-10-20T20:44:33.058331] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-06. [2025-10-20T20:44:33.415790] [DEBUG] [VALIDATOR ] PUK: All expected trading days are present. [2025-10-20T20:44:33.415923] [DEBUG] [VALIDATOR ] PUK: Primary data validation passed. [2025-10-20T20:44:33.416919] [INFO ] [MAIN ] PUK: Fetched primary data is valid. [2025-10-20T20:44:33.416981] [DEBUG] [LOCAL ] PUK: Saving primary data. [2025-10-20T20:44:33.418272] [DEBUG] [LOCAL ] PUK: Trimmed 65358 records to 44463 (keeping 200 days of data) [2025-10-20T20:44:33.698188] [DEBUG] [LOCAL ] PUK: Primary data saved (44463 records). [2025-10-20T20:44:33.698336] [INFO ] [MAIN ] PUK: Full history saved. [2025-10-20T20:44:33.698417] [INFO ] [MAIN ] Update Progress: 1744/2367 symbols completed. PUK took 33.5s. Estimated remaining: 1h 20m 12s. Rate: 7.8 symbols/min [2025-10-20T20:44:33.854225] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-05. [2025-10-20T20:44:33.874785] [INFO ] [MAIN ] PWR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:33.920514] [DEBUG] [LOCAL ] PWR: Loaded 47297 existing records from primary storage. [2025-10-20T20:44:33.920901] [INFO ] [MAIN ] PWR: Loaded 47297 existing records. [2025-10-20T20:44:34.106684] [WARN ] [MAIN ] PWR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:34.107073] [INFO ] [MAIN ] PWR: Wiping existing primary data before full refresh. [2025-10-20T20:44:34.107325] [INFO ] [LOCAL ] PWR: Starting deletion of primary data. [2025-10-20T20:44:34.107829] [DEBUG] [LOCAL ] PWR: Deleted primary data file. [2025-10-20T20:44:34.131201] [INFO ] [MAIN ] PWR: Starting full primary refresh. [2025-10-20T20:44:34.199312] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T20:44:34.497120] [INFO ] [REMOTE ] PWR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:34.528338] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T20:44:34.651342] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-01. [2025-10-20T20:44:34.804121] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T20:44:35.451462] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-07. [2025-10-20T20:44:36.251362] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-06. [2025-10-20T20:44:36.328926] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T20:44:36.887782] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T20:44:36.960398] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-06 (0.71 seconds). [2025-10-20T20:44:37.051335] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-07. [2025-10-20T20:44:37.698152] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T20:44:37.851345] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-08. [2025-10-20T20:44:38.651408] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-02. [2025-10-20T20:44:39.451403] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-08. [2025-10-20T20:44:39.715914] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T20:44:40.160157] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T20:44:40.226811] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:44:40.251378] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-09. [2025-10-20T20:44:40.876529] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T20:44:41.051378] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-09. [2025-10-20T20:44:41.851391] [INFO ] [REMOTE ] PUMP: Fetching primary data for 2025-10. [2025-10-20T20:44:42.328918] [INFO ] [REMOTE ] PUMP: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T20:44:42.335837] [INFO ] [REMOTE ] PUMP: Full fetch complete (71359 records). [2025-10-20T20:44:42.336006] [DEBUG] [VALIDATOR ] PUMP: Validating primary data. [2025-10-20T20:44:42.336883] [FATAL] [VALIDATOR ] PUMP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:44:42.336977] [FATAL] [MAIN ] PUMP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:44:42.337021] [WARN ] [MAIN ] PUMP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:44:42.337085] [INFO ] [LOCAL ] PUMP: Starting deletion of primary data. [2025-10-20T20:44:42.337149] [DEBUG] [LOCAL ] PUMP: No primary data file found to delete. [2025-10-20T20:44:42.337186] [INFO ] [LOCAL ] PUMP: Starting deletion of derivative data. [2025-10-20T20:44:42.337226] [DEBUG] [LOCAL ] PUMP: No derivative data file found to delete. [2025-10-20T20:44:42.337284] [INFO ] [MAIN ] Update Progress: 1745/2367 symbols completed. PUMP took 35.7s. Estimated remaining: 1h 20m 4s. Rate: 7.8 symbols/min [2025-10-20T20:44:42.514151] [INFO ] [MAIN ] PX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:42.571912] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T20:44:42.570465] [DEBUG] [LOCAL ] PX: Loaded 34837 existing records from primary storage. [2025-10-20T20:44:42.581678] [INFO ] [MAIN ] PX: Loaded 34837 existing records. [2025-10-20T20:44:42.656565] [INFO ] [REMOTE ] PVL: Fetching primary data for 2025-10. [2025-10-20T20:44:42.760297] [WARN ] [MAIN ] PX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:42.760517] [INFO ] [MAIN ] PX: Wiping existing primary data before full refresh. [2025-10-20T20:44:42.760567] [INFO ] [LOCAL ] PX: Starting deletion of primary data. [2025-10-20T20:44:42.760821] [DEBUG] [LOCAL ] PX: Deleted primary data file. [2025-10-20T20:44:42.760880] [INFO ] [MAIN ] PX: Starting full primary refresh. [2025-10-20T20:44:43.110974] [INFO ] [REMOTE ] PX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:43.280585] [INFO ] [REMOTE ] PVL: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:44:43.281913] [INFO ] [REMOTE ] PVL: Full fetch complete (7091 records). [2025-10-20T20:44:43.281985] [DEBUG] [VALIDATOR ] PVL: Validating primary data. [2025-10-20T20:44:43.456501] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-03. [2025-10-20T20:44:43.691390] [DEBUG] [VALIDATOR ] PVL: All expected trading days are present. [2025-10-20T20:44:43.691464] [DEBUG] [VALIDATOR ] PVL: Primary data validation passed. [2025-10-20T20:44:43.691636] [INFO ] [MAIN ] PVL: Fetched primary data is valid. [2025-10-20T20:44:43.691714] [DEBUG] [LOCAL ] PVL: Saving primary data. [2025-10-20T20:44:43.692083] [DEBUG] [LOCAL ] PVL: Trimmed 7091 records to 4742 (keeping 200 days of data) [2025-10-20T20:44:43.718181] [DEBUG] [LOCAL ] PVL: Primary data saved (4742 records). [2025-10-20T20:44:43.718250] [INFO ] [MAIN ] PVL: Full history saved. [2025-10-20T20:44:43.718317] [INFO ] [MAIN ] Update Progress: 1746/2367 symbols completed. PVL took 20.7s. Estimated remaining: 1h 19m 54s. Rate: 7.8 symbols/min [2025-10-20T20:44:43.903306] [INFO ] [MAIN ] PXED: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:43.903524] [DEBUG] [LOCAL ] PXED: No existing data file found in primary storage. [2025-10-20T20:44:43.903590] [WARN ] [MAIN ] PXED: No existing data found, performing full refresh instead. [2025-10-20T20:44:43.903763] [INFO ] [MAIN ] PXED: Wiping existing primary data before full refresh. [2025-10-20T20:44:43.903826] [INFO ] [LOCAL ] PXED: Starting deletion of primary data. [2025-10-20T20:44:43.903893] [DEBUG] [LOCAL ] PXED: No primary data file found to delete. [2025-10-20T20:44:43.903971] [INFO ] [MAIN ] PXED: Starting full primary refresh. [2025-10-20T20:44:44.264332] [INFO ] [REMOTE ] PVH: Fetching primary data for 2025-10. [2025-10-20T20:44:44.274494] [INFO ] [REMOTE ] PXED: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:45.019255] [INFO ] [REMOTE ] PVH: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T20:44:45.026367] [INFO ] [REMOTE ] PVH: Full fetch complete (70720 records). [2025-10-20T20:44:45.026508] [DEBUG] [VALIDATOR ] PVH: Validating primary data. [2025-10-20T20:44:45.027185] [FATAL] [VALIDATOR ] PVH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:44:45.027270] [FATAL] [MAIN ] PVH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:44:45.027313] [WARN ] [MAIN ] PVH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:44:45.027351] [INFO ] [LOCAL ] PVH: Starting deletion of primary data. [2025-10-20T20:44:45.027408] [DEBUG] [LOCAL ] PVH: No primary data file found to delete. [2025-10-20T20:44:45.027443] [INFO ] [LOCAL ] PVH: Starting deletion of derivative data. [2025-10-20T20:44:45.027611] [DEBUG] [LOCAL ] PVH: Deleted derivative data file. [2025-10-20T20:44:45.027727] [INFO ] [MAIN ] Update Progress: 1747/2367 symbols completed. PVH took 29.6s. Estimated remaining: 1h 19m 44s. Rate: 7.8 symbols/min [2025-10-20T20:44:45.058920] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-01. [2025-10-20T20:44:45.205385] [INFO ] [MAIN ] QBTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:45.205601] [DEBUG] [LOCAL ] QBTS: No existing data file found in primary storage. [2025-10-20T20:44:45.205680] [WARN ] [MAIN ] QBTS: No existing data found, performing full refresh instead. [2025-10-20T20:44:45.205791] [INFO ] [MAIN ] QBTS: Wiping existing primary data before full refresh. [2025-10-20T20:44:45.205832] [INFO ] [LOCAL ] QBTS: Starting deletion of primary data. [2025-10-20T20:44:45.205873] [DEBUG] [LOCAL ] QBTS: No primary data file found to delete. [2025-10-20T20:44:45.205903] [INFO ] [MAIN ] QBTS: Starting full primary refresh. [2025-10-20T20:44:45.569375] [INFO ] [REMOTE ] QBTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:45.609340] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T20:44:45.851409] [INFO ] [REMOTE ] PXED: Fetching primary data for 2025-01. [2025-10-20T20:44:46.155801] [DEBUG] [REMOTE ] PXED: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:44:46.156154] [WARN ] [REMOTE ] PXED: No data returned for month 2025-01 [2025-10-20T20:44:46.156273] [FATAL] [REMOTE ] PXED: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:44:46.156324] [FATAL] [MAIN ] PXED: Failed to fetch primary data. [2025-10-20T20:44:46.156361] [WARN ] [MAIN ] PXED: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:44:46.156402] [INFO ] [LOCAL ] PXED: Starting deletion of primary data. [2025-10-20T20:44:46.156467] [DEBUG] [LOCAL ] PXED: No primary data file found to delete. [2025-10-20T20:44:46.156504] [INFO ] [LOCAL ] PXED: Starting deletion of derivative data. [2025-10-20T20:44:46.156596] [DEBUG] [LOCAL ] PXED: No derivative data file found to delete. [2025-10-20T20:44:46.156676] [INFO ] [MAIN ] Update Progress: 1748/2367 symbols completed. PXED took 2.4s. Estimated remaining: 1h 19m 34s. Rate: 7.8 symbols/min [2025-10-20T20:44:46.328489] [INFO ] [MAIN ] QD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:44:46.354602] [DEBUG] [LOCAL ] QD: Loaded 27874 existing records from primary storage. [2025-10-20T20:44:46.354915] [INFO ] [MAIN ] QD: Loaded 27874 existing records. [2025-10-20T20:44:46.540868] [WARN ] [MAIN ] QD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:44:46.541175] [INFO ] [MAIN ] QD: Wiping existing primary data before full refresh. [2025-10-20T20:44:46.541266] [INFO ] [LOCAL ] QD: Starting deletion of primary data. [2025-10-20T20:44:46.541616] [DEBUG] [LOCAL ] QD: Deleted primary data file. [2025-10-20T20:44:46.541713] [INFO ] [MAIN ] QD: Starting full primary refresh. [2025-10-20T20:44:46.658319] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-01. [2025-10-20T20:44:46.699486] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T20:44:46.946171] [INFO ] [REMOTE ] QD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:44:47.451409] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-02. [2025-10-20T20:44:48.251387] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-01. [2025-10-20T20:44:48.979059] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T20:44:49.056547] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-03. [2025-10-20T20:44:49.069602] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T20:44:49.284547] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-01 (2.63 seconds). [2025-10-20T20:44:49.851373] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-02. [2025-10-20T20:44:50.537562] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T20:44:50.651389] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-02. [2025-10-20T20:44:51.451372] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-04. [2025-10-20T20:44:52.082700] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:44:52.251375] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-03. [2025-10-20T20:44:52.460370] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-02 (2.61 seconds). [2025-10-20T20:44:52.960515] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T20:44:53.051372] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-05. [2025-10-20T20:44:53.565711] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T20:44:53.851491] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-04. [2025-10-20T20:44:54.529873] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T20:44:54.651367] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-06. [2025-10-20T20:44:55.451356] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-03. [2025-10-20T20:44:55.555791] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:44:56.142436] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T20:44:56.251387] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-05. [2025-10-20T20:44:57.051390] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-07. [2025-10-20T20:44:57.851452] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-04. [2025-10-20T20:44:57.913326] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T20:44:58.260122] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-03 (2.81 seconds). [2025-10-20T20:44:58.372102] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T20:44:58.651376] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-08. [2025-10-20T20:44:58.730962] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T20:44:59.451372] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-04. [2025-10-20T20:45:00.052106] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T20:45:00.251385] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-06. [2025-10-20T20:45:01.051423] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-05. [2025-10-20T20:45:01.851395] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-09. [2025-10-20T20:45:02.018425] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T20:45:02.131228] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-04 (2.68 seconds). [2025-10-20T20:45:02.382324] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T20:45:02.651376] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-05. [2025-10-20T20:45:03.227031] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T20:45:03.451385] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-06. [2025-10-20T20:45:04.251378] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-07. [2025-10-20T20:45:04.587510] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T20:45:05.051343] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-07. [2025-10-20T20:45:05.186399] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-05 (2.54 seconds). [2025-10-20T20:45:05.851405] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-06. [2025-10-20T20:45:06.171622] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T20:45:06.441464] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T20:45:06.651421] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-08. [2025-10-20T20:45:07.451365] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-08. [2025-10-20T20:45:08.251431] [INFO ] [REMOTE ] PX: Fetching primary data for 2025-10. [2025-10-20T20:45:08.416208] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T20:45:08.491265] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T20:45:08.649087] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-06 (2.80 seconds). [2025-10-20T20:45:09.051373] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-09. [2025-10-20T20:45:09.142091] [INFO ] [REMOTE ] PX: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T20:45:09.146526] [INFO ] [REMOTE ] PX: Full fetch complete (50011 records). [2025-10-20T20:45:09.146697] [DEBUG] [VALIDATOR ] PX: Validating primary data. [2025-10-20T20:45:09.534501] [DEBUG] [VALIDATOR ] PX: All expected trading days are present. [2025-10-20T20:45:09.534575] [DEBUG] [VALIDATOR ] PX: Primary data validation passed. [2025-10-20T20:45:09.535442] [INFO ] [MAIN ] PX: Fetched primary data is valid. [2025-10-20T20:45:09.535506] [DEBUG] [LOCAL ] PX: Saving primary data. [2025-10-20T20:45:09.536504] [DEBUG] [LOCAL ] PX: Trimmed 50011 records to 34272 (keeping 200 days of data) [2025-10-20T20:45:09.749832] [DEBUG] [LOCAL ] PX: Primary data saved (34272 records). [2025-10-20T20:45:09.749993] [INFO ] [MAIN ] PX: Full history saved. [2025-10-20T20:45:09.750074] [INFO ] [MAIN ] Update Progress: 1749/2367 symbols completed. PX took 27.4s. Estimated remaining: 1h 19m 32s. Rate: 7.8 symbols/min [2025-10-20T20:45:09.857178] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-09. [2025-10-20T20:45:09.930251] [INFO ] [MAIN ] QGEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:09.978921] [DEBUG] [LOCAL ] QGEN: Loaded 49853 existing records from primary storage. [2025-10-20T20:45:09.979256] [INFO ] [MAIN ] QGEN: Loaded 49853 existing records. [2025-10-20T20:45:10.156725] [WARN ] [MAIN ] QGEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:10.156954] [INFO ] [MAIN ] QGEN: Wiping existing primary data before full refresh. [2025-10-20T20:45:10.157001] [INFO ] [LOCAL ] QGEN: Starting deletion of primary data. [2025-10-20T20:45:10.157216] [DEBUG] [LOCAL ] QGEN: Deleted primary data file. [2025-10-20T20:45:10.157271] [INFO ] [MAIN ] QGEN: Starting full primary refresh. [2025-10-20T20:45:10.509108] [INFO ] [REMOTE ] QGEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:10.651361] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-01. [2025-10-20T20:45:10.893227] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T20:45:11.259592] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T20:45:11.451374] [INFO ] [REMOTE ] PWR: Fetching primary data for 2025-10. [2025-10-20T20:45:12.216080] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T20:45:12.261535] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-02. [2025-10-20T20:45:12.263512] [INFO ] [REMOTE ] PWR: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T20:45:12.270050] [INFO ] [REMOTE ] PWR: Full fetch complete (68848 records). [2025-10-20T20:45:12.270149] [DEBUG] [VALIDATOR ] PWR: Validating primary data. [2025-10-20T20:45:12.270946] [FATAL] [VALIDATOR ] PWR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:45:12.271038] [FATAL] [MAIN ] PWR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:45:12.271124] [WARN ] [MAIN ] PWR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:45:12.271210] [INFO ] [LOCAL ] PWR: Starting deletion of primary data. [2025-10-20T20:45:12.271285] [DEBUG] [LOCAL ] PWR: No primary data file found to delete. [2025-10-20T20:45:12.271322] [INFO ] [LOCAL ] PWR: Starting deletion of derivative data. [2025-10-20T20:45:12.271439] [DEBUG] [LOCAL ] PWR: Deleted derivative data file. [2025-10-20T20:45:12.271507] [INFO ] [MAIN ] Update Progress: 1750/2367 symbols completed. PWR took 38.6s. Estimated remaining: 1h 19m 22s. Rate: 7.8 symbols/min [2025-10-20T20:45:12.451773] [INFO ] [MAIN ] QS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:12.504830] [DEBUG] [LOCAL ] QS: Loaded 54539 existing records from primary storage. [2025-10-20T20:45:12.505165] [INFO ] [MAIN ] QS: Loaded 54539 existing records. [2025-10-20T20:45:12.684600] [WARN ] [MAIN ] QS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:12.684821] [INFO ] [MAIN ] QS: Wiping existing primary data before full refresh. [2025-10-20T20:45:12.684870] [INFO ] [LOCAL ] QS: Starting deletion of primary data. [2025-10-20T20:45:12.685085] [DEBUG] [LOCAL ] QS: Deleted primary data file. [2025-10-20T20:45:12.685143] [INFO ] [MAIN ] QS: Starting full primary refresh. [2025-10-20T20:45:13.057082] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-07. [2025-10-20T20:45:13.064071] [INFO ] [REMOTE ] QS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:13.851371] [INFO ] [REMOTE ] QD: Fetching primary data for 2025-10. [2025-10-20T20:45:13.932079] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T20:45:14.651364] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-03. [2025-10-20T20:45:15.074968] [INFO ] [REMOTE ] QD: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T20:45:15.078999] [INFO ] [REMOTE ] QD: Full fetch complete (39974 records). [2025-10-20T20:45:15.079079] [DEBUG] [VALIDATOR ] QD: Validating primary data. [2025-10-20T20:45:15.444890] [DEBUG] [VALIDATOR ] QD: All expected trading days are present. [2025-10-20T20:45:15.444987] [DEBUG] [VALIDATOR ] QD: Primary data validation passed. [2025-10-20T20:45:15.445759] [INFO ] [MAIN ] QD: Fetched primary data is valid. [2025-10-20T20:45:15.445820] [DEBUG] [LOCAL ] QD: Saving primary data. [2025-10-20T20:45:15.446744] [DEBUG] [LOCAL ] QD: Trimmed 39974 records to 27590 (keeping 200 days of data) [2025-10-20T20:45:15.489399] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-01. [2025-10-20T20:45:15.600763] [DEBUG] [LOCAL ] QD: Primary data saved (27590 records). [2025-10-20T20:45:15.600949] [INFO ] [MAIN ] QD: Full history saved. [2025-10-20T20:45:15.601039] [INFO ] [MAIN ] Update Progress: 1751/2367 symbols completed. QD took 29.4s. Estimated remaining: 1h 19m 13s. Rate: 7.8 symbols/min [2025-10-20T20:45:15.778877] [INFO ] [MAIN ] QSR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:15.847855] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-07 (2.79 seconds). [2025-10-20T20:45:15.880282] [DEBUG] [LOCAL ] QSR: Loaded 53233 existing records from primary storage. [2025-10-20T20:45:15.880661] [INFO ] [MAIN ] QSR: Loaded 53233 existing records. [2025-10-20T20:45:16.069167] [WARN ] [MAIN ] QSR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:16.069402] [INFO ] [MAIN ] QSR: Wiping existing primary data before full refresh. [2025-10-20T20:45:16.069462] [INFO ] [LOCAL ] QSR: Starting deletion of primary data. [2025-10-20T20:45:16.069796] [DEBUG] [LOCAL ] QSR: Deleted primary data file. [2025-10-20T20:45:16.069852] [INFO ] [MAIN ] QSR: Starting full primary refresh. [2025-10-20T20:45:16.256515] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-08. [2025-10-20T20:45:16.439798] [INFO ] [REMOTE ] QSR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:16.520878] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T20:45:17.051425] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-01. [2025-10-20T20:45:17.851409] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-04. [2025-10-20T20:45:17.928918] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-01 (2.44 seconds). [2025-10-20T20:45:18.651396] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-02. [2025-10-20T20:45:18.862556] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T20:45:18.984059] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-08 (2.73 seconds). [2025-10-20T20:45:19.451409] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-09. [2025-10-20T20:45:19.580437] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:45:20.251366] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-02. [2025-10-20T20:45:21.006418] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-02 (2.36 seconds). [2025-10-20T20:45:21.051344] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-03. [2025-10-20T20:45:21.851407] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-05. [2025-10-20T20:45:22.100982] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T20:45:22.318772] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-09 (2.87 seconds). [2025-10-20T20:45:22.651384] [INFO ] [REMOTE ] QBTS: Fetching primary data for 2025-10. [2025-10-20T20:45:23.205893] [INFO ] [REMOTE ] QBTS: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:45:23.213193] [INFO ] [REMOTE ] QBTS: Full fetch complete (77533 records). [2025-10-20T20:45:23.213320] [DEBUG] [VALIDATOR ] QBTS: Validating primary data. [2025-10-20T20:45:23.214073] [FATAL] [VALIDATOR ] QBTS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:45:23.214196] [FATAL] [MAIN ] QBTS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:45:23.214276] [WARN ] [MAIN ] QBTS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:45:23.214362] [INFO ] [LOCAL ] QBTS: Starting deletion of primary data. [2025-10-20T20:45:23.214437] [DEBUG] [LOCAL ] QBTS: No primary data file found to delete. [2025-10-20T20:45:23.214473] [INFO ] [LOCAL ] QBTS: Starting deletion of derivative data. [2025-10-20T20:45:23.214511] [DEBUG] [LOCAL ] QBTS: No derivative data file found to delete. [2025-10-20T20:45:23.214563] [INFO ] [MAIN ] Update Progress: 1752/2367 symbols completed. QBTS took 38.2s. Estimated remaining: 1h 19m 5s. Rate: 7.8 symbols/min [2025-10-20T20:45:23.389632] [INFO ] [MAIN ] QTWO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:23.426950] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T20:45:23.452473] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-03. [2025-10-20T20:45:23.471630] [DEBUG] [LOCAL ] QTWO: Loaded 40143 existing records from primary storage. [2025-10-20T20:45:23.472144] [INFO ] [MAIN ] QTWO: Loaded 40143 existing records. [2025-10-20T20:45:23.676733] [WARN ] [MAIN ] QTWO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:23.677082] [INFO ] [MAIN ] QTWO: Wiping existing primary data before full refresh. [2025-10-20T20:45:23.677283] [INFO ] [LOCAL ] QTWO: Starting deletion of primary data. [2025-10-20T20:45:23.677632] [DEBUG] [LOCAL ] QTWO: Deleted primary data file. [2025-10-20T20:45:23.677838] [INFO ] [MAIN ] QTWO: Starting full primary refresh. [2025-10-20T20:45:23.983847] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T20:45:24.203728] [INFO ] [REMOTE ] QTWO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:24.251462] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-01. [2025-10-20T20:45:25.051422] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-04. [2025-10-20T20:45:25.279531] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T20:45:25.645053] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T20:45:25.851422] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-02. [2025-10-20T20:45:26.651450] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-06. [2025-10-20T20:45:27.229711] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T20:45:27.323016] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-04 (2.27 seconds). [2025-10-20T20:45:27.451406] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-04. [2025-10-20T20:45:28.251399] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-05. [2025-10-20T20:45:28.499742] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T20:45:29.051349] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-07. [2025-10-20T20:45:29.264135] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T20:45:29.851372] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-05. [2025-10-20T20:45:30.289740] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T20:45:30.453727] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:45:30.651388] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-06. [2025-10-20T20:45:31.451465] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-08. [2025-10-20T20:45:31.907678] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T20:45:32.251356] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-06. [2025-10-20T20:45:32.885932] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-10-20T20:45:33.051384] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-03. [2025-10-20T20:45:33.238428] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T20:45:33.851369] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-09. [2025-10-20T20:45:34.063158] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T20:45:34.252877] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T20:45:34.651370] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-07. [2025-10-20T20:45:35.451420] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-07. [2025-10-20T20:45:35.695284] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T20:45:36.251345] [INFO ] [REMOTE ] QGEN: Fetching primary data for 2025-10. [2025-10-20T20:45:36.359284] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T20:45:37.051426] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-08. [2025-10-20T20:45:37.851363] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-04. [2025-10-20T20:45:37.929456] [INFO ] [REMOTE ] QGEN: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T20:45:37.936390] [INFO ] [REMOTE ] QGEN: Full fetch complete (70530 records). [2025-10-20T20:45:37.936681] [DEBUG] [VALIDATOR ] QGEN: Validating primary data. [2025-10-20T20:45:38.343326] [DEBUG] [VALIDATOR ] QGEN: All expected trading days are present. [2025-10-20T20:45:38.343440] [DEBUG] [VALIDATOR ] QGEN: Primary data validation passed. [2025-10-20T20:45:38.344506] [INFO ] [MAIN ] QGEN: Fetched primary data is valid. [2025-10-20T20:45:38.344680] [DEBUG] [LOCAL ] QGEN: Saving primary data. [2025-10-20T20:45:38.346113] [DEBUG] [LOCAL ] QGEN: Trimmed 70530 records to 49088 (keeping 200 days of data) [2025-10-20T20:45:38.462527] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-07 (3.01 seconds). [2025-10-20T20:45:38.664778] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-08. [2025-10-20T20:45:38.681793] [DEBUG] [LOCAL ] QGEN: Primary data saved (49088 records). [2025-10-20T20:45:38.681937] [INFO ] [MAIN ] QGEN: Full history saved. [2025-10-20T20:45:38.682043] [INFO ] [MAIN ] Update Progress: 1753/2367 symbols completed. QGEN took 28.9s. Estimated remaining: 1h 19m 0s. Rate: 7.8 symbols/min [2025-10-20T20:45:38.876713] [INFO ] [MAIN ] QUAD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:38.898827] [DEBUG] [LOCAL ] QUAD: Loaded 23589 existing records from primary storage. [2025-10-20T20:45:38.899067] [INFO ] [MAIN ] QUAD: Loaded 23589 existing records. [2025-10-20T20:45:39.081371] [WARN ] [MAIN ] QUAD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:39.081748] [INFO ] [MAIN ] QUAD: Wiping existing primary data before full refresh. [2025-10-20T20:45:39.081872] [INFO ] [LOCAL ] QUAD: Starting deletion of primary data. [2025-10-20T20:45:39.082297] [DEBUG] [LOCAL ] QUAD: Deleted primary data file. [2025-10-20T20:45:39.082437] [INFO ] [MAIN ] QUAD: Starting full primary refresh. [2025-10-20T20:45:39.199357] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-08 (2.15 seconds). [2025-10-20T20:45:39.515580] [INFO ] [REMOTE ] QUAD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:39.510535] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-09. [2025-10-20T20:45:39.518814] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T20:45:40.251386] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-01. [2025-10-20T20:45:41.051359] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-05. [2025-10-20T20:45:41.137807] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-08 (2.47 seconds). [2025-10-20T20:45:41.164475] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T20:45:41.215706] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T20:45:41.851416] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-09. [2025-10-20T20:45:42.599872] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T20:45:42.651363] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-06. [2025-10-20T20:45:43.451411] [INFO ] [REMOTE ] QSR: Fetching primary data for 2025-10. [2025-10-20T20:45:43.845332] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T20:45:44.251355] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-07. [2025-10-20T20:45:44.608311] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-09 (2.76 seconds). [2025-10-20T20:45:45.051410] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-02. [2025-10-20T20:45:45.120580] [INFO ] [REMOTE ] QSR: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T20:45:45.128787] [INFO ] [REMOTE ] QSR: Full fetch complete (76092 records). [2025-10-20T20:45:45.128951] [DEBUG] [VALIDATOR ] QSR: Validating primary data. [2025-10-20T20:45:45.550908] [DEBUG] [VALIDATOR ] QSR: All expected trading days are present. [2025-10-20T20:45:45.551481] [DEBUG] [VALIDATOR ] QSR: Primary data validation passed. [2025-10-20T20:45:45.553688] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-07 (1.30 seconds). [2025-10-20T20:45:45.552673] [INFO ] [MAIN ] QSR: Fetched primary data is valid. [2025-10-20T20:45:45.560603] [DEBUG] [LOCAL ] QSR: Saving primary data. [2025-10-20T20:45:45.562445] [DEBUG] [LOCAL ] QSR: Trimmed 76092 records to 52497 (keeping 200 days of data) [2025-10-20T20:45:45.857067] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-08. [2025-10-20T20:45:45.880145] [DEBUG] [LOCAL ] QSR: Primary data saved (52497 records). [2025-10-20T20:45:45.880260] [INFO ] [MAIN ] QSR: Full history saved. [2025-10-20T20:45:45.880625] [INFO ] [MAIN ] Update Progress: 1754/2367 symbols completed. QSR took 30.3s. Estimated remaining: 1h 18m 53s. Rate: 7.8 symbols/min [2025-10-20T20:45:46.050091] [INFO ] [MAIN ] QVCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:46.061483] [DEBUG] [LOCAL ] QVCC: Loaded 9007 existing records from primary storage. [2025-10-20T20:45:46.061725] [INFO ] [MAIN ] QVCC: Loaded 9007 existing records. [2025-10-20T20:45:46.250318] [WARN ] [MAIN ] QVCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:46.250736] [INFO ] [MAIN ] QVCC: Wiping existing primary data before full refresh. [2025-10-20T20:45:46.250855] [INFO ] [LOCAL ] QVCC: Starting deletion of primary data. [2025-10-20T20:45:46.251075] [DEBUG] [LOCAL ] QVCC: Deleted primary data file. [2025-10-20T20:45:46.251142] [INFO ] [MAIN ] QVCC: Starting full primary refresh. [2025-10-20T20:45:46.389245] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T20:45:46.645971] [INFO ] [REMOTE ] QVCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:46.651404] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-01. [2025-10-20T20:45:47.243971] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T20:45:47.451423] [INFO ] [REMOTE ] QS: Fetching primary data for 2025-10. [2025-10-20T20:45:47.506978] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T20:45:47.813801] [INFO ] [REMOTE ] QS: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T20:45:47.821313] [INFO ] [REMOTE ] QS: Full fetch complete (77530 records). [2025-10-20T20:45:47.821473] [DEBUG] [VALIDATOR ] QS: Validating primary data. [2025-10-20T20:45:47.822226] [FATAL] [VALIDATOR ] QS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:45:47.822334] [FATAL] [MAIN ] QS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:45:47.822406] [WARN ] [MAIN ] QS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:45:47.822506] [INFO ] [LOCAL ] QS: Starting deletion of primary data. [2025-10-20T20:45:47.822585] [DEBUG] [LOCAL ] QS: No primary data file found to delete. [2025-10-20T20:45:47.822622] [INFO ] [LOCAL ] QS: Starting deletion of derivative data. [2025-10-20T20:45:47.822786] [DEBUG] [LOCAL ] QS: Deleted derivative data file. [2025-10-20T20:45:47.822873] [INFO ] [MAIN ] Update Progress: 1755/2367 symbols completed. QS took 35.6s. Estimated remaining: 1h 18m 43s. Rate: 7.8 symbols/min [2025-10-20T20:45:48.012923] [INFO ] [MAIN ] QVCD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:48.020548] [DEBUG] [LOCAL ] QVCD: Loaded 4732 existing records from primary storage. [2025-10-20T20:45:48.020816] [INFO ] [MAIN ] QVCD: Loaded 4732 existing records. [2025-10-20T20:45:48.218775] [WARN ] [MAIN ] QVCD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:48.219018] [INFO ] [MAIN ] QVCD: Wiping existing primary data before full refresh. [2025-10-20T20:45:48.219071] [INFO ] [LOCAL ] QVCD: Starting deletion of primary data. [2025-10-20T20:45:48.219204] [DEBUG] [LOCAL ] QVCD: Deleted primary data file. [2025-10-20T20:45:48.219243] [INFO ] [MAIN ] QVCD: Starting full primary refresh. [2025-10-20T20:45:48.256688] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-03. [2025-10-20T20:45:48.585923] [INFO ] [REMOTE ] QVCD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:49.051394] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-09. [2025-10-20T20:45:49.568241] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T20:45:49.851411] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-04. [2025-10-20T20:45:50.651435] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-01. [2025-10-20T20:45:50.773361] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T20:45:51.186513] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T20:45:51.231782] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T20:45:51.451388] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-05. [2025-10-20T20:45:52.251434] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-02. [2025-10-20T20:45:52.790059] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T20:45:52.808584] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T20:45:53.051384] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-06. [2025-10-20T20:45:53.851446] [INFO ] [REMOTE ] QTWO: Fetching primary data for 2025-10. [2025-10-20T20:45:54.189959] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T20:45:54.651352] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-07. [2025-10-20T20:45:55.456581] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-03. [2025-10-20T20:45:55.470401] [INFO ] [REMOTE ] QTWO: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T20:45:55.493243] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:45:55.492544] [INFO ] [REMOTE ] QTWO: Full fetch complete (58160 records). [2025-10-20T20:45:55.493432] [DEBUG] [VALIDATOR ] QTWO: Validating primary data. [2025-10-20T20:45:55.878927] [DEBUG] [VALIDATOR ] QTWO: All expected trading days are present. [2025-10-20T20:45:55.879012] [DEBUG] [VALIDATOR ] QTWO: Primary data validation passed. [2025-10-20T20:45:55.879999] [INFO ] [MAIN ] QTWO: Fetched primary data is valid. [2025-10-20T20:45:55.880062] [DEBUG] [LOCAL ] QTWO: Saving primary data. [2025-10-20T20:45:55.881270] [DEBUG] [LOCAL ] QTWO: Trimmed 58160 records to 39535 (keeping 200 days of data) [2025-10-20T20:45:56.109899] [DEBUG] [LOCAL ] QTWO: Primary data saved (39535 records). [2025-10-20T20:45:56.110027] [INFO ] [MAIN ] QTWO: Full history saved. [2025-10-20T20:45:56.110100] [INFO ] [MAIN ] Update Progress: 1756/2367 symbols completed. QTWO took 32.9s. Estimated remaining: 1h 18m 35s. Rate: 7.8 symbols/min [2025-10-20T20:45:56.194263] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T20:45:56.256553] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-08. [2025-10-20T20:45:56.287888] [INFO ] [MAIN ] QXO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:45:56.338342] [DEBUG] [LOCAL ] QXO: Loaded 54207 existing records from primary storage. [2025-10-20T20:45:56.338668] [INFO ] [MAIN ] QXO: Loaded 54207 existing records. [2025-10-20T20:45:56.515056] [WARN ] [MAIN ] QXO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:45:56.515307] [INFO ] [MAIN ] QXO: Wiping existing primary data before full refresh. [2025-10-20T20:45:56.515360] [INFO ] [LOCAL ] QXO: Starting deletion of primary data. [2025-10-20T20:45:56.515577] [DEBUG] [LOCAL ] QXO: Deleted primary data file. [2025-10-20T20:45:56.515632] [INFO ] [MAIN ] QXO: Starting full primary refresh. [2025-10-20T20:45:56.858495] [INFO ] [REMOTE ] QXO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:45:57.051397] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-01. [2025-10-20T20:45:57.499143] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T20:45:57.851431] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-02. [2025-10-20T20:45:58.246629] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-10-20T20:45:58.651429] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-03. [2025-10-20T20:45:58.787408] [INFO ] [REMOTE ] QXO: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T20:45:59.451351] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-02. [2025-10-20T20:45:59.511276] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T20:46:00.251389] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-09. [2025-10-20T20:46:00.999113] [INFO ] [REMOTE ] QXO: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T20:46:01.042110] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T20:46:01.051327] [INFO ] [REMOTE ] QUAD: Fetching primary data for 2025-10. [2025-10-20T20:46:01.851354] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-04. [2025-10-20T20:46:02.117145] [INFO ] [REMOTE ] QUAD: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T20:46:02.120596] [INFO ] [REMOTE ] QUAD: Full fetch complete (34002 records). [2025-10-20T20:46:02.120735] [DEBUG] [VALIDATOR ] QUAD: Validating primary data. [2025-10-20T20:46:02.430783] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T20:46:02.504103] [DEBUG] [VALIDATOR ] QUAD: All expected trading days are present. [2025-10-20T20:46:02.504194] [DEBUG] [VALIDATOR ] QUAD: Primary data validation passed. [2025-10-20T20:46:02.504957] [INFO ] [MAIN ] QUAD: Fetched primary data is valid. [2025-10-20T20:46:02.505045] [DEBUG] [LOCAL ] QUAD: Saving primary data. [2025-10-20T20:46:02.506028] [DEBUG] [LOCAL ] QUAD: Trimmed 34002 records to 23177 (keeping 200 days of data) [2025-10-20T20:46:02.640428] [DEBUG] [LOCAL ] QUAD: Primary data saved (23177 records). [2025-10-20T20:46:02.640586] [INFO ] [MAIN ] QUAD: Full history saved. [2025-10-20T20:46:02.640704] [INFO ] [MAIN ] Update Progress: 1757/2367 symbols completed. QUAD took 24.0s. Estimated remaining: 1h 18m 27s. Rate: 7.8 symbols/min [2025-10-20T20:46:02.651365] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-05. [2025-10-20T20:46:03.451363] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-03. [2025-10-20T20:46:03.478894] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T20:46:04.251434] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-06. [2025-10-20T20:46:04.907241] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T20:46:05.051329] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-07. [2025-10-20T20:46:05.321133] [INFO ] [REMOTE ] QXO: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T20:46:05.456192] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T20:46:05.851405] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-08. [2025-10-20T20:46:06.540217] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-08 (0.69 seconds). [2025-10-20T20:46:06.651349] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-04. [2025-10-20T20:46:07.451380] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-04. [2025-10-20T20:46:07.527378] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T20:46:08.251368] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-09. [2025-10-20T20:46:08.872397] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T20:46:09.051406] [INFO ] [REMOTE ] QVCD: Fetching primary data for 2025-10. [2025-10-20T20:46:09.450732] [INFO ] [REMOTE ] QXO: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T20:46:09.686020] [INFO ] [REMOTE ] QVCD: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:46:09.687385] [INFO ] [REMOTE ] QVCD: Full fetch complete (6200 records). [2025-10-20T20:46:09.687458] [DEBUG] [VALIDATOR ] QVCD: Validating primary data. [2025-10-20T20:46:09.854217] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-05. [2025-10-20T20:46:10.063734] [DEBUG] [VALIDATOR ] QVCD: All expected trading days are present. [2025-10-20T20:46:10.063824] [DEBUG] [VALIDATOR ] QVCD: Primary data validation passed. [2025-10-20T20:46:10.064005] [INFO ] [MAIN ] QVCD: Fetched primary data is valid. [2025-10-20T20:46:10.064043] [DEBUG] [LOCAL ] QVCD: Saving primary data. [2025-10-20T20:46:10.064455] [DEBUG] [LOCAL ] QVCD: Trimmed 6200 records to 4608 (keeping 200 days of data) [2025-10-20T20:46:10.091404] [DEBUG] [LOCAL ] QVCD: Primary data saved (4608 records). [2025-10-20T20:46:10.091538] [INFO ] [MAIN ] QVCD: Full history saved. [2025-10-20T20:46:10.091615] [INFO ] [MAIN ] Update Progress: 1758/2367 symbols completed. QVCD took 22.3s. Estimated remaining: 1h 18m 19s. Rate: 7.8 symbols/min [2025-10-20T20:46:10.579249] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T20:46:10.651377] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-05. [2025-10-20T20:46:11.451423] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-06. [2025-10-20T20:46:12.295336] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T20:46:12.298273] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-07. [2025-10-20T20:46:12.683065] [INFO ] [REMOTE ] QXO: Incremental fetch complete for 2025-05 (2.03 seconds). [2025-10-20T20:46:12.908988] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T20:46:13.051400] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-08. [2025-10-20T20:46:13.851344] [INFO ] [REMOTE ] QXO: Fetching primary data for 2025-06. [2025-10-20T20:46:13.918011] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:46:13.958885] [DEBUG] [REMOTE ] QXO: Error fetching primary data for 2025-06: 'API call returned an empty response.' [2025-10-20T20:46:13.959266] [WARN ] [REMOTE ] QXO: No data returned for month 2025-06 [2025-10-20T20:46:13.959377] [FATAL] [REMOTE ] QXO: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:46:13.959495] [FATAL] [MAIN ] QXO: Failed to fetch primary data. [2025-10-20T20:46:13.959581] [WARN ] [MAIN ] QXO: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:46:13.959696] [INFO ] [LOCAL ] QXO: Starting deletion of primary data. [2025-10-20T20:46:13.959782] [DEBUG] [LOCAL ] QXO: No primary data file found to delete. [2025-10-20T20:46:13.959821] [INFO ] [LOCAL ] QXO: Starting deletion of derivative data. [2025-10-20T20:46:13.959934] [DEBUG] [LOCAL ] QXO: Deleted derivative data file. [2025-10-20T20:46:13.959994] [INFO ] [MAIN ] Update Progress: 1759/2367 symbols completed. QXO took 17.8s. Estimated remaining: 1h 18m 10s. Rate: 7.8 symbols/min [2025-10-20T20:46:14.651466] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-09. [2025-10-20T20:46:15.416413] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T20:46:15.451389] [INFO ] [REMOTE ] QVCC: Fetching primary data for 2025-10. [2025-10-20T20:46:16.019778] [INFO ] [REMOTE ] QVCC: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T20:46:16.021603] [INFO ] [REMOTE ] QVCC: Full fetch complete (11882 records). [2025-10-20T20:46:16.021695] [DEBUG] [VALIDATOR ] QVCC: Validating primary data. [2025-10-20T20:46:16.377318] [DEBUG] [VALIDATOR ] QVCC: All expected trading days are present. [2025-10-20T20:46:16.377389] [DEBUG] [VALIDATOR ] QVCC: Primary data validation passed. [2025-10-20T20:46:16.377673] [INFO ] [MAIN ] QVCC: Fetched primary data is valid. [2025-10-20T20:46:16.377731] [DEBUG] [LOCAL ] QVCC: Saving primary data. [2025-10-20T20:46:16.378185] [DEBUG] [LOCAL ] QVCC: Trimmed 11882 records to 8741 (keeping 200 days of data) [2025-10-20T20:46:16.427339] [DEBUG] [LOCAL ] QVCC: Primary data saved (8741 records). [2025-10-20T20:46:16.427440] [INFO ] [MAIN ] QVCC: Full history saved. [2025-10-20T20:46:16.427512] [INFO ] [MAIN ] Update Progress: 1760/2367 symbols completed. QVCC took 30.5s. Estimated remaining: 1h 18m 1s. Rate: 7.8 symbols/min [2025-10-20T20:46:16.980188] [INFO ] [MAIN ] RA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:17.074706] [INFO ] [MAIN ] RACE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:17.066247] [INFO ] [MAIN ] RAMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:17.136717] [INFO ] [MAIN ] R: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:17.155391] [DEBUG] [LOCAL ] RA: Loaded 22044 existing records from primary storage. [2025-10-20T20:46:17.155855] [INFO ] [MAIN ] RA: Loaded 22044 existing records. [2025-10-20T20:46:17.388853] [DEBUG] [LOCAL ] RACE: Loaded 33535 existing records from primary storage. [2025-10-20T20:46:17.395363] [WARN ] [MAIN ] RA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:17.395911] [DEBUG] [LOCAL ] RAMP: Loaded 38244 existing records from primary storage. [2025-10-20T20:46:17.404312] [INFO ] [MAIN ] RA: Wiping existing primary data before full refresh. [2025-10-20T20:46:17.405689] [INFO ] [LOCAL ] RA: Starting deletion of primary data. [2025-10-20T20:46:17.406512] [DEBUG] [LOCAL ] R: Loaded 28752 existing records from primary storage. [2025-10-20T20:46:17.405958] [INFO ] [MAIN ] RAMP: Loaded 38244 existing records. [2025-10-20T20:46:17.406840] [INFO ] [MAIN ] R: Loaded 28752 existing records. [2025-10-20T20:46:17.406103] [INFO ] [MAIN ] RACE: Loaded 33535 existing records. [2025-10-20T20:46:17.438211] [DEBUG] [LOCAL ] RA: Deleted primary data file. [2025-10-20T20:46:17.620376] [INFO ] [MAIN ] RA: Starting full primary refresh. [2025-10-20T20:46:18.058223] [WARN ] [MAIN ] R: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:18.063663] [INFO ] [MAIN ] R: Wiping existing primary data before full refresh. [2025-10-20T20:46:18.106837] [WARN ] [MAIN ] RACE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:18.100953] [INFO ] [LOCAL ] R: Starting deletion of primary data. [2025-10-20T20:46:18.053521] [WARN ] [MAIN ] RAMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:18.112815] [INFO ] [MAIN ] RACE: Wiping existing primary data before full refresh. [2025-10-20T20:46:18.123312] [INFO ] [LOCAL ] RACE: Starting deletion of primary data. [2025-10-20T20:46:18.128607] [INFO ] [MAIN ] RAMP: Wiping existing primary data before full refresh. [2025-10-20T20:46:18.133904] [INFO ] [LOCAL ] RAMP: Starting deletion of primary data. [2025-10-20T20:46:18.146573] [DEBUG] [LOCAL ] R: Deleted primary data file. [2025-10-20T20:46:18.159624] [INFO ] [MAIN ] R: Starting full primary refresh. [2025-10-20T20:46:18.133961] [DEBUG] [LOCAL ] RACE: Deleted primary data file. [2025-10-20T20:46:18.187260] [INFO ] [MAIN ] RACE: Starting full primary refresh. [2025-10-20T20:46:18.209462] [DEBUG] [LOCAL ] RAMP: Deleted primary data file. [2025-10-20T20:46:18.214979] [INFO ] [MAIN ] RAMP: Starting full primary refresh. [2025-10-20T20:46:18.913729] [INFO ] [REMOTE ] RA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:18.969928] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-01. [2025-10-20T20:46:19.474455] [INFO ] [REMOTE ] RACE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:19.469679] [INFO ] [REMOTE ] RAMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:19.485113] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-01. [2025-10-20T20:46:19.511095] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-01. [2025-10-20T20:46:19.540060] [INFO ] [REMOTE ] R: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:19.540379] [INFO ] [REMOTE ] R: Fetching primary data for 2025-01. [2025-10-20T20:46:20.655842] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T20:46:20.660365] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-02. [2025-10-20T20:46:20.835833] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T20:46:20.842625] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-02. [2025-10-20T20:46:20.923197] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-01 (1.38 seconds). [2025-10-20T20:46:21.051382] [INFO ] [REMOTE ] R: Fetching primary data for 2025-02. [2025-10-20T20:46:21.082616] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:46:21.478969] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:46:21.851385] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-02. [2025-10-20T20:46:22.047313] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T20:46:22.402291] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T20:46:22.651422] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-03. [2025-10-20T20:46:23.432875] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T20:46:23.451339] [INFO ] [REMOTE ] R: Fetching primary data for 2025-03. [2025-10-20T20:46:23.851187] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T20:46:24.251419] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-03. [2025-10-20T20:46:24.782442] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T20:46:25.051463] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-03. [2025-10-20T20:46:25.851345] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-04. [2025-10-20T20:46:26.034297] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T20:46:26.651406] [INFO ] [REMOTE ] R: Fetching primary data for 2025-04. [2025-10-20T20:46:26.693344] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T20:46:26.940909] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:46:27.451385] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-04. [2025-10-20T20:46:28.251380] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-05. [2025-10-20T20:46:28.386840] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T20:46:28.856786] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T20:46:29.051407] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-05. [2025-10-20T20:46:29.410854] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:46:29.851440] [INFO ] [REMOTE ] R: Fetching primary data for 2025-05. [2025-10-20T20:46:30.661834] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-06. [2025-10-20T20:46:30.664366] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T20:46:31.251349] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T20:46:31.451418] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-04. [2025-10-20T20:46:31.856584] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T20:46:32.251380] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-06. [2025-10-20T20:46:33.051382] [INFO ] [REMOTE ] R: Fetching primary data for 2025-06. [2025-10-20T20:46:33.262880] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T20:46:33.776805] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T20:46:33.851410] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-07. [2025-10-20T20:46:33.934259] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T20:46:34.651383] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-07. [2025-10-20T20:46:35.066258] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T20:46:35.451376] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-05. [2025-10-20T20:46:35.771188] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-07 (1.12 seconds). [2025-10-20T20:46:36.251368] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-08. [2025-10-20T20:46:36.636891] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T20:46:37.051418] [INFO ] [REMOTE ] R: Fetching primary data for 2025-07. [2025-10-20T20:46:37.792902] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T20:46:37.851335] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-09. [2025-10-20T20:46:38.515539] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T20:46:38.651382] [INFO ] [REMOTE ] R: Fetching primary data for 2025-08. [2025-10-20T20:46:39.027047] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:46:39.451425] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-06. [2025-10-20T20:46:39.586353] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T20:46:40.251370] [INFO ] [REMOTE ] R: Fetching primary data for 2025-09. [2025-10-20T20:46:41.051398] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-08. [2025-10-20T20:46:41.200797] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T20:46:41.517454] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T20:46:41.851394] [INFO ] [REMOTE ] RAMP: Fetching primary data for 2025-10. [2025-10-20T20:46:42.282223] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T20:46:42.410587] [INFO ] [REMOTE ] RAMP: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:46:42.416462] [INFO ] [REMOTE ] RAMP: Full fetch complete (54329 records). [2025-10-20T20:46:42.416555] [DEBUG] [VALIDATOR ] RAMP: Validating primary data. [2025-10-20T20:46:42.417198] [FATAL] [VALIDATOR ] RAMP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:46:42.417280] [FATAL] [MAIN ] RAMP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:46:42.417322] [WARN ] [MAIN ] RAMP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:46:42.417383] [INFO ] [LOCAL ] RAMP: Starting deletion of primary data. [2025-10-20T20:46:42.417443] [DEBUG] [LOCAL ] RAMP: No primary data file found to delete. [2025-10-20T20:46:42.417478] [INFO ] [LOCAL ] RAMP: Starting deletion of derivative data. [2025-10-20T20:46:42.417601] [DEBUG] [LOCAL ] RAMP: Deleted derivative data file. [2025-10-20T20:46:42.418112] [INFO ] [MAIN ] Update Progress: 1761/2367 symbols completed. RAMP took 26.0s. Estimated remaining: 1h 17m 59s. Rate: 7.8 symbols/min [2025-10-20T20:46:42.584388] [INFO ] [MAIN ] RBA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:42.629454] [DEBUG] [LOCAL ] RBA: Loaded 46764 existing records from primary storage. [2025-10-20T20:46:42.629795] [INFO ] [MAIN ] RBA: Loaded 46764 existing records. [2025-10-20T20:46:42.659627] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-07. [2025-10-20T20:46:42.811979] [WARN ] [MAIN ] RBA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:42.812199] [INFO ] [MAIN ] RBA: Wiping existing primary data before full refresh. [2025-10-20T20:46:42.812244] [INFO ] [LOCAL ] RBA: Starting deletion of primary data. [2025-10-20T20:46:42.812425] [DEBUG] [LOCAL ] RBA: Deleted primary data file. [2025-10-20T20:46:42.812462] [INFO ] [MAIN ] RBA: Starting full primary refresh. [2025-10-20T20:46:43.184528] [INFO ] [REMOTE ] RBA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:43.451373] [INFO ] [REMOTE ] R: Fetching primary data for 2025-10. [2025-10-20T20:46:44.092634] [INFO ] [REMOTE ] R: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T20:46:44.096729] [INFO ] [REMOTE ] R: Full fetch complete (42131 records). [2025-10-20T20:46:44.096824] [DEBUG] [VALIDATOR ] R: Validating primary data. [2025-10-20T20:46:44.097395] [FATAL] [VALIDATOR ] R: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:46:44.097475] [FATAL] [MAIN ] R: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:46:44.097558] [WARN ] [MAIN ] R: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:46:44.097627] [INFO ] [LOCAL ] R: Starting deletion of primary data. [2025-10-20T20:46:44.097723] [DEBUG] [LOCAL ] R: No primary data file found to delete. [2025-10-20T20:46:44.097800] [INFO ] [LOCAL ] R: Starting deletion of derivative data. [2025-10-20T20:46:44.097956] [DEBUG] [LOCAL ] R: Deleted derivative data file. [2025-10-20T20:46:44.098044] [INFO ] [MAIN ] Update Progress: 1762/2367 symbols completed. R took 27.5s. Estimated remaining: 1h 17m 50s. Rate: 7.8 symbols/min [2025-10-20T20:46:44.253716] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-01. [2025-10-20T20:46:44.280592] [INFO ] [MAIN ] RBC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:44.298120] [DEBUG] [LOCAL ] RBC: Loaded 15633 existing records from primary storage. [2025-10-20T20:46:44.298379] [INFO ] [MAIN ] RBC: Loaded 15633 existing records. [2025-10-20T20:46:44.433090] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T20:46:44.521135] [WARN ] [MAIN ] RBC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:44.521381] [INFO ] [MAIN ] RBC: Wiping existing primary data before full refresh. [2025-10-20T20:46:44.521430] [INFO ] [LOCAL ] RBC: Starting deletion of primary data. [2025-10-20T20:46:44.521679] [DEBUG] [LOCAL ] RBC: Deleted primary data file. [2025-10-20T20:46:44.521737] [INFO ] [MAIN ] RBC: Starting full primary refresh. [2025-10-20T20:46:44.884246] [INFO ] [REMOTE ] RBC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:45.051331] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-01. [2025-10-20T20:46:45.851358] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-09. [2025-10-20T20:46:45.953927] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:46:46.054130] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T20:46:46.651401] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-08. [2025-10-20T20:46:46.881088] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T20:46:47.451360] [INFO ] [REMOTE ] RA: Fetching primary data for 2025-10. [2025-10-20T20:46:48.251355] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-02. [2025-10-20T20:46:48.313587] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T20:46:48.330528] [INFO ] [REMOTE ] RA: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T20:46:48.334213] [INFO ] [REMOTE ] RA: Full fetch complete (31811 records). [2025-10-20T20:46:48.334322] [DEBUG] [VALIDATOR ] RA: Validating primary data. [2025-10-20T20:46:48.717512] [DEBUG] [VALIDATOR ] RA: All expected trading days are present. [2025-10-20T20:46:48.717623] [DEBUG] [VALIDATOR ] RA: Primary data validation passed. [2025-10-20T20:46:48.718370] [INFO ] [MAIN ] RA: Fetched primary data is valid. [2025-10-20T20:46:48.718433] [DEBUG] [LOCAL ] RA: Saving primary data. [2025-10-20T20:46:48.719165] [DEBUG] [LOCAL ] RA: Trimmed 31811 records to 21865 (keeping 200 days of data) [2025-10-20T20:46:48.852087] [DEBUG] [LOCAL ] RA: Primary data saved (21865 records). [2025-10-20T20:46:48.852220] [INFO ] [MAIN ] RA: Full history saved. [2025-10-20T20:46:48.852299] [INFO ] [MAIN ] Update Progress: 1763/2367 symbols completed. RA took 32.4s. Estimated remaining: 1h 17m 41s. Rate: 7.8 symbols/min [2025-10-20T20:46:49.018568] [INFO ] [MAIN ] RBLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:49.051370] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-09. [2025-10-20T20:46:49.070400] [DEBUG] [LOCAL ] RBLX: Loaded 54498 existing records from primary storage. [2025-10-20T20:46:49.070857] [INFO ] [MAIN ] RBLX: Loaded 54498 existing records. [2025-10-20T20:46:49.267737] [WARN ] [MAIN ] RBLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:49.268049] [INFO ] [MAIN ] RBLX: Wiping existing primary data before full refresh. [2025-10-20T20:46:49.268106] [INFO ] [LOCAL ] RBLX: Starting deletion of primary data. [2025-10-20T20:46:49.268302] [DEBUG] [LOCAL ] RBLX: Deleted primary data file. [2025-10-20T20:46:49.268339] [INFO ] [MAIN ] RBLX: Starting full primary refresh. [2025-10-20T20:46:49.406986] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T20:46:49.649355] [INFO ] [REMOTE ] RBLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:49.851423] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-02. [2025-10-20T20:46:50.623546] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T20:46:50.651363] [INFO ] [REMOTE ] RACE: Fetching primary data for 2025-10. [2025-10-20T20:46:51.451369] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-03. [2025-10-20T20:46:51.548360] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:46:52.251341] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-03. [2025-10-20T20:46:52.286134] [INFO ] [REMOTE ] RACE: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T20:46:52.292606] [INFO ] [REMOTE ] RACE: Full fetch complete (49266 records). [2025-10-20T20:46:52.292725] [DEBUG] [VALIDATOR ] RACE: Validating primary data. [2025-10-20T20:46:52.423042] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T20:46:52.674281] [DEBUG] [VALIDATOR ] RACE: All expected trading days are present. [2025-10-20T20:46:52.674365] [DEBUG] [VALIDATOR ] RACE: Primary data validation passed. [2025-10-20T20:46:52.675169] [INFO ] [MAIN ] RACE: Fetched primary data is valid. [2025-10-20T20:46:52.675230] [DEBUG] [LOCAL ] RACE: Saving primary data. [2025-10-20T20:46:52.676163] [DEBUG] [LOCAL ] RACE: Trimmed 49266 records to 32933 (keeping 200 days of data) [2025-10-20T20:46:52.869015] [DEBUG] [LOCAL ] RACE: Primary data saved (32933 records). [2025-10-20T20:46:52.869147] [INFO ] [MAIN ] RACE: Full history saved. [2025-10-20T20:46:52.869220] [INFO ] [MAIN ] Update Progress: 1764/2367 symbols completed. RACE took 36.4s. Estimated remaining: 1h 17m 32s. Rate: 7.8 symbols/min [2025-10-20T20:46:53.033496] [INFO ] [MAIN ] RBOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:46:53.041045] [DEBUG] [LOCAL ] RBOT: Loaded 6207 existing records from primary storage. [2025-10-20T20:46:53.041216] [INFO ] [MAIN ] RBOT: Loaded 6207 existing records. [2025-10-20T20:46:53.061368] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-04. [2025-10-20T20:46:53.216518] [WARN ] [MAIN ] RBOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:46:53.216745] [INFO ] [MAIN ] RBOT: Wiping existing primary data before full refresh. [2025-10-20T20:46:53.216796] [INFO ] [LOCAL ] RBOT: Starting deletion of primary data. [2025-10-20T20:46:53.216952] [DEBUG] [LOCAL ] RBOT: Deleted primary data file. [2025-10-20T20:46:53.217008] [INFO ] [MAIN ] RBOT: Starting full primary refresh. [2025-10-20T20:46:53.561231] [INFO ] [REMOTE ] RBOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:46:53.851344] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-01. [2025-10-20T20:46:53.958196] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T20:46:54.193102] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T20:46:54.442979] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T20:46:54.651385] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-02. [2025-10-20T20:46:55.431124] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:46:55.451342] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-04. [2025-10-20T20:46:56.251370] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-03. [2025-10-20T20:46:56.869673] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T20:46:57.051367] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-05. [2025-10-20T20:46:57.153809] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:46:57.805203] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:46:57.851344] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-06. [2025-10-20T20:46:58.553004] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T20:46:58.651372] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-04. [2025-10-20T20:46:59.317562] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:46:59.451358] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-05. [2025-10-20T20:47:00.015270] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T20:47:00.251364] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-05. [2025-10-20T20:47:01.051423] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-07. [2025-10-20T20:47:01.778859] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T20:47:01.851346] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-06. [2025-10-20T20:47:02.078999] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T20:47:02.651373] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-08. [2025-10-20T20:47:03.451340] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-01. [2025-10-20T20:47:03.560615] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T20:47:03.794977] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T20:47:04.251413] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-09. [2025-10-20T20:47:05.051432] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-06. [2025-10-20T20:47:05.294371] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T20:47:05.601388] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T20:47:05.851340] [INFO ] [REMOTE ] RBC: Fetching primary data for 2025-10. [2025-10-20T20:47:05.941090] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-01 (2.49 seconds). [2025-10-20T20:47:06.382365] [INFO ] [REMOTE ] RBC: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:47:06.385232] [INFO ] [REMOTE ] RBC: Full fetch complete (21695 records). [2025-10-20T20:47:06.385304] [DEBUG] [VALIDATOR ] RBC: Validating primary data. [2025-10-20T20:47:06.385810] [FATAL] [VALIDATOR ] RBC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:47:06.385889] [FATAL] [MAIN ] RBC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:47:06.385930] [WARN ] [MAIN ] RBC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:47:06.385968] [INFO ] [LOCAL ] RBC: Starting deletion of primary data. [2025-10-20T20:47:06.386035] [DEBUG] [LOCAL ] RBC: No primary data file found to delete. [2025-10-20T20:47:06.386098] [INFO ] [LOCAL ] RBC: Starting deletion of derivative data. [2025-10-20T20:47:06.386231] [DEBUG] [LOCAL ] RBC: Deleted derivative data file. [2025-10-20T20:47:06.386295] [INFO ] [MAIN ] Update Progress: 1765/2367 symbols completed. RBC took 22.3s. Estimated remaining: 1h 17m 26s. Rate: 7.8 symbols/min [2025-10-20T20:47:06.554165] [INFO ] [MAIN ] RBRK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:06.602927] [DEBUG] [LOCAL ] RBRK: Loaded 53944 existing records from primary storage. [2025-10-20T20:47:06.603227] [INFO ] [MAIN ] RBRK: Loaded 53944 existing records. [2025-10-20T20:47:06.656607] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-02. [2025-10-20T20:47:06.783479] [WARN ] [MAIN ] RBRK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:47:06.783718] [INFO ] [MAIN ] RBRK: Wiping existing primary data before full refresh. [2025-10-20T20:47:06.783771] [INFO ] [LOCAL ] RBRK: Starting deletion of primary data. [2025-10-20T20:47:06.783982] [DEBUG] [LOCAL ] RBRK: Deleted primary data file. [2025-10-20T20:47:06.784037] [INFO ] [MAIN ] RBRK: Starting full primary refresh. [2025-10-20T20:47:07.132446] [INFO ] [REMOTE ] RBRK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:07.451423] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-01. [2025-10-20T20:47:08.251370] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-07. [2025-10-20T20:47:09.051398] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-07. [2025-10-20T20:47:09.149503] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-02 (2.49 seconds). [2025-10-20T20:47:09.509435] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T20:47:09.812607] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T20:47:09.851327] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-08. [2025-10-20T20:47:10.002214] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T20:47:10.651414] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-02. [2025-10-20T20:47:11.456737] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-03. [2025-10-20T20:47:11.478875] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T20:47:12.251396] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-08. [2025-10-20T20:47:12.548435] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T20:47:12.897898] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T20:47:13.051356] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-09. [2025-10-20T20:47:13.657118] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-03 (2.20 seconds). [2025-10-20T20:47:13.851352] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-09. [2025-10-20T20:47:14.441039] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T20:47:14.651379] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-04. [2025-10-20T20:47:14.797261] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T20:47:15.451390] [INFO ] [REMOTE ] RBA: Fetching primary data for 2025-10. [2025-10-20T20:47:15.985975] [INFO ] [REMOTE ] RBA: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T20:47:15.993878] [INFO ] [REMOTE ] RBA: Full fetch complete (66166 records). [2025-10-20T20:47:15.994121] [DEBUG] [VALIDATOR ] RBA: Validating primary data. [2025-10-20T20:47:15.995342] [FATAL] [VALIDATOR ] RBA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:47:15.995487] [FATAL] [MAIN ] RBA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:47:15.995572] [WARN ] [MAIN ] RBA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:47:15.995687] [INFO ] [LOCAL ] RBA: Starting deletion of primary data. [2025-10-20T20:47:15.995793] [DEBUG] [LOCAL ] RBA: No primary data file found to delete. [2025-10-20T20:47:15.995859] [INFO ] [LOCAL ] RBA: Starting deletion of derivative data. [2025-10-20T20:47:15.996058] [DEBUG] [LOCAL ] RBA: Deleted derivative data file. [2025-10-20T20:47:15.996626] [INFO ] [MAIN ] Update Progress: 1766/2367 symbols completed. RBA took 33.6s. Estimated remaining: 1h 17m 19s. Rate: 7.8 symbols/min [2025-10-20T20:47:16.171127] [INFO ] [MAIN ] RC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:16.217362] [DEBUG] [LOCAL ] RC: Loaded 49641 existing records from primary storage. [2025-10-20T20:47:16.217715] [INFO ] [MAIN ] RC: Loaded 49641 existing records. [2025-10-20T20:47:16.259310] [INFO ] [REMOTE ] RBOT: Fetching primary data for 2025-10. [2025-10-20T20:47:16.396748] [WARN ] [MAIN ] RC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:47:16.397012] [INFO ] [MAIN ] RC: Wiping existing primary data before full refresh. [2025-10-20T20:47:16.397061] [INFO ] [LOCAL ] RC: Starting deletion of primary data. [2025-10-20T20:47:16.397254] [DEBUG] [LOCAL ] RC: Deleted primary data file. [2025-10-20T20:47:16.397292] [INFO ] [MAIN ] RC: Starting full primary refresh. [2025-10-20T20:47:16.781997] [INFO ] [REMOTE ] RC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:16.818418] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-04 (2.17 seconds). [2025-10-20T20:47:17.051340] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-03. [2025-10-20T20:47:17.283888] [INFO ] [REMOTE ] RBOT: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T20:47:17.287689] [INFO ] [REMOTE ] RBOT: Full fetch complete (9295 records). [2025-10-20T20:47:17.287773] [DEBUG] [VALIDATOR ] RBOT: Validating primary data. [2025-10-20T20:47:17.638967] [DEBUG] [VALIDATOR ] RBOT: All expected trading days are present. [2025-10-20T20:47:17.639041] [DEBUG] [VALIDATOR ] RBOT: Primary data validation passed. [2025-10-20T20:47:17.639214] [INFO ] [MAIN ] RBOT: Fetched primary data is valid. [2025-10-20T20:47:17.639251] [DEBUG] [LOCAL ] RBOT: Saving primary data. [2025-10-20T20:47:17.639690] [DEBUG] [LOCAL ] RBOT: Trimmed 9295 records to 6187 (keeping 200 days of data) [2025-10-20T20:47:17.676121] [DEBUG] [LOCAL ] RBOT: Primary data saved (6187 records). [2025-10-20T20:47:17.676334] [INFO ] [MAIN ] RBOT: Full history saved. [2025-10-20T20:47:17.676450] [INFO ] [MAIN ] Update Progress: 1767/2367 symbols completed. RBOT took 24.8s. Estimated remaining: 1h 17m 9s. Rate: 7.8 symbols/min [2025-10-20T20:47:17.856610] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-01. [2025-10-20T20:47:17.860396] [INFO ] [MAIN ] RCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:17.861552] [DEBUG] [LOCAL ] RCB: No existing data file found in primary storage. [2025-10-20T20:47:17.861608] [WARN ] [MAIN ] RCB: No existing data found, performing full refresh instead. [2025-10-20T20:47:17.861773] [INFO ] [MAIN ] RCB: Wiping existing primary data before full refresh. [2025-10-20T20:47:17.861820] [INFO ] [LOCAL ] RCB: Starting deletion of primary data. [2025-10-20T20:47:17.861866] [DEBUG] [LOCAL ] RCB: No primary data file found to delete. [2025-10-20T20:47:17.861901] [INFO ] [MAIN ] RCB: Starting full primary refresh. [2025-10-20T20:47:18.231851] [INFO ] [REMOTE ] RCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:18.651346] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-05. [2025-10-20T20:47:18.980060] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T20:47:19.451342] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-04. [2025-10-20T20:47:19.764869] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T20:47:20.251414] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-02. [2025-10-20T20:47:20.992848] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-05 (2.34 seconds). [2025-10-20T20:47:21.051363] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-01. [2025-10-20T20:47:21.185292] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:47:21.670678] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T20:47:21.689410] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T20:47:21.851389] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-02. [2025-10-20T20:47:22.499422] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:47:22.651335] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-03. [2025-10-20T20:47:23.170028] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T20:47:23.451408] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-04. [2025-10-20T20:47:24.065218] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-04 (0.61 seconds). [2025-10-20T20:47:24.251340] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-03. [2025-10-20T20:47:25.051376] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-05. [2025-10-20T20:47:25.669959] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T20:47:25.851410] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-05. [2025-10-20T20:47:26.230871] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T20:47:26.651444] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-06. [2025-10-20T20:47:27.132277] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T20:47:27.451421] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-07. [2025-10-20T20:47:27.713423] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T20:47:27.839125] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T20:47:28.251400] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-04. [2025-10-20T20:47:29.051491] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-06. [2025-10-20T20:47:29.851384] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-08. [2025-10-20T20:47:30.029882] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T20:47:30.287714] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-08 (0.44 seconds). [2025-10-20T20:47:30.651364] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-09. [2025-10-20T20:47:31.042530] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-10-20T20:47:31.321871] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-06 (2.27 seconds). [2025-10-20T20:47:31.451374] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-05. [2025-10-20T20:47:32.251371] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-07. [2025-10-20T20:47:33.051361] [INFO ] [REMOTE ] RCB: Fetching primary data for 2025-10. [2025-10-20T20:47:33.325365] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T20:47:33.509816] [INFO ] [REMOTE ] RCB: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T20:47:33.510689] [INFO ] [REMOTE ] RCB: Full fetch complete (3713 records). [2025-10-20T20:47:33.510757] [DEBUG] [VALIDATOR ] RCB: Validating primary data. [2025-10-20T20:47:33.683593] [WARN ] [VALIDATOR ] RCB: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T20:47:33.683875] [FATAL] [MAIN ] RCB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:47:33.683931] [WARN ] [MAIN ] RCB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:47:33.683970] [INFO ] [LOCAL ] RCB: Starting deletion of primary data. [2025-10-20T20:47:33.684022] [DEBUG] [LOCAL ] RCB: No primary data file found to delete. [2025-10-20T20:47:33.684055] [INFO ] [LOCAL ] RCB: Starting deletion of derivative data. [2025-10-20T20:47:33.684094] [DEBUG] [LOCAL ] RCB: No derivative data file found to delete. [2025-10-20T20:47:33.684186] [INFO ] [MAIN ] Update Progress: 1768/2367 symbols completed. RCB took 16.0s. Estimated remaining: 1h 17m 4s. Rate: 7.8 symbols/min [2025-10-20T20:47:33.848270] [INFO ] [MAIN ] RCD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:33.851801] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-06. [2025-10-20T20:47:33.856141] [DEBUG] [LOCAL ] RCD: Loaded 3432 existing records from primary storage. [2025-10-20T20:47:33.856411] [INFO ] [MAIN ] RCD: Loaded 3432 existing records. [2025-10-20T20:47:34.033602] [WARN ] [MAIN ] RCD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:47:34.033957] [INFO ] [MAIN ] RCD: Wiping existing primary data before full refresh. [2025-10-20T20:47:34.034036] [INFO ] [LOCAL ] RCD: Starting deletion of primary data. [2025-10-20T20:47:34.034315] [DEBUG] [LOCAL ] RCD: Deleted primary data file. [2025-10-20T20:47:34.034434] [INFO ] [MAIN ] RCD: Starting full primary refresh. [2025-10-20T20:47:34.388350] [INFO ] [REMOTE ] RCD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:34.419871] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-10-20T20:47:34.651404] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-08. [2025-10-20T20:47:35.451425] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-06. [2025-10-20T20:47:35.601141] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T20:47:36.251413] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-07. [2025-10-20T20:47:36.700002] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T20:47:37.051433] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-09. [2025-10-20T20:47:37.851360] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-01. [2025-10-20T20:47:37.914901] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-06 (2.46 seconds). [2025-10-20T20:47:38.215054] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T20:47:38.628744] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T20:47:38.651328] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-02. [2025-10-20T20:47:39.018579] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-02 (0.37 seconds). [2025-10-20T20:47:39.085593] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T20:47:39.451397] [INFO ] [REMOTE ] RBRK: Fetching primary data for 2025-10. [2025-10-20T20:47:40.251376] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-08. [2025-10-20T20:47:41.051424] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-03. [2025-10-20T20:47:41.418789] [INFO ] [REMOTE ] RBRK: Incremental fetch complete for 2025-10 (1.97 seconds). [2025-10-20T20:47:41.426467] [INFO ] [REMOTE ] RBRK: Full fetch complete (76702 records). [2025-10-20T20:47:41.426594] [DEBUG] [VALIDATOR ] RBRK: Validating primary data. [2025-10-20T20:47:41.828528] [DEBUG] [VALIDATOR ] RBRK: All expected trading days are present. [2025-10-20T20:47:41.828763] [DEBUG] [VALIDATOR ] RBRK: Primary data validation passed. [2025-10-20T20:47:41.830036] [INFO ] [MAIN ] RBRK: Fetched primary data is valid. [2025-10-20T20:47:41.830120] [DEBUG] [LOCAL ] RBRK: Saving primary data. [2025-10-20T20:47:41.840424] [DEBUG] [LOCAL ] RBRK: Trimmed 76702 records to 53169 (keeping 200 days of data) [2025-10-20T20:47:41.840313] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:47:41.893971] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-04. [2025-10-20T20:47:42.153501] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T20:47:42.210543] [DEBUG] [LOCAL ] RBRK: Primary data saved (53169 records). [2025-10-20T20:47:42.210713] [INFO ] [MAIN ] RBRK: Full history saved. [2025-10-20T20:47:42.210804] [INFO ] [MAIN ] Update Progress: 1769/2367 symbols completed. RBRK took 35.8s. Estimated remaining: 1h 16m 57s. Rate: 7.8 symbols/min [2025-10-20T20:47:42.377634] [INFO ] [MAIN ] RCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:42.419928] [DEBUG] [LOCAL ] RCI: Loaded 49399 existing records from primary storage. [2025-10-20T20:47:42.420220] [INFO ] [MAIN ] RCI: Loaded 49399 existing records. [2025-10-20T20:47:42.602377] [WARN ] [MAIN ] RCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:47:42.605758] [INFO ] [MAIN ] RCI: Wiping existing primary data before full refresh. [2025-10-20T20:47:42.605605] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T20:47:42.605909] [INFO ] [LOCAL ] RCI: Starting deletion of primary data. [2025-10-20T20:47:42.608295] [DEBUG] [LOCAL ] RCI: Deleted primary data file. [2025-10-20T20:47:42.608353] [INFO ] [MAIN ] RCI: Starting full primary refresh. [2025-10-20T20:47:42.661124] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-09. [2025-10-20T20:47:42.967851] [INFO ] [REMOTE ] RCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:43.451424] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-01. [2025-10-20T20:47:44.251387] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-07. [2025-10-20T20:47:44.487884] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T20:47:45.057052] [INFO ] [REMOTE ] RC: Fetching primary data for 2025-10. [2025-10-20T20:47:45.074271] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T20:47:45.851377] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-02. [2025-10-20T20:47:45.890015] [INFO ] [REMOTE ] RC: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T20:47:45.896969] [INFO ] [REMOTE ] RC: Full fetch complete (70701 records). [2025-10-20T20:47:45.897079] [DEBUG] [VALIDATOR ] RC: Validating primary data. [2025-10-20T20:47:45.897824] [FATAL] [VALIDATOR ] RC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:47:45.897904] [FATAL] [MAIN ] RC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:47:45.897980] [WARN ] [MAIN ] RC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:47:45.898029] [INFO ] [LOCAL ] RC: Starting deletion of primary data. [2025-10-20T20:47:45.898084] [DEBUG] [LOCAL ] RC: No primary data file found to delete. [2025-10-20T20:47:45.898125] [INFO ] [LOCAL ] RC: Starting deletion of derivative data. [2025-10-20T20:47:45.898475] [DEBUG] [LOCAL ] RC: Deleted derivative data file. [2025-10-20T20:47:45.898922] [INFO ] [MAIN ] Update Progress: 1770/2367 symbols completed. RC took 29.9s. Estimated remaining: 1h 16m 48s. Rate: 7.8 symbols/min [2025-10-20T20:47:46.066350] [INFO ] [MAIN ] RCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:46.066531] [DEBUG] [LOCAL ] RCL: No existing data file found in primary storage. [2025-10-20T20:47:46.066569] [WARN ] [MAIN ] RCL: No existing data found, performing full refresh instead. [2025-10-20T20:47:46.066700] [INFO ] [MAIN ] RCL: Wiping existing primary data before full refresh. [2025-10-20T20:47:46.066751] [INFO ] [LOCAL ] RCL: Starting deletion of primary data. [2025-10-20T20:47:46.066794] [DEBUG] [LOCAL ] RCL: No primary data file found to delete. [2025-10-20T20:47:46.066827] [INFO ] [MAIN ] RCL: Starting full primary refresh. [2025-10-20T20:47:46.424053] [INFO ] [REMOTE ] RCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:46.651346] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-01. [2025-10-20T20:47:46.841608] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-07 (2.59 seconds). [2025-10-20T20:47:47.451370] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-08. [2025-10-20T20:47:47.559068] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T20:47:48.251413] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-03. [2025-10-20T20:47:48.596068] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T20:47:49.051378] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-05. [2025-10-20T20:47:49.644060] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T20:47:49.727789] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-10-20T20:47:49.818974] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T20:47:49.851339] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-04. [2025-10-20T20:47:50.651393] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-02. [2025-10-20T20:47:51.451405] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-09. [2025-10-20T20:47:51.685410] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:47:52.251399] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-05. [2025-10-20T20:47:52.545071] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-02 (1.89 seconds). [2025-10-20T20:47:53.051361] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-06. [2025-10-20T20:47:53.527004] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T20:47:53.782054] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T20:47:53.851362] [INFO ] [REMOTE ] RBLX: Fetching primary data for 2025-10. [2025-10-20T20:47:53.920221] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T20:47:54.508566] [INFO ] [REMOTE ] RBLX: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T20:47:54.515986] [INFO ] [REMOTE ] RBLX: Full fetch complete (77496 records). [2025-10-20T20:47:54.516132] [DEBUG] [VALIDATOR ] RBLX: Validating primary data. [2025-10-20T20:47:54.516861] [FATAL] [VALIDATOR ] RBLX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:47:54.516945] [FATAL] [MAIN ] RBLX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:47:54.516988] [WARN ] [MAIN ] RBLX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:47:54.517051] [INFO ] [LOCAL ] RBLX: Starting deletion of primary data. [2025-10-20T20:47:54.517114] [DEBUG] [LOCAL ] RBLX: No primary data file found to delete. [2025-10-20T20:47:54.517150] [INFO ] [LOCAL ] RBLX: Starting deletion of derivative data. [2025-10-20T20:47:54.517266] [DEBUG] [LOCAL ] RBLX: Deleted derivative data file. [2025-10-20T20:47:54.517334] [INFO ] [MAIN ] Update Progress: 1771/2367 symbols completed. RBLX took 1m 6s. Estimated remaining: 1h 16m 40s. Rate: 7.8 symbols/min [2025-10-20T20:47:54.658124] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-03. [2025-10-20T20:47:54.715002] [INFO ] [MAIN ] RCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:47:54.728588] [DEBUG] [LOCAL ] RCS: Loaded 12509 existing records from primary storage. [2025-10-20T20:47:54.728816] [INFO ] [MAIN ] RCS: Loaded 12509 existing records. [2025-10-20T20:47:54.902535] [WARN ] [MAIN ] RCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:47:54.902776] [INFO ] [MAIN ] RCS: Wiping existing primary data before full refresh. [2025-10-20T20:47:54.902826] [INFO ] [LOCAL ] RCS: Starting deletion of primary data. [2025-10-20T20:47:54.902985] [DEBUG] [LOCAL ] RCS: Deleted primary data file. [2025-10-20T20:47:54.903022] [INFO ] [MAIN ] RCS: Starting full primary refresh. [2025-10-20T20:47:55.251157] [INFO ] [REMOTE ] RCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:47:55.451413] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-01. [2025-10-20T20:47:56.251431] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-07. [2025-10-20T20:47:56.447984] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T20:47:56.641585] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T20:47:56.811232] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T20:47:57.051428] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-04. [2025-10-20T20:47:57.851391] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-06. [2025-10-20T20:47:58.651468] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-08. [2025-10-20T20:47:59.177115] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T20:47:59.278981] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T20:47:59.451413] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-09. [2025-10-20T20:47:59.527580] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T20:48:00.075446] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T20:48:00.251358] [INFO ] [REMOTE ] RCD: Fetching primary data for 2025-10. [2025-10-20T20:48:00.714732] [INFO ] [REMOTE ] RCD: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T20:48:00.715505] [INFO ] [REMOTE ] RCD: Full fetch complete (5958 records). [2025-10-20T20:48:00.715571] [DEBUG] [VALIDATOR ] RCD: Validating primary data. [2025-10-20T20:48:01.062548] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-05. [2025-10-20T20:48:01.069667] [DEBUG] [VALIDATOR ] RCD: All expected trading days are present. [2025-10-20T20:48:01.069738] [DEBUG] [VALIDATOR ] RCD: Primary data validation passed. [2025-10-20T20:48:01.069953] [INFO ] [MAIN ] RCD: Fetched primary data is valid. [2025-10-20T20:48:01.070023] [DEBUG] [LOCAL ] RCD: Saving primary data. [2025-10-20T20:48:01.070572] [DEBUG] [LOCAL ] RCD: Trimmed 5958 records to 3357 (keeping 200 days of data) [2025-10-20T20:48:01.091110] [DEBUG] [LOCAL ] RCD: Primary data saved (3357 records). [2025-10-20T20:48:01.091198] [INFO ] [MAIN ] RCD: Full history saved. [2025-10-20T20:48:01.091268] [INFO ] [MAIN ] Update Progress: 1772/2367 symbols completed. RCD took 27.4s. Estimated remaining: 1h 16m 32s. Rate: 7.8 symbols/min [2025-10-20T20:48:01.257664] [INFO ] [MAIN ] RCUS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:01.295483] [DEBUG] [LOCAL ] RCUS: Loaded 43674 existing records from primary storage. [2025-10-20T20:48:01.295797] [INFO ] [MAIN ] RCUS: Loaded 43674 existing records. [2025-10-20T20:48:01.491954] [WARN ] [MAIN ] RCUS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:01.492280] [INFO ] [MAIN ] RCUS: Wiping existing primary data before full refresh. [2025-10-20T20:48:01.492347] [INFO ] [LOCAL ] RCUS: Starting deletion of primary data. [2025-10-20T20:48:01.492638] [DEBUG] [LOCAL ] RCUS: Deleted primary data file. [2025-10-20T20:48:01.492759] [INFO ] [MAIN ] RCUS: Starting full primary refresh. [2025-10-20T20:48:01.872193] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-02. [2025-10-20T20:48:01.879357] [INFO ] [REMOTE ] RCUS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:02.651430] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-01. [2025-10-20T20:48:02.930033] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T20:48:03.168929] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T20:48:03.451379] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-07. [2025-10-20T20:48:04.051542] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T20:48:04.251394] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-02. [2025-10-20T20:48:05.051440] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-06. [2025-10-20T20:48:05.333366] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T20:48:05.857544] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-03. [2025-10-20T20:48:05.862329] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T20:48:06.651349] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-08. [2025-10-20T20:48:06.679547] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T20:48:06.967713] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-06 (1.92 seconds). [2025-10-20T20:48:07.451366] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-04. [2025-10-20T20:48:08.107042] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T20:48:08.251346] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-09. [2025-10-20T20:48:08.361342] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T20:48:09.051419] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-03. [2025-10-20T20:48:09.851418] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-05. [2025-10-20T20:48:09.939160] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T20:48:10.412548] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T20:48:10.651362] [INFO ] [REMOTE ] RCI: Fetching primary data for 2025-10. [2025-10-20T20:48:10.709257] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T20:48:11.451415] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-04. [2025-10-20T20:48:12.248640] [INFO ] [REMOTE ] RCI: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T20:48:12.255963] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-06. [2025-10-20T20:48:12.257896] [INFO ] [REMOTE ] RCI: Full fetch complete (70690 records). [2025-10-20T20:48:12.258069] [DEBUG] [VALIDATOR ] RCI: Validating primary data. [2025-10-20T20:48:12.711115] [DEBUG] [VALIDATOR ] RCI: All expected trading days are present. [2025-10-20T20:48:12.711276] [DEBUG] [VALIDATOR ] RCI: Primary data validation passed. [2025-10-20T20:48:12.712520] [INFO ] [MAIN ] RCI: Fetched primary data is valid. [2025-10-20T20:48:12.712616] [DEBUG] [LOCAL ] RCI: Saving primary data. [2025-10-20T20:48:12.714184] [DEBUG] [LOCAL ] RCI: Trimmed 70690 records to 48624 (keeping 200 days of data) [2025-10-20T20:48:13.032871] [DEBUG] [LOCAL ] RCI: Primary data saved (48624 records). [2025-10-20T20:48:13.033125] [INFO ] [MAIN ] RCI: Full history saved. [2025-10-20T20:48:13.033283] [INFO ] [MAIN ] Update Progress: 1773/2367 symbols completed. RCI took 30.8s. Estimated remaining: 1h 16m 26s. Rate: 7.8 symbols/min [2025-10-20T20:48:13.056035] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-07. [2025-10-20T20:48:13.183190] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:48:13.279244] [INFO ] [MAIN ] RDDT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:13.280325] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T20:48:13.337213] [DEBUG] [LOCAL ] RDDT: Loaded 54555 existing records from primary storage. [2025-10-20T20:48:13.337536] [INFO ] [MAIN ] RDDT: Loaded 54555 existing records. [2025-10-20T20:48:13.531502] [WARN ] [MAIN ] RDDT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:13.531752] [INFO ] [MAIN ] RDDT: Wiping existing primary data before full refresh. [2025-10-20T20:48:13.531807] [INFO ] [LOCAL ] RDDT: Starting deletion of primary data. [2025-10-20T20:48:13.532075] [DEBUG] [LOCAL ] RDDT: Deleted primary data file. [2025-10-20T20:48:13.532132] [INFO ] [MAIN ] RDDT: Starting full primary refresh. [2025-10-20T20:48:13.857394] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-07. [2025-10-20T20:48:13.952327] [INFO ] [REMOTE ] RDDT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:14.651618] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-05. [2025-10-20T20:48:14.653264] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T20:48:15.453224] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-08. [2025-10-20T20:48:15.455841] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-07 (2.40 seconds). [2025-10-20T20:48:16.251435] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-08. [2025-10-20T20:48:16.373246] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:48:16.391014] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T20:48:17.051442] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-09. [2025-10-20T20:48:17.851408] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-01. [2025-10-20T20:48:17.985270] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T20:48:18.315353] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T20:48:18.651437] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-09. [2025-10-20T20:48:19.451458] [INFO ] [REMOTE ] RCS: Fetching primary data for 2025-10. [2025-10-20T20:48:20.251445] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-06. [2025-10-20T20:48:20.293349] [INFO ] [REMOTE ] RCS: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:48:20.299029] [INFO ] [REMOTE ] RCS: Full fetch complete (19302 records). [2025-10-20T20:48:20.299259] [DEBUG] [VALIDATOR ] RCS: Validating primary data. [2025-10-20T20:48:20.683847] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-01 (2.83 seconds). [2025-10-20T20:48:20.890751] [DEBUG] [VALIDATOR ] RCS: All expected trading days are present. [2025-10-20T20:48:20.890963] [DEBUG] [VALIDATOR ] RCS: Primary data validation passed. [2025-10-20T20:48:20.891791] [INFO ] [MAIN ] RCS: Fetched primary data is valid. [2025-10-20T20:48:20.891872] [DEBUG] [LOCAL ] RCS: Saving primary data. [2025-10-20T20:48:20.893051] [DEBUG] [LOCAL ] RCS: Trimmed 19302 records to 12476 (keeping 200 days of data) [2025-10-20T20:48:20.989740] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-09 (2.34 seconds). [2025-10-20T20:48:21.027142] [DEBUG] [LOCAL ] RCS: Primary data saved (12476 records). [2025-10-20T20:48:21.027311] [INFO ] [MAIN ] RCS: Full history saved. [2025-10-20T20:48:21.027435] [INFO ] [MAIN ] Update Progress: 1774/2367 symbols completed. RCS took 26.5s. Estimated remaining: 1h 16m 18s. Rate: 7.8 symbols/min [2025-10-20T20:48:21.060861] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-02. [2025-10-20T20:48:21.214739] [INFO ] [MAIN ] RDN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:21.254551] [DEBUG] [LOCAL ] RDN: Loaded 45209 existing records from primary storage. [2025-10-20T20:48:21.254873] [INFO ] [MAIN ] RDN: Loaded 45209 existing records. [2025-10-20T20:48:21.445248] [WARN ] [MAIN ] RDN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:21.445598] [INFO ] [MAIN ] RDN: Wiping existing primary data before full refresh. [2025-10-20T20:48:21.445728] [INFO ] [LOCAL ] RDN: Starting deletion of primary data. [2025-10-20T20:48:21.446092] [DEBUG] [LOCAL ] RDN: Deleted primary data file. [2025-10-20T20:48:21.446272] [INFO ] [MAIN ] RDN: Starting full primary refresh. [2025-10-20T20:48:21.736508] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T20:48:21.850286] [INFO ] [REMOTE ] RDN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:21.851338] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-01. [2025-10-20T20:48:22.651372] [INFO ] [REMOTE ] RCL: Fetching primary data for 2025-10. [2025-10-20T20:48:23.411466] [INFO ] [REMOTE ] RCL: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:48:23.418682] [INFO ] [REMOTE ] RCL: Full fetch complete (75092 records). [2025-10-20T20:48:23.418806] [DEBUG] [VALIDATOR ] RCL: Validating primary data. [2025-10-20T20:48:23.419603] [FATAL] [VALIDATOR ] RCL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:48:23.419726] [FATAL] [MAIN ] RCL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:48:23.419796] [WARN ] [MAIN ] RCL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:48:23.419870] [INFO ] [LOCAL ] RCL: Starting deletion of primary data. [2025-10-20T20:48:23.419934] [DEBUG] [LOCAL ] RCL: No primary data file found to delete. [2025-10-20T20:48:23.419971] [INFO ] [LOCAL ] RCL: Starting deletion of derivative data. [2025-10-20T20:48:23.420012] [DEBUG] [LOCAL ] RCL: No derivative data file found to delete. [2025-10-20T20:48:23.420069] [INFO ] [MAIN ] Update Progress: 1775/2367 symbols completed. RCL took 37.5s. Estimated remaining: 1h 16m 9s. Rate: 7.8 symbols/min [2025-10-20T20:48:23.456494] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-07. [2025-10-20T20:48:23.636908] [INFO ] [MAIN ] RDW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:23.639289] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T20:48:23.697996] [DEBUG] [LOCAL ] RDW: Loaded 53025 existing records from primary storage. [2025-10-20T20:48:23.698342] [INFO ] [MAIN ] RDW: Loaded 53025 existing records. [2025-10-20T20:48:23.880729] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-02 (2.82 seconds). [2025-10-20T20:48:23.917676] [WARN ] [MAIN ] RDW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:23.918209] [INFO ] [MAIN ] RDW: Wiping existing primary data before full refresh. [2025-10-20T20:48:23.918256] [INFO ] [LOCAL ] RDW: Starting deletion of primary data. [2025-10-20T20:48:23.918535] [DEBUG] [LOCAL ] RDW: Deleted primary data file. [2025-10-20T20:48:23.918574] [INFO ] [MAIN ] RDW: Starting full primary refresh. [2025-10-20T20:48:24.251681] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-02. [2025-10-20T20:48:24.275257] [INFO ] [REMOTE ] RDW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:25.018800] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T20:48:25.051345] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-08. [2025-10-20T20:48:25.851419] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-01. [2025-10-20T20:48:25.956278] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:48:26.381014] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T20:48:26.651378] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-03. [2025-10-20T20:48:27.451348] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-03. [2025-10-20T20:48:27.922471] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T20:48:28.251413] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-09. [2025-10-20T20:48:28.560490] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T20:48:29.051387] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-04. [2025-10-20T20:48:29.782423] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T20:48:29.851346] [INFO ] [REMOTE ] RCUS: Fetching primary data for 2025-10. [2025-10-20T20:48:30.018473] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-03 (2.57 seconds). [2025-10-20T20:48:30.268080] [INFO ] [REMOTE ] RCUS: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T20:48:30.274584] [INFO ] [REMOTE ] RCUS: Full fetch complete (60700 records). [2025-10-20T20:48:30.274727] [DEBUG] [VALIDATOR ] RCUS: Validating primary data. [2025-10-20T20:48:30.275385] [FATAL] [VALIDATOR ] RCUS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:48:30.275470] [FATAL] [MAIN ] RCUS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:48:30.275513] [WARN ] [MAIN ] RCUS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:48:30.275579] [INFO ] [LOCAL ] RCUS: Starting deletion of primary data. [2025-10-20T20:48:30.275666] [DEBUG] [LOCAL ] RCUS: No primary data file found to delete. [2025-10-20T20:48:30.275724] [INFO ] [LOCAL ] RCUS: Starting deletion of derivative data. [2025-10-20T20:48:30.275846] [DEBUG] [LOCAL ] RCUS: Deleted derivative data file. [2025-10-20T20:48:30.275916] [INFO ] [MAIN ] Update Progress: 1776/2367 symbols completed. RCUS took 29.2s. Estimated remaining: 1h 16m 1s. Rate: 7.8 symbols/min [2025-10-20T20:48:30.448475] [INFO ] [MAIN ] RDY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:30.492115] [DEBUG] [LOCAL ] RDY: Loaded 47210 existing records from primary storage. [2025-10-20T20:48:30.492396] [INFO ] [MAIN ] RDY: Loaded 47210 existing records. [2025-10-20T20:48:30.662552] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-04. [2025-10-20T20:48:30.667834] [WARN ] [MAIN ] RDY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:30.668252] [INFO ] [MAIN ] RDY: Wiping existing primary data before full refresh. [2025-10-20T20:48:30.669732] [INFO ] [LOCAL ] RDY: Starting deletion of primary data. [2025-10-20T20:48:30.670182] [DEBUG] [LOCAL ] RDY: Deleted primary data file. [2025-10-20T20:48:30.670237] [INFO ] [MAIN ] RDY: Starting full primary refresh. [2025-10-20T20:48:31.021672] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T20:48:31.078532] [INFO ] [REMOTE ] RDY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:31.451437] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-01. [2025-10-20T20:48:32.251442] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-05. [2025-10-20T20:48:33.051379] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-02. [2025-10-20T20:48:33.234695] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T20:48:33.273258] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-04 (2.61 seconds). [2025-10-20T20:48:33.485868] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T20:48:33.851407] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-06. [2025-10-20T20:48:34.651435] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-02. [2025-10-20T20:48:34.885424] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T20:48:35.448276] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T20:48:35.456809] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-05. [2025-10-20T20:48:36.251344] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-07. [2025-10-20T20:48:36.353076] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T20:48:37.051421] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-03. [2025-10-20T20:48:37.851399] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-03. [2025-10-20T20:48:37.962134] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T20:48:38.179700] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-05 (2.72 seconds). [2025-10-20T20:48:38.651361] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-08. [2025-10-20T20:48:39.000561] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T20:48:39.451375] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-04. [2025-10-20T20:48:39.680384] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T20:48:40.251352] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-04. [2025-10-20T20:48:40.329624] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T20:48:41.051359] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-09. [2025-10-20T20:48:41.423226] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T20:48:41.851347] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-05. [2025-10-20T20:48:42.257283] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-04 (2.01 seconds). [2025-10-20T20:48:42.651402] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-05. [2025-10-20T20:48:42.722090] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T20:48:43.451411] [INFO ] [REMOTE ] RDN: Fetching primary data for 2025-10. [2025-10-20T20:48:43.895082] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T20:48:44.077279] [INFO ] [REMOTE ] RDN: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:48:44.083113] [INFO ] [REMOTE ] RDN: Full fetch complete (65793 records). [2025-10-20T20:48:44.083213] [DEBUG] [VALIDATOR ] RDN: Validating primary data. [2025-10-20T20:48:44.083822] [FATAL] [VALIDATOR ] RDN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:48:44.083903] [FATAL] [MAIN ] RDN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:48:44.083970] [WARN ] [MAIN ] RDN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:48:44.084039] [INFO ] [LOCAL ] RDN: Starting deletion of primary data. [2025-10-20T20:48:44.084101] [DEBUG] [LOCAL ] RDN: No primary data file found to delete. [2025-10-20T20:48:44.084135] [INFO ] [LOCAL ] RDN: Starting deletion of derivative data. [2025-10-20T20:48:44.084276] [DEBUG] [LOCAL ] RDN: Deleted derivative data file. [2025-10-20T20:48:44.084386] [INFO ] [MAIN ] Update Progress: 1777/2367 symbols completed. RDN took 23.1s. Estimated remaining: 1h 15m 55s. Rate: 7.8 symbols/min [2025-10-20T20:48:44.253320] [INFO ] [MAIN ] RELX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:44.251905] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-06. [2025-10-20T20:48:44.297733] [DEBUG] [LOCAL ] RELX: Loaded 49179 existing records from primary storage. [2025-10-20T20:48:44.298310] [INFO ] [MAIN ] RELX: Loaded 49179 existing records. [2025-10-20T20:48:44.476482] [WARN ] [MAIN ] RELX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:44.476765] [INFO ] [MAIN ] RELX: Wiping existing primary data before full refresh. [2025-10-20T20:48:44.476826] [INFO ] [LOCAL ] RELX: Starting deletion of primary data. [2025-10-20T20:48:44.477172] [DEBUG] [LOCAL ] RELX: Deleted primary data file. [2025-10-20T20:48:44.477229] [INFO ] [MAIN ] RELX: Starting full primary refresh. [2025-10-20T20:48:44.603111] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T20:48:44.860971] [INFO ] [REMOTE ] RELX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:45.051328] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-01. [2025-10-20T20:48:45.851396] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-06. [2025-10-20T20:48:46.321142] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T20:48:46.350901] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-06 (2.10 seconds). [2025-10-20T20:48:46.651434] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-02. [2025-10-20T20:48:47.451462] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-06. [2025-10-20T20:48:47.838134] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T20:48:48.251396] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-07. [2025-10-20T20:48:48.530795] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-06 (2.68 seconds). [2025-10-20T20:48:48.722705] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-06 (1.27 seconds). [2025-10-20T20:48:49.051422] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-07. [2025-10-20T20:48:49.851363] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-07. [2025-10-20T20:48:50.597085] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T20:48:50.651340] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-08. [2025-10-20T20:48:51.459426] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-03. [2025-10-20T20:48:51.477855] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T20:48:51.756049] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-07 (2.70 seconds). [2025-10-20T20:48:52.251364] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-08. [2025-10-20T20:48:53.028067] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T20:48:53.051339] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-09. [2025-10-20T20:48:53.379572] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T20:48:53.662992] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T20:48:53.851426] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-04. [2025-10-20T20:48:54.651369] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-09. [2025-10-20T20:48:55.446135] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-09 (2.39 seconds). [2025-10-20T20:48:55.456726] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-08. [2025-10-20T20:48:55.546566] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:48:56.251352] [INFO ] [REMOTE ] RDW: Fetching primary data for 2025-10. [2025-10-20T20:48:56.331907] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T20:48:56.981590] [INFO ] [REMOTE ] RDW: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T20:48:56.988996] [INFO ] [REMOTE ] RDW: Full fetch complete (75081 records). [2025-10-20T20:48:56.989116] [DEBUG] [VALIDATOR ] RDW: Validating primary data. [2025-10-20T20:48:56.989910] [FATAL] [VALIDATOR ] RDW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:48:56.989994] [FATAL] [MAIN ] RDW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:48:56.990036] [WARN ] [MAIN ] RDW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:48:56.990101] [INFO ] [LOCAL ] RDW: Starting deletion of primary data. [2025-10-20T20:48:56.990163] [DEBUG] [LOCAL ] RDW: No primary data file found to delete. [2025-10-20T20:48:56.990198] [INFO ] [LOCAL ] RDW: Starting deletion of derivative data. [2025-10-20T20:48:56.990321] [DEBUG] [LOCAL ] RDW: Deleted derivative data file. [2025-10-20T20:48:56.990685] [INFO ] [MAIN ] Update Progress: 1778/2367 symbols completed. RDW took 33.6s. Estimated remaining: 1h 15m 49s. Rate: 7.8 symbols/min [2025-10-20T20:48:57.056462] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-05. [2025-10-20T20:48:57.160025] [INFO ] [MAIN ] RERE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:48:57.198132] [DEBUG] [LOCAL ] RERE: Loaded 42117 existing records from primary storage. [2025-10-20T20:48:57.198430] [INFO ] [MAIN ] RERE: Loaded 42117 existing records. [2025-10-20T20:48:57.373665] [WARN ] [MAIN ] RERE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:48:57.373887] [INFO ] [MAIN ] RERE: Wiping existing primary data before full refresh. [2025-10-20T20:48:57.373940] [INFO ] [LOCAL ] RERE: Starting deletion of primary data. [2025-10-20T20:48:57.374135] [DEBUG] [LOCAL ] RERE: Deleted primary data file. [2025-10-20T20:48:57.374189] [INFO ] [MAIN ] RERE: Starting full primary refresh. [2025-10-20T20:48:57.723079] [INFO ] [REMOTE ] RERE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:48:57.851349] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-01. [2025-10-20T20:48:58.115542] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-08 (2.66 seconds). [2025-10-20T20:48:58.651349] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-09. [2025-10-20T20:48:58.884334] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T20:48:58.917070] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T20:48:59.451386] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-06. [2025-10-20T20:49:00.251401] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-02. [2025-10-20T20:49:01.056448] [INFO ] [REMOTE ] RDY: Fetching primary data for 2025-10. [2025-10-20T20:49:01.069367] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T20:49:01.156394] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-09 (2.51 seconds). [2025-10-20T20:49:01.683529] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:49:01.851364] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-07. [2025-10-20T20:49:02.656554] [INFO ] [REMOTE ] RDDT: Fetching primary data for 2025-10. [2025-10-20T20:49:02.670826] [INFO ] [REMOTE ] RDY: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T20:49:02.677987] [INFO ] [REMOTE ] RDY: Full fetch complete (68716 records). [2025-10-20T20:49:02.678113] [DEBUG] [VALIDATOR ] RDY: Validating primary data. [2025-10-20T20:49:03.059678] [DEBUG] [VALIDATOR ] RDY: All expected trading days are present. [2025-10-20T20:49:03.059795] [DEBUG] [VALIDATOR ] RDY: Primary data validation passed. [2025-10-20T20:49:03.060963] [INFO ] [MAIN ] RDY: Fetched primary data is valid. [2025-10-20T20:49:03.061029] [DEBUG] [LOCAL ] RDY: Saving primary data. [2025-10-20T20:49:03.062547] [DEBUG] [LOCAL ] RDY: Trimmed 68716 records to 46400 (keeping 200 days of data) [2025-10-20T20:49:03.342369] [DEBUG] [LOCAL ] RDY: Primary data saved (46400 records). [2025-10-20T20:49:03.342610] [INFO ] [MAIN ] RDY: Full history saved. [2025-10-20T20:49:03.342788] [INFO ] [MAIN ] Update Progress: 1779/2367 symbols completed. RDY took 33.1s. Estimated remaining: 1h 15m 41s. Rate: 7.8 symbols/min [2025-10-20T20:49:03.455098] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-03. [2025-10-20T20:49:03.541277] [INFO ] [MAIN ] RES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:03.544611] [INFO ] [REMOTE ] RDDT: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T20:49:03.553602] [INFO ] [REMOTE ] RDDT: Full fetch complete (77540 records). [2025-10-20T20:49:03.553869] [DEBUG] [VALIDATOR ] RDDT: Validating primary data. [2025-10-20T20:49:03.555387] [FATAL] [VALIDATOR ] RDDT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:49:03.555506] [FATAL] [MAIN ] RDDT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:49:03.555559] [WARN ] [MAIN ] RDDT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:49:03.555597] [INFO ] [LOCAL ] RDDT: Starting deletion of primary data. [2025-10-20T20:49:03.555768] [DEBUG] [LOCAL ] RDDT: No primary data file found to delete. [2025-10-20T20:49:03.555815] [INFO ] [LOCAL ] RDDT: Starting deletion of derivative data. [2025-10-20T20:49:03.555959] [DEBUG] [LOCAL ] RDDT: Deleted derivative data file. [2025-10-20T20:49:03.556054] [INFO ] [MAIN ] Update Progress: 1780/2367 symbols completed. RDDT took 50.5s. Estimated remaining: 1h 15m 31s. Rate: 7.8 symbols/min [2025-10-20T20:49:03.744616] [INFO ] [MAIN ] REVG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:03.789093] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T20:49:03.787329] [DEBUG] [LOCAL ] RES: Loaded 47647 existing records from primary storage. [2025-10-20T20:49:03.799849] [INFO ] [MAIN ] RES: Loaded 47647 existing records. [2025-10-20T20:49:04.005369] [WARN ] [MAIN ] RES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:04.006559] [DEBUG] [LOCAL ] REVG: Loaded 40973 existing records from primary storage. [2025-10-20T20:49:04.006870] [INFO ] [MAIN ] RES: Wiping existing primary data before full refresh. [2025-10-20T20:49:04.007226] [INFO ] [MAIN ] REVG: Loaded 40973 existing records. [2025-10-20T20:49:04.007079] [INFO ] [LOCAL ] RES: Starting deletion of primary data. [2025-10-20T20:49:04.054405] [DEBUG] [LOCAL ] RES: Deleted primary data file. [2025-10-20T20:49:04.068852] [INFO ] [MAIN ] RES: Starting full primary refresh. [2025-10-20T20:49:04.312452] [WARN ] [MAIN ] REVG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:04.256307] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-08. [2025-10-20T20:49:04.313471] [INFO ] [MAIN ] REVG: Wiping existing primary data before full refresh. [2025-10-20T20:49:04.318701] [INFO ] [LOCAL ] REVG: Starting deletion of primary data. [2025-10-20T20:49:04.343237] [DEBUG] [LOCAL ] REVG: Deleted primary data file. [2025-10-20T20:49:04.419544] [INFO ] [MAIN ] REVG: Starting full primary refresh. [2025-10-20T20:49:04.788659] [INFO ] [REMOTE ] RES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:04.955472] [INFO ] [REMOTE ] REVG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:04.976699] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T20:49:05.051329] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-04. [2025-10-20T20:49:05.851370] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-01. [2025-10-20T20:49:06.291930] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-08 (2.04 seconds). [2025-10-20T20:49:06.366578] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T20:49:06.651345] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-05. [2025-10-20T20:49:07.375687] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T20:49:07.451363] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-01. [2025-10-20T20:49:08.059976] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T20:49:08.251356] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-06. [2025-10-20T20:49:08.678682] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T20:49:09.051372] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-09. [2025-10-20T20:49:09.232992] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T20:49:09.851368] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-02. [2025-10-20T20:49:10.651397] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-07. [2025-10-20T20:49:10.964524] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T20:49:11.402908] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T20:49:11.451326] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-03. [2025-10-20T20:49:12.251395] [INFO ] [REMOTE ] RELX: Fetching primary data for 2025-10. [2025-10-20T20:49:12.462162] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T20:49:13.051383] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-08. [2025-10-20T20:49:13.272980] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T20:49:13.701275] [INFO ] [REMOTE ] RELX: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T20:49:13.712363] [INFO ] [REMOTE ] RELX: Full fetch complete (68409 records). [2025-10-20T20:49:13.712610] [DEBUG] [VALIDATOR ] RELX: Validating primary data. [2025-10-20T20:49:13.856548] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-02. [2025-10-20T20:49:14.111551] [DEBUG] [VALIDATOR ] RELX: All expected trading days are present. [2025-10-20T20:49:14.111629] [DEBUG] [VALIDATOR ] RELX: Primary data validation passed. [2025-10-20T20:49:14.112635] [INFO ] [MAIN ] RELX: Fetched primary data is valid. [2025-10-20T20:49:14.112857] [DEBUG] [LOCAL ] RELX: Saving primary data. [2025-10-20T20:49:14.114132] [DEBUG] [LOCAL ] RELX: Trimmed 68409 records to 48572 (keeping 200 days of data) [2025-10-20T20:49:14.399105] [DEBUG] [LOCAL ] RELX: Primary data saved (48572 records). [2025-10-20T20:49:14.399232] [INFO ] [MAIN ] RELX: Full history saved. [2025-10-20T20:49:14.399309] [INFO ] [MAIN ] Update Progress: 1781/2367 symbols completed. RELX took 30.3s. Estimated remaining: 1h 15m 24s. Rate: 7.8 symbols/min [2025-10-20T20:49:14.568752] [INFO ] [MAIN ] REX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:14.584377] [DEBUG] [LOCAL ] REX: Loaded 14756 existing records from primary storage. [2025-10-20T20:49:14.584608] [INFO ] [MAIN ] REX: Loaded 14756 existing records. [2025-10-20T20:49:14.656441] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-04. [2025-10-20T20:49:14.772783] [WARN ] [MAIN ] REX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:14.773045] [INFO ] [MAIN ] REX: Wiping existing primary data before full refresh. [2025-10-20T20:49:14.794687] [INFO ] [LOCAL ] REX: Starting deletion of primary data. [2025-10-20T20:49:14.798713] [DEBUG] [LOCAL ] REX: Deleted primary data file. [2025-10-20T20:49:14.799148] [INFO ] [MAIN ] REX: Starting full primary refresh. [2025-10-20T20:49:14.800399] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T20:49:15.195607] [INFO ] [REMOTE ] REX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:15.385684] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T20:49:15.451362] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-03. [2025-10-20T20:49:16.251354] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-09. [2025-10-20T20:49:16.494017] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T20:49:16.845224] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T20:49:17.051353] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-04. [2025-10-20T20:49:17.851423] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-01. [2025-10-20T20:49:18.206790] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T20:49:18.651352] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-05. [2025-10-20T20:49:18.750374] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T20:49:18.798953] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T20:49:19.451369] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-05. [2025-10-20T20:49:20.251400] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-02. [2025-10-20T20:49:20.290506] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T20:49:21.059300] [INFO ] [REMOTE ] RERE: Fetching primary data for 2025-10. [2025-10-20T20:49:21.064566] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T20:49:21.172526] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:49:21.851384] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-06. [2025-10-20T20:49:22.607854] [INFO ] [REMOTE ] RERE: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T20:49:22.614039] [INFO ] [REMOTE ] RERE: Full fetch complete (57926 records). [2025-10-20T20:49:22.614150] [DEBUG] [VALIDATOR ] RERE: Validating primary data. [2025-10-20T20:49:22.656934] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-03. [2025-10-20T20:49:22.993113] [DEBUG] [VALIDATOR ] RERE: All expected trading days are present. [2025-10-20T20:49:22.993185] [DEBUG] [VALIDATOR ] RERE: Primary data validation passed. [2025-10-20T20:49:22.994078] [INFO ] [MAIN ] RERE: Fetched primary data is valid. [2025-10-20T20:49:22.994138] [DEBUG] [LOCAL ] RERE: Saving primary data. [2025-10-20T20:49:22.995201] [DEBUG] [LOCAL ] RERE: Trimmed 57926 records to 41702 (keeping 200 days of data) [2025-10-20T20:49:23.225275] [DEBUG] [LOCAL ] RERE: Primary data saved (41702 records). [2025-10-20T20:49:23.225397] [INFO ] [MAIN ] RERE: Full history saved. [2025-10-20T20:49:23.225775] [INFO ] [MAIN ] Update Progress: 1782/2367 symbols completed. RERE took 26.2s. Estimated remaining: 1h 15m 17s. Rate: 7.8 symbols/min [2025-10-20T20:49:23.403981] [INFO ] [MAIN ] REXR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:23.453381] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-06. [2025-10-20T20:49:23.455371] [DEBUG] [LOCAL ] REXR: Loaded 52912 existing records from primary storage. [2025-10-20T20:49:23.455832] [INFO ] [MAIN ] REXR: Loaded 52912 existing records. [2025-10-20T20:49:23.666836] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:49:23.665445] [WARN ] [MAIN ] REXR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:23.676438] [INFO ] [MAIN ] REXR: Wiping existing primary data before full refresh. [2025-10-20T20:49:23.676621] [INFO ] [LOCAL ] REXR: Starting deletion of primary data. [2025-10-20T20:49:23.677281] [DEBUG] [LOCAL ] REXR: Deleted primary data file. [2025-10-20T20:49:23.677355] [INFO ] [MAIN ] REXR: Starting full primary refresh. [2025-10-20T20:49:23.787684] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T20:49:24.037500] [INFO ] [REMOTE ] REXR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:24.251337] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-01. [2025-10-20T20:49:25.051413] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-07. [2025-10-20T20:49:25.083927] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T20:49:25.713008] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T20:49:25.851363] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-02. [2025-10-20T20:49:26.651344] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-04. [2025-10-20T20:49:26.984575] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T20:49:27.451415] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-07. [2025-10-20T20:49:27.657095] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T20:49:27.719423] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T20:49:28.251395] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-05. [2025-10-20T20:49:29.020969] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T20:49:29.051398] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-08. [2025-10-20T20:49:29.262992] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T20:49:29.851423] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-08. [2025-10-20T20:49:30.651406] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-06. [2025-10-20T20:49:30.675514] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-08 (1.62 seconds). [2025-10-20T20:49:31.451459] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-03. [2025-10-20T20:49:31.517119] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T20:49:31.782824] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:49:32.251379] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-07. [2025-10-20T20:49:33.051346] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-09. [2025-10-20T20:49:33.101377] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T20:49:33.144049] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T20:49:33.851354] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-04. [2025-10-20T20:49:34.656635] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-08. [2025-10-20T20:49:34.675373] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T20:49:35.306555] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T20:49:35.451407] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-09. [2025-10-20T20:49:35.551341] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T20:49:36.251415] [INFO ] [REMOTE ] RES: Fetching primary data for 2025-10. [2025-10-20T20:49:36.824440] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T20:49:36.854928] [INFO ] [REMOTE ] RES: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T20:49:36.860961] [INFO ] [REMOTE ] RES: Full fetch complete (68482 records). [2025-10-20T20:49:36.861055] [DEBUG] [VALIDATOR ] RES: Validating primary data. [2025-10-20T20:49:36.861673] [FATAL] [VALIDATOR ] RES: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:49:36.861757] [FATAL] [MAIN ] RES: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:49:36.861802] [WARN ] [MAIN ] RES: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:49:36.861869] [INFO ] [LOCAL ] RES: Starting deletion of primary data. [2025-10-20T20:49:36.861934] [DEBUG] [LOCAL ] RES: No primary data file found to delete. [2025-10-20T20:49:36.861988] [INFO ] [LOCAL ] RES: Starting deletion of derivative data. [2025-10-20T20:49:36.862145] [DEBUG] [LOCAL ] RES: Deleted derivative data file. [2025-10-20T20:49:36.862220] [INFO ] [MAIN ] Update Progress: 1783/2367 symbols completed. RES took 33.5s. Estimated remaining: 1h 15m 11s. Rate: 7.8 symbols/min [2025-10-20T20:49:37.035799] [INFO ] [MAIN ] REZI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:37.058340] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-05. [2025-10-20T20:49:37.079067] [DEBUG] [LOCAL ] REZI: Loaded 47764 existing records from primary storage. [2025-10-20T20:49:37.079340] [INFO ] [MAIN ] REZI: Loaded 47764 existing records. [2025-10-20T20:49:37.253860] [WARN ] [MAIN ] REZI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:37.254073] [INFO ] [MAIN ] REZI: Wiping existing primary data before full refresh. [2025-10-20T20:49:37.254119] [INFO ] [LOCAL ] REZI: Starting deletion of primary data. [2025-10-20T20:49:37.254324] [DEBUG] [LOCAL ] REZI: Deleted primary data file. [2025-10-20T20:49:37.254378] [INFO ] [MAIN ] REZI: Starting full primary refresh. [2025-10-20T20:49:37.603831] [INFO ] [REMOTE ] REZI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:37.851400] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-01. [2025-10-20T20:49:38.651409] [INFO ] [REMOTE ] REVG: Fetching primary data for 2025-10. [2025-10-20T20:49:39.199250] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:49:39.451429] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-02. [2025-10-20T20:49:40.151314] [INFO ] [REMOTE ] REVG: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T20:49:40.156762] [INFO ] [REMOTE ] REVG: Full fetch complete (57891 records). [2025-10-20T20:49:40.156858] [DEBUG] [VALIDATOR ] REVG: Validating primary data. [2025-10-20T20:49:40.253730] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-09. [2025-10-20T20:49:40.551486] [DEBUG] [VALIDATOR ] REVG: All expected trading days are present. [2025-10-20T20:49:40.551565] [DEBUG] [VALIDATOR ] REVG: Primary data validation passed. [2025-10-20T20:49:40.552464] [INFO ] [MAIN ] REVG: Fetched primary data is valid. [2025-10-20T20:49:40.552525] [DEBUG] [LOCAL ] REVG: Saving primary data. [2025-10-20T20:49:40.553632] [DEBUG] [LOCAL ] REVG: Trimmed 57891 records to 40371 (keeping 200 days of data) [2025-10-20T20:49:40.685620] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-05 (3.63 seconds). [2025-10-20T20:49:40.833554] [DEBUG] [LOCAL ] REVG: Primary data saved (40371 records). [2025-10-20T20:49:40.833718] [INFO ] [MAIN ] REVG: Full history saved. [2025-10-20T20:49:40.833810] [INFO ] [MAIN ] Update Progress: 1784/2367 symbols completed. REVG took 37.3s. Estimated remaining: 1h 15m 2s. Rate: 7.8 symbols/min [2025-10-20T20:49:41.004711] [INFO ] [MAIN ] RF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:41.025486] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T20:49:41.057735] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-03. [2025-10-20T20:49:41.082672] [DEBUG] [LOCAL ] RF: Loaded 54412 existing records from primary storage. [2025-10-20T20:49:41.083012] [INFO ] [MAIN ] RF: Loaded 54412 existing records. [2025-10-20T20:49:41.264606] [WARN ] [MAIN ] RF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:41.264891] [INFO ] [MAIN ] RF: Wiping existing primary data before full refresh. [2025-10-20T20:49:41.264950] [INFO ] [LOCAL ] RF: Starting deletion of primary data. [2025-10-20T20:49:41.265129] [DEBUG] [LOCAL ] RF: Deleted primary data file. [2025-10-20T20:49:41.265167] [INFO ] [MAIN ] RF: Starting full primary refresh. [2025-10-20T20:49:41.388883] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T20:49:41.628424] [INFO ] [REMOTE ] RF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:41.851340] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-01. [2025-10-20T20:49:42.651362] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-06. [2025-10-20T20:49:42.741196] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T20:49:43.451343] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-04. [2025-10-20T20:49:43.634798] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T20:49:44.251402] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-02. [2025-10-20T20:49:44.987067] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T20:49:45.051332] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-05. [2025-10-20T20:49:45.537684] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-06 (2.89 seconds). [2025-10-20T20:49:45.698502] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T20:49:45.851370] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-07. [2025-10-20T20:49:46.651454] [INFO ] [REMOTE ] REX: Fetching primary data for 2025-10. [2025-10-20T20:49:46.726117] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T20:49:46.930451] [INFO ] [REMOTE ] REX: Incremental fetch complete for 2025-10 (0.28 seconds). [2025-10-20T20:49:46.933564] [INFO ] [REMOTE ] REX: Full fetch complete (21069 records). [2025-10-20T20:49:46.933634] [DEBUG] [VALIDATOR ] REX: Validating primary data. [2025-10-20T20:49:46.934106] [FATAL] [VALIDATOR ] REX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:49:46.934193] [FATAL] [MAIN ] REX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:49:46.934237] [WARN ] [MAIN ] REX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:49:46.934276] [INFO ] [LOCAL ] REX: Starting deletion of primary data. [2025-10-20T20:49:46.934334] [DEBUG] [LOCAL ] REX: No primary data file found to delete. [2025-10-20T20:49:46.934369] [INFO ] [LOCAL ] REX: Starting deletion of derivative data. [2025-10-20T20:49:46.934481] [DEBUG] [LOCAL ] REX: Deleted derivative data file. [2025-10-20T20:49:46.934546] [INFO ] [MAIN ] Update Progress: 1785/2367 symbols completed. REX took 32.5s. Estimated remaining: 1h 14m 54s. Rate: 7.8 symbols/min [2025-10-20T20:49:47.106441] [INFO ] [MAIN ] RFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:49:47.119141] [DEBUG] [LOCAL ] RFI: Loaded 10522 existing records from primary storage. [2025-10-20T20:49:47.119358] [INFO ] [MAIN ] RFI: Loaded 10522 existing records. [2025-10-20T20:49:47.302722] [WARN ] [MAIN ] RFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:49:47.302994] [INFO ] [MAIN ] RFI: Wiping existing primary data before full refresh. [2025-10-20T20:49:47.303051] [INFO ] [LOCAL ] RFI: Starting deletion of primary data. [2025-10-20T20:49:47.303290] [DEBUG] [LOCAL ] RFI: Deleted primary data file. [2025-10-20T20:49:47.303346] [INFO ] [MAIN ] RFI: Starting full primary refresh. [2025-10-20T20:49:47.458057] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-03. [2025-10-20T20:49:47.664347] [INFO ] [REMOTE ] RFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:49:47.852788] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T20:49:48.251388] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-01. [2025-10-20T20:49:49.051348] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-06. [2025-10-20T20:49:49.278456] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:49:49.510893] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T20:49:49.851373] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-02. [2025-10-20T20:49:50.430232] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T20:49:50.651353] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-04. [2025-10-20T20:49:50.678302] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T20:49:51.451402] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-03. [2025-10-20T20:49:52.198894] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T20:49:52.251347] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-04. [2025-10-20T20:49:52.678089] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T20:49:53.029908] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T20:49:53.051353] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-05. [2025-10-20T20:49:53.851344] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-08. [2025-10-20T20:49:53.928390] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T20:49:54.651382] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-07. [2025-10-20T20:49:55.451430] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-06. [2025-10-20T20:49:55.577663] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T20:49:56.153932] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T20:49:56.253881] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-09. [2025-10-20T20:49:56.257081] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T20:49:57.051434] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-08. [2025-10-20T20:49:57.851361] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-07. [2025-10-20T20:49:58.053444] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T20:49:58.649396] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:49:58.659277] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-05. [2025-10-20T20:49:58.794808] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T20:49:59.451410] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-08. [2025-10-20T20:50:00.251366] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-09. [2025-10-20T20:50:00.565425] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T20:50:00.594602] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T20:50:01.051369] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-09. [2025-10-20T20:50:01.851405] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-06. [2025-10-20T20:50:01.923400] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:50:02.061937] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T20:50:02.651411] [INFO ] [REMOTE ] REZI: Fetching primary data for 2025-10. [2025-10-20T20:50:03.410555] [INFO ] [REMOTE ] REZI: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:50:03.417044] [INFO ] [REMOTE ] REZI: Full fetch complete (65836 records). [2025-10-20T20:50:03.417155] [DEBUG] [VALIDATOR ] REZI: Validating primary data. [2025-10-20T20:50:03.417830] [FATAL] [VALIDATOR ] REZI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:50:03.417931] [FATAL] [MAIN ] REZI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:50:03.418005] [WARN ] [MAIN ] REZI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:50:03.418077] [INFO ] [LOCAL ] REZI: Starting deletion of primary data. [2025-10-20T20:50:03.418141] [DEBUG] [LOCAL ] REZI: No primary data file found to delete. [2025-10-20T20:50:03.418176] [INFO ] [LOCAL ] REZI: Starting deletion of derivative data. [2025-10-20T20:50:03.418420] [DEBUG] [LOCAL ] REZI: Deleted derivative data file. [2025-10-20T20:50:03.418512] [INFO ] [MAIN ] Update Progress: 1786/2367 symbols completed. REZI took 26.6s. Estimated remaining: 1h 14m 49s. Rate: 7.8 symbols/min [2025-10-20T20:50:03.456460] [INFO ] [REMOTE ] REXR: Fetching primary data for 2025-10. [2025-10-20T20:50:03.595370] [INFO ] [MAIN ] RFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:03.611748] [DEBUG] [LOCAL ] RFL: Loaded 11542 existing records from primary storage. [2025-10-20T20:50:03.611971] [INFO ] [MAIN ] RFL: Loaded 11542 existing records. [2025-10-20T20:50:03.788316] [WARN ] [MAIN ] RFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:03.788793] [INFO ] [MAIN ] RFL: Wiping existing primary data before full refresh. [2025-10-20T20:50:03.788919] [INFO ] [LOCAL ] RFL: Starting deletion of primary data. [2025-10-20T20:50:03.789120] [DEBUG] [LOCAL ] RFL: Deleted primary data file. [2025-10-20T20:50:03.789217] [INFO ] [MAIN ] RFL: Starting full primary refresh. [2025-10-20T20:50:03.986359] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T20:50:04.233334] [INFO ] [REMOTE ] RFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:04.235442] [INFO ] [REMOTE ] REXR: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:50:04.243939] [INFO ] [REMOTE ] REXR: Full fetch complete (74948 records). [2025-10-20T20:50:04.244140] [DEBUG] [VALIDATOR ] REXR: Validating primary data. [2025-10-20T20:50:04.245265] [FATAL] [VALIDATOR ] REXR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:50:04.245400] [FATAL] [MAIN ] REXR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:50:04.245484] [WARN ] [MAIN ] REXR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:50:04.245579] [INFO ] [LOCAL ] REXR: Starting deletion of primary data. [2025-10-20T20:50:04.245733] [DEBUG] [LOCAL ] REXR: No primary data file found to delete. [2025-10-20T20:50:04.245807] [INFO ] [LOCAL ] REXR: Starting deletion of derivative data. [2025-10-20T20:50:04.246017] [DEBUG] [LOCAL ] REXR: Deleted derivative data file. [2025-10-20T20:50:04.246440] [INFO ] [MAIN ] Update Progress: 1787/2367 symbols completed. REXR took 41.0s. Estimated remaining: 1h 14m 39s. Rate: 7.8 symbols/min [2025-10-20T20:50:04.251774] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-01. [2025-10-20T20:50:04.415691] [INFO ] [MAIN ] RFM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:04.423679] [DEBUG] [LOCAL ] RFM: Loaded 4229 existing records from primary storage. [2025-10-20T20:50:04.423896] [INFO ] [MAIN ] RFM: Loaded 4229 existing records. [2025-10-20T20:50:04.598253] [WARN ] [MAIN ] RFM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:04.598445] [INFO ] [MAIN ] RFM: Wiping existing primary data before full refresh. [2025-10-20T20:50:04.598500] [INFO ] [LOCAL ] RFM: Starting deletion of primary data. [2025-10-20T20:50:04.598680] [DEBUG] [LOCAL ] RFM: Deleted primary data file. [2025-10-20T20:50:04.598735] [INFO ] [MAIN ] RFM: Starting full primary refresh. [2025-10-20T20:50:04.944113] [INFO ] [REMOTE ] RFM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:05.002187] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T20:50:05.051335] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-07. [2025-10-20T20:50:05.851373] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-02. [2025-10-20T20:50:06.298727] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-02 (0.45 seconds). [2025-10-20T20:50:06.651357] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-01. [2025-10-20T20:50:07.036688] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T20:50:07.246007] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T20:50:07.451376] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-08. [2025-10-20T20:50:08.251383] [INFO ] [REMOTE ] RFI: Fetching primary data for 2025-10. [2025-10-20T20:50:08.884122] [INFO ] [REMOTE ] RFI: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:50:08.885970] [INFO ] [REMOTE ] RFI: Full fetch complete (15212 records). [2025-10-20T20:50:08.886044] [DEBUG] [VALIDATOR ] RFI: Validating primary data. [2025-10-20T20:50:09.051986] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-02. [2025-10-20T20:50:09.280620] [DEBUG] [VALIDATOR ] RFI: All expected trading days are present. [2025-10-20T20:50:09.280749] [DEBUG] [VALIDATOR ] RFI: Primary data validation passed. [2025-10-20T20:50:09.281087] [INFO ] [MAIN ] RFI: Fetched primary data is valid. [2025-10-20T20:50:09.281157] [DEBUG] [LOCAL ] RFI: Saving primary data. [2025-10-20T20:50:09.281881] [DEBUG] [LOCAL ] RFI: Trimmed 15212 records to 10384 (keeping 200 days of data) [2025-10-20T20:50:09.347358] [DEBUG] [LOCAL ] RFI: Primary data saved (10384 records). [2025-10-20T20:50:09.347513] [INFO ] [MAIN ] RFI: Full history saved. [2025-10-20T20:50:09.347618] [INFO ] [MAIN ] Update Progress: 1788/2367 symbols completed. RFI took 22.4s. Estimated remaining: 1h 14m 30s. Rate: 7.8 symbols/min [2025-10-20T20:50:09.506573] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-08 (2.06 seconds). [2025-10-20T20:50:09.561888] [INFO ] [MAIN ] RFMZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:09.580393] [DEBUG] [LOCAL ] RFMZ: Loaded 8929 existing records from primary storage. [2025-10-20T20:50:09.580637] [INFO ] [MAIN ] RFMZ: Loaded 8929 existing records. [2025-10-20T20:50:09.697758] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:50:09.778948] [WARN ] [MAIN ] RFMZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:09.779183] [INFO ] [MAIN ] RFMZ: Wiping existing primary data before full refresh. [2025-10-20T20:50:09.779241] [INFO ] [LOCAL ] RFMZ: Starting deletion of primary data. [2025-10-20T20:50:09.779393] [DEBUG] [LOCAL ] RFMZ: Deleted primary data file. [2025-10-20T20:50:09.779430] [INFO ] [MAIN ] RFMZ: Starting full primary refresh. [2025-10-20T20:50:09.856548] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-03. [2025-10-20T20:50:10.133475] [INFO ] [REMOTE ] RFMZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:10.597165] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T20:50:10.651380] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-03. [2025-10-20T20:50:11.298906] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T20:50:11.451379] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-09. [2025-10-20T20:50:12.251386] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-04. [2025-10-20T20:50:12.921015] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-04 (0.67 seconds). [2025-10-20T20:50:13.051362] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-05. [2025-10-20T20:50:13.322577] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T20:50:13.659766] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T20:50:13.851404] [INFO ] [REMOTE ] RF: Fetching primary data for 2025-10. [2025-10-20T20:50:14.473894] [INFO ] [REMOTE ] RF: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T20:50:14.481256] [INFO ] [REMOTE ] RF: Full fetch complete (77406 records). [2025-10-20T20:50:14.481407] [DEBUG] [VALIDATOR ] RF: Validating primary data. [2025-10-20T20:50:14.482231] [FATAL] [VALIDATOR ] RF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:50:14.482319] [FATAL] [MAIN ] RF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:50:14.482362] [WARN ] [MAIN ] RF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:50:14.482399] [INFO ] [LOCAL ] RF: Starting deletion of primary data. [2025-10-20T20:50:14.482454] [DEBUG] [LOCAL ] RF: No primary data file found to delete. [2025-10-20T20:50:14.482490] [INFO ] [LOCAL ] RF: Starting deletion of derivative data. [2025-10-20T20:50:14.482621] [DEBUG] [LOCAL ] RF: Deleted derivative data file. [2025-10-20T20:50:14.482731] [INFO ] [MAIN ] Update Progress: 1789/2367 symbols completed. RF took 33.6s. Estimated remaining: 1h 14m 22s. Rate: 7.8 symbols/min [2025-10-20T20:50:14.655751] [INFO ] [MAIN ] RGA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:14.655851] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-01. [2025-10-20T20:50:14.687011] [DEBUG] [LOCAL ] RGA: Loaded 29820 existing records from primary storage. [2025-10-20T20:50:14.687340] [INFO ] [MAIN ] RGA: Loaded 29820 existing records. [2025-10-20T20:50:14.867344] [WARN ] [MAIN ] RGA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:14.867546] [INFO ] [MAIN ] RGA: Wiping existing primary data before full refresh. [2025-10-20T20:50:14.867592] [INFO ] [LOCAL ] RGA: Starting deletion of primary data. [2025-10-20T20:50:14.867816] [DEBUG] [LOCAL ] RGA: Deleted primary data file. [2025-10-20T20:50:14.867872] [INFO ] [MAIN ] RGA: Starting full primary refresh. [2025-10-20T20:50:15.217384] [INFO ] [REMOTE ] RGA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:15.451393] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-04. [2025-10-20T20:50:15.608858] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T20:50:16.251384] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-06. [2025-10-20T20:50:16.276770] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T20:50:16.883868] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:50:17.051384] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-05. [2025-10-20T20:50:17.851403] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-02. [2025-10-20T20:50:17.897097] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T20:50:18.374021] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T20:50:18.651447] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-06. [2025-10-20T20:50:19.451406] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-01. [2025-10-20T20:50:19.781532] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T20:50:20.251399] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-03. [2025-10-20T20:50:21.005266] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T20:50:21.005992] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T20:50:21.051333] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-02. [2025-10-20T20:50:21.851379] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-04. [2025-10-20T20:50:22.602009] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T20:50:22.638805] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T20:50:22.651692] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-03. [2025-10-20T20:50:23.451387] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-07. [2025-10-20T20:50:23.963494] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T20:50:24.075700] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T20:50:24.251415] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-07. [2025-10-20T20:50:25.051352] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-08. [2025-10-20T20:50:25.205205] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T20:50:25.573018] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T20:50:25.851391] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-08. [2025-10-20T20:50:26.587675] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T20:50:26.651381] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-09. [2025-10-20T20:50:27.451402] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-05. [2025-10-20T20:50:27.673251] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:50:28.035050] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T20:50:28.251380] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-09. [2025-10-20T20:50:28.925285] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T20:50:29.051378] [INFO ] [REMOTE ] RFM: Fetching primary data for 2025-10. [2025-10-20T20:50:29.603723] [INFO ] [REMOTE ] RFM: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:50:29.605303] [INFO ] [REMOTE ] RFM: Full fetch complete (5713 records). [2025-10-20T20:50:29.605415] [DEBUG] [VALIDATOR ] RFM: Validating primary data. [2025-10-20T20:50:29.856847] [INFO ] [REMOTE ] RFL: Fetching primary data for 2025-10. [2025-10-20T20:50:29.973958] [DEBUG] [VALIDATOR ] RFM: All expected trading days are present. [2025-10-20T20:50:29.974033] [DEBUG] [VALIDATOR ] RFM: Primary data validation passed. [2025-10-20T20:50:29.974202] [INFO ] [MAIN ] RFM: Fetched primary data is valid. [2025-10-20T20:50:29.974240] [DEBUG] [LOCAL ] RFM: Saving primary data. [2025-10-20T20:50:29.974600] [DEBUG] [LOCAL ] RFM: Trimmed 5713 records to 4195 (keeping 200 days of data) [2025-10-20T20:50:29.999718] [DEBUG] [LOCAL ] RFM: Primary data saved (4195 records). [2025-10-20T20:50:29.999830] [INFO ] [MAIN ] RFM: Full history saved. [2025-10-20T20:50:29.999904] [INFO ] [MAIN ] Update Progress: 1790/2367 symbols completed. RFM took 25.8s. Estimated remaining: 1h 14m 17s. Rate: 7.8 symbols/min [2025-10-20T20:50:30.171102] [INFO ] [MAIN ] RGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:30.195591] [DEBUG] [LOCAL ] RGR: Loaded 21263 existing records from primary storage. [2025-10-20T20:50:30.195903] [INFO ] [MAIN ] RGR: Loaded 21263 existing records. [2025-10-20T20:50:30.384123] [WARN ] [MAIN ] RGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:30.384350] [INFO ] [MAIN ] RGR: Wiping existing primary data before full refresh. [2025-10-20T20:50:30.384395] [INFO ] [LOCAL ] RGR: Starting deletion of primary data. [2025-10-20T20:50:30.384562] [DEBUG] [LOCAL ] RGR: Deleted primary data file. [2025-10-20T20:50:30.384599] [INFO ] [MAIN ] RGR: Starting full primary refresh. [2025-10-20T20:50:30.653935] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-06. [2025-10-20T20:50:30.762965] [INFO ] [REMOTE ] RGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:30.758971] [INFO ] [REMOTE ] RFL: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T20:50:30.765167] [INFO ] [REMOTE ] RFL: Full fetch complete (13217 records). [2025-10-20T20:50:30.765258] [DEBUG] [VALIDATOR ] RFL: Validating primary data. [2025-10-20T20:50:31.183166] [DEBUG] [VALIDATOR ] RFL: All expected trading days are present. [2025-10-20T20:50:31.183280] [DEBUG] [VALIDATOR ] RFL: Primary data validation passed. [2025-10-20T20:50:31.183684] [INFO ] [MAIN ] RFL: Fetched primary data is valid. [2025-10-20T20:50:31.183759] [DEBUG] [LOCAL ] RFL: Saving primary data. [2025-10-20T20:50:31.184326] [DEBUG] [LOCAL ] RFL: Trimmed 13217 records to 11457 (keeping 200 days of data) [2025-10-20T20:50:31.250345] [DEBUG] [LOCAL ] RFL: Primary data saved (11457 records). [2025-10-20T20:50:31.250489] [INFO ] [MAIN ] RFL: Full history saved. [2025-10-20T20:50:31.250574] [INFO ] [MAIN ] Update Progress: 1791/2367 symbols completed. RFL took 27.8s. Estimated remaining: 1h 14m 7s. Rate: 7.8 symbols/min [2025-10-20T20:50:31.444323] [INFO ] [MAIN ] RGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:31.449660] [DEBUG] [LOCAL ] RGT: Loaded 2793 existing records from primary storage. [2025-10-20T20:50:31.449856] [INFO ] [MAIN ] RGT: Loaded 2793 existing records. [2025-10-20T20:50:31.460843] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-01. [2025-10-20T20:50:31.559123] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T20:50:31.643943] [WARN ] [MAIN ] RGT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:31.644158] [INFO ] [MAIN ] RGT: Wiping existing primary data before full refresh. [2025-10-20T20:50:31.644204] [INFO ] [LOCAL ] RGT: Starting deletion of primary data. [2025-10-20T20:50:31.644333] [DEBUG] [LOCAL ] RGT: Deleted primary data file. [2025-10-20T20:50:31.644369] [INFO ] [MAIN ] RGT: Starting full primary refresh. [2025-10-20T20:50:32.013971] [INFO ] [REMOTE ] RGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:32.251359] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-01. [2025-10-20T20:50:32.605715] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T20:50:32.939050] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-01 (0.69 seconds). [2025-10-20T20:50:33.051333] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-02. [2025-10-20T20:50:33.851441] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-07. [2025-10-20T20:50:34.184691] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T20:50:34.549171] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T20:50:34.651351] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-08. [2025-10-20T20:50:35.329565] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T20:50:35.451415] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-03. [2025-10-20T20:50:36.251375] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-09. [2025-10-20T20:50:36.539122] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T20:50:36.887024] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T20:50:37.051635] [INFO ] [REMOTE ] RFMZ: Fetching primary data for 2025-10. [2025-10-20T20:50:37.796249] [INFO ] [REMOTE ] RFMZ: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T20:50:37.798207] [INFO ] [REMOTE ] RFMZ: Full fetch complete (12267 records). [2025-10-20T20:50:37.798274] [DEBUG] [VALIDATOR ] RFMZ: Validating primary data. [2025-10-20T20:50:37.859510] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-04. [2025-10-20T20:50:38.171039] [DEBUG] [VALIDATOR ] RFMZ: All expected trading days are present. [2025-10-20T20:50:38.171115] [DEBUG] [VALIDATOR ] RFMZ: Primary data validation passed. [2025-10-20T20:50:38.171396] [INFO ] [MAIN ] RFMZ: Fetched primary data is valid. [2025-10-20T20:50:38.171434] [DEBUG] [LOCAL ] RFMZ: Saving primary data. [2025-10-20T20:50:38.171950] [DEBUG] [LOCAL ] RFMZ: Trimmed 12267 records to 8876 (keeping 200 days of data) [2025-10-20T20:50:38.226855] [DEBUG] [LOCAL ] RFMZ: Primary data saved (8876 records). [2025-10-20T20:50:38.226987] [INFO ] [MAIN ] RFMZ: Full history saved. [2025-10-20T20:50:38.227065] [INFO ] [MAIN ] Update Progress: 1792/2367 symbols completed. RFMZ took 28.9s. Estimated remaining: 1h 13m 59s. Rate: 7.8 symbols/min [2025-10-20T20:50:38.399624] [INFO ] [MAIN ] RH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:38.444064] [DEBUG] [LOCAL ] RH: Loaded 47994 existing records from primary storage. [2025-10-20T20:50:38.444366] [INFO ] [MAIN ] RH: Loaded 47994 existing records. [2025-10-20T20:50:38.621693] [WARN ] [MAIN ] RH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:38.621952] [INFO ] [MAIN ] RH: Wiping existing primary data before full refresh. [2025-10-20T20:50:38.622001] [INFO ] [LOCAL ] RH: Starting deletion of primary data. [2025-10-20T20:50:38.622194] [DEBUG] [LOCAL ] RH: Deleted primary data file. [2025-10-20T20:50:38.622233] [INFO ] [MAIN ] RH: Starting full primary refresh. [2025-10-20T20:50:38.656541] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-04. [2025-10-20T20:50:38.985175] [INFO ] [REMOTE ] RH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:39.451352] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-02. [2025-10-20T20:50:39.497289] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T20:50:39.828975] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-02 (0.38 seconds). [2025-10-20T20:50:39.828832] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T20:50:40.251348] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-05. [2025-10-20T20:50:41.051455] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-03. [2025-10-20T20:50:41.530377] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T20:50:41.675632] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T20:50:41.851447] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-05. [2025-10-20T20:50:42.651380] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-04. [2025-10-20T20:50:43.289215] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T20:50:43.293280] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T20:50:43.451357] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-06. [2025-10-20T20:50:44.251366] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-01. [2025-10-20T20:50:44.550146] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T20:50:45.051392] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-06. [2025-10-20T20:50:45.748253] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T20:50:45.851344] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-02. [2025-10-20T20:50:46.056215] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T20:50:46.651472] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-07. [2025-10-20T20:50:47.164361] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T20:50:47.451381] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-05. [2025-10-20T20:50:48.019871] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T20:50:48.019712] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T20:50:48.251434] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-03. [2025-10-20T20:50:49.051398] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-06. [2025-10-20T20:50:49.682624] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:50:49.851339] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-07. [2025-10-20T20:50:50.359079] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-03 (2.11 seconds). [2025-10-20T20:50:50.608117] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T20:50:50.651338] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-08. [2025-10-20T20:50:51.147080] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T20:50:51.451365] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-08. [2025-10-20T20:50:52.251374] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-04. [2025-10-20T20:50:52.866056] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T20:50:53.051366] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-09. [2025-10-20T20:50:53.599548] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T20:50:53.851365] [INFO ] [REMOTE ] RGT: Fetching primary data for 2025-10. [2025-10-20T20:50:54.390244] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T20:50:54.406674] [INFO ] [REMOTE ] RGT: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:50:54.407506] [INFO ] [REMOTE ] RGT: Full fetch complete (3949 records). [2025-10-20T20:50:54.407572] [DEBUG] [VALIDATOR ] RGT: Validating primary data. [2025-10-20T20:50:54.656474] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-09. [2025-10-20T20:50:54.773781] [DEBUG] [VALIDATOR ] RGT: All expected trading days are present. [2025-10-20T20:50:54.773884] [DEBUG] [VALIDATOR ] RGT: Primary data validation passed. [2025-10-20T20:50:54.774023] [INFO ] [MAIN ] RGT: Fetched primary data is valid. [2025-10-20T20:50:54.774063] [DEBUG] [LOCAL ] RGT: Saving primary data. [2025-10-20T20:50:54.774438] [DEBUG] [LOCAL ] RGT: Trimmed 3949 records to 2730 (keeping 200 days of data) [2025-10-20T20:50:54.790832] [DEBUG] [LOCAL ] RGT: Primary data saved (2730 records). [2025-10-20T20:50:54.790899] [INFO ] [MAIN ] RGT: Full history saved. [2025-10-20T20:50:54.790966] [INFO ] [MAIN ] Update Progress: 1793/2367 symbols completed. RGT took 23.5s. Estimated remaining: 1h 13m 54s. Rate: 7.8 symbols/min [2025-10-20T20:50:54.963060] [INFO ] [MAIN ] RHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:55.008398] [DEBUG] [LOCAL ] RHI: Loaded 50243 existing records from primary storage. [2025-10-20T20:50:55.008814] [INFO ] [MAIN ] RHI: Loaded 50243 existing records. [2025-10-20T20:50:55.187059] [WARN ] [MAIN ] RHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:50:55.187249] [INFO ] [MAIN ] RHI: Wiping existing primary data before full refresh. [2025-10-20T20:50:55.187296] [INFO ] [LOCAL ] RHI: Starting deletion of primary data. [2025-10-20T20:50:55.187476] [DEBUG] [LOCAL ] RHI: Deleted primary data file. [2025-10-20T20:50:55.187514] [INFO ] [MAIN ] RHI: Starting full primary refresh. [2025-10-20T20:50:55.460960] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-05. [2025-10-20T20:50:55.541675] [INFO ] [REMOTE ] RHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:56.056094] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T20:50:56.251367] [INFO ] [REMOTE ] RGA: Fetching primary data for 2025-10. [2025-10-20T20:50:56.949613] [INFO ] [REMOTE ] RGA: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:50:56.954159] [INFO ] [REMOTE ] RGA: Full fetch complete (44933 records). [2025-10-20T20:50:56.954284] [DEBUG] [VALIDATOR ] RGA: Validating primary data. [2025-10-20T20:50:56.954906] [FATAL] [VALIDATOR ] RGA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:50:56.954994] [FATAL] [MAIN ] RGA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:50:56.955038] [WARN ] [MAIN ] RGA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:50:56.955078] [INFO ] [LOCAL ] RGA: Starting deletion of primary data. [2025-10-20T20:50:56.955136] [DEBUG] [LOCAL ] RGA: No primary data file found to delete. [2025-10-20T20:50:56.955173] [INFO ] [LOCAL ] RGA: Starting deletion of derivative data. [2025-10-20T20:50:56.955297] [DEBUG] [LOCAL ] RGA: Deleted derivative data file. [2025-10-20T20:50:56.955366] [INFO ] [MAIN ] Update Progress: 1794/2367 symbols completed. RGA took 42.5s. Estimated remaining: 1h 13m 44s. Rate: 7.8 symbols/min [2025-10-20T20:50:57.056350] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-01. [2025-10-20T20:50:57.137042] [INFO ] [MAIN ] RHLD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:50:57.137263] [DEBUG] [LOCAL ] RHLD: No existing data file found in primary storage. [2025-10-20T20:50:57.137349] [WARN ] [MAIN ] RHLD: No existing data found, performing full refresh instead. [2025-10-20T20:50:57.137494] [INFO ] [MAIN ] RHLD: Wiping existing primary data before full refresh. [2025-10-20T20:50:57.137551] [INFO ] [LOCAL ] RHLD: Starting deletion of primary data. [2025-10-20T20:50:57.137726] [DEBUG] [LOCAL ] RHLD: No primary data file found to delete. [2025-10-20T20:50:57.137811] [INFO ] [MAIN ] RHLD: Starting full primary refresh. [2025-10-20T20:50:57.342529] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T20:50:57.535065] [INFO ] [REMOTE ] RHLD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:50:57.851363] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-07. [2025-10-20T20:50:58.651405] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-06. [2025-10-20T20:50:58.813775] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T20:50:59.081193] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T20:50:59.451361] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-08. [2025-10-20T20:51:00.251352] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-02. [2025-10-20T20:51:00.584178] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T20:51:00.635004] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T20:51:01.051415] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-09. [2025-10-20T20:51:01.851358] [INFO ] [REMOTE ] RHLD: Fetching primary data for 2025-01. [2025-10-20T20:51:01.945222] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T20:51:02.160680] [DEBUG] [REMOTE ] RHLD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T20:51:02.160988] [WARN ] [REMOTE ] RHLD: No data returned for month 2025-01 [2025-10-20T20:51:02.161098] [FATAL] [REMOTE ] RHLD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T20:51:02.161194] [FATAL] [MAIN ] RHLD: Failed to fetch primary data. [2025-10-20T20:51:02.161279] [WARN ] [MAIN ] RHLD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T20:51:02.161374] [INFO ] [LOCAL ] RHLD: Starting deletion of primary data. [2025-10-20T20:51:02.161455] [DEBUG] [LOCAL ] RHLD: No primary data file found to delete. [2025-10-20T20:51:02.161492] [INFO ] [LOCAL ] RHLD: Starting deletion of derivative data. [2025-10-20T20:51:02.161532] [DEBUG] [LOCAL ] RHLD: No derivative data file found to delete. [2025-10-20T20:51:02.161578] [INFO ] [MAIN ] Update Progress: 1795/2367 symbols completed. RHLD took 5.2s. Estimated remaining: 1h 13m 36s. Rate: 7.8 symbols/min [2025-10-20T20:51:02.330776] [INFO ] [MAIN ] RHP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:02.345792] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T20:51:02.385877] [DEBUG] [LOCAL ] RHP: Loaded 41203 existing records from primary storage. [2025-10-20T20:51:02.386172] [INFO ] [MAIN ] RHP: Loaded 41203 existing records. [2025-10-20T20:51:02.559161] [WARN ] [MAIN ] RHP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:02.559397] [INFO ] [MAIN ] RHP: Wiping existing primary data before full refresh. [2025-10-20T20:51:02.559448] [INFO ] [LOCAL ] RHP: Starting deletion of primary data. [2025-10-20T20:51:02.559728] [DEBUG] [LOCAL ] RHP: Deleted primary data file. [2025-10-20T20:51:02.559783] [INFO ] [MAIN ] RHP: Starting full primary refresh. [2025-10-20T20:51:02.656560] [INFO ] [REMOTE ] RGR: Fetching primary data for 2025-10. [2025-10-20T20:51:02.913902] [INFO ] [REMOTE ] RHP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:03.067078] [INFO ] [REMOTE ] RGR: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T20:51:03.070222] [INFO ] [REMOTE ] RGR: Full fetch complete (28692 records). [2025-10-20T20:51:03.070297] [DEBUG] [VALIDATOR ] RGR: Validating primary data. [2025-10-20T20:51:03.070790] [FATAL] [VALIDATOR ] RGR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:51:03.070888] [FATAL] [MAIN ] RGR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:51:03.070964] [WARN ] [MAIN ] RGR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:51:03.071046] [INFO ] [LOCAL ] RGR: Starting deletion of primary data. [2025-10-20T20:51:03.071123] [DEBUG] [LOCAL ] RGR: No primary data file found to delete. [2025-10-20T20:51:03.071177] [INFO ] [LOCAL ] RGR: Starting deletion of derivative data. [2025-10-20T20:51:03.071317] [DEBUG] [LOCAL ] RGR: Deleted derivative data file. [2025-10-20T20:51:03.071396] [INFO ] [MAIN ] Update Progress: 1796/2367 symbols completed. RGR took 33.1s. Estimated remaining: 1h 13m 26s. Rate: 7.8 symbols/min [2025-10-20T20:51:03.247963] [INFO ] [MAIN ] RIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:03.299158] [DEBUG] [LOCAL ] RIG: Loaded 54455 existing records from primary storage. [2025-10-20T20:51:03.299477] [INFO ] [MAIN ] RIG: Loaded 54455 existing records. [2025-10-20T20:51:03.456481] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-01. [2025-10-20T20:51:03.498244] [WARN ] [MAIN ] RIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:03.498417] [INFO ] [MAIN ] RIG: Wiping existing primary data before full refresh. [2025-10-20T20:51:03.498497] [INFO ] [LOCAL ] RIG: Starting deletion of primary data. [2025-10-20T20:51:03.499062] [DEBUG] [LOCAL ] RIG: Deleted primary data file. [2025-10-20T20:51:03.499121] [INFO ] [MAIN ] RIG: Starting full primary refresh. [2025-10-20T20:51:03.847439] [INFO ] [REMOTE ] RIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:04.251403] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-07. [2025-10-20T20:51:05.051409] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-03. [2025-10-20T20:51:05.162377] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T20:51:05.851348] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-02. [2025-10-20T20:51:06.085015] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T20:51:06.651351] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-01. [2025-10-20T20:51:06.730252] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T20:51:06.922221] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T20:51:07.451340] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-03. [2025-10-20T20:51:08.251336] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-04. [2025-10-20T20:51:08.410599] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T20:51:09.051366] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-08. [2025-10-20T20:51:09.217024] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T20:51:09.851402] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-02. [2025-10-20T20:51:10.121424] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T20:51:10.531386] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T20:51:10.651339] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-09. [2025-10-20T20:51:11.451422] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-04. [2025-10-20T20:51:11.753104] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T20:51:12.251454] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-03. [2025-10-20T20:51:12.376945] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T20:51:12.919880] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T20:51:13.051336] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-05. [2025-10-20T20:51:13.851418] [INFO ] [REMOTE ] RH: Fetching primary data for 2025-10. [2025-10-20T20:51:13.891709] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T20:51:14.563106] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T20:51:14.635710] [INFO ] [REMOTE ] RH: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:51:14.641908] [INFO ] [REMOTE ] RH: Full fetch complete (65862 records). [2025-10-20T20:51:14.642046] [DEBUG] [VALIDATOR ] RH: Validating primary data. [2025-10-20T20:51:14.642691] [FATAL] [VALIDATOR ] RH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:51:14.642779] [FATAL] [MAIN ] RH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:51:14.642822] [WARN ] [MAIN ] RH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:51:14.642860] [INFO ] [LOCAL ] RH: Starting deletion of primary data. [2025-10-20T20:51:14.642915] [DEBUG] [LOCAL ] RH: No primary data file found to delete. [2025-10-20T20:51:14.642947] [INFO ] [LOCAL ] RH: Starting deletion of derivative data. [2025-10-20T20:51:14.643067] [DEBUG] [LOCAL ] RH: Deleted derivative data file. [2025-10-20T20:51:14.643135] [INFO ] [MAIN ] Update Progress: 1797/2367 symbols completed. RH took 36.4s. Estimated remaining: 1h 13m 20s. Rate: 7.8 symbols/min [2025-10-20T20:51:14.651337] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-06. [2025-10-20T20:51:15.451453] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-05. [2025-10-20T20:51:16.144194] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T20:51:16.251454] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-04. [2025-10-20T20:51:17.051360] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-07. [2025-10-20T20:51:17.140470] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:51:17.851406] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-06. [2025-10-20T20:51:18.062886] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T20:51:18.616553] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T20:51:18.651346] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-08. [2025-10-20T20:51:19.132988] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T20:51:19.451370] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-07. [2025-10-20T20:51:20.225911] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T20:51:20.251351] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-09. [2025-10-20T20:51:21.051441] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-05. [2025-10-20T20:51:21.361496] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T20:51:21.444020] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T20:51:21.851383] [INFO ] [REMOTE ] RHP: Fetching primary data for 2025-10. [2025-10-20T20:51:22.527540] [INFO ] [REMOTE ] RHP: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:51:22.533117] [INFO ] [REMOTE ] RHP: Full fetch complete (58085 records). [2025-10-20T20:51:22.533207] [DEBUG] [VALIDATOR ] RHP: Validating primary data. [2025-10-20T20:51:22.533880] [FATAL] [VALIDATOR ] RHP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:51:22.533960] [FATAL] [MAIN ] RHP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:51:22.534022] [WARN ] [MAIN ] RHP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:51:22.534059] [INFO ] [LOCAL ] RHP: Starting deletion of primary data. [2025-10-20T20:51:22.534114] [DEBUG] [LOCAL ] RHP: No primary data file found to delete. [2025-10-20T20:51:22.534150] [INFO ] [LOCAL ] RHP: Starting deletion of derivative data. [2025-10-20T20:51:22.534309] [DEBUG] [LOCAL ] RHP: Deleted derivative data file. [2025-10-20T20:51:22.534418] [INFO ] [MAIN ] Update Progress: 1798/2367 symbols completed. RHP took 20.4s. Estimated remaining: 1h 13m 12s. Rate: 7.8 symbols/min [2025-10-20T20:51:22.651467] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-08. [2025-10-20T20:51:22.984903] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T20:51:23.451417] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-06. [2025-10-20T20:51:24.424123] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T20:51:24.435397] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-09. [2025-10-20T20:51:25.715254] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-06 (2.26 seconds). [2025-10-20T20:51:25.725924] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-07. [2025-10-20T20:51:26.251171] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T20:51:26.261526] [INFO ] [REMOTE ] RHI: Fetching primary data for 2025-10. [2025-10-20T20:51:26.847832] [INFO ] [REMOTE ] RHI: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T20:51:26.854513] [INFO ] [REMOTE ] RHI: Full fetch complete (70399 records). [2025-10-20T20:51:26.854615] [DEBUG] [VALIDATOR ] RHI: Validating primary data. [2025-10-20T20:51:26.855328] [FATAL] [VALIDATOR ] RHI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:51:26.855411] [FATAL] [MAIN ] RHI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:51:26.855454] [WARN ] [MAIN ] RHI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:51:26.855492] [INFO ] [LOCAL ] RHI: Starting deletion of primary data. [2025-10-20T20:51:26.855547] [DEBUG] [LOCAL ] RHI: No primary data file found to delete. [2025-10-20T20:51:26.855581] [INFO ] [LOCAL ] RHI: Starting deletion of derivative data. [2025-10-20T20:51:26.855748] [DEBUG] [LOCAL ] RHI: Deleted derivative data file. [2025-10-20T20:51:26.855843] [INFO ] [MAIN ] Update Progress: 1799/2367 symbols completed. RHI took 32.1s. Estimated remaining: 1h 13m 3s. Rate: 7.8 symbols/min [2025-10-20T20:51:27.677321] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T20:51:27.688804] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-08. [2025-10-20T20:51:29.640147] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T20:51:29.651521] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-09. [2025-10-20T20:51:31.691769] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:51:31.703027] [INFO ] [REMOTE ] RIG: Fetching primary data for 2025-10. [2025-10-20T20:51:32.472779] [INFO ] [REMOTE ] RIG: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:51:32.480291] [INFO ] [REMOTE ] RIG: Full fetch complete (77456 records). [2025-10-20T20:51:32.480442] [DEBUG] [VALIDATOR ] RIG: Validating primary data. [2025-10-20T20:51:32.481274] [FATAL] [VALIDATOR ] RIG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:51:32.481365] [FATAL] [MAIN ] RIG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:51:32.481452] [WARN ] [MAIN ] RIG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:51:32.481539] [INFO ] [LOCAL ] RIG: Starting deletion of primary data. [2025-10-20T20:51:32.481632] [DEBUG] [LOCAL ] RIG: No primary data file found to delete. [2025-10-20T20:51:32.481711] [INFO ] [LOCAL ] RIG: Starting deletion of derivative data. [2025-10-20T20:51:32.481859] [DEBUG] [LOCAL ] RIG: Deleted derivative data file. [2025-10-20T20:51:32.482160] [INFO ] [MAIN ] Update Progress: 1800/2367 symbols completed. RIG took 29.4s. Estimated remaining: 1h 12m 55s. Rate: 7.8 symbols/min [2025-10-20T20:51:33.122923] [INFO ] [MAIN ] RIO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:33.165770] [INFO ] [MAIN ] RJF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:33.133187] [INFO ] [MAIN ] RIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:33.198308] [INFO ] [MAIN ] RITM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:33.211345] [DEBUG] [LOCAL ] RIV: Loaded 12779 existing records from primary storage. [2025-10-20T20:51:33.211979] [INFO ] [MAIN ] RIV: Loaded 12779 existing records. [2025-10-20T20:51:33.434724] [WARN ] [MAIN ] RIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:33.450298] [INFO ] [MAIN ] RIV: Wiping existing primary data before full refresh. [2025-10-20T20:51:33.450947] [INFO ] [LOCAL ] RIV: Starting deletion of primary data. [2025-10-20T20:51:33.452269] [DEBUG] [LOCAL ] RJF: Loaded 47356 existing records from primary storage. [2025-10-20T20:51:33.468461] [DEBUG] [LOCAL ] RIV: Deleted primary data file. [2025-10-20T20:51:33.468978] [INFO ] [MAIN ] RIV: Starting full primary refresh. [2025-10-20T20:51:33.474931] [DEBUG] [LOCAL ] RIO: Loaded 54181 existing records from primary storage. [2025-10-20T20:51:33.468760] [INFO ] [MAIN ] RJF: Loaded 47356 existing records. [2025-10-20T20:51:33.572984] [INFO ] [MAIN ] RIO: Loaded 54181 existing records. [2025-10-20T20:51:33.606542] [DEBUG] [LOCAL ] RITM: Loaded 53721 existing records from primary storage. [2025-10-20T20:51:33.715599] [INFO ] [MAIN ] RITM: Loaded 53721 existing records. [2025-10-20T20:51:34.059922] [WARN ] [MAIN ] RJF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:34.174609] [INFO ] [MAIN ] RJF: Wiping existing primary data before full refresh. [2025-10-20T20:51:34.205600] [INFO ] [LOCAL ] RJF: Starting deletion of primary data. [2025-10-20T20:51:34.260826] [WARN ] [MAIN ] RIO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:34.307292] [DEBUG] [LOCAL ] RJF: Deleted primary data file. [2025-10-20T20:51:34.346308] [INFO ] [MAIN ] RJF: Starting full primary refresh. [2025-10-20T20:51:34.358999] [INFO ] [MAIN ] RIO: Wiping existing primary data before full refresh. [2025-10-20T20:51:34.365014] [WARN ] [MAIN ] RITM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:34.401551] [INFO ] [MAIN ] RITM: Wiping existing primary data before full refresh. [2025-10-20T20:51:34.416568] [INFO ] [LOCAL ] RIO: Starting deletion of primary data. [2025-10-20T20:51:34.401699] [INFO ] [LOCAL ] RITM: Starting deletion of primary data. [2025-10-20T20:51:34.488440] [DEBUG] [LOCAL ] RITM: Deleted primary data file. [2025-10-20T20:51:34.499505] [DEBUG] [LOCAL ] RIO: Deleted primary data file. [2025-10-20T20:51:34.518708] [INFO ] [MAIN ] RITM: Starting full primary refresh. [2025-10-20T20:51:34.521542] [INFO ] [MAIN ] RIO: Starting full primary refresh. [2025-10-20T20:51:34.620245] [INFO ] [REMOTE ] RIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:34.729351] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-01. [2025-10-20T20:51:35.389836] [INFO ] [REMOTE ] RJF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:35.421058] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-01. [2025-10-20T20:51:35.572119] [INFO ] [REMOTE ] RIO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:35.584985] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-01. [2025-10-20T20:51:35.607335] [INFO ] [REMOTE ] RITM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:35.607854] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-01. [2025-10-20T20:51:35.877251] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T20:51:35.880493] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-02. [2025-10-20T20:51:36.768718] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T20:51:36.771146] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-03. [2025-10-20T20:51:37.031144] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T20:51:37.040877] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-02. [2025-10-20T20:51:37.219917] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T20:51:37.231590] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-02. [2025-10-20T20:51:37.485894] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T20:51:37.503359] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-02. [2025-10-20T20:51:37.504082] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-03 (0.73 seconds). [2025-10-20T20:51:37.507751] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-04. [2025-10-20T20:51:38.382611] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T20:51:38.385668] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-05. [2025-10-20T20:51:38.758528] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T20:51:38.767876] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-03. [2025-10-20T20:51:38.860501] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T20:51:38.870900] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-03. [2025-10-20T20:51:38.922262] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T20:51:38.940279] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T20:51:39.451381] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-06. [2025-10-20T20:51:40.251356] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-03. [2025-10-20T20:51:40.353312] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T20:51:40.722429] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-03 (1.95 seconds). [2025-10-20T20:51:41.051443] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-04. [2025-10-20T20:51:41.851357] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-07. [2025-10-20T20:51:41.957632] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-03 (3.09 seconds). [2025-10-20T20:51:42.590082] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T20:51:42.651333] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-04. [2025-10-20T20:51:42.683639] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T20:51:42.944852] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T20:51:43.451366] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-04. [2025-10-20T20:51:44.251407] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-05. [2025-10-20T20:51:45.029746] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-04 (2.38 seconds). [2025-10-20T20:51:45.051334] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-05. [2025-10-20T20:51:45.562436] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-04 (2.11 seconds). [2025-10-20T20:51:45.851347] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-08. [2025-10-20T20:51:45.944295] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:51:46.651347] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-05. [2025-10-20T20:51:46.738975] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T20:51:47.188807] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T20:51:47.451434] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-06. [2025-10-20T20:51:48.251410] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-09. [2025-10-20T20:51:48.588076] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T20:51:49.051379] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-06. [2025-10-20T20:51:49.118944] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:51:49.170603] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T20:51:49.851379] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-06. [2025-10-20T20:51:50.651365] [INFO ] [REMOTE ] RIV: Fetching primary data for 2025-10. [2025-10-20T20:51:50.852064] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T20:51:51.451349] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-07. [2025-10-20T20:51:51.482942] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T20:51:51.684785] [INFO ] [REMOTE ] RIV: Incremental fetch complete for 2025-10 (1.03 seconds). [2025-10-20T20:51:51.687361] [INFO ] [REMOTE ] RIV: Full fetch complete (17617 records). [2025-10-20T20:51:51.687428] [DEBUG] [VALIDATOR ] RIV: Validating primary data. [2025-10-20T20:51:52.045494] [DEBUG] [VALIDATOR ] RIV: All expected trading days are present. [2025-10-20T20:51:52.045568] [DEBUG] [VALIDATOR ] RIV: Primary data validation passed. [2025-10-20T20:51:52.045960] [INFO ] [MAIN ] RIV: Fetched primary data is valid. [2025-10-20T20:51:52.046020] [DEBUG] [LOCAL ] RIV: Saving primary data. [2025-10-20T20:51:52.046528] [DEBUG] [LOCAL ] RIV: Trimmed 17617 records to 12682 (keeping 200 days of data) [2025-10-20T20:51:52.122198] [DEBUG] [LOCAL ] RIV: Primary data saved (12682 records). [2025-10-20T20:51:52.122330] [INFO ] [MAIN ] RIV: Full history saved. [2025-10-20T20:51:52.122407] [INFO ] [MAIN ] Update Progress: 1801/2367 symbols completed. RIV took 19.6s. Estimated remaining: 1h 12m 51s. Rate: 7.8 symbols/min [2025-10-20T20:51:52.259814] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-07. [2025-10-20T20:51:52.291022] [INFO ] [MAIN ] RKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:51:52.343631] [DEBUG] [LOCAL ] RKT: Loaded 54509 existing records from primary storage. [2025-10-20T20:51:52.343956] [INFO ] [MAIN ] RKT: Loaded 54509 existing records. [2025-10-20T20:51:52.517058] [WARN ] [MAIN ] RKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:51:52.517277] [INFO ] [MAIN ] RKT: Wiping existing primary data before full refresh. [2025-10-20T20:51:52.517324] [INFO ] [LOCAL ] RKT: Starting deletion of primary data. [2025-10-20T20:51:52.517789] [DEBUG] [LOCAL ] RKT: Deleted primary data file. [2025-10-20T20:51:52.517844] [INFO ] [MAIN ] RKT: Starting full primary refresh. [2025-10-20T20:51:52.870586] [INFO ] [REMOTE ] RKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:51:53.051343] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-01. [2025-10-20T20:51:53.154274] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T20:51:53.851381] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-07. [2025-10-20T20:51:54.202729] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T20:51:54.651391] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-08. [2025-10-20T20:51:54.851475] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T20:51:55.451371] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-08. [2025-10-20T20:51:55.990660] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T20:51:56.251401] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-08. [2025-10-20T20:51:56.406728] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T20:51:57.040573] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T20:51:57.051395] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-09. [2025-10-20T20:51:57.851398] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-09. [2025-10-20T20:51:58.327576] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-08 (2.08 seconds). [2025-10-20T20:51:58.651399] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-09. [2025-10-20T20:51:58.836813] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T20:51:59.451399] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-02. [2025-10-20T20:51:59.856867] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T20:52:00.251419] [INFO ] [REMOTE ] RITM: Fetching primary data for 2025-10. [2025-10-20T20:52:00.816119] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-09 (2.16 seconds). [2025-10-20T20:52:01.051350] [INFO ] [REMOTE ] RJF: Fetching primary data for 2025-10. [2025-10-20T20:52:01.109103] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T20:52:01.138105] [INFO ] [REMOTE ] RITM: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T20:52:01.145470] [INFO ] [REMOTE ] RITM: Full fetch complete (76363 records). [2025-10-20T20:52:01.145618] [DEBUG] [VALIDATOR ] RITM: Validating primary data. [2025-10-20T20:52:01.146429] [FATAL] [VALIDATOR ] RITM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:52:01.146529] [FATAL] [MAIN ] RITM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:52:01.146597] [WARN ] [MAIN ] RITM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:52:01.146697] [INFO ] [LOCAL ] RITM: Starting deletion of primary data. [2025-10-20T20:52:01.146792] [DEBUG] [LOCAL ] RITM: No primary data file found to delete. [2025-10-20T20:52:01.146847] [INFO ] [LOCAL ] RITM: Starting deletion of derivative data. [2025-10-20T20:52:01.146966] [DEBUG] [LOCAL ] RITM: Deleted derivative data file. [2025-10-20T20:52:01.147035] [INFO ] [MAIN ] Update Progress: 1802/2367 symbols completed. RITM took 28.7s. Estimated remaining: 1h 12m 43s. Rate: 7.8 symbols/min [2025-10-20T20:52:01.328020] [INFO ] [MAIN ] RL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:01.368400] [DEBUG] [LOCAL ] RL: Loaded 40561 existing records from primary storage. [2025-10-20T20:52:01.368760] [INFO ] [MAIN ] RL: Loaded 40561 existing records. [2025-10-20T20:52:01.543597] [WARN ] [MAIN ] RL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:01.543860] [INFO ] [MAIN ] RL: Wiping existing primary data before full refresh. [2025-10-20T20:52:01.543913] [INFO ] [LOCAL ] RL: Starting deletion of primary data. [2025-10-20T20:52:01.544093] [DEBUG] [LOCAL ] RL: Deleted primary data file. [2025-10-20T20:52:01.544129] [INFO ] [MAIN ] RL: Starting full primary refresh. [2025-10-20T20:52:01.853387] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-03. [2025-10-20T20:52:01.970586] [INFO ] [REMOTE ] RJF: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T20:52:01.983937] [INFO ] [REMOTE ] RL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:01.991594] [INFO ] [REMOTE ] RJF: Full fetch complete (68417 records). [2025-10-20T20:52:01.991820] [DEBUG] [VALIDATOR ] RJF: Validating primary data. [2025-10-20T20:52:01.992715] [FATAL] [VALIDATOR ] RJF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:52:01.992805] [FATAL] [MAIN ] RJF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:52:01.992851] [WARN ] [MAIN ] RJF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:52:01.992920] [INFO ] [LOCAL ] RJF: Starting deletion of primary data. [2025-10-20T20:52:01.992988] [DEBUG] [LOCAL ] RJF: No primary data file found to delete. [2025-10-20T20:52:01.993025] [INFO ] [LOCAL ] RJF: Starting deletion of derivative data. [2025-10-20T20:52:01.993164] [DEBUG] [LOCAL ] RJF: Deleted derivative data file. [2025-10-20T20:52:01.993243] [INFO ] [MAIN ] Update Progress: 1803/2367 symbols completed. RJF took 29.5s. Estimated remaining: 1h 12m 34s. Rate: 7.8 symbols/min [2025-10-20T20:52:02.187246] [INFO ] [MAIN ] RLI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:02.221520] [DEBUG] [LOCAL ] RLI: Loaded 35031 existing records from primary storage. [2025-10-20T20:52:02.221830] [INFO ] [MAIN ] RLI: Loaded 35031 existing records. [2025-10-20T20:52:02.396113] [WARN ] [MAIN ] RLI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:02.396355] [INFO ] [MAIN ] RLI: Wiping existing primary data before full refresh. [2025-10-20T20:52:02.396401] [INFO ] [LOCAL ] RLI: Starting deletion of primary data. [2025-10-20T20:52:02.396629] [DEBUG] [LOCAL ] RLI: Deleted primary data file. [2025-10-20T20:52:02.396710] [INFO ] [MAIN ] RLI: Starting full primary refresh. [2025-10-20T20:52:02.659361] [INFO ] [REMOTE ] RIO: Fetching primary data for 2025-10. [2025-10-20T20:52:02.747442] [INFO ] [REMOTE ] RLI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:03.451354] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-01. [2025-10-20T20:52:03.879508] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T20:52:04.251421] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-01. [2025-10-20T20:52:04.735760] [INFO ] [REMOTE ] RIO: Incremental fetch complete for 2025-10 (2.08 seconds). [2025-10-20T20:52:04.743504] [INFO ] [REMOTE ] RIO: Full fetch complete (77512 records). [2025-10-20T20:52:04.744018] [DEBUG] [VALIDATOR ] RIO: Validating primary data. [2025-10-20T20:52:05.056460] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-04. [2025-10-20T20:52:05.141147] [DEBUG] [VALIDATOR ] RIO: All expected trading days are present. [2025-10-20T20:52:05.141242] [DEBUG] [VALIDATOR ] RIO: Primary data validation passed. [2025-10-20T20:52:05.142383] [INFO ] [MAIN ] RIO: Fetched primary data is valid. [2025-10-20T20:52:05.142466] [DEBUG] [LOCAL ] RIO: Saving primary data. [2025-10-20T20:52:05.144169] [DEBUG] [LOCAL ] RIO: Trimmed 77512 records to 53402 (keeping 200 days of data) [2025-10-20T20:52:05.270784] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T20:52:05.497034] [DEBUG] [LOCAL ] RIO: Primary data saved (53402 records). [2025-10-20T20:52:05.497167] [INFO ] [MAIN ] RIO: Full history saved. [2025-10-20T20:52:05.497260] [INFO ] [MAIN ] Update Progress: 1804/2367 symbols completed. RIO took 33.0s. Estimated remaining: 1h 12m 25s. Rate: 7.8 symbols/min [2025-10-20T20:52:05.670016] [INFO ] [MAIN ] RLJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:05.689151] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:52:05.734415] [DEBUG] [LOCAL ] RLJ: Loaded 48741 existing records from primary storage. [2025-10-20T20:52:05.734778] [INFO ] [MAIN ] RLJ: Loaded 48741 existing records. [2025-10-20T20:52:05.856590] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-02. [2025-10-20T20:52:05.911725] [WARN ] [MAIN ] RLJ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:05.911897] [INFO ] [MAIN ] RLJ: Wiping existing primary data before full refresh. [2025-10-20T20:52:05.911945] [INFO ] [LOCAL ] RLJ: Starting deletion of primary data. [2025-10-20T20:52:05.912123] [DEBUG] [LOCAL ] RLJ: Deleted primary data file. [2025-10-20T20:52:05.912163] [INFO ] [MAIN ] RLJ: Starting full primary refresh. [2025-10-20T20:52:06.265914] [INFO ] [REMOTE ] RLJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:06.651425] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-01. [2025-10-20T20:52:07.133297] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T20:52:07.377112] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T20:52:07.451344] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-05. [2025-10-20T20:52:08.107338] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T20:52:08.251385] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-02. [2025-10-20T20:52:09.051432] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-02. [2025-10-20T20:52:09.579739] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-05 (2.13 seconds). [2025-10-20T20:52:09.772819] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T20:52:09.851368] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-06. [2025-10-20T20:52:10.381091] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T20:52:10.651407] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-03. [2025-10-20T20:52:11.451384] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-03. [2025-10-20T20:52:11.781514] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T20:52:12.251405] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-07. [2025-10-20T20:52:12.385448] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:52:13.015194] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T20:52:13.051328] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-04. [2025-10-20T20:52:13.851359] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-04. [2025-10-20T20:52:14.636118] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T20:52:14.651326] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-08. [2025-10-20T20:52:14.905560] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T20:52:15.290834] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T20:52:15.451349] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-05. [2025-10-20T20:52:16.251362] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-03. [2025-10-20T20:52:16.650619] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T20:52:16.700541] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T20:52:17.051391] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-06. [2025-10-20T20:52:17.721106] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T20:52:17.851419] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-04. [2025-10-20T20:52:18.651133] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T20:52:18.659910] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-09. [2025-10-20T20:52:19.343779] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T20:52:19.451381] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-05. [2025-10-20T20:52:20.251400] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-05. [2025-10-20T20:52:20.472754] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T20:52:21.051399] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-06. [2025-10-20T20:52:21.105561] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-09 (2.45 seconds). [2025-10-20T20:52:21.851348] [INFO ] [REMOTE ] RKT: Fetching primary data for 2025-10. [2025-10-20T20:52:22.072267] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T20:52:22.102035] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T20:52:22.581528] [INFO ] [REMOTE ] RKT: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T20:52:22.589023] [INFO ] [REMOTE ] RKT: Full fetch complete (77200 records). [2025-10-20T20:52:22.589141] [DEBUG] [VALIDATOR ] RKT: Validating primary data. [2025-10-20T20:52:22.589849] [FATAL] [VALIDATOR ] RKT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:52:22.589952] [FATAL] [MAIN ] RKT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:52:22.589995] [WARN ] [MAIN ] RKT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:52:22.590057] [INFO ] [LOCAL ] RKT: Starting deletion of primary data. [2025-10-20T20:52:22.590121] [DEBUG] [LOCAL ] RKT: No primary data file found to delete. [2025-10-20T20:52:22.590155] [INFO ] [LOCAL ] RKT: Starting deletion of derivative data. [2025-10-20T20:52:22.590275] [DEBUG] [LOCAL ] RKT: Deleted derivative data file. [2025-10-20T20:52:22.590344] [INFO ] [MAIN ] Update Progress: 1805/2367 symbols completed. RKT took 30.5s. Estimated remaining: 1h 12m 20s. Rate: 7.8 symbols/min [2025-10-20T20:52:22.656548] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-07. [2025-10-20T20:52:22.760773] [INFO ] [MAIN ] RLTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:22.773635] [DEBUG] [LOCAL ] RLTY: Loaded 11297 existing records from primary storage. [2025-10-20T20:52:22.773891] [INFO ] [MAIN ] RLTY: Loaded 11297 existing records. [2025-10-20T20:52:22.952730] [WARN ] [MAIN ] RLTY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:22.952996] [INFO ] [MAIN ] RLTY: Wiping existing primary data before full refresh. [2025-10-20T20:52:22.953047] [INFO ] [LOCAL ] RLTY: Starting deletion of primary data. [2025-10-20T20:52:22.953284] [DEBUG] [LOCAL ] RLTY: Deleted primary data file. [2025-10-20T20:52:22.953339] [INFO ] [MAIN ] RLTY: Starting full primary refresh. [2025-10-20T20:52:23.302256] [INFO ] [REMOTE ] RLTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:23.451334] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-01. [2025-10-20T20:52:23.935258] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T20:52:24.132864] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T20:52:24.251378] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-02. [2025-10-20T20:52:25.051361] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-07. [2025-10-20T20:52:25.265040] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T20:52:25.851360] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-08. [2025-10-20T20:52:26.651384] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-03. [2025-10-20T20:52:26.698428] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T20:52:27.050919] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-08 (1.20 seconds). [2025-10-20T20:52:27.341416] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T20:52:27.451395] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-09. [2025-10-20T20:52:28.251421] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-04. [2025-10-20T20:52:29.051364] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-08. [2025-10-20T20:52:29.194170] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-04 (0.94 seconds). [2025-10-20T20:52:29.851397] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-05. [2025-10-20T20:52:30.057724] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-09 (2.61 seconds). [2025-10-20T20:52:30.651459] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-06. [2025-10-20T20:52:30.737032] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T20:52:30.986125] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T20:52:31.451419] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-09. [2025-10-20T20:52:32.233269] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T20:52:32.251332] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-07. [2025-10-20T20:52:33.051419] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-06. [2025-10-20T20:52:33.172802] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T20:52:33.585509] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-06 (0.53 seconds). [2025-10-20T20:52:33.851415] [INFO ] [REMOTE ] RLJ: Fetching primary data for 2025-10. [2025-10-20T20:52:33.887493] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T20:52:34.651382] [INFO ] [REMOTE ] RLI: Fetching primary data for 2025-10. [2025-10-20T20:52:35.326949] [INFO ] [REMOTE ] RLI: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:52:35.332977] [INFO ] [REMOTE ] RLI: Full fetch complete (48821 records). [2025-10-20T20:52:35.333133] [DEBUG] [VALIDATOR ] RLI: Validating primary data. [2025-10-20T20:52:35.333838] [FATAL] [VALIDATOR ] RLI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:52:35.333931] [FATAL] [MAIN ] RLI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:52:35.333977] [WARN ] [MAIN ] RLI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:52:35.334043] [INFO ] [LOCAL ] RLI: Starting deletion of primary data. [2025-10-20T20:52:35.334113] [DEBUG] [LOCAL ] RLI: No primary data file found to delete. [2025-10-20T20:52:35.334149] [INFO ] [LOCAL ] RLI: Starting deletion of derivative data. [2025-10-20T20:52:35.334331] [DEBUG] [LOCAL ] RLI: Deleted derivative data file. [2025-10-20T20:52:35.334786] [INFO ] [MAIN ] Update Progress: 1806/2367 symbols completed. RLI took 33.3s. Estimated remaining: 1h 12m 14s. Rate: 7.8 symbols/min [2025-10-20T20:52:35.456443] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-08. [2025-10-20T20:52:35.539949] [INFO ] [MAIN ] RLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:35.560524] [INFO ] [REMOTE ] RLJ: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T20:52:35.568354] [INFO ] [REMOTE ] RLJ: Full fetch complete (69422 records). [2025-10-20T20:52:35.568496] [DEBUG] [VALIDATOR ] RLJ: Validating primary data. [2025-10-20T20:52:35.894180] [DEBUG] [LOCAL ] RLX: Loaded 48812 existing records from primary storage. [2025-10-20T20:52:35.905237] [INFO ] [MAIN ] RLX: Loaded 48812 existing records. [2025-10-20T20:52:36.107750] [DEBUG] [VALIDATOR ] RLJ: All expected trading days are present. [2025-10-20T20:52:36.112999] [DEBUG] [VALIDATOR ] RLJ: Primary data validation passed. [2025-10-20T20:52:36.119281] [INFO ] [MAIN ] RLJ: Fetched primary data is valid. [2025-10-20T20:52:36.119374] [DEBUG] [LOCAL ] RLJ: Saving primary data. [2025-10-20T20:52:36.172353] [DEBUG] [LOCAL ] RLJ: Trimmed 69422 records to 48004 (keeping 200 days of data) [2025-10-20T20:52:36.172191] [WARN ] [MAIN ] RLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:36.172680] [INFO ] [MAIN ] RLX: Wiping existing primary data before full refresh. [2025-10-20T20:52:36.172833] [INFO ] [LOCAL ] RLX: Starting deletion of primary data. [2025-10-20T20:52:36.173446] [DEBUG] [LOCAL ] RLX: Deleted primary data file. [2025-10-20T20:52:36.173544] [INFO ] [MAIN ] RLX: Starting full primary refresh. [2025-10-20T20:52:36.261429] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-07. [2025-10-20T20:52:36.767564] [INFO ] [REMOTE ] RLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:36.823405] [DEBUG] [LOCAL ] RLJ: Primary data saved (48004 records). [2025-10-20T20:52:36.823536] [INFO ] [MAIN ] RLJ: Full history saved. [2025-10-20T20:52:36.823614] [INFO ] [MAIN ] Update Progress: 1807/2367 symbols completed. RLJ took 31.3s. Estimated remaining: 1h 12m 4s. Rate: 7.8 symbols/min [2025-10-20T20:52:37.004610] [INFO ] [MAIN ] RM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:37.013676] [DEBUG] [LOCAL ] RM: Loaded 7642 existing records from primary storage. [2025-10-20T20:52:37.013894] [INFO ] [MAIN ] RM: Loaded 7642 existing records. [2025-10-20T20:52:37.062523] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-01. [2025-10-20T20:52:37.199217] [WARN ] [MAIN ] RM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:37.205487] [INFO ] [MAIN ] RM: Wiping existing primary data before full refresh. [2025-10-20T20:52:37.207429] [INFO ] [LOCAL ] RM: Starting deletion of primary data. [2025-10-20T20:52:37.207761] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-07 (0.95 seconds). [2025-10-20T20:52:37.218519] [DEBUG] [LOCAL ] RM: Deleted primary data file. [2025-10-20T20:52:37.231991] [INFO ] [MAIN ] RM: Starting full primary refresh. [2025-10-20T20:52:37.265029] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-08 (1.81 seconds). [2025-10-20T20:52:37.588852] [INFO ] [REMOTE ] RM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:37.851394] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-01. [2025-10-20T20:52:38.506738] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T20:52:38.651385] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-02. [2025-10-20T20:52:38.939342] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T20:52:39.451367] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-09. [2025-10-20T20:52:40.251419] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-08. [2025-10-20T20:52:40.326975] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T20:52:40.720614] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T20:52:41.051353] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-03. [2025-10-20T20:52:41.093579] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T20:52:41.851443] [INFO ] [REMOTE ] RL: Fetching primary data for 2025-10. [2025-10-20T20:52:42.557465] [INFO ] [REMOTE ] RL: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T20:52:42.562318] [INFO ] [REMOTE ] RL: Full fetch complete (59239 records). [2025-10-20T20:52:42.562434] [DEBUG] [VALIDATOR ] RL: Validating primary data. [2025-10-20T20:52:42.563116] [FATAL] [VALIDATOR ] RL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:52:42.563199] [FATAL] [MAIN ] RL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:52:42.563240] [WARN ] [MAIN ] RL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:52:42.563307] [INFO ] [LOCAL ] RL: Starting deletion of primary data. [2025-10-20T20:52:42.563388] [DEBUG] [LOCAL ] RL: No primary data file found to delete. [2025-10-20T20:52:42.563442] [INFO ] [LOCAL ] RL: Starting deletion of derivative data. [2025-10-20T20:52:42.563585] [DEBUG] [LOCAL ] RL: Deleted derivative data file. [2025-10-20T20:52:42.563710] [INFO ] [MAIN ] Update Progress: 1808/2367 symbols completed. RL took 41.4s. Estimated remaining: 1h 11m 56s. Rate: 7.8 symbols/min [2025-10-20T20:52:42.661698] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-09. [2025-10-20T20:52:42.736855] [INFO ] [MAIN ] RMAX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:42.785624] [DEBUG] [LOCAL ] RMAX: Loaded 21894 existing records from primary storage. [2025-10-20T20:52:42.786467] [INFO ] [MAIN ] RMAX: Loaded 21894 existing records. [2025-10-20T20:52:42.788149] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T20:52:42.972970] [WARN ] [MAIN ] RMAX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:42.973175] [INFO ] [MAIN ] RMAX: Wiping existing primary data before full refresh. [2025-10-20T20:52:42.973221] [INFO ] [LOCAL ] RMAX: Starting deletion of primary data. [2025-10-20T20:52:42.973415] [DEBUG] [LOCAL ] RMAX: Deleted primary data file. [2025-10-20T20:52:42.973470] [INFO ] [MAIN ] RMAX: Starting full primary refresh. [2025-10-20T20:52:43.316708] [INFO ] [REMOTE ] RMAX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:43.451337] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-01. [2025-10-20T20:52:43.537022] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:52:44.251462] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-02. [2025-10-20T20:52:44.744065] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T20:52:45.035824] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:52:45.051325] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-03. [2025-10-20T20:52:45.851345] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-02. [2025-10-20T20:52:45.920839] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:52:46.619311] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-02 (0.77 seconds). [2025-10-20T20:52:46.651345] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-03. [2025-10-20T20:52:47.457638] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-04. [2025-10-20T20:52:47.459775] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T20:52:48.251426] [INFO ] [REMOTE ] RLTY: Fetching primary data for 2025-10. [2025-10-20T20:52:48.280899] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:52:49.049618] [INFO ] [REMOTE ] RLTY: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:52:49.051988] [INFO ] [REMOTE ] RLTY: Full fetch complete (17304 records). [2025-10-20T20:52:49.051783] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-04. [2025-10-20T20:52:49.053085] [DEBUG] [VALIDATOR ] RLTY: Validating primary data. [2025-10-20T20:52:49.412125] [DEBUG] [VALIDATOR ] RLTY: All expected trading days are present. [2025-10-20T20:52:49.412205] [DEBUG] [VALIDATOR ] RLTY: Primary data validation passed. [2025-10-20T20:52:49.412569] [INFO ] [MAIN ] RLTY: Fetched primary data is valid. [2025-10-20T20:52:49.412607] [DEBUG] [LOCAL ] RLTY: Saving primary data. [2025-10-20T20:52:49.413330] [DEBUG] [LOCAL ] RLTY: Trimmed 17304 records to 11193 (keeping 200 days of data) [2025-10-20T20:52:49.479435] [DEBUG] [LOCAL ] RLTY: Primary data saved (11193 records). [2025-10-20T20:52:49.479560] [INFO ] [MAIN ] RLTY: Full history saved. [2025-10-20T20:52:49.479634] [INFO ] [MAIN ] Update Progress: 1809/2367 symbols completed. RLTY took 26.9s. Estimated remaining: 1h 11m 48s. Rate: 7.8 symbols/min [2025-10-20T20:52:49.646035] [INFO ] [MAIN ] RMD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:49.685286] [DEBUG] [LOCAL ] RMD: Loaded 42676 existing records from primary storage. [2025-10-20T20:52:49.685559] [INFO ] [MAIN ] RMD: Loaded 42676 existing records. [2025-10-20T20:52:49.862222] [WARN ] [MAIN ] RMD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:52:49.857792] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-05. [2025-10-20T20:52:49.862597] [INFO ] [MAIN ] RMD: Wiping existing primary data before full refresh. [2025-10-20T20:52:49.863276] [INFO ] [LOCAL ] RMD: Starting deletion of primary data. [2025-10-20T20:52:49.863521] [DEBUG] [LOCAL ] RMD: Deleted primary data file. [2025-10-20T20:52:49.863561] [INFO ] [MAIN ] RMD: Starting full primary refresh. [2025-10-20T20:52:50.209892] [INFO ] [REMOTE ] RMD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:52:50.651354] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-01. [2025-10-20T20:52:50.842144] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T20:52:50.860867] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T20:52:51.451365] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-06. [2025-10-20T20:52:52.022393] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T20:52:52.049921] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T20:52:52.251392] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-07. [2025-10-20T20:52:53.051345] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-02. [2025-10-20T20:52:53.143278] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-07 (0.89 seconds). [2025-10-20T20:52:53.851383] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-04. [2025-10-20T20:52:54.651338] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-05. [2025-10-20T20:52:54.784957] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:52:54.837057] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T20:52:55.451420] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-03. [2025-10-20T20:52:56.251345] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-08. [2025-10-20T20:52:56.341456] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:52:57.051431] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-05. [2025-10-20T20:52:57.076365] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T20:52:57.292093] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T20:52:57.851346] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-09. [2025-10-20T20:52:57.919616] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:52:58.632076] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T20:52:58.651330] [INFO ] [REMOTE ] RM: Fetching primary data for 2025-10. [2025-10-20T20:52:59.351034] [INFO ] [REMOTE ] RM: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:52:59.352440] [INFO ] [REMOTE ] RM: Full fetch complete (11506 records). [2025-10-20T20:52:59.352507] [DEBUG] [VALIDATOR ] RM: Validating primary data. [2025-10-20T20:52:59.456488] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-06. [2025-10-20T20:52:59.714037] [DEBUG] [VALIDATOR ] RM: All expected trading days are present. [2025-10-20T20:52:59.714111] [DEBUG] [VALIDATOR ] RM: Primary data validation passed. [2025-10-20T20:52:59.714387] [INFO ] [MAIN ] RM: Fetched primary data is valid. [2025-10-20T20:52:59.714446] [DEBUG] [LOCAL ] RM: Saving primary data. [2025-10-20T20:52:59.714904] [DEBUG] [LOCAL ] RM: Trimmed 11506 records to 7548 (keeping 200 days of data) [2025-10-20T20:52:59.759433] [DEBUG] [LOCAL ] RM: Primary data saved (7548 records). [2025-10-20T20:52:59.759559] [INFO ] [MAIN ] RM: Full history saved. [2025-10-20T20:52:59.759636] [INFO ] [MAIN ] Update Progress: 1810/2367 symbols completed. RM took 22.9s. Estimated remaining: 1h 11m 41s. Rate: 7.8 symbols/min [2025-10-20T20:52:59.924373] [INFO ] [MAIN ] RMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:52:59.931758] [DEBUG] [LOCAL ] RMI: Loaded 4804 existing records from primary storage. [2025-10-20T20:52:59.931950] [INFO ] [MAIN ] RMI: Loaded 4804 existing records. [2025-10-20T20:53:00.105966] [WARN ] [MAIN ] RMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:00.106160] [INFO ] [MAIN ] RMI: Wiping existing primary data before full refresh. [2025-10-20T20:53:00.106205] [INFO ] [LOCAL ] RMI: Starting deletion of primary data. [2025-10-20T20:53:00.106361] [DEBUG] [LOCAL ] RMI: Deleted primary data file. [2025-10-20T20:53:00.106423] [INFO ] [MAIN ] RMI: Starting full primary refresh. [2025-10-20T20:53:00.276245] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-06. [2025-10-20T20:53:00.460767] [INFO ] [REMOTE ] RMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:00.720133] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T20:53:01.051374] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-07. [2025-10-20T20:53:01.851488] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-01. [2025-10-20T20:53:02.161509] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T20:53:02.276050] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T20:53:02.319375] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-10-20T20:53:02.651440] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-04. [2025-10-20T20:53:03.451390] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-08. [2025-10-20T20:53:04.251370] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-07. [2025-10-20T20:53:04.607183] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T20:53:04.913044] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T20:53:05.051381] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-05. [2025-10-20T20:53:05.756249] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T20:53:05.851374] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-08. [2025-10-20T20:53:06.651381] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-02. [2025-10-20T20:53:06.774386] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T20:53:07.150054] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T20:53:07.451422] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-06. [2025-10-20T20:53:07.698961] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-08 (1.85 seconds). [2025-10-20T20:53:08.251362] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-03. [2025-10-20T20:53:08.665306] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-03 (0.41 seconds). [2025-10-20T20:53:09.051354] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-09. [2025-10-20T20:53:09.075134] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T20:53:09.851355] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-07. [2025-10-20T20:53:10.163800] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T20:53:10.651386] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-04. [2025-10-20T20:53:11.361426] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T20:53:11.387238] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T20:53:11.451364] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-05. [2025-10-20T20:53:12.201237] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T20:53:12.251356] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-09. [2025-10-20T20:53:13.051364] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-06. [2025-10-20T20:53:13.851424] [INFO ] [REMOTE ] RMAX: Fetching primary data for 2025-10. [2025-10-20T20:53:14.132051] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T20:53:14.612732] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T20:53:14.651335] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-07. [2025-10-20T20:53:15.293629] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T20:53:15.451397] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-08. [2025-10-20T20:53:15.527211] [INFO ] [REMOTE ] RMAX: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T20:53:15.530322] [INFO ] [REMOTE ] RMAX: Full fetch complete (29978 records). [2025-10-20T20:53:15.530392] [DEBUG] [VALIDATOR ] RMAX: Validating primary data. [2025-10-20T20:53:15.900527] [DEBUG] [VALIDATOR ] RMAX: All expected trading days are present. [2025-10-20T20:53:15.900600] [DEBUG] [VALIDATOR ] RMAX: Primary data validation passed. [2025-10-20T20:53:15.901195] [INFO ] [MAIN ] RMAX: Fetched primary data is valid. [2025-10-20T20:53:15.901255] [DEBUG] [LOCAL ] RMAX: Saving primary data. [2025-10-20T20:53:15.901953] [DEBUG] [LOCAL ] RMAX: Trimmed 29978 records to 21719 (keeping 200 days of data) [2025-10-20T20:53:16.020405] [DEBUG] [LOCAL ] RMAX: Primary data saved (21719 records). [2025-10-20T20:53:16.020531] [INFO ] [MAIN ] RMAX: Full history saved. [2025-10-20T20:53:16.020606] [INFO ] [MAIN ] Update Progress: 1811/2367 symbols completed. RMAX took 33.5s. Estimated remaining: 1h 11m 36s. Rate: 7.8 symbols/min [2025-10-20T20:53:16.191945] [INFO ] [MAIN ] RMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:16.203823] [DEBUG] [LOCAL ] RMM: Loaded 10866 existing records from primary storage. [2025-10-20T20:53:16.204031] [INFO ] [MAIN ] RMM: Loaded 10866 existing records. [2025-10-20T20:53:16.260002] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-08. [2025-10-20T20:53:16.294407] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T20:53:16.394932] [WARN ] [MAIN ] RMM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:16.395124] [INFO ] [MAIN ] RMM: Wiping existing primary data before full refresh. [2025-10-20T20:53:16.395171] [INFO ] [LOCAL ] RMM: Starting deletion of primary data. [2025-10-20T20:53:16.395330] [DEBUG] [LOCAL ] RMM: Deleted primary data file. [2025-10-20T20:53:16.395369] [INFO ] [MAIN ] RMM: Starting full primary refresh. [2025-10-20T20:53:16.770370] [INFO ] [REMOTE ] RMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:17.051438] [INFO ] [REMOTE ] RLX: Fetching primary data for 2025-10. [2025-10-20T20:53:17.691100] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T20:53:17.851364] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-09. [2025-10-20T20:53:18.228606] [INFO ] [REMOTE ] RLX: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T20:53:18.235940] [INFO ] [REMOTE ] RLX: Full fetch complete (70177 records). [2025-10-20T20:53:18.236036] [DEBUG] [VALIDATOR ] RLX: Validating primary data. [2025-10-20T20:53:18.626626] [DEBUG] [VALIDATOR ] RLX: All expected trading days are present. [2025-10-20T20:53:18.626746] [DEBUG] [VALIDATOR ] RLX: Primary data validation passed. [2025-10-20T20:53:18.628122] [INFO ] [MAIN ] RLX: Fetched primary data is valid. [2025-10-20T20:53:18.628221] [DEBUG] [LOCAL ] RLX: Saving primary data. [2025-10-20T20:53:18.629567] [DEBUG] [LOCAL ] RLX: Trimmed 70177 records to 48187 (keeping 200 days of data) [2025-10-20T20:53:18.676708] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-09. [2025-10-20T20:53:18.903219] [DEBUG] [LOCAL ] RLX: Primary data saved (48187 records). [2025-10-20T20:53:18.903360] [INFO ] [MAIN ] RLX: Full history saved. [2025-10-20T20:53:18.903695] [INFO ] [MAIN ] Update Progress: 1812/2367 symbols completed. RLX took 43.6s. Estimated remaining: 1h 11m 26s. Rate: 7.8 symbols/min [2025-10-20T20:53:19.072770] [INFO ] [MAIN ] RMMZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:19.080981] [DEBUG] [LOCAL ] RMMZ: Loaded 4841 existing records from primary storage. [2025-10-20T20:53:19.081208] [INFO ] [MAIN ] RMMZ: Loaded 4841 existing records. [2025-10-20T20:53:19.262724] [WARN ] [MAIN ] RMMZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:19.262973] [INFO ] [MAIN ] RMMZ: Wiping existing primary data before full refresh. [2025-10-20T20:53:19.263031] [INFO ] [LOCAL ] RMMZ: Starting deletion of primary data. [2025-10-20T20:53:19.263225] [DEBUG] [LOCAL ] RMMZ: Deleted primary data file. [2025-10-20T20:53:19.263281] [INFO ] [MAIN ] RMMZ: Starting full primary refresh. [2025-10-20T20:53:19.492852] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:53:19.457086] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-01. [2025-10-20T20:53:19.635852] [INFO ] [REMOTE ] RMMZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:19.658677] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T20:53:20.251414] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-01. [2025-10-20T20:53:20.336153] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T20:53:20.886855] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T20:53:21.051402] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-02. [2025-10-20T20:53:21.851374] [INFO ] [REMOTE ] RMD: Fetching primary data for 2025-10. [2025-10-20T20:53:21.869032] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T20:53:22.355621] [INFO ] [REMOTE ] RMD: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T20:53:22.361720] [INFO ] [REMOTE ] RMD: Full fetch complete (61266 records). [2025-10-20T20:53:22.361884] [DEBUG] [VALIDATOR ] RMD: Validating primary data. [2025-10-20T20:53:22.362612] [FATAL] [VALIDATOR ] RMD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:53:22.362749] [FATAL] [MAIN ] RMD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:53:22.362791] [WARN ] [MAIN ] RMD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:53:22.362828] [INFO ] [LOCAL ] RMD: Starting deletion of primary data. [2025-10-20T20:53:22.362884] [DEBUG] [LOCAL ] RMD: No primary data file found to delete. [2025-10-20T20:53:22.362917] [INFO ] [LOCAL ] RMD: Starting deletion of derivative data. [2025-10-20T20:53:22.363036] [DEBUG] [LOCAL ] RMD: Deleted derivative data file. [2025-10-20T20:53:22.363105] [INFO ] [MAIN ] Update Progress: 1813/2367 symbols completed. RMD took 32.9s. Estimated remaining: 1h 11m 17s. Rate: 7.8 symbols/min [2025-10-20T20:53:22.531105] [INFO ] [MAIN ] RMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:22.548182] [DEBUG] [LOCAL ] RMT: Loaded 17753 existing records from primary storage. [2025-10-20T20:53:22.548388] [INFO ] [MAIN ] RMT: Loaded 17753 existing records. [2025-10-20T20:53:22.656563] [INFO ] [REMOTE ] RMI: Fetching primary data for 2025-10. [2025-10-20T20:53:22.746616] [WARN ] [MAIN ] RMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:22.746823] [INFO ] [MAIN ] RMT: Wiping existing primary data before full refresh. [2025-10-20T20:53:22.746868] [INFO ] [LOCAL ] RMT: Starting deletion of primary data. [2025-10-20T20:53:22.747053] [DEBUG] [LOCAL ] RMT: Deleted primary data file. [2025-10-20T20:53:22.747109] [INFO ] [MAIN ] RMT: Starting full primary refresh. [2025-10-20T20:53:23.092156] [INFO ] [REMOTE ] RMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:23.122003] [INFO ] [REMOTE ] RMI: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T20:53:23.123168] [INFO ] [REMOTE ] RMI: Full fetch complete (5999 records). [2025-10-20T20:53:23.123239] [DEBUG] [VALIDATOR ] RMI: Validating primary data. [2025-10-20T20:53:23.456524] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-03. [2025-10-20T20:53:23.500444] [DEBUG] [VALIDATOR ] RMI: All expected trading days are present. [2025-10-20T20:53:23.500570] [DEBUG] [VALIDATOR ] RMI: Primary data validation passed. [2025-10-20T20:53:23.500767] [INFO ] [MAIN ] RMI: Fetched primary data is valid. [2025-10-20T20:53:23.500826] [DEBUG] [LOCAL ] RMI: Saving primary data. [2025-10-20T20:53:23.501224] [DEBUG] [LOCAL ] RMI: Trimmed 5999 records to 4709 (keeping 200 days of data) [2025-10-20T20:53:23.530054] [DEBUG] [LOCAL ] RMI: Primary data saved (4709 records). [2025-10-20T20:53:23.530156] [INFO ] [MAIN ] RMI: Full history saved. [2025-10-20T20:53:23.530230] [INFO ] [MAIN ] Update Progress: 1814/2367 symbols completed. RMI took 23.8s. Estimated remaining: 1h 11m 8s. Rate: 7.8 symbols/min [2025-10-20T20:53:23.700965] [INFO ] [MAIN ] RNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:23.745358] [DEBUG] [LOCAL ] RNG: Loaded 48827 existing records from primary storage. [2025-10-20T20:53:23.745702] [INFO ] [MAIN ] RNG: Loaded 48827 existing records. [2025-10-20T20:53:23.923248] [WARN ] [MAIN ] RNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:23.923442] [INFO ] [MAIN ] RNG: Wiping existing primary data before full refresh. [2025-10-20T20:53:23.923488] [INFO ] [LOCAL ] RNG: Starting deletion of primary data. [2025-10-20T20:53:23.923879] [DEBUG] [LOCAL ] RNG: Deleted primary data file. [2025-10-20T20:53:23.923935] [INFO ] [MAIN ] RNG: Starting full primary refresh. [2025-10-20T20:53:24.253938] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-01. [2025-10-20T20:53:24.282319] [INFO ] [REMOTE ] RNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:24.304383] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:53:25.051447] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-01. [2025-10-20T20:53:25.454080] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T20:53:25.851397] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-04. [2025-10-20T20:53:26.461890] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T20:53:26.651422] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-02. [2025-10-20T20:53:26.751735] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T20:53:27.328375] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T20:53:27.451421] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-03. [2025-10-20T20:53:28.251405] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-02. [2025-10-20T20:53:28.311718] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T20:53:29.051411] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-05. [2025-10-20T20:53:29.730579] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T20:53:29.792604] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T20:53:29.851372] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-03. [2025-10-20T20:53:30.651440] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-02. [2025-10-20T20:53:31.172127] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T20:53:31.451347] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-03. [2025-10-20T20:53:31.611049] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T20:53:31.901755] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T20:53:32.251377] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-04. [2025-10-20T20:53:33.051377] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-04. [2025-10-20T20:53:33.316035] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T20:53:33.851459] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-06. [2025-10-20T20:53:34.517430] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T20:53:34.651350] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-04. [2025-10-20T20:53:34.710006] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T20:53:35.359145] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T20:53:35.451349] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-05. [2025-10-20T20:53:36.251349] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-05. [2025-10-20T20:53:36.573337] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T20:53:36.879724] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T20:53:37.051380] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-06. [2025-10-20T20:53:37.701043] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T20:53:37.851396] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-06. [2025-10-20T20:53:38.651455] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-07. [2025-10-20T20:53:38.837977] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T20:53:39.159300] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-07 (0.51 seconds). [2025-10-20T20:53:39.451387] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-07. [2025-10-20T20:53:40.241310] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:53:40.251320] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-08. [2025-10-20T20:53:41.051352] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-07. [2025-10-20T20:53:41.129601] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T20:53:41.851397] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-08. [2025-10-20T20:53:41.975375] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T20:53:42.488354] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T20:53:42.651368] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-08. [2025-10-20T20:53:43.451393] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-09. [2025-10-20T20:53:43.510330] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T20:53:44.084304] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T20:53:44.251346] [INFO ] [REMOTE ] RMMZ: Fetching primary data for 2025-10. [2025-10-20T20:53:44.806456] [INFO ] [REMOTE ] RMMZ: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:53:44.807606] [INFO ] [REMOTE ] RMMZ: Full fetch complete (6662 records). [2025-10-20T20:53:44.807720] [DEBUG] [VALIDATOR ] RMMZ: Validating primary data. [2025-10-20T20:53:45.056587] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-09. [2025-10-20T20:53:45.202863] [DEBUG] [VALIDATOR ] RMMZ: All expected trading days are present. [2025-10-20T20:53:45.202962] [DEBUG] [VALIDATOR ] RMMZ: Primary data validation passed. [2025-10-20T20:53:45.203177] [INFO ] [MAIN ] RMMZ: Fetched primary data is valid. [2025-10-20T20:53:45.203218] [DEBUG] [LOCAL ] RMMZ: Saving primary data. [2025-10-20T20:53:45.203612] [DEBUG] [LOCAL ] RMMZ: Trimmed 6662 records to 4802 (keeping 200 days of data) [2025-10-20T20:53:45.232360] [DEBUG] [LOCAL ] RMMZ: Primary data saved (4802 records). [2025-10-20T20:53:45.232434] [INFO ] [MAIN ] RMMZ: Full history saved. [2025-10-20T20:53:45.232513] [INFO ] [MAIN ] Update Progress: 1815/2367 symbols completed. RMMZ took 26.3s. Estimated remaining: 1h 11m 4s. Rate: 7.8 symbols/min [2025-10-20T20:53:45.403766] [INFO ] [MAIN ] RNGR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:45.425922] [DEBUG] [LOCAL ] RNGR: Loaded 21011 existing records from primary storage. [2025-10-20T20:53:45.426156] [INFO ] [MAIN ] RNGR: Loaded 21011 existing records. [2025-10-20T20:53:45.615826] [WARN ] [MAIN ] RNGR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:45.616038] [INFO ] [MAIN ] RNGR: Wiping existing primary data before full refresh. [2025-10-20T20:53:45.616113] [INFO ] [LOCAL ] RNGR: Starting deletion of primary data. [2025-10-20T20:53:45.616341] [DEBUG] [LOCAL ] RNGR: Deleted primary data file. [2025-10-20T20:53:45.616407] [INFO ] [MAIN ] RNGR: Starting full primary refresh. [2025-10-20T20:53:45.860954] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-09. [2025-10-20T20:53:45.991373] [INFO ] [REMOTE ] RNGR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:46.088444] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T20:53:46.575826] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T20:53:46.651417] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-01. [2025-10-20T20:53:47.451392] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-05. [2025-10-20T20:53:47.855196] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T20:53:48.251381] [INFO ] [REMOTE ] RMT: Fetching primary data for 2025-10. [2025-10-20T20:53:48.923102] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T20:53:48.942344] [INFO ] [REMOTE ] RMT: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:53:48.945559] [INFO ] [REMOTE ] RMT: Full fetch complete (25564 records). [2025-10-20T20:53:48.945630] [DEBUG] [VALIDATOR ] RMT: Validating primary data. [2025-10-20T20:53:49.056559] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-06. [2025-10-20T20:53:49.308183] [DEBUG] [VALIDATOR ] RMT: All expected trading days are present. [2025-10-20T20:53:49.308256] [DEBUG] [VALIDATOR ] RMT: Primary data validation passed. [2025-10-20T20:53:49.308769] [INFO ] [MAIN ] RMT: Fetched primary data is valid. [2025-10-20T20:53:49.308829] [DEBUG] [LOCAL ] RMT: Saving primary data. [2025-10-20T20:53:49.309412] [DEBUG] [LOCAL ] RMT: Trimmed 25564 records to 17390 (keeping 200 days of data) [2025-10-20T20:53:49.409742] [DEBUG] [LOCAL ] RMT: Primary data saved (17390 records). [2025-10-20T20:53:49.409881] [INFO ] [MAIN ] RMT: Full history saved. [2025-10-20T20:53:49.410003] [INFO ] [MAIN ] Update Progress: 1816/2367 symbols completed. RMT took 27.0s. Estimated remaining: 1h 10m 55s. Rate: 7.8 symbols/min [2025-10-20T20:53:49.584001] [INFO ] [MAIN ] RNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:49.599795] [DEBUG] [LOCAL ] RNP: Loaded 16390 existing records from primary storage. [2025-10-20T20:53:49.600018] [INFO ] [MAIN ] RNP: Loaded 16390 existing records. [2025-10-20T20:53:49.778698] [WARN ] [MAIN ] RNP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:49.778939] [INFO ] [MAIN ] RNP: Wiping existing primary data before full refresh. [2025-10-20T20:53:49.778999] [INFO ] [LOCAL ] RNP: Starting deletion of primary data. [2025-10-20T20:53:49.779245] [DEBUG] [LOCAL ] RNP: Deleted primary data file. [2025-10-20T20:53:49.779284] [INFO ] [MAIN ] RNP: Starting full primary refresh. [2025-10-20T20:53:49.856541] [INFO ] [REMOTE ] RMM: Fetching primary data for 2025-10. [2025-10-20T20:53:50.136969] [INFO ] [REMOTE ] RNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:50.651409] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-02. [2025-10-20T20:53:50.699480] [INFO ] [REMOTE ] RMM: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:53:50.702510] [INFO ] [REMOTE ] RMM: Full fetch complete (14814 records). [2025-10-20T20:53:50.702596] [DEBUG] [VALIDATOR ] RMM: Validating primary data. [2025-10-20T20:53:50.876364] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:53:51.127836] [DEBUG] [VALIDATOR ] RMM: All expected trading days are present. [2025-10-20T20:53:51.127954] [DEBUG] [VALIDATOR ] RMM: Primary data validation passed. [2025-10-20T20:53:51.128337] [INFO ] [MAIN ] RMM: Fetched primary data is valid. [2025-10-20T20:53:51.128378] [DEBUG] [LOCAL ] RMM: Saving primary data. [2025-10-20T20:53:51.129008] [DEBUG] [LOCAL ] RMM: Trimmed 14814 records to 10826 (keeping 200 days of data) [2025-10-20T20:53:51.196896] [DEBUG] [LOCAL ] RMM: Primary data saved (10826 records). [2025-10-20T20:53:51.197055] [INFO ] [MAIN ] RMM: Full history saved. [2025-10-20T20:53:51.197139] [INFO ] [MAIN ] Update Progress: 1817/2367 symbols completed. RMM took 35.2s. Estimated remaining: 1h 10m 46s. Rate: 7.8 symbols/min [2025-10-20T20:53:51.366344] [INFO ] [MAIN ] RNR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:53:51.400819] [DEBUG] [LOCAL ] RNR: Loaded 34313 existing records from primary storage. [2025-10-20T20:53:51.401171] [INFO ] [MAIN ] RNR: Loaded 34313 existing records. [2025-10-20T20:53:51.456522] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-07. [2025-10-20T20:53:51.590971] [WARN ] [MAIN ] RNR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:53:51.591841] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T20:53:51.592007] [INFO ] [MAIN ] RNR: Wiping existing primary data before full refresh. [2025-10-20T20:53:51.595309] [INFO ] [LOCAL ] RNR: Starting deletion of primary data. [2025-10-20T20:53:51.595505] [DEBUG] [LOCAL ] RNR: Deleted primary data file. [2025-10-20T20:53:51.595544] [INFO ] [MAIN ] RNR: Starting full primary refresh. [2025-10-20T20:53:51.948638] [INFO ] [REMOTE ] RNR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:53:52.251389] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-01. [2025-10-20T20:53:53.051398] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-01. [2025-10-20T20:53:53.317792] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T20:53:53.845130] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T20:53:53.852278] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-08. [2025-10-20T20:53:54.378499] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T20:53:54.651405] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-02. [2025-10-20T20:53:55.451340] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-02. [2025-10-20T20:53:55.615862] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T20:53:56.191815] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T20:53:56.251331] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-03. [2025-10-20T20:53:56.441788] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T20:53:57.051457] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-03. [2025-10-20T20:53:57.774057] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T20:53:57.851359] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-04. [2025-10-20T20:53:57.903592] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T20:53:58.651416] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-04. [2025-10-20T20:53:59.451351] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-03. [2025-10-20T20:53:59.531068] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T20:53:59.580106] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:54:00.251372] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-05. [2025-10-20T20:54:00.532459] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T20:54:01.051377] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-04. [2025-10-20T20:54:01.710969] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T20:54:01.851362] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-09. [2025-10-20T20:54:02.169510] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T20:54:02.651366] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-05. [2025-10-20T20:54:03.451414] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-05. [2025-10-20T20:54:03.767366] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T20:54:04.251330] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-06. [2025-10-20T20:54:04.432869] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-09 (2.58 seconds). [2025-10-20T20:54:04.957716] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T20:54:05.051363] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-06. [2025-10-20T20:54:05.778928] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-06 (1.53 seconds). [2025-10-20T20:54:05.819194] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T20:54:05.851343] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-07. [2025-10-20T20:54:06.651391] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-06. [2025-10-20T20:54:07.017166] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T20:54:07.451362] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-08. [2025-10-20T20:54:08.036456] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T20:54:08.251355] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-07. [2025-10-20T20:54:08.324415] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-08 (0.87 seconds). [2025-10-20T20:54:09.051375] [INFO ] [REMOTE ] RNG: Fetching primary data for 2025-10. [2025-10-20T20:54:09.828293] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T20:54:09.851339] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-07. [2025-10-20T20:54:10.653499] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-09. [2025-10-20T20:54:10.657746] [INFO ] [REMOTE ] RNG: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T20:54:10.665475] [INFO ] [REMOTE ] RNG: Full fetch complete (69848 records). [2025-10-20T20:54:10.665584] [DEBUG] [VALIDATOR ] RNG: Validating primary data. [2025-10-20T20:54:11.004145] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T20:54:11.082600] [DEBUG] [VALIDATOR ] RNG: All expected trading days are present. [2025-10-20T20:54:11.082705] [DEBUG] [VALIDATOR ] RNG: Primary data validation passed. [2025-10-20T20:54:11.084581] [INFO ] [MAIN ] RNG: Fetched primary data is valid. [2025-10-20T20:54:11.084662] [DEBUG] [LOCAL ] RNG: Saving primary data. [2025-10-20T20:54:11.085954] [DEBUG] [LOCAL ] RNG: Trimmed 69848 records to 48033 (keeping 200 days of data) [2025-10-20T20:54:11.366876] [DEBUG] [LOCAL ] RNG: Primary data saved (48033 records). [2025-10-20T20:54:11.367041] [INFO ] [MAIN ] RNG: Full history saved. [2025-10-20T20:54:11.367129] [INFO ] [MAIN ] Update Progress: 1818/2367 symbols completed. RNG took 47.8s. Estimated remaining: 1h 10m 42s. Rate: 7.8 symbols/min [2025-10-20T20:54:11.454332] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-08. [2025-10-20T20:54:11.591780] [INFO ] [MAIN ] RNST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:11.636732] [DEBUG] [LOCAL ] RNST: Loaded 36994 existing records from primary storage. [2025-10-20T20:54:11.637167] [INFO ] [MAIN ] RNST: Loaded 36994 existing records. [2025-10-20T20:54:11.774040] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T20:54:11.924784] [WARN ] [MAIN ] RNST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:11.925087] [INFO ] [MAIN ] RNST: Wiping existing primary data before full refresh. [2025-10-20T20:54:11.925159] [INFO ] [LOCAL ] RNST: Starting deletion of primary data. [2025-10-20T20:54:11.925439] [DEBUG] [LOCAL ] RNST: Deleted primary data file. [2025-10-20T20:54:11.925530] [INFO ] [MAIN ] RNST: Starting full primary refresh. [2025-10-20T20:54:12.256633] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-08. [2025-10-20T20:54:12.402361] [INFO ] [REMOTE ] RNST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:12.645476] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T20:54:13.051372] [INFO ] [REMOTE ] RNP: Fetching primary data for 2025-10. [2025-10-20T20:54:13.366101] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T20:54:13.851399] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-09. [2025-10-20T20:54:14.027390] [INFO ] [REMOTE ] RNP: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T20:54:14.030625] [INFO ] [REMOTE ] RNP: Full fetch complete (23587 records). [2025-10-20T20:54:14.030732] [DEBUG] [VALIDATOR ] RNP: Validating primary data. [2025-10-20T20:54:14.386927] [DEBUG] [VALIDATOR ] RNP: All expected trading days are present. [2025-10-20T20:54:14.387004] [DEBUG] [VALIDATOR ] RNP: Primary data validation passed. [2025-10-20T20:54:14.387504] [INFO ] [MAIN ] RNP: Fetched primary data is valid. [2025-10-20T20:54:14.387543] [DEBUG] [LOCAL ] RNP: Saving primary data. [2025-10-20T20:54:14.388347] [DEBUG] [LOCAL ] RNP: Trimmed 23587 records to 16118 (keeping 200 days of data) [2025-10-20T20:54:14.484084] [DEBUG] [LOCAL ] RNP: Primary data saved (16118 records). [2025-10-20T20:54:14.484215] [INFO ] [MAIN ] RNP: Full history saved. [2025-10-20T20:54:14.484289] [INFO ] [MAIN ] Update Progress: 1819/2367 symbols completed. RNP took 25.1s. Estimated remaining: 1h 10m 33s. Rate: 7.8 symbols/min [2025-10-20T20:54:14.651259] [INFO ] [MAIN ] ROG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:14.651430] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-09. [2025-10-20T20:54:14.677195] [DEBUG] [LOCAL ] ROG: Loaded 22472 existing records from primary storage. [2025-10-20T20:54:14.677425] [INFO ] [MAIN ] ROG: Loaded 22472 existing records. [2025-10-20T20:54:14.851470] [WARN ] [MAIN ] ROG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:14.851695] [INFO ] [MAIN ] ROG: Wiping existing primary data before full refresh. [2025-10-20T20:54:14.851754] [INFO ] [LOCAL ] ROG: Starting deletion of primary data. [2025-10-20T20:54:14.851948] [DEBUG] [LOCAL ] ROG: Deleted primary data file. [2025-10-20T20:54:14.852002] [INFO ] [MAIN ] ROG: Starting full primary refresh. [2025-10-20T20:54:15.192235] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T20:54:15.225104] [INFO ] [REMOTE ] ROG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:15.451387] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-01. [2025-10-20T20:54:15.995928] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T20:54:16.251383] [INFO ] [REMOTE ] RNR: Fetching primary data for 2025-10. [2025-10-20T20:54:16.690291] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T20:54:16.929035] [INFO ] [REMOTE ] RNR: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:54:16.933891] [INFO ] [REMOTE ] RNR: Full fetch complete (50918 records). [2025-10-20T20:54:16.933987] [DEBUG] [VALIDATOR ] RNR: Validating primary data. [2025-10-20T20:54:16.934603] [FATAL] [VALIDATOR ] RNR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:54:16.934711] [FATAL] [MAIN ] RNR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:54:16.934776] [WARN ] [MAIN ] RNR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:54:16.934816] [INFO ] [LOCAL ] RNR: Starting deletion of primary data. [2025-10-20T20:54:16.934872] [DEBUG] [LOCAL ] RNR: No primary data file found to delete. [2025-10-20T20:54:16.934906] [INFO ] [LOCAL ] RNR: Starting deletion of derivative data. [2025-10-20T20:54:16.935064] [DEBUG] [LOCAL ] RNR: Deleted derivative data file. [2025-10-20T20:54:16.935151] [INFO ] [MAIN ] Update Progress: 1820/2367 symbols completed. RNR took 25.7s. Estimated remaining: 1h 10m 23s. Rate: 7.8 symbols/min [2025-10-20T20:54:17.056552] [INFO ] [REMOTE ] RNGR: Fetching primary data for 2025-10. [2025-10-20T20:54:17.101662] [INFO ] [MAIN ] ROK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:17.142305] [DEBUG] [LOCAL ] ROK: Loaded 43878 existing records from primary storage. [2025-10-20T20:54:17.142599] [INFO ] [MAIN ] ROK: Loaded 43878 existing records. [2025-10-20T20:54:17.313160] [WARN ] [MAIN ] ROK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:17.313389] [INFO ] [MAIN ] ROK: Wiping existing primary data before full refresh. [2025-10-20T20:54:17.313436] [INFO ] [LOCAL ] ROK: Starting deletion of primary data. [2025-10-20T20:54:17.313687] [DEBUG] [LOCAL ] ROK: Deleted primary data file. [2025-10-20T20:54:17.313742] [INFO ] [MAIN ] ROK: Starting full primary refresh. [2025-10-20T20:54:17.655856] [INFO ] [REMOTE ] ROK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:17.851374] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-01. [2025-10-20T20:54:18.173921] [INFO ] [REMOTE ] RNGR: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T20:54:18.177504] [INFO ] [REMOTE ] RNGR: Full fetch complete (28529 records). [2025-10-20T20:54:18.177617] [DEBUG] [VALIDATOR ] RNGR: Validating primary data. [2025-10-20T20:54:18.571021] [DEBUG] [VALIDATOR ] RNGR: All expected trading days are present. [2025-10-20T20:54:18.571121] [DEBUG] [VALIDATOR ] RNGR: Primary data validation passed. [2025-10-20T20:54:18.571815] [INFO ] [MAIN ] RNGR: Fetched primary data is valid. [2025-10-20T20:54:18.571876] [DEBUG] [LOCAL ] RNGR: Saving primary data. [2025-10-20T20:54:18.572692] [DEBUG] [LOCAL ] RNGR: Trimmed 28529 records to 20833 (keeping 200 days of data) [2025-10-20T20:54:18.659338] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-02. [2025-10-20T20:54:18.701320] [DEBUG] [LOCAL ] RNGR: Primary data saved (20833 records). [2025-10-20T20:54:18.701446] [INFO ] [MAIN ] RNGR: Full history saved. [2025-10-20T20:54:18.701537] [INFO ] [MAIN ] Update Progress: 1821/2367 symbols completed. RNGR took 33.5s. Estimated remaining: 1h 10m 14s. Rate: 7.8 symbols/min [2025-10-20T20:54:18.873964] [INFO ] [MAIN ] ROL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:18.922479] [DEBUG] [LOCAL ] ROL: Loaded 51102 existing records from primary storage. [2025-10-20T20:54:18.922821] [INFO ] [MAIN ] ROL: Loaded 51102 existing records. [2025-10-20T20:54:19.106052] [WARN ] [MAIN ] ROL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:19.106259] [INFO ] [MAIN ] ROL: Wiping existing primary data before full refresh. [2025-10-20T20:54:19.106308] [INFO ] [LOCAL ] ROL: Starting deletion of primary data. [2025-10-20T20:54:19.106499] [DEBUG] [LOCAL ] ROL: Deleted primary data file. [2025-10-20T20:54:19.106539] [INFO ] [MAIN ] ROL: Starting full primary refresh. [2025-10-20T20:54:19.458848] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-01. [2025-10-20T20:54:19.480289] [INFO ] [REMOTE ] ROL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:19.505908] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T20:54:19.620720] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T20:54:20.251394] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-01. [2025-10-20T20:54:21.056582] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-03. [2025-10-20T20:54:21.069802] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T20:54:21.851393] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-02. [2025-10-20T20:54:21.970536] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T20:54:22.388273] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T20:54:22.651403] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-04. [2025-10-20T20:54:23.451396] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-02. [2025-10-20T20:54:23.584692] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:54:24.194910] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T20:54:24.251374] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-05. [2025-10-20T20:54:24.567138] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T20:54:25.051349] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-03. [2025-10-20T20:54:25.414441] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:54:25.851382] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-03. [2025-10-20T20:54:26.605629] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T20:54:26.651340] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-04. [2025-10-20T20:54:27.451402] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-06. [2025-10-20T20:54:27.749824] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T20:54:28.251381] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-04. [2025-10-20T20:54:28.477615] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:54:28.502904] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T20:54:29.051417] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-05. [2025-10-20T20:54:29.851345] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-07. [2025-10-20T20:54:30.112774] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T20:54:30.616055] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-05 (1.56 seconds). [2025-10-20T20:54:30.651338] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-06. [2025-10-20T20:54:31.118757] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T20:54:31.451390] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-05. [2025-10-20T20:54:32.069324] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T20:54:32.251396] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-08. [2025-10-20T20:54:33.051387] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-07. [2025-10-20T20:54:33.195188] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T20:54:33.502498] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T20:54:33.851416] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-09. [2025-10-20T20:54:34.529488] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T20:54:34.651345] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-02. [2025-10-20T20:54:34.888446] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T20:54:35.451408] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-06. [2025-10-20T20:54:36.251384] [INFO ] [REMOTE ] ROG: Fetching primary data for 2025-10. [2025-10-20T20:54:36.315438] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T20:54:36.812568] [INFO ] [REMOTE ] ROG: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:54:36.815239] [INFO ] [REMOTE ] ROG: Full fetch complete (30667 records). [2025-10-20T20:54:36.815316] [DEBUG] [VALIDATOR ] ROG: Validating primary data. [2025-10-20T20:54:36.815800] [FATAL] [VALIDATOR ] ROG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:54:36.815902] [FATAL] [MAIN ] ROG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:54:36.815944] [WARN ] [MAIN ] ROG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:54:36.816002] [INFO ] [LOCAL ] ROG: Starting deletion of primary data. [2025-10-20T20:54:36.816056] [DEBUG] [LOCAL ] ROG: No primary data file found to delete. [2025-10-20T20:54:36.816091] [INFO ] [LOCAL ] ROG: Starting deletion of derivative data. [2025-10-20T20:54:36.816227] [DEBUG] [LOCAL ] ROG: Deleted derivative data file. [2025-10-20T20:54:36.816311] [INFO ] [MAIN ] Update Progress: 1822/2367 symbols completed. ROG took 22.3s. Estimated remaining: 1h 10m 9s. Rate: 7.8 symbols/min [2025-10-20T20:54:36.994186] [INFO ] [MAIN ] RPM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:37.026749] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T20:54:37.058483] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-03. [2025-10-20T20:54:37.079549] [DEBUG] [LOCAL ] RPM: Loaded 42595 existing records from primary storage. [2025-10-20T20:54:37.080024] [INFO ] [MAIN ] RPM: Loaded 42595 existing records. [2025-10-20T20:54:37.258190] [WARN ] [MAIN ] RPM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:37.258598] [INFO ] [MAIN ] RPM: Wiping existing primary data before full refresh. [2025-10-20T20:54:37.258686] [INFO ] [LOCAL ] RPM: Starting deletion of primary data. [2025-10-20T20:54:37.259336] [DEBUG] [LOCAL ] RPM: Deleted primary data file. [2025-10-20T20:54:37.259396] [INFO ] [MAIN ] RPM: Starting full primary refresh. [2025-10-20T20:54:37.613900] [INFO ] [REMOTE ] RPM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:37.851391] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-08. [2025-10-20T20:54:38.540245] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T20:54:38.651339] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-04. [2025-10-20T20:54:39.374791] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T20:54:39.451339] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-09. [2025-10-20T20:54:40.251430] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-01. [2025-10-20T20:54:40.485620] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T20:54:40.921060] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T20:54:41.051359] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-05. [2025-10-20T20:54:41.851346] [INFO ] [REMOTE ] RNST: Fetching primary data for 2025-10. [2025-10-20T20:54:41.946467] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T20:54:42.417930] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T20:54:42.538762] [INFO ] [REMOTE ] RNST: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T20:54:42.543805] [INFO ] [REMOTE ] RNST: Full fetch complete (52647 records). [2025-10-20T20:54:42.543960] [DEBUG] [VALIDATOR ] RNST: Validating primary data. [2025-10-20T20:54:42.544570] [FATAL] [VALIDATOR ] RNST: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:54:42.544700] [FATAL] [MAIN ] RNST: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:54:42.544765] [WARN ] [MAIN ] RNST: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:54:42.544851] [INFO ] [LOCAL ] RNST: Starting deletion of primary data. [2025-10-20T20:54:42.544927] [DEBUG] [LOCAL ] RNST: No primary data file found to delete. [2025-10-20T20:54:42.544976] [INFO ] [LOCAL ] RNST: Starting deletion of derivative data. [2025-10-20T20:54:42.545148] [DEBUG] [LOCAL ] RNST: Deleted derivative data file. [2025-10-20T20:54:42.545260] [INFO ] [MAIN ] Update Progress: 1823/2367 symbols completed. RNST took 31.2s. Estimated remaining: 1h 10m 1s. Rate: 7.8 symbols/min [2025-10-20T20:54:42.656618] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-07. [2025-10-20T20:54:42.718639] [INFO ] [MAIN ] RPT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:42.738392] [DEBUG] [LOCAL ] RPT: Loaded 19322 existing records from primary storage. [2025-10-20T20:54:42.738696] [INFO ] [MAIN ] RPT: Loaded 19322 existing records. [2025-10-20T20:54:42.911635] [WARN ] [MAIN ] RPT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:42.911893] [INFO ] [MAIN ] RPT: Wiping existing primary data before full refresh. [2025-10-20T20:54:42.911961] [INFO ] [LOCAL ] RPT: Starting deletion of primary data. [2025-10-20T20:54:42.912182] [DEBUG] [LOCAL ] RPT: Deleted primary data file. [2025-10-20T20:54:42.912240] [INFO ] [MAIN ] RPT: Starting full primary refresh. [2025-10-20T20:54:43.260493] [INFO ] [REMOTE ] RPT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:43.451376] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-01. [2025-10-20T20:54:44.251357] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-02. [2025-10-20T20:54:44.454953] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T20:54:44.482748] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-01 (1.03 seconds). [2025-10-20T20:54:45.051365] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-08. [2025-10-20T20:54:45.534918] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T20:54:45.851358] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-03. [2025-10-20T20:54:46.435434] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T20:54:46.651400] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-06. [2025-10-20T20:54:47.451426] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-02. [2025-10-20T20:54:47.573409] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T20:54:48.104213] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T20:54:48.251341] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-07. [2025-10-20T20:54:48.527930] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-02 (1.08 seconds). [2025-10-20T20:54:49.051377] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-03. [2025-10-20T20:54:49.851348] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-09. [2025-10-20T20:54:50.047352] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T20:54:50.125667] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T20:54:50.651426] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-08. [2025-10-20T20:54:51.335258] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T20:54:51.451353] [INFO ] [REMOTE ] ROK: Fetching primary data for 2025-10. [2025-10-20T20:54:52.134132] [INFO ] [REMOTE ] ROK: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T20:54:52.140436] [INFO ] [REMOTE ] ROK: Full fetch complete (62525 records). [2025-10-20T20:54:52.140597] [DEBUG] [VALIDATOR ] ROK: Validating primary data. [2025-10-20T20:54:52.141380] [FATAL] [VALIDATOR ] ROK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:54:52.141468] [FATAL] [MAIN ] ROK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:54:52.141558] [WARN ] [MAIN ] ROK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:54:52.141687] [INFO ] [LOCAL ] ROK: Starting deletion of primary data. [2025-10-20T20:54:52.141947] [DEBUG] [LOCAL ] ROK: No primary data file found to delete. [2025-10-20T20:54:52.142144] [INFO ] [LOCAL ] ROK: Starting deletion of derivative data. [2025-10-20T20:54:52.142283] [DEBUG] [LOCAL ] ROK: Deleted derivative data file. [2025-10-20T20:54:52.142411] [INFO ] [MAIN ] Update Progress: 1824/2367 symbols completed. ROK took 35.2s. Estimated remaining: 1h 9m 54s. Rate: 7.8 symbols/min [2025-10-20T20:54:52.256465] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-04. [2025-10-20T20:54:52.316939] [INFO ] [MAIN ] RQI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:52.348631] [DEBUG] [LOCAL ] RQI: Loaded 29541 existing records from primary storage. [2025-10-20T20:54:52.349010] [INFO ] [MAIN ] RQI: Loaded 29541 existing records. [2025-10-20T20:54:52.562396] [WARN ] [MAIN ] RQI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:52.563338] [INFO ] [MAIN ] RQI: Wiping existing primary data before full refresh. [2025-10-20T20:54:52.565149] [INFO ] [LOCAL ] RQI: Starting deletion of primary data. [2025-10-20T20:54:52.565003] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T20:54:52.574949] [DEBUG] [LOCAL ] RQI: Deleted primary data file. [2025-10-20T20:54:52.575567] [INFO ] [MAIN ] RQI: Starting full primary refresh. [2025-10-20T20:54:52.970883] [INFO ] [REMOTE ] RQI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:53.051462] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-01. [2025-10-20T20:54:53.851414] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-04. [2025-10-20T20:54:53.983222] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T20:54:54.651798] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:54:54.651902] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-09. [2025-10-20T20:54:54.673624] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T20:54:55.451412] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-05. [2025-10-20T20:54:56.251424] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-05. [2025-10-20T20:54:56.503545] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-09 (1.85 seconds). [2025-10-20T20:54:56.518630] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T20:54:57.051367] [INFO ] [REMOTE ] ROL: Fetching primary data for 2025-10. [2025-10-20T20:54:57.600313] [INFO ] [REMOTE ] ROL: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:54:57.607554] [INFO ] [REMOTE ] ROL: Full fetch complete (72878 records). [2025-10-20T20:54:57.607721] [DEBUG] [VALIDATOR ] ROL: Validating primary data. [2025-10-20T20:54:57.608587] [FATAL] [VALIDATOR ] ROL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:54:57.608742] [FATAL] [MAIN ] ROL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:54:57.608798] [WARN ] [MAIN ] ROL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:54:57.608865] [INFO ] [LOCAL ] ROL: Starting deletion of primary data. [2025-10-20T20:54:57.608933] [DEBUG] [LOCAL ] ROL: No primary data file found to delete. [2025-10-20T20:54:57.608970] [INFO ] [LOCAL ] ROL: Starting deletion of derivative data. [2025-10-20T20:54:57.609107] [DEBUG] [LOCAL ] ROL: Deleted derivative data file. [2025-10-20T20:54:57.609517] [INFO ] [MAIN ] Update Progress: 1825/2367 symbols completed. ROL took 38.9s. Estimated remaining: 1h 9m 45s. Rate: 7.8 symbols/min [2025-10-20T20:54:57.791264] [INFO ] [MAIN ] RRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:54:57.826717] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T20:54:57.859862] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-06. [2025-10-20T20:54:57.883987] [DEBUG] [LOCAL ] RRC: Loaded 53680 existing records from primary storage. [2025-10-20T20:54:57.884320] [INFO ] [MAIN ] RRC: Loaded 53680 existing records. [2025-10-20T20:54:58.065829] [WARN ] [MAIN ] RRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:54:58.066081] [INFO ] [MAIN ] RRC: Wiping existing primary data before full refresh. [2025-10-20T20:54:58.066129] [INFO ] [LOCAL ] RRC: Starting deletion of primary data. [2025-10-20T20:54:58.066382] [DEBUG] [LOCAL ] RRC: Deleted primary data file. [2025-10-20T20:54:58.066422] [INFO ] [MAIN ] RRC: Starting full primary refresh. [2025-10-20T20:54:58.428320] [INFO ] [REMOTE ] RRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:54:58.651424] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-02. [2025-10-20T20:54:59.067172] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T20:54:59.451376] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-07. [2025-10-20T20:54:59.954768] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T20:55:00.142250] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T20:55:00.251428] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-01. [2025-10-20T20:55:01.051367] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-03. [2025-10-20T20:55:01.851447] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-06. [2025-10-20T20:55:02.183415] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-03 (1.13 seconds). [2025-10-20T20:55:02.293230] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-01 (2.04 seconds). [2025-10-20T20:55:02.651416] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-04. [2025-10-20T20:55:03.460529] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-02. [2025-10-20T20:55:03.463899] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T20:55:03.833337] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T20:55:04.251420] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-08. [2025-10-20T20:55:05.051487] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-05. [2025-10-20T20:55:05.290753] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T20:55:05.506071] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T20:55:05.851375] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-03. [2025-10-20T20:55:06.369395] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T20:55:06.651420] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-06. [2025-10-20T20:55:07.428926] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T20:55:07.451349] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-04. [2025-10-20T20:55:07.873002] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T20:55:08.251409] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-09. [2025-10-20T20:55:09.051358] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-07. [2025-10-20T20:55:09.272284] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T20:55:09.450841] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T20:55:09.851377] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-07. [2025-10-20T20:55:10.484765] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-07 (1.43 seconds). [2025-10-20T20:55:10.651371] [INFO ] [REMOTE ] RPT: Fetching primary data for 2025-10. [2025-10-20T20:55:11.451368] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-05. [2025-10-20T20:55:11.541927] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T20:55:11.645047] [INFO ] [REMOTE ] RPT: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T20:55:11.648359] [INFO ] [REMOTE ] RPT: Full fetch complete (28266 records). [2025-10-20T20:55:11.648444] [DEBUG] [VALIDATOR ] RPT: Validating primary data. [2025-10-20T20:55:12.043465] [DEBUG] [VALIDATOR ] RPT: All expected trading days are present. [2025-10-20T20:55:12.043545] [DEBUG] [VALIDATOR ] RPT: Primary data validation passed. [2025-10-20T20:55:12.044323] [INFO ] [MAIN ] RPT: Fetched primary data is valid. [2025-10-20T20:55:12.044387] [DEBUG] [LOCAL ] RPT: Saving primary data. [2025-10-20T20:55:12.045111] [DEBUG] [LOCAL ] RPT: Trimmed 28266 records to 19109 (keeping 200 days of data) [2025-10-20T20:55:12.152895] [DEBUG] [LOCAL ] RPT: Primary data saved (19109 records). [2025-10-20T20:55:12.152999] [INFO ] [MAIN ] RPT: Full history saved. [2025-10-20T20:55:12.153079] [INFO ] [MAIN ] Update Progress: 1826/2367 symbols completed. RPT took 29.6s. Estimated remaining: 1h 9m 40s. Rate: 7.8 symbols/min [2025-10-20T20:55:12.255116] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-08. [2025-10-20T20:55:12.326964] [INFO ] [MAIN ] RRX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:12.367343] [DEBUG] [LOCAL ] RRX: Loaded 41654 existing records from primary storage. [2025-10-20T20:55:12.367713] [INFO ] [MAIN ] RRX: Loaded 41654 existing records. [2025-10-20T20:55:12.559254] [WARN ] [MAIN ] RRX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:12.559502] [INFO ] [MAIN ] RRX: Wiping existing primary data before full refresh. [2025-10-20T20:55:12.559567] [INFO ] [LOCAL ] RRX: Starting deletion of primary data. [2025-10-20T20:55:12.559845] [DEBUG] [LOCAL ] RRX: Deleted primary data file. [2025-10-20T20:55:12.559909] [INFO ] [MAIN ] RRX: Starting full primary refresh. [2025-10-20T20:55:12.949599] [INFO ] [REMOTE ] RRX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:13.013229] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T20:55:13.051445] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-09. [2025-10-20T20:55:13.265259] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T20:55:13.765338] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T20:55:13.851358] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-06. [2025-10-20T20:55:14.651373] [INFO ] [REMOTE ] RPM: Fetching primary data for 2025-10. [2025-10-20T20:55:15.319463] [INFO ] [REMOTE ] RPM: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T20:55:15.325586] [INFO ] [REMOTE ] RPM: Full fetch complete (59628 records). [2025-10-20T20:55:15.325717] [DEBUG] [VALIDATOR ] RPM: Validating primary data. [2025-10-20T20:55:15.326347] [FATAL] [VALIDATOR ] RPM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:55:15.326432] [FATAL] [MAIN ] RPM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:55:15.326474] [WARN ] [MAIN ] RPM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:55:15.326510] [INFO ] [LOCAL ] RPM: Starting deletion of primary data. [2025-10-20T20:55:15.326564] [DEBUG] [LOCAL ] RPM: No primary data file found to delete. [2025-10-20T20:55:15.326599] [INFO ] [LOCAL ] RPM: Starting deletion of derivative data. [2025-10-20T20:55:15.326741] [DEBUG] [LOCAL ] RPM: Deleted derivative data file. [2025-10-20T20:55:15.326835] [INFO ] [MAIN ] Update Progress: 1827/2367 symbols completed. RPM took 38.5s. Estimated remaining: 1h 9m 31s. Rate: 7.8 symbols/min [2025-10-20T20:55:15.455007] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-01. [2025-10-20T20:55:15.498672] [INFO ] [MAIN ] RS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:15.523850] [DEBUG] [LOCAL ] RS: Loaded 25141 existing records from primary storage. [2025-10-20T20:55:15.524137] [INFO ] [MAIN ] RS: Loaded 25141 existing records. [2025-10-20T20:55:15.669446] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T20:55:15.741448] [WARN ] [MAIN ] RS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:15.741709] [INFO ] [MAIN ] RS: Wiping existing primary data before full refresh. [2025-10-20T20:55:15.741766] [INFO ] [LOCAL ] RS: Starting deletion of primary data. [2025-10-20T20:55:15.741942] [DEBUG] [LOCAL ] RS: Deleted primary data file. [2025-10-20T20:55:15.741980] [INFO ] [MAIN ] RS: Starting full primary refresh. [2025-10-20T20:55:16.099321] [INFO ] [REMOTE ] RS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:16.251373] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-08. [2025-10-20T20:55:17.051432] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-07. [2025-10-20T20:55:17.057000] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:55:17.302864] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T20:55:17.851367] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-09. [2025-10-20T20:55:18.651427] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-02. [2025-10-20T20:55:19.038755] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T20:55:19.070573] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T20:55:19.451358] [INFO ] [REMOTE ] RQI: Fetching primary data for 2025-10. [2025-10-20T20:55:20.180456] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T20:55:20.251348] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-03. [2025-10-20T20:55:20.295629] [INFO ] [REMOTE ] RQI: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T20:55:20.299954] [INFO ] [REMOTE ] RQI: Full fetch complete (42849 records). [2025-10-20T20:55:20.300048] [DEBUG] [VALIDATOR ] RQI: Validating primary data. [2025-10-20T20:55:20.678757] [DEBUG] [VALIDATOR ] RQI: All expected trading days are present. [2025-10-20T20:55:20.678844] [DEBUG] [VALIDATOR ] RQI: Primary data validation passed. [2025-10-20T20:55:20.679626] [INFO ] [MAIN ] RQI: Fetched primary data is valid. [2025-10-20T20:55:20.679732] [DEBUG] [LOCAL ] RQI: Saving primary data. [2025-10-20T20:55:20.680638] [DEBUG] [LOCAL ] RQI: Trimmed 42849 records to 29007 (keeping 200 days of data) [2025-10-20T20:55:20.854067] [DEBUG] [LOCAL ] RQI: Primary data saved (29007 records). [2025-10-20T20:55:20.854194] [INFO ] [MAIN ] RQI: Full history saved. [2025-10-20T20:55:20.854269] [INFO ] [MAIN ] Update Progress: 1828/2367 symbols completed. RQI took 28.7s. Estimated remaining: 1h 9m 22s. Rate: 7.8 symbols/min [2025-10-20T20:55:21.025124] [INFO ] [MAIN ] RSF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:21.031440] [DEBUG] [LOCAL ] RSF: Loaded 4371 existing records from primary storage. [2025-10-20T20:55:21.031609] [INFO ] [MAIN ] RSF: Loaded 4371 existing records. [2025-10-20T20:55:21.062162] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-01. [2025-10-20T20:55:21.210838] [WARN ] [MAIN ] RSF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:21.210995] [INFO ] [MAIN ] RSF: Wiping existing primary data before full refresh. [2025-10-20T20:55:21.211042] [INFO ] [LOCAL ] RSF: Starting deletion of primary data. [2025-10-20T20:55:21.211161] [DEBUG] [LOCAL ] RSF: Deleted primary data file. [2025-10-20T20:55:21.211198] [INFO ] [MAIN ] RSF: Starting full primary refresh. [2025-10-20T20:55:21.565284] [INFO ] [REMOTE ] RSF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:21.679835] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T20:55:21.851373] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-04. [2025-10-20T20:55:22.656288] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-01. [2025-10-20T20:55:22.660327] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T20:55:23.228412] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T20:55:23.275232] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T20:55:23.451396] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-02. [2025-10-20T20:55:24.251354] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-08. [2025-10-20T20:55:24.924241] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T20:55:25.051426] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-02. [2025-10-20T20:55:25.669164] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T20:55:25.851354] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-03. [2025-10-20T20:55:26.003114] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T20:55:26.256143] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T20:55:26.651370] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-04. [2025-10-20T20:55:27.132829] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T20:55:27.451432] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-05. [2025-10-20T20:55:28.173741] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T20:55:28.251421] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-03. [2025-10-20T20:55:29.051381] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-06. [2025-10-20T20:55:29.664723] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T20:55:29.678534] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:55:29.851373] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-04. [2025-10-20T20:55:30.651432] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-05. [2025-10-20T20:55:31.294453] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T20:55:31.451376] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-09. [2025-10-20T20:55:32.251392] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-07. [2025-10-20T20:55:32.335555] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T20:55:32.983252] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T20:55:33.051345] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-06. [2025-10-20T20:55:33.259362] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T20:55:33.851367] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-05. [2025-10-20T20:55:34.567393] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T20:55:34.651393] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-07. [2025-10-20T20:55:34.870739] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T20:55:35.451406] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-08. [2025-10-20T20:55:36.079329] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-08 (0.63 seconds). [2025-10-20T20:55:36.251467] [INFO ] [REMOTE ] RRC: Fetching primary data for 2025-10. [2025-10-20T20:55:36.317258] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T20:55:36.891562] [INFO ] [REMOTE ] RRC: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T20:55:36.899047] [INFO ] [REMOTE ] RRC: Full fetch complete (76360 records). [2025-10-20T20:55:36.899175] [DEBUG] [VALIDATOR ] RRC: Validating primary data. [2025-10-20T20:55:36.899925] [FATAL] [VALIDATOR ] RRC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:55:36.900012] [FATAL] [MAIN ] RRC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:55:36.900054] [WARN ] [MAIN ] RRC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:55:36.900092] [INFO ] [LOCAL ] RRC: Starting deletion of primary data. [2025-10-20T20:55:36.900146] [DEBUG] [LOCAL ] RRC: No primary data file found to delete. [2025-10-20T20:55:36.900180] [INFO ] [LOCAL ] RRC: Starting deletion of derivative data. [2025-10-20T20:55:36.900299] [DEBUG] [LOCAL ] RRC: Deleted derivative data file. [2025-10-20T20:55:36.900687] [INFO ] [MAIN ] Update Progress: 1829/2367 symbols completed. RRC took 39.3s. Estimated remaining: 1h 9m 17s. Rate: 7.8 symbols/min [2025-10-20T20:55:37.060160] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-09. [2025-10-20T20:55:37.092400] [INFO ] [MAIN ] RSG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:37.146522] [DEBUG] [LOCAL ] RSG: Loaded 48206 existing records from primary storage. [2025-10-20T20:55:37.146957] [INFO ] [MAIN ] RSG: Loaded 48206 existing records. [2025-10-20T20:55:37.336389] [WARN ] [MAIN ] RSG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:37.336694] [INFO ] [MAIN ] RSG: Wiping existing primary data before full refresh. [2025-10-20T20:55:37.336757] [INFO ] [LOCAL ] RSG: Starting deletion of primary data. [2025-10-20T20:55:37.337203] [DEBUG] [LOCAL ] RSG: Deleted primary data file. [2025-10-20T20:55:37.337488] [INFO ] [MAIN ] RSG: Starting full primary refresh. [2025-10-20T20:55:37.720312] [INFO ] [REMOTE ] RSG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:37.721206] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T20:55:37.851398] [INFO ] [REMOTE ] RSF: Fetching primary data for 2025-10. [2025-10-20T20:55:38.304582] [INFO ] [REMOTE ] RSF: Incremental fetch complete for 2025-10 (0.45 seconds). [2025-10-20T20:55:38.305791] [INFO ] [REMOTE ] RSF: Full fetch complete (5689 records). [2025-10-20T20:55:38.305860] [DEBUG] [VALIDATOR ] RSF: Validating primary data. [2025-10-20T20:55:38.658713] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-01. [2025-10-20T20:55:38.683241] [DEBUG] [VALIDATOR ] RSF: All expected trading days are present. [2025-10-20T20:55:38.683326] [DEBUG] [VALIDATOR ] RSF: Primary data validation passed. [2025-10-20T20:55:38.683492] [INFO ] [MAIN ] RSF: Fetched primary data is valid. [2025-10-20T20:55:38.683532] [DEBUG] [LOCAL ] RSF: Saving primary data. [2025-10-20T20:55:38.683995] [DEBUG] [LOCAL ] RSF: Trimmed 5689 records to 4311 (keeping 200 days of data) [2025-10-20T20:55:38.712165] [DEBUG] [LOCAL ] RSF: Primary data saved (4311 records). [2025-10-20T20:55:38.712324] [INFO ] [MAIN ] RSF: Full history saved. [2025-10-20T20:55:38.712411] [INFO ] [MAIN ] Update Progress: 1830/2367 symbols completed. RSF took 17.9s. Estimated remaining: 1h 9m 8s. Rate: 7.8 symbols/min [2025-10-20T20:55:38.884148] [INFO ] [MAIN ] RSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:38.929042] [DEBUG] [LOCAL ] RSI: Loaded 47708 existing records from primary storage. [2025-10-20T20:55:38.929412] [INFO ] [MAIN ] RSI: Loaded 47708 existing records. [2025-10-20T20:55:39.109766] [WARN ] [MAIN ] RSI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:39.109986] [INFO ] [MAIN ] RSI: Wiping existing primary data before full refresh. [2025-10-20T20:55:39.110034] [INFO ] [LOCAL ] RSI: Starting deletion of primary data. [2025-10-20T20:55:39.110262] [DEBUG] [LOCAL ] RSI: Deleted primary data file. [2025-10-20T20:55:39.110321] [INFO ] [MAIN ] RSI: Starting full primary refresh. [2025-10-20T20:55:39.461021] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-08. [2025-10-20T20:55:39.467796] [INFO ] [REMOTE ] RSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:40.251348] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-06. [2025-10-20T20:55:40.433408] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T20:55:41.057971] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-01. [2025-10-20T20:55:41.073938] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T20:55:41.535402] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-06 (1.28 seconds). [2025-10-20T20:55:41.851433] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-09. [2025-10-20T20:55:42.651400] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-02. [2025-10-20T20:55:42.818033] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T20:55:43.451385] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-02. [2025-10-20T20:55:43.521351] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T20:55:44.251415] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-07. [2025-10-20T20:55:44.380597] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T20:55:44.987428] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T20:55:45.051374] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-03. [2025-10-20T20:55:45.632391] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T20:55:45.851405] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-03. [2025-10-20T20:55:46.651360] [INFO ] [REMOTE ] RRX: Fetching primary data for 2025-10. [2025-10-20T20:55:46.905020] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T20:55:47.199026] [INFO ] [REMOTE ] RRX: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T20:55:47.207290] [INFO ] [REMOTE ] RRX: Full fetch complete (58562 records). [2025-10-20T20:55:47.207464] [DEBUG] [VALIDATOR ] RRX: Validating primary data. [2025-10-20T20:55:47.208318] [FATAL] [VALIDATOR ] RRX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:55:47.208430] [FATAL] [MAIN ] RRX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:55:47.208486] [WARN ] [MAIN ] RRX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:55:47.208541] [INFO ] [LOCAL ] RRX: Starting deletion of primary data. [2025-10-20T20:55:47.208618] [DEBUG] [LOCAL ] RRX: No primary data file found to delete. [2025-10-20T20:55:47.208708] [INFO ] [LOCAL ] RRX: Starting deletion of derivative data. [2025-10-20T20:55:47.208885] [DEBUG] [LOCAL ] RRX: Deleted derivative data file. [2025-10-20T20:55:47.208997] [INFO ] [MAIN ] Update Progress: 1831/2367 symbols completed. RRX took 35.1s. Estimated remaining: 1h 9m 0s. Rate: 7.8 symbols/min [2025-10-20T20:55:47.391746] [INFO ] [MAIN ] RSKD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:47.423712] [DEBUG] [LOCAL ] RSKD: Loaded 33929 existing records from primary storage. [2025-10-20T20:55:47.423978] [INFO ] [MAIN ] RSKD: Loaded 33929 existing records. [2025-10-20T20:55:47.456682] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-08. [2025-10-20T20:55:47.612306] [WARN ] [MAIN ] RSKD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:47.612762] [INFO ] [MAIN ] RSKD: Wiping existing primary data before full refresh. [2025-10-20T20:55:47.612899] [INFO ] [LOCAL ] RSKD: Starting deletion of primary data. [2025-10-20T20:55:47.613122] [DEBUG] [LOCAL ] RSKD: Deleted primary data file. [2025-10-20T20:55:47.613177] [INFO ] [MAIN ] RSKD: Starting full primary refresh. [2025-10-20T20:55:47.829239] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-03 (1.98 seconds). [2025-10-20T20:55:48.034007] [INFO ] [REMOTE ] RSKD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:48.251378] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-01. [2025-10-20T20:55:48.645150] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T20:55:49.051415] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-09. [2025-10-20T20:55:49.676961] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T20:55:49.851416] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-04. [2025-10-20T20:55:50.093618] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T20:55:50.651385] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-02. [2025-10-20T20:55:51.457471] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-04. [2025-10-20T20:55:51.479406] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T20:55:51.941298] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:55:52.251413] [INFO ] [REMOTE ] RS: Fetching primary data for 2025-10. [2025-10-20T20:55:52.905282] [INFO ] [REMOTE ] RS: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T20:55:52.908887] [INFO ] [REMOTE ] RS: Full fetch complete (38610 records). [2025-10-20T20:55:52.908964] [DEBUG] [VALIDATOR ] RS: Validating primary data. [2025-10-20T20:55:52.909461] [FATAL] [VALIDATOR ] RS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:55:52.909547] [FATAL] [MAIN ] RS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:55:52.909591] [WARN ] [MAIN ] RS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:55:52.909679] [INFO ] [LOCAL ] RS: Starting deletion of primary data. [2025-10-20T20:55:52.909768] [DEBUG] [LOCAL ] RS: No primary data file found to delete. [2025-10-20T20:55:52.909811] [INFO ] [LOCAL ] RS: Starting deletion of derivative data. [2025-10-20T20:55:52.909963] [DEBUG] [LOCAL ] RS: Deleted derivative data file. [2025-10-20T20:55:52.910045] [INFO ] [MAIN ] Update Progress: 1832/2367 symbols completed. RS took 37.6s. Estimated remaining: 1h 8m 52s. Rate: 7.8 symbols/min [2025-10-20T20:55:53.064326] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-05. [2025-10-20T20:55:53.081609] [INFO ] [MAIN ] RTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:55:53.119875] [DEBUG] [LOCAL ] RTO: Loaded 40597 existing records from primary storage. [2025-10-20T20:55:53.120180] [INFO ] [MAIN ] RTO: Loaded 40597 existing records. [2025-10-20T20:55:53.299258] [WARN ] [MAIN ] RTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:55:53.299568] [INFO ] [MAIN ] RTO: Wiping existing primary data before full refresh. [2025-10-20T20:55:53.299716] [INFO ] [LOCAL ] RTO: Starting deletion of primary data. [2025-10-20T20:55:53.300031] [DEBUG] [LOCAL ] RTO: Deleted primary data file. [2025-10-20T20:55:53.300088] [INFO ] [MAIN ] RTO: Starting full primary refresh. [2025-10-20T20:55:53.687169] [INFO ] [REMOTE ] RTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:55:53.689592] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-04 (2.23 seconds). [2025-10-20T20:55:53.851372] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-05. [2025-10-20T20:55:54.567109] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T20:55:54.651333] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-06. [2025-10-20T20:55:55.416923] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T20:55:55.451335] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-06. [2025-10-20T20:55:56.243542] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T20:55:56.252049] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-03. [2025-10-20T20:55:57.051421] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-07. [2025-10-20T20:55:57.149270] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T20:55:57.791504] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T20:55:57.851402] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-07. [2025-10-20T20:55:58.651361] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-04. [2025-10-20T20:55:58.847240] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T20:55:59.451356] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-01. [2025-10-20T20:55:59.703927] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T20:56:00.079451] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T20:56:00.251460] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-05. [2025-10-20T20:56:01.051358] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-08. [2025-10-20T20:56:01.370282] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T20:56:01.633420] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:56:01.661956] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T20:56:01.851358] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-06. [2025-10-20T20:56:02.651411] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-02. [2025-10-20T20:56:03.321418] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-06 (1.47 seconds). [2025-10-20T20:56:03.451337] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-07. [2025-10-20T20:56:04.084953] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:56:04.251383] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-03. [2025-10-20T20:56:04.718836] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T20:56:05.051443] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-08. [2025-10-20T20:56:05.851396] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-08. [2025-10-20T20:56:06.011742] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T20:56:06.066983] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T20:56:06.651379] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-04. [2025-10-20T20:56:07.377159] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T20:56:07.451364] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-09. [2025-10-20T20:56:08.251352] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-09. [2025-10-20T20:56:08.326087] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T20:56:08.632317] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T20:56:08.740011] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T20:56:09.051371] [INFO ] [REMOTE ] RSG: Fetching primary data for 2025-10. [2025-10-20T20:56:09.721987] [INFO ] [REMOTE ] RSG: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T20:56:09.728571] [INFO ] [REMOTE ] RSG: Full fetch complete (68090 records). [2025-10-20T20:56:09.728717] [DEBUG] [VALIDATOR ] RSG: Validating primary data. [2025-10-20T20:56:09.729448] [FATAL] [VALIDATOR ] RSG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:56:09.729535] [FATAL] [MAIN ] RSG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:56:09.729578] [WARN ] [MAIN ] RSG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:56:09.729661] [INFO ] [LOCAL ] RSG: Starting deletion of primary data. [2025-10-20T20:56:09.729743] [DEBUG] [LOCAL ] RSG: No primary data file found to delete. [2025-10-20T20:56:09.729780] [INFO ] [LOCAL ] RSG: Starting deletion of derivative data. [2025-10-20T20:56:09.729925] [DEBUG] [LOCAL ] RSG: Deleted derivative data file. [2025-10-20T20:56:09.730389] [INFO ] [MAIN ] Update Progress: 1833/2367 symbols completed. RSG took 32.8s. Estimated remaining: 1h 8m 47s. Rate: 7.8 symbols/min [2025-10-20T20:56:09.856462] [INFO ] [REMOTE ] RSKD: Fetching primary data for 2025-10. [2025-10-20T20:56:09.905474] [INFO ] [MAIN ] RTX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:09.961527] [DEBUG] [LOCAL ] RTX: Loaded 54486 existing records from primary storage. [2025-10-20T20:56:09.961877] [INFO ] [MAIN ] RTX: Loaded 54486 existing records. [2025-10-20T20:56:10.145410] [WARN ] [MAIN ] RTX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:56:10.145661] [INFO ] [MAIN ] RTX: Wiping existing primary data before full refresh. [2025-10-20T20:56:10.145722] [INFO ] [LOCAL ] RTX: Starting deletion of primary data. [2025-10-20T20:56:10.146028] [DEBUG] [LOCAL ] RTX: Deleted primary data file. [2025-10-20T20:56:10.146085] [INFO ] [MAIN ] RTX: Starting full primary refresh. [2025-10-20T20:56:10.526331] [INFO ] [REMOTE ] RTX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:10.651351] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-01. [2025-10-20T20:56:11.145004] [INFO ] [REMOTE ] RSKD: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T20:56:11.150266] [INFO ] [REMOTE ] RSKD: Full fetch complete (48336 records). [2025-10-20T20:56:11.150498] [DEBUG] [VALIDATOR ] RSKD: Validating primary data. [2025-10-20T20:56:11.273555] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T20:56:11.456703] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-02. [2025-10-20T20:56:11.572299] [DEBUG] [VALIDATOR ] RSKD: All expected trading days are present. [2025-10-20T20:56:11.572383] [DEBUG] [VALIDATOR ] RSKD: Primary data validation passed. [2025-10-20T20:56:11.573270] [INFO ] [MAIN ] RSKD: Fetched primary data is valid. [2025-10-20T20:56:11.573335] [DEBUG] [LOCAL ] RSKD: Saving primary data. [2025-10-20T20:56:11.574424] [DEBUG] [LOCAL ] RSKD: Trimmed 48336 records to 33373 (keeping 200 days of data) [2025-10-20T20:56:11.754589] [DEBUG] [LOCAL ] RSKD: Primary data saved (33373 records). [2025-10-20T20:56:11.754761] [INFO ] [MAIN ] RSKD: Full history saved. [2025-10-20T20:56:11.754848] [INFO ] [MAIN ] Update Progress: 1834/2367 symbols completed. RSKD took 24.5s. Estimated remaining: 1h 8m 37s. Rate: 7.8 symbols/min [2025-10-20T20:56:11.934204] [INFO ] [MAIN ] RVLV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:11.977912] [DEBUG] [LOCAL ] RVLV: Loaded 48144 existing records from primary storage. [2025-10-20T20:56:11.978231] [INFO ] [MAIN ] RVLV: Loaded 48144 existing records. [2025-10-20T20:56:12.162629] [WARN ] [MAIN ] RVLV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:56:12.163006] [INFO ] [MAIN ] RVLV: Wiping existing primary data before full refresh. [2025-10-20T20:56:12.163078] [INFO ] [LOCAL ] RVLV: Starting deletion of primary data. [2025-10-20T20:56:12.163442] [DEBUG] [LOCAL ] RVLV: Deleted primary data file. [2025-10-20T20:56:12.163500] [INFO ] [MAIN ] RVLV: Starting full primary refresh. [2025-10-20T20:56:12.253307] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-05. [2025-10-20T20:56:12.529244] [INFO ] [REMOTE ] RVLV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:12.560298] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T20:56:13.051423] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-03. [2025-10-20T20:56:13.415231] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T20:56:13.851349] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-06. [2025-10-20T20:56:13.919547] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T20:56:14.651422] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-04. [2025-10-20T20:56:15.105550] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-06 (1.25 seconds). [2025-10-20T20:56:15.451369] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-07. [2025-10-20T20:56:15.551167] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T20:56:16.251384] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-01. [2025-10-20T20:56:16.947876] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T20:56:17.051451] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-05. [2025-10-20T20:56:17.845039] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T20:56:17.854322] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-08. [2025-10-20T20:56:17.919824] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T20:56:18.651374] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-02. [2025-10-20T20:56:19.451353] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-09. [2025-10-20T20:56:19.522912] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T20:56:20.215856] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T20:56:20.260581] [INFO ] [REMOTE ] RSI: Fetching primary data for 2025-10. [2025-10-20T20:56:20.262861] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T20:56:20.959084] [INFO ] [REMOTE ] RSI: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T20:56:20.966018] [INFO ] [REMOTE ] RSI: Full fetch complete (69365 records). [2025-10-20T20:56:20.966123] [DEBUG] [VALIDATOR ] RSI: Validating primary data. [2025-10-20T20:56:20.966831] [FATAL] [VALIDATOR ] RSI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:56:20.966917] [FATAL] [MAIN ] RSI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:56:20.966958] [WARN ] [MAIN ] RSI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:56:20.966994] [INFO ] [LOCAL ] RSI: Starting deletion of primary data. [2025-10-20T20:56:20.967047] [DEBUG] [LOCAL ] RSI: No primary data file found to delete. [2025-10-20T20:56:20.967080] [INFO ] [LOCAL ] RSI: Starting deletion of derivative data. [2025-10-20T20:56:20.967197] [DEBUG] [LOCAL ] RSI: Deleted derivative data file. [2025-10-20T20:56:20.967263] [INFO ] [MAIN ] Update Progress: 1835/2367 symbols completed. RSI took 42.3s. Estimated remaining: 1h 8m 30s. Rate: 7.8 symbols/min [2025-10-20T20:56:21.056514] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-03. [2025-10-20T20:56:21.138599] [INFO ] [MAIN ] RVT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:21.167471] [DEBUG] [LOCAL ] RVT: Loaded 26714 existing records from primary storage. [2025-10-20T20:56:21.167754] [INFO ] [MAIN ] RVT: Loaded 26714 existing records. [2025-10-20T20:56:21.341816] [WARN ] [MAIN ] RVT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:56:21.341996] [INFO ] [MAIN ] RVT: Wiping existing primary data before full refresh. [2025-10-20T20:56:21.342041] [INFO ] [LOCAL ] RVT: Starting deletion of primary data. [2025-10-20T20:56:21.342211] [DEBUG] [LOCAL ] RVT: Deleted primary data file. [2025-10-20T20:56:21.342248] [INFO ] [MAIN ] RVT: Starting full primary refresh. [2025-10-20T20:56:21.688850] [INFO ] [REMOTE ] RVT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:21.851327] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-01. [2025-10-20T20:56:22.651432] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-09. [2025-10-20T20:56:22.760418] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T20:56:23.273409] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T20:56:23.451417] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-04. [2025-10-20T20:56:24.240984] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T20:56:24.253152] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-02. [2025-10-20T20:56:25.051342] [INFO ] [REMOTE ] RTO: Fetching primary data for 2025-10. [2025-10-20T20:56:25.165585] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T20:56:25.199336] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T20:56:25.851429] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-05. [2025-10-20T20:56:26.624003] [INFO ] [REMOTE ] RTO: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T20:56:26.630350] [INFO ] [REMOTE ] RTO: Full fetch complete (58833 records). [2025-10-20T20:56:26.630468] [DEBUG] [VALIDATOR ] RTO: Validating primary data. [2025-10-20T20:56:26.662562] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-06. [2025-10-20T20:56:27.010803] [DEBUG] [VALIDATOR ] RTO: All expected trading days are present. [2025-10-20T20:56:27.010876] [DEBUG] [VALIDATOR ] RTO: Primary data validation passed. [2025-10-20T20:56:27.011763] [INFO ] [MAIN ] RTO: Fetched primary data is valid. [2025-10-20T20:56:27.011823] [DEBUG] [LOCAL ] RTO: Saving primary data. [2025-10-20T20:56:27.012883] [DEBUG] [LOCAL ] RTO: Trimmed 58833 records to 39868 (keeping 200 days of data) [2025-10-20T20:56:27.247316] [DEBUG] [LOCAL ] RTO: Primary data saved (39868 records). [2025-10-20T20:56:27.247430] [INFO ] [MAIN ] RTO: Full history saved. [2025-10-20T20:56:27.247504] [INFO ] [MAIN ] Update Progress: 1836/2367 symbols completed. RTO took 34.3s. Estimated remaining: 1h 8m 22s. Rate: 7.8 symbols/min [2025-10-20T20:56:27.420799] [INFO ] [MAIN ] RVTY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:27.421063] [DEBUG] [LOCAL ] RVTY: No existing data file found in primary storage. [2025-10-20T20:56:27.421337] [WARN ] [MAIN ] RVTY: No existing data found, performing full refresh instead. [2025-10-20T20:56:27.421745] [INFO ] [MAIN ] RVTY: Wiping existing primary data before full refresh. [2025-10-20T20:56:27.421851] [INFO ] [LOCAL ] RVTY: Starting deletion of primary data. [2025-10-20T20:56:27.422008] [DEBUG] [LOCAL ] RVTY: No primary data file found to delete. [2025-10-20T20:56:27.422131] [INFO ] [MAIN ] RVTY: Starting full primary refresh. [2025-10-20T20:56:27.463321] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-03. [2025-10-20T20:56:27.665833] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T20:56:27.874993] [INFO ] [REMOTE ] RVTY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:27.876913] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T20:56:28.251409] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-07. [2025-10-20T20:56:28.988114] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-03 (1.52 seconds). [2025-10-20T20:56:29.051353] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-06. [2025-10-20T20:56:29.121358] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T20:56:29.851376] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-04. [2025-10-20T20:56:30.657296] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-01. [2025-10-20T20:56:30.669970] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T20:56:30.778143] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-04 (0.93 seconds). [2025-10-20T20:56:31.451416] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-08. [2025-10-20T20:56:31.936759] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T20:56:31.989201] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T20:56:32.251411] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-07. [2025-10-20T20:56:33.051444] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-05. [2025-10-20T20:56:33.705844] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T20:56:33.851348] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-02. [2025-10-20T20:56:34.156790] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T20:56:34.651393] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-08. [2025-10-20T20:56:35.284384] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T20:56:35.451347] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-03. [2025-10-20T20:56:36.251356] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-06. [2025-10-20T20:56:36.362087] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T20:56:37.051451] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-09. [2025-10-20T20:56:37.160351] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T20:56:37.301804] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T20:56:37.442518] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-10-20T20:56:37.851373] [INFO ] [REMOTE ] RTX: Fetching primary data for 2025-10. [2025-10-20T20:56:38.237575] [INFO ] [REMOTE ] RTX: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-10-20T20:56:38.245114] [INFO ] [REMOTE ] RTX: Full fetch complete (77485 records). [2025-10-20T20:56:38.245266] [DEBUG] [VALIDATOR ] RTX: Validating primary data. [2025-10-20T20:56:38.246014] [FATAL] [VALIDATOR ] RTX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:56:38.246101] [FATAL] [MAIN ] RTX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:56:38.246144] [WARN ] [MAIN ] RTX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:56:38.246210] [INFO ] [LOCAL ] RTX: Starting deletion of primary data. [2025-10-20T20:56:38.246274] [DEBUG] [LOCAL ] RTX: No primary data file found to delete. [2025-10-20T20:56:38.246310] [INFO ] [LOCAL ] RTX: Starting deletion of derivative data. [2025-10-20T20:56:38.246495] [DEBUG] [LOCAL ] RTX: Deleted derivative data file. [2025-10-20T20:56:38.246842] [INFO ] [MAIN ] Update Progress: 1837/2367 symbols completed. RTX took 28.5s. Estimated remaining: 1h 8m 15s. Rate: 7.8 symbols/min [2025-10-20T20:56:38.651376] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-09. [2025-10-20T20:56:39.451363] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-07. [2025-10-20T20:56:40.180517] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T20:56:40.251329] [INFO ] [REMOTE ] RVLV: Fetching primary data for 2025-10. [2025-10-20T20:56:40.440539] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T20:56:41.051370] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-08. [2025-10-20T20:56:41.763081] [INFO ] [REMOTE ] RVLV: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T20:56:41.770358] [INFO ] [REMOTE ] RVLV: Full fetch complete (69042 records). [2025-10-20T20:56:41.770490] [DEBUG] [VALIDATOR ] RVLV: Validating primary data. [2025-10-20T20:56:41.861546] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-04. [2025-10-20T20:56:42.172437] [DEBUG] [VALIDATOR ] RVLV: All expected trading days are present. [2025-10-20T20:56:42.172606] [DEBUG] [VALIDATOR ] RVLV: Primary data validation passed. [2025-10-20T20:56:42.179080] [INFO ] [MAIN ] RVLV: Fetched primary data is valid. [2025-10-20T20:56:42.188614] [DEBUG] [LOCAL ] RVLV: Saving primary data. [2025-10-20T20:56:42.190310] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T20:56:42.197412] [DEBUG] [LOCAL ] RVLV: Trimmed 69042 records to 47502 (keeping 200 days of data) [2025-10-20T20:56:42.468773] [DEBUG] [LOCAL ] RVLV: Primary data saved (47502 records). [2025-10-20T20:56:42.468914] [INFO ] [MAIN ] RVLV: Full history saved. [2025-10-20T20:56:42.468994] [INFO ] [MAIN ] Update Progress: 1838/2367 symbols completed. RVLV took 30.7s. Estimated remaining: 1h 8m 6s. Rate: 7.8 symbols/min [2025-10-20T20:56:42.651434] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-09. [2025-10-20T20:56:43.479887] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T20:56:43.491446] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-05. [2025-10-20T20:56:43.836766] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T20:56:44.251451] [INFO ] [REMOTE ] RVT: Fetching primary data for 2025-10. [2025-10-20T20:56:45.127562] [INFO ] [REMOTE ] RVT: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T20:56:45.131516] [INFO ] [REMOTE ] RVT: Full fetch complete (39171 records). [2025-10-20T20:56:45.131591] [DEBUG] [VALIDATOR ] RVT: Validating primary data. [2025-10-20T20:56:45.479119] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T20:56:45.544835] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-06. [2025-10-20T20:56:45.576941] [DEBUG] [VALIDATOR ] RVT: All expected trading days are present. [2025-10-20T20:56:45.577022] [DEBUG] [VALIDATOR ] RVT: Primary data validation passed. [2025-10-20T20:56:45.577858] [INFO ] [MAIN ] RVT: Fetched primary data is valid. [2025-10-20T20:56:45.577947] [DEBUG] [LOCAL ] RVT: Saving primary data. [2025-10-20T20:56:45.579006] [DEBUG] [LOCAL ] RVT: Trimmed 39171 records to 26373 (keeping 200 days of data) [2025-10-20T20:56:45.735427] [DEBUG] [LOCAL ] RVT: Primary data saved (26373 records). [2025-10-20T20:56:45.735532] [INFO ] [MAIN ] RVT: Full history saved. [2025-10-20T20:56:45.735609] [INFO ] [MAIN ] Update Progress: 1839/2367 symbols completed. RVT took 24.8s. Estimated remaining: 1h 7m 57s. Rate: 7.8 symbols/min [2025-10-20T20:56:47.239690] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T20:56:47.252355] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-07. [2025-10-20T20:56:48.865742] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T20:56:48.875987] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-08. [2025-10-20T20:56:50.226993] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T20:56:50.237225] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-09. [2025-10-20T20:56:51.972346] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T20:56:51.982418] [INFO ] [REMOTE ] RVTY: Fetching primary data for 2025-10. [2025-10-20T20:56:52.763839] [INFO ] [REMOTE ] RVTY: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T20:56:52.770729] [INFO ] [REMOTE ] RVTY: Full fetch complete (67565 records). [2025-10-20T20:56:52.770885] [DEBUG] [VALIDATOR ] RVTY: Validating primary data. [2025-10-20T20:56:52.771596] [FATAL] [VALIDATOR ] RVTY: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:56:52.771730] [FATAL] [MAIN ] RVTY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:56:52.771775] [WARN ] [MAIN ] RVTY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:56:52.771848] [INFO ] [LOCAL ] RVTY: Starting deletion of primary data. [2025-10-20T20:56:52.771917] [DEBUG] [LOCAL ] RVTY: No primary data file found to delete. [2025-10-20T20:56:52.771953] [INFO ] [LOCAL ] RVTY: Starting deletion of derivative data. [2025-10-20T20:56:52.771991] [DEBUG] [LOCAL ] RVTY: No derivative data file found to delete. [2025-10-20T20:56:52.772045] [INFO ] [MAIN ] Update Progress: 1840/2367 symbols completed. RVTY took 25.5s. Estimated remaining: 1h 7m 49s. Rate: 7.8 symbols/min [2025-10-20T20:56:53.452486] [INFO ] [MAIN ] RWTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:53.432012] [INFO ] [MAIN ] RWT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:53.464614] [INFO ] [MAIN ] RWTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:53.476848] [DEBUG] [LOCAL ] RWTN: No existing data file found in primary storage. [2025-10-20T20:56:53.479907] [INFO ] [MAIN ] RWTP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:56:53.480005] [DEBUG] [LOCAL ] RWTO: No existing data file found in primary storage. [2025-10-20T20:56:53.480092] [WARN ] [MAIN ] RWTO: No existing data found, performing full refresh instead. [2025-10-20T20:56:53.480402] [INFO ] [MAIN ] RWTO: Wiping existing primary data before full refresh. [2025-10-20T20:56:53.480928] [INFO ] [LOCAL ] RWTO: Starting deletion of primary data. [2025-10-20T20:56:53.481025] [DEBUG] [LOCAL ] RWTO: No primary data file found to delete. [2025-10-20T20:56:53.481068] [INFO ] [MAIN ] RWTO: Starting full primary refresh. [2025-10-20T20:56:53.480305] [WARN ] [MAIN ] RWTN: No existing data found, performing full refresh instead. [2025-10-20T20:56:53.480358] [DEBUG] [LOCAL ] RWTP: No existing data file found in primary storage. [2025-10-20T20:56:53.516711] [INFO ] [MAIN ] RWTN: Wiping existing primary data before full refresh. [2025-10-20T20:56:53.516596] [WARN ] [MAIN ] RWTP: No existing data found, performing full refresh instead. [2025-10-20T20:56:53.537072] [INFO ] [LOCAL ] RWTN: Starting deletion of primary data. [2025-10-20T20:56:53.548126] [INFO ] [MAIN ] RWTP: Wiping existing primary data before full refresh. [2025-10-20T20:56:53.548045] [DEBUG] [LOCAL ] RWTN: No primary data file found to delete. [2025-10-20T20:56:53.553539] [INFO ] [LOCAL ] RWTP: Starting deletion of primary data. [2025-10-20T20:56:53.561282] [DEBUG] [LOCAL ] RWTP: No primary data file found to delete. [2025-10-20T20:56:53.566616] [INFO ] [MAIN ] RWTP: Starting full primary refresh. [2025-10-20T20:56:53.588708] [INFO ] [MAIN ] RWTN: Starting full primary refresh. [2025-10-20T20:56:53.971263] [DEBUG] [LOCAL ] RWT: Loaded 41307 existing records from primary storage. [2025-10-20T20:56:54.098726] [INFO ] [MAIN ] RWT: Loaded 41307 existing records. [2025-10-20T20:56:54.669559] [INFO ] [REMOTE ] RWTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:54.718366] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-01. [2025-10-20T20:56:54.757167] [WARN ] [MAIN ] RWT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:56:54.792336] [INFO ] [REMOTE ] RWTP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:54.813587] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-01. [2025-10-20T20:56:54.798128] [INFO ] [MAIN ] RWT: Wiping existing primary data before full refresh. [2025-10-20T20:56:54.827912] [INFO ] [LOCAL ] RWT: Starting deletion of primary data. [2025-10-20T20:56:54.828288] [DEBUG] [LOCAL ] RWT: Deleted primary data file. [2025-10-20T20:56:54.835704] [INFO ] [MAIN ] RWT: Starting full primary refresh. [2025-10-20T20:56:54.857217] [INFO ] [REMOTE ] RWTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:54.920745] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-01. [2025-10-20T20:56:55.208025] [INFO ] [REMOTE ] RWT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:56:55.208352] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-01. [2025-10-20T20:56:55.353368] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T20:56:55.354888] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-02. [2025-10-20T20:56:55.502346] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T20:56:55.504250] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-02. [2025-10-20T20:56:55.680126] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T20:56:55.681466] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-02. [2025-10-20T20:56:56.002497] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T20:56:56.003695] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-03. [2025-10-20T20:56:56.073399] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T20:56:56.075696] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-03. [2025-10-20T20:56:56.144494] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-02 (0.46 seconds). [2025-10-20T20:56:56.251490] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-03. [2025-10-20T20:56:56.502030] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T20:56:56.564110] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T20:56:56.694385] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T20:56:56.728795] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T20:56:57.051419] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-02. [2025-10-20T20:56:57.851371] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-04. [2025-10-20T20:56:58.499200] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-04 (0.65 seconds). [2025-10-20T20:56:58.550154] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T20:56:58.651405] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-03. [2025-10-20T20:56:59.451377] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-04. [2025-10-20T20:56:59.963556] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T20:57:00.251378] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-05. [2025-10-20T20:57:00.314146] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T20:57:00.880233] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T20:57:01.051384] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-05. [2025-10-20T20:57:01.603102] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T20:57:01.851390] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-04. [2025-10-20T20:57:02.364349] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T20:57:02.651395] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-06. [2025-10-20T20:57:03.041221] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T20:57:03.451383] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-07. [2025-10-20T20:57:03.877509] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T20:57:04.251369] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-08. [2025-10-20T20:57:04.866892] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-08 (0.62 seconds). [2025-10-20T20:57:05.051369] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-05. [2025-10-20T20:57:05.534828] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-05 (0.48 seconds). [2025-10-20T20:57:05.851383] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-06. [2025-10-20T20:57:06.358764] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T20:57:06.651367] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-07. [2025-10-20T20:57:07.013305] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-07 (0.36 seconds). [2025-10-20T20:57:07.451362] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-09. [2025-10-20T20:57:07.977020] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T20:57:08.251395] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-04. [2025-10-20T20:57:09.051374] [INFO ] [REMOTE ] RWTN: Fetching primary data for 2025-10. [2025-10-20T20:57:09.472190] [INFO ] [REMOTE ] RWTN: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T20:57:09.473062] [INFO ] [REMOTE ] RWTN: Full fetch complete (3179 records). [2025-10-20T20:57:09.473157] [DEBUG] [VALIDATOR ] RWTN: Validating primary data. [2025-10-20T20:57:09.651911] [WARN ] [VALIDATOR ] RWTN: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-10-20T20:57:09.652112] [FATAL] [MAIN ] RWTN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:57:09.652179] [WARN ] [MAIN ] RWTN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:57:09.652229] [INFO ] [LOCAL ] RWTN: Starting deletion of primary data. [2025-10-20T20:57:09.652281] [DEBUG] [LOCAL ] RWTN: No primary data file found to delete. [2025-10-20T20:57:09.652315] [INFO ] [LOCAL ] RWTN: Starting deletion of derivative data. [2025-10-20T20:57:09.652353] [DEBUG] [LOCAL ] RWTN: No derivative data file found to delete. [2025-10-20T20:57:09.652404] [INFO ] [MAIN ] Update Progress: 1841/2367 symbols completed. RWTN took 16.9s. Estimated remaining: 1h 7m 44s. Rate: 7.8 symbols/min [2025-10-20T20:57:09.819542] [INFO ] [MAIN ] RXO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:09.862446] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-08. [2025-10-20T20:57:09.865153] [DEBUG] [LOCAL ] RXO: Loaded 49101 existing records from primary storage. [2025-10-20T20:57:09.865447] [INFO ] [MAIN ] RXO: Loaded 49101 existing records. [2025-10-20T20:57:10.046197] [WARN ] [MAIN ] RXO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:57:10.046842] [INFO ] [MAIN ] RXO: Wiping existing primary data before full refresh. [2025-10-20T20:57:10.056214] [INFO ] [LOCAL ] RXO: Starting deletion of primary data. [2025-10-20T20:57:10.071188] [DEBUG] [LOCAL ] RXO: Deleted primary data file. [2025-10-20T20:57:10.071308] [INFO ] [MAIN ] RXO: Starting full primary refresh. [2025-10-20T20:57:10.136209] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T20:57:10.435144] [INFO ] [REMOTE ] RXO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:10.438087] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T20:57:10.651378] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-01. [2025-10-20T20:57:11.451375] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-05. [2025-10-20T20:57:12.173187] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T20:57:12.251369] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-02. [2025-10-20T20:57:12.743779] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T20:57:13.051377] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-06. [2025-10-20T20:57:13.404676] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T20:57:13.514530] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T20:57:13.851409] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-06. [2025-10-20T20:57:14.651459] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-03. [2025-10-20T20:57:15.199576] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T20:57:15.451364] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-07. [2025-10-20T20:57:16.129895] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T20:57:16.187195] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T20:57:16.251464] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-08. [2025-10-20T20:57:16.932339] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T20:57:17.051356] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-09. [2025-10-20T20:57:17.686922] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T20:57:17.851409] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-04. [2025-10-20T20:57:18.651439] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-09. [2025-10-20T20:57:19.285893] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T20:57:19.451386] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-07. [2025-10-20T20:57:19.617869] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T20:57:20.251377] [INFO ] [REMOTE ] RWTP: Fetching primary data for 2025-10. [2025-10-20T20:57:20.661883] [INFO ] [REMOTE ] RWTP: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T20:57:20.662731] [INFO ] [REMOTE ] RWTP: Full fetch complete (4669 records). [2025-10-20T20:57:20.662801] [DEBUG] [VALIDATOR ] RWTP: Validating primary data. [2025-10-20T20:57:20.845769] [WARN ] [VALIDATOR ] RWTP: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T20:57:20.846049] [FATAL] [MAIN ] RWTP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:57:20.846118] [WARN ] [MAIN ] RWTP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:57:20.846188] [INFO ] [LOCAL ] RWTP: Starting deletion of primary data. [2025-10-20T20:57:20.846248] [DEBUG] [LOCAL ] RWTP: No primary data file found to delete. [2025-10-20T20:57:20.846283] [INFO ] [LOCAL ] RWTP: Starting deletion of derivative data. [2025-10-20T20:57:20.846320] [DEBUG] [LOCAL ] RWTP: No derivative data file found to delete. [2025-10-20T20:57:20.846374] [INFO ] [MAIN ] Update Progress: 1842/2367 symbols completed. RWTP took 28.0s. Estimated remaining: 1h 7m 38s. Rate: 7.8 symbols/min [2025-10-20T20:57:21.014296] [INFO ] [MAIN ] RY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:21.039194] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T20:57:21.051336] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-08. [2025-10-20T20:57:21.088422] [DEBUG] [LOCAL ] RY: Loaded 48438 existing records from primary storage. [2025-10-20T20:57:21.088741] [INFO ] [MAIN ] RY: Loaded 48438 existing records. [2025-10-20T20:57:21.259622] [WARN ] [MAIN ] RY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:57:21.259860] [INFO ] [MAIN ] RY: Wiping existing primary data before full refresh. [2025-10-20T20:57:21.259905] [INFO ] [LOCAL ] RY: Starting deletion of primary data. [2025-10-20T20:57:21.260115] [DEBUG] [LOCAL ] RY: Deleted primary data file. [2025-10-20T20:57:21.260169] [INFO ] [MAIN ] RY: Starting full primary refresh. [2025-10-20T20:57:21.607064] [INFO ] [REMOTE ] RY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:21.851394] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-01. [2025-10-20T20:57:22.583885] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T20:57:22.656852] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-09. [2025-10-20T20:57:22.674396] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T20:57:23.451411] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-05. [2025-10-20T20:57:24.251372] [INFO ] [REMOTE ] RWTO: Fetching primary data for 2025-10. [2025-10-20T20:57:24.293318] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T20:57:24.758737] [INFO ] [REMOTE ] RWTO: Incremental fetch complete for 2025-10 (0.51 seconds). [2025-10-20T20:57:24.759707] [INFO ] [REMOTE ] RWTO: Full fetch complete (4815 records). [2025-10-20T20:57:24.759773] [DEBUG] [VALIDATOR ] RWTO: Validating primary data. [2025-10-20T20:57:24.938404] [WARN ] [VALIDATOR ] RWTO: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T20:57:24.938631] [FATAL] [MAIN ] RWTO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:57:24.938734] [WARN ] [MAIN ] RWTO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:57:24.938827] [INFO ] [LOCAL ] RWTO: Starting deletion of primary data. [2025-10-20T20:57:24.938909] [DEBUG] [LOCAL ] RWTO: No primary data file found to delete. [2025-10-20T20:57:24.938946] [INFO ] [LOCAL ] RWTO: Starting deletion of derivative data. [2025-10-20T20:57:24.938984] [DEBUG] [LOCAL ] RWTO: No derivative data file found to delete. [2025-10-20T20:57:24.939036] [INFO ] [MAIN ] Update Progress: 1843/2367 symbols completed. RWTO took 32.2s. Estimated remaining: 1h 7m 29s. Rate: 7.8 symbols/min [2025-10-20T20:57:25.053325] [INFO ] [REMOTE ] RWT: Fetching primary data for 2025-10. [2025-10-20T20:57:25.122109] [INFO ] [MAIN ] RYAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:25.191257] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T20:57:25.188474] [DEBUG] [LOCAL ] RYAM: Loaded 35616 existing records from primary storage. [2025-10-20T20:57:25.201194] [INFO ] [MAIN ] RYAM: Loaded 35616 existing records. [2025-10-20T20:57:25.387908] [WARN ] [MAIN ] RYAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:57:25.388132] [INFO ] [MAIN ] RYAM: Wiping existing primary data before full refresh. [2025-10-20T20:57:25.388182] [INFO ] [LOCAL ] RYAM: Starting deletion of primary data. [2025-10-20T20:57:25.388374] [DEBUG] [LOCAL ] RYAM: Deleted primary data file. [2025-10-20T20:57:25.388429] [INFO ] [MAIN ] RYAM: Starting full primary refresh. [2025-10-20T20:57:25.759837] [INFO ] [REMOTE ] RYAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:25.774574] [INFO ] [REMOTE ] RWT: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T20:57:25.780078] [INFO ] [REMOTE ] RWT: Full fetch complete (57922 records). [2025-10-20T20:57:25.780178] [DEBUG] [VALIDATOR ] RWT: Validating primary data. [2025-10-20T20:57:25.780941] [FATAL] [VALIDATOR ] RWT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:57:25.781025] [FATAL] [MAIN ] RWT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:57:25.781067] [WARN ] [MAIN ] RWT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:57:25.781133] [INFO ] [LOCAL ] RWT: Starting deletion of primary data. [2025-10-20T20:57:25.781196] [DEBUG] [LOCAL ] RWT: No primary data file found to delete. [2025-10-20T20:57:25.781230] [INFO ] [LOCAL ] RWT: Starting deletion of derivative data. [2025-10-20T20:57:25.781354] [DEBUG] [LOCAL ] RWT: Deleted derivative data file. [2025-10-20T20:57:25.781723] [INFO ] [MAIN ] Update Progress: 1844/2367 symbols completed. RWT took 33.0s. Estimated remaining: 1h 7m 19s. Rate: 7.8 symbols/min [2025-10-20T20:57:25.856490] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-02. [2025-10-20T20:57:25.950723] [INFO ] [MAIN ] RYAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:25.950936] [DEBUG] [LOCAL ] RYAN: No existing data file found in primary storage. [2025-10-20T20:57:25.950975] [WARN ] [MAIN ] RYAN: No existing data found, performing full refresh instead. [2025-10-20T20:57:25.951087] [INFO ] [MAIN ] RYAN: Wiping existing primary data before full refresh. [2025-10-20T20:57:25.951152] [INFO ] [LOCAL ] RYAN: Starting deletion of primary data. [2025-10-20T20:57:25.951194] [DEBUG] [LOCAL ] RYAN: No primary data file found to delete. [2025-10-20T20:57:25.951226] [INFO ] [MAIN ] RYAN: Starting full primary refresh. [2025-10-20T20:57:26.312583] [INFO ] [REMOTE ] RYAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:26.450702] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T20:57:26.651460] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-03. [2025-10-20T20:57:27.460858] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-06. [2025-10-20T20:57:27.463470] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T20:57:28.251361] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-04. [2025-10-20T20:57:28.782134] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-04 (0.53 seconds). [2025-10-20T20:57:29.051429] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-01. [2025-10-20T20:57:29.117918] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T20:57:29.851392] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-07. [2025-10-20T20:57:30.651362] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-05. [2025-10-20T20:57:30.730957] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T20:57:31.189987] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T20:57:31.423625] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T20:57:31.451330] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-06. [2025-10-20T20:57:32.211343] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T20:57:32.251342] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-07. [2025-10-20T20:57:33.051408] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-01. [2025-10-20T20:57:33.087092] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T20:57:33.851436] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-08. [2025-10-20T20:57:33.929479] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T20:57:34.651372] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-02. [2025-10-20T20:57:35.287329] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T20:57:35.451372] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-09. [2025-10-20T20:57:35.553466] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T20:57:36.251367] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-02. [2025-10-20T20:57:37.051350] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-03. [2025-10-20T20:57:37.124632] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T20:57:37.746358] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T20:57:37.851407] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-08. [2025-10-20T20:57:38.196852] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T20:57:38.613232] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T20:57:38.651334] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-09. [2025-10-20T20:57:39.313435] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-09 (0.66 seconds). [2025-10-20T20:57:39.451359] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-03. [2025-10-20T20:57:40.251360] [INFO ] [REMOTE ] RY: Fetching primary data for 2025-10. [2025-10-20T20:57:41.056674] [INFO ] [REMOTE ] RXO: Fetching primary data for 2025-10. [2025-10-20T20:57:41.061434] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T20:57:41.825738] [INFO ] [REMOTE ] RXO: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:57:41.833718] [INFO ] [REMOTE ] RXO: Full fetch complete (68536 records). [2025-10-20T20:57:41.833889] [DEBUG] [VALIDATOR ] RXO: Validating primary data. [2025-10-20T20:57:41.834786] [FATAL] [VALIDATOR ] RXO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:57:41.834889] [FATAL] [MAIN ] RXO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:57:41.834936] [WARN ] [MAIN ] RXO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:57:41.834977] [INFO ] [LOCAL ] RXO: Starting deletion of primary data. [2025-10-20T20:57:41.835034] [DEBUG] [LOCAL ] RXO: No primary data file found to delete. [2025-10-20T20:57:41.835068] [INFO ] [LOCAL ] RXO: Starting deletion of derivative data. [2025-10-20T20:57:41.835195] [DEBUG] [LOCAL ] RXO: Deleted derivative data file. [2025-10-20T20:57:41.835272] [INFO ] [MAIN ] Update Progress: 1845/2367 symbols completed. RXO took 32.2s. Estimated remaining: 1h 7m 14s. Rate: 7.8 symbols/min [2025-10-20T20:57:41.856704] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-04. [2025-10-20T20:57:41.987841] [INFO ] [REMOTE ] RY: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T20:57:42.028284] [INFO ] [REMOTE ] RY: Full fetch complete (69268 records). [2025-10-20T20:57:42.045442] [DEBUG] [VALIDATOR ] RY: Validating primary data. [2025-10-20T20:57:42.047018] [INFO ] [MAIN ] RYI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:42.221253] [DEBUG] [LOCAL ] RYI: Loaded 22046 existing records from primary storage. [2025-10-20T20:57:42.238246] [INFO ] [MAIN ] RYI: Loaded 22046 existing records. [2025-10-20T20:57:42.621834] [WARN ] [MAIN ] RYI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:57:42.638673] [DEBUG] [VALIDATOR ] RY: All expected trading days are present. [2025-10-20T20:57:42.638776] [DEBUG] [VALIDATOR ] RY: Primary data validation passed. [2025-10-20T20:57:42.640039] [INFO ] [MAIN ] RY: Fetched primary data is valid. [2025-10-20T20:57:42.640136] [DEBUG] [LOCAL ] RY: Saving primary data. [2025-10-20T20:57:42.640429] [INFO ] [MAIN ] RYI: Wiping existing primary data before full refresh. [2025-10-20T20:57:42.640498] [INFO ] [LOCAL ] RYI: Starting deletion of primary data. [2025-10-20T20:57:42.640878] [DEBUG] [LOCAL ] RYI: Deleted primary data file. [2025-10-20T20:57:42.641010] [INFO ] [MAIN ] RYI: Starting full primary refresh. [2025-10-20T20:57:42.656486] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-04. [2025-10-20T20:57:42.699519] [DEBUG] [LOCAL ] RY: Trimmed 69268 records to 47633 (keeping 200 days of data) [2025-10-20T20:57:43.244353] [INFO ] [REMOTE ] RYI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:43.303121] [DEBUG] [LOCAL ] RY: Primary data saved (47633 records). [2025-10-20T20:57:43.303242] [INFO ] [MAIN ] RY: Full history saved. [2025-10-20T20:57:43.303318] [INFO ] [MAIN ] Update Progress: 1846/2367 symbols completed. RY took 22.5s. Estimated remaining: 1h 7m 4s. Rate: 7.8 symbols/min [2025-10-20T20:57:43.468803] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-01. [2025-10-20T20:57:43.471379] [INFO ] [MAIN ] RYN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:57:43.512939] [DEBUG] [LOCAL ] RYN: Loaded 46269 existing records from primary storage. [2025-10-20T20:57:43.513274] [INFO ] [MAIN ] RYN: Loaded 46269 existing records. [2025-10-20T20:57:43.691608] [WARN ] [MAIN ] RYN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:57:43.691915] [INFO ] [MAIN ] RYN: Wiping existing primary data before full refresh. [2025-10-20T20:57:43.692013] [INFO ] [LOCAL ] RYN: Starting deletion of primary data. [2025-10-20T20:57:43.692282] [DEBUG] [LOCAL ] RYN: Deleted primary data file. [2025-10-20T20:57:43.692338] [INFO ] [MAIN ] RYN: Starting full primary refresh. [2025-10-20T20:57:43.829141] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T20:57:44.094571] [INFO ] [REMOTE ] RYN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:57:44.180254] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T20:57:44.251357] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-01. [2025-10-20T20:57:44.440160] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T20:57:45.051426] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-02. [2025-10-20T20:57:45.851404] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-05. [2025-10-20T20:57:45.904510] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T20:57:45.975127] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:57:46.651443] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-02. [2025-10-20T20:57:47.272261] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T20:57:47.451417] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-05. [2025-10-20T20:57:48.027139] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T20:57:48.251421] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-03. [2025-10-20T20:57:49.052794] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T20:57:49.052950] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-03. [2025-10-20T20:57:49.770816] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T20:57:49.851393] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-06. [2025-10-20T20:57:49.953374] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T20:57:50.651426] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-04. [2025-10-20T20:57:51.148819] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T20:57:51.451378] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-06. [2025-10-20T20:57:52.113212] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T20:57:52.251345] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-05. [2025-10-20T20:57:53.010688] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T20:57:53.051379] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-07. [2025-10-20T20:57:53.549306] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T20:57:53.851398] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-04. [2025-10-20T20:57:54.253294] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-07 (1.20 seconds). [2025-10-20T20:57:54.651368] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-06. [2025-10-20T20:57:55.059423] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T20:57:55.451373] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-08. [2025-10-20T20:57:55.894397] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T20:57:56.251372] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-07. [2025-10-20T20:57:57.051381] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-07. [2025-10-20T20:57:57.099731] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T20:57:57.851357] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-09. [2025-10-20T20:57:57.898693] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T20:57:58.504879] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T20:57:58.651383] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-08. [2025-10-20T20:57:59.280344] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T20:57:59.451341] [INFO ] [REMOTE ] RYAN: Fetching primary data for 2025-10. [2025-10-20T20:58:00.220294] [INFO ] [REMOTE ] RYAN: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T20:58:00.227694] [INFO ] [REMOTE ] RYAN: Full fetch complete (64537 records). [2025-10-20T20:58:00.227860] [DEBUG] [VALIDATOR ] RYAN: Validating primary data. [2025-10-20T20:58:00.228849] [FATAL] [VALIDATOR ] RYAN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:58:00.228937] [FATAL] [MAIN ] RYAN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:58:00.229002] [WARN ] [MAIN ] RYAN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:58:00.229074] [INFO ] [LOCAL ] RYAN: Starting deletion of primary data. [2025-10-20T20:58:00.229140] [DEBUG] [LOCAL ] RYAN: No primary data file found to delete. [2025-10-20T20:58:00.229176] [INFO ] [LOCAL ] RYAN: Starting deletion of derivative data. [2025-10-20T20:58:00.229214] [DEBUG] [LOCAL ] RYAN: No derivative data file found to delete. [2025-10-20T20:58:00.229272] [INFO ] [MAIN ] Update Progress: 1847/2367 symbols completed. RYAN took 34.4s. Estimated remaining: 1h 6m 59s. Rate: 7.8 symbols/min [2025-10-20T20:58:00.260753] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-08. [2025-10-20T20:58:00.424527] [INFO ] [MAIN ] RZB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:00.436331] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T20:58:00.452449] [DEBUG] [LOCAL ] RZB: Loaded 6031 existing records from primary storage. [2025-10-20T20:58:00.452687] [INFO ] [MAIN ] RZB: Loaded 6031 existing records. [2025-10-20T20:58:00.626919] [WARN ] [MAIN ] RZB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:00.627120] [INFO ] [MAIN ] RZB: Wiping existing primary data before full refresh. [2025-10-20T20:58:00.627165] [INFO ] [LOCAL ] RZB: Starting deletion of primary data. [2025-10-20T20:58:00.627315] [DEBUG] [LOCAL ] RZB: Deleted primary data file. [2025-10-20T20:58:00.627352] [INFO ] [MAIN ] RZB: Starting full primary refresh. [2025-10-20T20:58:00.981868] [INFO ] [REMOTE ] RZB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:01.051408] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-05. [2025-10-20T20:58:01.854233] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-01. [2025-10-20T20:58:01.865380] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T20:58:01.935229] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T20:58:02.368494] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T20:58:02.651403] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-09. [2025-10-20T20:58:03.451401] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-09. [2025-10-20T20:58:04.251396] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-02. [2025-10-20T20:58:04.308474] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T20:58:04.899191] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T20:58:05.010366] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T20:58:05.051416] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-03. [2025-10-20T20:58:05.706801] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-03 (0.66 seconds). [2025-10-20T20:58:05.851379] [INFO ] [REMOTE ] RYN: Fetching primary data for 2025-10. [2025-10-20T20:58:06.606958] [INFO ] [REMOTE ] RYN: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T20:58:06.613497] [INFO ] [REMOTE ] RYN: Full fetch complete (65202 records). [2025-10-20T20:58:06.613616] [DEBUG] [VALIDATOR ] RYN: Validating primary data. [2025-10-20T20:58:06.614232] [FATAL] [VALIDATOR ] RYN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:58:06.614317] [FATAL] [MAIN ] RYN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:58:06.614359] [WARN ] [MAIN ] RYN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:58:06.614396] [INFO ] [LOCAL ] RYN: Starting deletion of primary data. [2025-10-20T20:58:06.614468] [DEBUG] [LOCAL ] RYN: No primary data file found to delete. [2025-10-20T20:58:06.614540] [INFO ] [LOCAL ] RYN: Starting deletion of derivative data. [2025-10-20T20:58:06.614728] [DEBUG] [LOCAL ] RYN: Deleted derivative data file. [2025-10-20T20:58:06.614833] [INFO ] [MAIN ] Update Progress: 1848/2367 symbols completed. RYN took 23.3s. Estimated remaining: 1h 6m 51s. Rate: 7.8 symbols/min [2025-10-20T20:58:06.656617] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-06. [2025-10-20T20:58:06.815331] [INFO ] [MAIN ] RZC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:06.823521] [DEBUG] [LOCAL ] RZC: Loaded 6724 existing records from primary storage. [2025-10-20T20:58:06.823707] [INFO ] [MAIN ] RZC: Loaded 6724 existing records. [2025-10-20T20:58:07.004627] [WARN ] [MAIN ] RZC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:07.004841] [INFO ] [MAIN ] RZC: Wiping existing primary data before full refresh. [2025-10-20T20:58:07.004889] [INFO ] [LOCAL ] RZC: Starting deletion of primary data. [2025-10-20T20:58:07.005068] [DEBUG] [LOCAL ] RZC: Deleted primary data file. [2025-10-20T20:58:07.005123] [INFO ] [MAIN ] RZC: Starting full primary refresh. [2025-10-20T20:58:07.363773] [INFO ] [REMOTE ] RZC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:07.451371] [INFO ] [REMOTE ] RYAM: Fetching primary data for 2025-10. [2025-10-20T20:58:07.889317] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T20:58:08.251382] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-04. [2025-10-20T20:58:08.761389] [INFO ] [REMOTE ] RYAM: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T20:58:08.766930] [INFO ] [REMOTE ] RYAM: Full fetch complete (49139 records). [2025-10-20T20:58:08.767094] [DEBUG] [VALIDATOR ] RYAM: Validating primary data. [2025-10-20T20:58:08.879329] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T20:58:09.056490] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-01. [2025-10-20T20:58:09.171819] [DEBUG] [VALIDATOR ] RYAM: All expected trading days are present. [2025-10-20T20:58:09.171913] [DEBUG] [VALIDATOR ] RYAM: Primary data validation passed. [2025-10-20T20:58:09.172767] [INFO ] [MAIN ] RYAM: Fetched primary data is valid. [2025-10-20T20:58:09.172828] [DEBUG] [LOCAL ] RYAM: Saving primary data. [2025-10-20T20:58:09.173914] [DEBUG] [LOCAL ] RYAM: Trimmed 49139 records to 35098 (keeping 200 days of data) [2025-10-20T20:58:09.369447] [DEBUG] [LOCAL ] RYAM: Primary data saved (35098 records). [2025-10-20T20:58:09.369589] [INFO ] [MAIN ] RYAM: Full history saved. [2025-10-20T20:58:09.369701] [INFO ] [MAIN ] Update Progress: 1849/2367 symbols completed. RYAM took 44.4s. Estimated remaining: 1h 6m 42s. Rate: 7.8 symbols/min [2025-10-20T20:58:09.571311] [INFO ] [MAIN ] S: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:09.623891] [DEBUG] [LOCAL ] S: Loaded 54316 existing records from primary storage. [2025-10-20T20:58:09.624252] [INFO ] [MAIN ] S: Loaded 54316 existing records. [2025-10-20T20:58:09.815281] [WARN ] [MAIN ] S: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:09.815615] [INFO ] [MAIN ] S: Wiping existing primary data before full refresh. [2025-10-20T20:58:09.815712] [INFO ] [LOCAL ] S: Starting deletion of primary data. [2025-10-20T20:58:09.816311] [DEBUG] [LOCAL ] S: Deleted primary data file. [2025-10-20T20:58:09.816377] [INFO ] [MAIN ] S: Starting full primary refresh. [2025-10-20T20:58:09.856625] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-07. [2025-10-20T20:58:09.921308] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T20:58:10.192083] [INFO ] [REMOTE ] S: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:10.651415] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-02. [2025-10-20T20:58:11.174324] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T20:58:11.451422] [INFO ] [REMOTE ] S: Fetching primary data for 2025-01. [2025-10-20T20:58:11.570699] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:58:12.251383] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-03. [2025-10-20T20:58:12.886548] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T20:58:13.051428] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-04. [2025-10-20T20:58:13.131803] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T20:58:13.851354] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-05. [2025-10-20T20:58:13.860398] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T20:58:14.465514] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T20:58:14.651381] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-05. [2025-10-20T20:58:15.208467] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T20:58:15.451378] [INFO ] [REMOTE ] S: Fetching primary data for 2025-02. [2025-10-20T20:58:16.251396] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-08. [2025-10-20T20:58:17.051632] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-06. [2025-10-20T20:58:17.363915] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T20:58:17.537502] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-10-20T20:58:17.851399] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-09. [2025-10-20T20:58:18.119594] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T20:58:18.651447] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-07. [2025-10-20T20:58:18.720340] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T20:58:19.216052] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T20:58:19.451352] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-08. [2025-10-20T20:58:20.158587] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T20:58:20.251402] [INFO ] [REMOTE ] RYI: Fetching primary data for 2025-10. [2025-10-20T20:58:21.051421] [INFO ] [REMOTE ] S: Fetching primary data for 2025-03. [2025-10-20T20:58:21.237101] [INFO ] [REMOTE ] RYI: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T20:58:21.240566] [INFO ] [REMOTE ] RYI: Full fetch complete (31918 records). [2025-10-20T20:58:21.240734] [DEBUG] [VALIDATOR ] RYI: Validating primary data. [2025-10-20T20:58:21.604240] [DEBUG] [VALIDATOR ] RYI: All expected trading days are present. [2025-10-20T20:58:21.604318] [DEBUG] [VALIDATOR ] RYI: Primary data validation passed. [2025-10-20T20:58:21.604999] [INFO ] [MAIN ] RYI: Fetched primary data is valid. [2025-10-20T20:58:21.605058] [DEBUG] [LOCAL ] RYI: Saving primary data. [2025-10-20T20:58:21.605860] [DEBUG] [LOCAL ] RYI: Trimmed 31918 records to 21736 (keeping 200 days of data) [2025-10-20T20:58:21.731635] [DEBUG] [LOCAL ] RYI: Primary data saved (21736 records). [2025-10-20T20:58:21.731782] [INFO ] [MAIN ] RYI: Full history saved. [2025-10-20T20:58:21.731854] [INFO ] [MAIN ] Update Progress: 1850/2367 symbols completed. RYI took 39.9s. Estimated remaining: 1h 6m 36s. Rate: 7.8 symbols/min [2025-10-20T20:58:21.856516] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-06. [2025-10-20T20:58:21.899987] [INFO ] [MAIN ] SA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:21.940446] [DEBUG] [LOCAL ] SA: Loaded 45114 existing records from primary storage. [2025-10-20T20:58:21.940748] [INFO ] [MAIN ] SA: Loaded 45114 existing records. [2025-10-20T20:58:22.115298] [WARN ] [MAIN ] SA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:22.115528] [INFO ] [MAIN ] SA: Wiping existing primary data before full refresh. [2025-10-20T20:58:22.115573] [INFO ] [LOCAL ] SA: Starting deletion of primary data. [2025-10-20T20:58:22.115773] [DEBUG] [LOCAL ] SA: Deleted primary data file. [2025-10-20T20:58:22.115827] [INFO ] [MAIN ] SA: Starting full primary refresh. [2025-10-20T20:58:22.472469] [INFO ] [REMOTE ] SA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:22.572627] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T20:58:22.651339] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-01. [2025-10-20T20:58:22.914228] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T20:58:23.451372] [INFO ] [REMOTE ] S: Fetching primary data for 2025-04. [2025-10-20T20:58:24.181786] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T20:58:24.251390] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-02. [2025-10-20T20:58:25.051422] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-07. [2025-10-20T20:58:25.522393] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T20:58:25.556535] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T20:58:25.839216] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T20:58:25.851335] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-08. [2025-10-20T20:58:26.621048] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T20:58:26.651362] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-09. [2025-10-20T20:58:27.303668] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T20:58:27.451387] [INFO ] [REMOTE ] S: Fetching primary data for 2025-05. [2025-10-20T20:58:28.251394] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-03. [2025-10-20T20:58:29.051394] [INFO ] [REMOTE ] RZC: Fetching primary data for 2025-10. [2025-10-20T20:58:29.526836] [INFO ] [REMOTE ] RZC: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T20:58:29.528709] [INFO ] [REMOTE ] RZC: Full fetch complete (9662 records). [2025-10-20T20:58:29.528804] [DEBUG] [VALIDATOR ] RZC: Validating primary data. [2025-10-20T20:58:29.688150] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-10-20T20:58:29.806457] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T20:58:29.854624] [INFO ] [REMOTE ] S: Fetching primary data for 2025-06. [2025-10-20T20:58:30.022439] [DEBUG] [VALIDATOR ] RZC: All expected trading days are present. [2025-10-20T20:58:30.022516] [DEBUG] [VALIDATOR ] RZC: Primary data validation passed. [2025-10-20T20:58:30.022763] [INFO ] [MAIN ] RZC: Fetched primary data is valid. [2025-10-20T20:58:30.022823] [DEBUG] [LOCAL ] RZC: Saving primary data. [2025-10-20T20:58:30.023250] [DEBUG] [LOCAL ] RZC: Trimmed 9662 records to 6586 (keeping 200 days of data) [2025-10-20T20:58:30.062829] [DEBUG] [LOCAL ] RZC: Primary data saved (6586 records). [2025-10-20T20:58:30.062959] [INFO ] [MAIN ] RZC: Full history saved. [2025-10-20T20:58:30.063040] [INFO ] [MAIN ] Update Progress: 1851/2367 symbols completed. RZC took 23.4s. Estimated remaining: 1h 6m 28s. Rate: 7.8 symbols/min [2025-10-20T20:58:30.229364] [INFO ] [MAIN ] SABA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:30.240486] [DEBUG] [LOCAL ] SABA: Loaded 9397 existing records from primary storage. [2025-10-20T20:58:30.240769] [INFO ] [MAIN ] SABA: Loaded 9397 existing records. [2025-10-20T20:58:30.425172] [WARN ] [MAIN ] SABA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:30.425392] [INFO ] [MAIN ] SABA: Wiping existing primary data before full refresh. [2025-10-20T20:58:30.425440] [INFO ] [LOCAL ] SABA: Starting deletion of primary data. [2025-10-20T20:58:30.425599] [DEBUG] [LOCAL ] SABA: Deleted primary data file. [2025-10-20T20:58:30.425706] [INFO ] [MAIN ] SABA: Starting full primary refresh. [2025-10-20T20:58:30.658901] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-09. [2025-10-20T20:58:30.773349] [INFO ] [REMOTE ] SABA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:31.269212] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T20:58:31.451387] [INFO ] [REMOTE ] RZB: Fetching primary data for 2025-10. [2025-10-20T20:58:31.963956] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T20:58:31.994306] [INFO ] [REMOTE ] RZB: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T20:58:31.995309] [INFO ] [REMOTE ] RZB: Full fetch complete (8333 records). [2025-10-20T20:58:31.995375] [DEBUG] [VALIDATOR ] RZB: Validating primary data. [2025-10-20T20:58:32.256908] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-04. [2025-10-20T20:58:32.356490] [DEBUG] [VALIDATOR ] RZB: All expected trading days are present. [2025-10-20T20:58:32.356572] [DEBUG] [VALIDATOR ] RZB: Primary data validation passed. [2025-10-20T20:58:32.356815] [INFO ] [MAIN ] RZB: Fetched primary data is valid. [2025-10-20T20:58:32.356875] [DEBUG] [LOCAL ] RZB: Saving primary data. [2025-10-20T20:58:32.357368] [DEBUG] [LOCAL ] RZB: Trimmed 8333 records to 5937 (keeping 200 days of data) [2025-10-20T20:58:32.395795] [DEBUG] [LOCAL ] RZB: Primary data saved (5937 records). [2025-10-20T20:58:32.395975] [INFO ] [MAIN ] RZB: Full history saved. [2025-10-20T20:58:32.396066] [INFO ] [MAIN ] Update Progress: 1852/2367 symbols completed. RZB took 32.2s. Estimated remaining: 1h 6m 19s. Rate: 7.8 symbols/min [2025-10-20T20:58:32.577388] [INFO ] [MAIN ] SAFE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:32.611276] [DEBUG] [LOCAL ] SAFE: Loaded 30751 existing records from primary storage. [2025-10-20T20:58:32.611594] [INFO ] [MAIN ] SAFE: Loaded 30751 existing records. [2025-10-20T20:58:32.788708] [WARN ] [MAIN ] SAFE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:32.788939] [INFO ] [MAIN ] SAFE: Wiping existing primary data before full refresh. [2025-10-20T20:58:32.788983] [INFO ] [LOCAL ] SAFE: Starting deletion of primary data. [2025-10-20T20:58:32.789157] [DEBUG] [LOCAL ] SAFE: Deleted primary data file. [2025-10-20T20:58:32.789195] [INFO ] [MAIN ] SAFE: Starting full primary refresh. [2025-10-20T20:58:33.057053] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-01. [2025-10-20T20:58:33.144914] [INFO ] [REMOTE ] SAFE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:33.760775] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T20:58:33.851335] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-05. [2025-10-20T20:58:34.229687] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T20:58:34.651368] [INFO ] [REMOTE ] S: Fetching primary data for 2025-07. [2025-10-20T20:58:35.456534] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-01. [2025-10-20T20:58:35.470590] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T20:58:36.251446] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-06. [2025-10-20T20:58:36.735973] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-07 (2.08 seconds). [2025-10-20T20:58:37.051435] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-02. [2025-10-20T20:58:37.116144] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T20:58:37.819299] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T20:58:37.851384] [INFO ] [REMOTE ] S: Fetching primary data for 2025-08. [2025-10-20T20:58:37.973390] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:58:38.651374] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-03. [2025-10-20T20:58:39.451436] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-07. [2025-10-20T20:58:39.552756] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T20:58:40.092503] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-08 (2.24 seconds). [2025-10-20T20:58:40.251387] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-04. [2025-10-20T20:58:40.719941] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T20:58:41.025456] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T20:58:41.051334] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-05. [2025-10-20T20:58:41.853929] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-02. [2025-10-20T20:58:41.856141] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T20:58:42.651375] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-06. [2025-10-20T20:58:43.145252] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T20:58:43.451374] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-03. [2025-10-20T20:58:43.592300] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T20:58:44.251382] [INFO ] [REMOTE ] S: Fetching primary data for 2025-09. [2025-10-20T20:58:44.793294] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T20:58:45.051385] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-07. [2025-10-20T20:58:45.856463] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-08. [2025-10-20T20:58:45.859897] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T20:58:46.555189] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-09 (2.30 seconds). [2025-10-20T20:58:46.651344] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-08. [2025-10-20T20:58:47.453283] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-04. [2025-10-20T20:58:47.455163] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T20:58:47.514227] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T20:58:48.251438] [INFO ] [REMOTE ] S: Fetching primary data for 2025-10. [2025-10-20T20:58:48.656314] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T20:58:49.051387] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-09. [2025-10-20T20:58:49.851349] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-09. [2025-10-20T20:58:49.933248] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T20:58:49.977764] [INFO ] [REMOTE ] S: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T20:58:49.985706] [INFO ] [REMOTE ] S: Full fetch complete (77659 records). [2025-10-20T20:58:49.985836] [DEBUG] [VALIDATOR ] S: Validating primary data. [2025-10-20T20:58:50.391303] [DEBUG] [VALIDATOR ] S: All expected trading days are present. [2025-10-20T20:58:50.391377] [DEBUG] [VALIDATOR ] S: Primary data validation passed. [2025-10-20T20:58:50.392416] [INFO ] [MAIN ] S: Fetched primary data is valid. [2025-10-20T20:58:50.392479] [DEBUG] [LOCAL ] S: Saving primary data. [2025-10-20T20:58:50.394022] [DEBUG] [LOCAL ] S: Trimmed 77659 records to 53538 (keeping 200 days of data) [2025-10-20T20:58:50.654564] [INFO ] [REMOTE ] SABA: Fetching primary data for 2025-10. [2025-10-20T20:58:50.703124] [DEBUG] [LOCAL ] S: Primary data saved (53538 records). [2025-10-20T20:58:50.703222] [INFO ] [MAIN ] S: Full history saved. [2025-10-20T20:58:50.703294] [INFO ] [MAIN ] Update Progress: 1853/2367 symbols completed. S took 41.3s. Estimated remaining: 1h 6m 14s. Rate: 7.8 symbols/min [2025-10-20T20:58:50.866419] [INFO ] [MAIN ] SAH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:50.885914] [DEBUG] [LOCAL ] SAH: Loaded 21442 existing records from primary storage. [2025-10-20T20:58:50.886140] [INFO ] [MAIN ] SAH: Loaded 21442 existing records. [2025-10-20T20:58:51.069048] [WARN ] [MAIN ] SAH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:51.070294] [INFO ] [MAIN ] SAH: Wiping existing primary data before full refresh. [2025-10-20T20:58:51.070354] [INFO ] [LOCAL ] SAH: Starting deletion of primary data. [2025-10-20T20:58:51.071092] [DEBUG] [LOCAL ] SAH: Deleted primary data file. [2025-10-20T20:58:51.071151] [INFO ] [MAIN ] SAH: Starting full primary refresh. [2025-10-20T20:58:51.451005] [INFO ] [REMOTE ] SABA: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T20:58:51.452857] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-05. [2025-10-20T20:58:51.450760] [INFO ] [REMOTE ] SAH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:51.454692] [INFO ] [REMOTE ] SABA: Full fetch complete (15110 records). [2025-10-20T20:58:51.454810] [DEBUG] [VALIDATOR ] SABA: Validating primary data. [2025-10-20T20:58:51.885498] [DEBUG] [VALIDATOR ] SABA: All expected trading days are present. [2025-10-20T20:58:51.885611] [DEBUG] [VALIDATOR ] SABA: Primary data validation passed. [2025-10-20T20:58:51.886525] [INFO ] [MAIN ] SABA: Fetched primary data is valid. [2025-10-20T20:58:51.887621] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T20:58:51.887683] [DEBUG] [LOCAL ] SABA: Saving primary data. [2025-10-20T20:58:51.898429] [DEBUG] [LOCAL ] SABA: Trimmed 15110 records to 9248 (keeping 200 days of data) [2025-10-20T20:58:51.955808] [DEBUG] [LOCAL ] SABA: Primary data saved (9248 records). [2025-10-20T20:58:51.955980] [INFO ] [MAIN ] SABA: Full history saved. [2025-10-20T20:58:51.956113] [INFO ] [MAIN ] Update Progress: 1854/2367 symbols completed. SABA took 21.9s. Estimated remaining: 1h 6m 4s. Rate: 7.8 symbols/min [2025-10-20T20:58:52.129407] [INFO ] [MAIN ] SAJ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:52.129581] [DEBUG] [LOCAL ] SAJ: No existing data file found in primary storage. [2025-10-20T20:58:52.129619] [WARN ] [MAIN ] SAJ: No existing data found, performing full refresh instead. [2025-10-20T20:58:52.129748] [INFO ] [MAIN ] SAJ: Wiping existing primary data before full refresh. [2025-10-20T20:58:52.129789] [INFO ] [LOCAL ] SAJ: Starting deletion of primary data. [2025-10-20T20:58:52.129830] [DEBUG] [LOCAL ] SAJ: No primary data file found to delete. [2025-10-20T20:58:52.129862] [INFO ] [MAIN ] SAJ: Starting full primary refresh. [2025-10-20T20:58:52.256473] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-01. [2025-10-20T20:58:52.492959] [INFO ] [REMOTE ] SAJ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:52.784290] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T20:58:53.051374] [INFO ] [REMOTE ] SA: Fetching primary data for 2025-10. [2025-10-20T20:58:53.448942] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T20:58:53.851442] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-01. [2025-10-20T20:58:54.168475] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-01 (0.32 seconds). [2025-10-20T20:58:54.651412] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-02. [2025-10-20T20:58:55.061059] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-02 (0.41 seconds). [2025-10-20T20:58:55.086555] [INFO ] [REMOTE ] SA: Incremental fetch complete for 2025-10 (2.04 seconds). [2025-10-20T20:58:55.094421] [INFO ] [REMOTE ] SA: Full fetch complete (63401 records). [2025-10-20T20:58:55.094587] [DEBUG] [VALIDATOR ] SA: Validating primary data. [2025-10-20T20:58:55.463030] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-03. [2025-10-20T20:58:55.523605] [DEBUG] [VALIDATOR ] SA: All expected trading days are present. [2025-10-20T20:58:55.523730] [DEBUG] [VALIDATOR ] SA: Primary data validation passed. [2025-10-20T20:58:55.524836] [INFO ] [MAIN ] SA: Fetched primary data is valid. [2025-10-20T20:58:55.524900] [DEBUG] [LOCAL ] SA: Saving primary data. [2025-10-20T20:58:55.526398] [DEBUG] [LOCAL ] SA: Trimmed 63401 records to 44579 (keeping 200 days of data) [2025-10-20T20:58:55.792091] [DEBUG] [LOCAL ] SA: Primary data saved (44579 records). [2025-10-20T20:58:55.792239] [INFO ] [MAIN ] SA: Full history saved. [2025-10-20T20:58:55.792322] [INFO ] [MAIN ] Update Progress: 1855/2367 symbols completed. SA took 34.1s. Estimated remaining: 1h 5m 56s. Rate: 7.8 symbols/min [2025-10-20T20:58:55.941631] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T20:58:55.974952] [INFO ] [MAIN ] SAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:58:55.997382] [DEBUG] [LOCAL ] SAM: Loaded 21607 existing records from primary storage. [2025-10-20T20:58:55.997686] [INFO ] [MAIN ] SAM: Loaded 21607 existing records. [2025-10-20T20:58:56.179743] [WARN ] [MAIN ] SAM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:58:56.179950] [INFO ] [MAIN ] SAM: Wiping existing primary data before full refresh. [2025-10-20T20:58:56.179999] [INFO ] [LOCAL ] SAM: Starting deletion of primary data. [2025-10-20T20:58:56.180203] [DEBUG] [LOCAL ] SAM: Deleted primary data file. [2025-10-20T20:58:56.180259] [INFO ] [MAIN ] SAM: Starting full primary refresh. [2025-10-20T20:58:56.256547] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-06. [2025-10-20T20:58:56.531184] [INFO ] [REMOTE ] SAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:58:57.051427] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-01. [2025-10-20T20:58:57.567039] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T20:58:57.851440] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-04. [2025-10-20T20:58:58.203434] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-04 (0.35 seconds). [2025-10-20T20:58:58.316416] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T20:58:58.651369] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-02. [2025-10-20T20:58:59.451360] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-07. [2025-10-20T20:58:59.759671] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T20:59:00.251377] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-03. [2025-10-20T20:59:00.819676] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T20:59:01.051391] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-05. [2025-10-20T20:59:01.433843] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-05 (0.38 seconds). [2025-10-20T20:59:01.637934] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T20:59:01.851383] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-06. [2025-10-20T20:59:02.247147] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-06 (0.40 seconds). [2025-10-20T20:59:02.651388] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-04. [2025-10-20T20:59:03.451429] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-07. [2025-10-20T20:59:03.900977] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T20:59:04.099404] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T20:59:04.251356] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-05. [2025-10-20T20:59:05.051356] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-02. [2025-10-20T20:59:05.405732] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T20:59:05.851398] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-06. [2025-10-20T20:59:06.269496] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T20:59:06.651361] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-03. [2025-10-20T20:59:06.927741] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T20:59:07.451372] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-07. [2025-10-20T20:59:07.826291] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T20:59:08.251394] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-04. [2025-10-20T20:59:08.827356] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-07 (1.38 seconds). [2025-10-20T20:59:09.051342] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-08. [2025-10-20T20:59:09.449057] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T20:59:09.851397] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-08. [2025-10-20T20:59:10.284721] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T20:59:10.358277] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T20:59:10.651388] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-08. [2025-10-20T20:59:11.451427] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-09. [2025-10-20T20:59:11.814163] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T20:59:11.860319] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T20:59:12.251371] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-05. [2025-10-20T20:59:13.051486] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-09. [2025-10-20T20:59:13.295199] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T20:59:13.851343] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-06. [2025-10-20T20:59:14.090878] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T20:59:14.651431] [INFO ] [REMOTE ] SAH: Fetching primary data for 2025-10. [2025-10-20T20:59:15.140922] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T20:59:15.206947] [INFO ] [REMOTE ] SAH: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T20:59:15.209610] [INFO ] [REMOTE ] SAH: Full fetch complete (30722 records). [2025-10-20T20:59:15.209711] [DEBUG] [VALIDATOR ] SAH: Validating primary data. [2025-10-20T20:59:15.210195] [FATAL] [VALIDATOR ] SAH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:59:15.210275] [FATAL] [MAIN ] SAH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:59:15.210317] [WARN ] [MAIN ] SAH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:59:15.210354] [INFO ] [LOCAL ] SAH: Starting deletion of primary data. [2025-10-20T20:59:15.210431] [DEBUG] [LOCAL ] SAH: No primary data file found to delete. [2025-10-20T20:59:15.210490] [INFO ] [LOCAL ] SAH: Starting deletion of derivative data. [2025-10-20T20:59:15.210626] [DEBUG] [LOCAL ] SAH: Deleted derivative data file. [2025-10-20T20:59:15.210737] [INFO ] [MAIN ] Update Progress: 1856/2367 symbols completed. SAH took 24.5s. Estimated remaining: 1h 5m 51s. Rate: 7.8 symbols/min [2025-10-20T20:59:15.376872] [INFO ] [MAIN ] SAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:15.432046] [DEBUG] [LOCAL ] SAN: Loaded 53738 existing records from primary storage. [2025-10-20T20:59:15.432371] [INFO ] [MAIN ] SAN: Loaded 53738 existing records. [2025-10-20T20:59:15.461075] [INFO ] [REMOTE ] SAJ: Fetching primary data for 2025-10. [2025-10-20T20:59:15.612447] [WARN ] [MAIN ] SAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:59:15.612685] [INFO ] [MAIN ] SAN: Wiping existing primary data before full refresh. [2025-10-20T20:59:15.612742] [INFO ] [LOCAL ] SAN: Starting deletion of primary data. [2025-10-20T20:59:15.612927] [DEBUG] [LOCAL ] SAN: Deleted primary data file. [2025-10-20T20:59:15.612964] [INFO ] [MAIN ] SAN: Starting full primary refresh. [2025-10-20T20:59:15.959707] [INFO ] [REMOTE ] SAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:15.960405] [INFO ] [REMOTE ] SAJ: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T20:59:15.961491] [INFO ] [REMOTE ] SAJ: Full fetch complete (934 records). [2025-10-20T20:59:15.961560] [DEBUG] [VALIDATOR ] SAJ: Validating primary data. [2025-10-20T20:59:16.137041] [WARN ] [VALIDATOR ] SAJ: Validation failed - insufficient historical data. Have 182 unique days, need 200 days [2025-10-20T20:59:16.137226] [FATAL] [MAIN ] SAJ: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:59:16.137312] [WARN ] [MAIN ] SAJ: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:59:16.137359] [INFO ] [LOCAL ] SAJ: Starting deletion of primary data. [2025-10-20T20:59:16.137416] [DEBUG] [LOCAL ] SAJ: No primary data file found to delete. [2025-10-20T20:59:16.137451] [INFO ] [LOCAL ] SAJ: Starting deletion of derivative data. [2025-10-20T20:59:16.137489] [DEBUG] [LOCAL ] SAJ: No derivative data file found to delete. [2025-10-20T20:59:16.137564] [INFO ] [MAIN ] Update Progress: 1857/2367 symbols completed. SAJ took 24.2s. Estimated remaining: 1h 5m 42s. Rate: 7.8 symbols/min [2025-10-20T20:59:16.256628] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-01. [2025-10-20T20:59:16.311630] [INFO ] [MAIN ] SAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:16.359060] [DEBUG] [LOCAL ] SAP: Loaded 50604 existing records from primary storage. [2025-10-20T20:59:16.359373] [INFO ] [MAIN ] SAP: Loaded 50604 existing records. [2025-10-20T20:59:16.533552] [WARN ] [MAIN ] SAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:59:16.533802] [INFO ] [MAIN ] SAP: Wiping existing primary data before full refresh. [2025-10-20T20:59:16.533852] [INFO ] [LOCAL ] SAP: Starting deletion of primary data. [2025-10-20T20:59:16.534314] [DEBUG] [LOCAL ] SAP: Deleted primary data file. [2025-10-20T20:59:16.534368] [INFO ] [MAIN ] SAP: Starting full primary refresh. [2025-10-20T20:59:16.893511] [INFO ] [REMOTE ] SAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:17.051394] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-07. [2025-10-20T20:59:17.851379] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-01. [2025-10-20T20:59:18.100255] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T20:59:18.453080] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T20:59:18.617394] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T20:59:18.651362] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-02. [2025-10-20T20:59:19.435089] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T20:59:19.451335] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-03. [2025-10-20T20:59:20.237517] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T20:59:20.251343] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-04. [2025-10-20T20:59:20.811568] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T20:59:21.051423] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-05. [2025-10-20T20:59:21.830750] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T20:59:21.851332] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-06. [2025-10-20T20:59:22.595931] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T20:59:22.651349] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-07. [2025-10-20T20:59:23.417222] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T20:59:23.451339] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-08. [2025-10-20T20:59:24.251353] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-08. [2025-10-20T20:59:24.281998] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T20:59:25.051433] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-02. [2025-10-20T20:59:25.562347] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T20:59:25.851350] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-09. [2025-10-20T20:59:26.539086] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T20:59:26.651385] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-09. [2025-10-20T20:59:27.197912] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T20:59:27.451390] [INFO ] [REMOTE ] SAFE: Fetching primary data for 2025-10. [2025-10-20T20:59:27.880105] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T20:59:28.085452] [INFO ] [REMOTE ] SAFE: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T20:59:28.089296] [INFO ] [REMOTE ] SAFE: Full fetch complete (44128 records). [2025-10-20T20:59:28.089375] [DEBUG] [VALIDATOR ] SAFE: Validating primary data. [2025-10-20T20:59:28.089952] [FATAL] [VALIDATOR ] SAFE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:59:28.090047] [FATAL] [MAIN ] SAFE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:59:28.090121] [WARN ] [MAIN ] SAFE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:59:28.090209] [INFO ] [LOCAL ] SAFE: Starting deletion of primary data. [2025-10-20T20:59:28.090306] [DEBUG] [LOCAL ] SAFE: No primary data file found to delete. [2025-10-20T20:59:28.090352] [INFO ] [LOCAL ] SAFE: Starting deletion of derivative data. [2025-10-20T20:59:28.090489] [DEBUG] [LOCAL ] SAFE: Deleted derivative data file. [2025-10-20T20:59:28.090991] [INFO ] [MAIN ] Update Progress: 1858/2367 symbols completed. SAFE took 55.7s. Estimated remaining: 1h 5m 35s. Rate: 7.8 symbols/min [2025-10-20T20:59:28.257487] [INFO ] [REMOTE ] SAM: Fetching primary data for 2025-10. [2025-10-20T20:59:28.257375] [INFO ] [MAIN ] SAR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:28.280838] [DEBUG] [LOCAL ] SAR: Loaded 17471 existing records from primary storage. [2025-10-20T20:59:28.281070] [INFO ] [MAIN ] SAR: Loaded 17471 existing records. [2025-10-20T20:59:28.458118] [WARN ] [MAIN ] SAR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:59:28.458348] [INFO ] [MAIN ] SAR: Wiping existing primary data before full refresh. [2025-10-20T20:59:28.458395] [INFO ] [LOCAL ] SAR: Starting deletion of primary data. [2025-10-20T20:59:28.458557] [DEBUG] [LOCAL ] SAR: Deleted primary data file. [2025-10-20T20:59:28.458593] [INFO ] [MAIN ] SAR: Starting full primary refresh. [2025-10-20T20:59:28.731661] [INFO ] [REMOTE ] SAM: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T20:59:28.764720] [INFO ] [REMOTE ] SAM: Full fetch complete (31066 records). [2025-10-20T20:59:28.765293] [DEBUG] [VALIDATOR ] SAM: Validating primary data. [2025-10-20T20:59:28.787013] [FATAL] [VALIDATOR ] SAM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:59:28.806102] [FATAL] [MAIN ] SAM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:59:28.817684] [WARN ] [MAIN ] SAM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:59:28.827678] [INFO ] [LOCAL ] SAM: Starting deletion of primary data. [2025-10-20T20:59:28.848120] [INFO ] [REMOTE ] SAR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:28.844466] [DEBUG] [LOCAL ] SAM: No primary data file found to delete. [2025-10-20T20:59:28.848237] [INFO ] [LOCAL ] SAM: Starting deletion of derivative data. [2025-10-20T20:59:28.848629] [DEBUG] [LOCAL ] SAM: Deleted derivative data file. [2025-10-20T20:59:28.848755] [INFO ] [MAIN ] Update Progress: 1859/2367 symbols completed. SAM took 33.1s. Estimated remaining: 1h 5m 25s. Rate: 7.8 symbols/min [2025-10-20T20:59:29.029786] [INFO ] [MAIN ] SARO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:29.052693] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-03. [2025-10-20T20:59:29.073612] [DEBUG] [LOCAL ] SARO: Loaded 49102 existing records from primary storage. [2025-10-20T20:59:29.073952] [INFO ] [MAIN ] SARO: Loaded 49102 existing records. [2025-10-20T20:59:29.248770] [WARN ] [MAIN ] SARO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:59:29.249016] [INFO ] [MAIN ] SARO: Wiping existing primary data before full refresh. [2025-10-20T20:59:29.249088] [INFO ] [LOCAL ] SARO: Starting deletion of primary data. [2025-10-20T20:59:29.249300] [DEBUG] [LOCAL ] SARO: Deleted primary data file. [2025-10-20T20:59:29.249356] [INFO ] [MAIN ] SARO: Starting full primary refresh. [2025-10-20T20:59:29.600690] [INFO ] [REMOTE ] SARO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:29.851409] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-01. [2025-10-20T20:59:30.651407] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-09. [2025-10-20T20:59:31.111498] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-03 (2.06 seconds). [2025-10-20T20:59:31.198127] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T20:59:31.456436] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-04. [2025-10-20T20:59:31.471689] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T20:59:32.251387] [INFO ] [REMOTE ] SAP: Fetching primary data for 2025-10. [2025-10-20T20:59:33.051376] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-01. [2025-10-20T20:59:33.474500] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T20:59:33.851410] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-05. [2025-10-20T20:59:34.171617] [INFO ] [REMOTE ] SAP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T20:59:34.185814] [INFO ] [REMOTE ] SAP: Full fetch complete (72232 records). [2025-10-20T20:59:34.186542] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T20:59:34.186612] [DEBUG] [VALIDATOR ] SAP: Validating primary data. [2025-10-20T20:59:34.607526] [DEBUG] [VALIDATOR ] SAP: All expected trading days are present. [2025-10-20T20:59:34.607625] [DEBUG] [VALIDATOR ] SAP: Primary data validation passed. [2025-10-20T20:59:34.608811] [INFO ] [MAIN ] SAP: Fetched primary data is valid. [2025-10-20T20:59:34.608876] [DEBUG] [LOCAL ] SAP: Saving primary data. [2025-10-20T20:59:34.610234] [DEBUG] [LOCAL ] SAP: Trimmed 72232 records to 49871 (keeping 200 days of data) [2025-10-20T20:59:34.662175] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-02. [2025-10-20T20:59:34.914476] [DEBUG] [LOCAL ] SAP: Primary data saved (49871 records). [2025-10-20T20:59:34.914608] [INFO ] [MAIN ] SAP: Full history saved. [2025-10-20T20:59:34.914714] [INFO ] [MAIN ] Update Progress: 1860/2367 symbols completed. SAP took 18.8s. Estimated remaining: 1h 5m 17s. Rate: 7.8 symbols/min [2025-10-20T20:59:35.081785] [INFO ] [MAIN ] SAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:35.081941] [DEBUG] [LOCAL ] SAT: No existing data file found in primary storage. [2025-10-20T20:59:35.081979] [WARN ] [MAIN ] SAT: No existing data found, performing full refresh instead. [2025-10-20T20:59:35.082086] [INFO ] [MAIN ] SAT: Wiping existing primary data before full refresh. [2025-10-20T20:59:35.082128] [INFO ] [LOCAL ] SAT: Starting deletion of primary data. [2025-10-20T20:59:35.082170] [DEBUG] [LOCAL ] SAT: No primary data file found to delete. [2025-10-20T20:59:35.082203] [INFO ] [MAIN ] SAT: Starting full primary refresh. [2025-10-20T20:59:35.446479] [INFO ] [REMOTE ] SAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:35.451336] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-01. [2025-10-20T20:59:35.581168] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T20:59:35.676225] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T20:59:35.918729] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-10-20T20:59:36.251369] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-02. [2025-10-20T20:59:36.689803] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T20:59:37.051411] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-03. [2025-10-20T20:59:37.528756] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-03 (0.48 seconds). [2025-10-20T20:59:37.851405] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-03. [2025-10-20T20:59:38.651417] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-04. [2025-10-20T20:59:39.057672] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-03 (1.21 seconds). [2025-10-20T20:59:39.148604] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T20:59:39.451368] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-04. [2025-10-20T20:59:40.251427] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-06. [2025-10-20T20:59:40.541583] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-04 (1.09 seconds). [2025-10-20T20:59:41.051370] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-05. [2025-10-20T20:59:41.467252] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T20:59:41.851373] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-05. [2025-10-20T20:59:42.163944] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T20:59:42.633714] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T20:59:42.651333] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-06. [2025-10-20T20:59:43.281514] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T20:59:43.451445] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-02. [2025-10-20T20:59:44.251374] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-06. [2025-10-20T20:59:44.663240] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-06 (0.41 seconds). [2025-10-20T20:59:44.873339] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T20:59:45.051380] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-07. [2025-10-20T20:59:45.851386] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-07. [2025-10-20T20:59:46.250856] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T20:59:46.651563] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-03. [2025-10-20T20:59:47.080632] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T20:59:47.451410] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-08. [2025-10-20T20:59:48.251385] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-07. [2025-10-20T20:59:48.379336] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T20:59:49.051392] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-04. [2025-10-20T20:59:49.374052] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T20:59:49.435186] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T20:59:49.851414] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-09. [2025-10-20T20:59:50.482801] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T20:59:50.651366] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-08. [2025-10-20T20:59:51.451413] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-08. [2025-10-20T20:59:51.806269] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T20:59:51.804902] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-08 (0.35 seconds). [2025-10-20T20:59:51.845526] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T20:59:52.251394] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-09. [2025-10-20T20:59:53.051408] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-05. [2025-10-20T20:59:53.345528] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T20:59:53.851389] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-09. [2025-10-20T20:59:54.242270] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-10-20T20:59:54.651374] [INFO ] [REMOTE ] SAT: Fetching primary data for 2025-10. [2025-10-20T20:59:54.737323] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T20:59:55.041211] [INFO ] [REMOTE ] SAT: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-10-20T20:59:55.041963] [INFO ] [REMOTE ] SAT: Full fetch complete (1483 records). [2025-10-20T20:59:55.042032] [DEBUG] [VALIDATOR ] SAT: Validating primary data. [2025-10-20T20:59:55.042361] [FATAL] [VALIDATOR ] SAT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T20:59:55.042440] [FATAL] [MAIN ] SAT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T20:59:55.042483] [WARN ] [MAIN ] SAT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T20:59:55.042547] [INFO ] [LOCAL ] SAT: Starting deletion of primary data. [2025-10-20T20:59:55.042610] [DEBUG] [LOCAL ] SAT: No primary data file found to delete. [2025-10-20T20:59:55.042674] [INFO ] [LOCAL ] SAT: Starting deletion of derivative data. [2025-10-20T20:59:55.042738] [DEBUG] [LOCAL ] SAT: No derivative data file found to delete. [2025-10-20T20:59:55.042790] [INFO ] [MAIN ] Update Progress: 1861/2367 symbols completed. SAT took 20.1s. Estimated remaining: 1h 5m 13s. Rate: 7.8 symbols/min [2025-10-20T20:59:55.210670] [INFO ] [MAIN ] SAY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:55.210825] [DEBUG] [LOCAL ] SAY: No existing data file found in primary storage. [2025-10-20T20:59:55.210862] [WARN ] [MAIN ] SAY: No existing data found, performing full refresh instead. [2025-10-20T20:59:55.210970] [INFO ] [MAIN ] SAY: Wiping existing primary data before full refresh. [2025-10-20T20:59:55.211010] [INFO ] [LOCAL ] SAY: Starting deletion of primary data. [2025-10-20T20:59:55.211051] [DEBUG] [LOCAL ] SAY: No primary data file found to delete. [2025-10-20T20:59:55.211123] [INFO ] [MAIN ] SAY: Starting full primary refresh. [2025-10-20T20:59:55.456516] [INFO ] [REMOTE ] SAN: Fetching primary data for 2025-10. [2025-10-20T20:59:55.585413] [INFO ] [REMOTE ] SAY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:56.251366] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-06. [2025-10-20T20:59:57.051350] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-01. [2025-10-20T20:59:57.355565] [INFO ] [REMOTE ] SAN: Incremental fetch complete for 2025-10 (1.90 seconds). [2025-10-20T20:59:57.363350] [INFO ] [REMOTE ] SAN: Full fetch complete (76553 records). [2025-10-20T20:59:57.363515] [DEBUG] [VALIDATOR ] SAN: Validating primary data. [2025-10-20T20:59:57.515210] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-10-20T20:59:57.784508] [DEBUG] [VALIDATOR ] SAN: All expected trading days are present. [2025-10-20T20:59:57.784713] [DEBUG] [VALIDATOR ] SAN: Primary data validation passed. [2025-10-20T20:59:57.785881] [INFO ] [MAIN ] SAN: Fetched primary data is valid. [2025-10-20T20:59:57.785958] [DEBUG] [LOCAL ] SAN: Saving primary data. [2025-10-20T20:59:57.787391] [DEBUG] [LOCAL ] SAN: Trimmed 76553 records to 52969 (keeping 200 days of data) [2025-10-20T20:59:57.867159] [INFO ] [REMOTE ] SAR: Fetching primary data for 2025-10. [2025-10-20T20:59:57.913456] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T20:59:58.124543] [DEBUG] [LOCAL ] SAN: Primary data saved (52969 records). [2025-10-20T20:59:58.124696] [INFO ] [MAIN ] SAN: Full history saved. [2025-10-20T20:59:58.124786] [INFO ] [MAIN ] Update Progress: 1862/2367 symbols completed. SAN took 42.9s. Estimated remaining: 1h 5m 4s. Rate: 7.8 symbols/min [2025-10-20T20:59:58.309786] [INFO ] [MAIN ] SAZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:58.309974] [DEBUG] [LOCAL ] SAZ: No existing data file found in primary storage. [2025-10-20T20:59:58.310010] [WARN ] [MAIN ] SAZ: No existing data found, performing full refresh instead. [2025-10-20T20:59:58.310121] [INFO ] [MAIN ] SAZ: Wiping existing primary data before full refresh. [2025-10-20T20:59:58.310161] [INFO ] [LOCAL ] SAZ: Starting deletion of primary data. [2025-10-20T20:59:58.310201] [DEBUG] [LOCAL ] SAZ: No primary data file found to delete. [2025-10-20T20:59:58.310233] [INFO ] [MAIN ] SAZ: Starting full primary refresh. [2025-10-20T20:59:58.658749] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-02. [2025-10-20T20:59:58.658553] [INFO ] [REMOTE ] SAZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T20:59:58.982891] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T20:59:59.046743] [INFO ] [REMOTE ] SAR: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T20:59:59.050575] [INFO ] [REMOTE ] SAR: Full fetch complete (24010 records). [2025-10-20T20:59:59.050700] [DEBUG] [VALIDATOR ] SAR: Validating primary data. [2025-10-20T20:59:59.406430] [DEBUG] [VALIDATOR ] SAR: All expected trading days are present. [2025-10-20T20:59:59.406507] [DEBUG] [VALIDATOR ] SAR: Primary data validation passed. [2025-10-20T20:59:59.407015] [INFO ] [MAIN ] SAR: Fetched primary data is valid. [2025-10-20T20:59:59.407079] [DEBUG] [LOCAL ] SAR: Saving primary data. [2025-10-20T20:59:59.407687] [DEBUG] [LOCAL ] SAR: Trimmed 24010 records to 17251 (keeping 200 days of data) [2025-10-20T20:59:59.451923] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-07. [2025-10-20T20:59:59.514916] [DEBUG] [LOCAL ] SAR: Primary data saved (17251 records). [2025-10-20T20:59:59.515068] [INFO ] [MAIN ] SAR: Full history saved. [2025-10-20T20:59:59.515153] [INFO ] [MAIN ] Update Progress: 1863/2367 symbols completed. SAR took 31.4s. Estimated remaining: 1h 4m 54s. Rate: 7.8 symbols/min [2025-10-20T20:59:59.680759] [INFO ] [MAIN ] SB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T20:59:59.709973] [DEBUG] [LOCAL ] SB: Loaded 33292 existing records from primary storage. [2025-10-20T20:59:59.710234] [INFO ] [MAIN ] SB: Loaded 33292 existing records. [2025-10-20T20:59:59.887242] [WARN ] [MAIN ] SB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T20:59:59.887435] [INFO ] [MAIN ] SB: Wiping existing primary data before full refresh. [2025-10-20T20:59:59.887481] [INFO ] [LOCAL ] SB: Starting deletion of primary data. [2025-10-20T20:59:59.887767] [DEBUG] [LOCAL ] SB: Deleted primary data file. [2025-10-20T20:59:59.887823] [INFO ] [MAIN ] SB: Starting full primary refresh. [2025-10-20T21:00:00.236757] [INFO ] [REMOTE ] SB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:00.251326] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-01. [2025-10-20T21:00:01.041969] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T21:00:01.051336] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-03. [2025-10-20T21:00:01.135787] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T21:00:01.417683] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-03 (0.37 seconds). [2025-10-20T21:00:01.851352] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-02. [2025-10-20T21:00:02.651369] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-08. [2025-10-20T21:00:03.308525] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T21:00:03.451344] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-03. [2025-10-20T21:00:04.251365] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-04. [2025-10-20T21:00:04.425853] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T21:00:04.662440] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-04 (0.41 seconds). [2025-10-20T21:00:05.056530] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-01. [2025-10-20T21:00:05.078512] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:00:05.627674] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T21:00:05.851385] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-09. [2025-10-20T21:00:06.651376] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-05. [2025-10-20T21:00:07.230842] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T21:00:07.451439] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-02. [2025-10-20T21:00:07.715188] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T21:00:07.881634] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-02 (0.43 seconds). [2025-10-20T21:00:08.251369] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-06. [2025-10-20T21:00:08.617537] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-06 (0.37 seconds). [2025-10-20T21:00:09.051382] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-07. [2025-10-20T21:00:09.516164] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-07 (0.46 seconds). [2025-10-20T21:00:09.851381] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-08. [2025-10-20T21:00:10.280018] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T21:00:10.651421] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-09. [2025-10-20T21:00:11.040735] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-10-20T21:00:11.451437] [INFO ] [REMOTE ] SAY: Fetching primary data for 2025-10. [2025-10-20T21:00:11.861770] [INFO ] [REMOTE ] SAY: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T21:00:11.862515] [INFO ] [REMOTE ] SAY: Full fetch complete (1386 records). [2025-10-20T21:00:11.862580] [DEBUG] [VALIDATOR ] SAY: Validating primary data. [2025-10-20T21:00:12.041214] [WARN ] [VALIDATOR ] SAY: Validation failed - insufficient historical data. Have 194 unique days, need 200 days [2025-10-20T21:00:12.041377] [FATAL] [MAIN ] SAY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:00:12.041443] [WARN ] [MAIN ] SAY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:00:12.041490] [INFO ] [LOCAL ] SAY: Starting deletion of primary data. [2025-10-20T21:00:12.041544] [DEBUG] [LOCAL ] SAY: No primary data file found to delete. [2025-10-20T21:00:12.041578] [INFO ] [LOCAL ] SAY: Starting deletion of derivative data. [2025-10-20T21:00:12.041617] [DEBUG] [LOCAL ] SAY: No derivative data file found to delete. [2025-10-20T21:00:12.041694] [INFO ] [MAIN ] Update Progress: 1864/2367 symbols completed. SAY took 17.0s. Estimated remaining: 1h 4m 48s. Rate: 7.8 symbols/min [2025-10-20T21:00:12.208463] [INFO ] [MAIN ] SBDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:12.208617] [DEBUG] [LOCAL ] SBDS: No existing data file found in primary storage. [2025-10-20T21:00:12.208682] [WARN ] [MAIN ] SBDS: No existing data found, performing full refresh instead. [2025-10-20T21:00:12.208823] [INFO ] [MAIN ] SBDS: Wiping existing primary data before full refresh. [2025-10-20T21:00:12.208913] [INFO ] [LOCAL ] SBDS: Starting deletion of primary data. [2025-10-20T21:00:12.208970] [DEBUG] [LOCAL ] SBDS: No primary data file found to delete. [2025-10-20T21:00:12.209002] [INFO ] [MAIN ] SBDS: Starting full primary refresh. [2025-10-20T21:00:12.261264] [INFO ] [REMOTE ] SARO: Fetching primary data for 2025-10. [2025-10-20T21:00:12.568114] [INFO ] [REMOTE ] SBDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:13.051406] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-04. [2025-10-20T21:00:13.851368] [INFO ] [REMOTE ] SBDS: Fetching primary data for 2025-01. [2025-10-20T21:00:13.998627] [INFO ] [REMOTE ] SARO: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T21:00:14.006682] [INFO ] [REMOTE ] SARO: Full fetch complete (69009 records). [2025-10-20T21:00:14.006843] [DEBUG] [VALIDATOR ] SARO: Validating primary data. [2025-10-20T21:00:14.127767] [DEBUG] [REMOTE ] SBDS: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:00:14.185582] [WARN ] [REMOTE ] SBDS: No data returned for month 2025-01 [2025-10-20T21:00:14.188823] [FATAL] [REMOTE ] SBDS: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:00:14.192322] [FATAL] [MAIN ] SBDS: Failed to fetch primary data. [2025-10-20T21:00:14.242820] [WARN ] [MAIN ] SBDS: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:00:14.248472] [INFO ] [LOCAL ] SBDS: Starting deletion of primary data. [2025-10-20T21:00:14.248576] [DEBUG] [LOCAL ] SBDS: No primary data file found to delete. [2025-10-20T21:00:14.248613] [INFO ] [LOCAL ] SBDS: Starting deletion of derivative data. [2025-10-20T21:00:14.248863] [DEBUG] [LOCAL ] SBDS: No derivative data file found to delete. [2025-10-20T21:00:14.248976] [INFO ] [MAIN ] Update Progress: 1865/2367 symbols completed. SBDS took 2.2s. Estimated remaining: 1h 4m 39s. Rate: 7.8 symbols/min [2025-10-20T21:00:14.608137] [INFO ] [MAIN ] SBH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:14.617089] [DEBUG] [VALIDATOR ] SARO: All expected trading days are present. [2025-10-20T21:00:14.622422] [DEBUG] [VALIDATOR ] SARO: Primary data validation passed. [2025-10-20T21:00:14.632522] [INFO ] [MAIN ] SARO: Fetched primary data is valid. [2025-10-20T21:00:14.633639] [DEBUG] [LOCAL ] SARO: Saving primary data. [2025-10-20T21:00:14.633562] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T21:00:14.643455] [DEBUG] [LOCAL ] SARO: Trimmed 69009 records to 48307 (keeping 200 days of data) [2025-10-20T21:00:14.690451] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-03. [2025-10-20T21:00:14.961105] [DEBUG] [LOCAL ] SBH: Loaded 50615 existing records from primary storage. [2025-10-20T21:00:14.976271] [DEBUG] [LOCAL ] SARO: Primary data saved (48307 records). [2025-10-20T21:00:14.976965] [INFO ] [MAIN ] SARO: Full history saved. [2025-10-20T21:00:14.976817] [INFO ] [MAIN ] SBH: Loaded 50615 existing records. [2025-10-20T21:00:14.982246] [INFO ] [MAIN ] Update Progress: 1866/2367 symbols completed. SARO took 46.1s. Estimated remaining: 1h 4m 29s. Rate: 7.8 symbols/min [2025-10-20T21:00:15.273027] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T21:00:15.318059] [WARN ] [MAIN ] SBH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:15.361079] [INFO ] [MAIN ] SBH: Wiping existing primary data before full refresh. [2025-10-20T21:00:15.362669] [INFO ] [MAIN ] SBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:15.362874] [INFO ] [LOCAL ] SBH: Starting deletion of primary data. [2025-10-20T21:00:15.363395] [DEBUG] [LOCAL ] SBH: Deleted primary data file. [2025-10-20T21:00:15.363454] [INFO ] [MAIN ] SBH: Starting full primary refresh. [2025-10-20T21:00:15.455789] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-05. [2025-10-20T21:00:15.498377] [DEBUG] [LOCAL ] SBI: Loaded 7760 existing records from primary storage. [2025-10-20T21:00:15.521212] [INFO ] [MAIN ] SBI: Loaded 7760 existing records. [2025-10-20T21:00:15.910409] [WARN ] [MAIN ] SBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:15.914366] [INFO ] [REMOTE ] SBH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:15.914695] [INFO ] [MAIN ] SBI: Wiping existing primary data before full refresh. [2025-10-20T21:00:15.915023] [INFO ] [LOCAL ] SBI: Starting deletion of primary data. [2025-10-20T21:00:15.915186] [DEBUG] [LOCAL ] SBI: Deleted primary data file. [2025-10-20T21:00:15.915224] [INFO ] [MAIN ] SBI: Starting full primary refresh. [2025-10-20T21:00:16.256604] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-01. [2025-10-20T21:00:16.261400] [INFO ] [REMOTE ] SBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:16.957343] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T21:00:17.051423] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-04. [2025-10-20T21:00:17.565391] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-04 (0.51 seconds). [2025-10-20T21:00:17.851439] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-05. [2025-10-20T21:00:17.915940] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:00:18.287034] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T21:00:18.651421] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-02. [2025-10-20T21:00:19.451423] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-01. [2025-10-20T21:00:20.199750] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-01 (0.75 seconds). [2025-10-20T21:00:20.204011] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T21:00:20.251331] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-02. [2025-10-20T21:00:20.741005] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T21:00:21.051408] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-03. [2025-10-20T21:00:21.829595] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T21:00:21.851343] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-04. [2025-10-20T21:00:22.585973] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T21:00:22.651369] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-06. [2025-10-20T21:00:23.451391] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-06. [2025-10-20T21:00:23.874195] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-06 (0.42 seconds). [2025-10-20T21:00:24.064317] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T21:00:24.251393] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-07. [2025-10-20T21:00:25.051416] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-03. [2025-10-20T21:00:25.752585] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T21:00:25.851361] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-08. [2025-10-20T21:00:26.651427] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-05. [2025-10-20T21:00:26.845201] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T21:00:27.112227] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T21:00:27.330637] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T21:00:27.451377] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-04. [2025-10-20T21:00:28.251431] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-06. [2025-10-20T21:00:29.051387] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-07. [2025-10-20T21:00:29.224343] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T21:00:29.251809] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T21:00:29.424084] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-07 (0.37 seconds). [2025-10-20T21:00:29.851413] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-08. [2025-10-20T21:00:30.324880] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T21:00:30.651411] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-05. [2025-10-20T21:00:31.451395] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-09. [2025-10-20T21:00:31.848086] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T21:00:32.251412] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-09. [2025-10-20T21:00:32.457839] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:00:33.051390] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-06. [2025-10-20T21:00:33.463423] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T21:00:33.851388] [INFO ] [REMOTE ] SAZ: Fetching primary data for 2025-10. [2025-10-20T21:00:34.267437] [INFO ] [REMOTE ] SAZ: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T21:00:34.268298] [INFO ] [REMOTE ] SAZ: Full fetch complete (1863 records). [2025-10-20T21:00:34.268367] [DEBUG] [VALIDATOR ] SAZ: Validating primary data. [2025-10-20T21:00:34.451192] [WARN ] [VALIDATOR ] SAZ: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T21:00:34.451343] [FATAL] [MAIN ] SAZ: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:00:34.451388] [WARN ] [MAIN ] SAZ: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:00:34.451426] [INFO ] [LOCAL ] SAZ: Starting deletion of primary data. [2025-10-20T21:00:34.451477] [DEBUG] [LOCAL ] SAZ: No primary data file found to delete. [2025-10-20T21:00:34.451511] [INFO ] [LOCAL ] SAZ: Starting deletion of derivative data. [2025-10-20T21:00:34.451557] [DEBUG] [LOCAL ] SAZ: No derivative data file found to delete. [2025-10-20T21:00:34.451609] [INFO ] [MAIN ] Update Progress: 1867/2367 symbols completed. SAZ took 36.3s. Estimated remaining: 1h 4m 25s. Rate: 7.8 symbols/min [2025-10-20T21:00:34.619520] [INFO ] [MAIN ] SBR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:34.628614] [DEBUG] [LOCAL ] SBR: Loaded 7937 existing records from primary storage. [2025-10-20T21:00:34.628803] [INFO ] [MAIN ] SBR: Loaded 7937 existing records. [2025-10-20T21:00:34.659106] [INFO ] [REMOTE ] SB: Fetching primary data for 2025-10. [2025-10-20T21:00:34.809062] [WARN ] [MAIN ] SBR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:34.809413] [INFO ] [MAIN ] SBR: Wiping existing primary data before full refresh. [2025-10-20T21:00:34.809489] [INFO ] [LOCAL ] SBR: Starting deletion of primary data. [2025-10-20T21:00:34.809726] [DEBUG] [LOCAL ] SBR: Deleted primary data file. [2025-10-20T21:00:34.809788] [INFO ] [MAIN ] SBR: Starting full primary refresh. [2025-10-20T21:00:34.935791] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-06 (1.88 seconds). [2025-10-20T21:00:35.192133] [INFO ] [REMOTE ] SBR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:35.451347] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-07. [2025-10-20T21:00:35.993676] [INFO ] [REMOTE ] SB: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T21:00:35.998466] [INFO ] [REMOTE ] SB: Full fetch complete (49482 records). [2025-10-20T21:00:35.998607] [DEBUG] [VALIDATOR ] SB: Validating primary data. [2025-10-20T21:00:36.256505] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-01. [2025-10-20T21:00:36.382419] [DEBUG] [VALIDATOR ] SB: All expected trading days are present. [2025-10-20T21:00:36.382495] [DEBUG] [VALIDATOR ] SB: Primary data validation passed. [2025-10-20T21:00:36.383343] [INFO ] [MAIN ] SB: Fetched primary data is valid. [2025-10-20T21:00:36.383403] [DEBUG] [LOCAL ] SB: Saving primary data. [2025-10-20T21:00:36.384327] [DEBUG] [LOCAL ] SB: Trimmed 49482 records to 32611 (keeping 200 days of data) [2025-10-20T21:00:36.565146] [DEBUG] [LOCAL ] SB: Primary data saved (32611 records). [2025-10-20T21:00:36.565275] [INFO ] [MAIN ] SB: Full history saved. [2025-10-20T21:00:36.565696] [INFO ] [MAIN ] Update Progress: 1868/2367 symbols completed. SB took 37.1s. Estimated remaining: 1h 4m 15s. Rate: 7.8 symbols/min [2025-10-20T21:00:36.732695] [INFO ] [MAIN ] SBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:36.778456] [DEBUG] [LOCAL ] SBS: Loaded 50057 existing records from primary storage. [2025-10-20T21:00:36.778795] [INFO ] [MAIN ] SBS: Loaded 50057 existing records. [2025-10-20T21:00:36.961065] [WARN ] [MAIN ] SBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:36.961271] [INFO ] [MAIN ] SBS: Wiping existing primary data before full refresh. [2025-10-20T21:00:36.961318] [INFO ] [LOCAL ] SBS: Starting deletion of primary data. [2025-10-20T21:00:36.961518] [DEBUG] [LOCAL ] SBS: Deleted primary data file. [2025-10-20T21:00:36.961574] [INFO ] [MAIN ] SBS: Starting full primary refresh. [2025-10-20T21:00:37.060239] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-07. [2025-10-20T21:00:37.209295] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T21:00:37.378920] [INFO ] [REMOTE ] SBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:37.373686] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T21:00:37.697248] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T21:00:37.851427] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-08. [2025-10-20T21:00:38.651377] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-02. [2025-10-20T21:00:39.451393] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-01. [2025-10-20T21:00:39.470592] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T21:00:39.787503] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T21:00:40.251428] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-03. [2025-10-20T21:00:41.056629] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-08. [2025-10-20T21:00:41.078316] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:00:41.098031] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T21:00:41.851385] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-02. [2025-10-20T21:00:41.895791] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T21:00:42.651458] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-09. [2025-10-20T21:00:43.372316] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T21:00:43.451419] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-03. [2025-10-20T21:00:44.251440] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-09. [2025-10-20T21:00:44.554797] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-09 (1.90 seconds). [2025-10-20T21:00:44.902253] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T21:00:45.051411] [INFO ] [REMOTE ] SBI: Fetching primary data for 2025-10. [2025-10-20T21:00:45.268026] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:00:45.714954] [INFO ] [REMOTE ] SBI: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T21:00:45.716718] [INFO ] [REMOTE ] SBI: Full fetch complete (10585 records). [2025-10-20T21:00:45.716791] [DEBUG] [VALIDATOR ] SBI: Validating primary data. [2025-10-20T21:00:45.856447] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-04. [2025-10-20T21:00:46.080765] [DEBUG] [VALIDATOR ] SBI: All expected trading days are present. [2025-10-20T21:00:46.080846] [DEBUG] [VALIDATOR ] SBI: Primary data validation passed. [2025-10-20T21:00:46.081124] [INFO ] [MAIN ] SBI: Fetched primary data is valid. [2025-10-20T21:00:46.081161] [DEBUG] [LOCAL ] SBI: Saving primary data. [2025-10-20T21:00:46.081599] [DEBUG] [LOCAL ] SBI: Trimmed 10585 records to 7716 (keeping 200 days of data) [2025-10-20T21:00:46.127785] [DEBUG] [LOCAL ] SBI: Primary data saved (7716 records). [2025-10-20T21:00:46.127966] [INFO ] [MAIN ] SBI: Full history saved. [2025-10-20T21:00:46.128070] [INFO ] [MAIN ] Update Progress: 1869/2367 symbols completed. SBI took 31.1s. Estimated remaining: 1h 4m 8s. Rate: 7.8 symbols/min [2025-10-20T21:00:46.294346] [INFO ] [MAIN ] SBSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:46.310434] [DEBUG] [LOCAL ] SBSI: Loaded 15203 existing records from primary storage. [2025-10-20T21:00:46.310681] [INFO ] [MAIN ] SBSI: Loaded 15203 existing records. [2025-10-20T21:00:46.488211] [WARN ] [MAIN ] SBSI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:46.488463] [INFO ] [MAIN ] SBSI: Wiping existing primary data before full refresh. [2025-10-20T21:00:46.488522] [INFO ] [LOCAL ] SBSI: Starting deletion of primary data. [2025-10-20T21:00:46.488722] [DEBUG] [LOCAL ] SBSI: Deleted primary data file. [2025-10-20T21:00:46.488781] [INFO ] [MAIN ] SBSI: Starting full primary refresh. [2025-10-20T21:00:46.652712] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-04. [2025-10-20T21:00:46.868552] [INFO ] [REMOTE ] SBSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:47.370195] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T21:00:47.451354] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-05. [2025-10-20T21:00:47.765005] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T21:00:48.251356] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-01. [2025-10-20T21:00:48.251589] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T21:00:49.051349] [INFO ] [REMOTE ] SBH: Fetching primary data for 2025-10. [2025-10-20T21:00:49.210255] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T21:00:49.456854] [INFO ] [REMOTE ] SBH: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T21:00:49.464077] [INFO ] [REMOTE ] SBH: Full fetch complete (72694 records). [2025-10-20T21:00:49.464216] [DEBUG] [VALIDATOR ] SBH: Validating primary data. [2025-10-20T21:00:49.465055] [FATAL] [VALIDATOR ] SBH: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:00:49.465161] [FATAL] [MAIN ] SBH: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:00:49.465230] [WARN ] [MAIN ] SBH: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:00:49.465294] [INFO ] [LOCAL ] SBH: Starting deletion of primary data. [2025-10-20T21:00:49.465357] [DEBUG] [LOCAL ] SBH: No primary data file found to delete. [2025-10-20T21:00:49.465391] [INFO ] [LOCAL ] SBH: Starting deletion of derivative data. [2025-10-20T21:00:49.465533] [DEBUG] [LOCAL ] SBH: Deleted derivative data file. [2025-10-20T21:00:49.465621] [INFO ] [MAIN ] Update Progress: 1870/2367 symbols completed. SBH took 35.2s. Estimated remaining: 1h 3m 59s. Rate: 7.8 symbols/min [2025-10-20T21:00:49.633881] [INFO ] [MAIN ] SBSW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:00:49.682834] [DEBUG] [LOCAL ] SBSW: Loaded 53952 existing records from primary storage. [2025-10-20T21:00:49.683113] [INFO ] [MAIN ] SBSW: Loaded 53952 existing records. [2025-10-20T21:00:49.857611] [WARN ] [MAIN ] SBSW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:00:49.857690] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-02. [2025-10-20T21:00:49.858726] [INFO ] [MAIN ] SBSW: Wiping existing primary data before full refresh. [2025-10-20T21:00:49.858783] [INFO ] [LOCAL ] SBSW: Starting deletion of primary data. [2025-10-20T21:00:49.858992] [DEBUG] [LOCAL ] SBSW: Deleted primary data file. [2025-10-20T21:00:49.859046] [INFO ] [MAIN ] SBSW: Starting full primary refresh. [2025-10-20T21:00:50.216705] [INFO ] [REMOTE ] SBSW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:00:50.651368] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-05. [2025-10-20T21:00:50.814490] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T21:00:51.451448] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-03. [2025-10-20T21:00:52.251467] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-06. [2025-10-20T21:00:52.335920] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-03 (0.88 seconds). [2025-10-20T21:00:52.457890] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:00:53.046863] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T21:00:53.051317] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-07. [2025-10-20T21:00:53.743180] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T21:00:53.851350] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-08. [2025-10-20T21:00:54.651353] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-06. [2025-10-20T21:00:54.698243] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:00:55.451392] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-01. [2025-10-20T21:00:56.108286] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T21:00:56.251379] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-07. [2025-10-20T21:00:57.051385] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-09. [2025-10-20T21:00:57.355990] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T21:00:57.764849] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T21:00:57.851337] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-08. [2025-10-20T21:00:57.914056] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T21:00:58.651415] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-04. [2025-10-20T21:00:59.451348] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-02. [2025-10-20T21:00:59.631097] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T21:00:59.667848] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T21:01:00.251396] [INFO ] [REMOTE ] SBR: Fetching primary data for 2025-10. [2025-10-20T21:01:01.051360] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-05. [2025-10-20T21:01:01.199420] [INFO ] [REMOTE ] SBR: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T21:01:01.201478] [INFO ] [REMOTE ] SBR: Full fetch complete (11351 records). [2025-10-20T21:01:01.201549] [DEBUG] [VALIDATOR ] SBR: Validating primary data. [2025-10-20T21:01:01.565721] [DEBUG] [VALIDATOR ] SBR: All expected trading days are present. [2025-10-20T21:01:01.565900] [DEBUG] [VALIDATOR ] SBR: Primary data validation passed. [2025-10-20T21:01:01.566227] [INFO ] [MAIN ] SBR: Fetched primary data is valid. [2025-10-20T21:01:01.566300] [DEBUG] [LOCAL ] SBR: Saving primary data. [2025-10-20T21:01:01.576268] [DEBUG] [LOCAL ] SBR: Trimmed 11351 records to 7870 (keeping 200 days of data) [2025-10-20T21:01:01.614887] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-02 (2.16 seconds). [2025-10-20T21:01:01.651723] [DEBUG] [LOCAL ] SBR: Primary data saved (7870 records). [2025-10-20T21:01:01.651832] [INFO ] [MAIN ] SBR: Full history saved. [2025-10-20T21:01:01.651904] [INFO ] [MAIN ] Update Progress: 1871/2367 symbols completed. SBR took 27.2s. Estimated remaining: 1h 3m 53s. Rate: 7.8 symbols/min [2025-10-20T21:01:01.838596] [INFO ] [MAIN ] SBXD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:01.835545] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T21:01:01.838847] [DEBUG] [LOCAL ] SBXD: No existing data file found in primary storage. [2025-10-20T21:01:01.841972] [WARN ] [MAIN ] SBXD: No existing data found, performing full refresh instead. [2025-10-20T21:01:01.842354] [INFO ] [MAIN ] SBXD: Wiping existing primary data before full refresh. [2025-10-20T21:01:01.842402] [INFO ] [LOCAL ] SBXD: Starting deletion of primary data. [2025-10-20T21:01:01.842446] [DEBUG] [LOCAL ] SBXD: No primary data file found to delete. [2025-10-20T21:01:01.842476] [INFO ] [MAIN ] SBXD: Starting full primary refresh. [2025-10-20T21:01:01.856566] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-03. [2025-10-20T21:01:02.245165] [INFO ] [REMOTE ] SBXD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:02.651355] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-01. [2025-10-20T21:01:03.071826] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T21:01:03.451363] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-02. [2025-10-20T21:01:03.827232] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T21:01:03.843676] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T21:01:04.251366] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-03. [2025-10-20T21:01:04.545075] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-03 (0.29 seconds). [2025-10-20T21:01:05.051369] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-06. [2025-10-20T21:01:05.851340] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-09. [2025-10-20T21:01:05.905284] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T21:01:06.651405] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-04. [2025-10-20T21:01:07.451443] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-04. [2025-10-20T21:01:07.673414] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T21:01:07.755602] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-04 (0.30 seconds). [2025-10-20T21:01:08.251372] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-05. [2025-10-20T21:01:08.674567] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-05 (0.42 seconds). [2025-10-20T21:01:08.730319] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T21:01:09.051420] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-05. [2025-10-20T21:01:09.851394] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-06. [2025-10-20T21:01:10.651401] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-07. [2025-10-20T21:01:11.451367] [INFO ] [REMOTE ] SBS: Fetching primary data for 2025-10. [2025-10-20T21:01:11.862853] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T21:01:11.873043] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-05 (2.82 seconds). [2025-10-20T21:01:11.894883] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-06 (2.04 seconds). [2025-10-20T21:01:12.251408] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-07. [2025-10-20T21:01:12.941041] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-07 (0.69 seconds). [2025-10-20T21:01:13.051381] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-08. [2025-10-20T21:01:13.857730] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-06. [2025-10-20T21:01:13.861921] [INFO ] [REMOTE ] SBS: Incremental fetch complete for 2025-10 (2.41 seconds). [2025-10-20T21:01:13.869466] [INFO ] [REMOTE ] SBS: Full fetch complete (70632 records). [2025-10-20T21:01:13.869570] [DEBUG] [VALIDATOR ] SBS: Validating primary data. [2025-10-20T21:01:14.263986] [DEBUG] [VALIDATOR ] SBS: All expected trading days are present. [2025-10-20T21:01:14.264114] [DEBUG] [VALIDATOR ] SBS: Primary data validation passed. [2025-10-20T21:01:14.265261] [INFO ] [MAIN ] SBS: Fetched primary data is valid. [2025-10-20T21:01:14.265328] [DEBUG] [LOCAL ] SBS: Saving primary data. [2025-10-20T21:01:14.266708] [DEBUG] [LOCAL ] SBS: Trimmed 70632 records to 49323 (keeping 200 days of data) [2025-10-20T21:01:14.475480] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T21:01:14.567060] [DEBUG] [LOCAL ] SBS: Primary data saved (49323 records). [2025-10-20T21:01:14.567191] [INFO ] [MAIN ] SBS: Full history saved. [2025-10-20T21:01:14.567516] [INFO ] [MAIN ] Update Progress: 1872/2367 symbols completed. SBS took 38.0s. Estimated remaining: 1h 3m 46s. Rate: 7.8 symbols/min [2025-10-20T21:01:14.656454] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-09. [2025-10-20T21:01:14.749352] [INFO ] [MAIN ] SCCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:14.796031] [DEBUG] [LOCAL ] SCCO: Loaded 50329 existing records from primary storage. [2025-10-20T21:01:14.796362] [INFO ] [MAIN ] SCCO: Loaded 50329 existing records. [2025-10-20T21:01:14.979117] [WARN ] [MAIN ] SCCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:01:14.979349] [INFO ] [MAIN ] SCCO: Wiping existing primary data before full refresh. [2025-10-20T21:01:14.979407] [INFO ] [LOCAL ] SCCO: Starting deletion of primary data. [2025-10-20T21:01:14.979619] [DEBUG] [LOCAL ] SCCO: Deleted primary data file. [2025-10-20T21:01:14.979717] [INFO ] [MAIN ] SCCO: Starting full primary refresh. [2025-10-20T21:01:15.346985] [INFO ] [REMOTE ] SCCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:15.451341] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-01. [2025-10-20T21:01:15.750326] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T21:01:15.792363] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-06 (1.93 seconds). [2025-10-20T21:01:16.251373] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-07. [2025-10-20T21:01:17.051404] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-08. [2025-10-20T21:01:17.203133] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:01:17.688873] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T21:01:17.851359] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-09. [2025-10-20T21:01:18.061963] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T21:01:18.251295] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T21:01:18.651419] [INFO ] [REMOTE ] SBXD: Fetching primary data for 2025-10. [2025-10-20T21:01:19.052238] [INFO ] [REMOTE ] SBXD: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T21:01:19.052967] [INFO ] [REMOTE ] SBXD: Full fetch complete (3126 records). [2025-10-20T21:01:19.053034] [DEBUG] [VALIDATOR ] SBXD: Validating primary data. [2025-10-20T21:01:19.228541] [WARN ] [VALIDATOR ] SBXD: Validation failed - insufficient historical data. Have 178 unique days, need 200 days [2025-10-20T21:01:19.228773] [FATAL] [MAIN ] SBXD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:01:19.228840] [WARN ] [MAIN ] SBXD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:01:19.228910] [INFO ] [LOCAL ] SBXD: Starting deletion of primary data. [2025-10-20T21:01:19.228972] [DEBUG] [LOCAL ] SBXD: No primary data file found to delete. [2025-10-20T21:01:19.229005] [INFO ] [LOCAL ] SBXD: Starting deletion of derivative data. [2025-10-20T21:01:19.229042] [DEBUG] [LOCAL ] SBXD: No derivative data file found to delete. [2025-10-20T21:01:19.229094] [INFO ] [MAIN ] Update Progress: 1873/2367 symbols completed. SBXD took 17.6s. Estimated remaining: 1h 3m 38s. Rate: 7.8 symbols/min [2025-10-20T21:01:19.398963] [INFO ] [MAIN ] SCD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:19.413937] [DEBUG] [LOCAL ] SCD: Loaded 13601 existing records from primary storage. [2025-10-20T21:01:19.414135] [INFO ] [MAIN ] SCD: Loaded 13601 existing records. [2025-10-20T21:01:19.456608] [INFO ] [REMOTE ] SBSI: Fetching primary data for 2025-10. [2025-10-20T21:01:19.594974] [WARN ] [MAIN ] SCD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:01:19.595192] [INFO ] [MAIN ] SCD: Wiping existing primary data before full refresh. [2025-10-20T21:01:19.595240] [INFO ] [LOCAL ] SCD: Starting deletion of primary data. [2025-10-20T21:01:19.595454] [DEBUG] [LOCAL ] SCD: Deleted primary data file. [2025-10-20T21:01:19.595512] [INFO ] [MAIN ] SCD: Starting full primary refresh. [2025-10-20T21:01:19.964039] [INFO ] [REMOTE ] SCD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:20.081881] [INFO ] [REMOTE ] SBSI: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T21:01:20.084557] [INFO ] [REMOTE ] SBSI: Full fetch complete (21237 records). [2025-10-20T21:01:20.084684] [DEBUG] [VALIDATOR ] SBSI: Validating primary data. [2025-10-20T21:01:20.085177] [FATAL] [VALIDATOR ] SBSI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:01:20.085261] [FATAL] [MAIN ] SBSI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:01:20.085309] [WARN ] [MAIN ] SBSI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:01:20.085390] [INFO ] [LOCAL ] SBSI: Starting deletion of primary data. [2025-10-20T21:01:20.085464] [DEBUG] [LOCAL ] SBSI: No primary data file found to delete. [2025-10-20T21:01:20.085511] [INFO ] [LOCAL ] SBSI: Starting deletion of derivative data. [2025-10-20T21:01:20.085737] [DEBUG] [LOCAL ] SBSI: Deleted derivative data file. [2025-10-20T21:01:20.085841] [INFO ] [MAIN ] Update Progress: 1874/2367 symbols completed. SBSI took 34.0s. Estimated remaining: 1h 3m 28s. Rate: 7.8 symbols/min [2025-10-20T21:01:20.256915] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-08. [2025-10-20T21:01:20.260633] [INFO ] [MAIN ] SCHW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:20.313493] [DEBUG] [LOCAL ] SCHW: Loaded 54465 existing records from primary storage. [2025-10-20T21:01:20.313827] [INFO ] [MAIN ] SCHW: Loaded 54465 existing records. [2025-10-20T21:01:20.490904] [WARN ] [MAIN ] SCHW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:01:20.491130] [INFO ] [MAIN ] SCHW: Wiping existing primary data before full refresh. [2025-10-20T21:01:20.491196] [INFO ] [LOCAL ] SCHW: Starting deletion of primary data. [2025-10-20T21:01:20.491448] [DEBUG] [LOCAL ] SCHW: Deleted primary data file. [2025-10-20T21:01:20.491513] [INFO ] [MAIN ] SCHW: Starting full primary refresh. [2025-10-20T21:01:20.859949] [INFO ] [REMOTE ] SCHW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:21.051410] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-01. [2025-10-20T21:01:21.851374] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-01. [2025-10-20T21:01:22.105742] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T21:01:22.308742] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T21:01:22.651370] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-02. [2025-10-20T21:01:23.445577] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-02 (0.79 seconds). [2025-10-20T21:01:23.451318] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-03. [2025-10-20T21:01:23.881830] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-01 (2.03 seconds). [2025-10-20T21:01:24.086123] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-03 (0.63 seconds). [2025-10-20T21:01:24.251392] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-04. [2025-10-20T21:01:24.944557] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T21:01:25.051397] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-05. [2025-10-20T21:01:25.851400] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-09. [2025-10-20T21:01:25.943583] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-05 (0.89 seconds). [2025-10-20T21:01:26.651406] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-06. [2025-10-20T21:01:27.451352] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-02. [2025-10-20T21:01:27.739847] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T21:01:28.004736] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-09 (2.15 seconds). [2025-10-20T21:01:28.251384] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-07. [2025-10-20T21:01:29.051382] [INFO ] [REMOTE ] SBSW: Fetching primary data for 2025-10. [2025-10-20T21:01:29.258417] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T21:01:29.310755] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-07 (1.06 seconds). [2025-10-20T21:01:29.408561] [INFO ] [REMOTE ] SBSW: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T21:01:29.416105] [INFO ] [REMOTE ] SBSW: Full fetch complete (76351 records). [2025-10-20T21:01:29.416244] [DEBUG] [VALIDATOR ] SBSW: Validating primary data. [2025-10-20T21:01:29.417216] [FATAL] [VALIDATOR ] SBSW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:01:29.417324] [FATAL] [MAIN ] SBSW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:01:29.417427] [WARN ] [MAIN ] SBSW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:01:29.417506] [INFO ] [LOCAL ] SBSW: Starting deletion of primary data. [2025-10-20T21:01:29.417596] [DEBUG] [LOCAL ] SBSW: No primary data file found to delete. [2025-10-20T21:01:29.417670] [INFO ] [LOCAL ] SBSW: Starting deletion of derivative data. [2025-10-20T21:01:29.417964] [DEBUG] [LOCAL ] SBSW: Deleted derivative data file. [2025-10-20T21:01:29.418073] [INFO ] [MAIN ] Update Progress: 1875/2367 symbols completed. SBSW took 40.0s. Estimated remaining: 1h 3m 21s. Rate: 7.8 symbols/min [2025-10-20T21:01:29.584312] [INFO ] [MAIN ] SCI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:29.625502] [DEBUG] [LOCAL ] SCI: Loaded 47084 existing records from primary storage. [2025-10-20T21:01:29.625826] [INFO ] [MAIN ] SCI: Loaded 47084 existing records. [2025-10-20T21:01:29.823085] [WARN ] [MAIN ] SCI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:01:29.823371] [INFO ] [MAIN ] SCI: Wiping existing primary data before full refresh. [2025-10-20T21:01:29.823444] [INFO ] [LOCAL ] SCI: Starting deletion of primary data. [2025-10-20T21:01:29.823707] [DEBUG] [LOCAL ] SCI: Deleted primary data file. [2025-10-20T21:01:29.823779] [INFO ] [MAIN ] SCI: Starting full primary refresh. [2025-10-20T21:01:29.861925] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-02. [2025-10-20T21:01:30.181286] [INFO ] [REMOTE ] SCI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:30.651435] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-03. [2025-10-20T21:01:31.451360] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-08. [2025-10-20T21:01:31.741519] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-02 (1.88 seconds). [2025-10-20T21:01:32.251348] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-03. [2025-10-20T21:01:32.336928] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:01:32.442829] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T21:01:33.051368] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-04. [2025-10-20T21:01:33.851399] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-09. [2025-10-20T21:01:33.897340] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T21:01:34.651359] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-04. [2025-10-20T21:01:34.763587] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T21:01:35.130831] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T21:01:35.451403] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-05. [2025-10-20T21:01:36.251425] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-01. [2025-10-20T21:01:36.699958] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T21:01:36.982603] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:01:37.051348] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-06. [2025-10-20T21:01:37.838003] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T21:01:37.851332] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-02. [2025-10-20T21:01:38.651351] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-05. [2025-10-20T21:01:38.915730] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T21:01:39.451406] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-07. [2025-10-20T21:01:39.606234] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T21:01:40.251368] [INFO ] [REMOTE ] SCD: Fetching primary data for 2025-10. [2025-10-20T21:01:40.577487] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:01:41.051363] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-03. [2025-10-20T21:01:41.162913] [INFO ] [REMOTE ] SCD: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T21:01:41.165500] [INFO ] [REMOTE ] SCD: Full fetch complete (17077 records). [2025-10-20T21:01:41.165580] [DEBUG] [VALIDATOR ] SCD: Validating primary data. [2025-10-20T21:01:41.562528] [DEBUG] [VALIDATOR ] SCD: All expected trading days are present. [2025-10-20T21:01:41.562614] [DEBUG] [VALIDATOR ] SCD: Primary data validation passed. [2025-10-20T21:01:41.563000] [INFO ] [MAIN ] SCD: Fetched primary data is valid. [2025-10-20T21:01:41.563053] [DEBUG] [LOCAL ] SCD: Saving primary data. [2025-10-20T21:01:41.563623] [DEBUG] [LOCAL ] SCD: Trimmed 17077 records to 13520 (keeping 200 days of data) [2025-10-20T21:01:41.658701] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-07 (2.21 seconds). [2025-10-20T21:01:41.691793] [DEBUG] [LOCAL ] SCD: Primary data saved (13520 records). [2025-10-20T21:01:41.691928] [INFO ] [MAIN ] SCD: Full history saved. [2025-10-20T21:01:41.692002] [INFO ] [MAIN ] Update Progress: 1876/2367 symbols completed. SCD took 22.5s. Estimated remaining: 1h 3m 14s. Rate: 7.8 symbols/min [2025-10-20T21:01:41.859519] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-08. [2025-10-20T21:01:41.893633] [INFO ] [MAIN ] SCL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:01:41.909839] [DEBUG] [LOCAL ] SCL: Loaded 15195 existing records from primary storage. [2025-10-20T21:01:41.910153] [INFO ] [MAIN ] SCL: Loaded 15195 existing records. [2025-10-20T21:01:42.087684] [WARN ] [MAIN ] SCL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:01:42.087924] [INFO ] [MAIN ] SCL: Wiping existing primary data before full refresh. [2025-10-20T21:01:42.087982] [INFO ] [LOCAL ] SCL: Starting deletion of primary data. [2025-10-20T21:01:42.088133] [DEBUG] [LOCAL ] SCL: Deleted primary data file. [2025-10-20T21:01:42.088184] [INFO ] [MAIN ] SCL: Starting full primary refresh. [2025-10-20T21:01:42.457759] [INFO ] [REMOTE ] SCL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:01:42.488311] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:01:42.651380] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-06. [2025-10-20T21:01:43.451411] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-01. [2025-10-20T21:01:43.688837] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T21:01:44.093145] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T21:01:44.169191] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T21:01:44.251349] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-07. [2025-10-20T21:01:45.051366] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-09. [2025-10-20T21:01:45.851343] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-04. [2025-10-20T21:01:46.407397] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T21:01:46.548126] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T21:01:46.651402] [INFO ] [REMOTE ] SCCO: Fetching primary data for 2025-10. [2025-10-20T21:01:47.324438] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T21:01:47.451379] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-02. [2025-10-20T21:01:47.807320] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-02 (0.36 seconds). [2025-10-20T21:01:48.141851] [INFO ] [REMOTE ] SCCO: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T21:01:48.149407] [INFO ] [REMOTE ] SCCO: Full fetch complete (71858 records). [2025-10-20T21:01:48.149534] [DEBUG] [VALIDATOR ] SCCO: Validating primary data. [2025-10-20T21:01:48.251392] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-05. [2025-10-20T21:01:48.541168] [DEBUG] [VALIDATOR ] SCCO: All expected trading days are present. [2025-10-20T21:01:48.541264] [DEBUG] [VALIDATOR ] SCCO: Primary data validation passed. [2025-10-20T21:01:48.542396] [INFO ] [MAIN ] SCCO: Fetched primary data is valid. [2025-10-20T21:01:48.542461] [DEBUG] [LOCAL ] SCCO: Saving primary data. [2025-10-20T21:01:48.543867] [DEBUG] [LOCAL ] SCCO: Trimmed 71858 records to 49602 (keeping 200 days of data) [2025-10-20T21:01:48.872346] [DEBUG] [LOCAL ] SCCO: Primary data saved (49602 records). [2025-10-20T21:01:48.872507] [INFO ] [MAIN ] SCCO: Full history saved. [2025-10-20T21:01:48.872937] [INFO ] [MAIN ] Update Progress: 1877/2367 symbols completed. SCCO took 34.3s. Estimated remaining: 1h 3m 6s. Rate: 7.8 symbols/min [2025-10-20T21:01:49.051474] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-08. [2025-10-20T21:01:49.851462] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-03. [2025-10-20T21:01:50.227335] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T21:01:50.641025] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T21:01:50.651331] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-04. [2025-10-20T21:01:51.124876] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T21:01:51.451372] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-09. [2025-10-20T21:01:51.834268] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T21:01:51.991939] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-10-20T21:01:52.251380] [INFO ] [REMOTE ] SCHW: Fetching primary data for 2025-10. [2025-10-20T21:01:52.610504] [INFO ] [REMOTE ] SCHW: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T21:01:52.618024] [INFO ] [REMOTE ] SCHW: Full fetch complete (77475 records). [2025-10-20T21:01:52.618186] [DEBUG] [VALIDATOR ] SCHW: Validating primary data. [2025-10-20T21:01:52.619027] [FATAL] [VALIDATOR ] SCHW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:01:52.619119] [FATAL] [MAIN ] SCHW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:01:52.619165] [WARN ] [MAIN ] SCHW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:01:52.619203] [INFO ] [LOCAL ] SCHW: Starting deletion of primary data. [2025-10-20T21:01:52.619261] [DEBUG] [LOCAL ] SCHW: No primary data file found to delete. [2025-10-20T21:01:52.619297] [INFO ] [LOCAL ] SCHW: Starting deletion of derivative data. [2025-10-20T21:01:52.619449] [DEBUG] [LOCAL ] SCHW: Deleted derivative data file. [2025-10-20T21:01:52.619539] [INFO ] [MAIN ] Update Progress: 1878/2367 symbols completed. SCHW took 32.5s. Estimated remaining: 1h 2m 58s. Rate: 7.8 symbols/min [2025-10-20T21:01:53.051394] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-06. [2025-10-20T21:01:53.851422] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-05. [2025-10-20T21:01:54.474294] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T21:01:54.651350] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-07. [2025-10-20T21:01:54.964101] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T21:01:55.451370] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-06. [2025-10-20T21:01:56.177588] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:01:56.251360] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-07. [2025-10-20T21:01:56.294462] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T21:01:57.051360] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-08. [2025-10-20T21:01:57.147596] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T21:01:57.851408] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-08. [2025-10-20T21:01:58.682117] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:01:58.691820] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-09. [2025-10-20T21:01:58.942211] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T21:01:59.451424] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-09. [2025-10-20T21:02:00.096206] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:02:00.251370] [INFO ] [REMOTE ] SCI: Fetching primary data for 2025-10. [2025-10-20T21:02:00.548462] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T21:02:01.051383] [INFO ] [REMOTE ] SCL: Fetching primary data for 2025-10. [2025-10-20T21:02:01.119609] [INFO ] [REMOTE ] SCI: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T21:02:01.125910] [INFO ] [REMOTE ] SCI: Full fetch complete (67358 records). [2025-10-20T21:02:01.126055] [DEBUG] [VALIDATOR ] SCI: Validating primary data. [2025-10-20T21:02:01.126800] [FATAL] [VALIDATOR ] SCI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:02:01.126886] [FATAL] [MAIN ] SCI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:02:01.126928] [WARN ] [MAIN ] SCI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:02:01.126964] [INFO ] [LOCAL ] SCI: Starting deletion of primary data. [2025-10-20T21:02:01.127019] [DEBUG] [LOCAL ] SCI: No primary data file found to delete. [2025-10-20T21:02:01.127052] [INFO ] [LOCAL ] SCI: Starting deletion of derivative data. [2025-10-20T21:02:01.127178] [DEBUG] [LOCAL ] SCI: Deleted derivative data file. [2025-10-20T21:02:01.127243] [INFO ] [MAIN ] Update Progress: 1879/2367 symbols completed. SCI took 31.7s. Estimated remaining: 1h 2m 50s. Rate: 7.8 symbols/min [2025-10-20T21:02:01.391005] [INFO ] [REMOTE ] SCL: Incremental fetch complete for 2025-10 (0.34 seconds). [2025-10-20T21:02:01.395109] [INFO ] [REMOTE ] SCL: Full fetch complete (21428 records). [2025-10-20T21:02:01.395281] [DEBUG] [VALIDATOR ] SCL: Validating primary data. [2025-10-20T21:02:01.395958] [FATAL] [VALIDATOR ] SCL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:02:01.396061] [FATAL] [MAIN ] SCL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:02:01.396117] [WARN ] [MAIN ] SCL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:02:01.396194] [INFO ] [LOCAL ] SCL: Starting deletion of primary data. [2025-10-20T21:02:01.396277] [DEBUG] [LOCAL ] SCL: No primary data file found to delete. [2025-10-20T21:02:01.396326] [INFO ] [LOCAL ] SCL: Starting deletion of derivative data. [2025-10-20T21:02:01.396466] [DEBUG] [LOCAL ] SCL: Deleted derivative data file. [2025-10-20T21:02:01.396559] [INFO ] [MAIN ] Update Progress: 1880/2367 symbols completed. SCL took 19.7s. Estimated remaining: 1h 2m 41s. Rate: 7.8 symbols/min [2025-10-20T21:02:02.098500] [INFO ] [MAIN ] SDHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:02.094370] [INFO ] [MAIN ] SCM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:02.144377] [INFO ] [MAIN ] SD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:02.179241] [INFO ] [MAIN ] SCS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:02.192142] [DEBUG] [LOCAL ] SDHC: Loaded 10499 existing records from primary storage. [2025-10-20T21:02:02.192620] [INFO ] [MAIN ] SDHC: Loaded 10499 existing records. [2025-10-20T21:02:02.304929] [DEBUG] [LOCAL ] SCM: Loaded 20521 existing records from primary storage. [2025-10-20T21:02:02.324745] [INFO ] [MAIN ] SCM: Loaded 20521 existing records. [2025-10-20T21:02:02.524685] [DEBUG] [LOCAL ] SD: Loaded 29067 existing records from primary storage. [2025-10-20T21:02:02.526234] [DEBUG] [LOCAL ] SCS: Loaded 44145 existing records from primary storage. [2025-10-20T21:02:02.547991] [INFO ] [MAIN ] SCS: Loaded 44145 existing records. [2025-10-20T21:02:02.547631] [WARN ] [MAIN ] SDHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:02.574294] [INFO ] [MAIN ] SD: Loaded 29067 existing records. [2025-10-20T21:02:02.622765] [INFO ] [MAIN ] SDHC: Wiping existing primary data before full refresh. [2025-10-20T21:02:02.689266] [WARN ] [MAIN ] SCM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:02.720522] [INFO ] [MAIN ] SCM: Wiping existing primary data before full refresh. [2025-10-20T21:02:02.674778] [INFO ] [LOCAL ] SDHC: Starting deletion of primary data. [2025-10-20T21:02:02.773758] [INFO ] [LOCAL ] SCM: Starting deletion of primary data. [2025-10-20T21:02:02.806434] [DEBUG] [LOCAL ] SCM: Deleted primary data file. [2025-10-20T21:02:02.788996] [DEBUG] [LOCAL ] SDHC: Deleted primary data file. [2025-10-20T21:02:02.806612] [INFO ] [MAIN ] SDHC: Starting full primary refresh. [2025-10-20T21:02:02.827888] [INFO ] [MAIN ] SCM: Starting full primary refresh. [2025-10-20T21:02:03.172784] [WARN ] [MAIN ] SCS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:03.194473] [INFO ] [MAIN ] SCS: Wiping existing primary data before full refresh. [2025-10-20T21:02:03.209270] [INFO ] [LOCAL ] SCS: Starting deletion of primary data. [2025-10-20T21:02:03.240591] [WARN ] [MAIN ] SD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:03.271731] [DEBUG] [LOCAL ] SCS: Deleted primary data file. [2025-10-20T21:02:03.271888] [INFO ] [MAIN ] SD: Wiping existing primary data before full refresh. [2025-10-20T21:02:03.284860] [INFO ] [LOCAL ] SD: Starting deletion of primary data. [2025-10-20T21:02:03.317785] [INFO ] [MAIN ] SCS: Starting full primary refresh. [2025-10-20T21:02:03.407845] [DEBUG] [LOCAL ] SD: Deleted primary data file. [2025-10-20T21:02:03.474944] [INFO ] [MAIN ] SD: Starting full primary refresh. [2025-10-20T21:02:04.072438] [INFO ] [REMOTE ] SDHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:04.129254] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-01. [2025-10-20T21:02:04.175895] [INFO ] [REMOTE ] SCM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:04.197826] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-01. [2025-10-20T21:02:04.529279] [INFO ] [REMOTE ] SCS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:04.555225] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-01. [2025-10-20T21:02:04.603450] [INFO ] [REMOTE ] SD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:04.604123] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-01. [2025-10-20T21:02:05.463920] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T21:02:05.467356] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-02. [2025-10-20T21:02:05.504103] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T21:02:05.589994] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T21:02:05.836761] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T21:02:05.851327] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-02. [2025-10-20T21:02:06.135230] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T21:02:06.651388] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-02. [2025-10-20T21:02:07.292618] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T21:02:07.451363] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-02. [2025-10-20T21:02:07.529496] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T21:02:08.251408] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-03. [2025-10-20T21:02:08.654889] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:02:09.051372] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-03. [2025-10-20T21:02:09.179290] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T21:02:09.851459] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-04. [2025-10-20T21:02:10.426443] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T21:02:10.651344] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-03. [2025-10-20T21:02:10.730983] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T21:02:11.451411] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-04. [2025-10-20T21:02:11.606828] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-03 (0.96 seconds). [2025-10-20T21:02:12.251380] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-05. [2025-10-20T21:02:12.929572] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T21:02:13.051348] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-04. [2025-10-20T21:02:13.107754] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:02:13.851384] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-06. [2025-10-20T21:02:14.255870] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T21:02:14.433705] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T21:02:14.651376] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-05. [2025-10-20T21:02:15.451447] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-05. [2025-10-20T21:02:15.891264] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-05 (1.24 seconds). [2025-10-20T21:02:16.251381] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-07. [2025-10-20T21:02:16.524185] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-05 (1.07 seconds). [2025-10-20T21:02:17.051353] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-06. [2025-10-20T21:02:17.113927] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T21:02:17.851353] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-08. [2025-10-20T21:02:17.878183] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T21:02:18.651362] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-03. [2025-10-20T21:02:18.681216] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T21:02:19.451433] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-09. [2025-10-20T21:02:19.941424] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T21:02:20.251351] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-04. [2025-10-20T21:02:20.405821] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T21:02:21.051392] [INFO ] [REMOTE ] SDHC: Fetching primary data for 2025-10. [2025-10-20T21:02:21.851359] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-06. [2025-10-20T21:02:21.902693] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T21:02:21.965961] [INFO ] [REMOTE ] SDHC: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T21:02:21.968226] [INFO ] [REMOTE ] SDHC: Full fetch complete (16799 records). [2025-10-20T21:02:21.968294] [DEBUG] [VALIDATOR ] SDHC: Validating primary data. [2025-10-20T21:02:22.346746] [DEBUG] [VALIDATOR ] SDHC: All expected trading days are present. [2025-10-20T21:02:22.346829] [DEBUG] [VALIDATOR ] SDHC: Primary data validation passed. [2025-10-20T21:02:22.347199] [INFO ] [MAIN ] SDHC: Fetched primary data is valid. [2025-10-20T21:02:22.347238] [DEBUG] [LOCAL ] SDHC: Saving primary data. [2025-10-20T21:02:22.347852] [DEBUG] [LOCAL ] SDHC: Trimmed 16799 records to 10386 (keeping 200 days of data) [2025-10-20T21:02:22.411082] [DEBUG] [LOCAL ] SDHC: Primary data saved (10386 records). [2025-10-20T21:02:22.411218] [INFO ] [MAIN ] SDHC: Full history saved. [2025-10-20T21:02:22.411295] [INFO ] [MAIN ] Update Progress: 1881/2367 symbols completed. SDHC took 21.0s. Estimated remaining: 1h 2m 36s. Rate: 7.8 symbols/min [2025-10-20T21:02:22.579721] [INFO ] [MAIN ] SDHY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:22.591598] [DEBUG] [LOCAL ] SDHY: Loaded 11214 existing records from primary storage. [2025-10-20T21:02:22.591795] [INFO ] [MAIN ] SDHY: Loaded 11214 existing records. [2025-10-20T21:02:22.656608] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-05. [2025-10-20T21:02:22.770843] [WARN ] [MAIN ] SDHY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:22.771029] [INFO ] [MAIN ] SDHY: Wiping existing primary data before full refresh. [2025-10-20T21:02:22.771093] [INFO ] [LOCAL ] SDHY: Starting deletion of primary data. [2025-10-20T21:02:22.771261] [DEBUG] [LOCAL ] SDHY: Deleted primary data file. [2025-10-20T21:02:22.771317] [INFO ] [MAIN ] SDHY: Starting full primary refresh. [2025-10-20T21:02:23.128735] [INFO ] [REMOTE ] SDHY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:23.300600] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T21:02:23.451405] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-07. [2025-10-20T21:02:23.953926] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-05 (1.30 seconds). [2025-10-20T21:02:24.251351] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-07. [2025-10-20T21:02:24.601621] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T21:02:25.051384] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-01. [2025-10-20T21:02:25.588549] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T21:02:25.851385] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-08. [2025-10-20T21:02:26.052816] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T21:02:26.651404] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-02. [2025-10-20T21:02:26.838828] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T21:02:27.451402] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-09. [2025-10-20T21:02:27.463532] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:02:28.251468] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-06. [2025-10-20T21:02:28.799364] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T21:02:29.051435] [INFO ] [REMOTE ] SD: Fetching primary data for 2025-10. [2025-10-20T21:02:29.804887] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T21:02:29.851351] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-08. [2025-10-20T21:02:30.395069] [INFO ] [REMOTE ] SD: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:02:30.399354] [INFO ] [REMOTE ] SD: Full fetch complete (40822 records). [2025-10-20T21:02:30.399458] [DEBUG] [VALIDATOR ] SD: Validating primary data. [2025-10-20T21:02:30.654978] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-07. [2025-10-20T21:02:30.774522] [DEBUG] [VALIDATOR ] SD: All expected trading days are present. [2025-10-20T21:02:30.774621] [DEBUG] [VALIDATOR ] SD: Primary data validation passed. [2025-10-20T21:02:30.775519] [INFO ] [MAIN ] SD: Fetched primary data is valid. [2025-10-20T21:02:30.775592] [DEBUG] [LOCAL ] SD: Saving primary data. [2025-10-20T21:02:30.776592] [DEBUG] [LOCAL ] SD: Trimmed 40822 records to 28534 (keeping 200 days of data) [2025-10-20T21:02:30.885448] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:02:30.964275] [DEBUG] [LOCAL ] SD: Primary data saved (28534 records). [2025-10-20T21:02:30.964405] [INFO ] [MAIN ] SD: Full history saved. [2025-10-20T21:02:30.964480] [INFO ] [MAIN ] Update Progress: 1882/2367 symbols completed. SD took 29.6s. Estimated remaining: 1h 2m 29s. Rate: 7.8 symbols/min [2025-10-20T21:02:31.129548] [INFO ] [MAIN ] SDRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:31.168623] [DEBUG] [LOCAL ] SDRL: Loaded 42072 existing records from primary storage. [2025-10-20T21:02:31.169101] [INFO ] [MAIN ] SDRL: Loaded 42072 existing records. [2025-10-20T21:02:31.345759] [WARN ] [MAIN ] SDRL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:31.345957] [INFO ] [MAIN ] SDRL: Wiping existing primary data before full refresh. [2025-10-20T21:02:31.346001] [INFO ] [LOCAL ] SDRL: Starting deletion of primary data. [2025-10-20T21:02:31.346361] [DEBUG] [LOCAL ] SDRL: Deleted primary data file. [2025-10-20T21:02:31.346416] [INFO ] [MAIN ] SDRL: Starting full primary refresh. [2025-10-20T21:02:31.456713] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-03. [2025-10-20T21:02:31.697708] [INFO ] [REMOTE ] SDRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:32.251372] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-01. [2025-10-20T21:02:32.329932] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T21:02:32.389676] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:02:33.051413] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-08. [2025-10-20T21:02:33.854015] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-04. [2025-10-20T21:02:33.859262] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:02:34.564034] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T21:02:34.607075] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T21:02:34.651333] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-05. [2025-10-20T21:02:35.451788] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-02. [2025-10-20T21:02:35.452354] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T21:02:36.251395] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-06. [2025-10-20T21:02:37.051407] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-09. [2025-10-20T21:02:37.117691] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:02:37.116277] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T21:02:37.851381] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-07. [2025-10-20T21:02:37.930602] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T21:02:38.651365] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-03. [2025-10-20T21:02:38.724125] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T21:02:39.451364] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-09. [2025-10-20T21:02:40.025031] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T21:02:40.251414] [INFO ] [REMOTE ] SCM: Fetching primary data for 2025-10. [2025-10-20T21:02:41.051363] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-04. [2025-10-20T21:02:41.197630] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T21:02:41.494947] [INFO ] [REMOTE ] SCM: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:02:41.499084] [INFO ] [REMOTE ] SCM: Full fetch complete (29191 records). [2025-10-20T21:02:41.499162] [DEBUG] [VALIDATOR ] SCM: Validating primary data. [2025-10-20T21:02:41.858633] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-08. [2025-10-20T21:02:41.871480] [DEBUG] [VALIDATOR ] SCM: All expected trading days are present. [2025-10-20T21:02:41.871553] [DEBUG] [VALIDATOR ] SCM: Primary data validation passed. [2025-10-20T21:02:41.872152] [INFO ] [MAIN ] SCM: Fetched primary data is valid. [2025-10-20T21:02:41.872212] [DEBUG] [LOCAL ] SCM: Saving primary data. [2025-10-20T21:02:41.872935] [DEBUG] [LOCAL ] SCM: Trimmed 29191 records to 20080 (keeping 200 days of data) [2025-10-20T21:02:41.993499] [DEBUG] [LOCAL ] SCM: Primary data saved (20080 records). [2025-10-20T21:02:41.993624] [INFO ] [MAIN ] SCM: Full history saved. [2025-10-20T21:02:41.993735] [INFO ] [MAIN ] Update Progress: 1883/2367 symbols completed. SCM took 40.6s. Estimated remaining: 1h 2m 22s. Rate: 7.8 symbols/min [2025-10-20T21:02:42.174581] [INFO ] [MAIN ] SE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:42.229903] [DEBUG] [LOCAL ] SE: Loaded 54082 existing records from primary storage. [2025-10-20T21:02:42.230379] [INFO ] [MAIN ] SE: Loaded 54082 existing records. [2025-10-20T21:02:42.457925] [WARN ] [MAIN ] SE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:42.458419] [INFO ] [MAIN ] SE: Wiping existing primary data before full refresh. [2025-10-20T21:02:42.458580] [INFO ] [LOCAL ] SE: Starting deletion of primary data. [2025-10-20T21:02:42.458835] [DEBUG] [LOCAL ] SE: Deleted primary data file. [2025-10-20T21:02:42.458893] [INFO ] [MAIN ] SE: Starting full primary refresh. [2025-10-20T21:02:42.668178] [INFO ] [REMOTE ] SCS: Fetching primary data for 2025-10. [2025-10-20T21:02:42.806580] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T21:02:42.938413] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T21:02:43.037712] [INFO ] [REMOTE ] SE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:43.451396] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-09. [2025-10-20T21:02:44.034376] [INFO ] [REMOTE ] SCS: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T21:02:44.041099] [INFO ] [REMOTE ] SCS: Full fetch complete (61089 records). [2025-10-20T21:02:44.041256] [DEBUG] [VALIDATOR ] SCS: Validating primary data. [2025-10-20T21:02:44.259754] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-05. [2025-10-20T21:02:44.452987] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-09 (1.00 seconds). [2025-10-20T21:02:44.456687] [DEBUG] [VALIDATOR ] SCS: All expected trading days are present. [2025-10-20T21:02:44.457099] [DEBUG] [VALIDATOR ] SCS: Primary data validation passed. [2025-10-20T21:02:44.458160] [INFO ] [MAIN ] SCS: Fetched primary data is valid. [2025-10-20T21:02:44.458223] [DEBUG] [LOCAL ] SCS: Saving primary data. [2025-10-20T21:02:44.459486] [DEBUG] [LOCAL ] SCS: Trimmed 61089 records to 43377 (keeping 200 days of data) [2025-10-20T21:02:44.723825] [DEBUG] [LOCAL ] SCS: Primary data saved (43377 records). [2025-10-20T21:02:44.723956] [INFO ] [MAIN ] SCS: Full history saved. [2025-10-20T21:02:44.724034] [INFO ] [MAIN ] Update Progress: 1884/2367 symbols completed. SCS took 43.3s. Estimated remaining: 1h 2m 13s. Rate: 7.8 symbols/min [2025-10-20T21:02:44.894967] [INFO ] [MAIN ] SEE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:44.940828] [DEBUG] [LOCAL ] SEE: Loaded 48915 existing records from primary storage. [2025-10-20T21:02:44.941137] [INFO ] [MAIN ] SEE: Loaded 48915 existing records. [2025-10-20T21:02:45.056532] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-01. [2025-10-20T21:02:45.121264] [WARN ] [MAIN ] SEE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:45.121547] [INFO ] [MAIN ] SEE: Wiping existing primary data before full refresh. [2025-10-20T21:02:45.121597] [INFO ] [LOCAL ] SEE: Starting deletion of primary data. [2025-10-20T21:02:45.121807] [DEBUG] [LOCAL ] SEE: Deleted primary data file. [2025-10-20T21:02:45.121864] [INFO ] [MAIN ] SEE: Starting full primary refresh. [2025-10-20T21:02:45.470815] [INFO ] [REMOTE ] SEE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:45.851384] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-01. [2025-10-20T21:02:46.073296] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:02:46.651386] [INFO ] [REMOTE ] SDHY: Fetching primary data for 2025-10. [2025-10-20T21:02:47.035915] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T21:02:47.451395] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-02. [2025-10-20T21:02:47.492174] [INFO ] [REMOTE ] SDHY: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T21:02:47.494909] [INFO ] [REMOTE ] SDHY: Full fetch complete (15787 records). [2025-10-20T21:02:47.495038] [DEBUG] [VALIDATOR ] SDHY: Validating primary data. [2025-10-20T21:02:47.696494] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T21:02:47.907194] [DEBUG] [VALIDATOR ] SDHY: All expected trading days are present. [2025-10-20T21:02:47.907269] [DEBUG] [VALIDATOR ] SDHY: Primary data validation passed. [2025-10-20T21:02:47.907669] [INFO ] [MAIN ] SDHY: Fetched primary data is valid. [2025-10-20T21:02:47.907733] [DEBUG] [LOCAL ] SDHY: Saving primary data. [2025-10-20T21:02:47.908271] [DEBUG] [LOCAL ] SDHY: Trimmed 15787 records to 11160 (keeping 200 days of data) [2025-10-20T21:02:47.976040] [DEBUG] [LOCAL ] SDHY: Primary data saved (11160 records). [2025-10-20T21:02:47.976173] [INFO ] [MAIN ] SDHY: Full history saved. [2025-10-20T21:02:47.976255] [INFO ] [MAIN ] Update Progress: 1885/2367 symbols completed. SDHY took 25.6s. Estimated remaining: 1h 2m 4s. Rate: 7.8 symbols/min [2025-10-20T21:02:48.145581] [INFO ] [MAIN ] SEG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:48.159111] [DEBUG] [LOCAL ] SEG: Loaded 12956 existing records from primary storage. [2025-10-20T21:02:48.159303] [INFO ] [MAIN ] SEG: Loaded 12956 existing records. [2025-10-20T21:02:48.258893] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-02. [2025-10-20T21:02:48.342182] [WARN ] [MAIN ] SEG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:48.342426] [INFO ] [MAIN ] SEG: Wiping existing primary data before full refresh. [2025-10-20T21:02:48.342484] [INFO ] [LOCAL ] SEG: Starting deletion of primary data. [2025-10-20T21:02:48.342693] [DEBUG] [LOCAL ] SEG: Deleted primary data file. [2025-10-20T21:02:48.342749] [INFO ] [MAIN ] SEG: Starting full primary refresh. [2025-10-20T21:02:48.699244] [INFO ] [REMOTE ] SEG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:49.044736] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T21:02:49.055069] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-06. [2025-10-20T21:02:49.554820] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T21:02:49.851370] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-03. [2025-10-20T21:02:50.251830] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T21:02:50.651385] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-07. [2025-10-20T21:02:51.349721] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T21:02:51.451355] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-04. [2025-10-20T21:02:52.200072] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T21:02:52.251384] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-01. [2025-10-20T21:02:52.834130] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-04 (1.38 seconds). [2025-10-20T21:02:53.051359] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-08. [2025-10-20T21:02:53.095708] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T21:02:53.851388] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-05. [2025-10-20T21:02:54.606006] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T21:02:54.651397] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-09. [2025-10-20T21:02:55.451370] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-02. [2025-10-20T21:02:55.594623] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T21:02:56.116963] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T21:02:56.198813] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:02:56.251353] [INFO ] [REMOTE ] SDRL: Fetching primary data for 2025-10. [2025-10-20T21:02:57.051373] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-03. [2025-10-20T21:02:57.684052] [INFO ] [REMOTE ] SDRL: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T21:02:57.689935] [INFO ] [REMOTE ] SDRL: Full fetch complete (61395 records). [2025-10-20T21:02:57.690043] [DEBUG] [VALIDATOR ] SDRL: Validating primary data. [2025-10-20T21:02:57.864850] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-03. [2025-10-20T21:02:58.091077] [DEBUG] [VALIDATOR ] SDRL: All expected trading days are present. [2025-10-20T21:02:58.091159] [DEBUG] [VALIDATOR ] SDRL: Primary data validation passed. [2025-10-20T21:02:58.092102] [INFO ] [MAIN ] SDRL: Fetched primary data is valid. [2025-10-20T21:02:58.092164] [DEBUG] [LOCAL ] SDRL: Saving primary data. [2025-10-20T21:02:58.093295] [DEBUG] [LOCAL ] SDRL: Trimmed 61395 records to 41446 (keeping 200 days of data) [2025-10-20T21:02:58.341582] [DEBUG] [LOCAL ] SDRL: Primary data saved (41446 records). [2025-10-20T21:02:58.341745] [INFO ] [MAIN ] SDRL: Full history saved. [2025-10-20T21:02:58.341840] [INFO ] [MAIN ] Update Progress: 1886/2367 symbols completed. SDRL took 27.4s. Estimated remaining: 1h 1m 57s. Rate: 7.8 symbols/min [2025-10-20T21:02:58.517895] [INFO ] [MAIN ] SEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:02:58.564683] [DEBUG] [LOCAL ] SEI: Loaded 50711 existing records from primary storage. [2025-10-20T21:02:58.565003] [INFO ] [MAIN ] SEI: Loaded 50711 existing records. [2025-10-20T21:02:58.659402] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-06. [2025-10-20T21:02:58.751820] [WARN ] [MAIN ] SEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:02:58.752055] [INFO ] [MAIN ] SEI: Wiping existing primary data before full refresh. [2025-10-20T21:02:58.752102] [INFO ] [LOCAL ] SEI: Starting deletion of primary data. [2025-10-20T21:02:58.752376] [DEBUG] [LOCAL ] SEI: Deleted primary data file. [2025-10-20T21:02:58.752431] [INFO ] [MAIN ] SEI: Starting full primary refresh. [2025-10-20T21:02:59.129576] [INFO ] [REMOTE ] SEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:02:59.141541] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T21:02:59.200957] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T21:02:59.451378] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-01. [2025-10-20T21:03:00.119099] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T21:03:00.251379] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-07. [2025-10-20T21:03:00.930835] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T21:03:01.051366] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-02. [2025-10-20T21:03:01.851387] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-04. [2025-10-20T21:03:02.033060] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T21:03:02.538279] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T21:03:02.660366] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-04. [2025-10-20T21:03:02.664390] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T21:03:03.451376] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-03. [2025-10-20T21:03:04.251384] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-08. [2025-10-20T21:03:04.922889] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-04 (2.26 seconds). [2025-10-20T21:03:05.051345] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-05. [2025-10-20T21:03:05.306950] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T21:03:05.628741] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T21:03:05.665213] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-05 (0.61 seconds). [2025-10-20T21:03:05.851372] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-05. [2025-10-20T21:03:06.651385] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-06. [2025-10-20T21:03:07.451408] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-04. [2025-10-20T21:03:07.516893] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T21:03:08.174303] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-05 (2.32 seconds). [2025-10-20T21:03:08.251385] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-07. [2025-10-20T21:03:09.051346] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-06. [2025-10-20T21:03:09.258746] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T21:03:09.258377] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T21:03:09.851440] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-09. [2025-10-20T21:03:10.651460] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-05. [2025-10-20T21:03:11.162854] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-06 (2.11 seconds). [2025-10-20T21:03:11.252392] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:03:11.451417] [INFO ] [REMOTE ] SEE: Fetching primary data for 2025-10. [2025-10-20T21:03:12.213077] [INFO ] [REMOTE ] SEE: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T21:03:12.220908] [INFO ] [REMOTE ] SEE: Full fetch complete (69783 records). [2025-10-20T21:03:12.221192] [DEBUG] [VALIDATOR ] SEE: Validating primary data. [2025-10-20T21:03:12.222387] [FATAL] [VALIDATOR ] SEE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:03:12.222613] [FATAL] [MAIN ] SEE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:03:12.222795] [WARN ] [MAIN ] SEE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:03:12.222875] [INFO ] [LOCAL ] SEE: Starting deletion of primary data. [2025-10-20T21:03:12.222972] [DEBUG] [LOCAL ] SEE: No primary data file found to delete. [2025-10-20T21:03:12.223049] [INFO ] [LOCAL ] SEE: Starting deletion of derivative data. [2025-10-20T21:03:12.223268] [DEBUG] [LOCAL ] SEE: Deleted derivative data file. [2025-10-20T21:03:12.223812] [INFO ] [MAIN ] Update Progress: 1887/2367 symbols completed. SEE took 27.5s. Estimated remaining: 1h 1m 51s. Rate: 7.8 symbols/min [2025-10-20T21:03:12.256491] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-08. [2025-10-20T21:03:12.411095] [INFO ] [MAIN ] SEM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:12.443470] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T21:03:12.493182] [DEBUG] [LOCAL ] SEM: Loaded 43811 existing records from primary storage. [2025-10-20T21:03:12.493515] [INFO ] [MAIN ] SEM: Loaded 43811 existing records. [2025-10-20T21:03:12.686591] [WARN ] [MAIN ] SEM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:12.686836] [INFO ] [MAIN ] SEM: Wiping existing primary data before full refresh. [2025-10-20T21:03:12.686889] [INFO ] [LOCAL ] SEM: Starting deletion of primary data. [2025-10-20T21:03:12.687066] [DEBUG] [LOCAL ] SEM: Deleted primary data file. [2025-10-20T21:03:12.687106] [INFO ] [MAIN ] SEM: Starting full primary refresh. [2025-10-20T21:03:13.058820] [INFO ] [REMOTE ] SEM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:13.058913] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-06. [2025-10-20T21:03:13.326930] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:03:13.851407] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-01. [2025-10-20T21:03:14.651394] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-07. [2025-10-20T21:03:14.743535] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:03:14.922089] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T21:03:15.451440] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-07. [2025-10-20T21:03:16.251445] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-09. [2025-10-20T21:03:16.491946] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T21:03:16.952827] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T21:03:17.051357] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-08. [2025-10-20T21:03:17.084158] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:03:17.851415] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-02. [2025-10-20T21:03:18.651343] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-08. [2025-10-20T21:03:18.867696] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T21:03:18.946990] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T21:03:19.116062] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-08 (0.46 seconds). [2025-10-20T21:03:19.451493] [INFO ] [REMOTE ] SEG: Fetching primary data for 2025-10. [2025-10-20T21:03:20.245884] [INFO ] [REMOTE ] SEG: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:03:20.247810] [INFO ] [REMOTE ] SEG: Full fetch complete (19782 records). [2025-10-20T21:03:20.247912] [DEBUG] [VALIDATOR ] SEG: Validating primary data. [2025-10-20T21:03:20.259362] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-09. [2025-10-20T21:03:20.626410] [DEBUG] [VALIDATOR ] SEG: All expected trading days are present. [2025-10-20T21:03:20.626491] [DEBUG] [VALIDATOR ] SEG: Primary data validation passed. [2025-10-20T21:03:20.626976] [INFO ] [MAIN ] SEG: Fetched primary data is valid. [2025-10-20T21:03:20.627039] [DEBUG] [LOCAL ] SEG: Saving primary data. [2025-10-20T21:03:20.627670] [DEBUG] [LOCAL ] SEG: Trimmed 19782 records to 12718 (keeping 200 days of data) [2025-10-20T21:03:20.701957] [DEBUG] [LOCAL ] SEG: Primary data saved (12718 records). [2025-10-20T21:03:20.702094] [INFO ] [MAIN ] SEG: Full history saved. [2025-10-20T21:03:20.702169] [INFO ] [MAIN ] Update Progress: 1888/2367 symbols completed. SEG took 32.7s. Estimated remaining: 1h 1m 43s. Rate: 7.8 symbols/min [2025-10-20T21:03:20.882328] [INFO ] [MAIN ] SEMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:20.921944] [DEBUG] [LOCAL ] SEMR: Loaded 42727 existing records from primary storage. [2025-10-20T21:03:20.922243] [INFO ] [MAIN ] SEMR: Loaded 42727 existing records. [2025-10-20T21:03:21.056493] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-09. [2025-10-20T21:03:21.110106] [WARN ] [MAIN ] SEMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:21.110322] [INFO ] [MAIN ] SEMR: Wiping existing primary data before full refresh. [2025-10-20T21:03:21.110371] [INFO ] [LOCAL ] SEMR: Starting deletion of primary data. [2025-10-20T21:03:21.110664] [DEBUG] [LOCAL ] SEMR: Deleted primary data file. [2025-10-20T21:03:21.110726] [INFO ] [MAIN ] SEMR: Starting full primary refresh. [2025-10-20T21:03:21.457839] [INFO ] [REMOTE ] SEMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:21.851351] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-03. [2025-10-20T21:03:21.956263] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T21:03:22.280849] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T21:03:22.651370] [INFO ] [REMOTE ] SE: Fetching primary data for 2025-10. [2025-10-20T21:03:23.025171] [INFO ] [REMOTE ] SE: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T21:03:23.033189] [INFO ] [REMOTE ] SE: Full fetch complete (76881 records). [2025-10-20T21:03:23.033334] [DEBUG] [VALIDATOR ] SE: Validating primary data. [2025-10-20T21:03:23.034122] [FATAL] [VALIDATOR ] SE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:03:23.034236] [FATAL] [MAIN ] SE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:03:23.034318] [WARN ] [MAIN ] SE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:03:23.034403] [INFO ] [LOCAL ] SE: Starting deletion of primary data. [2025-10-20T21:03:23.034517] [DEBUG] [LOCAL ] SE: No primary data file found to delete. [2025-10-20T21:03:23.034615] [INFO ] [LOCAL ] SE: Starting deletion of derivative data. [2025-10-20T21:03:23.034836] [DEBUG] [LOCAL ] SE: Deleted derivative data file. [2025-10-20T21:03:23.035213] [INFO ] [MAIN ] Update Progress: 1889/2367 symbols completed. SE took 41.0s. Estimated remaining: 1h 1m 34s. Rate: 7.8 symbols/min [2025-10-20T21:03:23.239194] [INFO ] [MAIN ] SES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:23.290108] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:03:23.315680] [DEBUG] [LOCAL ] SES: Loaded 52357 existing records from primary storage. [2025-10-20T21:03:23.315997] [INFO ] [MAIN ] SES: Loaded 52357 existing records. [2025-10-20T21:03:23.456502] [INFO ] [REMOTE ] SEI: Fetching primary data for 2025-10. [2025-10-20T21:03:23.492021] [WARN ] [MAIN ] SES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:23.492310] [INFO ] [MAIN ] SES: Wiping existing primary data before full refresh. [2025-10-20T21:03:23.492364] [INFO ] [LOCAL ] SES: Starting deletion of primary data. [2025-10-20T21:03:23.492551] [DEBUG] [LOCAL ] SES: Deleted primary data file. [2025-10-20T21:03:23.492607] [INFO ] [MAIN ] SES: Starting full primary refresh. [2025-10-20T21:03:23.866004] [INFO ] [REMOTE ] SES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:24.251347] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-01. [2025-10-20T21:03:24.290468] [INFO ] [REMOTE ] SEI: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T21:03:24.297602] [INFO ] [REMOTE ] SEI: Full fetch complete (70673 records). [2025-10-20T21:03:24.297742] [DEBUG] [VALIDATOR ] SEI: Validating primary data. [2025-10-20T21:03:24.298481] [FATAL] [VALIDATOR ] SEI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:03:24.298589] [FATAL] [MAIN ] SEI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:03:24.298632] [WARN ] [MAIN ] SEI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:03:24.298760] [INFO ] [LOCAL ] SEI: Starting deletion of primary data. [2025-10-20T21:03:24.298823] [DEBUG] [LOCAL ] SEI: No primary data file found to delete. [2025-10-20T21:03:24.298857] [INFO ] [LOCAL ] SEI: Starting deletion of derivative data. [2025-10-20T21:03:24.299062] [DEBUG] [LOCAL ] SEI: Deleted derivative data file. [2025-10-20T21:03:24.299151] [INFO ] [MAIN ] Update Progress: 1890/2367 symbols completed. SEI took 26.0s. Estimated remaining: 1h 1m 25s. Rate: 7.8 symbols/min [2025-10-20T21:03:24.465401] [INFO ] [MAIN ] SF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:24.504253] [DEBUG] [LOCAL ] SF: Loaded 42171 existing records from primary storage. [2025-10-20T21:03:24.504528] [INFO ] [MAIN ] SF: Loaded 42171 existing records. [2025-10-20T21:03:24.678170] [WARN ] [MAIN ] SF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:24.678329] [INFO ] [MAIN ] SF: Wiping existing primary data before full refresh. [2025-10-20T21:03:24.678374] [INFO ] [LOCAL ] SF: Starting deletion of primary data. [2025-10-20T21:03:24.678543] [DEBUG] [LOCAL ] SF: Deleted primary data file. [2025-10-20T21:03:24.678580] [INFO ] [MAIN ] SF: Starting full primary refresh. [2025-10-20T21:03:25.023131] [INFO ] [REMOTE ] SF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:25.051341] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-01. [2025-10-20T21:03:25.482252] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T21:03:25.851383] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-01. [2025-10-20T21:03:26.651425] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-02. [2025-10-20T21:03:27.451418] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-04. [2025-10-20T21:03:27.529903] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T21:03:27.648474] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T21:03:27.690363] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-01 (2.64 seconds). [2025-10-20T21:03:28.251377] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-03. [2025-10-20T21:03:29.051383] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-02. [2025-10-20T21:03:29.084708] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:03:29.694573] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:03:29.851386] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-02. [2025-10-20T21:03:30.651387] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-04. [2025-10-20T21:03:31.407989] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T21:03:31.451345] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-03. [2025-10-20T21:03:31.543758] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-02 (2.49 seconds). [2025-10-20T21:03:31.785288] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T21:03:32.251396] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-05. [2025-10-20T21:03:33.051399] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-05. [2025-10-20T21:03:33.199628] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T21:03:33.567189] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T21:03:33.851424] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-06. [2025-10-20T21:03:34.653506] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-04. [2025-10-20T21:03:34.662427] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T21:03:35.392336] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:03:35.451409] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-03. [2025-10-20T21:03:36.251391] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-07. [2025-10-20T21:03:36.575371] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T21:03:37.051403] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-06. [2025-10-20T21:03:37.851455] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-05. [2025-10-20T21:03:37.922445] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T21:03:38.023058] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-03 (2.57 seconds). [2025-10-20T21:03:38.639623] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T21:03:38.651331] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-07. [2025-10-20T21:03:39.243953] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T21:03:39.451375] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-08. [2025-10-20T21:03:40.197158] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T21:03:40.258108] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-08. [2025-10-20T21:03:40.286206] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T21:03:41.051387] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-04. [2025-10-20T21:03:41.851368] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-09. [2025-10-20T21:03:42.094324] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T21:03:42.612456] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T21:03:42.651337] [INFO ] [REMOTE ] SEM: Fetching primary data for 2025-10. [2025-10-20T21:03:43.147190] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T21:03:43.340893] [INFO ] [REMOTE ] SEM: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T21:03:43.346867] [INFO ] [REMOTE ] SEM: Full fetch complete (61317 records). [2025-10-20T21:03:43.346967] [DEBUG] [VALIDATOR ] SEM: Validating primary data. [2025-10-20T21:03:43.347574] [FATAL] [VALIDATOR ] SEM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:03:43.347679] [FATAL] [MAIN ] SEM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:03:43.347731] [WARN ] [MAIN ] SEM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:03:43.347771] [INFO ] [LOCAL ] SEM: Starting deletion of primary data. [2025-10-20T21:03:43.347840] [DEBUG] [LOCAL ] SEM: No primary data file found to delete. [2025-10-20T21:03:43.347926] [INFO ] [LOCAL ] SEM: Starting deletion of derivative data. [2025-10-20T21:03:43.348249] [DEBUG] [LOCAL ] SEM: Deleted derivative data file. [2025-10-20T21:03:43.348337] [INFO ] [MAIN ] Update Progress: 1891/2367 symbols completed. SEM took 31.1s. Estimated remaining: 1h 1m 20s. Rate: 7.8 symbols/min [2025-10-20T21:03:43.456555] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-09. [2025-10-20T21:03:43.517356] [INFO ] [MAIN ] SFB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:43.522887] [DEBUG] [LOCAL ] SFB: Loaded 3858 existing records from primary storage. [2025-10-20T21:03:43.523073] [INFO ] [MAIN ] SFB: Loaded 3858 existing records. [2025-10-20T21:03:43.698201] [WARN ] [MAIN ] SFB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:43.698382] [INFO ] [MAIN ] SFB: Wiping existing primary data before full refresh. [2025-10-20T21:03:43.698428] [INFO ] [LOCAL ] SFB: Starting deletion of primary data. [2025-10-20T21:03:43.698571] [DEBUG] [LOCAL ] SFB: Deleted primary data file. [2025-10-20T21:03:43.698626] [INFO ] [MAIN ] SFB: Starting full primary refresh. [2025-10-20T21:03:44.045602] [INFO ] [REMOTE ] SFB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:44.251402] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-01. [2025-10-20T21:03:44.840852] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T21:03:44.902526] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T21:03:45.051384] [INFO ] [REMOTE ] SEMR: Fetching primary data for 2025-10. [2025-10-20T21:03:45.851351] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-02. [2025-10-20T21:03:46.371256] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-02 (0.52 seconds). [2025-10-20T21:03:46.594465] [INFO ] [REMOTE ] SEMR: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T21:03:46.601373] [INFO ] [REMOTE ] SEMR: Full fetch complete (60586 records). [2025-10-20T21:03:46.601518] [DEBUG] [VALIDATOR ] SEMR: Validating primary data. [2025-10-20T21:03:46.656621] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-06. [2025-10-20T21:03:46.982513] [DEBUG] [VALIDATOR ] SEMR: All expected trading days are present. [2025-10-20T21:03:46.982590] [DEBUG] [VALIDATOR ] SEMR: Primary data validation passed. [2025-10-20T21:03:46.983497] [INFO ] [MAIN ] SEMR: Fetched primary data is valid. [2025-10-20T21:03:46.983558] [DEBUG] [LOCAL ] SEMR: Saving primary data. [2025-10-20T21:03:46.984707] [DEBUG] [LOCAL ] SEMR: Trimmed 60586 records to 42107 (keeping 200 days of data) [2025-10-20T21:03:47.220944] [DEBUG] [LOCAL ] SEMR: Primary data saved (42107 records). [2025-10-20T21:03:47.221089] [INFO ] [MAIN ] SEMR: Full history saved. [2025-10-20T21:03:47.221181] [INFO ] [MAIN ] Update Progress: 1892/2367 symbols completed. SEMR took 26.5s. Estimated remaining: 1h 1m 11s. Rate: 7.8 symbols/min [2025-10-20T21:03:47.389375] [INFO ] [MAIN ] SFBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:47.409594] [DEBUG] [LOCAL ] SFBS: Loaded 21563 existing records from primary storage. [2025-10-20T21:03:47.409879] [INFO ] [MAIN ] SFBS: Loaded 21563 existing records. [2025-10-20T21:03:47.463009] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-05. [2025-10-20T21:03:47.589482] [WARN ] [MAIN ] SFBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:47.589698] [INFO ] [MAIN ] SFBS: Wiping existing primary data before full refresh. [2025-10-20T21:03:47.589755] [INFO ] [LOCAL ] SFBS: Starting deletion of primary data. [2025-10-20T21:03:47.589957] [DEBUG] [LOCAL ] SFBS: Deleted primary data file. [2025-10-20T21:03:47.590011] [INFO ] [MAIN ] SFBS: Starting full primary refresh. [2025-10-20T21:03:47.943740] [INFO ] [REMOTE ] SFBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:48.218104] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:03:48.251336] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-07. [2025-10-20T21:03:49.051434] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-01. [2025-10-20T21:03:49.864990] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-03. [2025-10-20T21:03:49.871335] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T21:03:49.916414] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T21:03:50.087449] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T21:03:50.578915] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T21:03:50.651378] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-08. [2025-10-20T21:03:51.451432] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-06. [2025-10-20T21:03:52.251395] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-04. [2025-10-20T21:03:52.279505] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:03:52.941151] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T21:03:53.051357] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-09. [2025-10-20T21:03:53.860253] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-05. [2025-10-20T21:03:53.866114] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-06 (2.41 seconds). [2025-10-20T21:03:54.494203] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T21:03:54.550902] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T21:03:54.651397] [INFO ] [REMOTE ] SF: Fetching primary data for 2025-10. [2025-10-20T21:03:55.399771] [INFO ] [REMOTE ] SF: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:03:55.405579] [INFO ] [REMOTE ] SF: Full fetch complete (60059 records). [2025-10-20T21:03:55.405717] [DEBUG] [VALIDATOR ] SF: Validating primary data. [2025-10-20T21:03:55.406367] [FATAL] [VALIDATOR ] SF: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:03:55.406454] [FATAL] [MAIN ] SF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:03:55.406497] [WARN ] [MAIN ] SF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:03:55.406554] [INFO ] [LOCAL ] SF: Starting deletion of primary data. [2025-10-20T21:03:55.406613] [DEBUG] [LOCAL ] SF: No primary data file found to delete. [2025-10-20T21:03:55.406681] [INFO ] [LOCAL ] SF: Starting deletion of derivative data. [2025-10-20T21:03:55.406843] [DEBUG] [LOCAL ] SF: Deleted derivative data file. [2025-10-20T21:03:55.406916] [INFO ] [MAIN ] Update Progress: 1893/2367 symbols completed. SF took 31.1s. Estimated remaining: 1h 1m 4s. Rate: 7.8 symbols/min [2025-10-20T21:03:55.456600] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-07. [2025-10-20T21:03:55.590787] [INFO ] [MAIN ] SFL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:03:55.631406] [DEBUG] [LOCAL ] SFL: Loaded 46032 existing records from primary storage. [2025-10-20T21:03:55.631801] [INFO ] [MAIN ] SFL: Loaded 46032 existing records. [2025-10-20T21:03:55.805494] [WARN ] [MAIN ] SFL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:03:55.805711] [INFO ] [MAIN ] SFL: Wiping existing primary data before full refresh. [2025-10-20T21:03:55.805765] [INFO ] [LOCAL ] SFL: Starting deletion of primary data. [2025-10-20T21:03:55.805951] [DEBUG] [LOCAL ] SFL: Deleted primary data file. [2025-10-20T21:03:55.805988] [INFO ] [MAIN ] SFL: Starting full primary refresh. [2025-10-20T21:03:56.157887] [INFO ] [REMOTE ] SFL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:03:56.251340] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-01. [2025-10-20T21:03:57.051374] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-06. [2025-10-20T21:03:57.667432] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T21:03:57.750819] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-06 (0.70 seconds). [2025-10-20T21:03:57.856525] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-02. [2025-10-20T21:03:57.875329] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-07 (2.42 seconds). [2025-10-20T21:03:58.651377] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-07. [2025-10-20T21:03:59.361770] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T21:03:59.451383] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-02. [2025-10-20T21:03:59.487450] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:04:00.251385] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-08. [2025-10-20T21:04:00.458816] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T21:04:00.894027] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T21:04:01.051377] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-03. [2025-10-20T21:04:01.851363] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-03. [2025-10-20T21:04:02.202343] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:04:02.651378] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-04. [2025-10-20T21:04:03.451401] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-08. [2025-10-20T21:04:03.695730] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T21:04:03.768943] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:04:04.251430] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-04. [2025-10-20T21:04:05.051347] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-09. [2025-10-20T21:04:05.310154] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T21:04:05.682618] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T21:04:05.851370] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-09. [2025-10-20T21:04:05.949008] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-04 (1.70 seconds). [2025-10-20T21:04:06.651377] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-05. [2025-10-20T21:04:07.451433] [INFO ] [REMOTE ] SFB: Fetching primary data for 2025-10. [2025-10-20T21:04:08.088462] [INFO ] [REMOTE ] SFB: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:04:08.091266] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-09 (2.24 seconds). [2025-10-20T21:04:08.091352] [INFO ] [REMOTE ] SFB: Full fetch complete (5800 records). [2025-10-20T21:04:08.101904] [DEBUG] [VALIDATOR ] SFB: Validating primary data. [2025-10-20T21:04:08.256501] [INFO ] [REMOTE ] SES: Fetching primary data for 2025-10. [2025-10-20T21:04:08.474792] [DEBUG] [VALIDATOR ] SFB: All expected trading days are present. [2025-10-20T21:04:08.474969] [DEBUG] [VALIDATOR ] SFB: Primary data validation passed. [2025-10-20T21:04:08.475275] [INFO ] [MAIN ] SFB: Fetched primary data is valid. [2025-10-20T21:04:08.475419] [DEBUG] [LOCAL ] SFB: Saving primary data. [2025-10-20T21:04:08.475851] [DEBUG] [LOCAL ] SFB: Trimmed 5800 records to 3778 (keeping 200 days of data) [2025-10-20T21:04:08.498597] [DEBUG] [LOCAL ] SFB: Primary data saved (3778 records). [2025-10-20T21:04:08.498784] [INFO ] [MAIN ] SFB: Full history saved. [2025-10-20T21:04:08.498897] [INFO ] [MAIN ] Update Progress: 1894/2367 symbols completed. SFB took 25.2s. Estimated remaining: 1h 0m 57s. Rate: 7.8 symbols/min [2025-10-20T21:04:08.701409] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T21:04:08.704668] [INFO ] [MAIN ] SG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:08.767442] [DEBUG] [LOCAL ] SG: Loaded 54264 existing records from primary storage. [2025-10-20T21:04:08.767780] [INFO ] [MAIN ] SG: Loaded 54264 existing records. [2025-10-20T21:04:08.948810] [WARN ] [MAIN ] SG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:08.949116] [INFO ] [MAIN ] SG: Wiping existing primary data before full refresh. [2025-10-20T21:04:08.949170] [INFO ] [LOCAL ] SG: Starting deletion of primary data. [2025-10-20T21:04:08.949347] [DEBUG] [LOCAL ] SG: Deleted primary data file. [2025-10-20T21:04:08.949398] [INFO ] [MAIN ] SG: Starting full primary refresh. [2025-10-20T21:04:09.055678] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-05. [2025-10-20T21:04:09.361001] [INFO ] [REMOTE ] SG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:09.362790] [INFO ] [REMOTE ] SES: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:04:09.370743] [INFO ] [REMOTE ] SES: Full fetch complete (75647 records). [2025-10-20T21:04:09.370864] [DEBUG] [VALIDATOR ] SES: Validating primary data. [2025-10-20T21:04:09.750877] [DEBUG] [VALIDATOR ] SES: All expected trading days are present. [2025-10-20T21:04:09.750956] [DEBUG] [VALIDATOR ] SES: Primary data validation passed. [2025-10-20T21:04:09.752002] [INFO ] [MAIN ] SES: Fetched primary data is valid. [2025-10-20T21:04:09.752064] [DEBUG] [LOCAL ] SES: Saving primary data. [2025-10-20T21:04:09.753383] [DEBUG] [LOCAL ] SES: Trimmed 75647 records to 51598 (keeping 200 days of data) [2025-10-20T21:04:09.858240] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-01. [2025-10-20T21:04:09.934631] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-05 (0.88 seconds). [2025-10-20T21:04:10.064497] [DEBUG] [LOCAL ] SES: Primary data saved (51598 records). [2025-10-20T21:04:10.064627] [INFO ] [MAIN ] SES: Full history saved. [2025-10-20T21:04:10.064965] [INFO ] [MAIN ] Update Progress: 1895/2367 symbols completed. SES took 47.0s. Estimated remaining: 1h 0m 48s. Rate: 7.8 symbols/min [2025-10-20T21:04:10.231300] [INFO ] [MAIN ] SGHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:10.274985] [DEBUG] [LOCAL ] SGHC: Loaded 47745 existing records from primary storage. [2025-10-20T21:04:10.275268] [INFO ] [MAIN ] SGHC: Loaded 47745 existing records. [2025-10-20T21:04:10.454047] [WARN ] [MAIN ] SGHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:10.454297] [INFO ] [MAIN ] SGHC: Wiping existing primary data before full refresh. [2025-10-20T21:04:10.454359] [INFO ] [LOCAL ] SGHC: Starting deletion of primary data. [2025-10-20T21:04:10.454563] [DEBUG] [LOCAL ] SGHC: Deleted primary data file. [2025-10-20T21:04:10.454666] [INFO ] [MAIN ] SGHC: Starting full primary refresh. [2025-10-20T21:04:10.659094] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-06. [2025-10-20T21:04:10.808434] [INFO ] [REMOTE ] SGHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:11.451451] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-01. [2025-10-20T21:04:11.625995] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-06 (0.97 seconds). [2025-10-20T21:04:11.850127] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-01 (1.99 seconds). [2025-10-20T21:04:12.251377] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-07. [2025-10-20T21:04:12.954110] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T21:04:13.051348] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-02. [2025-10-20T21:04:13.420366] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T21:04:13.851436] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-06. [2025-10-20T21:04:14.276369] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T21:04:14.651416] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-08. [2025-10-20T21:04:15.422798] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T21:04:15.456464] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-02. [2025-10-20T21:04:15.470104] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:04:16.251353] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-07. [2025-10-20T21:04:17.051351] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-03. [2025-10-20T21:04:17.530485] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T21:04:17.851360] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-03. [2025-10-20T21:04:17.929345] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T21:04:18.587966] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T21:04:18.651339] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-04. [2025-10-20T21:04:19.451430] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-09. [2025-10-20T21:04:19.553470] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T21:04:20.256490] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-08. [2025-10-20T21:04:20.269275] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T21:04:20.624734] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T21:04:21.051431] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-04. [2025-10-20T21:04:21.807335] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T21:04:21.851347] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-09. [2025-10-20T21:04:22.637056] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T21:04:22.651349] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-05. [2025-10-20T21:04:23.451387] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-05. [2025-10-20T21:04:23.502746] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T21:04:24.251363] [INFO ] [REMOTE ] SFBS: Fetching primary data for 2025-10. [2025-10-20T21:04:24.654189] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T21:04:24.692716] [INFO ] [REMOTE ] SFBS: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T21:04:24.695984] [INFO ] [REMOTE ] SFBS: Full fetch complete (30188 records). [2025-10-20T21:04:24.696056] [DEBUG] [VALIDATOR ] SFBS: Validating primary data. [2025-10-20T21:04:24.696523] [FATAL] [VALIDATOR ] SFBS: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:04:24.696606] [FATAL] [MAIN ] SFBS: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:04:24.696676] [WARN ] [MAIN ] SFBS: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:04:24.696746] [INFO ] [LOCAL ] SFBS: Starting deletion of primary data. [2025-10-20T21:04:24.696808] [DEBUG] [LOCAL ] SFBS: No primary data file found to delete. [2025-10-20T21:04:24.696843] [INFO ] [LOCAL ] SFBS: Starting deletion of derivative data. [2025-10-20T21:04:24.696963] [DEBUG] [LOCAL ] SFBS: Deleted derivative data file. [2025-10-20T21:04:24.697029] [INFO ] [MAIN ] Update Progress: 1896/2367 symbols completed. SFBS took 37.5s. Estimated remaining: 1h 0m 42s. Rate: 7.8 symbols/min [2025-10-20T21:04:24.869376] [INFO ] [MAIN ] SGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:24.869734] [DEBUG] [LOCAL ] SGI: No existing data file found in primary storage. [2025-10-20T21:04:24.869835] [WARN ] [MAIN ] SGI: No existing data found, performing full refresh instead. [2025-10-20T21:04:24.869994] [INFO ] [MAIN ] SGI: Wiping existing primary data before full refresh. [2025-10-20T21:04:24.870121] [INFO ] [LOCAL ] SGI: Starting deletion of primary data. [2025-10-20T21:04:24.870268] [DEBUG] [LOCAL ] SGI: No primary data file found to delete. [2025-10-20T21:04:24.870364] [INFO ] [MAIN ] SGI: Starting full primary refresh. [2025-10-20T21:04:25.053500] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-06. [2025-10-20T21:04:25.089413] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T21:04:25.315310] [INFO ] [REMOTE ] SGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:25.851440] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-06. [2025-10-20T21:04:26.651423] [INFO ] [REMOTE ] SFL: Fetching primary data for 2025-10. [2025-10-20T21:04:27.064336] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-06 (2.01 seconds). [2025-10-20T21:04:27.437540] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T21:04:27.451343] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-07. [2025-10-20T21:04:28.251421] [INFO ] [REMOTE ] SGI: Fetching primary data for 2025-01. [2025-10-20T21:04:28.283820] [INFO ] [REMOTE ] SFL: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T21:04:28.294881] [INFO ] [REMOTE ] SFL: Full fetch complete (65932 records). [2025-10-20T21:04:28.295090] [DEBUG] [VALIDATOR ] SFL: Validating primary data. [2025-10-20T21:04:28.547845] [DEBUG] [REMOTE ] SGI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:04:28.584308] [WARN ] [REMOTE ] SGI: No data returned for month 2025-01 [2025-10-20T21:04:28.632453] [FATAL] [REMOTE ] SGI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:04:28.672687] [FATAL] [MAIN ] SGI: Failed to fetch primary data. [2025-10-20T21:04:28.696944] [WARN ] [MAIN ] SGI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:04:28.711442] [INFO ] [LOCAL ] SGI: Starting deletion of primary data. [2025-10-20T21:04:28.711927] [DEBUG] [LOCAL ] SGI: No primary data file found to delete. [2025-10-20T21:04:28.717401] [INFO ] [LOCAL ] SGI: Starting deletion of derivative data. [2025-10-20T21:04:28.752017] [DEBUG] [LOCAL ] SGI: No derivative data file found to delete. [2025-10-20T21:04:28.780805] [INFO ] [MAIN ] Update Progress: 1897/2367 symbols completed. SGI took 4.1s. Estimated remaining: 1h 0m 33s. Rate: 7.8 symbols/min [2025-10-20T21:04:28.918196] [DEBUG] [VALIDATOR ] SFL: All expected trading days are present. [2025-10-20T21:04:28.946899] [DEBUG] [VALIDATOR ] SFL: Primary data validation passed. [2025-10-20T21:04:28.970815] [INFO ] [MAIN ] SFL: Fetched primary data is valid. [2025-10-20T21:04:28.989509] [DEBUG] [LOCAL ] SFL: Saving primary data. [2025-10-20T21:04:29.048875] [DEBUG] [LOCAL ] SFL: Trimmed 65932 records to 45331 (keeping 200 days of data) [2025-10-20T21:04:29.070242] [INFO ] [MAIN ] SGU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:29.065399] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-07. [2025-10-20T21:04:29.217272] [DEBUG] [LOCAL ] SGU: Loaded 6653 existing records from primary storage. [2025-10-20T21:04:29.220098] [INFO ] [MAIN ] SGU: Loaded 6653 existing records. [2025-10-20T21:04:29.534092] [WARN ] [MAIN ] SGU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:29.534918] [INFO ] [MAIN ] SGU: Wiping existing primary data before full refresh. [2025-10-20T21:04:29.535469] [INFO ] [LOCAL ] SGU: Starting deletion of primary data. [2025-10-20T21:04:29.536072] [DEBUG] [LOCAL ] SGU: Deleted primary data file. [2025-10-20T21:04:29.536527] [INFO ] [MAIN ] SGU: Starting full primary refresh. [2025-10-20T21:04:29.675525] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-07 (2.22 seconds). [2025-10-20T21:04:29.868460] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-08. [2025-10-20T21:04:29.921008] [DEBUG] [LOCAL ] SFL: Primary data saved (45331 records). [2025-10-20T21:04:29.921452] [INFO ] [MAIN ] SFL: Full history saved. [2025-10-20T21:04:29.926997] [INFO ] [MAIN ] Update Progress: 1898/2367 symbols completed. SFL took 34.5s. Estimated remaining: 1h 0m 24s. Rate: 7.8 symbols/min [2025-10-20T21:04:30.098401] [INFO ] [REMOTE ] SGU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:30.177997] [INFO ] [MAIN ] SHAK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:30.223944] [DEBUG] [LOCAL ] SHAK: Loaded 49501 existing records from primary storage. [2025-10-20T21:04:30.224255] [INFO ] [MAIN ] SHAK: Loaded 49501 existing records. [2025-10-20T21:04:30.399815] [WARN ] [MAIN ] SHAK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:30.400033] [INFO ] [MAIN ] SHAK: Wiping existing primary data before full refresh. [2025-10-20T21:04:30.400079] [INFO ] [LOCAL ] SHAK: Starting deletion of primary data. [2025-10-20T21:04:30.400380] [DEBUG] [LOCAL ] SHAK: Deleted primary data file. [2025-10-20T21:04:30.400435] [INFO ] [MAIN ] SHAK: Starting full primary refresh. [2025-10-20T21:04:30.663741] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-01. [2025-10-20T21:04:30.752274] [INFO ] [REMOTE ] SHAK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:31.086858] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T21:04:31.162115] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T21:04:31.451484] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-01. [2025-10-20T21:04:31.498819] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:04:32.251465] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-02. [2025-10-20T21:04:32.916243] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T21:04:33.063814] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-09. [2025-10-20T21:04:33.088798] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T21:04:33.851465] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-02. [2025-10-20T21:04:34.651419] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-03. [2025-10-20T21:04:35.107881] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T21:04:35.141796] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T21:04:35.451431] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-04. [2025-10-20T21:04:35.643476] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T21:04:36.153775] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T21:04:36.251398] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-03. [2025-10-20T21:04:37.051483] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-08. [2025-10-20T21:04:37.851406] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-05. [2025-10-20T21:04:38.099476] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T21:04:38.516298] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T21:04:38.651396] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-04. [2025-10-20T21:04:39.221677] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-08 (2.17 seconds). [2025-10-20T21:04:39.451407] [INFO ] [REMOTE ] SGHC: Fetching primary data for 2025-10. [2025-10-20T21:04:40.251419] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-09. [2025-10-20T21:04:40.516890] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T21:04:41.051379] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-06. [2025-10-20T21:04:41.215488] [INFO ] [REMOTE ] SGHC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T21:04:41.223087] [INFO ] [REMOTE ] SGHC: Full fetch complete (65475 records). [2025-10-20T21:04:41.223235] [DEBUG] [VALIDATOR ] SGHC: Validating primary data. [2025-10-20T21:04:41.622277] [DEBUG] [VALIDATOR ] SGHC: All expected trading days are present. [2025-10-20T21:04:41.627537] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T21:04:41.627670] [DEBUG] [VALIDATOR ] SGHC: Primary data validation passed. [2025-10-20T21:04:41.630983] [INFO ] [MAIN ] SGHC: Fetched primary data is valid. [2025-10-20T21:04:41.631070] [DEBUG] [LOCAL ] SGHC: Saving primary data. [2025-10-20T21:04:41.632346] [DEBUG] [LOCAL ] SGHC: Trimmed 65475 records to 47247 (keeping 200 days of data) [2025-10-20T21:04:41.851763] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-07. [2025-10-20T21:04:41.951227] [DEBUG] [LOCAL ] SGHC: Primary data saved (47247 records). [2025-10-20T21:04:41.951708] [INFO ] [MAIN ] SGHC: Full history saved. [2025-10-20T21:04:41.952431] [INFO ] [MAIN ] Update Progress: 1899/2367 symbols completed. SGHC took 31.9s. Estimated remaining: 1h 0m 17s. Rate: 7.8 symbols/min [2025-10-20T21:04:41.954110] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T21:04:42.140590] [INFO ] [MAIN ] SHCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:42.166270] [DEBUG] [LOCAL ] SHCO: Loaded 25956 existing records from primary storage. [2025-10-20T21:04:42.166544] [INFO ] [MAIN ] SHCO: Loaded 25956 existing records. [2025-10-20T21:04:42.341694] [WARN ] [MAIN ] SHCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:42.341883] [INFO ] [MAIN ] SHCO: Wiping existing primary data before full refresh. [2025-10-20T21:04:42.341928] [INFO ] [LOCAL ] SHCO: Starting deletion of primary data. [2025-10-20T21:04:42.342155] [DEBUG] [LOCAL ] SHCO: Deleted primary data file. [2025-10-20T21:04:42.342192] [INFO ] [MAIN ] SHCO: Starting full primary refresh. [2025-10-20T21:04:42.657470] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-05. [2025-10-20T21:04:42.658059] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T21:04:42.766368] [INFO ] [REMOTE ] SHCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:43.451388] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-08. [2025-10-20T21:04:44.251351] [INFO ] [REMOTE ] SG: Fetching primary data for 2025-10. [2025-10-20T21:04:44.271039] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-08 (0.82 seconds). [2025-10-20T21:04:44.542377] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T21:04:44.980330] [INFO ] [REMOTE ] SG: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:04:44.987707] [INFO ] [REMOTE ] SG: Full fetch complete (77082 records). [2025-10-20T21:04:44.987839] [DEBUG] [VALIDATOR ] SG: Validating primary data. [2025-10-20T21:04:44.988788] [FATAL] [VALIDATOR ] SG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:04:44.988872] [FATAL] [MAIN ] SG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:04:44.988914] [WARN ] [MAIN ] SG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:04:44.988952] [INFO ] [LOCAL ] SG: Starting deletion of primary data. [2025-10-20T21:04:44.989008] [DEBUG] [LOCAL ] SG: No primary data file found to delete. [2025-10-20T21:04:44.989043] [INFO ] [LOCAL ] SG: Starting deletion of derivative data. [2025-10-20T21:04:44.989186] [DEBUG] [LOCAL ] SG: Deleted derivative data file. [2025-10-20T21:04:44.989277] [INFO ] [MAIN ] Update Progress: 1900/2367 symbols completed. SG took 36.5s. Estimated remaining: 1h 0m 8s. Rate: 7.8 symbols/min [2025-10-20T21:04:45.056587] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-01. [2025-10-20T21:04:45.164605] [INFO ] [MAIN ] SHEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:45.214812] [DEBUG] [LOCAL ] SHEL: Loaded 54453 existing records from primary storage. [2025-10-20T21:04:45.215132] [INFO ] [MAIN ] SHEL: Loaded 54453 existing records. [2025-10-20T21:04:45.395690] [WARN ] [MAIN ] SHEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:45.395932] [INFO ] [MAIN ] SHEL: Wiping existing primary data before full refresh. [2025-10-20T21:04:45.396001] [INFO ] [LOCAL ] SHEL: Starting deletion of primary data. [2025-10-20T21:04:45.396362] [DEBUG] [LOCAL ] SHEL: Deleted primary data file. [2025-10-20T21:04:45.396416] [INFO ] [MAIN ] SHEL: Starting full primary refresh. [2025-10-20T21:04:45.744008] [INFO ] [REMOTE ] SHEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:45.851403] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-09. [2025-10-20T21:04:46.296685] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T21:04:46.411583] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T21:04:46.651417] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-01. [2025-10-20T21:04:47.366710] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T21:04:47.451335] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-02. [2025-10-20T21:04:48.227507] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T21:04:48.251345] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-03. [2025-10-20T21:04:49.051407] [INFO ] [REMOTE ] SGU: Fetching primary data for 2025-10. [2025-10-20T21:04:49.109773] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T21:04:49.660145] [INFO ] [REMOTE ] SGU: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T21:04:49.661327] [INFO ] [REMOTE ] SGU: Full fetch complete (8945 records). [2025-10-20T21:04:49.661394] [DEBUG] [VALIDATOR ] SGU: Validating primary data. [2025-10-20T21:04:49.857342] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-06. [2025-10-20T21:04:50.019436] [DEBUG] [VALIDATOR ] SGU: All expected trading days are present. [2025-10-20T21:04:50.019506] [DEBUG] [VALIDATOR ] SGU: Primary data validation passed. [2025-10-20T21:04:50.019735] [INFO ] [MAIN ] SGU: Fetched primary data is valid. [2025-10-20T21:04:50.019794] [DEBUG] [LOCAL ] SGU: Saving primary data. [2025-10-20T21:04:50.020183] [DEBUG] [LOCAL ] SGU: Trimmed 8945 records to 6541 (keeping 200 days of data) [2025-10-20T21:04:50.057906] [DEBUG] [LOCAL ] SGU: Primary data saved (6541 records). [2025-10-20T21:04:50.058019] [INFO ] [MAIN ] SGU: Full history saved. [2025-10-20T21:04:50.058093] [INFO ] [MAIN ] Update Progress: 1901/2367 symbols completed. SGU took 21.3s. Estimated remaining: 59m 60s. Rate: 7.8 symbols/min [2025-10-20T21:04:50.223357] [INFO ] [MAIN ] SHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:04:50.252816] [DEBUG] [LOCAL ] SHG: Loaded 27006 existing records from primary storage. [2025-10-20T21:04:50.253155] [INFO ] [MAIN ] SHG: Loaded 27006 existing records. [2025-10-20T21:04:50.430840] [WARN ] [MAIN ] SHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:04:50.431068] [INFO ] [MAIN ] SHG: Wiping existing primary data before full refresh. [2025-10-20T21:04:50.431117] [INFO ] [LOCAL ] SHG: Starting deletion of primary data. [2025-10-20T21:04:50.431317] [DEBUG] [LOCAL ] SHG: Deleted primary data file. [2025-10-20T21:04:50.431379] [INFO ] [MAIN ] SHG: Starting full primary refresh. [2025-10-20T21:04:50.662889] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-04. [2025-10-20T21:04:50.794720] [INFO ] [REMOTE ] SHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:04:51.327462] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T21:04:51.451355] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-05. [2025-10-20T21:04:51.672202] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-06 (1.81 seconds). [2025-10-20T21:04:52.251410] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-01. [2025-10-20T21:04:52.297470] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T21:04:53.051424] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-06. [2025-10-20T21:04:53.571325] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T21:04:53.784464] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:04:53.851386] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-02. [2025-10-20T21:04:54.651366] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-02. [2025-10-20T21:04:55.248613] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T21:04:55.451355] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-07. [2025-10-20T21:04:55.743846] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T21:04:56.251367] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-07. [2025-10-20T21:04:57.051357] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-03. [2025-10-20T21:04:57.187448] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-07 (0.94 seconds). [2025-10-20T21:04:57.243369] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T21:04:57.851373] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-08. [2025-10-20T21:04:58.069403] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T21:04:58.424922] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-08 (0.57 seconds). [2025-10-20T21:04:58.651380] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-04. [2025-10-20T21:04:59.451389] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-03. [2025-10-20T21:05:00.025164] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T21:05:00.251368] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-09. [2025-10-20T21:05:01.004363] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T21:05:01.051350] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-08. [2025-10-20T21:05:01.121532] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T21:05:01.851377] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-05. [2025-10-20T21:05:01.996899] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-08 (0.95 seconds). [2025-10-20T21:05:02.651419] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-04. [2025-10-20T21:05:02.708308] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:05:03.451423] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-09. [2025-10-20T21:05:03.998571] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T21:05:04.256460] [INFO ] [REMOTE ] SHEL: Fetching primary data for 2025-10. [2025-10-20T21:05:04.276590] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:05:05.051365] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-05. [2025-10-20T21:05:05.851435] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-06. [2025-10-20T21:05:06.173418] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-05 (1.12 seconds). [2025-10-20T21:05:06.457325] [INFO ] [REMOTE ] SHEL: Incremental fetch complete for 2025-10 (2.20 seconds). [2025-10-20T21:05:06.465311] [INFO ] [REMOTE ] SHEL: Full fetch complete (77835 records). [2025-10-20T21:05:06.465432] [DEBUG] [VALIDATOR ] SHEL: Validating primary data. [2025-10-20T21:05:06.656560] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-06. [2025-10-20T21:05:06.862254] [DEBUG] [VALIDATOR ] SHEL: All expected trading days are present. [2025-10-20T21:05:06.862356] [DEBUG] [VALIDATOR ] SHEL: Primary data validation passed. [2025-10-20T21:05:06.863491] [INFO ] [MAIN ] SHEL: Fetched primary data is valid. [2025-10-20T21:05:06.863569] [DEBUG] [LOCAL ] SHEL: Saving primary data. [2025-10-20T21:05:06.865115] [DEBUG] [LOCAL ] SHEL: Trimmed 77835 records to 53675 (keeping 200 days of data) [2025-10-20T21:05:06.964598] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:05:07.198725] [DEBUG] [LOCAL ] SHEL: Primary data saved (53675 records). [2025-10-20T21:05:07.198855] [INFO ] [MAIN ] SHEL: Full history saved. [2025-10-20T21:05:07.198930] [INFO ] [MAIN ] Update Progress: 1902/2367 symbols completed. SHEL took 22.2s. Estimated remaining: 59m 55s. Rate: 7.8 symbols/min [2025-10-20T21:05:07.364108] [INFO ] [MAIN ] SHO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:07.405893] [DEBUG] [LOCAL ] SHO: Loaded 50038 existing records from primary storage. [2025-10-20T21:05:07.406183] [INFO ] [MAIN ] SHO: Loaded 50038 existing records. [2025-10-20T21:05:07.456432] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-07. [2025-10-20T21:05:07.603443] [WARN ] [MAIN ] SHO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:07.603703] [INFO ] [MAIN ] SHO: Wiping existing primary data before full refresh. [2025-10-20T21:05:07.603769] [INFO ] [LOCAL ] SHO: Starting deletion of primary data. [2025-10-20T21:05:07.604189] [DEBUG] [LOCAL ] SHO: Deleted primary data file. [2025-10-20T21:05:07.604250] [INFO ] [MAIN ] SHO: Starting full primary refresh. [2025-10-20T21:05:07.878322] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T21:05:07.994188] [INFO ] [REMOTE ] SHO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:08.251433] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-01. [2025-10-20T21:05:08.825887] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T21:05:09.051407] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-08. [2025-10-20T21:05:09.851388] [INFO ] [REMOTE ] SHAK: Fetching primary data for 2025-10. [2025-10-20T21:05:09.898796] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T21:05:10.625244] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T21:05:10.651335] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-09. [2025-10-20T21:05:10.697214] [INFO ] [REMOTE ] SHAK: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T21:05:10.703733] [INFO ] [REMOTE ] SHAK: Full fetch complete (70840 records). [2025-10-20T21:05:10.703863] [DEBUG] [VALIDATOR ] SHAK: Validating primary data. [2025-10-20T21:05:10.704538] [FATAL] [VALIDATOR ] SHAK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:05:10.704666] [FATAL] [MAIN ] SHAK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:05:10.704743] [WARN ] [MAIN ] SHAK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:05:10.704839] [INFO ] [LOCAL ] SHAK: Starting deletion of primary data. [2025-10-20T21:05:10.704919] [DEBUG] [LOCAL ] SHAK: No primary data file found to delete. [2025-10-20T21:05:10.704957] [INFO ] [LOCAL ] SHAK: Starting deletion of derivative data. [2025-10-20T21:05:10.705098] [DEBUG] [LOCAL ] SHAK: Deleted derivative data file. [2025-10-20T21:05:10.705184] [INFO ] [MAIN ] Update Progress: 1903/2367 symbols completed. SHAK took 40.7s. Estimated remaining: 59m 46s. Rate: 7.8 symbols/min [2025-10-20T21:05:10.875615] [INFO ] [MAIN ] SHOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:10.881269] [DEBUG] [LOCAL ] SHOC: Loaded 3283 existing records from primary storage. [2025-10-20T21:05:10.881452] [INFO ] [MAIN ] SHOC: Loaded 3283 existing records. [2025-10-20T21:05:11.060748] [WARN ] [MAIN ] SHOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:11.060975] [INFO ] [MAIN ] SHOC: Wiping existing primary data before full refresh. [2025-10-20T21:05:11.061023] [INFO ] [LOCAL ] SHOC: Starting deletion of primary data. [2025-10-20T21:05:11.061177] [DEBUG] [LOCAL ] SHOC: Deleted primary data file. [2025-10-20T21:05:11.061233] [INFO ] [MAIN ] SHOC: Starting full primary refresh. [2025-10-20T21:05:11.411108] [INFO ] [REMOTE ] SHOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:11.451398] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-02. [2025-10-20T21:05:11.947685] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T21:05:12.251394] [INFO ] [REMOTE ] SHCO: Fetching primary data for 2025-10. [2025-10-20T21:05:12.762592] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:05:13.051357] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-01. [2025-10-20T21:05:13.315673] [INFO ] [REMOTE ] SHCO: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:05:13.319970] [INFO ] [REMOTE ] SHCO: Full fetch complete (37585 records). [2025-10-20T21:05:13.320069] [DEBUG] [VALIDATOR ] SHCO: Validating primary data. [2025-10-20T21:05:13.691731] [DEBUG] [VALIDATOR ] SHCO: All expected trading days are present. [2025-10-20T21:05:13.691813] [DEBUG] [VALIDATOR ] SHCO: Primary data validation passed. [2025-10-20T21:05:13.692545] [INFO ] [MAIN ] SHCO: Fetched primary data is valid. [2025-10-20T21:05:13.692594] [DEBUG] [LOCAL ] SHCO: Saving primary data. [2025-10-20T21:05:13.693533] [DEBUG] [LOCAL ] SHCO: Trimmed 37585 records to 25358 (keeping 200 days of data) [2025-10-20T21:05:13.835063] [DEBUG] [LOCAL ] SHCO: Primary data saved (25358 records). [2025-10-20T21:05:13.835198] [INFO ] [MAIN ] SHCO: Full history saved. [2025-10-20T21:05:13.835279] [INFO ] [MAIN ] Update Progress: 1904/2367 symbols completed. SHCO took 31.9s. Estimated remaining: 59m 37s. Rate: 7.8 symbols/min [2025-10-20T21:05:13.861746] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-03. [2025-10-20T21:05:13.951974] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T21:05:14.020529] [INFO ] [MAIN ] SHW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:14.067932] [DEBUG] [LOCAL ] SHW: Loaded 51119 existing records from primary storage. [2025-10-20T21:05:14.068261] [INFO ] [MAIN ] SHW: Loaded 51119 existing records. [2025-10-20T21:05:14.245130] [WARN ] [MAIN ] SHW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:14.245364] [INFO ] [MAIN ] SHW: Wiping existing primary data before full refresh. [2025-10-20T21:05:14.245421] [INFO ] [LOCAL ] SHW: Starting deletion of primary data. [2025-10-20T21:05:14.245722] [DEBUG] [LOCAL ] SHW: Deleted primary data file. [2025-10-20T21:05:14.245777] [INFO ] [MAIN ] SHW: Starting full primary refresh. [2025-10-20T21:05:14.596079] [INFO ] [REMOTE ] SHW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:14.651350] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-07. [2025-10-20T21:05:15.451364] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-01. [2025-10-20T21:05:15.644459] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:05:16.072969] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T21:05:16.251436] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-02. [2025-10-20T21:05:17.047286] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T21:05:17.056980] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-04. [2025-10-20T21:05:17.183859] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T21:05:17.851474] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-08. [2025-10-20T21:05:18.651383] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-02. [2025-10-20T21:05:18.725080] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T21:05:19.079282] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T21:05:19.451383] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-09. [2025-10-20T21:05:20.251404] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-05. [2025-10-20T21:05:20.380707] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T21:05:20.509276] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T21:05:21.051411] [INFO ] [REMOTE ] SHG: Fetching primary data for 2025-10. [2025-10-20T21:05:21.851361] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-03. [2025-10-20T21:05:22.029491] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T21:05:22.292529] [INFO ] [REMOTE ] SHG: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:05:22.299003] [INFO ] [REMOTE ] SHG: Full fetch complete (41264 records). [2025-10-20T21:05:22.299125] [DEBUG] [VALIDATOR ] SHG: Validating primary data. [2025-10-20T21:05:22.659317] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-03. [2025-10-20T21:05:22.707305] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T21:05:22.707483] [DEBUG] [VALIDATOR ] SHG: All expected trading days are present. [2025-10-20T21:05:22.709926] [DEBUG] [VALIDATOR ] SHG: Primary data validation passed. [2025-10-20T21:05:22.711560] [INFO ] [MAIN ] SHG: Fetched primary data is valid. [2025-10-20T21:05:22.711613] [DEBUG] [LOCAL ] SHG: Saving primary data. [2025-10-20T21:05:22.712820] [DEBUG] [LOCAL ] SHG: Trimmed 41264 records to 26655 (keeping 200 days of data) [2025-10-20T21:05:22.868155] [DEBUG] [LOCAL ] SHG: Primary data saved (26655 records). [2025-10-20T21:05:22.868285] [INFO ] [MAIN ] SHG: Full history saved. [2025-10-20T21:05:22.868362] [INFO ] [MAIN ] Update Progress: 1905/2367 symbols completed. SHG took 32.8s. Estimated remaining: 59m 30s. Rate: 7.8 symbols/min [2025-10-20T21:05:23.031732] [INFO ] [MAIN ] SI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:23.031920] [DEBUG] [LOCAL ] SI: No existing data file found in primary storage. [2025-10-20T21:05:23.031958] [WARN ] [MAIN ] SI: No existing data found, performing full refresh instead. [2025-10-20T21:05:23.032064] [INFO ] [MAIN ] SI: Wiping existing primary data before full refresh. [2025-10-20T21:05:23.032103] [INFO ] [LOCAL ] SI: Starting deletion of primary data. [2025-10-20T21:05:23.032144] [DEBUG] [LOCAL ] SI: No primary data file found to delete. [2025-10-20T21:05:23.032175] [INFO ] [MAIN ] SI: Starting full primary refresh. [2025-10-20T21:05:23.377169] [INFO ] [REMOTE ] SI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:23.451352] [INFO ] [REMOTE ] SI: Fetching primary data for 2025-01. [2025-10-20T21:05:23.728800] [DEBUG] [REMOTE ] SI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:05:23.729120] [WARN ] [REMOTE ] SI: No data returned for month 2025-01 [2025-10-20T21:05:23.729229] [FATAL] [REMOTE ] SI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:05:23.729278] [FATAL] [MAIN ] SI: Failed to fetch primary data. [2025-10-20T21:05:23.729310] [WARN ] [MAIN ] SI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:05:23.729347] [INFO ] [LOCAL ] SI: Starting deletion of primary data. [2025-10-20T21:05:23.729400] [DEBUG] [LOCAL ] SI: No primary data file found to delete. [2025-10-20T21:05:23.729437] [INFO ] [LOCAL ] SI: Starting deletion of derivative data. [2025-10-20T21:05:23.729477] [DEBUG] [LOCAL ] SI: No derivative data file found to delete. [2025-10-20T21:05:23.729521] [INFO ] [MAIN ] Update Progress: 1906/2367 symbols completed. SI took 0.9s. Estimated remaining: 59m 20s. Rate: 7.8 symbols/min [2025-10-20T21:05:23.896752] [INFO ] [MAIN ] SID: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:23.936725] [DEBUG] [LOCAL ] SID: Loaded 43366 existing records from primary storage. [2025-10-20T21:05:23.937036] [INFO ] [MAIN ] SID: Loaded 43366 existing records. [2025-10-20T21:05:24.117035] [WARN ] [MAIN ] SID: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:24.117249] [INFO ] [MAIN ] SID: Wiping existing primary data before full refresh. [2025-10-20T21:05:24.117296] [INFO ] [LOCAL ] SID: Starting deletion of primary data. [2025-10-20T21:05:24.117528] [DEBUG] [LOCAL ] SID: Deleted primary data file. [2025-10-20T21:05:24.117583] [INFO ] [MAIN ] SID: Starting full primary refresh. [2025-10-20T21:05:24.265077] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-06. [2025-10-20T21:05:24.502501] [INFO ] [REMOTE ] SID: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:24.704278] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T21:05:25.051395] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-01. [2025-10-20T21:05:25.807573] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:05:25.851349] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-07. [2025-10-20T21:05:26.660161] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-04. [2025-10-20T21:05:26.665728] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:05:27.451380] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-04. [2025-10-20T21:05:27.721975] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T21:05:28.110130] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T21:05:28.252272] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T21:05:28.252355] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-02. [2025-10-20T21:05:29.051402] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-05. [2025-10-20T21:05:29.726680] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:05:29.851372] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-05. [2025-10-20T21:05:30.396782] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T21:05:30.651407] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-03. [2025-10-20T21:05:30.890281] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T21:05:31.451391] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-08. [2025-10-20T21:05:31.969061] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T21:05:32.251365] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-06. [2025-10-20T21:05:32.807279] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T21:05:33.051349] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-07. [2025-10-20T21:05:33.208432] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T21:05:33.651307] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-07 (0.60 seconds). [2025-10-20T21:05:33.851356] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-08. [2025-10-20T21:05:34.509218] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T21:05:34.651351] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-09. [2025-10-20T21:05:35.451395] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-06. [2025-10-20T21:05:36.251358] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-04. [2025-10-20T21:05:36.413343] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T21:05:36.882522] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:05:37.051357] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-07. [2025-10-20T21:05:37.855831] [INFO ] [REMOTE ] SHO: Fetching primary data for 2025-10. [2025-10-20T21:05:37.864563] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T21:05:38.635818] [INFO ] [REMOTE ] SHO: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:05:38.645004] [INFO ] [REMOTE ] SHO: Full fetch complete (70560 records). [2025-10-20T21:05:38.645359] [DEBUG] [VALIDATOR ] SHO: Validating primary data. [2025-10-20T21:05:38.646273] [FATAL] [VALIDATOR ] SHO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:05:38.646487] [FATAL] [MAIN ] SHO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:05:38.646621] [WARN ] [MAIN ] SHO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:05:38.646725] [INFO ] [LOCAL ] SHO: Starting deletion of primary data. [2025-10-20T21:05:38.646827] [DEBUG] [LOCAL ] SHO: No primary data file found to delete. [2025-10-20T21:05:38.646897] [INFO ] [LOCAL ] SHO: Starting deletion of derivative data. [2025-10-20T21:05:38.647110] [DEBUG] [LOCAL ] SHO: Deleted derivative data file. [2025-10-20T21:05:38.647201] [INFO ] [MAIN ] Update Progress: 1907/2367 symbols completed. SHO took 31.4s. Estimated remaining: 59m 14s. Rate: 7.8 symbols/min [2025-10-20T21:05:38.657433] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-05. [2025-10-20T21:05:38.848910] [INFO ] [MAIN ] SIG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:38.887614] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T21:05:38.936975] [DEBUG] [LOCAL ] SIG: Loaded 46214 existing records from primary storage. [2025-10-20T21:05:38.937301] [INFO ] [MAIN ] SIG: Loaded 46214 existing records. [2025-10-20T21:05:39.119097] [WARN ] [MAIN ] SIG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:39.119281] [INFO ] [MAIN ] SIG: Wiping existing primary data before full refresh. [2025-10-20T21:05:39.119328] [INFO ] [LOCAL ] SIG: Starting deletion of primary data. [2025-10-20T21:05:39.119549] [DEBUG] [LOCAL ] SIG: Deleted primary data file. [2025-10-20T21:05:39.119604] [INFO ] [MAIN ] SIG: Starting full primary refresh. [2025-10-20T21:05:39.467629] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-08. [2025-10-20T21:05:39.473240] [INFO ] [REMOTE ] SIG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:40.210841] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T21:05:40.251346] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-09. [2025-10-20T21:05:40.416132] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T21:05:41.051366] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-06. [2025-10-20T21:05:41.095751] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-09 (0.84 seconds). [2025-10-20T21:05:41.851452] [INFO ] [REMOTE ] SHW: Fetching primary data for 2025-10. [2025-10-20T21:05:42.559416] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:05:42.588133] [INFO ] [REMOTE ] SHW: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:05:42.595579] [INFO ] [REMOTE ] SHW: Full fetch complete (72956 records). [2025-10-20T21:05:42.595752] [DEBUG] [VALIDATOR ] SHW: Validating primary data. [2025-10-20T21:05:42.596532] [FATAL] [VALIDATOR ] SHW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:05:42.596681] [FATAL] [MAIN ] SHW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:05:42.596821] [WARN ] [MAIN ] SHW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:05:42.596894] [INFO ] [LOCAL ] SHW: Starting deletion of primary data. [2025-10-20T21:05:42.596972] [DEBUG] [LOCAL ] SHW: No primary data file found to delete. [2025-10-20T21:05:42.597026] [INFO ] [LOCAL ] SHW: Starting deletion of derivative data. [2025-10-20T21:05:42.597535] [DEBUG] [LOCAL ] SHW: Deleted derivative data file. [2025-10-20T21:05:42.597633] [INFO ] [MAIN ] Update Progress: 1908/2367 symbols completed. SHW took 28.8s. Estimated remaining: 59m 6s. Rate: 7.8 symbols/min [2025-10-20T21:05:42.658024] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-07. [2025-10-20T21:05:42.775361] [INFO ] [MAIN ] SII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:42.798074] [DEBUG] [LOCAL ] SII: Loaded 24434 existing records from primary storage. [2025-10-20T21:05:42.798394] [INFO ] [MAIN ] SII: Loaded 24434 existing records. [2025-10-20T21:05:42.970099] [WARN ] [MAIN ] SII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:42.970316] [INFO ] [MAIN ] SII: Wiping existing primary data before full refresh. [2025-10-20T21:05:42.970362] [INFO ] [LOCAL ] SII: Starting deletion of primary data. [2025-10-20T21:05:42.970566] [DEBUG] [LOCAL ] SII: Deleted primary data file. [2025-10-20T21:05:42.970627] [INFO ] [MAIN ] SII: Starting full primary refresh. [2025-10-20T21:05:43.321260] [INFO ] [REMOTE ] SII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:43.451398] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-01. [2025-10-20T21:05:44.251346] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-01. [2025-10-20T21:05:44.336068] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:05:44.407999] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T21:05:45.051369] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-09. [2025-10-20T21:05:45.619158] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T21:05:45.851357] [INFO ] [REMOTE ] SHOC: Fetching primary data for 2025-10. [2025-10-20T21:05:45.907629] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:05:46.467100] [INFO ] [REMOTE ] SHOC: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T21:05:46.468317] [INFO ] [REMOTE ] SHOC: Full fetch complete (6898 records). [2025-10-20T21:05:46.468386] [DEBUG] [VALIDATOR ] SHOC: Validating primary data. [2025-10-20T21:05:46.656542] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-02. [2025-10-20T21:05:46.829703] [DEBUG] [VALIDATOR ] SHOC: All expected trading days are present. [2025-10-20T21:05:46.829810] [DEBUG] [VALIDATOR ] SHOC: Primary data validation passed. [2025-10-20T21:05:46.830024] [INFO ] [MAIN ] SHOC: Fetched primary data is valid. [2025-10-20T21:05:46.830064] [DEBUG] [LOCAL ] SHOC: Saving primary data. [2025-10-20T21:05:46.830770] [DEBUG] [LOCAL ] SHOC: Trimmed 6898 records to 3140 (keeping 200 days of data) [2025-10-20T21:05:46.849837] [DEBUG] [LOCAL ] SHOC: Primary data saved (3140 records). [2025-10-20T21:05:46.849896] [INFO ] [MAIN ] SHOC: Full history saved. [2025-10-20T21:05:46.849967] [INFO ] [MAIN ] Update Progress: 1909/2367 symbols completed. SHOC took 36.1s. Estimated remaining: 58m 57s. Rate: 7.8 symbols/min [2025-10-20T21:05:47.015165] [INFO ] [MAIN ] SILA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:05:47.043926] [DEBUG] [LOCAL ] SILA: Loaded 30851 existing records from primary storage. [2025-10-20T21:05:47.044178] [INFO ] [MAIN ] SILA: Loaded 30851 existing records. [2025-10-20T21:05:47.222498] [WARN ] [MAIN ] SILA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:05:47.222832] [INFO ] [MAIN ] SILA: Wiping existing primary data before full refresh. [2025-10-20T21:05:47.222891] [INFO ] [LOCAL ] SILA: Starting deletion of primary data. [2025-10-20T21:05:47.223072] [DEBUG] [LOCAL ] SILA: Deleted primary data file. [2025-10-20T21:05:47.223128] [INFO ] [MAIN ] SILA: Starting full primary refresh. [2025-10-20T21:05:47.456691] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-08. [2025-10-20T21:05:47.569948] [INFO ] [REMOTE ] SILA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:05:47.763525] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-02 (1.11 seconds). [2025-10-20T21:05:48.251383] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-02. [2025-10-20T21:05:49.051426] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-03. [2025-10-20T21:05:49.192968] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T21:05:49.853821] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-01. [2025-10-20T21:05:49.854796] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T21:05:49.933784] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T21:05:50.651358] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-03. [2025-10-20T21:05:51.144113] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T21:05:51.451388] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-04. [2025-10-20T21:05:52.155257] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T21:05:52.251337] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-04. [2025-10-20T21:05:52.801091] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T21:05:53.051397] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-05. [2025-10-20T21:05:53.761500] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T21:05:53.851344] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-05. [2025-10-20T21:05:54.306288] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T21:05:54.651392] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-06. [2025-10-20T21:05:55.366920] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:05:55.451345] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-06. [2025-10-20T21:05:56.026916] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T21:05:56.251382] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-09. [2025-10-20T21:05:56.907286] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T21:05:57.051372] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-07. [2025-10-20T21:05:57.823533] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:05:57.851342] [INFO ] [REMOTE ] SID: Fetching primary data for 2025-10. [2025-10-20T21:05:58.651375] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-07. [2025-10-20T21:05:58.837209] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T21:05:59.413327] [INFO ] [REMOTE ] SID: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T21:05:59.420767] [INFO ] [REMOTE ] SID: Full fetch complete (61083 records). [2025-10-20T21:05:59.420904] [DEBUG] [VALIDATOR ] SID: Validating primary data. [2025-10-20T21:05:59.456200] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-02. [2025-10-20T21:05:59.875942] [DEBUG] [VALIDATOR ] SID: All expected trading days are present. [2025-10-20T21:05:59.876090] [DEBUG] [VALIDATOR ] SID: Primary data validation passed. [2025-10-20T21:05:59.877277] [INFO ] [MAIN ] SID: Fetched primary data is valid. [2025-10-20T21:05:59.877390] [DEBUG] [LOCAL ] SID: Saving primary data. [2025-10-20T21:05:59.893201] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-07 (1.24 seconds). [2025-10-20T21:05:59.892420] [DEBUG] [LOCAL ] SID: Trimmed 61083 records to 42841 (keeping 200 days of data) [2025-10-20T21:06:00.138565] [DEBUG] [LOCAL ] SID: Primary data saved (42841 records). [2025-10-20T21:06:00.138715] [INFO ] [MAIN ] SID: Full history saved. [2025-10-20T21:06:00.138800] [INFO ] [MAIN ] Update Progress: 1910/2367 symbols completed. SID took 36.4s. Estimated remaining: 58m 51s. Rate: 7.8 symbols/min [2025-10-20T21:06:00.256511] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-08. [2025-10-20T21:06:00.308141] [INFO ] [MAIN ] SITC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:00.345819] [DEBUG] [LOCAL ] SITC: Loaded 42442 existing records from primary storage. [2025-10-20T21:06:00.346139] [INFO ] [MAIN ] SITC: Loaded 42442 existing records. [2025-10-20T21:06:00.520604] [WARN ] [MAIN ] SITC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:00.520849] [INFO ] [MAIN ] SITC: Wiping existing primary data before full refresh. [2025-10-20T21:06:00.520895] [INFO ] [LOCAL ] SITC: Starting deletion of primary data. [2025-10-20T21:06:00.521061] [DEBUG] [LOCAL ] SITC: Deleted primary data file. [2025-10-20T21:06:00.521099] [INFO ] [MAIN ] SITC: Starting full primary refresh. [2025-10-20T21:06:00.876879] [INFO ] [REMOTE ] SITC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:01.051340] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-08. [2025-10-20T21:06:01.172569] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:06:01.851383] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-03. [2025-10-20T21:06:02.039369] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:06:02.067783] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T21:06:02.651408] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-09. [2025-10-20T21:06:03.018100] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T21:06:03.451387] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-01. [2025-10-20T21:06:03.847830] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T21:06:04.251385] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-09. [2025-10-20T21:06:04.844374] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T21:06:05.051371] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-04. [2025-10-20T21:06:05.851416] [INFO ] [REMOTE ] SII: Fetching primary data for 2025-10. [2025-10-20T21:06:05.908426] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T21:06:06.500248] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T21:06:06.651420] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-05. [2025-10-20T21:06:07.097206] [INFO ] [REMOTE ] SII: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T21:06:07.102079] [INFO ] [REMOTE ] SII: Full fetch complete (31369 records). [2025-10-20T21:06:07.102245] [DEBUG] [VALIDATOR ] SII: Validating primary data. [2025-10-20T21:06:07.454788] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-02. [2025-10-20T21:06:07.502464] [DEBUG] [VALIDATOR ] SII: All expected trading days are present. [2025-10-20T21:06:07.502569] [DEBUG] [VALIDATOR ] SII: Primary data validation passed. [2025-10-20T21:06:07.503931] [INFO ] [MAIN ] SII: Fetched primary data is valid. [2025-10-20T21:06:07.503996] [DEBUG] [LOCAL ] SII: Saving primary data. [2025-10-20T21:06:07.505109] [DEBUG] [LOCAL ] SII: Trimmed 31369 records to 24165 (keeping 200 days of data) [2025-10-20T21:06:07.668416] [DEBUG] [LOCAL ] SII: Primary data saved (24165 records). [2025-10-20T21:06:07.668618] [INFO ] [MAIN ] SII: Full history saved. [2025-10-20T21:06:07.668758] [INFO ] [MAIN ] Update Progress: 1911/2367 symbols completed. SII took 25.1s. Estimated remaining: 58m 43s. Rate: 7.8 symbols/min [2025-10-20T21:06:07.857969] [INFO ] [MAIN ] SITE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:07.899596] [DEBUG] [LOCAL ] SITE: Loaded 36106 existing records from primary storage. [2025-10-20T21:06:07.899982] [INFO ] [MAIN ] SITE: Loaded 36106 existing records. [2025-10-20T21:06:08.093073] [WARN ] [MAIN ] SITE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:08.093512] [INFO ] [MAIN ] SITE: Wiping existing primary data before full refresh. [2025-10-20T21:06:08.093594] [INFO ] [LOCAL ] SITE: Starting deletion of primary data. [2025-10-20T21:06:08.094002] [DEBUG] [LOCAL ] SITE: Deleted primary data file. [2025-10-20T21:06:08.094061] [INFO ] [MAIN ] SITE: Starting full primary refresh. [2025-10-20T21:06:08.227182] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T21:06:08.256449] [INFO ] [REMOTE ] SIG: Fetching primary data for 2025-10. [2025-10-20T21:06:08.504136] [INFO ] [REMOTE ] SITE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:09.035061] [INFO ] [REMOTE ] SIG: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:06:09.043219] [INFO ] [REMOTE ] SIG: Full fetch complete (67517 records). [2025-10-20T21:06:09.043442] [DEBUG] [VALIDATOR ] SIG: Validating primary data. [2025-10-20T21:06:09.044679] [FATAL] [VALIDATOR ] SIG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:06:09.044788] [FATAL] [MAIN ] SIG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:06:09.044843] [WARN ] [MAIN ] SIG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:06:09.044889] [INFO ] [LOCAL ] SIG: Starting deletion of primary data. [2025-10-20T21:06:09.044960] [DEBUG] [LOCAL ] SIG: No primary data file found to delete. [2025-10-20T21:06:09.044998] [INFO ] [LOCAL ] SIG: Starting deletion of derivative data. [2025-10-20T21:06:09.045155] [DEBUG] [LOCAL ] SIG: Deleted derivative data file. [2025-10-20T21:06:09.045257] [INFO ] [MAIN ] Update Progress: 1912/2367 symbols completed. SIG took 30.4s. Estimated remaining: 58m 34s. Rate: 7.8 symbols/min [2025-10-20T21:06:09.060594] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-01. [2025-10-20T21:06:09.298584] [INFO ] [MAIN ] SJM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:09.363896] [DEBUG] [LOCAL ] SJM: Loaded 50860 existing records from primary storage. [2025-10-20T21:06:09.364476] [INFO ] [MAIN ] SJM: Loaded 50860 existing records. [2025-10-20T21:06:09.579016] [WARN ] [MAIN ] SJM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:09.579371] [INFO ] [MAIN ] SJM: Wiping existing primary data before full refresh. [2025-10-20T21:06:09.579424] [INFO ] [LOCAL ] SJM: Starting deletion of primary data. [2025-10-20T21:06:09.579715] [DEBUG] [LOCAL ] SJM: Deleted primary data file. [2025-10-20T21:06:09.579774] [INFO ] [MAIN ] SJM: Starting full primary refresh. [2025-10-20T21:06:09.852732] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-06. [2025-10-20T21:06:10.001027] [INFO ] [REMOTE ] SJM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:10.494576] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T21:06:10.651407] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-02. [2025-10-20T21:06:11.093977] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T21:06:11.451373] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-07. [2025-10-20T21:06:11.862217] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-02 (4.41 seconds). [2025-10-20T21:06:12.159039] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T21:06:12.251394] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-03. [2025-10-20T21:06:12.637293] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T21:06:13.051417] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-08. [2025-10-20T21:06:13.857222] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-01. [2025-10-20T21:06:13.871447] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T21:06:13.932399] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T21:06:14.651448] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-09. [2025-10-20T21:06:15.451420] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-04. [2025-10-20T21:06:16.027186] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-01 (2.17 seconds). [2025-10-20T21:06:16.052036] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:06:16.251401] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-03. [2025-10-20T21:06:16.818794] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T21:06:17.051424] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-05. [2025-10-20T21:06:17.831881] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T21:06:17.851356] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-02. [2025-10-20T21:06:18.520246] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T21:06:18.651404] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-04. [2025-10-20T21:06:19.451370] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-06. [2025-10-20T21:06:19.697953] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:06:19.950529] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T21:06:20.251426] [INFO ] [REMOTE ] SILA: Fetching primary data for 2025-10. [2025-10-20T21:06:21.051349] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-03. [2025-10-20T21:06:21.090607] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T21:06:21.237748] [INFO ] [REMOTE ] SILA: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T21:06:21.242369] [INFO ] [REMOTE ] SILA: Full fetch complete (44446 records). [2025-10-20T21:06:21.242451] [DEBUG] [VALIDATOR ] SILA: Validating primary data. [2025-10-20T21:06:21.629309] [DEBUG] [VALIDATOR ] SILA: All expected trading days are present. [2025-10-20T21:06:21.629397] [DEBUG] [VALIDATOR ] SILA: Primary data validation passed. [2025-10-20T21:06:21.630321] [INFO ] [MAIN ] SILA: Fetched primary data is valid. [2025-10-20T21:06:21.630384] [DEBUG] [LOCAL ] SILA: Saving primary data. [2025-10-20T21:06:21.631337] [DEBUG] [LOCAL ] SILA: Trimmed 44446 records to 30253 (keeping 200 days of data) [2025-10-20T21:06:21.827724] [DEBUG] [LOCAL ] SILA: Primary data saved (30253 records). [2025-10-20T21:06:21.827887] [INFO ] [MAIN ] SILA: Full history saved. [2025-10-20T21:06:21.827992] [INFO ] [MAIN ] Update Progress: 1913/2367 symbols completed. SILA took 35.0s. Estimated remaining: 58m 27s. Rate: 7.8 symbols/min [2025-10-20T21:06:21.857620] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-07. [2025-10-20T21:06:21.998228] [INFO ] [MAIN ] SJT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:22.018183] [DEBUG] [LOCAL ] SJT: Loaded 22927 existing records from primary storage. [2025-10-20T21:06:22.018419] [INFO ] [MAIN ] SJT: Loaded 22927 existing records. [2025-10-20T21:06:22.198386] [WARN ] [MAIN ] SJT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:22.198605] [INFO ] [MAIN ] SJT: Wiping existing primary data before full refresh. [2025-10-20T21:06:22.198676] [INFO ] [LOCAL ] SJT: Starting deletion of primary data. [2025-10-20T21:06:22.198901] [DEBUG] [LOCAL ] SJT: Deleted primary data file. [2025-10-20T21:06:22.198956] [INFO ] [MAIN ] SJT: Starting full primary refresh. [2025-10-20T21:06:22.560764] [INFO ] [REMOTE ] SJT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:22.651344] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-01. [2025-10-20T21:06:22.939365] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T21:06:23.105907] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T21:06:23.451370] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-08. [2025-10-20T21:06:23.785024] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T21:06:24.251446] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-02. [2025-10-20T21:06:24.757634] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T21:06:25.051423] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-05. [2025-10-20T21:06:25.159031] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T21:06:25.851435] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-03. [2025-10-20T21:06:26.222294] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:06:26.651387] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-04. [2025-10-20T21:06:27.007967] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T21:06:27.451403] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-06. [2025-10-20T21:06:28.251412] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-04. [2025-10-20T21:06:28.503390] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T21:06:28.570290] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T21:06:29.051350] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-05. [2025-10-20T21:06:29.414788] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T21:06:29.851391] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-07. [2025-10-20T21:06:30.651438] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-09. [2025-10-20T21:06:30.785978] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T21:06:31.366586] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:06:31.451411] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-08. [2025-10-20T21:06:32.251368] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-05. [2025-10-20T21:06:32.518547] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-09 (1.87 seconds). [2025-10-20T21:06:32.967621] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T21:06:33.051427] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-06. [2025-10-20T21:06:33.112231] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:06:33.851389] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-06. [2025-10-20T21:06:34.651374] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-09. [2025-10-20T21:06:34.981008] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T21:06:35.126466] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-06 (2.08 seconds). [2025-10-20T21:06:35.451435] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-07. [2025-10-20T21:06:36.184366] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:06:36.251417] [INFO ] [REMOTE ] SITC: Fetching primary data for 2025-10. [2025-10-20T21:06:36.458571] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T21:06:36.982483] [INFO ] [REMOTE ] SITC: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:06:36.988614] [INFO ] [REMOTE ] SITC: Full fetch complete (60402 records). [2025-10-20T21:06:36.988790] [DEBUG] [VALIDATOR ] SITC: Validating primary data. [2025-10-20T21:06:36.989498] [FATAL] [VALIDATOR ] SITC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:06:36.989587] [FATAL] [MAIN ] SITC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:06:36.989629] [WARN ] [MAIN ] SITC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:06:36.989715] [INFO ] [LOCAL ] SITC: Starting deletion of primary data. [2025-10-20T21:06:36.989793] [DEBUG] [LOCAL ] SITC: No primary data file found to delete. [2025-10-20T21:06:36.989830] [INFO ] [LOCAL ] SITC: Starting deletion of derivative data. [2025-10-20T21:06:36.989948] [DEBUG] [LOCAL ] SITC: Deleted derivative data file. [2025-10-20T21:06:36.990019] [INFO ] [MAIN ] Update Progress: 1914/2367 symbols completed. SITC took 36.9s. Estimated remaining: 58m 21s. Rate: 7.8 symbols/min [2025-10-20T21:06:37.056457] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-08. [2025-10-20T21:06:37.166932] [INFO ] [MAIN ] SKE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:37.202963] [DEBUG] [LOCAL ] SKE: Loaded 37617 existing records from primary storage. [2025-10-20T21:06:37.203249] [INFO ] [MAIN ] SKE: Loaded 37617 existing records. [2025-10-20T21:06:37.381976] [WARN ] [MAIN ] SKE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:37.382154] [INFO ] [MAIN ] SKE: Wiping existing primary data before full refresh. [2025-10-20T21:06:37.382199] [INFO ] [LOCAL ] SKE: Starting deletion of primary data. [2025-10-20T21:06:37.382365] [DEBUG] [LOCAL ] SKE: Deleted primary data file. [2025-10-20T21:06:37.382402] [INFO ] [MAIN ] SKE: Starting full primary refresh. [2025-10-20T21:06:37.734638] [INFO ] [REMOTE ] SKE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:37.851358] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-01. [2025-10-20T21:06:38.301116] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-08 (1.24 seconds). [2025-10-20T21:06:38.651368] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-09. [2025-10-20T21:06:39.281412] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-01 (1.43 seconds). [2025-10-20T21:06:39.451454] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-07. [2025-10-20T21:06:39.923067] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T21:06:40.251443] [INFO ] [REMOTE ] SJT: Fetching primary data for 2025-10. [2025-10-20T21:06:41.051350] [INFO ] [REMOTE ] SITE: Fetching primary data for 2025-10. [2025-10-20T21:06:41.339606] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:06:41.457016] [INFO ] [REMOTE ] SJT: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T21:06:41.460864] [INFO ] [REMOTE ] SJT: Full fetch complete (31943 records). [2025-10-20T21:06:41.460943] [DEBUG] [VALIDATOR ] SJT: Validating primary data. [2025-10-20T21:06:41.824826] [DEBUG] [VALIDATOR ] SJT: All expected trading days are present. [2025-10-20T21:06:41.824903] [DEBUG] [VALIDATOR ] SJT: Primary data validation passed. [2025-10-20T21:06:41.825802] [INFO ] [MAIN ] SJT: Fetched primary data is valid. [2025-10-20T21:06:41.825863] [DEBUG] [LOCAL ] SJT: Saving primary data. [2025-10-20T21:06:41.826614] [DEBUG] [LOCAL ] SJT: Trimmed 31943 records to 22785 (keeping 200 days of data) [2025-10-20T21:06:41.868509] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-08. [2025-10-20T21:06:41.951006] [DEBUG] [LOCAL ] SJT: Primary data saved (22785 records). [2025-10-20T21:06:41.951123] [INFO ] [MAIN ] SJT: Full history saved. [2025-10-20T21:06:41.951194] [INFO ] [MAIN ] Update Progress: 1915/2367 symbols completed. SJT took 20.1s. Estimated remaining: 58m 13s. Rate: 7.8 symbols/min [2025-10-20T21:06:42.118633] [INFO ] [MAIN ] SKIL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:42.129769] [DEBUG] [LOCAL ] SKIL: Loaded 6839 existing records from primary storage. [2025-10-20T21:06:42.130000] [INFO ] [MAIN ] SKIL: Loaded 6839 existing records. [2025-10-20T21:06:42.311004] [WARN ] [MAIN ] SKIL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:42.311297] [INFO ] [MAIN ] SKIL: Wiping existing primary data before full refresh. [2025-10-20T21:06:42.311458] [INFO ] [LOCAL ] SKIL: Starting deletion of primary data. [2025-10-20T21:06:42.311719] [DEBUG] [LOCAL ] SKIL: Deleted primary data file. [2025-10-20T21:06:42.311803] [INFO ] [MAIN ] SKIL: Starting full primary refresh. [2025-10-20T21:06:42.430048] [INFO ] [REMOTE ] SITE: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T21:06:42.461124] [INFO ] [REMOTE ] SITE: Full fetch complete (50713 records). [2025-10-20T21:06:42.497723] [DEBUG] [VALIDATOR ] SITE: Validating primary data. [2025-10-20T21:06:42.659293] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-02. [2025-10-20T21:06:42.875565] [INFO ] [REMOTE ] SKIL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:43.094441] [DEBUG] [VALIDATOR ] SITE: All expected trading days are present. [2025-10-20T21:06:43.094518] [DEBUG] [VALIDATOR ] SITE: Primary data validation passed. [2025-10-20T21:06:43.095365] [INFO ] [MAIN ] SITE: Fetched primary data is valid. [2025-10-20T21:06:43.095427] [DEBUG] [LOCAL ] SITE: Saving primary data. [2025-10-20T21:06:43.096435] [DEBUG] [LOCAL ] SITE: Trimmed 50713 records to 35597 (keeping 200 days of data) [2025-10-20T21:06:43.304044] [DEBUG] [LOCAL ] SITE: Primary data saved (35597 records). [2025-10-20T21:06:43.304162] [INFO ] [MAIN ] SITE: Full history saved. [2025-10-20T21:06:43.304452] [INFO ] [MAIN ] Update Progress: 1916/2367 symbols completed. SITE took 35.6s. Estimated remaining: 58m 3s. Rate: 7.8 symbols/min [2025-10-20T21:06:43.456555] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-01. [2025-10-20T21:06:43.481561] [INFO ] [MAIN ] SKLZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:06:43.496720] [DEBUG] [LOCAL ] SKLZ: Loaded 11752 existing records from primary storage. [2025-10-20T21:06:43.497088] [INFO ] [MAIN ] SKLZ: Loaded 11752 existing records. [2025-10-20T21:06:43.698951] [WARN ] [MAIN ] SKLZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:06:43.699593] [INFO ] [MAIN ] SKLZ: Wiping existing primary data before full refresh. [2025-10-20T21:06:43.699784] [INFO ] [LOCAL ] SKLZ: Starting deletion of primary data. [2025-10-20T21:06:43.700026] [DEBUG] [LOCAL ] SKLZ: Deleted primary data file. [2025-10-20T21:06:43.700111] [INFO ] [MAIN ] SKLZ: Starting full primary refresh. [2025-10-20T21:06:43.918870] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T21:06:44.119522] [INFO ] [REMOTE ] SKLZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:06:44.162608] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T21:06:44.251337] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-03. [2025-10-20T21:06:44.339980] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:06:45.051381] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-01. [2025-10-20T21:06:45.688336] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:06:45.762769] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T21:06:45.851359] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-02. [2025-10-20T21:06:46.651359] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-02. [2025-10-20T21:06:46.712887] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T21:06:47.268119] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T21:06:47.451400] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-03. [2025-10-20T21:06:47.984915] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-03 (0.53 seconds). [2025-10-20T21:06:48.251402] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-04. [2025-10-20T21:06:48.991246] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T21:06:49.051347] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-05. [2025-10-20T21:06:49.693914] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-05 (0.64 seconds). [2025-10-20T21:06:49.851420] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-09. [2025-10-20T21:06:50.651421] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-03. [2025-10-20T21:06:51.335724] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T21:06:51.451344] [INFO ] [REMOTE ] SJM: Fetching primary data for 2025-10. [2025-10-20T21:06:51.763146] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T21:06:51.924007] [INFO ] [REMOTE ] SJM: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T21:06:51.931040] [INFO ] [REMOTE ] SJM: Full fetch complete (72015 records). [2025-10-20T21:06:51.931160] [DEBUG] [VALIDATOR ] SJM: Validating primary data. [2025-10-20T21:06:51.931976] [FATAL] [VALIDATOR ] SJM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:06:51.932062] [FATAL] [MAIN ] SJM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:06:51.932106] [WARN ] [MAIN ] SJM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:06:51.932145] [INFO ] [LOCAL ] SJM: Starting deletion of primary data. [2025-10-20T21:06:51.932202] [DEBUG] [LOCAL ] SJM: No primary data file found to delete. [2025-10-20T21:06:51.932242] [INFO ] [LOCAL ] SJM: Starting deletion of derivative data. [2025-10-20T21:06:51.932419] [DEBUG] [LOCAL ] SJM: Deleted derivative data file. [2025-10-20T21:06:51.932510] [INFO ] [MAIN ] Update Progress: 1917/2367 symbols completed. SJM took 42.9s. Estimated remaining: 57m 56s. Rate: 7.8 symbols/min [2025-10-20T21:06:52.251465] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-06. [2025-10-20T21:06:53.051435] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-04. [2025-10-20T21:06:53.118702] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-06 (0.87 seconds). [2025-10-20T21:06:53.851428] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-04. [2025-10-20T21:06:54.261295] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T21:06:54.651432] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-05. [2025-10-20T21:06:54.758879] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T21:06:55.451417] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-05. [2025-10-20T21:06:56.217963] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:06:56.251360] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-06. [2025-10-20T21:06:56.360736] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T21:06:57.051415] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-06. [2025-10-20T21:06:57.319197] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T21:06:57.851356] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-07. [2025-10-20T21:06:57.867809] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T21:06:58.651352] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-07. [2025-10-20T21:06:58.685566] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T21:06:59.451355] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-07. [2025-10-20T21:06:59.646683] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T21:07:00.251365] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-08. [2025-10-20T21:07:00.925365] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T21:07:01.051421] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-09. [2025-10-20T21:07:01.226838] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T21:07:01.851383] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-08. [2025-10-20T21:07:01.877282] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:07:02.651401] [INFO ] [REMOTE ] SKIL: Fetching primary data for 2025-10. [2025-10-20T21:07:03.371879] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T21:07:03.388549] [INFO ] [REMOTE ] SKIL: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:07:03.391853] [INFO ] [REMOTE ] SKIL: Full fetch complete (10406 records). [2025-10-20T21:07:03.391963] [DEBUG] [VALIDATOR ] SKIL: Validating primary data. [2025-10-20T21:07:03.459509] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-09. [2025-10-20T21:07:03.776021] [DEBUG] [VALIDATOR ] SKIL: All expected trading days are present. [2025-10-20T21:07:03.776100] [DEBUG] [VALIDATOR ] SKIL: Primary data validation passed. [2025-10-20T21:07:03.776331] [INFO ] [MAIN ] SKIL: Fetched primary data is valid. [2025-10-20T21:07:03.776369] [DEBUG] [LOCAL ] SKIL: Saving primary data. [2025-10-20T21:07:03.776851] [DEBUG] [LOCAL ] SKIL: Trimmed 10406 records to 6812 (keeping 200 days of data) [2025-10-20T21:07:03.817456] [DEBUG] [LOCAL ] SKIL: Primary data saved (6812 records). [2025-10-20T21:07:03.817595] [INFO ] [MAIN ] SKIL: Full history saved. [2025-10-20T21:07:03.817697] [INFO ] [MAIN ] Update Progress: 1918/2367 symbols completed. SKIL took 21.9s. Estimated remaining: 57m 49s. Rate: 7.8 symbols/min [2025-10-20T21:07:04.251369] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-08. [2025-10-20T21:07:05.286185] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T21:07:05.305115] [INFO ] [REMOTE ] SKE: Fetching primary data for 2025-10. [2025-10-20T21:07:05.309366] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-08 (1.06 seconds). [2025-10-20T21:07:05.851401] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-09. [2025-10-20T21:07:06.780213] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T21:07:06.783739] [INFO ] [REMOTE ] SKLZ: Fetching primary data for 2025-10. [2025-10-20T21:07:06.950760] [INFO ] [REMOTE ] SKE: Incremental fetch complete for 2025-10 (1.65 seconds). [2025-10-20T21:07:06.957766] [INFO ] [REMOTE ] SKE: Full fetch complete (52327 records). [2025-10-20T21:07:06.957931] [DEBUG] [VALIDATOR ] SKE: Validating primary data. [2025-10-20T21:07:07.330564] [DEBUG] [VALIDATOR ] SKE: All expected trading days are present. [2025-10-20T21:07:07.330693] [DEBUG] [VALIDATOR ] SKE: Primary data validation passed. [2025-10-20T21:07:07.331717] [INFO ] [MAIN ] SKE: Fetched primary data is valid. [2025-10-20T21:07:07.331781] [DEBUG] [LOCAL ] SKE: Saving primary data. [2025-10-20T21:07:07.332945] [DEBUG] [LOCAL ] SKE: Trimmed 52327 records to 37233 (keeping 200 days of data) [2025-10-20T21:07:07.441422] [INFO ] [REMOTE ] SKLZ: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T21:07:07.458925] [INFO ] [REMOTE ] SKLZ: Full fetch complete (17827 records). [2025-10-20T21:07:07.459145] [DEBUG] [VALIDATOR ] SKLZ: Validating primary data. [2025-10-20T21:07:07.667600] [DEBUG] [LOCAL ] SKE: Primary data saved (37233 records). [2025-10-20T21:07:07.673582] [INFO ] [MAIN ] SKE: Full history saved. [2025-10-20T21:07:07.684050] [INFO ] [MAIN ] Update Progress: 1919/2367 symbols completed. SKE took 30.7s. Estimated remaining: 57m 41s. Rate: 7.8 symbols/min [2025-10-20T21:07:07.931686] [DEBUG] [VALIDATOR ] SKLZ: All expected trading days are present. [2025-10-20T21:07:07.931777] [DEBUG] [VALIDATOR ] SKLZ: Primary data validation passed. [2025-10-20T21:07:07.932149] [INFO ] [MAIN ] SKLZ: Fetched primary data is valid. [2025-10-20T21:07:07.932191] [DEBUG] [LOCAL ] SKLZ: Saving primary data. [2025-10-20T21:07:07.932862] [DEBUG] [LOCAL ] SKLZ: Trimmed 17827 records to 11630 (keeping 200 days of data) [2025-10-20T21:07:07.997357] [DEBUG] [LOCAL ] SKLZ: Primary data saved (11630 records). [2025-10-20T21:07:07.997503] [INFO ] [MAIN ] SKLZ: Full history saved. [2025-10-20T21:07:07.997584] [INFO ] [MAIN ] Update Progress: 1920/2367 symbols completed. SKLZ took 24.7s. Estimated remaining: 57m 31s. Rate: 7.8 symbols/min [2025-10-20T21:07:08.659666] [INFO ] [MAIN ] SKYH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:08.718193] [INFO ] [MAIN ] SKM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:08.724943] [INFO ] [MAIN ] SKY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:08.726128] [INFO ] [MAIN ] SKT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:08.746212] [DEBUG] [LOCAL ] SKYH: Loaded 16482 existing records from primary storage. [2025-10-20T21:07:08.746583] [INFO ] [MAIN ] SKYH: Loaded 16482 existing records. [2025-10-20T21:07:08.894281] [DEBUG] [LOCAL ] SKM: Loaded 35792 existing records from primary storage. [2025-10-20T21:07:08.947271] [INFO ] [MAIN ] SKM: Loaded 35792 existing records. [2025-10-20T21:07:08.947964] [DEBUG] [LOCAL ] SKY: Loaded 39132 existing records from primary storage. [2025-10-20T21:07:08.984279] [INFO ] [MAIN ] SKY: Loaded 39132 existing records. [2025-10-20T21:07:09.037049] [WARN ] [MAIN ] SKYH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:09.100027] [INFO ] [MAIN ] SKYH: Wiping existing primary data before full refresh. [2025-10-20T21:07:09.112381] [INFO ] [LOCAL ] SKYH: Starting deletion of primary data. [2025-10-20T21:07:09.166397] [DEBUG] [LOCAL ] SKT: Loaded 43136 existing records from primary storage. [2025-10-20T21:07:09.195374] [INFO ] [MAIN ] SKT: Loaded 43136 existing records. [2025-10-20T21:07:09.205811] [DEBUG] [LOCAL ] SKYH: Deleted primary data file. [2025-10-20T21:07:09.235722] [INFO ] [MAIN ] SKYH: Starting full primary refresh. [2025-10-20T21:07:09.477817] [WARN ] [MAIN ] SKM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:09.541386] [INFO ] [MAIN ] SKM: Wiping existing primary data before full refresh. [2025-10-20T21:07:09.576553] [INFO ] [LOCAL ] SKM: Starting deletion of primary data. [2025-10-20T21:07:09.663315] [DEBUG] [LOCAL ] SKM: Deleted primary data file. [2025-10-20T21:07:09.728587] [WARN ] [MAIN ] SKY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:09.732243] [INFO ] [MAIN ] SKM: Starting full primary refresh. [2025-10-20T21:07:09.795505] [WARN ] [MAIN ] SKT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:09.781380] [INFO ] [MAIN ] SKY: Wiping existing primary data before full refresh. [2025-10-20T21:07:09.811217] [INFO ] [MAIN ] SKT: Wiping existing primary data before full refresh. [2025-10-20T21:07:09.910685] [INFO ] [LOCAL ] SKY: Starting deletion of primary data. [2025-10-20T21:07:09.901332] [INFO ] [LOCAL ] SKT: Starting deletion of primary data. [2025-10-20T21:07:10.013222] [DEBUG] [LOCAL ] SKT: Deleted primary data file. [2025-10-20T21:07:10.022281] [DEBUG] [LOCAL ] SKY: Deleted primary data file. [2025-10-20T21:07:10.059317] [INFO ] [MAIN ] SKY: Starting full primary refresh. [2025-10-20T21:07:10.054097] [INFO ] [MAIN ] SKT: Starting full primary refresh. [2025-10-20T21:07:10.523306] [INFO ] [REMOTE ] SKYH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:10.573147] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-01. [2025-10-20T21:07:11.080769] [INFO ] [REMOTE ] SKM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:11.120759] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-01. [2025-10-20T21:07:11.212731] [INFO ] [REMOTE ] SKT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:11.227601] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-01. [2025-10-20T21:07:11.271585] [INFO ] [REMOTE ] SKY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:11.272767] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-01. [2025-10-20T21:07:11.536951] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T21:07:11.539967] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-02. [2025-10-20T21:07:12.293234] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:07:12.299257] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-02. [2025-10-20T21:07:12.355404] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T21:07:12.363579] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-02. [2025-10-20T21:07:12.444351] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-02 (0.90 seconds). [2025-10-20T21:07:12.544328] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T21:07:13.051416] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-02. [2025-10-20T21:07:13.497590] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:07:13.857019] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-03. [2025-10-20T21:07:13.861960] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T21:07:14.651405] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-03. [2025-10-20T21:07:14.687294] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:07:14.862014] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T21:07:15.451362] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-04. [2025-10-20T21:07:15.557555] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T21:07:16.251453] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-03. [2025-10-20T21:07:17.051438] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-03. [2025-10-20T21:07:17.088511] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:07:17.783297] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T21:07:17.851427] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-04. [2025-10-20T21:07:18.651398] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-05. [2025-10-20T21:07:18.686414] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:07:19.396164] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T21:07:19.451331] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-05. [2025-10-20T21:07:20.177669] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:07:20.251377] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-04. [2025-10-20T21:07:21.020502] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:07:21.051388] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-06. [2025-10-20T21:07:21.374253] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:07:21.851387] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-04. [2025-10-20T21:07:22.633098] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:07:22.651345] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-07. [2025-10-20T21:07:23.190929] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-04 (1.34 seconds). [2025-10-20T21:07:23.451386] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-05. [2025-10-20T21:07:24.210199] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:07:24.251424] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-08. [2025-10-20T21:07:25.062845] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-06. [2025-10-20T21:07:25.071271] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T21:07:25.565891] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-08 (1.31 seconds). [2025-10-20T21:07:25.851384] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-06. [2025-10-20T21:07:26.571550] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:07:26.651426] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-09. [2025-10-20T21:07:27.409621] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:07:27.451341] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-07. [2025-10-20T21:07:28.251406] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-05. [2025-10-20T21:07:28.396473] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T21:07:28.709675] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T21:07:29.051397] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-08. [2025-10-20T21:07:29.108622] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:07:29.851370] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-07. [2025-10-20T21:07:30.607724] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T21:07:30.651357] [INFO ] [REMOTE ] SKY: Fetching primary data for 2025-10. [2025-10-20T21:07:31.318051] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T21:07:31.376830] [INFO ] [REMOTE ] SKY: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:07:31.382351] [INFO ] [REMOTE ] SKY: Full fetch complete (54050 records). [2025-10-20T21:07:31.382442] [DEBUG] [VALIDATOR ] SKY: Validating primary data. [2025-10-20T21:07:31.382988] [FATAL] [VALIDATOR ] SKY: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:07:31.383069] [FATAL] [MAIN ] SKY: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:07:31.383110] [WARN ] [MAIN ] SKY: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:07:31.383148] [INFO ] [LOCAL ] SKY: Starting deletion of primary data. [2025-10-20T21:07:31.383201] [DEBUG] [LOCAL ] SKY: No primary data file found to delete. [2025-10-20T21:07:31.383234] [INFO ] [LOCAL ] SKY: Starting deletion of derivative data. [2025-10-20T21:07:31.383351] [DEBUG] [LOCAL ] SKY: Deleted derivative data file. [2025-10-20T21:07:31.383413] [INFO ] [MAIN ] Update Progress: 1921/2367 symbols completed. SKY took 23.4s. Estimated remaining: 57m 27s. Rate: 7.8 symbols/min [2025-10-20T21:07:31.456577] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-09. [2025-10-20T21:07:31.585124] [INFO ] [MAIN ] SLAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:31.585331] [DEBUG] [LOCAL ] SLAI: No existing data file found in primary storage. [2025-10-20T21:07:31.585388] [WARN ] [MAIN ] SLAI: No existing data found, performing full refresh instead. [2025-10-20T21:07:31.585498] [INFO ] [MAIN ] SLAI: Wiping existing primary data before full refresh. [2025-10-20T21:07:31.585540] [INFO ] [LOCAL ] SLAI: Starting deletion of primary data. [2025-10-20T21:07:31.585582] [DEBUG] [LOCAL ] SLAI: No primary data file found to delete. [2025-10-20T21:07:31.585613] [INFO ] [MAIN ] SLAI: Starting full primary refresh. [2025-10-20T21:07:31.932885] [INFO ] [REMOTE ] SLAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:32.251386] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-06. [2025-10-20T21:07:33.051352] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-08. [2025-10-20T21:07:33.153422] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T21:07:33.154174] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T21:07:33.851377] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-07. [2025-10-20T21:07:34.489853] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:07:34.586175] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-07 (0.73 seconds). [2025-10-20T21:07:34.651376] [INFO ] [REMOTE ] SKT: Fetching primary data for 2025-10. [2025-10-20T21:07:35.116678] [INFO ] [REMOTE ] SKT: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T21:07:35.122508] [INFO ] [REMOTE ] SKT: Full fetch complete (60341 records). [2025-10-20T21:07:35.122695] [DEBUG] [VALIDATOR ] SKT: Validating primary data. [2025-10-20T21:07:35.123382] [FATAL] [VALIDATOR ] SKT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:07:35.123468] [FATAL] [MAIN ] SKT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:07:35.123516] [WARN ] [MAIN ] SKT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:07:35.123555] [INFO ] [LOCAL ] SKT: Starting deletion of primary data. [2025-10-20T21:07:35.123613] [DEBUG] [LOCAL ] SKT: No primary data file found to delete. [2025-10-20T21:07:35.123698] [INFO ] [LOCAL ] SKT: Starting deletion of derivative data. [2025-10-20T21:07:35.123839] [DEBUG] [LOCAL ] SKT: Deleted derivative data file. [2025-10-20T21:07:35.123910] [INFO ] [MAIN ] Update Progress: 1922/2367 symbols completed. SKT took 27.1s. Estimated remaining: 57m 18s. Rate: 7.8 symbols/min [2025-10-20T21:07:35.302486] [INFO ] [MAIN ] SLB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:35.352040] [DEBUG] [LOCAL ] SLB: Loaded 54477 existing records from primary storage. [2025-10-20T21:07:35.352341] [INFO ] [MAIN ] SLB: Loaded 54477 existing records. [2025-10-20T21:07:35.452267] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-08. [2025-10-20T21:07:35.533615] [WARN ] [MAIN ] SLB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:35.533831] [INFO ] [MAIN ] SLB: Wiping existing primary data before full refresh. [2025-10-20T21:07:35.533876] [INFO ] [LOCAL ] SLB: Starting deletion of primary data. [2025-10-20T21:07:35.534134] [DEBUG] [LOCAL ] SLB: Deleted primary data file. [2025-10-20T21:07:35.534189] [INFO ] [MAIN ] SLB: Starting full primary refresh. [2025-10-20T21:07:35.884169] [INFO ] [REMOTE ] SLB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:36.251392] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-09. [2025-10-20T21:07:36.616524] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T21:07:37.051367] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-09. [2025-10-20T21:07:37.363481] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-09 (1.11 seconds). [2025-10-20T21:07:37.851436] [INFO ] [REMOTE ] SKM: Fetching primary data for 2025-10. [2025-10-20T21:07:38.007269] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T21:07:38.651398] [INFO ] [REMOTE ] SKYH: Fetching primary data for 2025-10. [2025-10-20T21:07:39.135274] [INFO ] [REMOTE ] SKM: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T21:07:39.141322] [INFO ] [REMOTE ] SKM: Full fetch complete (49064 records). [2025-10-20T21:07:39.141457] [DEBUG] [VALIDATOR ] SKM: Validating primary data. [2025-10-20T21:07:39.291096] [INFO ] [REMOTE ] SKYH: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:07:39.320627] [INFO ] [REMOTE ] SKYH: Full fetch complete (22538 records). [2025-10-20T21:07:39.331945] [DEBUG] [VALIDATOR ] SKYH: Validating primary data. [2025-10-20T21:07:39.468023] [INFO ] [REMOTE ] SLAI: Fetching primary data for 2025-01. [2025-10-20T21:07:39.864156] [DEBUG] [REMOTE ] SLAI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:07:39.880135] [WARN ] [REMOTE ] SLAI: No data returned for month 2025-01 [2025-10-20T21:07:39.906240] [DEBUG] [VALIDATOR ] SKM: All expected trading days are present. [2025-10-20T21:07:39.919211] [DEBUG] [VALIDATOR ] SKM: Primary data validation passed. [2025-10-20T21:07:39.937394] [INFO ] [MAIN ] SKM: Fetched primary data is valid. [2025-10-20T21:07:39.903632] [FATAL] [REMOTE ] SLAI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:07:39.937496] [DEBUG] [LOCAL ] SKM: Saving primary data. [2025-10-20T21:07:39.966244] [DEBUG] [VALIDATOR ] SKYH: All expected trading days are present. [2025-10-20T21:07:39.954528] [FATAL] [MAIN ] SLAI: Failed to fetch primary data. [2025-10-20T21:07:39.966577] [DEBUG] [VALIDATOR ] SKYH: Primary data validation passed. [2025-10-20T21:07:39.966445] [WARN ] [MAIN ] SLAI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:07:39.966904] [INFO ] [MAIN ] SKYH: Fetched primary data is valid. [2025-10-20T21:07:39.967121] [DEBUG] [LOCAL ] SKYH: Saving primary data. [2025-10-20T21:07:39.967206] [INFO ] [LOCAL ] SLAI: Starting deletion of primary data. [2025-10-20T21:07:39.967338] [DEBUG] [LOCAL ] SKM: Trimmed 49064 records to 35239 (keeping 200 days of data) [2025-10-20T21:07:39.967896] [DEBUG] [LOCAL ] SLAI: No primary data file found to delete. [2025-10-20T21:07:39.968268] [INFO ] [LOCAL ] SLAI: Starting deletion of derivative data. [2025-10-20T21:07:39.968886] [DEBUG] [LOCAL ] SKYH: Trimmed 22538 records to 15998 (keeping 200 days of data) [2025-10-20T21:07:39.968811] [DEBUG] [LOCAL ] SLAI: No derivative data file found to delete. [2025-10-20T21:07:39.969174] [INFO ] [MAIN ] Update Progress: 1923/2367 symbols completed. SLAI took 8.6s. Estimated remaining: 57m 10s. Rate: 7.8 symbols/min [2025-10-20T21:07:40.255934] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-01. [2025-10-20T21:07:40.333417] [INFO ] [MAIN ] SLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:40.310123] [DEBUG] [LOCAL ] SKYH: Primary data saved (15998 records). [2025-10-20T21:07:40.336373] [INFO ] [MAIN ] SKYH: Full history saved. [2025-10-20T21:07:40.337474] [INFO ] [MAIN ] Update Progress: 1924/2367 symbols completed. SKYH took 32.3s. Estimated remaining: 57m 1s. Rate: 7.8 symbols/min [2025-10-20T21:07:40.580612] [INFO ] [MAIN ] SLG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:40.624500] [DEBUG] [LOCAL ] SLF: Loaded 39322 existing records from primary storage. [2025-10-20T21:07:40.636746] [INFO ] [MAIN ] SLF: Loaded 39322 existing records. [2025-10-20T21:07:40.748125] [DEBUG] [LOCAL ] SKM: Primary data saved (35239 records). [2025-10-20T21:07:40.760087] [INFO ] [MAIN ] SKM: Full history saved. [2025-10-20T21:07:40.760445] [INFO ] [MAIN ] Update Progress: 1925/2367 symbols completed. SKM took 32.8s. Estimated remaining: 56m 51s. Rate: 7.8 symbols/min [2025-10-20T21:07:40.897228] [WARN ] [MAIN ] SLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:40.934359] [INFO ] [MAIN ] SLF: Wiping existing primary data before full refresh. [2025-10-20T21:07:40.934619] [INFO ] [LOCAL ] SLF: Starting deletion of primary data. [2025-10-20T21:07:40.951811] [DEBUG] [LOCAL ] SLF: Deleted primary data file. [2025-10-20T21:07:40.978410] [INFO ] [MAIN ] SLF: Starting full primary refresh. [2025-10-20T21:07:41.011160] [DEBUG] [LOCAL ] SLG: Loaded 46349 existing records from primary storage. [2025-10-20T21:07:41.122729] [INFO ] [MAIN ] SLGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:07:41.114151] [INFO ] [MAIN ] SLG: Loaded 46349 existing records. [2025-10-20T21:07:41.577345] [WARN ] [MAIN ] SLG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:41.598681] [INFO ] [MAIN ] SLG: Wiping existing primary data before full refresh. [2025-10-20T21:07:41.605020] [DEBUG] [LOCAL ] SLGN: Loaded 41517 existing records from primary storage. [2025-10-20T21:07:41.654869] [INFO ] [MAIN ] SLGN: Loaded 41517 existing records. [2025-10-20T21:07:41.659492] [INFO ] [REMOTE ] SLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:41.654745] [INFO ] [LOCAL ] SLG: Starting deletion of primary data. [2025-10-20T21:07:41.689701] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-01. [2025-10-20T21:07:41.702625] [DEBUG] [LOCAL ] SLG: Deleted primary data file. [2025-10-20T21:07:41.729074] [INFO ] [MAIN ] SLG: Starting full primary refresh. [2025-10-20T21:07:41.972935] [WARN ] [MAIN ] SLGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:07:41.996417] [INFO ] [MAIN ] SLGN: Wiping existing primary data before full refresh. [2025-10-20T21:07:42.002244] [INFO ] [LOCAL ] SLGN: Starting deletion of primary data. [2025-10-20T21:07:42.033051] [DEBUG] [LOCAL ] SLGN: Deleted primary data file. [2025-10-20T21:07:42.044343] [INFO ] [MAIN ] SLGN: Starting full primary refresh. [2025-10-20T21:07:42.401626] [INFO ] [REMOTE ] SLG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:42.410530] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-01. [2025-10-20T21:07:42.645758] [INFO ] [REMOTE ] SLGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:07:42.658502] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-01. [2025-10-20T21:07:42.685364] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-01 (2.43 seconds). [2025-10-20T21:07:42.931038] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T21:07:43.451437] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-02. [2025-10-20T21:07:43.826184] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:07:44.069934] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:07:44.251394] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-02. [2025-10-20T21:07:45.051366] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-02. [2025-10-20T21:07:45.097691] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T21:07:45.858989] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-02. [2025-10-20T21:07:45.861356] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T21:07:46.613445] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T21:07:46.651358] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-03. [2025-10-20T21:07:47.339250] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T21:07:47.451370] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-03. [2025-10-20T21:07:48.251346] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-03. [2025-10-20T21:07:48.344985] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:07:49.007106] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T21:07:49.051335] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-04. [2025-10-20T21:07:49.553100] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T21:07:49.851398] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-04. [2025-10-20T21:07:50.325006] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T21:07:50.651351] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-03. [2025-10-20T21:07:51.451352] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-04. [2025-10-20T21:07:51.634223] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:07:52.251405] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-05. [2025-10-20T21:07:52.742205] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-03 (2.09 seconds). [2025-10-20T21:07:53.051344] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-05. [2025-10-20T21:07:53.109765] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T21:07:53.497022] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T21:07:53.851367] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-06. [2025-10-20T21:07:54.486578] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T21:07:54.651386] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-05. [2025-10-20T21:07:55.451370] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-06. [2025-10-20T21:07:55.506131] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:07:55.779732] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T21:07:56.251362] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-06. [2025-10-20T21:07:56.962622] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:07:57.051330] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-07. [2025-10-20T21:07:57.358294] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:07:57.851410] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-07. [2025-10-20T21:07:58.339137] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T21:07:58.651370] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-04. [2025-10-20T21:07:59.412813] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:07:59.451328] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-08. [2025-10-20T21:08:00.251391] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-07. [2025-10-20T21:08:00.860730] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-04 (2.21 seconds). [2025-10-20T21:08:01.051377] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-05. [2025-10-20T21:08:01.284376] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T21:08:01.736762] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T21:08:01.851409] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-09. [2025-10-20T21:08:02.651337] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-08. [2025-10-20T21:08:03.134886] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T21:08:03.456603] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-08. [2025-10-20T21:08:03.472596] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:08:04.251389] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-06. [2025-10-20T21:08:04.403998] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T21:08:05.051398] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-09. [2025-10-20T21:08:05.088501] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:08:05.851375] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-09. [2025-10-20T21:08:06.454862] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-06 (2.20 seconds). [2025-10-20T21:08:06.651391] [INFO ] [REMOTE ] SLGN: Fetching primary data for 2025-10. [2025-10-20T21:08:06.777426] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T21:08:07.136938] [INFO ] [REMOTE ] SLGN: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T21:08:07.143361] [INFO ] [REMOTE ] SLGN: Full fetch complete (58985 records). [2025-10-20T21:08:07.143494] [DEBUG] [VALIDATOR ] SLGN: Validating primary data. [2025-10-20T21:08:07.144229] [FATAL] [VALIDATOR ] SLGN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:08:07.144330] [FATAL] [MAIN ] SLGN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:07.144403] [WARN ] [MAIN ] SLGN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:07.144498] [INFO ] [LOCAL ] SLGN: Starting deletion of primary data. [2025-10-20T21:08:07.144605] [DEBUG] [LOCAL ] SLGN: No primary data file found to delete. [2025-10-20T21:08:07.144690] [INFO ] [LOCAL ] SLGN: Starting deletion of derivative data. [2025-10-20T21:08:07.144852] [DEBUG] [LOCAL ] SLGN: Deleted derivative data file. [2025-10-20T21:08:07.144948] [INFO ] [MAIN ] Update Progress: 1926/2367 symbols completed. SLGN took 26.4s. Estimated remaining: 56m 48s. Rate: 7.8 symbols/min [2025-10-20T21:08:07.328093] [INFO ] [MAIN ] SLNZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:07.328364] [DEBUG] [LOCAL ] SLNZ: No existing data file found in primary storage. [2025-10-20T21:08:07.328435] [WARN ] [MAIN ] SLNZ: No existing data found, performing full refresh instead. [2025-10-20T21:08:07.328736] [INFO ] [MAIN ] SLNZ: Wiping existing primary data before full refresh. [2025-10-20T21:08:07.328875] [INFO ] [LOCAL ] SLNZ: Starting deletion of primary data. [2025-10-20T21:08:07.329051] [DEBUG] [LOCAL ] SLNZ: No primary data file found to delete. [2025-10-20T21:08:07.329249] [INFO ] [MAIN ] SLNZ: Starting full primary refresh. [2025-10-20T21:08:07.460508] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-07. [2025-10-20T21:08:07.474416] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:08:07.746554] [INFO ] [REMOTE ] SLNZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:08.251394] [INFO ] [REMOTE ] SLG: Fetching primary data for 2025-10. [2025-10-20T21:08:08.821576] [INFO ] [REMOTE ] SLG: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T21:08:08.827746] [INFO ] [REMOTE ] SLG: Full fetch complete (64819 records). [2025-10-20T21:08:08.827890] [DEBUG] [VALIDATOR ] SLG: Validating primary data. [2025-10-20T21:08:08.828611] [FATAL] [VALIDATOR ] SLG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:08:08.828721] [FATAL] [MAIN ] SLG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:08.828764] [WARN ] [MAIN ] SLG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:08.828825] [INFO ] [LOCAL ] SLG: Starting deletion of primary data. [2025-10-20T21:08:08.828887] [DEBUG] [LOCAL ] SLG: No primary data file found to delete. [2025-10-20T21:08:08.828921] [INFO ] [LOCAL ] SLG: Starting deletion of derivative data. [2025-10-20T21:08:08.829035] [DEBUG] [LOCAL ] SLG: Deleted derivative data file. [2025-10-20T21:08:08.829106] [INFO ] [MAIN ] Update Progress: 1927/2367 symbols completed. SLG took 28.5s. Estimated remaining: 56m 39s. Rate: 7.8 symbols/min [2025-10-20T21:08:08.998182] [INFO ] [MAIN ] SLQT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:09.036704] [DEBUG] [LOCAL ] SLQT: Loaded 44143 existing records from primary storage. [2025-10-20T21:08:09.036985] [INFO ] [MAIN ] SLQT: Loaded 44143 existing records. [2025-10-20T21:08:09.057105] [INFO ] [REMOTE ] SLF: Fetching primary data for 2025-10. [2025-10-20T21:08:09.214004] [WARN ] [MAIN ] SLQT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:09.214238] [INFO ] [MAIN ] SLQT: Wiping existing primary data before full refresh. [2025-10-20T21:08:09.214284] [INFO ] [LOCAL ] SLQT: Starting deletion of primary data. [2025-10-20T21:08:09.214719] [DEBUG] [LOCAL ] SLQT: Deleted primary data file. [2025-10-20T21:08:09.214773] [INFO ] [MAIN ] SLQT: Starting full primary refresh. [2025-10-20T21:08:09.564275] [INFO ] [REMOTE ] SLQT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:09.827428] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-07 (2.37 seconds). [2025-10-20T21:08:09.851389] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-01. [2025-10-20T21:08:10.049409] [INFO ] [REMOTE ] SLF: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T21:08:10.055035] [INFO ] [REMOTE ] SLF: Full fetch complete (56484 records). [2025-10-20T21:08:10.055172] [DEBUG] [VALIDATOR ] SLF: Validating primary data. [2025-10-20T21:08:10.439572] [DEBUG] [VALIDATOR ] SLF: All expected trading days are present. [2025-10-20T21:08:10.439686] [DEBUG] [VALIDATOR ] SLF: Primary data validation passed. [2025-10-20T21:08:10.440726] [INFO ] [MAIN ] SLF: Fetched primary data is valid. [2025-10-20T21:08:10.440788] [DEBUG] [LOCAL ] SLF: Saving primary data. [2025-10-20T21:08:10.441960] [DEBUG] [LOCAL ] SLF: Trimmed 56484 records to 38690 (keeping 200 days of data) [2025-10-20T21:08:10.655371] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-01. [2025-10-20T21:08:10.682241] [DEBUG] [LOCAL ] SLF: Primary data saved (38690 records). [2025-10-20T21:08:10.682360] [INFO ] [MAIN ] SLF: Full history saved. [2025-10-20T21:08:10.682434] [INFO ] [MAIN ] Update Progress: 1928/2367 symbols completed. SLF took 30.7s. Estimated remaining: 56m 30s. Rate: 7.8 symbols/min [2025-10-20T21:08:10.855665] [INFO ] [MAIN ] SLVM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:10.884164] [DEBUG] [LOCAL ] SLVM: Loaded 28167 existing records from primary storage. [2025-10-20T21:08:10.884424] [INFO ] [MAIN ] SLVM: Loaded 28167 existing records. [2025-10-20T21:08:11.063181] [WARN ] [MAIN ] SLVM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:11.063395] [INFO ] [MAIN ] SLVM: Wiping existing primary data before full refresh. [2025-10-20T21:08:11.063444] [INFO ] [LOCAL ] SLVM: Starting deletion of primary data. [2025-10-20T21:08:11.063609] [DEBUG] [LOCAL ] SLVM: Deleted primary data file. [2025-10-20T21:08:11.063676] [INFO ] [MAIN ] SLVM: Starting full primary refresh. [2025-10-20T21:08:11.111032] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-10-20T21:08:11.422893] [INFO ] [REMOTE ] SLVM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:11.452482] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-02. [2025-10-20T21:08:11.457476] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:08:11.892068] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T21:08:12.251420] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-08. [2025-10-20T21:08:13.051400] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-03. [2025-10-20T21:08:13.133029] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-08 (0.88 seconds). [2025-10-20T21:08:13.655760] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T21:08:13.851425] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-01. [2025-10-20T21:08:14.651357] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-04. [2025-10-20T21:08:15.017169] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:08:15.077028] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T21:08:15.451439] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-09. [2025-10-20T21:08:16.251409] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-02. [2025-10-20T21:08:16.333218] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T21:08:17.051443] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-05. [2025-10-20T21:08:17.554846] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-02 (1.30 seconds). [2025-10-20T21:08:17.597937] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T21:08:17.851462] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-03. [2025-10-20T21:08:18.651416] [INFO ] [REMOTE ] SLB: Fetching primary data for 2025-10. [2025-10-20T21:08:19.243456] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T21:08:19.439517] [INFO ] [REMOTE ] SLB: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:08:19.447029] [INFO ] [REMOTE ] SLB: Full fetch complete (77487 records). [2025-10-20T21:08:19.447199] [DEBUG] [VALIDATOR ] SLB: Validating primary data. [2025-10-20T21:08:19.447974] [FATAL] [VALIDATOR ] SLB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:08:19.448064] [FATAL] [MAIN ] SLB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:19.448105] [WARN ] [MAIN ] SLB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:19.448189] [INFO ] [LOCAL ] SLB: Starting deletion of primary data. [2025-10-20T21:08:19.448269] [DEBUG] [LOCAL ] SLB: No primary data file found to delete. [2025-10-20T21:08:19.448317] [INFO ] [LOCAL ] SLB: Starting deletion of derivative data. [2025-10-20T21:08:19.448463] [DEBUG] [LOCAL ] SLB: Deleted derivative data file. [2025-10-20T21:08:19.448560] [INFO ] [MAIN ] Update Progress: 1929/2367 symbols completed. SLB took 44.3s. Estimated remaining: 56m 22s. Rate: 7.8 symbols/min [2025-10-20T21:08:19.459916] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-02. [2025-10-20T21:08:19.624024] [INFO ] [MAIN ] SM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:19.673116] [DEBUG] [LOCAL ] SM: Loaded 53152 existing records from primary storage. [2025-10-20T21:08:19.673441] [INFO ] [MAIN ] SM: Loaded 53152 existing records. [2025-10-20T21:08:19.848036] [WARN ] [MAIN ] SM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:19.848275] [INFO ] [MAIN ] SM: Wiping existing primary data before full refresh. [2025-10-20T21:08:19.848325] [INFO ] [LOCAL ] SM: Starting deletion of primary data. [2025-10-20T21:08:19.848508] [DEBUG] [LOCAL ] SM: Deleted primary data file. [2025-10-20T21:08:19.848547] [INFO ] [MAIN ] SM: Starting full primary refresh. [2025-10-20T21:08:20.199892] [INFO ] [REMOTE ] SM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:20.251356] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-01. [2025-10-20T21:08:21.051375] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-04. [2025-10-20T21:08:21.275079] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T21:08:21.851381] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-03. [2025-10-20T21:08:22.038602] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T21:08:22.106015] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T21:08:22.651423] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-05. [2025-10-20T21:08:23.440629] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T21:08:23.451329] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-04. [2025-10-20T21:08:23.605581] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T21:08:24.251413] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-06. [2025-10-20T21:08:24.733093] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T21:08:25.051381] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-06. [2025-10-20T21:08:25.429936] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T21:08:25.851365] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-02. [2025-10-20T21:08:26.120225] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T21:08:26.651381] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-07. [2025-10-20T21:08:27.042539] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-07 (0.39 seconds). [2025-10-20T21:08:27.451366] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-08. [2025-10-20T21:08:27.608466] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:08:27.842830] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T21:08:28.251387] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-09. [2025-10-20T21:08:28.640987] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-09 (0.39 seconds). [2025-10-20T21:08:29.051373] [INFO ] [REMOTE ] SLNZ: Fetching primary data for 2025-10. [2025-10-20T21:08:29.354211] [INFO ] [REMOTE ] SLNZ: Incremental fetch complete for 2025-10 (0.30 seconds). [2025-10-20T21:08:29.354933] [INFO ] [REMOTE ] SLNZ: Full fetch complete (1044 records). [2025-10-20T21:08:29.355002] [DEBUG] [VALIDATOR ] SLNZ: Validating primary data. [2025-10-20T21:08:29.567709] [WARN ] [VALIDATOR ] SLNZ: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-10-20T21:08:29.567934] [FATAL] [MAIN ] SLNZ: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:29.568003] [WARN ] [MAIN ] SLNZ: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:29.568066] [INFO ] [LOCAL ] SLNZ: Starting deletion of primary data. [2025-10-20T21:08:29.568161] [DEBUG] [LOCAL ] SLNZ: No primary data file found to delete. [2025-10-20T21:08:29.568211] [INFO ] [LOCAL ] SLNZ: Starting deletion of derivative data. [2025-10-20T21:08:29.568255] [DEBUG] [LOCAL ] SLNZ: No derivative data file found to delete. [2025-10-20T21:08:29.568312] [INFO ] [MAIN ] Update Progress: 1930/2367 symbols completed. SLNZ took 22.4s. Estimated remaining: 56m 15s. Rate: 7.8 symbols/min [2025-10-20T21:08:29.777067] [INFO ] [MAIN ] SMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:29.777263] [DEBUG] [LOCAL ] SMA: No existing data file found in primary storage. [2025-10-20T21:08:29.777308] [WARN ] [MAIN ] SMA: No existing data found, performing full refresh instead. [2025-10-20T21:08:29.777437] [INFO ] [MAIN ] SMA: Wiping existing primary data before full refresh. [2025-10-20T21:08:29.777514] [INFO ] [LOCAL ] SMA: Starting deletion of primary data. [2025-10-20T21:08:29.777601] [DEBUG] [LOCAL ] SMA: No primary data file found to delete. [2025-10-20T21:08:29.777701] [INFO ] [MAIN ] SMA: Starting full primary refresh. [2025-10-20T21:08:29.856471] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-07. [2025-10-20T21:08:30.217748] [INFO ] [REMOTE ] SMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:30.651368] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-03. [2025-10-20T21:08:31.185918] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T21:08:31.451417] [INFO ] [REMOTE ] SMA: Fetching primary data for 2025-01. [2025-10-20T21:08:31.707414] [DEBUG] [REMOTE ] SMA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:08:31.707743] [WARN ] [REMOTE ] SMA: No data returned for month 2025-01 [2025-10-20T21:08:31.707830] [FATAL] [REMOTE ] SMA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:08:31.707935] [FATAL] [MAIN ] SMA: Failed to fetch primary data. [2025-10-20T21:08:31.708004] [WARN ] [MAIN ] SMA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:08:31.708071] [INFO ] [LOCAL ] SMA: Starting deletion of primary data. [2025-10-20T21:08:31.708133] [DEBUG] [LOCAL ] SMA: No primary data file found to delete. [2025-10-20T21:08:31.708167] [INFO ] [LOCAL ] SMA: Starting deletion of derivative data. [2025-10-20T21:08:31.708205] [DEBUG] [LOCAL ] SMA: No derivative data file found to delete. [2025-10-20T21:08:31.708264] [INFO ] [MAIN ] Update Progress: 1931/2367 symbols completed. SMA took 2.1s. Estimated remaining: 56m 6s. Rate: 7.8 symbols/min [2025-10-20T21:08:31.879009] [INFO ] [MAIN ] SMBK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:31.889023] [DEBUG] [LOCAL ] SMBK: Loaded 8414 existing records from primary storage. [2025-10-20T21:08:31.889226] [INFO ] [MAIN ] SMBK: Loaded 8414 existing records. [2025-10-20T21:08:32.064889] [WARN ] [MAIN ] SMBK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:32.065340] [INFO ] [MAIN ] SMBK: Wiping existing primary data before full refresh. [2025-10-20T21:08:32.065422] [INFO ] [LOCAL ] SMBK: Starting deletion of primary data. [2025-10-20T21:08:32.065575] [DEBUG] [LOCAL ] SMBK: Deleted primary data file. [2025-10-20T21:08:32.065630] [INFO ] [MAIN ] SMBK: Starting full primary refresh. [2025-10-20T21:08:32.256577] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-08. [2025-10-20T21:08:32.424829] [INFO ] [REMOTE ] SMBK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:32.508999] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T21:08:33.051397] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-01. [2025-10-20T21:08:33.770005] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T21:08:33.856451] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-04. [2025-10-20T21:08:33.867819] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:08:34.651497] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-02. [2025-10-20T21:08:35.322696] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T21:08:35.451358] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-09. [2025-10-20T21:08:35.713796] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T21:08:36.251418] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-05. [2025-10-20T21:08:36.374781] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T21:08:37.051381] [INFO ] [REMOTE ] SLVM: Fetching primary data for 2025-10. [2025-10-20T21:08:37.777407] [INFO ] [REMOTE ] SLVM: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:08:37.782016] [INFO ] [REMOTE ] SLVM: Full fetch complete (39959 records). [2025-10-20T21:08:37.782100] [DEBUG] [VALIDATOR ] SLVM: Validating primary data. [2025-10-20T21:08:37.782634] [FATAL] [VALIDATOR ] SLVM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:08:37.782737] [FATAL] [MAIN ] SLVM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:37.782778] [WARN ] [MAIN ] SLVM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:37.782814] [INFO ] [LOCAL ] SLVM: Starting deletion of primary data. [2025-10-20T21:08:37.782891] [DEBUG] [LOCAL ] SLVM: No primary data file found to delete. [2025-10-20T21:08:37.782958] [INFO ] [LOCAL ] SLVM: Starting deletion of derivative data. [2025-10-20T21:08:37.783097] [DEBUG] [LOCAL ] SLVM: Deleted derivative data file. [2025-10-20T21:08:37.783185] [INFO ] [MAIN ] Update Progress: 1932/2367 symbols completed. SLVM took 27.1s. Estimated remaining: 55m 58s. Rate: 7.8 symbols/min [2025-10-20T21:08:37.857358] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-03. [2025-10-20T21:08:37.959189] [INFO ] [MAIN ] SMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:37.988795] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T21:08:38.007584] [DEBUG] [LOCAL ] SMC: Loaded 11371 existing records from primary storage. [2025-10-20T21:08:38.007825] [INFO ] [MAIN ] SMC: Loaded 11371 existing records. [2025-10-20T21:08:38.184433] [WARN ] [MAIN ] SMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:38.184635] [INFO ] [MAIN ] SMC: Wiping existing primary data before full refresh. [2025-10-20T21:08:38.184727] [INFO ] [LOCAL ] SMC: Starting deletion of primary data. [2025-10-20T21:08:38.184981] [DEBUG] [LOCAL ] SMC: Deleted primary data file. [2025-10-20T21:08:38.185037] [INFO ] [MAIN ] SMC: Starting full primary refresh. [2025-10-20T21:08:38.531639] [INFO ] [REMOTE ] SMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:38.651367] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-06. [2025-10-20T21:08:38.793250] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T21:08:39.451372] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-01. [2025-10-20T21:08:40.251484] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-04. [2025-10-20T21:08:40.349559] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T21:08:40.417254] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:08:41.051367] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-07. [2025-10-20T21:08:41.115531] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-04 (0.86 seconds). [2025-10-20T21:08:41.851449] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-02. [2025-10-20T21:08:42.651371] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-05. [2025-10-20T21:08:42.693523] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T21:08:42.915704] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-07 (1.86 seconds). [2025-10-20T21:08:43.229603] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T21:08:43.451383] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-05. [2025-10-20T21:08:44.251384] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-03. [2025-10-20T21:08:45.051394] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-06. [2025-10-20T21:08:45.053719] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T21:08:45.161190] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T21:08:45.588470] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-06 (0.54 seconds). [2025-10-20T21:08:45.851392] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-04. [2025-10-20T21:08:46.651414] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-08. [2025-10-20T21:08:46.920362] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T21:08:47.451421] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-05. [2025-10-20T21:08:48.251422] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-06. [2025-10-20T21:08:48.264409] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T21:08:48.436197] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:08:49.051370] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-06. [2025-10-20T21:08:49.828362] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:08:49.851338] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-09. [2025-10-20T21:08:49.910666] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T21:08:50.651381] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-07. [2025-10-20T21:08:51.451389] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-07. [2025-10-20T21:08:51.570781] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T21:08:51.709184] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T21:08:52.251417] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-08. [2025-10-20T21:08:52.845813] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T21:08:53.049489] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T21:08:53.051783] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-08. [2025-10-20T21:08:53.851376] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-09. [2025-10-20T21:08:54.640697] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T21:08:54.651333] [INFO ] [REMOTE ] SMBK: Fetching primary data for 2025-10. [2025-10-20T21:08:54.994928] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T21:08:55.451376] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-07. [2025-10-20T21:08:55.507358] [INFO ] [REMOTE ] SMBK: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T21:08:55.509522] [INFO ] [REMOTE ] SMBK: Full fetch complete (11830 records). [2025-10-20T21:08:55.509604] [DEBUG] [VALIDATOR ] SMBK: Validating primary data. [2025-10-20T21:08:55.886064] [DEBUG] [VALIDATOR ] SMBK: All expected trading days are present. [2025-10-20T21:08:55.886146] [DEBUG] [VALIDATOR ] SMBK: Primary data validation passed. [2025-10-20T21:08:55.886452] [INFO ] [MAIN ] SMBK: Fetched primary data is valid. [2025-10-20T21:08:55.886492] [DEBUG] [LOCAL ] SMBK: Saving primary data. [2025-10-20T21:08:55.887052] [DEBUG] [LOCAL ] SMBK: Trimmed 11830 records to 8274 (keeping 200 days of data) [2025-10-20T21:08:55.941299] [DEBUG] [LOCAL ] SMBK: Primary data saved (8274 records). [2025-10-20T21:08:55.941436] [INFO ] [MAIN ] SMBK: Full history saved. [2025-10-20T21:08:55.941512] [INFO ] [MAIN ] Update Progress: 1933/2367 symbols completed. SMBK took 24.2s. Estimated remaining: 55m 52s. Rate: 7.8 symbols/min [2025-10-20T21:08:56.112893] [INFO ] [MAIN ] SMFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:56.127818] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T21:08:56.163948] [DEBUG] [LOCAL ] SMFG: Loaded 49084 existing records from primary storage. [2025-10-20T21:08:56.164248] [INFO ] [MAIN ] SMFG: Loaded 49084 existing records. [2025-10-20T21:08:56.260434] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-08. [2025-10-20T21:08:56.342314] [WARN ] [MAIN ] SMFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:56.342532] [INFO ] [MAIN ] SMFG: Wiping existing primary data before full refresh. [2025-10-20T21:08:56.342588] [INFO ] [LOCAL ] SMFG: Starting deletion of primary data. [2025-10-20T21:08:56.342914] [DEBUG] [LOCAL ] SMFG: Deleted primary data file. [2025-10-20T21:08:56.342970] [INFO ] [MAIN ] SMFG: Starting full primary refresh. [2025-10-20T21:08:56.690401] [INFO ] [REMOTE ] SMFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:57.051372] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-09. [2025-10-20T21:08:57.220185] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T21:08:57.851364] [INFO ] [REMOTE ] SM: Fetching primary data for 2025-10. [2025-10-20T21:08:58.436868] [INFO ] [REMOTE ] SM: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T21:08:58.462621] [INFO ] [REMOTE ] SM: Full fetch complete (75475 records). [2025-10-20T21:08:58.465618] [DEBUG] [VALIDATOR ] SM: Validating primary data. [2025-10-20T21:08:58.466578] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T21:08:58.474990] [FATAL] [VALIDATOR ] SM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:08:58.475384] [FATAL] [MAIN ] SM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:08:58.475565] [WARN ] [MAIN ] SM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:08:58.475671] [INFO ] [LOCAL ] SM: Starting deletion of primary data. [2025-10-20T21:08:58.475757] [DEBUG] [LOCAL ] SM: No primary data file found to delete. [2025-10-20T21:08:58.475805] [INFO ] [LOCAL ] SM: Starting deletion of derivative data. [2025-10-20T21:08:58.476172] [DEBUG] [LOCAL ] SM: Deleted derivative data file. [2025-10-20T21:08:58.476271] [INFO ] [MAIN ] Update Progress: 1934/2367 symbols completed. SM took 39.0s. Estimated remaining: 55m 44s. Rate: 7.8 symbols/min [2025-10-20T21:08:58.658606] [INFO ] [MAIN ] SMG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:08:58.656436] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-09. [2025-10-20T21:08:58.699118] [DEBUG] [LOCAL ] SMG: Loaded 43860 existing records from primary storage. [2025-10-20T21:08:58.699401] [INFO ] [MAIN ] SMG: Loaded 43860 existing records. [2025-10-20T21:08:58.876711] [WARN ] [MAIN ] SMG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:08:58.876910] [INFO ] [MAIN ] SMG: Wiping existing primary data before full refresh. [2025-10-20T21:08:58.876956] [INFO ] [LOCAL ] SMG: Starting deletion of primary data. [2025-10-20T21:08:58.877243] [DEBUG] [LOCAL ] SMG: Deleted primary data file. [2025-10-20T21:08:58.877280] [INFO ] [MAIN ] SMG: Starting full primary refresh. [2025-10-20T21:08:59.224928] [INFO ] [REMOTE ] SMG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:08:59.451347] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-01. [2025-10-20T21:08:59.563239] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T21:09:00.251379] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-01. [2025-10-20T21:09:01.051351] [INFO ] [REMOTE ] SLQT: Fetching primary data for 2025-10. [2025-10-20T21:09:01.109551] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:09:01.851400] [INFO ] [REMOTE ] SMC: Fetching primary data for 2025-10. [2025-10-20T21:09:02.214013] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T21:09:02.597830] [INFO ] [REMOTE ] SMC: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:09:02.599865] [INFO ] [REMOTE ] SMC: Full fetch complete (16560 records). [2025-10-20T21:09:02.599933] [DEBUG] [VALIDATOR ] SMC: Validating primary data. [2025-10-20T21:09:02.656488] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-02. [2025-10-20T21:09:02.836437] [INFO ] [REMOTE ] SLQT: Incremental fetch complete for 2025-10 (1.79 seconds). [2025-10-20T21:09:02.936931] [INFO ] [REMOTE ] SLQT: Full fetch complete (63889 records). [2025-10-20T21:09:02.958541] [DEBUG] [VALIDATOR ] SLQT: Validating primary data. [2025-10-20T21:09:03.165071] [DEBUG] [VALIDATOR ] SMC: All expected trading days are present. [2025-10-20T21:09:03.165169] [DEBUG] [VALIDATOR ] SMC: Primary data validation passed. [2025-10-20T21:09:03.165834] [INFO ] [MAIN ] SMC: Fetched primary data is valid. [2025-10-20T21:09:03.165911] [DEBUG] [LOCAL ] SMC: Saving primary data. [2025-10-20T21:09:03.208350] [DEBUG] [LOCAL ] SMC: Trimmed 16560 records to 11111 (keeping 200 days of data) [2025-10-20T21:09:03.455509] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-02. [2025-10-20T21:09:03.463319] [DEBUG] [LOCAL ] SMC: Primary data saved (11111 records). [2025-10-20T21:09:03.478142] [INFO ] [MAIN ] SMC: Full history saved. [2025-10-20T21:09:03.519226] [INFO ] [MAIN ] Update Progress: 1935/2367 symbols completed. SMC took 25.7s. Estimated remaining: 55m 35s. Rate: 7.8 symbols/min [2025-10-20T21:09:03.535014] [DEBUG] [VALIDATOR ] SLQT: All expected trading days are present. [2025-10-20T21:09:03.580389] [DEBUG] [VALIDATOR ] SLQT: Primary data validation passed. [2025-10-20T21:09:03.604610] [INFO ] [MAIN ] SLQT: Fetched primary data is valid. [2025-10-20T21:09:03.611524] [DEBUG] [LOCAL ] SLQT: Saving primary data. [2025-10-20T21:09:03.652721] [DEBUG] [LOCAL ] SLQT: Trimmed 63889 records to 43489 (keeping 200 days of data) [2025-10-20T21:09:03.829300] [INFO ] [MAIN ] SMHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:03.925724] [DEBUG] [LOCAL ] SMHI: Loaded 13216 existing records from primary storage. [2025-10-20T21:09:03.954778] [INFO ] [MAIN ] SMHI: Loaded 13216 existing records. [2025-10-20T21:09:04.180627] [WARN ] [MAIN ] SMHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:04.183577] [INFO ] [MAIN ] SMHI: Wiping existing primary data before full refresh. [2025-10-20T21:09:04.192519] [INFO ] [LOCAL ] SMHI: Starting deletion of primary data. [2025-10-20T21:09:04.197884] [DEBUG] [LOCAL ] SMHI: Deleted primary data file. [2025-10-20T21:09:04.197991] [INFO ] [MAIN ] SMHI: Starting full primary refresh. [2025-10-20T21:09:04.240244] [DEBUG] [LOCAL ] SLQT: Primary data saved (43489 records). [2025-10-20T21:09:04.247411] [INFO ] [MAIN ] SLQT: Full history saved. [2025-10-20T21:09:04.260036] [INFO ] [MAIN ] Update Progress: 1936/2367 symbols completed. SLQT took 55.4s. Estimated remaining: 55m 26s. Rate: 7.8 symbols/min [2025-10-20T21:09:04.581504] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T21:09:04.666073] [INFO ] [MAIN ] SMP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:04.710489] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-03. [2025-10-20T21:09:04.796781] [INFO ] [REMOTE ] SMHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:04.805611] [DEBUG] [LOCAL ] SMP: Loaded 19099 existing records from primary storage. [2025-10-20T21:09:04.805854] [INFO ] [MAIN ] SMP: Loaded 19099 existing records. [2025-10-20T21:09:04.954620] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T21:09:05.014801] [WARN ] [MAIN ] SMP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:05.015004] [INFO ] [MAIN ] SMP: Wiping existing primary data before full refresh. [2025-10-20T21:09:05.015052] [INFO ] [LOCAL ] SMP: Starting deletion of primary data. [2025-10-20T21:09:05.015215] [DEBUG] [LOCAL ] SMP: Deleted primary data file. [2025-10-20T21:09:05.015254] [INFO ] [MAIN ] SMP: Starting full primary refresh. [2025-10-20T21:09:05.057968] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-03. [2025-10-20T21:09:05.371778] [INFO ] [REMOTE ] SMP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:05.851376] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-01. [2025-10-20T21:09:06.483485] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T21:09:06.651351] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-04. [2025-10-20T21:09:06.729374] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:09:06.888361] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:09:07.451401] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-01. [2025-10-20T21:09:08.251363] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-04. [2025-10-20T21:09:08.287013] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T21:09:08.434976] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:09:09.051377] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-05. [2025-10-20T21:09:09.851419] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-02. [2025-10-20T21:09:10.063318] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T21:09:10.651382] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-05. [2025-10-20T21:09:10.816616] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T21:09:10.905456] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T21:09:11.451455] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-03. [2025-10-20T21:09:12.251360] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-02. [2025-10-20T21:09:12.443357] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T21:09:12.481612] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T21:09:13.033230] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T21:09:13.051349] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-03. [2025-10-20T21:09:13.851401] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-04. [2025-10-20T21:09:14.338103] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T21:09:14.651389] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-06. [2025-10-20T21:09:15.025496] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T21:09:15.451391] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-04. [2025-10-20T21:09:16.251402] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-05. [2025-10-20T21:09:16.362075] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:09:16.750847] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T21:09:17.051352] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-05. [2025-10-20T21:09:17.205728] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T21:09:17.851383] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-07. [2025-10-20T21:09:18.311935] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T21:09:18.651429] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-06. [2025-10-20T21:09:19.451428] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-06. [2025-10-20T21:09:19.601411] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T21:09:20.104969] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T21:09:20.251378] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-07. [2025-10-20T21:09:20.586410] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T21:09:21.051377] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-06. [2025-10-20T21:09:21.857523] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-08. [2025-10-20T21:09:21.863553] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:09:21.976993] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T21:09:22.651367] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-07. [2025-10-20T21:09:23.451351] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-08. [2025-10-20T21:09:23.678749] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T21:09:23.839547] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-08 (1.98 seconds). [2025-10-20T21:09:24.251344] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-08. [2025-10-20T21:09:25.051408] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-07. [2025-10-20T21:09:25.100482] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:09:25.168769] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T21:09:25.851351] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-09. [2025-10-20T21:09:26.158738] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T21:09:26.651444] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-09. [2025-10-20T21:09:27.451358] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-08. [2025-10-20T21:09:27.656346] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T21:09:27.938390] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T21:09:28.251345] [INFO ] [REMOTE ] SMG: Fetching primary data for 2025-10. [2025-10-20T21:09:28.561924] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T21:09:28.669460] [INFO ] [REMOTE ] SMG: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T21:09:28.675357] [INFO ] [REMOTE ] SMG: Full fetch complete (62715 records). [2025-10-20T21:09:28.675459] [DEBUG] [VALIDATOR ] SMG: Validating primary data. [2025-10-20T21:09:28.676176] [FATAL] [VALIDATOR ] SMG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:09:28.676259] [FATAL] [MAIN ] SMG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:09:28.676303] [WARN ] [MAIN ] SMG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:09:28.676370] [INFO ] [LOCAL ] SMG: Starting deletion of primary data. [2025-10-20T21:09:28.676432] [DEBUG] [LOCAL ] SMG: No primary data file found to delete. [2025-10-20T21:09:28.676467] [INFO ] [LOCAL ] SMG: Starting deletion of derivative data. [2025-10-20T21:09:28.676635] [DEBUG] [LOCAL ] SMG: Deleted derivative data file. [2025-10-20T21:09:28.676749] [INFO ] [MAIN ] Update Progress: 1937/2367 symbols completed. SMG took 30.2s. Estimated remaining: 55m 22s. Rate: 7.8 symbols/min [2025-10-20T21:09:28.846161] [INFO ] [MAIN ] SMR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:28.895388] [DEBUG] [LOCAL ] SMR: Loaded 54579 existing records from primary storage. [2025-10-20T21:09:28.895717] [INFO ] [MAIN ] SMR: Loaded 54579 existing records. [2025-10-20T21:09:29.055360] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-09. [2025-10-20T21:09:29.078330] [WARN ] [MAIN ] SMR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:29.079249] [INFO ] [MAIN ] SMR: Wiping existing primary data before full refresh. [2025-10-20T21:09:29.079325] [INFO ] [LOCAL ] SMR: Starting deletion of primary data. [2025-10-20T21:09:29.079697] [DEBUG] [LOCAL ] SMR: Deleted primary data file. [2025-10-20T21:09:29.079768] [INFO ] [MAIN ] SMR: Starting full primary refresh. [2025-10-20T21:09:29.435676] [INFO ] [REMOTE ] SMR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:29.851380] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-09. [2025-10-20T21:09:29.981057] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T21:09:30.651362] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-01. [2025-10-20T21:09:30.895041] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T21:09:31.451384] [INFO ] [REMOTE ] SMHI: Fetching primary data for 2025-10. [2025-10-20T21:09:32.168462] [INFO ] [REMOTE ] SMHI: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T21:09:32.170254] [INFO ] [REMOTE ] SMHI: Full fetch complete (21515 records). [2025-10-20T21:09:32.170361] [DEBUG] [VALIDATOR ] SMHI: Validating primary data. [2025-10-20T21:09:32.256572] [INFO ] [REMOTE ] SMFG: Fetching primary data for 2025-10. [2025-10-20T21:09:32.574878] [DEBUG] [VALIDATOR ] SMHI: All expected trading days are present. [2025-10-20T21:09:32.574966] [DEBUG] [VALIDATOR ] SMHI: Primary data validation passed. [2025-10-20T21:09:32.575406] [INFO ] [MAIN ] SMHI: Fetched primary data is valid. [2025-10-20T21:09:32.575445] [DEBUG] [LOCAL ] SMHI: Saving primary data. [2025-10-20T21:09:32.576192] [DEBUG] [LOCAL ] SMHI: Trimmed 21515 records to 13011 (keeping 200 days of data) [2025-10-20T21:09:32.647701] [DEBUG] [LOCAL ] SMHI: Primary data saved (13011 records). [2025-10-20T21:09:32.647900] [INFO ] [MAIN ] SMHI: Full history saved. [2025-10-20T21:09:32.647995] [INFO ] [MAIN ] Update Progress: 1938/2367 symbols completed. SMHI took 29.1s. Estimated remaining: 55m 13s. Rate: 7.8 symbols/min [2025-10-20T21:09:32.816504] [INFO ] [MAIN ] SMRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:32.857223] [DEBUG] [LOCAL ] SMRT: Loaded 44571 existing records from primary storage. [2025-10-20T21:09:32.857506] [INFO ] [MAIN ] SMRT: Loaded 44571 existing records. [2025-10-20T21:09:33.035544] [WARN ] [MAIN ] SMRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:33.035759] [INFO ] [MAIN ] SMRT: Wiping existing primary data before full refresh. [2025-10-20T21:09:33.035809] [INFO ] [LOCAL ] SMRT: Starting deletion of primary data. [2025-10-20T21:09:33.036044] [DEBUG] [LOCAL ] SMRT: Deleted primary data file. [2025-10-20T21:09:33.036120] [INFO ] [MAIN ] SMRT: Starting full primary refresh. [2025-10-20T21:09:33.066943] [INFO ] [REMOTE ] SMP: Fetching primary data for 2025-10. [2025-10-20T21:09:33.430040] [INFO ] [REMOTE ] SMRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:33.458129] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-01 (2.81 seconds). [2025-10-20T21:09:33.677115] [INFO ] [REMOTE ] SMP: Incremental fetch complete for 2025-10 (0.61 seconds). [2025-10-20T21:09:33.679770] [INFO ] [REMOTE ] SMP: Full fetch complete (26346 records). [2025-10-20T21:09:33.679848] [DEBUG] [VALIDATOR ] SMP: Validating primary data. [2025-10-20T21:09:33.680501] [FATAL] [VALIDATOR ] SMP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:09:33.680593] [FATAL] [MAIN ] SMP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:09:33.680707] [WARN ] [MAIN ] SMP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:09:33.680800] [INFO ] [LOCAL ] SMP: Starting deletion of primary data. [2025-10-20T21:09:33.680895] [DEBUG] [LOCAL ] SMP: No primary data file found to delete. [2025-10-20T21:09:33.680968] [INFO ] [LOCAL ] SMP: Starting deletion of derivative data. [2025-10-20T21:09:33.681092] [DEBUG] [LOCAL ] SMP: Deleted derivative data file. [2025-10-20T21:09:33.681161] [INFO ] [MAIN ] Update Progress: 1939/2367 symbols completed. SMP took 29.4s. Estimated remaining: 55m 4s. Rate: 7.8 symbols/min [2025-10-20T21:09:33.849810] [INFO ] [MAIN ] SMWB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:33.851352] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-02. [2025-10-20T21:09:33.882833] [DEBUG] [LOCAL ] SMWB: Loaded 32540 existing records from primary storage. [2025-10-20T21:09:33.883356] [INFO ] [MAIN ] SMWB: Loaded 32540 existing records. [2025-10-20T21:09:34.074123] [WARN ] [MAIN ] SMWB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:34.085633] [INFO ] [MAIN ] SMWB: Wiping existing primary data before full refresh. [2025-10-20T21:09:34.092756] [INFO ] [LOCAL ] SMWB: Starting deletion of primary data. [2025-10-20T21:09:34.093036] [DEBUG] [LOCAL ] SMWB: Deleted primary data file. [2025-10-20T21:09:34.093485] [INFO ] [MAIN ] SMWB: Starting full primary refresh. [2025-10-20T21:09:34.094234] [INFO ] [REMOTE ] SMFG: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T21:09:34.197130] [INFO ] [REMOTE ] SMFG: Full fetch complete (69377 records). [2025-10-20T21:09:34.208391] [DEBUG] [VALIDATOR ] SMFG: Validating primary data. [2025-10-20T21:09:34.679444] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-01. [2025-10-20T21:09:34.734349] [INFO ] [REMOTE ] SMWB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:34.965009] [DEBUG] [VALIDATOR ] SMFG: All expected trading days are present. [2025-10-20T21:09:34.965081] [DEBUG] [VALIDATOR ] SMFG: Primary data validation passed. [2025-10-20T21:09:34.966044] [INFO ] [MAIN ] SMFG: Fetched primary data is valid. [2025-10-20T21:09:34.966106] [DEBUG] [LOCAL ] SMFG: Saving primary data. [2025-10-20T21:09:34.967492] [DEBUG] [LOCAL ] SMFG: Trimmed 69377 records to 48395 (keeping 200 days of data) [2025-10-20T21:09:35.278335] [DEBUG] [LOCAL ] SMFG: Primary data saved (48395 records). [2025-10-20T21:09:35.278476] [INFO ] [MAIN ] SMFG: Full history saved. [2025-10-20T21:09:35.278558] [INFO ] [MAIN ] Update Progress: 1940/2367 symbols completed. SMFG took 39.3s. Estimated remaining: 54m 55s. Rate: 7.8 symbols/min [2025-10-20T21:09:35.452919] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-01. [2025-10-20T21:09:35.454337] [INFO ] [MAIN ] SN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:35.502848] [DEBUG] [LOCAL ] SN: Loaded 50745 existing records from primary storage. [2025-10-20T21:09:35.503164] [INFO ] [MAIN ] SN: Loaded 50745 existing records. [2025-10-20T21:09:35.679932] [WARN ] [MAIN ] SN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:09:35.680133] [INFO ] [MAIN ] SN: Wiping existing primary data before full refresh. [2025-10-20T21:09:35.680179] [INFO ] [LOCAL ] SN: Starting deletion of primary data. [2025-10-20T21:09:35.680380] [DEBUG] [LOCAL ] SN: Deleted primary data file. [2025-10-20T21:09:35.680435] [INFO ] [MAIN ] SN: Starting full primary refresh. [2025-10-20T21:09:36.054665] [INFO ] [REMOTE ] SN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:36.104919] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-02 (2.25 seconds). [2025-10-20T21:09:36.241756] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T21:09:36.251324] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-02. [2025-10-20T21:09:36.551778] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T21:09:37.051370] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-01. [2025-10-20T21:09:37.799131] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T21:09:37.851398] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-03. [2025-10-20T21:09:38.508511] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T21:09:38.651339] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-02. [2025-10-20T21:09:39.451432] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-02. [2025-10-20T21:09:40.251445] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-03. [2025-10-20T21:09:40.372470] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:09:40.529879] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-03 (2.68 seconds). [2025-10-20T21:09:41.019947] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:09:41.051354] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-03. [2025-10-20T21:09:41.653719] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T21:09:41.851396] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-03. [2025-10-20T21:09:42.507796] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T21:09:42.651370] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-04. [2025-10-20T21:09:43.298438] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T21:09:43.451367] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-04. [2025-10-20T21:09:44.251418] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-04. [2025-10-20T21:09:44.315149] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T21:09:45.051439] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-04. [2025-10-20T21:09:45.240285] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T21:09:45.851434] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-05. [2025-10-20T21:09:46.486319] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:09:46.596799] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-04 (2.35 seconds). [2025-10-20T21:09:46.651341] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-05. [2025-10-20T21:09:47.451349] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-05. [2025-10-20T21:09:47.511100] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T21:09:48.251409] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-05. [2025-10-20T21:09:49.055379] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-06. [2025-10-20T21:09:49.074505] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T21:09:49.399869] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-05 (2.75 seconds). [2025-10-20T21:09:49.851411] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-06. [2025-10-20T21:09:49.957103] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T21:09:50.651367] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-06. [2025-10-20T21:09:50.784873] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T21:09:51.211212] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T21:09:51.451342] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-07. [2025-10-20T21:09:51.715934] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T21:09:52.251383] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-07. [2025-10-20T21:09:53.051421] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-06. [2025-10-20T21:09:53.135348] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T21:09:53.609839] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T21:09:53.851446] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-08. [2025-10-20T21:09:54.651373] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-08. [2025-10-20T21:09:55.279261] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T21:09:55.406848] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-06 (2.36 seconds). [2025-10-20T21:09:55.451365] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-09. [2025-10-20T21:09:56.251373] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-07. [2025-10-20T21:09:56.302772] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:09:56.736917] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-09 (1.29 seconds). [2025-10-20T21:09:57.051381] [INFO ] [REMOTE ] SMWB: Fetching primary data for 2025-10. [2025-10-20T21:09:57.742037] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T21:09:57.851337] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-07. [2025-10-20T21:09:58.236786] [INFO ] [REMOTE ] SMWB: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T21:09:58.240973] [INFO ] [REMOTE ] SMWB: Full fetch complete (48951 records). [2025-10-20T21:09:58.241101] [DEBUG] [VALIDATOR ] SMWB: Validating primary data. [2025-10-20T21:09:58.622095] [DEBUG] [VALIDATOR ] SMWB: All expected trading days are present. [2025-10-20T21:09:58.622166] [DEBUG] [VALIDATOR ] SMWB: Primary data validation passed. [2025-10-20T21:09:58.623017] [INFO ] [MAIN ] SMWB: Fetched primary data is valid. [2025-10-20T21:09:58.623077] [DEBUG] [LOCAL ] SMWB: Saving primary data. [2025-10-20T21:09:58.624042] [DEBUG] [LOCAL ] SMWB: Trimmed 48951 records to 31939 (keeping 200 days of data) [2025-10-20T21:09:58.667034] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-09. [2025-10-20T21:09:58.798005] [DEBUG] [LOCAL ] SMWB: Primary data saved (31939 records). [2025-10-20T21:09:58.798128] [INFO ] [MAIN ] SMWB: Full history saved. [2025-10-20T21:09:58.798557] [INFO ] [MAIN ] Update Progress: 1941/2367 symbols completed. SMWB took 25.1s. Estimated remaining: 54m 51s. Rate: 7.8 symbols/min [2025-10-20T21:09:58.967257] [INFO ] [MAIN ] SNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:09:58.967404] [DEBUG] [LOCAL ] SNA: No existing data file found in primary storage. [2025-10-20T21:09:58.967442] [WARN ] [MAIN ] SNA: No existing data found, performing full refresh instead. [2025-10-20T21:09:58.967550] [INFO ] [MAIN ] SNA: Wiping existing primary data before full refresh. [2025-10-20T21:09:58.967591] [INFO ] [LOCAL ] SNA: Starting deletion of primary data. [2025-10-20T21:09:58.967633] [DEBUG] [LOCAL ] SNA: No primary data file found to delete. [2025-10-20T21:09:58.967702] [INFO ] [MAIN ] SNA: Starting full primary refresh. [2025-10-20T21:09:59.314900] [INFO ] [REMOTE ] SNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:09:59.451333] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-01. [2025-10-20T21:10:00.251341] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-08. [2025-10-20T21:10:00.302807] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T21:10:00.779325] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-07 (2.93 seconds). [2025-10-20T21:10:00.949026] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T21:10:01.051445] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-08. [2025-10-20T21:10:01.851383] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-02. [2025-10-20T21:10:01.921069] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T21:10:02.651388] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-09. [2025-10-20T21:10:03.296060] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T21:10:03.451370] [INFO ] [REMOTE ] SMRT: Fetching primary data for 2025-10. [2025-10-20T21:10:03.943683] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-08 (2.89 seconds). [2025-10-20T21:10:04.118710] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T21:10:04.251452] [INFO ] [REMOTE ] SN: Fetching primary data for 2025-10. [2025-10-20T21:10:04.992760] [INFO ] [REMOTE ] SMRT: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T21:10:04.998907] [INFO ] [REMOTE ] SMRT: Full fetch complete (62275 records). [2025-10-20T21:10:04.999051] [DEBUG] [VALIDATOR ] SMRT: Validating primary data. [2025-10-20T21:10:05.061086] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-09. [2025-10-20T21:10:05.268859] [INFO ] [REMOTE ] SN: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T21:10:05.335713] [INFO ] [REMOTE ] SN: Full fetch complete (71374 records). [2025-10-20T21:10:05.344496] [DEBUG] [VALIDATOR ] SN: Validating primary data. [2025-10-20T21:10:05.390828] [FATAL] [VALIDATOR ] SN: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:10:05.416784] [FATAL] [MAIN ] SN: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:10:05.434695] [WARN ] [MAIN ] SN: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:10:05.438292] [DEBUG] [VALIDATOR ] SMRT: All expected trading days are present. [2025-10-20T21:10:05.438419] [DEBUG] [VALIDATOR ] SMRT: Primary data validation passed. [2025-10-20T21:10:05.439754] [INFO ] [MAIN ] SMRT: Fetched primary data is valid. [2025-10-20T21:10:05.438478] [INFO ] [LOCAL ] SN: Starting deletion of primary data. [2025-10-20T21:10:05.440014] [DEBUG] [LOCAL ] SMRT: Saving primary data. [2025-10-20T21:10:05.440336] [DEBUG] [LOCAL ] SN: No primary data file found to delete. [2025-10-20T21:10:05.440449] [INFO ] [LOCAL ] SN: Starting deletion of derivative data. [2025-10-20T21:10:05.441138] [DEBUG] [LOCAL ] SN: Deleted derivative data file. [2025-10-20T21:10:05.441357] [INFO ] [MAIN ] Update Progress: 1942/2367 symbols completed. SN took 30.2s. Estimated remaining: 54m 43s. Rate: 7.8 symbols/min [2025-10-20T21:10:05.476582] [DEBUG] [LOCAL ] SMRT: Trimmed 62275 records to 43776 (keeping 200 days of data) [2025-10-20T21:10:05.731114] [INFO ] [MAIN ] SNAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:05.851692] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-03. [2025-10-20T21:10:05.906310] [DEBUG] [LOCAL ] SMRT: Primary data saved (43776 records). [2025-10-20T21:10:05.906524] [INFO ] [MAIN ] SMRT: Full history saved. [2025-10-20T21:10:05.906774] [INFO ] [MAIN ] Update Progress: 1943/2367 symbols completed. SMRT took 33.3s. Estimated remaining: 54m 34s. Rate: 7.8 symbols/min [2025-10-20T21:10:06.001200] [DEBUG] [LOCAL ] SNAP: Loaded 54576 existing records from primary storage. [2025-10-20T21:10:06.026975] [INFO ] [MAIN ] SNAP: Loaded 54576 existing records. [2025-10-20T21:10:06.214441] [INFO ] [MAIN ] SNDA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:06.272731] [DEBUG] [LOCAL ] SNDA: Loaded 4539 existing records from primary storage. [2025-10-20T21:10:06.294084] [INFO ] [MAIN ] SNDA: Loaded 4539 existing records. [2025-10-20T21:10:06.293899] [WARN ] [MAIN ] SNAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:06.362533] [INFO ] [MAIN ] SNAP: Wiping existing primary data before full refresh. [2025-10-20T21:10:06.373507] [INFO ] [LOCAL ] SNAP: Starting deletion of primary data. [2025-10-20T21:10:06.394701] [DEBUG] [LOCAL ] SNAP: Deleted primary data file. [2025-10-20T21:10:06.394763] [INFO ] [MAIN ] SNAP: Starting full primary refresh. [2025-10-20T21:10:06.594584] [WARN ] [MAIN ] SNDA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:06.600690] [INFO ] [MAIN ] SNDA: Wiping existing primary data before full refresh. [2025-10-20T21:10:06.616205] [INFO ] [LOCAL ] SNDA: Starting deletion of primary data. [2025-10-20T21:10:06.638867] [DEBUG] [LOCAL ] SNDA: Deleted primary data file. [2025-10-20T21:10:06.652908] [INFO ] [MAIN ] SNDA: Starting full primary refresh. [2025-10-20T21:10:07.050138] [INFO ] [REMOTE ] SNAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:07.059957] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-01. [2025-10-20T21:10:07.243225] [INFO ] [REMOTE ] SNDA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:07.411506] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T21:10:07.451331] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-04. [2025-10-20T21:10:07.986254] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-09 (2.93 seconds). [2025-10-20T21:10:08.251438] [INFO ] [REMOTE ] SMR: Fetching primary data for 2025-10. [2025-10-20T21:10:08.976856] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T21:10:09.011387] [INFO ] [REMOTE ] SMR: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T21:10:09.018661] [INFO ] [REMOTE ] SMR: Full fetch complete (77587 records). [2025-10-20T21:10:09.018793] [DEBUG] [VALIDATOR ] SMR: Validating primary data. [2025-10-20T21:10:09.019610] [FATAL] [VALIDATOR ] SMR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:10:09.019716] [FATAL] [MAIN ] SMR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:10:09.019778] [WARN ] [MAIN ] SMR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:10:09.019848] [INFO ] [LOCAL ] SMR: Starting deletion of primary data. [2025-10-20T21:10:09.019911] [DEBUG] [LOCAL ] SMR: No primary data file found to delete. [2025-10-20T21:10:09.019946] [INFO ] [LOCAL ] SMR: Starting deletion of derivative data. [2025-10-20T21:10:09.020089] [DEBUG] [LOCAL ] SMR: Deleted derivative data file. [2025-10-20T21:10:09.020178] [INFO ] [MAIN ] Update Progress: 1944/2367 symbols completed. SMR took 40.3s. Estimated remaining: 54m 25s. Rate: 7.8 symbols/min [2025-10-20T21:10:09.067695] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-05. [2025-10-20T21:10:09.197489] [INFO ] [MAIN ] SNDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:09.235633] [DEBUG] [LOCAL ] SNDR: Loaded 42209 existing records from primary storage. [2025-10-20T21:10:09.236092] [INFO ] [MAIN ] SNDR: Loaded 42209 existing records. [2025-10-20T21:10:09.423453] [WARN ] [MAIN ] SNDR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:09.423799] [INFO ] [MAIN ] SNDR: Wiping existing primary data before full refresh. [2025-10-20T21:10:09.423861] [INFO ] [LOCAL ] SNDR: Starting deletion of primary data. [2025-10-20T21:10:09.424050] [DEBUG] [LOCAL ] SNDR: Deleted primary data file. [2025-10-20T21:10:09.424099] [INFO ] [MAIN ] SNDR: Starting full primary refresh. [2025-10-20T21:10:09.769480] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-01 (2.71 seconds). [2025-10-20T21:10:09.852495] [INFO ] [REMOTE ] SNDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:09.852575] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-01. [2025-10-20T21:10:10.427340] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T21:10:10.651360] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-02. [2025-10-20T21:10:10.766522] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T21:10:11.451406] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-01. [2025-10-20T21:10:12.251427] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-02. [2025-10-20T21:10:13.044078] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-02 (2.39 seconds). [2025-10-20T21:10:13.053722] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-06. [2025-10-20T21:10:13.062144] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:10:13.091706] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T21:10:13.851362] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-03. [2025-10-20T21:10:14.467625] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T21:10:14.651382] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-02. [2025-10-20T21:10:15.451410] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-07. [2025-10-20T21:10:16.190884] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-03 (2.34 seconds). [2025-10-20T21:10:16.251607] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-04. [2025-10-20T21:10:16.256052] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T21:10:16.826283] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T21:10:17.051387] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-03. [2025-10-20T21:10:17.851398] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-03. [2025-10-20T21:10:18.534199] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-03 (0.68 seconds). [2025-10-20T21:10:18.604898] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-03 (1.55 seconds). [2025-10-20T21:10:18.651349] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-04. [2025-10-20T21:10:18.858780] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-04 (2.61 seconds). [2025-10-20T21:10:19.451598] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-04. [2025-10-20T21:10:20.153526] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:10:20.171124] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T21:10:20.251421] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-05. [2025-10-20T21:10:21.051402] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-08. [2025-10-20T21:10:21.414511] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-08 (0.36 seconds). [2025-10-20T21:10:21.851371] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-09. [2025-10-20T21:10:22.496171] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T21:10:22.651395] [INFO ] [REMOTE ] SNA: Fetching primary data for 2025-10. [2025-10-20T21:10:22.701913] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-05 (2.45 seconds). [2025-10-20T21:10:23.123955] [INFO ] [REMOTE ] SNA: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T21:10:23.128156] [INFO ] [REMOTE ] SNA: Full fetch complete (39006 records). [2025-10-20T21:10:23.128268] [DEBUG] [VALIDATOR ] SNA: Validating primary data. [2025-10-20T21:10:23.128833] [FATAL] [VALIDATOR ] SNA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:10:23.128937] [FATAL] [MAIN ] SNA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:10:23.128980] [WARN ] [MAIN ] SNA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:10:23.129017] [INFO ] [LOCAL ] SNA: Starting deletion of primary data. [2025-10-20T21:10:23.129072] [DEBUG] [LOCAL ] SNA: No primary data file found to delete. [2025-10-20T21:10:23.129107] [INFO ] [LOCAL ] SNA: Starting deletion of derivative data. [2025-10-20T21:10:23.129147] [DEBUG] [LOCAL ] SNA: No derivative data file found to delete. [2025-10-20T21:10:23.129205] [INFO ] [MAIN ] Update Progress: 1945/2367 symbols completed. SNA took 24.3s. Estimated remaining: 54m 19s. Rate: 7.8 symbols/min [2025-10-20T21:10:23.302093] [INFO ] [MAIN ] SNN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:23.343100] [DEBUG] [LOCAL ] SNN: Loaded 44929 existing records from primary storage. [2025-10-20T21:10:23.343402] [INFO ] [MAIN ] SNN: Loaded 44929 existing records. [2025-10-20T21:10:23.456554] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-05. [2025-10-20T21:10:23.524800] [WARN ] [MAIN ] SNN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:23.524962] [INFO ] [MAIN ] SNN: Wiping existing primary data before full refresh. [2025-10-20T21:10:23.525006] [INFO ] [LOCAL ] SNN: Starting deletion of primary data. [2025-10-20T21:10:23.525195] [DEBUG] [LOCAL ] SNN: Deleted primary data file. [2025-10-20T21:10:23.525251] [INFO ] [MAIN ] SNN: Starting full primary refresh. [2025-10-20T21:10:23.873802] [INFO ] [REMOTE ] SNN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:24.126544] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T21:10:24.251356] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-06. [2025-10-20T21:10:25.051399] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-01. [2025-10-20T21:10:25.851427] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-05. [2025-10-20T21:10:26.525353] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-06 (2.27 seconds). [2025-10-20T21:10:26.651367] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-07. [2025-10-20T21:10:26.750111] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T21:10:27.053337] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:10:27.451371] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-02. [2025-10-20T21:10:28.251378] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-06. [2025-10-20T21:10:28.743555] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T21:10:29.051376] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-03. [2025-10-20T21:10:29.273465] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-07 (2.62 seconds). [2025-10-20T21:10:29.857992] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-08. [2025-10-20T21:10:29.864114] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T21:10:30.448472] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T21:10:30.651362] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-07. [2025-10-20T21:10:30.714835] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T21:10:31.451400] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-09. [2025-10-20T21:10:32.251357] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-06. [2025-10-20T21:10:32.319345] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T21:10:32.361940] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T21:10:32.883111] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T21:10:33.051405] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-08. [2025-10-20T21:10:33.851390] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-04. [2025-10-20T21:10:34.413467] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T21:10:34.651384] [INFO ] [REMOTE ] SNAP: Fetching primary data for 2025-10. [2025-10-20T21:10:35.404730] [INFO ] [REMOTE ] SNAP: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:10:35.412134] [INFO ] [REMOTE ] SNAP: Full fetch complete (77585 records). [2025-10-20T21:10:35.412271] [DEBUG] [VALIDATOR ] SNAP: Validating primary data. [2025-10-20T21:10:35.413117] [FATAL] [VALIDATOR ] SNAP: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:10:35.413237] [FATAL] [MAIN ] SNAP: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:10:35.413307] [WARN ] [MAIN ] SNAP: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:10:35.413375] [INFO ] [LOCAL ] SNAP: Starting deletion of primary data. [2025-10-20T21:10:35.413439] [DEBUG] [LOCAL ] SNAP: No primary data file found to delete. [2025-10-20T21:10:35.413474] [INFO ] [LOCAL ] SNAP: Starting deletion of derivative data. [2025-10-20T21:10:35.413598] [DEBUG] [LOCAL ] SNAP: Deleted derivative data file. [2025-10-20T21:10:35.413718] [INFO ] [MAIN ] Update Progress: 1946/2367 symbols completed. SNAP took 30.0s. Estimated remaining: 54m 12s. Rate: 7.8 symbols/min [2025-10-20T21:10:35.456607] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-07. [2025-10-20T21:10:35.595744] [INFO ] [MAIN ] SNOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:35.625308] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T21:10:35.681483] [DEBUG] [LOCAL ] SNOW: Loaded 54371 existing records from primary storage. [2025-10-20T21:10:35.681870] [INFO ] [MAIN ] SNOW: Loaded 54371 existing records. [2025-10-20T21:10:35.860597] [WARN ] [MAIN ] SNOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:35.860830] [INFO ] [MAIN ] SNOW: Wiping existing primary data before full refresh. [2025-10-20T21:10:35.860884] [INFO ] [LOCAL ] SNOW: Starting deletion of primary data. [2025-10-20T21:10:35.861096] [DEBUG] [LOCAL ] SNOW: Deleted primary data file. [2025-10-20T21:10:35.861150] [INFO ] [MAIN ] SNOW: Starting full primary refresh. [2025-10-20T21:10:36.152110] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-07 (0.70 seconds). [2025-10-20T21:10:36.218705] [INFO ] [REMOTE ] SNOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:36.251341] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-01. [2025-10-20T21:10:37.051393] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-09. [2025-10-20T21:10:37.851373] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-05. [2025-10-20T21:10:38.488531] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-01 (2.24 seconds). [2025-10-20T21:10:38.650732] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T21:10:38.659075] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-02. [2025-10-20T21:10:39.451368] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-08. [2025-10-20T21:10:39.501339] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T21:10:40.251402] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-06. [2025-10-20T21:10:40.343112] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T21:10:41.051351] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-09. [2025-10-20T21:10:41.143312] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-02 (2.48 seconds). [2025-10-20T21:10:41.543188] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-09 (0.49 seconds). [2025-10-20T21:10:41.856504] [INFO ] [REMOTE ] SNDR: Fetching primary data for 2025-10. [2025-10-20T21:10:41.872604] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:10:42.651352] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-03. [2025-10-20T21:10:42.701376] [INFO ] [REMOTE ] SNDR: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T21:10:42.708129] [INFO ] [REMOTE ] SNDR: Full fetch complete (59895 records). [2025-10-20T21:10:42.708274] [DEBUG] [VALIDATOR ] SNDR: Validating primary data. [2025-10-20T21:10:42.708993] [FATAL] [VALIDATOR ] SNDR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:10:42.709100] [FATAL] [MAIN ] SNDR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:10:42.709166] [WARN ] [MAIN ] SNDR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:10:42.709238] [INFO ] [LOCAL ] SNDR: Starting deletion of primary data. [2025-10-20T21:10:42.709302] [DEBUG] [LOCAL ] SNDR: No primary data file found to delete. [2025-10-20T21:10:42.709339] [INFO ] [LOCAL ] SNDR: Starting deletion of derivative data. [2025-10-20T21:10:42.709481] [DEBUG] [LOCAL ] SNDR: Deleted derivative data file. [2025-10-20T21:10:42.709570] [INFO ] [MAIN ] Update Progress: 1947/2367 symbols completed. SNDR took 33.7s. Estimated remaining: 54m 4s. Rate: 7.8 symbols/min [2025-10-20T21:10:42.887341] [INFO ] [MAIN ] SNV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:42.931065] [DEBUG] [LOCAL ] SNV: Loaded 49816 existing records from primary storage. [2025-10-20T21:10:42.931371] [INFO ] [MAIN ] SNV: Loaded 49816 existing records. [2025-10-20T21:10:43.115728] [WARN ] [MAIN ] SNV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:43.115937] [INFO ] [MAIN ] SNV: Wiping existing primary data before full refresh. [2025-10-20T21:10:43.115984] [INFO ] [LOCAL ] SNV: Starting deletion of primary data. [2025-10-20T21:10:43.116161] [DEBUG] [LOCAL ] SNV: Deleted primary data file. [2025-10-20T21:10:43.116200] [INFO ] [MAIN ] SNV: Starting full primary refresh. [2025-10-20T21:10:43.452779] [INFO ] [REMOTE ] SNDA: Fetching primary data for 2025-10. [2025-10-20T21:10:43.474499] [INFO ] [REMOTE ] SNV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:43.956522] [INFO ] [REMOTE ] SNDA: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T21:10:43.958094] [INFO ] [REMOTE ] SNDA: Full fetch complete (7537 records). [2025-10-20T21:10:43.958183] [DEBUG] [VALIDATOR ] SNDA: Validating primary data. [2025-10-20T21:10:44.253450] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-07. [2025-10-20T21:10:44.338360] [DEBUG] [VALIDATOR ] SNDA: All expected trading days are present. [2025-10-20T21:10:44.338448] [DEBUG] [VALIDATOR ] SNDA: Primary data validation passed. [2025-10-20T21:10:44.338725] [INFO ] [MAIN ] SNDA: Fetched primary data is valid. [2025-10-20T21:10:44.338785] [DEBUG] [LOCAL ] SNDA: Saving primary data. [2025-10-20T21:10:44.339177] [DEBUG] [LOCAL ] SNDA: Trimmed 7537 records to 4488 (keeping 200 days of data) [2025-10-20T21:10:44.364851] [DEBUG] [LOCAL ] SNDA: Primary data saved (4488 records). [2025-10-20T21:10:44.364929] [INFO ] [MAIN ] SNDA: Full history saved. [2025-10-20T21:10:44.364999] [INFO ] [MAIN ] Update Progress: 1948/2367 symbols completed. SNDA took 38.4s. Estimated remaining: 53m 55s. Rate: 7.8 symbols/min [2025-10-20T21:10:44.537177] [INFO ] [MAIN ] SNX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:44.576145] [DEBUG] [LOCAL ] SNX: Loaded 39987 existing records from primary storage. [2025-10-20T21:10:44.576603] [INFO ] [MAIN ] SNX: Loaded 39987 existing records. [2025-10-20T21:10:44.760835] [WARN ] [MAIN ] SNX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:44.761152] [INFO ] [MAIN ] SNX: Wiping existing primary data before full refresh. [2025-10-20T21:10:44.761367] [INFO ] [LOCAL ] SNX: Starting deletion of primary data. [2025-10-20T21:10:44.761628] [DEBUG] [LOCAL ] SNX: Deleted primary data file. [2025-10-20T21:10:44.761731] [INFO ] [MAIN ] SNX: Starting full primary refresh. [2025-10-20T21:10:45.051336] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-01. [2025-10-20T21:10:45.130879] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-03 (2.48 seconds). [2025-10-20T21:10:45.173323] [INFO ] [REMOTE ] SNX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:45.851343] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-01. [2025-10-20T21:10:45.986711] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:10:46.651376] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-08. [2025-10-20T21:10:46.993498] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T21:10:47.460092] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-04. [2025-10-20T21:10:47.462357] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:10:48.251364] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-02. [2025-10-20T21:10:48.338007] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T21:10:49.051424] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-02. [2025-10-20T21:10:49.742547] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T21:10:49.847920] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-04 (2.39 seconds). [2025-10-20T21:10:49.858015] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-09. [2025-10-20T21:10:50.386537] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T21:10:50.651370] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-05. [2025-10-20T21:10:51.451377] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-03. [2025-10-20T21:10:51.523550] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:10:52.251419] [INFO ] [REMOTE ] SNN: Fetching primary data for 2025-10. [2025-10-20T21:10:53.051376] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-03. [2025-10-20T21:10:53.208609] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-05 (2.56 seconds). [2025-10-20T21:10:53.265688] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T21:10:53.424675] [INFO ] [REMOTE ] SNN: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T21:10:53.430798] [INFO ] [REMOTE ] SNN: Full fetch complete (63124 records). [2025-10-20T21:10:53.430902] [DEBUG] [VALIDATOR ] SNN: Validating primary data. [2025-10-20T21:10:53.822108] [DEBUG] [VALIDATOR ] SNN: All expected trading days are present. [2025-10-20T21:10:53.822191] [DEBUG] [VALIDATOR ] SNN: Primary data validation passed. [2025-10-20T21:10:53.823157] [INFO ] [MAIN ] SNN: Fetched primary data is valid. [2025-10-20T21:10:53.823220] [DEBUG] [LOCAL ] SNN: Saving primary data. [2025-10-20T21:10:53.824499] [DEBUG] [LOCAL ] SNN: Trimmed 63124 records to 44407 (keeping 200 days of data) [2025-10-20T21:10:53.877181] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-06. [2025-10-20T21:10:54.094717] [DEBUG] [LOCAL ] SNN: Primary data saved (44407 records). [2025-10-20T21:10:54.094871] [INFO ] [MAIN ] SNN: Full history saved. [2025-10-20T21:10:54.095280] [INFO ] [MAIN ] Update Progress: 1949/2367 symbols completed. SNN took 31.0s. Estimated remaining: 53m 48s. Rate: 7.8 symbols/min [2025-10-20T21:10:54.262724] [INFO ] [MAIN ] SO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:10:54.314453] [DEBUG] [LOCAL ] SO: Loaded 54394 existing records from primary storage. [2025-10-20T21:10:54.314782] [INFO ] [MAIN ] SO: Loaded 54394 existing records. [2025-10-20T21:10:54.493532] [WARN ] [MAIN ] SO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:10:54.493747] [INFO ] [MAIN ] SO: Wiping existing primary data before full refresh. [2025-10-20T21:10:54.493819] [INFO ] [LOCAL ] SO: Starting deletion of primary data. [2025-10-20T21:10:54.494017] [DEBUG] [LOCAL ] SO: Deleted primary data file. [2025-10-20T21:10:54.494073] [INFO ] [MAIN ] SO: Starting full primary refresh. [2025-10-20T21:10:54.653474] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-04. [2025-10-20T21:10:54.864519] [INFO ] [REMOTE ] SO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:10:54.891285] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T21:10:55.451366] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-04. [2025-10-20T21:10:56.254801] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-01. [2025-10-20T21:10:56.268173] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-06 (2.39 seconds). [2025-10-20T21:10:56.668841] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T21:10:57.051366] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-07. [2025-10-20T21:10:57.231788] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:10:57.851382] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-05. [2025-10-20T21:10:58.229590] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-01 (1.97 seconds). [2025-10-20T21:10:58.651352] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-05. [2025-10-20T21:10:59.093217] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T21:10:59.141852] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T21:10:59.451374] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-02. [2025-10-20T21:11:00.218100] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:11:00.251352] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-06. [2025-10-20T21:11:01.051364] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-06. [2025-10-20T21:11:01.461245] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T21:11:01.851365] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-03. [2025-10-20T21:11:01.927446] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:11:02.656474] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-08. [2025-10-20T21:11:02.674973] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:11:03.451393] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-07. [2025-10-20T21:11:03.642317] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-08 (0.99 seconds). [2025-10-20T21:11:03.807115] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T21:11:04.251381] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-09. [2025-10-20T21:11:05.051350] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-04. [2025-10-20T21:11:05.114803] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T21:11:05.158017] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T21:11:05.851428] [INFO ] [REMOTE ] SNOW: Fetching primary data for 2025-10. [2025-10-20T21:11:06.656207] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-08. [2025-10-20T21:11:06.660598] [INFO ] [REMOTE ] SNOW: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T21:11:06.668936] [INFO ] [REMOTE ] SNOW: Full fetch complete (77342 records). [2025-10-20T21:11:06.669081] [DEBUG] [VALIDATOR ] SNOW: Validating primary data. [2025-10-20T21:11:06.669943] [FATAL] [VALIDATOR ] SNOW: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:11:06.670026] [FATAL] [MAIN ] SNOW: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:11:06.670090] [WARN ] [MAIN ] SNOW: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:11:06.670131] [INFO ] [LOCAL ] SNOW: Starting deletion of primary data. [2025-10-20T21:11:06.670186] [DEBUG] [LOCAL ] SNOW: No primary data file found to delete. [2025-10-20T21:11:06.670220] [INFO ] [LOCAL ] SNOW: Starting deletion of derivative data. [2025-10-20T21:11:06.670379] [DEBUG] [LOCAL ] SNOW: Deleted derivative data file. [2025-10-20T21:11:06.670475] [INFO ] [MAIN ] Update Progress: 1950/2367 symbols completed. SNOW took 31.3s. Estimated remaining: 53m 41s. Rate: 7.8 symbols/min [2025-10-20T21:11:06.838479] [INFO ] [MAIN ] SOBO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:06.883608] [DEBUG] [LOCAL ] SOBO: Loaded 40006 existing records from primary storage. [2025-10-20T21:11:06.883918] [INFO ] [MAIN ] SOBO: Loaded 40006 existing records. [2025-10-20T21:11:07.087032] [WARN ] [MAIN ] SOBO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:07.087332] [INFO ] [MAIN ] SOBO: Wiping existing primary data before full refresh. [2025-10-20T21:11:07.087471] [INFO ] [LOCAL ] SOBO: Starting deletion of primary data. [2025-10-20T21:11:07.087831] [DEBUG] [LOCAL ] SOBO: Deleted primary data file. [2025-10-20T21:11:07.087903] [INFO ] [MAIN ] SOBO: Starting full primary refresh. [2025-10-20T21:11:07.273389] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-10-20T21:11:07.460871] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-07. [2025-10-20T21:11:07.491248] [INFO ] [REMOTE ] SOBO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:08.251368] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-05. [2025-10-20T21:11:08.421215] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T21:11:09.051380] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-09. [2025-10-20T21:11:09.125991] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T21:11:09.851444] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-01. [2025-10-20T21:11:10.199470] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T21:11:10.651386] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-06. [2025-10-20T21:11:10.895512] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T21:11:11.259934] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T21:11:11.451366] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-02. [2025-10-20T21:11:12.251419] [INFO ] [REMOTE ] SNV: Fetching primary data for 2025-10. [2025-10-20T21:11:12.411611] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T21:11:13.051424] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-08. [2025-10-20T21:11:13.214731] [INFO ] [REMOTE ] SNV: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T21:11:13.222425] [INFO ] [REMOTE ] SNV: Full fetch complete (71398 records). [2025-10-20T21:11:13.222591] [DEBUG] [VALIDATOR ] SNV: Validating primary data. [2025-10-20T21:11:13.223295] [FATAL] [VALIDATOR ] SNV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:11:13.223388] [FATAL] [MAIN ] SNV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:11:13.223432] [WARN ] [MAIN ] SNV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:11:13.223475] [INFO ] [LOCAL ] SNV: Starting deletion of primary data. [2025-10-20T21:11:13.223563] [DEBUG] [LOCAL ] SNV: No primary data file found to delete. [2025-10-20T21:11:13.223664] [INFO ] [LOCAL ] SNV: Starting deletion of derivative data. [2025-10-20T21:11:13.223838] [DEBUG] [LOCAL ] SNV: Deleted derivative data file. [2025-10-20T21:11:13.223938] [INFO ] [MAIN ] Update Progress: 1951/2367 symbols completed. SNV took 30.5s. Estimated remaining: 53m 33s. Rate: 7.8 symbols/min [2025-10-20T21:11:13.431219] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-02 (1.98 seconds). [2025-10-20T21:11:13.455541] [INFO ] [MAIN ] SOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:13.518250] [DEBUG] [LOCAL ] SOC: Loaded 51745 existing records from primary storage. [2025-10-20T21:11:13.518617] [INFO ] [MAIN ] SOC: Loaded 51745 existing records. [2025-10-20T21:11:13.705158] [WARN ] [MAIN ] SOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:13.705402] [INFO ] [MAIN ] SOC: Wiping existing primary data before full refresh. [2025-10-20T21:11:13.705449] [INFO ] [LOCAL ] SOC: Starting deletion of primary data. [2025-10-20T21:11:13.705687] [DEBUG] [LOCAL ] SOC: Deleted primary data file. [2025-10-20T21:11:13.705744] [INFO ] [MAIN ] SOC: Starting full primary refresh. [2025-10-20T21:11:13.852582] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-03. [2025-10-20T21:11:14.066223] [INFO ] [REMOTE ] SOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:14.461020] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T21:11:14.651390] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-09. [2025-10-20T21:11:15.451356] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-07. [2025-10-20T21:11:15.609192] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T21:11:15.875293] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T21:11:16.251380] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-01. [2025-10-20T21:11:17.051395] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-04. [2025-10-20T21:11:17.345693] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:11:17.444231] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T21:11:17.851443] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-08. [2025-10-20T21:11:18.656481] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-02. [2025-10-20T21:11:18.673298] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T21:11:18.762455] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-08 (0.91 seconds). [2025-10-20T21:11:19.451400] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-09. [2025-10-20T21:11:20.251363] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-05. [2025-10-20T21:11:20.292775] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:11:20.347612] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T21:11:21.051426] [INFO ] [REMOTE ] SNX: Fetching primary data for 2025-10. [2025-10-20T21:11:21.629540] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T21:11:21.730445] [INFO ] [REMOTE ] SNX: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T21:11:21.735856] [INFO ] [REMOTE ] SNX: Full fetch complete (57123 records). [2025-10-20T21:11:21.735952] [DEBUG] [VALIDATOR ] SNX: Validating primary data. [2025-10-20T21:11:21.736480] [FATAL] [VALIDATOR ] SNX: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:11:21.736562] [FATAL] [MAIN ] SNX: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:11:21.736604] [WARN ] [MAIN ] SNX: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:11:21.736689] [INFO ] [LOCAL ] SNX: Starting deletion of primary data. [2025-10-20T21:11:21.736762] [DEBUG] [LOCAL ] SNX: No primary data file found to delete. [2025-10-20T21:11:21.736797] [INFO ] [LOCAL ] SNX: Starting deletion of derivative data. [2025-10-20T21:11:21.736942] [DEBUG] [LOCAL ] SNX: Deleted derivative data file. [2025-10-20T21:11:21.737025] [INFO ] [MAIN ] Update Progress: 1952/2367 symbols completed. SNX took 37.4s. Estimated remaining: 53m 25s. Rate: 7.8 symbols/min [2025-10-20T21:11:21.856576] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-03. [2025-10-20T21:11:21.913809] [INFO ] [MAIN ] SOJC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:21.920320] [DEBUG] [LOCAL ] SOJC: Loaded 4852 existing records from primary storage. [2025-10-20T21:11:21.920475] [INFO ] [MAIN ] SOJC: Loaded 4852 existing records. [2025-10-20T21:11:22.101668] [WARN ] [MAIN ] SOJC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:22.101876] [INFO ] [MAIN ] SOJC: Wiping existing primary data before full refresh. [2025-10-20T21:11:22.101922] [INFO ] [LOCAL ] SOJC: Starting deletion of primary data. [2025-10-20T21:11:22.102088] [DEBUG] [LOCAL ] SOJC: Deleted primary data file. [2025-10-20T21:11:22.102144] [INFO ] [MAIN ] SOJC: Starting full primary refresh. [2025-10-20T21:11:22.464674] [INFO ] [REMOTE ] SOJC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:22.651381] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-01. [2025-10-20T21:11:23.355316] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T21:11:23.451364] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-02. [2025-10-20T21:11:23.724681] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T21:11:24.087095] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T21:11:24.251423] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-06. [2025-10-20T21:11:25.051351] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-04. [2025-10-20T21:11:25.598182] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T21:11:25.851435] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-07. [2025-10-20T21:11:26.493073] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T21:11:26.651326] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-05. [2025-10-20T21:11:26.896057] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T21:11:27.451414] [INFO ] [REMOTE ] SO: Fetching primary data for 2025-10. [2025-10-20T21:11:28.182921] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:11:28.187986] [INFO ] [REMOTE ] SO: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:11:28.202089] [INFO ] [REMOTE ] SO: Full fetch complete (77383 records). [2025-10-20T21:11:28.202212] [DEBUG] [VALIDATOR ] SO: Validating primary data. [2025-10-20T21:11:28.202920] [FATAL] [VALIDATOR ] SO: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:11:28.203004] [FATAL] [MAIN ] SO: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:11:28.203068] [WARN ] [MAIN ] SO: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:11:28.203136] [INFO ] [LOCAL ] SO: Starting deletion of primary data. [2025-10-20T21:11:28.203200] [DEBUG] [LOCAL ] SO: No primary data file found to delete. [2025-10-20T21:11:28.203236] [INFO ] [LOCAL ] SO: Starting deletion of derivative data. [2025-10-20T21:11:28.203370] [DEBUG] [LOCAL ] SO: Deleted derivative data file. [2025-10-20T21:11:28.203748] [INFO ] [MAIN ] Update Progress: 1953/2367 symbols completed. SO took 34.1s. Estimated remaining: 53m 17s. Rate: 7.8 symbols/min [2025-10-20T21:11:28.256473] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-03. [2025-10-20T21:11:28.373511] [INFO ] [MAIN ] SOJD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:28.387475] [DEBUG] [LOCAL ] SOJD: Loaded 13136 existing records from primary storage. [2025-10-20T21:11:28.387722] [INFO ] [MAIN ] SOJD: Loaded 13136 existing records. [2025-10-20T21:11:28.561127] [WARN ] [MAIN ] SOJD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:28.561324] [INFO ] [MAIN ] SOJD: Wiping existing primary data before full refresh. [2025-10-20T21:11:28.561368] [INFO ] [LOCAL ] SOJD: Starting deletion of primary data. [2025-10-20T21:11:28.561534] [DEBUG] [LOCAL ] SOJD: Deleted primary data file. [2025-10-20T21:11:28.561571] [INFO ] [MAIN ] SOJD: Starting full primary refresh. [2025-10-20T21:11:28.910377] [INFO ] [REMOTE ] SOJD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:29.051340] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-01. [2025-10-20T21:11:29.069280] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T21:11:29.851345] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-06. [2025-10-20T21:11:29.934871] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:11:30.651414] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-04. [2025-10-20T21:11:31.448816] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T21:11:31.451320] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-05. [2025-10-20T21:11:31.946170] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:11:31.978027] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T21:11:32.251375] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-02. [2025-10-20T21:11:32.896797] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-02 (0.65 seconds). [2025-10-20T21:11:33.051376] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-08. [2025-10-20T21:11:33.851436] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-03. [2025-10-20T21:11:34.632961] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T21:11:34.651339] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-09. [2025-10-20T21:11:34.786763] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T21:11:35.451424] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-07. [2025-10-20T21:11:36.086637] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T21:11:36.251372] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-06. [2025-10-20T21:11:36.897432] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T21:11:37.051380] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-04. [2025-10-20T21:11:37.052941] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T21:11:37.851391] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-08. [2025-10-20T21:11:38.100296] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T21:11:38.651422] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-05. [2025-10-20T21:11:39.295903] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:11:39.451350] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-09. [2025-10-20T21:11:39.655864] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T21:11:40.251362] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-07. [2025-10-20T21:11:41.007125] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T21:11:41.051438] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-08. [2025-10-20T21:11:41.286961] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T21:11:41.711352] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T21:11:41.851391] [INFO ] [REMOTE ] SOC: Fetching primary data for 2025-10. [2025-10-20T21:11:42.651381] [INFO ] [REMOTE ] SOBO: Fetching primary data for 2025-10. [2025-10-20T21:11:43.451357] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-06. [2025-10-20T21:11:43.869122] [INFO ] [REMOTE ] SOC: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T21:11:43.876841] [INFO ] [REMOTE ] SOC: Full fetch complete (72120 records). [2025-10-20T21:11:43.876971] [DEBUG] [VALIDATOR ] SOC: Validating primary data. [2025-10-20T21:11:44.265309] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-09. [2025-10-20T21:11:44.266773] [DEBUG] [VALIDATOR ] SOC: All expected trading days are present. [2025-10-20T21:11:44.266844] [DEBUG] [VALIDATOR ] SOC: Primary data validation passed. [2025-10-20T21:11:44.273777] [INFO ] [MAIN ] SOC: Fetched primary data is valid. [2025-10-20T21:11:44.281874] [DEBUG] [LOCAL ] SOC: Saving primary data. [2025-10-20T21:11:44.286135] [INFO ] [REMOTE ] SOBO: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T21:11:44.302884] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-06 (0.85 seconds). [2025-10-20T21:11:44.302138] [DEBUG] [LOCAL ] SOC: Trimmed 72120 records to 51208 (keeping 200 days of data) [2025-10-20T21:11:44.302037] [INFO ] [REMOTE ] SOBO: Full fetch complete (58526 records). [2025-10-20T21:11:44.306718] [DEBUG] [VALIDATOR ] SOBO: Validating primary data. [2025-10-20T21:11:44.923843] [DEBUG] [VALIDATOR ] SOBO: All expected trading days are present. [2025-10-20T21:11:44.924517] [DEBUG] [VALIDATOR ] SOBO: Primary data validation passed. [2025-10-20T21:11:44.928857] [INFO ] [MAIN ] SOBO: Fetched primary data is valid. [2025-10-20T21:11:44.929807] [DEBUG] [LOCAL ] SOBO: Saving primary data. [2025-10-20T21:11:44.933985] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-09 (0.67 seconds). [2025-10-20T21:11:44.935836] [DEBUG] [LOCAL ] SOBO: Trimmed 58526 records to 39419 (keeping 200 days of data) [2025-10-20T21:11:45.073539] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-07. [2025-10-20T21:11:45.122441] [DEBUG] [LOCAL ] SOC: Primary data saved (51208 records). [2025-10-20T21:11:45.125858] [INFO ] [MAIN ] SOC: Full history saved. [2025-10-20T21:11:45.125969] [INFO ] [MAIN ] Update Progress: 1954/2367 symbols completed. SOC took 31.9s. Estimated remaining: 53m 12s. Rate: 7.8 symbols/min [2025-10-20T21:11:45.392997] [DEBUG] [LOCAL ] SOBO: Primary data saved (39419 records). [2025-10-20T21:11:45.394615] [INFO ] [MAIN ] SOBO: Full history saved. [2025-10-20T21:11:45.409347] [INFO ] [MAIN ] Update Progress: 1955/2367 symbols completed. SOBO took 38.7s. Estimated remaining: 53m 2s. Rate: 7.8 symbols/min [2025-10-20T21:11:45.411636] [INFO ] [MAIN ] SOJE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:45.545220] [DEBUG] [LOCAL ] SOJE: Loaded 8990 existing records from primary storage. [2025-10-20T21:11:45.563016] [INFO ] [MAIN ] SOJE: Loaded 8990 existing records. [2025-10-20T21:11:45.667973] [INFO ] [MAIN ] SOJF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:45.690814] [DEBUG] [LOCAL ] SOJF: No existing data file found in primary storage. [2025-10-20T21:11:45.690916] [WARN ] [MAIN ] SOJF: No existing data found, performing full refresh instead. [2025-10-20T21:11:45.696822] [INFO ] [MAIN ] SOJF: Wiping existing primary data before full refresh. [2025-10-20T21:11:45.696902] [INFO ] [LOCAL ] SOJF: Starting deletion of primary data. [2025-10-20T21:11:45.712346] [DEBUG] [LOCAL ] SOJF: No primary data file found to delete. [2025-10-20T21:11:45.735344] [INFO ] [MAIN ] SOJF: Starting full primary refresh. [2025-10-20T21:11:45.804817] [WARN ] [MAIN ] SOJE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:45.847101] [INFO ] [MAIN ] SOJE: Wiping existing primary data before full refresh. [2025-10-20T21:11:45.864674] [INFO ] [REMOTE ] SOJC: Fetching primary data for 2025-10. [2025-10-20T21:11:45.883266] [INFO ] [LOCAL ] SOJE: Starting deletion of primary data. [2025-10-20T21:11:45.896143] [DEBUG] [LOCAL ] SOJE: Deleted primary data file. [2025-10-20T21:11:45.896538] [INFO ] [MAIN ] SOJE: Starting full primary refresh. [2025-10-20T21:11:45.945865] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T21:11:46.328150] [INFO ] [REMOTE ] SOJF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:46.526061] [INFO ] [REMOTE ] SOJE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:46.529050] [INFO ] [REMOTE ] SOJC: Incremental fetch complete for 2025-10 (0.66 seconds). [2025-10-20T21:11:46.530391] [INFO ] [REMOTE ] SOJC: Full fetch complete (7259 records). [2025-10-20T21:11:46.530457] [DEBUG] [VALIDATOR ] SOJC: Validating primary data. [2025-10-20T21:11:46.666609] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-01. [2025-10-20T21:11:46.885960] [DEBUG] [VALIDATOR ] SOJC: All expected trading days are present. [2025-10-20T21:11:46.886023] [DEBUG] [VALIDATOR ] SOJC: Primary data validation passed. [2025-10-20T21:11:46.886210] [INFO ] [MAIN ] SOJC: Fetched primary data is valid. [2025-10-20T21:11:46.886247] [DEBUG] [LOCAL ] SOJC: Saving primary data. [2025-10-20T21:11:46.886619] [DEBUG] [LOCAL ] SOJC: Trimmed 7259 records to 4689 (keeping 200 days of data) [2025-10-20T21:11:46.914912] [DEBUG] [LOCAL ] SOJC: Primary data saved (4689 records). [2025-10-20T21:11:46.915014] [INFO ] [MAIN ] SOJC: Full history saved. [2025-10-20T21:11:46.915088] [INFO ] [MAIN ] Update Progress: 1956/2367 symbols completed. SOJC took 25.2s. Estimated remaining: 52m 53s. Rate: 7.8 symbols/min [2025-10-20T21:11:47.092308] [INFO ] [MAIN ] SOL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:11:47.105902] [DEBUG] [LOCAL ] SOL: Loaded 13596 existing records from primary storage. [2025-10-20T21:11:47.106080] [INFO ] [MAIN ] SOL: Loaded 13596 existing records. [2025-10-20T21:11:47.285810] [WARN ] [MAIN ] SOL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:11:47.285973] [INFO ] [MAIN ] SOL: Wiping existing primary data before full refresh. [2025-10-20T21:11:47.286020] [INFO ] [LOCAL ] SOL: Starting deletion of primary data. [2025-10-20T21:11:47.286167] [DEBUG] [LOCAL ] SOL: Deleted primary data file. [2025-10-20T21:11:47.286206] [INFO ] [MAIN ] SOL: Starting full primary refresh. [2025-10-20T21:11:47.464892] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-08. [2025-10-20T21:11:47.592801] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T21:11:47.668004] [INFO ] [REMOTE ] SOL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:11:48.251359] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-01. [2025-10-20T21:11:48.443957] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-08 (0.98 seconds). [2025-10-20T21:11:49.051389] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-02. [2025-10-20T21:11:49.221120] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T21:11:49.855428] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-02. [2025-10-20T21:11:49.857165] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:11:50.651334] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-09. [2025-10-20T21:11:50.923619] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T21:11:51.451356] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-03. [2025-10-20T21:11:51.539621] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T21:11:52.073803] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T21:11:52.251392] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-04. [2025-10-20T21:11:53.051410] [INFO ] [REMOTE ] SOJD: Fetching primary data for 2025-10. [2025-10-20T21:11:53.104181] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-04 (0.85 seconds). [2025-10-20T21:11:53.851396] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-05. [2025-10-20T21:11:53.906678] [INFO ] [REMOTE ] SOJD: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T21:11:53.908960] [INFO ] [REMOTE ] SOJD: Full fetch complete (19123 records). [2025-10-20T21:11:53.909027] [DEBUG] [VALIDATOR ] SOJD: Validating primary data. [2025-10-20T21:11:54.306020] [DEBUG] [VALIDATOR ] SOJD: All expected trading days are present. [2025-10-20T21:11:54.306092] [DEBUG] [VALIDATOR ] SOJD: Primary data validation passed. [2025-10-20T21:11:54.306433] [INFO ] [MAIN ] SOJD: Fetched primary data is valid. [2025-10-20T21:11:54.306470] [DEBUG] [LOCAL ] SOJD: Saving primary data. [2025-10-20T21:11:54.307007] [DEBUG] [LOCAL ] SOJD: Trimmed 19123 records to 12905 (keeping 200 days of data) [2025-10-20T21:11:54.382788] [DEBUG] [LOCAL ] SOJD: Primary data saved (12905 records). [2025-10-20T21:11:54.382914] [INFO ] [MAIN ] SOJD: Full history saved. [2025-10-20T21:11:54.382988] [INFO ] [MAIN ] Update Progress: 1957/2367 symbols completed. SOJD took 26.2s. Estimated remaining: 52m 46s. Rate: 7.8 symbols/min [2025-10-20T21:11:54.651396] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-03. [2025-10-20T21:11:54.706416] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:11:55.451354] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-01. [2025-10-20T21:11:55.767799] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T21:11:56.251369] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-06. [2025-10-20T21:11:56.550993] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T21:11:57.051365] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-02. [2025-10-20T21:11:57.071707] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T21:11:57.856784] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-07. [2025-10-20T21:11:57.858515] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:11:58.650671] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T21:11:58.652881] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-04. [2025-10-20T21:11:59.451433] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-08. [2025-10-20T21:11:59.493606] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T21:12:00.255638] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-03. [2025-10-20T21:12:00.258433] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T21:12:01.012679] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T21:12:01.051382] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-04. [2025-10-20T21:12:01.851371] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-09. [2025-10-20T21:12:01.881375] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T21:12:02.651402] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-05. [2025-10-20T21:12:02.685980] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:12:03.368801] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T21:12:03.451333] [INFO ] [REMOTE ] SOJE: Fetching primary data for 2025-10. [2025-10-20T21:12:04.158956] [INFO ] [REMOTE ] SOJE: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T21:12:04.160789] [INFO ] [REMOTE ] SOJE: Full fetch complete (13462 records). [2025-10-20T21:12:04.160867] [DEBUG] [VALIDATOR ] SOJE: Validating primary data. [2025-10-20T21:12:04.260335] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-06. [2025-10-20T21:12:04.555630] [DEBUG] [VALIDATOR ] SOJE: All expected trading days are present. [2025-10-20T21:12:04.555816] [DEBUG] [VALIDATOR ] SOJE: Primary data validation passed. [2025-10-20T21:12:04.556391] [INFO ] [MAIN ] SOJE: Fetched primary data is valid. [2025-10-20T21:12:04.556436] [DEBUG] [LOCAL ] SOJE: Saving primary data. [2025-10-20T21:12:04.557085] [DEBUG] [LOCAL ] SOJE: Trimmed 13462 records to 8803 (keeping 200 days of data) [2025-10-20T21:12:04.615066] [DEBUG] [LOCAL ] SOJE: Primary data saved (8803 records). [2025-10-20T21:12:04.615274] [INFO ] [MAIN ] SOJE: Full history saved. [2025-10-20T21:12:04.615387] [INFO ] [MAIN ] Update Progress: 1958/2367 symbols completed. SOJE took 19.5s. Estimated remaining: 52m 38s. Rate: 7.8 symbols/min [2025-10-20T21:12:05.020329] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T21:12:05.051378] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-07. [2025-10-20T21:12:05.851458] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-05. [2025-10-20T21:12:05.901336] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T21:12:06.618905] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T21:12:06.651421] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-06. [2025-10-20T21:12:07.451395] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-08. [2025-10-20T21:12:07.757077] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:12:08.251425] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-07. [2025-10-20T21:12:08.290226] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T21:12:09.051397] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-09. [2025-10-20T21:12:09.323422] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T21:12:09.828054] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T21:12:09.851363] [INFO ] [REMOTE ] SOJF: Fetching primary data for 2025-10. [2025-10-20T21:12:10.579464] [INFO ] [REMOTE ] SOJF: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:12:10.581026] [INFO ] [REMOTE ] SOJF: Full fetch complete (14056 records). [2025-10-20T21:12:10.581097] [DEBUG] [VALIDATOR ] SOJF: Validating primary data. [2025-10-20T21:12:10.658907] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-08. [2025-10-20T21:12:10.769218] [WARN ] [VALIDATOR ] SOJF: Validation failed - insufficient historical data. Have 191 unique days, need 200 days [2025-10-20T21:12:10.769548] [FATAL] [MAIN ] SOJF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:12:10.769618] [WARN ] [MAIN ] SOJF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:12:10.769701] [INFO ] [LOCAL ] SOJF: Starting deletion of primary data. [2025-10-20T21:12:10.769793] [DEBUG] [LOCAL ] SOJF: No primary data file found to delete. [2025-10-20T21:12:10.769830] [INFO ] [LOCAL ] SOJF: Starting deletion of derivative data. [2025-10-20T21:12:10.769868] [DEBUG] [LOCAL ] SOJF: No derivative data file found to delete. [2025-10-20T21:12:10.769926] [INFO ] [MAIN ] Update Progress: 1959/2367 symbols completed. SOJF took 25.4s. Estimated remaining: 52m 30s. Rate: 7.8 symbols/min [2025-10-20T21:12:11.583575] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T21:12:11.586553] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-09. [2025-10-20T21:12:12.511398] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T21:12:12.514443] [INFO ] [REMOTE ] SOL: Fetching primary data for 2025-10. [2025-10-20T21:12:13.585364] [INFO ] [REMOTE ] SOL: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T21:12:13.588733] [INFO ] [REMOTE ] SOL: Full fetch complete (21335 records). [2025-10-20T21:12:13.588832] [DEBUG] [VALIDATOR ] SOL: Validating primary data. [2025-10-20T21:12:13.961370] [DEBUG] [VALIDATOR ] SOL: All expected trading days are present. [2025-10-20T21:12:13.961446] [DEBUG] [VALIDATOR ] SOL: Primary data validation passed. [2025-10-20T21:12:13.961947] [INFO ] [MAIN ] SOL: Fetched primary data is valid. [2025-10-20T21:12:13.962007] [DEBUG] [LOCAL ] SOL: Saving primary data. [2025-10-20T21:12:13.962580] [DEBUG] [LOCAL ] SOL: Trimmed 21335 records to 13587 (keeping 200 days of data) [2025-10-20T21:12:14.038584] [DEBUG] [LOCAL ] SOL: Primary data saved (13587 records). [2025-10-20T21:12:14.038750] [INFO ] [MAIN ] SOL: Full history saved. [2025-10-20T21:12:14.038831] [INFO ] [MAIN ] Update Progress: 1960/2367 symbols completed. SOL took 27.1s. Estimated remaining: 52m 22s. Rate: 7.8 symbols/min [2025-10-20T21:12:14.603421] [INFO ] [MAIN ] SONY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:14.738978] [INFO ] [MAIN ] SOLV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:14.774500] [INFO ] [MAIN ] SOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:14.748816] [INFO ] [MAIN ] SON: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:14.781443] [DEBUG] [LOCAL ] SOR: Loaded 2914 existing records from primary storage. [2025-10-20T21:12:14.781971] [INFO ] [MAIN ] SOR: Loaded 2914 existing records. [2025-10-20T21:12:15.008944] [WARN ] [MAIN ] SOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:15.026775] [DEBUG] [LOCAL ] SOLV: Loaded 44343 existing records from primary storage. [2025-10-20T21:12:15.023434] [INFO ] [MAIN ] SOR: Wiping existing primary data before full refresh. [2025-10-20T21:12:15.027344] [INFO ] [LOCAL ] SOR: Starting deletion of primary data. [2025-10-20T21:12:15.027496] [INFO ] [MAIN ] SOLV: Loaded 44343 existing records. [2025-10-20T21:12:15.041399] [DEBUG] [LOCAL ] SOR: Deleted primary data file. [2025-10-20T21:12:15.041807] [INFO ] [MAIN ] SOR: Starting full primary refresh. [2025-10-20T21:12:15.041300] [DEBUG] [LOCAL ] SON: Loaded 44067 existing records from primary storage. [2025-10-20T21:12:15.091858] [INFO ] [MAIN ] SON: Loaded 44067 existing records. [2025-10-20T21:12:15.147155] [DEBUG] [LOCAL ] SONY: Loaded 54159 existing records from primary storage. [2025-10-20T21:12:15.206626] [INFO ] [MAIN ] SONY: Loaded 54159 existing records. [2025-10-20T21:12:15.557301] [WARN ] [MAIN ] SOLV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:15.584275] [INFO ] [MAIN ] SOLV: Wiping existing primary data before full refresh. [2025-10-20T21:12:15.620260] [INFO ] [LOCAL ] SOLV: Starting deletion of primary data. [2025-10-20T21:12:15.745769] [DEBUG] [LOCAL ] SOLV: Deleted primary data file. [2025-10-20T21:12:15.745680] [WARN ] [MAIN ] SONY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:15.796515] [WARN ] [MAIN ] SON: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:15.813374] [INFO ] [MAIN ] SONY: Wiping existing primary data before full refresh. [2025-10-20T21:12:15.783141] [INFO ] [MAIN ] SOLV: Starting full primary refresh. [2025-10-20T21:12:15.813663] [INFO ] [MAIN ] SON: Wiping existing primary data before full refresh. [2025-10-20T21:12:15.858584] [INFO ] [LOCAL ] SON: Starting deletion of primary data. [2025-10-20T21:12:15.858675] [INFO ] [LOCAL ] SONY: Starting deletion of primary data. [2025-10-20T21:12:15.906483] [DEBUG] [LOCAL ] SON: Deleted primary data file. [2025-10-20T21:12:15.922179] [DEBUG] [LOCAL ] SONY: Deleted primary data file. [2025-10-20T21:12:15.916972] [INFO ] [MAIN ] SON: Starting full primary refresh. [2025-10-20T21:12:15.941814] [INFO ] [MAIN ] SONY: Starting full primary refresh. [2025-10-20T21:12:16.459551] [INFO ] [REMOTE ] SOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:16.528071] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-01. [2025-10-20T21:12:16.981817] [INFO ] [REMOTE ] SOLV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:17.096831] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-01. [2025-10-20T21:12:17.093429] [INFO ] [REMOTE ] SONY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:17.122903] [INFO ] [REMOTE ] SON: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:17.118752] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-01. [2025-10-20T21:12:17.125066] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-01. [2025-10-20T21:12:17.430027] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T21:12:17.431791] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-02. [2025-10-20T21:12:18.060425] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T21:12:18.061678] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-03. [2025-10-20T21:12:18.614288] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T21:12:18.615597] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-04. [2025-10-20T21:12:18.690147] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T21:12:18.698948] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-02. [2025-10-20T21:12:18.775719] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T21:12:19.071913] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-10-20T21:12:19.182129] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-01 (2.06 seconds). [2025-10-20T21:12:19.451424] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-02. [2025-10-20T21:12:20.251413] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-05. [2025-10-20T21:12:20.274226] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T21:12:20.839052] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T21:12:21.051340] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-03. [2025-10-20T21:12:21.413131] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-02 (1.96 seconds). [2025-10-20T21:12:21.851402] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-06. [2025-10-20T21:12:22.463399] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T21:12:22.651395] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-07. [2025-10-20T21:12:22.740697] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:12:23.215356] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T21:12:23.451395] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-08. [2025-10-20T21:12:23.943440] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T21:12:24.251430] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-03. [2025-10-20T21:12:25.051387] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-09. [2025-10-20T21:12:25.588414] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-09 (0.54 seconds). [2025-10-20T21:12:25.851346] [INFO ] [REMOTE ] SOR: Fetching primary data for 2025-10. [2025-10-20T21:12:26.243316] [INFO ] [REMOTE ] SOR: Incremental fetch complete for 2025-10 (0.39 seconds). [2025-10-20T21:12:26.244440] [INFO ] [REMOTE ] SOR: Full fetch complete (4352 records). [2025-10-20T21:12:26.244694] [DEBUG] [VALIDATOR ] SOR: Validating primary data. [2025-10-20T21:12:26.375536] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-03 (2.12 seconds). [2025-10-20T21:12:26.635073] [DEBUG] [VALIDATOR ] SOR: All expected trading days are present. [2025-10-20T21:12:26.635154] [DEBUG] [VALIDATOR ] SOR: Primary data validation passed. [2025-10-20T21:12:26.635283] [INFO ] [MAIN ] SOR: Fetched primary data is valid. [2025-10-20T21:12:26.635324] [DEBUG] [LOCAL ] SOR: Saving primary data. [2025-10-20T21:12:26.635702] [DEBUG] [LOCAL ] SOR: Trimmed 4352 records to 2827 (keeping 200 days of data) [2025-10-20T21:12:26.653182] [DEBUG] [LOCAL ] SOR: Primary data saved (2827 records). [2025-10-20T21:12:26.651464] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-02. [2025-10-20T21:12:26.653402] [INFO ] [MAIN ] SOR: Full history saved. [2025-10-20T21:12:26.653514] [INFO ] [MAIN ] Update Progress: 1961/2367 symbols completed. SOR took 12.6s. Estimated remaining: 52m 15s. Rate: 7.8 symbols/min [2025-10-20T21:12:26.829822] [INFO ] [MAIN ] SOS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:26.840393] [DEBUG] [LOCAL ] SOS: Loaded 9578 existing records from primary storage. [2025-10-20T21:12:26.840566] [INFO ] [MAIN ] SOS: Loaded 9578 existing records. [2025-10-20T21:12:27.013441] [WARN ] [MAIN ] SOS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:27.013612] [INFO ] [MAIN ] SOS: Wiping existing primary data before full refresh. [2025-10-20T21:12:27.013687] [INFO ] [LOCAL ] SOS: Starting deletion of primary data. [2025-10-20T21:12:27.013873] [DEBUG] [LOCAL ] SOS: Deleted primary data file. [2025-10-20T21:12:27.013918] [INFO ] [MAIN ] SOS: Starting full primary refresh. [2025-10-20T21:12:27.360570] [INFO ] [REMOTE ] SOS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:27.451333] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-01. [2025-10-20T21:12:28.160768] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-01 (0.71 seconds). [2025-10-20T21:12:28.183956] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:12:28.251336] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-03. [2025-10-20T21:12:29.051414] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-04. [2025-10-20T21:12:29.851363] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-04. [2025-10-20T21:12:29.932155] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T21:12:30.651401] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-02. [2025-10-20T21:12:30.905807] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T21:12:31.288595] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T21:12:31.354525] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:12:31.451366] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-05. [2025-10-20T21:12:32.251420] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-04. [2025-10-20T21:12:33.051389] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-05. [2025-10-20T21:12:33.633585] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T21:12:33.672942] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T21:12:33.851372] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-05. [2025-10-20T21:12:34.459429] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T21:12:34.651361] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-06. [2025-10-20T21:12:35.451360] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-03. [2025-10-20T21:12:35.694956] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T21:12:35.876482] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-03 (0.43 seconds). [2025-10-20T21:12:36.242398] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T21:12:36.251340] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-06. [2025-10-20T21:12:37.051398] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-07. [2025-10-20T21:12:37.851353] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-04. [2025-10-20T21:12:38.248844] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-06 (2.00 seconds). [2025-10-20T21:12:38.374572] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T21:12:38.651498] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-06. [2025-10-20T21:12:38.653103] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T21:12:39.451369] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-08. [2025-10-20T21:12:40.249081] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:12:40.256578] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-07. [2025-10-20T21:12:41.051434] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-07. [2025-10-20T21:12:41.084708] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:12:41.851352] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-09. [2025-10-20T21:12:42.080074] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T21:12:42.651413] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-08. [2025-10-20T21:12:42.770582] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:12:43.451440] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-05. [2025-10-20T21:12:43.483241] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T21:12:44.124859] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T21:12:44.251397] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-08. [2025-10-20T21:12:44.366535] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T21:12:45.051420] [INFO ] [REMOTE ] SOLV: Fetching primary data for 2025-10. [2025-10-20T21:12:45.831906] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T21:12:45.832860] [INFO ] [REMOTE ] SOLV: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:12:45.851832] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-09. [2025-10-20T21:12:45.854048] [INFO ] [REMOTE ] SOLV: Full fetch complete (63180 records). [2025-10-20T21:12:45.854234] [DEBUG] [VALIDATOR ] SOLV: Validating primary data. [2025-10-20T21:12:45.855141] [FATAL] [VALIDATOR ] SOLV: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:12:45.855366] [FATAL] [MAIN ] SOLV: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:12:45.855444] [WARN ] [MAIN ] SOLV: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:12:45.855523] [INFO ] [LOCAL ] SOLV: Starting deletion of primary data. [2025-10-20T21:12:45.855594] [DEBUG] [LOCAL ] SOLV: No primary data file found to delete. [2025-10-20T21:12:45.855630] [INFO ] [LOCAL ] SOLV: Starting deletion of derivative data. [2025-10-20T21:12:45.855833] [DEBUG] [LOCAL ] SOLV: Deleted derivative data file. [2025-10-20T21:12:45.856182] [INFO ] [MAIN ] Update Progress: 1962/2367 symbols completed. SOLV took 31.8s. Estimated remaining: 52m 10s. Rate: 7.8 symbols/min [2025-10-20T21:12:46.033285] [INFO ] [MAIN ] SPB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:46.068161] [DEBUG] [LOCAL ] SPB: Loaded 34547 existing records from primary storage. [2025-10-20T21:12:46.068465] [INFO ] [MAIN ] SPB: Loaded 34547 existing records. [2025-10-20T21:12:46.244237] [WARN ] [MAIN ] SPB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:46.244462] [INFO ] [MAIN ] SPB: Wiping existing primary data before full refresh. [2025-10-20T21:12:46.244509] [INFO ] [LOCAL ] SPB: Starting deletion of primary data. [2025-10-20T21:12:46.244751] [DEBUG] [LOCAL ] SPB: Deleted primary data file. [2025-10-20T21:12:46.244812] [INFO ] [MAIN ] SPB: Starting full primary refresh. [2025-10-20T21:12:46.592609] [INFO ] [REMOTE ] SPB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:46.651362] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-09. [2025-10-20T21:12:47.451362] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-01. [2025-10-20T21:12:47.977563] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-09 (2.13 seconds). [2025-10-20T21:12:48.251385] [INFO ] [REMOTE ] SONY: Fetching primary data for 2025-10. [2025-10-20T21:12:48.283856] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T21:12:48.869620] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T21:12:49.051375] [INFO ] [REMOTE ] SON: Fetching primary data for 2025-10. [2025-10-20T21:12:49.832584] [INFO ] [REMOTE ] SON: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:12:49.839184] [INFO ] [REMOTE ] SON: Full fetch complete (62541 records). [2025-10-20T21:12:49.839363] [DEBUG] [VALIDATOR ] SON: Validating primary data. [2025-10-20T21:12:49.840246] [FATAL] [VALIDATOR ] SON: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:12:49.840341] [FATAL] [MAIN ] SON: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:12:49.840420] [WARN ] [MAIN ] SON: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:12:49.840508] [INFO ] [LOCAL ] SON: Starting deletion of primary data. [2025-10-20T21:12:49.840597] [DEBUG] [LOCAL ] SON: No primary data file found to delete. [2025-10-20T21:12:49.840638] [INFO ] [LOCAL ] SON: Starting deletion of derivative data. [2025-10-20T21:12:49.840867] [DEBUG] [LOCAL ] SON: Deleted derivative data file. [2025-10-20T21:12:49.840958] [INFO ] [MAIN ] Update Progress: 1963/2367 symbols completed. SON took 35.8s. Estimated remaining: 52m 1s. Rate: 7.8 symbols/min [2025-10-20T21:12:49.852440] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-06. [2025-10-20T21:12:50.023330] [INFO ] [MAIN ] SPCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:50.070110] [DEBUG] [LOCAL ] SPCE: Loaded 50750 existing records from primary storage. [2025-10-20T21:12:50.070404] [INFO ] [MAIN ] SPCE: Loaded 50750 existing records. [2025-10-20T21:12:50.269853] [WARN ] [MAIN ] SPCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:50.272403] [INFO ] [MAIN ] SPCE: Wiping existing primary data before full refresh. [2025-10-20T21:12:50.272614] [INFO ] [LOCAL ] SPCE: Starting deletion of primary data. [2025-10-20T21:12:50.273882] [INFO ] [REMOTE ] SONY: Incremental fetch complete for 2025-10 (2.02 seconds). [2025-10-20T21:12:50.273923] [DEBUG] [LOCAL ] SPCE: Deleted primary data file. [2025-10-20T21:12:50.276627] [INFO ] [MAIN ] SPCE: Starting full primary refresh. [2025-10-20T21:12:50.331719] [INFO ] [REMOTE ] SONY: Full fetch complete (77411 records). [2025-10-20T21:12:50.350722] [DEBUG] [VALIDATOR ] SONY: Validating primary data. [2025-10-20T21:12:50.602810] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T21:12:50.674853] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-02. [2025-10-20T21:12:50.979185] [INFO ] [REMOTE ] SPCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:51.061852] [DEBUG] [VALIDATOR ] SONY: All expected trading days are present. [2025-10-20T21:12:51.061935] [DEBUG] [VALIDATOR ] SONY: Primary data validation passed. [2025-10-20T21:12:51.063050] [INFO ] [MAIN ] SONY: Fetched primary data is valid. [2025-10-20T21:12:51.063114] [DEBUG] [LOCAL ] SONY: Saving primary data. [2025-10-20T21:12:51.064609] [DEBUG] [LOCAL ] SONY: Trimmed 77411 records to 53379 (keeping 200 days of data) [2025-10-20T21:12:51.390543] [DEBUG] [LOCAL ] SONY: Primary data saved (53379 records). [2025-10-20T21:12:51.390760] [INFO ] [MAIN ] SONY: Full history saved. [2025-10-20T21:12:51.390869] [INFO ] [MAIN ] Update Progress: 1964/2367 symbols completed. SONY took 37.4s. Estimated remaining: 51m 52s. Rate: 7.8 symbols/min [2025-10-20T21:12:51.456636] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-01. [2025-10-20T21:12:51.579766] [INFO ] [MAIN ] SPE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:12:51.590169] [DEBUG] [LOCAL ] SPE: Loaded 6321 existing records from primary storage. [2025-10-20T21:12:51.590383] [INFO ] [MAIN ] SPE: Loaded 6321 existing records. [2025-10-20T21:12:51.765129] [WARN ] [MAIN ] SPE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:12:51.765323] [INFO ] [MAIN ] SPE: Wiping existing primary data before full refresh. [2025-10-20T21:12:51.765367] [INFO ] [LOCAL ] SPE: Starting deletion of primary data. [2025-10-20T21:12:51.765543] [DEBUG] [LOCAL ] SPE: Deleted primary data file. [2025-10-20T21:12:51.765598] [INFO ] [MAIN ] SPE: Starting full primary refresh. [2025-10-20T21:12:52.123091] [INFO ] [REMOTE ] SPE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:12:52.145736] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:12:52.251381] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-03. [2025-10-20T21:12:53.051389] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-01. [2025-10-20T21:12:53.326675] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T21:12:53.729495] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T21:12:53.851366] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-02. [2025-10-20T21:12:53.914771] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T21:12:54.651363] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-02. [2025-10-20T21:12:55.451380] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-04. [2025-10-20T21:12:55.495171] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T21:12:55.701131] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:12:56.251375] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-07. [2025-10-20T21:12:56.975370] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T21:12:57.051362] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-05. [2025-10-20T21:12:57.097294] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T21:12:57.851360] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-03. [2025-10-20T21:12:58.503238] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-05 (1.45 seconds). [2025-10-20T21:12:58.651380] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-06. [2025-10-20T21:12:59.451347] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-08. [2025-10-20T21:12:59.923311] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-03 (2.07 seconds). [2025-10-20T21:13:00.113183] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T21:13:00.251381] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-04. [2025-10-20T21:13:00.524376] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:13:01.051422] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-07. [2025-10-20T21:13:01.851386] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-09. [2025-10-20T21:13:02.138550] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T21:13:02.606862] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:13:02.651399] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-05. [2025-10-20T21:13:03.123071] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T21:13:03.451378] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-03. [2025-10-20T21:13:04.219104] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T21:13:04.251328] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-04. [2025-10-20T21:13:04.800396] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T21:13:05.051423] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-06. [2025-10-20T21:13:05.076877] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T21:13:05.851363] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-08. [2025-10-20T21:13:06.651401] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-05. [2025-10-20T21:13:07.295887] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:13:07.348853] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-06 (2.30 seconds). [2025-10-20T21:13:07.363672] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T21:13:07.451355] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-09. [2025-10-20T21:13:08.251404] [INFO ] [REMOTE ] SOS: Fetching primary data for 2025-10. [2025-10-20T21:13:08.891719] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T21:13:09.051350] [INFO ] [REMOTE ] SPB: Fetching primary data for 2025-10. [2025-10-20T21:13:09.104096] [INFO ] [REMOTE ] SOS: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T21:13:09.106361] [INFO ] [REMOTE ] SOS: Full fetch complete (11220 records). [2025-10-20T21:13:09.106432] [DEBUG] [VALIDATOR ] SOS: Validating primary data. [2025-10-20T21:13:09.461011] [DEBUG] [VALIDATOR ] SOS: All expected trading days are present. [2025-10-20T21:13:09.461087] [DEBUG] [VALIDATOR ] SOS: Primary data validation passed. [2025-10-20T21:13:09.461378] [INFO ] [MAIN ] SOS: Fetched primary data is valid. [2025-10-20T21:13:09.461447] [DEBUG] [LOCAL ] SOS: Saving primary data. [2025-10-20T21:13:09.461926] [DEBUG] [LOCAL ] SOS: Trimmed 11220 records to 9555 (keeping 200 days of data) [2025-10-20T21:13:09.513562] [DEBUG] [LOCAL ] SOS: Primary data saved (9555 records). [2025-10-20T21:13:09.513696] [INFO ] [MAIN ] SOS: Full history saved. [2025-10-20T21:13:09.513784] [INFO ] [MAIN ] Update Progress: 1965/2367 symbols completed. SOS took 42.9s. Estimated remaining: 51m 47s. Rate: 7.8 symbols/min [2025-10-20T21:13:09.680757] [INFO ] [MAIN ] SPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:09.731209] [DEBUG] [LOCAL ] SPG: Loaded 50898 existing records from primary storage. [2025-10-20T21:13:09.731529] [INFO ] [MAIN ] SPG: Loaded 50898 existing records. [2025-10-20T21:13:09.856586] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-07. [2025-10-20T21:13:09.914139] [WARN ] [MAIN ] SPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:09.914332] [INFO ] [MAIN ] SPG: Wiping existing primary data before full refresh. [2025-10-20T21:13:09.914398] [INFO ] [LOCAL ] SPG: Starting deletion of primary data. [2025-10-20T21:13:09.914614] [DEBUG] [LOCAL ] SPG: Deleted primary data file. [2025-10-20T21:13:09.914702] [INFO ] [MAIN ] SPG: Starting full primary refresh. [2025-10-20T21:13:10.271051] [INFO ] [REMOTE ] SPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:10.295777] [INFO ] [REMOTE ] SPB: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:13:10.300278] [INFO ] [REMOTE ] SPB: Full fetch complete (47718 records). [2025-10-20T21:13:10.300378] [DEBUG] [VALIDATOR ] SPB: Validating primary data. [2025-10-20T21:13:10.653629] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-06. [2025-10-20T21:13:10.671085] [DEBUG] [VALIDATOR ] SPB: All expected trading days are present. [2025-10-20T21:13:10.671169] [DEBUG] [VALIDATOR ] SPB: Primary data validation passed. [2025-10-20T21:13:10.672355] [INFO ] [MAIN ] SPB: Fetched primary data is valid. [2025-10-20T21:13:10.672558] [DEBUG] [LOCAL ] SPB: Saving primary data. [2025-10-20T21:13:10.673959] [DEBUG] [LOCAL ] SPB: Trimmed 47718 records to 34136 (keeping 200 days of data) [2025-10-20T21:13:10.882748] [DEBUG] [LOCAL ] SPB: Primary data saved (34136 records). [2025-10-20T21:13:10.882871] [INFO ] [MAIN ] SPB: Full history saved. [2025-10-20T21:13:10.883214] [INFO ] [MAIN ] Update Progress: 1966/2367 symbols completed. SPB took 25.0s. Estimated remaining: 51m 38s. Rate: 7.8 symbols/min [2025-10-20T21:13:11.061131] [INFO ] [MAIN ] SPGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:11.104152] [DEBUG] [LOCAL ] SPGI: Loaded 46609 existing records from primary storage. [2025-10-20T21:13:11.104442] [INFO ] [MAIN ] SPGI: Loaded 46609 existing records. [2025-10-20T21:13:11.290186] [WARN ] [MAIN ] SPGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:11.290465] [INFO ] [MAIN ] SPGI: Wiping existing primary data before full refresh. [2025-10-20T21:13:11.290554] [INFO ] [LOCAL ] SPGI: Starting deletion of primary data. [2025-10-20T21:13:11.294070] [DEBUG] [LOCAL ] SPGI: Deleted primary data file. [2025-10-20T21:13:11.295129] [INFO ] [MAIN ] SPGI: Starting full primary refresh. [2025-10-20T21:13:11.328444] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T21:13:11.458142] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-07. [2025-10-20T21:13:11.673768] [INFO ] [REMOTE ] SPGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:12.139906] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-07 (2.28 seconds). [2025-10-20T21:13:12.251349] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-01. [2025-10-20T21:13:12.262249] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T21:13:13.051418] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-01. [2025-10-20T21:13:13.851342] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-08. [2025-10-20T21:13:14.151562] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T21:13:14.651446] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-08. [2025-10-20T21:13:15.016325] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-01 (1.96 seconds). [2025-10-20T21:13:15.118613] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-08 (0.47 seconds). [2025-10-20T21:13:15.451355] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-02. [2025-10-20T21:13:16.126630] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-08 (2.28 seconds). [2025-10-20T21:13:16.251440] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-09. [2025-10-20T21:13:17.047209] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T21:13:17.055968] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-02. [2025-10-20T21:13:17.063442] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T21:13:17.851423] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-03. [2025-10-20T21:13:18.651341] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-09. [2025-10-20T21:13:18.848459] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T21:13:19.451352] [INFO ] [REMOTE ] SPE: Fetching primary data for 2025-10. [2025-10-20T21:13:19.882111] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T21:13:20.087286] [INFO ] [REMOTE ] SPE: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:13:20.088567] [INFO ] [REMOTE ] SPE: Full fetch complete (9835 records). [2025-10-20T21:13:20.088634] [DEBUG] [VALIDATOR ] SPE: Validating primary data. [2025-10-20T21:13:20.257563] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-03. [2025-10-20T21:13:20.445294] [DEBUG] [VALIDATOR ] SPE: All expected trading days are present. [2025-10-20T21:13:20.445371] [DEBUG] [VALIDATOR ] SPE: Primary data validation passed. [2025-10-20T21:13:20.445607] [INFO ] [MAIN ] SPE: Fetched primary data is valid. [2025-10-20T21:13:20.445663] [DEBUG] [LOCAL ] SPE: Saving primary data. [2025-10-20T21:13:20.446123] [DEBUG] [LOCAL ] SPE: Trimmed 9835 records to 6265 (keeping 200 days of data) [2025-10-20T21:13:20.482946] [DEBUG] [LOCAL ] SPE: Primary data saved (6265 records). [2025-10-20T21:13:20.483055] [INFO ] [MAIN ] SPE: Full history saved. [2025-10-20T21:13:20.483132] [INFO ] [MAIN ] Update Progress: 1967/2367 symbols completed. SPE took 29.1s. Estimated remaining: 51m 30s. Rate: 7.8 symbols/min [2025-10-20T21:13:20.653975] [INFO ] [MAIN ] SPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:20.671974] [DEBUG] [LOCAL ] SPH: Loaded 16862 existing records from primary storage. [2025-10-20T21:13:20.672203] [INFO ] [MAIN ] SPH: Loaded 16862 existing records. [2025-10-20T21:13:20.891255] [WARN ] [MAIN ] SPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:20.904527] [INFO ] [MAIN ] SPH: Wiping existing primary data before full refresh. [2025-10-20T21:13:20.904616] [INFO ] [LOCAL ] SPH: Starting deletion of primary data. [2025-10-20T21:13:20.905303] [DEBUG] [LOCAL ] SPH: Deleted primary data file. [2025-10-20T21:13:20.906717] [INFO ] [MAIN ] SPH: Starting full primary refresh. [2025-10-20T21:13:20.906638] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T21:13:21.056501] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-04. [2025-10-20T21:13:21.299522] [INFO ] [REMOTE ] SPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:21.851393] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-01. [2025-10-20T21:13:22.253823] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-03 (2.00 seconds). [2025-10-20T21:13:22.651342] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-04. [2025-10-20T21:13:22.893749] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-01 (1.04 seconds). [2025-10-20T21:13:23.105227] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T21:13:23.451362] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-02. [2025-10-20T21:13:24.251364] [INFO ] [REMOTE ] SPCE: Fetching primary data for 2025-10. [2025-10-20T21:13:24.544884] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T21:13:24.590565] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T21:13:24.634187] [INFO ] [REMOTE ] SPCE: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-10-20T21:13:24.642113] [INFO ] [REMOTE ] SPCE: Full fetch complete (72619 records). [2025-10-20T21:13:24.642243] [DEBUG] [VALIDATOR ] SPCE: Validating primary data. [2025-10-20T21:13:25.055156] [DEBUG] [VALIDATOR ] SPCE: All expected trading days are present. [2025-10-20T21:13:25.054922] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-05. [2025-10-20T21:13:25.055292] [DEBUG] [VALIDATOR ] SPCE: Primary data validation passed. [2025-10-20T21:13:25.056479] [INFO ] [MAIN ] SPCE: Fetched primary data is valid. [2025-10-20T21:13:25.056560] [DEBUG] [LOCAL ] SPCE: Saving primary data. [2025-10-20T21:13:25.058512] [DEBUG] [LOCAL ] SPCE: Trimmed 72619 records to 50018 (keeping 200 days of data) [2025-10-20T21:13:25.336546] [DEBUG] [LOCAL ] SPCE: Primary data saved (50018 records). [2025-10-20T21:13:25.336716] [INFO ] [MAIN ] SPCE: Full history saved. [2025-10-20T21:13:25.336835] [INFO ] [MAIN ] Update Progress: 1968/2367 symbols completed. SPCE took 35.5s. Estimated remaining: 51m 22s. Rate: 7.8 symbols/min [2025-10-20T21:13:25.502085] [INFO ] [MAIN ] SPHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:25.541612] [DEBUG] [LOCAL ] SPHR: Loaded 41225 existing records from primary storage. [2025-10-20T21:13:25.541930] [INFO ] [MAIN ] SPHR: Loaded 41225 existing records. [2025-10-20T21:13:25.719021] [WARN ] [MAIN ] SPHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:25.719250] [INFO ] [MAIN ] SPHR: Wiping existing primary data before full refresh. [2025-10-20T21:13:25.719297] [INFO ] [LOCAL ] SPHR: Starting deletion of primary data. [2025-10-20T21:13:25.719541] [DEBUG] [LOCAL ] SPHR: Deleted primary data file. [2025-10-20T21:13:25.719597] [INFO ] [MAIN ] SPHR: Starting full primary refresh. [2025-10-20T21:13:25.856495] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-05. [2025-10-20T21:13:26.068882] [INFO ] [REMOTE ] SPHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:26.623041] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:13:26.651341] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-06. [2025-10-20T21:13:27.385863] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:13:27.451332] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-06. [2025-10-20T21:13:28.251360] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-03. [2025-10-20T21:13:28.408798] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-06 (1.76 seconds). [2025-10-20T21:13:29.051367] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-01. [2025-10-20T21:13:29.202848] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T21:13:29.475347] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T21:13:29.851408] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-04. [2025-10-20T21:13:30.577876] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:13:30.651336] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-02. [2025-10-20T21:13:30.883894] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-04 (1.03 seconds). [2025-10-20T21:13:31.451395] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-07. [2025-10-20T21:13:32.172436] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T21:13:32.251450] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-05. [2025-10-20T21:13:33.051348] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-03. [2025-10-20T21:13:33.367153] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T21:13:33.407458] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T21:13:33.851421] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-08. [2025-10-20T21:13:34.651350] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-07. [2025-10-20T21:13:34.822244] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T21:13:35.451416] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-04. [2025-10-20T21:13:35.840463] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T21:13:36.251355] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-09. [2025-10-20T21:13:36.570288] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T21:13:36.728815] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:13:37.056498] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-06. [2025-10-20T21:13:37.068300] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T21:13:37.851345] [INFO ] [REMOTE ] SPG: Fetching primary data for 2025-10. [2025-10-20T21:13:37.919930] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T21:13:38.651479] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-07. [2025-10-20T21:13:38.656394] [INFO ] [REMOTE ] SPG: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T21:13:38.664732] [INFO ] [REMOTE ] SPG: Full fetch complete (71796 records). [2025-10-20T21:13:38.664869] [DEBUG] [VALIDATOR ] SPG: Validating primary data. [2025-10-20T21:13:38.665576] [FATAL] [VALIDATOR ] SPG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:13:38.665687] [FATAL] [MAIN ] SPG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:13:38.665742] [WARN ] [MAIN ] SPG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:13:38.665784] [INFO ] [LOCAL ] SPG: Starting deletion of primary data. [2025-10-20T21:13:38.665842] [DEBUG] [LOCAL ] SPG: No primary data file found to delete. [2025-10-20T21:13:38.665896] [INFO ] [LOCAL ] SPG: Starting deletion of derivative data. [2025-10-20T21:13:38.666083] [DEBUG] [LOCAL ] SPG: Deleted derivative data file. [2025-10-20T21:13:38.666184] [INFO ] [MAIN ] Update Progress: 1969/2367 symbols completed. SPG took 29.2s. Estimated remaining: 51m 15s. Rate: 7.8 symbols/min [2025-10-20T21:13:38.840886] [INFO ] [MAIN ] SPIR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:38.876129] [DEBUG] [LOCAL ] SPIR: Loaded 38185 existing records from primary storage. [2025-10-20T21:13:38.876387] [INFO ] [MAIN ] SPIR: Loaded 38185 existing records. [2025-10-20T21:13:39.050186] [WARN ] [MAIN ] SPIR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:39.050386] [INFO ] [MAIN ] SPIR: Wiping existing primary data before full refresh. [2025-10-20T21:13:39.050431] [INFO ] [LOCAL ] SPIR: Starting deletion of primary data. [2025-10-20T21:13:39.050594] [DEBUG] [LOCAL ] SPIR: Deleted primary data file. [2025-10-20T21:13:39.050631] [INFO ] [MAIN ] SPIR: Starting full primary refresh. [2025-10-20T21:13:39.412031] [INFO ] [REMOTE ] SPIR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:39.451335] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-01. [2025-10-20T21:13:39.761255] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T21:13:40.251376] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-08. [2025-10-20T21:13:40.793115] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T21:13:41.051371] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-05. [2025-10-20T21:13:41.851434] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-02. [2025-10-20T21:13:42.204029] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T21:13:42.517151] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T21:13:42.651428] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-08. [2025-10-20T21:13:43.046725] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:13:43.451370] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-03. [2025-10-20T21:13:43.489612] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T21:13:44.251410] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-06. [2025-10-20T21:13:44.782778] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T21:13:45.051423] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-09. [2025-10-20T21:13:45.790419] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:13:45.851367] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-04. [2025-10-20T21:13:46.071396] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T21:13:46.651383] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-07. [2025-10-20T21:13:47.353887] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:13:47.451369] [INFO ] [REMOTE ] SPH: Fetching primary data for 2025-10. [2025-10-20T21:13:48.193017] [INFO ] [REMOTE ] SPH: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:13:48.195599] [INFO ] [REMOTE ] SPH: Full fetch complete (27801 records). [2025-10-20T21:13:48.195702] [DEBUG] [VALIDATOR ] SPH: Validating primary data. [2025-10-20T21:13:48.259516] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-09. [2025-10-20T21:13:48.577970] [DEBUG] [VALIDATOR ] SPH: All expected trading days are present. [2025-10-20T21:13:48.585532] [DEBUG] [VALIDATOR ] SPH: Primary data validation passed. [2025-10-20T21:13:48.598795] [INFO ] [MAIN ] SPH: Fetched primary data is valid. [2025-10-20T21:13:48.598948] [DEBUG] [LOCAL ] SPH: Saving primary data. [2025-10-20T21:13:48.600935] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T21:13:48.608907] [DEBUG] [LOCAL ] SPH: Trimmed 27801 records to 16660 (keeping 200 days of data) [2025-10-20T21:13:48.711053] [DEBUG] [LOCAL ] SPH: Primary data saved (16660 records). [2025-10-20T21:13:48.711184] [INFO ] [MAIN ] SPH: Full history saved. [2025-10-20T21:13:48.711259] [INFO ] [MAIN ] Update Progress: 1970/2367 symbols completed. SPH took 28.2s. Estimated remaining: 51m 8s. Rate: 7.8 symbols/min [2025-10-20T21:13:48.881482] [INFO ] [MAIN ] SPMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:48.889494] [DEBUG] [LOCAL ] SPMC: Loaded 5342 existing records from primary storage. [2025-10-20T21:13:48.889722] [INFO ] [MAIN ] SPMC: Loaded 5342 existing records. [2025-10-20T21:13:49.065226] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-05. [2025-10-20T21:13:49.096583] [WARN ] [MAIN ] SPMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:49.097615] [INFO ] [MAIN ] SPMC: Wiping existing primary data before full refresh. [2025-10-20T21:13:49.097793] [INFO ] [LOCAL ] SPMC: Starting deletion of primary data. [2025-10-20T21:13:49.098073] [DEBUG] [LOCAL ] SPMC: Deleted primary data file. [2025-10-20T21:13:49.098151] [INFO ] [MAIN ] SPMC: Starting full primary refresh. [2025-10-20T21:13:49.496515] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-09 (1.24 seconds). [2025-10-20T21:13:49.500230] [INFO ] [REMOTE ] SPMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:49.851381] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-08. [2025-10-20T21:13:50.655842] [INFO ] [REMOTE ] SPGI: Fetching primary data for 2025-10. [2025-10-20T21:13:50.658417] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T21:13:51.228301] [INFO ] [REMOTE ] SPGI: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T21:13:51.235165] [INFO ] [REMOTE ] SPGI: Full fetch complete (67373 records). [2025-10-20T21:13:51.235321] [DEBUG] [VALIDATOR ] SPGI: Validating primary data. [2025-10-20T21:13:51.235966] [FATAL] [VALIDATOR ] SPGI: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:13:51.236052] [FATAL] [MAIN ] SPGI: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:13:51.236098] [WARN ] [MAIN ] SPGI: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:13:51.236136] [INFO ] [LOCAL ] SPGI: Starting deletion of primary data. [2025-10-20T21:13:51.236190] [DEBUG] [LOCAL ] SPGI: No primary data file found to delete. [2025-10-20T21:13:51.236225] [INFO ] [LOCAL ] SPGI: Starting deletion of derivative data. [2025-10-20T21:13:51.236382] [DEBUG] [LOCAL ] SPGI: Deleted derivative data file. [2025-10-20T21:13:51.236796] [INFO ] [MAIN ] Update Progress: 1971/2367 symbols completed. SPGI took 40.4s. Estimated remaining: 50m 59s. Rate: 7.8 symbols/min [2025-10-20T21:13:51.405913] [INFO ] [MAIN ] SPME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:51.406096] [DEBUG] [LOCAL ] SPME: No existing data file found in primary storage. [2025-10-20T21:13:51.406245] [WARN ] [MAIN ] SPME: No existing data found, performing full refresh instead. [2025-10-20T21:13:51.406373] [INFO ] [MAIN ] SPME: Wiping existing primary data before full refresh. [2025-10-20T21:13:51.406439] [INFO ] [LOCAL ] SPME: Starting deletion of primary data. [2025-10-20T21:13:51.406574] [DEBUG] [LOCAL ] SPME: No primary data file found to delete. [2025-10-20T21:13:51.406658] [INFO ] [MAIN ] SPME: Starting full primary refresh. [2025-10-20T21:13:51.458990] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-06. [2025-10-20T21:13:51.642897] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T21:13:51.804854] [INFO ] [REMOTE ] SPME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:52.251374] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-09. [2025-10-20T21:13:52.823625] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T21:13:53.051377] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-07. [2025-10-20T21:13:53.641764] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T21:13:53.851389] [INFO ] [REMOTE ] SPHR: Fetching primary data for 2025-10. [2025-10-20T21:13:54.589564] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T21:13:54.651349] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-01. [2025-10-20T21:13:55.071037] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-01 (0.42 seconds). [2025-10-20T21:13:55.184103] [INFO ] [REMOTE ] SPHR: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T21:13:55.191168] [INFO ] [REMOTE ] SPHR: Full fetch complete (58742 records). [2025-10-20T21:13:55.191282] [DEBUG] [VALIDATOR ] SPHR: Validating primary data. [2025-10-20T21:13:55.456605] [INFO ] [REMOTE ] SPME: Fetching primary data for 2025-01. [2025-10-20T21:13:55.639123] [DEBUG] [VALIDATOR ] SPHR: All expected trading days are present. [2025-10-20T21:13:55.639212] [DEBUG] [VALIDATOR ] SPHR: Primary data validation passed. [2025-10-20T21:13:55.640312] [INFO ] [MAIN ] SPHR: Fetched primary data is valid. [2025-10-20T21:13:55.640406] [DEBUG] [LOCAL ] SPHR: Saving primary data. [2025-10-20T21:13:55.641672] [DEBUG] [LOCAL ] SPHR: Trimmed 58742 records to 40682 (keeping 200 days of data) [2025-10-20T21:13:55.813256] [DEBUG] [REMOTE ] SPME: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:13:55.823407] [WARN ] [REMOTE ] SPME: No data returned for month 2025-01 [2025-10-20T21:13:55.853510] [FATAL] [REMOTE ] SPME: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:13:55.884062] [FATAL] [MAIN ] SPME: Failed to fetch primary data. [2025-10-20T21:13:55.888160] [DEBUG] [LOCAL ] SPHR: Primary data saved (40682 records). [2025-10-20T21:13:55.888237] [WARN ] [MAIN ] SPME: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:13:55.888483] [INFO ] [LOCAL ] SPME: Starting deletion of primary data. [2025-10-20T21:13:55.888375] [INFO ] [MAIN ] SPHR: Full history saved. [2025-10-20T21:13:55.888622] [INFO ] [MAIN ] Update Progress: 1972/2367 symbols completed. SPHR took 30.6s. Estimated remaining: 50m 51s. Rate: 7.8 symbols/min [2025-10-20T21:13:55.888794] [DEBUG] [LOCAL ] SPME: No primary data file found to delete. [2025-10-20T21:13:55.900091] [INFO ] [LOCAL ] SPME: Starting deletion of derivative data. [2025-10-20T21:13:55.951546] [DEBUG] [LOCAL ] SPME: No derivative data file found to delete. [2025-10-20T21:13:55.967887] [INFO ] [MAIN ] Update Progress: 1973/2367 symbols completed. SPME took 4.7s. Estimated remaining: 50m 42s. Rate: 7.8 symbols/min [2025-10-20T21:13:56.217741] [INFO ] [MAIN ] SPNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:56.238699] [INFO ] [MAIN ] SPOT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:13:56.256868] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-08. [2025-10-20T21:13:56.286235] [DEBUG] [LOCAL ] SPNT: Loaded 35165 existing records from primary storage. [2025-10-20T21:13:56.286609] [INFO ] [MAIN ] SPNT: Loaded 35165 existing records. [2025-10-20T21:13:56.421798] [DEBUG] [LOCAL ] SPOT: Loaded 51040 existing records from primary storage. [2025-10-20T21:13:56.432934] [INFO ] [MAIN ] SPOT: Loaded 51040 existing records. [2025-10-20T21:13:56.629063] [WARN ] [MAIN ] SPNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:56.631688] [INFO ] [MAIN ] SPNT: Wiping existing primary data before full refresh. [2025-10-20T21:13:56.631993] [INFO ] [LOCAL ] SPNT: Starting deletion of primary data. [2025-10-20T21:13:56.642680] [DEBUG] [LOCAL ] SPNT: Deleted primary data file. [2025-10-20T21:13:56.653026] [INFO ] [MAIN ] SPNT: Starting full primary refresh. [2025-10-20T21:13:56.821927] [WARN ] [MAIN ] SPOT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:13:56.825243] [INFO ] [MAIN ] SPOT: Wiping existing primary data before full refresh. [2025-10-20T21:13:56.831031] [INFO ] [LOCAL ] SPOT: Starting deletion of primary data. [2025-10-20T21:13:56.841554] [DEBUG] [LOCAL ] SPOT: Deleted primary data file. [2025-10-20T21:13:56.851833] [INFO ] [MAIN ] SPOT: Starting full primary refresh. [2025-10-20T21:13:57.058972] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-02. [2025-10-20T21:13:57.436523] [INFO ] [REMOTE ] SPNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:57.523293] [INFO ] [REMOTE ] SPOT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:13:57.684410] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T21:13:57.851402] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-01. [2025-10-20T21:13:58.010049] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T21:13:58.651353] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-03. [2025-10-20T21:13:58.913016] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T21:13:59.145658] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T21:13:59.451371] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-04. [2025-10-20T21:14:00.203883] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-04 (0.75 seconds). [2025-10-20T21:14:00.251335] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-05. [2025-10-20T21:14:00.757105] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-05 (0.51 seconds). [2025-10-20T21:14:01.051384] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-02. [2025-10-20T21:14:01.851354] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-01. [2025-10-20T21:14:02.578873] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:14:02.651390] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-03. [2025-10-20T21:14:03.451406] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-09. [2025-10-20T21:14:04.047374] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-01 (2.20 seconds). [2025-10-20T21:14:04.104707] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T21:14:04.251419] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-02. [2025-10-20T21:14:05.057420] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-06. [2025-10-20T21:14:05.061868] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T21:14:05.814811] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T21:14:05.851332] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-07. [2025-10-20T21:14:06.292011] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T21:14:06.500249] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T21:14:06.651373] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-04. [2025-10-20T21:14:07.451418] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-08. [2025-10-20T21:14:08.115873] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T21:14:08.205210] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T21:14:08.251364] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-05. [2025-10-20T21:14:09.051434] [INFO ] [REMOTE ] SPIR: Fetching primary data for 2025-10. [2025-10-20T21:14:09.748705] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T21:14:09.851387] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-03. [2025-10-20T21:14:10.607907] [INFO ] [REMOTE ] SPIR: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T21:14:10.616829] [INFO ] [REMOTE ] SPIR: Full fetch complete (54099 records). [2025-10-20T21:14:10.617014] [DEBUG] [VALIDATOR ] SPIR: Validating primary data. [2025-10-20T21:14:10.661001] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-06. [2025-10-20T21:14:11.002140] [DEBUG] [VALIDATOR ] SPIR: All expected trading days are present. [2025-10-20T21:14:11.002220] [DEBUG] [VALIDATOR ] SPIR: Primary data validation passed. [2025-10-20T21:14:11.003136] [INFO ] [MAIN ] SPIR: Fetched primary data is valid. [2025-10-20T21:14:11.003197] [DEBUG] [LOCAL ] SPIR: Saving primary data. [2025-10-20T21:14:11.004307] [DEBUG] [LOCAL ] SPIR: Trimmed 54099 records to 37644 (keeping 200 days of data) [2025-10-20T21:14:11.222267] [DEBUG] [LOCAL ] SPIR: Primary data saved (37644 records). [2025-10-20T21:14:11.222410] [INFO ] [MAIN ] SPIR: Full history saved. [2025-10-20T21:14:11.222487] [INFO ] [MAIN ] Update Progress: 1974/2367 symbols completed. SPIR took 32.6s. Estimated remaining: 50m 35s. Rate: 7.8 symbols/min [2025-10-20T21:14:11.398042] [INFO ] [MAIN ] SPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:11.436050] [DEBUG] [LOCAL ] SPR: Loaded 41640 existing records from primary storage. [2025-10-20T21:14:11.436402] [INFO ] [MAIN ] SPR: Loaded 41640 existing records. [2025-10-20T21:14:11.467594] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-09. [2025-10-20T21:14:11.621161] [WARN ] [MAIN ] SPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:11.621455] [INFO ] [MAIN ] SPR: Wiping existing primary data before full refresh. [2025-10-20T21:14:11.621513] [INFO ] [LOCAL ] SPR: Starting deletion of primary data. [2025-10-20T21:14:11.622023] [DEBUG] [LOCAL ] SPR: Deleted primary data file. [2025-10-20T21:14:11.622079] [INFO ] [MAIN ] SPR: Starting full primary refresh. [2025-10-20T21:14:11.984052] [INFO ] [REMOTE ] SPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:12.035343] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T21:14:12.055058] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T21:14:12.161446] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-03 (2.31 seconds). [2025-10-20T21:14:12.251366] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-07. [2025-10-20T21:14:13.051421] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-01. [2025-10-20T21:14:13.715878] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T21:14:13.851428] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-04. [2025-10-20T21:14:14.601470] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:14:14.651349] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-02. [2025-10-20T21:14:15.451464] [INFO ] [REMOTE ] SPMC: Fetching primary data for 2025-10. [2025-10-20T21:14:16.247344] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T21:14:16.257932] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-08. [2025-10-20T21:14:16.381874] [INFO ] [REMOTE ] SPMC: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T21:14:16.384509] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-04 (2.53 seconds). [2025-10-20T21:14:16.400596] [INFO ] [REMOTE ] SPMC: Full fetch complete (8324 records). [2025-10-20T21:14:16.400826] [DEBUG] [VALIDATOR ] SPMC: Validating primary data. [2025-10-20T21:14:16.764536] [DEBUG] [VALIDATOR ] SPMC: All expected trading days are present. [2025-10-20T21:14:16.764606] [DEBUG] [VALIDATOR ] SPMC: Primary data validation passed. [2025-10-20T21:14:16.764826] [INFO ] [MAIN ] SPMC: Fetched primary data is valid. [2025-10-20T21:14:16.764885] [DEBUG] [LOCAL ] SPMC: Saving primary data. [2025-10-20T21:14:16.765263] [DEBUG] [LOCAL ] SPMC: Trimmed 8324 records to 5386 (keeping 200 days of data) [2025-10-20T21:14:16.797122] [DEBUG] [LOCAL ] SPMC: Primary data saved (5386 records). [2025-10-20T21:14:16.797190] [INFO ] [MAIN ] SPMC: Full history saved. [2025-10-20T21:14:16.797258] [INFO ] [MAIN ] Update Progress: 1975/2367 symbols completed. SPMC took 28.1s. Estimated remaining: 50m 27s. Rate: 7.8 symbols/min [2025-10-20T21:14:16.990954] [INFO ] [MAIN ] SPRU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:17.002901] [DEBUG] [LOCAL ] SPRU: Loaded 10000 existing records from primary storage. [2025-10-20T21:14:17.003126] [INFO ] [MAIN ] SPRU: Loaded 10000 existing records. [2025-10-20T21:14:17.060057] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-05. [2025-10-20T21:14:17.185498] [WARN ] [MAIN ] SPRU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:17.185748] [INFO ] [MAIN ] SPRU: Wiping existing primary data before full refresh. [2025-10-20T21:14:17.185801] [INFO ] [LOCAL ] SPRU: Starting deletion of primary data. [2025-10-20T21:14:17.185948] [DEBUG] [LOCAL ] SPRU: Deleted primary data file. [2025-10-20T21:14:17.185985] [INFO ] [MAIN ] SPRU: Starting full primary refresh. [2025-10-20T21:14:17.535929] [INFO ] [REMOTE ] SPRU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:17.731507] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T21:14:17.851424] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-01. [2025-10-20T21:14:18.547246] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T21:14:18.651392] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-03. [2025-10-20T21:14:19.295222] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-10-20T21:14:19.451419] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-06. [2025-10-20T21:14:20.251404] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-02. [2025-10-20T21:14:20.303071] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T21:14:20.872373] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-02 (0.62 seconds). [2025-10-20T21:14:21.051399] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-03. [2025-10-20T21:14:21.442237] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T21:14:21.695932] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T21:14:21.851487] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-07. [2025-10-20T21:14:22.651427] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-04. [2025-10-20T21:14:23.451417] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-09. [2025-10-20T21:14:24.256492] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-04. [2025-10-20T21:14:24.279420] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-10-20T21:14:24.321043] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T21:14:25.042122] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T21:14:25.051334] [INFO ] [REMOTE ] SPNT: Fetching primary data for 2025-10. [2025-10-20T21:14:25.162939] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T21:14:25.703210] [INFO ] [REMOTE ] SPNT: Incremental fetch complete for 2025-10 (0.65 seconds). [2025-10-20T21:14:25.707709] [INFO ] [REMOTE ] SPNT: Full fetch complete (48430 records). [2025-10-20T21:14:25.707792] [DEBUG] [VALIDATOR ] SPNT: Validating primary data. [2025-10-20T21:14:25.708475] [FATAL] [VALIDATOR ] SPNT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:14:25.708578] [FATAL] [MAIN ] SPNT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:14:25.708638] [WARN ] [MAIN ] SPNT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:14:25.708736] [INFO ] [LOCAL ] SPNT: Starting deletion of primary data. [2025-10-20T21:14:25.708842] [DEBUG] [LOCAL ] SPNT: No primary data file found to delete. [2025-10-20T21:14:25.708912] [INFO ] [LOCAL ] SPNT: Starting deletion of derivative data. [2025-10-20T21:14:25.709083] [DEBUG] [LOCAL ] SPNT: Deleted derivative data file. [2025-10-20T21:14:25.709193] [INFO ] [MAIN ] Update Progress: 1976/2367 symbols completed. SPNT took 29.8s. Estimated remaining: 50m 20s. Rate: 7.8 symbols/min [2025-10-20T21:14:25.856498] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-05. [2025-10-20T21:14:25.883206] [INFO ] [MAIN ] SPXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:25.908047] [DEBUG] [LOCAL ] SPXC: Loaded 23724 existing records from primary storage. [2025-10-20T21:14:25.908278] [INFO ] [MAIN ] SPXC: Loaded 23724 existing records. [2025-10-20T21:14:26.082383] [WARN ] [MAIN ] SPXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:26.082570] [INFO ] [MAIN ] SPXC: Wiping existing primary data before full refresh. [2025-10-20T21:14:26.082639] [INFO ] [LOCAL ] SPXC: Starting deletion of primary data. [2025-10-20T21:14:26.082899] [DEBUG] [LOCAL ] SPXC: Deleted primary data file. [2025-10-20T21:14:26.082964] [INFO ] [MAIN ] SPXC: Starting full primary refresh. [2025-10-20T21:14:26.454014] [INFO ] [REMOTE ] SPXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:26.651364] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-05. [2025-10-20T21:14:26.689527] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-05 (0.83 seconds). [2025-10-20T21:14:27.451356] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-01. [2025-10-20T21:14:28.187882] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T21:14:28.251328] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-06. [2025-10-20T21:14:28.700419] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T21:14:29.051372] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-08. [2025-10-20T21:14:29.809555] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:14:29.851338] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-07. [2025-10-20T21:14:29.901207] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:14:30.651357] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-06. [2025-10-20T21:14:31.314620] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T21:14:31.340362] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T21:14:31.451347] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-09. [2025-10-20T21:14:32.214370] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T21:14:32.251332] [INFO ] [REMOTE ] SPOT: Fetching primary data for 2025-10. [2025-10-20T21:14:32.962798] [INFO ] [REMOTE ] SPOT: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T21:14:32.969594] [INFO ] [REMOTE ] SPOT: Full fetch complete (73455 records). [2025-10-20T21:14:32.969744] [DEBUG] [VALIDATOR ] SPOT: Validating primary data. [2025-10-20T21:14:32.970523] [FATAL] [VALIDATOR ] SPOT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:14:32.970615] [FATAL] [MAIN ] SPOT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:14:32.970700] [WARN ] [MAIN ] SPOT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:14:32.970755] [INFO ] [LOCAL ] SPOT: Starting deletion of primary data. [2025-10-20T21:14:32.970845] [DEBUG] [LOCAL ] SPOT: No primary data file found to delete. [2025-10-20T21:14:32.970911] [INFO ] [LOCAL ] SPOT: Starting deletion of derivative data. [2025-10-20T21:14:32.971058] [DEBUG] [LOCAL ] SPOT: Deleted derivative data file. [2025-10-20T21:14:32.971150] [INFO ] [MAIN ] Update Progress: 1977/2367 symbols completed. SPOT took 37.0s. Estimated remaining: 50m 12s. Rate: 7.8 symbols/min [2025-10-20T21:14:33.056579] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-07. [2025-10-20T21:14:33.142696] [INFO ] [MAIN ] SPXX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:33.152055] [DEBUG] [LOCAL ] SPXX: Loaded 9463 existing records from primary storage. [2025-10-20T21:14:33.152253] [INFO ] [MAIN ] SPXX: Loaded 9463 existing records. [2025-10-20T21:14:33.331141] [WARN ] [MAIN ] SPXX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:33.331322] [INFO ] [MAIN ] SPXX: Wiping existing primary data before full refresh. [2025-10-20T21:14:33.331367] [INFO ] [LOCAL ] SPXX: Starting deletion of primary data. [2025-10-20T21:14:33.331548] [DEBUG] [LOCAL ] SPXX: Deleted primary data file. [2025-10-20T21:14:33.331585] [INFO ] [MAIN ] SPXX: Starting full primary refresh. [2025-10-20T21:14:33.677127] [INFO ] [REMOTE ] SPXX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:33.851345] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-08. [2025-10-20T21:14:33.925913] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-07 (0.87 seconds). [2025-10-20T21:14:34.651355] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-01. [2025-10-20T21:14:35.354781] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T21:14:35.383617] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T21:14:35.451374] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-08. [2025-10-20T21:14:36.251345] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-02. [2025-10-20T21:14:36.569169] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T21:14:37.051414] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-09. [2025-10-20T21:14:37.221022] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T21:14:37.851385] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-09. [2025-10-20T21:14:37.950485] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T21:14:38.651369] [INFO ] [REMOTE ] SPRU: Fetching primary data for 2025-10. [2025-10-20T21:14:39.454959] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-02. [2025-10-20T21:14:39.457201] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T21:14:39.596695] [INFO ] [REMOTE ] SPRU: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T21:14:39.599452] [INFO ] [REMOTE ] SPRU: Full fetch complete (12187 records). [2025-10-20T21:14:39.599537] [DEBUG] [VALIDATOR ] SPRU: Validating primary data. [2025-10-20T21:14:39.960567] [DEBUG] [VALIDATOR ] SPRU: All expected trading days are present. [2025-10-20T21:14:39.960664] [DEBUG] [VALIDATOR ] SPRU: Primary data validation passed. [2025-10-20T21:14:39.960953] [INFO ] [MAIN ] SPRU: Fetched primary data is valid. [2025-10-20T21:14:39.960992] [DEBUG] [LOCAL ] SPRU: Saving primary data. [2025-10-20T21:14:39.961473] [DEBUG] [LOCAL ] SPRU: Trimmed 12187 records to 10032 (keeping 200 days of data) [2025-10-20T21:14:40.021026] [DEBUG] [LOCAL ] SPRU: Primary data saved (10032 records). [2025-10-20T21:14:40.021190] [INFO ] [MAIN ] SPRU: Full history saved. [2025-10-20T21:14:40.021295] [INFO ] [MAIN ] Update Progress: 1978/2367 symbols completed. SPRU took 23.2s. Estimated remaining: 50m 4s. Rate: 7.8 symbols/min [2025-10-20T21:14:40.195238] [INFO ] [MAIN ] SQM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:40.238363] [DEBUG] [LOCAL ] SQM: Loaded 49549 existing records from primary storage. [2025-10-20T21:14:40.238666] [INFO ] [MAIN ] SQM: Loaded 49549 existing records. [2025-10-20T21:14:40.253679] [INFO ] [REMOTE ] SPR: Fetching primary data for 2025-10. [2025-10-20T21:14:40.347547] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T21:14:40.426726] [WARN ] [MAIN ] SQM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:40.426953] [INFO ] [MAIN ] SQM: Wiping existing primary data before full refresh. [2025-10-20T21:14:40.427006] [INFO ] [LOCAL ] SQM: Starting deletion of primary data. [2025-10-20T21:14:40.427259] [DEBUG] [LOCAL ] SQM: Deleted primary data file. [2025-10-20T21:14:40.427301] [INFO ] [MAIN ] SQM: Starting full primary refresh. [2025-10-20T21:14:40.777495] [INFO ] [REMOTE ] SQM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:41.051364] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-01. [2025-10-20T21:14:41.763782] [INFO ] [REMOTE ] SPR: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T21:14:41.769759] [INFO ] [REMOTE ] SPR: Full fetch complete (60148 records). [2025-10-20T21:14:41.769857] [DEBUG] [VALIDATOR ] SPR: Validating primary data. [2025-10-20T21:14:41.859326] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-03. [2025-10-20T21:14:42.149691] [DEBUG] [VALIDATOR ] SPR: All expected trading days are present. [2025-10-20T21:14:42.149773] [DEBUG] [VALIDATOR ] SPR: Primary data validation passed. [2025-10-20T21:14:42.150687] [INFO ] [MAIN ] SPR: Fetched primary data is valid. [2025-10-20T21:14:42.150746] [DEBUG] [LOCAL ] SPR: Saving primary data. [2025-10-20T21:14:42.151808] [DEBUG] [LOCAL ] SPR: Trimmed 60148 records to 40983 (keeping 200 days of data) [2025-10-20T21:14:42.386309] [DEBUG] [LOCAL ] SPR: Primary data saved (40983 records). [2025-10-20T21:14:42.386516] [INFO ] [MAIN ] SPR: Full history saved. [2025-10-20T21:14:42.386718] [INFO ] [MAIN ] Update Progress: 1979/2367 symbols completed. SPR took 31.2s. Estimated remaining: 49m 55s. Rate: 7.8 symbols/min [2025-10-20T21:14:42.533041] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T21:14:42.590683] [INFO ] [MAIN ] SQNS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:42.617758] [DEBUG] [LOCAL ] SQNS: Loaded 27898 existing records from primary storage. [2025-10-20T21:14:42.618088] [INFO ] [MAIN ] SQNS: Loaded 27898 existing records. [2025-10-20T21:14:42.659761] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-03. [2025-10-20T21:14:42.794849] [WARN ] [MAIN ] SQNS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:42.795152] [INFO ] [MAIN ] SQNS: Wiping existing primary data before full refresh. [2025-10-20T21:14:42.795238] [INFO ] [LOCAL ] SQNS: Starting deletion of primary data. [2025-10-20T21:14:42.802390] [DEBUG] [LOCAL ] SQNS: Deleted primary data file. [2025-10-20T21:14:42.802500] [INFO ] [MAIN ] SQNS: Starting full primary refresh. [2025-10-20T21:14:42.834223] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T21:14:43.156083] [INFO ] [REMOTE ] SQNS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:43.451428] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-01. [2025-10-20T21:14:43.783430] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T21:14:44.178907] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T21:14:44.251404] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-02. [2025-10-20T21:14:45.051380] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-04. [2025-10-20T21:14:45.102521] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T21:14:45.851338] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-03. [2025-10-20T21:14:45.933132] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T21:14:46.651346] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-05. [2025-10-20T21:14:46.679116] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T21:14:47.451353] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-02. [2025-10-20T21:14:47.495595] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T21:14:48.251399] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-04. [2025-10-20T21:14:48.920488] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:14:49.051428] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-06. [2025-10-20T21:14:49.685587] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:14:49.847281] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T21:14:49.851324] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-07. [2025-10-20T21:14:50.647899] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T21:14:50.651317] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-08. [2025-10-20T21:14:51.433993] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-08 (0.78 seconds). [2025-10-20T21:14:51.451331] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-09. [2025-10-20T21:14:52.251363] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-03. [2025-10-20T21:14:52.318629] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-09 (0.87 seconds). [2025-10-20T21:14:53.051361] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-05. [2025-10-20T21:14:53.851386] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-04. [2025-10-20T21:14:53.980298] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T21:14:54.037031] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T21:14:54.651363] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-04. [2025-10-20T21:14:54.693510] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T21:14:55.451420] [INFO ] [REMOTE ] SPXX: Fetching primary data for 2025-10. [2025-10-20T21:14:56.251376] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-06. [2025-10-20T21:14:56.378565] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:14:56.406608] [INFO ] [REMOTE ] SPXX: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T21:14:56.408965] [INFO ] [REMOTE ] SPXX: Full fetch complete (13449 records). [2025-10-20T21:14:56.409038] [DEBUG] [VALIDATOR ] SPXX: Validating primary data. [2025-10-20T21:14:56.767421] [DEBUG] [VALIDATOR ] SPXX: All expected trading days are present. [2025-10-20T21:14:56.767495] [DEBUG] [VALIDATOR ] SPXX: Primary data validation passed. [2025-10-20T21:14:56.767803] [INFO ] [MAIN ] SPXX: Fetched primary data is valid. [2025-10-20T21:14:56.767882] [DEBUG] [LOCAL ] SPXX: Saving primary data. [2025-10-20T21:14:56.768365] [DEBUG] [LOCAL ] SPXX: Trimmed 13449 records to 9266 (keeping 200 days of data) [2025-10-20T21:14:56.822592] [DEBUG] [LOCAL ] SPXX: Primary data saved (9266 records). [2025-10-20T21:14:56.822737] [INFO ] [MAIN ] SPXX: Full history saved. [2025-10-20T21:14:56.822819] [INFO ] [MAIN ] Update Progress: 1980/2367 symbols completed. SPXX took 23.9s. Estimated remaining: 49m 49s. Rate: 7.8 symbols/min [2025-10-20T21:14:56.994988] [INFO ] [MAIN ] SR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:14:57.031023] [DEBUG] [LOCAL ] SR: Loaded 32870 existing records from primary storage. [2025-10-20T21:14:57.031420] [INFO ] [MAIN ] SR: Loaded 32870 existing records. [2025-10-20T21:14:57.059809] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-05. [2025-10-20T21:14:57.249485] [WARN ] [MAIN ] SR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:14:57.249834] [INFO ] [MAIN ] SR: Wiping existing primary data before full refresh. [2025-10-20T21:14:57.249890] [INFO ] [LOCAL ] SR: Starting deletion of primary data. [2025-10-20T21:14:57.250069] [DEBUG] [LOCAL ] SR: Deleted primary data file. [2025-10-20T21:14:57.250124] [INFO ] [MAIN ] SR: Starting full primary refresh. [2025-10-20T21:14:57.549986] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T21:14:57.632227] [INFO ] [REMOTE ] SR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:14:57.851352] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-07. [2025-10-20T21:14:57.999537] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T21:14:58.651407] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-01. [2025-10-20T21:14:59.006519] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T21:14:59.451357] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-05. [2025-10-20T21:14:59.632823] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T21:15:00.251374] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-08. [2025-10-20T21:15:01.051417] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-06. [2025-10-20T21:15:01.228480] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T21:15:01.747598] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T21:15:01.851359] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-09. [2025-10-20T21:15:01.945229] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-06 (0.89 seconds). [2025-10-20T21:15:02.651385] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-07. [2025-10-20T21:15:03.277930] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T21:15:03.451389] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-02. [2025-10-20T21:15:04.251404] [INFO ] [REMOTE ] SPXC: Fetching primary data for 2025-10. [2025-10-20T21:15:04.893298] [INFO ] [REMOTE ] SPXC: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:15:04.896703] [INFO ] [REMOTE ] SPXC: Full fetch complete (33985 records). [2025-10-20T21:15:04.896798] [DEBUG] [VALIDATOR ] SPXC: Validating primary data. [2025-10-20T21:15:04.897243] [FATAL] [VALIDATOR ] SPXC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:15:04.897322] [FATAL] [MAIN ] SPXC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:15:04.897365] [WARN ] [MAIN ] SPXC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:15:04.897432] [INFO ] [LOCAL ] SPXC: Starting deletion of primary data. [2025-10-20T21:15:04.897492] [DEBUG] [LOCAL ] SPXC: No primary data file found to delete. [2025-10-20T21:15:04.897527] [INFO ] [LOCAL ] SPXC: Starting deletion of derivative data. [2025-10-20T21:15:04.897719] [DEBUG] [LOCAL ] SPXC: Deleted derivative data file. [2025-10-20T21:15:04.897801] [INFO ] [MAIN ] Update Progress: 1981/2367 symbols completed. SPXC took 39.2s. Estimated remaining: 49m 41s. Rate: 7.8 symbols/min [2025-10-20T21:15:05.016997] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:15:05.062669] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-06. [2025-10-20T21:15:05.094179] [INFO ] [MAIN ] SRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:05.129762] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-07 (2.48 seconds). [2025-10-20T21:15:05.184932] [DEBUG] [LOCAL ] SRE: Loaded 54192 existing records from primary storage. [2025-10-20T21:15:05.185259] [INFO ] [MAIN ] SRE: Loaded 54192 existing records. [2025-10-20T21:15:05.359703] [WARN ] [MAIN ] SRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:15:05.359895] [INFO ] [MAIN ] SRE: Wiping existing primary data before full refresh. [2025-10-20T21:15:05.359942] [INFO ] [LOCAL ] SRE: Starting deletion of primary data. [2025-10-20T21:15:05.360146] [DEBUG] [LOCAL ] SRE: Deleted primary data file. [2025-10-20T21:15:05.360200] [INFO ] [MAIN ] SRE: Starting full primary refresh. [2025-10-20T21:15:05.704881] [INFO ] [REMOTE ] SRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:05.851380] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-01. [2025-10-20T21:15:06.651417] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-03. [2025-10-20T21:15:06.796596] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T21:15:07.451384] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-07. [2025-10-20T21:15:07.716694] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T21:15:08.160546] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:15:08.251371] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-08. [2025-10-20T21:15:09.051397] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-04. [2025-10-20T21:15:09.323528] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T21:15:09.851372] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-02. [2025-10-20T21:15:10.376473] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T21:15:10.542385] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T21:15:10.651365] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-05. [2025-10-20T21:15:11.451344] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-08. [2025-10-20T21:15:11.715392] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T21:15:12.159234] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T21:15:12.251434] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-03. [2025-10-20T21:15:13.051364] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-06. [2025-10-20T21:15:13.320507] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T21:15:13.851432] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-09. [2025-10-20T21:15:14.280337] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T21:15:14.407518] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T21:15:14.651348] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-04. [2025-10-20T21:15:15.204032] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T21:15:15.451376] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-07. [2025-10-20T21:15:16.251348] [INFO ] [REMOTE ] SQNS: Fetching primary data for 2025-10. [2025-10-20T21:15:16.671731] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T21:15:17.008833] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:15:17.051336] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-08. [2025-10-20T21:15:17.503118] [INFO ] [REMOTE ] SQNS: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T21:15:17.506921] [INFO ] [REMOTE ] SQNS: Full fetch complete (33041 records). [2025-10-20T21:15:17.506996] [DEBUG] [VALIDATOR ] SQNS: Validating primary data. [2025-10-20T21:15:17.860691] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-09. [2025-10-20T21:15:17.874689] [DEBUG] [VALIDATOR ] SQNS: All expected trading days are present. [2025-10-20T21:15:17.874762] [DEBUG] [VALIDATOR ] SQNS: Primary data validation passed. [2025-10-20T21:15:17.875540] [INFO ] [MAIN ] SQNS: Fetched primary data is valid. [2025-10-20T21:15:17.875737] [DEBUG] [LOCAL ] SQNS: Saving primary data. [2025-10-20T21:15:17.876560] [DEBUG] [LOCAL ] SQNS: Trimmed 33041 records to 27927 (keeping 200 days of data) [2025-10-20T21:15:18.029400] [DEBUG] [LOCAL ] SQNS: Primary data saved (27927 records). [2025-10-20T21:15:18.029521] [INFO ] [MAIN ] SQNS: Full history saved. [2025-10-20T21:15:18.029595] [INFO ] [MAIN ] Update Progress: 1982/2367 symbols completed. SQNS took 35.6s. Estimated remaining: 49m 35s. Rate: 7.8 symbols/min [2025-10-20T21:15:18.199411] [INFO ] [MAIN ] SREA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:18.210407] [DEBUG] [LOCAL ] SREA: Loaded 10096 existing records from primary storage. [2025-10-20T21:15:18.210587] [INFO ] [MAIN ] SREA: Loaded 10096 existing records. [2025-10-20T21:15:18.394596] [WARN ] [MAIN ] SREA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:15:18.394832] [INFO ] [MAIN ] SREA: Wiping existing primary data before full refresh. [2025-10-20T21:15:18.394901] [INFO ] [LOCAL ] SREA: Starting deletion of primary data. [2025-10-20T21:15:18.395203] [DEBUG] [LOCAL ] SREA: Deleted primary data file. [2025-10-20T21:15:18.395254] [INFO ] [MAIN ] SREA: Starting full primary refresh. [2025-10-20T21:15:18.661093] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-05. [2025-10-20T21:15:18.692391] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:15:18.776569] [INFO ] [REMOTE ] SREA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:19.451331] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-09. [2025-10-20T21:15:19.717223] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T21:15:20.251351] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-01. [2025-10-20T21:15:20.698117] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T21:15:20.915575] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T21:15:21.051348] [INFO ] [REMOTE ] SR: Fetching primary data for 2025-10. [2025-10-20T21:15:21.231427] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-01 (0.98 seconds). [2025-10-20T21:15:21.688864] [INFO ] [REMOTE ] SR: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:15:21.692932] [INFO ] [REMOTE ] SR: Full fetch complete (46011 records). [2025-10-20T21:15:21.693022] [DEBUG] [VALIDATOR ] SR: Validating primary data. [2025-10-20T21:15:21.693597] [FATAL] [VALIDATOR ] SR: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:15:21.693695] [FATAL] [MAIN ] SR: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:15:21.693741] [WARN ] [MAIN ] SR: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:15:21.693805] [INFO ] [LOCAL ] SR: Starting deletion of primary data. [2025-10-20T21:15:21.693869] [DEBUG] [LOCAL ] SR: No primary data file found to delete. [2025-10-20T21:15:21.693904] [INFO ] [LOCAL ] SR: Starting deletion of derivative data. [2025-10-20T21:15:21.694018] [DEBUG] [LOCAL ] SR: Deleted derivative data file. [2025-10-20T21:15:21.694400] [INFO ] [MAIN ] Update Progress: 1983/2367 symbols completed. SR took 24.9s. Estimated remaining: 49m 26s. Rate: 7.8 symbols/min [2025-10-20T21:15:21.865149] [INFO ] [MAIN ] SRFM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:21.857010] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-06. [2025-10-20T21:15:21.903981] [DEBUG] [LOCAL ] SRFM: Loaded 39823 existing records from primary storage. [2025-10-20T21:15:21.904319] [INFO ] [MAIN ] SRFM: Loaded 39823 existing records. [2025-10-20T21:15:22.101723] [WARN ] [MAIN ] SRFM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:15:22.101912] [INFO ] [MAIN ] SRFM: Wiping existing primary data before full refresh. [2025-10-20T21:15:22.101958] [INFO ] [LOCAL ] SRFM: Starting deletion of primary data. [2025-10-20T21:15:22.102122] [DEBUG] [LOCAL ] SRFM: Deleted primary data file. [2025-10-20T21:15:22.102159] [INFO ] [MAIN ] SRFM: Starting full primary refresh. [2025-10-20T21:15:22.451130] [INFO ] [REMOTE ] SRFM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:22.651328] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-01. [2025-10-20T21:15:23.451410] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-02. [2025-10-20T21:15:23.716289] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-06 (1.86 seconds). [2025-10-20T21:15:23.764346] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T21:15:24.251428] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-02. [2025-10-20T21:15:24.269336] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-02 (0.82 seconds). [2025-10-20T21:15:25.051450] [INFO ] [REMOTE ] SQM: Fetching primary data for 2025-10. [2025-10-20T21:15:25.481394] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T21:15:25.851398] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-03. [2025-10-20T21:15:26.651382] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-07. [2025-10-20T21:15:26.877389] [INFO ] [REMOTE ] SQM: Incremental fetch complete for 2025-10 (1.83 seconds). [2025-10-20T21:15:26.885445] [INFO ] [REMOTE ] SQM: Full fetch complete (69780 records). [2025-10-20T21:15:26.885594] [DEBUG] [VALIDATOR ] SQM: Validating primary data. [2025-10-20T21:15:27.048251] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T21:15:27.307737] [DEBUG] [VALIDATOR ] SQM: All expected trading days are present. [2025-10-20T21:15:27.307886] [DEBUG] [VALIDATOR ] SQM: Primary data validation passed. [2025-10-20T21:15:27.308976] [INFO ] [MAIN ] SQM: Fetched primary data is valid. [2025-10-20T21:15:27.309039] [DEBUG] [LOCAL ] SQM: Saving primary data. [2025-10-20T21:15:27.310471] [DEBUG] [LOCAL ] SQM: Trimmed 69780 records to 48976 (keeping 200 days of data) [2025-10-20T21:15:27.471925] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-03. [2025-10-20T21:15:27.592393] [DEBUG] [LOCAL ] SQM: Primary data saved (48976 records). [2025-10-20T21:15:27.592530] [INFO ] [MAIN ] SQM: Full history saved. [2025-10-20T21:15:27.592607] [INFO ] [MAIN ] Update Progress: 1984/2367 symbols completed. SQM took 47.6s. Estimated remaining: 49m 18s. Rate: 7.8 symbols/min [2025-10-20T21:15:27.764546] [INFO ] [MAIN ] SRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:27.781559] [DEBUG] [LOCAL ] SRG: Loaded 15640 existing records from primary storage. [2025-10-20T21:15:27.781854] [INFO ] [MAIN ] SRG: Loaded 15640 existing records. [2025-10-20T21:15:27.965322] [WARN ] [MAIN ] SRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:15:27.965558] [INFO ] [MAIN ] SRG: Wiping existing primary data before full refresh. [2025-10-20T21:15:27.965605] [INFO ] [LOCAL ] SRG: Starting deletion of primary data. [2025-10-20T21:15:27.965787] [DEBUG] [LOCAL ] SRG: Deleted primary data file. [2025-10-20T21:15:27.965841] [INFO ] [MAIN ] SRG: Starting full primary refresh. [2025-10-20T21:15:28.183093] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T21:15:28.261599] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-04. [2025-10-20T21:15:28.356193] [INFO ] [REMOTE ] SRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:28.392065] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T21:15:29.051347] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-08. [2025-10-20T21:15:29.424621] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T21:15:29.851368] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-04. [2025-10-20T21:15:30.651345] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-05. [2025-10-20T21:15:30.748295] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-04 (0.90 seconds). [2025-10-20T21:15:30.987028] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T21:15:31.451397] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-09. [2025-10-20T21:15:31.656833] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-05 (1.01 seconds). [2025-10-20T21:15:32.251384] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-06. [2025-10-20T21:15:32.356752] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-09 (0.91 seconds). [2025-10-20T21:15:33.051376] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-05. [2025-10-20T21:15:33.769989] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T21:15:33.851380] [INFO ] [REMOTE ] SRE: Fetching primary data for 2025-10. [2025-10-20T21:15:34.002346] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T21:15:34.621484] [INFO ] [REMOTE ] SRE: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T21:15:34.628676] [INFO ] [REMOTE ] SRE: Full fetch complete (77116 records). [2025-10-20T21:15:34.628801] [DEBUG] [VALIDATOR ] SRE: Validating primary data. [2025-10-20T21:15:34.629434] [FATAL] [VALIDATOR ] SRE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:15:34.629520] [FATAL] [MAIN ] SRE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:15:34.629563] [WARN ] [MAIN ] SRE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:15:34.629669] [INFO ] [LOCAL ] SRE: Starting deletion of primary data. [2025-10-20T21:15:34.629761] [DEBUG] [LOCAL ] SRE: No primary data file found to delete. [2025-10-20T21:15:34.629808] [INFO ] [LOCAL ] SRE: Starting deletion of derivative data. [2025-10-20T21:15:34.629996] [DEBUG] [LOCAL ] SRE: Deleted derivative data file. [2025-10-20T21:15:34.630097] [INFO ] [MAIN ] Update Progress: 1985/2367 symbols completed. SRE took 29.7s. Estimated remaining: 49m 10s. Rate: 7.8 symbols/min [2025-10-20T21:15:34.659786] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-01. [2025-10-20T21:15:34.806483] [INFO ] [MAIN ] SRI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:34.831475] [DEBUG] [LOCAL ] SRI: Loaded 26831 existing records from primary storage. [2025-10-20T21:15:34.831742] [INFO ] [MAIN ] SRI: Loaded 26831 existing records. [2025-10-20T21:15:35.004188] [WARN ] [MAIN ] SRI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:15:35.004387] [INFO ] [MAIN ] SRI: Wiping existing primary data before full refresh. [2025-10-20T21:15:35.004433] [INFO ] [LOCAL ] SRI: Starting deletion of primary data. [2025-10-20T21:15:35.004637] [DEBUG] [LOCAL ] SRI: Deleted primary data file. [2025-10-20T21:15:35.004726] [INFO ] [MAIN ] SRI: Starting full primary refresh. [2025-10-20T21:15:35.353286] [INFO ] [REMOTE ] SRI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:35.451377] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-01. [2025-10-20T21:15:35.866422] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T21:15:36.251366] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-02. [2025-10-20T21:15:36.707638] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T21:15:37.051350] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-02. [2025-10-20T21:15:37.266462] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T21:15:37.851383] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-06. [2025-10-20T21:15:38.398363] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T21:15:38.651358] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-03. [2025-10-20T21:15:38.662982] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-06 (0.81 seconds). [2025-10-20T21:15:39.451374] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-07. [2025-10-20T21:15:39.759954] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T21:15:40.251360] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-04. [2025-10-20T21:15:40.314472] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T21:15:41.051341] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-08. [2025-10-20T21:15:41.446014] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T21:15:41.851343] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-05. [2025-10-20T21:15:41.862061] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T21:15:42.651348] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-09. [2025-10-20T21:15:42.889699] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T21:15:43.452017] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-03. [2025-10-20T21:15:43.453627] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-09 (0.80 seconds). [2025-10-20T21:15:44.251381] [INFO ] [REMOTE ] SREA: Fetching primary data for 2025-10. [2025-10-20T21:15:44.884531] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T21:15:44.987593] [INFO ] [REMOTE ] SREA: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:15:44.989993] [INFO ] [REMOTE ] SREA: Full fetch complete (14937 records). [2025-10-20T21:15:44.990080] [DEBUG] [VALIDATOR ] SREA: Validating primary data. [2025-10-20T21:15:45.056480] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-06. [2025-10-20T21:15:45.359017] [DEBUG] [VALIDATOR ] SREA: All expected trading days are present. [2025-10-20T21:15:45.359097] [DEBUG] [VALIDATOR ] SREA: Primary data validation passed. [2025-10-20T21:15:45.359416] [INFO ] [MAIN ] SREA: Fetched primary data is valid. [2025-10-20T21:15:45.359453] [DEBUG] [LOCAL ] SREA: Saving primary data. [2025-10-20T21:15:45.359944] [DEBUG] [LOCAL ] SREA: Trimmed 14937 records to 9830 (keeping 200 days of data) [2025-10-20T21:15:45.417432] [DEBUG] [LOCAL ] SREA: Primary data saved (9830 records). [2025-10-20T21:15:45.417559] [INFO ] [MAIN ] SREA: Full history saved. [2025-10-20T21:15:45.417633] [INFO ] [MAIN ] Update Progress: 1986/2367 symbols completed. SREA took 27.4s. Estimated remaining: 49m 3s. Rate: 7.8 symbols/min [2025-10-20T21:15:45.584549] [INFO ] [MAIN ] SRL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:45.584723] [DEBUG] [LOCAL ] SRL: No existing data file found in primary storage. [2025-10-20T21:15:45.584770] [WARN ] [MAIN ] SRL: No existing data found, performing full refresh instead. [2025-10-20T21:15:45.584874] [INFO ] [MAIN ] SRL: Wiping existing primary data before full refresh. [2025-10-20T21:15:45.584914] [INFO ] [LOCAL ] SRL: Starting deletion of primary data. [2025-10-20T21:15:45.584953] [DEBUG] [LOCAL ] SRL: No primary data file found to delete. [2025-10-20T21:15:45.584983] [INFO ] [MAIN ] SRL: Starting full primary refresh. [2025-10-20T21:15:45.859333] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-04. [2025-10-20T21:15:45.943209] [INFO ] [REMOTE ] SRL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:15:45.976252] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-06 (0.92 seconds). [2025-10-20T21:15:46.651369] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-07. [2025-10-20T21:15:47.132281] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T21:15:47.451383] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-05. [2025-10-20T21:15:48.251437] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-01. [2025-10-20T21:15:48.711233] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T21:15:48.753825] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T21:15:49.051408] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-02. [2025-10-20T21:15:49.234682] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-07 (2.58 seconds). [2025-10-20T21:15:49.534179] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T21:15:49.851420] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-03. [2025-10-20T21:15:50.299781] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-03 (0.45 seconds). [2025-10-20T21:15:50.651441] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-07. [2025-10-20T21:15:51.451357] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-08. [2025-10-20T21:15:51.655137] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-07 (1.00 seconds). [2025-10-20T21:15:52.251476] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-08. [2025-10-20T21:15:53.051413] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-06. [2025-10-20T21:15:53.247092] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T21:15:53.672008] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-08 (2.22 seconds). [2025-10-20T21:15:53.851407] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-09. [2025-10-20T21:15:54.085598] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T21:15:54.651438] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-04. [2025-10-20T21:15:55.110049] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-04 (0.46 seconds). [2025-10-20T21:15:55.451392] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-09. [2025-10-20T21:15:55.912382] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-09 (2.06 seconds). [2025-10-20T21:15:56.251343] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-07. [2025-10-20T21:15:56.482306] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T21:15:57.051356] [INFO ] [REMOTE ] SRFM: Fetching primary data for 2025-10. [2025-10-20T21:15:57.772830] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:15:57.851372] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-08. [2025-10-20T21:15:58.651378] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-05. [2025-10-20T21:15:59.079094] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T21:15:59.167512] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-05 (0.52 seconds). [2025-10-20T21:15:59.190882] [INFO ] [REMOTE ] SRFM: Incremental fetch complete for 2025-10 (2.14 seconds). [2025-10-20T21:15:59.198335] [INFO ] [REMOTE ] SRFM: Full fetch complete (50977 records). [2025-10-20T21:15:59.198418] [DEBUG] [VALIDATOR ] SRFM: Validating primary data. [2025-10-20T21:15:59.456532] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-09. [2025-10-20T21:15:59.582199] [DEBUG] [VALIDATOR ] SRFM: All expected trading days are present. [2025-10-20T21:15:59.582278] [DEBUG] [VALIDATOR ] SRFM: Primary data validation passed. [2025-10-20T21:15:59.583373] [INFO ] [MAIN ] SRFM: Fetched primary data is valid. [2025-10-20T21:15:59.583434] [DEBUG] [LOCAL ] SRFM: Saving primary data. [2025-10-20T21:15:59.584430] [DEBUG] [LOCAL ] SRFM: Trimmed 50977 records to 39590 (keeping 200 days of data) [2025-10-20T21:15:59.797825] [DEBUG] [LOCAL ] SRFM: Primary data saved (39590 records). [2025-10-20T21:15:59.797978] [INFO ] [MAIN ] SRFM: Full history saved. [2025-10-20T21:15:59.798250] [INFO ] [MAIN ] Update Progress: 1987/2367 symbols completed. SRFM took 38.1s. Estimated remaining: 48m 57s. Rate: 7.8 symbols/min [2025-10-20T21:15:59.963144] [INFO ] [MAIN ] SRV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:15:59.973544] [DEBUG] [LOCAL ] SRV: Loaded 7537 existing records from primary storage. [2025-10-20T21:15:59.973785] [INFO ] [MAIN ] SRV: Loaded 7537 existing records. [2025-10-20T21:16:00.150377] [WARN ] [MAIN ] SRV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:00.150532] [INFO ] [MAIN ] SRV: Wiping existing primary data before full refresh. [2025-10-20T21:16:00.150577] [INFO ] [LOCAL ] SRV: Starting deletion of primary data. [2025-10-20T21:16:00.150764] [DEBUG] [LOCAL ] SRV: Deleted primary data file. [2025-10-20T21:16:00.150819] [INFO ] [MAIN ] SRV: Starting full primary refresh. [2025-10-20T21:16:00.256544] [INFO ] [REMOTE ] SRG: Fetching primary data for 2025-10. [2025-10-20T21:16:00.510448] [INFO ] [REMOTE ] SRV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:00.613183] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T21:16:01.051449] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-01. [2025-10-20T21:16:01.229865] [INFO ] [REMOTE ] SRG: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T21:16:01.232144] [INFO ] [REMOTE ] SRG: Full fetch complete (23257 records). [2025-10-20T21:16:01.232214] [DEBUG] [VALIDATOR ] SRG: Validating primary data. [2025-10-20T21:16:01.605567] [DEBUG] [VALIDATOR ] SRG: All expected trading days are present. [2025-10-20T21:16:01.605691] [DEBUG] [VALIDATOR ] SRG: Primary data validation passed. [2025-10-20T21:16:01.606342] [INFO ] [MAIN ] SRG: Fetched primary data is valid. [2025-10-20T21:16:01.606408] [DEBUG] [LOCAL ] SRG: Saving primary data. [2025-10-20T21:16:01.607234] [DEBUG] [LOCAL ] SRG: Trimmed 23257 records to 14970 (keeping 200 days of data) [2025-10-20T21:16:01.690484] [DEBUG] [LOCAL ] SRG: Primary data saved (14970 records). [2025-10-20T21:16:01.690627] [INFO ] [MAIN ] SRG: Full history saved. [2025-10-20T21:16:01.690746] [INFO ] [MAIN ] Update Progress: 1988/2367 symbols completed. SRG took 34.1s. Estimated remaining: 48m 48s. Rate: 7.8 symbols/min [2025-10-20T21:16:01.851634] [INFO ] [REMOTE ] SRI: Fetching primary data for 2025-10. [2025-10-20T21:16:01.873347] [INFO ] [MAIN ] SSB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:01.914168] [DEBUG] [LOCAL ] SSB: Loaded 43425 existing records from primary storage. [2025-10-20T21:16:01.914429] [INFO ] [MAIN ] SSB: Loaded 43425 existing records. [2025-10-20T21:16:02.037489] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-01 (0.99 seconds). [2025-10-20T21:16:02.114859] [WARN ] [MAIN ] SSB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:02.115109] [INFO ] [MAIN ] SSB: Wiping existing primary data before full refresh. [2025-10-20T21:16:02.115160] [INFO ] [LOCAL ] SSB: Starting deletion of primary data. [2025-10-20T21:16:02.115391] [DEBUG] [LOCAL ] SSB: Deleted primary data file. [2025-10-20T21:16:02.115451] [INFO ] [MAIN ] SSB: Starting full primary refresh. [2025-10-20T21:16:02.523627] [INFO ] [REMOTE ] SSB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:02.651379] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-01. [2025-10-20T21:16:02.997021] [INFO ] [REMOTE ] SRI: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T21:16:03.000162] [INFO ] [REMOTE ] SRI: Full fetch complete (39764 records). [2025-10-20T21:16:03.000236] [DEBUG] [VALIDATOR ] SRI: Validating primary data. [2025-10-20T21:16:03.371000] [DEBUG] [VALIDATOR ] SRI: All expected trading days are present. [2025-10-20T21:16:03.371078] [DEBUG] [VALIDATOR ] SRI: Primary data validation passed. [2025-10-20T21:16:03.371884] [INFO ] [MAIN ] SRI: Fetched primary data is valid. [2025-10-20T21:16:03.371945] [DEBUG] [LOCAL ] SRI: Saving primary data. [2025-10-20T21:16:03.372762] [DEBUG] [LOCAL ] SRI: Trimmed 39764 records to 26431 (keeping 200 days of data) [2025-10-20T21:16:03.451435] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-02. [2025-10-20T21:16:03.525609] [DEBUG] [LOCAL ] SRI: Primary data saved (26431 records). [2025-10-20T21:16:03.525781] [INFO ] [MAIN ] SRI: Full history saved. [2025-10-20T21:16:03.525868] [INFO ] [MAIN ] Update Progress: 1989/2367 symbols completed. SRI took 28.9s. Estimated remaining: 48m 39s. Rate: 7.8 symbols/min [2025-10-20T21:16:03.694148] [INFO ] [MAIN ] SSD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:03.717382] [DEBUG] [LOCAL ] SSD: Loaded 22917 existing records from primary storage. [2025-10-20T21:16:03.717637] [INFO ] [MAIN ] SSD: Loaded 22917 existing records. [2025-10-20T21:16:03.895273] [WARN ] [MAIN ] SSD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:03.895464] [INFO ] [MAIN ] SSD: Wiping existing primary data before full refresh. [2025-10-20T21:16:03.895511] [INFO ] [LOCAL ] SSD: Starting deletion of primary data. [2025-10-20T21:16:03.895699] [DEBUG] [LOCAL ] SSD: Deleted primary data file. [2025-10-20T21:16:03.895773] [INFO ] [MAIN ] SSD: Starting full primary refresh. [2025-10-20T21:16:04.209725] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T21:16:04.257749] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-06. [2025-10-20T21:16:04.289058] [INFO ] [REMOTE ] SSD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:04.375717] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T21:16:04.651754] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-06 (0.39 seconds). [2025-10-20T21:16:05.051358] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-03. [2025-10-20T21:16:05.851397] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-01. [2025-10-20T21:16:05.899424] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T21:16:06.651365] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-04. [2025-10-20T21:16:07.072843] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-01 (1.22 seconds). [2025-10-20T21:16:07.451356] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-07. [2025-10-20T21:16:07.676091] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T21:16:07.851849] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T21:16:08.251379] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-02. [2025-10-20T21:16:09.051376] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-02. [2025-10-20T21:16:09.764460] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T21:16:09.851369] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-03. [2025-10-20T21:16:10.343351] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-02 (1.29 seconds). [2025-10-20T21:16:10.651364] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-05. [2025-10-20T21:16:11.456501] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-08. [2025-10-20T21:16:11.470164] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T21:16:11.505152] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T21:16:11.903752] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-08 (0.45 seconds). [2025-10-20T21:16:12.251418] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-03. [2025-10-20T21:16:13.051436] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-09. [2025-10-20T21:16:13.464740] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T21:16:13.480920] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T21:16:13.851355] [INFO ] [REMOTE ] SRL: Fetching primary data for 2025-10. [2025-10-20T21:16:14.286618] [INFO ] [REMOTE ] SRL: Incremental fetch complete for 2025-10 (0.44 seconds). [2025-10-20T21:16:14.287418] [INFO ] [REMOTE ] SRL: Full fetch complete (1680 records). [2025-10-20T21:16:14.287484] [DEBUG] [VALIDATOR ] SRL: Validating primary data. [2025-10-20T21:16:14.462869] [WARN ] [VALIDATOR ] SRL: Validation failed - insufficient historical data. Have 196 unique days, need 200 days [2025-10-20T21:16:14.463086] [FATAL] [MAIN ] SRL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:16:14.463134] [WARN ] [MAIN ] SRL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:16:14.463173] [INFO ] [LOCAL ] SRL: Starting deletion of primary data. [2025-10-20T21:16:14.463229] [DEBUG] [LOCAL ] SRL: No primary data file found to delete. [2025-10-20T21:16:14.463275] [INFO ] [LOCAL ] SRL: Starting deletion of derivative data. [2025-10-20T21:16:14.463313] [DEBUG] [LOCAL ] SRL: No derivative data file found to delete. [2025-10-20T21:16:14.463409] [INFO ] [MAIN ] Update Progress: 1990/2367 symbols completed. SRL took 29.0s. Estimated remaining: 48m 32s. Rate: 7.8 symbols/min [2025-10-20T21:16:14.631425] [INFO ] [MAIN ] SSL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:14.651354] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-06. [2025-10-20T21:16:14.669904] [DEBUG] [LOCAL ] SSL: Loaded 42084 existing records from primary storage. [2025-10-20T21:16:14.670168] [INFO ] [MAIN ] SSL: Loaded 42084 existing records. [2025-10-20T21:16:14.843797] [WARN ] [MAIN ] SSL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:14.843942] [INFO ] [MAIN ] SSL: Wiping existing primary data before full refresh. [2025-10-20T21:16:14.843988] [INFO ] [LOCAL ] SSL: Starting deletion of primary data. [2025-10-20T21:16:14.844150] [DEBUG] [LOCAL ] SSL: Deleted primary data file. [2025-10-20T21:16:14.844187] [INFO ] [MAIN ] SSL: Starting full primary refresh. [2025-10-20T21:16:15.189936] [INFO ] [REMOTE ] SSL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:15.451369] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-01. [2025-10-20T21:16:15.597942] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T21:16:16.251425] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-04. [2025-10-20T21:16:17.051410] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-04. [2025-10-20T21:16:17.380864] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T21:16:17.437599] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T21:16:17.851341] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-05. [2025-10-20T21:16:18.651387] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-07. [2025-10-20T21:16:18.834419] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:16:19.033963] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T21:16:19.167283] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T21:16:19.451425] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-02. [2025-10-20T21:16:20.251391] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-08. [2025-10-20T21:16:21.009519] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T21:16:21.051338] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-05. [2025-10-20T21:16:21.098980] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:16:21.851349] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-03. [2025-10-20T21:16:22.651361] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-09. [2025-10-20T21:16:22.725110] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T21:16:23.148068] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-09 (0.50 seconds). [2025-10-20T21:16:23.386260] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T21:16:23.451335] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-04. [2025-10-20T21:16:24.251370] [INFO ] [REMOTE ] SRV: Fetching primary data for 2025-10. [2025-10-20T21:16:24.988474] [INFO ] [REMOTE ] SRV: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:16:24.990069] [INFO ] [REMOTE ] SRV: Full fetch complete (11355 records). [2025-10-20T21:16:24.990136] [DEBUG] [VALIDATOR ] SRV: Validating primary data. [2025-10-20T21:16:25.053572] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-06. [2025-10-20T21:16:25.364003] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T21:16:25.398811] [DEBUG] [VALIDATOR ] SRV: All expected trading days are present. [2025-10-20T21:16:25.398900] [DEBUG] [VALIDATOR ] SRV: Primary data validation passed. [2025-10-20T21:16:25.399429] [INFO ] [MAIN ] SRV: Fetched primary data is valid. [2025-10-20T21:16:25.399486] [DEBUG] [LOCAL ] SRV: Saving primary data. [2025-10-20T21:16:25.399961] [DEBUG] [LOCAL ] SRV: Trimmed 11355 records to 7460 (keeping 200 days of data) [2025-10-20T21:16:25.443496] [DEBUG] [LOCAL ] SRV: Primary data saved (7460 records). [2025-10-20T21:16:25.443591] [INFO ] [MAIN ] SRV: Full history saved. [2025-10-20T21:16:25.443687] [INFO ] [MAIN ] Update Progress: 1991/2367 symbols completed. SRV took 25.6s. Estimated remaining: 48m 25s. Rate: 7.8 symbols/min [2025-10-20T21:16:25.631142] [INFO ] [MAIN ] SST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:25.631300] [DEBUG] [LOCAL ] SST: No existing data file found in primary storage. [2025-10-20T21:16:25.631336] [WARN ] [MAIN ] SST: No existing data found, performing full refresh instead. [2025-10-20T21:16:25.631470] [INFO ] [MAIN ] SST: Wiping existing primary data before full refresh. [2025-10-20T21:16:25.631511] [INFO ] [LOCAL ] SST: Starting deletion of primary data. [2025-10-20T21:16:25.631551] [DEBUG] [LOCAL ] SST: No primary data file found to delete. [2025-10-20T21:16:25.631581] [INFO ] [MAIN ] SST: Starting full primary refresh. [2025-10-20T21:16:25.858494] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-06. [2025-10-20T21:16:25.981354] [INFO ] [REMOTE ] SST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:26.651381] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-01. [2025-10-20T21:16:26.771263] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T21:16:27.069508] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T21:16:27.451424] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-05. [2025-10-20T21:16:27.861062] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T21:16:28.251394] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-07. [2025-10-20T21:16:29.051410] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-02. [2025-10-20T21:16:29.147581] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T21:16:29.851383] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-06. [2025-10-20T21:16:29.899878] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T21:16:29.943179] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T21:16:30.651381] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-03. [2025-10-20T21:16:31.451385] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-08. [2025-10-20T21:16:31.539001] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T21:16:31.623412] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T21:16:32.251403] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-07. [2025-10-20T21:16:32.955722] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T21:16:33.051342] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-09. [2025-10-20T21:16:33.517835] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T21:16:33.851367] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-08. [2025-10-20T21:16:34.533757] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T21:16:34.651343] [INFO ] [REMOTE ] SSB: Fetching primary data for 2025-10. [2025-10-20T21:16:35.063012] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-08 (1.21 seconds). [2025-10-20T21:16:35.327941] [INFO ] [REMOTE ] SSB: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T21:16:35.333747] [INFO ] [REMOTE ] SSB: Full fetch complete (61319 records). [2025-10-20T21:16:35.333927] [DEBUG] [VALIDATOR ] SSB: Validating primary data. [2025-10-20T21:16:35.334617] [FATAL] [VALIDATOR ] SSB: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:16:35.334717] [FATAL] [MAIN ] SSB: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:16:35.334759] [WARN ] [MAIN ] SSB: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:16:35.334833] [INFO ] [LOCAL ] SSB: Starting deletion of primary data. [2025-10-20T21:16:35.334909] [DEBUG] [LOCAL ] SSB: No primary data file found to delete. [2025-10-20T21:16:35.334963] [INFO ] [LOCAL ] SSB: Starting deletion of derivative data. [2025-10-20T21:16:35.335351] [DEBUG] [LOCAL ] SSB: Deleted derivative data file. [2025-10-20T21:16:35.335438] [INFO ] [MAIN ] Update Progress: 1992/2367 symbols completed. SSB took 33.6s. Estimated remaining: 48m 17s. Rate: 7.8 symbols/min [2025-10-20T21:16:35.456569] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-07. [2025-10-20T21:16:35.512317] [INFO ] [MAIN ] SSTK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:35.537828] [DEBUG] [LOCAL ] SSTK: Loaded 25830 existing records from primary storage. [2025-10-20T21:16:35.538147] [INFO ] [MAIN ] SSTK: Loaded 25830 existing records. [2025-10-20T21:16:35.711997] [WARN ] [MAIN ] SSTK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:35.712229] [INFO ] [MAIN ] SSTK: Wiping existing primary data before full refresh. [2025-10-20T21:16:35.712285] [INFO ] [LOCAL ] SSTK: Starting deletion of primary data. [2025-10-20T21:16:35.712477] [DEBUG] [LOCAL ] SSTK: Deleted primary data file. [2025-10-20T21:16:35.712514] [INFO ] [MAIN ] SSTK: Starting full primary refresh. [2025-10-20T21:16:36.056766] [INFO ] [REMOTE ] SSTK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:36.251366] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-09. [2025-10-20T21:16:37.051369] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-04. [2025-10-20T21:16:37.166269] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T21:16:37.431302] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T21:16:37.851414] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-01. [2025-10-20T21:16:37.975259] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T21:16:38.651369] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-08. [2025-10-20T21:16:39.456854] [INFO ] [REMOTE ] SSD: Fetching primary data for 2025-10. [2025-10-20T21:16:39.471608] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T21:16:40.055461] [INFO ] [REMOTE ] SSD: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T21:16:40.058780] [INFO ] [REMOTE ] SSD: Full fetch complete (32327 records). [2025-10-20T21:16:40.058859] [DEBUG] [VALIDATOR ] SSD: Validating primary data. [2025-10-20T21:16:40.059306] [FATAL] [VALIDATOR ] SSD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:16:40.059391] [FATAL] [MAIN ] SSD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:16:40.059435] [WARN ] [MAIN ] SSD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:16:40.059472] [INFO ] [LOCAL ] SSD: Starting deletion of primary data. [2025-10-20T21:16:40.059530] [DEBUG] [LOCAL ] SSD: No primary data file found to delete. [2025-10-20T21:16:40.059566] [INFO ] [LOCAL ] SSD: Starting deletion of derivative data. [2025-10-20T21:16:40.059765] [DEBUG] [LOCAL ] SSD: Deleted derivative data file. [2025-10-20T21:16:40.059865] [INFO ] [MAIN ] Update Progress: 1993/2367 symbols completed. SSD took 36.5s. Estimated remaining: 48m 9s. Rate: 7.8 symbols/min [2025-10-20T21:16:40.230492] [INFO ] [MAIN ] ST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:40.257063] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-05. [2025-10-20T21:16:40.279130] [DEBUG] [LOCAL ] ST: Loaded 50556 existing records from primary storage. [2025-10-20T21:16:40.279431] [INFO ] [MAIN ] ST: Loaded 50556 existing records. [2025-10-20T21:16:40.492179] [WARN ] [MAIN ] ST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:40.493426] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T21:16:40.502393] [INFO ] [MAIN ] ST: Wiping existing primary data before full refresh. [2025-10-20T21:16:40.505713] [INFO ] [LOCAL ] ST: Starting deletion of primary data. [2025-10-20T21:16:40.506237] [DEBUG] [LOCAL ] ST: Deleted primary data file. [2025-10-20T21:16:40.506423] [INFO ] [MAIN ] ST: Starting full primary refresh. [2025-10-20T21:16:40.875612] [INFO ] [REMOTE ] ST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:41.051351] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-09. [2025-10-20T21:16:41.197013] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-05 (0.94 seconds). [2025-10-20T21:16:41.851359] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-02. [2025-10-20T21:16:42.651423] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-06. [2025-10-20T21:16:42.788550] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T21:16:42.872955] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T21:16:43.451383] [INFO ] [REMOTE ] SSL: Fetching primary data for 2025-10. [2025-10-20T21:16:43.859075] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T21:16:44.251443] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-01. [2025-10-20T21:16:45.010540] [INFO ] [REMOTE ] SSL: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T21:16:45.017170] [INFO ] [REMOTE ] SSL: Full fetch complete (60086 records). [2025-10-20T21:16:45.017344] [DEBUG] [VALIDATOR ] SSL: Validating primary data. [2025-10-20T21:16:45.056485] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-07. [2025-10-20T21:16:45.412309] [DEBUG] [VALIDATOR ] SSL: All expected trading days are present. [2025-10-20T21:16:45.412385] [DEBUG] [VALIDATOR ] SSL: Primary data validation passed. [2025-10-20T21:16:45.413434] [INFO ] [MAIN ] SSL: Fetched primary data is valid. [2025-10-20T21:16:45.413497] [DEBUG] [LOCAL ] SSL: Saving primary data. [2025-10-20T21:16:45.414624] [DEBUG] [LOCAL ] SSL: Trimmed 60086 records to 41522 (keeping 200 days of data) [2025-10-20T21:16:45.644634] [DEBUG] [LOCAL ] SSL: Primary data saved (41522 records). [2025-10-20T21:16:45.644783] [INFO ] [MAIN ] SSL: Full history saved. [2025-10-20T21:16:45.644858] [INFO ] [MAIN ] Update Progress: 1994/2367 symbols completed. SSL took 31.2s. Estimated remaining: 48m 1s. Rate: 7.8 symbols/min [2025-10-20T21:16:45.811332] [INFO ] [MAIN ] STAG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:45.857987] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-03. [2025-10-20T21:16:45.860740] [DEBUG] [LOCAL ] STAG: Loaded 50883 existing records from primary storage. [2025-10-20T21:16:45.861028] [INFO ] [MAIN ] STAG: Loaded 50883 existing records. [2025-10-20T21:16:46.042524] [WARN ] [MAIN ] STAG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:46.042782] [INFO ] [MAIN ] STAG: Wiping existing primary data before full refresh. [2025-10-20T21:16:46.042849] [INFO ] [LOCAL ] STAG: Starting deletion of primary data. [2025-10-20T21:16:46.043378] [DEBUG] [LOCAL ] STAG: Deleted primary data file. [2025-10-20T21:16:46.043436] [INFO ] [MAIN ] STAG: Starting full primary refresh. [2025-10-20T21:16:46.171887] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T21:16:46.316814] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T21:16:46.456770] [INFO ] [REMOTE ] STAG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:46.651378] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-02. [2025-10-20T21:16:47.451358] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-01. [2025-10-20T21:16:47.585466] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T21:16:48.251429] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-04. [2025-10-20T21:16:48.339373] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T21:16:49.051357] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-08. [2025-10-20T21:16:49.190828] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-01 (1.74 seconds). [2025-10-20T21:16:49.373052] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:16:49.590225] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T21:16:49.851374] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-05. [2025-10-20T21:16:50.651356] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-09. [2025-10-20T21:16:51.036292] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T21:16:51.135474] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:16:51.451411] [INFO ] [REMOTE ] SST: Fetching primary data for 2025-10. [2025-10-20T21:16:52.128550] [INFO ] [REMOTE ] SST: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T21:16:52.129899] [INFO ] [REMOTE ] SST: Full fetch complete (18063 records). [2025-10-20T21:16:52.129965] [DEBUG] [VALIDATOR ] SST: Validating primary data. [2025-10-20T21:16:52.266636] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-03. [2025-10-20T21:16:52.489377] [DEBUG] [VALIDATOR ] SST: All expected trading days are present. [2025-10-20T21:16:52.489467] [DEBUG] [VALIDATOR ] SST: Primary data validation passed. [2025-10-20T21:16:52.489819] [INFO ] [MAIN ] SST: Fetched primary data is valid. [2025-10-20T21:16:52.489878] [DEBUG] [LOCAL ] SST: Saving primary data. [2025-10-20T21:16:52.490389] [DEBUG] [LOCAL ] SST: Trimmed 18063 records to 11489 (keeping 200 days of data) [2025-10-20T21:16:52.552456] [DEBUG] [LOCAL ] SST: Primary data saved (11489 records). [2025-10-20T21:16:52.552571] [INFO ] [MAIN ] SST: Full history saved. [2025-10-20T21:16:52.552639] [INFO ] [MAIN ] Update Progress: 1995/2367 symbols completed. SST took 27.1s. Estimated remaining: 47m 53s. Rate: 7.8 symbols/min [2025-10-20T21:16:52.716675] [INFO ] [MAIN ] STC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:16:52.734090] [DEBUG] [LOCAL ] STC: Loaded 17534 existing records from primary storage. [2025-10-20T21:16:52.734296] [INFO ] [MAIN ] STC: Loaded 17534 existing records. [2025-10-20T21:16:52.906046] [WARN ] [MAIN ] STC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:16:52.906216] [INFO ] [MAIN ] STC: Wiping existing primary data before full refresh. [2025-10-20T21:16:52.906260] [INFO ] [LOCAL ] STC: Starting deletion of primary data. [2025-10-20T21:16:52.906437] [DEBUG] [LOCAL ] STC: Deleted primary data file. [2025-10-20T21:16:52.906504] [INFO ] [MAIN ] STC: Starting full primary refresh. [2025-10-20T21:16:53.053283] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-06. [2025-10-20T21:16:53.256602] [INFO ] [REMOTE ] STC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:16:53.851368] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-01. [2025-10-20T21:16:54.125928] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T21:16:54.350111] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T21:16:54.651386] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-04. [2025-10-20T21:16:54.912124] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T21:16:55.451374] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-02. [2025-10-20T21:16:56.251370] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-07. [2025-10-20T21:16:56.426954] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:16:56.569945] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T21:16:57.051375] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-03. [2025-10-20T21:16:57.542216] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-07 (1.29 seconds). [2025-10-20T21:16:57.851368] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-08. [2025-10-20T21:16:58.103905] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T21:16:58.651375] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-02. [2025-10-20T21:16:59.186466] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T21:16:59.451384] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-05. [2025-10-20T21:17:00.251382] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-09. [2025-10-20T21:17:00.325129] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:17:00.851224] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T21:17:01.051382] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-04. [2025-10-20T21:17:01.441257] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T21:17:01.851431] [INFO ] [REMOTE ] SSTK: Fetching primary data for 2025-10. [2025-10-20T21:17:02.229249] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T21:17:02.517097] [INFO ] [REMOTE ] SSTK: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T21:17:02.522823] [INFO ] [REMOTE ] SSTK: Full fetch complete (41461 records). [2025-10-20T21:17:02.522980] [DEBUG] [VALIDATOR ] SSTK: Validating primary data. [2025-10-20T21:17:02.523743] [FATAL] [VALIDATOR ] SSTK: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:02.523841] [FATAL] [MAIN ] SSTK: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:02.523895] [WARN ] [MAIN ] SSTK: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:02.523974] [INFO ] [LOCAL ] SSTK: Starting deletion of primary data. [2025-10-20T21:17:02.524054] [DEBUG] [LOCAL ] SSTK: No primary data file found to delete. [2025-10-20T21:17:02.524103] [INFO ] [LOCAL ] SSTK: Starting deletion of derivative data. [2025-10-20T21:17:02.524304] [DEBUG] [LOCAL ] SSTK: Deleted derivative data file. [2025-10-20T21:17:02.524414] [INFO ] [MAIN ] Update Progress: 1996/2367 symbols completed. SSTK took 27.2s. Estimated remaining: 47m 46s. Rate: 7.8 symbols/min [2025-10-20T21:17:02.656494] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-06. [2025-10-20T21:17:02.702057] [INFO ] [MAIN ] STE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:02.735760] [DEBUG] [LOCAL ] STE: Loaded 35901 existing records from primary storage. [2025-10-20T21:17:02.736027] [INFO ] [MAIN ] STE: Loaded 35901 existing records. [2025-10-20T21:17:02.909738] [WARN ] [MAIN ] STE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:02.909945] [INFO ] [MAIN ] STE: Wiping existing primary data before full refresh. [2025-10-20T21:17:02.909992] [INFO ] [LOCAL ] STE: Starting deletion of primary data. [2025-10-20T21:17:02.910191] [DEBUG] [LOCAL ] STE: Deleted primary data file. [2025-10-20T21:17:02.910272] [INFO ] [MAIN ] STE: Starting full primary refresh. [2025-10-20T21:17:03.270507] [INFO ] [REMOTE ] STE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:03.451370] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-01. [2025-10-20T21:17:04.251487] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-03. [2025-10-20T21:17:04.459349] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T21:17:05.051355] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-07. [2025-10-20T21:17:05.141185] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T21:17:05.851414] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-02. [2025-10-20T21:17:06.080200] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:17:06.651430] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-04. [2025-10-20T21:17:06.697108] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T21:17:07.306450] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T21:17:07.451427] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-03. [2025-10-20T21:17:08.251376] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-08. [2025-10-20T21:17:08.496388] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T21:17:09.053450] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-05. [2025-10-20T21:17:09.056064] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T21:17:09.851418] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-04. [2025-10-20T21:17:09.926897] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T21:17:10.651422] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-09. [2025-10-20T21:17:10.884678] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T21:17:11.451431] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-06. [2025-10-20T21:17:11.540330] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T21:17:11.994223] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T21:17:12.251381] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-05. [2025-10-20T21:17:13.051416] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-05. [2025-10-20T21:17:13.360005] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T21:17:13.609263] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T21:17:13.851351] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-06. [2025-10-20T21:17:14.209026] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T21:17:14.651365] [INFO ] [REMOTE ] ST: Fetching primary data for 2025-10. [2025-10-20T21:17:15.140136] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T21:17:15.442921] [INFO ] [REMOTE ] ST: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:17:15.449904] [INFO ] [REMOTE ] ST: Full fetch complete (71819 records). [2025-10-20T21:17:15.450046] [DEBUG] [VALIDATOR ] ST: Validating primary data. [2025-10-20T21:17:15.450735] [FATAL] [VALIDATOR ] ST: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:15.450822] [FATAL] [MAIN ] ST: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:15.450867] [WARN ] [MAIN ] ST: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:15.450935] [INFO ] [LOCAL ] ST: Starting deletion of primary data. [2025-10-20T21:17:15.450999] [DEBUG] [LOCAL ] ST: No primary data file found to delete. [2025-10-20T21:17:15.451033] [INFO ] [LOCAL ] ST: Starting deletion of derivative data. [2025-10-20T21:17:15.451151] [DEBUG] [LOCAL ] ST: Deleted derivative data file. [2025-10-20T21:17:15.451224] [INFO ] [MAIN ] Update Progress: 1997/2367 symbols completed. ST took 35.4s. Estimated remaining: 47m 39s. Rate: 7.8 symbols/min [2025-10-20T21:17:15.451332] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-06. [2025-10-20T21:17:16.251367] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-07. [2025-10-20T21:17:16.622258] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T21:17:17.051372] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-07. [2025-10-20T21:17:17.851391] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-07. [2025-10-20T21:17:18.098833] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T21:17:18.157163] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T21:17:18.651390] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-08. [2025-10-20T21:17:19.401265] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T21:17:19.451346] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-08. [2025-10-20T21:17:19.716970] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:17:20.251416] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-08. [2025-10-20T21:17:20.617998] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-08 (1.17 seconds). [2025-10-20T21:17:21.051424] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-09. [2025-10-20T21:17:21.851439] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-09. [2025-10-20T21:17:21.952423] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T21:17:22.565004] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T21:17:22.651393] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-09. [2025-10-20T21:17:22.677208] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:17:23.451477] [INFO ] [REMOTE ] STC: Fetching primary data for 2025-10. [2025-10-20T21:17:24.016313] [INFO ] [REMOTE ] STC: Incremental fetch complete for 2025-10 (0.56 seconds). [2025-10-20T21:17:24.018751] [INFO ] [REMOTE ] STC: Full fetch complete (25436 records). [2025-10-20T21:17:24.018827] [DEBUG] [VALIDATOR ] STC: Validating primary data. [2025-10-20T21:17:24.019296] [FATAL] [VALIDATOR ] STC: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:24.019388] [FATAL] [MAIN ] STC: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:24.019460] [WARN ] [MAIN ] STC: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:24.019545] [INFO ] [LOCAL ] STC: Starting deletion of primary data. [2025-10-20T21:17:24.019621] [DEBUG] [LOCAL ] STC: No primary data file found to delete. [2025-10-20T21:17:24.019697] [INFO ] [LOCAL ] STC: Starting deletion of derivative data. [2025-10-20T21:17:24.019851] [DEBUG] [LOCAL ] STC: Deleted derivative data file. [2025-10-20T21:17:24.019950] [INFO ] [MAIN ] Update Progress: 1998/2367 symbols completed. STC took 31.5s. Estimated remaining: 47m 31s. Rate: 7.8 symbols/min [2025-10-20T21:17:24.251449] [INFO ] [REMOTE ] STE: Fetching primary data for 2025-10. [2025-10-20T21:17:24.449629] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T21:17:24.932028] [INFO ] [REMOTE ] STE: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T21:17:24.937102] [INFO ] [REMOTE ] STE: Full fetch complete (50789 records). [2025-10-20T21:17:24.937227] [DEBUG] [VALIDATOR ] STE: Validating primary data. [2025-10-20T21:17:24.937899] [FATAL] [VALIDATOR ] STE: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:24.937986] [FATAL] [MAIN ] STE: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:24.938028] [WARN ] [MAIN ] STE: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:24.938063] [INFO ] [LOCAL ] STE: Starting deletion of primary data. [2025-10-20T21:17:24.938121] [DEBUG] [LOCAL ] STE: No primary data file found to delete. [2025-10-20T21:17:24.938154] [INFO ] [LOCAL ] STE: Starting deletion of derivative data. [2025-10-20T21:17:24.938318] [DEBUG] [LOCAL ] STE: Deleted derivative data file. [2025-10-20T21:17:24.938390] [INFO ] [MAIN ] Update Progress: 1999/2367 symbols completed. STE took 22.4s. Estimated remaining: 47m 22s. Rate: 7.8 symbols/min [2025-10-20T21:17:25.051378] [INFO ] [REMOTE ] STAG: Fetching primary data for 2025-10. [2025-10-20T21:17:25.930624] [INFO ] [REMOTE ] STAG: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T21:17:25.940909] [INFO ] [REMOTE ] STAG: Full fetch complete (72190 records). [2025-10-20T21:17:25.941114] [DEBUG] [VALIDATOR ] STAG: Validating primary data. [2025-10-20T21:17:25.942063] [FATAL] [VALIDATOR ] STAG: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:25.942178] [FATAL] [MAIN ] STAG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:25.942235] [WARN ] [MAIN ] STAG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:25.942311] [INFO ] [LOCAL ] STAG: Starting deletion of primary data. [2025-10-20T21:17:25.942397] [DEBUG] [LOCAL ] STAG: No primary data file found to delete. [2025-10-20T21:17:25.942462] [INFO ] [LOCAL ] STAG: Starting deletion of derivative data. [2025-10-20T21:17:25.942628] [DEBUG] [LOCAL ] STAG: Deleted derivative data file. [2025-10-20T21:17:25.942763] [INFO ] [MAIN ] Update Progress: 2000/2367 symbols completed. STAG took 40.3s. Estimated remaining: 47m 14s. Rate: 7.8 symbols/min [2025-10-20T21:17:26.585131] [INFO ] [MAIN ] STEW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:26.603212] [INFO ] [MAIN ] STEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:26.668717] [INFO ] [MAIN ] STEM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:26.693128] [INFO ] [MAIN ] STG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:26.693522] [DEBUG] [LOCAL ] STG: No existing data file found in primary storage. [2025-10-20T21:17:26.693568] [WARN ] [MAIN ] STG: No existing data found, performing full refresh instead. [2025-10-20T21:17:26.693697] [INFO ] [MAIN ] STG: Wiping existing primary data before full refresh. [2025-10-20T21:17:26.693759] [INFO ] [LOCAL ] STG: Starting deletion of primary data. [2025-10-20T21:17:26.693812] [DEBUG] [LOCAL ] STG: No primary data file found to delete. [2025-10-20T21:17:26.693943] [INFO ] [MAIN ] STG: Starting full primary refresh. [2025-10-20T21:17:26.769074] [DEBUG] [LOCAL ] STEW: Loaded 15838 existing records from primary storage. [2025-10-20T21:17:26.794165] [INFO ] [MAIN ] STEW: Loaded 15838 existing records. [2025-10-20T21:17:26.928878] [DEBUG] [LOCAL ] STEL: Loaded 22979 existing records from primary storage. [2025-10-20T21:17:27.018823] [INFO ] [MAIN ] STEL: Loaded 22979 existing records. [2025-10-20T21:17:27.217809] [WARN ] [MAIN ] STEW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:27.208562] [DEBUG] [LOCAL ] STEM: Loaded 39412 existing records from primary storage. [2025-10-20T21:17:27.258188] [INFO ] [MAIN ] STEW: Wiping existing primary data before full refresh. [2025-10-20T21:17:27.266910] [INFO ] [MAIN ] STEM: Loaded 39412 existing records. [2025-10-20T21:17:27.304197] [INFO ] [LOCAL ] STEW: Starting deletion of primary data. [2025-10-20T21:17:27.467518] [DEBUG] [LOCAL ] STEW: Deleted primary data file. [2025-10-20T21:17:27.474666] [INFO ] [MAIN ] STEW: Starting full primary refresh. [2025-10-20T21:17:27.631977] [INFO ] [REMOTE ] STG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:27.698282] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-01. [2025-10-20T21:17:27.714338] [WARN ] [MAIN ] STEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:27.748003] [INFO ] [MAIN ] STEL: Wiping existing primary data before full refresh. [2025-10-20T21:17:27.792962] [INFO ] [LOCAL ] STEL: Starting deletion of primary data. [2025-10-20T21:17:27.796501] [WARN ] [MAIN ] STEM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:27.807851] [INFO ] [MAIN ] STEM: Wiping existing primary data before full refresh. [2025-10-20T21:17:27.807618] [DEBUG] [LOCAL ] STEL: Deleted primary data file. [2025-10-20T21:17:27.818391] [INFO ] [MAIN ] STEL: Starting full primary refresh. [2025-10-20T21:17:27.828740] [INFO ] [LOCAL ] STEM: Starting deletion of primary data. [2025-10-20T21:17:27.857079] [DEBUG] [LOCAL ] STEM: Deleted primary data file. [2025-10-20T21:17:27.857226] [INFO ] [MAIN ] STEM: Starting full primary refresh. [2025-10-20T21:17:28.366715] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T21:17:28.416558] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-02. [2025-10-20T21:17:28.595779] [INFO ] [REMOTE ] STEW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:28.648095] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-01. [2025-10-20T21:17:28.833712] [INFO ] [REMOTE ] STEM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:28.849247] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-01. [2025-10-20T21:17:28.854082] [INFO ] [REMOTE ] STEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:28.966640] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T21:17:29.051371] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-03. [2025-10-20T21:17:29.620550] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T21:17:29.687312] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-03 (0.64 seconds). [2025-10-20T21:17:29.851385] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-04. [2025-10-20T21:17:30.329291] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T21:17:30.651362] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-01. [2025-10-20T21:17:31.343077] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-01 (2.49 seconds). [2025-10-20T21:17:31.451455] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-02. [2025-10-20T21:17:31.842625] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T21:17:32.251415] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-02. [2025-10-20T21:17:32.280706] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-02 (0.83 seconds). [2025-10-20T21:17:33.051395] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-03. [2025-10-20T21:17:33.833482] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T21:17:33.851337] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-02. [2025-10-20T21:17:34.117675] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T21:17:34.651358] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-03. [2025-10-20T21:17:34.970598] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T21:17:35.451402] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-03. [2025-10-20T21:17:36.251408] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-04. [2025-10-20T21:17:36.571398] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-03 (1.92 seconds). [2025-10-20T21:17:36.748021] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T21:17:37.051423] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-04. [2025-10-20T21:17:37.197172] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T21:17:37.851391] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-05. [2025-10-20T21:17:38.439146] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T21:17:38.651415] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-04. [2025-10-20T21:17:38.937580] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-05 (1.09 seconds). [2025-10-20T21:17:39.451439] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-06. [2025-10-20T21:17:40.251359] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-05. [2025-10-20T21:17:40.469349] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T21:17:40.549316] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T21:17:41.051395] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-05. [2025-10-20T21:17:41.386545] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T21:17:41.446824] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-05 (0.40 seconds). [2025-10-20T21:17:41.851413] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-05. [2025-10-20T21:17:42.651345] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-06. [2025-10-20T21:17:42.978328] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-06 (0.33 seconds). [2025-10-20T21:17:43.451474] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-07. [2025-10-20T21:17:43.767455] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T21:17:44.249508] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-07 (0.80 seconds). [2025-10-20T21:17:44.251877] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-08. [2025-10-20T21:17:44.776126] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T21:17:45.051389] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-07. [2025-10-20T21:17:45.851426] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-06. [2025-10-20T21:17:46.063502] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T21:17:46.651438] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-06. [2025-10-20T21:17:47.451346] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-09. [2025-10-20T21:17:47.618775] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:17:47.839284] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T21:17:48.019470] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T21:17:48.251434] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-07. [2025-10-20T21:17:49.051419] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-08. [2025-10-20T21:17:49.478709] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T21:17:49.851424] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-08. [2025-10-20T21:17:49.883170] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T21:17:50.651445] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-09. [2025-10-20T21:17:51.010870] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T21:17:51.451415] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-09. [2025-10-20T21:17:51.496987] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T21:17:52.251387] [INFO ] [REMOTE ] STEW: Fetching primary data for 2025-10. [2025-10-20T21:17:52.654588] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T21:17:53.051361] [INFO ] [REMOTE ] STEL: Fetching primary data for 2025-10. [2025-10-20T21:17:53.136115] [INFO ] [REMOTE ] STEW: Incremental fetch complete for 2025-10 (0.88 seconds). [2025-10-20T21:17:53.138488] [INFO ] [REMOTE ] STEW: Full fetch complete (21762 records). [2025-10-20T21:17:53.138581] [DEBUG] [VALIDATOR ] STEW: Validating primary data. [2025-10-20T21:17:53.526068] [DEBUG] [VALIDATOR ] STEW: All expected trading days are present. [2025-10-20T21:17:53.526176] [DEBUG] [VALIDATOR ] STEW: Primary data validation passed. [2025-10-20T21:17:53.526753] [INFO ] [MAIN ] STEW: Fetched primary data is valid. [2025-10-20T21:17:53.526818] [DEBUG] [LOCAL ] STEW: Saving primary data. [2025-10-20T21:17:53.527421] [DEBUG] [LOCAL ] STEW: Trimmed 21762 records to 15442 (keeping 200 days of data) [2025-10-20T21:17:53.629299] [INFO ] [REMOTE ] STEL: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T21:17:53.632303] [DEBUG] [LOCAL ] STEW: Primary data saved (15442 records). [2025-10-20T21:17:53.635579] [INFO ] [MAIN ] STEW: Full history saved. [2025-10-20T21:17:53.635713] [INFO ] [MAIN ] Update Progress: 2001/2367 symbols completed. STEW took 27.7s. Estimated remaining: 47m 9s. Rate: 7.8 symbols/min [2025-10-20T21:17:53.635858] [INFO ] [REMOTE ] STEL: Full fetch complete (32015 records). [2025-10-20T21:17:53.684161] [DEBUG] [VALIDATOR ] STEL: Validating primary data. [2025-10-20T21:17:53.747279] [FATAL] [VALIDATOR ] STEL: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:17:53.751405] [FATAL] [MAIN ] STEL: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:53.757181] [WARN ] [MAIN ] STEL: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:53.767563] [INFO ] [LOCAL ] STEL: Starting deletion of primary data. [2025-10-20T21:17:53.793098] [DEBUG] [LOCAL ] STEL: No primary data file found to delete. [2025-10-20T21:17:53.806594] [INFO ] [LOCAL ] STEL: Starting deletion of derivative data. [2025-10-20T21:17:53.824012] [INFO ] [MAIN ] STK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:53.824071] [DEBUG] [LOCAL ] STEL: Deleted derivative data file. [2025-10-20T21:17:53.824356] [INFO ] [MAIN ] Update Progress: 2002/2367 symbols completed. STEL took 27.9s. Estimated remaining: 47m 0s. Rate: 7.8 symbols/min [2025-10-20T21:17:53.856620] [INFO ] [REMOTE ] STG: Fetching primary data for 2025-10. [2025-10-20T21:17:53.944381] [DEBUG] [LOCAL ] STK: Loaded 9914 existing records from primary storage. [2025-10-20T21:17:53.960087] [INFO ] [MAIN ] STK: Loaded 9914 existing records. [2025-10-20T21:17:54.110442] [INFO ] [MAIN ] STLA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:54.191537] [WARN ] [MAIN ] STK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:54.192637] [INFO ] [MAIN ] STK: Wiping existing primary data before full refresh. [2025-10-20T21:17:54.192732] [INFO ] [LOCAL ] STK: Starting deletion of primary data. [2025-10-20T21:17:54.192891] [DEBUG] [LOCAL ] STK: Deleted primary data file. [2025-10-20T21:17:54.192930] [INFO ] [MAIN ] STK: Starting full primary refresh. [2025-10-20T21:17:54.349348] [INFO ] [REMOTE ] STG: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T21:17:54.364194] [INFO ] [REMOTE ] STG: Full fetch complete (2496 records). [2025-10-20T21:17:54.411526] [DEBUG] [VALIDATOR ] STG: Validating primary data. [2025-10-20T21:17:54.554241] [DEBUG] [LOCAL ] STLA: Loaded 54335 existing records from primary storage. [2025-10-20T21:17:54.591790] [INFO ] [MAIN ] STLA: Loaded 54335 existing records. [2025-10-20T21:17:54.669462] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-07. [2025-10-20T21:17:54.840343] [INFO ] [REMOTE ] STK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:54.920903] [WARN ] [VALIDATOR ] STG: Validation failed - insufficient historical data. Have 188 unique days, need 200 days [2025-10-20T21:17:54.941821] [FATAL] [MAIN ] STG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:17:54.965343] [WARN ] [MAIN ] STG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:17:54.993554] [INFO ] [LOCAL ] STG: Starting deletion of primary data. [2025-10-20T21:17:55.000375] [WARN ] [MAIN ] STLA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:55.000722] [DEBUG] [LOCAL ] STG: No primary data file found to delete. [2025-10-20T21:17:55.000918] [INFO ] [MAIN ] STLA: Wiping existing primary data before full refresh. [2025-10-20T21:17:55.001060] [INFO ] [LOCAL ] STG: Starting deletion of derivative data. [2025-10-20T21:17:55.001165] [DEBUG] [LOCAL ] STG: No derivative data file found to delete. [2025-10-20T21:17:55.001245] [INFO ] [LOCAL ] STLA: Starting deletion of primary data. [2025-10-20T21:17:55.001429] [INFO ] [MAIN ] Update Progress: 2003/2367 symbols completed. STG took 29.0s. Estimated remaining: 46m 51s. Rate: 7.8 symbols/min [2025-10-20T21:17:55.013305] [DEBUG] [LOCAL ] STLA: Deleted primary data file. [2025-10-20T21:17:55.065775] [INFO ] [MAIN ] STLA: Starting full primary refresh. [2025-10-20T21:17:55.330099] [INFO ] [MAIN ] STM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:17:55.456939] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-01. [2025-10-20T21:17:55.540417] [INFO ] [REMOTE ] STLA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:55.577897] [DEBUG] [LOCAL ] STM: Loaded 54352 existing records from primary storage. [2025-10-20T21:17:55.578197] [INFO ] [MAIN ] STM: Loaded 54352 existing records. [2025-10-20T21:17:55.755202] [WARN ] [MAIN ] STM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:17:55.755415] [INFO ] [MAIN ] STM: Wiping existing primary data before full refresh. [2025-10-20T21:17:55.755471] [INFO ] [LOCAL ] STM: Starting deletion of primary data. [2025-10-20T21:17:55.755708] [DEBUG] [LOCAL ] STM: Deleted primary data file. [2025-10-20T21:17:55.755765] [INFO ] [MAIN ] STM: Starting full primary refresh. [2025-10-20T21:17:56.122885] [INFO ] [REMOTE ] STM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:17:56.129350] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-01 (0.67 seconds). [2025-10-20T21:17:56.251325] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-01. [2025-10-20T21:17:56.517127] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T21:17:57.051358] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-02. [2025-10-20T21:17:57.851415] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-08. [2025-10-20T21:17:57.917422] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-02 (0.87 seconds). [2025-10-20T21:17:58.319255] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T21:17:58.651422] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-03. [2025-10-20T21:17:59.293556] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:17:59.451381] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-01. [2025-10-20T21:17:59.503503] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T21:18:00.251464] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-04. [2025-10-20T21:18:01.051384] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-02. [2025-10-20T21:18:01.355361] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T21:18:01.595300] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-01 (2.14 seconds). [2025-10-20T21:18:01.851412] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-09. [2025-10-20T21:18:02.651442] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-05. [2025-10-20T21:18:03.140764] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-10-20T21:18:03.209586] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T21:18:03.223300] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T21:18:03.451410] [INFO ] [REMOTE ] STEM: Fetching primary data for 2025-10. [2025-10-20T21:18:04.251409] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-06. [2025-10-20T21:18:05.051472] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-02. [2025-10-20T21:18:05.067521] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T21:18:05.176834] [INFO ] [REMOTE ] STEM: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T21:18:05.182571] [INFO ] [REMOTE ] STEM: Full fetch complete (61079 records). [2025-10-20T21:18:05.182724] [DEBUG] [VALIDATOR ] STEM: Validating primary data. [2025-10-20T21:18:05.583930] [DEBUG] [VALIDATOR ] STEM: All expected trading days are present. [2025-10-20T21:18:05.584020] [DEBUG] [VALIDATOR ] STEM: Primary data validation passed. [2025-10-20T21:18:05.584979] [INFO ] [MAIN ] STEM: Fetched primary data is valid. [2025-10-20T21:18:05.585044] [DEBUG] [LOCAL ] STEM: Saving primary data. [2025-10-20T21:18:05.586282] [DEBUG] [LOCAL ] STEM: Trimmed 61079 records to 38621 (keeping 200 days of data) [2025-10-20T21:18:05.808156] [DEBUG] [LOCAL ] STEM: Primary data saved (38621 records). [2025-10-20T21:18:05.808365] [INFO ] [MAIN ] STEM: Full history saved. [2025-10-20T21:18:05.808789] [INFO ] [MAIN ] Update Progress: 2004/2367 symbols completed. STEM took 39.9s. Estimated remaining: 46m 44s. Rate: 7.8 symbols/min [2025-10-20T21:18:05.856705] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-03. [2025-10-20T21:18:05.983936] [INFO ] [MAIN ] STN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:06.008204] [DEBUG] [LOCAL ] STN: Loaded 23104 existing records from primary storage. [2025-10-20T21:18:06.008486] [INFO ] [MAIN ] STN: Loaded 23104 existing records. [2025-10-20T21:18:06.211221] [WARN ] [MAIN ] STN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:06.211468] [INFO ] [MAIN ] STN: Wiping existing primary data before full refresh. [2025-10-20T21:18:06.211516] [INFO ] [LOCAL ] STN: Starting deletion of primary data. [2025-10-20T21:18:06.211748] [DEBUG] [LOCAL ] STN: Deleted primary data file. [2025-10-20T21:18:06.211805] [INFO ] [MAIN ] STN: Starting full primary refresh. [2025-10-20T21:18:06.571926] [INFO ] [REMOTE ] STN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:06.651340] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-01. [2025-10-20T21:18:07.115625] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T21:18:07.451401] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-07. [2025-10-20T21:18:07.690560] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:18:07.837085] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T21:18:08.251397] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-02. [2025-10-20T21:18:08.286241] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-07 (0.83 seconds). [2025-10-20T21:18:09.051388] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-04. [2025-10-20T21:18:09.494332] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T21:18:09.851345] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-08. [2025-10-20T21:18:10.651446] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-03. [2025-10-20T21:18:10.705268] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:18:11.451364] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-09. [2025-10-20T21:18:11.699941] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-04 (2.65 seconds). [2025-10-20T21:18:11.992744] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-03 (1.34 seconds). [2025-10-20T21:18:12.251405] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-04. [2025-10-20T21:18:12.302841] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T21:18:13.051373] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-05. [2025-10-20T21:18:13.322905] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T21:18:13.851411] [INFO ] [REMOTE ] STK: Fetching primary data for 2025-10. [2025-10-20T21:18:14.651458] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-03. [2025-10-20T21:18:14.766592] [INFO ] [REMOTE ] STK: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T21:18:14.768912] [INFO ] [REMOTE ] STK: Full fetch complete (14366 records). [2025-10-20T21:18:14.769011] [DEBUG] [VALIDATOR ] STK: Validating primary data. [2025-10-20T21:18:15.159761] [DEBUG] [VALIDATOR ] STK: All expected trading days are present. [2025-10-20T21:18:15.159869] [DEBUG] [VALIDATOR ] STK: Primary data validation passed. [2025-10-20T21:18:15.160395] [INFO ] [MAIN ] STK: Fetched primary data is valid. [2025-10-20T21:18:15.160591] [DEBUG] [LOCAL ] STK: Saving primary data. [2025-10-20T21:18:15.161381] [DEBUG] [LOCAL ] STK: Trimmed 14366 records to 9842 (keeping 200 days of data) [2025-10-20T21:18:15.228498] [DEBUG] [LOCAL ] STK: Primary data saved (9842 records). [2025-10-20T21:18:15.228743] [INFO ] [MAIN ] STK: Full history saved. [2025-10-20T21:18:15.228950] [INFO ] [MAIN ] Update Progress: 2005/2367 symbols completed. STK took 21.6s. Estimated remaining: 46m 37s. Rate: 7.8 symbols/min [2025-10-20T21:18:15.339032] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-05 (2.29 seconds). [2025-10-20T21:18:15.451343] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-06. [2025-10-20T21:18:15.451274] [INFO ] [MAIN ] STNG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:15.492280] [DEBUG] [LOCAL ] STNG: Loaded 44277 existing records from primary storage. [2025-10-20T21:18:15.492588] [INFO ] [MAIN ] STNG: Loaded 44277 existing records. [2025-10-20T21:18:15.671549] [WARN ] [MAIN ] STNG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:15.671770] [INFO ] [MAIN ] STNG: Wiping existing primary data before full refresh. [2025-10-20T21:18:15.671818] [INFO ] [LOCAL ] STNG: Starting deletion of primary data. [2025-10-20T21:18:15.672000] [DEBUG] [LOCAL ] STNG: Deleted primary data file. [2025-10-20T21:18:15.672037] [INFO ] [MAIN ] STNG: Starting full primary refresh. [2025-10-20T21:18:16.026695] [INFO ] [REMOTE ] STNG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:16.251431] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-01. [2025-10-20T21:18:16.939110] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-03 (2.29 seconds). [2025-10-20T21:18:17.051370] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-04. [2025-10-20T21:18:17.513786] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-06 (2.06 seconds). [2025-10-20T21:18:17.851370] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-05. [2025-10-20T21:18:17.934465] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T21:18:18.651447] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-02. [2025-10-20T21:18:18.841257] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T21:18:19.311093] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-04 (2.26 seconds). [2025-10-20T21:18:19.451360] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-06. [2025-10-20T21:18:20.046689] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T21:18:20.251427] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-03. [2025-10-20T21:18:20.590426] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T21:18:21.051467] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-05. [2025-10-20T21:18:21.851414] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-07. [2025-10-20T21:18:22.055672] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T21:18:22.651451] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-04. [2025-10-20T21:18:23.251695] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-05 (2.20 seconds). [2025-10-20T21:18:23.451425] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-06. [2025-10-20T21:18:23.846348] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T21:18:24.110872] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T21:18:24.251380] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-05. [2025-10-20T21:18:25.051347] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-07. [2025-10-20T21:18:25.226056] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:18:25.835757] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T21:18:25.851332] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-06. [2025-10-20T21:18:25.895904] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T21:18:26.651397] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-07. [2025-10-20T21:18:27.410495] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:18:27.451348] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-07. [2025-10-20T21:18:28.251412] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-08. [2025-10-20T21:18:29.054059] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-08. [2025-10-20T21:18:29.057565] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-07 (2.41 seconds). [2025-10-20T21:18:29.190791] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T21:18:29.247321] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-08 (1.00 seconds). [2025-10-20T21:18:29.851406] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-08. [2025-10-20T21:18:30.651420] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-09. [2025-10-20T21:18:31.150475] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-08 (2.10 seconds). [2025-10-20T21:18:31.451353] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-08. [2025-10-20T21:18:31.609733] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T21:18:31.820794] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T21:18:32.251384] [INFO ] [REMOTE ] STN: Fetching primary data for 2025-10. [2025-10-20T21:18:32.878680] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T21:18:33.051358] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-09. [2025-10-20T21:18:33.292759] [INFO ] [REMOTE ] STN: Incremental fetch complete for 2025-10 (1.04 seconds). [2025-10-20T21:18:33.296031] [INFO ] [REMOTE ] STN: Full fetch complete (32644 records). [2025-10-20T21:18:33.296110] [DEBUG] [VALIDATOR ] STN: Validating primary data. [2025-10-20T21:18:33.670446] [DEBUG] [VALIDATOR ] STN: All expected trading days are present. [2025-10-20T21:18:33.670521] [DEBUG] [VALIDATOR ] STN: Primary data validation passed. [2025-10-20T21:18:33.671182] [INFO ] [MAIN ] STN: Fetched primary data is valid. [2025-10-20T21:18:33.671243] [DEBUG] [LOCAL ] STN: Saving primary data. [2025-10-20T21:18:33.672001] [DEBUG] [LOCAL ] STN: Trimmed 32644 records to 22776 (keeping 200 days of data) [2025-10-20T21:18:33.814308] [DEBUG] [LOCAL ] STN: Primary data saved (22776 records). [2025-10-20T21:18:33.814400] [INFO ] [MAIN ] STN: Full history saved. [2025-10-20T21:18:33.814475] [INFO ] [MAIN ] Update Progress: 2006/2367 symbols completed. STN took 28.0s. Estimated remaining: 46m 31s. Rate: 7.8 symbols/min [2025-10-20T21:18:33.857689] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-09. [2025-10-20T21:18:33.996463] [INFO ] [MAIN ] STRV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:34.021572] [DEBUG] [LOCAL ] STRV: Loaded 11268 existing records from primary storage. [2025-10-20T21:18:34.021804] [INFO ] [MAIN ] STRV: Loaded 11268 existing records. [2025-10-20T21:18:34.197986] [WARN ] [MAIN ] STRV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:34.198203] [INFO ] [MAIN ] STRV: Wiping existing primary data before full refresh. [2025-10-20T21:18:34.198250] [INFO ] [LOCAL ] STRV: Starting deletion of primary data. [2025-10-20T21:18:34.198402] [DEBUG] [LOCAL ] STRV: Deleted primary data file. [2025-10-20T21:18:34.198440] [INFO ] [MAIN ] STRV: Starting full primary refresh. [2025-10-20T21:18:34.559834] [INFO ] [REMOTE ] STRV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:34.651352] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-01. [2025-10-20T21:18:35.236629] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-09 (2.19 seconds). [2025-10-20T21:18:35.332027] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T21:18:35.451347] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-02. [2025-10-20T21:18:35.524610] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:18:36.251363] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-09. [2025-10-20T21:18:36.329770] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-02 (0.88 seconds). [2025-10-20T21:18:37.051355] [INFO ] [REMOTE ] STNG: Fetching primary data for 2025-10. [2025-10-20T21:18:37.851386] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-03. [2025-10-20T21:18:38.265010] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T21:18:38.295964] [INFO ] [REMOTE ] STNG: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:18:38.302303] [INFO ] [REMOTE ] STNG: Full fetch complete (64310 records). [2025-10-20T21:18:38.302421] [DEBUG] [VALIDATOR ] STNG: Validating primary data. [2025-10-20T21:18:38.653152] [INFO ] [REMOTE ] STM: Fetching primary data for 2025-10. [2025-10-20T21:18:38.719706] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T21:18:38.719391] [DEBUG] [VALIDATOR ] STNG: All expected trading days are present. [2025-10-20T21:18:38.723492] [DEBUG] [VALIDATOR ] STNG: Primary data validation passed. [2025-10-20T21:18:38.725037] [INFO ] [MAIN ] STNG: Fetched primary data is valid. [2025-10-20T21:18:38.725226] [DEBUG] [LOCAL ] STNG: Saving primary data. [2025-10-20T21:18:38.726383] [DEBUG] [LOCAL ] STNG: Trimmed 64310 records to 43623 (keeping 200 days of data) [2025-10-20T21:18:39.000074] [DEBUG] [LOCAL ] STNG: Primary data saved (43623 records). [2025-10-20T21:18:39.000312] [INFO ] [MAIN ] STNG: Full history saved. [2025-10-20T21:18:39.000397] [INFO ] [MAIN ] Update Progress: 2007/2367 symbols completed. STNG took 23.8s. Estimated remaining: 46m 23s. Rate: 7.8 symbols/min [2025-10-20T21:18:39.170199] [INFO ] [MAIN ] STT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:39.220635] [DEBUG] [LOCAL ] STT: Loaded 51813 existing records from primary storage. [2025-10-20T21:18:39.221007] [INFO ] [MAIN ] STT: Loaded 51813 existing records. [2025-10-20T21:18:39.431330] [WARN ] [MAIN ] STT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:39.431742] [INFO ] [MAIN ] STT: Wiping existing primary data before full refresh. [2025-10-20T21:18:39.431872] [INFO ] [LOCAL ] STT: Starting deletion of primary data. [2025-10-20T21:18:39.438569] [DEBUG] [LOCAL ] STT: Deleted primary data file. [2025-10-20T21:18:39.449524] [INFO ] [MAIN ] STT: Starting full primary refresh. [2025-10-20T21:18:39.464617] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-04. [2025-10-20T21:18:39.464474] [INFO ] [REMOTE ] STM: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T21:18:39.564951] [INFO ] [REMOTE ] STM: Full fetch complete (77306 records). [2025-10-20T21:18:39.565119] [DEBUG] [VALIDATOR ] STM: Validating primary data. [2025-10-20T21:18:39.581879] [FATAL] [VALIDATOR ] STM: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:18:39.592218] [FATAL] [MAIN ] STM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:18:39.604370] [WARN ] [MAIN ] STM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:18:39.616690] [INFO ] [LOCAL ] STM: Starting deletion of primary data. [2025-10-20T21:18:39.616770] [DEBUG] [LOCAL ] STM: No primary data file found to delete. [2025-10-20T21:18:39.616808] [INFO ] [LOCAL ] STM: Starting deletion of derivative data. [2025-10-20T21:18:39.639251] [DEBUG] [LOCAL ] STM: Deleted derivative data file. [2025-10-20T21:18:39.687340] [INFO ] [MAIN ] Update Progress: 2008/2367 symbols completed. STM took 44.7s. Estimated remaining: 46m 14s. Rate: 7.8 symbols/min [2025-10-20T21:18:39.988007] [INFO ] [REMOTE ] STT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:40.006045] [INFO ] [MAIN ] STUB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:40.006538] [DEBUG] [LOCAL ] STUB: No existing data file found in primary storage. [2025-10-20T21:18:40.006584] [WARN ] [MAIN ] STUB: No existing data found, performing full refresh instead. [2025-10-20T21:18:40.006726] [INFO ] [MAIN ] STUB: Wiping existing primary data before full refresh. [2025-10-20T21:18:40.006773] [INFO ] [LOCAL ] STUB: Starting deletion of primary data. [2025-10-20T21:18:40.006814] [DEBUG] [LOCAL ] STUB: No primary data file found to delete. [2025-10-20T21:18:40.006845] [INFO ] [MAIN ] STUB: Starting full primary refresh. [2025-10-20T21:18:40.254213] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-01. [2025-10-20T21:18:40.357251] [INFO ] [REMOTE ] STUB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:40.627157] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T21:18:41.051425] [INFO ] [REMOTE ] STUB: Fetching primary data for 2025-01. [2025-10-20T21:18:41.281419] [DEBUG] [REMOTE ] STUB: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:18:41.281716] [WARN ] [REMOTE ] STUB: No data returned for month 2025-01 [2025-10-20T21:18:41.281822] [FATAL] [REMOTE ] STUB: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:18:41.281925] [FATAL] [MAIN ] STUB: Failed to fetch primary data. [2025-10-20T21:18:41.282027] [WARN ] [MAIN ] STUB: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:18:41.282124] [INFO ] [LOCAL ] STUB: Starting deletion of primary data. [2025-10-20T21:18:41.282211] [DEBUG] [LOCAL ] STUB: No primary data file found to delete. [2025-10-20T21:18:41.282249] [INFO ] [LOCAL ] STUB: Starting deletion of derivative data. [2025-10-20T21:18:41.282316] [DEBUG] [LOCAL ] STUB: No derivative data file found to delete. [2025-10-20T21:18:41.282370] [INFO ] [MAIN ] Update Progress: 2009/2367 symbols completed. STUB took 1.6s. Estimated remaining: 46m 5s. Rate: 7.8 symbols/min [2025-10-20T21:18:41.447040] [INFO ] [MAIN ] STVN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:41.472449] [DEBUG] [LOCAL ] STVN: Loaded 27140 existing records from primary storage. [2025-10-20T21:18:41.472719] [INFO ] [MAIN ] STVN: Loaded 27140 existing records. [2025-10-20T21:18:41.646321] [WARN ] [MAIN ] STVN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:41.646521] [INFO ] [MAIN ] STVN: Wiping existing primary data before full refresh. [2025-10-20T21:18:41.646566] [INFO ] [LOCAL ] STVN: Starting deletion of primary data. [2025-10-20T21:18:41.646779] [DEBUG] [LOCAL ] STVN: Deleted primary data file. [2025-10-20T21:18:41.646834] [INFO ] [MAIN ] STVN: Starting full primary refresh. [2025-10-20T21:18:41.856504] [INFO ] [REMOTE ] STLA: Fetching primary data for 2025-10. [2025-10-20T21:18:42.030537] [INFO ] [REMOTE ] STVN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:42.178483] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T21:18:42.651359] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-02. [2025-10-20T21:18:42.688134] [INFO ] [REMOTE ] STLA: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T21:18:42.695447] [INFO ] [REMOTE ] STLA: Full fetch complete (77272 records). [2025-10-20T21:18:42.695573] [DEBUG] [VALIDATOR ] STLA: Validating primary data. [2025-10-20T21:18:42.696363] [FATAL] [VALIDATOR ] STLA: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:18:42.696490] [FATAL] [MAIN ] STLA: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:18:42.696533] [WARN ] [MAIN ] STLA: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:18:42.696598] [INFO ] [LOCAL ] STLA: Starting deletion of primary data. [2025-10-20T21:18:42.696690] [DEBUG] [LOCAL ] STLA: No primary data file found to delete. [2025-10-20T21:18:42.696747] [INFO ] [LOCAL ] STLA: Starting deletion of derivative data. [2025-10-20T21:18:42.696912] [DEBUG] [LOCAL ] STLA: Deleted derivative data file. [2025-10-20T21:18:42.697000] [INFO ] [MAIN ] Update Progress: 2010/2367 symbols completed. STLA took 48.9s. Estimated remaining: 45m 56s. Rate: 7.8 symbols/min [2025-10-20T21:18:42.868345] [INFO ] [MAIN ] STWD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:42.916580] [DEBUG] [LOCAL ] STWD: Loaded 52750 existing records from primary storage. [2025-10-20T21:18:42.916908] [INFO ] [MAIN ] STWD: Loaded 52750 existing records. [2025-10-20T21:18:43.089899] [WARN ] [MAIN ] STWD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:43.090066] [INFO ] [MAIN ] STWD: Wiping existing primary data before full refresh. [2025-10-20T21:18:43.090111] [INFO ] [LOCAL ] STWD: Starting deletion of primary data. [2025-10-20T21:18:43.090294] [DEBUG] [LOCAL ] STWD: Deleted primary data file. [2025-10-20T21:18:43.090330] [INFO ] [MAIN ] STWD: Starting full primary refresh. [2025-10-20T21:18:43.448227] [INFO ] [REMOTE ] STWD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:43.451336] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-01. [2025-10-20T21:18:44.251421] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-05. [2025-10-20T21:18:44.407193] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:18:45.051409] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-01. [2025-10-20T21:18:45.150390] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T21:18:45.289746] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T21:18:45.851368] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-02. [2025-10-20T21:18:45.979267] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-01 (0.93 seconds). [2025-10-20T21:18:46.651369] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-03. [2025-10-20T21:18:47.451355] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-06. [2025-10-20T21:18:47.594862] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T21:18:48.251406] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-03. [2025-10-20T21:18:48.294532] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T21:18:48.543794] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T21:18:49.051385] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-07. [2025-10-20T21:18:49.851353] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-04. [2025-10-20T21:18:49.874701] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-07 (0.82 seconds). [2025-10-20T21:18:50.121578] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T21:18:50.651378] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-08. [2025-10-20T21:18:51.451370] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-02. [2025-10-20T21:18:51.501928] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:18:51.728034] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T21:18:52.251361] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-05. [2025-10-20T21:18:52.653412] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:18:53.051416] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-09. [2025-10-20T21:18:53.851389] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-04. [2025-10-20T21:18:53.903659] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-09 (0.85 seconds). [2025-10-20T21:18:54.168228] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T21:18:54.651364] [INFO ] [REMOTE ] STRV: Fetching primary data for 2025-10. [2025-10-20T21:18:55.274527] [INFO ] [REMOTE ] STRV: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T21:18:55.276472] [INFO ] [REMOTE ] STRV: Full fetch complete (17453 records). [2025-10-20T21:18:55.276540] [DEBUG] [VALIDATOR ] STRV: Validating primary data. [2025-10-20T21:18:55.455034] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-06. [2025-10-20T21:18:55.647788] [DEBUG] [VALIDATOR ] STRV: All expected trading days are present. [2025-10-20T21:18:55.647941] [DEBUG] [VALIDATOR ] STRV: Primary data validation passed. [2025-10-20T21:18:55.648360] [INFO ] [MAIN ] STRV: Fetched primary data is valid. [2025-10-20T21:18:55.648435] [DEBUG] [LOCAL ] STRV: Saving primary data. [2025-10-20T21:18:55.649198] [DEBUG] [LOCAL ] STRV: Trimmed 17453 records to 11011 (keeping 200 days of data) [2025-10-20T21:18:55.733377] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T21:18:55.757075] [DEBUG] [LOCAL ] STRV: Primary data saved (11011 records). [2025-10-20T21:18:55.757188] [INFO ] [MAIN ] STRV: Full history saved. [2025-10-20T21:18:55.757259] [INFO ] [MAIN ] Update Progress: 2011/2367 symbols completed. STRV took 21.9s. Estimated remaining: 45m 49s. Rate: 7.8 symbols/min [2025-10-20T21:18:55.922935] [INFO ] [MAIN ] STXD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:18:55.926297] [DEBUG] [LOCAL ] STXD: Loaded 1262 existing records from primary storage. [2025-10-20T21:18:55.926426] [INFO ] [MAIN ] STXD: Loaded 1262 existing records. [2025-10-20T21:18:56.103057] [WARN ] [MAIN ] STXD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:18:56.103371] [INFO ] [MAIN ] STXD: Wiping existing primary data before full refresh. [2025-10-20T21:18:56.103434] [INFO ] [LOCAL ] STXD: Starting deletion of primary data. [2025-10-20T21:18:56.103620] [DEBUG] [LOCAL ] STXD: Deleted primary data file. [2025-10-20T21:18:56.103723] [INFO ] [MAIN ] STXD: Starting full primary refresh. [2025-10-20T21:18:56.260095] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-03. [2025-10-20T21:18:56.459844] [INFO ] [REMOTE ] STXD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:18:57.051352] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-05. [2025-10-20T21:18:57.321009] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:18:57.544922] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T21:18:57.851403] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-04. [2025-10-20T21:18:58.651422] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-07. [2025-10-20T21:18:58.968760] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T21:18:59.176989] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-04 (1.33 seconds). [2025-10-20T21:18:59.451442] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-01. [2025-10-20T21:18:59.955237] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-01 (0.50 seconds). [2025-10-20T21:19:00.251395] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-06. [2025-10-20T21:19:00.590477] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T21:19:01.051461] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-08. [2025-10-20T21:19:01.851367] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-05. [2025-10-20T21:19:02.045265] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T21:19:02.651395] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-02. [2025-10-20T21:19:02.795080] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T21:19:03.008567] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T21:19:03.135840] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T21:19:03.451368] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-03. [2025-10-20T21:19:03.971139] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T21:19:04.251378] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-06. [2025-10-20T21:19:05.051398] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-09. [2025-10-20T21:19:05.424006] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T21:19:05.605987] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-09 (0.55 seconds). [2025-10-20T21:19:05.851431] [INFO ] [REMOTE ] STT: Fetching primary data for 2025-10. [2025-10-20T21:19:06.651356] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-04. [2025-10-20T21:19:06.903251] [INFO ] [REMOTE ] STT: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T21:19:06.910470] [INFO ] [REMOTE ] STT: Full fetch complete (74119 records). [2025-10-20T21:19:06.910637] [DEBUG] [VALIDATOR ] STT: Validating primary data. [2025-10-20T21:19:06.911420] [FATAL] [VALIDATOR ] STT: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:19:06.911521] [FATAL] [MAIN ] STT: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:19:06.911565] [WARN ] [MAIN ] STT: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:19:06.911604] [INFO ] [LOCAL ] STT: Starting deletion of primary data. [2025-10-20T21:19:06.911709] [DEBUG] [LOCAL ] STT: No primary data file found to delete. [2025-10-20T21:19:06.911775] [INFO ] [LOCAL ] STT: Starting deletion of derivative data. [2025-10-20T21:19:06.911972] [DEBUG] [LOCAL ] STT: Deleted derivative data file. [2025-10-20T21:19:06.912080] [INFO ] [MAIN ] Update Progress: 2012/2367 symbols completed. STT took 27.9s. Estimated remaining: 45m 42s. Rate: 7.8 symbols/min [2025-10-20T21:19:07.097809] [INFO ] [MAIN ] STXE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:07.102288] [DEBUG] [LOCAL ] STXE: Loaded 2136 existing records from primary storage. [2025-10-20T21:19:07.102459] [INFO ] [MAIN ] STXE: Loaded 2136 existing records. [2025-10-20T21:19:07.276396] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T21:19:07.291308] [WARN ] [MAIN ] STXE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:07.291526] [INFO ] [MAIN ] STXE: Wiping existing primary data before full refresh. [2025-10-20T21:19:07.291604] [INFO ] [LOCAL ] STXE: Starting deletion of primary data. [2025-10-20T21:19:07.291785] [DEBUG] [LOCAL ] STXE: Deleted primary data file. [2025-10-20T21:19:07.291844] [INFO ] [MAIN ] STXE: Starting full primary refresh. [2025-10-20T21:19:07.453684] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-07. [2025-10-20T21:19:07.644697] [INFO ] [REMOTE ] STXE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:08.251367] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-07. [2025-10-20T21:19:09.051431] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-05. [2025-10-20T21:19:09.633338] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-05 (0.58 seconds). [2025-10-20T21:19:09.714803] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T21:19:09.851342] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-08. [2025-10-20T21:19:10.177236] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T21:19:10.651402] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-08. [2025-10-20T21:19:11.451372] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-01. [2025-10-20T21:19:11.764732] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-08 (1.91 seconds). [2025-10-20T21:19:12.030247] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T21:19:12.033407] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T21:19:12.251395] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-09. [2025-10-20T21:19:13.051347] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-09. [2025-10-20T21:19:13.646503] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:19:13.851421] [INFO ] [REMOTE ] STVN: Fetching primary data for 2025-10. [2025-10-20T21:19:14.572354] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T21:19:14.651332] [INFO ] [REMOTE ] STWD: Fetching primary data for 2025-10. [2025-10-20T21:19:15.039415] [INFO ] [REMOTE ] STVN: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T21:19:15.043548] [INFO ] [REMOTE ] STVN: Full fetch complete (39305 records). [2025-10-20T21:19:15.043622] [DEBUG] [VALIDATOR ] STVN: Validating primary data. [2025-10-20T21:19:15.415917] [DEBUG] [VALIDATOR ] STVN: All expected trading days are present. [2025-10-20T21:19:15.422175] [DEBUG] [VALIDATOR ] STVN: Primary data validation passed. [2025-10-20T21:19:15.433581] [INFO ] [MAIN ] STVN: Fetched primary data is valid. [2025-10-20T21:19:15.433836] [DEBUG] [LOCAL ] STVN: Saving primary data. [2025-10-20T21:19:15.435188] [INFO ] [REMOTE ] STWD: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:19:15.442459] [DEBUG] [LOCAL ] STVN: Trimmed 39305 records to 26823 (keeping 200 days of data) [2025-10-20T21:19:15.442598] [INFO ] [REMOTE ] STWD: Full fetch complete (74983 records). [2025-10-20T21:19:15.442782] [DEBUG] [VALIDATOR ] STWD: Validating primary data. [2025-10-20T21:19:15.444209] [FATAL] [VALIDATOR ] STWD: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:19:15.444403] [FATAL] [MAIN ] STWD: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:19:15.490778] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-02. [2025-10-20T21:19:15.500748] [WARN ] [MAIN ] STWD: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:19:15.532674] [INFO ] [LOCAL ] STWD: Starting deletion of primary data. [2025-10-20T21:19:15.534473] [DEBUG] [LOCAL ] STWD: No primary data file found to delete. [2025-10-20T21:19:15.539317] [INFO ] [LOCAL ] STWD: Starting deletion of derivative data. [2025-10-20T21:19:15.539745] [DEBUG] [LOCAL ] STWD: Deleted derivative data file. [2025-10-20T21:19:15.540120] [INFO ] [MAIN ] Update Progress: 2013/2367 symbols completed. STWD took 32.8s. Estimated remaining: 45m 35s. Rate: 7.8 symbols/min [2025-10-20T21:19:15.774373] [DEBUG] [LOCAL ] STVN: Primary data saved (26823 records). [2025-10-20T21:19:15.776083] [INFO ] [MAIN ] STXG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:15.776299] [INFO ] [MAIN ] STVN: Full history saved. [2025-10-20T21:19:15.776749] [INFO ] [MAIN ] Update Progress: 2014/2367 symbols completed. STVN took 34.5s. Estimated remaining: 45m 26s. Rate: 7.8 symbols/min [2025-10-20T21:19:15.859474] [DEBUG] [LOCAL ] STXG: Loaded 2452 existing records from primary storage. [2025-10-20T21:19:15.878165] [INFO ] [MAIN ] STXG: Loaded 2452 existing records. [2025-10-20T21:19:16.087816] [INFO ] [MAIN ] STXK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:16.149843] [DEBUG] [LOCAL ] STXK: Loaded 1548 existing records from primary storage. [2025-10-20T21:19:16.149552] [WARN ] [MAIN ] STXG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:16.153483] [INFO ] [MAIN ] STXG: Wiping existing primary data before full refresh. [2025-10-20T21:19:16.153412] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T21:19:16.155187] [INFO ] [LOCAL ] STXG: Starting deletion of primary data. [2025-10-20T21:19:16.155343] [INFO ] [MAIN ] STXK: Loaded 1548 existing records. [2025-10-20T21:19:16.212664] [DEBUG] [LOCAL ] STXG: Deleted primary data file. [2025-10-20T21:19:16.217876] [INFO ] [MAIN ] STXG: Starting full primary refresh. [2025-10-20T21:19:16.259620] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-06. [2025-10-20T21:19:16.494628] [WARN ] [MAIN ] STXK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:16.509576] [INFO ] [MAIN ] STXK: Wiping existing primary data before full refresh. [2025-10-20T21:19:16.517407] [INFO ] [LOCAL ] STXK: Starting deletion of primary data. [2025-10-20T21:19:16.580771] [DEBUG] [LOCAL ] STXK: Deleted primary data file. [2025-10-20T21:19:16.586584] [INFO ] [MAIN ] STXK: Starting full primary refresh. [2025-10-20T21:19:16.766736] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T21:19:16.805697] [INFO ] [REMOTE ] STXG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:17.040831] [INFO ] [REMOTE ] STXK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:17.051329] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-01. [2025-10-20T21:19:17.671627] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T21:19:17.851397] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-07. [2025-10-20T21:19:18.254304] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T21:19:18.651378] [INFO ] [REMOTE ] STXG: Fetching primary data for 2025-01. [2025-10-20T21:19:19.283640] [INFO ] [REMOTE ] STXG: Incremental fetch complete for 2025-01 (0.63 seconds). [2025-10-20T21:19:19.451385] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-02. [2025-10-20T21:19:19.944491] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T21:19:20.251392] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-08. [2025-10-20T21:19:20.660110] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T21:19:21.051421] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-09. [2025-10-20T21:19:21.464067] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-09 (0.41 seconds). [2025-10-20T21:19:21.851368] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-03. [2025-10-20T21:19:22.460974] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T21:19:22.651407] [INFO ] [REMOTE ] STXD: Fetching primary data for 2025-10. [2025-10-20T21:19:23.049026] [INFO ] [REMOTE ] STXD: Incremental fetch complete for 2025-10 (0.40 seconds). [2025-10-20T21:19:23.049832] [INFO ] [REMOTE ] STXD: Full fetch complete (2049 records). [2025-10-20T21:19:23.049905] [DEBUG] [VALIDATOR ] STXD: Validating primary data. [2025-10-20T21:19:23.394063] [DEBUG] [VALIDATOR ] STXD: All expected trading days are present. [2025-10-20T21:19:23.394133] [DEBUG] [VALIDATOR ] STXD: Primary data validation passed. [2025-10-20T21:19:23.394209] [INFO ] [MAIN ] STXD: Fetched primary data is valid. [2025-10-20T21:19:23.394246] [DEBUG] [LOCAL ] STXD: Saving primary data. [2025-10-20T21:19:23.394548] [DEBUG] [LOCAL ] STXD: Trimmed 2049 records to 1245 (keeping 200 days of data) [2025-10-20T21:19:23.402396] [DEBUG] [LOCAL ] STXD: Primary data saved (1245 records). [2025-10-20T21:19:23.402453] [INFO ] [MAIN ] STXD: Full history saved. [2025-10-20T21:19:23.402515] [INFO ] [MAIN ] Update Progress: 2015/2367 symbols completed. STXD took 27.6s. Estimated remaining: 45m 18s. Rate: 7.8 symbols/min [2025-10-20T21:19:23.456611] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-03. [2025-10-20T21:19:23.569787] [INFO ] [MAIN ] STXM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:23.569930] [DEBUG] [LOCAL ] STXM: No existing data file found in primary storage. [2025-10-20T21:19:23.569966] [WARN ] [MAIN ] STXM: No existing data found, performing full refresh instead. [2025-10-20T21:19:23.570072] [INFO ] [MAIN ] STXM: Wiping existing primary data before full refresh. [2025-10-20T21:19:23.570113] [INFO ] [LOCAL ] STXM: Starting deletion of primary data. [2025-10-20T21:19:23.570154] [DEBUG] [LOCAL ] STXM: No primary data file found to delete. [2025-10-20T21:19:23.570185] [INFO ] [MAIN ] STXM: Starting full primary refresh. [2025-10-20T21:19:23.932234] [INFO ] [REMOTE ] STXM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:24.010363] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T21:19:24.251377] [INFO ] [REMOTE ] STXG: Fetching primary data for 2025-02. [2025-10-20T21:19:24.791019] [INFO ] [REMOTE ] STXG: Incremental fetch complete for 2025-02 (0.54 seconds). [2025-10-20T21:19:25.051405] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-04. [2025-10-20T21:19:25.504887] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-04 (0.45 seconds). [2025-10-20T21:19:25.851400] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-04. [2025-10-20T21:19:26.397018] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-04 (0.55 seconds). [2025-10-20T21:19:26.651402] [INFO ] [REMOTE ] STXG: Fetching primary data for 2025-03. [2025-10-20T21:19:27.266388] [INFO ] [REMOTE ] STXG: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T21:19:27.451368] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-01. [2025-10-20T21:19:27.941748] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-10-20T21:19:28.251362] [INFO ] [REMOTE ] STXG: Fetching primary data for 2025-04. [2025-10-20T21:19:28.362012] [DEBUG] [REMOTE ] STXG: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T21:19:28.362332] [WARN ] [REMOTE ] STXG: No data returned for month 2025-04 [2025-10-20T21:19:28.362422] [FATAL] [REMOTE ] STXG: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:19:28.362507] [FATAL] [MAIN ] STXG: Failed to fetch primary data. [2025-10-20T21:19:28.362566] [WARN ] [MAIN ] STXG: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:19:28.362666] [INFO ] [LOCAL ] STXG: Starting deletion of primary data. [2025-10-20T21:19:28.362750] [DEBUG] [LOCAL ] STXG: No primary data file found to delete. [2025-10-20T21:19:28.362787] [INFO ] [LOCAL ] STXG: Starting deletion of derivative data. [2025-10-20T21:19:28.362912] [DEBUG] [LOCAL ] STXG: Deleted derivative data file. [2025-10-20T21:19:28.362976] [INFO ] [MAIN ] Update Progress: 2016/2367 symbols completed. STXG took 12.8s. Estimated remaining: 45m 10s. Rate: 7.8 symbols/min [2025-10-20T21:19:28.528860] [INFO ] [MAIN ] STXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:28.533003] [DEBUG] [LOCAL ] STXT: Loaded 2537 existing records from primary storage. [2025-10-20T21:19:28.533151] [INFO ] [MAIN ] STXT: Loaded 2537 existing records. [2025-10-20T21:19:28.706863] [WARN ] [MAIN ] STXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:28.707074] [INFO ] [MAIN ] STXT: Wiping existing primary data before full refresh. [2025-10-20T21:19:28.707123] [INFO ] [LOCAL ] STXT: Starting deletion of primary data. [2025-10-20T21:19:28.707266] [DEBUG] [LOCAL ] STXT: Deleted primary data file. [2025-10-20T21:19:28.707306] [INFO ] [MAIN ] STXT: Starting full primary refresh. [2025-10-20T21:19:29.056423] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-05. [2025-10-20T21:19:29.060593] [INFO ] [REMOTE ] STXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:29.550828] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:19:29.851384] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-05. [2025-10-20T21:19:30.339065] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:19:30.651379] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-06. [2025-10-20T21:19:31.132463] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-06 (0.48 seconds). [2025-10-20T21:19:31.451360] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-07. [2025-10-20T21:19:31.853399] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T21:19:32.251370] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-08. [2025-10-20T21:19:32.752255] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T21:19:33.051373] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-09. [2025-10-20T21:19:33.535999] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:19:33.851372] [INFO ] [REMOTE ] STXK: Fetching primary data for 2025-10. [2025-10-20T21:19:34.351220] [INFO ] [REMOTE ] STXK: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T21:19:34.352159] [INFO ] [REMOTE ] STXK: Full fetch complete (2425 records). [2025-10-20T21:19:34.352227] [DEBUG] [VALIDATOR ] STXK: Validating primary data. [2025-10-20T21:19:34.666636] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-06. [2025-10-20T21:19:34.707177] [DEBUG] [VALIDATOR ] STXK: All expected trading days are present. [2025-10-20T21:19:34.707255] [DEBUG] [VALIDATOR ] STXK: Primary data validation passed. [2025-10-20T21:19:34.707352] [INFO ] [MAIN ] STXK: Fetched primary data is valid. [2025-10-20T21:19:34.707387] [DEBUG] [LOCAL ] STXK: Saving primary data. [2025-10-20T21:19:34.707735] [DEBUG] [LOCAL ] STXK: Trimmed 2425 records to 1512 (keeping 200 days of data) [2025-10-20T21:19:34.717352] [DEBUG] [LOCAL ] STXK: Primary data saved (1512 records). [2025-10-20T21:19:34.717427] [INFO ] [MAIN ] STXK: Full history saved. [2025-10-20T21:19:34.717496] [INFO ] [MAIN ] Update Progress: 2017/2367 symbols completed. STXK took 18.9s. Estimated remaining: 45m 2s. Rate: 7.8 symbols/min [2025-10-20T21:19:34.886629] [INFO ] [MAIN ] STXV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:34.890757] [DEBUG] [LOCAL ] STXV: Loaded 1542 existing records from primary storage. [2025-10-20T21:19:34.890949] [INFO ] [MAIN ] STXV: Loaded 1542 existing records. [2025-10-20T21:19:35.067697] [WARN ] [MAIN ] STXV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:35.067973] [INFO ] [MAIN ] STXV: Wiping existing primary data before full refresh. [2025-10-20T21:19:35.068059] [INFO ] [LOCAL ] STXV: Starting deletion of primary data. [2025-10-20T21:19:35.068196] [DEBUG] [LOCAL ] STXV: Deleted primary data file. [2025-10-20T21:19:35.068236] [INFO ] [MAIN ] STXV: Starting full primary refresh. [2025-10-20T21:19:35.331274] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T21:19:35.438688] [INFO ] [REMOTE ] STXV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:35.451339] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-01. [2025-10-20T21:19:36.047885] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T21:19:36.251393] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-07. [2025-10-20T21:19:36.754574] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-07 (0.50 seconds). [2025-10-20T21:19:37.051420] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-01. [2025-10-20T21:19:37.672984] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T21:19:37.851423] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-02. [2025-10-20T21:19:38.293172] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-02 (0.44 seconds). [2025-10-20T21:19:38.651379] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-03. [2025-10-20T21:19:39.046896] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-03 (0.40 seconds). [2025-10-20T21:19:39.451428] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-08. [2025-10-20T21:19:40.027615] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T21:19:40.251377] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-09. [2025-10-20T21:19:40.810691] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-09 (0.56 seconds). [2025-10-20T21:19:41.051365] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-02. [2025-10-20T21:19:41.545700] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T21:19:41.851367] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-03. [2025-10-20T21:19:42.372040] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-03 (0.52 seconds). [2025-10-20T21:19:42.651373] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-04. [2025-10-20T21:19:43.135242] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T21:19:43.451376] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-04. [2025-10-20T21:19:43.883263] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T21:19:44.251445] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-02. [2025-10-20T21:19:44.776932] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T21:19:45.051398] [INFO ] [REMOTE ] STXE: Fetching primary data for 2025-10. [2025-10-20T21:19:45.583394] [INFO ] [REMOTE ] STXE: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T21:19:45.584398] [INFO ] [REMOTE ] STXE: Full fetch complete (3508 records). [2025-10-20T21:19:45.584470] [DEBUG] [VALIDATOR ] STXE: Validating primary data. [2025-10-20T21:19:45.856686] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-03. [2025-10-20T21:19:45.964952] [DEBUG] [VALIDATOR ] STXE: All expected trading days are present. [2025-10-20T21:19:45.965029] [DEBUG] [VALIDATOR ] STXE: Primary data validation passed. [2025-10-20T21:19:45.965143] [INFO ] [MAIN ] STXE: Fetched primary data is valid. [2025-10-20T21:19:45.965181] [DEBUG] [LOCAL ] STXE: Saving primary data. [2025-10-20T21:19:45.965526] [DEBUG] [LOCAL ] STXE: Trimmed 3508 records to 2087 (keeping 200 days of data) [2025-10-20T21:19:45.978415] [DEBUG] [LOCAL ] STXE: Primary data saved (2087 records). [2025-10-20T21:19:45.978494] [INFO ] [MAIN ] STXE: Full history saved. [2025-10-20T21:19:45.978573] [INFO ] [MAIN ] Update Progress: 2018/2367 symbols completed. STXE took 39.1s. Estimated remaining: 44m 55s. Rate: 7.8 symbols/min [2025-10-20T21:19:46.145787] [INFO ] [MAIN ] STZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:19:46.200140] [DEBUG] [LOCAL ] STZ: Loaded 53184 existing records from primary storage. [2025-10-20T21:19:46.200487] [INFO ] [MAIN ] STZ: Loaded 53184 existing records. [2025-10-20T21:19:46.376334] [WARN ] [MAIN ] STZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:19:46.376556] [INFO ] [MAIN ] STZ: Wiping existing primary data before full refresh. [2025-10-20T21:19:46.376607] [INFO ] [LOCAL ] STZ: Starting deletion of primary data. [2025-10-20T21:19:46.376835] [DEBUG] [LOCAL ] STZ: Deleted primary data file. [2025-10-20T21:19:46.376891] [INFO ] [MAIN ] STZ: Starting full primary refresh. [2025-10-20T21:19:46.561514] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T21:19:46.656614] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-05. [2025-10-20T21:19:46.746913] [INFO ] [REMOTE ] STZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:19:47.101556] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-05 (0.44 seconds). [2025-10-20T21:19:47.451369] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-01. [2025-10-20T21:19:48.251383] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-04. [2025-10-20T21:19:48.756297] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T21:19:49.051364] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-05. [2025-10-20T21:19:49.356314] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-01 (1.90 seconds). [2025-10-20T21:19:49.588941] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-05 (0.54 seconds). [2025-10-20T21:19:49.851362] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-05. [2025-10-20T21:19:50.341663] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:19:50.651405] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-06. [2025-10-20T21:19:51.115795] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-06 (0.46 seconds). [2025-10-20T21:19:51.451441] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-07. [2025-10-20T21:19:51.944749] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T21:19:52.251365] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-08. [2025-10-20T21:19:52.770801] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T21:19:53.051413] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-06. [2025-10-20T21:19:53.479426] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-06 (0.43 seconds). [2025-10-20T21:19:53.851370] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-07. [2025-10-20T21:19:54.246577] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T21:19:54.651374] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-02. [2025-10-20T21:19:55.451347] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-08. [2025-10-20T21:19:55.863264] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-08 (0.41 seconds). [2025-10-20T21:19:56.251290] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-06. [2025-10-20T21:19:56.689941] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T21:19:56.772758] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-06 (0.52 seconds). [2025-10-20T21:19:57.051420] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-07. [2025-10-20T21:19:57.674717] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-07 (0.62 seconds). [2025-10-20T21:19:57.851388] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-08. [2025-10-20T21:19:58.356119] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-08 (0.50 seconds). [2025-10-20T21:19:58.651387] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-09. [2025-10-20T21:19:59.217892] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-09 (0.57 seconds). [2025-10-20T21:19:59.451384] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-09. [2025-10-20T21:19:59.853460] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-09 (0.40 seconds). [2025-10-20T21:20:00.251359] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-03. [2025-10-20T21:20:01.051439] [INFO ] [REMOTE ] STXM: Fetching primary data for 2025-10. [2025-10-20T21:20:01.467727] [INFO ] [REMOTE ] STXM: Incremental fetch complete for 2025-10 (0.42 seconds). [2025-10-20T21:20:01.468491] [INFO ] [REMOTE ] STXM: Full fetch complete (1403 records). [2025-10-20T21:20:01.468559] [DEBUG] [VALIDATOR ] STXM: Validating primary data. [2025-10-20T21:20:01.649971] [WARN ] [VALIDATOR ] STXM: Validation failed - insufficient historical data. Have 197 unique days, need 200 days [2025-10-20T21:20:01.650159] [FATAL] [MAIN ] STXM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:20:01.650226] [WARN ] [MAIN ] STXM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:20:01.650296] [INFO ] [LOCAL ] STXM: Starting deletion of primary data. [2025-10-20T21:20:01.650358] [DEBUG] [LOCAL ] STXM: No primary data file found to delete. [2025-10-20T21:20:01.650393] [INFO ] [LOCAL ] STXM: Starting deletion of derivative data. [2025-10-20T21:20:01.650432] [DEBUG] [LOCAL ] STXM: No derivative data file found to delete. [2025-10-20T21:20:01.650486] [INFO ] [MAIN ] Update Progress: 2019/2367 symbols completed. STXM took 38.2s. Estimated remaining: 44m 48s. Rate: 7.8 symbols/min [2025-10-20T21:20:01.821407] [INFO ] [MAIN ] SU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:01.851397] [INFO ] [REMOTE ] STXV: Fetching primary data for 2025-10. [2025-10-20T21:20:01.876223] [DEBUG] [LOCAL ] SU: Loaded 54206 existing records from primary storage. [2025-10-20T21:20:01.876591] [INFO ] [MAIN ] SU: Loaded 54206 existing records. [2025-10-20T21:20:02.056046] [WARN ] [MAIN ] SU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:02.056262] [INFO ] [MAIN ] SU: Wiping existing primary data before full refresh. [2025-10-20T21:20:02.056309] [INFO ] [LOCAL ] SU: Starting deletion of primary data. [2025-10-20T21:20:02.056494] [DEBUG] [LOCAL ] SU: Deleted primary data file. [2025-10-20T21:20:02.056530] [INFO ] [MAIN ] SU: Starting full primary refresh. [2025-10-20T21:20:02.325365] [INFO ] [REMOTE ] STXV: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T21:20:02.339329] [INFO ] [REMOTE ] STXV: Full fetch complete (2519 records). [2025-10-20T21:20:02.350822] [DEBUG] [VALIDATOR ] STXV: Validating primary data. [2025-10-20T21:20:02.533735] [INFO ] [REMOTE ] SU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:02.660378] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-09. [2025-10-20T21:20:02.870568] [DEBUG] [VALIDATOR ] STXV: All expected trading days are present. [2025-10-20T21:20:02.870730] [DEBUG] [VALIDATOR ] STXV: Primary data validation passed. [2025-10-20T21:20:02.871046] [INFO ] [MAIN ] STXV: Fetched primary data is valid. [2025-10-20T21:20:02.871223] [DEBUG] [LOCAL ] STXV: Saving primary data. [2025-10-20T21:20:02.871841] [DEBUG] [LOCAL ] STXV: Trimmed 2519 records to 1488 (keeping 200 days of data) [2025-10-20T21:20:02.882864] [DEBUG] [LOCAL ] STXV: Primary data saved (1488 records). [2025-10-20T21:20:02.883211] [INFO ] [MAIN ] STXV: Full history saved. [2025-10-20T21:20:02.883356] [INFO ] [MAIN ] Update Progress: 2020/2367 symbols completed. STXV took 28.2s. Estimated remaining: 44m 40s. Rate: 7.8 symbols/min [2025-10-20T21:20:03.032491] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-03 (2.78 seconds). [2025-10-20T21:20:03.119070] [INFO ] [MAIN ] SUI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:03.161286] [DEBUG] [LOCAL ] SUI: Loaded 42659 existing records from primary storage. [2025-10-20T21:20:03.161596] [INFO ] [MAIN ] SUI: Loaded 42659 existing records. [2025-10-20T21:20:03.279237] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T21:20:03.365010] [WARN ] [MAIN ] SUI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:03.365234] [INFO ] [MAIN ] SUI: Wiping existing primary data before full refresh. [2025-10-20T21:20:03.365282] [INFO ] [LOCAL ] SUI: Starting deletion of primary data. [2025-10-20T21:20:03.365469] [DEBUG] [LOCAL ] SUI: Deleted primary data file. [2025-10-20T21:20:03.365542] [INFO ] [MAIN ] SUI: Starting full primary refresh. [2025-10-20T21:20:03.456476] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-01. [2025-10-20T21:20:03.803855] [INFO ] [REMOTE ] SUI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:04.251366] [INFO ] [REMOTE ] STXT: Fetching primary data for 2025-10. [2025-10-20T21:20:04.803845] [INFO ] [REMOTE ] STXT: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T21:20:04.804995] [INFO ] [REMOTE ] STXT: Full fetch complete (3882 records). [2025-10-20T21:20:04.805067] [DEBUG] [VALIDATOR ] STXT: Validating primary data. [2025-10-20T21:20:05.059911] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-01. [2025-10-20T21:20:05.188445] [DEBUG] [VALIDATOR ] STXT: All expected trading days are present. [2025-10-20T21:20:05.188535] [DEBUG] [VALIDATOR ] STXT: Primary data validation passed. [2025-10-20T21:20:05.188746] [INFO ] [MAIN ] STXT: Fetched primary data is valid. [2025-10-20T21:20:05.188808] [DEBUG] [LOCAL ] STXT: Saving primary data. [2025-10-20T21:20:05.189195] [DEBUG] [LOCAL ] STXT: Trimmed 3882 records to 2492 (keeping 200 days of data) [2025-10-20T21:20:05.205926] [DEBUG] [LOCAL ] STXT: Primary data saved (2492 records). [2025-10-20T21:20:05.206032] [INFO ] [MAIN ] STXT: Full history saved. [2025-10-20T21:20:05.206108] [INFO ] [MAIN ] Update Progress: 2021/2367 symbols completed. STXT took 36.8s. Estimated remaining: 44m 31s. Rate: 7.8 symbols/min [2025-10-20T21:20:05.383056] [INFO ] [MAIN ] SUN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:05.418628] [DEBUG] [LOCAL ] SUN: Loaded 39035 existing records from primary storage. [2025-10-20T21:20:05.418955] [INFO ] [MAIN ] SUN: Loaded 39035 existing records. [2025-10-20T21:20:05.599778] [WARN ] [MAIN ] SUN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:05.600162] [INFO ] [MAIN ] SUN: Wiping existing primary data before full refresh. [2025-10-20T21:20:05.600259] [INFO ] [LOCAL ] SUN: Starting deletion of primary data. [2025-10-20T21:20:05.610569] [DEBUG] [LOCAL ] SUN: Deleted primary data file. [2025-10-20T21:20:05.617976] [INFO ] [MAIN ] SUN: Starting full primary refresh. [2025-10-20T21:20:05.737429] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-01 (2.28 seconds). [2025-10-20T21:20:05.856471] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-02. [2025-10-20T21:20:06.005192] [INFO ] [REMOTE ] SUN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:06.458950] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T21:20:06.651389] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-04. [2025-10-20T21:20:07.451374] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-02. [2025-10-20T21:20:07.808331] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-02 (1.95 seconds). [2025-10-20T21:20:08.251444] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-01. [2025-10-20T21:20:08.945784] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T21:20:09.017625] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:20:09.051335] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-03. [2025-10-20T21:20:09.798091] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:20:09.851340] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-02. [2025-10-20T21:20:10.651365] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-05. [2025-10-20T21:20:10.844993] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T21:20:11.451418] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-04. [2025-10-20T21:20:11.575472] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:20:12.251361] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-03. [2025-10-20T21:20:12.930494] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-10-20T21:20:13.051360] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-06. [2025-10-20T21:20:13.205196] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T21:20:13.851362] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-03. [2025-10-20T21:20:14.181528] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T21:20:14.651359] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-04. [2025-10-20T21:20:15.451366] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-05. [2025-10-20T21:20:16.033545] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-06 (2.98 seconds). [2025-10-20T21:20:16.101871] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T21:20:16.251356] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-05. [2025-10-20T21:20:16.679475] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T21:20:17.051499] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-07. [2025-10-20T21:20:17.851391] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-06. [2025-10-20T21:20:17.876001] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T21:20:18.651379] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-06. [2025-10-20T21:20:19.391809] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:20:19.431716] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-07 (2.38 seconds). [2025-10-20T21:20:19.451332] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-08. [2025-10-20T21:20:19.536398] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-03 (5.69 seconds). [2025-10-20T21:20:20.129623] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:20:20.251373] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-04. [2025-10-20T21:20:21.051377] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-07. [2025-10-20T21:20:21.381184] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T21:20:21.827038] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T21:20:21.851341] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-05. [2025-10-20T21:20:22.651351] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-07. [2025-10-20T21:20:22.775479] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:20:23.451351] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-08. [2025-10-20T21:20:23.742898] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T21:20:24.251022] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T21:20:24.254677] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-06. [2025-10-20T21:20:25.052004] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-09. [2025-10-20T21:20:25.051925] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T21:20:25.851361] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-08. [2025-10-20T21:20:25.987449] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T21:20:26.042455] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T21:20:26.651399] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-07. [2025-10-20T21:20:27.224721] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T21:20:27.451380] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-09. [2025-10-20T21:20:28.251372] [INFO ] [REMOTE ] STZ: Fetching primary data for 2025-10. [2025-10-20T21:20:28.555742] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T21:20:29.051382] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-09. [2025-10-20T21:20:29.052985] [INFO ] [REMOTE ] STZ: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T21:20:29.065871] [INFO ] [REMOTE ] STZ: Full fetch complete (75797 records). [2025-10-20T21:20:29.066005] [DEBUG] [VALIDATOR ] STZ: Validating primary data. [2025-10-20T21:20:29.066935] [FATAL] [VALIDATOR ] STZ: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:20:29.067036] [FATAL] [MAIN ] STZ: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:20:29.067087] [WARN ] [MAIN ] STZ: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:20:29.067165] [INFO ] [LOCAL ] STZ: Starting deletion of primary data. [2025-10-20T21:20:29.067239] [DEBUG] [LOCAL ] STZ: No primary data file found to delete. [2025-10-20T21:20:29.067282] [INFO ] [LOCAL ] STZ: Starting deletion of derivative data. [2025-10-20T21:20:29.067488] [DEBUG] [LOCAL ] STZ: Deleted derivative data file. [2025-10-20T21:20:29.067593] [INFO ] [MAIN ] Update Progress: 2022/2367 symbols completed. STZ took 43.1s. Estimated remaining: 44m 26s. Rate: 7.8 symbols/min [2025-10-20T21:20:29.283283] [INFO ] [MAIN ] SUPV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:29.288045] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T21:20:29.340568] [DEBUG] [LOCAL ] SUPV: Loaded 48638 existing records from primary storage. [2025-10-20T21:20:29.340902] [INFO ] [MAIN ] SUPV: Loaded 48638 existing records. [2025-10-20T21:20:29.515473] [WARN ] [MAIN ] SUPV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:29.515701] [INFO ] [MAIN ] SUPV: Wiping existing primary data before full refresh. [2025-10-20T21:20:29.515757] [INFO ] [LOCAL ] SUPV: Starting deletion of primary data. [2025-10-20T21:20:29.515953] [DEBUG] [LOCAL ] SUPV: Deleted primary data file. [2025-10-20T21:20:29.516009] [INFO ] [MAIN ] SUPV: Starting full primary refresh. [2025-10-20T21:20:29.861438] [INFO ] [REMOTE ] SUI: Fetching primary data for 2025-10. [2025-10-20T21:20:29.866928] [INFO ] [REMOTE ] SUPV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:30.651383] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-08. [2025-10-20T21:20:30.711158] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T21:20:31.454817] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-01. [2025-10-20T21:20:31.459021] [INFO ] [REMOTE ] SUI: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T21:20:31.465780] [INFO ] [REMOTE ] SUI: Full fetch complete (60939 records). [2025-10-20T21:20:31.465882] [DEBUG] [VALIDATOR ] SUI: Validating primary data. [2025-10-20T21:20:31.854426] [DEBUG] [VALIDATOR ] SUI: All expected trading days are present. [2025-10-20T21:20:31.854503] [DEBUG] [VALIDATOR ] SUI: Primary data validation passed. [2025-10-20T21:20:31.855505] [INFO ] [MAIN ] SUI: Fetched primary data is valid. [2025-10-20T21:20:31.855569] [DEBUG] [LOCAL ] SUI: Saving primary data. [2025-10-20T21:20:31.856716] [DEBUG] [LOCAL ] SUI: Trimmed 60939 records to 42016 (keeping 200 days of data) [2025-10-20T21:20:32.114271] [DEBUG] [LOCAL ] SUI: Primary data saved (42016 records). [2025-10-20T21:20:32.114412] [INFO ] [MAIN ] SUI: Full history saved. [2025-10-20T21:20:32.114494] [INFO ] [MAIN ] Update Progress: 2023/2367 symbols completed. SUI took 29.2s. Estimated remaining: 44m 17s. Rate: 7.8 symbols/min [2025-10-20T21:20:32.260729] [INFO ] [REMOTE ] SUN: Fetching primary data for 2025-10. [2025-10-20T21:20:32.295885] [INFO ] [MAIN ] SUZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:32.338816] [DEBUG] [LOCAL ] SUZ: Loaded 46367 existing records from primary storage. [2025-10-20T21:20:32.339139] [INFO ] [MAIN ] SUZ: Loaded 46367 existing records. [2025-10-20T21:20:32.520185] [WARN ] [MAIN ] SUZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:32.520490] [INFO ] [MAIN ] SUZ: Wiping existing primary data before full refresh. [2025-10-20T21:20:32.520546] [INFO ] [LOCAL ] SUZ: Starting deletion of primary data. [2025-10-20T21:20:32.520772] [DEBUG] [LOCAL ] SUZ: Deleted primary data file. [2025-10-20T21:20:32.520830] [INFO ] [MAIN ] SUZ: Starting full primary refresh. [2025-10-20T21:20:32.931418] [INFO ] [REMOTE ] SUZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:32.957469] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-08 (2.31 seconds). [2025-10-20T21:20:33.051352] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-09. [2025-10-20T21:20:33.183695] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T21:20:33.851548] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-01. [2025-10-20T21:20:33.852186] [INFO ] [REMOTE ] SUN: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T21:20:33.860159] [INFO ] [REMOTE ] SUN: Full fetch complete (55489 records). [2025-10-20T21:20:33.860376] [DEBUG] [VALIDATOR ] SUN: Validating primary data. [2025-10-20T21:20:34.255021] [DEBUG] [VALIDATOR ] SUN: All expected trading days are present. [2025-10-20T21:20:34.255166] [DEBUG] [VALIDATOR ] SUN: Primary data validation passed. [2025-10-20T21:20:34.257043] [INFO ] [MAIN ] SUN: Fetched primary data is valid. [2025-10-20T21:20:34.257118] [DEBUG] [LOCAL ] SUN: Saving primary data. [2025-10-20T21:20:34.259062] [DEBUG] [LOCAL ] SUN: Trimmed 55489 records to 38845 (keeping 200 days of data) [2025-10-20T21:20:34.515695] [DEBUG] [LOCAL ] SUN: Primary data saved (38845 records). [2025-10-20T21:20:34.515902] [INFO ] [MAIN ] SUN: Full history saved. [2025-10-20T21:20:34.516005] [INFO ] [MAIN ] Update Progress: 2024/2367 symbols completed. SUN took 29.3s. Estimated remaining: 44m 9s. Rate: 7.8 symbols/min [2025-10-20T21:20:34.654455] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-02. [2025-10-20T21:20:34.709680] [INFO ] [MAIN ] SVV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:34.754712] [DEBUG] [LOCAL ] SVV: Loaded 40914 existing records from primary storage. [2025-10-20T21:20:34.766574] [INFO ] [MAIN ] SVV: Loaded 40914 existing records. [2025-10-20T21:20:34.872961] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T21:20:34.989795] [WARN ] [MAIN ] SVV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:34.990469] [INFO ] [MAIN ] SVV: Wiping existing primary data before full refresh. [2025-10-20T21:20:34.990582] [INFO ] [LOCAL ] SVV: Starting deletion of primary data. [2025-10-20T21:20:34.990910] [DEBUG] [LOCAL ] SVV: Deleted primary data file. [2025-10-20T21:20:34.990971] [INFO ] [MAIN ] SVV: Starting full primary refresh. [2025-10-20T21:20:35.369587] [INFO ] [REMOTE ] SVV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:35.395772] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:20:35.451335] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-02. [2025-10-20T21:20:35.911484] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-02 (1.26 seconds). [2025-10-20T21:20:36.251414] [INFO ] [REMOTE ] SU: Fetching primary data for 2025-10. [2025-10-20T21:20:36.734328] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T21:20:36.985500] [INFO ] [REMOTE ] SU: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:20:36.992855] [INFO ] [REMOTE ] SU: Full fetch complete (77176 records). [2025-10-20T21:20:36.992968] [DEBUG] [VALIDATOR ] SU: Validating primary data. [2025-10-20T21:20:36.993593] [FATAL] [VALIDATOR ] SU: Data is stale - most recent date (2025-10-17) is before last completed trading day (2025-10-20). Symbol may be delisted or halted. [2025-10-20T21:20:36.993727] [FATAL] [MAIN ] SU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:20:36.993772] [WARN ] [MAIN ] SU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:20:36.993809] [INFO ] [LOCAL ] SU: Starting deletion of primary data. [2025-10-20T21:20:36.993862] [DEBUG] [LOCAL ] SU: No primary data file found to delete. [2025-10-20T21:20:36.993903] [INFO ] [LOCAL ] SU: Starting deletion of derivative data. [2025-10-20T21:20:36.994017] [DEBUG] [LOCAL ] SU: Deleted derivative data file. [2025-10-20T21:20:36.994331] [INFO ] [MAIN ] Update Progress: 2025/2367 symbols completed. SU took 35.3s. Estimated remaining: 44m 0s. Rate: 7.8 symbols/min [2025-10-20T21:20:37.057100] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-01. [2025-10-20T21:20:37.171419] [INFO ] [MAIN ] SW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:37.222005] [DEBUG] [LOCAL ] SW: Loaded 53965 existing records from primary storage. [2025-10-20T21:20:37.222299] [INFO ] [MAIN ] SW: Loaded 53965 existing records. [2025-10-20T21:20:37.396139] [WARN ] [MAIN ] SW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:37.396304] [INFO ] [MAIN ] SW: Wiping existing primary data before full refresh. [2025-10-20T21:20:37.396347] [INFO ] [LOCAL ] SW: Starting deletion of primary data. [2025-10-20T21:20:37.396527] [DEBUG] [LOCAL ] SW: Deleted primary data file. [2025-10-20T21:20:37.396563] [INFO ] [MAIN ] SW: Starting full primary refresh. [2025-10-20T21:20:37.744934] [INFO ] [REMOTE ] SW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:37.851334] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-01. [2025-10-20T21:20:38.411844] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-01 (1.35 seconds). [2025-10-20T21:20:38.651439] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-03. [2025-10-20T21:20:39.451406] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-02. [2025-10-20T21:20:39.902814] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-01 (2.05 seconds). [2025-10-20T21:20:40.251402] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-03. [2025-10-20T21:20:40.409044] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T21:20:40.870125] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T21:20:41.051399] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-03. [2025-10-20T21:20:41.490123] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T21:20:41.851348] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-04. [2025-10-20T21:20:42.524012] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T21:20:42.651370] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-02. [2025-10-20T21:20:43.451357] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-04. [2025-10-20T21:20:43.583861] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:20:44.251398] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-05. [2025-10-20T21:20:44.357028] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T21:20:45.011525] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T21:20:45.051329] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-05. [2025-10-20T21:20:45.583207] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T21:20:45.851358] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-06. [2025-10-20T21:20:46.573337] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:20:46.651349] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-04. [2025-10-20T21:20:47.417829] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T21:20:47.451342] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-07. [2025-10-20T21:20:48.251428] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-03. [2025-10-20T21:20:48.322195] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T21:20:49.056552] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-06. [2025-10-20T21:20:49.071066] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T21:20:49.690494] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:20:49.851345] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-04. [2025-10-20T21:20:50.617819] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:20:50.651330] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-07. [2025-10-20T21:20:51.451360] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-08. [2025-10-20T21:20:51.788815] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T21:20:52.228623] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T21:20:52.251334] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-08. [2025-10-20T21:20:52.789737] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T21:20:53.051353] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-05. [2025-10-20T21:20:53.795445] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T21:20:53.851345] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-09. [2025-10-20T21:20:54.640866] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T21:20:54.651765] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-06. [2025-10-20T21:20:55.418488] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:20:55.451336] [INFO ] [REMOTE ] SUZ: Fetching primary data for 2025-10. [2025-10-20T21:20:56.015545] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T21:20:56.251372] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-09. [2025-10-20T21:20:56.621147] [INFO ] [REMOTE ] SUZ: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T21:20:56.628894] [INFO ] [REMOTE ] SUZ: Full fetch complete (66829 records). [2025-10-20T21:20:56.629060] [DEBUG] [VALIDATOR ] SUZ: Validating primary data. [2025-10-20T21:20:57.060737] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-05. [2025-10-20T21:20:57.064334] [DEBUG] [VALIDATOR ] SUZ: All expected trading days are present. [2025-10-20T21:20:57.064741] [DEBUG] [VALIDATOR ] SUZ: Primary data validation passed. [2025-10-20T21:20:57.066726] [INFO ] [MAIN ] SUZ: Fetched primary data is valid. [2025-10-20T21:20:57.066915] [DEBUG] [LOCAL ] SUZ: Saving primary data. [2025-10-20T21:20:57.068535] [DEBUG] [LOCAL ] SUZ: Trimmed 66829 records to 45627 (keeping 200 days of data) [2025-10-20T21:20:57.340956] [DEBUG] [LOCAL ] SUZ: Primary data saved (45627 records). [2025-10-20T21:20:57.341102] [INFO ] [MAIN ] SUZ: Full history saved. [2025-10-20T21:20:57.341180] [INFO ] [MAIN ] Update Progress: 2026/2367 symbols completed. SUZ took 25.2s. Estimated remaining: 43m 55s. Rate: 7.8 symbols/min [2025-10-20T21:20:57.518515] [INFO ] [MAIN ] SWK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:20:57.567287] [DEBUG] [LOCAL ] SWK: Loaded 53219 existing records from primary storage. [2025-10-20T21:20:57.567586] [INFO ] [MAIN ] SWK: Loaded 53219 existing records. [2025-10-20T21:20:57.762526] [WARN ] [MAIN ] SWK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:20:57.762765] [INFO ] [MAIN ] SWK: Wiping existing primary data before full refresh. [2025-10-20T21:20:57.762816] [INFO ] [LOCAL ] SWK: Starting deletion of primary data. [2025-10-20T21:20:57.763120] [DEBUG] [LOCAL ] SWK: Deleted primary data file. [2025-10-20T21:20:57.763175] [INFO ] [MAIN ] SWK: Starting full primary refresh. [2025-10-20T21:20:57.856023] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-07. [2025-10-20T21:20:58.139400] [INFO ] [REMOTE ] SWK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:20:58.167742] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T21:20:58.651365] [INFO ] [REMOTE ] SUPV: Fetching primary data for 2025-10. [2025-10-20T21:20:58.925920] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T21:20:59.254020] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T21:20:59.451392] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-01. [2025-10-20T21:21:00.251372] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-06. [2025-10-20T21:21:00.397452] [INFO ] [REMOTE ] SUPV: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T21:21:00.405137] [INFO ] [REMOTE ] SUPV: Full fetch complete (68982 records). [2025-10-20T21:21:00.405288] [DEBUG] [VALIDATOR ] SUPV: Validating primary data. [2025-10-20T21:21:00.800441] [DEBUG] [VALIDATOR ] SUPV: All expected trading days are present. [2025-10-20T21:21:00.800535] [DEBUG] [VALIDATOR ] SUPV: Primary data validation passed. [2025-10-20T21:21:00.801554] [INFO ] [MAIN ] SUPV: Fetched primary data is valid. [2025-10-20T21:21:00.801615] [DEBUG] [LOCAL ] SUPV: Saving primary data. [2025-10-20T21:21:00.802894] [DEBUG] [LOCAL ] SUPV: Trimmed 68982 records to 48009 (keeping 200 days of data) [2025-10-20T21:21:01.054447] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-08. [2025-10-20T21:21:01.078870] [DEBUG] [LOCAL ] SUPV: Primary data saved (48009 records). [2025-10-20T21:21:01.079018] [INFO ] [MAIN ] SUPV: Full history saved. [2025-10-20T21:21:01.079098] [INFO ] [MAIN ] Update Progress: 2027/2367 symbols completed. SUPV took 32.0s. Estimated remaining: 43m 46s. Rate: 7.8 symbols/min [2025-10-20T21:21:01.251455] [INFO ] [MAIN ] SWX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:01.282052] [DEBUG] [LOCAL ] SWX: Loaded 31339 existing records from primary storage. [2025-10-20T21:21:01.282309] [INFO ] [MAIN ] SWX: Loaded 31339 existing records. [2025-10-20T21:21:01.461784] [WARN ] [MAIN ] SWX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:01.462149] [INFO ] [MAIN ] SWX: Wiping existing primary data before full refresh. [2025-10-20T21:21:01.462218] [INFO ] [LOCAL ] SWX: Starting deletion of primary data. [2025-10-20T21:21:01.462444] [DEBUG] [LOCAL ] SWX: Deleted primary data file. [2025-10-20T21:21:01.462559] [INFO ] [MAIN ] SWX: Starting full primary refresh. [2025-10-20T21:21:01.607216] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-01 (2.16 seconds). [2025-10-20T21:21:01.853236] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-02. [2025-10-20T21:21:01.879531] [INFO ] [REMOTE ] SWX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:02.651370] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-01. [2025-10-20T21:21:03.610455] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:21:03.620491] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-03. [2025-10-20T21:21:03.978184] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T21:21:04.251445] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-02. [2025-10-20T21:21:05.484801] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T21:21:05.496037] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-04. [2025-10-20T21:21:05.582611] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-06 (5.33 seconds). [2025-10-20T21:21:05.641638] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T21:21:05.851427] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-07. [2025-10-20T21:21:06.108782] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-08 (5.05 seconds). [2025-10-20T21:21:06.651474] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-03. [2025-10-20T21:21:07.451449] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-09. [2025-10-20T21:21:07.595256] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T21:21:07.670943] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T21:21:07.702632] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-03 (1.05 seconds). [2025-10-20T21:21:08.251418] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-04. [2025-10-20T21:21:09.056495] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-08. [2025-10-20T21:21:09.069773] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:21:09.622931] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T21:21:09.851448] [INFO ] [REMOTE ] SVV: Fetching primary data for 2025-10. [2025-10-20T21:21:10.651445] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-05. [2025-10-20T21:21:10.916883] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T21:21:11.451448] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-05. [2025-10-20T21:21:11.667092] [INFO ] [REMOTE ] SVV: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T21:21:11.673733] [INFO ] [REMOTE ] SVV: Full fetch complete (57416 records). [2025-10-20T21:21:11.673913] [DEBUG] [VALIDATOR ] SVV: Validating primary data. [2025-10-20T21:21:12.059201] [DEBUG] [VALIDATOR ] SVV: All expected trading days are present. [2025-10-20T21:21:12.059307] [DEBUG] [VALIDATOR ] SVV: Primary data validation passed. [2025-10-20T21:21:12.060234] [INFO ] [MAIN ] SVV: Fetched primary data is valid. [2025-10-20T21:21:12.060316] [DEBUG] [LOCAL ] SVV: Saving primary data. [2025-10-20T21:21:12.061605] [DEBUG] [LOCAL ] SVV: Trimmed 57416 records to 40326 (keeping 200 days of data) [2025-10-20T21:21:12.194268] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T21:21:12.255680] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-09. [2025-10-20T21:21:12.321198] [DEBUG] [LOCAL ] SVV: Primary data saved (40326 records). [2025-10-20T21:21:12.321333] [INFO ] [MAIN ] SVV: Full history saved. [2025-10-20T21:21:12.321408] [INFO ] [MAIN ] Update Progress: 2028/2367 symbols completed. SVV took 37.8s. Estimated remaining: 43m 39s. Rate: 7.8 symbols/min [2025-10-20T21:21:12.486285] [INFO ] [MAIN ] SWZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:12.486445] [DEBUG] [LOCAL ] SWZ: No existing data file found in primary storage. [2025-10-20T21:21:12.486482] [WARN ] [MAIN ] SWZ: No existing data found, performing full refresh instead. [2025-10-20T21:21:12.486590] [INFO ] [MAIN ] SWZ: Wiping existing primary data before full refresh. [2025-10-20T21:21:12.486630] [INFO ] [LOCAL ] SWZ: Starting deletion of primary data. [2025-10-20T21:21:12.486712] [DEBUG] [LOCAL ] SWZ: No primary data file found to delete. [2025-10-20T21:21:12.486752] [INFO ] [MAIN ] SWZ: Starting full primary refresh. [2025-10-20T21:21:12.841162] [INFO ] [REMOTE ] SWZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:13.051418] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-01. [2025-10-20T21:21:13.416625] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T21:21:13.642293] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T21:21:13.851402] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-06. [2025-10-20T21:21:14.084787] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-09 (1.83 seconds). [2025-10-20T21:21:14.651382] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-02. [2025-10-20T21:21:15.068123] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-02 (0.42 seconds). [2025-10-20T21:21:15.451437] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-06. [2025-10-20T21:21:15.695584] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T21:21:16.251379] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-03. [2025-10-20T21:21:16.847113] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-06 (1.40 seconds). [2025-10-20T21:21:17.051398] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-07. [2025-10-20T21:21:17.851423] [INFO ] [REMOTE ] SW: Fetching primary data for 2025-10. [2025-10-20T21:21:18.651410] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-07. [2025-10-20T21:21:18.904451] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-07 (1.85 seconds). [2025-10-20T21:21:19.451412] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-08. [2025-10-20T21:21:19.547178] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-03 (3.30 seconds). [2025-10-20T21:21:19.570384] [INFO ] [REMOTE ] SW: Incremental fetch complete for 2025-10 (1.72 seconds). [2025-10-20T21:21:19.578124] [INFO ] [REMOTE ] SW: Full fetch complete (77085 records). [2025-10-20T21:21:19.578266] [DEBUG] [VALIDATOR ] SW: Validating primary data. [2025-10-20T21:21:19.968952] [DEBUG] [VALIDATOR ] SW: All expected trading days are present. [2025-10-20T21:21:19.969030] [DEBUG] [VALIDATOR ] SW: Primary data validation passed. [2025-10-20T21:21:19.970141] [INFO ] [MAIN ] SW: Fetched primary data is valid. [2025-10-20T21:21:19.970249] [DEBUG] [LOCAL ] SW: Saving primary data. [2025-10-20T21:21:19.971914] [DEBUG] [LOCAL ] SW: Trimmed 77085 records to 53179 (keeping 200 days of data) [2025-10-20T21:21:20.211194] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:21:20.252806] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-04. [2025-10-20T21:21:20.329095] [DEBUG] [LOCAL ] SW: Primary data saved (53179 records). [2025-10-20T21:21:20.329214] [INFO ] [MAIN ] SW: Full history saved. [2025-10-20T21:21:20.329547] [INFO ] [MAIN ] Update Progress: 2029/2367 symbols completed. SW took 43.3s. Estimated remaining: 43m 31s. Rate: 7.8 symbols/min [2025-10-20T21:21:20.495491] [INFO ] [MAIN ] SXC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:20.533008] [DEBUG] [LOCAL ] SXC: Loaded 42228 existing records from primary storage. [2025-10-20T21:21:20.533283] [INFO ] [MAIN ] SXC: Loaded 42228 existing records. [2025-10-20T21:21:20.705638] [WARN ] [MAIN ] SXC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:20.705865] [INFO ] [MAIN ] SXC: Wiping existing primary data before full refresh. [2025-10-20T21:21:20.705918] [INFO ] [LOCAL ] SXC: Starting deletion of primary data. [2025-10-20T21:21:20.706190] [DEBUG] [LOCAL ] SXC: Deleted primary data file. [2025-10-20T21:21:20.706245] [INFO ] [MAIN ] SXC: Starting full primary refresh. [2025-10-20T21:21:21.051681] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-08. [2025-10-20T21:21:21.065127] [INFO ] [REMOTE ] SXC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:21.095218] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T21:21:21.216860] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-08 (1.77 seconds). [2025-10-20T21:21:21.851417] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-05. [2025-10-20T21:21:22.348767] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:21:22.533393] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-05 (0.68 seconds). [2025-10-20T21:21:22.651364] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-09. [2025-10-20T21:21:23.451392] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-01. [2025-10-20T21:21:24.000041] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T21:21:24.251400] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-09. [2025-10-20T21:21:25.051420] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-06. [2025-10-20T21:21:25.091678] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T21:21:25.557487] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-06 (0.51 seconds). [2025-10-20T21:21:25.851378] [INFO ] [REMOTE ] SWX: Fetching primary data for 2025-10. [2025-10-20T21:21:26.651381] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-07. [2025-10-20T21:21:27.049735] [INFO ] [REMOTE ] SWX: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T21:21:27.056271] [INFO ] [REMOTE ] SWX: Full fetch complete (43953 records). [2025-10-20T21:21:27.056532] [DEBUG] [VALIDATOR ] SWX: Validating primary data. [2025-10-20T21:21:27.173639] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T21:21:27.460544] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-08. [2025-10-20T21:21:27.464385] [DEBUG] [VALIDATOR ] SWX: All expected trading days are present. [2025-10-20T21:21:27.464480] [DEBUG] [VALIDATOR ] SWX: Primary data validation passed. [2025-10-20T21:21:27.465593] [INFO ] [MAIN ] SWX: Fetched primary data is valid. [2025-10-20T21:21:27.465830] [DEBUG] [LOCAL ] SWX: Saving primary data. [2025-10-20T21:21:27.466925] [DEBUG] [LOCAL ] SWX: Trimmed 43953 records to 30791 (keeping 200 days of data) [2025-10-20T21:21:27.678439] [DEBUG] [LOCAL ] SWX: Primary data saved (30791 records). [2025-10-20T21:21:27.678592] [INFO ] [MAIN ] SWX: Full history saved. [2025-10-20T21:21:27.678700] [INFO ] [MAIN ] Update Progress: 2030/2367 symbols completed. SWX took 26.6s. Estimated remaining: 43m 24s. Rate: 7.8 symbols/min [2025-10-20T21:21:27.864144] [INFO ] [MAIN ] SXI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:27.877750] [DEBUG] [LOCAL ] SXI: Loaded 12191 existing records from primary storage. [2025-10-20T21:21:27.878027] [INFO ] [MAIN ] SXI: Loaded 12191 existing records. [2025-10-20T21:21:28.050708] [WARN ] [MAIN ] SXI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:28.050940] [INFO ] [MAIN ] SXI: Wiping existing primary data before full refresh. [2025-10-20T21:21:28.050987] [INFO ] [LOCAL ] SXI: Starting deletion of primary data. [2025-10-20T21:21:28.051164] [DEBUG] [LOCAL ] SXI: Deleted primary data file. [2025-10-20T21:21:28.051219] [INFO ] [MAIN ] SXI: Starting full primary refresh. [2025-10-20T21:21:28.256224] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-02. [2025-10-20T21:21:28.364749] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-08 (0.90 seconds). [2025-10-20T21:21:28.419754] [INFO ] [REMOTE ] SXI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:29.051363] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-01. [2025-10-20T21:21:29.778135] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-09 (5.53 seconds). [2025-10-20T21:21:29.795744] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T21:21:29.851371] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-02. [2025-10-20T21:21:29.888931] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:21:30.651405] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-03. [2025-10-20T21:21:30.704529] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T21:21:31.451429] [INFO ] [REMOTE ] SWK: Fetching primary data for 2025-10. [2025-10-20T21:21:32.145234] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-03 (1.49 seconds). [2025-10-20T21:21:32.251351] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-03. [2025-10-20T21:21:33.040344] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-03 (0.79 seconds). [2025-10-20T21:21:33.051329] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-04. [2025-10-20T21:21:33.145565] [INFO ] [REMOTE ] SWK: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T21:21:33.153149] [INFO ] [REMOTE ] SWK: Full fetch complete (75549 records). [2025-10-20T21:21:33.153270] [DEBUG] [VALIDATOR ] SWK: Validating primary data. [2025-10-20T21:21:33.534730] [DEBUG] [VALIDATOR ] SWK: All expected trading days are present. [2025-10-20T21:21:33.534819] [DEBUG] [VALIDATOR ] SWK: Primary data validation passed. [2025-10-20T21:21:33.535901] [INFO ] [MAIN ] SWK: Fetched primary data is valid. [2025-10-20T21:21:33.535966] [DEBUG] [LOCAL ] SWK: Saving primary data. [2025-10-20T21:21:33.537350] [DEBUG] [LOCAL ] SWK: Trimmed 75549 records to 52434 (keeping 200 days of data) [2025-10-20T21:21:33.857967] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-09. [2025-10-20T21:21:33.859681] [DEBUG] [LOCAL ] SWK: Primary data saved (52434 records). [2025-10-20T21:21:33.860762] [INFO ] [MAIN ] SWK: Full history saved. [2025-10-20T21:21:33.860862] [INFO ] [MAIN ] Update Progress: 2031/2367 symbols completed. SWK took 36.5s. Estimated remaining: 43m 16s. Rate: 7.8 symbols/min [2025-10-20T21:21:34.042593] [INFO ] [MAIN ] SXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:34.043691] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T21:21:34.075861] [DEBUG] [LOCAL ] SXT: Loaded 30246 existing records from primary storage. [2025-10-20T21:21:34.076154] [INFO ] [MAIN ] SXT: Loaded 30246 existing records. [2025-10-20T21:21:34.249440] [WARN ] [MAIN ] SXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:34.249696] [INFO ] [MAIN ] SXT: Wiping existing primary data before full refresh. [2025-10-20T21:21:34.249754] [INFO ] [LOCAL ] SXT: Starting deletion of primary data. [2025-10-20T21:21:34.249930] [DEBUG] [LOCAL ] SXT: Deleted primary data file. [2025-10-20T21:21:34.249967] [INFO ] [MAIN ] SXT: Starting full primary refresh. [2025-10-20T21:21:34.579090] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T21:21:34.614122] [INFO ] [REMOTE ] SXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:34.651335] [INFO ] [REMOTE ] SWZ: Fetching primary data for 2025-10. [2025-10-20T21:21:35.266505] [INFO ] [REMOTE ] SWZ: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T21:21:35.267887] [INFO ] [REMOTE ] SWZ: Full fetch complete (8283 records). [2025-10-20T21:21:35.267955] [DEBUG] [VALIDATOR ] SWZ: Validating primary data. [2025-10-20T21:21:35.457153] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-01. [2025-10-20T21:21:35.624333] [DEBUG] [VALIDATOR ] SWZ: All expected trading days are present. [2025-10-20T21:21:35.624405] [DEBUG] [VALIDATOR ] SWZ: Primary data validation passed. [2025-10-20T21:21:35.624607] [INFO ] [MAIN ] SWZ: Fetched primary data is valid. [2025-10-20T21:21:35.624664] [DEBUG] [LOCAL ] SWZ: Saving primary data. [2025-10-20T21:21:35.625072] [DEBUG] [LOCAL ] SWZ: Trimmed 8283 records to 6016 (keeping 200 days of data) [2025-10-20T21:21:35.657428] [DEBUG] [LOCAL ] SWZ: Primary data saved (6016 records). [2025-10-20T21:21:35.657528] [INFO ] [MAIN ] SWZ: Full history saved. [2025-10-20T21:21:35.657589] [INFO ] [MAIN ] Update Progress: 2032/2367 symbols completed. SWZ took 23.3s. Estimated remaining: 43m 7s. Rate: 7.8 symbols/min [2025-10-20T21:21:35.823197] [INFO ] [MAIN ] SYF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:35.872132] [DEBUG] [LOCAL ] SYF: Loaded 54234 existing records from primary storage. [2025-10-20T21:21:35.872421] [INFO ] [MAIN ] SYF: Loaded 54234 existing records. [2025-10-20T21:21:36.046918] [WARN ] [MAIN ] SYF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:36.047110] [INFO ] [MAIN ] SYF: Wiping existing primary data before full refresh. [2025-10-20T21:21:36.047157] [INFO ] [LOCAL ] SYF: Starting deletion of primary data. [2025-10-20T21:21:36.047341] [DEBUG] [LOCAL ] SYF: Deleted primary data file. [2025-10-20T21:21:36.047382] [INFO ] [MAIN ] SYF: Starting full primary refresh. [2025-10-20T21:21:36.256539] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-05. [2025-10-20T21:21:36.428231] [INFO ] [REMOTE ] SYF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:36.630685] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:21:37.051407] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-04. [2025-10-20T21:21:37.214358] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-05 (0.96 seconds). [2025-10-20T21:21:37.851372] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-06. [2025-10-20T21:21:38.451336] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T21:21:38.625248] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T21:21:38.651335] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-05. [2025-10-20T21:21:39.451380] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-01. [2025-10-20T21:21:40.257556] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-07. [2025-10-20T21:21:40.272858] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T21:21:41.051357] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-02. [2025-10-20T21:21:41.164567] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T21:21:41.255872] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T21:21:41.851382] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-08. [2025-10-20T21:21:42.114965] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T21:21:42.510937] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T21:21:42.651398] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-09. [2025-10-20T21:21:43.451490] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-06. [2025-10-20T21:21:43.479096] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:21:44.251382] [INFO ] [REMOTE ] SXI: Fetching primary data for 2025-10. [2025-10-20T21:21:45.051822] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-03. [2025-10-20T21:21:45.055051] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:21:45.139186] [INFO ] [REMOTE ] SXI: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T21:21:45.141985] [INFO ] [REMOTE ] SXI: Full fetch complete (15739 records). [2025-10-20T21:21:45.142058] [DEBUG] [VALIDATOR ] SXI: Validating primary data. [2025-10-20T21:21:45.522660] [DEBUG] [VALIDATOR ] SXI: All expected trading days are present. [2025-10-20T21:21:45.522754] [DEBUG] [VALIDATOR ] SXI: Primary data validation passed. [2025-10-20T21:21:45.523131] [INFO ] [MAIN ] SXI: Fetched primary data is valid. [2025-10-20T21:21:45.523170] [DEBUG] [LOCAL ] SXI: Saving primary data. [2025-10-20T21:21:45.523768] [DEBUG] [LOCAL ] SXI: Trimmed 15739 records to 12056 (keeping 200 days of data) [2025-10-20T21:21:45.600097] [DEBUG] [LOCAL ] SXI: Primary data saved (12056 records). [2025-10-20T21:21:45.600235] [INFO ] [MAIN ] SXI: Full history saved. [2025-10-20T21:21:45.600316] [INFO ] [MAIN ] Update Progress: 2033/2367 symbols completed. SXI took 17.9s. Estimated remaining: 42m 60s. Rate: 7.8 symbols/min [2025-10-20T21:21:45.792396] [INFO ] [MAIN ] SYK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:45.838377] [DEBUG] [LOCAL ] SYK: Loaded 47627 existing records from primary storage. [2025-10-20T21:21:45.838717] [INFO ] [MAIN ] SYK: Loaded 47627 existing records. [2025-10-20T21:21:45.856548] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-07. [2025-10-20T21:21:46.038976] [WARN ] [MAIN ] SYK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:46.039263] [INFO ] [MAIN ] SYK: Wiping existing primary data before full refresh. [2025-10-20T21:21:46.039364] [INFO ] [LOCAL ] SYK: Starting deletion of primary data. [2025-10-20T21:21:46.039600] [DEBUG] [LOCAL ] SYK: Deleted primary data file. [2025-10-20T21:21:46.039691] [INFO ] [MAIN ] SYK: Starting full primary refresh. [2025-10-20T21:21:46.357102] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T21:21:46.459467] [INFO ] [REMOTE ] SYK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:46.651391] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-02. [2025-10-20T21:21:47.398842] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-07 (1.54 seconds). [2025-10-20T21:21:47.451383] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-04. [2025-10-20T21:21:48.251425] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-08. [2025-10-20T21:21:48.351911] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T21:21:48.956985] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T21:21:49.051401] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-01. [2025-10-20T21:21:49.779838] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-08 (1.53 seconds). [2025-10-20T21:21:49.851398] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-03. [2025-10-20T21:21:50.651436] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-05. [2025-10-20T21:21:50.958190] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T21:21:51.451431] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-09. [2025-10-20T21:21:51.669363] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:21:51.696061] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T21:21:52.251373] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-04. [2025-10-20T21:21:53.051388] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-02. [2025-10-20T21:21:53.113860] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T21:21:53.851381] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-06. [2025-10-20T21:21:54.104085] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T21:21:54.651358] [INFO ] [REMOTE ] SXC: Fetching primary data for 2025-10. [2025-10-20T21:21:54.767816] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:21:54.989070] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T21:21:55.451407] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-03. [2025-10-20T21:21:56.160729] [INFO ] [REMOTE ] SXC: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T21:21:56.167213] [INFO ] [REMOTE ] SXC: Full fetch complete (61037 records). [2025-10-20T21:21:56.167356] [DEBUG] [VALIDATOR ] SXC: Validating primary data. [2025-10-20T21:21:56.259970] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-05. [2025-10-20T21:21:56.552806] [DEBUG] [VALIDATOR ] SXC: All expected trading days are present. [2025-10-20T21:21:56.552905] [DEBUG] [VALIDATOR ] SXC: Primary data validation passed. [2025-10-20T21:21:56.553975] [INFO ] [MAIN ] SXC: Fetched primary data is valid. [2025-10-20T21:21:56.554040] [DEBUG] [LOCAL ] SXC: Saving primary data. [2025-10-20T21:21:56.555359] [DEBUG] [LOCAL ] SXC: Trimmed 61037 records to 41615 (keeping 200 days of data) [2025-10-20T21:21:56.809341] [DEBUG] [LOCAL ] SXC: Primary data saved (41615 records). [2025-10-20T21:21:56.809499] [INFO ] [MAIN ] SXC: Full history saved. [2025-10-20T21:21:56.809883] [INFO ] [MAIN ] Update Progress: 2034/2367 symbols completed. SXC took 36.5s. Estimated remaining: 42m 52s. Rate: 7.8 symbols/min [2025-10-20T21:21:56.983325] [INFO ] [MAIN ] SYY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:21:57.032258] [DEBUG] [LOCAL ] SYY: Loaded 53990 existing records from primary storage. [2025-10-20T21:21:57.032579] [INFO ] [MAIN ] SYY: Loaded 53990 existing records. [2025-10-20T21:21:57.064446] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-07. [2025-10-20T21:21:57.212922] [WARN ] [MAIN ] SYY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:21:57.213157] [INFO ] [MAIN ] SYY: Wiping existing primary data before full refresh. [2025-10-20T21:21:57.213258] [INFO ] [LOCAL ] SYY: Starting deletion of primary data. [2025-10-20T21:21:57.213690] [DEBUG] [LOCAL ] SYY: Deleted primary data file. [2025-10-20T21:21:57.213818] [INFO ] [MAIN ] SYY: Starting full primary refresh. [2025-10-20T21:21:57.598702] [INFO ] [REMOTE ] SYY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:21:57.594423] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T21:21:57.851367] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-04. [2025-10-20T21:21:58.179426] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-05 (1.92 seconds). [2025-10-20T21:21:58.569695] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T21:21:58.651434] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-06. [2025-10-20T21:21:59.451341] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-08. [2025-10-20T21:21:59.814355] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T21:22:00.251387] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-05. [2025-10-20T21:22:00.370150] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T21:22:01.051489] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-01. [2025-10-20T21:22:01.097101] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:22:01.851407] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-09. [2025-10-20T21:22:02.069991] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T21:22:02.651389] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-07. [2025-10-20T21:22:03.002407] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T21:22:03.308088] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T21:22:03.451333] [INFO ] [REMOTE ] SXT: Fetching primary data for 2025-10. [2025-10-20T21:22:04.251394] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-06. [2025-10-20T21:22:04.649713] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T21:22:04.704118] [INFO ] [REMOTE ] SXT: Incremental fetch complete for 2025-10 (1.25 seconds). [2025-10-20T21:22:04.708725] [INFO ] [REMOTE ] SXT: Full fetch complete (39606 records). [2025-10-20T21:22:04.708806] [DEBUG] [VALIDATOR ] SXT: Validating primary data. [2025-10-20T21:22:05.056639] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-08. [2025-10-20T21:22:05.107962] [DEBUG] [VALIDATOR ] SXT: All expected trading days are present. [2025-10-20T21:22:05.108073] [DEBUG] [VALIDATOR ] SXT: Primary data validation passed. [2025-10-20T21:22:05.108906] [INFO ] [MAIN ] SXT: Fetched primary data is valid. [2025-10-20T21:22:05.108968] [DEBUG] [LOCAL ] SXT: Saving primary data. [2025-10-20T21:22:05.109974] [DEBUG] [LOCAL ] SXT: Trimmed 39606 records to 29870 (keeping 200 days of data) [2025-10-20T21:22:05.291354] [DEBUG] [LOCAL ] SXT: Primary data saved (29870 records). [2025-10-20T21:22:05.291487] [INFO ] [MAIN ] SXT: Full history saved. [2025-10-20T21:22:05.291563] [INFO ] [MAIN ] Update Progress: 2035/2367 symbols completed. SXT took 31.4s. Estimated remaining: 42m 45s. Rate: 7.8 symbols/min [2025-10-20T21:22:05.471849] [INFO ] [MAIN ] T: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:05.521338] [DEBUG] [LOCAL ] T: Loaded 54578 existing records from primary storage. [2025-10-20T21:22:05.521638] [INFO ] [MAIN ] T: Loaded 54578 existing records. [2025-10-20T21:22:05.697688] [WARN ] [MAIN ] T: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:05.697882] [INFO ] [MAIN ] T: Wiping existing primary data before full refresh. [2025-10-20T21:22:05.697944] [INFO ] [LOCAL ] T: Starting deletion of primary data. [2025-10-20T21:22:05.698178] [DEBUG] [LOCAL ] T: Deleted primary data file. [2025-10-20T21:22:05.698233] [INFO ] [MAIN ] T: Starting full primary refresh. [2025-10-20T21:22:05.852609] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-02. [2025-10-20T21:22:06.070533] [INFO ] [REMOTE ] T: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:06.095170] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T21:22:06.651349] [INFO ] [REMOTE ] T: Fetching primary data for 2025-01. [2025-10-20T21:22:06.952228] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T21:22:07.418155] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:22:07.451364] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-03. [2025-10-20T21:22:08.251362] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-07. [2025-10-20T21:22:08.959720] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:22:09.051388] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-09. [2025-10-20T21:22:09.150926] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-01 (2.50 seconds). [2025-10-20T21:22:09.851438] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-04. [2025-10-20T21:22:10.049239] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T21:22:10.651457] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-08. [2025-10-20T21:22:10.840432] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T21:22:11.451441] [INFO ] [REMOTE ] SYF: Fetching primary data for 2025-10. [2025-10-20T21:22:11.691374] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T21:22:12.251472] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-05. [2025-10-20T21:22:12.395606] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T21:22:13.051387] [INFO ] [REMOTE ] T: Fetching primary data for 2025-02. [2025-10-20T21:22:13.120570] [INFO ] [REMOTE ] SYF: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T21:22:13.128696] [INFO ] [REMOTE ] SYF: Full fetch complete (77565 records). [2025-10-20T21:22:13.128868] [DEBUG] [VALIDATOR ] SYF: Validating primary data. [2025-10-20T21:22:13.511467] [DEBUG] [VALIDATOR ] SYF: All expected trading days are present. [2025-10-20T21:22:13.511543] [DEBUG] [VALIDATOR ] SYF: Primary data validation passed. [2025-10-20T21:22:13.512552] [INFO ] [MAIN ] SYF: Fetched primary data is valid. [2025-10-20T21:22:13.512614] [DEBUG] [LOCAL ] SYF: Saving primary data. [2025-10-20T21:22:13.513962] [DEBUG] [LOCAL ] SYF: Trimmed 77565 records to 53453 (keeping 200 days of data) [2025-10-20T21:22:13.827994] [DEBUG] [LOCAL ] SYF: Primary data saved (53453 records). [2025-10-20T21:22:13.828145] [INFO ] [MAIN ] SYF: Full history saved. [2025-10-20T21:22:13.828245] [INFO ] [MAIN ] Update Progress: 2036/2367 symbols completed. SYF took 38.2s. Estimated remaining: 42m 37s. Rate: 7.8 symbols/min [2025-10-20T21:22:13.859136] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-09. [2025-10-20T21:22:14.002485] [INFO ] [MAIN ] TAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:14.042153] [DEBUG] [LOCAL ] TAC: Loaded 43866 existing records from primary storage. [2025-10-20T21:22:14.042586] [INFO ] [MAIN ] TAC: Loaded 43866 existing records. [2025-10-20T21:22:14.223093] [WARN ] [MAIN ] TAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:14.223365] [INFO ] [MAIN ] TAC: Wiping existing primary data before full refresh. [2025-10-20T21:22:14.223433] [INFO ] [LOCAL ] TAC: Starting deletion of primary data. [2025-10-20T21:22:14.223630] [DEBUG] [LOCAL ] TAC: Deleted primary data file. [2025-10-20T21:22:14.223799] [INFO ] [MAIN ] TAC: Starting full primary refresh. [2025-10-20T21:22:14.356596] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-05 (2.11 seconds). [2025-10-20T21:22:14.618893] [INFO ] [REMOTE ] TAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:14.651377] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-01. [2025-10-20T21:22:15.101560] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T21:22:15.451378] [INFO ] [REMOTE ] T: Fetching primary data for 2025-03. [2025-10-20T21:22:16.211299] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-10-20T21:22:16.251341] [INFO ] [REMOTE ] SYK: Fetching primary data for 2025-10. [2025-10-20T21:22:16.318252] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:22:17.051424] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-06. [2025-10-20T21:22:17.851356] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-02. [2025-10-20T21:22:17.908261] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-03 (2.46 seconds). [2025-10-20T21:22:17.938453] [INFO ] [REMOTE ] SYK: Incremental fetch complete for 2025-10 (1.69 seconds). [2025-10-20T21:22:17.945569] [INFO ] [REMOTE ] SYK: Full fetch complete (69183 records). [2025-10-20T21:22:17.945707] [DEBUG] [VALIDATOR ] SYK: Validating primary data. [2025-10-20T21:22:18.324847] [DEBUG] [VALIDATOR ] SYK: All expected trading days are present. [2025-10-20T21:22:18.324928] [DEBUG] [VALIDATOR ] SYK: Primary data validation passed. [2025-10-20T21:22:18.325969] [INFO ] [MAIN ] SYK: Fetched primary data is valid. [2025-10-20T21:22:18.326031] [DEBUG] [LOCAL ] SYK: Saving primary data. [2025-10-20T21:22:18.327310] [DEBUG] [LOCAL ] SYK: Trimmed 69183 records to 46886 (keeping 200 days of data) [2025-10-20T21:22:18.614281] [DEBUG] [LOCAL ] SYK: Primary data saved (46886 records). [2025-10-20T21:22:18.614402] [INFO ] [MAIN ] SYK: Full history saved. [2025-10-20T21:22:18.614474] [INFO ] [MAIN ] Update Progress: 2037/2367 symbols completed. SYK took 33.0s. Estimated remaining: 42m 29s. Rate: 7.8 symbols/min [2025-10-20T21:22:18.651336] [INFO ] [REMOTE ] T: Fetching primary data for 2025-04. [2025-10-20T21:22:18.942738] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T21:22:19.189612] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T21:22:19.451370] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-07. [2025-10-20T21:22:20.251434] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-03. [2025-10-20T21:22:21.089713] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-04 (2.44 seconds). [2025-10-20T21:22:21.101930] [INFO ] [REMOTE ] T: Fetching primary data for 2025-05. [2025-10-20T21:22:21.345203] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:22:21.851446] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-08. [2025-10-20T21:22:21.880285] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:22:22.651403] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-04. [2025-10-20T21:22:23.399538] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-05 (2.30 seconds). [2025-10-20T21:22:23.451341] [INFO ] [REMOTE ] T: Fetching primary data for 2025-06. [2025-10-20T21:22:23.635589] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:22:24.245045] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T21:22:24.253799] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-09. [2025-10-20T21:22:25.051427] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-05. [2025-10-20T21:22:25.660965] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-06 (2.21 seconds). [2025-10-20T21:22:25.851422] [INFO ] [REMOTE ] T: Fetching primary data for 2025-07. [2025-10-20T21:22:26.173013] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-09 (1.92 seconds). [2025-10-20T21:22:26.531055] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T21:22:26.651421] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-06. [2025-10-20T21:22:27.451380] [INFO ] [REMOTE ] SYY: Fetching primary data for 2025-10. [2025-10-20T21:22:28.214555] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:22:28.252106] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-07. [2025-10-20T21:22:28.254698] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-07 (2.40 seconds). [2025-10-20T21:22:29.051409] [INFO ] [REMOTE ] T: Fetching primary data for 2025-08. [2025-10-20T21:22:29.276142] [INFO ] [REMOTE ] SYY: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T21:22:29.284024] [INFO ] [REMOTE ] SYY: Full fetch complete (77222 records). [2025-10-20T21:22:29.284155] [DEBUG] [VALIDATOR ] SYY: Validating primary data. [2025-10-20T21:22:29.680327] [DEBUG] [VALIDATOR ] SYY: All expected trading days are present. [2025-10-20T21:22:29.680424] [DEBUG] [VALIDATOR ] SYY: Primary data validation passed. [2025-10-20T21:22:29.681595] [INFO ] [MAIN ] SYY: Fetched primary data is valid. [2025-10-20T21:22:29.681719] [DEBUG] [LOCAL ] SYY: Saving primary data. [2025-10-20T21:22:29.683174] [DEBUG] [LOCAL ] SYY: Trimmed 77222 records to 53207 (keeping 200 days of data) [2025-10-20T21:22:30.013230] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T21:22:30.033871] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-08. [2025-10-20T21:22:30.046397] [DEBUG] [LOCAL ] SYY: Primary data saved (53207 records). [2025-10-20T21:22:30.046521] [INFO ] [MAIN ] SYY: Full history saved. [2025-10-20T21:22:30.046824] [INFO ] [MAIN ] Update Progress: 2038/2367 symbols completed. SYY took 33.2s. Estimated remaining: 42m 22s. Rate: 7.8 symbols/min [2025-10-20T21:22:31.417835] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-08 (2.37 seconds). [2025-10-20T21:22:31.429916] [INFO ] [REMOTE ] T: Fetching primary data for 2025-09. [2025-10-20T21:22:31.469000] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:22:31.478077] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-09. [2025-10-20T21:22:32.933779] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T21:22:32.942448] [INFO ] [REMOTE ] TAC: Fetching primary data for 2025-10. [2025-10-20T21:22:34.049754] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-09 (2.62 seconds). [2025-10-20T21:22:34.061169] [INFO ] [REMOTE ] T: Fetching primary data for 2025-10. [2025-10-20T21:22:34.574129] [INFO ] [REMOTE ] TAC: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T21:22:34.581429] [INFO ] [REMOTE ] TAC: Full fetch complete (64666 records). [2025-10-20T21:22:34.581534] [DEBUG] [VALIDATOR ] TAC: Validating primary data. [2025-10-20T21:22:34.971619] [DEBUG] [VALIDATOR ] TAC: All expected trading days are present. [2025-10-20T21:22:34.971724] [DEBUG] [VALIDATOR ] TAC: Primary data validation passed. [2025-10-20T21:22:34.972704] [INFO ] [MAIN ] TAC: Fetched primary data is valid. [2025-10-20T21:22:34.972764] [DEBUG] [LOCAL ] TAC: Saving primary data. [2025-10-20T21:22:34.973965] [DEBUG] [LOCAL ] TAC: Trimmed 64666 records to 43339 (keeping 200 days of data) [2025-10-20T21:22:35.226159] [DEBUG] [LOCAL ] TAC: Primary data saved (43339 records). [2025-10-20T21:22:35.226281] [INFO ] [MAIN ] TAC: Full history saved. [2025-10-20T21:22:35.226354] [INFO ] [MAIN ] Update Progress: 2039/2367 symbols completed. TAC took 21.4s. Estimated remaining: 42m 14s. Rate: 7.8 symbols/min [2025-10-20T21:22:36.461312] [INFO ] [REMOTE ] T: Incremental fetch complete for 2025-10 (2.40 seconds). [2025-10-20T21:22:36.469707] [INFO ] [REMOTE ] T: Full fetch complete (77978 records). [2025-10-20T21:22:36.469851] [DEBUG] [VALIDATOR ] T: Validating primary data. [2025-10-20T21:22:36.861579] [DEBUG] [VALIDATOR ] T: All expected trading days are present. [2025-10-20T21:22:36.861681] [DEBUG] [VALIDATOR ] T: Primary data validation passed. [2025-10-20T21:22:36.862780] [INFO ] [MAIN ] T: Fetched primary data is valid. [2025-10-20T21:22:36.862840] [DEBUG] [LOCAL ] T: Saving primary data. [2025-10-20T21:22:36.864210] [DEBUG] [LOCAL ] T: Trimmed 77978 records to 53798 (keeping 200 days of data) [2025-10-20T21:22:37.185420] [DEBUG] [LOCAL ] T: Primary data saved (53798 records). [2025-10-20T21:22:37.185547] [INFO ] [MAIN ] T: Full history saved. [2025-10-20T21:22:37.185623] [INFO ] [MAIN ] Update Progress: 2040/2367 symbols completed. T took 31.9s. Estimated remaining: 42m 5s. Rate: 7.8 symbols/min [2025-10-20T21:22:37.809566] [INFO ] [MAIN ] TAP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:37.874345] [INFO ] [MAIN ] TALO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:37.890977] [INFO ] [MAIN ] TAK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:37.911793] [INFO ] [MAIN ] TAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:22:38.021301] [DEBUG] [LOCAL ] TAP: Loaded 53723 existing records from primary storage. [2025-10-20T21:22:38.023056] [INFO ] [MAIN ] TAP: Loaded 53723 existing records. [2025-10-20T21:22:38.022823] [DEBUG] [LOCAL ] TALO: Loaded 51714 existing records from primary storage. [2025-10-20T21:22:38.054428] [INFO ] [MAIN ] TALO: Loaded 51714 existing records. [2025-10-20T21:22:38.161147] [DEBUG] [LOCAL ] TAK: Loaded 50897 existing records from primary storage. [2025-10-20T21:22:38.184753] [INFO ] [MAIN ] TAK: Loaded 50897 existing records. [2025-10-20T21:22:38.260216] [DEBUG] [LOCAL ] TAL: Loaded 52497 existing records from primary storage. [2025-10-20T21:22:38.298735] [INFO ] [MAIN ] TAL: Loaded 52497 existing records. [2025-10-20T21:22:38.483823] [WARN ] [MAIN ] TAP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:38.499877] [INFO ] [MAIN ] TAP: Wiping existing primary data before full refresh. [2025-10-20T21:22:38.505146] [INFO ] [LOCAL ] TAP: Starting deletion of primary data. [2025-10-20T21:22:38.652377] [DEBUG] [LOCAL ] TAP: Deleted primary data file. [2025-10-20T21:22:38.648392] [WARN ] [MAIN ] TALO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:38.674917] [INFO ] [MAIN ] TAP: Starting full primary refresh. [2025-10-20T21:22:38.685351] [INFO ] [MAIN ] TALO: Wiping existing primary data before full refresh. [2025-10-20T21:22:38.788932] [INFO ] [LOCAL ] TALO: Starting deletion of primary data. [2025-10-20T21:22:38.846713] [WARN ] [MAIN ] TAK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:38.861047] [DEBUG] [LOCAL ] TALO: Deleted primary data file. [2025-10-20T21:22:38.893451] [INFO ] [MAIN ] TALO: Starting full primary refresh. [2025-10-20T21:22:38.938946] [WARN ] [MAIN ] TAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:22:38.995816] [INFO ] [MAIN ] TAK: Wiping existing primary data before full refresh. [2025-10-20T21:22:39.001473] [INFO ] [MAIN ] TAL: Wiping existing primary data before full refresh. [2025-10-20T21:22:39.024315] [INFO ] [LOCAL ] TAK: Starting deletion of primary data. [2025-10-20T21:22:39.045114] [INFO ] [LOCAL ] TAL: Starting deletion of primary data. [2025-10-20T21:22:39.070705] [DEBUG] [LOCAL ] TAK: Deleted primary data file. [2025-10-20T21:22:39.092906] [DEBUG] [LOCAL ] TAL: Deleted primary data file. [2025-10-20T21:22:39.111805] [INFO ] [MAIN ] TAK: Starting full primary refresh. [2025-10-20T21:22:39.111902] [INFO ] [MAIN ] TAL: Starting full primary refresh. [2025-10-20T21:22:39.891291] [INFO ] [REMOTE ] TAP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:39.939193] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-01. [2025-10-20T21:22:40.215982] [INFO ] [REMOTE ] TALO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:40.249948] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-01. [2025-10-20T21:22:40.324620] [INFO ] [REMOTE ] TAK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:40.352065] [INFO ] [REMOTE ] TAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:22:40.352390] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-01. [2025-10-20T21:22:40.346772] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-01. [2025-10-20T21:22:41.745866] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T21:22:41.758796] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-02. [2025-10-20T21:22:41.853878] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-01 (1.51 seconds). [2025-10-20T21:22:41.863819] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-02. [2025-10-20T21:22:41.948295] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T21:22:41.958244] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-02. [2025-10-20T21:22:42.101293] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:22:42.111567] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-02. [2025-10-20T21:22:43.209750] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T21:22:43.219615] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-03. [2025-10-20T21:22:43.288177] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T21:22:43.299153] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-03. [2025-10-20T21:22:43.376538] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T21:22:43.385919] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-03. [2025-10-20T21:22:43.437334] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-02 (1.33 seconds). [2025-10-20T21:22:43.447462] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-03. [2025-10-20T21:22:44.926506] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:22:44.937908] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-04. [2025-10-20T21:22:44.998891] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T21:22:45.008987] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-04. [2025-10-20T21:22:45.052952] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T21:22:45.064222] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-04. [2025-10-20T21:22:45.249896] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T21:22:45.851430] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-04. [2025-10-20T21:22:46.594736] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T21:22:46.651360] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-05. [2025-10-20T21:22:46.819046] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-04 (1.81 seconds). [2025-10-20T21:22:46.919667] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T21:22:47.451387] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-05. [2025-10-20T21:22:47.769501] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T21:22:48.251388] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-05. [2025-10-20T21:22:48.444088] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T21:22:49.047714] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T21:22:49.057728] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-06. [2025-10-20T21:22:49.851365] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-06. [2025-10-20T21:22:50.132382] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T21:22:50.640194] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:22:50.651326] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-07. [2025-10-20T21:22:51.038211] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T21:22:51.451361] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-07. [2025-10-20T21:22:52.251374] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-05. [2025-10-20T21:22:53.051376] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-06. [2025-10-20T21:23:05.139562] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-07 (13.69 seconds). [2025-10-20T21:23:05.156084] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-08. [2025-10-20T21:23:05.566964] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-07 (14.92 seconds). [2025-10-20T21:23:05.579615] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-08. [2025-10-20T21:23:06.651925] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-06 (13.60 seconds). [2025-10-20T21:23:06.663370] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-07. [2025-10-20T21:23:06.796478] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-05 (14.55 seconds). [2025-10-20T21:23:06.807928] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-06. [2025-10-20T21:23:07.263529] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T21:23:07.275453] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-09. [2025-10-20T21:23:08.456368] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:23:08.467834] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-07. [2025-10-20T21:23:08.533998] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T21:23:08.546216] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-08. [2025-10-20T21:23:08.956888] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T21:23:08.969598] [INFO ] [REMOTE ] TALO: Fetching primary data for 2025-10. [2025-10-20T21:23:09.301895] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-08 (4.15 seconds). [2025-10-20T21:23:09.314870] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-09. [2025-10-20T21:23:10.334916] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T21:23:10.348585] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-08. [2025-10-20T21:23:10.372683] [INFO ] [REMOTE ] TALO: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T21:23:10.381559] [INFO ] [REMOTE ] TALO: Full fetch complete (73449 records). [2025-10-20T21:23:10.381779] [DEBUG] [VALIDATOR ] TALO: Validating primary data. [2025-10-20T21:23:10.726820] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T21:23:10.792294] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-09. [2025-10-20T21:23:10.839943] [DEBUG] [VALIDATOR ] TALO: All expected trading days are present. [2025-10-20T21:23:10.840064] [DEBUG] [VALIDATOR ] TALO: Primary data validation passed. [2025-10-20T21:23:10.841840] [INFO ] [MAIN ] TALO: Fetched primary data is valid. [2025-10-20T21:23:10.841989] [DEBUG] [LOCAL ] TALO: Saving primary data. [2025-10-20T21:23:10.844526] [DEBUG] [LOCAL ] TALO: Trimmed 73449 records to 51031 (keeping 200 days of data) [2025-10-20T21:23:11.163169] [DEBUG] [LOCAL ] TALO: Primary data saved (51031 records). [2025-10-20T21:23:11.163463] [INFO ] [MAIN ] TALO: Full history saved. [2025-10-20T21:23:11.163574] [INFO ] [MAIN ] Update Progress: 2041/2367 symbols completed. TALO took 34.0s. Estimated remaining: 42m 2s. Rate: 7.8 symbols/min [2025-10-20T21:23:11.308045] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-09 (1.99 seconds). [2025-10-20T21:23:11.367894] [INFO ] [REMOTE ] TAL: Fetching primary data for 2025-10. [2025-10-20T21:23:11.397273] [INFO ] [MAIN ] TBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:11.413219] [DEBUG] [LOCAL ] TBB: Loaded 11659 existing records from primary storage. [2025-10-20T21:23:11.413500] [INFO ] [MAIN ] TBB: Loaded 11659 existing records. [2025-10-20T21:23:11.599360] [WARN ] [MAIN ] TBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:11.600219] [INFO ] [MAIN ] TBB: Wiping existing primary data before full refresh. [2025-10-20T21:23:11.600282] [INFO ] [LOCAL ] TBB: Starting deletion of primary data. [2025-10-20T21:23:11.600975] [DEBUG] [LOCAL ] TBB: Deleted primary data file. [2025-10-20T21:23:11.601034] [INFO ] [MAIN ] TBB: Starting full primary refresh. [2025-10-20T21:23:11.997076] [INFO ] [REMOTE ] TBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:11.997506] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-01. [2025-10-20T21:23:12.281019] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T21:23:12.292048] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-09. [2025-10-20T21:23:12.500705] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T21:23:12.512942] [INFO ] [REMOTE ] TAP: Fetching primary data for 2025-10. [2025-10-20T21:23:12.901630] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-01 (0.90 seconds). [2025-10-20T21:23:12.905019] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-02. [2025-10-20T21:23:13.051932] [INFO ] [REMOTE ] TAL: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:23:13.059373] [INFO ] [REMOTE ] TAL: Full fetch complete (75695 records). [2025-10-20T21:23:13.059504] [DEBUG] [VALIDATOR ] TAL: Validating primary data. [2025-10-20T21:23:13.453546] [DEBUG] [VALIDATOR ] TAL: All expected trading days are present. [2025-10-20T21:23:13.453727] [DEBUG] [VALIDATOR ] TAL: Primary data validation passed. [2025-10-20T21:23:13.455357] [INFO ] [MAIN ] TAL: Fetched primary data is valid. [2025-10-20T21:23:13.455447] [DEBUG] [LOCAL ] TAL: Saving primary data. [2025-10-20T21:23:13.457551] [DEBUG] [LOCAL ] TAL: Trimmed 75695 records to 51731 (keeping 200 days of data) [2025-10-20T21:23:13.660712] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T21:23:13.694505] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-03. [2025-10-20T21:23:13.817449] [DEBUG] [LOCAL ] TAL: Primary data saved (51731 records). [2025-10-20T21:23:13.817668] [INFO ] [MAIN ] TAL: Full history saved. [2025-10-20T21:23:13.817795] [INFO ] [MAIN ] Update Progress: 2042/2367 symbols completed. TAL took 36.6s. Estimated remaining: 41m 53s. Rate: 7.8 symbols/min [2025-10-20T21:23:14.020210] [INFO ] [MAIN ] TBBB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:14.056155] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T21:23:14.067754] [INFO ] [REMOTE ] TAK: Fetching primary data for 2025-10. [2025-10-20T21:23:14.103407] [DEBUG] [LOCAL ] TBBB: Loaded 36273 existing records from primary storage. [2025-10-20T21:23:14.103844] [INFO ] [MAIN ] TBBB: Loaded 36273 existing records. [2025-10-20T21:23:14.289372] [WARN ] [MAIN ] TBBB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:14.289845] [INFO ] [MAIN ] TBBB: Wiping existing primary data before full refresh. [2025-10-20T21:23:14.289961] [INFO ] [LOCAL ] TBBB: Starting deletion of primary data. [2025-10-20T21:23:14.298338] [DEBUG] [LOCAL ] TBBB: Deleted primary data file. [2025-10-20T21:23:14.310601] [INFO ] [MAIN ] TBBB: Starting full primary refresh. [2025-10-20T21:23:14.381376] [INFO ] [REMOTE ] TAP: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T21:23:14.441384] [INFO ] [REMOTE ] TAP: Full fetch complete (76510 records). [2025-10-20T21:23:14.451889] [DEBUG] [VALIDATOR ] TAP: Validating primary data. [2025-10-20T21:23:14.783394] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T21:23:14.890585] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-04. [2025-10-20T21:23:15.038932] [INFO ] [REMOTE ] TBBB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:15.063253] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-01. [2025-10-20T21:23:15.176506] [DEBUG] [VALIDATOR ] TAP: All expected trading days are present. [2025-10-20T21:23:15.176606] [DEBUG] [VALIDATOR ] TAP: Primary data validation passed. [2025-10-20T21:23:15.177793] [INFO ] [MAIN ] TAP: Fetched primary data is valid. [2025-10-20T21:23:15.177859] [DEBUG] [LOCAL ] TAP: Saving primary data. [2025-10-20T21:23:15.179500] [DEBUG] [LOCAL ] TAP: Trimmed 76510 records to 52985 (keeping 200 days of data) [2025-10-20T21:23:15.498891] [DEBUG] [LOCAL ] TAP: Primary data saved (52985 records). [2025-10-20T21:23:15.499211] [INFO ] [MAIN ] TAP: Full history saved. [2025-10-20T21:23:15.499308] [INFO ] [MAIN ] Update Progress: 2043/2367 symbols completed. TAP took 38.3s. Estimated remaining: 41m 44s. Rate: 7.8 symbols/min [2025-10-20T21:23:15.678374] [INFO ] [MAIN ] TBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:15.698505] [DEBUG] [LOCAL ] TBI: Loaded 22962 existing records from primary storage. [2025-10-20T21:23:15.698791] [INFO ] [MAIN ] TBI: Loaded 22962 existing records. [2025-10-20T21:23:15.827482] [INFO ] [REMOTE ] TAK: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T21:23:15.868590] [INFO ] [REMOTE ] TAK: Full fetch complete (71928 records). [2025-10-20T21:23:15.887809] [DEBUG] [VALIDATOR ] TAK: Validating primary data. [2025-10-20T21:23:15.907901] [WARN ] [MAIN ] TBI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:15.908605] [INFO ] [MAIN ] TBI: Wiping existing primary data before full refresh. [2025-10-20T21:23:15.908727] [INFO ] [LOCAL ] TBI: Starting deletion of primary data. [2025-10-20T21:23:15.914411] [DEBUG] [LOCAL ] TBI: Deleted primary data file. [2025-10-20T21:23:15.947798] [INFO ] [MAIN ] TBI: Starting full primary refresh. [2025-10-20T21:23:16.154428] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-04 (1.26 seconds). [2025-10-20T21:23:16.271976] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-05. [2025-10-20T21:23:16.430869] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T21:23:16.545085] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-02. [2025-10-20T21:23:16.664790] [INFO ] [REMOTE ] TBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:16.685777] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-01. [2025-10-20T21:23:16.759323] [DEBUG] [VALIDATOR ] TAK: All expected trading days are present. [2025-10-20T21:23:16.759414] [DEBUG] [VALIDATOR ] TAK: Primary data validation passed. [2025-10-20T21:23:16.760739] [INFO ] [MAIN ] TAK: Fetched primary data is valid. [2025-10-20T21:23:16.760819] [DEBUG] [LOCAL ] TAK: Saving primary data. [2025-10-20T21:23:16.762526] [DEBUG] [LOCAL ] TAK: Trimmed 71928 records to 50251 (keeping 200 days of data) [2025-10-20T21:23:16.996985] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T21:23:17.006076] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-06. [2025-10-20T21:23:17.076443] [DEBUG] [LOCAL ] TAK: Primary data saved (50251 records). [2025-10-20T21:23:17.076592] [INFO ] [MAIN ] TAK: Full history saved. [2025-10-20T21:23:17.076744] [INFO ] [MAIN ] Update Progress: 2044/2367 symbols completed. TAK took 39.9s. Estimated remaining: 41m 36s. Rate: 7.8 symbols/min [2025-10-20T21:23:17.242998] [INFO ] [MAIN ] TBLU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:17.243188] [DEBUG] [LOCAL ] TBLU: No existing data file found in primary storage. [2025-10-20T21:23:17.243224] [WARN ] [MAIN ] TBLU: No existing data found, performing full refresh instead. [2025-10-20T21:23:17.243334] [INFO ] [MAIN ] TBLU: Wiping existing primary data before full refresh. [2025-10-20T21:23:17.243374] [INFO ] [LOCAL ] TBLU: Starting deletion of primary data. [2025-10-20T21:23:17.243414] [DEBUG] [LOCAL ] TBLU: No primary data file found to delete. [2025-10-20T21:23:17.243444] [INFO ] [MAIN ] TBLU: Starting full primary refresh. [2025-10-20T21:23:17.596814] [INFO ] [REMOTE ] TBLU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:17.597118] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-01. [2025-10-20T21:23:17.703125] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T21:23:17.707546] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-02. [2025-10-20T21:23:17.771104] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T21:23:17.774027] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-07. [2025-10-20T21:23:17.879309] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-01 (0.28 seconds). [2025-10-20T21:23:17.880184] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-02. [2025-10-20T21:23:18.155788] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-02 (0.28 seconds). [2025-10-20T21:23:18.156961] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-03. [2025-10-20T21:23:18.278710] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T21:23:18.287080] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-03. [2025-10-20T21:23:18.439883] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-03 (0.28 seconds). [2025-10-20T21:23:18.440804] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-04. [2025-10-20T21:23:18.561829] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T21:23:18.651347] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-08. [2025-10-20T21:23:18.711194] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T21:23:18.720023] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-04 (0.28 seconds). [2025-10-20T21:23:19.170386] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-08 (0.52 seconds). [2025-10-20T21:23:19.346205] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T21:23:19.451465] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-05. [2025-10-20T21:23:20.167724] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T21:23:20.251378] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-06. [2025-10-20T21:23:20.606301] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T21:23:21.051362] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-09. [2025-10-20T21:23:21.851359] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-03. [2025-10-20T21:23:21.882746] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-09 (0.83 seconds). [2025-10-20T21:23:22.651400] [INFO ] [REMOTE ] TBB: Fetching primary data for 2025-10. [2025-10-20T21:23:22.993859] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T21:23:23.385481] [INFO ] [REMOTE ] TBB: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:23:23.387507] [INFO ] [REMOTE ] TBB: Full fetch complete (16508 records). [2025-10-20T21:23:23.387589] [DEBUG] [VALIDATOR ] TBB: Validating primary data. [2025-10-20T21:23:23.456560] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-04. [2025-10-20T21:23:23.786423] [DEBUG] [VALIDATOR ] TBB: All expected trading days are present. [2025-10-20T21:23:23.786521] [DEBUG] [VALIDATOR ] TBB: Primary data validation passed. [2025-10-20T21:23:23.786983] [INFO ] [MAIN ] TBB: Fetched primary data is valid. [2025-10-20T21:23:23.787044] [DEBUG] [LOCAL ] TBB: Saving primary data. [2025-10-20T21:23:23.787610] [DEBUG] [LOCAL ] TBB: Trimmed 16508 records to 11444 (keeping 200 days of data) [2025-10-20T21:23:23.864259] [DEBUG] [LOCAL ] TBB: Primary data saved (11444 records). [2025-10-20T21:23:23.864396] [INFO ] [MAIN ] TBB: Full history saved. [2025-10-20T21:23:23.864475] [INFO ] [MAIN ] Update Progress: 2045/2367 symbols completed. TBB took 12.7s. Estimated remaining: 41m 28s. Rate: 7.8 symbols/min [2025-10-20T21:23:24.032699] [INFO ] [MAIN ] TBN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:24.039216] [DEBUG] [LOCAL ] TBN: Loaded 4058 existing records from primary storage. [2025-10-20T21:23:24.039411] [INFO ] [MAIN ] TBN: Loaded 4058 existing records. [2025-10-20T21:23:24.218260] [WARN ] [MAIN ] TBN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:24.218459] [INFO ] [MAIN ] TBN: Wiping existing primary data before full refresh. [2025-10-20T21:23:24.218505] [INFO ] [LOCAL ] TBN: Starting deletion of primary data. [2025-10-20T21:23:24.218684] [DEBUG] [LOCAL ] TBN: Deleted primary data file. [2025-10-20T21:23:24.218740] [INFO ] [MAIN ] TBN: Starting full primary refresh. [2025-10-20T21:23:24.260205] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-04. [2025-10-20T21:23:24.585222] [INFO ] [REMOTE ] TBN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:24.602039] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T21:23:25.051383] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-05. [2025-10-20T21:23:25.776528] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T21:23:25.851339] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-05. [2025-10-20T21:23:26.336301] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-05 (1.28 seconds). [2025-10-20T21:23:26.651409] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-01. [2025-10-20T21:23:27.246133] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T21:23:27.250113] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T21:23:27.451500] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-06. [2025-10-20T21:23:28.251401] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-07. [2025-10-20T21:23:28.648584] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-07 (0.40 seconds). [2025-10-20T21:23:28.784386] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T21:23:29.051361] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-08. [2025-10-20T21:23:29.474468] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-08 (0.42 seconds). [2025-10-20T21:23:29.851358] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-07. [2025-10-20T21:23:30.651419] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-02. [2025-10-20T21:23:31.258097] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-02 (0.61 seconds). [2025-10-20T21:23:31.363486] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T21:23:31.451402] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-08. [2025-10-20T21:23:32.251374] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-09. [2025-10-20T21:23:32.599508] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-09 (0.35 seconds). [2025-10-20T21:23:32.801486] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T21:23:33.051350] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-06. [2025-10-20T21:23:33.851406] [INFO ] [REMOTE ] TBLU: Fetching primary data for 2025-10. [2025-10-20T21:23:34.084615] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T21:23:34.182251] [INFO ] [REMOTE ] TBLU: Incremental fetch complete for 2025-10 (0.33 seconds). [2025-10-20T21:23:34.183615] [INFO ] [REMOTE ] TBLU: Full fetch complete (1237 records). [2025-10-20T21:23:34.183756] [DEBUG] [VALIDATOR ] TBLU: Validating primary data. [2025-10-20T21:23:34.398367] [WARN ] [VALIDATOR ] TBLU: Validation failed - insufficient historical data. Have 198 unique days, need 200 days [2025-10-20T21:23:34.398584] [FATAL] [MAIN ] TBLU: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:23:34.398633] [WARN ] [MAIN ] TBLU: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:23:34.398729] [INFO ] [LOCAL ] TBLU: Starting deletion of primary data. [2025-10-20T21:23:34.398839] [DEBUG] [LOCAL ] TBLU: No primary data file found to delete. [2025-10-20T21:23:34.398915] [INFO ] [LOCAL ] TBLU: Starting deletion of derivative data. [2025-10-20T21:23:34.398983] [DEBUG] [LOCAL ] TBLU: No derivative data file found to delete. [2025-10-20T21:23:34.399042] [INFO ] [MAIN ] Update Progress: 2046/2367 symbols completed. TBLU took 17.3s. Estimated remaining: 41m 20s. Rate: 7.8 symbols/min [2025-10-20T21:23:34.587362] [INFO ] [MAIN ] TCAI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:34.587549] [DEBUG] [LOCAL ] TCAI: No existing data file found in primary storage. [2025-10-20T21:23:34.587588] [WARN ] [MAIN ] TCAI: No existing data found, performing full refresh instead. [2025-10-20T21:23:34.587727] [INFO ] [MAIN ] TCAI: Wiping existing primary data before full refresh. [2025-10-20T21:23:34.587773] [INFO ] [LOCAL ] TCAI: Starting deletion of primary data. [2025-10-20T21:23:34.587816] [DEBUG] [LOCAL ] TCAI: No primary data file found to delete. [2025-10-20T21:23:34.587849] [INFO ] [MAIN ] TCAI: Starting full primary refresh. [2025-10-20T21:23:34.657744] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-03. [2025-10-20T21:23:34.969852] [INFO ] [REMOTE ] TCAI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:35.261402] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T21:23:35.451386] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-09. [2025-10-20T21:23:36.251386] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-04. [2025-10-20T21:23:36.682749] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T21:23:36.812408] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T21:23:37.051384] [INFO ] [REMOTE ] TCAI: Fetching primary data for 2025-01. [2025-10-20T21:23:37.248510] [DEBUG] [REMOTE ] TCAI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:23:37.248881] [WARN ] [REMOTE ] TCAI: No data returned for month 2025-01 [2025-10-20T21:23:37.248968] [FATAL] [REMOTE ] TCAI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:23:37.249014] [FATAL] [MAIN ] TCAI: Failed to fetch primary data. [2025-10-20T21:23:37.249046] [WARN ] [MAIN ] TCAI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:23:37.249111] [INFO ] [LOCAL ] TCAI: Starting deletion of primary data. [2025-10-20T21:23:37.249172] [DEBUG] [LOCAL ] TCAI: No primary data file found to delete. [2025-10-20T21:23:37.249206] [INFO ] [LOCAL ] TCAI: Starting deletion of derivative data. [2025-10-20T21:23:37.249244] [DEBUG] [LOCAL ] TCAI: No derivative data file found to delete. [2025-10-20T21:23:37.249288] [INFO ] [MAIN ] Update Progress: 2047/2367 symbols completed. TCAI took 2.9s. Estimated remaining: 41m 12s. Rate: 7.8 symbols/min [2025-10-20T21:23:37.414929] [INFO ] [MAIN ] TCBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:37.426752] [DEBUG] [LOCAL ] TCBX: Loaded 10385 existing records from primary storage. [2025-10-20T21:23:37.426963] [INFO ] [MAIN ] TCBX: Loaded 10385 existing records. [2025-10-20T21:23:37.604458] [WARN ] [MAIN ] TCBX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:37.604698] [INFO ] [MAIN ] TCBX: Wiping existing primary data before full refresh. [2025-10-20T21:23:37.604755] [INFO ] [LOCAL ] TCBX: Starting deletion of primary data. [2025-10-20T21:23:37.604931] [DEBUG] [LOCAL ] TCBX: Deleted primary data file. [2025-10-20T21:23:37.604986] [INFO ] [MAIN ] TCBX: Starting full primary refresh. [2025-10-20T21:23:37.856509] [INFO ] [REMOTE ] TBBB: Fetching primary data for 2025-10. [2025-10-20T21:23:37.954894] [INFO ] [REMOTE ] TCBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:38.651373] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-05. [2025-10-20T21:23:39.077729] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T21:23:39.191502] [INFO ] [REMOTE ] TBBB: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T21:23:39.197188] [INFO ] [REMOTE ] TBBB: Full fetch complete (51912 records). [2025-10-20T21:23:39.197296] [DEBUG] [VALIDATOR ] TBBB: Validating primary data. [2025-10-20T21:23:39.461705] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-01. [2025-10-20T21:23:39.600971] [DEBUG] [VALIDATOR ] TBBB: All expected trading days are present. [2025-10-20T21:23:39.601049] [DEBUG] [VALIDATOR ] TBBB: Primary data validation passed. [2025-10-20T21:23:39.601938] [INFO ] [MAIN ] TBBB: Fetched primary data is valid. [2025-10-20T21:23:39.602003] [DEBUG] [LOCAL ] TBBB: Saving primary data. [2025-10-20T21:23:39.603084] [DEBUG] [LOCAL ] TBBB: Trimmed 51912 records to 35683 (keeping 200 days of data) [2025-10-20T21:23:39.800328] [DEBUG] [LOCAL ] TBBB: Primary data saved (35683 records). [2025-10-20T21:23:39.800453] [INFO ] [MAIN ] TBBB: Full history saved. [2025-10-20T21:23:39.800793] [INFO ] [MAIN ] Update Progress: 2048/2367 symbols completed. TBBB took 26.0s. Estimated remaining: 41m 3s. Rate: 7.8 symbols/min [2025-10-20T21:23:39.968582] [INFO ] [MAIN ] TCPA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:39.968868] [DEBUG] [LOCAL ] TCPA: No existing data file found in primary storage. [2025-10-20T21:23:39.968931] [WARN ] [MAIN ] TCPA: No existing data found, performing full refresh instead. [2025-10-20T21:23:39.969053] [INFO ] [MAIN ] TCPA: Wiping existing primary data before full refresh. [2025-10-20T21:23:39.969092] [INFO ] [LOCAL ] TCPA: Starting deletion of primary data. [2025-10-20T21:23:39.969135] [DEBUG] [LOCAL ] TCPA: No primary data file found to delete. [2025-10-20T21:23:39.969169] [INFO ] [MAIN ] TCPA: Starting full primary refresh. [2025-10-20T21:23:40.251365] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-07. [2025-10-20T21:23:40.321351] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T21:23:40.342855] [INFO ] [REMOTE ] TCPA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:41.051347] [INFO ] [REMOTE ] TCPA: Fetching primary data for 2025-01. [2025-10-20T21:23:41.248571] [DEBUG] [REMOTE ] TCPA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:23:41.248920] [WARN ] [REMOTE ] TCPA: No data returned for month 2025-01 [2025-10-20T21:23:41.249007] [FATAL] [REMOTE ] TCPA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:23:41.249054] [FATAL] [MAIN ] TCPA: Failed to fetch primary data. [2025-10-20T21:23:41.249109] [WARN ] [MAIN ] TCPA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:23:41.249177] [INFO ] [LOCAL ] TCPA: Starting deletion of primary data. [2025-10-20T21:23:41.249277] [DEBUG] [LOCAL ] TCPA: No primary data file found to delete. [2025-10-20T21:23:41.249319] [INFO ] [LOCAL ] TCPA: Starting deletion of derivative data. [2025-10-20T21:23:41.249359] [DEBUG] [LOCAL ] TCPA: No derivative data file found to delete. [2025-10-20T21:23:41.249403] [INFO ] [MAIN ] Update Progress: 2049/2367 symbols completed. TCPA took 1.4s. Estimated remaining: 40m 55s. Rate: 7.8 symbols/min [2025-10-20T21:23:41.434586] [INFO ] [MAIN ] TD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:41.436008] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-07 (1.18 seconds). [2025-10-20T21:23:41.489058] [DEBUG] [LOCAL ] TD: Loaded 53086 existing records from primary storage. [2025-10-20T21:23:41.489354] [INFO ] [MAIN ] TD: Loaded 53086 existing records. [2025-10-20T21:23:41.675075] [WARN ] [MAIN ] TD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:41.675294] [INFO ] [MAIN ] TD: Wiping existing primary data before full refresh. [2025-10-20T21:23:41.675340] [INFO ] [LOCAL ] TD: Starting deletion of primary data. [2025-10-20T21:23:41.675517] [DEBUG] [LOCAL ] TD: Deleted primary data file. [2025-10-20T21:23:41.675553] [INFO ] [MAIN ] TD: Starting full primary refresh. [2025-10-20T21:23:41.852574] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-06. [2025-10-20T21:23:42.033794] [INFO ] [REMOTE ] TD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:42.399242] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T21:23:42.651421] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-02. [2025-10-20T21:23:43.451391] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-01. [2025-10-20T21:23:43.499359] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T21:23:44.251400] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-07. [2025-10-20T21:23:44.808822] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-07 (0.56 seconds). [2025-10-20T21:23:45.051366] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-08. [2025-10-20T21:23:45.179035] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T21:23:45.851416] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-02. [2025-10-20T21:23:46.131992] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T21:23:46.651387] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-08. [2025-10-20T21:23:47.039140] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T21:23:47.302407] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T21:23:47.451377] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-03. [2025-10-20T21:23:48.251372] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-09. [2025-10-20T21:23:48.964802] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T21:23:49.051380] [INFO ] [REMOTE ] TBN: Fetching primary data for 2025-10. [2025-10-20T21:23:49.144976] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:23:49.597621] [INFO ] [REMOTE ] TBN: Incremental fetch complete for 2025-10 (0.55 seconds). [2025-10-20T21:23:49.599840] [INFO ] [REMOTE ] TBN: Full fetch complete (5931 records). [2025-10-20T21:23:49.599910] [DEBUG] [VALIDATOR ] TBN: Validating primary data. [2025-10-20T21:23:49.861879] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-04. [2025-10-20T21:23:50.007305] [DEBUG] [VALIDATOR ] TBN: All expected trading days are present. [2025-10-20T21:23:50.007390] [DEBUG] [VALIDATOR ] TBN: Primary data validation passed. [2025-10-20T21:23:50.007560] [INFO ] [MAIN ] TBN: Fetched primary data is valid. [2025-10-20T21:23:50.007598] [DEBUG] [LOCAL ] TBN: Saving primary data. [2025-10-20T21:23:50.008044] [DEBUG] [LOCAL ] TBN: Trimmed 5931 records to 3997 (keeping 200 days of data) [2025-10-20T21:23:50.033816] [DEBUG] [LOCAL ] TBN: Primary data saved (3997 records). [2025-10-20T21:23:50.033965] [INFO ] [MAIN ] TBN: Full history saved. [2025-10-20T21:23:50.034049] [INFO ] [MAIN ] Update Progress: 2050/2367 symbols completed. TBN took 26.2s. Estimated remaining: 40m 47s. Rate: 7.8 symbols/min [2025-10-20T21:23:50.201619] [INFO ] [MAIN ] TDC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:50.248309] [DEBUG] [LOCAL ] TDC: Loaded 47237 existing records from primary storage. [2025-10-20T21:23:50.248626] [INFO ] [MAIN ] TDC: Loaded 47237 existing records. [2025-10-20T21:23:50.428273] [WARN ] [MAIN ] TDC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:50.428503] [INFO ] [MAIN ] TDC: Wiping existing primary data before full refresh. [2025-10-20T21:23:50.428550] [INFO ] [LOCAL ] TDC: Starting deletion of primary data. [2025-10-20T21:23:50.428764] [DEBUG] [LOCAL ] TDC: Deleted primary data file. [2025-10-20T21:23:50.428821] [INFO ] [MAIN ] TDC: Starting full primary refresh. [2025-10-20T21:23:50.661530] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-09. [2025-10-20T21:23:50.814875] [INFO ] [REMOTE ] TDC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:51.451348] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-03. [2025-10-20T21:23:51.681034] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T21:23:51.778778] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-04 (1.92 seconds). [2025-10-20T21:23:51.944734] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-03 (0.49 seconds). [2025-10-20T21:23:52.251358] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-05. [2025-10-20T21:23:53.051384] [INFO ] [REMOTE ] TBI: Fetching primary data for 2025-10. [2025-10-20T21:23:53.851342] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-04. [2025-10-20T21:23:54.052570] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T21:23:54.198796] [INFO ] [REMOTE ] TBI: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T21:23:54.202781] [INFO ] [REMOTE ] TBI: Full fetch complete (32086 records). [2025-10-20T21:23:54.202871] [DEBUG] [VALIDATOR ] TBI: Validating primary data. [2025-10-20T21:23:54.567132] [DEBUG] [VALIDATOR ] TBI: All expected trading days are present. [2025-10-20T21:23:54.567205] [DEBUG] [VALIDATOR ] TBI: Primary data validation passed. [2025-10-20T21:23:54.567914] [INFO ] [MAIN ] TBI: Fetched primary data is valid. [2025-10-20T21:23:54.567976] [DEBUG] [LOCAL ] TBI: Saving primary data. [2025-10-20T21:23:54.568734] [DEBUG] [LOCAL ] TBI: Trimmed 32086 records to 22807 (keeping 200 days of data) [2025-10-20T21:23:54.653415] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-01. [2025-10-20T21:23:54.693123] [DEBUG] [LOCAL ] TBI: Primary data saved (22807 records). [2025-10-20T21:23:54.693233] [INFO ] [MAIN ] TBI: Full history saved. [2025-10-20T21:23:54.693305] [INFO ] [MAIN ] Update Progress: 2051/2367 symbols completed. TBI took 39.2s. Estimated remaining: 40m 39s. Rate: 7.8 symbols/min [2025-10-20T21:23:54.872040] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T21:23:54.889996] [INFO ] [MAIN ] TDF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:23:54.901987] [DEBUG] [LOCAL ] TDF: Loaded 6421 existing records from primary storage. [2025-10-20T21:23:54.902167] [INFO ] [MAIN ] TDF: Loaded 6421 existing records. [2025-10-20T21:23:55.085860] [WARN ] [MAIN ] TDF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:23:55.086092] [INFO ] [MAIN ] TDF: Wiping existing primary data before full refresh. [2025-10-20T21:23:55.086138] [INFO ] [LOCAL ] TDF: Starting deletion of primary data. [2025-10-20T21:23:55.086313] [DEBUG] [LOCAL ] TDF: Deleted primary data file. [2025-10-20T21:23:55.086369] [INFO ] [MAIN ] TDF: Starting full primary refresh. [2025-10-20T21:23:55.445056] [INFO ] [REMOTE ] TDF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:23:55.451340] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-01. [2025-10-20T21:23:56.123530] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T21:23:56.189024] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-01 (0.74 seconds). [2025-10-20T21:23:56.251347] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-02. [2025-10-20T21:23:57.051381] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-06. [2025-10-20T21:23:57.820666] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:23:57.851358] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-03. [2025-10-20T21:23:58.651408] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-02. [2025-10-20T21:23:58.794456] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T21:23:59.433413] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T21:23:59.451362] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-03. [2025-10-20T21:23:59.542995] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:24:00.225285] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T21:24:00.251351] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-04. [2025-10-20T21:24:01.041707] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T21:24:01.051334] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-05. [2025-10-20T21:24:01.705121] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T21:24:01.851385] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-04. [2025-10-20T21:24:02.651457] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-06. [2025-10-20T21:24:03.293109] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T21:24:03.451390] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-07. [2025-10-20T21:24:03.536206] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T21:24:04.095985] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T21:24:04.251387] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-08. [2025-10-20T21:24:04.891863] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-08 (0.64 seconds). [2025-10-20T21:24:05.051428] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-05. [2025-10-20T21:24:05.816464] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T21:24:05.851348] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-06. [2025-10-20T21:24:06.573836] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-06 (0.72 seconds). [2025-10-20T21:24:06.651350] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-07. [2025-10-20T21:24:07.451420] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-09. [2025-10-20T21:24:08.251444] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-07. [2025-10-20T21:24:08.269279] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T21:24:08.550673] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-07 (1.90 seconds). [2025-10-20T21:24:09.016027] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-07 (0.76 seconds). [2025-10-20T21:24:09.051378] [INFO ] [REMOTE ] TDF: Fetching primary data for 2025-10. [2025-10-20T21:24:09.690565] [INFO ] [REMOTE ] TDF: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:24:09.692190] [INFO ] [REMOTE ] TDF: Full fetch complete (10373 records). [2025-10-20T21:24:09.692262] [DEBUG] [VALIDATOR ] TDF: Validating primary data. [2025-10-20T21:24:09.864414] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-05. [2025-10-20T21:24:10.055562] [DEBUG] [VALIDATOR ] TDF: All expected trading days are present. [2025-10-20T21:24:10.055671] [DEBUG] [VALIDATOR ] TDF: Primary data validation passed. [2025-10-20T21:24:10.055972] [INFO ] [MAIN ] TDF: Fetched primary data is valid. [2025-10-20T21:24:10.056014] [DEBUG] [LOCAL ] TDF: Saving primary data. [2025-10-20T21:24:10.056482] [DEBUG] [LOCAL ] TDF: Trimmed 10373 records to 6264 (keeping 200 days of data) [2025-10-20T21:24:10.094449] [DEBUG] [LOCAL ] TDF: Primary data saved (6264 records). [2025-10-20T21:24:10.094596] [INFO ] [MAIN ] TDF: Full history saved. [2025-10-20T21:24:10.094702] [INFO ] [MAIN ] Update Progress: 2052/2367 symbols completed. TDF took 15.4s. Estimated remaining: 40m 32s. Rate: 7.8 symbols/min [2025-10-20T21:24:10.275854] [INFO ] [MAIN ] TDG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:10.302463] [DEBUG] [LOCAL ] TDG: Loaded 22499 existing records from primary storage. [2025-10-20T21:24:10.302789] [INFO ] [MAIN ] TDG: Loaded 22499 existing records. [2025-10-20T21:24:10.512196] [WARN ] [MAIN ] TDG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:10.512427] [INFO ] [MAIN ] TDG: Wiping existing primary data before full refresh. [2025-10-20T21:24:10.512474] [INFO ] [LOCAL ] TDG: Starting deletion of primary data. [2025-10-20T21:24:10.512700] [DEBUG] [LOCAL ] TDG: Deleted primary data file. [2025-10-20T21:24:10.512757] [INFO ] [MAIN ] TDG: Starting full primary refresh. [2025-10-20T21:24:10.656597] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-08. [2025-10-20T21:24:10.905983] [INFO ] [REMOTE ] TDG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:11.451388] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-01. [2025-10-20T21:24:11.596512] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T21:24:11.742273] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T21:24:12.251470] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-06. [2025-10-20T21:24:12.603259] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T21:24:13.051436] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-02. [2025-10-20T21:24:13.558535] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T21:24:13.851384] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-08. [2025-10-20T21:24:14.153739] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T21:24:14.651426] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-09. [2025-10-20T21:24:15.425970] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-09 (0.77 seconds). [2025-10-20T21:24:15.457321] [INFO ] [REMOTE ] TCBX: Fetching primary data for 2025-10. [2025-10-20T21:24:15.463448] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:24:16.225844] [INFO ] [REMOTE ] TCBX: Incremental fetch complete for 2025-10 (0.77 seconds). [2025-10-20T21:24:16.228000] [INFO ] [REMOTE ] TCBX: Full fetch complete (15923 records). [2025-10-20T21:24:16.228069] [DEBUG] [VALIDATOR ] TCBX: Validating primary data. [2025-10-20T21:24:16.257490] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-09. [2025-10-20T21:24:16.594409] [DEBUG] [VALIDATOR ] TCBX: All expected trading days are present. [2025-10-20T21:24:16.594484] [DEBUG] [VALIDATOR ] TCBX: Primary data validation passed. [2025-10-20T21:24:16.594853] [INFO ] [MAIN ] TCBX: Fetched primary data is valid. [2025-10-20T21:24:16.594914] [DEBUG] [LOCAL ] TCBX: Saving primary data. [2025-10-20T21:24:16.595382] [DEBUG] [LOCAL ] TCBX: Trimmed 15923 records to 10256 (keeping 200 days of data) [2025-10-20T21:24:16.653245] [DEBUG] [LOCAL ] TCBX: Primary data saved (10256 records). [2025-10-20T21:24:16.653370] [INFO ] [MAIN ] TCBX: Full history saved. [2025-10-20T21:24:16.653446] [INFO ] [MAIN ] Update Progress: 2053/2367 symbols completed. TCBX took 39.4s. Estimated remaining: 40m 25s. Rate: 7.8 symbols/min [2025-10-20T21:24:16.826673] [INFO ] [MAIN ] TDOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:16.875393] [DEBUG] [LOCAL ] TDOC: Loaded 54145 existing records from primary storage. [2025-10-20T21:24:16.875731] [INFO ] [MAIN ] TDOC: Loaded 54145 existing records. [2025-10-20T21:24:17.058412] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-03. [2025-10-20T21:24:17.058286] [WARN ] [MAIN ] TDOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:17.059608] [INFO ] [MAIN ] TDOC: Wiping existing primary data before full refresh. [2025-10-20T21:24:17.059699] [INFO ] [LOCAL ] TDOC: Starting deletion of primary data. [2025-10-20T21:24:17.059950] [DEBUG] [LOCAL ] TDOC: Deleted primary data file. [2025-10-20T21:24:17.060021] [INFO ] [MAIN ] TDOC: Starting full primary refresh. [2025-10-20T21:24:17.429106] [INFO ] [REMOTE ] TDOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:17.851426] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-01. [2025-10-20T21:24:18.061398] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T21:24:18.131804] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T21:24:18.651415] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-04. [2025-10-20T21:24:19.451399] [INFO ] [REMOTE ] TD: Fetching primary data for 2025-10. [2025-10-20T21:24:19.832793] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T21:24:19.920769] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T21:24:20.251378] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-05. [2025-10-20T21:24:20.709894] [INFO ] [REMOTE ] TD: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T21:24:20.717383] [INFO ] [REMOTE ] TD: Full fetch complete (75929 records). [2025-10-20T21:24:20.717511] [DEBUG] [VALIDATOR ] TD: Validating primary data. [2025-10-20T21:24:21.062698] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-07. [2025-10-20T21:24:21.101451] [DEBUG] [VALIDATOR ] TD: All expected trading days are present. [2025-10-20T21:24:21.101536] [DEBUG] [VALIDATOR ] TD: Primary data validation passed. [2025-10-20T21:24:21.102734] [INFO ] [MAIN ] TD: Fetched primary data is valid. [2025-10-20T21:24:21.102797] [DEBUG] [LOCAL ] TD: Saving primary data. [2025-10-20T21:24:21.104213] [DEBUG] [LOCAL ] TD: Trimmed 75929 records to 52287 (keeping 200 days of data) [2025-10-20T21:24:21.425877] [DEBUG] [LOCAL ] TD: Primary data saved (52287 records). [2025-10-20T21:24:21.426011] [INFO ] [MAIN ] TD: Full history saved. [2025-10-20T21:24:21.426291] [INFO ] [MAIN ] Update Progress: 2054/2367 symbols completed. TD took 40.2s. Estimated remaining: 40m 16s. Rate: 7.8 symbols/min [2025-10-20T21:24:21.621312] [INFO ] [MAIN ] TDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:21.623609] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-05 (1.37 seconds). [2025-10-20T21:24:21.677500] [DEBUG] [LOCAL ] TDS: Loaded 43099 existing records from primary storage. [2025-10-20T21:24:21.677977] [INFO ] [MAIN ] TDS: Loaded 43099 existing records. [2025-10-20T21:24:21.877986] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-06. [2025-10-20T21:24:21.882903] [WARN ] [MAIN ] TDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:21.884020] [INFO ] [MAIN ] TDS: Wiping existing primary data before full refresh. [2025-10-20T21:24:21.884078] [INFO ] [LOCAL ] TDS: Starting deletion of primary data. [2025-10-20T21:24:21.884310] [DEBUG] [LOCAL ] TDS: Deleted primary data file. [2025-10-20T21:24:21.884348] [INFO ] [MAIN ] TDS: Starting full primary refresh. [2025-10-20T21:24:22.268093] [INFO ] [REMOTE ] TDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:22.651403] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-02. [2025-10-20T21:24:22.793441] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:24:23.001510] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T21:24:23.451390] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-07. [2025-10-20T21:24:24.251363] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-01. [2025-10-20T21:24:24.489583] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T21:24:24.797016] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-02 (2.15 seconds). [2025-10-20T21:24:25.051379] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-08. [2025-10-20T21:24:25.316098] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T21:24:25.851379] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-02. [2025-10-20T21:24:26.651368] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-03. [2025-10-20T21:24:26.684238] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-08 (1.63 seconds). [2025-10-20T21:24:27.284169] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T21:24:27.451355] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-03. [2025-10-20T21:24:28.251346] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-08. [2025-10-20T21:24:28.792584] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T21:24:29.038204] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T21:24:29.051345] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-04. [2025-10-20T21:24:29.793471] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T21:24:29.851383] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-04. [2025-10-20T21:24:30.647575] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T21:24:30.656064] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-09. [2025-10-20T21:24:31.451386] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-05. [2025-10-20T21:24:31.848598] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-04 (2.00 seconds). [2025-10-20T21:24:32.257706] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-05. [2025-10-20T21:24:32.275119] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:24:33.051410] [INFO ] [REMOTE ] TDC: Fetching primary data for 2025-10. [2025-10-20T21:24:33.079840] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:24:33.851411] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-06. [2025-10-20T21:24:34.236870] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-05 (1.98 seconds). [2025-10-20T21:24:34.549014] [INFO ] [REMOTE ] TDC: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T21:24:34.556132] [INFO ] [REMOTE ] TDC: Full fetch complete (67711 records). [2025-10-20T21:24:34.556264] [DEBUG] [VALIDATOR ] TDC: Validating primary data. [2025-10-20T21:24:34.657595] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-06. [2025-10-20T21:24:34.964362] [DEBUG] [VALIDATOR ] TDC: All expected trading days are present. [2025-10-20T21:24:34.964442] [DEBUG] [VALIDATOR ] TDC: Primary data validation passed. [2025-10-20T21:24:34.965455] [INFO ] [MAIN ] TDC: Fetched primary data is valid. [2025-10-20T21:24:34.965515] [DEBUG] [LOCAL ] TDC: Saving primary data. [2025-10-20T21:24:34.966748] [DEBUG] [LOCAL ] TDC: Trimmed 67711 records to 46477 (keeping 200 days of data) [2025-10-20T21:24:35.229996] [DEBUG] [LOCAL ] TDC: Primary data saved (46477 records). [2025-10-20T21:24:35.230283] [INFO ] [MAIN ] TDC: Full history saved. [2025-10-20T21:24:35.230444] [INFO ] [MAIN ] Update Progress: 2055/2367 symbols completed. TDC took 45.2s. Estimated remaining: 40m 10s. Rate: 7.8 symbols/min [2025-10-20T21:24:35.425124] [INFO ] [MAIN ] TDW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:35.429027] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:24:35.456129] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-09. [2025-10-20T21:24:35.477197] [DEBUG] [LOCAL ] TDW: Loaded 45635 existing records from primary storage. [2025-10-20T21:24:35.477484] [INFO ] [MAIN ] TDW: Loaded 45635 existing records. [2025-10-20T21:24:35.659180] [WARN ] [MAIN ] TDW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:35.659412] [INFO ] [MAIN ] TDW: Wiping existing primary data before full refresh. [2025-10-20T21:24:35.659473] [INFO ] [LOCAL ] TDW: Starting deletion of primary data. [2025-10-20T21:24:35.659743] [DEBUG] [LOCAL ] TDW: Deleted primary data file. [2025-10-20T21:24:35.659817] [INFO ] [MAIN ] TDW: Starting full primary refresh. [2025-10-20T21:24:36.018601] [INFO ] [REMOTE ] TDW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:36.251352] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-01. [2025-10-20T21:24:36.648995] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T21:24:36.673802] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T21:24:37.051392] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-07. [2025-10-20T21:24:37.850071] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T21:24:37.858905] [INFO ] [REMOTE ] TDG: Fetching primary data for 2025-10. [2025-10-20T21:24:38.651361] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-07. [2025-10-20T21:24:38.987256] [INFO ] [REMOTE ] TDG: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T21:24:38.991002] [INFO ] [REMOTE ] TDG: Full fetch complete (32129 records). [2025-10-20T21:24:38.991082] [DEBUG] [VALIDATOR ] TDG: Validating primary data. [2025-10-20T21:24:39.411505] [DEBUG] [VALIDATOR ] TDG: All expected trading days are present. [2025-10-20T21:24:39.411760] [DEBUG] [VALIDATOR ] TDG: Primary data validation passed. [2025-10-20T21:24:39.412437] [INFO ] [MAIN ] TDG: Fetched primary data is valid. [2025-10-20T21:24:39.412539] [DEBUG] [LOCAL ] TDG: Saving primary data. [2025-10-20T21:24:39.413673] [DEBUG] [LOCAL ] TDG: Trimmed 32129 records to 22139 (keeping 200 days of data) [2025-10-20T21:24:39.494491] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-02. [2025-10-20T21:24:39.550480] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-07 (2.50 seconds). [2025-10-20T21:24:39.612955] [DEBUG] [LOCAL ] TDG: Primary data saved (22139 records). [2025-10-20T21:24:39.613096] [INFO ] [MAIN ] TDG: Full history saved. [2025-10-20T21:24:39.613177] [INFO ] [MAIN ] Update Progress: 2056/2367 symbols completed. TDG took 29.5s. Estimated remaining: 40m 1s. Rate: 7.8 symbols/min [2025-10-20T21:24:39.776745] [INFO ] [MAIN ] TDY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:39.798726] [DEBUG] [LOCAL ] TDY: Loaded 22564 existing records from primary storage. [2025-10-20T21:24:39.798968] [INFO ] [MAIN ] TDY: Loaded 22564 existing records. [2025-10-20T21:24:39.981356] [WARN ] [MAIN ] TDY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:39.981633] [INFO ] [MAIN ] TDY: Wiping existing primary data before full refresh. [2025-10-20T21:24:39.981746] [INFO ] [LOCAL ] TDY: Starting deletion of primary data. [2025-10-20T21:24:39.981923] [DEBUG] [LOCAL ] TDY: Deleted primary data file. [2025-10-20T21:24:39.981962] [INFO ] [MAIN ] TDY: Starting full primary refresh. [2025-10-20T21:24:40.261914] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-08. [2025-10-20T21:24:40.299154] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-07 (1.65 seconds). [2025-10-20T21:24:40.388887] [INFO ] [REMOTE ] TDY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:41.051420] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-01. [2025-10-20T21:24:41.169255] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:24:41.851444] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-03. [2025-10-20T21:24:42.226880] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T21:24:42.450618] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-08 (2.19 seconds). [2025-10-20T21:24:42.651384] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-02. [2025-10-20T21:24:43.420080] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T21:24:43.451432] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-09. [2025-10-20T21:24:43.774115] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T21:24:44.251369] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-08. [2025-10-20T21:24:45.051424] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-04. [2025-10-20T21:24:45.430693] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T21:24:45.509044] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T21:24:45.851374] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-09. [2025-10-20T21:24:46.651353] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-03. [2025-10-20T21:24:46.737097] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T21:24:47.417440] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:24:47.451333] [INFO ] [REMOTE ] TDS: Fetching primary data for 2025-10. [2025-10-20T21:24:47.814596] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-03 (1.16 seconds). [2025-10-20T21:24:48.251375] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-04. [2025-10-20T21:24:48.581041] [INFO ] [REMOTE ] TDS: Incremental fetch complete for 2025-10 (1.13 seconds). [2025-10-20T21:24:48.587257] [INFO ] [REMOTE ] TDS: Full fetch complete (61548 records). [2025-10-20T21:24:48.587369] [DEBUG] [VALIDATOR ] TDS: Validating primary data. [2025-10-20T21:24:48.978769] [DEBUG] [VALIDATOR ] TDS: All expected trading days are present. [2025-10-20T21:24:48.978874] [DEBUG] [VALIDATOR ] TDS: Primary data validation passed. [2025-10-20T21:24:48.979953] [INFO ] [MAIN ] TDS: Fetched primary data is valid. [2025-10-20T21:24:48.980013] [DEBUG] [LOCAL ] TDS: Saving primary data. [2025-10-20T21:24:48.981212] [DEBUG] [LOCAL ] TDS: Trimmed 61548 records to 42428 (keeping 200 days of data) [2025-10-20T21:24:49.070590] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-05. [2025-10-20T21:24:49.239214] [DEBUG] [LOCAL ] TDS: Primary data saved (42428 records). [2025-10-20T21:24:49.239337] [INFO ] [MAIN ] TDS: Full history saved. [2025-10-20T21:24:49.239709] [INFO ] [MAIN ] Update Progress: 2057/2367 symbols completed. TDS took 27.8s. Estimated remaining: 39m 54s. Rate: 7.8 symbols/min [2025-10-20T21:24:49.432176] [INFO ] [MAIN ] TE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:49.432322] [DEBUG] [LOCAL ] TE: No existing data file found in primary storage. [2025-10-20T21:24:49.432359] [WARN ] [MAIN ] TE: No existing data found, performing full refresh instead. [2025-10-20T21:24:49.432492] [INFO ] [MAIN ] TE: Wiping existing primary data before full refresh. [2025-10-20T21:24:49.432531] [INFO ] [LOCAL ] TE: Starting deletion of primary data. [2025-10-20T21:24:49.432571] [DEBUG] [LOCAL ] TE: No primary data file found to delete. [2025-10-20T21:24:49.432601] [INFO ] [MAIN ] TE: Starting full primary refresh. [2025-10-20T21:24:49.830310] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T21:24:49.835363] [INFO ] [REMOTE ] TE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:49.851330] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-05. [2025-10-20T21:24:50.651374] [INFO ] [REMOTE ] TDOC: Fetching primary data for 2025-10. [2025-10-20T21:24:50.696864] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:24:51.022253] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:24:51.451372] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-06. [2025-10-20T21:24:52.251373] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-06. [2025-10-20T21:24:52.584155] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T21:24:52.639162] [INFO ] [REMOTE ] TDOC: Incremental fetch complete for 2025-10 (1.99 seconds). [2025-10-20T21:24:52.647248] [INFO ] [REMOTE ] TDOC: Full fetch complete (77435 records). [2025-10-20T21:24:52.647407] [DEBUG] [VALIDATOR ] TDOC: Validating primary data. [2025-10-20T21:24:53.060601] [DEBUG] [VALIDATOR ] TDOC: All expected trading days are present. [2025-10-20T21:24:53.056820] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-07. [2025-10-20T21:24:53.060987] [DEBUG] [VALIDATOR ] TDOC: Primary data validation passed. [2025-10-20T21:24:53.063238] [INFO ] [MAIN ] TDOC: Fetched primary data is valid. [2025-10-20T21:24:53.063319] [DEBUG] [LOCAL ] TDOC: Saving primary data. [2025-10-20T21:24:53.064800] [DEBUG] [LOCAL ] TDOC: Trimmed 77435 records to 53368 (keeping 200 days of data) [2025-10-20T21:24:53.359919] [DEBUG] [LOCAL ] TDOC: Primary data saved (53368 records). [2025-10-20T21:24:53.360043] [INFO ] [MAIN ] TDOC: Full history saved. [2025-10-20T21:24:53.360116] [INFO ] [MAIN ] Update Progress: 2058/2367 symbols completed. TDOC took 36.7s. Estimated remaining: 39m 46s. Rate: 7.8 symbols/min [2025-10-20T21:24:53.535261] [INFO ] [MAIN ] TEAF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:53.545017] [DEBUG] [LOCAL ] TEAF: Loaded 7531 existing records from primary storage. [2025-10-20T21:24:53.545199] [INFO ] [MAIN ] TEAF: Loaded 7531 existing records. [2025-10-20T21:24:53.724069] [WARN ] [MAIN ] TEAF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:53.724269] [INFO ] [MAIN ] TEAF: Wiping existing primary data before full refresh. [2025-10-20T21:24:53.724314] [INFO ] [LOCAL ] TEAF: Starting deletion of primary data. [2025-10-20T21:24:53.724477] [DEBUG] [LOCAL ] TEAF: Deleted primary data file. [2025-10-20T21:24:53.724531] [INFO ] [MAIN ] TEAF: Starting full primary refresh. [2025-10-20T21:24:53.855774] [INFO ] [REMOTE ] TE: Fetching primary data for 2025-01. [2025-10-20T21:24:54.116628] [INFO ] [REMOTE ] TEAF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:54.119122] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:24:54.145027] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-07 (1.09 seconds). [2025-10-20T21:24:54.161812] [DEBUG] [REMOTE ] TE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:24:54.162041] [WARN ] [REMOTE ] TE: No data returned for month 2025-01 [2025-10-20T21:24:54.162126] [FATAL] [REMOTE ] TE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:24:54.162198] [FATAL] [MAIN ] TE: Failed to fetch primary data. [2025-10-20T21:24:54.162262] [WARN ] [MAIN ] TE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:24:54.162330] [INFO ] [LOCAL ] TE: Starting deletion of primary data. [2025-10-20T21:24:54.162392] [DEBUG] [LOCAL ] TE: No primary data file found to delete. [2025-10-20T21:24:54.162425] [INFO ] [LOCAL ] TE: Starting deletion of derivative data. [2025-10-20T21:24:54.162463] [DEBUG] [LOCAL ] TE: No derivative data file found to delete. [2025-10-20T21:24:54.162503] [INFO ] [MAIN ] Update Progress: 2059/2367 symbols completed. TE took 4.9s. Estimated remaining: 39m 37s. Rate: 7.8 symbols/min [2025-10-20T21:24:54.333536] [INFO ] [MAIN ] TECK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:24:54.388529] [DEBUG] [LOCAL ] TECK: Loaded 54213 existing records from primary storage. [2025-10-20T21:24:54.388907] [INFO ] [MAIN ] TECK: Loaded 54213 existing records. [2025-10-20T21:24:54.570895] [WARN ] [MAIN ] TECK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:24:54.571119] [INFO ] [MAIN ] TECK: Wiping existing primary data before full refresh. [2025-10-20T21:24:54.571170] [INFO ] [LOCAL ] TECK: Starting deletion of primary data. [2025-10-20T21:24:54.571400] [DEBUG] [LOCAL ] TECK: Deleted primary data file. [2025-10-20T21:24:54.571456] [INFO ] [MAIN ] TECK: Starting full primary refresh. [2025-10-20T21:24:54.656489] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-07. [2025-10-20T21:24:54.940712] [INFO ] [REMOTE ] TECK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:24:55.451463] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-01. [2025-10-20T21:24:56.256481] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-01. [2025-10-20T21:24:56.269789] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:24:56.811819] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T21:24:56.876747] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T21:24:57.051392] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-02. [2025-10-20T21:24:57.851423] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-08. [2025-10-20T21:24:58.651459] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-02. [2025-10-20T21:24:58.724294] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:24:59.220483] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T21:24:59.457716] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-03. [2025-10-20T21:24:59.463394] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:25:00.251386] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-09. [2025-10-20T21:25:01.051442] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-03. [2025-10-20T21:25:01.742934] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-03 (2.29 seconds). [2025-10-20T21:25:01.763126] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T21:25:01.851381] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-04. [2025-10-20T21:25:02.054749] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T21:25:02.651378] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-08. [2025-10-20T21:25:03.451373] [INFO ] [REMOTE ] TDW: Fetching primary data for 2025-10. [2025-10-20T21:25:03.678949] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:25:03.921637] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-04 (2.07 seconds). [2025-10-20T21:25:04.251424] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-09. [2025-10-20T21:25:04.740285] [INFO ] [REMOTE ] TDW: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T21:25:04.746736] [INFO ] [REMOTE ] TDW: Full fetch complete (66779 records). [2025-10-20T21:25:04.746842] [DEBUG] [VALIDATOR ] TDW: Validating primary data. [2025-10-20T21:25:05.052184] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-05. [2025-10-20T21:25:05.151228] [DEBUG] [VALIDATOR ] TDW: All expected trading days are present. [2025-10-20T21:25:05.151311] [DEBUG] [VALIDATOR ] TDW: Primary data validation passed. [2025-10-20T21:25:05.152425] [INFO ] [MAIN ] TDW: Fetched primary data is valid. [2025-10-20T21:25:05.152489] [DEBUG] [LOCAL ] TDW: Saving primary data. [2025-10-20T21:25:05.153929] [DEBUG] [LOCAL ] TDW: Trimmed 66779 records to 45039 (keeping 200 days of data) [2025-10-20T21:25:05.423326] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T21:25:05.436325] [DEBUG] [LOCAL ] TDW: Primary data saved (45039 records). [2025-10-20T21:25:05.436790] [INFO ] [MAIN ] TDW: Full history saved. [2025-10-20T21:25:05.436884] [INFO ] [MAIN ] Update Progress: 2060/2367 symbols completed. TDW took 30.2s. Estimated remaining: 39m 30s. Rate: 7.8 symbols/min [2025-10-20T21:25:05.606315] [INFO ] [MAIN ] TEF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:05.640305] [DEBUG] [LOCAL ] TEF: Loaded 37415 existing records from primary storage. [2025-10-20T21:25:05.640570] [INFO ] [MAIN ] TEF: Loaded 37415 existing records. [2025-10-20T21:25:05.819960] [WARN ] [MAIN ] TEF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:05.820221] [INFO ] [MAIN ] TEF: Wiping existing primary data before full refresh. [2025-10-20T21:25:05.820273] [INFO ] [LOCAL ] TEF: Starting deletion of primary data. [2025-10-20T21:25:05.820517] [DEBUG] [LOCAL ] TEF: Deleted primary data file. [2025-10-20T21:25:05.820575] [INFO ] [MAIN ] TEF: Starting full primary refresh. [2025-10-20T21:25:05.856700] [INFO ] [REMOTE ] TDY: Fetching primary data for 2025-10. [2025-10-20T21:25:06.181501] [INFO ] [REMOTE ] TEF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:06.651339] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-01. [2025-10-20T21:25:06.781561] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T21:25:06.961626] [INFO ] [REMOTE ] TDY: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T21:25:06.964703] [INFO ] [REMOTE ] TDY: Full fetch complete (31979 records). [2025-10-20T21:25:06.964778] [DEBUG] [VALIDATOR ] TDY: Validating primary data. [2025-10-20T21:25:07.346305] [DEBUG] [VALIDATOR ] TDY: All expected trading days are present. [2025-10-20T21:25:07.346380] [DEBUG] [VALIDATOR ] TDY: Primary data validation passed. [2025-10-20T21:25:07.347036] [INFO ] [MAIN ] TDY: Fetched primary data is valid. [2025-10-20T21:25:07.347096] [DEBUG] [LOCAL ] TDY: Saving primary data. [2025-10-20T21:25:07.347804] [DEBUG] [LOCAL ] TDY: Trimmed 31979 records to 22191 (keeping 200 days of data) [2025-10-20T21:25:07.457705] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-06. [2025-10-20T21:25:07.483837] [DEBUG] [LOCAL ] TDY: Primary data saved (22191 records). [2025-10-20T21:25:07.483938] [INFO ] [MAIN ] TDY: Full history saved. [2025-10-20T21:25:07.484011] [INFO ] [MAIN ] Update Progress: 2061/2367 symbols completed. TDY took 27.9s. Estimated remaining: 39m 21s. Rate: 7.8 symbols/min [2025-10-20T21:25:07.649417] [INFO ] [MAIN ] TEI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:07.669675] [DEBUG] [LOCAL ] TEI: Loaded 21545 existing records from primary storage. [2025-10-20T21:25:07.669917] [INFO ] [MAIN ] TEI: Loaded 21545 existing records. [2025-10-20T21:25:07.843712] [WARN ] [MAIN ] TEI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:07.843907] [INFO ] [MAIN ] TEI: Wiping existing primary data before full refresh. [2025-10-20T21:25:07.843952] [INFO ] [LOCAL ] TEI: Starting deletion of primary data. [2025-10-20T21:25:07.844114] [DEBUG] [LOCAL ] TEI: Deleted primary data file. [2025-10-20T21:25:07.844151] [INFO ] [MAIN ] TEI: Starting full primary refresh. [2025-10-20T21:25:08.201889] [INFO ] [REMOTE ] TEI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:08.222634] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T21:25:08.251347] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-02. [2025-10-20T21:25:09.051378] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-01. [2025-10-20T21:25:09.326288] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:25:09.758265] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T21:25:09.851342] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-03. [2025-10-20T21:25:10.148869] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T21:25:10.651377] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-04. [2025-10-20T21:25:11.395357] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T21:25:11.451334] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-07. [2025-10-20T21:25:11.559795] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T21:25:12.251372] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-02. [2025-10-20T21:25:13.051379] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-04. [2025-10-20T21:25:13.429255] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T21:25:13.607943] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-07 (2.16 seconds). [2025-10-20T21:25:13.851367] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-08. [2025-10-20T21:25:14.651353] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-03. [2025-10-20T21:25:14.682703] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:25:15.263318] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T21:25:15.451342] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-05. [2025-10-20T21:25:15.796825] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:25:16.251372] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-09. [2025-10-20T21:25:16.972574] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:25:17.051367] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-06. [2025-10-20T21:25:17.851365] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-04. [2025-10-20T21:25:18.164927] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T21:25:18.553792] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T21:25:18.574498] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-04 (0.72 seconds). [2025-10-20T21:25:18.651339] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-07. [2025-10-20T21:25:19.451367] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-05. [2025-10-20T21:25:20.065561] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T21:25:20.251383] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-08. [2025-10-20T21:25:20.610190] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T21:25:21.051457] [INFO ] [REMOTE ] TECK: Fetching primary data for 2025-10. [2025-10-20T21:25:21.408938] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T21:25:21.851360] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-05. [2025-10-20T21:25:22.385397] [INFO ] [REMOTE ] TECK: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T21:25:22.393439] [INFO ] [REMOTE ] TECK: Full fetch complete (77525 records). [2025-10-20T21:25:22.393592] [DEBUG] [VALIDATOR ] TECK: Validating primary data. [2025-10-20T21:25:22.669004] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-06. [2025-10-20T21:25:22.708706] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T21:25:22.798165] [DEBUG] [VALIDATOR ] TECK: All expected trading days are present. [2025-10-20T21:25:22.798238] [DEBUG] [VALIDATOR ] TECK: Primary data validation passed. [2025-10-20T21:25:22.799299] [INFO ] [MAIN ] TECK: Fetched primary data is valid. [2025-10-20T21:25:22.799360] [DEBUG] [LOCAL ] TECK: Saving primary data. [2025-10-20T21:25:22.800769] [DEBUG] [LOCAL ] TECK: Trimmed 77525 records to 53435 (keeping 200 days of data) [2025-10-20T21:25:23.128337] [DEBUG] [LOCAL ] TECK: Primary data saved (53435 records). [2025-10-20T21:25:23.128474] [INFO ] [MAIN ] TECK: Full history saved. [2025-10-20T21:25:23.128825] [INFO ] [MAIN ] Update Progress: 2062/2367 symbols completed. TECK took 29.0s. Estimated remaining: 39m 15s. Rate: 7.8 symbols/min [2025-10-20T21:25:23.298402] [INFO ] [MAIN ] TEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:23.349899] [DEBUG] [LOCAL ] TEL: Loaded 51795 existing records from primary storage. [2025-10-20T21:25:23.350189] [INFO ] [MAIN ] TEL: Loaded 51795 existing records. [2025-10-20T21:25:23.452686] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-06. [2025-10-20T21:25:23.486622] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T21:25:23.548298] [WARN ] [MAIN ] TEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:23.548501] [INFO ] [MAIN ] TEL: Wiping existing primary data before full refresh. [2025-10-20T21:25:23.548547] [INFO ] [LOCAL ] TEL: Starting deletion of primary data. [2025-10-20T21:25:23.548762] [DEBUG] [LOCAL ] TEL: Deleted primary data file. [2025-10-20T21:25:23.548817] [INFO ] [MAIN ] TEL: Starting full primary refresh. [2025-10-20T21:25:23.899348] [INFO ] [REMOTE ] TEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:24.251420] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-09. [2025-10-20T21:25:24.289501] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-06 (0.84 seconds). [2025-10-20T21:25:25.051391] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-01. [2025-10-20T21:25:25.738875] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-09 (1.49 seconds). [2025-10-20T21:25:25.851365] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-07. [2025-10-20T21:25:26.651354] [INFO ] [REMOTE ] TEF: Fetching primary data for 2025-10. [2025-10-20T21:25:26.721886] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:25:26.882921] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T21:25:27.451441] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-07. [2025-10-20T21:25:28.019775] [INFO ] [REMOTE ] TEF: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T21:25:28.027997] [INFO ] [REMOTE ] TEF: Full fetch complete (54572 records). [2025-10-20T21:25:28.028127] [DEBUG] [VALIDATOR ] TEF: Validating primary data. [2025-10-20T21:25:28.221291] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T21:25:28.253794] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-08. [2025-10-20T21:25:28.443533] [DEBUG] [VALIDATOR ] TEF: All expected trading days are present. [2025-10-20T21:25:28.443606] [DEBUG] [VALIDATOR ] TEF: Primary data validation passed. [2025-10-20T21:25:28.444543] [INFO ] [MAIN ] TEF: Fetched primary data is valid. [2025-10-20T21:25:28.444605] [DEBUG] [LOCAL ] TEF: Saving primary data. [2025-10-20T21:25:28.445626] [DEBUG] [LOCAL ] TEF: Trimmed 54572 records to 36882 (keeping 200 days of data) [2025-10-20T21:25:28.654159] [DEBUG] [LOCAL ] TEF: Primary data saved (36882 records). [2025-10-20T21:25:28.654318] [INFO ] [MAIN ] TEF: Full history saved. [2025-10-20T21:25:28.654415] [INFO ] [MAIN ] Update Progress: 2063/2367 symbols completed. TEF took 23.2s. Estimated remaining: 39m 7s. Rate: 7.8 symbols/min [2025-10-20T21:25:28.831576] [INFO ] [MAIN ] TEN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:28.861420] [DEBUG] [LOCAL ] TEN: Loaded 30927 existing records from primary storage. [2025-10-20T21:25:28.861697] [INFO ] [MAIN ] TEN: Loaded 30927 existing records. [2025-10-20T21:25:29.035207] [WARN ] [MAIN ] TEN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:29.035391] [INFO ] [MAIN ] TEN: Wiping existing primary data before full refresh. [2025-10-20T21:25:29.035456] [INFO ] [LOCAL ] TEN: Starting deletion of primary data. [2025-10-20T21:25:29.035790] [DEBUG] [LOCAL ] TEN: Deleted primary data file. [2025-10-20T21:25:29.035866] [INFO ] [MAIN ] TEN: Starting full primary refresh. [2025-10-20T21:25:29.056587] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-08. [2025-10-20T21:25:29.209194] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-08 (0.96 seconds). [2025-10-20T21:25:29.407718] [INFO ] [REMOTE ] TEN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:29.791813] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-08 (0.74 seconds). [2025-10-20T21:25:29.851359] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-02. [2025-10-20T21:25:30.651373] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-09. [2025-10-20T21:25:31.439939] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T21:25:31.451321] [INFO ] [REMOTE ] TEAF: Fetching primary data for 2025-10. [2025-10-20T21:25:31.517281] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:25:32.215996] [INFO ] [REMOTE ] TEAF: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T21:25:32.218483] [INFO ] [REMOTE ] TEAF: Full fetch complete (10117 records). [2025-10-20T21:25:32.218583] [DEBUG] [VALIDATOR ] TEAF: Validating primary data. [2025-10-20T21:25:32.258605] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-03. [2025-10-20T21:25:32.584358] [DEBUG] [VALIDATOR ] TEAF: All expected trading days are present. [2025-10-20T21:25:32.584438] [DEBUG] [VALIDATOR ] TEAF: Primary data validation passed. [2025-10-20T21:25:32.584696] [INFO ] [MAIN ] TEAF: Fetched primary data is valid. [2025-10-20T21:25:32.584758] [DEBUG] [LOCAL ] TEAF: Saving primary data. [2025-10-20T21:25:32.585162] [DEBUG] [LOCAL ] TEAF: Trimmed 10117 records to 7386 (keeping 200 days of data) [2025-10-20T21:25:32.628442] [DEBUG] [LOCAL ] TEAF: Primary data saved (7386 records). [2025-10-20T21:25:32.628540] [INFO ] [MAIN ] TEAF: Full history saved. [2025-10-20T21:25:32.628625] [INFO ] [MAIN ] Update Progress: 2064/2367 symbols completed. TEAF took 39.3s. Estimated remaining: 38m 58s. Rate: 7.8 symbols/min [2025-10-20T21:25:32.798793] [INFO ] [MAIN ] TEO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:32.820411] [DEBUG] [LOCAL ] TEO: Loaded 22984 existing records from primary storage. [2025-10-20T21:25:32.820691] [INFO ] [MAIN ] TEO: Loaded 22984 existing records. [2025-10-20T21:25:32.995409] [WARN ] [MAIN ] TEO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:32.995595] [INFO ] [MAIN ] TEO: Wiping existing primary data before full refresh. [2025-10-20T21:25:32.995659] [INFO ] [LOCAL ] TEO: Starting deletion of primary data. [2025-10-20T21:25:32.995847] [DEBUG] [LOCAL ] TEO: Deleted primary data file. [2025-10-20T21:25:32.995885] [INFO ] [MAIN ] TEO: Starting full primary refresh. [2025-10-20T21:25:33.056631] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-09. [2025-10-20T21:25:33.367509] [INFO ] [REMOTE ] TEO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:33.851340] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-01. [2025-10-20T21:25:33.977421] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-09 (0.92 seconds). [2025-10-20T21:25:34.049934] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T21:25:34.651386] [INFO ] [REMOTE ] TEI: Fetching primary data for 2025-10. [2025-10-20T21:25:35.152305] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T21:25:35.451368] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-01. [2025-10-20T21:25:35.712094] [INFO ] [REMOTE ] TEI: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:25:35.718549] [INFO ] [REMOTE ] TEI: Full fetch complete (30707 records). [2025-10-20T21:25:35.718746] [DEBUG] [VALIDATOR ] TEI: Validating primary data. [2025-10-20T21:25:36.187367] [DEBUG] [VALIDATOR ] TEI: All expected trading days are present. [2025-10-20T21:25:36.187489] [DEBUG] [VALIDATOR ] TEI: Primary data validation passed. [2025-10-20T21:25:36.189103] [INFO ] [MAIN ] TEI: Fetched primary data is valid. [2025-10-20T21:25:36.189197] [DEBUG] [LOCAL ] TEI: Saving primary data. [2025-10-20T21:25:36.190363] [DEBUG] [LOCAL ] TEI: Trimmed 30707 records to 21267 (keeping 200 days of data) [2025-10-20T21:25:36.263170] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-02. [2025-10-20T21:25:36.368187] [DEBUG] [LOCAL ] TEI: Primary data saved (21267 records). [2025-10-20T21:25:36.368348] [INFO ] [MAIN ] TEI: Full history saved. [2025-10-20T21:25:36.368450] [INFO ] [MAIN ] Update Progress: 2065/2367 symbols completed. TEI took 28.9s. Estimated remaining: 38m 50s. Rate: 7.8 symbols/min [2025-10-20T21:25:36.622991] [INFO ] [MAIN ] TEVA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:36.648401] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-01 (1.20 seconds). [2025-10-20T21:25:36.709261] [DEBUG] [LOCAL ] TEVA: Loaded 54406 existing records from primary storage. [2025-10-20T21:25:36.709788] [INFO ] [MAIN ] TEVA: Loaded 54406 existing records. [2025-10-20T21:25:36.919848] [WARN ] [MAIN ] TEVA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:36.920064] [INFO ] [MAIN ] TEVA: Wiping existing primary data before full refresh. [2025-10-20T21:25:36.920111] [INFO ] [LOCAL ] TEVA: Starting deletion of primary data. [2025-10-20T21:25:36.920345] [DEBUG] [LOCAL ] TEVA: Deleted primary data file. [2025-10-20T21:25:36.920401] [INFO ] [MAIN ] TEVA: Starting full primary refresh. [2025-10-20T21:25:37.051856] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-04. [2025-10-20T21:25:37.253053] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T21:25:37.313242] [INFO ] [REMOTE ] TEVA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:37.851410] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-02. [2025-10-20T21:25:38.651401] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-01. [2025-10-20T21:25:39.007806] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T21:25:39.040347] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T21:25:39.451430] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-03. [2025-10-20T21:25:40.251378] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-03. [2025-10-20T21:25:40.473479] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T21:25:40.827524] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T21:25:41.051430] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-02. [2025-10-20T21:25:41.313524] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T21:25:41.851377] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-05. [2025-10-20T21:25:42.651372] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-04. [2025-10-20T21:25:43.077734] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-02 (2.03 seconds). [2025-10-20T21:25:43.451366] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-03. [2025-10-20T21:25:43.607989] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T21:25:43.654840] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-04 (1.00 seconds). [2025-10-20T21:25:44.251388] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-04. [2025-10-20T21:25:45.051431] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-05. [2025-10-20T21:25:45.481592] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T21:25:45.684932] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:25:45.851382] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-06. [2025-10-20T21:25:45.996863] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T21:25:46.651422] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-04. [2025-10-20T21:25:47.451404] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-06. [2025-10-20T21:25:47.588058] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T21:25:48.251400] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-07. [2025-10-20T21:25:48.462065] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T21:25:48.616742] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T21:25:49.051355] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-05. [2025-10-20T21:25:49.851345] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-07. [2025-10-20T21:25:50.245680] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T21:25:50.277853] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-05 (1.23 seconds). [2025-10-20T21:25:50.651339] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-08. [2025-10-20T21:25:51.021527] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T21:25:51.451363] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-05. [2025-10-20T21:25:52.251337] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-08. [2025-10-20T21:25:52.358441] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T21:25:53.051354] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-09. [2025-10-20T21:25:53.390917] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-08 (1.14 seconds). [2025-10-20T21:25:53.633348] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-05 (2.18 seconds). [2025-10-20T21:25:53.851528] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-09. [2025-10-20T21:25:54.651355] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-06. [2025-10-20T21:25:54.745615] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-09 (1.69 seconds). [2025-10-20T21:25:55.209039] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T21:25:55.451390] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-06. [2025-10-20T21:25:55.865547] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-06 (1.21 seconds). [2025-10-20T21:25:56.251376] [INFO ] [REMOTE ] TEO: Fetching primary data for 2025-10. [2025-10-20T21:25:57.051410] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-07. [2025-10-20T21:25:57.331119] [INFO ] [REMOTE ] TEO: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T21:25:57.335180] [INFO ] [REMOTE ] TEO: Full fetch complete (32896 records). [2025-10-20T21:25:57.335261] [DEBUG] [VALIDATOR ] TEO: Validating primary data. [2025-10-20T21:25:57.697607] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-06 (2.25 seconds). [2025-10-20T21:25:57.744918] [DEBUG] [VALIDATOR ] TEO: All expected trading days are present. [2025-10-20T21:25:57.744991] [DEBUG] [VALIDATOR ] TEO: Primary data validation passed. [2025-10-20T21:25:57.745622] [INFO ] [MAIN ] TEO: Fetched primary data is valid. [2025-10-20T21:25:57.745693] [DEBUG] [LOCAL ] TEO: Saving primary data. [2025-10-20T21:25:57.746418] [DEBUG] [LOCAL ] TEO: Trimmed 32896 records to 22830 (keeping 200 days of data) [2025-10-20T21:25:57.853691] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-07. [2025-10-20T21:25:57.872276] [DEBUG] [LOCAL ] TEO: Primary data saved (22830 records). [2025-10-20T21:25:57.872368] [INFO ] [MAIN ] TEO: Full history saved. [2025-10-20T21:25:57.872440] [INFO ] [MAIN ] Update Progress: 2066/2367 symbols completed. TEO took 25.2s. Estimated remaining: 38m 44s. Rate: 7.8 symbols/min [2025-10-20T21:25:58.046415] [INFO ] [MAIN ] TEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:25:58.086965] [DEBUG] [LOCAL ] TEX: Loaded 44431 existing records from primary storage. [2025-10-20T21:25:58.087257] [INFO ] [MAIN ] TEX: Loaded 44431 existing records. [2025-10-20T21:25:58.295016] [WARN ] [MAIN ] TEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:25:58.295198] [INFO ] [MAIN ] TEX: Wiping existing primary data before full refresh. [2025-10-20T21:25:58.295243] [INFO ] [LOCAL ] TEX: Starting deletion of primary data. [2025-10-20T21:25:58.295434] [DEBUG] [LOCAL ] TEX: Deleted primary data file. [2025-10-20T21:25:58.295489] [INFO ] [MAIN ] TEX: Starting full primary refresh. [2025-10-20T21:25:58.564792] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T21:25:58.675999] [INFO ] [REMOTE ] TEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:25:58.670741] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-08. [2025-10-20T21:25:59.451435] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-01. [2025-10-20T21:26:00.014923] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T21:26:00.054037] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-07 (2.20 seconds). [2025-10-20T21:26:00.251354] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-08. [2025-10-20T21:26:01.051469] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-09. [2025-10-20T21:26:01.289775] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T21:26:01.851359] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-02. [2025-10-20T21:26:02.134837] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T21:26:02.481796] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T21:26:02.651340] [INFO ] [REMOTE ] TEN: Fetching primary data for 2025-10. [2025-10-20T21:26:03.133560] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T21:26:03.451362] [INFO ] [REMOTE ] TEL: Fetching primary data for 2025-10. [2025-10-20T21:26:03.619281] [INFO ] [REMOTE ] TEN: Incremental fetch complete for 2025-10 (0.97 seconds). [2025-10-20T21:26:03.623605] [INFO ] [REMOTE ] TEN: Full fetch complete (44061 records). [2025-10-20T21:26:03.623711] [DEBUG] [VALIDATOR ] TEN: Validating primary data. [2025-10-20T21:26:03.992432] [DEBUG] [VALIDATOR ] TEN: All expected trading days are present. [2025-10-20T21:26:03.992510] [DEBUG] [VALIDATOR ] TEN: Primary data validation passed. [2025-10-20T21:26:03.993670] [INFO ] [MAIN ] TEN: Fetched primary data is valid. [2025-10-20T21:26:03.993729] [DEBUG] [LOCAL ] TEN: Saving primary data. [2025-10-20T21:26:03.994607] [DEBUG] [LOCAL ] TEN: Trimmed 44061 records to 30597 (keeping 200 days of data) [2025-10-20T21:26:04.170815] [DEBUG] [LOCAL ] TEN: Primary data saved (30597 records). [2025-10-20T21:26:04.170991] [INFO ] [MAIN ] TEN: Full history saved. [2025-10-20T21:26:04.171076] [INFO ] [MAIN ] Update Progress: 2067/2367 symbols completed. TEN took 35.5s. Estimated remaining: 38m 36s. Rate: 7.8 symbols/min [2025-10-20T21:26:04.258912] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-09. [2025-10-20T21:26:04.338482] [INFO ] [MAIN ] TFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:04.387550] [DEBUG] [LOCAL ] TFC: Loaded 54447 existing records from primary storage. [2025-10-20T21:26:04.387933] [INFO ] [MAIN ] TFC: Loaded 54447 existing records. [2025-10-20T21:26:04.559969] [WARN ] [MAIN ] TFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:04.560161] [INFO ] [MAIN ] TFC: Wiping existing primary data before full refresh. [2025-10-20T21:26:04.560206] [INFO ] [LOCAL ] TFC: Starting deletion of primary data. [2025-10-20T21:26:04.560592] [DEBUG] [LOCAL ] TFC: Deleted primary data file. [2025-10-20T21:26:04.560677] [INFO ] [MAIN ] TFC: Starting full primary refresh. [2025-10-20T21:26:04.910097] [INFO ] [REMOTE ] TFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:05.031232] [INFO ] [REMOTE ] TEL: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T21:26:05.038747] [INFO ] [REMOTE ] TEL: Full fetch complete (73772 records). [2025-10-20T21:26:05.038870] [DEBUG] [VALIDATOR ] TEL: Validating primary data. [2025-10-20T21:26:05.059998] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-01. [2025-10-20T21:26:05.430046] [DEBUG] [VALIDATOR ] TEL: All expected trading days are present. [2025-10-20T21:26:05.430130] [DEBUG] [VALIDATOR ] TEL: Primary data validation passed. [2025-10-20T21:26:05.431231] [INFO ] [MAIN ] TEL: Fetched primary data is valid. [2025-10-20T21:26:05.431295] [DEBUG] [LOCAL ] TEL: Saving primary data. [2025-10-20T21:26:05.432631] [DEBUG] [LOCAL ] TEL: Trimmed 73772 records to 51047 (keeping 200 days of data) [2025-10-20T21:26:05.746362] [DEBUG] [LOCAL ] TEL: Primary data saved (51047 records). [2025-10-20T21:26:05.746499] [INFO ] [MAIN ] TEL: Full history saved. [2025-10-20T21:26:05.746883] [INFO ] [MAIN ] Update Progress: 2068/2367 symbols completed. TEL took 42.6s. Estimated remaining: 38m 28s. Rate: 7.8 symbols/min [2025-10-20T21:26:05.856458] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-03. [2025-10-20T21:26:05.919543] [INFO ] [MAIN ] TFII: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:05.953979] [DEBUG] [LOCAL ] TFII: Loaded 33349 existing records from primary storage. [2025-10-20T21:26:05.954260] [INFO ] [MAIN ] TFII: Loaded 33349 existing records. [2025-10-20T21:26:06.137456] [WARN ] [MAIN ] TFII: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:06.137844] [INFO ] [MAIN ] TFII: Wiping existing primary data before full refresh. [2025-10-20T21:26:06.137982] [INFO ] [LOCAL ] TFII: Starting deletion of primary data. [2025-10-20T21:26:06.138324] [DEBUG] [LOCAL ] TFII: Deleted primary data file. [2025-10-20T21:26:06.138378] [INFO ] [MAIN ] TFII: Starting full primary refresh. [2025-10-20T21:26:06.459992] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-09 (2.20 seconds). [2025-10-20T21:26:06.548148] [INFO ] [REMOTE ] TFII: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:06.651403] [INFO ] [REMOTE ] TEVA: Fetching primary data for 2025-10. [2025-10-20T21:26:06.950713] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T21:26:07.230835] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T21:26:07.451364] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-02. [2025-10-20T21:26:08.251390] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-01. [2025-10-20T21:26:08.564212] [INFO ] [REMOTE ] TEVA: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T21:26:08.572232] [INFO ] [REMOTE ] TEVA: Full fetch complete (77785 records). [2025-10-20T21:26:08.572455] [DEBUG] [VALIDATOR ] TEVA: Validating primary data. [2025-10-20T21:26:08.971928] [DEBUG] [VALIDATOR ] TEVA: All expected trading days are present. [2025-10-20T21:26:08.972008] [DEBUG] [VALIDATOR ] TEVA: Primary data validation passed. [2025-10-20T21:26:08.973218] [INFO ] [MAIN ] TEVA: Fetched primary data is valid. [2025-10-20T21:26:08.973282] [DEBUG] [LOCAL ] TEVA: Saving primary data. [2025-10-20T21:26:08.974735] [DEBUG] [LOCAL ] TEVA: Trimmed 77785 records to 53622 (keeping 200 days of data) [2025-10-20T21:26:09.059294] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-04. [2025-10-20T21:26:09.309461] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-02 (1.86 seconds). [2025-10-20T21:26:09.338078] [DEBUG] [LOCAL ] TEVA: Primary data saved (53622 records). [2025-10-20T21:26:09.338181] [INFO ] [MAIN ] TEVA: Full history saved. [2025-10-20T21:26:09.338249] [INFO ] [MAIN ] Update Progress: 2069/2367 symbols completed. TEVA took 33.0s. Estimated remaining: 38m 19s. Rate: 7.8 symbols/min [2025-10-20T21:26:09.511213] [INFO ] [MAIN ] TFIN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:09.526907] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-01 (1.28 seconds). [2025-10-20T21:26:09.551622] [DEBUG] [LOCAL ] TFIN: Loaded 24116 existing records from primary storage. [2025-10-20T21:26:09.551889] [INFO ] [MAIN ] TFIN: Loaded 24116 existing records. [2025-10-20T21:26:09.726920] [WARN ] [MAIN ] TFIN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:09.727105] [INFO ] [MAIN ] TFIN: Wiping existing primary data before full refresh. [2025-10-20T21:26:09.727151] [INFO ] [LOCAL ] TFIN: Starting deletion of primary data. [2025-10-20T21:26:09.727312] [DEBUG] [LOCAL ] TFIN: Deleted primary data file. [2025-10-20T21:26:09.727349] [INFO ] [MAIN ] TFIN: Starting full primary refresh. [2025-10-20T21:26:09.856459] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-03. [2025-10-20T21:26:10.131912] [INFO ] [REMOTE ] TFIN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:10.651390] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-02. [2025-10-20T21:26:10.701288] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:26:11.461202] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-05. [2025-10-20T21:26:11.482465] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:26:12.054099] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-02 (1.40 seconds). [2025-10-20T21:26:12.251365] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-04. [2025-10-20T21:26:12.988568] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:26:13.051341] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-01. [2025-10-20T21:26:13.851420] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-03. [2025-10-20T21:26:13.959058] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-01 (0.91 seconds). [2025-10-20T21:26:14.243963] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T21:26:14.651377] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-05. [2025-10-20T21:26:15.286605] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:26:15.451371] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-04. [2025-10-20T21:26:16.251337] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-06. [2025-10-20T21:26:16.564457] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T21:26:16.615950] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T21:26:17.051383] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-06. [2025-10-20T21:26:17.851393] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-05. [2025-10-20T21:26:17.885049] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T21:26:18.651392] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-07. [2025-10-20T21:26:18.824708] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:26:18.911010] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-05 (1.06 seconds). [2025-10-20T21:26:19.451387] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-07. [2025-10-20T21:26:20.209726] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:26:20.251395] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-02. [2025-10-20T21:26:21.051357] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-08. [2025-10-20T21:26:21.339542] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T21:26:21.376049] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-07 (1.92 seconds). [2025-10-20T21:26:21.851421] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-03. [2025-10-20T21:26:22.607817] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T21:26:22.651341] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-09. [2025-10-20T21:26:22.751838] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T21:26:23.451391] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-06. [2025-10-20T21:26:23.871463] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-09 (1.22 seconds). [2025-10-20T21:26:24.251434] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-04. [2025-10-20T21:26:24.708046] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T21:26:25.051392] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-08. [2025-10-20T21:26:25.498855] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-04 (1.25 seconds). [2025-10-20T21:26:25.851448] [INFO ] [REMOTE ] TEX: Fetching primary data for 2025-10. [2025-10-20T21:26:26.651484] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-05. [2025-10-20T21:26:26.841487] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T21:26:27.229722] [INFO ] [REMOTE ] TEX: Incremental fetch complete for 2025-10 (1.38 seconds). [2025-10-20T21:26:27.235960] [INFO ] [REMOTE ] TEX: Full fetch complete (64418 records). [2025-10-20T21:26:27.236109] [DEBUG] [VALIDATOR ] TEX: Validating primary data. [2025-10-20T21:26:27.457009] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-09. [2025-10-20T21:26:27.647817] [DEBUG] [VALIDATOR ] TEX: All expected trading days are present. [2025-10-20T21:26:27.648019] [DEBUG] [VALIDATOR ] TEX: Primary data validation passed. [2025-10-20T21:26:27.649198] [INFO ] [MAIN ] TEX: Fetched primary data is valid. [2025-10-20T21:26:27.649264] [DEBUG] [LOCAL ] TEX: Saving primary data. [2025-10-20T21:26:27.650579] [DEBUG] [LOCAL ] TEX: Trimmed 64418 records to 43641 (keeping 200 days of data) [2025-10-20T21:26:27.851578] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:26:27.946145] [DEBUG] [LOCAL ] TEX: Primary data saved (43641 records). [2025-10-20T21:26:27.946283] [INFO ] [MAIN ] TEX: Full history saved. [2025-10-20T21:26:27.946364] [INFO ] [MAIN ] Update Progress: 2070/2367 symbols completed. TEX took 30.1s. Estimated remaining: 38m 13s. Rate: 7.8 symbols/min [2025-10-20T21:26:28.111872] [INFO ] [MAIN ] TFPM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:28.147917] [DEBUG] [LOCAL ] TFPM: Loaded 40129 existing records from primary storage. [2025-10-20T21:26:28.148218] [INFO ] [MAIN ] TFPM: Loaded 40129 existing records. [2025-10-20T21:26:28.256517] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-06. [2025-10-20T21:26:28.324327] [WARN ] [MAIN ] TFPM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:28.324502] [INFO ] [MAIN ] TFPM: Wiping existing primary data before full refresh. [2025-10-20T21:26:28.324547] [INFO ] [LOCAL ] TFPM: Starting deletion of primary data. [2025-10-20T21:26:28.324805] [DEBUG] [LOCAL ] TFPM: Deleted primary data file. [2025-10-20T21:26:28.324860] [INFO ] [MAIN ] TFPM: Starting full primary refresh. [2025-10-20T21:26:28.680137] [INFO ] [REMOTE ] TFPM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:29.051425] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-07. [2025-10-20T21:26:29.363442] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:26:29.411293] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T21:26:29.851444] [INFO ] [REMOTE ] TFC: Fetching primary data for 2025-10. [2025-10-20T21:26:30.424788] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T21:26:30.651407] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-07. [2025-10-20T21:26:31.451390] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-08. [2025-10-20T21:26:31.490454] [INFO ] [REMOTE ] TFC: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T21:26:31.499037] [INFO ] [REMOTE ] TFC: Full fetch complete (77843 records). [2025-10-20T21:26:31.499244] [DEBUG] [VALIDATOR ] TFC: Validating primary data. [2025-10-20T21:26:31.868846] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-07 (1.22 seconds). [2025-10-20T21:26:31.947969] [DEBUG] [VALIDATOR ] TFC: All expected trading days are present. [2025-10-20T21:26:31.948054] [DEBUG] [VALIDATOR ] TFC: Primary data validation passed. [2025-10-20T21:26:31.949160] [INFO ] [MAIN ] TFC: Fetched primary data is valid. [2025-10-20T21:26:31.949222] [DEBUG] [LOCAL ] TFC: Saving primary data. [2025-10-20T21:26:31.950606] [DEBUG] [LOCAL ] TFC: Trimmed 77843 records to 53667 (keeping 200 days of data) [2025-10-20T21:26:32.251797] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-01. [2025-10-20T21:26:32.271053] [DEBUG] [LOCAL ] TFC: Primary data saved (53667 records). [2025-10-20T21:26:32.271191] [INFO ] [MAIN ] TFC: Full history saved. [2025-10-20T21:26:32.271269] [INFO ] [MAIN ] Update Progress: 2071/2367 symbols completed. TFC took 28.1s. Estimated remaining: 38m 5s. Rate: 7.8 symbols/min [2025-10-20T21:26:32.441046] [INFO ] [MAIN ] TFX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:32.477432] [DEBUG] [LOCAL ] TFX: Loaded 36700 existing records from primary storage. [2025-10-20T21:26:32.477804] [INFO ] [MAIN ] TFX: Loaded 36700 existing records. [2025-10-20T21:26:32.654794] [WARN ] [MAIN ] TFX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:32.655029] [INFO ] [MAIN ] TFX: Wiping existing primary data before full refresh. [2025-10-20T21:26:32.655199] [INFO ] [LOCAL ] TFX: Starting deletion of primary data. [2025-10-20T21:26:32.655545] [DEBUG] [LOCAL ] TFX: Deleted primary data file. [2025-10-20T21:26:32.655637] [INFO ] [MAIN ] TFX: Starting full primary refresh. [2025-10-20T21:26:32.802783] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-08 (1.35 seconds). [2025-10-20T21:26:33.034593] [INFO ] [REMOTE ] TFX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:33.051383] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-01. [2025-10-20T21:26:33.483863] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T21:26:33.851462] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-09. [2025-10-20T21:26:34.528499] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T21:26:34.651357] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-02. [2025-10-20T21:26:35.170307] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T21:26:35.451432] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-02. [2025-10-20T21:26:35.959314] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:26:36.251412] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-03. [2025-10-20T21:26:36.519022] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T21:26:37.051408] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-08. [2025-10-20T21:26:37.851359] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-03. [2025-10-20T21:26:37.944744] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-03 (1.69 seconds). [2025-10-20T21:26:38.082545] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:26:38.651388] [INFO ] [REMOTE ] TFII: Fetching primary data for 2025-10. [2025-10-20T21:26:39.185943] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-03 (1.33 seconds). [2025-10-20T21:26:39.451373] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-09. [2025-10-20T21:26:39.828351] [INFO ] [REMOTE ] TFII: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T21:26:39.832960] [INFO ] [REMOTE ] TFII: Full fetch complete (48020 records). [2025-10-20T21:26:39.833085] [DEBUG] [VALIDATOR ] TFII: Validating primary data. [2025-10-20T21:26:40.207215] [DEBUG] [VALIDATOR ] TFII: All expected trading days are present. [2025-10-20T21:26:40.207314] [DEBUG] [VALIDATOR ] TFII: Primary data validation passed. [2025-10-20T21:26:40.208200] [INFO ] [MAIN ] TFII: Fetched primary data is valid. [2025-10-20T21:26:40.208261] [DEBUG] [LOCAL ] TFII: Saving primary data. [2025-10-20T21:26:40.209262] [DEBUG] [LOCAL ] TFII: Trimmed 48020 records to 32631 (keeping 200 days of data) [2025-10-20T21:26:40.261969] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-04. [2025-10-20T21:26:40.408513] [DEBUG] [LOCAL ] TFII: Primary data saved (32631 records). [2025-10-20T21:26:40.408662] [INFO ] [MAIN ] TFII: Full history saved. [2025-10-20T21:26:40.409004] [INFO ] [MAIN ] Update Progress: 2072/2367 symbols completed. TFII took 34.7s. Estimated remaining: 37m 57s. Rate: 7.8 symbols/min [2025-10-20T21:26:40.578771] [INFO ] [MAIN ] TG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:40.592484] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T21:26:40.604934] [DEBUG] [LOCAL ] TG: Loaded 10766 existing records from primary storage. [2025-10-20T21:26:40.605125] [INFO ] [MAIN ] TG: Loaded 10766 existing records. [2025-10-20T21:26:40.777079] [WARN ] [MAIN ] TG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:40.777267] [INFO ] [MAIN ] TG: Wiping existing primary data before full refresh. [2025-10-20T21:26:40.777312] [INFO ] [LOCAL ] TG: Starting deletion of primary data. [2025-10-20T21:26:40.777463] [DEBUG] [LOCAL ] TG: Deleted primary data file. [2025-10-20T21:26:40.777498] [INFO ] [MAIN ] TG: Starting full primary refresh. [2025-10-20T21:26:41.051507] [INFO ] [REMOTE ] TFIN: Fetching primary data for 2025-10. [2025-10-20T21:26:41.141519] [INFO ] [REMOTE ] TG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:41.527257] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T21:26:41.851368] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-01. [2025-10-20T21:26:42.155830] [INFO ] [REMOTE ] TFIN: Incremental fetch complete for 2025-10 (1.10 seconds). [2025-10-20T21:26:42.159868] [INFO ] [REMOTE ] TFIN: Full fetch complete (32467 records). [2025-10-20T21:26:42.159941] [DEBUG] [VALIDATOR ] TFIN: Validating primary data. [2025-10-20T21:26:42.472093] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-01 (0.62 seconds). [2025-10-20T21:26:42.553543] [DEBUG] [VALIDATOR ] TFIN: All expected trading days are present. [2025-10-20T21:26:42.553615] [DEBUG] [VALIDATOR ] TFIN: Primary data validation passed. [2025-10-20T21:26:42.554292] [INFO ] [MAIN ] TFIN: Fetched primary data is valid. [2025-10-20T21:26:42.554349] [DEBUG] [LOCAL ] TFIN: Saving primary data. [2025-10-20T21:26:42.555140] [DEBUG] [LOCAL ] TFIN: Trimmed 32467 records to 23845 (keeping 200 days of data) [2025-10-20T21:26:42.651406] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-04. [2025-10-20T21:26:42.692597] [DEBUG] [LOCAL ] TFIN: Primary data saved (23845 records). [2025-10-20T21:26:42.692748] [INFO ] [MAIN ] TFIN: Full history saved. [2025-10-20T21:26:42.692833] [INFO ] [MAIN ] Update Progress: 2073/2367 symbols completed. TFIN took 33.4s. Estimated remaining: 37m 49s. Rate: 7.8 symbols/min [2025-10-20T21:26:42.862560] [INFO ] [MAIN ] TGE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:42.862737] [DEBUG] [LOCAL ] TGE: No existing data file found in primary storage. [2025-10-20T21:26:42.862785] [WARN ] [MAIN ] TGE: No existing data found, performing full refresh instead. [2025-10-20T21:26:42.862893] [INFO ] [MAIN ] TGE: Wiping existing primary data before full refresh. [2025-10-20T21:26:42.862935] [INFO ] [LOCAL ] TGE: Starting deletion of primary data. [2025-10-20T21:26:42.862977] [DEBUG] [LOCAL ] TGE: No primary data file found to delete. [2025-10-20T21:26:42.863011] [INFO ] [MAIN ] TGE: Starting full primary refresh. [2025-10-20T21:26:43.221403] [INFO ] [REMOTE ] TGE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:43.451373] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-02. [2025-10-20T21:26:44.251408] [INFO ] [REMOTE ] TGE: Fetching primary data for 2025-01. [2025-10-20T21:26:44.259397] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:26:44.286464] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:26:44.489696] [DEBUG] [REMOTE ] TGE: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:26:44.489998] [WARN ] [REMOTE ] TGE: No data returned for month 2025-01 [2025-10-20T21:26:44.490084] [FATAL] [REMOTE ] TGE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:26:44.490133] [FATAL] [MAIN ] TGE: Failed to fetch primary data. [2025-10-20T21:26:44.490167] [WARN ] [MAIN ] TGE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:26:44.490204] [INFO ] [LOCAL ] TGE: Starting deletion of primary data. [2025-10-20T21:26:44.490258] [DEBUG] [LOCAL ] TGE: No primary data file found to delete. [2025-10-20T21:26:44.490294] [INFO ] [LOCAL ] TGE: Starting deletion of derivative data. [2025-10-20T21:26:44.490333] [DEBUG] [LOCAL ] TGE: No derivative data file found to delete. [2025-10-20T21:26:44.490415] [INFO ] [MAIN ] Update Progress: 2074/2367 symbols completed. TGE took 1.8s. Estimated remaining: 37m 40s. Rate: 7.8 symbols/min [2025-10-20T21:26:44.667238] [INFO ] [MAIN ] TGLS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:44.697441] [DEBUG] [LOCAL ] TGLS: Loaded 27901 existing records from primary storage. [2025-10-20T21:26:44.697751] [INFO ] [MAIN ] TGLS: Loaded 27901 existing records. [2025-10-20T21:26:44.877987] [WARN ] [MAIN ] TGLS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:44.878205] [INFO ] [MAIN ] TGLS: Wiping existing primary data before full refresh. [2025-10-20T21:26:44.878250] [INFO ] [LOCAL ] TGLS: Starting deletion of primary data. [2025-10-20T21:26:44.878413] [DEBUG] [LOCAL ] TGLS: Deleted primary data file. [2025-10-20T21:26:44.878451] [INFO ] [MAIN ] TGLS: Starting full primary refresh. [2025-10-20T21:26:45.058604] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-03. [2025-10-20T21:26:45.264797] [INFO ] [REMOTE ] TGLS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:45.851399] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-01. [2025-10-20T21:26:46.009710] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T21:26:46.651377] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-04. [2025-10-20T21:26:47.188837] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T21:26:47.451371] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-05. [2025-10-20T21:26:47.480821] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-04 (0.83 seconds). [2025-10-20T21:26:48.251391] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-05. [2025-10-20T21:26:49.000708] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T21:26:49.051327] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-06. [2025-10-20T21:26:49.690166] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T21:26:49.851389] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-06. [2025-10-20T21:26:50.492450] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T21:26:50.651360] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-07. [2025-10-20T21:26:51.241903] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-06 (1.39 seconds). [2025-10-20T21:26:51.451391] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-02. [2025-10-20T21:26:52.124266] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T21:26:52.251334] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-08. [2025-10-20T21:26:52.602387] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T21:26:53.051447] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-07. [2025-10-20T21:26:53.514677] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-08 (1.26 seconds). [2025-10-20T21:26:53.851376] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-09. [2025-10-20T21:26:54.494915] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:26:54.651448] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-03. [2025-10-20T21:26:55.425051] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:26:55.451333] [INFO ] [REMOTE ] TFPM: Fetching primary data for 2025-10. [2025-10-20T21:26:55.732313] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T21:26:56.251387] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-04. [2025-10-20T21:26:57.037740] [INFO ] [REMOTE ] TFPM: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T21:26:57.044447] [INFO ] [REMOTE ] TFPM: Full fetch complete (52758 records). [2025-10-20T21:26:57.044544] [DEBUG] [VALIDATOR ] TFPM: Validating primary data. [2025-10-20T21:26:57.058117] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-08. [2025-10-20T21:26:57.438822] [DEBUG] [VALIDATOR ] TFPM: All expected trading days are present. [2025-10-20T21:26:57.438913] [DEBUG] [VALIDATOR ] TFPM: Primary data validation passed. [2025-10-20T21:26:57.440007] [INFO ] [MAIN ] TFPM: Fetched primary data is valid. [2025-10-20T21:26:57.440096] [DEBUG] [LOCAL ] TFPM: Saving primary data. [2025-10-20T21:26:57.441240] [DEBUG] [LOCAL ] TFPM: Trimmed 52758 records to 39797 (keeping 200 days of data) [2025-10-20T21:26:57.553218] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-04 (1.30 seconds). [2025-10-20T21:26:57.700374] [DEBUG] [LOCAL ] TFPM: Primary data saved (39797 records). [2025-10-20T21:26:57.700505] [INFO ] [MAIN ] TFPM: Full history saved. [2025-10-20T21:26:57.700580] [INFO ] [MAIN ] Update Progress: 2075/2367 symbols completed. TFPM took 29.8s. Estimated remaining: 37m 33s. Rate: 7.8 symbols/min [2025-10-20T21:26:57.852437] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-05. [2025-10-20T21:26:57.870175] [INFO ] [MAIN ] TGNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:26:57.916356] [DEBUG] [LOCAL ] TGNA: Loaded 49331 existing records from primary storage. [2025-10-20T21:26:57.916704] [INFO ] [MAIN ] TGNA: Loaded 49331 existing records. [2025-10-20T21:26:58.091583] [WARN ] [MAIN ] TGNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:26:58.091866] [INFO ] [MAIN ] TGNA: Wiping existing primary data before full refresh. [2025-10-20T21:26:58.091926] [INFO ] [LOCAL ] TGNA: Starting deletion of primary data. [2025-10-20T21:26:58.092134] [DEBUG] [LOCAL ] TGNA: Deleted primary data file. [2025-10-20T21:26:58.092190] [INFO ] [MAIN ] TGNA: Starting full primary refresh. [2025-10-20T21:26:58.472717] [INFO ] [REMOTE ] TGNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:26:58.494751] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:26:58.628875] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T21:26:58.651324] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-06. [2025-10-20T21:26:59.441952] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-06 (0.79 seconds). [2025-10-20T21:26:59.451317] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-07. [2025-10-20T21:27:00.218811] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T21:27:00.251371] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-08. [2025-10-20T21:27:01.051357] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-09. [2025-10-20T21:27:01.170430] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T21:27:01.851427] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-01. [2025-10-20T21:27:02.635204] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T21:27:02.651367] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-09. [2025-10-20T21:27:03.404588] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:27:03.438181] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T21:27:03.451328] [INFO ] [REMOTE ] TG: Fetching primary data for 2025-10. [2025-10-20T21:27:04.248404] [INFO ] [REMOTE ] TG: Incremental fetch complete for 2025-10 (0.80 seconds). [2025-10-20T21:27:04.250858] [INFO ] [REMOTE ] TG: Full fetch complete (15185 records). [2025-10-20T21:27:04.250925] [DEBUG] [VALIDATOR ] TG: Validating primary data. [2025-10-20T21:27:04.256740] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-05. [2025-10-20T21:27:04.626548] [DEBUG] [VALIDATOR ] TG: All expected trading days are present. [2025-10-20T21:27:04.626616] [DEBUG] [VALIDATOR ] TG: Primary data validation passed. [2025-10-20T21:27:04.626977] [INFO ] [MAIN ] TG: Fetched primary data is valid. [2025-10-20T21:27:04.627032] [DEBUG] [LOCAL ] TG: Saving primary data. [2025-10-20T21:27:04.627496] [DEBUG] [LOCAL ] TG: Trimmed 15185 records to 10704 (keeping 200 days of data) [2025-10-20T21:27:04.688312] [DEBUG] [LOCAL ] TG: Primary data saved (10704 records). [2025-10-20T21:27:04.688451] [INFO ] [MAIN ] TG: Full history saved. [2025-10-20T21:27:04.688525] [INFO ] [MAIN ] Update Progress: 2076/2367 symbols completed. TG took 24.3s. Estimated remaining: 37m 26s. Rate: 7.8 symbols/min [2025-10-20T21:27:04.858365] [INFO ] [MAIN ] TGS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:04.890543] [DEBUG] [LOCAL ] TGS: Loaded 33978 existing records from primary storage. [2025-10-20T21:27:04.890850] [INFO ] [MAIN ] TGS: Loaded 33978 existing records. [2025-10-20T21:27:05.060507] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-02. [2025-10-20T21:27:05.065742] [WARN ] [MAIN ] TGS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:05.066075] [INFO ] [MAIN ] TGS: Wiping existing primary data before full refresh. [2025-10-20T21:27:05.067082] [INFO ] [LOCAL ] TGS: Starting deletion of primary data. [2025-10-20T21:27:05.067400] [DEBUG] [LOCAL ] TGS: Deleted primary data file. [2025-10-20T21:27:05.067468] [INFO ] [MAIN ] TGS: Starting full primary refresh. [2025-10-20T21:27:05.425833] [INFO ] [REMOTE ] TGS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:05.686172] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T21:27:05.851440] [INFO ] [REMOTE ] TFX: Fetching primary data for 2025-10. [2025-10-20T21:27:06.656744] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-06. [2025-10-20T21:27:06.669066] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T21:27:07.127242] [INFO ] [REMOTE ] TFX: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T21:27:07.132790] [INFO ] [REMOTE ] TFX: Full fetch complete (54764 records). [2025-10-20T21:27:07.132930] [DEBUG] [VALIDATOR ] TFX: Validating primary data. [2025-10-20T21:27:07.451392] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-03. [2025-10-20T21:27:07.522180] [DEBUG] [VALIDATOR ] TFX: All expected trading days are present. [2025-10-20T21:27:07.522272] [DEBUG] [VALIDATOR ] TFX: Primary data validation passed. [2025-10-20T21:27:07.523263] [INFO ] [MAIN ] TFX: Fetched primary data is valid. [2025-10-20T21:27:07.523343] [DEBUG] [LOCAL ] TFX: Saving primary data. [2025-10-20T21:27:07.524753] [DEBUG] [LOCAL ] TFX: Trimmed 54764 records to 35956 (keeping 200 days of data) [2025-10-20T21:27:07.541425] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T21:27:07.764154] [DEBUG] [LOCAL ] TFX: Primary data saved (35956 records). [2025-10-20T21:27:07.764283] [INFO ] [MAIN ] TFX: Full history saved. [2025-10-20T21:27:07.764355] [INFO ] [MAIN ] Update Progress: 2077/2367 symbols completed. TFX took 35.5s. Estimated remaining: 37m 17s. Rate: 7.8 symbols/min [2025-10-20T21:27:08.251380] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-07. [2025-10-20T21:27:09.051391] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-01. [2025-10-20T21:27:09.236149] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:27:09.504321] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-07 (1.25 seconds). [2025-10-20T21:27:09.851392] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-04. [2025-10-20T21:27:10.491180] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-01 (1.44 seconds). [2025-10-20T21:27:10.651437] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-08. [2025-10-20T21:27:11.451403] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-02. [2025-10-20T21:27:11.625304] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T21:27:11.980251] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T21:27:12.251442] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-05. [2025-10-20T21:27:12.402686] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T21:27:13.051451] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-03. [2025-10-20T21:27:13.851364] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-09. [2025-10-20T21:27:13.925295] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T21:27:14.063294] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-03 (1.01 seconds). [2025-10-20T21:27:14.651381] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-04. [2025-10-20T21:27:15.061379] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T21:27:15.451420] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-06. [2025-10-20T21:27:16.082556] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:27:16.251389] [INFO ] [REMOTE ] TGLS: Fetching primary data for 2025-10. [2025-10-20T21:27:17.028568] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:27:17.051327] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-07. [2025-10-20T21:27:17.457845] [INFO ] [REMOTE ] TGLS: Incremental fetch complete for 2025-10 (1.21 seconds). [2025-10-20T21:27:17.461978] [INFO ] [REMOTE ] TGLS: Full fetch complete (40586 records). [2025-10-20T21:27:17.462082] [DEBUG] [VALIDATOR ] TGLS: Validating primary data. [2025-10-20T21:27:17.844939] [DEBUG] [VALIDATOR ] TGLS: All expected trading days are present. [2025-10-20T21:27:17.845037] [DEBUG] [VALIDATOR ] TGLS: Primary data validation passed. [2025-10-20T21:27:17.845852] [INFO ] [MAIN ] TGLS: Fetched primary data is valid. [2025-10-20T21:27:17.845915] [DEBUG] [LOCAL ] TGLS: Saving primary data. [2025-10-20T21:27:17.846919] [DEBUG] [LOCAL ] TGLS: Trimmed 40586 records to 27616 (keeping 200 days of data) [2025-10-20T21:27:17.899357] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-05. [2025-10-20T21:27:18.018536] [DEBUG] [LOCAL ] TGLS: Primary data saved (27616 records). [2025-10-20T21:27:18.018709] [INFO ] [MAIN ] TGLS: Full history saved. [2025-10-20T21:27:18.018805] [INFO ] [MAIN ] Update Progress: 2078/2367 symbols completed. TGLS took 33.5s. Estimated remaining: 37m 10s. Rate: 7.8 symbols/min [2025-10-20T21:27:18.685881] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T21:27:18.696319] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-08. [2025-10-20T21:27:18.932011] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-05 (1.03 seconds). [2025-10-20T21:27:19.451364] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-06. [2025-10-20T21:27:20.166119] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T21:27:20.251390] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-09. [2025-10-20T21:27:20.822146] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T21:27:21.051435] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-07. [2025-10-20T21:27:21.571613] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T21:27:21.851436] [INFO ] [REMOTE ] TGNA: Fetching primary data for 2025-10. [2025-10-20T21:27:22.516984] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-07 (1.47 seconds). [2025-10-20T21:27:22.651432] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-08. [2025-10-20T21:27:23.473637] [INFO ] [REMOTE ] TGNA: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T21:27:23.486188] [INFO ] [REMOTE ] TGNA: Full fetch complete (70118 records). [2025-10-20T21:27:23.486490] [DEBUG] [VALIDATOR ] TGNA: Validating primary data. [2025-10-20T21:27:23.989437] [DEBUG] [VALIDATOR ] TGNA: All expected trading days are present. [2025-10-20T21:27:23.989620] [DEBUG] [VALIDATOR ] TGNA: Primary data validation passed. [2025-10-20T21:27:23.990901] [INFO ] [MAIN ] TGNA: Fetched primary data is valid. [2025-10-20T21:27:23.991080] [DEBUG] [LOCAL ] TGNA: Saving primary data. [2025-10-20T21:27:24.015628] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T21:27:24.025488] [DEBUG] [LOCAL ] TGNA: Trimmed 70118 records to 48621 (keeping 200 days of data) [2025-10-20T21:27:24.026991] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-09. [2025-10-20T21:27:24.329073] [DEBUG] [LOCAL ] TGNA: Primary data saved (48621 records). [2025-10-20T21:27:24.329248] [INFO ] [MAIN ] TGNA: Full history saved. [2025-10-20T21:27:24.329341] [INFO ] [MAIN ] Update Progress: 2079/2367 symbols completed. TGNA took 26.6s. Estimated remaining: 37m 2s. Rate: 7.8 symbols/min [2025-10-20T21:27:25.599746] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:27:25.608605] [INFO ] [REMOTE ] TGS: Fetching primary data for 2025-10. [2025-10-20T21:27:26.963452] [INFO ] [REMOTE ] TGS: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T21:27:26.969217] [INFO ] [REMOTE ] TGS: Full fetch complete (48762 records). [2025-10-20T21:27:26.969314] [DEBUG] [VALIDATOR ] TGS: Validating primary data. [2025-10-20T21:27:27.348345] [DEBUG] [VALIDATOR ] TGS: All expected trading days are present. [2025-10-20T21:27:27.348423] [DEBUG] [VALIDATOR ] TGS: Primary data validation passed. [2025-10-20T21:27:27.349387] [INFO ] [MAIN ] TGS: Fetched primary data is valid. [2025-10-20T21:27:27.349447] [DEBUG] [LOCAL ] TGS: Saving primary data. [2025-10-20T21:27:27.350584] [DEBUG] [LOCAL ] TGS: Trimmed 48762 records to 33811 (keeping 200 days of data) [2025-10-20T21:27:27.549180] [DEBUG] [LOCAL ] TGS: Primary data saved (33811 records). [2025-10-20T21:27:27.549342] [INFO ] [MAIN ] TGS: Full history saved. [2025-10-20T21:27:27.549771] [INFO ] [MAIN ] Update Progress: 2080/2367 symbols completed. TGS took 22.9s. Estimated remaining: 36m 54s. Rate: 7.8 symbols/min [2025-10-20T21:27:28.253970] [INFO ] [MAIN ] TGT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:28.329249] [INFO ] [MAIN ] THIR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:28.327776] [INFO ] [MAIN ] THC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:28.320618] [INFO ] [MAIN ] THG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:28.337781] [DEBUG] [LOCAL ] THIR: Loaded 2413 existing records from primary storage. [2025-10-20T21:27:28.338325] [INFO ] [MAIN ] THIR: Loaded 2413 existing records. [2025-10-20T21:27:28.587833] [WARN ] [MAIN ] THIR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:28.564006] [DEBUG] [LOCAL ] THG: Loaded 22090 existing records from primary storage. [2025-10-20T21:27:28.589375] [INFO ] [MAIN ] THIR: Wiping existing primary data before full refresh. [2025-10-20T21:27:28.590929] [DEBUG] [LOCAL ] THC: Loaded 47601 existing records from primary storage. [2025-10-20T21:27:28.589981] [INFO ] [LOCAL ] THIR: Starting deletion of primary data. [2025-10-20T21:27:28.607429] [INFO ] [MAIN ] THC: Loaded 47601 existing records. [2025-10-20T21:27:28.589831] [INFO ] [MAIN ] THG: Loaded 22090 existing records. [2025-10-20T21:27:28.628292] [DEBUG] [LOCAL ] THIR: Deleted primary data file. [2025-10-20T21:27:28.684238] [DEBUG] [LOCAL ] TGT: Loaded 54558 existing records from primary storage. [2025-10-20T21:27:28.704253] [INFO ] [MAIN ] TGT: Loaded 54558 existing records. [2025-10-20T21:27:28.704402] [INFO ] [MAIN ] THIR: Starting full primary refresh. [2025-10-20T21:27:29.284146] [WARN ] [MAIN ] THC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:29.335266] [WARN ] [MAIN ] THG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:29.387233] [INFO ] [MAIN ] THC: Wiping existing primary data before full refresh. [2025-10-20T21:27:29.436293] [INFO ] [MAIN ] THG: Wiping existing primary data before full refresh. [2025-10-20T21:27:29.444373] [INFO ] [LOCAL ] THC: Starting deletion of primary data. [2025-10-20T21:27:29.482240] [WARN ] [MAIN ] TGT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:29.498376] [INFO ] [MAIN ] TGT: Wiping existing primary data before full refresh. [2025-10-20T21:27:29.509045] [INFO ] [LOCAL ] TGT: Starting deletion of primary data. [2025-10-20T21:27:29.524347] [INFO ] [LOCAL ] THG: Starting deletion of primary data. [2025-10-20T21:27:29.524413] [DEBUG] [LOCAL ] THC: Deleted primary data file. [2025-10-20T21:27:29.525206] [DEBUG] [LOCAL ] TGT: Deleted primary data file. [2025-10-20T21:27:29.525407] [INFO ] [MAIN ] THC: Starting full primary refresh. [2025-10-20T21:27:29.535929] [DEBUG] [LOCAL ] THG: Deleted primary data file. [2025-10-20T21:27:29.536062] [INFO ] [MAIN ] TGT: Starting full primary refresh. [2025-10-20T21:27:29.568909] [INFO ] [MAIN ] THG: Starting full primary refresh. [2025-10-20T21:27:29.757581] [INFO ] [REMOTE ] THIR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:29.826670] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-01. [2025-10-20T21:27:30.480589] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T21:27:30.525263] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-02. [2025-10-20T21:27:30.763759] [INFO ] [REMOTE ] THC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:30.771529] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-01. [2025-10-20T21:27:30.733975] [INFO ] [REMOTE ] THG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:30.776849] [INFO ] [REMOTE ] TGT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:30.777271] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-01. [2025-10-20T21:27:30.777619] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-01. [2025-10-20T21:27:31.028453] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T21:27:31.029461] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-03. [2025-10-20T21:27:31.378328] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T21:27:31.379471] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-04. [2025-10-20T21:27:31.803556] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T21:27:31.804855] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-05. [2025-10-20T21:27:31.851314] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-01 (1.07 seconds). [2025-10-20T21:27:31.855728] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-02. [2025-10-20T21:27:32.291719] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:27:32.292866] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-06. [2025-10-20T21:27:32.334551] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T21:27:32.445109] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:27:32.741167] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-06 (0.45 seconds). [2025-10-20T21:27:32.982789] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T21:27:33.051368] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-03. [2025-10-20T21:27:33.851383] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-02. [2025-10-20T21:27:34.168792] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T21:27:34.651379] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-04. [2025-10-20T21:27:35.451450] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-02. [2025-10-20T21:27:35.497197] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T21:27:36.091307] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-04 (1.44 seconds). [2025-10-20T21:27:36.251414] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-05. [2025-10-20T21:27:37.051350] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-07. [2025-10-20T21:27:37.374248] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-07 (0.32 seconds). [2025-10-20T21:27:37.388570] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T21:27:37.522753] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-02 (2.07 seconds). [2025-10-20T21:27:37.851365] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-08. [2025-10-20T21:27:38.413670] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T21:27:38.651369] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-06. [2025-10-20T21:27:39.451418] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-03. [2025-10-20T21:27:39.640869] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T21:27:40.251441] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-07. [2025-10-20T21:27:41.051367] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-09. [2025-10-20T21:27:41.288113] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T21:27:41.484750] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-07 (1.23 seconds). [2025-10-20T21:27:41.521393] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T21:27:41.851396] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-08. [2025-10-20T21:27:42.651349] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-03. [2025-10-20T21:27:42.891509] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T21:27:43.451426] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-09. [2025-10-20T21:27:44.251394] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-04. [2025-10-20T21:27:44.483194] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T21:27:45.051441] [INFO ] [REMOTE ] THIR: Fetching primary data for 2025-10. [2025-10-20T21:27:45.105722] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-10-20T21:27:45.401138] [INFO ] [REMOTE ] THIR: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T21:27:45.402124] [INFO ] [REMOTE ] THIR: Full fetch complete (2905 records). [2025-10-20T21:27:45.402191] [DEBUG] [VALIDATOR ] THIR: Validating primary data. [2025-10-20T21:27:45.780899] [DEBUG] [VALIDATOR ] THIR: All expected trading days are present. [2025-10-20T21:27:45.781032] [DEBUG] [VALIDATOR ] THIR: Primary data validation passed. [2025-10-20T21:27:45.781189] [INFO ] [MAIN ] THIR: Fetched primary data is valid. [2025-10-20T21:27:45.781264] [DEBUG] [LOCAL ] THIR: Saving primary data. [2025-10-20T21:27:45.781827] [DEBUG] [LOCAL ] THIR: Trimmed 2905 records to 2407 (keeping 200 days of data) [2025-10-20T21:27:45.795935] [DEBUG] [LOCAL ] THIR: Primary data saved (2407 records). [2025-10-20T21:27:45.796046] [INFO ] [MAIN ] THIR: Full history saved. [2025-10-20T21:27:45.796120] [INFO ] [MAIN ] Update Progress: 2081/2367 symbols completed. THIR took 18.2s. Estimated remaining: 36m 47s. Rate: 7.8 symbols/min [2025-10-20T21:27:45.857938] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-04. [2025-10-20T21:27:45.967503] [INFO ] [MAIN ] THLV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:45.971177] [DEBUG] [LOCAL ] THLV: Loaded 1535 existing records from primary storage. [2025-10-20T21:27:45.971327] [INFO ] [MAIN ] THLV: Loaded 1535 existing records. [2025-10-20T21:27:46.157193] [WARN ] [MAIN ] THLV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:46.175957] [INFO ] [MAIN ] THLV: Wiping existing primary data before full refresh. [2025-10-20T21:27:46.176232] [INFO ] [LOCAL ] THLV: Starting deletion of primary data. [2025-10-20T21:27:46.176735] [DEBUG] [LOCAL ] THLV: Deleted primary data file. [2025-10-20T21:27:46.176795] [INFO ] [MAIN ] THLV: Starting full primary refresh. [2025-10-20T21:27:46.177768] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-04 (1.93 seconds). [2025-10-20T21:27:46.534401] [INFO ] [REMOTE ] THLV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:46.651361] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-01. [2025-10-20T21:27:47.095151] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-01 (0.44 seconds). [2025-10-20T21:27:47.451412] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-05. [2025-10-20T21:27:48.251387] [INFO ] [REMOTE ] THG: Fetching primary data for 2025-10. [2025-10-20T21:27:48.340317] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-04 (2.48 seconds). [2025-10-20T21:27:49.051345] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-02. [2025-10-20T21:27:49.183908] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T21:27:49.238307] [INFO ] [REMOTE ] THG: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T21:27:49.241559] [INFO ] [REMOTE ] THG: Full fetch complete (31425 records). [2025-10-20T21:27:49.241631] [DEBUG] [VALIDATOR ] THG: Validating primary data. [2025-10-20T21:27:49.531185] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-02 (0.48 seconds). [2025-10-20T21:27:49.615781] [DEBUG] [VALIDATOR ] THG: All expected trading days are present. [2025-10-20T21:27:49.615886] [DEBUG] [VALIDATOR ] THG: Primary data validation passed. [2025-10-20T21:27:49.616522] [INFO ] [MAIN ] THG: Fetched primary data is valid. [2025-10-20T21:27:49.616562] [DEBUG] [LOCAL ] THG: Saving primary data. [2025-10-20T21:27:49.617241] [DEBUG] [LOCAL ] THG: Trimmed 31425 records to 21672 (keeping 200 days of data) [2025-10-20T21:27:49.747618] [DEBUG] [LOCAL ] THG: Primary data saved (21672 records). [2025-10-20T21:27:49.747767] [INFO ] [MAIN ] THG: Full history saved. [2025-10-20T21:27:49.747851] [INFO ] [MAIN ] Update Progress: 2082/2367 symbols completed. THG took 22.2s. Estimated remaining: 36m 39s. Rate: 7.8 symbols/min [2025-10-20T21:27:49.856535] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-03. [2025-10-20T21:27:49.917818] [INFO ] [MAIN ] THO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:27:49.954020] [DEBUG] [LOCAL ] THO: Loaded 41853 existing records from primary storage. [2025-10-20T21:27:49.954299] [INFO ] [MAIN ] THO: Loaded 41853 existing records. [2025-10-20T21:27:50.131787] [WARN ] [MAIN ] THO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:27:50.132040] [INFO ] [MAIN ] THO: Wiping existing primary data before full refresh. [2025-10-20T21:27:50.132091] [INFO ] [LOCAL ] THO: Starting deletion of primary data. [2025-10-20T21:27:50.132337] [DEBUG] [LOCAL ] THO: Deleted primary data file. [2025-10-20T21:27:50.132377] [INFO ] [MAIN ] THO: Starting full primary refresh. [2025-10-20T21:27:50.408303] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-03 (0.55 seconds). [2025-10-20T21:27:50.484734] [INFO ] [REMOTE ] THO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:27:50.651371] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-04. [2025-10-20T21:27:51.144443] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-04 (0.49 seconds). [2025-10-20T21:27:51.451340] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-06. [2025-10-20T21:27:52.251431] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-05. [2025-10-20T21:27:52.697829] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-05 (0.45 seconds). [2025-10-20T21:27:53.051461] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-06. [2025-10-20T21:27:53.109186] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:27:53.400331] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T21:27:53.851409] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-01. [2025-10-20T21:27:54.651422] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-07. [2025-10-20T21:27:55.433321] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T21:27:55.451326] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-02. [2025-10-20T21:27:56.058951] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T21:27:56.251348] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-08. [2025-10-20T21:27:56.604756] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-02 (1.15 seconds). [2025-10-20T21:27:57.051418] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-05. [2025-10-20T21:27:57.851358] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-07. [2025-10-20T21:27:57.900829] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:27:58.199063] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-07 (0.35 seconds). [2025-10-20T21:27:58.651347] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-03. [2025-10-20T21:27:59.109601] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-05 (2.06 seconds). [2025-10-20T21:27:59.451385] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-08. [2025-10-20T21:27:59.876695] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-08 (0.43 seconds). [2025-10-20T21:28:00.262602] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-09. [2025-10-20T21:28:00.290347] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T21:28:00.723214] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T21:28:01.051404] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-04. [2025-10-20T21:28:01.851359] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-09. [2025-10-20T21:28:02.651410] [INFO ] [REMOTE ] THLV: Fetching primary data for 2025-10. [2025-10-20T21:28:02.875819] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T21:28:03.000570] [INFO ] [REMOTE ] THLV: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T21:28:03.001416] [INFO ] [REMOTE ] THLV: Full fetch complete (2551 records). [2025-10-20T21:28:03.001487] [DEBUG] [VALIDATOR ] THLV: Validating primary data. [2025-10-20T21:28:03.360570] [DEBUG] [VALIDATOR ] THLV: All expected trading days are present. [2025-10-20T21:28:03.360671] [DEBUG] [VALIDATOR ] THLV: Primary data validation passed. [2025-10-20T21:28:03.360787] [INFO ] [MAIN ] THLV: Fetched primary data is valid. [2025-10-20T21:28:03.360826] [DEBUG] [LOCAL ] THLV: Saving primary data. [2025-10-20T21:28:03.361159] [DEBUG] [LOCAL ] THLV: Trimmed 2551 records to 1505 (keeping 200 days of data) [2025-10-20T21:28:03.369983] [DEBUG] [LOCAL ] THLV: Primary data saved (1505 records). [2025-10-20T21:28:03.370068] [INFO ] [MAIN ] THLV: Full history saved. [2025-10-20T21:28:03.370137] [INFO ] [MAIN ] Update Progress: 2083/2367 symbols completed. THLV took 17.6s. Estimated remaining: 36m 32s. Rate: 7.8 symbols/min [2025-10-20T21:28:03.451727] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-05. [2025-10-20T21:28:03.550890] [INFO ] [MAIN ] THQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:03.579002] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T21:28:03.607620] [DEBUG] [LOCAL ] THQ: Loaded 25643 existing records from primary storage. [2025-10-20T21:28:03.607923] [INFO ] [MAIN ] THQ: Loaded 25643 existing records. [2025-10-20T21:28:03.789385] [WARN ] [MAIN ] THQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:03.789661] [INFO ] [MAIN ] THQ: Wiping existing primary data before full refresh. [2025-10-20T21:28:03.789733] [INFO ] [LOCAL ] THQ: Starting deletion of primary data. [2025-10-20T21:28:03.789950] [DEBUG] [LOCAL ] THQ: Deleted primary data file. [2025-10-20T21:28:03.789989] [INFO ] [MAIN ] THQ: Starting full primary refresh. [2025-10-20T21:28:04.144869] [INFO ] [REMOTE ] THQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:04.251369] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-01. [2025-10-20T21:28:05.051384] [INFO ] [REMOTE ] THC: Fetching primary data for 2025-10. [2025-10-20T21:28:05.164624] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T21:28:05.519078] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T21:28:05.851456] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-02. [2025-10-20T21:28:06.564961] [INFO ] [REMOTE ] THC: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T21:28:06.578634] [INFO ] [REMOTE ] THC: Full fetch complete (68791 records). [2025-10-20T21:28:06.579169] [DEBUG] [VALIDATOR ] THC: Validating primary data. [2025-10-20T21:28:06.655123] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-06. [2025-10-20T21:28:06.908837] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-02 (1.06 seconds). [2025-10-20T21:28:06.989863] [DEBUG] [VALIDATOR ] THC: All expected trading days are present. [2025-10-20T21:28:06.989964] [DEBUG] [VALIDATOR ] THC: Primary data validation passed. [2025-10-20T21:28:06.991015] [INFO ] [MAIN ] THC: Fetched primary data is valid. [2025-10-20T21:28:06.991076] [DEBUG] [LOCAL ] THC: Saving primary data. [2025-10-20T21:28:06.992398] [DEBUG] [LOCAL ] THC: Trimmed 68791 records to 46820 (keeping 200 days of data) [2025-10-20T21:28:07.265902] [DEBUG] [LOCAL ] THC: Primary data saved (46820 records). [2025-10-20T21:28:07.266034] [INFO ] [MAIN ] THC: Full history saved. [2025-10-20T21:28:07.266113] [INFO ] [MAIN ] Update Progress: 2084/2367 symbols completed. THC took 39.7s. Estimated remaining: 36m 24s. Rate: 7.8 symbols/min [2025-10-20T21:28:07.436112] [INFO ] [MAIN ] THR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:07.455151] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-06. [2025-10-20T21:28:07.455067] [DEBUG] [LOCAL ] THR: Loaded 20074 existing records from primary storage. [2025-10-20T21:28:07.456435] [INFO ] [MAIN ] THR: Loaded 20074 existing records. [2025-10-20T21:28:07.634703] [WARN ] [MAIN ] THR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:07.634955] [INFO ] [MAIN ] THR: Wiping existing primary data before full refresh. [2025-10-20T21:28:07.635002] [INFO ] [LOCAL ] THR: Starting deletion of primary data. [2025-10-20T21:28:07.635173] [DEBUG] [LOCAL ] THR: Deleted primary data file. [2025-10-20T21:28:07.635245] [INFO ] [MAIN ] THR: Starting full primary refresh. [2025-10-20T21:28:07.987952] [INFO ] [REMOTE ] THR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:08.251422] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-01. [2025-10-20T21:28:08.404136] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T21:28:09.051509] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-07. [2025-10-20T21:28:09.220937] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T21:28:09.689108] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-06 (2.23 seconds). [2025-10-20T21:28:09.851430] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-07. [2025-10-20T21:28:10.551511] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T21:28:10.651389] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-08. [2025-10-20T21:28:11.451514] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-02. [2025-10-20T21:28:12.073865] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T21:28:12.174869] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-07 (2.32 seconds). [2025-10-20T21:28:12.200679] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-02 (0.75 seconds). [2025-10-20T21:28:12.251334] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-03. [2025-10-20T21:28:13.051454] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-09. [2025-10-20T21:28:13.399754] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:28:13.851410] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-08. [2025-10-20T21:28:14.603815] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:28:14.651339] [INFO ] [REMOTE ] THO: Fetching primary data for 2025-10. [2025-10-20T21:28:15.451428] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-04. [2025-10-20T21:28:16.015912] [INFO ] [REMOTE ] THO: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T21:28:16.023872] [INFO ] [REMOTE ] THO: Full fetch complete (60130 records). [2025-10-20T21:28:16.024039] [DEBUG] [VALIDATOR ] THO: Validating primary data. [2025-10-20T21:28:16.259799] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-03. [2025-10-20T21:28:16.367670] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T21:28:16.470269] [DEBUG] [VALIDATOR ] THO: All expected trading days are present. [2025-10-20T21:28:16.470372] [DEBUG] [VALIDATOR ] THO: Primary data validation passed. [2025-10-20T21:28:16.471428] [INFO ] [MAIN ] THO: Fetched primary data is valid. [2025-10-20T21:28:16.476714] [DEBUG] [LOCAL ] THO: Saving primary data. [2025-10-20T21:28:16.487246] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-04 (1.04 seconds). [2025-10-20T21:28:16.492731] [DEBUG] [LOCAL ] THO: Trimmed 60130 records to 41224 (keeping 200 days of data) [2025-10-20T21:28:16.736382] [DEBUG] [LOCAL ] THO: Primary data saved (41224 records). [2025-10-20T21:28:16.736512] [INFO ] [MAIN ] THO: Full history saved. [2025-10-20T21:28:16.736584] [INFO ] [MAIN ] Update Progress: 2085/2367 symbols completed. THO took 27.0s. Estimated remaining: 36m 17s. Rate: 7.8 symbols/min [2025-10-20T21:28:16.902473] [INFO ] [MAIN ] THS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:16.938896] [DEBUG] [LOCAL ] THS: Loaded 39102 existing records from primary storage. [2025-10-20T21:28:16.939197] [INFO ] [MAIN ] THS: Loaded 39102 existing records. [2025-10-20T21:28:17.056440] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-05. [2025-10-20T21:28:17.124230] [WARN ] [MAIN ] THS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:17.124544] [INFO ] [MAIN ] THS: Wiping existing primary data before full refresh. [2025-10-20T21:28:17.124597] [INFO ] [LOCAL ] THS: Starting deletion of primary data. [2025-10-20T21:28:17.124822] [DEBUG] [LOCAL ] THS: Deleted primary data file. [2025-10-20T21:28:17.124877] [INFO ] [MAIN ] THS: Starting full primary refresh. [2025-10-20T21:28:17.450014] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-03 (1.19 seconds). [2025-10-20T21:28:17.492140] [INFO ] [REMOTE ] THS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:17.851362] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-01. [2025-10-20T21:28:18.207713] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-05 (1.15 seconds). [2025-10-20T21:28:18.651441] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-09. [2025-10-20T21:28:19.260555] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T21:28:19.451403] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-06. [2025-10-20T21:28:20.251351] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-02. [2025-10-20T21:28:20.453687] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-06 (1.00 seconds). [2025-10-20T21:28:20.994568] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-09 (2.34 seconds). [2025-10-20T21:28:21.051357] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-07. [2025-10-20T21:28:21.602572] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T21:28:21.664831] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T21:28:21.851365] [INFO ] [REMOTE ] TGT: Fetching primary data for 2025-10. [2025-10-20T21:28:22.651454] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-03. [2025-10-20T21:28:23.451377] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-04. [2025-10-20T21:28:23.997325] [INFO ] [REMOTE ] TGT: Incremental fetch complete for 2025-10 (2.15 seconds). [2025-10-20T21:28:24.006628] [INFO ] [REMOTE ] TGT: Full fetch complete (77955 records). [2025-10-20T21:28:24.006845] [DEBUG] [VALIDATOR ] TGT: Validating primary data. [2025-10-20T21:28:24.163946] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:28:24.256466] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-08. [2025-10-20T21:28:24.470968] [DEBUG] [VALIDATOR ] TGT: All expected trading days are present. [2025-10-20T21:28:24.472965] [DEBUG] [VALIDATOR ] TGT: Primary data validation passed. [2025-10-20T21:28:24.472724] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T21:28:24.474597] [INFO ] [MAIN ] TGT: Fetched primary data is valid. [2025-10-20T21:28:24.474748] [DEBUG] [LOCAL ] TGT: Saving primary data. [2025-10-20T21:28:24.481237] [DEBUG] [LOCAL ] TGT: Trimmed 77955 records to 53777 (keeping 200 days of data) [2025-10-20T21:28:24.805445] [DEBUG] [LOCAL ] TGT: Primary data saved (53777 records). [2025-10-20T21:28:24.805573] [INFO ] [MAIN ] TGT: Full history saved. [2025-10-20T21:28:24.805673] [INFO ] [MAIN ] Update Progress: 2086/2367 symbols completed. TGT took 57.3s. Estimated remaining: 36m 9s. Rate: 7.8 symbols/min [2025-10-20T21:28:24.973225] [INFO ] [MAIN ] THW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:24.993961] [DEBUG] [LOCAL ] THW: Loaded 19941 existing records from primary storage. [2025-10-20T21:28:24.994267] [INFO ] [MAIN ] THW: Loaded 19941 existing records. [2025-10-20T21:28:25.061760] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-05. [2025-10-20T21:28:25.190351] [WARN ] [MAIN ] THW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:25.190547] [INFO ] [MAIN ] THW: Wiping existing primary data before full refresh. [2025-10-20T21:28:25.190637] [INFO ] [LOCAL ] THW: Starting deletion of primary data. [2025-10-20T21:28:25.190879] [DEBUG] [LOCAL ] THW: Deleted primary data file. [2025-10-20T21:28:25.190933] [INFO ] [MAIN ] THW: Starting full primary refresh. [2025-10-20T21:28:25.558204] [INFO ] [REMOTE ] THW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:25.559586] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:28:25.851354] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-09. [2025-10-20T21:28:25.931998] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T21:28:26.651354] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-04. [2025-10-20T21:28:26.953740] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T21:28:27.451376] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-06. [2025-10-20T21:28:28.101598] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T21:28:28.251339] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-05. [2025-10-20T21:28:28.473794] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T21:28:29.051408] [INFO ] [REMOTE ] THR: Fetching primary data for 2025-10. [2025-10-20T21:28:29.709954] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T21:28:29.851350] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-07. [2025-10-20T21:28:29.916742] [INFO ] [REMOTE ] THR: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T21:28:29.919634] [INFO ] [REMOTE ] THR: Full fetch complete (28823 records). [2025-10-20T21:28:29.919741] [DEBUG] [VALIDATOR ] THR: Validating primary data. [2025-10-20T21:28:30.280475] [DEBUG] [VALIDATOR ] THR: All expected trading days are present. [2025-10-20T21:28:30.280554] [DEBUG] [VALIDATOR ] THR: Primary data validation passed. [2025-10-20T21:28:30.281153] [INFO ] [MAIN ] THR: Fetched primary data is valid. [2025-10-20T21:28:30.281217] [DEBUG] [LOCAL ] THR: Saving primary data. [2025-10-20T21:28:30.281983] [DEBUG] [LOCAL ] THR: Trimmed 28823 records to 19630 (keeping 200 days of data) [2025-10-20T21:28:30.393808] [DEBUG] [LOCAL ] THR: Primary data saved (19630 records). [2025-10-20T21:28:30.393942] [INFO ] [MAIN ] THR: Full history saved. [2025-10-20T21:28:30.394019] [INFO ] [MAIN ] Update Progress: 2087/2367 symbols completed. THR took 23.1s. Estimated remaining: 36m 1s. Rate: 7.8 symbols/min [2025-10-20T21:28:30.564272] [INFO ] [MAIN ] TIC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:30.564464] [DEBUG] [LOCAL ] TIC: No existing data file found in primary storage. [2025-10-20T21:28:30.564501] [WARN ] [MAIN ] TIC: No existing data found, performing full refresh instead. [2025-10-20T21:28:30.564638] [INFO ] [MAIN ] TIC: Wiping existing primary data before full refresh. [2025-10-20T21:28:30.564727] [INFO ] [LOCAL ] TIC: Starting deletion of primary data. [2025-10-20T21:28:30.564788] [DEBUG] [LOCAL ] TIC: No primary data file found to delete. [2025-10-20T21:28:30.564822] [INFO ] [MAIN ] TIC: Starting full primary refresh. [2025-10-20T21:28:30.661116] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-01. [2025-10-20T21:28:30.929200] [INFO ] [REMOTE ] TIC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:31.299297] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T21:28:31.451370] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-08. [2025-10-20T21:28:31.866903] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T21:28:32.251440] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-06. [2025-10-20T21:28:32.878282] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T21:28:33.051398] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-02. [2025-10-20T21:28:33.202484] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T21:28:33.851365] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-07. [2025-10-20T21:28:34.051562] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T21:28:34.651457] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-09. [2025-10-20T21:28:35.374280] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:28:35.451408] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-08. [2025-10-20T21:28:35.854279] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-09 (1.20 seconds). [2025-10-20T21:28:36.251459] [INFO ] [REMOTE ] THQ: Fetching primary data for 2025-10. [2025-10-20T21:28:36.935870] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T21:28:37.051321] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-09. [2025-10-20T21:28:37.359835] [INFO ] [REMOTE ] THQ: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:28:37.363488] [INFO ] [REMOTE ] THQ: Full fetch complete (34892 records). [2025-10-20T21:28:37.363559] [DEBUG] [VALIDATOR ] THQ: Validating primary data. [2025-10-20T21:28:37.722754] [DEBUG] [VALIDATOR ] THQ: All expected trading days are present. [2025-10-20T21:28:37.722864] [DEBUG] [VALIDATOR ] THQ: Primary data validation passed. [2025-10-20T21:28:37.723417] [INFO ] [MAIN ] THQ: Fetched primary data is valid. [2025-10-20T21:28:37.723455] [DEBUG] [LOCAL ] THQ: Saving primary data. [2025-10-20T21:28:37.724233] [DEBUG] [LOCAL ] THQ: Trimmed 34892 records to 25401 (keeping 200 days of data) [2025-10-20T21:28:37.855537] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-03. [2025-10-20T21:28:37.871995] [DEBUG] [LOCAL ] THQ: Primary data saved (25401 records). [2025-10-20T21:28:37.872062] [INFO ] [MAIN ] THQ: Full history saved. [2025-10-20T21:28:37.872173] [INFO ] [MAIN ] Update Progress: 2088/2367 symbols completed. THQ took 34.5s. Estimated remaining: 35m 53s. Rate: 7.8 symbols/min [2025-10-20T21:28:38.038008] [INFO ] [MAIN ] TIMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:38.073110] [DEBUG] [LOCAL ] TIMB: Loaded 35736 existing records from primary storage. [2025-10-20T21:28:38.073361] [INFO ] [MAIN ] TIMB: Loaded 35736 existing records. [2025-10-20T21:28:38.246362] [WARN ] [MAIN ] TIMB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:38.246526] [INFO ] [MAIN ] TIMB: Wiping existing primary data before full refresh. [2025-10-20T21:28:38.246572] [INFO ] [LOCAL ] TIMB: Starting deletion of primary data. [2025-10-20T21:28:38.246764] [DEBUG] [LOCAL ] TIMB: Deleted primary data file. [2025-10-20T21:28:38.246818] [INFO ] [MAIN ] TIMB: Starting full primary refresh. [2025-10-20T21:28:38.602871] [INFO ] [REMOTE ] TIMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:38.625242] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-09 (1.57 seconds). [2025-10-20T21:28:38.651329] [INFO ] [REMOTE ] THS: Fetching primary data for 2025-10. [2025-10-20T21:28:38.880250] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T21:28:39.451452] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-01. [2025-10-20T21:28:40.064703] [INFO ] [REMOTE ] THS: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T21:28:40.070406] [INFO ] [REMOTE ] THS: Full fetch complete (54512 records). [2025-10-20T21:28:40.070494] [DEBUG] [VALIDATOR ] THS: Validating primary data. [2025-10-20T21:28:40.256603] [INFO ] [REMOTE ] TIC: Fetching primary data for 2025-01. [2025-10-20T21:28:40.446740] [DEBUG] [VALIDATOR ] THS: All expected trading days are present. [2025-10-20T21:28:40.446814] [DEBUG] [VALIDATOR ] THS: Primary data validation passed. [2025-10-20T21:28:40.447725] [INFO ] [MAIN ] THS: Fetched primary data is valid. [2025-10-20T21:28:40.447786] [DEBUG] [LOCAL ] THS: Saving primary data. [2025-10-20T21:28:40.448845] [DEBUG] [LOCAL ] THS: Trimmed 54512 records to 38682 (keeping 200 days of data) [2025-10-20T21:28:40.602422] [DEBUG] [REMOTE ] TIC: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:28:40.611408] [WARN ] [REMOTE ] TIC: No data returned for month 2025-01 [2025-10-20T21:28:40.640676] [FATAL] [REMOTE ] TIC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:28:40.650707] [FATAL] [MAIN ] TIC: Failed to fetch primary data. [2025-10-20T21:28:40.657702] [WARN ] [MAIN ] TIC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:28:40.666504] [DEBUG] [LOCAL ] THS: Primary data saved (38682 records). [2025-10-20T21:28:40.666714] [INFO ] [MAIN ] THS: Full history saved. [2025-10-20T21:28:40.663204] [INFO ] [LOCAL ] TIC: Starting deletion of primary data. [2025-10-20T21:28:40.666982] [INFO ] [MAIN ] Update Progress: 2089/2367 symbols completed. THS took 23.9s. Estimated remaining: 35m 45s. Rate: 7.8 symbols/min [2025-10-20T21:28:40.688214] [DEBUG] [LOCAL ] TIC: No primary data file found to delete. [2025-10-20T21:28:40.688522] [INFO ] [LOCAL ] TIC: Starting deletion of derivative data. [2025-10-20T21:28:40.699196] [DEBUG] [LOCAL ] TIC: No derivative data file found to delete. [2025-10-20T21:28:40.731596] [INFO ] [MAIN ] Update Progress: 2090/2367 symbols completed. TIC took 10.3s. Estimated remaining: 35m 36s. Rate: 7.8 symbols/min [2025-10-20T21:28:41.029316] [INFO ] [MAIN ] TIXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:41.050639] [INFO ] [MAIN ] TISI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:28:41.051421] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-04. [2025-10-20T21:28:41.081061] [DEBUG] [LOCAL ] TISI: Loaded 3366 existing records from primary storage. [2025-10-20T21:28:41.082628] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:28:41.091078] [INFO ] [MAIN ] TISI: Loaded 3366 existing records. [2025-10-20T21:28:41.284858] [DEBUG] [LOCAL ] TIXT: Loaded 32881 existing records from primary storage. [2025-10-20T21:28:41.290175] [WARN ] [MAIN ] TISI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:41.290393] [INFO ] [MAIN ] TISI: Wiping existing primary data before full refresh. [2025-10-20T21:28:41.290460] [INFO ] [LOCAL ] TISI: Starting deletion of primary data. [2025-10-20T21:28:41.290692] [INFO ] [MAIN ] TIXT: Loaded 32881 existing records. [2025-10-20T21:28:41.332856] [DEBUG] [LOCAL ] TISI: Deleted primary data file. [2025-10-20T21:28:41.360036] [INFO ] [MAIN ] TISI: Starting full primary refresh. [2025-10-20T21:28:41.604576] [WARN ] [MAIN ] TIXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:28:41.631238] [INFO ] [MAIN ] TIXT: Wiping existing primary data before full refresh. [2025-10-20T21:28:41.637571] [INFO ] [LOCAL ] TIXT: Starting deletion of primary data. [2025-10-20T21:28:41.678169] [DEBUG] [LOCAL ] TIXT: Deleted primary data file. [2025-10-20T21:28:41.743687] [INFO ] [MAIN ] TIXT: Starting full primary refresh. [2025-10-20T21:28:41.863528] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-02. [2025-10-20T21:28:42.012609] [INFO ] [REMOTE ] TISI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:42.281555] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-04 (1.23 seconds). [2025-10-20T21:28:42.298399] [INFO ] [REMOTE ] TIXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:28:42.651386] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-05. [2025-10-20T21:28:43.227630] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T21:28:43.451444] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-03. [2025-10-20T21:28:43.676314] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T21:28:44.251343] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-01. [2025-10-20T21:28:44.988708] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T21:28:45.051332] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-04. [2025-10-20T21:28:45.650761] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T21:28:45.851336] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-02. [2025-10-20T21:28:46.590289] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T21:28:46.651338] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-05. [2025-10-20T21:28:47.028111] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T21:28:47.451419] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-01. [2025-10-20T21:28:47.988758] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-01 (0.54 seconds). [2025-10-20T21:28:48.049636] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T21:28:48.251357] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-02. [2025-10-20T21:28:48.725808] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T21:28:49.051352] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-06. [2025-10-20T21:28:49.851418] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-03. [2025-10-20T21:28:50.539292] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T21:28:50.604274] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-03 (0.75 seconds). [2025-10-20T21:28:50.651332] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-04. [2025-10-20T21:28:51.451412] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-06. [2025-10-20T21:28:51.727622] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T21:28:52.185278] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:28:52.251351] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-07. [2025-10-20T21:28:53.051346] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-07. [2025-10-20T21:28:53.364765] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-07 (1.11 seconds). [2025-10-20T21:28:53.851398] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-03. [2025-10-20T21:28:54.307752] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T21:28:54.462561] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-03 (0.61 seconds). [2025-10-20T21:28:54.651382] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-05. [2025-10-20T21:28:55.451443] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-04. [2025-10-20T21:28:55.756329] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T21:28:56.228571] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-04 (0.78 seconds). [2025-10-20T21:28:56.251331] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-05. [2025-10-20T21:28:56.821349] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T21:28:57.051366] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-08. [2025-10-20T21:28:57.851498] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-06. [2025-10-20T21:28:58.456449] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T21:28:58.651367] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-08. [2025-10-20T21:28:59.209338] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-06 (1.36 seconds). [2025-10-20T21:28:59.399899] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T21:28:59.451393] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-09. [2025-10-20T21:29:00.251364] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-07. [2025-10-20T21:29:00.439933] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T21:29:01.051436] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-06. [2025-10-20T21:29:01.687565] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T21:29:01.689883] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:29:01.851435] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-09. [2025-10-20T21:29:02.651385] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-08. [2025-10-20T21:29:03.183142] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T21:29:03.451421] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-07. [2025-10-20T21:29:03.906328] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T21:29:04.063877] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-08 (1.41 seconds). [2025-10-20T21:29:04.251403] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-08. [2025-10-20T21:29:04.737758] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-08 (0.49 seconds). [2025-10-20T21:29:05.051368] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-09. [2025-10-20T21:29:05.851535] [INFO ] [REMOTE ] TIMB: Fetching primary data for 2025-10. [2025-10-20T21:29:06.490127] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T21:29:06.651361] [INFO ] [REMOTE ] TIXT: Fetching primary data for 2025-10. [2025-10-20T21:29:07.030435] [INFO ] [REMOTE ] TIMB: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T21:29:07.034987] [INFO ] [REMOTE ] TIMB: Full fetch complete (53292 records). [2025-10-20T21:29:07.035129] [DEBUG] [VALIDATOR ] TIMB: Validating primary data. [2025-10-20T21:29:07.441402] [DEBUG] [VALIDATOR ] TIMB: All expected trading days are present. [2025-10-20T21:29:07.442143] [DEBUG] [VALIDATOR ] TIMB: Primary data validation passed. [2025-10-20T21:29:07.444227] [INFO ] [MAIN ] TIMB: Fetched primary data is valid. [2025-10-20T21:29:07.444293] [DEBUG] [LOCAL ] TIMB: Saving primary data. [2025-10-20T21:29:07.445741] [DEBUG] [LOCAL ] TIMB: Trimmed 53292 records to 35027 (keeping 200 days of data) [2025-10-20T21:29:07.500185] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-09. [2025-10-20T21:29:07.708958] [INFO ] [REMOTE ] TIXT: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:29:07.725670] [DEBUG] [LOCAL ] TIMB: Primary data saved (35027 records). [2025-10-20T21:29:07.726157] [INFO ] [MAIN ] TIMB: Full history saved. [2025-10-20T21:29:07.725873] [INFO ] [REMOTE ] TIXT: Full fetch complete (46149 records). [2025-10-20T21:29:07.726908] [INFO ] [MAIN ] Update Progress: 2091/2367 symbols completed. TIMB took 29.9s. Estimated remaining: 35m 31s. Rate: 7.8 symbols/min [2025-10-20T21:29:07.727362] [DEBUG] [VALIDATOR ] TIXT: Validating primary data. [2025-10-20T21:29:08.095331] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T21:29:08.085034] [INFO ] [MAIN ] TJX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:08.256700] [INFO ] [REMOTE ] TISI: Fetching primary data for 2025-10. [2025-10-20T21:29:08.338409] [DEBUG] [VALIDATOR ] TIXT: All expected trading days are present. [2025-10-20T21:29:08.338502] [DEBUG] [VALIDATOR ] TIXT: Primary data validation passed. [2025-10-20T21:29:08.339520] [INFO ] [MAIN ] TIXT: Fetched primary data is valid. [2025-10-20T21:29:08.339833] [DEBUG] [LOCAL ] TIXT: Saving primary data. [2025-10-20T21:29:08.341102] [DEBUG] [LOCAL ] TIXT: Trimmed 46149 records to 32585 (keeping 200 days of data) [2025-10-20T21:29:08.532037] [DEBUG] [LOCAL ] TJX: Loaded 54444 existing records from primary storage. [2025-10-20T21:29:08.533511] [INFO ] [MAIN ] TJX: Loaded 54444 existing records. [2025-10-20T21:29:08.678605] [DEBUG] [LOCAL ] TIXT: Primary data saved (32585 records). [2025-10-20T21:29:08.696301] [INFO ] [MAIN ] TIXT: Full history saved. [2025-10-20T21:29:08.714499] [INFO ] [MAIN ] Update Progress: 2092/2367 symbols completed. TIXT took 28.0s. Estimated remaining: 35m 22s. Rate: 7.8 symbols/min [2025-10-20T21:29:08.799783] [WARN ] [MAIN ] TJX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:08.809134] [INFO ] [MAIN ] TJX: Wiping existing primary data before full refresh. [2025-10-20T21:29:08.800021] [INFO ] [REMOTE ] TISI: Incremental fetch complete for 2025-10 (0.54 seconds). [2025-10-20T21:29:08.809252] [INFO ] [LOCAL ] TJX: Starting deletion of primary data. [2025-10-20T21:29:08.828001] [INFO ] [REMOTE ] TISI: Full fetch complete (4829 records). [2025-10-20T21:29:08.838912] [DEBUG] [VALIDATOR ] TISI: Validating primary data. [2025-10-20T21:29:08.846354] [DEBUG] [LOCAL ] TJX: Deleted primary data file. [2025-10-20T21:29:08.868433] [INFO ] [MAIN ] TJX: Starting full primary refresh. [2025-10-20T21:29:09.061550] [INFO ] [REMOTE ] THW: Fetching primary data for 2025-10. [2025-10-20T21:29:09.140591] [INFO ] [MAIN ] TK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:09.567764] [DEBUG] [LOCAL ] TK: Loaded 39692 existing records from primary storage. [2025-10-20T21:29:09.618032] [INFO ] [MAIN ] TK: Loaded 39692 existing records. [2025-10-20T21:29:09.729660] [INFO ] [REMOTE ] TJX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:09.821434] [DEBUG] [VALIDATOR ] TISI: All expected trading days are present. [2025-10-20T21:29:09.829496] [DEBUG] [VALIDATOR ] TISI: Primary data validation passed. [2025-10-20T21:29:09.832078] [INFO ] [MAIN ] TISI: Fetched primary data is valid. [2025-10-20T21:29:09.832453] [DEBUG] [LOCAL ] TISI: Saving primary data. [2025-10-20T21:29:09.854061] [DEBUG] [LOCAL ] TISI: Trimmed 4829 records to 3305 (keeping 200 days of data) [2025-10-20T21:29:09.853920] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-01. [2025-10-20T21:29:09.914054] [WARN ] [MAIN ] TK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:09.929336] [INFO ] [MAIN ] TK: Wiping existing primary data before full refresh. [2025-10-20T21:29:09.930483] [INFO ] [LOCAL ] TK: Starting deletion of primary data. [2025-10-20T21:29:09.934825] [DEBUG] [LOCAL ] TK: Deleted primary data file. [2025-10-20T21:29:09.935764] [DEBUG] [LOCAL ] TISI: Primary data saved (3305 records). [2025-10-20T21:29:09.935825] [INFO ] [MAIN ] TK: Starting full primary refresh. [2025-10-20T21:29:09.941088] [INFO ] [MAIN ] TISI: Full history saved. [2025-10-20T21:29:09.980283] [INFO ] [MAIN ] Update Progress: 2093/2367 symbols completed. TISI took 29.3s. Estimated remaining: 35m 14s. Rate: 7.8 symbols/min [2025-10-20T21:29:10.373455] [INFO ] [REMOTE ] THW: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T21:29:10.422078] [INFO ] [MAIN ] TKC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:10.443635] [INFO ] [REMOTE ] THW: Full fetch complete (28800 records). [2025-10-20T21:29:10.498708] [DEBUG] [VALIDATOR ] THW: Validating primary data. [2025-10-20T21:29:10.503844] [INFO ] [REMOTE ] TK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:10.654633] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-01. [2025-10-20T21:29:10.811599] [DEBUG] [LOCAL ] TKC: Loaded 40157 existing records from primary storage. [2025-10-20T21:29:10.822524] [INFO ] [MAIN ] TKC: Loaded 40157 existing records. [2025-10-20T21:29:11.037989] [DEBUG] [VALIDATOR ] THW: All expected trading days are present. [2025-10-20T21:29:11.043197] [DEBUG] [VALIDATOR ] THW: Primary data validation passed. [2025-10-20T21:29:11.059018] [INFO ] [MAIN ] THW: Fetched primary data is valid. [2025-10-20T21:29:11.059098] [DEBUG] [LOCAL ] THW: Saving primary data. [2025-10-20T21:29:11.101626] [WARN ] [MAIN ] TKC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:11.101955] [DEBUG] [LOCAL ] THW: Trimmed 28800 records to 19667 (keeping 200 days of data) [2025-10-20T21:29:11.102126] [INFO ] [MAIN ] TKC: Wiping existing primary data before full refresh. [2025-10-20T21:29:11.102447] [INFO ] [LOCAL ] TKC: Starting deletion of primary data. [2025-10-20T21:29:11.102971] [DEBUG] [LOCAL ] TKC: Deleted primary data file. [2025-10-20T21:29:11.103024] [INFO ] [MAIN ] TKC: Starting full primary refresh. [2025-10-20T21:29:11.526799] [DEBUG] [LOCAL ] THW: Primary data saved (19667 records). [2025-10-20T21:29:11.545618] [INFO ] [MAIN ] THW: Full history saved. [2025-10-20T21:29:11.545945] [INFO ] [MAIN ] Update Progress: 2094/2367 symbols completed. THW took 46.7s. Estimated remaining: 35m 5s. Rate: 7.8 symbols/min [2025-10-20T21:29:11.714759] [INFO ] [REMOTE ] TKC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:11.724891] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-01. [2025-10-20T21:29:11.820236] [INFO ] [MAIN ] TKO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:11.864929] [DEBUG] [LOCAL ] TKO: Loaded 44357 existing records from primary storage. [2025-10-20T21:29:11.877145] [INFO ] [MAIN ] TKO: Loaded 44357 existing records. [2025-10-20T21:29:11.998176] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-01 (2.14 seconds). [2025-10-20T21:29:12.088925] [WARN ] [MAIN ] TKO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:12.089172] [INFO ] [MAIN ] TKO: Wiping existing primary data before full refresh. [2025-10-20T21:29:12.089392] [INFO ] [LOCAL ] TKO: Starting deletion of primary data. [2025-10-20T21:29:12.089802] [DEBUG] [LOCAL ] TKO: Deleted primary data file. [2025-10-20T21:29:12.089915] [INFO ] [MAIN ] TKO: Starting full primary refresh. [2025-10-20T21:29:12.259533] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-02. [2025-10-20T21:29:12.249214] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T21:29:12.475918] [INFO ] [REMOTE ] TKO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:12.781470] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-01 (1.06 seconds). [2025-10-20T21:29:13.051400] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-02. [2025-10-20T21:29:13.851381] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-01. [2025-10-20T21:29:14.089478] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T21:29:14.417143] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T21:29:14.651400] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-03. [2025-10-20T21:29:15.451423] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-02. [2025-10-20T21:29:15.502360] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-01 (1.65 seconds). [2025-10-20T21:29:16.251410] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-02. [2025-10-20T21:29:16.704618] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T21:29:16.763049] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:29:17.051423] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-04. [2025-10-20T21:29:17.851448] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-03. [2025-10-20T21:29:17.986898] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T21:29:18.651415] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-03. [2025-10-20T21:29:19.150566] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-04 (2.10 seconds). [2025-10-20T21:29:19.353959] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T21:29:19.451332] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-04. [2025-10-20T21:29:20.103826] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T21:29:20.251395] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-03. [2025-10-20T21:29:20.922969] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T21:29:21.051347] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-05. [2025-10-20T21:29:21.851364] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-05. [2025-10-20T21:29:21.921441] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T21:29:22.651356] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-04. [2025-10-20T21:29:23.200362] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T21:29:23.313298] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T21:29:23.451370] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-06. [2025-10-20T21:29:24.251358] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-06. [2025-10-20T21:29:24.333527] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T21:29:25.051375] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-04. [2025-10-20T21:29:25.323516] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:29:25.854322] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-07. [2025-10-20T21:29:25.865502] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T21:29:26.296110] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T21:29:26.651457] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-05. [2025-10-20T21:29:27.451411] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-05. [2025-10-20T21:29:28.024010] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-07 (2.17 seconds). [2025-10-20T21:29:28.258297] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-08. [2025-10-20T21:29:28.277114] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:29:28.960882] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T21:29:29.051444] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-06. [2025-10-20T21:29:29.851371] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-07. [2025-10-20T21:29:30.305778] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-08 (2.05 seconds). [2025-10-20T21:29:30.652218] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-09. [2025-10-20T21:29:30.654816] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:29:30.951314] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-07 (1.10 seconds). [2025-10-20T21:29:31.451449] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-06. [2025-10-20T21:29:32.251356] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-07. [2025-10-20T21:29:32.474841] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T21:29:33.037609] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T21:29:33.051365] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-07. [2025-10-20T21:29:33.690991] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:29:33.851397] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-08. [2025-10-20T21:29:34.641630] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T21:29:34.651327] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-08. [2025-10-20T21:29:34.791025] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T21:29:35.451364] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-08. [2025-10-20T21:29:35.779029] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-08 (1.13 seconds). [2025-10-20T21:29:36.251379] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-09. [2025-10-20T21:29:37.057230] [INFO ] [REMOTE ] TJX: Fetching primary data for 2025-10. [2025-10-20T21:29:37.062712] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:29:37.858444] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-09. [2025-10-20T21:29:37.863584] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T21:29:38.651370] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-09. [2025-10-20T21:29:38.941525] [INFO ] [REMOTE ] TJX: Incremental fetch complete for 2025-10 (1.88 seconds). [2025-10-20T21:29:38.950963] [INFO ] [REMOTE ] TJX: Full fetch complete (77833 records). [2025-10-20T21:29:38.951167] [DEBUG] [VALIDATOR ] TJX: Validating primary data. [2025-10-20T21:29:39.351288] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-09 (0.70 seconds). [2025-10-20T21:29:39.401219] [DEBUG] [VALIDATOR ] TJX: All expected trading days are present. [2025-10-20T21:29:39.401775] [DEBUG] [VALIDATOR ] TJX: Primary data validation passed. [2025-10-20T21:29:39.403274] [INFO ] [MAIN ] TJX: Fetched primary data is valid. [2025-10-20T21:29:39.403372] [DEBUG] [LOCAL ] TJX: Saving primary data. [2025-10-20T21:29:39.451399] [INFO ] [REMOTE ] TKC: Fetching primary data for 2025-10. [2025-10-20T21:29:39.453036] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T21:29:39.454565] [DEBUG] [LOCAL ] TJX: Trimmed 77833 records to 53664 (keeping 200 days of data) [2025-10-20T21:29:39.794407] [DEBUG] [LOCAL ] TJX: Primary data saved (53664 records). [2025-10-20T21:29:39.794566] [INFO ] [MAIN ] TJX: Full history saved. [2025-10-20T21:29:39.794669] [INFO ] [MAIN ] Update Progress: 2095/2367 symbols completed. TJX took 32.1s. Estimated remaining: 35m 0s. Rate: 7.8 symbols/min [2025-10-20T21:29:39.963902] [INFO ] [MAIN ] TKR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:40.001177] [DEBUG] [LOCAL ] TKR: Loaded 40207 existing records from primary storage. [2025-10-20T21:29:40.001511] [INFO ] [MAIN ] TKR: Loaded 40207 existing records. [2025-10-20T21:29:40.185920] [WARN ] [MAIN ] TKR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:40.186174] [INFO ] [MAIN ] TKR: Wiping existing primary data before full refresh. [2025-10-20T21:29:40.186223] [INFO ] [LOCAL ] TKR: Starting deletion of primary data. [2025-10-20T21:29:40.186466] [DEBUG] [LOCAL ] TKR: Deleted primary data file. [2025-10-20T21:29:40.186540] [INFO ] [MAIN ] TKR: Starting full primary refresh. [2025-10-20T21:29:40.256512] [INFO ] [REMOTE ] TK: Fetching primary data for 2025-10. [2025-10-20T21:29:40.551083] [INFO ] [REMOTE ] TKR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:40.949896] [INFO ] [REMOTE ] TKC: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T21:29:40.956714] [INFO ] [REMOTE ] TKC: Full fetch complete (54302 records). [2025-10-20T21:29:40.956810] [DEBUG] [VALIDATOR ] TKC: Validating primary data. [2025-10-20T21:29:41.054542] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-01. [2025-10-20T21:29:41.337948] [DEBUG] [VALIDATOR ] TKC: All expected trading days are present. [2025-10-20T21:29:41.338044] [DEBUG] [VALIDATOR ] TKC: Primary data validation passed. [2025-10-20T21:29:41.338992] [INFO ] [MAIN ] TKC: Fetched primary data is valid. [2025-10-20T21:29:41.339096] [DEBUG] [LOCAL ] TKC: Saving primary data. [2025-10-20T21:29:41.340219] [DEBUG] [LOCAL ] TKC: Trimmed 54302 records to 39869 (keeping 200 days of data) [2025-10-20T21:29:41.566784] [DEBUG] [LOCAL ] TKC: Primary data saved (39869 records). [2025-10-20T21:29:41.566998] [INFO ] [MAIN ] TKC: Full history saved. [2025-10-20T21:29:41.572316] [INFO ] [MAIN ] Update Progress: 2096/2367 symbols completed. TKC took 31.6s. Estimated remaining: 34m 52s. Rate: 7.8 symbols/min [2025-10-20T21:29:41.625788] [INFO ] [REMOTE ] TK: Incremental fetch complete for 2025-10 (1.37 seconds). [2025-10-20T21:29:41.687494] [INFO ] [REMOTE ] TK: Full fetch complete (56335 records). [2025-10-20T21:29:41.690184] [DEBUG] [VALIDATOR ] TK: Validating primary data. [2025-10-20T21:29:41.866476] [INFO ] [MAIN ] TLK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:41.874446] [INFO ] [REMOTE ] TKO: Fetching primary data for 2025-10. [2025-10-20T21:29:42.174008] [DEBUG] [LOCAL ] TLK: Loaded 39277 existing records from primary storage. [2025-10-20T21:29:42.209546] [DEBUG] [VALIDATOR ] TK: All expected trading days are present. [2025-10-20T21:29:42.209744] [INFO ] [MAIN ] TLK: Loaded 39277 existing records. [2025-10-20T21:29:42.253168] [DEBUG] [VALIDATOR ] TK: Primary data validation passed. [2025-10-20T21:29:42.286377] [INFO ] [MAIN ] TK: Fetched primary data is valid. [2025-10-20T21:29:42.291727] [DEBUG] [LOCAL ] TK: Saving primary data. [2025-10-20T21:29:42.397137] [DEBUG] [LOCAL ] TK: Trimmed 56335 records to 39202 (keeping 200 days of data) [2025-10-20T21:29:42.396936] [WARN ] [MAIN ] TLK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:42.397544] [INFO ] [MAIN ] TLK: Wiping existing primary data before full refresh. [2025-10-20T21:29:42.397841] [INFO ] [LOCAL ] TLK: Starting deletion of primary data. [2025-10-20T21:29:42.398764] [DEBUG] [LOCAL ] TLK: Deleted primary data file. [2025-10-20T21:29:42.398822] [INFO ] [MAIN ] TLK: Starting full primary refresh. [2025-10-20T21:29:42.803557] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:29:42.855173] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-02. [2025-10-20T21:29:42.899504] [INFO ] [REMOTE ] TLK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:43.031135] [DEBUG] [LOCAL ] TK: Primary data saved (39202 records). [2025-10-20T21:29:43.031435] [INFO ] [MAIN ] TK: Full history saved. [2025-10-20T21:29:43.031587] [INFO ] [MAIN ] Update Progress: 2097/2367 symbols completed. TK took 34.3s. Estimated remaining: 34m 43s. Rate: 7.8 symbols/min [2025-10-20T21:29:43.200085] [INFO ] [MAIN ] TLYS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:43.218603] [INFO ] [REMOTE ] TKO: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:29:43.225770] [INFO ] [REMOTE ] TKO: Full fetch complete (65488 records). [2025-10-20T21:29:43.226010] [DEBUG] [VALIDATOR ] TKO: Validating primary data. [2025-10-20T21:29:43.349482] [DEBUG] [LOCAL ] TLYS: Loaded 16688 existing records from primary storage. [2025-10-20T21:29:43.370704] [INFO ] [MAIN ] TLYS: Loaded 16688 existing records. [2025-10-20T21:29:43.454860] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-01. [2025-10-20T21:29:43.770295] [WARN ] [MAIN ] TLYS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:43.813126] [INFO ] [MAIN ] TLYS: Wiping existing primary data before full refresh. [2025-10-20T21:29:43.814257] [INFO ] [LOCAL ] TLYS: Starting deletion of primary data. [2025-10-20T21:29:43.847467] [DEBUG] [LOCAL ] TLYS: Deleted primary data file. [2025-10-20T21:29:43.851050] [DEBUG] [VALIDATOR ] TKO: All expected trading days are present. [2025-10-20T21:29:43.850939] [INFO ] [MAIN ] TLYS: Starting full primary refresh. [2025-10-20T21:29:43.856310] [DEBUG] [VALIDATOR ] TKO: Primary data validation passed. [2025-10-20T21:29:43.884896] [INFO ] [MAIN ] TKO: Fetched primary data is valid. [2025-10-20T21:29:43.909775] [DEBUG] [LOCAL ] TKO: Saving primary data. [2025-10-20T21:29:43.961354] [DEBUG] [LOCAL ] TKO: Trimmed 65488 records to 43606 (keeping 200 days of data) [2025-10-20T21:29:44.413225] [INFO ] [REMOTE ] TLYS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:44.422994] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-01. [2025-10-20T21:29:44.480050] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T21:29:44.536603] [DEBUG] [LOCAL ] TKO: Primary data saved (43606 records). [2025-10-20T21:29:44.536769] [INFO ] [MAIN ] TKO: Full history saved. [2025-10-20T21:29:44.536853] [INFO ] [MAIN ] Update Progress: 2098/2367 symbols completed. TKO took 33.0s. Estimated remaining: 34m 35s. Rate: 7.8 symbols/min [2025-10-20T21:29:44.708365] [INFO ] [MAIN ] TM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:29:44.744066] [DEBUG] [LOCAL ] TM: Loaded 36853 existing records from primary storage. [2025-10-20T21:29:44.744465] [INFO ] [MAIN ] TM: Loaded 36853 existing records. [2025-10-20T21:29:44.940427] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T21:29:44.945264] [WARN ] [MAIN ] TM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:29:44.945480] [INFO ] [MAIN ] TM: Wiping existing primary data before full refresh. [2025-10-20T21:29:44.945546] [INFO ] [LOCAL ] TM: Starting deletion of primary data. [2025-10-20T21:29:44.953271] [DEBUG] [LOCAL ] TM: Deleted primary data file. [2025-10-20T21:29:44.953331] [INFO ] [MAIN ] TM: Starting full primary refresh. [2025-10-20T21:29:45.052611] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-02. [2025-10-20T21:29:45.101957] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T21:29:45.319613] [INFO ] [REMOTE ] TM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:29:45.851363] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-03. [2025-10-20T21:29:46.397437] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-02 (1.34 seconds). [2025-10-20T21:29:46.651366] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-02. [2025-10-20T21:29:47.075118] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-03 (1.22 seconds). [2025-10-20T21:29:47.386836] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-02 (0.74 seconds). [2025-10-20T21:29:47.451375] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-03. [2025-10-20T21:29:48.251434] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-04. [2025-10-20T21:29:49.051352] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-01. [2025-10-20T21:29:49.182769] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T21:29:49.851357] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-03. [2025-10-20T21:29:49.889752] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:29:50.651365] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-04. [2025-10-20T21:29:50.744387] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T21:29:50.799612] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-03 (0.95 seconds). [2025-10-20T21:29:51.451449] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-05. [2025-10-20T21:29:52.141351] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T21:29:52.251433] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-05. [2025-10-20T21:29:53.051453] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-04. [2025-10-20T21:29:53.096499] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T21:29:53.835591] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-05 (1.58 seconds). [2025-10-20T21:29:53.852179] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-06. [2025-10-20T21:29:53.855417] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T21:29:54.651477] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-06. [2025-10-20T21:29:55.236616] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-06 (1.38 seconds). [2025-10-20T21:29:55.451436] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-07. [2025-10-20T21:29:55.876601] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T21:29:56.251376] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-07. [2025-10-20T21:29:56.890951] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:29:57.051414] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-08. [2025-10-20T21:29:57.770761] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:29:57.851429] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-08. [2025-10-20T21:29:58.502870] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T21:29:58.651373] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-05. [2025-10-20T21:29:58.945935] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-08 (1.09 seconds). [2025-10-20T21:29:59.451349] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-02. [2025-10-20T21:29:59.761595] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-05 (1.11 seconds). [2025-10-20T21:30:00.251390] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-09. [2025-10-20T21:30:01.051415] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-06. [2025-10-20T21:30:01.354460] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-02 (1.90 seconds). [2025-10-20T21:30:01.652089] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:30:01.851466] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-09. [2025-10-20T21:30:02.421676] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T21:30:02.651376] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-07. [2025-10-20T21:30:03.400821] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:30:03.451342] [INFO ] [REMOTE ] TKR: Fetching primary data for 2025-10. [2025-10-20T21:30:03.781336] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-07 (1.13 seconds). [2025-10-20T21:30:04.251376] [INFO ] [REMOTE ] TLK: Fetching primary data for 2025-10. [2025-10-20T21:30:04.556803] [INFO ] [REMOTE ] TKR: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:30:04.564627] [INFO ] [REMOTE ] TKR: Full fetch complete (57997 records). [2025-10-20T21:30:04.564849] [DEBUG] [VALIDATOR ] TKR: Validating primary data. [2025-10-20T21:30:04.952754] [DEBUG] [VALIDATOR ] TKR: All expected trading days are present. [2025-10-20T21:30:04.952838] [DEBUG] [VALIDATOR ] TKR: Primary data validation passed. [2025-10-20T21:30:04.953742] [INFO ] [MAIN ] TKR: Fetched primary data is valid. [2025-10-20T21:30:04.953803] [DEBUG] [LOCAL ] TKR: Saving primary data. [2025-10-20T21:30:04.954903] [DEBUG] [LOCAL ] TKR: Trimmed 57997 records to 39466 (keeping 200 days of data) [2025-10-20T21:30:05.056673] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-03. [2025-10-20T21:30:05.187802] [DEBUG] [LOCAL ] TKR: Primary data saved (39466 records). [2025-10-20T21:30:05.187963] [INFO ] [MAIN ] TKR: Full history saved. [2025-10-20T21:30:05.188241] [INFO ] [MAIN ] Update Progress: 2099/2367 symbols completed. TKR took 25.4s. Estimated remaining: 34m 29s. Rate: 7.8 symbols/min [2025-10-20T21:30:05.354052] [INFO ] [MAIN ] TME: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:05.407068] [DEBUG] [LOCAL ] TME: Loaded 53970 existing records from primary storage. [2025-10-20T21:30:05.407407] [INFO ] [MAIN ] TME: Loaded 53970 existing records. [2025-10-20T21:30:05.612221] [WARN ] [MAIN ] TME: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:05.612548] [INFO ] [MAIN ] TME: Wiping existing primary data before full refresh. [2025-10-20T21:30:05.612607] [INFO ] [LOCAL ] TME: Starting deletion of primary data. [2025-10-20T21:30:05.612815] [DEBUG] [LOCAL ] TME: Deleted primary data file. [2025-10-20T21:30:05.612868] [INFO ] [MAIN ] TME: Starting full primary refresh. [2025-10-20T21:30:05.741395] [INFO ] [REMOTE ] TLK: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T21:30:05.787727] [INFO ] [REMOTE ] TLK: Full fetch complete (57539 records). [2025-10-20T21:30:05.811881] [DEBUG] [VALIDATOR ] TLK: Validating primary data. [2025-10-20T21:30:05.877603] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-08. [2025-10-20T21:30:06.209280] [INFO ] [REMOTE ] TME: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:06.409636] [DEBUG] [VALIDATOR ] TLK: All expected trading days are present. [2025-10-20T21:30:06.409751] [DEBUG] [VALIDATOR ] TLK: Primary data validation passed. [2025-10-20T21:30:06.410711] [INFO ] [MAIN ] TLK: Fetched primary data is valid. [2025-10-20T21:30:06.410772] [DEBUG] [LOCAL ] TLK: Saving primary data. [2025-10-20T21:30:06.411847] [DEBUG] [LOCAL ] TLK: Trimmed 57539 records to 38763 (keeping 200 days of data) [2025-10-20T21:30:06.640331] [DEBUG] [LOCAL ] TLK: Primary data saved (38763 records). [2025-10-20T21:30:06.640473] [INFO ] [MAIN ] TLK: Full history saved. [2025-10-20T21:30:06.640552] [INFO ] [MAIN ] Update Progress: 2100/2367 symbols completed. TLK took 25.1s. Estimated remaining: 34m 20s. Rate: 7.8 symbols/min [2025-10-20T21:30:06.655465] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-01. [2025-10-20T21:30:06.767075] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T21:30:06.829273] [INFO ] [MAIN ] TMHC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:06.850042] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T21:30:06.892304] [DEBUG] [LOCAL ] TMHC: Loaded 46217 existing records from primary storage. [2025-10-20T21:30:06.892609] [INFO ] [MAIN ] TMHC: Loaded 46217 existing records. [2025-10-20T21:30:07.064203] [WARN ] [MAIN ] TMHC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:07.064441] [INFO ] [MAIN ] TMHC: Wiping existing primary data before full refresh. [2025-10-20T21:30:07.064488] [INFO ] [LOCAL ] TMHC: Starting deletion of primary data. [2025-10-20T21:30:07.064809] [DEBUG] [LOCAL ] TMHC: Deleted primary data file. [2025-10-20T21:30:07.064873] [INFO ] [MAIN ] TMHC: Starting full primary refresh. [2025-10-20T21:30:07.412936] [INFO ] [REMOTE ] TMHC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:07.451370] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-01. [2025-10-20T21:30:08.251348] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-09. [2025-10-20T21:30:08.671617] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T21:30:09.051422] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-04. [2025-10-20T21:30:09.120530] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:30:09.223304] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T21:30:09.851385] [INFO ] [REMOTE ] TLYS: Fetching primary data for 2025-10. [2025-10-20T21:30:10.651389] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-02. [2025-10-20T21:30:10.783490] [INFO ] [REMOTE ] TLYS: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T21:30:10.785256] [INFO ] [REMOTE ] TLYS: Full fetch complete (21110 records). [2025-10-20T21:30:10.785340] [DEBUG] [VALIDATOR ] TLYS: Validating primary data. [2025-10-20T21:30:11.163204] [DEBUG] [VALIDATOR ] TLYS: All expected trading days are present. [2025-10-20T21:30:11.163371] [DEBUG] [VALIDATOR ] TLYS: Primary data validation passed. [2025-10-20T21:30:11.163864] [INFO ] [MAIN ] TLYS: Fetched primary data is valid. [2025-10-20T21:30:11.163940] [DEBUG] [LOCAL ] TLYS: Saving primary data. [2025-10-20T21:30:11.172762] [DEBUG] [LOCAL ] TLYS: Trimmed 21110 records to 16437 (keeping 200 days of data) [2025-10-20T21:30:11.188813] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T21:30:11.288690] [DEBUG] [LOCAL ] TLYS: Primary data saved (16437 records). [2025-10-20T21:30:11.288899] [INFO ] [MAIN ] TLYS: Full history saved. [2025-10-20T21:30:11.288985] [INFO ] [MAIN ] Update Progress: 2101/2367 symbols completed. TLYS took 28.3s. Estimated remaining: 34m 12s. Rate: 7.8 symbols/min [2025-10-20T21:30:11.459779] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-05. [2025-10-20T21:30:11.498916] [INFO ] [MAIN ] TMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:11.547089] [DEBUG] [LOCAL ] TMO: Loaded 53193 existing records from primary storage. [2025-10-20T21:30:11.547448] [INFO ] [MAIN ] TMO: Loaded 53193 existing records. [2025-10-20T21:30:11.731953] [WARN ] [MAIN ] TMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:11.732184] [INFO ] [MAIN ] TMO: Wiping existing primary data before full refresh. [2025-10-20T21:30:11.732230] [INFO ] [LOCAL ] TMO: Starting deletion of primary data. [2025-10-20T21:30:11.732462] [DEBUG] [LOCAL ] TMO: Deleted primary data file. [2025-10-20T21:30:11.732518] [INFO ] [MAIN ] TMO: Starting full primary refresh. [2025-10-20T21:30:12.082702] [INFO ] [REMOTE ] TMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:12.251378] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-01. [2025-10-20T21:30:12.641806] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T21:30:12.722848] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-05 (1.26 seconds). [2025-10-20T21:30:13.051422] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-06. [2025-10-20T21:30:13.851396] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-02. [2025-10-20T21:30:14.027560] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T21:30:14.651456] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-03. [2025-10-20T21:30:14.693971] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T21:30:15.311238] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T21:30:15.451351] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-03. [2025-10-20T21:30:16.251415] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-02. [2025-10-20T21:30:16.794782] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T21:30:17.051344] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-04. [2025-10-20T21:30:17.236296] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:30:17.851458] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-07. [2025-10-20T21:30:18.074186] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T21:30:18.651413] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-04. [2025-10-20T21:30:18.948214] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-04 (1.90 seconds). [2025-10-20T21:30:19.458103] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-05. [2025-10-20T21:30:19.462521] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:30:19.960862] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T21:30:20.251391] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-08. [2025-10-20T21:30:21.051383] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-03. [2025-10-20T21:30:21.446008] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-05 (1.99 seconds). [2025-10-20T21:30:21.813521] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T21:30:21.851395] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-05. [2025-10-20T21:30:22.651350] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-06. [2025-10-20T21:30:22.689821] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T21:30:23.367690] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:30:23.451398] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-09. [2025-10-20T21:30:24.251394] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-04. [2025-10-20T21:30:24.524602] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:30:24.870741] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T21:30:25.051396] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-06. [2025-10-20T21:30:25.851396] [INFO ] [REMOTE ] TM: Fetching primary data for 2025-10. [2025-10-20T21:30:26.388958] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-04 (2.14 seconds). [2025-10-20T21:30:26.595272] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:30:26.651412] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-07. [2025-10-20T21:30:27.251475] [INFO ] [REMOTE ] TM: Incremental fetch complete for 2025-10 (1.40 seconds). [2025-10-20T21:30:27.256389] [INFO ] [REMOTE ] TM: Full fetch complete (52141 records). [2025-10-20T21:30:27.256549] [DEBUG] [VALIDATOR ] TM: Validating primary data. [2025-10-20T21:30:27.456472] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-05. [2025-10-20T21:30:27.660833] [DEBUG] [VALIDATOR ] TM: All expected trading days are present. [2025-10-20T21:30:27.660927] [DEBUG] [VALIDATOR ] TM: Primary data validation passed. [2025-10-20T21:30:27.661935] [INFO ] [MAIN ] TM: Fetched primary data is valid. [2025-10-20T21:30:27.662001] [DEBUG] [LOCAL ] TM: Saving primary data. [2025-10-20T21:30:27.663231] [DEBUG] [LOCAL ] TM: Trimmed 52141 records to 36215 (keeping 200 days of data) [2025-10-20T21:30:27.884927] [DEBUG] [LOCAL ] TM: Primary data saved (36215 records). [2025-10-20T21:30:27.885099] [INFO ] [MAIN ] TM: Full history saved. [2025-10-20T21:30:27.885183] [INFO ] [MAIN ] Update Progress: 2102/2367 symbols completed. TM took 43.3s. Estimated remaining: 34m 5s. Rate: 7.8 symbols/min [2025-10-20T21:30:28.054828] [INFO ] [MAIN ] TNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:28.070398] [DEBUG] [LOCAL ] TNC: Loaded 14601 existing records from primary storage. [2025-10-20T21:30:28.070638] [INFO ] [MAIN ] TNC: Loaded 14601 existing records. [2025-10-20T21:30:28.259466] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-07. [2025-10-20T21:30:28.265335] [WARN ] [MAIN ] TNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:28.267230] [INFO ] [MAIN ] TNC: Wiping existing primary data before full refresh. [2025-10-20T21:30:28.267356] [INFO ] [LOCAL ] TNC: Starting deletion of primary data. [2025-10-20T21:30:28.267735] [DEBUG] [LOCAL ] TNC: Deleted primary data file. [2025-10-20T21:30:28.267916] [INFO ] [MAIN ] TNC: Starting full primary refresh. [2025-10-20T21:30:28.563732] [INFO ] [REMOTE ] TMHC: Incremental fetch complete for 2025-07 (1.91 seconds). [2025-10-20T21:30:28.686792] [INFO ] [REMOTE ] TNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:29.051460] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-01. [2025-10-20T21:30:29.696484] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-05 (2.24 seconds). [2025-10-20T21:30:29.851366] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-06. [2025-10-20T21:30:30.066931] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-01 (1.02 seconds). [2025-10-20T21:30:30.100828] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T21:30:30.651363] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-02. [2025-10-20T21:30:31.451382] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-08. [2025-10-20T21:30:31.737287] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T21:30:31.773940] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T21:30:32.251395] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-07. [2025-10-20T21:30:33.051364] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-03. [2025-10-20T21:30:33.851362] [INFO ] [REMOTE ] TMHC: Fetching primary data for 2025-08. [2025-10-20T21:30:33.956308] [DEBUG] [REMOTE ] TMHC: Error fetching primary data for 2025-08: 'API call returned an empty response.' [2025-10-20T21:30:33.956545] [WARN ] [REMOTE ] TMHC: No data returned for month 2025-08 [2025-10-20T21:30:33.956632] [FATAL] [REMOTE ] TMHC: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:30:33.956761] [FATAL] [MAIN ] TMHC: Failed to fetch primary data. [2025-10-20T21:30:33.956826] [WARN ] [MAIN ] TMHC: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:30:33.956896] [INFO ] [LOCAL ] TMHC: Starting deletion of primary data. [2025-10-20T21:30:33.956956] [DEBUG] [LOCAL ] TMHC: No primary data file found to delete. [2025-10-20T21:30:33.956991] [INFO ] [LOCAL ] TMHC: Starting deletion of derivative data. [2025-10-20T21:30:33.957182] [DEBUG] [LOCAL ] TMHC: Deleted derivative data file. [2025-10-20T21:30:33.957264] [INFO ] [MAIN ] Update Progress: 2103/2367 symbols completed. TMHC took 27.3s. Estimated remaining: 33m 57s. Rate: 7.8 symbols/min [2025-10-20T21:30:34.129263] [INFO ] [MAIN ] TNET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:34.161147] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-08 (2.71 seconds). [2025-10-20T21:30:34.197829] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:30:34.215247] [DEBUG] [LOCAL ] TNET: Loaded 28661 existing records from primary storage. [2025-10-20T21:30:34.215546] [INFO ] [MAIN ] TNET: Loaded 28661 existing records. [2025-10-20T21:30:34.389803] [WARN ] [MAIN ] TNET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:34.390016] [INFO ] [MAIN ] TNET: Wiping existing primary data before full refresh. [2025-10-20T21:30:34.390062] [INFO ] [LOCAL ] TNET: Starting deletion of primary data. [2025-10-20T21:30:34.390279] [DEBUG] [LOCAL ] TNET: Deleted primary data file. [2025-10-20T21:30:34.390333] [INFO ] [MAIN ] TNET: Starting full primary refresh. [2025-10-20T21:30:34.656222] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-09. [2025-10-20T21:30:34.748077] [INFO ] [REMOTE ] TNET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:34.781411] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-07 (2.53 seconds). [2025-10-20T21:30:35.451378] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-08. [2025-10-20T21:30:36.251356] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-04. [2025-10-20T21:30:36.586537] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T21:30:37.051386] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-01. [2025-10-20T21:30:37.353036] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-08 (1.90 seconds). [2025-10-20T21:30:37.410906] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-04 (1.16 seconds). [2025-10-20T21:30:37.851376] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-05. [2025-10-20T21:30:38.223077] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:30:38.609724] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-05 (0.76 seconds). [2025-10-20T21:30:38.651388] [INFO ] [REMOTE ] TME: Fetching primary data for 2025-10. [2025-10-20T21:30:39.451523] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-06. [2025-10-20T21:30:40.251369] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-09. [2025-10-20T21:30:40.456636] [INFO ] [REMOTE ] TME: Incremental fetch complete for 2025-10 (1.81 seconds). [2025-10-20T21:30:40.457607] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-06 (1.01 seconds). [2025-10-20T21:30:40.469005] [INFO ] [REMOTE ] TME: Full fetch complete (77166 records). [2025-10-20T21:30:40.469152] [DEBUG] [VALIDATOR ] TME: Validating primary data. [2025-10-20T21:30:40.876070] [DEBUG] [VALIDATOR ] TME: All expected trading days are present. [2025-10-20T21:30:40.876187] [DEBUG] [VALIDATOR ] TME: Primary data validation passed. [2025-10-20T21:30:40.877416] [INFO ] [MAIN ] TME: Fetched primary data is valid. [2025-10-20T21:30:40.877482] [DEBUG] [LOCAL ] TME: Saving primary data. [2025-10-20T21:30:40.879110] [DEBUG] [LOCAL ] TME: Trimmed 77166 records to 53210 (keeping 200 days of data) [2025-10-20T21:30:41.053899] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-07. [2025-10-20T21:30:41.197058] [DEBUG] [LOCAL ] TME: Primary data saved (53210 records). [2025-10-20T21:30:41.197190] [INFO ] [MAIN ] TME: Full history saved. [2025-10-20T21:30:41.197457] [INFO ] [MAIN ] Update Progress: 2104/2367 symbols completed. TME took 36.0s. Estimated remaining: 33m 50s. Rate: 7.8 symbols/min [2025-10-20T21:30:41.394496] [INFO ] [MAIN ] TNGY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:41.394726] [DEBUG] [LOCAL ] TNGY: No existing data file found in primary storage. [2025-10-20T21:30:41.394777] [WARN ] [MAIN ] TNGY: No existing data found, performing full refresh instead. [2025-10-20T21:30:41.394943] [INFO ] [MAIN ] TNGY: Wiping existing primary data before full refresh. [2025-10-20T21:30:41.394993] [INFO ] [LOCAL ] TNGY: Starting deletion of primary data. [2025-10-20T21:30:41.395035] [DEBUG] [LOCAL ] TNGY: No primary data file found to delete. [2025-10-20T21:30:41.395067] [INFO ] [MAIN ] TNGY: Starting full primary refresh. [2025-10-20T21:30:41.787546] [INFO ] [REMOTE ] TNGY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:41.851349] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-02. [2025-10-20T21:30:41.964765] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T21:30:42.134560] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-09 (1.88 seconds). [2025-10-20T21:30:42.651435] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-08. [2025-10-20T21:30:43.218857] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T21:30:43.451405] [INFO ] [REMOTE ] TMO: Fetching primary data for 2025-10. [2025-10-20T21:30:43.497870] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:30:44.251359] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-09. [2025-10-20T21:30:45.051343] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-03. [2025-10-20T21:30:45.183400] [INFO ] [REMOTE ] TMO: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T21:30:45.191218] [INFO ] [REMOTE ] TMO: Full fetch complete (75696 records). [2025-10-20T21:30:45.191348] [DEBUG] [VALIDATOR ] TMO: Validating primary data. [2025-10-20T21:30:45.280603] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-09 (1.03 seconds). [2025-10-20T21:30:45.608757] [DEBUG] [VALIDATOR ] TMO: All expected trading days are present. [2025-10-20T21:30:45.608841] [DEBUG] [VALIDATOR ] TMO: Primary data validation passed. [2025-10-20T21:30:45.609906] [INFO ] [MAIN ] TMO: Fetched primary data is valid. [2025-10-20T21:30:45.609969] [DEBUG] [LOCAL ] TMO: Saving primary data. [2025-10-20T21:30:45.611405] [DEBUG] [LOCAL ] TMO: Trimmed 75696 records to 52402 (keeping 200 days of data) [2025-10-20T21:30:45.857479] [INFO ] [REMOTE ] TNGY: Fetching primary data for 2025-01. [2025-10-20T21:30:45.935239] [DEBUG] [LOCAL ] TMO: Primary data saved (52402 records). [2025-10-20T21:30:45.935359] [INFO ] [MAIN ] TMO: Full history saved. [2025-10-20T21:30:45.935433] [INFO ] [MAIN ] Update Progress: 2105/2367 symbols completed. TMO took 34.6s. Estimated remaining: 33m 42s. Rate: 7.8 symbols/min [2025-10-20T21:30:46.107385] [INFO ] [MAIN ] TNK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:46.110031] [DEBUG] [REMOTE ] TNGY: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:30:46.110471] [WARN ] [REMOTE ] TNGY: No data returned for month 2025-01 [2025-10-20T21:30:46.110597] [FATAL] [REMOTE ] TNGY: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:30:46.110697] [FATAL] [MAIN ] TNGY: Failed to fetch primary data. [2025-10-20T21:30:46.110757] [WARN ] [MAIN ] TNGY: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:30:46.110836] [INFO ] [LOCAL ] TNGY: Starting deletion of primary data. [2025-10-20T21:30:46.110911] [DEBUG] [LOCAL ] TNGY: No primary data file found to delete. [2025-10-20T21:30:46.110956] [INFO ] [LOCAL ] TNGY: Starting deletion of derivative data. [2025-10-20T21:30:46.111008] [DEBUG] [LOCAL ] TNGY: No derivative data file found to delete. [2025-10-20T21:30:46.111129] [INFO ] [MAIN ] Update Progress: 2106/2367 symbols completed. TNGY took 4.9s. Estimated remaining: 33m 33s. Rate: 7.8 symbols/min [2025-10-20T21:30:46.315137] [INFO ] [MAIN ] TNL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:46.311631] [DEBUG] [LOCAL ] TNK: Loaded 35859 existing records from primary storage. [2025-10-20T21:30:46.315748] [INFO ] [MAIN ] TNK: Loaded 35859 existing records. [2025-10-20T21:30:46.512670] [WARN ] [MAIN ] TNK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:46.513177] [INFO ] [MAIN ] TNK: Wiping existing primary data before full refresh. [2025-10-20T21:30:46.513285] [INFO ] [LOCAL ] TNK: Starting deletion of primary data. [2025-10-20T21:30:46.513553] [DEBUG] [LOCAL ] TNK: Deleted primary data file. [2025-10-20T21:30:46.513766] [INFO ] [MAIN ] TNK: Starting full primary refresh. [2025-10-20T21:30:46.644811] [DEBUG] [LOCAL ] TNL: Loaded 42362 existing records from primary storage. [2025-10-20T21:30:46.654617] [INFO ] [REMOTE ] TNC: Fetching primary data for 2025-10. [2025-10-20T21:30:46.658951] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T21:30:46.659120] [INFO ] [MAIN ] TNL: Loaded 42362 existing records. [2025-10-20T21:30:47.089149] [WARN ] [MAIN ] TNL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:47.125944] [INFO ] [MAIN ] TNL: Wiping existing primary data before full refresh. [2025-10-20T21:30:47.163143] [INFO ] [REMOTE ] TNK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:47.163283] [INFO ] [LOCAL ] TNL: Starting deletion of primary data. [2025-10-20T21:30:47.164110] [DEBUG] [LOCAL ] TNL: Deleted primary data file. [2025-10-20T21:30:47.164191] [INFO ] [MAIN ] TNL: Starting full primary refresh. [2025-10-20T21:30:47.457755] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-01. [2025-10-20T21:30:47.489739] [INFO ] [REMOTE ] TNC: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T21:30:47.523999] [INFO ] [REMOTE ] TNC: Full fetch complete (22302 records). [2025-10-20T21:30:47.552963] [DEBUG] [VALIDATOR ] TNC: Validating primary data. [2025-10-20T21:30:47.557100] [INFO ] [REMOTE ] TNL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:47.943087] [DEBUG] [VALIDATOR ] TNC: All expected trading days are present. [2025-10-20T21:30:47.943179] [DEBUG] [VALIDATOR ] TNC: Primary data validation passed. [2025-10-20T21:30:47.943636] [INFO ] [MAIN ] TNC: Fetched primary data is valid. [2025-10-20T21:30:47.943723] [DEBUG] [LOCAL ] TNC: Saving primary data. [2025-10-20T21:30:47.944511] [DEBUG] [LOCAL ] TNC: Trimmed 22302 records to 14395 (keeping 200 days of data) [2025-10-20T21:30:48.031629] [DEBUG] [LOCAL ] TNC: Primary data saved (14395 records). [2025-10-20T21:30:48.031805] [INFO ] [MAIN ] TNC: Full history saved. [2025-10-20T21:30:48.031883] [INFO ] [MAIN ] Update Progress: 2107/2367 symbols completed. TNC took 20.1s. Estimated remaining: 33m 24s. Rate: 7.8 symbols/min [2025-10-20T21:30:48.204144] [INFO ] [MAIN ] TOL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:30:48.251806] [DEBUG] [LOCAL ] TOL: Loaded 51370 existing records from primary storage. [2025-10-20T21:30:48.252016] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-01. [2025-10-20T21:30:48.252447] [INFO ] [MAIN ] TOL: Loaded 51370 existing records. [2025-10-20T21:30:48.430141] [WARN ] [MAIN ] TOL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:30:48.430350] [INFO ] [MAIN ] TOL: Wiping existing primary data before full refresh. [2025-10-20T21:30:48.430423] [INFO ] [LOCAL ] TOL: Starting deletion of primary data. [2025-10-20T21:30:48.430857] [DEBUG] [LOCAL ] TOL: Deleted primary data file. [2025-10-20T21:30:48.430943] [INFO ] [MAIN ] TOL: Starting full primary refresh. [2025-10-20T21:30:48.785051] [INFO ] [REMOTE ] TOL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:30:49.056468] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-04. [2025-10-20T21:30:49.066274] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:30:49.801988] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:30:49.851335] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-02. [2025-10-20T21:30:50.459213] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-04 (1.40 seconds). [2025-10-20T21:30:50.651396] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-05. [2025-10-20T21:30:51.318156] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:30:51.451348] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-02. [2025-10-20T21:30:51.868481] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T21:30:52.251387] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-06. [2025-10-20T21:30:52.844027] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T21:30:53.051429] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-03. [2025-10-20T21:30:53.355783] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T21:30:53.851392] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-01. [2025-10-20T21:30:54.442560] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T21:30:54.651428] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-07. [2025-10-20T21:30:55.451454] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-03. [2025-10-20T21:30:55.634634] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T21:30:55.956886] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T21:30:56.251464] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-02. [2025-10-20T21:30:56.963376] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:30:57.051354] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-04. [2025-10-20T21:30:57.851367] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-04. [2025-10-20T21:30:57.959505] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T21:30:58.257879] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T21:30:58.651384] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-05. [2025-10-20T21:30:59.414422] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T21:30:59.451354] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-08. [2025-10-20T21:31:00.256478] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-03. [2025-10-20T21:31:00.277612] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:31:01.051387] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-06. [2025-10-20T21:31:01.385315] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-08 (1.93 seconds). [2025-10-20T21:31:01.851423] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-09. [2025-10-20T21:31:02.649101] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:31:02.657127] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-05. [2025-10-20T21:31:02.762629] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-03 (2.51 seconds). [2025-10-20T21:31:03.398280] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:31:03.451342] [INFO ] [REMOTE ] TNET: Fetching primary data for 2025-10. [2025-10-20T21:31:04.208708] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T21:31:04.228613] [INFO ] [REMOTE ] TNET: Incremental fetch complete for 2025-10 (0.78 seconds). [2025-10-20T21:31:04.232929] [INFO ] [REMOTE ] TNET: Full fetch complete (41812 records). [2025-10-20T21:31:04.233071] [DEBUG] [VALIDATOR ] TNET: Validating primary data. [2025-10-20T21:31:04.256510] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-06. [2025-10-20T21:31:04.631131] [DEBUG] [VALIDATOR ] TNET: All expected trading days are present. [2025-10-20T21:31:04.631258] [DEBUG] [VALIDATOR ] TNET: Primary data validation passed. [2025-10-20T21:31:04.632090] [INFO ] [MAIN ] TNET: Fetched primary data is valid. [2025-10-20T21:31:04.632152] [DEBUG] [LOCAL ] TNET: Saving primary data. [2025-10-20T21:31:04.633084] [DEBUG] [LOCAL ] TNET: Trimmed 41812 records to 28275 (keeping 200 days of data) [2025-10-20T21:31:04.804192] [DEBUG] [LOCAL ] TNET: Primary data saved (28275 records). [2025-10-20T21:31:04.804332] [INFO ] [MAIN ] TNET: Full history saved. [2025-10-20T21:31:04.804410] [INFO ] [MAIN ] Update Progress: 2108/2367 symbols completed. TNET took 30.8s. Estimated remaining: 33m 18s. Rate: 7.8 symbols/min [2025-10-20T21:31:04.974110] [INFO ] [MAIN ] TOST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:05.021337] [DEBUG] [LOCAL ] TOST: Loaded 54470 existing records from primary storage. [2025-10-20T21:31:05.021699] [INFO ] [MAIN ] TOST: Loaded 54470 existing records. [2025-10-20T21:31:05.056519] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-07. [2025-10-20T21:31:05.196802] [WARN ] [MAIN ] TOST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:05.197004] [INFO ] [MAIN ] TOST: Wiping existing primary data before full refresh. [2025-10-20T21:31:05.197049] [INFO ] [LOCAL ] TOST: Starting deletion of primary data. [2025-10-20T21:31:05.197364] [DEBUG] [LOCAL ] TOST: Deleted primary data file. [2025-10-20T21:31:05.197420] [INFO ] [MAIN ] TOST: Starting full primary refresh. [2025-10-20T21:31:05.551663] [INFO ] [REMOTE ] TOST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:05.739352] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:31:05.851354] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-07. [2025-10-20T21:31:06.651418] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-01. [2025-10-20T21:31:06.721352] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T21:31:07.196427] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-07 (1.35 seconds). [2025-10-20T21:31:07.451439] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-08. [2025-10-20T21:31:08.251347] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-04. [2025-10-20T21:31:08.532476] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-01 (1.88 seconds). [2025-10-20T21:31:09.036587] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T21:31:09.051333] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-09. [2025-10-20T21:31:09.711553] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T21:31:09.851393] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-05. [2025-10-20T21:31:10.468254] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T21:31:10.651432] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-02. [2025-10-20T21:31:11.451416] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-08. [2025-10-20T21:31:11.713241] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T21:31:12.251382] [INFO ] [REMOTE ] TNK: Fetching primary data for 2025-10. [2025-10-20T21:31:12.497342] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:31:12.946311] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T21:31:13.051381] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-09. [2025-10-20T21:31:13.587012] [INFO ] [REMOTE ] TNK: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:31:13.592449] [INFO ] [REMOTE ] TNK: Full fetch complete (51973 records). [2025-10-20T21:31:13.592602] [DEBUG] [VALIDATOR ] TNK: Validating primary data. [2025-10-20T21:31:13.856632] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-03. [2025-10-20T21:31:13.971998] [DEBUG] [VALIDATOR ] TNK: All expected trading days are present. [2025-10-20T21:31:13.972094] [DEBUG] [VALIDATOR ] TNK: Primary data validation passed. [2025-10-20T21:31:13.973021] [INFO ] [MAIN ] TNK: Fetched primary data is valid. [2025-10-20T21:31:13.973089] [DEBUG] [LOCAL ] TNK: Saving primary data. [2025-10-20T21:31:13.974253] [DEBUG] [LOCAL ] TNK: Trimmed 51973 records to 35462 (keeping 200 days of data) [2025-10-20T21:31:14.185590] [DEBUG] [LOCAL ] TNK: Primary data saved (35462 records). [2025-10-20T21:31:14.185763] [INFO ] [MAIN ] TNK: Full history saved. [2025-10-20T21:31:14.185849] [INFO ] [MAIN ] Update Progress: 2109/2367 symbols completed. TNK took 28.3s. Estimated remaining: 33m 10s. Rate: 7.8 symbols/min [2025-10-20T21:31:14.368336] [INFO ] [MAIN ] TPB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:14.393707] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T21:31:14.427281] [DEBUG] [LOCAL ] TPB: Loaded 26442 existing records from primary storage. [2025-10-20T21:31:14.427569] [INFO ] [MAIN ] TPB: Loaded 26442 existing records. [2025-10-20T21:31:14.628803] [WARN ] [MAIN ] TPB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:14.629084] [INFO ] [MAIN ] TPB: Wiping existing primary data before full refresh. [2025-10-20T21:31:14.629133] [INFO ] [LOCAL ] TPB: Starting deletion of primary data. [2025-10-20T21:31:14.629378] [DEBUG] [LOCAL ] TPB: Deleted primary data file. [2025-10-20T21:31:14.629439] [INFO ] [MAIN ] TPB: Starting full primary refresh. [2025-10-20T21:31:14.662419] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-06. [2025-10-20T21:31:15.032371] [INFO ] [REMOTE ] TPB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:15.451369] [INFO ] [REMOTE ] TNL: Fetching primary data for 2025-10. [2025-10-20T21:31:15.685124] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:31:16.251352] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-04. [2025-10-20T21:31:16.323755] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:31:16.675229] [INFO ] [REMOTE ] TNL: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T21:31:16.681278] [INFO ] [REMOTE ] TNL: Full fetch complete (59629 records). [2025-10-20T21:31:16.681393] [DEBUG] [VALIDATOR ] TNL: Validating primary data. [2025-10-20T21:31:17.055738] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-01. [2025-10-20T21:31:17.072109] [DEBUG] [VALIDATOR ] TNL: All expected trading days are present. [2025-10-20T21:31:17.072165] [DEBUG] [VALIDATOR ] TNL: Primary data validation passed. [2025-10-20T21:31:17.073268] [INFO ] [MAIN ] TNL: Fetched primary data is valid. [2025-10-20T21:31:17.073433] [DEBUG] [LOCAL ] TNL: Saving primary data. [2025-10-20T21:31:17.074734] [DEBUG] [LOCAL ] TNL: Trimmed 59629 records to 41798 (keeping 200 days of data) [2025-10-20T21:31:17.330291] [DEBUG] [LOCAL ] TNL: Primary data saved (41798 records). [2025-10-20T21:31:17.330417] [INFO ] [MAIN ] TNL: Full history saved. [2025-10-20T21:31:17.330496] [INFO ] [MAIN ] Update Progress: 2110/2367 symbols completed. TNL took 31.2s. Estimated remaining: 33m 2s. Rate: 7.8 symbols/min [2025-10-20T21:31:17.545305] [INFO ] [MAIN ] TPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:17.587776] [DEBUG] [LOCAL ] TPC: Loaded 39604 existing records from primary storage. [2025-10-20T21:31:17.588129] [INFO ] [MAIN ] TPC: Loaded 39604 existing records. [2025-10-20T21:31:17.783474] [WARN ] [MAIN ] TPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:17.783752] [INFO ] [MAIN ] TPC: Wiping existing primary data before full refresh. [2025-10-20T21:31:17.783964] [INFO ] [LOCAL ] TPC: Starting deletion of primary data. [2025-10-20T21:31:17.784263] [DEBUG] [LOCAL ] TPC: Deleted primary data file. [2025-10-20T21:31:17.784348] [INFO ] [MAIN ] TPC: Starting full primary refresh. [2025-10-20T21:31:17.854401] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-07. [2025-10-20T21:31:18.126332] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T21:31:18.226807] [INFO ] [REMOTE ] TPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:18.239637] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-01 (1.18 seconds). [2025-10-20T21:31:18.651380] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-01. [2025-10-20T21:31:19.451386] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-02. [2025-10-20T21:31:19.998095] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T21:31:20.072065] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-01 (1.42 seconds). [2025-10-20T21:31:20.251378] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-08. [2025-10-20T21:31:20.672224] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T21:31:21.051396] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-02. [2025-10-20T21:31:21.851367] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-05. [2025-10-20T21:31:21.997722] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T21:31:22.401006] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T21:31:22.651397] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-03. [2025-10-20T21:31:23.451423] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-09. [2025-10-20T21:31:23.764705] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T21:31:23.793079] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-03 (1.14 seconds). [2025-10-20T21:31:24.251462] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-06. [2025-10-20T21:31:25.051351] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-03. [2025-10-20T21:31:25.147754] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T21:31:25.851416] [INFO ] [REMOTE ] TOL: Fetching primary data for 2025-10. [2025-10-20T21:31:25.953367] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T21:31:26.525690] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T21:31:26.651336] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-04. [2025-10-20T21:31:27.451365] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-07. [2025-10-20T21:31:27.552377] [INFO ] [REMOTE ] TOL: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T21:31:27.559884] [INFO ] [REMOTE ] TOL: Full fetch complete (73789 records). [2025-10-20T21:31:27.560038] [DEBUG] [VALIDATOR ] TOL: Validating primary data. [2025-10-20T21:31:28.013234] [DEBUG] [VALIDATOR ] TOL: All expected trading days are present. [2025-10-20T21:31:28.013376] [DEBUG] [VALIDATOR ] TOL: Primary data validation passed. [2025-10-20T21:31:28.014515] [INFO ] [MAIN ] TOL: Fetched primary data is valid. [2025-10-20T21:31:28.014683] [DEBUG] [LOCAL ] TOL: Saving primary data. [2025-10-20T21:31:28.016272] [DEBUG] [LOCAL ] TOL: Trimmed 73789 records to 50629 (keeping 200 days of data) [2025-10-20T21:31:28.258917] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-04. [2025-10-20T21:31:28.292551] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:31:28.389838] [DEBUG] [LOCAL ] TOL: Primary data saved (50629 records). [2025-10-20T21:31:28.389979] [INFO ] [MAIN ] TOL: Full history saved. [2025-10-20T21:31:28.390055] [INFO ] [MAIN ] Update Progress: 2111/2367 symbols completed. TOL took 40.4s. Estimated remaining: 32m 55s. Rate: 7.8 symbols/min [2025-10-20T21:31:28.557039] [INFO ] [MAIN ] TPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:28.598425] [DEBUG] [LOCAL ] TPH: Loaded 45484 existing records from primary storage. [2025-10-20T21:31:28.598788] [INFO ] [MAIN ] TPH: Loaded 45484 existing records. [2025-10-20T21:31:28.777631] [WARN ] [MAIN ] TPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:28.777867] [INFO ] [MAIN ] TPH: Wiping existing primary data before full refresh. [2025-10-20T21:31:28.777945] [INFO ] [LOCAL ] TPH: Starting deletion of primary data. [2025-10-20T21:31:28.778209] [DEBUG] [LOCAL ] TPH: Deleted primary data file. [2025-10-20T21:31:28.778248] [INFO ] [MAIN ] TPH: Starting full primary refresh. [2025-10-20T21:31:29.056471] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-05. [2025-10-20T21:31:29.171761] [INFO ] [REMOTE ] TPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:29.280783] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-04 (1.02 seconds). [2025-10-20T21:31:29.451147] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-07 (2.00 seconds). [2025-10-20T21:31:29.851396] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-05. [2025-10-20T21:31:30.651352] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-08. [2025-10-20T21:31:30.702705] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T21:31:30.763393] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-05 (0.91 seconds). [2025-10-20T21:31:31.451396] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-06. [2025-10-20T21:31:32.251388] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-01. [2025-10-20T21:31:32.552450] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-06 (1.10 seconds). [2025-10-20T21:31:32.724002] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-08 (2.07 seconds). [2025-10-20T21:31:33.051417] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-07. [2025-10-20T21:31:33.721147] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T21:31:33.851336] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-02. [2025-10-20T21:31:34.327247] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T21:31:34.651431] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-08. [2025-10-20T21:31:35.163809] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:31:35.451413] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-03. [2025-10-20T21:31:35.938633] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T21:31:36.251368] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-09. [2025-10-20T21:31:36.744782] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T21:31:37.051375] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-09. [2025-10-20T21:31:37.851399] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-06. [2025-10-20T21:31:38.298872] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T21:31:38.301083] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-09 (2.05 seconds). [2025-10-20T21:31:38.651438] [INFO ] [REMOTE ] TPB: Fetching primary data for 2025-10. [2025-10-20T21:31:39.451436] [INFO ] [REMOTE ] TOST: Fetching primary data for 2025-10. [2025-10-20T21:31:40.136228] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-10-20T21:31:40.245095] [INFO ] [REMOTE ] TPB: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T21:31:40.250139] [INFO ] [REMOTE ] TPB: Full fetch complete (36067 records). [2025-10-20T21:31:40.250310] [DEBUG] [VALIDATOR ] TPB: Validating primary data. [2025-10-20T21:31:40.262059] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-07. [2025-10-20T21:31:40.629861] [DEBUG] [VALIDATOR ] TPB: All expected trading days are present. [2025-10-20T21:31:40.629948] [DEBUG] [VALIDATOR ] TPB: Primary data validation passed. [2025-10-20T21:31:40.630806] [INFO ] [MAIN ] TPB: Fetched primary data is valid. [2025-10-20T21:31:40.630869] [DEBUG] [LOCAL ] TPB: Saving primary data. [2025-10-20T21:31:40.631712] [DEBUG] [LOCAL ] TPB: Trimmed 36067 records to 26151 (keeping 200 days of data) [2025-10-20T21:31:40.787100] [DEBUG] [LOCAL ] TPB: Primary data saved (26151 records). [2025-10-20T21:31:40.787233] [INFO ] [MAIN ] TPB: Full history saved. [2025-10-20T21:31:40.787309] [INFO ] [MAIN ] Update Progress: 2112/2367 symbols completed. TPB took 26.6s. Estimated remaining: 32m 48s. Rate: 7.8 symbols/min [2025-10-20T21:31:40.955812] [INFO ] [MAIN ] TPL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:40.969089] [DEBUG] [LOCAL ] TPL: Loaded 12558 existing records from primary storage. [2025-10-20T21:31:40.969272] [INFO ] [MAIN ] TPL: Loaded 12558 existing records. [2025-10-20T21:31:41.065220] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-04. [2025-10-20T21:31:41.120959] [INFO ] [REMOTE ] TOST: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T21:31:41.180184] [WARN ] [MAIN ] TPL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:41.180519] [INFO ] [MAIN ] TPL: Wiping existing primary data before full refresh. [2025-10-20T21:31:41.180600] [INFO ] [LOCAL ] TPL: Starting deletion of primary data. [2025-10-20T21:31:41.180320] [INFO ] [REMOTE ] TOST: Full fetch complete (77852 records). [2025-10-20T21:31:41.180842] [DEBUG] [VALIDATOR ] TOST: Validating primary data. [2025-10-20T21:31:41.181035] [DEBUG] [LOCAL ] TPL: Deleted primary data file. [2025-10-20T21:31:41.181107] [INFO ] [MAIN ] TPL: Starting full primary refresh. [2025-10-20T21:31:41.942558] [DEBUG] [VALIDATOR ] TOST: All expected trading days are present. [2025-10-20T21:31:41.921033] [INFO ] [REMOTE ] TPL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:41.942976] [DEBUG] [VALIDATOR ] TOST: Primary data validation passed. [2025-10-20T21:31:41.942918] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-01. [2025-10-20T21:31:41.944245] [INFO ] [MAIN ] TOST: Fetched primary data is valid. [2025-10-20T21:31:41.945317] [DEBUG] [LOCAL ] TOST: Saving primary data. [2025-10-20T21:31:41.947171] [DEBUG] [LOCAL ] TOST: Trimmed 77852 records to 53690 (keeping 200 days of data) [2025-10-20T21:31:42.027373] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T21:31:42.286485] [DEBUG] [LOCAL ] TOST: Primary data saved (53690 records). [2025-10-20T21:31:42.286671] [INFO ] [MAIN ] TOST: Full history saved. [2025-10-20T21:31:42.286766] [INFO ] [MAIN ] Update Progress: 2113/2367 symbols completed. TOST took 37.5s. Estimated remaining: 32m 39s. Rate: 7.8 symbols/min [2025-10-20T21:31:42.453702] [INFO ] [MAIN ] TPR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:42.507170] [DEBUG] [LOCAL ] TPR: Loaded 54177 existing records from primary storage. [2025-10-20T21:31:42.507479] [INFO ] [MAIN ] TPR: Loaded 54177 existing records. [2025-10-20T21:31:42.659757] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-08. [2025-10-20T21:31:42.693969] [WARN ] [MAIN ] TPR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:42.694756] [INFO ] [MAIN ] TPR: Wiping existing primary data before full refresh. [2025-10-20T21:31:42.695018] [INFO ] [LOCAL ] TPR: Starting deletion of primary data. [2025-10-20T21:31:42.695619] [DEBUG] [LOCAL ] TPR: Deleted primary data file. [2025-10-20T21:31:42.695929] [INFO ] [MAIN ] TPR: Starting full primary refresh. [2025-10-20T21:31:42.886308] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T21:31:42.891708] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T21:31:43.104713] [INFO ] [REMOTE ] TPR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:43.451433] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-01. [2025-10-20T21:31:44.239569] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T21:31:44.251337] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-09. [2025-10-20T21:31:45.051417] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-05. [2025-10-20T21:31:45.359902] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T21:31:45.782302] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:31:45.851371] [INFO ] [REMOTE ] TPC: Fetching primary data for 2025-10. [2025-10-20T21:31:46.229426] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T21:31:46.651459] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-02. [2025-10-20T21:31:47.168902] [INFO ] [REMOTE ] TPC: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T21:31:47.174677] [INFO ] [REMOTE ] TPC: Full fetch complete (55446 records). [2025-10-20T21:31:47.174836] [DEBUG] [VALIDATOR ] TPC: Validating primary data. [2025-10-20T21:31:47.460575] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-02. [2025-10-20T21:31:47.555332] [DEBUG] [VALIDATOR ] TPC: All expected trading days are present. [2025-10-20T21:31:47.555424] [DEBUG] [VALIDATOR ] TPC: Primary data validation passed. [2025-10-20T21:31:47.556384] [INFO ] [MAIN ] TPC: Fetched primary data is valid. [2025-10-20T21:31:47.556449] [DEBUG] [LOCAL ] TPC: Saving primary data. [2025-10-20T21:31:47.557627] [DEBUG] [LOCAL ] TPC: Trimmed 55446 records to 39183 (keeping 200 days of data) [2025-10-20T21:31:47.783281] [DEBUG] [LOCAL ] TPC: Primary data saved (39183 records). [2025-10-20T21:31:47.783434] [INFO ] [MAIN ] TPC: Full history saved. [2025-10-20T21:31:47.783801] [INFO ] [MAIN ] Update Progress: 2114/2367 symbols completed. TPC took 30.5s. Estimated remaining: 32m 31s. Rate: 7.8 symbols/min [2025-10-20T21:31:47.954577] [INFO ] [MAIN ] TPVG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:31:47.980807] [DEBUG] [LOCAL ] TPVG: Loaded 27655 existing records from primary storage. [2025-10-20T21:31:47.981089] [INFO ] [MAIN ] TPVG: Loaded 27655 existing records. [2025-10-20T21:31:48.168952] [WARN ] [MAIN ] TPVG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:31:48.169199] [INFO ] [MAIN ] TPVG: Wiping existing primary data before full refresh. [2025-10-20T21:31:48.169263] [INFO ] [LOCAL ] TPVG: Starting deletion of primary data. [2025-10-20T21:31:48.169525] [DEBUG] [LOCAL ] TPVG: Deleted primary data file. [2025-10-20T21:31:48.169594] [INFO ] [MAIN ] TPVG: Starting full primary refresh. [2025-10-20T21:31:48.256529] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-06. [2025-10-20T21:31:48.551788] [INFO ] [REMOTE ] TPVG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:31:48.580001] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-02 (1.93 seconds). [2025-10-20T21:31:48.684741] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-02 (1.22 seconds). [2025-10-20T21:31:49.051398] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-03. [2025-10-20T21:31:49.812394] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:31:49.851345] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-07. [2025-10-20T21:31:50.651359] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-01. [2025-10-20T21:31:50.897629] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T21:31:51.418519] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-01 (0.77 seconds). [2025-10-20T21:31:51.451382] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-03. [2025-10-20T21:31:51.521002] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T21:31:52.251384] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-02. [2025-10-20T21:31:52.354742] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-03 (0.90 seconds). [2025-10-20T21:31:53.051389] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-04. [2025-10-20T21:31:53.299462] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-02 (1.05 seconds). [2025-10-20T21:31:53.851438] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-08. [2025-10-20T21:31:54.113024] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-04 (1.06 seconds). [2025-10-20T21:31:54.651385] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-04. [2025-10-20T21:31:55.223433] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T21:31:55.451413] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-05. [2025-10-20T21:31:56.251409] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-09. [2025-10-20T21:31:56.369423] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T21:31:56.538102] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T21:31:57.051433] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-06. [2025-10-20T21:31:57.785932] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:31:57.851368] [INFO ] [REMOTE ] TPH: Fetching primary data for 2025-10. [2025-10-20T21:31:58.070574] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-06 (1.02 seconds). [2025-10-20T21:31:58.651382] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-07. [2025-10-20T21:31:59.336497] [INFO ] [REMOTE ] TPH: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T21:31:59.342844] [INFO ] [REMOTE ] TPH: Full fetch complete (64905 records). [2025-10-20T21:31:59.342951] [DEBUG] [VALIDATOR ] TPH: Validating primary data. [2025-10-20T21:31:59.456436] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-03. [2025-10-20T21:31:59.725395] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T21:31:59.749259] [DEBUG] [VALIDATOR ] TPH: All expected trading days are present. [2025-10-20T21:31:59.749349] [DEBUG] [VALIDATOR ] TPH: Primary data validation passed. [2025-10-20T21:31:59.750389] [INFO ] [MAIN ] TPH: Fetched primary data is valid. [2025-10-20T21:31:59.750453] [DEBUG] [LOCAL ] TPH: Saving primary data. [2025-10-20T21:31:59.751701] [DEBUG] [LOCAL ] TPH: Trimmed 64905 records to 44829 (keeping 200 days of data) [2025-10-20T21:32:00.000849] [DEBUG] [LOCAL ] TPH: Primary data saved (44829 records). [2025-10-20T21:32:00.000988] [INFO ] [MAIN ] TPH: Full history saved. [2025-10-20T21:32:00.001062] [INFO ] [MAIN ] Update Progress: 2115/2367 symbols completed. TPH took 31.6s. Estimated remaining: 32m 24s. Rate: 7.8 symbols/min [2025-10-20T21:32:00.167258] [INFO ] [MAIN ] TPYP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:00.176783] [DEBUG] [LOCAL ] TPYP: Loaded 8009 existing records from primary storage. [2025-10-20T21:32:00.176967] [INFO ] [MAIN ] TPYP: Loaded 8009 existing records. [2025-10-20T21:32:00.256516] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-05. [2025-10-20T21:32:00.353251] [WARN ] [MAIN ] TPYP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:00.353463] [INFO ] [MAIN ] TPYP: Wiping existing primary data before full refresh. [2025-10-20T21:32:00.353508] [INFO ] [LOCAL ] TPYP: Starting deletion of primary data. [2025-10-20T21:32:00.353711] [DEBUG] [LOCAL ] TPYP: Deleted primary data file. [2025-10-20T21:32:00.353769] [INFO ] [MAIN ] TPYP: Starting full primary refresh. [2025-10-20T21:32:00.711729] [INFO ] [REMOTE ] TPYP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:00.836322] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-03 (1.38 seconds). [2025-10-20T21:32:01.051435] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-01. [2025-10-20T21:32:01.851363] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-08. [2025-10-20T21:32:01.896980] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T21:32:01.909204] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-01 (0.86 seconds). [2025-10-20T21:32:02.651366] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-02. [2025-10-20T21:32:02.793269] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-08 (0.94 seconds). [2025-10-20T21:32:03.451413] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-04. [2025-10-20T21:32:03.615935] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T21:32:04.251461] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-06. [2025-10-20T21:32:04.987843] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T21:32:05.051430] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-09. [2025-10-20T21:32:05.513893] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-09 (0.46 seconds). [2025-10-20T21:32:05.851368] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-03. [2025-10-20T21:32:06.610199] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-06 (2.36 seconds). [2025-10-20T21:32:06.651403] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-07. [2025-10-20T21:32:06.719327] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-03 (0.87 seconds). [2025-10-20T21:32:07.451486] [INFO ] [REMOTE ] TPL: Fetching primary data for 2025-10. [2025-10-20T21:32:08.251470] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-04. [2025-10-20T21:32:08.381867] [INFO ] [REMOTE ] TPL: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T21:32:08.384820] [INFO ] [REMOTE ] TPL: Full fetch complete (17596 records). [2025-10-20T21:32:08.384907] [DEBUG] [VALIDATOR ] TPL: Validating primary data. [2025-10-20T21:32:08.754846] [DEBUG] [VALIDATOR ] TPL: All expected trading days are present. [2025-10-20T21:32:08.754947] [DEBUG] [VALIDATOR ] TPL: Primary data validation passed. [2025-10-20T21:32:08.755354] [INFO ] [MAIN ] TPL: Fetched primary data is valid. [2025-10-20T21:32:08.755424] [DEBUG] [LOCAL ] TPL: Saving primary data. [2025-10-20T21:32:08.756768] [DEBUG] [LOCAL ] TPL: Trimmed 17596 records to 12450 (keeping 200 days of data) [2025-10-20T21:32:08.870853] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-07 (2.22 seconds). [2025-10-20T21:32:08.870963] [DEBUG] [LOCAL ] TPL: Primary data saved (12450 records). [2025-10-20T21:32:08.882603] [INFO ] [MAIN ] TPL: Full history saved. [2025-10-20T21:32:08.882943] [INFO ] [MAIN ] Update Progress: 2116/2367 symbols completed. TPL took 28.1s. Estimated remaining: 32m 16s. Rate: 7.8 symbols/min [2025-10-20T21:32:09.051417] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-08. [2025-10-20T21:32:09.054183] [INFO ] [MAIN ] TPZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:09.062389] [DEBUG] [LOCAL ] TPZ: Loaded 4739 existing records from primary storage. [2025-10-20T21:32:09.062572] [INFO ] [MAIN ] TPZ: Loaded 4739 existing records. [2025-10-20T21:32:09.171602] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-04 (0.92 seconds). [2025-10-20T21:32:09.256516] [WARN ] [MAIN ] TPZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:09.256716] [INFO ] [MAIN ] TPZ: Wiping existing primary data before full refresh. [2025-10-20T21:32:09.256782] [INFO ] [LOCAL ] TPZ: Starting deletion of primary data. [2025-10-20T21:32:09.256982] [DEBUG] [LOCAL ] TPZ: Deleted primary data file. [2025-10-20T21:32:09.257046] [INFO ] [MAIN ] TPZ: Starting full primary refresh. [2025-10-20T21:32:09.604271] [INFO ] [REMOTE ] TPZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:09.851416] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-01. [2025-10-20T21:32:10.644735] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T21:32:10.651321] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-02. [2025-10-20T21:32:11.046761] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-08 (2.00 seconds). [2025-10-20T21:32:11.137539] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T21:32:11.451389] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-09. [2025-10-20T21:32:12.252785] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-05. [2025-10-20T21:32:12.261576] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T21:32:13.021389] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T21:32:13.051352] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-03. [2025-10-20T21:32:13.789783] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T21:32:13.851395] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-04. [2025-10-20T21:32:14.650112] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T21:32:14.652457] [INFO ] [REMOTE ] TPR: Fetching primary data for 2025-10. [2025-10-20T21:32:15.451389] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-06. [2025-10-20T21:32:16.220998] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T21:32:16.251398] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-05. [2025-10-20T21:32:16.429243] [INFO ] [REMOTE ] TPR: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T21:32:16.437269] [INFO ] [REMOTE ] TPR: Full fetch complete (77502 records). [2025-10-20T21:32:16.437416] [DEBUG] [VALIDATOR ] TPR: Validating primary data. [2025-10-20T21:32:16.837003] [DEBUG] [VALIDATOR ] TPR: All expected trading days are present. [2025-10-20T21:32:16.837095] [DEBUG] [VALIDATOR ] TPR: Primary data validation passed. [2025-10-20T21:32:16.838211] [INFO ] [MAIN ] TPR: Fetched primary data is valid. [2025-10-20T21:32:16.838275] [DEBUG] [LOCAL ] TPR: Saving primary data. [2025-10-20T21:32:16.839753] [DEBUG] [LOCAL ] TPR: Trimmed 77502 records to 53371 (keeping 200 days of data) [2025-10-20T21:32:17.052904] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-05. [2025-10-20T21:32:17.171873] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T21:32:17.192591] [DEBUG] [LOCAL ] TPR: Primary data saved (53371 records). [2025-10-20T21:32:17.192734] [INFO ] [MAIN ] TPR: Full history saved. [2025-10-20T21:32:17.192813] [INFO ] [MAIN ] Update Progress: 2117/2367 symbols completed. TPR took 34.9s. Estimated remaining: 32m 9s. Rate: 7.8 symbols/min [2025-10-20T21:32:17.644493] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-05 (0.59 seconds). [2025-10-20T21:32:17.851409] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-06. [2025-10-20T21:32:18.651354] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-07. [2025-10-20T21:32:19.028894] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-06 (1.18 seconds). [2025-10-20T21:32:19.094770] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-07 (0.44 seconds). [2025-10-20T21:32:19.451370] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-06. [2025-10-20T21:32:19.955622] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-06 (0.50 seconds). [2025-10-20T21:32:20.251356] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-07. [2025-10-20T21:32:21.051350] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-07. [2025-10-20T21:32:21.421767] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T21:32:21.468357] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-07 (0.42 seconds). [2025-10-20T21:32:21.851420] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-08. [2025-10-20T21:32:22.564540] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-08 (0.71 seconds). [2025-10-20T21:32:22.651387] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-09. [2025-10-20T21:32:23.337231] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T21:32:23.451368] [INFO ] [REMOTE ] TPYP: Fetching primary data for 2025-10. [2025-10-20T21:32:24.095869] [INFO ] [REMOTE ] TPYP: Incremental fetch complete for 2025-10 (0.64 seconds). [2025-10-20T21:32:24.097494] [INFO ] [REMOTE ] TPYP: Full fetch complete (12466 records). [2025-10-20T21:32:24.097583] [DEBUG] [VALIDATOR ] TPYP: Validating primary data. [2025-10-20T21:32:24.256484] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-08. [2025-10-20T21:32:24.477682] [DEBUG] [VALIDATOR ] TPYP: All expected trading days are present. [2025-10-20T21:32:24.477883] [DEBUG] [VALIDATOR ] TPYP: Primary data validation passed. [2025-10-20T21:32:24.478304] [INFO ] [MAIN ] TPYP: Fetched primary data is valid. [2025-10-20T21:32:24.478347] [DEBUG] [LOCAL ] TPYP: Saving primary data. [2025-10-20T21:32:24.479200] [DEBUG] [LOCAL ] TPYP: Trimmed 12466 records to 7931 (keeping 200 days of data) [2025-10-20T21:32:24.528600] [DEBUG] [LOCAL ] TPYP: Primary data saved (7931 records). [2025-10-20T21:32:24.528774] [INFO ] [MAIN ] TPYP: Full history saved. [2025-10-20T21:32:24.528860] [INFO ] [MAIN ] Update Progress: 2118/2367 symbols completed. TPYP took 24.5s. Estimated remaining: 32m 1s. Rate: 7.8 symbols/min [2025-10-20T21:32:25.051411] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-08. [2025-10-20T21:32:25.552960] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:32:25.604791] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T21:32:25.851444] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-09. [2025-10-20T21:32:26.430390] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T21:32:26.651438] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-09. [2025-10-20T21:32:27.451357] [INFO ] [REMOTE ] TPZ: Fetching primary data for 2025-10. [2025-10-20T21:32:27.934125] [INFO ] [REMOTE ] TPZ: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T21:32:27.935464] [INFO ] [REMOTE ] TPZ: Full fetch complete (8608 records). [2025-10-20T21:32:27.935626] [DEBUG] [VALIDATOR ] TPZ: Validating primary data. [2025-10-20T21:32:28.264578] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-09 (1.61 seconds). [2025-10-20T21:32:28.325329] [INFO ] [REMOTE ] TPVG: Fetching primary data for 2025-10. [2025-10-20T21:32:28.325515] [DEBUG] [VALIDATOR ] TPZ: All expected trading days are present. [2025-10-20T21:32:28.325595] [DEBUG] [VALIDATOR ] TPZ: Primary data validation passed. [2025-10-20T21:32:28.326542] [INFO ] [MAIN ] TPZ: Fetched primary data is valid. [2025-10-20T21:32:28.326593] [DEBUG] [LOCAL ] TPZ: Saving primary data. [2025-10-20T21:32:28.327061] [DEBUG] [LOCAL ] TPZ: Trimmed 8608 records to 4653 (keeping 200 days of data) [2025-10-20T21:32:28.356099] [DEBUG] [LOCAL ] TPZ: Primary data saved (4653 records). [2025-10-20T21:32:28.356222] [INFO ] [MAIN ] TPZ: Full history saved. [2025-10-20T21:32:28.356297] [INFO ] [MAIN ] Update Progress: 2119/2367 symbols completed. TPZ took 19.5s. Estimated remaining: 31m 53s. Rate: 7.8 symbols/min [2025-10-20T21:32:29.278128] [INFO ] [REMOTE ] TPVG: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T21:32:29.283124] [INFO ] [REMOTE ] TPVG: Full fetch complete (39369 records). [2025-10-20T21:32:29.283261] [DEBUG] [VALIDATOR ] TPVG: Validating primary data. [2025-10-20T21:32:29.700989] [DEBUG] [VALIDATOR ] TPVG: All expected trading days are present. [2025-10-20T21:32:29.701087] [DEBUG] [VALIDATOR ] TPVG: Primary data validation passed. [2025-10-20T21:32:29.701886] [INFO ] [MAIN ] TPVG: Fetched primary data is valid. [2025-10-20T21:32:29.701952] [DEBUG] [LOCAL ] TPVG: Saving primary data. [2025-10-20T21:32:29.703030] [DEBUG] [LOCAL ] TPVG: Trimmed 39369 records to 27321 (keeping 200 days of data) [2025-10-20T21:32:29.859064] [DEBUG] [LOCAL ] TPVG: Primary data saved (27321 records). [2025-10-20T21:32:29.859197] [INFO ] [MAIN ] TPVG: Full history saved. [2025-10-20T21:32:29.859274] [INFO ] [MAIN ] Update Progress: 2120/2367 symbols completed. TPVG took 42.1s. Estimated remaining: 31m 44s. Rate: 7.8 symbols/min [2025-10-20T21:32:30.501556] [INFO ] [MAIN ] TRAK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:30.568758] [INFO ] [MAIN ] TRC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:30.545218] [INFO ] [MAIN ] TREX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:30.615135] [INFO ] [MAIN ] TR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:30.632850] [DEBUG] [LOCAL ] TRAK: Loaded 9840 existing records from primary storage. [2025-10-20T21:32:30.633704] [INFO ] [MAIN ] TRAK: Loaded 9840 existing records. [2025-10-20T21:32:30.701558] [DEBUG] [LOCAL ] TRC: Loaded 15736 existing records from primary storage. [2025-10-20T21:32:30.738282] [INFO ] [MAIN ] TRC: Loaded 15736 existing records. [2025-10-20T21:32:30.778751] [DEBUG] [LOCAL ] TR: Loaded 19196 existing records from primary storage. [2025-10-20T21:32:30.852760] [INFO ] [MAIN ] TR: Loaded 19196 existing records. [2025-10-20T21:32:31.071053] [DEBUG] [LOCAL ] TREX: Loaded 50560 existing records from primary storage. [2025-10-20T21:32:30.991052] [WARN ] [MAIN ] TRAK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:31.076590] [INFO ] [MAIN ] TREX: Loaded 50560 existing records. [2025-10-20T21:32:31.148736] [INFO ] [MAIN ] TRAK: Wiping existing primary data before full refresh. [2025-10-20T21:32:31.165492] [INFO ] [LOCAL ] TRAK: Starting deletion of primary data. [2025-10-20T21:32:31.230487] [WARN ] [MAIN ] TRC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:31.258981] [DEBUG] [LOCAL ] TRAK: Deleted primary data file. [2025-10-20T21:32:31.293845] [INFO ] [MAIN ] TRAK: Starting full primary refresh. [2025-10-20T21:32:31.328081] [INFO ] [MAIN ] TRC: Wiping existing primary data before full refresh. [2025-10-20T21:32:31.416330] [INFO ] [LOCAL ] TRC: Starting deletion of primary data. [2025-10-20T21:32:31.416234] [WARN ] [MAIN ] TR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:31.436840] [DEBUG] [LOCAL ] TRC: Deleted primary data file. [2025-10-20T21:32:31.490998] [INFO ] [MAIN ] TR: Wiping existing primary data before full refresh. [2025-10-20T21:32:31.485737] [INFO ] [MAIN ] TRC: Starting full primary refresh. [2025-10-20T21:32:31.519730] [INFO ] [LOCAL ] TR: Starting deletion of primary data. [2025-10-20T21:32:31.663422] [WARN ] [MAIN ] TREX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:31.659571] [DEBUG] [LOCAL ] TR: Deleted primary data file. [2025-10-20T21:32:31.721134] [INFO ] [MAIN ] TR: Starting full primary refresh. [2025-10-20T21:32:31.721025] [INFO ] [MAIN ] TREX: Wiping existing primary data before full refresh. [2025-10-20T21:32:31.738912] [INFO ] [LOCAL ] TREX: Starting deletion of primary data. [2025-10-20T21:32:31.774478] [DEBUG] [LOCAL ] TREX: Deleted primary data file. [2025-10-20T21:32:31.796239] [INFO ] [MAIN ] TREX: Starting full primary refresh. [2025-10-20T21:32:32.400742] [INFO ] [REMOTE ] TRAK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:32.430817] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-01. [2025-10-20T21:32:32.709267] [INFO ] [REMOTE ] TRC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:32.792209] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-01. [2025-10-20T21:32:32.941191] [INFO ] [REMOTE ] TR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:32.956506] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-01. [2025-10-20T21:32:32.978773] [INFO ] [REMOTE ] TREX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:32.979241] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-01. [2025-10-20T21:32:33.311776] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:32:33.314178] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-02. [2025-10-20T21:32:33.795489] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T21:32:33.798570] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-02. [2025-10-20T21:32:33.871517] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T21:32:33.873749] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-03. [2025-10-20T21:32:34.002375] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-01 (1.05 seconds). [2025-10-20T21:32:34.525332] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:32:34.653830] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-02. [2025-10-20T21:32:34.655873] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T21:32:34.731396] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T21:32:35.451407] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-04. [2025-10-20T21:32:36.180565] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:32:36.251373] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-03. [2025-10-20T21:32:36.428018] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T21:32:37.051373] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-05. [2025-10-20T21:32:37.695803] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:32:37.765341] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T21:32:37.851373] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-04. [2025-10-20T21:32:38.651406] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-02. [2025-10-20T21:32:39.451373] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-03. [2025-10-20T21:32:39.480802] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:32:39.539831] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T21:32:40.251402] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-03. [2025-10-20T21:32:40.444953] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T21:32:41.051349] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-04. [2025-10-20T21:32:41.234477] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T21:32:41.851358] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-06. [2025-10-20T21:32:41.871741] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T21:32:42.602311] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T21:32:42.651341] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-05. [2025-10-20T21:32:43.451484] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-04. [2025-10-20T21:32:44.256472] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-05. [2025-10-20T21:32:44.274138] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-05 (1.62 seconds). [2025-10-20T21:32:44.438731] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T21:32:45.051439] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-05. [2025-10-20T21:32:45.464023] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T21:32:45.851384] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-07. [2025-10-20T21:32:46.374350] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T21:32:46.651477] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-06. [2025-10-20T21:32:46.664924] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T21:32:47.451363] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-06. [2025-10-20T21:32:47.719460] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T21:32:48.251472] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-08. [2025-10-20T21:32:48.788445] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-08 (0.54 seconds). [2025-10-20T21:32:49.051419] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-09. [2025-10-20T21:32:49.157720] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:32:49.536186] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:32:49.851367] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-07. [2025-10-20T21:32:50.651412] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-06. [2025-10-20T21:32:51.022066] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T21:32:51.451390] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-08. [2025-10-20T21:32:51.579032] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-06 (0.93 seconds). [2025-10-20T21:32:52.251356] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-07. [2025-10-20T21:32:52.462080] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T21:32:53.051485] [INFO ] [REMOTE ] TRAK: Fetching primary data for 2025-10. [2025-10-20T21:32:53.755999] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T21:32:53.841996] [INFO ] [REMOTE ] TRAK: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:32:53.844804] [INFO ] [REMOTE ] TRAK: Full fetch complete (13694 records). [2025-10-20T21:32:53.844874] [DEBUG] [VALIDATOR ] TRAK: Validating primary data. [2025-10-20T21:32:53.856481] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-09. [2025-10-20T21:32:54.204293] [DEBUG] [VALIDATOR ] TRAK: All expected trading days are present. [2025-10-20T21:32:54.204377] [DEBUG] [VALIDATOR ] TRAK: Primary data validation passed. [2025-10-20T21:32:54.204691] [INFO ] [MAIN ] TRAK: Fetched primary data is valid. [2025-10-20T21:32:54.204752] [DEBUG] [LOCAL ] TRAK: Saving primary data. [2025-10-20T21:32:54.205203] [DEBUG] [LOCAL ] TRAK: Trimmed 13694 records to 9660 (keeping 200 days of data) [2025-10-20T21:32:54.264557] [DEBUG] [LOCAL ] TRAK: Primary data saved (9660 records). [2025-10-20T21:32:54.264730] [INFO ] [MAIN ] TRAK: Full history saved. [2025-10-20T21:32:54.264814] [INFO ] [MAIN ] Update Progress: 2121/2367 symbols completed. TRAK took 24.4s. Estimated remaining: 31m 39s. Rate: 7.8 symbols/min [2025-10-20T21:32:54.429302] [INFO ] [MAIN ] TRGP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:54.480569] [DEBUG] [LOCAL ] TRGP: Loaded 51552 existing records from primary storage. [2025-10-20T21:32:54.480922] [INFO ] [MAIN ] TRGP: Loaded 51552 existing records. [2025-10-20T21:32:54.656015] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-07. [2025-10-20T21:32:54.660680] [WARN ] [MAIN ] TRGP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:54.661622] [INFO ] [MAIN ] TRGP: Wiping existing primary data before full refresh. [2025-10-20T21:32:54.661710] [INFO ] [LOCAL ] TRGP: Starting deletion of primary data. [2025-10-20T21:32:54.662111] [DEBUG] [LOCAL ] TRGP: Deleted primary data file. [2025-10-20T21:32:54.662168] [INFO ] [MAIN ] TRGP: Starting full primary refresh. [2025-10-20T21:32:55.028207] [INFO ] [REMOTE ] TRGP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:55.024479] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T21:32:55.451442] [INFO ] [REMOTE ] TR: Fetching primary data for 2025-10. [2025-10-20T21:32:55.577202] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-07 (0.92 seconds). [2025-10-20T21:32:56.251389] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-08. [2025-10-20T21:32:56.339979] [INFO ] [REMOTE ] TR: Incremental fetch complete for 2025-10 (0.89 seconds). [2025-10-20T21:32:56.343039] [INFO ] [REMOTE ] TR: Full fetch complete (25480 records). [2025-10-20T21:32:56.343129] [DEBUG] [VALIDATOR ] TR: Validating primary data. [2025-10-20T21:32:56.714781] [DEBUG] [VALIDATOR ] TR: All expected trading days are present. [2025-10-20T21:32:56.714888] [DEBUG] [VALIDATOR ] TR: Primary data validation passed. [2025-10-20T21:32:56.715873] [INFO ] [MAIN ] TR: Fetched primary data is valid. [2025-10-20T21:32:56.715942] [DEBUG] [LOCAL ] TR: Saving primary data. [2025-10-20T21:32:56.716941] [DEBUG] [LOCAL ] TR: Trimmed 25480 records to 19094 (keeping 200 days of data) [2025-10-20T21:32:56.830804] [DEBUG] [LOCAL ] TR: Primary data saved (19094 records). [2025-10-20T21:32:56.830964] [INFO ] [MAIN ] TR: Full history saved. [2025-10-20T21:32:56.831046] [INFO ] [MAIN ] Update Progress: 2122/2367 symbols completed. TR took 27.0s. Estimated remaining: 31m 30s. Rate: 7.8 symbols/min [2025-10-20T21:32:57.006972] [INFO ] [MAIN ] TRN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:32:57.046254] [DEBUG] [LOCAL ] TRN: Loaded 37211 existing records from primary storage. [2025-10-20T21:32:57.046672] [INFO ] [MAIN ] TRN: Loaded 37211 existing records. [2025-10-20T21:32:57.058446] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-08. [2025-10-20T21:32:57.184501] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T21:32:57.243192] [WARN ] [MAIN ] TRN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:32:57.243433] [INFO ] [MAIN ] TRN: Wiping existing primary data before full refresh. [2025-10-20T21:32:57.243490] [INFO ] [LOCAL ] TRN: Starting deletion of primary data. [2025-10-20T21:32:57.243746] [DEBUG] [LOCAL ] TRN: Deleted primary data file. [2025-10-20T21:32:57.243802] [INFO ] [MAIN ] TRN: Starting full primary refresh. [2025-10-20T21:32:57.593994] [INFO ] [REMOTE ] TRN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:32:57.851537] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-09. [2025-10-20T21:32:58.651375] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-01. [2025-10-20T21:32:58.779018] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-09 (0.93 seconds). [2025-10-20T21:32:58.843434] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:32:59.451430] [INFO ] [REMOTE ] TRC: Fetching primary data for 2025-10. [2025-10-20T21:33:00.251350] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-01. [2025-10-20T21:33:00.284837] [INFO ] [REMOTE ] TRC: Incremental fetch complete for 2025-10 (0.83 seconds). [2025-10-20T21:33:00.287130] [INFO ] [REMOTE ] TRC: Full fetch complete (21821 records). [2025-10-20T21:33:00.287198] [DEBUG] [VALIDATOR ] TRC: Validating primary data. [2025-10-20T21:33:00.598900] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T21:33:00.688711] [DEBUG] [VALIDATOR ] TRC: All expected trading days are present. [2025-10-20T21:33:00.688794] [DEBUG] [VALIDATOR ] TRC: Primary data validation passed. [2025-10-20T21:33:00.689231] [INFO ] [MAIN ] TRC: Fetched primary data is valid. [2025-10-20T21:33:00.689268] [DEBUG] [LOCAL ] TRC: Saving primary data. [2025-10-20T21:33:00.689843] [DEBUG] [LOCAL ] TRC: Trimmed 21821 records to 15518 (keeping 200 days of data) [2025-10-20T21:33:00.782759] [DEBUG] [LOCAL ] TRC: Primary data saved (15518 records). [2025-10-20T21:33:00.782975] [INFO ] [MAIN ] TRC: Full history saved. [2025-10-20T21:33:00.783083] [INFO ] [MAIN ] Update Progress: 2123/2367 symbols completed. TRC took 30.9s. Estimated remaining: 31m 22s. Rate: 7.8 symbols/min [2025-10-20T21:33:00.955488] [INFO ] [MAIN ] TRNO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:00.995490] [DEBUG] [LOCAL ] TRNO: Loaded 44531 existing records from primary storage. [2025-10-20T21:33:00.995827] [INFO ] [MAIN ] TRNO: Loaded 44531 existing records. [2025-10-20T21:33:01.058598] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-02. [2025-10-20T21:33:01.199685] [WARN ] [MAIN ] TRNO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:01.200003] [INFO ] [MAIN ] TRNO: Wiping existing primary data before full refresh. [2025-10-20T21:33:01.200066] [INFO ] [LOCAL ] TRNO: Starting deletion of primary data. [2025-10-20T21:33:01.200295] [DEBUG] [LOCAL ] TRNO: Deleted primary data file. [2025-10-20T21:33:01.200353] [INFO ] [MAIN ] TRNO: Starting full primary refresh. [2025-10-20T21:33:01.564555] [INFO ] [REMOTE ] TRNO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:01.709232] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T21:33:01.851365] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-01. [2025-10-20T21:33:02.651372] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-02. [2025-10-20T21:33:02.911041] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:33:03.451696] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-03. [2025-10-20T21:33:03.452571] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T21:33:04.066893] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T21:33:04.251363] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-03. [2025-10-20T21:33:05.051356] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-02. [2025-10-20T21:33:05.313894] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T21:33:05.691752] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:33:05.851359] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-04. [2025-10-20T21:33:06.618201] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:33:06.651328] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-03. [2025-10-20T21:33:07.214368] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T21:33:07.451375] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-04. [2025-10-20T21:33:08.251414] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-05. [2025-10-20T21:33:08.303518] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T21:33:09.051357] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-04. [2025-10-20T21:33:09.332444] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-04 (1.88 seconds). [2025-10-20T21:33:09.779240] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:33:09.851351] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-09. [2025-10-20T21:33:10.651370] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-05. [2025-10-20T21:33:10.786919] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T21:33:11.190143] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-09 (1.34 seconds). [2025-10-20T21:33:11.451446] [INFO ] [REMOTE ] TREX: Fetching primary data for 2025-10. [2025-10-20T21:33:12.251376] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-06. [2025-10-20T21:33:12.426623] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T21:33:13.051370] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-06. [2025-10-20T21:33:13.164305] [INFO ] [REMOTE ] TREX: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T21:33:13.171923] [INFO ] [REMOTE ] TREX: Full fetch complete (71700 records). [2025-10-20T21:33:13.172093] [DEBUG] [VALIDATOR ] TREX: Validating primary data. [2025-10-20T21:33:13.569031] [DEBUG] [VALIDATOR ] TREX: All expected trading days are present. [2025-10-20T21:33:13.569195] [DEBUG] [VALIDATOR ] TREX: Primary data validation passed. [2025-10-20T21:33:13.570571] [INFO ] [MAIN ] TREX: Fetched primary data is valid. [2025-10-20T21:33:13.570679] [DEBUG] [LOCAL ] TREX: Saving primary data. [2025-10-20T21:33:13.590439] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T21:33:13.597474] [DEBUG] [LOCAL ] TREX: Trimmed 71700 records to 49841 (keeping 200 days of data) [2025-10-20T21:33:13.866628] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-07. [2025-10-20T21:33:13.902629] [DEBUG] [LOCAL ] TREX: Primary data saved (49841 records). [2025-10-20T21:33:13.902808] [INFO ] [MAIN ] TREX: Full history saved. [2025-10-20T21:33:13.903338] [INFO ] [MAIN ] Update Progress: 2124/2367 symbols completed. TREX took 44.0s. Estimated remaining: 31m 15s. Rate: 7.8 symbols/min [2025-10-20T21:33:14.079177] [INFO ] [MAIN ] TROX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:14.130773] [DEBUG] [LOCAL ] TROX: Loaded 52933 existing records from primary storage. [2025-10-20T21:33:14.131140] [INFO ] [MAIN ] TROX: Loaded 52933 existing records. [2025-10-20T21:33:14.315332] [WARN ] [MAIN ] TROX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:14.315611] [INFO ] [MAIN ] TROX: Wiping existing primary data before full refresh. [2025-10-20T21:33:14.315884] [INFO ] [LOCAL ] TROX: Starting deletion of primary data. [2025-10-20T21:33:14.316106] [DEBUG] [LOCAL ] TROX: Deleted primary data file. [2025-10-20T21:33:14.316145] [INFO ] [MAIN ] TROX: Starting full primary refresh. [2025-10-20T21:33:14.656497] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-05. [2025-10-20T21:33:14.718218] [INFO ] [REMOTE ] TROX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:14.922261] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:33:15.451418] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-07. [2025-10-20T21:33:15.630718] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T21:33:16.251422] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-08. [2025-10-20T21:33:16.356101] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T21:33:16.769339] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-07 (1.32 seconds). [2025-10-20T21:33:17.051409] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-01. [2025-10-20T21:33:17.731330] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T21:33:17.851366] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-09. [2025-10-20T21:33:18.661479] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-01 (1.61 seconds). [2025-10-20T21:33:18.655163] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-06. [2025-10-20T21:33:18.746791] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-09 (0.90 seconds). [2025-10-20T21:33:19.451411] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-02. [2025-10-20T21:33:19.857248] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T21:33:20.251407] [INFO ] [REMOTE ] TRN: Fetching primary data for 2025-10. [2025-10-20T21:33:21.013510] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T21:33:21.051404] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-08. [2025-10-20T21:33:21.467926] [INFO ] [REMOTE ] TRN: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T21:33:21.473351] [INFO ] [REMOTE ] TRN: Full fetch complete (52657 records). [2025-10-20T21:33:21.473496] [DEBUG] [VALIDATOR ] TRN: Validating primary data. [2025-10-20T21:33:21.857415] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-07. [2025-10-20T21:33:21.876951] [DEBUG] [VALIDATOR ] TRN: All expected trading days are present. [2025-10-20T21:33:21.877052] [DEBUG] [VALIDATOR ] TRN: Primary data validation passed. [2025-10-20T21:33:21.878039] [INFO ] [MAIN ] TRN: Fetched primary data is valid. [2025-10-20T21:33:21.878218] [DEBUG] [LOCAL ] TRN: Saving primary data. [2025-10-20T21:33:21.879427] [DEBUG] [LOCAL ] TRN: Trimmed 52657 records to 36521 (keeping 200 days of data) [2025-10-20T21:33:22.103985] [DEBUG] [LOCAL ] TRN: Primary data saved (36521 records). [2025-10-20T21:33:22.104152] [INFO ] [MAIN ] TRN: Full history saved. [2025-10-20T21:33:22.104238] [INFO ] [MAIN ] Update Progress: 2125/2367 symbols completed. TRN took 25.3s. Estimated remaining: 31m 7s. Rate: 7.8 symbols/min [2025-10-20T21:33:22.270369] [INFO ] [MAIN ] TRP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:22.318794] [DEBUG] [LOCAL ] TRP: Loaded 52912 existing records from primary storage. [2025-10-20T21:33:22.319120] [INFO ] [MAIN ] TRP: Loaded 52912 existing records. [2025-10-20T21:33:22.493217] [WARN ] [MAIN ] TRP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:22.493425] [INFO ] [MAIN ] TRP: Wiping existing primary data before full refresh. [2025-10-20T21:33:22.493468] [INFO ] [LOCAL ] TRP: Starting deletion of primary data. [2025-10-20T21:33:22.493715] [DEBUG] [LOCAL ] TRP: Deleted primary data file. [2025-10-20T21:33:22.493775] [INFO ] [MAIN ] TRP: Starting full primary refresh. [2025-10-20T21:33:22.652944] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-03. [2025-10-20T21:33:22.876735] [INFO ] [REMOTE ] TRP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:22.911563] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T21:33:23.428415] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T21:33:23.451349] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-08. [2025-10-20T21:33:24.251394] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-09. [2025-10-20T21:33:24.485264] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:33:24.670914] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T21:33:25.051402] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-04. [2025-10-20T21:33:25.546911] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-09 (1.30 seconds). [2025-10-20T21:33:25.851418] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-01. [2025-10-20T21:33:26.651406] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-09. [2025-10-20T21:33:26.847550] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T21:33:27.451348] [INFO ] [REMOTE ] TRGP: Fetching primary data for 2025-10. [2025-10-20T21:33:27.647378] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-01 (1.80 seconds). [2025-10-20T21:33:27.804836] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T21:33:28.251432] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-02. [2025-10-20T21:33:29.025807] [INFO ] [REMOTE ] TRGP: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T21:33:29.033166] [INFO ] [REMOTE ] TRGP: Full fetch complete (73650 records). [2025-10-20T21:33:29.033313] [DEBUG] [VALIDATOR ] TRGP: Validating primary data. [2025-10-20T21:33:29.056457] [INFO ] [REMOTE ] TRNO: Fetching primary data for 2025-10. [2025-10-20T21:33:29.417292] [DEBUG] [VALIDATOR ] TRGP: All expected trading days are present. [2025-10-20T21:33:29.417382] [DEBUG] [VALIDATOR ] TRGP: Primary data validation passed. [2025-10-20T21:33:29.418588] [INFO ] [MAIN ] TRGP: Fetched primary data is valid. [2025-10-20T21:33:29.418675] [DEBUG] [LOCAL ] TRGP: Saving primary data. [2025-10-20T21:33:29.420084] [DEBUG] [LOCAL ] TRGP: Trimmed 73650 records to 50800 (keeping 200 days of data) [2025-10-20T21:33:29.739005] [DEBUG] [LOCAL ] TRGP: Primary data saved (50800 records). [2025-10-20T21:33:29.739340] [INFO ] [MAIN ] TRGP: Full history saved. [2025-10-20T21:33:29.739438] [INFO ] [MAIN ] Update Progress: 2126/2367 symbols completed. TRGP took 35.5s. Estimated remaining: 30m 60s. Rate: 7.8 symbols/min [2025-10-20T21:33:29.859028] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-05. [2025-10-20T21:33:29.945382] [INFO ] [MAIN ] TRTX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:29.974043] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:33:30.011912] [DEBUG] [LOCAL ] TRTX: Loaded 37790 existing records from primary storage. [2025-10-20T21:33:30.012201] [INFO ] [MAIN ] TRTX: Loaded 37790 existing records. [2025-10-20T21:33:30.186967] [WARN ] [MAIN ] TRTX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:30.187211] [INFO ] [MAIN ] TRTX: Wiping existing primary data before full refresh. [2025-10-20T21:33:30.187272] [INFO ] [LOCAL ] TRTX: Starting deletion of primary data. [2025-10-20T21:33:30.187496] [DEBUG] [LOCAL ] TRTX: Deleted primary data file. [2025-10-20T21:33:30.187562] [INFO ] [MAIN ] TRTX: Starting full primary refresh. [2025-10-20T21:33:30.567193] [INFO ] [REMOTE ] TRTX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:30.583390] [INFO ] [REMOTE ] TRNO: Incremental fetch complete for 2025-10 (1.53 seconds). [2025-10-20T21:33:30.589563] [INFO ] [REMOTE ] TRNO: Full fetch complete (63002 records). [2025-10-20T21:33:30.589753] [DEBUG] [VALIDATOR ] TRNO: Validating primary data. [2025-10-20T21:33:30.656539] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-03. [2025-10-20T21:33:30.979221] [DEBUG] [VALIDATOR ] TRNO: All expected trading days are present. [2025-10-20T21:33:30.979311] [DEBUG] [VALIDATOR ] TRNO: Primary data validation passed. [2025-10-20T21:33:30.980303] [INFO ] [MAIN ] TRNO: Fetched primary data is valid. [2025-10-20T21:33:30.980368] [DEBUG] [LOCAL ] TRNO: Saving primary data. [2025-10-20T21:33:30.981671] [DEBUG] [LOCAL ] TRNO: Trimmed 63002 records to 43861 (keeping 200 days of data) [2025-10-20T21:33:31.248368] [DEBUG] [LOCAL ] TRNO: Primary data saved (43861 records). [2025-10-20T21:33:31.248504] [INFO ] [MAIN ] TRNO: Full history saved. [2025-10-20T21:33:31.248582] [INFO ] [MAIN ] Update Progress: 2127/2367 symbols completed. TRNO took 30.5s. Estimated remaining: 30m 51s. Rate: 7.8 symbols/min [2025-10-20T21:33:31.425449] [INFO ] [MAIN ] TRU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:31.452636] [INFO ] [REMOTE ] TRTX: Fetching primary data for 2025-01. [2025-10-20T21:33:31.477136] [DEBUG] [LOCAL ] TRU: Loaded 52194 existing records from primary storage. [2025-10-20T21:33:31.477470] [INFO ] [MAIN ] TRU: Loaded 52194 existing records. [2025-10-20T21:33:31.661057] [WARN ] [MAIN ] TRU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:31.661465] [INFO ] [MAIN ] TRU: Wiping existing primary data before full refresh. [2025-10-20T21:33:31.661689] [INFO ] [LOCAL ] TRU: Starting deletion of primary data. [2025-10-20T21:33:31.661905] [DEBUG] [LOCAL ] TRU: Deleted primary data file. [2025-10-20T21:33:31.661953] [INFO ] [MAIN ] TRU: Starting full primary refresh. [2025-10-20T21:33:31.793601] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:33:32.063842] [INFO ] [REMOTE ] TRU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:32.251354] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-06. [2025-10-20T21:33:32.510942] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T21:33:32.788718] [INFO ] [REMOTE ] TRTX: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T21:33:33.051422] [INFO ] [REMOTE ] TRTX: Fetching primary data for 2025-02. [2025-10-20T21:33:33.851368] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-01. [2025-10-20T21:33:33.968190] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T21:33:34.040816] [INFO ] [REMOTE ] TRTX: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T21:33:34.651371] [INFO ] [REMOTE ] TRTX: Fetching primary data for 2025-03. [2025-10-20T21:33:35.451376] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-07. [2025-10-20T21:33:35.671111] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T21:33:36.166150] [INFO ] [REMOTE ] TRTX: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:33:36.251395] [INFO ] [REMOTE ] TRTX: Fetching primary data for 2025-04. [2025-10-20T21:33:36.355986] [DEBUG] [REMOTE ] TRTX: Error fetching primary data for 2025-04: 'API call returned an empty response.' [2025-10-20T21:33:36.356257] [WARN ] [REMOTE ] TRTX: No data returned for month 2025-04 [2025-10-20T21:33:36.356363] [FATAL] [REMOTE ] TRTX: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:33:36.356479] [FATAL] [MAIN ] TRTX: Failed to fetch primary data. [2025-10-20T21:33:36.356561] [WARN ] [MAIN ] TRTX: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:33:36.356671] [INFO ] [LOCAL ] TRTX: Starting deletion of primary data. [2025-10-20T21:33:36.356755] [DEBUG] [LOCAL ] TRTX: No primary data file found to delete. [2025-10-20T21:33:36.356793] [INFO ] [LOCAL ] TRTX: Starting deletion of derivative data. [2025-10-20T21:33:36.356942] [DEBUG] [LOCAL ] TRTX: Deleted derivative data file. [2025-10-20T21:33:36.357028] [INFO ] [MAIN ] Update Progress: 2128/2367 symbols completed. TRTX took 6.6s. Estimated remaining: 30m 43s. Rate: 7.8 symbols/min [2025-10-20T21:33:36.522709] [INFO ] [MAIN ] TRV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:36.575407] [DEBUG] [LOCAL ] TRV: Loaded 48191 existing records from primary storage. [2025-10-20T21:33:36.575760] [INFO ] [MAIN ] TRV: Loaded 48191 existing records. [2025-10-20T21:33:36.749157] [WARN ] [MAIN ] TRV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:36.749434] [INFO ] [MAIN ] TRV: Wiping existing primary data before full refresh. [2025-10-20T21:33:36.749630] [INFO ] [LOCAL ] TRV: Starting deletion of primary data. [2025-10-20T21:33:36.750092] [DEBUG] [LOCAL ] TRV: Deleted primary data file. [2025-10-20T21:33:36.750148] [INFO ] [MAIN ] TRV: Starting full primary refresh. [2025-10-20T21:33:37.057912] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-02. [2025-10-20T21:33:37.146852] [INFO ] [REMOTE ] TRV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:37.185663] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:33:37.851377] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-04. [2025-10-20T21:33:38.651390] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-08. [2025-10-20T21:33:38.733895] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T21:33:39.451354] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-03. [2025-10-20T21:33:39.643847] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T21:33:39.980076] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-08 (1.33 seconds). [2025-10-20T21:33:40.251413] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-09. [2025-10-20T21:33:41.051376] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-05. [2025-10-20T21:33:41.135045] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T21:33:41.851380] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-04. [2025-10-20T21:33:42.049091] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T21:33:42.449291] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T21:33:42.651434] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-06. [2025-10-20T21:33:43.451463] [INFO ] [REMOTE ] TROX: Fetching primary data for 2025-10. [2025-10-20T21:33:43.574427] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T21:33:44.251349] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-05. [2025-10-20T21:33:44.346367] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T21:33:44.758464] [INFO ] [REMOTE ] TROX: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T21:33:44.766263] [INFO ] [REMOTE ] TROX: Full fetch complete (74047 records). [2025-10-20T21:33:44.766406] [DEBUG] [VALIDATOR ] TROX: Validating primary data. [2025-10-20T21:33:45.058053] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-01. [2025-10-20T21:33:45.158720] [DEBUG] [VALIDATOR ] TROX: All expected trading days are present. [2025-10-20T21:33:45.158812] [DEBUG] [VALIDATOR ] TROX: Primary data validation passed. [2025-10-20T21:33:45.159938] [INFO ] [MAIN ] TROX: Fetched primary data is valid. [2025-10-20T21:33:45.159999] [DEBUG] [LOCAL ] TROX: Saving primary data. [2025-10-20T21:33:45.161355] [DEBUG] [LOCAL ] TROX: Trimmed 74047 records to 52248 (keeping 200 days of data) [2025-10-20T21:33:45.459884] [DEBUG] [LOCAL ] TROX: Primary data saved (52248 records). [2025-10-20T21:33:45.460054] [INFO ] [MAIN ] TROX: Full history saved. [2025-10-20T21:33:45.460344] [INFO ] [MAIN ] Update Progress: 2129/2367 symbols completed. TROX took 31.6s. Estimated remaining: 30m 36s. Rate: 7.8 symbols/min [2025-10-20T21:33:45.629111] [INFO ] [MAIN ] TS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:45.680504] [DEBUG] [LOCAL ] TS: Loaded 52431 existing records from primary storage. [2025-10-20T21:33:45.680861] [INFO ] [MAIN ] TS: Loaded 52431 existing records. [2025-10-20T21:33:45.854813] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-07. [2025-10-20T21:33:45.857951] [WARN ] [MAIN ] TS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:45.859117] [INFO ] [MAIN ] TS: Wiping existing primary data before full refresh. [2025-10-20T21:33:45.859180] [INFO ] [LOCAL ] TS: Starting deletion of primary data. [2025-10-20T21:33:45.859707] [DEBUG] [LOCAL ] TS: Deleted primary data file. [2025-10-20T21:33:45.859762] [INFO ] [MAIN ] TS: Starting full primary refresh. [2025-10-20T21:33:46.182184] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:33:46.271833] [INFO ] [REMOTE ] TS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:46.651361] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-06. [2025-10-20T21:33:46.843026] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T21:33:47.458431] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-01. [2025-10-20T21:33:47.463965] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:33:48.256518] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-08. [2025-10-20T21:33:48.275172] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:33:49.051358] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-02. [2025-10-20T21:33:49.230082] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T21:33:49.851364] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-02. [2025-10-20T21:33:49.969147] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T21:33:50.651360] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-09. [2025-10-20T21:33:50.809500] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:33:51.451382] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-03. [2025-10-20T21:33:51.612923] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:33:52.251408] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-07. [2025-10-20T21:33:52.395150] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T21:33:53.051384] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-03. [2025-10-20T21:33:53.253925] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T21:33:53.851367] [INFO ] [REMOTE ] TRP: Fetching primary data for 2025-10. [2025-10-20T21:33:53.992556] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T21:33:54.651352] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-04. [2025-10-20T21:33:54.769762] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T21:33:55.048230] [INFO ] [REMOTE ] TRP: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T21:33:55.056029] [INFO ] [REMOTE ] TRP: Full fetch complete (75764 records). [2025-10-20T21:33:55.056205] [DEBUG] [VALIDATOR ] TRP: Validating primary data. [2025-10-20T21:33:55.447778] [DEBUG] [VALIDATOR ] TRP: All expected trading days are present. [2025-10-20T21:33:55.447862] [DEBUG] [VALIDATOR ] TRP: Primary data validation passed. [2025-10-20T21:33:55.448971] [INFO ] [MAIN ] TRP: Fetched primary data is valid. [2025-10-20T21:33:55.449084] [DEBUG] [LOCAL ] TRP: Saving primary data. [2025-10-20T21:33:55.450449] [DEBUG] [LOCAL ] TRP: Trimmed 75764 records to 52150 (keeping 200 days of data) [2025-10-20T21:33:55.451383] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-04. [2025-10-20T21:33:55.770783] [DEBUG] [LOCAL ] TRP: Primary data saved (52150 records). [2025-10-20T21:33:55.770956] [INFO ] [MAIN ] TRP: Full history saved. [2025-10-20T21:33:55.771041] [INFO ] [MAIN ] Update Progress: 2130/2367 symbols completed. TRP took 33.7s. Estimated remaining: 30m 28s. Rate: 7.8 symbols/min [2025-10-20T21:33:55.941223] [INFO ] [MAIN ] TSE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:33:55.969530] [DEBUG] [LOCAL ] TSE: Loaded 28113 existing records from primary storage. [2025-10-20T21:33:55.969878] [INFO ] [MAIN ] TSE: Loaded 28113 existing records. [2025-10-20T21:33:56.152306] [WARN ] [MAIN ] TSE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:33:56.152526] [INFO ] [MAIN ] TSE: Wiping existing primary data before full refresh. [2025-10-20T21:33:56.152572] [INFO ] [LOCAL ] TSE: Starting deletion of primary data. [2025-10-20T21:33:56.152813] [DEBUG] [LOCAL ] TSE: Deleted primary data file. [2025-10-20T21:33:56.152870] [INFO ] [MAIN ] TSE: Starting full primary refresh. [2025-10-20T21:33:56.252428] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-08. [2025-10-20T21:33:56.512818] [INFO ] [REMOTE ] TSE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:33:56.637415] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-04 (1.99 seconds). [2025-10-20T21:33:57.051459] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-01. [2025-10-20T21:33:57.340392] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T21:33:57.820532] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T21:33:57.851334] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-09. [2025-10-20T21:33:58.225994] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:33:58.651388] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-05. [2025-10-20T21:33:59.451381] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-02. [2025-10-20T21:33:59.496040] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T21:34:00.251501] [INFO ] [REMOTE ] TRU: Fetching primary data for 2025-10. [2025-10-20T21:34:00.394630] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T21:34:00.454621] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T21:34:01.051431] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-05. [2025-10-20T21:34:01.851360] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-06. [2025-10-20T21:34:02.181348] [INFO ] [REMOTE ] TRU: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T21:34:02.189555] [INFO ] [REMOTE ] TRU: Full fetch complete (74104 records). [2025-10-20T21:34:02.189768] [DEBUG] [VALIDATOR ] TRU: Validating primary data. [2025-10-20T21:34:02.606190] [DEBUG] [VALIDATOR ] TRU: All expected trading days are present. [2025-10-20T21:34:02.606338] [DEBUG] [VALIDATOR ] TRU: Primary data validation passed. [2025-10-20T21:34:02.607451] [INFO ] [MAIN ] TRU: Fetched primary data is valid. [2025-10-20T21:34:02.607631] [DEBUG] [LOCAL ] TRU: Saving primary data. [2025-10-20T21:34:02.609094] [DEBUG] [LOCAL ] TRU: Trimmed 74104 records to 51460 (keeping 200 days of data) [2025-10-20T21:34:02.662167] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-03. [2025-10-20T21:34:02.949481] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-05 (1.90 seconds). [2025-10-20T21:34:02.978805] [DEBUG] [LOCAL ] TRU: Primary data saved (51460 records). [2025-10-20T21:34:02.978944] [INFO ] [MAIN ] TRU: Full history saved. [2025-10-20T21:34:02.979021] [INFO ] [MAIN ] Update Progress: 2131/2367 symbols completed. TRU took 31.7s. Estimated remaining: 30m 21s. Rate: 7.8 symbols/min [2025-10-20T21:34:03.146167] [INFO ] [MAIN ] TSI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:03.155458] [DEBUG] [LOCAL ] TSI: Loaded 8285 existing records from primary storage. [2025-10-20T21:34:03.155679] [INFO ] [MAIN ] TSI: Loaded 8285 existing records. [2025-10-20T21:34:03.331609] [WARN ] [MAIN ] TSI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:03.331826] [INFO ] [MAIN ] TSI: Wiping existing primary data before full refresh. [2025-10-20T21:34:03.331908] [INFO ] [LOCAL ] TSI: Starting deletion of primary data. [2025-10-20T21:34:03.332065] [DEBUG] [LOCAL ] TSI: Deleted primary data file. [2025-10-20T21:34:03.332102] [INFO ] [MAIN ] TSI: Starting full primary refresh. [2025-10-20T21:34:03.453137] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-06. [2025-10-20T21:34:03.718460] [INFO ] [REMOTE ] TSI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:03.723415] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:34:03.750837] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-03 (1.09 seconds). [2025-10-20T21:34:04.251415] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-07. [2025-10-20T21:34:05.051367] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-04. [2025-10-20T21:34:05.191277] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T21:34:05.851395] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-07. [2025-10-20T21:34:06.027733] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-04 (0.98 seconds). [2025-10-20T21:34:06.062325] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-07 (1.81 seconds). [2025-10-20T21:34:06.651428] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-05. [2025-10-20T21:34:07.451459] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-08. [2025-10-20T21:34:07.584917] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:34:07.671752] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T21:34:08.251429] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-08. [2025-10-20T21:34:09.051435] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-06. [2025-10-20T21:34:09.276487] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T21:34:09.851383] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-09. [2025-10-20T21:34:09.971607] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T21:34:10.000429] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-06 (0.95 seconds). [2025-10-20T21:34:10.651356] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-09. [2025-10-20T21:34:11.456534] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-01. [2025-10-20T21:34:11.472291] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:34:12.101767] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T21:34:12.251406] [INFO ] [REMOTE ] TRV: Fetching primary data for 2025-10. [2025-10-20T21:34:12.432150] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-09 (1.78 seconds). [2025-10-20T21:34:13.051443] [INFO ] [REMOTE ] TS: Fetching primary data for 2025-10. [2025-10-20T21:34:13.851429] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-07. [2025-10-20T21:34:13.950396] [INFO ] [REMOTE ] TRV: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T21:34:13.958201] [INFO ] [REMOTE ] TRV: Full fetch complete (69960 records). [2025-10-20T21:34:13.958408] [DEBUG] [VALIDATOR ] TRV: Validating primary data. [2025-10-20T21:34:14.392554] [DEBUG] [VALIDATOR ] TRV: All expected trading days are present. [2025-10-20T21:34:14.392676] [DEBUG] [VALIDATOR ] TRV: Primary data validation passed. [2025-10-20T21:34:14.393791] [INFO ] [MAIN ] TRV: Fetched primary data is valid. [2025-10-20T21:34:14.393874] [DEBUG] [LOCAL ] TRV: Saving primary data. [2025-10-20T21:34:14.395297] [DEBUG] [LOCAL ] TRV: Trimmed 69960 records to 47484 (keeping 200 days of data) [2025-10-20T21:34:14.664261] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-02. [2025-10-20T21:34:14.692956] [DEBUG] [LOCAL ] TRV: Primary data saved (47484 records). [2025-10-20T21:34:14.693787] [INFO ] [MAIN ] TRV: Full history saved. [2025-10-20T21:34:14.693913] [INFO ] [MAIN ] Update Progress: 2132/2367 symbols completed. TRV took 38.3s. Estimated remaining: 30m 13s. Rate: 7.8 symbols/min [2025-10-20T21:34:14.795699] [INFO ] [REMOTE ] TS: Incremental fetch complete for 2025-10 (1.74 seconds). [2025-10-20T21:34:14.843022] [INFO ] [REMOTE ] TS: Full fetch complete (74833 records). [2025-10-20T21:34:14.853636] [DEBUG] [VALIDATOR ] TS: Validating primary data. [2025-10-20T21:34:14.957232] [INFO ] [MAIN ] TSLX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:15.304276] [DEBUG] [LOCAL ] TSLX: Loaded 36454 existing records from primary storage. [2025-10-20T21:34:15.330834] [INFO ] [MAIN ] TSLX: Loaded 36454 existing records. [2025-10-20T21:34:15.437068] [DEBUG] [VALIDATOR ] TS: All expected trading days are present. [2025-10-20T21:34:15.451944] [DEBUG] [VALIDATOR ] TS: Primary data validation passed. [2025-10-20T21:34:15.469816] [INFO ] [MAIN ] TS: Fetched primary data is valid. [2025-10-20T21:34:15.475115] [DEBUG] [LOCAL ] TS: Saving primary data. [2025-10-20T21:34:15.573786] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:34:15.606046] [DEBUG] [LOCAL ] TS: Trimmed 74833 records to 51678 (keeping 200 days of data) [2025-10-20T21:34:15.612202] [WARN ] [MAIN ] TSLX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:15.612955] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-08. [2025-10-20T21:34:15.612715] [INFO ] [MAIN ] TSLX: Wiping existing primary data before full refresh. [2025-10-20T21:34:15.613110] [INFO ] [LOCAL ] TSLX: Starting deletion of primary data. [2025-10-20T21:34:15.614908] [DEBUG] [LOCAL ] TSLX: Deleted primary data file. [2025-10-20T21:34:15.614981] [INFO ] [MAIN ] TSLX: Starting full primary refresh. [2025-10-20T21:34:15.790131] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-02 (1.13 seconds). [2025-10-20T21:34:16.287045] [INFO ] [REMOTE ] TSLX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:16.282291] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-03. [2025-10-20T21:34:16.310327] [DEBUG] [LOCAL ] TS: Primary data saved (51678 records). [2025-10-20T21:34:16.310441] [INFO ] [MAIN ] TS: Full history saved. [2025-10-20T21:34:16.310770] [INFO ] [MAIN ] Update Progress: 2133/2367 symbols completed. TS took 30.9s. Estimated remaining: 30m 5s. Rate: 7.8 symbols/min [2025-10-20T21:34:16.474284] [INFO ] [MAIN ] TSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:16.530004] [DEBUG] [LOCAL ] TSM: Loaded 54599 existing records from primary storage. [2025-10-20T21:34:16.530332] [INFO ] [MAIN ] TSM: Loaded 54599 existing records. [2025-10-20T21:34:16.706206] [WARN ] [MAIN ] TSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:16.706502] [INFO ] [MAIN ] TSM: Wiping existing primary data before full refresh. [2025-10-20T21:34:16.706550] [INFO ] [LOCAL ] TSM: Starting deletion of primary data. [2025-10-20T21:34:16.706766] [DEBUG] [LOCAL ] TSM: Deleted primary data file. [2025-10-20T21:34:16.706821] [INFO ] [MAIN ] TSM: Starting full primary refresh. [2025-10-20T21:34:17.051632] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-01. [2025-10-20T21:34:17.116362] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T21:34:17.149207] [INFO ] [REMOTE ] TSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:17.211840] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T21:34:17.851411] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-04. [2025-10-20T21:34:18.302964] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T21:34:18.621125] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T21:34:18.651408] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-09. [2025-10-20T21:34:19.451434] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-01. [2025-10-20T21:34:19.719930] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T21:34:20.251377] [INFO ] [REMOTE ] TSE: Fetching primary data for 2025-10. [2025-10-20T21:34:21.051401] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-02. [2025-10-20T21:34:21.360751] [INFO ] [REMOTE ] TSE: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:34:21.366270] [INFO ] [REMOTE ] TSE: Full fetch complete (40755 records). [2025-10-20T21:34:21.366466] [DEBUG] [VALIDATOR ] TSE: Validating primary data. [2025-10-20T21:34:21.752837] [DEBUG] [VALIDATOR ] TSE: All expected trading days are present. [2025-10-20T21:34:21.752916] [DEBUG] [VALIDATOR ] TSE: Primary data validation passed. [2025-10-20T21:34:21.753700] [INFO ] [MAIN ] TSE: Fetched primary data is valid. [2025-10-20T21:34:21.753761] [DEBUG] [LOCAL ] TSE: Saving primary data. [2025-10-20T21:34:21.754533] [DEBUG] [LOCAL ] TSE: Trimmed 40755 records to 27846 (keeping 200 days of data) [2025-10-20T21:34:21.853762] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-05. [2025-10-20T21:34:21.912596] [DEBUG] [LOCAL ] TSE: Primary data saved (27846 records). [2025-10-20T21:34:21.912746] [INFO ] [MAIN ] TSE: Full history saved. [2025-10-20T21:34:21.912829] [INFO ] [MAIN ] Update Progress: 2134/2367 symbols completed. TSE took 26.1s. Estimated remaining: 29m 57s. Rate: 7.8 symbols/min [2025-10-20T21:34:22.097060] [INFO ] [MAIN ] TSN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:22.178938] [DEBUG] [LOCAL ] TSN: Loaded 53821 existing records from primary storage. [2025-10-20T21:34:22.182003] [INFO ] [MAIN ] TSN: Loaded 53821 existing records. [2025-10-20T21:34:22.181800] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-01 (2.73 seconds). [2025-10-20T21:34:22.263119] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-05 (0.41 seconds). [2025-10-20T21:34:22.383267] [WARN ] [MAIN ] TSN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:22.383549] [INFO ] [MAIN ] TSN: Wiping existing primary data before full refresh. [2025-10-20T21:34:22.383793] [INFO ] [LOCAL ] TSN: Starting deletion of primary data. [2025-10-20T21:34:22.390131] [DEBUG] [LOCAL ] TSN: Deleted primary data file. [2025-10-20T21:34:22.400199] [INFO ] [MAIN ] TSN: Starting full primary refresh. [2025-10-20T21:34:22.443384] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-02 (1.39 seconds). [2025-10-20T21:34:22.656537] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-02. [2025-10-20T21:34:22.760917] [INFO ] [REMOTE ] TSN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:23.451431] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-03. [2025-10-20T21:34:24.251393] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-06. [2025-10-20T21:34:24.926061] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T21:34:24.940912] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-06 (0.69 seconds). [2025-10-20T21:34:25.060045] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-04. [2025-10-20T21:34:25.066687] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-02 (2.41 seconds). [2025-10-20T21:34:25.851389] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-07. [2025-10-20T21:34:26.534063] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T21:34:26.638761] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T21:34:26.651348] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-08. [2025-10-20T21:34:27.451419] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-01. [2025-10-20T21:34:27.511979] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-08 (0.86 seconds). [2025-10-20T21:34:28.251385] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-03. [2025-10-20T21:34:29.051355] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-05. [2025-10-20T21:34:29.266752] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T21:34:29.851427] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-09. [2025-10-20T21:34:30.493870] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T21:34:30.630265] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-09 (0.78 seconds). [2025-10-20T21:34:30.651331] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-06. [2025-10-20T21:34:31.066790] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-03 (2.82 seconds). [2025-10-20T21:34:31.451377] [INFO ] [REMOTE ] TSI: Fetching primary data for 2025-10. [2025-10-20T21:34:32.094683] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T21:34:32.251387] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-04. [2025-10-20T21:34:32.395012] [INFO ] [REMOTE ] TSI: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T21:34:32.397621] [INFO ] [REMOTE ] TSI: Full fetch complete (13990 records). [2025-10-20T21:34:32.397747] [DEBUG] [VALIDATOR ] TSI: Validating primary data. [2025-10-20T21:34:32.775631] [DEBUG] [VALIDATOR ] TSI: All expected trading days are present. [2025-10-20T21:34:32.775766] [DEBUG] [VALIDATOR ] TSI: Primary data validation passed. [2025-10-20T21:34:32.776115] [INFO ] [MAIN ] TSI: Fetched primary data is valid. [2025-10-20T21:34:32.776154] [DEBUG] [LOCAL ] TSI: Saving primary data. [2025-10-20T21:34:32.776617] [DEBUG] [LOCAL ] TSI: Trimmed 13990 records to 8213 (keeping 200 days of data) [2025-10-20T21:34:32.822634] [DEBUG] [LOCAL ] TSI: Primary data saved (8213 records). [2025-10-20T21:34:32.822777] [INFO ] [MAIN ] TSI: Full history saved. [2025-10-20T21:34:32.822853] [INFO ] [MAIN ] Update Progress: 2135/2367 symbols completed. TSI took 29.8s. Estimated remaining: 29m 50s. Rate: 7.8 symbols/min [2025-10-20T21:34:33.006876] [INFO ] [MAIN ] TSQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:33.019714] [DEBUG] [LOCAL ] TSQ: Loaded 11828 existing records from primary storage. [2025-10-20T21:34:33.019963] [INFO ] [MAIN ] TSQ: Loaded 11828 existing records. [2025-10-20T21:34:33.056627] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-07. [2025-10-20T21:34:33.200594] [WARN ] [MAIN ] TSQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:33.200922] [INFO ] [MAIN ] TSQ: Wiping existing primary data before full refresh. [2025-10-20T21:34:33.200977] [INFO ] [LOCAL ] TSQ: Starting deletion of primary data. [2025-10-20T21:34:33.201172] [DEBUG] [LOCAL ] TSQ: Deleted primary data file. [2025-10-20T21:34:33.201229] [INFO ] [MAIN ] TSQ: Starting full primary refresh. [2025-10-20T21:34:33.561337] [INFO ] [REMOTE ] TSQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:33.851371] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-01. [2025-10-20T21:34:34.537882] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T21:34:34.555028] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T21:34:34.651383] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-02. [2025-10-20T21:34:34.997388] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-04 (2.75 seconds). [2025-10-20T21:34:35.410999] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T21:34:35.451334] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-03. [2025-10-20T21:34:36.248726] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-03 (0.80 seconds). [2025-10-20T21:34:36.252330] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-08. [2025-10-20T21:34:37.051375] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-05. [2025-10-20T21:34:37.685940] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-08 (1.43 seconds). [2025-10-20T21:34:37.851380] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-09. [2025-10-20T21:34:38.651373] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-02. [2025-10-20T21:34:39.208345] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T21:34:39.451423] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-04. [2025-10-20T21:34:40.026081] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-05 (2.97 seconds). [2025-10-20T21:34:40.166486] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T21:34:40.241170] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T21:34:40.251340] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-03. [2025-10-20T21:34:41.051461] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-06. [2025-10-20T21:34:41.851369] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-05. [2025-10-20T21:34:42.092134] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T21:34:42.651369] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-04. [2025-10-20T21:34:42.836869] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-05 (0.99 seconds). [2025-10-20T21:34:43.405234] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-06 (2.35 seconds). [2025-10-20T21:34:43.451431] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-07. [2025-10-20T21:34:44.251419] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-06. [2025-10-20T21:34:44.400662] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-04 (1.75 seconds). [2025-10-20T21:34:45.051344] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-05. [2025-10-20T21:34:45.206437] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-06 (0.96 seconds). [2025-10-20T21:34:45.851374] [INFO ] [REMOTE ] TSLX: Fetching primary data for 2025-10. [2025-10-20T21:34:46.210328] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-07 (2.76 seconds). [2025-10-20T21:34:46.651364] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-08. [2025-10-20T21:34:46.979256] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:34:47.045466] [INFO ] [REMOTE ] TSLX: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T21:34:47.051511] [INFO ] [REMOTE ] TSLX: Full fetch complete (50567 records). [2025-10-20T21:34:47.051615] [DEBUG] [VALIDATOR ] TSLX: Validating primary data. [2025-10-20T21:34:47.423522] [DEBUG] [VALIDATOR ] TSLX: All expected trading days are present. [2025-10-20T21:34:47.423603] [DEBUG] [VALIDATOR ] TSLX: Primary data validation passed. [2025-10-20T21:34:47.424490] [INFO ] [MAIN ] TSLX: Fetched primary data is valid. [2025-10-20T21:34:47.424552] [DEBUG] [LOCAL ] TSLX: Saving primary data. [2025-10-20T21:34:47.425565] [DEBUG] [LOCAL ] TSLX: Trimmed 50567 records to 35864 (keeping 200 days of data) [2025-10-20T21:34:47.477439] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-07. [2025-10-20T21:34:47.647023] [DEBUG] [LOCAL ] TSLX: Primary data saved (35864 records). [2025-10-20T21:34:47.647181] [INFO ] [MAIN ] TSLX: Full history saved. [2025-10-20T21:34:47.647267] [INFO ] [MAIN ] Update Progress: 2136/2367 symbols completed. TSLX took 33.0s. Estimated remaining: 29m 43s. Rate: 7.8 symbols/min [2025-10-20T21:34:47.833869] [INFO ] [MAIN ] TT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:47.879448] [DEBUG] [LOCAL ] TT: Loaded 47906 existing records from primary storage. [2025-10-20T21:34:47.879798] [INFO ] [MAIN ] TT: Loaded 47906 existing records. [2025-10-20T21:34:48.081784] [WARN ] [MAIN ] TT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:48.082051] [INFO ] [MAIN ] TT: Wiping existing primary data before full refresh. [2025-10-20T21:34:48.082097] [INFO ] [LOCAL ] TT: Starting deletion of primary data. [2025-10-20T21:34:48.082728] [DEBUG] [LOCAL ] TT: Deleted primary data file. [2025-10-20T21:34:48.082786] [INFO ] [MAIN ] TT: Starting full primary refresh. [2025-10-20T21:34:48.256501] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-06. [2025-10-20T21:34:48.387445] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-07 (0.91 seconds). [2025-10-20T21:34:48.451096] [INFO ] [REMOTE ] TT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:49.051333] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-01. [2025-10-20T21:34:49.457434] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-08 (2.81 seconds). [2025-10-20T21:34:49.851348] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-09. [2025-10-20T21:34:50.219797] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-06 (1.96 seconds). [2025-10-20T21:34:50.651370] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-07. [2025-10-20T21:34:50.770504] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T21:34:51.451430] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-02. [2025-10-20T21:34:52.251416] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-08. [2025-10-20T21:34:52.601346] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-09 (2.75 seconds). [2025-10-20T21:34:52.689572] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T21:34:53.021077] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T21:34:53.051336] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-09. [2025-10-20T21:34:53.182946] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T21:34:53.765117] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T21:34:53.851425] [INFO ] [REMOTE ] TSM: Fetching primary data for 2025-10. [2025-10-20T21:34:54.651415] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-03. [2025-10-20T21:34:55.451512] [INFO ] [REMOTE ] TSQ: Fetching primary data for 2025-10. [2025-10-20T21:34:56.171450] [INFO ] [REMOTE ] TSQ: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T21:34:56.173584] [INFO ] [REMOTE ] TSQ: Full fetch complete (16323 records). [2025-10-20T21:34:56.173703] [DEBUG] [VALIDATOR ] TSQ: Validating primary data. [2025-10-20T21:34:56.268911] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-08. [2025-10-20T21:34:56.609749] [INFO ] [REMOTE ] TSM: Incremental fetch complete for 2025-10 (2.76 seconds). [2025-10-20T21:34:56.610140] [DEBUG] [VALIDATOR ] TSQ: All expected trading days are present. [2025-10-20T21:34:56.618114] [DEBUG] [VALIDATOR ] TSQ: Primary data validation passed. [2025-10-20T21:34:56.619535] [INFO ] [MAIN ] TSQ: Fetched primary data is valid. [2025-10-20T21:34:56.619783] [INFO ] [REMOTE ] TSM: Full fetch complete (77999 records). [2025-10-20T21:34:56.620031] [DEBUG] [LOCAL ] TSQ: Saving primary data. [2025-10-20T21:34:56.620277] [DEBUG] [VALIDATOR ] TSM: Validating primary data. [2025-10-20T21:34:56.634155] [DEBUG] [LOCAL ] TSQ: Trimmed 16323 records to 11603 (keeping 200 days of data) [2025-10-20T21:34:56.876400] [DEBUG] [LOCAL ] TSQ: Primary data saved (11603 records). [2025-10-20T21:34:56.914687] [INFO ] [MAIN ] TSQ: Full history saved. [2025-10-20T21:34:56.945722] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-03 (2.29 seconds). [2025-10-20T21:34:56.951478] [INFO ] [MAIN ] Update Progress: 2137/2367 symbols completed. TSQ took 24.1s. Estimated remaining: 29m 35s. Rate: 7.8 symbols/min [2025-10-20T21:34:57.061794] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-04. [2025-10-20T21:34:57.370165] [INFO ] [MAIN ] TTAM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:57.370683] [DEBUG] [VALIDATOR ] TSM: All expected trading days are present. [2025-10-20T21:34:57.370531] [DEBUG] [LOCAL ] TTAM: No existing data file found in primary storage. [2025-10-20T21:34:57.370869] [DEBUG] [VALIDATOR ] TSM: Primary data validation passed. [2025-10-20T21:34:57.372305] [INFO ] [MAIN ] TSM: Fetched primary data is valid. [2025-10-20T21:34:57.372402] [DEBUG] [LOCAL ] TSM: Saving primary data. [2025-10-20T21:34:57.370972] [WARN ] [MAIN ] TTAM: No existing data found, performing full refresh instead. [2025-10-20T21:34:57.372790] [INFO ] [MAIN ] TTAM: Wiping existing primary data before full refresh. [2025-10-20T21:34:57.372841] [INFO ] [LOCAL ] TTAM: Starting deletion of primary data. [2025-10-20T21:34:57.372887] [DEBUG] [LOCAL ] TTAM: No primary data file found to delete. [2025-10-20T21:34:57.372922] [INFO ] [MAIN ] TTAM: Starting full primary refresh. [2025-10-20T21:34:57.479837] [DEBUG] [LOCAL ] TSM: Trimmed 77999 records to 53819 (keeping 200 days of data) [2025-10-20T21:34:57.992557] [INFO ] [REMOTE ] TTAM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:57.993602] [INFO ] [REMOTE ] TTAM: Fetching primary data for 2025-01. [2025-10-20T21:34:58.141882] [DEBUG] [LOCAL ] TSM: Primary data saved (53819 records). [2025-10-20T21:34:58.142173] [INFO ] [MAIN ] TSM: Full history saved. [2025-10-20T21:34:58.142852] [INFO ] [MAIN ] Update Progress: 2138/2367 symbols completed. TSM took 41.8s. Estimated remaining: 29m 27s. Rate: 7.8 symbols/min [2025-10-20T21:34:58.204354] [DEBUG] [REMOTE ] TTAM: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:34:58.233339] [WARN ] [REMOTE ] TTAM: No data returned for month 2025-01 [2025-10-20T21:34:58.238773] [FATAL] [REMOTE ] TTAM: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:34:58.246070] [FATAL] [MAIN ] TTAM: Failed to fetch primary data. [2025-10-20T21:34:58.259460] [WARN ] [MAIN ] TTAM: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:34:58.259715] [INFO ] [LOCAL ] TTAM: Starting deletion of primary data. [2025-10-20T21:34:58.260043] [DEBUG] [LOCAL ] TTAM: No primary data file found to delete. [2025-10-20T21:34:58.260500] [INFO ] [LOCAL ] TTAM: Starting deletion of derivative data. [2025-10-20T21:34:58.271237] [DEBUG] [LOCAL ] TTAM: No derivative data file found to delete. [2025-10-20T21:34:58.281769] [INFO ] [MAIN ] Update Progress: 2139/2367 symbols completed. TTAM took 1.3s. Estimated remaining: 29m 18s. Rate: 7.8 symbols/min [2025-10-20T21:34:58.361458] [INFO ] [MAIN ] TTC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:58.561819] [INFO ] [MAIN ] TTE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:34:58.566271] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-08 (2.30 seconds). [2025-10-20T21:34:58.618494] [DEBUG] [LOCAL ] TTC: Loaded 42525 existing records from primary storage. [2025-10-20T21:34:58.619018] [INFO ] [MAIN ] TTC: Loaded 42525 existing records. [2025-10-20T21:34:58.651384] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-09. [2025-10-20T21:34:58.842893] [WARN ] [MAIN ] TTC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:58.844187] [INFO ] [MAIN ] TTC: Wiping existing primary data before full refresh. [2025-10-20T21:34:58.844405] [INFO ] [LOCAL ] TTC: Starting deletion of primary data. [2025-10-20T21:34:58.846793] [DEBUG] [LOCAL ] TTC: Deleted primary data file. [2025-10-20T21:34:58.846887] [INFO ] [MAIN ] TTC: Starting full primary refresh. [2025-10-20T21:34:58.846734] [DEBUG] [LOCAL ] TTE: Loaded 52350 existing records from primary storage. [2025-10-20T21:34:58.883738] [INFO ] [MAIN ] TTE: Loaded 52350 existing records. [2025-10-20T21:34:59.289537] [WARN ] [MAIN ] TTE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:34:59.318257] [INFO ] [MAIN ] TTE: Wiping existing primary data before full refresh. [2025-10-20T21:34:59.339266] [INFO ] [LOCAL ] TTE: Starting deletion of primary data. [2025-10-20T21:34:59.353515] [DEBUG] [LOCAL ] TTE: Deleted primary data file. [2025-10-20T21:34:59.353778] [INFO ] [MAIN ] TTE: Starting full primary refresh. [2025-10-20T21:34:59.519122] [INFO ] [REMOTE ] TTC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:34:59.552210] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-01. [2025-10-20T21:34:59.797226] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-04 (2.74 seconds). [2025-10-20T21:34:59.833341] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T21:34:59.924411] [INFO ] [REMOTE ] TTE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:00.251379] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-01. [2025-10-20T21:35:01.051453] [INFO ] [REMOTE ] TSN: Fetching primary data for 2025-10. [2025-10-20T21:35:01.477831] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-01 (1.93 seconds). [2025-10-20T21:35:01.851435] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-05. [2025-10-20T21:35:02.332346] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T21:35:02.409863] [INFO ] [REMOTE ] TSN: Incremental fetch complete for 2025-10 (1.36 seconds). [2025-10-20T21:35:02.417835] [INFO ] [REMOTE ] TSN: Full fetch complete (76659 records). [2025-10-20T21:35:02.418020] [DEBUG] [VALIDATOR ] TSN: Validating primary data. [2025-10-20T21:35:02.656527] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-02. [2025-10-20T21:35:02.816188] [DEBUG] [VALIDATOR ] TSN: All expected trading days are present. [2025-10-20T21:35:02.816275] [DEBUG] [VALIDATOR ] TSN: Primary data validation passed. [2025-10-20T21:35:02.817466] [INFO ] [MAIN ] TSN: Fetched primary data is valid. [2025-10-20T21:35:02.817530] [DEBUG] [LOCAL ] TSN: Saving primary data. [2025-10-20T21:35:02.819017] [DEBUG] [LOCAL ] TSN: Trimmed 76659 records to 53079 (keeping 200 days of data) [2025-10-20T21:35:03.153489] [DEBUG] [LOCAL ] TSN: Primary data saved (53079 records). [2025-10-20T21:35:03.153615] [INFO ] [MAIN ] TSN: Full history saved. [2025-10-20T21:35:03.153726] [INFO ] [MAIN ] Update Progress: 2140/2367 symbols completed. TSN took 41.2s. Estimated remaining: 29m 10s. Rate: 7.8 symbols/min [2025-10-20T21:35:03.324602] [INFO ] [MAIN ] TTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:03.366757] [DEBUG] [LOCAL ] TTI: Loaded 47129 existing records from primary storage. [2025-10-20T21:35:03.367074] [INFO ] [MAIN ] TTI: Loaded 47129 existing records. [2025-10-20T21:35:03.459829] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-02. [2025-10-20T21:35:03.552048] [WARN ] [MAIN ] TTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:03.552333] [INFO ] [MAIN ] TTI: Wiping existing primary data before full refresh. [2025-10-20T21:35:03.552399] [INFO ] [LOCAL ] TTI: Starting deletion of primary data. [2025-10-20T21:35:03.552621] [DEBUG] [LOCAL ] TTI: Deleted primary data file. [2025-10-20T21:35:03.552835] [INFO ] [MAIN ] TTI: Starting full primary refresh. [2025-10-20T21:35:03.863387] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-05 (2.01 seconds). [2025-10-20T21:35:03.957075] [INFO ] [REMOTE ] TTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:04.251415] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-01. [2025-10-20T21:35:04.703473] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T21:35:05.051360] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-06. [2025-10-20T21:35:05.223345] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-02 (1.76 seconds). [2025-10-20T21:35:05.816017] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T21:35:05.851342] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-02. [2025-10-20T21:35:06.651401] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-03. [2025-10-20T21:35:06.732932] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:35:07.451349] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-07. [2025-10-20T21:35:07.674891] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T21:35:08.177313] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T21:35:08.251378] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-03. [2025-10-20T21:35:09.051348] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-04. [2025-10-20T21:35:09.165273] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T21:35:09.851604] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-08. [2025-10-20T21:35:09.990673] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T21:35:10.651380] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-04. [2025-10-20T21:35:10.739347] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T21:35:11.218823] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-08 (1.37 seconds). [2025-10-20T21:35:11.451403] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-03. [2025-10-20T21:35:12.251418] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-09. [2025-10-20T21:35:12.477982] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T21:35:13.051408] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-05. [2025-10-20T21:35:13.386553] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T21:35:13.851352] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-05. [2025-10-20T21:35:13.960301] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T21:35:14.655109] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-04. [2025-10-20T21:35:14.657822] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T21:35:14.981344] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T21:35:15.451402] [INFO ] [REMOTE ] TT: Fetching primary data for 2025-10. [2025-10-20T21:35:16.251415] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-06. [2025-10-20T21:35:16.631534] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-04 (1.98 seconds). [2025-10-20T21:35:17.010298] [INFO ] [REMOTE ] TT: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T21:35:17.016989] [INFO ] [REMOTE ] TT: Full fetch complete (70039 records). [2025-10-20T21:35:17.017129] [DEBUG] [VALIDATOR ] TT: Validating primary data. [2025-10-20T21:35:17.056457] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-06. [2025-10-20T21:35:17.414007] [DEBUG] [VALIDATOR ] TT: All expected trading days are present. [2025-10-20T21:35:17.414159] [DEBUG] [VALIDATOR ] TT: Primary data validation passed. [2025-10-20T21:35:17.415262] [INFO ] [MAIN ] TT: Fetched primary data is valid. [2025-10-20T21:35:17.415339] [DEBUG] [LOCAL ] TT: Saving primary data. [2025-10-20T21:35:17.416812] [DEBUG] [LOCAL ] TT: Trimmed 70039 records to 47092 (keeping 200 days of data) [2025-10-20T21:35:17.663721] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-06 (1.41 seconds). [2025-10-20T21:35:17.759074] [DEBUG] [LOCAL ] TT: Primary data saved (47092 records). [2025-10-20T21:35:17.759216] [INFO ] [MAIN ] TT: Full history saved. [2025-10-20T21:35:17.759299] [INFO ] [MAIN ] Update Progress: 2141/2367 symbols completed. TT took 30.1s. Estimated remaining: 29m 3s. Rate: 7.8 symbols/min [2025-10-20T21:35:17.860110] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-05. [2025-10-20T21:35:17.933495] [INFO ] [MAIN ] TU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:17.983507] [DEBUG] [LOCAL ] TU: Loaded 50563 existing records from primary storage. [2025-10-20T21:35:17.983861] [INFO ] [MAIN ] TU: Loaded 50563 existing records. [2025-10-20T21:35:18.160347] [WARN ] [MAIN ] TU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:18.160607] [INFO ] [MAIN ] TU: Wiping existing primary data before full refresh. [2025-10-20T21:35:18.160685] [INFO ] [LOCAL ] TU: Starting deletion of primary data. [2025-10-20T21:35:18.160905] [DEBUG] [LOCAL ] TU: Deleted primary data file. [2025-10-20T21:35:18.160943] [INFO ] [MAIN ] TU: Starting full primary refresh. [2025-10-20T21:35:18.509572] [INFO ] [REMOTE ] TU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:18.651368] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-01. [2025-10-20T21:35:18.691195] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T21:35:19.451450] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-07. [2025-10-20T21:35:19.720277] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T21:35:19.957803] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T21:35:20.251357] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-07. [2025-10-20T21:35:21.056611] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-02. [2025-10-20T21:35:21.071673] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T21:35:21.770223] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:35:21.851401] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-08. [2025-10-20T21:35:22.651434] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-06. [2025-10-20T21:35:22.683955] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:35:23.330614] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T21:35:23.451422] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-03. [2025-10-20T21:35:24.251364] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-08. [2025-10-20T21:35:24.495451] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-06 (1.84 seconds). [2025-10-20T21:35:25.051385] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-09. [2025-10-20T21:35:25.170404] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T21:35:25.414579] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T21:35:25.851379] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-04. [2025-10-20T21:35:26.214335] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T21:35:26.651377] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-09. [2025-10-20T21:35:27.451363] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-07. [2025-10-20T21:35:27.592811] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T21:35:28.256689] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-05. [2025-10-20T21:35:28.268424] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:35:28.915743] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T21:35:29.051367] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-08. [2025-10-20T21:35:29.541667] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T21:35:29.851370] [INFO ] [REMOTE ] TTC: Fetching primary data for 2025-10. [2025-10-20T21:35:30.562918] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T21:35:30.651417] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-09. [2025-10-20T21:35:31.157211] [INFO ] [REMOTE ] TTC: Incremental fetch complete for 2025-10 (1.31 seconds). [2025-10-20T21:35:31.163423] [INFO ] [REMOTE ] TTC: Full fetch complete (62050 records). [2025-10-20T21:35:31.163567] [DEBUG] [VALIDATOR ] TTC: Validating primary data. [2025-10-20T21:35:31.457051] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-06. [2025-10-20T21:35:31.556007] [DEBUG] [VALIDATOR ] TTC: All expected trading days are present. [2025-10-20T21:35:31.556096] [DEBUG] [VALIDATOR ] TTC: Primary data validation passed. [2025-10-20T21:35:31.557325] [INFO ] [MAIN ] TTC: Fetched primary data is valid. [2025-10-20T21:35:31.557401] [DEBUG] [LOCAL ] TTC: Saving primary data. [2025-10-20T21:35:31.558871] [DEBUG] [LOCAL ] TTC: Trimmed 62050 records to 41748 (keeping 200 days of data) [2025-10-20T21:35:31.814271] [DEBUG] [LOCAL ] TTC: Primary data saved (41748 records). [2025-10-20T21:35:31.814411] [INFO ] [MAIN ] TTC: Full history saved. [2025-10-20T21:35:31.814776] [INFO ] [MAIN ] Update Progress: 2142/2367 symbols completed. TTC took 33.7s. Estimated remaining: 28m 56s. Rate: 7.8 symbols/min [2025-10-20T21:35:31.984347] [INFO ] [MAIN ] TUYA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:32.027244] [DEBUG] [LOCAL ] TUYA: Loaded 46657 existing records from primary storage. [2025-10-20T21:35:32.027536] [INFO ] [MAIN ] TUYA: Loaded 46657 existing records. [2025-10-20T21:35:32.215414] [WARN ] [MAIN ] TUYA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:32.215705] [INFO ] [MAIN ] TUYA: Wiping existing primary data before full refresh. [2025-10-20T21:35:32.215764] [INFO ] [LOCAL ] TUYA: Starting deletion of primary data. [2025-10-20T21:35:32.215941] [DEBUG] [LOCAL ] TUYA: Deleted primary data file. [2025-10-20T21:35:32.215989] [INFO ] [MAIN ] TUYA: Starting full primary refresh. [2025-10-20T21:35:32.258561] [INFO ] [REMOTE ] TTI: Fetching primary data for 2025-10. [2025-10-20T21:35:32.576535] [INFO ] [REMOTE ] TUYA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:32.605362] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-09 (1.95 seconds). [2025-10-20T21:35:33.051363] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-01. [2025-10-20T21:35:33.109193] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:35:33.851416] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-07. [2025-10-20T21:35:33.934486] [INFO ] [REMOTE ] TTI: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:35:33.941959] [INFO ] [REMOTE ] TTI: Full fetch complete (66336 records). [2025-10-20T21:35:33.942065] [DEBUG] [VALIDATOR ] TTI: Validating primary data. [2025-10-20T21:35:34.329988] [DEBUG] [VALIDATOR ] TTI: All expected trading days are present. [2025-10-20T21:35:34.330074] [DEBUG] [VALIDATOR ] TTI: Primary data validation passed. [2025-10-20T21:35:34.331114] [INFO ] [MAIN ] TTI: Fetched primary data is valid. [2025-10-20T21:35:34.331176] [DEBUG] [LOCAL ] TTI: Saving primary data. [2025-10-20T21:35:34.332536] [DEBUG] [LOCAL ] TTI: Trimmed 66336 records to 46578 (keeping 200 days of data) [2025-10-20T21:35:34.622520] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T21:35:34.636985] [DEBUG] [LOCAL ] TTI: Primary data saved (46578 records). [2025-10-20T21:35:34.637108] [INFO ] [MAIN ] TTI: Full history saved. [2025-10-20T21:35:34.637194] [INFO ] [MAIN ] Update Progress: 2143/2367 symbols completed. TTI took 31.5s. Estimated remaining: 28m 48s. Rate: 7.8 symbols/min [2025-10-20T21:35:34.657670] [INFO ] [REMOTE ] TTE: Fetching primary data for 2025-10. [2025-10-20T21:35:34.806727] [INFO ] [MAIN ] TV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:34.842735] [DEBUG] [LOCAL ] TV: Loaded 42121 existing records from primary storage. [2025-10-20T21:35:34.843032] [INFO ] [MAIN ] TV: Loaded 42121 existing records. [2025-10-20T21:35:35.014967] [WARN ] [MAIN ] TV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:35.015172] [INFO ] [MAIN ] TV: Wiping existing primary data before full refresh. [2025-10-20T21:35:35.015218] [INFO ] [LOCAL ] TV: Starting deletion of primary data. [2025-10-20T21:35:35.015458] [DEBUG] [LOCAL ] TV: Deleted primary data file. [2025-10-20T21:35:35.015498] [INFO ] [MAIN ] TV: Starting full primary refresh. [2025-10-20T21:35:35.368040] [INFO ] [REMOTE ] TV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:35.451332] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-01. [2025-10-20T21:35:35.593886] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T21:35:36.251341] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-02. [2025-10-20T21:35:36.530992] [INFO ] [REMOTE ] TTE: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T21:35:36.538604] [INFO ] [REMOTE ] TTE: Full fetch complete (75114 records). [2025-10-20T21:35:36.538795] [DEBUG] [VALIDATOR ] TTE: Validating primary data. [2025-10-20T21:35:36.940551] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T21:35:36.976616] [DEBUG] [VALIDATOR ] TTE: All expected trading days are present. [2025-10-20T21:35:36.977452] [DEBUG] [VALIDATOR ] TTE: Primary data validation passed. [2025-10-20T21:35:36.978575] [INFO ] [MAIN ] TTE: Fetched primary data is valid. [2025-10-20T21:35:36.978691] [DEBUG] [LOCAL ] TTE: Saving primary data. [2025-10-20T21:35:36.980022] [DEBUG] [LOCAL ] TTE: Trimmed 75114 records to 51594 (keeping 200 days of data) [2025-10-20T21:35:37.066130] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-02. [2025-10-20T21:35:37.289203] [DEBUG] [LOCAL ] TTE: Primary data saved (51594 records). [2025-10-20T21:35:37.289334] [INFO ] [MAIN ] TTE: Full history saved. [2025-10-20T21:35:37.289422] [INFO ] [MAIN ] Update Progress: 2144/2367 symbols completed. TTE took 39.0s. Estimated remaining: 28m 40s. Rate: 7.8 symbols/min [2025-10-20T21:35:37.475348] [INFO ] [MAIN ] TVC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:37.480774] [DEBUG] [LOCAL ] TVC: Loaded 3240 existing records from primary storage. [2025-10-20T21:35:37.480923] [INFO ] [MAIN ] TVC: Loaded 3240 existing records. [2025-10-20T21:35:37.653622] [WARN ] [MAIN ] TVC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:37.653827] [INFO ] [MAIN ] TVC: Wiping existing primary data before full refresh. [2025-10-20T21:35:37.653873] [INFO ] [LOCAL ] TVC: Starting deletion of primary data. [2025-10-20T21:35:37.654003] [DEBUG] [LOCAL ] TVC: Deleted primary data file. [2025-10-20T21:35:37.654039] [INFO ] [MAIN ] TVC: Starting full primary refresh. [2025-10-20T21:35:37.856540] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-08. [2025-10-20T21:35:38.004478] [INFO ] [REMOTE ] TVC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:38.100816] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:35:38.524675] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-02 (1.46 seconds). [2025-10-20T21:35:38.651340] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-03. [2025-10-20T21:35:39.451363] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-01. [2025-10-20T21:35:39.521824] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T21:35:40.056685] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T21:35:40.251405] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-02. [2025-10-20T21:35:40.326537] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T21:35:40.945080] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T21:35:41.051426] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-04. [2025-10-20T21:35:41.851386] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-03. [2025-10-20T21:35:42.651355] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-09. [2025-10-20T21:35:42.760708] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T21:35:43.423707] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T21:35:43.451333] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-04. [2025-10-20T21:35:44.099165] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T21:35:44.251345] [INFO ] [REMOTE ] TU: Fetching primary data for 2025-10. [2025-10-20T21:35:45.037758] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T21:35:45.051949] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-05. [2025-10-20T21:35:45.806202] [INFO ] [REMOTE ] TU: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T21:35:45.813620] [INFO ] [REMOTE ] TU: Full fetch complete (72887 records). [2025-10-20T21:35:45.813802] [DEBUG] [VALIDATOR ] TU: Validating primary data. [2025-10-20T21:35:45.856567] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-05. [2025-10-20T21:35:46.212251] [DEBUG] [VALIDATOR ] TU: All expected trading days are present. [2025-10-20T21:35:46.212357] [DEBUG] [VALIDATOR ] TU: Primary data validation passed. [2025-10-20T21:35:46.213539] [INFO ] [MAIN ] TU: Fetched primary data is valid. [2025-10-20T21:35:46.213602] [DEBUG] [LOCAL ] TU: Saving primary data. [2025-10-20T21:35:46.214980] [DEBUG] [LOCAL ] TU: Trimmed 72887 records to 49805 (keeping 200 days of data) [2025-10-20T21:35:46.512228] [DEBUG] [LOCAL ] TU: Primary data saved (49805 records). [2025-10-20T21:35:46.512437] [INFO ] [MAIN ] TU: Full history saved. [2025-10-20T21:35:46.512526] [INFO ] [MAIN ] Update Progress: 2145/2367 symbols completed. TU took 28.8s. Estimated remaining: 28m 32s. Rate: 7.8 symbols/min [2025-10-20T21:35:46.656472] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-03. [2025-10-20T21:35:46.683418] [INFO ] [MAIN ] TVE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:46.688469] [DEBUG] [LOCAL ] TVE: Loaded 2658 existing records from primary storage. [2025-10-20T21:35:46.688616] [INFO ] [MAIN ] TVE: Loaded 2658 existing records. [2025-10-20T21:35:46.869697] [WARN ] [MAIN ] TVE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:46.870082] [INFO ] [MAIN ] TVE: Wiping existing primary data before full refresh. [2025-10-20T21:35:46.870145] [INFO ] [LOCAL ] TVE: Starting deletion of primary data. [2025-10-20T21:35:46.880408] [DEBUG] [LOCAL ] TVE: Deleted primary data file. [2025-10-20T21:35:46.893786] [INFO ] [MAIN ] TVE: Starting full primary refresh. [2025-10-20T21:35:46.979158] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:35:47.268419] [INFO ] [REMOTE ] TVE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:47.309870] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T21:35:47.451409] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-04. [2025-10-20T21:35:47.520324] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T21:35:47.875292] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-04 (0.42 seconds). [2025-10-20T21:35:48.251364] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-06. [2025-10-20T21:35:49.051471] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-01. [2025-10-20T21:35:49.534919] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-01 (0.48 seconds). [2025-10-20T21:35:49.764571] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:35:49.851367] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-07. [2025-10-20T21:35:50.651432] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-05. [2025-10-20T21:35:51.201011] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T21:35:51.446869] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T21:35:51.455513] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-06. [2025-10-20T21:35:52.038119] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T21:35:52.251420] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-07. [2025-10-20T21:35:52.729401] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-07 (0.48 seconds). [2025-10-20T21:35:53.051426] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-08. [2025-10-20T21:35:53.528716] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T21:35:53.851410] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-09. [2025-10-20T21:35:54.319586] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-09 (0.47 seconds). [2025-10-20T21:35:54.651417] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-06. [2025-10-20T21:35:55.451433] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-08. [2025-10-20T21:35:56.251383] [INFO ] [REMOTE ] TVC: Fetching primary data for 2025-10. [2025-10-20T21:35:56.299210] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:35:56.727619] [INFO ] [REMOTE ] TVC: Incremental fetch complete for 2025-10 (0.48 seconds). [2025-10-20T21:35:56.728597] [INFO ] [REMOTE ] TVC: Full fetch complete (5106 records). [2025-10-20T21:35:56.728688] [DEBUG] [VALIDATOR ] TVC: Validating primary data. [2025-10-20T21:35:57.058013] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-02. [2025-10-20T21:35:57.089503] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:35:57.130316] [DEBUG] [VALIDATOR ] TVC: All expected trading days are present. [2025-10-20T21:35:57.130418] [DEBUG] [VALIDATOR ] TVC: Primary data validation passed. [2025-10-20T21:35:57.131038] [INFO ] [MAIN ] TVC: Fetched primary data is valid. [2025-10-20T21:35:57.131096] [DEBUG] [LOCAL ] TVC: Saving primary data. [2025-10-20T21:35:57.131524] [DEBUG] [LOCAL ] TVC: Trimmed 5106 records to 3148 (keeping 200 days of data) [2025-10-20T21:35:57.150028] [DEBUG] [LOCAL ] TVC: Primary data saved (3148 records). [2025-10-20T21:35:57.150086] [INFO ] [MAIN ] TVC: Full history saved. [2025-10-20T21:35:57.150152] [INFO ] [MAIN ] Update Progress: 2146/2367 symbols completed. TVC took 19.9s. Estimated remaining: 28m 25s. Rate: 7.8 symbols/min [2025-10-20T21:35:57.314039] [INFO ] [MAIN ] TWI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:35:57.346026] [DEBUG] [LOCAL ] TWI: Loaded 34457 existing records from primary storage. [2025-10-20T21:35:57.346386] [INFO ] [MAIN ] TWI: Loaded 34457 existing records. [2025-10-20T21:35:57.525033] [WARN ] [MAIN ] TWI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:35:57.525251] [INFO ] [MAIN ] TWI: Wiping existing primary data before full refresh. [2025-10-20T21:35:57.525299] [INFO ] [LOCAL ] TWI: Starting deletion of primary data. [2025-10-20T21:35:57.525518] [DEBUG] [LOCAL ] TWI: Deleted primary data file. [2025-10-20T21:35:57.525555] [INFO ] [MAIN ] TWI: Starting full primary refresh. [2025-10-20T21:35:57.768873] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-02 (0.71 seconds). [2025-10-20T21:35:57.856495] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-03. [2025-10-20T21:35:57.887076] [INFO ] [REMOTE ] TWI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:35:58.523470] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T21:35:58.651376] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-01. [2025-10-20T21:35:59.451443] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-07. [2025-10-20T21:36:00.037256] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-01 (1.39 seconds). [2025-10-20T21:36:00.251414] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-09. [2025-10-20T21:36:01.051402] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-02. [2025-10-20T21:36:01.507796] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-07 (2.06 seconds). [2025-10-20T21:36:01.779016] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:36:01.851372] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-08. [2025-10-20T21:36:02.536772] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T21:36:02.651340] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-04. [2025-10-20T21:36:03.129269] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-04 (0.48 seconds). [2025-10-20T21:36:03.451434] [INFO ] [REMOTE ] TV: Fetching primary data for 2025-10. [2025-10-20T21:36:03.558244] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T21:36:04.251477] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-03. [2025-10-20T21:36:04.511350] [INFO ] [REMOTE ] TV: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:36:04.516467] [INFO ] [REMOTE ] TV: Full fetch complete (61100 records). [2025-10-20T21:36:04.516601] [DEBUG] [VALIDATOR ] TV: Validating primary data. [2025-10-20T21:36:04.910261] [DEBUG] [VALIDATOR ] TV: All expected trading days are present. [2025-10-20T21:36:04.910353] [DEBUG] [VALIDATOR ] TV: Primary data validation passed. [2025-10-20T21:36:04.911318] [INFO ] [MAIN ] TV: Fetched primary data is valid. [2025-10-20T21:36:04.911383] [DEBUG] [LOCAL ] TV: Saving primary data. [2025-10-20T21:36:04.912628] [DEBUG] [LOCAL ] TV: Trimmed 61100 records to 41489 (keeping 200 days of data) [2025-10-20T21:36:05.067142] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-05. [2025-10-20T21:36:05.145797] [DEBUG] [LOCAL ] TV: Primary data saved (41489 records). [2025-10-20T21:36:05.145925] [INFO ] [MAIN ] TV: Full history saved. [2025-10-20T21:36:05.146003] [INFO ] [MAIN ] Update Progress: 2147/2367 symbols completed. TV took 30.5s. Estimated remaining: 28m 17s. Rate: 7.8 symbols/min [2025-10-20T21:36:05.316548] [INFO ] [MAIN ] TWLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:05.364839] [DEBUG] [LOCAL ] TWLO: Loaded 53590 existing records from primary storage. [2025-10-20T21:36:05.365133] [INFO ] [MAIN ] TWLO: Loaded 53590 existing records. [2025-10-20T21:36:05.555321] [WARN ] [MAIN ] TWLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:05.555570] [INFO ] [MAIN ] TWLO: Wiping existing primary data before full refresh. [2025-10-20T21:36:05.555695] [INFO ] [LOCAL ] TWLO: Starting deletion of primary data. [2025-10-20T21:36:05.556215] [DEBUG] [LOCAL ] TWLO: Deleted primary data file. [2025-10-20T21:36:05.556273] [INFO ] [MAIN ] TWLO: Starting full primary refresh. [2025-10-20T21:36:05.624832] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-05 (0.56 seconds). [2025-10-20T21:36:05.856555] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-06. [2025-10-20T21:36:05.882506] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:36:05.968999] [INFO ] [REMOTE ] TWLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:06.485337] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T21:36:06.651400] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-04. [2025-10-20T21:36:07.451426] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-01. [2025-10-20T21:36:08.153151] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:36:08.251341] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-05. [2025-10-20T21:36:09.051444] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-09. [2025-10-20T21:36:09.531201] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-01 (2.08 seconds). [2025-10-20T21:36:09.632019] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-05 (1.38 seconds). [2025-10-20T21:36:09.851358] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-02. [2025-10-20T21:36:10.656490] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-06. [2025-10-20T21:36:10.671509] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-09 (1.62 seconds). [2025-10-20T21:36:11.451356] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-07. [2025-10-20T21:36:11.867737] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T21:36:11.900237] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-07 (0.45 seconds). [2025-10-20T21:36:12.144585] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T21:36:12.251457] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-07. [2025-10-20T21:36:13.051381] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-03. [2025-10-20T21:36:13.699937] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T21:36:13.851368] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-08. [2025-10-20T21:36:14.651422] [INFO ] [REMOTE ] TUYA: Fetching primary data for 2025-10. [2025-10-20T21:36:15.074532] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-03 (2.02 seconds). [2025-10-20T21:36:15.104271] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T21:36:15.451425] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-09. [2025-10-20T21:36:16.206025] [INFO ] [REMOTE ] TUYA: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T21:36:16.212680] [INFO ] [REMOTE ] TUYA: Full fetch complete (66708 records). [2025-10-20T21:36:16.212826] [DEBUG] [VALIDATOR ] TUYA: Validating primary data. [2025-10-20T21:36:16.256749] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-08. [2025-10-20T21:36:16.624732] [DEBUG] [VALIDATOR ] TUYA: All expected trading days are present. [2025-10-20T21:36:16.624875] [DEBUG] [VALIDATOR ] TUYA: Primary data validation passed. [2025-10-20T21:36:16.626788] [INFO ] [MAIN ] TUYA: Fetched primary data is valid. [2025-10-20T21:36:16.627558] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T21:36:16.627711] [DEBUG] [LOCAL ] TUYA: Saving primary data. [2025-10-20T21:36:16.636072] [DEBUG] [LOCAL ] TUYA: Trimmed 66708 records to 45927 (keeping 200 days of data) [2025-10-20T21:36:16.804108] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-08 (0.55 seconds). [2025-10-20T21:36:16.893716] [DEBUG] [LOCAL ] TUYA: Primary data saved (45927 records). [2025-10-20T21:36:16.893844] [INFO ] [MAIN ] TUYA: Full history saved. [2025-10-20T21:36:16.894227] [INFO ] [MAIN ] Update Progress: 2148/2367 symbols completed. TUYA took 45.1s. Estimated remaining: 28m 10s. Rate: 7.8 symbols/min [2025-10-20T21:36:17.062507] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-09. [2025-10-20T21:36:17.066420] [INFO ] [MAIN ] TWN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:17.074045] [DEBUG] [LOCAL ] TWN: Loaded 3361 existing records from primary storage. [2025-10-20T21:36:17.074246] [INFO ] [MAIN ] TWN: Loaded 3361 existing records. [2025-10-20T21:36:17.250974] [WARN ] [MAIN ] TWN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:17.251216] [INFO ] [MAIN ] TWN: Wiping existing primary data before full refresh. [2025-10-20T21:36:17.251262] [INFO ] [LOCAL ] TWN: Starting deletion of primary data. [2025-10-20T21:36:17.251388] [DEBUG] [LOCAL ] TWN: Deleted primary data file. [2025-10-20T21:36:17.251424] [INFO ] [MAIN ] TWN: Starting full primary refresh. [2025-10-20T21:36:17.601613] [INFO ] [REMOTE ] TWN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:17.645175] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T21:36:17.851379] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-01. [2025-10-20T21:36:18.416474] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-01 (0.57 seconds). [2025-10-20T21:36:18.651416] [INFO ] [REMOTE ] TVE: Fetching primary data for 2025-10. [2025-10-20T21:36:19.034825] [INFO ] [REMOTE ] TVE: Incremental fetch complete for 2025-10 (0.38 seconds). [2025-10-20T21:36:19.035706] [INFO ] [REMOTE ] TVE: Full fetch complete (4229 records). [2025-10-20T21:36:19.035774] [DEBUG] [VALIDATOR ] TVE: Validating primary data. [2025-10-20T21:36:19.381530] [DEBUG] [VALIDATOR ] TVE: All expected trading days are present. [2025-10-20T21:36:19.381607] [DEBUG] [VALIDATOR ] TVE: Primary data validation passed. [2025-10-20T21:36:19.381748] [INFO ] [MAIN ] TVE: Fetched primary data is valid. [2025-10-20T21:36:19.381812] [DEBUG] [LOCAL ] TVE: Saving primary data. [2025-10-20T21:36:19.382141] [DEBUG] [LOCAL ] TVE: Trimmed 4229 records to 2584 (keeping 200 days of data) [2025-10-20T21:36:19.397136] [DEBUG] [LOCAL ] TVE: Primary data saved (2584 records). [2025-10-20T21:36:19.397193] [INFO ] [MAIN ] TVE: Full history saved. [2025-10-20T21:36:19.397257] [INFO ] [MAIN ] Update Progress: 2149/2367 symbols completed. TVE took 32.9s. Estimated remaining: 28m 1s. Rate: 7.8 symbols/min [2025-10-20T21:36:19.456618] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-02. [2025-10-20T21:36:19.562441] [INFO ] [MAIN ] TWO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:19.604814] [DEBUG] [LOCAL ] TWO: Loaded 49363 existing records from primary storage. [2025-10-20T21:36:19.605141] [INFO ] [MAIN ] TWO: Loaded 49363 existing records. [2025-10-20T21:36:19.780573] [WARN ] [MAIN ] TWO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:19.780829] [INFO ] [MAIN ] TWO: Wiping existing primary data before full refresh. [2025-10-20T21:36:19.780876] [INFO ] [LOCAL ] TWO: Starting deletion of primary data. [2025-10-20T21:36:19.781094] [DEBUG] [LOCAL ] TWO: Deleted primary data file. [2025-10-20T21:36:19.781149] [INFO ] [MAIN ] TWO: Starting full primary refresh. [2025-10-20T21:36:20.135667] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-02 (0.68 seconds). [2025-10-20T21:36:20.141295] [INFO ] [REMOTE ] TWO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:20.251361] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-01. [2025-10-20T21:36:21.051373] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-03. [2025-10-20T21:36:21.673881] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T21:36:21.851343] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-04. [2025-10-20T21:36:21.922072] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:36:22.651373] [INFO ] [REMOTE ] TWI: Fetching primary data for 2025-10. [2025-10-20T21:36:23.451353] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-02. [2025-10-20T21:36:23.799784] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-04 (1.95 seconds). [2025-10-20T21:36:24.062993] [INFO ] [REMOTE ] TWI: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T21:36:24.068622] [INFO ] [REMOTE ] TWI: Full fetch complete (52166 records). [2025-10-20T21:36:24.068816] [DEBUG] [VALIDATOR ] TWI: Validating primary data. [2025-10-20T21:36:24.257308] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-05. [2025-10-20T21:36:24.471402] [DEBUG] [VALIDATOR ] TWI: All expected trading days are present. [2025-10-20T21:36:24.471502] [DEBUG] [VALIDATOR ] TWI: Primary data validation passed. [2025-10-20T21:36:24.472542] [INFO ] [MAIN ] TWI: Fetched primary data is valid. [2025-10-20T21:36:24.472685] [DEBUG] [LOCAL ] TWI: Saving primary data. [2025-10-20T21:36:24.473876] [DEBUG] [LOCAL ] TWI: Trimmed 52166 records to 33967 (keeping 200 days of data) [2025-10-20T21:36:24.669969] [DEBUG] [LOCAL ] TWI: Primary data saved (33967 records). [2025-10-20T21:36:24.670118] [INFO ] [MAIN ] TWI: Full history saved. [2025-10-20T21:36:24.670201] [INFO ] [MAIN ] Update Progress: 2150/2367 symbols completed. TWI took 27.5s. Estimated remaining: 27m 53s. Rate: 7.8 symbols/min [2025-10-20T21:36:24.859554] [INFO ] [MAIN ] TWOD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:24.859746] [DEBUG] [LOCAL ] TWOD: No existing data file found in primary storage. [2025-10-20T21:36:24.859794] [WARN ] [MAIN ] TWOD: No existing data found, performing full refresh instead. [2025-10-20T21:36:24.859903] [INFO ] [MAIN ] TWOD: Wiping existing primary data before full refresh. [2025-10-20T21:36:24.859945] [INFO ] [LOCAL ] TWOD: Starting deletion of primary data. [2025-10-20T21:36:24.859987] [DEBUG] [LOCAL ] TWOD: No primary data file found to delete. [2025-10-20T21:36:24.860022] [INFO ] [MAIN ] TWOD: Starting full primary refresh. [2025-10-20T21:36:25.066872] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-04. [2025-10-20T21:36:25.176376] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-02 (1.73 seconds). [2025-10-20T21:36:25.278240] [INFO ] [REMOTE ] TWOD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:25.757428] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T21:36:25.851398] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-03. [2025-10-20T21:36:26.333015] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T21:36:26.651374] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-05. [2025-10-20T21:36:27.451425] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-06. [2025-10-20T21:36:27.462324] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T21:36:27.559978] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T21:36:28.251370] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-06. [2025-10-20T21:36:28.718087] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-06 (0.47 seconds). [2025-10-20T21:36:29.051346] [INFO ] [REMOTE ] TWOD: Fetching primary data for 2025-01. [2025-10-20T21:36:29.255112] [DEBUG] [REMOTE ] TWOD: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:36:29.271718] [WARN ] [REMOTE ] TWOD: No data returned for month 2025-01 [2025-10-20T21:36:29.272067] [FATAL] [REMOTE ] TWOD: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:36:29.273469] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-06 (1.82 seconds). [2025-10-20T21:36:29.283486] [FATAL] [MAIN ] TWOD: Failed to fetch primary data. [2025-10-20T21:36:29.283875] [WARN ] [MAIN ] TWOD: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:36:29.284097] [INFO ] [LOCAL ] TWOD: Starting deletion of primary data. [2025-10-20T21:36:29.284187] [DEBUG] [LOCAL ] TWOD: No primary data file found to delete. [2025-10-20T21:36:29.284623] [INFO ] [LOCAL ] TWOD: Starting deletion of derivative data. [2025-10-20T21:36:29.284731] [DEBUG] [LOCAL ] TWOD: No derivative data file found to delete. [2025-10-20T21:36:29.284790] [INFO ] [MAIN ] Update Progress: 2151/2367 symbols completed. TWOD took 4.6s. Estimated remaining: 27m 45s. Rate: 7.8 symbols/min [2025-10-20T21:36:29.449064] [INFO ] [MAIN ] TX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:29.473669] [DEBUG] [LOCAL ] TX: Loaded 23667 existing records from primary storage. [2025-10-20T21:36:29.473933] [INFO ] [MAIN ] TX: Loaded 23667 existing records. [2025-10-20T21:36:29.648005] [WARN ] [MAIN ] TX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:29.648219] [INFO ] [MAIN ] TX: Wiping existing primary data before full refresh. [2025-10-20T21:36:29.648273] [INFO ] [LOCAL ] TX: Starting deletion of primary data. [2025-10-20T21:36:29.648444] [DEBUG] [LOCAL ] TX: Deleted primary data file. [2025-10-20T21:36:29.648481] [INFO ] [MAIN ] TX: Starting full primary refresh. [2025-10-20T21:36:29.856617] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-07. [2025-10-20T21:36:29.997138] [INFO ] [REMOTE ] TX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:30.527068] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-07 (0.67 seconds). [2025-10-20T21:36:30.651358] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-04. [2025-10-20T21:36:31.451406] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-08. [2025-10-20T21:36:31.740235] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-08 (0.29 seconds). [2025-10-20T21:36:32.237104] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T21:36:32.251338] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-05. [2025-10-20T21:36:33.051425] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-07. [2025-10-20T21:36:33.851388] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-01. [2025-10-20T21:36:33.890850] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-05 (1.64 seconds). [2025-10-20T21:36:34.651355] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-09. [2025-10-20T21:36:34.965089] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T21:36:35.005429] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T21:36:35.229315] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-09 (0.58 seconds). [2025-10-20T21:36:35.451374] [INFO ] [REMOTE ] TWN: Fetching primary data for 2025-10. [2025-10-20T21:36:36.038509] [INFO ] [REMOTE ] TWN: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T21:36:36.040033] [INFO ] [REMOTE ] TWN: Full fetch complete (5226 records). [2025-10-20T21:36:36.040105] [DEBUG] [VALIDATOR ] TWN: Validating primary data. [2025-10-20T21:36:36.256723] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-08. [2025-10-20T21:36:36.410993] [DEBUG] [VALIDATOR ] TWN: All expected trading days are present. [2025-10-20T21:36:36.411078] [DEBUG] [VALIDATOR ] TWN: Primary data validation passed. [2025-10-20T21:36:36.411272] [INFO ] [MAIN ] TWN: Fetched primary data is valid. [2025-10-20T21:36:36.411312] [DEBUG] [LOCAL ] TWN: Saving primary data. [2025-10-20T21:36:36.411721] [DEBUG] [LOCAL ] TWN: Trimmed 5226 records to 3258 (keeping 200 days of data) [2025-10-20T21:36:36.430221] [DEBUG] [LOCAL ] TWN: Primary data saved (3258 records). [2025-10-20T21:36:36.430289] [INFO ] [MAIN ] TWN: Full history saved. [2025-10-20T21:36:36.430357] [INFO ] [MAIN ] Update Progress: 2152/2367 symbols completed. TWN took 19.5s. Estimated remaining: 27m 38s. Rate: 7.8 symbols/min [2025-10-20T21:36:36.643090] [INFO ] [MAIN ] TXNM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:36.690212] [DEBUG] [LOCAL ] TXNM: Loaded 43953 existing records from primary storage. [2025-10-20T21:36:36.690627] [INFO ] [MAIN ] TXNM: Loaded 43953 existing records. [2025-10-20T21:36:36.884093] [WARN ] [MAIN ] TXNM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:36.884360] [INFO ] [MAIN ] TXNM: Wiping existing primary data before full refresh. [2025-10-20T21:36:36.884407] [INFO ] [LOCAL ] TXNM: Starting deletion of primary data. [2025-10-20T21:36:36.884625] [DEBUG] [LOCAL ] TXNM: Deleted primary data file. [2025-10-20T21:36:36.884717] [INFO ] [MAIN ] TXNM: Starting full primary refresh. [2025-10-20T21:36:37.060100] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-02. [2025-10-20T21:36:37.281269] [INFO ] [REMOTE ] TXNM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:37.851384] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-01. [2025-10-20T21:36:38.413437] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T21:36:38.466739] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T21:36:38.651377] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-06. [2025-10-20T21:36:39.386170] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:36:39.451333] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-02. [2025-10-20T21:36:40.251360] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-03. [2025-10-20T21:36:40.357977] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:36:40.896441] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T21:36:41.051389] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-07. [2025-10-20T21:36:41.533031] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-03 (1.28 seconds). [2025-10-20T21:36:41.851460] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-04. [2025-10-20T21:36:42.651373] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-09. [2025-10-20T21:36:43.027237] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T21:36:43.065816] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-04 (1.21 seconds). [2025-10-20T21:36:43.451421] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-03. [2025-10-20T21:36:44.251444] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-08. [2025-10-20T21:36:44.608375] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T21:36:44.654371] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T21:36:45.051392] [INFO ] [REMOTE ] TWLO: Fetching primary data for 2025-10. [2025-10-20T21:36:45.720194] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T21:36:45.851376] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-04. [2025-10-20T21:36:46.651426] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-05. [2025-10-20T21:36:46.834758] [INFO ] [REMOTE ] TWLO: Incremental fetch complete for 2025-10 (1.78 seconds). [2025-10-20T21:36:46.842622] [INFO ] [REMOTE ] TWLO: Full fetch complete (76640 records). [2025-10-20T21:36:46.842813] [DEBUG] [VALIDATOR ] TWLO: Validating primary data. [2025-10-20T21:36:47.242408] [DEBUG] [VALIDATOR ] TWLO: All expected trading days are present. [2025-10-20T21:36:47.242524] [DEBUG] [VALIDATOR ] TWLO: Primary data validation passed. [2025-10-20T21:36:47.243759] [INFO ] [MAIN ] TWLO: Fetched primary data is valid. [2025-10-20T21:36:47.243940] [DEBUG] [LOCAL ] TWLO: Saving primary data. [2025-10-20T21:36:47.266556] [DEBUG] [LOCAL ] TWLO: Trimmed 76640 records to 52836 (keeping 200 days of data) [2025-10-20T21:36:47.270703] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T21:36:47.457193] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-09. [2025-10-20T21:36:47.503324] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T21:36:47.617277] [DEBUG] [LOCAL ] TWLO: Primary data saved (52836 records). [2025-10-20T21:36:47.617424] [INFO ] [MAIN ] TWLO: Full history saved. [2025-10-20T21:36:47.617514] [INFO ] [MAIN ] Update Progress: 2153/2367 symbols completed. TWLO took 42.5s. Estimated remaining: 27m 30s. Rate: 7.8 symbols/min [2025-10-20T21:36:47.785429] [INFO ] [MAIN ] TXO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:47.806286] [DEBUG] [LOCAL ] TXO: Loaded 20129 existing records from primary storage. [2025-10-20T21:36:47.806548] [INFO ] [MAIN ] TXO: Loaded 20129 existing records. [2025-10-20T21:36:47.983037] [WARN ] [MAIN ] TXO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:47.983301] [INFO ] [MAIN ] TXO: Wiping existing primary data before full refresh. [2025-10-20T21:36:47.983351] [INFO ] [LOCAL ] TXO: Starting deletion of primary data. [2025-10-20T21:36:47.983699] [DEBUG] [LOCAL ] TXO: Deleted primary data file. [2025-10-20T21:36:47.983764] [INFO ] [MAIN ] TXO: Starting full primary refresh. [2025-10-20T21:36:48.256744] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-06. [2025-10-20T21:36:48.368081] [INFO ] [REMOTE ] TXO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:49.051355] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-05. [2025-10-20T21:36:49.317043] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-09 (1.86 seconds). [2025-10-20T21:36:49.379027] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T21:36:49.851426] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-07. [2025-10-20T21:36:50.651363] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-01. [2025-10-20T21:36:50.718549] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T21:36:51.044690] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T21:36:51.451407] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-08. [2025-10-20T21:36:51.532759] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:36:52.251401] [INFO ] [REMOTE ] TWO: Fetching primary data for 2025-10. [2025-10-20T21:36:52.475273] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T21:36:53.051420] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-09. [2025-10-20T21:36:53.851353] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-02. [2025-10-20T21:36:53.885246] [INFO ] [REMOTE ] TWO: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T21:36:53.892710] [INFO ] [REMOTE ] TWO: Full fetch complete (69291 records). [2025-10-20T21:36:53.892907] [DEBUG] [VALIDATOR ] TWO: Validating primary data. [2025-10-20T21:36:53.937540] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T21:36:54.298432] [DEBUG] [VALIDATOR ] TWO: All expected trading days are present. [2025-10-20T21:36:54.298514] [DEBUG] [VALIDATOR ] TWO: Primary data validation passed. [2025-10-20T21:36:54.299541] [INFO ] [MAIN ] TWO: Fetched primary data is valid. [2025-10-20T21:36:54.299603] [DEBUG] [LOCAL ] TWO: Saving primary data. [2025-10-20T21:36:54.300875] [DEBUG] [LOCAL ] TWO: Trimmed 69291 records to 48624 (keeping 200 days of data) [2025-10-20T21:36:54.588744] [DEBUG] [LOCAL ] TWO: Primary data saved (48624 records). [2025-10-20T21:36:54.588869] [INFO ] [MAIN ] TWO: Full history saved. [2025-10-20T21:36:54.588945] [INFO ] [MAIN ] Update Progress: 2154/2367 symbols completed. TWO took 35.2s. Estimated remaining: 27m 23s. Rate: 7.8 symbols/min [2025-10-20T21:36:54.656456] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-06. [2025-10-20T21:36:54.761451] [INFO ] [MAIN ] TXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:36:54.805359] [DEBUG] [LOCAL ] TXT: Loaded 50302 existing records from primary storage. [2025-10-20T21:36:54.810915] [INFO ] [MAIN ] TXT: Loaded 50302 existing records. [2025-10-20T21:36:54.886859] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T21:36:55.001577] [WARN ] [MAIN ] TXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:36:55.001818] [INFO ] [MAIN ] TXT: Wiping existing primary data before full refresh. [2025-10-20T21:36:55.001877] [INFO ] [LOCAL ] TXT: Starting deletion of primary data. [2025-10-20T21:36:55.002080] [DEBUG] [LOCAL ] TXT: Deleted primary data file. [2025-10-20T21:36:55.002134] [INFO ] [MAIN ] TXT: Starting full primary refresh. [2025-10-20T21:36:55.355734] [INFO ] [REMOTE ] TXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:36:55.451343] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-01. [2025-10-20T21:36:56.203314] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T21:36:56.251358] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-07. [2025-10-20T21:36:57.051372] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-03. [2025-10-20T21:36:57.204572] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:36:57.838344] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T21:36:57.851320] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-08. [2025-10-20T21:36:57.942406] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T21:36:58.651372] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-04. [2025-10-20T21:36:59.348703] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T21:36:59.451397] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-09. [2025-10-20T21:36:59.491764] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-04 (0.84 seconds). [2025-10-20T21:37:00.251380] [INFO ] [REMOTE ] TX: Fetching primary data for 2025-10. [2025-10-20T21:37:00.884570] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T21:37:01.051365] [INFO ] [REMOTE ] TXNM: Fetching primary data for 2025-10. [2025-10-20T21:37:01.123573] [INFO ] [REMOTE ] TX: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T21:37:01.127560] [INFO ] [REMOTE ] TX: Full fetch complete (35667 records). [2025-10-20T21:37:01.127689] [DEBUG] [VALIDATOR ] TX: Validating primary data. [2025-10-20T21:37:01.490966] [DEBUG] [VALIDATOR ] TX: All expected trading days are present. [2025-10-20T21:37:01.491067] [DEBUG] [VALIDATOR ] TX: Primary data validation passed. [2025-10-20T21:37:01.491816] [INFO ] [MAIN ] TX: Fetched primary data is valid. [2025-10-20T21:37:01.491876] [DEBUG] [LOCAL ] TX: Saving primary data. [2025-10-20T21:37:01.492674] [DEBUG] [LOCAL ] TX: Trimmed 35667 records to 23414 (keeping 200 days of data) [2025-10-20T21:37:01.627382] [DEBUG] [LOCAL ] TX: Primary data saved (23414 records). [2025-10-20T21:37:01.627511] [INFO ] [MAIN ] TX: Full history saved. [2025-10-20T21:37:01.627586] [INFO ] [MAIN ] Update Progress: 2155/2367 symbols completed. TX took 32.3s. Estimated remaining: 27m 15s. Rate: 7.8 symbols/min [2025-10-20T21:37:01.793080] [INFO ] [MAIN ] TY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:01.803175] [DEBUG] [LOCAL ] TY: Loaded 7674 existing records from primary storage. [2025-10-20T21:37:01.803352] [INFO ] [MAIN ] TY: Loaded 7674 existing records. [2025-10-20T21:37:01.864565] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-02. [2025-10-20T21:37:01.978459] [WARN ] [MAIN ] TY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:01.978677] [INFO ] [MAIN ] TY: Wiping existing primary data before full refresh. [2025-10-20T21:37:01.978735] [INFO ] [LOCAL ] TY: Starting deletion of primary data. [2025-10-20T21:37:01.978884] [DEBUG] [LOCAL ] TY: Deleted primary data file. [2025-10-20T21:37:01.978921] [INFO ] [MAIN ] TY: Starting full primary refresh. [2025-10-20T21:37:02.329581] [INFO ] [REMOTE ] TY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:02.514471] [INFO ] [REMOTE ] TXNM: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T21:37:02.521212] [INFO ] [REMOTE ] TXNM: Full fetch complete (63014 records). [2025-10-20T21:37:02.521394] [DEBUG] [VALIDATOR ] TXNM: Validating primary data. [2025-10-20T21:37:02.656844] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-01. [2025-10-20T21:37:02.927093] [DEBUG] [VALIDATOR ] TXNM: All expected trading days are present. [2025-10-20T21:37:02.927177] [DEBUG] [VALIDATOR ] TXNM: Primary data validation passed. [2025-10-20T21:37:02.928296] [INFO ] [MAIN ] TXNM: Fetched primary data is valid. [2025-10-20T21:37:02.928358] [DEBUG] [LOCAL ] TXNM: Saving primary data. [2025-10-20T21:37:02.929563] [DEBUG] [LOCAL ] TXNM: Trimmed 63014 records to 43149 (keeping 200 days of data) [2025-10-20T21:37:03.186482] [DEBUG] [LOCAL ] TXNM: Primary data saved (43149 records). [2025-10-20T21:37:03.186601] [INFO ] [MAIN ] TXNM: Full history saved. [2025-10-20T21:37:03.187028] [INFO ] [MAIN ] Update Progress: 2156/2367 symbols completed. TXNM took 26.8s. Estimated remaining: 27m 6s. Rate: 7.8 symbols/min [2025-10-20T21:37:03.355547] [INFO ] [MAIN ] TYG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:03.374760] [DEBUG] [LOCAL ] TYG: Loaded 14089 existing records from primary storage. [2025-10-20T21:37:03.375003] [INFO ] [MAIN ] TYG: Loaded 14089 existing records. [2025-10-20T21:37:03.452565] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-05. [2025-10-20T21:37:03.563754] [WARN ] [MAIN ] TYG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:03.564083] [INFO ] [MAIN ] TYG: Wiping existing primary data before full refresh. [2025-10-20T21:37:03.564197] [INFO ] [LOCAL ] TYG: Starting deletion of primary data. [2025-10-20T21:37:03.564463] [DEBUG] [LOCAL ] TYG: Deleted primary data file. [2025-10-20T21:37:03.564521] [INFO ] [MAIN ] TYG: Starting full primary refresh. [2025-10-20T21:37:03.781999] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T21:37:03.900489] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T21:37:04.002610] [INFO ] [REMOTE ] TYG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:04.251409] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-02. [2025-10-20T21:37:04.742663] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-05 (1.29 seconds). [2025-10-20T21:37:04.810052] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T21:37:05.051440] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-01. [2025-10-20T21:37:05.851356] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-06. [2025-10-20T21:37:06.188102] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T21:37:06.651407] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-02. [2025-10-20T21:37:06.995680] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T21:37:07.451406] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-07. [2025-10-20T21:37:07.639173] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T21:37:08.251360] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-03. [2025-10-20T21:37:08.459084] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-07 (1.01 seconds). [2025-10-20T21:37:09.051369] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-08. [2025-10-20T21:37:09.851349] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-03. [2025-10-20T21:37:10.004195] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T21:37:10.059142] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-08 (1.01 seconds). [2025-10-20T21:37:10.559212] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-03 (0.71 seconds). [2025-10-20T21:37:10.651375] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-04. [2025-10-20T21:37:11.285669] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T21:37:11.451379] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-05. [2025-10-20T21:37:12.251354] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-09. [2025-10-20T21:37:12.258719] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-05 (0.81 seconds). [2025-10-20T21:37:13.051417] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-03. [2025-10-20T21:37:13.271408] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T21:37:13.851392] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-06. [2025-10-20T21:37:14.588731] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-06 (0.74 seconds). [2025-10-20T21:37:14.624630] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T21:37:14.651329] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-04. [2025-10-20T21:37:15.451387] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-07. [2025-10-20T21:37:15.696417] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T21:37:16.251378] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-05. [2025-10-20T21:37:16.355249] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-07 (0.90 seconds). [2025-10-20T21:37:16.937891] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-05 (0.69 seconds). [2025-10-20T21:37:17.051376] [INFO ] [REMOTE ] TXO: Fetching primary data for 2025-10. [2025-10-20T21:37:17.851392] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-06. [2025-10-20T21:37:18.067310] [INFO ] [REMOTE ] TXO: Incremental fetch complete for 2025-10 (1.02 seconds). [2025-10-20T21:37:18.071480] [INFO ] [REMOTE ] TXO: Full fetch complete (27047 records). [2025-10-20T21:37:18.071620] [DEBUG] [VALIDATOR ] TXO: Validating primary data. [2025-10-20T21:37:18.449833] [DEBUG] [VALIDATOR ] TXO: All expected trading days are present. [2025-10-20T21:37:18.449924] [DEBUG] [VALIDATOR ] TXO: Primary data validation passed. [2025-10-20T21:37:18.450737] [INFO ] [MAIN ] TXO: Fetched primary data is valid. [2025-10-20T21:37:18.450801] [DEBUG] [LOCAL ] TXO: Saving primary data. [2025-10-20T21:37:18.451608] [DEBUG] [LOCAL ] TXO: Trimmed 27047 records to 19974 (keeping 200 days of data) [2025-10-20T21:37:18.579397] [DEBUG] [LOCAL ] TXO: Primary data saved (19974 records). [2025-10-20T21:37:18.579609] [INFO ] [MAIN ] TXO: Full history saved. [2025-10-20T21:37:18.579743] [INFO ] [MAIN ] Update Progress: 2157/2367 symbols completed. TXO took 31.0s. Estimated remaining: 26m 59s. Rate: 7.8 symbols/min [2025-10-20T21:37:18.651363] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-08. [2025-10-20T21:37:18.730480] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T21:37:19.370873] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T21:37:19.451357] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-07. [2025-10-20T21:37:20.251352] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-09. [2025-10-20T21:37:20.488911] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T21:37:21.051346] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-08. [2025-10-20T21:37:21.069731] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T21:37:21.851352] [INFO ] [REMOTE ] TY: Fetching primary data for 2025-10. [2025-10-20T21:37:21.881259] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-08 (0.83 seconds). [2025-10-20T21:37:22.449478] [INFO ] [REMOTE ] TY: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T21:37:22.451004] [INFO ] [REMOTE ] TY: Full fetch complete (11391 records). [2025-10-20T21:37:22.451072] [DEBUG] [VALIDATOR ] TY: Validating primary data. [2025-10-20T21:37:22.657322] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-04. [2025-10-20T21:37:22.805260] [DEBUG] [VALIDATOR ] TY: All expected trading days are present. [2025-10-20T21:37:22.805350] [DEBUG] [VALIDATOR ] TY: Primary data validation passed. [2025-10-20T21:37:22.805606] [INFO ] [MAIN ] TY: Fetched primary data is valid. [2025-10-20T21:37:22.805663] [DEBUG] [LOCAL ] TY: Saving primary data. [2025-10-20T21:37:22.806160] [DEBUG] [LOCAL ] TY: Trimmed 11391 records to 7553 (keeping 200 days of data) [2025-10-20T21:37:22.851670] [DEBUG] [LOCAL ] TY: Primary data saved (7553 records). [2025-10-20T21:37:22.851805] [INFO ] [MAIN ] TY: Full history saved. [2025-10-20T21:37:22.851880] [INFO ] [MAIN ] Update Progress: 2158/2367 symbols completed. TY took 21.2s. Estimated remaining: 26m 51s. Rate: 7.8 symbols/min [2025-10-20T21:37:23.451363] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-09. [2025-10-20T21:37:24.268193] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T21:37:24.271518] [INFO ] [REMOTE ] TYG: Fetching primary data for 2025-10. [2025-10-20T21:37:24.523084] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T21:37:25.051406] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-05. [2025-10-20T21:37:25.123800] [INFO ] [REMOTE ] TYG: Incremental fetch complete for 2025-10 (0.85 seconds). [2025-10-20T21:37:25.125933] [INFO ] [REMOTE ] TYG: Full fetch complete (22263 records). [2025-10-20T21:37:25.126001] [DEBUG] [VALIDATOR ] TYG: Validating primary data. [2025-10-20T21:37:25.481551] [DEBUG] [VALIDATOR ] TYG: All expected trading days are present. [2025-10-20T21:37:25.481632] [DEBUG] [VALIDATOR ] TYG: Primary data validation passed. [2025-10-20T21:37:25.482130] [INFO ] [MAIN ] TYG: Fetched primary data is valid. [2025-10-20T21:37:25.482176] [DEBUG] [LOCAL ] TYG: Saving primary data. [2025-10-20T21:37:25.482756] [DEBUG] [LOCAL ] TYG: Trimmed 22263 records to 13996 (keeping 200 days of data) [2025-10-20T21:37:25.564800] [DEBUG] [LOCAL ] TYG: Primary data saved (13996 records). [2025-10-20T21:37:25.564915] [INFO ] [MAIN ] TYG: Full history saved. [2025-10-20T21:37:25.564990] [INFO ] [MAIN ] Update Progress: 2159/2367 symbols completed. TYG took 22.4s. Estimated remaining: 26m 43s. Rate: 7.8 symbols/min [2025-10-20T21:37:26.888173] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T21:37:26.898617] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-06. [2025-10-20T21:37:28.628025] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T21:37:28.638018] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-07. [2025-10-20T21:37:30.532231] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:37:30.543088] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-08. [2025-10-20T21:37:32.366533] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T21:37:32.377736] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-09. [2025-10-20T21:37:33.193450] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T21:37:33.204530] [INFO ] [REMOTE ] TXT: Fetching primary data for 2025-10. [2025-10-20T21:37:34.841180] [INFO ] [REMOTE ] TXT: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T21:37:34.848458] [INFO ] [REMOTE ] TXT: Full fetch complete (72404 records). [2025-10-20T21:37:34.848631] [DEBUG] [VALIDATOR ] TXT: Validating primary data. [2025-10-20T21:37:35.259432] [DEBUG] [VALIDATOR ] TXT: All expected trading days are present. [2025-10-20T21:37:35.259547] [DEBUG] [VALIDATOR ] TXT: Primary data validation passed. [2025-10-20T21:37:35.261010] [INFO ] [MAIN ] TXT: Fetched primary data is valid. [2025-10-20T21:37:35.261092] [DEBUG] [LOCAL ] TXT: Saving primary data. [2025-10-20T21:37:35.262914] [DEBUG] [LOCAL ] TXT: Trimmed 72404 records to 49541 (keeping 200 days of data) [2025-10-20T21:37:35.574162] [DEBUG] [LOCAL ] TXT: Primary data saved (49541 records). [2025-10-20T21:37:35.574313] [INFO ] [MAIN ] TXT: Full history saved. [2025-10-20T21:37:35.574406] [INFO ] [MAIN ] Update Progress: 2160/2367 symbols completed. TXT took 41.0s. Estimated remaining: 26m 36s. Rate: 7.8 symbols/min [2025-10-20T21:37:36.250804] [INFO ] [MAIN ] UA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:36.249935] [INFO ] [MAIN ] U: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:36.270117] [INFO ] [MAIN ] TYL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:36.287560] [INFO ] [MAIN ] UAA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:36.311783] [DEBUG] [LOCAL ] TYL: Loaded 22658 existing records from primary storage. [2025-10-20T21:37:36.337779] [INFO ] [MAIN ] TYL: Loaded 22658 existing records. [2025-10-20T21:37:36.489182] [DEBUG] [LOCAL ] U: Loaded 54579 existing records from primary storage. [2025-10-20T21:37:36.510008] [DEBUG] [LOCAL ] UA: Loaded 51219 existing records from primary storage. [2025-10-20T21:37:36.515323] [INFO ] [MAIN ] U: Loaded 54579 existing records. [2025-10-20T21:37:36.544618] [INFO ] [MAIN ] UA: Loaded 51219 existing records. [2025-10-20T21:37:36.569032] [DEBUG] [LOCAL ] UAA: Loaded 54277 existing records from primary storage. [2025-10-20T21:37:36.678847] [INFO ] [MAIN ] UAA: Loaded 54277 existing records. [2025-10-20T21:37:36.673619] [WARN ] [MAIN ] TYL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:36.807398] [INFO ] [MAIN ] TYL: Wiping existing primary data before full refresh. [2025-10-20T21:37:36.873556] [INFO ] [LOCAL ] TYL: Starting deletion of primary data. [2025-10-20T21:37:36.936935] [DEBUG] [LOCAL ] TYL: Deleted primary data file. [2025-10-20T21:37:37.040556] [INFO ] [MAIN ] TYL: Starting full primary refresh. [2025-10-20T21:37:37.043607] [WARN ] [MAIN ] U: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:37.141454] [INFO ] [MAIN ] U: Wiping existing primary data before full refresh. [2025-10-20T21:37:37.201030] [WARN ] [MAIN ] UA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:37.223098] [INFO ] [LOCAL ] U: Starting deletion of primary data. [2025-10-20T21:37:37.246284] [INFO ] [MAIN ] UA: Wiping existing primary data before full refresh. [2025-10-20T21:37:37.254663] [WARN ] [MAIN ] UAA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:37.270388] [INFO ] [MAIN ] UAA: Wiping existing primary data before full refresh. [2025-10-20T21:37:37.267079] [INFO ] [LOCAL ] UA: Starting deletion of primary data. [2025-10-20T21:37:37.270795] [INFO ] [LOCAL ] UAA: Starting deletion of primary data. [2025-10-20T21:37:37.267163] [DEBUG] [LOCAL ] U: Deleted primary data file. [2025-10-20T21:37:37.270921] [DEBUG] [LOCAL ] UA: Deleted primary data file. [2025-10-20T21:37:37.271289] [INFO ] [MAIN ] U: Starting full primary refresh. [2025-10-20T21:37:37.286914] [DEBUG] [LOCAL ] UAA: Deleted primary data file. [2025-10-20T21:37:37.287057] [INFO ] [MAIN ] UA: Starting full primary refresh. [2025-10-20T21:37:37.341910] [INFO ] [MAIN ] UAA: Starting full primary refresh. [2025-10-20T21:37:38.264209] [INFO ] [REMOTE ] TYL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:38.415143] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-01. [2025-10-20T21:37:38.685154] [INFO ] [REMOTE ] UA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:38.767821] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-01. [2025-10-20T21:37:38.766664] [INFO ] [REMOTE ] UAA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:38.792120] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-01. [2025-10-20T21:37:38.791625] [INFO ] [REMOTE ] U: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:38.793301] [INFO ] [REMOTE ] U: Fetching primary data for 2025-01. [2025-10-20T21:37:39.704805] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T21:37:39.709003] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-02. [2025-10-20T21:37:40.232813] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T21:37:40.243268] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-02. [2025-10-20T21:37:40.325966] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:37:40.336833] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-02. [2025-10-20T21:37:40.878193] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T21:37:40.882919] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-03. [2025-10-20T21:37:41.043944] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-01 (2.25 seconds). [2025-10-20T21:37:41.056352] [INFO ] [REMOTE ] U: Fetching primary data for 2025-02. [2025-10-20T21:37:41.879545] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:37:41.889808] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-03. [2025-10-20T21:37:41.979435] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:37:41.989423] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-03. [2025-10-20T21:37:42.115583] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-03 (1.23 seconds). [2025-10-20T21:37:42.120786] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-04. [2025-10-20T21:37:42.997189] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-02 (1.94 seconds). [2025-10-20T21:37:43.007425] [INFO ] [REMOTE ] U: Fetching primary data for 2025-03. [2025-10-20T21:37:43.189674] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T21:37:43.195795] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-05. [2025-10-20T21:37:43.303230] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T21:37:43.314041] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-04. [2025-10-20T21:37:43.817091] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:37:43.828588] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-04. [2025-10-20T21:37:43.993687] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T21:37:43.998515] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-06. [2025-10-20T21:37:44.919904] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T21:37:44.931253] [INFO ] [REMOTE ] U: Fetching primary data for 2025-04. [2025-10-20T21:37:45.069401] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T21:37:45.074273] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-07. [2025-10-20T21:37:45.134349] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T21:37:45.144933] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-05. [2025-10-20T21:37:45.461900] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:37:45.851398] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-05. [2025-10-20T21:37:46.224176] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T21:37:46.485926] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-05 (1.34 seconds). [2025-10-20T21:37:46.651371] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-08. [2025-10-20T21:37:47.223749] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T21:37:47.295040] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T21:37:47.456846] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-06. [2025-10-20T21:37:47.463971] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-08 (0.81 seconds). [2025-10-20T21:37:48.251375] [INFO ] [REMOTE ] U: Fetching primary data for 2025-05. [2025-10-20T21:37:49.051393] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-09. [2025-10-20T21:37:49.114024] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:37:49.851444] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-07. [2025-10-20T21:37:50.198679] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T21:37:50.651423] [INFO ] [REMOTE ] TYL: Fetching primary data for 2025-10. [2025-10-20T21:37:50.688393] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-05 (2.44 seconds). [2025-10-20T21:37:51.128160] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T21:37:51.451428] [INFO ] [REMOTE ] U: Fetching primary data for 2025-06. [2025-10-20T21:37:51.842971] [INFO ] [REMOTE ] TYL: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T21:37:51.846861] [INFO ] [REMOTE ] TYL: Full fetch complete (31673 records). [2025-10-20T21:37:51.846976] [DEBUG] [VALIDATOR ] TYL: Validating primary data. [2025-10-20T21:37:52.226274] [DEBUG] [VALIDATOR ] TYL: All expected trading days are present. [2025-10-20T21:37:52.226370] [DEBUG] [VALIDATOR ] TYL: Primary data validation passed. [2025-10-20T21:37:52.227157] [INFO ] [MAIN ] TYL: Fetched primary data is valid. [2025-10-20T21:37:52.227223] [DEBUG] [LOCAL ] TYL: Saving primary data. [2025-10-20T21:37:52.228244] [DEBUG] [LOCAL ] TYL: Trimmed 31673 records to 22229 (keeping 200 days of data) [2025-10-20T21:37:52.278970] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-08. [2025-10-20T21:37:52.371063] [DEBUG] [LOCAL ] TYL: Primary data saved (22229 records). [2025-10-20T21:37:52.371381] [INFO ] [MAIN ] TYL: Full history saved. [2025-10-20T21:37:52.371518] [INFO ] [MAIN ] Update Progress: 2161/2367 symbols completed. TYL took 16.8s. Estimated remaining: 26m 29s. Rate: 7.8 symbols/min [2025-10-20T21:37:52.541542] [INFO ] [MAIN ] UAN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:52.549457] [DEBUG] [LOCAL ] UAN: Loaded 5477 existing records from primary storage. [2025-10-20T21:37:52.549723] [INFO ] [MAIN ] UAN: Loaded 5477 existing records. [2025-10-20T21:37:52.726774] [WARN ] [MAIN ] UAN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:52.726955] [INFO ] [MAIN ] UAN: Wiping existing primary data before full refresh. [2025-10-20T21:37:52.727002] [INFO ] [LOCAL ] UAN: Starting deletion of primary data. [2025-10-20T21:37:52.727183] [DEBUG] [LOCAL ] UAN: Deleted primary data file. [2025-10-20T21:37:52.727240] [INFO ] [MAIN ] UAN: Starting full primary refresh. [2025-10-20T21:37:53.075903] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-06. [2025-10-20T21:37:53.080509] [INFO ] [REMOTE ] UAN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:37:53.729856] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-10-20T21:37:53.851356] [INFO ] [REMOTE ] U: Fetching primary data for 2025-07. [2025-10-20T21:37:54.247382] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-08 (1.97 seconds). [2025-10-20T21:37:54.651371] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-09. [2025-10-20T21:37:54.759442] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:37:55.451427] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-01. [2025-10-20T21:37:56.155980] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-01 (0.70 seconds). [2025-10-20T21:37:56.251340] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-07. [2025-10-20T21:37:56.285335] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T21:37:56.337538] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-07 (2.49 seconds). [2025-10-20T21:37:57.051370] [INFO ] [REMOTE ] UA: Fetching primary data for 2025-10. [2025-10-20T21:37:57.851412] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-02. [2025-10-20T21:37:58.184357] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T21:37:58.240327] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-02 (0.39 seconds). [2025-10-20T21:37:58.322322] [INFO ] [REMOTE ] UA: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T21:37:58.330727] [INFO ] [REMOTE ] UA: Full fetch complete (73180 records). [2025-10-20T21:37:58.330899] [DEBUG] [VALIDATOR ] UA: Validating primary data. [2025-10-20T21:37:58.656513] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-08. [2025-10-20T21:37:58.739979] [DEBUG] [VALIDATOR ] UA: All expected trading days are present. [2025-10-20T21:37:58.740063] [DEBUG] [VALIDATOR ] UA: Primary data validation passed. [2025-10-20T21:37:58.741154] [INFO ] [MAIN ] UA: Fetched primary data is valid. [2025-10-20T21:37:58.741216] [DEBUG] [LOCAL ] UA: Saving primary data. [2025-10-20T21:37:58.742503] [DEBUG] [LOCAL ] UA: Trimmed 73180 records to 50486 (keeping 200 days of data) [2025-10-20T21:37:59.021679] [DEBUG] [LOCAL ] UA: Primary data saved (50486 records). [2025-10-20T21:37:59.021818] [INFO ] [MAIN ] UA: Full history saved. [2025-10-20T21:37:59.021893] [INFO ] [MAIN ] Update Progress: 2162/2367 symbols completed. UA took 23.4s. Estimated remaining: 26m 21s. Rate: 7.8 symbols/min [2025-10-20T21:37:59.185544] [INFO ] [MAIN ] UBER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:37:59.234945] [DEBUG] [LOCAL ] UBER: Loaded 54586 existing records from primary storage. [2025-10-20T21:37:59.235253] [INFO ] [MAIN ] UBER: Loaded 54586 existing records. [2025-10-20T21:37:59.406833] [WARN ] [MAIN ] UBER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:37:59.407042] [INFO ] [MAIN ] UBER: Wiping existing primary data before full refresh. [2025-10-20T21:37:59.407104] [INFO ] [LOCAL ] UBER: Starting deletion of primary data. [2025-10-20T21:37:59.407343] [DEBUG] [LOCAL ] UBER: Deleted primary data file. [2025-10-20T21:37:59.407399] [INFO ] [MAIN ] UBER: Starting full primary refresh. [2025-10-20T21:37:59.460471] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-03. [2025-10-20T21:37:59.765883] [INFO ] [REMOTE ] UBER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:00.110024] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-03 (0.65 seconds). [2025-10-20T21:38:00.251389] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-01. [2025-10-20T21:38:00.450529] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-08 (1.79 seconds). [2025-10-20T21:38:01.051395] [INFO ] [REMOTE ] U: Fetching primary data for 2025-08. [2025-10-20T21:38:01.851384] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-04. [2025-10-20T21:38:02.656633] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-09. [2025-10-20T21:38:02.659102] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-04 (0.81 seconds). [2025-10-20T21:38:02.884625] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-01 (2.63 seconds). [2025-10-20T21:38:03.257173] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T21:38:03.451388] [INFO ] [REMOTE ] U: Fetching primary data for 2025-09. [2025-10-20T21:38:04.251429] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-02. [2025-10-20T21:38:04.634391] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T21:38:05.051408] [INFO ] [REMOTE ] UAA: Fetching primary data for 2025-10. [2025-10-20T21:38:05.486935] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T21:38:05.851442] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-05. [2025-10-20T21:38:06.503060] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-02 (2.25 seconds). [2025-10-20T21:38:06.518275] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-05 (0.67 seconds). [2025-10-20T21:38:06.651371] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-03. [2025-10-20T21:38:06.979445] [INFO ] [REMOTE ] UAA: Incremental fetch complete for 2025-10 (1.93 seconds). [2025-10-20T21:38:06.987617] [INFO ] [REMOTE ] UAA: Full fetch complete (77643 records). [2025-10-20T21:38:06.987817] [DEBUG] [VALIDATOR ] UAA: Validating primary data. [2025-10-20T21:38:07.391091] [DEBUG] [VALIDATOR ] UAA: All expected trading days are present. [2025-10-20T21:38:07.391188] [DEBUG] [VALIDATOR ] UAA: Primary data validation passed. [2025-10-20T21:38:07.392400] [INFO ] [MAIN ] UAA: Fetched primary data is valid. [2025-10-20T21:38:07.392471] [DEBUG] [LOCAL ] UAA: Saving primary data. [2025-10-20T21:38:07.394269] [DEBUG] [LOCAL ] UAA: Trimmed 77643 records to 53499 (keeping 200 days of data) [2025-10-20T21:38:07.475221] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-06. [2025-10-20T21:38:07.698285] [DEBUG] [LOCAL ] UAA: Primary data saved (53499 records). [2025-10-20T21:38:07.698430] [INFO ] [MAIN ] UAA: Full history saved. [2025-10-20T21:38:07.698510] [INFO ] [MAIN ] Update Progress: 2163/2367 symbols completed. UAA took 32.1s. Estimated remaining: 26m 13s. Rate: 7.8 symbols/min [2025-10-20T21:38:07.868152] [INFO ] [MAIN ] UBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:07.916401] [DEBUG] [LOCAL ] UBS: Loaded 53087 existing records from primary storage. [2025-10-20T21:38:07.916743] [INFO ] [MAIN ] UBS: Loaded 53087 existing records. [2025-10-20T21:38:08.102598] [WARN ] [MAIN ] UBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:08.102835] [INFO ] [MAIN ] UBS: Wiping existing primary data before full refresh. [2025-10-20T21:38:08.102883] [INFO ] [LOCAL ] UBS: Starting deletion of primary data. [2025-10-20T21:38:08.103084] [DEBUG] [LOCAL ] UBS: Deleted primary data file. [2025-10-20T21:38:08.103122] [INFO ] [MAIN ] UBS: Starting full primary refresh. [2025-10-20T21:38:08.208999] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:38:08.260727] [INFO ] [REMOTE ] U: Fetching primary data for 2025-10. [2025-10-20T21:38:08.468909] [INFO ] [REMOTE ] UBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:08.976162] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-03 (2.32 seconds). [2025-10-20T21:38:09.051405] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-01. [2025-10-20T21:38:09.851425] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-07. [2025-10-20T21:38:10.619523] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T21:38:10.645076] [INFO ] [REMOTE ] U: Incremental fetch complete for 2025-10 (2.38 seconds). [2025-10-20T21:38:10.652505] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-08. [2025-10-20T21:38:10.654927] [INFO ] [REMOTE ] U: Full fetch complete (77975 records). [2025-10-20T21:38:10.655033] [DEBUG] [VALIDATOR ] U: Validating primary data. [2025-10-20T21:38:11.001089] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-01 (1.95 seconds). [2025-10-20T21:38:11.104981] [DEBUG] [VALIDATOR ] U: All expected trading days are present. [2025-10-20T21:38:11.105073] [DEBUG] [VALIDATOR ] U: Primary data validation passed. [2025-10-20T21:38:11.106244] [INFO ] [MAIN ] U: Fetched primary data is valid. [2025-10-20T21:38:11.106319] [DEBUG] [LOCAL ] U: Saving primary data. [2025-10-20T21:38:11.107833] [DEBUG] [LOCAL ] U: Trimmed 77975 records to 53799 (keeping 200 days of data) [2025-10-20T21:38:11.414853] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T21:38:11.432900] [DEBUG] [LOCAL ] U: Primary data saved (53799 records). [2025-10-20T21:38:11.433024] [INFO ] [MAIN ] U: Full history saved. [2025-10-20T21:38:11.433099] [INFO ] [MAIN ] Update Progress: 2164/2367 symbols completed. U took 35.8s. Estimated remaining: 26m 5s. Rate: 7.8 symbols/min [2025-10-20T21:38:11.456585] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-04. [2025-10-20T21:38:11.616372] [INFO ] [MAIN ] UCB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:11.655702] [DEBUG] [LOCAL ] UCB: Loaded 38433 existing records from primary storage. [2025-10-20T21:38:11.655983] [INFO ] [MAIN ] UCB: Loaded 38433 existing records. [2025-10-20T21:38:11.834702] [WARN ] [MAIN ] UCB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:11.834932] [INFO ] [MAIN ] UCB: Wiping existing primary data before full refresh. [2025-10-20T21:38:11.834979] [INFO ] [LOCAL ] UCB: Starting deletion of primary data. [2025-10-20T21:38:11.835194] [DEBUG] [LOCAL ] UCB: Deleted primary data file. [2025-10-20T21:38:11.835251] [INFO ] [MAIN ] UCB: Starting full primary refresh. [2025-10-20T21:38:12.182750] [INFO ] [REMOTE ] UCB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:12.251345] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-01. [2025-10-20T21:38:13.051428] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-09. [2025-10-20T21:38:13.657630] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-09 (0.61 seconds). [2025-10-20T21:38:13.788812] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:38:13.851349] [INFO ] [REMOTE ] UAN: Fetching primary data for 2025-10. [2025-10-20T21:38:13.991079] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-04 (2.53 seconds). [2025-10-20T21:38:14.339182] [INFO ] [REMOTE ] UAN: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T21:38:14.340387] [INFO ] [REMOTE ] UAN: Full fetch complete (7930 records). [2025-10-20T21:38:14.340459] [DEBUG] [VALIDATOR ] UAN: Validating primary data. [2025-10-20T21:38:14.656849] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-02. [2025-10-20T21:38:14.721163] [DEBUG] [VALIDATOR ] UAN: All expected trading days are present. [2025-10-20T21:38:14.721242] [DEBUG] [VALIDATOR ] UAN: Primary data validation passed. [2025-10-20T21:38:14.721466] [INFO ] [MAIN ] UAN: Fetched primary data is valid. [2025-10-20T21:38:14.721508] [DEBUG] [LOCAL ] UAN: Saving primary data. [2025-10-20T21:38:14.722077] [DEBUG] [LOCAL ] UAN: Trimmed 7930 records to 5412 (keeping 200 days of data) [2025-10-20T21:38:14.760237] [DEBUG] [LOCAL ] UAN: Primary data saved (5412 records). [2025-10-20T21:38:14.760367] [INFO ] [MAIN ] UAN: Full history saved. [2025-10-20T21:38:14.760442] [INFO ] [MAIN ] Update Progress: 2165/2367 symbols completed. UAN took 22.4s. Estimated remaining: 25m 57s. Rate: 7.8 symbols/min [2025-10-20T21:38:14.928024] [INFO ] [MAIN ] UDR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:14.973859] [DEBUG] [LOCAL ] UDR: Loaded 52826 existing records from primary storage. [2025-10-20T21:38:14.974204] [INFO ] [MAIN ] UDR: Loaded 52826 existing records. [2025-10-20T21:38:15.164470] [WARN ] [MAIN ] UDR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:15.164735] [INFO ] [MAIN ] UDR: Wiping existing primary data before full refresh. [2025-10-20T21:38:15.164788] [INFO ] [LOCAL ] UDR: Starting deletion of primary data. [2025-10-20T21:38:15.165066] [DEBUG] [LOCAL ] UDR: Deleted primary data file. [2025-10-20T21:38:15.165106] [INFO ] [MAIN ] UDR: Starting full primary refresh. [2025-10-20T21:38:15.456466] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-05. [2025-10-20T21:38:15.547035] [INFO ] [REMOTE ] UDR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:16.134195] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T21:38:16.251389] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-01. [2025-10-20T21:38:17.051361] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-02. [2025-10-20T21:38:17.851358] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-03. [2025-10-20T21:38:17.856959] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-05 (2.40 seconds). [2025-10-20T21:38:17.919753] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:38:18.625868] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-02 (1.57 seconds). [2025-10-20T21:38:18.651353] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-03. [2025-10-20T21:38:19.457134] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-02. [2025-10-20T21:38:19.462903] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-03 (1.61 seconds). [2025-10-20T21:38:20.251381] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-06. [2025-10-20T21:38:20.616236] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-03 (1.96 seconds). [2025-10-20T21:38:21.051382] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-04. [2025-10-20T21:38:21.197442] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-02 (1.74 seconds). [2025-10-20T21:38:21.851446] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-03. [2025-10-20T21:38:22.473142] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-10-20T21:38:22.573050] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T21:38:22.651405] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-07. [2025-10-20T21:38:23.451410] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-04. [2025-10-20T21:38:23.535140] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T21:38:24.251373] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-04. [2025-10-20T21:38:24.955960] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:38:25.051375] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-05. [2025-10-20T21:38:25.257771] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-07 (2.61 seconds). [2025-10-20T21:38:25.680790] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:38:25.851374] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-08. [2025-10-20T21:38:26.651356] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-05. [2025-10-20T21:38:26.881208] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T21:38:27.451373] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-05. [2025-10-20T21:38:28.053399] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T21:38:28.251470] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-06. [2025-10-20T21:38:28.973565] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-08 (3.12 seconds). [2025-10-20T21:38:29.051371] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-09. [2025-10-20T21:38:29.130000] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T21:38:29.851391] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-06. [2025-10-20T21:38:30.048298] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T21:38:30.651421] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-06. [2025-10-20T21:38:31.329365] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:38:31.451351] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-07. [2025-10-20T21:38:31.506533] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-09 (2.46 seconds). [2025-10-20T21:38:31.958293] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T21:38:32.251417] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-07. [2025-10-20T21:38:32.902430] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T21:38:33.051339] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-08. [2025-10-20T21:38:33.851342] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-07. [2025-10-20T21:38:33.969347] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:38:34.472999] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-08 (1.42 seconds). [2025-10-20T21:38:34.651381] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-09. [2025-10-20T21:38:35.451347] [INFO ] [REMOTE ] UBER: Fetching primary data for 2025-10. [2025-10-20T21:38:35.738420] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:38:35.812025] [INFO ] [REMOTE ] UBER: Incremental fetch complete for 2025-10 (0.36 seconds). [2025-10-20T21:38:35.824557] [INFO ] [REMOTE ] UBER: Full fetch complete (77986 records). [2025-10-20T21:38:35.824988] [DEBUG] [VALIDATOR ] UBER: Validating primary data. [2025-10-20T21:38:36.185351] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:38:36.253244] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-08. [2025-10-20T21:38:36.299211] [DEBUG] [VALIDATOR ] UBER: All expected trading days are present. [2025-10-20T21:38:36.299301] [DEBUG] [VALIDATOR ] UBER: Primary data validation passed. [2025-10-20T21:38:36.300402] [INFO ] [MAIN ] UBER: Fetched primary data is valid. [2025-10-20T21:38:36.300467] [DEBUG] [LOCAL ] UBER: Saving primary data. [2025-10-20T21:38:36.301930] [DEBUG] [LOCAL ] UBER: Trimmed 77986 records to 53806 (keeping 200 days of data) [2025-10-20T21:38:36.620498] [DEBUG] [LOCAL ] UBER: Primary data saved (53806 records). [2025-10-20T21:38:36.620639] [INFO ] [MAIN ] UBER: Full history saved. [2025-10-20T21:38:36.621049] [INFO ] [MAIN ] Update Progress: 2166/2367 symbols completed. UBER took 37.6s. Estimated remaining: 25m 51s. Rate: 7.8 symbols/min [2025-10-20T21:38:36.788559] [INFO ] [MAIN ] UE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:36.788783] [DEBUG] [LOCAL ] UE: No existing data file found in primary storage. [2025-10-20T21:38:36.788828] [WARN ] [MAIN ] UE: No existing data found, performing full refresh instead. [2025-10-20T21:38:36.788959] [INFO ] [MAIN ] UE: Wiping existing primary data before full refresh. [2025-10-20T21:38:36.788999] [INFO ] [LOCAL ] UE: Starting deletion of primary data. [2025-10-20T21:38:36.789039] [DEBUG] [LOCAL ] UE: No primary data file found to delete. [2025-10-20T21:38:36.789069] [INFO ] [MAIN ] UE: Starting full primary refresh. [2025-10-20T21:38:37.057390] [INFO ] [REMOTE ] UCB: Fetching primary data for 2025-10. [2025-10-20T21:38:37.136819] [INFO ] [REMOTE ] UE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:37.851344] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-08. [2025-10-20T21:38:38.142707] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T21:38:38.397217] [INFO ] [REMOTE ] UCB: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:38:38.402936] [INFO ] [REMOTE ] UCB: Full fetch complete (54914 records). [2025-10-20T21:38:38.403029] [DEBUG] [VALIDATOR ] UCB: Validating primary data. [2025-10-20T21:38:38.661025] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-09. [2025-10-20T21:38:38.779084] [DEBUG] [VALIDATOR ] UCB: All expected trading days are present. [2025-10-20T21:38:38.779162] [DEBUG] [VALIDATOR ] UCB: Primary data validation passed. [2025-10-20T21:38:38.780179] [INFO ] [MAIN ] UCB: Fetched primary data is valid. [2025-10-20T21:38:38.780242] [DEBUG] [LOCAL ] UCB: Saving primary data. [2025-10-20T21:38:38.781271] [DEBUG] [LOCAL ] UCB: Trimmed 54914 records to 37801 (keeping 200 days of data) [2025-10-20T21:38:39.009297] [DEBUG] [LOCAL ] UCB: Primary data saved (37801 records). [2025-10-20T21:38:39.009460] [INFO ] [MAIN ] UCB: Full history saved. [2025-10-20T21:38:39.009542] [INFO ] [MAIN ] Update Progress: 2167/2367 symbols completed. UCB took 27.6s. Estimated remaining: 25m 43s. Rate: 7.8 symbols/min [2025-10-20T21:38:39.209156] [INFO ] [MAIN ] UFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:39.220081] [DEBUG] [LOCAL ] UFI: Loaded 8274 existing records from primary storage. [2025-10-20T21:38:39.220309] [INFO ] [MAIN ] UFI: Loaded 8274 existing records. [2025-10-20T21:38:39.401843] [WARN ] [MAIN ] UFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:39.402152] [INFO ] [MAIN ] UFI: Wiping existing primary data before full refresh. [2025-10-20T21:38:39.402201] [INFO ] [LOCAL ] UFI: Starting deletion of primary data. [2025-10-20T21:38:39.402386] [DEBUG] [LOCAL ] UFI: Deleted primary data file. [2025-10-20T21:38:39.402441] [INFO ] [MAIN ] UFI: Starting full primary refresh. [2025-10-20T21:38:39.456200] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-01. [2025-10-20T21:38:39.800728] [INFO ] [REMOTE ] UFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:39.803340] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T21:38:39.844706] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-08 (1.99 seconds). [2025-10-20T21:38:40.251370] [INFO ] [REMOTE ] UDR: Fetching primary data for 2025-10. [2025-10-20T21:38:40.998626] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:38:41.051336] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-02. [2025-10-20T21:38:41.856485] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-01. [2025-10-20T21:38:41.871457] [INFO ] [REMOTE ] UDR: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T21:38:41.879511] [INFO ] [REMOTE ] UDR: Full fetch complete (75104 records). [2025-10-20T21:38:41.879677] [DEBUG] [VALIDATOR ] UDR: Validating primary data. [2025-10-20T21:38:42.277460] [DEBUG] [VALIDATOR ] UDR: All expected trading days are present. [2025-10-20T21:38:42.277572] [DEBUG] [VALIDATOR ] UDR: Primary data validation passed. [2025-10-20T21:38:42.278862] [INFO ] [MAIN ] UDR: Fetched primary data is valid. [2025-10-20T21:38:42.278935] [DEBUG] [LOCAL ] UDR: Saving primary data. [2025-10-20T21:38:42.280370] [DEBUG] [LOCAL ] UDR: Trimmed 75104 records to 52109 (keeping 200 days of data) [2025-10-20T21:38:42.406833] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-01 (0.55 seconds). [2025-10-20T21:38:42.565461] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-02 (1.51 seconds). [2025-10-20T21:38:42.649533] [DEBUG] [LOCAL ] UDR: Primary data saved (52109 records). [2025-10-20T21:38:42.649713] [INFO ] [MAIN ] UDR: Full history saved. [2025-10-20T21:38:42.649807] [INFO ] [MAIN ] Update Progress: 2168/2367 symbols completed. UDR took 27.9s. Estimated remaining: 25m 35s. Rate: 7.8 symbols/min [2025-10-20T21:38:42.661425] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-09. [2025-10-20T21:38:42.841961] [INFO ] [MAIN ] UGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:42.889989] [DEBUG] [LOCAL ] UGI: Loaded 51333 existing records from primary storage. [2025-10-20T21:38:42.890305] [INFO ] [MAIN ] UGI: Loaded 51333 existing records. [2025-10-20T21:38:43.070201] [WARN ] [MAIN ] UGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:43.070537] [INFO ] [MAIN ] UGI: Wiping existing primary data before full refresh. [2025-10-20T21:38:43.070595] [INFO ] [LOCAL ] UGI: Starting deletion of primary data. [2025-10-20T21:38:43.071231] [DEBUG] [LOCAL ] UGI: Deleted primary data file. [2025-10-20T21:38:43.071291] [INFO ] [MAIN ] UGI: Starting full primary refresh. [2025-10-20T21:38:43.438343] [INFO ] [REMOTE ] UGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:43.451404] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-02. [2025-10-20T21:38:44.227786] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T21:38:44.251410] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-03. [2025-10-20T21:38:44.502607] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T21:38:45.051384] [INFO ] [REMOTE ] UBS: Fetching primary data for 2025-10. [2025-10-20T21:38:45.776968] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T21:38:45.851394] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-04. [2025-10-20T21:38:46.651337] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-03. [2025-10-20T21:38:46.825211] [INFO ] [REMOTE ] UBS: Incremental fetch complete for 2025-10 (1.77 seconds). [2025-10-20T21:38:46.832876] [INFO ] [REMOTE ] UBS: Full fetch complete (75958 records). [2025-10-20T21:38:46.832992] [DEBUG] [VALIDATOR ] UBS: Validating primary data. [2025-10-20T21:38:47.247496] [DEBUG] [VALIDATOR ] UBS: All expected trading days are present. [2025-10-20T21:38:47.247585] [DEBUG] [VALIDATOR ] UBS: Primary data validation passed. [2025-10-20T21:38:47.248689] [INFO ] [MAIN ] UBS: Fetched primary data is valid. [2025-10-20T21:38:47.248750] [DEBUG] [LOCAL ] UBS: Saving primary data. [2025-10-20T21:38:47.250153] [DEBUG] [LOCAL ] UBS: Trimmed 75958 records to 52297 (keeping 200 days of data) [2025-10-20T21:38:47.451717] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-01. [2025-10-20T21:38:47.491557] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T21:38:47.577120] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:38:47.609009] [DEBUG] [LOCAL ] UBS: Primary data saved (52297 records). [2025-10-20T21:38:47.609131] [INFO ] [MAIN ] UBS: Full history saved. [2025-10-20T21:38:47.609203] [INFO ] [MAIN ] Update Progress: 2169/2367 symbols completed. UBS took 39.9s. Estimated remaining: 25m 27s. Rate: 7.8 symbols/min [2025-10-20T21:38:47.777682] [INFO ] [MAIN ] UGP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:38:47.817274] [DEBUG] [LOCAL ] UGP: Loaded 43519 existing records from primary storage. [2025-10-20T21:38:47.817613] [INFO ] [MAIN ] UGP: Loaded 43519 existing records. [2025-10-20T21:38:47.993131] [WARN ] [MAIN ] UGP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:38:47.993442] [INFO ] [MAIN ] UGP: Wiping existing primary data before full refresh. [2025-10-20T21:38:47.993499] [INFO ] [LOCAL ] UGP: Starting deletion of primary data. [2025-10-20T21:38:47.993861] [DEBUG] [LOCAL ] UGP: Deleted primary data file. [2025-10-20T21:38:47.993917] [INFO ] [MAIN ] UGP: Starting full primary refresh. [2025-10-20T21:38:48.258589] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-04. [2025-10-20T21:38:48.337494] [INFO ] [REMOTE ] UGP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:38:49.051417] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-05. [2025-10-20T21:38:49.083234] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-04 (0.82 seconds). [2025-10-20T21:38:49.183943] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T21:38:49.851423] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-05. [2025-10-20T21:38:50.264431] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T21:38:50.602026] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T21:38:50.651453] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-01. [2025-10-20T21:38:51.451371] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-06. [2025-10-20T21:38:52.128499] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-01 (1.48 seconds). [2025-10-20T21:38:52.251427] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-02. [2025-10-20T21:38:52.916369] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-06 (1.46 seconds). [2025-10-20T21:38:53.051352] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-06. [2025-10-20T21:38:53.703970] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-06 (0.65 seconds). [2025-10-20T21:38:53.851362] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-02. [2025-10-20T21:38:53.904482] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T21:38:54.651367] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-07. [2025-10-20T21:38:55.257720] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T21:38:55.451418] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-03. [2025-10-20T21:38:55.840116] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T21:38:56.251423] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-08. [2025-10-20T21:38:57.051370] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-03. [2025-10-20T21:38:57.238902] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-03 (1.79 seconds). [2025-10-20T21:38:57.855810] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-07. [2025-10-20T21:38:57.858449] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:38:58.497366] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-07 (0.64 seconds). [2025-10-20T21:38:58.535303] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T21:38:58.651396] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-04. [2025-10-20T21:38:59.451385] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-04. [2025-10-20T21:39:00.251387] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T21:39:00.251459] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-08. [2025-10-20T21:39:01.048515] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T21:39:01.051321] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-09. [2025-10-20T21:39:01.392981] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T21:39:01.769687] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T21:39:01.851382] [INFO ] [REMOTE ] UFI: Fetching primary data for 2025-10. [2025-10-20T21:39:02.587378] [INFO ] [REMOTE ] UFI: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:39:02.589394] [INFO ] [REMOTE ] UFI: Full fetch complete (12334 records). [2025-10-20T21:39:02.589478] [DEBUG] [VALIDATOR ] UFI: Validating primary data. [2025-10-20T21:39:02.656490] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-09. [2025-10-20T21:39:02.966615] [DEBUG] [VALIDATOR ] UFI: All expected trading days are present. [2025-10-20T21:39:02.966767] [DEBUG] [VALIDATOR ] UFI: Primary data validation passed. [2025-10-20T21:39:02.967202] [INFO ] [MAIN ] UFI: Fetched primary data is valid. [2025-10-20T21:39:02.967270] [DEBUG] [LOCAL ] UFI: Saving primary data. [2025-10-20T21:39:02.968103] [DEBUG] [LOCAL ] UFI: Trimmed 12334 records to 8161 (keeping 200 days of data) [2025-10-20T21:39:03.015161] [DEBUG] [LOCAL ] UFI: Primary data saved (8161 records). [2025-10-20T21:39:03.015300] [INFO ] [MAIN ] UFI: Full history saved. [2025-10-20T21:39:03.015378] [INFO ] [MAIN ] Update Progress: 2170/2367 symbols completed. UFI took 24.0s. Estimated remaining: 25m 20s. Rate: 7.8 symbols/min [2025-10-20T21:39:03.208981] [INFO ] [MAIN ] UHAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:03.227023] [DEBUG] [LOCAL ] UHAL: Loaded 17102 existing records from primary storage. [2025-10-20T21:39:03.227299] [INFO ] [MAIN ] UHAL: Loaded 17102 existing records. [2025-10-20T21:39:03.424791] [WARN ] [MAIN ] UHAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:03.425047] [INFO ] [MAIN ] UHAL: Wiping existing primary data before full refresh. [2025-10-20T21:39:03.425096] [INFO ] [LOCAL ] UHAL: Starting deletion of primary data. [2025-10-20T21:39:03.425289] [DEBUG] [LOCAL ] UHAL: Deleted primary data file. [2025-10-20T21:39:03.425345] [INFO ] [MAIN ] UHAL: Starting full primary refresh. [2025-10-20T21:39:03.461351] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-05. [2025-10-20T21:39:03.816072] [INFO ] [REMOTE ] UHAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:04.136126] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T21:39:04.251380] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-05. [2025-10-20T21:39:05.051429] [INFO ] [REMOTE ] UE: Fetching primary data for 2025-10. [2025-10-20T21:39:05.090979] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:39:05.855856] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-06. [2025-10-20T21:39:05.861584] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-05 (1.61 seconds). [2025-10-20T21:39:06.350806] [INFO ] [REMOTE ] UE: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T21:39:06.357172] [INFO ] [REMOTE ] UE: Full fetch complete (61150 records). [2025-10-20T21:39:06.357349] [DEBUG] [VALIDATOR ] UE: Validating primary data. [2025-10-20T21:39:06.660946] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-01. [2025-10-20T21:39:06.758594] [DEBUG] [VALIDATOR ] UE: All expected trading days are present. [2025-10-20T21:39:06.758708] [DEBUG] [VALIDATOR ] UE: Primary data validation passed. [2025-10-20T21:39:06.759704] [INFO ] [MAIN ] UE: Fetched primary data is valid. [2025-10-20T21:39:06.759767] [DEBUG] [LOCAL ] UE: Saving primary data. [2025-10-20T21:39:06.761006] [DEBUG] [LOCAL ] UE: Trimmed 61150 records to 42603 (keeping 200 days of data) [2025-10-20T21:39:07.049084] [DEBUG] [LOCAL ] UE: Primary data saved (42603 records). [2025-10-20T21:39:07.049227] [INFO ] [MAIN ] UE: Full history saved. [2025-10-20T21:39:07.049304] [INFO ] [MAIN ] Update Progress: 2171/2367 symbols completed. UE took 30.4s. Estimated remaining: 25m 12s. Rate: 7.8 symbols/min [2025-10-20T21:39:07.238045] [INFO ] [MAIN ] UHS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:07.277924] [DEBUG] [LOCAL ] UHS: Loaded 40593 existing records from primary storage. [2025-10-20T21:39:07.278264] [INFO ] [MAIN ] UHS: Loaded 40593 existing records. [2025-10-20T21:39:07.461761] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-06. [2025-10-20T21:39:07.467789] [WARN ] [MAIN ] UHS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:07.468358] [INFO ] [MAIN ] UHS: Wiping existing primary data before full refresh. [2025-10-20T21:39:07.468532] [INFO ] [LOCAL ] UHS: Starting deletion of primary data. [2025-10-20T21:39:07.469513] [DEBUG] [LOCAL ] UHS: Deleted primary data file. [2025-10-20T21:39:07.469616] [INFO ] [MAIN ] UHS: Starting full primary refresh. [2025-10-20T21:39:07.623355] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:39:07.823718] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T21:39:07.882149] [INFO ] [REMOTE ] UHS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:08.251416] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-01. [2025-10-20T21:39:09.041185] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:39:09.051337] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-07. [2025-10-20T21:39:09.770621] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T21:39:09.851350] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-02. [2025-10-20T21:39:10.188342] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T21:39:10.651392] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-08. [2025-10-20T21:39:11.386113] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:39:11.451362] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-03. [2025-10-20T21:39:12.251372] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-02. [2025-10-20T21:39:12.300236] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:39:13.051376] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-09. [2025-10-20T21:39:13.136582] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T21:39:13.173926] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T21:39:13.851466] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-04. [2025-10-20T21:39:14.563102] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T21:39:14.651478] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-03. [2025-10-20T21:39:15.451364] [INFO ] [REMOTE ] UGP: Fetching primary data for 2025-10. [2025-10-20T21:39:15.533962] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T21:39:15.666846] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T21:39:16.251424] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-05. [2025-10-20T21:39:16.691983] [INFO ] [REMOTE ] UGP: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:39:16.700377] [INFO ] [REMOTE ] UGP: Full fetch complete (62213 records). [2025-10-20T21:39:16.700611] [DEBUG] [VALIDATOR ] UGP: Validating primary data. [2025-10-20T21:39:17.056775] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-04. [2025-10-20T21:39:17.104846] [DEBUG] [VALIDATOR ] UGP: All expected trading days are present. [2025-10-20T21:39:17.104939] [DEBUG] [VALIDATOR ] UGP: Primary data validation passed. [2025-10-20T21:39:17.106076] [INFO ] [MAIN ] UGP: Fetched primary data is valid. [2025-10-20T21:39:17.106141] [DEBUG] [LOCAL ] UGP: Saving primary data. [2025-10-20T21:39:17.107462] [DEBUG] [LOCAL ] UGP: Trimmed 62213 records to 42987 (keeping 200 days of data) [2025-10-20T21:39:17.361992] [DEBUG] [LOCAL ] UGP: Primary data saved (42987 records). [2025-10-20T21:39:17.362358] [INFO ] [MAIN ] UGP: Full history saved. [2025-10-20T21:39:17.362463] [INFO ] [MAIN ] Update Progress: 2172/2367 symbols completed. UGP took 29.8s. Estimated remaining: 25m 4s. Rate: 7.8 symbols/min [2025-10-20T21:39:17.545699] [INFO ] [MAIN ] UHT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:17.558176] [DEBUG] [LOCAL ] UHT: Loaded 10371 existing records from primary storage. [2025-10-20T21:39:17.558386] [INFO ] [MAIN ] UHT: Loaded 10371 existing records. [2025-10-20T21:39:17.748376] [WARN ] [MAIN ] UHT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:17.749079] [INFO ] [MAIN ] UHT: Wiping existing primary data before full refresh. [2025-10-20T21:39:17.749226] [INFO ] [LOCAL ] UHT: Starting deletion of primary data. [2025-10-20T21:39:17.749434] [DEBUG] [LOCAL ] UHT: Deleted primary data file. [2025-10-20T21:39:17.749489] [INFO ] [MAIN ] UHT: Starting full primary refresh. [2025-10-20T21:39:17.852741] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-07. [2025-10-20T21:39:17.964582] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-04 (0.91 seconds). [2025-10-20T21:39:18.161520] [INFO ] [REMOTE ] UHT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:18.163109] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T21:39:18.651385] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-06. [2025-10-20T21:39:19.451383] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-01. [2025-10-20T21:39:19.654295] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T21:39:20.251372] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-08. [2025-10-20T21:39:20.291192] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T21:39:20.334010] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:39:21.051422] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-02. [2025-10-20T21:39:21.851391] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-05. [2025-10-20T21:39:22.012409] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T21:39:22.050234] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-02 (1.00 seconds). [2025-10-20T21:39:22.651401] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-09. [2025-10-20T21:39:22.799309] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-05 (0.95 seconds). [2025-10-20T21:39:23.451416] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-07. [2025-10-20T21:39:24.251349] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-03. [2025-10-20T21:39:24.350040] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T21:39:25.051397] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-06. [2025-10-20T21:39:25.091369] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-03 (0.84 seconds). [2025-10-20T21:39:25.169739] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:39:25.851395] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-08. [2025-10-20T21:39:26.043636] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-06 (0.99 seconds). [2025-10-20T21:39:26.651403] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-04. [2025-10-20T21:39:27.255454] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-08 (1.40 seconds). [2025-10-20T21:39:27.292229] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-04 (0.64 seconds). [2025-10-20T21:39:27.451429] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-07. [2025-10-20T21:39:28.251381] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-05. [2025-10-20T21:39:28.521314] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T21:39:29.023087] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T21:39:29.051379] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-06. [2025-10-20T21:39:29.607447] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-06 (0.56 seconds). [2025-10-20T21:39:29.851371] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-07. [2025-10-20T21:39:30.651378] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-09. [2025-10-20T21:39:30.712595] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T21:39:31.360049] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T21:39:31.451375] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-08. [2025-10-20T21:39:32.251408] [INFO ] [REMOTE ] UGI: Fetching primary data for 2025-10. [2025-10-20T21:39:32.608123] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-08 (1.16 seconds). [2025-10-20T21:39:33.051422] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-08. [2025-10-20T21:39:33.844450] [INFO ] [REMOTE ] UGI: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T21:39:33.851856] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-09. [2025-10-20T21:39:33.854391] [INFO ] [REMOTE ] UGI: Full fetch complete (73444 records). [2025-10-20T21:39:33.854713] [DEBUG] [VALIDATOR ] UGI: Validating primary data. [2025-10-20T21:39:33.983141] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T21:39:34.289343] [DEBUG] [VALIDATOR ] UGI: All expected trading days are present. [2025-10-20T21:39:34.289472] [DEBUG] [VALIDATOR ] UGI: Primary data validation passed. [2025-10-20T21:39:34.290974] [INFO ] [MAIN ] UGI: Fetched primary data is valid. [2025-10-20T21:39:34.291049] [DEBUG] [LOCAL ] UGI: Saving primary data. [2025-10-20T21:39:34.293327] [DEBUG] [LOCAL ] UGI: Trimmed 73444 records to 50511 (keeping 200 days of data) [2025-10-20T21:39:34.634197] [DEBUG] [LOCAL ] UGI: Primary data saved (50511 records). [2025-10-20T21:39:34.634472] [INFO ] [MAIN ] UGI: Full history saved. [2025-10-20T21:39:34.634599] [INFO ] [MAIN ] Update Progress: 2173/2367 symbols completed. UGI took 52.0s. Estimated remaining: 24m 57s. Rate: 7.8 symbols/min [2025-10-20T21:39:34.657884] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-09. [2025-10-20T21:39:34.821732] [INFO ] [MAIN ] UI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:34.840559] [DEBUG] [LOCAL ] UI: Loaded 14297 existing records from primary storage. [2025-10-20T21:39:34.840934] [INFO ] [MAIN ] UI: Loaded 14297 existing records. [2025-10-20T21:39:35.033495] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T21:39:35.051829] [WARN ] [MAIN ] UI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:35.052873] [INFO ] [MAIN ] UI: Wiping existing primary data before full refresh. [2025-10-20T21:39:35.053311] [INFO ] [LOCAL ] UI: Starting deletion of primary data. [2025-10-20T21:39:35.053563] [DEBUG] [LOCAL ] UI: Deleted primary data file. [2025-10-20T21:39:35.053611] [INFO ] [MAIN ] UI: Starting full primary refresh. [2025-10-20T21:39:35.432954] [INFO ] [REMOTE ] UI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:35.451353] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-01. [2025-10-20T21:39:35.466791] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T21:39:36.251366] [INFO ] [REMOTE ] UHAL: Fetching primary data for 2025-10. [2025-10-20T21:39:36.336429] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T21:39:37.051414] [INFO ] [REMOTE ] UHS: Fetching primary data for 2025-10. [2025-10-20T21:39:37.165272] [INFO ] [REMOTE ] UHAL: Incremental fetch complete for 2025-10 (0.91 seconds). [2025-10-20T21:39:37.168946] [INFO ] [REMOTE ] UHAL: Full fetch complete (23417 records). [2025-10-20T21:39:37.169090] [DEBUG] [VALIDATOR ] UHAL: Validating primary data. [2025-10-20T21:39:37.554765] [DEBUG] [VALIDATOR ] UHAL: All expected trading days are present. [2025-10-20T21:39:37.554858] [DEBUG] [VALIDATOR ] UHAL: Primary data validation passed. [2025-10-20T21:39:37.555323] [INFO ] [MAIN ] UHAL: Fetched primary data is valid. [2025-10-20T21:39:37.555363] [DEBUG] [LOCAL ] UHAL: Saving primary data. [2025-10-20T21:39:37.556030] [DEBUG] [LOCAL ] UHAL: Trimmed 23417 records to 16899 (keeping 200 days of data) [2025-10-20T21:39:37.657062] [DEBUG] [LOCAL ] UHAL: Primary data saved (16899 records). [2025-10-20T21:39:37.657205] [INFO ] [MAIN ] UHAL: Full history saved. [2025-10-20T21:39:37.657287] [INFO ] [MAIN ] Update Progress: 2174/2367 symbols completed. UHAL took 34.6s. Estimated remaining: 24m 49s. Rate: 7.8 symbols/min [2025-10-20T21:39:37.840458] [INFO ] [MAIN ] UIS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:37.858457] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-02. [2025-10-20T21:39:37.876203] [DEBUG] [LOCAL ] UIS: Loaded 37313 existing records from primary storage. [2025-10-20T21:39:37.876538] [INFO ] [MAIN ] UIS: Loaded 37313 existing records. [2025-10-20T21:39:38.058869] [WARN ] [MAIN ] UIS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:38.059372] [INFO ] [MAIN ] UIS: Wiping existing primary data before full refresh. [2025-10-20T21:39:38.059740] [INFO ] [LOCAL ] UIS: Starting deletion of primary data. [2025-10-20T21:39:38.059968] [DEBUG] [LOCAL ] UIS: Deleted primary data file. [2025-10-20T21:39:38.060024] [INFO ] [MAIN ] UIS: Starting full primary refresh. [2025-10-20T21:39:38.367252] [INFO ] [REMOTE ] UHS: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T21:39:38.418609] [INFO ] [REMOTE ] UHS: Full fetch complete (58249 records). [2025-10-20T21:39:38.447715] [INFO ] [REMOTE ] UIS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:38.448776] [DEBUG] [VALIDATOR ] UHS: Validating primary data. [2025-10-20T21:39:38.662291] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-01. [2025-10-20T21:39:38.831669] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T21:39:38.919354] [DEBUG] [VALIDATOR ] UHS: All expected trading days are present. [2025-10-20T21:39:38.919478] [DEBUG] [VALIDATOR ] UHS: Primary data validation passed. [2025-10-20T21:39:38.921785] [INFO ] [MAIN ] UHS: Fetched primary data is valid. [2025-10-20T21:39:38.921863] [DEBUG] [LOCAL ] UHS: Saving primary data. [2025-10-20T21:39:38.923636] [DEBUG] [LOCAL ] UHS: Trimmed 58249 records to 39895 (keeping 200 days of data) [2025-10-20T21:39:39.203551] [DEBUG] [LOCAL ] UHS: Primary data saved (39895 records). [2025-10-20T21:39:39.203737] [INFO ] [MAIN ] UHS: Full history saved. [2025-10-20T21:39:39.204272] [INFO ] [MAIN ] Update Progress: 2175/2367 symbols completed. UHS took 32.2s. Estimated remaining: 24m 41s. Rate: 7.8 symbols/min [2025-10-20T21:39:39.371832] [INFO ] [MAIN ] UL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:39.421871] [DEBUG] [LOCAL ] UL: Loaded 53858 existing records from primary storage. [2025-10-20T21:39:39.422219] [INFO ] [MAIN ] UL: Loaded 53858 existing records. [2025-10-20T21:39:39.456619] [INFO ] [REMOTE ] UHT: Fetching primary data for 2025-10. [2025-10-20T21:39:39.598269] [WARN ] [MAIN ] UL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:39.598504] [INFO ] [MAIN ] UL: Wiping existing primary data before full refresh. [2025-10-20T21:39:39.598554] [INFO ] [LOCAL ] UL: Starting deletion of primary data. [2025-10-20T21:39:39.598807] [DEBUG] [LOCAL ] UL: Deleted primary data file. [2025-10-20T21:39:39.598863] [INFO ] [MAIN ] UL: Starting full primary refresh. [2025-10-20T21:39:39.945836] [INFO ] [REMOTE ] UL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:40.132110] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T21:39:40.251352] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-02. [2025-10-20T21:39:40.325135] [INFO ] [REMOTE ] UHT: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T21:39:40.327724] [INFO ] [REMOTE ] UHT: Full fetch complete (15099 records). [2025-10-20T21:39:40.327809] [DEBUG] [VALIDATOR ] UHT: Validating primary data. [2025-10-20T21:39:40.693082] [DEBUG] [VALIDATOR ] UHT: All expected trading days are present. [2025-10-20T21:39:40.693164] [DEBUG] [VALIDATOR ] UHT: Primary data validation passed. [2025-10-20T21:39:40.693627] [INFO ] [MAIN ] UHT: Fetched primary data is valid. [2025-10-20T21:39:40.693699] [DEBUG] [LOCAL ] UHT: Saving primary data. [2025-10-20T21:39:40.694177] [DEBUG] [LOCAL ] UHT: Trimmed 15099 records to 10229 (keeping 200 days of data) [2025-10-20T21:39:40.756683] [DEBUG] [LOCAL ] UHT: Primary data saved (10229 records). [2025-10-20T21:39:40.756841] [INFO ] [MAIN ] UHT: Full history saved. [2025-10-20T21:39:40.756923] [INFO ] [MAIN ] Update Progress: 2176/2367 symbols completed. UHT took 23.4s. Estimated remaining: 24m 33s. Rate: 7.8 symbols/min [2025-10-20T21:39:40.924629] [INFO ] [MAIN ] ULS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:39:40.962680] [DEBUG] [LOCAL ] ULS: Loaded 39538 existing records from primary storage. [2025-10-20T21:39:40.963063] [INFO ] [MAIN ] ULS: Loaded 39538 existing records. [2025-10-20T21:39:41.060764] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-01. [2025-10-20T21:39:41.171206] [WARN ] [MAIN ] ULS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:39:41.171568] [INFO ] [MAIN ] ULS: Wiping existing primary data before full refresh. [2025-10-20T21:39:41.171630] [INFO ] [LOCAL ] ULS: Starting deletion of primary data. [2025-10-20T21:39:41.171881] [DEBUG] [LOCAL ] ULS: Deleted primary data file. [2025-10-20T21:39:41.171971] [INFO ] [MAIN ] ULS: Starting full primary refresh. [2025-10-20T21:39:41.532149] [INFO ] [REMOTE ] ULS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:39:41.570152] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-02 (1.32 seconds). [2025-10-20T21:39:41.851404] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-03. [2025-10-20T21:39:42.651377] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-01. [2025-10-20T21:39:42.885243] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-01 (1.82 seconds). [2025-10-20T21:39:43.215877] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T21:39:43.451392] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-02. [2025-10-20T21:39:43.890338] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T21:39:44.251378] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-04. [2025-10-20T21:39:44.933692] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T21:39:45.051394] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-03. [2025-10-20T21:39:45.658729] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T21:39:45.851453] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-03. [2025-10-20T21:39:46.651416] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-02. [2025-10-20T21:39:46.824547] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-03 (0.97 seconds). [2025-10-20T21:39:46.869112] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:39:47.451381] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-04. [2025-10-20T21:39:47.577339] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-02 (0.93 seconds). [2025-10-20T21:39:48.251365] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-03. [2025-10-20T21:39:49.051351] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-05. [2025-10-20T21:39:49.133255] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-04 (1.68 seconds). [2025-10-20T21:39:49.232117] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T21:39:49.851389] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-04. [2025-10-20T21:39:50.582996] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:39:50.651342] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-05. [2025-10-20T21:39:50.737892] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-04 (0.89 seconds). [2025-10-20T21:39:51.451408] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-05. [2025-10-20T21:39:52.251364] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-06. [2025-10-20T21:39:52.318728] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-05 (0.87 seconds). [2025-10-20T21:39:52.449515] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-05 (1.80 seconds). [2025-10-20T21:39:53.051396] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-06. [2025-10-20T21:39:53.672222] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T21:39:53.851379] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-06. [2025-10-20T21:39:54.651367] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-07. [2025-10-20T21:39:54.671813] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-06 (0.82 seconds). [2025-10-20T21:39:54.924298] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-06 (1.87 seconds). [2025-10-20T21:39:55.451430] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-04. [2025-10-20T21:39:56.113351] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T21:39:56.251365] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-08. [2025-10-20T21:39:57.051415] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-07. [2025-10-20T21:39:57.189441] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-04 (1.74 seconds). [2025-10-20T21:39:57.478128] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-08 (1.23 seconds). [2025-10-20T21:39:57.851397] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-09. [2025-10-20T21:39:57.926704] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-07 (0.88 seconds). [2025-10-20T21:39:58.651434] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-08. [2025-10-20T21:39:59.258520] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T21:39:59.405475] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T21:39:59.451403] [INFO ] [REMOTE ] UIS: Fetching primary data for 2025-10. [2025-10-20T21:40:00.251486] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-09. [2025-10-20T21:40:00.952777] [INFO ] [REMOTE ] UIS: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T21:40:00.958049] [INFO ] [REMOTE ] UIS: Full fetch complete (52008 records). [2025-10-20T21:40:00.958208] [DEBUG] [VALIDATOR ] UIS: Validating primary data. [2025-10-20T21:40:01.054979] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-05. [2025-10-20T21:40:01.346675] [DEBUG] [VALIDATOR ] UIS: All expected trading days are present. [2025-10-20T21:40:01.346829] [DEBUG] [VALIDATOR ] UIS: Primary data validation passed. [2025-10-20T21:40:01.347791] [INFO ] [MAIN ] UIS: Fetched primary data is valid. [2025-10-20T21:40:01.347911] [DEBUG] [LOCAL ] UIS: Saving primary data. [2025-10-20T21:40:01.349029] [DEBUG] [LOCAL ] UIS: Trimmed 52008 records to 36775 (keeping 200 days of data) [2025-10-20T21:40:01.458326] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T21:40:01.566322] [DEBUG] [LOCAL ] UIS: Primary data saved (36775 records). [2025-10-20T21:40:01.566451] [INFO ] [MAIN ] UIS: Full history saved. [2025-10-20T21:40:01.566527] [INFO ] [MAIN ] Update Progress: 2177/2367 symbols completed. UIS took 23.9s. Estimated remaining: 24m 26s. Rate: 7.8 symbols/min [2025-10-20T21:40:01.731297] [INFO ] [MAIN ] UMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:01.779557] [DEBUG] [LOCAL ] UMC: Loaded 53463 existing records from primary storage. [2025-10-20T21:40:01.779933] [INFO ] [MAIN ] UMC: Loaded 53463 existing records. [2025-10-20T21:40:01.857423] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-07. [2025-10-20T21:40:01.976900] [WARN ] [MAIN ] UMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:01.977148] [INFO ] [MAIN ] UMC: Wiping existing primary data before full refresh. [2025-10-20T21:40:01.977198] [INFO ] [LOCAL ] UMC: Starting deletion of primary data. [2025-10-20T21:40:01.977487] [DEBUG] [LOCAL ] UMC: Deleted primary data file. [2025-10-20T21:40:01.977546] [INFO ] [MAIN ] UMC: Starting full primary refresh. [2025-10-20T21:40:02.337224] [INFO ] [REMOTE ] UMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:02.467488] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T21:40:02.651369] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-06. [2025-10-20T21:40:03.451364] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-01. [2025-10-20T21:40:03.893375] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T21:40:04.166938] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T21:40:04.251351] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-07. [2025-10-20T21:40:05.051384] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-08. [2025-10-20T21:40:05.340428] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-01 (1.89 seconds). [2025-10-20T21:40:05.657272] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T21:40:05.851367] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-08. [2025-10-20T21:40:06.651369] [INFO ] [REMOTE ] UI: Fetching primary data for 2025-10. [2025-10-20T21:40:06.920588] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T21:40:07.463829] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-08 (1.61 seconds). [2025-10-20T21:40:07.462275] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-02. [2025-10-20T21:40:07.793046] [INFO ] [REMOTE ] UI: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T21:40:07.796396] [INFO ] [REMOTE ] UI: Full fetch complete (19591 records). [2025-10-20T21:40:07.796478] [DEBUG] [VALIDATOR ] UI: Validating primary data. [2025-10-20T21:40:08.156419] [DEBUG] [VALIDATOR ] UI: All expected trading days are present. [2025-10-20T21:40:08.156506] [DEBUG] [VALIDATOR ] UI: Primary data validation passed. [2025-10-20T21:40:08.156899] [INFO ] [MAIN ] UI: Fetched primary data is valid. [2025-10-20T21:40:08.156994] [DEBUG] [LOCAL ] UI: Saving primary data. [2025-10-20T21:40:08.157687] [DEBUG] [LOCAL ] UI: Trimmed 19591 records to 14122 (keeping 200 days of data) [2025-10-20T21:40:08.247571] [DEBUG] [LOCAL ] UI: Primary data saved (14122 records). [2025-10-20T21:40:08.247737] [INFO ] [MAIN ] UI: Full history saved. [2025-10-20T21:40:08.247821] [INFO ] [MAIN ] Update Progress: 2178/2367 symbols completed. UI took 33.6s. Estimated remaining: 24m 18s. Rate: 7.8 symbols/min [2025-10-20T21:40:08.259639] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-09. [2025-10-20T21:40:08.423590] [INFO ] [MAIN ] UMH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:08.457538] [DEBUG] [LOCAL ] UMH: Loaded 34724 existing records from primary storage. [2025-10-20T21:40:08.457840] [INFO ] [MAIN ] UMH: Loaded 34724 existing records. [2025-10-20T21:40:08.633465] [WARN ] [MAIN ] UMH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:08.633688] [INFO ] [MAIN ] UMH: Wiping existing primary data before full refresh. [2025-10-20T21:40:08.633758] [INFO ] [LOCAL ] UMH: Starting deletion of primary data. [2025-10-20T21:40:08.633971] [DEBUG] [LOCAL ] UMH: Deleted primary data file. [2025-10-20T21:40:08.634037] [INFO ] [MAIN ] UMH: Starting full primary refresh. [2025-10-20T21:40:08.990201] [INFO ] [REMOTE ] UMH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:09.051368] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-09. [2025-10-20T21:40:09.151040] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T21:40:09.851377] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-01. [2025-10-20T21:40:10.470677] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-09 (1.42 seconds). [2025-10-20T21:40:10.519996] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-09 (2.26 seconds). [2025-10-20T21:40:10.651413] [INFO ] [REMOTE ] UL: Fetching primary data for 2025-10. [2025-10-20T21:40:11.140957] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T21:40:11.451394] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-02. [2025-10-20T21:40:12.251359] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-03. [2025-10-20T21:40:12.538054] [INFO ] [REMOTE ] UL: Incremental fetch complete for 2025-10 (1.89 seconds). [2025-10-20T21:40:12.545999] [INFO ] [REMOTE ] UL: Full fetch complete (77081 records). [2025-10-20T21:40:12.546125] [DEBUG] [VALIDATOR ] UL: Validating primary data. [2025-10-20T21:40:12.676438] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T21:40:12.963333] [DEBUG] [VALIDATOR ] UL: All expected trading days are present. [2025-10-20T21:40:12.963425] [DEBUG] [VALIDATOR ] UL: Primary data validation passed. [2025-10-20T21:40:12.964514] [INFO ] [MAIN ] UL: Fetched primary data is valid. [2025-10-20T21:40:12.964575] [DEBUG] [LOCAL ] UL: Saving primary data. [2025-10-20T21:40:12.965984] [DEBUG] [LOCAL ] UL: Trimmed 77081 records to 53076 (keeping 200 days of data) [2025-10-20T21:40:13.052870] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-03. [2025-10-20T21:40:13.291312] [DEBUG] [LOCAL ] UL: Primary data saved (53076 records). [2025-10-20T21:40:13.291439] [INFO ] [MAIN ] UL: Full history saved. [2025-10-20T21:40:13.291794] [INFO ] [MAIN ] Update Progress: 2179/2367 symbols completed. UL took 34.1s. Estimated remaining: 24m 10s. Rate: 7.8 symbols/min [2025-10-20T21:40:13.458147] [INFO ] [MAIN ] UNF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:13.472933] [DEBUG] [LOCAL ] UNF: Loaded 12140 existing records from primary storage. [2025-10-20T21:40:13.473157] [INFO ] [MAIN ] UNF: Loaded 12140 existing records. [2025-10-20T21:40:13.653430] [WARN ] [MAIN ] UNF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:13.653684] [INFO ] [MAIN ] UNF: Wiping existing primary data before full refresh. [2025-10-20T21:40:13.653741] [INFO ] [LOCAL ] UNF: Starting deletion of primary data. [2025-10-20T21:40:13.653893] [DEBUG] [LOCAL ] UNF: Deleted primary data file. [2025-10-20T21:40:13.653929] [INFO ] [MAIN ] UNF: Starting full primary refresh. [2025-10-20T21:40:13.856472] [INFO ] [REMOTE ] ULS: Fetching primary data for 2025-10. [2025-10-20T21:40:14.020165] [INFO ] [REMOTE ] UNF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:14.050102] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T21:40:14.079475] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-03 (1.03 seconds). [2025-10-20T21:40:14.651386] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-01. [2025-10-20T21:40:15.210544] [INFO ] [REMOTE ] ULS: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T21:40:15.215591] [INFO ] [REMOTE ] ULS: Full fetch complete (53923 records). [2025-10-20T21:40:15.215718] [DEBUG] [VALIDATOR ] ULS: Validating primary data. [2025-10-20T21:40:15.459582] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-04. [2025-10-20T21:40:15.609202] [DEBUG] [VALIDATOR ] ULS: All expected trading days are present. [2025-10-20T21:40:15.609293] [DEBUG] [VALIDATOR ] ULS: Primary data validation passed. [2025-10-20T21:40:15.610380] [INFO ] [MAIN ] ULS: Fetched primary data is valid. [2025-10-20T21:40:15.610445] [DEBUG] [LOCAL ] ULS: Saving primary data. [2025-10-20T21:40:15.611562] [DEBUG] [LOCAL ] ULS: Trimmed 53923 records to 38818 (keeping 200 days of data) [2025-10-20T21:40:15.840633] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T21:40:15.864903] [DEBUG] [LOCAL ] ULS: Primary data saved (38818 records). [2025-10-20T21:40:15.865005] [INFO ] [MAIN ] ULS: Full history saved. [2025-10-20T21:40:15.865077] [INFO ] [MAIN ] Update Progress: 2180/2367 symbols completed. ULS took 35.1s. Estimated remaining: 24m 2s. Rate: 7.8 symbols/min [2025-10-20T21:40:16.033232] [INFO ] [MAIN ] UNFI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:16.071854] [DEBUG] [LOCAL ] UNFI: Loaded 43232 existing records from primary storage. [2025-10-20T21:40:16.072214] [INFO ] [MAIN ] UNFI: Loaded 43232 existing records. [2025-10-20T21:40:16.260463] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-04. [2025-10-20T21:40:16.264945] [WARN ] [MAIN ] UNFI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:16.266000] [INFO ] [MAIN ] UNFI: Wiping existing primary data before full refresh. [2025-10-20T21:40:16.266057] [INFO ] [LOCAL ] UNFI: Starting deletion of primary data. [2025-10-20T21:40:16.266239] [DEBUG] [LOCAL ] UNFI: Deleted primary data file. [2025-10-20T21:40:16.266277] [INFO ] [MAIN ] UNFI: Starting full primary refresh. [2025-10-20T21:40:16.612049] [INFO ] [REMOTE ] UNFI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:17.051427] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-02. [2025-10-20T21:40:17.422146] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-04 (1.96 seconds). [2025-10-20T21:40:17.544427] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-02 (0.49 seconds). [2025-10-20T21:40:17.719040] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T21:40:17.851396] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-01. [2025-10-20T21:40:18.651372] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-03. [2025-10-20T21:40:19.262631] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T21:40:19.435753] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T21:40:19.451344] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-04. [2025-10-20T21:40:20.251402] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-05. [2025-10-20T21:40:20.548883] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-04 (1.10 seconds). [2025-10-20T21:40:21.051393] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-02. [2025-10-20T21:40:21.851407] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-05. [2025-10-20T21:40:21.925540] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-05 (1.67 seconds). [2025-10-20T21:40:22.461476] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T21:40:22.651399] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-03. [2025-10-20T21:40:23.261600] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T21:40:23.451469] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-05. [2025-10-20T21:40:24.191776] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-03 (1.54 seconds). [2025-10-20T21:40:24.229497] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-05 (0.78 seconds). [2025-10-20T21:40:24.251334] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-04. [2025-10-20T21:40:25.051401] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-06. [2025-10-20T21:40:25.757499] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-04 (1.51 seconds). [2025-10-20T21:40:25.851364] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-05. [2025-10-20T21:40:25.934166] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T21:40:26.651410] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-06. [2025-10-20T21:40:27.354851] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T21:40:27.451365] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-07. [2025-10-20T21:40:28.251343] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-06. [2025-10-20T21:40:28.355673] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T21:40:28.423722] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T21:40:29.051363] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-08. [2025-10-20T21:40:29.851374] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-06. [2025-10-20T21:40:29.937198] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-06 (1.69 seconds). [2025-10-20T21:40:30.431129] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T21:40:30.651403] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-07. [2025-10-20T21:40:30.757627] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-06 (0.91 seconds). [2025-10-20T21:40:31.451424] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-07. [2025-10-20T21:40:32.251441] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-09. [2025-10-20T21:40:32.406492] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T21:40:33.051367] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-08. [2025-10-20T21:40:33.213528] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-07 (1.76 seconds). [2025-10-20T21:40:33.265964] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T21:40:33.851380] [INFO ] [REMOTE ] UMH: Fetching primary data for 2025-10. [2025-10-20T21:40:34.651353] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-08. [2025-10-20T21:40:34.694208] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:40:34.795482] [INFO ] [REMOTE ] UMH: Incremental fetch complete for 2025-10 (0.94 seconds). [2025-10-20T21:40:34.801502] [INFO ] [REMOTE ] UMH: Full fetch complete (47280 records). [2025-10-20T21:40:34.801607] [DEBUG] [VALIDATOR ] UMH: Validating primary data. [2025-10-20T21:40:35.179638] [DEBUG] [VALIDATOR ] UMH: All expected trading days are present. [2025-10-20T21:40:35.179761] [DEBUG] [VALIDATOR ] UMH: Primary data validation passed. [2025-10-20T21:40:35.180613] [INFO ] [MAIN ] UMH: Fetched primary data is valid. [2025-10-20T21:40:35.180677] [DEBUG] [LOCAL ] UMH: Saving primary data. [2025-10-20T21:40:35.181712] [DEBUG] [LOCAL ] UMH: Trimmed 47280 records to 34230 (keeping 200 days of data) [2025-10-20T21:40:35.388595] [DEBUG] [LOCAL ] UMH: Primary data saved (34230 records). [2025-10-20T21:40:35.388780] [INFO ] [MAIN ] UMH: Full history saved. [2025-10-20T21:40:35.388878] [INFO ] [MAIN ] Update Progress: 2181/2367 symbols completed. UMH took 27.1s. Estimated remaining: 23m 55s. Rate: 7.8 symbols/min [2025-10-20T21:40:35.456636] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-07. [2025-10-20T21:40:35.562544] [INFO ] [MAIN ] UNH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:35.613158] [DEBUG] [LOCAL ] UNH: Loaded 54598 existing records from primary storage. [2025-10-20T21:40:35.613478] [INFO ] [MAIN ] UNH: Loaded 54598 existing records. [2025-10-20T21:40:35.794819] [WARN ] [MAIN ] UNH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:35.795092] [INFO ] [MAIN ] UNH: Wiping existing primary data before full refresh. [2025-10-20T21:40:35.795148] [INFO ] [LOCAL ] UNH: Starting deletion of primary data. [2025-10-20T21:40:35.795371] [DEBUG] [LOCAL ] UNH: Deleted primary data file. [2025-10-20T21:40:35.795427] [INFO ] [MAIN ] UNH: Starting full primary refresh. [2025-10-20T21:40:36.156537] [INFO ] [REMOTE ] UNH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:36.251338] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-09. [2025-10-20T21:40:36.484209] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T21:40:36.524403] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T21:40:37.051417] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-09. [2025-10-20T21:40:37.770071] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T21:40:37.851374] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-08. [2025-10-20T21:40:38.582715] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-08 (0.73 seconds). [2025-10-20T21:40:38.651367] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-09. [2025-10-20T21:40:39.059200] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-09 (2.01 seconds). [2025-10-20T21:40:39.451342] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-01. [2025-10-20T21:40:39.535433] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-09 (0.88 seconds). [2025-10-20T21:40:40.251382] [INFO ] [REMOTE ] UNF: Fetching primary data for 2025-10. [2025-10-20T21:40:41.051370] [INFO ] [REMOTE ] UNFI: Fetching primary data for 2025-10. [2025-10-20T21:40:41.170794] [INFO ] [REMOTE ] UNF: Incremental fetch complete for 2025-10 (0.92 seconds). [2025-10-20T21:40:41.173452] [INFO ] [REMOTE ] UNF: Full fetch complete (17768 records). [2025-10-20T21:40:41.173554] [DEBUG] [VALIDATOR ] UNF: Validating primary data. [2025-10-20T21:40:41.614572] [DEBUG] [VALIDATOR ] UNF: All expected trading days are present. [2025-10-20T21:40:41.614718] [DEBUG] [VALIDATOR ] UNF: Primary data validation passed. [2025-10-20T21:40:41.615321] [INFO ] [MAIN ] UNF: Fetched primary data is valid. [2025-10-20T21:40:41.615430] [DEBUG] [LOCAL ] UNF: Saving primary data. [2025-10-20T21:40:41.616552] [DEBUG] [LOCAL ] UNF: Trimmed 17768 records to 11812 (keeping 200 days of data) [2025-10-20T21:40:41.694859] [DEBUG] [LOCAL ] UNF: Primary data saved (11812 records). [2025-10-20T21:40:41.695071] [INFO ] [MAIN ] UNF: Full history saved. [2025-10-20T21:40:41.695228] [INFO ] [MAIN ] Update Progress: 2182/2367 symbols completed. UNF took 28.4s. Estimated remaining: 23m 48s. Rate: 7.8 symbols/min [2025-10-20T21:40:41.859147] [INFO ] [REMOTE ] UMC: Fetching primary data for 2025-10. [2025-10-20T21:40:41.895584] [INFO ] [MAIN ] UNM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:41.929892] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-01 (2.48 seconds). [2025-10-20T21:40:41.983175] [DEBUG] [LOCAL ] UNM: Loaded 48817 existing records from primary storage. [2025-10-20T21:40:41.983490] [INFO ] [MAIN ] UNM: Loaded 48817 existing records. [2025-10-20T21:40:42.178469] [WARN ] [MAIN ] UNM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:42.178799] [INFO ] [MAIN ] UNM: Wiping existing primary data before full refresh. [2025-10-20T21:40:42.178859] [INFO ] [LOCAL ] UNM: Starting deletion of primary data. [2025-10-20T21:40:42.179419] [DEBUG] [LOCAL ] UNM: Deleted primary data file. [2025-10-20T21:40:42.179478] [INFO ] [MAIN ] UNM: Starting full primary refresh. [2025-10-20T21:40:42.545564] [INFO ] [REMOTE ] UNM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:42.566128] [INFO ] [REMOTE ] UNFI: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T21:40:42.572792] [INFO ] [REMOTE ] UNFI: Full fetch complete (59841 records). [2025-10-20T21:40:42.572895] [DEBUG] [VALIDATOR ] UNFI: Validating primary data. [2025-10-20T21:40:42.661546] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-01. [2025-10-20T21:40:42.949962] [DEBUG] [VALIDATOR ] UNFI: All expected trading days are present. [2025-10-20T21:40:42.950048] [DEBUG] [VALIDATOR ] UNFI: Primary data validation passed. [2025-10-20T21:40:42.951005] [INFO ] [MAIN ] UNFI: Fetched primary data is valid. [2025-10-20T21:40:42.951067] [DEBUG] [LOCAL ] UNFI: Saving primary data. [2025-10-20T21:40:42.952244] [DEBUG] [LOCAL ] UNFI: Trimmed 59841 records to 42612 (keeping 200 days of data) [2025-10-20T21:40:43.188557] [DEBUG] [LOCAL ] UNFI: Primary data saved (42612 records). [2025-10-20T21:40:43.188708] [INFO ] [MAIN ] UNFI: Full history saved. [2025-10-20T21:40:43.188792] [INFO ] [MAIN ] Update Progress: 2183/2367 symbols completed. UNFI took 27.3s. Estimated remaining: 23m 39s. Rate: 7.8 symbols/min [2025-10-20T21:40:43.358347] [INFO ] [MAIN ] UNMA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:43.364583] [DEBUG] [LOCAL ] UNMA: Loaded 3559 existing records from primary storage. [2025-10-20T21:40:43.364803] [INFO ] [MAIN ] UNMA: Loaded 3559 existing records. [2025-10-20T21:40:43.461720] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-02. [2025-10-20T21:40:43.541720] [WARN ] [MAIN ] UNMA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:43.541910] [INFO ] [MAIN ] UNMA: Wiping existing primary data before full refresh. [2025-10-20T21:40:43.541956] [INFO ] [LOCAL ] UNMA: Starting deletion of primary data. [2025-10-20T21:40:43.542103] [DEBUG] [LOCAL ] UNMA: Deleted primary data file. [2025-10-20T21:40:43.542158] [INFO ] [MAIN ] UNMA: Starting full primary refresh. [2025-10-20T21:40:43.896210] [INFO ] [REMOTE ] UNMA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:43.918434] [INFO ] [REMOTE ] UMC: Incremental fetch complete for 2025-10 (2.06 seconds). [2025-10-20T21:40:43.926692] [INFO ] [REMOTE ] UMC: Full fetch complete (76674 records). [2025-10-20T21:40:43.926841] [DEBUG] [VALIDATOR ] UMC: Validating primary data. [2025-10-20T21:40:44.253747] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-01. [2025-10-20T21:40:44.363246] [DEBUG] [VALIDATOR ] UMC: All expected trading days are present. [2025-10-20T21:40:44.363368] [DEBUG] [VALIDATOR ] UMC: Primary data validation passed. [2025-10-20T21:40:44.364732] [INFO ] [MAIN ] UMC: Fetched primary data is valid. [2025-10-20T21:40:44.364845] [DEBUG] [LOCAL ] UMC: Saving primary data. [2025-10-20T21:40:44.390957] [DEBUG] [LOCAL ] UMC: Trimmed 76674 records to 52683 (keeping 200 days of data) [2025-10-20T21:40:44.392967] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T21:40:44.720692] [DEBUG] [LOCAL ] UMC: Primary data saved (52683 records). [2025-10-20T21:40:44.720857] [INFO ] [MAIN ] UMC: Full history saved. [2025-10-20T21:40:44.720941] [INFO ] [MAIN ] Update Progress: 2184/2367 symbols completed. UMC took 43.2s. Estimated remaining: 23m 31s. Rate: 7.8 symbols/min [2025-10-20T21:40:44.903598] [INFO ] [MAIN ] UNP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:44.908337] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-01 (0.65 seconds). [2025-10-20T21:40:44.958466] [DEBUG] [LOCAL ] UNP: Loaded 54105 existing records from primary storage. [2025-10-20T21:40:44.958821] [INFO ] [MAIN ] UNP: Loaded 54105 existing records. [2025-10-20T21:40:45.060362] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-02. [2025-10-20T21:40:45.142054] [WARN ] [MAIN ] UNP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:45.142262] [INFO ] [MAIN ] UNP: Wiping existing primary data before full refresh. [2025-10-20T21:40:45.142309] [INFO ] [LOCAL ] UNP: Starting deletion of primary data. [2025-10-20T21:40:45.142518] [DEBUG] [LOCAL ] UNP: Deleted primary data file. [2025-10-20T21:40:45.142572] [INFO ] [MAIN ] UNP: Starting full primary refresh. [2025-10-20T21:40:45.497151] [INFO ] [REMOTE ] UNP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:45.674554] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-02 (2.21 seconds). [2025-10-20T21:40:45.689541] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T21:40:45.851380] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-03. [2025-10-20T21:40:46.418322] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-03 (0.57 seconds). [2025-10-20T21:40:46.651415] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-01. [2025-10-20T21:40:47.451377] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-04. [2025-10-20T21:40:48.008291] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-04 (0.56 seconds). [2025-10-20T21:40:48.251417] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-05. [2025-10-20T21:40:48.490093] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T21:40:48.798793] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-05 (0.55 seconds). [2025-10-20T21:40:49.051421] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-02. [2025-10-20T21:40:49.851398] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-06. [2025-10-20T21:40:50.441506] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-06 (0.59 seconds). [2025-10-20T21:40:50.651399] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-07. [2025-10-20T21:40:50.868260] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T21:40:51.231419] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T21:40:51.451369] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-08. [2025-10-20T21:40:51.930484] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T21:40:52.251381] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-03. [2025-10-20T21:40:53.051380] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-03. [2025-10-20T21:40:53.851386] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-02. [2025-10-20T21:40:54.485781] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-03 (2.23 seconds). [2025-10-20T21:40:54.651692] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-09. [2025-10-20T21:40:54.905738] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-03 (1.85 seconds). [2025-10-20T21:40:55.018474] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T21:40:55.268132] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T21:40:55.451400] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-04. [2025-10-20T21:40:56.251414] [INFO ] [REMOTE ] UNMA: Fetching primary data for 2025-10. [2025-10-20T21:40:56.831456] [INFO ] [REMOTE ] UNMA: Incremental fetch complete for 2025-10 (0.58 seconds). [2025-10-20T21:40:56.832579] [INFO ] [REMOTE ] UNMA: Full fetch complete (5470 records). [2025-10-20T21:40:56.832683] [DEBUG] [VALIDATOR ] UNMA: Validating primary data. [2025-10-20T21:40:57.059259] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-04. [2025-10-20T21:40:57.204999] [DEBUG] [VALIDATOR ] UNMA: All expected trading days are present. [2025-10-20T21:40:57.205082] [DEBUG] [VALIDATOR ] UNMA: Primary data validation passed. [2025-10-20T21:40:57.205250] [INFO ] [MAIN ] UNMA: Fetched primary data is valid. [2025-10-20T21:40:57.205287] [DEBUG] [LOCAL ] UNMA: Saving primary data. [2025-10-20T21:40:57.205721] [DEBUG] [LOCAL ] UNMA: Trimmed 5470 records to 3457 (keeping 200 days of data) [2025-10-20T21:40:57.226695] [DEBUG] [LOCAL ] UNMA: Primary data saved (3457 records). [2025-10-20T21:40:57.226939] [INFO ] [MAIN ] UNMA: Full history saved. [2025-10-20T21:40:57.227085] [INFO ] [MAIN ] Update Progress: 2185/2367 symbols completed. UNMA took 14.0s. Estimated remaining: 23m 24s. Rate: 7.8 symbols/min [2025-10-20T21:40:57.401530] [INFO ] [MAIN ] UP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:40:57.443819] [DEBUG] [LOCAL ] UP: Loaded 44937 existing records from primary storage. [2025-10-20T21:40:57.444139] [INFO ] [MAIN ] UP: Loaded 44937 existing records. [2025-10-20T21:40:57.630768] [WARN ] [MAIN ] UP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:40:57.631198] [INFO ] [MAIN ] UP: Wiping existing primary data before full refresh. [2025-10-20T21:40:57.631290] [INFO ] [LOCAL ] UP: Starting deletion of primary data. [2025-10-20T21:40:57.631679] [DEBUG] [LOCAL ] UP: Deleted primary data file. [2025-10-20T21:40:57.631754] [INFO ] [MAIN ] UP: Starting full primary refresh. [2025-10-20T21:40:57.755271] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-04 (2.30 seconds). [2025-10-20T21:40:57.864013] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-03. [2025-10-20T21:40:58.044767] [INFO ] [REMOTE ] UP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:40:58.651432] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-01. [2025-10-20T21:40:59.451355] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-05. [2025-10-20T21:40:59.684015] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:40:59.756098] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-04 (2.70 seconds). [2025-10-20T21:41:00.182192] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:41:00.251329] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-02. [2025-10-20T21:41:01.051425] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-04. [2025-10-20T21:41:01.417691] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T21:41:01.452343] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:41:01.851420] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-05. [2025-10-20T21:41:02.573211] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T21:41:02.651408] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-03. [2025-10-20T21:41:03.451393] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-06. [2025-10-20T21:41:04.233519] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T21:41:04.251340] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-04. [2025-10-20T21:41:04.717577] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-05 (2.87 seconds). [2025-10-20T21:41:05.051413] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-06. [2025-10-20T21:41:05.279405] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-06 (1.83 seconds). [2025-10-20T21:41:05.420581] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T21:41:05.851421] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-05. [2025-10-20T21:41:06.651369] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-07. [2025-10-20T21:41:07.451348] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-05. [2025-10-20T21:41:07.544125] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T21:41:07.687768] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-06 (2.64 seconds). [2025-10-20T21:41:08.251395] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-07. [2025-10-20T21:41:08.679437] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T21:41:08.991872] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T21:41:09.051361] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-08. [2025-10-20T21:41:09.851354] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-06. [2025-10-20T21:41:10.651390] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-06. [2025-10-20T21:41:10.992116] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T21:41:11.075824] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-07 (2.82 seconds). [2025-10-20T21:41:11.451386] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-09. [2025-10-20T21:41:11.511054] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:41:11.818846] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T21:41:12.251374] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-08. [2025-10-20T21:41:13.037310] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T21:41:13.051339] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-07. [2025-10-20T21:41:13.851372] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-07. [2025-10-20T21:41:14.623729] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T21:41:14.651329] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-08. [2025-10-20T21:41:15.050351] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-08 (2.80 seconds). [2025-10-20T21:41:15.451340] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-09. [2025-10-20T21:41:15.636410] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-07 (1.79 seconds). [2025-10-20T21:41:16.251370] [INFO ] [REMOTE ] UNP: Fetching primary data for 2025-10. [2025-10-20T21:41:16.317443] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-08 (1.67 seconds). [2025-10-20T21:41:16.658090] [INFO ] [REMOTE ] UNP: Incremental fetch complete for 2025-10 (0.41 seconds). [2025-10-20T21:41:16.667878] [INFO ] [REMOTE ] UNP: Full fetch complete (77127 records). [2025-10-20T21:41:16.668079] [DEBUG] [VALIDATOR ] UNP: Validating primary data. [2025-10-20T21:41:17.060581] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-09. [2025-10-20T21:41:17.064767] [DEBUG] [VALIDATOR ] UNP: All expected trading days are present. [2025-10-20T21:41:17.066061] [DEBUG] [VALIDATOR ] UNP: Primary data validation passed. [2025-10-20T21:41:17.067205] [INFO ] [MAIN ] UNP: Fetched primary data is valid. [2025-10-20T21:41:17.067360] [DEBUG] [LOCAL ] UNP: Saving primary data. [2025-10-20T21:41:17.068877] [DEBUG] [LOCAL ] UNP: Trimmed 77127 records to 53362 (keeping 200 days of data) [2025-10-20T21:41:17.400481] [DEBUG] [LOCAL ] UNP: Primary data saved (53362 records). [2025-10-20T21:41:17.400634] [INFO ] [MAIN ] UNP: Full history saved. [2025-10-20T21:41:17.400755] [INFO ] [MAIN ] Update Progress: 2186/2367 symbols completed. UNP took 32.7s. Estimated remaining: 23m 17s. Rate: 7.8 symbols/min [2025-10-20T21:41:17.572319] [INFO ] [MAIN ] UPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:17.621198] [DEBUG] [LOCAL ] UPS: Loaded 54499 existing records from primary storage. [2025-10-20T21:41:17.621523] [INFO ] [MAIN ] UPS: Loaded 54499 existing records. [2025-10-20T21:41:17.798983] [WARN ] [MAIN ] UPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:17.799245] [INFO ] [MAIN ] UPS: Wiping existing primary data before full refresh. [2025-10-20T21:41:17.799382] [INFO ] [LOCAL ] UPS: Starting deletion of primary data. [2025-10-20T21:41:17.799595] [DEBUG] [LOCAL ] UPS: Deleted primary data file. [2025-10-20T21:41:17.799634] [INFO ] [MAIN ] UPS: Starting full primary refresh. [2025-10-20T21:41:17.867185] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-08. [2025-10-20T21:41:18.177824] [INFO ] [REMOTE ] UPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:18.210578] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-09 (2.76 seconds). [2025-10-20T21:41:18.651417] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-01. [2025-10-20T21:41:19.203634] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-09 (2.14 seconds). [2025-10-20T21:41:19.451415] [INFO ] [REMOTE ] UP: Fetching primary data for 2025-10. [2025-10-20T21:41:19.590965] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T21:41:20.251381] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-09. [2025-10-20T21:41:20.924368] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-01 (2.27 seconds). [2025-10-20T21:41:21.051342] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-02. [2025-10-20T21:41:21.372329] [INFO ] [REMOTE ] UP: Incremental fetch complete for 2025-10 (1.92 seconds). [2025-10-20T21:41:21.379800] [INFO ] [REMOTE ] UP: Full fetch complete (63084 records). [2025-10-20T21:41:21.379910] [DEBUG] [VALIDATOR ] UP: Validating primary data. [2025-10-20T21:41:21.783582] [DEBUG] [VALIDATOR ] UP: All expected trading days are present. [2025-10-20T21:41:21.783739] [DEBUG] [VALIDATOR ] UP: Primary data validation passed. [2025-10-20T21:41:21.784772] [INFO ] [MAIN ] UP: Fetched primary data is valid. [2025-10-20T21:41:21.784850] [DEBUG] [LOCAL ] UP: Saving primary data. [2025-10-20T21:41:21.812534] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T21:41:21.811338] [DEBUG] [LOCAL ] UP: Trimmed 63084 records to 44329 (keeping 200 days of data) [2025-10-20T21:41:21.866224] [INFO ] [REMOTE ] UNM: Fetching primary data for 2025-10. [2025-10-20T21:41:22.067734] [DEBUG] [LOCAL ] UP: Primary data saved (44329 records). [2025-10-20T21:41:22.067864] [INFO ] [MAIN ] UP: Full history saved. [2025-10-20T21:41:22.067975] [INFO ] [MAIN ] Update Progress: 2187/2367 symbols completed. UP took 24.8s. Estimated remaining: 23m 9s. Rate: 7.8 symbols/min [2025-10-20T21:41:22.253745] [INFO ] [MAIN ] URI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:22.253936] [DEBUG] [LOCAL ] URI: No existing data file found in primary storage. [2025-10-20T21:41:22.253973] [WARN ] [MAIN ] URI: No existing data found, performing full refresh instead. [2025-10-20T21:41:22.254080] [INFO ] [MAIN ] URI: Wiping existing primary data before full refresh. [2025-10-20T21:41:22.254120] [INFO ] [LOCAL ] URI: Starting deletion of primary data. [2025-10-20T21:41:22.254161] [DEBUG] [LOCAL ] URI: No primary data file found to delete. [2025-10-20T21:41:22.254191] [INFO ] [MAIN ] URI: Starting full primary refresh. [2025-10-20T21:41:22.609271] [INFO ] [REMOTE ] URI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:22.651369] [INFO ] [REMOTE ] UNH: Fetching primary data for 2025-10. [2025-10-20T21:41:23.092768] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-02 (2.04 seconds). [2025-10-20T21:41:23.154484] [INFO ] [REMOTE ] UNM: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T21:41:23.161205] [INFO ] [REMOTE ] UNM: Full fetch complete (70006 records). [2025-10-20T21:41:23.161338] [DEBUG] [VALIDATOR ] UNM: Validating primary data. [2025-10-20T21:41:23.454486] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-03. [2025-10-20T21:41:23.562568] [DEBUG] [VALIDATOR ] UNM: All expected trading days are present. [2025-10-20T21:41:23.562815] [DEBUG] [VALIDATOR ] UNM: Primary data validation passed. [2025-10-20T21:41:23.563900] [INFO ] [MAIN ] UNM: Fetched primary data is valid. [2025-10-20T21:41:23.563980] [DEBUG] [LOCAL ] UNM: Saving primary data. [2025-10-20T21:41:23.585041] [INFO ] [REMOTE ] UNH: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T21:41:23.587897] [DEBUG] [LOCAL ] UNM: Trimmed 70006 records to 48078 (keeping 200 days of data) [2025-10-20T21:41:23.593527] [INFO ] [REMOTE ] UNH: Full fetch complete (77960 records). [2025-10-20T21:41:23.593617] [DEBUG] [VALIDATOR ] UNH: Validating primary data. [2025-10-20T21:41:24.253550] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-01. [2025-10-20T21:41:24.316914] [DEBUG] [VALIDATOR ] UNH: All expected trading days are present. [2025-10-20T21:41:24.317972] [DEBUG] [VALIDATOR ] UNH: Primary data validation passed. [2025-10-20T21:41:24.319296] [INFO ] [MAIN ] UNH: Fetched primary data is valid. [2025-10-20T21:41:24.319375] [DEBUG] [LOCAL ] UNH: Saving primary data. [2025-10-20T21:41:24.337334] [DEBUG] [LOCAL ] UNM: Primary data saved (48078 records). [2025-10-20T21:41:24.337528] [INFO ] [MAIN ] UNM: Full history saved. [2025-10-20T21:41:24.333749] [DEBUG] [LOCAL ] UNH: Trimmed 77960 records to 53819 (keeping 200 days of data) [2025-10-20T21:41:24.338062] [INFO ] [MAIN ] Update Progress: 2188/2367 symbols completed. UNM took 42.6s. Estimated remaining: 23m 1s. Rate: 7.8 symbols/min [2025-10-20T21:41:24.637246] [INFO ] [MAIN ] USA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:24.885233] [DEBUG] [LOCAL ] UNH: Primary data saved (53819 records). [2025-10-20T21:41:24.885400] [INFO ] [MAIN ] UNH: Full history saved. [2025-10-20T21:41:24.885549] [INFO ] [MAIN ] Update Progress: 2189/2367 symbols completed. UNH took 49.5s. Estimated remaining: 22m 53s. Rate: 7.8 symbols/min [2025-10-20T21:41:24.886129] [DEBUG] [LOCAL ] USA: Loaded 36917 existing records from primary storage. [2025-10-20T21:41:24.917827] [INFO ] [MAIN ] USA: Loaded 36917 existing records. [2025-10-20T21:41:25.223468] [INFO ] [MAIN ] USAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:25.243829] [WARN ] [MAIN ] USA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:25.244478] [INFO ] [MAIN ] USA: Wiping existing primary data before full refresh. [2025-10-20T21:41:25.244633] [INFO ] [LOCAL ] USA: Starting deletion of primary data. [2025-10-20T21:41:25.244863] [DEBUG] [LOCAL ] USA: Deleted primary data file. [2025-10-20T21:41:25.244906] [INFO ] [MAIN ] USA: Starting full primary refresh. [2025-10-20T21:41:25.464566] [DEBUG] [LOCAL ] USAC: Loaded 22019 existing records from primary storage. [2025-10-20T21:41:25.527035] [INFO ] [MAIN ] USAC: Loaded 22019 existing records. [2025-10-20T21:41:25.750799] [INFO ] [REMOTE ] USA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:25.804893] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-01. [2025-10-20T21:41:25.828402] [WARN ] [MAIN ] USAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:25.830037] [INFO ] [MAIN ] USAC: Wiping existing primary data before full refresh. [2025-10-20T21:41:25.830216] [INFO ] [LOCAL ] USAC: Starting deletion of primary data. [2025-10-20T21:41:25.830474] [DEBUG] [LOCAL ] USAC: Deleted primary data file. [2025-10-20T21:41:25.830554] [INFO ] [MAIN ] USAC: Starting full primary refresh. [2025-10-20T21:41:26.006853] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-03 (2.55 seconds). [2025-10-20T21:41:26.125353] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-04. [2025-10-20T21:41:26.127205] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-01 (1.87 seconds). [2025-10-20T21:41:26.297472] [INFO ] [REMOTE ] USAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:26.651389] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-02. [2025-10-20T21:41:27.293830] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-01 (1.49 seconds). [2025-10-20T21:41:27.451399] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-02. [2025-10-20T21:41:28.100753] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-02 (1.45 seconds). [2025-10-20T21:41:28.251333] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-03. [2025-10-20T21:41:28.550308] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-04 (2.42 seconds). [2025-10-20T21:41:28.729211] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-02 (1.28 seconds). [2025-10-20T21:41:29.051376] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-05. [2025-10-20T21:41:29.856508] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-03. [2025-10-20T21:41:29.874617] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T21:41:30.651438] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-01. [2025-10-20T21:41:31.304381] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-03 (1.45 seconds). [2025-10-20T21:41:31.363212] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-05 (2.31 seconds). [2025-10-20T21:41:31.451373] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-06. [2025-10-20T21:41:31.956178] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T21:41:32.251415] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-04. [2025-10-20T21:41:33.051400] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-02. [2025-10-20T21:41:33.622537] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-06 (2.17 seconds). [2025-10-20T21:41:33.708939] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-04 (1.46 seconds). [2025-10-20T21:41:33.851384] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-07. [2025-10-20T21:41:33.973016] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T21:41:34.651416] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-05. [2025-10-20T21:41:35.451394] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-03. [2025-10-20T21:41:35.973202] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T21:41:36.115428] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-07 (2.26 seconds). [2025-10-20T21:41:36.251429] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-08. [2025-10-20T21:41:36.756490] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-03 (1.31 seconds). [2025-10-20T21:41:37.051385] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-04. [2025-10-20T21:41:37.851447] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-04. [2025-10-20T21:41:38.186010] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T21:41:38.663498] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-06. [2025-10-20T21:41:38.669999] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-08 (2.42 seconds). [2025-10-20T21:41:39.451358] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-05. [2025-10-20T21:41:39.630130] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:41:39.991696] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-06 (1.33 seconds). [2025-10-20T21:41:40.251434] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-07. [2025-10-20T21:41:40.369624] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T21:41:41.051412] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-06. [2025-10-20T21:41:41.702416] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-07 (1.45 seconds). [2025-10-20T21:41:41.851397] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-08. [2025-10-20T21:41:42.208142] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T21:41:42.651460] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-09. [2025-10-20T21:41:43.196345] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T21:41:43.451397] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-05. [2025-10-20T21:41:44.251408] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-07. [2025-10-20T21:41:44.943082] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-05 (1.49 seconds). [2025-10-20T21:41:45.000341] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-09 (2.35 seconds). [2025-10-20T21:41:45.051388] [INFO ] [REMOTE ] UPS: Fetching primary data for 2025-10. [2025-10-20T21:41:45.214884] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-07 (0.96 seconds). [2025-10-20T21:41:45.776117] [INFO ] [REMOTE ] UPS: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T21:41:45.784040] [INFO ] [REMOTE ] UPS: Full fetch complete (77880 records). [2025-10-20T21:41:45.784196] [DEBUG] [VALIDATOR ] UPS: Validating primary data. [2025-10-20T21:41:45.856497] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-09. [2025-10-20T21:41:46.181529] [DEBUG] [VALIDATOR ] UPS: All expected trading days are present. [2025-10-20T21:41:46.181612] [DEBUG] [VALIDATOR ] UPS: Primary data validation passed. [2025-10-20T21:41:46.182761] [INFO ] [MAIN ] UPS: Fetched primary data is valid. [2025-10-20T21:41:46.182823] [DEBUG] [LOCAL ] UPS: Saving primary data. [2025-10-20T21:41:46.184248] [DEBUG] [LOCAL ] UPS: Trimmed 77880 records to 53720 (keeping 200 days of data) [2025-10-20T21:41:46.516189] [DEBUG] [LOCAL ] UPS: Primary data saved (53720 records). [2025-10-20T21:41:46.516365] [INFO ] [MAIN ] UPS: Full history saved. [2025-10-20T21:41:46.516465] [INFO ] [MAIN ] Update Progress: 2190/2367 symbols completed. UPS took 29.1s. Estimated remaining: 22m 46s. Rate: 7.8 symbols/min [2025-10-20T21:41:46.656760] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-06. [2025-10-20T21:41:46.690494] [INFO ] [MAIN ] USB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:46.740469] [DEBUG] [LOCAL ] USB: Loaded 54468 existing records from primary storage. [2025-10-20T21:41:46.740826] [INFO ] [MAIN ] USB: Loaded 54468 existing records. [2025-10-20T21:41:46.920967] [WARN ] [MAIN ] USB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:46.921165] [INFO ] [MAIN ] USB: Wiping existing primary data before full refresh. [2025-10-20T21:41:46.921213] [INFO ] [LOCAL ] USB: Starting deletion of primary data. [2025-10-20T21:41:46.921394] [DEBUG] [LOCAL ] USB: Deleted primary data file. [2025-10-20T21:41:46.921433] [INFO ] [MAIN ] USB: Starting full primary refresh. [2025-10-20T21:41:47.303920] [INFO ] [REMOTE ] USB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:47.371555] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-09 (1.52 seconds). [2025-10-20T21:41:47.451446] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-01. [2025-10-20T21:41:48.000358] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T21:41:48.251378] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-08. [2025-10-20T21:41:49.021621] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T21:41:49.051368] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-07. [2025-10-20T21:41:49.308175] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T21:41:49.851372] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-02. [2025-10-20T21:41:50.261403] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T21:41:50.651420] [INFO ] [REMOTE ] USA: Fetching primary data for 2025-10. [2025-10-20T21:41:51.451391] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-08. [2025-10-20T21:41:51.901945] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T21:41:52.144292] [INFO ] [REMOTE ] USA: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T21:41:52.150568] [INFO ] [REMOTE ] USA: Full fetch complete (52517 records). [2025-10-20T21:41:52.150767] [DEBUG] [VALIDATOR ] USA: Validating primary data. [2025-10-20T21:41:52.251737] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-09. [2025-10-20T21:41:52.531145] [DEBUG] [VALIDATOR ] USA: All expected trading days are present. [2025-10-20T21:41:52.531256] [DEBUG] [VALIDATOR ] USA: Primary data validation passed. [2025-10-20T21:41:52.532262] [INFO ] [MAIN ] USA: Fetched primary data is valid. [2025-10-20T21:41:52.532341] [DEBUG] [LOCAL ] USA: Saving primary data. [2025-10-20T21:41:52.533688] [DEBUG] [LOCAL ] USA: Trimmed 52517 records to 36476 (keeping 200 days of data) [2025-10-20T21:41:52.644248] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-08 (1.19 seconds). [2025-10-20T21:41:52.768735] [DEBUG] [LOCAL ] USA: Primary data saved (36476 records). [2025-10-20T21:41:52.768882] [INFO ] [MAIN ] USA: Full history saved. [2025-10-20T21:41:52.768964] [INFO ] [MAIN ] Update Progress: 2191/2367 symbols completed. USA took 28.4s. Estimated remaining: 22m 38s. Rate: 7.8 symbols/min [2025-10-20T21:41:52.933708] [INFO ] [MAIN ] USFD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:52.980462] [DEBUG] [LOCAL ] USFD: Loaded 51936 existing records from primary storage. [2025-10-20T21:41:52.980798] [INFO ] [MAIN ] USFD: Loaded 51936 existing records. [2025-10-20T21:41:53.056460] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-03. [2025-10-20T21:41:53.157387] [WARN ] [MAIN ] USFD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:53.157575] [INFO ] [MAIN ] USFD: Wiping existing primary data before full refresh. [2025-10-20T21:41:53.157620] [INFO ] [LOCAL ] USFD: Starting deletion of primary data. [2025-10-20T21:41:53.157849] [DEBUG] [LOCAL ] USFD: Deleted primary data file. [2025-10-20T21:41:53.157925] [INFO ] [MAIN ] USFD: Starting full primary refresh. [2025-10-20T21:41:53.507308] [INFO ] [REMOTE ] USFD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:53.522296] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T21:41:53.851371] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-01. [2025-10-20T21:41:54.651410] [INFO ] [REMOTE ] USAC: Fetching primary data for 2025-10. [2025-10-20T21:41:55.066243] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T21:41:55.451414] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-09. [2025-10-20T21:41:55.510616] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:41:55.597759] [INFO ] [REMOTE ] USAC: Incremental fetch complete for 2025-10 (0.95 seconds). [2025-10-20T21:41:55.601177] [INFO ] [REMOTE ] USAC: Full fetch complete (35178 records). [2025-10-20T21:41:55.601252] [DEBUG] [VALIDATOR ] USAC: Validating primary data. [2025-10-20T21:41:55.975296] [DEBUG] [VALIDATOR ] USAC: All expected trading days are present. [2025-10-20T21:41:55.975377] [DEBUG] [VALIDATOR ] USAC: Primary data validation passed. [2025-10-20T21:41:55.976056] [INFO ] [MAIN ] USAC: Fetched primary data is valid. [2025-10-20T21:41:55.976116] [DEBUG] [LOCAL ] USAC: Saving primary data. [2025-10-20T21:41:55.976839] [DEBUG] [LOCAL ] USAC: Trimmed 35178 records to 21832 (keeping 200 days of data) [2025-10-20T21:41:56.100953] [DEBUG] [LOCAL ] USAC: Primary data saved (21832 records). [2025-10-20T21:41:56.101073] [INFO ] [MAIN ] USAC: Full history saved. [2025-10-20T21:41:56.101143] [INFO ] [MAIN ] Update Progress: 2192/2367 symbols completed. USAC took 31.2s. Estimated remaining: 22m 30s. Rate: 7.8 symbols/min [2025-10-20T21:41:56.258543] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-02. [2025-10-20T21:41:56.266811] [INFO ] [MAIN ] USNA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:41:56.285521] [DEBUG] [LOCAL ] USNA: Loaded 18290 existing records from primary storage. [2025-10-20T21:41:56.285771] [INFO ] [MAIN ] USNA: Loaded 18290 existing records. [2025-10-20T21:41:56.459340] [WARN ] [MAIN ] USNA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:41:56.459586] [INFO ] [MAIN ] USNA: Wiping existing primary data before full refresh. [2025-10-20T21:41:56.459637] [INFO ] [LOCAL ] USNA: Starting deletion of primary data. [2025-10-20T21:41:56.459843] [DEBUG] [LOCAL ] USNA: Deleted primary data file. [2025-10-20T21:41:56.459897] [INFO ] [MAIN ] USNA: Starting full primary refresh. [2025-10-20T21:41:56.821999] [INFO ] [REMOTE ] USNA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:41:56.903573] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-09 (1.45 seconds). [2025-10-20T21:41:57.051423] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-04. [2025-10-20T21:41:57.851404] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-01. [2025-10-20T21:41:57.919387] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T21:41:58.651418] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-03. [2025-10-20T21:41:58.933527] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-01 (1.08 seconds). [2025-10-20T21:41:59.084706] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-04 (2.03 seconds). [2025-10-20T21:41:59.451399] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-02. [2025-10-20T21:42:00.251439] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-05. [2025-10-20T21:42:00.389190] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T21:42:00.431874] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-02 (0.98 seconds). [2025-10-20T21:42:01.051391] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-04. [2025-10-20T21:42:01.851371] [INFO ] [REMOTE ] URI: Fetching primary data for 2025-10. [2025-10-20T21:42:02.221398] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-05 (1.97 seconds). [2025-10-20T21:42:02.416813] [INFO ] [REMOTE ] URI: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T21:42:02.421481] [INFO ] [REMOTE ] URI: Full fetch complete (52833 records). [2025-10-20T21:42:02.421625] [DEBUG] [VALIDATOR ] URI: Validating primary data. [2025-10-20T21:42:02.652078] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-03. [2025-10-20T21:42:02.817922] [DEBUG] [VALIDATOR ] URI: All expected trading days are present. [2025-10-20T21:42:02.818075] [DEBUG] [VALIDATOR ] URI: Primary data validation passed. [2025-10-20T21:42:02.819013] [INFO ] [MAIN ] URI: Fetched primary data is valid. [2025-10-20T21:42:02.819118] [DEBUG] [LOCAL ] URI: Saving primary data. [2025-10-20T21:42:02.836295] [DEBUG] [LOCAL ] URI: Trimmed 52833 records to 34532 (keeping 200 days of data) [2025-10-20T21:42:02.848750] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T21:42:03.067995] [DEBUG] [LOCAL ] URI: Primary data saved (34532 records). [2025-10-20T21:42:03.068122] [INFO ] [MAIN ] URI: Full history saved. [2025-10-20T21:42:03.068188] [INFO ] [MAIN ] Update Progress: 2193/2367 symbols completed. URI took 41.0s. Estimated remaining: 22m 22s. Rate: 7.8 symbols/min [2025-10-20T21:42:03.233203] [INFO ] [MAIN ] USPH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:03.250198] [DEBUG] [LOCAL ] USPH: Loaded 17392 existing records from primary storage. [2025-10-20T21:42:03.250405] [INFO ] [MAIN ] USPH: Loaded 17392 existing records. [2025-10-20T21:42:03.424195] [WARN ] [MAIN ] USPH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:03.424393] [INFO ] [MAIN ] USPH: Wiping existing primary data before full refresh. [2025-10-20T21:42:03.424446] [INFO ] [LOCAL ] USPH: Starting deletion of primary data. [2025-10-20T21:42:03.424605] [DEBUG] [LOCAL ] USPH: Deleted primary data file. [2025-10-20T21:42:03.424671] [INFO ] [MAIN ] USPH: Starting full primary refresh. [2025-10-20T21:42:03.456568] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-05. [2025-10-20T21:42:03.781425] [INFO ] [REMOTE ] USPH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:03.799727] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:42:04.251359] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-01. [2025-10-20T21:42:05.051339] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-04. [2025-10-20T21:42:05.135750] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-01 (0.88 seconds). [2025-10-20T21:42:05.263254] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:42:05.851425] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-02. [2025-10-20T21:42:06.296007] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T21:42:06.651354] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-06. [2025-10-20T21:42:06.757980] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T21:42:07.451431] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-05. [2025-10-20T21:42:08.251369] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-06. [2025-10-20T21:42:08.425889] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-06 (1.77 seconds). [2025-10-20T21:42:08.580815] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T21:42:09.051413] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-07. [2025-10-20T21:42:09.851421] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-06. [2025-10-20T21:42:10.044942] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T21:42:10.651380] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-03. [2025-10-20T21:42:10.749300] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T21:42:10.788145] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-06 (0.94 seconds). [2025-10-20T21:42:11.451357] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-08. [2025-10-20T21:42:11.674403] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T21:42:12.251423] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-07. [2025-10-20T21:42:13.051379] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-07. [2025-10-20T21:42:13.139524] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-08 (1.69 seconds). [2025-10-20T21:42:13.178409] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T21:42:13.851365] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-04. [2025-10-20T21:42:14.651354] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-08. [2025-10-20T21:42:14.899698] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T21:42:15.038630] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T21:42:15.451413] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-09. [2025-10-20T21:42:15.503006] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-08 (0.85 seconds). [2025-10-20T21:42:16.251359] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-05. [2025-10-20T21:42:16.979462] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:42:17.051343] [INFO ] [REMOTE ] USFD: Fetching primary data for 2025-10. [2025-10-20T21:42:17.253927] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T21:42:17.851400] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-09. [2025-10-20T21:42:18.651382] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-08. [2025-10-20T21:42:18.728437] [INFO ] [REMOTE ] USFD: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:42:18.745609] [INFO ] [REMOTE ] USFD: Full fetch complete (74358 records). [2025-10-20T21:42:18.745956] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-09 (0.89 seconds). [2025-10-20T21:42:18.749395] [DEBUG] [VALIDATOR ] USFD: Validating primary data. [2025-10-20T21:42:19.161803] [DEBUG] [VALIDATOR ] USFD: All expected trading days are present. [2025-10-20T21:42:19.161900] [DEBUG] [VALIDATOR ] USFD: Primary data validation passed. [2025-10-20T21:42:19.163126] [INFO ] [MAIN ] USFD: Fetched primary data is valid. [2025-10-20T21:42:19.163209] [DEBUG] [LOCAL ] USFD: Saving primary data. [2025-10-20T21:42:19.164771] [DEBUG] [LOCAL ] USFD: Trimmed 74358 records to 51212 (keeping 200 days of data) [2025-10-20T21:42:19.461344] [INFO ] [REMOTE ] USNA: Fetching primary data for 2025-10. [2025-10-20T21:42:19.483698] [DEBUG] [LOCAL ] USFD: Primary data saved (51212 records). [2025-10-20T21:42:19.483817] [INFO ] [MAIN ] USFD: Full history saved. [2025-10-20T21:42:19.484089] [INFO ] [MAIN ] Update Progress: 2194/2367 symbols completed. USFD took 26.7s. Estimated remaining: 22m 15s. Rate: 7.8 symbols/min [2025-10-20T21:42:19.673206] [INFO ] [MAIN ] UTF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:19.706596] [DEBUG] [LOCAL ] UTF: Loaded 31389 existing records from primary storage. [2025-10-20T21:42:19.706894] [INFO ] [MAIN ] UTF: Loaded 31389 existing records. [2025-10-20T21:42:19.888172] [WARN ] [MAIN ] UTF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:19.888431] [INFO ] [MAIN ] UTF: Wiping existing primary data before full refresh. [2025-10-20T21:42:19.888480] [INFO ] [LOCAL ] UTF: Starting deletion of primary data. [2025-10-20T21:42:19.888705] [DEBUG] [LOCAL ] UTF: Deleted primary data file. [2025-10-20T21:42:19.888763] [INFO ] [MAIN ] UTF: Starting full primary refresh. [2025-10-20T21:42:20.256383] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-06. [2025-10-20T21:42:20.288136] [INFO ] [REMOTE ] UTF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:20.427065] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:42:20.456460] [INFO ] [REMOTE ] USNA: Incremental fetch complete for 2025-10 (1.00 seconds). [2025-10-20T21:42:20.459950] [INFO ] [REMOTE ] USNA: Full fetch complete (27026 records). [2025-10-20T21:42:20.460022] [DEBUG] [VALIDATOR ] USNA: Validating primary data. [2025-10-20T21:42:20.824813] [DEBUG] [VALIDATOR ] USNA: All expected trading days are present. [2025-10-20T21:42:20.824944] [DEBUG] [VALIDATOR ] USNA: Primary data validation passed. [2025-10-20T21:42:20.825713] [INFO ] [MAIN ] USNA: Fetched primary data is valid. [2025-10-20T21:42:20.825772] [DEBUG] [LOCAL ] USNA: Saving primary data. [2025-10-20T21:42:20.826396] [DEBUG] [LOCAL ] USNA: Trimmed 27026 records to 18032 (keeping 200 days of data) [2025-10-20T21:42:20.927024] [DEBUG] [LOCAL ] USNA: Primary data saved (18032 records). [2025-10-20T21:42:20.927157] [INFO ] [MAIN ] USNA: Full history saved. [2025-10-20T21:42:20.927232] [INFO ] [MAIN ] Update Progress: 2195/2367 symbols completed. USNA took 24.8s. Estimated remaining: 22m 7s. Rate: 7.8 symbols/min [2025-10-20T21:42:21.064355] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-01. [2025-10-20T21:42:21.096958] [INFO ] [MAIN ] UTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:21.134998] [DEBUG] [LOCAL ] UTI: Loaded 41052 existing records from primary storage. [2025-10-20T21:42:21.135295] [INFO ] [MAIN ] UTI: Loaded 41052 existing records. [2025-10-20T21:42:21.313605] [WARN ] [MAIN ] UTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:21.313989] [INFO ] [MAIN ] UTI: Wiping existing primary data before full refresh. [2025-10-20T21:42:21.314098] [INFO ] [LOCAL ] UTI: Starting deletion of primary data. [2025-10-20T21:42:21.314328] [DEBUG] [LOCAL ] UTI: Deleted primary data file. [2025-10-20T21:42:21.314384] [INFO ] [MAIN ] UTI: Starting full primary refresh. [2025-10-20T21:42:21.445793] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T21:42:21.676258] [INFO ] [REMOTE ] UTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:21.851340] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-09. [2025-10-20T21:42:22.315186] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-01 (1.25 seconds). [2025-10-20T21:42:22.651386] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-01. [2025-10-20T21:42:23.451365] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-07. [2025-10-20T21:42:23.744738] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-09 (1.89 seconds). [2025-10-20T21:42:24.225234] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-01 (1.57 seconds). [2025-10-20T21:42:24.251338] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-02. [2025-10-20T21:42:24.532510] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-07 (1.08 seconds). [2025-10-20T21:42:25.051416] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-08. [2025-10-20T21:42:25.754560] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T21:42:25.813710] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-08 (0.76 seconds). [2025-10-20T21:42:25.851357] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-03. [2025-10-20T21:42:26.651377] [INFO ] [REMOTE ] USB: Fetching primary data for 2025-10. [2025-10-20T21:42:27.345116] [INFO ] [REMOTE ] USB: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T21:42:27.353097] [INFO ] [REMOTE ] USB: Full fetch complete (77867 records). [2025-10-20T21:42:27.353280] [DEBUG] [VALIDATOR ] USB: Validating primary data. [2025-10-20T21:42:27.452532] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-09. [2025-10-20T21:42:27.744542] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T21:42:27.796906] [DEBUG] [VALIDATOR ] USB: All expected trading days are present. [2025-10-20T21:42:27.797071] [DEBUG] [VALIDATOR ] USB: Primary data validation passed. [2025-10-20T21:42:27.798788] [INFO ] [MAIN ] USB: Fetched primary data is valid. [2025-10-20T21:42:27.798867] [DEBUG] [LOCAL ] USB: Saving primary data. [2025-10-20T21:42:27.800973] [DEBUG] [LOCAL ] USB: Trimmed 77867 records to 53688 (keeping 200 days of data) [2025-10-20T21:42:28.144959] [DEBUG] [LOCAL ] USB: Primary data saved (53688 records). [2025-10-20T21:42:28.145097] [INFO ] [MAIN ] USB: Full history saved. [2025-10-20T21:42:28.145176] [INFO ] [MAIN ] Update Progress: 2196/2367 symbols completed. USB took 41.6s. Estimated remaining: 21m 59s. Rate: 7.8 symbols/min [2025-10-20T21:42:28.256486] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-04. [2025-10-20T21:42:28.322685] [INFO ] [MAIN ] UTL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:28.337067] [DEBUG] [LOCAL ] UTL: Loaded 12006 existing records from primary storage. [2025-10-20T21:42:28.337283] [INFO ] [MAIN ] UTL: Loaded 12006 existing records. [2025-10-20T21:42:28.533343] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-09 (1.08 seconds). [2025-10-20T21:42:28.532679] [WARN ] [MAIN ] UTL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:28.537841] [INFO ] [MAIN ] UTL: Wiping existing primary data before full refresh. [2025-10-20T21:42:28.537936] [INFO ] [LOCAL ] UTL: Starting deletion of primary data. [2025-10-20T21:42:28.538155] [DEBUG] [LOCAL ] UTL: Deleted primary data file. [2025-10-20T21:42:28.538210] [INFO ] [MAIN ] UTL: Starting full primary refresh. [2025-10-20T21:42:28.884735] [INFO ] [REMOTE ] UTL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:29.051409] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-01. [2025-10-20T21:42:29.694037] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-01 (0.64 seconds). [2025-10-20T21:42:29.851351] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-02. [2025-10-20T21:42:29.894408] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:42:30.651372] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-05. [2025-10-20T21:42:31.084482] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T21:42:31.451364] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-03. [2025-10-20T21:42:32.251351] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-02. [2025-10-20T21:42:32.349198] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T21:42:32.714184] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T21:42:32.816769] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-02 (0.57 seconds). [2025-10-20T21:42:33.051424] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-03. [2025-10-20T21:42:33.822083] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T21:42:33.851343] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-04. [2025-10-20T21:42:34.641604] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-04 (0.79 seconds). [2025-10-20T21:42:34.651327] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-05. [2025-10-20T21:42:35.441153] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T21:42:35.451317] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-06. [2025-10-20T21:42:36.217394] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T21:42:36.251346] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-07. [2025-10-20T21:42:37.051382] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-06. [2025-10-20T21:42:37.091706] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T21:42:37.851405] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-08. [2025-10-20T21:42:38.548154] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T21:42:38.651464] [INFO ] [REMOTE ] USPH: Fetching primary data for 2025-10. [2025-10-20T21:42:38.933005] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T21:42:39.351017] [INFO ] [REMOTE ] USPH: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T21:42:39.354054] [INFO ] [REMOTE ] USPH: Full fetch complete (23032 records). [2025-10-20T21:42:39.354143] [DEBUG] [VALIDATOR ] USPH: Validating primary data. [2025-10-20T21:42:39.456351] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-09. [2025-10-20T21:42:39.743489] [DEBUG] [VALIDATOR ] USPH: All expected trading days are present. [2025-10-20T21:42:39.743584] [DEBUG] [VALIDATOR ] USPH: Primary data validation passed. [2025-10-20T21:42:39.744142] [INFO ] [MAIN ] USPH: Fetched primary data is valid. [2025-10-20T21:42:39.744208] [DEBUG] [LOCAL ] USPH: Saving primary data. [2025-10-20T21:42:39.745419] [DEBUG] [LOCAL ] USPH: Trimmed 23032 records to 17179 (keeping 200 days of data) [2025-10-20T21:42:39.856935] [DEBUG] [LOCAL ] USPH: Primary data saved (17179 records). [2025-10-20T21:42:39.857105] [INFO ] [MAIN ] USPH: Full history saved. [2025-10-20T21:42:39.857195] [INFO ] [MAIN ] Update Progress: 2197/2367 symbols completed. USPH took 36.8s. Estimated remaining: 21m 52s. Rate: 7.8 symbols/min [2025-10-20T21:42:40.251431] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-04. [2025-10-20T21:42:40.434010] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T21:42:41.051454] [INFO ] [REMOTE ] UTL: Fetching primary data for 2025-10. [2025-10-20T21:42:41.381320] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-04 (1.13 seconds). [2025-10-20T21:42:41.851405] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-05. [2025-10-20T21:42:41.862296] [INFO ] [REMOTE ] UTL: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T21:42:41.864912] [INFO ] [REMOTE ] UTL: Full fetch complete (15798 records). [2025-10-20T21:42:41.865005] [DEBUG] [VALIDATOR ] UTL: Validating primary data. [2025-10-20T21:42:42.244737] [DEBUG] [VALIDATOR ] UTL: All expected trading days are present. [2025-10-20T21:42:42.244831] [DEBUG] [VALIDATOR ] UTL: Primary data validation passed. [2025-10-20T21:42:42.245219] [INFO ] [MAIN ] UTL: Fetched primary data is valid. [2025-10-20T21:42:42.245258] [DEBUG] [LOCAL ] UTL: Saving primary data. [2025-10-20T21:42:42.245904] [DEBUG] [LOCAL ] UTL: Trimmed 15798 records to 11878 (keeping 200 days of data) [2025-10-20T21:42:42.317522] [DEBUG] [LOCAL ] UTL: Primary data saved (11878 records). [2025-10-20T21:42:42.317680] [INFO ] [MAIN ] UTL: Full history saved. [2025-10-20T21:42:42.317774] [INFO ] [MAIN ] Update Progress: 2198/2367 symbols completed. UTL took 14.2s. Estimated remaining: 21m 44s. Rate: 7.8 symbols/min [2025-10-20T21:42:42.651485] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-07. [2025-10-20T21:42:43.172786] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T21:42:43.451355] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-06. [2025-10-20T21:42:43.795431] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-07 (1.14 seconds). [2025-10-20T21:42:44.251465] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-08. [2025-10-20T21:42:44.690301] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T21:42:45.051424] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-07. [2025-10-20T21:42:45.800931] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-08 (1.55 seconds). [2025-10-20T21:42:45.851359] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-09. [2025-10-20T21:42:46.377535] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T21:42:46.651491] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-08. [2025-10-20T21:42:47.059917] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T21:42:47.450891] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T21:42:47.456243] [INFO ] [REMOTE ] UTI: Fetching primary data for 2025-10. [2025-10-20T21:42:48.251367] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-09. [2025-10-20T21:42:48.699513] [INFO ] [REMOTE ] UTI: Incremental fetch complete for 2025-10 (1.24 seconds). [2025-10-20T21:42:48.705425] [INFO ] [REMOTE ] UTI: Full fetch complete (58726 records). [2025-10-20T21:42:48.705552] [DEBUG] [VALIDATOR ] UTI: Validating primary data. [2025-10-20T21:42:49.097831] [DEBUG] [VALIDATOR ] UTI: All expected trading days are present. [2025-10-20T21:42:49.097942] [DEBUG] [VALIDATOR ] UTI: Primary data validation passed. [2025-10-20T21:42:49.098908] [INFO ] [MAIN ] UTI: Fetched primary data is valid. [2025-10-20T21:42:49.098969] [DEBUG] [LOCAL ] UTI: Saving primary data. [2025-10-20T21:42:49.100228] [DEBUG] [LOCAL ] UTI: Trimmed 58726 records to 40371 (keeping 200 days of data) [2025-10-20T21:42:49.332153] [DEBUG] [LOCAL ] UTI: Primary data saved (40371 records). [2025-10-20T21:42:49.332349] [INFO ] [MAIN ] UTI: Full history saved. [2025-10-20T21:42:49.332432] [INFO ] [MAIN ] Update Progress: 2199/2367 symbols completed. UTI took 28.4s. Estimated remaining: 21m 36s. Rate: 7.8 symbols/min [2025-10-20T21:42:49.575949] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-09 (1.32 seconds). [2025-10-20T21:42:49.584881] [INFO ] [REMOTE ] UTF: Fetching primary data for 2025-10. [2025-10-20T21:42:51.150095] [INFO ] [REMOTE ] UTF: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T21:42:51.157324] [INFO ] [REMOTE ] UTF: Full fetch complete (43660 records). [2025-10-20T21:42:51.157474] [DEBUG] [VALIDATOR ] UTF: Validating primary data. [2025-10-20T21:42:51.536391] [DEBUG] [VALIDATOR ] UTF: All expected trading days are present. [2025-10-20T21:42:51.536481] [DEBUG] [VALIDATOR ] UTF: Primary data validation passed. [2025-10-20T21:42:51.537377] [INFO ] [MAIN ] UTF: Fetched primary data is valid. [2025-10-20T21:42:51.537438] [DEBUG] [LOCAL ] UTF: Saving primary data. [2025-10-20T21:42:51.538502] [DEBUG] [LOCAL ] UTF: Trimmed 43660 records to 30922 (keeping 200 days of data) [2025-10-20T21:42:51.725409] [DEBUG] [LOCAL ] UTF: Primary data saved (30922 records). [2025-10-20T21:42:51.725666] [INFO ] [MAIN ] UTF: Full history saved. [2025-10-20T21:42:51.726166] [INFO ] [MAIN ] Update Progress: 2200/2367 symbols completed. UTF took 32.2s. Estimated remaining: 21m 28s. Rate: 7.8 symbols/min [2025-10-20T21:42:52.340786] [INFO ] [MAIN ] UVV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:52.413133] [INFO ] [MAIN ] UWMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:52.410324] [INFO ] [MAIN ] UTZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:52.415255] [INFO ] [MAIN ] UVE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:42:52.450209] [DEBUG] [LOCAL ] UVV: Loaded 22980 existing records from primary storage. [2025-10-20T21:42:52.450630] [INFO ] [MAIN ] UVV: Loaded 22980 existing records. [2025-10-20T21:42:52.564287] [DEBUG] [LOCAL ] UVE: Loaded 22261 existing records from primary storage. [2025-10-20T21:42:52.592065] [INFO ] [MAIN ] UVE: Loaded 22261 existing records. [2025-10-20T21:42:52.672826] [DEBUG] [LOCAL ] UTZ: Loaded 45179 existing records from primary storage. [2025-10-20T21:42:52.725076] [INFO ] [MAIN ] UTZ: Loaded 45179 existing records. [2025-10-20T21:42:52.781993] [WARN ] [MAIN ] UVV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:52.828504] [DEBUG] [LOCAL ] UWMC: Loaded 53731 existing records from primary storage. [2025-10-20T21:42:52.848935] [INFO ] [MAIN ] UWMC: Loaded 53731 existing records. [2025-10-20T21:42:52.849071] [INFO ] [MAIN ] UVV: Wiping existing primary data before full refresh. [2025-10-20T21:42:52.874287] [INFO ] [LOCAL ] UVV: Starting deletion of primary data. [2025-10-20T21:42:52.960059] [DEBUG] [LOCAL ] UVV: Deleted primary data file. [2025-10-20T21:42:52.987766] [INFO ] [MAIN ] UVV: Starting full primary refresh. [2025-10-20T21:42:53.171765] [WARN ] [MAIN ] UVE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:53.244771] [INFO ] [MAIN ] UVE: Wiping existing primary data before full refresh. [2025-10-20T21:42:53.244998] [INFO ] [LOCAL ] UVE: Starting deletion of primary data. [2025-10-20T21:42:53.345549] [WARN ] [MAIN ] UTZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:53.304034] [DEBUG] [LOCAL ] UVE: Deleted primary data file. [2025-10-20T21:42:53.385239] [INFO ] [MAIN ] UVE: Starting full primary refresh. [2025-10-20T21:42:53.384978] [WARN ] [MAIN ] UWMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:42:53.396395] [INFO ] [MAIN ] UTZ: Wiping existing primary data before full refresh. [2025-10-20T21:42:53.406169] [INFO ] [LOCAL ] UTZ: Starting deletion of primary data. [2025-10-20T21:42:53.486090] [INFO ] [MAIN ] UWMC: Wiping existing primary data before full refresh. [2025-10-20T21:42:53.517032] [DEBUG] [LOCAL ] UTZ: Deleted primary data file. [2025-10-20T21:42:53.520315] [INFO ] [LOCAL ] UWMC: Starting deletion of primary data. [2025-10-20T21:42:53.520269] [INFO ] [MAIN ] UTZ: Starting full primary refresh. [2025-10-20T21:42:53.590683] [DEBUG] [LOCAL ] UWMC: Deleted primary data file. [2025-10-20T21:42:53.690365] [INFO ] [MAIN ] UWMC: Starting full primary refresh. [2025-10-20T21:42:54.314594] [INFO ] [REMOTE ] UVV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:54.435741] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-01. [2025-10-20T21:42:54.585188] [INFO ] [REMOTE ] UVE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:54.632320] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-01. [2025-10-20T21:42:54.678444] [INFO ] [REMOTE ] UTZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:54.702314] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-01. [2025-10-20T21:42:54.740561] [INFO ] [REMOTE ] UWMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:42:54.740852] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-01. [2025-10-20T21:42:55.583949] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-01 (1.15 seconds). [2025-10-20T21:42:55.587883] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-02. [2025-10-20T21:42:55.724746] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T21:42:55.728697] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-02. [2025-10-20T21:42:55.997145] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-01 (1.29 seconds). [2025-10-20T21:42:56.006561] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-02. [2025-10-20T21:42:56.345804] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-01 (1.60 seconds). [2025-10-20T21:42:56.355741] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-02. [2025-10-20T21:42:56.398449] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-02 (0.67 seconds). [2025-10-20T21:42:56.402771] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-03. [2025-10-20T21:42:56.535280] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-02 (0.95 seconds). [2025-10-20T21:42:57.051420] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-03. [2025-10-20T21:42:57.247290] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T21:42:57.459116] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-03 (1.06 seconds). [2025-10-20T21:42:57.629592] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-02 (1.27 seconds). [2025-10-20T21:42:57.851358] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-04. [2025-10-20T21:42:58.054760] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T21:42:58.651346] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-03. [2025-10-20T21:42:59.002258] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T21:42:59.451357] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-03. [2025-10-20T21:43:00.017138] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-03 (1.37 seconds). [2025-10-20T21:43:00.251429] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-04. [2025-10-20T21:43:01.051406] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-04. [2025-10-20T21:43:01.544259] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T21:43:01.589200] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T21:43:01.851428] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-04. [2025-10-20T21:43:02.651434] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-05. [2025-10-20T21:43:02.694679] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:43:03.451372] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-05. [2025-10-20T21:43:03.612368] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T21:43:03.833228] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-05 (1.18 seconds). [2025-10-20T21:43:04.251377] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-05. [2025-10-20T21:43:05.051452] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-06. [2025-10-20T21:43:05.113776] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T21:43:05.851444] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-06. [2025-10-20T21:43:06.248841] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T21:43:06.293575] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T21:43:06.651421] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-06. [2025-10-20T21:43:07.087928] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-06 (1.24 seconds). [2025-10-20T21:43:07.451418] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-07. [2025-10-20T21:43:08.251365] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-05. [2025-10-20T21:43:08.363902] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:43:08.786159] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-07 (1.33 seconds). [2025-10-20T21:43:09.051357] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-07. [2025-10-20T21:43:09.421724] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:43:09.851391] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-06. [2025-10-20T21:43:10.651329] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-08. [2025-10-20T21:43:10.709457] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T21:43:11.052054] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-06 (1.20 seconds). [2025-10-20T21:43:11.451364] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-08. [2025-10-20T21:43:11.695091] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-08 (1.04 seconds). [2025-10-20T21:43:12.251375] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-09. [2025-10-20T21:43:13.051434] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-07. [2025-10-20T21:43:13.236012] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-08 (1.78 seconds). [2025-10-20T21:43:13.276301] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-09 (1.02 seconds). [2025-10-20T21:43:13.851410] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-07. [2025-10-20T21:43:14.024295] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T21:43:14.651362] [INFO ] [REMOTE ] UVE: Fetching primary data for 2025-10. [2025-10-20T21:43:15.437741] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-07 (1.59 seconds). [2025-10-20T21:43:15.451326] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-08. [2025-10-20T21:43:15.716126] [INFO ] [REMOTE ] UVE: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:43:15.719580] [INFO ] [REMOTE ] UVE: Full fetch complete (30312 records). [2025-10-20T21:43:15.719675] [DEBUG] [VALIDATOR ] UVE: Validating primary data. [2025-10-20T21:43:16.080155] [DEBUG] [VALIDATOR ] UVE: All expected trading days are present. [2025-10-20T21:43:16.080237] [DEBUG] [VALIDATOR ] UVE: Primary data validation passed. [2025-10-20T21:43:16.080849] [INFO ] [MAIN ] UVE: Fetched primary data is valid. [2025-10-20T21:43:16.080909] [DEBUG] [LOCAL ] UVE: Saving primary data. [2025-10-20T21:43:16.081578] [DEBUG] [LOCAL ] UVE: Trimmed 30312 records to 22011 (keeping 200 days of data) [2025-10-20T21:43:16.211502] [DEBUG] [LOCAL ] UVE: Primary data saved (22011 records). [2025-10-20T21:43:16.211636] [INFO ] [MAIN ] UVE: Full history saved. [2025-10-20T21:43:16.211749] [INFO ] [MAIN ] Update Progress: 2201/2367 symbols completed. UVE took 24.5s. Estimated remaining: 21m 22s. Rate: 7.8 symbols/min [2025-10-20T21:43:16.256625] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-09. [2025-10-20T21:43:16.382024] [INFO ] [MAIN ] UZD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:16.389727] [DEBUG] [LOCAL ] UZD: Loaded 6137 existing records from primary storage. [2025-10-20T21:43:16.389895] [INFO ] [MAIN ] UZD: Loaded 6137 existing records. [2025-10-20T21:43:16.566840] [WARN ] [MAIN ] UZD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:16.567038] [INFO ] [MAIN ] UZD: Wiping existing primary data before full refresh. [2025-10-20T21:43:16.567083] [INFO ] [LOCAL ] UZD: Starting deletion of primary data. [2025-10-20T21:43:16.567228] [DEBUG] [LOCAL ] UZD: Deleted primary data file. [2025-10-20T21:43:16.567265] [INFO ] [MAIN ] UZD: Starting full primary refresh. [2025-10-20T21:43:16.894855] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:43:16.957392] [INFO ] [REMOTE ] UZD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:17.051362] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-01. [2025-10-20T21:43:17.855561] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-08. [2025-10-20T21:43:17.856516] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-01 (0.81 seconds). [2025-10-20T21:43:18.062743] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T21:43:18.651427] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-02. [2025-10-20T21:43:19.121005] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-08 (1.27 seconds). [2025-10-20T21:43:19.251789] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-02 (0.60 seconds). [2025-10-20T21:43:19.451360] [INFO ] [REMOTE ] UWMC: Fetching primary data for 2025-10. [2025-10-20T21:43:20.251381] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-09. [2025-10-20T21:43:21.020264] [INFO ] [REMOTE ] UWMC: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T21:43:21.028382] [INFO ] [REMOTE ] UWMC: Full fetch complete (75918 records). [2025-10-20T21:43:21.028543] [DEBUG] [VALIDATOR ] UWMC: Validating primary data. [2025-10-20T21:43:21.061615] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-03. [2025-10-20T21:43:21.419990] [DEBUG] [VALIDATOR ] UWMC: All expected trading days are present. [2025-10-20T21:43:21.420073] [DEBUG] [VALIDATOR ] UWMC: Primary data validation passed. [2025-10-20T21:43:21.421150] [INFO ] [MAIN ] UWMC: Fetched primary data is valid. [2025-10-20T21:43:21.421212] [DEBUG] [LOCAL ] UWMC: Saving primary data. [2025-10-20T21:43:21.422551] [DEBUG] [LOCAL ] UWMC: Trimmed 75918 records to 52957 (keeping 200 days of data) [2025-10-20T21:43:21.736268] [DEBUG] [LOCAL ] UWMC: Primary data saved (52957 records). [2025-10-20T21:43:21.736447] [INFO ] [MAIN ] UWMC: Full history saved. [2025-10-20T21:43:21.736551] [INFO ] [MAIN ] Update Progress: 2202/2367 symbols completed. UWMC took 30.0s. Estimated remaining: 21m 14s. Rate: 7.8 symbols/min [2025-10-20T21:43:21.859412] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-09. [2025-10-20T21:43:21.972434] [INFO ] [MAIN ] UZE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:21.974098] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-03 (0.91 seconds). [2025-10-20T21:43:21.977931] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T21:43:21.994263] [DEBUG] [LOCAL ] UZE: Loaded 7329 existing records from primary storage. [2025-10-20T21:43:21.994544] [INFO ] [MAIN ] UZE: Loaded 7329 existing records. [2025-10-20T21:43:22.179326] [WARN ] [MAIN ] UZE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:22.179569] [INFO ] [MAIN ] UZE: Wiping existing primary data before full refresh. [2025-10-20T21:43:22.179615] [INFO ] [LOCAL ] UZE: Starting deletion of primary data. [2025-10-20T21:43:22.179819] [DEBUG] [LOCAL ] UZE: Deleted primary data file. [2025-10-20T21:43:22.179885] [INFO ] [MAIN ] UZE: Starting full primary refresh. [2025-10-20T21:43:22.536402] [INFO ] [REMOTE ] UZE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:22.651341] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-01. [2025-10-20T21:43:22.994091] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T21:43:23.137222] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-10-20T21:43:23.451348] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-02. [2025-10-20T21:43:23.997923] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-02 (0.55 seconds). [2025-10-20T21:43:24.251377] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-04. [2025-10-20T21:43:24.936384] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-04 (0.69 seconds). [2025-10-20T21:43:25.051381] [INFO ] [REMOTE ] UTZ: Fetching primary data for 2025-10. [2025-10-20T21:43:25.851372] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-03. [2025-10-20T21:43:26.331460] [INFO ] [REMOTE ] UTZ: Incremental fetch complete for 2025-10 (1.28 seconds). [2025-10-20T21:43:26.338179] [INFO ] [REMOTE ] UTZ: Full fetch complete (64631 records). [2025-10-20T21:43:26.338356] [DEBUG] [VALIDATOR ] UTZ: Validating primary data. [2025-10-20T21:43:26.623510] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-03 (0.77 seconds). [2025-10-20T21:43:26.656773] [INFO ] [REMOTE ] UVV: Fetching primary data for 2025-10. [2025-10-20T21:43:26.754341] [DEBUG] [VALIDATOR ] UTZ: All expected trading days are present. [2025-10-20T21:43:26.754425] [DEBUG] [VALIDATOR ] UTZ: Primary data validation passed. [2025-10-20T21:43:26.755961] [INFO ] [MAIN ] UTZ: Fetched primary data is valid. [2025-10-20T21:43:26.756026] [DEBUG] [LOCAL ] UTZ: Saving primary data. [2025-10-20T21:43:26.757267] [DEBUG] [LOCAL ] UTZ: Trimmed 64631 records to 44488 (keeping 200 days of data) [2025-10-20T21:43:27.021223] [DEBUG] [LOCAL ] UTZ: Primary data saved (44488 records). [2025-10-20T21:43:27.021362] [INFO ] [MAIN ] UTZ: Full history saved. [2025-10-20T21:43:27.021468] [INFO ] [MAIN ] Update Progress: 2203/2367 symbols completed. UTZ took 35.3s. Estimated remaining: 21m 6s. Rate: 7.8 symbols/min [2025-10-20T21:43:27.186352] [INFO ] [MAIN ] UZF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:27.186514] [DEBUG] [LOCAL ] UZF: No existing data file found in primary storage. [2025-10-20T21:43:27.186551] [WARN ] [MAIN ] UZF: No existing data found, performing full refresh instead. [2025-10-20T21:43:27.186690] [INFO ] [MAIN ] UZF: Wiping existing primary data before full refresh. [2025-10-20T21:43:27.186743] [INFO ] [LOCAL ] UZF: Starting deletion of primary data. [2025-10-20T21:43:27.186784] [DEBUG] [LOCAL ] UZF: No primary data file found to delete. [2025-10-20T21:43:27.186815] [INFO ] [MAIN ] UZF: Starting full primary refresh. [2025-10-20T21:43:27.456040] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-04. [2025-10-20T21:43:27.533809] [INFO ] [REMOTE ] UZF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:27.776776] [INFO ] [REMOTE ] UVV: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T21:43:27.780022] [INFO ] [REMOTE ] UVV: Full fetch complete (30191 records). [2025-10-20T21:43:27.780116] [DEBUG] [VALIDATOR ] UVV: Validating primary data. [2025-10-20T21:43:28.148276] [DEBUG] [VALIDATOR ] UVV: All expected trading days are present. [2025-10-20T21:43:28.148391] [DEBUG] [VALIDATOR ] UVV: Primary data validation passed. [2025-10-20T21:43:28.149283] [INFO ] [MAIN ] UVV: Fetched primary data is valid. [2025-10-20T21:43:28.149346] [DEBUG] [LOCAL ] UVV: Saving primary data. [2025-10-20T21:43:28.150076] [DEBUG] [LOCAL ] UVV: Trimmed 30191 records to 22837 (keeping 200 days of data) [2025-10-20T21:43:28.222688] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-04 (0.77 seconds). [2025-10-20T21:43:28.256901] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-05. [2025-10-20T21:43:28.309958] [DEBUG] [LOCAL ] UVV: Primary data saved (22837 records). [2025-10-20T21:43:28.310093] [INFO ] [MAIN ] UVV: Full history saved. [2025-10-20T21:43:28.310174] [INFO ] [MAIN ] Update Progress: 2204/2367 symbols completed. UVV took 36.6s. Estimated remaining: 20m 58s. Rate: 7.8 symbols/min [2025-10-20T21:43:28.482976] [INFO ] [MAIN ] V: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:28.534518] [DEBUG] [LOCAL ] V: Loaded 54532 existing records from primary storage. [2025-10-20T21:43:28.534887] [INFO ] [MAIN ] V: Loaded 54532 existing records. [2025-10-20T21:43:28.712203] [WARN ] [MAIN ] V: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:28.712385] [INFO ] [MAIN ] V: Wiping existing primary data before full refresh. [2025-10-20T21:43:28.712432] [INFO ] [LOCAL ] V: Starting deletion of primary data. [2025-10-20T21:43:28.712616] [DEBUG] [LOCAL ] V: Deleted primary data file. [2025-10-20T21:43:28.712694] [INFO ] [MAIN ] V: Starting full primary refresh. [2025-10-20T21:43:28.995095] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T21:43:29.055672] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-06. [2025-10-20T21:43:29.083054] [INFO ] [REMOTE ] V: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:29.689265] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T21:43:29.851385] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-05. [2025-10-20T21:43:30.485967] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T21:43:30.651438] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-01. [2025-10-20T21:43:31.246352] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-01 (0.59 seconds). [2025-10-20T21:43:31.451367] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-02. [2025-10-20T21:43:31.922439] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-02 (0.47 seconds). [2025-10-20T21:43:32.251381] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-03. [2025-10-20T21:43:32.956286] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T21:43:33.051399] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-06. [2025-10-20T21:43:33.657192] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T21:43:33.851366] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-07. [2025-10-20T21:43:34.324272] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T21:43:34.651383] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-04. [2025-10-20T21:43:35.315679] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-04 (0.66 seconds). [2025-10-20T21:43:35.451422] [INFO ] [REMOTE ] V: Fetching primary data for 2025-01. [2025-10-20T21:43:36.251437] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-08. [2025-10-20T21:43:37.021545] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T21:43:37.051372] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-09. [2025-10-20T21:43:37.576689] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-01 (2.13 seconds). [2025-10-20T21:43:37.693935] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T21:43:37.851364] [INFO ] [REMOTE ] UZD: Fetching primary data for 2025-10. [2025-10-20T21:43:38.339370] [INFO ] [REMOTE ] UZD: Incremental fetch complete for 2025-10 (0.49 seconds). [2025-10-20T21:43:38.340355] [INFO ] [REMOTE ] UZD: Full fetch complete (8967 records). [2025-10-20T21:43:38.340422] [DEBUG] [VALIDATOR ] UZD: Validating primary data. [2025-10-20T21:43:38.656496] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-05. [2025-10-20T21:43:38.696266] [DEBUG] [VALIDATOR ] UZD: All expected trading days are present. [2025-10-20T21:43:38.696349] [DEBUG] [VALIDATOR ] UZD: Primary data validation passed. [2025-10-20T21:43:38.696570] [INFO ] [MAIN ] UZD: Fetched primary data is valid. [2025-10-20T21:43:38.696610] [DEBUG] [LOCAL ] UZD: Saving primary data. [2025-10-20T21:43:38.697047] [DEBUG] [LOCAL ] UZD: Trimmed 8967 records to 5930 (keeping 200 days of data) [2025-10-20T21:43:38.732929] [DEBUG] [LOCAL ] UZD: Primary data saved (5930 records). [2025-10-20T21:43:38.733066] [INFO ] [MAIN ] UZD: Full history saved. [2025-10-20T21:43:38.733148] [INFO ] [MAIN ] Update Progress: 2205/2367 symbols completed. UZD took 22.5s. Estimated remaining: 20m 50s. Rate: 7.8 symbols/min [2025-10-20T21:43:38.905328] [INFO ] [MAIN ] VAC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:38.936483] [DEBUG] [LOCAL ] VAC: Loaded 34464 existing records from primary storage. [2025-10-20T21:43:38.936794] [INFO ] [MAIN ] VAC: Loaded 34464 existing records. [2025-10-20T21:43:39.114144] [WARN ] [MAIN ] VAC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:39.114377] [INFO ] [MAIN ] VAC: Wiping existing primary data before full refresh. [2025-10-20T21:43:39.114433] [INFO ] [LOCAL ] VAC: Starting deletion of primary data. [2025-10-20T21:43:39.114621] [DEBUG] [LOCAL ] VAC: Deleted primary data file. [2025-10-20T21:43:39.114702] [INFO ] [MAIN ] VAC: Starting full primary refresh. [2025-10-20T21:43:39.395219] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T21:43:39.452848] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-07. [2025-10-20T21:43:39.476221] [INFO ] [REMOTE ] VAC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:40.104809] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T21:43:40.251369] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-06. [2025-10-20T21:43:40.862761] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T21:43:41.051432] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-07. [2025-10-20T21:43:41.521338] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T21:43:41.851419] [INFO ] [REMOTE ] V: Fetching primary data for 2025-02. [2025-10-20T21:43:42.651426] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-08. [2025-10-20T21:43:43.451420] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-08. [2025-10-20T21:43:43.580315] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T21:43:44.091165] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-02 (2.24 seconds). [2025-10-20T21:43:44.247319] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-08 (0.80 seconds). [2025-10-20T21:43:44.251328] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-09. [2025-10-20T21:43:44.848202] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T21:43:45.051419] [INFO ] [REMOTE ] UZF: Fetching primary data for 2025-10. [2025-10-20T21:43:45.524983] [INFO ] [REMOTE ] UZF: Incremental fetch complete for 2025-10 (0.47 seconds). [2025-10-20T21:43:45.526007] [INFO ] [REMOTE ] UZF: Full fetch complete (8449 records). [2025-10-20T21:43:45.526074] [DEBUG] [VALIDATOR ] UZF: Validating primary data. [2025-10-20T21:43:45.704809] [WARN ] [VALIDATOR ] UZF: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T21:43:45.705087] [FATAL] [MAIN ] UZF: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:43:45.705157] [WARN ] [MAIN ] UZF: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:43:45.705205] [INFO ] [LOCAL ] UZF: Starting deletion of primary data. [2025-10-20T21:43:45.705262] [DEBUG] [LOCAL ] UZF: No primary data file found to delete. [2025-10-20T21:43:45.705297] [INFO ] [LOCAL ] UZF: Starting deletion of derivative data. [2025-10-20T21:43:45.705339] [DEBUG] [LOCAL ] UZF: No derivative data file found to delete. [2025-10-20T21:43:45.705394] [INFO ] [MAIN ] Update Progress: 2206/2367 symbols completed. UZF took 18.7s. Estimated remaining: 20m 42s. Rate: 7.8 symbols/min [2025-10-20T21:43:45.851888] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-01. [2025-10-20T21:43:45.874812] [INFO ] [MAIN ] VAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:45.922732] [DEBUG] [LOCAL ] VAL: Loaded 50829 existing records from primary storage. [2025-10-20T21:43:45.923084] [INFO ] [MAIN ] VAL: Loaded 50829 existing records. [2025-10-20T21:43:46.096144] [WARN ] [MAIN ] VAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:46.096364] [INFO ] [MAIN ] VAL: Wiping existing primary data before full refresh. [2025-10-20T21:43:46.096418] [INFO ] [LOCAL ] VAL: Starting deletion of primary data. [2025-10-20T21:43:46.096590] [DEBUG] [LOCAL ] VAL: Deleted primary data file. [2025-10-20T21:43:46.096698] [INFO ] [MAIN ] VAL: Starting full primary refresh. [2025-10-20T21:43:46.443102] [INFO ] [REMOTE ] VAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:46.651355] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-01. [2025-10-20T21:43:47.191845] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T21:43:47.451379] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-02. [2025-10-20T21:43:48.150018] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T21:43:48.251403] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-09. [2025-10-20T21:43:48.867261] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T21:43:48.919127] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:43:49.051442] [INFO ] [REMOTE ] V: Fetching primary data for 2025-03. [2025-10-20T21:43:49.851407] [INFO ] [REMOTE ] UZE: Fetching primary data for 2025-10. [2025-10-20T21:43:50.479932] [INFO ] [REMOTE ] UZE: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T21:43:50.481261] [INFO ] [REMOTE ] UZE: Full fetch complete (9823 records). [2025-10-20T21:43:50.481332] [DEBUG] [VALIDATOR ] UZE: Validating primary data. [2025-10-20T21:43:50.664898] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-02. [2025-10-20T21:43:50.850278] [DEBUG] [VALIDATOR ] UZE: All expected trading days are present. [2025-10-20T21:43:50.850413] [DEBUG] [VALIDATOR ] UZE: Primary data validation passed. [2025-10-20T21:43:50.850679] [INFO ] [MAIN ] UZE: Fetched primary data is valid. [2025-10-20T21:43:50.850737] [DEBUG] [LOCAL ] UZE: Saving primary data. [2025-10-20T21:43:50.851153] [DEBUG] [LOCAL ] UZE: Trimmed 9823 records to 7056 (keeping 200 days of data) [2025-10-20T21:43:50.891496] [DEBUG] [LOCAL ] UZE: Primary data saved (7056 records). [2025-10-20T21:43:50.891572] [INFO ] [MAIN ] UZE: Full history saved. [2025-10-20T21:43:50.891661] [INFO ] [MAIN ] Update Progress: 2207/2367 symbols completed. UZE took 29.2s. Estimated remaining: 20m 34s. Rate: 7.8 symbols/min [2025-10-20T21:43:51.061905] [INFO ] [MAIN ] VALE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:43:51.111328] [DEBUG] [LOCAL ] VALE: Loaded 54473 existing records from primary storage. [2025-10-20T21:43:51.111613] [INFO ] [MAIN ] VALE: Loaded 54473 existing records. [2025-10-20T21:43:51.289594] [WARN ] [MAIN ] VALE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:43:51.289877] [INFO ] [MAIN ] VALE: Wiping existing primary data before full refresh. [2025-10-20T21:43:51.289979] [INFO ] [LOCAL ] VALE: Starting deletion of primary data. [2025-10-20T21:43:51.290174] [DEBUG] [LOCAL ] VALE: Deleted primary data file. [2025-10-20T21:43:51.290211] [INFO ] [MAIN ] VALE: Starting full primary refresh. [2025-10-20T21:43:51.472690] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-03. [2025-10-20T21:43:51.625303] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-03 (2.57 seconds). [2025-10-20T21:43:51.709323] [INFO ] [REMOTE ] VALE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:43:52.251455] [INFO ] [REMOTE ] V: Fetching primary data for 2025-04. [2025-10-20T21:43:52.306419] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:43:52.548295] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T21:43:53.051470] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-01. [2025-10-20T21:43:53.851394] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-04. [2025-10-20T21:43:54.467344] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-04 (2.22 seconds). [2025-10-20T21:43:54.651377] [INFO ] [REMOTE ] V: Fetching primary data for 2025-05. [2025-10-20T21:43:54.743261] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T21:43:55.264912] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-04 (1.41 seconds). [2025-10-20T21:43:55.451491] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-02. [2025-10-20T21:43:56.251487] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-05. [2025-10-20T21:43:56.928054] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-10-20T21:43:57.051340] [INFO ] [REMOTE ] V: Fetching primary data for 2025-06. [2025-10-20T21:43:57.411499] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-05 (1.16 seconds). [2025-10-20T21:43:57.446090] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-02 (1.99 seconds). [2025-10-20T21:43:57.851383] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-03. [2025-10-20T21:43:58.651371] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-03. [2025-10-20T21:43:59.328454] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-06 (2.28 seconds). [2025-10-20T21:43:59.451347] [INFO ] [REMOTE ] V: Fetching primary data for 2025-07. [2025-10-20T21:43:59.782626] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T21:44:00.251357] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-04. [2025-10-20T21:44:00.356495] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T21:44:01.051369] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-04. [2025-10-20T21:44:01.818113] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-07 (2.37 seconds). [2025-10-20T21:44:01.856452] [INFO ] [REMOTE ] V: Fetching primary data for 2025-08. [2025-10-20T21:44:01.876435] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:44:02.444775] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T21:44:02.651414] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-05. [2025-10-20T21:44:03.451409] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-05. [2025-10-20T21:44:04.109818] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-08 (2.25 seconds). [2025-10-20T21:44:04.251426] [INFO ] [REMOTE ] V: Fetching primary data for 2025-09. [2025-10-20T21:44:04.695742] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-05 (2.04 seconds). [2025-10-20T21:44:05.051374] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-06. [2025-10-20T21:44:05.241474] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-05 (1.79 seconds). [2025-10-20T21:44:05.851377] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-06. [2025-10-20T21:44:06.471678] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-09 (2.22 seconds). [2025-10-20T21:44:06.651359] [INFO ] [REMOTE ] V: Fetching primary data for 2025-10. [2025-10-20T21:44:07.072575] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-06 (2.02 seconds). [2025-10-20T21:44:07.175391] [INFO ] [REMOTE ] V: Incremental fetch complete for 2025-10 (0.52 seconds). [2025-10-20T21:44:07.183436] [INFO ] [REMOTE ] V: Full fetch complete (77928 records). [2025-10-20T21:44:07.183586] [DEBUG] [VALIDATOR ] V: Validating primary data. [2025-10-20T21:44:07.462232] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-06 (1.61 seconds). [2025-10-20T21:44:07.460496] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-06. [2025-10-20T21:44:07.614578] [DEBUG] [VALIDATOR ] V: All expected trading days are present. [2025-10-20T21:44:07.614702] [DEBUG] [VALIDATOR ] V: Primary data validation passed. [2025-10-20T21:44:07.616799] [INFO ] [MAIN ] V: Fetched primary data is valid. [2025-10-20T21:44:07.616867] [DEBUG] [LOCAL ] V: Saving primary data. [2025-10-20T21:44:07.618411] [DEBUG] [LOCAL ] V: Trimmed 77928 records to 53752 (keeping 200 days of data) [2025-10-20T21:44:07.947117] [DEBUG] [LOCAL ] V: Primary data saved (53752 records). [2025-10-20T21:44:07.947249] [INFO ] [MAIN ] V: Full history saved. [2025-10-20T21:44:07.947326] [INFO ] [MAIN ] Update Progress: 2208/2367 symbols completed. V took 39.6s. Estimated remaining: 20m 27s. Rate: 7.8 symbols/min [2025-10-20T21:44:08.115551] [INFO ] [MAIN ] VATE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:08.122950] [DEBUG] [LOCAL ] VATE: Loaded 5266 existing records from primary storage. [2025-10-20T21:44:08.123151] [INFO ] [MAIN ] VATE: Loaded 5266 existing records. [2025-10-20T21:44:08.257683] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-07. [2025-10-20T21:44:08.296877] [WARN ] [MAIN ] VATE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:08.297153] [INFO ] [MAIN ] VATE: Wiping existing primary data before full refresh. [2025-10-20T21:44:08.297207] [INFO ] [LOCAL ] VATE: Starting deletion of primary data. [2025-10-20T21:44:08.297358] [DEBUG] [LOCAL ] VATE: Deleted primary data file. [2025-10-20T21:44:08.297413] [INFO ] [MAIN ] VATE: Starting full primary refresh. [2025-10-20T21:44:08.643675] [INFO ] [REMOTE ] VATE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:08.888246] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:44:09.051351] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-07. [2025-10-20T21:44:09.851426] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-01. [2025-10-20T21:44:10.491680] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:44:10.531115] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-07 (2.27 seconds). [2025-10-20T21:44:10.651432] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-08. [2025-10-20T21:44:10.938337] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T21:44:11.451376] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-02. [2025-10-20T21:44:11.899505] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-08 (1.25 seconds). [2025-10-20T21:44:12.251407] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-08. [2025-10-20T21:44:12.365518] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-02 (0.91 seconds). [2025-10-20T21:44:13.051432] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-09. [2025-10-20T21:44:13.851416] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-03. [2025-10-20T21:44:14.345539] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T21:44:14.399297] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-09 (1.35 seconds). [2025-10-20T21:44:14.627595] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-03 (0.78 seconds). [2025-10-20T21:44:14.651345] [INFO ] [REMOTE ] VAC: Fetching primary data for 2025-10. [2025-10-20T21:44:15.451387] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-04. [2025-10-20T21:44:15.786771] [INFO ] [REMOTE ] VAC: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T21:44:15.791367] [INFO ] [REMOTE ] VAC: Full fetch complete (48728 records). [2025-10-20T21:44:15.791480] [DEBUG] [VALIDATOR ] VAC: Validating primary data. [2025-10-20T21:44:16.179153] [DEBUG] [VALIDATOR ] VAC: All expected trading days are present. [2025-10-20T21:44:16.184956] [DEBUG] [VALIDATOR ] VAC: Primary data validation passed. [2025-10-20T21:44:16.186823] [INFO ] [MAIN ] VAC: Fetched primary data is valid. [2025-10-20T21:44:16.186925] [DEBUG] [LOCAL ] VAC: Saving primary data. [2025-10-20T21:44:16.187185] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T21:44:16.194487] [DEBUG] [LOCAL ] VAC: Trimmed 48728 records to 33877 (keeping 200 days of data) [2025-10-20T21:44:16.257865] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-05. [2025-10-20T21:44:16.413091] [DEBUG] [LOCAL ] VAC: Primary data saved (33877 records). [2025-10-20T21:44:16.413235] [INFO ] [MAIN ] VAC: Full history saved. [2025-10-20T21:44:16.413316] [INFO ] [MAIN ] Update Progress: 2209/2367 symbols completed. VAC took 37.7s. Estimated remaining: 20m 20s. Rate: 7.8 symbols/min [2025-10-20T21:44:16.578342] [INFO ] [MAIN ] VBF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:16.587486] [DEBUG] [LOCAL ] VBF: Loaded 7375 existing records from primary storage. [2025-10-20T21:44:16.587669] [INFO ] [MAIN ] VBF: Loaded 7375 existing records. [2025-10-20T21:44:16.759437] [WARN ] [MAIN ] VBF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:16.759680] [INFO ] [MAIN ] VBF: Wiping existing primary data before full refresh. [2025-10-20T21:44:16.759739] [INFO ] [LOCAL ] VBF: Starting deletion of primary data. [2025-10-20T21:44:16.759883] [DEBUG] [LOCAL ] VBF: Deleted primary data file. [2025-10-20T21:44:16.759923] [INFO ] [MAIN ] VBF: Starting full primary refresh. [2025-10-20T21:44:17.049711] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-05 (0.79 seconds). [2025-10-20T21:44:17.056624] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-07. [2025-10-20T21:44:17.140482] [INFO ] [REMOTE ] VBF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:17.851350] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-06. [2025-10-20T21:44:18.455147] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-06 (0.60 seconds). [2025-10-20T21:44:18.651348] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-07. [2025-10-20T21:44:18.780467] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:44:19.136730] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-07 (0.49 seconds). [2025-10-20T21:44:19.451366] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-08. [2025-10-20T21:44:20.251370] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-08. [2025-10-20T21:44:20.847079] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-08 (0.60 seconds). [2025-10-20T21:44:21.051343] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-09. [2025-10-20T21:44:21.214075] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-08 (1.76 seconds). [2025-10-20T21:44:21.674085] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-09 (0.62 seconds). [2025-10-20T21:44:21.851383] [INFO ] [REMOTE ] VATE: Fetching primary data for 2025-10. [2025-10-20T21:44:22.384273] [INFO ] [REMOTE ] VATE: Incremental fetch complete for 2025-10 (0.53 seconds). [2025-10-20T21:44:22.385534] [INFO ] [REMOTE ] VATE: Full fetch complete (10528 records). [2025-10-20T21:44:22.385600] [DEBUG] [VALIDATOR ] VATE: Validating primary data. [2025-10-20T21:44:22.656497] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-09. [2025-10-20T21:44:22.739759] [DEBUG] [VALIDATOR ] VATE: All expected trading days are present. [2025-10-20T21:44:22.739875] [DEBUG] [VALIDATOR ] VATE: Primary data validation passed. [2025-10-20T21:44:22.740153] [INFO ] [MAIN ] VATE: Fetched primary data is valid. [2025-10-20T21:44:22.740192] [DEBUG] [LOCAL ] VATE: Saving primary data. [2025-10-20T21:44:22.740604] [DEBUG] [LOCAL ] VATE: Trimmed 10528 records to 5107 (keeping 200 days of data) [2025-10-20T21:44:22.767850] [DEBUG] [LOCAL ] VATE: Primary data saved (5107 records). [2025-10-20T21:44:22.767966] [INFO ] [MAIN ] VATE: Full history saved. [2025-10-20T21:44:22.768052] [INFO ] [MAIN ] Update Progress: 2210/2367 symbols completed. VATE took 14.8s. Estimated remaining: 20m 12s. Rate: 7.8 symbols/min [2025-10-20T21:44:22.935331] [INFO ] [MAIN ] VCV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:22.953287] [DEBUG] [LOCAL ] VCV: Loaded 13798 existing records from primary storage. [2025-10-20T21:44:22.953521] [INFO ] [MAIN ] VCV: Loaded 13798 existing records. [2025-10-20T21:44:23.133007] [WARN ] [MAIN ] VCV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:23.133194] [INFO ] [MAIN ] VCV: Wiping existing primary data before full refresh. [2025-10-20T21:44:23.133238] [INFO ] [LOCAL ] VCV: Starting deletion of primary data. [2025-10-20T21:44:23.133413] [DEBUG] [LOCAL ] VCV: Deleted primary data file. [2025-10-20T21:44:23.133468] [INFO ] [MAIN ] VCV: Starting full primary refresh. [2025-10-20T21:44:23.456804] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-09. [2025-10-20T21:44:23.480417] [INFO ] [REMOTE ] VCV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:24.251360] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-01. [2025-10-20T21:44:24.330285] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:44:24.637816] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T21:44:25.051346] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-02. [2025-10-20T21:44:25.528839] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-09 (2.07 seconds). [2025-10-20T21:44:25.814446] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T21:44:25.851335] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-03. [2025-10-20T21:44:26.651449] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-01. [2025-10-20T21:44:26.671210] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-03 (0.82 seconds). [2025-10-20T21:44:27.451428] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-04. [2025-10-20T21:44:27.487110] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T21:44:28.077954] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-04 (0.63 seconds). [2025-10-20T21:44:28.251391] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-05. [2025-10-20T21:44:28.886127] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T21:44:29.051423] [INFO ] [REMOTE ] VALE: Fetching primary data for 2025-10. [2025-10-20T21:44:29.851368] [INFO ] [REMOTE ] VAL: Fetching primary data for 2025-10. [2025-10-20T21:44:30.651407] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-06. [2025-10-20T21:44:30.924007] [INFO ] [REMOTE ] VALE: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T21:44:30.934765] [INFO ] [REMOTE ] VALE: Full fetch complete (77873 records). [2025-10-20T21:44:30.935022] [DEBUG] [VALIDATOR ] VALE: Validating primary data. [2025-10-20T21:44:31.372715] [DEBUG] [VALIDATOR ] VALE: All expected trading days are present. [2025-10-20T21:44:31.372862] [DEBUG] [VALIDATOR ] VALE: Primary data validation passed. [2025-10-20T21:44:31.374612] [INFO ] [MAIN ] VALE: Fetched primary data is valid. [2025-10-20T21:44:31.375788] [DEBUG] [LOCAL ] VALE: Saving primary data. [2025-10-20T21:44:31.402866] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-06 (0.75 seconds). [2025-10-20T21:44:31.404191] [INFO ] [REMOTE ] VAL: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T21:44:31.415378] [DEBUG] [LOCAL ] VALE: Trimmed 77873 records to 53693 (keeping 200 days of data) [2025-10-20T21:44:31.415524] [INFO ] [REMOTE ] VAL: Full fetch complete (73027 records). [2025-10-20T21:44:31.415854] [DEBUG] [VALIDATOR ] VAL: Validating primary data. [2025-10-20T21:44:31.508531] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-07. [2025-10-20T21:44:32.182897] [DEBUG] [VALIDATOR ] VAL: All expected trading days are present. [2025-10-20T21:44:32.188422] [DEBUG] [LOCAL ] VALE: Primary data saved (53693 records). [2025-10-20T21:44:32.183904] [DEBUG] [VALIDATOR ] VAL: Primary data validation passed. [2025-10-20T21:44:32.190546] [INFO ] [MAIN ] VAL: Fetched primary data is valid. [2025-10-20T21:44:32.190734] [INFO ] [MAIN ] VALE: Full history saved. [2025-10-20T21:44:32.191155] [INFO ] [MAIN ] Update Progress: 2211/2367 symbols completed. VALE took 41.3s. Estimated remaining: 20m 4s. Rate: 7.8 symbols/min [2025-10-20T21:44:32.191249] [DEBUG] [LOCAL ] VAL: Saving primary data. [2025-10-20T21:44:32.259628] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-02. [2025-10-20T21:44:32.296456] [DEBUG] [LOCAL ] VAL: Trimmed 73027 records to 50021 (keeping 200 days of data) [2025-10-20T21:44:32.471802] [INFO ] [MAIN ] VEEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:32.477257] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T21:44:32.680620] [DEBUG] [LOCAL ] VEEV: Loaded 48056 existing records from primary storage. [2025-10-20T21:44:32.691627] [INFO ] [MAIN ] VEEV: Loaded 48056 existing records. [2025-10-20T21:44:32.879268] [DEBUG] [LOCAL ] VAL: Primary data saved (50021 records). [2025-10-20T21:44:32.941217] [INFO ] [MAIN ] VAL: Full history saved. [2025-10-20T21:44:32.944701] [INFO ] [MAIN ] Update Progress: 2212/2367 symbols completed. VAL took 47.2s. Estimated remaining: 19m 56s. Rate: 7.8 symbols/min [2025-10-20T21:44:32.944466] [WARN ] [MAIN ] VEEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:32.992791] [INFO ] [MAIN ] VEEV: Wiping existing primary data before full refresh. [2025-10-20T21:44:32.998018] [INFO ] [LOCAL ] VEEV: Starting deletion of primary data. [2025-10-20T21:44:33.037735] [DEBUG] [LOCAL ] VEEV: Deleted primary data file. [2025-10-20T21:44:33.051094] [INFO ] [MAIN ] VEEV: Starting full primary refresh. [2025-10-20T21:44:33.059293] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-08. [2025-10-20T21:44:33.223420] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T21:44:33.251456] [INFO ] [MAIN ] VEL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:33.411039] [DEBUG] [LOCAL ] VEL: Loaded 12350 existing records from primary storage. [2025-10-20T21:44:33.422467] [INFO ] [MAIN ] VEL: Loaded 12350 existing records. [2025-10-20T21:44:33.653212] [INFO ] [REMOTE ] VEEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:33.719703] [WARN ] [MAIN ] VEL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:33.719980] [INFO ] [MAIN ] VEL: Wiping existing primary data before full refresh. [2025-10-20T21:44:33.720030] [INFO ] [LOCAL ] VEL: Starting deletion of primary data. [2025-10-20T21:44:33.720242] [DEBUG] [LOCAL ] VEL: Deleted primary data file. [2025-10-20T21:44:33.720341] [INFO ] [MAIN ] VEL: Starting full primary refresh. [2025-10-20T21:44:33.866955] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-01. [2025-10-20T21:44:34.027061] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T21:44:34.094968] [INFO ] [REMOTE ] VEL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:34.651374] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-03. [2025-10-20T21:44:35.451427] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-09. [2025-10-20T21:44:35.514688] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-03 (0.86 seconds). [2025-10-20T21:44:35.655976] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T21:44:36.188682] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T21:44:36.251341] [INFO ] [REMOTE ] VBF: Fetching primary data for 2025-10. [2025-10-20T21:44:37.044970] [INFO ] [REMOTE ] VBF: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:44:37.047144] [INFO ] [REMOTE ] VBF: Full fetch complete (10932 records). [2025-10-20T21:44:37.047213] [DEBUG] [VALIDATOR ] VBF: Validating primary data. [2025-10-20T21:44:37.058529] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-04. [2025-10-20T21:44:37.409717] [DEBUG] [VALIDATOR ] VBF: All expected trading days are present. [2025-10-20T21:44:37.409802] [DEBUG] [VALIDATOR ] VBF: Primary data validation passed. [2025-10-20T21:44:37.410050] [INFO ] [MAIN ] VBF: Fetched primary data is valid. [2025-10-20T21:44:37.410087] [DEBUG] [LOCAL ] VBF: Saving primary data. [2025-10-20T21:44:37.410499] [DEBUG] [LOCAL ] VBF: Trimmed 10932 records to 7267 (keeping 200 days of data) [2025-10-20T21:44:37.455532] [DEBUG] [LOCAL ] VBF: Primary data saved (7267 records). [2025-10-20T21:44:37.455708] [INFO ] [MAIN ] VBF: Full history saved. [2025-10-20T21:44:37.455802] [INFO ] [MAIN ] Update Progress: 2213/2367 symbols completed. VBF took 21.0s. Estimated remaining: 19m 48s. Rate: 7.8 symbols/min [2025-10-20T21:44:37.638497] [INFO ] [MAIN ] VET: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:37.683634] [DEBUG] [LOCAL ] VET: Loaded 44787 existing records from primary storage. [2025-10-20T21:44:37.684030] [INFO ] [MAIN ] VET: Loaded 44787 existing records. [2025-10-20T21:44:37.852983] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-01. [2025-10-20T21:44:37.874078] [WARN ] [MAIN ] VET: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:37.875194] [INFO ] [MAIN ] VET: Wiping existing primary data before full refresh. [2025-10-20T21:44:37.875277] [INFO ] [LOCAL ] VET: Starting deletion of primary data. [2025-10-20T21:44:37.875528] [DEBUG] [LOCAL ] VET: Deleted primary data file. [2025-10-20T21:44:37.875586] [INFO ] [MAIN ] VET: Starting full primary refresh. [2025-10-20T21:44:38.169858] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-04 (1.11 seconds). [2025-10-20T21:44:38.281446] [INFO ] [REMOTE ] VET: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:38.651433] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-01. [2025-10-20T21:44:38.670679] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T21:44:39.451376] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-05. [2025-10-20T21:44:40.178688] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:44:40.251364] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-02. [2025-10-20T21:44:40.303955] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-05 (0.85 seconds). [2025-10-20T21:44:40.968426] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T21:44:41.051360] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-06. [2025-10-20T21:44:41.851349] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-02. [2025-10-20T21:44:42.080708] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-06 (1.03 seconds). [2025-10-20T21:44:42.651435] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-02. [2025-10-20T21:44:43.372313] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T21:44:43.451424] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-07. [2025-10-20T21:44:43.819303] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T21:44:44.062740] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T21:44:44.251395] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-03. [2025-10-20T21:44:45.051358] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-03. [2025-10-20T21:44:45.851432] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-08. [2025-10-20T21:44:45.982373] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-03 (1.73 seconds). [2025-10-20T21:44:46.038246] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-03 (0.99 seconds). [2025-10-20T21:44:46.446264] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T21:44:46.651413] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-03. [2025-10-20T21:44:47.451416] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-04. [2025-10-20T21:44:48.060510] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-03 (1.41 seconds). [2025-10-20T21:44:48.251483] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-09. [2025-10-20T21:44:49.051399] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-04. [2025-10-20T21:44:49.115189] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-04 (1.66 seconds). [2025-10-20T21:44:49.320689] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-09 (1.07 seconds). [2025-10-20T21:44:49.851401] [INFO ] [REMOTE ] VCV: Fetching primary data for 2025-10. [2025-10-20T21:44:49.924363] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-04 (0.87 seconds). [2025-10-20T21:44:50.562098] [INFO ] [REMOTE ] VCV: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T21:44:50.564024] [INFO ] [REMOTE ] VCV: Full fetch complete (18438 records). [2025-10-20T21:44:50.564127] [DEBUG] [VALIDATOR ] VCV: Validating primary data. [2025-10-20T21:44:50.656699] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-04. [2025-10-20T21:44:50.950695] [DEBUG] [VALIDATOR ] VCV: All expected trading days are present. [2025-10-20T21:44:50.950787] [DEBUG] [VALIDATOR ] VCV: Primary data validation passed. [2025-10-20T21:44:50.951174] [INFO ] [MAIN ] VCV: Fetched primary data is valid. [2025-10-20T21:44:50.951214] [DEBUG] [LOCAL ] VCV: Saving primary data. [2025-10-20T21:44:50.951859] [DEBUG] [LOCAL ] VCV: Trimmed 18438 records to 13517 (keeping 200 days of data) [2025-10-20T21:44:51.034240] [DEBUG] [LOCAL ] VCV: Primary data saved (13517 records). [2025-10-20T21:44:51.034387] [INFO ] [MAIN ] VCV: Full history saved. [2025-10-20T21:44:51.034466] [INFO ] [MAIN ] Update Progress: 2214/2367 symbols completed. VCV took 28.3s. Estimated remaining: 19m 41s. Rate: 7.8 symbols/min [2025-10-20T21:44:51.215115] [INFO ] [MAIN ] VFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:44:51.278360] [DEBUG] [LOCAL ] VFC: Loaded 54384 existing records from primary storage. [2025-10-20T21:44:51.278835] [INFO ] [MAIN ] VFC: Loaded 54384 existing records. [2025-10-20T21:44:51.457476] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-05. [2025-10-20T21:44:51.478755] [WARN ] [MAIN ] VFC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:44:51.480577] [INFO ] [MAIN ] VFC: Wiping existing primary data before full refresh. [2025-10-20T21:44:51.480668] [INFO ] [LOCAL ] VFC: Starting deletion of primary data. [2025-10-20T21:44:51.481030] [DEBUG] [LOCAL ] VFC: Deleted primary data file. [2025-10-20T21:44:51.481100] [INFO ] [MAIN ] VFC: Starting full primary refresh. [2025-10-20T21:44:51.867031] [INFO ] [REMOTE ] VFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:44:52.251373] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-01. [2025-10-20T21:44:52.429743] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-05 (0.97 seconds). [2025-10-20T21:44:52.513466] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-04 (1.86 seconds). [2025-10-20T21:44:53.051413] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-06. [2025-10-20T21:44:53.850095] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T21:44:53.853252] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-05. [2025-10-20T21:44:54.095747] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T21:44:54.651387] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-05. [2025-10-20T21:44:55.178303] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-05 (1.33 seconds). [2025-10-20T21:44:55.451440] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-07. [2025-10-20T21:44:56.221699] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T21:44:56.251372] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-06. [2025-10-20T21:44:56.386859] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T21:44:57.051424] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-08. [2025-10-20T21:44:57.851358] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-06. [2025-10-20T21:44:57.938618] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-08 (0.89 seconds). [2025-10-20T21:44:57.966674] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T21:44:58.651418] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-02. [2025-10-20T21:44:59.451350] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-07. [2025-10-20T21:44:59.697429] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T21:45:00.251348] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-07. [2025-10-20T21:45:00.313440] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T21:45:01.051413] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-03. [2025-10-20T21:45:01.415929] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-07 (1.96 seconds). [2025-10-20T21:45:01.851412] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-09. [2025-10-20T21:45:02.217038] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-07 (1.97 seconds). [2025-10-20T21:45:02.529365] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T21:45:02.558153] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-09 (0.71 seconds). [2025-10-20T21:45:02.651426] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-04. [2025-10-20T21:45:03.451390] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-08. [2025-10-20T21:45:04.251407] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-08. [2025-10-20T21:45:04.850929] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-04 (2.20 seconds). [2025-10-20T21:45:05.051374] [INFO ] [REMOTE ] VEL: Fetching primary data for 2025-10. [2025-10-20T21:45:05.316724] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T21:45:05.766240] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-08 (1.51 seconds). [2025-10-20T21:45:05.843226] [INFO ] [REMOTE ] VEL: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:45:05.846431] [INFO ] [REMOTE ] VEL: Full fetch complete (17176 records). [2025-10-20T21:45:05.846626] [DEBUG] [VALIDATOR ] VEL: Validating primary data. [2025-10-20T21:45:05.858709] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-09. [2025-10-20T21:45:06.229214] [DEBUG] [VALIDATOR ] VEL: All expected trading days are present. [2025-10-20T21:45:06.229321] [DEBUG] [VALIDATOR ] VEL: Primary data validation passed. [2025-10-20T21:45:06.230181] [INFO ] [MAIN ] VEL: Fetched primary data is valid. [2025-10-20T21:45:06.230264] [DEBUG] [LOCAL ] VEL: Saving primary data. [2025-10-20T21:45:06.231012] [DEBUG] [LOCAL ] VEL: Trimmed 17176 records to 12088 (keeping 200 days of data) [2025-10-20T21:45:06.313057] [DEBUG] [LOCAL ] VEL: Primary data saved (12088 records). [2025-10-20T21:45:06.313259] [INFO ] [MAIN ] VEL: Full history saved. [2025-10-20T21:45:06.313363] [INFO ] [MAIN ] Update Progress: 2215/2367 symbols completed. VEL took 33.4s. Estimated remaining: 19m 34s. Rate: 7.8 symbols/min [2025-10-20T21:45:06.495922] [INFO ] [MAIN ] VG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:06.496157] [DEBUG] [LOCAL ] VG: No existing data file found in primary storage. [2025-10-20T21:45:06.496196] [WARN ] [MAIN ] VG: No existing data found, performing full refresh instead. [2025-10-20T21:45:06.496337] [INFO ] [MAIN ] VG: Wiping existing primary data before full refresh. [2025-10-20T21:45:06.496379] [INFO ] [LOCAL ] VG: Starting deletion of primary data. [2025-10-20T21:45:06.496420] [DEBUG] [LOCAL ] VG: No primary data file found to delete. [2025-10-20T21:45:06.496453] [INFO ] [MAIN ] VG: Starting full primary refresh. [2025-10-20T21:45:06.655730] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-05. [2025-10-20T21:45:06.871889] [INFO ] [REMOTE ] VG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:07.451411] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-01. [2025-10-20T21:45:07.608786] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T21:45:08.251423] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-09. [2025-10-20T21:45:08.419719] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T21:45:08.804459] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-05 (2.15 seconds). [2025-10-20T21:45:09.051378] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-06. [2025-10-20T21:45:09.585685] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-09 (1.33 seconds). [2025-10-20T21:45:09.851370] [INFO ] [REMOTE ] VEEV: Fetching primary data for 2025-10. [2025-10-20T21:45:10.651357] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-02. [2025-10-20T21:45:10.994286] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-06 (1.94 seconds). [2025-10-20T21:45:11.463151] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-07. [2025-10-20T21:45:11.465579] [INFO ] [REMOTE ] VEEV: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T21:45:11.473099] [INFO ] [REMOTE ] VEEV: Full fetch complete (68429 records). [2025-10-20T21:45:11.473209] [DEBUG] [VALIDATOR ] VEEV: Validating primary data. [2025-10-20T21:45:11.861305] [DEBUG] [VALIDATOR ] VEEV: All expected trading days are present. [2025-10-20T21:45:11.861393] [DEBUG] [VALIDATOR ] VEEV: Primary data validation passed. [2025-10-20T21:45:11.862515] [INFO ] [MAIN ] VEEV: Fetched primary data is valid. [2025-10-20T21:45:11.862579] [DEBUG] [LOCAL ] VEEV: Saving primary data. [2025-10-20T21:45:11.863936] [DEBUG] [LOCAL ] VEEV: Trimmed 68429 records to 47245 (keeping 200 days of data) [2025-10-20T21:45:12.147141] [DEBUG] [LOCAL ] VEEV: Primary data saved (47245 records). [2025-10-20T21:45:12.147384] [INFO ] [MAIN ] VEEV: Full history saved. [2025-10-20T21:45:12.147525] [INFO ] [MAIN ] Update Progress: 2216/2367 symbols completed. VEEV took 40.0s. Estimated remaining: 19m 26s. Rate: 7.8 symbols/min [2025-10-20T21:45:12.253413] [INFO ] [REMOTE ] VET: Fetching primary data for 2025-10. [2025-10-20T21:45:12.332447] [INFO ] [MAIN ] VGI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:12.340195] [DEBUG] [LOCAL ] VGI: Loaded 5280 existing records from primary storage. [2025-10-20T21:45:12.340527] [INFO ] [MAIN ] VGI: Loaded 5280 existing records. [2025-10-20T21:45:12.483935] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T21:45:12.574154] [WARN ] [MAIN ] VGI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:12.574401] [INFO ] [MAIN ] VGI: Wiping existing primary data before full refresh. [2025-10-20T21:45:12.574450] [INFO ] [LOCAL ] VGI: Starting deletion of primary data. [2025-10-20T21:45:12.574588] [DEBUG] [LOCAL ] VGI: Deleted primary data file. [2025-10-20T21:45:12.574626] [INFO ] [MAIN ] VGI: Starting full primary refresh. [2025-10-20T21:45:12.927406] [INFO ] [REMOTE ] VGI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:13.051338] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-01. [2025-10-20T21:45:13.493876] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T21:45:13.768196] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-01 (0.72 seconds). [2025-10-20T21:45:13.834069] [INFO ] [REMOTE ] VET: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T21:45:13.840761] [INFO ] [REMOTE ] VET: Full fetch complete (64564 records). [2025-10-20T21:45:13.840897] [DEBUG] [VALIDATOR ] VET: Validating primary data. [2025-10-20T21:45:13.856576] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-08. [2025-10-20T21:45:14.225733] [DEBUG] [VALIDATOR ] VET: All expected trading days are present. [2025-10-20T21:45:14.225840] [DEBUG] [VALIDATOR ] VET: Primary data validation passed. [2025-10-20T21:45:14.226883] [INFO ] [MAIN ] VET: Fetched primary data is valid. [2025-10-20T21:45:14.226946] [DEBUG] [LOCAL ] VET: Saving primary data. [2025-10-20T21:45:14.228250] [DEBUG] [LOCAL ] VET: Trimmed 64564 records to 44200 (keeping 200 days of data) [2025-10-20T21:45:14.478436] [DEBUG] [LOCAL ] VET: Primary data saved (44200 records). [2025-10-20T21:45:14.478534] [INFO ] [MAIN ] VET: Full history saved. [2025-10-20T21:45:14.478611] [INFO ] [MAIN ] Update Progress: 2217/2367 symbols completed. VET took 37.0s. Estimated remaining: 19m 18s. Rate: 7.8 symbols/min [2025-10-20T21:45:14.644084] [INFO ] [MAIN ] VGM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:14.653819] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-03. [2025-10-20T21:45:14.663793] [DEBUG] [LOCAL ] VGM: Loaded 18936 existing records from primary storage. [2025-10-20T21:45:14.664016] [INFO ] [MAIN ] VGM: Loaded 18936 existing records. [2025-10-20T21:45:14.858631] [WARN ] [MAIN ] VGM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:14.858878] [INFO ] [MAIN ] VGM: Wiping existing primary data before full refresh. [2025-10-20T21:45:14.858925] [INFO ] [LOCAL ] VGM: Starting deletion of primary data. [2025-10-20T21:45:14.859127] [DEBUG] [LOCAL ] VGM: Deleted primary data file. [2025-10-20T21:45:14.859181] [INFO ] [MAIN ] VGM: Starting full primary refresh. [2025-10-20T21:45:15.210396] [INFO ] [REMOTE ] VGM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:15.451361] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-01. [2025-10-20T21:45:15.962975] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T21:45:16.251354] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-02. [2025-10-20T21:45:16.273566] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-01 (0.82 seconds). [2025-10-20T21:45:16.523334] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T21:45:16.601049] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-10-20T21:45:17.051419] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-04. [2025-10-20T21:45:17.851405] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-03. [2025-10-20T21:45:18.441232] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-03 (0.59 seconds). [2025-10-20T21:45:18.651340] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-02. [2025-10-20T21:45:19.177682] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-04 (2.13 seconds). [2025-10-20T21:45:19.293850] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T21:45:19.451368] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-03. [2025-10-20T21:45:20.251377] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-04. [2025-10-20T21:45:20.379910] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T21:45:20.836570] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-04 (0.59 seconds). [2025-10-20T21:45:21.051376] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-04. [2025-10-20T21:45:21.851377] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-05. [2025-10-20T21:45:22.174502] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:45:22.651373] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-09. [2025-10-20T21:45:23.451363] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-05. [2025-10-20T21:45:23.931569] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T21:45:24.160881] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T21:45:24.251381] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-06. [2025-10-20T21:45:24.465801] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-09 (1.81 seconds). [2025-10-20T21:45:25.051432] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-05. [2025-10-20T21:45:25.400054] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-06 (1.15 seconds). [2025-10-20T21:45:25.779953] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-05 (0.73 seconds). [2025-10-20T21:45:25.851349] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-06. [2025-10-20T21:45:26.422078] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-06 (0.57 seconds). [2025-10-20T21:45:26.651374] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-07. [2025-10-20T21:45:27.262359] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T21:45:27.451361] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-06. [2025-10-20T21:45:28.251400] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-08. [2025-10-20T21:45:28.896502] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-08 (0.65 seconds). [2025-10-20T21:45:29.051416] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-09. [2025-10-20T21:45:29.423751] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-06 (1.97 seconds). [2025-10-20T21:45:29.840026] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-09 (0.79 seconds). [2025-10-20T21:45:29.851326] [INFO ] [REMOTE ] VGI: Fetching primary data for 2025-10. [2025-10-20T21:45:30.550989] [INFO ] [REMOTE ] VGI: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T21:45:30.552461] [INFO ] [REMOTE ] VGI: Full fetch complete (7607 records). [2025-10-20T21:45:30.552535] [DEBUG] [VALIDATOR ] VGI: Validating primary data. [2025-10-20T21:45:30.658125] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-07. [2025-10-20T21:45:30.918015] [DEBUG] [VALIDATOR ] VGI: All expected trading days are present. [2025-10-20T21:45:30.918099] [DEBUG] [VALIDATOR ] VGI: Primary data validation passed. [2025-10-20T21:45:30.918311] [INFO ] [MAIN ] VGI: Fetched primary data is valid. [2025-10-20T21:45:30.918349] [DEBUG] [LOCAL ] VGI: Saving primary data. [2025-10-20T21:45:30.918799] [DEBUG] [LOCAL ] VGI: Trimmed 7607 records to 5227 (keeping 200 days of data) [2025-10-20T21:45:30.949075] [DEBUG] [LOCAL ] VGI: Primary data saved (5227 records). [2025-10-20T21:45:30.949206] [INFO ] [MAIN ] VGI: Full history saved. [2025-10-20T21:45:30.949280] [INFO ] [MAIN ] Update Progress: 2218/2367 symbols completed. VGI took 18.8s. Estimated remaining: 19m 11s. Rate: 7.8 symbols/min [2025-10-20T21:45:31.118778] [INFO ] [MAIN ] VHI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:31.124006] [DEBUG] [LOCAL ] VHI: Loaded 2811 existing records from primary storage. [2025-10-20T21:45:31.124201] [INFO ] [MAIN ] VHI: Loaded 2811 existing records. [2025-10-20T21:45:31.301134] [WARN ] [MAIN ] VHI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:31.301370] [INFO ] [MAIN ] VHI: Wiping existing primary data before full refresh. [2025-10-20T21:45:31.301425] [INFO ] [LOCAL ] VHI: Starting deletion of primary data. [2025-10-20T21:45:31.301557] [DEBUG] [LOCAL ] VHI: Deleted primary data file. [2025-10-20T21:45:31.301595] [INFO ] [MAIN ] VHI: Starting full primary refresh. [2025-10-20T21:45:31.456515] [INFO ] [REMOTE ] VFC: Fetching primary data for 2025-10. [2025-10-20T21:45:31.669938] [INFO ] [REMOTE ] VHI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:31.724956] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T21:45:32.251401] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-01. [2025-10-20T21:45:32.862913] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-01 (0.61 seconds). [2025-10-20T21:45:33.051379] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-07. [2025-10-20T21:45:33.214380] [INFO ] [REMOTE ] VFC: Incremental fetch complete for 2025-10 (1.76 seconds). [2025-10-20T21:45:33.222322] [INFO ] [REMOTE ] VFC: Full fetch complete (77749 records). [2025-10-20T21:45:33.222481] [DEBUG] [VALIDATOR ] VFC: Validating primary data. [2025-10-20T21:45:33.627585] [DEBUG] [VALIDATOR ] VFC: All expected trading days are present. [2025-10-20T21:45:33.627745] [DEBUG] [VALIDATOR ] VFC: Primary data validation passed. [2025-10-20T21:45:33.628911] [INFO ] [MAIN ] VFC: Fetched primary data is valid. [2025-10-20T21:45:33.628974] [DEBUG] [LOCAL ] VFC: Saving primary data. [2025-10-20T21:45:33.630455] [DEBUG] [LOCAL ] VFC: Trimmed 77749 records to 53607 (keeping 200 days of data) [2025-10-20T21:45:33.854693] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-08. [2025-10-20T21:45:33.953628] [DEBUG] [LOCAL ] VFC: Primary data saved (53607 records). [2025-10-20T21:45:33.953789] [INFO ] [MAIN ] VFC: Full history saved. [2025-10-20T21:45:33.953870] [INFO ] [MAIN ] Update Progress: 2219/2367 symbols completed. VFC took 42.9s. Estimated remaining: 19m 2s. Rate: 7.8 symbols/min [2025-10-20T21:45:34.121511] [INFO ] [MAIN ] VIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:34.121697] [DEBUG] [LOCAL ] VIA: No existing data file found in primary storage. [2025-10-20T21:45:34.121747] [WARN ] [MAIN ] VIA: No existing data found, performing full refresh instead. [2025-10-20T21:45:34.121880] [INFO ] [MAIN ] VIA: Wiping existing primary data before full refresh. [2025-10-20T21:45:34.121945] [INFO ] [LOCAL ] VIA: Starting deletion of primary data. [2025-10-20T21:45:34.121994] [DEBUG] [LOCAL ] VIA: No primary data file found to delete. [2025-10-20T21:45:34.122026] [INFO ] [MAIN ] VIA: Starting full primary refresh. [2025-10-20T21:45:34.471665] [INFO ] [REMOTE ] VIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:34.651391] [INFO ] [REMOTE ] VIA: Fetching primary data for 2025-01. [2025-10-20T21:45:34.692518] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T21:45:34.850690] [DEBUG] [REMOTE ] VIA: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:45:34.850955] [WARN ] [REMOTE ] VIA: No data returned for month 2025-01 [2025-10-20T21:45:34.851043] [FATAL] [REMOTE ] VIA: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:45:34.851112] [FATAL] [MAIN ] VIA: Failed to fetch primary data. [2025-10-20T21:45:34.851177] [WARN ] [MAIN ] VIA: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:45:34.851249] [INFO ] [LOCAL ] VIA: Starting deletion of primary data. [2025-10-20T21:45:34.851312] [DEBUG] [LOCAL ] VIA: No primary data file found to delete. [2025-10-20T21:45:34.851348] [INFO ] [LOCAL ] VIA: Starting deletion of derivative data. [2025-10-20T21:45:34.851386] [DEBUG] [LOCAL ] VIA: No derivative data file found to delete. [2025-10-20T21:45:34.851429] [INFO ] [MAIN ] Update Progress: 2220/2367 symbols completed. VIA took 0.9s. Estimated remaining: 18m 54s. Rate: 7.8 symbols/min [2025-10-20T21:45:35.025394] [INFO ] [MAIN ] VICI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:35.074608] [DEBUG] [LOCAL ] VICI: Loaded 54433 existing records from primary storage. [2025-10-20T21:45:35.074978] [INFO ] [MAIN ] VICI: Loaded 54433 existing records. [2025-10-20T21:45:35.260556] [WARN ] [MAIN ] VICI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:35.260861] [INFO ] [MAIN ] VICI: Wiping existing primary data before full refresh. [2025-10-20T21:45:35.260997] [INFO ] [LOCAL ] VICI: Starting deletion of primary data. [2025-10-20T21:45:35.261431] [DEBUG] [LOCAL ] VICI: Deleted primary data file. [2025-10-20T21:45:35.261486] [INFO ] [MAIN ] VICI: Starting full primary refresh. [2025-10-20T21:45:35.452379] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-09. [2025-10-20T21:45:35.482034] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-07 (2.43 seconds). [2025-10-20T21:45:35.683559] [INFO ] [REMOTE ] VICI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:36.251463] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-01. [2025-10-20T21:45:36.593507] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-09 (1.14 seconds). [2025-10-20T21:45:37.051374] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-08. [2025-10-20T21:45:37.851369] [INFO ] [REMOTE ] VGM: Fetching primary data for 2025-10. [2025-10-20T21:45:38.319031] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-01 (2.07 seconds). [2025-10-20T21:45:38.441587] [INFO ] [REMOTE ] VGM: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T21:45:38.444694] [INFO ] [REMOTE ] VGM: Full fetch complete (25802 records). [2025-10-20T21:45:38.444766] [DEBUG] [VALIDATOR ] VGM: Validating primary data. [2025-10-20T21:45:38.658776] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-02. [2025-10-20T21:45:38.829141] [DEBUG] [VALIDATOR ] VGM: All expected trading days are present. [2025-10-20T21:45:38.829222] [DEBUG] [VALIDATOR ] VGM: Primary data validation passed. [2025-10-20T21:45:38.829754] [INFO ] [MAIN ] VGM: Fetched primary data is valid. [2025-10-20T21:45:38.829813] [DEBUG] [LOCAL ] VGM: Saving primary data. [2025-10-20T21:45:38.830496] [DEBUG] [LOCAL ] VGM: Trimmed 25802 records to 18689 (keeping 200 days of data) [2025-10-20T21:45:38.939538] [DEBUG] [LOCAL ] VGM: Primary data saved (18689 records). [2025-10-20T21:45:38.939697] [INFO ] [MAIN ] VGM: Full history saved. [2025-10-20T21:45:38.939787] [INFO ] [MAIN ] Update Progress: 2221/2367 symbols completed. VGM took 24.5s. Estimated remaining: 18m 46s. Rate: 7.8 symbols/min [2025-10-20T21:45:39.110513] [INFO ] [MAIN ] VIK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:39.144972] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-08 (2.09 seconds). [2025-10-20T21:45:39.198885] [DEBUG] [LOCAL ] VIK: Loaded 53232 existing records from primary storage. [2025-10-20T21:45:39.199293] [INFO ] [MAIN ] VIK: Loaded 53232 existing records. [2025-10-20T21:45:39.314305] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-02 (0.66 seconds). [2025-10-20T21:45:39.391471] [WARN ] [MAIN ] VIK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:39.391709] [INFO ] [MAIN ] VIK: Wiping existing primary data before full refresh. [2025-10-20T21:45:39.391764] [INFO ] [LOCAL ] VIK: Starting deletion of primary data. [2025-10-20T21:45:39.391978] [DEBUG] [LOCAL ] VIK: Deleted primary data file. [2025-10-20T21:45:39.392033] [INFO ] [MAIN ] VIK: Starting full primary refresh. [2025-10-20T21:45:39.456551] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-03. [2025-10-20T21:45:39.746551] [INFO ] [REMOTE ] VIK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:39.958703] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-03 (0.50 seconds). [2025-10-20T21:45:40.251442] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-01. [2025-10-20T21:45:41.051451] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-09. [2025-10-20T21:45:41.851350] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-04. [2025-10-20T21:45:41.964126] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T21:45:42.354900] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-04 (0.50 seconds). [2025-10-20T21:45:42.651353] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-02. [2025-10-20T21:45:43.032153] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T21:45:43.451394] [INFO ] [REMOTE ] VG: Fetching primary data for 2025-10. [2025-10-20T21:45:44.251406] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-02. [2025-10-20T21:45:44.494072] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T21:45:45.051365] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-03. [2025-10-20T21:45:45.362011] [INFO ] [REMOTE ] VG: Incremental fetch complete for 2025-10 (1.91 seconds). [2025-10-20T21:45:45.370460] [INFO ] [REMOTE ] VG: Full fetch complete (71909 records). [2025-10-20T21:45:45.370637] [DEBUG] [VALIDATOR ] VG: Validating primary data. [2025-10-20T21:45:45.576314] [WARN ] [VALIDATOR ] VG: Validation failed - insufficient historical data. Have 186 unique days, need 200 days [2025-10-20T21:45:45.577101] [FATAL] [MAIN ] VG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:45:45.591465] [WARN ] [MAIN ] VG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:45:45.601120] [INFO ] [LOCAL ] VG: Starting deletion of primary data. [2025-10-20T21:45:45.601287] [DEBUG] [LOCAL ] VG: No primary data file found to delete. [2025-10-20T21:45:45.601340] [INFO ] [LOCAL ] VG: Starting deletion of derivative data. [2025-10-20T21:45:45.601385] [DEBUG] [LOCAL ] VG: No derivative data file found to delete. [2025-10-20T21:45:45.601469] [INFO ] [MAIN ] Update Progress: 2222/2367 symbols completed. VG took 39.3s. Estimated remaining: 18m 39s. Rate: 7.8 symbols/min [2025-10-20T21:45:45.676607] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T21:45:45.794333] [INFO ] [MAIN ] VIPS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:45:45.844193] [DEBUG] [LOCAL ] VIPS: Loaded 52050 existing records from primary storage. [2025-10-20T21:45:45.844510] [INFO ] [MAIN ] VIPS: Loaded 52050 existing records. [2025-10-20T21:45:45.857151] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-03. [2025-10-20T21:45:46.028344] [WARN ] [MAIN ] VIPS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:45:46.028592] [INFO ] [MAIN ] VIPS: Wiping existing primary data before full refresh. [2025-10-20T21:45:46.028663] [INFO ] [LOCAL ] VIPS: Starting deletion of primary data. [2025-10-20T21:45:46.028881] [DEBUG] [LOCAL ] VIPS: Deleted primary data file. [2025-10-20T21:45:46.028921] [INFO ] [MAIN ] VIPS: Starting full primary refresh. [2025-10-20T21:45:46.383405] [INFO ] [REMOTE ] VIPS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:45:46.651344] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-01. [2025-10-20T21:45:47.150410] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-03 (2.10 seconds). [2025-10-20T21:45:47.451469] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-04. [2025-10-20T21:45:47.826807] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-03 (1.97 seconds). [2025-10-20T21:45:48.251410] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-04. [2025-10-20T21:45:48.287327] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T21:45:49.030857] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-04 (1.58 seconds). [2025-10-20T21:45:49.051340] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-05. [2025-10-20T21:45:49.851478] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-02. [2025-10-20T21:45:50.021406] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T21:45:50.651360] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-05. [2025-10-20T21:45:50.984899] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:45:51.451406] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-06. [2025-10-20T21:45:51.530202] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T21:45:52.116388] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-05 (1.47 seconds). [2025-10-20T21:45:52.251341] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-06. [2025-10-20T21:45:53.051394] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-05. [2025-10-20T21:45:53.305220] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T21:45:53.481633] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-05 (0.43 seconds). [2025-10-20T21:45:53.822283] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T21:45:53.851365] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-07. [2025-10-20T21:45:54.651400] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-06. [2025-10-20T21:45:55.143351] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-06 (0.49 seconds). [2025-10-20T21:45:55.451375] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-03. [2025-10-20T21:45:55.598745] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T21:45:56.251445] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-08. [2025-10-20T21:45:57.051385] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-07. [2025-10-20T21:45:57.262850] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T21:45:57.630680] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-07 (0.58 seconds). [2025-10-20T21:45:57.851409] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-08. [2025-10-20T21:45:57.968716] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T21:45:58.329914] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-08 (0.48 seconds). [2025-10-20T21:45:58.651381] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-09. [2025-10-20T21:45:59.134864] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:45:59.451400] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-09. [2025-10-20T21:46:00.251393] [INFO ] [REMOTE ] VHI: Fetching primary data for 2025-10. [2025-10-20T21:46:00.876613] [INFO ] [REMOTE ] VHI: Incremental fetch complete for 2025-10 (0.63 seconds). [2025-10-20T21:46:00.877595] [INFO ] [REMOTE ] VHI: Full fetch complete (4343 records). [2025-10-20T21:46:00.877694] [DEBUG] [VALIDATOR ] VHI: Validating primary data. [2025-10-20T21:46:01.065050] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-07. [2025-10-20T21:46:01.264144] [DEBUG] [VALIDATOR ] VHI: All expected trading days are present. [2025-10-20T21:46:01.264246] [DEBUG] [VALIDATOR ] VHI: Primary data validation passed. [2025-10-20T21:46:01.264477] [INFO ] [MAIN ] VHI: Fetched primary data is valid. [2025-10-20T21:46:01.264528] [DEBUG] [LOCAL ] VHI: Saving primary data. [2025-10-20T21:46:01.264940] [DEBUG] [LOCAL ] VHI: Trimmed 4343 records to 2733 (keeping 200 days of data) [2025-10-20T21:46:01.282080] [DEBUG] [LOCAL ] VHI: Primary data saved (2733 records). [2025-10-20T21:46:01.282163] [INFO ] [MAIN ] VHI: Full history saved. [2025-10-20T21:46:01.282232] [INFO ] [MAIN ] Update Progress: 2223/2367 symbols completed. VHI took 30.3s. Estimated remaining: 18m 31s. Rate: 7.8 symbols/min [2025-10-20T21:46:01.476535] [INFO ] [MAIN ] VIRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:01.518890] [DEBUG] [LOCAL ] VIRT: Loaded 47324 existing records from primary storage. [2025-10-20T21:46:01.519209] [INFO ] [MAIN ] VIRT: Loaded 47324 existing records. [2025-10-20T21:46:01.729761] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-09 (2.28 seconds). [2025-10-20T21:46:01.732864] [WARN ] [MAIN ] VIRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:01.743594] [INFO ] [MAIN ] VIRT: Wiping existing primary data before full refresh. [2025-10-20T21:46:01.743901] [INFO ] [LOCAL ] VIRT: Starting deletion of primary data. [2025-10-20T21:46:01.744277] [DEBUG] [LOCAL ] VIRT: Deleted primary data file. [2025-10-20T21:46:01.744337] [INFO ] [MAIN ] VIRT: Starting full primary refresh. [2025-10-20T21:46:01.856437] [INFO ] [REMOTE ] VICI: Fetching primary data for 2025-10. [2025-10-20T21:46:02.130955] [INFO ] [REMOTE ] VIRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:02.603232] [INFO ] [REMOTE ] VICI: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:46:02.611228] [INFO ] [REMOTE ] VICI: Full fetch complete (77801 records). [2025-10-20T21:46:02.611348] [DEBUG] [VALIDATOR ] VICI: Validating primary data. [2025-10-20T21:46:02.654938] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-04. [2025-10-20T21:46:03.008467] [DEBUG] [VALIDATOR ] VICI: All expected trading days are present. [2025-10-20T21:46:03.008670] [DEBUG] [VALIDATOR ] VICI: Primary data validation passed. [2025-10-20T21:46:03.010264] [INFO ] [MAIN ] VICI: Fetched primary data is valid. [2025-10-20T21:46:03.010346] [DEBUG] [LOCAL ] VICI: Saving primary data. [2025-10-20T21:46:03.041264] [DEBUG] [LOCAL ] VICI: Trimmed 77801 records to 53653 (keeping 200 days of data) [2025-10-20T21:46:03.042480] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T21:46:03.372049] [DEBUG] [LOCAL ] VICI: Primary data saved (53653 records). [2025-10-20T21:46:03.372187] [INFO ] [MAIN ] VICI: Full history saved. [2025-10-20T21:46:03.372260] [INFO ] [MAIN ] Update Progress: 2224/2367 symbols completed. VICI took 28.5s. Estimated remaining: 18m 23s. Rate: 7.8 symbols/min [2025-10-20T21:46:03.456579] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-01. [2025-10-20T21:46:03.544745] [INFO ] [MAIN ] VIST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:03.586578] [DEBUG] [LOCAL ] VIST: Loaded 47252 existing records from primary storage. [2025-10-20T21:46:03.586898] [INFO ] [MAIN ] VIST: Loaded 47252 existing records. [2025-10-20T21:46:03.762074] [WARN ] [MAIN ] VIST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:03.762286] [INFO ] [MAIN ] VIST: Wiping existing primary data before full refresh. [2025-10-20T21:46:03.762332] [INFO ] [LOCAL ] VIST: Starting deletion of primary data. [2025-10-20T21:46:03.762507] [DEBUG] [LOCAL ] VIST: Deleted primary data file. [2025-10-20T21:46:03.762543] [INFO ] [MAIN ] VIST: Starting full primary refresh. [2025-10-20T21:46:04.112768] [INFO ] [REMOTE ] VIST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:04.251346] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-01. [2025-10-20T21:46:04.492898] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T21:46:04.862978] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-01 (1.41 seconds). [2025-10-20T21:46:05.051369] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-02. [2025-10-20T21:46:05.851351] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-08. [2025-10-20T21:46:05.950017] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T21:46:06.652349] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-05. [2025-10-20T21:46:06.652248] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T21:46:07.451377] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-02. [2025-10-20T21:46:07.653998] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T21:46:08.251471] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-09. [2025-10-20T21:46:08.468119] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T21:46:09.051352] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-06. [2025-10-20T21:46:09.121112] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-02 (1.67 seconds). [2025-10-20T21:46:09.851429] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-03. [2025-10-20T21:46:09.960208] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T21:46:10.659842] [INFO ] [REMOTE ] VIK: Fetching primary data for 2025-10. [2025-10-20T21:46:10.683789] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T21:46:11.451355] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-07. [2025-10-20T21:46:11.609123] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-03 (1.76 seconds). [2025-10-20T21:46:12.251434] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-03. [2025-10-20T21:46:12.356180] [INFO ] [REMOTE ] VIK: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T21:46:12.364392] [INFO ] [REMOTE ] VIK: Full fetch complete (75978 records). [2025-10-20T21:46:12.364545] [DEBUG] [VALIDATOR ] VIK: Validating primary data. [2025-10-20T21:46:12.793924] [DEBUG] [VALIDATOR ] VIK: All expected trading days are present. [2025-10-20T21:46:12.794013] [DEBUG] [VALIDATOR ] VIK: Primary data validation passed. [2025-10-20T21:46:12.795371] [INFO ] [MAIN ] VIK: Fetched primary data is valid. [2025-10-20T21:46:12.795461] [DEBUG] [LOCAL ] VIK: Saving primary data. [2025-10-20T21:46:12.797033] [DEBUG] [LOCAL ] VIK: Trimmed 75978 records to 52392 (keeping 200 days of data) [2025-10-20T21:46:13.083010] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-04. [2025-10-20T21:46:13.132548] [DEBUG] [LOCAL ] VIK: Primary data saved (52392 records). [2025-10-20T21:46:13.132843] [INFO ] [MAIN ] VIK: Full history saved. [2025-10-20T21:46:13.132949] [INFO ] [MAIN ] Update Progress: 2225/2367 symbols completed. VIK took 34.2s. Estimated remaining: 18m 16s. Rate: 7.8 symbols/min [2025-10-20T21:46:13.271554] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T21:46:13.370205] [INFO ] [MAIN ] VIV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:13.415187] [DEBUG] [LOCAL ] VIV: Loaded 45447 existing records from primary storage. [2025-10-20T21:46:13.415525] [INFO ] [MAIN ] VIV: Loaded 45447 existing records. [2025-10-20T21:46:13.614122] [WARN ] [MAIN ] VIV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:13.614341] [INFO ] [MAIN ] VIV: Wiping existing primary data before full refresh. [2025-10-20T21:46:13.614391] [INFO ] [LOCAL ] VIV: Starting deletion of primary data. [2025-10-20T21:46:13.614606] [DEBUG] [LOCAL ] VIV: Deleted primary data file. [2025-10-20T21:46:13.614696] [INFO ] [MAIN ] VIV: Starting full primary refresh. [2025-10-20T21:46:13.854946] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-08. [2025-10-20T21:46:14.027125] [INFO ] [REMOTE ] VIV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:14.053511] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T21:46:14.651357] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-04. [2025-10-20T21:46:14.752445] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-04 (1.67 seconds). [2025-10-20T21:46:15.451359] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-05. [2025-10-20T21:46:15.556926] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T21:46:16.251372] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-01. [2025-10-20T21:46:16.371003] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T21:46:17.004601] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T21:46:17.051368] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-05. [2025-10-20T21:46:17.624797] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-01 (1.37 seconds). [2025-10-20T21:46:17.851384] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-06. [2025-10-20T21:46:18.249747] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:46:18.651383] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-09. [2025-10-20T21:46:19.418455] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T21:46:19.451347] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-07. [2025-10-20T21:46:20.251407] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-06. [2025-10-20T21:46:20.380780] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T21:46:21.051382] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-02. [2025-10-20T21:46:21.171288] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:46:21.763313] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:46:21.851338] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-08. [2025-10-20T21:46:22.230693] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-02 (1.18 seconds). [2025-10-20T21:46:22.651426] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-07. [2025-10-20T21:46:23.452418] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T21:46:23.451405] [INFO ] [REMOTE ] VIPS: Fetching primary data for 2025-10. [2025-10-20T21:46:24.251380] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-03. [2025-10-20T21:46:24.371584] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:46:25.051362] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-09. [2025-10-20T21:46:25.074112] [INFO ] [REMOTE ] VIPS: Incremental fetch complete for 2025-10 (1.62 seconds). [2025-10-20T21:46:25.082818] [INFO ] [REMOTE ] VIPS: Full fetch complete (74846 records). [2025-10-20T21:46:25.082929] [DEBUG] [VALIDATOR ] VIPS: Validating primary data. [2025-10-20T21:46:25.507559] [DEBUG] [VALIDATOR ] VIPS: All expected trading days are present. [2025-10-20T21:46:25.507659] [DEBUG] [VALIDATOR ] VIPS: Primary data validation passed. [2025-10-20T21:46:25.508786] [INFO ] [MAIN ] VIPS: Fetched primary data is valid. [2025-10-20T21:46:25.508846] [DEBUG] [LOCAL ] VIPS: Saving primary data. [2025-10-20T21:46:25.510203] [DEBUG] [LOCAL ] VIPS: Trimmed 74846 records to 51269 (keeping 200 days of data) [2025-10-20T21:46:25.830296] [DEBUG] [LOCAL ] VIPS: Primary data saved (51269 records). [2025-10-20T21:46:25.844787] [INFO ] [MAIN ] VIPS: Full history saved. [2025-10-20T21:46:25.844949] [INFO ] [MAIN ] Update Progress: 2226/2367 symbols completed. VIPS took 40.2s. Estimated remaining: 18m 8s. Rate: 7.8 symbols/min [2025-10-20T21:46:25.856318] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-08. [2025-10-20T21:46:25.891556] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T21:46:26.027513] [INFO ] [MAIN ] VKQ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:26.045036] [DEBUG] [LOCAL ] VKQ: Loaded 18083 existing records from primary storage. [2025-10-20T21:46:26.045270] [INFO ] [MAIN ] VKQ: Loaded 18083 existing records. [2025-10-20T21:46:26.246073] [WARN ] [MAIN ] VKQ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:26.246278] [INFO ] [MAIN ] VKQ: Wiping existing primary data before full refresh. [2025-10-20T21:46:26.246324] [INFO ] [LOCAL ] VKQ: Starting deletion of primary data. [2025-10-20T21:46:26.246498] [DEBUG] [LOCAL ] VKQ: Deleted primary data file. [2025-10-20T21:46:26.246578] [INFO ] [MAIN ] VKQ: Starting full primary refresh. [2025-10-20T21:46:26.647236] [INFO ] [REMOTE ] VKQ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:26.651327] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-01. [2025-10-20T21:46:26.711479] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T21:46:27.152475] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:46:27.451411] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-09. [2025-10-20T21:46:27.620199] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-01 (0.97 seconds). [2025-10-20T21:46:28.251436] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-04. [2025-10-20T21:46:29.051455] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-02. [2025-10-20T21:46:29.193537] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T21:46:29.455425] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-04 (1.20 seconds). [2025-10-20T21:46:29.851624] [INFO ] [REMOTE ] VIRT: Fetching primary data for 2025-10. [2025-10-20T21:46:29.887755] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-02 (0.84 seconds). [2025-10-20T21:46:30.651409] [INFO ] [REMOTE ] VIST: Fetching primary data for 2025-10. [2025-10-20T21:46:31.327855] [INFO ] [REMOTE ] VIRT: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T21:46:31.334567] [INFO ] [REMOTE ] VIRT: Full fetch complete (67608 records). [2025-10-20T21:46:31.334711] [DEBUG] [VALIDATOR ] VIRT: Validating primary data. [2025-10-20T21:46:31.456468] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-03. [2025-10-20T21:46:31.723000] [DEBUG] [VALIDATOR ] VIRT: All expected trading days are present. [2025-10-20T21:46:31.723080] [DEBUG] [VALIDATOR ] VIRT: Primary data validation passed. [2025-10-20T21:46:31.724097] [INFO ] [MAIN ] VIRT: Fetched primary data is valid. [2025-10-20T21:46:31.724159] [DEBUG] [LOCAL ] VIRT: Saving primary data. [2025-10-20T21:46:31.725409] [DEBUG] [LOCAL ] VIRT: Trimmed 67608 records to 46585 (keeping 200 days of data) [2025-10-20T21:46:32.007985] [DEBUG] [LOCAL ] VIRT: Primary data saved (46585 records). [2025-10-20T21:46:32.008134] [INFO ] [MAIN ] VIRT: Full history saved. [2025-10-20T21:46:32.008556] [INFO ] [MAIN ] Update Progress: 2227/2367 symbols completed. VIRT took 30.7s. Estimated remaining: 18m 0s. Rate: 7.8 symbols/min [2025-10-20T21:46:32.187022] [INFO ] [MAIN ] VLN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:32.218329] [DEBUG] [LOCAL ] VLN: Loaded 31295 existing records from primary storage. [2025-10-20T21:46:32.218609] [INFO ] [MAIN ] VLN: Loaded 31295 existing records. [2025-10-20T21:46:32.261517] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-05. [2025-10-20T21:46:32.329149] [INFO ] [REMOTE ] VIST: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:46:32.370153] [INFO ] [REMOTE ] VIST: Full fetch complete (67515 records). [2025-10-20T21:46:32.385775] [DEBUG] [VALIDATOR ] VIST: Validating primary data. [2025-10-20T21:46:32.484481] [WARN ] [MAIN ] VLN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:32.518420] [INFO ] [MAIN ] VLN: Wiping existing primary data before full refresh. [2025-10-20T21:46:32.542088] [INFO ] [LOCAL ] VLN: Starting deletion of primary data. [2025-10-20T21:46:32.564294] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-03 (1.11 seconds). [2025-10-20T21:46:32.564625] [DEBUG] [LOCAL ] VLN: Deleted primary data file. [2025-10-20T21:46:32.579482] [INFO ] [MAIN ] VLN: Starting full primary refresh. [2025-10-20T21:46:33.054740] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-04. [2025-10-20T21:46:33.233342] [DEBUG] [VALIDATOR ] VIST: All expected trading days are present. [2025-10-20T21:46:33.252439] [DEBUG] [VALIDATOR ] VIST: Primary data validation passed. [2025-10-20T21:46:33.259867] [INFO ] [MAIN ] VIST: Fetched primary data is valid. [2025-10-20T21:46:33.265439] [DEBUG] [LOCAL ] VIST: Saving primary data. [2025-10-20T21:46:33.297231] [INFO ] [REMOTE ] VLN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:33.298749] [DEBUG] [LOCAL ] VIST: Trimmed 67515 records to 46676 (keeping 200 days of data) [2025-10-20T21:46:33.583172] [DEBUG] [LOCAL ] VIST: Primary data saved (46676 records). [2025-10-20T21:46:33.583325] [INFO ] [MAIN ] VIST: Full history saved. [2025-10-20T21:46:33.583407] [INFO ] [MAIN ] Update Progress: 2228/2367 symbols completed. VIST took 30.2s. Estimated remaining: 17m 52s. Rate: 7.8 symbols/min [2025-10-20T21:46:33.752390] [INFO ] [MAIN ] VLO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:33.780538] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:46:33.836763] [DEBUG] [LOCAL ] VLO: Loaded 53782 existing records from primary storage. [2025-10-20T21:46:33.837103] [INFO ] [MAIN ] VLO: Loaded 53782 existing records. [2025-10-20T21:46:33.857208] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-01. [2025-10-20T21:46:34.014810] [WARN ] [MAIN ] VLO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:34.015019] [INFO ] [MAIN ] VLO: Wiping existing primary data before full refresh. [2025-10-20T21:46:34.015066] [INFO ] [LOCAL ] VLO: Starting deletion of primary data. [2025-10-20T21:46:34.015443] [DEBUG] [LOCAL ] VLO: Deleted primary data file. [2025-10-20T21:46:34.015481] [INFO ] [MAIN ] VLO: Starting full primary refresh. [2025-10-20T21:46:34.376360] [INFO ] [REMOTE ] VLO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:34.378749] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T21:46:34.651423] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-01. [2025-10-20T21:46:35.381916] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T21:46:35.451371] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-02. [2025-10-20T21:46:36.260563] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:46:36.259391] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-06. [2025-10-20T21:46:36.464598] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T21:46:37.051435] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-02. [2025-10-20T21:46:37.820075] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T21:46:37.851361] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-07. [2025-10-20T21:46:38.651434] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-05. [2025-10-20T21:46:38.817534] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T21:46:39.451441] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-03. [2025-10-20T21:46:39.553074] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T21:46:39.572953] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T21:46:40.251483] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-03. [2025-10-20T21:46:41.051457] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-08. [2025-10-20T21:46:41.316750] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-03 (1.07 seconds). [2025-10-20T21:46:41.350734] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-03 (1.90 seconds). [2025-10-20T21:46:41.851395] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-06. [2025-10-20T21:46:42.386717] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-08 (1.34 seconds). [2025-10-20T21:46:42.489947] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T21:46:42.651336] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-04. [2025-10-20T21:46:43.451427] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-09. [2025-10-20T21:46:44.251406] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-07. [2025-10-20T21:46:44.563508] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-04 (1.91 seconds). [2025-10-20T21:46:44.962170] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T21:46:45.051344] [INFO ] [REMOTE ] VIV: Fetching primary data for 2025-10. [2025-10-20T21:46:45.237887] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-07 (0.99 seconds). [2025-10-20T21:46:45.851393] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-04. [2025-10-20T21:46:46.547375] [INFO ] [REMOTE ] VIV: Incremental fetch complete for 2025-10 (1.50 seconds). [2025-10-20T21:46:46.553597] [INFO ] [REMOTE ] VIV: Full fetch complete (64434 records). [2025-10-20T21:46:46.553728] [DEBUG] [VALIDATOR ] VIV: Validating primary data. [2025-10-20T21:46:46.656912] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-08. [2025-10-20T21:46:46.955914] [DEBUG] [VALIDATOR ] VIV: All expected trading days are present. [2025-10-20T21:46:46.955994] [DEBUG] [VALIDATOR ] VIV: Primary data validation passed. [2025-10-20T21:46:46.956994] [INFO ] [MAIN ] VIV: Fetched primary data is valid. [2025-10-20T21:46:46.957061] [DEBUG] [LOCAL ] VIV: Saving primary data. [2025-10-20T21:46:46.958291] [DEBUG] [LOCAL ] VIV: Trimmed 64434 records to 44697 (keeping 200 days of data) [2025-10-20T21:46:47.089085] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-04 (1.24 seconds). [2025-10-20T21:46:47.243068] [DEBUG] [LOCAL ] VIV: Primary data saved (44697 records). [2025-10-20T21:46:47.243191] [INFO ] [MAIN ] VIV: Full history saved. [2025-10-20T21:46:47.243265] [INFO ] [MAIN ] Update Progress: 2229/2367 symbols completed. VIV took 34.1s. Estimated remaining: 17m 45s. Rate: 7.8 symbols/min [2025-10-20T21:46:47.408853] [INFO ] [MAIN ] VLRS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:47.443956] [DEBUG] [LOCAL ] VLRS: Loaded 40231 existing records from primary storage. [2025-10-20T21:46:47.444259] [INFO ] [MAIN ] VLRS: Loaded 40231 existing records. [2025-10-20T21:46:47.456910] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-05. [2025-10-20T21:46:47.626321] [WARN ] [MAIN ] VLRS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:47.626713] [INFO ] [MAIN ] VLRS: Wiping existing primary data before full refresh. [2025-10-20T21:46:47.626768] [INFO ] [LOCAL ] VLRS: Starting deletion of primary data. [2025-10-20T21:46:47.627070] [DEBUG] [LOCAL ] VLRS: Deleted primary data file. [2025-10-20T21:46:47.627128] [INFO ] [MAIN ] VLRS: Starting full primary refresh. [2025-10-20T21:46:47.878484] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-08 (1.22 seconds). [2025-10-20T21:46:47.994612] [INFO ] [REMOTE ] VLRS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:48.251355] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-01. [2025-10-20T21:46:48.655691] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:46:49.051455] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-09. [2025-10-20T21:46:49.788539] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:46:49.851376] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-02. [2025-10-20T21:46:50.138435] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-09 (1.09 seconds). [2025-10-20T21:46:50.651429] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-05. [2025-10-20T21:46:51.094404] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-02 (1.24 seconds). [2025-10-20T21:46:51.451477] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-06. [2025-10-20T21:46:52.251439] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-03. [2025-10-20T21:46:52.542877] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T21:46:52.569253] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-06 (1.12 seconds). [2025-10-20T21:46:53.051436] [INFO ] [REMOTE ] VKQ: Fetching primary data for 2025-10. [2025-10-20T21:46:53.750371] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T21:46:53.798826] [INFO ] [REMOTE ] VKQ: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:46:53.802061] [INFO ] [REMOTE ] VKQ: Full fetch complete (24608 records). [2025-10-20T21:46:53.802165] [DEBUG] [VALIDATOR ] VKQ: Validating primary data. [2025-10-20T21:46:53.856579] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-06. [2025-10-20T21:46:54.182540] [DEBUG] [VALIDATOR ] VKQ: All expected trading days are present. [2025-10-20T21:46:54.182612] [DEBUG] [VALIDATOR ] VKQ: Primary data validation passed. [2025-10-20T21:46:54.183129] [INFO ] [MAIN ] VKQ: Fetched primary data is valid. [2025-10-20T21:46:54.183183] [DEBUG] [LOCAL ] VKQ: Saving primary data. [2025-10-20T21:46:54.183839] [DEBUG] [LOCAL ] VKQ: Trimmed 24608 records to 17880 (keeping 200 days of data) [2025-10-20T21:46:54.284770] [DEBUG] [LOCAL ] VKQ: Primary data saved (17880 records). [2025-10-20T21:46:54.284895] [INFO ] [MAIN ] VKQ: Full history saved. [2025-10-20T21:46:54.284967] [INFO ] [MAIN ] Update Progress: 2230/2367 symbols completed. VKQ took 28.4s. Estimated remaining: 17m 37s. Rate: 7.8 symbols/min [2025-10-20T21:46:54.451232] [INFO ] [MAIN ] VLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:46:54.461331] [DEBUG] [LOCAL ] VLT: Loaded 4468 existing records from primary storage. [2025-10-20T21:46:54.461514] [INFO ] [MAIN ] VLT: Loaded 4468 existing records. [2025-10-20T21:46:54.635725] [WARN ] [MAIN ] VLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:46:54.635944] [INFO ] [MAIN ] VLT: Wiping existing primary data before full refresh. [2025-10-20T21:46:54.635995] [INFO ] [LOCAL ] VLT: Starting deletion of primary data. [2025-10-20T21:46:54.636166] [DEBUG] [LOCAL ] VLT: Deleted primary data file. [2025-10-20T21:46:54.636221] [INFO ] [MAIN ] VLT: Starting full primary refresh. [2025-10-20T21:46:54.656701] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-04. [2025-10-20T21:46:55.001087] [INFO ] [REMOTE ] VLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:46:55.451351] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-07. [2025-10-20T21:46:55.749540] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T21:46:56.251416] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-01. [2025-10-20T21:46:56.388678] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:46:56.861354] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T21:46:57.042481] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-01 (0.79 seconds). [2025-10-20T21:46:57.051324] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-08. [2025-10-20T21:46:57.851405] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-07. [2025-10-20T21:46:58.623932] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-08 (1.57 seconds). [2025-10-20T21:46:58.651354] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-05. [2025-10-20T21:46:59.451352] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-02. [2025-10-20T21:46:59.866727] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T21:47:00.093086] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-02 (0.64 seconds). [2025-10-20T21:47:00.251352] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-08. [2025-10-20T21:47:00.280229] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:47:01.051403] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-03. [2025-10-20T21:47:01.737438] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-03 (0.69 seconds). [2025-10-20T21:47:01.851348] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-04. [2025-10-20T21:47:02.133596] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T21:47:02.281079] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T21:47:02.651382] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-05. [2025-10-20T21:47:03.296555] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-05 (0.65 seconds). [2025-10-20T21:47:03.451367] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-06. [2025-10-20T21:47:04.029935] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-06 (0.58 seconds). [2025-10-20T21:47:04.251364] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-06. [2025-10-20T21:47:05.051411] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-09. [2025-10-20T21:47:05.700533] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T21:47:05.851362] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-07. [2025-10-20T21:47:06.651468] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-07. [2025-10-20T21:47:07.027874] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-09 (1.98 seconds). [2025-10-20T21:47:07.430023] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T21:47:07.453035] [INFO ] [REMOTE ] VLO: Fetching primary data for 2025-10. [2025-10-20T21:47:07.458676] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:47:08.163981] [INFO ] [REMOTE ] VLO: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T21:47:08.172483] [INFO ] [REMOTE ] VLO: Full fetch complete (76923 records). [2025-10-20T21:47:08.172716] [DEBUG] [VALIDATOR ] VLO: Validating primary data. [2025-10-20T21:47:08.256723] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-09. [2025-10-20T21:47:08.568551] [DEBUG] [VALIDATOR ] VLO: All expected trading days are present. [2025-10-20T21:47:08.568675] [DEBUG] [VALIDATOR ] VLO: Primary data validation passed. [2025-10-20T21:47:08.569867] [INFO ] [MAIN ] VLO: Fetched primary data is valid. [2025-10-20T21:47:08.569932] [DEBUG] [LOCAL ] VLO: Saving primary data. [2025-10-20T21:47:08.571404] [DEBUG] [LOCAL ] VLO: Trimmed 76923 records to 52982 (keeping 200 days of data) [2025-10-20T21:47:08.903392] [DEBUG] [LOCAL ] VLO: Primary data saved (52982 records). [2025-10-20T21:47:08.903549] [INFO ] [MAIN ] VLO: Full history saved. [2025-10-20T21:47:08.903634] [INFO ] [MAIN ] Update Progress: 2231/2367 symbols completed. VLO took 35.3s. Estimated remaining: 17m 30s. Rate: 7.8 symbols/min [2025-10-20T21:47:09.068329] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-08. [2025-10-20T21:47:09.070924] [INFO ] [MAIN ] VLTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:09.114231] [DEBUG] [LOCAL ] VLTO: Loaded 46842 existing records from primary storage. [2025-10-20T21:47:09.114523] [INFO ] [MAIN ] VLTO: Loaded 46842 existing records. [2025-10-20T21:47:09.287765] [WARN ] [MAIN ] VLTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:09.287952] [INFO ] [MAIN ] VLTO: Wiping existing primary data before full refresh. [2025-10-20T21:47:09.288010] [INFO ] [LOCAL ] VLTO: Starting deletion of primary data. [2025-10-20T21:47:09.288234] [DEBUG] [LOCAL ] VLTO: Deleted primary data file. [2025-10-20T21:47:09.288292] [INFO ] [MAIN ] VLTO: Starting full primary refresh. [2025-10-20T21:47:09.645624] [INFO ] [REMOTE ] VLTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:09.682813] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-09 (1.43 seconds). [2025-10-20T21:47:09.740718] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-08 (0.67 seconds). [2025-10-20T21:47:09.851332] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-01. [2025-10-20T21:47:10.651426] [INFO ] [REMOTE ] VLN: Fetching primary data for 2025-10. [2025-10-20T21:47:11.451394] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-09. [2025-10-20T21:47:11.522435] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:47:12.056099] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-09 (0.60 seconds). [2025-10-20T21:47:12.132335] [INFO ] [REMOTE ] VLN: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T21:47:12.138288] [INFO ] [REMOTE ] VLN: Full fetch complete (43431 records). [2025-10-20T21:47:12.138411] [DEBUG] [VALIDATOR ] VLN: Validating primary data. [2025-10-20T21:47:12.256558] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-08. [2025-10-20T21:47:12.530938] [DEBUG] [VALIDATOR ] VLN: All expected trading days are present. [2025-10-20T21:47:12.531006] [DEBUG] [VALIDATOR ] VLN: Primary data validation passed. [2025-10-20T21:47:12.531942] [INFO ] [MAIN ] VLN: Fetched primary data is valid. [2025-10-20T21:47:12.532005] [DEBUG] [LOCAL ] VLN: Saving primary data. [2025-10-20T21:47:12.532976] [DEBUG] [LOCAL ] VLN: Trimmed 43431 records to 30887 (keeping 200 days of data) [2025-10-20T21:47:12.697211] [DEBUG] [LOCAL ] VLN: Primary data saved (30887 records). [2025-10-20T21:47:12.697332] [INFO ] [MAIN ] VLN: Full history saved. [2025-10-20T21:47:12.697403] [INFO ] [MAIN ] Update Progress: 2232/2367 symbols completed. VLN took 40.7s. Estimated remaining: 17m 22s. Rate: 7.8 symbols/min [2025-10-20T21:47:12.869350] [INFO ] [MAIN ] VMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:12.908863] [DEBUG] [LOCAL ] VMC: Loaded 43121 existing records from primary storage. [2025-10-20T21:47:12.909141] [INFO ] [MAIN ] VMC: Loaded 43121 existing records. [2025-10-20T21:47:13.053446] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-02. [2025-10-20T21:47:13.118204] [WARN ] [MAIN ] VMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:13.118429] [INFO ] [MAIN ] VMC: Wiping existing primary data before full refresh. [2025-10-20T21:47:13.118481] [INFO ] [LOCAL ] VMC: Starting deletion of primary data. [2025-10-20T21:47:13.118701] [DEBUG] [LOCAL ] VMC: Deleted primary data file. [2025-10-20T21:47:13.118760] [INFO ] [MAIN ] VMC: Starting full primary refresh. [2025-10-20T21:47:13.475605] [INFO ] [REMOTE ] VMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:13.793241] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T21:47:13.851331] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-09. [2025-10-20T21:47:14.651424] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-01. [2025-10-20T21:47:14.876724] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-02 (1.82 seconds). [2025-10-20T21:47:15.230573] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T21:47:15.451373] [INFO ] [REMOTE ] VLRS: Fetching primary data for 2025-10. [2025-10-20T21:47:16.251372] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-03. [2025-10-20T21:47:16.276874] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:47:16.711536] [INFO ] [REMOTE ] VLRS: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T21:47:16.716942] [INFO ] [REMOTE ] VLRS: Full fetch complete (58351 records). [2025-10-20T21:47:16.717036] [DEBUG] [VALIDATOR ] VLRS: Validating primary data. [2025-10-20T21:47:17.061713] [INFO ] [REMOTE ] VLT: Fetching primary data for 2025-10. [2025-10-20T21:47:17.097985] [DEBUG] [VALIDATOR ] VLRS: All expected trading days are present. [2025-10-20T21:47:17.098060] [DEBUG] [VALIDATOR ] VLRS: Primary data validation passed. [2025-10-20T21:47:17.099126] [INFO ] [MAIN ] VLRS: Fetched primary data is valid. [2025-10-20T21:47:17.099300] [DEBUG] [LOCAL ] VLRS: Saving primary data. [2025-10-20T21:47:17.100463] [DEBUG] [LOCAL ] VLRS: Trimmed 58351 records to 39472 (keeping 200 days of data) [2025-10-20T21:47:17.312163] [DEBUG] [LOCAL ] VLRS: Primary data saved (39472 records). [2025-10-20T21:47:17.312294] [INFO ] [MAIN ] VLRS: Full history saved. [2025-10-20T21:47:17.312372] [INFO ] [MAIN ] Update Progress: 2233/2367 symbols completed. VLRS took 30.1s. Estimated remaining: 17m 14s. Rate: 7.8 symbols/min [2025-10-20T21:47:17.478218] [INFO ] [MAIN ] VMI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:17.495675] [DEBUG] [LOCAL ] VMI: Loaded 15248 existing records from primary storage. [2025-10-20T21:47:17.495897] [INFO ] [MAIN ] VMI: Loaded 15248 existing records. [2025-10-20T21:47:17.678367] [WARN ] [MAIN ] VMI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:17.678707] [INFO ] [REMOTE ] VLT: Incremental fetch complete for 2025-10 (0.62 seconds). [2025-10-20T21:47:17.680506] [INFO ] [REMOTE ] VLT: Full fetch complete (7480 records). [2025-10-20T21:47:17.680734] [DEBUG] [VALIDATOR ] VLT: Validating primary data. [2025-10-20T21:47:17.681029] [INFO ] [MAIN ] VMI: Wiping existing primary data before full refresh. [2025-10-20T21:47:17.716994] [INFO ] [LOCAL ] VMI: Starting deletion of primary data. [2025-10-20T21:47:17.752973] [DEBUG] [LOCAL ] VMI: Deleted primary data file. [2025-10-20T21:47:17.771436] [INFO ] [MAIN ] VMI: Starting full primary refresh. [2025-10-20T21:47:17.860697] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-02. [2025-10-20T21:47:18.389112] [DEBUG] [VALIDATOR ] VLT: All expected trading days are present. [2025-10-20T21:47:18.389280] [DEBUG] [VALIDATOR ] VLT: Primary data validation passed. [2025-10-20T21:47:18.389996] [INFO ] [MAIN ] VLT: Fetched primary data is valid. [2025-10-20T21:47:18.395359] [DEBUG] [LOCAL ] VLT: Saving primary data. [2025-10-20T21:47:18.465176] [DEBUG] [LOCAL ] VLT: Trimmed 7480 records to 4389 (keeping 200 days of data) [2025-10-20T21:47:18.468750] [INFO ] [REMOTE ] VMI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:18.523613] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-03 (2.27 seconds). [2025-10-20T21:47:18.523754] [DEBUG] [LOCAL ] VLT: Primary data saved (4389 records). [2025-10-20T21:47:18.524022] [INFO ] [MAIN ] VLT: Full history saved. [2025-10-20T21:47:18.533722] [INFO ] [MAIN ] Update Progress: 2234/2367 symbols completed. VLT took 24.2s. Estimated remaining: 17m 6s. Rate: 7.8 symbols/min [2025-10-20T21:47:18.656437] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-01. [2025-10-20T21:47:18.707094] [INFO ] [MAIN ] VMO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:18.729087] [DEBUG] [LOCAL ] VMO: Loaded 21084 existing records from primary storage. [2025-10-20T21:47:18.729354] [INFO ] [MAIN ] VMO: Loaded 21084 existing records. [2025-10-20T21:47:18.907299] [WARN ] [MAIN ] VMO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:18.907538] [INFO ] [MAIN ] VMO: Wiping existing primary data before full refresh. [2025-10-20T21:47:18.907586] [INFO ] [LOCAL ] VMO: Starting deletion of primary data. [2025-10-20T21:47:18.907805] [DEBUG] [LOCAL ] VMO: Deleted primary data file. [2025-10-20T21:47:18.907863] [INFO ] [MAIN ] VMO: Starting full primary refresh. [2025-10-20T21:47:19.262083] [INFO ] [REMOTE ] VMO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:19.451378] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-01. [2025-10-20T21:47:19.597797] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T21:47:19.726241] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-02 (1.87 seconds). [2025-10-20T21:47:20.251412] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-02. [2025-10-20T21:47:20.455534] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T21:47:21.051371] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-02. [2025-10-20T21:47:21.246051] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-02 (0.99 seconds). [2025-10-20T21:47:21.851345] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-03. [2025-10-20T21:47:22.219419] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T21:47:22.651344] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-03. [2025-10-20T21:47:22.874486] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-03 (1.02 seconds). [2025-10-20T21:47:23.451364] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-04. [2025-10-20T21:47:23.751706] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T21:47:24.251423] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-03. [2025-10-20T21:47:24.528099] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T21:47:25.051413] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-05. [2025-10-20T21:47:25.851470] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-04. [2025-10-20T21:47:25.961163] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T21:47:25.984957] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-05 (0.93 seconds). [2025-10-20T21:47:26.651393] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-06. [2025-10-20T21:47:27.344452] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-04 (1.49 seconds). [2025-10-20T21:47:27.380876] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:47:27.451363] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-05. [2025-10-20T21:47:28.251355] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-07. [2025-10-20T21:47:29.051346] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-04. [2025-10-20T21:47:29.268287] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T21:47:29.290547] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T21:47:29.851416] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-08. [2025-10-20T21:47:30.218590] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T21:47:30.651340] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-05. [2025-10-20T21:47:30.882360] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:47:31.451348] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-04. [2025-10-20T21:47:31.656099] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T21:47:32.251365] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-06. [2025-10-20T21:47:33.051395] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-06. [2025-10-20T21:47:33.231720] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:47:33.380943] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-06 (1.13 seconds). [2025-10-20T21:47:33.851375] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-07. [2025-10-20T21:47:34.531013] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:47:34.592698] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-07 (0.74 seconds). [2025-10-20T21:47:34.651377] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-08. [2025-10-20T21:47:35.451411] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-09. [2025-10-20T21:47:35.621756] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T21:47:36.251394] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-09. [2025-10-20T21:47:36.437552] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-09 (0.99 seconds). [2025-10-20T21:47:37.051377] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-05. [2025-10-20T21:47:37.407607] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T21:47:37.851394] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-07. [2025-10-20T21:47:38.470437] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-05 (1.42 seconds). [2025-10-20T21:47:38.651473] [INFO ] [REMOTE ] VMO: Fetching primary data for 2025-10. [2025-10-20T21:47:39.451378] [INFO ] [REMOTE ] VMI: Fetching primary data for 2025-10. [2025-10-20T21:47:39.545623] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T21:47:39.633966] [INFO ] [REMOTE ] VMO: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T21:47:39.637160] [INFO ] [REMOTE ] VMO: Full fetch complete (30828 records). [2025-10-20T21:47:39.637234] [DEBUG] [VALIDATOR ] VMO: Validating primary data. [2025-10-20T21:47:40.011271] [DEBUG] [VALIDATOR ] VMO: All expected trading days are present. [2025-10-20T21:47:40.011353] [DEBUG] [VALIDATOR ] VMO: Primary data validation passed. [2025-10-20T21:47:40.011998] [INFO ] [MAIN ] VMO: Fetched primary data is valid. [2025-10-20T21:47:40.012057] [DEBUG] [LOCAL ] VMO: Saving primary data. [2025-10-20T21:47:40.012770] [DEBUG] [LOCAL ] VMO: Trimmed 30828 records to 20784 (keeping 200 days of data) [2025-10-20T21:47:40.134395] [DEBUG] [LOCAL ] VMO: Primary data saved (20784 records). [2025-10-20T21:47:40.134574] [INFO ] [MAIN ] VMO: Full history saved. [2025-10-20T21:47:40.134697] [INFO ] [MAIN ] Update Progress: 2235/2367 symbols completed. VMO took 21.6s. Estimated remaining: 16m 59s. Rate: 7.8 symbols/min [2025-10-20T21:47:40.256462] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-06. [2025-10-20T21:47:40.307180] [INFO ] [MAIN ] VNCE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:40.314449] [INFO ] [REMOTE ] VMI: Incremental fetch complete for 2025-10 (0.86 seconds). [2025-10-20T21:47:40.317081] [INFO ] [REMOTE ] VMI: Full fetch complete (21239 records). [2025-10-20T21:47:40.317154] [DEBUG] [VALIDATOR ] VMI: Validating primary data. [2025-10-20T21:47:40.426455] [DEBUG] [LOCAL ] VNCE: Loaded 9180 existing records from primary storage. [2025-10-20T21:47:40.426851] [INFO ] [MAIN ] VNCE: Loaded 9180 existing records. [2025-10-20T21:47:40.804402] [WARN ] [MAIN ] VNCE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:40.820483] [INFO ] [MAIN ] VNCE: Wiping existing primary data before full refresh. [2025-10-20T21:47:40.825708] [INFO ] [LOCAL ] VNCE: Starting deletion of primary data. [2025-10-20T21:47:40.877639] [DEBUG] [LOCAL ] VNCE: Deleted primary data file. [2025-10-20T21:47:40.887309] [INFO ] [MAIN ] VNCE: Starting full primary refresh. [2025-10-20T21:47:40.920371] [DEBUG] [VALIDATOR ] VMI: All expected trading days are present. [2025-10-20T21:47:40.945379] [DEBUG] [VALIDATOR ] VMI: Primary data validation passed. [2025-10-20T21:47:40.961020] [INFO ] [MAIN ] VMI: Fetched primary data is valid. [2025-10-20T21:47:40.961104] [DEBUG] [LOCAL ] VMI: Saving primary data. [2025-10-20T21:47:40.995222] [DEBUG] [LOCAL ] VMI: Trimmed 21239 records to 14963 (keeping 200 days of data) [2025-10-20T21:47:41.056055] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-08. [2025-10-20T21:47:41.302616] [DEBUG] [LOCAL ] VMI: Primary data saved (14963 records). [2025-10-20T21:47:41.320928] [INFO ] [MAIN ] VMI: Full history saved. [2025-10-20T21:47:41.340538] [INFO ] [MAIN ] Update Progress: 2236/2367 symbols completed. VMI took 24.0s. Estimated remaining: 16m 51s. Rate: 7.8 symbols/min [2025-10-20T21:47:41.394134] [INFO ] [REMOTE ] VNCE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:41.532887] [INFO ] [MAIN ] VNO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:47:41.576639] [DEBUG] [LOCAL ] VNO: Loaded 49695 existing records from primary storage. [2025-10-20T21:47:41.576995] [INFO ] [MAIN ] VNO: Loaded 49695 existing records. [2025-10-20T21:47:41.754792] [WARN ] [MAIN ] VNO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:47:41.755026] [INFO ] [MAIN ] VNO: Wiping existing primary data before full refresh. [2025-10-20T21:47:41.755148] [INFO ] [LOCAL ] VNO: Starting deletion of primary data. [2025-10-20T21:47:41.755418] [DEBUG] [LOCAL ] VNO: Deleted primary data file. [2025-10-20T21:47:41.755493] [INFO ] [MAIN ] VNO: Starting full primary refresh. [2025-10-20T21:47:41.854675] [INFO ] [REMOTE ] VNCE: Fetching primary data for 2025-01. [2025-10-20T21:47:42.052053] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T21:47:42.147373] [INFO ] [REMOTE ] VNO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:47:42.651334] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-07. [2025-10-20T21:47:42.752192] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T21:47:43.404018] [INFO ] [REMOTE ] VNCE: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:47:43.451326] [INFO ] [REMOTE ] VNCE: Fetching primary data for 2025-02. [2025-10-20T21:47:44.229445] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T21:47:44.246545] [INFO ] [REMOTE ] VNCE: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T21:47:44.251323] [INFO ] [REMOTE ] VNCE: Fetching primary data for 2025-03. [2025-10-20T21:47:44.361384] [DEBUG] [REMOTE ] VNCE: Error fetching primary data for 2025-03: 'API call returned an empty response.' [2025-10-20T21:47:44.361784] [WARN ] [REMOTE ] VNCE: No data returned for month 2025-03 [2025-10-20T21:47:44.361890] [FATAL] [REMOTE ] VNCE: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:47:44.362016] [FATAL] [MAIN ] VNCE: Failed to fetch primary data. [2025-10-20T21:47:44.362086] [WARN ] [MAIN ] VNCE: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:47:44.362161] [INFO ] [LOCAL ] VNCE: Starting deletion of primary data. [2025-10-20T21:47:44.362236] [DEBUG] [LOCAL ] VNCE: No primary data file found to delete. [2025-10-20T21:47:44.362275] [INFO ] [LOCAL ] VNCE: Starting deletion of derivative data. [2025-10-20T21:47:44.362730] [DEBUG] [LOCAL ] VNCE: Deleted derivative data file. [2025-10-20T21:47:44.363012] [INFO ] [MAIN ] Update Progress: 2237/2367 symbols completed. VNCE took 4.2s. Estimated remaining: 16m 43s. Rate: 7.8 symbols/min [2025-10-20T21:47:45.051436] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-01. [2025-10-20T21:47:45.851434] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-08. [2025-10-20T21:47:46.651407] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-09. [2025-10-20T21:47:46.684380] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:47:47.440154] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T21:47:47.451332] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-09. [2025-10-20T21:47:48.244408] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-09 (1.59 seconds). [2025-10-20T21:47:48.253908] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-02. [2025-10-20T21:47:49.049231] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-09 (1.60 seconds). [2025-10-20T21:47:49.057478] [INFO ] [REMOTE ] VLTO: Fetching primary data for 2025-10. [2025-10-20T21:47:49.841883] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T21:47:49.851328] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-03. [2025-10-20T21:47:50.640347] [INFO ] [REMOTE ] VLTO: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T21:47:50.647503] [INFO ] [REMOTE ] VLTO: Full fetch complete (67372 records). [2025-10-20T21:47:50.647683] [DEBUG] [VALIDATOR ] VLTO: Validating primary data. [2025-10-20T21:47:50.680613] [INFO ] [REMOTE ] VMC: Fetching primary data for 2025-10. [2025-10-20T21:47:51.046820] [DEBUG] [VALIDATOR ] VLTO: All expected trading days are present. [2025-10-20T21:47:51.046905] [DEBUG] [VALIDATOR ] VLTO: Primary data validation passed. [2025-10-20T21:47:51.047943] [INFO ] [MAIN ] VLTO: Fetched primary data is valid. [2025-10-20T21:47:51.048008] [DEBUG] [LOCAL ] VLTO: Saving primary data. [2025-10-20T21:47:51.049274] [DEBUG] [LOCAL ] VLTO: Trimmed 67372 records to 46138 (keeping 200 days of data) [2025-10-20T21:47:51.336896] [DEBUG] [LOCAL ] VLTO: Primary data saved (46138 records). [2025-10-20T21:47:51.337091] [INFO ] [MAIN ] VLTO: Full history saved. [2025-10-20T21:47:51.337232] [INFO ] [MAIN ] Update Progress: 2238/2367 symbols completed. VLTO took 42.4s. Estimated remaining: 16m 35s. Rate: 7.8 symbols/min [2025-10-20T21:47:51.352927] [INFO ] [REMOTE ] VMC: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T21:47:51.358795] [INFO ] [REMOTE ] VMC: Full fetch complete (62373 records). [2025-10-20T21:47:51.358897] [DEBUG] [VALIDATOR ] VMC: Validating primary data. [2025-10-20T21:47:51.753393] [DEBUG] [VALIDATOR ] VMC: All expected trading days are present. [2025-10-20T21:47:51.753580] [DEBUG] [VALIDATOR ] VMC: Primary data validation passed. [2025-10-20T21:47:51.754681] [INFO ] [MAIN ] VMC: Fetched primary data is valid. [2025-10-20T21:47:51.754763] [DEBUG] [LOCAL ] VMC: Saving primary data. [2025-10-20T21:47:51.781430] [DEBUG] [LOCAL ] VMC: Trimmed 62373 records to 42371 (keeping 200 days of data) [2025-10-20T21:47:51.782370] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-03 (1.93 seconds). [2025-10-20T21:47:51.793268] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-04. [2025-10-20T21:47:52.050372] [DEBUG] [LOCAL ] VMC: Primary data saved (42371 records). [2025-10-20T21:47:52.050502] [INFO ] [MAIN ] VMC: Full history saved. [2025-10-20T21:47:52.050677] [INFO ] [MAIN ] Update Progress: 2239/2367 symbols completed. VMC took 39.4s. Estimated remaining: 16m 27s. Rate: 7.8 symbols/min [2025-10-20T21:47:53.518758] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:47:53.529062] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-05. [2025-10-20T21:47:55.242837] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T21:47:55.253109] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-06. [2025-10-20T21:47:56.869004] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:47:56.879125] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-07. [2025-10-20T21:47:58.567306] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T21:47:58.579662] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-08. [2025-10-20T21:48:00.316941] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T21:48:00.328283] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-09. [2025-10-20T21:48:02.048599] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T21:48:02.060809] [INFO ] [REMOTE ] VNO: Fetching primary data for 2025-10. [2025-10-20T21:48:03.640508] [INFO ] [REMOTE ] VNO: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T21:48:03.647991] [INFO ] [REMOTE ] VNO: Full fetch complete (70567 records). [2025-10-20T21:48:03.648127] [DEBUG] [VALIDATOR ] VNO: Validating primary data. [2025-10-20T21:48:04.052584] [DEBUG] [VALIDATOR ] VNO: All expected trading days are present. [2025-10-20T21:48:04.052737] [DEBUG] [VALIDATOR ] VNO: Primary data validation passed. [2025-10-20T21:48:04.054538] [INFO ] [MAIN ] VNO: Fetched primary data is valid. [2025-10-20T21:48:04.054609] [DEBUG] [LOCAL ] VNO: Saving primary data. [2025-10-20T21:48:04.056562] [DEBUG] [LOCAL ] VNO: Trimmed 70567 records to 48936 (keeping 200 days of data) [2025-10-20T21:48:04.356747] [DEBUG] [LOCAL ] VNO: Primary data saved (48936 records). [2025-10-20T21:48:04.356946] [INFO ] [MAIN ] VNO: Full history saved. [2025-10-20T21:48:04.357047] [INFO ] [MAIN ] Update Progress: 2240/2367 symbols completed. VNO took 23.0s. Estimated remaining: 16m 20s. Rate: 7.8 symbols/min [2025-10-20T21:48:05.007002] [INFO ] [MAIN ] VNT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:05.069084] [INFO ] [MAIN ] VOYA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:05.108071] [INFO ] [MAIN ] VOC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:05.129792] [INFO ] [MAIN ] VOYG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:05.130239] [DEBUG] [LOCAL ] VOYG: No existing data file found in primary storage. [2025-10-20T21:48:05.130707] [WARN ] [MAIN ] VOYG: No existing data found, performing full refresh instead. [2025-10-20T21:48:05.130843] [INFO ] [MAIN ] VOYG: Wiping existing primary data before full refresh. [2025-10-20T21:48:05.130900] [INFO ] [LOCAL ] VOYG: Starting deletion of primary data. [2025-10-20T21:48:05.130945] [DEBUG] [LOCAL ] VOYG: No primary data file found to delete. [2025-10-20T21:48:05.130986] [INFO ] [MAIN ] VOYG: Starting full primary refresh. [2025-10-20T21:48:05.225333] [DEBUG] [LOCAL ] VOC: Loaded 9096 existing records from primary storage. [2025-10-20T21:48:05.241815] [INFO ] [MAIN ] VOC: Loaded 9096 existing records. [2025-10-20T21:48:05.493510] [DEBUG] [LOCAL ] VNT: Loaded 44708 existing records from primary storage. [2025-10-20T21:48:05.529924] [INFO ] [MAIN ] VNT: Loaded 44708 existing records. [2025-10-20T21:48:05.776918] [WARN ] [MAIN ] VOC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:05.764570] [DEBUG] [LOCAL ] VOYA: Loaded 42815 existing records from primary storage. [2025-10-20T21:48:05.798846] [INFO ] [MAIN ] VOC: Wiping existing primary data before full refresh. [2025-10-20T21:48:05.831161] [INFO ] [LOCAL ] VOC: Starting deletion of primary data. [2025-10-20T21:48:05.842103] [INFO ] [MAIN ] VOYA: Loaded 42815 existing records. [2025-10-20T21:48:05.857414] [DEBUG] [LOCAL ] VOC: Deleted primary data file. [2025-10-20T21:48:05.909806] [INFO ] [MAIN ] VOC: Starting full primary refresh. [2025-10-20T21:48:06.007072] [INFO ] [REMOTE ] VOYG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:06.078784] [WARN ] [MAIN ] VNT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:06.157161] [INFO ] [MAIN ] VNT: Wiping existing primary data before full refresh. [2025-10-20T21:48:06.168345] [INFO ] [LOCAL ] VNT: Starting deletion of primary data. [2025-10-20T21:48:06.157798] [INFO ] [REMOTE ] VOYG: Fetching primary data for 2025-01. [2025-10-20T21:48:06.194543] [DEBUG] [LOCAL ] VNT: Deleted primary data file. [2025-10-20T21:48:06.205478] [INFO ] [MAIN ] VNT: Starting full primary refresh. [2025-10-20T21:48:06.511594] [WARN ] [MAIN ] VOYA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:06.521938] [DEBUG] [REMOTE ] VOYG: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:48:06.558885] [INFO ] [MAIN ] VOYA: Wiping existing primary data before full refresh. [2025-10-20T21:48:06.567068] [WARN ] [REMOTE ] VOYG: No data returned for month 2025-01 [2025-10-20T21:48:06.592045] [FATAL] [REMOTE ] VOYG: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:48:06.592254] [INFO ] [LOCAL ] VOYA: Starting deletion of primary data. [2025-10-20T21:48:06.592206] [FATAL] [MAIN ] VOYG: Failed to fetch primary data. [2025-10-20T21:48:06.624440] [DEBUG] [LOCAL ] VOYA: Deleted primary data file. [2025-10-20T21:48:06.644236] [INFO ] [MAIN ] VOYA: Starting full primary refresh. [2025-10-20T21:48:06.638521] [WARN ] [MAIN ] VOYG: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:48:06.750448] [INFO ] [LOCAL ] VOYG: Starting deletion of primary data. [2025-10-20T21:48:06.930629] [DEBUG] [LOCAL ] VOYG: No primary data file found to delete. [2025-10-20T21:48:06.930547] [INFO ] [REMOTE ] VOC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:06.966063] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-01. [2025-10-20T21:48:06.966132] [INFO ] [LOCAL ] VOYG: Starting deletion of derivative data. [2025-10-20T21:48:07.010705] [DEBUG] [LOCAL ] VOYG: No derivative data file found to delete. [2025-10-20T21:48:07.037826] [INFO ] [MAIN ] Update Progress: 2241/2367 symbols completed. VOYG took 2.7s. Estimated remaining: 16m 12s. Rate: 7.8 symbols/min [2025-10-20T21:48:07.366382] [INFO ] [REMOTE ] VNT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:07.395086] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-01. [2025-10-20T21:48:07.496007] [INFO ] [MAIN ] VPG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:07.536000] [INFO ] [REMOTE ] VOYA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:07.536727] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-01. [2025-10-20T21:48:07.555747] [DEBUG] [LOCAL ] VPG: Loaded 17444 existing records from primary storage. [2025-10-20T21:48:07.556006] [INFO ] [MAIN ] VPG: Loaded 17444 existing records. [2025-10-20T21:48:07.772810] [WARN ] [MAIN ] VPG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:07.773018] [INFO ] [MAIN ] VPG: Wiping existing primary data before full refresh. [2025-10-20T21:48:07.773064] [INFO ] [LOCAL ] VPG: Starting deletion of primary data. [2025-10-20T21:48:07.773229] [DEBUG] [LOCAL ] VPG: Deleted primary data file. [2025-10-20T21:48:07.773291] [INFO ] [MAIN ] VPG: Starting full primary refresh. [2025-10-20T21:48:07.882744] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-01 (0.92 seconds). [2025-10-20T21:48:07.896425] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-02. [2025-10-20T21:48:08.137852] [INFO ] [REMOTE ] VPG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:08.138188] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-01. [2025-10-20T21:48:08.749859] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-02 (0.85 seconds). [2025-10-20T21:48:08.752480] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-03. [2025-10-20T21:48:09.027551] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T21:48:09.038793] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-02. [2025-10-20T21:48:09.054716] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T21:48:09.063426] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-02. [2025-10-20T21:48:09.145085] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:48:09.154122] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-02. [2025-10-20T21:48:09.513370] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-03 (0.76 seconds). [2025-10-20T21:48:09.515580] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-04. [2025-10-20T21:48:09.961716] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-02 (0.92 seconds). [2025-10-20T21:48:09.964942] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-03. [2025-10-20T21:48:10.569672] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-04 (1.05 seconds). [2025-10-20T21:48:10.572561] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-05. [2025-10-20T21:48:10.601043] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T21:48:10.609721] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-03. [2025-10-20T21:48:10.634601] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T21:48:10.643618] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-03. [2025-10-20T21:48:10.815493] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-03 (0.85 seconds). [2025-10-20T21:48:10.818934] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-04. [2025-10-20T21:48:11.060755] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-05 (0.49 seconds). [2025-10-20T21:48:11.063077] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-06. [2025-10-20T21:48:11.840273] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-06 (0.78 seconds). [2025-10-20T21:48:11.842795] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-07. [2025-10-20T21:48:12.012281] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T21:48:12.017705] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-05. [2025-10-20T21:48:12.068141] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-03 (1.46 seconds). [2025-10-20T21:48:12.224058] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T21:48:12.251364] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-04. [2025-10-20T21:48:12.678959] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T21:48:13.051400] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-08. [2025-10-20T21:48:13.094720] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T21:48:13.712199] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-08 (0.66 seconds). [2025-10-20T21:48:13.851342] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-09. [2025-10-20T21:48:13.879138] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:48:14.651354] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-05. [2025-10-20T21:48:14.670052] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T21:48:15.451399] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-06. [2025-10-20T21:48:16.183290] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:48:16.251352] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-06. [2025-10-20T21:48:16.529996] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T21:48:17.051356] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-04. [2025-10-20T21:48:17.766333] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T21:48:17.851406] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-07. [2025-10-20T21:48:18.651684] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-07. [2025-10-20T21:48:18.651589] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-04 (1.60 seconds). [2025-10-20T21:48:19.452769] [INFO ] [REMOTE ] VOC: Fetching primary data for 2025-10. [2025-10-20T21:48:19.458913] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-07 (1.61 seconds). [2025-10-20T21:48:19.622824] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-07 (0.97 seconds). [2025-10-20T21:48:20.178926] [INFO ] [REMOTE ] VOC: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:48:20.180546] [INFO ] [REMOTE ] VOC: Full fetch complete (13140 records). [2025-10-20T21:48:20.180615] [DEBUG] [VALIDATOR ] VOC: Validating primary data. [2025-10-20T21:48:20.256527] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-08. [2025-10-20T21:48:20.540090] [DEBUG] [VALIDATOR ] VOC: All expected trading days are present. [2025-10-20T21:48:20.540167] [DEBUG] [VALIDATOR ] VOC: Primary data validation passed. [2025-10-20T21:48:20.540449] [INFO ] [MAIN ] VOC: Fetched primary data is valid. [2025-10-20T21:48:20.540487] [DEBUG] [LOCAL ] VOC: Saving primary data. [2025-10-20T21:48:20.540967] [DEBUG] [LOCAL ] VOC: Trimmed 13140 records to 9011 (keeping 200 days of data) [2025-10-20T21:48:20.593183] [DEBUG] [LOCAL ] VOC: Primary data saved (9011 records). [2025-10-20T21:48:20.593342] [INFO ] [MAIN ] VOC: Full history saved. [2025-10-20T21:48:20.593430] [INFO ] [MAIN ] Update Progress: 2242/2367 symbols completed. VOC took 16.2s. Estimated remaining: 16m 4s. Rate: 7.8 symbols/min [2025-10-20T21:48:20.767952] [INFO ] [MAIN ] VPV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:20.775950] [DEBUG] [LOCAL ] VPV: Loaded 5502 existing records from primary storage. [2025-10-20T21:48:20.776154] [INFO ] [MAIN ] VPV: Loaded 5502 existing records. [2025-10-20T21:48:20.954952] [WARN ] [MAIN ] VPV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:20.955197] [INFO ] [MAIN ] VPV: Wiping existing primary data before full refresh. [2025-10-20T21:48:20.955247] [INFO ] [LOCAL ] VPV: Starting deletion of primary data. [2025-10-20T21:48:20.955415] [DEBUG] [LOCAL ] VPV: Deleted primary data file. [2025-10-20T21:48:20.955486] [INFO ] [MAIN ] VPV: Starting full primary refresh. [2025-10-20T21:48:21.056788] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-05. [2025-10-20T21:48:21.329597] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:48:21.335514] [INFO ] [REMOTE ] VPV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:21.851386] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-01. [2025-10-20T21:48:22.651429] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-09. [2025-10-20T21:48:22.685966] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:48:22.725735] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-01 (0.87 seconds). [2025-10-20T21:48:23.451400] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-02. [2025-10-20T21:48:23.598731] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-09 (0.95 seconds). [2025-10-20T21:48:24.211191] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-02 (0.76 seconds). [2025-10-20T21:48:24.251340] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-03. [2025-10-20T21:48:24.872329] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-03 (0.62 seconds). [2025-10-20T21:48:25.051371] [INFO ] [REMOTE ] VPG: Fetching primary data for 2025-10. [2025-10-20T21:48:25.851357] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-08. [2025-10-20T21:48:25.865930] [INFO ] [REMOTE ] VPG: Incremental fetch complete for 2025-10 (0.81 seconds). [2025-10-20T21:48:25.868559] [INFO ] [REMOTE ] VPG: Full fetch complete (23020 records). [2025-10-20T21:48:25.868635] [DEBUG] [VALIDATOR ] VPG: Validating primary data. [2025-10-20T21:48:26.236860] [DEBUG] [VALIDATOR ] VPG: All expected trading days are present. [2025-10-20T21:48:26.236948] [DEBUG] [VALIDATOR ] VPG: Primary data validation passed. [2025-10-20T21:48:26.237416] [INFO ] [MAIN ] VPG: Fetched primary data is valid. [2025-10-20T21:48:26.237455] [DEBUG] [LOCAL ] VPG: Saving primary data. [2025-10-20T21:48:26.238233] [DEBUG] [LOCAL ] VPG: Trimmed 23020 records to 17102 (keeping 200 days of data) [2025-10-20T21:48:26.336833] [DEBUG] [LOCAL ] VPG: Primary data saved (17102 records). [2025-10-20T21:48:26.336984] [INFO ] [MAIN ] VPG: Full history saved. [2025-10-20T21:48:26.337061] [INFO ] [MAIN ] Update Progress: 2243/2367 symbols completed. VPG took 19.3s. Estimated remaining: 15m 57s. Rate: 7.8 symbols/min [2025-10-20T21:48:26.507209] [INFO ] [MAIN ] VRE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:26.541259] [DEBUG] [LOCAL ] VRE: Loaded 35807 existing records from primary storage. [2025-10-20T21:48:26.541532] [INFO ] [MAIN ] VRE: Loaded 35807 existing records. [2025-10-20T21:48:26.656726] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-04. [2025-10-20T21:48:26.725662] [WARN ] [MAIN ] VRE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:26.725874] [INFO ] [MAIN ] VRE: Wiping existing primary data before full refresh. [2025-10-20T21:48:26.725920] [INFO ] [LOCAL ] VRE: Starting deletion of primary data. [2025-10-20T21:48:26.726104] [DEBUG] [LOCAL ] VRE: Deleted primary data file. [2025-10-20T21:48:26.726143] [INFO ] [MAIN ] VRE: Starting full primary refresh. [2025-10-20T21:48:27.085288] [INFO ] [REMOTE ] VRE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:27.455452] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-01. [2025-10-20T21:48:27.458071] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-04 (0.80 seconds). [2025-10-20T21:48:27.493577] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:48:28.251351] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-05. [2025-10-20T21:48:28.718523] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T21:48:28.853770] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-05 (0.60 seconds). [2025-10-20T21:48:29.051356] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-02. [2025-10-20T21:48:29.851407] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-09. [2025-10-20T21:48:30.194116] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T21:48:30.651375] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-06. [2025-10-20T21:48:31.060172] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-09 (1.21 seconds). [2025-10-20T21:48:31.280480] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-06 (0.63 seconds). [2025-10-20T21:48:31.451424] [INFO ] [REMOTE ] VNT: Fetching primary data for 2025-10. [2025-10-20T21:48:32.251477] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-06. [2025-10-20T21:48:32.885125] [INFO ] [REMOTE ] VNT: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T21:48:32.892341] [INFO ] [REMOTE ] VNT: Full fetch complete (64337 records). [2025-10-20T21:48:32.892516] [DEBUG] [VALIDATOR ] VNT: Validating primary data. [2025-10-20T21:48:33.064373] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-03. [2025-10-20T21:48:33.276455] [DEBUG] [VALIDATOR ] VNT: All expected trading days are present. [2025-10-20T21:48:33.276535] [DEBUG] [VALIDATOR ] VNT: Primary data validation passed. [2025-10-20T21:48:33.277568] [INFO ] [MAIN ] VNT: Fetched primary data is valid. [2025-10-20T21:48:33.277630] [DEBUG] [LOCAL ] VNT: Saving primary data. [2025-10-20T21:48:33.278926] [DEBUG] [LOCAL ] VNT: Trimmed 64337 records to 43986 (keeping 200 days of data) [2025-10-20T21:48:33.540686] [DEBUG] [LOCAL ] VNT: Primary data saved (43986 records). [2025-10-20T21:48:33.540965] [INFO ] [MAIN ] VNT: Full history saved. [2025-10-20T21:48:33.541135] [INFO ] [MAIN ] Update Progress: 2244/2367 symbols completed. VNT took 29.2s. Estimated remaining: 15m 49s. Rate: 7.8 symbols/min [2025-10-20T21:48:33.715527] [INFO ] [MAIN ] VRT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:33.740956] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-06 (1.49 seconds). [2025-10-20T21:48:33.797361] [DEBUG] [LOCAL ] VRT: Loaded 54533 existing records from primary storage. [2025-10-20T21:48:33.797785] [INFO ] [MAIN ] VRT: Loaded 54533 existing records. [2025-10-20T21:48:33.859884] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-07. [2025-10-20T21:48:33.982988] [WARN ] [MAIN ] VRT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:33.983241] [INFO ] [MAIN ] VRT: Wiping existing primary data before full refresh. [2025-10-20T21:48:33.983294] [INFO ] [LOCAL ] VRT: Starting deletion of primary data. [2025-10-20T21:48:33.983732] [DEBUG] [LOCAL ] VRT: Deleted primary data file. [2025-10-20T21:48:33.983788] [INFO ] [MAIN ] VRT: Starting full primary refresh. [2025-10-20T21:48:34.333750] [INFO ] [REMOTE ] VRT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:34.493604] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T21:48:34.506603] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-07 (0.65 seconds). [2025-10-20T21:48:34.651354] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-07. [2025-10-20T21:48:35.451418] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-08. [2025-10-20T21:48:36.042815] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T21:48:36.206169] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T21:48:36.251400] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-09. [2025-10-20T21:48:36.938605] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T21:48:37.051452] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-01. [2025-10-20T21:48:37.851395] [INFO ] [REMOTE ] VPV: Fetching primary data for 2025-10. [2025-10-20T21:48:38.446326] [INFO ] [REMOTE ] VPV: Incremental fetch complete for 2025-10 (0.59 seconds). [2025-10-20T21:48:38.447638] [INFO ] [REMOTE ] VPV: Full fetch complete (8919 records). [2025-10-20T21:48:38.447728] [DEBUG] [VALIDATOR ] VPV: Validating primary data. [2025-10-20T21:48:38.656551] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-04. [2025-10-20T21:48:38.801237] [DEBUG] [VALIDATOR ] VPV: All expected trading days are present. [2025-10-20T21:48:38.801338] [DEBUG] [VALIDATOR ] VPV: Primary data validation passed. [2025-10-20T21:48:38.801545] [INFO ] [MAIN ] VPV: Fetched primary data is valid. [2025-10-20T21:48:38.801582] [DEBUG] [LOCAL ] VPV: Saving primary data. [2025-10-20T21:48:38.802012] [DEBUG] [LOCAL ] VPV: Trimmed 8919 records to 5429 (keeping 200 days of data) [2025-10-20T21:48:38.834453] [DEBUG] [LOCAL ] VPV: Primary data saved (5429 records). [2025-10-20T21:48:38.834576] [INFO ] [MAIN ] VPV: Full history saved. [2025-10-20T21:48:38.834676] [INFO ] [MAIN ] Update Progress: 2245/2367 symbols completed. VPV took 18.2s. Estimated remaining: 15m 41s. Rate: 7.8 symbols/min [2025-10-20T21:48:39.023683] [INFO ] [MAIN ] VRTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:39.036177] [DEBUG] [LOCAL ] VRTS: Loaded 9481 existing records from primary storage. [2025-10-20T21:48:39.036425] [INFO ] [MAIN ] VRTS: Loaded 9481 existing records. [2025-10-20T21:48:39.238332] [WARN ] [MAIN ] VRTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:39.238708] [INFO ] [MAIN ] VRTS: Wiping existing primary data before full refresh. [2025-10-20T21:48:39.238787] [INFO ] [LOCAL ] VRTS: Starting deletion of primary data. [2025-10-20T21:48:39.238970] [DEBUG] [LOCAL ] VRTS: Deleted primary data file. [2025-10-20T21:48:39.239017] [INFO ] [MAIN ] VRTS: Starting full primary refresh. [2025-10-20T21:48:39.465969] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-08. [2025-10-20T21:48:39.627091] [INFO ] [REMOTE ] VRTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:39.660055] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-01 (2.61 seconds). [2025-10-20T21:48:40.158785] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T21:48:40.251368] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-01. [2025-10-20T21:48:40.738238] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-01 (0.49 seconds). [2025-10-20T21:48:40.770023] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:48:41.051367] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-09. [2025-10-20T21:48:41.851380] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-05. [2025-10-20T21:48:42.605182] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:48:42.651374] [INFO ] [REMOTE ] VOYA: Fetching primary data for 2025-10. [2025-10-20T21:48:43.238117] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T21:48:43.451379] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-02. [2025-10-20T21:48:44.094034] [INFO ] [REMOTE ] VOYA: Incremental fetch complete for 2025-10 (1.44 seconds). [2025-10-20T21:48:44.101180] [INFO ] [REMOTE ] VOYA: Full fetch complete (61745 records). [2025-10-20T21:48:44.101411] [DEBUG] [VALIDATOR ] VOYA: Validating primary data. [2025-10-20T21:48:44.256586] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-06. [2025-10-20T21:48:44.512732] [DEBUG] [VALIDATOR ] VOYA: All expected trading days are present. [2025-10-20T21:48:44.512822] [DEBUG] [VALIDATOR ] VOYA: Primary data validation passed. [2025-10-20T21:48:44.513804] [INFO ] [MAIN ] VOYA: Fetched primary data is valid. [2025-10-20T21:48:44.513867] [DEBUG] [LOCAL ] VOYA: Saving primary data. [2025-10-20T21:48:44.515121] [DEBUG] [LOCAL ] VOYA: Trimmed 61745 records to 42073 (keeping 200 days of data) [2025-10-20T21:48:44.767621] [DEBUG] [LOCAL ] VOYA: Primary data saved (42073 records). [2025-10-20T21:48:44.767776] [INFO ] [MAIN ] VOYA: Full history saved. [2025-10-20T21:48:44.767855] [INFO ] [MAIN ] Update Progress: 2246/2367 symbols completed. VOYA took 40.4s. Estimated remaining: 15m 33s. Rate: 7.8 symbols/min [2025-10-20T21:48:44.937170] [INFO ] [MAIN ] VSCO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:44.986959] [DEBUG] [LOCAL ] VSCO: Loaded 53511 existing records from primary storage. [2025-10-20T21:48:44.987292] [INFO ] [MAIN ] VSCO: Loaded 53511 existing records. [2025-10-20T21:48:45.056726] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-02. [2025-10-20T21:48:45.171532] [WARN ] [MAIN ] VSCO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:45.171870] [INFO ] [MAIN ] VSCO: Wiping existing primary data before full refresh. [2025-10-20T21:48:45.171932] [INFO ] [LOCAL ] VSCO: Starting deletion of primary data. [2025-10-20T21:48:45.172209] [DEBUG] [LOCAL ] VSCO: Deleted primary data file. [2025-10-20T21:48:45.172276] [INFO ] [MAIN ] VSCO: Starting full primary refresh. [2025-10-20T21:48:45.533100] [INFO ] [REMOTE ] VSCO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:45.708442] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T21:48:45.745796] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-02 (0.69 seconds). [2025-10-20T21:48:45.851333] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-03. [2025-10-20T21:48:45.985273] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-02 (2.53 seconds). [2025-10-20T21:48:46.294225] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-03 (0.44 seconds). [2025-10-20T21:48:46.651432] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-04. [2025-10-20T21:48:47.394732] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-04 (0.74 seconds). [2025-10-20T21:48:47.451395] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-05. [2025-10-20T21:48:48.160837] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-05 (0.71 seconds). [2025-10-20T21:48:48.251403] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-01. [2025-10-20T21:48:49.051487] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-03. [2025-10-20T21:48:49.851372] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-07. [2025-10-20T21:48:49.937875] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T21:48:50.651385] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-06. [2025-10-20T21:48:51.359082] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T21:48:51.451397] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-02. [2025-10-20T21:48:51.491207] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-03 (2.44 seconds). [2025-10-20T21:48:51.552083] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T21:48:52.251407] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-07. [2025-10-20T21:48:53.051423] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-04. [2025-10-20T21:48:53.065818] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T21:48:53.103098] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T21:48:53.851404] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-08. [2025-10-20T21:48:54.619555] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-08 (0.77 seconds). [2025-10-20T21:48:54.651347] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-09. [2025-10-20T21:48:55.376263] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-09 (0.72 seconds). [2025-10-20T21:48:55.451395] [INFO ] [REMOTE ] VRTS: Fetching primary data for 2025-10. [2025-10-20T21:48:55.522025] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-04 (2.47 seconds). [2025-10-20T21:48:56.193045] [INFO ] [REMOTE ] VRTS: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:48:56.195034] [INFO ] [REMOTE ] VRTS: Full fetch complete (11711 records). [2025-10-20T21:48:56.195122] [DEBUG] [VALIDATOR ] VRTS: Validating primary data. [2025-10-20T21:48:56.256635] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-03. [2025-10-20T21:48:56.579932] [DEBUG] [VALIDATOR ] VRTS: All expected trading days are present. [2025-10-20T21:48:56.580015] [DEBUG] [VALIDATOR ] VRTS: Primary data validation passed. [2025-10-20T21:48:56.580309] [INFO ] [MAIN ] VRTS: Fetched primary data is valid. [2025-10-20T21:48:56.580346] [DEBUG] [LOCAL ] VRTS: Saving primary data. [2025-10-20T21:48:56.580817] [DEBUG] [LOCAL ] VRTS: Trimmed 11711 records to 9391 (keeping 200 days of data) [2025-10-20T21:48:56.638227] [DEBUG] [LOCAL ] VRTS: Primary data saved (9391 records). [2025-10-20T21:48:56.638359] [INFO ] [MAIN ] VRTS: Full history saved. [2025-10-20T21:48:56.638434] [INFO ] [MAIN ] Update Progress: 2247/2367 symbols completed. VRTS took 17.8s. Estimated remaining: 15m 26s. Rate: 7.8 symbols/min [2025-10-20T21:48:56.808823] [INFO ] [MAIN ] VSH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:48:56.853181] [DEBUG] [LOCAL ] VSH: Loaded 48966 existing records from primary storage. [2025-10-20T21:48:56.853512] [INFO ] [MAIN ] VSH: Loaded 48966 existing records. [2025-10-20T21:48:57.030922] [WARN ] [MAIN ] VSH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:48:57.031167] [INFO ] [MAIN ] VSH: Wiping existing primary data before full refresh. [2025-10-20T21:48:57.031215] [INFO ] [LOCAL ] VSH: Starting deletion of primary data. [2025-10-20T21:48:57.031425] [DEBUG] [LOCAL ] VSH: Deleted primary data file. [2025-10-20T21:48:57.031479] [INFO ] [MAIN ] VSH: Starting full primary refresh. [2025-10-20T21:48:57.062221] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-05. [2025-10-20T21:48:57.382180] [INFO ] [REMOTE ] VSH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:48:57.851382] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-01. [2025-10-20T21:48:58.307354] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-03 (2.05 seconds). [2025-10-20T21:48:58.651396] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-04. [2025-10-20T21:48:59.451403] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-08. [2025-10-20T21:48:59.695973] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-01 (1.84 seconds). [2025-10-20T21:48:59.739539] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-05 (2.68 seconds). [2025-10-20T21:49:00.251379] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-02. [2025-10-20T21:49:00.287960] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:49:00.744430] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T21:49:01.051373] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-06. [2025-10-20T21:49:01.673291] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T21:49:01.851360] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-05. [2025-10-20T21:49:02.651370] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-03. [2025-10-20T21:49:03.411794] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-06 (2.36 seconds). [2025-10-20T21:49:03.451332] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-07. [2025-10-20T21:49:03.708212] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T21:49:03.967356] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T21:49:04.251371] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-06. [2025-10-20T21:49:05.051384] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-09. [2025-10-20T21:49:05.851419] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-04. [2025-10-20T21:49:06.033918] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-07 (2.58 seconds). [2025-10-20T21:49:06.197926] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-06 (1.95 seconds). [2025-10-20T21:49:06.651444] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-07. [2025-10-20T21:49:06.701246] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T21:49:07.204069] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T21:49:07.451444] [INFO ] [REMOTE ] VRE: Fetching primary data for 2025-10. [2025-10-20T21:49:08.251372] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-05. [2025-10-20T21:49:08.526758] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-07 (1.88 seconds). [2025-10-20T21:49:08.736947] [INFO ] [REMOTE ] VRE: Incremental fetch complete for 2025-10 (1.29 seconds). [2025-10-20T21:49:08.742422] [INFO ] [REMOTE ] VRE: Full fetch complete (48628 records). [2025-10-20T21:49:08.742550] [DEBUG] [VALIDATOR ] VRE: Validating primary data. [2025-10-20T21:49:09.056521] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-08. [2025-10-20T21:49:09.124080] [DEBUG] [VALIDATOR ] VRE: All expected trading days are present. [2025-10-20T21:49:09.124167] [DEBUG] [VALIDATOR ] VRE: Primary data validation passed. [2025-10-20T21:49:09.125069] [INFO ] [MAIN ] VRE: Fetched primary data is valid. [2025-10-20T21:49:09.125129] [DEBUG] [LOCAL ] VRE: Saving primary data. [2025-10-20T21:49:09.126188] [DEBUG] [LOCAL ] VRE: Trimmed 48628 records to 35362 (keeping 200 days of data) [2025-10-20T21:49:09.337698] [DEBUG] [LOCAL ] VRE: Primary data saved (35362 records). [2025-10-20T21:49:09.337844] [INFO ] [MAIN ] VRE: Full history saved. [2025-10-20T21:49:09.337942] [INFO ] [MAIN ] Update Progress: 2248/2367 symbols completed. VRE took 43.0s. Estimated remaining: 15m 18s. Rate: 7.8 symbols/min [2025-10-20T21:49:09.502875] [INFO ] [MAIN ] VST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:09.553096] [DEBUG] [LOCAL ] VST: Loaded 54489 existing records from primary storage. [2025-10-20T21:49:09.553394] [INFO ] [MAIN ] VST: Loaded 54489 existing records. [2025-10-20T21:49:09.733012] [WARN ] [MAIN ] VST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:09.733337] [INFO ] [MAIN ] VST: Wiping existing primary data before full refresh. [2025-10-20T21:49:09.733403] [INFO ] [LOCAL ] VST: Starting deletion of primary data. [2025-10-20T21:49:09.733731] [DEBUG] [LOCAL ] VST: Deleted primary data file. [2025-10-20T21:49:09.733807] [INFO ] [MAIN ] VST: Starting full primary refresh. [2025-10-20T21:49:09.856501] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-08. [2025-10-20T21:49:10.112907] [INFO ] [REMOTE ] VST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:10.140127] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T21:49:10.651438] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-06. [2025-10-20T21:49:11.451352] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-01. [2025-10-20T21:49:11.676107] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-08 (2.62 seconds). [2025-10-20T21:49:11.743680] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T21:49:12.222056] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-06 (1.57 seconds). [2025-10-20T21:49:12.251334] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-07. [2025-10-20T21:49:13.051351] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-09. [2025-10-20T21:49:13.851438] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-09. [2025-10-20T21:49:13.888901] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-07 (1.64 seconds). [2025-10-20T21:49:13.981573] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-01 (2.53 seconds). [2025-10-20T21:49:14.651358] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-02. [2025-10-20T21:49:14.867569] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T21:49:15.451360] [INFO ] [REMOTE ] VSCO: Fetching primary data for 2025-10. [2025-10-20T21:49:16.236272] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-09 (2.38 seconds). [2025-10-20T21:49:16.251333] [INFO ] [REMOTE ] VRT: Fetching primary data for 2025-10. [2025-10-20T21:49:16.743424] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-02 (2.09 seconds). [2025-10-20T21:49:17.051404] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-08. [2025-10-20T21:49:17.196439] [INFO ] [REMOTE ] VSCO: Incremental fetch complete for 2025-10 (1.75 seconds). [2025-10-20T21:49:17.204390] [INFO ] [REMOTE ] VSCO: Full fetch complete (76435 records). [2025-10-20T21:49:17.204590] [DEBUG] [VALIDATOR ] VSCO: Validating primary data. [2025-10-20T21:49:17.593154] [DEBUG] [VALIDATOR ] VSCO: All expected trading days are present. [2025-10-20T21:49:17.593272] [DEBUG] [VALIDATOR ] VSCO: Primary data validation passed. [2025-10-20T21:49:17.594516] [INFO ] [MAIN ] VSCO: Fetched primary data is valid. [2025-10-20T21:49:17.594579] [DEBUG] [LOCAL ] VSCO: Saving primary data. [2025-10-20T21:49:17.596178] [DEBUG] [LOCAL ] VSCO: Trimmed 76435 records to 52717 (keeping 200 days of data) [2025-10-20T21:49:17.855196] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-03. [2025-10-20T21:49:17.902813] [DEBUG] [LOCAL ] VSCO: Primary data saved (52717 records). [2025-10-20T21:49:17.902968] [INFO ] [MAIN ] VSCO: Full history saved. [2025-10-20T21:49:17.903070] [INFO ] [MAIN ] Update Progress: 2249/2367 symbols completed. VSCO took 33.1s. Estimated remaining: 15m 10s. Rate: 7.8 symbols/min [2025-10-20T21:49:18.082509] [INFO ] [MAIN ] VSTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:18.124230] [DEBUG] [LOCAL ] VSTS: Loaded 49618 existing records from primary storage. [2025-10-20T21:49:18.124536] [INFO ] [MAIN ] VSTS: Loaded 49618 existing records. [2025-10-20T21:49:18.305142] [WARN ] [MAIN ] VSTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:18.305348] [INFO ] [MAIN ] VSTS: Wiping existing primary data before full refresh. [2025-10-20T21:49:18.305394] [INFO ] [LOCAL ] VSTS: Starting deletion of primary data. [2025-10-20T21:49:18.305765] [DEBUG] [LOCAL ] VSTS: Deleted primary data file. [2025-10-20T21:49:18.305820] [INFO ] [MAIN ] VSTS: Starting full primary refresh. [2025-10-20T21:49:18.665412] [INFO ] [REMOTE ] VSTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:18.665851] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-01. [2025-10-20T21:49:18.688239] [INFO ] [REMOTE ] VRT: Incremental fetch complete for 2025-10 (2.44 seconds). [2025-10-20T21:49:18.695977] [INFO ] [REMOTE ] VRT: Full fetch complete (77932 records). [2025-10-20T21:49:18.696099] [DEBUG] [VALIDATOR ] VRT: Validating primary data. [2025-10-20T21:49:19.011811] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T21:49:19.152591] [DEBUG] [VALIDATOR ] VRT: All expected trading days are present. [2025-10-20T21:49:19.152696] [DEBUG] [VALIDATOR ] VRT: Primary data validation passed. [2025-10-20T21:49:19.153790] [INFO ] [MAIN ] VRT: Fetched primary data is valid. [2025-10-20T21:49:19.153851] [DEBUG] [LOCAL ] VRT: Saving primary data. [2025-10-20T21:49:19.155223] [DEBUG] [LOCAL ] VRT: Trimmed 77932 records to 53753 (keeping 200 days of data) [2025-10-20T21:49:19.463464] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-09. [2025-10-20T21:49:19.489251] [DEBUG] [LOCAL ] VRT: Primary data saved (53753 records). [2025-10-20T21:49:19.489340] [INFO ] [MAIN ] VRT: Full history saved. [2025-10-20T21:49:19.489412] [INFO ] [MAIN ] Update Progress: 2250/2367 symbols completed. VRT took 45.9s. Estimated remaining: 15m 2s. Rate: 7.8 symbols/min [2025-10-20T21:49:19.655107] [INFO ] [MAIN ] VTEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:19.693291] [DEBUG] [LOCAL ] VTEX: Loaded 43610 existing records from primary storage. [2025-10-20T21:49:19.693591] [INFO ] [MAIN ] VTEX: Loaded 43610 existing records. [2025-10-20T21:49:19.870572] [WARN ] [MAIN ] VTEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:19.871010] [INFO ] [MAIN ] VTEX: Wiping existing primary data before full refresh. [2025-10-20T21:49:19.871068] [INFO ] [LOCAL ] VTEX: Starting deletion of primary data. [2025-10-20T21:49:19.871244] [DEBUG] [LOCAL ] VTEX: Deleted primary data file. [2025-10-20T21:49:19.871281] [INFO ] [MAIN ] VTEX: Starting full primary refresh. [2025-10-20T21:49:20.227396] [INFO ] [REMOTE ] VTEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:20.251326] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-01. [2025-10-20T21:49:20.306140] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-01 (1.64 seconds). [2025-10-20T21:49:20.507267] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-03 (2.65 seconds). [2025-10-20T21:49:20.653997] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-09 (1.19 seconds). [2025-10-20T21:49:21.051402] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-02. [2025-10-20T21:49:21.702879] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T21:49:21.851447] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-02. [2025-10-20T21:49:22.238225] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-02 (1.19 seconds). [2025-10-20T21:49:22.651360] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-04. [2025-10-20T21:49:23.283521] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T21:49:23.451397] [INFO ] [REMOTE ] VSH: Fetching primary data for 2025-10. [2025-10-20T21:49:24.251418] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-03. [2025-10-20T21:49:25.026100] [INFO ] [REMOTE ] VSH: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T21:49:25.039475] [INFO ] [REMOTE ] VSH: Full fetch complete (70935 records). [2025-10-20T21:49:25.039840] [DEBUG] [VALIDATOR ] VSH: Validating primary data. [2025-10-20T21:49:25.055848] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-03. [2025-10-20T21:49:25.528056] [DEBUG] [VALIDATOR ] VSH: All expected trading days are present. [2025-10-20T21:49:25.528639] [DEBUG] [VALIDATOR ] VSH: Primary data validation passed. [2025-10-20T21:49:25.530235] [INFO ] [MAIN ] VSH: Fetched primary data is valid. [2025-10-20T21:49:25.532230] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-04 (2.88 seconds). [2025-10-20T21:49:25.532341] [DEBUG] [LOCAL ] VSH: Saving primary data. [2025-10-20T21:49:25.544952] [DEBUG] [LOCAL ] VSH: Trimmed 70935 records to 48263 (keeping 200 days of data) [2025-10-20T21:49:25.841464] [DEBUG] [LOCAL ] VSH: Primary data saved (48263 records). [2025-10-20T21:49:25.841665] [INFO ] [MAIN ] VSH: Full history saved. [2025-10-20T21:49:25.841958] [INFO ] [MAIN ] Update Progress: 2251/2367 symbols completed. VSH took 29.2s. Estimated remaining: 14m 55s. Rate: 7.8 symbols/min [2025-10-20T21:49:25.856548] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-05. [2025-10-20T21:49:26.071504] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:49:26.073329] [INFO ] [MAIN ] VTLE: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:26.130796] [DEBUG] [LOCAL ] VTLE: Loaded 48615 existing records from primary storage. [2025-10-20T21:49:26.131118] [INFO ] [MAIN ] VTLE: Loaded 48615 existing records. [2025-10-20T21:49:26.324547] [WARN ] [MAIN ] VTLE: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:26.324789] [INFO ] [MAIN ] VTLE: Wiping existing primary data before full refresh. [2025-10-20T21:49:26.324840] [INFO ] [LOCAL ] VTLE: Starting deletion of primary data. [2025-10-20T21:49:26.325032] [DEBUG] [LOCAL ] VTLE: Deleted primary data file. [2025-10-20T21:49:26.325070] [INFO ] [MAIN ] VTLE: Starting full primary refresh. [2025-10-20T21:49:26.656522] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-04. [2025-10-20T21:49:26.711502] [INFO ] [REMOTE ] VTLE: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:26.881124] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T21:49:27.451384] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-04. [2025-10-20T21:49:28.251409] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-01. [2025-10-20T21:49:28.295993] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:49:28.422487] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-05 (2.57 seconds). [2025-10-20T21:49:28.999511] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-04 (1.55 seconds). [2025-10-20T21:49:29.051348] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-05. [2025-10-20T21:49:29.807843] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T21:49:29.851366] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-02. [2025-10-20T21:49:30.651348] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-06. [2025-10-20T21:49:30.823366] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T21:49:31.451411] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-05. [2025-10-20T21:49:31.476278] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T21:49:32.251425] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-06. [2025-10-20T21:49:33.012410] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-06 (2.36 seconds). [2025-10-20T21:49:33.047499] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T21:49:33.056456] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-07. [2025-10-20T21:49:33.831314] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-06 (1.58 seconds). [2025-10-20T21:49:33.851336] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-07. [2025-10-20T21:49:34.651434] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-06. [2025-10-20T21:49:35.165870] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T21:49:35.451436] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-08. [2025-10-20T21:49:35.628016] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-07 (2.57 seconds). [2025-10-20T21:49:36.128000] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:49:36.251348] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-08. [2025-10-20T21:49:37.041774] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T21:49:37.052529] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-07. [2025-10-20T21:49:37.851427] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-09. [2025-10-20T21:49:38.605070] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-07 (1.55 seconds). [2025-10-20T21:49:38.651356] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-08. [2025-10-20T21:49:38.744286] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T21:49:39.451342] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-09. [2025-10-20T21:49:39.491000] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-09 (1.64 seconds). [2025-10-20T21:49:40.251354] [INFO ] [REMOTE ] VSTS: Fetching primary data for 2025-10. [2025-10-20T21:49:40.307163] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-08 (1.66 seconds). [2025-10-20T21:49:41.051416] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-09. [2025-10-20T21:49:41.800974] [INFO ] [REMOTE ] VSTS: Incremental fetch complete for 2025-10 (1.55 seconds). [2025-10-20T21:49:41.808262] [INFO ] [REMOTE ] VSTS: Full fetch complete (69786 records). [2025-10-20T21:49:41.808449] [DEBUG] [VALIDATOR ] VSTS: Validating primary data. [2025-10-20T21:49:41.852209] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-03. [2025-10-20T21:49:41.981473] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-09 (2.53 seconds). [2025-10-20T21:49:42.245878] [DEBUG] [VALIDATOR ] VSTS: All expected trading days are present. [2025-10-20T21:49:42.246003] [DEBUG] [VALIDATOR ] VSTS: Primary data validation passed. [2025-10-20T21:49:42.247127] [INFO ] [MAIN ] VSTS: Fetched primary data is valid. [2025-10-20T21:49:42.247187] [DEBUG] [LOCAL ] VSTS: Saving primary data. [2025-10-20T21:49:42.248548] [DEBUG] [LOCAL ] VSTS: Trimmed 69786 records to 48911 (keeping 200 days of data) [2025-10-20T21:49:42.516723] [DEBUG] [LOCAL ] VSTS: Primary data saved (48911 records). [2025-10-20T21:49:42.516847] [INFO ] [MAIN ] VSTS: Full history saved. [2025-10-20T21:49:42.516926] [INFO ] [MAIN ] Update Progress: 2252/2367 symbols completed. VSTS took 24.6s. Estimated remaining: 14m 47s. Rate: 7.8 symbols/min [2025-10-20T21:49:42.652374] [INFO ] [REMOTE ] VST: Fetching primary data for 2025-10. [2025-10-20T21:49:42.722533] [INFO ] [MAIN ] VTMX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:42.724684] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:49:42.749918] [DEBUG] [LOCAL ] VTMX: Loaded 17871 existing records from primary storage. [2025-10-20T21:49:42.750156] [INFO ] [MAIN ] VTMX: Loaded 17871 existing records. [2025-10-20T21:49:42.931913] [WARN ] [MAIN ] VTMX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:42.932096] [INFO ] [MAIN ] VTMX: Wiping existing primary data before full refresh. [2025-10-20T21:49:42.932142] [INFO ] [LOCAL ] VTMX: Starting deletion of primary data. [2025-10-20T21:49:42.932323] [DEBUG] [LOCAL ] VTMX: Deleted primary data file. [2025-10-20T21:49:42.932361] [INFO ] [MAIN ] VTMX: Starting full primary refresh. [2025-10-20T21:49:43.313190] [INFO ] [REMOTE ] VTMX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:43.439079] [INFO ] [REMOTE ] VST: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:49:43.447230] [INFO ] [REMOTE ] VST: Full fetch complete (77888 records). [2025-10-20T21:49:43.447355] [DEBUG] [VALIDATOR ] VST: Validating primary data. [2025-10-20T21:49:43.459335] [INFO ] [REMOTE ] VTEX: Fetching primary data for 2025-10. [2025-10-20T21:49:43.796751] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T21:49:43.894963] [DEBUG] [VALIDATOR ] VST: All expected trading days are present. [2025-10-20T21:49:43.895045] [DEBUG] [VALIDATOR ] VST: Primary data validation passed. [2025-10-20T21:49:43.896190] [INFO ] [MAIN ] VST: Fetched primary data is valid. [2025-10-20T21:49:43.896251] [DEBUG] [LOCAL ] VST: Saving primary data. [2025-10-20T21:49:43.897603] [DEBUG] [LOCAL ] VST: Trimmed 77888 records to 53709 (keeping 200 days of data) [2025-10-20T21:49:44.232716] [DEBUG] [LOCAL ] VST: Primary data saved (53709 records). [2025-10-20T21:49:44.232854] [INFO ] [MAIN ] VST: Full history saved. [2025-10-20T21:49:44.232926] [INFO ] [MAIN ] Update Progress: 2253/2367 symbols completed. VST took 34.9s. Estimated remaining: 14m 39s. Rate: 7.8 symbols/min [2025-10-20T21:49:44.256603] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-04. [2025-10-20T21:49:44.403364] [INFO ] [MAIN ] VTN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:44.411540] [DEBUG] [LOCAL ] VTN: Loaded 7166 existing records from primary storage. [2025-10-20T21:49:44.411728] [INFO ] [MAIN ] VTN: Loaded 7166 existing records. [2025-10-20T21:49:44.593108] [WARN ] [MAIN ] VTN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:44.593309] [INFO ] [MAIN ] VTN: Wiping existing primary data before full refresh. [2025-10-20T21:49:44.593355] [INFO ] [LOCAL ] VTN: Starting deletion of primary data. [2025-10-20T21:49:44.593512] [DEBUG] [LOCAL ] VTN: Deleted primary data file. [2025-10-20T21:49:44.593550] [INFO ] [MAIN ] VTN: Starting full primary refresh. [2025-10-20T21:49:44.959698] [INFO ] [REMOTE ] VTN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:45.030547] [INFO ] [REMOTE ] VTEX: Incremental fetch complete for 2025-10 (1.57 seconds). [2025-10-20T21:49:45.036791] [INFO ] [REMOTE ] VTEX: Full fetch complete (60978 records). [2025-10-20T21:49:45.036890] [DEBUG] [VALIDATOR ] VTEX: Validating primary data. [2025-10-20T21:49:45.057504] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-01. [2025-10-20T21:49:45.426997] [DEBUG] [VALIDATOR ] VTEX: All expected trading days are present. [2025-10-20T21:49:45.427074] [DEBUG] [VALIDATOR ] VTEX: Primary data validation passed. [2025-10-20T21:49:45.428489] [INFO ] [MAIN ] VTEX: Fetched primary data is valid. [2025-10-20T21:49:45.428557] [DEBUG] [LOCAL ] VTEX: Saving primary data. [2025-10-20T21:49:45.429859] [DEBUG] [LOCAL ] VTEX: Trimmed 60978 records to 42953 (keeping 200 days of data) [2025-10-20T21:49:45.666666] [DEBUG] [LOCAL ] VTEX: Primary data saved (42953 records). [2025-10-20T21:49:45.666916] [INFO ] [MAIN ] VTEX: Full history saved. [2025-10-20T21:49:45.667014] [INFO ] [MAIN ] Update Progress: 2254/2367 symbols completed. VTEX took 26.2s. Estimated remaining: 14m 31s. Rate: 7.8 symbols/min [2025-10-20T21:49:45.837816] [INFO ] [MAIN ] VTOL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:49:45.851354] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-01. [2025-10-20T21:49:45.861082] [DEBUG] [LOCAL ] VTOL: Loaded 19920 existing records from primary storage. [2025-10-20T21:49:45.861290] [INFO ] [MAIN ] VTOL: Loaded 19920 existing records. [2025-10-20T21:49:46.041844] [WARN ] [MAIN ] VTOL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:49:46.042244] [INFO ] [MAIN ] VTOL: Wiping existing primary data before full refresh. [2025-10-20T21:49:46.042424] [INFO ] [LOCAL ] VTOL: Starting deletion of primary data. [2025-10-20T21:49:46.043406] [DEBUG] [LOCAL ] VTOL: Deleted primary data file. [2025-10-20T21:49:46.043465] [INFO ] [MAIN ] VTOL: Starting full primary refresh. [2025-10-20T21:49:46.148086] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-01 (1.09 seconds). [2025-10-20T21:49:46.372683] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T21:49:46.430547] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-01 (0.58 seconds). [2025-10-20T21:49:46.472549] [INFO ] [REMOTE ] VTOL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:49:46.651372] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-01. [2025-10-20T21:49:47.453682] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-02. [2025-10-20T21:49:47.455823] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-01 (0.80 seconds). [2025-10-20T21:49:48.251421] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-02. [2025-10-20T21:49:48.475431] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-02 (1.02 seconds). [2025-10-20T21:49:48.974959] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T21:49:49.051337] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-03. [2025-10-20T21:49:49.851392] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-05. [2025-10-20T21:49:49.858561] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-03 (0.81 seconds). [2025-10-20T21:49:50.651391] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-04. [2025-10-20T21:49:51.380848] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-04 (0.73 seconds). [2025-10-20T21:49:51.451413] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-05. [2025-10-20T21:49:51.547549] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T21:49:52.068757] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T21:49:52.251387] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-06. [2025-10-20T21:49:52.920431] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-06 (0.67 seconds). [2025-10-20T21:49:53.051364] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-03. [2025-10-20T21:49:53.851378] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-06. [2025-10-20T21:49:53.995004] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-03 (0.94 seconds). [2025-10-20T21:49:54.651382] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-04. [2025-10-20T21:49:55.451359] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-07. [2025-10-20T21:49:55.536272] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:49:55.720963] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T21:49:55.973327] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-07 (0.52 seconds). [2025-10-20T21:49:56.251369] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-07. [2025-10-20T21:49:57.051370] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-08. [2025-10-20T21:49:57.839151] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T21:49:57.851319] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-09. [2025-10-20T21:49:57.961817] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-07 (1.71 seconds). [2025-10-20T21:49:58.591438] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-09 (0.74 seconds). [2025-10-20T21:49:58.651380] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-08. [2025-10-20T21:49:59.451379] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-05. [2025-10-20T21:50:00.251378] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-02. [2025-10-20T21:50:00.292360] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:50:00.531031] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-05 (1.08 seconds). [2025-10-20T21:50:01.051358] [INFO ] [REMOTE ] VTN: Fetching primary data for 2025-10. [2025-10-20T21:50:01.287470] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T21:50:01.800621] [INFO ] [REMOTE ] VTN: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:50:01.802473] [INFO ] [REMOTE ] VTN: Full fetch complete (10828 records). [2025-10-20T21:50:01.802581] [DEBUG] [VALIDATOR ] VTN: Validating primary data. [2025-10-20T21:50:01.860947] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-06. [2025-10-20T21:50:02.213520] [DEBUG] [VALIDATOR ] VTN: All expected trading days are present. [2025-10-20T21:50:02.213622] [DEBUG] [VALIDATOR ] VTN: Primary data validation passed. [2025-10-20T21:50:02.213928] [INFO ] [MAIN ] VTN: Fetched primary data is valid. [2025-10-20T21:50:02.213976] [DEBUG] [LOCAL ] VTN: Saving primary data. [2025-10-20T21:50:02.214402] [DEBUG] [LOCAL ] VTN: Trimmed 10828 records to 7075 (keeping 200 days of data) [2025-10-20T21:50:02.257327] [DEBUG] [LOCAL ] VTN: Primary data saved (7075 records). [2025-10-20T21:50:02.257474] [INFO ] [MAIN ] VTN: Full history saved. [2025-10-20T21:50:02.257555] [INFO ] [MAIN ] Update Progress: 2255/2367 symbols completed. VTN took 18.0s. Estimated remaining: 14m 24s. Rate: 7.8 symbols/min [2025-10-20T21:50:02.431596] [INFO ] [MAIN ] VTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:02.486922] [DEBUG] [LOCAL ] VTR: Loaded 53526 existing records from primary storage. [2025-10-20T21:50:02.487368] [INFO ] [MAIN ] VTR: Loaded 53526 existing records. [2025-10-20T21:50:02.656301] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-09. [2025-10-20T21:50:02.677793] [WARN ] [MAIN ] VTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:02.678169] [INFO ] [MAIN ] VTR: Wiping existing primary data before full refresh. [2025-10-20T21:50:02.678218] [INFO ] [LOCAL ] VTR: Starting deletion of primary data. [2025-10-20T21:50:02.678412] [DEBUG] [LOCAL ] VTR: Deleted primary data file. [2025-10-20T21:50:02.678453] [INFO ] [MAIN ] VTR: Starting full primary refresh. [2025-10-20T21:50:03.038269] [INFO ] [REMOTE ] VTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:03.055801] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-06 (1.19 seconds). [2025-10-20T21:50:03.451376] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-07. [2025-10-20T21:50:04.166508] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T21:50:04.251392] [INFO ] [REMOTE ] VTLE: Fetching primary data for 2025-10. [2025-10-20T21:50:04.484459] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-07 (1.03 seconds). [2025-10-20T21:50:05.051409] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-01. [2025-10-20T21:50:05.765218] [INFO ] [REMOTE ] VTLE: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T21:50:05.771353] [INFO ] [REMOTE ] VTLE: Full fetch complete (68546 records). [2025-10-20T21:50:05.771485] [DEBUG] [VALIDATOR ] VTLE: Validating primary data. [2025-10-20T21:50:05.861680] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-03. [2025-10-20T21:50:06.156624] [DEBUG] [VALIDATOR ] VTLE: All expected trading days are present. [2025-10-20T21:50:06.156735] [DEBUG] [VALIDATOR ] VTLE: Primary data validation passed. [2025-10-20T21:50:06.157778] [INFO ] [MAIN ] VTLE: Fetched primary data is valid. [2025-10-20T21:50:06.157840] [DEBUG] [LOCAL ] VTLE: Saving primary data. [2025-10-20T21:50:06.159068] [DEBUG] [LOCAL ] VTLE: Trimmed 68546 records to 47796 (keeping 200 days of data) [2025-10-20T21:50:06.434508] [DEBUG] [LOCAL ] VTLE: Primary data saved (47796 records). [2025-10-20T21:50:06.434807] [INFO ] [MAIN ] VTLE: Full history saved. [2025-10-20T21:50:06.434911] [INFO ] [MAIN ] Update Progress: 2256/2367 symbols completed. VTLE took 40.6s. Estimated remaining: 14m 16s. Rate: 7.8 symbols/min [2025-10-20T21:50:06.617368] [INFO ] [MAIN ] VTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:06.649292] [DEBUG] [LOCAL ] VTS: Loaded 30394 existing records from primary storage. [2025-10-20T21:50:06.649585] [INFO ] [MAIN ] VTS: Loaded 30394 existing records. [2025-10-20T21:50:06.680193] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-08. [2025-10-20T21:50:06.838445] [WARN ] [MAIN ] VTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:06.838992] [INFO ] [MAIN ] VTS: Wiping existing primary data before full refresh. [2025-10-20T21:50:06.839165] [INFO ] [LOCAL ] VTS: Starting deletion of primary data. [2025-10-20T21:50:06.839447] [DEBUG] [LOCAL ] VTS: Deleted primary data file. [2025-10-20T21:50:06.839532] [INFO ] [MAIN ] VTS: Starting full primary refresh. [2025-10-20T21:50:06.964753] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-03 (1.10 seconds). [2025-10-20T21:50:07.072427] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-01 (2.02 seconds). [2025-10-20T21:50:07.292876] [INFO ] [REMOTE ] VTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:07.398616] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-08 (0.72 seconds). [2025-10-20T21:50:07.451367] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-02. [2025-10-20T21:50:08.251376] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-09. [2025-10-20T21:50:09.051361] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-04. [2025-10-20T21:50:09.264231] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-02 (1.81 seconds). [2025-10-20T21:50:09.312977] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-09 (1.06 seconds). [2025-10-20T21:50:09.851356] [INFO ] [REMOTE ] VTMX: Fetching primary data for 2025-10. [2025-10-20T21:50:10.136164] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-04 (1.08 seconds). [2025-10-20T21:50:10.651363] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-03. [2025-10-20T21:50:10.807412] [INFO ] [REMOTE ] VTMX: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T21:50:10.810201] [INFO ] [REMOTE ] VTMX: Full fetch complete (25601 records). [2025-10-20T21:50:10.810277] [DEBUG] [VALIDATOR ] VTMX: Validating primary data. [2025-10-20T21:50:11.207168] [DEBUG] [VALIDATOR ] VTMX: All expected trading days are present. [2025-10-20T21:50:11.207284] [DEBUG] [VALIDATOR ] VTMX: Primary data validation passed. [2025-10-20T21:50:11.207947] [INFO ] [MAIN ] VTMX: Fetched primary data is valid. [2025-10-20T21:50:11.208013] [DEBUG] [LOCAL ] VTMX: Saving primary data. [2025-10-20T21:50:11.208771] [DEBUG] [LOCAL ] VTMX: Trimmed 25601 records to 17621 (keeping 200 days of data) [2025-10-20T21:50:11.323761] [DEBUG] [LOCAL ] VTMX: Primary data saved (17621 records). [2025-10-20T21:50:11.323869] [INFO ] [MAIN ] VTMX: Full history saved. [2025-10-20T21:50:11.323947] [INFO ] [MAIN ] Update Progress: 2257/2367 symbols completed. VTMX took 28.8s. Estimated remaining: 14m 8s. Rate: 7.8 symbols/min [2025-10-20T21:50:11.456481] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-05. [2025-10-20T21:50:11.519753] [INFO ] [MAIN ] VVR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:11.550753] [DEBUG] [LOCAL ] VVR: Loaded 31776 existing records from primary storage. [2025-10-20T21:50:11.551028] [INFO ] [MAIN ] VVR: Loaded 31776 existing records. [2025-10-20T21:50:11.745893] [WARN ] [MAIN ] VVR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:11.746170] [INFO ] [MAIN ] VVR: Wiping existing primary data before full refresh. [2025-10-20T21:50:11.746221] [INFO ] [LOCAL ] VVR: Starting deletion of primary data. [2025-10-20T21:50:11.746401] [DEBUG] [LOCAL ] VVR: Deleted primary data file. [2025-10-20T21:50:11.746440] [INFO ] [MAIN ] VVR: Starting full primary refresh. [2025-10-20T21:50:12.129092] [INFO ] [REMOTE ] VVR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:12.251331] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-01. [2025-10-20T21:50:12.625554] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:50:13.051362] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-01. [2025-10-20T21:50:13.711883] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T21:50:13.851368] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-02. [2025-10-20T21:50:14.158136] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T21:50:14.651324] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-02. [2025-10-20T21:50:14.791946] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T21:50:15.451364] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-03. [2025-10-20T21:50:15.738248] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T21:50:15.914675] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-03 (5.26 seconds). [2025-10-20T21:50:16.251418] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-04. [2025-10-20T21:50:16.855048] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T21:50:17.051414] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-03. [2025-10-20T21:50:17.851425] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-06. [2025-10-20T21:50:18.054875] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-04 (1.80 seconds). [2025-10-20T21:50:18.440057] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T21:50:18.651457] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-04. [2025-10-20T21:50:18.908985] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T21:50:19.451355] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-04. [2025-10-20T21:50:20.083902] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-04 (1.43 seconds). [2025-10-20T21:50:20.251441] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-05. [2025-10-20T21:50:20.866515] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T21:50:21.051358] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-07. [2025-10-20T21:50:21.416494] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:50:21.851366] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-06. [2025-10-20T21:50:22.067248] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-07 (1.02 seconds). [2025-10-20T21:50:22.651384] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-08. [2025-10-20T21:50:22.915488] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-06 (1.06 seconds). [2025-10-20T21:50:23.451367] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-05. [2025-10-20T21:50:23.681216] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:50:24.251472] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-07. [2025-10-20T21:50:25.051422] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-05. [2025-10-20T21:50:25.258179] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:50:25.523495] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T21:50:25.851394] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-08. [2025-10-20T21:50:26.068861] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-05 (1.02 seconds). [2025-10-20T21:50:26.651483] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-09. [2025-10-20T21:50:27.154944] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-08 (1.30 seconds). [2025-10-20T21:50:27.451390] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-06. [2025-10-20T21:50:27.634891] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-09 (0.98 seconds). [2025-10-20T21:50:28.251369] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-09. [2025-10-20T21:50:29.051404] [INFO ] [REMOTE ] VTOL: Fetching primary data for 2025-10. [2025-10-20T21:50:29.229274] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-06 (1.78 seconds). [2025-10-20T21:50:29.727879] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T21:50:29.851337] [INFO ] [REMOTE ] VVR: Fetching primary data for 2025-10. [2025-10-20T21:50:29.953508] [INFO ] [REMOTE ] VTOL: Incremental fetch complete for 2025-10 (0.90 seconds). [2025-10-20T21:50:29.956391] [INFO ] [REMOTE ] VTOL: Full fetch complete (26351 records). [2025-10-20T21:50:29.956469] [DEBUG] [VALIDATOR ] VTOL: Validating primary data. [2025-10-20T21:50:30.315531] [DEBUG] [VALIDATOR ] VTOL: All expected trading days are present. [2025-10-20T21:50:30.315622] [DEBUG] [VALIDATOR ] VTOL: Primary data validation passed. [2025-10-20T21:50:30.316191] [INFO ] [MAIN ] VTOL: Fetched primary data is valid. [2025-10-20T21:50:30.316237] [DEBUG] [LOCAL ] VTOL: Saving primary data. [2025-10-20T21:50:30.316889] [DEBUG] [LOCAL ] VTOL: Trimmed 26351 records to 19532 (keeping 200 days of data) [2025-10-20T21:50:30.425066] [DEBUG] [LOCAL ] VTOL: Primary data saved (19532 records). [2025-10-20T21:50:30.425193] [INFO ] [MAIN ] VTOL: Full history saved. [2025-10-20T21:50:30.425267] [INFO ] [MAIN ] Update Progress: 2258/2367 symbols completed. VTOL took 44.8s. Estimated remaining: 14m 1s. Rate: 7.8 symbols/min [2025-10-20T21:50:30.591247] [INFO ] [MAIN ] VVV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:30.637358] [DEBUG] [LOCAL ] VVV: Loaded 51015 existing records from primary storage. [2025-10-20T21:50:30.637700] [INFO ] [MAIN ] VVV: Loaded 51015 existing records. [2025-10-20T21:50:30.657755] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-07. [2025-10-20T21:50:30.814212] [WARN ] [MAIN ] VVV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:30.814394] [INFO ] [MAIN ] VVV: Wiping existing primary data before full refresh. [2025-10-20T21:50:30.814440] [INFO ] [LOCAL ] VVV: Starting deletion of primary data. [2025-10-20T21:50:30.814685] [DEBUG] [LOCAL ] VVV: Deleted primary data file. [2025-10-20T21:50:30.814739] [INFO ] [MAIN ] VVV: Starting full primary refresh. [2025-10-20T21:50:31.161303] [INFO ] [REMOTE ] VVV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:31.178794] [INFO ] [REMOTE ] VVR: Incremental fetch complete for 2025-10 (1.33 seconds). [2025-10-20T21:50:31.184253] [INFO ] [REMOTE ] VVR: Full fetch complete (46760 records). [2025-10-20T21:50:31.184340] [DEBUG] [VALIDATOR ] VVR: Validating primary data. [2025-10-20T21:50:31.456722] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-06. [2025-10-20T21:50:31.560178] [DEBUG] [VALIDATOR ] VVR: All expected trading days are present. [2025-10-20T21:50:31.560256] [DEBUG] [VALIDATOR ] VVR: Primary data validation passed. [2025-10-20T21:50:31.561013] [INFO ] [MAIN ] VVR: Fetched primary data is valid. [2025-10-20T21:50:31.561098] [DEBUG] [LOCAL ] VVR: Saving primary data. [2025-10-20T21:50:31.562008] [DEBUG] [LOCAL ] VVR: Trimmed 46760 records to 31242 (keeping 200 days of data) [2025-10-20T21:50:31.734300] [DEBUG] [LOCAL ] VVR: Primary data saved (31242 records). [2025-10-20T21:50:31.734429] [INFO ] [MAIN ] VVR: Full history saved. [2025-10-20T21:50:31.734504] [INFO ] [MAIN ] Update Progress: 2259/2367 symbols completed. VVR took 20.4s. Estimated remaining: 13m 53s. Rate: 7.8 symbols/min [2025-10-20T21:50:31.899381] [INFO ] [MAIN ] VVX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:31.921917] [DEBUG] [LOCAL ] VVX: Loaded 23319 existing records from primary storage. [2025-10-20T21:50:31.922147] [INFO ] [MAIN ] VVX: Loaded 23319 existing records. [2025-10-20T21:50:32.098741] [WARN ] [MAIN ] VVX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:32.098991] [INFO ] [MAIN ] VVX: Wiping existing primary data before full refresh. [2025-10-20T21:50:32.099044] [INFO ] [LOCAL ] VVX: Starting deletion of primary data. [2025-10-20T21:50:32.099236] [DEBUG] [LOCAL ] VVX: Deleted primary data file. [2025-10-20T21:50:32.099288] [INFO ] [MAIN ] VVX: Starting full primary refresh. [2025-10-20T21:50:32.269622] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-01. [2025-10-20T21:50:32.450733] [INFO ] [REMOTE ] VVX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:32.485378] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T21:50:32.889125] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:50:33.051400] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-08. [2025-10-20T21:50:33.851368] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-07. [2025-10-20T21:50:33.955489] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-01 (1.69 seconds). [2025-10-20T21:50:34.651452] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-02. [2025-10-20T21:50:34.791403] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T21:50:35.108561] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-07 (1.26 seconds). [2025-10-20T21:50:35.451439] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-08. [2025-10-20T21:50:35.961975] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:50:36.251432] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-01. [2025-10-20T21:50:36.383634] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-08 (0.93 seconds). [2025-10-20T21:50:37.051362] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-09. [2025-10-20T21:50:37.381690] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-01 (1.13 seconds). [2025-10-20T21:50:37.851387] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-03. [2025-10-20T21:50:38.651393] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-09. [2025-10-20T21:50:38.761937] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T21:50:39.451391] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-02. [2025-10-20T21:50:39.506204] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-03 (1.65 seconds). [2025-10-20T21:50:40.018386] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T21:50:40.251407] [INFO ] [REMOTE ] VTS: Fetching primary data for 2025-10. [2025-10-20T21:50:40.521690] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-02 (1.07 seconds). [2025-10-20T21:50:41.051406] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-03. [2025-10-20T21:50:41.452532] [INFO ] [REMOTE ] VTS: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T21:50:41.456976] [INFO ] [REMOTE ] VTS: Full fetch complete (41567 records). [2025-10-20T21:50:41.457103] [DEBUG] [VALIDATOR ] VTS: Validating primary data. [2025-10-20T21:50:41.827938] [DEBUG] [VALIDATOR ] VTS: All expected trading days are present. [2025-10-20T21:50:41.828031] [DEBUG] [VALIDATOR ] VTS: Primary data validation passed. [2025-10-20T21:50:41.828884] [INFO ] [MAIN ] VTS: Fetched primary data is valid. [2025-10-20T21:50:41.829057] [DEBUG] [LOCAL ] VTS: Saving primary data. [2025-10-20T21:50:41.830488] [DEBUG] [LOCAL ] VTS: Trimmed 41567 records to 29986 (keeping 200 days of data) [2025-10-20T21:50:41.881312] [INFO ] [REMOTE ] VTR: Fetching primary data for 2025-10. [2025-10-20T21:50:42.015788] [DEBUG] [LOCAL ] VTS: Primary data saved (29986 records). [2025-10-20T21:50:42.015922] [INFO ] [MAIN ] VTS: Full history saved. [2025-10-20T21:50:42.016005] [INFO ] [MAIN ] Update Progress: 2260/2367 symbols completed. VTS took 35.6s. Estimated remaining: 13m 46s. Rate: 7.8 symbols/min [2025-10-20T21:50:42.188221] [INFO ] [MAIN ] VYX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:42.205737] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-03 (1.15 seconds). [2025-10-20T21:50:42.256177] [DEBUG] [LOCAL ] VYX: Loaded 49165 existing records from primary storage. [2025-10-20T21:50:42.256495] [INFO ] [MAIN ] VYX: Loaded 49165 existing records. [2025-10-20T21:50:42.437215] [WARN ] [MAIN ] VYX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:42.437423] [INFO ] [MAIN ] VYX: Wiping existing primary data before full refresh. [2025-10-20T21:50:42.437471] [INFO ] [LOCAL ] VYX: Starting deletion of primary data. [2025-10-20T21:50:42.437705] [DEBUG] [LOCAL ] VYX: Deleted primary data file. [2025-10-20T21:50:42.437771] [INFO ] [MAIN ] VYX: Starting full primary refresh. [2025-10-20T21:50:42.655006] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-04. [2025-10-20T21:50:42.843909] [INFO ] [REMOTE ] VTR: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T21:50:42.842634] [INFO ] [REMOTE ] VYX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:42.851789] [INFO ] [REMOTE ] VTR: Full fetch complete (76478 records). [2025-10-20T21:50:42.851893] [DEBUG] [VALIDATOR ] VTR: Validating primary data. [2025-10-20T21:50:43.243635] [DEBUG] [VALIDATOR ] VTR: All expected trading days are present. [2025-10-20T21:50:43.243759] [DEBUG] [VALIDATOR ] VTR: Primary data validation passed. [2025-10-20T21:50:43.245973] [INFO ] [MAIN ] VTR: Fetched primary data is valid. [2025-10-20T21:50:43.246046] [DEBUG] [LOCAL ] VTR: Saving primary data. [2025-10-20T21:50:43.247515] [DEBUG] [LOCAL ] VTR: Trimmed 76478 records to 52720 (keeping 200 days of data) [2025-10-20T21:50:43.455574] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-04. [2025-10-20T21:50:43.568105] [DEBUG] [LOCAL ] VTR: Primary data saved (52720 records). [2025-10-20T21:50:43.568243] [INFO ] [MAIN ] VTR: Full history saved. [2025-10-20T21:50:43.568319] [INFO ] [MAIN ] Update Progress: 2261/2367 symbols completed. VTR took 41.3s. Estimated remaining: 13m 38s. Rate: 7.8 symbols/min [2025-10-20T21:50:43.751879] [INFO ] [MAIN ] VZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:43.801616] [DEBUG] [LOCAL ] VZ: Loaded 54571 existing records from primary storage. [2025-10-20T21:50:43.801975] [INFO ] [MAIN ] VZ: Loaded 54571 existing records. [2025-10-20T21:50:43.982495] [WARN ] [MAIN ] VZ: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:43.982780] [INFO ] [MAIN ] VZ: Wiping existing primary data before full refresh. [2025-10-20T21:50:43.982840] [INFO ] [LOCAL ] VZ: Starting deletion of primary data. [2025-10-20T21:50:43.983025] [DEBUG] [LOCAL ] VZ: Deleted primary data file. [2025-10-20T21:50:43.983064] [INFO ] [MAIN ] VZ: Starting full primary refresh. [2025-10-20T21:50:44.255323] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-01. [2025-10-20T21:50:44.373884] [INFO ] [REMOTE ] VZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:44.471884] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T21:50:44.608438] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T21:50:45.051380] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-05. [2025-10-20T21:50:45.851409] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-01. [2025-10-20T21:50:46.007421] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T21:50:46.651347] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-05. [2025-10-20T21:50:46.820720] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-05 (1.77 seconds). [2025-10-20T21:50:47.451348] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-06. [2025-10-20T21:50:47.786002] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-05 (1.13 seconds). [2025-10-20T21:50:48.214402] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-01 (2.36 seconds). [2025-10-20T21:50:48.251339] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-06. [2025-10-20T21:50:48.708868] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-06 (1.26 seconds). [2025-10-20T21:50:49.051357] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-07. [2025-10-20T21:50:49.471556] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-06 (1.22 seconds). [2025-10-20T21:50:49.851371] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-07. [2025-10-20T21:50:50.651351] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-02. [2025-10-20T21:50:50.727755] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T21:50:50.900567] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T21:50:51.451387] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-08. [2025-10-20T21:50:52.251380] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-02. [2025-10-20T21:50:52.296252] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:50:53.051356] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-08. [2025-10-20T21:50:53.255703] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-08 (1.80 seconds). [2025-10-20T21:50:53.851377] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-09. [2025-10-20T21:50:54.259503] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-02 (2.01 seconds). [2025-10-20T21:50:54.525183] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-08 (1.47 seconds). [2025-10-20T21:50:54.651391] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-09. [2025-10-20T21:50:55.120980] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-09 (1.27 seconds). [2025-10-20T21:50:55.451397] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-03. [2025-10-20T21:50:56.024203] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T21:50:56.251390] [INFO ] [REMOTE ] VVX: Fetching primary data for 2025-10. [2025-10-20T21:50:57.051388] [INFO ] [REMOTE ] VVV: Fetching primary data for 2025-10. [2025-10-20T21:50:57.311675] [INFO ] [REMOTE ] VVX: Incremental fetch complete for 2025-10 (1.06 seconds). [2025-10-20T21:50:57.315334] [INFO ] [REMOTE ] VVX: Full fetch complete (33105 records). [2025-10-20T21:50:57.315471] [DEBUG] [VALIDATOR ] VVX: Validating primary data. [2025-10-20T21:50:57.716815] [DEBUG] [VALIDATOR ] VVX: All expected trading days are present. [2025-10-20T21:50:57.716899] [DEBUG] [VALIDATOR ] VVX: Primary data validation passed. [2025-10-20T21:50:57.717570] [INFO ] [MAIN ] VVX: Fetched primary data is valid. [2025-10-20T21:50:57.717609] [DEBUG] [LOCAL ] VVX: Saving primary data. [2025-10-20T21:50:57.718385] [DEBUG] [LOCAL ] VVX: Trimmed 33105 records to 23053 (keeping 200 days of data) [2025-10-20T21:50:57.853257] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-03. [2025-10-20T21:50:57.858329] [DEBUG] [LOCAL ] VVX: Primary data saved (23053 records). [2025-10-20T21:50:57.859869] [INFO ] [MAIN ] VVX: Full history saved. [2025-10-20T21:50:57.859983] [INFO ] [MAIN ] Update Progress: 2262/2367 symbols completed. VVX took 26.1s. Estimated remaining: 13m 30s. Rate: 7.8 symbols/min [2025-10-20T21:50:58.060159] [INFO ] [MAIN ] W: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:58.092740] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-03 (2.64 seconds). [2025-10-20T21:50:58.148847] [DEBUG] [LOCAL ] W: Loaded 54205 existing records from primary storage. [2025-10-20T21:50:58.149164] [INFO ] [MAIN ] W: Loaded 54205 existing records. [2025-10-20T21:50:58.340048] [WARN ] [MAIN ] W: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:58.340391] [INFO ] [MAIN ] W: Wiping existing primary data before full refresh. [2025-10-20T21:50:58.340453] [INFO ] [LOCAL ] W: Starting deletion of primary data. [2025-10-20T21:50:58.341079] [DEBUG] [LOCAL ] W: Deleted primary data file. [2025-10-20T21:50:58.341138] [INFO ] [MAIN ] W: Starting full primary refresh. [2025-10-20T21:50:58.655070] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-04. [2025-10-20T21:50:58.708230] [INFO ] [REMOTE ] W: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:50:58.735977] [INFO ] [REMOTE ] VVV: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:50:58.744390] [INFO ] [REMOTE ] VVV: Full fetch complete (72514 records). [2025-10-20T21:50:58.744551] [DEBUG] [VALIDATOR ] VVV: Validating primary data. [2025-10-20T21:50:59.146932] [DEBUG] [VALIDATOR ] VVV: All expected trading days are present. [2025-10-20T21:50:59.147022] [DEBUG] [VALIDATOR ] VVV: Primary data validation passed. [2025-10-20T21:50:59.148150] [INFO ] [MAIN ] VVV: Fetched primary data is valid. [2025-10-20T21:50:59.148219] [DEBUG] [LOCAL ] VVV: Saving primary data. [2025-10-20T21:50:59.149637] [DEBUG] [LOCAL ] VVV: Trimmed 72514 records to 50231 (keeping 200 days of data) [2025-10-20T21:50:59.441982] [DEBUG] [LOCAL ] VVV: Primary data saved (50231 records). [2025-10-20T21:50:59.442114] [INFO ] [MAIN ] VVV: Full history saved. [2025-10-20T21:50:59.442189] [INFO ] [MAIN ] Update Progress: 2263/2367 symbols completed. VVV took 29.0s. Estimated remaining: 13m 22s. Rate: 7.8 symbols/min [2025-10-20T21:50:59.474943] [INFO ] [REMOTE ] W: Fetching primary data for 2025-01. [2025-10-20T21:50:59.620896] [INFO ] [MAIN ] WAB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:50:59.652377] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-03 (1.80 seconds). [2025-10-20T21:50:59.702085] [DEBUG] [LOCAL ] WAB: Loaded 44602 existing records from primary storage. [2025-10-20T21:50:59.702491] [INFO ] [MAIN ] WAB: Loaded 44602 existing records. [2025-10-20T21:50:59.891513] [WARN ] [MAIN ] WAB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:50:59.891774] [INFO ] [MAIN ] WAB: Wiping existing primary data before full refresh. [2025-10-20T21:50:59.891823] [INFO ] [LOCAL ] WAB: Starting deletion of primary data. [2025-10-20T21:50:59.892054] [DEBUG] [LOCAL ] WAB: Deleted primary data file. [2025-10-20T21:50:59.892110] [INFO ] [MAIN ] WAB: Starting full primary refresh. [2025-10-20T21:51:00.259189] [INFO ] [REMOTE ] WAB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:00.259299] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-04. [2025-10-20T21:51:01.051386] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-01. [2025-10-20T21:51:01.188218] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-04 (2.53 seconds). [2025-10-20T21:51:01.282725] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T21:51:01.851369] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-05. [2025-10-20T21:51:02.027240] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T21:51:02.651448] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-05. [2025-10-20T21:51:02.811823] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-01 (1.76 seconds). [2025-10-20T21:51:03.451403] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-02. [2025-10-20T21:51:04.130841] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-05 (2.28 seconds). [2025-10-20T21:51:04.171549] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-05 (1.52 seconds). [2025-10-20T21:51:04.251342] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-06. [2025-10-20T21:51:05.051360] [INFO ] [REMOTE ] W: Fetching primary data for 2025-02. [2025-10-20T21:51:05.243947] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-02 (1.79 seconds). [2025-10-20T21:51:05.860387] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-03. [2025-10-20T21:51:05.867011] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-06 (1.62 seconds). [2025-10-20T21:51:06.651367] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-07. [2025-10-20T21:51:06.731322] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-02 (1.68 seconds). [2025-10-20T21:51:07.451358] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-06. [2025-10-20T21:51:07.678547] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-03 (1.82 seconds). [2025-10-20T21:51:08.251367] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-04. [2025-10-20T21:51:08.345672] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T21:51:09.051484] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-08. [2025-10-20T21:51:09.692899] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-06 (2.24 seconds). [2025-10-20T21:51:09.851398] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-07. [2025-10-20T21:51:10.068680] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T21:51:10.651436] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-05. [2025-10-20T21:51:10.704281] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:51:11.451425] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-09. [2025-10-20T21:51:12.251370] [INFO ] [REMOTE ] W: Fetching primary data for 2025-03. [2025-10-20T21:51:12.299287] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T21:51:12.492753] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-07 (2.64 seconds). [2025-10-20T21:51:13.051356] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-08. [2025-10-20T21:51:13.131129] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T21:51:13.731507] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T21:51:13.851369] [INFO ] [REMOTE ] VYX: Fetching primary data for 2025-10. [2025-10-20T21:51:14.651426] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-06. [2025-10-20T21:51:15.235787] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-08 (2.18 seconds). [2025-10-20T21:51:15.273627] [INFO ] [REMOTE ] VYX: Incremental fetch complete for 2025-10 (1.42 seconds). [2025-10-20T21:51:15.280508] [INFO ] [REMOTE ] VYX: Full fetch complete (70211 records). [2025-10-20T21:51:15.280687] [DEBUG] [VALIDATOR ] VYX: Validating primary data. [2025-10-20T21:51:15.454484] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-09. [2025-10-20T21:51:15.681523] [DEBUG] [VALIDATOR ] VYX: All expected trading days are present. [2025-10-20T21:51:15.681608] [DEBUG] [VALIDATOR ] VYX: Primary data validation passed. [2025-10-20T21:51:15.682812] [INFO ] [MAIN ] VYX: Fetched primary data is valid. [2025-10-20T21:51:15.682876] [DEBUG] [LOCAL ] VYX: Saving primary data. [2025-10-20T21:51:15.684261] [DEBUG] [LOCAL ] VYX: Trimmed 70211 records to 48363 (keeping 200 days of data) [2025-10-20T21:51:15.961405] [DEBUG] [LOCAL ] VYX: Primary data saved (48363 records). [2025-10-20T21:51:15.961551] [INFO ] [MAIN ] VYX: Full history saved. [2025-10-20T21:51:15.961637] [INFO ] [MAIN ] Update Progress: 2264/2367 symbols completed. VYX took 33.9s. Estimated remaining: 13m 15s. Rate: 7.8 symbols/min [2025-10-20T21:51:16.130936] [INFO ] [MAIN ] WAL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:16.171341] [DEBUG] [LOCAL ] WAL: Loaded 46291 existing records from primary storage. [2025-10-20T21:51:16.171634] [INFO ] [MAIN ] WAL: Loaded 46291 existing records. [2025-10-20T21:51:16.276899] [INFO ] [REMOTE ] W: Fetching primary data for 2025-04. [2025-10-20T21:51:16.358554] [WARN ] [MAIN ] WAL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:16.359381] [INFO ] [MAIN ] WAL: Wiping existing primary data before full refresh. [2025-10-20T21:51:16.367211] [INFO ] [LOCAL ] WAL: Starting deletion of primary data. [2025-10-20T21:51:16.380755] [DEBUG] [LOCAL ] WAL: Deleted primary data file. [2025-10-20T21:51:16.380857] [INFO ] [MAIN ] WAL: Starting full primary refresh. [2025-10-20T21:51:16.381536] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-06 (1.73 seconds). [2025-10-20T21:51:16.747127] [INFO ] [REMOTE ] WAL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:17.051365] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-07. [2025-10-20T21:51:17.851352] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-01. [2025-10-20T21:51:18.001034] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-09 (2.55 seconds). [2025-10-20T21:51:18.460911] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-04 (2.18 seconds). [2025-10-20T21:51:18.651350] [INFO ] [REMOTE ] W: Fetching primary data for 2025-05. [2025-10-20T21:51:18.778420] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-07 (1.73 seconds). [2025-10-20T21:51:19.451408] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-08. [2025-10-20T21:51:19.568454] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T21:51:20.251348] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-02. [2025-10-20T21:51:20.732201] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-05 (2.08 seconds). [2025-10-20T21:51:20.771449] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-08 (1.32 seconds). [2025-10-20T21:51:21.051359] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-09. [2025-10-20T21:51:21.834898] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-02 (1.58 seconds). [2025-10-20T21:51:21.851344] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-03. [2025-10-20T21:51:22.651434] [INFO ] [REMOTE ] VZ: Fetching primary data for 2025-10. [2025-10-20T21:51:22.722216] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:51:23.393185] [INFO ] [REMOTE ] VZ: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:51:23.401557] [INFO ] [REMOTE ] VZ: Full fetch complete (77971 records). [2025-10-20T21:51:23.401748] [DEBUG] [VALIDATOR ] VZ: Validating primary data. [2025-10-20T21:51:23.458464] [INFO ] [REMOTE ] W: Fetching primary data for 2025-06. [2025-10-20T21:51:23.832209] [DEBUG] [VALIDATOR ] VZ: All expected trading days are present. [2025-10-20T21:51:23.832304] [DEBUG] [VALIDATOR ] VZ: Primary data validation passed. [2025-10-20T21:51:23.833668] [INFO ] [MAIN ] VZ: Fetched primary data is valid. [2025-10-20T21:51:23.833750] [DEBUG] [LOCAL ] VZ: Saving primary data. [2025-10-20T21:51:23.862761] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-03 (2.01 seconds). [2025-10-20T21:51:23.868105] [DEBUG] [LOCAL ] VZ: Trimmed 77971 records to 53791 (keeping 200 days of data) [2025-10-20T21:51:24.212498] [DEBUG] [LOCAL ] VZ: Primary data saved (53791 records). [2025-10-20T21:51:24.212635] [INFO ] [MAIN ] VZ: Full history saved. [2025-10-20T21:51:24.212754] [INFO ] [MAIN ] Update Progress: 2265/2367 symbols completed. VZ took 40.6s. Estimated remaining: 13m 7s. Rate: 7.8 symbols/min [2025-10-20T21:51:24.256627] [INFO ] [REMOTE ] WAB: Fetching primary data for 2025-10. [2025-10-20T21:51:24.390323] [INFO ] [MAIN ] WAT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:24.425190] [DEBUG] [LOCAL ] WAT: Loaded 36355 existing records from primary storage. [2025-10-20T21:51:24.425466] [INFO ] [MAIN ] WAT: Loaded 36355 existing records. [2025-10-20T21:51:24.625681] [WARN ] [MAIN ] WAT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:24.625956] [INFO ] [MAIN ] WAT: Wiping existing primary data before full refresh. [2025-10-20T21:51:24.626040] [INFO ] [LOCAL ] WAT: Starting deletion of primary data. [2025-10-20T21:51:24.626253] [DEBUG] [LOCAL ] WAT: Deleted primary data file. [2025-10-20T21:51:24.626295] [INFO ] [MAIN ] WAT: Starting full primary refresh. [2025-10-20T21:51:25.000884] [INFO ] [REMOTE ] WAT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:25.019460] [INFO ] [REMOTE ] WAB: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T21:51:25.025948] [INFO ] [REMOTE ] WAB: Full fetch complete (64434 records). [2025-10-20T21:51:25.026081] [DEBUG] [VALIDATOR ] WAB: Validating primary data. [2025-10-20T21:51:25.057487] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-04. [2025-10-20T21:51:25.417338] [DEBUG] [VALIDATOR ] WAB: All expected trading days are present. [2025-10-20T21:51:25.417506] [DEBUG] [VALIDATOR ] WAB: Primary data validation passed. [2025-10-20T21:51:25.418676] [INFO ] [MAIN ] WAB: Fetched primary data is valid. [2025-10-20T21:51:25.418782] [DEBUG] [LOCAL ] WAB: Saving primary data. [2025-10-20T21:51:25.420292] [DEBUG] [LOCAL ] WAB: Trimmed 64434 records to 43863 (keeping 200 days of data) [2025-10-20T21:51:25.450709] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-06 (1.99 seconds). [2025-10-20T21:51:25.735152] [DEBUG] [LOCAL ] WAB: Primary data saved (43863 records). [2025-10-20T21:51:25.735302] [INFO ] [MAIN ] WAB: Full history saved. [2025-10-20T21:51:25.735384] [INFO ] [MAIN ] Update Progress: 2266/2367 symbols completed. WAB took 26.3s. Estimated remaining: 12m 59s. Rate: 7.8 symbols/min [2025-10-20T21:51:25.856506] [INFO ] [REMOTE ] W: Fetching primary data for 2025-07. [2025-10-20T21:51:25.912377] [INFO ] [MAIN ] WBI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:25.912510] [DEBUG] [LOCAL ] WBI: No existing data file found in primary storage. [2025-10-20T21:51:25.912546] [WARN ] [MAIN ] WBI: No existing data found, performing full refresh instead. [2025-10-20T21:51:25.912681] [INFO ] [MAIN ] WBI: Wiping existing primary data before full refresh. [2025-10-20T21:51:25.912734] [INFO ] [LOCAL ] WBI: Starting deletion of primary data. [2025-10-20T21:51:25.912775] [DEBUG] [LOCAL ] WBI: No primary data file found to delete. [2025-10-20T21:51:25.912806] [INFO ] [MAIN ] WBI: Starting full primary refresh. [2025-10-20T21:51:26.278006] [INFO ] [REMOTE ] WBI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:26.651391] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-01. [2025-10-20T21:51:26.898459] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-04 (1.84 seconds). [2025-10-20T21:51:27.451710] [INFO ] [REMOTE ] WBI: Fetching primary data for 2025-01. [2025-10-20T21:51:27.655196] [DEBUG] [REMOTE ] WBI: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:51:27.655738] [WARN ] [REMOTE ] WBI: No data returned for month 2025-01 [2025-10-20T21:51:27.655872] [FATAL] [REMOTE ] WBI: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:51:27.656011] [FATAL] [MAIN ] WBI: Failed to fetch primary data. [2025-10-20T21:51:27.656078] [WARN ] [MAIN ] WBI: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:51:27.656134] [INFO ] [LOCAL ] WBI: Starting deletion of primary data. [2025-10-20T21:51:27.656218] [DEBUG] [LOCAL ] WBI: No primary data file found to delete. [2025-10-20T21:51:27.656271] [INFO ] [LOCAL ] WBI: Starting deletion of derivative data. [2025-10-20T21:51:27.656325] [DEBUG] [LOCAL ] WBI: No derivative data file found to delete. [2025-10-20T21:51:27.656394] [INFO ] [MAIN ] Update Progress: 2267/2367 symbols completed. WBI took 1.9s. Estimated remaining: 12m 51s. Rate: 7.8 symbols/min [2025-10-20T21:51:27.918787] [INFO ] [MAIN ] WBS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:27.994910] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T21:51:27.999592] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-07 (2.14 seconds). [2025-10-20T21:51:28.083754] [DEBUG] [LOCAL ] WBS: Loaded 49516 existing records from primary storage. [2025-10-20T21:51:28.084236] [INFO ] [MAIN ] WBS: Loaded 49516 existing records. [2025-10-20T21:51:28.265346] [INFO ] [REMOTE ] W: Fetching primary data for 2025-08. [2025-10-20T21:51:28.311697] [WARN ] [MAIN ] WBS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:28.311921] [INFO ] [MAIN ] WBS: Wiping existing primary data before full refresh. [2025-10-20T21:51:28.311973] [INFO ] [LOCAL ] WBS: Starting deletion of primary data. [2025-10-20T21:51:28.312247] [DEBUG] [LOCAL ] WBS: Deleted primary data file. [2025-10-20T21:51:28.312311] [INFO ] [MAIN ] WBS: Starting full primary refresh. [2025-10-20T21:51:28.733986] [INFO ] [REMOTE ] WBS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:29.051398] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-02. [2025-10-20T21:51:29.851403] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-01. [2025-10-20T21:51:30.276358] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-08 (2.01 seconds). [2025-10-20T21:51:30.536449] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-02 (1.49 seconds). [2025-10-20T21:51:30.651416] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-03. [2025-10-20T21:51:31.451383] [INFO ] [REMOTE ] W: Fetching primary data for 2025-09. [2025-10-20T21:51:31.554093] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T21:51:31.653400] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-03 (1.00 seconds). [2025-10-20T21:51:32.251444] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-05. [2025-10-20T21:51:33.051349] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-02. [2025-10-20T21:51:33.365437] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T21:51:33.817570] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:51:33.851328] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-06. [2025-10-20T21:51:34.577905] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:51:34.651335] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-03. [2025-10-20T21:51:35.328021] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-06 (1.48 seconds). [2025-10-20T21:51:35.451368] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-04. [2025-10-20T21:51:36.251408] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-07. [2025-10-20T21:51:36.520590] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-03 (1.87 seconds). [2025-10-20T21:51:36.897990] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-04 (1.45 seconds). [2025-10-20T21:51:37.051368] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-05. [2025-10-20T21:51:37.851410] [INFO ] [REMOTE ] W: Fetching primary data for 2025-10. [2025-10-20T21:51:38.035756] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-07 (1.78 seconds). [2025-10-20T21:51:38.458911] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-05 (1.41 seconds). [2025-10-20T21:51:38.651462] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-06. [2025-10-20T21:51:39.451376] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-04. [2025-10-20T21:51:39.669796] [INFO ] [REMOTE ] W: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T21:51:39.678446] [INFO ] [REMOTE ] W: Full fetch complete (77526 records). [2025-10-20T21:51:39.678605] [DEBUG] [VALIDATOR ] W: Validating primary data. [2025-10-20T21:51:39.882219] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-06 (1.23 seconds). [2025-10-20T21:51:40.141415] [DEBUG] [VALIDATOR ] W: All expected trading days are present. [2025-10-20T21:51:40.141509] [DEBUG] [VALIDATOR ] W: Primary data validation passed. [2025-10-20T21:51:40.142781] [INFO ] [MAIN ] W: Fetched primary data is valid. [2025-10-20T21:51:40.142849] [DEBUG] [LOCAL ] W: Saving primary data. [2025-10-20T21:51:40.144331] [DEBUG] [LOCAL ] W: Trimmed 77526 records to 53421 (keeping 200 days of data) [2025-10-20T21:51:40.262344] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-07. [2025-10-20T21:51:40.456051] [DEBUG] [LOCAL ] W: Primary data saved (53421 records). [2025-10-20T21:51:40.456185] [INFO ] [MAIN ] W: Full history saved. [2025-10-20T21:51:40.456260] [INFO ] [MAIN ] Update Progress: 2268/2367 symbols completed. W took 42.6s. Estimated remaining: 12m 44s. Rate: 7.8 symbols/min [2025-10-20T21:51:40.630760] [INFO ] [MAIN ] WBX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:40.642771] [DEBUG] [LOCAL ] WBX: Loaded 10496 existing records from primary storage. [2025-10-20T21:51:40.643058] [INFO ] [MAIN ] WBX: Loaded 10496 existing records. [2025-10-20T21:51:40.825361] [WARN ] [MAIN ] WBX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:40.825585] [INFO ] [MAIN ] WBX: Wiping existing primary data before full refresh. [2025-10-20T21:51:40.825632] [INFO ] [LOCAL ] WBX: Starting deletion of primary data. [2025-10-20T21:51:40.825814] [DEBUG] [LOCAL ] WBX: Deleted primary data file. [2025-10-20T21:51:40.825873] [INFO ] [MAIN ] WBX: Starting full primary refresh. [2025-10-20T21:51:41.054066] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-08. [2025-10-20T21:51:41.196802] [INFO ] [REMOTE ] WBX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:41.228514] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:51:41.851352] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-01. [2025-10-20T21:51:41.942716] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-07 (1.68 seconds). [2025-10-20T21:51:42.575041] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T21:51:42.651401] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-09. [2025-10-20T21:51:43.080062] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T21:51:43.451445] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-08. [2025-10-20T21:51:44.040361] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T21:51:44.251407] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-05. [2025-10-20T21:51:44.576269] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T21:51:45.051369] [INFO ] [REMOTE ] WAL: Fetching primary data for 2025-10. [2025-10-20T21:51:45.821973] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T21:51:45.851359] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-06. [2025-10-20T21:51:46.661230] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-09. [2025-10-20T21:51:46.666023] [INFO ] [REMOTE ] WAL: Incremental fetch complete for 2025-10 (1.61 seconds). [2025-10-20T21:51:46.673896] [INFO ] [REMOTE ] WAL: Full fetch complete (67216 records). [2025-10-20T21:51:46.674034] [DEBUG] [VALIDATOR ] WAL: Validating primary data. [2025-10-20T21:51:47.073813] [DEBUG] [VALIDATOR ] WAL: All expected trading days are present. [2025-10-20T21:51:47.073908] [DEBUG] [VALIDATOR ] WAL: Primary data validation passed. [2025-10-20T21:51:47.075079] [INFO ] [MAIN ] WAL: Fetched primary data is valid. [2025-10-20T21:51:47.075146] [DEBUG] [LOCAL ] WAL: Saving primary data. [2025-10-20T21:51:47.076517] [DEBUG] [LOCAL ] WAL: Trimmed 67216 records to 45683 (keeping 200 days of data) [2025-10-20T21:51:47.353255] [DEBUG] [LOCAL ] WAL: Primary data saved (45683 records). [2025-10-20T21:51:47.353386] [INFO ] [MAIN ] WAL: Full history saved. [2025-10-20T21:51:47.353465] [INFO ] [MAIN ] Update Progress: 2269/2367 symbols completed. WAL took 31.4s. Estimated remaining: 12m 36s. Rate: 7.8 symbols/min [2025-10-20T21:51:47.454064] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-02. [2025-10-20T21:51:47.529321] [INFO ] [MAIN ] WCC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:47.560947] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:51:47.602833] [DEBUG] [LOCAL ] WCC: Loaded 39530 existing records from primary storage. [2025-10-20T21:51:47.603146] [INFO ] [MAIN ] WCC: Loaded 39530 existing records. [2025-10-20T21:51:47.779442] [WARN ] [MAIN ] WCC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:47.779668] [INFO ] [MAIN ] WCC: Wiping existing primary data before full refresh. [2025-10-20T21:51:47.779729] [INFO ] [LOCAL ] WCC: Starting deletion of primary data. [2025-10-20T21:51:47.779956] [DEBUG] [LOCAL ] WCC: Deleted primary data file. [2025-10-20T21:51:47.780011] [INFO ] [MAIN ] WCC: Starting full primary refresh. [2025-10-20T21:51:48.136987] [INFO ] [REMOTE ] WCC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:48.251376] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-07. [2025-10-20T21:51:48.289947] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T21:51:48.760467] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-02 (1.31 seconds). [2025-10-20T21:51:49.051393] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-03. [2025-10-20T21:51:49.851421] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-01. [2025-10-20T21:51:50.023748] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T21:51:50.307884] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-03 (1.26 seconds). [2025-10-20T21:51:50.651449] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-04. [2025-10-20T21:51:51.451408] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-08. [2025-10-20T21:51:51.482762] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:51:51.528868] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T21:51:52.251407] [INFO ] [REMOTE ] WAT: Fetching primary data for 2025-10. [2025-10-20T21:51:52.847622] [INFO ] [REMOTE ] WAT: Incremental fetch complete for 2025-10 (0.60 seconds). [2025-10-20T21:51:52.853468] [INFO ] [REMOTE ] WAT: Full fetch complete (50768 records). [2025-10-20T21:51:52.853593] [DEBUG] [VALIDATOR ] WAT: Validating primary data. [2025-10-20T21:51:53.057204] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-02. [2025-10-20T21:51:53.241871] [DEBUG] [VALIDATOR ] WAT: All expected trading days are present. [2025-10-20T21:51:53.242059] [DEBUG] [VALIDATOR ] WAT: Primary data validation passed. [2025-10-20T21:51:53.243309] [INFO ] [MAIN ] WAT: Fetched primary data is valid. [2025-10-20T21:51:53.253535] [DEBUG] [LOCAL ] WAT: Saving primary data. [2025-10-20T21:51:53.274777] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T21:51:53.284812] [DEBUG] [LOCAL ] WAT: Trimmed 50768 records to 35792 (keeping 200 days of data) [2025-10-20T21:51:53.499307] [DEBUG] [LOCAL ] WAT: Primary data saved (35792 records). [2025-10-20T21:51:53.499446] [INFO ] [MAIN ] WAT: Full history saved. [2025-10-20T21:51:53.499523] [INFO ] [MAIN ] Update Progress: 2270/2367 symbols completed. WAT took 29.3s. Estimated remaining: 12m 28s. Rate: 7.8 symbols/min [2025-10-20T21:51:53.666926] [INFO ] [MAIN ] WCN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:51:53.711253] [DEBUG] [LOCAL ] WCN: Loaded 48725 existing records from primary storage. [2025-10-20T21:51:53.711539] [INFO ] [MAIN ] WCN: Loaded 48725 existing records. [2025-10-20T21:51:53.859732] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-05. [2025-10-20T21:51:53.890470] [WARN ] [MAIN ] WCN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:51:53.890771] [INFO ] [MAIN ] WCN: Wiping existing primary data before full refresh. [2025-10-20T21:51:53.890823] [INFO ] [LOCAL ] WCN: Starting deletion of primary data. [2025-10-20T21:51:53.891158] [DEBUG] [LOCAL ] WCN: Deleted primary data file. [2025-10-20T21:51:53.891196] [INFO ] [MAIN ] WCN: Starting full primary refresh. [2025-10-20T21:51:54.249179] [INFO ] [REMOTE ] WCN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:51:54.651336] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-09. [2025-10-20T21:51:54.755496] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T21:51:54.895421] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-05 (1.04 seconds). [2025-10-20T21:51:55.451374] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-06. [2025-10-20T21:51:56.151589] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-09 (1.50 seconds). [2025-10-20T21:51:56.176524] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-06 (0.73 seconds). [2025-10-20T21:51:56.251352] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-07. [2025-10-20T21:51:56.935410] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-07 (0.68 seconds). [2025-10-20T21:51:57.051345] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-08. [2025-10-20T21:51:57.641123] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-08 (0.59 seconds). [2025-10-20T21:51:57.851397] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-03. [2025-10-20T21:51:58.651390] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-09. [2025-10-20T21:51:59.300322] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-09 (0.65 seconds). [2025-10-20T21:51:59.451356] [INFO ] [REMOTE ] WBS: Fetching primary data for 2025-10. [2025-10-20T21:51:59.518852] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T21:52:00.251369] [INFO ] [REMOTE ] WBX: Fetching primary data for 2025-10. [2025-10-20T21:52:00.983516] [INFO ] [REMOTE ] WBX: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T21:52:00.985385] [INFO ] [REMOTE ] WBX: Full fetch complete (22301 records). [2025-10-20T21:52:00.985462] [DEBUG] [VALIDATOR ] WBX: Validating primary data. [2025-10-20T21:52:01.051942] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-01. [2025-10-20T21:52:01.321863] [INFO ] [REMOTE ] WBS: Incremental fetch complete for 2025-10 (1.87 seconds). [2025-10-20T21:52:01.365457] [INFO ] [REMOTE ] WBS: Full fetch complete (70337 records). [2025-10-20T21:52:01.392227] [DEBUG] [VALIDATOR ] WBS: Validating primary data. [2025-10-20T21:52:01.392422] [DEBUG] [VALIDATOR ] WBX: All expected trading days are present. [2025-10-20T21:52:01.392582] [DEBUG] [VALIDATOR ] WBX: Primary data validation passed. [2025-10-20T21:52:01.393083] [INFO ] [MAIN ] WBX: Fetched primary data is valid. [2025-10-20T21:52:01.393213] [DEBUG] [LOCAL ] WBX: Saving primary data. [2025-10-20T21:52:01.471972] [DEBUG] [LOCAL ] WBX: Trimmed 22301 records to 10196 (keeping 200 days of data) [2025-10-20T21:52:01.639612] [DEBUG] [LOCAL ] WBX: Primary data saved (10196 records). [2025-10-20T21:52:01.658865] [INFO ] [MAIN ] WBX: Full history saved. [2025-10-20T21:52:01.688123] [INFO ] [MAIN ] Update Progress: 2271/2367 symbols completed. WBX took 21.2s. Estimated remaining: 12m 20s. Rate: 7.8 symbols/min [2025-10-20T21:52:01.855181] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-04. [2025-10-20T21:52:02.054401] [DEBUG] [VALIDATOR ] WBS: All expected trading days are present. [2025-10-20T21:52:02.055957] [DEBUG] [VALIDATOR ] WBS: Primary data validation passed. [2025-10-20T21:52:02.055885] [INFO ] [MAIN ] WD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:02.057315] [INFO ] [MAIN ] WBS: Fetched primary data is valid. [2025-10-20T21:52:02.057750] [DEBUG] [LOCAL ] WBS: Saving primary data. [2025-10-20T21:52:02.059749] [DEBUG] [LOCAL ] WBS: Trimmed 70337 records to 48845 (keeping 200 days of data) [2025-10-20T21:52:02.189391] [DEBUG] [LOCAL ] WD: Loaded 23354 existing records from primary storage. [2025-10-20T21:52:02.195336] [INFO ] [MAIN ] WD: Loaded 23354 existing records. [2025-10-20T21:52:02.523782] [WARN ] [MAIN ] WD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:02.524209] [INFO ] [MAIN ] WD: Wiping existing primary data before full refresh. [2025-10-20T21:52:02.529382] [INFO ] [LOCAL ] WD: Starting deletion of primary data. [2025-10-20T21:52:02.541239] [DEBUG] [LOCAL ] WD: Deleted primary data file. [2025-10-20T21:52:02.541564] [INFO ] [MAIN ] WD: Starting full primary refresh. [2025-10-20T21:52:02.631344] [DEBUG] [LOCAL ] WBS: Primary data saved (48845 records). [2025-10-20T21:52:02.655699] [INFO ] [MAIN ] WBS: Full history saved. [2025-10-20T21:52:02.658809] [INFO ] [MAIN ] Update Progress: 2272/2367 symbols completed. WBS took 35.0s. Estimated remaining: 12m 12s. Rate: 7.8 symbols/min [2025-10-20T21:52:03.020978] [INFO ] [MAIN ] WDH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:03.110995] [INFO ] [REMOTE ] WD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:03.111386] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-01. [2025-10-20T21:52:03.143347] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-01 (2.09 seconds). [2025-10-20T21:52:03.159413] [DEBUG] [LOCAL ] WDH: Loaded 18351 existing records from primary storage. [2025-10-20T21:52:03.160226] [INFO ] [MAIN ] WDH: Loaded 18351 existing records. [2025-10-20T21:52:03.337252] [WARN ] [MAIN ] WDH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:03.337470] [INFO ] [MAIN ] WDH: Wiping existing primary data before full refresh. [2025-10-20T21:52:03.337519] [INFO ] [LOCAL ] WDH: Starting deletion of primary data. [2025-10-20T21:52:03.337776] [DEBUG] [LOCAL ] WDH: Deleted primary data file. [2025-10-20T21:52:03.337836] [INFO ] [MAIN ] WDH: Starting full primary refresh. [2025-10-20T21:52:03.455288] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-02. [2025-10-20T21:52:03.701298] [INFO ] [REMOTE ] WDH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:03.729474] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T21:52:04.253668] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-05. [2025-10-20T21:52:04.255435] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-01 (1.14 seconds). [2025-10-20T21:52:05.051367] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-01. [2025-10-20T21:52:05.296590] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-02 (1.84 seconds). [2025-10-20T21:52:05.511246] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-01 (0.46 seconds). [2025-10-20T21:52:05.840069] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-05 (1.59 seconds). [2025-10-20T21:52:05.851332] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-02. [2025-10-20T21:52:06.651337] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-06. [2025-10-20T21:52:06.662104] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:52:07.451418] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-03. [2025-10-20T21:52:08.099353] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-06 (1.45 seconds). [2025-10-20T21:52:08.251359] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-02. [2025-10-20T21:52:09.051389] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-03. [2025-10-20T21:52:09.170962] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-03 (1.72 seconds). [2025-10-20T21:52:09.395614] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T21:52:09.653712] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T21:52:09.851422] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-04. [2025-10-20T21:52:10.651347] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-03. [2025-10-20T21:52:10.813348] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T21:52:11.451373] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-07. [2025-10-20T21:52:11.831291] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-03 (1.18 seconds). [2025-10-20T21:52:12.251384] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-05. [2025-10-20T21:52:12.733141] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T21:52:13.022036] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-05 (0.77 seconds). [2025-10-20T21:52:13.051341] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-06. [2025-10-20T21:52:13.851432] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-04. [2025-10-20T21:52:13.949837] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-06 (0.90 seconds). [2025-10-20T21:52:14.651383] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-04. [2025-10-20T21:52:15.451356] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-07. [2025-10-20T21:52:15.702942] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-04 (1.85 seconds). [2025-10-20T21:52:15.772972] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:52:16.251377] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-05. [2025-10-20T21:52:16.308149] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-07 (0.86 seconds). [2025-10-20T21:52:17.051424] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-05. [2025-10-20T21:52:17.851349] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-08. [2025-10-20T21:52:18.010887] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-05 (1.76 seconds). [2025-10-20T21:52:18.251217] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:52:18.651355] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-06. [2025-10-20T21:52:19.344193] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T21:52:19.451352] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-08. [2025-10-20T21:52:19.737797] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T21:52:20.149093] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-08 (0.70 seconds). [2025-10-20T21:52:20.251408] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-07. [2025-10-20T21:52:21.051445] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-09. [2025-10-20T21:52:21.526442] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-07 (1.28 seconds). [2025-10-20T21:52:21.851448] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-06. [2025-10-20T21:52:22.651446] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-08. [2025-10-20T21:52:22.680796] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-09 (1.63 seconds). [2025-10-20T21:52:23.451387] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-09. [2025-10-20T21:52:23.532456] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:52:23.723829] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:52:24.251369] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-09. [2025-10-20T21:52:24.418457] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T21:52:24.936408] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-09 (0.69 seconds). [2025-10-20T21:52:25.051385] [INFO ] [REMOTE ] WD: Fetching primary data for 2025-10. [2025-10-20T21:52:25.851366] [INFO ] [REMOTE ] WDH: Fetching primary data for 2025-10. [2025-10-20T21:52:25.981219] [INFO ] [REMOTE ] WD: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T21:52:25.984056] [INFO ] [REMOTE ] WD: Full fetch complete (33170 records). [2025-10-20T21:52:25.984129] [DEBUG] [VALIDATOR ] WD: Validating primary data. [2025-10-20T21:52:26.371460] [DEBUG] [VALIDATOR ] WD: All expected trading days are present. [2025-10-20T21:52:26.371539] [DEBUG] [VALIDATOR ] WD: Primary data validation passed. [2025-10-20T21:52:26.372297] [INFO ] [MAIN ] WD: Fetched primary data is valid. [2025-10-20T21:52:26.372359] [DEBUG] [LOCAL ] WD: Saving primary data. [2025-10-20T21:52:26.373123] [DEBUG] [LOCAL ] WD: Trimmed 33170 records to 22815 (keeping 200 days of data) [2025-10-20T21:52:26.507150] [DEBUG] [LOCAL ] WD: Primary data saved (22815 records). [2025-10-20T21:52:26.507271] [INFO ] [MAIN ] WD: Full history saved. [2025-10-20T21:52:26.507346] [INFO ] [MAIN ] Update Progress: 2273/2367 symbols completed. WD took 24.8s. Estimated remaining: 12m 5s. Rate: 7.8 symbols/min [2025-10-20T21:52:26.659618] [INFO ] [REMOTE ] WCC: Fetching primary data for 2025-10. [2025-10-20T21:52:26.679362] [INFO ] [MAIN ] WDI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:26.702376] [DEBUG] [LOCAL ] WDI: Loaded 21497 existing records from primary storage. [2025-10-20T21:52:26.702675] [INFO ] [MAIN ] WDI: Loaded 21497 existing records. [2025-10-20T21:52:26.890258] [WARN ] [MAIN ] WDI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:26.890473] [INFO ] [MAIN ] WDI: Wiping existing primary data before full refresh. [2025-10-20T21:52:26.890523] [INFO ] [LOCAL ] WDI: Starting deletion of primary data. [2025-10-20T21:52:26.890760] [DEBUG] [LOCAL ] WDI: Deleted primary data file. [2025-10-20T21:52:26.890820] [INFO ] [MAIN ] WDI: Starting full primary refresh. [2025-10-20T21:52:27.027085] [INFO ] [REMOTE ] WDH: Incremental fetch complete for 2025-10 (1.18 seconds). [2025-10-20T21:52:27.073022] [INFO ] [REMOTE ] WDH: Full fetch complete (23825 records). [2025-10-20T21:52:27.091740] [DEBUG] [VALIDATOR ] WDH: Validating primary data. [2025-10-20T21:52:27.456040] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-07. [2025-10-20T21:52:27.478218] [INFO ] [REMOTE ] WDI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:27.665782] [DEBUG] [VALIDATOR ] WDH: All expected trading days are present. [2025-10-20T21:52:27.665860] [DEBUG] [VALIDATOR ] WDH: Primary data validation passed. [2025-10-20T21:52:27.666339] [INFO ] [MAIN ] WDH: Fetched primary data is valid. [2025-10-20T21:52:27.666376] [DEBUG] [LOCAL ] WDH: Saving primary data. [2025-10-20T21:52:27.667030] [DEBUG] [LOCAL ] WDH: Trimmed 23825 records to 18081 (keeping 200 days of data) [2025-10-20T21:52:27.775617] [DEBUG] [LOCAL ] WDH: Primary data saved (18081 records). [2025-10-20T21:52:27.775876] [INFO ] [MAIN ] WDH: Full history saved. [2025-10-20T21:52:27.776018] [INFO ] [MAIN ] Update Progress: 2274/2367 symbols completed. WDH took 25.1s. Estimated remaining: 11m 57s. Rate: 7.8 symbols/min [2025-10-20T21:52:27.917255] [INFO ] [REMOTE ] WCC: Incremental fetch complete for 2025-10 (1.26 seconds). [2025-10-20T21:52:27.943950] [INFO ] [REMOTE ] WCC: Full fetch complete (57382 records). [2025-10-20T21:52:27.962010] [DEBUG] [VALIDATOR ] WCC: Validating primary data. [2025-10-20T21:52:28.064245] [INFO ] [MAIN ] WDS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:28.260964] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-01. [2025-10-20T21:52:28.343211] [DEBUG] [LOCAL ] WDS: Loaded 43811 existing records from primary storage. [2025-10-20T21:52:28.385947] [INFO ] [MAIN ] WDS: Loaded 43811 existing records. [2025-10-20T21:52:28.499160] [DEBUG] [VALIDATOR ] WCC: All expected trading days are present. [2025-10-20T21:52:28.504377] [DEBUG] [VALIDATOR ] WCC: Primary data validation passed. [2025-10-20T21:52:28.516699] [INFO ] [MAIN ] WCC: Fetched primary data is valid. [2025-10-20T21:52:28.524891] [DEBUG] [LOCAL ] WCC: Saving primary data. [2025-10-20T21:52:28.579506] [DEBUG] [LOCAL ] WCC: Trimmed 57382 records to 38838 (keeping 200 days of data) [2025-10-20T21:52:28.606775] [WARN ] [MAIN ] WDS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:28.607104] [INFO ] [MAIN ] WDS: Wiping existing primary data before full refresh. [2025-10-20T21:52:28.607366] [INFO ] [LOCAL ] WDS: Starting deletion of primary data. [2025-10-20T21:52:28.607561] [DEBUG] [LOCAL ] WDS: Deleted primary data file. [2025-10-20T21:52:28.607602] [INFO ] [MAIN ] WDS: Starting full primary refresh. [2025-10-20T21:52:29.207810] [INFO ] [REMOTE ] WDS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:29.208310] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-01. [2025-10-20T21:52:29.266438] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-01 (1.01 seconds). [2025-10-20T21:52:29.288563] [DEBUG] [LOCAL ] WCC: Primary data saved (38838 records). [2025-10-20T21:52:29.290012] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T21:52:29.289127] [INFO ] [MAIN ] WCC: Full history saved. [2025-10-20T21:52:29.300434] [INFO ] [MAIN ] Update Progress: 2275/2367 symbols completed. WCC took 41.9s. Estimated remaining: 11m 50s. Rate: 7.8 symbols/min [2025-10-20T21:52:29.476769] [INFO ] [MAIN ] WEA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:29.486468] [DEBUG] [LOCAL ] WEA: Loaded 7901 existing records from primary storage. [2025-10-20T21:52:29.486695] [INFO ] [MAIN ] WEA: Loaded 7901 existing records. [2025-10-20T21:52:29.662156] [WARN ] [MAIN ] WEA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:29.662343] [INFO ] [MAIN ] WEA: Wiping existing primary data before full refresh. [2025-10-20T21:52:29.662408] [INFO ] [LOCAL ] WEA: Starting deletion of primary data. [2025-10-20T21:52:29.662548] [DEBUG] [LOCAL ] WEA: Deleted primary data file. [2025-10-20T21:52:29.662592] [INFO ] [MAIN ] WEA: Starting full primary refresh. [2025-10-20T21:52:29.857709] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-02. [2025-10-20T21:52:30.020241] [INFO ] [REMOTE ] WEA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:30.651392] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-08. [2025-10-20T21:52:30.763042] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:52:30.801631] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-02 (0.94 seconds). [2025-10-20T21:52:31.451436] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-02. [2025-10-20T21:52:32.249900] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-08 (1.60 seconds). [2025-10-20T21:52:32.253734] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-01. [2025-10-20T21:52:32.643775] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-01 (0.39 seconds). [2025-10-20T21:52:32.979744] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-02 (1.53 seconds). [2025-10-20T21:52:33.051381] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-03. [2025-10-20T21:52:33.851385] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-02. [2025-10-20T21:52:34.205583] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-02 (0.35 seconds). [2025-10-20T21:52:34.651344] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-09. [2025-10-20T21:52:34.713424] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T21:52:35.451420] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-04. [2025-10-20T21:52:36.251339] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-03. [2025-10-20T21:52:36.443326] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-09 (1.79 seconds). [2025-10-20T21:52:36.988901] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-03 (0.74 seconds). [2025-10-20T21:52:37.051337] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-04. [2025-10-20T21:52:37.140189] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T21:52:37.851362] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-03. [2025-10-20T21:52:38.175915] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-04 (1.12 seconds). [2025-10-20T21:52:38.433634] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-03 (0.58 seconds). [2025-10-20T21:52:38.651390] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-05. [2025-10-20T21:52:39.451359] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-05. [2025-10-20T21:52:39.746602] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-05 (1.10 seconds). [2025-10-20T21:52:40.251424] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-04. [2025-10-20T21:52:40.959411] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-04 (0.71 seconds). [2025-10-20T21:52:41.051381] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-06. [2025-10-20T21:52:41.084992] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:52:41.851403] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-06. [2025-10-20T21:52:42.088129] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-06 (1.04 seconds). [2025-10-20T21:52:42.651376] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-05. [2025-10-20T21:52:43.386789] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T21:52:43.451397] [INFO ] [REMOTE ] WCN: Fetching primary data for 2025-10. [2025-10-20T21:52:43.602229] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T21:52:44.251381] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-07. [2025-10-20T21:52:45.052130] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-07. [2025-10-20T21:52:45.054128] [INFO ] [REMOTE ] WCN: Incremental fetch complete for 2025-10 (1.60 seconds). [2025-10-20T21:52:45.061491] [INFO ] [REMOTE ] WCN: Full fetch complete (69447 records). [2025-10-20T21:52:45.061631] [DEBUG] [VALIDATOR ] WCN: Validating primary data. [2025-10-20T21:52:45.454455] [DEBUG] [VALIDATOR ] WCN: All expected trading days are present. [2025-10-20T21:52:45.454532] [DEBUG] [VALIDATOR ] WCN: Primary data validation passed. [2025-10-20T21:52:45.455605] [INFO ] [MAIN ] WCN: Fetched primary data is valid. [2025-10-20T21:52:45.455689] [DEBUG] [LOCAL ] WCN: Saving primary data. [2025-10-20T21:52:45.457052] [DEBUG] [LOCAL ] WCN: Trimmed 69447 records to 47917 (keeping 200 days of data) [2025-10-20T21:52:45.745514] [DEBUG] [LOCAL ] WCN: Primary data saved (47917 records). [2025-10-20T21:52:45.745636] [INFO ] [MAIN ] WCN: Full history saved. [2025-10-20T21:52:45.745749] [INFO ] [MAIN ] Update Progress: 2276/2367 symbols completed. WCN took 52.2s. Estimated remaining: 11m 42s. Rate: 7.8 symbols/min [2025-10-20T21:52:45.855442] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-06. [2025-10-20T21:52:45.920900] [INFO ] [MAIN ] WEAV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:52:45.947205] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T21:52:45.992919] [DEBUG] [LOCAL ] WEAV: Loaded 42907 existing records from primary storage. [2025-10-20T21:52:45.993201] [INFO ] [MAIN ] WEAV: Loaded 42907 existing records. [2025-10-20T21:52:46.171035] [WARN ] [MAIN ] WEAV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:52:46.171347] [INFO ] [MAIN ] WEAV: Wiping existing primary data before full refresh. [2025-10-20T21:52:46.171392] [INFO ] [LOCAL ] WEAV: Starting deletion of primary data. [2025-10-20T21:52:46.171603] [DEBUG] [LOCAL ] WEAV: Deleted primary data file. [2025-10-20T21:52:46.171697] [INFO ] [MAIN ] WEAV: Starting full primary refresh. [2025-10-20T21:52:46.324609] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T21:52:46.541501] [INFO ] [REMOTE ] WEAV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:52:46.651355] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-08. [2025-10-20T21:52:46.712325] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-06 (0.86 seconds). [2025-10-20T21:52:47.452440] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-01. [2025-10-20T21:52:48.098435] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T21:52:48.251421] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-08. [2025-10-20T21:52:48.983480] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:52:49.051337] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-02. [2025-10-20T21:52:49.271185] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-08 (1.02 seconds). [2025-10-20T21:52:49.851398] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-09. [2025-10-20T21:52:50.611700] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-02 (1.56 seconds). [2025-10-20T21:52:50.651339] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-03. [2025-10-20T21:52:51.392043] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-09 (1.54 seconds). [2025-10-20T21:52:51.451344] [INFO ] [REMOTE ] WDS: Fetching primary data for 2025-10. [2025-10-20T21:52:52.181695] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T21:52:52.251329] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-04. [2025-10-20T21:52:52.592338] [INFO ] [REMOTE ] WDS: Incremental fetch complete for 2025-10 (1.14 seconds). [2025-10-20T21:52:52.598708] [INFO ] [REMOTE ] WDS: Full fetch complete (64198 records). [2025-10-20T21:52:52.598811] [DEBUG] [VALIDATOR ] WDS: Validating primary data. [2025-10-20T21:52:52.973838] [DEBUG] [VALIDATOR ] WDS: All expected trading days are present. [2025-10-20T21:52:52.973919] [DEBUG] [VALIDATOR ] WDS: Primary data validation passed. [2025-10-20T21:52:52.974969] [INFO ] [MAIN ] WDS: Fetched primary data is valid. [2025-10-20T21:52:52.975030] [DEBUG] [LOCAL ] WDS: Saving primary data. [2025-10-20T21:52:52.976210] [DEBUG] [LOCAL ] WDS: Trimmed 64198 records to 43142 (keeping 200 days of data) [2025-10-20T21:52:53.062987] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-07. [2025-10-20T21:52:53.233914] [DEBUG] [LOCAL ] WDS: Primary data saved (43142 records). [2025-10-20T21:52:53.234045] [INFO ] [MAIN ] WDS: Full history saved. [2025-10-20T21:52:53.234126] [INFO ] [MAIN ] Update Progress: 2277/2367 symbols completed. WDS took 25.5s. Estimated remaining: 11m 34s. Rate: 7.8 symbols/min [2025-10-20T21:52:53.418238] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T21:52:53.851373] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-05. [2025-10-20T21:52:53.868985] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-07 (0.81 seconds). [2025-10-20T21:52:54.651445] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-08. [2025-10-20T21:52:55.327809] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T21:52:55.376807] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-05 (1.53 seconds). [2025-10-20T21:52:55.451349] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-06. [2025-10-20T21:52:56.251380] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-09. [2025-10-20T21:52:56.891322] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-06 (1.44 seconds). [2025-10-20T21:52:57.014345] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T21:52:57.051374] [INFO ] [REMOTE ] WEA: Fetching primary data for 2025-10. [2025-10-20T21:52:57.732840] [INFO ] [REMOTE ] WEA: Incremental fetch complete for 2025-10 (0.68 seconds). [2025-10-20T21:52:57.734631] [INFO ] [REMOTE ] WEA: Full fetch complete (10092 records). [2025-10-20T21:52:57.734724] [DEBUG] [VALIDATOR ] WEA: Validating primary data. [2025-10-20T21:52:57.856486] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-07. [2025-10-20T21:52:58.088690] [DEBUG] [VALIDATOR ] WEA: All expected trading days are present. [2025-10-20T21:52:58.088774] [DEBUG] [VALIDATOR ] WEA: Primary data validation passed. [2025-10-20T21:52:58.089015] [INFO ] [MAIN ] WEA: Fetched primary data is valid. [2025-10-20T21:52:58.089053] [DEBUG] [LOCAL ] WEA: Saving primary data. [2025-10-20T21:52:58.089477] [DEBUG] [LOCAL ] WEA: Trimmed 10092 records to 7826 (keeping 200 days of data) [2025-10-20T21:52:58.134564] [DEBUG] [LOCAL ] WEA: Primary data saved (7826 records). [2025-10-20T21:52:58.134696] [INFO ] [MAIN ] WEA: Full history saved. [2025-10-20T21:52:58.134818] [INFO ] [MAIN ] Update Progress: 2278/2367 symbols completed. WEA took 28.8s. Estimated remaining: 11m 27s. Rate: 7.8 symbols/min [2025-10-20T21:52:58.651376] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-09. [2025-10-20T21:52:59.603886] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-07 (1.75 seconds). [2025-10-20T21:52:59.613387] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-08. [2025-10-20T21:52:59.659671] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-09 (1.01 seconds). [2025-10-20T21:53:00.251368] [INFO ] [REMOTE ] WDI: Fetching primary data for 2025-10. [2025-10-20T21:53:01.318865] [INFO ] [REMOTE ] WDI: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T21:53:01.322209] [INFO ] [REMOTE ] WDI: Full fetch complete (30324 records). [2025-10-20T21:53:01.322286] [DEBUG] [VALIDATOR ] WDI: Validating primary data. [2025-10-20T21:53:01.529828] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T21:53:01.623394] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-09. [2025-10-20T21:53:01.753378] [DEBUG] [VALIDATOR ] WDI: All expected trading days are present. [2025-10-20T21:53:01.753463] [DEBUG] [VALIDATOR ] WDI: Primary data validation passed. [2025-10-20T21:53:01.754118] [INFO ] [MAIN ] WDI: Fetched primary data is valid. [2025-10-20T21:53:01.754179] [DEBUG] [LOCAL ] WDI: Saving primary data. [2025-10-20T21:53:01.754873] [DEBUG] [LOCAL ] WDI: Trimmed 30324 records to 21311 (keeping 200 days of data) [2025-10-20T21:53:01.879446] [DEBUG] [LOCAL ] WDI: Primary data saved (21311 records). [2025-10-20T21:53:01.879579] [INFO ] [MAIN ] WDI: Full history saved. [2025-10-20T21:53:01.879677] [INFO ] [MAIN ] Update Progress: 2279/2367 symbols completed. WDI took 35.4s. Estimated remaining: 11m 19s. Rate: 7.8 symbols/min [2025-10-20T21:53:03.176952] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:53:03.186449] [INFO ] [REMOTE ] WEAV: Fetching primary data for 2025-10. [2025-10-20T21:53:04.647361] [INFO ] [REMOTE ] WEAV: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T21:53:04.654202] [INFO ] [REMOTE ] WEAV: Full fetch complete (61855 records). [2025-10-20T21:53:04.654327] [DEBUG] [VALIDATOR ] WEAV: Validating primary data. [2025-10-20T21:53:05.052221] [DEBUG] [VALIDATOR ] WEAV: All expected trading days are present. [2025-10-20T21:53:05.052344] [DEBUG] [VALIDATOR ] WEAV: Primary data validation passed. [2025-10-20T21:53:05.053381] [INFO ] [MAIN ] WEAV: Fetched primary data is valid. [2025-10-20T21:53:05.053445] [DEBUG] [LOCAL ] WEAV: Saving primary data. [2025-10-20T21:53:05.054717] [DEBUG] [LOCAL ] WEAV: Trimmed 61855 records to 42365 (keeping 200 days of data) [2025-10-20T21:53:05.284060] [DEBUG] [LOCAL ] WEAV: Primary data saved (42365 records). [2025-10-20T21:53:05.284183] [INFO ] [MAIN ] WEAV: Full history saved. [2025-10-20T21:53:05.284257] [INFO ] [MAIN ] Update Progress: 2280/2367 symbols completed. WEAV took 19.5s. Estimated remaining: 11m 11s. Rate: 7.8 symbols/min [2025-10-20T21:53:05.905992] [INFO ] [MAIN ] WEX: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:05.962573] [INFO ] [MAIN ] WES: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:06.016519] [INFO ] [MAIN ] WEC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:06.042519] [INFO ] [MAIN ] WELL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:06.074637] [DEBUG] [LOCAL ] WEX: Loaded 33480 existing records from primary storage. [2025-10-20T21:53:06.075184] [INFO ] [MAIN ] WEX: Loaded 33480 existing records. [2025-10-20T21:53:06.278923] [DEBUG] [LOCAL ] WES: Loaded 49260 existing records from primary storage. [2025-10-20T21:53:06.336135] [WARN ] [MAIN ] WEX: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:06.336729] [INFO ] [MAIN ] WES: Loaded 49260 existing records. [2025-10-20T21:53:06.364512] [DEBUG] [LOCAL ] WELL: Loaded 53089 existing records from primary storage. [2025-10-20T21:53:06.364618] [INFO ] [MAIN ] WEX: Wiping existing primary data before full refresh. [2025-10-20T21:53:06.386173] [INFO ] [LOCAL ] WEX: Starting deletion of primary data. [2025-10-20T21:53:06.402897] [DEBUG] [LOCAL ] WEC: Loaded 52934 existing records from primary storage. [2025-10-20T21:53:06.402039] [INFO ] [MAIN ] WELL: Loaded 53089 existing records. [2025-10-20T21:53:06.415543] [INFO ] [MAIN ] WEC: Loaded 52934 existing records. [2025-10-20T21:53:06.451167] [DEBUG] [LOCAL ] WEX: Deleted primary data file. [2025-10-20T21:53:06.461165] [INFO ] [MAIN ] WEX: Starting full primary refresh. [2025-10-20T21:53:06.931992] [WARN ] [MAIN ] WES: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:06.986075] [WARN ] [MAIN ] WELL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:07.033155] [INFO ] [MAIN ] WELL: Wiping existing primary data before full refresh. [2025-10-20T21:53:07.041312] [INFO ] [MAIN ] WES: Wiping existing primary data before full refresh. [2025-10-20T21:53:07.048585] [INFO ] [LOCAL ] WELL: Starting deletion of primary data. [2025-10-20T21:53:07.063857] [INFO ] [LOCAL ] WES: Starting deletion of primary data. [2025-10-20T21:53:07.079626] [DEBUG] [LOCAL ] WES: Deleted primary data file. [2025-10-20T21:53:07.079521] [WARN ] [MAIN ] WEC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:07.102917] [INFO ] [MAIN ] WEC: Wiping existing primary data before full refresh. [2025-10-20T21:53:07.109717] [DEBUG] [LOCAL ] WELL: Deleted primary data file. [2025-10-20T21:53:07.116617] [INFO ] [MAIN ] WES: Starting full primary refresh. [2025-10-20T21:53:07.125982] [INFO ] [MAIN ] WELL: Starting full primary refresh. [2025-10-20T21:53:07.134691] [INFO ] [LOCAL ] WEC: Starting deletion of primary data. [2025-10-20T21:53:07.240440] [DEBUG] [LOCAL ] WEC: Deleted primary data file. [2025-10-20T21:53:07.265569] [INFO ] [MAIN ] WEC: Starting full primary refresh. [2025-10-20T21:53:07.635203] [INFO ] [REMOTE ] WEX: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:07.684006] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-01. [2025-10-20T21:53:08.241853] [INFO ] [REMOTE ] WELL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:08.263034] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-01. [2025-10-20T21:53:08.384405] [INFO ] [REMOTE ] WES: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:08.399982] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-01. [2025-10-20T21:53:08.399192] [INFO ] [REMOTE ] WEC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:08.401348] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-01. [2025-10-20T21:53:09.243441] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-01 (1.56 seconds). [2025-10-20T21:53:09.249950] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-02. [2025-10-20T21:53:09.804134] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:53:09.814449] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-02. [2025-10-20T21:53:10.065729] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-01 (1.66 seconds). [2025-10-20T21:53:10.075952] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-02. [2025-10-20T21:53:10.170523] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-01 (1.77 seconds). [2025-10-20T21:53:10.181038] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-02. [2025-10-20T21:53:10.424507] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-02 (1.17 seconds). [2025-10-20T21:53:10.432431] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-03. [2025-10-20T21:53:11.565347] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-02 (1.75 seconds). [2025-10-20T21:53:11.575517] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-03. [2025-10-20T21:53:11.814328] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:53:11.823381] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-03. [2025-10-20T21:53:11.859720] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T21:53:11.869549] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-03. [2025-10-20T21:53:12.031166] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-03 (1.60 seconds). [2025-10-20T21:53:12.251361] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-04. [2025-10-20T21:53:13.196588] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-03 (1.62 seconds). [2025-10-20T21:53:13.208950] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-04. [2025-10-20T21:53:13.262083] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-03 (1.44 seconds). [2025-10-20T21:53:13.768253] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-04 (1.52 seconds). [2025-10-20T21:53:13.783174] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T21:53:13.851448] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-04. [2025-10-20T21:53:14.651479] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-04. [2025-10-20T21:53:15.179563] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-04 (1.97 seconds). [2025-10-20T21:53:15.451406] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-05. [2025-10-20T21:53:15.644110] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-04 (1.79 seconds). [2025-10-20T21:53:16.025310] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-04 (1.37 seconds). [2025-10-20T21:53:16.251453] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-05. [2025-10-20T21:53:16.589723] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T21:53:17.051398] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-05. [2025-10-20T21:53:17.851398] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-06. [2025-10-20T21:53:18.251346] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-05 (2.00 seconds). [2025-10-20T21:53:18.651399] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-06. [2025-10-20T21:53:18.828915] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T21:53:19.203177] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T21:53:19.451404] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-06. [2025-10-20T21:53:20.151999] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T21:53:20.251340] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-07. [2025-10-20T21:53:20.740614] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T21:53:21.051407] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-07. [2025-10-20T21:53:21.851406] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-07. [2025-10-20T21:53:22.281387] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-07 (2.03 seconds). [2025-10-20T21:53:22.469275] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-07 (1.42 seconds). [2025-10-20T21:53:22.651385] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-08. [2025-10-20T21:53:23.451433] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-05. [2025-10-20T21:53:23.625402] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T21:53:24.039394] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T21:53:24.251435] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-08. [2025-10-20T21:53:24.837746] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-05 (1.39 seconds). [2025-10-20T21:53:25.051402] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-06. [2025-10-20T21:53:25.851372] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-08. [2025-10-20T21:53:26.094404] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T21:53:26.651360] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-09. [2025-10-20T21:53:26.694090] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-06 (1.64 seconds). [2025-10-20T21:53:27.339729] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T21:53:27.451345] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-09. [2025-10-20T21:53:28.251363] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-07. [2025-10-20T21:53:28.405229] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-09 (1.75 seconds). [2025-10-20T21:53:28.824567] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T21:53:29.051369] [INFO ] [REMOTE ] WEC: Fetching primary data for 2025-10. [2025-10-20T21:53:29.806028] [INFO ] [REMOTE ] WEC: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:53:29.813880] [INFO ] [REMOTE ] WEC: Full fetch complete (75847 records). [2025-10-20T21:53:29.814027] [DEBUG] [VALIDATOR ] WEC: Validating primary data. [2025-10-20T21:53:29.856458] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-09. [2025-10-20T21:53:30.138912] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:53:30.253489] [DEBUG] [VALIDATOR ] WEC: All expected trading days are present. [2025-10-20T21:53:30.253580] [DEBUG] [VALIDATOR ] WEC: Primary data validation passed. [2025-10-20T21:53:30.254726] [INFO ] [MAIN ] WEC: Fetched primary data is valid. [2025-10-20T21:53:30.254788] [DEBUG] [LOCAL ] WEC: Saving primary data. [2025-10-20T21:53:30.256105] [DEBUG] [LOCAL ] WEC: Trimmed 75847 records to 52188 (keeping 200 days of data) [2025-10-20T21:53:30.564522] [DEBUG] [LOCAL ] WEC: Primary data saved (52188 records). [2025-10-20T21:53:30.564672] [INFO ] [MAIN ] WEC: Full history saved. [2025-10-20T21:53:30.564766] [INFO ] [MAIN ] Update Progress: 2281/2367 symbols completed. WEC took 25.3s. Estimated remaining: 11m 4s. Rate: 7.8 symbols/min [2025-10-20T21:53:30.656597] [INFO ] [REMOTE ] WELL: Fetching primary data for 2025-10. [2025-10-20T21:53:30.739127] [INFO ] [MAIN ] WF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:30.751150] [DEBUG] [LOCAL ] WF: Loaded 11094 existing records from primary storage. [2025-10-20T21:53:30.751339] [INFO ] [MAIN ] WF: Loaded 11094 existing records. [2025-10-20T21:53:30.924360] [WARN ] [MAIN ] WF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:30.924732] [INFO ] [MAIN ] WF: Wiping existing primary data before full refresh. [2025-10-20T21:53:30.924965] [INFO ] [LOCAL ] WF: Starting deletion of primary data. [2025-10-20T21:53:30.925137] [DEBUG] [LOCAL ] WF: Deleted primary data file. [2025-10-20T21:53:30.925216] [INFO ] [MAIN ] WF: Starting full primary refresh. [2025-10-20T21:53:31.293847] [INFO ] [REMOTE ] WF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:31.295487] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-09 (1.44 seconds). [2025-10-20T21:53:31.324299] [INFO ] [REMOTE ] WELL: Incremental fetch complete for 2025-10 (0.67 seconds). [2025-10-20T21:53:31.331946] [INFO ] [REMOTE ] WELL: Full fetch complete (75996 records). [2025-10-20T21:53:31.332076] [DEBUG] [VALIDATOR ] WELL: Validating primary data. [2025-10-20T21:53:31.456456] [INFO ] [REMOTE ] WEX: Fetching primary data for 2025-10. [2025-10-20T21:53:31.722977] [DEBUG] [VALIDATOR ] WELL: All expected trading days are present. [2025-10-20T21:53:31.723058] [DEBUG] [VALIDATOR ] WELL: Primary data validation passed. [2025-10-20T21:53:31.724180] [INFO ] [MAIN ] WELL: Fetched primary data is valid. [2025-10-20T21:53:31.724242] [DEBUG] [LOCAL ] WELL: Saving primary data. [2025-10-20T21:53:31.725632] [DEBUG] [LOCAL ] WELL: Trimmed 75996 records to 52322 (keeping 200 days of data) [2025-10-20T21:53:32.041265] [DEBUG] [LOCAL ] WELL: Primary data saved (52322 records). [2025-10-20T21:53:32.041403] [INFO ] [MAIN ] WELL: Full history saved. [2025-10-20T21:53:32.041478] [INFO ] [MAIN ] Update Progress: 2282/2367 symbols completed. WELL took 26.7s. Estimated remaining: 10m 56s. Rate: 7.8 symbols/min [2025-10-20T21:53:32.209449] [INFO ] [MAIN ] WFC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:32.209617] [DEBUG] [LOCAL ] WFC: No existing data file found in primary storage. [2025-10-20T21:53:32.209685] [WARN ] [MAIN ] WFC: No existing data found, performing full refresh instead. [2025-10-20T21:53:32.209827] [INFO ] [MAIN ] WFC: Wiping existing primary data before full refresh. [2025-10-20T21:53:32.209889] [INFO ] [LOCAL ] WFC: Starting deletion of primary data. [2025-10-20T21:53:32.209935] [DEBUG] [LOCAL ] WFC: No primary data file found to delete. [2025-10-20T21:53:32.209968] [INFO ] [MAIN ] WFC: Starting full primary refresh. [2025-10-20T21:53:32.256548] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-08. [2025-10-20T21:53:32.510784] [INFO ] [REMOTE ] WEX: Incremental fetch complete for 2025-10 (1.05 seconds). [2025-10-20T21:53:32.539311] [INFO ] [REMOTE ] WEX: Full fetch complete (50470 records). [2025-10-20T21:53:32.567166] [DEBUG] [VALIDATOR ] WEX: Validating primary data. [2025-10-20T21:53:32.663375] [INFO ] [REMOTE ] WFC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:32.972562] [DEBUG] [VALIDATOR ] WEX: All expected trading days are present. [2025-10-20T21:53:32.972638] [DEBUG] [VALIDATOR ] WEX: Primary data validation passed. [2025-10-20T21:53:32.973479] [INFO ] [MAIN ] WEX: Fetched primary data is valid. [2025-10-20T21:53:32.973520] [DEBUG] [LOCAL ] WEX: Saving primary data. [2025-10-20T21:53:32.974472] [DEBUG] [LOCAL ] WEX: Trimmed 50470 records to 32742 (keeping 200 days of data) [2025-10-20T21:53:33.057712] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-01. [2025-10-20T21:53:33.165433] [DEBUG] [LOCAL ] WEX: Primary data saved (32742 records). [2025-10-20T21:53:33.165563] [INFO ] [MAIN ] WEX: Full history saved. [2025-10-20T21:53:33.165639] [INFO ] [MAIN ] Update Progress: 2283/2367 symbols completed. WEX took 27.9s. Estimated remaining: 10m 48s. Rate: 7.8 symbols/min [2025-10-20T21:53:33.330420] [INFO ] [MAIN ] WFG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:33.353830] [DEBUG] [LOCAL ] WFG: Loaded 25178 existing records from primary storage. [2025-10-20T21:53:33.354078] [INFO ] [MAIN ] WFG: Loaded 25178 existing records. [2025-10-20T21:53:33.529047] [WARN ] [MAIN ] WFG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:33.529252] [INFO ] [MAIN ] WFG: Wiping existing primary data before full refresh. [2025-10-20T21:53:33.529297] [INFO ] [LOCAL ] WFG: Starting deletion of primary data. [2025-10-20T21:53:33.529481] [DEBUG] [LOCAL ] WFG: Deleted primary data file. [2025-10-20T21:53:33.529535] [INFO ] [MAIN ] WFG: Starting full primary refresh. [2025-10-20T21:53:33.865762] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-01. [2025-10-20T21:53:33.888830] [INFO ] [REMOTE ] WFG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:33.897932] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T21:53:34.076692] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-08 (1.82 seconds). [2025-10-20T21:53:34.651405] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-02. [2025-10-20T21:53:35.449368] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-02 (0.80 seconds). [2025-10-20T21:53:35.451990] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-01. [2025-10-20T21:53:35.876876] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-01 (2.01 seconds). [2025-10-20T21:53:36.251426] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-09. [2025-10-20T21:53:36.613882] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T21:53:37.051429] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-03. [2025-10-20T21:53:37.775948] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-03 (0.72 seconds). [2025-10-20T21:53:37.851407] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-04. [2025-10-20T21:53:37.982037] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T21:53:38.651406] [INFO ] [REMOTE ] WES: Fetching primary data for 2025-10. [2025-10-20T21:53:38.840060] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-04 (0.99 seconds). [2025-10-20T21:53:39.451385] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-02. [2025-10-20T21:53:40.245484] [INFO ] [REMOTE ] WES: Incremental fetch complete for 2025-10 (1.59 seconds). [2025-10-20T21:53:40.252454] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-02. [2025-10-20T21:53:40.254038] [INFO ] [REMOTE ] WES: Full fetch complete (70595 records). [2025-10-20T21:53:40.254121] [DEBUG] [VALIDATOR ] WES: Validating primary data. [2025-10-20T21:53:40.632732] [DEBUG] [VALIDATOR ] WES: All expected trading days are present. [2025-10-20T21:53:40.632835] [DEBUG] [VALIDATOR ] WES: Primary data validation passed. [2025-10-20T21:53:40.633844] [INFO ] [MAIN ] WES: Fetched primary data is valid. [2025-10-20T21:53:40.633905] [DEBUG] [LOCAL ] WES: Saving primary data. [2025-10-20T21:53:40.635148] [DEBUG] [LOCAL ] WES: Trimmed 70595 records to 48673 (keeping 200 days of data) [2025-10-20T21:53:40.939390] [DEBUG] [LOCAL ] WES: Primary data saved (48673 records). [2025-10-20T21:53:40.939540] [INFO ] [MAIN ] WES: Full history saved. [2025-10-20T21:53:40.939624] [INFO ] [MAIN ] Update Progress: 2284/2367 symbols completed. WES took 35.7s. Estimated remaining: 10m 40s. Rate: 7.8 symbols/min [2025-10-20T21:53:41.056468] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-05. [2025-10-20T21:53:41.120336] [INFO ] [MAIN ] WGO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:41.156266] [DEBUG] [LOCAL ] WGO: Loaded 38653 existing records from primary storage. [2025-10-20T21:53:41.156610] [INFO ] [MAIN ] WGO: Loaded 38653 existing records. [2025-10-20T21:53:41.337945] [WARN ] [MAIN ] WGO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:41.338432] [INFO ] [MAIN ] WGO: Wiping existing primary data before full refresh. [2025-10-20T21:53:41.338516] [INFO ] [LOCAL ] WGO: Starting deletion of primary data. [2025-10-20T21:53:41.338741] [DEBUG] [LOCAL ] WGO: Deleted primary data file. [2025-10-20T21:53:41.338797] [INFO ] [MAIN ] WGO: Starting full primary refresh. [2025-10-20T21:53:41.631624] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-02 (2.18 seconds). [2025-10-20T21:53:41.717671] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-02 (1.47 seconds). [2025-10-20T21:53:41.761549] [INFO ] [REMOTE ] WGO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:41.851368] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-03. [2025-10-20T21:53:41.898396] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-05 (0.84 seconds). [2025-10-20T21:53:42.651461] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-01. [2025-10-20T21:53:43.451404] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-06. [2025-10-20T21:53:44.251420] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-03. [2025-10-20T21:53:44.305424] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-03 (2.45 seconds). [2025-10-20T21:53:44.327103] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-06 (0.88 seconds). [2025-10-20T21:53:44.362547] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T21:53:45.051357] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-02. [2025-10-20T21:53:45.491732] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T21:53:45.851452] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-04. [2025-10-20T21:53:46.491555] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T21:53:46.651384] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-03. [2025-10-20T21:53:47.451415] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-07. [2025-10-20T21:53:48.142532] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-04 (2.29 seconds). [2025-10-20T21:53:48.238624] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T21:53:48.251330] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-04. [2025-10-20T21:53:48.495009] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-07 (1.04 seconds). [2025-10-20T21:53:49.051386] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-08. [2025-10-20T21:53:49.734091] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T21:53:49.860117] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-04. [2025-10-20T21:53:49.862960] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T21:53:50.651430] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-09. [2025-10-20T21:53:51.130862] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-04 (1.27 seconds). [2025-10-20T21:53:51.408937] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T21:53:51.451330] [INFO ] [REMOTE ] WF: Fetching primary data for 2025-10. [2025-10-20T21:53:52.251404] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-05. [2025-10-20T21:53:52.273141] [INFO ] [REMOTE ] WF: Incremental fetch complete for 2025-10 (0.82 seconds). [2025-10-20T21:53:52.275467] [INFO ] [REMOTE ] WF: Full fetch complete (16172 records). [2025-10-20T21:53:52.275537] [DEBUG] [VALIDATOR ] WF: Validating primary data. [2025-10-20T21:53:52.637854] [DEBUG] [VALIDATOR ] WF: All expected trading days are present. [2025-10-20T21:53:52.637979] [DEBUG] [VALIDATOR ] WF: Primary data validation passed. [2025-10-20T21:53:52.638345] [INFO ] [MAIN ] WF: Fetched primary data is valid. [2025-10-20T21:53:52.638386] [DEBUG] [LOCAL ] WF: Saving primary data. [2025-10-20T21:53:52.639016] [DEBUG] [LOCAL ] WF: Trimmed 16172 records to 10893 (keeping 200 days of data) [2025-10-20T21:53:52.705990] [DEBUG] [LOCAL ] WF: Primary data saved (10893 records). [2025-10-20T21:53:52.706131] [INFO ] [MAIN ] WF: Full history saved. [2025-10-20T21:53:52.706210] [INFO ] [MAIN ] Update Progress: 2285/2367 symbols completed. WF took 22.1s. Estimated remaining: 10m 33s. Rate: 7.8 symbols/min [2025-10-20T21:53:52.872732] [INFO ] [MAIN ] WH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:53:52.913282] [DEBUG] [LOCAL ] WH: Loaded 45923 existing records from primary storage. [2025-10-20T21:53:52.913632] [INFO ] [MAIN ] WH: Loaded 45923 existing records. [2025-10-20T21:53:53.056587] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-05. [2025-10-20T21:53:53.093862] [WARN ] [MAIN ] WH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:53:53.094017] [INFO ] [MAIN ] WH: Wiping existing primary data before full refresh. [2025-10-20T21:53:53.094061] [INFO ] [LOCAL ] WH: Starting deletion of primary data. [2025-10-20T21:53:53.094468] [DEBUG] [LOCAL ] WH: Deleted primary data file. [2025-10-20T21:53:53.094522] [INFO ] [MAIN ] WH: Starting full primary refresh. [2025-10-20T21:53:53.450098] [INFO ] [REMOTE ] WH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:53:53.462156] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T21:53:53.851366] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-06. [2025-10-20T21:53:54.517616] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-05 (1.46 seconds). [2025-10-20T21:53:54.611038] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-06 (0.76 seconds). [2025-10-20T21:53:54.651358] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-07. [2025-10-20T21:53:55.451380] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-06. [2025-10-20T21:53:55.816102] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T21:53:56.251424] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-01. [2025-10-20T21:53:57.041492] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T21:53:57.051330] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-07. [2025-10-20T21:53:57.801160] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-01 (1.55 seconds). [2025-10-20T21:53:57.851345] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-02. [2025-10-20T21:53:58.620702] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T21:53:58.651342] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-08. [2025-10-20T21:53:58.945230] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T21:53:59.451454] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-08. [2025-10-20T21:54:00.111916] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T21:54:00.251330] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-09. [2025-10-20T21:54:00.937449] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T21:54:01.051424] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-09. [2025-10-20T21:54:01.851342] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-03. [2025-10-20T21:54:02.023352] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T21:54:02.303975] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T21:54:02.651364] [INFO ] [REMOTE ] WFG: Fetching primary data for 2025-10. [2025-10-20T21:54:03.278492] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T21:54:03.451338] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-05. [2025-10-20T21:54:03.797691] [INFO ] [REMOTE ] WFG: Incremental fetch complete for 2025-10 (1.15 seconds). [2025-10-20T21:54:03.801502] [INFO ] [REMOTE ] WFG: Full fetch complete (36424 records). [2025-10-20T21:54:03.801581] [DEBUG] [VALIDATOR ] WFG: Validating primary data. [2025-10-20T21:54:04.169459] [DEBUG] [VALIDATOR ] WFG: All expected trading days are present. [2025-10-20T21:54:04.169539] [DEBUG] [VALIDATOR ] WFG: Primary data validation passed. [2025-10-20T21:54:04.170258] [INFO ] [MAIN ] WFG: Fetched primary data is valid. [2025-10-20T21:54:04.170319] [DEBUG] [LOCAL ] WFG: Saving primary data. [2025-10-20T21:54:04.171167] [DEBUG] [LOCAL ] WFG: Trimmed 36424 records to 24746 (keeping 200 days of data) [2025-10-20T21:54:04.270137] [INFO ] [REMOTE ] WGO: Fetching primary data for 2025-10. [2025-10-20T21:54:04.334725] [DEBUG] [LOCAL ] WFG: Primary data saved (24746 records). [2025-10-20T21:54:04.334847] [INFO ] [MAIN ] WFG: Full history saved. [2025-10-20T21:54:04.334919] [INFO ] [MAIN ] Update Progress: 2286/2367 symbols completed. WFG took 31.2s. Estimated remaining: 10m 25s. Rate: 7.8 symbols/min [2025-10-20T21:54:04.501327] [INFO ] [MAIN ] WHD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:04.537519] [DEBUG] [LOCAL ] WHD: Loaded 40730 existing records from primary storage. [2025-10-20T21:54:04.537850] [INFO ] [MAIN ] WHD: Loaded 40730 existing records. [2025-10-20T21:54:04.713084] [WARN ] [MAIN ] WHD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:04.713286] [INFO ] [MAIN ] WHD: Wiping existing primary data before full refresh. [2025-10-20T21:54:04.713332] [INFO ] [LOCAL ] WHD: Starting deletion of primary data. [2025-10-20T21:54:04.713517] [DEBUG] [LOCAL ] WHD: Deleted primary data file. [2025-10-20T21:54:04.713554] [INFO ] [MAIN ] WHD: Starting full primary refresh. [2025-10-20T21:54:05.055118] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-04. [2025-10-20T21:54:05.069161] [INFO ] [REMOTE ] WHD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:05.594070] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-05 (2.14 seconds). [2025-10-20T21:54:05.607411] [INFO ] [REMOTE ] WGO: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:54:05.613275] [INFO ] [REMOTE ] WGO: Full fetch complete (57015 records). [2025-10-20T21:54:05.613398] [DEBUG] [VALIDATOR ] WGO: Validating primary data. [2025-10-20T21:54:05.856919] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-01. [2025-10-20T21:54:06.007853] [DEBUG] [VALIDATOR ] WGO: All expected trading days are present. [2025-10-20T21:54:06.007976] [DEBUG] [VALIDATOR ] WGO: Primary data validation passed. [2025-10-20T21:54:06.009157] [INFO ] [MAIN ] WGO: Fetched primary data is valid. [2025-10-20T21:54:06.009271] [DEBUG] [LOCAL ] WGO: Saving primary data. [2025-10-20T21:54:06.010590] [DEBUG] [LOCAL ] WGO: Trimmed 57015 records to 37980 (keeping 200 days of data) [2025-10-20T21:54:06.237039] [DEBUG] [LOCAL ] WGO: Primary data saved (37980 records). [2025-10-20T21:54:06.237178] [INFO ] [MAIN ] WGO: Full history saved. [2025-10-20T21:54:06.237255] [INFO ] [MAIN ] Update Progress: 2287/2367 symbols completed. WGO took 25.3s. Estimated remaining: 10m 17s. Rate: 7.8 symbols/min [2025-10-20T21:54:06.411745] [INFO ] [MAIN ] WHG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:06.418921] [DEBUG] [LOCAL ] WHG: Loaded 3165 existing records from primary storage. [2025-10-20T21:54:06.419173] [INFO ] [MAIN ] WHG: Loaded 3165 existing records. [2025-10-20T21:54:06.661829] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-06. [2025-10-20T21:54:06.669344] [WARN ] [MAIN ] WHG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:06.670402] [INFO ] [MAIN ] WHG: Wiping existing primary data before full refresh. [2025-10-20T21:54:06.670477] [INFO ] [LOCAL ] WHG: Starting deletion of primary data. [2025-10-20T21:54:06.661745] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T21:54:06.670676] [DEBUG] [LOCAL ] WHG: Deleted primary data file. [2025-10-20T21:54:06.670744] [INFO ] [MAIN ] WHG: Starting full primary refresh. [2025-10-20T21:54:07.116900] [INFO ] [REMOTE ] WHG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:07.383670] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:54:07.451424] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-01. [2025-10-20T21:54:07.957718] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-01 (0.51 seconds). [2025-10-20T21:54:08.251377] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-02. [2025-10-20T21:54:08.794497] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T21:54:09.051374] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-05. [2025-10-20T21:54:09.677249] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T21:54:09.851376] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-02. [2025-10-20T21:54:10.441608] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-02 (0.59 seconds). [2025-10-20T21:54:10.651402] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-03. [2025-10-20T21:54:10.739387] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T21:54:11.071049] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-03 (0.42 seconds). [2025-10-20T21:54:11.451369] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-07. [2025-10-20T21:54:12.251414] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-04. [2025-10-20T21:54:12.832831] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-04 (0.58 seconds). [2025-10-20T21:54:13.051357] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-05. [2025-10-20T21:54:13.674759] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-05 (0.62 seconds). [2025-10-20T21:54:13.678442] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-07 (2.23 seconds). [2025-10-20T21:54:13.851370] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-08. [2025-10-20T21:54:14.651374] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-06. [2025-10-20T21:54:15.312329] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-06 (0.66 seconds). [2025-10-20T21:54:15.451393] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-07. [2025-10-20T21:54:15.720336] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-08 (1.87 seconds). [2025-10-20T21:54:16.018749] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-07 (0.57 seconds). [2025-10-20T21:54:16.251435] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-06. [2025-10-20T21:54:17.051434] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-09. [2025-10-20T21:54:17.800262] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T21:54:17.851431] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-07. [2025-10-20T21:54:18.651396] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-03. [2025-10-20T21:54:18.887775] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-09 (1.84 seconds). [2025-10-20T21:54:19.451416] [INFO ] [REMOTE ] WFC: Fetching primary data for 2025-10. [2025-10-20T21:54:19.620427] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T21:54:20.144355] [INFO ] [REMOTE ] WFC: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T21:54:20.152421] [INFO ] [REMOTE ] WFC: Full fetch complete (77925 records). [2025-10-20T21:54:20.152553] [DEBUG] [VALIDATOR ] WFC: Validating primary data. [2025-10-20T21:54:20.255097] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-08. [2025-10-20T21:54:20.493738] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-03 (1.84 seconds). [2025-10-20T21:54:20.596430] [DEBUG] [VALIDATOR ] WFC: All expected trading days are present. [2025-10-20T21:54:20.596514] [DEBUG] [VALIDATOR ] WFC: Primary data validation passed. [2025-10-20T21:54:20.597684] [INFO ] [MAIN ] WFC: Fetched primary data is valid. [2025-10-20T21:54:20.597745] [DEBUG] [LOCAL ] WFC: Saving primary data. [2025-10-20T21:54:20.599136] [DEBUG] [LOCAL ] WFC: Trimmed 77925 records to 53748 (keeping 200 days of data) [2025-10-20T21:54:20.787359] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-08 (0.53 seconds). [2025-10-20T21:54:20.928210] [DEBUG] [LOCAL ] WFC: Primary data saved (53748 records). [2025-10-20T21:54:20.928355] [INFO ] [MAIN ] WFC: Full history saved. [2025-10-20T21:54:20.928443] [INFO ] [MAIN ] Update Progress: 2288/2367 symbols completed. WFC took 48.9s. Estimated remaining: 10m 10s. Rate: 7.8 symbols/min [2025-10-20T21:54:21.065979] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-09. [2025-10-20T21:54:21.097566] [INFO ] [MAIN ] WHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:21.140663] [DEBUG] [LOCAL ] WHR: Loaded 49531 existing records from primary storage. [2025-10-20T21:54:21.140964] [INFO ] [MAIN ] WHR: Loaded 49531 existing records. [2025-10-20T21:54:21.319428] [WARN ] [MAIN ] WHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:21.319635] [INFO ] [MAIN ] WHR: Wiping existing primary data before full refresh. [2025-10-20T21:54:21.319717] [INFO ] [LOCAL ] WHR: Starting deletion of primary data. [2025-10-20T21:54:21.320061] [DEBUG] [LOCAL ] WHR: Deleted primary data file. [2025-10-20T21:54:21.320098] [INFO ] [MAIN ] WHR: Starting full primary refresh. [2025-10-20T21:54:21.598853] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-09 (0.53 seconds). [2025-10-20T21:54:21.675879] [INFO ] [REMOTE ] WHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:21.851365] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-08. [2025-10-20T21:54:22.651384] [INFO ] [REMOTE ] WHG: Fetching primary data for 2025-10. [2025-10-20T21:54:23.018732] [INFO ] [REMOTE ] WHG: Incremental fetch complete for 2025-10 (0.37 seconds). [2025-10-20T21:54:23.019475] [INFO ] [REMOTE ] WHG: Full fetch complete (4417 records). [2025-10-20T21:54:23.019541] [DEBUG] [VALIDATOR ] WHG: Validating primary data. [2025-10-20T21:54:23.395481] [DEBUG] [VALIDATOR ] WHG: All expected trading days are present. [2025-10-20T21:54:23.395577] [DEBUG] [VALIDATOR ] WHG: Primary data validation passed. [2025-10-20T21:54:23.395791] [INFO ] [MAIN ] WHG: Fetched primary data is valid. [2025-10-20T21:54:23.395862] [DEBUG] [LOCAL ] WHG: Saving primary data. [2025-10-20T21:54:23.396402] [DEBUG] [LOCAL ] WHG: Trimmed 4417 records to 3075 (keeping 200 days of data) [2025-10-20T21:54:23.414913] [DEBUG] [LOCAL ] WHG: Primary data saved (3075 records). [2025-10-20T21:54:23.414990] [INFO ] [MAIN ] WHG: Full history saved. [2025-10-20T21:54:23.415058] [INFO ] [MAIN ] Update Progress: 2289/2367 symbols completed. WHG took 17.2s. Estimated remaining: 10m 2s. Rate: 7.8 symbols/min [2025-10-20T21:54:23.456506] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-01. [2025-10-20T21:54:23.596803] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T21:54:23.629913] [INFO ] [MAIN ] WIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:23.637364] [DEBUG] [LOCAL ] WIA: Loaded 5051 existing records from primary storage. [2025-10-20T21:54:23.637571] [INFO ] [MAIN ] WIA: Loaded 5051 existing records. [2025-10-20T21:54:23.815586] [WARN ] [MAIN ] WIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:23.815811] [INFO ] [MAIN ] WIA: Wiping existing primary data before full refresh. [2025-10-20T21:54:23.815859] [INFO ] [LOCAL ] WIA: Starting deletion of primary data. [2025-10-20T21:54:23.816017] [DEBUG] [LOCAL ] WIA: Deleted primary data file. [2025-10-20T21:54:23.816072] [INFO ] [MAIN ] WIA: Starting full primary refresh. [2025-10-20T21:54:24.177425] [INFO ] [REMOTE ] WIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:24.251358] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-01. [2025-10-20T21:54:24.855178] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-01 (0.60 seconds). [2025-10-20T21:54:25.051341] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-02. [2025-10-20T21:54:25.134490] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-01 (1.68 seconds). [2025-10-20T21:54:25.613519] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-02 (0.56 seconds). [2025-10-20T21:54:25.851386] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-04. [2025-10-20T21:54:26.651426] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-02. [2025-10-20T21:54:27.451395] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-09. [2025-10-20T21:54:27.490506] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-04 (1.64 seconds). [2025-10-20T21:54:28.250040] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T21:54:28.258387] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-05. [2025-10-20T21:54:29.009946] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-09 (1.56 seconds). [2025-10-20T21:54:29.051399] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-03. [2025-10-20T21:54:29.772930] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-05 (1.51 seconds). [2025-10-20T21:54:29.851351] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-06. [2025-10-20T21:54:30.651362] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-03. [2025-10-20T21:54:30.830234] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:54:30.899228] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-06 (1.05 seconds). [2025-10-20T21:54:31.248029] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T21:54:31.451394] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-04. [2025-10-20T21:54:32.056385] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T21:54:32.251359] [INFO ] [REMOTE ] WH: Fetching primary data for 2025-10. [2025-10-20T21:54:33.051369] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-05. [2025-10-20T21:54:33.716279] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-05 (0.66 seconds). [2025-10-20T21:54:33.716963] [INFO ] [REMOTE ] WH: Incremental fetch complete for 2025-10 (1.47 seconds). [2025-10-20T21:54:33.725432] [INFO ] [REMOTE ] WH: Full fetch complete (65505 records). [2025-10-20T21:54:33.725537] [DEBUG] [VALIDATOR ] WH: Validating primary data. [2025-10-20T21:54:33.863517] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-06. [2025-10-20T21:54:34.129447] [DEBUG] [VALIDATOR ] WH: All expected trading days are present. [2025-10-20T21:54:34.129539] [DEBUG] [VALIDATOR ] WH: Primary data validation passed. [2025-10-20T21:54:34.130828] [INFO ] [MAIN ] WH: Fetched primary data is valid. [2025-10-20T21:54:34.130894] [DEBUG] [LOCAL ] WH: Saving primary data. [2025-10-20T21:54:34.132286] [DEBUG] [LOCAL ] WH: Trimmed 65505 records to 45259 (keeping 200 days of data) [2025-10-20T21:54:34.411269] [DEBUG] [LOCAL ] WH: Primary data saved (45259 records). [2025-10-20T21:54:34.411391] [INFO ] [MAIN ] WH: Full history saved. [2025-10-20T21:54:34.411467] [INFO ] [MAIN ] Update Progress: 2290/2367 symbols completed. WH took 41.7s. Estimated remaining: 9m 54s. Rate: 7.8 symbols/min [2025-10-20T21:54:34.584968] [INFO ] [MAIN ] WINN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:34.597929] [DEBUG] [LOCAL ] WINN: Loaded 11678 existing records from primary storage. [2025-10-20T21:54:34.598155] [INFO ] [MAIN ] WINN: Loaded 11678 existing records. [2025-10-20T21:54:34.666810] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T21:54:34.655675] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-04. [2025-10-20T21:54:34.789223] [WARN ] [MAIN ] WINN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:34.789444] [INFO ] [MAIN ] WINN: Wiping existing primary data before full refresh. [2025-10-20T21:54:34.789491] [INFO ] [LOCAL ] WINN: Starting deletion of primary data. [2025-10-20T21:54:34.789718] [DEBUG] [LOCAL ] WINN: Deleted primary data file. [2025-10-20T21:54:34.789791] [INFO ] [MAIN ] WINN: Starting full primary refresh. [2025-10-20T21:54:35.160210] [INFO ] [REMOTE ] WINN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:35.451361] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-01. [2025-10-20T21:54:36.251374] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-07. [2025-10-20T21:54:36.405397] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T21:54:36.714369] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-04 (2.06 seconds). [2025-10-20T21:54:37.051415] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-02. [2025-10-20T21:54:37.692229] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-07 (1.44 seconds). [2025-10-20T21:54:37.851364] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-08. [2025-10-20T21:54:37.937190] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-02 (0.89 seconds). [2025-10-20T21:54:38.651422] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-07. [2025-10-20T21:54:39.244383] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-07 (0.59 seconds). [2025-10-20T21:54:39.288084] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-08 (1.44 seconds). [2025-10-20T21:54:39.451381] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-03. [2025-10-20T21:54:40.251422] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-05. [2025-10-20T21:54:40.282074] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-03 (0.83 seconds). [2025-10-20T21:54:41.051386] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-09. [2025-10-20T21:54:41.851417] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-04. [2025-10-20T21:54:42.075278] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-05 (1.82 seconds). [2025-10-20T21:54:42.556466] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-09 (1.51 seconds). [2025-10-20T21:54:42.651338] [INFO ] [REMOTE ] WHD: Fetching primary data for 2025-10. [2025-10-20T21:54:42.807715] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-04 (0.96 seconds). [2025-10-20T21:54:43.451424] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-05. [2025-10-20T21:54:44.110386] [INFO ] [REMOTE ] WHD: Incremental fetch complete for 2025-10 (1.46 seconds). [2025-10-20T21:54:44.116476] [INFO ] [REMOTE ] WHD: Full fetch complete (57900 records). [2025-10-20T21:54:44.116579] [DEBUG] [VALIDATOR ] WHD: Validating primary data. [2025-10-20T21:54:44.256471] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-06. [2025-10-20T21:54:44.456046] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-05 (1.00 seconds). [2025-10-20T21:54:44.547363] [DEBUG] [VALIDATOR ] WHD: All expected trading days are present. [2025-10-20T21:54:44.547437] [DEBUG] [VALIDATOR ] WHD: Primary data validation passed. [2025-10-20T21:54:44.548423] [INFO ] [MAIN ] WHD: Fetched primary data is valid. [2025-10-20T21:54:44.548483] [DEBUG] [LOCAL ] WHD: Saving primary data. [2025-10-20T21:54:44.549505] [DEBUG] [LOCAL ] WHD: Trimmed 57900 records to 40110 (keeping 200 days of data) [2025-10-20T21:54:44.783076] [DEBUG] [LOCAL ] WHD: Primary data saved (40110 records). [2025-10-20T21:54:44.783193] [INFO ] [MAIN ] WHD: Full history saved. [2025-10-20T21:54:44.783264] [INFO ] [MAIN ] Update Progress: 2291/2367 symbols completed. WHD took 40.4s. Estimated remaining: 9m 47s. Rate: 7.8 symbols/min [2025-10-20T21:54:44.954086] [INFO ] [MAIN ] WIT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:45.000850] [DEBUG] [LOCAL ] WIT: Loaded 53266 existing records from primary storage. [2025-10-20T21:54:45.001156] [INFO ] [MAIN ] WIT: Loaded 53266 existing records. [2025-10-20T21:54:45.062413] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-08. [2025-10-20T21:54:45.177064] [WARN ] [MAIN ] WIT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:45.177329] [INFO ] [MAIN ] WIT: Wiping existing primary data before full refresh. [2025-10-20T21:54:45.177440] [INFO ] [LOCAL ] WIT: Starting deletion of primary data. [2025-10-20T21:54:45.177716] [DEBUG] [LOCAL ] WIT: Deleted primary data file. [2025-10-20T21:54:45.177772] [INFO ] [MAIN ] WIT: Starting full primary refresh. [2025-10-20T21:54:45.521714] [INFO ] [REMOTE ] WIT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:45.746848] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-08 (0.68 seconds). [2025-10-20T21:54:45.851375] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-09. [2025-10-20T21:54:45.959582] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T21:54:46.482234] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-09 (0.63 seconds). [2025-10-20T21:54:46.651424] [INFO ] [REMOTE ] WIA: Fetching primary data for 2025-10. [2025-10-20T21:54:47.225573] [INFO ] [REMOTE ] WIA: Incremental fetch complete for 2025-10 (0.57 seconds). [2025-10-20T21:54:47.227289] [INFO ] [REMOTE ] WIA: Full fetch complete (7162 records). [2025-10-20T21:54:47.227365] [DEBUG] [VALIDATOR ] WIA: Validating primary data. [2025-10-20T21:54:47.456536] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-07. [2025-10-20T21:54:47.601969] [DEBUG] [VALIDATOR ] WIA: All expected trading days are present. [2025-10-20T21:54:47.602056] [DEBUG] [VALIDATOR ] WIA: Primary data validation passed. [2025-10-20T21:54:47.602256] [INFO ] [MAIN ] WIA: Fetched primary data is valid. [2025-10-20T21:54:47.602365] [DEBUG] [LOCAL ] WIA: Saving primary data. [2025-10-20T21:54:47.602829] [DEBUG] [LOCAL ] WIA: Trimmed 7162 records to 5002 (keeping 200 days of data) [2025-10-20T21:54:47.632823] [DEBUG] [LOCAL ] WIA: Primary data saved (5002 records). [2025-10-20T21:54:47.632925] [INFO ] [MAIN ] WIA: Full history saved. [2025-10-20T21:54:47.633000] [INFO ] [MAIN ] Update Progress: 2292/2367 symbols completed. WIA took 24.2s. Estimated remaining: 9m 39s. Rate: 7.8 symbols/min [2025-10-20T21:54:47.808607] [INFO ] [MAIN ] WIW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:47.831699] [DEBUG] [LOCAL ] WIW: Loaded 20553 existing records from primary storage. [2025-10-20T21:54:47.832011] [INFO ] [MAIN ] WIW: Loaded 20553 existing records. [2025-10-20T21:54:48.012831] [WARN ] [MAIN ] WIW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:48.013187] [INFO ] [MAIN ] WIW: Wiping existing primary data before full refresh. [2025-10-20T21:54:48.013251] [INFO ] [LOCAL ] WIW: Starting deletion of primary data. [2025-10-20T21:54:48.014025] [DEBUG] [LOCAL ] WIW: Deleted primary data file. [2025-10-20T21:54:48.014086] [INFO ] [MAIN ] WIW: Starting full primary refresh. [2025-10-20T21:54:48.256526] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-06. [2025-10-20T21:54:48.367592] [INFO ] [REMOTE ] WIW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:49.054515] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-01. [2025-10-20T21:54:49.058514] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T21:54:49.442124] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-07 (1.99 seconds). [2025-10-20T21:54:49.739014] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-01 (0.68 seconds). [2025-10-20T21:54:49.851336] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-02. [2025-10-20T21:54:50.651377] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-07. [2025-10-20T21:54:50.818262] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T21:54:51.417941] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-07 (0.77 seconds). [2025-10-20T21:54:51.451337] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-08. [2025-10-20T21:54:52.239770] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-08 (0.79 seconds). [2025-10-20T21:54:52.251332] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-08. [2025-10-20T21:54:53.051408] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-09. [2025-10-20T21:54:53.807401] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T21:54:53.851408] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-01. [2025-10-20T21:54:54.085330] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-08 (1.83 seconds). [2025-10-20T21:54:54.651430] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-09. [2025-10-20T21:54:55.451349] [INFO ] [REMOTE ] WINN: Fetching primary data for 2025-10. [2025-10-20T21:54:55.715770] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T21:54:55.911755] [INFO ] [REMOTE ] WINN: Incremental fetch complete for 2025-10 (0.46 seconds). [2025-10-20T21:54:55.913872] [INFO ] [REMOTE ] WINN: Full fetch complete (17385 records). [2025-10-20T21:54:55.913944] [DEBUG] [VALIDATOR ] WINN: Validating primary data. [2025-10-20T21:54:56.251738] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-03. [2025-10-20T21:54:56.289134] [DEBUG] [VALIDATOR ] WINN: All expected trading days are present. [2025-10-20T21:54:56.289230] [DEBUG] [VALIDATOR ] WINN: Primary data validation passed. [2025-10-20T21:54:56.289696] [INFO ] [MAIN ] WINN: Fetched primary data is valid. [2025-10-20T21:54:56.289771] [DEBUG] [LOCAL ] WINN: Saving primary data. [2025-10-20T21:54:56.290394] [DEBUG] [LOCAL ] WINN: Trimmed 17385 records to 11461 (keeping 200 days of data) [2025-10-20T21:54:56.360357] [DEBUG] [LOCAL ] WINN: Primary data saved (11461 records). [2025-10-20T21:54:56.360474] [INFO ] [MAIN ] WINN: Full history saved. [2025-10-20T21:54:56.360548] [INFO ] [MAIN ] Update Progress: 2293/2367 symbols completed. WINN took 21.9s. Estimated remaining: 9m 31s. Rate: 7.8 symbols/min [2025-10-20T21:54:56.530669] [INFO ] [MAIN ] WK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:54:56.559854] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T21:54:56.599458] [DEBUG] [LOCAL ] WK: Loaded 37823 existing records from primary storage. [2025-10-20T21:54:56.599760] [INFO ] [MAIN ] WK: Loaded 37823 existing records. [2025-10-20T21:54:56.773145] [WARN ] [MAIN ] WK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:54:56.773371] [INFO ] [MAIN ] WK: Wiping existing primary data before full refresh. [2025-10-20T21:54:56.773418] [INFO ] [LOCAL ] WK: Starting deletion of primary data. [2025-10-20T21:54:56.773599] [DEBUG] [LOCAL ] WK: Deleted primary data file. [2025-10-20T21:54:56.773637] [INFO ] [MAIN ] WK: Starting full primary refresh. [2025-10-20T21:54:57.056534] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-02. [2025-10-20T21:54:57.123734] [INFO ] [REMOTE ] WK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:54:57.375791] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-03 (1.12 seconds). [2025-10-20T21:54:57.851424] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-01. [2025-10-20T21:54:58.651432] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-04. [2025-10-20T21:54:58.711344] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-02 (1.65 seconds). [2025-10-20T21:54:59.247081] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-01 (1.40 seconds). [2025-10-20T21:54:59.451362] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-03. [2025-10-20T21:54:59.800994] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T21:55:00.251394] [INFO ] [REMOTE ] WHR: Fetching primary data for 2025-10. [2025-10-20T21:55:01.051431] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-02. [2025-10-20T21:55:01.591799] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-03 (2.14 seconds). [2025-10-20T21:55:01.851389] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-05. [2025-10-20T21:55:02.086431] [INFO ] [REMOTE ] WHR: Incremental fetch complete for 2025-10 (1.84 seconds). [2025-10-20T21:55:02.094543] [INFO ] [REMOTE ] WHR: Full fetch complete (69560 records). [2025-10-20T21:55:02.094803] [DEBUG] [VALIDATOR ] WHR: Validating primary data. [2025-10-20T21:55:02.527598] [DEBUG] [VALIDATOR ] WHR: All expected trading days are present. [2025-10-20T21:55:02.527753] [DEBUG] [VALIDATOR ] WHR: Primary data validation passed. [2025-10-20T21:55:02.529128] [INFO ] [MAIN ] WHR: Fetched primary data is valid. [2025-10-20T21:55:02.529204] [DEBUG] [LOCAL ] WHR: Saving primary data. [2025-10-20T21:55:02.552579] [DEBUG] [LOCAL ] WHR: Trimmed 69560 records to 48938 (keeping 200 days of data) [2025-10-20T21:55:02.554637] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-02 (1.50 seconds). [2025-10-20T21:55:02.652052] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-03. [2025-10-20T21:55:02.858051] [DEBUG] [LOCAL ] WHR: Primary data saved (48938 records). [2025-10-20T21:55:02.858250] [INFO ] [MAIN ] WHR: Full history saved. [2025-10-20T21:55:02.858498] [INFO ] [MAIN ] Update Progress: 2294/2367 symbols completed. WHR took 41.9s. Estimated remaining: 9m 23s. Rate: 7.8 symbols/min [2025-10-20T21:55:03.020553] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-05 (1.17 seconds). [2025-10-20T21:55:03.043611] [INFO ] [MAIN ] WKC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:03.082661] [DEBUG] [LOCAL ] WKC: Loaded 35103 existing records from primary storage. [2025-10-20T21:55:03.082939] [INFO ] [MAIN ] WKC: Loaded 35103 existing records. [2025-10-20T21:55:03.264372] [WARN ] [MAIN ] WKC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:03.264578] [INFO ] [MAIN ] WKC: Wiping existing primary data before full refresh. [2025-10-20T21:55:03.264625] [INFO ] [LOCAL ] WKC: Starting deletion of primary data. [2025-10-20T21:55:03.264848] [DEBUG] [LOCAL ] WKC: Deleted primary data file. [2025-10-20T21:55:03.264895] [INFO ] [MAIN ] WKC: Starting full primary refresh. [2025-10-20T21:55:03.458609] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-04. [2025-10-20T21:55:03.619451] [INFO ] [REMOTE ] WKC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:04.164739] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-03 (1.51 seconds). [2025-10-20T21:55:04.251369] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-04. [2025-10-20T21:55:05.051364] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-01. [2025-10-20T21:55:05.343899] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-04 (1.89 seconds). [2025-10-20T21:55:05.851453] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-06. [2025-10-20T21:55:05.937512] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-04 (1.69 seconds). [2025-10-20T21:55:06.047689] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T21:55:06.496270] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-06 (0.64 seconds). [2025-10-20T21:55:06.651354] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-07. [2025-10-20T21:55:07.451372] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-02. [2025-10-20T21:55:07.725760] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-07 (1.07 seconds). [2025-10-20T21:55:08.251370] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-05. [2025-10-20T21:55:08.832460] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-02 (1.38 seconds). [2025-10-20T21:55:09.051399] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-05. [2025-10-20T21:55:09.851373] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-03. [2025-10-20T21:55:10.099917] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T21:55:10.552748] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-05 (1.50 seconds). [2025-10-20T21:55:10.651351] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-06. [2025-10-20T21:55:10.929160] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-03 (1.08 seconds). [2025-10-20T21:55:11.451398] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-04. [2025-10-20T21:55:11.991851] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T21:55:12.251385] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-08. [2025-10-20T21:55:12.935883] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T21:55:13.051398] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-05. [2025-10-20T21:55:13.321168] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T21:55:13.851393] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-07. [2025-10-20T21:55:14.490772] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-05 (1.44 seconds). [2025-10-20T21:55:14.651383] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-09. [2025-10-20T21:55:15.334313] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-07 (1.48 seconds). [2025-10-20T21:55:15.451390] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-06. [2025-10-20T21:55:15.754196] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-09 (1.10 seconds). [2025-10-20T21:55:16.251391] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-08. [2025-10-20T21:55:16.794885] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T21:55:17.051389] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-06. [2025-10-20T21:55:17.737462] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-08 (1.49 seconds). [2025-10-20T21:55:17.851484] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-07. [2025-10-20T21:55:18.651441] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-09. [2025-10-20T21:55:18.760531] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T21:55:19.190255] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-07 (1.34 seconds). [2025-10-20T21:55:19.451356] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-07. [2025-10-20T21:55:20.197778] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:55:20.251357] [INFO ] [REMOTE ] WK: Fetching primary data for 2025-10. [2025-10-20T21:55:21.051418] [INFO ] [REMOTE ] WIW: Fetching primary data for 2025-10. [2025-10-20T21:55:21.269590] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T21:55:21.605265] [INFO ] [REMOTE ] WK: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T21:55:21.611762] [INFO ] [REMOTE ] WK: Full fetch complete (53421 records). [2025-10-20T21:55:21.611950] [DEBUG] [VALIDATOR ] WK: Validating primary data. [2025-10-20T21:55:21.761701] [INFO ] [REMOTE ] WIW: Incremental fetch complete for 2025-10 (0.71 seconds). [2025-10-20T21:55:21.799307] [INFO ] [REMOTE ] WIW: Full fetch complete (29164 records). [2025-10-20T21:55:21.814755] [DEBUG] [VALIDATOR ] WIW: Validating primary data. [2025-10-20T21:55:21.856485] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-08. [2025-10-20T21:55:22.190143] [DEBUG] [VALIDATOR ] WK: All expected trading days are present. [2025-10-20T21:55:22.191042] [DEBUG] [VALIDATOR ] WK: Primary data validation passed. [2025-10-20T21:55:22.201336] [INFO ] [MAIN ] WK: Fetched primary data is valid. [2025-10-20T21:55:22.206512] [DEBUG] [LOCAL ] WK: Saving primary data. [2025-10-20T21:55:22.306457] [DEBUG] [LOCAL ] WK: Trimmed 53421 records to 37168 (keeping 200 days of data) [2025-10-20T21:55:22.510877] [DEBUG] [VALIDATOR ] WIW: All expected trading days are present. [2025-10-20T21:55:22.512392] [DEBUG] [VALIDATOR ] WIW: Primary data validation passed. [2025-10-20T21:55:22.518432] [INFO ] [MAIN ] WIW: Fetched primary data is valid. [2025-10-20T21:55:22.522864] [DEBUG] [LOCAL ] WIW: Saving primary data. [2025-10-20T21:55:22.532000] [DEBUG] [LOCAL ] WIW: Trimmed 29164 records to 20283 (keeping 200 days of data) [2025-10-20T21:55:22.674839] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-08. [2025-10-20T21:55:22.739326] [DEBUG] [LOCAL ] WIW: Primary data saved (20283 records). [2025-10-20T21:55:22.741571] [INFO ] [MAIN ] WIW: Full history saved. [2025-10-20T21:55:22.746443] [INFO ] [MAIN ] Update Progress: 2295/2367 symbols completed. WIW took 35.1s. Estimated remaining: 9m 16s. Rate: 7.8 symbols/min [2025-10-20T21:55:22.897163] [DEBUG] [LOCAL ] WK: Primary data saved (37168 records). [2025-10-20T21:55:22.916481] [INFO ] [MAIN ] WK: Full history saved. [2025-10-20T21:55:22.923429] [INFO ] [MAIN ] Update Progress: 2296/2367 symbols completed. WK took 26.6s. Estimated remaining: 9m 8s. Rate: 7.8 symbols/min [2025-10-20T21:55:22.943019] [INFO ] [MAIN ] WLK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:23.114938] [INFO ] [MAIN ] WLKP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:23.134006] [DEBUG] [LOCAL ] WLKP: Loaded 7181 existing records from primary storage. [2025-10-20T21:55:23.134231] [INFO ] [MAIN ] WLKP: Loaded 7181 existing records. [2025-10-20T21:55:23.288667] [DEBUG] [LOCAL ] WLK: Loaded 46398 existing records from primary storage. [2025-10-20T21:55:23.299322] [INFO ] [MAIN ] WLK: Loaded 46398 existing records. [2025-10-20T21:55:23.334323] [WARN ] [MAIN ] WLKP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:23.342352] [INFO ] [MAIN ] WLKP: Wiping existing primary data before full refresh. [2025-10-20T21:55:23.350112] [INFO ] [LOCAL ] WLKP: Starting deletion of primary data. [2025-10-20T21:55:23.356133] [DEBUG] [LOCAL ] WLKP: Deleted primary data file. [2025-10-20T21:55:23.356214] [INFO ] [MAIN ] WLKP: Starting full primary refresh. [2025-10-20T21:55:23.769369] [WARN ] [MAIN ] WLK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:23.810610] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T21:55:23.810761] [INFO ] [MAIN ] WLK: Wiping existing primary data before full refresh. [2025-10-20T21:55:23.810995] [INFO ] [LOCAL ] WLK: Starting deletion of primary data. [2025-10-20T21:55:23.831743] [DEBUG] [LOCAL ] WLK: Deleted primary data file. [2025-10-20T21:55:23.836951] [INFO ] [MAIN ] WLK: Starting full primary refresh. [2025-10-20T21:55:23.888909] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-09. [2025-10-20T21:55:24.082679] [INFO ] [REMOTE ] WLKP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:24.254469] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-01. [2025-10-20T21:55:24.346298] [INFO ] [REMOTE ] WLK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:24.376213] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T21:55:25.018595] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-01 (0.76 seconds). [2025-10-20T21:55:25.051370] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-02. [2025-10-20T21:55:25.417089] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-09 (1.53 seconds). [2025-10-20T21:55:25.773128] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-02 (0.72 seconds). [2025-10-20T21:55:25.851376] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-01. [2025-10-20T21:55:26.651420] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-09. [2025-10-20T21:55:27.302087] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T21:55:27.451405] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-03. [2025-10-20T21:55:28.117620] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-03 (0.67 seconds). [2025-10-20T21:55:28.251374] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-04. [2025-10-20T21:55:28.372487] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T21:55:28.953612] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-04 (0.70 seconds). [2025-10-20T21:55:29.051383] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-05. [2025-10-20T21:55:29.793279] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T21:55:29.851403] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-06. [2025-10-20T21:55:30.621133] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-06 (0.77 seconds). [2025-10-20T21:55:30.651336] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-07. [2025-10-20T21:55:31.364986] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-07 (0.71 seconds). [2025-10-20T21:55:31.451347] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-02. [2025-10-20T21:55:32.251386] [INFO ] [REMOTE ] WIT: Fetching primary data for 2025-10. [2025-10-20T21:55:32.889780] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-02 (1.44 seconds). [2025-10-20T21:55:33.051393] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-03. [2025-10-20T21:55:33.851380] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-08. [2025-10-20T21:55:33.921118] [INFO ] [REMOTE ] WIT: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T21:55:33.929135] [INFO ] [REMOTE ] WIT: Full fetch complete (75986 records). [2025-10-20T21:55:33.929277] [DEBUG] [VALIDATOR ] WIT: Validating primary data. [2025-10-20T21:55:34.324701] [DEBUG] [VALIDATOR ] WIT: All expected trading days are present. [2025-10-20T21:55:34.324797] [DEBUG] [VALIDATOR ] WIT: Primary data validation passed. [2025-10-20T21:55:34.326058] [INFO ] [MAIN ] WIT: Fetched primary data is valid. [2025-10-20T21:55:34.326133] [DEBUG] [LOCAL ] WIT: Saving primary data. [2025-10-20T21:55:34.327694] [DEBUG] [LOCAL ] WIT: Trimmed 75986 records to 52534 (keeping 200 days of data) [2025-10-20T21:55:34.427114] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-08 (0.58 seconds). [2025-10-20T21:55:34.637824] [DEBUG] [LOCAL ] WIT: Primary data saved (52534 records). [2025-10-20T21:55:34.637982] [INFO ] [MAIN ] WIT: Full history saved. [2025-10-20T21:55:34.638069] [INFO ] [MAIN ] Update Progress: 2297/2367 symbols completed. WIT took 49.9s. Estimated remaining: 9m 0s. Rate: 7.8 symbols/min [2025-10-20T21:55:34.658696] [INFO ] [REMOTE ] WKC: Fetching primary data for 2025-10. [2025-10-20T21:55:34.833956] [INFO ] [MAIN ] WLTG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:34.836042] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:55:34.834668] [DEBUG] [LOCAL ] WLTG: No existing data file found in primary storage. [2025-10-20T21:55:34.844762] [WARN ] [MAIN ] WLTG: No existing data found, performing full refresh instead. [2025-10-20T21:55:34.845238] [INFO ] [MAIN ] WLTG: Wiping existing primary data before full refresh. [2025-10-20T21:55:34.845562] [INFO ] [LOCAL ] WLTG: Starting deletion of primary data. [2025-10-20T21:55:34.845621] [DEBUG] [LOCAL ] WLTG: No primary data file found to delete. [2025-10-20T21:55:34.845693] [INFO ] [MAIN ] WLTG: Starting full primary refresh. [2025-10-20T21:55:35.198743] [INFO ] [REMOTE ] WLTG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:35.451368] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-09. [2025-10-20T21:55:35.855099] [INFO ] [REMOTE ] WKC: Incremental fetch complete for 2025-10 (1.20 seconds). [2025-10-20T21:55:35.860009] [INFO ] [REMOTE ] WKC: Full fetch complete (50080 records). [2025-10-20T21:55:35.860103] [DEBUG] [VALIDATOR ] WKC: Validating primary data. [2025-10-20T21:55:36.258888] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T21:55:36.255633] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-04. [2025-10-20T21:55:36.259900] [DEBUG] [VALIDATOR ] WKC: All expected trading days are present. [2025-10-20T21:55:36.262637] [DEBUG] [VALIDATOR ] WKC: Primary data validation passed. [2025-10-20T21:55:36.263724] [INFO ] [MAIN ] WKC: Fetched primary data is valid. [2025-10-20T21:55:36.263786] [DEBUG] [LOCAL ] WKC: Saving primary data. [2025-10-20T21:55:36.264909] [DEBUG] [LOCAL ] WKC: Trimmed 50080 records to 34592 (keeping 200 days of data) [2025-10-20T21:55:36.471278] [DEBUG] [LOCAL ] WKC: Primary data saved (34592 records). [2025-10-20T21:55:36.471408] [INFO ] [MAIN ] WKC: Full history saved. [2025-10-20T21:55:36.471483] [INFO ] [MAIN ] Update Progress: 2298/2367 symbols completed. WKC took 33.6s. Estimated remaining: 8m 52s. Rate: 7.8 symbols/min [2025-10-20T21:55:36.657372] [INFO ] [MAIN ] WLY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:36.686938] [DEBUG] [LOCAL ] WLY: Loaded 30295 existing records from primary storage. [2025-10-20T21:55:36.687240] [INFO ] [MAIN ] WLY: Loaded 30295 existing records. [2025-10-20T21:55:36.871234] [WARN ] [MAIN ] WLY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:36.871408] [INFO ] [MAIN ] WLY: Wiping existing primary data before full refresh. [2025-10-20T21:55:36.871453] [INFO ] [LOCAL ] WLY: Starting deletion of primary data. [2025-10-20T21:55:36.871631] [DEBUG] [LOCAL ] WLY: Deleted primary data file. [2025-10-20T21:55:36.871715] [INFO ] [MAIN ] WLY: Starting full primary refresh. [2025-10-20T21:55:37.058851] [INFO ] [REMOTE ] WLKP: Fetching primary data for 2025-10. [2025-10-20T21:55:37.225442] [INFO ] [REMOTE ] WLY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:37.795960] [INFO ] [REMOTE ] WLKP: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T21:55:37.797832] [INFO ] [REMOTE ] WLKP: Full fetch complete (10081 records). [2025-10-20T21:55:37.797925] [DEBUG] [VALIDATOR ] WLKP: Validating primary data. [2025-10-20T21:55:37.856526] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-01. [2025-10-20T21:55:38.172541] [DEBUG] [VALIDATOR ] WLKP: All expected trading days are present. [2025-10-20T21:55:38.172674] [DEBUG] [VALIDATOR ] WLKP: Primary data validation passed. [2025-10-20T21:55:38.173010] [INFO ] [MAIN ] WLKP: Fetched primary data is valid. [2025-10-20T21:55:38.173086] [DEBUG] [LOCAL ] WLKP: Saving primary data. [2025-10-20T21:55:38.195943] [DEBUG] [LOCAL ] WLKP: Trimmed 10081 records to 7204 (keeping 200 days of data) [2025-10-20T21:55:38.196720] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T21:55:38.248547] [DEBUG] [LOCAL ] WLKP: Primary data saved (7204 records). [2025-10-20T21:55:38.248697] [INFO ] [MAIN ] WLKP: Full history saved. [2025-10-20T21:55:38.248792] [INFO ] [MAIN ] Update Progress: 2299/2367 symbols completed. WLKP took 15.3s. Estimated remaining: 8m 45s. Rate: 7.8 symbols/min [2025-10-20T21:55:38.327733] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-10-20T21:55:38.422178] [INFO ] [MAIN ] WM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:38.472962] [DEBUG] [LOCAL ] WM: Loaded 51698 existing records from primary storage. [2025-10-20T21:55:38.473353] [INFO ] [MAIN ] WM: Loaded 51698 existing records. [2025-10-20T21:55:38.655671] [WARN ] [MAIN ] WM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:38.655791] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-02. [2025-10-20T21:55:38.657221] [INFO ] [MAIN ] WM: Wiping existing primary data before full refresh. [2025-10-20T21:55:38.657299] [INFO ] [LOCAL ] WM: Starting deletion of primary data. [2025-10-20T21:55:38.657609] [DEBUG] [LOCAL ] WM: Deleted primary data file. [2025-10-20T21:55:38.657714] [INFO ] [MAIN ] WM: Starting full primary refresh. [2025-10-20T21:55:39.006901] [INFO ] [REMOTE ] WM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:55:39.053068] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-02 (0.40 seconds). [2025-10-20T21:55:39.451388] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-01. [2025-10-20T21:55:40.251388] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-05. [2025-10-20T21:55:41.051354] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-03. [2025-10-20T21:55:41.312255] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-01 (1.86 seconds). [2025-10-20T21:55:41.402483] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-03 (0.35 seconds). [2025-10-20T21:55:41.851404] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-04. [2025-10-20T21:55:41.937771] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-05 (1.69 seconds). [2025-10-20T21:55:42.284716] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-04 (0.43 seconds). [2025-10-20T21:55:42.651363] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-02. [2025-10-20T21:55:43.451429] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-01. [2025-10-20T21:55:44.251357] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-05. [2025-10-20T21:55:44.486235] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T21:55:44.552157] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-01 (1.10 seconds). [2025-10-20T21:55:44.560717] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-05 (0.31 seconds). [2025-10-20T21:55:45.051446] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-06. [2025-10-20T21:55:45.851417] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-02. [2025-10-20T21:55:46.652057] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:55:46.652189] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-06. [2025-10-20T21:55:46.708933] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T21:55:47.002758] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-06 (0.35 seconds). [2025-10-20T21:55:47.451380] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-03. [2025-10-20T21:55:48.251374] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-07. [2025-10-20T21:55:48.816146] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-03 (1.36 seconds). [2025-10-20T21:55:49.051402] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-04. [2025-10-20T21:55:49.851388] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-03. [2025-10-20T21:55:49.925102] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-07 (1.67 seconds). [2025-10-20T21:55:50.120362] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-04 (1.07 seconds). [2025-10-20T21:55:50.651378] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-07. [2025-10-20T21:55:51.081783] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-07 (0.43 seconds). [2025-10-20T21:55:51.451354] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-08. [2025-10-20T21:55:51.890725] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-03 (2.04 seconds). [2025-10-20T21:55:52.251376] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-04. [2025-10-20T21:55:53.051371] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-08. [2025-10-20T21:55:53.092313] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T21:55:53.437355] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-08 (0.39 seconds). [2025-10-20T21:55:53.851397] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-09. [2025-10-20T21:55:54.029145] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:55:54.651353] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-05. [2025-10-20T21:55:55.451383] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-05. [2025-10-20T21:55:55.611444] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-09 (1.76 seconds). [2025-10-20T21:55:56.084192] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T21:55:56.251367] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-06. [2025-10-20T21:55:57.051381] [INFO ] [REMOTE ] WLK: Fetching primary data for 2025-10. [2025-10-20T21:55:57.263979] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T21:55:57.601122] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T21:55:57.851389] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-06. [2025-10-20T21:55:58.633124] [INFO ] [REMOTE ] WLK: Incremental fetch complete for 2025-10 (1.58 seconds). [2025-10-20T21:55:58.640085] [INFO ] [REMOTE ] WLK: Full fetch complete (63939 records). [2025-10-20T21:55:58.640191] [DEBUG] [VALIDATOR ] WLK: Validating primary data. [2025-10-20T21:55:58.661256] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-09. [2025-10-20T21:55:59.016788] [DEBUG] [VALIDATOR ] WLK: All expected trading days are present. [2025-10-20T21:55:59.016878] [DEBUG] [VALIDATOR ] WLK: Primary data validation passed. [2025-10-20T21:55:59.017971] [INFO ] [MAIN ] WLK: Fetched primary data is valid. [2025-10-20T21:55:59.018034] [DEBUG] [LOCAL ] WLK: Saving primary data. [2025-10-20T21:55:59.019266] [DEBUG] [LOCAL ] WLK: Trimmed 63939 records to 45903 (keeping 200 days of data) [2025-10-20T21:55:59.137140] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-09 (0.48 seconds). [2025-10-20T21:55:59.299140] [DEBUG] [LOCAL ] WLK: Primary data saved (45903 records). [2025-10-20T21:55:59.299423] [INFO ] [MAIN ] WLK: Full history saved. [2025-10-20T21:55:59.299543] [INFO ] [MAIN ] Update Progress: 2300/2367 symbols completed. WLK took 36.5s. Estimated remaining: 8m 37s. Rate: 7.8 symbols/min [2025-10-20T21:55:59.452680] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-07. [2025-10-20T21:55:59.473515] [INFO ] [MAIN ] WMB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:55:59.504381] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:55:59.555544] [DEBUG] [LOCAL ] WMB: Loaded 54452 existing records from primary storage. [2025-10-20T21:55:59.555900] [INFO ] [MAIN ] WMB: Loaded 54452 existing records. [2025-10-20T21:55:59.728759] [WARN ] [MAIN ] WMB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:55:59.728972] [INFO ] [MAIN ] WMB: Wiping existing primary data before full refresh. [2025-10-20T21:55:59.729020] [INFO ] [LOCAL ] WMB: Starting deletion of primary data. [2025-10-20T21:55:59.729212] [DEBUG] [LOCAL ] WMB: Deleted primary data file. [2025-10-20T21:55:59.729250] [INFO ] [MAIN ] WMB: Starting full primary refresh. [2025-10-20T21:56:00.075271] [INFO ] [REMOTE ] WMB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:00.251390] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-01. [2025-10-20T21:56:01.057493] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-07. [2025-10-20T21:56:01.071415] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-07 (1.62 seconds). [2025-10-20T21:56:01.851349] [INFO ] [REMOTE ] WLTG: Fetching primary data for 2025-10. [2025-10-20T21:56:01.960428] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-01 (1.71 seconds). [2025-10-20T21:56:02.198507] [INFO ] [REMOTE ] WLTG: Incremental fetch complete for 2025-10 (0.35 seconds). [2025-10-20T21:56:02.199295] [INFO ] [REMOTE ] WLTG: Full fetch complete (1679 records). [2025-10-20T21:56:02.199362] [DEBUG] [VALIDATOR ] WLTG: Validating primary data. [2025-10-20T21:56:02.380453] [WARN ] [VALIDATOR ] WLTG: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T21:56:02.380681] [FATAL] [MAIN ] WLTG: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:56:02.380743] [WARN ] [MAIN ] WLTG: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:56:02.380785] [INFO ] [LOCAL ] WLTG: Starting deletion of primary data. [2025-10-20T21:56:02.380838] [DEBUG] [LOCAL ] WLTG: No primary data file found to delete. [2025-10-20T21:56:02.380873] [INFO ] [LOCAL ] WLTG: Starting deletion of derivative data. [2025-10-20T21:56:02.380912] [DEBUG] [LOCAL ] WLTG: No derivative data file found to delete. [2025-10-20T21:56:02.380965] [INFO ] [MAIN ] Update Progress: 2301/2367 symbols completed. WLTG took 27.7s. Estimated remaining: 8m 29s. Rate: 7.8 symbols/min [2025-10-20T21:56:02.556749] [INFO ] [MAIN ] WMK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:02.572327] [DEBUG] [LOCAL ] WMK: Loaded 14886 existing records from primary storage. [2025-10-20T21:56:02.573182] [INFO ] [MAIN ] WMK: Loaded 14886 existing records. [2025-10-20T21:56:02.651760] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-02. [2025-10-20T21:56:02.760674] [WARN ] [MAIN ] WMK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:02.760884] [INFO ] [MAIN ] WMK: Wiping existing primary data before full refresh. [2025-10-20T21:56:02.760931] [INFO ] [LOCAL ] WMK: Starting deletion of primary data. [2025-10-20T21:56:02.761104] [DEBUG] [LOCAL ] WMK: Deleted primary data file. [2025-10-20T21:56:02.761144] [INFO ] [MAIN ] WMK: Starting full primary refresh. [2025-10-20T21:56:03.118923] [INFO ] [REMOTE ] WMK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:03.153679] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-07 (2.10 seconds). [2025-10-20T21:56:03.451423] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-08. [2025-10-20T21:56:04.251426] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-01. [2025-10-20T21:56:04.504410] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-02 (1.85 seconds). [2025-10-20T21:56:04.567111] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-08 (1.12 seconds). [2025-10-20T21:56:05.051503] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-09. [2025-10-20T21:56:05.210422] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T21:56:05.851389] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-02. [2025-10-20T21:56:06.306022] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-09 (1.25 seconds). [2025-10-20T21:56:06.655628] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-08. [2025-10-20T21:56:06.658538] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-02 (0.81 seconds). [2025-10-20T21:56:07.451436] [INFO ] [REMOTE ] WLY: Fetching primary data for 2025-10. [2025-10-20T21:56:08.251355] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-03. [2025-10-20T21:56:08.540388] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-08 (1.88 seconds). [2025-10-20T21:56:08.566279] [INFO ] [REMOTE ] WLY: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:56:08.571062] [INFO ] [REMOTE ] WLY: Full fetch complete (42951 records). [2025-10-20T21:56:08.571174] [DEBUG] [VALIDATOR ] WLY: Validating primary data. [2025-10-20T21:56:08.941129] [DEBUG] [VALIDATOR ] WLY: All expected trading days are present. [2025-10-20T21:56:08.941218] [DEBUG] [VALIDATOR ] WLY: Primary data validation passed. [2025-10-20T21:56:08.942077] [INFO ] [MAIN ] WLY: Fetched primary data is valid. [2025-10-20T21:56:08.942172] [DEBUG] [LOCAL ] WLY: Saving primary data. [2025-10-20T21:56:08.943137] [DEBUG] [LOCAL ] WLY: Trimmed 42951 records to 29783 (keeping 200 days of data) [2025-10-20T21:56:09.059072] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-03. [2025-10-20T21:56:09.122446] [DEBUG] [LOCAL ] WLY: Primary data saved (29783 records). [2025-10-20T21:56:09.123197] [INFO ] [MAIN ] WLY: Full history saved. [2025-10-20T21:56:09.123300] [INFO ] [MAIN ] Update Progress: 2302/2367 symbols completed. WLY took 32.7s. Estimated remaining: 8m 22s. Rate: 7.8 symbols/min [2025-10-20T21:56:09.305696] [INFO ] [MAIN ] WMS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:09.345891] [DEBUG] [LOCAL ] WMS: Loaded 40206 existing records from primary storage. [2025-10-20T21:56:09.346194] [INFO ] [MAIN ] WMS: Loaded 40206 existing records. [2025-10-20T21:56:09.526110] [WARN ] [MAIN ] WMS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:09.526318] [INFO ] [MAIN ] WMS: Wiping existing primary data before full refresh. [2025-10-20T21:56:09.526367] [INFO ] [LOCAL ] WMS: Starting deletion of primary data. [2025-10-20T21:56:09.526587] [DEBUG] [LOCAL ] WMS: Deleted primary data file. [2025-10-20T21:56:09.526678] [INFO ] [MAIN ] WMS: Starting full primary refresh. [2025-10-20T21:56:09.857832] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-09. [2025-10-20T21:56:09.886489] [INFO ] [REMOTE ] WMS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:10.038811] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-03 (0.98 seconds). [2025-10-20T21:56:10.186786] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-03 (1.94 seconds). [2025-10-20T21:56:10.651331] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-04. [2025-10-20T21:56:11.451426] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-01. [2025-10-20T21:56:11.901474] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-09 (2.04 seconds). [2025-10-20T21:56:12.251372] [INFO ] [REMOTE ] WM: Fetching primary data for 2025-10. [2025-10-20T21:56:12.735611] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-04 (2.08 seconds). [2025-10-20T21:56:12.943057] [INFO ] [REMOTE ] WM: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T21:56:12.970377] [INFO ] [REMOTE ] WM: Full fetch complete (73858 records). [2025-10-20T21:56:12.970624] [DEBUG] [VALIDATOR ] WM: Validating primary data. [2025-10-20T21:56:12.972573] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-01 (1.52 seconds). [2025-10-20T21:56:13.056458] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-04. [2025-10-20T21:56:13.361999] [DEBUG] [VALIDATOR ] WM: All expected trading days are present. [2025-10-20T21:56:13.362075] [DEBUG] [VALIDATOR ] WM: Primary data validation passed. [2025-10-20T21:56:13.363128] [INFO ] [MAIN ] WM: Fetched primary data is valid. [2025-10-20T21:56:13.363189] [DEBUG] [LOCAL ] WM: Saving primary data. [2025-10-20T21:56:13.364451] [DEBUG] [LOCAL ] WM: Trimmed 73858 records to 50913 (keeping 200 days of data) [2025-10-20T21:56:13.669085] [DEBUG] [LOCAL ] WM: Primary data saved (50913 records). [2025-10-20T21:56:13.669209] [INFO ] [MAIN ] WM: Full history saved. [2025-10-20T21:56:13.669282] [INFO ] [MAIN ] Update Progress: 2303/2367 symbols completed. WM took 35.4s. Estimated remaining: 8m 14s. Rate: 7.8 symbols/min [2025-10-20T21:56:13.834734] [INFO ] [MAIN ] WMT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:13.861034] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-02. [2025-10-20T21:56:13.885043] [DEBUG] [LOCAL ] WMT: Loaded 54580 existing records from primary storage. [2025-10-20T21:56:13.885360] [INFO ] [MAIN ] WMT: Loaded 54580 existing records. [2025-10-20T21:56:14.064389] [WARN ] [MAIN ] WMT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:14.064624] [INFO ] [MAIN ] WMT: Wiping existing primary data before full refresh. [2025-10-20T21:56:14.064711] [INFO ] [LOCAL ] WMT: Starting deletion of primary data. [2025-10-20T21:56:14.064943] [DEBUG] [LOCAL ] WMT: Deleted primary data file. [2025-10-20T21:56:14.064991] [INFO ] [MAIN ] WMT: Starting full primary refresh. [2025-10-20T21:56:14.373144] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-04 (1.32 seconds). [2025-10-20T21:56:14.462771] [INFO ] [REMOTE ] WMT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:14.651428] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-05. [2025-10-20T21:56:15.396792] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-02 (1.54 seconds). [2025-10-20T21:56:15.450018] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-05 (0.80 seconds). [2025-10-20T21:56:15.451444] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-05. [2025-10-20T21:56:16.251395] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-03. [2025-10-20T21:56:17.051388] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-06. [2025-10-20T21:56:17.496483] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-05 (2.05 seconds). [2025-10-20T21:56:17.811983] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T21:56:17.851336] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-04. [2025-10-20T21:56:17.886007] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-06 (0.83 seconds). [2025-10-20T21:56:18.651379] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-06. [2025-10-20T21:56:19.318762] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T21:56:19.451423] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-05. [2025-10-20T21:56:20.251444] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-01. [2025-10-20T21:56:20.561011] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-06 (1.91 seconds). [2025-10-20T21:56:21.051426] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-07. [2025-10-20T21:56:21.083052] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T21:56:21.851347] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-06. [2025-10-20T21:56:21.900278] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-07 (0.85 seconds). [2025-10-20T21:56:22.602504] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-01 (2.35 seconds). [2025-10-20T21:56:22.651339] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-02. [2025-10-20T21:56:23.396623] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T21:56:23.451334] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-07. [2025-10-20T21:56:24.251386] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-08. [2025-10-20T21:56:25.029834] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T21:56:25.061335] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-07. [2025-10-20T21:56:25.067018] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-02 (2.42 seconds). [2025-10-20T21:56:25.086639] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T21:56:25.851389] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-09. [2025-10-20T21:56:26.651425] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-03. [2025-10-20T21:56:26.889962] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T21:56:27.082535] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-07 (2.02 seconds). [2025-10-20T21:56:27.451365] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-08. [2025-10-20T21:56:28.251409] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-08. [2025-10-20T21:56:29.051351] [INFO ] [REMOTE ] WMK: Fetching primary data for 2025-10. [2025-10-20T21:56:29.392054] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-08 (1.94 seconds). [2025-10-20T21:56:29.393715] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-03 (2.74 seconds). [2025-10-20T21:56:29.830460] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-08 (1.58 seconds). [2025-10-20T21:56:29.851334] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-09. [2025-10-20T21:56:29.887134] [INFO ] [REMOTE ] WMK: Incremental fetch complete for 2025-10 (0.84 seconds). [2025-10-20T21:56:29.889705] [INFO ] [REMOTE ] WMK: Full fetch complete (20041 records). [2025-10-20T21:56:29.889784] [DEBUG] [VALIDATOR ] WMK: Validating primary data. [2025-10-20T21:56:30.257971] [DEBUG] [VALIDATOR ] WMK: All expected trading days are present. [2025-10-20T21:56:30.258060] [DEBUG] [VALIDATOR ] WMK: Primary data validation passed. [2025-10-20T21:56:30.258520] [INFO ] [MAIN ] WMK: Fetched primary data is valid. [2025-10-20T21:56:30.258559] [DEBUG] [LOCAL ] WMK: Saving primary data. [2025-10-20T21:56:30.259133] [DEBUG] [LOCAL ] WMK: Trimmed 20041 records to 14754 (keeping 200 days of data) [2025-10-20T21:56:30.345793] [DEBUG] [LOCAL ] WMK: Primary data saved (14754 records). [2025-10-20T21:56:30.345920] [INFO ] [MAIN ] WMK: Full history saved. [2025-10-20T21:56:30.345995] [INFO ] [MAIN ] Update Progress: 2304/2367 symbols completed. WMK took 28.0s. Estimated remaining: 8m 6s. Rate: 7.8 symbols/min [2025-10-20T21:56:30.511897] [INFO ] [MAIN ] WNC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:30.548543] [DEBUG] [LOCAL ] WNC: Loaded 39048 existing records from primary storage. [2025-10-20T21:56:30.548847] [INFO ] [MAIN ] WNC: Loaded 39048 existing records. [2025-10-20T21:56:30.656550] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-09. [2025-10-20T21:56:30.728810] [WARN ] [MAIN ] WNC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:30.728998] [INFO ] [MAIN ] WNC: Wiping existing primary data before full refresh. [2025-10-20T21:56:30.729046] [INFO ] [LOCAL ] WNC: Starting deletion of primary data. [2025-10-20T21:56:30.729245] [DEBUG] [LOCAL ] WNC: Deleted primary data file. [2025-10-20T21:56:30.729303] [INFO ] [MAIN ] WNC: Starting full primary refresh. [2025-10-20T21:56:31.108397] [INFO ] [REMOTE ] WNC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:31.451403] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-01. [2025-10-20T21:56:31.562947] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-09 (1.71 seconds). [2025-10-20T21:56:32.251353] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-04. [2025-10-20T21:56:32.657589] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-09 (2.00 seconds). [2025-10-20T21:56:32.691102] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-01 (1.24 seconds). [2025-10-20T21:56:33.051445] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-02. [2025-10-20T21:56:33.851396] [INFO ] [REMOTE ] WMB: Fetching primary data for 2025-10. [2025-10-20T21:56:34.527822] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-02 (1.48 seconds). [2025-10-20T21:56:34.609185] [INFO ] [REMOTE ] WMB: Incremental fetch complete for 2025-10 (0.76 seconds). [2025-10-20T21:56:34.616963] [INFO ] [REMOTE ] WMB: Full fetch complete (77851 records). [2025-10-20T21:56:34.617097] [DEBUG] [VALIDATOR ] WMB: Validating primary data. [2025-10-20T21:56:34.658893] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-03. [2025-10-20T21:56:35.012700] [DEBUG] [VALIDATOR ] WMB: All expected trading days are present. [2025-10-20T21:56:35.012896] [DEBUG] [VALIDATOR ] WMB: Primary data validation passed. [2025-10-20T21:56:35.014162] [INFO ] [MAIN ] WMB: Fetched primary data is valid. [2025-10-20T21:56:35.014230] [DEBUG] [LOCAL ] WMB: Saving primary data. [2025-10-20T21:56:35.015634] [DEBUG] [LOCAL ] WMB: Trimmed 77851 records to 53672 (keeping 200 days of data) [2025-10-20T21:56:35.161160] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-04 (2.91 seconds). [2025-10-20T21:56:35.393329] [DEBUG] [LOCAL ] WMB: Primary data saved (53672 records). [2025-10-20T21:56:35.393477] [INFO ] [MAIN ] WMB: Full history saved. [2025-10-20T21:56:35.393556] [INFO ] [MAIN ] Update Progress: 2305/2367 symbols completed. WMB took 36.1s. Estimated remaining: 7m 59s. Rate: 7.8 symbols/min [2025-10-20T21:56:35.456561] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-05. [2025-10-20T21:56:35.561449] [INFO ] [MAIN ] WOLF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:35.610722] [DEBUG] [LOCAL ] WOLF: Loaded 53926 existing records from primary storage. [2025-10-20T21:56:35.611038] [INFO ] [MAIN ] WOLF: Loaded 53926 existing records. [2025-10-20T21:56:35.785164] [WARN ] [MAIN ] WOLF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:35.785364] [INFO ] [MAIN ] WOLF: Wiping existing primary data before full refresh. [2025-10-20T21:56:35.785411] [INFO ] [LOCAL ] WOLF: Starting deletion of primary data. [2025-10-20T21:56:35.785675] [DEBUG] [LOCAL ] WOLF: Deleted primary data file. [2025-10-20T21:56:35.785734] [INFO ] [MAIN ] WOLF: Starting full primary refresh. [2025-10-20T21:56:36.139717] [INFO ] [REMOTE ] WOLF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:36.251382] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-01. [2025-10-20T21:56:36.368659] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-03 (1.71 seconds). [2025-10-20T21:56:37.051417] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-04. [2025-10-20T21:56:37.851366] [INFO ] [REMOTE ] WMS: Fetching primary data for 2025-10. [2025-10-20T21:56:38.046605] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-05 (2.59 seconds). [2025-10-20T21:56:38.625000] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-01 (2.37 seconds). [2025-10-20T21:56:38.651337] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-02. [2025-10-20T21:56:38.704230] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-04 (1.65 seconds). [2025-10-20T21:56:38.835463] [INFO ] [REMOTE ] WMS: Incremental fetch complete for 2025-10 (0.98 seconds). [2025-10-20T21:56:38.841594] [INFO ] [REMOTE ] WMS: Full fetch complete (57109 records). [2025-10-20T21:56:38.841754] [DEBUG] [VALIDATOR ] WMS: Validating primary data. [2025-10-20T21:56:39.242895] [DEBUG] [VALIDATOR ] WMS: All expected trading days are present. [2025-10-20T21:56:39.242983] [DEBUG] [VALIDATOR ] WMS: Primary data validation passed. [2025-10-20T21:56:39.243944] [INFO ] [MAIN ] WMS: Fetched primary data is valid. [2025-10-20T21:56:39.244006] [DEBUG] [LOCAL ] WMS: Saving primary data. [2025-10-20T21:56:39.245104] [DEBUG] [LOCAL ] WMS: Trimmed 57109 records to 39655 (keeping 200 days of data) [2025-10-20T21:56:39.454499] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-06. [2025-10-20T21:56:39.495282] [DEBUG] [LOCAL ] WMS: Primary data saved (39655 records). [2025-10-20T21:56:39.495419] [INFO ] [MAIN ] WMS: Full history saved. [2025-10-20T21:56:39.495496] [INFO ] [MAIN ] Update Progress: 2306/2367 symbols completed. WMS took 30.4s. Estimated remaining: 7m 51s. Rate: 7.8 symbols/min [2025-10-20T21:56:39.660029] [INFO ] [MAIN ] WOR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:39.682737] [DEBUG] [LOCAL ] WOR: Loaded 24332 existing records from primary storage. [2025-10-20T21:56:39.682990] [INFO ] [MAIN ] WOR: Loaded 24332 existing records. [2025-10-20T21:56:39.857088] [WARN ] [MAIN ] WOR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:39.857312] [INFO ] [MAIN ] WOR: Wiping existing primary data before full refresh. [2025-10-20T21:56:39.857357] [INFO ] [LOCAL ] WOR: Starting deletion of primary data. [2025-10-20T21:56:39.857523] [DEBUG] [LOCAL ] WOR: Deleted primary data file. [2025-10-20T21:56:39.857560] [INFO ] [MAIN ] WOR: Starting full primary refresh. [2025-10-20T21:56:40.239957] [INFO ] [REMOTE ] WOR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:40.251335] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-01. [2025-10-20T21:56:40.709615] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T21:56:41.051382] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-03. [2025-10-20T21:56:41.508208] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-01 (1.26 seconds). [2025-10-20T21:56:41.857298] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-02. [2025-10-20T21:56:41.877391] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-06 (2.42 seconds). [2025-10-20T21:56:42.651374] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-05. [2025-10-20T21:56:42.961449] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-02 (1.10 seconds). [2025-10-20T21:56:43.287462] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-03 (2.24 seconds). [2025-10-20T21:56:43.451357] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-03. [2025-10-20T21:56:44.205576] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T21:56:44.251358] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-04. [2025-10-20T21:56:44.803023] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-03 (1.35 seconds). [2025-10-20T21:56:45.051377] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-07. [2025-10-20T21:56:45.851385] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-04. [2025-10-20T21:56:46.600178] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-04 (2.35 seconds). [2025-10-20T21:56:46.651349] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-05. [2025-10-20T21:56:47.241676] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-04 (1.39 seconds). [2025-10-20T21:56:47.451417] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-06. [2025-10-20T21:56:47.691381] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-07 (2.64 seconds). [2025-10-20T21:56:48.251429] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-08. [2025-10-20T21:56:48.565611] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:56:49.051345] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-07. [2025-10-20T21:56:49.141394] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-05 (2.49 seconds). [2025-10-20T21:56:49.851373] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-06. [2025-10-20T21:56:50.507836] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T21:56:50.651365] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-08. [2025-10-20T21:56:50.774560] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-08 (2.52 seconds). [2025-10-20T21:56:51.451383] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-09. [2025-10-20T21:56:52.076154] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-06 (2.22 seconds). [2025-10-20T21:56:52.113015] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T21:56:52.251374] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-09. [2025-10-20T21:56:53.051367] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-05. [2025-10-20T21:56:53.507999] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-09 (1.26 seconds). [2025-10-20T21:56:53.544230] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T21:56:53.804861] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-05 (0.75 seconds). [2025-10-20T21:56:53.851407] [INFO ] [REMOTE ] WMT: Fetching primary data for 2025-10. [2025-10-20T21:56:54.546882] [INFO ] [REMOTE ] WMT: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T21:56:54.554990] [INFO ] [REMOTE ] WMT: Full fetch complete (77980 records). [2025-10-20T21:56:54.555125] [DEBUG] [VALIDATOR ] WMT: Validating primary data. [2025-10-20T21:56:54.658782] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-06. [2025-10-20T21:56:54.948873] [DEBUG] [VALIDATOR ] WMT: All expected trading days are present. [2025-10-20T21:56:54.948957] [DEBUG] [VALIDATOR ] WMT: Primary data validation passed. [2025-10-20T21:56:54.950083] [INFO ] [MAIN ] WMT: Fetched primary data is valid. [2025-10-20T21:56:54.950145] [DEBUG] [LOCAL ] WMT: Saving primary data. [2025-10-20T21:56:54.951508] [DEBUG] [LOCAL ] WMT: Trimmed 77980 records to 53800 (keeping 200 days of data) [2025-10-20T21:56:55.282468] [DEBUG] [LOCAL ] WMT: Primary data saved (53800 records). [2025-10-20T21:56:55.282602] [INFO ] [MAIN ] WMT: Full history saved. [2025-10-20T21:56:55.282697] [INFO ] [MAIN ] Update Progress: 2307/2367 symbols completed. WMT took 41.6s. Estimated remaining: 7m 43s. Rate: 7.8 symbols/min [2025-10-20T21:56:55.451673] [INFO ] [MAIN ] WOW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:56:55.451740] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-07. [2025-10-20T21:56:55.483624] [DEBUG] [LOCAL ] WOW: Loaded 31535 existing records from primary storage. [2025-10-20T21:56:55.483917] [INFO ] [MAIN ] WOW: Loaded 31535 existing records. [2025-10-20T21:56:55.663151] [WARN ] [MAIN ] WOW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:56:55.663435] [INFO ] [MAIN ] WOW: Wiping existing primary data before full refresh. [2025-10-20T21:56:55.663494] [INFO ] [LOCAL ] WOW: Starting deletion of primary data. [2025-10-20T21:56:55.663715] [DEBUG] [LOCAL ] WOW: Deleted primary data file. [2025-10-20T21:56:55.663770] [INFO ] [MAIN ] WOW: Starting full primary refresh. [2025-10-20T21:56:55.961694] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-06 (1.30 seconds). [2025-10-20T21:56:56.042976] [INFO ] [REMOTE ] WOW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:56:56.251375] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-01. [2025-10-20T21:56:57.051351] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-07. [2025-10-20T21:56:57.411781] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-01 (1.16 seconds). [2025-10-20T21:56:57.851410] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-02. [2025-10-20T21:56:57.919262] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-07 (2.47 seconds). [2025-10-20T21:56:58.413227] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T21:56:58.651372] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-08. [2025-10-20T21:56:58.975467] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-02 (1.12 seconds). [2025-10-20T21:56:59.451489] [INFO ] [REMOTE ] WNC: Fetching primary data for 2025-10. [2025-10-20T21:57:00.251395] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-08. [2025-10-20T21:57:00.776501] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-08 (2.13 seconds). [2025-10-20T21:57:00.801696] [INFO ] [REMOTE ] WNC: Incremental fetch complete for 2025-10 (1.35 seconds). [2025-10-20T21:57:00.807033] [INFO ] [REMOTE ] WNC: Full fetch complete (57844 records). [2025-10-20T21:57:00.807135] [DEBUG] [VALIDATOR ] WNC: Validating primary data. [2025-10-20T21:57:01.056498] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-03. [2025-10-20T21:57:01.222870] [DEBUG] [VALIDATOR ] WNC: All expected trading days are present. [2025-10-20T21:57:01.222965] [DEBUG] [VALIDATOR ] WNC: Primary data validation passed. [2025-10-20T21:57:01.223950] [INFO ] [MAIN ] WNC: Fetched primary data is valid. [2025-10-20T21:57:01.224014] [DEBUG] [LOCAL ] WNC: Saving primary data. [2025-10-20T21:57:01.225198] [DEBUG] [LOCAL ] WNC: Trimmed 57844 records to 38400 (keeping 200 days of data) [2025-10-20T21:57:01.461607] [DEBUG] [LOCAL ] WNC: Primary data saved (38400 records). [2025-10-20T21:57:01.462027] [INFO ] [MAIN ] WNC: Full history saved. [2025-10-20T21:57:01.462146] [INFO ] [MAIN ] Update Progress: 2308/2367 symbols completed. WNC took 31.1s. Estimated remaining: 7m 35s. Rate: 7.8 symbols/min [2025-10-20T21:57:01.628803] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T21:57:01.676515] [INFO ] [MAIN ] WPC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:01.720029] [DEBUG] [LOCAL ] WPC: Loaded 48956 existing records from primary storage. [2025-10-20T21:57:01.720350] [INFO ] [MAIN ] WPC: Loaded 48956 existing records. [2025-10-20T21:57:01.856449] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-09. [2025-10-20T21:57:01.925138] [WARN ] [MAIN ] WPC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:01.925348] [INFO ] [MAIN ] WPC: Wiping existing primary data before full refresh. [2025-10-20T21:57:01.925394] [INFO ] [LOCAL ] WPC: Starting deletion of primary data. [2025-10-20T21:57:01.925764] [DEBUG] [LOCAL ] WPC: Deleted primary data file. [2025-10-20T21:57:01.925822] [INFO ] [MAIN ] WPC: Starting full primary refresh. [2025-10-20T21:57:02.286152] [INFO ] [REMOTE ] WPC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:02.379475] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T21:57:02.651423] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-01. [2025-10-20T21:57:03.451452] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-09. [2025-10-20T21:57:04.256680] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-04. [2025-10-20T21:57:04.285058] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-01 (1.63 seconds). [2025-10-20T21:57:04.436444] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-09 (2.58 seconds). [2025-10-20T21:57:04.495710] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-09 (1.04 seconds). [2025-10-20T21:57:05.051387] [INFO ] [REMOTE ] WOR: Fetching primary data for 2025-10. [2025-10-20T21:57:05.543074] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-04 (1.29 seconds). [2025-10-20T21:57:05.851397] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-02. [2025-10-20T21:57:06.162269] [INFO ] [REMOTE ] WOR: Incremental fetch complete for 2025-10 (1.11 seconds). [2025-10-20T21:57:06.166004] [INFO ] [REMOTE ] WOR: Full fetch complete (36351 records). [2025-10-20T21:57:06.166108] [DEBUG] [VALIDATOR ] WOR: Validating primary data. [2025-10-20T21:57:06.541670] [DEBUG] [VALIDATOR ] WOR: All expected trading days are present. [2025-10-20T21:57:06.541786] [DEBUG] [VALIDATOR ] WOR: Primary data validation passed. [2025-10-20T21:57:06.542524] [INFO ] [MAIN ] WOR: Fetched primary data is valid. [2025-10-20T21:57:06.542564] [DEBUG] [LOCAL ] WOR: Saving primary data. [2025-10-20T21:57:06.543301] [DEBUG] [LOCAL ] WOR: Trimmed 36351 records to 23740 (keeping 200 days of data) [2025-10-20T21:57:06.654679] [INFO ] [REMOTE ] WOLF: Fetching primary data for 2025-10. [2025-10-20T21:57:06.685440] [DEBUG] [LOCAL ] WOR: Primary data saved (23740 records). [2025-10-20T21:57:06.685542] [INFO ] [MAIN ] WOR: Full history saved. [2025-10-20T21:57:06.685614] [INFO ] [MAIN ] Update Progress: 2309/2367 symbols completed. WOR took 27.2s. Estimated remaining: 7m 28s. Rate: 7.8 symbols/min [2025-10-20T21:57:06.852794] [INFO ] [MAIN ] WPM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:06.899802] [DEBUG] [LOCAL ] WPM: Loaded 53554 existing records from primary storage. [2025-10-20T21:57:06.900099] [INFO ] [MAIN ] WPM: Loaded 53554 existing records. [2025-10-20T21:57:07.077017] [WARN ] [MAIN ] WPM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:07.077189] [INFO ] [MAIN ] WPM: Wiping existing primary data before full refresh. [2025-10-20T21:57:07.077240] [INFO ] [LOCAL ] WPM: Starting deletion of primary data. [2025-10-20T21:57:07.077614] [DEBUG] [LOCAL ] WPM: Deleted primary data file. [2025-10-20T21:57:07.077701] [INFO ] [MAIN ] WPM: Starting full primary refresh. [2025-10-20T21:57:07.449506] [INFO ] [REMOTE ] WPM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:07.451330] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-01. [2025-10-20T21:57:07.627486] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-02 (1.78 seconds). [2025-10-20T21:57:08.251378] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-05. [2025-10-20T21:57:08.958458] [INFO ] [REMOTE ] WOLF: Incremental fetch complete for 2025-10 (2.30 seconds). [2025-10-20T21:57:08.966296] [INFO ] [REMOTE ] WOLF: Full fetch complete (77300 records). [2025-10-20T21:57:08.966439] [DEBUG] [VALIDATOR ] WOLF: Validating primary data. [2025-10-20T21:57:09.059821] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-03. [2025-10-20T21:57:09.377100] [DEBUG] [VALIDATOR ] WOLF: All expected trading days are present. [2025-10-20T21:57:09.389914] [DEBUG] [VALIDATOR ] WOLF: Primary data validation passed. [2025-10-20T21:57:09.392430] [INFO ] [MAIN ] WOLF: Fetched primary data is valid. [2025-10-20T21:57:09.392519] [DEBUG] [LOCAL ] WOLF: Saving primary data. [2025-10-20T21:57:09.390932] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-05 (1.14 seconds). [2025-10-20T21:57:09.425050] [DEBUG] [LOCAL ] WOLF: Trimmed 77300 records to 53128 (keeping 200 days of data) [2025-10-20T21:57:09.427073] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-01 (1.98 seconds). [2025-10-20T21:57:09.732198] [DEBUG] [LOCAL ] WOLF: Primary data saved (53128 records). [2025-10-20T21:57:09.732338] [INFO ] [MAIN ] WOLF: Full history saved. [2025-10-20T21:57:09.732423] [INFO ] [MAIN ] Update Progress: 2310/2367 symbols completed. WOLF took 34.3s. Estimated remaining: 7m 20s. Rate: 7.8 symbols/min [2025-10-20T21:57:09.856523] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-02. [2025-10-20T21:57:09.905733] [INFO ] [MAIN ] WPP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:09.938346] [DEBUG] [LOCAL ] WPP: Loaded 34978 existing records from primary storage. [2025-10-20T21:57:09.938614] [INFO ] [MAIN ] WPP: Loaded 34978 existing records. [2025-10-20T21:57:10.111589] [WARN ] [MAIN ] WPP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:10.111809] [INFO ] [MAIN ] WPP: Wiping existing primary data before full refresh. [2025-10-20T21:57:10.111855] [INFO ] [LOCAL ] WPP: Starting deletion of primary data. [2025-10-20T21:57:10.112037] [DEBUG] [LOCAL ] WPP: Deleted primary data file. [2025-10-20T21:57:10.112074] [INFO ] [MAIN ] WPP: Starting full primary refresh. [2025-10-20T21:57:10.460086] [INFO ] [REMOTE ] WPP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:10.637866] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-03 (1.58 seconds). [2025-10-20T21:57:10.651329] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-04. [2025-10-20T21:57:11.451356] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-06. [2025-10-20T21:57:11.629735] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-02 (1.77 seconds). [2025-10-20T21:57:12.251351] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-03. [2025-10-20T21:57:12.434739] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-04 (1.78 seconds). [2025-10-20T21:57:12.566236] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-06 (1.11 seconds). [2025-10-20T21:57:13.051426] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-07. [2025-10-20T21:57:13.851388] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-05. [2025-10-20T21:57:13.895997] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-03 (1.64 seconds). [2025-10-20T21:57:14.262061] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-07 (1.21 seconds). [2025-10-20T21:57:14.651384] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-01. [2025-10-20T21:57:15.451388] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-04. [2025-10-20T21:57:15.635055] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-05 (1.78 seconds). [2025-10-20T21:57:15.771516] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-01 (1.12 seconds). [2025-10-20T21:57:16.251429] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-02. [2025-10-20T21:57:17.051351] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-08. [2025-10-20T21:57:17.390082] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T21:57:17.468512] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-04 (2.02 seconds). [2025-10-20T21:57:17.851387] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-06. [2025-10-20T21:57:18.548171] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T21:57:18.651367] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-09. [2025-10-20T21:57:19.452114] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-03. [2025-10-20T21:57:19.452006] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-06 (1.60 seconds). [2025-10-20T21:57:19.596249] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T21:57:20.251354] [INFO ] [REMOTE ] WOW: Fetching primary data for 2025-10. [2025-10-20T21:57:20.882923] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-03 (1.43 seconds). [2025-10-20T21:57:21.051462] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-07. [2025-10-20T21:57:21.555917] [INFO ] [REMOTE ] WOW: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T21:57:21.561185] [INFO ] [REMOTE ] WOW: Full fetch complete (42927 records). [2025-10-20T21:57:21.561322] [DEBUG] [VALIDATOR ] WOW: Validating primary data. [2025-10-20T21:57:21.856504] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-04. [2025-10-20T21:57:21.930286] [DEBUG] [VALIDATOR ] WOW: All expected trading days are present. [2025-10-20T21:57:21.930356] [DEBUG] [VALIDATOR ] WOW: Primary data validation passed. [2025-10-20T21:57:21.931216] [INFO ] [MAIN ] WOW: Fetched primary data is valid. [2025-10-20T21:57:21.931296] [DEBUG] [LOCAL ] WOW: Saving primary data. [2025-10-20T21:57:21.932205] [DEBUG] [LOCAL ] WOW: Trimmed 42927 records to 31259 (keeping 200 days of data) [2025-10-20T21:57:22.101105] [DEBUG] [LOCAL ] WOW: Primary data saved (31259 records). [2025-10-20T21:57:22.101238] [INFO ] [MAIN ] WOW: Full history saved. [2025-10-20T21:57:22.101314] [INFO ] [MAIN ] Update Progress: 2311/2367 symbols completed. WOW took 26.8s. Estimated remaining: 7m 12s. Rate: 7.8 symbols/min [2025-10-20T21:57:22.267179] [INFO ] [MAIN ] WRB: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:22.313874] [DEBUG] [LOCAL ] WRB: Loaded 51552 existing records from primary storage. [2025-10-20T21:57:22.314160] [INFO ] [MAIN ] WRB: Loaded 51552 existing records. [2025-10-20T21:57:22.488080] [WARN ] [MAIN ] WRB: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:22.488286] [INFO ] [MAIN ] WRB: Wiping existing primary data before full refresh. [2025-10-20T21:57:22.488332] [INFO ] [LOCAL ] WRB: Starting deletion of primary data. [2025-10-20T21:57:22.488511] [DEBUG] [LOCAL ] WRB: Deleted primary data file. [2025-10-20T21:57:22.488548] [INFO ] [MAIN ] WRB: Starting full primary refresh. [2025-10-20T21:57:22.660152] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-05. [2025-10-20T21:57:22.846290] [INFO ] [REMOTE ] WRB: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:22.875742] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-07 (1.82 seconds). [2025-10-20T21:57:23.393170] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-04 (1.54 seconds). [2025-10-20T21:57:23.451376] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-01. [2025-10-20T21:57:24.251372] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-05. [2025-10-20T21:57:24.586849] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-05 (1.93 seconds). [2025-10-20T21:57:24.685396] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-01 (1.23 seconds). [2025-10-20T21:57:25.051407] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-08. [2025-10-20T21:57:25.154719] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-05 (0.90 seconds). [2025-10-20T21:57:25.851377] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-02. [2025-10-20T21:57:26.651349] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-06. [2025-10-20T21:57:26.749103] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T21:57:27.451357] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-09. [2025-10-20T21:57:27.566886] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-02 (1.72 seconds). [2025-10-20T21:57:28.251404] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-06. [2025-10-20T21:57:28.309503] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:57:29.051399] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-07. [2025-10-20T21:57:29.127453] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-09 (1.68 seconds). [2025-10-20T21:57:29.676831] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:57:29.851442] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-03. [2025-10-20T21:57:30.651365] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-07. [2025-10-20T21:57:30.981998] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-07 (1.93 seconds). [2025-10-20T21:57:31.451358] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-08. [2025-10-20T21:57:31.603091] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-03 (1.75 seconds). [2025-10-20T21:57:32.222193] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-07 (1.57 seconds). [2025-10-20T21:57:32.251360] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-08. [2025-10-20T21:57:33.051437] [INFO ] [REMOTE ] WPC: Fetching primary data for 2025-10. [2025-10-20T21:57:33.369798] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-08 (1.92 seconds). [2025-10-20T21:57:33.701814] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-08 (1.45 seconds). [2025-10-20T21:57:33.851400] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-04. [2025-10-20T21:57:34.651382] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-09. [2025-10-20T21:57:34.718749] [INFO ] [REMOTE ] WPC: Incremental fetch complete for 2025-10 (1.67 seconds). [2025-10-20T21:57:34.726334] [INFO ] [REMOTE ] WPC: Full fetch complete (70560 records). [2025-10-20T21:57:34.726474] [DEBUG] [VALIDATOR ] WPC: Validating primary data. [2025-10-20T21:57:35.106169] [DEBUG] [VALIDATOR ] WPC: All expected trading days are present. [2025-10-20T21:57:35.106257] [DEBUG] [VALIDATOR ] WPC: Primary data validation passed. [2025-10-20T21:57:35.107372] [INFO ] [MAIN ] WPC: Fetched primary data is valid. [2025-10-20T21:57:35.107433] [DEBUG] [LOCAL ] WPC: Saving primary data. [2025-10-20T21:57:35.108780] [DEBUG] [LOCAL ] WPC: Trimmed 70560 records to 48228 (keeping 200 days of data) [2025-10-20T21:57:35.401618] [DEBUG] [LOCAL ] WPC: Primary data saved (48228 records). [2025-10-20T21:57:35.401782] [INFO ] [MAIN ] WPC: Full history saved. [2025-10-20T21:57:35.401877] [INFO ] [MAIN ] Update Progress: 2312/2367 symbols completed. WPC took 33.9s. Estimated remaining: 7m 5s. Rate: 7.8 symbols/min [2025-10-20T21:57:35.456553] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-09. [2025-10-20T21:57:35.577378] [INFO ] [MAIN ] WRBY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:35.610942] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T21:57:35.662733] [DEBUG] [LOCAL ] WRBY: Loaded 51470 existing records from primary storage. [2025-10-20T21:57:35.663060] [INFO ] [MAIN ] WRBY: Loaded 51470 existing records. [2025-10-20T21:57:35.839275] [WARN ] [MAIN ] WRBY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:35.839477] [INFO ] [MAIN ] WRBY: Wiping existing primary data before full refresh. [2025-10-20T21:57:35.839524] [INFO ] [LOCAL ] WRBY: Starting deletion of primary data. [2025-10-20T21:57:35.839749] [DEBUG] [LOCAL ] WRBY: Deleted primary data file. [2025-10-20T21:57:35.839807] [INFO ] [MAIN ] WRBY: Starting full primary refresh. [2025-10-20T21:57:36.195631] [INFO ] [REMOTE ] WRBY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:36.251340] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-01. [2025-10-20T21:57:36.681617] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-09 (2.03 seconds). [2025-10-20T21:57:37.005940] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-09 (1.55 seconds). [2025-10-20T21:57:37.051401] [INFO ] [REMOTE ] WPP: Fetching primary data for 2025-10. [2025-10-20T21:57:37.526342] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T21:57:37.851379] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-02. [2025-10-20T21:57:38.480037] [INFO ] [REMOTE ] WPP: Incremental fetch complete for 2025-10 (1.43 seconds). [2025-10-20T21:57:38.485916] [INFO ] [REMOTE ] WPP: Full fetch complete (47457 records). [2025-10-20T21:57:38.486032] [DEBUG] [VALIDATOR ] WPP: Validating primary data. [2025-10-20T21:57:38.657050] [INFO ] [REMOTE ] WPM: Fetching primary data for 2025-10. [2025-10-20T21:57:38.918074] [DEBUG] [VALIDATOR ] WPP: All expected trading days are present. [2025-10-20T21:57:38.918164] [DEBUG] [VALIDATOR ] WPP: Primary data validation passed. [2025-10-20T21:57:38.919085] [INFO ] [MAIN ] WPP: Fetched primary data is valid. [2025-10-20T21:57:38.919185] [DEBUG] [LOCAL ] WPP: Saving primary data. [2025-10-20T21:57:38.920197] [DEBUG] [LOCAL ] WPP: Trimmed 47457 records to 34370 (keeping 200 days of data) [2025-10-20T21:57:39.131539] [DEBUG] [LOCAL ] WPP: Primary data saved (34370 records). [2025-10-20T21:57:39.131707] [INFO ] [MAIN ] WPP: Full history saved. [2025-10-20T21:57:39.131799] [INFO ] [MAIN ] Update Progress: 2313/2367 symbols completed. WPP took 29.4s. Estimated remaining: 6m 57s. Rate: 7.8 symbols/min [2025-10-20T21:57:39.296487] [INFO ] [MAIN ] WS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:39.318755] [DEBUG] [LOCAL ] WS: Loaded 23730 existing records from primary storage. [2025-10-20T21:57:39.319017] [INFO ] [MAIN ] WS: Loaded 23730 existing records. [2025-10-20T21:57:39.469039] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-05. [2025-10-20T21:57:39.499877] [WARN ] [MAIN ] WS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:39.500401] [INFO ] [MAIN ] WS: Wiping existing primary data before full refresh. [2025-10-20T21:57:39.500522] [INFO ] [LOCAL ] WS: Starting deletion of primary data. [2025-10-20T21:57:39.500790] [DEBUG] [LOCAL ] WS: Deleted primary data file. [2025-10-20T21:57:39.500901] [INFO ] [MAIN ] WS: Starting full primary refresh. [2025-10-20T21:57:39.684097] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-02 (1.83 seconds). [2025-10-20T21:57:39.902107] [INFO ] [REMOTE ] WS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:40.251350] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-01. [2025-10-20T21:57:40.476256] [INFO ] [REMOTE ] WPM: Incremental fetch complete for 2025-10 (1.82 seconds). [2025-10-20T21:57:40.484032] [INFO ] [REMOTE ] WPM: Full fetch complete (76374 records). [2025-10-20T21:57:40.484152] [DEBUG] [VALIDATOR ] WPM: Validating primary data. [2025-10-20T21:57:40.876762] [DEBUG] [VALIDATOR ] WPM: All expected trading days are present. [2025-10-20T21:57:40.876835] [DEBUG] [VALIDATOR ] WPM: Primary data validation passed. [2025-10-20T21:57:40.877935] [INFO ] [MAIN ] WPM: Fetched primary data is valid. [2025-10-20T21:57:40.878004] [DEBUG] [LOCAL ] WPM: Saving primary data. [2025-10-20T21:57:40.879382] [DEBUG] [LOCAL ] WPM: Trimmed 76374 records to 52786 (keeping 200 days of data) [2025-10-20T21:57:41.055360] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-03. [2025-10-20T21:57:41.201173] [DEBUG] [LOCAL ] WPM: Primary data saved (52786 records). [2025-10-20T21:57:41.201305] [INFO ] [MAIN ] WPM: Full history saved. [2025-10-20T21:57:41.201378] [INFO ] [MAIN ] Update Progress: 2314/2367 symbols completed. WPM took 34.5s. Estimated remaining: 6m 49s. Rate: 7.8 symbols/min [2025-10-20T21:57:41.391193] [INFO ] [MAIN ] WSM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:41.411188] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-05 (1.94 seconds). [2025-10-20T21:57:41.442639] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-01 (1.19 seconds). [2025-10-20T21:57:41.486860] [DEBUG] [LOCAL ] WSM: Loaded 49738 existing records from primary storage. [2025-10-20T21:57:41.487161] [INFO ] [MAIN ] WSM: Loaded 49738 existing records. [2025-10-20T21:57:41.659262] [WARN ] [MAIN ] WSM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:41.659474] [INFO ] [MAIN ] WSM: Wiping existing primary data before full refresh. [2025-10-20T21:57:41.659519] [INFO ] [LOCAL ] WSM: Starting deletion of primary data. [2025-10-20T21:57:41.659738] [DEBUG] [LOCAL ] WSM: Deleted primary data file. [2025-10-20T21:57:41.659795] [INFO ] [MAIN ] WSM: Starting full primary refresh. [2025-10-20T21:57:41.857113] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-02. [2025-10-20T21:57:42.047082] [INFO ] [REMOTE ] WSM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:57:42.651355] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-06. [2025-10-20T21:57:42.824889] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T21:57:43.086004] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T21:57:43.451457] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-03. [2025-10-20T21:57:44.251371] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-04. [2025-10-20T21:57:44.310092] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-06 (1.66 seconds). [2025-10-20T21:57:44.738253] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-03 (1.29 seconds). [2025-10-20T21:57:45.051380] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-04. [2025-10-20T21:57:45.733638] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-04 (1.48 seconds). [2025-10-20T21:57:45.851350] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-05. [2025-10-20T21:57:46.223723] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-04 (1.17 seconds). [2025-10-20T21:57:46.651361] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-07. [2025-10-20T21:57:47.451342] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-05. [2025-10-20T21:57:47.553682] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-05 (1.70 seconds). [2025-10-20T21:57:48.189293] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-05 (0.74 seconds). [2025-10-20T21:57:48.251347] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-06. [2025-10-20T21:57:48.420580] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-07 (1.77 seconds). [2025-10-20T21:57:49.051445] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-06. [2025-10-20T21:57:49.540270] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-06 (1.29 seconds). [2025-10-20T21:57:49.851381] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-08. [2025-10-20T21:57:50.486130] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:57:50.651386] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-07. [2025-10-20T21:57:51.451417] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-01. [2025-10-20T21:57:51.506356] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-08 (1.65 seconds). [2025-10-20T21:57:52.251371] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-07. [2025-10-20T21:57:52.375378] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T21:57:53.043663] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-07 (0.79 seconds). [2025-10-20T21:57:53.051314] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-08. [2025-10-20T21:57:53.235114] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T21:57:53.851410] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-02. [2025-10-20T21:57:54.101996] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-08 (1.05 seconds). [2025-10-20T21:57:54.651378] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-09. [2025-10-20T21:57:55.451357] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-09. [2025-10-20T21:57:55.510117] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T21:57:56.251394] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-03. [2025-10-20T21:57:56.321710] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-09 (1.67 seconds). [2025-10-20T21:57:56.625935] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-09 (1.17 seconds). [2025-10-20T21:57:57.051417] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-08. [2025-10-20T21:57:57.851384] [INFO ] [REMOTE ] WS: Fetching primary data for 2025-10. [2025-10-20T21:57:58.158453] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-03 (1.91 seconds). [2025-10-20T21:57:58.651366] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-04. [2025-10-20T21:57:58.779256] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T21:57:58.920471] [INFO ] [REMOTE ] WS: Incremental fetch complete for 2025-10 (1.07 seconds). [2025-10-20T21:57:58.924134] [INFO ] [REMOTE ] WS: Full fetch complete (35719 records). [2025-10-20T21:57:58.924207] [DEBUG] [VALIDATOR ] WS: Validating primary data. [2025-10-20T21:57:59.293970] [DEBUG] [VALIDATOR ] WS: All expected trading days are present. [2025-10-20T21:57:59.294068] [DEBUG] [VALIDATOR ] WS: Primary data validation passed. [2025-10-20T21:57:59.295084] [INFO ] [MAIN ] WS: Fetched primary data is valid. [2025-10-20T21:57:59.295148] [DEBUG] [LOCAL ] WS: Saving primary data. [2025-10-20T21:57:59.296082] [DEBUG] [LOCAL ] WS: Trimmed 35719 records to 23540 (keeping 200 days of data) [2025-10-20T21:57:59.443520] [DEBUG] [LOCAL ] WS: Primary data saved (23540 records). [2025-10-20T21:57:59.443698] [INFO ] [MAIN ] WS: Full history saved. [2025-10-20T21:57:59.443796] [INFO ] [MAIN ] Update Progress: 2315/2367 symbols completed. WS took 20.3s. Estimated remaining: 6m 42s. Rate: 7.8 symbols/min [2025-10-20T21:57:59.456574] [INFO ] [REMOTE ] WRB: Fetching primary data for 2025-10. [2025-10-20T21:57:59.623677] [INFO ] [MAIN ] WSO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:57:59.650110] [DEBUG] [LOCAL ] WSO: Loaded 26167 existing records from primary storage. [2025-10-20T21:57:59.650370] [INFO ] [MAIN ] WSO: Loaded 26167 existing records. [2025-10-20T21:57:59.823616] [WARN ] [MAIN ] WSO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:57:59.823858] [INFO ] [MAIN ] WSO: Wiping existing primary data before full refresh. [2025-10-20T21:57:59.823902] [INFO ] [LOCAL ] WSO: Starting deletion of primary data. [2025-10-20T21:57:59.824094] [DEBUG] [LOCAL ] WSO: Deleted primary data file. [2025-10-20T21:57:59.824149] [INFO ] [MAIN ] WSO: Starting full primary refresh. [2025-10-20T21:58:00.210626] [INFO ] [REMOTE ] WRB: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:58:00.208004] [INFO ] [REMOTE ] WSO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:00.219442] [INFO ] [REMOTE ] WRB: Full fetch complete (73801 records). [2025-10-20T21:58:00.219613] [DEBUG] [VALIDATOR ] WRB: Validating primary data. [2025-10-20T21:58:00.256507] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-09. [2025-10-20T21:58:00.617307] [DEBUG] [VALIDATOR ] WRB: All expected trading days are present. [2025-10-20T21:58:00.617472] [DEBUG] [VALIDATOR ] WRB: Primary data validation passed. [2025-10-20T21:58:00.618615] [INFO ] [MAIN ] WRB: Fetched primary data is valid. [2025-10-20T21:58:00.618840] [DEBUG] [LOCAL ] WRB: Saving primary data. [2025-10-20T21:58:00.620513] [DEBUG] [LOCAL ] WRB: Trimmed 73801 records to 50781 (keeping 200 days of data) [2025-10-20T21:58:00.776237] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T21:58:00.970452] [DEBUG] [LOCAL ] WRB: Primary data saved (50781 records). [2025-10-20T21:58:00.970598] [INFO ] [MAIN ] WRB: Full history saved. [2025-10-20T21:58:00.970701] [INFO ] [MAIN ] Update Progress: 2316/2367 symbols completed. WRB took 38.9s. Estimated remaining: 6m 34s. Rate: 7.8 symbols/min [2025-10-20T21:58:01.059270] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-05. [2025-10-20T21:58:01.141180] [INFO ] [MAIN ] WSR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:01.164982] [DEBUG] [LOCAL ] WSR: Loaded 23238 existing records from primary storage. [2025-10-20T21:58:01.165323] [INFO ] [MAIN ] WSR: Loaded 23238 existing records. [2025-10-20T21:58:01.339708] [WARN ] [MAIN ] WSR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:01.339980] [INFO ] [MAIN ] WSR: Wiping existing primary data before full refresh. [2025-10-20T21:58:01.340032] [INFO ] [LOCAL ] WSR: Starting deletion of primary data. [2025-10-20T21:58:01.340382] [DEBUG] [LOCAL ] WSR: Deleted primary data file. [2025-10-20T21:58:01.340443] [INFO ] [MAIN ] WSR: Starting full primary refresh. [2025-10-20T21:58:01.719461] [INFO ] [REMOTE ] WSR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:01.851352] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-01. [2025-10-20T21:58:02.348146] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-09 (2.09 seconds). [2025-10-20T21:58:02.651423] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-01. [2025-10-20T21:58:02.928309] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T21:58:02.962778] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-01 (1.11 seconds). [2025-10-20T21:58:03.381889] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-01 (0.73 seconds). [2025-10-20T21:58:03.451377] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-06. [2025-10-20T21:58:04.251395] [INFO ] [REMOTE ] WRBY: Fetching primary data for 2025-10. [2025-10-20T21:58:05.051373] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-02. [2025-10-20T21:58:05.188012] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T21:58:05.439605] [INFO ] [REMOTE ] WRBY: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T21:58:05.447183] [INFO ] [REMOTE ] WRBY: Full fetch complete (73722 records). [2025-10-20T21:58:05.447338] [DEBUG] [VALIDATOR ] WRBY: Validating primary data. [2025-10-20T21:58:05.845791] [DEBUG] [VALIDATOR ] WRBY: All expected trading days are present. [2025-10-20T21:58:05.845869] [DEBUG] [VALIDATOR ] WRBY: Primary data validation passed. [2025-10-20T21:58:05.846971] [INFO ] [MAIN ] WRBY: Fetched primary data is valid. [2025-10-20T21:58:05.847034] [DEBUG] [LOCAL ] WRBY: Saving primary data. [2025-10-20T21:58:05.848421] [DEBUG] [LOCAL ] WRBY: Trimmed 73722 records to 50728 (keeping 200 days of data) [2025-10-20T21:58:05.895378] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-02. [2025-10-20T21:58:06.139431] [DEBUG] [LOCAL ] WRBY: Primary data saved (50728 records). [2025-10-20T21:58:06.139733] [INFO ] [MAIN ] WRBY: Full history saved. [2025-10-20T21:58:06.139902] [INFO ] [MAIN ] Update Progress: 2317/2367 symbols completed. WRBY took 30.7s. Estimated remaining: 6m 26s. Rate: 7.8 symbols/min [2025-10-20T21:58:06.206534] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-02 (1.16 seconds). [2025-10-20T21:58:06.651406] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-07. [2025-10-20T21:58:06.852415] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-02 (0.96 seconds). [2025-10-20T21:58:07.451368] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-03. [2025-10-20T21:58:08.251323] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-03. [2025-10-20T21:58:08.536792] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T21:58:08.649738] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-03 (1.20 seconds). [2025-10-20T21:58:09.051391] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-04. [2025-10-20T21:58:09.177107] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-03 (0.93 seconds). [2025-10-20T21:58:09.851343] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-04. [2025-10-20T21:58:09.931444] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-04 (0.88 seconds). [2025-10-20T21:58:10.651377] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-08. [2025-10-20T21:58:11.276218] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T21:58:11.451370] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-05. [2025-10-20T21:58:12.251374] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-05. [2025-10-20T21:58:12.491562] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T21:58:12.674142] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-05 (1.22 seconds). [2025-10-20T21:58:13.051387] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-09. [2025-10-20T21:58:13.170085] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-05 (0.92 seconds). [2025-10-20T21:58:13.851375] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-06. [2025-10-20T21:58:14.447822] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-09 (1.40 seconds). [2025-10-20T21:58:14.651336] [INFO ] [REMOTE ] WSM: Fetching primary data for 2025-10. [2025-10-20T21:58:14.995696] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T21:58:15.367619] [INFO ] [REMOTE ] WSM: Incremental fetch complete for 2025-10 (0.72 seconds). [2025-10-20T21:58:15.374335] [INFO ] [REMOTE ] WSM: Full fetch complete (71475 records). [2025-10-20T21:58:15.374455] [DEBUG] [VALIDATOR ] WSM: Validating primary data. [2025-10-20T21:58:15.456550] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-06. [2025-10-20T21:58:15.775224] [DEBUG] [VALIDATOR ] WSM: All expected trading days are present. [2025-10-20T21:58:15.775300] [DEBUG] [VALIDATOR ] WSM: Primary data validation passed. [2025-10-20T21:58:15.776372] [INFO ] [MAIN ] WSM: Fetched primary data is valid. [2025-10-20T21:58:15.776433] [DEBUG] [LOCAL ] WSM: Saving primary data. [2025-10-20T21:58:15.777812] [DEBUG] [LOCAL ] WSM: Trimmed 71475 records to 48892 (keeping 200 days of data) [2025-10-20T21:58:16.076550] [DEBUG] [LOCAL ] WSM: Primary data saved (48892 records). [2025-10-20T21:58:16.076704] [INFO ] [MAIN ] WSM: Full history saved. [2025-10-20T21:58:16.076792] [INFO ] [MAIN ] Update Progress: 2318/2367 symbols completed. WSM took 34.9s. Estimated remaining: 6m 18s. Rate: 7.8 symbols/min [2025-10-20T21:58:16.251397] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-07. [2025-10-20T21:58:16.622506] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-06 (1.17 seconds). [2025-10-20T21:58:17.032887] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-07 (0.78 seconds). [2025-10-20T21:58:17.051331] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-08. [2025-10-20T21:58:17.851348] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-07. [2025-10-20T21:58:18.086046] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-08 (1.03 seconds). [2025-10-20T21:58:18.651370] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-09. [2025-10-20T21:58:19.013739] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-07 (1.16 seconds). [2025-10-20T21:58:19.451383] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-08. [2025-10-20T21:58:19.511016] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-09 (0.86 seconds). [2025-10-20T21:58:20.251415] [INFO ] [REMOTE ] WSR: Fetching primary data for 2025-10. [2025-10-20T21:58:20.373075] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-08 (0.92 seconds). [2025-10-20T21:58:21.051349] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-09. [2025-10-20T21:58:21.239972] [INFO ] [REMOTE ] WSR: Incremental fetch complete for 2025-10 (0.99 seconds). [2025-10-20T21:58:21.243277] [INFO ] [REMOTE ] WSR: Full fetch complete (32734 records). [2025-10-20T21:58:21.243350] [DEBUG] [VALIDATOR ] WSR: Validating primary data. [2025-10-20T21:58:21.624305] [DEBUG] [VALIDATOR ] WSR: All expected trading days are present. [2025-10-20T21:58:21.624381] [DEBUG] [VALIDATOR ] WSR: Primary data validation passed. [2025-10-20T21:58:21.625069] [INFO ] [MAIN ] WSR: Fetched primary data is valid. [2025-10-20T21:58:21.625129] [DEBUG] [LOCAL ] WSR: Saving primary data. [2025-10-20T21:58:21.625858] [DEBUG] [LOCAL ] WSR: Trimmed 32734 records to 22806 (keeping 200 days of data) [2025-10-20T21:58:21.759813] [DEBUG] [LOCAL ] WSR: Primary data saved (22806 records). [2025-10-20T21:58:21.759935] [INFO ] [MAIN ] WSR: Full history saved. [2025-10-20T21:58:21.760007] [INFO ] [MAIN ] Update Progress: 2319/2367 symbols completed. WSR took 20.8s. Estimated remaining: 6m 10s. Rate: 7.8 symbols/min [2025-10-20T21:58:22.443172] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-09 (1.39 seconds). [2025-10-20T21:58:22.449922] [INFO ] [REMOTE ] WSO: Fetching primary data for 2025-10. [2025-10-20T21:58:23.789045] [INFO ] [REMOTE ] WSO: Incremental fetch complete for 2025-10 (1.34 seconds). [2025-10-20T21:58:23.793720] [INFO ] [REMOTE ] WSO: Full fetch complete (35767 records). [2025-10-20T21:58:23.793814] [DEBUG] [VALIDATOR ] WSO: Validating primary data. [2025-10-20T21:58:24.159758] [DEBUG] [VALIDATOR ] WSO: All expected trading days are present. [2025-10-20T21:58:24.159830] [DEBUG] [VALIDATOR ] WSO: Primary data validation passed. [2025-10-20T21:58:24.160602] [INFO ] [MAIN ] WSO: Fetched primary data is valid. [2025-10-20T21:58:24.160663] [DEBUG] [LOCAL ] WSO: Saving primary data. [2025-10-20T21:58:24.161406] [DEBUG] [LOCAL ] WSO: Trimmed 35767 records to 25899 (keeping 200 days of data) [2025-10-20T21:58:24.317144] [DEBUG] [LOCAL ] WSO: Primary data saved (25899 records). [2025-10-20T21:58:24.317269] [INFO ] [MAIN ] WSO: Full history saved. [2025-10-20T21:58:24.317342] [INFO ] [MAIN ] Update Progress: 2320/2367 symbols completed. WSO took 24.9s. Estimated remaining: 6m 3s. Rate: 7.8 symbols/min [2025-10-20T21:58:24.972989] [INFO ] [MAIN ] WTI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:24.976397] [INFO ] [MAIN ] WT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:25.018764] [INFO ] [MAIN ] WTM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:25.021341] [INFO ] [MAIN ] WST: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:25.021853] [DEBUG] [LOCAL ] WTM: No existing data file found in primary storage. [2025-10-20T21:58:25.022021] [WARN ] [MAIN ] WTM: No existing data found, performing full refresh instead. [2025-10-20T21:58:25.022275] [INFO ] [MAIN ] WTM: Wiping existing primary data before full refresh. [2025-10-20T21:58:25.022329] [INFO ] [LOCAL ] WTM: Starting deletion of primary data. [2025-10-20T21:58:25.022375] [DEBUG] [LOCAL ] WTM: No primary data file found to delete. [2025-10-20T21:58:25.022409] [INFO ] [MAIN ] WTM: Starting full primary refresh. [2025-10-20T21:58:25.338450] [DEBUG] [LOCAL ] WTI: Loaded 45285 existing records from primary storage. [2025-10-20T21:58:25.366919] [INFO ] [MAIN ] WTI: Loaded 45285 existing records. [2025-10-20T21:58:25.427610] [DEBUG] [LOCAL ] WST: Loaded 37318 existing records from primary storage. [2025-10-20T21:58:25.473808] [INFO ] [MAIN ] WST: Loaded 37318 existing records. [2025-10-20T21:58:25.433685] [DEBUG] [LOCAL ] WT: Loaded 48844 existing records from primary storage. [2025-10-20T21:58:25.529479] [INFO ] [MAIN ] WT: Loaded 48844 existing records. [2025-10-20T21:58:25.622850] [INFO ] [REMOTE ] WTM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:25.683802] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-01. [2025-10-20T21:58:25.945858] [WARN ] [MAIN ] WTI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:25.978946] [WARN ] [MAIN ] WST: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:25.979168] [INFO ] [MAIN ] WTI: Wiping existing primary data before full refresh. [2025-10-20T21:58:25.992895] [INFO ] [MAIN ] WST: Wiping existing primary data before full refresh. [2025-10-20T21:58:26.003291] [INFO ] [LOCAL ] WTI: Starting deletion of primary data. [2025-10-20T21:58:26.033281] [WARN ] [MAIN ] WT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:26.033537] [DEBUG] [LOCAL ] WTI: Deleted primary data file. [2025-10-20T21:58:26.033796] [INFO ] [MAIN ] WT: Wiping existing primary data before full refresh. [2025-10-20T21:58:26.033742] [INFO ] [MAIN ] WTI: Starting full primary refresh. [2025-10-20T21:58:26.033442] [INFO ] [LOCAL ] WST: Starting deletion of primary data. [2025-10-20T21:58:26.045254] [INFO ] [LOCAL ] WT: Starting deletion of primary data. [2025-10-20T21:58:26.054668] [DEBUG] [LOCAL ] WST: Deleted primary data file. [2025-10-20T21:58:26.089415] [DEBUG] [LOCAL ] WT: Deleted primary data file. [2025-10-20T21:58:26.089353] [INFO ] [MAIN ] WST: Starting full primary refresh. [2025-10-20T21:58:26.109674] [INFO ] [MAIN ] WT: Starting full primary refresh. [2025-10-20T21:58:26.644583] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-01 (0.96 seconds). [2025-10-20T21:58:26.702203] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-02. [2025-10-20T21:58:27.049383] [INFO ] [REMOTE ] WST: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:27.108541] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-01. [2025-10-20T21:58:27.198452] [INFO ] [REMOTE ] WT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:27.193772] [INFO ] [REMOTE ] WTI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:27.199082] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-01. [2025-10-20T21:58:27.199453] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-01. [2025-10-20T21:58:27.231493] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-02 (0.53 seconds). [2025-10-20T21:58:27.232388] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-03. [2025-10-20T21:58:27.625013] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-03 (0.39 seconds). [2025-10-20T21:58:27.626058] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-04. [2025-10-20T21:58:28.141328] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-04 (0.52 seconds). [2025-10-20T21:58:28.251348] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-05. [2025-10-20T21:58:28.696618] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T21:58:28.748621] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-05 (0.50 seconds). [2025-10-20T21:58:28.868913] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-01 (1.67 seconds). [2025-10-20T21:58:28.903399] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-01 (1.70 seconds). [2025-10-20T21:58:29.051348] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-06. [2025-10-20T21:58:29.600941] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-06 (0.55 seconds). [2025-10-20T21:58:29.851400] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-02. [2025-10-20T21:58:30.651460] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-07. [2025-10-20T21:58:31.121990] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-07 (0.47 seconds). [2025-10-20T21:58:31.451351] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-02. [2025-10-20T21:58:31.477090] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:58:32.251366] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-02. [2025-10-20T21:58:32.865839] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-02 (1.41 seconds). [2025-10-20T21:58:33.051384] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-08. [2025-10-20T21:58:33.614258] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-08 (0.56 seconds). [2025-10-20T21:58:33.851364] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-09. [2025-10-20T21:58:33.877135] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-02 (1.63 seconds). [2025-10-20T21:58:34.494253] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-09 (0.64 seconds). [2025-10-20T21:58:34.651358] [INFO ] [REMOTE ] WTM: Fetching primary data for 2025-10. [2025-10-20T21:58:35.336714] [INFO ] [REMOTE ] WTM: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T21:58:35.338033] [INFO ] [REMOTE ] WTM: Full fetch complete (2757 records). [2025-10-20T21:58:35.338112] [DEBUG] [VALIDATOR ] WTM: Validating primary data. [2025-10-20T21:58:35.456497] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-03. [2025-10-20T21:58:35.524148] [WARN ] [VALIDATOR ] WTM: Validation failed - insufficient historical data. Have 199 unique days, need 200 days [2025-10-20T21:58:35.524379] [FATAL] [MAIN ] WTM: Fetched primary data is invalid. Refusing to save it. [2025-10-20T21:58:35.524450] [WARN ] [MAIN ] WTM: Deleting existing primary and derivative data due to validation failure. [2025-10-20T21:58:35.524523] [INFO ] [LOCAL ] WTM: Starting deletion of primary data. [2025-10-20T21:58:35.524584] [DEBUG] [LOCAL ] WTM: No primary data file found to delete. [2025-10-20T21:58:35.524620] [INFO ] [LOCAL ] WTM: Starting deletion of derivative data. [2025-10-20T21:58:35.524698] [DEBUG] [LOCAL ] WTM: No derivative data file found to delete. [2025-10-20T21:58:35.524770] [INFO ] [MAIN ] Update Progress: 2321/2367 symbols completed. WTM took 11.2s. Estimated remaining: 5m 55s. Rate: 7.8 symbols/min [2025-10-20T21:58:35.698521] [INFO ] [MAIN ] WTRG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:35.747009] [DEBUG] [LOCAL ] WTRG: Loaded 51704 existing records from primary storage. [2025-10-20T21:58:35.747308] [INFO ] [MAIN ] WTRG: Loaded 51704 existing records. [2025-10-20T21:58:35.921895] [WARN ] [MAIN ] WTRG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:35.922078] [INFO ] [MAIN ] WTRG: Wiping existing primary data before full refresh. [2025-10-20T21:58:35.922124] [INFO ] [LOCAL ] WTRG: Starting deletion of primary data. [2025-10-20T21:58:35.922297] [DEBUG] [LOCAL ] WTRG: Deleted primary data file. [2025-10-20T21:58:35.922334] [INFO ] [MAIN ] WTRG: Starting full primary refresh. [2025-10-20T21:58:36.251929] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-03. [2025-10-20T21:58:36.277528] [INFO ] [REMOTE ] WTRG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:37.051418] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-03. [2025-10-20T21:58:37.133211] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T21:58:37.851367] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-01. [2025-10-20T21:58:37.915211] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T21:58:38.651403] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-04. [2025-10-20T21:58:38.681444] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-03 (1.63 seconds). [2025-10-20T21:58:39.395601] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T21:58:39.451378] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-02. [2025-10-20T21:58:40.251385] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-04. [2025-10-20T21:58:40.364858] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-04 (1.71 seconds). [2025-10-20T21:58:41.051412] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-04. [2025-10-20T21:58:41.147296] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-02 (1.70 seconds). [2025-10-20T21:58:41.851377] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-05. [2025-10-20T21:58:41.884069] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:58:42.651349] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-03. [2025-10-20T21:58:42.781540] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-04 (1.73 seconds). [2025-10-20T21:58:43.066215] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-05 (1.21 seconds). [2025-10-20T21:58:43.451358] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-06. [2025-10-20T21:58:44.251387] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-05. [2025-10-20T21:58:44.422766] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-03 (1.77 seconds). [2025-10-20T21:58:45.051437] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-04. [2025-10-20T21:58:45.255383] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T21:58:45.726543] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-05 (1.48 seconds). [2025-10-20T21:58:45.851345] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-06. [2025-10-20T21:58:46.520998] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T21:58:46.651428] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-07. [2025-10-20T21:58:47.451405] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-05. [2025-10-20T21:58:47.503575] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T21:58:48.251447] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-05. [2025-10-20T21:58:48.279862] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-07 (1.63 seconds). [2025-10-20T21:58:48.884965] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-05 (1.43 seconds). [2025-10-20T21:58:49.051397] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-06. [2025-10-20T21:58:49.851418] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-08. [2025-10-20T21:58:50.087258] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T21:58:50.481703] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-06 (1.43 seconds). [2025-10-20T21:58:50.651369] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-06. [2025-10-20T21:58:51.443928] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-08 (1.59 seconds). [2025-10-20T21:58:51.451860] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-07. [2025-10-20T21:58:52.251349] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-09. [2025-10-20T21:58:52.327782] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-06 (1.68 seconds). [2025-10-20T21:58:52.972039] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-07 (1.52 seconds). [2025-10-20T21:58:53.051356] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-08. [2025-10-20T21:58:53.827185] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T21:58:53.851339] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-07. [2025-10-20T21:58:54.440724] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-08 (1.39 seconds). [2025-10-20T21:58:54.651403] [INFO ] [REMOTE ] WTI: Fetching primary data for 2025-10. [2025-10-20T21:58:55.451348] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-09. [2025-10-20T21:58:55.595130] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-07 (1.74 seconds). [2025-10-20T21:58:55.700003] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-09 (0.25 seconds). [2025-10-20T21:58:56.212551] [INFO ] [REMOTE ] WTI: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T21:58:56.219727] [INFO ] [REMOTE ] WTI: Full fetch complete (63982 records). [2025-10-20T21:58:56.219873] [DEBUG] [VALIDATOR ] WTI: Validating primary data. [2025-10-20T21:58:56.256574] [INFO ] [REMOTE ] WST: Fetching primary data for 2025-10. [2025-10-20T21:58:56.601227] [DEBUG] [VALIDATOR ] WTI: All expected trading days are present. [2025-10-20T21:58:56.601306] [DEBUG] [VALIDATOR ] WTI: Primary data validation passed. [2025-10-20T21:58:56.602347] [INFO ] [MAIN ] WTI: Fetched primary data is valid. [2025-10-20T21:58:56.602410] [DEBUG] [LOCAL ] WTI: Saving primary data. [2025-10-20T21:58:56.603622] [DEBUG] [LOCAL ] WTI: Trimmed 63982 records to 44707 (keeping 200 days of data) [2025-10-20T21:58:56.855612] [DEBUG] [LOCAL ] WTI: Primary data saved (44707 records). [2025-10-20T21:58:56.855762] [INFO ] [MAIN ] WTI: Full history saved. [2025-10-20T21:58:56.855845] [INFO ] [MAIN ] Update Progress: 2322/2367 symbols completed. WTI took 32.5s. Estimated remaining: 5m 48s. Rate: 7.8 symbols/min [2025-10-20T21:58:57.046355] [INFO ] [MAIN ] WTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:57.048755] [INFO ] [REMOTE ] WST: Incremental fetch complete for 2025-10 (0.79 seconds). [2025-10-20T21:58:57.054515] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-08. [2025-10-20T21:58:57.057220] [INFO ] [REMOTE ] WST: Full fetch complete (54322 records). [2025-10-20T21:58:57.057443] [DEBUG] [VALIDATOR ] WST: Validating primary data. [2025-10-20T21:58:57.181619] [DEBUG] [LOCAL ] WTS: Loaded 17595 existing records from primary storage. [2025-10-20T21:58:57.197343] [INFO ] [MAIN ] WTS: Loaded 17595 existing records. [2025-10-20T21:58:57.597592] [WARN ] [MAIN ] WTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:57.633250] [INFO ] [MAIN ] WTS: Wiping existing primary data before full refresh. [2025-10-20T21:58:57.649907] [DEBUG] [VALIDATOR ] WST: All expected trading days are present. [2025-10-20T21:58:57.649969] [DEBUG] [VALIDATOR ] WST: Primary data validation passed. [2025-10-20T21:58:57.650908] [INFO ] [MAIN ] WST: Fetched primary data is valid. [2025-10-20T21:58:57.650985] [DEBUG] [LOCAL ] WST: Saving primary data. [2025-10-20T21:58:57.647142] [INFO ] [LOCAL ] WTS: Starting deletion of primary data. [2025-10-20T21:58:57.651375] [DEBUG] [LOCAL ] WTS: Deleted primary data file. [2025-10-20T21:58:57.651438] [INFO ] [MAIN ] WTS: Starting full primary refresh. [2025-10-20T21:58:57.740999] [DEBUG] [LOCAL ] WST: Trimmed 54322 records to 36673 (keeping 200 days of data) [2025-10-20T21:58:57.862112] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-07. [2025-10-20T21:58:58.145369] [INFO ] [REMOTE ] WTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:58.246851] [DEBUG] [LOCAL ] WST: Primary data saved (36673 records). [2025-10-20T21:58:58.247000] [INFO ] [MAIN ] WST: Full history saved. [2025-10-20T21:58:58.247115] [INFO ] [MAIN ] Update Progress: 2323/2367 symbols completed. WST took 33.9s. Estimated remaining: 5m 40s. Rate: 7.8 symbols/min [2025-10-20T21:58:58.412429] [INFO ] [MAIN ] WTTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:58:58.452721] [DEBUG] [LOCAL ] WTTR: Loaded 45312 existing records from primary storage. [2025-10-20T21:58:58.453002] [INFO ] [MAIN ] WTTR: Loaded 45312 existing records. [2025-10-20T21:58:58.634394] [WARN ] [MAIN ] WTTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:58:58.634724] [INFO ] [MAIN ] WTTR: Wiping existing primary data before full refresh. [2025-10-20T21:58:58.634822] [INFO ] [LOCAL ] WTTR: Starting deletion of primary data. [2025-10-20T21:58:58.635137] [DEBUG] [LOCAL ] WTTR: Deleted primary data file. [2025-10-20T21:58:58.635206] [INFO ] [MAIN ] WTTR: Starting full primary refresh. [2025-10-20T21:58:58.665360] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-01. [2025-10-20T21:58:59.011316] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-08 (1.96 seconds). [2025-10-20T21:58:59.053212] [INFO ] [REMOTE ] WTTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:58:59.451382] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-09. [2025-10-20T21:58:59.815475] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-07 (1.95 seconds). [2025-10-20T21:58:59.836091] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-01 (1.17 seconds). [2025-10-20T21:59:00.251374] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-08. [2025-10-20T21:59:00.934032] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-09 (1.48 seconds). [2025-10-20T21:59:01.051373] [INFO ] [REMOTE ] WT: Fetching primary data for 2025-10. [2025-10-20T21:59:01.851388] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-02. [2025-10-20T21:59:02.107860] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-08 (1.86 seconds). [2025-10-20T21:59:02.651407] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-09. [2025-10-20T21:59:02.727006] [INFO ] [REMOTE ] WT: Incremental fetch complete for 2025-10 (1.68 seconds). [2025-10-20T21:59:02.734958] [INFO ] [REMOTE ] WT: Full fetch complete (69811 records). [2025-10-20T21:59:02.735085] [DEBUG] [VALIDATOR ] WT: Validating primary data. [2025-10-20T21:59:03.050201] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-02 (1.20 seconds). [2025-10-20T21:59:03.169264] [DEBUG] [VALIDATOR ] WT: All expected trading days are present. [2025-10-20T21:59:03.169379] [DEBUG] [VALIDATOR ] WT: Primary data validation passed. [2025-10-20T21:59:03.171749] [INFO ] [MAIN ] WT: Fetched primary data is valid. [2025-10-20T21:59:03.171826] [DEBUG] [LOCAL ] WT: Saving primary data. [2025-10-20T21:59:03.173872] [DEBUG] [LOCAL ] WT: Trimmed 69811 records to 48165 (keeping 200 days of data) [2025-10-20T21:59:03.453035] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-01. [2025-10-20T21:59:03.492118] [DEBUG] [LOCAL ] WT: Primary data saved (48165 records). [2025-10-20T21:59:03.492456] [INFO ] [MAIN ] WT: Full history saved. [2025-10-20T21:59:03.492575] [INFO ] [MAIN ] Update Progress: 2324/2367 symbols completed. WT took 39.2s. Estimated remaining: 5m 32s. Rate: 7.8 symbols/min [2025-10-20T21:59:03.670596] [INFO ] [MAIN ] WU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:03.726485] [DEBUG] [LOCAL ] WU: Loaded 54258 existing records from primary storage. [2025-10-20T21:59:03.726987] [INFO ] [MAIN ] WU: Loaded 54258 existing records. [2025-10-20T21:59:03.917571] [WARN ] [MAIN ] WU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:03.917876] [INFO ] [MAIN ] WU: Wiping existing primary data before full refresh. [2025-10-20T21:59:03.917954] [INFO ] [LOCAL ] WU: Starting deletion of primary data. [2025-10-20T21:59:03.918329] [DEBUG] [LOCAL ] WU: Deleted primary data file. [2025-10-20T21:59:03.918390] [INFO ] [MAIN ] WU: Starting full primary refresh. [2025-10-20T21:59:04.254230] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-03. [2025-10-20T21:59:04.300529] [INFO ] [REMOTE ] WU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:04.373122] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-09 (1.72 seconds). [2025-10-20T21:59:05.040351] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-01 (1.59 seconds). [2025-10-20T21:59:05.051345] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-01. [2025-10-20T21:59:05.420808] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T21:59:05.851416] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-02. [2025-10-20T21:59:06.651424] [INFO ] [REMOTE ] WTRG: Fetching primary data for 2025-10. [2025-10-20T21:59:06.834680] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T21:59:07.373999] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T21:59:07.451373] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-03. [2025-10-20T21:59:08.251349] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-04. [2025-10-20T21:59:08.361824] [INFO ] [REMOTE ] WTRG: Incremental fetch complete for 2025-10 (1.71 seconds). [2025-10-20T21:59:08.369755] [INFO ] [REMOTE ] WTRG: Full fetch complete (73856 records). [2025-10-20T21:59:08.369889] [DEBUG] [VALIDATOR ] WTRG: Validating primary data. [2025-10-20T21:59:08.793557] [DEBUG] [VALIDATOR ] WTRG: All expected trading days are present. [2025-10-20T21:59:08.793758] [DEBUG] [VALIDATOR ] WTRG: Primary data validation passed. [2025-10-20T21:59:08.794904] [INFO ] [MAIN ] WTRG: Fetched primary data is valid. [2025-10-20T21:59:08.794980] [DEBUG] [LOCAL ] WTRG: Saving primary data. [2025-10-20T21:59:08.796458] [DEBUG] [LOCAL ] WTRG: Trimmed 73856 records to 50966 (keeping 200 days of data) [2025-10-20T21:59:09.064609] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-02. [2025-10-20T21:59:09.116081] [DEBUG] [LOCAL ] WTRG: Primary data saved (50966 records). [2025-10-20T21:59:09.116307] [INFO ] [MAIN ] WTRG: Full history saved. [2025-10-20T21:59:09.116481] [INFO ] [MAIN ] Update Progress: 2325/2367 symbols completed. WTRG took 33.6s. Estimated remaining: 5m 24s. Rate: 7.8 symbols/min [2025-10-20T21:59:09.339957] [INFO ] [MAIN ] WWW: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:09.342064] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-03 (1.89 seconds). [2025-10-20T21:59:09.412525] [DEBUG] [LOCAL ] WWW: Loaded 49933 existing records from primary storage. [2025-10-20T21:59:09.412906] [INFO ] [MAIN ] WWW: Loaded 49933 existing records. [2025-10-20T21:59:09.612415] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-04 (1.36 seconds). [2025-10-20T21:59:09.628344] [WARN ] [MAIN ] WWW: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:09.629311] [INFO ] [MAIN ] WWW: Wiping existing primary data before full refresh. [2025-10-20T21:59:09.629383] [INFO ] [LOCAL ] WWW: Starting deletion of primary data. [2025-10-20T21:59:09.629584] [DEBUG] [LOCAL ] WWW: Deleted primary data file. [2025-10-20T21:59:09.629623] [INFO ] [MAIN ] WWW: Starting full primary refresh. [2025-10-20T21:59:09.860086] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-05. [2025-10-20T21:59:10.001784] [INFO ] [REMOTE ] WWW: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:10.651350] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-01. [2025-10-20T21:59:10.772391] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-02 (1.71 seconds). [2025-10-20T21:59:11.061975] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-05 (1.20 seconds). [2025-10-20T21:59:11.451378] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-03. [2025-10-20T21:59:12.256499] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-06. [2025-10-20T21:59:12.270568] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T21:59:13.051393] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-02. [2025-10-20T21:59:13.415268] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-06 (1.16 seconds). [2025-10-20T21:59:13.847905] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-03 (2.40 seconds). [2025-10-20T21:59:13.858030] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-07. [2025-10-20T21:59:14.651347] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-04. [2025-10-20T21:59:14.740806] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-02 (1.69 seconds). [2025-10-20T21:59:14.905751] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T21:59:15.451430] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-08. [2025-10-20T21:59:16.251374] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-03. [2025-10-20T21:59:16.521203] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-04 (1.87 seconds). [2025-10-20T21:59:16.548012] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-08 (1.10 seconds). [2025-10-20T21:59:17.051392] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-04. [2025-10-20T21:59:17.823586] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T21:59:17.851328] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-04. [2025-10-20T21:59:18.651428] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-05. [2025-10-20T21:59:18.682765] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T21:59:19.451395] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-05. [2025-10-20T21:59:19.571994] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T21:59:20.251413] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-09. [2025-10-20T21:59:20.507753] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-05 (1.86 seconds). [2025-10-20T21:59:21.051371] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-06. [2025-10-20T21:59:21.113700] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-05 (1.66 seconds). [2025-10-20T21:59:21.191034] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-09 (0.94 seconds). [2025-10-20T21:59:21.851398] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-06. [2025-10-20T21:59:22.651422] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-05. [2025-10-20T21:59:22.800527] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T21:59:23.390277] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T21:59:23.451352] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-07. [2025-10-20T21:59:24.251345] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-07. [2025-10-20T21:59:24.364972] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-05 (1.71 seconds). [2025-10-20T21:59:25.010177] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-07 (1.56 seconds). [2025-10-20T21:59:25.051362] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-08. [2025-10-20T21:59:25.851370] [INFO ] [REMOTE ] WTS: Fetching primary data for 2025-10. [2025-10-20T21:59:26.196062] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-07 (1.94 seconds). [2025-10-20T21:59:26.601462] [INFO ] [REMOTE ] WTS: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T21:59:26.607507] [INFO ] [REMOTE ] WTS: Full fetch complete (25677 records). [2025-10-20T21:59:26.607391] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T21:59:26.616676] [DEBUG] [VALIDATOR ] WTS: Validating primary data. [2025-10-20T21:59:26.651353] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-06. [2025-10-20T21:59:26.991559] [DEBUG] [VALIDATOR ] WTS: All expected trading days are present. [2025-10-20T21:59:26.991686] [DEBUG] [VALIDATOR ] WTS: Primary data validation passed. [2025-10-20T21:59:26.992566] [INFO ] [MAIN ] WTS: Fetched primary data is valid. [2025-10-20T21:59:26.992627] [DEBUG] [LOCAL ] WTS: Saving primary data. [2025-10-20T21:59:26.993488] [DEBUG] [LOCAL ] WTS: Trimmed 25677 records to 17355 (keeping 200 days of data) [2025-10-20T21:59:27.106763] [DEBUG] [LOCAL ] WTS: Primary data saved (17355 records). [2025-10-20T21:59:27.106973] [INFO ] [MAIN ] WTS: Full history saved. [2025-10-20T21:59:27.107069] [INFO ] [MAIN ] Update Progress: 2326/2367 symbols completed. WTS took 30.3s. Estimated remaining: 5m 17s. Rate: 7.8 symbols/min [2025-10-20T21:59:27.276796] [INFO ] [MAIN ] WY: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:27.329842] [DEBUG] [LOCAL ] WY: Loaded 53997 existing records from primary storage. [2025-10-20T21:59:27.330253] [INFO ] [MAIN ] WY: Loaded 53997 existing records. [2025-10-20T21:59:27.456506] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-09. [2025-10-20T21:59:27.514660] [WARN ] [MAIN ] WY: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:27.515060] [INFO ] [MAIN ] WY: Wiping existing primary data before full refresh. [2025-10-20T21:59:27.515149] [INFO ] [LOCAL ] WY: Starting deletion of primary data. [2025-10-20T21:59:27.515416] [DEBUG] [LOCAL ] WY: Deleted primary data file. [2025-10-20T21:59:27.515473] [INFO ] [MAIN ] WY: Starting full primary refresh. [2025-10-20T21:59:27.881906] [INFO ] [REMOTE ] WY: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:28.251357] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-01. [2025-10-20T21:59:28.279815] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-06 (1.63 seconds). [2025-10-20T21:59:29.051388] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-07. [2025-10-20T21:59:29.111000] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-09 (1.65 seconds). [2025-10-20T21:59:29.851424] [INFO ] [REMOTE ] WTTR: Fetching primary data for 2025-10. [2025-10-20T21:59:30.166018] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-01 (1.91 seconds). [2025-10-20T21:59:30.651374] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-08. [2025-10-20T21:59:30.711100] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-07 (1.66 seconds). [2025-10-20T21:59:31.395442] [INFO ] [REMOTE ] WTTR: Incremental fetch complete for 2025-10 (1.54 seconds). [2025-10-20T21:59:31.402143] [INFO ] [REMOTE ] WTTR: Full fetch complete (65583 records). [2025-10-20T21:59:31.402253] [DEBUG] [VALIDATOR ] WTTR: Validating primary data. [2025-10-20T21:59:31.456730] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-08. [2025-10-20T21:59:31.808789] [DEBUG] [VALIDATOR ] WTTR: All expected trading days are present. [2025-10-20T21:59:31.808881] [DEBUG] [VALIDATOR ] WTTR: Primary data validation passed. [2025-10-20T21:59:31.810085] [INFO ] [MAIN ] WTTR: Fetched primary data is valid. [2025-10-20T21:59:31.810151] [DEBUG] [LOCAL ] WTTR: Saving primary data. [2025-10-20T21:59:31.811516] [DEBUG] [LOCAL ] WTTR: Trimmed 65583 records to 44633 (keeping 200 days of data) [2025-10-20T21:59:32.075279] [DEBUG] [LOCAL ] WTTR: Primary data saved (44633 records). [2025-10-20T21:59:32.075427] [INFO ] [MAIN ] WTTR: Full history saved. [2025-10-20T21:59:32.075511] [INFO ] [MAIN ] Update Progress: 2327/2367 symbols completed. WTTR took 33.8s. Estimated remaining: 5m 9s. Rate: 7.8 symbols/min [2025-10-20T21:59:32.245506] [INFO ] [MAIN ] XFLT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:32.251342] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-02. [2025-10-20T21:59:32.275512] [DEBUG] [LOCAL ] XFLT: Loaded 30459 existing records from primary storage. [2025-10-20T21:59:32.275785] [INFO ] [MAIN ] XFLT: Loaded 30459 existing records. [2025-10-20T21:59:32.456571] [WARN ] [MAIN ] XFLT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:32.456810] [INFO ] [MAIN ] XFLT: Wiping existing primary data before full refresh. [2025-10-20T21:59:32.456857] [INFO ] [LOCAL ] XFLT: Starting deletion of primary data. [2025-10-20T21:59:32.457063] [DEBUG] [LOCAL ] XFLT: Deleted primary data file. [2025-10-20T21:59:32.457100] [INFO ] [MAIN ] XFLT: Starting full primary refresh. [2025-10-20T21:59:32.864360] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T21:59:32.868317] [INFO ] [REMOTE ] XFLT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:32.918154] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-08 (1.46 seconds). [2025-10-20T21:59:33.051374] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-01. [2025-10-20T21:59:33.851384] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-09. [2025-10-20T21:59:33.886395] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T21:59:34.363969] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-01 (1.31 seconds). [2025-10-20T21:59:34.651377] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-03. [2025-10-20T21:59:35.451357] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-09. [2025-10-20T21:59:35.668054] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-09 (1.82 seconds). [2025-10-20T21:59:36.251362] [INFO ] [REMOTE ] WU: Fetching primary data for 2025-10. [2025-10-20T21:59:36.431966] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T21:59:36.922074] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-09 (1.47 seconds). [2025-10-20T21:59:37.051333] [INFO ] [REMOTE ] WWW: Fetching primary data for 2025-10. [2025-10-20T21:59:37.851393] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-04. [2025-10-20T21:59:37.955819] [INFO ] [REMOTE ] WU: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T21:59:37.964961] [INFO ] [REMOTE ] WU: Full fetch complete (77513 records). [2025-10-20T21:59:37.965094] [DEBUG] [VALIDATOR ] WU: Validating primary data. [2025-10-20T21:59:38.359822] [DEBUG] [VALIDATOR ] WU: All expected trading days are present. [2025-10-20T21:59:38.359895] [DEBUG] [VALIDATOR ] WU: Primary data validation passed. [2025-10-20T21:59:38.361020] [INFO ] [MAIN ] WU: Fetched primary data is valid. [2025-10-20T21:59:38.361083] [DEBUG] [LOCAL ] WU: Saving primary data. [2025-10-20T21:59:38.362470] [DEBUG] [LOCAL ] WU: Trimmed 77513 records to 53492 (keeping 200 days of data) [2025-10-20T21:59:38.658462] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-02. [2025-10-20T21:59:38.691742] [INFO ] [REMOTE ] WWW: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T21:59:38.712173] [DEBUG] [LOCAL ] WU: Primary data saved (53492 records). [2025-10-20T21:59:38.708636] [INFO ] [REMOTE ] WWW: Full fetch complete (70588 records). [2025-10-20T21:59:38.712435] [INFO ] [MAIN ] WU: Full history saved. [2025-10-20T21:59:38.712581] [INFO ] [MAIN ] Update Progress: 2328/2367 symbols completed. WU took 35.2s. Estimated remaining: 5m 1s. Rate: 7.8 symbols/min [2025-10-20T21:59:38.712366] [DEBUG] [VALIDATOR ] WWW: Validating primary data. [2025-10-20T21:59:39.055119] [INFO ] [MAIN ] XHR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:39.275323] [DEBUG] [VALIDATOR ] WWW: All expected trading days are present. [2025-10-20T21:59:39.275418] [DEBUG] [VALIDATOR ] WWW: Primary data validation passed. [2025-10-20T21:59:39.276525] [INFO ] [MAIN ] WWW: Fetched primary data is valid. [2025-10-20T21:59:39.276724] [DEBUG] [LOCAL ] WWW: Saving primary data. [2025-10-20T21:59:39.287892] [DEBUG] [LOCAL ] WWW: Trimmed 70588 records to 49279 (keeping 200 days of data) [2025-10-20T21:59:39.408908] [DEBUG] [LOCAL ] XHR: Loaded 44119 existing records from primary storage. [2025-10-20T21:59:39.416994] [INFO ] [MAIN ] XHR: Loaded 44119 existing records. [2025-10-20T21:59:39.702730] [WARN ] [MAIN ] XHR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:39.707164] [INFO ] [MAIN ] XHR: Wiping existing primary data before full refresh. [2025-10-20T21:59:39.707331] [INFO ] [LOCAL ] XHR: Starting deletion of primary data. [2025-10-20T21:59:39.713700] [DEBUG] [LOCAL ] XHR: Deleted primary data file. [2025-10-20T21:59:39.713877] [INFO ] [MAIN ] XHR: Starting full primary refresh. [2025-10-20T21:59:40.078938] [DEBUG] [LOCAL ] WWW: Primary data saved (49279 records). [2025-10-20T21:59:40.084972] [INFO ] [MAIN ] WWW: Full history saved. [2025-10-20T21:59:40.090525] [INFO ] [MAIN ] Update Progress: 2329/2367 symbols completed. WWW took 31.0s. Estimated remaining: 4m 53s. Rate: 7.8 symbols/min [2025-10-20T21:59:40.259483] [INFO ] [REMOTE ] XHR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:40.280094] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-02 (1.62 seconds). [2025-10-20T21:59:40.302251] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-03. [2025-10-20T21:59:40.301970] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-01. [2025-10-20T21:59:40.311113] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-04 (2.46 seconds). [2025-10-20T21:59:40.409377] [INFO ] [MAIN ] XIFR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:40.409602] [DEBUG] [LOCAL ] XIFR: No existing data file found in primary storage. [2025-10-20T21:59:40.409671] [WARN ] [MAIN ] XIFR: No existing data found, performing full refresh instead. [2025-10-20T21:59:40.409799] [INFO ] [MAIN ] XIFR: Wiping existing primary data before full refresh. [2025-10-20T21:59:40.409840] [INFO ] [LOCAL ] XIFR: Starting deletion of primary data. [2025-10-20T21:59:40.409882] [DEBUG] [LOCAL ] XIFR: No primary data file found to delete. [2025-10-20T21:59:40.409915] [INFO ] [MAIN ] XIFR: Starting full primary refresh. [2025-10-20T21:59:40.763466] [INFO ] [REMOTE ] XIFR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:41.051373] [INFO ] [REMOTE ] XIFR: Fetching primary data for 2025-01. [2025-10-20T21:59:41.687682] [DEBUG] [REMOTE ] XIFR: Error fetching primary data for 2025-01: Invalid API call. Please retry or visit the documentation (https://www.alphavantage.co/documentation/) for TIME_SERIES_INTRADAY. [2025-10-20T21:59:41.691272] [WARN ] [REMOTE ] XIFR: No data returned for month 2025-01 [2025-10-20T21:59:41.693363] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T21:59:41.692301] [FATAL] [REMOTE ] XIFR: API call failed - aborting fetch to avoid wasting further API calls [2025-10-20T21:59:41.702009] [FATAL] [MAIN ] XIFR: Failed to fetch primary data. [2025-10-20T21:59:41.702474] [WARN ] [MAIN ] XIFR: Deleting existing primary and derivative data due to fetch failure. [2025-10-20T21:59:41.702566] [INFO ] [LOCAL ] XIFR: Starting deletion of primary data. [2025-10-20T21:59:41.702703] [DEBUG] [LOCAL ] XIFR: No primary data file found to delete. [2025-10-20T21:59:41.702887] [INFO ] [LOCAL ] XIFR: Starting deletion of derivative data. [2025-10-20T21:59:41.703018] [DEBUG] [LOCAL ] XIFR: No derivative data file found to delete. [2025-10-20T21:59:41.703104] [INFO ] [MAIN ] Update Progress: 2330/2367 symbols completed. XIFR took 1.6s. Estimated remaining: 4m 45s. Rate: 7.8 symbols/min [2025-10-20T21:59:41.833305] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-01 (1.53 seconds). [2025-10-20T21:59:41.857099] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-04. [2025-10-20T21:59:41.915583] [INFO ] [MAIN ] XOM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:41.969055] [DEBUG] [LOCAL ] XOM: Loaded 54573 existing records from primary storage. [2025-10-20T21:59:41.969416] [INFO ] [MAIN ] XOM: Loaded 54573 existing records. [2025-10-20T21:59:42.145081] [WARN ] [MAIN ] XOM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:42.145278] [INFO ] [MAIN ] XOM: Wiping existing primary data before full refresh. [2025-10-20T21:59:42.145325] [INFO ] [LOCAL ] XOM: Starting deletion of primary data. [2025-10-20T21:59:42.145523] [DEBUG] [LOCAL ] XOM: Deleted primary data file. [2025-10-20T21:59:42.145560] [INFO ] [MAIN ] XOM: Starting full primary refresh. [2025-10-20T21:59:42.489483] [INFO ] [REMOTE ] XOM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:42.651375] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-01. [2025-10-20T21:59:43.209779] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-04 (1.35 seconds). [2025-10-20T21:59:43.451433] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-05. [2025-10-20T21:59:44.251352] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-02. [2025-10-20T21:59:44.703592] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-05 (1.25 seconds). [2025-10-20T21:59:44.830716] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-01 (2.18 seconds). [2025-10-20T21:59:45.051420] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-02. [2025-10-20T21:59:45.258324] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-02 (1.01 seconds). [2025-10-20T21:59:45.851427] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-03. [2025-10-20T21:59:46.651428] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-05. [2025-10-20T21:59:47.110572] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-02 (2.06 seconds). [2025-10-20T21:59:47.419481] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-03 (1.57 seconds). [2025-10-20T21:59:47.451336] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-04. [2025-10-20T21:59:48.251406] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-06. [2025-10-20T21:59:48.565233] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-05 (1.91 seconds). [2025-10-20T21:59:49.062489] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-03. [2025-10-20T21:59:49.068377] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-04 (1.62 seconds). [2025-10-20T21:59:49.393989] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-06 (1.14 seconds). [2025-10-20T21:59:49.851399] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-05. [2025-10-20T21:59:50.651445] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-06. [2025-10-20T21:59:51.390715] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T21:59:51.445017] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-03 (2.38 seconds). [2025-10-20T21:59:51.456223] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-07. [2025-10-20T21:59:52.251356] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-04. [2025-10-20T21:59:52.389617] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-06 (1.74 seconds). [2025-10-20T21:59:52.812510] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-07 (1.36 seconds). [2025-10-20T21:59:53.051452] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-06. [2025-10-20T21:59:53.851372] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-08. [2025-10-20T21:59:54.650089] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-04 (2.40 seconds). [2025-10-20T21:59:54.661725] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-07. [2025-10-20T21:59:54.722500] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-06 (1.67 seconds). [2025-10-20T21:59:55.132668] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-08 (1.28 seconds). [2025-10-20T21:59:55.451422] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-09. [2025-10-20T21:59:56.251377] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-05. [2025-10-20T21:59:56.351246] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-07 (1.69 seconds). [2025-10-20T21:59:56.412526] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-09 (0.96 seconds). [2025-10-20T21:59:57.051428] [INFO ] [REMOTE ] XFLT: Fetching primary data for 2025-10. [2025-10-20T21:59:57.851352] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-07. [2025-10-20T21:59:58.175287] [INFO ] [REMOTE ] XFLT: Incremental fetch complete for 2025-10 (1.12 seconds). [2025-10-20T21:59:58.180227] [INFO ] [REMOTE ] XFLT: Full fetch complete (43969 records). [2025-10-20T21:59:58.180348] [DEBUG] [VALIDATOR ] XFLT: Validating primary data. [2025-10-20T21:59:58.563314] [DEBUG] [VALIDATOR ] XFLT: All expected trading days are present. [2025-10-20T21:59:58.563423] [DEBUG] [VALIDATOR ] XFLT: Primary data validation passed. [2025-10-20T21:59:58.564482] [INFO ] [MAIN ] XFLT: Fetched primary data is valid. [2025-10-20T21:59:58.564597] [DEBUG] [LOCAL ] XFLT: Saving primary data. [2025-10-20T21:59:58.565879] [DEBUG] [LOCAL ] XFLT: Trimmed 43969 records to 29989 (keeping 200 days of data) [2025-10-20T21:59:58.651479] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-08. [2025-10-20T21:59:58.682263] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-05 (2.43 seconds). [2025-10-20T21:59:58.782144] [DEBUG] [LOCAL ] XFLT: Primary data saved (29989 records). [2025-10-20T21:59:58.782322] [INFO ] [MAIN ] XFLT: Full history saved. [2025-10-20T21:59:58.782423] [INFO ] [MAIN ] Update Progress: 2331/2367 symbols completed. XFLT took 26.7s. Estimated remaining: 4m 38s. Rate: 7.8 symbols/min [2025-10-20T21:59:58.946123] [INFO ] [MAIN ] XPER: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T21:59:58.974916] [DEBUG] [LOCAL ] XPER: Loaded 31034 existing records from primary storage. [2025-10-20T21:59:58.975184] [INFO ] [MAIN ] XPER: Loaded 31034 existing records. [2025-10-20T21:59:59.149210] [WARN ] [MAIN ] XPER: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T21:59:59.149404] [INFO ] [MAIN ] XPER: Wiping existing primary data before full refresh. [2025-10-20T21:59:59.149451] [INFO ] [LOCAL ] XPER: Starting deletion of primary data. [2025-10-20T21:59:59.149629] [DEBUG] [LOCAL ] XPER: Deleted primary data file. [2025-10-20T21:59:59.149711] [INFO ] [MAIN ] XPER: Starting full primary refresh. [2025-10-20T21:59:59.451754] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-06. [2025-10-20T21:59:59.547227] [INFO ] [REMOTE ] XPER: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T21:59:59.552851] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T22:00:00.251460] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-08. [2025-10-20T22:00:00.764878] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-08 (2.11 seconds). [2025-10-20T22:00:01.051381] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-01. [2025-10-20T22:00:01.771395] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T22:00:01.851404] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-09. [2025-10-20T22:00:02.026846] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-06 (2.58 seconds). [2025-10-20T22:00:02.553703] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-01 (1.50 seconds). [2025-10-20T22:00:02.651391] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-07. [2025-10-20T22:00:03.308582] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T22:00:03.451369] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-02. [2025-10-20T22:00:04.251384] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-09. [2025-10-20T22:00:04.884994] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T22:00:05.010195] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-07 (2.36 seconds). [2025-10-20T22:00:05.051335] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-08. [2025-10-20T22:00:05.384381] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-09 (1.13 seconds). [2025-10-20T22:00:05.851370] [INFO ] [REMOTE ] WY: Fetching primary data for 2025-10. [2025-10-20T22:00:06.579015] [INFO ] [REMOTE ] WY: Incremental fetch complete for 2025-10 (0.73 seconds). [2025-10-20T22:00:06.586796] [INFO ] [REMOTE ] WY: Full fetch complete (77036 records). [2025-10-20T22:00:06.586938] [DEBUG] [VALIDATOR ] WY: Validating primary data. [2025-10-20T22:00:06.656501] [INFO ] [REMOTE ] XHR: Fetching primary data for 2025-10. [2025-10-20T22:00:06.999459] [DEBUG] [VALIDATOR ] WY: All expected trading days are present. [2025-10-20T22:00:06.999704] [DEBUG] [VALIDATOR ] WY: Primary data validation passed. [2025-10-20T22:00:07.001008] [INFO ] [MAIN ] WY: Fetched primary data is valid. [2025-10-20T22:00:07.001147] [DEBUG] [LOCAL ] WY: Saving primary data. [2025-10-20T22:00:07.002964] [DEBUG] [LOCAL ] WY: Trimmed 77036 records to 53216 (keeping 200 days of data) [2025-10-20T22:00:07.265002] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-08 (2.21 seconds). [2025-10-20T22:00:07.370703] [DEBUG] [LOCAL ] WY: Primary data saved (53216 records). [2025-10-20T22:00:07.370833] [INFO ] [MAIN ] WY: Full history saved. [2025-10-20T22:00:07.370910] [INFO ] [MAIN ] Update Progress: 2332/2367 symbols completed. WY took 40.3s. Estimated remaining: 4m 30s. Rate: 7.8 symbols/min [2025-10-20T22:00:07.459041] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-09. [2025-10-20T22:00:07.540370] [INFO ] [MAIN ] XPEV: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:07.600421] [DEBUG] [LOCAL ] XPEV: Loaded 54409 existing records from primary storage. [2025-10-20T22:00:07.600779] [INFO ] [MAIN ] XPEV: Loaded 54409 existing records. [2025-10-20T22:00:07.789550] [WARN ] [MAIN ] XPEV: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:07.789770] [INFO ] [MAIN ] XPEV: Wiping existing primary data before full refresh. [2025-10-20T22:00:07.789818] [INFO ] [LOCAL ] XPEV: Starting deletion of primary data. [2025-10-20T22:00:07.790001] [DEBUG] [LOCAL ] XPEV: Deleted primary data file. [2025-10-20T22:00:07.790039] [INFO ] [MAIN ] XPEV: Starting full primary refresh. [2025-10-20T22:00:08.147860] [INFO ] [REMOTE ] XPEV: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:08.165233] [INFO ] [REMOTE ] XHR: Incremental fetch complete for 2025-10 (1.51 seconds). [2025-10-20T22:00:08.171430] [INFO ] [REMOTE ] XHR: Full fetch complete (61819 records). [2025-10-20T22:00:08.171567] [DEBUG] [VALIDATOR ] XHR: Validating primary data. [2025-10-20T22:00:08.256617] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-03. [2025-10-20T22:00:08.560168] [DEBUG] [VALIDATOR ] XHR: All expected trading days are present. [2025-10-20T22:00:08.560248] [DEBUG] [VALIDATOR ] XHR: Primary data validation passed. [2025-10-20T22:00:08.561286] [INFO ] [MAIN ] XHR: Fetched primary data is valid. [2025-10-20T22:00:08.561351] [DEBUG] [LOCAL ] XHR: Saving primary data. [2025-10-20T22:00:08.562600] [DEBUG] [LOCAL ] XHR: Trimmed 61819 records to 43474 (keeping 200 days of data) [2025-10-20T22:00:08.815988] [DEBUG] [LOCAL ] XHR: Primary data saved (43474 records). [2025-10-20T22:00:08.816110] [INFO ] [MAIN ] XHR: Full history saved. [2025-10-20T22:00:08.816182] [INFO ] [MAIN ] Update Progress: 2333/2367 symbols completed. XHR took 30.1s. Estimated remaining: 4m 22s. Rate: 7.8 symbols/min [2025-10-20T22:00:08.983698] [INFO ] [MAIN ] XPO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:09.031073] [DEBUG] [LOCAL ] XPO: Loaded 48306 existing records from primary storage. [2025-10-20T22:00:09.031434] [INFO ] [MAIN ] XPO: Loaded 48306 existing records. [2025-10-20T22:00:09.062671] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-01. [2025-10-20T22:00:09.210660] [WARN ] [MAIN ] XPO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:09.210858] [INFO ] [MAIN ] XPO: Wiping existing primary data before full refresh. [2025-10-20T22:00:09.210905] [INFO ] [LOCAL ] XPO: Starting deletion of primary data. [2025-10-20T22:00:09.211088] [DEBUG] [LOCAL ] XPO: Deleted primary data file. [2025-10-20T22:00:09.211125] [INFO ] [MAIN ] XPO: Starting full primary refresh. [2025-10-20T22:00:09.562986] [INFO ] [REMOTE ] XPO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:09.654305] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-03 (1.40 seconds). [2025-10-20T22:00:09.784267] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-09 (2.33 seconds). [2025-10-20T22:00:09.851374] [INFO ] [REMOTE ] XOM: Fetching primary data for 2025-10. [2025-10-20T22:00:10.595772] [INFO ] [REMOTE ] XOM: Incremental fetch complete for 2025-10 (0.74 seconds). [2025-10-20T22:00:10.603802] [INFO ] [REMOTE ] XOM: Full fetch complete (77973 records). [2025-10-20T22:00:10.603930] [DEBUG] [VALIDATOR ] XOM: Validating primary data. [2025-10-20T22:00:10.662542] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-01. [2025-10-20T22:00:11.022701] [DEBUG] [VALIDATOR ] XOM: All expected trading days are present. [2025-10-20T22:00:11.022790] [DEBUG] [VALIDATOR ] XOM: Primary data validation passed. [2025-10-20T22:00:11.023929] [INFO ] [MAIN ] XOM: Fetched primary data is valid. [2025-10-20T22:00:11.023989] [DEBUG] [LOCAL ] XOM: Saving primary data. [2025-10-20T22:00:11.025367] [DEBUG] [LOCAL ] XOM: Trimmed 77973 records to 53793 (keeping 200 days of data) [2025-10-20T22:00:11.361332] [DEBUG] [LOCAL ] XOM: Primary data saved (53793 records). [2025-10-20T22:00:11.361456] [INFO ] [MAIN ] XOM: Full history saved. [2025-10-20T22:00:11.361530] [INFO ] [MAIN ] Update Progress: 2334/2367 symbols completed. XOM took 29.7s. Estimated remaining: 4m 15s. Rate: 7.8 symbols/min [2025-10-20T22:00:11.454068] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-04. [2025-10-20T22:00:11.537781] [INFO ] [MAIN ] XPOF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:11.566350] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-01 (2.50 seconds). [2025-10-20T22:00:11.605553] [DEBUG] [LOCAL ] XPOF: Loaded 37953 existing records from primary storage. [2025-10-20T22:00:11.605925] [INFO ] [MAIN ] XPOF: Loaded 37953 existing records. [2025-10-20T22:00:11.778103] [WARN ] [MAIN ] XPOF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:11.778300] [INFO ] [MAIN ] XPOF: Wiping existing primary data before full refresh. [2025-10-20T22:00:11.778347] [INFO ] [LOCAL ] XPOF: Starting deletion of primary data. [2025-10-20T22:00:11.778566] [DEBUG] [LOCAL ] XPOF: Deleted primary data file. [2025-10-20T22:00:11.778621] [INFO ] [MAIN ] XPOF: Starting full primary refresh. [2025-10-20T22:00:12.128143] [INFO ] [REMOTE ] XPOF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:12.251374] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-01. [2025-10-20T22:00:12.395143] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-01 (1.73 seconds). [2025-10-20T22:00:12.467419] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-04 (1.01 seconds). [2025-10-20T22:00:13.051400] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-05. [2025-10-20T22:00:13.523108] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-01 (1.27 seconds). [2025-10-20T22:00:13.851385] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-02. [2025-10-20T22:00:14.450345] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-05 (1.40 seconds). [2025-10-20T22:00:14.651385] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-02. [2025-10-20T22:00:15.451370] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-02. [2025-10-20T22:00:15.511292] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T22:00:16.251389] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-03. [2025-10-20T22:00:16.658993] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-02 (1.21 seconds). [2025-10-20T22:00:17.014281] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-02 (2.36 seconds). [2025-10-20T22:00:17.051377] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-03. [2025-10-20T22:00:17.851377] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-03. [2025-10-20T22:00:18.079047] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T22:00:18.292749] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-03 (1.24 seconds). [2025-10-20T22:00:18.651438] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-06. [2025-10-20T22:00:19.451393] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-04. [2025-10-20T22:00:19.744046] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-06 (1.09 seconds). [2025-10-20T22:00:20.251403] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-07. [2025-10-20T22:00:20.451461] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-03 (2.60 seconds). [2025-10-20T22:00:20.984469] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T22:00:21.051335] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-05. [2025-10-20T22:00:21.183103] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-07 (0.93 seconds). [2025-10-20T22:00:21.851377] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-08. [2025-10-20T22:00:22.651360] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-04. [2025-10-20T22:00:22.683123] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-05 (1.63 seconds). [2025-10-20T22:00:23.231058] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-08 (1.38 seconds). [2025-10-20T22:00:23.451397] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-04. [2025-10-20T22:00:24.212378] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-04 (1.56 seconds). [2025-10-20T22:00:24.251371] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-06. [2025-10-20T22:00:25.051395] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-09. [2025-10-20T22:00:25.603594] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-06 (1.35 seconds). [2025-10-20T22:00:25.851387] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-07. [2025-10-20T22:00:26.035027] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-04 (2.58 seconds). [2025-10-20T22:00:26.435972] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-09 (1.38 seconds). [2025-10-20T22:00:26.651370] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-05. [2025-10-20T22:00:27.044910] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-07 (1.19 seconds). [2025-10-20T22:00:27.451392] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-08. [2025-10-20T22:00:28.251374] [INFO ] [REMOTE ] XPER: Fetching primary data for 2025-10. [2025-10-20T22:00:28.459556] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-05 (1.81 seconds). [2025-10-20T22:00:28.536054] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-08 (1.08 seconds). [2025-10-20T22:00:29.051366] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-09. [2025-10-20T22:00:29.554957] [INFO ] [REMOTE ] XPER: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T22:00:29.560363] [INFO ] [REMOTE ] XPER: Full fetch complete (43734 records). [2025-10-20T22:00:29.560470] [DEBUG] [VALIDATOR ] XPER: Validating primary data. [2025-10-20T22:00:29.856555] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-05. [2025-10-20T22:00:29.959528] [DEBUG] [VALIDATOR ] XPER: All expected trading days are present. [2025-10-20T22:00:29.959607] [DEBUG] [VALIDATOR ] XPER: Primary data validation passed. [2025-10-20T22:00:29.960520] [INFO ] [MAIN ] XPER: Fetched primary data is valid. [2025-10-20T22:00:29.960581] [DEBUG] [LOCAL ] XPER: Saving primary data. [2025-10-20T22:00:29.961561] [DEBUG] [LOCAL ] XPER: Trimmed 43734 records to 30766 (keeping 200 days of data) [2025-10-20T22:00:30.135663] [DEBUG] [LOCAL ] XPER: Primary data saved (30766 records). [2025-10-20T22:00:30.135812] [INFO ] [MAIN ] XPER: Full history saved. [2025-10-20T22:00:30.135890] [INFO ] [MAIN ] Update Progress: 2335/2367 symbols completed. XPER took 31.4s. Estimated remaining: 4m 7s. Rate: 7.8 symbols/min [2025-10-20T22:00:30.313402] [INFO ] [MAIN ] XPRO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:30.357323] [DEBUG] [LOCAL ] XPRO: Loaded 46552 existing records from primary storage. [2025-10-20T22:00:30.357822] [INFO ] [MAIN ] XPRO: Loaded 46552 existing records. [2025-10-20T22:00:30.514085] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-09 (1.46 seconds). [2025-10-20T22:00:30.575147] [WARN ] [MAIN ] XPRO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:30.575588] [INFO ] [MAIN ] XPRO: Wiping existing primary data before full refresh. [2025-10-20T22:00:30.575636] [INFO ] [LOCAL ] XPRO: Starting deletion of primary data. [2025-10-20T22:00:30.575890] [DEBUG] [LOCAL ] XPRO: Deleted primary data file. [2025-10-20T22:00:30.575950] [INFO ] [MAIN ] XPRO: Starting full primary refresh. [2025-10-20T22:00:30.656686] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-06. [2025-10-20T22:00:30.927332] [INFO ] [REMOTE ] XPRO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:31.451412] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-01. [2025-10-20T22:00:32.255141] [INFO ] [REMOTE ] XPOF: Fetching primary data for 2025-10. [2025-10-20T22:00:32.269413] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-05 (2.41 seconds). [2025-10-20T22:00:32.403423] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-06 (1.75 seconds). [2025-10-20T22:00:33.033092] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T22:00:33.051338] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-02. [2025-10-20T22:00:33.477135] [INFO ] [REMOTE ] XPOF: Incremental fetch complete for 2025-10 (1.22 seconds). [2025-10-20T22:00:33.482445] [INFO ] [REMOTE ] XPOF: Full fetch complete (52975 records). [2025-10-20T22:00:33.482539] [DEBUG] [VALIDATOR ] XPOF: Validating primary data. [2025-10-20T22:00:33.851416] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-07. [2025-10-20T22:00:33.865838] [DEBUG] [VALIDATOR ] XPOF: All expected trading days are present. [2025-10-20T22:00:33.865917] [DEBUG] [VALIDATOR ] XPOF: Primary data validation passed. [2025-10-20T22:00:33.866862] [INFO ] [MAIN ] XPOF: Fetched primary data is valid. [2025-10-20T22:00:33.866924] [DEBUG] [LOCAL ] XPOF: Saving primary data. [2025-10-20T22:00:33.868048] [DEBUG] [LOCAL ] XPOF: Trimmed 52975 records to 37197 (keeping 200 days of data) [2025-10-20T22:00:34.073101] [DEBUG] [LOCAL ] XPOF: Primary data saved (37197 records). [2025-10-20T22:00:34.073225] [INFO ] [MAIN ] XPOF: Full history saved. [2025-10-20T22:00:34.073298] [INFO ] [MAIN ] Update Progress: 2336/2367 symbols completed. XPOF took 22.7s. Estimated remaining: 3m 59s. Rate: 7.8 symbols/min [2025-10-20T22:00:34.261786] [INFO ] [MAIN ] XYF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:34.286030] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-02 (1.23 seconds). [2025-10-20T22:00:34.317740] [DEBUG] [LOCAL ] XYF: Loaded 28115 existing records from primary storage. [2025-10-20T22:00:34.318019] [INFO ] [MAIN ] XYF: Loaded 28115 existing records. [2025-10-20T22:00:34.494116] [WARN ] [MAIN ] XYF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:34.494319] [INFO ] [MAIN ] XYF: Wiping existing primary data before full refresh. [2025-10-20T22:00:34.494365] [INFO ] [LOCAL ] XYF: Starting deletion of primary data. [2025-10-20T22:00:34.494538] [DEBUG] [LOCAL ] XYF: Deleted primary data file. [2025-10-20T22:00:34.494576] [INFO ] [MAIN ] XYF: Starting full primary refresh. [2025-10-20T22:00:34.653390] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-06. [2025-10-20T22:00:34.861938] [INFO ] [REMOTE ] XYF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:35.451439] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-03. [2025-10-20T22:00:35.834429] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-07 (1.98 seconds). [2025-10-20T22:00:36.251448] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-08. [2025-10-20T22:00:37.031985] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-06 (2.38 seconds). [2025-10-20T22:00:37.051327] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-07. [2025-10-20T22:00:37.117361] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-03 (1.67 seconds). [2025-10-20T22:00:37.793267] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-08 (1.54 seconds). [2025-10-20T22:00:37.851335] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-09. [2025-10-20T22:00:38.651432] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-01. [2025-10-20T22:00:39.451412] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-04. [2025-10-20T22:00:39.495043] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-01 (0.84 seconds). [2025-10-20T22:00:39.496704] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-07 (2.45 seconds). [2025-10-20T22:00:39.619032] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T22:00:40.251361] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-08. [2025-10-20T22:00:40.951427] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-04 (1.50 seconds). [2025-10-20T22:00:41.051349] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-05. [2025-10-20T22:00:41.851404] [INFO ] [REMOTE ] XPO: Fetching primary data for 2025-10. [2025-10-20T22:00:42.651337] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-02. [2025-10-20T22:00:42.734741] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-05 (1.68 seconds). [2025-10-20T22:00:42.746321] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-08 (2.49 seconds). [2025-10-20T22:00:43.146397] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-02 (0.50 seconds). [2025-10-20T22:00:43.451429] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-06. [2025-10-20T22:00:44.251361] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-09. [2025-10-20T22:00:44.420920] [INFO ] [REMOTE ] XPO: Incremental fetch complete for 2025-10 (2.57 seconds). [2025-10-20T22:00:44.427828] [INFO ] [REMOTE ] XPO: Full fetch complete (69703 records). [2025-10-20T22:00:44.427979] [DEBUG] [VALIDATOR ] XPO: Validating primary data. [2025-10-20T22:00:44.834377] [DEBUG] [VALIDATOR ] XPO: All expected trading days are present. [2025-10-20T22:00:44.834455] [DEBUG] [VALIDATOR ] XPO: Primary data validation passed. [2025-10-20T22:00:44.835717] [INFO ] [MAIN ] XPO: Fetched primary data is valid. [2025-10-20T22:00:44.835874] [DEBUG] [LOCAL ] XPO: Saving primary data. [2025-10-20T22:00:44.837290] [DEBUG] [LOCAL ] XPO: Trimmed 69703 records to 47569 (keeping 200 days of data) [2025-10-20T22:00:45.056407] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-03. [2025-10-20T22:00:45.099350] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-06 (1.65 seconds). [2025-10-20T22:00:45.158151] [DEBUG] [LOCAL ] XPO: Primary data saved (47569 records). [2025-10-20T22:00:45.158272] [INFO ] [MAIN ] XPO: Full history saved. [2025-10-20T22:00:45.158350] [INFO ] [MAIN ] Update Progress: 2337/2367 symbols completed. XPO took 36.3s. Estimated remaining: 3m 52s. Rate: 7.8 symbols/min [2025-10-20T22:00:45.339720] [INFO ] [MAIN ] XYL: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:45.384932] [DEBUG] [LOCAL ] XYL: Loaded 48469 existing records from primary storage. [2025-10-20T22:00:45.385252] [INFO ] [MAIN ] XYL: Loaded 48469 existing records. [2025-10-20T22:00:45.562051] [WARN ] [MAIN ] XYL: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:45.562238] [INFO ] [MAIN ] XYL: Wiping existing primary data before full refresh. [2025-10-20T22:00:45.562283] [INFO ] [LOCAL ] XYL: Starting deletion of primary data. [2025-10-20T22:00:45.562595] [DEBUG] [LOCAL ] XYL: Deleted primary data file. [2025-10-20T22:00:45.562680] [INFO ] [MAIN ] XYL: Starting full primary refresh. [2025-10-20T22:00:45.856605] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-07. [2025-10-20T22:00:45.914157] [INFO ] [REMOTE ] XYL: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:46.358766] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T22:00:46.651384] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-04. [2025-10-20T22:00:46.731014] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-09 (2.48 seconds). [2025-10-20T22:00:47.438881] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-07 (1.58 seconds). [2025-10-20T22:00:47.451350] [INFO ] [REMOTE ] XPEV: Fetching primary data for 2025-10. [2025-10-20T22:00:47.802049] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T22:00:48.251422] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-01. [2025-10-20T22:00:49.051430] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-05. [2025-10-20T22:00:49.726913] [INFO ] [REMOTE ] XPEV: Incremental fetch complete for 2025-10 (2.28 seconds). [2025-10-20T22:00:49.735271] [INFO ] [REMOTE ] XPEV: Full fetch complete (77779 records). [2025-10-20T22:00:49.735401] [DEBUG] [VALIDATOR ] XPEV: Validating primary data. [2025-10-20T22:00:49.858799] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-08. [2025-10-20T22:00:50.165389] [DEBUG] [VALIDATOR ] XPEV: All expected trading days are present. [2025-10-20T22:00:50.165568] [DEBUG] [VALIDATOR ] XPEV: Primary data validation passed. [2025-10-20T22:00:50.167162] [INFO ] [MAIN ] XPEV: Fetched primary data is valid. [2025-10-20T22:00:50.167316] [DEBUG] [LOCAL ] XPEV: Saving primary data. [2025-10-20T22:00:50.169396] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-01 (1.92 seconds). [2025-10-20T22:00:50.179865] [DEBUG] [LOCAL ] XPEV: Trimmed 77779 records to 53624 (keeping 200 days of data) [2025-10-20T22:00:50.368331] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-05 (1.32 seconds). [2025-10-20T22:00:50.515393] [DEBUG] [LOCAL ] XPEV: Primary data saved (53624 records). [2025-10-20T22:00:50.515539] [INFO ] [MAIN ] XPEV: Full history saved. [2025-10-20T22:00:50.515623] [INFO ] [MAIN ] Update Progress: 2338/2367 symbols completed. XPEV took 43.1s. Estimated remaining: 3m 44s. Rate: 7.8 symbols/min [2025-10-20T22:00:50.659301] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-06. [2025-10-20T22:00:50.692009] [INFO ] [MAIN ] XYZ: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:50.692247] [DEBUG] [LOCAL ] XYZ: No existing data file found in primary storage. [2025-10-20T22:00:50.692293] [WARN ] [MAIN ] XYZ: No existing data found, performing full refresh instead. [2025-10-20T22:00:50.692412] [INFO ] [MAIN ] XYZ: Wiping existing primary data before full refresh. [2025-10-20T22:00:50.692454] [INFO ] [LOCAL ] XYZ: Starting deletion of primary data. [2025-10-20T22:00:50.692542] [DEBUG] [LOCAL ] XYZ: No primary data file found to delete. [2025-10-20T22:00:50.692595] [INFO ] [MAIN ] XYZ: Starting full primary refresh. [2025-10-20T22:00:51.045601] [INFO ] [REMOTE ] XYZ: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:51.451352] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-01. [2025-10-20T22:00:51.591621] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-08 (1.73 seconds). [2025-10-20T22:00:51.740706] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-06 (1.08 seconds). [2025-10-20T22:00:52.251426] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-09. [2025-10-20T22:00:52.753460] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-01 (1.30 seconds). [2025-10-20T22:00:53.051449] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-07. [2025-10-20T22:00:53.832938] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T22:00:53.851355] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-02. [2025-10-20T22:00:54.196481] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-07 (1.15 seconds). [2025-10-20T22:00:54.651374] [INFO ] [REMOTE ] XPRO: Fetching primary data for 2025-10. [2025-10-20T22:00:55.443478] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T22:00:55.452370] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-08. [2025-10-20T22:00:56.065107] [INFO ] [REMOTE ] XPRO: Incremental fetch complete for 2025-10 (1.41 seconds). [2025-10-20T22:00:56.071816] [INFO ] [REMOTE ] XPRO: Full fetch complete (66435 records). [2025-10-20T22:00:56.071925] [DEBUG] [VALIDATOR ] XPRO: Validating primary data. [2025-10-20T22:00:56.261895] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-02. [2025-10-20T22:00:56.454162] [DEBUG] [VALIDATOR ] XPRO: All expected trading days are present. [2025-10-20T22:00:56.454235] [DEBUG] [VALIDATOR ] XPRO: Primary data validation passed. [2025-10-20T22:00:56.455299] [INFO ] [MAIN ] XPRO: Fetched primary data is valid. [2025-10-20T22:00:56.455360] [DEBUG] [LOCAL ] XPRO: Saving primary data. [2025-10-20T22:00:56.456587] [DEBUG] [LOCAL ] XPRO: Trimmed 66435 records to 45873 (keeping 200 days of data) [2025-10-20T22:00:56.707499] [DEBUG] [LOCAL ] XPRO: Primary data saved (45873 records). [2025-10-20T22:00:56.707620] [INFO ] [MAIN ] XPRO: Full history saved. [2025-10-20T22:00:56.707725] [INFO ] [MAIN ] Update Progress: 2339/2367 symbols completed. XPRO took 26.6s. Estimated remaining: 3m 36s. Rate: 7.8 symbols/min [2025-10-20T22:00:56.878333] [INFO ] [MAIN ] YALA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:00:56.914920] [DEBUG] [LOCAL ] YALA: Loaded 37162 existing records from primary storage. [2025-10-20T22:00:56.915230] [INFO ] [MAIN ] YALA: Loaded 37162 existing records. [2025-10-20T22:00:57.015027] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-08 (1.56 seconds). [2025-10-20T22:00:57.054075] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-03. [2025-10-20T22:00:57.123050] [WARN ] [MAIN ] YALA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:00:57.123267] [INFO ] [MAIN ] YALA: Wiping existing primary data before full refresh. [2025-10-20T22:00:57.123334] [INFO ] [LOCAL ] YALA: Starting deletion of primary data. [2025-10-20T22:00:57.123529] [DEBUG] [LOCAL ] YALA: Deleted primary data file. [2025-10-20T22:00:57.123584] [INFO ] [MAIN ] YALA: Starting full primary refresh. [2025-10-20T22:00:57.465908] [INFO ] [REMOTE ] YALA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:00:57.851384] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-09. [2025-10-20T22:00:58.286102] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-02 (2.02 seconds). [2025-10-20T22:00:58.651351] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-03. [2025-10-20T22:00:58.916525] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-03 (1.86 seconds). [2025-10-20T22:00:58.967447] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T22:00:59.451441] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-01. [2025-10-20T22:01:00.251354] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-04. [2025-10-20T22:01:00.392674] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-01 (0.94 seconds). [2025-10-20T22:01:01.051353] [INFO ] [REMOTE ] XYF: Fetching primary data for 2025-10. [2025-10-20T22:01:01.299548] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-03 (2.65 seconds). [2025-10-20T22:01:01.851382] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-02. [2025-10-20T22:01:02.131560] [INFO ] [REMOTE ] XYF: Incremental fetch complete for 2025-10 (1.08 seconds). [2025-10-20T22:01:02.135564] [INFO ] [REMOTE ] XYF: Full fetch complete (33518 records). [2025-10-20T22:01:02.135719] [DEBUG] [VALIDATOR ] XYF: Validating primary data. [2025-10-20T22:01:02.542067] [DEBUG] [VALIDATOR ] XYF: All expected trading days are present. [2025-10-20T22:01:02.542282] [DEBUG] [VALIDATOR ] XYF: Primary data validation passed. [2025-10-20T22:01:02.543155] [INFO ] [MAIN ] XYF: Fetched primary data is valid. [2025-10-20T22:01:02.543240] [DEBUG] [LOCAL ] XYF: Saving primary data. [2025-10-20T22:01:02.544814] [DEBUG] [LOCAL ] XYF: Trimmed 33518 records to 27638 (keeping 200 days of data) [2025-10-20T22:01:02.669773] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-04 (2.42 seconds). [2025-10-20T22:01:02.664980] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-04. [2025-10-20T22:01:02.706887] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-02 (0.86 seconds). [2025-10-20T22:01:02.784836] [DEBUG] [LOCAL ] XYF: Primary data saved (27638 records). [2025-10-20T22:01:02.784993] [INFO ] [MAIN ] XYF: Full history saved. [2025-10-20T22:01:02.785076] [INFO ] [MAIN ] Update Progress: 2340/2367 symbols completed. XYF took 28.7s. Estimated remaining: 3m 28s. Rate: 7.8 symbols/min [2025-10-20T22:01:02.950044] [INFO ] [MAIN ] YELP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:02.988726] [DEBUG] [LOCAL ] YELP: Loaded 46079 existing records from primary storage. [2025-10-20T22:01:02.989020] [INFO ] [MAIN ] YELP: Loaded 46079 existing records. [2025-10-20T22:01:03.163853] [WARN ] [MAIN ] YELP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:03.164070] [INFO ] [MAIN ] YELP: Wiping existing primary data before full refresh. [2025-10-20T22:01:03.164117] [INFO ] [LOCAL ] YELP: Starting deletion of primary data. [2025-10-20T22:01:03.164432] [DEBUG] [LOCAL ] YELP: Deleted primary data file. [2025-10-20T22:01:03.164492] [INFO ] [MAIN ] YELP: Starting full primary refresh. [2025-10-20T22:01:03.456603] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-05. [2025-10-20T22:01:03.515155] [INFO ] [REMOTE ] YELP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:04.251352] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-03. [2025-10-20T22:01:04.927912] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-04 (2.26 seconds). [2025-10-20T22:01:05.051385] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-05. [2025-10-20T22:01:05.138841] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-03 (0.89 seconds). [2025-10-20T22:01:05.302134] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-05 (1.85 seconds). [2025-10-20T22:01:05.851419] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-06. [2025-10-20T22:01:06.651379] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-01. [2025-10-20T22:01:07.348143] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-05 (2.30 seconds). [2025-10-20T22:01:07.437203] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-06 (1.59 seconds). [2025-10-20T22:01:07.451331] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-07. [2025-10-20T22:01:07.857111] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-01 (1.21 seconds). [2025-10-20T22:01:08.251368] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-02. [2025-10-20T22:01:09.051391] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-04. [2025-10-20T22:01:09.150169] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-07 (1.70 seconds). [2025-10-20T22:01:09.685456] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-02 (1.43 seconds). [2025-10-20T22:01:09.851406] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-06. [2025-10-20T22:01:10.651540] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-08. [2025-10-20T22:01:10.682478] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-04 (1.63 seconds). [2025-10-20T22:01:11.451436] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-03. [2025-10-20T22:01:11.740782] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-06 (1.89 seconds). [2025-10-20T22:01:12.251366] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-05. [2025-10-20T22:01:12.364047] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-08 (1.71 seconds). [2025-10-20T22:01:12.845094] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-03 (1.39 seconds). [2025-10-20T22:01:13.051448] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-04. [2025-10-20T22:01:13.851354] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-07. [2025-10-20T22:01:13.900363] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-05 (1.65 seconds). [2025-10-20T22:01:14.642029] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T22:01:14.651381] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-09. [2025-10-20T22:01:15.451397] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-06. [2025-10-20T22:01:16.196942] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-07 (2.35 seconds). [2025-10-20T22:01:16.251338] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-08. [2025-10-20T22:01:16.356096] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T22:01:16.994064] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-06 (1.54 seconds). [2025-10-20T22:01:17.051373] [INFO ] [REMOTE ] XYL: Fetching primary data for 2025-10. [2025-10-20T22:01:17.804096] [INFO ] [REMOTE ] XYL: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T22:01:17.811428] [INFO ] [REMOTE ] XYL: Full fetch complete (70057 records). [2025-10-20T22:01:17.811561] [DEBUG] [VALIDATOR ] XYL: Validating primary data. [2025-10-20T22:01:17.856679] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-07. [2025-10-20T22:01:18.200159] [DEBUG] [VALIDATOR ] XYL: All expected trading days are present. [2025-10-20T22:01:18.200239] [DEBUG] [VALIDATOR ] XYL: Primary data validation passed. [2025-10-20T22:01:18.201326] [INFO ] [MAIN ] XYL: Fetched primary data is valid. [2025-10-20T22:01:18.201389] [DEBUG] [LOCAL ] XYL: Saving primary data. [2025-10-20T22:01:18.202745] [DEBUG] [LOCAL ] XYL: Trimmed 70057 records to 47669 (keeping 200 days of data) [2025-10-20T22:01:18.496058] [DEBUG] [LOCAL ] XYL: Primary data saved (47669 records). [2025-10-20T22:01:18.496200] [INFO ] [MAIN ] XYL: Full history saved. [2025-10-20T22:01:18.496280] [INFO ] [MAIN ] Update Progress: 2341/2367 symbols completed. XYL took 33.3s. Estimated remaining: 3m 21s. Rate: 7.8 symbols/min [2025-10-20T22:01:18.695128] [INFO ] [MAIN ] YETI: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:18.696885] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-05. [2025-10-20T22:01:18.696836] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-08 (2.45 seconds). [2025-10-20T22:01:18.755712] [DEBUG] [LOCAL ] YETI: Loaded 53250 existing records from primary storage. [2025-10-20T22:01:18.756027] [INFO ] [MAIN ] YETI: Loaded 53250 existing records. [2025-10-20T22:01:18.934545] [WARN ] [MAIN ] YETI: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:18.934806] [INFO ] [MAIN ] YETI: Wiping existing primary data before full refresh. [2025-10-20T22:01:18.934885] [INFO ] [LOCAL ] YETI: Starting deletion of primary data. [2025-10-20T22:01:18.935188] [DEBUG] [LOCAL ] YETI: Deleted primary data file. [2025-10-20T22:01:18.941504] [INFO ] [MAIN ] YETI: Starting full primary refresh. [2025-10-20T22:01:19.027986] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-07 (1.17 seconds). [2025-10-20T22:01:19.311228] [INFO ] [REMOTE ] YETI: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:19.451337] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-01. [2025-10-20T22:01:19.886128] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-05 (1.19 seconds). [2025-10-20T22:01:20.251382] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-08. [2025-10-20T22:01:20.806816] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-01 (1.36 seconds). [2025-10-20T22:01:21.051427] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-02. [2025-10-20T22:01:21.754111] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T22:01:21.851360] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-09. [2025-10-20T22:01:22.651949] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-06. [2025-10-20T22:01:22.656325] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-02 (1.60 seconds). [2025-10-20T22:01:23.081292] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-09 (1.23 seconds). [2025-10-20T22:01:23.451370] [INFO ] [REMOTE ] YALA: Fetching primary data for 2025-10. [2025-10-20T22:01:24.199330] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-06 (1.55 seconds). [2025-10-20T22:01:24.251370] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-07. [2025-10-20T22:01:24.617099] [INFO ] [REMOTE ] YALA: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T22:01:24.621401] [INFO ] [REMOTE ] YALA: Full fetch complete (47112 records). [2025-10-20T22:01:24.621495] [DEBUG] [VALIDATOR ] YALA: Validating primary data. [2025-10-20T22:01:25.008829] [DEBUG] [VALIDATOR ] YALA: All expected trading days are present. [2025-10-20T22:01:25.008903] [DEBUG] [VALIDATOR ] YALA: Primary data validation passed. [2025-10-20T22:01:25.009839] [INFO ] [MAIN ] YALA: Fetched primary data is valid. [2025-10-20T22:01:25.009901] [DEBUG] [LOCAL ] YALA: Saving primary data. [2025-10-20T22:01:25.010892] [DEBUG] [LOCAL ] YALA: Trimmed 47112 records to 36606 (keeping 200 days of data) [2025-10-20T22:01:25.053384] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-03. [2025-10-20T22:01:25.213913] [DEBUG] [LOCAL ] YALA: Primary data saved (36606 records). [2025-10-20T22:01:25.214042] [INFO ] [MAIN ] YALA: Full history saved. [2025-10-20T22:01:25.214123] [INFO ] [MAIN ] Update Progress: 2342/2367 symbols completed. YALA took 28.5s. Estimated remaining: 3m 13s. Rate: 7.8 symbols/min [2025-10-20T22:01:25.394964] [INFO ] [MAIN ] YEXT: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:25.434463] [DEBUG] [LOCAL ] YEXT: Loaded 43806 existing records from primary storage. [2025-10-20T22:01:25.434841] [INFO ] [MAIN ] YEXT: Loaded 43806 existing records. [2025-10-20T22:01:25.652254] [WARN ] [MAIN ] YEXT: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:25.652538] [INFO ] [MAIN ] YEXT: Wiping existing primary data before full refresh. [2025-10-20T22:01:25.652592] [INFO ] [LOCAL ] YEXT: Starting deletion of primary data. [2025-10-20T22:01:25.652800] [DEBUG] [LOCAL ] YEXT: Deleted primary data file. [2025-10-20T22:01:25.652854] [INFO ] [MAIN ] YEXT: Starting full primary refresh. [2025-10-20T22:01:25.858714] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-09. [2025-10-20T22:01:26.028589] [INFO ] [REMOTE ] YEXT: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:26.054941] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-07 (1.80 seconds). [2025-10-20T22:01:26.651420] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-08. [2025-10-20T22:01:26.885042] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-03 (1.83 seconds). [2025-10-20T22:01:27.451365] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-04. [2025-10-20T22:01:27.814463] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-09 (1.96 seconds). [2025-10-20T22:01:28.251380] [INFO ] [REMOTE ] XYZ: Fetching primary data for 2025-10. [2025-10-20T22:01:28.291169] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-08 (1.64 seconds). [2025-10-20T22:01:28.998899] [INFO ] [REMOTE ] XYZ: Incremental fetch complete for 2025-10 (0.75 seconds). [2025-10-20T22:01:29.007133] [INFO ] [REMOTE ] XYZ: Full fetch complete (73633 records). [2025-10-20T22:01:29.007291] [DEBUG] [VALIDATOR ] XYZ: Validating primary data. [2025-10-20T22:01:29.051838] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-09. [2025-10-20T22:01:29.215601] [WARN ] [VALIDATOR ] XYZ: Validation failed - insufficient historical data. Have 189 unique days, need 200 days [2025-10-20T22:01:29.216435] [FATAL] [MAIN ] XYZ: Fetched primary data is invalid. Refusing to save it. [2025-10-20T22:01:29.216626] [WARN ] [MAIN ] XYZ: Deleting existing primary and derivative data due to validation failure. [2025-10-20T22:01:29.216862] [INFO ] [LOCAL ] XYZ: Starting deletion of primary data. [2025-10-20T22:01:29.216971] [DEBUG] [LOCAL ] XYZ: No primary data file found to delete. [2025-10-20T22:01:29.217045] [INFO ] [LOCAL ] XYZ: Starting deletion of derivative data. [2025-10-20T22:01:29.217139] [DEBUG] [LOCAL ] XYZ: No derivative data file found to delete. [2025-10-20T22:01:29.217220] [INFO ] [MAIN ] Update Progress: 2343/2367 symbols completed. XYZ took 38.7s. Estimated remaining: 3m 5s. Rate: 7.8 symbols/min [2025-10-20T22:01:29.389479] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-04 (1.94 seconds). [2025-10-20T22:01:29.473464] [INFO ] [MAIN ] YMM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:29.530207] [DEBUG] [LOCAL ] YMM: Loaded 53373 existing records from primary storage. [2025-10-20T22:01:29.530515] [INFO ] [MAIN ] YMM: Loaded 53373 existing records. [2025-10-20T22:01:29.709984] [WARN ] [MAIN ] YMM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:29.710219] [INFO ] [MAIN ] YMM: Wiping existing primary data before full refresh. [2025-10-20T22:01:29.710264] [INFO ] [LOCAL ] YMM: Starting deletion of primary data. [2025-10-20T22:01:29.710518] [DEBUG] [LOCAL ] YMM: Deleted primary data file. [2025-10-20T22:01:29.710572] [INFO ] [MAIN ] YMM: Starting full primary refresh. [2025-10-20T22:01:29.856556] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-01. [2025-10-20T22:01:30.080354] [INFO ] [REMOTE ] YMM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:30.651438] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-05. [2025-10-20T22:01:30.707747] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-09 (1.66 seconds). [2025-10-20T22:01:31.435797] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-01 (1.58 seconds). [2025-10-20T22:01:31.451331] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-02. [2025-10-20T22:01:32.251410] [INFO ] [REMOTE ] YELP: Fetching primary data for 2025-10. [2025-10-20T22:01:32.395578] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-05 (1.74 seconds). [2025-10-20T22:01:32.490707] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-02 (1.04 seconds). [2025-10-20T22:01:33.051351] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-03. [2025-10-20T22:01:33.344755] [INFO ] [REMOTE ] YELP: Incremental fetch complete for 2025-10 (1.09 seconds). [2025-10-20T22:01:33.351556] [INFO ] [REMOTE ] YELP: Full fetch complete (64801 records). [2025-10-20T22:01:33.351684] [DEBUG] [VALIDATOR ] YELP: Validating primary data. [2025-10-20T22:01:33.731152] [DEBUG] [VALIDATOR ] YELP: All expected trading days are present. [2025-10-20T22:01:33.731246] [DEBUG] [VALIDATOR ] YELP: Primary data validation passed. [2025-10-20T22:01:33.732296] [INFO ] [MAIN ] YELP: Fetched primary data is valid. [2025-10-20T22:01:33.732357] [DEBUG] [LOCAL ] YELP: Saving primary data. [2025-10-20T22:01:33.733531] [DEBUG] [LOCAL ] YELP: Trimmed 64801 records to 45389 (keeping 200 days of data) [2025-10-20T22:01:33.890328] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-01. [2025-10-20T22:01:33.991363] [DEBUG] [LOCAL ] YELP: Primary data saved (45389 records). [2025-10-20T22:01:33.991468] [INFO ] [MAIN ] YELP: Full history saved. [2025-10-20T22:01:33.991546] [INFO ] [MAIN ] Update Progress: 2344/2367 symbols completed. YELP took 31.2s. Estimated remaining: 2m 58s. Rate: 7.8 symbols/min [2025-10-20T22:01:34.160212] [INFO ] [MAIN ] YOU: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:34.204044] [DEBUG] [LOCAL ] YOU: Loaded 50363 existing records from primary storage. [2025-10-20T22:01:34.204341] [INFO ] [MAIN ] YOU: Loaded 50363 existing records. [2025-10-20T22:01:34.388411] [WARN ] [MAIN ] YOU: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:34.388739] [INFO ] [MAIN ] YOU: Wiping existing primary data before full refresh. [2025-10-20T22:01:34.388795] [INFO ] [LOCAL ] YOU: Starting deletion of primary data. [2025-10-20T22:01:34.389013] [DEBUG] [LOCAL ] YOU: Deleted primary data file. [2025-10-20T22:01:34.389070] [INFO ] [MAIN ] YOU: Starting full primary refresh. [2025-10-20T22:01:34.653768] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-06. [2025-10-20T22:01:34.763928] [INFO ] [REMOTE ] YOU: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:34.790563] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T22:01:35.451363] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-01. [2025-10-20T22:01:35.607597] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-01 (1.72 seconds). [2025-10-20T22:01:36.251366] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-04. [2025-10-20T22:01:36.453952] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-06 (1.80 seconds). [2025-10-20T22:01:36.989720] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-01 (1.54 seconds). [2025-10-20T22:01:37.051338] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-02. [2025-10-20T22:01:37.719489] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-04 (1.47 seconds). [2025-10-20T22:01:37.851385] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-05. [2025-10-20T22:01:38.651437] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-07. [2025-10-20T22:01:38.694671] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-02 (1.64 seconds). [2025-10-20T22:01:39.212747] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-05 (1.36 seconds). [2025-10-20T22:01:39.451429] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-06. [2025-10-20T22:01:40.251354] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-03. [2025-10-20T22:01:40.519183] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-07 (1.87 seconds). [2025-10-20T22:01:40.953821] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-06 (1.50 seconds). [2025-10-20T22:01:41.051331] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-07. [2025-10-20T22:01:41.809676] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-03 (1.56 seconds). [2025-10-20T22:01:41.851339] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-04. [2025-10-20T22:01:42.538829] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-07 (1.49 seconds). [2025-10-20T22:01:42.651338] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-08. [2025-10-20T22:01:43.451440] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-02. [2025-10-20T22:01:43.686149] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-04 (1.83 seconds). [2025-10-20T22:01:43.763806] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-08 (1.11 seconds). [2025-10-20T22:01:44.251348] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-08. [2025-10-20T22:01:45.060898] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-09. [2025-10-20T22:01:45.063260] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T22:01:45.851430] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-05. [2025-10-20T22:01:46.089038] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T22:01:46.641327] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-09 (1.58 seconds). [2025-10-20T22:01:46.651352] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-09. [2025-10-20T22:01:47.451361] [INFO ] [REMOTE ] YEXT: Fetching primary data for 2025-10. [2025-10-20T22:01:47.719380] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-05 (1.87 seconds). [2025-10-20T22:01:48.251381] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-03. [2025-10-20T22:01:48.353156] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-09 (1.70 seconds). [2025-10-20T22:01:48.902448] [INFO ] [REMOTE ] YEXT: Incremental fetch complete for 2025-10 (1.45 seconds). [2025-10-20T22:01:48.908930] [INFO ] [REMOTE ] YEXT: Full fetch complete (63209 records). [2025-10-20T22:01:48.909024] [DEBUG] [VALIDATOR ] YEXT: Validating primary data. [2025-10-20T22:01:49.056583] [INFO ] [REMOTE ] YETI: Fetching primary data for 2025-10. [2025-10-20T22:01:49.288471] [DEBUG] [VALIDATOR ] YEXT: All expected trading days are present. [2025-10-20T22:01:49.288541] [DEBUG] [VALIDATOR ] YEXT: Primary data validation passed. [2025-10-20T22:01:49.289491] [INFO ] [MAIN ] YEXT: Fetched primary data is valid. [2025-10-20T22:01:49.289552] [DEBUG] [LOCAL ] YEXT: Saving primary data. [2025-10-20T22:01:49.290723] [DEBUG] [LOCAL ] YEXT: Trimmed 63209 records to 43147 (keeping 200 days of data) [2025-10-20T22:01:49.524904] [DEBUG] [LOCAL ] YEXT: Primary data saved (43147 records). [2025-10-20T22:01:49.525050] [INFO ] [MAIN ] YEXT: Full history saved. [2025-10-20T22:01:49.525366] [INFO ] [MAIN ] Update Progress: 2345/2367 symbols completed. YEXT took 24.3s. Estimated remaining: 2m 50s. Rate: 7.8 symbols/min [2025-10-20T22:01:49.693890] [INFO ] [MAIN ] YPF: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:49.726792] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-03 (1.48 seconds). [2025-10-20T22:01:49.778061] [DEBUG] [LOCAL ] YPF: Loaded 51423 existing records from primary storage. [2025-10-20T22:01:49.778388] [INFO ] [MAIN ] YPF: Loaded 51423 existing records. [2025-10-20T22:01:49.856518] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-06. [2025-10-20T22:01:49.956282] [WARN ] [MAIN ] YPF: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:49.956479] [INFO ] [MAIN ] YPF: Wiping existing primary data before full refresh. [2025-10-20T22:01:49.956528] [INFO ] [LOCAL ] YPF: Starting deletion of primary data. [2025-10-20T22:01:49.956762] [DEBUG] [LOCAL ] YPF: Deleted primary data file. [2025-10-20T22:01:49.956817] [INFO ] [MAIN ] YPF: Starting full primary refresh. [2025-10-20T22:01:50.302526] [INFO ] [REMOTE ] YPF: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:50.651356] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-01. [2025-10-20T22:01:50.687970] [INFO ] [REMOTE ] YETI: Incremental fetch complete for 2025-10 (1.63 seconds). [2025-10-20T22:01:50.695370] [INFO ] [REMOTE ] YETI: Full fetch complete (75515 records). [2025-10-20T22:01:50.695490] [DEBUG] [VALIDATOR ] YETI: Validating primary data. [2025-10-20T22:01:51.079412] [DEBUG] [VALIDATOR ] YETI: All expected trading days are present. [2025-10-20T22:01:51.079491] [DEBUG] [VALIDATOR ] YETI: Primary data validation passed. [2025-10-20T22:01:51.080638] [INFO ] [MAIN ] YETI: Fetched primary data is valid. [2025-10-20T22:01:51.080720] [DEBUG] [LOCAL ] YETI: Saving primary data. [2025-10-20T22:01:51.082047] [DEBUG] [LOCAL ] YETI: Trimmed 75515 records to 52543 (keeping 200 days of data) [2025-10-20T22:01:51.377806] [DEBUG] [LOCAL ] YETI: Primary data saved (52543 records). [2025-10-20T22:01:51.377960] [INFO ] [MAIN ] YETI: Full history saved. [2025-10-20T22:01:51.378052] [INFO ] [MAIN ] Update Progress: 2346/2367 symbols completed. YETI took 32.9s. Estimated remaining: 2m 42s. Rate: 7.8 symbols/min [2025-10-20T22:01:51.452783] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-04. [2025-10-20T22:01:51.550017] [INFO ] [MAIN ] YRD: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:01:51.581163] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-06 (1.72 seconds). [2025-10-20T22:01:51.596192] [DEBUG] [LOCAL ] YRD: Loaded 11657 existing records from primary storage. [2025-10-20T22:01:51.596422] [INFO ] [MAIN ] YRD: Loaded 11657 existing records. [2025-10-20T22:01:51.770055] [WARN ] [MAIN ] YRD: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:01:51.770284] [INFO ] [MAIN ] YRD: Wiping existing primary data before full refresh. [2025-10-20T22:01:51.770330] [INFO ] [LOCAL ] YRD: Starting deletion of primary data. [2025-10-20T22:01:51.770487] [DEBUG] [LOCAL ] YRD: Deleted primary data file. [2025-10-20T22:01:51.770525] [INFO ] [MAIN ] YRD: Starting full primary refresh. [2025-10-20T22:01:52.133705] [INFO ] [REMOTE ] YRD: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:01:52.251348] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-07. [2025-10-20T22:01:52.501380] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-01 (1.85 seconds). [2025-10-20T22:01:53.055128] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-02. [2025-10-20T22:01:53.060845] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-04 (1.61 seconds). [2025-10-20T22:01:53.851362] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-05. [2025-10-20T22:01:54.078257] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-07 (1.83 seconds). [2025-10-20T22:01:54.646619] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-02 (1.59 seconds). [2025-10-20T22:01:54.655549] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-01. [2025-10-20T22:01:55.451366] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-03. [2025-10-20T22:01:55.546017] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-01 (0.89 seconds). [2025-10-20T22:01:55.922746] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-05 (2.07 seconds). [2025-10-20T22:01:56.251440] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-08. [2025-10-20T22:01:57.051351] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-06. [2025-10-20T22:01:57.191550] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-03 (1.74 seconds). [2025-10-20T22:01:57.851366] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-04. [2025-10-20T22:01:58.005779] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-08 (1.75 seconds). [2025-10-20T22:01:58.614188] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-06 (1.56 seconds). [2025-10-20T22:01:58.651341] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-07. [2025-10-20T22:01:59.451389] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-09. [2025-10-20T22:01:59.573844] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-04 (1.72 seconds). [2025-10-20T22:02:00.251407] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-05. [2025-10-20T22:02:00.545166] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-07 (1.89 seconds). [2025-10-20T22:02:01.051345] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-08. [2025-10-20T22:02:01.177781] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T22:02:01.851404] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-02. [2025-10-20T22:02:02.197944] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-05 (1.95 seconds). [2025-10-20T22:02:02.651376] [INFO ] [REMOTE ] YOU: Fetching primary data for 2025-10. [2025-10-20T22:02:02.816732] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-02 (0.97 seconds). [2025-10-20T22:02:02.886488] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T22:02:03.451361] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-09. [2025-10-20T22:02:04.251352] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-03. [2025-10-20T22:02:04.287318] [INFO ] [REMOTE ] YOU: Incremental fetch complete for 2025-10 (1.64 seconds). [2025-10-20T22:02:04.294619] [INFO ] [REMOTE ] YOU: Full fetch complete (72333 records). [2025-10-20T22:02:04.294761] [DEBUG] [VALIDATOR ] YOU: Validating primary data. [2025-10-20T22:02:04.686216] [DEBUG] [VALIDATOR ] YOU: All expected trading days are present. [2025-10-20T22:02:04.686294] [DEBUG] [VALIDATOR ] YOU: Primary data validation passed. [2025-10-20T22:02:04.687383] [INFO ] [MAIN ] YOU: Fetched primary data is valid. [2025-10-20T22:02:04.687445] [DEBUG] [LOCAL ] YOU: Saving primary data. [2025-10-20T22:02:04.688806] [DEBUG] [LOCAL ] YOU: Trimmed 72333 records to 49573 (keeping 200 days of data) [2025-10-20T22:02:05.000619] [DEBUG] [LOCAL ] YOU: Primary data saved (49573 records). [2025-10-20T22:02:05.000762] [INFO ] [MAIN ] YOU: Full history saved. [2025-10-20T22:02:05.000841] [INFO ] [MAIN ] Update Progress: 2347/2367 symbols completed. YOU took 31.0s. Estimated remaining: 2m 34s. Rate: 7.8 symbols/min [2025-10-20T22:02:05.056714] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-06. [2025-10-20T22:02:05.178622] [INFO ] [MAIN ] YSG: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:05.203533] [DEBUG] [LOCAL ] YSG: Loaded 25852 existing records from primary storage. [2025-10-20T22:02:05.203797] [INFO ] [MAIN ] YSG: Loaded 25852 existing records. [2025-10-20T22:02:05.389482] [WARN ] [MAIN ] YSG: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:05.389908] [INFO ] [MAIN ] YSG: Wiping existing primary data before full refresh. [2025-10-20T22:02:05.390080] [INFO ] [LOCAL ] YSG: Starting deletion of primary data. [2025-10-20T22:02:05.400566] [DEBUG] [LOCAL ] YSG: Deleted primary data file. [2025-10-20T22:02:05.418081] [INFO ] [MAIN ] YSG: Starting full primary refresh. [2025-10-20T22:02:05.475806] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-09 (2.02 seconds). [2025-10-20T22:02:05.553003] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-03 (1.30 seconds). [2025-10-20T22:02:05.814599] [INFO ] [REMOTE ] YSG: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:05.851325] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-01. [2025-10-20T22:02:06.651437] [INFO ] [REMOTE ] YMM: Fetching primary data for 2025-10. [2025-10-20T22:02:06.765505] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-06 (1.71 seconds). [2025-10-20T22:02:06.804207] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-01 (0.95 seconds). [2025-10-20T22:02:07.451376] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-02. [2025-10-20T22:02:08.215829] [INFO ] [REMOTE ] YMM: Incremental fetch complete for 2025-10 (1.56 seconds). [2025-10-20T22:02:08.232489] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-02 (0.78 seconds). [2025-10-20T22:02:08.231910] [INFO ] [REMOTE ] YMM: Full fetch complete (76467 records). [2025-10-20T22:02:08.235926] [DEBUG] [VALIDATOR ] YMM: Validating primary data. [2025-10-20T22:02:08.259295] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-04. [2025-10-20T22:02:08.670151] [DEBUG] [VALIDATOR ] YMM: All expected trading days are present. [2025-10-20T22:02:08.670232] [DEBUG] [VALIDATOR ] YMM: Primary data validation passed. [2025-10-20T22:02:08.671374] [INFO ] [MAIN ] YMM: Fetched primary data is valid. [2025-10-20T22:02:08.671437] [DEBUG] [LOCAL ] YMM: Saving primary data. [2025-10-20T22:02:08.672924] [DEBUG] [LOCAL ] YMM: Trimmed 76467 records to 52587 (keeping 200 days of data) [2025-10-20T22:02:09.004632] [DEBUG] [LOCAL ] YMM: Primary data saved (52587 records). [2025-10-20T22:02:09.004809] [INFO ] [MAIN ] YMM: Full history saved. [2025-10-20T22:02:09.004894] [INFO ] [MAIN ] Update Progress: 2348/2367 symbols completed. YMM took 39.8s. Estimated remaining: 2m 27s. Rate: 7.8 symbols/min [2025-10-20T22:02:09.062048] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-03. [2025-10-20T22:02:09.187785] [INFO ] [MAIN ] YUM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:09.240102] [DEBUG] [LOCAL ] YUM: Loaded 51918 existing records from primary storage. [2025-10-20T22:02:09.240436] [INFO ] [MAIN ] YUM: Loaded 51918 existing records. [2025-10-20T22:02:09.424790] [WARN ] [MAIN ] YUM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:09.435142] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-04 (1.18 seconds). [2025-10-20T22:02:09.434801] [INFO ] [MAIN ] YUM: Wiping existing primary data before full refresh. [2025-10-20T22:02:09.438972] [INFO ] [LOCAL ] YUM: Starting deletion of primary data. [2025-10-20T22:02:09.439548] [DEBUG] [LOCAL ] YUM: Deleted primary data file. [2025-10-20T22:02:09.439618] [INFO ] [MAIN ] YUM: Starting full primary refresh. [2025-10-20T22:02:09.812540] [INFO ] [REMOTE ] YUM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:09.851352] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-01. [2025-10-20T22:02:10.228769] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-03 (1.17 seconds). [2025-10-20T22:02:10.651392] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-04. [2025-10-20T22:02:11.451372] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-07. [2025-10-20T22:02:11.604845] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-04 (0.95 seconds). [2025-10-20T22:02:11.663401] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-01 (1.81 seconds). [2025-10-20T22:02:12.251444] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-05. [2025-10-20T22:02:13.051341] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-05. [2025-10-20T22:02:13.073538] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-05 (0.82 seconds). [2025-10-20T22:02:13.168269] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-07 (1.72 seconds). [2025-10-20T22:02:13.624176] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-05 (0.57 seconds). [2025-10-20T22:02:13.851391] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-02. [2025-10-20T22:02:14.651409] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-08. [2025-10-20T22:02:15.451373] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-06. [2025-10-20T22:02:15.507946] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-02 (1.66 seconds). [2025-10-20T22:02:16.254416] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-06. [2025-10-20T22:02:16.255529] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-06 (0.80 seconds). [2025-10-20T22:02:16.328393] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-08 (1.68 seconds). [2025-10-20T22:02:17.051395] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-09. [2025-10-20T22:02:17.620611] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-06 (1.37 seconds). [2025-10-20T22:02:17.851389] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-07. [2025-10-20T22:02:18.651390] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-07. [2025-10-20T22:02:18.692626] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-07 (0.84 seconds). [2025-10-20T22:02:18.780533] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-09 (1.73 seconds). [2025-10-20T22:02:19.451373] [INFO ] [REMOTE ] YPF: Fetching primary data for 2025-10. [2025-10-20T22:02:20.037068] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-07 (1.39 seconds). [2025-10-20T22:02:20.251415] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-03. [2025-10-20T22:02:21.051406] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-08. [2025-10-20T22:02:21.155594] [INFO ] [REMOTE ] YPF: Incremental fetch complete for 2025-10 (1.70 seconds). [2025-10-20T22:02:21.163153] [INFO ] [REMOTE ] YPF: Full fetch complete (73890 records). [2025-10-20T22:02:21.163272] [DEBUG] [VALIDATOR ] YPF: Validating primary data. [2025-10-20T22:02:21.544582] [DEBUG] [VALIDATOR ] YPF: All expected trading days are present. [2025-10-20T22:02:21.544700] [DEBUG] [VALIDATOR ] YPF: Primary data validation passed. [2025-10-20T22:02:21.545851] [INFO ] [MAIN ] YPF: Fetched primary data is valid. [2025-10-20T22:02:21.545910] [DEBUG] [LOCAL ] YPF: Saving primary data. [2025-10-20T22:02:21.547397] [DEBUG] [LOCAL ] YPF: Trimmed 73890 records to 50760 (keeping 200 days of data) [2025-10-20T22:02:21.841351] [DEBUG] [LOCAL ] YPF: Primary data saved (50760 records). [2025-10-20T22:02:21.841480] [INFO ] [MAIN ] YPF: Full history saved. [2025-10-20T22:02:21.841554] [INFO ] [MAIN ] Update Progress: 2349/2367 symbols completed. YPF took 32.3s. Estimated remaining: 2m 19s. Rate: 7.8 symbols/min [2025-10-20T22:02:21.856971] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-08. [2025-10-20T22:02:22.015700] [INFO ] [MAIN ] YUMC: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:22.067503] [DEBUG] [LOCAL ] YUMC: Loaded 53359 existing records from primary storage. [2025-10-20T22:02:22.068005] [INFO ] [MAIN ] YUMC: Loaded 53359 existing records. [2025-10-20T22:02:22.278237] [WARN ] [MAIN ] YUMC: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:22.280565] [INFO ] [MAIN ] YUMC: Wiping existing primary data before full refresh. [2025-10-20T22:02:22.280477] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-03 (2.03 seconds). [2025-10-20T22:02:22.280769] [INFO ] [LOCAL ] YUMC: Starting deletion of primary data. [2025-10-20T22:02:22.292913] [DEBUG] [LOCAL ] YUMC: Deleted primary data file. [2025-10-20T22:02:22.292995] [INFO ] [MAIN ] YUMC: Starting full primary refresh. [2025-10-20T22:02:22.414126] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-08 (1.36 seconds). [2025-10-20T22:02:22.606592] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-08 (0.75 seconds). [2025-10-20T22:02:22.658173] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-04. [2025-10-20T22:02:22.686850] [INFO ] [REMOTE ] YUMC: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:23.451442] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-09. [2025-10-20T22:02:24.251381] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-01. [2025-10-20T22:02:24.266504] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-09 (0.82 seconds). [2025-10-20T22:02:24.477272] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T22:02:25.051377] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-05. [2025-10-20T22:02:25.726000] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-01 (1.47 seconds). [2025-10-20T22:02:25.851330] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-02. [2025-10-20T22:02:26.651441] [INFO ] [REMOTE ] YRD: Fetching primary data for 2025-10. [2025-10-20T22:02:26.786132] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-05 (1.73 seconds). [2025-10-20T22:02:27.218496] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-02 (1.37 seconds). [2025-10-20T22:02:27.451339] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-03. [2025-10-20T22:02:27.526123] [INFO ] [REMOTE ] YRD: Incremental fetch complete for 2025-10 (0.87 seconds). [2025-10-20T22:02:27.528223] [INFO ] [REMOTE ] YRD: Full fetch complete (18977 records). [2025-10-20T22:02:27.528291] [DEBUG] [VALIDATOR ] YRD: Validating primary data. [2025-10-20T22:02:27.888103] [DEBUG] [VALIDATOR ] YRD: All expected trading days are present. [2025-10-20T22:02:27.888178] [DEBUG] [VALIDATOR ] YRD: Primary data validation passed. [2025-10-20T22:02:27.888570] [INFO ] [MAIN ] YRD: Fetched primary data is valid. [2025-10-20T22:02:27.888608] [DEBUG] [LOCAL ] YRD: Saving primary data. [2025-10-20T22:02:27.889151] [DEBUG] [LOCAL ] YRD: Trimmed 18977 records to 11354 (keeping 200 days of data) [2025-10-20T22:02:27.955663] [DEBUG] [LOCAL ] YRD: Primary data saved (11354 records). [2025-10-20T22:02:27.955796] [INFO ] [MAIN ] YRD: Full history saved. [2025-10-20T22:02:27.955872] [INFO ] [MAIN ] Update Progress: 2350/2367 symbols completed. YRD took 36.6s. Estimated remaining: 2m 11s. Rate: 7.8 symbols/min [2025-10-20T22:02:28.124747] [INFO ] [MAIN ] ZBH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:28.173054] [DEBUG] [LOCAL ] ZBH: Loaded 51528 existing records from primary storage. [2025-10-20T22:02:28.173347] [INFO ] [MAIN ] ZBH: Loaded 51528 existing records. [2025-10-20T22:02:28.256612] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-09. [2025-10-20T22:02:28.356507] [WARN ] [MAIN ] ZBH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:28.356725] [INFO ] [MAIN ] ZBH: Wiping existing primary data before full refresh. [2025-10-20T22:02:28.356822] [INFO ] [LOCAL ] ZBH: Starting deletion of primary data. [2025-10-20T22:02:28.357084] [DEBUG] [LOCAL ] ZBH: Deleted primary data file. [2025-10-20T22:02:28.357153] [INFO ] [MAIN ] ZBH: Starting full primary refresh. [2025-10-20T22:02:28.716754] [INFO ] [REMOTE ] ZBH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:29.051347] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-01. [2025-10-20T22:02:29.262255] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-03 (1.81 seconds). [2025-10-20T22:02:29.404273] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-09 (1.15 seconds). [2025-10-20T22:02:29.851376] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-04. [2025-10-20T22:02:30.651368] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-06. [2025-10-20T22:02:30.841820] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-01 (1.79 seconds). [2025-10-20T22:02:31.270598] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-04 (1.42 seconds). [2025-10-20T22:02:31.451356] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-05. [2025-10-20T22:02:32.251441] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-02. [2025-10-20T22:02:32.505167] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-06 (1.85 seconds). [2025-10-20T22:02:33.051399] [INFO ] [REMOTE ] YSG: Fetching primary data for 2025-10. [2025-10-20T22:02:33.288629] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-05 (1.84 seconds). [2025-10-20T22:02:33.862327] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-06. [2025-10-20T22:02:33.865011] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-02 (1.61 seconds). [2025-10-20T22:02:34.223720] [INFO ] [REMOTE ] YSG: Incremental fetch complete for 2025-10 (1.17 seconds). [2025-10-20T22:02:34.227994] [INFO ] [REMOTE ] YSG: Full fetch complete (32515 records). [2025-10-20T22:02:34.228073] [DEBUG] [VALIDATOR ] YSG: Validating primary data. [2025-10-20T22:02:34.611795] [DEBUG] [VALIDATOR ] YSG: All expected trading days are present. [2025-10-20T22:02:34.611869] [DEBUG] [VALIDATOR ] YSG: Primary data validation passed. [2025-10-20T22:02:34.612498] [INFO ] [MAIN ] YSG: Fetched primary data is valid. [2025-10-20T22:02:34.612536] [DEBUG] [LOCAL ] YSG: Saving primary data. [2025-10-20T22:02:34.613249] [DEBUG] [LOCAL ] YSG: Trimmed 32515 records to 25657 (keeping 200 days of data) [2025-10-20T22:02:34.657027] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-07. [2025-10-20T22:02:34.752619] [DEBUG] [LOCAL ] YSG: Primary data saved (25657 records). [2025-10-20T22:02:34.752799] [INFO ] [MAIN ] YSG: Full history saved. [2025-10-20T22:02:34.752879] [INFO ] [MAIN ] Update Progress: 2351/2367 symbols completed. YSG took 29.8s. Estimated remaining: 2m 4s. Rate: 7.8 symbols/min [2025-10-20T22:02:34.917688] [INFO ] [MAIN ] ZEPP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:34.935527] [DEBUG] [LOCAL ] ZEPP: Loaded 19285 existing records from primary storage. [2025-10-20T22:02:34.935771] [INFO ] [MAIN ] ZEPP: Loaded 19285 existing records. [2025-10-20T22:02:35.109170] [WARN ] [MAIN ] ZEPP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:35.109445] [INFO ] [MAIN ] ZEPP: Wiping existing primary data before full refresh. [2025-10-20T22:02:35.109495] [INFO ] [LOCAL ] ZEPP: Starting deletion of primary data. [2025-10-20T22:02:35.109700] [DEBUG] [LOCAL ] ZEPP: Deleted primary data file. [2025-10-20T22:02:35.109754] [INFO ] [MAIN ] ZEPP: Starting full primary refresh. [2025-10-20T22:02:35.455095] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-03. [2025-10-20T22:02:35.459686] [INFO ] [REMOTE ] ZEPP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:35.558821] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-06 (1.70 seconds). [2025-10-20T22:02:36.171285] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-07 (1.51 seconds). [2025-10-20T22:02:36.251416] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-01. [2025-10-20T22:02:36.716782] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-01 (0.47 seconds). [2025-10-20T22:02:37.051433] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-02. [2025-10-20T22:02:37.118603] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-03 (1.66 seconds). [2025-10-20T22:02:37.373542] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-02 (0.32 seconds). [2025-10-20T22:02:37.851426] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-03. [2025-10-20T22:02:38.451936] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-03 (0.60 seconds). [2025-10-20T22:02:38.651413] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-08. [2025-10-20T22:02:39.451369] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-04. [2025-10-20T22:02:39.994961] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-04 (0.54 seconds). [2025-10-20T22:02:40.251381] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-05. [2025-10-20T22:02:40.542956] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-08 (1.89 seconds). [2025-10-20T22:02:40.882747] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T22:02:41.051393] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-04. [2025-10-20T22:02:41.851381] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-07. [2025-10-20T22:02:42.651371] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-09. [2025-10-20T22:02:42.876166] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-04 (1.82 seconds). [2025-10-20T22:02:43.451345] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-05. [2025-10-20T22:02:43.692531] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-07 (1.84 seconds). [2025-10-20T22:02:44.006541] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T22:02:44.251417] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-06. [2025-10-20T22:02:44.856573] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T22:02:45.051405] [INFO ] [REMOTE ] YUM: Fetching primary data for 2025-10. [2025-10-20T22:02:45.343789] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-05 (1.89 seconds). [2025-10-20T22:02:45.754735] [INFO ] [REMOTE ] YUM: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T22:02:45.762426] [INFO ] [REMOTE ] YUM: Full fetch complete (74685 records). [2025-10-20T22:02:45.762573] [DEBUG] [VALIDATOR ] YUM: Validating primary data. [2025-10-20T22:02:45.859557] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-08. [2025-10-20T22:02:46.154058] [DEBUG] [VALIDATOR ] YUM: All expected trading days are present. [2025-10-20T22:02:46.154144] [DEBUG] [VALIDATOR ] YUM: Primary data validation passed. [2025-10-20T22:02:46.155282] [INFO ] [MAIN ] YUM: Fetched primary data is valid. [2025-10-20T22:02:46.155345] [DEBUG] [LOCAL ] YUM: Saving primary data. [2025-10-20T22:02:46.156722] [DEBUG] [LOCAL ] YUM: Trimmed 74685 records to 51173 (keeping 200 days of data) [2025-10-20T22:02:46.475400] [DEBUG] [LOCAL ] YUM: Primary data saved (51173 records). [2025-10-20T22:02:46.475567] [INFO ] [MAIN ] YUM: Full history saved. [2025-10-20T22:02:46.475725] [INFO ] [MAIN ] Update Progress: 2352/2367 symbols completed. YUM took 37.5s. Estimated remaining: 1m 56s. Rate: 7.8 symbols/min [2025-10-20T22:02:46.644490] [INFO ] [MAIN ] ZETA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:46.651400] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-06. [2025-10-20T22:02:46.697534] [DEBUG] [LOCAL ] ZETA: Loaded 54397 existing records from primary storage. [2025-10-20T22:02:46.697894] [INFO ] [MAIN ] ZETA: Loaded 54397 existing records. [2025-10-20T22:02:46.876282] [WARN ] [MAIN ] ZETA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:46.876489] [INFO ] [MAIN ] ZETA: Wiping existing primary data before full refresh. [2025-10-20T22:02:46.876537] [INFO ] [LOCAL ] ZETA: Starting deletion of primary data. [2025-10-20T22:02:46.876820] [DEBUG] [LOCAL ] ZETA: Deleted primary data file. [2025-10-20T22:02:46.876878] [INFO ] [MAIN ] ZETA: Starting full primary refresh. [2025-10-20T22:02:47.235897] [INFO ] [REMOTE ] ZETA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:47.451391] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-01. [2025-10-20T22:02:47.698439] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-08 (1.84 seconds). [2025-10-20T22:02:47.965262] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-06 (1.31 seconds). [2025-10-20T22:02:48.251433] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-07. [2025-10-20T22:02:49.051449] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-09. [2025-10-20T22:02:49.391822] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-01 (1.94 seconds). [2025-10-20T22:02:49.661170] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-07 (1.41 seconds). [2025-10-20T22:02:49.851378] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-08. [2025-10-20T22:02:50.651376] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-07. [2025-10-20T22:02:50.788608] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-09 (1.74 seconds). [2025-10-20T22:02:51.373237] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-08 (1.52 seconds). [2025-10-20T22:02:51.451335] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-09. [2025-10-20T22:02:52.017428] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-07 (1.37 seconds). [2025-10-20T22:02:52.251424] [INFO ] [REMOTE ] YUMC: Fetching primary data for 2025-10. [2025-10-20T22:02:52.859342] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-09 (1.41 seconds). [2025-10-20T22:02:53.051421] [INFO ] [REMOTE ] ZEPP: Fetching primary data for 2025-10. [2025-10-20T22:02:53.851400] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-02. [2025-10-20T22:02:53.983274] [INFO ] [REMOTE ] YUMC: Incremental fetch complete for 2025-10 (1.73 seconds). [2025-10-20T22:02:53.991430] [INFO ] [REMOTE ] YUMC: Full fetch complete (76454 records). [2025-10-20T22:02:53.991593] [DEBUG] [VALIDATOR ] YUMC: Validating primary data. [2025-10-20T22:02:54.321494] [INFO ] [REMOTE ] ZEPP: Incremental fetch complete for 2025-10 (1.27 seconds). [2025-10-20T22:02:54.405169] [INFO ] [REMOTE ] ZEPP: Full fetch complete (21005 records). [2025-10-20T22:02:54.409101] [DEBUG] [VALIDATOR ] YUMC: All expected trading days are present. [2025-10-20T22:02:54.409012] [DEBUG] [VALIDATOR ] ZEPP: Validating primary data. [2025-10-20T22:02:54.409384] [DEBUG] [VALIDATOR ] YUMC: Primary data validation passed. [2025-10-20T22:02:54.410693] [INFO ] [MAIN ] YUMC: Fetched primary data is valid. [2025-10-20T22:02:54.410773] [DEBUG] [LOCAL ] YUMC: Saving primary data. [2025-10-20T22:02:54.515885] [DEBUG] [LOCAL ] YUMC: Trimmed 76454 records to 52595 (keeping 200 days of data) [2025-10-20T22:02:54.692830] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-08. [2025-10-20T22:02:55.052281] [DEBUG] [VALIDATOR ] ZEPP: All expected trading days are present. [2025-10-20T22:02:55.056074] [DEBUG] [VALIDATOR ] ZEPP: Primary data validation passed. [2025-10-20T22:02:55.056739] [INFO ] [MAIN ] ZEPP: Fetched primary data is valid. [2025-10-20T22:02:55.092239] [DEBUG] [LOCAL ] ZEPP: Saving primary data. [2025-10-20T22:02:55.124753] [DEBUG] [LOCAL ] ZEPP: Trimmed 21005 records to 19430 (keeping 200 days of data) [2025-10-20T22:02:55.134340] [DEBUG] [LOCAL ] YUMC: Primary data saved (52595 records). [2025-10-20T22:02:55.135046] [INFO ] [MAIN ] YUMC: Full history saved. [2025-10-20T22:02:55.135143] [INFO ] [MAIN ] Update Progress: 2353/2367 symbols completed. YUMC took 33.3s. Estimated remaining: 1m 48s. Rate: 7.8 symbols/min [2025-10-20T22:02:55.435401] [INFO ] [MAIN ] ZGN: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:55.460593] [DEBUG] [LOCAL ] ZEPP: Primary data saved (19430 records). [2025-10-20T22:02:55.460855] [INFO ] [MAIN ] ZEPP: Full history saved. [2025-10-20T22:02:55.461009] [INFO ] [MAIN ] Update Progress: 2354/2367 symbols completed. ZEPP took 20.7s. Estimated remaining: 1m 40s. Rate: 7.8 symbols/min [2025-10-20T22:02:55.687676] [DEBUG] [LOCAL ] ZGN: Loaded 41539 existing records from primary storage. [2025-10-20T22:02:55.712699] [INFO ] [MAIN ] ZH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:02:55.713856] [INFO ] [MAIN ] ZGN: Loaded 41539 existing records. [2025-10-20T22:02:56.009549] [DEBUG] [LOCAL ] ZH: Loaded 25565 existing records from primary storage. [2025-10-20T22:02:56.008012] [WARN ] [MAIN ] ZGN: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:56.010303] [INFO ] [MAIN ] ZH: Loaded 25565 existing records. [2025-10-20T22:02:56.010406] [INFO ] [MAIN ] ZGN: Wiping existing primary data before full refresh. [2025-10-20T22:02:56.010595] [INFO ] [LOCAL ] ZGN: Starting deletion of primary data. [2025-10-20T22:02:56.055893] [DEBUG] [LOCAL ] ZGN: Deleted primary data file. [2025-10-20T22:02:56.098774] [INFO ] [MAIN ] ZGN: Starting full primary refresh. [2025-10-20T22:02:56.300357] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-02 (2.45 seconds). [2025-10-20T22:02:56.335218] [WARN ] [MAIN ] ZH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:02:56.383097] [INFO ] [MAIN ] ZH: Wiping existing primary data before full refresh. [2025-10-20T22:02:56.388630] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-03. [2025-10-20T22:02:56.399140] [INFO ] [LOCAL ] ZH: Starting deletion of primary data. [2025-10-20T22:02:56.413259] [DEBUG] [LOCAL ] ZH: Deleted primary data file. [2025-10-20T22:02:56.424378] [INFO ] [MAIN ] ZH: Starting full primary refresh. [2025-10-20T22:02:56.919091] [INFO ] [REMOTE ] ZGN: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:56.958078] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-01. [2025-10-20T22:02:57.079988] [INFO ] [REMOTE ] ZH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:02:57.082054] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-08 (2.39 seconds). [2025-10-20T22:02:57.080756] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-01. [2025-10-20T22:02:57.851371] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-09. [2025-10-20T22:02:58.077131] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-01 (1.00 seconds). [2025-10-20T22:02:58.541767] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-03 (2.15 seconds). [2025-10-20T22:02:58.578855] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-01 (1.62 seconds). [2025-10-20T22:02:58.654274] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-04. [2025-10-20T22:02:58.660262] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-09 (0.81 seconds). [2025-10-20T22:02:59.451371] [INFO ] [REMOTE ] ZBH: Fetching primary data for 2025-10. [2025-10-20T22:03:00.142859] [INFO ] [REMOTE ] ZBH: Incremental fetch complete for 2025-10 (0.69 seconds). [2025-10-20T22:03:00.150065] [INFO ] [REMOTE ] ZBH: Full fetch complete (73952 records). [2025-10-20T22:03:00.150231] [DEBUG] [VALIDATOR ] ZBH: Validating primary data. [2025-10-20T22:03:00.254534] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-02. [2025-10-20T22:03:00.562619] [DEBUG] [VALIDATOR ] ZBH: All expected trading days are present. [2025-10-20T22:03:00.562749] [DEBUG] [VALIDATOR ] ZBH: Primary data validation passed. [2025-10-20T22:03:00.563852] [INFO ] [MAIN ] ZBH: Fetched primary data is valid. [2025-10-20T22:03:00.563928] [DEBUG] [LOCAL ] ZBH: Saving primary data. [2025-10-20T22:03:00.565477] [DEBUG] [LOCAL ] ZBH: Trimmed 73952 records to 50791 (keeping 200 days of data) [2025-10-20T22:03:00.701184] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-04 (2.05 seconds). [2025-10-20T22:03:00.929164] [DEBUG] [LOCAL ] ZBH: Primary data saved (50791 records). [2025-10-20T22:03:00.929286] [INFO ] [MAIN ] ZBH: Full history saved. [2025-10-20T22:03:00.929363] [INFO ] [MAIN ] Update Progress: 2355/2367 symbols completed. ZBH took 33.0s. Estimated remaining: 1m 33s. Rate: 7.8 symbols/min [2025-10-20T22:03:01.056634] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-02. [2025-10-20T22:03:01.107351] [INFO ] [MAIN ] ZIM: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:01.161307] [DEBUG] [LOCAL ] ZIM: Loaded 54263 existing records from primary storage. [2025-10-20T22:03:01.161595] [INFO ] [MAIN ] ZIM: Loaded 54263 existing records. [2025-10-20T22:03:01.337485] [WARN ] [MAIN ] ZIM: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:01.337695] [INFO ] [MAIN ] ZIM: Wiping existing primary data before full refresh. [2025-10-20T22:03:01.337751] [INFO ] [LOCAL ] ZIM: Starting deletion of primary data. [2025-10-20T22:03:01.338009] [DEBUG] [LOCAL ] ZIM: Deleted primary data file. [2025-10-20T22:03:01.338064] [INFO ] [MAIN ] ZIM: Starting full primary refresh. [2025-10-20T22:03:01.703975] [INFO ] [REMOTE ] ZIM: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:01.805705] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T22:03:01.851337] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-03. [2025-10-20T22:03:02.475716] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-02 (1.42 seconds). [2025-10-20T22:03:02.651365] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-03. [2025-10-20T22:03:03.317609] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-03 (1.47 seconds). [2025-10-20T22:03:03.451371] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-05. [2025-10-20T22:03:04.185573] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-03 (1.53 seconds). [2025-10-20T22:03:04.251393] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-01. [2025-10-20T22:03:05.051388] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-04. [2025-10-20T22:03:05.567818] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-05 (2.12 seconds). [2025-10-20T22:03:05.851382] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-06. [2025-10-20T22:03:06.027433] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-01 (1.78 seconds). [2025-10-20T22:03:06.205151] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-04 (1.15 seconds). [2025-10-20T22:03:06.651381] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-05. [2025-10-20T22:03:07.451443] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-04. [2025-10-20T22:03:07.830112] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-06 (1.98 seconds). [2025-10-20T22:03:08.188221] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-05 (1.54 seconds). [2025-10-20T22:03:08.251367] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-06. [2025-10-20T22:03:08.764136] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-04 (1.31 seconds). [2025-10-20T22:03:09.051367] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-05. [2025-10-20T22:03:09.667466] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-06 (1.42 seconds). [2025-10-20T22:03:09.851356] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-07. [2025-10-20T22:03:09.914038] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-05 (0.86 seconds). [2025-10-20T22:03:10.651363] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-07. [2025-10-20T22:03:11.351736] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-07 (1.50 seconds). [2025-10-20T22:03:11.451407] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-02. [2025-10-20T22:03:12.251391] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-08. [2025-10-20T22:03:12.841551] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-07 (2.19 seconds). [2025-10-20T22:03:13.051403] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-08. [2025-10-20T22:03:13.325960] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-08 (1.07 seconds). [2025-10-20T22:03:13.504026] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-02 (2.05 seconds). [2025-10-20T22:03:13.851431] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-06. [2025-10-20T22:03:14.651399] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-03. [2025-10-20T22:03:14.920491] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-06 (1.07 seconds). [2025-10-20T22:03:15.429387] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-08 (2.38 seconds). [2025-10-20T22:03:15.451341] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-09. [2025-10-20T22:03:16.251360] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-07. [2025-10-20T22:03:16.829527] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-03 (2.18 seconds). [2025-10-20T22:03:17.051390] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-04. [2025-10-20T22:03:17.358571] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-09 (1.91 seconds). [2025-10-20T22:03:17.517376] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-07 (1.27 seconds). [2025-10-20T22:03:17.851367] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-08. [2025-10-20T22:03:18.651425] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-09. [2025-10-20T22:03:19.137385] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-08 (1.29 seconds). [2025-10-20T22:03:19.175499] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-04 (2.12 seconds). [2025-10-20T22:03:19.451430] [INFO ] [REMOTE ] ZETA: Fetching primary data for 2025-10. [2025-10-20T22:03:20.021555] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-09 (1.37 seconds). [2025-10-20T22:03:20.251405] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-05. [2025-10-20T22:03:21.051446] [INFO ] [REMOTE ] ZGN: Fetching primary data for 2025-10. [2025-10-20T22:03:21.808965] [INFO ] [REMOTE ] ZETA: Incremental fetch complete for 2025-10 (2.36 seconds). [2025-10-20T22:03:21.818770] [INFO ] [REMOTE ] ZETA: Full fetch complete (77733 records). [2025-10-20T22:03:21.818902] [DEBUG] [VALIDATOR ] ZETA: Validating primary data. [2025-10-20T22:03:21.862361] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-09. [2025-10-20T22:03:22.224501] [DEBUG] [VALIDATOR ] ZETA: All expected trading days are present. [2025-10-20T22:03:22.224598] [DEBUG] [VALIDATOR ] ZETA: Primary data validation passed. [2025-10-20T22:03:22.225781] [INFO ] [MAIN ] ZETA: Fetched primary data is valid. [2025-10-20T22:03:22.225858] [DEBUG] [LOCAL ] ZETA: Saving primary data. [2025-10-20T22:03:22.244435] [INFO ] [REMOTE ] ZGN: Incremental fetch complete for 2025-10 (1.19 seconds). [2025-10-20T22:03:22.250604] [DEBUG] [LOCAL ] ZETA: Trimmed 77733 records to 53618 (keeping 200 days of data) [2025-10-20T22:03:22.251366] [INFO ] [REMOTE ] ZGN: Full fetch complete (58720 records). [2025-10-20T22:03:22.251615] [DEBUG] [VALIDATOR ] ZGN: Validating primary data. [2025-10-20T22:03:22.886725] [DEBUG] [VALIDATOR ] ZGN: All expected trading days are present. [2025-10-20T22:03:22.887468] [DEBUG] [VALIDATOR ] ZGN: Primary data validation passed. [2025-10-20T22:03:22.981077] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-09 (1.12 seconds). [2025-10-20T22:03:22.959619] [INFO ] [MAIN ] ZGN: Fetched primary data is valid. [2025-10-20T22:03:22.981277] [DEBUG] [LOCAL ] ZGN: Saving primary data. [2025-10-20T22:03:23.023080] [INFO ] [REMOTE ] ZH: Fetching primary data for 2025-10. [2025-10-20T22:03:23.031839] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-05 (2.78 seconds). [2025-10-20T22:03:23.030436] [DEBUG] [LOCAL ] ZGN: Trimmed 58720 records to 40935 (keeping 200 days of data) [2025-10-20T22:03:23.139556] [DEBUG] [LOCAL ] ZETA: Primary data saved (53618 records). [2025-10-20T22:03:23.140838] [INFO ] [MAIN ] ZETA: Full history saved. [2025-10-20T22:03:23.141221] [INFO ] [MAIN ] Update Progress: 2356/2367 symbols completed. ZETA took 36.7s. Estimated remaining: 1m 25s. Rate: 7.8 symbols/min [2025-10-20T22:03:23.392390] [INFO ] [MAIN ] ZIP: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:23.454434] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-06. [2025-10-20T22:03:23.481380] [DEBUG] [LOCAL ] ZGN: Primary data saved (40935 records). [2025-10-20T22:03:23.481495] [INFO ] [MAIN ] ZGN: Full history saved. [2025-10-20T22:03:23.481567] [INFO ] [MAIN ] Update Progress: 2357/2367 symbols completed. ZGN took 28.3s. Estimated remaining: 1m 17s. Rate: 7.8 symbols/min [2025-10-20T22:03:23.496325] [DEBUG] [LOCAL ] ZIP: Loaded 44273 existing records from primary storage. [2025-10-20T22:03:23.496583] [INFO ] [MAIN ] ZIP: Loaded 44273 existing records. [2025-10-20T22:03:23.677235] [WARN ] [MAIN ] ZIP: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:23.677445] [INFO ] [MAIN ] ZIP: Wiping existing primary data before full refresh. [2025-10-20T22:03:23.677491] [INFO ] [LOCAL ] ZIP: Starting deletion of primary data. [2025-10-20T22:03:23.677750] [DEBUG] [LOCAL ] ZIP: Deleted primary data file. [2025-10-20T22:03:23.677806] [INFO ] [MAIN ] ZIP: Starting full primary refresh. [2025-10-20T22:03:23.978244] [INFO ] [REMOTE ] ZH: Incremental fetch complete for 2025-10 (0.96 seconds). [2025-10-20T22:03:24.002453] [INFO ] [REMOTE ] ZH: Full fetch complete (39875 records). [2025-10-20T22:03:24.026873] [DEBUG] [VALIDATOR ] ZH: Validating primary data. [2025-10-20T22:03:24.132622] [INFO ] [REMOTE ] ZIP: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:24.282483] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-01. [2025-10-20T22:03:24.440831] [DEBUG] [VALIDATOR ] ZH: All expected trading days are present. [2025-10-20T22:03:24.440901] [DEBUG] [VALIDATOR ] ZH: Primary data validation passed. [2025-10-20T22:03:24.441673] [INFO ] [MAIN ] ZH: Fetched primary data is valid. [2025-10-20T22:03:24.441733] [DEBUG] [LOCAL ] ZH: Saving primary data. [2025-10-20T22:03:24.442501] [DEBUG] [LOCAL ] ZH: Trimmed 39875 records to 24968 (keeping 200 days of data) [2025-10-20T22:03:24.577320] [DEBUG] [LOCAL ] ZH: Primary data saved (24968 records). [2025-10-20T22:03:24.577440] [INFO ] [MAIN ] ZH: Full history saved. [2025-10-20T22:03:24.577516] [INFO ] [MAIN ] Update Progress: 2358/2367 symbols completed. ZH took 29.1s. Estimated remaining: 1m 9s. Rate: 7.8 symbols/min [2025-10-20T22:03:25.625916] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-01 (1.34 seconds). [2025-10-20T22:03:25.632198] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-02. [2025-10-20T22:03:25.768229] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-06 (2.31 seconds). [2025-10-20T22:03:25.851343] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-07. [2025-10-20T22:03:26.770768] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-02 (1.14 seconds). [2025-10-20T22:03:26.777397] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-03. [2025-10-20T22:03:27.891253] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-07 (2.04 seconds). [2025-10-20T22:03:27.902848] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-08. [2025-10-20T22:03:28.276194] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-03 (1.50 seconds). [2025-10-20T22:03:28.286017] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-04. [2025-10-20T22:03:29.855429] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-04 (1.57 seconds). [2025-10-20T22:03:29.865542] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-05. [2025-10-20T22:03:30.219934] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-08 (2.32 seconds). [2025-10-20T22:03:30.231270] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-09. [2025-10-20T22:03:31.414996] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-05 (1.55 seconds). [2025-10-20T22:03:31.426515] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-06. [2025-10-20T22:03:32.165546] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-09 (1.93 seconds). [2025-10-20T22:03:32.178107] [INFO ] [REMOTE ] ZIM: Fetching primary data for 2025-10. [2025-10-20T22:03:32.948284] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T22:03:32.957501] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-07. [2025-10-20T22:03:33.840131] [INFO ] [REMOTE ] ZIM: Incremental fetch complete for 2025-10 (1.66 seconds). [2025-10-20T22:03:33.848166] [INFO ] [REMOTE ] ZIM: Full fetch complete (77596 records). [2025-10-20T22:03:33.848280] [DEBUG] [VALIDATOR ] ZIM: Validating primary data. [2025-10-20T22:03:34.243863] [DEBUG] [VALIDATOR ] ZIM: All expected trading days are present. [2025-10-20T22:03:34.244020] [DEBUG] [VALIDATOR ] ZIM: Primary data validation passed. [2025-10-20T22:03:34.245329] [INFO ] [MAIN ] ZIM: Fetched primary data is valid. [2025-10-20T22:03:34.245459] [DEBUG] [LOCAL ] ZIM: Saving primary data. [2025-10-20T22:03:34.269143] [DEBUG] [LOCAL ] ZIM: Trimmed 77596 records to 53475 (keeping 200 days of data) [2025-10-20T22:03:34.272258] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-07 (1.31 seconds). [2025-10-20T22:03:34.369604] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-08. [2025-10-20T22:03:34.606288] [DEBUG] [LOCAL ] ZIM: Primary data saved (53475 records). [2025-10-20T22:03:34.606483] [INFO ] [MAIN ] ZIM: Full history saved. [2025-10-20T22:03:34.606590] [INFO ] [MAIN ] Update Progress: 2359/2367 symbols completed. ZIM took 33.7s. Estimated remaining: 1m 2s. Rate: 7.8 symbols/min [2025-10-20T22:03:36.067398] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-08 (1.70 seconds). [2025-10-20T22:03:36.077320] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-09. [2025-10-20T22:03:37.257401] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-09 (1.18 seconds). [2025-10-20T22:03:37.266687] [INFO ] [REMOTE ] ZIP: Fetching primary data for 2025-10. [2025-10-20T22:03:38.784983] [INFO ] [REMOTE ] ZIP: Incremental fetch complete for 2025-10 (1.52 seconds). [2025-10-20T22:03:38.791379] [INFO ] [REMOTE ] ZIP: Full fetch complete (60632 records). [2025-10-20T22:03:38.791491] [DEBUG] [VALIDATOR ] ZIP: Validating primary data. [2025-10-20T22:03:39.178243] [DEBUG] [VALIDATOR ] ZIP: All expected trading days are present. [2025-10-20T22:03:39.178322] [DEBUG] [VALIDATOR ] ZIP: Primary data validation passed. [2025-10-20T22:03:39.179353] [INFO ] [MAIN ] ZIP: Fetched primary data is valid. [2025-10-20T22:03:39.179416] [DEBUG] [LOCAL ] ZIP: Saving primary data. [2025-10-20T22:03:39.180561] [DEBUG] [LOCAL ] ZIP: Trimmed 60632 records to 43557 (keeping 200 days of data) [2025-10-20T22:03:39.426795] [DEBUG] [LOCAL ] ZIP: Primary data saved (43557 records). [2025-10-20T22:03:39.426974] [INFO ] [MAIN ] ZIP: Full history saved. [2025-10-20T22:03:39.427092] [INFO ] [MAIN ] Update Progress: 2360/2367 symbols completed. ZIP took 16.3s. Estimated remaining: 54.0s. Rate: 7.8 symbols/min [2025-10-20T22:03:40.083002] [INFO ] [MAIN ] ZTR: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:40.087699] [INFO ] [MAIN ] ZK: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:40.105313] [INFO ] [MAIN ] ZTO: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:40.177099] [INFO ] [MAIN ] ZKH: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:40.192971] [DEBUG] [LOCAL ] ZTR: Loaded 16868 existing records from primary storage. [2025-10-20T22:03:40.193357] [INFO ] [MAIN ] ZTR: Loaded 16868 existing records. [2025-10-20T22:03:40.227469] [DEBUG] [LOCAL ] ZKH: Loaded 7505 existing records from primary storage. [2025-10-20T22:03:40.238775] [INFO ] [MAIN ] ZKH: Loaded 7505 existing records. [2025-10-20T22:03:40.417321] [DEBUG] [LOCAL ] ZK: Loaded 39593 existing records from primary storage. [2025-10-20T22:03:40.447296] [INFO ] [MAIN ] ZK: Loaded 39593 existing records. [2025-10-20T22:03:40.718103] [WARN ] [MAIN ] ZTR: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:40.715007] [DEBUG] [LOCAL ] ZTO: Loaded 50253 existing records from primary storage. [2025-10-20T22:03:40.718533] [INFO ] [MAIN ] ZTO: Loaded 50253 existing records. [2025-10-20T22:03:40.718686] [INFO ] [MAIN ] ZTR: Wiping existing primary data before full refresh. [2025-10-20T22:03:40.732486] [WARN ] [MAIN ] ZKH: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:40.732610] [INFO ] [LOCAL ] ZTR: Starting deletion of primary data. [2025-10-20T22:03:40.756455] [DEBUG] [LOCAL ] ZTR: Deleted primary data file. [2025-10-20T22:03:40.756920] [INFO ] [MAIN ] ZTR: Starting full primary refresh. [2025-10-20T22:03:40.772428] [INFO ] [MAIN ] ZKH: Wiping existing primary data before full refresh. [2025-10-20T22:03:40.844613] [INFO ] [LOCAL ] ZKH: Starting deletion of primary data. [2025-10-20T22:03:40.922318] [WARN ] [MAIN ] ZK: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:40.919186] [DEBUG] [LOCAL ] ZKH: Deleted primary data file. [2025-10-20T22:03:40.970674] [INFO ] [MAIN ] ZKH: Starting full primary refresh. [2025-10-20T22:03:40.965415] [INFO ] [MAIN ] ZK: Wiping existing primary data before full refresh. [2025-10-20T22:03:41.002742] [INFO ] [LOCAL ] ZK: Starting deletion of primary data. [2025-10-20T22:03:41.052134] [DEBUG] [LOCAL ] ZK: Deleted primary data file. [2025-10-20T22:03:41.112691] [INFO ] [MAIN ] ZK: Starting full primary refresh. [2025-10-20T22:03:41.467696] [WARN ] [MAIN ] ZTO: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:41.511136] [INFO ] [MAIN ] ZTO: Wiping existing primary data before full refresh. [2025-10-20T22:03:41.522121] [INFO ] [LOCAL ] ZTO: Starting deletion of primary data. [2025-10-20T22:03:41.622896] [DEBUG] [LOCAL ] ZTO: Deleted primary data file. [2025-10-20T22:03:41.656324] [INFO ] [MAIN ] ZTO: Starting full primary refresh. [2025-10-20T22:03:42.093519] [INFO ] [REMOTE ] ZTR: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:42.190771] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-01. [2025-10-20T22:03:42.279594] [INFO ] [REMOTE ] ZKH: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:42.328188] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-01. [2025-10-20T22:03:42.375514] [INFO ] [REMOTE ] ZK: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:42.413137] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-01. [2025-10-20T22:03:42.504175] [INFO ] [REMOTE ] ZTO: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:42.504467] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-01. [2025-10-20T22:03:43.111407] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-01 (0.78 seconds). [2025-10-20T22:03:43.114541] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-02. [2025-10-20T22:03:43.521582] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T22:03:43.527253] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-02. [2025-10-20T22:03:43.729971] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-01 (1.32 seconds). [2025-10-20T22:03:43.737518] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-02. [2025-10-20T22:03:43.748371] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-02 (0.63 seconds). [2025-10-20T22:03:43.752617] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-03. [2025-10-20T22:03:43.831054] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-01 (1.33 seconds). [2025-10-20T22:03:43.841099] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-02. [2025-10-20T22:03:44.226683] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-03 (0.47 seconds). [2025-10-20T22:03:44.229449] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-04. [2025-10-20T22:03:44.616865] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-02 (1.09 seconds). [2025-10-20T22:03:44.622101] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-03. [2025-10-20T22:03:44.829768] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-04 (0.60 seconds). [2025-10-20T22:03:44.831915] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-05. [2025-10-20T22:03:45.203197] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-02 (1.36 seconds). [2025-10-20T22:03:45.213598] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-03. [2025-10-20T22:03:45.259756] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-02 (1.52 seconds). [2025-10-20T22:03:45.319938] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-03 (0.70 seconds). [2025-10-20T22:03:45.457861] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-05 (0.63 seconds). [2025-10-20T22:03:45.851413] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-04. [2025-10-20T22:03:46.651367] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-03. [2025-10-20T22:03:46.994545] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-03 (1.78 seconds). [2025-10-20T22:03:47.039677] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-04 (1.19 seconds). [2025-10-20T22:03:47.451383] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-06. [2025-10-20T22:03:48.065950] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-06 (0.61 seconds). [2025-10-20T22:03:48.251377] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-07. [2025-10-20T22:03:48.349847] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-03 (1.70 seconds). [2025-10-20T22:03:48.865277] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-07 (0.61 seconds). [2025-10-20T22:03:49.051445] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-08. [2025-10-20T22:03:49.851389] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-05. [2025-10-20T22:03:49.894286] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-08 (0.84 seconds). [2025-10-20T22:03:50.568441] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-05 (0.72 seconds). [2025-10-20T22:03:50.651368] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-06. [2025-10-20T22:03:51.451367] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-04. [2025-10-20T22:03:51.627527] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-06 (0.98 seconds). [2025-10-20T22:03:52.251374] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-09. [2025-10-20T22:03:52.977284] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-04 (1.53 seconds). [2025-10-20T22:03:53.011968] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-09 (0.76 seconds). [2025-10-20T22:03:53.051344] [INFO ] [REMOTE ] ZKH: Fetching primary data for 2025-10. [2025-10-20T22:03:53.553546] [INFO ] [REMOTE ] ZKH: Incremental fetch complete for 2025-10 (0.50 seconds). [2025-10-20T22:03:53.554779] [INFO ] [REMOTE ] ZKH: Full fetch complete (11831 records). [2025-10-20T22:03:53.554845] [DEBUG] [VALIDATOR ] ZKH: Validating primary data. [2025-10-20T22:03:53.863033] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-04. [2025-10-20T22:03:53.904030] [DEBUG] [VALIDATOR ] ZKH: All expected trading days are present. [2025-10-20T22:03:53.904268] [DEBUG] [VALIDATOR ] ZKH: Primary data validation passed. [2025-10-20T22:03:53.904462] [INFO ] [MAIN ] ZKH: Fetched primary data is valid. [2025-10-20T22:03:53.904503] [DEBUG] [LOCAL ] ZKH: Saving primary data. [2025-10-20T22:03:53.904971] [DEBUG] [LOCAL ] ZKH: Trimmed 11831 records to 7399 (keeping 200 days of data) [2025-10-20T22:03:53.943096] [DEBUG] [LOCAL ] ZKH: Primary data saved (7399 records). [2025-10-20T22:03:53.943178] [INFO ] [MAIN ] ZKH: Full history saved. [2025-10-20T22:03:53.943253] [INFO ] [MAIN ] Update Progress: 2361/2367 symbols completed. ZKH took 14.5s. Estimated remaining: 46.3s. Rate: 7.8 symbols/min [2025-10-20T22:03:54.104987] [INFO ] [MAIN ] ZTS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:03:54.155789] [DEBUG] [LOCAL ] ZTS: Loaded 53916 existing records from primary storage. [2025-10-20T22:03:54.156105] [INFO ] [MAIN ] ZTS: Loaded 53916 existing records. [2025-10-20T22:03:54.332230] [WARN ] [MAIN ] ZTS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:03:54.332422] [INFO ] [MAIN ] ZTS: Wiping existing primary data before full refresh. [2025-10-20T22:03:54.332470] [INFO ] [LOCAL ] ZTS: Starting deletion of primary data. [2025-10-20T22:03:54.332690] [DEBUG] [LOCAL ] ZTS: Deleted primary data file. [2025-10-20T22:03:54.332745] [INFO ] [MAIN ] ZTS: Starting full primary refresh. [2025-10-20T22:03:54.670710] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-07. [2025-10-20T22:03:54.675128] [INFO ] [REMOTE ] ZTS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:03:55.451412] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-05. [2025-10-20T22:03:55.631004] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-04 (1.77 seconds). [2025-10-20T22:03:55.725530] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-07 (1.05 seconds). [2025-10-20T22:03:56.251416] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-08. [2025-10-20T22:03:57.051415] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-05. [2025-10-20T22:03:57.167736] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-05 (1.72 seconds). [2025-10-20T22:03:57.221597] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-08 (0.97 seconds). [2025-10-20T22:03:57.851375] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-01. [2025-10-20T22:03:58.651372] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-09. [2025-10-20T22:03:58.882140] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-05 (1.83 seconds). [2025-10-20T22:03:59.451420] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-06. [2025-10-20T22:03:59.597616] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-01 (1.75 seconds). [2025-10-20T22:03:59.620373] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-09 (0.97 seconds). [2025-10-20T22:04:00.251417] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-02. [2025-10-20T22:04:01.051364] [INFO ] [REMOTE ] ZTR: Fetching primary data for 2025-10. [2025-10-20T22:04:01.585541] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-06 (2.13 seconds). [2025-10-20T22:04:01.851351] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-06. [2025-10-20T22:04:01.984510] [INFO ] [REMOTE ] ZTR: Incremental fetch complete for 2025-10 (0.93 seconds). [2025-10-20T22:04:01.987843] [INFO ] [REMOTE ] ZTR: Full fetch complete (26789 records). [2025-10-20T22:04:01.988001] [DEBUG] [VALIDATOR ] ZTR: Validating primary data. [2025-10-20T22:04:02.368853] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-02 (2.12 seconds). [2025-10-20T22:04:02.389178] [DEBUG] [VALIDATOR ] ZTR: All expected trading days are present. [2025-10-20T22:04:02.389515] [DEBUG] [VALIDATOR ] ZTR: Primary data validation passed. [2025-10-20T22:04:02.390126] [INFO ] [MAIN ] ZTR: Fetched primary data is valid. [2025-10-20T22:04:02.390204] [DEBUG] [LOCAL ] ZTR: Saving primary data. [2025-10-20T22:04:02.391068] [DEBUG] [LOCAL ] ZTR: Trimmed 26789 records to 16614 (keeping 200 days of data) [2025-10-20T22:04:02.486408] [DEBUG] [LOCAL ] ZTR: Primary data saved (16614 records). [2025-10-20T22:04:02.486533] [INFO ] [MAIN ] ZTR: Full history saved. [2025-10-20T22:04:02.486606] [INFO ] [MAIN ] Update Progress: 2362/2367 symbols completed. ZTR took 23.1s. Estimated remaining: 38.6s. Rate: 7.8 symbols/min [2025-10-20T22:04:02.651976] [INFO ] [MAIN ] ZVIA: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:04:02.652066] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-07. [2025-10-20T22:04:02.684861] [DEBUG] [LOCAL ] ZVIA: Loaded 36311 existing records from primary storage. [2025-10-20T22:04:02.685123] [INFO ] [MAIN ] ZVIA: Loaded 36311 existing records. [2025-10-20T22:04:02.859411] [WARN ] [MAIN ] ZVIA: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:04:02.859615] [INFO ] [MAIN ] ZVIA: Wiping existing primary data before full refresh. [2025-10-20T22:04:02.859689] [INFO ] [LOCAL ] ZVIA: Starting deletion of primary data. [2025-10-20T22:04:02.859904] [DEBUG] [LOCAL ] ZVIA: Deleted primary data file. [2025-10-20T22:04:02.859947] [INFO ] [MAIN ] ZVIA: Starting full primary refresh. [2025-10-20T22:04:03.213268] [INFO ] [REMOTE ] ZVIA: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:04:03.361804] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-06 (1.51 seconds). [2025-10-20T22:04:03.451360] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-01. [2025-10-20T22:04:04.114670] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T22:04:04.251369] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-03. [2025-10-20T22:04:04.915495] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-01 (1.46 seconds). [2025-10-20T22:04:05.051358] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-08. [2025-10-20T22:04:05.851424] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-02. [2025-10-20T22:04:05.933009] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-03 (1.68 seconds). [2025-10-20T22:04:06.651420] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-04. [2025-10-20T22:04:06.793893] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-08 (1.74 seconds). [2025-10-20T22:04:07.204086] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-02 (1.35 seconds). [2025-10-20T22:04:07.451373] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-03. [2025-10-20T22:04:08.245569] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-04 (1.59 seconds). [2025-10-20T22:04:08.255983] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-07. [2025-10-20T22:04:08.773455] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-03 (1.32 seconds). [2025-10-20T22:04:09.051379] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-05. [2025-10-20T22:04:09.656760] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-07 (1.40 seconds). [2025-10-20T22:04:09.851368] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-08. [2025-10-20T22:04:10.651413] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-09. [2025-10-20T22:04:10.933943] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-05 (1.88 seconds). [2025-10-20T22:04:11.350238] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-08 (1.50 seconds). [2025-10-20T22:04:11.451453] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-09. [2025-10-20T22:04:12.251360] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-06. [2025-10-20T22:04:12.416870] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-09 (1.77 seconds). [2025-10-20T22:04:12.609885] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-09 (1.16 seconds). [2025-10-20T22:04:13.051377] [INFO ] [REMOTE ] ZK: Fetching primary data for 2025-10. [2025-10-20T22:04:13.851355] [INFO ] [REMOTE ] ZTO: Fetching primary data for 2025-10. [2025-10-20T22:04:14.043152] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-06 (1.79 seconds). [2025-10-20T22:04:14.528446] [INFO ] [REMOTE ] ZK: Incremental fetch complete for 2025-10 (1.48 seconds). [2025-10-20T22:04:14.534162] [INFO ] [REMOTE ] ZK: Full fetch complete (56952 records). [2025-10-20T22:04:14.534266] [DEBUG] [VALIDATOR ] ZK: Validating primary data. [2025-10-20T22:04:14.656484] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-04. [2025-10-20T22:04:14.912016] [DEBUG] [VALIDATOR ] ZK: All expected trading days are present. [2025-10-20T22:04:14.912240] [DEBUG] [VALIDATOR ] ZK: Primary data validation passed. [2025-10-20T22:04:14.913273] [INFO ] [MAIN ] ZK: Fetched primary data is valid. [2025-10-20T22:04:14.913756] [DEBUG] [LOCAL ] ZK: Saving primary data. [2025-10-20T22:04:14.915057] [DEBUG] [LOCAL ] ZK: Trimmed 56952 records to 39023 (keeping 200 days of data) [2025-10-20T22:04:15.152875] [INFO ] [REMOTE ] ZTO: Incremental fetch complete for 2025-10 (1.30 seconds). [2025-10-20T22:04:15.167878] [DEBUG] [LOCAL ] ZK: Primary data saved (39023 records). [2025-10-20T22:04:15.168161] [INFO ] [MAIN ] ZK: Full history saved. [2025-10-20T22:04:15.168280] [INFO ] [MAIN ] Update Progress: 2363/2367 symbols completed. ZK took 35.7s. Estimated remaining: 30.9s. Rate: 7.8 symbols/min [2025-10-20T22:04:15.168098] [INFO ] [REMOTE ] ZTO: Full fetch complete (72853 records). [2025-10-20T22:04:15.188999] [DEBUG] [VALIDATOR ] ZTO: Validating primary data. [2025-10-20T22:04:15.482852] [INFO ] [MAIN ] ZWS: Starting incremental primary update from 2025-10-10 to 2025-10-20. [2025-10-20T22:04:15.480717] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-07. [2025-10-20T22:04:15.759944] [DEBUG] [VALIDATOR ] ZTO: All expected trading days are present. [2025-10-20T22:04:15.761112] [DEBUG] [VALIDATOR ] ZTO: Primary data validation passed. [2025-10-20T22:04:15.762958] [INFO ] [MAIN ] ZTO: Fetched primary data is valid. [2025-10-20T22:04:15.763053] [DEBUG] [LOCAL ] ZTO: Saving primary data. [2025-10-20T22:04:15.761689] [DEBUG] [LOCAL ] ZWS: Loaded 43041 existing records from primary storage. [2025-10-20T22:04:15.763532] [INFO ] [MAIN ] ZWS: Loaded 43041 existing records. [2025-10-20T22:04:15.864932] [DEBUG] [LOCAL ] ZTO: Trimmed 72853 records to 49462 (keeping 200 days of data) [2025-10-20T22:04:16.042068] [WARN ] [MAIN ] ZWS: Insufficient historical data. Have data from 2025-03-31, but need from 2025-01-02 for 200 day retention. Performing full refresh. [2025-10-20T22:04:16.043007] [INFO ] [MAIN ] ZWS: Wiping existing primary data before full refresh. [2025-10-20T22:04:16.045830] [INFO ] [LOCAL ] ZWS: Starting deletion of primary data. [2025-10-20T22:04:16.052472] [DEBUG] [LOCAL ] ZWS: Deleted primary data file. [2025-10-20T22:04:16.055819] [INFO ] [MAIN ] ZWS: Starting full primary refresh. [2025-10-20T22:04:16.420032] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-04 (1.76 seconds). [2025-10-20T22:04:16.505471] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-05. [2025-10-20T22:04:16.705826] [INFO ] [REMOTE ] ZWS: Fetching primary data covering 200 trading days from 2025-01-02 to 2025-10-20 (requires 10 months of data pull). [2025-10-20T22:04:16.705984] [DEBUG] [LOCAL ] ZTO: Primary data saved (49462 records). [2025-10-20T22:04:16.706561] [INFO ] [MAIN ] ZTO: Full history saved. [2025-10-20T22:04:16.707479] [INFO ] [MAIN ] Update Progress: 2364/2367 symbols completed. ZTO took 37.3s. Estimated remaining: 23.2s. Rate: 7.8 symbols/min [2025-10-20T22:04:17.051708] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-01. [2025-10-20T22:04:17.492599] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-07 (2.01 seconds). [2025-10-20T22:04:17.851430] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-08. [2025-10-20T22:04:18.070762] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-05 (1.57 seconds). [2025-10-20T22:04:18.501684] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-01 (1.45 seconds). [2025-10-20T22:04:18.651434] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-02. [2025-10-20T22:04:19.451443] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-06. [2025-10-20T22:04:19.799829] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-08 (1.95 seconds). [2025-10-20T22:04:20.205067] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-02 (1.55 seconds). [2025-10-20T22:04:20.251350] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-03. [2025-10-20T22:04:20.790347] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-06 (1.34 seconds). [2025-10-20T22:04:21.051429] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-09. [2025-10-20T22:04:21.844244] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-03 (1.59 seconds). [2025-10-20T22:04:21.852878] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-07. [2025-10-20T22:04:22.651364] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-04. [2025-10-20T22:04:23.021442] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-09 (1.97 seconds). [2025-10-20T22:04:23.310867] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-07 (1.46 seconds). [2025-10-20T22:04:23.451384] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-08. [2025-10-20T22:04:23.932124] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-04 (1.28 seconds). [2025-10-20T22:04:24.251449] [INFO ] [REMOTE ] ZTS: Fetching primary data for 2025-10. [2025-10-20T22:04:24.952411] [INFO ] [REMOTE ] ZTS: Incremental fetch complete for 2025-10 (0.70 seconds). [2025-10-20T22:04:24.960083] [INFO ] [REMOTE ] ZTS: Full fetch complete (76882 records). [2025-10-20T22:04:24.960211] [DEBUG] [VALIDATOR ] ZTS: Validating primary data. [2025-10-20T22:04:25.054467] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-05. [2025-10-20T22:04:25.166783] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-08 (1.72 seconds). [2025-10-20T22:04:25.389076] [DEBUG] [VALIDATOR ] ZTS: All expected trading days are present. [2025-10-20T22:04:25.389155] [DEBUG] [VALIDATOR ] ZTS: Primary data validation passed. [2025-10-20T22:04:25.390306] [INFO ] [MAIN ] ZTS: Fetched primary data is valid. [2025-10-20T22:04:25.390368] [DEBUG] [LOCAL ] ZTS: Saving primary data. [2025-10-20T22:04:25.391778] [DEBUG] [LOCAL ] ZTS: Trimmed 76882 records to 53138 (keeping 200 days of data) [2025-10-20T22:04:25.715422] [DEBUG] [LOCAL ] ZTS: Primary data saved (53138 records). [2025-10-20T22:04:25.715551] [INFO ] [MAIN ] ZTS: Full history saved. [2025-10-20T22:04:25.715866] [INFO ] [MAIN ] Update Progress: 2365/2367 symbols completed. ZTS took 31.8s. Estimated remaining: 15.4s. Rate: 7.8 symbols/min [2025-10-20T22:04:25.851418] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-09. [2025-10-20T22:04:26.652456] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-05 (1.60 seconds). [2025-10-20T22:04:26.661619] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-06. [2025-10-20T22:04:27.214615] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-09 (1.36 seconds). [2025-10-20T22:04:27.451473] [INFO ] [REMOTE ] ZVIA: Fetching primary data for 2025-10. [2025-10-20T22:04:28.178816] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-06 (1.52 seconds). [2025-10-20T22:04:28.251401] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-07. [2025-10-20T22:04:28.768726] [INFO ] [REMOTE ] ZVIA: Incremental fetch complete for 2025-10 (1.32 seconds). [2025-10-20T22:04:28.774396] [INFO ] [REMOTE ] ZVIA: Full fetch complete (52765 records). [2025-10-20T22:04:28.774534] [DEBUG] [VALIDATOR ] ZVIA: Validating primary data. [2025-10-20T22:04:29.159239] [DEBUG] [VALIDATOR ] ZVIA: All expected trading days are present. [2025-10-20T22:04:29.159326] [DEBUG] [VALIDATOR ] ZVIA: Primary data validation passed. [2025-10-20T22:04:29.160378] [INFO ] [MAIN ] ZVIA: Fetched primary data is valid. [2025-10-20T22:04:29.160441] [DEBUG] [LOCAL ] ZVIA: Saving primary data. [2025-10-20T22:04:29.161557] [DEBUG] [LOCAL ] ZVIA: Trimmed 52765 records to 35831 (keeping 200 days of data) [2025-10-20T22:04:29.362462] [DEBUG] [LOCAL ] ZVIA: Primary data saved (35831 records). [2025-10-20T22:04:29.362603] [INFO ] [MAIN ] ZVIA: Full history saved. [2025-10-20T22:04:29.362707] [INFO ] [MAIN ] Update Progress: 2366/2367 symbols completed. ZVIA took 26.9s. Estimated remaining: 7.7s. Rate: 7.8 symbols/min [2025-10-20T22:04:29.848045] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-07 (1.60 seconds). [2025-10-20T22:04:29.858006] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-08. [2025-10-20T22:04:31.339049] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-08 (1.48 seconds). [2025-10-20T22:04:31.348668] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-09. [2025-10-20T22:04:33.150126] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-09 (1.80 seconds). [2025-10-20T22:04:33.159271] [INFO ] [REMOTE ] ZWS: Fetching primary data for 2025-10. [2025-10-20T22:04:34.646213] [INFO ] [REMOTE ] ZWS: Incremental fetch complete for 2025-10 (1.49 seconds). [2025-10-20T22:04:34.652534] [INFO ] [REMOTE ] ZWS: Full fetch complete (62118 records). [2025-10-20T22:04:34.652631] [DEBUG] [VALIDATOR ] ZWS: Validating primary data. [2025-10-20T22:04:35.055189] [DEBUG] [VALIDATOR ] ZWS: All expected trading days are present. [2025-10-20T22:04:35.055286] [DEBUG] [VALIDATOR ] ZWS: Primary data validation passed. [2025-10-20T22:04:35.056335] [INFO ] [MAIN ] ZWS: Fetched primary data is valid. [2025-10-20T22:04:35.056399] [DEBUG] [LOCAL ] ZWS: Saving primary data. [2025-10-20T22:04:35.057533] [DEBUG] [LOCAL ] ZWS: Trimmed 62118 records to 42373 (keeping 200 days of data) [2025-10-20T22:04:35.304292] [DEBUG] [LOCAL ] ZWS: Primary data saved (42373 records). [2025-10-20T22:04:35.304406] [INFO ] [MAIN ] ZWS: Full history saved. [2025-10-20T22:04:35.304479] [INFO ] [MAIN ] Update Progress: 2367/2367 symbols completed. ZWS took 20.1s. Estimated remaining: 0.0s. Rate: 7.8 symbols/min [2025-10-20T22:04:35.305284] [INFO ] [MAIN ] Parallel update completed in 5h 4m 31s [2025-10-20T22:04:35.305365] [WARN ] [MAIN ] Failed symbols (490): AAM, AAUC, AA, ABCB, ABBV, ACA, ACI, ACR, AD, ACV... [2025-10-20T22:04:35.305446] [INFO ] [MAIN ] Update success rate: 79.3% (1877/2367 symbols) [2025-10-20T22:04:38.976144] [DEBUG] [EMAIL ] Email sent to 'chris.punches@silogroup.org' with subject: 'COMPLETE_OK: TDM Primary Update Completed (1877/2367 @ 79.30%)' [2025-10-20T22:04:38.976600] [INFO ] [MAIN ] Primary update successful. Now generating derivatives... [2025-10-20T22:04:38.976809] [INFO ] [MAIN ] Refreshing symbols list before derivative generation... [2025-10-20T22:04:38.976860] [INFO ] [MAIN ] Starting TDM symbol refresh process. [2025-10-20T22:04:38.976946] [INFO ] [MAIN ] Fetching latest NYSE symbols list. [2025-10-20T22:04:38.976982] [DEBUG] [SYMBOL ] Starting fetch of NYSE Symbols. [2025-10-20T22:04:40.000157] [DEBUG] [SYMBOL ] Found 2419 valid NYSE symbols. [2025-10-20T22:04:40.000279] [INFO ] [SYMBOL ] Completed fetch of NYSE Symbols. [2025-10-20T22:04:40.000467] [INFO ] [MAIN ] Filtered out 54 blacklisted symbols. [2025-10-20T22:04:40.000971] [INFO ] [LOCAL ] Symbols cache updated with 2367 symbols. [2025-10-20T22:04:40.001026] [INFO ] [MAIN ] Successfully refreshed symbols list. [2025-10-20T22:04:40.001056] [INFO ] [MAIN ] Finished TDM symbol refresh process. [2025-10-20T22:04:40.001110] [INFO ] [MAIN ] Pruning stale symbols... [2025-10-20T22:04:40.001140] [INFO ] [MAIN ] Starting TDM stale symbol pruning process. [2025-10-20T22:04:40.001168] [INFO ] [LOCAL ] Reading cached symbols list. [2025-10-20T22:04:40.001583] [DEBUG] [LOCAL ] Found 2367 symbols in cached list. [2025-10-20T22:04:40.001621] [INFO ] [LOCAL ] Completed reading cached symbols list. [2025-10-20T22:04:40.001835] [INFO ] [MAIN ] Including 54 blacklisted symbols for pruning. [2025-10-20T22:04:40.001883] [INFO ] [MAIN ] Pruning data for symbols not in current list of 2367 valid symbols. [2025-10-20T22:04:40.001954] [INFO ] [LOCAL ] Starting pruning of local symbol history data for inactive/invalid symbols. [2025-10-20T22:04:40.001993] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/primary_storage' [2025-10-20T22:04:40.028177] [DEBUG] [LOCAL ] Checking directory: '/opt/tdm/output/data/derivative_storage' [2025-10-20T22:04:40.049876] [INFO ] [LOCAL ] Deleted 0 obsolete data files. [2025-10-20T22:04:40.049932] [INFO ] [LOCAL ] Completed pruning of unlisted local symbol history data. [2025-10-20T22:04:40.049963] [INFO ] [MAIN ] Successfully pruned stale symbol data. [2025-10-20T22:04:40.049991] [INFO ] [MAIN ] Finished TDM stale symbol pruning process. [2025-10-20T22:04:40.050041] [INFO ] [MAIN ] Running derivative generation in parallel mode [2025-10-20T22:04:40.050070] [INFO ] [MAIN ] Wiping all existing derivative data before regeneration. [2025-10-20T22:04:40.050101] [INFO ] [LOCAL ] Starting deletion of all local derivative data. [2025-10-20T22:04:40.051117] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CINT.csv [2025-10-20T22:04:40.051233] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMC.csv [2025-10-20T22:04:40.051341] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FF.csv [2025-10-20T22:04:40.051414] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PLOW.csv [2025-10-20T22:04:40.051483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTN.csv [2025-10-20T22:04:40.051574] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SBR.csv [2025-10-20T22:04:40.051708] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BTI.csv [2025-10-20T22:04:40.051842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TX.csv [2025-10-20T22:04:40.051939] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTZ.csv [2025-10-20T22:04:40.052014] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RSF.csv [2025-10-20T22:04:40.052084] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UTZ.csv [2025-10-20T22:04:40.052204] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ERO.csv [2025-10-20T22:04:40.052318] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OPP.csv [2025-10-20T22:04:40.052413] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OBDC.csv [2025-10-20T22:04:40.052505] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JLL.csv [2025-10-20T22:04:40.052634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HII.csv [2025-10-20T22:04:40.052766] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AGD.csv [2025-10-20T22:04:40.052873] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMWL.csv [2025-10-20T22:04:40.052943] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MDST.csv [2025-10-20T22:04:40.053010] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KEN.csv [2025-10-20T22:04:40.053090] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EGO.csv [2025-10-20T22:04:40.053163] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NRK.csv [2025-10-20T22:04:40.053240] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HRL.csv [2025-10-20T22:04:40.053345] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DXYZ.csv [2025-10-20T22:04:40.053435] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPFD.csv [2025-10-20T22:04:40.053517] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMX.csv [2025-10-20T22:04:40.053583] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BV.csv [2025-10-20T22:04:40.053672] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MYN.csv [2025-10-20T22:04:40.053768] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHE.csv [2025-10-20T22:04:40.053839] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RZC.csv [2025-10-20T22:04:40.053927] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NABL.csv [2025-10-20T22:04:40.054019] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SII.csv [2025-10-20T22:04:40.054114] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NCV.csv [2025-10-20T22:04:40.054216] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNK.csv [2025-10-20T22:04:40.054297] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIN.csv [2025-10-20T22:04:40.054379] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TREX.csv [2025-10-20T22:04:40.054485] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NTST.csv [2025-10-20T22:04:40.054554] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SKM.csv [2025-10-20T22:04:40.054659] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PCF.csv [2025-10-20T22:04:40.054775] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVN.csv [2025-10-20T22:04:40.054844] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UGI.csv [2025-10-20T22:04:40.054910] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CEPU.csv [2025-10-20T22:04:40.054989] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMPX.csv [2025-10-20T22:04:40.055053] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTB.csv [2025-10-20T22:04:40.055120] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AAT.csv [2025-10-20T22:04:40.055186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WMS.csv [2025-10-20T22:04:40.055283] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CSV.csv [2025-10-20T22:04:40.055387] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VKQ.csv [2025-10-20T22:04:40.055475] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MLNK.csv [2025-10-20T22:04:40.055558] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FINV.csv [2025-10-20T22:04:40.055622] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/S.csv [2025-10-20T22:04:40.055751] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FSS.csv [2025-10-20T22:04:40.055876] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JELD.csv [2025-10-20T22:04:40.055949] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JKS.csv [2025-10-20T22:04:40.056036] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EOS.csv [2025-10-20T22:04:40.056116] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NCA.csv [2025-10-20T22:04:40.056221] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WAT.csv [2025-10-20T22:04:40.056310] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QUAD.csv [2025-10-20T22:04:40.056394] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUR.csv [2025-10-20T22:04:40.056493] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FNB.csv [2025-10-20T22:04:40.056559] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRMB.csv [2025-10-20T22:04:40.056687] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LAZ.csv [2025-10-20T22:04:40.058451] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YPF.csv [2025-10-20T22:04:40.058636] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UTL.csv [2025-10-20T22:04:40.058832] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UFI.csv [2025-10-20T22:04:40.058934] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFC.csv [2025-10-20T22:04:40.059024] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PEO.csv [2025-10-20T22:04:40.059105] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTHY.csv [2025-10-20T22:04:40.059184] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPHR.csv [2025-10-20T22:04:40.059288] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GCV.csv [2025-10-20T22:04:40.059377] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BP.csv [2025-10-20T22:04:40.059472] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRC.csv [2025-10-20T22:04:40.059554] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDOT.csv [2025-10-20T22:04:40.059624] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NVS.csv [2025-10-20T22:04:40.059763] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FVR.csv [2025-10-20T22:04:40.059841] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NVGS.csv [2025-10-20T22:04:40.059922] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OMF.csv [2025-10-20T22:04:40.060000] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALL.csv [2025-10-20T22:04:40.060095] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVB.csv [2025-10-20T22:04:40.060196] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OR.csv [2025-10-20T22:04:40.060275] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMR.csv [2025-10-20T22:04:40.060352] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVH.csv [2025-10-20T22:04:40.060415] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLCO.csv [2025-10-20T22:04:40.060481] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVI.csv [2025-10-20T22:04:40.060546] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WY.csv [2025-10-20T22:04:40.060608] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BR.csv [2025-10-20T22:04:40.060723] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TGLS.csv [2025-10-20T22:04:40.060831] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TD.csv [2025-10-20T22:04:40.060923] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMU.csv [2025-10-20T22:04:40.061003] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XYL.csv [2025-10-20T22:04:40.061091] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LUMN.csv [2025-10-20T22:04:40.061179] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMA.csv [2025-10-20T22:04:40.061256] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OPFI.csv [2025-10-20T22:04:40.061347] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGR.csv [2025-10-20T22:04:40.061440] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AFGD.csv [2025-10-20T22:04:40.061510] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MHF.csv [2025-10-20T22:04:40.061572] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WBX.csv [2025-10-20T22:04:40.061637] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KMPB.csv [2025-10-20T22:04:40.061763] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMBC.csv [2025-10-20T22:04:40.061868] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GTY.csv [2025-10-20T22:04:40.061970] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VPG.csv [2025-10-20T22:04:40.062062] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CUZ.csv [2025-10-20T22:04:40.062142] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRX.csv [2025-10-20T22:04:40.062209] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIPO.csv [2025-10-20T22:04:40.062302] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FPI.csv [2025-10-20T22:04:40.062399] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/H.csv [2025-10-20T22:04:40.062466] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNQ.csv [2025-10-20T22:04:40.062533] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAY.csv [2025-10-20T22:04:40.062601] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HEQ.csv [2025-10-20T22:04:40.062693] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WIT.csv [2025-10-20T22:04:40.062785] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WTRG.csv [2025-10-20T22:04:40.062853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UGP.csv [2025-10-20T22:04:40.062935] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPOR.csv [2025-10-20T22:04:40.063012] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVTR.csv [2025-10-20T22:04:40.063077] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DRH.csv [2025-10-20T22:04:40.063163] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FCPT.csv [2025-10-20T22:04:40.063254] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFN.csv [2025-10-20T22:04:40.063321] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HAE.csv [2025-10-20T22:04:40.063388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TOL.csv [2025-10-20T22:04:40.063458] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LYB.csv [2025-10-20T22:04:40.063523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNF.csv [2025-10-20T22:04:40.063587] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOR.csv [2025-10-20T22:04:40.063703] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IMAX.csv [2025-10-20T22:04:40.063808] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIZN.csv [2025-10-20T22:04:40.063878] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WS.csv [2025-10-20T22:04:40.063947] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAL.csv [2025-10-20T22:04:40.064024] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLR.csv [2025-10-20T22:04:40.064087] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BK.csv [2025-10-20T22:04:40.064275] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IIM.csv [2025-10-20T22:04:40.064347] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WCN.csv [2025-10-20T22:04:40.064412] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NNY.csv [2025-10-20T22:04:40.064477] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VLO.csv [2025-10-20T22:04:40.064594] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NGVT.csv [2025-10-20T22:04:40.064729] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DOV.csv [2025-10-20T22:04:40.064812] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KMPR.csv [2025-10-20T22:04:40.064891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVTL.csv [2025-10-20T22:04:40.064973] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KWR.csv [2025-10-20T22:04:40.065060] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AORT.csv [2025-10-20T22:04:40.065127] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BWNB.csv [2025-10-20T22:04:40.065199] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SAN.csv [2025-10-20T22:04:40.065264] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USB.csv [2025-10-20T22:04:40.065327] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RGT.csv [2025-10-20T22:04:40.065389] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BMY.csv [2025-10-20T22:04:40.065462] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLO.csv [2025-10-20T22:04:40.065538] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCS.csv [2025-10-20T22:04:40.065602] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YSG.csv [2025-10-20T22:04:40.065767] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATO.csv [2025-10-20T22:04:40.065870] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DSL.csv [2025-10-20T22:04:40.065938] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NREF.csv [2025-10-20T22:04:40.066001] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIVI.csv [2025-10-20T22:04:40.066080] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNT.csv [2025-10-20T22:04:40.066172] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KIM.csv [2025-10-20T22:04:40.066259] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRT.csv [2025-10-20T22:04:40.066324] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBAI.csv [2025-10-20T22:04:40.066388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TMO.csv [2025-10-20T22:04:40.066450] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGRB.csv [2025-10-20T22:04:40.066519] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LTH.csv [2025-10-20T22:04:40.066581] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRO.csv [2025-10-20T22:04:40.066683] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PCN.csv [2025-10-20T22:04:40.066788] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ARDT.csv [2025-10-20T22:04:40.066853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFSI.csv [2025-10-20T22:04:40.066920] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LNG.csv [2025-10-20T22:04:40.066992] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LXU.csv [2025-10-20T22:04:40.067054] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRGP.csv [2025-10-20T22:04:40.067118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UHAL.csv [2025-10-20T22:04:40.067180] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LOB.csv [2025-10-20T22:04:40.067242] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XPER.csv [2025-10-20T22:04:40.067328] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FPH.csv [2025-10-20T22:04:40.067391] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WST.csv [2025-10-20T22:04:40.067454] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPK.csv [2025-10-20T22:04:40.067534] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GL.csv [2025-10-20T22:04:40.067612] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IRS.csv [2025-10-20T22:04:40.067719] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FR.csv [2025-10-20T22:04:40.067802] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PIM.csv [2025-10-20T22:04:40.067866] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTR.csv [2025-10-20T22:04:40.067951] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MD.csv [2025-10-20T22:04:40.068046] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NFJ.csv [2025-10-20T22:04:40.068126] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FHI.csv [2025-10-20T22:04:40.068206] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SRG.csv [2025-10-20T22:04:40.068308] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ACM.csv [2025-10-20T22:04:40.068374] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PKE.csv [2025-10-20T22:04:40.068438] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CWAN.csv [2025-10-20T22:04:40.068510] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPA.csv [2025-10-20T22:04:40.068573] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZTR.csv [2025-10-20T22:04:40.068683] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FNV.csv [2025-10-20T22:04:40.068788] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DNOW.csv [2025-10-20T22:04:40.068869] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBZ.csv [2025-10-20T22:04:40.068951] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPMC.csv [2025-10-20T22:04:40.069053] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BWA.csv [2025-10-20T22:04:40.069119] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MYE.csv [2025-10-20T22:04:40.069183] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JRI.csv [2025-10-20T22:04:40.069246] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNY.csv [2025-10-20T22:04:40.069323] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GGG.csv [2025-10-20T22:04:40.069387] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HYT.csv [2025-10-20T22:04:40.069467] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPC.csv [2025-10-20T22:04:40.069542] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GUT.csv [2025-10-20T22:04:40.069617] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHC.csv [2025-10-20T22:04:40.069714] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNED.csv [2025-10-20T22:04:40.069808] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DXC.csv [2025-10-20T22:04:40.069873] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BWLP.csv [2025-10-20T22:04:40.069947] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNA.csv [2025-10-20T22:04:40.070008] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFG.csv [2025-10-20T22:04:40.070069] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TXNM.csv [2025-10-20T22:04:40.070173] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ULS.csv [2025-10-20T22:04:40.070246] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PNI.csv [2025-10-20T22:04:40.070337] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IFS.csv [2025-10-20T22:04:40.070433] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BC.csv [2025-10-20T22:04:40.070502] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMSD.csv [2025-10-20T22:04:40.070573] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFSG.csv [2025-10-20T22:04:40.070670] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TYL.csv [2025-10-20T22:04:40.070775] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TIMB.csv [2025-10-20T22:04:40.070891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EE.csv [2025-10-20T22:04:40.070958] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPS.csv [2025-10-20T22:04:40.071038] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRK.csv [2025-10-20T22:04:40.071106] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAC.csv [2025-10-20T22:04:40.071170] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVEO.csv [2025-10-20T22:04:40.071250] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HI.csv [2025-10-20T22:04:40.071318] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSCI.csv [2025-10-20T22:04:40.071388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RNG.csv [2025-10-20T22:04:40.071483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DLNG.csv [2025-10-20T22:04:40.071581] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGRD.csv [2025-10-20T22:04:40.071676] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NNN.csv [2025-10-20T22:04:40.071766] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ABEV.csv [2025-10-20T22:04:40.071840] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VMI.csv [2025-10-20T22:04:40.071907] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FHN.csv [2025-10-20T22:04:40.071975] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PGZ.csv [2025-10-20T22:04:40.072052] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMBT.csv [2025-10-20T22:04:40.072119] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QD.csv [2025-10-20T22:04:40.072187] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VICI.csv [2025-10-20T22:04:40.072262] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IHG.csv [2025-10-20T22:04:40.072329] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NRDY.csv [2025-10-20T22:04:40.072410] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPC.csv [2025-10-20T22:04:40.072484] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRX.csv [2025-10-20T22:04:40.072553] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRC.csv [2025-10-20T22:04:40.072619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAM.csv [2025-10-20T22:04:40.072756] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BHE.csv [2025-10-20T22:04:40.072842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UBER.csv [2025-10-20T22:04:40.072912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEO.csv [2025-10-20T22:04:40.072983] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DEI.csv [2025-10-20T22:04:40.073060] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AJG.csv [2025-10-20T22:04:40.073128] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GTN.csv [2025-10-20T22:04:40.073192] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AFB.csv [2025-10-20T22:04:40.073255] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BFLY.csv [2025-10-20T22:04:40.073342] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBU.csv [2025-10-20T22:04:40.073429] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHY.csv [2025-10-20T22:04:40.073499] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ITGR.csv [2025-10-20T22:04:40.073567] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SRV.csv [2025-10-20T22:04:40.073633] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MRC.csv [2025-10-20T22:04:40.073870] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WOR.csv [2025-10-20T22:04:40.073950] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NGL.csv [2025-10-20T22:04:40.074014] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BFZ.csv [2025-10-20T22:04:40.074082] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TLK.csv [2025-10-20T22:04:40.074148] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WD.csv [2025-10-20T22:04:40.074210] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BVN.csv [2025-10-20T22:04:40.074277] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WEAV.csv [2025-10-20T22:04:40.074345] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MHN.csv [2025-10-20T22:04:40.074412] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WK.csv [2025-10-20T22:04:40.074482] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHWY.csv [2025-10-20T22:04:40.074562] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CSL.csv [2025-10-20T22:04:40.074629] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMR.csv [2025-10-20T22:04:40.074743] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WLY.csv [2025-10-20T22:04:40.074845] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGF.csv [2025-10-20T22:04:40.074938] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEL.csv [2025-10-20T22:04:40.075021] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRBK.csv [2025-10-20T22:04:40.075090] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TIXT.csv [2025-10-20T22:04:40.075158] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RBRK.csv [2025-10-20T22:04:40.075248] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EOG.csv [2025-10-20T22:04:40.075314] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/COMP.csv [2025-10-20T22:04:40.075380] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LRN.csv [2025-10-20T22:04:40.075446] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGA.csv [2025-10-20T22:04:40.075513] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SXC.csv [2025-10-20T22:04:40.075592] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HUN.csv [2025-10-20T22:04:40.075689] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JHI.csv [2025-10-20T22:04:40.075789] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MEG.csv [2025-10-20T22:04:40.075901] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMT.csv [2025-10-20T22:04:40.076042] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LC.csv [2025-10-20T22:04:40.076122] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ES.csv [2025-10-20T22:04:40.076186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BARK.csv [2025-10-20T22:04:40.076260] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HYTR.csv [2025-10-20T22:04:40.076326] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WHG.csv [2025-10-20T22:04:40.076400] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HESM.csv [2025-10-20T22:04:40.076473] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XHR.csv [2025-10-20T22:04:40.076547] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TROX.csv [2025-10-20T22:04:40.076611] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNW.csv [2025-10-20T22:04:40.076726] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MDV.csv [2025-10-20T22:04:40.076817] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BILL.csv [2025-10-20T22:04:40.076887] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WRBY.csv [2025-10-20T22:04:40.076952] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IQI.csv [2025-10-20T22:04:40.077030] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCI.csv [2025-10-20T22:04:40.077092] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BCX.csv [2025-10-20T22:04:40.077177] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOC.csv [2025-10-20T22:04:40.077267] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NIC.csv [2025-10-20T22:04:40.077332] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BIPC.csv [2025-10-20T22:04:40.077408] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALUR.csv [2025-10-20T22:04:40.077483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CEE.csv [2025-10-20T22:04:40.077548] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PHK.csv [2025-10-20T22:04:40.077612] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SKE.csv [2025-10-20T22:04:40.077729] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLXR.csv [2025-10-20T22:04:40.077822] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JPM.csv [2025-10-20T22:04:40.077902] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTS.csv [2025-10-20T22:04:40.077993] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDI.csv [2025-10-20T22:04:40.078086] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLI.csv [2025-10-20T22:04:40.078153] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QSR.csv [2025-10-20T22:04:40.078218] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFSV.csv [2025-10-20T22:04:40.078283] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAA.csv [2025-10-20T22:04:40.078347] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THS.csv [2025-10-20T22:04:40.078423] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDIV.csv [2025-10-20T22:04:40.078486] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BALL.csv [2025-10-20T22:04:40.078550] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IR.csv [2025-10-20T22:04:40.078637] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DDL.csv [2025-10-20T22:04:40.078750] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVLG.csv [2025-10-20T22:04:40.078821] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMI.csv [2025-10-20T22:04:40.078901] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FNF.csv [2025-10-20T22:04:40.078974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CW.csv [2025-10-20T22:04:40.079053] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GBCI.csv [2025-10-20T22:04:40.079130] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRDN.csv [2025-10-20T22:04:40.079194] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MYI.csv [2025-10-20T22:04:40.079287] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PACK.csv [2025-10-20T22:04:40.079373] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/COP.csv [2025-10-20T22:04:40.079440] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RMT.csv [2025-10-20T22:04:40.079513] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PLYM.csv [2025-10-20T22:04:40.079592] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DINO.csv [2025-10-20T22:04:40.079686] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VAC.csv [2025-10-20T22:04:40.079811] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMG.csv [2025-10-20T22:04:40.079901] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AU.csv [2025-10-20T22:04:40.079975] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AKR.csv [2025-10-20T22:04:40.080052] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMM.csv [2025-10-20T22:04:40.080114] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BIPH.csv [2025-10-20T22:04:40.080192] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ADT.csv [2025-10-20T22:04:40.080256] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLDR.csv [2025-10-20T22:04:40.080331] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIT.csv [2025-10-20T22:04:40.080393] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ARCO.csv [2025-10-20T22:04:40.080469] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NBHC.csv [2025-10-20T22:04:40.080548] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EHC.csv [2025-10-20T22:04:40.080612] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PPT.csv [2025-10-20T22:04:40.080716] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THLV.csv [2025-10-20T22:04:40.080810] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AXTA.csv [2025-10-20T22:04:40.080878] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MA.csv [2025-10-20T22:04:40.080946] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/APAM.csv [2025-10-20T22:04:40.081010] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ADC.csv [2025-10-20T22:04:40.081074] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VIRT.csv [2025-10-20T22:04:40.081139] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETSY.csv [2025-10-20T22:04:40.081276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SAR.csv [2025-10-20T22:04:40.081369] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIG.csv [2025-10-20T22:04:40.081446] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTG.csv [2025-10-20T22:04:40.081552] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BW.csv [2025-10-20T22:04:40.081631] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FRA.csv [2025-10-20T22:04:40.081824] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MODG.csv [2025-10-20T22:04:40.081906] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AQNB.csv [2025-10-20T22:04:40.081972] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STK.csv [2025-10-20T22:04:40.082067] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ABR.csv [2025-10-20T22:04:40.082174] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATR.csv [2025-10-20T22:04:40.082255] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EFXT.csv [2025-10-20T22:04:40.082333] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CF.csv [2025-10-20T22:04:40.082409] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GROV.csv [2025-10-20T22:04:40.082472] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLX.csv [2025-10-20T22:04:40.082549] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRGY.csv [2025-10-20T22:04:40.082622] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HHH.csv [2025-10-20T22:04:40.082731] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZTO.csv [2025-10-20T22:04:40.082811] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BUD.csv [2025-10-20T22:04:40.082891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRAK.csv [2025-10-20T22:04:40.082992] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LPG.csv [2025-10-20T22:04:40.083057] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BDC.csv [2025-10-20T22:04:40.083121] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UVV.csv [2025-10-20T22:04:40.083186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JPC.csv [2025-10-20T22:04:40.083248] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VZ.csv [2025-10-20T22:04:40.083327] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECL.csv [2025-10-20T22:04:40.083398] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTH.csv [2025-10-20T22:04:40.083492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HUM.csv [2025-10-20T22:04:40.083580] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CTVA.csv [2025-10-20T22:04:40.083673] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STXT.csv [2025-10-20T22:04:40.083766] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VSTS.csv [2025-10-20T22:04:40.083913] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALK.csv [2025-10-20T22:04:40.084015] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FICO.csv [2025-10-20T22:04:40.084081] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USA.csv [2025-10-20T22:04:40.084142] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LNC.csv [2025-10-20T22:04:40.084205] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSB.csv [2025-10-20T22:04:40.084306] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FSCO.csv [2025-10-20T22:04:40.084368] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KF.csv [2025-10-20T22:04:40.084445] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DVA.csv [2025-10-20T22:04:40.084524] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPRU.csv [2025-10-20T22:04:40.084612] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IFN.csv [2025-10-20T22:04:40.084723] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPB.csv [2025-10-20T22:04:40.084819] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EXK.csv [2025-10-20T22:04:40.084914] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKKT.csv [2025-10-20T22:04:40.085000] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHGG.csv [2025-10-20T22:04:40.085089] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FIS.csv [2025-10-20T22:04:40.085152] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SHOC.csv [2025-10-20T22:04:40.085215] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JBGS.csv [2025-10-20T22:04:40.085276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKV.csv [2025-10-20T22:04:40.085341] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVD.csv [2025-10-20T22:04:40.085410] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KLC.csv [2025-10-20T22:04:40.085486] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SLQT.csv [2025-10-20T22:04:40.085588] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FVRR.csv [2025-10-20T22:04:40.085693] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASBA.csv [2025-10-20T22:04:40.085785] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVNA.csv [2025-10-20T22:04:40.085878] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IGR.csv [2025-10-20T22:04:40.085974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NBXG.csv [2025-10-20T22:04:40.086051] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BZH.csv [2025-10-20T22:04:40.086123] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WPP.csv [2025-10-20T22:04:40.086203] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KB.csv [2025-10-20T22:04:40.086284] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AEFC.csv [2025-10-20T22:04:40.086365] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SUI.csv [2025-10-20T22:04:40.086458] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAI.csv [2025-10-20T22:04:40.086523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CULP.csv [2025-10-20T22:04:40.086591] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SHEL.csv [2025-10-20T22:04:40.086686] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VNT.csv [2025-10-20T22:04:40.086797] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETV.csv [2025-10-20T22:04:40.086867] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WM.csv [2025-10-20T22:04:40.086932] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CURV.csv [2025-10-20T22:04:40.086998] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDW.csv [2025-10-20T22:04:40.087074] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CYH.csv [2025-10-20T22:04:40.087137] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BJ.csv [2025-10-20T22:04:40.087218] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THW.csv [2025-10-20T22:04:40.087322] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VNO.csv [2025-10-20T22:04:40.087413] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EXG.csv [2025-10-20T22:04:40.087525] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/COUR.csv [2025-10-20T22:04:40.087614] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCK.csv [2025-10-20T22:04:40.087723] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IVT.csv [2025-10-20T22:04:40.087808] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XPRO.csv [2025-10-20T22:04:40.087876] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSA.csv [2025-10-20T22:04:40.087969] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SDHC.csv [2025-10-20T22:04:40.088058] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BTU.csv [2025-10-20T22:04:40.088132] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VLTO.csv [2025-10-20T22:04:40.088197] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEI.csv [2025-10-20T22:04:40.088264] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IBM.csv [2025-10-20T22:04:40.088324] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AXR.csv [2025-10-20T22:04:40.088401] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EAT.csv [2025-10-20T22:04:40.088466] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LION.csv [2025-10-20T22:04:40.088536] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BXC.csv [2025-10-20T22:04:40.088598] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BGR.csv [2025-10-20T22:04:40.088698] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RLTY.csv [2025-10-20T22:04:40.088800] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBAN.csv [2025-10-20T22:04:40.088879] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HXL.csv [2025-10-20T22:04:40.088946] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VALE.csv [2025-10-20T22:04:40.089018] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IRM.csv [2025-10-20T22:04:40.089083] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CION.csv [2025-10-20T22:04:40.089157] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FMS.csv [2025-10-20T22:04:40.089233] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AWR.csv [2025-10-20T22:04:40.089311] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CFR.csv [2025-10-20T22:04:40.089388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EFC.csv [2025-10-20T22:04:40.089451] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRT.csv [2025-10-20T22:04:40.089534] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GENI.csv [2025-10-20T22:04:40.089599] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VFC.csv [2025-10-20T22:04:40.089820] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMFG.csv [2025-10-20T22:04:40.089934] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECC.csv [2025-10-20T22:04:40.090014] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HGTY.csv [2025-10-20T22:04:40.090081] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NJR.csv [2025-10-20T22:04:40.090159] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CR.csv [2025-10-20T22:04:40.090236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GFI.csv [2025-10-20T22:04:40.090310] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CB.csv [2025-10-20T22:04:40.090375] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NGG.csv [2025-10-20T22:04:40.090440] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MDT.csv [2025-10-20T22:04:40.090516] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DTM.csv [2025-10-20T22:04:40.090580] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TJX.csv [2025-10-20T22:04:40.090660] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BMN.csv [2025-10-20T22:04:40.090754] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AHH.csv [2025-10-20T22:04:40.090824] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HYI.csv [2025-10-20T22:04:40.090900] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KBDC.csv [2025-10-20T22:04:40.090963] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOAH.csv [2025-10-20T22:04:40.091027] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPR.csv [2025-10-20T22:04:40.091089] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DTG.csv [2025-10-20T22:04:40.091153] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KOS.csv [2025-10-20T22:04:40.091230] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GETY.csv [2025-10-20T22:04:40.091295] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRH.csv [2025-10-20T22:04:40.091362] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HKD.csv [2025-10-20T22:04:40.091425] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WAB.csv [2025-10-20T22:04:40.091487] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/REVG.csv [2025-10-20T22:04:40.091548] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AZZ.csv [2025-10-20T22:04:40.091611] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VRT.csv [2025-10-20T22:04:40.091707] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ITUB.csv [2025-10-20T22:04:40.091815] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AXP.csv [2025-10-20T22:04:40.091884] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HASI.csv [2025-10-20T22:04:40.091967] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVNS.csv [2025-10-20T22:04:40.092032] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SILA.csv [2025-10-20T22:04:40.092094] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WIW.csv [2025-10-20T22:04:40.092156] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TNC.csv [2025-10-20T22:04:40.092218] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NCDL.csv [2025-10-20T22:04:40.092284] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HR.csv [2025-10-20T22:04:40.092359] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CUBE.csv [2025-10-20T22:04:40.092436] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRND.csv [2025-10-20T22:04:40.092523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GCTS.csv [2025-10-20T22:04:40.092600] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KNX.csv [2025-10-20T22:04:40.092735] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GBX.csv [2025-10-20T22:04:40.092834] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ACCO.csv [2025-10-20T22:04:40.092899] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTEX.csv [2025-10-20T22:04:40.092974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AL.csv [2025-10-20T22:04:40.093078] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WEX.csv [2025-10-20T22:04:40.093147] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NC.csv [2025-10-20T22:04:40.093210] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TNL.csv [2025-10-20T22:04:40.093271] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VATE.csv [2025-10-20T22:04:40.093349] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FDS.csv [2025-10-20T22:04:40.093427] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AWK.csv [2025-10-20T22:04:40.093505] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EARN.csv [2025-10-20T22:04:40.093581] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CTRI.csv [2025-10-20T22:04:40.093705] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EOI.csv [2025-10-20T22:04:40.093814] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FG.csv [2025-10-20T22:04:40.093880] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TVE.csv [2025-10-20T22:04:40.093959] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LUV.csv [2025-10-20T22:04:40.094045] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TXT.csv [2025-10-20T22:04:40.094109] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLE.csv [2025-10-20T22:04:40.094174] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHMI.csv [2025-10-20T22:04:40.094236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LW.csv [2025-10-20T22:04:40.094308] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTWO.csv [2025-10-20T22:04:40.094371] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/J.csv [2025-10-20T22:04:40.094443] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CAE.csv [2025-10-20T22:04:40.094503] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AXS.csv [2025-10-20T22:04:40.094577] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INFY.csv [2025-10-20T22:04:40.094638] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSN.csv [2025-10-20T22:04:40.094749] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TLYS.csv [2025-10-20T22:04:40.094818] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBRE.csv [2025-10-20T22:04:40.094888] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TWO.csv [2025-10-20T22:04:40.094949] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SFB.csv [2025-10-20T22:04:40.095010] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDY.csv [2025-10-20T22:04:40.095087] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLG.csv [2025-10-20T22:04:40.095149] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OIA.csv [2025-10-20T22:04:40.095229] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RNGR.csv [2025-10-20T22:04:40.095318] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TNET.csv [2025-10-20T22:04:40.095382] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NEM.csv [2025-10-20T22:04:40.095455] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNL.csv [2025-10-20T22:04:40.095523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JBND.csv [2025-10-20T22:04:40.095598] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DTB.csv [2025-10-20T22:04:40.095709] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESAB.csv [2025-10-20T22:04:40.095802] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BDJ.csv [2025-10-20T22:04:40.095871] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SONY.csv [2025-10-20T22:04:40.095940] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VOC.csv [2025-10-20T22:04:40.096020] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LAC.csv [2025-10-20T22:04:40.096108] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PD.csv [2025-10-20T22:04:40.096177] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JXN.csv [2025-10-20T22:04:40.096241] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LDI.csv [2025-10-20T22:04:40.096312] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ABM.csv [2025-10-20T22:04:40.096378] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNRC.csv [2025-10-20T22:04:40.096443] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MIY.csv [2025-10-20T22:04:40.096509] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WIA.csv [2025-10-20T22:04:40.096580] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MITP.csv [2025-10-20T22:04:40.096694] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DG.csv [2025-10-20T22:04:40.096809] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MANU.csv [2025-10-20T22:04:40.096887] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MHD.csv [2025-10-20T22:04:40.096951] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ELME.csv [2025-10-20T22:04:40.097018] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XOM.csv [2025-10-20T22:04:40.097084] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VIPS.csv [2025-10-20T22:04:40.097148] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOL.csv [2025-10-20T22:04:40.097228] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRCS.csv [2025-10-20T22:04:40.097328] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPO.csv [2025-10-20T22:04:40.097409] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GD.csv [2025-10-20T22:04:40.097475] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPYP.csv [2025-10-20T22:04:40.097583] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KTF.csv [2025-10-20T22:04:40.097806] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FDP.csv [2025-10-20T22:04:40.097898] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RERE.csv [2025-10-20T22:04:40.098009] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBU.csv [2025-10-20T22:04:40.098106] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PMO.csv [2025-10-20T22:04:40.098198] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STNG.csv [2025-10-20T22:04:40.098267] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UHT.csv [2025-10-20T22:04:40.098336] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPAC.csv [2025-10-20T22:04:40.098402] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VYX.csv [2025-10-20T22:04:40.098468] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NNI.csv [2025-10-20T22:04:40.098536] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETO.csv [2025-10-20T22:04:40.098606] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CUK.csv [2025-10-20T22:04:40.098753] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LUCK.csv [2025-10-20T22:04:40.098855] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LVS.csv [2025-10-20T22:04:40.098924] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JFR.csv [2025-10-20T22:04:40.098989] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RELX.csv [2025-10-20T22:04:40.099057] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IEX.csv [2025-10-20T22:04:40.099123] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MET.csv [2025-10-20T22:04:40.099189] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UBS.csv [2025-10-20T22:04:40.099264] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EAI.csv [2025-10-20T22:04:40.099328] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SQNS.csv [2025-10-20T22:04:40.099405] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETN.csv [2025-10-20T22:04:40.099483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KVYO.csv [2025-10-20T22:04:40.099588] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IVR.csv [2025-10-20T22:04:40.099684] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VET.csv [2025-10-20T22:04:40.099788] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ENIC.csv [2025-10-20T22:04:40.099856] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SBS.csv [2025-10-20T22:04:40.099949] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTV.csv [2025-10-20T22:04:40.100039] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CII.csv [2025-10-20T22:04:40.100107] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WDI.csv [2025-10-20T22:04:40.100172] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALTG.csv [2025-10-20T22:04:40.100286] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LOCL.csv [2025-10-20T22:04:40.100350] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSE.csv [2025-10-20T22:04:40.100412] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BIT.csv [2025-10-20T22:04:40.100498] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CAL.csv [2025-10-20T22:04:40.100571] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHH.csv [2025-10-20T22:04:40.100634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXC.csv [2025-10-20T22:04:40.100772] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DUKB.csv [2025-10-20T22:04:40.100858] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ED.csv [2025-10-20T22:04:40.100933] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JGH.csv [2025-10-20T22:04:40.101005] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KMT.csv [2025-10-20T22:04:40.101069] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QGEN.csv [2025-10-20T22:04:40.101142] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IBTA.csv [2025-10-20T22:04:40.101205] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/APH.csv [2025-10-20T22:04:40.101282] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NI.csv [2025-10-20T22:04:40.101346] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TWI.csv [2025-10-20T22:04:40.101416] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFAO.csv [2025-10-20T22:04:40.101483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UWMC.csv [2025-10-20T22:04:40.101548] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KFY.csv [2025-10-20T22:04:40.101611] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VEEV.csv [2025-10-20T22:04:40.101711] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOC.csv [2025-10-20T22:04:40.101796] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WES.csv [2025-10-20T22:04:40.101864] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JRS.csv [2025-10-20T22:04:40.101947] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IIPR.csv [2025-10-20T22:04:40.102011] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMHI.csv [2025-10-20T22:04:40.102074] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YETI.csv [2025-10-20T22:04:40.102135] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/U.csv [2025-10-20T22:04:40.102200] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TAC.csv [2025-10-20T22:04:40.102269] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KR.csv [2025-10-20T22:04:40.102346] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JCE.csv [2025-10-20T22:04:40.102418] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALEX.csv [2025-10-20T22:04:40.102481] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNP.csv [2025-10-20T22:04:40.102558] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DOW.csv [2025-10-20T22:04:40.102635] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FBP.csv [2025-10-20T22:04:40.102743] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LEN.csv [2025-10-20T22:04:40.102813] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UAA.csv [2025-10-20T22:04:40.102878] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUA.csv [2025-10-20T22:04:40.102949] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JNJ.csv [2025-10-20T22:04:40.103022] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGY.csv [2025-10-20T22:04:40.103084] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LH.csv [2025-10-20T22:04:40.103145] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KGS.csv [2025-10-20T22:04:40.103221] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOOT.csv [2025-10-20T22:04:40.103301] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AESI.csv [2025-10-20T22:04:40.103401] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATHS.csv [2025-10-20T22:04:40.103495] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DNA.csv [2025-10-20T22:04:40.103560] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CSTM.csv [2025-10-20T22:04:40.103624] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEPH.csv [2025-10-20T22:04:40.103731] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RLX.csv [2025-10-20T22:04:40.103810] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AFGE.csv [2025-10-20T22:04:40.103874] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NAC.csv [2025-10-20T22:04:40.103945] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NVO.csv [2025-10-20T22:04:40.104008] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BTZ.csv [2025-10-20T22:04:40.104075] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ENVA.csv [2025-10-20T22:04:40.104191] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AWF.csv [2025-10-20T22:04:40.104263] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SYY.csv [2025-10-20T22:04:40.104337] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVR.csv [2025-10-20T22:04:40.104405] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFAN.csv [2025-10-20T22:04:40.104467] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZH.csv [2025-10-20T22:04:40.104528] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMAI.csv [2025-10-20T22:04:40.104594] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMP.csv [2025-10-20T22:04:40.104712] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLF.csv [2025-10-20T22:04:40.104803] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SD.csv [2025-10-20T22:04:40.104867] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SKLZ.csv [2025-10-20T22:04:40.104943] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FINS.csv [2025-10-20T22:04:40.105004] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LANV.csv [2025-10-20T22:04:40.105078] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALX.csv [2025-10-20T22:04:40.105144] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TBN.csv [2025-10-20T22:04:40.105212] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFA.csv [2025-10-20T22:04:40.105277] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDO.csv [2025-10-20T22:04:40.105356] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KKR.csv [2025-10-20T22:04:40.105432] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HP.csv [2025-10-20T22:04:40.105496] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PCQ.csv [2025-10-20T22:04:40.105576] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPR.csv [2025-10-20T22:04:40.105799] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RVT.csv [2025-10-20T22:04:40.105889] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BIPJ.csv [2025-10-20T22:04:40.105964] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDL.csv [2025-10-20T22:04:40.106029] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRIM.csv [2025-10-20T22:04:40.106118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EIC.csv [2025-10-20T22:04:40.106206] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CWK.csv [2025-10-20T22:04:40.106273] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PGP.csv [2025-10-20T22:04:40.106337] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RYAM.csv [2025-10-20T22:04:40.106399] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BHC.csv [2025-10-20T22:04:40.106467] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WTI.csv [2025-10-20T22:04:40.106545] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DOLE.csv [2025-10-20T22:04:40.106610] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WHD.csv [2025-10-20T22:04:40.106708] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMCM.csv [2025-10-20T22:04:40.106798] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMO.csv [2025-10-20T22:04:40.106869] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WWW.csv [2025-10-20T22:04:40.106940] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NIO.csv [2025-10-20T22:04:40.107006] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USFD.csv [2025-10-20T22:04:40.107078] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HOV.csv [2025-10-20T22:04:40.107171] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MVO.csv [2025-10-20T22:04:40.107244] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LDP.csv [2025-10-20T22:04:40.107315] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSDL.csv [2025-10-20T22:04:40.107392] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FIGS.csv [2025-10-20T22:04:40.107457] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MG.csv [2025-10-20T22:04:40.107523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ODC.csv [2025-10-20T22:04:40.107587] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JOBY.csv [2025-10-20T22:04:40.107676] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TKR.csv [2025-10-20T22:04:40.107778] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ONL.csv [2025-10-20T22:04:40.107845] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SRI.csv [2025-10-20T22:04:40.107921] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BGSF.csv [2025-10-20T22:04:40.108028] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TNK.csv [2025-10-20T22:04:40.108097] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/M.csv [2025-10-20T22:04:40.108159] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AGX.csv [2025-10-20T22:04:40.108223] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VIST.csv [2025-10-20T22:04:40.108286] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RMAX.csv [2025-10-20T22:04:40.108351] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDC.csv [2025-10-20T22:04:40.108414] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SUPV.csv [2025-10-20T22:04:40.108476] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BAP.csv [2025-10-20T22:04:40.108553] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPR.csv [2025-10-20T22:04:40.108628] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AYI.csv [2025-10-20T22:04:40.108743] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LU.csv [2025-10-20T22:04:40.108823] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPQ.csv [2025-10-20T22:04:40.108900] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALV.csv [2025-10-20T22:04:40.108963] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAXS.csv [2025-10-20T22:04:40.109032] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMT.csv [2025-10-20T22:04:40.109096] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEF.csv [2025-10-20T22:04:40.109167] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PVL.csv [2025-10-20T22:04:40.109230] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DHT.csv [2025-10-20T22:04:40.109302] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IGD.csv [2025-10-20T22:04:40.109367] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LITB.csv [2025-10-20T22:04:40.109430] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SUN.csv [2025-10-20T22:04:40.109492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RSKD.csv [2025-10-20T22:04:40.109599] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UL.csv [2025-10-20T22:04:40.109698] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MATV.csv [2025-10-20T22:04:40.109806] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FOUR.csv [2025-10-20T22:04:40.109871] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NAZ.csv [2025-10-20T22:04:40.109934] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TBI.csv [2025-10-20T22:04:40.110028] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FUL.csv [2025-10-20T22:04:40.110129] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DQ.csv [2025-10-20T22:04:40.110202] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPNG.csv [2025-10-20T22:04:40.110265] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NDMO.csv [2025-10-20T22:04:40.110327] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AZO.csv [2025-10-20T22:04:40.110389] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/COOK.csv [2025-10-20T22:04:40.110452] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YUMC.csv [2025-10-20T22:04:40.110513] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PSBD.csv [2025-10-20T22:04:40.110590] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOC.csv [2025-10-20T22:04:40.110683] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TKO.csv [2025-10-20T22:04:40.110777] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMSC.csv [2025-10-20T22:04:40.110843] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SGHC.csv [2025-10-20T22:04:40.110906] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DX.csv [2025-10-20T22:04:40.110971] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTW.csv [2025-10-20T22:04:40.111033] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WMK.csv [2025-10-20T22:04:40.111111] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SID.csv [2025-10-20T22:04:40.111197] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TFPM.csv [2025-10-20T22:04:40.111261] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPIR.csv [2025-10-20T22:04:40.111333] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KOF.csv [2025-10-20T22:04:40.111395] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GYLD.csv [2025-10-20T22:04:40.111456] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMM.csv [2025-10-20T22:04:40.111531] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMP.csv [2025-10-20T22:04:40.111592] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VMO.csv [2025-10-20T22:04:40.111700] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HE.csv [2025-10-20T22:04:40.111805] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBW.csv [2025-10-20T22:04:40.111891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WKC.csv [2025-10-20T22:04:40.111956] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AOS.csv [2025-10-20T22:04:40.112033] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FOF.csv [2025-10-20T22:04:40.112110] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETR.csv [2025-10-20T22:04:40.112193] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BXMT.csv [2025-10-20T22:04:40.112254] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VAL.csv [2025-10-20T22:04:40.112315] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZVIA.csv [2025-10-20T22:04:40.112375] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEVA.csv [2025-10-20T22:04:40.112451] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOMD.csv [2025-10-20T22:04:40.112525] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVS.csv [2025-10-20T22:04:40.112594] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CL.csv [2025-10-20T22:04:40.112705] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KNF.csv [2025-10-20T22:04:40.112794] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRW.csv [2025-10-20T22:04:40.112859] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SUZ.csv [2025-10-20T22:04:40.112938] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CGV.csv [2025-10-20T22:04:40.113001] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WT.csv [2025-10-20T22:04:40.113081] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DBI.csv [2025-10-20T22:04:40.113146] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECCU.csv [2025-10-20T22:04:40.113219] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ELAN.csv [2025-10-20T22:04:40.113281] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BCAT.csv [2025-10-20T22:04:40.113344] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMS.csv [2025-10-20T22:04:40.113404] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KOP.csv [2025-10-20T22:04:40.113466] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CX.csv [2025-10-20T22:04:40.113525] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WRB.csv [2025-10-20T22:04:40.113589] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NTB.csv [2025-10-20T22:04:40.113786] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PNNT.csv [2025-10-20T22:04:40.113880] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SXT.csv [2025-10-20T22:04:40.113951] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GTES.csv [2025-10-20T22:04:40.114031] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPRK.csv [2025-10-20T22:04:40.114104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVF.csv [2025-10-20T22:04:40.114183] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMN.csv [2025-10-20T22:04:40.114251] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZTS.csv [2025-10-20T22:04:40.114321] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DRD.csv [2025-10-20T22:04:40.114399] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRC.csv [2025-10-20T22:04:40.114465] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRN.csv [2025-10-20T22:04:40.114545] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPN.csv [2025-10-20T22:04:40.114608] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BURL.csv [2025-10-20T22:04:40.114717] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HTD.csv [2025-10-20T22:04:40.114816] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHG.csv [2025-10-20T22:04:40.114893] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LLY.csv [2025-10-20T22:04:40.114962] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CALX.csv [2025-10-20T22:04:40.115076] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIW.csv [2025-10-20T22:04:40.115157] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCO.csv [2025-10-20T22:04:40.115224] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SREA.csv [2025-10-20T22:04:40.115289] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDG.csv [2025-10-20T22:04:40.115354] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PBR.csv [2025-10-20T22:04:40.115416] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CSR.csv [2025-10-20T22:04:40.115492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATEN.csv [2025-10-20T22:04:40.115557] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VMC.csv [2025-10-20T22:04:40.115634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STXV.csv [2025-10-20T22:04:40.115757] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GLOB.csv [2025-10-20T22:04:40.115867] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVA.csv [2025-10-20T22:04:40.115935] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YMM.csv [2025-10-20T22:04:40.116000] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YALA.csv [2025-10-20T22:04:40.116072] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IP.csv [2025-10-20T22:04:40.116135] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVY.csv [2025-10-20T22:04:40.116198] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNO.csv [2025-10-20T22:04:40.116263] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NEXA.csv [2025-10-20T22:04:40.116326] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEPC.csv [2025-10-20T22:04:40.116397] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/K.csv [2025-10-20T22:04:40.116475] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HDB.csv [2025-10-20T22:04:40.116556] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QTWO.csv [2025-10-20T22:04:40.116622] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOK.csv [2025-10-20T22:04:40.116730] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPCE.csv [2025-10-20T22:04:40.116812] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LOAR.csv [2025-10-20T22:04:40.116886] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPF.csv [2025-10-20T22:04:40.116963] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EFX.csv [2025-10-20T22:04:40.117030] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLIO.csv [2025-10-20T22:04:40.117101] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FMX.csv [2025-10-20T22:04:40.117181] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RTO.csv [2025-10-20T22:04:40.117284] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOA.csv [2025-10-20T22:04:40.117360] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JLS.csv [2025-10-20T22:04:40.117426] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TM.csv [2025-10-20T22:04:40.117504] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ERJ.csv [2025-10-20T22:04:40.117568] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WNC.csv [2025-10-20T22:04:40.117631] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPLX.csv [2025-10-20T22:04:40.117744] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PX.csv [2025-10-20T22:04:40.117817] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MED.csv [2025-10-20T22:04:40.117880] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPE.csv [2025-10-20T22:04:40.117967] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VIK.csv [2025-10-20T22:04:40.118066] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INGR.csv [2025-10-20T22:04:40.118145] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FFWM.csv [2025-10-20T22:04:40.118214] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPF.csv [2025-10-20T22:04:40.118276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSI.csv [2025-10-20T22:04:40.118340] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MDU.csv [2025-10-20T22:04:40.118402] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTMX.csv [2025-10-20T22:04:40.118478] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HBM.csv [2025-10-20T22:04:40.118548] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BUR.csv [2025-10-20T22:04:40.118625] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECG.csv [2025-10-20T22:04:40.118743] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EFT.csv [2025-10-20T22:04:40.118829] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CANG.csv [2025-10-20T22:04:40.118895] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TTI.csv [2025-10-20T22:04:40.118958] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UIS.csv [2025-10-20T22:04:40.119022] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LII.csv [2025-10-20T22:04:40.119096] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INVH.csv [2025-10-20T22:04:40.119173] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPRT.csv [2025-10-20T22:04:40.119242] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GCI.csv [2025-10-20T22:04:40.119306] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIO.csv [2025-10-20T22:04:40.119368] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOTE.csv [2025-10-20T22:04:40.119429] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SABA.csv [2025-10-20T22:04:40.119490] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VSH.csv [2025-10-20T22:04:40.119554] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PCOR.csv [2025-10-20T22:04:40.119631] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVK.csv [2025-10-20T22:04:40.119748] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHLD.csv [2025-10-20T22:04:40.119842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DDD.csv [2025-10-20T22:04:40.119921] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESE.csv [2025-10-20T22:04:40.120000] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GE.csv [2025-10-20T22:04:40.120070] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HGV.csv [2025-10-20T22:04:40.120149] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KGC.csv [2025-10-20T22:04:40.120224] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FT.csv [2025-10-20T22:04:40.120287] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MQY.csv [2025-10-20T22:04:40.120363] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GOOS.csv [2025-10-20T22:04:40.120466] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNMA.csv [2025-10-20T22:04:40.120541] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BSAC.csv [2025-10-20T22:04:40.120633] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAR.csv [2025-10-20T22:04:40.120754] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EB.csv [2025-10-20T22:04:40.120825] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SKIL.csv [2025-10-20T22:04:40.120888] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSM.csv [2025-10-20T22:04:40.120950] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BFH.csv [2025-10-20T22:04:40.121014] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RIV.csv [2025-10-20T22:04:40.121086] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTOL.csv [2025-10-20T22:04:40.121178] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATMU.csv [2025-10-20T22:04:40.121264] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AXL.csv [2025-10-20T22:04:40.121329] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MLI.csv [2025-10-20T22:04:40.121391] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUC.csv [2025-10-20T22:04:40.121464] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GWRE.csv [2025-10-20T22:04:40.121531] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SFL.csv [2025-10-20T22:04:40.121592] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WINN.csv [2025-10-20T22:04:40.121809] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GBAB.csv [2025-10-20T22:04:40.121913] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIV.csv [2025-10-20T22:04:40.121980] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GEL.csv [2025-10-20T22:04:40.122056] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EHI.csv [2025-10-20T22:04:40.122118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VGM.csv [2025-10-20T22:04:40.122180] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMTD.csv [2025-10-20T22:04:40.122256] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASX.csv [2025-10-20T22:04:40.122317] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WOLF.csv [2025-10-20T22:04:40.122389] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ELC.csv [2025-10-20T22:04:40.122463] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CWH.csv [2025-10-20T22:04:40.122525] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SNN.csv [2025-10-20T22:04:40.122584] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LCII.csv [2025-10-20T22:04:40.122679] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOS.csv [2025-10-20T22:04:40.122782] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KNOP.csv [2025-10-20T22:04:40.122853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZIM.csv [2025-10-20T22:04:40.122934] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MBC.csv [2025-10-20T22:04:40.123009] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLF.csv [2025-10-20T22:04:40.123071] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BMI.csv [2025-10-20T22:04:40.123135] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAIN.csv [2025-10-20T22:04:40.123200] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TAK.csv [2025-10-20T22:04:40.123265] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEAF.csv [2025-10-20T22:04:40.123342] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FSM.csv [2025-10-20T22:04:40.123425] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ELS.csv [2025-10-20T22:04:40.123493] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WH.csv [2025-10-20T22:04:40.123557] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPV.csv [2025-10-20T22:04:40.123621] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TGNA.csv [2025-10-20T22:04:40.123734] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CC.csv [2025-10-20T22:04:40.123814] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMWB.csv [2025-10-20T22:04:40.123944] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EBF.csv [2025-10-20T22:04:40.124011] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HOUS.csv [2025-10-20T22:04:40.124091] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FRO.csv [2025-10-20T22:04:40.124157] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOG.csv [2025-10-20T22:04:40.124235] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPRI.csv [2025-10-20T22:04:40.124301] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LPL.csv [2025-10-20T22:04:40.124377] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EDN.csv [2025-10-20T22:04:40.124464] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GOTU.csv [2025-10-20T22:04:40.124528] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LAD.csv [2025-10-20T22:04:40.124594] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BSTZ.csv [2025-10-20T22:04:40.124707] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOX.csv [2025-10-20T22:04:40.124798] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STEW.csv [2025-10-20T22:04:40.124866] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DIN.csv [2025-10-20T22:04:40.124931] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JOF.csv [2025-10-20T22:04:40.124996] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THR.csv [2025-10-20T22:04:40.125058] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NE.csv [2025-10-20T22:04:40.125122] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WHR.csv [2025-10-20T22:04:40.125210] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DSX.csv [2025-10-20T22:04:40.125276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SCM.csv [2025-10-20T22:04:40.125351] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AM.csv [2025-10-20T22:04:40.125415] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LEG.csv [2025-10-20T22:04:40.125480] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MNSO.csv [2025-10-20T22:04:40.125556] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ARW.csv [2025-10-20T22:04:40.125633] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHM.csv [2025-10-20T22:04:40.125759] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EW.csv [2025-10-20T22:04:40.125844] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPWR.csv [2025-10-20T22:04:40.125920] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FGN.csv [2025-10-20T22:04:40.126032] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BTO.csv [2025-10-20T22:04:40.126104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DD.csv [2025-10-20T22:04:40.126181] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CWEN.csv [2025-10-20T22:04:40.126246] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TWLO.csv [2025-10-20T22:04:40.126309] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRNO.csv [2025-10-20T22:04:40.126371] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PBT.csv [2025-10-20T22:04:40.126434] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UDR.csv [2025-10-20T22:04:40.126514] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETG.csv [2025-10-20T22:04:40.126589] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CAAP.csv [2025-10-20T22:04:40.126683] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GSL.csv [2025-10-20T22:04:40.126785] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMPY.csv [2025-10-20T22:04:40.126854] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ISD.csv [2025-10-20T22:04:40.126919] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THG.csv [2025-10-20T22:04:40.126993] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETB.csv [2025-10-20T22:04:40.127056] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AROC.csv [2025-10-20T22:04:40.127138] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNK.csv [2025-10-20T22:04:40.127201] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MOV.csv [2025-10-20T22:04:40.127263] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RYI.csv [2025-10-20T22:04:40.127356] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FE.csv [2025-10-20T22:04:40.127436] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STXK.csv [2025-10-20T22:04:40.127502] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RDY.csv [2025-10-20T22:04:40.127578] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IFF.csv [2025-10-20T22:04:40.127690] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AGM.csv [2025-10-20T22:04:40.127779] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UTI.csv [2025-10-20T22:04:40.127853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMP.csv [2025-10-20T22:04:40.127917] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SCS.csv [2025-10-20T22:04:40.128030] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RCS.csv [2025-10-20T22:04:40.128113] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GAM.csv [2025-10-20T22:04:40.128177] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QVCC.csv [2025-10-20T22:04:40.128239] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNT.csv [2025-10-20T22:04:40.128311] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFSI.csv [2025-10-20T22:04:40.128390] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/COF.csv [2025-10-20T22:04:40.128454] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VIV.csv [2025-10-20T22:04:40.128516] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNH.csv [2025-10-20T22:04:40.128577] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CXM.csv [2025-10-20T22:04:40.128642] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRL.csv [2025-10-20T22:04:40.128787] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TGT.csv [2025-10-20T22:04:40.128856] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CACI.csv [2025-10-20T22:04:40.128920] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PERF.csv [2025-10-20T22:04:40.128998] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DV.csv [2025-10-20T22:04:40.129060] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BG.csv [2025-10-20T22:04:40.129122] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AFGC.csv [2025-10-20T22:04:40.129190] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPCT.csv [2025-10-20T22:04:40.129265] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALSN.csv [2025-10-20T22:04:40.129327] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UPS.csv [2025-10-20T22:04:40.129389] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNC.csv [2025-10-20T22:04:40.129465] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IQV.csv [2025-10-20T22:04:40.129541] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESI.csv [2025-10-20T22:04:40.129622] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCJ.csv [2025-10-20T22:04:40.129835] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TG.csv [2025-10-20T22:04:40.129912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BSM.csv [2025-10-20T22:04:40.129978] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WDH.csv [2025-10-20T22:04:40.130039] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBNA.csv [2025-10-20T22:04:40.130115] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/G.csv [2025-10-20T22:04:40.130179] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRT.csv [2025-10-20T22:04:40.130241] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VCV.csv [2025-10-20T22:04:40.130334] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESNT.csv [2025-10-20T22:04:40.130435] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNH.csv [2025-10-20T22:04:40.130567] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOE.csv [2025-10-20T22:04:40.130665] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRNT.csv [2025-10-20T22:04:40.130769] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETW.csv [2025-10-20T22:04:40.130836] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INSW.csv [2025-10-20T22:04:40.130899] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VEL.csv [2025-10-20T22:04:40.130972] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GS.csv [2025-10-20T22:04:40.131050] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CARS.csv [2025-10-20T22:04:40.131128] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AEO.csv [2025-10-20T22:04:40.131203] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IPG.csv [2025-10-20T22:04:40.131268] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPC.csv [2025-10-20T22:04:40.131329] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LFT.csv [2025-10-20T22:04:40.131393] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPI.csv [2025-10-20T22:04:40.131470] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPJA.csv [2025-10-20T22:04:40.131576] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKU.csv [2025-10-20T22:04:40.131689] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IIIN.csv [2025-10-20T22:04:40.131783] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAC.csv [2025-10-20T22:04:40.131853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TTE.csv [2025-10-20T22:04:40.131919] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TK.csv [2025-10-20T22:04:40.131983] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DEO.csv [2025-10-20T22:04:40.132046] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLD.csv [2025-10-20T22:04:40.132123] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GLP.csv [2025-10-20T22:04:40.132216] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KBH.csv [2025-10-20T22:04:40.132282] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USAC.csv [2025-10-20T22:04:40.132353] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BYM.csv [2025-10-20T22:04:40.132430] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLT.csv [2025-10-20T22:04:40.132505] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDT.csv [2025-10-20T22:04:40.132571] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSM.csv [2025-10-20T22:04:40.132633] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBDO.csv [2025-10-20T22:04:40.132762] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALLY.csv [2025-10-20T22:04:40.132842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KBR.csv [2025-10-20T22:04:40.132908] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HOG.csv [2025-10-20T22:04:40.132974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZKH.csv [2025-10-20T22:04:40.133038] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RQI.csv [2025-10-20T22:04:40.133105] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/T.csv [2025-10-20T22:04:40.133171] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OBK.csv [2025-10-20T22:04:40.133249] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HZO.csv [2025-10-20T22:04:40.133314] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STRV.csv [2025-10-20T22:04:40.133386] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLX.csv [2025-10-20T22:04:40.133461] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FXED.csv [2025-10-20T22:04:40.133526] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THQ.csv [2025-10-20T22:04:40.133602] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCU.csv [2025-10-20T22:04:40.133803] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GWH.csv [2025-10-20T22:04:40.133914] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMD.csv [2025-10-20T22:04:40.133994] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DOUG.csv [2025-10-20T22:04:40.134059] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MEGI.csv [2025-10-20T22:04:40.134137] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CE.csv [2025-10-20T22:04:40.134200] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LMND.csv [2025-10-20T22:04:40.134262] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LNN.csv [2025-10-20T22:04:40.134336] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMG.csv [2025-10-20T22:04:40.134399] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PML.csv [2025-10-20T22:04:40.134461] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIG.csv [2025-10-20T22:04:40.134525] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUE.csv [2025-10-20T22:04:40.134588] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SCCO.csv [2025-10-20T22:04:40.134691] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FFC.csv [2025-10-20T22:04:40.134782] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPK.csv [2025-10-20T22:04:40.134852] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEX.csv [2025-10-20T22:04:40.134917] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PTA.csv [2025-10-20T22:04:40.134992] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMC.csv [2025-10-20T22:04:40.135056] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SES.csv [2025-10-20T22:04:40.135122] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVAL.csv [2025-10-20T22:04:40.135185] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NGS.csv [2025-10-20T22:04:40.135259] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AFL.csv [2025-10-20T22:04:40.135325] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SWX.csv [2025-10-20T22:04:40.135442] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GSBD.csv [2025-10-20T22:04:40.135505] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NEE.csv [2025-10-20T22:04:40.135566] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STXE.csv [2025-10-20T22:04:40.135633] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KEP.csv [2025-10-20T22:04:40.135754] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LDOS.csv [2025-10-20T22:04:40.135826] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RFMZ.csv [2025-10-20T22:04:40.135912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EC.csv [2025-10-20T22:04:40.135978] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LPX.csv [2025-10-20T22:04:40.136042] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOBO.csv [2025-10-20T22:04:40.136102] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MEC.csv [2025-10-20T22:04:40.136164] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CI.csv [2025-10-20T22:04:40.136227] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VOYA.csv [2025-10-20T22:04:40.136298] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NEU.csv [2025-10-20T22:04:40.136368] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LXP.csv [2025-10-20T22:04:40.136443] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NFG.csv [2025-10-20T22:04:40.136515] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ENR.csv [2025-10-20T22:04:40.136579] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KMX.csv [2025-10-20T22:04:40.136670] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VST.csv [2025-10-20T22:04:40.136763] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRO.csv [2025-10-20T22:04:40.136829] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZK.csv [2025-10-20T22:04:40.136895] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNFI.csv [2025-10-20T22:04:40.137013] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/E.csv [2025-10-20T22:04:40.137104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NET.csv [2025-10-20T22:04:40.137173] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GRC.csv [2025-10-20T22:04:40.137236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TR.csv [2025-10-20T22:04:40.137297] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RM.csv [2025-10-20T22:04:40.137373] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HGER.csv [2025-10-20T22:04:40.137443] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KEX.csv [2025-10-20T22:04:40.137505] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKT.csv [2025-10-20T22:04:40.137569] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ING.csv [2025-10-20T22:04:40.137666] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ARMK.csv [2025-10-20T22:04:40.137774] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPE.csv [2025-10-20T22:04:40.137840] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SGU.csv [2025-10-20T22:04:40.137902] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OWLT.csv [2025-10-20T22:04:40.138022] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BXMX.csv [2025-10-20T22:04:40.138101] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WTTR.csv [2025-10-20T22:04:40.138165] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VRTS.csv [2025-10-20T22:04:40.138227] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAN.csv [2025-10-20T22:04:40.138315] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOW.csv [2025-10-20T22:04:40.138417] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KFRC.csv [2025-10-20T22:04:40.138482] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PSF.csv [2025-10-20T22:04:40.138559] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GME.csv [2025-10-20T22:04:40.138635] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HCC.csv [2025-10-20T22:04:40.138752] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BGB.csv [2025-10-20T22:04:40.138821] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASGI.csv [2025-10-20T22:04:40.138884] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UCB.csv [2025-10-20T22:04:40.138945] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VLT.csv [2025-10-20T22:04:40.139019] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGRE.csv [2025-10-20T22:04:40.139080] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USPH.csv [2025-10-20T22:04:40.139155] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCL.csv [2025-10-20T22:04:40.139230] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DKS.csv [2025-10-20T22:04:40.139293] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BUI.csv [2025-10-20T22:04:40.139354] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFH.csv [2025-10-20T22:04:40.139428] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETX.csv [2025-10-20T22:04:40.139492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IRT.csv [2025-10-20T22:04:40.139568] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ACN.csv [2025-10-20T22:04:40.139628] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEPJ.csv [2025-10-20T22:04:40.139759] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IHD.csv [2025-10-20T22:04:40.139846] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KD.csv [2025-10-20T22:04:40.139916] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNJ.csv [2025-10-20T22:04:40.139991] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GOLF.csv [2025-10-20T22:04:40.140052] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MIR.csv [2025-10-20T22:04:40.140128] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NATL.csv [2025-10-20T22:04:40.140188] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOW.csv [2025-10-20T22:04:40.140247] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SITE.csv [2025-10-20T22:04:40.140322] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AG.csv [2025-10-20T22:04:40.140387] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIF.csv [2025-10-20T22:04:40.140464] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KRG.csv [2025-10-20T22:04:40.140558] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GEO.csv [2025-10-20T22:04:40.140637] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KIO.csv [2025-10-20T22:04:40.140758] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KN.csv [2025-10-20T22:04:40.140842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FCT.csv [2025-10-20T22:04:40.140921] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GATX.csv [2025-10-20T22:04:40.140987] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOJC.csv [2025-10-20T22:04:40.141054] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WEC.csv [2025-10-20T22:04:40.141138] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HD.csv [2025-10-20T22:04:40.141204] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NAN.csv [2025-10-20T22:04:40.141270] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WGO.csv [2025-10-20T22:04:40.141341] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CARR.csv [2025-10-20T22:04:40.141414] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JHX.csv [2025-10-20T22:04:40.141491] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GKOS.csv [2025-10-20T22:04:40.141554] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/QVCD.csv [2025-10-20T22:04:40.141619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSD.csv [2025-10-20T22:04:40.141831] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMD.csv [2025-10-20T22:04:40.141921] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ICL.csv [2025-10-20T22:04:40.141991] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WSR.csv [2025-10-20T22:04:40.142077] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DB.csv [2025-10-20T22:04:40.142154] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ADNT.csv [2025-10-20T22:04:40.142225] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MQT.csv [2025-10-20T22:04:40.142305] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVTR.csv [2025-10-20T22:04:40.142369] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INGM.csv [2025-10-20T22:04:40.142446] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GLW.csv [2025-10-20T22:04:40.142552] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MITT.csv [2025-10-20T22:04:40.142670] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EQT.csv [2025-10-20T22:04:40.142772] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALE.csv [2025-10-20T22:04:40.142842] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MOD.csv [2025-10-20T22:04:40.142912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HBI.csv [2025-10-20T22:04:40.142976] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NIE.csv [2025-10-20T22:04:40.143043] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPS.csv [2025-10-20T22:04:40.143111] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPL.csv [2025-10-20T22:04:40.143181] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KNSL.csv [2025-10-20T22:04:40.143244] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STVN.csv [2025-10-20T22:04:40.143306] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CON.csv [2025-10-20T22:04:40.143371] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WAL.csv [2025-10-20T22:04:40.143444] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BTA.csv [2025-10-20T22:04:40.143528] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LND.csv [2025-10-20T22:04:40.143614] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GFF.csv [2025-10-20T22:04:40.143728] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JILL.csv [2025-10-20T22:04:40.143815] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SB.csv [2025-10-20T22:04:40.143897] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JMIA.csv [2025-10-20T22:04:40.143961] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GIC.csv [2025-10-20T22:04:40.144027] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UMC.csv [2025-10-20T22:04:40.144089] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ONIT.csv [2025-10-20T22:04:40.144153] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TGS.csv [2025-10-20T22:04:40.144218] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TU.csv [2025-10-20T22:04:40.144281] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RIO.csv [2025-10-20T22:04:40.144356] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRBD.csv [2025-10-20T22:04:40.144422] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IGI.csv [2025-10-20T22:04:40.144497] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASPN.csv [2025-10-20T22:04:40.144575] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HMY.csv [2025-10-20T22:04:40.144639] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TV.csv [2025-10-20T22:04:40.144770] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TBB.csv [2025-10-20T22:04:40.144838] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BCH.csv [2025-10-20T22:04:40.144902] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RCD.csv [2025-10-20T22:04:40.144966] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SNDA.csv [2025-10-20T22:04:40.145041] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCK.csv [2025-10-20T22:04:40.145106] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CAH.csv [2025-10-20T22:04:40.145186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HL.csv [2025-10-20T22:04:40.145265] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FND.csv [2025-10-20T22:04:40.145341] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FRT.csv [2025-10-20T22:04:40.145404] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TISI.csv [2025-10-20T22:04:40.145468] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SRFM.csv [2025-10-20T22:04:40.145530] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BMO.csv [2025-10-20T22:04:40.145596] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PKST.csv [2025-10-20T22:04:40.145699] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMH.csv [2025-10-20T22:04:40.145787] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KNTK.csv [2025-10-20T22:04:40.145866] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATI.csv [2025-10-20T22:04:40.145929] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPC.csv [2025-10-20T22:04:40.145997] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNM.csv [2025-10-20T22:04:40.146061] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IDA.csv [2025-10-20T22:04:40.146136] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNI.csv [2025-10-20T22:04:40.146212] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETD.csv [2025-10-20T22:04:40.146351] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MHK.csv [2025-10-20T22:04:40.146414] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TALO.csv [2025-10-20T22:04:40.146480] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CDLR.csv [2025-10-20T22:04:40.146543] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TKC.csv [2025-10-20T22:04:40.146605] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SKYH.csv [2025-10-20T22:04:40.146692] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MXF.csv [2025-10-20T22:04:40.146795] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FIHL.csv [2025-10-20T22:04:40.146861] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DKL.csv [2025-10-20T22:04:40.146924] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MVT.csv [2025-10-20T22:04:40.146985] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BHR.csv [2025-10-20T22:04:40.147046] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDF.csv [2025-10-20T22:04:40.147109] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZIP.csv [2025-10-20T22:04:40.147170] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SHCO.csv [2025-10-20T22:04:40.147266] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HY.csv [2025-10-20T22:04:40.147394] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSGS.csv [2025-10-20T22:04:40.147483] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TOST.csv [2025-10-20T22:04:40.147561] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BOND.csv [2025-10-20T22:04:40.147624] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AON.csv [2025-10-20T22:04:40.147728] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CXW.csv [2025-10-20T22:04:40.147803] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NLOP.csv [2025-10-20T22:04:40.147866] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RPT.csv [2025-10-20T22:04:40.147992] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KSS.csv [2025-10-20T22:04:40.148078] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JEF.csv [2025-10-20T22:04:40.148177] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NQP.csv [2025-10-20T22:04:40.148258] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBY.csv [2025-10-20T22:04:40.148323] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VRE.csv [2025-10-20T22:04:40.148385] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMTM.csv [2025-10-20T22:04:40.148460] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PBA.csv [2025-10-20T22:04:40.148538] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LBRT.csv [2025-10-20T22:04:40.148601] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BAK.csv [2025-10-20T22:04:40.148699] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RMI.csv [2025-10-20T22:04:40.148791] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDX.csv [2025-10-20T22:04:40.148853] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VHI.csv [2025-10-20T22:04:40.148932] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FI.csv [2025-10-20T22:04:40.148996] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAO.csv [2025-10-20T22:04:40.149058] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YOU.csv [2025-10-20T22:04:40.149118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ONTF.csv [2025-10-20T22:04:40.149179] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBAR.csv [2025-10-20T22:04:40.149253] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ENOV.csv [2025-10-20T22:04:40.149313] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KORE.csv [2025-10-20T22:04:40.149388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EBS.csv [2025-10-20T22:04:40.149449] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VVV.csv [2025-10-20T22:04:40.149512] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PSFE.csv [2025-10-20T22:04:40.149574] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TEN.csv [2025-10-20T22:04:40.149634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YEXT.csv [2025-10-20T22:04:40.149822] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HNI.csv [2025-10-20T22:04:40.149946] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KMI.csv [2025-10-20T22:04:40.150024] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHCT.csv [2025-10-20T22:04:40.150086] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KTB.csv [2025-10-20T22:04:40.150148] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IIF.csv [2025-10-20T22:04:40.150248] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVEX.csv [2025-10-20T22:04:40.150316] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PACS.csv [2025-10-20T22:04:40.150386] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAN.csv [2025-10-20T22:04:40.150451] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RMMZ.csv [2025-10-20T22:04:40.150519] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PINE.csv [2025-10-20T22:04:40.150584] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GXO.csv [2025-10-20T22:04:40.150669] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMF.csv [2025-10-20T22:04:40.150761] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STXD.csv [2025-10-20T22:04:40.150827] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BA.csv [2025-10-20T22:04:40.150897] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YUM.csv [2025-10-20T22:04:40.150963] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VLN.csv [2025-10-20T22:04:40.151035] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SQM.csv [2025-10-20T22:04:40.151106] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FIX.csv [2025-10-20T22:04:40.151175] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLNG.csv [2025-10-20T22:04:40.151252] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DLY.csv [2025-10-20T22:04:40.151317] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFL.csv [2025-10-20T22:04:40.151380] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MOS.csv [2025-10-20T22:04:40.151458] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FSLY.csv [2025-10-20T22:04:40.151532] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GWW.csv [2025-10-20T22:04:40.151616] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NVG.csv [2025-10-20T22:04:40.151736] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XFLT.csv [2025-10-20T22:04:40.151827] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DUK.csv [2025-10-20T22:04:40.151899] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GIB.csv [2025-10-20T22:04:40.151968] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HTH.csv [2025-10-20T22:04:40.152045] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HAL.csv [2025-10-20T22:04:40.152111] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/USNA.csv [2025-10-20T22:04:40.152176] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRV.csv [2025-10-20T22:04:40.152241] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SYK.csv [2025-10-20T22:04:40.152306] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RNP.csv [2025-10-20T22:04:40.152382] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FET.csv [2025-10-20T22:04:40.152462] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FMN.csv [2025-10-20T22:04:40.152537] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMCR.csv [2025-10-20T22:04:40.152613] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ADX.csv [2025-10-20T22:04:40.152748] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CFG.csv [2025-10-20T22:04:40.152831] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THIR.csv [2025-10-20T22:04:40.152897] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OPAD.csv [2025-10-20T22:04:40.152960] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BCE.csv [2025-10-20T22:04:40.153027] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HSHP.csv [2025-10-20T22:04:40.153092] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KRP.csv [2025-10-20T22:04:40.153156] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LYG.csv [2025-10-20T22:04:40.153228] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MKL.csv [2025-10-20T22:04:40.153303] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IDT.csv [2025-10-20T22:04:40.153415] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/W.csv [2025-10-20T22:04:40.153487] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WU.csv [2025-10-20T22:04:40.153590] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASAN.csv [2025-10-20T22:04:40.153698] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FCN.csv [2025-10-20T22:04:40.153789] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CXT.csv [2025-10-20T22:04:40.153861] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLH.csv [2025-10-20T22:04:40.153929] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RLJ.csv [2025-10-20T22:04:40.154002] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WTS.csv [2025-10-20T22:04:40.154070] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HUBB.csv [2025-10-20T22:04:40.154135] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TUYA.csv [2025-10-20T22:04:40.154198] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PUK.csv [2025-10-20T22:04:40.154260] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BSBR.csv [2025-10-20T22:04:40.154323] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MSGE.csv [2025-10-20T22:04:40.154385] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RCI.csv [2025-10-20T22:04:40.154461] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FAF.csv [2025-10-20T22:04:40.154526] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KAR.csv [2025-10-20T22:04:40.154601] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HGLB.csv [2025-10-20T22:04:40.154713] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HMC.csv [2025-10-20T22:04:40.154819] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GAP.csv [2025-10-20T22:04:40.154900] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/F.csv [2025-10-20T22:04:40.154968] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PSQH.csv [2025-10-20T22:04:40.155033] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UVE.csv [2025-10-20T22:04:40.155108] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EEX.csv [2025-10-20T22:04:40.155185] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRM.csv [2025-10-20T22:04:40.155253] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HRTG.csv [2025-10-20T22:04:40.155316] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HRI.csv [2025-10-20T22:04:40.155378] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SXI.csv [2025-10-20T22:04:40.155442] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MOH.csv [2025-10-20T22:04:40.155537] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBT.csv [2025-10-20T22:04:40.155612] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GSK.csv [2025-10-20T22:04:40.155726] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AVNT.csv [2025-10-20T22:04:40.155808] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TCBX.csv [2025-10-20T22:04:40.155878] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HWM.csv [2025-10-20T22:04:40.155942] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HSY.csv [2025-10-20T22:04:40.156005] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PTY.csv [2025-10-20T22:04:40.156140] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMSA.csv [2025-10-20T22:04:40.156211] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALG.csv [2025-10-20T22:04:40.156275] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IPI.csv [2025-10-20T22:04:40.156337] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NRT.csv [2025-10-20T22:04:40.156400] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LOW.csv [2025-10-20T22:04:40.156462] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/V.csv [2025-10-20T22:04:40.156523] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTDR.csv [2025-10-20T22:04:40.156584] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SDHY.csv [2025-10-20T22:04:40.156675] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TY.csv [2025-10-20T22:04:40.156782] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HOMB.csv [2025-10-20T22:04:40.156854] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WEA.csv [2025-10-20T22:04:40.156920] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLW.csv [2025-10-20T22:04:40.156983] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KYN.csv [2025-10-20T22:04:40.157050] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCS.csv [2025-10-20T22:04:40.157110] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEPI.csv [2025-10-20T22:04:40.157173] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LYV.csv [2025-10-20T22:04:40.157236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPXX.csv [2025-10-20T22:04:40.157313] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMCO.csv [2025-10-20T22:04:40.157388] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FRGE.csv [2025-10-20T22:04:40.157451] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XPEV.csv [2025-10-20T22:04:40.157512] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YRD.csv [2025-10-20T22:04:40.157587] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESRT.csv [2025-10-20T22:04:40.157801] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KW.csv [2025-10-20T22:04:40.157891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PMM.csv [2025-10-20T22:04:40.157998] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ENB.csv [2025-10-20T22:04:40.158090] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ITW.csv [2025-10-20T22:04:40.158155] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SVV.csv [2025-10-20T22:04:40.158218] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SLF.csv [2025-10-20T22:04:40.158278] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPV.csv [2025-10-20T22:04:40.158352] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTX.csv [2025-10-20T22:04:40.158415] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DLB.csv [2025-10-20T22:04:40.158478] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WSM.csv [2025-10-20T22:04:40.158539] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ANF.csv [2025-10-20T22:04:40.158602] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MGM.csv [2025-10-20T22:04:40.158693] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ANET.csv [2025-10-20T22:04:40.158800] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THC.csv [2025-10-20T22:04:40.158934] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BUXX.csv [2025-10-20T22:04:40.159014] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLN.csv [2025-10-20T22:04:40.159077] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WBS.csv [2025-10-20T22:04:40.159138] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZEPP.csv [2025-10-20T22:04:40.159217] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIA.csv [2025-10-20T22:04:40.159278] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UNM.csv [2025-10-20T22:04:40.159339] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VLRS.csv [2025-10-20T22:04:40.159401] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OEC.csv [2025-10-20T22:04:40.159472] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPKI.csv [2025-10-20T22:04:40.159543] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IT.csv [2025-10-20T22:04:40.159604] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DBRG.csv [2025-10-20T22:04:40.159699] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVTC.csv [2025-10-20T22:04:40.159790] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CVE.csv [2025-10-20T22:04:40.159862] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCI.csv [2025-10-20T22:04:40.159932] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CATO.csv [2025-10-20T22:04:40.159999] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IX.csv [2025-10-20T22:04:40.160062] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/L.csv [2025-10-20T22:04:40.160138] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EGY.csv [2025-10-20T22:04:40.160201] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BAX.csv [2025-10-20T22:04:40.160276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MYD.csv [2025-10-20T22:04:40.160341] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDS.csv [2025-10-20T22:04:40.160467] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AR.csv [2025-10-20T22:04:40.160545] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NBR.csv [2025-10-20T22:04:40.160609] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPAY.csv [2025-10-20T22:04:40.160722] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SEG.csv [2025-10-20T22:04:40.160809] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIEN.csv [2025-10-20T22:04:40.160875] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBD.csv [2025-10-20T22:04:40.160939] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BRCC.csv [2025-10-20T22:04:40.161009] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AWI.csv [2025-10-20T22:04:40.161084] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMC.csv [2025-10-20T22:04:40.161146] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMU.csv [2025-10-20T22:04:40.161223] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GEF.csv [2025-10-20T22:04:40.161298] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNS.csv [2025-10-20T22:04:40.161367] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MNR.csv [2025-10-20T22:04:40.161429] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CMRE.csv [2025-10-20T22:04:40.161492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DLR.csv [2025-10-20T22:04:40.161565] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GM.csv [2025-10-20T22:04:40.161662] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JCI.csv [2025-10-20T22:04:40.161757] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RFL.csv [2025-10-20T22:04:40.161932] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXG.csv [2025-10-20T22:04:40.162003] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIO.csv [2025-10-20T22:04:40.162081] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HAYW.csv [2025-10-20T22:04:40.162145] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MP.csv [2025-10-20T22:04:40.162212] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLUT.csv [2025-10-20T22:04:40.162276] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUFG.csv [2025-10-20T22:04:40.162340] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LEO.csv [2025-10-20T22:04:40.162404] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/A.csv [2025-10-20T22:04:40.162481] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPT.csv [2025-10-20T22:04:40.162544] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UZE.csv [2025-10-20T22:04:40.162605] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CXH.csv [2025-10-20T22:04:40.162693] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NRP.csv [2025-10-20T22:04:40.162790] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NEA.csv [2025-10-20T22:04:40.162857] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAS.csv [2025-10-20T22:04:40.162922] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LVWR.csv [2025-10-20T22:04:40.162987] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INSP.csv [2025-10-20T22:04:40.163049] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEP.csv [2025-10-20T22:04:40.163113] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTS.csv [2025-10-20T22:04:40.163187] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GBTG.csv [2025-10-20T22:04:40.163248] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TWN.csv [2025-10-20T22:04:40.163323] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CGAU.csv [2025-10-20T22:04:40.163386] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CNH.csv [2025-10-20T22:04:40.163461] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IHS.csv [2025-10-20T22:04:40.163524] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WLK.csv [2025-10-20T22:04:40.163586] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XPOF.csv [2025-10-20T22:04:40.163672] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BME.csv [2025-10-20T22:04:40.163761] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DDT.csv [2025-10-20T22:04:40.163835] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GTLS.csv [2025-10-20T22:04:40.163898] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CYD.csv [2025-10-20T22:04:40.163963] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDS.csv [2025-10-20T22:04:40.164026] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VSCO.csv [2025-10-20T22:04:40.164104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NZF.csv [2025-10-20T22:04:40.164209] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLX.csv [2025-10-20T22:04:40.164281] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKSY.csv [2025-10-20T22:04:40.164359] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GES.csv [2025-10-20T22:04:40.164423] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAX.csv [2025-10-20T22:04:40.164486] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OLP.csv [2025-10-20T22:04:40.164556] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MLP.csv [2025-10-20T22:04:40.164632] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMTB.csv [2025-10-20T22:04:40.164752] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INN.csv [2025-10-20T22:04:40.164824] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PHR.csv [2025-10-20T22:04:40.164889] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NCZ.csv [2025-10-20T22:04:40.164977] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HCA.csv [2025-10-20T22:04:40.165054] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FSK.csv [2025-10-20T22:04:40.165128] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CDRE.csv [2025-10-20T22:04:40.165189] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LSPD.csv [2025-10-20T22:04:40.165255] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LGI.csv [2025-10-20T22:04:40.165316] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIM.csv [2025-10-20T22:04:40.165381] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHN.csv [2025-10-20T22:04:40.165442] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SEMR.csv [2025-10-20T22:04:40.165515] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JOE.csv [2025-10-20T22:04:40.165578] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PSTL.csv [2025-10-20T22:04:40.165783] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GFR.csv [2025-10-20T22:04:40.165872] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VGI.csv [2025-10-20T22:04:40.165939] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IBN.csv [2025-10-20T22:04:40.166002] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZWS.csv [2025-10-20T22:04:40.166088] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NAD.csv [2025-10-20T22:04:40.166165] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FBIN.csv [2025-10-20T22:04:40.166237] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HLLY.csv [2025-10-20T22:04:40.166313] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GFL.csv [2025-10-20T22:04:40.166386] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ABG.csv [2025-10-20T22:04:40.166448] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/YELP.csv [2025-10-20T22:04:40.166508] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UI.csv [2025-10-20T22:04:40.166580] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ITT.csv [2025-10-20T22:04:40.166691] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EG.csv [2025-10-20T22:04:40.166795] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HQL.csv [2025-10-20T22:04:40.166860] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CPA.csv [2025-10-20T22:04:40.166922] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOJE.csv [2025-10-20T22:04:40.166995] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AGCO.csv [2025-10-20T22:04:40.167057] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXP.csv [2025-10-20T22:04:40.167118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TFIN.csv [2025-10-20T22:04:40.167183] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HUYA.csv [2025-10-20T22:04:40.167259] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MATX.csv [2025-10-20T22:04:40.167331] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BWSN.csv [2025-10-20T22:04:40.167407] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EBR.csv [2025-10-20T22:04:40.167472] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NPK.csv [2025-10-20T22:04:40.167534] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DSMC.csv [2025-10-20T22:04:40.167624] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FMY.csv [2025-10-20T22:04:40.167726] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AN.csv [2025-10-20T22:04:40.167814] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AEG.csv [2025-10-20T22:04:40.167880] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UZD.csv [2025-10-20T22:04:40.167964] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LADR.csv [2025-10-20T22:04:40.168131] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HUBS.csv [2025-10-20T22:04:40.168199] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMRT.csv [2025-10-20T22:04:40.168263] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/APG.csv [2025-10-20T22:04:40.168331] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ESTC.csv [2025-10-20T22:04:40.168395] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MLM.csv [2025-10-20T22:04:40.168457] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JHS.csv [2025-10-20T22:04:40.168560] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCR.csv [2025-10-20T22:04:40.168691] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CSAN.csv [2025-10-20T22:04:40.168782] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CXE.csv [2025-10-20T22:04:40.168859] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCRD.csv [2025-10-20T22:04:40.168922] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AHT.csv [2025-10-20T22:04:40.168990] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OOMA.csv [2025-10-20T22:04:40.169058] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AGO.csv [2025-10-20T22:04:40.169121] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RA.csv [2025-10-20T22:04:40.169186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HSBC.csv [2025-10-20T22:04:40.169250] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LAW.csv [2025-10-20T22:04:40.169330] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SW.csv [2025-10-20T22:04:40.169418] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRS.csv [2025-10-20T22:04:40.169486] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TME.csv [2025-10-20T22:04:40.169550] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TYG.csv [2025-10-20T22:04:40.169619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AES.csv [2025-10-20T22:04:40.169745] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KAI.csv [2025-10-20T22:04:40.169819] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JBI.csv [2025-10-20T22:04:40.169885] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UA.csv [2025-10-20T22:04:40.169954] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CTOS.csv [2025-10-20T22:04:40.170035] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CDE.csv [2025-10-20T22:04:40.170111] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NKX.csv [2025-10-20T22:04:40.170188] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVC.csv [2025-10-20T22:04:40.170253] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WOW.csv [2025-10-20T22:04:40.170318] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NUW.csv [2025-10-20T22:04:40.170381] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPX.csv [2025-10-20T22:04:40.170444] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SSL.csv [2025-10-20T22:04:40.170512] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HVT.csv [2025-10-20T22:04:40.170584] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DELL.csv [2025-10-20T22:04:40.170671] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/THO.csv [2025-10-20T22:04:40.170764] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TAP.csv [2025-10-20T22:04:40.170845] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NGVC.csv [2025-10-20T22:04:40.170912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFLT.csv [2025-10-20T22:04:40.170997] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EQBK.csv [2025-10-20T22:04:40.171067] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLB.csv [2025-10-20T22:04:40.171144] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALLE.csv [2025-10-20T22:04:40.171206] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLCO.csv [2025-10-20T22:04:40.171269] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UMH.csv [2025-10-20T22:04:40.171344] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JBL.csv [2025-10-20T22:04:40.171417] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FUN.csv [2025-10-20T22:04:40.171479] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BN.csv [2025-10-20T22:04:40.171556] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EQNR.csv [2025-10-20T22:04:40.171627] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IAE.csv [2025-10-20T22:04:40.171749] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MHO.csv [2025-10-20T22:04:40.171827] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MPW.csv [2025-10-20T22:04:40.171891] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AOMR.csv [2025-10-20T22:04:40.171954] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SMBK.csv [2025-10-20T22:04:40.172072] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MAGN.csv [2025-10-20T22:04:40.172136] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SPH.csv [2025-10-20T22:04:40.172212] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FOA.csv [2025-10-20T22:04:40.172273] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TVC.csv [2025-10-20T22:04:40.172346] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HMN.csv [2025-10-20T22:04:40.172409] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WSO.csv [2025-10-20T22:04:40.172471] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIO.csv [2025-10-20T22:04:40.172534] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VVX.csv [2025-10-20T22:04:40.172602] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IDE.csv [2025-10-20T22:04:40.172699] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WDS.csv [2025-10-20T22:04:40.172803] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FERG.csv [2025-10-20T22:04:40.172882] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FBK.csv [2025-10-20T22:04:40.172957] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASIX.csv [2025-10-20T22:04:40.173047] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FOR.csv [2025-10-20T22:04:40.173118] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EOD.csv [2025-10-20T22:04:40.173181] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WPC.csv [2025-10-20T22:04:40.173244] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XYF.csv [2025-10-20T22:04:40.173305] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MBI.csv [2025-10-20T22:04:40.173367] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TECK.csv [2025-10-20T22:04:40.173435] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCB.csv [2025-10-20T22:04:40.173497] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RFM.csv [2025-10-20T22:04:40.173567] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MKC.csv [2025-10-20T22:04:40.173775] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DLX.csv [2025-10-20T22:04:40.173873] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IVZ.csv [2025-10-20T22:04:40.173940] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MMC.csv [2025-10-20T22:04:40.174017] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FDX.csv [2025-10-20T22:04:40.174080] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCN.csv [2025-10-20T22:04:40.174143] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LHX.csv [2025-10-20T22:04:40.174205] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WF.csv [2025-10-20T22:04:40.174287] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GMRE.csv [2025-10-20T22:04:40.174348] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NUV.csv [2025-10-20T22:04:40.174416] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTN.csv [2025-10-20T22:04:40.174480] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BWXT.csv [2025-10-20T22:04:40.174542] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ANRO.csv [2025-10-20T22:04:40.174619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ACEL.csv [2025-10-20T22:04:40.174718] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPVG.csv [2025-10-20T22:04:40.174797] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TFC.csv [2025-10-20T22:04:40.174874] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GNE.csv [2025-10-20T22:04:40.174939] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VPV.csv [2025-10-20T22:04:40.175001] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SARO.csv [2025-10-20T22:04:40.175106] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIG.csv [2025-10-20T22:04:40.175210] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BLND.csv [2025-10-20T22:04:40.175274] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MLR.csv [2025-10-20T22:04:40.175349] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NHI.csv [2025-10-20T22:04:40.175504] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPAM.csv [2025-10-20T22:04:40.175595] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPZ.csv [2025-10-20T22:04:40.175689] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DRLL.csv [2025-10-20T22:04:40.175779] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STEM.csv [2025-10-20T22:04:40.175844] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBN.csv [2025-10-20T22:04:40.175924] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHD.csv [2025-10-20T22:04:40.176004] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETY.csv [2025-10-20T22:04:40.176071] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZETA.csv [2025-10-20T22:04:40.176136] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UP.csv [2025-10-20T22:04:40.176224] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLS.csv [2025-10-20T22:04:40.176300] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ACHR.csv [2025-10-20T22:04:40.176378] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CCIF.csv [2025-10-20T22:04:40.176456] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALB.csv [2025-10-20T22:04:40.176517] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLDT.csv [2025-10-20T22:04:40.176581] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ARES.csv [2025-10-20T22:04:40.176702] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HCI.csv [2025-10-20T22:04:40.176796] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VBF.csv [2025-10-20T22:04:40.176864] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RY.csv [2025-10-20T22:04:40.176931] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SDRL.csv [2025-10-20T22:04:40.177007] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ATKR.csv [2025-10-20T22:04:40.177098] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BX.csv [2025-10-20T22:04:40.177171] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MWA.csv [2025-10-20T22:04:40.177236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IBP.csv [2025-10-20T22:04:40.177299] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BIRK.csv [2025-10-20T22:04:40.177363] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RMM.csv [2025-10-20T22:04:40.177433] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LOMA.csv [2025-10-20T22:04:40.177515] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EQR.csv [2025-10-20T22:04:40.177592] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECO.csv [2025-10-20T22:04:40.177685] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SBI.csv [2025-10-20T22:04:40.177788] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HRB.csv [2025-10-20T22:04:40.177856] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DRI.csv [2025-10-20T22:04:40.177920] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RBOT.csv [2025-10-20T22:04:40.178048] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GEV.csv [2025-10-20T22:04:40.178132] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GAB.csv [2025-10-20T22:04:40.178209] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALC.csv [2025-10-20T22:04:40.178284] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMRC.csv [2025-10-20T22:04:40.178355] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HQH.csv [2025-10-20T22:04:40.178432] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DY.csv [2025-10-20T22:04:40.178495] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BMA.csv [2025-10-20T22:04:40.178571] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HFRO.csv [2025-10-20T22:04:40.178634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NBB.csv [2025-10-20T22:04:40.178752] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ALIT.csv [2025-10-20T22:04:40.178827] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UAN.csv [2025-10-20T22:04:40.178893] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRM.csv [2025-10-20T22:04:40.178968] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AME.csv [2025-10-20T22:04:40.179031] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LZB.csv [2025-10-20T22:04:40.179112] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CABO.csv [2025-10-20T22:04:40.179197] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KKRS.csv [2025-10-20T22:04:40.179317] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXN.csv [2025-10-20T22:04:40.179401] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SOJD.csv [2025-10-20T22:04:40.179480] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HBB.csv [2025-10-20T22:04:40.179544] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PDM.csv [2025-10-20T22:04:40.179609] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KRC.csv [2025-10-20T22:04:40.179708] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IGA.csv [2025-10-20T22:04:40.179804] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASG.csv [2025-10-20T22:04:40.179868] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OCFT.csv [2025-10-20T22:04:40.179945] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GGB.csv [2025-10-20T22:04:40.180008] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LTM.csv [2025-10-20T22:04:40.180086] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FC.csv [2025-10-20T22:04:40.180167] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CADE.csv [2025-10-20T22:04:40.180238] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ECAT.csv [2025-10-20T22:04:40.180315] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GMED.csv [2025-10-20T22:04:40.180390] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EL.csv [2025-10-20T22:04:40.180460] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBVA.csv [2025-10-20T22:04:40.180524] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SJT.csv [2025-10-20T22:04:40.180587] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MEI.csv [2025-10-20T22:04:40.180680] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXDT.csv [2025-10-20T22:04:40.180792] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BEKE.csv [2025-10-20T22:04:40.180863] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BGS.csv [2025-10-20T22:04:40.180928] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUSA.csv [2025-10-20T22:04:40.181002] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASGN.csv [2025-10-20T22:04:40.181104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTI.csv [2025-10-20T22:04:40.181192] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VTLE.csv [2025-10-20T22:04:40.181263] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ELV.csv [2025-10-20T22:04:40.181333] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BAH.csv [2025-10-20T22:04:40.181413] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSQ.csv [2025-10-20T22:04:40.181476] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DIAX.csv [2025-10-20T22:04:40.181537] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBDC.csv [2025-10-20T22:04:40.181597] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NRUC.csv [2025-10-20T22:04:40.181820] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GIL.csv [2025-10-20T22:04:40.181911] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UHS.csv [2025-10-20T22:04:40.181977] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BSX.csv [2025-10-20T22:04:40.182054] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DTE.csv [2025-10-20T22:04:40.182124] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IAG.csv [2025-10-20T22:04:40.182188] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SHG.csv [2025-10-20T22:04:40.182261] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KFS.csv [2025-10-20T22:04:40.182322] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/APD.csv [2025-10-20T22:04:40.182383] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NOV.csv [2025-10-20T22:04:40.182458] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BORR.csv [2025-10-20T22:04:40.182542] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BY.csv [2025-10-20T22:04:40.182609] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IONQ.csv [2025-10-20T22:04:40.182705] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TBBB.csv [2025-10-20T22:04:40.182789] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPH.csv [2025-10-20T22:04:40.182852] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TFII.csv [2025-10-20T22:04:40.182913] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NIM.csv [2025-10-20T22:04:40.182974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/XPO.csv [2025-10-20T22:04:40.183034] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RZB.csv [2025-10-20T22:04:40.183101] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LXFR.csv [2025-10-20T22:04:40.183169] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CBL.csv [2025-10-20T22:04:40.183231] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BCSF.csv [2025-10-20T22:04:40.183293] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BKN.csv [2025-10-20T22:04:40.183361] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WFG.csv [2025-10-20T22:04:40.183438] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DOC.csv [2025-10-20T22:04:40.183499] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TSLX.csv [2025-10-20T22:04:40.183617] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AUNA.csv [2025-10-20T22:04:40.183721] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVG.csv [2025-10-20T22:04:40.183803] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JHG.csv [2025-10-20T22:04:40.183871] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PFO.csv [2025-10-20T22:04:40.183932] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/VVR.csv [2025-10-20T22:04:40.184018] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EPD.csv [2025-10-20T22:04:40.184104] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ANVS.csv [2025-10-20T22:04:40.184186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FUBO.csv [2025-10-20T22:04:40.184255] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/IOT.csv [2025-10-20T22:04:40.184327] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NXJ.csv [2025-10-20T22:04:40.184403] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CLW.csv [2025-10-20T22:04:40.184479] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAC.csv [2025-10-20T22:04:40.184542] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CHPT.csv [2025-10-20T22:04:40.184615] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ORN.csv [2025-10-20T22:04:40.184742] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GOF.csv [2025-10-20T22:04:40.184825] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TAL.csv [2025-10-20T22:04:40.184907] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EMR.csv [2025-10-20T22:04:40.184985] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GHI.csv [2025-10-20T22:04:40.185057] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DIS.csv [2025-10-20T22:04:40.185122] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUX.csv [2025-10-20T22:04:40.185186] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TS.csv [2025-10-20T22:04:40.185251] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ICE.csv [2025-10-20T22:04:40.185313] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CODI.csv [2025-10-20T22:04:40.185378] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LZM.csv [2025-10-20T22:04:40.185447] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CAT.csv [2025-10-20T22:04:40.185572] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HAFN.csv [2025-10-20T22:04:40.185638] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMZ.csv [2025-10-20T22:04:40.185765] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CRH.csv [2025-10-20T22:04:40.185886] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CIB.csv [2025-10-20T22:04:40.185961] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TPB.csv [2025-10-20T22:04:40.186038] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/JQC.csv [2025-10-20T22:04:40.186116] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDDY.csv [2025-10-20T22:04:40.186181] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MCY.csv [2025-10-20T22:04:40.186257] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FN.csv [2025-10-20T22:04:40.186361] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MFM.csv [2025-10-20T22:04:40.186441] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/INFA.csv [2025-10-20T22:04:40.186508] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RACE.csv [2025-10-20T22:04:40.186571] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BAM.csv [2025-10-20T22:04:40.186634] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MIN.csv [2025-10-20T22:04:40.186744] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIX.csv [2025-10-20T22:04:40.186817] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TFX.csv [2025-10-20T22:04:40.186881] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMS.csv [2025-10-20T22:04:40.186947] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RFI.csv [2025-10-20T22:04:40.187023] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDO.csv [2025-10-20T22:04:40.187087] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PCM.csv [2025-10-20T22:04:40.187151] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TDOC.csv [2025-10-20T22:04:40.187236] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WMT.csv [2025-10-20T22:04:40.187329] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PKX.csv [2025-10-20T22:04:40.187408] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EDU.csv [2025-10-20T22:04:40.187495] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BXSL.csv [2025-10-20T22:04:40.187561] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WPM.csv [2025-10-20T22:04:40.187625] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZGN.csv [2025-10-20T22:04:40.187743] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AIR.csv [2025-10-20T22:04:40.187821] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DAVA.csv [2025-10-20T22:04:40.187899] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ETJ.csv [2025-10-20T22:04:40.188006] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FLC.csv [2025-10-20T22:04:40.188088] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NINE.csv [2025-10-20T22:04:40.188165] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/LEVI.csv [2025-10-20T22:04:40.188230] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NAT.csv [2025-10-20T22:04:40.188292] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BBUC.csv [2025-10-20T22:04:40.188355] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WMB.csv [2025-10-20T22:04:40.188431] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FTK.csv [2025-10-20T22:04:40.188492] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BABA.csv [2025-10-20T22:04:40.188564] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GGZ.csv [2025-10-20T22:04:40.188639] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ET.csv [2025-10-20T22:04:40.188766] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/UTF.csv [2025-10-20T22:04:40.188848] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AMT.csv [2025-10-20T22:04:40.188912] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ZBH.csv [2025-10-20T22:04:40.188973] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/STN.csv [2025-10-20T22:04:40.189035] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GUG.csv [2025-10-20T22:04:40.189095] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BGT.csv [2025-10-20T22:04:40.189169] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HPP.csv [2025-10-20T22:04:40.189232] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MTD.csv [2025-10-20T22:04:40.189307] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GDV.csv [2025-10-20T22:04:40.189390] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPI.csv [2025-10-20T22:04:40.189453] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WELL.csv [2025-10-20T22:04:40.189544] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FMC.csv [2025-10-20T22:04:40.189619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/ASH.csv [2025-10-20T22:04:40.189828] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/RVLV.csv [2025-10-20T22:04:40.189922] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EVT.csv [2025-10-20T22:04:40.189990] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TT.csv [2025-10-20T22:04:40.190053] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SYF.csv [2025-10-20T22:04:40.190113] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PAR.csv [2025-10-20T22:04:40.190174] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MT.csv [2025-10-20T22:04:40.190249] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WCC.csv [2025-10-20T22:04:40.190336] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/OIS.csv [2025-10-20T22:04:40.190401] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BDX.csv [2025-10-20T22:04:40.190473] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KREF.csv [2025-10-20T22:04:40.190536] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/MUJ.csv [2025-10-20T22:04:40.190621] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KRO.csv [2025-10-20T22:04:40.190722] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/AAP.csv [2025-10-20T22:04:40.190820] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FCX.csv [2025-10-20T22:04:40.190887] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/NMI.csv [2025-10-20T22:04:40.190964] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/GPMT.csv [2025-10-20T22:04:40.191038] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EXP.csv [2025-10-20T22:04:40.191125] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/FPF.csv [2025-10-20T22:04:40.191191] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SAP.csv [2025-10-20T22:04:40.191252] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/WLKP.csv [2025-10-20T22:04:40.191314] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SCD.csv [2025-10-20T22:04:40.191374] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/DCO.csv [2025-10-20T22:04:40.191435] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRP.csv [2025-10-20T22:04:40.191495] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TTC.csv [2025-10-20T22:04:40.191557] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/APLE.csv [2025-10-20T22:04:40.191619] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/KVUE.csv [2025-10-20T22:04:40.191715] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SWK.csv [2025-10-20T22:04:40.191836] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/EXR.csv [2025-10-20T22:04:40.191909] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/PRKS.csv [2025-10-20T22:04:40.191974] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CTDD.csv [2025-10-20T22:04:40.192036] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/SA.csv [2025-10-20T22:04:40.192102] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/BNL.csv [2025-10-20T22:04:40.192164] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TXO.csv [2025-10-20T22:04:40.192223] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/TRU.csv [2025-10-20T22:04:40.192292] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/HIMS.csv [2025-10-20T22:04:40.192366] [DEBUG] [LOCAL ] Deleted: /opt/tdm/output/data/derivative_storage/CM.csv [2025-10-20T22:04:40.192440] [INFO ] [LOCAL ] Deleted 1774 derivative data files. [2025-10-20T22:04:40.192470] [INFO ] [LOCAL ] Scanning primary storage for available symbols. [2025-10-20T22:04:40.194503] [INFO ] [LOCAL ] Found 1877 symbols with primary data. [2025-10-20T22:04:40.194742] [INFO ] [MAIN ] Starting parallel derivative generation with 3 worker threads for 1877 symbols [2025-10-20T22:04:40.195474] [INFO ] [MAIN ] A: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:40.195637] [INFO ] [MAIN ] AAMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:40.196047] [INFO ] [MAIN ] AAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:40.220061] [DEBUG] [LOCAL ] AAMI: Loaded 24762 existing records from primary storage. [2025-10-20T22:04:40.220266] [INFO ] [MAIN ] AAMI: Loaded 24762 primary data records. [2025-10-20T22:04:40.562086] [DEBUG] [LOCAL ] A: Loaded 50738 existing records from primary storage. [2025-10-20T22:04:40.577255] [INFO ] [MAIN ] A: Loaded 50738 primary data records. [2025-10-20T22:04:40.599257] [DEBUG] [LOCAL ] AAP: Loaded 51783 existing records from primary storage. [2025-10-20T22:04:40.616261] [INFO ] [MAIN ] AAP: Loaded 51783 primary data records. [2025-10-20T22:04:41.194116] [DEBUG] [MAIN ] AAMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:41.195710] [INFO ] [MAIN ] AAMI: Generated 1794 derivative records from 24762 primary records. [2025-10-20T22:04:41.195846] [DEBUG] [LOCAL ] AAMI: Saving derivative data. [2025-10-20T22:04:41.214678] [DEBUG] [LOCAL ] AAMI: Derivative data saved (1794 records). [2025-10-20T22:04:41.214939] [INFO ] [MAIN ] AAMI: Successfully saved derivative data. [2025-10-20T22:04:41.216872] [INFO ] [MAIN ] Derivative Progress: 1/1877 symbols completed. AAMI took 1.0s. Estimated remaining: 31m 58s. Rate: 58.7 symbols/min [2025-10-20T22:04:41.217126] [INFO ] [MAIN ] AAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:41.236456] [DEBUG] [MAIN ] A: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:41.238292] [INFO ] [MAIN ] A: Generated 1794 derivative records from 50738 primary records. [2025-10-20T22:04:41.239914] [DEBUG] [LOCAL ] A: Saving derivative data. [2025-10-20T22:04:41.281392] [DEBUG] [MAIN ] AAP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:41.282330] [INFO ] [MAIN ] AAP: Generated 1794 derivative records from 51783 primary records. [2025-10-20T22:04:41.282507] [DEBUG] [LOCAL ] AAP: Saving derivative data. [2025-10-20T22:04:41.284159] [DEBUG] [LOCAL ] A: Derivative data saved (1794 records). [2025-10-20T22:04:41.285608] [INFO ] [MAIN ] A: Successfully saved derivative data. [2025-10-20T22:04:41.287860] [INFO ] [MAIN ] Derivative Progress: 2/1877 symbols completed. A took 1.1s. Estimated remaining: 17m 5s. Rate: 109.8 symbols/min [2025-10-20T22:04:41.299183] [INFO ] [MAIN ] AB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:41.311982] [DEBUG] [LOCAL ] AAP: Derivative data saved (1794 records). [2025-10-20T22:04:41.313409] [INFO ] [MAIN ] AAP: Successfully saved derivative data. [2025-10-20T22:04:41.316560] [INFO ] [MAIN ] Derivative Progress: 3/1877 symbols completed. AAP took 1.1s. Estimated remaining: 11m 41s. Rate: 160.4 symbols/min [2025-10-20T22:04:41.318078] [DEBUG] [LOCAL ] AAT: Loaded 30441 existing records from primary storage. [2025-10-20T22:04:41.316902] [INFO ] [MAIN ] ABEV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:41.318241] [INFO ] [MAIN ] AAT: Loaded 30441 primary data records. [2025-10-20T22:04:41.519080] [DEBUG] [LOCAL ] AB: Loaded 25030 existing records from primary storage. [2025-10-20T22:04:41.624872] [INFO ] [MAIN ] AB: Loaded 25030 primary data records. [2025-10-20T22:04:41.831242] [DEBUG] [LOCAL ] ABEV: Loaded 53291 existing records from primary storage. [2025-10-20T22:04:41.831613] [INFO ] [MAIN ] ABEV: Loaded 53291 primary data records. [2025-10-20T22:04:42.331347] [DEBUG] [MAIN ] AAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:42.414166] [INFO ] [MAIN ] AAT: Generated 1794 derivative records from 30441 primary records. [2025-10-20T22:04:42.528797] [DEBUG] [LOCAL ] AAT: Saving derivative data. [2025-10-20T22:04:42.559923] [DEBUG] [LOCAL ] AAT: Derivative data saved (1794 records). [2025-10-20T22:04:42.560607] [INFO ] [MAIN ] AAT: Successfully saved derivative data. [2025-10-20T22:04:42.563110] [INFO ] [MAIN ] Derivative Progress: 4/1877 symbols completed. AAT took 1.3s. Estimated remaining: 18m 29s. Rate: 101.3 symbols/min [2025-10-20T22:04:42.563809] [INFO ] [MAIN ] ABG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:42.596243] [DEBUG] [MAIN ] AB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:42.597528] [INFO ] [MAIN ] AB: Generated 1794 derivative records from 25030 primary records. [2025-10-20T22:04:42.597800] [DEBUG] [LOCAL ] AB: Saving derivative data. [2025-10-20T22:04:42.628448] [DEBUG] [LOCAL ] ABG: Loaded 18128 existing records from primary storage. [2025-10-20T22:04:42.628758] [INFO ] [MAIN ] ABG: Loaded 18128 primary data records. [2025-10-20T22:04:42.762243] [DEBUG] [LOCAL ] AB: Derivative data saved (1794 records). [2025-10-20T22:04:42.772028] [DEBUG] [MAIN ] ABEV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:42.778218] [INFO ] [MAIN ] ABEV: Generated 1794 derivative records from 53291 primary records. [2025-10-20T22:04:42.776954] [INFO ] [MAIN ] AB: Successfully saved derivative data. [2025-10-20T22:04:42.783627] [DEBUG] [LOCAL ] ABEV: Saving derivative data. [2025-10-20T22:04:42.825207] [INFO ] [MAIN ] Derivative Progress: 5/1877 symbols completed. AB took 1.5s. Estimated remaining: 16m 25s. Rate: 114.0 symbols/min [2025-10-20T22:04:42.857804] [INFO ] [MAIN ] ABM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:42.900239] [DEBUG] [LOCAL ] ABEV: Derivative data saved (1794 records). [2025-10-20T22:04:43.028943] [INFO ] [MAIN ] ABEV: Successfully saved derivative data. [2025-10-20T22:04:43.039962] [INFO ] [MAIN ] Derivative Progress: 6/1877 symbols completed. ABEV took 1.7s. Estimated remaining: 14m 47s. Rate: 126.5 symbols/min [2025-10-20T22:04:43.040208] [INFO ] [MAIN ] ABR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:43.077804] [DEBUG] [LOCAL ] ABM: Loaded 32177 existing records from primary storage. [2025-10-20T22:04:43.083461] [INFO ] [MAIN ] ABM: Loaded 32177 primary data records. [2025-10-20T22:04:43.130142] [DEBUG] [MAIN ] ABG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:43.180963] [INFO ] [MAIN ] ABG: Generated 1794 derivative records from 18128 primary records. [2025-10-20T22:04:43.198980] [DEBUG] [LOCAL ] ABG: Saving derivative data. [2025-10-20T22:04:43.335021] [DEBUG] [LOCAL ] ABR: Loaded 51949 existing records from primary storage. [2025-10-20T22:04:43.443288] [INFO ] [MAIN ] ABR: Loaded 51949 primary data records. [2025-10-20T22:04:43.454722] [DEBUG] [LOCAL ] ABG: Derivative data saved (1794 records). [2025-10-20T22:04:43.454926] [INFO ] [MAIN ] ABG: Successfully saved derivative data. [2025-10-20T22:04:43.456822] [INFO ] [MAIN ] Derivative Progress: 7/1877 symbols completed. ABG took 0.9s. Estimated remaining: 14m 31s. Rate: 128.7 symbols/min [2025-10-20T22:04:43.488050] [INFO ] [MAIN ] ABT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:43.797517] [DEBUG] [MAIN ] ABM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:43.798694] [INFO ] [MAIN ] ABM: Generated 1794 derivative records from 32177 primary records. [2025-10-20T22:04:43.798957] [DEBUG] [LOCAL ] ABM: Saving derivative data. [2025-10-20T22:04:43.818620] [DEBUG] [LOCAL ] ABM: Derivative data saved (1794 records). [2025-10-20T22:04:43.820576] [INFO ] [MAIN ] ABM: Successfully saved derivative data. [2025-10-20T22:04:43.838776] [INFO ] [MAIN ] Derivative Progress: 8/1877 symbols completed. ABM took 1.0s. Estimated remaining: 14m 11s. Rate: 131.7 symbols/min [2025-10-20T22:04:43.840079] [DEBUG] [LOCAL ] ABT: Loaded 53622 existing records from primary storage. [2025-10-20T22:04:43.840339] [INFO ] [MAIN ] ABT: Loaded 53622 primary data records. [2025-10-20T22:04:43.840547] [INFO ] [MAIN ] ACCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:43.891267] [DEBUG] [MAIN ] ABR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:43.903389] [INFO ] [MAIN ] ABR: Generated 1794 derivative records from 51949 primary records. [2025-10-20T22:04:43.925189] [DEBUG] [LOCAL ] ABR: Saving derivative data. [2025-10-20T22:04:43.994829] [DEBUG] [LOCAL ] ABR: Derivative data saved (1794 records). [2025-10-20T22:04:44.053108] [INFO ] [MAIN ] ABR: Successfully saved derivative data. [2025-10-20T22:04:44.163782] [INFO ] [MAIN ] Derivative Progress: 9/1877 symbols completed. ABR took 1.1s. Estimated remaining: 13m 44s. Rate: 136.0 symbols/min [2025-10-20T22:04:44.172230] [INFO ] [MAIN ] ACEL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:44.187093] [DEBUG] [LOCAL ] ACCO: Loaded 36838 existing records from primary storage. [2025-10-20T22:04:44.187594] [INFO ] [MAIN ] ACCO: Loaded 36838 primary data records. [2025-10-20T22:04:44.322733] [DEBUG] [LOCAL ] ACEL: Loaded 26577 existing records from primary storage. [2025-10-20T22:04:44.328801] [INFO ] [MAIN ] ACEL: Loaded 26577 primary data records. [2025-10-20T22:04:44.878308] [DEBUG] [MAIN ] ABT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:44.879764] [INFO ] [MAIN ] ABT: Generated 1794 derivative records from 53622 primary records. [2025-10-20T22:04:44.879905] [DEBUG] [LOCAL ] ABT: Saving derivative data. [2025-10-20T22:04:44.898367] [DEBUG] [LOCAL ] ABT: Derivative data saved (1794 records). [2025-10-20T22:04:44.899978] [INFO ] [MAIN ] ABT: Successfully saved derivative data. [2025-10-20T22:04:44.902504] [INFO ] [MAIN ] Derivative Progress: 10/1877 symbols completed. ABT took 1.4s. Estimated remaining: 14m 39s. Rate: 127.4 symbols/min [2025-10-20T22:04:44.903573] [INFO ] [MAIN ] ACHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:44.926381] [DEBUG] [MAIN ] ACCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:44.927507] [INFO ] [MAIN ] ACCO: Generated 1794 derivative records from 36838 primary records. [2025-10-20T22:04:44.927763] [DEBUG] [LOCAL ] ACCO: Saving derivative data. [2025-10-20T22:04:44.946212] [DEBUG] [LOCAL ] ACCO: Derivative data saved (1794 records). [2025-10-20T22:04:44.948122] [INFO ] [MAIN ] ACCO: Successfully saved derivative data. [2025-10-20T22:04:44.950362] [INFO ] [MAIN ] Derivative Progress: 11/1877 symbols completed. ACCO took 1.1s. Estimated remaining: 13m 27s. Rate: 138.8 symbols/min [2025-10-20T22:04:44.950815] [INFO ] [MAIN ] ACM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:44.964429] [DEBUG] [MAIN ] ACEL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:44.965349] [INFO ] [MAIN ] ACEL: Generated 1794 derivative records from 26577 primary records. [2025-10-20T22:04:44.965545] [DEBUG] [LOCAL ] ACEL: Saving derivative data. [2025-10-20T22:04:44.998325] [DEBUG] [LOCAL ] ACEL: Derivative data saved (1794 records). [2025-10-20T22:04:44.998625] [INFO ] [MAIN ] ACEL: Successfully saved derivative data. [2025-10-20T22:04:45.000936] [INFO ] [MAIN ] Derivative Progress: 12/1877 symbols completed. ACEL took 0.8s. Estimated remaining: 12m 27s. Rate: 149.8 symbols/min [2025-10-20T22:04:45.001931] [INFO ] [MAIN ] ACN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:45.020913] [DEBUG] [LOCAL ] ACHR: Loaded 53818 existing records from primary storage. [2025-10-20T22:04:45.021118] [INFO ] [MAIN ] ACHR: Loaded 53818 primary data records. [2025-10-20T22:04:45.181083] [DEBUG] [LOCAL ] ACM: Loaded 42890 existing records from primary storage. [2025-10-20T22:04:45.202302] [INFO ] [MAIN ] ACM: Loaded 42890 primary data records. [2025-10-20T22:04:45.537990] [DEBUG] [LOCAL ] ACN: Loaded 53440 existing records from primary storage. [2025-10-20T22:04:45.540478] [INFO ] [MAIN ] ACN: Loaded 53440 primary data records. [2025-10-20T22:04:46.003105] [DEBUG] [MAIN ] ACHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:46.004168] [INFO ] [MAIN ] ACHR: Generated 1794 derivative records from 53818 primary records. [2025-10-20T22:04:46.004265] [DEBUG] [LOCAL ] ACHR: Saving derivative data. [2025-10-20T22:04:46.024199] [DEBUG] [LOCAL ] ACHR: Derivative data saved (1794 records). [2025-10-20T22:04:46.024318] [INFO ] [MAIN ] ACHR: Successfully saved derivative data. [2025-10-20T22:04:46.026236] [INFO ] [MAIN ] Derivative Progress: 13/1877 symbols completed. ACHR took 1.1s. Estimated remaining: 13m 56s. Rate: 133.8 symbols/min [2025-10-20T22:04:46.039448] [DEBUG] [MAIN ] ACM: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:04:46.040215] [INFO ] [MAIN ] ACM: Generated 1793 derivative records from 42890 primary records. [2025-10-20T22:04:46.040373] [DEBUG] [LOCAL ] ACM: Saving derivative data. [2025-10-20T22:04:46.056484] [DEBUG] [LOCAL ] ACM: Derivative data saved (1793 records). [2025-10-20T22:04:46.056628] [INFO ] [MAIN ] ACM: Successfully saved derivative data. [2025-10-20T22:04:46.060009] [INFO ] [MAIN ] Derivative Progress: 14/1877 symbols completed. ACM took 1.1s. Estimated remaining: 13m 1s. Rate: 143.2 symbols/min [2025-10-20T22:04:46.073361] [DEBUG] [MAIN ] ACN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:46.073943] [INFO ] [MAIN ] ACN: Generated 1794 derivative records from 53440 primary records. [2025-10-20T22:04:46.074003] [DEBUG] [LOCAL ] ACN: Saving derivative data. [2025-10-20T22:04:46.089067] [DEBUG] [LOCAL ] ACN: Derivative data saved (1794 records). [2025-10-20T22:04:46.089124] [INFO ] [MAIN ] ACN: Successfully saved derivative data. [2025-10-20T22:04:46.091950] [INFO ] [MAIN ] Derivative Progress: 15/1877 symbols completed. ACN took 1.1s. Estimated remaining: 12m 12s. Rate: 152.6 symbols/min [2025-10-20T22:04:46.092368] [INFO ] [MAIN ] ACP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:46.092540] [INFO ] [MAIN ] ADC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:46.092636] [INFO ] [MAIN ] ACVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:46.142210] [DEBUG] [LOCAL ] ACP: Loaded 31109 existing records from primary storage. [2025-10-20T22:04:46.142359] [INFO ] [MAIN ] ACP: Loaded 31109 primary data records. [2025-10-20T22:04:46.258480] [DEBUG] [LOCAL ] ACVA: Loaded 49252 existing records from primary storage. [2025-10-20T22:04:46.310781] [INFO ] [MAIN ] ACVA: Loaded 49252 primary data records. [2025-10-20T22:04:46.298275] [DEBUG] [LOCAL ] ADC: Loaded 46431 existing records from primary storage. [2025-10-20T22:04:46.319423] [INFO ] [MAIN ] ADC: Loaded 46431 primary data records. [2025-10-20T22:04:47.127865] [DEBUG] [MAIN ] ACP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:47.129203] [INFO ] [MAIN ] ACP: Generated 1794 derivative records from 31109 primary records. [2025-10-20T22:04:47.129380] [DEBUG] [LOCAL ] ACP: Saving derivative data. [2025-10-20T22:04:47.147023] [DEBUG] [LOCAL ] ACP: Derivative data saved (1794 records). [2025-10-20T22:04:47.147198] [INFO ] [MAIN ] ACP: Successfully saved derivative data. [2025-10-20T22:04:47.149029] [INFO ] [MAIN ] Derivative Progress: 16/1877 symbols completed. ACP took 1.1s. Estimated remaining: 13m 29s. Rate: 138.0 symbols/min [2025-10-20T22:04:47.149344] [INFO ] [MAIN ] ADNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:47.170503] [DEBUG] [MAIN ] ADC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:47.171817] [INFO ] [MAIN ] ADC: Generated 1794 derivative records from 46431 primary records. [2025-10-20T22:04:47.171997] [DEBUG] [LOCAL ] ADC: Saving derivative data. [2025-10-20T22:04:47.197742] [DEBUG] [MAIN ] ACVA: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:04:47.202383] [DEBUG] [LOCAL ] ADC: Derivative data saved (1794 records). [2025-10-20T22:04:47.202753] [INFO ] [MAIN ] ADC: Successfully saved derivative data. [2025-10-20T22:04:47.204962] [INFO ] [MAIN ] Derivative Progress: 17/1877 symbols completed. ADC took 1.1s. Estimated remaining: 12m 47s. Rate: 145.5 symbols/min [2025-10-20T22:04:47.206313] [INFO ] [MAIN ] ADT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:47.206058] [INFO ] [MAIN ] ACVA: Generated 1793 derivative records from 49252 primary records. [2025-10-20T22:04:47.206767] [DEBUG] [LOCAL ] ACVA: Saving derivative data. [2025-10-20T22:04:47.234926] [DEBUG] [LOCAL ] ACVA: Derivative data saved (1793 records). [2025-10-20T22:04:47.235225] [INFO ] [MAIN ] ACVA: Successfully saved derivative data. [2025-10-20T22:04:47.238153] [INFO ] [MAIN ] Derivative Progress: 18/1877 symbols completed. ACVA took 1.1s. Estimated remaining: 12m 7s. Rate: 153.3 symbols/min [2025-10-20T22:04:47.238431] [INFO ] [MAIN ] ADX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:47.256062] [DEBUG] [LOCAL ] ADNT: Loaded 46530 existing records from primary storage. [2025-10-20T22:04:47.256393] [INFO ] [MAIN ] ADNT: Loaded 46530 primary data records. [2025-10-20T22:04:47.440477] [DEBUG] [LOCAL ] ADX: Loaded 27841 existing records from primary storage. [2025-10-20T22:04:47.484454] [INFO ] [MAIN ] ADX: Loaded 27841 primary data records. [2025-10-20T22:04:47.491545] [DEBUG] [LOCAL ] ADT: Loaded 53438 existing records from primary storage. [2025-10-20T22:04:47.635423] [INFO ] [MAIN ] ADT: Loaded 53438 primary data records. [2025-10-20T22:04:48.261365] [DEBUG] [MAIN ] ADNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:48.263094] [INFO ] [MAIN ] ADNT: Generated 1794 derivative records from 46530 primary records. [2025-10-20T22:04:48.263528] [DEBUG] [LOCAL ] ADNT: Saving derivative data. [2025-10-20T22:04:48.283218] [DEBUG] [LOCAL ] ADNT: Derivative data saved (1794 records). [2025-10-20T22:04:48.283738] [INFO ] [MAIN ] ADNT: Successfully saved derivative data. [2025-10-20T22:04:48.287454] [INFO ] [MAIN ] Derivative Progress: 19/1877 symbols completed. ADNT took 1.1s. Estimated remaining: 13m 11s. Rate: 140.9 symbols/min [2025-10-20T22:04:48.287959] [INFO ] [MAIN ] AEE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:48.305506] [DEBUG] [MAIN ] ADX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:48.306460] [INFO ] [MAIN ] ADX: Generated 1794 derivative records from 27841 primary records. [2025-10-20T22:04:48.306666] [DEBUG] [LOCAL ] ADX: Saving derivative data. [2025-10-20T22:04:48.327436] [DEBUG] [LOCAL ] ADX: Derivative data saved (1794 records). [2025-10-20T22:04:48.327812] [INFO ] [MAIN ] ADX: Successfully saved derivative data. [2025-10-20T22:04:48.330061] [INFO ] [MAIN ] Derivative Progress: 20/1877 symbols completed. ADX took 1.1s. Estimated remaining: 12m 35s. Rate: 147.5 symbols/min [2025-10-20T22:04:48.343428] [INFO ] [MAIN ] AEFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:48.344021] [DEBUG] [MAIN ] ADT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:48.345239] [INFO ] [MAIN ] ADT: Generated 1794 derivative records from 53438 primary records. [2025-10-20T22:04:48.345359] [DEBUG] [LOCAL ] ADT: Saving derivative data. [2025-10-20T22:04:48.374027] [DEBUG] [LOCAL ] ADT: Derivative data saved (1794 records). [2025-10-20T22:04:48.374269] [INFO ] [MAIN ] ADT: Successfully saved derivative data. [2025-10-20T22:04:48.376539] [INFO ] [MAIN ] Derivative Progress: 21/1877 symbols completed. ADT took 1.2s. Estimated remaining: 12m 3s. Rate: 154.0 symbols/min [2025-10-20T22:04:48.381115] [INFO ] [MAIN ] AEG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:48.381010] [DEBUG] [LOCAL ] AEFC: Loaded 8025 existing records from primary storage. [2025-10-20T22:04:48.381686] [INFO ] [MAIN ] AEFC: Loaded 8025 primary data records. [2025-10-20T22:04:48.504314] [DEBUG] [LOCAL ] AEE: Loaded 50155 existing records from primary storage. [2025-10-20T22:04:48.504681] [INFO ] [MAIN ] AEE: Loaded 50155 primary data records. [2025-10-20T22:04:48.934343] [DEBUG] [LOCAL ] AEG: Loaded 49602 existing records from primary storage. [2025-10-20T22:04:49.052755] [INFO ] [MAIN ] AEG: Loaded 49602 primary data records. [2025-10-20T22:04:49.383984] [DEBUG] [MAIN ] AEFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:49.384903] [INFO ] [MAIN ] AEFC: Generated 1794 derivative records from 8025 primary records. [2025-10-20T22:04:49.385003] [DEBUG] [LOCAL ] AEFC: Saving derivative data. [2025-10-20T22:04:49.403040] [DEBUG] [LOCAL ] AEFC: Derivative data saved (1794 records). [2025-10-20T22:04:49.403153] [INFO ] [MAIN ] AEFC: Successfully saved derivative data. [2025-10-20T22:04:49.407536] [INFO ] [MAIN ] Derivative Progress: 22/1877 symbols completed. AEFC took 1.1s. Estimated remaining: 12m 57s. Rate: 143.3 symbols/min [2025-10-20T22:04:49.408001] [INFO ] [MAIN ] AEM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:49.422048] [DEBUG] [MAIN ] AEE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:49.422802] [INFO ] [MAIN ] AEE: Generated 1794 derivative records from 50155 primary records. [2025-10-20T22:04:49.423357] [DEBUG] [LOCAL ] AEE: Saving derivative data. [2025-10-20T22:04:49.441104] [DEBUG] [LOCAL ] AEE: Derivative data saved (1794 records). [2025-10-20T22:04:49.441521] [INFO ] [MAIN ] AEE: Successfully saved derivative data. [2025-10-20T22:04:49.443499] [INFO ] [MAIN ] Derivative Progress: 23/1877 symbols completed. AEE took 1.2s. Estimated remaining: 12m 26s. Rate: 149.2 symbols/min [2025-10-20T22:04:49.443803] [INFO ] [MAIN ] AEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:49.463713] [DEBUG] [MAIN ] AEG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:49.464401] [INFO ] [MAIN ] AEG: Generated 1794 derivative records from 49602 primary records. [2025-10-20T22:04:49.464574] [DEBUG] [LOCAL ] AEG: Saving derivative data. [2025-10-20T22:04:49.494102] [DEBUG] [LOCAL ] AEG: Derivative data saved (1794 records). [2025-10-20T22:04:49.494414] [INFO ] [MAIN ] AEG: Successfully saved derivative data. [2025-10-20T22:04:49.496822] [INFO ] [MAIN ] Derivative Progress: 24/1877 symbols completed. AEG took 1.1s. Estimated remaining: 11m 58s. Rate: 154.8 symbols/min [2025-10-20T22:04:49.497149] [INFO ] [MAIN ] AES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:49.515493] [DEBUG] [LOCAL ] AEM: Loaded 53477 existing records from primary storage. [2025-10-20T22:04:49.515891] [INFO ] [MAIN ] AEM: Loaded 53477 primary data records. [2025-10-20T22:04:49.696915] [DEBUG] [LOCAL ] AEO: Loaded 53565 existing records from primary storage. [2025-10-20T22:04:49.719201] [INFO ] [MAIN ] AEO: Loaded 53565 primary data records. [2025-10-20T22:04:49.980948] [DEBUG] [LOCAL ] AES: Loaded 53668 existing records from primary storage. [2025-10-20T22:04:49.981526] [INFO ] [MAIN ] AES: Loaded 53668 primary data records. [2025-10-20T22:04:50.551723] [DEBUG] [MAIN ] AEM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:50.554518] [INFO ] [MAIN ] AEM: Generated 1794 derivative records from 53477 primary records. [2025-10-20T22:04:50.554977] [DEBUG] [LOCAL ] AEM: Saving derivative data. [2025-10-20T22:04:50.572422] [DEBUG] [LOCAL ] AEM: Derivative data saved (1794 records). [2025-10-20T22:04:50.573009] [INFO ] [MAIN ] AEM: Successfully saved derivative data. [2025-10-20T22:04:50.575104] [INFO ] [MAIN ] Derivative Progress: 25/1877 symbols completed. AEM took 1.2s. Estimated remaining: 12m 49s. Rate: 144.5 symbols/min [2025-10-20T22:04:50.575233] [INFO ] [MAIN ] AESI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:50.596485] [DEBUG] [MAIN ] AEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:50.597176] [INFO ] [MAIN ] AEO: Generated 1794 derivative records from 53565 primary records. [2025-10-20T22:04:50.597484] [DEBUG] [LOCAL ] AEO: Saving derivative data. [2025-10-20T22:04:50.616268] [DEBUG] [LOCAL ] AEO: Derivative data saved (1794 records). [2025-10-20T22:04:50.616441] [INFO ] [MAIN ] AEO: Successfully saved derivative data. [2025-10-20T22:04:50.633133] [DEBUG] [MAIN ] AES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:50.632583] [INFO ] [MAIN ] Derivative Progress: 26/1877 symbols completed. AEO took 1.2s. Estimated remaining: 12m 23s. Rate: 149.5 symbols/min [2025-10-20T22:04:50.633387] [INFO ] [MAIN ] AFB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:50.634511] [INFO ] [MAIN ] AES: Generated 1794 derivative records from 53668 primary records. [2025-10-20T22:04:50.634753] [DEBUG] [LOCAL ] AES: Saving derivative data. [2025-10-20T22:04:50.665449] [DEBUG] [LOCAL ] AES: Derivative data saved (1794 records). [2025-10-20T22:04:50.665630] [INFO ] [MAIN ] AES: Successfully saved derivative data. [2025-10-20T22:04:50.667620] [INFO ] [MAIN ] Derivative Progress: 27/1877 symbols completed. AES took 1.2s. Estimated remaining: 11m 58s. Rate: 154.7 symbols/min [2025-10-20T22:04:50.667903] [INFO ] [MAIN ] AFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:50.673635] [DEBUG] [LOCAL ] AFB: Loaded 11115 existing records from primary storage. [2025-10-20T22:04:50.673892] [INFO ] [MAIN ] AFB: Loaded 11115 primary data records. [2025-10-20T22:04:50.801556] [DEBUG] [LOCAL ] AESI: Loaded 49828 existing records from primary storage. [2025-10-20T22:04:50.802382] [INFO ] [MAIN ] AESI: Loaded 49828 primary data records. [2025-10-20T22:04:51.124409] [DEBUG] [LOCAL ] AFG: Loaded 31954 existing records from primary storage. [2025-10-20T22:04:51.125177] [INFO ] [MAIN ] AFG: Loaded 31954 primary data records. [2025-10-20T22:04:51.647130] [DEBUG] [MAIN ] AFB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:51.648363] [INFO ] [MAIN ] AFB: Generated 1794 derivative records from 11115 primary records. [2025-10-20T22:04:51.648880] [DEBUG] [LOCAL ] AFB: Saving derivative data. [2025-10-20T22:04:51.680080] [DEBUG] [MAIN ] AESI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:51.679671] [DEBUG] [LOCAL ] AFB: Derivative data saved (1794 records). [2025-10-20T22:04:51.681027] [INFO ] [MAIN ] AESI: Generated 1794 derivative records from 49828 primary records. [2025-10-20T22:04:51.681890] [DEBUG] [LOCAL ] AESI: Saving derivative data. [2025-10-20T22:04:51.680316] [INFO ] [MAIN ] AFB: Successfully saved derivative data. [2025-10-20T22:04:51.684365] [INFO ] [MAIN ] Derivative Progress: 28/1877 symbols completed. AFB took 1.1s. Estimated remaining: 12m 39s. Rate: 146.2 symbols/min [2025-10-20T22:04:51.713675] [DEBUG] [LOCAL ] AESI: Derivative data saved (1794 records). [2025-10-20T22:04:51.712592] [DEBUG] [MAIN ] AFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:51.713796] [INFO ] [MAIN ] AESI: Successfully saved derivative data. [2025-10-20T22:04:51.716384] [INFO ] [MAIN ] Derivative Progress: 29/1877 symbols completed. AESI took 1.1s. Estimated remaining: 12m 14s. Rate: 151.0 symbols/min [2025-10-20T22:04:51.714427] [INFO ] [MAIN ] AFG: Generated 1794 derivative records from 31954 primary records. [2025-10-20T22:04:51.716573] [DEBUG] [LOCAL ] AFG: Saving derivative data. [2025-10-20T22:04:51.731968] [DEBUG] [LOCAL ] AFG: Derivative data saved (1794 records). [2025-10-20T22:04:51.732058] [INFO ] [MAIN ] AFG: Successfully saved derivative data. [2025-10-20T22:04:51.734796] [INFO ] [MAIN ] Derivative Progress: 30/1877 symbols completed. AFG took 1.1s. Estimated remaining: 11m 50s. Rate: 156.0 symbols/min [2025-10-20T22:04:51.735154] [INFO ] [MAIN ] AFGB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:51.735248] [INFO ] [MAIN ] AFGC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:51.735789] [INFO ] [MAIN ] AFGD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:51.743945] [DEBUG] [LOCAL ] AFGB: Loaded 2245 existing records from primary storage. [2025-10-20T22:04:51.745957] [DEBUG] [LOCAL ] AFGC: Loaded 2561 existing records from primary storage. [2025-10-20T22:04:51.746062] [INFO ] [MAIN ] AFGC: Loaded 2561 primary data records. [2025-10-20T22:04:51.766763] [INFO ] [MAIN ] AFGB: Loaded 2245 primary data records. [2025-10-20T22:04:51.743799] [DEBUG] [LOCAL ] AFGD: Loaded 1775 existing records from primary storage. [2025-10-20T22:04:51.865198] [INFO ] [MAIN ] AFGD: Loaded 1775 primary data records. [2025-10-20T22:04:52.748633] [DEBUG] [MAIN ] AFGB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:52.751072] [INFO ] [MAIN ] AFGB: Generated 1794 derivative records from 2245 primary records. [2025-10-20T22:04:52.760161] [DEBUG] [LOCAL ] AFGB: Saving derivative data. [2025-10-20T22:04:52.775446] [DEBUG] [MAIN ] AFGC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:52.784470] [INFO ] [MAIN ] AFGC: Generated 1794 derivative records from 2561 primary records. [2025-10-20T22:04:52.784624] [DEBUG] [LOCAL ] AFGC: Saving derivative data. [2025-10-20T22:04:52.783399] [DEBUG] [LOCAL ] AFGB: Derivative data saved (1794 records). [2025-10-20T22:04:52.798912] [INFO ] [MAIN ] AFGB: Successfully saved derivative data. [2025-10-20T22:04:52.805519] [DEBUG] [MAIN ] AFGD: Filtered 7001 bins outside market hours, 7823 incomplete bins [2025-10-20T22:04:52.810562] [INFO ] [MAIN ] AFGD: Generated 1791 derivative records from 1775 primary records. [2025-10-20T22:04:52.812675] [INFO ] [MAIN ] Derivative Progress: 31/1877 symbols completed. AFGB took 1.1s. Estimated remaining: 12m 31s. Rate: 147.4 symbols/min [2025-10-20T22:04:52.816375] [DEBUG] [LOCAL ] AFGD: Saving derivative data. [2025-10-20T22:04:52.816701] [INFO ] [MAIN ] AFGE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:52.816456] [DEBUG] [LOCAL ] AFGC: Derivative data saved (1794 records). [2025-10-20T22:04:52.817831] [INFO ] [MAIN ] AFGC: Successfully saved derivative data. [2025-10-20T22:04:52.820292] [INFO ] [MAIN ] Derivative Progress: 32/1877 symbols completed. AFGC took 1.1s. Estimated remaining: 12m 8s. Rate: 152.1 symbols/min [2025-10-20T22:04:52.820623] [INFO ] [MAIN ] AFL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:52.834767] [DEBUG] [LOCAL ] AFGD: Derivative data saved (1791 records). [2025-10-20T22:04:52.835527] [INFO ] [MAIN ] AFGD: Successfully saved derivative data. [2025-10-20T22:04:52.838258] [INFO ] [MAIN ] Derivative Progress: 33/1877 symbols completed. AFGD took 1.1s. Estimated remaining: 11m 47s. Rate: 156.6 symbols/min [2025-10-20T22:04:52.838532] [INFO ] [MAIN ] AG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:52.841785] [DEBUG] [LOCAL ] AFGE: Loaded 2313 existing records from primary storage. [2025-10-20T22:04:52.842237] [INFO ] [MAIN ] AFGE: Loaded 2313 primary data records. [2025-10-20T22:04:53.196535] [DEBUG] [LOCAL ] AFL: Loaded 53182 existing records from primary storage. [2025-10-20T22:04:53.212466] [INFO ] [MAIN ] AFL: Loaded 53182 primary data records. [2025-10-20T22:04:53.213372] [DEBUG] [LOCAL ] AG: Loaded 53644 existing records from primary storage. [2025-10-20T22:04:53.236063] [INFO ] [MAIN ] AG: Loaded 53644 primary data records. [2025-10-20T22:04:53.546611] [DEBUG] [MAIN ] AFGE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:53.702209] [INFO ] [MAIN ] AFGE: Generated 1794 derivative records from 2313 primary records. [2025-10-20T22:04:53.731105] [DEBUG] [LOCAL ] AFGE: Saving derivative data. [2025-10-20T22:04:53.865727] [DEBUG] [LOCAL ] AFGE: Derivative data saved (1794 records). [2025-10-20T22:04:53.865947] [INFO ] [MAIN ] AFGE: Successfully saved derivative data. [2025-10-20T22:04:53.868340] [INFO ] [MAIN ] Derivative Progress: 34/1877 symbols completed. AFGE took 1.1s. Estimated remaining: 12m 21s. Rate: 149.2 symbols/min [2025-10-20T22:04:53.868790] [INFO ] [MAIN ] AGCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:53.919919] [DEBUG] [MAIN ] AFL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:53.920292] [DEBUG] [MAIN ] AG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:53.921211] [INFO ] [MAIN ] AG: Generated 1794 derivative records from 53644 primary records. [2025-10-20T22:04:53.922154] [INFO ] [MAIN ] AFL: Generated 1794 derivative records from 53182 primary records. [2025-10-20T22:04:53.922441] [DEBUG] [LOCAL ] AFL: Saving derivative data. [2025-10-20T22:04:53.922289] [DEBUG] [LOCAL ] AG: Saving derivative data. [2025-10-20T22:04:53.946880] [DEBUG] [LOCAL ] AG: Derivative data saved (1794 records). [2025-10-20T22:04:53.948106] [INFO ] [MAIN ] AG: Successfully saved derivative data. [2025-10-20T22:04:53.950361] [INFO ] [MAIN ] Derivative Progress: 35/1877 symbols completed. AG took 1.1s. Estimated remaining: 12m 4s. Rate: 152.7 symbols/min [2025-10-20T22:04:53.954995] [INFO ] [MAIN ] AGD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:53.959032] [DEBUG] [LOCAL ] AFL: Derivative data saved (1794 records). [2025-10-20T22:04:53.973340] [INFO ] [MAIN ] AFL: Successfully saved derivative data. [2025-10-20T22:04:53.977847] [DEBUG] [LOCAL ] AGCO: Loaded 42303 existing records from primary storage. [2025-10-20T22:04:53.976723] [INFO ] [MAIN ] Derivative Progress: 36/1877 symbols completed. AFL took 1.2s. Estimated remaining: 11m 45s. Rate: 156.7 symbols/min [2025-10-20T22:04:53.978124] [INFO ] [MAIN ] AGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:53.978573] [INFO ] [MAIN ] AGCO: Loaded 42303 primary data records. [2025-10-20T22:04:54.112565] [DEBUG] [LOCAL ] AGD: Loaded 19307 existing records from primary storage. [2025-10-20T22:04:54.117972] [INFO ] [MAIN ] AGD: Loaded 19307 primary data records. [2025-10-20T22:04:54.649578] [DEBUG] [LOCAL ] AGI: Loaded 53423 existing records from primary storage. [2025-10-20T22:04:54.650455] [INFO ] [MAIN ] AGI: Loaded 53423 primary data records. [2025-10-20T22:04:54.975707] [DEBUG] [MAIN ] AGCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:54.976634] [INFO ] [MAIN ] AGCO: Generated 1794 derivative records from 42303 primary records. [2025-10-20T22:04:54.977211] [DEBUG] [LOCAL ] AGCO: Saving derivative data. [2025-10-20T22:04:54.998326] [DEBUG] [LOCAL ] AGCO: Derivative data saved (1794 records). [2025-10-20T22:04:55.006520] [INFO ] [MAIN ] AGCO: Successfully saved derivative data. [2025-10-20T22:04:55.008776] [INFO ] [MAIN ] Derivative Progress: 37/1877 symbols completed. AGCO took 1.1s. Estimated remaining: 12m 17s. Rate: 149.9 symbols/min [2025-10-20T22:04:55.013505] [INFO ] [MAIN ] AGM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:55.014902] [DEBUG] [MAIN ] AGD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:55.017005] [INFO ] [MAIN ] AGD: Generated 1794 derivative records from 19307 primary records. [2025-10-20T22:04:55.017290] [DEBUG] [LOCAL ] AGD: Saving derivative data. [2025-10-20T22:04:55.033764] [DEBUG] [LOCAL ] AGD: Derivative data saved (1794 records). [2025-10-20T22:04:55.033972] [INFO ] [MAIN ] AGD: Successfully saved derivative data. [2025-10-20T22:04:55.039241] [INFO ] [MAIN ] Derivative Progress: 38/1877 symbols completed. AGD took 1.1s. Estimated remaining: 11m 58s. Rate: 153.6 symbols/min [2025-10-20T22:04:55.039441] [INFO ] [MAIN ] AGO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:55.048584] [DEBUG] [LOCAL ] AGM: Loaded 10702 existing records from primary storage. [2025-10-20T22:04:55.048990] [INFO ] [MAIN ] AGM: Loaded 10702 primary data records. [2025-10-20T22:04:55.100100] [DEBUG] [MAIN ] AGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:55.141500] [INFO ] [MAIN ] AGI: Generated 1794 derivative records from 53423 primary records. [2025-10-20T22:04:55.165352] [DEBUG] [LOCAL ] AGI: Saving derivative data. [2025-10-20T22:04:55.186880] [DEBUG] [LOCAL ] AGO: Loaded 27832 existing records from primary storage. [2025-10-20T22:04:55.214122] [INFO ] [MAIN ] AGO: Loaded 27832 primary data records. [2025-10-20T22:04:55.228829] [DEBUG] [LOCAL ] AGI: Derivative data saved (1794 records). [2025-10-20T22:04:55.263586] [INFO ] [MAIN ] AGI: Successfully saved derivative data. [2025-10-20T22:04:55.315137] [INFO ] [MAIN ] Derivative Progress: 39/1877 symbols completed. AGI took 1.3s. Estimated remaining: 11m 53s. Rate: 154.8 symbols/min [2025-10-20T22:04:55.455758] [INFO ] [MAIN ] AGX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:55.735318] [DEBUG] [LOCAL ] AGX: Loaded 32688 existing records from primary storage. [2025-10-20T22:04:55.736708] [INFO ] [MAIN ] AGX: Loaded 32688 primary data records. [2025-10-20T22:04:55.834944] [DEBUG] [MAIN ] AGM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:55.857132] [INFO ] [MAIN ] AGM: Generated 1794 derivative records from 10702 primary records. [2025-10-20T22:04:55.857579] [DEBUG] [LOCAL ] AGM: Saving derivative data. [2025-10-20T22:04:55.895366] [DEBUG] [LOCAL ] AGM: Derivative data saved (1794 records). [2025-10-20T22:04:55.900598] [INFO ] [MAIN ] AGM: Successfully saved derivative data. [2025-10-20T22:04:55.917160] [INFO ] [MAIN ] Derivative Progress: 40/1877 symbols completed. AGM took 0.9s. Estimated remaining: 12m 2s. Rate: 152.6 symbols/min [2025-10-20T22:04:55.926249] [INFO ] [MAIN ] AHH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:56.088225] [DEBUG] [MAIN ] AGO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:56.089178] [INFO ] [MAIN ] AGO: Generated 1794 derivative records from 27832 primary records. [2025-10-20T22:04:56.089293] [DEBUG] [LOCAL ] AGO: Saving derivative data. [2025-10-20T22:04:56.107688] [DEBUG] [LOCAL ] AGO: Derivative data saved (1794 records). [2025-10-20T22:04:56.107968] [INFO ] [MAIN ] AGO: Successfully saved derivative data. [2025-10-20T22:04:56.110395] [INFO ] [MAIN ] Derivative Progress: 41/1877 symbols completed. AGO took 1.1s. Estimated remaining: 11m 53s. Rate: 154.6 symbols/min [2025-10-20T22:04:56.110738] [INFO ] [MAIN ] AHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:56.141849] [DEBUG] [LOCAL ] AHH: Loaded 40379 existing records from primary storage. [2025-10-20T22:04:56.150004] [INFO ] [MAIN ] AHH: Loaded 40379 primary data records. [2025-10-20T22:04:56.185883] [DEBUG] [MAIN ] AGX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:56.230471] [INFO ] [MAIN ] AGX: Generated 1794 derivative records from 32688 primary records. [2025-10-20T22:04:56.235879] [DEBUG] [LOCAL ] AGX: Saving derivative data. [2025-10-20T22:04:56.291478] [DEBUG] [LOCAL ] AGX: Derivative data saved (1794 records). [2025-10-20T22:04:56.316716] [INFO ] [MAIN ] AGX: Successfully saved derivative data. [2025-10-20T22:04:56.342260] [INFO ] [MAIN ] Derivative Progress: 42/1877 symbols completed. AGX took 0.9s. Estimated remaining: 11m 45s. Rate: 156.1 symbols/min [2025-10-20T22:04:56.351314] [INFO ] [MAIN ] AHT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:56.362816] [DEBUG] [LOCAL ] AHR: Loaded 49006 existing records from primary storage. [2025-10-20T22:04:56.377618] [INFO ] [MAIN ] AHR: Loaded 49006 primary data records. [2025-10-20T22:04:56.592778] [DEBUG] [LOCAL ] AHT: Loaded 6438 existing records from primary storage. [2025-10-20T22:04:56.607111] [INFO ] [MAIN ] AHT: Loaded 6438 primary data records. [2025-10-20T22:04:57.190747] [DEBUG] [MAIN ] AHH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:57.193104] [INFO ] [MAIN ] AHH: Generated 1794 derivative records from 40379 primary records. [2025-10-20T22:04:57.193414] [DEBUG] [LOCAL ] AHH: Saving derivative data. [2025-10-20T22:04:57.213031] [DEBUG] [LOCAL ] AHH: Derivative data saved (1794 records). [2025-10-20T22:04:57.213496] [INFO ] [MAIN ] AHH: Successfully saved derivative data. [2025-10-20T22:04:57.215551] [INFO ] [MAIN ] Derivative Progress: 43/1877 symbols completed. AHH took 1.3s. Estimated remaining: 12m 6s. Rate: 151.6 symbols/min [2025-10-20T22:04:57.229360] [DEBUG] [MAIN ] AHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:57.230189] [INFO ] [MAIN ] AHR: Generated 1794 derivative records from 49006 primary records. [2025-10-20T22:04:57.230422] [DEBUG] [LOCAL ] AHR: Saving derivative data. [2025-10-20T22:04:57.247866] [DEBUG] [LOCAL ] AHR: Derivative data saved (1794 records). [2025-10-20T22:04:57.249257] [INFO ] [MAIN ] AHR: Successfully saved derivative data. [2025-10-20T22:04:57.252589] [INFO ] [MAIN ] Derivative Progress: 44/1877 symbols completed. AHR took 1.1s. Estimated remaining: 11m 51s. Rate: 154.8 symbols/min [2025-10-20T22:04:57.265845] [DEBUG] [MAIN ] AHT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:57.266669] [INFO ] [MAIN ] AHT: Generated 1794 derivative records from 6438 primary records. [2025-10-20T22:04:57.266738] [DEBUG] [LOCAL ] AHT: Saving derivative data. [2025-10-20T22:04:57.281739] [DEBUG] [LOCAL ] AHT: Derivative data saved (1794 records). [2025-10-20T22:04:57.281807] [INFO ] [MAIN ] AHT: Successfully saved derivative data. [2025-10-20T22:04:57.285056] [INFO ] [MAIN ] Derivative Progress: 45/1877 symbols completed. AHT took 0.9s. Estimated remaining: 11m 36s. Rate: 158.0 symbols/min [2025-10-20T22:04:57.285686] [INFO ] [MAIN ] AIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:57.286283] [INFO ] [MAIN ] AIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:57.285568] [INFO ] [MAIN ] AIG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:57.304147] [DEBUG] [LOCAL ] AIO: Loaded 13123 existing records from primary storage. [2025-10-20T22:04:57.311119] [INFO ] [MAIN ] AIO: Loaded 13123 primary data records. [2025-10-20T22:04:57.419336] [DEBUG] [LOCAL ] AIN: Loaded 24466 existing records from primary storage. [2025-10-20T22:04:57.428009] [INFO ] [MAIN ] AIN: Loaded 24466 primary data records. [2025-10-20T22:04:57.785251] [DEBUG] [LOCAL ] AIG: Loaded 53465 existing records from primary storage. [2025-10-20T22:04:57.796689] [INFO ] [MAIN ] AIG: Loaded 53465 primary data records. [2025-10-20T22:04:58.389893] [DEBUG] [MAIN ] AIO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:04:58.389325] [DEBUG] [MAIN ] AIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:58.391138] [INFO ] [MAIN ] AIO: Generated 1793 derivative records from 13123 primary records. [2025-10-20T22:04:58.392363] [INFO ] [MAIN ] AIN: Generated 1794 derivative records from 24466 primary records. [2025-10-20T22:04:58.392743] [DEBUG] [LOCAL ] AIN: Saving derivative data. [2025-10-20T22:04:58.392616] [DEBUG] [LOCAL ] AIO: Saving derivative data. [2025-10-20T22:04:58.428681] [DEBUG] [LOCAL ] AIN: Derivative data saved (1794 records). [2025-10-20T22:04:58.430492] [INFO ] [MAIN ] AIN: Successfully saved derivative data. [2025-10-20T22:04:58.440524] [INFO ] [MAIN ] Derivative Progress: 46/1877 symbols completed. AIN took 1.2s. Estimated remaining: 12m 6s. Rate: 151.3 symbols/min [2025-10-20T22:04:58.430126] [DEBUG] [LOCAL ] AIO: Derivative data saved (1793 records). [2025-10-20T22:04:58.445823] [INFO ] [MAIN ] AIO: Successfully saved derivative data. [2025-10-20T22:04:58.446151] [DEBUG] [MAIN ] AIG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:58.446347] [INFO ] [MAIN ] AIR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:58.449692] [INFO ] [MAIN ] Derivative Progress: 47/1877 symbols completed. AIO took 1.2s. Estimated remaining: 11m 51s. Rate: 154.5 symbols/min [2025-10-20T22:04:58.451218] [INFO ] [MAIN ] AIG: Generated 1794 derivative records from 53465 primary records. [2025-10-20T22:04:58.450390] [INFO ] [MAIN ] AIT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:58.451387] [DEBUG] [LOCAL ] AIG: Saving derivative data. [2025-10-20T22:04:58.468805] [DEBUG] [LOCAL ] AIG: Derivative data saved (1794 records). [2025-10-20T22:04:58.468974] [INFO ] [MAIN ] AIG: Successfully saved derivative data. [2025-10-20T22:04:58.471269] [INFO ] [MAIN ] Derivative Progress: 48/1877 symbols completed. AIG took 1.2s. Estimated remaining: 11m 36s. Rate: 157.6 symbols/min [2025-10-20T22:04:58.471470] [INFO ] [MAIN ] AIV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:58.499751] [DEBUG] [LOCAL ] AIR: Loaded 28742 existing records from primary storage. [2025-10-20T22:04:58.499887] [INFO ] [MAIN ] AIR: Loaded 28742 primary data records. [2025-10-20T22:04:58.623028] [DEBUG] [LOCAL ] AIT: Loaded 26804 existing records from primary storage. [2025-10-20T22:04:58.628955] [INFO ] [MAIN ] AIT: Loaded 26804 primary data records. [2025-10-20T22:04:58.750698] [DEBUG] [LOCAL ] AIV: Loaded 41799 existing records from primary storage. [2025-10-20T22:04:58.857007] [INFO ] [MAIN ] AIV: Loaded 41799 primary data records. [2025-10-20T22:04:59.470538] [DEBUG] [MAIN ] AIR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:59.471580] [INFO ] [MAIN ] AIR: Generated 1794 derivative records from 28742 primary records. [2025-10-20T22:04:59.472020] [DEBUG] [LOCAL ] AIR: Saving derivative data. [2025-10-20T22:04:59.488872] [DEBUG] [LOCAL ] AIR: Derivative data saved (1794 records). [2025-10-20T22:04:59.489634] [INFO ] [MAIN ] AIR: Successfully saved derivative data. [2025-10-20T22:04:59.492918] [INFO ] [MAIN ] Derivative Progress: 49/1877 symbols completed. AIR took 1.0s. Estimated remaining: 11m 60s. Rate: 152.3 symbols/min [2025-10-20T22:04:59.495145] [INFO ] [MAIN ] AIZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:59.512481] [DEBUG] [MAIN ] AIT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:59.513454] [INFO ] [MAIN ] AIT: Generated 1794 derivative records from 26804 primary records. [2025-10-20T22:04:59.513667] [DEBUG] [LOCAL ] AIT: Saving derivative data. [2025-10-20T22:04:59.534408] [DEBUG] [MAIN ] AIV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:04:59.542550] [INFO ] [MAIN ] AIV: Generated 1794 derivative records from 41799 primary records. [2025-10-20T22:04:59.543832] [DEBUG] [LOCAL ] AIT: Derivative data saved (1794 records). [2025-10-20T22:04:59.543990] [INFO ] [MAIN ] AIT: Successfully saved derivative data. [2025-10-20T22:04:59.546336] [DEBUG] [LOCAL ] AIV: Saving derivative data. [2025-10-20T22:04:59.546235] [INFO ] [MAIN ] Derivative Progress: 50/1877 symbols completed. AIT took 1.1s. Estimated remaining: 11m 47s. Rate: 155.0 symbols/min [2025-10-20T22:04:59.547335] [INFO ] [MAIN ] AIZN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:59.571268] [DEBUG] [LOCAL ] AIV: Derivative data saved (1794 records). [2025-10-20T22:04:59.571434] [INFO ] [MAIN ] AIV: Successfully saved derivative data. [2025-10-20T22:04:59.573538] [INFO ] [MAIN ] Derivative Progress: 51/1877 symbols completed. AIV took 1.1s. Estimated remaining: 11m 34s. Rate: 157.9 symbols/min [2025-10-20T22:04:59.574264] [INFO ] [MAIN ] AJG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:04:59.589349] [DEBUG] [LOCAL ] AIZ: Loaded 29471 existing records from primary storage. [2025-10-20T22:04:59.593019] [INFO ] [MAIN ] AIZ: Loaded 29471 primary data records. [2025-10-20T22:04:59.592822] [DEBUG] [LOCAL ] AIZN: Loaded 3848 existing records from primary storage. [2025-10-20T22:04:59.593377] [INFO ] [MAIN ] AIZN: Loaded 3848 primary data records. [2025-10-20T22:05:00.235507] [DEBUG] [LOCAL ] AJG: Loaded 46490 existing records from primary storage. [2025-10-20T22:05:00.244224] [INFO ] [MAIN ] AJG: Loaded 46490 primary data records. [2025-10-20T22:05:00.609742] [DEBUG] [MAIN ] AIZN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:00.610584] [INFO ] [MAIN ] AIZN: Generated 1794 derivative records from 3848 primary records. [2025-10-20T22:05:00.611441] [DEBUG] [LOCAL ] AIZN: Saving derivative data. [2025-10-20T22:05:00.635991] [DEBUG] [LOCAL ] AIZN: Derivative data saved (1794 records). [2025-10-20T22:05:00.641287] [INFO ] [MAIN ] AIZN: Successfully saved derivative data. [2025-10-20T22:05:00.643459] [INFO ] [MAIN ] Derivative Progress: 52/1877 symbols completed. AIZN took 1.1s. Estimated remaining: 11m 58s. Rate: 152.6 symbols/min [2025-10-20T22:05:00.641194] [DEBUG] [MAIN ] AIZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:00.644616] [INFO ] [MAIN ] AIZ: Generated 1794 derivative records from 29471 primary records. [2025-10-20T22:05:00.644873] [DEBUG] [LOCAL ] AIZ: Saving derivative data. [2025-10-20T22:05:00.643716] [INFO ] [MAIN ] AKR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:00.661153] [DEBUG] [LOCAL ] AIZ: Derivative data saved (1794 records). [2025-10-20T22:05:00.661227] [INFO ] [MAIN ] AIZ: Successfully saved derivative data. [2025-10-20T22:05:00.663293] [INFO ] [MAIN ] Derivative Progress: 53/1877 symbols completed. AIZ took 1.2s. Estimated remaining: 11m 44s. Rate: 155.4 symbols/min [2025-10-20T22:05:00.663566] [INFO ] [MAIN ] AL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:00.681580] [DEBUG] [MAIN ] AJG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:00.682407] [INFO ] [MAIN ] AJG: Generated 1794 derivative records from 46490 primary records. [2025-10-20T22:05:00.682511] [DEBUG] [LOCAL ] AJG: Saving derivative data. [2025-10-20T22:05:00.700070] [DEBUG] [LOCAL ] AJG: Derivative data saved (1794 records). [2025-10-20T22:05:00.700420] [INFO ] [MAIN ] AJG: Successfully saved derivative data. [2025-10-20T22:05:00.703747] [INFO ] [MAIN ] Derivative Progress: 54/1877 symbols completed. AJG took 1.1s. Estimated remaining: 11m 32s. Rate: 158.0 symbols/min [2025-10-20T22:05:00.704435] [INFO ] [MAIN ] ALB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:00.761773] [DEBUG] [LOCAL ] AKR: Loaded 46127 existing records from primary storage. [2025-10-20T22:05:00.762003] [INFO ] [MAIN ] AKR: Loaded 46127 primary data records. [2025-10-20T22:05:00.760829] [DEBUG] [LOCAL ] AL: Loaded 44827 existing records from primary storage. [2025-10-20T22:05:00.762109] [INFO ] [MAIN ] AL: Loaded 44827 primary data records. [2025-10-20T22:05:01.411026] [DEBUG] [LOCAL ] ALB: Loaded 53378 existing records from primary storage. [2025-10-20T22:05:01.423163] [INFO ] [MAIN ] ALB: Loaded 53378 primary data records. [2025-10-20T22:05:01.771662] [DEBUG] [MAIN ] AKR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:01.773031] [INFO ] [MAIN ] AKR: Generated 1794 derivative records from 46127 primary records. [2025-10-20T22:05:01.773562] [DEBUG] [LOCAL ] AKR: Saving derivative data. [2025-10-20T22:05:01.808589] [DEBUG] [LOCAL ] AKR: Derivative data saved (1794 records). [2025-10-20T22:05:01.810098] [INFO ] [MAIN ] AKR: Successfully saved derivative data. [2025-10-20T22:05:01.810697] [DEBUG] [MAIN ] AL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:01.814686] [INFO ] [MAIN ] AL: Generated 1794 derivative records from 44827 primary records. [2025-10-20T22:05:01.815061] [DEBUG] [LOCAL ] AL: Saving derivative data. [2025-10-20T22:05:01.813551] [INFO ] [MAIN ] Derivative Progress: 55/1877 symbols completed. AKR took 1.2s. Estimated remaining: 11m 56s. Rate: 152.6 symbols/min [2025-10-20T22:05:01.816159] [INFO ] [MAIN ] ALC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:01.835985] [DEBUG] [LOCAL ] AL: Derivative data saved (1794 records). [2025-10-20T22:05:01.836484] [INFO ] [MAIN ] AL: Successfully saved derivative data. [2025-10-20T22:05:01.838883] [INFO ] [MAIN ] Derivative Progress: 56/1877 symbols completed. AL took 1.2s. Estimated remaining: 11m 44s. Rate: 155.2 symbols/min [2025-10-20T22:05:01.839098] [INFO ] [MAIN ] ALE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:01.856560] [DEBUG] [MAIN ] ALB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:01.857416] [INFO ] [MAIN ] ALB: Generated 1794 derivative records from 53378 primary records. [2025-10-20T22:05:01.857538] [DEBUG] [LOCAL ] ALB: Saving derivative data. [2025-10-20T22:05:01.875916] [DEBUG] [LOCAL ] ALB: Derivative data saved (1794 records). [2025-10-20T22:05:01.876092] [INFO ] [MAIN ] ALB: Successfully saved derivative data. [2025-10-20T22:05:01.879206] [INFO ] [MAIN ] Derivative Progress: 57/1877 symbols completed. ALB took 1.2s. Estimated remaining: 11m 32s. Rate: 157.7 symbols/min [2025-10-20T22:05:01.879543] [INFO ] [MAIN ] ALEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:01.919369] [DEBUG] [LOCAL ] ALE: Loaded 32104 existing records from primary storage. [2025-10-20T22:05:01.934613] [INFO ] [MAIN ] ALE: Loaded 32104 primary data records. [2025-10-20T22:05:01.950884] [DEBUG] [LOCAL ] ALC: Loaded 51212 existing records from primary storage. [2025-10-20T22:05:02.014423] [INFO ] [MAIN ] ALC: Loaded 51212 primary data records. [2025-10-20T22:05:02.101077] [DEBUG] [LOCAL ] ALEX: Loaded 31005 existing records from primary storage. [2025-10-20T22:05:02.145298] [INFO ] [MAIN ] ALEX: Loaded 31005 primary data records. [2025-10-20T22:05:02.948002] [DEBUG] [MAIN ] ALE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:02.948892] [INFO ] [MAIN ] ALE: Generated 1794 derivative records from 32104 primary records. [2025-10-20T22:05:02.949081] [DEBUG] [LOCAL ] ALE: Saving derivative data. [2025-10-20T22:05:02.971184] [DEBUG] [MAIN ] ALC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:02.974581] [INFO ] [MAIN ] ALC: Generated 1794 derivative records from 51212 primary records. [2025-10-20T22:05:02.974838] [DEBUG] [LOCAL ] ALC: Saving derivative data. [2025-10-20T22:05:02.980327] [DEBUG] [LOCAL ] ALE: Derivative data saved (1794 records). [2025-10-20T22:05:02.987717] [INFO ] [MAIN ] ALE: Successfully saved derivative data. [2025-10-20T22:05:02.993675] [INFO ] [MAIN ] Derivative Progress: 58/1877 symbols completed. ALE took 1.2s. Estimated remaining: 11m 55s. Rate: 152.6 symbols/min [2025-10-20T22:05:02.999260] [DEBUG] [LOCAL ] ALC: Derivative data saved (1794 records). [2025-10-20T22:05:03.000095] [INFO ] [MAIN ] ALC: Successfully saved derivative data. [2025-10-20T22:05:03.002964] [INFO ] [MAIN ] Derivative Progress: 59/1877 symbols completed. ALC took 1.2s. Estimated remaining: 11m 43s. Rate: 155.2 symbols/min [2025-10-20T22:05:03.015388] [DEBUG] [MAIN ] ALEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:03.015949] [INFO ] [MAIN ] ALEX: Generated 1794 derivative records from 31005 primary records. [2025-10-20T22:05:03.016008] [DEBUG] [LOCAL ] ALEX: Saving derivative data. [2025-10-20T22:05:03.033710] [DEBUG] [LOCAL ] ALEX: Derivative data saved (1794 records). [2025-10-20T22:05:03.033803] [INFO ] [MAIN ] ALEX: Successfully saved derivative data. [2025-10-20T22:05:03.036135] [INFO ] [MAIN ] Derivative Progress: 60/1877 symbols completed. ALEX took 1.2s. Estimated remaining: 11m 32s. Rate: 157.6 symbols/min [2025-10-20T22:05:03.036474] [INFO ] [MAIN ] ALG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:03.036605] [INFO ] [MAIN ] ALIT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:03.036875] [INFO ] [MAIN ] ALK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:03.053807] [DEBUG] [LOCAL ] ALG: Loaded 9748 existing records from primary storage. [2025-10-20T22:05:03.054022] [INFO ] [MAIN ] ALG: Loaded 9748 primary data records. [2025-10-20T22:05:03.296936] [DEBUG] [LOCAL ] ALIT: Loaded 52790 existing records from primary storage. [2025-10-20T22:05:03.434766] [DEBUG] [LOCAL ] ALK: Loaded 53085 existing records from primary storage. [2025-10-20T22:05:03.433575] [INFO ] [MAIN ] ALIT: Loaded 52790 primary data records. [2025-10-20T22:05:03.435107] [INFO ] [MAIN ] ALK: Loaded 53085 primary data records. [2025-10-20T22:05:04.055336] [DEBUG] [MAIN ] ALG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:04.056831] [INFO ] [MAIN ] ALG: Generated 1794 derivative records from 9748 primary records. [2025-10-20T22:05:04.057238] [DEBUG] [LOCAL ] ALG: Saving derivative data. [2025-10-20T22:05:04.075772] [DEBUG] [LOCAL ] ALG: Derivative data saved (1794 records). [2025-10-20T22:05:04.075920] [INFO ] [MAIN ] ALG: Successfully saved derivative data. [2025-10-20T22:05:04.078464] [INFO ] [MAIN ] Derivative Progress: 61/1877 symbols completed. ALG took 1.0s. Estimated remaining: 11m 51s. Rate: 153.2 symbols/min [2025-10-20T22:05:04.078831] [INFO ] [MAIN ] ALL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:04.094426] [DEBUG] [MAIN ] ALIT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:04.095290] [INFO ] [MAIN ] ALIT: Generated 1794 derivative records from 52790 primary records. [2025-10-20T22:05:04.095493] [DEBUG] [LOCAL ] ALIT: Saving derivative data. [2025-10-20T22:05:04.108544] [DEBUG] [MAIN ] ALK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:04.117236] [INFO ] [MAIN ] ALK: Generated 1794 derivative records from 53085 primary records. [2025-10-20T22:05:04.119929] [DEBUG] [LOCAL ] ALK: Saving derivative data. [2025-10-20T22:05:04.123960] [DEBUG] [LOCAL ] ALIT: Derivative data saved (1794 records). [2025-10-20T22:05:04.124154] [INFO ] [MAIN ] ALIT: Successfully saved derivative data. [2025-10-20T22:05:04.125992] [INFO ] [MAIN ] Derivative Progress: 62/1877 symbols completed. ALIT took 1.1s. Estimated remaining: 11m 41s. Rate: 155.4 symbols/min [2025-10-20T22:05:04.126889] [INFO ] [MAIN ] ALLE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:04.142607] [DEBUG] [LOCAL ] ALK: Derivative data saved (1794 records). [2025-10-20T22:05:04.142804] [INFO ] [MAIN ] ALK: Successfully saved derivative data. [2025-10-20T22:05:04.145137] [INFO ] [MAIN ] Derivative Progress: 63/1877 symbols completed. ALK took 1.1s. Estimated remaining: 11m 30s. Rate: 157.8 symbols/min [2025-10-20T22:05:04.145354] [INFO ] [MAIN ] ALLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:04.180369] [DEBUG] [LOCAL ] ALL: Loaded 49721 existing records from primary storage. [2025-10-20T22:05:04.180562] [INFO ] [MAIN ] ALL: Loaded 49721 primary data records. [2025-10-20T22:05:04.360468] [DEBUG] [LOCAL ] ALLE: Loaded 44026 existing records from primary storage. [2025-10-20T22:05:04.400730] [INFO ] [MAIN ] ALLE: Loaded 44026 primary data records. [2025-10-20T22:05:04.567268] [DEBUG] [LOCAL ] ALLY: Loaded 53224 existing records from primary storage. [2025-10-20T22:05:04.574073] [INFO ] [MAIN ] ALLY: Loaded 53224 primary data records. [2025-10-20T22:05:05.197667] [DEBUG] [MAIN ] ALL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:05.199146] [INFO ] [MAIN ] ALL: Generated 1794 derivative records from 49721 primary records. [2025-10-20T22:05:05.199246] [DEBUG] [LOCAL ] ALL: Saving derivative data. [2025-10-20T22:05:05.217491] [DEBUG] [LOCAL ] ALL: Derivative data saved (1794 records). [2025-10-20T22:05:05.217885] [INFO ] [MAIN ] ALL: Successfully saved derivative data. [2025-10-20T22:05:05.220081] [INFO ] [MAIN ] Derivative Progress: 64/1877 symbols completed. ALL took 1.1s. Estimated remaining: 11m 49s. Rate: 153.4 symbols/min [2025-10-20T22:05:05.220349] [INFO ] [MAIN ] ALSN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:05.254516] [DEBUG] [MAIN ] ALLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:05.254128] [DEBUG] [MAIN ] ALLE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:05.255408] [INFO ] [MAIN ] ALLY: Generated 1794 derivative records from 53224 primary records. [2025-10-20T22:05:05.256257] [INFO ] [MAIN ] ALLE: Generated 1794 derivative records from 44026 primary records. [2025-10-20T22:05:05.255494] [DEBUG] [LOCAL ] ALLY: Saving derivative data. [2025-10-20T22:05:05.256913] [DEBUG] [LOCAL ] ALLE: Saving derivative data. [2025-10-20T22:05:05.281829] [DEBUG] [LOCAL ] ALLY: Derivative data saved (1794 records). [2025-10-20T22:05:05.281942] [INFO ] [MAIN ] ALLY: Successfully saved derivative data. [2025-10-20T22:05:05.287788] [INFO ] [MAIN ] Derivative Progress: 65/1877 symbols completed. ALLY took 1.1s. Estimated remaining: 11m 40s. Rate: 155.4 symbols/min [2025-10-20T22:05:05.289606] [INFO ] [MAIN ] ALTG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:05.291503] [DEBUG] [LOCAL ] ALLE: Derivative data saved (1794 records). [2025-10-20T22:05:05.291973] [INFO ] [MAIN ] ALLE: Successfully saved derivative data. [2025-10-20T22:05:05.293828] [INFO ] [MAIN ] Derivative Progress: 66/1877 symbols completed. ALLE took 1.2s. Estimated remaining: 11m 29s. Rate: 157.8 symbols/min [2025-10-20T22:05:05.303475] [INFO ] [MAIN ] ALUR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:05.328194] [DEBUG] [LOCAL ] ALSN: Loaded 43332 existing records from primary storage. [2025-10-20T22:05:05.332450] [DEBUG] [LOCAL ] ALUR: Loaded 9345 existing records from primary storage. [2025-10-20T22:05:05.332572] [INFO ] [MAIN ] ALSN: Loaded 43332 primary data records. [2025-10-20T22:05:05.339275] [INFO ] [MAIN ] ALUR: Loaded 9345 primary data records. [2025-10-20T22:05:05.351936] [DEBUG] [LOCAL ] ALTG: Loaded 23275 existing records from primary storage. [2025-10-20T22:05:05.382403] [INFO ] [MAIN ] ALTG: Loaded 23275 primary data records. [2025-10-20T22:05:06.296793] [DEBUG] [MAIN ] ALSN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:06.299513] [INFO ] [MAIN ] ALSN: Generated 1794 derivative records from 43332 primary records. [2025-10-20T22:05:06.299810] [DEBUG] [LOCAL ] ALSN: Saving derivative data. [2025-10-20T22:05:06.330124] [DEBUG] [LOCAL ] ALSN: Derivative data saved (1794 records). [2025-10-20T22:05:06.328089] [DEBUG] [MAIN ] ALUR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:06.331106] [INFO ] [MAIN ] ALUR: Generated 1794 derivative records from 9345 primary records. [2025-10-20T22:05:06.331226] [DEBUG] [LOCAL ] ALUR: Saving derivative data. [2025-10-20T22:05:06.331915] [INFO ] [MAIN ] ALSN: Successfully saved derivative data. [2025-10-20T22:05:06.353318] [DEBUG] [MAIN ] ALTG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:06.346404] [INFO ] [MAIN ] Derivative Progress: 67/1877 symbols completed. ALSN took 1.1s. Estimated remaining: 11m 46s. Rate: 153.7 symbols/min [2025-10-20T22:05:06.361014] [DEBUG] [LOCAL ] ALUR: Derivative data saved (1794 records). [2025-10-20T22:05:06.361788] [INFO ] [MAIN ] ALTG: Generated 1794 derivative records from 23275 primary records. [2025-10-20T22:05:06.360186] [INFO ] [MAIN ] ALV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:06.361945] [INFO ] [MAIN ] ALUR: Successfully saved derivative data. [2025-10-20T22:05:06.364096] [INFO ] [MAIN ] Derivative Progress: 68/1877 symbols completed. ALUR took 1.1s. Estimated remaining: 11m 36s. Rate: 155.9 symbols/min [2025-10-20T22:05:06.362031] [DEBUG] [LOCAL ] ALTG: Saving derivative data. [2025-10-20T22:05:06.365307] [INFO ] [MAIN ] ALX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:06.380449] [DEBUG] [LOCAL ] ALTG: Derivative data saved (1794 records). [2025-10-20T22:05:06.380670] [INFO ] [MAIN ] ALTG: Successfully saved derivative data. [2025-10-20T22:05:06.382708] [INFO ] [MAIN ] Derivative Progress: 69/1877 symbols completed. ALTG took 1.1s. Estimated remaining: 11m 26s. Rate: 158.1 symbols/min [2025-10-20T22:05:06.384706] [INFO ] [MAIN ] AM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:06.389278] [DEBUG] [LOCAL ] ALX: Loaded 7183 existing records from primary storage. [2025-10-20T22:05:06.389502] [INFO ] [MAIN ] ALX: Loaded 7183 primary data records. [2025-10-20T22:05:06.731693] [DEBUG] [LOCAL ] ALV: Loaded 42190 existing records from primary storage. [2025-10-20T22:05:06.732994] [INFO ] [MAIN ] ALV: Loaded 42190 primary data records. [2025-10-20T22:05:06.865479] [DEBUG] [LOCAL ] AM: Loaded 51887 existing records from primary storage. [2025-10-20T22:05:06.877855] [INFO ] [MAIN ] AM: Loaded 51887 primary data records. [2025-10-20T22:05:07.289110] [DEBUG] [MAIN ] ALX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:07.416164] [INFO ] [MAIN ] ALX: Generated 1794 derivative records from 7183 primary records. [2025-10-20T22:05:07.416846] [DEBUG] [LOCAL ] ALX: Saving derivative data. [2025-10-20T22:05:07.432811] [DEBUG] [LOCAL ] ALX: Derivative data saved (1794 records). [2025-10-20T22:05:07.433782] [INFO ] [MAIN ] ALX: Successfully saved derivative data. [2025-10-20T22:05:07.436208] [INFO ] [MAIN ] Derivative Progress: 70/1877 symbols completed. ALX took 1.1s. Estimated remaining: 11m 43s. Rate: 154.2 symbols/min [2025-10-20T22:05:07.436710] [INFO ] [MAIN ] AMBC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:07.461002] [DEBUG] [MAIN ] ALV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:07.462140] [INFO ] [MAIN ] ALV: Generated 1794 derivative records from 42190 primary records. [2025-10-20T22:05:07.462226] [DEBUG] [LOCAL ] ALV: Saving derivative data. [2025-10-20T22:05:07.494927] [DEBUG] [LOCAL ] ALV: Derivative data saved (1794 records). [2025-10-20T22:05:07.495332] [INFO ] [MAIN ] ALV: Successfully saved derivative data. [2025-10-20T22:05:07.497226] [INFO ] [MAIN ] Derivative Progress: 71/1877 symbols completed. ALV took 1.1s. Estimated remaining: 11m 34s. Rate: 156.0 symbols/min [2025-10-20T22:05:07.521933] [INFO ] [MAIN ] AMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:07.525332] [DEBUG] [LOCAL ] AMBC: Loaded 36189 existing records from primary storage. [2025-10-20T22:05:07.525440] [INFO ] [MAIN ] AMBC: Loaded 36189 primary data records. [2025-10-20T22:05:07.524814] [DEBUG] [MAIN ] AM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:07.538241] [INFO ] [MAIN ] AM: Generated 1794 derivative records from 51887 primary records. [2025-10-20T22:05:07.547992] [DEBUG] [LOCAL ] AM: Saving derivative data. [2025-10-20T22:05:07.635427] [DEBUG] [LOCAL ] AM: Derivative data saved (1794 records). [2025-10-20T22:05:07.640888] [INFO ] [MAIN ] AM: Successfully saved derivative data. [2025-10-20T22:05:07.661766] [INFO ] [MAIN ] Derivative Progress: 72/1877 symbols completed. AM took 1.3s. Estimated remaining: 11m 29s. Rate: 157.3 symbols/min [2025-10-20T22:05:07.679705] [INFO ] [MAIN ] AMCR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:07.895510] [DEBUG] [LOCAL ] AMC: Loaded 53766 existing records from primary storage. [2025-10-20T22:05:07.896387] [INFO ] [MAIN ] AMC: Loaded 53766 primary data records. [2025-10-20T22:05:08.226305] [DEBUG] [LOCAL ] AMCR: Loaded 53670 existing records from primary storage. [2025-10-20T22:05:08.226955] [INFO ] [MAIN ] AMCR: Loaded 53670 primary data records. [2025-10-20T22:05:08.349842] [DEBUG] [MAIN ] AMBC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:08.350844] [INFO ] [MAIN ] AMBC: Generated 1794 derivative records from 36189 primary records. [2025-10-20T22:05:08.361613] [DEBUG] [LOCAL ] AMBC: Saving derivative data. [2025-10-20T22:05:08.591608] [DEBUG] [LOCAL ] AMBC: Derivative data saved (1794 records). [2025-10-20T22:05:08.597482] [INFO ] [MAIN ] AMBC: Successfully saved derivative data. [2025-10-20T22:05:08.607992] [INFO ] [MAIN ] Derivative Progress: 73/1877 symbols completed. AMBC took 1.2s. Estimated remaining: 11m 42s. Rate: 154.2 symbols/min [2025-10-20T22:05:08.629540] [DEBUG] [MAIN ] AMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:08.630676] [INFO ] [MAIN ] AMC: Generated 1794 derivative records from 53766 primary records. [2025-10-20T22:05:08.630914] [DEBUG] [LOCAL ] AMC: Saving derivative data. [2025-10-20T22:05:08.647517] [DEBUG] [LOCAL ] AMC: Derivative data saved (1794 records). [2025-10-20T22:05:08.647625] [INFO ] [MAIN ] AMC: Successfully saved derivative data. [2025-10-20T22:05:08.649557] [INFO ] [MAIN ] Derivative Progress: 74/1877 symbols completed. AMC took 1.1s. Estimated remaining: 11m 33s. Rate: 156.0 symbols/min [2025-10-20T22:05:08.664818] [DEBUG] [MAIN ] AMCR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:08.665455] [INFO ] [MAIN ] AMCR: Generated 1794 derivative records from 53670 primary records. [2025-10-20T22:05:08.665518] [DEBUG] [LOCAL ] AMCR: Saving derivative data. [2025-10-20T22:05:08.680862] [DEBUG] [LOCAL ] AMCR: Derivative data saved (1794 records). [2025-10-20T22:05:08.680950] [INFO ] [MAIN ] AMCR: Successfully saved derivative data. [2025-10-20T22:05:08.682898] [INFO ] [MAIN ] Derivative Progress: 75/1877 symbols completed. AMCR took 1.0s. Estimated remaining: 11m 24s. Rate: 158.0 symbols/min [2025-10-20T22:05:08.683347] [INFO ] [MAIN ] AME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:08.683541] [INFO ] [MAIN ] AMG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:08.683435] [INFO ] [MAIN ] AMH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:08.705186] [DEBUG] [LOCAL ] AMG: Loaded 20174 existing records from primary storage. [2025-10-20T22:05:08.705424] [INFO ] [MAIN ] AMG: Loaded 20174 primary data records. [2025-10-20T22:05:08.933269] [DEBUG] [LOCAL ] AME: Loaded 48570 existing records from primary storage. [2025-10-20T22:05:09.064927] [INFO ] [MAIN ] AME: Loaded 48570 primary data records. [2025-10-20T22:05:09.087843] [DEBUG] [LOCAL ] AMH: Loaded 51831 existing records from primary storage. [2025-10-20T22:05:09.104690] [INFO ] [MAIN ] AMH: Loaded 51831 primary data records. [2025-10-20T22:05:09.716710] [DEBUG] [MAIN ] AMG: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:09.717745] [INFO ] [MAIN ] AMG: Generated 1793 derivative records from 20174 primary records. [2025-10-20T22:05:09.718061] [DEBUG] [LOCAL ] AMG: Saving derivative data. [2025-10-20T22:05:09.762812] [DEBUG] [LOCAL ] AMG: Derivative data saved (1793 records). [2025-10-20T22:05:09.762990] [INFO ] [MAIN ] AMG: Successfully saved derivative data. [2025-10-20T22:05:09.759115] [DEBUG] [MAIN ] AME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:09.765780] [INFO ] [MAIN ] Derivative Progress: 76/1877 symbols completed. AMG took 1.1s. Estimated remaining: 11m 41s. Rate: 154.2 symbols/min [2025-10-20T22:05:09.767595] [INFO ] [MAIN ] AME: Generated 1794 derivative records from 48570 primary records. [2025-10-20T22:05:09.767773] [INFO ] [MAIN ] AMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:09.767901] [DEBUG] [LOCAL ] AME: Saving derivative data. [2025-10-20T22:05:09.797952] [DEBUG] [LOCAL ] AME: Derivative data saved (1794 records). [2025-10-20T22:05:09.798567] [INFO ] [MAIN ] AME: Successfully saved derivative data. [2025-10-20T22:05:09.800711] [INFO ] [MAIN ] Derivative Progress: 77/1877 symbols completed. AME took 1.1s. Estimated remaining: 11m 32s. Rate: 156.0 symbols/min [2025-10-20T22:05:09.800975] [INFO ] [MAIN ] AMP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:09.798806] [DEBUG] [MAIN ] AMH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:09.802102] [INFO ] [MAIN ] AMH: Generated 1794 derivative records from 51831 primary records. [2025-10-20T22:05:09.802259] [DEBUG] [LOCAL ] AMH: Saving derivative data. [2025-10-20T22:05:09.818188] [DEBUG] [LOCAL ] AMH: Derivative data saved (1794 records). [2025-10-20T22:05:09.818328] [INFO ] [MAIN ] AMH: Successfully saved derivative data. [2025-10-20T22:05:09.820701] [INFO ] [MAIN ] Derivative Progress: 78/1877 symbols completed. AMH took 1.1s. Estimated remaining: 11m 23s. Rate: 158.0 symbols/min [2025-10-20T22:05:09.820964] [INFO ] [MAIN ] AMPX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:09.860353] [DEBUG] [LOCAL ] AMN: Loaded 41731 existing records from primary storage. [2025-10-20T22:05:09.860576] [INFO ] [MAIN ] AMN: Loaded 41731 primary data records. [2025-10-20T22:05:09.981423] [DEBUG] [LOCAL ] AMP: Loaded 31952 existing records from primary storage. [2025-10-20T22:05:09.996317] [INFO ] [MAIN ] AMP: Loaded 31952 primary data records. [2025-10-20T22:05:10.184175] [DEBUG] [LOCAL ] AMPX: Loaded 52385 existing records from primary storage. [2025-10-20T22:05:10.304627] [INFO ] [MAIN ] AMPX: Loaded 52385 primary data records. [2025-10-20T22:05:10.903395] [DEBUG] [MAIN ] AMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:10.904486] [INFO ] [MAIN ] AMN: Generated 1794 derivative records from 41731 primary records. [2025-10-20T22:05:10.904736] [DEBUG] [LOCAL ] AMN: Saving derivative data. [2025-10-20T22:05:10.922813] [DEBUG] [LOCAL ] AMN: Derivative data saved (1794 records). [2025-10-20T22:05:10.923823] [INFO ] [MAIN ] AMN: Successfully saved derivative data. [2025-10-20T22:05:10.925977] [INFO ] [MAIN ] Derivative Progress: 79/1877 symbols completed. AMN took 1.2s. Estimated remaining: 11m 39s. Rate: 154.2 symbols/min [2025-10-20T22:05:10.926228] [INFO ] [MAIN ] AMPY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:10.946723] [DEBUG] [MAIN ] AMP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:10.947632] [INFO ] [MAIN ] AMP: Generated 1794 derivative records from 31952 primary records. [2025-10-20T22:05:10.947759] [DEBUG] [LOCAL ] AMP: Saving derivative data. [2025-10-20T22:05:10.968923] [DEBUG] [MAIN ] AMPX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:10.969848] [INFO ] [MAIN ] AMPX: Generated 1794 derivative records from 52385 primary records. [2025-10-20T22:05:10.969928] [DEBUG] [LOCAL ] AMPX: Saving derivative data. [2025-10-20T22:05:10.978532] [DEBUG] [LOCAL ] AMP: Derivative data saved (1794 records). [2025-10-20T22:05:10.978995] [INFO ] [MAIN ] AMP: Successfully saved derivative data. [2025-10-20T22:05:10.988144] [INFO ] [MAIN ] Derivative Progress: 80/1877 symbols completed. AMP took 1.2s. Estimated remaining: 11m 32s. Rate: 155.9 symbols/min [2025-10-20T22:05:11.000904] [INFO ] [MAIN ] AMR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:11.005511] [DEBUG] [LOCAL ] AMPX: Derivative data saved (1794 records). [2025-10-20T22:05:11.005843] [INFO ] [MAIN ] AMPX: Successfully saved derivative data. [2025-10-20T22:05:11.008071] [INFO ] [MAIN ] Derivative Progress: 81/1877 symbols completed. AMPX took 1.2s. Estimated remaining: 11m 23s. Rate: 157.7 symbols/min [2025-10-20T22:05:11.009057] [INFO ] [MAIN ] AMRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:11.024221] [DEBUG] [LOCAL ] AMPY: Loaded 39461 existing records from primary storage. [2025-10-20T22:05:11.024365] [INFO ] [MAIN ] AMPY: Loaded 39461 primary data records. [2025-10-20T22:05:11.343741] [DEBUG] [LOCAL ] AMRC: Loaded 38558 existing records from primary storage. [2025-10-20T22:05:11.356634] [INFO ] [MAIN ] AMRC: Loaded 38558 primary data records. [2025-10-20T22:05:11.357699] [DEBUG] [LOCAL ] AMR: Loaded 28978 existing records from primary storage. [2025-10-20T22:05:11.369229] [INFO ] [MAIN ] AMR: Loaded 28978 primary data records. [2025-10-20T22:05:11.983959] [DEBUG] [MAIN ] AMPY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:11.985139] [INFO ] [MAIN ] AMPY: Generated 1794 derivative records from 39461 primary records. [2025-10-20T22:05:11.985336] [DEBUG] [LOCAL ] AMPY: Saving derivative data. [2025-10-20T22:05:12.002280] [DEBUG] [LOCAL ] AMPY: Derivative data saved (1794 records). [2025-10-20T22:05:12.002683] [INFO ] [MAIN ] AMPY: Successfully saved derivative data. [2025-10-20T22:05:12.005183] [INFO ] [MAIN ] Derivative Progress: 82/1877 symbols completed. AMPY took 1.1s. Estimated remaining: 11m 36s. Rate: 154.7 symbols/min [2025-10-20T22:05:12.005467] [INFO ] [MAIN ] AMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:12.037833] [DEBUG] [MAIN ] AMRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:12.038284] [DEBUG] [MAIN ] AMR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:12.039620] [INFO ] [MAIN ] AMR: Generated 1794 derivative records from 28978 primary records. [2025-10-20T22:05:12.039067] [INFO ] [MAIN ] AMRC: Generated 1794 derivative records from 38558 primary records. [2025-10-20T22:05:12.039854] [DEBUG] [LOCAL ] AMR: Saving derivative data. [2025-10-20T22:05:12.039749] [DEBUG] [LOCAL ] AMRC: Saving derivative data. [2025-10-20T22:05:12.068521] [DEBUG] [LOCAL ] AMRC: Derivative data saved (1794 records). [2025-10-20T22:05:12.070402] [INFO ] [MAIN ] AMRC: Successfully saved derivative data. [2025-10-20T22:05:12.072442] [INFO ] [MAIN ] Derivative Progress: 83/1877 symbols completed. AMRC took 1.1s. Estimated remaining: 11m 29s. Rate: 156.2 symbols/min [2025-10-20T22:05:12.072624] [INFO ] [MAIN ] AMTB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:12.074301] [DEBUG] [LOCAL ] AMR: Derivative data saved (1794 records). [2025-10-20T22:05:12.074598] [INFO ] [MAIN ] AMR: Successfully saved derivative data. [2025-10-20T22:05:12.076556] [INFO ] [MAIN ] Derivative Progress: 84/1877 symbols completed. AMR took 1.1s. Estimated remaining: 11m 21s. Rate: 158.1 symbols/min [2025-10-20T22:05:12.077019] [INFO ] [MAIN ] AMTD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:12.096827] [DEBUG] [LOCAL ] AMTD: Loaded 3824 existing records from primary storage. [2025-10-20T22:05:12.097006] [INFO ] [MAIN ] AMTD: Loaded 3824 primary data records. [2025-10-20T22:05:12.198029] [DEBUG] [LOCAL ] AMTB: Loaded 20539 existing records from primary storage. [2025-10-20T22:05:12.222253] [INFO ] [MAIN ] AMTB: Loaded 20539 primary data records. [2025-10-20T22:05:12.226317] [DEBUG] [LOCAL ] AMT: Loaded 52454 existing records from primary storage. [2025-10-20T22:05:12.226433] [INFO ] [MAIN ] AMT: Loaded 52454 primary data records. [2025-10-20T22:05:13.062077] [DEBUG] [MAIN ] AMTD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:13.063144] [INFO ] [MAIN ] AMTD: Generated 1794 derivative records from 3824 primary records. [2025-10-20T22:05:13.063671] [DEBUG] [LOCAL ] AMTD: Saving derivative data. [2025-10-20T22:05:13.079506] [DEBUG] [LOCAL ] AMTD: Derivative data saved (1794 records). [2025-10-20T22:05:13.079738] [INFO ] [MAIN ] AMTD: Successfully saved derivative data. [2025-10-20T22:05:13.082134] [INFO ] [MAIN ] Derivative Progress: 85/1877 symbols completed. AMTD took 1.0s. Estimated remaining: 11m 33s. Rate: 155.1 symbols/min [2025-10-20T22:05:13.082755] [INFO ] [MAIN ] AMTM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:13.109773] [DEBUG] [MAIN ] AMTB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:13.110988] [INFO ] [MAIN ] AMTB: Generated 1794 derivative records from 20539 primary records. [2025-10-20T22:05:13.111162] [DEBUG] [LOCAL ] AMTB: Saving derivative data. [2025-10-20T22:05:13.110208] [DEBUG] [MAIN ] AMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:13.112355] [INFO ] [MAIN ] AMT: Generated 1794 derivative records from 52454 primary records. [2025-10-20T22:05:13.120887] [DEBUG] [LOCAL ] AMT: Saving derivative data. [2025-10-20T22:05:13.144270] [DEBUG] [LOCAL ] AMT: Derivative data saved (1794 records). [2025-10-20T22:05:13.137570] [DEBUG] [LOCAL ] AMTB: Derivative data saved (1794 records). [2025-10-20T22:05:13.144478] [INFO ] [MAIN ] AMT: Successfully saved derivative data. [2025-10-20T22:05:13.146378] [INFO ] [MAIN ] Derivative Progress: 86/1877 symbols completed. AMT took 1.1s. Estimated remaining: 11m 26s. Rate: 156.6 symbols/min [2025-10-20T22:05:13.146541] [INFO ] [MAIN ] AMWL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:13.147062] [INFO ] [MAIN ] AMTB: Successfully saved derivative data. [2025-10-20T22:05:13.149385] [INFO ] [MAIN ] Derivative Progress: 87/1877 symbols completed. AMTB took 1.1s. Estimated remaining: 11m 18s. Rate: 158.4 symbols/min [2025-10-20T22:05:13.149638] [INFO ] [MAIN ] AMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:13.169309] [DEBUG] [LOCAL ] AMWL: Loaded 9705 existing records from primary storage. [2025-10-20T22:05:13.169531] [INFO ] [MAIN ] AMWL: Loaded 9705 primary data records. [2025-10-20T22:05:13.287578] [DEBUG] [LOCAL ] AMTM: Loaded 50401 existing records from primary storage. [2025-10-20T22:05:13.287946] [INFO ] [MAIN ] AMTM: Loaded 50401 primary data records. [2025-10-20T22:05:13.685326] [DEBUG] [LOCAL ] AMX: Loaded 49207 existing records from primary storage. [2025-10-20T22:05:13.691377] [INFO ] [MAIN ] AMX: Loaded 49207 primary data records. [2025-10-20T22:05:14.134977] [DEBUG] [MAIN ] AMWL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:14.136079] [INFO ] [MAIN ] AMWL: Generated 1794 derivative records from 9705 primary records. [2025-10-20T22:05:14.136282] [DEBUG] [LOCAL ] AMWL: Saving derivative data. [2025-10-20T22:05:14.161611] [DEBUG] [MAIN ] AMTM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:14.166219] [INFO ] [MAIN ] AMTM: Generated 1794 derivative records from 50401 primary records. [2025-10-20T22:05:14.165380] [DEBUG] [LOCAL ] AMWL: Derivative data saved (1794 records). [2025-10-20T22:05:14.166330] [DEBUG] [LOCAL ] AMTM: Saving derivative data. [2025-10-20T22:05:14.166438] [INFO ] [MAIN ] AMWL: Successfully saved derivative data. [2025-10-20T22:05:14.169334] [INFO ] [MAIN ] Derivative Progress: 88/1877 symbols completed. AMWL took 1.0s. Estimated remaining: 11m 31s. Rate: 155.4 symbols/min [2025-10-20T22:05:14.185223] [DEBUG] [LOCAL ] AMTM: Derivative data saved (1794 records). [2025-10-20T22:05:14.185416] [INFO ] [MAIN ] AMTM: Successfully saved derivative data. [2025-10-20T22:05:14.187478] [INFO ] [MAIN ] Derivative Progress: 89/1877 symbols completed. AMTM took 1.1s. Estimated remaining: 11m 23s. Rate: 157.1 symbols/min [2025-10-20T22:05:14.202393] [DEBUG] [MAIN ] AMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:14.203058] [INFO ] [MAIN ] AMX: Generated 1794 derivative records from 49207 primary records. [2025-10-20T22:05:14.203118] [DEBUG] [LOCAL ] AMX: Saving derivative data. [2025-10-20T22:05:14.218594] [DEBUG] [LOCAL ] AMX: Derivative data saved (1794 records). [2025-10-20T22:05:14.218688] [INFO ] [MAIN ] AMX: Successfully saved derivative data. [2025-10-20T22:05:14.220752] [INFO ] [MAIN ] Derivative Progress: 90/1877 symbols completed. AMX took 1.1s. Estimated remaining: 11m 16s. Rate: 158.7 symbols/min [2025-10-20T22:05:14.221189] [INFO ] [MAIN ] AN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:14.221311] [INFO ] [MAIN ] ANF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:14.221921] [INFO ] [MAIN ] ANET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:14.306580] [DEBUG] [LOCAL ] AN: Loaded 31264 existing records from primary storage. [2025-10-20T22:05:14.307849] [INFO ] [MAIN ] AN: Loaded 31264 primary data records. [2025-10-20T22:05:14.307534] [DEBUG] [LOCAL ] ANET: Loaded 53766 existing records from primary storage. [2025-10-20T22:05:14.322964] [INFO ] [MAIN ] ANET: Loaded 53766 primary data records. [2025-10-20T22:05:14.379110] [DEBUG] [LOCAL ] ANF: Loaded 52054 existing records from primary storage. [2025-10-20T22:05:14.404868] [INFO ] [MAIN ] ANF: Loaded 52054 primary data records. [2025-10-20T22:05:15.284588] [DEBUG] [MAIN ] AN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:15.298131] [INFO ] [MAIN ] AN: Generated 1794 derivative records from 31264 primary records. [2025-10-20T22:05:15.299186] [DEBUG] [LOCAL ] AN: Saving derivative data. [2025-10-20T22:05:15.299975] [DEBUG] [MAIN ] ANET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:15.313407] [DEBUG] [MAIN ] ANF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:15.314251] [INFO ] [MAIN ] ANET: Generated 1794 derivative records from 53766 primary records. [2025-10-20T22:05:15.315202] [INFO ] [MAIN ] ANF: Generated 1794 derivative records from 52054 primary records. [2025-10-20T22:05:15.326082] [DEBUG] [LOCAL ] ANET: Saving derivative data. [2025-10-20T22:05:15.330212] [DEBUG] [LOCAL ] AN: Derivative data saved (1794 records). [2025-10-20T22:05:15.322381] [DEBUG] [LOCAL ] ANF: Saving derivative data. [2025-10-20T22:05:15.331074] [INFO ] [MAIN ] AN: Successfully saved derivative data. [2025-10-20T22:05:15.333127] [INFO ] [MAIN ] Derivative Progress: 91/1877 symbols completed. AN took 1.1s. Estimated remaining: 11m 30s. Rate: 155.4 symbols/min [2025-10-20T22:05:15.341299] [INFO ] [MAIN ] ANRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:15.365602] [DEBUG] [LOCAL ] ANET: Derivative data saved (1794 records). [2025-10-20T22:05:15.366333] [INFO ] [MAIN ] ANET: Successfully saved derivative data. [2025-10-20T22:05:15.371269] [DEBUG] [LOCAL ] ANF: Derivative data saved (1794 records). [2025-10-20T22:05:15.373396] [INFO ] [MAIN ] Derivative Progress: 92/1877 symbols completed. ANET took 1.2s. Estimated remaining: 11m 23s. Rate: 156.9 symbols/min [2025-10-20T22:05:15.373565] [INFO ] [MAIN ] ANF: Successfully saved derivative data. [2025-10-20T22:05:15.376674] [INFO ] [MAIN ] Derivative Progress: 93/1877 symbols completed. ANF took 1.2s. Estimated remaining: 11m 15s. Rate: 158.6 symbols/min [2025-10-20T22:05:15.373775] [INFO ] [MAIN ] ANVS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:15.377403] [INFO ] [MAIN ] AOD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:15.386170] [DEBUG] [LOCAL ] ANRO: Loaded 19540 existing records from primary storage. [2025-10-20T22:05:15.386293] [INFO ] [MAIN ] ANRO: Loaded 19540 primary data records. [2025-10-20T22:05:15.567582] [DEBUG] [LOCAL ] ANVS: Loaded 24001 existing records from primary storage. [2025-10-20T22:05:15.569982] [INFO ] [MAIN ] ANVS: Loaded 24001 primary data records. [2025-10-20T22:05:15.799308] [DEBUG] [LOCAL ] AOD: Loaded 30963 existing records from primary storage. [2025-10-20T22:05:15.826808] [INFO ] [MAIN ] AOD: Loaded 30963 primary data records. [2025-10-20T22:05:16.393245] [DEBUG] [MAIN ] ANRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:16.394851] [INFO ] [MAIN ] ANRO: Generated 1794 derivative records from 19540 primary records. [2025-10-20T22:05:16.395014] [DEBUG] [LOCAL ] ANRO: Saving derivative data. [2025-10-20T22:05:16.413966] [DEBUG] [LOCAL ] ANRO: Derivative data saved (1794 records). [2025-10-20T22:05:16.414093] [INFO ] [MAIN ] ANRO: Successfully saved derivative data. [2025-10-20T22:05:16.416020] [INFO ] [MAIN ] Derivative Progress: 94/1877 symbols completed. ANRO took 1.1s. Estimated remaining: 11m 27s. Rate: 155.7 symbols/min [2025-10-20T22:05:16.416337] [INFO ] [MAIN ] AOMR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:16.434898] [DEBUG] [MAIN ] ANVS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:16.435831] [INFO ] [MAIN ] ANVS: Generated 1794 derivative records from 24001 primary records. [2025-10-20T22:05:16.436082] [DEBUG] [LOCAL ] ANVS: Saving derivative data. [2025-10-20T22:05:16.458384] [DEBUG] [LOCAL ] ANVS: Derivative data saved (1794 records). [2025-10-20T22:05:16.464039] [INFO ] [MAIN ] ANVS: Successfully saved derivative data. [2025-10-20T22:05:16.473699] [INFO ] [MAIN ] Derivative Progress: 95/1877 symbols completed. ANVS took 1.1s. Estimated remaining: 11m 21s. Rate: 157.1 symbols/min [2025-10-20T22:05:16.479869] [DEBUG] [LOCAL ] AOMR: Loaded 16433 existing records from primary storage. [2025-10-20T22:05:16.479439] [INFO ] [MAIN ] AON: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:16.480564] [INFO ] [MAIN ] AOMR: Loaded 16433 primary data records. [2025-10-20T22:05:16.480448] [DEBUG] [MAIN ] AOD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:16.481802] [INFO ] [MAIN ] AOD: Generated 1794 derivative records from 30963 primary records. [2025-10-20T22:05:16.502586] [DEBUG] [LOCAL ] AOD: Saving derivative data. [2025-10-20T22:05:16.599312] [DEBUG] [LOCAL ] AOD: Derivative data saved (1794 records). [2025-10-20T22:05:16.602985] [INFO ] [MAIN ] AOD: Successfully saved derivative data. [2025-10-20T22:05:16.605816] [INFO ] [MAIN ] Derivative Progress: 96/1877 symbols completed. AOD took 1.2s. Estimated remaining: 11m 16s. Rate: 158.2 symbols/min [2025-10-20T22:05:16.622141] [INFO ] [MAIN ] AORT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:16.828349] [DEBUG] [LOCAL ] AON: Loaded 45175 existing records from primary storage. [2025-10-20T22:05:16.829388] [INFO ] [MAIN ] AON: Loaded 45175 primary data records. [2025-10-20T22:05:16.971017] [DEBUG] [LOCAL ] AORT: Loaded 28981 existing records from primary storage. [2025-10-20T22:05:16.981901] [INFO ] [MAIN ] AORT: Loaded 28981 primary data records. [2025-10-20T22:05:17.512625] [DEBUG] [MAIN ] AOMR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:17.514363] [INFO ] [MAIN ] AOMR: Generated 1794 derivative records from 16433 primary records. [2025-10-20T22:05:17.514821] [DEBUG] [LOCAL ] AOMR: Saving derivative data. [2025-10-20T22:05:17.537975] [DEBUG] [LOCAL ] AOMR: Derivative data saved (1794 records). [2025-10-20T22:05:17.538322] [INFO ] [MAIN ] AOMR: Successfully saved derivative data. [2025-10-20T22:05:17.540679] [INFO ] [MAIN ] Derivative Progress: 97/1877 symbols completed. AOMR took 1.1s. Estimated remaining: 11m 25s. Rate: 155.8 symbols/min [2025-10-20T22:05:17.541013] [INFO ] [MAIN ] AOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:17.559776] [DEBUG] [MAIN ] AON: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:17.560943] [INFO ] [MAIN ] AON: Generated 1794 derivative records from 45175 primary records. [2025-10-20T22:05:17.561110] [DEBUG] [LOCAL ] AON: Saving derivative data. [2025-10-20T22:05:17.578861] [DEBUG] [LOCAL ] AON: Derivative data saved (1794 records). [2025-10-20T22:05:17.578930] [INFO ] [MAIN ] AON: Successfully saved derivative data. [2025-10-20T22:05:17.580941] [INFO ] [MAIN ] Derivative Progress: 98/1877 symbols completed. AON took 1.1s. Estimated remaining: 11m 19s. Rate: 157.3 symbols/min [2025-10-20T22:05:17.581149] [INFO ] [MAIN ] AP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:17.608017] [DEBUG] [MAIN ] AORT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:17.610482] [INFO ] [MAIN ] AORT: Generated 1794 derivative records from 28981 primary records. [2025-10-20T22:05:17.610607] [DEBUG] [LOCAL ] AORT: Saving derivative data. [2025-10-20T22:05:17.626833] [DEBUG] [LOCAL ] AP: Loaded 7408 existing records from primary storage. [2025-10-20T22:05:17.630077] [INFO ] [MAIN ] AP: Loaded 7408 primary data records. [2025-10-20T22:05:17.711576] [DEBUG] [LOCAL ] AORT: Derivative data saved (1794 records). [2025-10-20T22:05:17.730410] [DEBUG] [LOCAL ] AOS: Loaded 48689 existing records from primary storage. [2025-10-20T22:05:17.756225] [INFO ] [MAIN ] AORT: Successfully saved derivative data. [2025-10-20T22:05:17.756448] [INFO ] [MAIN ] AOS: Loaded 48689 primary data records. [2025-10-20T22:05:17.758608] [INFO ] [MAIN ] Derivative Progress: 99/1877 symbols completed. AORT took 1.1s. Estimated remaining: 11m 15s. Rate: 158.1 symbols/min [2025-10-20T22:05:17.786832] [INFO ] [MAIN ] APAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:18.324082] [DEBUG] [LOCAL ] APAM: Loaded 36007 existing records from primary storage. [2025-10-20T22:05:18.325219] [INFO ] [MAIN ] APAM: Loaded 36007 primary data records. [2025-10-20T22:05:18.639127] [DEBUG] [MAIN ] AP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:18.640177] [INFO ] [MAIN ] AP: Generated 1794 derivative records from 7408 primary records. [2025-10-20T22:05:18.640337] [DEBUG] [LOCAL ] AP: Saving derivative data. [2025-10-20T22:05:18.657687] [DEBUG] [LOCAL ] AP: Derivative data saved (1794 records). [2025-10-20T22:05:18.659139] [INFO ] [MAIN ] AP: Successfully saved derivative data. [2025-10-20T22:05:18.661577] [INFO ] [MAIN ] Derivative Progress: 100/1877 symbols completed. AP took 1.1s. Estimated remaining: 11m 24s. Rate: 156.0 symbols/min [2025-10-20T22:05:18.662636] [INFO ] [MAIN ] APD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:18.680295] [DEBUG] [MAIN ] AOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:18.681112] [INFO ] [MAIN ] AOS: Generated 1794 derivative records from 48689 primary records. [2025-10-20T22:05:18.682392] [DEBUG] [LOCAL ] AOS: Saving derivative data. [2025-10-20T22:05:18.698513] [DEBUG] [LOCAL ] AOS: Derivative data saved (1794 records). [2025-10-20T22:05:18.698692] [INFO ] [MAIN ] AOS: Successfully saved derivative data. [2025-10-20T22:05:18.701024] [INFO ] [MAIN ] Derivative Progress: 101/1877 symbols completed. AOS took 1.2s. Estimated remaining: 11m 17s. Rate: 157.4 symbols/min [2025-10-20T22:05:18.701341] [INFO ] [MAIN ] APG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:18.734007] [DEBUG] [MAIN ] APAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:18.734922] [INFO ] [MAIN ] APAM: Generated 1794 derivative records from 36007 primary records. [2025-10-20T22:05:18.735180] [DEBUG] [LOCAL ] APAM: Saving derivative data. [2025-10-20T22:05:18.752801] [DEBUG] [LOCAL ] APAM: Derivative data saved (1794 records). [2025-10-20T22:05:18.753142] [INFO ] [MAIN ] APAM: Successfully saved derivative data. [2025-10-20T22:05:18.767911] [INFO ] [MAIN ] Derivative Progress: 102/1877 symbols completed. APAM took 1.0s. Estimated remaining: 11m 11s. Rate: 158.7 symbols/min [2025-10-20T22:05:18.769796] [INFO ] [MAIN ] APH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:18.782459] [DEBUG] [LOCAL ] APD: Loaded 45522 existing records from primary storage. [2025-10-20T22:05:18.782815] [INFO ] [MAIN ] APD: Loaded 45522 primary data records. [2025-10-20T22:05:18.913780] [DEBUG] [LOCAL ] APG: Loaded 50628 existing records from primary storage. [2025-10-20T22:05:18.919856] [INFO ] [MAIN ] APG: Loaded 50628 primary data records. [2025-10-20T22:05:19.450493] [DEBUG] [LOCAL ] APH: Loaded 53691 existing records from primary storage. [2025-10-20T22:05:19.450915] [INFO ] [MAIN ] APH: Loaded 53691 primary data records. [2025-10-20T22:05:19.773943] [DEBUG] [MAIN ] APD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:19.775382] [INFO ] [MAIN ] APD: Generated 1794 derivative records from 45522 primary records. [2025-10-20T22:05:19.775700] [DEBUG] [LOCAL ] APD: Saving derivative data. [2025-10-20T22:05:19.793392] [DEBUG] [LOCAL ] APD: Derivative data saved (1794 records). [2025-10-20T22:05:19.795167] [INFO ] [MAIN ] APD: Successfully saved derivative data. [2025-10-20T22:05:19.797766] [INFO ] [MAIN ] Derivative Progress: 103/1877 symbols completed. APD took 1.1s. Estimated remaining: 11m 22s. Rate: 156.0 symbols/min [2025-10-20T22:05:19.813190] [DEBUG] [MAIN ] APG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:19.814616] [INFO ] [MAIN ] APG: Generated 1794 derivative records from 50628 primary records. [2025-10-20T22:05:19.814743] [DEBUG] [LOCAL ] APG: Saving derivative data. [2025-10-20T22:05:19.836304] [DEBUG] [LOCAL ] APG: Derivative data saved (1794 records). [2025-10-20T22:05:19.836598] [INFO ] [MAIN ] APG: Successfully saved derivative data. [2025-10-20T22:05:19.851243] [INFO ] [MAIN ] Derivative Progress: 104/1877 symbols completed. APG took 1.1s. Estimated remaining: 11m 16s. Rate: 157.4 symbols/min [2025-10-20T22:05:19.851736] [DEBUG] [MAIN ] APH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:19.852551] [INFO ] [MAIN ] APH: Generated 1794 derivative records from 53691 primary records. [2025-10-20T22:05:19.852617] [DEBUG] [LOCAL ] APH: Saving derivative data. [2025-10-20T22:05:19.867950] [DEBUG] [LOCAL ] APH: Derivative data saved (1794 records). [2025-10-20T22:05:19.868013] [INFO ] [MAIN ] APH: Successfully saved derivative data. [2025-10-20T22:05:19.870143] [INFO ] [MAIN ] Derivative Progress: 105/1877 symbols completed. APH took 1.1s. Estimated remaining: 11m 10s. Rate: 158.8 symbols/min [2025-10-20T22:05:19.870631] [INFO ] [MAIN ] APLE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:19.871227] [INFO ] [MAIN ] AQN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:19.870820] [INFO ] [MAIN ] APOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:19.884070] [DEBUG] [LOCAL ] APOS: Loaded 7187 existing records from primary storage. [2025-10-20T22:05:19.884289] [INFO ] [MAIN ] APOS: Loaded 7187 primary data records. [2025-10-20T22:05:20.271493] [DEBUG] [LOCAL ] AQN: Loaded 51714 existing records from primary storage. [2025-10-20T22:05:20.281972] [INFO ] [MAIN ] AQN: Loaded 51714 primary data records. [2025-10-20T22:05:20.294608] [DEBUG] [LOCAL ] APLE: Loaded 51975 existing records from primary storage. [2025-10-20T22:05:20.322401] [INFO ] [MAIN ] APLE: Loaded 51975 primary data records. [2025-10-20T22:05:20.992391] [DEBUG] [MAIN ] APOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:20.994063] [INFO ] [MAIN ] APOS: Generated 1794 derivative records from 7187 primary records. [2025-10-20T22:05:20.994169] [DEBUG] [LOCAL ] APOS: Saving derivative data. [2025-10-20T22:05:21.012171] [DEBUG] [LOCAL ] APOS: Derivative data saved (1794 records). [2025-10-20T22:05:21.013321] [INFO ] [MAIN ] APOS: Successfully saved derivative data. [2025-10-20T22:05:21.015336] [INFO ] [MAIN ] Derivative Progress: 106/1877 symbols completed. APOS took 1.1s. Estimated remaining: 11m 22s. Rate: 155.8 symbols/min [2025-10-20T22:05:21.015630] [INFO ] [MAIN ] AQNB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:21.022886] [DEBUG] [LOCAL ] AQNB: Loaded 3972 existing records from primary storage. [2025-10-20T22:05:21.023296] [INFO ] [MAIN ] AQNB: Loaded 3972 primary data records. [2025-10-20T22:05:21.189612] [DEBUG] [MAIN ] AQN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:21.190105] [DEBUG] [MAIN ] APLE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:21.357740] [INFO ] [MAIN ] AQN: Generated 1794 derivative records from 51714 primary records. [2025-10-20T22:05:21.360078] [INFO ] [MAIN ] APLE: Generated 1794 derivative records from 51975 primary records. [2025-10-20T22:05:21.358682] [DEBUG] [LOCAL ] AQN: Saving derivative data. [2025-10-20T22:05:21.360444] [DEBUG] [LOCAL ] APLE: Saving derivative data. [2025-10-20T22:05:21.397400] [DEBUG] [LOCAL ] AQN: Derivative data saved (1794 records). [2025-10-20T22:05:21.392915] [DEBUG] [LOCAL ] APLE: Derivative data saved (1794 records). [2025-10-20T22:05:21.398533] [INFO ] [MAIN ] AQN: Successfully saved derivative data. [2025-10-20T22:05:21.400588] [INFO ] [MAIN ] Derivative Progress: 107/1877 symbols completed. AQN took 1.5s. Estimated remaining: 11m 22s. Rate: 155.8 symbols/min [2025-10-20T22:05:21.398028] [INFO ] [MAIN ] APLE: Successfully saved derivative data. [2025-10-20T22:05:21.400915] [INFO ] [MAIN ] AR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:21.404120] [INFO ] [MAIN ] Derivative Progress: 108/1877 symbols completed. APLE took 1.5s. Estimated remaining: 11m 15s. Rate: 157.2 symbols/min [2025-10-20T22:05:21.404578] [INFO ] [MAIN ] ARCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:21.428001] [DEBUG] [MAIN ] AQNB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:21.428800] [INFO ] [MAIN ] AQNB: Generated 1794 derivative records from 3972 primary records. [2025-10-20T22:05:21.428946] [DEBUG] [LOCAL ] AQNB: Saving derivative data. [2025-10-20T22:05:21.459136] [DEBUG] [LOCAL ] AQNB: Derivative data saved (1794 records). [2025-10-20T22:05:21.459619] [INFO ] [MAIN ] AQNB: Successfully saved derivative data. [2025-10-20T22:05:21.471163] [INFO ] [MAIN ] Derivative Progress: 109/1877 symbols completed. AQNB took 0.5s. Estimated remaining: 11m 10s. Rate: 158.4 symbols/min [2025-10-20T22:05:21.471399] [INFO ] [MAIN ] ARDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:21.503954] [DEBUG] [LOCAL ] AR: Loaded 53590 existing records from primary storage. [2025-10-20T22:05:21.508904] [INFO ] [MAIN ] AR: Loaded 53590 primary data records. [2025-10-20T22:05:21.521589] [DEBUG] [LOCAL ] ARCO: Loaded 44434 existing records from primary storage. [2025-10-20T22:05:21.546621] [INFO ] [MAIN ] ARCO: Loaded 44434 primary data records. [2025-10-20T22:05:21.617924] [DEBUG] [LOCAL ] ARDC: Loaded 15280 existing records from primary storage. [2025-10-20T22:05:21.703058] [INFO ] [MAIN ] ARDC: Loaded 15280 primary data records. [2025-10-20T22:05:22.525060] [DEBUG] [MAIN ] AR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:22.528582] [INFO ] [MAIN ] AR: Generated 1794 derivative records from 53590 primary records. [2025-10-20T22:05:22.528710] [DEBUG] [LOCAL ] AR: Saving derivative data. [2025-10-20T22:05:22.544187] [DEBUG] [MAIN ] ARCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:22.557355] [INFO ] [MAIN ] ARCO: Generated 1794 derivative records from 44434 primary records. [2025-10-20T22:05:22.558915] [DEBUG] [LOCAL ] ARCO: Saving derivative data. [2025-10-20T22:05:22.564526] [DEBUG] [LOCAL ] AR: Derivative data saved (1794 records). [2025-10-20T22:05:22.565172] [INFO ] [MAIN ] AR: Successfully saved derivative data. [2025-10-20T22:05:22.574212] [INFO ] [MAIN ] Derivative Progress: 110/1877 symbols completed. AR took 1.2s. Estimated remaining: 11m 21s. Rate: 155.7 symbols/min [2025-10-20T22:05:22.574529] [INFO ] [MAIN ] ARDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:22.596959] [DEBUG] [LOCAL ] ARCO: Derivative data saved (1794 records). [2025-10-20T22:05:22.597947] [INFO ] [MAIN ] ARCO: Successfully saved derivative data. [2025-10-20T22:05:22.598171] [DEBUG] [MAIN ] ARDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:22.602017] [INFO ] [MAIN ] ARDC: Generated 1794 derivative records from 15280 primary records. [2025-10-20T22:05:22.600983] [INFO ] [MAIN ] Derivative Progress: 111/1877 symbols completed. ARCO took 1.2s. Estimated remaining: 11m 15s. Rate: 157.1 symbols/min [2025-10-20T22:05:22.602371] [DEBUG] [LOCAL ] ARDC: Saving derivative data. [2025-10-20T22:05:22.602975] [INFO ] [MAIN ] ARE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:22.619938] [DEBUG] [LOCAL ] ARDC: Derivative data saved (1794 records). [2025-10-20T22:05:22.620162] [INFO ] [MAIN ] ARDC: Successfully saved derivative data. [2025-10-20T22:05:22.622451] [INFO ] [MAIN ] Derivative Progress: 112/1877 symbols completed. ARDC took 1.2s. Estimated remaining: 11m 9s. Rate: 158.4 symbols/min [2025-10-20T22:05:22.622608] [INFO ] [MAIN ] ARES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:22.645411] [DEBUG] [LOCAL ] ARDT: Loaded 27434 existing records from primary storage. [2025-10-20T22:05:22.645565] [INFO ] [MAIN ] ARDT: Loaded 27434 primary data records. [2025-10-20T22:05:22.981623] [DEBUG] [LOCAL ] ARE: Loaded 50965 existing records from primary storage. [2025-10-20T22:05:22.982390] [INFO ] [MAIN ] ARE: Loaded 50965 primary data records. [2025-10-20T22:05:23.054724] [DEBUG] [LOCAL ] ARES: Loaded 49703 existing records from primary storage. [2025-10-20T22:05:23.091767] [INFO ] [MAIN ] ARES: Loaded 49703 primary data records. [2025-10-20T22:05:23.619850] [DEBUG] [MAIN ] ARDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:23.620849] [INFO ] [MAIN ] ARDT: Generated 1794 derivative records from 27434 primary records. [2025-10-20T22:05:23.621026] [DEBUG] [LOCAL ] ARDT: Saving derivative data. [2025-10-20T22:05:23.638559] [DEBUG] [LOCAL ] ARDT: Derivative data saved (1794 records). [2025-10-20T22:05:23.639382] [INFO ] [MAIN ] ARDT: Successfully saved derivative data. [2025-10-20T22:05:23.642582] [INFO ] [MAIN ] Derivative Progress: 113/1877 symbols completed. ARDT took 1.1s. Estimated remaining: 11m 18s. Rate: 156.0 symbols/min [2025-10-20T22:05:23.643731] [INFO ] [MAIN ] ARMK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:23.659384] [DEBUG] [MAIN ] ARE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:23.660257] [INFO ] [MAIN ] ARE: Generated 1794 derivative records from 50965 primary records. [2025-10-20T22:05:23.660384] [DEBUG] [LOCAL ] ARE: Saving derivative data. [2025-10-20T22:05:23.678243] [DEBUG] [LOCAL ] ARE: Derivative data saved (1794 records). [2025-10-20T22:05:23.678591] [INFO ] [MAIN ] ARE: Successfully saved derivative data. [2025-10-20T22:05:23.680482] [INFO ] [MAIN ] Derivative Progress: 114/1877 symbols completed. ARE took 1.1s. Estimated remaining: 11m 13s. Rate: 157.3 symbols/min [2025-10-20T22:05:23.680828] [INFO ] [MAIN ] AROC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:23.699832] [DEBUG] [MAIN ] ARES: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:23.700518] [INFO ] [MAIN ] ARES: Generated 1793 derivative records from 49703 primary records. [2025-10-20T22:05:23.700682] [DEBUG] [LOCAL ] ARES: Saving derivative data. [2025-10-20T22:05:23.729821] [DEBUG] [LOCAL ] ARES: Derivative data saved (1793 records). [2025-10-20T22:05:23.730609] [INFO ] [MAIN ] ARES: Successfully saved derivative data. [2025-10-20T22:05:23.733116] [INFO ] [MAIN ] Derivative Progress: 115/1877 symbols completed. ARES took 1.1s. Estimated remaining: 11m 7s. Rate: 158.5 symbols/min [2025-10-20T22:05:23.748863] [INFO ] [MAIN ] ARW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:23.750471] [DEBUG] [LOCAL ] ARMK: Loaded 50421 existing records from primary storage. [2025-10-20T22:05:23.750630] [INFO ] [MAIN ] ARMK: Loaded 50421 primary data records. [2025-10-20T22:05:23.988258] [DEBUG] [LOCAL ] ARW: Loaded 33236 existing records from primary storage. [2025-10-20T22:05:23.996034] [DEBUG] [LOCAL ] AROC: Loaded 49092 existing records from primary storage. [2025-10-20T22:05:24.101713] [INFO ] [MAIN ] AROC: Loaded 49092 primary data records. [2025-10-20T22:05:24.101477] [INFO ] [MAIN ] ARW: Loaded 33236 primary data records. [2025-10-20T22:05:24.796502] [DEBUG] [MAIN ] ARMK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:24.797683] [INFO ] [MAIN ] ARMK: Generated 1794 derivative records from 50421 primary records. [2025-10-20T22:05:24.798064] [DEBUG] [LOCAL ] ARMK: Saving derivative data. [2025-10-20T22:05:24.821397] [DEBUG] [LOCAL ] ARMK: Derivative data saved (1794 records). [2025-10-20T22:05:24.821833] [INFO ] [MAIN ] ARMK: Successfully saved derivative data. [2025-10-20T22:05:24.824086] [INFO ] [MAIN ] Derivative Progress: 116/1877 symbols completed. ARMK took 1.2s. Estimated remaining: 11m 18s. Rate: 156.0 symbols/min [2025-10-20T22:05:24.824548] [INFO ] [MAIN ] ASAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:24.839359] [DEBUG] [MAIN ] AROC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:24.840382] [INFO ] [MAIN ] AROC: Generated 1794 derivative records from 49092 primary records. [2025-10-20T22:05:24.840567] [DEBUG] [LOCAL ] AROC: Saving derivative data. [2025-10-20T22:05:24.869781] [DEBUG] [LOCAL ] AROC: Derivative data saved (1794 records). [2025-10-20T22:05:24.874207] [INFO ] [MAIN ] AROC: Successfully saved derivative data. [2025-10-20T22:05:24.877329] [DEBUG] [MAIN ] ARW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:24.876997] [INFO ] [MAIN ] Derivative Progress: 117/1877 symbols completed. AROC took 1.2s. Estimated remaining: 11m 12s. Rate: 157.1 symbols/min [2025-10-20T22:05:24.877680] [INFO ] [MAIN ] ASBA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:24.878544] [INFO ] [MAIN ] ARW: Generated 1794 derivative records from 33236 primary records. [2025-10-20T22:05:24.878988] [DEBUG] [LOCAL ] ARW: Saving derivative data. [2025-10-20T22:05:24.896556] [DEBUG] [LOCAL ] ARW: Derivative data saved (1794 records). [2025-10-20T22:05:24.896831] [INFO ] [MAIN ] ARW: Successfully saved derivative data. [2025-10-20T22:05:24.899495] [INFO ] [MAIN ] Derivative Progress: 118/1877 symbols completed. ARW took 1.2s. Estimated remaining: 11m 6s. Rate: 158.4 symbols/min [2025-10-20T22:05:24.902297] [DEBUG] [LOCAL ] ASBA: Loaded 3520 existing records from primary storage. [2025-10-20T22:05:24.902746] [INFO ] [MAIN ] ASBA: Loaded 3520 primary data records. [2025-10-20T22:05:25.068584] [DEBUG] [LOCAL ] ASAN: Loaded 52732 existing records from primary storage. [2025-10-20T22:05:25.091384] [INFO ] [MAIN ] ASAN: Loaded 52732 primary data records. [2025-10-20T22:05:25.560836] [DEBUG] [MAIN ] ASBA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:25.561582] [INFO ] [MAIN ] ASBA: Generated 1794 derivative records from 3520 primary records. [2025-10-20T22:05:25.561888] [DEBUG] [LOCAL ] ASBA: Saving derivative data. [2025-10-20T22:05:25.576536] [DEBUG] [LOCAL ] ASBA: Derivative data saved (1794 records). [2025-10-20T22:05:25.576794] [INFO ] [MAIN ] ASBA: Successfully saved derivative data. [2025-10-20T22:05:25.578710] [INFO ] [MAIN ] Derivative Progress: 119/1877 symbols completed. ASBA took 0.7s. Estimated remaining: 11m 10s. Rate: 157.3 symbols/min [2025-10-20T22:05:25.593635] [DEBUG] [MAIN ] ASAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:25.594197] [INFO ] [MAIN ] ASAN: Generated 1794 derivative records from 52732 primary records. [2025-10-20T22:05:25.594253] [DEBUG] [LOCAL ] ASAN: Saving derivative data. [2025-10-20T22:05:25.609376] [DEBUG] [LOCAL ] ASAN: Derivative data saved (1794 records). [2025-10-20T22:05:25.609437] [INFO ] [MAIN ] ASAN: Successfully saved derivative data. [2025-10-20T22:05:25.611364] [INFO ] [MAIN ] Derivative Progress: 120/1877 symbols completed. ASAN took 0.8s. Estimated remaining: 11m 5s. Rate: 158.5 symbols/min [2025-10-20T22:05:25.611854] [INFO ] [MAIN ] ASGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:25.611917] [INFO ] [MAIN ] ASGN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:25.611795] [INFO ] [MAIN ] ASG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:25.652983] [DEBUG] [LOCAL ] ASGI: Loaded 23006 existing records from primary storage. [2025-10-20T22:05:25.652541] [DEBUG] [LOCAL ] ASG: Loaded 19504 existing records from primary storage. [2025-10-20T22:05:25.653229] [INFO ] [MAIN ] ASGI: Loaded 23006 primary data records. [2025-10-20T22:05:25.653183] [INFO ] [MAIN ] ASG: Loaded 19504 primary data records. [2025-10-20T22:05:25.859570] [DEBUG] [LOCAL ] ASGN: Loaded 36724 existing records from primary storage. [2025-10-20T22:05:25.880261] [INFO ] [MAIN ] ASGN: Loaded 36724 primary data records. [2025-10-20T22:05:26.644876] [DEBUG] [MAIN ] ASG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:26.652703] [INFO ] [MAIN ] ASG: Generated 1794 derivative records from 19504 primary records. [2025-10-20T22:05:26.652800] [DEBUG] [LOCAL ] ASG: Saving derivative data. [2025-10-20T22:05:26.651348] [DEBUG] [MAIN ] ASGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:26.654021] [INFO ] [MAIN ] ASGI: Generated 1794 derivative records from 23006 primary records. [2025-10-20T22:05:26.655875] [DEBUG] [LOCAL ] ASGI: Saving derivative data. [2025-10-20T22:05:26.692528] [DEBUG] [LOCAL ] ASG: Derivative data saved (1794 records). [2025-10-20T22:05:26.700936] [DEBUG] [LOCAL ] ASGI: Derivative data saved (1794 records). [2025-10-20T22:05:26.701521] [INFO ] [MAIN ] ASG: Successfully saved derivative data. [2025-10-20T22:05:26.701748] [INFO ] [MAIN ] ASGI: Successfully saved derivative data. [2025-10-20T22:05:26.702033] [DEBUG] [MAIN ] ASGN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:26.703973] [INFO ] [MAIN ] Derivative Progress: 121/1877 symbols completed. ASGI took 1.1s. Estimated remaining: 11m 15s. Rate: 156.1 symbols/min [2025-10-20T22:05:26.706816] [INFO ] [MAIN ] ASGN: Generated 1794 derivative records from 36724 primary records. [2025-10-20T22:05:26.706111] [INFO ] [MAIN ] ASH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:26.707034] [INFO ] [MAIN ] Derivative Progress: 122/1877 symbols completed. ASG took 1.1s. Estimated remaining: 11m 9s. Rate: 157.4 symbols/min [2025-10-20T22:05:26.707217] [INFO ] [MAIN ] ASIX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:26.707724] [DEBUG] [LOCAL ] ASGN: Saving derivative data. [2025-10-20T22:05:26.723845] [DEBUG] [LOCAL ] ASGN: Derivative data saved (1794 records). [2025-10-20T22:05:26.724011] [INFO ] [MAIN ] ASGN: Successfully saved derivative data. [2025-10-20T22:05:26.726269] [INFO ] [MAIN ] Derivative Progress: 123/1877 symbols completed. ASGN took 1.1s. Estimated remaining: 11m 4s. Rate: 158.6 symbols/min [2025-10-20T22:05:26.726582] [INFO ] [MAIN ] ASPN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:26.753713] [DEBUG] [LOCAL ] ASIX: Loaded 23205 existing records from primary storage. [2025-10-20T22:05:26.753851] [INFO ] [MAIN ] ASIX: Loaded 23205 primary data records. [2025-10-20T22:05:26.872397] [DEBUG] [LOCAL ] ASH: Loaded 37563 existing records from primary storage. [2025-10-20T22:05:26.894420] [INFO ] [MAIN ] ASH: Loaded 37563 primary data records. [2025-10-20T22:05:27.196712] [DEBUG] [LOCAL ] ASPN: Loaded 49915 existing records from primary storage. [2025-10-20T22:05:27.201983] [INFO ] [MAIN ] ASPN: Loaded 49915 primary data records. [2025-10-20T22:05:27.745315] [DEBUG] [MAIN ] ASIX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:27.746981] [INFO ] [MAIN ] ASIX: Generated 1794 derivative records from 23205 primary records. [2025-10-20T22:05:27.747246] [DEBUG] [LOCAL ] ASIX: Saving derivative data. [2025-10-20T22:05:27.769308] [DEBUG] [LOCAL ] ASIX: Derivative data saved (1794 records). [2025-10-20T22:05:27.769558] [INFO ] [MAIN ] ASIX: Successfully saved derivative data. [2025-10-20T22:05:27.771718] [INFO ] [MAIN ] Derivative Progress: 124/1877 symbols completed. ASIX took 1.1s. Estimated remaining: 11m 13s. Rate: 156.4 symbols/min [2025-10-20T22:05:27.772097] [INFO ] [MAIN ] ASX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:27.785738] [DEBUG] [MAIN ] ASH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:27.786755] [INFO ] [MAIN ] ASH: Generated 1794 derivative records from 37563 primary records. [2025-10-20T22:05:27.787039] [DEBUG] [LOCAL ] ASH: Saving derivative data. [2025-10-20T22:05:27.805160] [DEBUG] [LOCAL ] ASH: Derivative data saved (1794 records). [2025-10-20T22:05:27.805269] [INFO ] [MAIN ] ASH: Successfully saved derivative data. [2025-10-20T22:05:27.807741] [INFO ] [MAIN ] Derivative Progress: 125/1877 symbols completed. ASH took 1.1s. Estimated remaining: 11m 7s. Rate: 157.5 symbols/min [2025-10-20T22:05:27.815828] [INFO ] [MAIN ] ATEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:27.821352] [DEBUG] [MAIN ] ASPN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:27.822087] [INFO ] [MAIN ] ASPN: Generated 1794 derivative records from 49915 primary records. [2025-10-20T22:05:27.822161] [DEBUG] [LOCAL ] ASPN: Saving derivative data. [2025-10-20T22:05:27.839111] [DEBUG] [LOCAL ] ASPN: Derivative data saved (1794 records). [2025-10-20T22:05:27.839181] [INFO ] [MAIN ] ASPN: Successfully saved derivative data. [2025-10-20T22:05:27.841066] [INFO ] [MAIN ] Derivative Progress: 126/1877 symbols completed. ASPN took 1.1s. Estimated remaining: 11m 2s. Rate: 158.7 symbols/min [2025-10-20T22:05:27.841347] [INFO ] [MAIN ] ATHM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:27.907902] [DEBUG] [LOCAL ] ATEN: Loaded 38022 existing records from primary storage. [2025-10-20T22:05:27.908084] [INFO ] [MAIN ] ATEN: Loaded 38022 primary data records. [2025-10-20T22:05:27.914689] [DEBUG] [LOCAL ] ASX: Loaded 52882 existing records from primary storage. [2025-10-20T22:05:27.940970] [INFO ] [MAIN ] ASX: Loaded 52882 primary data records. [2025-10-20T22:05:28.041821] [DEBUG] [LOCAL ] ATHM: Loaded 31516 existing records from primary storage. [2025-10-20T22:05:28.093606] [INFO ] [MAIN ] ATHM: Loaded 31516 primary data records. [2025-10-20T22:05:28.896338] [DEBUG] [MAIN ] ATEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:28.897409] [INFO ] [MAIN ] ATEN: Generated 1794 derivative records from 38022 primary records. [2025-10-20T22:05:28.897828] [DEBUG] [LOCAL ] ATEN: Saving derivative data. [2025-10-20T22:05:28.920296] [DEBUG] [LOCAL ] ATEN: Derivative data saved (1794 records). [2025-10-20T22:05:28.933180] [INFO ] [MAIN ] ATEN: Successfully saved derivative data. [2025-10-20T22:05:28.935229] [INFO ] [MAIN ] Derivative Progress: 127/1877 symbols completed. ATEN took 1.1s. Estimated remaining: 11m 12s. Rate: 156.3 symbols/min [2025-10-20T22:05:28.943322] [DEBUG] [MAIN ] ASX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:28.948815] [INFO ] [MAIN ] ASX: Generated 1794 derivative records from 52882 primary records. [2025-10-20T22:05:28.949709] [DEBUG] [LOCAL ] ASX: Saving derivative data. [2025-10-20T22:05:28.943484] [INFO ] [MAIN ] ATHS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:28.949532] [DEBUG] [MAIN ] ATHM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:28.960086] [INFO ] [MAIN ] ATHM: Generated 1794 derivative records from 31516 primary records. [2025-10-20T22:05:28.961695] [DEBUG] [LOCAL ] ATHM: Saving derivative data. [2025-10-20T22:05:28.969210] [DEBUG] [LOCAL ] ASX: Derivative data saved (1794 records). [2025-10-20T22:05:28.976594] [INFO ] [MAIN ] ASX: Successfully saved derivative data. [2025-10-20T22:05:28.988055] [DEBUG] [LOCAL ] ATHM: Derivative data saved (1794 records). [2025-10-20T22:05:28.983958] [INFO ] [MAIN ] Derivative Progress: 128/1877 symbols completed. ASX took 1.2s. Estimated remaining: 11m 7s. Rate: 157.4 symbols/min [2025-10-20T22:05:28.988408] [INFO ] [MAIN ] ATI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:28.988298] [INFO ] [MAIN ] ATHM: Successfully saved derivative data. [2025-10-20T22:05:28.990603] [INFO ] [MAIN ] Derivative Progress: 129/1877 symbols completed. ATHM took 1.1s. Estimated remaining: 11m 1s. Rate: 158.6 symbols/min [2025-10-20T22:05:28.991268] [INFO ] [MAIN ] ATKR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:28.996821] [DEBUG] [LOCAL ] ATHS: Loaded 6572 existing records from primary storage. [2025-10-20T22:05:28.997078] [INFO ] [MAIN ] ATHS: Loaded 6572 primary data records. [2025-10-20T22:05:29.336997] [DEBUG] [LOCAL ] ATKR: Loaded 35028 existing records from primary storage. [2025-10-20T22:05:29.337672] [INFO ] [MAIN ] ATKR: Loaded 35028 primary data records. [2025-10-20T22:05:29.460499] [DEBUG] [LOCAL ] ATI: Loaded 51311 existing records from primary storage. [2025-10-20T22:05:29.462543] [INFO ] [MAIN ] ATI: Loaded 51311 primary data records. [2025-10-20T22:05:29.984881] [DEBUG] [MAIN ] ATHS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:30.000804] [INFO ] [MAIN ] ATHS: Generated 1794 derivative records from 6572 primary records. [2025-10-20T22:05:30.001130] [DEBUG] [LOCAL ] ATHS: Saving derivative data. [2025-10-20T22:05:30.019536] [DEBUG] [LOCAL ] ATHS: Derivative data saved (1794 records). [2025-10-20T22:05:30.020450] [INFO ] [MAIN ] ATHS: Successfully saved derivative data. [2025-10-20T22:05:30.022572] [INFO ] [MAIN ] Derivative Progress: 130/1877 symbols completed. ATHS took 1.1s. Estimated remaining: 11m 10s. Rate: 156.5 symbols/min [2025-10-20T22:05:30.023057] [INFO ] [MAIN ] ATMU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:30.049985] [DEBUG] [MAIN ] ATKR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:30.051105] [INFO ] [MAIN ] ATKR: Generated 1793 derivative records from 35028 primary records. [2025-10-20T22:05:30.051457] [DEBUG] [LOCAL ] ATKR: Saving derivative data. [2025-10-20T22:05:30.075143] [DEBUG] [LOCAL ] ATKR: Derivative data saved (1793 records). [2025-10-20T22:05:30.075361] [INFO ] [MAIN ] ATKR: Successfully saved derivative data. [2025-10-20T22:05:30.088580] [INFO ] [MAIN ] Derivative Progress: 131/1877 symbols completed. ATKR took 1.1s. Estimated remaining: 11m 5s. Rate: 157.5 symbols/min [2025-10-20T22:05:30.089002] [INFO ] [MAIN ] ATO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:30.102183] [DEBUG] [MAIN ] ATI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:30.102936] [INFO ] [MAIN ] ATI: Generated 1794 derivative records from 51311 primary records. [2025-10-20T22:05:30.103104] [DEBUG] [LOCAL ] ATI: Saving derivative data. [2025-10-20T22:05:30.119325] [DEBUG] [LOCAL ] ATI: Derivative data saved (1794 records). [2025-10-20T22:05:30.119410] [INFO ] [MAIN ] ATI: Successfully saved derivative data. [2025-10-20T22:05:30.121453] [INFO ] [MAIN ] Derivative Progress: 132/1877 symbols completed. ATI took 1.1s. Estimated remaining: 11m 0s. Rate: 158.6 symbols/min [2025-10-20T22:05:30.134132] [INFO ] [MAIN ] ATR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:30.135274] [DEBUG] [LOCAL ] ATMU: Loaded 38808 existing records from primary storage. [2025-10-20T22:05:30.135748] [INFO ] [MAIN ] ATMU: Loaded 38808 primary data records. [2025-10-20T22:05:30.443586] [DEBUG] [LOCAL ] ATO: Loaded 41744 existing records from primary storage. [2025-10-20T22:05:30.497054] [DEBUG] [LOCAL ] ATR: Loaded 30554 existing records from primary storage. [2025-10-20T22:05:30.497210] [INFO ] [MAIN ] ATO: Loaded 41744 primary data records. [2025-10-20T22:05:30.498076] [INFO ] [MAIN ] ATR: Loaded 30554 primary data records. [2025-10-20T22:05:31.166034] [DEBUG] [MAIN ] ATMU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:31.167265] [INFO ] [MAIN ] ATMU: Generated 1794 derivative records from 38808 primary records. [2025-10-20T22:05:31.168233] [DEBUG] [LOCAL ] ATMU: Saving derivative data. [2025-10-20T22:05:31.188189] [DEBUG] [LOCAL ] ATMU: Derivative data saved (1794 records). [2025-10-20T22:05:31.188605] [INFO ] [MAIN ] ATMU: Successfully saved derivative data. [2025-10-20T22:05:31.191055] [INFO ] [MAIN ] Derivative Progress: 133/1877 symbols completed. ATMU took 1.2s. Estimated remaining: 11m 9s. Rate: 156.5 symbols/min [2025-10-20T22:05:31.208733] [DEBUG] [MAIN ] ATR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:31.209745] [INFO ] [MAIN ] ATR: Generated 1794 derivative records from 30554 primary records. [2025-10-20T22:05:31.210740] [DEBUG] [LOCAL ] ATR: Saving derivative data. [2025-10-20T22:05:31.238265] [DEBUG] [MAIN ] ATO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:31.244354] [INFO ] [MAIN ] ATO: Generated 1794 derivative records from 41744 primary records. [2025-10-20T22:05:31.244707] [DEBUG] [LOCAL ] ATO: Saving derivative data. [2025-10-20T22:05:31.246469] [DEBUG] [LOCAL ] ATR: Derivative data saved (1794 records). [2025-10-20T22:05:31.246852] [INFO ] [MAIN ] ATR: Successfully saved derivative data. [2025-10-20T22:05:31.249224] [INFO ] [MAIN ] Derivative Progress: 134/1877 symbols completed. ATR took 1.1s. Estimated remaining: 11m 4s. Rate: 157.5 symbols/min [2025-10-20T22:05:31.264322] [DEBUG] [LOCAL ] ATO: Derivative data saved (1794 records). [2025-10-20T22:05:31.264409] [INFO ] [MAIN ] ATO: Successfully saved derivative data. [2025-10-20T22:05:31.266250] [INFO ] [MAIN ] Derivative Progress: 135/1877 symbols completed. ATO took 1.2s. Estimated remaining: 10m 59s. Rate: 158.6 symbols/min [2025-10-20T22:05:31.266927] [INFO ] [MAIN ] AU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:31.266815] [INFO ] [MAIN ] ATS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:31.267461] [INFO ] [MAIN ] ATUS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:31.292306] [DEBUG] [LOCAL ] ATS: Loaded 17325 existing records from primary storage. [2025-10-20T22:05:31.292545] [INFO ] [MAIN ] ATS: Loaded 17325 primary data records. [2025-10-20T22:05:31.487395] [DEBUG] [LOCAL ] AU: Loaded 53089 existing records from primary storage. [2025-10-20T22:05:31.536819] [INFO ] [MAIN ] AU: Loaded 53089 primary data records. [2025-10-20T22:05:31.655869] [DEBUG] [LOCAL ] ATUS: Loaded 49947 existing records from primary storage. [2025-10-20T22:05:31.656352] [INFO ] [MAIN ] ATUS: Loaded 49947 primary data records. [2025-10-20T22:05:32.367212] [DEBUG] [MAIN ] ATS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:32.370266] [INFO ] [MAIN ] ATS: Generated 1794 derivative records from 17325 primary records. [2025-10-20T22:05:32.370572] [DEBUG] [LOCAL ] ATS: Saving derivative data. [2025-10-20T22:05:32.389780] [DEBUG] [LOCAL ] ATS: Derivative data saved (1794 records). [2025-10-20T22:05:32.390035] [INFO ] [MAIN ] ATS: Successfully saved derivative data. [2025-10-20T22:05:32.391968] [INFO ] [MAIN ] Derivative Progress: 136/1877 symbols completed. ATS took 1.1s. Estimated remaining: 11m 8s. Rate: 156.3 symbols/min [2025-10-20T22:05:32.392242] [INFO ] [MAIN ] AUB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:32.425807] [DEBUG] [MAIN ] ATUS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:32.426107] [DEBUG] [MAIN ] AU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:32.426968] [INFO ] [MAIN ] ATUS: Generated 1794 derivative records from 49947 primary records. [2025-10-20T22:05:32.427838] [INFO ] [MAIN ] AU: Generated 1794 derivative records from 53089 primary records. [2025-10-20T22:05:32.427105] [DEBUG] [LOCAL ] ATUS: Saving derivative data. [2025-10-20T22:05:32.428038] [DEBUG] [LOCAL ] AU: Saving derivative data. [2025-10-20T22:05:32.455846] [DEBUG] [LOCAL ] AU: Derivative data saved (1794 records). [2025-10-20T22:05:32.455985] [INFO ] [MAIN ] AU: Successfully saved derivative data. [2025-10-20T22:05:32.459175] [INFO ] [MAIN ] Derivative Progress: 137/1877 symbols completed. AU took 1.2s. Estimated remaining: 11m 4s. Rate: 157.3 symbols/min [2025-10-20T22:05:32.459795] [INFO ] [MAIN ] AUNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:32.473069] [DEBUG] [LOCAL ] ATUS: Derivative data saved (1794 records). [2025-10-20T22:05:32.473501] [INFO ] [MAIN ] ATUS: Successfully saved derivative data. [2025-10-20T22:05:32.476361] [INFO ] [MAIN ] Derivative Progress: 138/1877 symbols completed. ATUS took 1.2s. Estimated remaining: 10m 59s. Rate: 158.4 symbols/min [2025-10-20T22:05:32.476601] [INFO ] [MAIN ] AVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:32.482565] [DEBUG] [LOCAL ] AUNA: Loaded 5212 existing records from primary storage. [2025-10-20T22:05:32.483380] [INFO ] [MAIN ] AUNA: Loaded 5212 primary data records. [2025-10-20T22:05:32.589264] [DEBUG] [LOCAL ] AUB: Loaded 45950 existing records from primary storage. [2025-10-20T22:05:32.600312] [INFO ] [MAIN ] AUB: Loaded 45950 primary data records. [2025-10-20T22:05:32.940105] [DEBUG] [LOCAL ] AVA: Loaded 38503 existing records from primary storage. [2025-10-20T22:05:32.986394] [INFO ] [MAIN ] AVA: Loaded 38503 primary data records. [2025-10-20T22:05:33.442516] [DEBUG] [MAIN ] AUNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:33.444800] [INFO ] [MAIN ] AUNA: Generated 1794 derivative records from 5212 primary records. [2025-10-20T22:05:33.445011] [DEBUG] [LOCAL ] AUNA: Saving derivative data. [2025-10-20T22:05:33.462792] [DEBUG] [LOCAL ] AUNA: Derivative data saved (1794 records). [2025-10-20T22:05:33.470771] [INFO ] [MAIN ] AUNA: Successfully saved derivative data. [2025-10-20T22:05:33.472874] [INFO ] [MAIN ] Derivative Progress: 139/1877 symbols completed. AUNA took 1.0s. Estimated remaining: 11m 6s. Rate: 156.5 symbols/min [2025-10-20T22:05:33.477756] [INFO ] [MAIN ] AVAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:33.479537] [DEBUG] [MAIN ] AUB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:33.480242] [INFO ] [MAIN ] AUB: Generated 1794 derivative records from 45950 primary records. [2025-10-20T22:05:33.480494] [DEBUG] [LOCAL ] AUB: Saving derivative data. [2025-10-20T22:05:33.501058] [DEBUG] [LOCAL ] AUB: Derivative data saved (1794 records). [2025-10-20T22:05:33.501322] [INFO ] [MAIN ] AUB: Successfully saved derivative data. [2025-10-20T22:05:33.503398] [INFO ] [MAIN ] Derivative Progress: 140/1877 symbols completed. AUB took 1.1s. Estimated remaining: 11m 1s. Rate: 157.6 symbols/min [2025-10-20T22:05:33.511223] [INFO ] [MAIN ] AVB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:33.521314] [DEBUG] [LOCAL ] AVAL: Loaded 8528 existing records from primary storage. [2025-10-20T22:05:33.520530] [DEBUG] [MAIN ] AVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:33.521512] [INFO ] [MAIN ] AVAL: Loaded 8528 primary data records. [2025-10-20T22:05:33.522326] [INFO ] [MAIN ] AVA: Generated 1794 derivative records from 38503 primary records. [2025-10-20T22:05:33.522470] [DEBUG] [LOCAL ] AVA: Saving derivative data. [2025-10-20T22:05:33.624421] [DEBUG] [LOCAL ] AVA: Derivative data saved (1794 records). [2025-10-20T22:05:33.642638] [INFO ] [MAIN ] AVA: Successfully saved derivative data. [2025-10-20T22:05:33.644528] [INFO ] [MAIN ] Derivative Progress: 141/1877 symbols completed. AVA took 1.2s. Estimated remaining: 10m 58s. Rate: 158.3 symbols/min [2025-10-20T22:05:33.655418] [INFO ] [MAIN ] AVD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:33.859398] [DEBUG] [LOCAL ] AVB: Loaded 42300 existing records from primary storage. [2025-10-20T22:05:33.865362] [INFO ] [MAIN ] AVB: Loaded 42300 primary data records. [2025-10-20T22:05:33.978799] [DEBUG] [LOCAL ] AVD: Loaded 21480 existing records from primary storage. [2025-10-20T22:05:33.994580] [INFO ] [MAIN ] AVD: Loaded 21480 primary data records. [2025-10-20T22:05:34.519355] [DEBUG] [MAIN ] AVAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:34.521389] [INFO ] [MAIN ] AVAL: Generated 1794 derivative records from 8528 primary records. [2025-10-20T22:05:34.521576] [DEBUG] [LOCAL ] AVAL: Saving derivative data. [2025-10-20T22:05:34.537900] [DEBUG] [LOCAL ] AVAL: Derivative data saved (1794 records). [2025-10-20T22:05:34.538253] [INFO ] [MAIN ] AVAL: Successfully saved derivative data. [2025-10-20T22:05:34.541718] [INFO ] [MAIN ] Derivative Progress: 142/1877 symbols completed. AVAL took 1.1s. Estimated remaining: 11m 4s. Rate: 156.8 symbols/min [2025-10-20T22:05:34.542398] [INFO ] [MAIN ] AVK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:34.564245] [DEBUG] [MAIN ] AVB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:34.565371] [INFO ] [MAIN ] AVB: Generated 1794 derivative records from 42300 primary records. [2025-10-20T22:05:34.565561] [DEBUG] [LOCAL ] AVB: Saving derivative data. [2025-10-20T22:05:34.604513] [DEBUG] [LOCAL ] AVB: Derivative data saved (1794 records). [2025-10-20T22:05:34.609501] [INFO ] [MAIN ] AVB: Successfully saved derivative data. [2025-10-20T22:05:34.612240] [DEBUG] [LOCAL ] AVK: Loaded 18702 existing records from primary storage. [2025-10-20T22:05:34.611559] [INFO ] [MAIN ] Derivative Progress: 143/1877 symbols completed. AVB took 1.1s. Estimated remaining: 10m 60s. Rate: 157.7 symbols/min [2025-10-20T22:05:34.612400] [INFO ] [MAIN ] AVNS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:34.612538] [INFO ] [MAIN ] AVK: Loaded 18702 primary data records. [2025-10-20T22:05:34.613365] [DEBUG] [MAIN ] AVD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:34.651437] [INFO ] [MAIN ] AVD: Generated 1794 derivative records from 21480 primary records. [2025-10-20T22:05:34.670835] [DEBUG] [LOCAL ] AVD: Saving derivative data. [2025-10-20T22:05:34.836461] [DEBUG] [LOCAL ] AVD: Derivative data saved (1794 records). [2025-10-20T22:05:34.836830] [INFO ] [MAIN ] AVD: Successfully saved derivative data. [2025-10-20T22:05:34.964061] [INFO ] [MAIN ] Derivative Progress: 144/1877 symbols completed. AVD took 1.3s. Estimated remaining: 10m 59s. Rate: 157.8 symbols/min [2025-10-20T22:05:34.976148] [INFO ] [MAIN ] AVNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:34.990115] [DEBUG] [LOCAL ] AVNS: Loaded 33631 existing records from primary storage. [2025-10-20T22:05:34.991938] [INFO ] [MAIN ] AVNS: Loaded 33631 primary data records. [2025-10-20T22:05:35.266176] [DEBUG] [LOCAL ] AVNT: Loaded 38938 existing records from primary storage. [2025-10-20T22:05:35.420600] [INFO ] [MAIN ] AVNT: Loaded 38938 primary data records. [2025-10-20T22:05:35.471555] [DEBUG] [MAIN ] AVK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:35.515182] [INFO ] [MAIN ] AVK: Generated 1794 derivative records from 18702 primary records. [2025-10-20T22:05:35.545914] [DEBUG] [LOCAL ] AVK: Saving derivative data. [2025-10-20T22:05:35.848533] [DEBUG] [LOCAL ] AVK: Derivative data saved (1794 records). [2025-10-20T22:05:35.863886] [INFO ] [MAIN ] AVK: Successfully saved derivative data. [2025-10-20T22:05:35.866987] [INFO ] [MAIN ] Derivative Progress: 145/1877 symbols completed. AVK took 1.3s. Estimated remaining: 11m 5s. Rate: 156.3 symbols/min [2025-10-20T22:05:35.867664] [INFO ] [MAIN ] AVTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:35.890278] [DEBUG] [MAIN ] AVNS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:35.891141] [INFO ] [MAIN ] AVNS: Generated 1794 derivative records from 33631 primary records. [2025-10-20T22:05:35.891233] [DEBUG] [LOCAL ] AVNS: Saving derivative data. [2025-10-20T22:05:35.910423] [DEBUG] [LOCAL ] AVNS: Derivative data saved (1794 records). [2025-10-20T22:05:35.910875] [INFO ] [MAIN ] AVNS: Successfully saved derivative data. [2025-10-20T22:05:35.914725] [INFO ] [MAIN ] Derivative Progress: 146/1877 symbols completed. AVNS took 1.3s. Estimated remaining: 11m 1s. Rate: 157.2 symbols/min [2025-10-20T22:05:35.915081] [INFO ] [MAIN ] AVY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:35.934492] [DEBUG] [MAIN ] AVNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:35.951820] [INFO ] [MAIN ] AVNT: Generated 1794 derivative records from 38938 primary records. [2025-10-20T22:05:35.952026] [DEBUG] [LOCAL ] AVNT: Saving derivative data. [2025-10-20T22:05:35.972587] [DEBUG] [LOCAL ] AVNT: Derivative data saved (1794 records). [2025-10-20T22:05:35.973699] [INFO ] [MAIN ] AVNT: Successfully saved derivative data. [2025-10-20T22:05:35.976868] [INFO ] [MAIN ] Derivative Progress: 147/1877 symbols completed. AVNT took 1.0s. Estimated remaining: 10m 56s. Rate: 158.1 symbols/min [2025-10-20T22:05:35.994997] [INFO ] [MAIN ] AWF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:35.996687] [DEBUG] [LOCAL ] AVTR: Loaded 53572 existing records from primary storage. [2025-10-20T22:05:35.996910] [INFO ] [MAIN ] AVTR: Loaded 53572 primary data records. [2025-10-20T22:05:36.193561] [DEBUG] [LOCAL ] AVY: Loaded 38172 existing records from primary storage. [2025-10-20T22:05:36.223696] [INFO ] [MAIN ] AVY: Loaded 38172 primary data records. [2025-10-20T22:05:36.335798] [DEBUG] [LOCAL ] AWF: Loaded 24036 existing records from primary storage. [2025-10-20T22:05:36.357222] [INFO ] [MAIN ] AWF: Loaded 24036 primary data records. [2025-10-20T22:05:36.990528] [DEBUG] [MAIN ] AVTR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:36.991516] [INFO ] [MAIN ] AVTR: Generated 1794 derivative records from 53572 primary records. [2025-10-20T22:05:36.991948] [DEBUG] [LOCAL ] AVTR: Saving derivative data. [2025-10-20T22:05:37.009518] [DEBUG] [LOCAL ] AVTR: Derivative data saved (1794 records). [2025-10-20T22:05:37.009881] [INFO ] [MAIN ] AVTR: Successfully saved derivative data. [2025-10-20T22:05:37.011898] [INFO ] [MAIN ] Derivative Progress: 148/1877 symbols completed. AVTR took 1.1s. Estimated remaining: 11m 4s. Rate: 156.3 symbols/min [2025-10-20T22:05:37.027919] [DEBUG] [MAIN ] AVY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:37.028666] [INFO ] [MAIN ] AVY: Generated 1794 derivative records from 38172 primary records. [2025-10-20T22:05:37.028740] [DEBUG] [LOCAL ] AVY: Saving derivative data. [2025-10-20T22:05:37.058533] [DEBUG] [LOCAL ] AVY: Derivative data saved (1794 records). [2025-10-20T22:05:37.058241] [DEBUG] [MAIN ] AWF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:37.058630] [INFO ] [MAIN ] AVY: Successfully saved derivative data. [2025-10-20T22:05:37.059279] [INFO ] [MAIN ] AWF: Generated 1794 derivative records from 24036 primary records. [2025-10-20T22:05:37.061374] [INFO ] [MAIN ] Derivative Progress: 149/1877 symbols completed. AVY took 1.1s. Estimated remaining: 10m 60s. Rate: 157.2 symbols/min [2025-10-20T22:05:37.061481] [DEBUG] [LOCAL ] AWF: Saving derivative data. [2025-10-20T22:05:37.076163] [DEBUG] [LOCAL ] AWF: Derivative data saved (1794 records). [2025-10-20T22:05:37.076218] [INFO ] [MAIN ] AWF: Successfully saved derivative data. [2025-10-20T22:05:37.078298] [INFO ] [MAIN ] Derivative Progress: 150/1877 symbols completed. AWF took 1.1s. Estimated remaining: 10m 55s. Rate: 158.2 symbols/min [2025-10-20T22:05:37.078754] [INFO ] [MAIN ] AWK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:37.078703] [INFO ] [MAIN ] AWI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:37.078896] [INFO ] [MAIN ] AWR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:37.108571] [DEBUG] [LOCAL ] AWR: Loaded 22778 existing records from primary storage. [2025-10-20T22:05:37.108759] [INFO ] [MAIN ] AWR: Loaded 22778 primary data records. [2025-10-20T22:05:37.191005] [DEBUG] [LOCAL ] AWI: Loaded 25285 existing records from primary storage. [2025-10-20T22:05:37.219286] [INFO ] [MAIN ] AWI: Loaded 25285 primary data records. [2025-10-20T22:05:37.406269] [DEBUG] [LOCAL ] AWK: Loaded 48510 existing records from primary storage. [2025-10-20T22:05:37.538195] [INFO ] [MAIN ] AWK: Loaded 48510 primary data records. [2025-10-20T22:05:38.049298] [DEBUG] [MAIN ] AWR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:38.052143] [INFO ] [MAIN ] AWR: Generated 1794 derivative records from 22778 primary records. [2025-10-20T22:05:38.052410] [DEBUG] [LOCAL ] AWR: Saving derivative data. [2025-10-20T22:05:38.076537] [DEBUG] [MAIN ] AWI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:38.080278] [INFO ] [MAIN ] AWI: Generated 1793 derivative records from 25285 primary records. [2025-10-20T22:05:38.081393] [DEBUG] [LOCAL ] AWI: Saving derivative data. [2025-10-20T22:05:38.081320] [DEBUG] [LOCAL ] AWR: Derivative data saved (1794 records). [2025-10-20T22:05:38.082099] [INFO ] [MAIN ] AWR: Successfully saved derivative data. [2025-10-20T22:05:38.084308] [INFO ] [MAIN ] Derivative Progress: 151/1877 symbols completed. AWR took 1.0s. Estimated remaining: 11m 2s. Rate: 156.5 symbols/min [2025-10-20T22:05:38.095482] [INFO ] [MAIN ] AX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:38.099875] [DEBUG] [LOCAL ] AWI: Derivative data saved (1793 records). [2025-10-20T22:05:38.100240] [INFO ] [MAIN ] AWI: Successfully saved derivative data. [2025-10-20T22:05:38.102585] [INFO ] [MAIN ] Derivative Progress: 152/1877 symbols completed. AWI took 1.0s. Estimated remaining: 10m 57s. Rate: 157.5 symbols/min [2025-10-20T22:05:38.104273] [INFO ] [MAIN ] AXL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:38.119388] [DEBUG] [MAIN ] AWK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:38.120089] [INFO ] [MAIN ] AWK: Generated 1794 derivative records from 48510 primary records. [2025-10-20T22:05:38.120181] [DEBUG] [LOCAL ] AWK: Saving derivative data. [2025-10-20T22:05:38.143613] [DEBUG] [LOCAL ] AWK: Derivative data saved (1794 records). [2025-10-20T22:05:38.143845] [INFO ] [MAIN ] AWK: Successfully saved derivative data. [2025-10-20T22:05:38.145870] [INFO ] [MAIN ] Derivative Progress: 153/1877 symbols completed. AWK took 1.1s. Estimated remaining: 10m 53s. Rate: 158.4 symbols/min [2025-10-20T22:05:38.146124] [INFO ] [MAIN ] AXP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:38.157003] [DEBUG] [LOCAL ] AX: Loaded 28103 existing records from primary storage. [2025-10-20T22:05:38.157246] [INFO ] [MAIN ] AX: Loaded 28103 primary data records. [2025-10-20T22:05:38.364305] [DEBUG] [LOCAL ] AXL: Loaded 50539 existing records from primary storage. [2025-10-20T22:05:38.404792] [INFO ] [MAIN ] AXL: Loaded 50539 primary data records. [2025-10-20T22:05:38.652291] [DEBUG] [LOCAL ] AXP: Loaded 53169 existing records from primary storage. [2025-10-20T22:05:38.668004] [INFO ] [MAIN ] AXP: Loaded 53169 primary data records. [2025-10-20T22:05:39.160202] [DEBUG] [MAIN ] AX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:39.163023] [INFO ] [MAIN ] AX: Generated 1794 derivative records from 28103 primary records. [2025-10-20T22:05:39.163186] [DEBUG] [LOCAL ] AX: Saving derivative data. [2025-10-20T22:05:39.196975] [DEBUG] [LOCAL ] AX: Derivative data saved (1794 records). [2025-10-20T22:05:39.197424] [DEBUG] [MAIN ] AXL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:39.197626] [INFO ] [MAIN ] AX: Successfully saved derivative data. [2025-10-20T22:05:39.199898] [INFO ] [MAIN ] Derivative Progress: 154/1877 symbols completed. AX took 1.1s. Estimated remaining: 11m 0s. Rate: 156.6 symbols/min [2025-10-20T22:05:39.201145] [INFO ] [MAIN ] AXR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:39.200794] [INFO ] [MAIN ] AXL: Generated 1794 derivative records from 50539 primary records. [2025-10-20T22:05:39.201326] [DEBUG] [LOCAL ] AXL: Saving derivative data. [2025-10-20T22:05:39.220596] [DEBUG] [LOCAL ] AXL: Derivative data saved (1794 records). [2025-10-20T22:05:39.221705] [INFO ] [MAIN ] AXL: Successfully saved derivative data. [2025-10-20T22:05:39.226584] [DEBUG] [LOCAL ] AXR: Loaded 2925 existing records from primary storage. [2025-10-20T22:05:39.226347] [INFO ] [MAIN ] Derivative Progress: 155/1877 symbols completed. AXL took 1.1s. Estimated remaining: 10m 56s. Rate: 157.5 symbols/min [2025-10-20T22:05:39.234494] [INFO ] [MAIN ] AXR: Loaded 2925 primary data records. [2025-10-20T22:05:39.239495] [INFO ] [MAIN ] AXS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:39.277440] [DEBUG] [MAIN ] AXP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:39.311937] [INFO ] [MAIN ] AXP: Generated 1794 derivative records from 53169 primary records. [2025-10-20T22:05:39.328965] [DEBUG] [LOCAL ] AXP: Saving derivative data. [2025-10-20T22:05:39.402947] [DEBUG] [LOCAL ] AXP: Derivative data saved (1794 records). [2025-10-20T22:05:39.411759] [INFO ] [MAIN ] AXP: Successfully saved derivative data. [2025-10-20T22:05:39.439309] [INFO ] [MAIN ] Derivative Progress: 156/1877 symbols completed. AXP took 1.3s. Estimated remaining: 10m 54s. Rate: 158.0 symbols/min [2025-10-20T22:05:39.569872] [INFO ] [MAIN ] AXTA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:39.588975] [DEBUG] [LOCAL ] AXS: Loaded 38702 existing records from primary storage. [2025-10-20T22:05:39.589365] [INFO ] [MAIN ] AXS: Loaded 38702 primary data records. [2025-10-20T22:05:39.924966] [DEBUG] [LOCAL ] AXTA: Loaded 52286 existing records from primary storage. [2025-10-20T22:05:39.925450] [INFO ] [MAIN ] AXTA: Loaded 52286 primary data records. [2025-10-20T22:05:39.984479] [DEBUG] [MAIN ] AXR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:40.011883] [INFO ] [MAIN ] AXR: Generated 1794 derivative records from 2925 primary records. [2025-10-20T22:05:40.019614] [DEBUG] [LOCAL ] AXR: Saving derivative data. [2025-10-20T22:05:40.088975] [DEBUG] [LOCAL ] AXR: Derivative data saved (1794 records). [2025-10-20T22:05:40.142290] [INFO ] [MAIN ] AXR: Successfully saved derivative data. [2025-10-20T22:05:40.254576] [INFO ] [MAIN ] Derivative Progress: 157/1877 symbols completed. AXR took 1.1s. Estimated remaining: 10m 58s. Rate: 156.8 symbols/min [2025-10-20T22:05:40.263462] [INFO ] [MAIN ] AYI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:40.285851] [DEBUG] [MAIN ] AXS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:40.288041] [INFO ] [MAIN ] AXS: Generated 1794 derivative records from 38702 primary records. [2025-10-20T22:05:40.288381] [DEBUG] [LOCAL ] AXS: Saving derivative data. [2025-10-20T22:05:40.311706] [DEBUG] [LOCAL ] AXS: Derivative data saved (1794 records). [2025-10-20T22:05:40.320102] [INFO ] [MAIN ] AXS: Successfully saved derivative data. [2025-10-20T22:05:40.322873] [INFO ] [MAIN ] Derivative Progress: 158/1877 symbols completed. AXS took 1.1s. Estimated remaining: 10m 54s. Rate: 157.7 symbols/min [2025-10-20T22:05:40.323381] [INFO ] [MAIN ] AZO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:40.324677] [DEBUG] [LOCAL ] AYI: Loaded 24589 existing records from primary storage. [2025-10-20T22:05:40.324861] [INFO ] [MAIN ] AYI: Loaded 24589 primary data records. [2025-10-20T22:05:40.435469] [DEBUG] [MAIN ] AXTA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:40.446488] [DEBUG] [LOCAL ] AZO: Loaded 10945 existing records from primary storage. [2025-10-20T22:05:40.455017] [INFO ] [MAIN ] AXTA: Generated 1794 derivative records from 52286 primary records. [2025-10-20T22:05:40.455143] [DEBUG] [LOCAL ] AXTA: Saving derivative data. [2025-10-20T22:05:40.455293] [INFO ] [MAIN ] AZO: Loaded 10945 primary data records. [2025-10-20T22:05:40.801747] [DEBUG] [LOCAL ] AXTA: Derivative data saved (1794 records). [2025-10-20T22:05:40.816373] [INFO ] [MAIN ] AXTA: Successfully saved derivative data. [2025-10-20T22:05:40.818845] [INFO ] [MAIN ] Derivative Progress: 159/1877 symbols completed. AXTA took 1.2s. Estimated remaining: 10m 55s. Rate: 157.4 symbols/min [2025-10-20T22:05:40.829501] [INFO ] [MAIN ] AZZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:41.037298] [DEBUG] [LOCAL ] AZZ: Loaded 22882 existing records from primary storage. [2025-10-20T22:05:41.038034] [INFO ] [MAIN ] AZZ: Loaded 22882 primary data records. [2025-10-20T22:05:41.159033] [DEBUG] [MAIN ] AYI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:41.159902] [INFO ] [MAIN ] AYI: Generated 1793 derivative records from 24589 primary records. [2025-10-20T22:05:41.160106] [DEBUG] [LOCAL ] AYI: Saving derivative data. [2025-10-20T22:05:41.379419] [DEBUG] [MAIN ] AZO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:41.381929] [INFO ] [MAIN ] AZO: Generated 1794 derivative records from 10945 primary records. [2025-10-20T22:05:41.385417] [DEBUG] [LOCAL ] AZO: Saving derivative data. [2025-10-20T22:05:41.385490] [DEBUG] [LOCAL ] AYI: Derivative data saved (1793 records). [2025-10-20T22:05:41.386166] [INFO ] [MAIN ] AYI: Successfully saved derivative data. [2025-10-20T22:05:41.388268] [INFO ] [MAIN ] Derivative Progress: 160/1877 symbols completed. AYI took 1.1s. Estimated remaining: 10m 57s. Rate: 156.9 symbols/min [2025-10-20T22:05:41.388929] [INFO ] [MAIN ] BA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:41.409615] [DEBUG] [LOCAL ] AZO: Derivative data saved (1794 records). [2025-10-20T22:05:41.410271] [INFO ] [MAIN ] AZO: Successfully saved derivative data. [2025-10-20T22:05:41.412663] [INFO ] [MAIN ] Derivative Progress: 161/1877 symbols completed. AZO took 1.1s. Estimated remaining: 10m 52s. Rate: 157.8 symbols/min [2025-10-20T22:05:41.412864] [INFO ] [MAIN ] BABA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:41.450440] [DEBUG] [MAIN ] AZZ: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:41.451143] [INFO ] [MAIN ] AZZ: Generated 1793 derivative records from 22882 primary records. [2025-10-20T22:05:41.451305] [DEBUG] [LOCAL ] AZZ: Saving derivative data. [2025-10-20T22:05:41.469498] [DEBUG] [LOCAL ] AZZ: Derivative data saved (1793 records). [2025-10-20T22:05:41.469718] [INFO ] [MAIN ] AZZ: Successfully saved derivative data. [2025-10-20T22:05:41.487036] [INFO ] [MAIN ] Derivative Progress: 162/1877 symbols completed. AZZ took 0.7s. Estimated remaining: 10m 49s. Rate: 158.6 symbols/min [2025-10-20T22:05:41.487328] [INFO ] [MAIN ] BAH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:41.500836] [DEBUG] [LOCAL ] BA: Loaded 53790 existing records from primary storage. [2025-10-20T22:05:41.501213] [INFO ] [MAIN ] BA: Loaded 53790 primary data records. [2025-10-20T22:05:41.624572] [DEBUG] [LOCAL ] BABA: Loaded 53820 existing records from primary storage. [2025-10-20T22:05:41.630679] [INFO ] [MAIN ] BABA: Loaded 53820 primary data records. [2025-10-20T22:05:42.177430] [DEBUG] [LOCAL ] BAH: Loaded 51683 existing records from primary storage. [2025-10-20T22:05:42.178088] [INFO ] [MAIN ] BAH: Loaded 51683 primary data records. [2025-10-20T22:05:42.551195] [DEBUG] [MAIN ] BA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:42.552920] [INFO ] [MAIN ] BA: Generated 1794 derivative records from 53790 primary records. [2025-10-20T22:05:42.553273] [DEBUG] [LOCAL ] BA: Saving derivative data. [2025-10-20T22:05:42.574221] [DEBUG] [LOCAL ] BA: Derivative data saved (1794 records). [2025-10-20T22:05:42.575101] [INFO ] [MAIN ] BA: Successfully saved derivative data. [2025-10-20T22:05:42.577113] [INFO ] [MAIN ] Derivative Progress: 163/1877 symbols completed. BA took 1.2s. Estimated remaining: 10m 56s. Rate: 156.8 symbols/min [2025-10-20T22:05:42.590283] [DEBUG] [MAIN ] BABA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:42.591066] [INFO ] [MAIN ] BABA: Generated 1794 derivative records from 53820 primary records. [2025-10-20T22:05:42.591674] [DEBUG] [LOCAL ] BABA: Saving derivative data. [2025-10-20T22:05:42.609352] [DEBUG] [LOCAL ] BABA: Derivative data saved (1794 records). [2025-10-20T22:05:42.609572] [INFO ] [MAIN ] BABA: Successfully saved derivative data. [2025-10-20T22:05:42.611755] [INFO ] [MAIN ] Derivative Progress: 164/1877 symbols completed. BABA took 1.2s. Estimated remaining: 10m 52s. Rate: 157.6 symbols/min [2025-10-20T22:05:42.627982] [DEBUG] [MAIN ] BAH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:42.628590] [INFO ] [MAIN ] BAH: Generated 1794 derivative records from 51683 primary records. [2025-10-20T22:05:42.628674] [DEBUG] [LOCAL ] BAH: Saving derivative data. [2025-10-20T22:05:42.644033] [DEBUG] [LOCAL ] BAH: Derivative data saved (1794 records). [2025-10-20T22:05:42.644098] [INFO ] [MAIN ] BAH: Successfully saved derivative data. [2025-10-20T22:05:42.646271] [INFO ] [MAIN ] Derivative Progress: 165/1877 symbols completed. BAH took 1.2s. Estimated remaining: 10m 48s. Rate: 158.5 symbols/min [2025-10-20T22:05:42.646774] [INFO ] [MAIN ] BALL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:42.646851] [INFO ] [MAIN ] BAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:42.646704] [INFO ] [MAIN ] BAK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:42.708087] [DEBUG] [LOCAL ] BAK: Loaded 39438 existing records from primary storage. [2025-10-20T22:05:42.708284] [INFO ] [MAIN ] BAK: Loaded 39438 primary data records. [2025-10-20T22:05:42.838298] [DEBUG] [LOCAL ] BAM: Loaded 52103 existing records from primary storage. [2025-10-20T22:05:42.838405] [DEBUG] [LOCAL ] BALL: Loaded 51927 existing records from primary storage. [2025-10-20T22:05:42.841724] [INFO ] [MAIN ] BAM: Loaded 52103 primary data records. [2025-10-20T22:05:42.841837] [INFO ] [MAIN ] BALL: Loaded 51927 primary data records. [2025-10-20T22:05:43.684311] [DEBUG] [MAIN ] BAK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:43.685442] [INFO ] [MAIN ] BAK: Generated 1794 derivative records from 39438 primary records. [2025-10-20T22:05:43.685541] [DEBUG] [LOCAL ] BAK: Saving derivative data. [2025-10-20T22:05:43.731065] [DEBUG] [LOCAL ] BAK: Derivative data saved (1794 records). [2025-10-20T22:05:43.736909] [INFO ] [MAIN ] BAK: Successfully saved derivative data. [2025-10-20T22:05:43.731389] [DEBUG] [MAIN ] BALL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:43.737248] [DEBUG] [MAIN ] BAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:43.739014] [INFO ] [MAIN ] Derivative Progress: 166/1877 symbols completed. BAK took 1.1s. Estimated remaining: 10m 55s. Rate: 156.7 symbols/min [2025-10-20T22:05:43.741108] [INFO ] [MAIN ] BANC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:43.740004] [INFO ] [MAIN ] BALL: Generated 1794 derivative records from 51927 primary records. [2025-10-20T22:05:43.740978] [INFO ] [MAIN ] BAM: Generated 1794 derivative records from 52103 primary records. [2025-10-20T22:05:43.741766] [DEBUG] [LOCAL ] BAM: Saving derivative data. [2025-10-20T22:05:43.741619] [DEBUG] [LOCAL ] BALL: Saving derivative data. [2025-10-20T22:05:43.770615] [DEBUG] [LOCAL ] BALL: Derivative data saved (1794 records). [2025-10-20T22:05:43.771881] [INFO ] [MAIN ] BALL: Successfully saved derivative data. [2025-10-20T22:05:43.775748] [INFO ] [MAIN ] Derivative Progress: 167/1877 symbols completed. BALL took 1.1s. Estimated remaining: 10m 51s. Rate: 157.6 symbols/min [2025-10-20T22:05:43.776294] [INFO ] [MAIN ] BAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:43.779014] [DEBUG] [LOCAL ] BAM: Derivative data saved (1794 records). [2025-10-20T22:05:43.780873] [INFO ] [MAIN ] BAM: Successfully saved derivative data. [2025-10-20T22:05:43.783291] [INFO ] [MAIN ] Derivative Progress: 168/1877 symbols completed. BAM took 1.1s. Estimated remaining: 10m 47s. Rate: 158.5 symbols/min [2025-10-20T22:05:43.783674] [INFO ] [MAIN ] BARK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:43.820558] [DEBUG] [LOCAL ] BAP: Loaded 25879 existing records from primary storage. [2025-10-20T22:05:43.822270] [INFO ] [MAIN ] BAP: Loaded 25879 primary data records. [2025-10-20T22:05:43.929548] [DEBUG] [LOCAL ] BANC: Loaded 51387 existing records from primary storage. [2025-10-20T22:05:43.952586] [INFO ] [MAIN ] BANC: Loaded 51387 primary data records. [2025-10-20T22:05:44.085423] [DEBUG] [LOCAL ] BARK: Loaded 38200 existing records from primary storage. [2025-10-20T22:05:44.233499] [INFO ] [MAIN ] BARK: Loaded 38200 primary data records. [2025-10-20T22:05:44.833953] [DEBUG] [MAIN ] BAP: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:44.834778] [INFO ] [MAIN ] BAP: Generated 1793 derivative records from 25879 primary records. [2025-10-20T22:05:44.834990] [DEBUG] [LOCAL ] BAP: Saving derivative data. [2025-10-20T22:05:44.857682] [DEBUG] [LOCAL ] BAP: Derivative data saved (1793 records). [2025-10-20T22:05:44.858086] [INFO ] [MAIN ] BAP: Successfully saved derivative data. [2025-10-20T22:05:44.860000] [INFO ] [MAIN ] Derivative Progress: 169/1877 symbols completed. BAP took 1.1s. Estimated remaining: 10m 54s. Rate: 156.8 symbols/min [2025-10-20T22:05:44.874402] [INFO ] [MAIN ] BAX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:44.874718] [DEBUG] [MAIN ] BANC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:44.875515] [INFO ] [MAIN ] BANC: Generated 1794 derivative records from 51387 primary records. [2025-10-20T22:05:44.875959] [DEBUG] [LOCAL ] BANC: Saving derivative data. [2025-10-20T22:05:44.907619] [DEBUG] [MAIN ] BARK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:44.910269] [INFO ] [MAIN ] BARK: Generated 1794 derivative records from 38200 primary records. [2025-10-20T22:05:44.911077] [DEBUG] [LOCAL ] BARK: Saving derivative data. [2025-10-20T22:05:44.926742] [DEBUG] [LOCAL ] BANC: Derivative data saved (1794 records). [2025-10-20T22:05:44.931579] [INFO ] [MAIN ] BANC: Successfully saved derivative data. [2025-10-20T22:05:44.931481] [DEBUG] [LOCAL ] BARK: Derivative data saved (1794 records). [2025-10-20T22:05:44.934091] [INFO ] [MAIN ] BARK: Successfully saved derivative data. [2025-10-20T22:05:44.934019] [INFO ] [MAIN ] Derivative Progress: 170/1877 symbols completed. BANC took 1.2s. Estimated remaining: 10m 50s. Rate: 157.6 symbols/min [2025-10-20T22:05:44.934400] [INFO ] [MAIN ] BBAI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:44.936595] [INFO ] [MAIN ] Derivative Progress: 171/1877 symbols completed. BARK took 1.2s. Estimated remaining: 10m 46s. Rate: 158.5 symbols/min [2025-10-20T22:05:44.936739] [INFO ] [MAIN ] BBAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:44.980638] [DEBUG] [LOCAL ] BAX: Loaded 53537 existing records from primary storage. [2025-10-20T22:05:44.981008] [INFO ] [MAIN ] BAX: Loaded 53537 primary data records. [2025-10-20T22:05:45.222153] [DEBUG] [LOCAL ] BBAR: Loaded 42660 existing records from primary storage. [2025-10-20T22:05:45.222395] [INFO ] [MAIN ] BBAR: Loaded 42660 primary data records. [2025-10-20T22:05:45.243992] [DEBUG] [LOCAL ] BBAI: Loaded 53820 existing records from primary storage. [2025-10-20T22:05:45.259610] [INFO ] [MAIN ] BBAI: Loaded 53820 primary data records. [2025-10-20T22:05:45.982192] [DEBUG] [MAIN ] BAX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:45.984120] [INFO ] [MAIN ] BAX: Generated 1794 derivative records from 53537 primary records. [2025-10-20T22:05:45.984528] [DEBUG] [LOCAL ] BAX: Saving derivative data. [2025-10-20T22:05:46.002255] [DEBUG] [LOCAL ] BAX: Derivative data saved (1794 records). [2025-10-20T22:05:46.002420] [INFO ] [MAIN ] BAX: Successfully saved derivative data. [2025-10-20T22:05:46.004371] [INFO ] [MAIN ] Derivative Progress: 172/1877 symbols completed. BAX took 1.1s. Estimated remaining: 10m 52s. Rate: 156.8 symbols/min [2025-10-20T22:05:46.005339] [INFO ] [MAIN ] BBD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:46.025803] [DEBUG] [MAIN ] BBAI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:46.026728] [INFO ] [MAIN ] BBAI: Generated 1794 derivative records from 53820 primary records. [2025-10-20T22:05:46.026919] [DEBUG] [LOCAL ] BBAI: Saving derivative data. [2025-10-20T22:05:46.040055] [DEBUG] [MAIN ] BBAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:46.041514] [INFO ] [MAIN ] BBAR: Generated 1794 derivative records from 42660 primary records. [2025-10-20T22:05:46.056273] [DEBUG] [LOCAL ] BBAI: Derivative data saved (1794 records). [2025-10-20T22:05:46.056441] [DEBUG] [LOCAL ] BBAR: Saving derivative data. [2025-10-20T22:05:46.057359] [INFO ] [MAIN ] BBAI: Successfully saved derivative data. [2025-10-20T22:05:46.059488] [INFO ] [MAIN ] Derivative Progress: 173/1877 symbols completed. BBAI took 1.1s. Estimated remaining: 10m 49s. Rate: 157.6 symbols/min [2025-10-20T22:05:46.066735] [INFO ] [MAIN ] BBDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:46.075322] [DEBUG] [LOCAL ] BBAR: Derivative data saved (1794 records). [2025-10-20T22:05:46.075589] [INFO ] [MAIN ] BBAR: Successfully saved derivative data. [2025-10-20T22:05:46.077575] [INFO ] [MAIN ] Derivative Progress: 174/1877 symbols completed. BBAR took 1.1s. Estimated remaining: 10m 45s. Rate: 158.5 symbols/min [2025-10-20T22:05:46.091570] [INFO ] [MAIN ] BBDO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:46.099944] [DEBUG] [LOCAL ] BBDO: Loaded 5012 existing records from primary storage. [2025-10-20T22:05:46.115591] [INFO ] [MAIN ] BBDO: Loaded 5012 primary data records. [2025-10-20T22:05:46.145341] [DEBUG] [LOCAL ] BBD: Loaded 53276 existing records from primary storage. [2025-10-20T22:05:46.156561] [INFO ] [MAIN ] BBD: Loaded 53276 primary data records. [2025-10-20T22:05:46.295129] [DEBUG] [LOCAL ] BBDC: Loaded 33048 existing records from primary storage. [2025-10-20T22:05:46.320689] [INFO ] [MAIN ] BBDC: Loaded 33048 primary data records. [2025-10-20T22:05:47.136979] [DEBUG] [MAIN ] BBDO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:47.153007] [INFO ] [MAIN ] BBDO: Generated 1794 derivative records from 5012 primary records. [2025-10-20T22:05:47.154157] [DEBUG] [MAIN ] BBD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:47.154548] [DEBUG] [LOCAL ] BBDO: Saving derivative data. [2025-10-20T22:05:47.156100] [INFO ] [MAIN ] BBD: Generated 1794 derivative records from 53276 primary records. [2025-10-20T22:05:47.170159] [DEBUG] [LOCAL ] BBD: Saving derivative data. [2025-10-20T22:05:47.183427] [DEBUG] [LOCAL ] BBDO: Derivative data saved (1794 records). [2025-10-20T22:05:47.195107] [INFO ] [MAIN ] BBDO: Successfully saved derivative data. [2025-10-20T22:05:47.190978] [DEBUG] [MAIN ] BBDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:47.198038] [INFO ] [MAIN ] BBDC: Generated 1794 derivative records from 33048 primary records. [2025-10-20T22:05:47.197120] [INFO ] [MAIN ] Derivative Progress: 175/1877 symbols completed. BBDO took 1.1s. Estimated remaining: 10m 52s. Rate: 156.7 symbols/min [2025-10-20T22:05:47.201861] [DEBUG] [LOCAL ] BBD: Derivative data saved (1794 records). [2025-10-20T22:05:47.201958] [INFO ] [MAIN ] BBD: Successfully saved derivative data. [2025-10-20T22:05:47.203998] [INFO ] [MAIN ] BBN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:47.204120] [INFO ] [MAIN ] Derivative Progress: 176/1877 symbols completed. BBD took 1.2s. Estimated remaining: 10m 48s. Rate: 157.6 symbols/min [2025-10-20T22:05:47.204566] [INFO ] [MAIN ] BBU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:47.204874] [DEBUG] [LOCAL ] BBDC: Saving derivative data. [2025-10-20T22:05:47.221316] [DEBUG] [LOCAL ] BBDC: Derivative data saved (1794 records). [2025-10-20T22:05:47.222925] [INFO ] [MAIN ] BBDC: Successfully saved derivative data. [2025-10-20T22:05:47.225148] [INFO ] [MAIN ] Derivative Progress: 177/1877 symbols completed. BBDC took 1.2s. Estimated remaining: 10m 44s. Rate: 158.4 symbols/min [2025-10-20T22:05:47.225506] [INFO ] [MAIN ] BBUC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:47.229744] [DEBUG] [LOCAL ] BBU: Loaded 5727 existing records from primary storage. [2025-10-20T22:05:47.229876] [INFO ] [MAIN ] BBU: Loaded 5727 primary data records. [2025-10-20T22:05:47.350174] [DEBUG] [LOCAL ] BBN: Loaded 17924 existing records from primary storage. [2025-10-20T22:05:47.355960] [INFO ] [MAIN ] BBN: Loaded 17924 primary data records. [2025-10-20T22:05:47.403858] [DEBUG] [LOCAL ] BBUC: Loaded 13935 existing records from primary storage. [2025-10-20T22:05:47.606071] [INFO ] [MAIN ] BBUC: Loaded 13935 primary data records. [2025-10-20T22:05:48.199142] [DEBUG] [MAIN ] BBU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:48.200110] [INFO ] [MAIN ] BBU: Generated 1794 derivative records from 5727 primary records. [2025-10-20T22:05:48.200554] [DEBUG] [LOCAL ] BBU: Saving derivative data. [2025-10-20T22:05:48.217903] [DEBUG] [LOCAL ] BBU: Derivative data saved (1794 records). [2025-10-20T22:05:48.218027] [INFO ] [MAIN ] BBU: Successfully saved derivative data. [2025-10-20T22:05:48.220819] [INFO ] [MAIN ] Derivative Progress: 178/1877 symbols completed. BBU took 1.0s. Estimated remaining: 10m 49s. Rate: 157.0 symbols/min [2025-10-20T22:05:48.235262] [DEBUG] [MAIN ] BBN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:48.236072] [INFO ] [MAIN ] BBN: Generated 1794 derivative records from 17924 primary records. [2025-10-20T22:05:48.236251] [DEBUG] [LOCAL ] BBN: Saving derivative data. [2025-10-20T22:05:48.267301] [DEBUG] [LOCAL ] BBN: Derivative data saved (1794 records). [2025-10-20T22:05:48.267703] [INFO ] [MAIN ] BBN: Successfully saved derivative data. [2025-10-20T22:05:48.267524] [DEBUG] [MAIN ] BBUC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:48.269675] [INFO ] [MAIN ] Derivative Progress: 179/1877 symbols completed. BBN took 1.1s. Estimated remaining: 10m 46s. Rate: 157.8 symbols/min [2025-10-20T22:05:48.270376] [INFO ] [MAIN ] BBUC: Generated 1794 derivative records from 13935 primary records. [2025-10-20T22:05:48.270470] [DEBUG] [LOCAL ] BBUC: Saving derivative data. [2025-10-20T22:05:48.285304] [DEBUG] [LOCAL ] BBUC: Derivative data saved (1794 records). [2025-10-20T22:05:48.285360] [INFO ] [MAIN ] BBUC: Successfully saved derivative data. [2025-10-20T22:05:48.287307] [INFO ] [MAIN ] Derivative Progress: 180/1877 symbols completed. BBUC took 1.1s. Estimated remaining: 10m 42s. Rate: 158.6 symbols/min [2025-10-20T22:05:48.287717] [INFO ] [MAIN ] BBWI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:48.287677] [INFO ] [MAIN ] BBVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:48.287825] [INFO ] [MAIN ] BBW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:48.327251] [DEBUG] [LOCAL ] BBW: Loaded 27378 existing records from primary storage. [2025-10-20T22:05:48.341062] [INFO ] [MAIN ] BBW: Loaded 27378 primary data records. [2025-10-20T22:05:48.477136] [DEBUG] [LOCAL ] BBVA: Loaded 49119 existing records from primary storage. [2025-10-20T22:05:48.476393] [DEBUG] [LOCAL ] BBWI: Loaded 53584 existing records from primary storage. [2025-10-20T22:05:48.493185] [INFO ] [MAIN ] BBVA: Loaded 49119 primary data records. [2025-10-20T22:05:48.498695] [INFO ] [MAIN ] BBWI: Loaded 53584 primary data records. [2025-10-20T22:05:49.311933] [DEBUG] [MAIN ] BBW: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:49.312666] [INFO ] [MAIN ] BBW: Generated 1793 derivative records from 27378 primary records. [2025-10-20T22:05:49.312770] [DEBUG] [LOCAL ] BBW: Saving derivative data. [2025-10-20T22:05:49.330109] [DEBUG] [LOCAL ] BBW: Derivative data saved (1793 records). [2025-10-20T22:05:49.330345] [INFO ] [MAIN ] BBW: Successfully saved derivative data. [2025-10-20T22:05:49.332992] [INFO ] [MAIN ] Derivative Progress: 181/1877 symbols completed. BBW took 1.0s. Estimated remaining: 10m 48s. Rate: 157.1 symbols/min [2025-10-20T22:05:49.333722] [INFO ] [MAIN ] BBY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:49.349990] [DEBUG] [MAIN ] BBVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:49.362540] [INFO ] [MAIN ] BBVA: Generated 1794 derivative records from 49119 primary records. [2025-10-20T22:05:49.363078] [DEBUG] [MAIN ] BBWI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:49.363165] [DEBUG] [LOCAL ] BBVA: Saving derivative data. [2025-10-20T22:05:49.363766] [INFO ] [MAIN ] BBWI: Generated 1793 derivative records from 53584 primary records. [2025-10-20T22:05:49.364377] [DEBUG] [LOCAL ] BBWI: Saving derivative data. [2025-10-20T22:05:49.392771] [DEBUG] [LOCAL ] BBVA: Derivative data saved (1794 records). [2025-10-20T22:05:49.393888] [INFO ] [MAIN ] BBVA: Successfully saved derivative data. [2025-10-20T22:05:49.396140] [INFO ] [MAIN ] Derivative Progress: 182/1877 symbols completed. BBVA took 1.1s. Estimated remaining: 10m 44s. Rate: 157.8 symbols/min [2025-10-20T22:05:49.396909] [DEBUG] [LOCAL ] BBWI: Derivative data saved (1793 records). [2025-10-20T22:05:49.397234] [INFO ] [MAIN ] BBWI: Successfully saved derivative data. [2025-10-20T22:05:49.397081] [INFO ] [MAIN ] BC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:49.400105] [INFO ] [MAIN ] Derivative Progress: 183/1877 symbols completed. BBWI took 1.1s. Estimated remaining: 10m 41s. Rate: 158.7 symbols/min [2025-10-20T22:05:49.400530] [INFO ] [MAIN ] BCAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:49.449392] [DEBUG] [LOCAL ] BBY: Loaded 53436 existing records from primary storage. [2025-10-20T22:05:49.459182] [INFO ] [MAIN ] BBY: Loaded 53436 primary data records. [2025-10-20T22:05:49.590033] [DEBUG] [LOCAL ] BCAT: Loaded 34298 existing records from primary storage. [2025-10-20T22:05:49.595757] [INFO ] [MAIN ] BCAT: Loaded 34298 primary data records. [2025-10-20T22:05:49.596331] [DEBUG] [LOCAL ] BC: Loaded 43999 existing records from primary storage. [2025-10-20T22:05:49.674758] [INFO ] [MAIN ] BC: Loaded 43999 primary data records. [2025-10-20T22:05:50.436714] [DEBUG] [MAIN ] BBY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:50.437823] [INFO ] [MAIN ] BBY: Generated 1794 derivative records from 53436 primary records. [2025-10-20T22:05:50.438245] [DEBUG] [LOCAL ] BBY: Saving derivative data. [2025-10-20T22:05:50.454722] [DEBUG] [LOCAL ] BBY: Derivative data saved (1794 records). [2025-10-20T22:05:50.456451] [INFO ] [MAIN ] BBY: Successfully saved derivative data. [2025-10-20T22:05:50.458371] [INFO ] [MAIN ] Derivative Progress: 184/1877 symbols completed. BBY took 1.1s. Estimated remaining: 10m 47s. Rate: 157.1 symbols/min [2025-10-20T22:05:50.459364] [INFO ] [MAIN ] BCC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:50.473883] [DEBUG] [MAIN ] BCAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:50.474721] [INFO ] [MAIN ] BCAT: Generated 1794 derivative records from 34298 primary records. [2025-10-20T22:05:50.474864] [DEBUG] [LOCAL ] BCAT: Saving derivative data. [2025-10-20T22:05:50.492491] [DEBUG] [LOCAL ] BCAT: Derivative data saved (1794 records). [2025-10-20T22:05:50.493277] [INFO ] [MAIN ] BCAT: Successfully saved derivative data. [2025-10-20T22:05:50.495701] [INFO ] [MAIN ] Derivative Progress: 185/1877 symbols completed. BCAT took 1.1s. Estimated remaining: 10m 43s. Rate: 157.9 symbols/min [2025-10-20T22:05:50.495891] [INFO ] [MAIN ] BCE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:50.517757] [DEBUG] [MAIN ] BC: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:50.518530] [INFO ] [MAIN ] BC: Generated 1793 derivative records from 43999 primary records. [2025-10-20T22:05:50.518681] [DEBUG] [LOCAL ] BC: Saving derivative data. [2025-10-20T22:05:50.544080] [DEBUG] [LOCAL ] BC: Derivative data saved (1793 records). [2025-10-20T22:05:50.544252] [INFO ] [MAIN ] BC: Successfully saved derivative data. [2025-10-20T22:05:50.546351] [INFO ] [MAIN ] Derivative Progress: 186/1877 symbols completed. BC took 1.1s. Estimated remaining: 10m 40s. Rate: 158.6 symbols/min [2025-10-20T22:05:50.546510] [INFO ] [MAIN ] BCH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:50.547437] [DEBUG] [LOCAL ] BCC: Loaded 28671 existing records from primary storage. [2025-10-20T22:05:50.548017] [INFO ] [MAIN ] BCC: Loaded 28671 primary data records. [2025-10-20T22:05:50.903326] [DEBUG] [LOCAL ] BCE: Loaded 52271 existing records from primary storage. [2025-10-20T22:05:50.896983] [DEBUG] [LOCAL ] BCH: Loaded 30410 existing records from primary storage. [2025-10-20T22:05:50.912771] [INFO ] [MAIN ] BCH: Loaded 30410 primary data records. [2025-10-20T22:05:50.912852] [INFO ] [MAIN ] BCE: Loaded 52271 primary data records. [2025-10-20T22:05:51.544611] [DEBUG] [MAIN ] BCC: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:51.545793] [INFO ] [MAIN ] BCC: Generated 1793 derivative records from 28671 primary records. [2025-10-20T22:05:51.546166] [DEBUG] [LOCAL ] BCC: Saving derivative data. [2025-10-20T22:05:51.565103] [DEBUG] [LOCAL ] BCC: Derivative data saved (1793 records). [2025-10-20T22:05:51.565625] [INFO ] [MAIN ] BCC: Successfully saved derivative data. [2025-10-20T22:05:51.567857] [INFO ] [MAIN ] Derivative Progress: 187/1877 symbols completed. BCC took 1.1s. Estimated remaining: 10m 45s. Rate: 157.2 symbols/min [2025-10-20T22:05:51.568179] [INFO ] [MAIN ] BCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:51.595386] [DEBUG] [MAIN ] BCH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:51.600507] [DEBUG] [MAIN ] BCE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:51.602253] [INFO ] [MAIN ] BCE: Generated 1794 derivative records from 52271 primary records. [2025-10-20T22:05:51.601393] [INFO ] [MAIN ] BCH: Generated 1794 derivative records from 30410 primary records. [2025-10-20T22:05:51.602475] [DEBUG] [LOCAL ] BCE: Saving derivative data. [2025-10-20T22:05:51.602958] [DEBUG] [LOCAL ] BCH: Saving derivative data. [2025-10-20T22:05:51.625923] [DEBUG] [LOCAL ] BCE: Derivative data saved (1794 records). [2025-10-20T22:05:51.633298] [INFO ] [MAIN ] BCE: Successfully saved derivative data. [2025-10-20T22:05:51.637018] [INFO ] [MAIN ] Derivative Progress: 188/1877 symbols completed. BCE took 1.1s. Estimated remaining: 10m 42s. Rate: 157.9 symbols/min [2025-10-20T22:05:51.641360] [INFO ] [MAIN ] BCSF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:51.644550] [DEBUG] [LOCAL ] BCH: Derivative data saved (1794 records). [2025-10-20T22:05:51.646293] [INFO ] [MAIN ] BCH: Successfully saved derivative data. [2025-10-20T22:05:51.656748] [INFO ] [MAIN ] Derivative Progress: 189/1877 symbols completed. BCH took 1.1s. Estimated remaining: 10m 38s. Rate: 158.7 symbols/min [2025-10-20T22:05:51.657510] [INFO ] [MAIN ] BCX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:51.657300] [DEBUG] [LOCAL ] BCO: Loaded 25219 existing records from primary storage. [2025-10-20T22:05:51.658104] [INFO ] [MAIN ] BCO: Loaded 25219 primary data records. [2025-10-20T22:05:51.860769] [DEBUG] [LOCAL ] BCX: Loaded 22104 existing records from primary storage. [2025-10-20T22:05:51.892306] [INFO ] [MAIN ] BCX: Loaded 22104 primary data records. [2025-10-20T22:05:51.999851] [DEBUG] [LOCAL ] BCSF: Loaded 28311 existing records from primary storage. [2025-10-20T22:05:52.009269] [INFO ] [MAIN ] BCSF: Loaded 28311 primary data records. [2025-10-20T22:05:52.690199] [DEBUG] [MAIN ] BCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:52.691965] [INFO ] [MAIN ] BCO: Generated 1794 derivative records from 25219 primary records. [2025-10-20T22:05:52.692130] [DEBUG] [LOCAL ] BCO: Saving derivative data. [2025-10-20T22:05:52.709926] [DEBUG] [LOCAL ] BCO: Derivative data saved (1794 records). [2025-10-20T22:05:52.710184] [INFO ] [MAIN ] BCO: Successfully saved derivative data. [2025-10-20T22:05:52.712727] [INFO ] [MAIN ] Derivative Progress: 190/1877 symbols completed. BCO took 1.1s. Estimated remaining: 10m 44s. Rate: 157.2 symbols/min [2025-10-20T22:05:52.713006] [INFO ] [MAIN ] BDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:52.733750] [DEBUG] [MAIN ] BCX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:52.735147] [INFO ] [MAIN ] BCX: Generated 1794 derivative records from 22104 primary records. [2025-10-20T22:05:52.735311] [DEBUG] [LOCAL ] BCX: Saving derivative data. [2025-10-20T22:05:52.749661] [DEBUG] [MAIN ] BCSF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:52.750708] [INFO ] [MAIN ] BCSF: Generated 1794 derivative records from 28311 primary records. [2025-10-20T22:05:52.751096] [DEBUG] [LOCAL ] BCSF: Saving derivative data. [2025-10-20T22:05:52.766059] [DEBUG] [LOCAL ] BCX: Derivative data saved (1794 records). [2025-10-20T22:05:52.766225] [INFO ] [MAIN ] BCX: Successfully saved derivative data. [2025-10-20T22:05:52.770878] [INFO ] [MAIN ] Derivative Progress: 191/1877 symbols completed. BCX took 1.1s. Estimated remaining: 10m 41s. Rate: 157.9 symbols/min [2025-10-20T22:05:52.791258] [INFO ] [MAIN ] BDJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:52.796578] [DEBUG] [LOCAL ] BCSF: Derivative data saved (1794 records). [2025-10-20T22:05:52.796930] [INFO ] [MAIN ] BCSF: Successfully saved derivative data. [2025-10-20T22:05:52.799276] [INFO ] [MAIN ] Derivative Progress: 192/1877 symbols completed. BCSF took 1.2s. Estimated remaining: 10m 37s. Rate: 158.7 symbols/min [2025-10-20T22:05:52.799554] [INFO ] [MAIN ] BDN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:52.815342] [DEBUG] [LOCAL ] BDC: Loaded 24267 existing records from primary storage. [2025-10-20T22:05:52.815681] [INFO ] [MAIN ] BDC: Loaded 24267 primary data records. [2025-10-20T22:05:52.953737] [DEBUG] [LOCAL ] BDJ: Loaded 29444 existing records from primary storage. [2025-10-20T22:05:52.957511] [INFO ] [MAIN ] BDJ: Loaded 29444 primary data records. [2025-10-20T22:05:53.312321] [DEBUG] [LOCAL ] BDN: Loaded 47639 existing records from primary storage. [2025-10-20T22:05:53.317889] [INFO ] [MAIN ] BDN: Loaded 47639 primary data records. [2025-10-20T22:05:53.863614] [DEBUG] [MAIN ] BDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:53.865062] [INFO ] [MAIN ] BDC: Generated 1794 derivative records from 24267 primary records. [2025-10-20T22:05:53.865317] [DEBUG] [LOCAL ] BDC: Saving derivative data. [2025-10-20T22:05:53.885254] [DEBUG] [LOCAL ] BDC: Derivative data saved (1794 records). [2025-10-20T22:05:53.885556] [INFO ] [MAIN ] BDC: Successfully saved derivative data. [2025-10-20T22:05:53.887774] [INFO ] [MAIN ] Derivative Progress: 193/1877 symbols completed. BDC took 1.2s. Estimated remaining: 10m 43s. Rate: 157.1 symbols/min [2025-10-20T22:05:53.900956] [DEBUG] [MAIN ] BDJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:53.901826] [INFO ] [MAIN ] BDJ: Generated 1794 derivative records from 29444 primary records. [2025-10-20T22:05:53.901975] [DEBUG] [LOCAL ] BDJ: Saving derivative data. [2025-10-20T22:05:53.917765] [DEBUG] [LOCAL ] BDJ: Derivative data saved (1794 records). [2025-10-20T22:05:53.918078] [INFO ] [MAIN ] BDJ: Successfully saved derivative data. [2025-10-20T22:05:53.920152] [INFO ] [MAIN ] Derivative Progress: 194/1877 symbols completed. BDJ took 1.1s. Estimated remaining: 10m 40s. Rate: 157.9 symbols/min [2025-10-20T22:05:53.935416] [DEBUG] [MAIN ] BDN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:53.936018] [INFO ] [MAIN ] BDN: Generated 1794 derivative records from 47639 primary records. [2025-10-20T22:05:53.936078] [DEBUG] [LOCAL ] BDN: Saving derivative data. [2025-10-20T22:05:53.954098] [DEBUG] [LOCAL ] BDN: Derivative data saved (1794 records). [2025-10-20T22:05:53.954178] [INFO ] [MAIN ] BDN: Successfully saved derivative data. [2025-10-20T22:05:53.956826] [INFO ] [MAIN ] Derivative Progress: 195/1877 symbols completed. BDN took 1.2s. Estimated remaining: 10m 36s. Rate: 158.6 symbols/min [2025-10-20T22:05:53.957263] [INFO ] [MAIN ] BDX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:53.957338] [INFO ] [MAIN ] BEKE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:53.957471] [INFO ] [MAIN ] BEP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:54.025849] [DEBUG] [LOCAL ] BEP: Loaded 39006 existing records from primary storage. [2025-10-20T22:05:54.026007] [INFO ] [MAIN ] BEP: Loaded 39006 primary data records. [2025-10-20T22:05:54.136302] [DEBUG] [LOCAL ] BEKE: Loaded 53198 existing records from primary storage. [2025-10-20T22:05:54.187397] [INFO ] [MAIN ] BEKE: Loaded 53198 primary data records. [2025-10-20T22:05:54.179739] [DEBUG] [LOCAL ] BDX: Loaded 52166 existing records from primary storage. [2025-10-20T22:05:54.191188] [INFO ] [MAIN ] BDX: Loaded 52166 primary data records. [2025-10-20T22:05:55.057667] [DEBUG] [MAIN ] BEP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:55.061191] [INFO ] [MAIN ] BEP: Generated 1794 derivative records from 39006 primary records. [2025-10-20T22:05:55.061341] [DEBUG] [LOCAL ] BEP: Saving derivative data. [2025-10-20T22:05:55.092779] [DEBUG] [LOCAL ] BEP: Derivative data saved (1794 records). [2025-10-20T22:05:55.093284] [DEBUG] [MAIN ] BEKE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:55.094809] [INFO ] [MAIN ] BEKE: Generated 1794 derivative records from 53198 primary records. [2025-10-20T22:05:55.094003] [INFO ] [MAIN ] BEP: Successfully saved derivative data. [2025-10-20T22:05:55.096754] [INFO ] [MAIN ] Derivative Progress: 196/1877 symbols completed. BEP took 1.1s. Estimated remaining: 10m 42s. Rate: 157.0 symbols/min [2025-10-20T22:05:55.096852] [DEBUG] [LOCAL ] BEKE: Saving derivative data. [2025-10-20T22:05:55.097761] [INFO ] [MAIN ] BEPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:55.121008] [DEBUG] [LOCAL ] BEKE: Derivative data saved (1794 records). [2025-10-20T22:05:55.121106] [INFO ] [MAIN ] BEKE: Successfully saved derivative data. [2025-10-20T22:05:55.128561] [INFO ] [MAIN ] Derivative Progress: 197/1877 symbols completed. BEKE took 1.2s. Estimated remaining: 10m 39s. Rate: 157.7 symbols/min [2025-10-20T22:05:55.129206] [INFO ] [MAIN ] BEPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:55.128936] [DEBUG] [MAIN ] BDX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:55.130171] [INFO ] [MAIN ] BDX: Generated 1794 derivative records from 52166 primary records. [2025-10-20T22:05:55.130671] [DEBUG] [LOCAL ] BDX: Saving derivative data. [2025-10-20T22:05:55.148472] [DEBUG] [LOCAL ] BDX: Derivative data saved (1794 records). [2025-10-20T22:05:55.148819] [INFO ] [MAIN ] BDX: Successfully saved derivative data. [2025-10-20T22:05:55.150960] [INFO ] [MAIN ] Derivative Progress: 198/1877 symbols completed. BDX took 1.2s. Estimated remaining: 10m 36s. Rate: 158.5 symbols/min [2025-10-20T22:05:55.151292] [INFO ] [MAIN ] BEPI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:55.158187] [DEBUG] [LOCAL ] BEPH: Loaded 4950 existing records from primary storage. [2025-10-20T22:05:55.158399] [INFO ] [MAIN ] BEPH: Loaded 4950 primary data records. [2025-10-20T22:05:55.219810] [DEBUG] [LOCAL ] BEPI: Loaded 3594 existing records from primary storage. [2025-10-20T22:05:55.239826] [INFO ] [MAIN ] BEPI: Loaded 3594 primary data records. [2025-10-20T22:05:55.466954] [DEBUG] [LOCAL ] BEPC: Loaded 46428 existing records from primary storage. [2025-10-20T22:05:55.483112] [INFO ] [MAIN ] BEPC: Loaded 46428 primary data records. [2025-10-20T22:05:56.142007] [DEBUG] [MAIN ] BEPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:56.141604] [DEBUG] [MAIN ] BEPI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:56.142965] [INFO ] [MAIN ] BEPH: Generated 1794 derivative records from 4950 primary records. [2025-10-20T22:05:56.143849] [INFO ] [MAIN ] BEPI: Generated 1794 derivative records from 3594 primary records. [2025-10-20T22:05:56.143222] [DEBUG] [LOCAL ] BEPH: Saving derivative data. [2025-10-20T22:05:56.144298] [DEBUG] [LOCAL ] BEPI: Saving derivative data. [2025-10-20T22:05:56.173194] [DEBUG] [LOCAL ] BEPI: Derivative data saved (1794 records). [2025-10-20T22:05:56.173387] [INFO ] [MAIN ] BEPI: Successfully saved derivative data. [2025-10-20T22:05:56.173134] [DEBUG] [LOCAL ] BEPH: Derivative data saved (1794 records). [2025-10-20T22:05:56.177632] [INFO ] [MAIN ] Derivative Progress: 199/1877 symbols completed. BEPI took 1.0s. Estimated remaining: 10m 41s. Rate: 157.1 symbols/min [2025-10-20T22:05:56.175063] [INFO ] [MAIN ] BEPH: Successfully saved derivative data. [2025-10-20T22:05:56.178072] [INFO ] [MAIN ] BEPJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:56.179990] [INFO ] [MAIN ] Derivative Progress: 200/1877 symbols completed. BEPH took 1.1s. Estimated remaining: 10m 37s. Rate: 157.9 symbols/min [2025-10-20T22:05:56.180197] [INFO ] [MAIN ] BFAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:56.195274] [DEBUG] [MAIN ] BEPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:56.196131] [INFO ] [MAIN ] BEPC: Generated 1794 derivative records from 46428 primary records. [2025-10-20T22:05:56.196344] [DEBUG] [LOCAL ] BEPC: Saving derivative data. [2025-10-20T22:05:56.199253] [DEBUG] [LOCAL ] BEPJ: Loaded 3084 existing records from primary storage. [2025-10-20T22:05:56.199599] [INFO ] [MAIN ] BEPJ: Loaded 3084 primary data records. [2025-10-20T22:05:56.297406] [DEBUG] [LOCAL ] BEPC: Derivative data saved (1794 records). [2025-10-20T22:05:56.316803] [INFO ] [MAIN ] BEPC: Successfully saved derivative data. [2025-10-20T22:05:56.330403] [INFO ] [MAIN ] Derivative Progress: 201/1877 symbols completed. BEPC took 1.2s. Estimated remaining: 10m 35s. Rate: 158.4 symbols/min [2025-10-20T22:05:56.330765] [INFO ] [MAIN ] BFH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:56.552328] [DEBUG] [LOCAL ] BFAM: Loaded 33897 existing records from primary storage. [2025-10-20T22:05:56.561163] [INFO ] [MAIN ] BFAM: Loaded 33897 primary data records. [2025-10-20T22:05:56.783813] [DEBUG] [LOCAL ] BFH: Loaded 41581 existing records from primary storage. [2025-10-20T22:05:56.889493] [INFO ] [MAIN ] BFH: Loaded 41581 primary data records. [2025-10-20T22:05:57.225308] [DEBUG] [MAIN ] BEPJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:57.226437] [INFO ] [MAIN ] BEPJ: Generated 1794 derivative records from 3084 primary records. [2025-10-20T22:05:57.226520] [DEBUG] [LOCAL ] BEPJ: Saving derivative data. [2025-10-20T22:05:57.243610] [DEBUG] [LOCAL ] BEPJ: Derivative data saved (1794 records). [2025-10-20T22:05:57.244112] [INFO ] [MAIN ] BEPJ: Successfully saved derivative data. [2025-10-20T22:05:57.246245] [INFO ] [MAIN ] Derivative Progress: 202/1877 symbols completed. BEPJ took 1.1s. Estimated remaining: 10m 39s. Rate: 157.3 symbols/min [2025-10-20T22:05:57.246502] [INFO ] [MAIN ] BFLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:57.264942] [DEBUG] [MAIN ] BFAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:57.265739] [INFO ] [MAIN ] BFAM: Generated 1794 derivative records from 33897 primary records. [2025-10-20T22:05:57.266021] [DEBUG] [LOCAL ] BFAM: Saving derivative data. [2025-10-20T22:05:57.284016] [DEBUG] [LOCAL ] BFAM: Derivative data saved (1794 records). [2025-10-20T22:05:57.284586] [INFO ] [MAIN ] BFAM: Successfully saved derivative data. [2025-10-20T22:05:57.286753] [INFO ] [MAIN ] Derivative Progress: 203/1877 symbols completed. BFAM took 1.1s. Estimated remaining: 10m 36s. Rate: 158.0 symbols/min [2025-10-20T22:05:57.286985] [INFO ] [MAIN ] BFS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:57.318253] [DEBUG] [MAIN ] BFH: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:05:57.319133] [INFO ] [MAIN ] BFH: Generated 1793 derivative records from 41581 primary records. [2025-10-20T22:05:57.319258] [DEBUG] [LOCAL ] BFH: Saving derivative data. [2025-10-20T22:05:57.331756] [DEBUG] [LOCAL ] BFS: Loaded 9614 existing records from primary storage. [2025-10-20T22:05:57.340129] [DEBUG] [LOCAL ] BFH: Derivative data saved (1793 records). [2025-10-20T22:05:57.354444] [INFO ] [MAIN ] BFH: Successfully saved derivative data. [2025-10-20T22:05:57.358083] [INFO ] [MAIN ] BFS: Loaded 9614 primary data records. [2025-10-20T22:05:57.357963] [INFO ] [MAIN ] Derivative Progress: 204/1877 symbols completed. BFH took 1.0s. Estimated remaining: 10m 33s. Rate: 158.6 symbols/min [2025-10-20T22:05:57.363617] [INFO ] [MAIN ] BFZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:57.402853] [DEBUG] [LOCAL ] BFLY: Loaded 51857 existing records from primary storage. [2025-10-20T22:05:57.423734] [INFO ] [MAIN ] BFLY: Loaded 51857 primary data records. [2025-10-20T22:05:57.654773] [DEBUG] [LOCAL ] BFZ: Loaded 12050 existing records from primary storage. [2025-10-20T22:05:57.767345] [INFO ] [MAIN ] BFZ: Loaded 12050 primary data records. [2025-10-20T22:05:58.355633] [DEBUG] [MAIN ] BFLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:58.357250] [INFO ] [MAIN ] BFLY: Generated 1794 derivative records from 51857 primary records. [2025-10-20T22:05:58.356530] [DEBUG] [MAIN ] BFS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:58.358061] [DEBUG] [LOCAL ] BFLY: Saving derivative data. [2025-10-20T22:05:58.359037] [INFO ] [MAIN ] BFS: Generated 1794 derivative records from 9614 primary records. [2025-10-20T22:05:58.359461] [DEBUG] [LOCAL ] BFS: Saving derivative data. [2025-10-20T22:05:58.383192] [DEBUG] [LOCAL ] BFLY: Derivative data saved (1794 records). [2025-10-20T22:05:58.386605] [INFO ] [MAIN ] BFLY: Successfully saved derivative data. [2025-10-20T22:05:58.389300] [INFO ] [MAIN ] Derivative Progress: 205/1877 symbols completed. BFLY took 1.1s. Estimated remaining: 10m 38s. Rate: 157.3 symbols/min [2025-10-20T22:05:58.392214] [INFO ] [MAIN ] BG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:58.395580] [DEBUG] [LOCAL ] BFS: Derivative data saved (1794 records). [2025-10-20T22:05:58.395972] [INFO ] [MAIN ] BFS: Successfully saved derivative data. [2025-10-20T22:05:58.398412] [INFO ] [MAIN ] Derivative Progress: 206/1877 symbols completed. BFS took 1.1s. Estimated remaining: 10m 34s. Rate: 158.0 symbols/min [2025-10-20T22:05:58.398737] [INFO ] [MAIN ] BGB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:58.416208] [DEBUG] [MAIN ] BFZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:58.417040] [INFO ] [MAIN ] BFZ: Generated 1794 derivative records from 12050 primary records. [2025-10-20T22:05:58.417223] [DEBUG] [LOCAL ] BFZ: Saving derivative data. [2025-10-20T22:05:58.439205] [DEBUG] [LOCAL ] BFZ: Derivative data saved (1794 records). [2025-10-20T22:05:58.439397] [INFO ] [MAIN ] BFZ: Successfully saved derivative data. [2025-10-20T22:05:58.441866] [INFO ] [MAIN ] Derivative Progress: 207/1877 symbols completed. BFZ took 1.1s. Estimated remaining: 10m 31s. Rate: 158.7 symbols/min [2025-10-20T22:05:58.442114] [INFO ] [MAIN ] BGH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:58.462752] [DEBUG] [LOCAL ] BGB: Loaded 17876 existing records from primary storage. [2025-10-20T22:05:58.462910] [INFO ] [MAIN ] BGB: Loaded 17876 primary data records. [2025-10-20T22:05:58.593406] [DEBUG] [LOCAL ] BG: Loaded 50585 existing records from primary storage. [2025-10-20T22:05:58.615531] [DEBUG] [LOCAL ] BGH: Loaded 17201 existing records from primary storage. [2025-10-20T22:05:58.636479] [INFO ] [MAIN ] BGH: Loaded 17201 primary data records. [2025-10-20T22:05:58.620740] [INFO ] [MAIN ] BG: Loaded 50585 primary data records. [2025-10-20T22:05:59.433888] [DEBUG] [MAIN ] BGB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:59.435178] [INFO ] [MAIN ] BGB: Generated 1794 derivative records from 17876 primary records. [2025-10-20T22:05:59.435405] [DEBUG] [LOCAL ] BGB: Saving derivative data. [2025-10-20T22:05:59.454617] [DEBUG] [LOCAL ] BGB: Derivative data saved (1794 records). [2025-10-20T22:05:59.455573] [INFO ] [MAIN ] BGB: Successfully saved derivative data. [2025-10-20T22:05:59.465988] [INFO ] [MAIN ] Derivative Progress: 208/1877 symbols completed. BGB took 1.1s. Estimated remaining: 10m 36s. Rate: 157.4 symbols/min [2025-10-20T22:05:59.473609] [DEBUG] [MAIN ] BGH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:59.474629] [INFO ] [MAIN ] BGH: Generated 1794 derivative records from 17201 primary records. [2025-10-20T22:05:59.474841] [DEBUG] [LOCAL ] BGH: Saving derivative data. [2025-10-20T22:05:59.488133] [DEBUG] [MAIN ] BG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:05:59.488826] [INFO ] [MAIN ] BG: Generated 1794 derivative records from 50585 primary records. [2025-10-20T22:05:59.488888] [DEBUG] [LOCAL ] BG: Saving derivative data. [2025-10-20T22:05:59.521675] [DEBUG] [LOCAL ] BGH: Derivative data saved (1794 records). [2025-10-20T22:05:59.522344] [DEBUG] [LOCAL ] BG: Derivative data saved (1794 records). [2025-10-20T22:05:59.522439] [INFO ] [MAIN ] BGH: Successfully saved derivative data. [2025-10-20T22:05:59.522585] [INFO ] [MAIN ] BG: Successfully saved derivative data. [2025-10-20T22:05:59.525118] [INFO ] [MAIN ] Derivative Progress: 209/1877 symbols completed. BGH took 1.1s. Estimated remaining: 10m 33s. Rate: 158.1 symbols/min [2025-10-20T22:05:59.528111] [INFO ] [MAIN ] Derivative Progress: 210/1877 symbols completed. BG took 1.1s. Estimated remaining: 10m 30s. Rate: 158.8 symbols/min [2025-10-20T22:05:59.528534] [INFO ] [MAIN ] BGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:59.528719] [INFO ] [MAIN ] BGSF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:59.528601] [INFO ] [MAIN ] BGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:05:59.544934] [DEBUG] [LOCAL ] BGSF: Loaded 8128 existing records from primary storage. [2025-10-20T22:05:59.545125] [INFO ] [MAIN ] BGSF: Loaded 8128 primary data records. [2025-10-20T22:05:59.620674] [DEBUG] [LOCAL ] BGR: Loaded 13505 existing records from primary storage. [2025-10-20T22:05:59.650240] [INFO ] [MAIN ] BGR: Loaded 13505 primary data records. [2025-10-20T22:05:59.971916] [DEBUG] [LOCAL ] BGS: Loaded 50571 existing records from primary storage. [2025-10-20T22:05:59.982343] [INFO ] [MAIN ] BGS: Loaded 50571 primary data records. [2025-10-20T22:06:00.532372] [DEBUG] [MAIN ] BGSF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:00.533722] [INFO ] [MAIN ] BGSF: Generated 1794 derivative records from 8128 primary records. [2025-10-20T22:06:00.533933] [DEBUG] [LOCAL ] BGSF: Saving derivative data. [2025-10-20T22:06:00.552057] [DEBUG] [LOCAL ] BGSF: Derivative data saved (1794 records). [2025-10-20T22:06:00.552437] [INFO ] [MAIN ] BGSF: Successfully saved derivative data. [2025-10-20T22:06:00.554566] [INFO ] [MAIN ] Derivative Progress: 211/1877 symbols completed. BGSF took 1.0s. Estimated remaining: 10m 35s. Rate: 157.5 symbols/min [2025-10-20T22:06:00.554843] [INFO ] [MAIN ] BGT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:00.568231] [DEBUG] [MAIN ] BGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:00.569018] [INFO ] [MAIN ] BGR: Generated 1794 derivative records from 13505 primary records. [2025-10-20T22:06:00.569114] [DEBUG] [LOCAL ] BGR: Saving derivative data. [2025-10-20T22:06:00.585708] [DEBUG] [LOCAL ] BGR: Derivative data saved (1794 records). [2025-10-20T22:06:00.586436] [INFO ] [MAIN ] BGR: Successfully saved derivative data. [2025-10-20T22:06:00.593888] [INFO ] [MAIN ] Derivative Progress: 212/1877 symbols completed. BGR took 1.1s. Estimated remaining: 10m 31s. Rate: 158.2 symbols/min [2025-10-20T22:06:00.595014] [INFO ] [MAIN ] BGY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:00.609399] [DEBUG] [MAIN ] BGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:00.615943] [DEBUG] [LOCAL ] BGT: Loaded 16314 existing records from primary storage. [2025-10-20T22:06:00.615634] [INFO ] [MAIN ] BGS: Generated 1794 derivative records from 50571 primary records. [2025-10-20T22:06:00.616198] [DEBUG] [LOCAL ] BGS: Saving derivative data. [2025-10-20T22:06:00.616052] [INFO ] [MAIN ] BGT: Loaded 16314 primary data records. [2025-10-20T22:06:00.667614] [DEBUG] [LOCAL ] BGS: Derivative data saved (1794 records). [2025-10-20T22:06:00.691754] [INFO ] [MAIN ] BGS: Successfully saved derivative data. [2025-10-20T22:06:00.708821] [INFO ] [MAIN ] Derivative Progress: 213/1877 symbols completed. BGS took 1.2s. Estimated remaining: 10m 29s. Rate: 158.7 symbols/min [2025-10-20T22:06:00.720968] [INFO ] [MAIN ] BHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:00.752554] [DEBUG] [LOCAL ] BGY: Loaded 21632 existing records from primary storage. [2025-10-20T22:06:00.771020] [INFO ] [MAIN ] BGY: Loaded 21632 primary data records. [2025-10-20T22:06:01.155831] [DEBUG] [LOCAL ] BHC: Loaded 46695 existing records from primary storage. [2025-10-20T22:06:01.156223] [INFO ] [MAIN ] BHC: Loaded 46695 primary data records. [2025-10-20T22:06:01.595304] [DEBUG] [MAIN ] BGT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:01.597383] [INFO ] [MAIN ] BGT: Generated 1794 derivative records from 16314 primary records. [2025-10-20T22:06:01.599244] [DEBUG] [LOCAL ] BGT: Saving derivative data. [2025-10-20T22:06:01.625937] [DEBUG] [MAIN ] BGY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:01.630444] [INFO ] [MAIN ] BGY: Generated 1794 derivative records from 21632 primary records. [2025-10-20T22:06:01.629519] [DEBUG] [LOCAL ] BGT: Derivative data saved (1794 records). [2025-10-20T22:06:01.630632] [INFO ] [MAIN ] BGT: Successfully saved derivative data. [2025-10-20T22:06:01.632692] [INFO ] [MAIN ] Derivative Progress: 214/1877 symbols completed. BGT took 1.1s. Estimated remaining: 10m 33s. Rate: 157.7 symbols/min [2025-10-20T22:06:01.630584] [DEBUG] [LOCAL ] BGY: Saving derivative data. [2025-10-20T22:06:01.632973] [INFO ] [MAIN ] BHE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:01.650719] [DEBUG] [LOCAL ] BGY: Derivative data saved (1794 records). [2025-10-20T22:06:01.650875] [INFO ] [MAIN ] BGY: Successfully saved derivative data. [2025-10-20T22:06:01.653036] [INFO ] [MAIN ] Derivative Progress: 215/1877 symbols completed. BGY took 1.1s. Estimated remaining: 10m 30s. Rate: 158.4 symbols/min [2025-10-20T22:06:01.653357] [INFO ] [MAIN ] BHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:01.677768] [DEBUG] [MAIN ] BHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:01.678559] [INFO ] [MAIN ] BHC: Generated 1794 derivative records from 46695 primary records. [2025-10-20T22:06:01.678699] [DEBUG] [LOCAL ] BHC: Saving derivative data. [2025-10-20T22:06:01.698798] [DEBUG] [LOCAL ] BHE: Loaded 23853 existing records from primary storage. [2025-10-20T22:06:01.703338] [INFO ] [MAIN ] BHE: Loaded 23853 primary data records. [2025-10-20T22:06:01.773287] [DEBUG] [LOCAL ] BHC: Derivative data saved (1794 records). [2025-10-20T22:06:01.786076] [INFO ] [MAIN ] BHC: Successfully saved derivative data. [2025-10-20T22:06:01.825753] [DEBUG] [LOCAL ] BHR: Loaded 22056 existing records from primary storage. [2025-10-20T22:06:01.824632] [INFO ] [MAIN ] Derivative Progress: 216/1877 symbols completed. BHC took 1.1s. Estimated remaining: 10m 28s. Rate: 158.8 symbols/min [2025-10-20T22:06:01.830899] [INFO ] [MAIN ] BHR: Loaded 22056 primary data records. [2025-10-20T22:06:01.831526] [INFO ] [MAIN ] BILL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:02.479800] [DEBUG] [LOCAL ] BILL: Loaded 52513 existing records from primary storage. [2025-10-20T22:06:02.480557] [INFO ] [MAIN ] BILL: Loaded 52513 primary data records. [2025-10-20T22:06:02.618035] [DEBUG] [MAIN ] BHE: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:02.625197] [INFO ] [MAIN ] BHE: Generated 1793 derivative records from 23853 primary records. [2025-10-20T22:06:02.635789] [DEBUG] [LOCAL ] BHE: Saving derivative data. [2025-10-20T22:06:02.845928] [DEBUG] [LOCAL ] BHE: Derivative data saved (1793 records). [2025-10-20T22:06:02.854474] [INFO ] [MAIN ] BHE: Successfully saved derivative data. [2025-10-20T22:06:02.856593] [INFO ] [MAIN ] Derivative Progress: 217/1877 symbols completed. BHE took 1.2s. Estimated remaining: 10m 32s. Rate: 157.5 symbols/min [2025-10-20T22:06:02.863917] [INFO ] [MAIN ] BIPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:02.863668] [DEBUG] [MAIN ] BHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:02.867096] [INFO ] [MAIN ] BHR: Generated 1794 derivative records from 22056 primary records. [2025-10-20T22:06:02.867209] [DEBUG] [LOCAL ] BHR: Saving derivative data. [2025-10-20T22:06:02.885805] [DEBUG] [LOCAL ] BHR: Derivative data saved (1794 records). [2025-10-20T22:06:02.886145] [INFO ] [MAIN ] BHR: Successfully saved derivative data. [2025-10-20T22:06:02.888383] [INFO ] [MAIN ] Derivative Progress: 218/1877 symbols completed. BHR took 1.2s. Estimated remaining: 10m 29s. Rate: 158.2 symbols/min [2025-10-20T22:06:02.890621] [INFO ] [MAIN ] BIPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:02.897151] [DEBUG] [LOCAL ] BIPH: Loaded 2395 existing records from primary storage. [2025-10-20T22:06:02.906999] [INFO ] [MAIN ] BIPH: Loaded 2395 primary data records. [2025-10-20T22:06:03.048107] [DEBUG] [LOCAL ] BIPC: Loaded 41246 existing records from primary storage. [2025-10-20T22:06:03.048455] [DEBUG] [MAIN ] BILL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:03.048680] [INFO ] [MAIN ] BIPC: Loaded 41246 primary data records. [2025-10-20T22:06:03.050026] [INFO ] [MAIN ] BILL: Generated 1794 derivative records from 52513 primary records. [2025-10-20T22:06:03.060624] [DEBUG] [LOCAL ] BILL: Saving derivative data. [2025-10-20T22:06:03.353798] [DEBUG] [LOCAL ] BILL: Derivative data saved (1794 records). [2025-10-20T22:06:03.372765] [INFO ] [MAIN ] BILL: Successfully saved derivative data. [2025-10-20T22:06:03.385408] [INFO ] [MAIN ] Derivative Progress: 219/1877 symbols completed. BILL took 1.6s. Estimated remaining: 10m 30s. Rate: 158.0 symbols/min [2025-10-20T22:06:03.391405] [INFO ] [MAIN ] BIPJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:03.571870] [DEBUG] [LOCAL ] BIPJ: Loaded 3133 existing records from primary storage. [2025-10-20T22:06:03.572046] [INFO ] [MAIN ] BIPJ: Loaded 3133 primary data records. [2025-10-20T22:06:03.902778] [DEBUG] [MAIN ] BIPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:03.903809] [INFO ] [MAIN ] BIPH: Generated 1794 derivative records from 2395 primary records. [2025-10-20T22:06:03.904919] [DEBUG] [LOCAL ] BIPH: Saving derivative data. [2025-10-20T22:06:03.935635] [DEBUG] [LOCAL ] BIPH: Derivative data saved (1794 records). [2025-10-20T22:06:03.936588] [INFO ] [MAIN ] BIPH: Successfully saved derivative data. [2025-10-20T22:06:03.936501] [DEBUG] [MAIN ] BIPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:03.939926] [INFO ] [MAIN ] BIPC: Generated 1794 derivative records from 41246 primary records. [2025-10-20T22:06:03.938821] [INFO ] [MAIN ] Derivative Progress: 220/1877 symbols completed. BIPH took 1.0s. Estimated remaining: 10m 31s. Rate: 157.6 symbols/min [2025-10-20T22:06:03.940253] [INFO ] [MAIN ] BIRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:03.940315] [DEBUG] [LOCAL ] BIPC: Saving derivative data. [2025-10-20T22:06:03.958965] [DEBUG] [LOCAL ] BIPC: Derivative data saved (1794 records). [2025-10-20T22:06:03.960034] [INFO ] [MAIN ] BIPC: Successfully saved derivative data. [2025-10-20T22:06:03.963012] [INFO ] [MAIN ] Derivative Progress: 221/1877 symbols completed. BIPC took 1.1s. Estimated remaining: 10m 28s. Rate: 158.3 symbols/min [2025-10-20T22:06:03.963365] [INFO ] [MAIN ] BIT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:03.977144] [DEBUG] [MAIN ] BIPJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:03.977883] [INFO ] [MAIN ] BIPJ: Generated 1794 derivative records from 3133 primary records. [2025-10-20T22:06:03.978015] [DEBUG] [LOCAL ] BIPJ: Saving derivative data. [2025-10-20T22:06:03.992787] [DEBUG] [LOCAL ] BIPJ: Derivative data saved (1794 records). [2025-10-20T22:06:04.005788] [INFO ] [MAIN ] BIPJ: Successfully saved derivative data. [2025-10-20T22:06:04.007988] [INFO ] [MAIN ] Derivative Progress: 222/1877 symbols completed. BIPJ took 0.6s. Estimated remaining: 10m 25s. Rate: 158.9 symbols/min [2025-10-20T22:06:04.008333] [INFO ] [MAIN ] BJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:04.025972] [DEBUG] [LOCAL ] BIT: Loaded 23780 existing records from primary storage. [2025-10-20T22:06:04.041167] [INFO ] [MAIN ] BIT: Loaded 23780 primary data records. [2025-10-20T22:06:04.047682] [DEBUG] [LOCAL ] BIRK: Loaded 51198 existing records from primary storage. [2025-10-20T22:06:04.066182] [INFO ] [MAIN ] BIRK: Loaded 51198 primary data records. [2025-10-20T22:06:04.620519] [DEBUG] [LOCAL ] BJ: Loaded 51581 existing records from primary storage. [2025-10-20T22:06:04.716632] [INFO ] [MAIN ] BJ: Loaded 51581 primary data records. [2025-10-20T22:06:05.039170] [DEBUG] [MAIN ] BIT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:05.052068] [INFO ] [MAIN ] BIT: Generated 1794 derivative records from 23780 primary records. [2025-10-20T22:06:05.052362] [DEBUG] [LOCAL ] BIT: Saving derivative data. [2025-10-20T22:06:05.052600] [DEBUG] [MAIN ] BIRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:05.054058] [INFO ] [MAIN ] BIRK: Generated 1794 derivative records from 51198 primary records. [2025-10-20T22:06:05.054414] [DEBUG] [LOCAL ] BIRK: Saving derivative data. [2025-10-20T22:06:05.077198] [DEBUG] [LOCAL ] BIT: Derivative data saved (1794 records). [2025-10-20T22:06:05.085395] [INFO ] [MAIN ] BIT: Successfully saved derivative data. [2025-10-20T22:06:05.084542] [DEBUG] [LOCAL ] BIRK: Derivative data saved (1794 records). [2025-10-20T22:06:05.089035] [INFO ] [MAIN ] BIRK: Successfully saved derivative data. [2025-10-20T22:06:05.088180] [INFO ] [MAIN ] Derivative Progress: 223/1877 symbols completed. BIT took 1.1s. Estimated remaining: 10m 30s. Rate: 157.6 symbols/min [2025-10-20T22:06:05.092009] [INFO ] [MAIN ] Derivative Progress: 224/1877 symbols completed. BIRK took 1.2s. Estimated remaining: 10m 26s. Rate: 158.3 symbols/min [2025-10-20T22:06:05.104448] [DEBUG] [MAIN ] BJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:05.105034] [INFO ] [MAIN ] BJ: Generated 1794 derivative records from 51581 primary records. [2025-10-20T22:06:05.105094] [DEBUG] [LOCAL ] BJ: Saving derivative data. [2025-10-20T22:06:05.120229] [DEBUG] [LOCAL ] BJ: Derivative data saved (1794 records). [2025-10-20T22:06:05.120285] [INFO ] [MAIN ] BJ: Successfully saved derivative data. [2025-10-20T22:06:05.122838] [INFO ] [MAIN ] Derivative Progress: 225/1877 symbols completed. BJ took 1.1s. Estimated remaining: 10m 24s. Rate: 159.0 symbols/min [2025-10-20T22:06:05.123470] [INFO ] [MAIN ] BKKT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:05.123393] [INFO ] [MAIN ] BK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:05.123577] [INFO ] [MAIN ] BKE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:05.172762] [DEBUG] [LOCAL ] BKE: Loaded 33953 existing records from primary storage. [2025-10-20T22:06:05.173067] [INFO ] [MAIN ] BKE: Loaded 33953 primary data records. [2025-10-20T22:06:05.304811] [DEBUG] [LOCAL ] BK: Loaded 53546 existing records from primary storage. [2025-10-20T22:06:05.363441] [DEBUG] [LOCAL ] BKKT: Loaded 37052 existing records from primary storage. [2025-10-20T22:06:05.386720] [INFO ] [MAIN ] BK: Loaded 53546 primary data records. [2025-10-20T22:06:05.501046] [INFO ] [MAIN ] BKKT: Loaded 37052 primary data records. [2025-10-20T22:06:06.128826] [DEBUG] [MAIN ] BKE: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:06.130405] [INFO ] [MAIN ] BKE: Generated 1793 derivative records from 33953 primary records. [2025-10-20T22:06:06.130709] [DEBUG] [LOCAL ] BKE: Saving derivative data. [2025-10-20T22:06:06.147340] [DEBUG] [LOCAL ] BKE: Derivative data saved (1793 records). [2025-10-20T22:06:06.147473] [INFO ] [MAIN ] BKE: Successfully saved derivative data. [2025-10-20T22:06:06.149371] [INFO ] [MAIN ] Derivative Progress: 226/1877 symbols completed. BKE took 1.0s. Estimated remaining: 10m 28s. Rate: 157.8 symbols/min [2025-10-20T22:06:06.149630] [INFO ] [MAIN ] BKN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:06.178424] [DEBUG] [MAIN ] BK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:06.183372] [DEBUG] [MAIN ] BKKT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:06.184107] [INFO ] [MAIN ] BKKT: Generated 1793 derivative records from 37052 primary records. [2025-10-20T22:06:06.184758] [INFO ] [MAIN ] BK: Generated 1794 derivative records from 53546 primary records. [2025-10-20T22:06:06.184997] [DEBUG] [LOCAL ] BKKT: Saving derivative data. [2025-10-20T22:06:06.188736] [DEBUG] [LOCAL ] BK: Saving derivative data. [2025-10-20T22:06:06.196391] [DEBUG] [LOCAL ] BKN: Loaded 7285 existing records from primary storage. [2025-10-20T22:06:06.209189] [INFO ] [MAIN ] BKN: Loaded 7285 primary data records. [2025-10-20T22:06:06.238737] [DEBUG] [LOCAL ] BKKT: Derivative data saved (1793 records). [2025-10-20T22:06:06.251045] [DEBUG] [LOCAL ] BK: Derivative data saved (1794 records). [2025-10-20T22:06:06.251146] [INFO ] [MAIN ] BK: Successfully saved derivative data. [2025-10-20T22:06:06.251253] [INFO ] [MAIN ] BKKT: Successfully saved derivative data. [2025-10-20T22:06:06.258274] [INFO ] [MAIN ] Derivative Progress: 227/1877 symbols completed. BK took 1.1s. Estimated remaining: 10m 26s. Rate: 158.3 symbols/min [2025-10-20T22:06:06.261086] [INFO ] [MAIN ] BKSY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:06.261321] [INFO ] [MAIN ] Derivative Progress: 228/1877 symbols completed. BKKT took 1.1s. Estimated remaining: 10m 22s. Rate: 158.9 symbols/min [2025-10-20T22:06:06.289460] [INFO ] [MAIN ] BKT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:06.560776] [DEBUG] [LOCAL ] BKT: Loaded 12778 existing records from primary storage. [2025-10-20T22:06:06.561235] [INFO ] [MAIN ] BKT: Loaded 12778 primary data records. [2025-10-20T22:06:06.717535] [DEBUG] [LOCAL ] BKSY: Loaded 48890 existing records from primary storage. [2025-10-20T22:06:06.745068] [INFO ] [MAIN ] BKSY: Loaded 48890 primary data records. [2025-10-20T22:06:07.167978] [DEBUG] [MAIN ] BKN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:07.269743] [INFO ] [MAIN ] BKN: Generated 1794 derivative records from 7285 primary records. [2025-10-20T22:06:07.269993] [DEBUG] [LOCAL ] BKN: Saving derivative data. [2025-10-20T22:06:07.291057] [DEBUG] [LOCAL ] BKN: Derivative data saved (1794 records). [2025-10-20T22:06:07.292891] [INFO ] [MAIN ] BKN: Successfully saved derivative data. [2025-10-20T22:06:07.295943] [INFO ] [MAIN ] Derivative Progress: 229/1877 symbols completed. BKN took 1.1s. Estimated remaining: 10m 27s. Rate: 157.7 symbols/min [2025-10-20T22:06:07.296499] [INFO ] [MAIN ] BKU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:07.317414] [DEBUG] [MAIN ] BKT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:07.318941] [INFO ] [MAIN ] BKT: Generated 1794 derivative records from 12778 primary records. [2025-10-20T22:06:07.319259] [DEBUG] [LOCAL ] BKT: Saving derivative data. [2025-10-20T22:06:07.340118] [DEBUG] [LOCAL ] BKT: Derivative data saved (1794 records). [2025-10-20T22:06:07.342129] [INFO ] [MAIN ] BKT: Successfully saved derivative data. [2025-10-20T22:06:07.345313] [INFO ] [MAIN ] Derivative Progress: 230/1877 symbols completed. BKT took 1.1s. Estimated remaining: 10m 24s. Rate: 158.3 symbols/min [2025-10-20T22:06:07.356004] [INFO ] [MAIN ] BKV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:07.370094] [DEBUG] [MAIN ] BKSY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:07.370749] [INFO ] [MAIN ] BKSY: Generated 1794 derivative records from 48890 primary records. [2025-10-20T22:06:07.370855] [DEBUG] [LOCAL ] BKSY: Saving derivative data. [2025-10-20T22:06:07.387342] [DEBUG] [LOCAL ] BKSY: Derivative data saved (1794 records). [2025-10-20T22:06:07.400311] [INFO ] [MAIN ] BKSY: Successfully saved derivative data. [2025-10-20T22:06:07.402518] [INFO ] [MAIN ] Derivative Progress: 231/1877 symbols completed. BKSY took 1.1s. Estimated remaining: 10m 21s. Rate: 158.9 symbols/min [2025-10-20T22:06:07.402724] [INFO ] [MAIN ] BLCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:07.404303] [DEBUG] [LOCAL ] BKU: Loaded 42263 existing records from primary storage. [2025-10-20T22:06:07.404396] [INFO ] [MAIN ] BKU: Loaded 42263 primary data records. [2025-10-20T22:06:07.524820] [DEBUG] [LOCAL ] BKV: Loaded 29615 existing records from primary storage. [2025-10-20T22:06:07.535784] [INFO ] [MAIN ] BKV: Loaded 29615 primary data records. [2025-10-20T22:06:07.843363] [DEBUG] [LOCAL ] BLCO: Loaded 35097 existing records from primary storage. [2025-10-20T22:06:07.844996] [INFO ] [MAIN ] BLCO: Loaded 35097 primary data records. [2025-10-20T22:06:08.435093] [DEBUG] [MAIN ] BKU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:08.436129] [INFO ] [MAIN ] BKU: Generated 1794 derivative records from 42263 primary records. [2025-10-20T22:06:08.436278] [DEBUG] [LOCAL ] BKU: Saving derivative data. [2025-10-20T22:06:08.454793] [DEBUG] [LOCAL ] BKU: Derivative data saved (1794 records). [2025-10-20T22:06:08.455005] [INFO ] [MAIN ] BKU: Successfully saved derivative data. [2025-10-20T22:06:08.457005] [INFO ] [MAIN ] Derivative Progress: 232/1877 symbols completed. BKU took 1.2s. Estimated remaining: 10m 26s. Rate: 157.7 symbols/min [2025-10-20T22:06:08.457479] [INFO ] [MAIN ] BLD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:08.481976] [DEBUG] [MAIN ] BKV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:08.483522] [INFO ] [MAIN ] BKV: Generated 1794 derivative records from 29615 primary records. [2025-10-20T22:06:08.485282] [DEBUG] [LOCAL ] BKV: Saving derivative data. [2025-10-20T22:06:08.519042] [DEBUG] [MAIN ] BLCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:08.532553] [DEBUG] [LOCAL ] BKV: Derivative data saved (1794 records). [2025-10-20T22:06:08.533581] [INFO ] [MAIN ] BKV: Successfully saved derivative data. [2025-10-20T22:06:08.533425] [INFO ] [MAIN ] BLCO: Generated 1794 derivative records from 35097 primary records. [2025-10-20T22:06:08.534125] [DEBUG] [LOCAL ] BLD: Loaded 27298 existing records from primary storage. [2025-10-20T22:06:08.536131] [INFO ] [MAIN ] Derivative Progress: 233/1877 symbols completed. BKV took 1.2s. Estimated remaining: 10m 23s. Rate: 158.2 symbols/min [2025-10-20T22:06:08.536485] [INFO ] [MAIN ] BLD: Loaded 27298 primary data records. [2025-10-20T22:06:08.536228] [DEBUG] [LOCAL ] BLCO: Saving derivative data. [2025-10-20T22:06:08.537597] [INFO ] [MAIN ] BLDR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:08.608924] [DEBUG] [LOCAL ] BLCO: Derivative data saved (1794 records). [2025-10-20T22:06:08.634395] [INFO ] [MAIN ] BLCO: Successfully saved derivative data. [2025-10-20T22:06:08.636722] [INFO ] [MAIN ] Derivative Progress: 234/1877 symbols completed. BLCO took 1.2s. Estimated remaining: 10m 21s. Rate: 158.7 symbols/min [2025-10-20T22:06:08.658005] [INFO ] [MAIN ] BLE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:08.871612] [DEBUG] [LOCAL ] BLE: Loaded 13804 existing records from primary storage. [2025-10-20T22:06:08.871879] [INFO ] [MAIN ] BLE: Loaded 13804 primary data records. [2025-10-20T22:06:09.054770] [DEBUG] [LOCAL ] BLDR: Loaded 51892 existing records from primary storage. [2025-10-20T22:06:09.064389] [INFO ] [MAIN ] BLDR: Loaded 51892 primary data records. [2025-10-20T22:06:09.522909] [DEBUG] [MAIN ] BLD: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:09.523972] [INFO ] [MAIN ] BLD: Generated 1793 derivative records from 27298 primary records. [2025-10-20T22:06:09.524245] [DEBUG] [LOCAL ] BLD: Saving derivative data. [2025-10-20T22:06:09.541454] [DEBUG] [LOCAL ] BLD: Derivative data saved (1793 records). [2025-10-20T22:06:09.541854] [INFO ] [MAIN ] BLD: Successfully saved derivative data. [2025-10-20T22:06:09.543858] [INFO ] [MAIN ] Derivative Progress: 235/1877 symbols completed. BLD took 1.1s. Estimated remaining: 10m 24s. Rate: 157.8 symbols/min [2025-10-20T22:06:09.544196] [INFO ] [MAIN ] BLND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:09.562906] [DEBUG] [MAIN ] BLE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:09.563616] [INFO ] [MAIN ] BLE: Generated 1794 derivative records from 13804 primary records. [2025-10-20T22:06:09.563937] [DEBUG] [LOCAL ] BLE: Saving derivative data. [2025-10-20T22:06:09.582158] [DEBUG] [LOCAL ] BLE: Derivative data saved (1794 records). [2025-10-20T22:06:09.582426] [INFO ] [MAIN ] BLE: Successfully saved derivative data. [2025-10-20T22:06:09.584370] [INFO ] [MAIN ] Derivative Progress: 236/1877 symbols completed. BLE took 0.9s. Estimated remaining: 10m 22s. Rate: 158.4 symbols/min [2025-10-20T22:06:09.584620] [INFO ] [MAIN ] BLW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:09.597858] [DEBUG] [MAIN ] BLDR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:09.608995] [INFO ] [MAIN ] BLDR: Generated 1794 derivative records from 51892 primary records. [2025-10-20T22:06:09.609105] [DEBUG] [LOCAL ] BLDR: Saving derivative data. [2025-10-20T22:06:09.627822] [DEBUG] [LOCAL ] BLDR: Derivative data saved (1794 records). [2025-10-20T22:06:09.628813] [INFO ] [MAIN ] BLDR: Successfully saved derivative data. [2025-10-20T22:06:09.630940] [INFO ] [MAIN ] Derivative Progress: 237/1877 symbols completed. BLDR took 1.1s. Estimated remaining: 10m 19s. Rate: 159.0 symbols/min [2025-10-20T22:06:09.646020] [INFO ] [MAIN ] BLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:09.652296] [DEBUG] [LOCAL ] BLW: Loaded 13124 existing records from primary storage. [2025-10-20T22:06:09.652847] [INFO ] [MAIN ] BLW: Loaded 13124 primary data records. [2025-10-20T22:06:09.686747] [DEBUG] [LOCAL ] BLND: Loaded 49271 existing records from primary storage. [2025-10-20T22:06:09.710100] [INFO ] [MAIN ] BLND: Loaded 49271 primary data records. [2025-10-20T22:06:09.924846] [DEBUG] [LOCAL ] BLX: Loaded 15575 existing records from primary storage. [2025-10-20T22:06:09.992418] [INFO ] [MAIN ] BLX: Loaded 15575 primary data records. [2025-10-20T22:06:10.691990] [DEBUG] [MAIN ] BLW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:10.705557] [DEBUG] [MAIN ] BLND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:10.706981] [INFO ] [MAIN ] BLND: Generated 1794 derivative records from 49271 primary records. [2025-10-20T22:06:10.707076] [INFO ] [MAIN ] BLW: Generated 1794 derivative records from 13124 primary records. [2025-10-20T22:06:10.707352] [DEBUG] [LOCAL ] BLW: Saving derivative data. [2025-10-20T22:06:10.708086] [DEBUG] [LOCAL ] BLND: Saving derivative data. [2025-10-20T22:06:10.741831] [DEBUG] [LOCAL ] BLND: Derivative data saved (1794 records). [2025-10-20T22:06:10.733479] [DEBUG] [LOCAL ] BLW: Derivative data saved (1794 records). [2025-10-20T22:06:10.742978] [INFO ] [MAIN ] BLND: Successfully saved derivative data. [2025-10-20T22:06:10.742903] [INFO ] [MAIN ] BLW: Successfully saved derivative data. [2025-10-20T22:06:10.746054] [INFO ] [MAIN ] Derivative Progress: 238/1877 symbols completed. BLND took 1.2s. Estimated remaining: 10m 24s. Rate: 157.7 symbols/min [2025-10-20T22:06:10.751979] [INFO ] [MAIN ] Derivative Progress: 239/1877 symbols completed. BLW took 1.2s. Estimated remaining: 10m 21s. Rate: 158.4 symbols/min [2025-10-20T22:06:10.764709] [DEBUG] [MAIN ] BLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:10.765385] [INFO ] [MAIN ] BLX: Generated 1794 derivative records from 15575 primary records. [2025-10-20T22:06:10.765445] [DEBUG] [LOCAL ] BLX: Saving derivative data. [2025-10-20T22:06:10.780312] [DEBUG] [LOCAL ] BLX: Derivative data saved (1794 records). [2025-10-20T22:06:10.780371] [INFO ] [MAIN ] BLX: Successfully saved derivative data. [2025-10-20T22:06:10.782912] [INFO ] [MAIN ] Derivative Progress: 240/1877 symbols completed. BLX took 1.1s. Estimated remaining: 10m 18s. Rate: 159.0 symbols/min [2025-10-20T22:06:10.783483] [INFO ] [MAIN ] BME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:10.783405] [INFO ] [MAIN ] BMA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:10.784315] [INFO ] [MAIN ] BMEZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:10.798664] [DEBUG] [LOCAL ] BME: Loaded 11258 existing records from primary storage. [2025-10-20T22:06:10.798823] [INFO ] [MAIN ] BME: Loaded 11258 primary data records. [2025-10-20T22:06:10.933166] [DEBUG] [LOCAL ] BMEZ: Loaded 25392 existing records from primary storage. [2025-10-20T22:06:11.024082] [INFO ] [MAIN ] BMEZ: Loaded 25392 primary data records. [2025-10-20T22:06:11.025003] [DEBUG] [LOCAL ] BMA: Loaded 36470 existing records from primary storage. [2025-10-20T22:06:11.141935] [INFO ] [MAIN ] BMA: Loaded 36470 primary data records. [2025-10-20T22:06:11.774407] [DEBUG] [MAIN ] BME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:11.775667] [INFO ] [MAIN ] BME: Generated 1794 derivative records from 11258 primary records. [2025-10-20T22:06:11.775783] [DEBUG] [LOCAL ] BME: Saving derivative data. [2025-10-20T22:06:11.793915] [DEBUG] [LOCAL ] BME: Derivative data saved (1794 records). [2025-10-20T22:06:11.794129] [INFO ] [MAIN ] BME: Successfully saved derivative data. [2025-10-20T22:06:11.796368] [INFO ] [MAIN ] Derivative Progress: 241/1877 symbols completed. BME took 1.0s. Estimated remaining: 10m 22s. Rate: 157.9 symbols/min [2025-10-20T22:06:11.796723] [INFO ] [MAIN ] BMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:11.817297] [DEBUG] [MAIN ] BMEZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:11.830444] [INFO ] [MAIN ] BMEZ: Generated 1794 derivative records from 25392 primary records. [2025-10-20T22:06:11.829758] [DEBUG] [MAIN ] BMA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:11.830532] [DEBUG] [LOCAL ] BMEZ: Saving derivative data. [2025-10-20T22:06:11.836956] [INFO ] [MAIN ] BMA: Generated 1794 derivative records from 36470 primary records. [2025-10-20T22:06:11.837620] [DEBUG] [LOCAL ] BMA: Saving derivative data. [2025-10-20T22:06:11.877259] [DEBUG] [LOCAL ] BMEZ: Derivative data saved (1794 records). [2025-10-20T22:06:11.879855] [INFO ] [MAIN ] BMEZ: Successfully saved derivative data. [2025-10-20T22:06:11.881931] [INFO ] [MAIN ] Derivative Progress: 242/1877 symbols completed. BMEZ took 1.1s. Estimated remaining: 10m 19s. Rate: 158.4 symbols/min [2025-10-20T22:06:11.883430] [DEBUG] [LOCAL ] BMA: Derivative data saved (1794 records). [2025-10-20T22:06:11.882360] [DEBUG] [LOCAL ] BMI: Loaded 24793 existing records from primary storage. [2025-10-20T22:06:11.883597] [INFO ] [MAIN ] BMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:11.884263] [INFO ] [MAIN ] BMI: Loaded 24793 primary data records. [2025-10-20T22:06:11.884081] [INFO ] [MAIN ] BMA: Successfully saved derivative data. [2025-10-20T22:06:11.886458] [INFO ] [MAIN ] Derivative Progress: 243/1877 symbols completed. BMA took 1.1s. Estimated remaining: 10m 17s. Rate: 159.0 symbols/min [2025-10-20T22:06:11.886792] [INFO ] [MAIN ] BMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:11.946451] [DEBUG] [LOCAL ] BMN: Loaded 2592 existing records from primary storage. [2025-10-20T22:06:11.960778] [INFO ] [MAIN ] BMN: Loaded 2592 primary data records. [2025-10-20T22:06:12.552227] [DEBUG] [LOCAL ] BMO: Loaded 41223 existing records from primary storage. [2025-10-20T22:06:12.552698] [INFO ] [MAIN ] BMO: Loaded 41223 primary data records. [2025-10-20T22:06:12.887811] [DEBUG] [MAIN ] BMI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:12.890246] [INFO ] [MAIN ] BMI: Generated 1793 derivative records from 24793 primary records. [2025-10-20T22:06:12.893276] [DEBUG] [LOCAL ] BMI: Saving derivative data. [2025-10-20T22:06:12.907808] [DEBUG] [MAIN ] BMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:12.909033] [INFO ] [MAIN ] BMN: Generated 1794 derivative records from 2592 primary records. [2025-10-20T22:06:12.909142] [DEBUG] [LOCAL ] BMN: Saving derivative data. [2025-10-20T22:06:12.926281] [DEBUG] [LOCAL ] BMI: Derivative data saved (1793 records). [2025-10-20T22:06:12.933684] [INFO ] [MAIN ] BMI: Successfully saved derivative data. [2025-10-20T22:06:12.939969] [INFO ] [MAIN ] Derivative Progress: 244/1877 symbols completed. BMI took 1.1s. Estimated remaining: 10m 21s. Rate: 157.9 symbols/min [2025-10-20T22:06:12.940419] [INFO ] [MAIN ] BMY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:12.943691] [DEBUG] [LOCAL ] BMN: Derivative data saved (1794 records). [2025-10-20T22:06:12.943948] [INFO ] [MAIN ] BMN: Successfully saved derivative data. [2025-10-20T22:06:12.946546] [INFO ] [MAIN ] Derivative Progress: 245/1877 symbols completed. BMN took 1.1s. Estimated remaining: 10m 18s. Rate: 158.5 symbols/min [2025-10-20T22:06:12.946836] [INFO ] [MAIN ] BN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:12.967711] [DEBUG] [MAIN ] BMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:12.968636] [INFO ] [MAIN ] BMO: Generated 1794 derivative records from 41223 primary records. [2025-10-20T22:06:12.968946] [DEBUG] [LOCAL ] BMO: Saving derivative data. [2025-10-20T22:06:12.986488] [DEBUG] [LOCAL ] BMO: Derivative data saved (1794 records). [2025-10-20T22:06:12.986700] [INFO ] [MAIN ] BMO: Successfully saved derivative data. [2025-10-20T22:06:12.989186] [INFO ] [MAIN ] Derivative Progress: 246/1877 symbols completed. BMO took 1.1s. Estimated remaining: 10m 15s. Rate: 159.1 symbols/min [2025-10-20T22:06:12.989504] [INFO ] [MAIN ] BNED: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:13.040252] [DEBUG] [LOCAL ] BN: Loaded 52986 existing records from primary storage. [2025-10-20T22:06:13.041225] [INFO ] [MAIN ] BN: Loaded 52986 primary data records. [2025-10-20T22:06:13.183788] [DEBUG] [LOCAL ] BNED: Loaded 28483 existing records from primary storage. [2025-10-20T22:06:13.185003] [DEBUG] [LOCAL ] BMY: Loaded 53757 existing records from primary storage. [2025-10-20T22:06:13.185138] [INFO ] [MAIN ] BNED: Loaded 28483 primary data records. [2025-10-20T22:06:13.195618] [INFO ] [MAIN ] BMY: Loaded 53757 primary data records. [2025-10-20T22:06:14.014894] [DEBUG] [MAIN ] BN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:14.015800] [INFO ] [MAIN ] BN: Generated 1794 derivative records from 52986 primary records. [2025-10-20T22:06:14.016744] [DEBUG] [LOCAL ] BN: Saving derivative data. [2025-10-20T22:06:14.034035] [DEBUG] [LOCAL ] BN: Derivative data saved (1794 records). [2025-10-20T22:06:14.034273] [INFO ] [MAIN ] BN: Successfully saved derivative data. [2025-10-20T22:06:14.047802] [DEBUG] [MAIN ] BNED: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:14.048551] [INFO ] [MAIN ] BNED: Generated 1794 derivative records from 28483 primary records. [2025-10-20T22:06:14.050898] [INFO ] [MAIN ] Derivative Progress: 247/1877 symbols completed. BN took 1.1s. Estimated remaining: 10m 19s. Rate: 157.9 symbols/min [2025-10-20T22:06:14.051047] [DEBUG] [LOCAL ] BNED: Saving derivative data. [2025-10-20T22:06:14.051180] [INFO ] [MAIN ] BNH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:14.077589] [DEBUG] [MAIN ] BMY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:14.078534] [INFO ] [MAIN ] BMY: Generated 1794 derivative records from 53757 primary records. [2025-10-20T22:06:14.082728] [DEBUG] [LOCAL ] BMY: Saving derivative data. [2025-10-20T22:06:14.099504] [DEBUG] [LOCAL ] BNH: Loaded 4731 existing records from primary storage. [2025-10-20T22:06:14.107187] [INFO ] [MAIN ] BNH: Loaded 4731 primary data records. [2025-10-20T22:06:14.131361] [DEBUG] [LOCAL ] BMY: Derivative data saved (1794 records). [2025-10-20T22:06:14.130627] [DEBUG] [LOCAL ] BNED: Derivative data saved (1794 records). [2025-10-20T22:06:14.141811] [INFO ] [MAIN ] BNED: Successfully saved derivative data. [2025-10-20T22:06:14.144116] [INFO ] [MAIN ] Derivative Progress: 248/1877 symbols completed. BNED took 1.2s. Estimated remaining: 10m 17s. Rate: 158.4 symbols/min [2025-10-20T22:06:14.141728] [INFO ] [MAIN ] BMY: Successfully saved derivative data. [2025-10-20T22:06:14.151441] [INFO ] [MAIN ] BNJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:14.158772] [INFO ] [MAIN ] Derivative Progress: 249/1877 symbols completed. BMY took 1.2s. Estimated remaining: 10m 14s. Rate: 159.0 symbols/min [2025-10-20T22:06:14.180728] [INFO ] [MAIN ] BNL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:14.227832] [DEBUG] [LOCAL ] BNJ: Loaded 2531 existing records from primary storage. [2025-10-20T22:06:14.238313] [INFO ] [MAIN ] BNJ: Loaded 2531 primary data records. [2025-10-20T22:06:14.795411] [DEBUG] [MAIN ] BNH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:14.796584] [INFO ] [MAIN ] BNH: Generated 1794 derivative records from 4731 primary records. [2025-10-20T22:06:14.796735] [DEBUG] [LOCAL ] BNH: Saving derivative data. [2025-10-20T22:06:14.794944] [DEBUG] [LOCAL ] BNL: Loaded 47210 existing records from primary storage. [2025-10-20T22:06:14.797611] [INFO ] [MAIN ] BNL: Loaded 47210 primary data records. [2025-10-20T22:06:14.899086] [DEBUG] [LOCAL ] BNH: Derivative data saved (1794 records). [2025-10-20T22:06:14.910855] [INFO ] [MAIN ] BNH: Successfully saved derivative data. [2025-10-20T22:06:14.937098] [DEBUG] [MAIN ] BNJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:14.936108] [INFO ] [MAIN ] Derivative Progress: 250/1877 symbols completed. BNH took 0.9s. Estimated remaining: 10m 17s. Rate: 158.3 symbols/min [2025-10-20T22:06:14.958599] [INFO ] [MAIN ] BNJ: Generated 1794 derivative records from 2531 primary records. [2025-10-20T22:06:14.985304] [DEBUG] [LOCAL ] BNJ: Saving derivative data. [2025-10-20T22:06:14.995114] [INFO ] [MAIN ] BNS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:15.038356] [DEBUG] [LOCAL ] BNJ: Derivative data saved (1794 records). [2025-10-20T22:06:15.038464] [INFO ] [MAIN ] BNJ: Successfully saved derivative data. [2025-10-20T22:06:15.157006] [INFO ] [MAIN ] Derivative Progress: 251/1877 symbols completed. BNJ took 1.0s. Estimated remaining: 10m 15s. Rate: 158.6 symbols/min [2025-10-20T22:06:15.157841] [INFO ] [MAIN ] BNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:15.165503] [DEBUG] [LOCAL ] BNT: Loaded 3553 existing records from primary storage. [2025-10-20T22:06:15.166143] [INFO ] [MAIN ] BNT: Loaded 3553 primary data records. [2025-10-20T22:06:15.493879] [DEBUG] [LOCAL ] BNS: Loaded 51449 existing records from primary storage. [2025-10-20T22:06:15.495437] [INFO ] [MAIN ] BNS: Loaded 51449 primary data records. [2025-10-20T22:06:15.599527] [DEBUG] [MAIN ] BNL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:15.607691] [INFO ] [MAIN ] BNL: Generated 1794 derivative records from 47210 primary records. [2025-10-20T22:06:15.607806] [DEBUG] [LOCAL ] BNL: Saving derivative data. [2025-10-20T22:06:15.653565] [DEBUG] [LOCAL ] BNL: Derivative data saved (1794 records). [2025-10-20T22:06:15.666594] [INFO ] [MAIN ] BNL: Successfully saved derivative data. [2025-10-20T22:06:15.668728] [INFO ] [MAIN ] Derivative Progress: 252/1877 symbols completed. BNL took 1.5s. Estimated remaining: 10m 16s. Rate: 158.4 symbols/min [2025-10-20T22:06:15.677325] [INFO ] [MAIN ] BNY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:15.826892] [DEBUG] [LOCAL ] BNY: Loaded 7267 existing records from primary storage. [2025-10-20T22:06:15.828258] [INFO ] [MAIN ] BNY: Loaded 7267 primary data records. [2025-10-20T22:06:15.917539] [DEBUG] [MAIN ] BNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:15.944527] [INFO ] [MAIN ] BNT: Generated 1794 derivative records from 3553 primary records. [2025-10-20T22:06:15.945403] [DEBUG] [LOCAL ] BNT: Saving derivative data. [2025-10-20T22:06:16.039373] [DEBUG] [LOCAL ] BNT: Derivative data saved (1794 records). [2025-10-20T22:06:16.039570] [INFO ] [MAIN ] BNT: Successfully saved derivative data. [2025-10-20T22:06:16.201099] [INFO ] [MAIN ] Derivative Progress: 253/1877 symbols completed. BNT took 1.0s. Estimated remaining: 10m 16s. Rate: 158.1 symbols/min [2025-10-20T22:06:16.220906] [DEBUG] [MAIN ] BNS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:16.221623] [INFO ] [MAIN ] BNS: Generated 1794 derivative records from 51449 primary records. [2025-10-20T22:06:16.221733] [DEBUG] [LOCAL ] BNS: Saving derivative data. [2025-10-20T22:06:16.238253] [DEBUG] [LOCAL ] BNS: Derivative data saved (1794 records). [2025-10-20T22:06:16.238351] [INFO ] [MAIN ] BNS: Successfully saved derivative data. [2025-10-20T22:06:16.240308] [INFO ] [MAIN ] Derivative Progress: 254/1877 symbols completed. BNS took 1.2s. Estimated remaining: 10m 14s. Rate: 158.7 symbols/min [2025-10-20T22:06:16.257098] [DEBUG] [MAIN ] BNY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:16.257740] [INFO ] [MAIN ] BNY: Generated 1794 derivative records from 7267 primary records. [2025-10-20T22:06:16.257803] [DEBUG] [LOCAL ] BNY: Saving derivative data. [2025-10-20T22:06:16.271867] [DEBUG] [LOCAL ] BNY: Derivative data saved (1794 records). [2025-10-20T22:06:16.271935] [INFO ] [MAIN ] BNY: Successfully saved derivative data. [2025-10-20T22:06:16.273836] [INFO ] [MAIN ] Derivative Progress: 255/1877 symbols completed. BNY took 0.6s. Estimated remaining: 10m 11s. Rate: 159.2 symbols/min [2025-10-20T22:06:16.274416] [INFO ] [MAIN ] BOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:16.274555] [INFO ] [MAIN ] BOE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:16.274841] [INFO ] [MAIN ] BOND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:16.312362] [DEBUG] [LOCAL ] BOE: Loaded 18109 existing records from primary storage. [2025-10-20T22:06:16.312937] [INFO ] [MAIN ] BOE: Loaded 18109 primary data records. [2025-10-20T22:06:16.312850] [DEBUG] [LOCAL ] BOC: Loaded 18418 existing records from primary storage. [2025-10-20T22:06:16.349969] [INFO ] [MAIN ] BOC: Loaded 18418 primary data records. [2025-10-20T22:06:16.445970] [DEBUG] [LOCAL ] BOND: Loaded 32481 existing records from primary storage. [2025-10-20T22:06:16.459937] [INFO ] [MAIN ] BOND: Loaded 32481 primary data records. [2025-10-20T22:06:17.316315] [DEBUG] [MAIN ] BOE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:17.317836] [INFO ] [MAIN ] BOE: Generated 1794 derivative records from 18109 primary records. [2025-10-20T22:06:17.318090] [DEBUG] [LOCAL ] BOE: Saving derivative data. [2025-10-20T22:06:17.339404] [DEBUG] [LOCAL ] BOE: Derivative data saved (1794 records). [2025-10-20T22:06:17.339982] [INFO ] [MAIN ] BOE: Successfully saved derivative data. [2025-10-20T22:06:17.342342] [INFO ] [MAIN ] Derivative Progress: 256/1877 symbols completed. BOE took 1.1s. Estimated remaining: 10m 15s. Rate: 158.1 symbols/min [2025-10-20T22:06:17.348517] [INFO ] [MAIN ] BOOT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:17.365822] [DEBUG] [MAIN ] BOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:17.368310] [INFO ] [MAIN ] BOC: Generated 1794 derivative records from 18418 primary records. [2025-10-20T22:06:17.368613] [DEBUG] [LOCAL ] BOC: Saving derivative data. [2025-10-20T22:06:17.383813] [DEBUG] [MAIN ] BOND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:17.395897] [INFO ] [MAIN ] BOND: Generated 1794 derivative records from 32481 primary records. [2025-10-20T22:06:17.402572] [DEBUG] [LOCAL ] BOND: Saving derivative data. [2025-10-20T22:06:17.438891] [DEBUG] [LOCAL ] BOND: Derivative data saved (1794 records). [2025-10-20T22:06:17.435570] [DEBUG] [LOCAL ] BOC: Derivative data saved (1794 records). [2025-10-20T22:06:17.439412] [INFO ] [MAIN ] BOC: Successfully saved derivative data. [2025-10-20T22:06:17.439293] [INFO ] [MAIN ] BOND: Successfully saved derivative data. [2025-10-20T22:06:17.452598] [INFO ] [MAIN ] Derivative Progress: 257/1877 symbols completed. BOC took 1.2s. Estimated remaining: 10m 13s. Rate: 158.5 symbols/min [2025-10-20T22:06:17.455497] [INFO ] [MAIN ] BORR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:17.455596] [INFO ] [MAIN ] Derivative Progress: 258/1877 symbols completed. BOND took 1.2s. Estimated remaining: 10m 10s. Rate: 159.2 symbols/min [2025-10-20T22:06:17.456611] [INFO ] [MAIN ] BOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:17.475928] [DEBUG] [LOCAL ] BOOT: Loaded 39698 existing records from primary storage. [2025-10-20T22:06:17.476330] [INFO ] [MAIN ] BOOT: Loaded 39698 primary data records. [2025-10-20T22:06:17.600499] [DEBUG] [LOCAL ] BOW: Loaded 22961 existing records from primary storage. [2025-10-20T22:06:17.629161] [INFO ] [MAIN ] BOW: Loaded 22961 primary data records. [2025-10-20T22:06:17.858047] [DEBUG] [LOCAL ] BORR: Loaded 51118 existing records from primary storage. [2025-10-20T22:06:17.893878] [INFO ] [MAIN ] BORR: Loaded 51118 primary data records. [2025-10-20T22:06:18.579280] [DEBUG] [MAIN ] BOOT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:18.581037] [INFO ] [MAIN ] BOOT: Generated 1794 derivative records from 39698 primary records. [2025-10-20T22:06:18.581513] [DEBUG] [LOCAL ] BOOT: Saving derivative data. [2025-10-20T22:06:18.601435] [DEBUG] [LOCAL ] BOOT: Derivative data saved (1794 records). [2025-10-20T22:06:18.601588] [INFO ] [MAIN ] BOOT: Successfully saved derivative data. [2025-10-20T22:06:18.604707] [INFO ] [MAIN ] Derivative Progress: 259/1877 symbols completed. BOOT took 1.3s. Estimated remaining: 10m 15s. Rate: 157.9 symbols/min [2025-10-20T22:06:18.605958] [INFO ] [MAIN ] BOX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:18.627553] [DEBUG] [MAIN ] BOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:18.629512] [INFO ] [MAIN ] BOW: Generated 1794 derivative records from 22961 primary records. [2025-10-20T22:06:18.629764] [DEBUG] [LOCAL ] BOW: Saving derivative data. [2025-10-20T22:06:18.652399] [DEBUG] [MAIN ] BORR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:18.662282] [DEBUG] [LOCAL ] BOW: Derivative data saved (1794 records). [2025-10-20T22:06:18.663826] [INFO ] [MAIN ] BORR: Generated 1794 derivative records from 51118 primary records. [2025-10-20T22:06:18.663960] [INFO ] [MAIN ] BOW: Successfully saved derivative data. [2025-10-20T22:06:18.667043] [DEBUG] [LOCAL ] BORR: Saving derivative data. [2025-10-20T22:06:18.666909] [INFO ] [MAIN ] Derivative Progress: 260/1877 symbols completed. BOW took 1.2s. Estimated remaining: 10m 12s. Rate: 158.4 symbols/min [2025-10-20T22:06:18.668887] [INFO ] [MAIN ] BP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:18.685490] [DEBUG] [LOCAL ] BORR: Derivative data saved (1794 records). [2025-10-20T22:06:18.685667] [INFO ] [MAIN ] BORR: Successfully saved derivative data. [2025-10-20T22:06:18.687973] [INFO ] [MAIN ] Derivative Progress: 261/1877 symbols completed. BORR took 1.2s. Estimated remaining: 10m 10s. Rate: 159.0 symbols/min [2025-10-20T22:06:18.688462] [INFO ] [MAIN ] BR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:18.731291] [DEBUG] [LOCAL ] BOX: Loaded 50187 existing records from primary storage. [2025-10-20T22:06:18.731503] [INFO ] [MAIN ] BOX: Loaded 50187 primary data records. [2025-10-20T22:06:18.838754] [DEBUG] [LOCAL ] BR: Loaded 34741 existing records from primary storage. [2025-10-20T22:06:18.849807] [INFO ] [MAIN ] BR: Loaded 34741 primary data records. [2025-10-20T22:06:19.181793] [DEBUG] [LOCAL ] BP: Loaded 53708 existing records from primary storage. [2025-10-20T22:06:19.199184] [INFO ] [MAIN ] BP: Loaded 53708 primary data records. [2025-10-20T22:06:19.739074] [DEBUG] [MAIN ] BOX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:19.740188] [INFO ] [MAIN ] BOX: Generated 1794 derivative records from 50187 primary records. [2025-10-20T22:06:19.740432] [DEBUG] [LOCAL ] BOX: Saving derivative data. [2025-10-20T22:06:19.759245] [DEBUG] [LOCAL ] BOX: Derivative data saved (1794 records). [2025-10-20T22:06:19.760142] [INFO ] [MAIN ] BOX: Successfully saved derivative data. [2025-10-20T22:06:19.762528] [INFO ] [MAIN ] Derivative Progress: 262/1877 symbols completed. BOX took 1.2s. Estimated remaining: 10m 14s. Rate: 157.9 symbols/min [2025-10-20T22:06:19.762912] [INFO ] [MAIN ] BRBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:19.776803] [DEBUG] [MAIN ] BR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:19.778900] [INFO ] [MAIN ] BR: Generated 1794 derivative records from 34741 primary records. [2025-10-20T22:06:19.779278] [DEBUG] [LOCAL ] BR: Saving derivative data. [2025-10-20T22:06:19.797711] [DEBUG] [LOCAL ] BR: Derivative data saved (1794 records). [2025-10-20T22:06:19.797882] [INFO ] [MAIN ] BR: Successfully saved derivative data. [2025-10-20T22:06:19.800034] [INFO ] [MAIN ] Derivative Progress: 263/1877 symbols completed. BR took 1.1s. Estimated remaining: 10m 11s. Rate: 158.4 symbols/min [2025-10-20T22:06:19.808136] [INFO ] [MAIN ] BRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:19.813817] [DEBUG] [MAIN ] BP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:19.814458] [INFO ] [MAIN ] BP: Generated 1794 derivative records from 53708 primary records. [2025-10-20T22:06:19.814699] [DEBUG] [LOCAL ] BP: Saving derivative data. [2025-10-20T22:06:19.830789] [DEBUG] [LOCAL ] BP: Derivative data saved (1794 records). [2025-10-20T22:06:19.830955] [INFO ] [MAIN ] BP: Successfully saved derivative data. [2025-10-20T22:06:19.833167] [INFO ] [MAIN ] Derivative Progress: 264/1877 symbols completed. BP took 1.2s. Estimated remaining: 10m 9s. Rate: 159.0 symbols/min [2025-10-20T22:06:19.833343] [INFO ] [MAIN ] BRCC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:19.860430] [DEBUG] [LOCAL ] BRC: Loaded 18533 existing records from primary storage. [2025-10-20T22:06:19.860708] [INFO ] [MAIN ] BRC: Loaded 18533 primary data records. [2025-10-20T22:06:20.038871] [DEBUG] [LOCAL ] BRBR: Loaded 51831 existing records from primary storage. [2025-10-20T22:06:20.054161] [INFO ] [MAIN ] BRBR: Loaded 51831 primary data records. [2025-10-20T22:06:20.206634] [DEBUG] [LOCAL ] BRCC: Loaded 40137 existing records from primary storage. [2025-10-20T22:06:20.236260] [INFO ] [MAIN ] BRCC: Loaded 40137 primary data records. [2025-10-20T22:06:20.879009] [DEBUG] [MAIN ] BRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:20.880367] [INFO ] [MAIN ] BRC: Generated 1794 derivative records from 18533 primary records. [2025-10-20T22:06:20.880782] [DEBUG] [LOCAL ] BRC: Saving derivative data. [2025-10-20T22:06:20.898908] [DEBUG] [LOCAL ] BRC: Derivative data saved (1794 records). [2025-10-20T22:06:20.899133] [INFO ] [MAIN ] BRC: Successfully saved derivative data. [2025-10-20T22:06:20.901223] [INFO ] [MAIN ] Derivative Progress: 265/1877 symbols completed. BRC took 1.1s. Estimated remaining: 10m 13s. Rate: 157.9 symbols/min [2025-10-20T22:06:20.901466] [INFO ] [MAIN ] BRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:20.928827] [DEBUG] [MAIN ] BRBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:20.934819] [INFO ] [MAIN ] BRBR: Generated 1794 derivative records from 51831 primary records. [2025-10-20T22:06:20.934920] [DEBUG] [LOCAL ] BRBR: Saving derivative data. [2025-10-20T22:06:20.935221] [DEBUG] [MAIN ] BRCC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:20.936943] [INFO ] [MAIN ] BRCC: Generated 1794 derivative records from 40137 primary records. [2025-10-20T22:06:20.937271] [DEBUG] [LOCAL ] BRCC: Saving derivative data. [2025-10-20T22:06:20.953464] [DEBUG] [LOCAL ] BRBR: Derivative data saved (1794 records). [2025-10-20T22:06:20.962106] [INFO ] [MAIN ] BRBR: Successfully saved derivative data. [2025-10-20T22:06:20.968058] [INFO ] [MAIN ] Derivative Progress: 266/1877 symbols completed. BRBR took 1.2s. Estimated remaining: 10m 10s. Rate: 158.4 symbols/min [2025-10-20T22:06:20.971848] [DEBUG] [LOCAL ] BRCC: Derivative data saved (1794 records). [2025-10-20T22:06:20.972111] [INFO ] [MAIN ] BROS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:20.972205] [INFO ] [MAIN ] BRCC: Successfully saved derivative data. [2025-10-20T22:06:20.974816] [INFO ] [MAIN ] Derivative Progress: 267/1877 symbols completed. BRCC took 1.1s. Estimated remaining: 10m 8s. Rate: 159.0 symbols/min [2025-10-20T22:06:20.975114] [INFO ] [MAIN ] BRSP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:21.022767] [DEBUG] [LOCAL ] BRO: Loaded 52094 existing records from primary storage. [2025-10-20T22:06:21.023048] [INFO ] [MAIN ] BRO: Loaded 52094 primary data records. [2025-10-20T22:06:21.123410] [DEBUG] [LOCAL ] BRSP: Loaded 36685 existing records from primary storage. [2025-10-20T22:06:21.141756] [INFO ] [MAIN ] BRSP: Loaded 36685 primary data records. [2025-10-20T22:06:21.356832] [DEBUG] [LOCAL ] BROS: Loaded 53300 existing records from primary storage. [2025-10-20T22:06:21.362943] [INFO ] [MAIN ] BROS: Loaded 53300 primary data records. [2025-10-20T22:06:22.090074] [DEBUG] [MAIN ] BRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:22.091786] [INFO ] [MAIN ] BRO: Generated 1794 derivative records from 52094 primary records. [2025-10-20T22:06:22.092327] [DEBUG] [LOCAL ] BRO: Saving derivative data. [2025-10-20T22:06:22.121142] [DEBUG] [MAIN ] BRSP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:22.124758] [DEBUG] [LOCAL ] BRO: Derivative data saved (1794 records). [2025-10-20T22:06:22.125796] [INFO ] [MAIN ] BRO: Successfully saved derivative data. [2025-10-20T22:06:22.139623] [INFO ] [MAIN ] Derivative Progress: 268/1877 symbols completed. BRO took 1.2s. Estimated remaining: 10m 12s. Rate: 157.7 symbols/min [2025-10-20T22:06:22.125470] [INFO ] [MAIN ] BRSP: Generated 1794 derivative records from 36685 primary records. [2025-10-20T22:06:22.140627] [DEBUG] [LOCAL ] BRSP: Saving derivative data. [2025-10-20T22:06:22.148543] [DEBUG] [MAIN ] BROS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:22.152830] [INFO ] [MAIN ] BROS: Generated 1794 derivative records from 53300 primary records. [2025-10-20T22:06:22.154103] [DEBUG] [LOCAL ] BROS: Saving derivative data. [2025-10-20T22:06:22.170434] [DEBUG] [LOCAL ] BRSP: Derivative data saved (1794 records). [2025-10-20T22:06:22.172208] [DEBUG] [LOCAL ] BROS: Derivative data saved (1794 records). [2025-10-20T22:06:22.172279] [INFO ] [MAIN ] BROS: Successfully saved derivative data. [2025-10-20T22:06:22.172338] [INFO ] [MAIN ] BRSP: Successfully saved derivative data. [2025-10-20T22:06:22.174419] [INFO ] [MAIN ] Derivative Progress: 269/1877 symbols completed. BROS took 1.2s. Estimated remaining: 10m 10s. Rate: 158.3 symbols/min [2025-10-20T22:06:22.176565] [INFO ] [MAIN ] Derivative Progress: 270/1877 symbols completed. BRSP took 1.2s. Estimated remaining: 10m 7s. Rate: 158.9 symbols/min [2025-10-20T22:06:22.177055] [INFO ] [MAIN ] BRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:22.177134] [INFO ] [MAIN ] BRX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:22.177252] [INFO ] [MAIN ] BRW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:22.186871] [DEBUG] [LOCAL ] BRT: Loaded 5249 existing records from primary storage. [2025-10-20T22:06:22.187104] [INFO ] [MAIN ] BRT: Loaded 5249 primary data records. [2025-10-20T22:06:22.315700] [DEBUG] [LOCAL ] BRW: Loaded 20943 existing records from primary storage. [2025-10-20T22:06:22.378470] [INFO ] [MAIN ] BRW: Loaded 20943 primary data records. [2025-10-20T22:06:22.672176] [DEBUG] [LOCAL ] BRX: Loaded 51836 existing records from primary storage. [2025-10-20T22:06:22.683802] [INFO ] [MAIN ] BRX: Loaded 51836 primary data records. [2025-10-20T22:06:23.183296] [DEBUG] [MAIN ] BRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:23.184822] [INFO ] [MAIN ] BRT: Generated 1794 derivative records from 5249 primary records. [2025-10-20T22:06:23.185006] [DEBUG] [LOCAL ] BRT: Saving derivative data. [2025-10-20T22:06:23.201583] [DEBUG] [LOCAL ] BRT: Derivative data saved (1794 records). [2025-10-20T22:06:23.201839] [INFO ] [MAIN ] BRT: Successfully saved derivative data. [2025-10-20T22:06:23.203928] [INFO ] [MAIN ] Derivative Progress: 271/1877 symbols completed. BRT took 1.0s. Estimated remaining: 10m 10s. Rate: 157.8 symbols/min [2025-10-20T22:06:23.204226] [INFO ] [MAIN ] BSAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:23.221588] [DEBUG] [MAIN ] BRW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:23.222579] [INFO ] [MAIN ] BRW: Generated 1794 derivative records from 20943 primary records. [2025-10-20T22:06:23.222910] [DEBUG] [LOCAL ] BRW: Saving derivative data. [2025-10-20T22:06:23.240776] [DEBUG] [LOCAL ] BRW: Derivative data saved (1794 records). [2025-10-20T22:06:23.241219] [INFO ] [MAIN ] BRW: Successfully saved derivative data. [2025-10-20T22:06:23.243306] [INFO ] [MAIN ] Derivative Progress: 272/1877 symbols completed. BRW took 1.1s. Estimated remaining: 10m 8s. Rate: 158.4 symbols/min [2025-10-20T22:06:23.243530] [INFO ] [MAIN ] BSBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:23.264353] [DEBUG] [MAIN ] BRX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:23.265157] [INFO ] [MAIN ] BRX: Generated 1794 derivative records from 51836 primary records. [2025-10-20T22:06:23.265420] [DEBUG] [LOCAL ] BRX: Saving derivative data. [2025-10-20T22:06:23.292340] [DEBUG] [LOCAL ] BSAC: Loaded 32970 existing records from primary storage. [2025-10-20T22:06:23.293435] [INFO ] [MAIN ] BSAC: Loaded 32970 primary data records. [2025-10-20T22:06:23.293335] [DEBUG] [LOCAL ] BRX: Derivative data saved (1794 records). [2025-10-20T22:06:23.293574] [INFO ] [MAIN ] BRX: Successfully saved derivative data. [2025-10-20T22:06:23.296102] [INFO ] [MAIN ] Derivative Progress: 273/1877 symbols completed. BRX took 1.1s. Estimated remaining: 10m 6s. Rate: 158.9 symbols/min [2025-10-20T22:06:23.296500] [INFO ] [MAIN ] BSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:23.418175] [DEBUG] [LOCAL ] BSL: Loaded 8235 existing records from primary storage. [2025-10-20T22:06:23.439133] [INFO ] [MAIN ] BSL: Loaded 8235 primary data records. [2025-10-20T22:06:23.510882] [DEBUG] [LOCAL ] BSBR: Loaded 35912 existing records from primary storage. [2025-10-20T22:06:23.575984] [INFO ] [MAIN ] BSBR: Loaded 35912 primary data records. [2025-10-20T22:06:24.314877] [DEBUG] [MAIN ] BSAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:24.316087] [INFO ] [MAIN ] BSAC: Generated 1794 derivative records from 32970 primary records. [2025-10-20T22:06:24.316260] [DEBUG] [LOCAL ] BSAC: Saving derivative data. [2025-10-20T22:06:24.332671] [DEBUG] [LOCAL ] BSAC: Derivative data saved (1794 records). [2025-10-20T22:06:24.333145] [INFO ] [MAIN ] BSAC: Successfully saved derivative data. [2025-10-20T22:06:24.335828] [INFO ] [MAIN ] Derivative Progress: 274/1877 symbols completed. BSAC took 1.1s. Estimated remaining: 10m 9s. Rate: 157.9 symbols/min [2025-10-20T22:06:24.336085] [INFO ] [MAIN ] BSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:24.352550] [DEBUG] [MAIN ] BSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:24.353286] [INFO ] [MAIN ] BSL: Generated 1794 derivative records from 8235 primary records. [2025-10-20T22:06:24.353545] [DEBUG] [LOCAL ] BSL: Saving derivative data. [2025-10-20T22:06:24.370240] [DEBUG] [LOCAL ] BSL: Derivative data saved (1794 records). [2025-10-20T22:06:24.370966] [INFO ] [MAIN ] BSL: Successfully saved derivative data. [2025-10-20T22:06:24.372990] [INFO ] [MAIN ] Derivative Progress: 275/1877 symbols completed. BSL took 1.1s. Estimated remaining: 10m 7s. Rate: 158.4 symbols/min [2025-10-20T22:06:24.373682] [INFO ] [MAIN ] BST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:24.394551] [DEBUG] [MAIN ] BSBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:24.395337] [INFO ] [MAIN ] BSBR: Generated 1794 derivative records from 35912 primary records. [2025-10-20T22:06:24.395471] [DEBUG] [LOCAL ] BSBR: Saving derivative data. [2025-10-20T22:06:24.420334] [DEBUG] [LOCAL ] BSBR: Derivative data saved (1794 records). [2025-10-20T22:06:24.421341] [INFO ] [MAIN ] BSBR: Successfully saved derivative data. [2025-10-20T22:06:24.427992] [DEBUG] [LOCAL ] BSM: Loaded 32482 existing records from primary storage. [2025-10-20T22:06:24.427055] [INFO ] [MAIN ] Derivative Progress: 276/1877 symbols completed. BSBR took 1.2s. Estimated remaining: 10m 5s. Rate: 158.9 symbols/min [2025-10-20T22:06:24.428192] [INFO ] [MAIN ] BSM: Loaded 32482 primary data records. [2025-10-20T22:06:24.428332] [INFO ] [MAIN ] BSTZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:24.505289] [DEBUG] [LOCAL ] BST: Loaded 18723 existing records from primary storage. [2025-10-20T22:06:24.543139] [INFO ] [MAIN ] BST: Loaded 18723 primary data records. [2025-10-20T22:06:24.701606] [DEBUG] [LOCAL ] BSTZ: Loaded 24687 existing records from primary storage. [2025-10-20T22:06:24.844467] [INFO ] [MAIN ] BSTZ: Loaded 24687 primary data records. [2025-10-20T22:06:25.350982] [DEBUG] [MAIN ] BSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:25.352381] [INFO ] [MAIN ] BSM: Generated 1794 derivative records from 32482 primary records. [2025-10-20T22:06:25.352475] [DEBUG] [LOCAL ] BSM: Saving derivative data. [2025-10-20T22:06:25.370155] [DEBUG] [LOCAL ] BSM: Derivative data saved (1794 records). [2025-10-20T22:06:25.370306] [INFO ] [MAIN ] BSM: Successfully saved derivative data. [2025-10-20T22:06:25.372475] [INFO ] [MAIN ] Derivative Progress: 277/1877 symbols completed. BSM took 1.0s. Estimated remaining: 10m 8s. Rate: 158.0 symbols/min [2025-10-20T22:06:25.374883] [INFO ] [MAIN ] BSX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:25.389677] [DEBUG] [MAIN ] BST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:25.390497] [INFO ] [MAIN ] BST: Generated 1794 derivative records from 18723 primary records. [2025-10-20T22:06:25.390809] [DEBUG] [LOCAL ] BST: Saving derivative data. [2025-10-20T22:06:25.407192] [DEBUG] [LOCAL ] BST: Derivative data saved (1794 records). [2025-10-20T22:06:25.407349] [INFO ] [MAIN ] BST: Successfully saved derivative data. [2025-10-20T22:06:25.410231] [INFO ] [MAIN ] Derivative Progress: 278/1877 symbols completed. BST took 1.0s. Estimated remaining: 10m 5s. Rate: 158.5 symbols/min [2025-10-20T22:06:25.411410] [INFO ] [MAIN ] BTA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:25.425779] [DEBUG] [MAIN ] BSTZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:25.426405] [INFO ] [MAIN ] BSTZ: Generated 1794 derivative records from 24687 primary records. [2025-10-20T22:06:25.426563] [DEBUG] [LOCAL ] BSTZ: Saving derivative data. [2025-10-20T22:06:25.447392] [DEBUG] [LOCAL ] BSTZ: Derivative data saved (1794 records). [2025-10-20T22:06:25.447570] [INFO ] [MAIN ] BSTZ: Successfully saved derivative data. [2025-10-20T22:06:25.449575] [INFO ] [MAIN ] Derivative Progress: 279/1877 symbols completed. BSTZ took 1.0s. Estimated remaining: 10m 3s. Rate: 159.0 symbols/min [2025-10-20T22:06:25.450043] [INFO ] [MAIN ] BTE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:25.449827] [DEBUG] [LOCAL ] BTA: Loaded 5756 existing records from primary storage. [2025-10-20T22:06:25.450599] [INFO ] [MAIN ] BTA: Loaded 5756 primary data records. [2025-10-20T22:06:25.654023] [DEBUG] [LOCAL ] BSX: Loaded 53688 existing records from primary storage. [2025-10-20T22:06:25.676769] [INFO ] [MAIN ] BSX: Loaded 53688 primary data records. [2025-10-20T22:06:25.942862] [DEBUG] [LOCAL ] BTE: Loaded 52728 existing records from primary storage. [2025-10-20T22:06:25.954262] [INFO ] [MAIN ] BTE: Loaded 52728 primary data records. [2025-10-20T22:06:26.521672] [DEBUG] [MAIN ] BTA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:26.523076] [INFO ] [MAIN ] BTA: Generated 1794 derivative records from 5756 primary records. [2025-10-20T22:06:26.524094] [DEBUG] [LOCAL ] BTA: Saving derivative data. [2025-10-20T22:06:26.542212] [DEBUG] [LOCAL ] BTA: Derivative data saved (1794 records). [2025-10-20T22:06:26.542893] [INFO ] [MAIN ] BTA: Successfully saved derivative data. [2025-10-20T22:06:26.545413] [INFO ] [MAIN ] Derivative Progress: 280/1877 symbols completed. BTA took 1.1s. Estimated remaining: 10m 7s. Rate: 158.0 symbols/min [2025-10-20T22:06:26.545957] [INFO ] [MAIN ] BTI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:26.566571] [DEBUG] [MAIN ] BSX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:26.567531] [INFO ] [MAIN ] BSX: Generated 1794 derivative records from 53688 primary records. [2025-10-20T22:06:26.568079] [DEBUG] [LOCAL ] BSX: Saving derivative data. [2025-10-20T22:06:26.585983] [DEBUG] [LOCAL ] BSX: Derivative data saved (1794 records). [2025-10-20T22:06:26.586250] [INFO ] [MAIN ] BSX: Successfully saved derivative data. [2025-10-20T22:06:26.588174] [INFO ] [MAIN ] Derivative Progress: 281/1877 symbols completed. BSX took 1.2s. Estimated remaining: 10m 4s. Rate: 158.5 symbols/min [2025-10-20T22:06:26.588326] [INFO ] [MAIN ] BTO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:26.606792] [DEBUG] [MAIN ] BTE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:26.607805] [INFO ] [MAIN ] BTE: Generated 1794 derivative records from 52728 primary records. [2025-10-20T22:06:26.610528] [DEBUG] [LOCAL ] BTE: Saving derivative data. [2025-10-20T22:06:26.637369] [DEBUG] [LOCAL ] BTO: Loaded 8905 existing records from primary storage. [2025-10-20T22:06:26.642243] [INFO ] [MAIN ] BTO: Loaded 8905 primary data records. [2025-10-20T22:06:26.769703] [DEBUG] [LOCAL ] BTE: Derivative data saved (1794 records). [2025-10-20T22:06:26.768800] [DEBUG] [LOCAL ] BTI: Loaded 53594 existing records from primary storage. [2025-10-20T22:06:26.785377] [INFO ] [MAIN ] BTE: Successfully saved derivative data. [2025-10-20T22:06:26.785463] [INFO ] [MAIN ] BTI: Loaded 53594 primary data records. [2025-10-20T22:06:26.819609] [INFO ] [MAIN ] Derivative Progress: 282/1877 symbols completed. BTE took 1.4s. Estimated remaining: 10m 3s. Rate: 158.7 symbols/min [2025-10-20T22:06:26.880831] [INFO ] [MAIN ] BTT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:27.271895] [DEBUG] [LOCAL ] BTT: Loaded 15488 existing records from primary storage. [2025-10-20T22:06:27.280524] [INFO ] [MAIN ] BTT: Loaded 15488 primary data records. [2025-10-20T22:06:27.596941] [DEBUG] [MAIN ] BTO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:27.598265] [INFO ] [MAIN ] BTO: Generated 1794 derivative records from 8905 primary records. [2025-10-20T22:06:27.598556] [DEBUG] [LOCAL ] BTO: Saving derivative data. [2025-10-20T22:06:27.616911] [DEBUG] [LOCAL ] BTO: Derivative data saved (1794 records). [2025-10-20T22:06:27.617050] [INFO ] [MAIN ] BTO: Successfully saved derivative data. [2025-10-20T22:06:27.619042] [INFO ] [MAIN ] Derivative Progress: 283/1877 symbols completed. BTO took 1.0s. Estimated remaining: 10m 5s. Rate: 158.1 symbols/min [2025-10-20T22:06:27.639040] [DEBUG] [MAIN ] BTI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:27.640620] [INFO ] [MAIN ] BTI: Generated 1794 derivative records from 53594 primary records. [2025-10-20T22:06:27.640851] [DEBUG] [LOCAL ] BTI: Saving derivative data. [2025-10-20T22:06:27.657956] [DEBUG] [LOCAL ] BTI: Derivative data saved (1794 records). [2025-10-20T22:06:27.658094] [INFO ] [MAIN ] BTI: Successfully saved derivative data. [2025-10-20T22:06:27.661749] [INFO ] [MAIN ] Derivative Progress: 284/1877 symbols completed. BTI took 1.1s. Estimated remaining: 10m 3s. Rate: 158.6 symbols/min [2025-10-20T22:06:27.677367] [DEBUG] [MAIN ] BTT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:27.678033] [INFO ] [MAIN ] BTT: Generated 1794 derivative records from 15488 primary records. [2025-10-20T22:06:27.678093] [DEBUG] [LOCAL ] BTT: Saving derivative data. [2025-10-20T22:06:27.692754] [DEBUG] [LOCAL ] BTT: Derivative data saved (1794 records). [2025-10-20T22:06:27.692810] [INFO ] [MAIN ] BTT: Successfully saved derivative data. [2025-10-20T22:06:27.695132] [INFO ] [MAIN ] Derivative Progress: 285/1877 symbols completed. BTT took 0.8s. Estimated remaining: 10m 0s. Rate: 159.1 symbols/min [2025-10-20T22:06:27.695918] [INFO ] [MAIN ] BTZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:27.696077] [INFO ] [MAIN ] BUD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:27.695842] [INFO ] [MAIN ] BTU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:27.719965] [DEBUG] [LOCAL ] BTZ: Loaded 22556 existing records from primary storage. [2025-10-20T22:06:27.720095] [INFO ] [MAIN ] BTZ: Loaded 22556 primary data records. [2025-10-20T22:06:27.935302] [DEBUG] [LOCAL ] BUD: Loaded 52605 existing records from primary storage. [2025-10-20T22:06:27.952346] [INFO ] [MAIN ] BUD: Loaded 52605 primary data records. [2025-10-20T22:06:27.952244] [DEBUG] [LOCAL ] BTU: Loaded 53336 existing records from primary storage. [2025-10-20T22:06:28.121317] [INFO ] [MAIN ] BTU: Loaded 53336 primary data records. [2025-10-20T22:06:28.754871] [DEBUG] [MAIN ] BTZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:28.757400] [INFO ] [MAIN ] BTZ: Generated 1794 derivative records from 22556 primary records. [2025-10-20T22:06:28.758311] [DEBUG] [LOCAL ] BTZ: Saving derivative data. [2025-10-20T22:06:28.776441] [DEBUG] [LOCAL ] BTZ: Derivative data saved (1794 records). [2025-10-20T22:06:28.776583] [INFO ] [MAIN ] BTZ: Successfully saved derivative data. [2025-10-20T22:06:28.778794] [INFO ] [MAIN ] Derivative Progress: 286/1877 symbols completed. BTZ took 1.1s. Estimated remaining: 10m 4s. Rate: 158.0 symbols/min [2025-10-20T22:06:28.779455] [INFO ] [MAIN ] BUI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:28.802242] [DEBUG] [MAIN ] BUD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:28.803172] [INFO ] [MAIN ] BUD: Generated 1794 derivative records from 52605 primary records. [2025-10-20T22:06:28.804085] [DEBUG] [LOCAL ] BUD: Saving derivative data. [2025-10-20T22:06:28.811152] [DEBUG] [LOCAL ] BUI: Loaded 11949 existing records from primary storage. [2025-10-20T22:06:28.811340] [INFO ] [MAIN ] BUI: Loaded 11949 primary data records. [2025-10-20T22:06:28.871396] [DEBUG] [MAIN ] BTU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:28.881462] [INFO ] [MAIN ] BTU: Generated 1794 derivative records from 53336 primary records. [2025-10-20T22:06:28.893684] [DEBUG] [LOCAL ] BTU: Saving derivative data. [2025-10-20T22:06:28.903632] [DEBUG] [LOCAL ] BUD: Derivative data saved (1794 records). [2025-10-20T22:06:28.903755] [INFO ] [MAIN ] BUD: Successfully saved derivative data. [2025-10-20T22:06:28.926200] [INFO ] [MAIN ] Derivative Progress: 287/1877 symbols completed. BUD took 1.2s. Estimated remaining: 10m 2s. Rate: 158.4 symbols/min [2025-10-20T22:06:28.934063] [DEBUG] [LOCAL ] BTU: Derivative data saved (1794 records). [2025-10-20T22:06:28.934217] [INFO ] [MAIN ] BUR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:28.939745] [INFO ] [MAIN ] BTU: Successfully saved derivative data. [2025-10-20T22:06:28.947285] [INFO ] [MAIN ] Derivative Progress: 288/1877 symbols completed. BTU took 1.3s. Estimated remaining: 10m 0s. Rate: 158.9 symbols/min [2025-10-20T22:06:28.961764] [INFO ] [MAIN ] BURL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:29.231997] [DEBUG] [LOCAL ] BURL: Loaded 45583 existing records from primary storage. [2025-10-20T22:06:29.233440] [INFO ] [MAIN ] BURL: Loaded 45583 primary data records. [2025-10-20T22:06:29.233236] [DEBUG] [LOCAL ] BUR: Loaded 45494 existing records from primary storage. [2025-10-20T22:06:29.267989] [DEBUG] [MAIN ] BUI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:29.285238] [INFO ] [MAIN ] BUR: Loaded 45494 primary data records. [2025-10-20T22:06:29.306535] [INFO ] [MAIN ] BUI: Generated 1794 derivative records from 11949 primary records. [2025-10-20T22:06:29.315714] [DEBUG] [LOCAL ] BUI: Saving derivative data. [2025-10-20T22:06:29.499287] [DEBUG] [LOCAL ] BUI: Derivative data saved (1794 records). [2025-10-20T22:06:29.526137] [INFO ] [MAIN ] BUI: Successfully saved derivative data. [2025-10-20T22:06:29.538924] [INFO ] [MAIN ] Derivative Progress: 289/1877 symbols completed. BUI took 0.8s. Estimated remaining: 10m 1s. Rate: 158.6 symbols/min [2025-10-20T22:06:29.573705] [INFO ] [MAIN ] BUXX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:29.898808] [DEBUG] [LOCAL ] BUXX: Loaded 15466 existing records from primary storage. [2025-10-20T22:06:29.899812] [INFO ] [MAIN ] BUXX: Loaded 15466 primary data records. [2025-10-20T22:06:30.104788] [DEBUG] [MAIN ] BURL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:30.108410] [INFO ] [MAIN ] BURL: Generated 1794 derivative records from 45583 primary records. [2025-10-20T22:06:30.248976] [DEBUG] [LOCAL ] BURL: Saving derivative data. [2025-10-20T22:06:30.293475] [DEBUG] [LOCAL ] BURL: Derivative data saved (1794 records). [2025-10-20T22:06:30.301883] [INFO ] [MAIN ] BURL: Successfully saved derivative data. [2025-10-20T22:06:30.308803] [INFO ] [MAIN ] Derivative Progress: 290/1877 symbols completed. BURL took 1.3s. Estimated remaining: 10m 3s. Rate: 158.0 symbols/min [2025-10-20T22:06:30.309481] [DEBUG] [MAIN ] BUR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:30.309598] [INFO ] [MAIN ] BV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:30.310674] [INFO ] [MAIN ] BUR: Generated 1794 derivative records from 45494 primary records. [2025-10-20T22:06:30.311569] [DEBUG] [LOCAL ] BUR: Saving derivative data. [2025-10-20T22:06:30.329251] [DEBUG] [LOCAL ] BUR: Derivative data saved (1794 records). [2025-10-20T22:06:30.329355] [INFO ] [MAIN ] BUR: Successfully saved derivative data. [2025-10-20T22:06:30.331264] [INFO ] [MAIN ] Derivative Progress: 291/1877 symbols completed. BUR took 1.4s. Estimated remaining: 10m 0s. Rate: 158.5 symbols/min [2025-10-20T22:06:30.331623] [INFO ] [MAIN ] BVN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:30.359865] [DEBUG] [MAIN ] BUXX: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:30.360696] [INFO ] [MAIN ] BUXX: Generated 1793 derivative records from 15466 primary records. [2025-10-20T22:06:30.360868] [DEBUG] [LOCAL ] BUXX: Saving derivative data. [2025-10-20T22:06:30.376607] [DEBUG] [LOCAL ] BUXX: Derivative data saved (1793 records). [2025-10-20T22:06:30.397389] [INFO ] [MAIN ] BUXX: Successfully saved derivative data. [2025-10-20T22:06:30.399565] [INFO ] [MAIN ] Derivative Progress: 292/1877 symbols completed. BUXX took 0.8s. Estimated remaining: 9m 58s. Rate: 159.0 symbols/min [2025-10-20T22:06:30.399817] [INFO ] [MAIN ] BW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:30.401295] [DEBUG] [LOCAL ] BV: Loaded 38556 existing records from primary storage. [2025-10-20T22:06:30.401521] [INFO ] [MAIN ] BV: Loaded 38556 primary data records. [2025-10-20T22:06:30.520010] [DEBUG] [LOCAL ] BVN: Loaded 44477 existing records from primary storage. [2025-10-20T22:06:30.524260] [INFO ] [MAIN ] BVN: Loaded 44477 primary data records. [2025-10-20T22:06:30.850262] [DEBUG] [LOCAL ] BW: Loaded 40452 existing records from primary storage. [2025-10-20T22:06:30.856292] [INFO ] [MAIN ] BW: Loaded 40452 primary data records. [2025-10-20T22:06:31.362782] [DEBUG] [MAIN ] BV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:31.363736] [INFO ] [MAIN ] BV: Generated 1794 derivative records from 38556 primary records. [2025-10-20T22:06:31.363980] [DEBUG] [LOCAL ] BV: Saving derivative data. [2025-10-20T22:06:31.383151] [DEBUG] [LOCAL ] BV: Derivative data saved (1794 records). [2025-10-20T22:06:31.383308] [INFO ] [MAIN ] BV: Successfully saved derivative data. [2025-10-20T22:06:31.385888] [INFO ] [MAIN ] Derivative Progress: 293/1877 symbols completed. BV took 1.1s. Estimated remaining: 10m 1s. Rate: 158.1 symbols/min [2025-10-20T22:06:31.386581] [INFO ] [MAIN ] BWA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:31.401081] [DEBUG] [MAIN ] BVN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:31.402035] [INFO ] [MAIN ] BVN: Generated 1794 derivative records from 44477 primary records. [2025-10-20T22:06:31.416290] [DEBUG] [MAIN ] BW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:31.416378] [DEBUG] [LOCAL ] BVN: Saving derivative data. [2025-10-20T22:06:31.417836] [INFO ] [MAIN ] BW: Generated 1794 derivative records from 40452 primary records. [2025-10-20T22:06:31.418150] [DEBUG] [LOCAL ] BW: Saving derivative data. [2025-10-20T22:06:31.447769] [DEBUG] [LOCAL ] BVN: Derivative data saved (1794 records). [2025-10-20T22:06:31.447976] [INFO ] [MAIN ] BVN: Successfully saved derivative data. [2025-10-20T22:06:31.452315] [INFO ] [MAIN ] Derivative Progress: 294/1877 symbols completed. BVN took 1.1s. Estimated remaining: 9m 59s. Rate: 158.6 symbols/min [2025-10-20T22:06:31.450392] [DEBUG] [LOCAL ] BW: Derivative data saved (1794 records). [2025-10-20T22:06:31.452622] [INFO ] [MAIN ] BWG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:31.452980] [INFO ] [MAIN ] BW: Successfully saved derivative data. [2025-10-20T22:06:31.455316] [INFO ] [MAIN ] Derivative Progress: 295/1877 symbols completed. BW took 1.1s. Estimated remaining: 9m 57s. Rate: 159.1 symbols/min [2025-10-20T22:06:31.455556] [INFO ] [MAIN ] BWLP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:31.481217] [DEBUG] [LOCAL ] BWG: Loaded 12624 existing records from primary storage. [2025-10-20T22:06:31.481396] [INFO ] [MAIN ] BWG: Loaded 12624 primary data records. [2025-10-20T22:06:31.609403] [DEBUG] [LOCAL ] BWA: Loaded 52108 existing records from primary storage. [2025-10-20T22:06:31.625838] [INFO ] [MAIN ] BWA: Loaded 52108 primary data records. [2025-10-20T22:06:31.671003] [DEBUG] [LOCAL ] BWLP: Loaded 32227 existing records from primary storage. [2025-10-20T22:06:31.711687] [INFO ] [MAIN ] BWLP: Loaded 32227 primary data records. [2025-10-20T22:06:32.463837] [DEBUG] [MAIN ] BWG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:32.465430] [INFO ] [MAIN ] BWG: Generated 1794 derivative records from 12624 primary records. [2025-10-20T22:06:32.465529] [DEBUG] [LOCAL ] BWG: Saving derivative data. [2025-10-20T22:06:32.482947] [DEBUG] [LOCAL ] BWG: Derivative data saved (1794 records). [2025-10-20T22:06:32.495134] [INFO ] [MAIN ] BWG: Successfully saved derivative data. [2025-10-20T22:06:32.497673] [INFO ] [MAIN ] Derivative Progress: 296/1877 symbols completed. BWG took 1.0s. Estimated remaining: 9m 60s. Rate: 158.1 symbols/min [2025-10-20T22:06:32.498974] [DEBUG] [MAIN ] BWA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:32.499102] [INFO ] [MAIN ] BWMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:32.500527] [INFO ] [MAIN ] BWA: Generated 1794 derivative records from 52108 primary records. [2025-10-20T22:06:32.500957] [DEBUG] [LOCAL ] BWA: Saving derivative data. [2025-10-20T22:06:32.532699] [DEBUG] [LOCAL ] BWA: Derivative data saved (1794 records). [2025-10-20T22:06:32.532602] [DEBUG] [MAIN ] BWLP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:32.533546] [INFO ] [MAIN ] BWLP: Generated 1794 derivative records from 32227 primary records. [2025-10-20T22:06:32.532884] [INFO ] [MAIN ] BWA: Successfully saved derivative data. [2025-10-20T22:06:32.533693] [DEBUG] [LOCAL ] BWLP: Saving derivative data. [2025-10-20T22:06:32.535730] [INFO ] [MAIN ] Derivative Progress: 297/1877 symbols completed. BWA took 1.1s. Estimated remaining: 9m 58s. Rate: 158.6 symbols/min [2025-10-20T22:06:32.540505] [DEBUG] [LOCAL ] BWMX: Loaded 7887 existing records from primary storage. [2025-10-20T22:06:32.540703] [INFO ] [MAIN ] BWMX: Loaded 7887 primary data records. [2025-10-20T22:06:32.540224] [INFO ] [MAIN ] BWNB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:32.598604] [DEBUG] [LOCAL ] BWLP: Derivative data saved (1794 records). [2025-10-20T22:06:32.610530] [INFO ] [MAIN ] BWLP: Successfully saved derivative data. [2025-10-20T22:06:32.633306] [INFO ] [MAIN ] Derivative Progress: 298/1877 symbols completed. BWLP took 1.2s. Estimated remaining: 9m 56s. Rate: 159.0 symbols/min [2025-10-20T22:06:32.681511] [DEBUG] [LOCAL ] BWNB: Loaded 4101 existing records from primary storage. [2025-10-20T22:06:32.692007] [INFO ] [MAIN ] BWNB: Loaded 4101 primary data records. [2025-10-20T22:06:33.183370] [DEBUG] [MAIN ] BWMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:33.184126] [INFO ] [MAIN ] BWMX: Generated 1794 derivative records from 7887 primary records. [2025-10-20T22:06:33.184271] [DEBUG] [LOCAL ] BWMX: Saving derivative data. [2025-10-20T22:06:33.201619] [DEBUG] [LOCAL ] BWMX: Derivative data saved (1794 records). [2025-10-20T22:06:33.201781] [INFO ] [MAIN ] BWMX: Successfully saved derivative data. [2025-10-20T22:06:33.203791] [INFO ] [MAIN ] Derivative Progress: 299/1877 symbols completed. BWMX took 0.7s. Estimated remaining: 9m 56s. Rate: 158.7 symbols/min [2025-10-20T22:06:33.216384] [DEBUG] [MAIN ] BWNB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:33.216977] [INFO ] [MAIN ] BWNB: Generated 1794 derivative records from 4101 primary records. [2025-10-20T22:06:33.217037] [DEBUG] [LOCAL ] BWNB: Saving derivative data. [2025-10-20T22:06:33.230469] [DEBUG] [LOCAL ] BWNB: Derivative data saved (1794 records). [2025-10-20T22:06:33.230534] [INFO ] [MAIN ] BWNB: Successfully saved derivative data. [2025-10-20T22:06:33.233021] [INFO ] [MAIN ] Derivative Progress: 300/1877 symbols completed. BWNB took 0.7s. Estimated remaining: 9m 54s. Rate: 159.2 symbols/min [2025-10-20T22:06:33.233545] [INFO ] [MAIN ] BX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:33.233498] [INFO ] [MAIN ] BWSN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:33.233933] [INFO ] [MAIN ] BWXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:33.242309] [DEBUG] [LOCAL ] BWSN: Loaded 5120 existing records from primary storage. [2025-10-20T22:06:33.242549] [INFO ] [MAIN ] BWSN: Loaded 5120 primary data records. [2025-10-20T22:06:33.590529] [DEBUG] [LOCAL ] BWXT: Loaded 47285 existing records from primary storage. [2025-10-20T22:06:33.591194] [INFO ] [MAIN ] BWXT: Loaded 47285 primary data records. [2025-10-20T22:06:33.701118] [DEBUG] [LOCAL ] BX: Loaded 53483 existing records from primary storage. [2025-10-20T22:06:33.709721] [INFO ] [MAIN ] BX: Loaded 53483 primary data records. [2025-10-20T22:06:34.259529] [DEBUG] [MAIN ] BWSN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:34.260684] [INFO ] [MAIN ] BWSN: Generated 1794 derivative records from 5120 primary records. [2025-10-20T22:06:34.260814] [DEBUG] [LOCAL ] BWSN: Saving derivative data. [2025-10-20T22:06:34.278821] [DEBUG] [LOCAL ] BWSN: Derivative data saved (1794 records). [2025-10-20T22:06:34.279896] [INFO ] [MAIN ] BWSN: Successfully saved derivative data. [2025-10-20T22:06:34.282689] [INFO ] [MAIN ] Derivative Progress: 301/1877 symbols completed. BWSN took 1.0s. Estimated remaining: 9m 57s. Rate: 158.3 symbols/min [2025-10-20T22:06:34.283039] [INFO ] [MAIN ] BXC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:34.308867] [DEBUG] [MAIN ] BWXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:34.309686] [INFO ] [MAIN ] BWXT: Generated 1794 derivative records from 47285 primary records. [2025-10-20T22:06:34.315722] [DEBUG] [LOCAL ] BWXT: Saving derivative data. [2025-10-20T22:06:34.315614] [DEBUG] [LOCAL ] BXC: Loaded 13571 existing records from primary storage. [2025-10-20T22:06:34.316930] [INFO ] [MAIN ] BXC: Loaded 13571 primary data records. [2025-10-20T22:06:34.345300] [DEBUG] [LOCAL ] BWXT: Derivative data saved (1794 records). [2025-10-20T22:06:34.345520] [INFO ] [MAIN ] BWXT: Successfully saved derivative data. [2025-10-20T22:06:34.357730] [INFO ] [MAIN ] Derivative Progress: 302/1877 symbols completed. BWXT took 1.1s. Estimated remaining: 9m 55s. Rate: 158.7 symbols/min [2025-10-20T22:06:34.374087] [INFO ] [MAIN ] BXMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:34.418247] [DEBUG] [MAIN ] BX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:34.436663] [INFO ] [MAIN ] BX: Generated 1794 derivative records from 53483 primary records. [2025-10-20T22:06:34.459269] [DEBUG] [LOCAL ] BX: Saving derivative data. [2025-10-20T22:06:34.552122] [DEBUG] [LOCAL ] BX: Derivative data saved (1794 records). [2025-10-20T22:06:34.555470] [INFO ] [MAIN ] BX: Successfully saved derivative data. [2025-10-20T22:06:34.672934] [INFO ] [MAIN ] Derivative Progress: 303/1877 symbols completed. BX took 1.4s. Estimated remaining: 9m 55s. Rate: 158.8 symbols/min [2025-10-20T22:06:34.673844] [INFO ] [MAIN ] BXMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:34.711059] [DEBUG] [LOCAL ] BXMT: Loaded 47292 existing records from primary storage. [2025-10-20T22:06:34.711481] [INFO ] [MAIN ] BXMT: Loaded 47292 primary data records. [2025-10-20T22:06:34.763610] [DEBUG] [LOCAL ] BXMX: Loaded 19355 existing records from primary storage. [2025-10-20T22:06:34.763942] [INFO ] [MAIN ] BXMX: Loaded 19355 primary data records. [2025-10-20T22:06:35.208065] [DEBUG] [MAIN ] BXC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:35.208794] [INFO ] [MAIN ] BXC: Generated 1794 derivative records from 13571 primary records. [2025-10-20T22:06:35.273712] [DEBUG] [LOCAL ] BXC: Saving derivative data. [2025-10-20T22:06:35.356603] [DEBUG] [LOCAL ] BXC: Derivative data saved (1794 records). [2025-10-20T22:06:35.357569] [INFO ] [MAIN ] BXC: Successfully saved derivative data. [2025-10-20T22:06:35.360631] [INFO ] [MAIN ] Derivative Progress: 304/1877 symbols completed. BXC took 1.1s. Estimated remaining: 9m 56s. Rate: 158.4 symbols/min [2025-10-20T22:06:35.361470] [INFO ] [MAIN ] BXSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:35.402525] [DEBUG] [MAIN ] BXMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:35.403841] [INFO ] [MAIN ] BXMT: Generated 1794 derivative records from 47292 primary records. [2025-10-20T22:06:35.403994] [DEBUG] [LOCAL ] BXMT: Saving derivative data. [2025-10-20T22:06:35.404207] [DEBUG] [MAIN ] BXMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:35.405570] [INFO ] [MAIN ] BXMX: Generated 1794 derivative records from 19355 primary records. [2025-10-20T22:06:35.405946] [DEBUG] [LOCAL ] BXMX: Saving derivative data. [2025-10-20T22:06:35.447005] [DEBUG] [LOCAL ] BXMX: Derivative data saved (1794 records). [2025-10-20T22:06:35.447634] [DEBUG] [LOCAL ] BXMT: Derivative data saved (1794 records). [2025-10-20T22:06:35.447732] [INFO ] [MAIN ] BXMX: Successfully saved derivative data. [2025-10-20T22:06:35.448562] [INFO ] [MAIN ] BXMT: Successfully saved derivative data. [2025-10-20T22:06:35.463451] [INFO ] [MAIN ] Derivative Progress: 305/1877 symbols completed. BXMX took 0.8s. Estimated remaining: 9m 54s. Rate: 158.8 symbols/min [2025-10-20T22:06:35.467010] [INFO ] [MAIN ] BY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:35.467204] [INFO ] [MAIN ] Derivative Progress: 306/1877 symbols completed. BXMT took 1.1s. Estimated remaining: 9m 52s. Rate: 159.3 symbols/min [2025-10-20T22:06:35.467699] [INFO ] [MAIN ] BYM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:35.468964] [DEBUG] [LOCAL ] BXSL: Loaded 48208 existing records from primary storage. [2025-10-20T22:06:35.469409] [INFO ] [MAIN ] BXSL: Loaded 48208 primary data records. [2025-10-20T22:06:35.587614] [DEBUG] [LOCAL ] BYM: Loaded 11543 existing records from primary storage. [2025-10-20T22:06:35.621821] [DEBUG] [LOCAL ] BY: Loaded 21868 existing records from primary storage. [2025-10-20T22:06:35.634709] [INFO ] [MAIN ] BY: Loaded 21868 primary data records. [2025-10-20T22:06:35.621412] [INFO ] [MAIN ] BYM: Loaded 11543 primary data records. [2025-10-20T22:06:36.463799] [DEBUG] [MAIN ] BXSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:36.465439] [INFO ] [MAIN ] BXSL: Generated 1794 derivative records from 48208 primary records. [2025-10-20T22:06:36.466084] [DEBUG] [LOCAL ] BXSL: Saving derivative data. [2025-10-20T22:06:36.485273] [DEBUG] [LOCAL ] BXSL: Derivative data saved (1794 records). [2025-10-20T22:06:36.487578] [INFO ] [MAIN ] BXSL: Successfully saved derivative data. [2025-10-20T22:06:36.491728] [INFO ] [MAIN ] Derivative Progress: 307/1877 symbols completed. BXSL took 1.1s. Estimated remaining: 9m 55s. Rate: 158.4 symbols/min [2025-10-20T22:06:36.492017] [INFO ] [MAIN ] BZH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:36.519611] [DEBUG] [MAIN ] BYM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:36.520037] [DEBUG] [MAIN ] BY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:36.520954] [INFO ] [MAIN ] BYM: Generated 1794 derivative records from 11543 primary records. [2025-10-20T22:06:36.521639] [INFO ] [MAIN ] BY: Generated 1794 derivative records from 21868 primary records. [2025-10-20T22:06:36.521570] [DEBUG] [LOCAL ] BYM: Saving derivative data. [2025-10-20T22:06:36.521753] [DEBUG] [LOCAL ] BY: Saving derivative data. [2025-10-20T22:06:36.548838] [DEBUG] [LOCAL ] BYM: Derivative data saved (1794 records). [2025-10-20T22:06:36.552847] [DEBUG] [LOCAL ] BY: Derivative data saved (1794 records). [2025-10-20T22:06:36.553023] [INFO ] [MAIN ] BY: Successfully saved derivative data. [2025-10-20T22:06:36.552043] [INFO ] [MAIN ] BYM: Successfully saved derivative data. [2025-10-20T22:06:36.555068] [INFO ] [MAIN ] Derivative Progress: 308/1877 symbols completed. BYM took 1.1s. Estimated remaining: 9m 53s. Rate: 158.8 symbols/min [2025-10-20T22:06:36.557628] [INFO ] [MAIN ] Derivative Progress: 309/1877 symbols completed. BY took 1.1s. Estimated remaining: 9m 50s. Rate: 159.3 symbols/min [2025-10-20T22:06:36.555270] [INFO ] [MAIN ] CAAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:36.557791] [INFO ] [MAIN ] CABO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:36.593771] [DEBUG] [LOCAL ] BZH: Loaded 32291 existing records from primary storage. [2025-10-20T22:06:36.594054] [INFO ] [MAIN ] BZH: Loaded 32291 primary data records. [2025-10-20T22:06:36.680834] [DEBUG] [LOCAL ] CABO: Loaded 16723 existing records from primary storage. [2025-10-20T22:06:36.680943] [DEBUG] [LOCAL ] CAAP: Loaded 19913 existing records from primary storage. [2025-10-20T22:06:36.703073] [INFO ] [MAIN ] CAAP: Loaded 19913 primary data records. [2025-10-20T22:06:36.708445] [INFO ] [MAIN ] CABO: Loaded 16723 primary data records. [2025-10-20T22:06:37.530319] [DEBUG] [MAIN ] BZH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:37.531405] [INFO ] [MAIN ] BZH: Generated 1794 derivative records from 32291 primary records. [2025-10-20T22:06:37.531743] [DEBUG] [LOCAL ] BZH: Saving derivative data. [2025-10-20T22:06:37.551011] [DEBUG] [LOCAL ] BZH: Derivative data saved (1794 records). [2025-10-20T22:06:37.552076] [INFO ] [MAIN ] BZH: Successfully saved derivative data. [2025-10-20T22:06:37.554140] [INFO ] [MAIN ] Derivative Progress: 310/1877 symbols completed. BZH took 1.1s. Estimated remaining: 9m 53s. Rate: 158.5 symbols/min [2025-10-20T22:06:37.554493] [INFO ] [MAIN ] CACI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:37.573712] [DEBUG] [MAIN ] CABO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:37.576062] [INFO ] [MAIN ] CABO: Generated 1794 derivative records from 16723 primary records. [2025-10-20T22:06:37.576279] [DEBUG] [LOCAL ] CABO: Saving derivative data. [2025-10-20T22:06:37.591984] [DEBUG] [MAIN ] CAAP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:37.613435] [DEBUG] [LOCAL ] CABO: Derivative data saved (1794 records). [2025-10-20T22:06:37.614857] [INFO ] [MAIN ] CABO: Successfully saved derivative data. [2025-10-20T22:06:37.614578] [INFO ] [MAIN ] CAAP: Generated 1794 derivative records from 19913 primary records. [2025-10-20T22:06:37.618078] [INFO ] [MAIN ] Derivative Progress: 311/1877 symbols completed. CABO took 1.1s. Estimated remaining: 9m 51s. Rate: 158.9 symbols/min [2025-10-20T22:06:37.618330] [DEBUG] [LOCAL ] CAAP: Saving derivative data. [2025-10-20T22:06:37.619692] [INFO ] [MAIN ] CADE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:37.646091] [DEBUG] [LOCAL ] CACI: Loaded 22704 existing records from primary storage. [2025-10-20T22:06:37.647677] [INFO ] [MAIN ] CACI: Loaded 22704 primary data records. [2025-10-20T22:06:37.647540] [DEBUG] [LOCAL ] CAAP: Derivative data saved (1794 records). [2025-10-20T22:06:37.669712] [INFO ] [MAIN ] CAAP: Successfully saved derivative data. [2025-10-20T22:06:37.700869] [INFO ] [MAIN ] Derivative Progress: 312/1877 symbols completed. CAAP took 1.1s. Estimated remaining: 9m 49s. Rate: 159.3 symbols/min [2025-10-20T22:06:37.732954] [INFO ] [MAIN ] CAE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:38.004848] [DEBUG] [LOCAL ] CADE: Loaded 49658 existing records from primary storage. [2025-10-20T22:06:38.005938] [INFO ] [MAIN ] CADE: Loaded 49658 primary data records. [2025-10-20T22:06:38.091334] [DEBUG] [LOCAL ] CAE: Loaded 39223 existing records from primary storage. [2025-10-20T22:06:38.121589] [INFO ] [MAIN ] CAE: Loaded 39223 primary data records. [2025-10-20T22:06:38.675945] [DEBUG] [MAIN ] CACI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:38.677747] [INFO ] [MAIN ] CACI: Generated 1794 derivative records from 22704 primary records. [2025-10-20T22:06:38.678020] [DEBUG] [LOCAL ] CACI: Saving derivative data. [2025-10-20T22:06:38.702054] [DEBUG] [LOCAL ] CACI: Derivative data saved (1794 records). [2025-10-20T22:06:38.702366] [INFO ] [MAIN ] CACI: Successfully saved derivative data. [2025-10-20T22:06:38.705877] [INFO ] [MAIN ] Derivative Progress: 313/1877 symbols completed. CACI took 1.2s. Estimated remaining: 9m 52s. Rate: 158.5 symbols/min [2025-10-20T22:06:38.726727] [DEBUG] [MAIN ] CADE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:38.727491] [INFO ] [MAIN ] CADE: Generated 1794 derivative records from 49658 primary records. [2025-10-20T22:06:38.727671] [DEBUG] [LOCAL ] CADE: Saving derivative data. [2025-10-20T22:06:38.746453] [DEBUG] [LOCAL ] CADE: Derivative data saved (1794 records). [2025-10-20T22:06:38.758811] [INFO ] [MAIN ] CADE: Successfully saved derivative data. [2025-10-20T22:06:38.760962] [INFO ] [MAIN ] Derivative Progress: 314/1877 symbols completed. CADE took 1.1s. Estimated remaining: 9m 50s. Rate: 158.9 symbols/min [2025-10-20T22:06:38.761253] [DEBUG] [MAIN ] CAE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:38.762266] [INFO ] [MAIN ] CAE: Generated 1794 derivative records from 39223 primary records. [2025-10-20T22:06:38.762335] [DEBUG] [LOCAL ] CAE: Saving derivative data. [2025-10-20T22:06:38.780548] [DEBUG] [LOCAL ] CAE: Derivative data saved (1794 records). [2025-10-20T22:06:38.780758] [INFO ] [MAIN ] CAE: Successfully saved derivative data. [2025-10-20T22:06:38.783983] [INFO ] [MAIN ] Derivative Progress: 315/1877 symbols completed. CAE took 1.1s. Estimated remaining: 9m 48s. Rate: 159.4 symbols/min [2025-10-20T22:06:38.784719] [INFO ] [MAIN ] CAH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:38.784847] [INFO ] [MAIN ] CAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:38.784620] [INFO ] [MAIN ] CAG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:38.872099] [DEBUG] [LOCAL ] CAH: Loaded 51463 existing records from primary storage. [2025-10-20T22:06:38.872778] [INFO ] [MAIN ] CAH: Loaded 51463 primary data records. [2025-10-20T22:06:38.928819] [DEBUG] [LOCAL ] CAL: Loaded 42891 existing records from primary storage. [2025-10-20T22:06:38.955499] [INFO ] [MAIN ] CAL: Loaded 42891 primary data records. [2025-10-20T22:06:39.068218] [DEBUG] [LOCAL ] CAG: Loaded 53665 existing records from primary storage. [2025-10-20T22:06:39.105794] [INFO ] [MAIN ] CAG: Loaded 53665 primary data records. [2025-10-20T22:06:39.822441] [DEBUG] [MAIN ] CAH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:39.823388] [INFO ] [MAIN ] CAH: Generated 1794 derivative records from 51463 primary records. [2025-10-20T22:06:39.823676] [DEBUG] [LOCAL ] CAH: Saving derivative data. [2025-10-20T22:06:39.842492] [DEBUG] [LOCAL ] CAH: Derivative data saved (1794 records). [2025-10-20T22:06:39.842601] [INFO ] [MAIN ] CAH: Successfully saved derivative data. [2025-10-20T22:06:39.844400] [INFO ] [MAIN ] Derivative Progress: 316/1877 symbols completed. CAH took 1.1s. Estimated remaining: 9m 51s. Rate: 158.5 symbols/min [2025-10-20T22:06:39.844616] [INFO ] [MAIN ] CALX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:39.859182] [DEBUG] [MAIN ] CAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:39.867273] [INFO ] [MAIN ] CAL: Generated 1794 derivative records from 42891 primary records. [2025-10-20T22:06:39.871962] [DEBUG] [LOCAL ] CAL: Saving derivative data. [2025-10-20T22:06:39.873112] [DEBUG] [MAIN ] CAG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:39.873977] [INFO ] [MAIN ] CAG: Generated 1794 derivative records from 53665 primary records. [2025-10-20T22:06:39.881264] [DEBUG] [LOCAL ] CAG: Saving derivative data. [2025-10-20T22:06:39.889633] [DEBUG] [LOCAL ] CAL: Derivative data saved (1794 records). [2025-10-20T22:06:39.897061] [INFO ] [MAIN ] CAL: Successfully saved derivative data. [2025-10-20T22:06:39.898874] [INFO ] [MAIN ] Derivative Progress: 317/1877 symbols completed. CAL took 1.1s. Estimated remaining: 9m 49s. Rate: 158.9 symbols/min [2025-10-20T22:06:39.900004] [INFO ] [MAIN ] CANG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:39.907093] [DEBUG] [LOCAL ] CAG: Derivative data saved (1794 records). [2025-10-20T22:06:39.907199] [INFO ] [MAIN ] CAG: Successfully saved derivative data. [2025-10-20T22:06:39.909771] [INFO ] [MAIN ] Derivative Progress: 318/1877 symbols completed. CAG took 1.1s. Estimated remaining: 9m 47s. Rate: 159.4 symbols/min [2025-10-20T22:06:39.910084] [INFO ] [MAIN ] CARR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:39.953127] [DEBUG] [LOCAL ] CANG: Loaded 27136 existing records from primary storage. [2025-10-20T22:06:39.953274] [INFO ] [MAIN ] CANG: Loaded 27136 primary data records. [2025-10-20T22:06:39.953828] [DEBUG] [LOCAL ] CALX: Loaded 37496 existing records from primary storage. [2025-10-20T22:06:39.954022] [INFO ] [MAIN ] CALX: Loaded 37496 primary data records. [2025-10-20T22:06:40.211504] [DEBUG] [LOCAL ] CARR: Loaded 53603 existing records from primary storage. [2025-10-20T22:06:40.274683] [INFO ] [MAIN ] CARR: Loaded 53603 primary data records. [2025-10-20T22:06:40.922510] [DEBUG] [MAIN ] CANG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:40.923400] [INFO ] [MAIN ] CANG: Generated 1794 derivative records from 27136 primary records. [2025-10-20T22:06:40.924290] [DEBUG] [LOCAL ] CANG: Saving derivative data. [2025-10-20T22:06:40.949569] [DEBUG] [LOCAL ] CANG: Derivative data saved (1794 records). [2025-10-20T22:06:40.949750] [INFO ] [MAIN ] CANG: Successfully saved derivative data. [2025-10-20T22:06:40.956109] [INFO ] [MAIN ] Derivative Progress: 319/1877 symbols completed. CANG took 1.1s. Estimated remaining: 9m 50s. Rate: 158.5 symbols/min [2025-10-20T22:06:40.956317] [INFO ] [MAIN ] CARS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:40.958114] [DEBUG] [MAIN ] CALX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:40.960279] [INFO ] [MAIN ] CALX: Generated 1794 derivative records from 37496 primary records. [2025-10-20T22:06:40.961026] [DEBUG] [LOCAL ] CALX: Saving derivative data. [2025-10-20T22:06:40.990167] [DEBUG] [LOCAL ] CALX: Derivative data saved (1794 records). [2025-10-20T22:06:40.989160] [DEBUG] [MAIN ] CARR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:40.990384] [INFO ] [MAIN ] CALX: Successfully saved derivative data. [2025-10-20T22:06:40.993223] [INFO ] [MAIN ] Derivative Progress: 320/1877 symbols completed. CALX took 1.1s. Estimated remaining: 9m 48s. Rate: 158.9 symbols/min [2025-10-20T22:06:40.991255] [INFO ] [MAIN ] CARR: Generated 1794 derivative records from 53603 primary records. [2025-10-20T22:06:40.993416] [DEBUG] [LOCAL ] CARR: Saving derivative data. [2025-10-20T22:06:40.993545] [INFO ] [MAIN ] CAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:41.019845] [DEBUG] [LOCAL ] CARR: Derivative data saved (1794 records). [2025-10-20T22:06:41.019980] [INFO ] [MAIN ] CARR: Successfully saved derivative data. [2025-10-20T22:06:41.022704] [INFO ] [MAIN ] Derivative Progress: 321/1877 symbols completed. CARR took 1.1s. Estimated remaining: 9m 46s. Rate: 159.4 symbols/min [2025-10-20T22:06:41.023043] [INFO ] [MAIN ] CATO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:41.037465] [DEBUG] [LOCAL ] CARS: Loaded 39802 existing records from primary storage. [2025-10-20T22:06:41.039297] [INFO ] [MAIN ] CARS: Loaded 39802 primary data records. [2025-10-20T22:06:41.113371] [DEBUG] [LOCAL ] CATO: Loaded 6648 existing records from primary storage. [2025-10-20T22:06:41.130333] [INFO ] [MAIN ] CATO: Loaded 6648 primary data records. [2025-10-20T22:06:41.531547] [DEBUG] [LOCAL ] CAT: Loaded 52966 existing records from primary storage. [2025-10-20T22:06:41.546395] [INFO ] [MAIN ] CAT: Loaded 52966 primary data records. [2025-10-20T22:06:42.064050] [DEBUG] [MAIN ] CARS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:42.065715] [INFO ] [MAIN ] CARS: Generated 1794 derivative records from 39802 primary records. [2025-10-20T22:06:42.065950] [DEBUG] [LOCAL ] CARS: Saving derivative data. [2025-10-20T22:06:42.086240] [DEBUG] [LOCAL ] CARS: Derivative data saved (1794 records). [2025-10-20T22:06:42.086337] [INFO ] [MAIN ] CARS: Successfully saved derivative data. [2025-10-20T22:06:42.088258] [INFO ] [MAIN ] Derivative Progress: 322/1877 symbols completed. CARS took 1.1s. Estimated remaining: 9m 49s. Rate: 158.5 symbols/min [2025-10-20T22:06:42.096080] [INFO ] [MAIN ] CAVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:42.101899] [DEBUG] [MAIN ] CATO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:42.102749] [INFO ] [MAIN ] CATO: Generated 1794 derivative records from 6648 primary records. [2025-10-20T22:06:42.103267] [DEBUG] [LOCAL ] CATO: Saving derivative data. [2025-10-20T22:06:42.118185] [DEBUG] [LOCAL ] CATO: Derivative data saved (1794 records). [2025-10-20T22:06:42.118373] [INFO ] [MAIN ] CATO: Successfully saved derivative data. [2025-10-20T22:06:42.120409] [INFO ] [MAIN ] Derivative Progress: 323/1877 symbols completed. CATO took 1.1s. Estimated remaining: 9m 47s. Rate: 158.9 symbols/min [2025-10-20T22:06:42.121481] [INFO ] [MAIN ] CB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:42.136410] [DEBUG] [MAIN ] CAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:42.137203] [INFO ] [MAIN ] CAT: Generated 1794 derivative records from 52966 primary records. [2025-10-20T22:06:42.137371] [DEBUG] [LOCAL ] CAT: Saving derivative data. [2025-10-20T22:06:42.155024] [DEBUG] [LOCAL ] CAT: Derivative data saved (1794 records). [2025-10-20T22:06:42.155090] [INFO ] [MAIN ] CAT: Successfully saved derivative data. [2025-10-20T22:06:42.157845] [INFO ] [MAIN ] Derivative Progress: 324/1877 symbols completed. CAT took 1.2s. Estimated remaining: 9m 45s. Rate: 159.4 symbols/min [2025-10-20T22:06:42.158066] [INFO ] [MAIN ] CBAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:42.184138] [DEBUG] [LOCAL ] CBAN: Loaded 10299 existing records from primary storage. [2025-10-20T22:06:42.193449] [INFO ] [MAIN ] CBAN: Loaded 10299 primary data records. [2025-10-20T22:06:42.304267] [DEBUG] [LOCAL ] CAVA: Loaded 53552 existing records from primary storage. [2025-10-20T22:06:42.329344] [INFO ] [MAIN ] CAVA: Loaded 53552 primary data records. [2025-10-20T22:06:42.378115] [DEBUG] [LOCAL ] CB: Loaded 49967 existing records from primary storage. [2025-10-20T22:06:42.390038] [INFO ] [MAIN ] CB: Loaded 49967 primary data records. [2025-10-20T22:06:43.148507] [DEBUG] [MAIN ] CBAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:43.149360] [INFO ] [MAIN ] CBAN: Generated 1794 derivative records from 10299 primary records. [2025-10-20T22:06:43.149531] [DEBUG] [LOCAL ] CBAN: Saving derivative data. [2025-10-20T22:06:43.168489] [DEBUG] [LOCAL ] CBAN: Derivative data saved (1794 records). [2025-10-20T22:06:43.168710] [INFO ] [MAIN ] CBAN: Successfully saved derivative data. [2025-10-20T22:06:43.170733] [INFO ] [MAIN ] Derivative Progress: 325/1877 symbols completed. CBAN took 1.0s. Estimated remaining: 9m 47s. Rate: 158.6 symbols/min [2025-10-20T22:06:43.170931] [INFO ] [MAIN ] CBL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:43.185752] [DEBUG] [MAIN ] CB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:43.186460] [INFO ] [MAIN ] CB: Generated 1794 derivative records from 49967 primary records. [2025-10-20T22:06:43.186713] [DEBUG] [LOCAL ] CB: Saving derivative data. [2025-10-20T22:06:43.221711] [DEBUG] [MAIN ] CAVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:43.228612] [DEBUG] [LOCAL ] CB: Derivative data saved (1794 records). [2025-10-20T22:06:43.229792] [DEBUG] [LOCAL ] CBL: Loaded 16230 existing records from primary storage. [2025-10-20T22:06:43.229450] [INFO ] [MAIN ] CAVA: Generated 1794 derivative records from 53552 primary records. [2025-10-20T22:06:43.230099] [INFO ] [MAIN ] CB: Successfully saved derivative data. [2025-10-20T22:06:43.230045] [DEBUG] [LOCAL ] CAVA: Saving derivative data. [2025-10-20T22:06:43.229937] [INFO ] [MAIN ] CBL: Loaded 16230 primary data records. [2025-10-20T22:06:43.244054] [INFO ] [MAIN ] Derivative Progress: 326/1877 symbols completed. CB took 1.1s. Estimated remaining: 9m 45s. Rate: 159.0 symbols/min [2025-10-20T22:06:43.244282] [INFO ] [MAIN ] CBNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:43.356212] [DEBUG] [LOCAL ] CBNA: Loaded 2900 existing records from primary storage. [2025-10-20T22:06:43.366561] [INFO ] [MAIN ] CBNA: Loaded 2900 primary data records. [2025-10-20T22:06:43.416047] [DEBUG] [LOCAL ] CAVA: Derivative data saved (1794 records). [2025-10-20T22:06:43.452387] [INFO ] [MAIN ] CAVA: Successfully saved derivative data. [2025-10-20T22:06:43.495597] [INFO ] [MAIN ] Derivative Progress: 327/1877 symbols completed. CAVA took 1.4s. Estimated remaining: 9m 44s. Rate: 159.1 symbols/min [2025-10-20T22:06:43.514749] [INFO ] [MAIN ] CBRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:43.934453] [DEBUG] [MAIN ] CBL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:43.952346] [INFO ] [MAIN ] CBL: Generated 1794 derivative records from 16230 primary records. [2025-10-20T22:06:43.954864] [DEBUG] [LOCAL ] CBRE: Loaded 50305 existing records from primary storage. [2025-10-20T22:06:43.953535] [DEBUG] [LOCAL ] CBL: Saving derivative data. [2025-10-20T22:06:43.956717] [INFO ] [MAIN ] CBRE: Loaded 50305 primary data records. [2025-10-20T22:06:44.031432] [DEBUG] [MAIN ] CBNA: Filtered 7001 bins outside market hours, 7823 incomplete bins [2025-10-20T22:06:44.042769] [INFO ] [MAIN ] CBNA: Generated 1791 derivative records from 2900 primary records. [2025-10-20T22:06:44.031519] [DEBUG] [LOCAL ] CBL: Derivative data saved (1794 records). [2025-10-20T22:06:44.061680] [DEBUG] [LOCAL ] CBNA: Saving derivative data. [2025-10-20T22:06:44.078222] [INFO ] [MAIN ] CBL: Successfully saved derivative data. [2025-10-20T22:06:44.080235] [INFO ] [MAIN ] Derivative Progress: 328/1877 symbols completed. CBL took 0.9s. Estimated remaining: 9m 45s. Rate: 158.9 symbols/min [2025-10-20T22:06:44.181146] [DEBUG] [LOCAL ] CBNA: Derivative data saved (1791 records). [2025-10-20T22:06:44.202299] [INFO ] [MAIN ] CBNA: Successfully saved derivative data. [2025-10-20T22:06:44.228415] [INFO ] [MAIN ] Derivative Progress: 329/1877 symbols completed. CBNA took 1.0s. Estimated remaining: 9m 44s. Rate: 159.2 symbols/min [2025-10-20T22:06:44.481602] [DEBUG] [MAIN ] CBRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:44.482356] [INFO ] [MAIN ] CBRE: Generated 1794 derivative records from 50305 primary records. [2025-10-20T22:06:44.482429] [DEBUG] [LOCAL ] CBRE: Saving derivative data. [2025-10-20T22:06:44.499491] [DEBUG] [LOCAL ] CBRE: Derivative data saved (1794 records). [2025-10-20T22:06:44.499630] [INFO ] [MAIN ] CBRE: Successfully saved derivative data. [2025-10-20T22:06:44.501805] [INFO ] [MAIN ] Derivative Progress: 330/1877 symbols completed. CBRE took 1.0s. Estimated remaining: 9m 43s. Rate: 159.3 symbols/min [2025-10-20T22:06:44.502484] [INFO ] [MAIN ] CBZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:44.502861] [INFO ] [MAIN ] CBT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:44.502725] [INFO ] [MAIN ] CBU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:44.552393] [DEBUG] [LOCAL ] CBU: Loaded 22862 existing records from primary storage. [2025-10-20T22:06:44.552705] [INFO ] [MAIN ] CBU: Loaded 22862 primary data records. [2025-10-20T22:06:44.696885] [DEBUG] [LOCAL ] CBZ: Loaded 34284 existing records from primary storage. [2025-10-20T22:06:44.703426] [DEBUG] [LOCAL ] CBT: Loaded 31150 existing records from primary storage. [2025-10-20T22:06:44.731969] [INFO ] [MAIN ] CBT: Loaded 31150 primary data records. [2025-10-20T22:06:44.703617] [INFO ] [MAIN ] CBZ: Loaded 34284 primary data records. [2025-10-20T22:06:45.599211] [DEBUG] [MAIN ] CBU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:45.600560] [INFO ] [MAIN ] CBU: Generated 1794 derivative records from 22862 primary records. [2025-10-20T22:06:45.600930] [DEBUG] [LOCAL ] CBU: Saving derivative data. [2025-10-20T22:06:45.618556] [DEBUG] [LOCAL ] CBU: Derivative data saved (1794 records). [2025-10-20T22:06:45.618915] [INFO ] [MAIN ] CBU: Successfully saved derivative data. [2025-10-20T22:06:45.622786] [INFO ] [MAIN ] Derivative Progress: 331/1877 symbols completed. CBU took 1.1s. Estimated remaining: 9m 46s. Rate: 158.3 symbols/min [2025-10-20T22:06:45.623893] [INFO ] [MAIN ] CC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:45.638690] [DEBUG] [MAIN ] CBZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:45.639561] [INFO ] [MAIN ] CBZ: Generated 1794 derivative records from 34284 primary records. [2025-10-20T22:06:45.639698] [DEBUG] [LOCAL ] CBZ: Saving derivative data. [2025-10-20T22:06:45.658714] [DEBUG] [LOCAL ] CBZ: Derivative data saved (1794 records). [2025-10-20T22:06:45.659298] [INFO ] [MAIN ] CBZ: Successfully saved derivative data. [2025-10-20T22:06:45.674108] [INFO ] [MAIN ] Derivative Progress: 332/1877 symbols completed. CBZ took 1.2s. Estimated remaining: 9m 44s. Rate: 158.8 symbols/min [2025-10-20T22:06:45.674378] [INFO ] [MAIN ] CCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:45.675594] [DEBUG] [MAIN ] CBT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:45.676714] [INFO ] [MAIN ] CBT: Generated 1794 derivative records from 31150 primary records. [2025-10-20T22:06:45.676923] [DEBUG] [LOCAL ] CBT: Saving derivative data. [2025-10-20T22:06:45.693824] [DEBUG] [LOCAL ] CBT: Derivative data saved (1794 records). [2025-10-20T22:06:45.694037] [INFO ] [MAIN ] CBT: Successfully saved derivative data. [2025-10-20T22:06:45.696140] [INFO ] [MAIN ] Derivative Progress: 333/1877 symbols completed. CBT took 1.2s. Estimated remaining: 9m 42s. Rate: 159.2 symbols/min [2025-10-20T22:06:45.696387] [INFO ] [MAIN ] CCIF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:45.737592] [DEBUG] [LOCAL ] CC: Loaded 53030 existing records from primary storage. [2025-10-20T22:06:45.737821] [INFO ] [MAIN ] CC: Loaded 53030 primary data records. [2025-10-20T22:06:45.811740] [DEBUG] [LOCAL ] CCIF: Loaded 15294 existing records from primary storage. [2025-10-20T22:06:45.833892] [INFO ] [MAIN ] CCIF: Loaded 15294 primary data records. [2025-10-20T22:06:45.972926] [DEBUG] [LOCAL ] CCI: Loaded 53116 existing records from primary storage. [2025-10-20T22:06:46.061855] [INFO ] [MAIN ] CCI: Loaded 53116 primary data records. [2025-10-20T22:06:46.736046] [DEBUG] [MAIN ] CC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:46.737230] [INFO ] [MAIN ] CC: Generated 1794 derivative records from 53030 primary records. [2025-10-20T22:06:46.738081] [DEBUG] [LOCAL ] CC: Saving derivative data. [2025-10-20T22:06:46.757734] [DEBUG] [LOCAL ] CC: Derivative data saved (1794 records). [2025-10-20T22:06:46.758394] [INFO ] [MAIN ] CC: Successfully saved derivative data. [2025-10-20T22:06:46.773706] [DEBUG] [MAIN ] CCIF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:46.773296] [INFO ] [MAIN ] Derivative Progress: 334/1877 symbols completed. CC took 1.1s. Estimated remaining: 9m 45s. Rate: 158.3 symbols/min [2025-10-20T22:06:46.774879] [INFO ] [MAIN ] CCIF: Generated 1794 derivative records from 15294 primary records. [2025-10-20T22:06:46.776006] [DEBUG] [LOCAL ] CCIF: Saving derivative data. [2025-10-20T22:06:46.776925] [INFO ] [MAIN ] CCJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:46.800866] [DEBUG] [LOCAL ] CCIF: Derivative data saved (1794 records). [2025-10-20T22:06:46.806256] [INFO ] [MAIN ] CCIF: Successfully saved derivative data. [2025-10-20T22:06:46.808471] [INFO ] [MAIN ] Derivative Progress: 335/1877 symbols completed. CCIF took 1.1s. Estimated remaining: 9m 43s. Rate: 158.8 symbols/min [2025-10-20T22:06:46.806188] [DEBUG] [MAIN ] CCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:46.808814] [INFO ] [MAIN ] CCK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:46.810057] [INFO ] [MAIN ] CCI: Generated 1794 derivative records from 53116 primary records. [2025-10-20T22:06:46.810212] [DEBUG] [LOCAL ] CCI: Saving derivative data. [2025-10-20T22:06:46.829714] [DEBUG] [LOCAL ] CCI: Derivative data saved (1794 records). [2025-10-20T22:06:46.829846] [INFO ] [MAIN ] CCI: Successfully saved derivative data. [2025-10-20T22:06:46.832194] [INFO ] [MAIN ] Derivative Progress: 336/1877 symbols completed. CCI took 1.2s. Estimated remaining: 9m 41s. Rate: 159.2 symbols/min [2025-10-20T22:06:46.832557] [INFO ] [MAIN ] CCL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:46.878238] [DEBUG] [LOCAL ] CCJ: Loaded 53690 existing records from primary storage. [2025-10-20T22:06:46.878474] [INFO ] [MAIN ] CCJ: Loaded 53690 primary data records. [2025-10-20T22:06:47.032830] [DEBUG] [LOCAL ] CCK: Loaded 46482 existing records from primary storage. [2025-10-20T22:06:47.043500] [INFO ] [MAIN ] CCK: Loaded 46482 primary data records. [2025-10-20T22:06:47.313429] [DEBUG] [LOCAL ] CCL: Loaded 53800 existing records from primary storage. [2025-10-20T22:06:47.319445] [INFO ] [MAIN ] CCL: Loaded 53800 primary data records. [2025-10-20T22:06:47.865726] [DEBUG] [MAIN ] CCJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:47.866442] [INFO ] [MAIN ] CCJ: Generated 1794 derivative records from 53690 primary records. [2025-10-20T22:06:47.866615] [DEBUG] [LOCAL ] CCJ: Saving derivative data. [2025-10-20T22:06:47.884564] [DEBUG] [LOCAL ] CCJ: Derivative data saved (1794 records). [2025-10-20T22:06:47.884662] [INFO ] [MAIN ] CCJ: Successfully saved derivative data. [2025-10-20T22:06:47.886830] [INFO ] [MAIN ] Derivative Progress: 337/1877 symbols completed. CCJ took 1.1s. Estimated remaining: 9m 44s. Rate: 158.3 symbols/min [2025-10-20T22:06:47.887915] [INFO ] [MAIN ] CCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:47.906909] [DEBUG] [MAIN ] CCK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:47.907700] [INFO ] [MAIN ] CCK: Generated 1794 derivative records from 46482 primary records. [2025-10-20T22:06:47.907904] [DEBUG] [LOCAL ] CCK: Saving derivative data. [2025-10-20T22:06:47.935145] [DEBUG] [MAIN ] CCL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:47.937088] [INFO ] [MAIN ] CCL: Generated 1794 derivative records from 53800 primary records. [2025-10-20T22:06:47.939309] [DEBUG] [LOCAL ] CCL: Saving derivative data. [2025-10-20T22:06:47.939222] [DEBUG] [LOCAL ] CCK: Derivative data saved (1794 records). [2025-10-20T22:06:47.940141] [INFO ] [MAIN ] CCK: Successfully saved derivative data. [2025-10-20T22:06:47.942036] [INFO ] [MAIN ] Derivative Progress: 338/1877 symbols completed. CCK took 1.1s. Estimated remaining: 9m 42s. Rate: 158.8 symbols/min [2025-10-20T22:06:47.942449] [INFO ] [MAIN ] CCRD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:47.968580] [DEBUG] [LOCAL ] CCL: Derivative data saved (1794 records). [2025-10-20T22:06:47.968891] [INFO ] [MAIN ] CCL: Successfully saved derivative data. [2025-10-20T22:06:47.970932] [INFO ] [MAIN ] Derivative Progress: 339/1877 symbols completed. CCL took 1.1s. Estimated remaining: 9m 40s. Rate: 159.2 symbols/min [2025-10-20T22:06:47.971150] [INFO ] [MAIN ] CCS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:47.979715] [DEBUG] [LOCAL ] CCRD: Loaded 9603 existing records from primary storage. [2025-10-20T22:06:47.980491] [INFO ] [MAIN ] CCRD: Loaded 9603 primary data records. [2025-10-20T22:06:48.062983] [DEBUG] [LOCAL ] CCO: Loaded 48483 existing records from primary storage. [2025-10-20T22:06:48.103902] [INFO ] [MAIN ] CCO: Loaded 48483 primary data records. [2025-10-20T22:06:48.210067] [DEBUG] [LOCAL ] CCS: Loaded 30084 existing records from primary storage. [2025-10-20T22:06:48.242606] [INFO ] [MAIN ] CCS: Loaded 30084 primary data records. [2025-10-20T22:06:48.982779] [DEBUG] [MAIN ] CCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:48.982566] [DEBUG] [MAIN ] CCRD: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:06:48.983953] [INFO ] [MAIN ] CCRD: Generated 1793 derivative records from 9603 primary records. [2025-10-20T22:06:48.984858] [DEBUG] [LOCAL ] CCRD: Saving derivative data. [2025-10-20T22:06:48.984719] [INFO ] [MAIN ] CCO: Generated 1794 derivative records from 48483 primary records. [2025-10-20T22:06:48.985885] [DEBUG] [LOCAL ] CCO: Saving derivative data. [2025-10-20T22:06:49.010495] [DEBUG] [LOCAL ] CCRD: Derivative data saved (1793 records). [2025-10-20T22:06:49.013279] [INFO ] [MAIN ] CCRD: Successfully saved derivative data. [2025-10-20T22:06:49.017537] [DEBUG] [LOCAL ] CCO: Derivative data saved (1794 records). [2025-10-20T22:06:49.019575] [INFO ] [MAIN ] Derivative Progress: 340/1877 symbols completed. CCRD took 1.1s. Estimated remaining: 9m 42s. Rate: 158.4 symbols/min [2025-10-20T22:06:49.019705] [INFO ] [MAIN ] CCO: Successfully saved derivative data. [2025-10-20T22:06:49.019939] [INFO ] [MAIN ] CCU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:49.023133] [INFO ] [MAIN ] Derivative Progress: 341/1877 symbols completed. CCO took 1.1s. Estimated remaining: 9m 40s. Rate: 158.8 symbols/min [2025-10-20T22:06:49.024191] [INFO ] [MAIN ] CDE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:49.038970] [DEBUG] [MAIN ] CCS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:49.039717] [INFO ] [MAIN ] CCS: Generated 1794 derivative records from 30084 primary records. [2025-10-20T22:06:49.039918] [DEBUG] [LOCAL ] CCS: Saving derivative data. [2025-10-20T22:06:49.067504] [DEBUG] [LOCAL ] CCS: Derivative data saved (1794 records). [2025-10-20T22:06:49.068681] [INFO ] [MAIN ] CCS: Successfully saved derivative data. [2025-10-20T22:06:49.072731] [INFO ] [MAIN ] Derivative Progress: 342/1877 symbols completed. CCS took 1.1s. Estimated remaining: 9m 38s. Rate: 159.2 symbols/min [2025-10-20T22:06:49.070686] [DEBUG] [LOCAL ] CCU: Loaded 21982 existing records from primary storage. [2025-10-20T22:06:49.072925] [INFO ] [MAIN ] CDLR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:49.073488] [INFO ] [MAIN ] CCU: Loaded 21982 primary data records. [2025-10-20T22:06:49.235443] [DEBUG] [LOCAL ] CDLR: Loaded 10221 existing records from primary storage. [2025-10-20T22:06:49.244107] [INFO ] [MAIN ] CDLR: Loaded 10221 primary data records. [2025-10-20T22:06:49.505736] [DEBUG] [LOCAL ] CDE: Loaded 53643 existing records from primary storage. [2025-10-20T22:06:49.517280] [INFO ] [MAIN ] CDE: Loaded 53643 primary data records. [2025-10-20T22:06:50.059390] [DEBUG] [MAIN ] CCU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:50.061042] [INFO ] [MAIN ] CCU: Generated 1794 derivative records from 21982 primary records. [2025-10-20T22:06:50.061448] [DEBUG] [LOCAL ] CCU: Saving derivative data. [2025-10-20T22:06:50.080828] [DEBUG] [LOCAL ] CCU: Derivative data saved (1794 records). [2025-10-20T22:06:50.082056] [INFO ] [MAIN ] CCU: Successfully saved derivative data. [2025-10-20T22:06:50.084348] [INFO ] [MAIN ] Derivative Progress: 343/1877 symbols completed. CCU took 1.1s. Estimated remaining: 9m 41s. Rate: 158.4 symbols/min [2025-10-20T22:06:50.099715] [DEBUG] [MAIN ] CDLR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:50.100423] [INFO ] [MAIN ] CDLR: Generated 1794 derivative records from 10221 primary records. [2025-10-20T22:06:50.100576] [DEBUG] [LOCAL ] CDLR: Saving derivative data. [2025-10-20T22:06:50.118571] [DEBUG] [LOCAL ] CDLR: Derivative data saved (1794 records). [2025-10-20T22:06:50.118772] [INFO ] [MAIN ] CDLR: Successfully saved derivative data. [2025-10-20T22:06:50.133107] [DEBUG] [MAIN ] CDE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:50.132767] [INFO ] [MAIN ] Derivative Progress: 344/1877 symbols completed. CDLR took 1.1s. Estimated remaining: 9m 39s. Rate: 158.8 symbols/min [2025-10-20T22:06:50.133916] [INFO ] [MAIN ] CDE: Generated 1794 derivative records from 53643 primary records. [2025-10-20T22:06:50.133977] [DEBUG] [LOCAL ] CDE: Saving derivative data. [2025-10-20T22:06:50.148715] [DEBUG] [LOCAL ] CDE: Derivative data saved (1794 records). [2025-10-20T22:06:50.148796] [INFO ] [MAIN ] CDE: Successfully saved derivative data. [2025-10-20T22:06:50.150703] [INFO ] [MAIN ] Derivative Progress: 345/1877 symbols completed. CDE took 1.1s. Estimated remaining: 9m 37s. Rate: 159.3 symbols/min [2025-10-20T22:06:50.151129] [INFO ] [MAIN ] CDRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:50.151299] [INFO ] [MAIN ] CE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:50.151184] [INFO ] [MAIN ] CEE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:50.159706] [DEBUG] [LOCAL ] CEE: Loaded 5304 existing records from primary storage. [2025-10-20T22:06:50.159950] [INFO ] [MAIN ] CEE: Loaded 5304 primary data records. [2025-10-20T22:06:50.395949] [DEBUG] [LOCAL ] CDRE: Loaded 24879 existing records from primary storage. [2025-10-20T22:06:50.511789] [INFO ] [MAIN ] CDRE: Loaded 24879 primary data records. [2025-10-20T22:06:50.630877] [DEBUG] [LOCAL ] CE: Loaded 51312 existing records from primary storage. [2025-10-20T22:06:50.641944] [INFO ] [MAIN ] CE: Loaded 51312 primary data records. [2025-10-20T22:06:51.134437] [DEBUG] [MAIN ] CEE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:51.135878] [INFO ] [MAIN ] CEE: Generated 1794 derivative records from 5304 primary records. [2025-10-20T22:06:51.136122] [DEBUG] [LOCAL ] CEE: Saving derivative data. [2025-10-20T22:06:51.165788] [DEBUG] [LOCAL ] CEE: Derivative data saved (1794 records). [2025-10-20T22:06:51.166220] [DEBUG] [MAIN ] CDRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:51.165994] [INFO ] [MAIN ] CEE: Successfully saved derivative data. [2025-10-20T22:06:51.168513] [INFO ] [MAIN ] Derivative Progress: 346/1877 symbols completed. CEE took 1.0s. Estimated remaining: 9m 40s. Rate: 158.5 symbols/min [2025-10-20T22:06:51.169399] [INFO ] [MAIN ] CDRE: Generated 1794 derivative records from 24879 primary records. [2025-10-20T22:06:51.168678] [INFO ] [MAIN ] CEPU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:51.169670] [DEBUG] [LOCAL ] CDRE: Saving derivative data. [2025-10-20T22:06:51.186016] [DEBUG] [LOCAL ] CDRE: Derivative data saved (1794 records). [2025-10-20T22:06:51.186443] [INFO ] [MAIN ] CDRE: Successfully saved derivative data. [2025-10-20T22:06:51.188451] [INFO ] [MAIN ] Derivative Progress: 347/1877 symbols completed. CDRE took 1.0s. Estimated remaining: 9m 38s. Rate: 158.9 symbols/min [2025-10-20T22:06:51.188746] [INFO ] [MAIN ] CF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:51.211600] [DEBUG] [MAIN ] CE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:51.212327] [INFO ] [MAIN ] CE: Generated 1794 derivative records from 51312 primary records. [2025-10-20T22:06:51.212450] [DEBUG] [LOCAL ] CE: Saving derivative data. [2025-10-20T22:06:51.238486] [DEBUG] [LOCAL ] CEPU: Loaded 26925 existing records from primary storage. [2025-10-20T22:06:51.239028] [INFO ] [MAIN ] CEPU: Loaded 26925 primary data records. [2025-10-20T22:06:51.238969] [DEBUG] [LOCAL ] CE: Derivative data saved (1794 records). [2025-10-20T22:06:51.239240] [INFO ] [MAIN ] CE: Successfully saved derivative data. [2025-10-20T22:06:51.297958] [INFO ] [MAIN ] Derivative Progress: 348/1877 symbols completed. CE took 1.1s. Estimated remaining: 9m 36s. Rate: 159.3 symbols/min [2025-10-20T22:06:51.324242] [INFO ] [MAIN ] CFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:51.566927] [DEBUG] [LOCAL ] CF: Loaded 52126 existing records from primary storage. [2025-10-20T22:06:51.567123] [INFO ] [MAIN ] CF: Loaded 52126 primary data records. [2025-10-20T22:06:51.897788] [DEBUG] [LOCAL ] CFG: Loaded 53623 existing records from primary storage. [2025-10-20T22:06:51.898955] [INFO ] [MAIN ] CFG: Loaded 53623 primary data records. [2025-10-20T22:06:52.075896] [DEBUG] [MAIN ] CEPU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:52.250521] [INFO ] [MAIN ] CEPU: Generated 1794 derivative records from 26925 primary records. [2025-10-20T22:06:52.258993] [DEBUG] [LOCAL ] CEPU: Saving derivative data. [2025-10-20T22:06:52.277315] [DEBUG] [LOCAL ] CEPU: Derivative data saved (1794 records). [2025-10-20T22:06:52.277605] [INFO ] [MAIN ] CEPU: Successfully saved derivative data. [2025-10-20T22:06:52.279919] [INFO ] [MAIN ] Derivative Progress: 349/1877 symbols completed. CEPU took 1.1s. Estimated remaining: 9m 38s. Rate: 158.5 symbols/min [2025-10-20T22:06:52.280192] [INFO ] [MAIN ] CFR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:52.297337] [DEBUG] [MAIN ] CF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:52.298066] [INFO ] [MAIN ] CF: Generated 1794 derivative records from 52126 primary records. [2025-10-20T22:06:52.298288] [DEBUG] [LOCAL ] CF: Saving derivative data. [2025-10-20T22:06:52.315090] [DEBUG] [LOCAL ] CF: Derivative data saved (1794 records). [2025-10-20T22:06:52.315253] [INFO ] [MAIN ] CF: Successfully saved derivative data. [2025-10-20T22:06:52.317295] [INFO ] [MAIN ] Derivative Progress: 350/1877 symbols completed. CF took 1.1s. Estimated remaining: 9m 36s. Rate: 158.9 symbols/min [2025-10-20T22:06:52.318708] [INFO ] [MAIN ] CGAU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:52.341675] [DEBUG] [MAIN ] CFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:52.342400] [INFO ] [MAIN ] CFG: Generated 1794 derivative records from 53623 primary records. [2025-10-20T22:06:52.342510] [DEBUG] [LOCAL ] CFG: Saving derivative data. [2025-10-20T22:06:52.368122] [DEBUG] [LOCAL ] CFG: Derivative data saved (1794 records). [2025-10-20T22:06:52.368996] [DEBUG] [LOCAL ] CFR: Loaded 32637 existing records from primary storage. [2025-10-20T22:06:52.369205] [INFO ] [MAIN ] CFG: Successfully saved derivative data. [2025-10-20T22:06:52.369278] [INFO ] [MAIN ] CFR: Loaded 32637 primary data records. [2025-10-20T22:06:52.371603] [INFO ] [MAIN ] Derivative Progress: 351/1877 symbols completed. CFG took 1.0s. Estimated remaining: 9m 35s. Rate: 159.3 symbols/min [2025-10-20T22:06:52.371892] [INFO ] [MAIN ] CGV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:52.486829] [DEBUG] [LOCAL ] CGV: Loaded 1384 existing records from primary storage. [2025-10-20T22:06:52.499303] [INFO ] [MAIN ] CGV: Loaded 1384 primary data records. [2025-10-20T22:06:52.500136] [DEBUG] [LOCAL ] CGAU: Loaded 43818 existing records from primary storage. [2025-10-20T22:06:52.521864] [INFO ] [MAIN ] CGAU: Loaded 43818 primary data records. [2025-10-20T22:06:53.299722] [DEBUG] [MAIN ] CFR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:53.301877] [INFO ] [MAIN ] CFR: Generated 1794 derivative records from 32637 primary records. [2025-10-20T22:06:53.302092] [DEBUG] [LOCAL ] CFR: Saving derivative data. [2025-10-20T22:06:53.325474] [DEBUG] [MAIN ] CGV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:53.326094] [INFO ] [MAIN ] CGV: Generated 1794 derivative records from 1384 primary records. [2025-10-20T22:06:53.330721] [DEBUG] [LOCAL ] CGV: Saving derivative data. [2025-10-20T22:06:53.351717] [DEBUG] [MAIN ] CGAU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:53.356010] [INFO ] [MAIN ] CGAU: Generated 1794 derivative records from 43818 primary records. [2025-10-20T22:06:53.356773] [DEBUG] [LOCAL ] CGAU: Saving derivative data. [2025-10-20T22:06:53.362114] [DEBUG] [LOCAL ] CFR: Derivative data saved (1794 records). [2025-10-20T22:06:53.362365] [INFO ] [MAIN ] CFR: Successfully saved derivative data. [2025-10-20T22:06:53.364414] [INFO ] [MAIN ] Derivative Progress: 352/1877 symbols completed. CFR took 1.1s. Estimated remaining: 9m 37s. Rate: 158.6 symbols/min [2025-10-20T22:06:53.360289] [DEBUG] [LOCAL ] CGV: Derivative data saved (1794 records). [2025-10-20T22:06:53.364567] [INFO ] [MAIN ] CHCT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:53.375177] [INFO ] [MAIN ] CGV: Successfully saved derivative data. [2025-10-20T22:06:53.377488] [INFO ] [MAIN ] Derivative Progress: 353/1877 symbols completed. CGV took 1.0s. Estimated remaining: 9m 35s. Rate: 159.0 symbols/min [2025-10-20T22:06:53.381138] [INFO ] [MAIN ] CHD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:53.382138] [DEBUG] [LOCAL ] CGAU: Derivative data saved (1794 records). [2025-10-20T22:06:53.382663] [INFO ] [MAIN ] CGAU: Successfully saved derivative data. [2025-10-20T22:06:53.384837] [INFO ] [MAIN ] Derivative Progress: 354/1877 symbols completed. CGAU took 1.1s. Estimated remaining: 9m 33s. Rate: 159.5 symbols/min [2025-10-20T22:06:53.385150] [INFO ] [MAIN ] CHE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:53.409129] [DEBUG] [LOCAL ] CHCT: Loaded 24343 existing records from primary storage. [2025-10-20T22:06:53.409355] [INFO ] [MAIN ] CHCT: Loaded 24343 primary data records. [2025-10-20T22:06:53.457364] [DEBUG] [LOCAL ] CHE: Loaded 15304 existing records from primary storage. [2025-10-20T22:06:53.478118] [INFO ] [MAIN ] CHE: Loaded 15304 primary data records. [2025-10-20T22:06:53.671734] [DEBUG] [LOCAL ] CHD: Loaded 52329 existing records from primary storage. [2025-10-20T22:06:53.712826] [INFO ] [MAIN ] CHD: Loaded 52329 primary data records. [2025-10-20T22:06:54.387329] [DEBUG] [MAIN ] CHCT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:54.388170] [INFO ] [MAIN ] CHCT: Generated 1794 derivative records from 24343 primary records. [2025-10-20T22:06:54.388422] [DEBUG] [LOCAL ] CHCT: Saving derivative data. [2025-10-20T22:06:54.407025] [DEBUG] [LOCAL ] CHCT: Derivative data saved (1794 records). [2025-10-20T22:06:54.407226] [INFO ] [MAIN ] CHCT: Successfully saved derivative data. [2025-10-20T22:06:54.409307] [INFO ] [MAIN ] Derivative Progress: 355/1877 symbols completed. CHCT took 1.0s. Estimated remaining: 9m 35s. Rate: 158.7 symbols/min [2025-10-20T22:06:54.409554] [INFO ] [MAIN ] CHGG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:54.422970] [DEBUG] [MAIN ] CHE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:54.424525] [INFO ] [MAIN ] CHE: Generated 1794 derivative records from 15304 primary records. [2025-10-20T22:06:54.424704] [DEBUG] [LOCAL ] CHE: Saving derivative data. [2025-10-20T22:06:54.449120] [DEBUG] [LOCAL ] CHE: Derivative data saved (1794 records). [2025-10-20T22:06:54.449266] [INFO ] [MAIN ] CHE: Successfully saved derivative data. [2025-10-20T22:06:54.456000] [INFO ] [MAIN ] Derivative Progress: 356/1877 symbols completed. CHE took 1.1s. Estimated remaining: 9m 34s. Rate: 159.1 symbols/min [2025-10-20T22:06:54.456539] [DEBUG] [MAIN ] CHD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:54.457492] [INFO ] [MAIN ] CHH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:54.457390] [INFO ] [MAIN ] CHD: Generated 1794 derivative records from 52329 primary records. [2025-10-20T22:06:54.457705] [DEBUG] [LOCAL ] CHD: Saving derivative data. [2025-10-20T22:06:54.474522] [DEBUG] [LOCAL ] CHD: Derivative data saved (1794 records). [2025-10-20T22:06:54.474800] [INFO ] [MAIN ] CHD: Successfully saved derivative data. [2025-10-20T22:06:54.476927] [INFO ] [MAIN ] Derivative Progress: 357/1877 symbols completed. CHD took 1.1s. Estimated remaining: 9m 32s. Rate: 159.5 symbols/min [2025-10-20T22:06:54.486988] [INFO ] [MAIN ] CHMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:54.519103] [DEBUG] [LOCAL ] CHGG: Loaded 47673 existing records from primary storage. [2025-10-20T22:06:54.519711] [INFO ] [MAIN ] CHGG: Loaded 47673 primary data records. [2025-10-20T22:06:54.586031] [DEBUG] [LOCAL ] CHH: Loaded 32329 existing records from primary storage. [2025-10-20T22:06:54.603008] [INFO ] [MAIN ] CHH: Loaded 32329 primary data records. [2025-10-20T22:06:54.585600] [DEBUG] [LOCAL ] CHMI: Loaded 21466 existing records from primary storage. [2025-10-20T22:06:54.631776] [INFO ] [MAIN ] CHMI: Loaded 21466 primary data records. [2025-10-20T22:06:55.440989] [DEBUG] [MAIN ] CHGG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:55.441969] [INFO ] [MAIN ] CHGG: Generated 1794 derivative records from 47673 primary records. [2025-10-20T22:06:55.442136] [DEBUG] [LOCAL ] CHGG: Saving derivative data. [2025-10-20T22:06:55.468798] [DEBUG] [LOCAL ] CHGG: Derivative data saved (1794 records). [2025-10-20T22:06:55.474277] [INFO ] [MAIN ] CHGG: Successfully saved derivative data. [2025-10-20T22:06:55.476670] [INFO ] [MAIN ] Derivative Progress: 358/1877 symbols completed. CHGG took 1.1s. Estimated remaining: 9m 34s. Rate: 158.8 symbols/min [2025-10-20T22:06:55.477043] [DEBUG] [MAIN ] CHMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:55.477988] [INFO ] [MAIN ] CHMI: Generated 1794 derivative records from 21466 primary records. [2025-10-20T22:06:55.478191] [DEBUG] [LOCAL ] CHMI: Saving derivative data. [2025-10-20T22:06:55.501801] [DEBUG] [MAIN ] CHH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:55.503386] [INFO ] [MAIN ] CHH: Generated 1794 derivative records from 32329 primary records. [2025-10-20T22:06:55.504830] [DEBUG] [LOCAL ] CHH: Saving derivative data. [2025-10-20T22:06:55.506815] [DEBUG] [LOCAL ] CHMI: Derivative data saved (1794 records). [2025-10-20T22:06:55.507030] [INFO ] [MAIN ] CHMI: Successfully saved derivative data. [2025-10-20T22:06:55.509045] [INFO ] [MAIN ] Derivative Progress: 359/1877 symbols completed. CHMI took 1.0s. Estimated remaining: 9m 32s. Rate: 159.2 symbols/min [2025-10-20T22:06:55.523445] [DEBUG] [LOCAL ] CHH: Derivative data saved (1794 records). [2025-10-20T22:06:55.523501] [INFO ] [MAIN ] CHH: Successfully saved derivative data. [2025-10-20T22:06:55.525903] [INFO ] [MAIN ] Derivative Progress: 360/1877 symbols completed. CHH took 1.1s. Estimated remaining: 9m 30s. Rate: 159.6 symbols/min [2025-10-20T22:06:55.526473] [INFO ] [MAIN ] CHPT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:55.526583] [INFO ] [MAIN ] CHWY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:55.526420] [INFO ] [MAIN ] CHN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:55.535385] [DEBUG] [LOCAL ] CHN: Loaded 5002 existing records from primary storage. [2025-10-20T22:06:55.535759] [INFO ] [MAIN ] CHN: Loaded 5002 primary data records. [2025-10-20T22:06:55.878933] [DEBUG] [LOCAL ] CHPT: Loaded 47293 existing records from primary storage. [2025-10-20T22:06:55.894568] [INFO ] [MAIN ] CHPT: Loaded 47293 primary data records. [2025-10-20T22:06:55.906474] [DEBUG] [LOCAL ] CHWY: Loaded 53656 existing records from primary storage. [2025-10-20T22:06:55.916600] [INFO ] [MAIN ] CHWY: Loaded 53656 primary data records. [2025-10-20T22:06:56.549755] [DEBUG] [MAIN ] CHN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:56.551133] [INFO ] [MAIN ] CHN: Generated 1794 derivative records from 5002 primary records. [2025-10-20T22:06:56.552800] [DEBUG] [LOCAL ] CHN: Saving derivative data. [2025-10-20T22:06:56.575869] [DEBUG] [LOCAL ] CHN: Derivative data saved (1794 records). [2025-10-20T22:06:56.576264] [INFO ] [MAIN ] CHN: Successfully saved derivative data. [2025-10-20T22:06:56.579396] [INFO ] [MAIN ] Derivative Progress: 361/1877 symbols completed. CHN took 1.1s. Estimated remaining: 9m 33s. Rate: 158.8 symbols/min [2025-10-20T22:06:56.579975] [INFO ] [MAIN ] CI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:56.598048] [DEBUG] [MAIN ] CHWY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:56.601289] [INFO ] [MAIN ] CHWY: Generated 1794 derivative records from 53656 primary records. [2025-10-20T22:06:56.601495] [DEBUG] [LOCAL ] CHWY: Saving derivative data. [2025-10-20T22:06:56.630280] [DEBUG] [LOCAL ] CHWY: Derivative data saved (1794 records). [2025-10-20T22:06:56.630474] [INFO ] [MAIN ] CHWY: Successfully saved derivative data. [2025-10-20T22:06:56.632786] [INFO ] [MAIN ] Derivative Progress: 362/1877 symbols completed. CHWY took 1.1s. Estimated remaining: 9m 31s. Rate: 159.2 symbols/min [2025-10-20T22:06:56.632928] [INFO ] [MAIN ] CIA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:56.626182] [DEBUG] [MAIN ] CHPT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:56.634012] [INFO ] [MAIN ] CHPT: Generated 1794 derivative records from 47293 primary records. [2025-10-20T22:06:56.634346] [DEBUG] [LOCAL ] CHPT: Saving derivative data. [2025-10-20T22:06:56.650678] [DEBUG] [LOCAL ] CHPT: Derivative data saved (1794 records). [2025-10-20T22:06:56.650931] [INFO ] [MAIN ] CHPT: Successfully saved derivative data. [2025-10-20T22:06:56.653190] [INFO ] [MAIN ] Derivative Progress: 363/1877 symbols completed. CHPT took 1.1s. Estimated remaining: 9m 29s. Rate: 159.6 symbols/min [2025-10-20T22:06:56.657700] [INFO ] [MAIN ] CIB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:56.675676] [DEBUG] [LOCAL ] CIA: Loaded 16550 existing records from primary storage. [2025-10-20T22:06:56.675798] [INFO ] [MAIN ] CIA: Loaded 16550 primary data records. [2025-10-20T22:06:56.771470] [DEBUG] [LOCAL ] CI: Loaded 49691 existing records from primary storage. [2025-10-20T22:06:56.790786] [INFO ] [MAIN ] CI: Loaded 49691 primary data records. [2025-10-20T22:06:56.912616] [DEBUG] [LOCAL ] CIB: Loaded 30372 existing records from primary storage. [2025-10-20T22:06:56.961331] [INFO ] [MAIN ] CIB: Loaded 30372 primary data records. [2025-10-20T22:06:57.622058] [DEBUG] [MAIN ] CIA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:57.626750] [INFO ] [MAIN ] CIA: Generated 1794 derivative records from 16550 primary records. [2025-10-20T22:06:57.627048] [DEBUG] [LOCAL ] CIA: Saving derivative data. [2025-10-20T22:06:57.655674] [DEBUG] [MAIN ] CI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:57.656410] [INFO ] [MAIN ] CI: Generated 1794 derivative records from 49691 primary records. [2025-10-20T22:06:57.656615] [DEBUG] [LOCAL ] CI: Saving derivative data. [2025-10-20T22:06:57.655733] [DEBUG] [LOCAL ] CIA: Derivative data saved (1794 records). [2025-10-20T22:06:57.657126] [INFO ] [MAIN ] CIA: Successfully saved derivative data. [2025-10-20T22:06:57.658972] [INFO ] [MAIN ] Derivative Progress: 364/1877 symbols completed. CIA took 1.0s. Estimated remaining: 9m 31s. Rate: 158.9 symbols/min [2025-10-20T22:06:57.659622] [INFO ] [MAIN ] CIEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:57.678171] [DEBUG] [LOCAL ] CI: Derivative data saved (1794 records). [2025-10-20T22:06:57.678507] [INFO ] [MAIN ] CI: Successfully saved derivative data. [2025-10-20T22:06:57.680949] [INFO ] [MAIN ] Derivative Progress: 365/1877 symbols completed. CI took 1.1s. Estimated remaining: 9m 30s. Rate: 159.3 symbols/min [2025-10-20T22:06:57.693929] [INFO ] [MAIN ] CIF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:57.693818] [DEBUG] [MAIN ] CIB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:57.694997] [INFO ] [MAIN ] CIB: Generated 1794 derivative records from 30372 primary records. [2025-10-20T22:06:57.695249] [DEBUG] [LOCAL ] CIB: Saving derivative data. [2025-10-20T22:06:57.711954] [DEBUG] [LOCAL ] CIB: Derivative data saved (1794 records). [2025-10-20T22:06:57.712140] [INFO ] [MAIN ] CIB: Successfully saved derivative data. [2025-10-20T22:06:57.714825] [INFO ] [MAIN ] Derivative Progress: 366/1877 symbols completed. CIB took 1.1s. Estimated remaining: 9m 28s. Rate: 159.7 symbols/min [2025-10-20T22:06:57.715065] [INFO ] [MAIN ] CIG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:57.718244] [DEBUG] [LOCAL ] CIF: Loaded 4025 existing records from primary storage. [2025-10-20T22:06:57.718776] [INFO ] [MAIN ] CIF: Loaded 4025 primary data records. [2025-10-20T22:06:58.068690] [DEBUG] [LOCAL ] CIEN: Loaded 52129 existing records from primary storage. [2025-10-20T22:06:58.068934] [INFO ] [MAIN ] CIEN: Loaded 52129 primary data records. [2025-10-20T22:06:58.186436] [DEBUG] [LOCAL ] CIG: Loaded 44324 existing records from primary storage. [2025-10-20T22:06:58.197354] [INFO ] [MAIN ] CIG: Loaded 44324 primary data records. [2025-10-20T22:06:58.683834] [DEBUG] [MAIN ] CIF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:58.685485] [INFO ] [MAIN ] CIF: Generated 1794 derivative records from 4025 primary records. [2025-10-20T22:06:58.686000] [DEBUG] [LOCAL ] CIF: Saving derivative data. [2025-10-20T22:06:58.700857] [DEBUG] [LOCAL ] CIF: Derivative data saved (1794 records). [2025-10-20T22:06:58.700964] [INFO ] [MAIN ] CIF: Successfully saved derivative data. [2025-10-20T22:06:58.703561] [INFO ] [MAIN ] Derivative Progress: 367/1877 symbols completed. CIF took 1.0s. Estimated remaining: 9m 30s. Rate: 159.0 symbols/min [2025-10-20T22:06:58.704887] [INFO ] [MAIN ] CII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:58.721788] [DEBUG] [MAIN ] CIEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:58.722636] [INFO ] [MAIN ] CIEN: Generated 1794 derivative records from 52129 primary records. [2025-10-20T22:06:58.722981] [DEBUG] [LOCAL ] CIEN: Saving derivative data. [2025-10-20T22:06:58.750258] [DEBUG] [LOCAL ] CII: Loaded 15328 existing records from primary storage. [2025-10-20T22:06:58.750449] [INFO ] [MAIN ] CII: Loaded 15328 primary data records. [2025-10-20T22:06:58.744834] [DEBUG] [LOCAL ] CIEN: Derivative data saved (1794 records). [2025-10-20T22:06:58.750661] [INFO ] [MAIN ] CIEN: Successfully saved derivative data. [2025-10-20T22:06:58.763575] [INFO ] [MAIN ] Derivative Progress: 368/1877 symbols completed. CIEN took 1.1s. Estimated remaining: 9m 28s. Rate: 159.3 symbols/min [2025-10-20T22:06:58.763814] [INFO ] [MAIN ] CIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:58.804636] [DEBUG] [MAIN ] CIG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:58.811345] [INFO ] [MAIN ] CIG: Generated 1794 derivative records from 44324 primary records. [2025-10-20T22:06:58.820849] [DEBUG] [LOCAL ] CIG: Saving derivative data. [2025-10-20T22:06:58.893387] [DEBUG] [LOCAL ] CIG: Derivative data saved (1794 records). [2025-10-20T22:06:58.930165] [INFO ] [MAIN ] CIG: Successfully saved derivative data. [2025-10-20T22:06:58.960403] [INFO ] [MAIN ] Derivative Progress: 369/1877 symbols completed. CIG took 1.2s. Estimated remaining: 9m 27s. Rate: 159.5 symbols/min [2025-10-20T22:06:58.969020] [INFO ] [MAIN ] CIMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:59.083794] [DEBUG] [LOCAL ] CIMN: Loaded 2756 existing records from primary storage. [2025-10-20T22:06:59.083925] [INFO ] [MAIN ] CIMN: Loaded 2756 primary data records. [2025-10-20T22:06:59.188563] [DEBUG] [LOCAL ] CIM: Loaded 37626 existing records from primary storage. [2025-10-20T22:06:59.199992] [INFO ] [MAIN ] CIM: Loaded 37626 primary data records. [2025-10-20T22:06:59.747995] [DEBUG] [MAIN ] CII: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:59.748780] [INFO ] [MAIN ] CII: Generated 1794 derivative records from 15328 primary records. [2025-10-20T22:06:59.749738] [DEBUG] [LOCAL ] CII: Saving derivative data. [2025-10-20T22:06:59.767291] [DEBUG] [LOCAL ] CII: Derivative data saved (1794 records). [2025-10-20T22:06:59.767516] [INFO ] [MAIN ] CII: Successfully saved derivative data. [2025-10-20T22:06:59.769638] [INFO ] [MAIN ] Derivative Progress: 370/1877 symbols completed. CII took 1.1s. Estimated remaining: 9m 28s. Rate: 159.1 symbols/min [2025-10-20T22:06:59.770023] [INFO ] [MAIN ] CINT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:59.786906] [DEBUG] [MAIN ] CIMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:59.787580] [INFO ] [MAIN ] CIMN: Generated 1794 derivative records from 2756 primary records. [2025-10-20T22:06:59.787874] [DEBUG] [LOCAL ] CIMN: Saving derivative data. [2025-10-20T22:06:59.808698] [DEBUG] [LOCAL ] CIMN: Derivative data saved (1794 records). [2025-10-20T22:06:59.808811] [INFO ] [MAIN ] CIMN: Successfully saved derivative data. [2025-10-20T22:06:59.810849] [INFO ] [MAIN ] Derivative Progress: 371/1877 symbols completed. CIMN took 0.8s. Estimated remaining: 9m 27s. Rate: 159.4 symbols/min [2025-10-20T22:06:59.811342] [INFO ] [MAIN ] CIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:06:59.823466] [DEBUG] [LOCAL ] CINT: Loaded 20229 existing records from primary storage. [2025-10-20T22:06:59.823913] [INFO ] [MAIN ] CINT: Loaded 20229 primary data records. [2025-10-20T22:06:59.836792] [DEBUG] [MAIN ] CIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:06:59.879859] [INFO ] [MAIN ] CIM: Generated 1794 derivative records from 37626 primary records. [2025-10-20T22:06:59.896595] [DEBUG] [LOCAL ] CIM: Saving derivative data. [2025-10-20T22:06:59.945555] [DEBUG] [LOCAL ] CIM: Derivative data saved (1794 records). [2025-10-20T22:06:59.956731] [DEBUG] [LOCAL ] CIO: Loaded 21090 existing records from primary storage. [2025-10-20T22:06:59.956843] [INFO ] [MAIN ] CIO: Loaded 21090 primary data records. [2025-10-20T22:06:59.950851] [INFO ] [MAIN ] CIM: Successfully saved derivative data. [2025-10-20T22:06:59.959485] [INFO ] [MAIN ] Derivative Progress: 372/1877 symbols completed. CIM took 1.2s. Estimated remaining: 9m 25s. Rate: 159.7 symbols/min [2025-10-20T22:07:00.007085] [INFO ] [MAIN ] CION: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:00.480195] [DEBUG] [LOCAL ] CION: Loaded 28316 existing records from primary storage. [2025-10-20T22:07:00.482289] [INFO ] [MAIN ] CION: Loaded 28316 primary data records. [2025-10-20T22:07:00.803328] [DEBUG] [MAIN ] CINT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:00.805626] [INFO ] [MAIN ] CINT: Generated 1793 derivative records from 20229 primary records. [2025-10-20T22:07:00.805801] [DEBUG] [LOCAL ] CINT: Saving derivative data. [2025-10-20T22:07:00.821305] [DEBUG] [MAIN ] CIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:00.822440] [INFO ] [MAIN ] CIO: Generated 1794 derivative records from 21090 primary records. [2025-10-20T22:07:00.829283] [DEBUG] [LOCAL ] CIO: Saving derivative data. [2025-10-20T22:07:00.837158] [DEBUG] [LOCAL ] CINT: Derivative data saved (1793 records). [2025-10-20T22:07:00.837999] [INFO ] [MAIN ] CINT: Successfully saved derivative data. [2025-10-20T22:07:00.847024] [INFO ] [MAIN ] Derivative Progress: 373/1877 symbols completed. CINT took 1.1s. Estimated remaining: 9m 27s. Rate: 159.1 symbols/min [2025-10-20T22:07:00.856001] [DEBUG] [LOCAL ] CIO: Derivative data saved (1794 records). [2025-10-20T22:07:00.856110] [INFO ] [MAIN ] CIO: Successfully saved derivative data. [2025-10-20T22:07:00.858515] [INFO ] [MAIN ] Derivative Progress: 374/1877 symbols completed. CIO took 1.0s. Estimated remaining: 9m 25s. Rate: 159.5 symbols/min [2025-10-20T22:07:00.876290] [DEBUG] [MAIN ] CION: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:00.876874] [INFO ] [MAIN ] CION: Generated 1794 derivative records from 28316 primary records. [2025-10-20T22:07:00.876941] [DEBUG] [LOCAL ] CION: Saving derivative data. [2025-10-20T22:07:00.891738] [DEBUG] [LOCAL ] CION: Derivative data saved (1794 records). [2025-10-20T22:07:00.891797] [INFO ] [MAIN ] CION: Successfully saved derivative data. [2025-10-20T22:07:00.893962] [INFO ] [MAIN ] Derivative Progress: 375/1877 symbols completed. CION took 0.9s. Estimated remaining: 9m 24s. Rate: 159.9 symbols/min [2025-10-20T22:07:00.894467] [INFO ] [MAIN ] CL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:00.894397] [INFO ] [MAIN ] CIVI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:00.894626] [INFO ] [MAIN ] CLB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:00.953324] [DEBUG] [LOCAL ] CLB: Loaded 33606 existing records from primary storage. [2025-10-20T22:07:00.953599] [INFO ] [MAIN ] CLB: Loaded 33606 primary data records. [2025-10-20T22:07:01.090545] [DEBUG] [LOCAL ] CIVI: Loaded 52736 existing records from primary storage. [2025-10-20T22:07:01.112435] [DEBUG] [LOCAL ] CL: Loaded 53649 existing records from primary storage. [2025-10-20T22:07:01.130713] [INFO ] [MAIN ] CIVI: Loaded 52736 primary data records. [2025-10-20T22:07:01.130979] [INFO ] [MAIN ] CL: Loaded 53649 primary data records. [2025-10-20T22:07:01.956234] [DEBUG] [MAIN ] CLB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:01.957275] [INFO ] [MAIN ] CLB: Generated 1794 derivative records from 33606 primary records. [2025-10-20T22:07:01.957447] [DEBUG] [LOCAL ] CLB: Saving derivative data. [2025-10-20T22:07:01.974755] [DEBUG] [LOCAL ] CLB: Derivative data saved (1794 records). [2025-10-20T22:07:01.974836] [INFO ] [MAIN ] CLB: Successfully saved derivative data. [2025-10-20T22:07:01.976508] [INFO ] [MAIN ] Derivative Progress: 376/1877 symbols completed. CLB took 1.1s. Estimated remaining: 9m 26s. Rate: 159.1 symbols/min [2025-10-20T22:07:01.976776] [INFO ] [MAIN ] CLCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:02.009012] [DEBUG] [MAIN ] CIVI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:02.010121] [INFO ] [MAIN ] CIVI: Generated 1794 derivative records from 52736 primary records. [2025-10-20T22:07:02.014146] [DEBUG] [MAIN ] CL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:02.014953] [INFO ] [MAIN ] CL: Generated 1794 derivative records from 53649 primary records. [2025-10-20T22:07:02.014284] [DEBUG] [LOCAL ] CIVI: Saving derivative data. [2025-10-20T22:07:02.015107] [DEBUG] [LOCAL ] CL: Saving derivative data. [2025-10-20T22:07:02.047076] [DEBUG] [LOCAL ] CL: Derivative data saved (1794 records). [2025-10-20T22:07:02.049058] [INFO ] [MAIN ] CL: Successfully saved derivative data. [2025-10-20T22:07:02.049422] [DEBUG] [LOCAL ] CLCO: Loaded 13189 existing records from primary storage. [2025-10-20T22:07:02.051748] [DEBUG] [LOCAL ] CIVI: Derivative data saved (1794 records). [2025-10-20T22:07:02.053993] [INFO ] [MAIN ] Derivative Progress: 377/1877 symbols completed. CL took 1.2s. Estimated remaining: 9m 24s. Rate: 159.5 symbols/min [2025-10-20T22:07:02.054328] [INFO ] [MAIN ] CLDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:02.054132] [INFO ] [MAIN ] CIVI: Successfully saved derivative data. [2025-10-20T22:07:02.054254] [INFO ] [MAIN ] CLCO: Loaded 13189 primary data records. [2025-10-20T22:07:02.069391] [INFO ] [MAIN ] Derivative Progress: 378/1877 symbols completed. CIVI took 1.2s. Estimated remaining: 9m 23s. Rate: 159.9 symbols/min [2025-10-20T22:07:02.079366] [INFO ] [MAIN ] CLF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:02.263228] [DEBUG] [LOCAL ] CLDT: Loaded 25365 existing records from primary storage. [2025-10-20T22:07:02.377750] [INFO ] [MAIN ] CLDT: Loaded 25365 primary data records. [2025-10-20T22:07:02.556753] [DEBUG] [LOCAL ] CLF: Loaded 53741 existing records from primary storage. [2025-10-20T22:07:02.585727] [INFO ] [MAIN ] CLF: Loaded 53741 primary data records. [2025-10-20T22:07:03.066256] [DEBUG] [MAIN ] CLCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:03.078030] [INFO ] [MAIN ] CLCO: Generated 1794 derivative records from 13189 primary records. [2025-10-20T22:07:03.078336] [DEBUG] [LOCAL ] CLCO: Saving derivative data. [2025-10-20T22:07:03.100497] [DEBUG] [LOCAL ] CLCO: Derivative data saved (1794 records). [2025-10-20T22:07:03.102975] [INFO ] [MAIN ] CLCO: Successfully saved derivative data. [2025-10-20T22:07:03.105073] [INFO ] [MAIN ] Derivative Progress: 379/1877 symbols completed. CLCO took 1.1s. Estimated remaining: 9m 25s. Rate: 159.1 symbols/min [2025-10-20T22:07:03.106585] [INFO ] [MAIN ] CLH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:03.123811] [DEBUG] [MAIN ] CLDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:03.124987] [INFO ] [MAIN ] CLDT: Generated 1794 derivative records from 25365 primary records. [2025-10-20T22:07:03.125273] [DEBUG] [LOCAL ] CLDT: Saving derivative data. [2025-10-20T22:07:03.143234] [DEBUG] [LOCAL ] CLDT: Derivative data saved (1794 records). [2025-10-20T22:07:03.143494] [INFO ] [MAIN ] CLDT: Successfully saved derivative data. [2025-10-20T22:07:03.145813] [INFO ] [MAIN ] Derivative Progress: 380/1877 symbols completed. CLDT took 1.1s. Estimated remaining: 9m 23s. Rate: 159.5 symbols/min [2025-10-20T22:07:03.147039] [INFO ] [MAIN ] CLPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:03.172320] [DEBUG] [MAIN ] CLF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:03.176160] [INFO ] [MAIN ] CLF: Generated 1794 derivative records from 53741 primary records. [2025-10-20T22:07:03.176313] [DEBUG] [LOCAL ] CLF: Saving derivative data. [2025-10-20T22:07:03.201524] [DEBUG] [LOCAL ] CLH: Loaded 27808 existing records from primary storage. [2025-10-20T22:07:03.206733] [DEBUG] [LOCAL ] CLF: Derivative data saved (1794 records). [2025-10-20T22:07:03.206945] [INFO ] [MAIN ] CLH: Loaded 27808 primary data records. [2025-10-20T22:07:03.207071] [INFO ] [MAIN ] CLF: Successfully saved derivative data. [2025-10-20T22:07:03.209518] [INFO ] [MAIN ] Derivative Progress: 381/1877 symbols completed. CLF took 1.1s. Estimated remaining: 9m 22s. Rate: 159.8 symbols/min [2025-10-20T22:07:03.209746] [INFO ] [MAIN ] CLS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:03.222008] [DEBUG] [LOCAL ] CLPR: Loaded 10162 existing records from primary storage. [2025-10-20T22:07:03.232401] [INFO ] [MAIN ] CLPR: Loaded 10162 primary data records. [2025-10-20T22:07:03.905422] [DEBUG] [LOCAL ] CLS: Loaded 53569 existing records from primary storage. [2025-10-20T22:07:03.906390] [INFO ] [MAIN ] CLS: Loaded 53569 primary data records. [2025-10-20T22:07:04.133034] [DEBUG] [MAIN ] CLH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:04.269432] [DEBUG] [MAIN ] CLPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:04.268840] [INFO ] [MAIN ] CLH: Generated 1794 derivative records from 27808 primary records. [2025-10-20T22:07:04.269605] [DEBUG] [LOCAL ] CLH: Saving derivative data. [2025-10-20T22:07:04.271235] [INFO ] [MAIN ] CLPR: Generated 1794 derivative records from 10162 primary records. [2025-10-20T22:07:04.272115] [DEBUG] [LOCAL ] CLPR: Saving derivative data. [2025-10-20T22:07:04.307093] [DEBUG] [LOCAL ] CLPR: Derivative data saved (1794 records). [2025-10-20T22:07:04.307320] [INFO ] [MAIN ] CLPR: Successfully saved derivative data. [2025-10-20T22:07:04.310218] [INFO ] [MAIN ] Derivative Progress: 382/1877 symbols completed. CLPR took 1.2s. Estimated remaining: 9m 24s. Rate: 159.0 symbols/min [2025-10-20T22:07:04.311868] [DEBUG] [LOCAL ] CLH: Derivative data saved (1794 records). [2025-10-20T22:07:04.312120] [INFO ] [MAIN ] CLVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:04.312193] [INFO ] [MAIN ] CLH: Successfully saved derivative data. [2025-10-20T22:07:04.315878] [INFO ] [MAIN ] Derivative Progress: 383/1877 symbols completed. CLH took 1.2s. Estimated remaining: 9m 22s. Rate: 159.4 symbols/min [2025-10-20T22:07:04.317690] [INFO ] [MAIN ] CLW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:04.347006] [DEBUG] [MAIN ] CLS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:04.348004] [INFO ] [MAIN ] CLS: Generated 1794 derivative records from 53569 primary records. [2025-10-20T22:07:04.355297] [DEBUG] [LOCAL ] CLS: Saving derivative data. [2025-10-20T22:07:04.357870] [DEBUG] [LOCAL ] CLW: Loaded 21460 existing records from primary storage. [2025-10-20T22:07:04.365899] [INFO ] [MAIN ] CLW: Loaded 21460 primary data records. [2025-10-20T22:07:04.431155] [DEBUG] [LOCAL ] CLS: Derivative data saved (1794 records). [2025-10-20T22:07:04.443415] [INFO ] [MAIN ] CLS: Successfully saved derivative data. [2025-10-20T22:07:04.446159] [INFO ] [MAIN ] Derivative Progress: 384/1877 symbols completed. CLS took 1.2s. Estimated remaining: 9m 21s. Rate: 159.7 symbols/min [2025-10-20T22:07:04.455144] [INFO ] [MAIN ] CLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:04.513898] [DEBUG] [LOCAL ] CLVT: Loaded 51923 existing records from primary storage. [2025-10-20T22:07:04.514542] [INFO ] [MAIN ] CLVT: Loaded 51923 primary data records. [2025-10-20T22:07:05.049238] [DEBUG] [LOCAL ] CLX: Loaded 51778 existing records from primary storage. [2025-10-20T22:07:05.049833] [INFO ] [MAIN ] CLX: Loaded 51778 primary data records. [2025-10-20T22:07:05.278026] [DEBUG] [MAIN ] CLW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:05.282213] [INFO ] [MAIN ] CLW: Generated 1794 derivative records from 21460 primary records. [2025-10-20T22:07:05.282443] [DEBUG] [LOCAL ] CLW: Saving derivative data. [2025-10-20T22:07:05.425487] [DEBUG] [LOCAL ] CLW: Derivative data saved (1794 records). [2025-10-20T22:07:05.426672] [INFO ] [MAIN ] CLW: Successfully saved derivative data. [2025-10-20T22:07:05.431084] [INFO ] [MAIN ] Derivative Progress: 385/1877 symbols completed. CLW took 1.1s. Estimated remaining: 9m 23s. Rate: 159.1 symbols/min [2025-10-20T22:07:05.431700] [INFO ] [MAIN ] CM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:05.446975] [DEBUG] [MAIN ] CLVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:05.447862] [INFO ] [MAIN ] CLVT: Generated 1794 derivative records from 51923 primary records. [2025-10-20T22:07:05.448111] [DEBUG] [LOCAL ] CLVT: Saving derivative data. [2025-10-20T22:07:05.464946] [DEBUG] [LOCAL ] CLVT: Derivative data saved (1794 records). [2025-10-20T22:07:05.465174] [INFO ] [MAIN ] CLVT: Successfully saved derivative data. [2025-10-20T22:07:05.467583] [INFO ] [MAIN ] Derivative Progress: 386/1877 symbols completed. CLVT took 1.2s. Estimated remaining: 9m 21s. Rate: 159.4 symbols/min [2025-10-20T22:07:05.467828] [INFO ] [MAIN ] CMA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:05.484993] [DEBUG] [MAIN ] CLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:05.485794] [INFO ] [MAIN ] CLX: Generated 1794 derivative records from 51778 primary records. [2025-10-20T22:07:05.485910] [DEBUG] [LOCAL ] CLX: Saving derivative data. [2025-10-20T22:07:05.512861] [DEBUG] [LOCAL ] CLX: Derivative data saved (1794 records). [2025-10-20T22:07:05.513140] [INFO ] [MAIN ] CLX: Successfully saved derivative data. [2025-10-20T22:07:05.515390] [INFO ] [MAIN ] Derivative Progress: 387/1877 symbols completed. CLX took 1.1s. Estimated remaining: 9m 20s. Rate: 159.8 symbols/min [2025-10-20T22:07:05.516828] [INFO ] [MAIN ] CMBT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:05.534775] [DEBUG] [LOCAL ] CM: Loaded 48575 existing records from primary storage. [2025-10-20T22:07:05.535046] [INFO ] [MAIN ] CM: Loaded 48575 primary data records. [2025-10-20T22:07:05.765467] [DEBUG] [LOCAL ] CMBT: Loaded 27012 existing records from primary storage. [2025-10-20T22:07:05.894634] [INFO ] [MAIN ] CMBT: Loaded 27012 primary data records. [2025-10-20T22:07:05.909219] [DEBUG] [LOCAL ] CMA: Loaded 50177 existing records from primary storage. [2025-10-20T22:07:05.940210] [INFO ] [MAIN ] CMA: Loaded 50177 primary data records. [2025-10-20T22:07:06.550970] [DEBUG] [MAIN ] CM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:06.552584] [INFO ] [MAIN ] CM: Generated 1794 derivative records from 48575 primary records. [2025-10-20T22:07:06.552746] [DEBUG] [LOCAL ] CM: Saving derivative data. [2025-10-20T22:07:06.570369] [DEBUG] [LOCAL ] CM: Derivative data saved (1794 records). [2025-10-20T22:07:06.570893] [INFO ] [MAIN ] CM: Successfully saved derivative data. [2025-10-20T22:07:06.572890] [INFO ] [MAIN ] Derivative Progress: 388/1877 symbols completed. CM took 1.1s. Estimated remaining: 9m 22s. Rate: 159.0 symbols/min [2025-10-20T22:07:06.590577] [DEBUG] [MAIN ] CMBT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:06.591581] [INFO ] [MAIN ] CMBT: Generated 1794 derivative records from 27012 primary records. [2025-10-20T22:07:06.591706] [DEBUG] [LOCAL ] CMBT: Saving derivative data. [2025-10-20T22:07:06.609680] [DEBUG] [LOCAL ] CMBT: Derivative data saved (1794 records). [2025-10-20T22:07:06.610380] [INFO ] [MAIN ] CMBT: Successfully saved derivative data. [2025-10-20T22:07:06.612400] [INFO ] [MAIN ] Derivative Progress: 389/1877 symbols completed. CMBT took 1.1s. Estimated remaining: 9m 20s. Rate: 159.4 symbols/min [2025-10-20T22:07:06.625263] [DEBUG] [MAIN ] CMA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:06.625970] [INFO ] [MAIN ] CMA: Generated 1794 derivative records from 50177 primary records. [2025-10-20T22:07:06.626032] [DEBUG] [LOCAL ] CMA: Saving derivative data. [2025-10-20T22:07:06.641288] [DEBUG] [LOCAL ] CMA: Derivative data saved (1794 records). [2025-10-20T22:07:06.641348] [INFO ] [MAIN ] CMA: Successfully saved derivative data. [2025-10-20T22:07:06.643688] [INFO ] [MAIN ] Derivative Progress: 390/1877 symbols completed. CMA took 1.2s. Estimated remaining: 9m 18s. Rate: 159.8 symbols/min [2025-10-20T22:07:06.644149] [INFO ] [MAIN ] CMCM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:06.644262] [INFO ] [MAIN ] CMP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:06.644081] [INFO ] [MAIN ] CMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:06.651990] [DEBUG] [LOCAL ] CMCM: Loaded 4827 existing records from primary storage. [2025-10-20T22:07:06.652477] [INFO ] [MAIN ] CMCM: Loaded 4827 primary data records. [2025-10-20T22:07:06.974902] [DEBUG] [LOCAL ] CMP: Loaded 35799 existing records from primary storage. [2025-10-20T22:07:06.975381] [INFO ] [MAIN ] CMP: Loaded 35799 primary data records. [2025-10-20T22:07:07.025700] [DEBUG] [LOCAL ] CMC: Loaded 45569 existing records from primary storage. [2025-10-20T22:07:07.038743] [INFO ] [MAIN ] CMC: Loaded 45569 primary data records. [2025-10-20T22:07:07.678452] [DEBUG] [MAIN ] CMCM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:07.680010] [INFO ] [MAIN ] CMCM: Generated 1794 derivative records from 4827 primary records. [2025-10-20T22:07:07.680189] [DEBUG] [LOCAL ] CMCM: Saving derivative data. [2025-10-20T22:07:07.695148] [DEBUG] [LOCAL ] CMCM: Derivative data saved (1794 records). [2025-10-20T22:07:07.695379] [INFO ] [MAIN ] CMCM: Successfully saved derivative data. [2025-10-20T22:07:07.697669] [INFO ] [MAIN ] Derivative Progress: 391/1877 symbols completed. CMCM took 1.1s. Estimated remaining: 9m 21s. Rate: 159.0 symbols/min [2025-10-20T22:07:07.698081] [INFO ] [MAIN ] CMPO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:07.718144] [DEBUG] [MAIN ] CMP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:07.719068] [INFO ] [MAIN ] CMP: Generated 1794 derivative records from 35799 primary records. [2025-10-20T22:07:07.719550] [DEBUG] [LOCAL ] CMP: Saving derivative data. [2025-10-20T22:07:07.752142] [DEBUG] [LOCAL ] CMP: Derivative data saved (1794 records). [2025-10-20T22:07:07.752305] [INFO ] [MAIN ] CMP: Successfully saved derivative data. [2025-10-20T22:07:07.755133] [INFO ] [MAIN ] Derivative Progress: 392/1877 symbols completed. CMP took 1.1s. Estimated remaining: 9m 19s. Rate: 159.4 symbols/min [2025-10-20T22:07:07.755600] [INFO ] [MAIN ] CMRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:07.756025] [DEBUG] [MAIN ] CMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:07.766299] [INFO ] [MAIN ] CMC: Generated 1794 derivative records from 45569 primary records. [2025-10-20T22:07:07.766533] [DEBUG] [LOCAL ] CMC: Saving derivative data. [2025-10-20T22:07:07.782054] [DEBUG] [LOCAL ] CMC: Derivative data saved (1794 records). [2025-10-20T22:07:07.782932] [INFO ] [MAIN ] CMC: Successfully saved derivative data. [2025-10-20T22:07:07.795976] [INFO ] [MAIN ] Derivative Progress: 393/1877 symbols completed. CMC took 1.2s. Estimated remaining: 9m 17s. Rate: 159.8 symbols/min [2025-10-20T22:07:07.797272] [INFO ] [MAIN ] CMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:07.798385] [DEBUG] [LOCAL ] CMPO: Loaded 38711 existing records from primary storage. [2025-10-20T22:07:07.798744] [INFO ] [MAIN ] CMPO: Loaded 38711 primary data records. [2025-10-20T22:07:07.958809] [DEBUG] [LOCAL ] CMRE: Loaded 35844 existing records from primary storage. [2025-10-20T22:07:07.973575] [INFO ] [MAIN ] CMRE: Loaded 35844 primary data records. [2025-10-20T22:07:08.305713] [DEBUG] [LOCAL ] CMS: Loaded 52585 existing records from primary storage. [2025-10-20T22:07:08.335305] [INFO ] [MAIN ] CMS: Loaded 52585 primary data records. [2025-10-20T22:07:08.829986] [DEBUG] [MAIN ] CMPO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:08.831064] [INFO ] [MAIN ] CMPO: Generated 1794 derivative records from 38711 primary records. [2025-10-20T22:07:08.832176] [DEBUG] [LOCAL ] CMPO: Saving derivative data. [2025-10-20T22:07:08.852566] [DEBUG] [LOCAL ] CMPO: Derivative data saved (1794 records). [2025-10-20T22:07:08.853586] [INFO ] [MAIN ] CMPO: Successfully saved derivative data. [2025-10-20T22:07:08.859197] [INFO ] [MAIN ] Derivative Progress: 394/1877 symbols completed. CMPO took 1.2s. Estimated remaining: 9m 20s. Rate: 159.0 symbols/min [2025-10-20T22:07:08.860123] [INFO ] [MAIN ] CMSA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:08.876634] [DEBUG] [MAIN ] CMRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:08.881261] [DEBUG] [LOCAL ] CMSA: Loaded 3515 existing records from primary storage. [2025-10-20T22:07:08.877805] [INFO ] [MAIN ] CMRE: Generated 1794 derivative records from 35844 primary records. [2025-10-20T22:07:08.881737] [DEBUG] [LOCAL ] CMRE: Saving derivative data. [2025-10-20T22:07:08.881599] [INFO ] [MAIN ] CMSA: Loaded 3515 primary data records. [2025-10-20T22:07:08.951796] [DEBUG] [LOCAL ] CMRE: Derivative data saved (1794 records). [2025-10-20T22:07:08.964817] [INFO ] [MAIN ] CMRE: Successfully saved derivative data. [2025-10-20T22:07:08.977448] [INFO ] [MAIN ] Derivative Progress: 395/1877 symbols completed. CMRE took 1.2s. Estimated remaining: 9m 18s. Rate: 159.3 symbols/min [2025-10-20T22:07:08.977791] [INFO ] [MAIN ] CMSC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:09.006721] [DEBUG] [MAIN ] CMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:09.018058] [INFO ] [MAIN ] CMS: Generated 1794 derivative records from 52585 primary records. [2025-10-20T22:07:09.018163] [DEBUG] [LOCAL ] CMS: Saving derivative data. [2025-10-20T22:07:09.090626] [DEBUG] [LOCAL ] CMS: Derivative data saved (1794 records). [2025-10-20T22:07:09.103028] [INFO ] [MAIN ] CMS: Successfully saved derivative data. [2025-10-20T22:07:09.147591] [DEBUG] [LOCAL ] CMSC: Loaded 4847 existing records from primary storage. [2025-10-20T22:07:09.150065] [INFO ] [MAIN ] Derivative Progress: 396/1877 symbols completed. CMS took 1.4s. Estimated remaining: 9m 17s. Rate: 159.5 symbols/min [2025-10-20T22:07:09.150408] [INFO ] [MAIN ] CMSC: Loaded 4847 primary data records. [2025-10-20T22:07:09.150272] [INFO ] [MAIN ] CMSD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:09.386078] [DEBUG] [LOCAL ] CMSD: Loaded 7682 existing records from primary storage. [2025-10-20T22:07:09.386541] [INFO ] [MAIN ] CMSD: Loaded 7682 primary data records. [2025-10-20T22:07:09.964067] [DEBUG] [MAIN ] CMSA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:09.966185] [INFO ] [MAIN ] CMSA: Generated 1794 derivative records from 3515 primary records. [2025-10-20T22:07:09.966799] [DEBUG] [LOCAL ] CMSA: Saving derivative data. [2025-10-20T22:07:09.982982] [DEBUG] [LOCAL ] CMSA: Derivative data saved (1794 records). [2025-10-20T22:07:09.983209] [INFO ] [MAIN ] CMSA: Successfully saved derivative data. [2025-10-20T22:07:09.985499] [INFO ] [MAIN ] Derivative Progress: 397/1877 symbols completed. CMSA took 1.1s. Estimated remaining: 9m 18s. Rate: 159.0 symbols/min [2025-10-20T22:07:09.985799] [INFO ] [MAIN ] CMU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:10.011217] [DEBUG] [LOCAL ] CMU: Loaded 6357 existing records from primary storage. [2025-10-20T22:07:10.011693] [DEBUG] [MAIN ] CMSC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:10.012473] [INFO ] [MAIN ] CMSC: Generated 1794 derivative records from 4847 primary records. [2025-10-20T22:07:10.012687] [DEBUG] [LOCAL ] CMSC: Saving derivative data. [2025-10-20T22:07:10.012782] [INFO ] [MAIN ] CMU: Loaded 6357 primary data records. [2025-10-20T22:07:10.084065] [DEBUG] [LOCAL ] CMSC: Derivative data saved (1794 records). [2025-10-20T22:07:10.084793] [INFO ] [MAIN ] CMSC: Successfully saved derivative data. [2025-10-20T22:07:10.086874] [INFO ] [MAIN ] Derivative Progress: 398/1877 symbols completed. CMSC took 1.1s. Estimated remaining: 9m 17s. Rate: 159.3 symbols/min [2025-10-20T22:07:10.087123] [INFO ] [MAIN ] CNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:10.140626] [DEBUG] [MAIN ] CMSD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:10.164147] [INFO ] [MAIN ] CMSD: Generated 1794 derivative records from 7682 primary records. [2025-10-20T22:07:10.164340] [DEBUG] [LOCAL ] CMSD: Saving derivative data. [2025-10-20T22:07:10.242477] [DEBUG] [LOCAL ] CMSD: Derivative data saved (1794 records). [2025-10-20T22:07:10.244970] [INFO ] [MAIN ] CMSD: Successfully saved derivative data. [2025-10-20T22:07:10.263932] [INFO ] [MAIN ] Derivative Progress: 399/1877 symbols completed. CMSD took 1.1s. Estimated remaining: 9m 16s. Rate: 159.5 symbols/min [2025-10-20T22:07:10.264902] [INFO ] [MAIN ] CNC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:10.407333] [DEBUG] [LOCAL ] CNA: Loaded 32763 existing records from primary storage. [2025-10-20T22:07:10.407795] [INFO ] [MAIN ] CNA: Loaded 32763 primary data records. [2025-10-20T22:07:10.734873] [DEBUG] [LOCAL ] CNC: Loaded 53778 existing records from primary storage. [2025-10-20T22:07:10.736103] [INFO ] [MAIN ] CNC: Loaded 53778 primary data records. [2025-10-20T22:07:10.824753] [DEBUG] [MAIN ] CMU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:10.843822] [INFO ] [MAIN ] CMU: Generated 1794 derivative records from 6357 primary records. [2025-10-20T22:07:10.859401] [DEBUG] [LOCAL ] CMU: Saving derivative data. [2025-10-20T22:07:10.905183] [DEBUG] [LOCAL ] CMU: Derivative data saved (1794 records). [2025-10-20T22:07:10.951624] [INFO ] [MAIN ] CMU: Successfully saved derivative data. [2025-10-20T22:07:10.958503] [INFO ] [MAIN ] Derivative Progress: 400/1877 symbols completed. CMU took 1.0s. Estimated remaining: 9m 17s. Rate: 159.2 symbols/min [2025-10-20T22:07:11.071458] [INFO ] [MAIN ] CNH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:11.093269] [DEBUG] [MAIN ] CNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:11.094142] [INFO ] [MAIN ] CNA: Generated 1794 derivative records from 32763 primary records. [2025-10-20T22:07:11.094222] [DEBUG] [LOCAL ] CNA: Saving derivative data. [2025-10-20T22:07:11.111135] [DEBUG] [LOCAL ] CNA: Derivative data saved (1794 records). [2025-10-20T22:07:11.111268] [INFO ] [MAIN ] CNA: Successfully saved derivative data. [2025-10-20T22:07:11.113512] [INFO ] [MAIN ] Derivative Progress: 401/1877 symbols completed. CNA took 1.0s. Estimated remaining: 9m 16s. Rate: 159.4 symbols/min [2025-10-20T22:07:11.113820] [INFO ] [MAIN ] CNI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:11.146571] [DEBUG] [MAIN ] CNC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:11.147230] [INFO ] [MAIN ] CNC: Generated 1794 derivative records from 53778 primary records. [2025-10-20T22:07:11.147397] [DEBUG] [LOCAL ] CNC: Saving derivative data. [2025-10-20T22:07:11.163535] [DEBUG] [LOCAL ] CNC: Derivative data saved (1794 records). [2025-10-20T22:07:11.163739] [INFO ] [MAIN ] CNC: Successfully saved derivative data. [2025-10-20T22:07:11.180210] [INFO ] [MAIN ] Derivative Progress: 402/1877 symbols completed. CNC took 0.9s. Estimated remaining: 9m 14s. Rate: 159.8 symbols/min [2025-10-20T22:07:11.181683] [INFO ] [MAIN ] CNK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:11.183007] [DEBUG] [LOCAL ] CNH: Loaded 53548 existing records from primary storage. [2025-10-20T22:07:11.183289] [INFO ] [MAIN ] CNH: Loaded 53548 primary data records. [2025-10-20T22:07:11.530854] [DEBUG] [LOCAL ] CNI: Loaded 51678 existing records from primary storage. [2025-10-20T22:07:11.531282] [INFO ] [MAIN ] CNI: Loaded 51678 primary data records. [2025-10-20T22:07:11.609493] [DEBUG] [LOCAL ] CNK: Loaded 52920 existing records from primary storage. [2025-10-20T22:07:11.636945] [INFO ] [MAIN ] CNK: Loaded 52920 primary data records. [2025-10-20T22:07:12.201575] [DEBUG] [MAIN ] CNH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:12.217866] [INFO ] [MAIN ] CNH: Generated 1794 derivative records from 53548 primary records. [2025-10-20T22:07:12.218587] [DEBUG] [LOCAL ] CNH: Saving derivative data. [2025-10-20T22:07:12.236682] [DEBUG] [LOCAL ] CNH: Derivative data saved (1794 records). [2025-10-20T22:07:12.237407] [INFO ] [MAIN ] CNH: Successfully saved derivative data. [2025-10-20T22:07:12.239885] [INFO ] [MAIN ] Derivative Progress: 403/1877 symbols completed. CNH took 1.2s. Estimated remaining: 9m 16s. Rate: 159.0 symbols/min [2025-10-20T22:07:12.262207] [DEBUG] [MAIN ] CNI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:12.263237] [INFO ] [MAIN ] CNI: Generated 1794 derivative records from 51678 primary records. [2025-10-20T22:07:12.264142] [DEBUG] [LOCAL ] CNI: Saving derivative data. [2025-10-20T22:07:12.294529] [DEBUG] [LOCAL ] CNI: Derivative data saved (1794 records). [2025-10-20T22:07:12.294443] [DEBUG] [MAIN ] CNK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:12.294755] [INFO ] [MAIN ] CNI: Successfully saved derivative data. [2025-10-20T22:07:12.295454] [INFO ] [MAIN ] CNK: Generated 1794 derivative records from 52920 primary records. [2025-10-20T22:07:12.297977] [INFO ] [MAIN ] Derivative Progress: 404/1877 symbols completed. CNI took 1.2s. Estimated remaining: 9m 15s. Rate: 159.4 symbols/min [2025-10-20T22:07:12.298097] [DEBUG] [LOCAL ] CNK: Saving derivative data. [2025-10-20T22:07:12.314215] [DEBUG] [LOCAL ] CNK: Derivative data saved (1794 records). [2025-10-20T22:07:12.314334] [INFO ] [MAIN ] CNK: Successfully saved derivative data. [2025-10-20T22:07:12.316876] [INFO ] [MAIN ] Derivative Progress: 405/1877 symbols completed. CNK took 1.1s. Estimated remaining: 9m 13s. Rate: 159.7 symbols/min [2025-10-20T22:07:12.317274] [INFO ] [MAIN ] CNM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:12.317421] [INFO ] [MAIN ] CNQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:12.317556] [INFO ] [MAIN ] CNO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:12.390675] [DEBUG] [LOCAL ] CNQ: Loaded 53583 existing records from primary storage. [2025-10-20T22:07:12.390904] [INFO ] [MAIN ] CNQ: Loaded 53583 primary data records. [2025-10-20T22:07:12.529103] [DEBUG] [LOCAL ] CNO: Loaded 40510 existing records from primary storage. [2025-10-20T22:07:12.530326] [DEBUG] [LOCAL ] CNM: Loaded 51031 existing records from primary storage. [2025-10-20T22:07:12.536066] [INFO ] [MAIN ] CNO: Loaded 40510 primary data records. [2025-10-20T22:07:12.541325] [INFO ] [MAIN ] CNM: Loaded 51031 primary data records. [2025-10-20T22:07:13.391258] [DEBUG] [MAIN ] CNQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:13.392298] [INFO ] [MAIN ] CNQ: Generated 1794 derivative records from 53583 primary records. [2025-10-20T22:07:13.392503] [DEBUG] [LOCAL ] CNQ: Saving derivative data. [2025-10-20T22:07:13.413687] [DEBUG] [LOCAL ] CNQ: Derivative data saved (1794 records). [2025-10-20T22:07:13.426587] [INFO ] [MAIN ] CNQ: Successfully saved derivative data. [2025-10-20T22:07:13.427576] [DEBUG] [MAIN ] CNO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:13.429604] [INFO ] [MAIN ] Derivative Progress: 406/1877 symbols completed. CNQ took 1.1s. Estimated remaining: 9m 15s. Rate: 159.0 symbols/min [2025-10-20T22:07:13.430611] [INFO ] [MAIN ] CNO: Generated 1794 derivative records from 40510 primary records. [2025-10-20T22:07:13.431046] [DEBUG] [LOCAL ] CNO: Saving derivative data. [2025-10-20T22:07:13.431226] [INFO ] [MAIN ] CODI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:13.453496] [DEBUG] [LOCAL ] CNO: Derivative data saved (1794 records). [2025-10-20T22:07:13.466541] [INFO ] [MAIN ] CNO: Successfully saved derivative data. [2025-10-20T22:07:13.467228] [DEBUG] [MAIN ] CNM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:13.469434] [INFO ] [MAIN ] Derivative Progress: 407/1877 symbols completed. CNO took 1.2s. Estimated remaining: 9m 14s. Rate: 159.3 symbols/min [2025-10-20T22:07:13.470473] [INFO ] [MAIN ] CNM: Generated 1794 derivative records from 51031 primary records. [2025-10-20T22:07:13.470599] [DEBUG] [LOCAL ] CNM: Saving derivative data. [2025-10-20T22:07:13.470843] [INFO ] [MAIN ] COF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:13.489234] [DEBUG] [LOCAL ] CNM: Derivative data saved (1794 records). [2025-10-20T22:07:13.499753] [INFO ] [MAIN ] CNM: Successfully saved derivative data. [2025-10-20T22:07:13.502530] [INFO ] [MAIN ] Derivative Progress: 408/1877 symbols completed. CNM took 1.2s. Estimated remaining: 9m 12s. Rate: 159.7 symbols/min [2025-10-20T22:07:13.503044] [INFO ] [MAIN ] COLD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:13.526000] [DEBUG] [LOCAL ] CODI: Loaded 41816 existing records from primary storage. [2025-10-20T22:07:13.526277] [INFO ] [MAIN ] CODI: Loaded 41816 primary data records. [2025-10-20T22:07:13.910046] [DEBUG] [LOCAL ] COLD: Loaded 52476 existing records from primary storage. [2025-10-20T22:07:13.911824] [INFO ] [MAIN ] COLD: Loaded 52476 primary data records. [2025-10-20T22:07:13.911101] [DEBUG] [LOCAL ] COF: Loaded 53625 existing records from primary storage. [2025-10-20T22:07:13.933105] [INFO ] [MAIN ] COF: Loaded 53625 primary data records. [2025-10-20T22:07:14.551996] [DEBUG] [MAIN ] CODI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:14.553252] [INFO ] [MAIN ] CODI: Generated 1794 derivative records from 41816 primary records. [2025-10-20T22:07:14.553515] [DEBUG] [LOCAL ] CODI: Saving derivative data. [2025-10-20T22:07:14.570819] [DEBUG] [LOCAL ] CODI: Derivative data saved (1794 records). [2025-10-20T22:07:14.571008] [INFO ] [MAIN ] CODI: Successfully saved derivative data. [2025-10-20T22:07:14.573024] [INFO ] [MAIN ] Derivative Progress: 409/1877 symbols completed. CODI took 1.1s. Estimated remaining: 9m 14s. Rate: 159.0 symbols/min [2025-10-20T22:07:14.573738] [INFO ] [MAIN ] COMP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:14.590796] [DEBUG] [MAIN ] COLD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:14.593823] [INFO ] [MAIN ] COLD: Generated 1794 derivative records from 52476 primary records. [2025-10-20T22:07:14.594110] [DEBUG] [LOCAL ] COLD: Saving derivative data. [2025-10-20T22:07:14.607901] [DEBUG] [MAIN ] COF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:14.623359] [INFO ] [MAIN ] COF: Generated 1794 derivative records from 53625 primary records. [2025-10-20T22:07:14.624526] [DEBUG] [LOCAL ] COLD: Derivative data saved (1794 records). [2025-10-20T22:07:14.624923] [DEBUG] [LOCAL ] COF: Saving derivative data. [2025-10-20T22:07:14.624851] [INFO ] [MAIN ] COLD: Successfully saved derivative data. [2025-10-20T22:07:14.628029] [INFO ] [MAIN ] Derivative Progress: 410/1877 symbols completed. COLD took 1.1s. Estimated remaining: 9m 13s. Rate: 159.3 symbols/min [2025-10-20T22:07:14.628542] [INFO ] [MAIN ] CON: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:14.657994] [DEBUG] [LOCAL ] COF: Derivative data saved (1794 records). [2025-10-20T22:07:14.658334] [INFO ] [MAIN ] COF: Successfully saved derivative data. [2025-10-20T22:07:14.660343] [INFO ] [MAIN ] Derivative Progress: 411/1877 symbols completed. COF took 1.2s. Estimated remaining: 9m 11s. Rate: 159.6 symbols/min [2025-10-20T22:07:14.660612] [INFO ] [MAIN ] COOK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:14.681450] [DEBUG] [LOCAL ] COMP: Loaded 53404 existing records from primary storage. [2025-10-20T22:07:14.681760] [INFO ] [MAIN ] COMP: Loaded 53404 primary data records. [2025-10-20T22:07:14.806177] [DEBUG] [LOCAL ] COOK: Loaded 24923 existing records from primary storage. [2025-10-20T22:07:14.843214] [INFO ] [MAIN ] COOK: Loaded 24923 primary data records. [2025-10-20T22:07:14.893687] [DEBUG] [LOCAL ] CON: Loaded 38877 existing records from primary storage. [2025-10-20T22:07:14.906010] [INFO ] [MAIN ] CON: Loaded 38877 primary data records. [2025-10-20T22:07:15.640337] [DEBUG] [MAIN ] COMP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:15.641579] [INFO ] [MAIN ] COMP: Generated 1794 derivative records from 53404 primary records. [2025-10-20T22:07:15.642239] [DEBUG] [LOCAL ] COMP: Saving derivative data. [2025-10-20T22:07:15.662674] [DEBUG] [LOCAL ] COMP: Derivative data saved (1794 records). [2025-10-20T22:07:15.662950] [INFO ] [MAIN ] COMP: Successfully saved derivative data. [2025-10-20T22:07:15.664863] [INFO ] [MAIN ] Derivative Progress: 412/1877 symbols completed. COMP took 1.1s. Estimated remaining: 9m 13s. Rate: 159.0 symbols/min [2025-10-20T22:07:15.665288] [INFO ] [MAIN ] COP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:15.685870] [DEBUG] [MAIN ] COOK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:15.688158] [INFO ] [MAIN ] COOK: Generated 1794 derivative records from 24923 primary records. [2025-10-20T22:07:15.688233] [DEBUG] [LOCAL ] COOK: Saving derivative data. [2025-10-20T22:07:15.712773] [DEBUG] [MAIN ] CON: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:15.716917] [DEBUG] [LOCAL ] COOK: Derivative data saved (1794 records). [2025-10-20T22:07:15.717233] [INFO ] [MAIN ] COOK: Successfully saved derivative data. [2025-10-20T22:07:15.719139] [INFO ] [MAIN ] Derivative Progress: 413/1877 symbols completed. COOK took 1.1s. Estimated remaining: 9m 11s. Rate: 159.3 symbols/min [2025-10-20T22:07:15.720273] [INFO ] [MAIN ] CON: Generated 1794 derivative records from 38877 primary records. [2025-10-20T22:07:15.719291] [INFO ] [MAIN ] COTY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:15.720497] [DEBUG] [LOCAL ] CON: Saving derivative data. [2025-10-20T22:07:15.738535] [DEBUG] [LOCAL ] CON: Derivative data saved (1794 records). [2025-10-20T22:07:15.738783] [INFO ] [MAIN ] CON: Successfully saved derivative data. [2025-10-20T22:07:15.741310] [INFO ] [MAIN ] Derivative Progress: 414/1877 symbols completed. CON took 1.1s. Estimated remaining: 9m 10s. Rate: 159.7 symbols/min [2025-10-20T22:07:15.741510] [INFO ] [MAIN ] COUR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:15.816633] [DEBUG] [LOCAL ] COP: Loaded 53727 existing records from primary storage. [2025-10-20T22:07:15.817810] [INFO ] [MAIN ] COP: Loaded 53727 primary data records. [2025-10-20T22:07:15.817483] [DEBUG] [LOCAL ] COTY: Loaded 53275 existing records from primary storage. [2025-10-20T22:07:15.827671] [INFO ] [MAIN ] COTY: Loaded 53275 primary data records. [2025-10-20T22:07:16.060752] [DEBUG] [LOCAL ] COUR: Loaded 49404 existing records from primary storage. [2025-10-20T22:07:16.061085] [INFO ] [MAIN ] COUR: Loaded 49404 primary data records. [2025-10-20T22:07:16.812479] [DEBUG] [MAIN ] COP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:16.825613] [INFO ] [MAIN ] COP: Generated 1794 derivative records from 53727 primary records. [2025-10-20T22:07:16.829092] [DEBUG] [LOCAL ] COP: Saving derivative data. [2025-10-20T22:07:16.829000] [DEBUG] [MAIN ] COTY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:16.838113] [INFO ] [MAIN ] COTY: Generated 1794 derivative records from 53275 primary records. [2025-10-20T22:07:16.842163] [DEBUG] [LOCAL ] COTY: Saving derivative data. [2025-10-20T22:07:16.857814] [DEBUG] [MAIN ] COUR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:16.873596] [DEBUG] [LOCAL ] COTY: Derivative data saved (1794 records). [2025-10-20T22:07:16.874006] [INFO ] [MAIN ] COTY: Successfully saved derivative data. [2025-10-20T22:07:16.869672] [INFO ] [MAIN ] COUR: Generated 1794 derivative records from 49404 primary records. [2025-10-20T22:07:16.876373] [INFO ] [MAIN ] Derivative Progress: 415/1877 symbols completed. COTY took 1.2s. Estimated remaining: 9m 12s. Rate: 158.9 symbols/min [2025-10-20T22:07:16.876826] [DEBUG] [LOCAL ] COUR: Saving derivative data. [2025-10-20T22:07:16.877548] [INFO ] [MAIN ] CPA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:16.880052] [DEBUG] [LOCAL ] COP: Derivative data saved (1794 records). [2025-10-20T22:07:16.887577] [INFO ] [MAIN ] COP: Successfully saved derivative data. [2025-10-20T22:07:16.889536] [INFO ] [MAIN ] Derivative Progress: 416/1877 symbols completed. COP took 1.2s. Estimated remaining: 9m 10s. Rate: 159.3 symbols/min [2025-10-20T22:07:16.891123] [INFO ] [MAIN ] CPAY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:16.898357] [DEBUG] [LOCAL ] COUR: Derivative data saved (1794 records). [2025-10-20T22:07:16.898625] [INFO ] [MAIN ] COUR: Successfully saved derivative data. [2025-10-20T22:07:16.900837] [INFO ] [MAIN ] Derivative Progress: 417/1877 symbols completed. COUR took 1.2s. Estimated remaining: 9m 9s. Rate: 159.7 symbols/min [2025-10-20T22:07:16.901025] [INFO ] [MAIN ] CPF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:16.940693] [DEBUG] [LOCAL ] CPA: Loaded 28158 existing records from primary storage. [2025-10-20T22:07:16.940168] [DEBUG] [LOCAL ] CPF: Loaded 15307 existing records from primary storage. [2025-10-20T22:07:16.940952] [INFO ] [MAIN ] CPA: Loaded 28158 primary data records. [2025-10-20T22:07:16.941129] [INFO ] [MAIN ] CPF: Loaded 15307 primary data records. [2025-10-20T22:07:17.040942] [DEBUG] [LOCAL ] CPAY: Loaded 30345 existing records from primary storage. [2025-10-20T22:07:17.090095] [INFO ] [MAIN ] CPAY: Loaded 30345 primary data records. [2025-10-20T22:07:18.011424] [DEBUG] [MAIN ] CPF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:18.013162] [INFO ] [MAIN ] CPF: Generated 1794 derivative records from 15307 primary records. [2025-10-20T22:07:18.012170] [DEBUG] [MAIN ] CPA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:18.013696] [DEBUG] [LOCAL ] CPF: Saving derivative data. [2025-10-20T22:07:18.015681] [INFO ] [MAIN ] CPA: Generated 1794 derivative records from 28158 primary records. [2025-10-20T22:07:18.015986] [DEBUG] [LOCAL ] CPA: Saving derivative data. [2025-10-20T22:07:18.043786] [DEBUG] [LOCAL ] CPF: Derivative data saved (1794 records). [2025-10-20T22:07:18.047679] [INFO ] [MAIN ] CPF: Successfully saved derivative data. [2025-10-20T22:07:18.054290] [INFO ] [MAIN ] Derivative Progress: 418/1877 symbols completed. CPF took 1.2s. Estimated remaining: 9m 11s. Rate: 158.9 symbols/min [2025-10-20T22:07:18.052172] [DEBUG] [LOCAL ] CPA: Derivative data saved (1794 records). [2025-10-20T22:07:18.054698] [INFO ] [MAIN ] CPA: Successfully saved derivative data. [2025-10-20T22:07:18.057260] [INFO ] [MAIN ] Derivative Progress: 419/1877 symbols completed. CPA took 1.2s. Estimated remaining: 9m 9s. Rate: 159.3 symbols/min [2025-10-20T22:07:18.070494] [DEBUG] [MAIN ] CPAY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:18.071256] [INFO ] [MAIN ] CPAY: Generated 1794 derivative records from 30345 primary records. [2025-10-20T22:07:18.071319] [DEBUG] [LOCAL ] CPAY: Saving derivative data. [2025-10-20T22:07:18.087465] [DEBUG] [LOCAL ] CPAY: Derivative data saved (1794 records). [2025-10-20T22:07:18.087539] [INFO ] [MAIN ] CPAY: Successfully saved derivative data. [2025-10-20T22:07:18.089977] [INFO ] [MAIN ] Derivative Progress: 420/1877 symbols completed. CPAY took 1.2s. Estimated remaining: 9m 8s. Rate: 159.6 symbols/min [2025-10-20T22:07:18.090431] [INFO ] [MAIN ] CPRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:18.090957] [INFO ] [MAIN ] CPNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:18.091108] [INFO ] [MAIN ] CPK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:18.109603] [DEBUG] [LOCAL ] CPK: Loaded 14051 existing records from primary storage. [2025-10-20T22:07:18.110412] [INFO ] [MAIN ] CPK: Loaded 14051 primary data records. [2025-10-20T22:07:18.315822] [DEBUG] [LOCAL ] CPRI: Loaded 52442 existing records from primary storage. [2025-10-20T22:07:18.337525] [INFO ] [MAIN ] CPRI: Loaded 52442 primary data records. [2025-10-20T22:07:18.537529] [DEBUG] [LOCAL ] CPNG: Loaded 53680 existing records from primary storage. [2025-10-20T22:07:18.550743] [INFO ] [MAIN ] CPNG: Loaded 53680 primary data records. [2025-10-20T22:07:19.116076] [DEBUG] [MAIN ] CPK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:19.116891] [INFO ] [MAIN ] CPK: Generated 1794 derivative records from 14051 primary records. [2025-10-20T22:07:19.117214] [DEBUG] [LOCAL ] CPK: Saving derivative data. [2025-10-20T22:07:19.134370] [DEBUG] [LOCAL ] CPK: Derivative data saved (1794 records). [2025-10-20T22:07:19.136039] [INFO ] [MAIN ] CPK: Successfully saved derivative data. [2025-10-20T22:07:19.139304] [INFO ] [MAIN ] Derivative Progress: 421/1877 symbols completed. CPK took 1.0s. Estimated remaining: 9m 10s. Rate: 158.9 symbols/min [2025-10-20T22:07:19.139590] [INFO ] [MAIN ] CPS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:19.155772] [DEBUG] [MAIN ] CPRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:19.158594] [INFO ] [MAIN ] CPRI: Generated 1794 derivative records from 52442 primary records. [2025-10-20T22:07:19.158801] [DEBUG] [LOCAL ] CPRI: Saving derivative data. [2025-10-20T22:07:19.177390] [DEBUG] [LOCAL ] CPRI: Derivative data saved (1794 records). [2025-10-20T22:07:19.177633] [INFO ] [MAIN ] CPRI: Successfully saved derivative data. [2025-10-20T22:07:19.192250] [INFO ] [MAIN ] Derivative Progress: 422/1877 symbols completed. CPRI took 1.1s. Estimated remaining: 9m 8s. Rate: 159.2 symbols/min [2025-10-20T22:07:19.204777] [DEBUG] [MAIN ] CPNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:19.206192] [INFO ] [MAIN ] CPNG: Generated 1794 derivative records from 53680 primary records. [2025-10-20T22:07:19.206347] [DEBUG] [LOCAL ] CPNG: Saving derivative data. [2025-10-20T22:07:19.206505] [INFO ] [MAIN ] CPT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:19.229094] [DEBUG] [LOCAL ] CPS: Loaded 25047 existing records from primary storage. [2025-10-20T22:07:19.230016] [INFO ] [MAIN ] CPS: Loaded 25047 primary data records. [2025-10-20T22:07:19.255145] [DEBUG] [LOCAL ] CPNG: Derivative data saved (1794 records). [2025-10-20T22:07:19.255261] [INFO ] [MAIN ] CPNG: Successfully saved derivative data. [2025-10-20T22:07:19.257731] [INFO ] [MAIN ] Derivative Progress: 423/1877 symbols completed. CPNG took 1.2s. Estimated remaining: 9m 7s. Rate: 159.6 symbols/min [2025-10-20T22:07:19.268236] [INFO ] [MAIN ] CR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:19.561366] [DEBUG] [LOCAL ] CR: Loaded 24066 existing records from primary storage. [2025-10-20T22:07:19.561605] [INFO ] [MAIN ] CR: Loaded 24066 primary data records. [2025-10-20T22:07:19.612607] [DEBUG] [LOCAL ] CPT: Loaded 44551 existing records from primary storage. [2025-10-20T22:07:19.652753] [INFO ] [MAIN ] CPT: Loaded 44551 primary data records. [2025-10-20T22:07:20.229525] [DEBUG] [MAIN ] CPS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:20.231758] [INFO ] [MAIN ] CPS: Generated 1794 derivative records from 25047 primary records. [2025-10-20T22:07:20.231969] [DEBUG] [LOCAL ] CPS: Saving derivative data. [2025-10-20T22:07:20.248579] [DEBUG] [LOCAL ] CPS: Derivative data saved (1794 records). [2025-10-20T22:07:20.249062] [INFO ] [MAIN ] CPS: Successfully saved derivative data. [2025-10-20T22:07:20.252172] [INFO ] [MAIN ] Derivative Progress: 424/1877 symbols completed. CPS took 1.1s. Estimated remaining: 9m 8s. Rate: 158.9 symbols/min [2025-10-20T22:07:20.252366] [INFO ] [MAIN ] CRBD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:20.274530] [DEBUG] [MAIN ] CR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:20.278502] [DEBUG] [LOCAL ] CRBD: Loaded 8858 existing records from primary storage. [2025-10-20T22:07:20.280132] [INFO ] [MAIN ] CR: Generated 1793 derivative records from 24066 primary records. [2025-10-20T22:07:20.281014] [INFO ] [MAIN ] CRBD: Loaded 8858 primary data records. [2025-10-20T22:07:20.292062] [DEBUG] [LOCAL ] CR: Saving derivative data. [2025-10-20T22:07:20.414215] [DEBUG] [LOCAL ] CR: Derivative data saved (1793 records). [2025-10-20T22:07:20.414913] [INFO ] [MAIN ] CR: Successfully saved derivative data. [2025-10-20T22:07:20.416893] [INFO ] [MAIN ] Derivative Progress: 425/1877 symbols completed. CR took 1.1s. Estimated remaining: 9m 7s. Rate: 159.2 symbols/min [2025-10-20T22:07:20.417558] [DEBUG] [MAIN ] CPT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:20.428948] [INFO ] [MAIN ] CPT: Generated 1794 derivative records from 44551 primary records. [2025-10-20T22:07:20.429058] [INFO ] [MAIN ] CRBG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:20.444551] [DEBUG] [LOCAL ] CPT: Saving derivative data. [2025-10-20T22:07:20.632617] [DEBUG] [LOCAL ] CPT: Derivative data saved (1794 records). [2025-10-20T22:07:20.633438] [INFO ] [MAIN ] CPT: Successfully saved derivative data. [2025-10-20T22:07:20.635799] [INFO ] [MAIN ] Derivative Progress: 426/1877 symbols completed. CPT took 1.4s. Estimated remaining: 9m 6s. Rate: 159.3 symbols/min [2025-10-20T22:07:20.636068] [INFO ] [MAIN ] CRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:20.660010] [DEBUG] [MAIN ] CRBD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:20.660683] [INFO ] [MAIN ] CRBD: Generated 1794 derivative records from 8858 primary records. [2025-10-20T22:07:20.660749] [DEBUG] [LOCAL ] CRBD: Saving derivative data. [2025-10-20T22:07:20.690233] [DEBUG] [LOCAL ] CRBD: Derivative data saved (1794 records). [2025-10-20T22:07:20.690438] [INFO ] [MAIN ] CRBD: Successfully saved derivative data. [2025-10-20T22:07:20.692398] [INFO ] [MAIN ] Derivative Progress: 427/1877 symbols completed. CRBD took 0.4s. Estimated remaining: 9m 5s. Rate: 159.6 symbols/min [2025-10-20T22:07:20.692611] [INFO ] [MAIN ] CRGY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:20.722384] [DEBUG] [LOCAL ] CRBG: Loaded 52602 existing records from primary storage. [2025-10-20T22:07:20.722746] [INFO ] [MAIN ] CRBG: Loaded 52602 primary data records. [2025-10-20T22:07:20.842817] [DEBUG] [LOCAL ] CRC: Loaded 44124 existing records from primary storage. [2025-10-20T22:07:20.856494] [INFO ] [MAIN ] CRC: Loaded 44124 primary data records. [2025-10-20T22:07:21.033952] [DEBUG] [LOCAL ] CRGY: Loaded 52514 existing records from primary storage. [2025-10-20T22:07:21.172353] [INFO ] [MAIN ] CRGY: Loaded 52514 primary data records. [2025-10-20T22:07:21.744120] [DEBUG] [MAIN ] CRBG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:21.745423] [INFO ] [MAIN ] CRBG: Generated 1794 derivative records from 52602 primary records. [2025-10-20T22:07:21.745971] [DEBUG] [LOCAL ] CRBG: Saving derivative data. [2025-10-20T22:07:21.765634] [DEBUG] [LOCAL ] CRBG: Derivative data saved (1794 records). [2025-10-20T22:07:21.765965] [INFO ] [MAIN ] CRBG: Successfully saved derivative data. [2025-10-20T22:07:21.768117] [INFO ] [MAIN ] Derivative Progress: 428/1877 symbols completed. CRBG took 1.3s. Estimated remaining: 9m 7s. Rate: 158.9 symbols/min [2025-10-20T22:07:21.768514] [INFO ] [MAIN ] CRH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:21.782699] [DEBUG] [MAIN ] CRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:21.784669] [INFO ] [MAIN ] CRC: Generated 1794 derivative records from 44124 primary records. [2025-10-20T22:07:21.785487] [DEBUG] [LOCAL ] CRC: Saving derivative data. [2025-10-20T22:07:21.815072] [DEBUG] [LOCAL ] CRC: Derivative data saved (1794 records). [2025-10-20T22:07:21.815686] [INFO ] [MAIN ] CRC: Successfully saved derivative data. [2025-10-20T22:07:21.815562] [DEBUG] [MAIN ] CRGY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:21.818896] [INFO ] [MAIN ] CRGY: Generated 1794 derivative records from 52514 primary records. [2025-10-20T22:07:21.818054] [INFO ] [MAIN ] Derivative Progress: 429/1877 symbols completed. CRC took 1.2s. Estimated remaining: 9m 6s. Rate: 159.3 symbols/min [2025-10-20T22:07:21.819049] [DEBUG] [LOCAL ] CRGY: Saving derivative data. [2025-10-20T22:07:21.819532] [INFO ] [MAIN ] CRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:21.835311] [DEBUG] [LOCAL ] CRGY: Derivative data saved (1794 records). [2025-10-20T22:07:21.835445] [INFO ] [MAIN ] CRGY: Successfully saved derivative data. [2025-10-20T22:07:21.837602] [INFO ] [MAIN ] Derivative Progress: 430/1877 symbols completed. CRGY took 1.1s. Estimated remaining: 9m 4s. Rate: 159.6 symbols/min [2025-10-20T22:07:21.837883] [INFO ] [MAIN ] CRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:21.894762] [DEBUG] [LOCAL ] CRH: Loaded 53439 existing records from primary storage. [2025-10-20T22:07:21.908899] [INFO ] [MAIN ] CRH: Loaded 53439 primary data records. [2025-10-20T22:07:21.955937] [DEBUG] [LOCAL ] CRI: Loaded 49604 existing records from primary storage. [2025-10-20T22:07:21.991161] [INFO ] [MAIN ] CRI: Loaded 49604 primary data records. [2025-10-20T22:07:22.121548] [DEBUG] [LOCAL ] CRK: Loaded 52250 existing records from primary storage. [2025-10-20T22:07:22.157715] [INFO ] [MAIN ] CRK: Loaded 52250 primary data records. [2025-10-20T22:07:22.896253] [DEBUG] [MAIN ] CRH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:22.899412] [INFO ] [MAIN ] CRH: Generated 1794 derivative records from 53439 primary records. [2025-10-20T22:07:22.901840] [DEBUG] [LOCAL ] CRH: Saving derivative data. [2025-10-20T22:07:22.927049] [DEBUG] [MAIN ] CRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:22.935618] [DEBUG] [LOCAL ] CRH: Derivative data saved (1794 records). [2025-10-20T22:07:22.936451] [INFO ] [MAIN ] CRH: Successfully saved derivative data. [2025-10-20T22:07:22.938260] [INFO ] [MAIN ] CRI: Generated 1794 derivative records from 49604 primary records. [2025-10-20T22:07:22.942939] [INFO ] [MAIN ] Derivative Progress: 431/1877 symbols completed. CRH took 1.2s. Estimated remaining: 9m 6s. Rate: 158.9 symbols/min [2025-10-20T22:07:22.943098] [DEBUG] [LOCAL ] CRI: Saving derivative data. [2025-10-20T22:07:22.943395] [INFO ] [MAIN ] CRL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:22.960336] [DEBUG] [MAIN ] CRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:22.972799] [INFO ] [MAIN ] CRK: Generated 1794 derivative records from 52250 primary records. [2025-10-20T22:07:22.977433] [DEBUG] [LOCAL ] CRK: Saving derivative data. [2025-10-20T22:07:22.977317] [DEBUG] [LOCAL ] CRI: Derivative data saved (1794 records). [2025-10-20T22:07:22.977670] [INFO ] [MAIN ] CRI: Successfully saved derivative data. [2025-10-20T22:07:22.981314] [INFO ] [MAIN ] Derivative Progress: 432/1877 symbols completed. CRI took 1.2s. Estimated remaining: 9m 5s. Rate: 159.2 symbols/min [2025-10-20T22:07:22.981484] [INFO ] [MAIN ] CRM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:22.998188] [DEBUG] [LOCAL ] CRK: Derivative data saved (1794 records). [2025-10-20T22:07:22.998413] [INFO ] [MAIN ] CRK: Successfully saved derivative data. [2025-10-20T22:07:23.005001] [INFO ] [MAIN ] Derivative Progress: 433/1877 symbols completed. CRK took 1.2s. Estimated remaining: 9m 3s. Rate: 159.6 symbols/min [2025-10-20T22:07:23.053405] [DEBUG] [LOCAL ] CRL: Loaded 42666 existing records from primary storage. [2025-10-20T22:07:23.066039] [INFO ] [MAIN ] CRL: Loaded 42666 primary data records. [2025-10-20T22:07:23.227563] [DEBUG] [LOCAL ] CRM: Loaded 53770 existing records from primary storage. [2025-10-20T22:07:23.243777] [INFO ] [MAIN ] CRM: Loaded 53770 primary data records. [2025-10-20T22:07:23.723996] [DEBUG] [MAIN ] CRL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:23.726598] [INFO ] [MAIN ] CRL: Generated 1794 derivative records from 42666 primary records. [2025-10-20T22:07:23.727010] [DEBUG] [LOCAL ] CRL: Saving derivative data. [2025-10-20T22:07:23.742450] [DEBUG] [MAIN ] CRM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:23.750428] [INFO ] [MAIN ] CRM: Generated 1794 derivative records from 53770 primary records. [2025-10-20T22:07:23.752345] [DEBUG] [LOCAL ] CRM: Saving derivative data. [2025-10-20T22:07:23.759824] [DEBUG] [LOCAL ] CRL: Derivative data saved (1794 records). [2025-10-20T22:07:23.760296] [INFO ] [MAIN ] CRL: Successfully saved derivative data. [2025-10-20T22:07:23.763494] [INFO ] [MAIN ] Derivative Progress: 434/1877 symbols completed. CRL took 0.8s. Estimated remaining: 9m 4s. Rate: 159.2 symbols/min [2025-10-20T22:07:23.778877] [DEBUG] [LOCAL ] CRM: Derivative data saved (1794 records). [2025-10-20T22:07:23.778956] [INFO ] [MAIN ] CRM: Successfully saved derivative data. [2025-10-20T22:07:23.780954] [INFO ] [MAIN ] Derivative Progress: 435/1877 symbols completed. CRM took 0.8s. Estimated remaining: 9m 2s. Rate: 159.5 symbols/min [2025-10-20T22:07:23.781546] [INFO ] [MAIN ] CRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:23.781392] [INFO ] [MAIN ] CRS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:23.781473] [INFO ] [MAIN ] CSAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:23.791703] [DEBUG] [LOCAL ] CRT: Loaded 4802 existing records from primary storage. [2025-10-20T22:07:23.791917] [INFO ] [MAIN ] CRT: Loaded 4802 primary data records. [2025-10-20T22:07:24.023896] [DEBUG] [LOCAL ] CSAN: Loaded 42287 existing records from primary storage. [2025-10-20T22:07:24.026766] [INFO ] [MAIN ] CSAN: Loaded 42287 primary data records. [2025-10-20T22:07:24.292254] [DEBUG] [LOCAL ] CRS: Loaded 43124 existing records from primary storage. [2025-10-20T22:07:24.306503] [INFO ] [MAIN ] CRS: Loaded 43124 primary data records. [2025-10-20T22:07:24.837376] [DEBUG] [MAIN ] CRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:24.839081] [INFO ] [MAIN ] CRT: Generated 1794 derivative records from 4802 primary records. [2025-10-20T22:07:24.839801] [DEBUG] [LOCAL ] CRT: Saving derivative data. [2025-10-20T22:07:24.861383] [DEBUG] [LOCAL ] CRT: Derivative data saved (1794 records). [2025-10-20T22:07:24.861678] [INFO ] [MAIN ] CRT: Successfully saved derivative data. [2025-10-20T22:07:24.864481] [INFO ] [MAIN ] Derivative Progress: 436/1877 symbols completed. CRT took 1.1s. Estimated remaining: 9m 4s. Rate: 158.9 symbols/min [2025-10-20T22:07:24.864797] [INFO ] [MAIN ] CSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:24.881811] [DEBUG] [MAIN ] CSAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:24.882624] [INFO ] [MAIN ] CSAN: Generated 1794 derivative records from 42287 primary records. [2025-10-20T22:07:24.882721] [DEBUG] [LOCAL ] CSAN: Saving derivative data. [2025-10-20T22:07:24.909187] [DEBUG] [MAIN ] CRS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:24.911365] [INFO ] [MAIN ] CRS: Generated 1794 derivative records from 43124 primary records. [2025-10-20T22:07:24.911458] [DEBUG] [LOCAL ] CRS: Saving derivative data. [2025-10-20T22:07:24.912116] [DEBUG] [LOCAL ] CSAN: Derivative data saved (1794 records). [2025-10-20T22:07:24.912852] [INFO ] [MAIN ] CSAN: Successfully saved derivative data. [2025-10-20T22:07:24.914818] [INFO ] [MAIN ] Derivative Progress: 437/1877 symbols completed. CSAN took 1.1s. Estimated remaining: 9m 3s. Rate: 159.2 symbols/min [2025-10-20T22:07:24.922395] [INFO ] [MAIN ] CSR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:24.930969] [DEBUG] [LOCAL ] CRS: Derivative data saved (1794 records). [2025-10-20T22:07:24.931163] [INFO ] [MAIN ] CRS: Successfully saved derivative data. [2025-10-20T22:07:24.933324] [INFO ] [MAIN ] Derivative Progress: 438/1877 symbols completed. CRS took 1.2s. Estimated remaining: 9m 1s. Rate: 159.5 symbols/min [2025-10-20T22:07:24.943192] [INFO ] [MAIN ] CSTM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:24.955304] [DEBUG] [LOCAL ] CSR: Loaded 13476 existing records from primary storage. [2025-10-20T22:07:24.965090] [INFO ] [MAIN ] CSR: Loaded 13476 primary data records. [2025-10-20T22:07:24.977131] [DEBUG] [LOCAL ] CSL: Loaded 31310 existing records from primary storage. [2025-10-20T22:07:25.008550] [INFO ] [MAIN ] CSL: Loaded 31310 primary data records. [2025-10-20T22:07:25.592852] [DEBUG] [LOCAL ] CSTM: Loaded 47473 existing records from primary storage. [2025-10-20T22:07:25.594036] [INFO ] [MAIN ] CSTM: Loaded 47473 primary data records. [2025-10-20T22:07:25.977959] [DEBUG] [MAIN ] CSR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:26.030454] [INFO ] [MAIN ] CSR: Generated 1794 derivative records from 13476 primary records. [2025-10-20T22:07:26.031041] [DEBUG] [LOCAL ] CSR: Saving derivative data. [2025-10-20T22:07:26.050782] [DEBUG] [LOCAL ] CSR: Derivative data saved (1794 records). [2025-10-20T22:07:26.051137] [INFO ] [MAIN ] CSR: Successfully saved derivative data. [2025-10-20T22:07:26.053517] [INFO ] [MAIN ] Derivative Progress: 439/1877 symbols completed. CSR took 1.1s. Estimated remaining: 9m 3s. Rate: 158.8 symbols/min [2025-10-20T22:07:26.054148] [INFO ] [MAIN ] CSV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:26.068190] [DEBUG] [MAIN ] CSL: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:26.069570] [INFO ] [MAIN ] CSL: Generated 1793 derivative records from 31310 primary records. [2025-10-20T22:07:26.070610] [DEBUG] [LOCAL ] CSL: Saving derivative data. [2025-10-20T22:07:26.091469] [DEBUG] [LOCAL ] CSL: Derivative data saved (1793 records). [2025-10-20T22:07:26.091686] [INFO ] [MAIN ] CSL: Successfully saved derivative data. [2025-10-20T22:07:26.093886] [INFO ] [MAIN ] Derivative Progress: 440/1877 symbols completed. CSL took 1.2s. Estimated remaining: 9m 2s. Rate: 159.1 symbols/min [2025-10-20T22:07:26.094188] [INFO ] [MAIN ] CTBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:26.100412] [DEBUG] [LOCAL ] CSV: Loaded 12576 existing records from primary storage. [2025-10-20T22:07:26.100677] [INFO ] [MAIN ] CSV: Loaded 12576 primary data records. [2025-10-20T22:07:26.207757] [DEBUG] [LOCAL ] CTBB: Loaded 6573 existing records from primary storage. [2025-10-20T22:07:26.234580] [INFO ] [MAIN ] CTBB: Loaded 6573 primary data records. [2025-10-20T22:07:26.240411] [DEBUG] [MAIN ] CSTM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:26.314100] [INFO ] [MAIN ] CSTM: Generated 1794 derivative records from 47473 primary records. [2025-10-20T22:07:26.342006] [DEBUG] [LOCAL ] CSTM: Saving derivative data. [2025-10-20T22:07:26.563630] [DEBUG] [LOCAL ] CSTM: Derivative data saved (1794 records). [2025-10-20T22:07:26.569014] [INFO ] [MAIN ] CSTM: Successfully saved derivative data. [2025-10-20T22:07:26.584723] [INFO ] [MAIN ] Derivative Progress: 441/1877 symbols completed. CSTM took 1.6s. Estimated remaining: 9m 2s. Rate: 159.0 symbols/min [2025-10-20T22:07:26.594685] [INFO ] [MAIN ] CTDD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:26.748114] [DEBUG] [LOCAL ] CTDD: Loaded 5284 existing records from primary storage. [2025-10-20T22:07:26.748612] [INFO ] [MAIN ] CTDD: Loaded 5284 primary data records. [2025-10-20T22:07:27.070483] [DEBUG] [MAIN ] CSV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:27.071455] [INFO ] [MAIN ] CSV: Generated 1794 derivative records from 12576 primary records. [2025-10-20T22:07:27.071797] [DEBUG] [LOCAL ] CSV: Saving derivative data. [2025-10-20T22:07:27.092523] [DEBUG] [LOCAL ] CSV: Derivative data saved (1794 records). [2025-10-20T22:07:27.092728] [INFO ] [MAIN ] CSV: Successfully saved derivative data. [2025-10-20T22:07:27.107620] [INFO ] [MAIN ] Derivative Progress: 442/1877 symbols completed. CSV took 1.1s. Estimated remaining: 9m 2s. Rate: 158.9 symbols/min [2025-10-20T22:07:27.108550] [INFO ] [MAIN ] CTOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:27.108104] [DEBUG] [MAIN ] CTBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:27.109938] [INFO ] [MAIN ] CTBB: Generated 1794 derivative records from 6573 primary records. [2025-10-20T22:07:27.110140] [DEBUG] [LOCAL ] CTBB: Saving derivative data. [2025-10-20T22:07:27.127757] [DEBUG] [LOCAL ] CTBB: Derivative data saved (1794 records). [2025-10-20T22:07:27.127981] [INFO ] [MAIN ] CTBB: Successfully saved derivative data. [2025-10-20T22:07:27.130061] [INFO ] [MAIN ] Derivative Progress: 443/1877 symbols completed. CTBB took 1.0s. Estimated remaining: 9m 0s. Rate: 159.2 symbols/min [2025-10-20T22:07:27.130252] [INFO ] [MAIN ] CTRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:27.158242] [DEBUG] [MAIN ] CTDD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:27.159097] [INFO ] [MAIN ] CTDD: Generated 1794 derivative records from 5284 primary records. [2025-10-20T22:07:27.159321] [DEBUG] [LOCAL ] CTDD: Saving derivative data. [2025-10-20T22:07:27.174703] [DEBUG] [LOCAL ] CTDD: Derivative data saved (1794 records). [2025-10-20T22:07:27.175507] [INFO ] [MAIN ] CTDD: Successfully saved derivative data. [2025-10-20T22:07:27.187536] [INFO ] [MAIN ] Derivative Progress: 444/1877 symbols completed. CTDD took 0.6s. Estimated remaining: 8m 59s. Rate: 159.5 symbols/min [2025-10-20T22:07:27.188798] [INFO ] [MAIN ] CTRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:27.189949] [DEBUG] [LOCAL ] CTOS: Loaded 37326 existing records from primary storage. [2025-10-20T22:07:27.190128] [INFO ] [MAIN ] CTOS: Loaded 37326 primary data records. [2025-10-20T22:07:27.393973] [DEBUG] [LOCAL ] CTRE: Loaded 49507 existing records from primary storage. [2025-10-20T22:07:27.468572] [INFO ] [MAIN ] CTRE: Loaded 49507 primary data records. [2025-10-20T22:07:27.646252] [DEBUG] [LOCAL ] CTRI: Loaded 40518 existing records from primary storage. [2025-10-20T22:07:27.667565] [INFO ] [MAIN ] CTRI: Loaded 40518 primary data records. [2025-10-20T22:07:28.216907] [DEBUG] [MAIN ] CTOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:28.218051] [INFO ] [MAIN ] CTOS: Generated 1794 derivative records from 37326 primary records. [2025-10-20T22:07:28.218598] [DEBUG] [LOCAL ] CTOS: Saving derivative data. [2025-10-20T22:07:28.236207] [DEBUG] [LOCAL ] CTOS: Derivative data saved (1794 records). [2025-10-20T22:07:28.236729] [INFO ] [MAIN ] CTOS: Successfully saved derivative data. [2025-10-20T22:07:28.239038] [INFO ] [MAIN ] Derivative Progress: 445/1877 symbols completed. CTOS took 1.1s. Estimated remaining: 9m 1s. Rate: 158.9 symbols/min [2025-10-20T22:07:28.239415] [INFO ] [MAIN ] CTVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:28.257964] [DEBUG] [MAIN ] CTRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:28.258852] [INFO ] [MAIN ] CTRE: Generated 1794 derivative records from 49507 primary records. [2025-10-20T22:07:28.258974] [DEBUG] [LOCAL ] CTRE: Saving derivative data. [2025-10-20T22:07:28.277352] [DEBUG] [LOCAL ] CTRE: Derivative data saved (1794 records). [2025-10-20T22:07:28.277521] [INFO ] [MAIN ] CTRE: Successfully saved derivative data. [2025-10-20T22:07:28.279711] [INFO ] [MAIN ] Derivative Progress: 446/1877 symbols completed. CTRE took 1.1s. Estimated remaining: 8m 59s. Rate: 159.2 symbols/min [2025-10-20T22:07:28.279904] [INFO ] [MAIN ] CUBE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:28.294584] [DEBUG] [MAIN ] CTRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:28.295412] [INFO ] [MAIN ] CTRI: Generated 1794 derivative records from 40518 primary records. [2025-10-20T22:07:28.295639] [DEBUG] [LOCAL ] CTRI: Saving derivative data. [2025-10-20T22:07:28.312460] [DEBUG] [LOCAL ] CTRI: Derivative data saved (1794 records). [2025-10-20T22:07:28.312559] [INFO ] [MAIN ] CTRI: Successfully saved derivative data. [2025-10-20T22:07:28.314811] [INFO ] [MAIN ] Derivative Progress: 447/1877 symbols completed. CTRI took 1.1s. Estimated remaining: 8m 58s. Rate: 159.5 symbols/min [2025-10-20T22:07:28.315002] [INFO ] [MAIN ] CUBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:28.351702] [DEBUG] [LOCAL ] CTVA: Loaded 53302 existing records from primary storage. [2025-10-20T22:07:28.351977] [INFO ] [MAIN ] CTVA: Loaded 53302 primary data records. [2025-10-20T22:07:28.500886] [DEBUG] [LOCAL ] CUBI: Loaded 27276 existing records from primary storage. [2025-10-20T22:07:28.523064] [INFO ] [MAIN ] CUBI: Loaded 27276 primary data records. [2025-10-20T22:07:28.517859] [DEBUG] [LOCAL ] CUBE: Loaded 50888 existing records from primary storage. [2025-10-20T22:07:28.606258] [INFO ] [MAIN ] CUBE: Loaded 50888 primary data records. [2025-10-20T22:07:29.325281] [DEBUG] [MAIN ] CTVA: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:29.326781] [INFO ] [MAIN ] CTVA: Generated 1793 derivative records from 53302 primary records. [2025-10-20T22:07:29.327529] [DEBUG] [LOCAL ] CTVA: Saving derivative data. [2025-10-20T22:07:29.345550] [DEBUG] [LOCAL ] CTVA: Derivative data saved (1793 records). [2025-10-20T22:07:29.345812] [INFO ] [MAIN ] CTVA: Successfully saved derivative data. [2025-10-20T22:07:29.347891] [INFO ] [MAIN ] Derivative Progress: 448/1877 symbols completed. CTVA took 1.1s. Estimated remaining: 8m 60s. Rate: 158.9 symbols/min [2025-10-20T22:07:29.365471] [DEBUG] [MAIN ] CUBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:29.366240] [INFO ] [MAIN ] CUBI: Generated 1794 derivative records from 27276 primary records. [2025-10-20T22:07:29.366379] [DEBUG] [LOCAL ] CUBI: Saving derivative data. [2025-10-20T22:07:29.384636] [DEBUG] [LOCAL ] CUBI: Derivative data saved (1794 records). [2025-10-20T22:07:29.396913] [INFO ] [MAIN ] CUBI: Successfully saved derivative data. [2025-10-20T22:07:29.397172] [DEBUG] [MAIN ] CUBE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:29.399280] [INFO ] [MAIN ] Derivative Progress: 449/1877 symbols completed. CUBI took 1.1s. Estimated remaining: 8m 58s. Rate: 159.2 symbols/min [2025-10-20T22:07:29.400287] [INFO ] [MAIN ] CUBE: Generated 1794 derivative records from 50888 primary records. [2025-10-20T22:07:29.400485] [DEBUG] [LOCAL ] CUBE: Saving derivative data. [2025-10-20T22:07:29.415743] [DEBUG] [LOCAL ] CUBE: Derivative data saved (1794 records). [2025-10-20T22:07:29.415802] [INFO ] [MAIN ] CUBE: Successfully saved derivative data. [2025-10-20T22:07:29.418180] [INFO ] [MAIN ] Derivative Progress: 450/1877 symbols completed. CUBE took 1.1s. Estimated remaining: 8m 57s. Rate: 159.6 symbols/min [2025-10-20T22:07:29.418564] [INFO ] [MAIN ] CUK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:29.418729] [INFO ] [MAIN ] CULP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:29.418665] [INFO ] [MAIN ] CURV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:29.424719] [DEBUG] [LOCAL ] CULP: Loaded 2882 existing records from primary storage. [2025-10-20T22:07:29.425064] [INFO ] [MAIN ] CULP: Loaded 2882 primary data records. [2025-10-20T22:07:29.632308] [DEBUG] [LOCAL ] CURV: Loaded 27263 existing records from primary storage. [2025-10-20T22:07:29.745340] [INFO ] [MAIN ] CURV: Loaded 27263 primary data records. [2025-10-20T22:07:29.777784] [DEBUG] [LOCAL ] CUK: Loaded 51072 existing records from primary storage. [2025-10-20T22:07:29.798275] [INFO ] [MAIN ] CUK: Loaded 51072 primary data records. [2025-10-20T22:07:30.416684] [DEBUG] [MAIN ] CULP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:30.417871] [INFO ] [MAIN ] CULP: Generated 1794 derivative records from 2882 primary records. [2025-10-20T22:07:30.419297] [DEBUG] [LOCAL ] CULP: Saving derivative data. [2025-10-20T22:07:30.437859] [DEBUG] [LOCAL ] CULP: Derivative data saved (1794 records). [2025-10-20T22:07:30.438604] [INFO ] [MAIN ] CULP: Successfully saved derivative data. [2025-10-20T22:07:30.440952] [INFO ] [MAIN ] Derivative Progress: 451/1877 symbols completed. CULP took 1.0s. Estimated remaining: 8m 58s. Rate: 158.9 symbols/min [2025-10-20T22:07:30.441208] [INFO ] [MAIN ] CUZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:30.463509] [DEBUG] [MAIN ] CURV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:30.464867] [INFO ] [MAIN ] CURV: Generated 1794 derivative records from 27263 primary records. [2025-10-20T22:07:30.465962] [DEBUG] [LOCAL ] CURV: Saving derivative data. [2025-10-20T22:07:30.495102] [DEBUG] [MAIN ] CUK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:30.496748] [INFO ] [MAIN ] CUK: Generated 1794 derivative records from 51072 primary records. [2025-10-20T22:07:30.497597] [DEBUG] [LOCAL ] CUK: Saving derivative data. [2025-10-20T22:07:30.510941] [DEBUG] [LOCAL ] CURV: Derivative data saved (1794 records). [2025-10-20T22:07:30.522233] [INFO ] [MAIN ] CURV: Successfully saved derivative data. [2025-10-20T22:07:30.527606] [INFO ] [MAIN ] Derivative Progress: 452/1877 symbols completed. CURV took 1.1s. Estimated remaining: 8m 57s. Rate: 159.2 symbols/min [2025-10-20T22:07:30.528582] [INFO ] [MAIN ] CVE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:30.531928] [DEBUG] [LOCAL ] CUK: Derivative data saved (1794 records). [2025-10-20T22:07:30.532184] [INFO ] [MAIN ] CUK: Successfully saved derivative data. [2025-10-20T22:07:30.534406] [INFO ] [MAIN ] Derivative Progress: 453/1877 symbols completed. CUK took 1.1s. Estimated remaining: 8m 55s. Rate: 159.6 symbols/min [2025-10-20T22:07:30.534609] [INFO ] [MAIN ] CVEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:30.552264] [DEBUG] [LOCAL ] CUZ: Loaded 49575 existing records from primary storage. [2025-10-20T22:07:30.552570] [INFO ] [MAIN ] CUZ: Loaded 49575 primary data records. [2025-10-20T22:07:30.667787] [DEBUG] [LOCAL ] CVEO: Loaded 15328 existing records from primary storage. [2025-10-20T22:07:30.671761] [INFO ] [MAIN ] CVEO: Loaded 15328 primary data records. [2025-10-20T22:07:31.045866] [DEBUG] [LOCAL ] CVE: Loaded 53567 existing records from primary storage. [2025-10-20T22:07:31.089162] [INFO ] [MAIN ] CVE: Loaded 53567 primary data records. [2025-10-20T22:07:31.562676] [DEBUG] [MAIN ] CUZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:31.564414] [INFO ] [MAIN ] CUZ: Generated 1794 derivative records from 49575 primary records. [2025-10-20T22:07:31.564985] [DEBUG] [LOCAL ] CUZ: Saving derivative data. [2025-10-20T22:07:31.584113] [DEBUG] [LOCAL ] CUZ: Derivative data saved (1794 records). [2025-10-20T22:07:31.584300] [INFO ] [MAIN ] CUZ: Successfully saved derivative data. [2025-10-20T22:07:31.586445] [INFO ] [MAIN ] Derivative Progress: 454/1877 symbols completed. CUZ took 1.1s. Estimated remaining: 8m 57s. Rate: 158.9 symbols/min [2025-10-20T22:07:31.586702] [INFO ] [MAIN ] CVI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:31.601581] [DEBUG] [MAIN ] CVEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:31.602742] [INFO ] [MAIN ] CVEO: Generated 1794 derivative records from 15328 primary records. [2025-10-20T22:07:31.603025] [DEBUG] [LOCAL ] CVEO: Saving derivative data. [2025-10-20T22:07:31.620117] [DEBUG] [LOCAL ] CVEO: Derivative data saved (1794 records). [2025-10-20T22:07:31.620382] [INFO ] [MAIN ] CVEO: Successfully saved derivative data. [2025-10-20T22:07:31.622311] [INFO ] [MAIN ] Derivative Progress: 455/1877 symbols completed. CVEO took 1.1s. Estimated remaining: 8m 56s. Rate: 159.3 symbols/min [2025-10-20T22:07:31.622596] [INFO ] [MAIN ] CVLG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:31.639922] [DEBUG] [MAIN ] CVE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:31.640804] [INFO ] [MAIN ] CVE: Generated 1794 derivative records from 53567 primary records. [2025-10-20T22:07:31.640917] [DEBUG] [LOCAL ] CVE: Saving derivative data. [2025-10-20T22:07:31.677339] [DEBUG] [LOCAL ] CVE: Derivative data saved (1794 records). [2025-10-20T22:07:31.677898] [INFO ] [MAIN ] CVE: Successfully saved derivative data. [2025-10-20T22:07:31.680564] [INFO ] [MAIN ] Derivative Progress: 456/1877 symbols completed. CVE took 1.2s. Estimated remaining: 8m 54s. Rate: 159.5 symbols/min [2025-10-20T22:07:31.680816] [INFO ] [MAIN ] CVNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:31.677769] [DEBUG] [LOCAL ] CVLG: Loaded 15478 existing records from primary storage. [2025-10-20T22:07:31.681370] [INFO ] [MAIN ] CVLG: Loaded 15478 primary data records. [2025-10-20T22:07:31.801840] [DEBUG] [LOCAL ] CVI: Loaded 46745 existing records from primary storage. [2025-10-20T22:07:31.813151] [INFO ] [MAIN ] CVI: Loaded 46745 primary data records. [2025-10-20T22:07:32.357887] [DEBUG] [LOCAL ] CVNA: Loaded 52842 existing records from primary storage. [2025-10-20T22:07:32.358322] [INFO ] [MAIN ] CVNA: Loaded 52842 primary data records. [2025-10-20T22:07:32.679170] [DEBUG] [MAIN ] CVLG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:32.681672] [INFO ] [MAIN ] CVLG: Generated 1794 derivative records from 15478 primary records. [2025-10-20T22:07:32.682032] [DEBUG] [LOCAL ] CVLG: Saving derivative data. [2025-10-20T22:07:32.710281] [DEBUG] [LOCAL ] CVLG: Derivative data saved (1794 records). [2025-10-20T22:07:32.716083] [INFO ] [MAIN ] CVLG: Successfully saved derivative data. [2025-10-20T22:07:32.716011] [DEBUG] [MAIN ] CVI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:32.718355] [INFO ] [MAIN ] Derivative Progress: 457/1877 symbols completed. CVLG took 1.1s. Estimated remaining: 8m 56s. Rate: 158.9 symbols/min [2025-10-20T22:07:32.719608] [INFO ] [MAIN ] CVI: Generated 1794 derivative records from 46745 primary records. [2025-10-20T22:07:32.719974] [DEBUG] [LOCAL ] CVI: Saving derivative data. [2025-10-20T22:07:32.720168] [INFO ] [MAIN ] CVS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:32.751992] [DEBUG] [MAIN ] CVNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:32.753415] [DEBUG] [LOCAL ] CVI: Derivative data saved (1794 records). [2025-10-20T22:07:32.754274] [INFO ] [MAIN ] CVNA: Generated 1794 derivative records from 52842 primary records. [2025-10-20T22:07:32.753543] [INFO ] [MAIN ] CVI: Successfully saved derivative data. [2025-10-20T22:07:32.756526] [INFO ] [MAIN ] Derivative Progress: 458/1877 symbols completed. CVI took 1.2s. Estimated remaining: 8m 55s. Rate: 159.2 symbols/min [2025-10-20T22:07:32.756608] [DEBUG] [LOCAL ] CVNA: Saving derivative data. [2025-10-20T22:07:32.756864] [INFO ] [MAIN ] CVX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:32.776060] [DEBUG] [LOCAL ] CVNA: Derivative data saved (1794 records). [2025-10-20T22:07:32.776250] [INFO ] [MAIN ] CVNA: Successfully saved derivative data. [2025-10-20T22:07:32.778341] [INFO ] [MAIN ] Derivative Progress: 459/1877 symbols completed. CVNA took 1.1s. Estimated remaining: 8m 53s. Rate: 159.6 symbols/min [2025-10-20T22:07:32.778479] [INFO ] [MAIN ] CW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:32.829686] [DEBUG] [LOCAL ] CW: Loaded 22034 existing records from primary storage. [2025-10-20T22:07:32.829818] [INFO ] [MAIN ] CW: Loaded 22034 primary data records. [2025-10-20T22:07:32.851554] [DEBUG] [LOCAL ] CVS: Loaded 53694 existing records from primary storage. [2025-10-20T22:07:32.865334] [INFO ] [MAIN ] CVS: Loaded 53694 primary data records. [2025-10-20T22:07:33.080535] [DEBUG] [LOCAL ] CVX: Loaded 53760 existing records from primary storage. [2025-10-20T22:07:33.102450] [INFO ] [MAIN ] CVX: Loaded 53760 primary data records. [2025-10-20T22:07:33.858802] [DEBUG] [MAIN ] CVS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:33.859899] [INFO ] [MAIN ] CVS: Generated 1794 derivative records from 53694 primary records. [2025-10-20T22:07:33.860198] [DEBUG] [LOCAL ] CVS: Saving derivative data. [2025-10-20T22:07:33.874488] [DEBUG] [MAIN ] CW: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:33.883111] [INFO ] [MAIN ] CW: Generated 1793 derivative records from 22034 primary records. [2025-10-20T22:07:33.887683] [DEBUG] [LOCAL ] CW: Saving derivative data. [2025-10-20T22:07:33.892178] [DEBUG] [LOCAL ] CVS: Derivative data saved (1794 records). [2025-10-20T22:07:33.892526] [INFO ] [MAIN ] CVS: Successfully saved derivative data. [2025-10-20T22:07:33.894858] [INFO ] [MAIN ] Derivative Progress: 460/1877 symbols completed. CVS took 1.2s. Estimated remaining: 8m 55s. Rate: 158.9 symbols/min [2025-10-20T22:07:33.905690] [INFO ] [MAIN ] CWAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:33.913960] [DEBUG] [LOCAL ] CW: Derivative data saved (1793 records). [2025-10-20T22:07:33.914214] [INFO ] [MAIN ] CW: Successfully saved derivative data. [2025-10-20T22:07:33.916207] [INFO ] [MAIN ] Derivative Progress: 461/1877 symbols completed. CW took 1.1s. Estimated remaining: 8m 54s. Rate: 159.2 symbols/min [2025-10-20T22:07:33.916449] [INFO ] [MAIN ] CWEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:33.931020] [DEBUG] [MAIN ] CVX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:33.931743] [INFO ] [MAIN ] CVX: Generated 1794 derivative records from 53760 primary records. [2025-10-20T22:07:33.931847] [DEBUG] [LOCAL ] CVX: Saving derivative data. [2025-10-20T22:07:33.949254] [DEBUG] [LOCAL ] CVX: Derivative data saved (1794 records). [2025-10-20T22:07:33.949459] [INFO ] [MAIN ] CVX: Successfully saved derivative data. [2025-10-20T22:07:33.951355] [INFO ] [MAIN ] Derivative Progress: 462/1877 symbols completed. CVX took 1.2s. Estimated remaining: 8m 52s. Rate: 159.5 symbols/min [2025-10-20T22:07:33.951620] [INFO ] [MAIN ] CWH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:34.000682] [DEBUG] [LOCAL ] CWAN: Loaded 52668 existing records from primary storage. [2025-10-20T22:07:34.000917] [INFO ] [MAIN ] CWAN: Loaded 52668 primary data records. [2025-10-20T22:07:34.135006] [DEBUG] [LOCAL ] CWEN: Loaded 45314 existing records from primary storage. [2025-10-20T22:07:34.150960] [INFO ] [MAIN ] CWEN: Loaded 45314 primary data records. [2025-10-20T22:07:34.316910] [DEBUG] [LOCAL ] CWH: Loaded 51314 existing records from primary storage. [2025-10-20T22:07:34.323009] [INFO ] [MAIN ] CWH: Loaded 51314 primary data records. [2025-10-20T22:07:34.977411] [DEBUG] [MAIN ] CWAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:34.990461] [INFO ] [MAIN ] CWAN: Generated 1794 derivative records from 52668 primary records. [2025-10-20T22:07:34.990771] [DEBUG] [LOCAL ] CWAN: Saving derivative data. [2025-10-20T22:07:34.992334] [DEBUG] [MAIN ] CWEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:34.993129] [INFO ] [MAIN ] CWEN: Generated 1794 derivative records from 45314 primary records. [2025-10-20T22:07:35.004354] [DEBUG] [LOCAL ] CWEN: Saving derivative data. [2025-10-20T22:07:35.009793] [DEBUG] [LOCAL ] CWAN: Derivative data saved (1794 records). [2025-10-20T22:07:35.016923] [INFO ] [MAIN ] CWAN: Successfully saved derivative data. [2025-10-20T22:07:35.026674] [INFO ] [MAIN ] Derivative Progress: 463/1877 symbols completed. CWAN took 1.1s. Estimated remaining: 8m 54s. Rate: 158.9 symbols/min [2025-10-20T22:07:35.028379] [DEBUG] [LOCAL ] CWEN: Derivative data saved (1794 records). [2025-10-20T22:07:35.029217] [INFO ] [MAIN ] CWEN: Successfully saved derivative data. [2025-10-20T22:07:35.032282] [INFO ] [MAIN ] Derivative Progress: 464/1877 symbols completed. CWEN took 1.1s. Estimated remaining: 8m 52s. Rate: 159.2 symbols/min [2025-10-20T22:07:35.047153] [DEBUG] [MAIN ] CWH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:35.047722] [INFO ] [MAIN ] CWH: Generated 1794 derivative records from 51314 primary records. [2025-10-20T22:07:35.047783] [DEBUG] [LOCAL ] CWH: Saving derivative data. [2025-10-20T22:07:35.062739] [DEBUG] [LOCAL ] CWH: Derivative data saved (1794 records). [2025-10-20T22:07:35.062795] [INFO ] [MAIN ] CWH: Successfully saved derivative data. [2025-10-20T22:07:35.065523] [INFO ] [MAIN ] Derivative Progress: 465/1877 symbols completed. CWH took 1.1s. Estimated remaining: 8m 51s. Rate: 159.5 symbols/min [2025-10-20T22:07:35.066080] [INFO ] [MAIN ] CWK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:35.066252] [INFO ] [MAIN ] CX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:35.066750] [INFO ] [MAIN ] CXE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:35.076715] [DEBUG] [LOCAL ] CXE: Loaded 6342 existing records from primary storage. [2025-10-20T22:07:35.076984] [INFO ] [MAIN ] CXE: Loaded 6342 primary data records. [2025-10-20T22:07:35.410046] [DEBUG] [LOCAL ] CWK: Loaded 49668 existing records from primary storage. [2025-10-20T22:07:35.410457] [INFO ] [MAIN ] CWK: Loaded 49668 primary data records. [2025-10-20T22:07:35.543314] [DEBUG] [LOCAL ] CX: Loaded 52835 existing records from primary storage. [2025-10-20T22:07:35.570410] [INFO ] [MAIN ] CX: Loaded 52835 primary data records. [2025-10-20T22:07:35.921527] [DEBUG] [MAIN ] CXE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:35.945497] [INFO ] [MAIN ] CXE: Generated 1794 derivative records from 6342 primary records. [2025-10-20T22:07:36.049814] [DEBUG] [LOCAL ] CXE: Saving derivative data. [2025-10-20T22:07:36.123332] [DEBUG] [LOCAL ] CXE: Derivative data saved (1794 records). [2025-10-20T22:07:36.123680] [INFO ] [MAIN ] CXE: Successfully saved derivative data. [2025-10-20T22:07:36.125863] [INFO ] [MAIN ] Derivative Progress: 466/1877 symbols completed. CXE took 1.1s. Estimated remaining: 8m 53s. Rate: 158.9 symbols/min [2025-10-20T22:07:36.126058] [INFO ] [MAIN ] CXH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:36.137433] [DEBUG] [LOCAL ] CXH: Loaded 4421 existing records from primary storage. [2025-10-20T22:07:36.150061] [INFO ] [MAIN ] CXH: Loaded 4421 primary data records. [2025-10-20T22:07:36.162128] [DEBUG] [MAIN ] CWK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:36.182599] [INFO ] [MAIN ] CWK: Generated 1794 derivative records from 49668 primary records. [2025-10-20T22:07:36.188190] [DEBUG] [LOCAL ] CWK: Saving derivative data. [2025-10-20T22:07:36.273274] [DEBUG] [LOCAL ] CWK: Derivative data saved (1794 records). [2025-10-20T22:07:36.289154] [INFO ] [MAIN ] CWK: Successfully saved derivative data. [2025-10-20T22:07:36.315014] [INFO ] [MAIN ] Derivative Progress: 467/1877 symbols completed. CWK took 1.2s. Estimated remaining: 8m 52s. Rate: 159.1 symbols/min [2025-10-20T22:07:36.315213] [INFO ] [MAIN ] CXM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:36.491150] [DEBUG] [MAIN ] CX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:36.493207] [INFO ] [MAIN ] CX: Generated 1794 derivative records from 52835 primary records. [2025-10-20T22:07:36.493842] [DEBUG] [LOCAL ] CX: Saving derivative data. [2025-10-20T22:07:36.512699] [DEBUG] [LOCAL ] CX: Derivative data saved (1794 records). [2025-10-20T22:07:36.526060] [INFO ] [MAIN ] CX: Successfully saved derivative data. [2025-10-20T22:07:36.528394] [INFO ] [MAIN ] Derivative Progress: 468/1877 symbols completed. CX took 1.5s. Estimated remaining: 8m 51s. Rate: 159.2 symbols/min [2025-10-20T22:07:36.528952] [INFO ] [MAIN ] CXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:36.572774] [DEBUG] [MAIN ] CXH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:36.572450] [DEBUG] [LOCAL ] CXM: Loaded 47532 existing records from primary storage. [2025-10-20T22:07:36.572911] [INFO ] [MAIN ] CXM: Loaded 47532 primary data records. [2025-10-20T22:07:36.573692] [INFO ] [MAIN ] CXH: Generated 1794 derivative records from 4421 primary records. [2025-10-20T22:07:36.573782] [DEBUG] [LOCAL ] CXH: Saving derivative data. [2025-10-20T22:07:36.705605] [DEBUG] [LOCAL ] CXT: Loaded 30937 existing records from primary storage. [2025-10-20T22:07:36.709220] [INFO ] [MAIN ] CXT: Loaded 30937 primary data records. [2025-10-20T22:07:36.709145] [DEBUG] [LOCAL ] CXH: Derivative data saved (1794 records). [2025-10-20T22:07:36.709491] [INFO ] [MAIN ] CXH: Successfully saved derivative data. [2025-10-20T22:07:36.711533] [INFO ] [MAIN ] Derivative Progress: 469/1877 symbols completed. CXH took 0.6s. Estimated remaining: 8m 50s. Rate: 159.4 symbols/min [2025-10-20T22:07:36.756004] [INFO ] [MAIN ] CXW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:37.280946] [DEBUG] [LOCAL ] CXW: Loaded 43775 existing records from primary storage. [2025-10-20T22:07:37.281696] [INFO ] [MAIN ] CXW: Loaded 43775 primary data records. [2025-10-20T22:07:37.411196] [DEBUG] [MAIN ] CXM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:37.424409] [INFO ] [MAIN ] CXM: Generated 1794 derivative records from 47532 primary records. [2025-10-20T22:07:37.431021] [DEBUG] [LOCAL ] CXM: Saving derivative data. [2025-10-20T22:07:37.700151] [DEBUG] [MAIN ] CXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:37.704152] [DEBUG] [LOCAL ] CXM: Derivative data saved (1794 records). [2025-10-20T22:07:37.704433] [INFO ] [MAIN ] CXM: Successfully saved derivative data. [2025-10-20T22:07:37.705322] [INFO ] [MAIN ] CXT: Generated 1794 derivative records from 30937 primary records. [2025-10-20T22:07:37.709089] [DEBUG] [LOCAL ] CXT: Saving derivative data. [2025-10-20T22:07:37.708567] [INFO ] [MAIN ] Derivative Progress: 470/1877 symbols completed. CXM took 1.4s. Estimated remaining: 8m 51s. Rate: 158.9 symbols/min [2025-10-20T22:07:37.711038] [INFO ] [MAIN ] CYD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:37.737890] [DEBUG] [LOCAL ] CXT: Derivative data saved (1794 records). [2025-10-20T22:07:37.738137] [INFO ] [MAIN ] CXT: Successfully saved derivative data. [2025-10-20T22:07:37.741244] [INFO ] [MAIN ] Derivative Progress: 471/1877 symbols completed. CXT took 1.2s. Estimated remaining: 8m 50s. Rate: 159.2 symbols/min [2025-10-20T22:07:37.741764] [INFO ] [MAIN ] CYH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:37.790262] [DEBUG] [LOCAL ] CYD: Loaded 27167 existing records from primary storage. [2025-10-20T22:07:37.791008] [INFO ] [MAIN ] CYD: Loaded 27167 primary data records. [2025-10-20T22:07:37.790748] [DEBUG] [MAIN ] CXW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:37.797680] [INFO ] [MAIN ] CXW: Generated 1794 derivative records from 43775 primary records. [2025-10-20T22:07:37.843777] [DEBUG] [LOCAL ] CXW: Saving derivative data. [2025-10-20T22:07:37.995211] [DEBUG] [LOCAL ] CYH: Loaded 47742 existing records from primary storage. [2025-10-20T22:07:38.001607] [INFO ] [MAIN ] CYH: Loaded 47742 primary data records. [2025-10-20T22:07:38.007173] [DEBUG] [LOCAL ] CXW: Derivative data saved (1794 records). [2025-10-20T22:07:38.012502] [INFO ] [MAIN ] CXW: Successfully saved derivative data. [2025-10-20T22:07:38.015109] [INFO ] [MAIN ] Derivative Progress: 472/1877 symbols completed. CXW took 1.3s. Estimated remaining: 8m 49s. Rate: 159.3 symbols/min [2025-10-20T22:07:38.015280] [INFO ] [MAIN ] D: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:38.583246] [DEBUG] [LOCAL ] D: Loaded 53649 existing records from primary storage. [2025-10-20T22:07:38.585791] [INFO ] [MAIN ] D: Loaded 53649 primary data records. [2025-10-20T22:07:38.682790] [DEBUG] [MAIN ] CYD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:38.707840] [INFO ] [MAIN ] CYD: Generated 1794 derivative records from 27167 primary records. [2025-10-20T22:07:38.725333] [DEBUG] [LOCAL ] CYD: Saving derivative data. [2025-10-20T22:07:38.821721] [DEBUG] [LOCAL ] CYD: Derivative data saved (1794 records). [2025-10-20T22:07:38.934882] [INFO ] [MAIN ] CYD: Successfully saved derivative data. [2025-10-20T22:07:38.934783] [DEBUG] [MAIN ] CYH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:38.937705] [INFO ] [MAIN ] CYH: Generated 1794 derivative records from 47742 primary records. [2025-10-20T22:07:38.936851] [INFO ] [MAIN ] Derivative Progress: 473/1877 symbols completed. CYD took 1.2s. Estimated remaining: 8m 51s. Rate: 158.8 symbols/min [2025-10-20T22:07:38.937907] [INFO ] [MAIN ] DAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:38.938017] [DEBUG] [LOCAL ] CYH: Saving derivative data. [2025-10-20T22:07:38.963061] [DEBUG] [LOCAL ] CYH: Derivative data saved (1794 records). [2025-10-20T22:07:38.963785] [INFO ] [MAIN ] CYH: Successfully saved derivative data. [2025-10-20T22:07:38.966391] [INFO ] [MAIN ] Derivative Progress: 474/1877 symbols completed. CYH took 1.2s. Estimated remaining: 8m 49s. Rate: 159.1 symbols/min [2025-10-20T22:07:38.967038] [INFO ] [MAIN ] DAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:38.974171] [DEBUG] [LOCAL ] DAC: Loaded 12590 existing records from primary storage. [2025-10-20T22:07:38.974367] [INFO ] [MAIN ] DAC: Loaded 12590 primary data records. [2025-10-20T22:07:39.320280] [DEBUG] [MAIN ] D: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:39.321904] [INFO ] [MAIN ] D: Generated 1794 derivative records from 53649 primary records. [2025-10-20T22:07:39.322040] [DEBUG] [LOCAL ] D: Saving derivative data. [2025-10-20T22:07:39.354048] [DEBUG] [LOCAL ] DAL: Loaded 53753 existing records from primary storage. [2025-10-20T22:07:39.355629] [INFO ] [MAIN ] DAL: Loaded 53753 primary data records. [2025-10-20T22:07:39.355139] [DEBUG] [LOCAL ] D: Derivative data saved (1794 records). [2025-10-20T22:07:39.356045] [INFO ] [MAIN ] D: Successfully saved derivative data. [2025-10-20T22:07:39.389001] [INFO ] [MAIN ] Derivative Progress: 475/1877 symbols completed. D took 1.4s. Estimated remaining: 8m 49s. Rate: 159.0 symbols/min [2025-10-20T22:07:39.394311] [INFO ] [MAIN ] DAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:39.686588] [DEBUG] [MAIN ] DAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:39.688034] [INFO ] [MAIN ] DAC: Generated 1794 derivative records from 12590 primary records. [2025-10-20T22:07:39.689480] [DEBUG] [LOCAL ] DAC: Saving derivative data. [2025-10-20T22:07:39.717544] [DEBUG] [LOCAL ] DAN: Loaded 49407 existing records from primary storage. [2025-10-20T22:07:39.722024] [INFO ] [MAIN ] DAN: Loaded 49407 primary data records. [2025-10-20T22:07:39.721739] [DEBUG] [LOCAL ] DAC: Derivative data saved (1794 records). [2025-10-20T22:07:39.722361] [INFO ] [MAIN ] DAC: Successfully saved derivative data. [2025-10-20T22:07:39.736442] [INFO ] [MAIN ] Derivative Progress: 476/1877 symbols completed. DAC took 0.8s. Estimated remaining: 8m 48s. Rate: 159.1 symbols/min [2025-10-20T22:07:39.762495] [INFO ] [MAIN ] DAO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:39.946363] [DEBUG] [MAIN ] DAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:40.094478] [DEBUG] [LOCAL ] DAO: Loaded 19694 existing records from primary storage. [2025-10-20T22:07:40.103918] [INFO ] [MAIN ] DAL: Generated 1794 derivative records from 53753 primary records. [2025-10-20T22:07:40.104923] [INFO ] [MAIN ] DAO: Loaded 19694 primary data records. [2025-10-20T22:07:40.105022] [DEBUG] [LOCAL ] DAL: Saving derivative data. [2025-10-20T22:07:40.211998] [DEBUG] [LOCAL ] DAL: Derivative data saved (1794 records). [2025-10-20T22:07:40.225620] [INFO ] [MAIN ] DAL: Successfully saved derivative data. [2025-10-20T22:07:40.231556] [INFO ] [MAIN ] Derivative Progress: 477/1877 symbols completed. DAL took 1.3s. Estimated remaining: 8m 48s. Rate: 159.0 symbols/min [2025-10-20T22:07:40.243068] [INFO ] [MAIN ] DAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:40.446706] [DEBUG] [MAIN ] DAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:40.448123] [INFO ] [MAIN ] DAN: Generated 1794 derivative records from 49407 primary records. [2025-10-20T22:07:40.448225] [DEBUG] [LOCAL ] DAN: Saving derivative data. [2025-10-20T22:07:40.480393] [DEBUG] [LOCAL ] DAN: Derivative data saved (1794 records). [2025-10-20T22:07:40.480836] [INFO ] [MAIN ] DAN: Successfully saved derivative data. [2025-10-20T22:07:40.483434] [INFO ] [MAIN ] Derivative Progress: 478/1877 symbols completed. DAN took 1.1s. Estimated remaining: 8m 48s. Rate: 159.1 symbols/min [2025-10-20T22:07:40.504897] [DEBUG] [LOCAL ] DAR: Loaded 51998 existing records from primary storage. [2025-10-20T22:07:40.505225] [INFO ] [MAIN ] DAR: Loaded 51998 primary data records. [2025-10-20T22:07:40.561359] [DEBUG] [MAIN ] DAO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:40.593841] [INFO ] [MAIN ] DAO: Generated 1794 derivative records from 19694 primary records. [2025-10-20T22:07:40.599058] [DEBUG] [LOCAL ] DAO: Saving derivative data. [2025-10-20T22:07:40.684411] [DEBUG] [LOCAL ] DAO: Derivative data saved (1794 records). [2025-10-20T22:07:40.692069] [INFO ] [MAIN ] DAO: Successfully saved derivative data. [2025-10-20T22:07:40.695803] [INFO ] [MAIN ] Derivative Progress: 479/1877 symbols completed. DAO took 0.9s. Estimated remaining: 8m 47s. Rate: 159.2 symbols/min [2025-10-20T22:07:40.845992] [DEBUG] [MAIN ] DAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:40.846630] [INFO ] [MAIN ] DAR: Generated 1794 derivative records from 51998 primary records. [2025-10-20T22:07:40.846718] [DEBUG] [LOCAL ] DAR: Saving derivative data. [2025-10-20T22:07:40.862069] [DEBUG] [LOCAL ] DAR: Derivative data saved (1794 records). [2025-10-20T22:07:40.862156] [INFO ] [MAIN ] DAR: Successfully saved derivative data. [2025-10-20T22:07:40.864120] [INFO ] [MAIN ] Derivative Progress: 480/1877 symbols completed. DAR took 0.6s. Estimated remaining: 8m 46s. Rate: 159.4 symbols/min [2025-10-20T22:07:40.864776] [INFO ] [MAIN ] DAVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:40.865065] [INFO ] [MAIN ] DB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:40.864913] [INFO ] [MAIN ] DAY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:40.920234] [DEBUG] [LOCAL ] DAY: Loaded 49167 existing records from primary storage. [2025-10-20T22:07:40.920379] [INFO ] [MAIN ] DAY: Loaded 49167 primary data records. [2025-10-20T22:07:41.053393] [DEBUG] [LOCAL ] DAVA: Loaded 41996 existing records from primary storage. [2025-10-20T22:07:41.118863] [DEBUG] [LOCAL ] DB: Loaded 52484 existing records from primary storage. [2025-10-20T22:07:41.109227] [INFO ] [MAIN ] DAVA: Loaded 41996 primary data records. [2025-10-20T22:07:41.148187] [INFO ] [MAIN ] DB: Loaded 52484 primary data records. [2025-10-20T22:07:41.910760] [DEBUG] [MAIN ] DAY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:41.911815] [INFO ] [MAIN ] DAY: Generated 1794 derivative records from 49167 primary records. [2025-10-20T22:07:41.912315] [DEBUG] [LOCAL ] DAY: Saving derivative data. [2025-10-20T22:07:41.930462] [DEBUG] [LOCAL ] DAY: Derivative data saved (1794 records). [2025-10-20T22:07:41.930774] [INFO ] [MAIN ] DAY: Successfully saved derivative data. [2025-10-20T22:07:41.932743] [INFO ] [MAIN ] Derivative Progress: 481/1877 symbols completed. DAY took 1.1s. Estimated remaining: 8m 47s. Rate: 158.8 symbols/min [2025-10-20T22:07:41.933087] [INFO ] [MAIN ] DBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:41.950361] [DEBUG] [MAIN ] DAVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:41.951173] [INFO ] [MAIN ] DAVA: Generated 1794 derivative records from 41996 primary records. [2025-10-20T22:07:41.951389] [DEBUG] [LOCAL ] DAVA: Saving derivative data. [2025-10-20T22:07:41.983544] [DEBUG] [MAIN ] DB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:41.983133] [DEBUG] [LOCAL ] DAVA: Derivative data saved (1794 records). [2025-10-20T22:07:41.984447] [INFO ] [MAIN ] DB: Generated 1794 derivative records from 52484 primary records. [2025-10-20T22:07:41.984583] [INFO ] [MAIN ] DAVA: Successfully saved derivative data. [2025-10-20T22:07:41.987083] [INFO ] [MAIN ] Derivative Progress: 482/1877 symbols completed. DAVA took 1.1s. Estimated remaining: 8m 46s. Rate: 159.1 symbols/min [2025-10-20T22:07:41.987310] [DEBUG] [LOCAL ] DB: Saving derivative data. [2025-10-20T22:07:41.987410] [INFO ] [MAIN ] DBRG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:42.014393] [DEBUG] [LOCAL ] DB: Derivative data saved (1794 records). [2025-10-20T22:07:42.014589] [INFO ] [MAIN ] DB: Successfully saved derivative data. [2025-10-20T22:07:42.016680] [INFO ] [MAIN ] Derivative Progress: 483/1877 symbols completed. DB took 1.2s. Estimated remaining: 8m 45s. Rate: 159.4 symbols/min [2025-10-20T22:07:42.016874] [INFO ] [MAIN ] DCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:42.031450] [DEBUG] [LOCAL ] DBI: Loaded 43623 existing records from primary storage. [2025-10-20T22:07:42.031704] [INFO ] [MAIN ] DBI: Loaded 43623 primary data records. [2025-10-20T22:07:42.145059] [DEBUG] [LOCAL ] DCO: Loaded 14765 existing records from primary storage. [2025-10-20T22:07:42.157360] [INFO ] [MAIN ] DCO: Loaded 14765 primary data records. [2025-10-20T22:07:42.490617] [DEBUG] [LOCAL ] DBRG: Loaded 49985 existing records from primary storage. [2025-10-20T22:07:42.490872] [INFO ] [MAIN ] DBRG: Loaded 49985 primary data records. [2025-10-20T22:07:43.024570] [DEBUG] [MAIN ] DBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:43.025357] [INFO ] [MAIN ] DBI: Generated 1794 derivative records from 43623 primary records. [2025-10-20T22:07:43.025447] [DEBUG] [LOCAL ] DBI: Saving derivative data. [2025-10-20T22:07:43.043480] [DEBUG] [LOCAL ] DBI: Derivative data saved (1794 records). [2025-10-20T22:07:43.044518] [INFO ] [MAIN ] DBI: Successfully saved derivative data. [2025-10-20T22:07:43.046548] [INFO ] [MAIN ] Derivative Progress: 484/1877 symbols completed. DBI took 1.1s. Estimated remaining: 8m 46s. Rate: 158.8 symbols/min [2025-10-20T22:07:43.046856] [INFO ] [MAIN ] DD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:43.061239] [DEBUG] [MAIN ] DCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:43.062086] [INFO ] [MAIN ] DCO: Generated 1794 derivative records from 14765 primary records. [2025-10-20T22:07:43.062177] [DEBUG] [LOCAL ] DCO: Saving derivative data. [2025-10-20T22:07:43.092916] [DEBUG] [MAIN ] DBRG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:43.093396] [DEBUG] [LOCAL ] DCO: Derivative data saved (1794 records). [2025-10-20T22:07:43.093560] [INFO ] [MAIN ] DCO: Successfully saved derivative data. [2025-10-20T22:07:43.099175] [INFO ] [MAIN ] DBRG: Generated 1794 derivative records from 49985 primary records. [2025-10-20T22:07:43.098104] [INFO ] [MAIN ] Derivative Progress: 485/1877 symbols completed. DCO took 1.1s. Estimated remaining: 8m 45s. Rate: 159.1 symbols/min [2025-10-20T22:07:43.099496] [INFO ] [MAIN ] DDD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:43.099671] [DEBUG] [LOCAL ] DBRG: Saving derivative data. [2025-10-20T22:07:43.116856] [DEBUG] [LOCAL ] DBRG: Derivative data saved (1794 records). [2025-10-20T22:07:43.117082] [INFO ] [MAIN ] DBRG: Successfully saved derivative data. [2025-10-20T22:07:43.119156] [INFO ] [MAIN ] Derivative Progress: 486/1877 symbols completed. DBRG took 1.1s. Estimated remaining: 8m 44s. Rate: 159.4 symbols/min [2025-10-20T22:07:43.119332] [INFO ] [MAIN ] DDL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:43.158045] [DEBUG] [LOCAL ] DD: Loaded 52934 existing records from primary storage. [2025-10-20T22:07:43.178273] [INFO ] [MAIN ] DD: Loaded 52934 primary data records. [2025-10-20T22:07:43.311854] [DEBUG] [LOCAL ] DDL: Loaded 37211 existing records from primary storage. [2025-10-20T22:07:43.323554] [DEBUG] [LOCAL ] DDD: Loaded 50939 existing records from primary storage. [2025-10-20T22:07:43.323706] [INFO ] [MAIN ] DDL: Loaded 37211 primary data records. [2025-10-20T22:07:43.323838] [INFO ] [MAIN ] DDD: Loaded 50939 primary data records. [2025-10-20T22:07:44.196598] [DEBUG] [MAIN ] DD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:44.199263] [INFO ] [MAIN ] DD: Generated 1794 derivative records from 52934 primary records. [2025-10-20T22:07:44.199447] [DEBUG] [LOCAL ] DD: Saving derivative data. [2025-10-20T22:07:44.232964] [DEBUG] [LOCAL ] DD: Derivative data saved (1794 records). [2025-10-20T22:07:44.235808] [INFO ] [MAIN ] DD: Successfully saved derivative data. [2025-10-20T22:07:44.234808] [DEBUG] [MAIN ] DDL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:44.238925] [INFO ] [MAIN ] DDL: Generated 1794 derivative records from 37211 primary records. [2025-10-20T22:07:44.237952] [INFO ] [MAIN ] Derivative Progress: 487/1877 symbols completed. DD took 1.2s. Estimated remaining: 8m 45s. Rate: 158.8 symbols/min [2025-10-20T22:07:44.247146] [DEBUG] [LOCAL ] DDL: Saving derivative data. [2025-10-20T22:07:44.247274] [INFO ] [MAIN ] DDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:44.253728] [DEBUG] [MAIN ] DDD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:44.255291] [INFO ] [MAIN ] DDD: Generated 1794 derivative records from 50939 primary records. [2025-10-20T22:07:44.262484] [DEBUG] [LOCAL ] DDD: Saving derivative data. [2025-10-20T22:07:44.271302] [DEBUG] [LOCAL ] DDL: Derivative data saved (1794 records). [2025-10-20T22:07:44.271908] [INFO ] [MAIN ] DDL: Successfully saved derivative data. [2025-10-20T22:07:44.281129] [INFO ] [MAIN ] Derivative Progress: 488/1877 symbols completed. DDL took 1.2s. Estimated remaining: 8m 44s. Rate: 159.1 symbols/min [2025-10-20T22:07:44.287981] [INFO ] [MAIN ] DEI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:44.289129] [DEBUG] [LOCAL ] DDD: Derivative data saved (1794 records). [2025-10-20T22:07:44.289291] [INFO ] [MAIN ] DDD: Successfully saved derivative data. [2025-10-20T22:07:44.294129] [DEBUG] [LOCAL ] DDT: Loaded 1978 existing records from primary storage. [2025-10-20T22:07:44.293866] [INFO ] [MAIN ] Derivative Progress: 489/1877 symbols completed. DDD took 1.2s. Estimated remaining: 8m 43s. Rate: 159.4 symbols/min [2025-10-20T22:07:44.294514] [INFO ] [MAIN ] DDT: Loaded 1978 primary data records. [2025-10-20T22:07:44.294435] [INFO ] [MAIN ] DELL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:44.628424] [DEBUG] [LOCAL ] DEI: Loaded 48313 existing records from primary storage. [2025-10-20T22:07:44.629329] [INFO ] [MAIN ] DEI: Loaded 48313 primary data records. [2025-10-20T22:07:44.758852] [DEBUG] [LOCAL ] DELL: Loaded 53736 existing records from primary storage. [2025-10-20T22:07:44.786253] [INFO ] [MAIN ] DELL: Loaded 53736 primary data records. [2025-10-20T22:07:45.076963] [DEBUG] [MAIN ] DDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:45.079099] [INFO ] [MAIN ] DDT: Generated 1794 derivative records from 1978 primary records. [2025-10-20T22:07:45.095281] [DEBUG] [LOCAL ] DDT: Saving derivative data. [2025-10-20T22:07:45.288697] [DEBUG] [LOCAL ] DDT: Derivative data saved (1794 records). [2025-10-20T22:07:45.289174] [INFO ] [MAIN ] DDT: Successfully saved derivative data. [2025-10-20T22:07:45.291199] [INFO ] [MAIN ] Derivative Progress: 490/1877 symbols completed. DDT took 1.0s. Estimated remaining: 8m 44s. Rate: 158.8 symbols/min [2025-10-20T22:07:45.291566] [INFO ] [MAIN ] DEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:45.315938] [DEBUG] [MAIN ] DEI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:45.317065] [INFO ] [MAIN ] DEI: Generated 1794 derivative records from 48313 primary records. [2025-10-20T22:07:45.317351] [DEBUG] [LOCAL ] DEI: Saving derivative data. [2025-10-20T22:07:45.337825] [DEBUG] [LOCAL ] DEI: Derivative data saved (1794 records). [2025-10-20T22:07:45.338049] [INFO ] [MAIN ] DEI: Successfully saved derivative data. [2025-10-20T22:07:45.339948] [INFO ] [MAIN ] Derivative Progress: 491/1877 symbols completed. DEI took 1.1s. Estimated remaining: 8m 43s. Rate: 159.1 symbols/min [2025-10-20T22:07:45.352749] [INFO ] [MAIN ] DFP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:45.353557] [DEBUG] [MAIN ] DELL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:45.354421] [INFO ] [MAIN ] DELL: Generated 1794 derivative records from 53736 primary records. [2025-10-20T22:07:45.354536] [DEBUG] [LOCAL ] DELL: Saving derivative data. [2025-10-20T22:07:45.386121] [DEBUG] [LOCAL ] DELL: Derivative data saved (1794 records). [2025-10-20T22:07:45.386220] [INFO ] [MAIN ] DELL: Successfully saved derivative data. [2025-10-20T22:07:45.388184] [INFO ] [MAIN ] Derivative Progress: 492/1877 symbols completed. DELL took 1.1s. Estimated remaining: 8m 41s. Rate: 159.4 symbols/min [2025-10-20T22:07:45.393341] [DEBUG] [LOCAL ] DFP: Loaded 11121 existing records from primary storage. [2025-10-20T22:07:45.388674] [INFO ] [MAIN ] DG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:45.393830] [INFO ] [MAIN ] DFP: Loaded 11121 primary data records. [2025-10-20T22:07:45.517015] [DEBUG] [LOCAL ] DEO: Loaded 49256 existing records from primary storage. [2025-10-20T22:07:45.529528] [INFO ] [MAIN ] DEO: Loaded 49256 primary data records. [2025-10-20T22:07:46.130530] [DEBUG] [LOCAL ] DG: Loaded 53269 existing records from primary storage. [2025-10-20T22:07:46.131506] [INFO ] [MAIN ] DG: Loaded 53269 primary data records. [2025-10-20T22:07:46.440306] [DEBUG] [MAIN ] DFP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:46.441383] [INFO ] [MAIN ] DFP: Generated 1794 derivative records from 11121 primary records. [2025-10-20T22:07:46.441693] [DEBUG] [LOCAL ] DFP: Saving derivative data. [2025-10-20T22:07:46.459445] [DEBUG] [LOCAL ] DFP: Derivative data saved (1794 records). [2025-10-20T22:07:46.459855] [INFO ] [MAIN ] DFP: Successfully saved derivative data. [2025-10-20T22:07:46.461983] [INFO ] [MAIN ] Derivative Progress: 493/1877 symbols completed. DFP took 1.1s. Estimated remaining: 8m 43s. Rate: 158.8 symbols/min [2025-10-20T22:07:46.478413] [DEBUG] [MAIN ] DEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:46.479279] [INFO ] [MAIN ] DEO: Generated 1794 derivative records from 49256 primary records. [2025-10-20T22:07:46.479435] [DEBUG] [LOCAL ] DEO: Saving derivative data. [2025-10-20T22:07:46.495701] [DEBUG] [LOCAL ] DEO: Derivative data saved (1794 records). [2025-10-20T22:07:46.495804] [INFO ] [MAIN ] DEO: Successfully saved derivative data. [2025-10-20T22:07:46.497707] [INFO ] [MAIN ] Derivative Progress: 494/1877 symbols completed. DEO took 1.2s. Estimated remaining: 8m 42s. Rate: 159.1 symbols/min [2025-10-20T22:07:46.512399] [DEBUG] [MAIN ] DG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:46.512952] [INFO ] [MAIN ] DG: Generated 1794 derivative records from 53269 primary records. [2025-10-20T22:07:46.513009] [DEBUG] [LOCAL ] DG: Saving derivative data. [2025-10-20T22:07:46.527997] [DEBUG] [LOCAL ] DG: Derivative data saved (1794 records). [2025-10-20T22:07:46.528061] [INFO ] [MAIN ] DG: Successfully saved derivative data. [2025-10-20T22:07:46.530511] [INFO ] [MAIN ] Derivative Progress: 495/1877 symbols completed. DG took 1.1s. Estimated remaining: 8m 40s. Rate: 159.4 symbols/min [2025-10-20T22:07:46.531023] [INFO ] [MAIN ] DHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:46.530948] [INFO ] [MAIN ] DHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:46.531124] [INFO ] [MAIN ] DHT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:46.619416] [DEBUG] [LOCAL ] DHR: Loaded 53381 existing records from primary storage. [2025-10-20T22:07:46.620671] [INFO ] [MAIN ] DHR: Loaded 53381 primary data records. [2025-10-20T22:07:46.634850] [DEBUG] [LOCAL ] DHI: Loaded 53365 existing records from primary storage. [2025-10-20T22:07:46.666704] [INFO ] [MAIN ] DHI: Loaded 53365 primary data records. [2025-10-20T22:07:46.771300] [DEBUG] [LOCAL ] DHT: Loaded 47915 existing records from primary storage. [2025-10-20T22:07:46.809438] [INFO ] [MAIN ] DHT: Loaded 47915 primary data records. [2025-10-20T22:07:47.574976] [DEBUG] [MAIN ] DHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:47.575882] [INFO ] [MAIN ] DHI: Generated 1794 derivative records from 53365 primary records. [2025-10-20T22:07:47.574611] [DEBUG] [MAIN ] DHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:47.579298] [INFO ] [MAIN ] DHR: Generated 1794 derivative records from 53381 primary records. [2025-10-20T22:07:47.580186] [DEBUG] [LOCAL ] DHR: Saving derivative data. [2025-10-20T22:07:47.577434] [DEBUG] [LOCAL ] DHI: Saving derivative data. [2025-10-20T22:07:47.620740] [DEBUG] [LOCAL ] DHR: Derivative data saved (1794 records). [2025-10-20T22:07:47.624807] [DEBUG] [LOCAL ] DHI: Derivative data saved (1794 records). [2025-10-20T22:07:47.625070] [INFO ] [MAIN ] DHR: Successfully saved derivative data. [2025-10-20T22:07:47.627231] [INFO ] [MAIN ] Derivative Progress: 496/1877 symbols completed. DHR took 1.1s. Estimated remaining: 8m 42s. Rate: 158.8 symbols/min [2025-10-20T22:07:47.627371] [INFO ] [MAIN ] DHI: Successfully saved derivative data. [2025-10-20T22:07:47.625431] [DEBUG] [MAIN ] DHT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:47.627623] [INFO ] [MAIN ] DHX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:47.630849] [INFO ] [MAIN ] DHT: Generated 1794 derivative records from 47915 primary records. [2025-10-20T22:07:47.629594] [INFO ] [MAIN ] Derivative Progress: 497/1877 symbols completed. DHI took 1.1s. Estimated remaining: 8m 40s. Rate: 159.1 symbols/min [2025-10-20T22:07:47.631110] [INFO ] [MAIN ] DIAX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:47.631501] [DEBUG] [LOCAL ] DHT: Saving derivative data. [2025-10-20T22:07:47.651869] [DEBUG] [LOCAL ] DHT: Derivative data saved (1794 records). [2025-10-20T22:07:47.651938] [INFO ] [MAIN ] DHT: Successfully saved derivative data. [2025-10-20T22:07:47.654713] [INFO ] [MAIN ] Derivative Progress: 498/1877 symbols completed. DHT took 1.1s. Estimated remaining: 8m 39s. Rate: 159.4 symbols/min [2025-10-20T22:07:47.654940] [INFO ] [MAIN ] DIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:47.665613] [DEBUG] [LOCAL ] DHX: Loaded 13199 existing records from primary storage. [2025-10-20T22:07:47.665789] [INFO ] [MAIN ] DHX: Loaded 13199 primary data records. [2025-10-20T22:07:47.719021] [DEBUG] [LOCAL ] DIAX: Loaded 12903 existing records from primary storage. [2025-10-20T22:07:47.745371] [INFO ] [MAIN ] DIAX: Loaded 12903 primary data records. [2025-10-20T22:07:48.154633] [DEBUG] [LOCAL ] DIN: Loaded 37006 existing records from primary storage. [2025-10-20T22:07:48.172909] [INFO ] [MAIN ] DIN: Loaded 37006 primary data records. [2025-10-20T22:07:48.662057] [DEBUG] [MAIN ] DIAX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:48.662445] [DEBUG] [MAIN ] DHX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:48.663189] [INFO ] [MAIN ] DIAX: Generated 1794 derivative records from 12903 primary records. [2025-10-20T22:07:48.663365] [DEBUG] [LOCAL ] DIAX: Saving derivative data. [2025-10-20T22:07:48.664389] [INFO ] [MAIN ] DHX: Generated 1794 derivative records from 13199 primary records. [2025-10-20T22:07:48.666002] [DEBUG] [LOCAL ] DHX: Saving derivative data. [2025-10-20T22:07:48.681515] [DEBUG] [LOCAL ] DIAX: Derivative data saved (1794 records). [2025-10-20T22:07:48.688351] [INFO ] [MAIN ] DIAX: Successfully saved derivative data. [2025-10-20T22:07:48.691363] [INFO ] [MAIN ] Derivative Progress: 499/1877 symbols completed. DIAX took 1.1s. Estimated remaining: 8m 41s. Rate: 158.8 symbols/min [2025-10-20T22:07:48.698193] [DEBUG] [LOCAL ] DHX: Derivative data saved (1794 records). [2025-10-20T22:07:48.698410] [INFO ] [MAIN ] DHX: Successfully saved derivative data. [2025-10-20T22:07:48.700290] [INFO ] [MAIN ] DINO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:48.700362] [INFO ] [MAIN ] Derivative Progress: 500/1877 symbols completed. DHX took 1.1s. Estimated remaining: 8m 39s. Rate: 159.1 symbols/min [2025-10-20T22:07:48.701035] [INFO ] [MAIN ] DIS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:48.718256] [DEBUG] [MAIN ] DIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:48.719192] [INFO ] [MAIN ] DIN: Generated 1794 derivative records from 37006 primary records. [2025-10-20T22:07:48.719383] [DEBUG] [LOCAL ] DIN: Saving derivative data. [2025-10-20T22:07:48.735413] [DEBUG] [LOCAL ] DIN: Derivative data saved (1794 records). [2025-10-20T22:07:48.735599] [INFO ] [MAIN ] DIN: Successfully saved derivative data. [2025-10-20T22:07:48.737613] [INFO ] [MAIN ] Derivative Progress: 501/1877 symbols completed. DIN took 1.1s. Estimated remaining: 8m 38s. Rate: 159.4 symbols/min [2025-10-20T22:07:48.737894] [INFO ] [MAIN ] DK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:48.789347] [DEBUG] [LOCAL ] DINO: Loaded 52842 existing records from primary storage. [2025-10-20T22:07:48.789572] [INFO ] [MAIN ] DINO: Loaded 52842 primary data records. [2025-10-20T22:07:48.881143] [DEBUG] [LOCAL ] DIS: Loaded 53788 existing records from primary storage. [2025-10-20T22:07:48.893186] [INFO ] [MAIN ] DIS: Loaded 53788 primary data records. [2025-10-20T22:07:49.076608] [DEBUG] [LOCAL ] DK: Loaded 49346 existing records from primary storage. [2025-10-20T22:07:49.235213] [INFO ] [MAIN ] DK: Loaded 49346 primary data records. [2025-10-20T22:07:49.762363] [DEBUG] [MAIN ] DINO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:49.763903] [INFO ] [MAIN ] DINO: Generated 1794 derivative records from 52842 primary records. [2025-10-20T22:07:49.764142] [DEBUG] [LOCAL ] DINO: Saving derivative data. [2025-10-20T22:07:49.788676] [DEBUG] [MAIN ] DIS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:49.790501] [INFO ] [MAIN ] DIS: Generated 1794 derivative records from 53788 primary records. [2025-10-20T22:07:49.795068] [DEBUG] [LOCAL ] DIS: Saving derivative data. [2025-10-20T22:07:49.801514] [DEBUG] [LOCAL ] DINO: Derivative data saved (1794 records). [2025-10-20T22:07:49.823139] [DEBUG] [MAIN ] DK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:49.823313] [INFO ] [MAIN ] DINO: Successfully saved derivative data. [2025-10-20T22:07:49.827555] [INFO ] [MAIN ] DK: Generated 1794 derivative records from 49346 primary records. [2025-10-20T22:07:49.834313] [DEBUG] [LOCAL ] DK: Saving derivative data. [2025-10-20T22:07:49.834226] [DEBUG] [LOCAL ] DIS: Derivative data saved (1794 records). [2025-10-20T22:07:49.829723] [INFO ] [MAIN ] Derivative Progress: 502/1877 symbols completed. DINO took 1.1s. Estimated remaining: 8m 39s. Rate: 158.8 symbols/min [2025-10-20T22:07:49.843011] [INFO ] [MAIN ] DIS: Successfully saved derivative data. [2025-10-20T22:07:49.835480] [INFO ] [MAIN ] DKL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:49.845934] [INFO ] [MAIN ] Derivative Progress: 503/1877 symbols completed. DIS took 1.1s. Estimated remaining: 8m 38s. Rate: 159.1 symbols/min [2025-10-20T22:07:49.855069] [DEBUG] [LOCAL ] DK: Derivative data saved (1794 records). [2025-10-20T22:07:49.851138] [INFO ] [MAIN ] DKS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:49.855335] [INFO ] [MAIN ] DK: Successfully saved derivative data. [2025-10-20T22:07:49.857866] [INFO ] [MAIN ] Derivative Progress: 504/1877 symbols completed. DK took 1.1s. Estimated remaining: 8m 37s. Rate: 159.4 symbols/min [2025-10-20T22:07:49.859517] [INFO ] [MAIN ] DLB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:49.877962] [DEBUG] [LOCAL ] DKL: Loaded 17203 existing records from primary storage. [2025-10-20T22:07:49.878700] [INFO ] [MAIN ] DKL: Loaded 17203 primary data records. [2025-10-20T22:07:50.026844] [DEBUG] [LOCAL ] DLB: Loaded 32311 existing records from primary storage. [2025-10-20T22:07:50.056342] [INFO ] [MAIN ] DLB: Loaded 32311 primary data records. [2025-10-20T22:07:50.121796] [DEBUG] [LOCAL ] DKS: Loaded 49551 existing records from primary storage. [2025-10-20T22:07:50.146903] [INFO ] [MAIN ] DKS: Loaded 49551 primary data records. [2025-10-20T22:07:50.879607] [DEBUG] [MAIN ] DKL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:50.880578] [INFO ] [MAIN ] DKL: Generated 1794 derivative records from 17203 primary records. [2025-10-20T22:07:50.880996] [DEBUG] [LOCAL ] DKL: Saving derivative data. [2025-10-20T22:07:50.898910] [DEBUG] [LOCAL ] DKL: Derivative data saved (1794 records). [2025-10-20T22:07:50.899280] [INFO ] [MAIN ] DKL: Successfully saved derivative data. [2025-10-20T22:07:50.901184] [INFO ] [MAIN ] Derivative Progress: 505/1877 symbols completed. DKL took 1.1s. Estimated remaining: 8m 38s. Rate: 158.9 symbols/min [2025-10-20T22:07:50.902241] [INFO ] [MAIN ] DLNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:50.917341] [DEBUG] [MAIN ] DLB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:50.918119] [INFO ] [MAIN ] DLB: Generated 1794 derivative records from 32311 primary records. [2025-10-20T22:07:50.918317] [DEBUG] [LOCAL ] DLB: Saving derivative data. [2025-10-20T22:07:50.935408] [DEBUG] [LOCAL ] DLNG: Loaded 6143 existing records from primary storage. [2025-10-20T22:07:50.938699] [INFO ] [MAIN ] DLNG: Loaded 6143 primary data records. [2025-10-20T22:07:50.953817] [DEBUG] [LOCAL ] DLB: Derivative data saved (1794 records). [2025-10-20T22:07:50.959180] [INFO ] [MAIN ] DLB: Successfully saved derivative data. [2025-10-20T22:07:50.997078] [INFO ] [MAIN ] Derivative Progress: 506/1877 symbols completed. DLB took 1.1s. Estimated remaining: 8m 37s. Rate: 159.1 symbols/min [2025-10-20T22:07:51.018571] [INFO ] [MAIN ] DLR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:51.019009] [DEBUG] [MAIN ] DKS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:51.033241] [INFO ] [MAIN ] DKS: Generated 1794 derivative records from 49551 primary records. [2025-10-20T22:07:51.047095] [DEBUG] [LOCAL ] DKS: Saving derivative data. [2025-10-20T22:07:51.112842] [DEBUG] [LOCAL ] DKS: Derivative data saved (1794 records). [2025-10-20T22:07:51.126143] [INFO ] [MAIN ] DKS: Successfully saved derivative data. [2025-10-20T22:07:51.160751] [INFO ] [MAIN ] Derivative Progress: 507/1877 symbols completed. DKS took 1.3s. Estimated remaining: 8m 36s. Rate: 159.3 symbols/min [2025-10-20T22:07:51.161718] [INFO ] [MAIN ] DLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:51.322596] [DEBUG] [LOCAL ] DLX: Loaded 28318 existing records from primary storage. [2025-10-20T22:07:51.337682] [INFO ] [MAIN ] DLX: Loaded 28318 primary data records. [2025-10-20T22:07:51.370596] [DEBUG] [LOCAL ] DLR: Loaded 50645 existing records from primary storage. [2025-10-20T22:07:51.394953] [INFO ] [MAIN ] DLR: Loaded 50645 primary data records. [2025-10-20T22:07:51.562094] [DEBUG] [MAIN ] DLNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:51.592411] [INFO ] [MAIN ] DLNG: Generated 1794 derivative records from 6143 primary records. [2025-10-20T22:07:51.613482] [DEBUG] [LOCAL ] DLNG: Saving derivative data. [2025-10-20T22:07:52.002630] [DEBUG] [LOCAL ] DLNG: Derivative data saved (1794 records). [2025-10-20T22:07:52.002881] [INFO ] [MAIN ] DLNG: Successfully saved derivative data. [2025-10-20T22:07:52.004859] [INFO ] [MAIN ] Derivative Progress: 508/1877 symbols completed. DLNG took 1.1s. Estimated remaining: 8m 37s. Rate: 158.9 symbols/min [2025-10-20T22:07:52.027576] [DEBUG] [MAIN ] DLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:52.028404] [INFO ] [MAIN ] DLX: Generated 1794 derivative records from 28318 primary records. [2025-10-20T22:07:52.029140] [DEBUG] [LOCAL ] DLX: Saving derivative data. [2025-10-20T22:07:52.054006] [DEBUG] [MAIN ] DLR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:52.058027] [DEBUG] [LOCAL ] DLX: Derivative data saved (1794 records). [2025-10-20T22:07:52.058711] [INFO ] [MAIN ] DLR: Generated 1794 derivative records from 50645 primary records. [2025-10-20T22:07:52.058770] [INFO ] [MAIN ] DLX: Successfully saved derivative data. [2025-10-20T22:07:52.060511] [INFO ] [MAIN ] Derivative Progress: 509/1877 symbols completed. DLX took 0.9s. Estimated remaining: 8m 36s. Rate: 159.2 symbols/min [2025-10-20T22:07:52.060589] [DEBUG] [LOCAL ] DLR: Saving derivative data. [2025-10-20T22:07:52.075710] [DEBUG] [LOCAL ] DLR: Derivative data saved (1794 records). [2025-10-20T22:07:52.075767] [INFO ] [MAIN ] DLR: Successfully saved derivative data. [2025-10-20T22:07:52.078162] [INFO ] [MAIN ] Derivative Progress: 510/1877 symbols completed. DLR took 1.1s. Estimated remaining: 8m 34s. Rate: 159.5 symbols/min [2025-10-20T22:07:52.078567] [INFO ] [MAIN ] DMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:52.078511] [INFO ] [MAIN ] DMB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:52.079215] [INFO ] [MAIN ] DLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:52.097111] [DEBUG] [LOCAL ] DMB: Loaded 10378 existing records from primary storage. [2025-10-20T22:07:52.101068] [DEBUG] [LOCAL ] DMO: Loaded 9868 existing records from primary storage. [2025-10-20T22:07:52.101195] [INFO ] [MAIN ] DMO: Loaded 9868 primary data records. [2025-10-20T22:07:52.106522] [INFO ] [MAIN ] DMB: Loaded 10378 primary data records. [2025-10-20T22:07:52.246416] [DEBUG] [LOCAL ] DLY: Loaded 20770 existing records from primary storage. [2025-10-20T22:07:52.274509] [INFO ] [MAIN ] DLY: Loaded 20770 primary data records. [2025-10-20T22:07:53.018271] [DEBUG] [MAIN ] DMB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:53.019722] [INFO ] [MAIN ] DMB: Generated 1794 derivative records from 10378 primary records. [2025-10-20T22:07:53.022911] [DEBUG] [LOCAL ] DMB: Saving derivative data. [2025-10-20T22:07:53.053978] [DEBUG] [MAIN ] DMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:53.056153] [DEBUG] [LOCAL ] DMB: Derivative data saved (1794 records). [2025-10-20T22:07:53.065020] [INFO ] [MAIN ] DMB: Successfully saved derivative data. [2025-10-20T22:07:53.064911] [INFO ] [MAIN ] DMO: Generated 1794 derivative records from 9868 primary records. [2025-10-20T22:07:53.065224] [DEBUG] [LOCAL ] DMO: Saving derivative data. [2025-10-20T22:07:53.067720] [INFO ] [MAIN ] Derivative Progress: 511/1877 symbols completed. DMB took 1.0s. Estimated remaining: 8m 36s. Rate: 159.0 symbols/min [2025-10-20T22:07:53.081262] [INFO ] [MAIN ] DNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:53.081746] [DEBUG] [MAIN ] DLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:53.087287] [INFO ] [MAIN ] DLY: Generated 1794 derivative records from 20770 primary records. [2025-10-20T22:07:53.090793] [DEBUG] [LOCAL ] DLY: Saving derivative data. [2025-10-20T22:07:53.092590] [DEBUG] [LOCAL ] DMO: Derivative data saved (1794 records). [2025-10-20T22:07:53.093830] [INFO ] [MAIN ] DMO: Successfully saved derivative data. [2025-10-20T22:07:53.096686] [INFO ] [MAIN ] Derivative Progress: 512/1877 symbols completed. DMO took 1.0s. Estimated remaining: 8m 34s. Rate: 159.3 symbols/min [2025-10-20T22:07:53.104255] [INFO ] [MAIN ] DNOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:53.112775] [DEBUG] [LOCAL ] DLY: Derivative data saved (1794 records). [2025-10-20T22:07:53.113405] [INFO ] [MAIN ] DLY: Successfully saved derivative data. [2025-10-20T22:07:53.115401] [INFO ] [MAIN ] Derivative Progress: 513/1877 symbols completed. DLY took 1.0s. Estimated remaining: 8m 33s. Rate: 159.5 symbols/min [2025-10-20T22:07:53.115839] [INFO ] [MAIN ] DNP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:53.170615] [DEBUG] [LOCAL ] DNP: Loaded 33154 existing records from primary storage. [2025-10-20T22:07:53.170837] [INFO ] [MAIN ] DNP: Loaded 33154 primary data records. [2025-10-20T22:07:53.278428] [DEBUG] [LOCAL ] DNA: Loaded 48575 existing records from primary storage. [2025-10-20T22:07:53.302020] [INFO ] [MAIN ] DNA: Loaded 48575 primary data records. [2025-10-20T22:07:53.346797] [DEBUG] [LOCAL ] DNOW: Loaded 43858 existing records from primary storage. [2025-10-20T22:07:53.391075] [INFO ] [MAIN ] DNOW: Loaded 43858 primary data records. [2025-10-20T22:07:54.197097] [DEBUG] [MAIN ] DNP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:54.198327] [INFO ] [MAIN ] DNP: Generated 1794 derivative records from 33154 primary records. [2025-10-20T22:07:54.198420] [DEBUG] [LOCAL ] DNP: Saving derivative data. [2025-10-20T22:07:54.217452] [DEBUG] [LOCAL ] DNP: Derivative data saved (1794 records). [2025-10-20T22:07:54.217794] [INFO ] [MAIN ] DNP: Successfully saved derivative data. [2025-10-20T22:07:54.219674] [INFO ] [MAIN ] Derivative Progress: 514/1877 symbols completed. DNP took 1.1s. Estimated remaining: 8m 35s. Rate: 158.9 symbols/min [2025-10-20T22:07:54.220040] [INFO ] [MAIN ] DOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:54.233375] [DEBUG] [MAIN ] DNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:54.236029] [INFO ] [MAIN ] DNA: Generated 1794 derivative records from 48575 primary records. [2025-10-20T22:07:54.236317] [DEBUG] [LOCAL ] DNA: Saving derivative data. [2025-10-20T22:07:54.249743] [DEBUG] [MAIN ] DNOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:54.250399] [INFO ] [MAIN ] DNOW: Generated 1794 derivative records from 43858 primary records. [2025-10-20T22:07:54.257503] [DEBUG] [LOCAL ] DNOW: Saving derivative data. [2025-10-20T22:07:54.266100] [DEBUG] [LOCAL ] DNA: Derivative data saved (1794 records). [2025-10-20T22:07:54.276671] [INFO ] [MAIN ] DNA: Successfully saved derivative data. [2025-10-20T22:07:54.281734] [INFO ] [MAIN ] Derivative Progress: 515/1877 symbols completed. DNA took 1.2s. Estimated remaining: 8m 33s. Rate: 159.2 symbols/min [2025-10-20T22:07:54.283075] [INFO ] [MAIN ] DOLE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:54.282964] [DEBUG] [LOCAL ] DNOW: Derivative data saved (1794 records). [2025-10-20T22:07:54.283604] [INFO ] [MAIN ] DNOW: Successfully saved derivative data. [2025-10-20T22:07:54.285483] [INFO ] [MAIN ] Derivative Progress: 516/1877 symbols completed. DNOW took 1.2s. Estimated remaining: 8m 32s. Rate: 159.5 symbols/min [2025-10-20T22:07:54.285715] [INFO ] [MAIN ] DOUG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:54.359619] [DEBUG] [LOCAL ] DOC: Loaded 53556 existing records from primary storage. [2025-10-20T22:07:54.360625] [DEBUG] [LOCAL ] DOUG: Loaded 33965 existing records from primary storage. [2025-10-20T22:07:54.360161] [INFO ] [MAIN ] DOC: Loaded 53556 primary data records. [2025-10-20T22:07:54.361067] [INFO ] [MAIN ] DOUG: Loaded 33965 primary data records. [2025-10-20T22:07:54.526760] [DEBUG] [LOCAL ] DOLE: Loaded 39134 existing records from primary storage. [2025-10-20T22:07:54.563815] [INFO ] [MAIN ] DOLE: Loaded 39134 primary data records. [2025-10-20T22:07:55.296230] [DEBUG] [MAIN ] DOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:55.297561] [INFO ] [MAIN ] DOC: Generated 1794 derivative records from 53556 primary records. [2025-10-20T22:07:55.296461] [DEBUG] [MAIN ] DOUG: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:55.297749] [DEBUG] [LOCAL ] DOC: Saving derivative data. [2025-10-20T22:07:55.298530] [INFO ] [MAIN ] DOUG: Generated 1793 derivative records from 33965 primary records. [2025-10-20T22:07:55.300071] [DEBUG] [LOCAL ] DOUG: Saving derivative data. [2025-10-20T22:07:55.317483] [DEBUG] [LOCAL ] DOC: Derivative data saved (1794 records). [2025-10-20T22:07:55.325798] [INFO ] [MAIN ] DOC: Successfully saved derivative data. [2025-10-20T22:07:55.327596] [INFO ] [MAIN ] Derivative Progress: 517/1877 symbols completed. DOC took 1.1s. Estimated remaining: 8m 33s. Rate: 159.0 symbols/min [2025-10-20T22:07:55.341825] [INFO ] [MAIN ] DOV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:55.342115] [DEBUG] [LOCAL ] DOUG: Derivative data saved (1793 records). [2025-10-20T22:07:55.346789] [INFO ] [MAIN ] DOUG: Successfully saved derivative data. [2025-10-20T22:07:55.349731] [DEBUG] [MAIN ] DOLE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:55.350503] [INFO ] [MAIN ] DOLE: Generated 1794 derivative records from 39134 primary records. [2025-10-20T22:07:55.350672] [DEBUG] [LOCAL ] DOLE: Saving derivative data. [2025-10-20T22:07:55.349170] [INFO ] [MAIN ] Derivative Progress: 518/1877 symbols completed. DOUG took 1.1s. Estimated remaining: 8m 32s. Rate: 159.3 symbols/min [2025-10-20T22:07:55.351713] [INFO ] [MAIN ] DOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:55.367213] [DEBUG] [LOCAL ] DOLE: Derivative data saved (1794 records). [2025-10-20T22:07:55.367374] [INFO ] [MAIN ] DOLE: Successfully saved derivative data. [2025-10-20T22:07:55.369215] [INFO ] [MAIN ] Derivative Progress: 519/1877 symbols completed. DOLE took 1.1s. Estimated remaining: 8m 31s. Rate: 159.5 symbols/min [2025-10-20T22:07:55.369416] [INFO ] [MAIN ] DPG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:55.385068] [DEBUG] [LOCAL ] DPG: Loaded 13502 existing records from primary storage. [2025-10-20T22:07:55.385455] [INFO ] [MAIN ] DPG: Loaded 13502 primary data records. [2025-10-20T22:07:55.534167] [DEBUG] [LOCAL ] DOV: Loaded 45575 existing records from primary storage. [2025-10-20T22:07:55.552464] [INFO ] [MAIN ] DOV: Loaded 45575 primary data records. [2025-10-20T22:07:55.846123] [DEBUG] [LOCAL ] DOW: Loaded 53733 existing records from primary storage. [2025-10-20T22:07:55.856769] [INFO ] [MAIN ] DOW: Loaded 53733 primary data records. [2025-10-20T22:07:56.383131] [DEBUG] [MAIN ] DPG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:56.385214] [INFO ] [MAIN ] DPG: Generated 1794 derivative records from 13502 primary records. [2025-10-20T22:07:56.385884] [DEBUG] [LOCAL ] DPG: Saving derivative data. [2025-10-20T22:07:56.403019] [DEBUG] [LOCAL ] DPG: Derivative data saved (1794 records). [2025-10-20T22:07:56.403155] [INFO ] [MAIN ] DPG: Successfully saved derivative data. [2025-10-20T22:07:56.405316] [INFO ] [MAIN ] Derivative Progress: 520/1877 symbols completed. DPG took 1.0s. Estimated remaining: 8m 32s. Rate: 159.0 symbols/min [2025-10-20T22:07:56.405454] [INFO ] [MAIN ] DQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:56.425725] [DEBUG] [MAIN ] DOV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:56.438278] [INFO ] [MAIN ] DOV: Generated 1794 derivative records from 45575 primary records. [2025-10-20T22:07:56.438626] [DEBUG] [LOCAL ] DOV: Saving derivative data. [2025-10-20T22:07:56.438951] [DEBUG] [MAIN ] DOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:56.440036] [INFO ] [MAIN ] DOW: Generated 1794 derivative records from 53733 primary records. [2025-10-20T22:07:56.440265] [DEBUG] [LOCAL ] DOW: Saving derivative data. [2025-10-20T22:07:56.456207] [DEBUG] [LOCAL ] DOV: Derivative data saved (1794 records). [2025-10-20T22:07:56.456355] [INFO ] [MAIN ] DOV: Successfully saved derivative data. [2025-10-20T22:07:56.465428] [INFO ] [MAIN ] Derivative Progress: 521/1877 symbols completed. DOV took 1.1s. Estimated remaining: 8m 31s. Rate: 159.3 symbols/min [2025-10-20T22:07:56.465561] [INFO ] [MAIN ] DRD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:56.474201] [DEBUG] [LOCAL ] DOW: Derivative data saved (1794 records). [2025-10-20T22:07:56.474350] [INFO ] [MAIN ] DOW: Successfully saved derivative data. [2025-10-20T22:07:56.476443] [INFO ] [MAIN ] Derivative Progress: 522/1877 symbols completed. DOW took 1.1s. Estimated remaining: 8m 30s. Rate: 159.6 symbols/min [2025-10-20T22:07:56.476740] [INFO ] [MAIN ] DRH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:56.506994] [DEBUG] [LOCAL ] DQ: Loaded 43901 existing records from primary storage. [2025-10-20T22:07:56.507143] [INFO ] [MAIN ] DQ: Loaded 43901 primary data records. [2025-10-20T22:07:56.642441] [DEBUG] [LOCAL ] DRD: Loaded 37805 existing records from primary storage. [2025-10-20T22:07:56.646420] [INFO ] [MAIN ] DRD: Loaded 37805 primary data records. [2025-10-20T22:07:56.896265] [DEBUG] [LOCAL ] DRH: Loaded 50315 existing records from primary storage. [2025-10-20T22:07:56.953668] [INFO ] [MAIN ] DRH: Loaded 50315 primary data records. [2025-10-20T22:07:57.466085] [DEBUG] [MAIN ] DQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:57.467002] [INFO ] [MAIN ] DQ: Generated 1794 derivative records from 43901 primary records. [2025-10-20T22:07:57.467419] [DEBUG] [LOCAL ] DQ: Saving derivative data. [2025-10-20T22:07:57.487512] [DEBUG] [LOCAL ] DQ: Derivative data saved (1794 records). [2025-10-20T22:07:57.500262] [INFO ] [MAIN ] DQ: Successfully saved derivative data. [2025-10-20T22:07:57.500198] [DEBUG] [MAIN ] DRD: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:57.502196] [INFO ] [MAIN ] Derivative Progress: 523/1877 symbols completed. DQ took 1.1s. Estimated remaining: 8m 31s. Rate: 159.0 symbols/min [2025-10-20T22:07:57.503165] [INFO ] [MAIN ] DRD: Generated 1793 derivative records from 37805 primary records. [2025-10-20T22:07:57.503340] [DEBUG] [LOCAL ] DRD: Saving derivative data. [2025-10-20T22:07:57.528675] [DEBUG] [LOCAL ] DRD: Derivative data saved (1793 records). [2025-10-20T22:07:57.533257] [INFO ] [MAIN ] DRD: Successfully saved derivative data. [2025-10-20T22:07:57.533170] [DEBUG] [MAIN ] DRH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:57.535041] [INFO ] [MAIN ] Derivative Progress: 524/1877 symbols completed. DRD took 1.1s. Estimated remaining: 8m 30s. Rate: 159.3 symbols/min [2025-10-20T22:07:57.535741] [INFO ] [MAIN ] DRH: Generated 1794 derivative records from 50315 primary records. [2025-10-20T22:07:57.535922] [DEBUG] [LOCAL ] DRH: Saving derivative data. [2025-10-20T22:07:57.550357] [DEBUG] [LOCAL ] DRH: Derivative data saved (1794 records). [2025-10-20T22:07:57.550412] [INFO ] [MAIN ] DRH: Successfully saved derivative data. [2025-10-20T22:07:57.552454] [INFO ] [MAIN ] Derivative Progress: 525/1877 symbols completed. DRH took 1.1s. Estimated remaining: 8m 28s. Rate: 159.6 symbols/min [2025-10-20T22:07:57.552803] [INFO ] [MAIN ] DRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:57.552854] [INFO ] [MAIN ] DRLL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:57.553199] [INFO ] [MAIN ] DSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:57.565993] [DEBUG] [LOCAL ] DRLL: Loaded 8856 existing records from primary storage. [2025-10-20T22:07:57.566180] [INFO ] [MAIN ] DRLL: Loaded 8856 primary data records. [2025-10-20T22:07:57.774088] [DEBUG] [LOCAL ] DSL: Loaded 29017 existing records from primary storage. [2025-10-20T22:07:57.861872] [INFO ] [MAIN ] DSL: Loaded 29017 primary data records. [2025-10-20T22:07:58.031510] [DEBUG] [LOCAL ] DRI: Loaded 47643 existing records from primary storage. [2025-10-20T22:07:58.032333] [INFO ] [MAIN ] DRI: Loaded 47643 primary data records. [2025-10-20T22:07:58.559803] [DEBUG] [MAIN ] DRLL: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:58.561330] [INFO ] [MAIN ] DRLL: Generated 1793 derivative records from 8856 primary records. [2025-10-20T22:07:58.563547] [DEBUG] [LOCAL ] DRLL: Saving derivative data. [2025-10-20T22:07:58.583553] [DEBUG] [LOCAL ] DRLL: Derivative data saved (1793 records). [2025-10-20T22:07:58.583847] [INFO ] [MAIN ] DRLL: Successfully saved derivative data. [2025-10-20T22:07:58.585972] [INFO ] [MAIN ] Derivative Progress: 526/1877 symbols completed. DRLL took 1.0s. Estimated remaining: 8m 30s. Rate: 159.1 symbols/min [2025-10-20T22:07:58.586793] [INFO ] [MAIN ] DSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:58.611570] [DEBUG] [MAIN ] DSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:58.612928] [INFO ] [MAIN ] DSL: Generated 1794 derivative records from 29017 primary records. [2025-10-20T22:07:58.613079] [DEBUG] [LOCAL ] DSL: Saving derivative data. [2025-10-20T22:07:58.632948] [DEBUG] [LOCAL ] DSM: Loaded 14073 existing records from primary storage. [2025-10-20T22:07:58.635135] [INFO ] [MAIN ] DSM: Loaded 14073 primary data records. [2025-10-20T22:07:58.639974] [DEBUG] [LOCAL ] DSL: Derivative data saved (1794 records). [2025-10-20T22:07:58.640446] [INFO ] [MAIN ] DSL: Successfully saved derivative data. [2025-10-20T22:07:58.679758] [INFO ] [MAIN ] Derivative Progress: 527/1877 symbols completed. DSL took 1.1s. Estimated remaining: 8m 28s. Rate: 159.3 symbols/min [2025-10-20T22:07:58.709251] [INFO ] [MAIN ] DSMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:58.762298] [DEBUG] [MAIN ] DRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:58.767880] [INFO ] [MAIN ] DRI: Generated 1794 derivative records from 47643 primary records. [2025-10-20T22:07:58.769965] [DEBUG] [LOCAL ] DSMC: Loaded 1838 existing records from primary storage. [2025-10-20T22:07:58.770036] [DEBUG] [LOCAL ] DRI: Saving derivative data. [2025-10-20T22:07:58.770200] [INFO ] [MAIN ] DSMC: Loaded 1838 primary data records. [2025-10-20T22:07:59.119722] [DEBUG] [LOCAL ] DRI: Derivative data saved (1794 records). [2025-10-20T22:07:59.130427] [INFO ] [MAIN ] DRI: Successfully saved derivative data. [2025-10-20T22:07:59.138621] [INFO ] [MAIN ] Derivative Progress: 528/1877 symbols completed. DRI took 1.6s. Estimated remaining: 8m 28s. Rate: 159.2 symbols/min [2025-10-20T22:07:59.139003] [INFO ] [MAIN ] DSX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:59.333299] [DEBUG] [LOCAL ] DSX: Loaded 23744 existing records from primary storage. [2025-10-20T22:07:59.334113] [INFO ] [MAIN ] DSX: Loaded 23744 primary data records. [2025-10-20T22:07:59.543350] [DEBUG] [MAIN ] DSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:07:59.747590] [INFO ] [MAIN ] DSM: Generated 1794 derivative records from 14073 primary records. [2025-10-20T22:07:59.749032] [DEBUG] [LOCAL ] DSM: Saving derivative data. [2025-10-20T22:07:59.784355] [DEBUG] [MAIN ] DSMC: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:07:59.789018] [INFO ] [MAIN ] DSMC: Generated 1793 derivative records from 1838 primary records. [2025-10-20T22:07:59.789361] [DEBUG] [LOCAL ] DSMC: Saving derivative data. [2025-10-20T22:07:59.792905] [DEBUG] [LOCAL ] DSM: Derivative data saved (1794 records). [2025-10-20T22:07:59.793529] [INFO ] [MAIN ] DSM: Successfully saved derivative data. [2025-10-20T22:07:59.796349] [INFO ] [MAIN ] Derivative Progress: 529/1877 symbols completed. DSM took 1.2s. Estimated remaining: 8m 29s. Rate: 159.0 symbols/min [2025-10-20T22:07:59.796757] [INFO ] [MAIN ] DT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:59.813216] [DEBUG] [LOCAL ] DSMC: Derivative data saved (1793 records). [2025-10-20T22:07:59.813318] [INFO ] [MAIN ] DSMC: Successfully saved derivative data. [2025-10-20T22:07:59.815512] [INFO ] [MAIN ] Derivative Progress: 530/1877 symbols completed. DSMC took 1.1s. Estimated remaining: 8m 27s. Rate: 159.3 symbols/min [2025-10-20T22:07:59.815950] [INFO ] [MAIN ] DTB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:07:59.824872] [DEBUG] [LOCAL ] DTB: Loaded 4937 existing records from primary storage. [2025-10-20T22:07:59.825053] [INFO ] [MAIN ] DTB: Loaded 4937 primary data records. [2025-10-20T22:08:00.068516] [DEBUG] [MAIN ] DSX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:00.076214] [INFO ] [MAIN ] DSX: Generated 1794 derivative records from 23744 primary records. [2025-10-20T22:08:00.193106] [DEBUG] [LOCAL ] DSX: Saving derivative data. [2025-10-20T22:08:00.235942] [DEBUG] [LOCAL ] DT: Loaded 52952 existing records from primary storage. [2025-10-20T22:08:00.237141] [INFO ] [MAIN ] DT: Loaded 52952 primary data records. [2025-10-20T22:08:00.241934] [DEBUG] [LOCAL ] DSX: Derivative data saved (1794 records). [2025-10-20T22:08:00.248613] [INFO ] [MAIN ] DSX: Successfully saved derivative data. [2025-10-20T22:08:00.268620] [INFO ] [MAIN ] Derivative Progress: 531/1877 symbols completed. DSX took 1.1s. Estimated remaining: 8m 27s. Rate: 159.2 symbols/min [2025-10-20T22:08:00.311204] [INFO ] [MAIN ] DTE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:00.584577] [DEBUG] [MAIN ] DTB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:00.598717] [INFO ] [MAIN ] DTB: Generated 1794 derivative records from 4937 primary records. [2025-10-20T22:08:00.599124] [DEBUG] [LOCAL ] DTB: Saving derivative data. [2025-10-20T22:08:00.616664] [DEBUG] [LOCAL ] DTB: Derivative data saved (1794 records). [2025-10-20T22:08:00.616969] [INFO ] [MAIN ] DTB: Successfully saved derivative data. [2025-10-20T22:08:00.619113] [INFO ] [MAIN ] Derivative Progress: 532/1877 symbols completed. DTB took 0.8s. Estimated remaining: 8m 27s. Rate: 159.3 symbols/min [2025-10-20T22:08:00.620797] [INFO ] [MAIN ] DTG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:00.639763] [DEBUG] [LOCAL ] DTE: Loaded 47244 existing records from primary storage. [2025-10-20T22:08:00.642278] [INFO ] [MAIN ] DTE: Loaded 47244 primary data records. [2025-10-20T22:08:00.711718] [DEBUG] [LOCAL ] DTG: Loaded 5735 existing records from primary storage. [2025-10-20T22:08:00.725915] [INFO ] [MAIN ] DTG: Loaded 5735 primary data records. [2025-10-20T22:08:00.870845] [DEBUG] [MAIN ] DT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:00.909415] [INFO ] [MAIN ] DT: Generated 1794 derivative records from 52952 primary records. [2025-10-20T22:08:00.957232] [DEBUG] [LOCAL ] DT: Saving derivative data. [2025-10-20T22:08:01.158086] [DEBUG] [LOCAL ] DT: Derivative data saved (1794 records). [2025-10-20T22:08:01.171459] [INFO ] [MAIN ] DT: Successfully saved derivative data. [2025-10-20T22:08:01.173795] [INFO ] [MAIN ] Derivative Progress: 533/1877 symbols completed. DT took 1.4s. Estimated remaining: 8m 27s. Rate: 159.1 symbols/min [2025-10-20T22:08:01.179954] [INFO ] [MAIN ] DTM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:01.364455] [DEBUG] [MAIN ] DTE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:01.365706] [INFO ] [MAIN ] DTE: Generated 1794 derivative records from 47244 primary records. [2025-10-20T22:08:01.365875] [DEBUG] [LOCAL ] DTE: Saving derivative data. [2025-10-20T22:08:01.385161] [DEBUG] [LOCAL ] DTE: Derivative data saved (1794 records). [2025-10-20T22:08:01.385307] [INFO ] [MAIN ] DTE: Successfully saved derivative data. [2025-10-20T22:08:01.402797] [INFO ] [MAIN ] Derivative Progress: 534/1877 symbols completed. DTE took 1.1s. Estimated remaining: 8m 26s. Rate: 159.2 symbols/min [2025-10-20T22:08:01.404717] [INFO ] [MAIN ] DUK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:01.419067] [DEBUG] [LOCAL ] DTM: Loaded 43821 existing records from primary storage. [2025-10-20T22:08:01.419362] [INFO ] [MAIN ] DTM: Loaded 43821 primary data records. [2025-10-20T22:08:01.419697] [DEBUG] [MAIN ] DTG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:01.420704] [INFO ] [MAIN ] DTG: Generated 1794 derivative records from 5735 primary records. [2025-10-20T22:08:01.432486] [DEBUG] [LOCAL ] DTG: Saving derivative data. [2025-10-20T22:08:01.551871] [DEBUG] [LOCAL ] DTG: Derivative data saved (1794 records). [2025-10-20T22:08:01.585388] [INFO ] [MAIN ] DTG: Successfully saved derivative data. [2025-10-20T22:08:01.596304] [INFO ] [MAIN ] Derivative Progress: 535/1877 symbols completed. DTG took 1.0s. Estimated remaining: 8m 25s. Rate: 159.4 symbols/min [2025-10-20T22:08:01.596561] [INFO ] [MAIN ] DUKB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:01.746028] [DEBUG] [LOCAL ] DUKB: Loaded 6196 existing records from primary storage. [2025-10-20T22:08:01.746245] [INFO ] [MAIN ] DUKB: Loaded 6196 primary data records. [2025-10-20T22:08:01.905865] [DEBUG] [LOCAL ] DUK: Loaded 53437 existing records from primary storage. [2025-10-20T22:08:01.922126] [INFO ] [MAIN ] DUK: Loaded 53437 primary data records. [2025-10-20T22:08:02.472971] [DEBUG] [MAIN ] DTM: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:02.474689] [INFO ] [MAIN ] DTM: Generated 1793 derivative records from 43821 primary records. [2025-10-20T22:08:02.474800] [DEBUG] [LOCAL ] DTM: Saving derivative data. [2025-10-20T22:08:02.492721] [DEBUG] [LOCAL ] DTM: Derivative data saved (1793 records). [2025-10-20T22:08:02.492844] [INFO ] [MAIN ] DTM: Successfully saved derivative data. [2025-10-20T22:08:02.495102] [INFO ] [MAIN ] Derivative Progress: 536/1877 symbols completed. DTM took 1.3s. Estimated remaining: 8m 26s. Rate: 159.0 symbols/min [2025-10-20T22:08:02.495910] [INFO ] [MAIN ] DV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:02.515399] [DEBUG] [MAIN ] DUKB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:02.516538] [INFO ] [MAIN ] DUKB: Generated 1794 derivative records from 6196 primary records. [2025-10-20T22:08:02.516720] [DEBUG] [LOCAL ] DUKB: Saving derivative data. [2025-10-20T22:08:02.535272] [DEBUG] [LOCAL ] DUKB: Derivative data saved (1794 records). [2025-10-20T22:08:02.535908] [INFO ] [MAIN ] DUKB: Successfully saved derivative data. [2025-10-20T22:08:02.538998] [INFO ] [MAIN ] Derivative Progress: 537/1877 symbols completed. DUKB took 0.9s. Estimated remaining: 8m 25s. Rate: 159.2 symbols/min [2025-10-20T22:08:02.540010] [INFO ] [MAIN ] DVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:02.553887] [DEBUG] [MAIN ] DUK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:02.554719] [INFO ] [MAIN ] DUK: Generated 1794 derivative records from 53437 primary records. [2025-10-20T22:08:02.554830] [DEBUG] [LOCAL ] DUK: Saving derivative data. [2025-10-20T22:08:02.584983] [DEBUG] [LOCAL ] DUK: Derivative data saved (1794 records). [2025-10-20T22:08:02.585114] [INFO ] [MAIN ] DUK: Successfully saved derivative data. [2025-10-20T22:08:02.587229] [INFO ] [MAIN ] Derivative Progress: 538/1877 symbols completed. DUK took 1.2s. Estimated remaining: 8m 24s. Rate: 159.5 symbols/min [2025-10-20T22:08:02.617737] [DEBUG] [LOCAL ] DV: Loaded 51250 existing records from primary storage. [2025-10-20T22:08:02.617869] [INFO ] [MAIN ] DV: Loaded 51250 primary data records. [2025-10-20T22:08:02.759870] [DEBUG] [LOCAL ] DVA: Loaded 40887 existing records from primary storage. [2025-10-20T22:08:02.804158] [INFO ] [MAIN ] DVA: Loaded 40887 primary data records. [2025-10-20T22:08:03.234892] [DEBUG] [MAIN ] DV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:03.235721] [INFO ] [MAIN ] DV: Generated 1794 derivative records from 51250 primary records. [2025-10-20T22:08:03.235795] [DEBUG] [LOCAL ] DV: Saving derivative data. [2025-10-20T22:08:03.252891] [DEBUG] [LOCAL ] DV: Derivative data saved (1794 records). [2025-10-20T22:08:03.253156] [INFO ] [MAIN ] DV: Successfully saved derivative data. [2025-10-20T22:08:03.255212] [INFO ] [MAIN ] Derivative Progress: 539/1877 symbols completed. DV took 0.8s. Estimated remaining: 8m 24s. Rate: 159.3 symbols/min [2025-10-20T22:08:03.272400] [DEBUG] [MAIN ] DVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:03.273029] [INFO ] [MAIN ] DVA: Generated 1794 derivative records from 40887 primary records. [2025-10-20T22:08:03.273089] [DEBUG] [LOCAL ] DVA: Saving derivative data. [2025-10-20T22:08:03.288313] [DEBUG] [LOCAL ] DVA: Derivative data saved (1794 records). [2025-10-20T22:08:03.288371] [INFO ] [MAIN ] DVA: Successfully saved derivative data. [2025-10-20T22:08:03.290224] [INFO ] [MAIN ] Derivative Progress: 540/1877 symbols completed. DVA took 0.8s. Estimated remaining: 8m 23s. Rate: 159.5 symbols/min [2025-10-20T22:08:03.290728] [INFO ] [MAIN ] DXC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:03.290826] [INFO ] [MAIN ] DX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:03.290612] [INFO ] [MAIN ] DVN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:03.360976] [DEBUG] [LOCAL ] DXC: Loaded 49946 existing records from primary storage. [2025-10-20T22:08:03.375874] [INFO ] [MAIN ] DXC: Loaded 49946 primary data records. [2025-10-20T22:08:03.422563] [DEBUG] [LOCAL ] DX: Loaded 52526 existing records from primary storage. [2025-10-20T22:08:03.435179] [DEBUG] [LOCAL ] DVN: Loaded 53699 existing records from primary storage. [2025-10-20T22:08:03.455566] [INFO ] [MAIN ] DX: Loaded 52526 primary data records. [2025-10-20T22:08:03.455639] [INFO ] [MAIN ] DVN: Loaded 53699 primary data records. [2025-10-20T22:08:04.347191] [DEBUG] [MAIN ] DXC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:04.348255] [INFO ] [MAIN ] DXC: Generated 1794 derivative records from 49946 primary records. [2025-10-20T22:08:04.348570] [DEBUG] [LOCAL ] DXC: Saving derivative data. [2025-10-20T22:08:04.369151] [DEBUG] [LOCAL ] DXC: Derivative data saved (1794 records). [2025-10-20T22:08:04.369332] [INFO ] [MAIN ] DXC: Successfully saved derivative data. [2025-10-20T22:08:04.373074] [INFO ] [MAIN ] Derivative Progress: 541/1877 symbols completed. DXC took 1.1s. Estimated remaining: 8m 24s. Rate: 159.0 symbols/min [2025-10-20T22:08:04.373301] [INFO ] [MAIN ] DXYZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:04.387714] [DEBUG] [MAIN ] DVN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:04.388702] [INFO ] [MAIN ] DVN: Generated 1794 derivative records from 53699 primary records. [2025-10-20T22:08:04.388933] [DEBUG] [LOCAL ] DVN: Saving derivative data. [2025-10-20T22:08:04.410002] [DEBUG] [MAIN ] DX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:04.414894] [INFO ] [MAIN ] DX: Generated 1794 derivative records from 52526 primary records. [2025-10-20T22:08:04.415753] [DEBUG] [LOCAL ] DX: Saving derivative data. [2025-10-20T22:08:04.419398] [DEBUG] [LOCAL ] DVN: Derivative data saved (1794 records). [2025-10-20T22:08:04.420242] [INFO ] [MAIN ] DVN: Successfully saved derivative data. [2025-10-20T22:08:04.429487] [INFO ] [MAIN ] Derivative Progress: 542/1877 symbols completed. DVN took 1.1s. Estimated remaining: 8m 23s. Rate: 159.2 symbols/min [2025-10-20T22:08:04.434170] [INFO ] [MAIN ] DY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:04.439555] [DEBUG] [LOCAL ] DX: Derivative data saved (1794 records). [2025-10-20T22:08:04.439916] [INFO ] [MAIN ] DX: Successfully saved derivative data. [2025-10-20T22:08:04.442110] [INFO ] [MAIN ] Derivative Progress: 543/1877 symbols completed. DX took 1.2s. Estimated remaining: 8m 22s. Rate: 159.5 symbols/min [2025-10-20T22:08:04.442317] [INFO ] [MAIN ] E: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:04.493938] [DEBUG] [LOCAL ] DY: Loaded 28264 existing records from primary storage. [2025-10-20T22:08:04.494774] [DEBUG] [LOCAL ] DXYZ: Loaded 37229 existing records from primary storage. [2025-10-20T22:08:04.495212] [INFO ] [MAIN ] DXYZ: Loaded 37229 primary data records. [2025-10-20T22:08:04.495139] [INFO ] [MAIN ] DY: Loaded 28264 primary data records. [2025-10-20T22:08:04.705264] [DEBUG] [LOCAL ] E: Loaded 33355 existing records from primary storage. [2025-10-20T22:08:04.734474] [INFO ] [MAIN ] E: Loaded 33355 primary data records. [2025-10-20T22:08:05.445516] [DEBUG] [MAIN ] DXYZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:05.446622] [INFO ] [MAIN ] DXYZ: Generated 1794 derivative records from 37229 primary records. [2025-10-20T22:08:05.446859] [DEBUG] [LOCAL ] DXYZ: Saving derivative data. [2025-10-20T22:08:05.465549] [DEBUG] [LOCAL ] DXYZ: Derivative data saved (1794 records). [2025-10-20T22:08:05.466435] [INFO ] [MAIN ] DXYZ: Successfully saved derivative data. [2025-10-20T22:08:05.468871] [INFO ] [MAIN ] Derivative Progress: 544/1877 symbols completed. DXYZ took 1.1s. Estimated remaining: 8m 23s. Rate: 159.0 symbols/min [2025-10-20T22:08:05.469389] [INFO ] [MAIN ] EAF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:05.482632] [DEBUG] [MAIN ] DY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:05.485494] [INFO ] [MAIN ] DY: Generated 1794 derivative records from 28264 primary records. [2025-10-20T22:08:05.485664] [DEBUG] [LOCAL ] DY: Saving derivative data. [2025-10-20T22:08:05.509503] [DEBUG] [LOCAL ] DY: Derivative data saved (1794 records). [2025-10-20T22:08:05.514162] [INFO ] [MAIN ] DY: Successfully saved derivative data. [2025-10-20T22:08:05.516234] [INFO ] [MAIN ] Derivative Progress: 545/1877 symbols completed. DY took 1.1s. Estimated remaining: 8m 22s. Rate: 159.3 symbols/min [2025-10-20T22:08:05.516793] [INFO ] [MAIN ] EAI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:05.516629] [DEBUG] [MAIN ] E: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:05.517900] [INFO ] [MAIN ] E: Generated 1794 derivative records from 33355 primary records. [2025-10-20T22:08:05.518088] [DEBUG] [LOCAL ] E: Saving derivative data. [2025-10-20T22:08:05.543878] [DEBUG] [LOCAL ] EAI: Loaded 3940 existing records from primary storage. [2025-10-20T22:08:05.545793] [INFO ] [MAIN ] EAI: Loaded 3940 primary data records. [2025-10-20T22:08:05.567485] [DEBUG] [LOCAL ] E: Derivative data saved (1794 records). [2025-10-20T22:08:05.584029] [INFO ] [MAIN ] E: Successfully saved derivative data. [2025-10-20T22:08:05.591560] [INFO ] [MAIN ] Derivative Progress: 546/1877 symbols completed. E took 1.1s. Estimated remaining: 8m 21s. Rate: 159.5 symbols/min [2025-10-20T22:08:05.613208] [INFO ] [MAIN ] EARN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:05.665768] [DEBUG] [LOCAL ] EAF: Loaded 41341 existing records from primary storage. [2025-10-20T22:08:05.682697] [INFO ] [MAIN ] EAF: Loaded 41341 primary data records. [2025-10-20T22:08:06.006575] [DEBUG] [LOCAL ] EARN: Loaded 28002 existing records from primary storage. [2025-10-20T22:08:06.022878] [INFO ] [MAIN ] EARN: Loaded 28002 primary data records. [2025-10-20T22:08:06.526562] [DEBUG] [MAIN ] EAI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:06.527404] [INFO ] [MAIN ] EAI: Generated 1794 derivative records from 3940 primary records. [2025-10-20T22:08:06.527593] [DEBUG] [LOCAL ] EAI: Saving derivative data. [2025-10-20T22:08:06.546265] [DEBUG] [LOCAL ] EAI: Derivative data saved (1794 records). [2025-10-20T22:08:06.548312] [INFO ] [MAIN ] EAI: Successfully saved derivative data. [2025-10-20T22:08:06.550631] [INFO ] [MAIN ] Derivative Progress: 547/1877 symbols completed. EAI took 1.0s. Estimated remaining: 8m 22s. Rate: 159.0 symbols/min [2025-10-20T22:08:06.551200] [INFO ] [MAIN ] EAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:06.566455] [DEBUG] [MAIN ] EAF: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:06.569154] [INFO ] [MAIN ] EAF: Generated 1793 derivative records from 41341 primary records. [2025-10-20T22:08:06.569474] [DEBUG] [LOCAL ] EAF: Saving derivative data. [2025-10-20T22:08:06.593277] [DEBUG] [MAIN ] EARN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:06.597407] [INFO ] [MAIN ] EARN: Generated 1794 derivative records from 28002 primary records. [2025-10-20T22:08:06.597534] [DEBUG] [LOCAL ] EARN: Saving derivative data. [2025-10-20T22:08:06.601338] [DEBUG] [LOCAL ] EAF: Derivative data saved (1793 records). [2025-10-20T22:08:06.601510] [INFO ] [MAIN ] EAF: Successfully saved derivative data. [2025-10-20T22:08:06.604608] [INFO ] [MAIN ] Derivative Progress: 548/1877 symbols completed. EAF took 1.1s. Estimated remaining: 8m 21s. Rate: 159.3 symbols/min [2025-10-20T22:08:06.604811] [INFO ] [MAIN ] EB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:06.622598] [DEBUG] [LOCAL ] EARN: Derivative data saved (1794 records). [2025-10-20T22:08:06.622785] [INFO ] [MAIN ] EARN: Successfully saved derivative data. [2025-10-20T22:08:06.625123] [INFO ] [MAIN ] Derivative Progress: 549/1877 symbols completed. EARN took 1.0s. Estimated remaining: 8m 19s. Rate: 159.6 symbols/min [2025-10-20T22:08:06.625365] [INFO ] [MAIN ] EBF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:06.663562] [DEBUG] [LOCAL ] EBF: Loaded 19140 existing records from primary storage. [2025-10-20T22:08:06.678600] [INFO ] [MAIN ] EBF: Loaded 19140 primary data records. [2025-10-20T22:08:06.701791] [DEBUG] [LOCAL ] EAT: Loaded 49806 existing records from primary storage. [2025-10-20T22:08:06.759824] [DEBUG] [LOCAL ] EB: Loaded 36273 existing records from primary storage. [2025-10-20T22:08:06.766996] [INFO ] [MAIN ] EB: Loaded 36273 primary data records. [2025-10-20T22:08:06.759300] [INFO ] [MAIN ] EAT: Loaded 49806 primary data records. [2025-10-20T22:08:07.692780] [DEBUG] [MAIN ] EBF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:07.693241] [DEBUG] [MAIN ] EB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:07.692484] [DEBUG] [MAIN ] EAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:07.694395] [INFO ] [MAIN ] EBF: Generated 1794 derivative records from 19140 primary records. [2025-10-20T22:08:07.695314] [DEBUG] [LOCAL ] EBF: Saving derivative data. [2025-10-20T22:08:07.695114] [INFO ] [MAIN ] EAT: Generated 1794 derivative records from 49806 primary records. [2025-10-20T22:08:07.696121] [INFO ] [MAIN ] EB: Generated 1794 derivative records from 36273 primary records. [2025-10-20T22:08:07.697470] [DEBUG] [LOCAL ] EB: Saving derivative data. [2025-10-20T22:08:07.697566] [DEBUG] [LOCAL ] EAT: Saving derivative data. [2025-10-20T22:08:07.735235] [DEBUG] [LOCAL ] EB: Derivative data saved (1794 records). [2025-10-20T22:08:07.748116] [DEBUG] [LOCAL ] EAT: Derivative data saved (1794 records). [2025-10-20T22:08:07.748231] [INFO ] [MAIN ] EB: Successfully saved derivative data. [2025-10-20T22:08:07.750185] [INFO ] [MAIN ] Derivative Progress: 550/1877 symbols completed. EB took 1.1s. Estimated remaining: 8m 21s. Rate: 159.0 symbols/min [2025-10-20T22:08:07.741358] [DEBUG] [LOCAL ] EBF: Derivative data saved (1794 records). [2025-10-20T22:08:07.750455] [INFO ] [MAIN ] EBF: Successfully saved derivative data. [2025-10-20T22:08:07.753626] [INFO ] [MAIN ] EAT: Successfully saved derivative data. [2025-10-20T22:08:07.753521] [INFO ] [MAIN ] EBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:07.753859] [INFO ] [MAIN ] Derivative Progress: 551/1877 symbols completed. EBF took 1.1s. Estimated remaining: 8m 19s. Rate: 159.3 symbols/min [2025-10-20T22:08:07.756807] [INFO ] [MAIN ] EBS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:07.756995] [INFO ] [MAIN ] Derivative Progress: 552/1877 symbols completed. EAT took 1.2s. Estimated remaining: 8m 18s. Rate: 159.6 symbols/min [2025-10-20T22:08:07.757166] [INFO ] [MAIN ] EC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:07.835741] [DEBUG] [LOCAL ] EBR: Loaded 43386 existing records from primary storage. [2025-10-20T22:08:07.851720] [DEBUG] [LOCAL ] EC: Loaded 48929 existing records from primary storage. [2025-10-20T22:08:07.851873] [INFO ] [MAIN ] EC: Loaded 48929 primary data records. [2025-10-20T22:08:07.851067] [INFO ] [MAIN ] EBR: Loaded 43386 primary data records. [2025-10-20T22:08:07.875265] [DEBUG] [LOCAL ] EBS: Loaded 46671 existing records from primary storage. [2025-10-20T22:08:07.950866] [INFO ] [MAIN ] EBS: Loaded 46671 primary data records. [2025-10-20T22:08:08.885557] [DEBUG] [MAIN ] EBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:08.889053] [INFO ] [MAIN ] EBR: Generated 1794 derivative records from 43386 primary records. [2025-10-20T22:08:08.889257] [DEBUG] [LOCAL ] EBR: Saving derivative data. [2025-10-20T22:08:08.910538] [DEBUG] [MAIN ] EC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:08.915980] [INFO ] [MAIN ] EC: Generated 1794 derivative records from 48929 primary records. [2025-10-20T22:08:08.916174] [DEBUG] [LOCAL ] EC: Saving derivative data. [2025-10-20T22:08:08.932856] [DEBUG] [LOCAL ] EBR: Derivative data saved (1794 records). [2025-10-20T22:08:08.934076] [INFO ] [MAIN ] EBR: Successfully saved derivative data. [2025-10-20T22:08:08.937346] [DEBUG] [LOCAL ] EC: Derivative data saved (1794 records). [2025-10-20T22:08:08.939470] [INFO ] [MAIN ] Derivative Progress: 553/1877 symbols completed. EBR took 1.2s. Estimated remaining: 8m 20s. Rate: 159.0 symbols/min [2025-10-20T22:08:08.941496] [INFO ] [MAIN ] EC: Successfully saved derivative data. [2025-10-20T22:08:08.943637] [INFO ] [MAIN ] Derivative Progress: 554/1877 symbols completed. EC took 1.2s. Estimated remaining: 8m 19s. Rate: 159.2 symbols/min [2025-10-20T22:08:08.956340] [DEBUG] [MAIN ] EBS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:08.956959] [INFO ] [MAIN ] EBS: Generated 1794 derivative records from 46671 primary records. [2025-10-20T22:08:08.957019] [DEBUG] [LOCAL ] EBS: Saving derivative data. [2025-10-20T22:08:08.971889] [DEBUG] [LOCAL ] EBS: Derivative data saved (1794 records). [2025-10-20T22:08:08.971982] [INFO ] [MAIN ] EBS: Successfully saved derivative data. [2025-10-20T22:08:08.974000] [INFO ] [MAIN ] Derivative Progress: 555/1877 symbols completed. EBS took 1.2s. Estimated remaining: 8m 17s. Rate: 159.5 symbols/min [2025-10-20T22:08:08.974514] [INFO ] [MAIN ] ECCU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:08.974451] [INFO ] [MAIN ] ECAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:08.974616] [INFO ] [MAIN ] ECC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:08.980965] [DEBUG] [LOCAL ] ECCU: Loaded 2179 existing records from primary storage. [2025-10-20T22:08:08.981274] [INFO ] [MAIN ] ECCU: Loaded 2179 primary data records. [2025-10-20T22:08:09.333073] [DEBUG] [LOCAL ] ECAT: Loaded 29346 existing records from primary storage. [2025-10-20T22:08:09.346159] [INFO ] [MAIN ] ECAT: Loaded 29346 primary data records. [2025-10-20T22:08:09.368637] [DEBUG] [LOCAL ] ECC: Loaded 43475 existing records from primary storage. [2025-10-20T22:08:09.384400] [INFO ] [MAIN ] ECC: Loaded 43475 primary data records. [2025-10-20T22:08:10.015273] [DEBUG] [MAIN ] ECCU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:10.017151] [INFO ] [MAIN ] ECCU: Generated 1794 derivative records from 2179 primary records. [2025-10-20T22:08:10.017311] [DEBUG] [LOCAL ] ECCU: Saving derivative data. [2025-10-20T22:08:10.033046] [DEBUG] [LOCAL ] ECCU: Derivative data saved (1794 records). [2025-10-20T22:08:10.033164] [INFO ] [MAIN ] ECCU: Successfully saved derivative data. [2025-10-20T22:08:10.035188] [INFO ] [MAIN ] Derivative Progress: 556/1877 symbols completed. ECCU took 1.1s. Estimated remaining: 8m 19s. Rate: 159.0 symbols/min [2025-10-20T22:08:10.035362] [INFO ] [MAIN ] ECG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:10.056328] [DEBUG] [MAIN ] ECAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:10.069485] [INFO ] [MAIN ] ECAT: Generated 1794 derivative records from 29346 primary records. [2025-10-20T22:08:10.069751] [DEBUG] [LOCAL ] ECAT: Saving derivative data. [2025-10-20T22:08:10.070168] [DEBUG] [MAIN ] ECC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:10.071899] [INFO ] [MAIN ] ECC: Generated 1794 derivative records from 43475 primary records. [2025-10-20T22:08:10.071993] [DEBUG] [LOCAL ] ECC: Saving derivative data. [2025-10-20T22:08:10.113728] [DEBUG] [LOCAL ] ECC: Derivative data saved (1794 records). [2025-10-20T22:08:10.099215] [DEBUG] [LOCAL ] ECAT: Derivative data saved (1794 records). [2025-10-20T22:08:10.113995] [INFO ] [MAIN ] ECC: Successfully saved derivative data. [2025-10-20T22:08:10.113916] [INFO ] [MAIN ] ECAT: Successfully saved derivative data. [2025-10-20T22:08:10.116085] [INFO ] [MAIN ] Derivative Progress: 557/1877 symbols completed. ECC took 1.1s. Estimated remaining: 8m 17s. Rate: 159.2 symbols/min [2025-10-20T22:08:10.118722] [INFO ] [MAIN ] ECL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:10.118829] [INFO ] [MAIN ] Derivative Progress: 558/1877 symbols completed. ECAT took 1.1s. Estimated remaining: 8m 16s. Rate: 159.5 symbols/min [2025-10-20T22:08:10.119116] [INFO ] [MAIN ] ECO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:10.132412] [DEBUG] [LOCAL ] ECG: Loaded 32902 existing records from primary storage. [2025-10-20T22:08:10.132566] [INFO ] [MAIN ] ECG: Loaded 32902 primary data records. [2025-10-20T22:08:10.295874] [DEBUG] [LOCAL ] ECO: Loaded 18204 existing records from primary storage. [2025-10-20T22:08:10.328857] [INFO ] [MAIN ] ECO: Loaded 18204 primary data records. [2025-10-20T22:08:10.564086] [DEBUG] [LOCAL ] ECL: Loaded 46730 existing records from primary storage. [2025-10-20T22:08:10.567958] [INFO ] [MAIN ] ECL: Loaded 46730 primary data records. [2025-10-20T22:08:11.085730] [DEBUG] [MAIN ] ECG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:11.092167] [INFO ] [MAIN ] ECG: Generated 1794 derivative records from 32902 primary records. [2025-10-20T22:08:11.100462] [DEBUG] [LOCAL ] ECG: Saving derivative data. [2025-10-20T22:08:11.120578] [DEBUG] [LOCAL ] ECG: Derivative data saved (1794 records). [2025-10-20T22:08:11.121279] [INFO ] [MAIN ] ECG: Successfully saved derivative data. [2025-10-20T22:08:11.124330] [INFO ] [MAIN ] Derivative Progress: 559/1877 symbols completed. ECG took 1.1s. Estimated remaining: 8m 17s. Rate: 159.0 symbols/min [2025-10-20T22:08:11.125016] [INFO ] [MAIN ] ED: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:11.145545] [DEBUG] [MAIN ] ECO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:11.146698] [INFO ] [MAIN ] ECO: Generated 1794 derivative records from 18204 primary records. [2025-10-20T22:08:11.147193] [DEBUG] [LOCAL ] ECO: Saving derivative data. [2025-10-20T22:08:11.165723] [DEBUG] [LOCAL ] ECO: Derivative data saved (1794 records). [2025-10-20T22:08:11.166093] [INFO ] [MAIN ] ECO: Successfully saved derivative data. [2025-10-20T22:08:11.168728] [INFO ] [MAIN ] Derivative Progress: 560/1877 symbols completed. ECO took 1.0s. Estimated remaining: 8m 16s. Rate: 159.3 symbols/min [2025-10-20T22:08:11.169341] [INFO ] [MAIN ] EDN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:11.186707] [DEBUG] [MAIN ] ECL: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:11.187395] [INFO ] [MAIN ] ECL: Generated 1793 derivative records from 46730 primary records. [2025-10-20T22:08:11.187504] [DEBUG] [LOCAL ] ECL: Saving derivative data. [2025-10-20T22:08:11.221267] [DEBUG] [LOCAL ] ECL: Derivative data saved (1793 records). [2025-10-20T22:08:11.221457] [INFO ] [MAIN ] ECL: Successfully saved derivative data. [2025-10-20T22:08:11.223495] [INFO ] [MAIN ] Derivative Progress: 561/1877 symbols completed. ECL took 1.1s. Estimated remaining: 8m 15s. Rate: 159.5 symbols/min [2025-10-20T22:08:11.223805] [INFO ] [MAIN ] EDU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:11.233500] [DEBUG] [LOCAL ] EDN: Loaded 19210 existing records from primary storage. [2025-10-20T22:08:11.233684] [INFO ] [MAIN ] EDN: Loaded 19210 primary data records. [2025-10-20T22:08:11.371912] [DEBUG] [LOCAL ] ED: Loaded 52649 existing records from primary storage. [2025-10-20T22:08:11.414829] [INFO ] [MAIN ] ED: Loaded 52649 primary data records. [2025-10-20T22:08:11.626508] [DEBUG] [LOCAL ] EDU: Loaded 44811 existing records from primary storage. [2025-10-20T22:08:11.641184] [INFO ] [MAIN ] EDU: Loaded 44811 primary data records. [2025-10-20T22:08:12.226929] [DEBUG] [MAIN ] EDN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:12.228483] [INFO ] [MAIN ] EDN: Generated 1794 derivative records from 19210 primary records. [2025-10-20T22:08:12.228579] [DEBUG] [LOCAL ] EDN: Saving derivative data. [2025-10-20T22:08:12.244756] [DEBUG] [LOCAL ] EDN: Derivative data saved (1794 records). [2025-10-20T22:08:12.245324] [INFO ] [MAIN ] EDN: Successfully saved derivative data. [2025-10-20T22:08:12.248273] [INFO ] [MAIN ] Derivative Progress: 562/1877 symbols completed. EDN took 1.1s. Estimated remaining: 8m 16s. Rate: 159.0 symbols/min [2025-10-20T22:08:12.248461] [INFO ] [MAIN ] EE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:12.270300] [DEBUG] [MAIN ] ED: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:12.271428] [INFO ] [MAIN ] ED: Generated 1794 derivative records from 52649 primary records. [2025-10-20T22:08:12.271674] [DEBUG] [LOCAL ] ED: Saving derivative data. [2025-10-20T22:08:12.304514] [DEBUG] [LOCAL ] ED: Derivative data saved (1794 records). [2025-10-20T22:08:12.304693] [INFO ] [MAIN ] ED: Successfully saved derivative data. [2025-10-20T22:08:12.315158] [INFO ] [MAIN ] Derivative Progress: 563/1877 symbols completed. ED took 1.2s. Estimated remaining: 8m 15s. Rate: 159.2 symbols/min [2025-10-20T22:08:12.315484] [INFO ] [MAIN ] EEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:12.316936] [DEBUG] [MAIN ] EDU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:12.317785] [INFO ] [MAIN ] EDU: Generated 1794 derivative records from 44811 primary records. [2025-10-20T22:08:12.317847] [DEBUG] [LOCAL ] EDU: Saving derivative data. [2025-10-20T22:08:12.347089] [DEBUG] [LOCAL ] EDU: Derivative data saved (1794 records). [2025-10-20T22:08:12.347782] [DEBUG] [LOCAL ] EE: Loaded 32694 existing records from primary storage. [2025-10-20T22:08:12.347305] [INFO ] [MAIN ] EDU: Successfully saved derivative data. [2025-10-20T22:08:12.347954] [INFO ] [MAIN ] EE: Loaded 32694 primary data records. [2025-10-20T22:08:12.350069] [INFO ] [MAIN ] Derivative Progress: 564/1877 symbols completed. EDU took 1.1s. Estimated remaining: 8m 14s. Rate: 159.5 symbols/min [2025-10-20T22:08:12.350387] [INFO ] [MAIN ] EFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:12.366910] [DEBUG] [LOCAL ] EEX: Loaded 10029 existing records from primary storage. [2025-10-20T22:08:12.398755] [INFO ] [MAIN ] EEX: Loaded 10029 primary data records. [2025-10-20T22:08:12.983010] [DEBUG] [LOCAL ] EFC: Loaded 46426 existing records from primary storage. [2025-10-20T22:08:12.984342] [INFO ] [MAIN ] EFC: Loaded 46426 primary data records. [2025-10-20T22:08:13.312482] [DEBUG] [MAIN ] EEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:13.314268] [INFO ] [MAIN ] EEX: Generated 1794 derivative records from 10029 primary records. [2025-10-20T22:08:13.315768] [DEBUG] [LOCAL ] EEX: Saving derivative data. [2025-10-20T22:08:13.344012] [DEBUG] [LOCAL ] EEX: Derivative data saved (1794 records). [2025-10-20T22:08:13.344489] [INFO ] [MAIN ] EEX: Successfully saved derivative data. [2025-10-20T22:08:13.344436] [DEBUG] [MAIN ] EE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:13.347439] [INFO ] [MAIN ] EE: Generated 1794 derivative records from 32694 primary records. [2025-10-20T22:08:13.346534] [INFO ] [MAIN ] Derivative Progress: 565/1877 symbols completed. EEX took 1.0s. Estimated remaining: 8m 15s. Rate: 159.0 symbols/min [2025-10-20T22:08:13.347597] [INFO ] [MAIN ] EFT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:13.347806] [DEBUG] [LOCAL ] EE: Saving derivative data. [2025-10-20T22:08:13.365262] [DEBUG] [LOCAL ] EE: Derivative data saved (1794 records). [2025-10-20T22:08:13.371492] [INFO ] [MAIN ] EE: Successfully saved derivative data. [2025-10-20T22:08:13.373911] [INFO ] [MAIN ] Derivative Progress: 566/1877 symbols completed. EE took 1.1s. Estimated remaining: 8m 14s. Rate: 159.3 symbols/min [2025-10-20T22:08:13.386029] [INFO ] [MAIN ] EFX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:13.391852] [DEBUG] [LOCAL ] EFT: Loaded 12980 existing records from primary storage. [2025-10-20T22:08:13.392543] [DEBUG] [MAIN ] EFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:13.393410] [INFO ] [MAIN ] EFC: Generated 1794 derivative records from 46426 primary records. [2025-10-20T22:08:13.393582] [DEBUG] [LOCAL ] EFC: Saving derivative data. [2025-10-20T22:08:13.392792] [INFO ] [MAIN ] EFT: Loaded 12980 primary data records. [2025-10-20T22:08:13.452088] [DEBUG] [LOCAL ] EFC: Derivative data saved (1794 records). [2025-10-20T22:08:13.464049] [INFO ] [MAIN ] EFC: Successfully saved derivative data. [2025-10-20T22:08:13.467547] [INFO ] [MAIN ] Derivative Progress: 567/1877 symbols completed. EFC took 1.1s. Estimated remaining: 8m 13s. Rate: 159.5 symbols/min [2025-10-20T22:08:13.480429] [INFO ] [MAIN ] EFXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:13.736890] [DEBUG] [LOCAL ] EFX: Loaded 44781 existing records from primary storage. [2025-10-20T22:08:13.737864] [INFO ] [MAIN ] EFX: Loaded 44781 primary data records. [2025-10-20T22:08:13.827228] [DEBUG] [LOCAL ] EFXT: Loaded 29637 existing records from primary storage. [2025-10-20T22:08:13.854373] [INFO ] [MAIN ] EFXT: Loaded 29637 primary data records. [2025-10-20T22:08:14.401995] [DEBUG] [MAIN ] EFT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:14.405566] [INFO ] [MAIN ] EFT: Generated 1794 derivative records from 12980 primary records. [2025-10-20T22:08:14.406054] [DEBUG] [LOCAL ] EFT: Saving derivative data. [2025-10-20T22:08:14.423149] [DEBUG] [LOCAL ] EFT: Derivative data saved (1794 records). [2025-10-20T22:08:14.423939] [INFO ] [MAIN ] EFT: Successfully saved derivative data. [2025-10-20T22:08:14.426121] [INFO ] [MAIN ] Derivative Progress: 568/1877 symbols completed. EFT took 1.1s. Estimated remaining: 8m 14s. Rate: 159.1 symbols/min [2025-10-20T22:08:14.446785] [DEBUG] [MAIN ] EFX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:14.447540] [INFO ] [MAIN ] EFX: Generated 1794 derivative records from 44781 primary records. [2025-10-20T22:08:14.447714] [DEBUG] [LOCAL ] EFX: Saving derivative data. [2025-10-20T22:08:14.460576] [DEBUG] [MAIN ] EFXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:14.472599] [INFO ] [MAIN ] EFXT: Generated 1794 derivative records from 29637 primary records. [2025-10-20T22:08:14.475299] [DEBUG] [LOCAL ] EFXT: Saving derivative data. [2025-10-20T22:08:14.480050] [DEBUG] [LOCAL ] EFX: Derivative data saved (1794 records). [2025-10-20T22:08:14.480264] [INFO ] [MAIN ] EFX: Successfully saved derivative data. [2025-10-20T22:08:14.482211] [INFO ] [MAIN ] Derivative Progress: 569/1877 symbols completed. EFX took 1.1s. Estimated remaining: 8m 13s. Rate: 159.3 symbols/min [2025-10-20T22:08:14.496609] [DEBUG] [LOCAL ] EFXT: Derivative data saved (1794 records). [2025-10-20T22:08:14.496701] [INFO ] [MAIN ] EFXT: Successfully saved derivative data. [2025-10-20T22:08:14.498872] [INFO ] [MAIN ] Derivative Progress: 570/1877 symbols completed. EFXT took 1.0s. Estimated remaining: 8m 11s. Rate: 159.6 symbols/min [2025-10-20T22:08:14.499248] [INFO ] [MAIN ] EG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:14.499825] [INFO ] [MAIN ] EGO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:14.499368] [INFO ] [MAIN ] EGY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:14.534634] [DEBUG] [LOCAL ] EG: Loaded 27781 existing records from primary storage. [2025-10-20T22:08:14.547114] [INFO ] [MAIN ] EG: Loaded 27781 primary data records. [2025-10-20T22:08:14.649052] [DEBUG] [LOCAL ] EGY: Loaded 41464 existing records from primary storage. [2025-10-20T22:08:14.681588] [INFO ] [MAIN ] EGY: Loaded 41464 primary data records. [2025-10-20T22:08:14.693321] [DEBUG] [LOCAL ] EGO: Loaded 51745 existing records from primary storage. [2025-10-20T22:08:14.703940] [INFO ] [MAIN ] EGO: Loaded 51745 primary data records. [2025-10-20T22:08:15.506081] [DEBUG] [MAIN ] EG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:15.507534] [INFO ] [MAIN ] EG: Generated 1794 derivative records from 27781 primary records. [2025-10-20T22:08:15.507767] [DEBUG] [LOCAL ] EG: Saving derivative data. [2025-10-20T22:08:15.525107] [DEBUG] [LOCAL ] EG: Derivative data saved (1794 records). [2025-10-20T22:08:15.525428] [INFO ] [MAIN ] EG: Successfully saved derivative data. [2025-10-20T22:08:15.527623] [INFO ] [MAIN ] Derivative Progress: 571/1877 symbols completed. EG took 1.0s. Estimated remaining: 8m 13s. Rate: 159.1 symbols/min [2025-10-20T22:08:15.527969] [INFO ] [MAIN ] EHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:15.554166] [DEBUG] [MAIN ] EGY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:15.559920] [INFO ] [MAIN ] EGY: Generated 1794 derivative records from 41464 primary records. [2025-10-20T22:08:15.559242] [DEBUG] [MAIN ] EGO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:15.560530] [INFO ] [MAIN ] EGO: Generated 1794 derivative records from 51745 primary records. [2025-10-20T22:08:15.560687] [DEBUG] [LOCAL ] EGY: Saving derivative data. [2025-10-20T22:08:15.561598] [DEBUG] [LOCAL ] EGO: Saving derivative data. [2025-10-20T22:08:15.592068] [DEBUG] [LOCAL ] EGO: Derivative data saved (1794 records). [2025-10-20T22:08:15.592013] [DEBUG] [LOCAL ] EGY: Derivative data saved (1794 records). [2025-10-20T22:08:15.592264] [INFO ] [MAIN ] EGY: Successfully saved derivative data. [2025-10-20T22:08:15.592345] [INFO ] [MAIN ] EGO: Successfully saved derivative data. [2025-10-20T22:08:15.594222] [INFO ] [MAIN ] Derivative Progress: 572/1877 symbols completed. EGY took 1.1s. Estimated remaining: 8m 11s. Rate: 159.3 symbols/min [2025-10-20T22:08:15.596249] [INFO ] [MAIN ] EHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:15.596483] [INFO ] [MAIN ] Derivative Progress: 573/1877 symbols completed. EGO took 1.1s. Estimated remaining: 8m 10s. Rate: 159.6 symbols/min [2025-10-20T22:08:15.596712] [INFO ] [MAIN ] EIC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:15.617595] [DEBUG] [LOCAL ] EHI: Loaded 10543 existing records from primary storage. [2025-10-20T22:08:15.617858] [INFO ] [MAIN ] EHI: Loaded 10543 primary data records. [2025-10-20T22:08:15.747811] [DEBUG] [LOCAL ] EHC: Loaded 41026 existing records from primary storage. [2025-10-20T22:08:15.751408] [DEBUG] [LOCAL ] EIC: Loaded 22699 existing records from primary storage. [2025-10-20T22:08:15.751853] [INFO ] [MAIN ] EIC: Loaded 22699 primary data records. [2025-10-20T22:08:15.751718] [INFO ] [MAIN ] EHC: Loaded 41026 primary data records. [2025-10-20T22:08:16.608915] [DEBUG] [MAIN ] EHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:16.613070] [INFO ] [MAIN ] EHI: Generated 1794 derivative records from 10543 primary records. [2025-10-20T22:08:16.613516] [DEBUG] [LOCAL ] EHI: Saving derivative data. [2025-10-20T22:08:16.641992] [DEBUG] [MAIN ] EIC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:16.645846] [INFO ] [MAIN ] EIC: Generated 1794 derivative records from 22699 primary records. [2025-10-20T22:08:16.648933] [DEBUG] [LOCAL ] EIC: Saving derivative data. [2025-10-20T22:08:16.649110] [DEBUG] [LOCAL ] EHI: Derivative data saved (1794 records). [2025-10-20T22:08:16.650512] [INFO ] [MAIN ] EHI: Successfully saved derivative data. [2025-10-20T22:08:16.668186] [INFO ] [MAIN ] Derivative Progress: 574/1877 symbols completed. EHI took 1.1s. Estimated remaining: 8m 11s. Rate: 159.1 symbols/min [2025-10-20T22:08:16.682268] [DEBUG] [LOCAL ] EIC: Derivative data saved (1794 records). [2025-10-20T22:08:16.682699] [DEBUG] [MAIN ] EHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:16.683038] [INFO ] [MAIN ] EIC: Successfully saved derivative data. [2025-10-20T22:08:16.683949] [INFO ] [MAIN ] EHC: Generated 1794 derivative records from 41026 primary records. [2025-10-20T22:08:16.686686] [DEBUG] [LOCAL ] EHC: Saving derivative data. [2025-10-20T22:08:16.686596] [INFO ] [MAIN ] EL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:16.686812] [INFO ] [MAIN ] Derivative Progress: 575/1877 symbols completed. EIC took 1.1s. Estimated remaining: 8m 10s. Rate: 159.4 symbols/min [2025-10-20T22:08:16.696986] [INFO ] [MAIN ] ELAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:16.705505] [DEBUG] [LOCAL ] EHC: Derivative data saved (1794 records). [2025-10-20T22:08:16.705850] [INFO ] [MAIN ] EHC: Successfully saved derivative data. [2025-10-20T22:08:16.707855] [INFO ] [MAIN ] Derivative Progress: 576/1877 symbols completed. EHC took 1.2s. Estimated remaining: 8m 9s. Rate: 159.6 symbols/min [2025-10-20T22:08:16.708201] [INFO ] [MAIN ] ELC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:16.715365] [DEBUG] [LOCAL ] ELC: Loaded 3476 existing records from primary storage. [2025-10-20T22:08:16.715686] [INFO ] [MAIN ] ELC: Loaded 3476 primary data records. [2025-10-20T22:08:17.077970] [DEBUG] [LOCAL ] ELAN: Loaded 53315 existing records from primary storage. [2025-10-20T22:08:17.078363] [INFO ] [MAIN ] ELAN: Loaded 53315 primary data records. [2025-10-20T22:08:17.084416] [DEBUG] [LOCAL ] EL: Loaded 53387 existing records from primary storage. [2025-10-20T22:08:17.102390] [INFO ] [MAIN ] EL: Loaded 53387 primary data records. [2025-10-20T22:08:17.791507] [DEBUG] [MAIN ] ELC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:17.792736] [INFO ] [MAIN ] ELC: Generated 1794 derivative records from 3476 primary records. [2025-10-20T22:08:17.793175] [DEBUG] [LOCAL ] ELC: Saving derivative data. [2025-10-20T22:08:17.811677] [DEBUG] [LOCAL ] ELC: Derivative data saved (1794 records). [2025-10-20T22:08:17.811823] [INFO ] [MAIN ] ELC: Successfully saved derivative data. [2025-10-20T22:08:17.814661] [INFO ] [MAIN ] Derivative Progress: 577/1877 symbols completed. ELC took 1.1s. Estimated remaining: 8m 10s. Rate: 159.1 symbols/min [2025-10-20T22:08:17.815416] [INFO ] [MAIN ] ELME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:17.851638] [DEBUG] [MAIN ] ELAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:17.853091] [INFO ] [MAIN ] ELAN: Generated 1794 derivative records from 53315 primary records. [2025-10-20T22:08:17.851989] [DEBUG] [MAIN ] EL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:17.853995] [INFO ] [MAIN ] EL: Generated 1794 derivative records from 53387 primary records. [2025-10-20T22:08:17.854161] [DEBUG] [LOCAL ] EL: Saving derivative data. [2025-10-20T22:08:17.854107] [DEBUG] [LOCAL ] ELAN: Saving derivative data. [2025-10-20T22:08:17.899221] [DEBUG] [LOCAL ] ELAN: Derivative data saved (1794 records). [2025-10-20T22:08:17.899413] [INFO ] [MAIN ] ELAN: Successfully saved derivative data. [2025-10-20T22:08:17.901373] [DEBUG] [LOCAL ] EL: Derivative data saved (1794 records). [2025-10-20T22:08:17.901551] [INFO ] [MAIN ] EL: Successfully saved derivative data. [2025-10-20T22:08:17.904183] [INFO ] [MAIN ] Derivative Progress: 578/1877 symbols completed. EL took 1.2s. Estimated remaining: 8m 9s. Rate: 159.3 symbols/min [2025-10-20T22:08:17.904397] [INFO ] [MAIN ] ELS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:17.907220] [INFO ] [MAIN ] Derivative Progress: 579/1877 symbols completed. ELAN took 1.2s. Estimated remaining: 8m 8s. Rate: 159.6 symbols/min [2025-10-20T22:08:17.907973] [INFO ] [MAIN ] ELV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:17.922454] [DEBUG] [LOCAL ] ELME: Loaded 39361 existing records from primary storage. [2025-10-20T22:08:17.922684] [INFO ] [MAIN ] ELME: Loaded 39361 primary data records. [2025-10-20T22:08:18.259949] [DEBUG] [LOCAL ] ELS: Loaded 49903 existing records from primary storage. [2025-10-20T22:08:18.260917] [INFO ] [MAIN ] ELS: Loaded 49903 primary data records. [2025-10-20T22:08:18.381182] [DEBUG] [LOCAL ] ELV: Loaded 50756 existing records from primary storage. [2025-10-20T22:08:18.392477] [INFO ] [MAIN ] ELV: Loaded 50756 primary data records. [2025-10-20T22:08:18.903805] [DEBUG] [MAIN ] ELME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:18.905766] [INFO ] [MAIN ] ELME: Generated 1794 derivative records from 39361 primary records. [2025-10-20T22:08:18.906261] [DEBUG] [LOCAL ] ELME: Saving derivative data. [2025-10-20T22:08:18.925434] [DEBUG] [LOCAL ] ELME: Derivative data saved (1794 records). [2025-10-20T22:08:18.926429] [INFO ] [MAIN ] ELME: Successfully saved derivative data. [2025-10-20T22:08:18.928801] [INFO ] [MAIN ] Derivative Progress: 580/1877 symbols completed. ELME took 1.1s. Estimated remaining: 8m 9s. Rate: 159.1 symbols/min [2025-10-20T22:08:18.929063] [INFO ] [MAIN ] EMD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:18.947602] [DEBUG] [MAIN ] ELS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:18.948684] [INFO ] [MAIN ] ELS: Generated 1794 derivative records from 49903 primary records. [2025-10-20T22:08:18.948853] [DEBUG] [LOCAL ] ELS: Saving derivative data. [2025-10-20T22:08:18.970125] [DEBUG] [LOCAL ] ELS: Derivative data saved (1794 records). [2025-10-20T22:08:18.970457] [INFO ] [MAIN ] ELS: Successfully saved derivative data. [2025-10-20T22:08:18.972231] [INFO ] [MAIN ] Derivative Progress: 581/1877 symbols completed. ELS took 1.1s. Estimated remaining: 8m 8s. Rate: 159.3 symbols/min [2025-10-20T22:08:18.972392] [INFO ] [MAIN ] EME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:18.998070] [DEBUG] [LOCAL ] EMD: Loaded 20926 existing records from primary storage. [2025-10-20T22:08:18.998743] [INFO ] [MAIN ] EMD: Loaded 20926 primary data records. [2025-10-20T22:08:18.998455] [DEBUG] [MAIN ] ELV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:19.000048] [INFO ] [MAIN ] ELV: Generated 1794 derivative records from 50756 primary records. [2025-10-20T22:08:19.000161] [DEBUG] [LOCAL ] ELV: Saving derivative data. [2025-10-20T22:08:19.114308] [DEBUG] [LOCAL ] ELV: Derivative data saved (1794 records). [2025-10-20T22:08:19.132892] [INFO ] [MAIN ] ELV: Successfully saved derivative data. [2025-10-20T22:08:19.159428] [DEBUG] [LOCAL ] EME: Loaded 30809 existing records from primary storage. [2025-10-20T22:08:19.158538] [INFO ] [MAIN ] Derivative Progress: 582/1877 symbols completed. ELV took 1.3s. Estimated remaining: 8m 7s. Rate: 159.5 symbols/min [2025-10-20T22:08:19.160135] [INFO ] [MAIN ] EMF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:19.160038] [INFO ] [MAIN ] EME: Loaded 30809 primary data records. [2025-10-20T22:08:19.341317] [DEBUG] [LOCAL ] EMF: Loaded 3172 existing records from primary storage. [2025-10-20T22:08:19.484916] [INFO ] [MAIN ] EMF: Loaded 3172 primary data records. [2025-10-20T22:08:20.056240] [DEBUG] [MAIN ] EMD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:20.057239] [INFO ] [MAIN ] EMD: Generated 1794 derivative records from 20926 primary records. [2025-10-20T22:08:20.057815] [DEBUG] [LOCAL ] EMD: Saving derivative data. [2025-10-20T22:08:20.078190] [DEBUG] [LOCAL ] EMD: Derivative data saved (1794 records). [2025-10-20T22:08:20.085990] [INFO ] [MAIN ] EMD: Successfully saved derivative data. [2025-10-20T22:08:20.093149] [INFO ] [MAIN ] Derivative Progress: 583/1877 symbols completed. EMD took 1.2s. Estimated remaining: 8m 8s. Rate: 159.1 symbols/min [2025-10-20T22:08:20.091223] [DEBUG] [MAIN ] EME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:20.094295] [INFO ] [MAIN ] EME: Generated 1794 derivative records from 30809 primary records. [2025-10-20T22:08:20.094469] [DEBUG] [LOCAL ] EME: Saving derivative data. [2025-10-20T22:08:20.111451] [DEBUG] [LOCAL ] EME: Derivative data saved (1794 records). [2025-10-20T22:08:20.113487] [INFO ] [MAIN ] EME: Successfully saved derivative data. [2025-10-20T22:08:20.115385] [INFO ] [MAIN ] Derivative Progress: 584/1877 symbols completed. EME took 1.1s. Estimated remaining: 8m 7s. Rate: 159.3 symbols/min [2025-10-20T22:08:20.128412] [DEBUG] [MAIN ] EMF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:20.129095] [INFO ] [MAIN ] EMF: Generated 1794 derivative records from 3172 primary records. [2025-10-20T22:08:20.129156] [DEBUG] [LOCAL ] EMF: Saving derivative data. [2025-10-20T22:08:20.142452] [DEBUG] [LOCAL ] EMF: Derivative data saved (1794 records). [2025-10-20T22:08:20.142510] [INFO ] [MAIN ] EMF: Successfully saved derivative data. [2025-10-20T22:08:20.144463] [INFO ] [MAIN ] Derivative Progress: 585/1877 symbols completed. EMF took 1.0s. Estimated remaining: 8m 6s. Rate: 159.6 symbols/min [2025-10-20T22:08:20.144983] [INFO ] [MAIN ] EMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:20.145080] [INFO ] [MAIN ] EMP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:20.144908] [INFO ] [MAIN ] EMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:20.150531] [DEBUG] [LOCAL ] EMP: Loaded 2945 existing records from primary storage. [2025-10-20T22:08:20.150883] [INFO ] [MAIN ] EMP: Loaded 2945 primary data records. [2025-10-20T22:08:20.250942] [DEBUG] [LOCAL ] EMO: Loaded 13342 existing records from primary storage. [2025-10-20T22:08:20.273708] [INFO ] [MAIN ] EMO: Loaded 13342 primary data records. [2025-10-20T22:08:20.688488] [DEBUG] [LOCAL ] EMN: Loaded 50838 existing records from primary storage. [2025-10-20T22:08:20.794942] [INFO ] [MAIN ] EMN: Loaded 50838 primary data records. [2025-10-20T22:08:21.169782] [DEBUG] [MAIN ] EMP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:21.171668] [DEBUG] [MAIN ] EMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:21.172439] [INFO ] [MAIN ] EMP: Generated 1794 derivative records from 2945 primary records. [2025-10-20T22:08:21.173679] [INFO ] [MAIN ] EMO: Generated 1794 derivative records from 13342 primary records. [2025-10-20T22:08:21.173930] [DEBUG] [LOCAL ] EMO: Saving derivative data. [2025-10-20T22:08:21.174498] [DEBUG] [LOCAL ] EMP: Saving derivative data. [2025-10-20T22:08:21.201144] [DEBUG] [LOCAL ] EMP: Derivative data saved (1794 records). [2025-10-20T22:08:21.204108] [INFO ] [MAIN ] EMP: Successfully saved derivative data. [2025-10-20T22:08:21.206928] [INFO ] [MAIN ] Derivative Progress: 586/1877 symbols completed. EMP took 1.1s. Estimated remaining: 8m 7s. Rate: 159.1 symbols/min [2025-10-20T22:08:21.208309] [DEBUG] [LOCAL ] EMO: Derivative data saved (1794 records). [2025-10-20T22:08:21.208879] [INFO ] [MAIN ] EMO: Successfully saved derivative data. [2025-10-20T22:08:21.210742] [INFO ] [MAIN ] Derivative Progress: 587/1877 symbols completed. EMO took 1.1s. Estimated remaining: 8m 6s. Rate: 159.4 symbols/min [2025-10-20T22:08:21.207867] [INFO ] [MAIN ] EMR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:21.211316] [INFO ] [MAIN ] ENB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:21.229321] [DEBUG] [MAIN ] EMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:21.230143] [INFO ] [MAIN ] EMN: Generated 1794 derivative records from 50838 primary records. [2025-10-20T22:08:21.230301] [DEBUG] [LOCAL ] EMN: Saving derivative data. [2025-10-20T22:08:21.246699] [DEBUG] [LOCAL ] EMN: Derivative data saved (1794 records). [2025-10-20T22:08:21.246898] [INFO ] [MAIN ] EMN: Successfully saved derivative data. [2025-10-20T22:08:21.249113] [INFO ] [MAIN ] Derivative Progress: 588/1877 symbols completed. EMN took 1.1s. Estimated remaining: 8m 5s. Rate: 159.6 symbols/min [2025-10-20T22:08:21.250056] [INFO ] [MAIN ] ENIC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:21.321255] [DEBUG] [LOCAL ] EMR: Loaded 53176 existing records from primary storage. [2025-10-20T22:08:21.321493] [INFO ] [MAIN ] EMR: Loaded 53176 primary data records. [2025-10-20T22:08:21.342683] [DEBUG] [LOCAL ] ENIC: Loaded 36571 existing records from primary storage. [2025-10-20T22:08:21.375496] [INFO ] [MAIN ] ENIC: Loaded 36571 primary data records. [2025-10-20T22:08:21.370182] [DEBUG] [LOCAL ] ENB: Loaded 53511 existing records from primary storage. [2025-10-20T22:08:21.403279] [INFO ] [MAIN ] ENB: Loaded 53511 primary data records. [2025-10-20T22:08:22.352309] [DEBUG] [MAIN ] EMR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:22.353918] [INFO ] [MAIN ] EMR: Generated 1794 derivative records from 53176 primary records. [2025-10-20T22:08:22.354013] [DEBUG] [LOCAL ] EMR: Saving derivative data. [2025-10-20T22:08:22.401370] [DEBUG] [LOCAL ] EMR: Derivative data saved (1794 records). [2025-10-20T22:08:22.400907] [DEBUG] [MAIN ] ENIC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:22.402773] [INFO ] [MAIN ] ENIC: Generated 1794 derivative records from 36571 primary records. [2025-10-20T22:08:22.402882] [DEBUG] [LOCAL ] ENIC: Saving derivative data. [2025-10-20T22:08:22.401896] [DEBUG] [MAIN ] ENB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:22.404520] [INFO ] [MAIN ] ENB: Generated 1794 derivative records from 53511 primary records. [2025-10-20T22:08:22.403765] [INFO ] [MAIN ] EMR: Successfully saved derivative data. [2025-10-20T22:08:22.404741] [DEBUG] [LOCAL ] ENB: Saving derivative data. [2025-10-20T22:08:22.407013] [INFO ] [MAIN ] Derivative Progress: 589/1877 symbols completed. EMR took 1.2s. Estimated remaining: 8m 6s. Rate: 159.0 symbols/min [2025-10-20T22:08:22.408191] [INFO ] [MAIN ] ENOV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:22.435697] [DEBUG] [LOCAL ] ENB: Derivative data saved (1794 records). [2025-10-20T22:08:22.435909] [INFO ] [MAIN ] ENB: Successfully saved derivative data. [2025-10-20T22:08:22.438494] [INFO ] [MAIN ] Derivative Progress: 590/1877 symbols completed. ENB took 1.2s. Estimated remaining: 8m 5s. Rate: 159.3 symbols/min [2025-10-20T22:08:22.442733] [INFO ] [MAIN ] ENR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:22.442551] [DEBUG] [LOCAL ] ENIC: Derivative data saved (1794 records). [2025-10-20T22:08:22.443379] [INFO ] [MAIN ] ENIC: Successfully saved derivative data. [2025-10-20T22:08:22.445549] [INFO ] [MAIN ] Derivative Progress: 591/1877 symbols completed. ENIC took 1.2s. Estimated remaining: 8m 4s. Rate: 159.5 symbols/min [2025-10-20T22:08:22.445793] [INFO ] [MAIN ] ENVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:22.489279] [DEBUG] [LOCAL ] ENVA: Loaded 25413 existing records from primary storage. [2025-10-20T22:08:22.489415] [INFO ] [MAIN ] ENVA: Loaded 25413 primary data records. [2025-10-20T22:08:22.625553] [DEBUG] [LOCAL ] ENR: Loaded 43825 existing records from primary storage. [2025-10-20T22:08:22.629610] [DEBUG] [LOCAL ] ENOV: Loaded 44779 existing records from primary storage. [2025-10-20T22:08:22.629783] [INFO ] [MAIN ] ENR: Loaded 43825 primary data records. [2025-10-20T22:08:22.630009] [INFO ] [MAIN ] ENOV: Loaded 44779 primary data records. [2025-10-20T22:08:23.464958] [DEBUG] [MAIN ] ENVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:23.465967] [INFO ] [MAIN ] ENVA: Generated 1794 derivative records from 25413 primary records. [2025-10-20T22:08:23.466227] [DEBUG] [LOCAL ] ENVA: Saving derivative data. [2025-10-20T22:08:23.482891] [DEBUG] [LOCAL ] ENVA: Derivative data saved (1794 records). [2025-10-20T22:08:23.483126] [INFO ] [MAIN ] ENVA: Successfully saved derivative data. [2025-10-20T22:08:23.486623] [INFO ] [MAIN ] Derivative Progress: 592/1877 symbols completed. ENVA took 1.0s. Estimated remaining: 8m 5s. Rate: 159.1 symbols/min [2025-10-20T22:08:23.488310] [INFO ] [MAIN ] EOD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:23.503223] [DEBUG] [MAIN ] ENR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:23.516471] [INFO ] [MAIN ] ENR: Generated 1794 derivative records from 43825 primary records. [2025-10-20T22:08:23.516736] [DEBUG] [LOCAL ] ENR: Saving derivative data. [2025-10-20T22:08:23.517837] [DEBUG] [MAIN ] ENOV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:23.518665] [INFO ] [MAIN ] ENOV: Generated 1794 derivative records from 44779 primary records. [2025-10-20T22:08:23.531240] [DEBUG] [LOCAL ] ENOV: Saving derivative data. [2025-10-20T22:08:23.535618] [DEBUG] [LOCAL ] ENR: Derivative data saved (1794 records). [2025-10-20T22:08:23.539806] [INFO ] [MAIN ] ENR: Successfully saved derivative data. [2025-10-20T22:08:23.542039] [INFO ] [MAIN ] Derivative Progress: 593/1877 symbols completed. ENR took 1.1s. Estimated remaining: 8m 4s. Rate: 159.3 symbols/min [2025-10-20T22:08:23.553469] [INFO ] [MAIN ] EOG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:23.563983] [DEBUG] [LOCAL ] EOD: Loaded 15056 existing records from primary storage. [2025-10-20T22:08:23.565029] [DEBUG] [LOCAL ] ENOV: Derivative data saved (1794 records). [2025-10-20T22:08:23.565100] [INFO ] [MAIN ] EOD: Loaded 15056 primary data records. [2025-10-20T22:08:23.565502] [INFO ] [MAIN ] ENOV: Successfully saved derivative data. [2025-10-20T22:08:23.572533] [INFO ] [MAIN ] Derivative Progress: 594/1877 symbols completed. ENOV took 1.2s. Estimated remaining: 8m 2s. Rate: 159.6 symbols/min [2025-10-20T22:08:23.589017] [INFO ] [MAIN ] EOI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:23.907079] [DEBUG] [LOCAL ] EOI: Loaded 10601 existing records from primary storage. [2025-10-20T22:08:23.915056] [INFO ] [MAIN ] EOI: Loaded 10601 primary data records. [2025-10-20T22:08:24.045077] [DEBUG] [LOCAL ] EOG: Loaded 53391 existing records from primary storage. [2025-10-20T22:08:24.067238] [INFO ] [MAIN ] EOG: Loaded 53391 primary data records. [2025-10-20T22:08:24.574278] [DEBUG] [MAIN ] EOD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:24.575562] [INFO ] [MAIN ] EOD: Generated 1794 derivative records from 15056 primary records. [2025-10-20T22:08:24.575925] [DEBUG] [LOCAL ] EOD: Saving derivative data. [2025-10-20T22:08:24.595296] [DEBUG] [LOCAL ] EOD: Derivative data saved (1794 records). [2025-10-20T22:08:24.595791] [INFO ] [MAIN ] EOD: Successfully saved derivative data. [2025-10-20T22:08:24.597778] [INFO ] [MAIN ] Derivative Progress: 595/1877 symbols completed. EOD took 1.1s. Estimated remaining: 8m 4s. Rate: 159.1 symbols/min [2025-10-20T22:08:24.598186] [INFO ] [MAIN ] EOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:24.616451] [DEBUG] [MAIN ] EOI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:24.617503] [INFO ] [MAIN ] EOI: Generated 1794 derivative records from 10601 primary records. [2025-10-20T22:08:24.617600] [DEBUG] [LOCAL ] EOI: Saving derivative data. [2025-10-20T22:08:24.638705] [DEBUG] [LOCAL ] EOI: Derivative data saved (1794 records). [2025-10-20T22:08:24.639095] [INFO ] [MAIN ] EOI: Successfully saved derivative data. [2025-10-20T22:08:24.641228] [INFO ] [MAIN ] Derivative Progress: 596/1877 symbols completed. EOI took 1.1s. Estimated remaining: 8m 2s. Rate: 159.3 symbols/min [2025-10-20T22:08:24.646779] [INFO ] [MAIN ] EPAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:24.649244] [DEBUG] [LOCAL ] EOS: Loaded 14852 existing records from primary storage. [2025-10-20T22:08:24.651104] [INFO ] [MAIN ] EOS: Loaded 14852 primary data records. [2025-10-20T22:08:24.727867] [DEBUG] [MAIN ] EOG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:24.746049] [INFO ] [MAIN ] EOG: Generated 1794 derivative records from 53391 primary records. [2025-10-20T22:08:24.772296] [DEBUG] [LOCAL ] EOG: Saving derivative data. [2025-10-20T22:08:24.857707] [DEBUG] [LOCAL ] EOG: Derivative data saved (1794 records). [2025-10-20T22:08:24.856086] [DEBUG] [LOCAL ] EPAC: Loaded 26516 existing records from primary storage. [2025-10-20T22:08:24.857862] [INFO ] [MAIN ] EOG: Successfully saved derivative data. [2025-10-20T22:08:24.860295] [INFO ] [MAIN ] EPAC: Loaded 26516 primary data records. [2025-10-20T22:08:24.860197] [INFO ] [MAIN ] Derivative Progress: 597/1877 symbols completed. EOG took 1.3s. Estimated remaining: 8m 2s. Rate: 159.4 symbols/min [2025-10-20T22:08:24.887979] [INFO ] [MAIN ] EPAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:25.344002] [DEBUG] [LOCAL ] EPAM: Loaded 39399 existing records from primary storage. [2025-10-20T22:08:25.344767] [INFO ] [MAIN ] EPAM: Loaded 39399 primary data records. [2025-10-20T22:08:25.369304] [DEBUG] [MAIN ] EOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:25.422968] [INFO ] [MAIN ] EOS: Generated 1794 derivative records from 14852 primary records. [2025-10-20T22:08:25.445487] [DEBUG] [LOCAL ] EOS: Saving derivative data. [2025-10-20T22:08:25.629719] [DEBUG] [LOCAL ] EOS: Derivative data saved (1794 records). [2025-10-20T22:08:25.716862] [INFO ] [MAIN ] EOS: Successfully saved derivative data. [2025-10-20T22:08:25.720948] [INFO ] [MAIN ] Derivative Progress: 598/1877 symbols completed. EOS took 1.1s. Estimated remaining: 8m 2s. Rate: 159.1 symbols/min [2025-10-20T22:08:25.734266] [DEBUG] [MAIN ] EPAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:25.734965] [INFO ] [MAIN ] EPAC: Generated 1794 derivative records from 26516 primary records. [2025-10-20T22:08:25.735619] [DEBUG] [LOCAL ] EPAC: Saving derivative data. [2025-10-20T22:08:25.753938] [DEBUG] [LOCAL ] EPAC: Derivative data saved (1794 records). [2025-10-20T22:08:25.754022] [INFO ] [MAIN ] EPAC: Successfully saved derivative data. [2025-10-20T22:08:25.755930] [INFO ] [MAIN ] Derivative Progress: 599/1877 symbols completed. EPAC took 1.1s. Estimated remaining: 8m 1s. Rate: 159.3 symbols/min [2025-10-20T22:08:25.774995] [DEBUG] [MAIN ] EPAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:25.775579] [INFO ] [MAIN ] EPAM: Generated 1794 derivative records from 39399 primary records. [2025-10-20T22:08:25.775638] [DEBUG] [LOCAL ] EPAM: Saving derivative data. [2025-10-20T22:08:25.791022] [DEBUG] [LOCAL ] EPAM: Derivative data saved (1794 records). [2025-10-20T22:08:25.791082] [INFO ] [MAIN ] EPAM: Successfully saved derivative data. [2025-10-20T22:08:25.792831] [INFO ] [MAIN ] Derivative Progress: 600/1877 symbols completed. EPAM took 0.9s. Estimated remaining: 8m 0s. Rate: 159.6 symbols/min [2025-10-20T22:08:25.793301] [INFO ] [MAIN ] EPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:25.793228] [INFO ] [MAIN ] EPD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:25.793402] [INFO ] [MAIN ] EPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:25.853528] [DEBUG] [LOCAL ] EPC: Loaded 39805 existing records from primary storage. [2025-10-20T22:08:25.854035] [INFO ] [MAIN ] EPC: Loaded 39805 primary data records. [2025-10-20T22:08:25.986256] [DEBUG] [LOCAL ] EPR: Loaded 43859 existing records from primary storage. [2025-10-20T22:08:26.003682] [INFO ] [MAIN ] EPR: Loaded 43859 primary data records. [2025-10-20T22:08:25.998387] [DEBUG] [LOCAL ] EPD: Loaded 53612 existing records from primary storage. [2025-10-20T22:08:26.027296] [INFO ] [MAIN ] EPD: Loaded 53612 primary data records. [2025-10-20T22:08:26.835414] [DEBUG] [MAIN ] EPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:26.838523] [INFO ] [MAIN ] EPC: Generated 1794 derivative records from 39805 primary records. [2025-10-20T22:08:26.838926] [DEBUG] [LOCAL ] EPC: Saving derivative data. [2025-10-20T22:08:26.860924] [DEBUG] [MAIN ] EPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:26.867593] [INFO ] [MAIN ] EPR: Generated 1794 derivative records from 43859 primary records. [2025-10-20T22:08:26.867736] [DEBUG] [LOCAL ] EPR: Saving derivative data. [2025-10-20T22:08:26.871039] [DEBUG] [LOCAL ] EPC: Derivative data saved (1794 records). [2025-10-20T22:08:26.871615] [INFO ] [MAIN ] EPC: Successfully saved derivative data. [2025-10-20T22:08:26.875538] [INFO ] [MAIN ] Derivative Progress: 601/1877 symbols completed. EPC took 1.1s. Estimated remaining: 8m 1s. Rate: 159.1 symbols/min [2025-10-20T22:08:26.888243] [INFO ] [MAIN ] EPRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:26.893611] [DEBUG] [LOCAL ] EPR: Derivative data saved (1794 records). [2025-10-20T22:08:26.900222] [INFO ] [MAIN ] EPR: Successfully saved derivative data. [2025-10-20T22:08:26.911271] [INFO ] [MAIN ] Derivative Progress: 602/1877 symbols completed. EPR took 1.1s. Estimated remaining: 8m 0s. Rate: 159.3 symbols/min [2025-10-20T22:08:26.912488] [INFO ] [MAIN ] EQBK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:26.913065] [DEBUG] [MAIN ] EPD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:26.914813] [INFO ] [MAIN ] EPD: Generated 1794 derivative records from 53612 primary records. [2025-10-20T22:08:26.915001] [DEBUG] [LOCAL ] EPD: Saving derivative data. [2025-10-20T22:08:26.953337] [DEBUG] [LOCAL ] EPD: Derivative data saved (1794 records). [2025-10-20T22:08:26.953578] [INFO ] [MAIN ] EPD: Successfully saved derivative data. [2025-10-20T22:08:26.955830] [INFO ] [MAIN ] Derivative Progress: 603/1877 symbols completed. EPD took 1.2s. Estimated remaining: 7m 59s. Rate: 159.6 symbols/min [2025-10-20T22:08:26.956072] [INFO ] [MAIN ] EQNR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:26.961709] [DEBUG] [LOCAL ] EQBK: Loaded 11425 existing records from primary storage. [2025-10-20T22:08:26.962333] [INFO ] [MAIN ] EQBK: Loaded 11425 primary data records. [2025-10-20T22:08:27.081235] [DEBUG] [LOCAL ] EPRT: Loaded 48488 existing records from primary storage. [2025-10-20T22:08:27.108543] [INFO ] [MAIN ] EPRT: Loaded 48488 primary data records. [2025-10-20T22:08:27.416370] [DEBUG] [LOCAL ] EQNR: Loaded 53058 existing records from primary storage. [2025-10-20T22:08:27.427437] [INFO ] [MAIN ] EQNR: Loaded 53058 primary data records. [2025-10-20T22:08:27.966523] [DEBUG] [MAIN ] EQBK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:27.967884] [INFO ] [MAIN ] EQBK: Generated 1794 derivative records from 11425 primary records. [2025-10-20T22:08:27.967992] [DEBUG] [LOCAL ] EQBK: Saving derivative data. [2025-10-20T22:08:27.984165] [DEBUG] [LOCAL ] EQBK: Derivative data saved (1794 records). [2025-10-20T22:08:27.987480] [INFO ] [MAIN ] EQBK: Successfully saved derivative data. [2025-10-20T22:08:27.989449] [INFO ] [MAIN ] Derivative Progress: 604/1877 symbols completed. EQBK took 1.1s. Estimated remaining: 8m 0s. Rate: 159.1 symbols/min [2025-10-20T22:08:27.989725] [INFO ] [MAIN ] EQR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:28.004018] [DEBUG] [MAIN ] EPRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:28.004813] [INFO ] [MAIN ] EPRT: Generated 1794 derivative records from 48488 primary records. [2025-10-20T22:08:28.005085] [DEBUG] [LOCAL ] EPRT: Saving derivative data. [2025-10-20T22:08:28.022117] [DEBUG] [LOCAL ] EPRT: Derivative data saved (1794 records). [2025-10-20T22:08:28.024966] [INFO ] [MAIN ] EPRT: Successfully saved derivative data. [2025-10-20T22:08:28.039350] [INFO ] [MAIN ] Derivative Progress: 605/1877 symbols completed. EPRT took 1.2s. Estimated remaining: 7m 59s. Rate: 159.3 symbols/min [2025-10-20T22:08:28.040179] [INFO ] [MAIN ] EQT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:28.040082] [DEBUG] [MAIN ] EQNR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:28.041076] [INFO ] [MAIN ] EQNR: Generated 1794 derivative records from 53058 primary records. [2025-10-20T22:08:28.041423] [DEBUG] [LOCAL ] EQNR: Saving derivative data. [2025-10-20T22:08:28.070519] [DEBUG] [LOCAL ] EQNR: Derivative data saved (1794 records). [2025-10-20T22:08:28.070803] [INFO ] [MAIN ] EQNR: Successfully saved derivative data. [2025-10-20T22:08:28.073538] [INFO ] [MAIN ] Derivative Progress: 606/1877 symbols completed. EQNR took 1.1s. Estimated remaining: 7m 58s. Rate: 159.6 symbols/min [2025-10-20T22:08:28.073802] [INFO ] [MAIN ] EQV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:28.080516] [DEBUG] [LOCAL ] EQV: Loaded 2745 existing records from primary storage. [2025-10-20T22:08:28.081210] [INFO ] [MAIN ] EQV: Loaded 2745 primary data records. [2025-10-20T22:08:28.188488] [DEBUG] [LOCAL ] EQR: Loaded 51795 existing records from primary storage. [2025-10-20T22:08:28.198944] [INFO ] [MAIN ] EQR: Loaded 51795 primary data records. [2025-10-20T22:08:28.424622] [DEBUG] [LOCAL ] EQT: Loaded 53685 existing records from primary storage. [2025-10-20T22:08:28.574382] [INFO ] [MAIN ] EQT: Loaded 53685 primary data records. [2025-10-20T22:08:29.046368] [DEBUG] [MAIN ] EQV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:29.047627] [INFO ] [MAIN ] EQV: Generated 1794 derivative records from 2745 primary records. [2025-10-20T22:08:29.048618] [DEBUG] [LOCAL ] EQV: Saving derivative data. [2025-10-20T22:08:29.078582] [DEBUG] [LOCAL ] EQV: Derivative data saved (1794 records). [2025-10-20T22:08:29.078517] [DEBUG] [MAIN ] EQR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:29.079387] [INFO ] [MAIN ] EQR: Generated 1794 derivative records from 51795 primary records. [2025-10-20T22:08:29.079582] [DEBUG] [LOCAL ] EQR: Saving derivative data. [2025-10-20T22:08:29.078721] [INFO ] [MAIN ] EQV: Successfully saved derivative data. [2025-10-20T22:08:29.082541] [INFO ] [MAIN ] Derivative Progress: 607/1877 symbols completed. EQV took 1.0s. Estimated remaining: 7m 59s. Rate: 159.1 symbols/min [2025-10-20T22:08:29.090083] [INFO ] [MAIN ] ERJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:29.098188] [DEBUG] [LOCAL ] EQR: Derivative data saved (1794 records). [2025-10-20T22:08:29.098251] [INFO ] [MAIN ] EQR: Successfully saved derivative data. [2025-10-20T22:08:29.100042] [INFO ] [MAIN ] Derivative Progress: 608/1877 symbols completed. EQR took 1.1s. Estimated remaining: 7m 58s. Rate: 159.4 symbols/min [2025-10-20T22:08:29.100405] [INFO ] [MAIN ] ERO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:29.119558] [DEBUG] [MAIN ] EQT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:29.120363] [INFO ] [MAIN ] EQT: Generated 1794 derivative records from 53685 primary records. [2025-10-20T22:08:29.120460] [DEBUG] [LOCAL ] EQT: Saving derivative data. [2025-10-20T22:08:29.140445] [DEBUG] [LOCAL ] EQT: Derivative data saved (1794 records). [2025-10-20T22:08:29.140674] [INFO ] [MAIN ] EQT: Successfully saved derivative data. [2025-10-20T22:08:29.142942] [INFO ] [MAIN ] Derivative Progress: 609/1877 symbols completed. EQT took 1.1s. Estimated remaining: 7m 57s. Rate: 159.6 symbols/min [2025-10-20T22:08:29.143270] [INFO ] [MAIN ] ES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:29.196455] [DEBUG] [LOCAL ] ERJ: Loaded 50301 existing records from primary storage. [2025-10-20T22:08:29.197813] [INFO ] [MAIN ] ERJ: Loaded 50301 primary data records. [2025-10-20T22:08:29.198458] [DEBUG] [LOCAL ] ERO: Loaded 42144 existing records from primary storage. [2025-10-20T22:08:29.198550] [INFO ] [MAIN ] ERO: Loaded 42144 primary data records. [2025-10-20T22:08:29.866161] [DEBUG] [LOCAL ] ES: Loaded 52707 existing records from primary storage. [2025-10-20T22:08:29.868105] [INFO ] [MAIN ] ES: Loaded 52707 primary data records. [2025-10-20T22:08:30.267531] [DEBUG] [MAIN ] ERJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:30.269244] [INFO ] [MAIN ] ERJ: Generated 1794 derivative records from 50301 primary records. [2025-10-20T22:08:30.269601] [DEBUG] [LOCAL ] ERJ: Saving derivative data. [2025-10-20T22:08:30.295110] [DEBUG] [MAIN ] ERO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:30.304171] [INFO ] [MAIN ] ERO: Generated 1794 derivative records from 42144 primary records. [2025-10-20T22:08:30.303081] [DEBUG] [LOCAL ] ERJ: Derivative data saved (1794 records). [2025-10-20T22:08:30.304595] [INFO ] [MAIN ] ERJ: Successfully saved derivative data. [2025-10-20T22:08:30.307015] [INFO ] [MAIN ] Derivative Progress: 610/1877 symbols completed. ERJ took 1.2s. Estimated remaining: 7m 58s. Rate: 159.1 symbols/min [2025-10-20T22:08:30.307198] [DEBUG] [LOCAL ] ERO: Saving derivative data. [2025-10-20T22:08:30.307305] [INFO ] [MAIN ] ESAB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:30.328052] [DEBUG] [LOCAL ] ERO: Derivative data saved (1794 records). [2025-10-20T22:08:30.332585] [INFO ] [MAIN ] ERO: Successfully saved derivative data. [2025-10-20T22:08:30.334907] [INFO ] [MAIN ] Derivative Progress: 611/1877 symbols completed. ERO took 1.2s. Estimated remaining: 7m 57s. Rate: 159.3 symbols/min [2025-10-20T22:08:30.335541] [INFO ] [MAIN ] ESE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:30.364458] [DEBUG] [MAIN ] ES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:30.365248] [INFO ] [MAIN ] ES: Generated 1794 derivative records from 52707 primary records. [2025-10-20T22:08:30.365412] [DEBUG] [LOCAL ] ES: Saving derivative data. [2025-10-20T22:08:30.384866] [DEBUG] [LOCAL ] ES: Derivative data saved (1794 records). [2025-10-20T22:08:30.395341] [INFO ] [MAIN ] ES: Successfully saved derivative data. [2025-10-20T22:08:30.397422] [INFO ] [MAIN ] Derivative Progress: 612/1877 symbols completed. ES took 1.3s. Estimated remaining: 7m 56s. Rate: 159.5 symbols/min [2025-10-20T22:08:30.398954] [DEBUG] [LOCAL ] ESAB: Loaded 26063 existing records from primary storage. [2025-10-20T22:08:30.399096] [INFO ] [MAIN ] ESAB: Loaded 26063 primary data records. [2025-10-20T22:08:30.399220] [INFO ] [MAIN ] ESI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:30.487357] [DEBUG] [LOCAL ] ESE: Loaded 16467 existing records from primary storage. [2025-10-20T22:08:30.498969] [INFO ] [MAIN ] ESE: Loaded 16467 primary data records. [2025-10-20T22:08:30.838877] [DEBUG] [LOCAL ] ESI: Loaded 50643 existing records from primary storage. [2025-10-20T22:08:30.878615] [INFO ] [MAIN ] ESI: Loaded 50643 primary data records. [2025-10-20T22:08:31.542803] [DEBUG] [MAIN ] ESAB: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:31.543762] [INFO ] [MAIN ] ESAB: Generated 1793 derivative records from 26063 primary records. [2025-10-20T22:08:31.544160] [DEBUG] [LOCAL ] ESAB: Saving derivative data. [2025-10-20T22:08:31.556913] [DEBUG] [MAIN ] ESE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:31.566458] [INFO ] [MAIN ] ESE: Generated 1794 derivative records from 16467 primary records. [2025-10-20T22:08:31.566682] [DEBUG] [LOCAL ] ESE: Saving derivative data. [2025-10-20T22:08:31.574282] [DEBUG] [LOCAL ] ESAB: Derivative data saved (1793 records). [2025-10-20T22:08:31.584764] [INFO ] [MAIN ] ESAB: Successfully saved derivative data. [2025-10-20T22:08:31.588007] [INFO ] [MAIN ] Derivative Progress: 613/1877 symbols completed. ESAB took 1.3s. Estimated remaining: 7m 57s. Rate: 159.0 symbols/min [2025-10-20T22:08:31.592941] [DEBUG] [LOCAL ] ESE: Derivative data saved (1794 records). [2025-10-20T22:08:31.594039] [INFO ] [MAIN ] ESE: Successfully saved derivative data. [2025-10-20T22:08:31.596170] [INFO ] [MAIN ] Derivative Progress: 614/1877 symbols completed. ESE took 1.3s. Estimated remaining: 7m 56s. Rate: 159.2 symbols/min [2025-10-20T22:08:31.609958] [DEBUG] [MAIN ] ESI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:31.610638] [INFO ] [MAIN ] ESI: Generated 1794 derivative records from 50643 primary records. [2025-10-20T22:08:31.610761] [DEBUG] [LOCAL ] ESI: Saving derivative data. [2025-10-20T22:08:31.627067] [DEBUG] [LOCAL ] ESI: Derivative data saved (1794 records). [2025-10-20T22:08:31.627139] [INFO ] [MAIN ] ESI: Successfully saved derivative data. [2025-10-20T22:08:31.629416] [INFO ] [MAIN ] Derivative Progress: 615/1877 symbols completed. ESI took 1.2s. Estimated remaining: 7m 55s. Rate: 159.4 symbols/min [2025-10-20T22:08:31.629914] [INFO ] [MAIN ] ESRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:31.630014] [INFO ] [MAIN ] ESTC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:31.629857] [INFO ] [MAIN ] ESNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:31.686485] [DEBUG] [LOCAL ] ESNT: Loaded 38177 existing records from primary storage. [2025-10-20T22:08:31.699689] [INFO ] [MAIN ] ESNT: Loaded 38177 primary data records. [2025-10-20T22:08:31.744504] [DEBUG] [LOCAL ] ESRT: Loaded 46564 existing records from primary storage. [2025-10-20T22:08:31.771478] [INFO ] [MAIN ] ESRT: Loaded 46564 primary data records. [2025-10-20T22:08:31.884827] [DEBUG] [LOCAL ] ESTC: Loaded 50306 existing records from primary storage. [2025-10-20T22:08:31.907162] [INFO ] [MAIN ] ESTC: Loaded 50306 primary data records. [2025-10-20T22:08:32.660755] [DEBUG] [MAIN ] ESNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:32.661992] [INFO ] [MAIN ] ESNT: Generated 1794 derivative records from 38177 primary records. [2025-10-20T22:08:32.662355] [DEBUG] [LOCAL ] ESNT: Saving derivative data. [2025-10-20T22:08:32.681942] [DEBUG] [LOCAL ] ESNT: Derivative data saved (1794 records). [2025-10-20T22:08:32.697742] [INFO ] [MAIN ] ESNT: Successfully saved derivative data. [2025-10-20T22:08:32.699974] [INFO ] [MAIN ] Derivative Progress: 616/1877 symbols completed. ESNT took 1.1s. Estimated remaining: 7m 56s. Rate: 159.0 symbols/min [2025-10-20T22:08:32.700479] [DEBUG] [MAIN ] ESRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:32.701478] [INFO ] [MAIN ] ESRT: Generated 1794 derivative records from 46564 primary records. [2025-10-20T22:08:32.702195] [DEBUG] [LOCAL ] ESRT: Saving derivative data. [2025-10-20T22:08:32.700175] [INFO ] [MAIN ] ET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:32.732640] [DEBUG] [LOCAL ] ESRT: Derivative data saved (1794 records). [2025-10-20T22:08:32.732859] [INFO ] [MAIN ] ESRT: Successfully saved derivative data. [2025-10-20T22:08:32.734977] [INFO ] [MAIN ] Derivative Progress: 617/1877 symbols completed. ESRT took 1.1s. Estimated remaining: 7m 55s. Rate: 159.2 symbols/min [2025-10-20T22:08:32.733135] [DEBUG] [MAIN ] ESTC: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:32.735304] [INFO ] [MAIN ] ETB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:32.736121] [INFO ] [MAIN ] ESTC: Generated 1793 derivative records from 50306 primary records. [2025-10-20T22:08:32.736532] [DEBUG] [LOCAL ] ESTC: Saving derivative data. [2025-10-20T22:08:32.754682] [DEBUG] [LOCAL ] ESTC: Derivative data saved (1793 records). [2025-10-20T22:08:32.755040] [INFO ] [MAIN ] ESTC: Successfully saved derivative data. [2025-10-20T22:08:32.758038] [INFO ] [MAIN ] Derivative Progress: 618/1877 symbols completed. ESTC took 1.1s. Estimated remaining: 7m 54s. Rate: 159.4 symbols/min [2025-10-20T22:08:32.758335] [INFO ] [MAIN ] ETD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:32.768670] [DEBUG] [LOCAL ] ETB: Loaded 11975 existing records from primary storage. [2025-10-20T22:08:32.768890] [INFO ] [MAIN ] ETB: Loaded 11975 primary data records. [2025-10-20T22:08:32.943850] [DEBUG] [LOCAL ] ET: Loaded 53789 existing records from primary storage. [2025-10-20T22:08:32.980067] [INFO ] [MAIN ] ET: Loaded 53789 primary data records. [2025-10-20T22:08:33.063303] [DEBUG] [LOCAL ] ETD: Loaded 27370 existing records from primary storage. [2025-10-20T22:08:33.186689] [INFO ] [MAIN ] ETD: Loaded 27370 primary data records. [2025-10-20T22:08:33.818112] [DEBUG] [MAIN ] ETB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:33.819032] [INFO ] [MAIN ] ETB: Generated 1794 derivative records from 11975 primary records. [2025-10-20T22:08:33.821767] [DEBUG] [LOCAL ] ETB: Saving derivative data. [2025-10-20T22:08:33.851578] [DEBUG] [LOCAL ] ETB: Derivative data saved (1794 records). [2025-10-20T22:08:33.852588] [INFO ] [MAIN ] ETB: Successfully saved derivative data. [2025-10-20T22:08:33.852398] [DEBUG] [MAIN ] ET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:33.855442] [INFO ] [MAIN ] ET: Generated 1794 derivative records from 53789 primary records. [2025-10-20T22:08:33.854364] [INFO ] [MAIN ] Derivative Progress: 619/1877 symbols completed. ETB took 1.1s. Estimated remaining: 7m 55s. Rate: 158.9 symbols/min [2025-10-20T22:08:33.855597] [DEBUG] [LOCAL ] ET: Saving derivative data. [2025-10-20T22:08:33.856596] [INFO ] [MAIN ] ETG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:33.875210] [DEBUG] [LOCAL ] ET: Derivative data saved (1794 records). [2025-10-20T22:08:33.875791] [INFO ] [MAIN ] ET: Successfully saved derivative data. [2025-10-20T22:08:33.877743] [INFO ] [MAIN ] Derivative Progress: 620/1877 symbols completed. ET took 1.2s. Estimated remaining: 7m 54s. Rate: 159.2 symbols/min [2025-10-20T22:08:33.878015] [INFO ] [MAIN ] ETJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:33.899483] [DEBUG] [MAIN ] ETD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:33.900534] [INFO ] [MAIN ] ETD: Generated 1794 derivative records from 27370 primary records. [2025-10-20T22:08:33.900740] [DEBUG] [LOCAL ] ETD: Saving derivative data. [2025-10-20T22:08:33.942219] [DEBUG] [LOCAL ] ETG: Loaded 16667 existing records from primary storage. [2025-10-20T22:08:33.942788] [INFO ] [MAIN ] ETG: Loaded 16667 primary data records. [2025-10-20T22:08:33.943501] [DEBUG] [LOCAL ] ETJ: Loaded 16676 existing records from primary storage. [2025-10-20T22:08:33.946013] [DEBUG] [LOCAL ] ETD: Derivative data saved (1794 records). [2025-10-20T22:08:33.946149] [INFO ] [MAIN ] ETD: Successfully saved derivative data. [2025-10-20T22:08:33.966734] [INFO ] [MAIN ] ETJ: Loaded 16676 primary data records. [2025-10-20T22:08:33.989507] [INFO ] [MAIN ] Derivative Progress: 621/1877 symbols completed. ETD took 1.2s. Estimated remaining: 7m 53s. Rate: 159.4 symbols/min [2025-10-20T22:08:33.989778] [INFO ] [MAIN ] ETN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:34.683774] [DEBUG] [LOCAL ] ETN: Loaded 52584 existing records from primary storage. [2025-10-20T22:08:34.686050] [INFO ] [MAIN ] ETN: Loaded 52584 primary data records. [2025-10-20T22:08:35.015072] [DEBUG] [MAIN ] ETG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:35.015754] [DEBUG] [MAIN ] ETJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:35.016621] [INFO ] [MAIN ] ETG: Generated 1794 derivative records from 16667 primary records. [2025-10-20T22:08:35.018081] [INFO ] [MAIN ] ETJ: Generated 1794 derivative records from 16676 primary records. [2025-10-20T22:08:35.018303] [DEBUG] [LOCAL ] ETJ: Saving derivative data. [2025-10-20T22:08:35.018523] [DEBUG] [LOCAL ] ETG: Saving derivative data. [2025-10-20T22:08:35.047834] [DEBUG] [LOCAL ] ETJ: Derivative data saved (1794 records). [2025-10-20T22:08:35.051391] [INFO ] [MAIN ] ETJ: Successfully saved derivative data. [2025-10-20T22:08:35.053987] [INFO ] [MAIN ] Derivative Progress: 622/1877 symbols completed. ETJ took 1.2s. Estimated remaining: 7m 54s. Rate: 158.9 symbols/min [2025-10-20T22:08:35.052205] [DEBUG] [LOCAL ] ETG: Derivative data saved (1794 records). [2025-10-20T22:08:35.054495] [INFO ] [MAIN ] ETO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:35.054381] [INFO ] [MAIN ] ETG: Successfully saved derivative data. [2025-10-20T22:08:35.056777] [INFO ] [MAIN ] Derivative Progress: 623/1877 symbols completed. ETG took 1.2s. Estimated remaining: 7m 53s. Rate: 159.2 symbols/min [2025-10-20T22:08:35.057479] [INFO ] [MAIN ] ETR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:35.067347] [DEBUG] [LOCAL ] ETO: Loaded 7007 existing records from primary storage. [2025-10-20T22:08:35.067503] [INFO ] [MAIN ] ETO: Loaded 7007 primary data records. [2025-10-20T22:08:35.192584] [DEBUG] [MAIN ] ETN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:35.193304] [INFO ] [MAIN ] ETN: Generated 1794 derivative records from 52584 primary records. [2025-10-20T22:08:35.193690] [DEBUG] [LOCAL ] ETN: Saving derivative data. [2025-10-20T22:08:35.412542] [DEBUG] [LOCAL ] ETN: Derivative data saved (1794 records). [2025-10-20T22:08:35.413291] [INFO ] [MAIN ] ETN: Successfully saved derivative data. [2025-10-20T22:08:35.415484] [INFO ] [MAIN ] Derivative Progress: 624/1877 symbols completed. ETN took 1.4s. Estimated remaining: 7m 52s. Rate: 159.2 symbols/min [2025-10-20T22:08:35.415894] [INFO ] [MAIN ] ETSY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:35.435526] [DEBUG] [LOCAL ] ETR: Loaded 53196 existing records from primary storage. [2025-10-20T22:08:35.435779] [INFO ] [MAIN ] ETR: Loaded 53196 primary data records. [2025-10-20T22:08:35.773582] [DEBUG] [MAIN ] ETO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:35.774497] [INFO ] [MAIN ] ETO: Generated 1794 derivative records from 7007 primary records. [2025-10-20T22:08:35.775028] [DEBUG] [LOCAL ] ETO: Saving derivative data. [2025-10-20T22:08:35.776351] [DEBUG] [LOCAL ] ETSY: Loaded 53569 existing records from primary storage. [2025-10-20T22:08:35.777810] [INFO ] [MAIN ] ETSY: Loaded 53569 primary data records. [2025-10-20T22:08:35.828839] [DEBUG] [LOCAL ] ETO: Derivative data saved (1794 records). [2025-10-20T22:08:35.862048] [INFO ] [MAIN ] ETO: Successfully saved derivative data. [2025-10-20T22:08:35.888098] [INFO ] [MAIN ] Derivative Progress: 625/1877 symbols completed. ETO took 0.8s. Estimated remaining: 7m 52s. Rate: 159.1 symbols/min [2025-10-20T22:08:35.899698] [INFO ] [MAIN ] ETV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:36.160275] [DEBUG] [MAIN ] ETR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:36.161675] [INFO ] [MAIN ] ETR: Generated 1794 derivative records from 53196 primary records. [2025-10-20T22:08:36.161001] [DEBUG] [LOCAL ] ETV: Loaded 22127 existing records from primary storage. [2025-10-20T22:08:36.162044] [DEBUG] [LOCAL ] ETR: Saving derivative data. [2025-10-20T22:08:36.162276] [INFO ] [MAIN ] ETV: Loaded 22127 primary data records. [2025-10-20T22:08:36.306317] [DEBUG] [LOCAL ] ETR: Derivative data saved (1794 records). [2025-10-20T22:08:36.311720] [INFO ] [MAIN ] ETR: Successfully saved derivative data. [2025-10-20T22:08:36.335819] [INFO ] [MAIN ] Derivative Progress: 626/1877 symbols completed. ETR took 1.3s. Estimated remaining: 7m 52s. Rate: 159.1 symbols/min [2025-10-20T22:08:36.365328] [INFO ] [MAIN ] ETW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:36.500280] [DEBUG] [MAIN ] ETSY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:36.501321] [INFO ] [MAIN ] ETSY: Generated 1794 derivative records from 53569 primary records. [2025-10-20T22:08:36.501740] [DEBUG] [LOCAL ] ETSY: Saving derivative data. [2025-10-20T22:08:36.525291] [DEBUG] [LOCAL ] ETSY: Derivative data saved (1794 records). [2025-10-20T22:08:36.525539] [INFO ] [MAIN ] ETSY: Successfully saved derivative data. [2025-10-20T22:08:36.527474] [INFO ] [MAIN ] Derivative Progress: 627/1877 symbols completed. ETSY took 1.1s. Estimated remaining: 7m 51s. Rate: 159.2 symbols/min [2025-10-20T22:08:36.528283] [INFO ] [MAIN ] ETX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:36.537101] [DEBUG] [LOCAL ] ETW: Loaded 25419 existing records from primary storage. [2025-10-20T22:08:36.537262] [INFO ] [MAIN ] ETW: Loaded 25419 primary data records. [2025-10-20T22:08:36.635050] [DEBUG] [LOCAL ] ETX: Loaded 5277 existing records from primary storage. [2025-10-20T22:08:36.646564] [INFO ] [MAIN ] ETX: Loaded 5277 primary data records. [2025-10-20T22:08:36.763181] [DEBUG] [MAIN ] ETV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:36.790330] [INFO ] [MAIN ] ETV: Generated 1794 derivative records from 22127 primary records. [2025-10-20T22:08:36.790433] [DEBUG] [LOCAL ] ETV: Saving derivative data. [2025-10-20T22:08:37.100860] [DEBUG] [LOCAL ] ETV: Derivative data saved (1794 records). [2025-10-20T22:08:37.165196] [INFO ] [MAIN ] ETV: Successfully saved derivative data. [2025-10-20T22:08:37.167631] [INFO ] [MAIN ] Derivative Progress: 628/1877 symbols completed. ETV took 1.3s. Estimated remaining: 7m 51s. Rate: 159.0 symbols/min [2025-10-20T22:08:37.191297] [DEBUG] [MAIN ] ETW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:37.192106] [INFO ] [MAIN ] ETW: Generated 1794 derivative records from 25419 primary records. [2025-10-20T22:08:37.192302] [DEBUG] [LOCAL ] ETW: Saving derivative data. [2025-10-20T22:08:37.222465] [DEBUG] [LOCAL ] ETW: Derivative data saved (1794 records). [2025-10-20T22:08:37.222881] [INFO ] [MAIN ] ETW: Successfully saved derivative data. [2025-10-20T22:08:37.224935] [INFO ] [MAIN ] Derivative Progress: 629/1877 symbols completed. ETW took 0.9s. Estimated remaining: 7m 50s. Rate: 159.2 symbols/min [2025-10-20T22:08:37.222770] [DEBUG] [MAIN ] ETX: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:37.225724] [INFO ] [MAIN ] ETX: Generated 1793 derivative records from 5277 primary records. [2025-10-20T22:08:37.225795] [DEBUG] [LOCAL ] ETX: Saving derivative data. [2025-10-20T22:08:37.239854] [DEBUG] [LOCAL ] ETX: Derivative data saved (1793 records). [2025-10-20T22:08:37.239910] [INFO ] [MAIN ] ETX: Successfully saved derivative data. [2025-10-20T22:08:37.242111] [INFO ] [MAIN ] Derivative Progress: 630/1877 symbols completed. ETX took 0.7s. Estimated remaining: 7m 49s. Rate: 159.5 symbols/min [2025-10-20T22:08:37.242615] [INFO ] [MAIN ] ETY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:37.242754] [INFO ] [MAIN ] EVEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:37.243196] [INFO ] [MAIN ] EVC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:37.281707] [DEBUG] [LOCAL ] EVC: Loaded 18043 existing records from primary storage. [2025-10-20T22:08:37.283117] [DEBUG] [LOCAL ] ETY: Loaded 22685 existing records from primary storage. [2025-10-20T22:08:37.283325] [INFO ] [MAIN ] ETY: Loaded 22685 primary data records. [2025-10-20T22:08:37.283419] [INFO ] [MAIN ] EVC: Loaded 18043 primary data records. [2025-10-20T22:08:37.459990] [DEBUG] [LOCAL ] EVEX: Loaded 32891 existing records from primary storage. [2025-10-20T22:08:37.522360] [INFO ] [MAIN ] EVEX: Loaded 32891 primary data records. [2025-10-20T22:08:38.268875] [DEBUG] [MAIN ] EVC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:38.269883] [INFO ] [MAIN ] EVC: Generated 1794 derivative records from 18043 primary records. [2025-10-20T22:08:38.270719] [DEBUG] [LOCAL ] EVC: Saving derivative data. [2025-10-20T22:08:38.267793] [DEBUG] [MAIN ] ETY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:38.271446] [INFO ] [MAIN ] ETY: Generated 1794 derivative records from 22685 primary records. [2025-10-20T22:08:38.280038] [DEBUG] [LOCAL ] ETY: Saving derivative data. [2025-10-20T22:08:38.288377] [DEBUG] [LOCAL ] EVC: Derivative data saved (1794 records). [2025-10-20T22:08:38.288777] [INFO ] [MAIN ] EVC: Successfully saved derivative data. [2025-10-20T22:08:38.290710] [INFO ] [MAIN ] Derivative Progress: 631/1877 symbols completed. EVC took 1.0s. Estimated remaining: 7m 50s. Rate: 159.0 symbols/min [2025-10-20T22:08:38.290946] [INFO ] [MAIN ] EVF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:38.315017] [DEBUG] [MAIN ] EVEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:38.315883] [INFO ] [MAIN ] EVEX: Generated 1794 derivative records from 32891 primary records. [2025-10-20T22:08:38.316390] [DEBUG] [LOCAL ] EVEX: Saving derivative data. [2025-10-20T22:08:38.321156] [DEBUG] [LOCAL ] ETY: Derivative data saved (1794 records). [2025-10-20T22:08:38.321778] [INFO ] [MAIN ] ETY: Successfully saved derivative data. [2025-10-20T22:08:38.324243] [INFO ] [MAIN ] Derivative Progress: 632/1877 symbols completed. ETY took 1.1s. Estimated remaining: 7m 49s. Rate: 159.2 symbols/min [2025-10-20T22:08:38.324909] [INFO ] [MAIN ] EVG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:38.351253] [DEBUG] [LOCAL ] EVEX: Derivative data saved (1794 records). [2025-10-20T22:08:38.351506] [INFO ] [MAIN ] EVEX: Successfully saved derivative data. [2025-10-20T22:08:38.354445] [INFO ] [MAIN ] Derivative Progress: 633/1877 symbols completed. EVEX took 1.1s. Estimated remaining: 7m 48s. Rate: 159.5 symbols/min [2025-10-20T22:08:38.359576] [INFO ] [MAIN ] EVH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:38.360082] [DEBUG] [LOCAL ] EVF: Loaded 9820 existing records from primary storage. [2025-10-20T22:08:38.361770] [INFO ] [MAIN ] EVF: Loaded 9820 primary data records. [2025-10-20T22:08:38.442718] [DEBUG] [LOCAL ] EVG: Loaded 5867 existing records from primary storage. [2025-10-20T22:08:38.459320] [INFO ] [MAIN ] EVG: Loaded 5867 primary data records. [2025-10-20T22:08:38.856749] [DEBUG] [LOCAL ] EVH: Loaded 49646 existing records from primary storage. [2025-10-20T22:08:38.859704] [INFO ] [MAIN ] EVH: Loaded 49646 primary data records. [2025-10-20T22:08:39.332901] [DEBUG] [MAIN ] EVF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:39.334581] [INFO ] [MAIN ] EVF: Generated 1794 derivative records from 9820 primary records. [2025-10-20T22:08:39.335064] [DEBUG] [LOCAL ] EVF: Saving derivative data. [2025-10-20T22:08:39.351017] [DEBUG] [LOCAL ] EVF: Derivative data saved (1794 records). [2025-10-20T22:08:39.352354] [INFO ] [MAIN ] EVF: Successfully saved derivative data. [2025-10-20T22:08:39.355192] [INFO ] [MAIN ] Derivative Progress: 634/1877 symbols completed. EVF took 1.1s. Estimated remaining: 7m 49s. Rate: 159.1 symbols/min [2025-10-20T22:08:39.357175] [INFO ] [MAIN ] EVN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:39.372034] [DEBUG] [MAIN ] EVG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:39.373010] [INFO ] [MAIN ] EVG: Generated 1794 derivative records from 5867 primary records. [2025-10-20T22:08:39.373286] [DEBUG] [LOCAL ] EVG: Saving derivative data. [2025-10-20T22:08:39.395061] [DEBUG] [LOCAL ] EVN: Loaded 10724 existing records from primary storage. [2025-10-20T22:08:39.397271] [DEBUG] [LOCAL ] EVG: Derivative data saved (1794 records). [2025-10-20T22:08:39.397563] [INFO ] [MAIN ] EVN: Loaded 10724 primary data records. [2025-10-20T22:08:39.398565] [INFO ] [MAIN ] EVG: Successfully saved derivative data. [2025-10-20T22:08:39.401393] [INFO ] [MAIN ] Derivative Progress: 635/1877 symbols completed. EVG took 1.1s. Estimated remaining: 7m 48s. Rate: 159.3 symbols/min [2025-10-20T22:08:39.477230] [INFO ] [MAIN ] EVR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:39.518763] [DEBUG] [MAIN ] EVH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:39.540492] [INFO ] [MAIN ] EVH: Generated 1794 derivative records from 49646 primary records. [2025-10-20T22:08:39.596568] [DEBUG] [LOCAL ] EVH: Saving derivative data. [2025-10-20T22:08:39.727239] [DEBUG] [LOCAL ] EVH: Derivative data saved (1794 records). [2025-10-20T22:08:39.728312] [INFO ] [MAIN ] EVH: Successfully saved derivative data. [2025-10-20T22:08:39.739508] [INFO ] [MAIN ] Derivative Progress: 636/1877 symbols completed. EVH took 1.4s. Estimated remaining: 7m 47s. Rate: 159.3 symbols/min [2025-10-20T22:08:39.739840] [INFO ] [MAIN ] EVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:39.754992] [DEBUG] [LOCAL ] EVR: Loaded 34068 existing records from primary storage. [2025-10-20T22:08:39.755423] [INFO ] [MAIN ] EVR: Loaded 34068 primary data records. [2025-10-20T22:08:39.868885] [DEBUG] [LOCAL ] EVT: Loaded 16762 existing records from primary storage. [2025-10-20T22:08:39.876795] [INFO ] [MAIN ] EVT: Loaded 16762 primary data records. [2025-10-20T22:08:39.897911] [DEBUG] [MAIN ] EVN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:39.953468] [INFO ] [MAIN ] EVN: Generated 1794 derivative records from 10724 primary records. [2025-10-20T22:08:40.034700] [DEBUG] [LOCAL ] EVN: Saving derivative data. [2025-10-20T22:08:40.250557] [DEBUG] [LOCAL ] EVN: Derivative data saved (1794 records). [2025-10-20T22:08:40.274504] [INFO ] [MAIN ] EVN: Successfully saved derivative data. [2025-10-20T22:08:40.305827] [INFO ] [MAIN ] Derivative Progress: 637/1877 symbols completed. EVN took 0.9s. Estimated remaining: 7m 47s. Rate: 159.2 symbols/min [2025-10-20T22:08:40.306319] [INFO ] [MAIN ] EVTC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:40.465812] [DEBUG] [MAIN ] EVR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:40.466771] [INFO ] [MAIN ] EVR: Generated 1794 derivative records from 34068 primary records. [2025-10-20T22:08:40.467024] [DEBUG] [LOCAL ] EVR: Saving derivative data. [2025-10-20T22:08:40.468615] [DEBUG] [LOCAL ] EVTC: Loaded 26895 existing records from primary storage. [2025-10-20T22:08:40.469298] [INFO ] [MAIN ] EVTC: Loaded 26895 primary data records. [2025-10-20T22:08:40.555161] [DEBUG] [LOCAL ] EVR: Derivative data saved (1794 records). [2025-10-20T22:08:40.572336] [INFO ] [MAIN ] EVR: Successfully saved derivative data. [2025-10-20T22:08:40.580416] [INFO ] [MAIN ] Derivative Progress: 638/1877 symbols completed. EVR took 1.1s. Estimated remaining: 7m 47s. Rate: 159.2 symbols/min [2025-10-20T22:08:40.594271] [INFO ] [MAIN ] EVTL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:40.690819] [DEBUG] [MAIN ] EVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:40.800112] [INFO ] [MAIN ] EVT: Generated 1794 derivative records from 16762 primary records. [2025-10-20T22:08:40.808756] [DEBUG] [LOCAL ] EVT: Saving derivative data. [2025-10-20T22:08:40.827728] [DEBUG] [LOCAL ] EVT: Derivative data saved (1794 records). [2025-10-20T22:08:40.837698] [INFO ] [MAIN ] EVT: Successfully saved derivative data. [2025-10-20T22:08:40.840277] [INFO ] [MAIN ] Derivative Progress: 639/1877 symbols completed. EVT took 1.1s. Estimated remaining: 7m 46s. Rate: 159.3 symbols/min [2025-10-20T22:08:40.841114] [INFO ] [MAIN ] EVTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:40.858808] [DEBUG] [LOCAL ] EVTL: Loaded 41005 existing records from primary storage. [2025-10-20T22:08:40.859246] [INFO ] [MAIN ] EVTL: Loaded 41005 primary data records. [2025-10-20T22:08:40.983637] [DEBUG] [LOCAL ] EVTR: Loaded 25188 existing records from primary storage. [2025-10-20T22:08:41.036401] [DEBUG] [MAIN ] EVTC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:41.035901] [INFO ] [MAIN ] EVTR: Loaded 25188 primary data records. [2025-10-20T22:08:41.056065] [INFO ] [MAIN ] EVTC: Generated 1794 derivative records from 26895 primary records. [2025-10-20T22:08:41.098255] [DEBUG] [LOCAL ] EVTC: Saving derivative data. [2025-10-20T22:08:41.328588] [DEBUG] [LOCAL ] EVTC: Derivative data saved (1794 records). [2025-10-20T22:08:41.339938] [INFO ] [MAIN ] EVTC: Successfully saved derivative data. [2025-10-20T22:08:41.399072] [INFO ] [MAIN ] Derivative Progress: 640/1877 symbols completed. EVTC took 1.1s. Estimated remaining: 7m 46s. Rate: 159.2 symbols/min [2025-10-20T22:08:41.405029] [INFO ] [MAIN ] EW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:41.560018] [DEBUG] [MAIN ] EVTL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:41.562340] [INFO ] [MAIN ] EVTL: Generated 1794 derivative records from 41005 primary records. [2025-10-20T22:08:41.562599] [DEBUG] [LOCAL ] EVTL: Saving derivative data. [2025-10-20T22:08:41.584319] [DEBUG] [MAIN ] EVTR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:41.591054] [DEBUG] [LOCAL ] EVTL: Derivative data saved (1794 records). [2025-10-20T22:08:41.591847] [INFO ] [MAIN ] EVTL: Successfully saved derivative data. [2025-10-20T22:08:41.594963] [INFO ] [MAIN ] Derivative Progress: 641/1877 symbols completed. EVTL took 1.0s. Estimated remaining: 7m 45s. Rate: 159.3 symbols/min [2025-10-20T22:08:41.592790] [INFO ] [MAIN ] EVTR: Generated 1793 derivative records from 25188 primary records. [2025-10-20T22:08:41.595158] [INFO ] [MAIN ] EXG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:41.611124] [DEBUG] [LOCAL ] EVTR: Saving derivative data. [2025-10-20T22:08:41.620633] [DEBUG] [LOCAL ] EW: Loaded 53400 existing records from primary storage. [2025-10-20T22:08:41.624705] [INFO ] [MAIN ] EW: Loaded 53400 primary data records. [2025-10-20T22:08:41.628951] [DEBUG] [LOCAL ] EVTR: Derivative data saved (1793 records). [2025-10-20T22:08:41.667022] [INFO ] [MAIN ] EVTR: Successfully saved derivative data. [2025-10-20T22:08:41.711042] [INFO ] [MAIN ] Derivative Progress: 642/1877 symbols completed. EVTR took 0.9s. Estimated remaining: 7m 45s. Rate: 159.5 symbols/min [2025-10-20T22:08:41.716517] [INFO ] [MAIN ] EXK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:41.837277] [DEBUG] [LOCAL ] EXG: Loaded 33887 existing records from primary storage. [2025-10-20T22:08:41.961530] [INFO ] [MAIN ] EXG: Loaded 33887 primary data records. [2025-10-20T22:08:42.311243] [DEBUG] [LOCAL ] EXK: Loaded 53185 existing records from primary storage. [2025-10-20T22:08:42.311538] [INFO ] [MAIN ] EXK: Loaded 53185 primary data records. [2025-10-20T22:08:42.484035] [DEBUG] [MAIN ] EW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:42.506296] [INFO ] [MAIN ] EW: Generated 1794 derivative records from 53400 primary records. [2025-10-20T22:08:42.506399] [DEBUG] [LOCAL ] EW: Saving derivative data. [2025-10-20T22:08:42.664865] [DEBUG] [LOCAL ] EW: Derivative data saved (1794 records). [2025-10-20T22:08:42.665140] [INFO ] [MAIN ] EW: Successfully saved derivative data. [2025-10-20T22:08:42.667321] [INFO ] [MAIN ] Derivative Progress: 643/1877 symbols completed. EW took 1.3s. Estimated remaining: 7m 45s. Rate: 159.1 symbols/min [2025-10-20T22:08:42.685249] [DEBUG] [MAIN ] EXG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:42.686005] [INFO ] [MAIN ] EXG: Generated 1794 derivative records from 33887 primary records. [2025-10-20T22:08:42.686145] [DEBUG] [LOCAL ] EXG: Saving derivative data. [2025-10-20T22:08:42.702606] [DEBUG] [LOCAL ] EXG: Derivative data saved (1794 records). [2025-10-20T22:08:42.702935] [INFO ] [MAIN ] EXG: Successfully saved derivative data. [2025-10-20T22:08:42.705490] [INFO ] [MAIN ] Derivative Progress: 644/1877 symbols completed. EXG took 1.1s. Estimated remaining: 7m 44s. Rate: 159.3 symbols/min [2025-10-20T22:08:42.721390] [DEBUG] [MAIN ] EXK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:42.721984] [INFO ] [MAIN ] EXK: Generated 1794 derivative records from 53185 primary records. [2025-10-20T22:08:42.722044] [DEBUG] [LOCAL ] EXK: Saving derivative data. [2025-10-20T22:08:42.736756] [DEBUG] [LOCAL ] EXK: Derivative data saved (1794 records). [2025-10-20T22:08:42.736815] [INFO ] [MAIN ] EXK: Successfully saved derivative data. [2025-10-20T22:08:42.739179] [INFO ] [MAIN ] Derivative Progress: 645/1877 symbols completed. EXK took 1.0s. Estimated remaining: 7m 43s. Rate: 159.6 symbols/min [2025-10-20T22:08:42.739670] [INFO ] [MAIN ] EXR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:42.739578] [INFO ] [MAIN ] EXP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:42.739791] [INFO ] [MAIN ] F: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:42.779264] [DEBUG] [LOCAL ] EXP: Loaded 29414 existing records from primary storage. [2025-10-20T22:08:42.779391] [INFO ] [MAIN ] EXP: Loaded 29414 primary data records. [2025-10-20T22:08:42.947570] [DEBUG] [LOCAL ] EXR: Loaded 45611 existing records from primary storage. [2025-10-20T22:08:42.982029] [INFO ] [MAIN ] EXR: Loaded 45611 primary data records. [2025-10-20T22:08:43.120902] [DEBUG] [LOCAL ] F: Loaded 53820 existing records from primary storage. [2025-10-20T22:08:43.142333] [INFO ] [MAIN ] F: Loaded 53820 primary data records. [2025-10-20T22:08:43.786570] [DEBUG] [MAIN ] EXP: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:43.788104] [INFO ] [MAIN ] EXP: Generated 1793 derivative records from 29414 primary records. [2025-10-20T22:08:43.788205] [DEBUG] [LOCAL ] EXP: Saving derivative data. [2025-10-20T22:08:43.807060] [DEBUG] [LOCAL ] EXP: Derivative data saved (1793 records). [2025-10-20T22:08:43.807382] [INFO ] [MAIN ] EXP: Successfully saved derivative data. [2025-10-20T22:08:43.809316] [INFO ] [MAIN ] Derivative Progress: 646/1877 symbols completed. EXP took 1.1s. Estimated remaining: 7m 44s. Rate: 159.1 symbols/min [2025-10-20T22:08:43.809923] [INFO ] [MAIN ] FAF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:43.848623] [DEBUG] [MAIN ] EXR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:43.848191] [DEBUG] [MAIN ] F: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:43.849797] [INFO ] [MAIN ] F: Generated 1794 derivative records from 53820 primary records. [2025-10-20T22:08:43.850970] [INFO ] [MAIN ] EXR: Generated 1794 derivative records from 45611 primary records. [2025-10-20T22:08:43.851199] [DEBUG] [LOCAL ] EXR: Saving derivative data. [2025-10-20T22:08:43.851394] [DEBUG] [LOCAL ] F: Saving derivative data. [2025-10-20T22:08:43.891884] [DEBUG] [LOCAL ] F: Derivative data saved (1794 records). [2025-10-20T22:08:43.894051] [INFO ] [MAIN ] F: Successfully saved derivative data. [2025-10-20T22:08:43.898459] [INFO ] [MAIN ] Derivative Progress: 647/1877 symbols completed. F took 1.2s. Estimated remaining: 7m 43s. Rate: 159.3 symbols/min [2025-10-20T22:08:43.898736] [INFO ] [MAIN ] FBIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:43.895521] [DEBUG] [LOCAL ] EXR: Derivative data saved (1794 records). [2025-10-20T22:08:43.899032] [INFO ] [MAIN ] EXR: Successfully saved derivative data. [2025-10-20T22:08:43.901593] [INFO ] [MAIN ] Derivative Progress: 648/1877 symbols completed. EXR took 1.2s. Estimated remaining: 7m 42s. Rate: 159.5 symbols/min [2025-10-20T22:08:43.901953] [INFO ] [MAIN ] FBK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:43.917237] [DEBUG] [LOCAL ] FAF: Loaded 41567 existing records from primary storage. [2025-10-20T22:08:43.917415] [INFO ] [MAIN ] FAF: Loaded 41567 primary data records. [2025-10-20T22:08:44.114232] [DEBUG] [LOCAL ] FBK: Loaded 27519 existing records from primary storage. [2025-10-20T22:08:44.130828] [INFO ] [MAIN ] FBK: Loaded 27519 primary data records. [2025-10-20T22:08:44.388474] [DEBUG] [LOCAL ] FBIN: Loaded 50414 existing records from primary storage. [2025-10-20T22:08:44.406623] [INFO ] [MAIN ] FBIN: Loaded 50414 primary data records. [2025-10-20T22:08:44.886336] [DEBUG] [MAIN ] FAF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:44.887983] [INFO ] [MAIN ] FAF: Generated 1794 derivative records from 41567 primary records. [2025-10-20T22:08:44.889004] [DEBUG] [LOCAL ] FAF: Saving derivative data. [2025-10-20T22:08:44.907267] [DEBUG] [LOCAL ] FAF: Derivative data saved (1794 records). [2025-10-20T22:08:44.907366] [INFO ] [MAIN ] FAF: Successfully saved derivative data. [2025-10-20T22:08:44.909471] [INFO ] [MAIN ] Derivative Progress: 649/1877 symbols completed. FAF took 1.1s. Estimated remaining: 7m 43s. Rate: 159.1 symbols/min [2025-10-20T22:08:44.909614] [INFO ] [MAIN ] FBP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:44.928592] [DEBUG] [MAIN ] FBK: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:44.929629] [INFO ] [MAIN ] FBK: Generated 1793 derivative records from 27519 primary records. [2025-10-20T22:08:44.930030] [DEBUG] [LOCAL ] FBK: Saving derivative data. [2025-10-20T22:08:44.946764] [DEBUG] [LOCAL ] FBK: Derivative data saved (1793 records). [2025-10-20T22:08:44.947010] [INFO ] [MAIN ] FBK: Successfully saved derivative data. [2025-10-20T22:08:44.949343] [INFO ] [MAIN ] Derivative Progress: 650/1877 symbols completed. FBK took 1.0s. Estimated remaining: 7m 42s. Rate: 159.3 symbols/min [2025-10-20T22:08:44.949705] [INFO ] [MAIN ] FC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:44.966032] [DEBUG] [MAIN ] FBIN: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:44.980266] [INFO ] [MAIN ] FBIN: Generated 1793 derivative records from 50414 primary records. [2025-10-20T22:08:44.980405] [DEBUG] [LOCAL ] FBIN: Saving derivative data. [2025-10-20T22:08:45.014451] [DEBUG] [LOCAL ] FBIN: Derivative data saved (1793 records). [2025-10-20T22:08:45.015639] [INFO ] [MAIN ] FBIN: Successfully saved derivative data. [2025-10-20T22:08:45.016488] [DEBUG] [LOCAL ] FC: Loaded 14647 existing records from primary storage. [2025-10-20T22:08:45.019478] [INFO ] [MAIN ] Derivative Progress: 651/1877 symbols completed. FBIN took 1.1s. Estimated remaining: 7m 41s. Rate: 159.5 symbols/min [2025-10-20T22:08:45.017188] [DEBUG] [LOCAL ] FBP: Loaded 45752 existing records from primary storage. [2025-10-20T22:08:45.019846] [INFO ] [MAIN ] FC: Loaded 14647 primary data records. [2025-10-20T22:08:45.020040] [INFO ] [MAIN ] FBP: Loaded 45752 primary data records. [2025-10-20T22:08:45.019760] [INFO ] [MAIN ] FCN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:45.681432] [DEBUG] [LOCAL ] FCN: Loaded 25974 existing records from primary storage. [2025-10-20T22:08:45.698790] [INFO ] [MAIN ] FCN: Loaded 25974 primary data records. [2025-10-20T22:08:46.037909] [DEBUG] [MAIN ] FBP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:46.051518] [INFO ] [MAIN ] FBP: Generated 1794 derivative records from 45752 primary records. [2025-10-20T22:08:46.052111] [DEBUG] [MAIN ] FC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:46.052274] [DEBUG] [LOCAL ] FBP: Saving derivative data. [2025-10-20T22:08:46.053039] [INFO ] [MAIN ] FC: Generated 1794 derivative records from 14647 primary records. [2025-10-20T22:08:46.054248] [DEBUG] [LOCAL ] FC: Saving derivative data. [2025-10-20T22:08:46.070162] [DEBUG] [LOCAL ] FBP: Derivative data saved (1794 records). [2025-10-20T22:08:46.070416] [INFO ] [MAIN ] FBP: Successfully saved derivative data. [2025-10-20T22:08:46.072426] [INFO ] [MAIN ] Derivative Progress: 652/1877 symbols completed. FBP took 1.2s. Estimated remaining: 7m 42s. Rate: 159.1 symbols/min [2025-10-20T22:08:46.087999] [DEBUG] [LOCAL ] FC: Derivative data saved (1794 records). [2025-10-20T22:08:46.081177] [INFO ] [MAIN ] FCPT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:46.088610] [INFO ] [MAIN ] FC: Successfully saved derivative data. [2025-10-20T22:08:46.090695] [INFO ] [MAIN ] Derivative Progress: 653/1877 symbols completed. FC took 1.1s. Estimated remaining: 7m 41s. Rate: 159.3 symbols/min [2025-10-20T22:08:46.091104] [INFO ] [MAIN ] FCT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:46.113781] [DEBUG] [MAIN ] FCN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:46.114740] [INFO ] [MAIN ] FCN: Generated 1794 derivative records from 25974 primary records. [2025-10-20T22:08:46.114860] [DEBUG] [LOCAL ] FCN: Saving derivative data. [2025-10-20T22:08:46.133944] [DEBUG] [LOCAL ] FCT: Loaded 14667 existing records from primary storage. [2025-10-20T22:08:46.137822] [INFO ] [MAIN ] FCT: Loaded 14667 primary data records. [2025-10-20T22:08:46.137753] [DEBUG] [LOCAL ] FCN: Derivative data saved (1794 records). [2025-10-20T22:08:46.170712] [INFO ] [MAIN ] FCN: Successfully saved derivative data. [2025-10-20T22:08:46.185091] [INFO ] [MAIN ] Derivative Progress: 654/1877 symbols completed. FCN took 1.2s. Estimated remaining: 7m 40s. Rate: 159.5 symbols/min [2025-10-20T22:08:46.207009] [INFO ] [MAIN ] FCX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:46.294141] [DEBUG] [LOCAL ] FCPT: Loaded 37210 existing records from primary storage. [2025-10-20T22:08:46.315152] [INFO ] [MAIN ] FCPT: Loaded 37210 primary data records. [2025-10-20T22:08:46.798080] [DEBUG] [LOCAL ] FCX: Loaded 53701 existing records from primary storage. [2025-10-20T22:08:46.798448] [INFO ] [MAIN ] FCX: Loaded 53701 primary data records. [2025-10-20T22:08:46.931823] [DEBUG] [MAIN ] FCT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:46.963674] [INFO ] [MAIN ] FCT: Generated 1794 derivative records from 14667 primary records. [2025-10-20T22:08:46.988685] [DEBUG] [LOCAL ] FCT: Saving derivative data. [2025-10-20T22:08:47.137387] [DEBUG] [LOCAL ] FCT: Derivative data saved (1794 records). [2025-10-20T22:08:47.138399] [INFO ] [MAIN ] FCT: Successfully saved derivative data. [2025-10-20T22:08:47.138091] [DEBUG] [MAIN ] FCPT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:47.140706] [INFO ] [MAIN ] Derivative Progress: 655/1877 symbols completed. FCT took 1.0s. Estimated remaining: 7m 41s. Rate: 159.1 symbols/min [2025-10-20T22:08:47.141897] [INFO ] [MAIN ] FCPT: Generated 1794 derivative records from 37210 primary records. [2025-10-20T22:08:47.141009] [INFO ] [MAIN ] FDP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:47.142847] [DEBUG] [LOCAL ] FCPT: Saving derivative data. [2025-10-20T22:08:47.161421] [DEBUG] [LOCAL ] FCPT: Derivative data saved (1794 records). [2025-10-20T22:08:47.161772] [INFO ] [MAIN ] FCPT: Successfully saved derivative data. [2025-10-20T22:08:47.164197] [INFO ] [MAIN ] Derivative Progress: 656/1877 symbols completed. FCPT took 1.1s. Estimated remaining: 7m 40s. Rate: 159.4 symbols/min [2025-10-20T22:08:47.164545] [INFO ] [MAIN ] FDS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:47.184043] [DEBUG] [MAIN ] FCX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:47.184765] [INFO ] [MAIN ] FCX: Generated 1794 derivative records from 53701 primary records. [2025-10-20T22:08:47.192847] [DEBUG] [LOCAL ] FCX: Saving derivative data. [2025-10-20T22:08:47.223308] [DEBUG] [LOCAL ] FCX: Derivative data saved (1794 records). [2025-10-20T22:08:47.224937] [INFO ] [MAIN ] FCX: Successfully saved derivative data. [2025-10-20T22:08:47.227138] [INFO ] [MAIN ] Derivative Progress: 657/1877 symbols completed. FCX took 1.0s. Estimated remaining: 7m 39s. Rate: 159.6 symbols/min [2025-10-20T22:08:47.227436] [INFO ] [MAIN ] FDX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:47.237806] [DEBUG] [LOCAL ] FDS: Loaded 27869 existing records from primary storage. [2025-10-20T22:08:47.238067] [INFO ] [MAIN ] FDS: Loaded 27869 primary data records. [2025-10-20T22:08:47.250639] [DEBUG] [LOCAL ] FDP: Loaded 29728 existing records from primary storage. [2025-10-20T22:08:47.276413] [INFO ] [MAIN ] FDP: Loaded 29728 primary data records. [2025-10-20T22:08:47.911381] [DEBUG] [LOCAL ] FDX: Loaded 51326 existing records from primary storage. [2025-10-20T22:08:47.911671] [INFO ] [MAIN ] FDX: Loaded 51326 primary data records. [2025-10-20T22:08:48.248525] [DEBUG] [MAIN ] FDS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:48.249917] [INFO ] [MAIN ] FDS: Generated 1794 derivative records from 27869 primary records. [2025-10-20T22:08:48.262304] [DEBUG] [LOCAL ] FDS: Saving derivative data. [2025-10-20T22:08:48.264185] [DEBUG] [MAIN ] FDP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:48.273481] [INFO ] [MAIN ] FDP: Generated 1794 derivative records from 29728 primary records. [2025-10-20T22:08:48.273669] [DEBUG] [LOCAL ] FDP: Saving derivative data. [2025-10-20T22:08:48.292995] [DEBUG] [LOCAL ] FDS: Derivative data saved (1794 records). [2025-10-20T22:08:48.297247] [INFO ] [MAIN ] FDS: Successfully saved derivative data. [2025-10-20T22:08:48.299264] [INFO ] [MAIN ] Derivative Progress: 658/1877 symbols completed. FDS took 1.1s. Estimated remaining: 7m 40s. Rate: 159.1 symbols/min [2025-10-20T22:08:48.299577] [DEBUG] [LOCAL ] FDP: Derivative data saved (1794 records). [2025-10-20T22:08:48.300006] [INFO ] [MAIN ] FDP: Successfully saved derivative data. [2025-10-20T22:08:48.302207] [INFO ] [MAIN ] Derivative Progress: 659/1877 symbols completed. FDP took 1.2s. Estimated remaining: 7m 39s. Rate: 159.4 symbols/min [2025-10-20T22:08:48.322342] [DEBUG] [MAIN ] FDX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:48.322928] [INFO ] [MAIN ] FDX: Generated 1794 derivative records from 51326 primary records. [2025-10-20T22:08:48.322987] [DEBUG] [LOCAL ] FDX: Saving derivative data. [2025-10-20T22:08:48.338418] [DEBUG] [LOCAL ] FDX: Derivative data saved (1794 records). [2025-10-20T22:08:48.338493] [INFO ] [MAIN ] FDX: Successfully saved derivative data. [2025-10-20T22:08:48.340447] [INFO ] [MAIN ] Derivative Progress: 660/1877 symbols completed. FDX took 1.1s. Estimated remaining: 7m 38s. Rate: 159.6 symbols/min [2025-10-20T22:08:48.340921] [INFO ] [MAIN ] FERG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:48.340865] [INFO ] [MAIN ] FE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:48.341190] [INFO ] [MAIN ] FET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:48.360290] [DEBUG] [LOCAL ] FET: Loaded 12965 existing records from primary storage. [2025-10-20T22:08:48.360583] [INFO ] [MAIN ] FET: Loaded 12965 primary data records. [2025-10-20T22:08:48.566847] [DEBUG] [LOCAL ] FERG: Loaded 48769 existing records from primary storage. [2025-10-20T22:08:48.583841] [INFO ] [MAIN ] FERG: Loaded 48769 primary data records. [2025-10-20T22:08:48.770518] [DEBUG] [LOCAL ] FE: Loaded 53436 existing records from primary storage. [2025-10-20T22:08:48.800699] [INFO ] [MAIN ] FE: Loaded 53436 primary data records. [2025-10-20T22:08:49.363888] [DEBUG] [MAIN ] FET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:49.364992] [INFO ] [MAIN ] FET: Generated 1794 derivative records from 12965 primary records. [2025-10-20T22:08:49.365447] [DEBUG] [LOCAL ] FET: Saving derivative data. [2025-10-20T22:08:49.384447] [DEBUG] [LOCAL ] FET: Derivative data saved (1794 records). [2025-10-20T22:08:49.385030] [INFO ] [MAIN ] FET: Successfully saved derivative data. [2025-10-20T22:08:49.387227] [INFO ] [MAIN ] Derivative Progress: 661/1877 symbols completed. FET took 1.0s. Estimated remaining: 7m 38s. Rate: 159.2 symbols/min [2025-10-20T22:08:49.387417] [INFO ] [MAIN ] FF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:49.407434] [DEBUG] [MAIN ] FERG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:49.410003] [INFO ] [MAIN ] FERG: Generated 1794 derivative records from 48769 primary records. [2025-10-20T22:08:49.416120] [DEBUG] [LOCAL ] FERG: Saving derivative data. [2025-10-20T22:08:49.453246] [DEBUG] [LOCAL ] FF: Loaded 24052 existing records from primary storage. [2025-10-20T22:08:49.459006] [DEBUG] [LOCAL ] FERG: Derivative data saved (1794 records). [2025-10-20T22:08:49.459581] [INFO ] [MAIN ] FERG: Successfully saved derivative data. [2025-10-20T22:08:49.462322] [INFO ] [MAIN ] Derivative Progress: 662/1877 symbols completed. FERG took 1.1s. Estimated remaining: 7m 37s. Rate: 159.3 symbols/min [2025-10-20T22:08:49.462702] [DEBUG] [MAIN ] FE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:49.463618] [INFO ] [MAIN ] FE: Generated 1794 derivative records from 53436 primary records. [2025-10-20T22:08:49.459506] [INFO ] [MAIN ] FF: Loaded 24052 primary data records. [2025-10-20T22:08:49.463833] [DEBUG] [LOCAL ] FE: Saving derivative data. [2025-10-20T22:08:49.469624] [INFO ] [MAIN ] FFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:49.529462] [DEBUG] [LOCAL ] FE: Derivative data saved (1794 records). [2025-10-20T22:08:49.534944] [INFO ] [MAIN ] FE: Successfully saved derivative data. [2025-10-20T22:08:49.544576] [INFO ] [MAIN ] Derivative Progress: 663/1877 symbols completed. FE took 1.2s. Estimated remaining: 7m 37s. Rate: 159.5 symbols/min [2025-10-20T22:08:49.554703] [INFO ] [MAIN ] FFWM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:49.709231] [DEBUG] [LOCAL ] FFC: Loaded 16622 existing records from primary storage. [2025-10-20T22:08:49.851135] [INFO ] [MAIN ] FFC: Loaded 16622 primary data records. [2025-10-20T22:08:49.969493] [DEBUG] [LOCAL ] FFWM: Loaded 35833 existing records from primary storage. [2025-10-20T22:08:49.981467] [INFO ] [MAIN ] FFWM: Loaded 35833 primary data records. [2025-10-20T22:08:50.502085] [DEBUG] [MAIN ] FF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:50.504632] [INFO ] [MAIN ] FF: Generated 1794 derivative records from 24052 primary records. [2025-10-20T22:08:50.505260] [DEBUG] [LOCAL ] FF: Saving derivative data. [2025-10-20T22:08:50.524049] [DEBUG] [LOCAL ] FF: Derivative data saved (1794 records). [2025-10-20T22:08:50.524180] [INFO ] [MAIN ] FF: Successfully saved derivative data. [2025-10-20T22:08:50.527286] [INFO ] [MAIN ] Derivative Progress: 664/1877 symbols completed. FF took 1.1s. Estimated remaining: 7m 37s. Rate: 159.1 symbols/min [2025-10-20T22:08:50.527520] [INFO ] [MAIN ] FG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:50.555498] [DEBUG] [MAIN ] FFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:50.556560] [INFO ] [MAIN ] FFC: Generated 1794 derivative records from 16622 primary records. [2025-10-20T22:08:50.556782] [DEBUG] [LOCAL ] FFC: Saving derivative data. [2025-10-20T22:08:50.576173] [DEBUG] [LOCAL ] FFC: Derivative data saved (1794 records). [2025-10-20T22:08:50.576414] [INFO ] [MAIN ] FFC: Successfully saved derivative data. [2025-10-20T22:08:50.578553] [INFO ] [MAIN ] Derivative Progress: 665/1877 symbols completed. FFC took 1.1s. Estimated remaining: 7m 36s. Rate: 159.4 symbols/min [2025-10-20T22:08:50.578854] [INFO ] [MAIN ] FGN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:50.600960] [DEBUG] [LOCAL ] FG: Loaded 21737 existing records from primary storage. [2025-10-20T22:08:50.601467] [INFO ] [MAIN ] FG: Loaded 21737 primary data records. [2025-10-20T22:08:50.601371] [DEBUG] [MAIN ] FFWM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:50.633991] [INFO ] [MAIN ] FFWM: Generated 1794 derivative records from 35833 primary records. [2025-10-20T22:08:50.674029] [DEBUG] [LOCAL ] FGN: Loaded 3636 existing records from primary storage. [2025-10-20T22:08:50.674100] [DEBUG] [LOCAL ] FFWM: Saving derivative data. [2025-10-20T22:08:50.696147] [INFO ] [MAIN ] FGN: Loaded 3636 primary data records. [2025-10-20T22:08:50.800254] [DEBUG] [LOCAL ] FFWM: Derivative data saved (1794 records). [2025-10-20T22:08:50.825724] [INFO ] [MAIN ] FFWM: Successfully saved derivative data. [2025-10-20T22:08:50.850870] [INFO ] [MAIN ] Derivative Progress: 666/1877 symbols completed. FFWM took 1.3s. Estimated remaining: 7m 36s. Rate: 159.4 symbols/min [2025-10-20T22:08:50.906723] [INFO ] [MAIN ] FHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:51.300879] [DEBUG] [LOCAL ] FHI: Loaded 39793 existing records from primary storage. [2025-10-20T22:08:51.301180] [INFO ] [MAIN ] FHI: Loaded 39793 primary data records. [2025-10-20T22:08:51.301529] [DEBUG] [MAIN ] FG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:51.302678] [INFO ] [MAIN ] FG: Generated 1794 derivative records from 21737 primary records. [2025-10-20T22:08:51.316186] [DEBUG] [LOCAL ] FG: Saving derivative data. [2025-10-20T22:08:51.419250] [DEBUG] [LOCAL ] FG: Derivative data saved (1794 records). [2025-10-20T22:08:51.411436] [DEBUG] [MAIN ] FGN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:51.425398] [INFO ] [MAIN ] FG: Successfully saved derivative data. [2025-10-20T22:08:51.428322] [INFO ] [MAIN ] Derivative Progress: 667/1877 symbols completed. FG took 0.9s. Estimated remaining: 7m 36s. Rate: 159.3 symbols/min [2025-10-20T22:08:51.426187] [INFO ] [MAIN ] FGN: Generated 1794 derivative records from 3636 primary records. [2025-10-20T22:08:51.439562] [INFO ] [MAIN ] FHN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:51.439855] [DEBUG] [LOCAL ] FGN: Saving derivative data. [2025-10-20T22:08:51.530022] [DEBUG] [LOCAL ] FGN: Derivative data saved (1794 records). [2025-10-20T22:08:51.536976] [INFO ] [MAIN ] FGN: Successfully saved derivative data. [2025-10-20T22:08:51.539368] [INFO ] [MAIN ] Derivative Progress: 668/1877 symbols completed. FGN took 1.0s. Estimated remaining: 7m 35s. Rate: 159.5 symbols/min [2025-10-20T22:08:51.539515] [INFO ] [MAIN ] FI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:51.746482] [DEBUG] [LOCAL ] FHN: Loaded 53394 existing records from primary storage. [2025-10-20T22:08:51.747167] [INFO ] [MAIN ] FHN: Loaded 53394 primary data records. [2025-10-20T22:08:51.870828] [DEBUG] [LOCAL ] FI: Loaded 53585 existing records from primary storage. [2025-10-20T22:08:51.899301] [INFO ] [MAIN ] FI: Loaded 53585 primary data records. [2025-10-20T22:08:51.910598] [DEBUG] [MAIN ] FHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:51.939210] [INFO ] [MAIN ] FHI: Generated 1794 derivative records from 39793 primary records. [2025-10-20T22:08:51.958423] [DEBUG] [LOCAL ] FHI: Saving derivative data. [2025-10-20T22:08:52.219978] [DEBUG] [LOCAL ] FHI: Derivative data saved (1794 records). [2025-10-20T22:08:52.231541] [INFO ] [MAIN ] FHI: Successfully saved derivative data. [2025-10-20T22:08:52.234775] [INFO ] [MAIN ] Derivative Progress: 669/1877 symbols completed. FHI took 1.3s. Estimated remaining: 7m 35s. Rate: 159.3 symbols/min [2025-10-20T22:08:52.234961] [INFO ] [MAIN ] FICO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:52.464522] [DEBUG] [MAIN ] FHN: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:52.466570] [INFO ] [MAIN ] FHN: Generated 1793 derivative records from 53394 primary records. [2025-10-20T22:08:52.465304] [DEBUG] [LOCAL ] FICO: Loaded 21862 existing records from primary storage. [2025-10-20T22:08:52.466840] [DEBUG] [LOCAL ] FHN: Saving derivative data. [2025-10-20T22:08:52.467317] [INFO ] [MAIN ] FICO: Loaded 21862 primary data records. [2025-10-20T22:08:52.565107] [DEBUG] [LOCAL ] FHN: Derivative data saved (1793 records). [2025-10-20T22:08:52.579192] [INFO ] [MAIN ] FHN: Successfully saved derivative data. [2025-10-20T22:08:52.586428] [INFO ] [MAIN ] Derivative Progress: 670/1877 symbols completed. FHN took 1.1s. Estimated remaining: 7m 35s. Rate: 159.3 symbols/min [2025-10-20T22:08:52.607032] [INFO ] [MAIN ] FIGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:52.835608] [DEBUG] [MAIN ] FI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:52.837824] [INFO ] [MAIN ] FI: Generated 1793 derivative records from 53585 primary records. [2025-10-20T22:08:52.838047] [DEBUG] [LOCAL ] FI: Saving derivative data. [2025-10-20T22:08:52.859921] [DEBUG] [LOCAL ] FI: Derivative data saved (1793 records). [2025-10-20T22:08:52.860672] [INFO ] [MAIN ] FI: Successfully saved derivative data. [2025-10-20T22:08:52.863429] [INFO ] [MAIN ] Derivative Progress: 671/1877 symbols completed. FI took 1.3s. Estimated remaining: 7m 34s. Rate: 159.3 symbols/min [2025-10-20T22:08:52.864303] [INFO ] [MAIN ] FIHL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:52.907751] [DEBUG] [LOCAL ] FIGS: Loaded 48620 existing records from primary storage. [2025-10-20T22:08:52.908296] [INFO ] [MAIN ] FIGS: Loaded 48620 primary data records. [2025-10-20T22:08:52.971038] [DEBUG] [MAIN ] FICO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:53.005854] [INFO ] [MAIN ] FICO: Generated 1794 derivative records from 21862 primary records. [2025-10-20T22:08:53.038881] [DEBUG] [LOCAL ] FICO: Saving derivative data. [2025-10-20T22:08:53.052759] [DEBUG] [LOCAL ] FIHL: Loaded 39199 existing records from primary storage. [2025-10-20T22:08:53.053268] [INFO ] [MAIN ] FIHL: Loaded 39199 primary data records. [2025-10-20T22:08:53.236682] [DEBUG] [LOCAL ] FICO: Derivative data saved (1794 records). [2025-10-20T22:08:53.260949] [INFO ] [MAIN ] FICO: Successfully saved derivative data. [2025-10-20T22:08:53.387797] [INFO ] [MAIN ] Derivative Progress: 672/1877 symbols completed. FICO took 1.2s. Estimated remaining: 7m 34s. Rate: 159.2 symbols/min [2025-10-20T22:08:53.404347] [INFO ] [MAIN ] FINS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:53.580085] [DEBUG] [LOCAL ] FINS: Loaded 8283 existing records from primary storage. [2025-10-20T22:08:53.580247] [INFO ] [MAIN ] FINS: Loaded 8283 primary data records. [2025-10-20T22:08:53.942480] [DEBUG] [MAIN ] FIGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:53.943399] [INFO ] [MAIN ] FIGS: Generated 1794 derivative records from 48620 primary records. [2025-10-20T22:08:53.943608] [DEBUG] [LOCAL ] FIGS: Saving derivative data. [2025-10-20T22:08:53.965094] [DEBUG] [LOCAL ] FIGS: Derivative data saved (1794 records). [2025-10-20T22:08:53.977419] [INFO ] [MAIN ] FIGS: Successfully saved derivative data. [2025-10-20T22:08:53.979482] [INFO ] [MAIN ] Derivative Progress: 673/1877 symbols completed. FIGS took 1.4s. Estimated remaining: 7m 34s. Rate: 159.1 symbols/min [2025-10-20T22:08:53.980920] [DEBUG] [MAIN ] FIHL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:53.981747] [INFO ] [MAIN ] FIHL: Generated 1794 derivative records from 39199 primary records. [2025-10-20T22:08:53.981827] [DEBUG] [LOCAL ] FIHL: Saving derivative data. [2025-10-20T22:08:53.998890] [DEBUG] [LOCAL ] FIHL: Derivative data saved (1794 records). [2025-10-20T22:08:53.999090] [INFO ] [MAIN ] FIHL: Successfully saved derivative data. [2025-10-20T22:08:54.001349] [INFO ] [MAIN ] Derivative Progress: 674/1877 symbols completed. FIHL took 1.1s. Estimated remaining: 7m 33s. Rate: 159.3 symbols/min [2025-10-20T22:08:54.018584] [DEBUG] [MAIN ] FINS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:54.019230] [INFO ] [MAIN ] FINS: Generated 1794 derivative records from 8283 primary records. [2025-10-20T22:08:54.019292] [DEBUG] [LOCAL ] FINS: Saving derivative data. [2025-10-20T22:08:54.034673] [DEBUG] [LOCAL ] FINS: Derivative data saved (1794 records). [2025-10-20T22:08:54.034758] [INFO ] [MAIN ] FINS: Successfully saved derivative data. [2025-10-20T22:08:54.037015] [INFO ] [MAIN ] Derivative Progress: 675/1877 symbols completed. FINS took 0.6s. Estimated remaining: 7m 32s. Rate: 159.5 symbols/min [2025-10-20T22:08:54.037599] [INFO ] [MAIN ] FIX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:54.037413] [INFO ] [MAIN ] FINV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:54.037486] [INFO ] [MAIN ] FIS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:54.076177] [DEBUG] [LOCAL ] FIX: Loaded 28617 existing records from primary storage. [2025-10-20T22:08:54.076409] [INFO ] [MAIN ] FIX: Loaded 28617 primary data records. [2025-10-20T22:08:54.217077] [DEBUG] [LOCAL ] FINV: Loaded 45271 existing records from primary storage. [2025-10-20T22:08:54.223012] [INFO ] [MAIN ] FINV: Loaded 45271 primary data records. [2025-10-20T22:08:54.322067] [DEBUG] [LOCAL ] FIS: Loaded 53327 existing records from primary storage. [2025-10-20T22:08:54.381815] [INFO ] [MAIN ] FIS: Loaded 53327 primary data records. [2025-10-20T22:08:55.109173] [DEBUG] [MAIN ] FIX: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:08:55.110780] [INFO ] [MAIN ] FIX: Generated 1793 derivative records from 28617 primary records. [2025-10-20T22:08:55.111936] [DEBUG] [LOCAL ] FIX: Saving derivative data. [2025-10-20T22:08:55.130359] [DEBUG] [LOCAL ] FIX: Derivative data saved (1793 records). [2025-10-20T22:08:55.130817] [INFO ] [MAIN ] FIX: Successfully saved derivative data. [2025-10-20T22:08:55.132902] [INFO ] [MAIN ] Derivative Progress: 676/1877 symbols completed. FIX took 1.1s. Estimated remaining: 7m 33s. Rate: 159.1 symbols/min [2025-10-20T22:08:55.133264] [INFO ] [MAIN ] FLC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:55.145796] [DEBUG] [LOCAL ] FLC: Loaded 5417 existing records from primary storage. [2025-10-20T22:08:55.170181] [INFO ] [MAIN ] FLC: Loaded 5417 primary data records. [2025-10-20T22:08:55.175914] [DEBUG] [MAIN ] FINV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:55.183683] [INFO ] [MAIN ] FINV: Generated 1794 derivative records from 45271 primary records. [2025-10-20T22:08:55.176345] [DEBUG] [MAIN ] FIS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:55.195465] [INFO ] [MAIN ] FIS: Generated 1794 derivative records from 53327 primary records. [2025-10-20T22:08:55.195581] [DEBUG] [LOCAL ] FIS: Saving derivative data. [2025-10-20T22:08:55.188920] [DEBUG] [LOCAL ] FINV: Saving derivative data. [2025-10-20T22:08:55.286530] [DEBUG] [LOCAL ] FIS: Derivative data saved (1794 records). [2025-10-20T22:08:55.298573] [INFO ] [MAIN ] FIS: Successfully saved derivative data. [2025-10-20T22:08:55.310530] [INFO ] [MAIN ] Derivative Progress: 677/1877 symbols completed. FIS took 1.3s. Estimated remaining: 7m 32s. Rate: 159.2 symbols/min [2025-10-20T22:08:55.310705] [INFO ] [MAIN ] FLG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:55.312184] [DEBUG] [LOCAL ] FINV: Derivative data saved (1794 records). [2025-10-20T22:08:55.316132] [INFO ] [MAIN ] FINV: Successfully saved derivative data. [2025-10-20T22:08:55.318203] [INFO ] [MAIN ] Derivative Progress: 678/1877 symbols completed. FINV took 1.3s. Estimated remaining: 7m 31s. Rate: 159.5 symbols/min [2025-10-20T22:08:55.318804] [INFO ] [MAIN ] FLNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:55.555945] [DEBUG] [LOCAL ] FLNG: Loaded 33591 existing records from primary storage. [2025-10-20T22:08:55.556491] [INFO ] [MAIN ] FLNG: Loaded 33591 primary data records. [2025-10-20T22:08:55.686394] [DEBUG] [LOCAL ] FLG: Loaded 53322 existing records from primary storage. [2025-10-20T22:08:55.704156] [INFO ] [MAIN ] FLG: Loaded 53322 primary data records. [2025-10-20T22:08:55.726590] [DEBUG] [MAIN ] FLC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:55.798793] [INFO ] [MAIN ] FLC: Generated 1794 derivative records from 5417 primary records. [2025-10-20T22:08:55.842507] [DEBUG] [LOCAL ] FLC: Saving derivative data. [2025-10-20T22:08:56.082213] [DEBUG] [LOCAL ] FLC: Derivative data saved (1794 records). [2025-10-20T22:08:56.082357] [INFO ] [MAIN ] FLC: Successfully saved derivative data. [2025-10-20T22:08:56.116040] [INFO ] [MAIN ] Derivative Progress: 679/1877 symbols completed. FLC took 1.0s. Estimated remaining: 7m 32s. Rate: 159.2 symbols/min [2025-10-20T22:08:56.250840] [INFO ] [MAIN ] FLO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:56.276090] [DEBUG] [MAIN ] FLNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:56.276997] [INFO ] [MAIN ] FLNG: Generated 1794 derivative records from 33591 primary records. [2025-10-20T22:08:56.277477] [DEBUG] [LOCAL ] FLNG: Saving derivative data. [2025-10-20T22:08:56.293467] [DEBUG] [LOCAL ] FLNG: Derivative data saved (1794 records). [2025-10-20T22:08:56.293873] [INFO ] [MAIN ] FLNG: Successfully saved derivative data. [2025-10-20T22:08:56.297067] [INFO ] [MAIN ] Derivative Progress: 680/1877 symbols completed. FLNG took 1.0s. Estimated remaining: 7m 31s. Rate: 159.3 symbols/min [2025-10-20T22:08:56.312269] [INFO ] [MAIN ] FLR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:56.326530] [DEBUG] [MAIN ] FLG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:56.327282] [INFO ] [MAIN ] FLG: Generated 1794 derivative records from 53322 primary records. [2025-10-20T22:08:56.327390] [DEBUG] [LOCAL ] FLG: Saving derivative data. [2025-10-20T22:08:56.343844] [DEBUG] [LOCAL ] FLG: Derivative data saved (1794 records). [2025-10-20T22:08:56.344175] [INFO ] [MAIN ] FLG: Successfully saved derivative data. [2025-10-20T22:08:56.361203] [INFO ] [MAIN ] Derivative Progress: 681/1877 symbols completed. FLG took 1.1s. Estimated remaining: 7m 30s. Rate: 159.5 symbols/min [2025-10-20T22:08:56.361535] [INFO ] [MAIN ] FLS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:56.363128] [DEBUG] [LOCAL ] FLO: Loaded 52083 existing records from primary storage. [2025-10-20T22:08:56.363597] [INFO ] [MAIN ] FLO: Loaded 52083 primary data records. [2025-10-20T22:08:56.696846] [DEBUG] [LOCAL ] FLR: Loaded 53258 existing records from primary storage. [2025-10-20T22:08:56.697769] [INFO ] [MAIN ] FLR: Loaded 53258 primary data records. [2025-10-20T22:08:56.822092] [DEBUG] [LOCAL ] FLS: Loaded 50452 existing records from primary storage. [2025-10-20T22:08:56.823501] [INFO ] [MAIN ] FLS: Loaded 50452 primary data records. [2025-10-20T22:08:57.342527] [DEBUG] [MAIN ] FLO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:57.344735] [INFO ] [MAIN ] FLO: Generated 1794 derivative records from 52083 primary records. [2025-10-20T22:08:57.345463] [DEBUG] [LOCAL ] FLO: Saving derivative data. [2025-10-20T22:08:57.364066] [DEBUG] [LOCAL ] FLO: Derivative data saved (1794 records). [2025-10-20T22:08:57.364151] [INFO ] [MAIN ] FLO: Successfully saved derivative data. [2025-10-20T22:08:57.366382] [INFO ] [MAIN ] Derivative Progress: 682/1877 symbols completed. FLO took 1.1s. Estimated remaining: 7m 31s. Rate: 159.1 symbols/min [2025-10-20T22:08:57.366773] [INFO ] [MAIN ] FLUT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:57.385115] [DEBUG] [MAIN ] FLR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:57.385837] [INFO ] [MAIN ] FLR: Generated 1794 derivative records from 53258 primary records. [2025-10-20T22:08:57.386042] [DEBUG] [LOCAL ] FLR: Saving derivative data. [2025-10-20T22:08:57.403434] [DEBUG] [LOCAL ] FLR: Derivative data saved (1794 records). [2025-10-20T22:08:57.404289] [INFO ] [MAIN ] FLR: Successfully saved derivative data. [2025-10-20T22:08:57.406769] [INFO ] [MAIN ] Derivative Progress: 683/1877 symbols completed. FLR took 1.1s. Estimated remaining: 7m 30s. Rate: 159.3 symbols/min [2025-10-20T22:08:57.420376] [INFO ] [MAIN ] FLXR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:57.433752] [DEBUG] [MAIN ] FLS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:57.434386] [INFO ] [MAIN ] FLS: Generated 1794 derivative records from 50452 primary records. [2025-10-20T22:08:57.434494] [DEBUG] [LOCAL ] FLS: Saving derivative data. [2025-10-20T22:08:57.449790] [DEBUG] [LOCAL ] FLS: Derivative data saved (1794 records). [2025-10-20T22:08:57.449878] [INFO ] [MAIN ] FLS: Successfully saved derivative data. [2025-10-20T22:08:57.452006] [INFO ] [MAIN ] Derivative Progress: 684/1877 symbols completed. FLS took 1.1s. Estimated remaining: 7m 29s. Rate: 159.5 symbols/min [2025-10-20T22:08:57.452801] [INFO ] [MAIN ] FMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:57.473211] [DEBUG] [LOCAL ] FLUT: Loaded 50697 existing records from primary storage. [2025-10-20T22:08:57.473490] [INFO ] [MAIN ] FLUT: Loaded 50697 primary data records. [2025-10-20T22:08:57.560930] [DEBUG] [LOCAL ] FLXR: Loaded 21242 existing records from primary storage. [2025-10-20T22:08:57.578995] [INFO ] [MAIN ] FLXR: Loaded 21242 primary data records. [2025-10-20T22:08:58.123864] [DEBUG] [LOCAL ] FMC: Loaded 51892 existing records from primary storage. [2025-10-20T22:08:58.124235] [INFO ] [MAIN ] FMC: Loaded 51892 primary data records. [2025-10-20T22:08:58.448764] [DEBUG] [MAIN ] FLUT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:58.452280] [INFO ] [MAIN ] FLUT: Generated 1794 derivative records from 50697 primary records. [2025-10-20T22:08:58.465085] [DEBUG] [LOCAL ] FLUT: Saving derivative data. [2025-10-20T22:08:58.465388] [DEBUG] [MAIN ] FLXR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:58.466764] [INFO ] [MAIN ] FLXR: Generated 1794 derivative records from 21242 primary records. [2025-10-20T22:08:58.467392] [DEBUG] [LOCAL ] FLXR: Saving derivative data. [2025-10-20T22:08:58.497146] [DEBUG] [LOCAL ] FLUT: Derivative data saved (1794 records). [2025-10-20T22:08:58.499128] [DEBUG] [LOCAL ] FLXR: Derivative data saved (1794 records). [2025-10-20T22:08:58.499467] [INFO ] [MAIN ] FLUT: Successfully saved derivative data. [2025-10-20T22:08:58.499541] [INFO ] [MAIN ] FLXR: Successfully saved derivative data. [2025-10-20T22:08:58.501583] [INFO ] [MAIN ] Derivative Progress: 685/1877 symbols completed. FLUT took 1.1s. Estimated remaining: 7m 29s. Rate: 159.1 symbols/min [2025-10-20T22:08:58.503849] [INFO ] [MAIN ] Derivative Progress: 686/1877 symbols completed. FLXR took 1.1s. Estimated remaining: 7m 28s. Rate: 159.3 symbols/min [2025-10-20T22:08:58.503985] [INFO ] [MAIN ] FMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:58.503794] [INFO ] [MAIN ] FMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:58.526384] [DEBUG] [LOCAL ] FMN: Loaded 5517 existing records from primary storage. [2025-10-20T22:08:58.527029] [INFO ] [MAIN ] FMN: Loaded 5517 primary data records. [2025-10-20T22:08:58.526956] [DEBUG] [MAIN ] FMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:58.539430] [INFO ] [MAIN ] FMC: Generated 1794 derivative records from 51892 primary records. [2025-10-20T22:08:58.572104] [DEBUG] [LOCAL ] FMC: Saving derivative data. [2025-10-20T22:08:58.663006] [DEBUG] [LOCAL ] FMC: Derivative data saved (1794 records). [2025-10-20T22:08:58.678966] [INFO ] [MAIN ] FMC: Successfully saved derivative data. [2025-10-20T22:08:58.707061] [INFO ] [MAIN ] Derivative Progress: 687/1877 symbols completed. FMC took 1.3s. Estimated remaining: 7m 28s. Rate: 159.5 symbols/min [2025-10-20T22:08:58.691388] [DEBUG] [LOCAL ] FMS: Loaded 30419 existing records from primary storage. [2025-10-20T22:08:58.744339] [INFO ] [MAIN ] FMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:58.857098] [INFO ] [MAIN ] FMS: Loaded 30419 primary data records. [2025-10-20T22:08:59.192261] [DEBUG] [LOCAL ] FMX: Loaded 40649 existing records from primary storage. [2025-10-20T22:08:59.192636] [INFO ] [MAIN ] FMX: Loaded 40649 primary data records. [2025-10-20T22:08:59.315080] [DEBUG] [MAIN ] FMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:59.337217] [INFO ] [MAIN ] FMN: Generated 1794 derivative records from 5517 primary records. [2025-10-20T22:08:59.342581] [DEBUG] [LOCAL ] FMN: Saving derivative data. [2025-10-20T22:08:59.528025] [DEBUG] [LOCAL ] FMN: Derivative data saved (1794 records). [2025-10-20T22:08:59.528239] [INFO ] [MAIN ] FMN: Successfully saved derivative data. [2025-10-20T22:08:59.531103] [INFO ] [MAIN ] Derivative Progress: 688/1877 symbols completed. FMN took 1.0s. Estimated remaining: 7m 28s. Rate: 159.2 symbols/min [2025-10-20T22:08:59.547999] [DEBUG] [MAIN ] FMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:59.548863] [INFO ] [MAIN ] FMS: Generated 1794 derivative records from 30419 primary records. [2025-10-20T22:08:59.548944] [DEBUG] [LOCAL ] FMS: Saving derivative data. [2025-10-20T22:08:59.566172] [DEBUG] [LOCAL ] FMS: Derivative data saved (1794 records). [2025-10-20T22:08:59.566299] [INFO ] [MAIN ] FMS: Successfully saved derivative data. [2025-10-20T22:08:59.568347] [INFO ] [MAIN ] Derivative Progress: 689/1877 symbols completed. FMS took 1.1s. Estimated remaining: 7m 27s. Rate: 159.4 symbols/min [2025-10-20T22:08:59.585347] [DEBUG] [MAIN ] FMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:08:59.586029] [INFO ] [MAIN ] FMX: Generated 1794 derivative records from 40649 primary records. [2025-10-20T22:08:59.586092] [DEBUG] [LOCAL ] FMX: Saving derivative data. [2025-10-20T22:08:59.601326] [DEBUG] [LOCAL ] FMX: Derivative data saved (1794 records). [2025-10-20T22:08:59.601388] [INFO ] [MAIN ] FMX: Successfully saved derivative data. [2025-10-20T22:08:59.603691] [INFO ] [MAIN ] Derivative Progress: 690/1877 symbols completed. FMX took 0.9s. Estimated remaining: 7m 26s. Rate: 159.6 symbols/min [2025-10-20T22:08:59.604086] [INFO ] [MAIN ] FMY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:59.604222] [INFO ] [MAIN ] FN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:59.604517] [INFO ] [MAIN ] FNB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:08:59.610825] [DEBUG] [LOCAL ] FMY: Loaded 1381 existing records from primary storage. [2025-10-20T22:08:59.611083] [INFO ] [MAIN ] FMY: Loaded 1381 primary data records. [2025-10-20T22:08:59.959523] [DEBUG] [LOCAL ] FN: Loaded 37620 existing records from primary storage. [2025-10-20T22:08:59.959772] [INFO ] [MAIN ] FN: Loaded 37620 primary data records. [2025-10-20T22:09:00.094441] [DEBUG] [LOCAL ] FNB: Loaded 52605 existing records from primary storage. [2025-10-20T22:09:00.105320] [INFO ] [MAIN ] FNB: Loaded 52605 primary data records. [2025-10-20T22:09:00.606187] [DEBUG] [MAIN ] FMY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:00.607190] [INFO ] [MAIN ] FMY: Generated 1794 derivative records from 1381 primary records. [2025-10-20T22:09:00.607599] [DEBUG] [LOCAL ] FMY: Saving derivative data. [2025-10-20T22:09:00.624856] [DEBUG] [LOCAL ] FMY: Derivative data saved (1794 records). [2025-10-20T22:09:00.625113] [INFO ] [MAIN ] FMY: Successfully saved derivative data. [2025-10-20T22:09:00.627246] [INFO ] [MAIN ] Derivative Progress: 691/1877 symbols completed. FMY took 1.0s. Estimated remaining: 7m 27s. Rate: 159.2 symbols/min [2025-10-20T22:09:00.627517] [INFO ] [MAIN ] FND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:00.647264] [DEBUG] [MAIN ] FN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:00.648062] [INFO ] [MAIN ] FN: Generated 1794 derivative records from 37620 primary records. [2025-10-20T22:09:00.648256] [DEBUG] [LOCAL ] FN: Saving derivative data. [2025-10-20T22:09:00.666313] [DEBUG] [LOCAL ] FN: Derivative data saved (1794 records). [2025-10-20T22:09:00.668115] [INFO ] [MAIN ] FN: Successfully saved derivative data. [2025-10-20T22:09:00.670309] [INFO ] [MAIN ] Derivative Progress: 692/1877 symbols completed. FN took 1.1s. Estimated remaining: 7m 26s. Rate: 159.4 symbols/min [2025-10-20T22:09:00.670598] [INFO ] [MAIN ] FNF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:00.684160] [DEBUG] [MAIN ] FNB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:00.698886] [INFO ] [MAIN ] FNB: Generated 1794 derivative records from 52605 primary records. [2025-10-20T22:09:00.699216] [DEBUG] [LOCAL ] FNB: Saving derivative data. [2025-10-20T22:09:00.715551] [DEBUG] [LOCAL ] FNB: Derivative data saved (1794 records). [2025-10-20T22:09:00.716669] [INFO ] [MAIN ] FNB: Successfully saved derivative data. [2025-10-20T22:09:00.718942] [INFO ] [MAIN ] Derivative Progress: 693/1877 symbols completed. FNB took 1.1s. Estimated remaining: 7m 25s. Rate: 159.6 symbols/min [2025-10-20T22:09:00.719166] [INFO ] [MAIN ] FNV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:00.738029] [DEBUG] [LOCAL ] FND: Loaded 52091 existing records from primary storage. [2025-10-20T22:09:00.738383] [INFO ] [MAIN ] FND: Loaded 52091 primary data records. [2025-10-20T22:09:00.961943] [DEBUG] [LOCAL ] FNF: Loaded 48842 existing records from primary storage. [2025-10-20T22:09:00.962262] [INFO ] [MAIN ] FNF: Loaded 48842 primary data records. [2025-10-20T22:09:01.237752] [DEBUG] [LOCAL ] FNV: Loaded 46152 existing records from primary storage. [2025-10-20T22:09:01.253903] [INFO ] [MAIN ] FNV: Loaded 46152 primary data records. [2025-10-20T22:09:01.833849] [DEBUG] [MAIN ] FND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:01.837715] [INFO ] [MAIN ] FND: Generated 1794 derivative records from 52091 primary records. [2025-10-20T22:09:01.838850] [DEBUG] [LOCAL ] FND: Saving derivative data. [2025-10-20T22:09:01.857286] [DEBUG] [LOCAL ] FND: Derivative data saved (1794 records). [2025-10-20T22:09:01.857393] [INFO ] [MAIN ] FND: Successfully saved derivative data. [2025-10-20T22:09:01.859906] [INFO ] [MAIN ] Derivative Progress: 694/1877 symbols completed. FND took 1.2s. Estimated remaining: 7m 26s. Rate: 159.1 symbols/min [2025-10-20T22:09:01.860283] [INFO ] [MAIN ] FOA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:01.888355] [DEBUG] [MAIN ] FNF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:01.889451] [INFO ] [MAIN ] FNF: Generated 1794 derivative records from 48842 primary records. [2025-10-20T22:09:01.901622] [DEBUG] [LOCAL ] FNF: Saving derivative data. [2025-10-20T22:09:01.908183] [DEBUG] [LOCAL ] FOA: Loaded 13064 existing records from primary storage. [2025-10-20T22:09:01.907899] [DEBUG] [MAIN ] FNV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:01.916134] [INFO ] [MAIN ] FNV: Generated 1794 derivative records from 46152 primary records. [2025-10-20T22:09:01.908507] [INFO ] [MAIN ] FOA: Loaded 13064 primary data records. [2025-10-20T22:09:01.917094] [DEBUG] [LOCAL ] FNV: Saving derivative data. [2025-10-20T22:09:02.016375] [DEBUG] [LOCAL ] FNF: Derivative data saved (1794 records). [2025-10-20T22:09:02.042856] [DEBUG] [LOCAL ] FNV: Derivative data saved (1794 records). [2025-10-20T22:09:02.042235] [INFO ] [MAIN ] FNF: Successfully saved derivative data. [2025-10-20T22:09:02.043311] [INFO ] [MAIN ] FNV: Successfully saved derivative data. [2025-10-20T22:09:02.055972] [INFO ] [MAIN ] Derivative Progress: 695/1877 symbols completed. FNV took 1.3s. Estimated remaining: 7m 25s. Rate: 159.2 symbols/min [2025-10-20T22:09:02.068297] [INFO ] [MAIN ] FOF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:02.079005] [INFO ] [MAIN ] Derivative Progress: 696/1877 symbols completed. FNF took 1.4s. Estimated remaining: 7m 24s. Rate: 159.5 symbols/min [2025-10-20T22:09:02.084314] [INFO ] [MAIN ] FOR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:02.287723] [DEBUG] [LOCAL ] FOF: Loaded 8615 existing records from primary storage. [2025-10-20T22:09:02.290271] [INFO ] [MAIN ] FOF: Loaded 8615 primary data records. [2025-10-20T22:09:02.290679] [DEBUG] [LOCAL ] FOR: Loaded 19738 existing records from primary storage. [2025-10-20T22:09:02.290994] [INFO ] [MAIN ] FOR: Loaded 19738 primary data records. [2025-10-20T22:09:02.950505] [DEBUG] [MAIN ] FOA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:02.952698] [INFO ] [MAIN ] FOA: Generated 1794 derivative records from 13064 primary records. [2025-10-20T22:09:02.952969] [DEBUG] [LOCAL ] FOA: Saving derivative data. [2025-10-20T22:09:02.971928] [DEBUG] [LOCAL ] FOA: Derivative data saved (1794 records). [2025-10-20T22:09:02.972076] [INFO ] [MAIN ] FOA: Successfully saved derivative data. [2025-10-20T22:09:02.974086] [INFO ] [MAIN ] Derivative Progress: 697/1877 symbols completed. FOA took 1.1s. Estimated remaining: 7m 25s. Rate: 159.1 symbols/min [2025-10-20T22:09:02.974619] [INFO ] [MAIN ] FOUR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:03.003312] [DEBUG] [MAIN ] FOF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:03.009699] [INFO ] [MAIN ] FOF: Generated 1794 derivative records from 8615 primary records. [2025-10-20T22:09:03.010871] [DEBUG] [MAIN ] FOR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:03.010505] [DEBUG] [LOCAL ] FOF: Saving derivative data. [2025-10-20T22:09:03.012158] [INFO ] [MAIN ] FOR: Generated 1794 derivative records from 19738 primary records. [2025-10-20T22:09:03.013366] [DEBUG] [LOCAL ] FOR: Saving derivative data. [2025-10-20T22:09:03.048453] [DEBUG] [LOCAL ] FOR: Derivative data saved (1794 records). [2025-10-20T22:09:03.048930] [INFO ] [MAIN ] FOR: Successfully saved derivative data. [2025-10-20T22:09:03.050962] [INFO ] [MAIN ] Derivative Progress: 698/1877 symbols completed. FOR took 1.0s. Estimated remaining: 7m 24s. Rate: 159.3 symbols/min [2025-10-20T22:09:03.045855] [DEBUG] [LOCAL ] FOF: Derivative data saved (1794 records). [2025-10-20T22:09:03.065603] [INFO ] [MAIN ] FPF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:03.066099] [INFO ] [MAIN ] FOF: Successfully saved derivative data. [2025-10-20T22:09:03.069242] [INFO ] [MAIN ] Derivative Progress: 699/1877 symbols completed. FOF took 1.0s. Estimated remaining: 7m 23s. Rate: 159.5 symbols/min [2025-10-20T22:09:03.069635] [INFO ] [MAIN ] FPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:03.104639] [DEBUG] [LOCAL ] FOUR: Loaded 49761 existing records from primary storage. [2025-10-20T22:09:03.105067] [INFO ] [MAIN ] FOUR: Loaded 49761 primary data records. [2025-10-20T22:09:03.169991] [DEBUG] [LOCAL ] FPF: Loaded 18844 existing records from primary storage. [2025-10-20T22:09:03.198295] [INFO ] [MAIN ] FPF: Loaded 18844 primary data records. [2025-10-20T22:09:03.209480] [DEBUG] [LOCAL ] FPH: Loaded 19328 existing records from primary storage. [2025-10-20T22:09:03.256380] [INFO ] [MAIN ] FPH: Loaded 19328 primary data records. [2025-10-20T22:09:04.106182] [DEBUG] [MAIN ] FOUR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:04.107097] [INFO ] [MAIN ] FOUR: Generated 1794 derivative records from 49761 primary records. [2025-10-20T22:09:04.107358] [DEBUG] [LOCAL ] FOUR: Saving derivative data. [2025-10-20T22:09:04.129067] [DEBUG] [LOCAL ] FOUR: Derivative data saved (1794 records). [2025-10-20T22:09:04.130105] [INFO ] [MAIN ] FOUR: Successfully saved derivative data. [2025-10-20T22:09:04.133047] [INFO ] [MAIN ] Derivative Progress: 700/1877 symbols completed. FOUR took 1.2s. Estimated remaining: 7m 24s. Rate: 159.1 symbols/min [2025-10-20T22:09:04.141193] [INFO ] [MAIN ] FPI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:04.160099] [DEBUG] [MAIN ] FPF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:04.160865] [DEBUG] [MAIN ] FPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:04.161818] [INFO ] [MAIN ] FPF: Generated 1794 derivative records from 18844 primary records. [2025-10-20T22:09:04.163123] [INFO ] [MAIN ] FPH: Generated 1794 derivative records from 19328 primary records. [2025-10-20T22:09:04.161954] [DEBUG] [LOCAL ] FPF: Saving derivative data. [2025-10-20T22:09:04.164151] [DEBUG] [LOCAL ] FPH: Saving derivative data. [2025-10-20T22:09:04.196495] [DEBUG] [LOCAL ] FPF: Derivative data saved (1794 records). [2025-10-20T22:09:04.199116] [DEBUG] [LOCAL ] FPH: Derivative data saved (1794 records). [2025-10-20T22:09:04.199692] [INFO ] [MAIN ] FPH: Successfully saved derivative data. [2025-10-20T22:09:04.199495] [INFO ] [MAIN ] FPF: Successfully saved derivative data. [2025-10-20T22:09:04.202603] [INFO ] [MAIN ] Derivative Progress: 701/1877 symbols completed. FPF took 1.1s. Estimated remaining: 7m 23s. Rate: 159.3 symbols/min [2025-10-20T22:09:04.205572] [INFO ] [MAIN ] FR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:04.205763] [INFO ] [MAIN ] Derivative Progress: 702/1877 symbols completed. FPH took 1.1s. Estimated remaining: 7m 22s. Rate: 159.5 symbols/min [2025-10-20T22:09:04.206852] [INFO ] [MAIN ] FRA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:04.232493] [DEBUG] [LOCAL ] FRA: Loaded 14261 existing records from primary storage. [2025-10-20T22:09:04.243110] [INFO ] [MAIN ] FRA: Loaded 14261 primary data records. [2025-10-20T22:09:04.243593] [DEBUG] [LOCAL ] FPI: Loaded 31464 existing records from primary storage. [2025-10-20T22:09:04.243714] [INFO ] [MAIN ] FPI: Loaded 31464 primary data records. [2025-10-20T22:09:04.762838] [DEBUG] [LOCAL ] FR: Loaded 46684 existing records from primary storage. [2025-10-20T22:09:04.773668] [INFO ] [MAIN ] FR: Loaded 46684 primary data records. [2025-10-20T22:09:05.220898] [DEBUG] [MAIN ] FRA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:05.222021] [INFO ] [MAIN ] FRA: Generated 1794 derivative records from 14261 primary records. [2025-10-20T22:09:05.222237] [DEBUG] [LOCAL ] FRA: Saving derivative data. [2025-10-20T22:09:05.238706] [DEBUG] [MAIN ] FPI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:05.248472] [INFO ] [MAIN ] FPI: Generated 1794 derivative records from 31464 primary records. [2025-10-20T22:09:05.251296] [DEBUG] [LOCAL ] FPI: Saving derivative data. [2025-10-20T22:09:05.256187] [DEBUG] [LOCAL ] FRA: Derivative data saved (1794 records). [2025-10-20T22:09:05.256809] [INFO ] [MAIN ] FRA: Successfully saved derivative data. [2025-10-20T22:09:05.259112] [INFO ] [MAIN ] Derivative Progress: 703/1877 symbols completed. FRA took 1.1s. Estimated remaining: 7m 23s. Rate: 159.1 symbols/min [2025-10-20T22:09:05.282593] [DEBUG] [LOCAL ] FPI: Derivative data saved (1794 records). [2025-10-20T22:09:05.283768] [INFO ] [MAIN ] FPI: Successfully saved derivative data. [2025-10-20T22:09:05.287161] [INFO ] [MAIN ] Derivative Progress: 704/1877 symbols completed. FPI took 1.1s. Estimated remaining: 7m 22s. Rate: 159.3 symbols/min [2025-10-20T22:09:05.300423] [DEBUG] [MAIN ] FR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:05.301050] [INFO ] [MAIN ] FR: Generated 1794 derivative records from 46684 primary records. [2025-10-20T22:09:05.301112] [DEBUG] [LOCAL ] FR: Saving derivative data. [2025-10-20T22:09:05.316128] [DEBUG] [LOCAL ] FR: Derivative data saved (1794 records). [2025-10-20T22:09:05.316188] [INFO ] [MAIN ] FR: Successfully saved derivative data. [2025-10-20T22:09:05.318177] [INFO ] [MAIN ] Derivative Progress: 705/1877 symbols completed. FR took 1.1s. Estimated remaining: 7m 21s. Rate: 159.5 symbols/min [2025-10-20T22:09:05.318778] [INFO ] [MAIN ] FRGE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:05.319225] [INFO ] [MAIN ] FRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:05.318917] [INFO ] [MAIN ] FRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:05.338409] [DEBUG] [LOCAL ] FRGE: Loaded 15767 existing records from primary storage. [2025-10-20T22:09:05.338562] [INFO ] [MAIN ] FRGE: Loaded 15767 primary data records. [2025-10-20T22:09:05.500432] [DEBUG] [LOCAL ] FRT: Loaded 41648 existing records from primary storage. [2025-10-20T22:09:05.528314] [INFO ] [MAIN ] FRT: Loaded 41648 primary data records. [2025-10-20T22:09:05.821524] [DEBUG] [LOCAL ] FRO: Loaded 51691 existing records from primary storage. [2025-10-20T22:09:05.837405] [INFO ] [MAIN ] FRO: Loaded 51691 primary data records. [2025-10-20T22:09:06.387676] [DEBUG] [MAIN ] FRGE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:06.388884] [INFO ] [MAIN ] FRGE: Generated 1794 derivative records from 15767 primary records. [2025-10-20T22:09:06.389144] [DEBUG] [LOCAL ] FRGE: Saving derivative data. [2025-10-20T22:09:06.408691] [DEBUG] [LOCAL ] FRGE: Derivative data saved (1794 records). [2025-10-20T22:09:06.408903] [INFO ] [MAIN ] FRGE: Successfully saved derivative data. [2025-10-20T22:09:06.411148] [INFO ] [MAIN ] Derivative Progress: 706/1877 symbols completed. FRGE took 1.1s. Estimated remaining: 7m 22s. Rate: 159.1 symbols/min [2025-10-20T22:09:06.411462] [INFO ] [MAIN ] FSCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:06.427084] [DEBUG] [MAIN ] FRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:06.428115] [INFO ] [MAIN ] FRT: Generated 1794 derivative records from 41648 primary records. [2025-10-20T22:09:06.428399] [DEBUG] [LOCAL ] FRT: Saving derivative data. [2025-10-20T22:09:06.450062] [DEBUG] [LOCAL ] FRT: Derivative data saved (1794 records). [2025-10-20T22:09:06.450466] [INFO ] [MAIN ] FRT: Successfully saved derivative data. [2025-10-20T22:09:06.452471] [INFO ] [MAIN ] Derivative Progress: 707/1877 symbols completed. FRT took 1.1s. Estimated remaining: 7m 21s. Rate: 159.3 symbols/min [2025-10-20T22:09:06.474317] [INFO ] [MAIN ] FSK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:06.474626] [DEBUG] [MAIN ] FRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:06.475605] [INFO ] [MAIN ] FRO: Generated 1794 derivative records from 51691 primary records. [2025-10-20T22:09:06.475756] [DEBUG] [LOCAL ] FRO: Saving derivative data. [2025-10-20T22:09:06.493832] [DEBUG] [LOCAL ] FRO: Derivative data saved (1794 records). [2025-10-20T22:09:06.504408] [INFO ] [MAIN ] FRO: Successfully saved derivative data. [2025-10-20T22:09:06.506568] [INFO ] [MAIN ] Derivative Progress: 708/1877 symbols completed. FRO took 1.2s. Estimated remaining: 7m 20s. Rate: 159.5 symbols/min [2025-10-20T22:09:06.507967] [INFO ] [MAIN ] FSLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:06.509344] [DEBUG] [LOCAL ] FSCO: Loaded 38696 existing records from primary storage. [2025-10-20T22:09:06.509455] [INFO ] [MAIN ] FSCO: Loaded 38696 primary data records. [2025-10-20T22:09:06.863426] [DEBUG] [LOCAL ] FSK: Loaded 49506 existing records from primary storage. [2025-10-20T22:09:06.865335] [INFO ] [MAIN ] FSK: Loaded 49506 primary data records. [2025-10-20T22:09:06.864183] [DEBUG] [LOCAL ] FSLY: Loaded 51345 existing records from primary storage. [2025-10-20T22:09:06.866179] [INFO ] [MAIN ] FSLY: Loaded 51345 primary data records. [2025-10-20T22:09:07.514286] [DEBUG] [MAIN ] FSCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:07.515350] [INFO ] [MAIN ] FSCO: Generated 1794 derivative records from 38696 primary records. [2025-10-20T22:09:07.516012] [DEBUG] [LOCAL ] FSCO: Saving derivative data. [2025-10-20T22:09:07.537284] [DEBUG] [LOCAL ] FSCO: Derivative data saved (1794 records). [2025-10-20T22:09:07.537502] [INFO ] [MAIN ] FSCO: Successfully saved derivative data. [2025-10-20T22:09:07.539831] [INFO ] [MAIN ] Derivative Progress: 709/1877 symbols completed. FSCO took 1.1s. Estimated remaining: 7m 20s. Rate: 159.1 symbols/min [2025-10-20T22:09:07.540497] [INFO ] [MAIN ] FSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:07.564149] [DEBUG] [MAIN ] FSLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:07.565077] [INFO ] [MAIN ] FSLY: Generated 1794 derivative records from 51345 primary records. [2025-10-20T22:09:07.565414] [DEBUG] [LOCAL ] FSLY: Saving derivative data. [2025-10-20T22:09:07.579670] [DEBUG] [MAIN ] FSK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:07.587524] [INFO ] [MAIN ] FSK: Generated 1794 derivative records from 49506 primary records. [2025-10-20T22:09:07.591589] [DEBUG] [LOCAL ] FSK: Saving derivative data. [2025-10-20T22:09:07.596460] [DEBUG] [LOCAL ] FSLY: Derivative data saved (1794 records). [2025-10-20T22:09:07.596817] [INFO ] [MAIN ] FSLY: Successfully saved derivative data. [2025-10-20T22:09:07.605892] [INFO ] [MAIN ] Derivative Progress: 710/1877 symbols completed. FSLY took 1.1s. Estimated remaining: 7m 20s. Rate: 159.3 symbols/min [2025-10-20T22:09:07.606120] [INFO ] [MAIN ] FSS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:07.628771] [DEBUG] [LOCAL ] FSK: Derivative data saved (1794 records). [2025-10-20T22:09:07.628919] [INFO ] [MAIN ] FSK: Successfully saved derivative data. [2025-10-20T22:09:07.631085] [INFO ] [MAIN ] Derivative Progress: 711/1877 symbols completed. FSK took 1.2s. Estimated remaining: 7m 19s. Rate: 159.5 symbols/min [2025-10-20T22:09:07.631421] [INFO ] [MAIN ] FT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:07.654800] [DEBUG] [LOCAL ] FSM: Loaded 53303 existing records from primary storage. [2025-10-20T22:09:07.655137] [INFO ] [MAIN ] FSM: Loaded 53303 primary data records. [2025-10-20T22:09:07.677983] [DEBUG] [LOCAL ] FT: Loaded 6734 existing records from primary storage. [2025-10-20T22:09:07.704097] [INFO ] [MAIN ] FT: Loaded 6734 primary data records. [2025-10-20T22:09:07.945688] [DEBUG] [LOCAL ] FSS: Loaded 33463 existing records from primary storage. [2025-10-20T22:09:07.978211] [INFO ] [MAIN ] FSS: Loaded 33463 primary data records. [2025-10-20T22:09:08.629962] [DEBUG] [MAIN ] FSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:08.644395] [INFO ] [MAIN ] FSM: Generated 1794 derivative records from 53303 primary records. [2025-10-20T22:09:08.644528] [DEBUG] [LOCAL ] FSM: Saving derivative data. [2025-10-20T22:09:08.643531] [DEBUG] [MAIN ] FT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:08.646455] [INFO ] [MAIN ] FT: Generated 1794 derivative records from 6734 primary records. [2025-10-20T22:09:08.654319] [DEBUG] [LOCAL ] FT: Saving derivative data. [2025-10-20T22:09:08.663619] [DEBUG] [LOCAL ] FSM: Derivative data saved (1794 records). [2025-10-20T22:09:08.664103] [INFO ] [MAIN ] FSM: Successfully saved derivative data. [2025-10-20T22:09:08.676903] [INFO ] [MAIN ] Derivative Progress: 712/1877 symbols completed. FSM took 1.1s. Estimated remaining: 7m 19s. Rate: 159.1 symbols/min [2025-10-20T22:09:08.677437] [INFO ] [MAIN ] FTHY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:08.683084] [DEBUG] [LOCAL ] FT: Derivative data saved (1794 records). [2025-10-20T22:09:08.683384] [INFO ] [MAIN ] FT: Successfully saved derivative data. [2025-10-20T22:09:08.685919] [INFO ] [MAIN ] Derivative Progress: 713/1877 symbols completed. FT took 1.1s. Estimated remaining: 7m 18s. Rate: 159.3 symbols/min [2025-10-20T22:09:08.686439] [INFO ] [MAIN ] FTI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:08.711608] [DEBUG] [LOCAL ] FTHY: Loaded 12520 existing records from primary storage. [2025-10-20T22:09:08.712236] [INFO ] [MAIN ] FTHY: Loaded 12520 primary data records. [2025-10-20T22:09:08.712563] [DEBUG] [MAIN ] FSS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:08.743997] [INFO ] [MAIN ] FSS: Generated 1794 derivative records from 33463 primary records. [2025-10-20T22:09:08.756104] [DEBUG] [LOCAL ] FSS: Saving derivative data. [2025-10-20T22:09:08.830958] [DEBUG] [LOCAL ] FSS: Derivative data saved (1794 records). [2025-10-20T22:09:08.848708] [INFO ] [MAIN ] FSS: Successfully saved derivative data. [2025-10-20T22:09:08.861560] [INFO ] [MAIN ] Derivative Progress: 714/1877 symbols completed. FSS took 1.3s. Estimated remaining: 7m 18s. Rate: 159.5 symbols/min [2025-10-20T22:09:08.887753] [INFO ] [MAIN ] FTK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:09.058122] [DEBUG] [LOCAL ] FTI: Loaded 53290 existing records from primary storage. [2025-10-20T22:09:09.059182] [INFO ] [MAIN ] FTI: Loaded 53290 primary data records. [2025-10-20T22:09:09.236879] [DEBUG] [LOCAL ] FTK: Loaded 32881 existing records from primary storage. [2025-10-20T22:09:09.243844] [INFO ] [MAIN ] FTK: Loaded 32881 primary data records. [2025-10-20T22:09:09.512627] [DEBUG] [MAIN ] FTHY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:09.580454] [INFO ] [MAIN ] FTHY: Generated 1794 derivative records from 12520 primary records. [2025-10-20T22:09:09.580601] [DEBUG] [LOCAL ] FTHY: Saving derivative data. [2025-10-20T22:09:09.758889] [DEBUG] [LOCAL ] FTHY: Derivative data saved (1794 records). [2025-10-20T22:09:09.759262] [INFO ] [MAIN ] FTHY: Successfully saved derivative data. [2025-10-20T22:09:09.762339] [INFO ] [MAIN ] Derivative Progress: 715/1877 symbols completed. FTHY took 1.1s. Estimated remaining: 7m 18s. Rate: 159.1 symbols/min [2025-10-20T22:09:09.762609] [INFO ] [MAIN ] FTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:09.783202] [DEBUG] [MAIN ] FTI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:09.784041] [INFO ] [MAIN ] FTI: Generated 1794 derivative records from 53290 primary records. [2025-10-20T22:09:09.784257] [DEBUG] [LOCAL ] FTI: Saving derivative data. [2025-10-20T22:09:09.801373] [DEBUG] [LOCAL ] FTI: Derivative data saved (1794 records). [2025-10-20T22:09:09.802455] [INFO ] [MAIN ] FTI: Successfully saved derivative data. [2025-10-20T22:09:09.804840] [INFO ] [MAIN ] Derivative Progress: 716/1877 symbols completed. FTI took 1.1s. Estimated remaining: 7m 17s. Rate: 159.3 symbols/min [2025-10-20T22:09:09.806686] [INFO ] [MAIN ] FTV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:09.831615] [DEBUG] [MAIN ] FTK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:09.832419] [INFO ] [MAIN ] FTK: Generated 1794 derivative records from 32881 primary records. [2025-10-20T22:09:09.832527] [DEBUG] [LOCAL ] FTK: Saving derivative data. [2025-10-20T22:09:09.861967] [DEBUG] [LOCAL ] FTK: Derivative data saved (1794 records). [2025-10-20T22:09:09.862112] [INFO ] [MAIN ] FTK: Successfully saved derivative data. [2025-10-20T22:09:09.864242] [INFO ] [MAIN ] Derivative Progress: 717/1877 symbols completed. FTK took 1.0s. Estimated remaining: 7m 16s. Rate: 159.5 symbols/min [2025-10-20T22:09:09.864467] [INFO ] [MAIN ] FTWO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:09.865767] [DEBUG] [LOCAL ] FTS: Loaded 42558 existing records from primary storage. [2025-10-20T22:09:09.865918] [INFO ] [MAIN ] FTS: Loaded 42558 primary data records. [2025-10-20T22:09:09.960373] [DEBUG] [LOCAL ] FTWO: Loaded 1955 existing records from primary storage. [2025-10-20T22:09:09.985391] [INFO ] [MAIN ] FTWO: Loaded 1955 primary data records. [2025-10-20T22:09:10.297628] [DEBUG] [LOCAL ] FTV: Loaded 53059 existing records from primary storage. [2025-10-20T22:09:10.304150] [INFO ] [MAIN ] FTV: Loaded 53059 primary data records. [2025-10-20T22:09:10.829627] [DEBUG] [MAIN ] FTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:10.830568] [INFO ] [MAIN ] FTS: Generated 1794 derivative records from 42558 primary records. [2025-10-20T22:09:10.830864] [DEBUG] [LOCAL ] FTS: Saving derivative data. [2025-10-20T22:09:10.849995] [DEBUG] [LOCAL ] FTS: Derivative data saved (1794 records). [2025-10-20T22:09:10.850224] [INFO ] [MAIN ] FTS: Successfully saved derivative data. [2025-10-20T22:09:10.852320] [INFO ] [MAIN ] Derivative Progress: 718/1877 symbols completed. FTS took 1.1s. Estimated remaining: 7m 17s. Rate: 159.2 symbols/min [2025-10-20T22:09:10.865693] [DEBUG] [MAIN ] FTWO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:10.866350] [INFO ] [MAIN ] FTWO: Generated 1793 derivative records from 1955 primary records. [2025-10-20T22:09:10.866422] [DEBUG] [LOCAL ] FTWO: Saving derivative data. [2025-10-20T22:09:10.882749] [DEBUG] [LOCAL ] FTWO: Derivative data saved (1793 records). [2025-10-20T22:09:10.882909] [INFO ] [MAIN ] FTWO: Successfully saved derivative data. [2025-10-20T22:09:10.885060] [INFO ] [MAIN ] Derivative Progress: 719/1877 symbols completed. FTWO took 1.0s. Estimated remaining: 7m 16s. Rate: 159.4 symbols/min [2025-10-20T22:09:10.897407] [DEBUG] [MAIN ] FTV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:10.898115] [INFO ] [MAIN ] FTV: Generated 1794 derivative records from 53059 primary records. [2025-10-20T22:09:10.898177] [DEBUG] [LOCAL ] FTV: Saving derivative data. [2025-10-20T22:09:10.913387] [DEBUG] [LOCAL ] FTV: Derivative data saved (1794 records). [2025-10-20T22:09:10.913444] [INFO ] [MAIN ] FTV: Successfully saved derivative data. [2025-10-20T22:09:10.915522] [INFO ] [MAIN ] Derivative Progress: 720/1877 symbols completed. FTV took 1.1s. Estimated remaining: 7m 15s. Rate: 159.6 symbols/min [2025-10-20T22:09:10.916010] [INFO ] [MAIN ] FUN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:10.915948] [INFO ] [MAIN ] FUBO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:10.916089] [INFO ] [MAIN ] FUL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:10.975330] [DEBUG] [LOCAL ] FUL: Loaded 32097 existing records from primary storage. [2025-10-20T22:09:10.975584] [INFO ] [MAIN ] FUL: Loaded 32097 primary data records. [2025-10-20T22:09:11.110186] [DEBUG] [LOCAL ] FUN: Loaded 52012 existing records from primary storage. [2025-10-20T22:09:11.110407] [INFO ] [MAIN ] FUN: Loaded 52012 primary data records. [2025-10-20T22:09:11.111124] [DEBUG] [LOCAL ] FUBO: Loaded 53718 existing records from primary storage. [2025-10-20T22:09:11.122957] [INFO ] [MAIN ] FUBO: Loaded 53718 primary data records. [2025-10-20T22:09:11.983671] [DEBUG] [MAIN ] FUL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:11.984849] [INFO ] [MAIN ] FUL: Generated 1794 derivative records from 32097 primary records. [2025-10-20T22:09:11.984942] [DEBUG] [LOCAL ] FUL: Saving derivative data. [2025-10-20T22:09:12.003267] [DEBUG] [LOCAL ] FUL: Derivative data saved (1794 records). [2025-10-20T22:09:12.003563] [INFO ] [MAIN ] FUL: Successfully saved derivative data. [2025-10-20T22:09:12.013878] [INFO ] [MAIN ] Derivative Progress: 721/1877 symbols completed. FUL took 1.1s. Estimated remaining: 7m 16s. Rate: 159.1 symbols/min [2025-10-20T22:09:12.019682] [INFO ] [MAIN ] FVR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:12.020826] [DEBUG] [MAIN ] FUBO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:12.021872] [INFO ] [MAIN ] FUBO: Generated 1794 derivative records from 53718 primary records. [2025-10-20T22:09:12.022148] [DEBUG] [LOCAL ] FUBO: Saving derivative data. [2025-10-20T22:09:12.048669] [DEBUG] [MAIN ] FUN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:12.049525] [INFO ] [MAIN ] FUN: Generated 1794 derivative records from 52012 primary records. [2025-10-20T22:09:12.052560] [DEBUG] [LOCAL ] FUBO: Derivative data saved (1794 records). [2025-10-20T22:09:12.052500] [DEBUG] [LOCAL ] FUN: Saving derivative data. [2025-10-20T22:09:12.052897] [INFO ] [MAIN ] FUBO: Successfully saved derivative data. [2025-10-20T22:09:12.055408] [INFO ] [MAIN ] Derivative Progress: 722/1877 symbols completed. FUBO took 1.1s. Estimated remaining: 7m 15s. Rate: 159.3 symbols/min [2025-10-20T22:09:12.055842] [INFO ] [MAIN ] FVRR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:12.077268] [DEBUG] [LOCAL ] FUN: Derivative data saved (1794 records). [2025-10-20T22:09:12.077422] [INFO ] [MAIN ] FUN: Successfully saved derivative data. [2025-10-20T22:09:12.079844] [INFO ] [MAIN ] Derivative Progress: 723/1877 symbols completed. FUN took 1.2s. Estimated remaining: 7m 14s. Rate: 159.6 symbols/min [2025-10-20T22:09:12.080088] [INFO ] [MAIN ] FXED: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:12.091431] [DEBUG] [LOCAL ] FXED: Loaded 1237 existing records from primary storage. [2025-10-20T22:09:12.092045] [INFO ] [MAIN ] FXED: Loaded 1237 primary data records. [2025-10-20T22:09:12.091955] [DEBUG] [LOCAL ] FVR: Loaded 20901 existing records from primary storage. [2025-10-20T22:09:12.113452] [INFO ] [MAIN ] FVR: Loaded 20901 primary data records. [2025-10-20T22:09:12.600396] [DEBUG] [LOCAL ] FVRR: Loaded 40712 existing records from primary storage. [2025-10-20T22:09:12.702145] [INFO ] [MAIN ] FVRR: Loaded 40712 primary data records. [2025-10-20T22:09:13.053981] [DEBUG] [MAIN ] FXED: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:13.068334] [INFO ] [MAIN ] FXED: Generated 1793 derivative records from 1237 primary records. [2025-10-20T22:09:13.068696] [DEBUG] [LOCAL ] FXED: Saving derivative data. [2025-10-20T22:09:13.069110] [DEBUG] [MAIN ] FVR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:13.076780] [INFO ] [MAIN ] FVR: Generated 1794 derivative records from 20901 primary records. [2025-10-20T22:09:13.085423] [DEBUG] [LOCAL ] FXED: Derivative data saved (1793 records). [2025-10-20T22:09:13.085861] [DEBUG] [LOCAL ] FVR: Saving derivative data. [2025-10-20T22:09:13.086247] [INFO ] [MAIN ] FXED: Successfully saved derivative data. [2025-10-20T22:09:13.089082] [INFO ] [MAIN ] Derivative Progress: 724/1877 symbols completed. FXED took 1.0s. Estimated remaining: 7m 15s. Rate: 159.2 symbols/min [2025-10-20T22:09:13.090089] [INFO ] [MAIN ] G: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:13.108019] [DEBUG] [LOCAL ] FVR: Derivative data saved (1794 records). [2025-10-20T22:09:13.108239] [INFO ] [MAIN ] FVR: Successfully saved derivative data. [2025-10-20T22:09:13.111123] [INFO ] [MAIN ] Derivative Progress: 725/1877 symbols completed. FVR took 1.1s. Estimated remaining: 7m 14s. Rate: 159.4 symbols/min [2025-10-20T22:09:13.111687] [INFO ] [MAIN ] GAB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:13.128588] [DEBUG] [MAIN ] FVRR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:13.129440] [INFO ] [MAIN ] FVRR: Generated 1794 derivative records from 40712 primary records. [2025-10-20T22:09:13.129578] [DEBUG] [LOCAL ] FVRR: Saving derivative data. [2025-10-20T22:09:13.146315] [DEBUG] [LOCAL ] FVRR: Derivative data saved (1794 records). [2025-10-20T22:09:13.146502] [INFO ] [MAIN ] FVRR: Successfully saved derivative data. [2025-10-20T22:09:13.148710] [INFO ] [MAIN ] Derivative Progress: 726/1877 symbols completed. FVRR took 1.1s. Estimated remaining: 7m 13s. Rate: 159.6 symbols/min [2025-10-20T22:09:13.149013] [INFO ] [MAIN ] GAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:13.164886] [DEBUG] [LOCAL ] GAM: Loaded 3901 existing records from primary storage. [2025-10-20T22:09:13.174787] [INFO ] [MAIN ] GAM: Loaded 3901 primary data records. [2025-10-20T22:09:13.325439] [DEBUG] [LOCAL ] G: Loaded 50114 existing records from primary storage. [2025-10-20T22:09:13.314337] [DEBUG] [LOCAL ] GAB: Loaded 27744 existing records from primary storage. [2025-10-20T22:09:13.349211] [INFO ] [MAIN ] G: Loaded 50114 primary data records. [2025-10-20T22:09:13.358071] [INFO ] [MAIN ] GAB: Loaded 27744 primary data records. [2025-10-20T22:09:14.175073] [DEBUG] [MAIN ] GAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:14.175930] [INFO ] [MAIN ] GAM: Generated 1794 derivative records from 3901 primary records. [2025-10-20T22:09:14.176385] [DEBUG] [LOCAL ] GAM: Saving derivative data. [2025-10-20T22:09:14.191478] [DEBUG] [LOCAL ] GAM: Derivative data saved (1794 records). [2025-10-20T22:09:14.191675] [INFO ] [MAIN ] GAM: Successfully saved derivative data. [2025-10-20T22:09:14.194711] [INFO ] [MAIN ] Derivative Progress: 727/1877 symbols completed. GAM took 1.0s. Estimated remaining: 7m 13s. Rate: 159.2 symbols/min [2025-10-20T22:09:14.195764] [INFO ] [MAIN ] GAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:14.224153] [DEBUG] [MAIN ] GAB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:14.224494] [DEBUG] [MAIN ] G: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:14.225926] [INFO ] [MAIN ] G: Generated 1794 derivative records from 50114 primary records. [2025-10-20T22:09:14.226076] [DEBUG] [LOCAL ] G: Saving derivative data. [2025-10-20T22:09:14.225332] [INFO ] [MAIN ] GAB: Generated 1794 derivative records from 27744 primary records. [2025-10-20T22:09:14.226577] [DEBUG] [LOCAL ] GAB: Saving derivative data. [2025-10-20T22:09:14.253850] [DEBUG] [LOCAL ] G: Derivative data saved (1794 records). [2025-10-20T22:09:14.255445] [INFO ] [MAIN ] G: Successfully saved derivative data. [2025-10-20T22:09:14.258160] [INFO ] [MAIN ] Derivative Progress: 728/1877 symbols completed. G took 1.2s. Estimated remaining: 7m 13s. Rate: 159.4 symbols/min [2025-10-20T22:09:14.258522] [INFO ] [MAIN ] GATX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:14.260353] [DEBUG] [LOCAL ] GAB: Derivative data saved (1794 records). [2025-10-20T22:09:14.260531] [INFO ] [MAIN ] GAB: Successfully saved derivative data. [2025-10-20T22:09:14.265065] [INFO ] [MAIN ] Derivative Progress: 729/1877 symbols completed. GAB took 1.2s. Estimated remaining: 7m 12s. Rate: 159.6 symbols/min [2025-10-20T22:09:14.265443] [INFO ] [MAIN ] GBAB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:14.302938] [DEBUG] [LOCAL ] GATX: Loaded 18106 existing records from primary storage. [2025-10-20T22:09:14.302625] [DEBUG] [LOCAL ] GBAB: Loaded 12847 existing records from primary storage. [2025-10-20T22:09:14.303173] [INFO ] [MAIN ] GATX: Loaded 18106 primary data records. [2025-10-20T22:09:14.303330] [INFO ] [MAIN ] GBAB: Loaded 12847 primary data records. [2025-10-20T22:09:14.506875] [DEBUG] [LOCAL ] GAP: Loaded 53653 existing records from primary storage. [2025-10-20T22:09:14.525568] [INFO ] [MAIN ] GAP: Loaded 53653 primary data records. [2025-10-20T22:09:15.278125] [DEBUG] [MAIN ] GBAB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:15.281062] [INFO ] [MAIN ] GBAB: Generated 1794 derivative records from 12847 primary records. [2025-10-20T22:09:15.281337] [DEBUG] [LOCAL ] GBAB: Saving derivative data. [2025-10-20T22:09:15.312295] [DEBUG] [LOCAL ] GBAB: Derivative data saved (1794 records). [2025-10-20T22:09:15.312405] [INFO ] [MAIN ] GBAB: Successfully saved derivative data. [2025-10-20T22:09:15.315786] [INFO ] [MAIN ] Derivative Progress: 730/1877 symbols completed. GBAB took 1.1s. Estimated remaining: 7m 12s. Rate: 159.2 symbols/min [2025-10-20T22:09:15.316817] [INFO ] [MAIN ] GBCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:15.318702] [DEBUG] [MAIN ] GATX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:15.319935] [INFO ] [MAIN ] GATX: Generated 1794 derivative records from 18106 primary records. [2025-10-20T22:09:15.320191] [DEBUG] [LOCAL ] GATX: Saving derivative data. [2025-10-20T22:09:15.346670] [DEBUG] [MAIN ] GAP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:15.351819] [INFO ] [MAIN ] GAP: Generated 1794 derivative records from 53653 primary records. [2025-10-20T22:09:15.352010] [DEBUG] [LOCAL ] GAP: Saving derivative data. [2025-10-20T22:09:15.351928] [DEBUG] [LOCAL ] GATX: Derivative data saved (1794 records). [2025-10-20T22:09:15.352345] [INFO ] [MAIN ] GATX: Successfully saved derivative data. [2025-10-20T22:09:15.354711] [INFO ] [MAIN ] Derivative Progress: 731/1877 symbols completed. GATX took 1.1s. Estimated remaining: 7m 11s. Rate: 159.4 symbols/min [2025-10-20T22:09:15.355336] [INFO ] [MAIN ] GBTG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:15.372921] [DEBUG] [LOCAL ] GAP: Derivative data saved (1794 records). [2025-10-20T22:09:15.373085] [INFO ] [MAIN ] GAP: Successfully saved derivative data. [2025-10-20T22:09:15.376039] [INFO ] [MAIN ] Derivative Progress: 732/1877 symbols completed. GAP took 1.2s. Estimated remaining: 7m 10s. Rate: 159.6 symbols/min [2025-10-20T22:09:15.376282] [INFO ] [MAIN ] GBX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:15.435801] [DEBUG] [LOCAL ] GBCI: Loaded 41215 existing records from primary storage. [2025-10-20T22:09:15.436069] [INFO ] [MAIN ] GBCI: Loaded 41215 primary data records. [2025-10-20T22:09:15.512996] [DEBUG] [LOCAL ] GBTG: Loaded 40310 existing records from primary storage. [2025-10-20T22:09:15.535178] [DEBUG] [LOCAL ] GBX: Loaded 28567 existing records from primary storage. [2025-10-20T22:09:15.545962] [INFO ] [MAIN ] GBX: Loaded 28567 primary data records. [2025-10-20T22:09:15.584205] [INFO ] [MAIN ] GBTG: Loaded 40310 primary data records. [2025-10-20T22:09:16.462110] [DEBUG] [MAIN ] GBCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:16.462960] [INFO ] [MAIN ] GBCI: Generated 1794 derivative records from 41215 primary records. [2025-10-20T22:09:16.463254] [DEBUG] [LOCAL ] GBCI: Saving derivative data. [2025-10-20T22:09:16.481178] [DEBUG] [LOCAL ] GBCI: Derivative data saved (1794 records). [2025-10-20T22:09:16.481593] [INFO ] [MAIN ] GBCI: Successfully saved derivative data. [2025-10-20T22:09:16.483477] [INFO ] [MAIN ] Derivative Progress: 733/1877 symbols completed. GBCI took 1.2s. Estimated remaining: 7m 11s. Rate: 159.2 symbols/min [2025-10-20T22:09:16.500203] [DEBUG] [MAIN ] GBTG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:16.501668] [INFO ] [MAIN ] GBTG: Generated 1794 derivative records from 40310 primary records. [2025-10-20T22:09:16.501764] [DEBUG] [LOCAL ] GBTG: Saving derivative data. [2025-10-20T22:09:16.527221] [DEBUG] [MAIN ] GBX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:16.531688] [INFO ] [MAIN ] GBX: Generated 1794 derivative records from 28567 primary records. [2025-10-20T22:09:16.531028] [DEBUG] [LOCAL ] GBTG: Derivative data saved (1794 records). [2025-10-20T22:09:16.531795] [DEBUG] [LOCAL ] GBX: Saving derivative data. [2025-10-20T22:09:16.532347] [INFO ] [MAIN ] GBTG: Successfully saved derivative data. [2025-10-20T22:09:16.534533] [INFO ] [MAIN ] Derivative Progress: 734/1877 symbols completed. GBTG took 1.2s. Estimated remaining: 7m 10s. Rate: 159.4 symbols/min [2025-10-20T22:09:16.549141] [DEBUG] [LOCAL ] GBX: Derivative data saved (1794 records). [2025-10-20T22:09:16.549202] [INFO ] [MAIN ] GBX: Successfully saved derivative data. [2025-10-20T22:09:16.551882] [INFO ] [MAIN ] Derivative Progress: 735/1877 symbols completed. GBX took 1.2s. Estimated remaining: 7m 9s. Rate: 159.6 symbols/min [2025-10-20T22:09:16.552426] [INFO ] [MAIN ] GCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:16.552891] [INFO ] [MAIN ] GCTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:16.552544] [INFO ] [MAIN ] GCV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:16.565832] [DEBUG] [LOCAL ] GCV: Loaded 7831 existing records from primary storage. [2025-10-20T22:09:16.566179] [INFO ] [MAIN ] GCV: Loaded 7831 primary data records. [2025-10-20T22:09:16.701036] [DEBUG] [LOCAL ] GCTS: Loaded 20245 existing records from primary storage. [2025-10-20T22:09:16.706966] [INFO ] [MAIN ] GCTS: Loaded 20245 primary data records. [2025-10-20T22:09:16.837919] [DEBUG] [LOCAL ] GCI: Loaded 43764 existing records from primary storage. [2025-10-20T22:09:16.857477] [INFO ] [MAIN ] GCI: Loaded 43764 primary data records. [2025-10-20T22:09:17.704113] [DEBUG] [MAIN ] GCV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:17.706594] [INFO ] [MAIN ] GCV: Generated 1794 derivative records from 7831 primary records. [2025-10-20T22:09:17.707087] [DEBUG] [LOCAL ] GCV: Saving derivative data. [2025-10-20T22:09:17.727263] [DEBUG] [LOCAL ] GCV: Derivative data saved (1794 records). [2025-10-20T22:09:17.732805] [INFO ] [MAIN ] GCV: Successfully saved derivative data. [2025-10-20T22:09:17.735552] [INFO ] [MAIN ] Derivative Progress: 736/1877 symbols completed. GCV took 1.2s. Estimated remaining: 7m 10s. Rate: 159.1 symbols/min [2025-10-20T22:09:17.735827] [INFO ] [MAIN ] GD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:17.749978] [DEBUG] [MAIN ] GCTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:17.750887] [INFO ] [MAIN ] GCTS: Generated 1794 derivative records from 20245 primary records. [2025-10-20T22:09:17.751280] [DEBUG] [LOCAL ] GCTS: Saving derivative data. [2025-10-20T22:09:17.783496] [DEBUG] [LOCAL ] GCTS: Derivative data saved (1794 records). [2025-10-20T22:09:17.781036] [DEBUG] [MAIN ] GCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:17.783599] [INFO ] [MAIN ] GCTS: Successfully saved derivative data. [2025-10-20T22:09:17.786484] [INFO ] [MAIN ] GCI: Generated 1794 derivative records from 43764 primary records. [2025-10-20T22:09:17.786558] [DEBUG] [LOCAL ] GCI: Saving derivative data. [2025-10-20T22:09:17.785790] [INFO ] [MAIN ] Derivative Progress: 737/1877 symbols completed. GCTS took 1.2s. Estimated remaining: 7m 9s. Rate: 159.3 symbols/min [2025-10-20T22:09:17.787406] [INFO ] [MAIN ] GDDY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:17.804192] [DEBUG] [LOCAL ] GCI: Derivative data saved (1794 records). [2025-10-20T22:09:17.804277] [INFO ] [MAIN ] GCI: Successfully saved derivative data. [2025-10-20T22:09:17.806355] [INFO ] [MAIN ] Derivative Progress: 738/1877 symbols completed. GCI took 1.3s. Estimated remaining: 7m 8s. Rate: 159.5 symbols/min [2025-10-20T22:09:17.806692] [INFO ] [MAIN ] GDIV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:17.826255] [DEBUG] [LOCAL ] GDIV: Loaded 3580 existing records from primary storage. [2025-10-20T22:09:17.826410] [INFO ] [MAIN ] GDIV: Loaded 3580 primary data records. [2025-10-20T22:09:17.959107] [DEBUG] [LOCAL ] GD: Loaded 46577 existing records from primary storage. [2025-10-20T22:09:17.970245] [INFO ] [MAIN ] GD: Loaded 46577 primary data records. [2025-10-20T22:09:18.276033] [DEBUG] [LOCAL ] GDDY: Loaded 49758 existing records from primary storage. [2025-10-20T22:09:18.282713] [INFO ] [MAIN ] GDDY: Loaded 49758 primary data records. [2025-10-20T22:09:18.841466] [DEBUG] [MAIN ] GDIV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:18.842635] [INFO ] [MAIN ] GDIV: Generated 1793 derivative records from 3580 primary records. [2025-10-20T22:09:18.843181] [DEBUG] [LOCAL ] GDIV: Saving derivative data. [2025-10-20T22:09:18.859670] [DEBUG] [LOCAL ] GDIV: Derivative data saved (1793 records). [2025-10-20T22:09:18.859972] [INFO ] [MAIN ] GDIV: Successfully saved derivative data. [2025-10-20T22:09:18.861913] [INFO ] [MAIN ] Derivative Progress: 739/1877 symbols completed. GDIV took 1.1s. Estimated remaining: 7m 9s. Rate: 159.1 symbols/min [2025-10-20T22:09:18.862227] [INFO ] [MAIN ] GDL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:18.882403] [DEBUG] [LOCAL ] GDL: Loaded 2525 existing records from primary storage. [2025-10-20T22:09:18.883057] [DEBUG] [MAIN ] GD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:18.883158] [INFO ] [MAIN ] GDL: Loaded 2525 primary data records. [2025-10-20T22:09:18.883822] [INFO ] [MAIN ] GD: Generated 1794 derivative records from 46577 primary records. [2025-10-20T22:09:18.921529] [DEBUG] [LOCAL ] GD: Saving derivative data. [2025-10-20T22:09:19.016493] [DEBUG] [MAIN ] GDDY: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:19.021496] [DEBUG] [LOCAL ] GD: Derivative data saved (1794 records). [2025-10-20T22:09:19.022928] [INFO ] [MAIN ] GDDY: Generated 1793 derivative records from 49758 primary records. [2025-10-20T22:09:19.023227] [DEBUG] [LOCAL ] GDDY: Saving derivative data. [2025-10-20T22:09:19.023323] [INFO ] [MAIN ] GD: Successfully saved derivative data. [2025-10-20T22:09:19.027268] [INFO ] [MAIN ] Derivative Progress: 740/1877 symbols completed. GD took 1.3s. Estimated remaining: 7m 8s. Rate: 159.2 symbols/min [2025-10-20T22:09:19.119140] [DEBUG] [LOCAL ] GDDY: Derivative data saved (1793 records). [2025-10-20T22:09:19.115827] [INFO ] [MAIN ] GDO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:19.204879] [INFO ] [MAIN ] GDDY: Successfully saved derivative data. [2025-10-20T22:09:19.241758] [INFO ] [MAIN ] Derivative Progress: 741/1877 symbols completed. GDDY took 1.5s. Estimated remaining: 7m 8s. Rate: 159.3 symbols/min [2025-10-20T22:09:19.242312] [INFO ] [MAIN ] GDOT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:19.255454] [DEBUG] [LOCAL ] GDO: Loaded 5840 existing records from primary storage. [2025-10-20T22:09:19.256039] [INFO ] [MAIN ] GDO: Loaded 5840 primary data records. [2025-10-20T22:09:19.605864] [DEBUG] [LOCAL ] GDOT: Loaded 37755 existing records from primary storage. [2025-10-20T22:09:19.617060] [INFO ] [MAIN ] GDOT: Loaded 37755 primary data records. [2025-10-20T22:09:19.645472] [DEBUG] [MAIN ] GDL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:19.666192] [INFO ] [MAIN ] GDL: Generated 1794 derivative records from 2525 primary records. [2025-10-20T22:09:19.666449] [DEBUG] [LOCAL ] GDL: Saving derivative data. [2025-10-20T22:09:19.763592] [DEBUG] [LOCAL ] GDL: Derivative data saved (1794 records). [2025-10-20T22:09:19.788034] [INFO ] [MAIN ] GDL: Successfully saved derivative data. [2025-10-20T22:09:19.790868] [INFO ] [MAIN ] Derivative Progress: 742/1877 symbols completed. GDL took 0.9s. Estimated remaining: 7m 8s. Rate: 159.2 symbols/min [2025-10-20T22:09:19.802430] [INFO ] [MAIN ] GDV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:20.092088] [DEBUG] [MAIN ] GDO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:20.093944] [INFO ] [MAIN ] GDO: Generated 1794 derivative records from 5840 primary records. [2025-10-20T22:09:20.094448] [DEBUG] [LOCAL ] GDO: Saving derivative data. [2025-10-20T22:09:20.115674] [DEBUG] [LOCAL ] GDV: Loaded 19418 existing records from primary storage. [2025-10-20T22:09:20.119756] [DEBUG] [LOCAL ] GDO: Derivative data saved (1794 records). [2025-10-20T22:09:20.120237] [INFO ] [MAIN ] GDO: Successfully saved derivative data. [2025-10-20T22:09:20.122336] [INFO ] [MAIN ] Derivative Progress: 743/1877 symbols completed. GDO took 1.0s. Estimated remaining: 7m 7s. Rate: 159.3 symbols/min [2025-10-20T22:09:20.122491] [INFO ] [MAIN ] GDV: Loaded 19418 primary data records. [2025-10-20T22:09:20.123326] [INFO ] [MAIN ] GE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:20.459275] [DEBUG] [MAIN ] GDOT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:20.461302] [INFO ] [MAIN ] GDOT: Generated 1794 derivative records from 37755 primary records. [2025-10-20T22:09:20.461965] [DEBUG] [LOCAL ] GDOT: Saving derivative data. [2025-10-20T22:09:20.500326] [DEBUG] [LOCAL ] GDOT: Derivative data saved (1794 records). [2025-10-20T22:09:20.503206] [INFO ] [MAIN ] GDOT: Successfully saved derivative data. [2025-10-20T22:09:20.506942] [INFO ] [MAIN ] Derivative Progress: 744/1877 symbols completed. GDOT took 1.3s. Estimated remaining: 7m 7s. Rate: 159.3 symbols/min [2025-10-20T22:09:20.507430] [INFO ] [MAIN ] GEF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:20.526608] [DEBUG] [LOCAL ] GE: Loaded 53698 existing records from primary storage. [2025-10-20T22:09:20.527301] [INFO ] [MAIN ] GE: Loaded 53698 primary data records. [2025-10-20T22:09:20.668763] [DEBUG] [LOCAL ] GEF: Loaded 20852 existing records from primary storage. [2025-10-20T22:09:20.692311] [INFO ] [MAIN ] GEF: Loaded 20852 primary data records. [2025-10-20T22:09:20.735811] [DEBUG] [MAIN ] GDV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:20.828573] [INFO ] [MAIN ] GDV: Generated 1794 derivative records from 19418 primary records. [2025-10-20T22:09:20.912600] [DEBUG] [LOCAL ] GDV: Saving derivative data. [2025-10-20T22:09:21.011565] [DEBUG] [LOCAL ] GDV: Derivative data saved (1794 records). [2025-10-20T22:09:21.023956] [INFO ] [MAIN ] GDV: Successfully saved derivative data. [2025-10-20T22:09:21.026330] [INFO ] [MAIN ] Derivative Progress: 745/1877 symbols completed. GDV took 1.2s. Estimated remaining: 7m 7s. Rate: 159.2 symbols/min [2025-10-20T22:09:21.079990] [INFO ] [MAIN ] GEL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:21.250409] [DEBUG] [MAIN ] GE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:21.260601] [INFO ] [MAIN ] GE: Generated 1794 derivative records from 53698 primary records. [2025-10-20T22:09:21.261321] [DEBUG] [LOCAL ] GE: Saving derivative data. [2025-10-20T22:09:21.280694] [DEBUG] [LOCAL ] GE: Derivative data saved (1794 records). [2025-10-20T22:09:21.293905] [INFO ] [MAIN ] GE: Successfully saved derivative data. [2025-10-20T22:09:21.298273] [INFO ] [MAIN ] Derivative Progress: 746/1877 symbols completed. GE took 1.2s. Estimated remaining: 7m 6s. Rate: 159.2 symbols/min [2025-10-20T22:09:21.298573] [INFO ] [MAIN ] GENI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:21.300519] [DEBUG] [LOCAL ] GEL: Loaded 31500 existing records from primary storage. [2025-10-20T22:09:21.300934] [INFO ] [MAIN ] GEL: Loaded 31500 primary data records. [2025-10-20T22:09:21.539413] [DEBUG] [MAIN ] GEF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:21.666014] [INFO ] [MAIN ] GEF: Generated 1794 derivative records from 20852 primary records. [2025-10-20T22:09:21.680046] [DEBUG] [LOCAL ] GEF: Saving derivative data. [2025-10-20T22:09:21.700050] [DEBUG] [LOCAL ] GEF: Derivative data saved (1794 records). [2025-10-20T22:09:21.718840] [DEBUG] [LOCAL ] GENI: Loaded 52943 existing records from primary storage. [2025-10-20T22:09:21.719275] [INFO ] [MAIN ] GENI: Loaded 52943 primary data records. [2025-10-20T22:09:21.717538] [INFO ] [MAIN ] GEF: Successfully saved derivative data. [2025-10-20T22:09:21.722740] [INFO ] [MAIN ] Derivative Progress: 747/1877 symbols completed. GEF took 1.2s. Estimated remaining: 7m 6s. Rate: 159.2 symbols/min [2025-10-20T22:09:21.760160] [INFO ] [MAIN ] GEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:22.089673] [DEBUG] [MAIN ] GEL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:22.090979] [INFO ] [MAIN ] GEL: Generated 1794 derivative records from 31500 primary records. [2025-10-20T22:09:22.091101] [DEBUG] [LOCAL ] GEL: Saving derivative data. [2025-10-20T22:09:22.128622] [DEBUG] [LOCAL ] GEL: Derivative data saved (1794 records). [2025-10-20T22:09:22.128839] [INFO ] [MAIN ] GEL: Successfully saved derivative data. [2025-10-20T22:09:22.130971] [INFO ] [MAIN ] Derivative Progress: 748/1877 symbols completed. GEL took 1.1s. Estimated remaining: 7m 6s. Rate: 159.2 symbols/min [2025-10-20T22:09:22.132205] [DEBUG] [LOCAL ] GEO: Loaded 51753 existing records from primary storage. [2025-10-20T22:09:22.133637] [INFO ] [MAIN ] GEO: Loaded 51753 primary data records. [2025-10-20T22:09:22.442229] [DEBUG] [MAIN ] GENI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:22.443086] [INFO ] [MAIN ] GENI: Generated 1794 derivative records from 52943 primary records. [2025-10-20T22:09:22.443238] [DEBUG] [LOCAL ] GENI: Saving derivative data. [2025-10-20T22:09:22.461448] [DEBUG] [LOCAL ] GENI: Derivative data saved (1794 records). [2025-10-20T22:09:22.461566] [INFO ] [MAIN ] GENI: Successfully saved derivative data. [2025-10-20T22:09:22.464270] [INFO ] [MAIN ] Derivative Progress: 749/1877 symbols completed. GENI took 1.2s. Estimated remaining: 7m 5s. Rate: 159.2 symbols/min [2025-10-20T22:09:22.483778] [DEBUG] [MAIN ] GEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:22.484370] [INFO ] [MAIN ] GEO: Generated 1794 derivative records from 51753 primary records. [2025-10-20T22:09:22.484427] [DEBUG] [LOCAL ] GEO: Saving derivative data. [2025-10-20T22:09:22.499798] [DEBUG] [LOCAL ] GEO: Derivative data saved (1794 records). [2025-10-20T22:09:22.499855] [INFO ] [MAIN ] GEO: Successfully saved derivative data. [2025-10-20T22:09:22.502205] [INFO ] [MAIN ] Derivative Progress: 750/1877 symbols completed. GEO took 0.7s. Estimated remaining: 7m 4s. Rate: 159.4 symbols/min [2025-10-20T22:09:22.502602] [INFO ] [MAIN ] GES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:22.502756] [INFO ] [MAIN ] GETY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:22.503568] [INFO ] [MAIN ] GEV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:22.574544] [DEBUG] [LOCAL ] GES: Loaded 38154 existing records from primary storage. [2025-10-20T22:09:22.575787] [DEBUG] [LOCAL ] GETY: Loaded 35268 existing records from primary storage. [2025-10-20T22:09:22.576102] [INFO ] [MAIN ] GES: Loaded 38154 primary data records. [2025-10-20T22:09:22.576219] [INFO ] [MAIN ] GETY: Loaded 35268 primary data records. [2025-10-20T22:09:22.776581] [DEBUG] [LOCAL ] GEV: Loaded 52992 existing records from primary storage. [2025-10-20T22:09:22.811024] [INFO ] [MAIN ] GEV: Loaded 52992 primary data records. [2025-10-20T22:09:23.621281] [DEBUG] [MAIN ] GES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:23.620838] [DEBUG] [MAIN ] GETY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:23.622332] [INFO ] [MAIN ] GETY: Generated 1794 derivative records from 35268 primary records. [2025-10-20T22:09:23.622777] [DEBUG] [LOCAL ] GETY: Saving derivative data. [2025-10-20T22:09:23.623701] [INFO ] [MAIN ] GES: Generated 1794 derivative records from 38154 primary records. [2025-10-20T22:09:23.625101] [DEBUG] [LOCAL ] GES: Saving derivative data. [2025-10-20T22:09:23.661245] [DEBUG] [LOCAL ] GETY: Derivative data saved (1794 records). [2025-10-20T22:09:23.665726] [INFO ] [MAIN ] GETY: Successfully saved derivative data. [2025-10-20T22:09:23.671973] [INFO ] [MAIN ] Derivative Progress: 751/1877 symbols completed. GETY took 1.2s. Estimated remaining: 7m 5s. Rate: 159.0 symbols/min [2025-10-20T22:09:23.669222] [DEBUG] [LOCAL ] GES: Derivative data saved (1794 records). [2025-10-20T22:09:23.672224] [INFO ] [MAIN ] GF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:23.672675] [INFO ] [MAIN ] GES: Successfully saved derivative data. [2025-10-20T22:09:23.677308] [INFO ] [MAIN ] Derivative Progress: 752/1877 symbols completed. GES took 1.2s. Estimated remaining: 7m 4s. Rate: 159.2 symbols/min [2025-10-20T22:09:23.677694] [INFO ] [MAIN ] GFF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:23.685433] [DEBUG] [LOCAL ] GF: Loaded 2662 existing records from primary storage. [2025-10-20T22:09:23.685900] [INFO ] [MAIN ] GF: Loaded 2662 primary data records. [2025-10-20T22:09:23.758845] [DEBUG] [MAIN ] GEV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:23.795134] [INFO ] [MAIN ] GEV: Generated 1794 derivative records from 52992 primary records. [2025-10-20T22:09:23.800734] [DEBUG] [LOCAL ] GEV: Saving derivative data. [2025-10-20T22:09:23.916895] [DEBUG] [LOCAL ] GEV: Derivative data saved (1794 records). [2025-10-20T22:09:23.929029] [DEBUG] [LOCAL ] GFF: Loaded 29664 existing records from primary storage. [2025-10-20T22:09:23.959729] [INFO ] [MAIN ] GFF: Loaded 29664 primary data records. [2025-10-20T22:09:23.959879] [INFO ] [MAIN ] GEV: Successfully saved derivative data. [2025-10-20T22:09:24.009203] [INFO ] [MAIN ] Derivative Progress: 753/1877 symbols completed. GEV took 1.5s. Estimated remaining: 7m 4s. Rate: 159.2 symbols/min [2025-10-20T22:09:24.244784] [INFO ] [MAIN ] GFI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:24.642844] [DEBUG] [LOCAL ] GFI: Loaded 53116 existing records from primary storage. [2025-10-20T22:09:24.644058] [INFO ] [MAIN ] GFI: Loaded 53116 primary data records. [2025-10-20T22:09:24.722260] [DEBUG] [MAIN ] GF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:24.731951] [INFO ] [MAIN ] GF: Generated 1794 derivative records from 2662 primary records. [2025-10-20T22:09:24.773323] [DEBUG] [LOCAL ] GF: Saving derivative data. [2025-10-20T22:09:24.912072] [DEBUG] [LOCAL ] GF: Derivative data saved (1794 records). [2025-10-20T22:09:24.913268] [INFO ] [MAIN ] GF: Successfully saved derivative data. [2025-10-20T22:09:24.915844] [INFO ] [MAIN ] Derivative Progress: 754/1877 symbols completed. GF took 1.2s. Estimated remaining: 7m 4s. Rate: 158.9 symbols/min [2025-10-20T22:09:25.032718] [INFO ] [MAIN ] GFL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:25.048329] [DEBUG] [MAIN ] GFF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:25.049166] [INFO ] [MAIN ] GFF: Generated 1794 derivative records from 29664 primary records. [2025-10-20T22:09:25.049407] [DEBUG] [LOCAL ] GFF: Saving derivative data. [2025-10-20T22:09:25.066940] [DEBUG] [LOCAL ] GFF: Derivative data saved (1794 records). [2025-10-20T22:09:25.067292] [INFO ] [MAIN ] GFF: Successfully saved derivative data. [2025-10-20T22:09:25.069279] [INFO ] [MAIN ] Derivative Progress: 755/1877 symbols completed. GFF took 1.4s. Estimated remaining: 7m 3s. Rate: 159.0 symbols/min [2025-10-20T22:09:25.070043] [INFO ] [MAIN ] GFR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:25.081677] [DEBUG] [LOCAL ] GFR: Loaded 7510 existing records from primary storage. [2025-10-20T22:09:25.081956] [INFO ] [MAIN ] GFR: Loaded 7510 primary data records. [2025-10-20T22:09:25.211787] [DEBUG] [MAIN ] GFI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:25.218458] [INFO ] [MAIN ] GFI: Generated 1794 derivative records from 53116 primary records. [2025-10-20T22:09:25.234930] [DEBUG] [LOCAL ] GFI: Saving derivative data. [2025-10-20T22:09:25.426729] [DEBUG] [LOCAL ] GFL: Loaded 47569 existing records from primary storage. [2025-10-20T22:09:25.444591] [DEBUG] [LOCAL ] GFI: Derivative data saved (1794 records). [2025-10-20T22:09:25.444969] [INFO ] [MAIN ] GFI: Successfully saved derivative data. [2025-10-20T22:09:25.446961] [INFO ] [MAIN ] Derivative Progress: 756/1877 symbols completed. GFI took 1.2s. Estimated remaining: 7m 3s. Rate: 159.0 symbols/min [2025-10-20T22:09:25.444790] [INFO ] [MAIN ] GFL: Loaded 47569 primary data records. [2025-10-20T22:09:25.447328] [INFO ] [MAIN ] GGB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:25.805844] [DEBUG] [LOCAL ] GGB: Loaded 51536 existing records from primary storage. [2025-10-20T22:09:25.806849] [INFO ] [MAIN ] GGB: Loaded 51536 primary data records. [2025-10-20T22:09:25.878375] [DEBUG] [MAIN ] GFR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:25.908308] [INFO ] [MAIN ] GFR: Generated 1794 derivative records from 7510 primary records. [2025-10-20T22:09:25.913580] [DEBUG] [LOCAL ] GFR: Saving derivative data. [2025-10-20T22:09:25.988918] [DEBUG] [LOCAL ] GFR: Derivative data saved (1794 records). [2025-10-20T22:09:26.014743] [INFO ] [MAIN ] GFR: Successfully saved derivative data. [2025-10-20T22:09:26.016849] [INFO ] [MAIN ] Derivative Progress: 757/1877 symbols completed. GFR took 0.9s. Estimated remaining: 7m 3s. Rate: 158.9 symbols/min [2025-10-20T22:09:26.020638] [INFO ] [MAIN ] GGG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:26.151233] [DEBUG] [MAIN ] GFL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:26.152819] [INFO ] [MAIN ] GFL: Generated 1794 derivative records from 47569 primary records. [2025-10-20T22:09:26.153492] [DEBUG] [LOCAL ] GFL: Saving derivative data. [2025-10-20T22:09:26.170056] [DEBUG] [LOCAL ] GFL: Derivative data saved (1794 records). [2025-10-20T22:09:26.179479] [INFO ] [MAIN ] GFL: Successfully saved derivative data. [2025-10-20T22:09:26.181705] [INFO ] [MAIN ] Derivative Progress: 758/1877 symbols completed. GFL took 1.1s. Estimated remaining: 7m 2s. Rate: 159.0 symbols/min [2025-10-20T22:09:26.181877] [INFO ] [MAIN ] GGT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:26.204347] [DEBUG] [LOCAL ] GGG: Loaded 40585 existing records from primary storage. [2025-10-20T22:09:26.204751] [INFO ] [MAIN ] GGG: Loaded 40585 primary data records. [2025-10-20T22:09:26.271728] [DEBUG] [MAIN ] GGB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:26.286001] [INFO ] [MAIN ] GGB: Generated 1794 derivative records from 51536 primary records. [2025-10-20T22:09:26.291478] [DEBUG] [LOCAL ] GGT: Loaded 14586 existing records from primary storage. [2025-10-20T22:09:26.303332] [INFO ] [MAIN ] GGT: Loaded 14586 primary data records. [2025-10-20T22:09:26.298135] [DEBUG] [LOCAL ] GGB: Saving derivative data. [2025-10-20T22:09:26.430909] [DEBUG] [LOCAL ] GGB: Derivative data saved (1794 records). [2025-10-20T22:09:26.457773] [INFO ] [MAIN ] GGB: Successfully saved derivative data. [2025-10-20T22:09:26.493972] [INFO ] [MAIN ] Derivative Progress: 759/1877 symbols completed. GGB took 1.0s. Estimated remaining: 7m 2s. Rate: 159.1 symbols/min [2025-10-20T22:09:26.672333] [INFO ] [MAIN ] GGZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:26.752663] [DEBUG] [LOCAL ] GGZ: Loaded 2412 existing records from primary storage. [2025-10-20T22:09:26.753130] [INFO ] [MAIN ] GGZ: Loaded 2412 primary data records. [2025-10-20T22:09:27.195661] [DEBUG] [MAIN ] GGT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:27.201697] [INFO ] [MAIN ] GGT: Generated 1794 derivative records from 14586 primary records. [2025-10-20T22:09:27.202414] [DEBUG] [MAIN ] GGG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:27.203144] [INFO ] [MAIN ] GGG: Generated 1794 derivative records from 40585 primary records. [2025-10-20T22:09:27.202165] [DEBUG] [LOCAL ] GGT: Saving derivative data. [2025-10-20T22:09:27.204031] [DEBUG] [LOCAL ] GGG: Saving derivative data. [2025-10-20T22:09:27.233287] [DEBUG] [LOCAL ] GGT: Derivative data saved (1794 records). [2025-10-20T22:09:27.234478] [DEBUG] [LOCAL ] GGG: Derivative data saved (1794 records). [2025-10-20T22:09:27.235424] [INFO ] [MAIN ] GGG: Successfully saved derivative data. [2025-10-20T22:09:27.237454] [INFO ] [MAIN ] Derivative Progress: 760/1877 symbols completed. GGG took 1.2s. Estimated remaining: 7m 2s. Rate: 158.9 symbols/min [2025-10-20T22:09:27.233603] [INFO ] [MAIN ] GGT: Successfully saved derivative data. [2025-10-20T22:09:27.241116] [INFO ] [MAIN ] Derivative Progress: 761/1877 symbols completed. GGT took 1.1s. Estimated remaining: 7m 1s. Rate: 159.1 symbols/min [2025-10-20T22:09:27.238450] [INFO ] [MAIN ] GHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:27.241456] [INFO ] [MAIN ] GHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:27.254866] [DEBUG] [MAIN ] GGZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:27.255531] [INFO ] [MAIN ] GGZ: Generated 1794 derivative records from 2412 primary records. [2025-10-20T22:09:27.256381] [DEBUG] [LOCAL ] GGZ: Saving derivative data. [2025-10-20T22:09:27.260460] [DEBUG] [LOCAL ] GHC: Loaded 2956 existing records from primary storage. [2025-10-20T22:09:27.266068] [INFO ] [MAIN ] GHC: Loaded 2956 primary data records. [2025-10-20T22:09:27.338089] [DEBUG] [LOCAL ] GHG: Loaded 3282 existing records from primary storage. [2025-10-20T22:09:27.343329] [INFO ] [MAIN ] GHG: Loaded 3282 primary data records. [2025-10-20T22:09:27.349675] [DEBUG] [LOCAL ] GGZ: Derivative data saved (1794 records). [2025-10-20T22:09:27.415213] [INFO ] [MAIN ] GGZ: Successfully saved derivative data. [2025-10-20T22:09:27.446472] [INFO ] [MAIN ] Derivative Progress: 762/1877 symbols completed. GGZ took 0.8s. Estimated remaining: 7m 0s. Rate: 159.2 symbols/min [2025-10-20T22:09:27.468420] [INFO ] [MAIN ] GHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:27.876728] [DEBUG] [LOCAL ] GHI: Loaded 7558 existing records from primary storage. [2025-10-20T22:09:27.877044] [INFO ] [MAIN ] GHI: Loaded 7558 primary data records. [2025-10-20T22:09:28.249793] [DEBUG] [MAIN ] GHG: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:28.250709] [INFO ] [MAIN ] GHG: Generated 1793 derivative records from 3282 primary records. [2025-10-20T22:09:28.250803] [DEBUG] [LOCAL ] GHG: Saving derivative data. [2025-10-20T22:09:28.272023] [DEBUG] [MAIN ] GHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:28.275615] [INFO ] [MAIN ] GHC: Generated 1794 derivative records from 2956 primary records. [2025-10-20T22:09:28.279201] [DEBUG] [LOCAL ] GHG: Derivative data saved (1793 records). [2025-10-20T22:09:28.279433] [INFO ] [MAIN ] GHG: Successfully saved derivative data. [2025-10-20T22:09:28.281537] [INFO ] [MAIN ] Derivative Progress: 763/1877 symbols completed. GHG took 1.0s. Estimated remaining: 7m 1s. Rate: 158.9 symbols/min [2025-10-20T22:09:28.279349] [DEBUG] [LOCAL ] GHC: Saving derivative data. [2025-10-20T22:09:28.297365] [DEBUG] [LOCAL ] GHC: Derivative data saved (1794 records). [2025-10-20T22:09:28.297497] [INFO ] [MAIN ] GHC: Successfully saved derivative data. [2025-10-20T22:09:28.299558] [INFO ] [MAIN ] Derivative Progress: 764/1877 symbols completed. GHC took 1.1s. Estimated remaining: 6m 60s. Rate: 159.1 symbols/min [2025-10-20T22:09:28.315354] [DEBUG] [MAIN ] GHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:28.316037] [INFO ] [MAIN ] GHI: Generated 1794 derivative records from 7558 primary records. [2025-10-20T22:09:28.316097] [DEBUG] [LOCAL ] GHI: Saving derivative data. [2025-10-20T22:09:28.330782] [DEBUG] [LOCAL ] GHI: Derivative data saved (1794 records). [2025-10-20T22:09:28.330880] [INFO ] [MAIN ] GHI: Successfully saved derivative data. [2025-10-20T22:09:28.333016] [INFO ] [MAIN ] Derivative Progress: 765/1877 symbols completed. GHI took 0.9s. Estimated remaining: 6m 59s. Rate: 159.3 symbols/min [2025-10-20T22:09:28.333438] [INFO ] [MAIN ] GHM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:28.333381] [INFO ] [MAIN ] GHLD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:28.333590] [INFO ] [MAIN ] GHY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:28.342262] [DEBUG] [LOCAL ] GHLD: Loaded 4301 existing records from primary storage. [2025-10-20T22:09:28.342424] [INFO ] [MAIN ] GHLD: Loaded 4301 primary data records. [2025-10-20T22:09:28.447916] [DEBUG] [LOCAL ] GHM: Loaded 14369 existing records from primary storage. [2025-10-20T22:09:28.471510] [DEBUG] [LOCAL ] GHY: Loaded 20931 existing records from primary storage. [2025-10-20T22:09:28.465729] [INFO ] [MAIN ] GHM: Loaded 14369 primary data records. [2025-10-20T22:09:28.504516] [INFO ] [MAIN ] GHY: Loaded 20931 primary data records. [2025-10-20T22:09:29.268708] [DEBUG] [MAIN ] GHLD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:29.269731] [INFO ] [MAIN ] GHLD: Generated 1794 derivative records from 4301 primary records. [2025-10-20T22:09:29.269979] [DEBUG] [LOCAL ] GHLD: Saving derivative data. [2025-10-20T22:09:29.284465] [DEBUG] [LOCAL ] GHLD: Derivative data saved (1794 records). [2025-10-20T22:09:29.284820] [INFO ] [MAIN ] GHLD: Successfully saved derivative data. [2025-10-20T22:09:29.286938] [INFO ] [MAIN ] Derivative Progress: 766/1877 symbols completed. GHLD took 1.0s. Estimated remaining: 6m 59s. Rate: 159.0 symbols/min [2025-10-20T22:09:29.287324] [INFO ] [MAIN ] GIB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:29.309909] [DEBUG] [MAIN ] GHY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:29.310703] [INFO ] [MAIN ] GHY: Generated 1794 derivative records from 20931 primary records. [2025-10-20T22:09:29.310874] [DEBUG] [LOCAL ] GHY: Saving derivative data. [2025-10-20T22:09:29.324095] [DEBUG] [MAIN ] GHM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:29.324912] [INFO ] [MAIN ] GHM: Generated 1794 derivative records from 14369 primary records. [2025-10-20T22:09:29.337598] [DEBUG] [LOCAL ] GHM: Saving derivative data. [2025-10-20T22:09:29.339592] [DEBUG] [LOCAL ] GHY: Derivative data saved (1794 records). [2025-10-20T22:09:29.340296] [INFO ] [MAIN ] GHY: Successfully saved derivative data. [2025-10-20T22:09:29.342212] [INFO ] [MAIN ] Derivative Progress: 767/1877 symbols completed. GHY took 1.0s. Estimated remaining: 6m 58s. Rate: 159.2 symbols/min [2025-10-20T22:09:29.342684] [INFO ] [MAIN ] GIC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:29.380087] [DEBUG] [LOCAL ] GHM: Derivative data saved (1794 records). [2025-10-20T22:09:29.380361] [INFO ] [MAIN ] GHM: Successfully saved derivative data. [2025-10-20T22:09:29.382866] [INFO ] [MAIN ] Derivative Progress: 768/1877 symbols completed. GHM took 1.0s. Estimated remaining: 6m 58s. Rate: 159.3 symbols/min [2025-10-20T22:09:29.397007] [INFO ] [MAIN ] GIL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:29.397690] [DEBUG] [LOCAL ] GIB: Loaded 31496 existing records from primary storage. [2025-10-20T22:09:29.397803] [INFO ] [MAIN ] GIB: Loaded 31496 primary data records. [2025-10-20T22:09:29.450862] [DEBUG] [LOCAL ] GIC: Loaded 13388 existing records from primary storage. [2025-10-20T22:09:29.472018] [INFO ] [MAIN ] GIC: Loaded 13388 primary data records. [2025-10-20T22:09:30.059704] [DEBUG] [LOCAL ] GIL: Loaded 43989 existing records from primary storage. [2025-10-20T22:09:30.060290] [INFO ] [MAIN ] GIL: Loaded 43989 primary data records. [2025-10-20T22:09:30.381382] [DEBUG] [MAIN ] GIB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:30.382185] [INFO ] [MAIN ] GIB: Generated 1794 derivative records from 31496 primary records. [2025-10-20T22:09:30.382354] [DEBUG] [LOCAL ] GIB: Saving derivative data. [2025-10-20T22:09:30.400523] [DEBUG] [LOCAL ] GIB: Derivative data saved (1794 records). [2025-10-20T22:09:30.402395] [INFO ] [MAIN ] GIB: Successfully saved derivative data. [2025-10-20T22:09:30.405135] [INFO ] [MAIN ] Derivative Progress: 769/1877 symbols completed. GIB took 1.1s. Estimated remaining: 6m 58s. Rate: 159.0 symbols/min [2025-10-20T22:09:30.405926] [INFO ] [MAIN ] GKOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:30.420516] [DEBUG] [MAIN ] GIC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:30.421297] [INFO ] [MAIN ] GIC: Generated 1794 derivative records from 13388 primary records. [2025-10-20T22:09:30.421391] [DEBUG] [LOCAL ] GIC: Saving derivative data. [2025-10-20T22:09:30.437872] [DEBUG] [LOCAL ] GIC: Derivative data saved (1794 records). [2025-10-20T22:09:30.438023] [INFO ] [MAIN ] GIC: Successfully saved derivative data. [2025-10-20T22:09:30.439983] [INFO ] [MAIN ] Derivative Progress: 770/1877 symbols completed. GIC took 1.1s. Estimated remaining: 6m 57s. Rate: 159.2 symbols/min [2025-10-20T22:09:30.440305] [INFO ] [MAIN ] GL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:30.469470] [DEBUG] [MAIN ] GIL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:30.470184] [INFO ] [MAIN ] GIL: Generated 1794 derivative records from 43989 primary records. [2025-10-20T22:09:30.470310] [DEBUG] [LOCAL ] GIL: Saving derivative data. [2025-10-20T22:09:30.487130] [DEBUG] [LOCAL ] GIL: Derivative data saved (1794 records). [2025-10-20T22:09:30.487355] [INFO ] [MAIN ] GIL: Successfully saved derivative data. [2025-10-20T22:09:30.489245] [INFO ] [MAIN ] Derivative Progress: 771/1877 symbols completed. GIL took 1.1s. Estimated remaining: 6m 56s. Rate: 159.4 symbols/min [2025-10-20T22:09:30.489496] [INFO ] [MAIN ] GLOB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:30.511915] [DEBUG] [LOCAL ] GKOS: Loaded 42502 existing records from primary storage. [2025-10-20T22:09:30.512068] [INFO ] [MAIN ] GKOS: Loaded 42502 primary data records. [2025-10-20T22:09:30.632382] [DEBUG] [LOCAL ] GL: Loaded 37483 existing records from primary storage. [2025-10-20T22:09:30.633022] [INFO ] [MAIN ] GL: Loaded 37483 primary data records. [2025-10-20T22:09:31.162526] [DEBUG] [LOCAL ] GLOB: Loaded 48058 existing records from primary storage. [2025-10-20T22:09:31.164012] [INFO ] [MAIN ] GLOB: Loaded 48058 primary data records. [2025-10-20T22:09:31.489529] [DEBUG] [MAIN ] GKOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:31.490787] [INFO ] [MAIN ] GKOS: Generated 1794 derivative records from 42502 primary records. [2025-10-20T22:09:31.491498] [DEBUG] [LOCAL ] GKOS: Saving derivative data. [2025-10-20T22:09:31.512163] [DEBUG] [LOCAL ] GKOS: Derivative data saved (1794 records). [2025-10-20T22:09:31.526163] [INFO ] [MAIN ] GKOS: Successfully saved derivative data. [2025-10-20T22:09:31.526047] [DEBUG] [MAIN ] GL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:31.528464] [INFO ] [MAIN ] Derivative Progress: 772/1877 symbols completed. GKOS took 1.1s. Estimated remaining: 6m 57s. Rate: 159.0 symbols/min [2025-10-20T22:09:31.529468] [INFO ] [MAIN ] GL: Generated 1794 derivative records from 37483 primary records. [2025-10-20T22:09:31.529932] [DEBUG] [LOCAL ] GL: Saving derivative data. [2025-10-20T22:09:31.531186] [INFO ] [MAIN ] GLP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:31.550326] [DEBUG] [LOCAL ] GL: Derivative data saved (1794 records). [2025-10-20T22:09:31.550677] [INFO ] [MAIN ] GL: Successfully saved derivative data. [2025-10-20T22:09:31.555829] [INFO ] [MAIN ] Derivative Progress: 773/1877 symbols completed. GL took 1.1s. Estimated remaining: 6m 56s. Rate: 159.2 symbols/min [2025-10-20T22:09:31.556342] [INFO ] [MAIN ] GLW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:31.562874] [DEBUG] [LOCAL ] GLP: Loaded 9318 existing records from primary storage. [2025-10-20T22:09:31.563173] [INFO ] [MAIN ] GLP: Loaded 9318 primary data records. [2025-10-20T22:09:31.694968] [DEBUG] [MAIN ] GLOB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:31.735413] [INFO ] [MAIN ] GLOB: Generated 1794 derivative records from 48058 primary records. [2025-10-20T22:09:31.753003] [DEBUG] [LOCAL ] GLOB: Saving derivative data. [2025-10-20T22:09:31.928480] [DEBUG] [LOCAL ] GLOB: Derivative data saved (1794 records). [2025-10-20T22:09:31.928686] [INFO ] [MAIN ] GLOB: Successfully saved derivative data. [2025-10-20T22:09:31.930674] [INFO ] [MAIN ] Derivative Progress: 774/1877 symbols completed. GLOB took 1.4s. Estimated remaining: 6m 56s. Rate: 159.2 symbols/min [2025-10-20T22:09:31.930985] [INFO ] [MAIN ] GM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:31.951906] [DEBUG] [LOCAL ] GLW: Loaded 53635 existing records from primary storage. [2025-10-20T22:09:31.952129] [INFO ] [MAIN ] GLW: Loaded 53635 primary data records. [2025-10-20T22:09:32.314236] [DEBUG] [MAIN ] GLP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:32.316916] [INFO ] [MAIN ] GLP: Generated 1794 derivative records from 9318 primary records. [2025-10-20T22:09:32.317359] [DEBUG] [LOCAL ] GLP: Saving derivative data. [2025-10-20T22:09:32.352669] [DEBUG] [LOCAL ] GLP: Derivative data saved (1794 records). [2025-10-20T22:09:32.355412] [INFO ] [MAIN ] GLP: Successfully saved derivative data. [2025-10-20T22:09:32.354912] [DEBUG] [LOCAL ] GM: Loaded 53729 existing records from primary storage. [2025-10-20T22:09:32.358615] [INFO ] [MAIN ] Derivative Progress: 775/1877 symbols completed. GLP took 0.8s. Estimated remaining: 6m 55s. Rate: 159.2 symbols/min [2025-10-20T22:09:32.359020] [INFO ] [MAIN ] GM: Loaded 53729 primary data records. [2025-10-20T22:09:32.359249] [INFO ] [MAIN ] GME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:32.720760] [DEBUG] [MAIN ] GLW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:32.737413] [INFO ] [MAIN ] GLW: Generated 1794 derivative records from 53635 primary records. [2025-10-20T22:09:32.738399] [DEBUG] [LOCAL ] GLW: Saving derivative data. [2025-10-20T22:09:32.740246] [DEBUG] [LOCAL ] GME: Loaded 53781 existing records from primary storage. [2025-10-20T22:09:32.740747] [INFO ] [MAIN ] GME: Loaded 53781 primary data records. [2025-10-20T22:09:32.823748] [DEBUG] [LOCAL ] GLW: Derivative data saved (1794 records). [2025-10-20T22:09:32.835639] [INFO ] [MAIN ] GLW: Successfully saved derivative data. [2025-10-20T22:09:32.837848] [INFO ] [MAIN ] Derivative Progress: 776/1877 symbols completed. GLW took 1.3s. Estimated remaining: 6m 55s. Rate: 159.1 symbols/min [2025-10-20T22:09:32.838147] [INFO ] [MAIN ] GMED: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:33.097676] [DEBUG] [MAIN ] GM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:33.099410] [INFO ] [MAIN ] GM: Generated 1794 derivative records from 53729 primary records. [2025-10-20T22:09:33.099919] [DEBUG] [LOCAL ] GM: Saving derivative data. [2025-10-20T22:09:33.138948] [DEBUG] [LOCAL ] GMED: Loaded 48170 existing records from primary storage. [2025-10-20T22:09:33.140013] [DEBUG] [LOCAL ] GM: Derivative data saved (1794 records). [2025-10-20T22:09:33.140231] [INFO ] [MAIN ] GM: Successfully saved derivative data. [2025-10-20T22:09:33.142497] [INFO ] [MAIN ] Derivative Progress: 777/1877 symbols completed. GM took 1.2s. Estimated remaining: 6m 55s. Rate: 159.1 symbols/min [2025-10-20T22:09:33.140281] [INFO ] [MAIN ] GMED: Loaded 48170 primary data records. [2025-10-20T22:09:33.142809] [INFO ] [MAIN ] GMRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:33.384021] [DEBUG] [MAIN ] GME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:33.480808] [INFO ] [MAIN ] GME: Generated 1794 derivative records from 53781 primary records. [2025-10-20T22:09:33.482522] [DEBUG] [LOCAL ] GMRE: Loaded 36823 existing records from primary storage. [2025-10-20T22:09:33.482691] [INFO ] [MAIN ] GMRE: Loaded 36823 primary data records. [2025-10-20T22:09:33.481590] [DEBUG] [LOCAL ] GME: Saving derivative data. [2025-10-20T22:09:33.589340] [DEBUG] [LOCAL ] GME: Derivative data saved (1794 records). [2025-10-20T22:09:33.594631] [INFO ] [MAIN ] GME: Successfully saved derivative data. [2025-10-20T22:09:33.613556] [INFO ] [MAIN ] Derivative Progress: 778/1877 symbols completed. GME took 1.3s. Estimated remaining: 6m 54s. Rate: 159.1 symbols/min [2025-10-20T22:09:33.831965] [DEBUG] [MAIN ] GMED: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:33.833155] [INFO ] [MAIN ] GMED: Generated 1794 derivative records from 48170 primary records. [2025-10-20T22:09:33.833269] [DEBUG] [LOCAL ] GMED: Saving derivative data. [2025-10-20T22:09:33.850999] [DEBUG] [LOCAL ] GMED: Derivative data saved (1794 records). [2025-10-20T22:09:33.851131] [INFO ] [MAIN ] GMED: Successfully saved derivative data. [2025-10-20T22:09:33.853299] [INFO ] [MAIN ] Derivative Progress: 779/1877 symbols completed. GMED took 1.0s. Estimated remaining: 6m 54s. Rate: 159.2 symbols/min [2025-10-20T22:09:33.870805] [DEBUG] [MAIN ] GMRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:33.871370] [INFO ] [MAIN ] GMRE: Generated 1794 derivative records from 36823 primary records. [2025-10-20T22:09:33.871427] [DEBUG] [LOCAL ] GMRE: Saving derivative data. [2025-10-20T22:09:33.886050] [DEBUG] [LOCAL ] GMRE: Derivative data saved (1794 records). [2025-10-20T22:09:33.886119] [INFO ] [MAIN ] GMRE: Successfully saved derivative data. [2025-10-20T22:09:33.888044] [INFO ] [MAIN ] Derivative Progress: 780/1877 symbols completed. GMRE took 0.7s. Estimated remaining: 6m 53s. Rate: 159.3 symbols/min [2025-10-20T22:09:33.888491] [INFO ] [MAIN ] GNL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:33.888543] [INFO ] [MAIN ] GNK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:33.888444] [INFO ] [MAIN ] GNE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:33.911186] [DEBUG] [LOCAL ] GNE: Loaded 18091 existing records from primary storage. [2025-10-20T22:09:33.920450] [INFO ] [MAIN ] GNE: Loaded 18091 primary data records. [2025-10-20T22:09:34.049385] [DEBUG] [LOCAL ] GNK: Loaded 39773 existing records from primary storage. [2025-10-20T22:09:34.060615] [INFO ] [MAIN ] GNK: Loaded 39773 primary data records. [2025-10-20T22:09:34.197588] [DEBUG] [LOCAL ] GNL: Loaded 46630 existing records from primary storage. [2025-10-20T22:09:34.382597] [INFO ] [MAIN ] GNL: Loaded 46630 primary data records. [2025-10-20T22:09:34.921245] [DEBUG] [MAIN ] GNE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:34.923877] [INFO ] [MAIN ] GNE: Generated 1794 derivative records from 18091 primary records. [2025-10-20T22:09:34.924105] [DEBUG] [LOCAL ] GNE: Saving derivative data. [2025-10-20T22:09:34.952915] [DEBUG] [LOCAL ] GNE: Derivative data saved (1794 records). [2025-10-20T22:09:34.953047] [INFO ] [MAIN ] GNE: Successfully saved derivative data. [2025-10-20T22:09:34.955294] [INFO ] [MAIN ] Derivative Progress: 781/1877 symbols completed. GNE took 1.1s. Estimated remaining: 6m 54s. Rate: 159.0 symbols/min [2025-10-20T22:09:34.955814] [INFO ] [MAIN ] GNRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:34.955626] [DEBUG] [MAIN ] GNK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:34.957216] [INFO ] [MAIN ] GNK: Generated 1794 derivative records from 39773 primary records. [2025-10-20T22:09:34.957362] [DEBUG] [LOCAL ] GNK: Saving derivative data. [2025-10-20T22:09:34.974094] [DEBUG] [LOCAL ] GNK: Derivative data saved (1794 records). [2025-10-20T22:09:34.974411] [INFO ] [MAIN ] GNK: Successfully saved derivative data. [2025-10-20T22:09:34.977547] [INFO ] [MAIN ] Derivative Progress: 782/1877 symbols completed. GNK took 1.1s. Estimated remaining: 6m 53s. Rate: 159.2 symbols/min [2025-10-20T22:09:34.979312] [INFO ] [MAIN ] GNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:34.998959] [DEBUG] [MAIN ] GNL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:34.999858] [INFO ] [MAIN ] GNL: Generated 1794 derivative records from 46630 primary records. [2025-10-20T22:09:35.000045] [DEBUG] [LOCAL ] GNL: Saving derivative data. [2025-10-20T22:09:35.026626] [DEBUG] [LOCAL ] GNL: Derivative data saved (1794 records). [2025-10-20T22:09:35.026956] [DEBUG] [LOCAL ] GNT: Loaded 8645 existing records from primary storage. [2025-10-20T22:09:35.027208] [INFO ] [MAIN ] GNL: Successfully saved derivative data. [2025-10-20T22:09:35.029298] [INFO ] [MAIN ] Derivative Progress: 783/1877 symbols completed. GNL took 1.1s. Estimated remaining: 6m 52s. Rate: 159.3 symbols/min [2025-10-20T22:09:35.029474] [INFO ] [MAIN ] GNW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:35.029795] [INFO ] [MAIN ] GNT: Loaded 8645 primary data records. [2025-10-20T22:09:35.201147] [DEBUG] [LOCAL ] GNRC: Loaded 45450 existing records from primary storage. [2025-10-20T22:09:35.246202] [INFO ] [MAIN ] GNRC: Loaded 45450 primary data records. [2025-10-20T22:09:35.490026] [DEBUG] [LOCAL ] GNW: Loaded 51392 existing records from primary storage. [2025-10-20T22:09:35.506272] [INFO ] [MAIN ] GNW: Loaded 51392 primary data records. [2025-10-20T22:09:36.054836] [DEBUG] [MAIN ] GNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:36.056286] [INFO ] [MAIN ] GNT: Generated 1794 derivative records from 8645 primary records. [2025-10-20T22:09:36.056865] [DEBUG] [LOCAL ] GNT: Saving derivative data. [2025-10-20T22:09:36.074036] [DEBUG] [LOCAL ] GNT: Derivative data saved (1794 records). [2025-10-20T22:09:36.075445] [INFO ] [MAIN ] GNT: Successfully saved derivative data. [2025-10-20T22:09:36.077384] [INFO ] [MAIN ] Derivative Progress: 784/1877 symbols completed. GNT took 1.1s. Estimated remaining: 6m 52s. Rate: 159.0 symbols/min [2025-10-20T22:09:36.077716] [INFO ] [MAIN ] GOF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:36.095961] [DEBUG] [MAIN ] GNRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:36.097083] [INFO ] [MAIN ] GNRC: Generated 1794 derivative records from 45450 primary records. [2025-10-20T22:09:36.097275] [DEBUG] [LOCAL ] GNRC: Saving derivative data. [2025-10-20T22:09:36.117397] [DEBUG] [LOCAL ] GNRC: Derivative data saved (1794 records). [2025-10-20T22:09:36.117742] [INFO ] [MAIN ] GNRC: Successfully saved derivative data. [2025-10-20T22:09:36.130743] [DEBUG] [MAIN ] GNW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:36.133044] [INFO ] [MAIN ] Derivative Progress: 785/1877 symbols completed. GNRC took 1.2s. Estimated remaining: 6m 52s. Rate: 159.2 symbols/min [2025-10-20T22:09:36.133254] [INFO ] [MAIN ] GOLF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:36.134787] [INFO ] [MAIN ] GNW: Generated 1794 derivative records from 51392 primary records. [2025-10-20T22:09:36.134905] [DEBUG] [LOCAL ] GNW: Saving derivative data. [2025-10-20T22:09:36.160043] [DEBUG] [LOCAL ] GNW: Derivative data saved (1794 records). [2025-10-20T22:09:36.160346] [INFO ] [MAIN ] GNW: Successfully saved derivative data. [2025-10-20T22:09:36.162375] [INFO ] [MAIN ] Derivative Progress: 786/1877 symbols completed. GNW took 1.1s. Estimated remaining: 6m 51s. Rate: 159.3 symbols/min [2025-10-20T22:09:36.162680] [INFO ] [MAIN ] GOOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:36.181456] [DEBUG] [LOCAL ] GOF: Loaded 42707 existing records from primary storage. [2025-10-20T22:09:36.188571] [INFO ] [MAIN ] GOF: Loaded 42707 primary data records. [2025-10-20T22:09:36.336603] [DEBUG] [LOCAL ] GOLF: Loaded 30669 existing records from primary storage. [2025-10-20T22:09:36.357609] [INFO ] [MAIN ] GOLF: Loaded 30669 primary data records. [2025-10-20T22:09:36.557896] [DEBUG] [LOCAL ] GOOS: Loaded 44284 existing records from primary storage. [2025-10-20T22:09:36.580627] [INFO ] [MAIN ] GOOS: Loaded 44284 primary data records. [2025-10-20T22:09:37.168859] [DEBUG] [MAIN ] GOF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:37.170529] [INFO ] [MAIN ] GOF: Generated 1794 derivative records from 42707 primary records. [2025-10-20T22:09:37.170922] [DEBUG] [LOCAL ] GOF: Saving derivative data. [2025-10-20T22:09:37.189451] [DEBUG] [LOCAL ] GOF: Derivative data saved (1794 records). [2025-10-20T22:09:37.190066] [INFO ] [MAIN ] GOF: Successfully saved derivative data. [2025-10-20T22:09:37.192602] [INFO ] [MAIN ] Derivative Progress: 787/1877 symbols completed. GOF took 1.1s. Estimated remaining: 6m 51s. Rate: 159.0 symbols/min [2025-10-20T22:09:37.192811] [INFO ] [MAIN ] GOTU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:37.216436] [DEBUG] [MAIN ] GOOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:37.217602] [INFO ] [MAIN ] GOOS: Generated 1794 derivative records from 44284 primary records. [2025-10-20T22:09:37.217877] [DEBUG] [LOCAL ] GOOS: Saving derivative data. [2025-10-20T22:09:37.240524] [DEBUG] [MAIN ] GOLF: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:37.241403] [INFO ] [MAIN ] GOLF: Generated 1793 derivative records from 30669 primary records. [2025-10-20T22:09:37.241813] [DEBUG] [LOCAL ] GOLF: Saving derivative data. [2025-10-20T22:09:37.264213] [DEBUG] [LOCAL ] GOOS: Derivative data saved (1794 records). [2025-10-20T22:09:37.264398] [INFO ] [MAIN ] GOOS: Successfully saved derivative data. [2025-10-20T22:09:37.270017] [INFO ] [MAIN ] Derivative Progress: 788/1877 symbols completed. GOOS took 1.1s. Estimated remaining: 6m 51s. Rate: 159.2 symbols/min [2025-10-20T22:09:37.272262] [INFO ] [MAIN ] GPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:37.285596] [DEBUG] [LOCAL ] GOTU: Loaded 36945 existing records from primary storage. [2025-10-20T22:09:37.287471] [DEBUG] [LOCAL ] GOLF: Derivative data saved (1793 records). [2025-10-20T22:09:37.287821] [INFO ] [MAIN ] GOTU: Loaded 36945 primary data records. [2025-10-20T22:09:37.287745] [INFO ] [MAIN ] GOLF: Successfully saved derivative data. [2025-10-20T22:09:37.290021] [INFO ] [MAIN ] Derivative Progress: 789/1877 symbols completed. GOLF took 1.2s. Estimated remaining: 6m 50s. Rate: 159.3 symbols/min [2025-10-20T22:09:37.321286] [INFO ] [MAIN ] GPI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:37.609014] [DEBUG] [LOCAL ] GPI: Loaded 14844 existing records from primary storage. [2025-10-20T22:09:37.609278] [INFO ] [MAIN ] GPI: Loaded 14844 primary data records. [2025-10-20T22:09:37.779614] [DEBUG] [LOCAL ] GPC: Loaded 45947 existing records from primary storage. [2025-10-20T22:09:37.797111] [INFO ] [MAIN ] GPC: Loaded 45947 primary data records. [2025-10-20T22:09:38.170390] [DEBUG] [MAIN ] GOTU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:38.350411] [INFO ] [MAIN ] GOTU: Generated 1794 derivative records from 36945 primary records. [2025-10-20T22:09:38.364938] [DEBUG] [LOCAL ] GOTU: Saving derivative data. [2025-10-20T22:09:38.383323] [DEBUG] [LOCAL ] GOTU: Derivative data saved (1794 records). [2025-10-20T22:09:38.384483] [INFO ] [MAIN ] GOTU: Successfully saved derivative data. [2025-10-20T22:09:38.387335] [INFO ] [MAIN ] Derivative Progress: 790/1877 symbols completed. GOTU took 1.2s. Estimated remaining: 6m 50s. Rate: 159.0 symbols/min [2025-10-20T22:09:38.387480] [INFO ] [MAIN ] GPJA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:38.407818] [DEBUG] [LOCAL ] GPJA: Loaded 4118 existing records from primary storage. [2025-10-20T22:09:38.414091] [INFO ] [MAIN ] GPJA: Loaded 4118 primary data records. [2025-10-20T22:09:38.413982] [DEBUG] [MAIN ] GPI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:38.420436] [INFO ] [MAIN ] GPI: Generated 1793 derivative records from 14844 primary records. [2025-10-20T22:09:38.453183] [DEBUG] [LOCAL ] GPI: Saving derivative data. [2025-10-20T22:09:38.549631] [DEBUG] [LOCAL ] GPI: Derivative data saved (1793 records). [2025-10-20T22:09:38.549881] [INFO ] [MAIN ] GPI: Successfully saved derivative data. [2025-10-20T22:09:38.551771] [INFO ] [MAIN ] Derivative Progress: 791/1877 symbols completed. GPI took 1.2s. Estimated remaining: 6m 50s. Rate: 159.1 symbols/min [2025-10-20T22:09:38.568368] [INFO ] [MAIN ] GPK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:38.644745] [DEBUG] [MAIN ] GPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:38.651078] [INFO ] [MAIN ] GPC: Generated 1794 derivative records from 45947 primary records. [2025-10-20T22:09:38.754244] [DEBUG] [LOCAL ] GPC: Saving derivative data. [2025-10-20T22:09:38.788112] [DEBUG] [LOCAL ] GPC: Derivative data saved (1794 records). [2025-10-20T22:09:38.789184] [INFO ] [MAIN ] GPC: Successfully saved derivative data. [2025-10-20T22:09:38.791848] [INFO ] [MAIN ] Derivative Progress: 792/1877 symbols completed. GPC took 1.5s. Estimated remaining: 6m 49s. Rate: 159.1 symbols/min [2025-10-20T22:09:38.792192] [INFO ] [MAIN ] GPMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:38.829729] [DEBUG] [MAIN ] GPJA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:38.852862] [INFO ] [MAIN ] GPJA: Generated 1794 derivative records from 4118 primary records. [2025-10-20T22:09:38.854189] [DEBUG] [LOCAL ] GPK: Loaded 52784 existing records from primary storage. [2025-10-20T22:09:38.854345] [INFO ] [MAIN ] GPK: Loaded 52784 primary data records. [2025-10-20T22:09:38.853073] [DEBUG] [LOCAL ] GPJA: Saving derivative data. [2025-10-20T22:09:38.941415] [DEBUG] [LOCAL ] GPJA: Derivative data saved (1794 records). [2025-10-20T22:09:38.967238] [INFO ] [MAIN ] GPJA: Successfully saved derivative data. [2025-10-20T22:09:38.990278] [DEBUG] [LOCAL ] GPMT: Loaded 25982 existing records from primary storage. [2025-10-20T22:09:38.992874] [INFO ] [MAIN ] Derivative Progress: 793/1877 symbols completed. GPJA took 0.6s. Estimated remaining: 6m 48s. Rate: 159.2 symbols/min [2025-10-20T22:09:39.013830] [INFO ] [MAIN ] GPMT: Loaded 25982 primary data records. [2025-10-20T22:09:39.522964] [DEBUG] [MAIN ] GPK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:39.523781] [INFO ] [MAIN ] GPK: Generated 1794 derivative records from 52784 primary records. [2025-10-20T22:09:39.523957] [DEBUG] [LOCAL ] GPK: Saving derivative data. [2025-10-20T22:09:39.541405] [DEBUG] [LOCAL ] GPK: Derivative data saved (1794 records). [2025-10-20T22:09:39.543084] [INFO ] [MAIN ] GPK: Successfully saved derivative data. [2025-10-20T22:09:39.546789] [INFO ] [MAIN ] Derivative Progress: 794/1877 symbols completed. GPK took 1.0s. Estimated remaining: 6m 48s. Rate: 159.1 symbols/min [2025-10-20T22:09:39.560164] [DEBUG] [MAIN ] GPMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:39.560885] [INFO ] [MAIN ] GPMT: Generated 1794 derivative records from 25982 primary records. [2025-10-20T22:09:39.560949] [DEBUG] [LOCAL ] GPMT: Saving derivative data. [2025-10-20T22:09:39.576126] [DEBUG] [LOCAL ] GPMT: Derivative data saved (1794 records). [2025-10-20T22:09:39.576215] [INFO ] [MAIN ] GPMT: Successfully saved derivative data. [2025-10-20T22:09:39.578089] [INFO ] [MAIN ] Derivative Progress: 795/1877 symbols completed. GPMT took 0.8s. Estimated remaining: 6m 47s. Rate: 159.3 symbols/min [2025-10-20T22:09:39.578449] [INFO ] [MAIN ] GPN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:39.579030] [INFO ] [MAIN ] GPOR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:39.578504] [INFO ] [MAIN ] GPRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:39.614355] [DEBUG] [LOCAL ] GPOR: Loaded 23397 existing records from primary storage. [2025-10-20T22:09:39.614515] [INFO ] [MAIN ] GPOR: Loaded 23397 primary data records. [2025-10-20T22:09:39.754728] [DEBUG] [LOCAL ] GPRK: Loaded 39821 existing records from primary storage. [2025-10-20T22:09:39.770805] [INFO ] [MAIN ] GPRK: Loaded 39821 primary data records. [2025-10-20T22:09:39.777873] [DEBUG] [LOCAL ] GPN: Loaded 51943 existing records from primary storage. [2025-10-20T22:09:39.803909] [INFO ] [MAIN ] GPN: Loaded 51943 primary data records. [2025-10-20T22:09:40.676722] [DEBUG] [MAIN ] GPOR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:40.677769] [INFO ] [MAIN ] GPOR: Generated 1793 derivative records from 23397 primary records. [2025-10-20T22:09:40.678070] [DEBUG] [LOCAL ] GPOR: Saving derivative data. [2025-10-20T22:09:40.698697] [DEBUG] [LOCAL ] GPOR: Derivative data saved (1793 records). [2025-10-20T22:09:40.699450] [INFO ] [MAIN ] GPOR: Successfully saved derivative data. [2025-10-20T22:09:40.702804] [INFO ] [MAIN ] Derivative Progress: 796/1877 symbols completed. GPOR took 1.1s. Estimated remaining: 6m 48s. Rate: 158.9 symbols/min [2025-10-20T22:09:40.703458] [INFO ] [MAIN ] GRBK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:40.733276] [DEBUG] [MAIN ] GPRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:40.739578] [DEBUG] [MAIN ] GPN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:40.740547] [INFO ] [MAIN ] GPRK: Generated 1794 derivative records from 39821 primary records. [2025-10-20T22:09:40.740969] [DEBUG] [LOCAL ] GPRK: Saving derivative data. [2025-10-20T22:09:40.742441] [INFO ] [MAIN ] GPN: Generated 1794 derivative records from 51943 primary records. [2025-10-20T22:09:40.742879] [DEBUG] [LOCAL ] GPN: Saving derivative data. [2025-10-20T22:09:40.780258] [DEBUG] [LOCAL ] GPRK: Derivative data saved (1794 records). [2025-10-20T22:09:40.780406] [INFO ] [MAIN ] GPRK: Successfully saved derivative data. [2025-10-20T22:09:40.782963] [INFO ] [MAIN ] Derivative Progress: 797/1877 symbols completed. GPRK took 1.2s. Estimated remaining: 6m 47s. Rate: 159.1 symbols/min [2025-10-20T22:09:40.785823] [INFO ] [MAIN ] GRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:40.787581] [DEBUG] [LOCAL ] GPN: Derivative data saved (1794 records). [2025-10-20T22:09:40.798495] [DEBUG] [LOCAL ] GRBK: Loaded 26937 existing records from primary storage. [2025-10-20T22:09:40.798803] [INFO ] [MAIN ] GRBK: Loaded 26937 primary data records. [2025-10-20T22:09:40.798024] [INFO ] [MAIN ] GPN: Successfully saved derivative data. [2025-10-20T22:09:40.803028] [INFO ] [MAIN ] Derivative Progress: 798/1877 symbols completed. GPN took 1.2s. Estimated remaining: 6m 46s. Rate: 159.3 symbols/min [2025-10-20T22:09:40.824862] [INFO ] [MAIN ] GRDN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:40.919044] [DEBUG] [LOCAL ] GRC: Loaded 9306 existing records from primary storage. [2025-10-20T22:09:40.923694] [INFO ] [MAIN ] GRC: Loaded 9306 primary data records. [2025-10-20T22:09:41.110207] [DEBUG] [LOCAL ] GRDN: Loaded 21063 existing records from primary storage. [2025-10-20T22:09:41.241352] [INFO ] [MAIN ] GRDN: Loaded 21063 primary data records. [2025-10-20T22:09:41.794838] [DEBUG] [MAIN ] GRBK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:41.796250] [INFO ] [MAIN ] GRBK: Generated 1794 derivative records from 26937 primary records. [2025-10-20T22:09:41.796733] [DEBUG] [LOCAL ] GRBK: Saving derivative data. [2025-10-20T22:09:41.796590] [DEBUG] [MAIN ] GRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:41.798543] [INFO ] [MAIN ] GRC: Generated 1794 derivative records from 9306 primary records. [2025-10-20T22:09:41.798669] [DEBUG] [LOCAL ] GRC: Saving derivative data. [2025-10-20T22:09:41.823070] [DEBUG] [LOCAL ] GRBK: Derivative data saved (1794 records). [2025-10-20T22:09:41.825948] [INFO ] [MAIN ] GRBK: Successfully saved derivative data. [2025-10-20T22:09:41.828603] [INFO ] [MAIN ] Derivative Progress: 799/1877 symbols completed. GRBK took 1.1s. Estimated remaining: 6m 47s. Rate: 158.9 symbols/min [2025-10-20T22:09:41.830218] [INFO ] [MAIN ] GRND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:41.834108] [DEBUG] [LOCAL ] GRC: Derivative data saved (1794 records). [2025-10-20T22:09:41.835453] [INFO ] [MAIN ] GRC: Successfully saved derivative data. [2025-10-20T22:09:41.838236] [INFO ] [MAIN ] Derivative Progress: 800/1877 symbols completed. GRC took 1.1s. Estimated remaining: 6m 46s. Rate: 159.1 symbols/min [2025-10-20T22:09:41.838683] [INFO ] [MAIN ] GRNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:41.851964] [DEBUG] [MAIN ] GRDN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:41.852672] [INFO ] [MAIN ] GRDN: Generated 1794 derivative records from 21063 primary records. [2025-10-20T22:09:41.852777] [DEBUG] [LOCAL ] GRDN: Saving derivative data. [2025-10-20T22:09:41.869932] [DEBUG] [LOCAL ] GRDN: Derivative data saved (1794 records). [2025-10-20T22:09:41.870200] [INFO ] [MAIN ] GRDN: Successfully saved derivative data. [2025-10-20T22:09:41.873739] [INFO ] [MAIN ] Derivative Progress: 801/1877 symbols completed. GRDN took 1.0s. Estimated remaining: 6m 45s. Rate: 159.3 symbols/min [2025-10-20T22:09:41.873950] [INFO ] [MAIN ] GROV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:41.902870] [DEBUG] [LOCAL ] GROV: Loaded 8747 existing records from primary storage. [2025-10-20T22:09:41.913534] [INFO ] [MAIN ] GROV: Loaded 8747 primary data records. [2025-10-20T22:09:41.945629] [DEBUG] [LOCAL ] GRNT: Loaded 33992 existing records from primary storage. [2025-10-20T22:09:41.958059] [INFO ] [MAIN ] GRNT: Loaded 33992 primary data records. [2025-10-20T22:09:42.115824] [DEBUG] [LOCAL ] GRND: Loaded 50199 existing records from primary storage. [2025-10-20T22:09:42.138215] [INFO ] [MAIN ] GRND: Loaded 50199 primary data records. [2025-10-20T22:09:42.906770] [DEBUG] [MAIN ] GROV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:42.907801] [INFO ] [MAIN ] GROV: Generated 1794 derivative records from 8747 primary records. [2025-10-20T22:09:42.908824] [DEBUG] [LOCAL ] GROV: Saving derivative data. [2025-10-20T22:09:42.940319] [DEBUG] [LOCAL ] GROV: Derivative data saved (1794 records). [2025-10-20T22:09:42.940730] [DEBUG] [MAIN ] GRNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:42.941438] [INFO ] [MAIN ] GRNT: Generated 1794 derivative records from 33992 primary records. [2025-10-20T22:09:42.940432] [INFO ] [MAIN ] GROV: Successfully saved derivative data. [2025-10-20T22:09:42.943384] [DEBUG] [LOCAL ] GRNT: Saving derivative data. [2025-10-20T22:09:42.943312] [INFO ] [MAIN ] Derivative Progress: 802/1877 symbols completed. GROV took 1.1s. Estimated remaining: 6m 46s. Rate: 158.9 symbols/min [2025-10-20T22:09:42.944447] [INFO ] [MAIN ] GRX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:42.972144] [DEBUG] [LOCAL ] GRNT: Derivative data saved (1794 records). [2025-10-20T22:09:42.972344] [INFO ] [MAIN ] GRNT: Successfully saved derivative data. [2025-10-20T22:09:42.970994] [DEBUG] [MAIN ] GRND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:42.975327] [INFO ] [MAIN ] Derivative Progress: 803/1877 symbols completed. GRNT took 1.1s. Estimated remaining: 6m 45s. Rate: 159.1 symbols/min [2025-10-20T22:09:42.973210] [INFO ] [MAIN ] GRND: Generated 1794 derivative records from 50199 primary records. [2025-10-20T22:09:42.976602] [INFO ] [MAIN ] GS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:42.977243] [DEBUG] [LOCAL ] GRND: Saving derivative data. [2025-10-20T22:09:42.981335] [DEBUG] [LOCAL ] GRX: Loaded 4840 existing records from primary storage. [2025-10-20T22:09:42.989204] [INFO ] [MAIN ] GRX: Loaded 4840 primary data records. [2025-10-20T22:09:43.074024] [DEBUG] [LOCAL ] GRND: Derivative data saved (1794 records). [2025-10-20T22:09:43.092310] [INFO ] [MAIN ] GRND: Successfully saved derivative data. [2025-10-20T22:09:43.101306] [INFO ] [MAIN ] Derivative Progress: 804/1877 symbols completed. GRND took 1.3s. Estimated remaining: 6m 44s. Rate: 159.3 symbols/min [2025-10-20T22:09:43.110431] [INFO ] [MAIN ] GSBD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:43.361297] [DEBUG] [LOCAL ] GS: Loaded 51784 existing records from primary storage. [2025-10-20T22:09:43.361913] [INFO ] [MAIN ] GS: Loaded 51784 primary data records. [2025-10-20T22:09:43.454662] [DEBUG] [LOCAL ] GSBD: Loaded 39645 existing records from primary storage. [2025-10-20T22:09:43.465782] [INFO ] [MAIN ] GSBD: Loaded 39645 primary data records. [2025-10-20T22:09:43.788673] [DEBUG] [MAIN ] GRX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:43.813763] [INFO ] [MAIN ] GRX: Generated 1794 derivative records from 4840 primary records. [2025-10-20T22:09:43.819538] [DEBUG] [LOCAL ] GRX: Saving derivative data. [2025-10-20T22:09:43.894734] [DEBUG] [LOCAL ] GRX: Derivative data saved (1794 records). [2025-10-20T22:09:43.895043] [INFO ] [MAIN ] GRX: Successfully saved derivative data. [2025-10-20T22:09:43.897298] [INFO ] [MAIN ] Derivative Progress: 805/1877 symbols completed. GRX took 1.0s. Estimated remaining: 6m 44s. Rate: 159.0 symbols/min [2025-10-20T22:09:43.897451] [INFO ] [MAIN ] GSK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:44.039020] [DEBUG] [MAIN ] GS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:44.039771] [INFO ] [MAIN ] GS: Generated 1794 derivative records from 51784 primary records. [2025-10-20T22:09:44.039934] [DEBUG] [LOCAL ] GS: Saving derivative data. [2025-10-20T22:09:44.056416] [DEBUG] [LOCAL ] GS: Derivative data saved (1794 records). [2025-10-20T22:09:44.070721] [INFO ] [MAIN ] GS: Successfully saved derivative data. [2025-10-20T22:09:44.074366] [INFO ] [MAIN ] Derivative Progress: 806/1877 symbols completed. GS took 1.1s. Estimated remaining: 6m 44s. Rate: 159.1 symbols/min [2025-10-20T22:09:44.074843] [INFO ] [MAIN ] GSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:44.088703] [DEBUG] [MAIN ] GSBD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:44.089468] [INFO ] [MAIN ] GSBD: Generated 1794 derivative records from 39645 primary records. [2025-10-20T22:09:44.090422] [DEBUG] [LOCAL ] GSBD: Saving derivative data. [2025-10-20T22:09:44.123036] [DEBUG] [LOCAL ] GSK: Loaded 53512 existing records from primary storage. [2025-10-20T22:09:44.123594] [DEBUG] [LOCAL ] GSBD: Derivative data saved (1794 records). [2025-10-20T22:09:44.123768] [INFO ] [MAIN ] GSK: Loaded 53512 primary data records. [2025-10-20T22:09:44.124316] [INFO ] [MAIN ] GSBD: Successfully saved derivative data. [2025-10-20T22:09:44.163115] [INFO ] [MAIN ] Derivative Progress: 807/1877 symbols completed. GSBD took 1.1s. Estimated remaining: 6m 43s. Rate: 159.3 symbols/min [2025-10-20T22:09:44.191323] [INFO ] [MAIN ] GTES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:44.273741] [DEBUG] [LOCAL ] GSL: Loaded 33197 existing records from primary storage. [2025-10-20T22:09:44.300948] [INFO ] [MAIN ] GSL: Loaded 33197 primary data records. [2025-10-20T22:09:44.675446] [DEBUG] [LOCAL ] GTES: Loaded 50321 existing records from primary storage. [2025-10-20T22:09:44.781698] [INFO ] [MAIN ] GTES: Loaded 50321 primary data records. [2025-10-20T22:09:45.112011] [DEBUG] [MAIN ] GSK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:45.113572] [INFO ] [MAIN ] GSK: Generated 1794 derivative records from 53512 primary records. [2025-10-20T22:09:45.113912] [DEBUG] [LOCAL ] GSK: Saving derivative data. [2025-10-20T22:09:45.134264] [DEBUG] [LOCAL ] GSK: Derivative data saved (1794 records). [2025-10-20T22:09:45.135545] [INFO ] [MAIN ] GSK: Successfully saved derivative data. [2025-10-20T22:09:45.137728] [INFO ] [MAIN ] Derivative Progress: 808/1877 symbols completed. GSK took 1.2s. Estimated remaining: 6m 43s. Rate: 159.0 symbols/min [2025-10-20T22:09:45.151054] [DEBUG] [MAIN ] GSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:45.151879] [INFO ] [MAIN ] GSL: Generated 1794 derivative records from 33197 primary records. [2025-10-20T22:09:45.152778] [DEBUG] [LOCAL ] GSL: Saving derivative data. [2025-10-20T22:09:45.169031] [DEBUG] [LOCAL ] GSL: Derivative data saved (1794 records). [2025-10-20T22:09:45.169166] [INFO ] [MAIN ] GSL: Successfully saved derivative data. [2025-10-20T22:09:45.171110] [INFO ] [MAIN ] Derivative Progress: 809/1877 symbols completed. GSL took 1.1s. Estimated remaining: 6m 43s. Rate: 159.2 symbols/min [2025-10-20T22:09:45.187113] [DEBUG] [MAIN ] GTES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:45.187612] [INFO ] [MAIN ] GTES: Generated 1794 derivative records from 50321 primary records. [2025-10-20T22:09:45.187692] [DEBUG] [LOCAL ] GTES: Saving derivative data. [2025-10-20T22:09:45.202541] [DEBUG] [LOCAL ] GTES: Derivative data saved (1794 records). [2025-10-20T22:09:45.202596] [INFO ] [MAIN ] GTES: Successfully saved derivative data. [2025-10-20T22:09:45.204692] [INFO ] [MAIN ] Derivative Progress: 810/1877 symbols completed. GTES took 1.0s. Estimated remaining: 6m 42s. Rate: 159.3 symbols/min [2025-10-20T22:09:45.205130] [INFO ] [MAIN ] GTN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:45.205227] [INFO ] [MAIN ] GTY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:45.205076] [INFO ] [MAIN ] GTLS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:45.262092] [DEBUG] [LOCAL ] GTY: Loaded 30157 existing records from primary storage. [2025-10-20T22:09:45.262315] [INFO ] [MAIN ] GTY: Loaded 30157 primary data records. [2025-10-20T22:09:45.392200] [DEBUG] [LOCAL ] GTLS: Loaded 43326 existing records from primary storage. [2025-10-20T22:09:45.393617] [INFO ] [MAIN ] GTLS: Loaded 43326 primary data records. [2025-10-20T22:09:45.404552] [DEBUG] [LOCAL ] GTN: Loaded 44989 existing records from primary storage. [2025-10-20T22:09:45.433566] [INFO ] [MAIN ] GTN: Loaded 44989 primary data records. [2025-10-20T22:09:46.268851] [DEBUG] [MAIN ] GTY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:46.269691] [INFO ] [MAIN ] GTY: Generated 1794 derivative records from 30157 primary records. [2025-10-20T22:09:46.269938] [DEBUG] [LOCAL ] GTY: Saving derivative data. [2025-10-20T22:09:46.289443] [DEBUG] [LOCAL ] GTY: Derivative data saved (1794 records). [2025-10-20T22:09:46.303056] [INFO ] [MAIN ] GTY: Successfully saved derivative data. [2025-10-20T22:09:46.305834] [INFO ] [MAIN ] Derivative Progress: 811/1877 symbols completed. GTY took 1.1s. Estimated remaining: 6m 42s. Rate: 159.0 symbols/min [2025-10-20T22:09:46.306121] [INFO ] [MAIN ] GUG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:46.307336] [DEBUG] [MAIN ] GTLS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:46.320680] [INFO ] [MAIN ] GTLS: Generated 1794 derivative records from 43326 primary records. [2025-10-20T22:09:46.320858] [DEBUG] [LOCAL ] GTLS: Saving derivative data. [2025-10-20T22:09:46.319831] [DEBUG] [MAIN ] GTN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:46.322051] [INFO ] [MAIN ] GTN: Generated 1794 derivative records from 44989 primary records. [2025-10-20T22:09:46.329542] [DEBUG] [LOCAL ] GTN: Saving derivative data. [2025-10-20T22:09:46.340793] [DEBUG] [LOCAL ] GTLS: Derivative data saved (1794 records). [2025-10-20T22:09:46.340906] [INFO ] [MAIN ] GTLS: Successfully saved derivative data. [2025-10-20T22:09:46.342969] [INFO ] [MAIN ] Derivative Progress: 812/1877 symbols completed. GTLS took 1.1s. Estimated remaining: 6m 42s. Rate: 159.1 symbols/min [2025-10-20T22:09:46.350040] [INFO ] [MAIN ] GUT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:46.357970] [DEBUG] [LOCAL ] GTN: Derivative data saved (1794 records). [2025-10-20T22:09:46.362848] [INFO ] [MAIN ] GTN: Successfully saved derivative data. [2025-10-20T22:09:46.365542] [DEBUG] [LOCAL ] GUG: Loaded 12194 existing records from primary storage. [2025-10-20T22:09:46.365062] [INFO ] [MAIN ] Derivative Progress: 813/1877 symbols completed. GTN took 1.2s. Estimated remaining: 6m 41s. Rate: 159.3 symbols/min [2025-10-20T22:09:46.365813] [INFO ] [MAIN ] GUG: Loaded 12194 primary data records. [2025-10-20T22:09:46.365935] [INFO ] [MAIN ] GWH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:46.532724] [DEBUG] [LOCAL ] GUT: Loaded 15737 existing records from primary storage. [2025-10-20T22:09:46.569976] [INFO ] [MAIN ] GUT: Loaded 15737 primary data records. [2025-10-20T22:09:46.689756] [DEBUG] [LOCAL ] GWH: Loaded 19537 existing records from primary storage. [2025-10-20T22:09:46.723926] [INFO ] [MAIN ] GWH: Loaded 19537 primary data records. [2025-10-20T22:09:47.330539] [DEBUG] [MAIN ] GUG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:47.332280] [INFO ] [MAIN ] GUG: Generated 1794 derivative records from 12194 primary records. [2025-10-20T22:09:47.332862] [DEBUG] [LOCAL ] GUG: Saving derivative data. [2025-10-20T22:09:47.351971] [DEBUG] [LOCAL ] GUG: Derivative data saved (1794 records). [2025-10-20T22:09:47.352189] [INFO ] [MAIN ] GUG: Successfully saved derivative data. [2025-10-20T22:09:47.354411] [INFO ] [MAIN ] Derivative Progress: 814/1877 symbols completed. GUG took 1.0s. Estimated remaining: 6m 41s. Rate: 159.0 symbols/min [2025-10-20T22:09:47.354741] [INFO ] [MAIN ] GWRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:47.371546] [DEBUG] [MAIN ] GUT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:47.372219] [INFO ] [MAIN ] GUT: Generated 1794 derivative records from 15737 primary records. [2025-10-20T22:09:47.372437] [DEBUG] [LOCAL ] GUT: Saving derivative data. [2025-10-20T22:09:47.391722] [DEBUG] [LOCAL ] GUT: Derivative data saved (1794 records). [2025-10-20T22:09:47.391977] [INFO ] [MAIN ] GUT: Successfully saved derivative data. [2025-10-20T22:09:47.395505] [INFO ] [MAIN ] Derivative Progress: 815/1877 symbols completed. GUT took 1.0s. Estimated remaining: 6m 40s. Rate: 159.2 symbols/min [2025-10-20T22:09:47.396119] [INFO ] [MAIN ] GWW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:47.410370] [DEBUG] [MAIN ] GWH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:47.411336] [INFO ] [MAIN ] GWH: Generated 1794 derivative records from 19537 primary records. [2025-10-20T22:09:47.411485] [DEBUG] [LOCAL ] GWH: Saving derivative data. [2025-10-20T22:09:47.443856] [DEBUG] [LOCAL ] GWH: Derivative data saved (1794 records). [2025-10-20T22:09:47.444783] [INFO ] [MAIN ] GWH: Successfully saved derivative data. [2025-10-20T22:09:47.446818] [INFO ] [MAIN ] Derivative Progress: 816/1877 symbols completed. GWH took 1.1s. Estimated remaining: 6m 40s. Rate: 159.3 symbols/min [2025-10-20T22:09:47.466963] [DEBUG] [LOCAL ] GWRE: Loaded 39801 existing records from primary storage. [2025-10-20T22:09:47.465992] [DEBUG] [LOCAL ] GWW: Loaded 19418 existing records from primary storage. [2025-10-20T22:09:47.467319] [INFO ] [MAIN ] GWRE: Loaded 39801 primary data records. [2025-10-20T22:09:47.467442] [INFO ] [MAIN ] GWW: Loaded 19418 primary data records. [2025-10-20T22:09:47.467206] [INFO ] [MAIN ] GXO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:48.140261] [DEBUG] [LOCAL ] GXO: Loaded 46617 existing records from primary storage. [2025-10-20T22:09:48.141337] [INFO ] [MAIN ] GXO: Loaded 46617 primary data records. [2025-10-20T22:09:48.469040] [DEBUG] [MAIN ] GWRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:48.471376] [INFO ] [MAIN ] GWRE: Generated 1794 derivative records from 39801 primary records. [2025-10-20T22:09:48.471478] [DEBUG] [LOCAL ] GWRE: Saving derivative data. [2025-10-20T22:09:48.504481] [DEBUG] [MAIN ] GWW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:48.504158] [DEBUG] [LOCAL ] GWRE: Derivative data saved (1794 records). [2025-10-20T22:09:48.505502] [INFO ] [MAIN ] GWW: Generated 1794 derivative records from 19418 primary records. [2025-10-20T22:09:48.505831] [DEBUG] [LOCAL ] GWW: Saving derivative data. [2025-10-20T22:09:48.505911] [INFO ] [MAIN ] GWRE: Successfully saved derivative data. [2025-10-20T22:09:48.509273] [INFO ] [MAIN ] Derivative Progress: 817/1877 symbols completed. GWRE took 1.2s. Estimated remaining: 6m 40s. Rate: 159.0 symbols/min [2025-10-20T22:09:48.517267] [INFO ] [MAIN ] GYLD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:48.528131] [DEBUG] [LOCAL ] GWW: Derivative data saved (1794 records). [2025-10-20T22:09:48.528397] [INFO ] [MAIN ] GWW: Successfully saved derivative data. [2025-10-20T22:09:48.530777] [INFO ] [MAIN ] Derivative Progress: 818/1877 symbols completed. GWW took 1.1s. Estimated remaining: 6m 39s. Rate: 159.2 symbols/min [2025-10-20T22:09:48.531042] [INFO ] [MAIN ] H: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:48.532953] [DEBUG] [LOCAL ] GYLD: Loaded 1544 existing records from primary storage. [2025-10-20T22:09:48.533551] [INFO ] [MAIN ] GYLD: Loaded 1544 primary data records. [2025-10-20T22:09:48.650554] [DEBUG] [MAIN ] GXO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:48.651514] [INFO ] [MAIN ] GXO: Generated 1793 derivative records from 46617 primary records. [2025-10-20T22:09:48.657121] [DEBUG] [LOCAL ] GXO: Saving derivative data. [2025-10-20T22:09:48.746273] [DEBUG] [LOCAL ] GXO: Derivative data saved (1793 records). [2025-10-20T22:09:48.860943] [INFO ] [MAIN ] GXO: Successfully saved derivative data. [2025-10-20T22:09:48.878087] [INFO ] [MAIN ] Derivative Progress: 819/1877 symbols completed. GXO took 1.4s. Estimated remaining: 6m 39s. Rate: 159.2 symbols/min [2025-10-20T22:09:48.879309] [INFO ] [MAIN ] HAE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:48.898308] [DEBUG] [LOCAL ] H: Loaded 44459 existing records from primary storage. [2025-10-20T22:09:48.899035] [INFO ] [MAIN ] H: Loaded 44459 primary data records. [2025-10-20T22:09:49.227296] [DEBUG] [LOCAL ] HAE: Loaded 39456 existing records from primary storage. [2025-10-20T22:09:49.227838] [INFO ] [MAIN ] HAE: Loaded 39456 primary data records. [2025-10-20T22:09:49.372250] [DEBUG] [MAIN ] GYLD: Filtered 7001 bins outside market hours, 7823 incomplete bins [2025-10-20T22:09:49.392819] [INFO ] [MAIN ] GYLD: Generated 1791 derivative records from 1544 primary records. [2025-10-20T22:09:49.424894] [DEBUG] [LOCAL ] GYLD: Saving derivative data. [2025-10-20T22:09:49.558056] [DEBUG] [LOCAL ] GYLD: Derivative data saved (1791 records). [2025-10-20T22:09:49.559588] [INFO ] [MAIN ] GYLD: Successfully saved derivative data. [2025-10-20T22:09:49.562096] [INFO ] [MAIN ] Derivative Progress: 820/1877 symbols completed. GYLD took 1.0s. Estimated remaining: 6m 39s. Rate: 159.0 symbols/min [2025-10-20T22:09:49.562287] [INFO ] [MAIN ] HAFN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:49.580034] [DEBUG] [MAIN ] H: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:49.580765] [INFO ] [MAIN ] H: Generated 1794 derivative records from 44459 primary records. [2025-10-20T22:09:49.581121] [DEBUG] [LOCAL ] H: Saving derivative data. [2025-10-20T22:09:49.598032] [DEBUG] [LOCAL ] H: Derivative data saved (1794 records). [2025-10-20T22:09:49.598266] [INFO ] [MAIN ] H: Successfully saved derivative data. [2025-10-20T22:09:49.600207] [INFO ] [MAIN ] Derivative Progress: 821/1877 symbols completed. H took 1.1s. Estimated remaining: 6m 38s. Rate: 159.2 symbols/min [2025-10-20T22:09:49.600682] [INFO ] [MAIN ] HAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:49.626810] [DEBUG] [MAIN ] HAE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:49.627462] [INFO ] [MAIN ] HAE: Generated 1794 derivative records from 39456 primary records. [2025-10-20T22:09:49.627672] [DEBUG] [LOCAL ] HAE: Saving derivative data. [2025-10-20T22:09:49.653972] [DEBUG] [LOCAL ] HAFN: Loaded 42843 existing records from primary storage. [2025-10-20T22:09:49.656200] [INFO ] [MAIN ] HAFN: Loaded 42843 primary data records. [2025-10-20T22:09:49.663495] [DEBUG] [LOCAL ] HAE: Derivative data saved (1794 records). [2025-10-20T22:09:49.699348] [INFO ] [MAIN ] HAE: Successfully saved derivative data. [2025-10-20T22:09:49.760943] [INFO ] [MAIN ] Derivative Progress: 822/1877 symbols completed. HAE took 0.9s. Estimated remaining: 6m 37s. Rate: 159.3 symbols/min [2025-10-20T22:09:49.780700] [INFO ] [MAIN ] HASI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:49.985702] [DEBUG] [LOCAL ] HAL: Loaded 53718 existing records from primary storage. [2025-10-20T22:09:49.994480] [INFO ] [MAIN ] HAL: Loaded 53718 primary data records. [2025-10-20T22:09:50.235596] [DEBUG] [LOCAL ] HASI: Loaded 47810 existing records from primary storage. [2025-10-20T22:09:50.349967] [INFO ] [MAIN ] HASI: Loaded 47810 primary data records. [2025-10-20T22:09:50.665152] [DEBUG] [MAIN ] HAFN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:50.666269] [INFO ] [MAIN ] HAFN: Generated 1794 derivative records from 42843 primary records. [2025-10-20T22:09:50.666368] [DEBUG] [LOCAL ] HAFN: Saving derivative data. [2025-10-20T22:09:50.685813] [DEBUG] [LOCAL ] HAFN: Derivative data saved (1794 records). [2025-10-20T22:09:50.685976] [INFO ] [MAIN ] HAFN: Successfully saved derivative data. [2025-10-20T22:09:50.688169] [INFO ] [MAIN ] Derivative Progress: 823/1877 symbols completed. HAFN took 1.1s. Estimated remaining: 6m 38s. Rate: 159.0 symbols/min [2025-10-20T22:09:50.700942] [DEBUG] [MAIN ] HAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:50.701893] [INFO ] [MAIN ] HAL: Generated 1794 derivative records from 53718 primary records. [2025-10-20T22:09:50.702446] [DEBUG] [LOCAL ] HAL: Saving derivative data. [2025-10-20T22:09:50.718791] [DEBUG] [LOCAL ] HAL: Derivative data saved (1794 records). [2025-10-20T22:09:50.718951] [INFO ] [MAIN ] HAL: Successfully saved derivative data. [2025-10-20T22:09:50.721051] [INFO ] [MAIN ] Derivative Progress: 824/1877 symbols completed. HAL took 1.1s. Estimated remaining: 6m 37s. Rate: 159.2 symbols/min [2025-10-20T22:09:50.735730] [DEBUG] [MAIN ] HASI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:50.736302] [INFO ] [MAIN ] HASI: Generated 1794 derivative records from 47810 primary records. [2025-10-20T22:09:50.736360] [DEBUG] [LOCAL ] HASI: Saving derivative data. [2025-10-20T22:09:50.751492] [DEBUG] [LOCAL ] HASI: Derivative data saved (1794 records). [2025-10-20T22:09:50.751547] [INFO ] [MAIN ] HASI: Successfully saved derivative data. [2025-10-20T22:09:50.753413] [INFO ] [MAIN ] Derivative Progress: 825/1877 symbols completed. HASI took 1.0s. Estimated remaining: 6m 36s. Rate: 159.4 symbols/min [2025-10-20T22:09:50.754083] [INFO ] [MAIN ] HBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:50.753999] [INFO ] [MAIN ] HBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:50.753947] [INFO ] [MAIN ] HAYW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:50.764068] [DEBUG] [LOCAL ] HBB: Loaded 6459 existing records from primary storage. [2025-10-20T22:09:50.764213] [INFO ] [MAIN ] HBB: Loaded 6459 primary data records. [2025-10-20T22:09:51.111764] [DEBUG] [LOCAL ] HAYW: Loaded 48833 existing records from primary storage. [2025-10-20T22:09:51.112067] [INFO ] [MAIN ] HAYW: Loaded 48833 primary data records. [2025-10-20T22:09:51.124713] [DEBUG] [LOCAL ] HBI: Loaded 52948 existing records from primary storage. [2025-10-20T22:09:51.144798] [INFO ] [MAIN ] HBI: Loaded 52948 primary data records. [2025-10-20T22:09:51.743188] [DEBUG] [MAIN ] HBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:51.744286] [INFO ] [MAIN ] HBB: Generated 1794 derivative records from 6459 primary records. [2025-10-20T22:09:51.744389] [DEBUG] [LOCAL ] HBB: Saving derivative data. [2025-10-20T22:09:51.762538] [DEBUG] [LOCAL ] HBB: Derivative data saved (1794 records). [2025-10-20T22:09:51.762935] [INFO ] [MAIN ] HBB: Successfully saved derivative data. [2025-10-20T22:09:51.765141] [INFO ] [MAIN ] Derivative Progress: 826/1877 symbols completed. HBB took 1.0s. Estimated remaining: 6m 36s. Rate: 159.1 symbols/min [2025-10-20T22:09:51.765297] [INFO ] [MAIN ] HBM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:51.785491] [DEBUG] [MAIN ] HBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:51.786435] [INFO ] [MAIN ] HBI: Generated 1794 derivative records from 52948 primary records. [2025-10-20T22:09:51.786557] [DEBUG] [LOCAL ] HBI: Saving derivative data. [2025-10-20T22:09:51.810399] [DEBUG] [MAIN ] HAYW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:51.815801] [DEBUG] [LOCAL ] HBI: Derivative data saved (1794 records). [2025-10-20T22:09:51.813521] [INFO ] [MAIN ] HAYW: Generated 1794 derivative records from 48833 primary records. [2025-10-20T22:09:51.816010] [DEBUG] [LOCAL ] HAYW: Saving derivative data. [2025-10-20T22:09:51.816082] [INFO ] [MAIN ] HBI: Successfully saved derivative data. [2025-10-20T22:09:51.818242] [INFO ] [MAIN ] Derivative Progress: 827/1877 symbols completed. HBI took 1.1s. Estimated remaining: 6m 36s. Rate: 159.2 symbols/min [2025-10-20T22:09:51.825887] [INFO ] [MAIN ] HCA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:51.834370] [DEBUG] [LOCAL ] HAYW: Derivative data saved (1794 records). [2025-10-20T22:09:51.847516] [INFO ] [MAIN ] HAYW: Successfully saved derivative data. [2025-10-20T22:09:51.849478] [INFO ] [MAIN ] Derivative Progress: 828/1877 symbols completed. HAYW took 1.1s. Estimated remaining: 6m 35s. Rate: 159.4 symbols/min [2025-10-20T22:09:51.849777] [INFO ] [MAIN ] HCC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:51.871363] [DEBUG] [LOCAL ] HBM: Loaded 52976 existing records from primary storage. [2025-10-20T22:09:51.871633] [INFO ] [MAIN ] HBM: Loaded 52976 primary data records. [2025-10-20T22:09:52.227470] [DEBUG] [LOCAL ] HCA: Loaded 46957 existing records from primary storage. [2025-10-20T22:09:52.235944] [DEBUG] [LOCAL ] HCC: Loaded 45162 existing records from primary storage. [2025-10-20T22:09:52.236673] [INFO ] [MAIN ] HCA: Loaded 46957 primary data records. [2025-10-20T22:09:52.236610] [INFO ] [MAIN ] HCC: Loaded 45162 primary data records. [2025-10-20T22:09:52.854530] [DEBUG] [MAIN ] HBM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:52.855428] [INFO ] [MAIN ] HBM: Generated 1794 derivative records from 52976 primary records. [2025-10-20T22:09:52.855517] [DEBUG] [LOCAL ] HBM: Saving derivative data. [2025-10-20T22:09:52.873183] [DEBUG] [LOCAL ] HBM: Derivative data saved (1794 records). [2025-10-20T22:09:52.873365] [INFO ] [MAIN ] HBM: Successfully saved derivative data. [2025-10-20T22:09:52.875526] [INFO ] [MAIN ] Derivative Progress: 829/1877 symbols completed. HBM took 1.1s. Estimated remaining: 6m 35s. Rate: 159.1 symbols/min [2025-10-20T22:09:52.875910] [INFO ] [MAIN ] HCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:52.894390] [DEBUG] [MAIN ] HCC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:52.895226] [INFO ] [MAIN ] HCC: Generated 1794 derivative records from 45162 primary records. [2025-10-20T22:09:52.895983] [DEBUG] [LOCAL ] HCC: Saving derivative data. [2025-10-20T22:09:52.921055] [DEBUG] [MAIN ] HCA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:52.922772] [INFO ] [MAIN ] HCA: Generated 1794 derivative records from 46957 primary records. [2025-10-20T22:09:52.930886] [DEBUG] [LOCAL ] HCA: Saving derivative data. [2025-10-20T22:09:52.935300] [DEBUG] [LOCAL ] HCI: Loaded 14113 existing records from primary storage. [2025-10-20T22:09:52.935737] [INFO ] [MAIN ] HCI: Loaded 14113 primary data records. [2025-10-20T22:09:52.934806] [DEBUG] [LOCAL ] HCC: Derivative data saved (1794 records). [2025-10-20T22:09:52.936051] [INFO ] [MAIN ] HCC: Successfully saved derivative data. [2025-10-20T22:09:52.959426] [INFO ] [MAIN ] Derivative Progress: 830/1877 symbols completed. HCC took 1.1s. Estimated remaining: 6m 35s. Rate: 159.2 symbols/min [2025-10-20T22:09:52.972718] [INFO ] [MAIN ] HD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:53.008236] [DEBUG] [LOCAL ] HCA: Derivative data saved (1794 records). [2025-10-20T22:09:53.029229] [INFO ] [MAIN ] HCA: Successfully saved derivative data. [2025-10-20T22:09:53.041789] [INFO ] [MAIN ] Derivative Progress: 831/1877 symbols completed. HCA took 1.2s. Estimated remaining: 6m 34s. Rate: 159.4 symbols/min [2025-10-20T22:09:53.041973] [INFO ] [MAIN ] HDB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:53.325410] [DEBUG] [LOCAL ] HD: Loaded 53482 existing records from primary storage. [2025-10-20T22:09:53.326448] [INFO ] [MAIN ] HD: Loaded 53482 primary data records. [2025-10-20T22:09:53.327318] [DEBUG] [LOCAL ] HDB: Loaded 51936 existing records from primary storage. [2025-10-20T22:09:53.370165] [INFO ] [MAIN ] HDB: Loaded 51936 primary data records. [2025-10-20T22:09:53.394944] [DEBUG] [MAIN ] HCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:53.434378] [INFO ] [MAIN ] HCI: Generated 1794 derivative records from 14113 primary records. [2025-10-20T22:09:53.446131] [DEBUG] [LOCAL ] HCI: Saving derivative data. [2025-10-20T22:09:53.623518] [DEBUG] [LOCAL ] HCI: Derivative data saved (1794 records). [2025-10-20T22:09:53.708420] [INFO ] [MAIN ] HCI: Successfully saved derivative data. [2025-10-20T22:09:53.817677] [INFO ] [MAIN ] Derivative Progress: 832/1877 symbols completed. HCI took 0.9s. Estimated remaining: 6m 34s. Rate: 159.2 symbols/min [2025-10-20T22:09:53.971607] [INFO ] [MAIN ] HE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:54.036234] [DEBUG] [MAIN ] HD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:54.035988] [DEBUG] [MAIN ] HDB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:54.037151] [INFO ] [MAIN ] HDB: Generated 1794 derivative records from 51936 primary records. [2025-10-20T22:09:54.037314] [DEBUG] [LOCAL ] HDB: Saving derivative data. [2025-10-20T22:09:54.038490] [INFO ] [MAIN ] HD: Generated 1794 derivative records from 53482 primary records. [2025-10-20T22:09:54.038804] [DEBUG] [LOCAL ] HD: Saving derivative data. [2025-10-20T22:09:54.065362] [DEBUG] [LOCAL ] HDB: Derivative data saved (1794 records). [2025-10-20T22:09:54.067767] [INFO ] [MAIN ] HDB: Successfully saved derivative data. [2025-10-20T22:09:54.070974] [INFO ] [MAIN ] Derivative Progress: 833/1877 symbols completed. HDB took 1.0s. Estimated remaining: 6m 33s. Rate: 159.2 symbols/min [2025-10-20T22:09:54.071720] [INFO ] [MAIN ] HEI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:54.073113] [DEBUG] [LOCAL ] HD: Derivative data saved (1794 records). [2025-10-20T22:09:54.073343] [INFO ] [MAIN ] HD: Successfully saved derivative data. [2025-10-20T22:09:54.075414] [INFO ] [MAIN ] Derivative Progress: 834/1877 symbols completed. HD took 1.1s. Estimated remaining: 6m 33s. Rate: 159.4 symbols/min [2025-10-20T22:09:54.089184] [INFO ] [MAIN ] HEQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:54.092072] [DEBUG] [LOCAL ] HE: Loaded 48865 existing records from primary storage. [2025-10-20T22:09:54.092231] [INFO ] [MAIN ] HE: Loaded 48865 primary data records. [2025-10-20T22:09:54.201286] [DEBUG] [LOCAL ] HEQ: Loaded 6822 existing records from primary storage. [2025-10-20T22:09:54.205871] [INFO ] [MAIN ] HEQ: Loaded 6822 primary data records. [2025-10-20T22:09:54.408347] [DEBUG] [LOCAL ] HEI: Loaded 30890 existing records from primary storage. [2025-10-20T22:09:54.411551] [INFO ] [MAIN ] HEI: Loaded 30890 primary data records. [2025-10-20T22:09:55.064962] [DEBUG] [MAIN ] HE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:55.066156] [INFO ] [MAIN ] HE: Generated 1794 derivative records from 48865 primary records. [2025-10-20T22:09:55.067108] [DEBUG] [LOCAL ] HE: Saving derivative data. [2025-10-20T22:09:55.085792] [DEBUG] [LOCAL ] HE: Derivative data saved (1794 records). [2025-10-20T22:09:55.086830] [INFO ] [MAIN ] HE: Successfully saved derivative data. [2025-10-20T22:09:55.088904] [INFO ] [MAIN ] Derivative Progress: 835/1877 symbols completed. HE took 1.1s. Estimated remaining: 6m 33s. Rate: 159.1 symbols/min [2025-10-20T22:09:55.089479] [INFO ] [MAIN ] HESM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:55.104126] [DEBUG] [MAIN ] HEQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:55.105309] [INFO ] [MAIN ] HEQ: Generated 1794 derivative records from 6822 primary records. [2025-10-20T22:09:55.105525] [DEBUG] [LOCAL ] HEQ: Saving derivative data. [2025-10-20T22:09:55.141031] [DEBUG] [LOCAL ] HEQ: Derivative data saved (1794 records). [2025-10-20T22:09:55.141760] [DEBUG] [MAIN ] HEI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:55.142501] [INFO ] [MAIN ] HEI: Generated 1794 derivative records from 30890 primary records. [2025-10-20T22:09:55.141831] [INFO ] [MAIN ] HEQ: Successfully saved derivative data. [2025-10-20T22:09:55.142592] [DEBUG] [LOCAL ] HEI: Saving derivative data. [2025-10-20T22:09:55.144638] [INFO ] [MAIN ] Derivative Progress: 836/1877 symbols completed. HEQ took 1.1s. Estimated remaining: 6m 32s. Rate: 159.3 symbols/min [2025-10-20T22:09:55.145266] [INFO ] [MAIN ] HFRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:55.161932] [DEBUG] [LOCAL ] HEI: Derivative data saved (1794 records). [2025-10-20T22:09:55.172320] [INFO ] [MAIN ] HEI: Successfully saved derivative data. [2025-10-20T22:09:55.174392] [INFO ] [MAIN ] Derivative Progress: 837/1877 symbols completed. HEI took 1.1s. Estimated remaining: 6m 31s. Rate: 159.4 symbols/min [2025-10-20T22:09:55.174783] [INFO ] [MAIN ] HGER: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:55.192310] [DEBUG] [LOCAL ] HFRO: Loaded 16506 existing records from primary storage. [2025-10-20T22:09:55.192591] [INFO ] [MAIN ] HFRO: Loaded 16506 primary data records. [2025-10-20T22:09:55.282404] [DEBUG] [LOCAL ] HGER: Loaded 15250 existing records from primary storage. [2025-10-20T22:09:55.294062] [INFO ] [MAIN ] HGER: Loaded 15250 primary data records. [2025-10-20T22:09:55.300400] [DEBUG] [LOCAL ] HESM: Loaded 48651 existing records from primary storage. [2025-10-20T22:09:55.313980] [INFO ] [MAIN ] HESM: Loaded 48651 primary data records. [2025-10-20T22:09:56.197878] [DEBUG] [MAIN ] HFRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:56.210619] [INFO ] [MAIN ] HFRO: Generated 1794 derivative records from 16506 primary records. [2025-10-20T22:09:56.211925] [DEBUG] [LOCAL ] HFRO: Saving derivative data. [2025-10-20T22:09:56.211851] [DEBUG] [MAIN ] HGER: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:56.214138] [INFO ] [MAIN ] HGER: Generated 1794 derivative records from 15250 primary records. [2025-10-20T22:09:56.229881] [DEBUG] [LOCAL ] HGER: Saving derivative data. [2025-10-20T22:09:56.229989] [DEBUG] [LOCAL ] HFRO: Derivative data saved (1794 records). [2025-10-20T22:09:56.231448] [INFO ] [MAIN ] HFRO: Successfully saved derivative data. [2025-10-20T22:09:56.233637] [INFO ] [MAIN ] Derivative Progress: 838/1877 symbols completed. HFRO took 1.1s. Estimated remaining: 6m 32s. Rate: 159.1 symbols/min [2025-10-20T22:09:56.268110] [DEBUG] [MAIN ] HESM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:56.271425] [DEBUG] [LOCAL ] HGER: Derivative data saved (1794 records). [2025-10-20T22:09:56.271819] [INFO ] [MAIN ] HGER: Successfully saved derivative data. [2025-10-20T22:09:56.272762] [INFO ] [MAIN ] HESM: Generated 1794 derivative records from 48651 primary records. [2025-10-20T22:09:56.275440] [INFO ] [MAIN ] Derivative Progress: 839/1877 symbols completed. HGER took 1.1s. Estimated remaining: 6m 31s. Rate: 159.3 symbols/min [2025-10-20T22:09:56.275543] [DEBUG] [LOCAL ] HESM: Saving derivative data. [2025-10-20T22:09:56.292831] [DEBUG] [LOCAL ] HESM: Derivative data saved (1794 records). [2025-10-20T22:09:56.292935] [INFO ] [MAIN ] HESM: Successfully saved derivative data. [2025-10-20T22:09:56.295563] [INFO ] [MAIN ] Derivative Progress: 840/1877 symbols completed. HESM took 1.2s. Estimated remaining: 6m 30s. Rate: 159.4 symbols/min [2025-10-20T22:09:56.296203] [INFO ] [MAIN ] HGLB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:56.296331] [INFO ] [MAIN ] HGTY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:56.296749] [INFO ] [MAIN ] HGV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:56.315062] [DEBUG] [LOCAL ] HGLB: Loaded 9696 existing records from primary storage. [2025-10-20T22:09:56.315399] [INFO ] [MAIN ] HGLB: Loaded 9696 primary data records. [2025-10-20T22:09:56.395126] [DEBUG] [LOCAL ] HGTY: Loaded 13382 existing records from primary storage. [2025-10-20T22:09:56.426976] [INFO ] [MAIN ] HGTY: Loaded 13382 primary data records. [2025-10-20T22:09:56.547988] [DEBUG] [LOCAL ] HGV: Loaded 46172 existing records from primary storage. [2025-10-20T22:09:56.560470] [INFO ] [MAIN ] HGV: Loaded 46172 primary data records. [2025-10-20T22:09:57.328547] [DEBUG] [MAIN ] HGLB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:57.329728] [INFO ] [MAIN ] HGLB: Generated 1794 derivative records from 9696 primary records. [2025-10-20T22:09:57.330054] [DEBUG] [LOCAL ] HGLB: Saving derivative data. [2025-10-20T22:09:57.351253] [DEBUG] [LOCAL ] HGLB: Derivative data saved (1794 records). [2025-10-20T22:09:57.352526] [INFO ] [MAIN ] HGLB: Successfully saved derivative data. [2025-10-20T22:09:57.354771] [INFO ] [MAIN ] Derivative Progress: 841/1877 symbols completed. HGLB took 1.1s. Estimated remaining: 6m 31s. Rate: 159.1 symbols/min [2025-10-20T22:09:57.355095] [INFO ] [MAIN ] HHH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:57.370045] [DEBUG] [MAIN ] HGV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:57.372964] [INFO ] [MAIN ] HGV: Generated 1794 derivative records from 46172 primary records. [2025-10-20T22:09:57.373250] [DEBUG] [LOCAL ] HGV: Saving derivative data. [2025-10-20T22:09:57.396088] [DEBUG] [MAIN ] HGTY: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:09:57.403127] [INFO ] [MAIN ] HGTY: Generated 1793 derivative records from 13382 primary records. [2025-10-20T22:09:57.409182] [DEBUG] [LOCAL ] HGTY: Saving derivative data. [2025-10-20T22:09:57.409916] [DEBUG] [LOCAL ] HGV: Derivative data saved (1794 records). [2025-10-20T22:09:57.410228] [INFO ] [MAIN ] HGV: Successfully saved derivative data. [2025-10-20T22:09:57.413026] [INFO ] [MAIN ] Derivative Progress: 842/1877 symbols completed. HGV took 1.1s. Estimated remaining: 6m 30s. Rate: 159.3 symbols/min [2025-10-20T22:09:57.413266] [INFO ] [MAIN ] HI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:57.438716] [DEBUG] [LOCAL ] HGTY: Derivative data saved (1793 records). [2025-10-20T22:09:57.438958] [INFO ] [MAIN ] HGTY: Successfully saved derivative data. [2025-10-20T22:09:57.441878] [INFO ] [MAIN ] Derivative Progress: 843/1877 symbols completed. HGTY took 1.1s. Estimated remaining: 6m 29s. Rate: 159.4 symbols/min [2025-10-20T22:09:57.442147] [INFO ] [MAIN ] HIG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:57.456062] [DEBUG] [LOCAL ] HHH: Loaded 33185 existing records from primary storage. [2025-10-20T22:09:57.456334] [INFO ] [MAIN ] HHH: Loaded 33185 primary data records. [2025-10-20T22:09:57.588586] [DEBUG] [LOCAL ] HI: Loaded 36061 existing records from primary storage. [2025-10-20T22:09:57.599978] [INFO ] [MAIN ] HI: Loaded 36061 primary data records. [2025-10-20T22:09:57.977446] [DEBUG] [LOCAL ] HIG: Loaded 51188 existing records from primary storage. [2025-10-20T22:09:57.999063] [INFO ] [MAIN ] HIG: Loaded 51188 primary data records. [2025-10-20T22:09:58.467772] [DEBUG] [MAIN ] HHH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:58.468899] [INFO ] [MAIN ] HHH: Generated 1794 derivative records from 33185 primary records. [2025-10-20T22:09:58.468993] [DEBUG] [LOCAL ] HHH: Saving derivative data. [2025-10-20T22:09:58.486250] [DEBUG] [LOCAL ] HHH: Derivative data saved (1794 records). [2025-10-20T22:09:58.489025] [INFO ] [MAIN ] HHH: Successfully saved derivative data. [2025-10-20T22:09:58.491718] [INFO ] [MAIN ] Derivative Progress: 844/1877 symbols completed. HHH took 1.1s. Estimated remaining: 6m 30s. Rate: 159.1 symbols/min [2025-10-20T22:09:58.492046] [INFO ] [MAIN ] HII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:58.506075] [DEBUG] [MAIN ] HI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:58.507221] [INFO ] [MAIN ] HI: Generated 1794 derivative records from 36061 primary records. [2025-10-20T22:09:58.507310] [DEBUG] [LOCAL ] HI: Saving derivative data. [2025-10-20T22:09:58.526301] [DEBUG] [LOCAL ] HI: Derivative data saved (1794 records). [2025-10-20T22:09:58.526622] [INFO ] [MAIN ] HI: Successfully saved derivative data. [2025-10-20T22:09:58.536841] [INFO ] [MAIN ] Derivative Progress: 845/1877 symbols completed. HI took 1.1s. Estimated remaining: 6m 29s. Rate: 159.3 symbols/min [2025-10-20T22:09:58.542461] [INFO ] [MAIN ] HIMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:58.542119] [DEBUG] [MAIN ] HIG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:58.543861] [INFO ] [MAIN ] HIG: Generated 1794 derivative records from 51188 primary records. [2025-10-20T22:09:58.544013] [DEBUG] [LOCAL ] HIG: Saving derivative data. [2025-10-20T22:09:58.569880] [DEBUG] [LOCAL ] HIG: Derivative data saved (1794 records). [2025-10-20T22:09:58.570567] [INFO ] [MAIN ] HIG: Successfully saved derivative data. [2025-10-20T22:09:58.572889] [INFO ] [MAIN ] Derivative Progress: 846/1877 symbols completed. HIG took 1.1s. Estimated remaining: 6m 28s. Rate: 159.4 symbols/min [2025-10-20T22:09:58.573158] [INFO ] [MAIN ] HIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:58.585468] [DEBUG] [LOCAL ] HII: Loaded 33610 existing records from primary storage. [2025-10-20T22:09:58.585751] [INFO ] [MAIN ] HII: Loaded 33610 primary data records. [2025-10-20T22:09:58.764496] [DEBUG] [LOCAL ] HIO: Loaded 24124 existing records from primary storage. [2025-10-20T22:09:58.793865] [INFO ] [MAIN ] HIO: Loaded 24124 primary data records. [2025-10-20T22:09:58.981563] [DEBUG] [LOCAL ] HIMS: Loaded 53817 existing records from primary storage. [2025-10-20T22:09:59.001420] [INFO ] [MAIN ] HIMS: Loaded 53817 primary data records. [2025-10-20T22:09:59.582662] [DEBUG] [MAIN ] HII: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:59.585436] [INFO ] [MAIN ] HII: Generated 1794 derivative records from 33610 primary records. [2025-10-20T22:09:59.586113] [DEBUG] [LOCAL ] HII: Saving derivative data. [2025-10-20T22:09:59.603756] [DEBUG] [LOCAL ] HII: Derivative data saved (1794 records). [2025-10-20T22:09:59.604179] [INFO ] [MAIN ] HII: Successfully saved derivative data. [2025-10-20T22:09:59.606520] [INFO ] [MAIN ] Derivative Progress: 847/1877 symbols completed. HII took 1.1s. Estimated remaining: 6m 28s. Rate: 159.1 symbols/min [2025-10-20T22:09:59.607114] [INFO ] [MAIN ] HIPO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:59.627576] [DEBUG] [MAIN ] HIMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:59.636431] [INFO ] [MAIN ] HIMS: Generated 1794 derivative records from 53817 primary records. [2025-10-20T22:09:59.641521] [DEBUG] [LOCAL ] HIMS: Saving derivative data. [2025-10-20T22:09:59.641978] [DEBUG] [MAIN ] HIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:09:59.648675] [INFO ] [MAIN ] HIO: Generated 1794 derivative records from 24124 primary records. [2025-10-20T22:09:59.648791] [DEBUG] [LOCAL ] HIO: Saving derivative data. [2025-10-20T22:09:59.671667] [DEBUG] [LOCAL ] HIMS: Derivative data saved (1794 records). [2025-10-20T22:09:59.678501] [INFO ] [MAIN ] HIMS: Successfully saved derivative data. [2025-10-20T22:09:59.686149] [INFO ] [MAIN ] Derivative Progress: 848/1877 symbols completed. HIMS took 1.1s. Estimated remaining: 6m 28s. Rate: 159.3 symbols/min [2025-10-20T22:09:59.684040] [DEBUG] [LOCAL ] HIPO: Loaded 21623 existing records from primary storage. [2025-10-20T22:09:59.688420] [INFO ] [MAIN ] HIPO: Loaded 21623 primary data records. [2025-10-20T22:09:59.688871] [INFO ] [MAIN ] HIW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:09:59.712368] [DEBUG] [LOCAL ] HIO: Derivative data saved (1794 records). [2025-10-20T22:09:59.712686] [INFO ] [MAIN ] HIO: Successfully saved derivative data. [2025-10-20T22:09:59.730986] [INFO ] [MAIN ] Derivative Progress: 849/1877 symbols completed. HIO took 1.2s. Estimated remaining: 6m 27s. Rate: 159.4 symbols/min [2025-10-20T22:09:59.743350] [INFO ] [MAIN ] HIX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:00.060933] [DEBUG] [LOCAL ] HIX: Loaded 20398 existing records from primary storage. [2025-10-20T22:10:00.062241] [DEBUG] [LOCAL ] HIW: Loaded 44503 existing records from primary storage. [2025-10-20T22:10:00.062761] [INFO ] [MAIN ] HIX: Loaded 20398 primary data records. [2025-10-20T22:10:00.063113] [INFO ] [MAIN ] HIW: Loaded 44503 primary data records. [2025-10-20T22:10:00.723140] [DEBUG] [MAIN ] HIPO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:00.725087] [INFO ] [MAIN ] HIPO: Generated 1794 derivative records from 21623 primary records. [2025-10-20T22:10:00.726002] [DEBUG] [LOCAL ] HIPO: Saving derivative data. [2025-10-20T22:10:00.744492] [DEBUG] [LOCAL ] HIPO: Derivative data saved (1794 records). [2025-10-20T22:10:00.744565] [INFO ] [MAIN ] HIPO: Successfully saved derivative data. [2025-10-20T22:10:00.746857] [INFO ] [MAIN ] Derivative Progress: 850/1877 symbols completed. HIPO took 1.1s. Estimated remaining: 6m 27s. Rate: 159.1 symbols/min [2025-10-20T22:10:00.747449] [INFO ] [MAIN ] HKD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:00.765894] [DEBUG] [MAIN ] HIX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:00.766780] [INFO ] [MAIN ] HIX: Generated 1794 derivative records from 20398 primary records. [2025-10-20T22:10:00.767004] [DEBUG] [LOCAL ] HIX: Saving derivative data. [2025-10-20T22:10:00.790632] [DEBUG] [LOCAL ] HIX: Derivative data saved (1794 records). [2025-10-20T22:10:00.790833] [INFO ] [MAIN ] HIX: Successfully saved derivative data. [2025-10-20T22:10:00.800520] [INFO ] [MAIN ] Derivative Progress: 851/1877 symbols completed. HIX took 1.1s. Estimated remaining: 6m 27s. Rate: 159.3 symbols/min [2025-10-20T22:10:00.812567] [INFO ] [MAIN ] HL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:00.813144] [DEBUG] [LOCAL ] HKD: Loaded 18743 existing records from primary storage. [2025-10-20T22:10:00.813948] [INFO ] [MAIN ] HKD: Loaded 18743 primary data records. [2025-10-20T22:10:00.813542] [DEBUG] [MAIN ] HIW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:00.836533] [INFO ] [MAIN ] HIW: Generated 1794 derivative records from 44503 primary records. [2025-10-20T22:10:00.870769] [DEBUG] [LOCAL ] HIW: Saving derivative data. [2025-10-20T22:10:00.935510] [DEBUG] [LOCAL ] HIW: Derivative data saved (1794 records). [2025-10-20T22:10:00.935790] [INFO ] [MAIN ] HIW: Successfully saved derivative data. [2025-10-20T22:10:00.959082] [INFO ] [MAIN ] Derivative Progress: 852/1877 symbols completed. HIW took 1.3s. Estimated remaining: 6m 26s. Rate: 159.4 symbols/min [2025-10-20T22:10:00.985989] [INFO ] [MAIN ] HLF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:01.180778] [DEBUG] [LOCAL ] HL: Loaded 53676 existing records from primary storage. [2025-10-20T22:10:01.181207] [INFO ] [MAIN ] HL: Loaded 53676 primary data records. [2025-10-20T22:10:01.290352] [DEBUG] [LOCAL ] HLF: Loaded 48540 existing records from primary storage. [2025-10-20T22:10:01.302449] [INFO ] [MAIN ] HLF: Loaded 48540 primary data records. [2025-10-20T22:10:01.665944] [DEBUG] [MAIN ] HKD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:01.672977] [INFO ] [MAIN ] HKD: Generated 1794 derivative records from 18743 primary records. [2025-10-20T22:10:01.694779] [DEBUG] [LOCAL ] HKD: Saving derivative data. [2025-10-20T22:10:01.924072] [DEBUG] [LOCAL ] HKD: Derivative data saved (1794 records). [2025-10-20T22:10:01.945688] [INFO ] [MAIN ] HKD: Successfully saved derivative data. [2025-10-20T22:10:01.957684] [INFO ] [MAIN ] Derivative Progress: 853/1877 symbols completed. HKD took 1.2s. Estimated remaining: 6m 26s. Rate: 159.1 symbols/min [2025-10-20T22:10:01.981304] [DEBUG] [MAIN ] HL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:01.982365] [INFO ] [MAIN ] HL: Generated 1794 derivative records from 53676 primary records. [2025-10-20T22:10:01.982441] [DEBUG] [LOCAL ] HL: Saving derivative data. [2025-10-20T22:10:02.001359] [DEBUG] [LOCAL ] HL: Derivative data saved (1794 records). [2025-10-20T22:10:02.001548] [INFO ] [MAIN ] HL: Successfully saved derivative data. [2025-10-20T22:10:02.003796] [INFO ] [MAIN ] Derivative Progress: 854/1877 symbols completed. HL took 1.2s. Estimated remaining: 6m 25s. Rate: 159.2 symbols/min [2025-10-20T22:10:02.019842] [DEBUG] [MAIN ] HLF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:02.020571] [INFO ] [MAIN ] HLF: Generated 1794 derivative records from 48540 primary records. [2025-10-20T22:10:02.020876] [DEBUG] [LOCAL ] HLF: Saving derivative data. [2025-10-20T22:10:02.036543] [DEBUG] [LOCAL ] HLF: Derivative data saved (1794 records). [2025-10-20T22:10:02.036604] [INFO ] [MAIN ] HLF: Successfully saved derivative data. [2025-10-20T22:10:02.039185] [INFO ] [MAIN ] Derivative Progress: 855/1877 symbols completed. HLF took 1.1s. Estimated remaining: 6m 25s. Rate: 159.4 symbols/min [2025-10-20T22:10:02.040067] [INFO ] [MAIN ] HLLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:02.040001] [INFO ] [MAIN ] HLIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:02.039902] [INFO ] [MAIN ] HLI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:02.087540] [DEBUG] [LOCAL ] HLIO: Loaded 28259 existing records from primary storage. [2025-10-20T22:10:02.098187] [INFO ] [MAIN ] HLIO: Loaded 28259 primary data records. [2025-10-20T22:10:02.099396] [DEBUG] [LOCAL ] HLLY: Loaded 33339 existing records from primary storage. [2025-10-20T22:10:02.141078] [DEBUG] [LOCAL ] HLI: Loaded 30785 existing records from primary storage. [2025-10-20T22:10:02.139817] [INFO ] [MAIN ] HLLY: Loaded 33339 primary data records. [2025-10-20T22:10:02.175348] [INFO ] [MAIN ] HLI: Loaded 30785 primary data records. [2025-10-20T22:10:03.147398] [DEBUG] [MAIN ] HLIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:03.149321] [INFO ] [MAIN ] HLIO: Generated 1794 derivative records from 28259 primary records. [2025-10-20T22:10:03.150477] [DEBUG] [LOCAL ] HLIO: Saving derivative data. [2025-10-20T22:10:03.181866] [DEBUG] [LOCAL ] HLIO: Derivative data saved (1794 records). [2025-10-20T22:10:03.182737] [DEBUG] [MAIN ] HLLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:03.182953] [INFO ] [MAIN ] HLIO: Successfully saved derivative data. [2025-10-20T22:10:03.185816] [INFO ] [MAIN ] HLLY: Generated 1794 derivative records from 33339 primary records. [2025-10-20T22:10:03.185039] [INFO ] [MAIN ] Derivative Progress: 856/1877 symbols completed. HLIO took 1.1s. Estimated remaining: 6m 25s. Rate: 159.0 symbols/min [2025-10-20T22:10:03.186043] [DEBUG] [LOCAL ] HLLY: Saving derivative data. [2025-10-20T22:10:03.186265] [INFO ] [MAIN ] HLN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:03.199992] [DEBUG] [MAIN ] HLI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:03.207880] [INFO ] [MAIN ] HLI: Generated 1794 derivative records from 30785 primary records. [2025-10-20T22:10:03.211942] [DEBUG] [LOCAL ] HLI: Saving derivative data. [2025-10-20T22:10:03.216471] [DEBUG] [LOCAL ] HLLY: Derivative data saved (1794 records). [2025-10-20T22:10:03.217102] [INFO ] [MAIN ] HLLY: Successfully saved derivative data. [2025-10-20T22:10:03.226468] [INFO ] [MAIN ] Derivative Progress: 857/1877 symbols completed. HLLY took 1.2s. Estimated remaining: 6m 24s. Rate: 159.2 symbols/min [2025-10-20T22:10:03.233664] [INFO ] [MAIN ] HLT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:03.234735] [DEBUG] [LOCAL ] HLI: Derivative data saved (1794 records). [2025-10-20T22:10:03.234840] [INFO ] [MAIN ] HLI: Successfully saved derivative data. [2025-10-20T22:10:03.237102] [INFO ] [MAIN ] Derivative Progress: 858/1877 symbols completed. HLI took 1.2s. Estimated remaining: 6m 24s. Rate: 159.4 symbols/min [2025-10-20T22:10:03.237717] [INFO ] [MAIN ] HLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:03.281311] [DEBUG] [LOCAL ] HLN: Loaded 53136 existing records from primary storage. [2025-10-20T22:10:03.281624] [INFO ] [MAIN ] HLN: Loaded 53136 primary data records. [2025-10-20T22:10:03.480060] [DEBUG] [LOCAL ] HLX: Loaded 47458 existing records from primary storage. [2025-10-20T22:10:03.481026] [INFO ] [MAIN ] HLX: Loaded 47458 primary data records. [2025-10-20T22:10:03.480904] [DEBUG] [LOCAL ] HLT: Loaded 50463 existing records from primary storage. [2025-10-20T22:10:03.496009] [INFO ] [MAIN ] HLT: Loaded 50463 primary data records. [2025-10-20T22:10:04.309587] [DEBUG] [MAIN ] HLN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:04.312162] [INFO ] [MAIN ] HLN: Generated 1794 derivative records from 53136 primary records. [2025-10-20T22:10:04.312531] [DEBUG] [LOCAL ] HLN: Saving derivative data. [2025-10-20T22:10:04.330947] [DEBUG] [LOCAL ] HLN: Derivative data saved (1794 records). [2025-10-20T22:10:04.331303] [INFO ] [MAIN ] HLN: Successfully saved derivative data. [2025-10-20T22:10:04.333214] [INFO ] [MAIN ] Derivative Progress: 859/1877 symbols completed. HLN took 1.1s. Estimated remaining: 6m 24s. Rate: 159.0 symbols/min [2025-10-20T22:10:04.333599] [INFO ] [MAIN ] HMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:04.353404] [DEBUG] [MAIN ] HLT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:04.355999] [INFO ] [MAIN ] HLT: Generated 1794 derivative records from 50463 primary records. [2025-10-20T22:10:04.356183] [DEBUG] [LOCAL ] HLT: Saving derivative data. [2025-10-20T22:10:04.386168] [DEBUG] [LOCAL ] HLT: Derivative data saved (1794 records). [2025-10-20T22:10:04.386723] [INFO ] [MAIN ] HLT: Successfully saved derivative data. [2025-10-20T22:10:04.388535] [INFO ] [MAIN ] Derivative Progress: 860/1877 symbols completed. HLT took 1.2s. Estimated remaining: 6m 23s. Rate: 159.2 symbols/min [2025-10-20T22:10:04.386561] [DEBUG] [MAIN ] HLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:04.388924] [INFO ] [MAIN ] HMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:04.389741] [INFO ] [MAIN ] HLX: Generated 1794 derivative records from 47458 primary records. [2025-10-20T22:10:04.390277] [DEBUG] [LOCAL ] HLX: Saving derivative data. [2025-10-20T22:10:04.415745] [DEBUG] [LOCAL ] HLX: Derivative data saved (1794 records). [2025-10-20T22:10:04.415896] [INFO ] [MAIN ] HLX: Successfully saved derivative data. [2025-10-20T22:10:04.417833] [INFO ] [MAIN ] Derivative Progress: 861/1877 symbols completed. HLX took 1.2s. Estimated remaining: 6m 23s. Rate: 159.3 symbols/min [2025-10-20T22:10:04.418149] [INFO ] [MAIN ] HMY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:04.438528] [DEBUG] [LOCAL ] HMC: Loaded 46338 existing records from primary storage. [2025-10-20T22:10:04.438806] [INFO ] [MAIN ] HMC: Loaded 46338 primary data records. [2025-10-20T22:10:04.547925] [DEBUG] [LOCAL ] HMN: Loaded 23459 existing records from primary storage. [2025-10-20T22:10:04.548220] [INFO ] [MAIN ] HMN: Loaded 23459 primary data records. [2025-10-20T22:10:04.961191] [DEBUG] [LOCAL ] HMY: Loaded 53166 existing records from primary storage. [2025-10-20T22:10:05.074167] [INFO ] [MAIN ] HMY: Loaded 53166 primary data records. [2025-10-20T22:10:05.413949] [DEBUG] [MAIN ] HMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:05.415975] [INFO ] [MAIN ] HMC: Generated 1794 derivative records from 46338 primary records. [2025-10-20T22:10:05.416324] [DEBUG] [LOCAL ] HMC: Saving derivative data. [2025-10-20T22:10:05.440320] [DEBUG] [LOCAL ] HMC: Derivative data saved (1794 records). [2025-10-20T22:10:05.453606] [INFO ] [MAIN ] HMC: Successfully saved derivative data. [2025-10-20T22:10:05.454067] [DEBUG] [MAIN ] HMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:05.454907] [INFO ] [MAIN ] HMN: Generated 1794 derivative records from 23459 primary records. [2025-10-20T22:10:05.456846] [INFO ] [MAIN ] Derivative Progress: 862/1877 symbols completed. HMC took 1.1s. Estimated remaining: 6m 23s. Rate: 159.0 symbols/min [2025-10-20T22:10:05.457589] [INFO ] [MAIN ] HNI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:05.457509] [DEBUG] [LOCAL ] HMN: Saving derivative data. [2025-10-20T22:10:05.476889] [DEBUG] [LOCAL ] HMN: Derivative data saved (1794 records). [2025-10-20T22:10:05.477182] [INFO ] [MAIN ] HMN: Successfully saved derivative data. [2025-10-20T22:10:05.479327] [INFO ] [MAIN ] Derivative Progress: 863/1877 symbols completed. HMN took 1.1s. Estimated remaining: 6m 22s. Rate: 159.2 symbols/min [2025-10-20T22:10:05.479759] [INFO ] [MAIN ] HOG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:05.501596] [DEBUG] [MAIN ] HMY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:05.502385] [INFO ] [MAIN ] HMY: Generated 1794 derivative records from 53166 primary records. [2025-10-20T22:10:05.502564] [DEBUG] [LOCAL ] HMY: Saving derivative data. [2025-10-20T22:10:05.526124] [DEBUG] [LOCAL ] HMY: Derivative data saved (1794 records). [2025-10-20T22:10:05.526363] [INFO ] [MAIN ] HMY: Successfully saved derivative data. [2025-10-20T22:10:05.528806] [INFO ] [MAIN ] Derivative Progress: 864/1877 symbols completed. HMY took 1.1s. Estimated remaining: 6m 21s. Rate: 159.3 symbols/min [2025-10-20T22:10:05.531475] [INFO ] [MAIN ] HOMB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:05.542423] [DEBUG] [LOCAL ] HNI: Loaded 29783 existing records from primary storage. [2025-10-20T22:10:05.542708] [INFO ] [MAIN ] HNI: Loaded 29783 primary data records. [2025-10-20T22:10:05.881150] [DEBUG] [LOCAL ] HOG: Loaded 50715 existing records from primary storage. [2025-10-20T22:10:05.894001] [INFO ] [MAIN ] HOG: Loaded 50715 primary data records. [2025-10-20T22:10:05.906433] [DEBUG] [LOCAL ] HOMB: Loaded 43754 existing records from primary storage. [2025-10-20T22:10:05.916091] [INFO ] [MAIN ] HOMB: Loaded 43754 primary data records. [2025-10-20T22:10:06.532699] [DEBUG] [MAIN ] HNI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:06.533694] [INFO ] [MAIN ] HNI: Generated 1794 derivative records from 29783 primary records. [2025-10-20T22:10:06.533887] [DEBUG] [LOCAL ] HNI: Saving derivative data. [2025-10-20T22:10:06.550701] [DEBUG] [LOCAL ] HNI: Derivative data saved (1794 records). [2025-10-20T22:10:06.550996] [INFO ] [MAIN ] HNI: Successfully saved derivative data. [2025-10-20T22:10:06.553039] [INFO ] [MAIN ] Derivative Progress: 865/1877 symbols completed. HNI took 1.1s. Estimated remaining: 6m 22s. Rate: 159.0 symbols/min [2025-10-20T22:10:06.553612] [INFO ] [MAIN ] HOUS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:06.573425] [DEBUG] [MAIN ] HOG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:06.586800] [INFO ] [MAIN ] HOG: Generated 1794 derivative records from 50715 primary records. [2025-10-20T22:10:06.586100] [DEBUG] [MAIN ] HOMB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:06.587402] [INFO ] [MAIN ] HOMB: Generated 1794 derivative records from 43754 primary records. [2025-10-20T22:10:06.587488] [DEBUG] [LOCAL ] HOG: Saving derivative data. [2025-10-20T22:10:06.587565] [DEBUG] [LOCAL ] HOMB: Saving derivative data. [2025-10-20T22:10:06.626113] [DEBUG] [LOCAL ] HOMB: Derivative data saved (1794 records). [2025-10-20T22:10:06.627481] [INFO ] [MAIN ] HOMB: Successfully saved derivative data. [2025-10-20T22:10:06.629371] [INFO ] [MAIN ] Derivative Progress: 866/1877 symbols completed. HOMB took 1.1s. Estimated remaining: 6m 21s. Rate: 159.2 symbols/min [2025-10-20T22:10:06.630005] [INFO ] [MAIN ] HOV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:06.629911] [DEBUG] [LOCAL ] HOG: Derivative data saved (1794 records). [2025-10-20T22:10:06.630268] [INFO ] [MAIN ] HOG: Successfully saved derivative data. [2025-10-20T22:10:06.632267] [INFO ] [MAIN ] Derivative Progress: 867/1877 symbols completed. HOG took 1.2s. Estimated remaining: 6m 20s. Rate: 159.4 symbols/min [2025-10-20T22:10:06.633047] [INFO ] [MAIN ] HP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:06.647608] [DEBUG] [LOCAL ] HOUS: Loaded 45656 existing records from primary storage. [2025-10-20T22:10:06.647900] [INFO ] [MAIN ] HOUS: Loaded 45656 primary data records. [2025-10-20T22:10:06.748230] [DEBUG] [LOCAL ] HOV: Loaded 14023 existing records from primary storage. [2025-10-20T22:10:06.759495] [INFO ] [MAIN ] HOV: Loaded 14023 primary data records. [2025-10-20T22:10:07.274385] [DEBUG] [LOCAL ] HP: Loaded 51484 existing records from primary storage. [2025-10-20T22:10:07.274905] [INFO ] [MAIN ] HP: Loaded 51484 primary data records. [2025-10-20T22:10:07.610460] [DEBUG] [MAIN ] HOUS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:07.612429] [INFO ] [MAIN ] HOUS: Generated 1794 derivative records from 45656 primary records. [2025-10-20T22:10:07.612736] [DEBUG] [LOCAL ] HOUS: Saving derivative data. [2025-10-20T22:10:07.636580] [DEBUG] [LOCAL ] HOUS: Derivative data saved (1794 records). [2025-10-20T22:10:07.636962] [INFO ] [MAIN ] HOUS: Successfully saved derivative data. [2025-10-20T22:10:07.652580] [DEBUG] [MAIN ] HOV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:07.653436] [INFO ] [MAIN ] HOV: Generated 1793 derivative records from 14023 primary records. [2025-10-20T22:10:07.653883] [DEBUG] [LOCAL ] HOV: Saving derivative data. [2025-10-20T22:10:07.651755] [INFO ] [MAIN ] Derivative Progress: 868/1877 symbols completed. HOUS took 1.1s. Estimated remaining: 6m 21s. Rate: 159.0 symbols/min [2025-10-20T22:10:07.671122] [DEBUG] [LOCAL ] HOV: Derivative data saved (1793 records). [2025-10-20T22:10:07.671209] [INFO ] [MAIN ] HOV: Successfully saved derivative data. [2025-10-20T22:10:07.673731] [INFO ] [MAIN ] Derivative Progress: 869/1877 symbols completed. HOV took 1.0s. Estimated remaining: 6m 20s. Rate: 159.2 symbols/min [2025-10-20T22:10:07.689699] [DEBUG] [MAIN ] HP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:07.690299] [INFO ] [MAIN ] HP: Generated 1794 derivative records from 51484 primary records. [2025-10-20T22:10:07.690358] [DEBUG] [LOCAL ] HP: Saving derivative data. [2025-10-20T22:10:07.705389] [DEBUG] [LOCAL ] HP: Derivative data saved (1794 records). [2025-10-20T22:10:07.705450] [INFO ] [MAIN ] HP: Successfully saved derivative data. [2025-10-20T22:10:07.707414] [INFO ] [MAIN ] Derivative Progress: 870/1877 symbols completed. HP took 1.1s. Estimated remaining: 6m 19s. Rate: 159.4 symbols/min [2025-10-20T22:10:07.708221] [INFO ] [MAIN ] HPF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:07.708101] [INFO ] [MAIN ] HPE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:07.708542] [INFO ] [MAIN ] HPI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:07.723268] [DEBUG] [LOCAL ] HPF: Loaded 7516 existing records from primary storage. [2025-10-20T22:10:07.723465] [INFO ] [MAIN ] HPF: Loaded 7516 primary data records. [2025-10-20T22:10:07.759202] [DEBUG] [LOCAL ] HPI: Loaded 10790 existing records from primary storage. [2025-10-20T22:10:07.791671] [INFO ] [MAIN ] HPI: Loaded 10790 primary data records. [2025-10-20T22:10:08.383736] [DEBUG] [LOCAL ] HPE: Loaded 53697 existing records from primary storage. [2025-10-20T22:10:08.397233] [INFO ] [MAIN ] HPE: Loaded 53697 primary data records. [2025-10-20T22:10:08.737488] [DEBUG] [MAIN ] HPF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:08.740091] [INFO ] [MAIN ] HPF: Generated 1794 derivative records from 7516 primary records. [2025-10-20T22:10:08.740344] [DEBUG] [LOCAL ] HPF: Saving derivative data. [2025-10-20T22:10:08.765891] [DEBUG] [MAIN ] HPI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:08.769583] [INFO ] [MAIN ] HPI: Generated 1794 derivative records from 10790 primary records. [2025-10-20T22:10:08.773524] [DEBUG] [LOCAL ] HPF: Derivative data saved (1794 records). [2025-10-20T22:10:08.773809] [DEBUG] [LOCAL ] HPI: Saving derivative data. [2025-10-20T22:10:08.774077] [INFO ] [MAIN ] HPF: Successfully saved derivative data. [2025-10-20T22:10:08.777095] [INFO ] [MAIN ] Derivative Progress: 871/1877 symbols completed. HPF took 1.1s. Estimated remaining: 6m 20s. Rate: 159.0 symbols/min [2025-10-20T22:10:08.777520] [INFO ] [MAIN ] HPP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:08.795488] [DEBUG] [LOCAL ] HPI: Derivative data saved (1794 records). [2025-10-20T22:10:08.795870] [INFO ] [MAIN ] HPI: Successfully saved derivative data. [2025-10-20T22:10:08.798286] [INFO ] [MAIN ] Derivative Progress: 872/1877 symbols completed. HPI took 1.1s. Estimated remaining: 6m 19s. Rate: 159.2 symbols/min [2025-10-20T22:10:08.798616] [INFO ] [MAIN ] HPQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:08.816341] [DEBUG] [MAIN ] HPE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:08.817209] [INFO ] [MAIN ] HPE: Generated 1794 derivative records from 53697 primary records. [2025-10-20T22:10:08.817366] [DEBUG] [LOCAL ] HPE: Saving derivative data. [2025-10-20T22:10:08.834246] [DEBUG] [LOCAL ] HPE: Derivative data saved (1794 records). [2025-10-20T22:10:08.834549] [INFO ] [MAIN ] HPE: Successfully saved derivative data. [2025-10-20T22:10:08.836432] [INFO ] [MAIN ] Derivative Progress: 873/1877 symbols completed. HPE took 1.1s. Estimated remaining: 6m 18s. Rate: 159.4 symbols/min [2025-10-20T22:10:08.836703] [INFO ] [MAIN ] HPS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:08.874157] [DEBUG] [LOCAL ] HPS: Loaded 10377 existing records from primary storage. [2025-10-20T22:10:08.874355] [INFO ] [MAIN ] HPS: Loaded 10377 primary data records. [2025-10-20T22:10:08.961367] [DEBUG] [LOCAL ] HPP: Loaded 51262 existing records from primary storage. [2025-10-20T22:10:08.968314] [INFO ] [MAIN ] HPP: Loaded 51262 primary data records. [2025-10-20T22:10:09.047218] [DEBUG] [LOCAL ] HPQ: Loaded 53669 existing records from primary storage. [2025-10-20T22:10:09.054543] [INFO ] [MAIN ] HPQ: Loaded 53669 primary data records. [2025-10-20T22:10:09.831670] [DEBUG] [MAIN ] HPS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:09.833376] [INFO ] [MAIN ] HPS: Generated 1794 derivative records from 10377 primary records. [2025-10-20T22:10:09.833600] [DEBUG] [LOCAL ] HPS: Saving derivative data. [2025-10-20T22:10:09.849915] [DEBUG] [LOCAL ] HPS: Derivative data saved (1794 records). [2025-10-20T22:10:09.850103] [INFO ] [MAIN ] HPS: Successfully saved derivative data. [2025-10-20T22:10:09.865048] [INFO ] [MAIN ] Derivative Progress: 874/1877 symbols completed. HPS took 1.0s. Estimated remaining: 6m 18s. Rate: 159.1 symbols/min [2025-10-20T22:10:09.866189] [INFO ] [MAIN ] HQH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:09.865368] [DEBUG] [MAIN ] HPP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:09.868150] [INFO ] [MAIN ] HPP: Generated 1794 derivative records from 51262 primary records. [2025-10-20T22:10:09.868266] [DEBUG] [LOCAL ] HPP: Saving derivative data. [2025-10-20T22:10:09.892934] [DEBUG] [MAIN ] HPQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:09.895823] [INFO ] [MAIN ] HPQ: Generated 1794 derivative records from 53669 primary records. [2025-10-20T22:10:09.896825] [DEBUG] [LOCAL ] HPQ: Saving derivative data. [2025-10-20T22:10:09.897830] [DEBUG] [LOCAL ] HPP: Derivative data saved (1794 records). [2025-10-20T22:10:09.898034] [INFO ] [MAIN ] HPP: Successfully saved derivative data. [2025-10-20T22:10:09.900580] [INFO ] [MAIN ] Derivative Progress: 875/1877 symbols completed. HPP took 1.1s. Estimated remaining: 6m 18s. Rate: 159.2 symbols/min [2025-10-20T22:10:09.900967] [INFO ] [MAIN ] HQL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:09.916251] [DEBUG] [LOCAL ] HPQ: Derivative data saved (1794 records). [2025-10-20T22:10:09.916336] [INFO ] [MAIN ] HPQ: Successfully saved derivative data. [2025-10-20T22:10:09.918430] [INFO ] [MAIN ] Derivative Progress: 876/1877 symbols completed. HPQ took 1.1s. Estimated remaining: 6m 17s. Rate: 159.4 symbols/min [2025-10-20T22:10:09.918566] [INFO ] [MAIN ] HR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:09.949450] [DEBUG] [LOCAL ] HQH: Loaded 25838 existing records from primary storage. [2025-10-20T22:10:09.949888] [DEBUG] [LOCAL ] HQL: Loaded 16958 existing records from primary storage. [2025-10-20T22:10:09.950102] [INFO ] [MAIN ] HQH: Loaded 25838 primary data records. [2025-10-20T22:10:09.950210] [INFO ] [MAIN ] HQL: Loaded 16958 primary data records. [2025-10-20T22:10:10.337624] [DEBUG] [LOCAL ] HR: Loaded 52115 existing records from primary storage. [2025-10-20T22:10:10.346015] [INFO ] [MAIN ] HR: Loaded 52115 primary data records. [2025-10-20T22:10:10.943015] [DEBUG] [MAIN ] HQH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:10.957066] [INFO ] [MAIN ] HQH: Generated 1794 derivative records from 25838 primary records. [2025-10-20T22:10:10.956209] [DEBUG] [MAIN ] HQL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:10.958100] [INFO ] [MAIN ] HQL: Generated 1794 derivative records from 16958 primary records. [2025-10-20T22:10:10.958261] [DEBUG] [LOCAL ] HQH: Saving derivative data. [2025-10-20T22:10:10.958364] [DEBUG] [LOCAL ] HQL: Saving derivative data. [2025-10-20T22:10:10.984471] [DEBUG] [LOCAL ] HQH: Derivative data saved (1794 records). [2025-10-20T22:10:10.988836] [INFO ] [MAIN ] HQH: Successfully saved derivative data. [2025-10-20T22:10:10.994470] [INFO ] [MAIN ] Derivative Progress: 877/1877 symbols completed. HQH took 1.1s. Estimated remaining: 6m 17s. Rate: 159.1 symbols/min [2025-10-20T22:10:10.992486] [DEBUG] [LOCAL ] HQL: Derivative data saved (1794 records). [2025-10-20T22:10:10.994768] [INFO ] [MAIN ] HRB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:10.994977] [INFO ] [MAIN ] HQL: Successfully saved derivative data. [2025-10-20T22:10:10.997908] [INFO ] [MAIN ] Derivative Progress: 878/1877 symbols completed. HQL took 1.1s. Estimated remaining: 6m 16s. Rate: 159.2 symbols/min [2025-10-20T22:10:10.998580] [INFO ] [MAIN ] HRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:11.017114] [DEBUG] [MAIN ] HR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:11.017713] [INFO ] [MAIN ] HR: Generated 1794 derivative records from 52115 primary records. [2025-10-20T22:10:11.017829] [DEBUG] [LOCAL ] HR: Saving derivative data. [2025-10-20T22:10:11.034168] [DEBUG] [LOCAL ] HR: Derivative data saved (1794 records). [2025-10-20T22:10:11.034443] [INFO ] [MAIN ] HR: Successfully saved derivative data. [2025-10-20T22:10:11.036431] [INFO ] [MAIN ] Derivative Progress: 879/1877 symbols completed. HR took 1.1s. Estimated remaining: 6m 16s. Rate: 159.4 symbols/min [2025-10-20T22:10:11.036800] [INFO ] [MAIN ] HRL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:11.090127] [DEBUG] [LOCAL ] HRI: Loaded 32387 existing records from primary storage. [2025-10-20T22:10:11.089604] [DEBUG] [LOCAL ] HRB: Loaded 48355 existing records from primary storage. [2025-10-20T22:10:11.090358] [INFO ] [MAIN ] HRB: Loaded 48355 primary data records. [2025-10-20T22:10:11.090419] [INFO ] [MAIN ] HRI: Loaded 32387 primary data records. [2025-10-20T22:10:11.371156] [DEBUG] [LOCAL ] HRL: Loaded 53128 existing records from primary storage. [2025-10-20T22:10:11.415752] [INFO ] [MAIN ] HRL: Loaded 53128 primary data records. [2025-10-20T22:10:12.101326] [DEBUG] [MAIN ] HRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:12.115457] [INFO ] [MAIN ] HRI: Generated 1794 derivative records from 32387 primary records. [2025-10-20T22:10:12.115790] [DEBUG] [MAIN ] HRB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:12.117617] [DEBUG] [LOCAL ] HRI: Saving derivative data. [2025-10-20T22:10:12.119027] [INFO ] [MAIN ] HRB: Generated 1794 derivative records from 48355 primary records. [2025-10-20T22:10:12.119986] [DEBUG] [LOCAL ] HRB: Saving derivative data. [2025-10-20T22:10:12.155899] [DEBUG] [LOCAL ] HRI: Derivative data saved (1794 records). [2025-10-20T22:10:12.160112] [INFO ] [MAIN ] HRI: Successfully saved derivative data. [2025-10-20T22:10:12.166249] [INFO ] [MAIN ] Derivative Progress: 880/1877 symbols completed. HRI took 1.2s. Estimated remaining: 6m 16s. Rate: 159.0 symbols/min [2025-10-20T22:10:12.166689] [INFO ] [MAIN ] HRTG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:12.167746] [DEBUG] [LOCAL ] HRB: Derivative data saved (1794 records). [2025-10-20T22:10:12.160446] [DEBUG] [MAIN ] HRL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:12.168045] [INFO ] [MAIN ] HRB: Successfully saved derivative data. [2025-10-20T22:10:12.170891] [INFO ] [MAIN ] Derivative Progress: 881/1877 symbols completed. HRB took 1.2s. Estimated remaining: 6m 15s. Rate: 159.2 symbols/min [2025-10-20T22:10:12.168869] [INFO ] [MAIN ] HRL: Generated 1794 derivative records from 53128 primary records. [2025-10-20T22:10:12.171162] [INFO ] [MAIN ] HSBC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:12.171768] [DEBUG] [LOCAL ] HRL: Saving derivative data. [2025-10-20T22:10:12.188672] [DEBUG] [LOCAL ] HRL: Derivative data saved (1794 records). [2025-10-20T22:10:12.188819] [INFO ] [MAIN ] HRL: Successfully saved derivative data. [2025-10-20T22:10:12.191008] [INFO ] [MAIN ] Derivative Progress: 882/1877 symbols completed. HRL took 1.2s. Estimated remaining: 6m 15s. Rate: 159.4 symbols/min [2025-10-20T22:10:12.191256] [INFO ] [MAIN ] HSHP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:12.214936] [DEBUG] [LOCAL ] HSHP: Loaded 14817 existing records from primary storage. [2025-10-20T22:10:12.215070] [INFO ] [MAIN ] HSHP: Loaded 14817 primary data records. [2025-10-20T22:10:12.314156] [DEBUG] [LOCAL ] HRTG: Loaded 34587 existing records from primary storage. [2025-10-20T22:10:12.330059] [INFO ] [MAIN ] HRTG: Loaded 34587 primary data records. [2025-10-20T22:10:12.545340] [DEBUG] [LOCAL ] HSBC: Loaded 50998 existing records from primary storage. [2025-10-20T22:10:12.669604] [INFO ] [MAIN ] HSBC: Loaded 50998 primary data records. [2025-10-20T22:10:13.183840] [DEBUG] [MAIN ] HSHP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:13.185403] [INFO ] [MAIN ] HSHP: Generated 1794 derivative records from 14817 primary records. [2025-10-20T22:10:13.187004] [DEBUG] [LOCAL ] HSHP: Saving derivative data. [2025-10-20T22:10:13.215829] [DEBUG] [LOCAL ] HSHP: Derivative data saved (1794 records). [2025-10-20T22:10:13.214956] [DEBUG] [MAIN ] HRTG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:13.216092] [INFO ] [MAIN ] HSHP: Successfully saved derivative data. [2025-10-20T22:10:13.220201] [INFO ] [MAIN ] Derivative Progress: 883/1877 symbols completed. HSHP took 1.0s. Estimated remaining: 6m 15s. Rate: 159.1 symbols/min [2025-10-20T22:10:13.217681] [INFO ] [MAIN ] HRTG: Generated 1794 derivative records from 34587 primary records. [2025-10-20T22:10:13.220346] [DEBUG] [LOCAL ] HRTG: Saving derivative data. [2025-10-20T22:10:13.247698] [DEBUG] [MAIN ] HSBC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:13.249177] [INFO ] [MAIN ] HSBC: Generated 1794 derivative records from 50998 primary records. [2025-10-20T22:10:13.250194] [DEBUG] [LOCAL ] HRTG: Derivative data saved (1794 records). [2025-10-20T22:10:13.250265] [DEBUG] [LOCAL ] HSBC: Saving derivative data. [2025-10-20T22:10:13.250323] [INFO ] [MAIN ] HRTG: Successfully saved derivative data. [2025-10-20T22:10:13.252540] [INFO ] [MAIN ] Derivative Progress: 884/1877 symbols completed. HRTG took 1.1s. Estimated remaining: 6m 14s. Rate: 159.3 symbols/min [2025-10-20T22:10:13.267483] [DEBUG] [LOCAL ] HSBC: Derivative data saved (1794 records). [2025-10-20T22:10:13.267544] [INFO ] [MAIN ] HSBC: Successfully saved derivative data. [2025-10-20T22:10:13.269500] [INFO ] [MAIN ] Derivative Progress: 885/1877 symbols completed. HSBC took 1.1s. Estimated remaining: 6m 13s. Rate: 159.4 symbols/min [2025-10-20T22:10:13.269877] [INFO ] [MAIN ] HSY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:13.269990] [INFO ] [MAIN ] HTGC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:13.270232] [INFO ] [MAIN ] HTD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:13.287991] [DEBUG] [LOCAL ] HTD: Loaded 14305 existing records from primary storage. [2025-10-20T22:10:13.288262] [INFO ] [MAIN ] HTD: Loaded 14305 primary data records. [2025-10-20T22:10:13.627052] [DEBUG] [LOCAL ] HTGC: Loaded 47761 existing records from primary storage. [2025-10-20T22:10:13.637611] [INFO ] [MAIN ] HTGC: Loaded 47761 primary data records. [2025-10-20T22:10:13.637366] [DEBUG] [LOCAL ] HSY: Loaded 50468 existing records from primary storage. [2025-10-20T22:10:13.637959] [INFO ] [MAIN ] HSY: Loaded 50468 primary data records. [2025-10-20T22:10:14.295141] [DEBUG] [MAIN ] HTD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:14.296113] [INFO ] [MAIN ] HTD: Generated 1794 derivative records from 14305 primary records. [2025-10-20T22:10:14.296323] [DEBUG] [LOCAL ] HTD: Saving derivative data. [2025-10-20T22:10:14.315335] [DEBUG] [LOCAL ] HTD: Derivative data saved (1794 records). [2025-10-20T22:10:14.315693] [INFO ] [MAIN ] HTD: Successfully saved derivative data. [2025-10-20T22:10:14.318312] [INFO ] [MAIN ] Derivative Progress: 886/1877 symbols completed. HTD took 1.0s. Estimated remaining: 6m 14s. Rate: 159.1 symbols/min [2025-10-20T22:10:14.326202] [INFO ] [MAIN ] HTH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:14.332059] [DEBUG] [MAIN ] HTGC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:14.333016] [INFO ] [MAIN ] HTGC: Generated 1794 derivative records from 47761 primary records. [2025-10-20T22:10:14.333351] [DEBUG] [LOCAL ] HTGC: Saving derivative data. [2025-10-20T22:10:14.349958] [DEBUG] [LOCAL ] HTGC: Derivative data saved (1794 records). [2025-10-20T22:10:14.350145] [INFO ] [MAIN ] HTGC: Successfully saved derivative data. [2025-10-20T22:10:14.352472] [INFO ] [MAIN ] Derivative Progress: 887/1877 symbols completed. HTGC took 1.1s. Estimated remaining: 6m 13s. Rate: 159.3 symbols/min [2025-10-20T22:10:14.354599] [INFO ] [MAIN ] HUBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:14.369427] [DEBUG] [MAIN ] HSY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:14.370176] [INFO ] [MAIN ] HSY: Generated 1794 derivative records from 50468 primary records. [2025-10-20T22:10:14.370413] [DEBUG] [LOCAL ] HSY: Saving derivative data. [2025-10-20T22:10:14.404722] [DEBUG] [LOCAL ] HSY: Derivative data saved (1794 records). [2025-10-20T22:10:14.404990] [INFO ] [MAIN ] HSY: Successfully saved derivative data. [2025-10-20T22:10:14.406966] [INFO ] [MAIN ] Derivative Progress: 888/1877 symbols completed. HSY took 1.1s. Estimated remaining: 6m 12s. Rate: 159.4 symbols/min [2025-10-20T22:10:14.407108] [INFO ] [MAIN ] HUBS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:14.407880] [DEBUG] [LOCAL ] HTH: Loaded 32664 existing records from primary storage. [2025-10-20T22:10:14.408363] [INFO ] [MAIN ] HTH: Loaded 32664 primary data records. [2025-10-20T22:10:14.595732] [DEBUG] [LOCAL ] HUBB: Loaded 36261 existing records from primary storage. [2025-10-20T22:10:14.618563] [INFO ] [MAIN ] HUBB: Loaded 36261 primary data records. [2025-10-20T22:10:14.854455] [DEBUG] [LOCAL ] HUBS: Loaded 40086 existing records from primary storage. [2025-10-20T22:10:14.860324] [INFO ] [MAIN ] HUBS: Loaded 40086 primary data records. [2025-10-20T22:10:15.364607] [DEBUG] [MAIN ] HTH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:15.365617] [INFO ] [MAIN ] HTH: Generated 1794 derivative records from 32664 primary records. [2025-10-20T22:10:15.366034] [DEBUG] [LOCAL ] HTH: Saving derivative data. [2025-10-20T22:10:15.383755] [DEBUG] [LOCAL ] HTH: Derivative data saved (1794 records). [2025-10-20T22:10:15.384232] [INFO ] [MAIN ] HTH: Successfully saved derivative data. [2025-10-20T22:10:15.386679] [INFO ] [MAIN ] Derivative Progress: 889/1877 symbols completed. HTH took 1.1s. Estimated remaining: 6m 13s. Rate: 159.1 symbols/min [2025-10-20T22:10:15.387037] [INFO ] [MAIN ] HUM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:15.404022] [DEBUG] [MAIN ] HUBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:15.405208] [INFO ] [MAIN ] HUBB: Generated 1794 derivative records from 36261 primary records. [2025-10-20T22:10:15.405347] [DEBUG] [LOCAL ] HUBB: Saving derivative data. [2025-10-20T22:10:15.424097] [DEBUG] [LOCAL ] HUBB: Derivative data saved (1794 records). [2025-10-20T22:10:15.424274] [INFO ] [MAIN ] HUBB: Successfully saved derivative data. [2025-10-20T22:10:15.426472] [INFO ] [MAIN ] Derivative Progress: 890/1877 symbols completed. HUBB took 1.1s. Estimated remaining: 6m 12s. Rate: 159.3 symbols/min [2025-10-20T22:10:15.439323] [INFO ] [MAIN ] HUN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:15.439937] [DEBUG] [MAIN ] HUBS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:15.441030] [INFO ] [MAIN ] HUBS: Generated 1794 derivative records from 40086 primary records. [2025-10-20T22:10:15.441101] [DEBUG] [LOCAL ] HUBS: Saving derivative data. [2025-10-20T22:10:15.457902] [DEBUG] [LOCAL ] HUBS: Derivative data saved (1794 records). [2025-10-20T22:10:15.468200] [INFO ] [MAIN ] HUBS: Successfully saved derivative data. [2025-10-20T22:10:15.470242] [INFO ] [MAIN ] Derivative Progress: 891/1877 symbols completed. HUBS took 1.1s. Estimated remaining: 6m 11s. Rate: 159.5 symbols/min [2025-10-20T22:10:15.470491] [INFO ] [MAIN ] HUYA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:15.503803] [DEBUG] [LOCAL ] HUM: Loaded 49347 existing records from primary storage. [2025-10-20T22:10:15.504056] [INFO ] [MAIN ] HUM: Loaded 49347 primary data records. [2025-10-20T22:10:15.622512] [DEBUG] [LOCAL ] HUN: Loaded 53219 existing records from primary storage. [2025-10-20T22:10:15.638525] [INFO ] [MAIN ] HUN: Loaded 53219 primary data records. [2025-10-20T22:10:15.992065] [DEBUG] [LOCAL ] HUYA: Loaded 43607 existing records from primary storage. [2025-10-20T22:10:16.015408] [INFO ] [MAIN ] HUYA: Loaded 43607 primary data records. [2025-10-20T22:10:16.493759] [DEBUG] [MAIN ] HUM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:16.495382] [INFO ] [MAIN ] HUM: Generated 1794 derivative records from 49347 primary records. [2025-10-20T22:10:16.495546] [DEBUG] [LOCAL ] HUM: Saving derivative data. [2025-10-20T22:10:16.515537] [DEBUG] [LOCAL ] HUM: Derivative data saved (1794 records). [2025-10-20T22:10:16.528240] [INFO ] [MAIN ] HUM: Successfully saved derivative data. [2025-10-20T22:10:16.528926] [DEBUG] [MAIN ] HUN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:16.531967] [INFO ] [MAIN ] HUN: Generated 1794 derivative records from 53219 primary records. [2025-10-20T22:10:16.530972] [INFO ] [MAIN ] Derivative Progress: 892/1877 symbols completed. HUM took 1.1s. Estimated remaining: 6m 11s. Rate: 159.1 symbols/min [2025-10-20T22:10:16.532190] [INFO ] [MAIN ] HVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:16.532101] [DEBUG] [LOCAL ] HUN: Saving derivative data. [2025-10-20T22:10:16.551346] [DEBUG] [LOCAL ] HUN: Derivative data saved (1794 records). [2025-10-20T22:10:16.551499] [INFO ] [MAIN ] HUN: Successfully saved derivative data. [2025-10-20T22:10:16.553491] [INFO ] [MAIN ] Derivative Progress: 893/1877 symbols completed. HUN took 1.1s. Estimated remaining: 6m 11s. Rate: 159.3 symbols/min [2025-10-20T22:10:16.554002] [INFO ] [MAIN ] HWM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:16.567962] [DEBUG] [MAIN ] HUYA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:16.568677] [INFO ] [MAIN ] HUYA: Generated 1794 derivative records from 43607 primary records. [2025-10-20T22:10:16.568805] [DEBUG] [LOCAL ] HUYA: Saving derivative data. [2025-10-20T22:10:16.589714] [DEBUG] [LOCAL ] HUYA: Derivative data saved (1794 records). [2025-10-20T22:10:16.595065] [INFO ] [MAIN ] HUYA: Successfully saved derivative data. [2025-10-20T22:10:16.597558] [INFO ] [MAIN ] Derivative Progress: 894/1877 symbols completed. HUYA took 1.1s. Estimated remaining: 6m 10s. Rate: 159.5 symbols/min [2025-10-20T22:10:16.597753] [INFO ] [MAIN ] HXL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:16.598198] [DEBUG] [LOCAL ] HVT: Loaded 14991 existing records from primary storage. [2025-10-20T22:10:16.598774] [INFO ] [MAIN ] HVT: Loaded 14991 primary data records. [2025-10-20T22:10:16.955536] [DEBUG] [LOCAL ] HWM: Loaded 52717 existing records from primary storage. [2025-10-20T22:10:16.955823] [INFO ] [MAIN ] HWM: Loaded 52717 primary data records. [2025-10-20T22:10:16.956463] [DEBUG] [LOCAL ] HXL: Loaded 45814 existing records from primary storage. [2025-10-20T22:10:16.969055] [INFO ] [MAIN ] HXL: Loaded 45814 primary data records. [2025-10-20T22:10:17.597001] [DEBUG] [MAIN ] HVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:17.598500] [INFO ] [MAIN ] HVT: Generated 1794 derivative records from 14991 primary records. [2025-10-20T22:10:17.598671] [DEBUG] [LOCAL ] HVT: Saving derivative data. [2025-10-20T22:10:17.617154] [DEBUG] [LOCAL ] HVT: Derivative data saved (1794 records). [2025-10-20T22:10:17.618676] [INFO ] [MAIN ] HVT: Successfully saved derivative data. [2025-10-20T22:10:17.620965] [INFO ] [MAIN ] Derivative Progress: 895/1877 symbols completed. HVT took 1.1s. Estimated remaining: 6m 10s. Rate: 159.1 symbols/min [2025-10-20T22:10:17.621108] [INFO ] [MAIN ] HY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:17.650618] [DEBUG] [LOCAL ] HY: Loaded 10046 existing records from primary storage. [2025-10-20T22:10:17.663455] [DEBUG] [MAIN ] HWM: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:17.664825] [INFO ] [MAIN ] HWM: Generated 1793 derivative records from 52717 primary records. [2025-10-20T22:10:17.665304] [DEBUG] [LOCAL ] HWM: Saving derivative data. [2025-10-20T22:10:17.665208] [DEBUG] [MAIN ] HXL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:17.665531] [INFO ] [MAIN ] HY: Loaded 10046 primary data records. [2025-10-20T22:10:17.672541] [INFO ] [MAIN ] HXL: Generated 1794 derivative records from 45814 primary records. [2025-10-20T22:10:17.679908] [DEBUG] [LOCAL ] HXL: Saving derivative data. [2025-10-20T22:10:17.753474] [DEBUG] [LOCAL ] HWM: Derivative data saved (1793 records). [2025-10-20T22:10:17.759305] [INFO ] [MAIN ] HWM: Successfully saved derivative data. [2025-10-20T22:10:17.809457] [INFO ] [MAIN ] Derivative Progress: 896/1877 symbols completed. HWM took 1.3s. Estimated remaining: 6m 10s. Rate: 159.2 symbols/min [2025-10-20T22:10:17.814402] [INFO ] [MAIN ] HYI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:17.814259] [DEBUG] [LOCAL ] HXL: Derivative data saved (1794 records). [2025-10-20T22:10:17.840633] [INFO ] [MAIN ] HXL: Successfully saved derivative data. [2025-10-20T22:10:17.855938] [INFO ] [MAIN ] Derivative Progress: 897/1877 symbols completed. HXL took 1.3s. Estimated remaining: 6m 9s. Rate: 159.4 symbols/min [2025-10-20T22:10:17.888244] [INFO ] [MAIN ] HYT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:18.037098] [DEBUG] [LOCAL ] HYI: Loaded 9771 existing records from primary storage. [2025-10-20T22:10:18.037430] [INFO ] [MAIN ] HYI: Loaded 9771 primary data records. [2025-10-20T22:10:18.355175] [DEBUG] [LOCAL ] HYT: Loaded 33697 existing records from primary storage. [2025-10-20T22:10:18.355884] [INFO ] [MAIN ] HYT: Loaded 33697 primary data records. [2025-10-20T22:10:18.662440] [DEBUG] [MAIN ] HY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:18.671980] [INFO ] [MAIN ] HY: Generated 1794 derivative records from 10046 primary records. [2025-10-20T22:10:18.672535] [DEBUG] [LOCAL ] HY: Saving derivative data. [2025-10-20T22:10:18.693261] [DEBUG] [LOCAL ] HY: Derivative data saved (1794 records). [2025-10-20T22:10:18.705933] [INFO ] [MAIN ] HY: Successfully saved derivative data. [2025-10-20T22:10:18.708226] [INFO ] [MAIN ] Derivative Progress: 898/1877 symbols completed. HY took 1.1s. Estimated remaining: 6m 9s. Rate: 159.2 symbols/min [2025-10-20T22:10:18.709151] [DEBUG] [MAIN ] HYI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:18.709892] [INFO ] [MAIN ] HYI: Generated 1794 derivative records from 9771 primary records. [2025-10-20T22:10:18.709997] [DEBUG] [LOCAL ] HYI: Saving derivative data. [2025-10-20T22:10:18.726171] [DEBUG] [LOCAL ] HYI: Derivative data saved (1794 records). [2025-10-20T22:10:18.726243] [INFO ] [MAIN ] HYI: Successfully saved derivative data. [2025-10-20T22:10:18.728371] [INFO ] [MAIN ] Derivative Progress: 899/1877 symbols completed. HYI took 0.9s. Estimated remaining: 6m 8s. Rate: 159.3 symbols/min [2025-10-20T22:10:18.744111] [DEBUG] [MAIN ] HYT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:18.744605] [INFO ] [MAIN ] HYT: Generated 1794 derivative records from 33697 primary records. [2025-10-20T22:10:18.744687] [DEBUG] [LOCAL ] HYT: Saving derivative data. [2025-10-20T22:10:18.759542] [DEBUG] [LOCAL ] HYT: Derivative data saved (1794 records). [2025-10-20T22:10:18.759597] [INFO ] [MAIN ] HYT: Successfully saved derivative data. [2025-10-20T22:10:18.761764] [INFO ] [MAIN ] Derivative Progress: 900/1877 symbols completed. HYT took 0.9s. Estimated remaining: 6m 8s. Rate: 159.5 symbols/min [2025-10-20T22:10:18.762163] [INFO ] [MAIN ] HYTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:18.762322] [INFO ] [MAIN ] HZO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:18.762227] [INFO ] [MAIN ] IAE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:18.768440] [DEBUG] [LOCAL ] HYTR: Loaded 3098 existing records from primary storage. [2025-10-20T22:10:18.770268] [INFO ] [MAIN ] HYTR: Loaded 3098 primary data records. [2025-10-20T22:10:18.822143] [DEBUG] [LOCAL ] IAE: Loaded 6325 existing records from primary storage. [2025-10-20T22:10:18.822236] [INFO ] [MAIN ] IAE: Loaded 6325 primary data records. [2025-10-20T22:10:18.970884] [DEBUG] [LOCAL ] HZO: Loaded 28914 existing records from primary storage. [2025-10-20T22:10:19.000481] [INFO ] [MAIN ] HZO: Loaded 28914 primary data records. [2025-10-20T22:10:19.773866] [DEBUG] [MAIN ] HYTR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:19.774979] [INFO ] [MAIN ] HYTR: Generated 1793 derivative records from 3098 primary records. [2025-10-20T22:10:19.775244] [DEBUG] [LOCAL ] HYTR: Saving derivative data. [2025-10-20T22:10:19.791045] [DEBUG] [LOCAL ] HYTR: Derivative data saved (1793 records). [2025-10-20T22:10:19.792063] [INFO ] [MAIN ] HYTR: Successfully saved derivative data. [2025-10-20T22:10:19.794970] [INFO ] [MAIN ] Derivative Progress: 901/1877 symbols completed. HYTR took 1.0s. Estimated remaining: 6m 8s. Rate: 159.2 symbols/min [2025-10-20T22:10:19.795193] [INFO ] [MAIN ] IAG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:19.809858] [DEBUG] [MAIN ] IAE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:19.811829] [INFO ] [MAIN ] IAE: Generated 1794 derivative records from 6325 primary records. [2025-10-20T22:10:19.812065] [DEBUG] [LOCAL ] IAE: Saving derivative data. [2025-10-20T22:10:19.831372] [DEBUG] [MAIN ] HZO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:19.833473] [INFO ] [MAIN ] HZO: Generated 1794 derivative records from 28914 primary records. [2025-10-20T22:10:19.839871] [DEBUG] [LOCAL ] IAE: Derivative data saved (1794 records). [2025-10-20T22:10:19.839966] [DEBUG] [LOCAL ] HZO: Saving derivative data. [2025-10-20T22:10:19.840091] [INFO ] [MAIN ] IAE: Successfully saved derivative data. [2025-10-20T22:10:19.842846] [INFO ] [MAIN ] Derivative Progress: 902/1877 symbols completed. IAE took 1.1s. Estimated remaining: 6m 7s. Rate: 159.3 symbols/min [2025-10-20T22:10:19.843249] [INFO ] [MAIN ] IBM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:19.859263] [DEBUG] [LOCAL ] HZO: Derivative data saved (1794 records). [2025-10-20T22:10:19.859469] [INFO ] [MAIN ] HZO: Successfully saved derivative data. [2025-10-20T22:10:19.861840] [INFO ] [MAIN ] Derivative Progress: 903/1877 symbols completed. HZO took 1.1s. Estimated remaining: 6m 6s. Rate: 159.5 symbols/min [2025-10-20T22:10:19.862209] [INFO ] [MAIN ] IBN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:19.905771] [DEBUG] [LOCAL ] IAG: Loaded 53491 existing records from primary storage. [2025-10-20T22:10:19.906123] [INFO ] [MAIN ] IAG: Loaded 53491 primary data records. [2025-10-20T22:10:20.089271] [DEBUG] [LOCAL ] IBM: Loaded 53629 existing records from primary storage. [2025-10-20T22:10:20.136924] [INFO ] [MAIN ] IBM: Loaded 53629 primary data records. [2025-10-20T22:10:20.147983] [DEBUG] [LOCAL ] IBN: Loaded 52844 existing records from primary storage. [2025-10-20T22:10:20.154361] [INFO ] [MAIN ] IBN: Loaded 52844 primary data records. [2025-10-20T22:10:20.915211] [DEBUG] [MAIN ] IAG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:20.916202] [INFO ] [MAIN ] IAG: Generated 1794 derivative records from 53491 primary records. [2025-10-20T22:10:20.916340] [DEBUG] [LOCAL ] IAG: Saving derivative data. [2025-10-20T22:10:20.936324] [DEBUG] [LOCAL ] IAG: Derivative data saved (1794 records). [2025-10-20T22:10:20.936456] [INFO ] [MAIN ] IAG: Successfully saved derivative data. [2025-10-20T22:10:20.938604] [INFO ] [MAIN ] Derivative Progress: 904/1877 symbols completed. IAG took 1.1s. Estimated remaining: 6m 7s. Rate: 159.2 symbols/min [2025-10-20T22:10:20.939353] [INFO ] [MAIN ] IBP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:20.976183] [DEBUG] [MAIN ] IBM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:20.977211] [INFO ] [MAIN ] IBM: Generated 1794 derivative records from 53629 primary records. [2025-10-20T22:10:20.977599] [DEBUG] [LOCAL ] IBM: Saving derivative data. [2025-10-20T22:10:20.977458] [DEBUG] [MAIN ] IBN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:20.978689] [INFO ] [MAIN ] IBN: Generated 1794 derivative records from 52844 primary records. [2025-10-20T22:10:20.978784] [DEBUG] [LOCAL ] IBN: Saving derivative data. [2025-10-20T22:10:21.007781] [DEBUG] [LOCAL ] IBN: Derivative data saved (1794 records). [2025-10-20T22:10:21.015821] [INFO ] [MAIN ] IBN: Successfully saved derivative data. [2025-10-20T22:10:21.019080] [INFO ] [MAIN ] Derivative Progress: 905/1877 symbols completed. IBN took 1.2s. Estimated remaining: 6m 6s. Rate: 159.3 symbols/min [2025-10-20T22:10:21.019327] [INFO ] [MAIN ] IBTA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:21.022875] [DEBUG] [LOCAL ] IBM: Derivative data saved (1794 records). [2025-10-20T22:10:21.022997] [INFO ] [MAIN ] IBM: Successfully saved derivative data. [2025-10-20T22:10:21.025157] [INFO ] [MAIN ] Derivative Progress: 906/1877 symbols completed. IBM took 1.2s. Estimated remaining: 6m 5s. Rate: 159.5 symbols/min [2025-10-20T22:10:21.025449] [INFO ] [MAIN ] ICE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:21.036926] [DEBUG] [LOCAL ] IBP: Loaded 28823 existing records from primary storage. [2025-10-20T22:10:21.037100] [INFO ] [MAIN ] IBP: Loaded 28823 primary data records. [2025-10-20T22:10:21.357259] [DEBUG] [LOCAL ] IBTA: Loaded 36338 existing records from primary storage. [2025-10-20T22:10:21.370498] [INFO ] [MAIN ] IBTA: Loaded 36338 primary data records. [2025-10-20T22:10:21.510503] [DEBUG] [LOCAL ] ICE: Loaded 53012 existing records from primary storage. [2025-10-20T22:10:21.527444] [INFO ] [MAIN ] ICE: Loaded 53012 primary data records. [2025-10-20T22:10:22.048865] [DEBUG] [MAIN ] IBP: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:22.050382] [INFO ] [MAIN ] IBP: Generated 1793 derivative records from 28823 primary records. [2025-10-20T22:10:22.051257] [DEBUG] [LOCAL ] IBP: Saving derivative data. [2025-10-20T22:10:22.067868] [DEBUG] [LOCAL ] IBP: Derivative data saved (1793 records). [2025-10-20T22:10:22.068129] [INFO ] [MAIN ] IBP: Successfully saved derivative data. [2025-10-20T22:10:22.070566] [INFO ] [MAIN ] Derivative Progress: 907/1877 symbols completed. IBP took 1.1s. Estimated remaining: 6m 6s. Rate: 159.2 symbols/min [2025-10-20T22:10:22.070877] [INFO ] [MAIN ] ICL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:22.087763] [DEBUG] [MAIN ] IBTA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:22.088563] [INFO ] [MAIN ] IBTA: Generated 1794 derivative records from 36338 primary records. [2025-10-20T22:10:22.088965] [DEBUG] [LOCAL ] IBTA: Saving derivative data. [2025-10-20T22:10:22.105888] [DEBUG] [LOCAL ] IBTA: Derivative data saved (1794 records). [2025-10-20T22:10:22.106475] [INFO ] [MAIN ] IBTA: Successfully saved derivative data. [2025-10-20T22:10:22.108631] [INFO ] [MAIN ] Derivative Progress: 908/1877 symbols completed. IBTA took 1.1s. Estimated remaining: 6m 5s. Rate: 159.3 symbols/min [2025-10-20T22:10:22.108825] [INFO ] [MAIN ] IDA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:22.134075] [DEBUG] [MAIN ] ICE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:22.134777] [INFO ] [MAIN ] ICE: Generated 1794 derivative records from 53012 primary records. [2025-10-20T22:10:22.134900] [DEBUG] [LOCAL ] ICE: Saving derivative data. [2025-10-20T22:10:22.151104] [DEBUG] [LOCAL ] ICE: Derivative data saved (1794 records). [2025-10-20T22:10:22.162562] [INFO ] [MAIN ] ICE: Successfully saved derivative data. [2025-10-20T22:10:22.166013] [DEBUG] [LOCAL ] ICL: Loaded 38359 existing records from primary storage. [2025-10-20T22:10:22.164881] [INFO ] [MAIN ] Derivative Progress: 909/1877 symbols completed. ICE took 1.1s. Estimated remaining: 6m 4s. Rate: 159.5 symbols/min [2025-10-20T22:10:22.166433] [INFO ] [MAIN ] IDE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:22.166285] [INFO ] [MAIN ] ICL: Loaded 38359 primary data records. [2025-10-20T22:10:22.285829] [DEBUG] [LOCAL ] IDE: Loaded 9091 existing records from primary storage. [2025-10-20T22:10:22.305740] [INFO ] [MAIN ] IDE: Loaded 9091 primary data records. [2025-10-20T22:10:22.352463] [DEBUG] [LOCAL ] IDA: Loaded 32795 existing records from primary storage. [2025-10-20T22:10:22.352587] [INFO ] [MAIN ] IDA: Loaded 32795 primary data records. [2025-10-20T22:10:23.116138] [DEBUG] [MAIN ] ICL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:23.116895] [INFO ] [MAIN ] ICL: Generated 1794 derivative records from 38359 primary records. [2025-10-20T22:10:23.117121] [DEBUG] [LOCAL ] ICL: Saving derivative data. [2025-10-20T22:10:23.133028] [DEBUG] [LOCAL ] ICL: Derivative data saved (1794 records). [2025-10-20T22:10:23.133143] [INFO ] [MAIN ] ICL: Successfully saved derivative data. [2025-10-20T22:10:23.135044] [INFO ] [MAIN ] Derivative Progress: 910/1877 symbols completed. ICL took 1.1s. Estimated remaining: 6m 4s. Rate: 159.2 symbols/min [2025-10-20T22:10:23.135333] [INFO ] [MAIN ] IDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:23.165369] [DEBUG] [MAIN ] IDE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:23.165537] [DEBUG] [MAIN ] IDA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:23.166325] [INFO ] [MAIN ] IDE: Generated 1794 derivative records from 9091 primary records. [2025-10-20T22:10:23.167009] [INFO ] [MAIN ] IDA: Generated 1794 derivative records from 32795 primary records. [2025-10-20T22:10:23.167091] [DEBUG] [LOCAL ] IDE: Saving derivative data. [2025-10-20T22:10:23.167570] [DEBUG] [LOCAL ] IDA: Saving derivative data. [2025-10-20T22:10:23.199476] [DEBUG] [LOCAL ] IDA: Derivative data saved (1794 records). [2025-10-20T22:10:23.201604] [INFO ] [MAIN ] IDA: Successfully saved derivative data. [2025-10-20T22:10:23.203670] [INFO ] [MAIN ] Derivative Progress: 911/1877 symbols completed. IDA took 1.1s. Estimated remaining: 6m 4s. Rate: 159.4 symbols/min [2025-10-20T22:10:23.209506] [INFO ] [MAIN ] IEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:23.211031] [DEBUG] [LOCAL ] IDT: Loaded 18522 existing records from primary storage. [2025-10-20T22:10:23.211922] [DEBUG] [LOCAL ] IDE: Derivative data saved (1794 records). [2025-10-20T22:10:23.212245] [INFO ] [MAIN ] IDE: Successfully saved derivative data. [2025-10-20T22:10:23.212305] [INFO ] [MAIN ] IDT: Loaded 18522 primary data records. [2025-10-20T22:10:23.214339] [INFO ] [MAIN ] Derivative Progress: 912/1877 symbols completed. IDE took 1.0s. Estimated remaining: 6m 3s. Rate: 159.5 symbols/min [2025-10-20T22:10:23.214550] [INFO ] [MAIN ] IFF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:23.551633] [DEBUG] [LOCAL ] IEX: Loaded 38423 existing records from primary storage. [2025-10-20T22:10:23.552463] [INFO ] [MAIN ] IEX: Loaded 38423 primary data records. [2025-10-20T22:10:23.687420] [DEBUG] [LOCAL ] IFF: Loaded 51225 existing records from primary storage. [2025-10-20T22:10:23.704486] [INFO ] [MAIN ] IFF: Loaded 51225 primary data records. [2025-10-20T22:10:24.233900] [DEBUG] [MAIN ] IDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:24.234958] [INFO ] [MAIN ] IDT: Generated 1794 derivative records from 18522 primary records. [2025-10-20T22:10:24.235083] [DEBUG] [LOCAL ] IDT: Saving derivative data. [2025-10-20T22:10:24.252707] [DEBUG] [LOCAL ] IDT: Derivative data saved (1794 records). [2025-10-20T22:10:24.254172] [INFO ] [MAIN ] IDT: Successfully saved derivative data. [2025-10-20T22:10:24.256923] [INFO ] [MAIN ] Derivative Progress: 913/1877 symbols completed. IDT took 1.1s. Estimated remaining: 6m 3s. Rate: 159.2 symbols/min [2025-10-20T22:10:24.273441] [DEBUG] [MAIN ] IEX: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:24.274239] [INFO ] [MAIN ] IEX: Generated 1793 derivative records from 38423 primary records. [2025-10-20T22:10:24.274426] [DEBUG] [LOCAL ] IEX: Saving derivative data. [2025-10-20T22:10:24.290398] [DEBUG] [LOCAL ] IEX: Derivative data saved (1793 records). [2025-10-20T22:10:24.290464] [INFO ] [MAIN ] IEX: Successfully saved derivative data. [2025-10-20T22:10:24.292335] [INFO ] [MAIN ] Derivative Progress: 914/1877 symbols completed. IEX took 1.1s. Estimated remaining: 6m 3s. Rate: 159.4 symbols/min [2025-10-20T22:10:24.306916] [DEBUG] [MAIN ] IFF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:24.307418] [INFO ] [MAIN ] IFF: Generated 1794 derivative records from 51225 primary records. [2025-10-20T22:10:24.307474] [DEBUG] [LOCAL ] IFF: Saving derivative data. [2025-10-20T22:10:24.322897] [DEBUG] [LOCAL ] IFF: Derivative data saved (1794 records). [2025-10-20T22:10:24.322958] [INFO ] [MAIN ] IFF: Successfully saved derivative data. [2025-10-20T22:10:24.325557] [INFO ] [MAIN ] Derivative Progress: 915/1877 symbols completed. IFF took 1.1s. Estimated remaining: 6m 2s. Rate: 159.5 symbols/min [2025-10-20T22:10:24.325932] [INFO ] [MAIN ] IFN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:24.326013] [INFO ] [MAIN ] IGA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:24.326535] [INFO ] [MAIN ] IFS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:24.349535] [DEBUG] [LOCAL ] IGA: Loaded 10268 existing records from primary storage. [2025-10-20T22:10:24.349780] [INFO ] [MAIN ] IGA: Loaded 10268 primary data records. [2025-10-20T22:10:24.429313] [DEBUG] [LOCAL ] IFS: Loaded 22330 existing records from primary storage. [2025-10-20T22:10:24.441940] [DEBUG] [LOCAL ] IFN: Loaded 18570 existing records from primary storage. [2025-10-20T22:10:24.466430] [INFO ] [MAIN ] IFN: Loaded 18570 primary data records. [2025-10-20T22:10:24.458771] [INFO ] [MAIN ] IFS: Loaded 22330 primary data records. [2025-10-20T22:10:25.285308] [DEBUG] [MAIN ] IGA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:25.286223] [INFO ] [MAIN ] IGA: Generated 1794 derivative records from 10268 primary records. [2025-10-20T22:10:25.286553] [DEBUG] [LOCAL ] IGA: Saving derivative data. [2025-10-20T22:10:25.302121] [DEBUG] [LOCAL ] IGA: Derivative data saved (1794 records). [2025-10-20T22:10:25.302305] [INFO ] [MAIN ] IGA: Successfully saved derivative data. [2025-10-20T22:10:25.304257] [INFO ] [MAIN ] Derivative Progress: 916/1877 symbols completed. IGA took 1.0s. Estimated remaining: 6m 2s. Rate: 159.3 symbols/min [2025-10-20T22:10:25.304507] [INFO ] [MAIN ] IGD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:25.337742] [DEBUG] [MAIN ] IFS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:25.338599] [DEBUG] [MAIN ] IFN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:25.344722] [INFO ] [MAIN ] IFN: Generated 1794 derivative records from 18570 primary records. [2025-10-20T22:10:25.344856] [DEBUG] [LOCAL ] IFN: Saving derivative data. [2025-10-20T22:10:25.339575] [INFO ] [MAIN ] IFS: Generated 1794 derivative records from 22330 primary records. [2025-10-20T22:10:25.345031] [DEBUG] [LOCAL ] IFS: Saving derivative data. [2025-10-20T22:10:25.383988] [DEBUG] [LOCAL ] IFS: Derivative data saved (1794 records). [2025-10-20T22:10:25.386611] [DEBUG] [LOCAL ] IGD: Loaded 19266 existing records from primary storage. [2025-10-20T22:10:25.386853] [INFO ] [MAIN ] IGD: Loaded 19266 primary data records. [2025-10-20T22:10:25.386965] [INFO ] [MAIN ] IFS: Successfully saved derivative data. [2025-10-20T22:10:25.385874] [DEBUG] [LOCAL ] IFN: Derivative data saved (1794 records). [2025-10-20T22:10:25.389301] [INFO ] [MAIN ] Derivative Progress: 917/1877 symbols completed. IFS took 1.1s. Estimated remaining: 6m 1s. Rate: 159.4 symbols/min [2025-10-20T22:10:25.394512] [INFO ] [MAIN ] IFN: Successfully saved derivative data. [2025-10-20T22:10:25.429610] [INFO ] [MAIN ] IGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:25.440104] [INFO ] [MAIN ] Derivative Progress: 918/1877 symbols completed. IFN took 1.1s. Estimated remaining: 6m 1s. Rate: 159.5 symbols/min [2025-10-20T22:10:25.458938] [INFO ] [MAIN ] IGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:25.519206] [DEBUG] [LOCAL ] IGI: Loaded 4077 existing records from primary storage. [2025-10-20T22:10:25.541003] [INFO ] [MAIN ] IGI: Loaded 4077 primary data records. [2025-10-20T22:10:25.861208] [DEBUG] [LOCAL ] IGR: Loaded 30502 existing records from primary storage. [2025-10-20T22:10:25.867445] [INFO ] [MAIN ] IGR: Loaded 30502 primary data records. [2025-10-20T22:10:26.378784] [DEBUG] [MAIN ] IGD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:26.380431] [INFO ] [MAIN ] IGD: Generated 1794 derivative records from 19266 primary records. [2025-10-20T22:10:26.380818] [DEBUG] [LOCAL ] IGD: Saving derivative data. [2025-10-20T22:10:26.398894] [DEBUG] [LOCAL ] IGD: Derivative data saved (1794 records). [2025-10-20T22:10:26.399126] [INFO ] [MAIN ] IGD: Successfully saved derivative data. [2025-10-20T22:10:26.402717] [INFO ] [MAIN ] Derivative Progress: 919/1877 symbols completed. IGD took 1.1s. Estimated remaining: 6m 1s. Rate: 159.3 symbols/min [2025-10-20T22:10:26.402981] [INFO ] [MAIN ] IH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:26.417361] [DEBUG] [MAIN ] IGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:26.418948] [INFO ] [MAIN ] IGI: Generated 1794 derivative records from 4077 primary records. [2025-10-20T22:10:26.419051] [DEBUG] [LOCAL ] IGI: Saving derivative data. [2025-10-20T22:10:26.432478] [DEBUG] [LOCAL ] IH: Loaded 2768 existing records from primary storage. [2025-10-20T22:10:26.438611] [DEBUG] [LOCAL ] IGI: Derivative data saved (1794 records). [2025-10-20T22:10:26.438830] [INFO ] [MAIN ] IGI: Successfully saved derivative data. [2025-10-20T22:10:26.438758] [INFO ] [MAIN ] IH: Loaded 2768 primary data records. [2025-10-20T22:10:26.441184] [INFO ] [MAIN ] Derivative Progress: 920/1877 symbols completed. IGI took 1.0s. Estimated remaining: 6m 0s. Rate: 159.4 symbols/min [2025-10-20T22:10:26.452700] [INFO ] [MAIN ] IHD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:26.497367] [DEBUG] [MAIN ] IGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:26.530720] [INFO ] [MAIN ] IGR: Generated 1794 derivative records from 30502 primary records. [2025-10-20T22:10:26.548786] [DEBUG] [LOCAL ] IGR: Saving derivative data. [2025-10-20T22:10:26.605112] [DEBUG] [LOCAL ] IHD: Loaded 7362 existing records from primary storage. [2025-10-20T22:10:26.605813] [DEBUG] [LOCAL ] IGR: Derivative data saved (1794 records). [2025-10-20T22:10:26.605911] [INFO ] [MAIN ] IGR: Successfully saved derivative data. [2025-10-20T22:10:26.608855] [INFO ] [MAIN ] Derivative Progress: 921/1877 symbols completed. IGR took 1.1s. Estimated remaining: 5m 60s. Rate: 159.5 symbols/min [2025-10-20T22:10:26.605999] [INFO ] [MAIN ] IHD: Loaded 7362 primary data records. [2025-10-20T22:10:26.609045] [INFO ] [MAIN ] IHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:27.095357] [DEBUG] [LOCAL ] IHG: Loaded 24215 existing records from primary storage. [2025-10-20T22:10:27.109093] [INFO ] [MAIN ] IHG: Loaded 24215 primary data records. [2025-10-20T22:10:27.463387] [DEBUG] [MAIN ] IH: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:27.464327] [INFO ] [MAIN ] IH: Generated 1793 derivative records from 2768 primary records. [2025-10-20T22:10:27.464422] [DEBUG] [LOCAL ] IH: Saving derivative data. [2025-10-20T22:10:27.480483] [DEBUG] [LOCAL ] IH: Derivative data saved (1793 records). [2025-10-20T22:10:27.482397] [INFO ] [MAIN ] IH: Successfully saved derivative data. [2025-10-20T22:10:27.484778] [INFO ] [MAIN ] Derivative Progress: 922/1877 symbols completed. IH took 1.1s. Estimated remaining: 5m 60s. Rate: 159.3 symbols/min [2025-10-20T22:10:27.484949] [INFO ] [MAIN ] IHS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:27.499778] [DEBUG] [MAIN ] IHD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:27.501669] [INFO ] [MAIN ] IHD: Generated 1794 derivative records from 7362 primary records. [2025-10-20T22:10:27.501978] [DEBUG] [LOCAL ] IHD: Saving derivative data. [2025-10-20T22:10:27.517674] [DEBUG] [LOCAL ] IHD: Derivative data saved (1794 records). [2025-10-20T22:10:27.517899] [INFO ] [MAIN ] IHD: Successfully saved derivative data. [2025-10-20T22:10:27.519898] [INFO ] [MAIN ] Derivative Progress: 923/1877 symbols completed. IHD took 1.1s. Estimated remaining: 5m 59s. Rate: 159.4 symbols/min [2025-10-20T22:10:27.520321] [INFO ] [MAIN ] IIF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:27.546900] [DEBUG] [MAIN ] IHG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:27.550210] [DEBUG] [LOCAL ] IIF: Loaded 5448 existing records from primary storage. [2025-10-20T22:10:27.551171] [INFO ] [MAIN ] IHG: Generated 1794 derivative records from 24215 primary records. [2025-10-20T22:10:27.551314] [DEBUG] [LOCAL ] IHG: Saving derivative data. [2025-10-20T22:10:27.551415] [INFO ] [MAIN ] IIF: Loaded 5448 primary data records. [2025-10-20T22:10:27.621608] [DEBUG] [LOCAL ] IHG: Derivative data saved (1794 records). [2025-10-20T22:10:27.642950] [DEBUG] [LOCAL ] IHS: Loaded 37267 existing records from primary storage. [2025-10-20T22:10:27.648290] [INFO ] [MAIN ] IHG: Successfully saved derivative data. [2025-10-20T22:10:27.657061] [INFO ] [MAIN ] Derivative Progress: 924/1877 symbols completed. IHG took 1.0s. Estimated remaining: 5m 58s. Rate: 159.6 symbols/min [2025-10-20T22:10:27.662351] [INFO ] [MAIN ] IIIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:27.662606] [INFO ] [MAIN ] IHS: Loaded 37267 primary data records. [2025-10-20T22:10:28.071538] [DEBUG] [LOCAL ] IIIN: Loaded 16310 existing records from primary storage. [2025-10-20T22:10:28.095168] [INFO ] [MAIN ] IIIN: Loaded 16310 primary data records. [2025-10-20T22:10:28.550562] [DEBUG] [MAIN ] IIF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:28.559205] [INFO ] [MAIN ] IIF: Generated 1794 derivative records from 5448 primary records. [2025-10-20T22:10:28.564682] [DEBUG] [LOCAL ] IIF: Saving derivative data. [2025-10-20T22:10:28.565156] [DEBUG] [MAIN ] IHS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:28.566713] [INFO ] [MAIN ] IHS: Generated 1794 derivative records from 37267 primary records. [2025-10-20T22:10:28.566871] [DEBUG] [LOCAL ] IHS: Saving derivative data. [2025-10-20T22:10:28.591055] [DEBUG] [LOCAL ] IIF: Derivative data saved (1794 records). [2025-10-20T22:10:28.591322] [INFO ] [MAIN ] IIF: Successfully saved derivative data. [2025-10-20T22:10:28.593565] [INFO ] [MAIN ] Derivative Progress: 925/1877 symbols completed. IIF took 1.1s. Estimated remaining: 5m 59s. Rate: 159.3 symbols/min [2025-10-20T22:10:28.596508] [INFO ] [MAIN ] IIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:28.601279] [DEBUG] [LOCAL ] IHS: Derivative data saved (1794 records). [2025-10-20T22:10:28.601510] [INFO ] [MAIN ] IHS: Successfully saved derivative data. [2025-10-20T22:10:28.603561] [INFO ] [MAIN ] Derivative Progress: 926/1877 symbols completed. IHS took 1.1s. Estimated remaining: 5m 58s. Rate: 159.5 symbols/min [2025-10-20T22:10:28.603940] [INFO ] [MAIN ] IIPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:28.626236] [DEBUG] [MAIN ] IIIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:28.627008] [INFO ] [MAIN ] IIIN: Generated 1794 derivative records from 16310 primary records. [2025-10-20T22:10:28.627193] [DEBUG] [LOCAL ] IIIN: Saving derivative data. [2025-10-20T22:10:28.641201] [DEBUG] [LOCAL ] IIM: Loaded 15541 existing records from primary storage. [2025-10-20T22:10:28.645347] [INFO ] [MAIN ] IIM: Loaded 15541 primary data records. [2025-10-20T22:10:28.660455] [DEBUG] [LOCAL ] IIIN: Derivative data saved (1794 records). [2025-10-20T22:10:28.685000] [INFO ] [MAIN ] IIIN: Successfully saved derivative data. [2025-10-20T22:10:28.712990] [INFO ] [MAIN ] Derivative Progress: 927/1877 symbols completed. IIIN took 1.1s. Estimated remaining: 5m 57s. Rate: 159.6 symbols/min [2025-10-20T22:10:28.746702] [INFO ] [MAIN ] IMAX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:28.825775] [DEBUG] [LOCAL ] IIPR: Loaded 29071 existing records from primary storage. [2025-10-20T22:10:28.836336] [INFO ] [MAIN ] IIPR: Loaded 29071 primary data records. [2025-10-20T22:10:29.281941] [DEBUG] [LOCAL ] IMAX: Loaded 43484 existing records from primary storage. [2025-10-20T22:10:29.283461] [INFO ] [MAIN ] IMAX: Loaded 43484 primary data records. [2025-10-20T22:10:29.655027] [DEBUG] [MAIN ] IIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:29.655838] [INFO ] [MAIN ] IIM: Generated 1794 derivative records from 15541 primary records. [2025-10-20T22:10:29.655938] [DEBUG] [LOCAL ] IIM: Saving derivative data. [2025-10-20T22:10:29.674528] [DEBUG] [LOCAL ] IIM: Derivative data saved (1794 records). [2025-10-20T22:10:29.675843] [INFO ] [MAIN ] IIM: Successfully saved derivative data. [2025-10-20T22:10:29.678521] [INFO ] [MAIN ] Derivative Progress: 928/1877 symbols completed. IIM took 1.1s. Estimated remaining: 5m 57s. Rate: 159.3 symbols/min [2025-10-20T22:10:29.692693] [DEBUG] [MAIN ] IIPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:29.694404] [INFO ] [MAIN ] IIPR: Generated 1794 derivative records from 29071 primary records. [2025-10-20T22:10:29.694526] [DEBUG] [LOCAL ] IIPR: Saving derivative data. [2025-10-20T22:10:29.710973] [DEBUG] [LOCAL ] IIPR: Derivative data saved (1794 records). [2025-10-20T22:10:29.711060] [INFO ] [MAIN ] IIPR: Successfully saved derivative data. [2025-10-20T22:10:29.713168] [INFO ] [MAIN ] Derivative Progress: 929/1877 symbols completed. IIPR took 1.1s. Estimated remaining: 5m 57s. Rate: 159.5 symbols/min [2025-10-20T22:10:29.728539] [DEBUG] [MAIN ] IMAX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:29.729106] [INFO ] [MAIN ] IMAX: Generated 1794 derivative records from 43484 primary records. [2025-10-20T22:10:29.729165] [DEBUG] [LOCAL ] IMAX: Saving derivative data. [2025-10-20T22:10:29.744300] [DEBUG] [LOCAL ] IMAX: Derivative data saved (1794 records). [2025-10-20T22:10:29.744366] [INFO ] [MAIN ] IMAX: Successfully saved derivative data. [2025-10-20T22:10:29.746302] [INFO ] [MAIN ] Derivative Progress: 930/1877 symbols completed. IMAX took 1.0s. Estimated remaining: 5m 56s. Rate: 159.6 symbols/min [2025-10-20T22:10:29.746734] [INFO ] [MAIN ] INFA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:29.746847] [INFO ] [MAIN ] INFY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:29.746947] [INFO ] [MAIN ] ING: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:29.815312] [DEBUG] [LOCAL ] INFA: Loaded 48813 existing records from primary storage. [2025-10-20T22:10:29.815524] [INFO ] [MAIN ] INFA: Loaded 48813 primary data records. [2025-10-20T22:10:29.949976] [DEBUG] [LOCAL ] INFY: Loaded 53597 existing records from primary storage. [2025-10-20T22:10:29.956602] [INFO ] [MAIN ] INFY: Loaded 53597 primary data records. [2025-10-20T22:10:29.956511] [DEBUG] [LOCAL ] ING: Loaded 50721 existing records from primary storage. [2025-10-20T22:10:29.956900] [INFO ] [MAIN ] ING: Loaded 50721 primary data records. [2025-10-20T22:10:30.776539] [DEBUG] [MAIN ] INFA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:30.777269] [INFO ] [MAIN ] INFA: Generated 1794 derivative records from 48813 primary records. [2025-10-20T22:10:30.777471] [DEBUG] [LOCAL ] INFA: Saving derivative data. [2025-10-20T22:10:30.794830] [DEBUG] [LOCAL ] INFA: Derivative data saved (1794 records). [2025-10-20T22:10:30.796051] [INFO ] [MAIN ] INFA: Successfully saved derivative data. [2025-10-20T22:10:30.799398] [INFO ] [MAIN ] Derivative Progress: 931/1877 symbols completed. INFA took 1.1s. Estimated remaining: 5m 56s. Rate: 159.3 symbols/min [2025-10-20T22:10:30.799685] [INFO ] [MAIN ] INGM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:30.815443] [DEBUG] [MAIN ] ING: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:30.816499] [INFO ] [MAIN ] ING: Generated 1794 derivative records from 50721 primary records. [2025-10-20T22:10:30.816627] [DEBUG] [LOCAL ] ING: Saving derivative data. [2025-10-20T22:10:30.845729] [DEBUG] [LOCAL ] ING: Derivative data saved (1794 records). [2025-10-20T22:10:30.846122] [INFO ] [MAIN ] ING: Successfully saved derivative data. [2025-10-20T22:10:30.848500] [INFO ] [MAIN ] Derivative Progress: 932/1877 symbols completed. ING took 1.1s. Estimated remaining: 5m 56s. Rate: 159.5 symbols/min [2025-10-20T22:10:30.845949] [DEBUG] [MAIN ] INFY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:30.848810] [INFO ] [MAIN ] INGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:30.849894] [INFO ] [MAIN ] INFY: Generated 1794 derivative records from 53597 primary records. [2025-10-20T22:10:30.849967] [DEBUG] [LOCAL ] INFY: Saving derivative data. [2025-10-20T22:10:30.873526] [DEBUG] [LOCAL ] INFY: Derivative data saved (1794 records). [2025-10-20T22:10:30.873674] [INFO ] [MAIN ] INFY: Successfully saved derivative data. [2025-10-20T22:10:30.876188] [INFO ] [MAIN ] Derivative Progress: 933/1877 symbols completed. INFY took 1.1s. Estimated remaining: 5m 55s. Rate: 159.6 symbols/min [2025-10-20T22:10:30.876529] [INFO ] [MAIN ] INN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:30.886851] [DEBUG] [LOCAL ] INGM: Loaded 28268 existing records from primary storage. [2025-10-20T22:10:30.887071] [INFO ] [MAIN ] INGM: Loaded 28268 primary data records. [2025-10-20T22:10:31.029586] [DEBUG] [LOCAL ] INGR: Loaded 33774 existing records from primary storage. [2025-10-20T22:10:31.051404] [INFO ] [MAIN ] INGR: Loaded 33774 primary data records. [2025-10-20T22:10:31.304593] [DEBUG] [LOCAL ] INN: Loaded 41658 existing records from primary storage. [2025-10-20T22:10:31.351986] [INFO ] [MAIN ] INN: Loaded 41658 primary data records. [2025-10-20T22:10:31.973037] [DEBUG] [MAIN ] INGM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:31.974817] [INFO ] [MAIN ] INGM: Generated 1794 derivative records from 28268 primary records. [2025-10-20T22:10:31.975081] [DEBUG] [LOCAL ] INGM: Saving derivative data. [2025-10-20T22:10:32.005880] [DEBUG] [LOCAL ] INGM: Derivative data saved (1794 records). [2025-10-20T22:10:32.005995] [INFO ] [MAIN ] INGM: Successfully saved derivative data. [2025-10-20T22:10:32.008033] [INFO ] [MAIN ] Derivative Progress: 934/1877 symbols completed. INGM took 1.2s. Estimated remaining: 5m 55s. Rate: 159.3 symbols/min [2025-10-20T22:10:32.008293] [INFO ] [MAIN ] INSP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:32.009488] [DEBUG] [MAIN ] INGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:32.010511] [INFO ] [MAIN ] INGR: Generated 1794 derivative records from 33774 primary records. [2025-10-20T22:10:32.011565] [DEBUG] [LOCAL ] INGR: Saving derivative data. [2025-10-20T22:10:32.029915] [DEBUG] [LOCAL ] INGR: Derivative data saved (1794 records). [2025-10-20T22:10:32.042917] [INFO ] [MAIN ] INGR: Successfully saved derivative data. [2025-10-20T22:10:32.045593] [INFO ] [MAIN ] Derivative Progress: 935/1877 symbols completed. INGR took 1.2s. Estimated remaining: 5m 54s. Rate: 159.4 symbols/min [2025-10-20T22:10:32.042841] [DEBUG] [MAIN ] INN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:32.046498] [INFO ] [MAIN ] INN: Generated 1794 derivative records from 41658 primary records. [2025-10-20T22:10:32.046745] [INFO ] [MAIN ] INSW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:32.046845] [DEBUG] [LOCAL ] INN: Saving derivative data. [2025-10-20T22:10:32.063212] [DEBUG] [LOCAL ] INN: Derivative data saved (1794 records). [2025-10-20T22:10:32.063387] [INFO ] [MAIN ] INN: Successfully saved derivative data. [2025-10-20T22:10:32.065383] [INFO ] [MAIN ] Derivative Progress: 936/1877 symbols completed. INN took 1.2s. Estimated remaining: 5m 54s. Rate: 159.6 symbols/min [2025-10-20T22:10:32.065714] [INFO ] [MAIN ] INVH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:32.094636] [DEBUG] [LOCAL ] INSP: Loaded 38552 existing records from primary storage. [2025-10-20T22:10:32.104514] [INFO ] [MAIN ] INSP: Loaded 38552 primary data records. [2025-10-20T22:10:32.216302] [DEBUG] [LOCAL ] INSW: Loaded 39541 existing records from primary storage. [2025-10-20T22:10:32.263442] [INFO ] [MAIN ] INSW: Loaded 39541 primary data records. [2025-10-20T22:10:32.419595] [DEBUG] [LOCAL ] INVH: Loaded 53195 existing records from primary storage. [2025-10-20T22:10:32.462056] [INFO ] [MAIN ] INVH: Loaded 53195 primary data records. [2025-10-20T22:10:33.352423] [DEBUG] [MAIN ] INSP: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:33.354354] [INFO ] [MAIN ] INSP: Generated 1793 derivative records from 38552 primary records. [2025-10-20T22:10:33.356188] [DEBUG] [LOCAL ] INSP: Saving derivative data. [2025-10-20T22:10:33.406083] [DEBUG] [LOCAL ] INSP: Derivative data saved (1793 records). [2025-10-20T22:10:33.405978] [DEBUG] [MAIN ] INSW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:33.407458] [INFO ] [MAIN ] INSW: Generated 1794 derivative records from 39541 primary records. [2025-10-20T22:10:33.407717] [DEBUG] [LOCAL ] INSW: Saving derivative data. [2025-10-20T22:10:33.406535] [INFO ] [MAIN ] INSP: Successfully saved derivative data. [2025-10-20T22:10:33.411671] [INFO ] [MAIN ] Derivative Progress: 937/1877 symbols completed. INSP took 1.4s. Estimated remaining: 5m 54s. Rate: 159.2 symbols/min [2025-10-20T22:10:33.424513] [INFO ] [MAIN ] INVX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:33.438326] [DEBUG] [LOCAL ] INSW: Derivative data saved (1794 records). [2025-10-20T22:10:33.439219] [INFO ] [MAIN ] INSW: Successfully saved derivative data. [2025-10-20T22:10:33.441838] [INFO ] [MAIN ] Derivative Progress: 938/1877 symbols completed. INSW took 1.4s. Estimated remaining: 5m 54s. Rate: 159.3 symbols/min [2025-10-20T22:10:33.442809] [INFO ] [MAIN ] IONQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:33.468266] [DEBUG] [MAIN ] INVH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:33.469250] [INFO ] [MAIN ] INVH: Generated 1794 derivative records from 53195 primary records. [2025-10-20T22:10:33.478092] [DEBUG] [LOCAL ] INVH: Saving derivative data. [2025-10-20T22:10:33.500284] [DEBUG] [LOCAL ] INVH: Derivative data saved (1794 records). [2025-10-20T22:10:33.510126] [INFO ] [MAIN ] INVH: Successfully saved derivative data. [2025-10-20T22:10:33.513862] [INFO ] [MAIN ] Derivative Progress: 939/1877 symbols completed. INVH took 1.4s. Estimated remaining: 5m 53s. Rate: 159.5 symbols/min [2025-10-20T22:10:33.514286] [INFO ] [MAIN ] IOT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:33.515877] [DEBUG] [LOCAL ] INVX: Loaded 29241 existing records from primary storage. [2025-10-20T22:10:33.516401] [INFO ] [MAIN ] INVX: Loaded 29241 primary data records. [2025-10-20T22:10:33.788700] [DEBUG] [LOCAL ] IONQ: Loaded 53788 existing records from primary storage. [2025-10-20T22:10:33.931903] [INFO ] [MAIN ] IONQ: Loaded 53788 primary data records. [2025-10-20T22:10:34.036189] [DEBUG] [LOCAL ] IOT: Loaded 53478 existing records from primary storage. [2025-10-20T22:10:34.049870] [INFO ] [MAIN ] IOT: Loaded 53478 primary data records. [2025-10-20T22:10:34.623569] [DEBUG] [MAIN ] INVX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:34.625274] [INFO ] [MAIN ] INVX: Generated 1794 derivative records from 29241 primary records. [2025-10-20T22:10:34.625580] [DEBUG] [LOCAL ] INVX: Saving derivative data. [2025-10-20T22:10:34.648089] [DEBUG] [LOCAL ] INVX: Derivative data saved (1794 records). [2025-10-20T22:10:34.648298] [INFO ] [MAIN ] INVX: Successfully saved derivative data. [2025-10-20T22:10:34.650515] [INFO ] [MAIN ] Derivative Progress: 940/1877 symbols completed. INVX took 1.2s. Estimated remaining: 5m 53s. Rate: 159.1 symbols/min [2025-10-20T22:10:34.650916] [INFO ] [MAIN ] IP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:34.664542] [DEBUG] [MAIN ] IONQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:34.666175] [INFO ] [MAIN ] IONQ: Generated 1794 derivative records from 53788 primary records. [2025-10-20T22:10:34.666720] [DEBUG] [LOCAL ] IONQ: Saving derivative data. [2025-10-20T22:10:34.683853] [DEBUG] [LOCAL ] IONQ: Derivative data saved (1794 records). [2025-10-20T22:10:34.696910] [INFO ] [MAIN ] IONQ: Successfully saved derivative data. [2025-10-20T22:10:34.698925] [INFO ] [MAIN ] Derivative Progress: 941/1877 symbols completed. IONQ took 1.3s. Estimated remaining: 5m 53s. Rate: 159.3 symbols/min [2025-10-20T22:10:34.699324] [DEBUG] [MAIN ] IOT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:34.700186] [INFO ] [MAIN ] IOT: Generated 1794 derivative records from 53478 primary records. [2025-10-20T22:10:34.700298] [DEBUG] [LOCAL ] IOT: Saving derivative data. [2025-10-20T22:10:34.700419] [INFO ] [MAIN ] IPG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:34.717288] [DEBUG] [LOCAL ] IOT: Derivative data saved (1794 records). [2025-10-20T22:10:34.717488] [INFO ] [MAIN ] IOT: Successfully saved derivative data. [2025-10-20T22:10:34.719596] [INFO ] [MAIN ] Derivative Progress: 942/1877 symbols completed. IOT took 1.2s. Estimated remaining: 5m 52s. Rate: 159.4 symbols/min [2025-10-20T22:10:34.719901] [INFO ] [MAIN ] IPI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:34.770020] [DEBUG] [LOCAL ] IPI: Loaded 21513 existing records from primary storage. [2025-10-20T22:10:34.770357] [INFO ] [MAIN ] IPI: Loaded 21513 primary data records. [2025-10-20T22:10:34.782503] [DEBUG] [LOCAL ] IP: Loaded 53306 existing records from primary storage. [2025-10-20T22:10:34.802469] [INFO ] [MAIN ] IP: Loaded 53306 primary data records. [2025-10-20T22:10:35.044590] [DEBUG] [LOCAL ] IPG: Loaded 53330 existing records from primary storage. [2025-10-20T22:10:35.076495] [INFO ] [MAIN ] IPG: Loaded 53330 primary data records. [2025-10-20T22:10:35.780893] [DEBUG] [MAIN ] IPI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:35.780370] [DEBUG] [MAIN ] IP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:35.782194] [INFO ] [MAIN ] IP: Generated 1794 derivative records from 53306 primary records. [2025-10-20T22:10:35.783227] [INFO ] [MAIN ] IPI: Generated 1794 derivative records from 21513 primary records. [2025-10-20T22:10:35.783592] [DEBUG] [LOCAL ] IPI: Saving derivative data. [2025-10-20T22:10:35.783335] [DEBUG] [LOCAL ] IP: Saving derivative data. [2025-10-20T22:10:35.812413] [DEBUG] [LOCAL ] IP: Derivative data saved (1794 records). [2025-10-20T22:10:35.812577] [INFO ] [MAIN ] IP: Successfully saved derivative data. [2025-10-20T22:10:35.827173] [INFO ] [MAIN ] Derivative Progress: 943/1877 symbols completed. IP took 1.2s. Estimated remaining: 5m 52s. Rate: 159.1 symbols/min [2025-10-20T22:10:35.833122] [DEBUG] [MAIN ] IPG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:35.836916] [DEBUG] [LOCAL ] IPI: Derivative data saved (1794 records). [2025-10-20T22:10:35.837012] [INFO ] [MAIN ] IPI: Successfully saved derivative data. [2025-10-20T22:10:35.839919] [INFO ] [MAIN ] Derivative Progress: 944/1877 symbols completed. IPI took 1.1s. Estimated remaining: 5m 52s. Rate: 159.3 symbols/min [2025-10-20T22:10:35.837803] [INFO ] [MAIN ] IPG: Generated 1794 derivative records from 53330 primary records. [2025-10-20T22:10:35.840152] [DEBUG] [LOCAL ] IPG: Saving derivative data. [2025-10-20T22:10:35.855562] [DEBUG] [LOCAL ] IPG: Derivative data saved (1794 records). [2025-10-20T22:10:35.855621] [INFO ] [MAIN ] IPG: Successfully saved derivative data. [2025-10-20T22:10:35.858135] [INFO ] [MAIN ] Derivative Progress: 945/1877 symbols completed. IPG took 1.2s. Estimated remaining: 5m 51s. Rate: 159.4 symbols/min [2025-10-20T22:10:35.858678] [INFO ] [MAIN ] IQV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:35.858793] [INFO ] [MAIN ] IR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:35.858556] [INFO ] [MAIN ] IQI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:35.884032] [DEBUG] [LOCAL ] IQI: Loaded 17832 existing records from primary storage. [2025-10-20T22:10:35.884249] [INFO ] [MAIN ] IQI: Loaded 17832 primary data records. [2025-10-20T22:10:36.154372] [DEBUG] [LOCAL ] IQV: Loaded 50022 existing records from primary storage. [2025-10-20T22:10:36.175416] [INFO ] [MAIN ] IQV: Loaded 50022 primary data records. [2025-10-20T22:10:36.289783] [DEBUG] [LOCAL ] IR: Loaded 52765 existing records from primary storage. [2025-10-20T22:10:36.290085] [INFO ] [MAIN ] IR: Loaded 52765 primary data records. [2025-10-20T22:10:36.937277] [DEBUG] [MAIN ] IQI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:36.938558] [INFO ] [MAIN ] IQI: Generated 1794 derivative records from 17832 primary records. [2025-10-20T22:10:36.938690] [DEBUG] [LOCAL ] IQI: Saving derivative data. [2025-10-20T22:10:36.957796] [DEBUG] [LOCAL ] IQI: Derivative data saved (1794 records). [2025-10-20T22:10:36.958034] [INFO ] [MAIN ] IQI: Successfully saved derivative data. [2025-10-20T22:10:36.960324] [INFO ] [MAIN ] Derivative Progress: 946/1877 symbols completed. IQI took 1.1s. Estimated remaining: 5m 51s. Rate: 159.1 symbols/min [2025-10-20T22:10:36.960577] [INFO ] [MAIN ] IRM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:36.978538] [DEBUG] [MAIN ] IQV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:36.980600] [INFO ] [MAIN ] IQV: Generated 1793 derivative records from 50022 primary records. [2025-10-20T22:10:36.980869] [DEBUG] [LOCAL ] IQV: Saving derivative data. [2025-10-20T22:10:36.994327] [DEBUG] [MAIN ] IR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:37.002285] [INFO ] [MAIN ] IR: Generated 1794 derivative records from 52765 primary records. [2025-10-20T22:10:37.002441] [DEBUG] [LOCAL ] IR: Saving derivative data. [2025-10-20T22:10:37.024375] [DEBUG] [LOCAL ] IQV: Derivative data saved (1793 records). [2025-10-20T22:10:37.026929] [INFO ] [MAIN ] IQV: Successfully saved derivative data. [2025-10-20T22:10:37.026836] [DEBUG] [LOCAL ] IR: Derivative data saved (1794 records). [2025-10-20T22:10:37.029019] [INFO ] [MAIN ] IR: Successfully saved derivative data. [2025-10-20T22:10:37.028951] [INFO ] [MAIN ] Derivative Progress: 947/1877 symbols completed. IQV took 1.2s. Estimated remaining: 5m 50s. Rate: 159.2 symbols/min [2025-10-20T22:10:37.029232] [INFO ] [MAIN ] IRS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:37.041391] [INFO ] [MAIN ] Derivative Progress: 948/1877 symbols completed. IR took 1.2s. Estimated remaining: 5m 50s. Rate: 159.4 symbols/min [2025-10-20T22:10:37.041679] [INFO ] [MAIN ] IRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:37.063626] [DEBUG] [LOCAL ] IRM: Loaded 49015 existing records from primary storage. [2025-10-20T22:10:37.068737] [INFO ] [MAIN ] IRM: Loaded 49015 primary data records. [2025-10-20T22:10:37.183591] [DEBUG] [LOCAL ] IRS: Loaded 23243 existing records from primary storage. [2025-10-20T22:10:37.212909] [INFO ] [MAIN ] IRS: Loaded 23243 primary data records. [2025-10-20T22:10:37.531683] [DEBUG] [LOCAL ] IRT: Loaded 50527 existing records from primary storage. [2025-10-20T22:10:37.532387] [INFO ] [MAIN ] IRT: Loaded 50527 primary data records. [2025-10-20T22:10:38.123975] [DEBUG] [MAIN ] IRM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:38.125012] [INFO ] [MAIN ] IRM: Generated 1794 derivative records from 49015 primary records. [2025-10-20T22:10:38.127723] [DEBUG] [LOCAL ] IRM: Saving derivative data. [2025-10-20T22:10:38.157778] [DEBUG] [MAIN ] IRS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:38.160032] [INFO ] [MAIN ] IRS: Generated 1794 derivative records from 23243 primary records. [2025-10-20T22:10:38.160415] [DEBUG] [LOCAL ] IRS: Saving derivative data. [2025-10-20T22:10:38.160334] [DEBUG] [LOCAL ] IRM: Derivative data saved (1794 records). [2025-10-20T22:10:38.160519] [INFO ] [MAIN ] IRM: Successfully saved derivative data. [2025-10-20T22:10:38.163103] [INFO ] [MAIN ] Derivative Progress: 949/1877 symbols completed. IRM took 1.2s. Estimated remaining: 5m 50s. Rate: 159.1 symbols/min [2025-10-20T22:10:38.163829] [INFO ] [MAIN ] ISD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:38.179405] [DEBUG] [LOCAL ] IRS: Derivative data saved (1794 records). [2025-10-20T22:10:38.179822] [INFO ] [MAIN ] IRS: Successfully saved derivative data. [2025-10-20T22:10:38.181905] [INFO ] [MAIN ] Derivative Progress: 950/1877 symbols completed. IRS took 1.2s. Estimated remaining: 5m 49s. Rate: 159.2 symbols/min [2025-10-20T22:10:38.182197] [INFO ] [MAIN ] IT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:38.205759] [DEBUG] [MAIN ] IRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:38.206811] [INFO ] [MAIN ] IRT: Generated 1794 derivative records from 50527 primary records. [2025-10-20T22:10:38.207002] [DEBUG] [LOCAL ] IRT: Saving derivative data. [2025-10-20T22:10:38.216130] [DEBUG] [LOCAL ] ISD: Loaded 15782 existing records from primary storage. [2025-10-20T22:10:38.225018] [INFO ] [MAIN ] ISD: Loaded 15782 primary data records. [2025-10-20T22:10:38.231800] [DEBUG] [LOCAL ] IRT: Derivative data saved (1794 records). [2025-10-20T22:10:38.244133] [INFO ] [MAIN ] IRT: Successfully saved derivative data. [2025-10-20T22:10:38.257332] [INFO ] [MAIN ] Derivative Progress: 951/1877 symbols completed. IRT took 1.2s. Estimated remaining: 5m 49s. Rate: 159.4 symbols/min [2025-10-20T22:10:38.267910] [INFO ] [MAIN ] ITGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:38.399863] [DEBUG] [LOCAL ] IT: Loaded 42199 existing records from primary storage. [2025-10-20T22:10:38.440434] [INFO ] [MAIN ] IT: Loaded 42199 primary data records. [2025-10-20T22:10:38.656768] [DEBUG] [LOCAL ] ITGR: Loaded 31097 existing records from primary storage. [2025-10-20T22:10:38.673860] [INFO ] [MAIN ] ITGR: Loaded 31097 primary data records. [2025-10-20T22:10:39.224328] [DEBUG] [MAIN ] ISD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:39.225490] [INFO ] [MAIN ] ISD: Generated 1794 derivative records from 15782 primary records. [2025-10-20T22:10:39.225685] [DEBUG] [LOCAL ] ISD: Saving derivative data. [2025-10-20T22:10:39.244582] [DEBUG] [LOCAL ] ISD: Derivative data saved (1794 records). [2025-10-20T22:10:39.245258] [INFO ] [MAIN ] ISD: Successfully saved derivative data. [2025-10-20T22:10:39.247161] [INFO ] [MAIN ] Derivative Progress: 952/1877 symbols completed. ISD took 1.1s. Estimated remaining: 5m 49s. Rate: 159.1 symbols/min [2025-10-20T22:10:39.247418] [INFO ] [MAIN ] ITT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:39.266885] [DEBUG] [MAIN ] ITGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:39.269336] [INFO ] [MAIN ] ITGR: Generated 1794 derivative records from 31097 primary records. [2025-10-20T22:10:39.269622] [DEBUG] [LOCAL ] ITGR: Saving derivative data. [2025-10-20T22:10:39.296371] [DEBUG] [MAIN ] IT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:39.298728] [INFO ] [MAIN ] IT: Generated 1793 derivative records from 42199 primary records. [2025-10-20T22:10:39.299424] [DEBUG] [LOCAL ] ITGR: Derivative data saved (1794 records). [2025-10-20T22:10:39.299142] [DEBUG] [LOCAL ] IT: Saving derivative data. [2025-10-20T22:10:39.299562] [INFO ] [MAIN ] ITGR: Successfully saved derivative data. [2025-10-20T22:10:39.302272] [INFO ] [MAIN ] Derivative Progress: 953/1877 symbols completed. ITGR took 1.0s. Estimated remaining: 5m 48s. Rate: 159.2 symbols/min [2025-10-20T22:10:39.302688] [INFO ] [MAIN ] ITUB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:39.336968] [DEBUG] [LOCAL ] IT: Derivative data saved (1793 records). [2025-10-20T22:10:39.338181] [INFO ] [MAIN ] IT: Successfully saved derivative data. [2025-10-20T22:10:39.340226] [INFO ] [MAIN ] Derivative Progress: 954/1877 symbols completed. IT took 1.2s. Estimated remaining: 5m 47s. Rate: 159.4 symbols/min [2025-10-20T22:10:39.340511] [INFO ] [MAIN ] ITW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:39.341854] [DEBUG] [LOCAL ] ITT: Loaded 32539 existing records from primary storage. [2025-10-20T22:10:39.341931] [INFO ] [MAIN ] ITT: Loaded 32539 primary data records. [2025-10-20T22:10:39.677487] [DEBUG] [LOCAL ] ITUB: Loaded 53327 existing records from primary storage. [2025-10-20T22:10:39.677709] [INFO ] [MAIN ] ITUB: Loaded 53327 primary data records. [2025-10-20T22:10:39.871763] [DEBUG] [LOCAL ] ITW: Loaded 45637 existing records from primary storage. [2025-10-20T22:10:39.889113] [INFO ] [MAIN ] ITW: Loaded 45637 primary data records. [2025-10-20T22:10:40.302138] [DEBUG] [MAIN ] ITT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:40.356966] [INFO ] [MAIN ] ITT: Generated 1793 derivative records from 32539 primary records. [2025-10-20T22:10:40.357875] [DEBUG] [LOCAL ] ITT: Saving derivative data. [2025-10-20T22:10:40.376232] [DEBUG] [LOCAL ] ITT: Derivative data saved (1793 records). [2025-10-20T22:10:40.376398] [INFO ] [MAIN ] ITT: Successfully saved derivative data. [2025-10-20T22:10:40.379361] [INFO ] [MAIN ] Derivative Progress: 955/1877 symbols completed. ITT took 1.1s. Estimated remaining: 5m 48s. Rate: 159.1 symbols/min [2025-10-20T22:10:40.379921] [INFO ] [MAIN ] IVR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:40.400174] [DEBUG] [MAIN ] ITUB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:40.401157] [INFO ] [MAIN ] ITUB: Generated 1794 derivative records from 53327 primary records. [2025-10-20T22:10:40.401425] [DEBUG] [LOCAL ] ITUB: Saving derivative data. [2025-10-20T22:10:40.418855] [DEBUG] [LOCAL ] ITUB: Derivative data saved (1794 records). [2025-10-20T22:10:40.419138] [INFO ] [MAIN ] ITUB: Successfully saved derivative data. [2025-10-20T22:10:40.421672] [INFO ] [MAIN ] Derivative Progress: 956/1877 symbols completed. ITUB took 1.1s. Estimated remaining: 5m 47s. Rate: 159.2 symbols/min [2025-10-20T22:10:40.421830] [INFO ] [MAIN ] IVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:40.451859] [DEBUG] [MAIN ] ITW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:40.452541] [INFO ] [MAIN ] ITW: Generated 1794 derivative records from 45637 primary records. [2025-10-20T22:10:40.452691] [DEBUG] [LOCAL ] ITW: Saving derivative data. [2025-10-20T22:10:40.484695] [DEBUG] [LOCAL ] ITW: Derivative data saved (1794 records). [2025-10-20T22:10:40.484515] [DEBUG] [LOCAL ] IVR: Loaded 47946 existing records from primary storage. [2025-10-20T22:10:40.484846] [INFO ] [MAIN ] ITW: Successfully saved derivative data. [2025-10-20T22:10:40.485131] [INFO ] [MAIN ] IVR: Loaded 47946 primary data records. [2025-10-20T22:10:40.487281] [INFO ] [MAIN ] Derivative Progress: 957/1877 symbols completed. ITW took 1.1s. Estimated remaining: 5m 46s. Rate: 159.4 symbols/min [2025-10-20T22:10:40.487702] [INFO ] [MAIN ] IVZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:40.611690] [DEBUG] [LOCAL ] IVT: Loaded 31523 existing records from primary storage. [2025-10-20T22:10:40.617129] [INFO ] [MAIN ] IVT: Loaded 31523 primary data records. [2025-10-20T22:10:41.135243] [DEBUG] [LOCAL ] IVZ: Loaded 53129 existing records from primary storage. [2025-10-20T22:10:41.136035] [INFO ] [MAIN ] IVZ: Loaded 53129 primary data records. [2025-10-20T22:10:41.459799] [DEBUG] [MAIN ] IVR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:41.460898] [INFO ] [MAIN ] IVR: Generated 1794 derivative records from 47946 primary records. [2025-10-20T22:10:41.461009] [DEBUG] [LOCAL ] IVR: Saving derivative data. [2025-10-20T22:10:41.493934] [DEBUG] [LOCAL ] IVR: Derivative data saved (1794 records). [2025-10-20T22:10:41.494057] [INFO ] [MAIN ] IVR: Successfully saved derivative data. [2025-10-20T22:10:41.491447] [DEBUG] [MAIN ] IVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:41.495938] [INFO ] [MAIN ] IVT: Generated 1794 derivative records from 31523 primary records. [2025-10-20T22:10:41.496054] [DEBUG] [LOCAL ] IVT: Saving derivative data. [2025-10-20T22:10:41.498074] [INFO ] [MAIN ] Derivative Progress: 958/1877 symbols completed. IVR took 1.1s. Estimated remaining: 5m 47s. Rate: 159.1 symbols/min [2025-10-20T22:10:41.514666] [DEBUG] [LOCAL ] IVT: Derivative data saved (1794 records). [2025-10-20T22:10:41.514817] [INFO ] [MAIN ] IVT: Successfully saved derivative data. [2025-10-20T22:10:41.517371] [INFO ] [MAIN ] Derivative Progress: 959/1877 symbols completed. IVT took 1.1s. Estimated remaining: 5m 46s. Rate: 159.2 symbols/min [2025-10-20T22:10:41.533221] [DEBUG] [MAIN ] IVZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:41.533762] [INFO ] [MAIN ] IVZ: Generated 1794 derivative records from 53129 primary records. [2025-10-20T22:10:41.533821] [DEBUG] [LOCAL ] IVZ: Saving derivative data. [2025-10-20T22:10:41.548614] [DEBUG] [LOCAL ] IVZ: Derivative data saved (1794 records). [2025-10-20T22:10:41.548723] [INFO ] [MAIN ] IVZ: Successfully saved derivative data. [2025-10-20T22:10:41.550852] [INFO ] [MAIN ] Derivative Progress: 960/1877 symbols completed. IVZ took 1.1s. Estimated remaining: 5m 45s. Rate: 159.4 symbols/min [2025-10-20T22:10:41.551265] [INFO ] [MAIN ] IX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:41.551379] [INFO ] [MAIN ] J: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:41.551482] [INFO ] [MAIN ] JBGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:41.578093] [DEBUG] [LOCAL ] IX: Loaded 22282 existing records from primary storage. [2025-10-20T22:10:41.578388] [INFO ] [MAIN ] IX: Loaded 22282 primary data records. [2025-10-20T22:10:41.792839] [DEBUG] [LOCAL ] J: Loaded 40502 existing records from primary storage. [2025-10-20T22:10:41.814700] [INFO ] [MAIN ] J: Loaded 40502 primary data records. [2025-10-20T22:10:41.984675] [DEBUG] [LOCAL ] JBGS: Loaded 41953 existing records from primary storage. [2025-10-20T22:10:42.016778] [INFO ] [MAIN ] JBGS: Loaded 41953 primary data records. [2025-10-20T22:10:42.597745] [DEBUG] [MAIN ] IX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:42.599899] [INFO ] [MAIN ] IX: Generated 1794 derivative records from 22282 primary records. [2025-10-20T22:10:42.599996] [DEBUG] [LOCAL ] IX: Saving derivative data. [2025-10-20T22:10:42.619125] [DEBUG] [LOCAL ] IX: Derivative data saved (1794 records). [2025-10-20T22:10:42.619274] [INFO ] [MAIN ] IX: Successfully saved derivative data. [2025-10-20T22:10:42.621166] [INFO ] [MAIN ] Derivative Progress: 961/1877 symbols completed. IX took 1.1s. Estimated remaining: 5m 45s. Rate: 159.1 symbols/min [2025-10-20T22:10:42.621425] [INFO ] [MAIN ] JBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:42.635681] [DEBUG] [MAIN ] J: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:42.636578] [INFO ] [MAIN ] J: Generated 1794 derivative records from 40502 primary records. [2025-10-20T22:10:42.637636] [DEBUG] [LOCAL ] J: Saving derivative data. [2025-10-20T22:10:42.656002] [DEBUG] [LOCAL ] J: Derivative data saved (1794 records). [2025-10-20T22:10:42.668775] [INFO ] [MAIN ] J: Successfully saved derivative data. [2025-10-20T22:10:42.671687] [INFO ] [MAIN ] Derivative Progress: 962/1877 symbols completed. J took 1.1s. Estimated remaining: 5m 45s. Rate: 159.2 symbols/min [2025-10-20T22:10:42.672343] [INFO ] [MAIN ] JBL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:42.672035] [DEBUG] [MAIN ] JBGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:42.673432] [INFO ] [MAIN ] JBGS: Generated 1794 derivative records from 41953 primary records. [2025-10-20T22:10:42.673530] [DEBUG] [LOCAL ] JBGS: Saving derivative data. [2025-10-20T22:10:42.699049] [DEBUG] [LOCAL ] JBGS: Derivative data saved (1794 records). [2025-10-20T22:10:42.699232] [INFO ] [MAIN ] JBGS: Successfully saved derivative data. [2025-10-20T22:10:42.701281] [INFO ] [MAIN ] Derivative Progress: 963/1877 symbols completed. JBGS took 1.1s. Estimated remaining: 5m 44s. Rate: 159.4 symbols/min [2025-10-20T22:10:42.701549] [INFO ] [MAIN ] JBND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:42.718690] [DEBUG] [LOCAL ] JBI: Loaded 42783 existing records from primary storage. [2025-10-20T22:10:42.718941] [INFO ] [MAIN ] JBI: Loaded 42783 primary data records. [2025-10-20T22:10:42.872674] [DEBUG] [LOCAL ] JBND: Loaded 27243 existing records from primary storage. [2025-10-20T22:10:42.878066] [INFO ] [MAIN ] JBND: Loaded 27243 primary data records. [2025-10-20T22:10:42.968311] [DEBUG] [LOCAL ] JBL: Loaded 48383 existing records from primary storage. [2025-10-20T22:10:43.012531] [INFO ] [MAIN ] JBL: Loaded 48383 primary data records. [2025-10-20T22:10:43.693771] [DEBUG] [MAIN ] JBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:43.695329] [INFO ] [MAIN ] JBI: Generated 1794 derivative records from 42783 primary records. [2025-10-20T22:10:43.695587] [DEBUG] [LOCAL ] JBI: Saving derivative data. [2025-10-20T22:10:43.714094] [DEBUG] [LOCAL ] JBI: Derivative data saved (1794 records). [2025-10-20T22:10:43.714178] [INFO ] [MAIN ] JBI: Successfully saved derivative data. [2025-10-20T22:10:43.716098] [INFO ] [MAIN ] Derivative Progress: 964/1877 symbols completed. JBI took 1.1s. Estimated remaining: 5m 44s. Rate: 159.1 symbols/min [2025-10-20T22:10:43.716383] [INFO ] [MAIN ] JCE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:43.739332] [DEBUG] [MAIN ] JBND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:43.740209] [INFO ] [MAIN ] JBND: Generated 1794 derivative records from 27243 primary records. [2025-10-20T22:10:43.743916] [DEBUG] [LOCAL ] JBND: Saving derivative data. [2025-10-20T22:10:43.744351] [DEBUG] [LOCAL ] JCE: Loaded 7308 existing records from primary storage. [2025-10-20T22:10:43.745440] [INFO ] [MAIN ] JCE: Loaded 7308 primary data records. [2025-10-20T22:10:43.854111] [DEBUG] [LOCAL ] JBND: Derivative data saved (1794 records). [2025-10-20T22:10:43.885681] [INFO ] [MAIN ] JBND: Successfully saved derivative data. [2025-10-20T22:10:43.888602] [INFO ] [MAIN ] Derivative Progress: 965/1877 symbols completed. JBND took 1.2s. Estimated remaining: 5m 44s. Rate: 159.2 symbols/min [2025-10-20T22:10:43.890327] [DEBUG] [MAIN ] JBL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:43.889958] [INFO ] [MAIN ] JCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:43.912963] [INFO ] [MAIN ] JBL: Generated 1794 derivative records from 48383 primary records. [2025-10-20T22:10:43.975677] [DEBUG] [LOCAL ] JBL: Saving derivative data. [2025-10-20T22:10:44.137563] [DEBUG] [LOCAL ] JBL: Derivative data saved (1794 records). [2025-10-20T22:10:44.138061] [INFO ] [MAIN ] JBL: Successfully saved derivative data. [2025-10-20T22:10:44.140869] [INFO ] [MAIN ] Derivative Progress: 966/1877 symbols completed. JBL took 1.5s. Estimated remaining: 5m 43s. Rate: 159.3 symbols/min [2025-10-20T22:10:44.141157] [INFO ] [MAIN ] JEF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:44.177517] [DEBUG] [MAIN ] JCE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:44.178260] [INFO ] [MAIN ] JCE: Generated 1794 derivative records from 7308 primary records. [2025-10-20T22:10:44.178370] [DEBUG] [LOCAL ] JCE: Saving derivative data. [2025-10-20T22:10:44.194227] [DEBUG] [LOCAL ] JCE: Derivative data saved (1794 records). [2025-10-20T22:10:44.194404] [INFO ] [MAIN ] JCE: Successfully saved derivative data. [2025-10-20T22:10:44.212632] [INFO ] [MAIN ] Derivative Progress: 967/1877 symbols completed. JCE took 0.5s. Estimated remaining: 5m 43s. Rate: 159.4 symbols/min [2025-10-20T22:10:44.212866] [INFO ] [MAIN ] JELD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:44.224949] [DEBUG] [LOCAL ] JCI: Loaded 53523 existing records from primary storage. [2025-10-20T22:10:44.225630] [INFO ] [MAIN ] JCI: Loaded 53523 primary data records. [2025-10-20T22:10:44.383272] [DEBUG] [LOCAL ] JEF: Loaded 50184 existing records from primary storage. [2025-10-20T22:10:44.434912] [INFO ] [MAIN ] JEF: Loaded 50184 primary data records. [2025-10-20T22:10:44.670247] [DEBUG] [LOCAL ] JELD: Loaded 46009 existing records from primary storage. [2025-10-20T22:10:44.676081] [INFO ] [MAIN ] JELD: Loaded 46009 primary data records. [2025-10-20T22:10:45.185919] [DEBUG] [MAIN ] JCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:45.187898] [INFO ] [MAIN ] JCI: Generated 1794 derivative records from 53523 primary records. [2025-10-20T22:10:45.188021] [DEBUG] [LOCAL ] JCI: Saving derivative data. [2025-10-20T22:10:45.207139] [DEBUG] [LOCAL ] JCI: Derivative data saved (1794 records). [2025-10-20T22:10:45.207584] [INFO ] [MAIN ] JCI: Successfully saved derivative data. [2025-10-20T22:10:45.209537] [INFO ] [MAIN ] Derivative Progress: 968/1877 symbols completed. JCI took 1.3s. Estimated remaining: 5m 43s. Rate: 159.1 symbols/min [2025-10-20T22:10:45.209974] [INFO ] [MAIN ] JFR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:45.231314] [DEBUG] [MAIN ] JEF: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:45.232470] [INFO ] [MAIN ] JEF: Generated 1793 derivative records from 50184 primary records. [2025-10-20T22:10:45.232725] [DEBUG] [LOCAL ] JEF: Saving derivative data. [2025-10-20T22:10:45.254384] [DEBUG] [MAIN ] JELD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:45.256571] [INFO ] [MAIN ] JELD: Generated 1794 derivative records from 46009 primary records. [2025-10-20T22:10:45.271684] [DEBUG] [LOCAL ] JELD: Saving derivative data. [2025-10-20T22:10:45.271569] [DEBUG] [LOCAL ] JEF: Derivative data saved (1793 records). [2025-10-20T22:10:45.272499] [INFO ] [MAIN ] JEF: Successfully saved derivative data. [2025-10-20T22:10:45.274852] [INFO ] [MAIN ] Derivative Progress: 969/1877 symbols completed. JEF took 1.1s. Estimated remaining: 5m 42s. Rate: 159.3 symbols/min [2025-10-20T22:10:45.281893] [INFO ] [MAIN ] JGH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:45.291304] [DEBUG] [LOCAL ] JELD: Derivative data saved (1794 records). [2025-10-20T22:10:45.291532] [INFO ] [MAIN ] JELD: Successfully saved derivative data. [2025-10-20T22:10:45.303110] [INFO ] [MAIN ] Derivative Progress: 970/1877 symbols completed. JELD took 1.1s. Estimated remaining: 5m 41s. Rate: 159.4 symbols/min [2025-10-20T22:10:45.303389] [INFO ] [MAIN ] JHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:45.305945] [DEBUG] [LOCAL ] JFR: Loaded 30952 existing records from primary storage. [2025-10-20T22:10:45.309203] [INFO ] [MAIN ] JFR: Loaded 30952 primary data records. [2025-10-20T22:10:45.405429] [DEBUG] [LOCAL ] JGH: Loaded 12697 existing records from primary storage. [2025-10-20T22:10:45.425446] [INFO ] [MAIN ] JGH: Loaded 12697 primary data records. [2025-10-20T22:10:45.988906] [DEBUG] [LOCAL ] JHG: Loaded 47569 existing records from primary storage. [2025-10-20T22:10:46.003539] [INFO ] [MAIN ] JHG: Loaded 47569 primary data records. [2025-10-20T22:10:46.328697] [DEBUG] [MAIN ] JFR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:46.331574] [INFO ] [MAIN ] JFR: Generated 1794 derivative records from 30952 primary records. [2025-10-20T22:10:46.332501] [DEBUG] [LOCAL ] JFR: Saving derivative data. [2025-10-20T22:10:46.350306] [DEBUG] [LOCAL ] JFR: Derivative data saved (1794 records). [2025-10-20T22:10:46.351224] [INFO ] [MAIN ] JFR: Successfully saved derivative data. [2025-10-20T22:10:46.353244] [INFO ] [MAIN ] Derivative Progress: 971/1877 symbols completed. JFR took 1.1s. Estimated remaining: 5m 42s. Rate: 159.1 symbols/min [2025-10-20T22:10:46.353524] [INFO ] [MAIN ] JHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:46.367928] [DEBUG] [MAIN ] JGH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:46.368893] [INFO ] [MAIN ] JGH: Generated 1794 derivative records from 12697 primary records. [2025-10-20T22:10:46.369090] [DEBUG] [LOCAL ] JGH: Saving derivative data. [2025-10-20T22:10:46.387078] [DEBUG] [LOCAL ] JHI: Loaded 4157 existing records from primary storage. [2025-10-20T22:10:46.389279] [DEBUG] [LOCAL ] JGH: Derivative data saved (1794 records). [2025-10-20T22:10:46.389433] [INFO ] [MAIN ] JHI: Loaded 4157 primary data records. [2025-10-20T22:10:46.389514] [INFO ] [MAIN ] JGH: Successfully saved derivative data. [2025-10-20T22:10:46.402998] [INFO ] [MAIN ] Derivative Progress: 972/1877 symbols completed. JGH took 1.1s. Estimated remaining: 5m 41s. Rate: 159.3 symbols/min [2025-10-20T22:10:46.423414] [INFO ] [MAIN ] JHS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:46.521941] [DEBUG] [LOCAL ] JHS: Loaded 2931 existing records from primary storage. [2025-10-20T22:10:46.549346] [INFO ] [MAIN ] JHS: Loaded 2931 primary data records. [2025-10-20T22:10:46.707083] [DEBUG] [MAIN ] JHG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:46.788924] [INFO ] [MAIN ] JHG: Generated 1794 derivative records from 47569 primary records. [2025-10-20T22:10:46.794625] [DEBUG] [LOCAL ] JHG: Saving derivative data. [2025-10-20T22:10:46.908919] [DEBUG] [LOCAL ] JHG: Derivative data saved (1794 records). [2025-10-20T22:10:46.919791] [INFO ] [MAIN ] JHG: Successfully saved derivative data. [2025-10-20T22:10:47.027840] [INFO ] [MAIN ] Derivative Progress: 973/1877 symbols completed. JHG took 1.7s. Estimated remaining: 5m 41s. Rate: 159.1 symbols/min [2025-10-20T22:10:47.056877] [DEBUG] [MAIN ] JHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:47.057742] [INFO ] [MAIN ] JHI: Generated 1794 derivative records from 4157 primary records. [2025-10-20T22:10:47.057901] [DEBUG] [LOCAL ] JHI: Saving derivative data. [2025-10-20T22:10:47.073319] [DEBUG] [LOCAL ] JHI: Derivative data saved (1794 records). [2025-10-20T22:10:47.073491] [INFO ] [MAIN ] JHI: Successfully saved derivative data. [2025-10-20T22:10:47.075505] [INFO ] [MAIN ] Derivative Progress: 974/1877 symbols completed. JHI took 0.7s. Estimated remaining: 5m 40s. Rate: 159.3 symbols/min [2025-10-20T22:10:47.090166] [DEBUG] [MAIN ] JHS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:47.090699] [INFO ] [MAIN ] JHS: Generated 1794 derivative records from 2931 primary records. [2025-10-20T22:10:47.090758] [DEBUG] [LOCAL ] JHS: Saving derivative data. [2025-10-20T22:10:47.103535] [DEBUG] [LOCAL ] JHS: Derivative data saved (1794 records). [2025-10-20T22:10:47.103591] [INFO ] [MAIN ] JHS: Successfully saved derivative data. [2025-10-20T22:10:47.105589] [INFO ] [MAIN ] Derivative Progress: 975/1877 symbols completed. JHS took 0.7s. Estimated remaining: 5m 39s. Rate: 159.4 symbols/min [2025-10-20T22:10:47.106030] [INFO ] [MAIN ] JHX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:47.106324] [INFO ] [MAIN ] JILL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:47.106382] [INFO ] [MAIN ] JKS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:47.125533] [DEBUG] [LOCAL ] JILL: Loaded 15604 existing records from primary storage. [2025-10-20T22:10:47.125792] [INFO ] [MAIN ] JILL: Loaded 15604 primary data records. [2025-10-20T22:10:47.341298] [DEBUG] [LOCAL ] JKS: Loaded 39426 existing records from primary storage. [2025-10-20T22:10:47.360056] [INFO ] [MAIN ] JKS: Loaded 39426 primary data records. [2025-10-20T22:10:47.484823] [DEBUG] [LOCAL ] JHX: Loaded 53195 existing records from primary storage. [2025-10-20T22:10:47.506159] [INFO ] [MAIN ] JHX: Loaded 53195 primary data records. [2025-10-20T22:10:48.136564] [DEBUG] [MAIN ] JILL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:48.141448] [INFO ] [MAIN ] JILL: Generated 1794 derivative records from 15604 primary records. [2025-10-20T22:10:48.142236] [DEBUG] [LOCAL ] JILL: Saving derivative data. [2025-10-20T22:10:48.162087] [DEBUG] [LOCAL ] JILL: Derivative data saved (1794 records). [2025-10-20T22:10:48.162441] [INFO ] [MAIN ] JILL: Successfully saved derivative data. [2025-10-20T22:10:48.164909] [INFO ] [MAIN ] Derivative Progress: 976/1877 symbols completed. JILL took 1.1s. Estimated remaining: 5m 40s. Rate: 159.1 symbols/min [2025-10-20T22:10:48.165110] [INFO ] [MAIN ] JLL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:48.187979] [DEBUG] [MAIN ] JHX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:48.188911] [INFO ] [MAIN ] JHX: Generated 1794 derivative records from 53195 primary records. [2025-10-20T22:10:48.189071] [DEBUG] [LOCAL ] JHX: Saving derivative data. [2025-10-20T22:10:48.211703] [DEBUG] [MAIN ] JKS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:48.213246] [INFO ] [MAIN ] JKS: Generated 1794 derivative records from 39426 primary records. [2025-10-20T22:10:48.213604] [DEBUG] [LOCAL ] JKS: Saving derivative data. [2025-10-20T22:10:48.250843] [DEBUG] [LOCAL ] JHX: Derivative data saved (1794 records). [2025-10-20T22:10:48.254544] [INFO ] [MAIN ] JHX: Successfully saved derivative data. [2025-10-20T22:10:48.255673] [DEBUG] [LOCAL ] JLL: Loaded 31753 existing records from primary storage. [2025-10-20T22:10:48.257994] [INFO ] [MAIN ] JLL: Loaded 31753 primary data records. [2025-10-20T22:10:48.255058] [DEBUG] [LOCAL ] JKS: Derivative data saved (1794 records). [2025-10-20T22:10:48.258257] [INFO ] [MAIN ] JKS: Successfully saved derivative data. [2025-10-20T22:10:48.257897] [INFO ] [MAIN ] Derivative Progress: 977/1877 symbols completed. JHX took 1.2s. Estimated remaining: 5m 39s. Rate: 159.3 symbols/min [2025-10-20T22:10:48.292820] [INFO ] [MAIN ] JLS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:48.293263] [INFO ] [MAIN ] Derivative Progress: 978/1877 symbols completed. JKS took 1.2s. Estimated remaining: 5m 38s. Rate: 159.4 symbols/min [2025-10-20T22:10:48.299242] [INFO ] [MAIN ] JMIA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:48.375147] [DEBUG] [LOCAL ] JLS: Loaded 4066 existing records from primary storage. [2025-10-20T22:10:48.391067] [INFO ] [MAIN ] JLS: Loaded 4066 primary data records. [2025-10-20T22:10:48.782627] [DEBUG] [LOCAL ] JMIA: Loaded 51222 existing records from primary storage. [2025-10-20T22:10:48.899175] [INFO ] [MAIN ] JMIA: Loaded 51222 primary data records. [2025-10-20T22:10:49.226676] [DEBUG] [MAIN ] JLL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:49.228237] [INFO ] [MAIN ] JLL: Generated 1794 derivative records from 31753 primary records. [2025-10-20T22:10:49.228813] [DEBUG] [LOCAL ] JLL: Saving derivative data. [2025-10-20T22:10:49.250636] [DEBUG] [LOCAL ] JLL: Derivative data saved (1794 records). [2025-10-20T22:10:49.263734] [INFO ] [MAIN ] JLL: Successfully saved derivative data. [2025-10-20T22:10:49.266071] [INFO ] [MAIN ] Derivative Progress: 979/1877 symbols completed. JLL took 1.1s. Estimated remaining: 5m 39s. Rate: 159.2 symbols/min [2025-10-20T22:10:49.266527] [INFO ] [MAIN ] JNJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:49.266807] [DEBUG] [MAIN ] JLS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:49.267807] [INFO ] [MAIN ] JLS: Generated 1794 derivative records from 4066 primary records. [2025-10-20T22:10:49.268433] [DEBUG] [LOCAL ] JLS: Saving derivative data. [2025-10-20T22:10:49.283920] [DEBUG] [LOCAL ] JLS: Derivative data saved (1794 records). [2025-10-20T22:10:49.284180] [INFO ] [MAIN ] JLS: Successfully saved derivative data. [2025-10-20T22:10:49.286491] [INFO ] [MAIN ] Derivative Progress: 980/1877 symbols completed. JLS took 1.0s. Estimated remaining: 5m 38s. Rate: 159.3 symbols/min [2025-10-20T22:10:49.286690] [INFO ] [MAIN ] JOBY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:49.307104] [DEBUG] [MAIN ] JMIA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:49.308075] [INFO ] [MAIN ] JMIA: Generated 1794 derivative records from 51222 primary records. [2025-10-20T22:10:49.308139] [DEBUG] [LOCAL ] JMIA: Saving derivative data. [2025-10-20T22:10:49.324237] [DEBUG] [LOCAL ] JMIA: Derivative data saved (1794 records). [2025-10-20T22:10:49.324390] [INFO ] [MAIN ] JMIA: Successfully saved derivative data. [2025-10-20T22:10:49.326701] [INFO ] [MAIN ] Derivative Progress: 981/1877 symbols completed. JMIA took 1.0s. Estimated remaining: 5m 37s. Rate: 159.5 symbols/min [2025-10-20T22:10:49.326951] [INFO ] [MAIN ] JOE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:49.379489] [DEBUG] [LOCAL ] JOE: Loaded 23890 existing records from primary storage. [2025-10-20T22:10:49.379832] [INFO ] [MAIN ] JOE: Loaded 23890 primary data records. [2025-10-20T22:10:49.508829] [DEBUG] [LOCAL ] JNJ: Loaded 53753 existing records from primary storage. [2025-10-20T22:10:49.526873] [INFO ] [MAIN ] JNJ: Loaded 53753 primary data records. [2025-10-20T22:10:49.527676] [DEBUG] [LOCAL ] JOBY: Loaded 53809 existing records from primary storage. [2025-10-20T22:10:49.538624] [INFO ] [MAIN ] JOBY: Loaded 53809 primary data records. [2025-10-20T22:10:50.396287] [DEBUG] [MAIN ] JOE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:50.397186] [INFO ] [MAIN ] JOE: Generated 1794 derivative records from 23890 primary records. [2025-10-20T22:10:50.397407] [DEBUG] [LOCAL ] JOE: Saving derivative data. [2025-10-20T22:10:50.414072] [DEBUG] [LOCAL ] JOE: Derivative data saved (1794 records). [2025-10-20T22:10:50.414321] [INFO ] [MAIN ] JOE: Successfully saved derivative data. [2025-10-20T22:10:50.416163] [INFO ] [MAIN ] Derivative Progress: 982/1877 symbols completed. JOE took 1.1s. Estimated remaining: 5m 37s. Rate: 159.1 symbols/min [2025-10-20T22:10:50.416416] [INFO ] [MAIN ] JOF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:50.447770] [DEBUG] [MAIN ] JNJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:50.448689] [INFO ] [MAIN ] JNJ: Generated 1794 derivative records from 53753 primary records. [2025-10-20T22:10:50.450751] [DEBUG] [LOCAL ] JNJ: Saving derivative data. [2025-10-20T22:10:50.447432] [DEBUG] [MAIN ] JOBY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:50.452392] [INFO ] [MAIN ] JOBY: Generated 1794 derivative records from 53809 primary records. [2025-10-20T22:10:50.468147] [DEBUG] [LOCAL ] JOF: Loaded 6925 existing records from primary storage. [2025-10-20T22:10:50.470121] [INFO ] [MAIN ] JOF: Loaded 6925 primary data records. [2025-10-20T22:10:50.471044] [DEBUG] [LOCAL ] JOBY: Saving derivative data. [2025-10-20T22:10:50.485159] [DEBUG] [LOCAL ] JNJ: Derivative data saved (1794 records). [2025-10-20T22:10:50.494792] [INFO ] [MAIN ] JNJ: Successfully saved derivative data. [2025-10-20T22:10:50.504385] [INFO ] [MAIN ] Derivative Progress: 983/1877 symbols completed. JNJ took 1.2s. Estimated remaining: 5m 37s. Rate: 159.3 symbols/min [2025-10-20T22:10:50.551031] [INFO ] [MAIN ] JPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:50.560288] [DEBUG] [LOCAL ] JOBY: Derivative data saved (1794 records). [2025-10-20T22:10:50.566714] [INFO ] [MAIN ] JOBY: Successfully saved derivative data. [2025-10-20T22:10:50.585857] [INFO ] [MAIN ] Derivative Progress: 984/1877 symbols completed. JOBY took 1.3s. Estimated remaining: 5m 36s. Rate: 159.4 symbols/min [2025-10-20T22:10:50.596531] [INFO ] [MAIN ] JPM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:50.814489] [DEBUG] [LOCAL ] JPC: Loaded 37073 existing records from primary storage. [2025-10-20T22:10:50.815118] [INFO ] [MAIN ] JPC: Loaded 37073 primary data records. [2025-10-20T22:10:50.980428] [DEBUG] [LOCAL ] JPM: Loaded 53794 existing records from primary storage. [2025-10-20T22:10:51.034357] [INFO ] [MAIN ] JPM: Loaded 53794 primary data records. [2025-10-20T22:10:51.276014] [DEBUG] [MAIN ] JOF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:51.298316] [INFO ] [MAIN ] JOF: Generated 1794 derivative records from 6925 primary records. [2025-10-20T22:10:51.303561] [DEBUG] [LOCAL ] JOF: Saving derivative data. [2025-10-20T22:10:51.487547] [DEBUG] [LOCAL ] JOF: Derivative data saved (1794 records). [2025-10-20T22:10:51.488018] [INFO ] [MAIN ] JOF: Successfully saved derivative data. [2025-10-20T22:10:51.490473] [INFO ] [MAIN ] Derivative Progress: 985/1877 symbols completed. JOF took 1.1s. Estimated remaining: 5m 36s. Rate: 159.2 symbols/min [2025-10-20T22:10:51.490790] [INFO ] [MAIN ] JQC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:51.513024] [DEBUG] [MAIN ] JPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:51.513909] [INFO ] [MAIN ] JPC: Generated 1794 derivative records from 37073 primary records. [2025-10-20T22:10:51.514214] [DEBUG] [LOCAL ] JPC: Saving derivative data. [2025-10-20T22:10:51.539076] [DEBUG] [LOCAL ] JPC: Derivative data saved (1794 records). [2025-10-20T22:10:51.539313] [INFO ] [MAIN ] JPC: Successfully saved derivative data. [2025-10-20T22:10:51.541307] [INFO ] [MAIN ] Derivative Progress: 986/1877 symbols completed. JPC took 1.0s. Estimated remaining: 5m 36s. Rate: 159.3 symbols/min [2025-10-20T22:10:51.541502] [INFO ] [MAIN ] JRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:51.569443] [DEBUG] [MAIN ] JPM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:51.570881] [INFO ] [MAIN ] JPM: Generated 1794 derivative records from 53794 primary records. [2025-10-20T22:10:51.570982] [DEBUG] [LOCAL ] JPM: Saving derivative data. [2025-10-20T22:10:51.570024] [DEBUG] [LOCAL ] JQC: Loaded 33824 existing records from primary storage. [2025-10-20T22:10:51.571874] [INFO ] [MAIN ] JQC: Loaded 33824 primary data records. [2025-10-20T22:10:51.597793] [DEBUG] [LOCAL ] JPM: Derivative data saved (1794 records). [2025-10-20T22:10:51.638576] [INFO ] [MAIN ] JPM: Successfully saved derivative data. [2025-10-20T22:10:51.653452] [INFO ] [MAIN ] Derivative Progress: 987/1877 symbols completed. JPM took 1.1s. Estimated remaining: 5m 35s. Rate: 159.4 symbols/min [2025-10-20T22:10:51.668085] [INFO ] [MAIN ] JRS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:51.691743] [DEBUG] [LOCAL ] JRI: Loaded 15423 existing records from primary storage. [2025-10-20T22:10:51.698301] [INFO ] [MAIN ] JRI: Loaded 15423 primary data records. [2025-10-20T22:10:51.876854] [DEBUG] [LOCAL ] JRS: Loaded 11623 existing records from primary storage. [2025-10-20T22:10:52.032157] [INFO ] [MAIN ] JRS: Loaded 11623 primary data records. [2025-10-20T22:10:52.571960] [DEBUG] [MAIN ] JQC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:52.573180] [INFO ] [MAIN ] JQC: Generated 1794 derivative records from 33824 primary records. [2025-10-20T22:10:52.573509] [DEBUG] [LOCAL ] JQC: Saving derivative data. [2025-10-20T22:10:52.590674] [DEBUG] [LOCAL ] JQC: Derivative data saved (1794 records). [2025-10-20T22:10:52.593069] [INFO ] [MAIN ] JQC: Successfully saved derivative data. [2025-10-20T22:10:52.595498] [INFO ] [MAIN ] Derivative Progress: 988/1877 symbols completed. JQC took 1.1s. Estimated remaining: 5m 35s. Rate: 159.2 symbols/min [2025-10-20T22:10:52.609386] [DEBUG] [MAIN ] JRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:52.612194] [INFO ] [MAIN ] JRI: Generated 1794 derivative records from 15423 primary records. [2025-10-20T22:10:52.613029] [DEBUG] [LOCAL ] JRI: Saving derivative data. [2025-10-20T22:10:52.641951] [DEBUG] [LOCAL ] JRI: Derivative data saved (1794 records). [2025-10-20T22:10:52.642115] [INFO ] [MAIN ] JRI: Successfully saved derivative data. [2025-10-20T22:10:52.644198] [INFO ] [MAIN ] Derivative Progress: 989/1877 symbols completed. JRI took 1.1s. Estimated remaining: 5m 34s. Rate: 159.3 symbols/min [2025-10-20T22:10:52.644452] [DEBUG] [MAIN ] JRS: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:10:52.645233] [INFO ] [MAIN ] JRS: Generated 1793 derivative records from 11623 primary records. [2025-10-20T22:10:52.645295] [DEBUG] [LOCAL ] JRS: Saving derivative data. [2025-10-20T22:10:52.659920] [DEBUG] [LOCAL ] JRS: Derivative data saved (1793 records). [2025-10-20T22:10:52.659991] [INFO ] [MAIN ] JRS: Successfully saved derivative data. [2025-10-20T22:10:52.662339] [INFO ] [MAIN ] Derivative Progress: 990/1877 symbols completed. JRS took 1.0s. Estimated remaining: 5m 34s. Rate: 159.5 symbols/min [2025-10-20T22:10:52.662854] [INFO ] [MAIN ] K: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:52.662962] [INFO ] [MAIN ] KAI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:52.662785] [INFO ] [MAIN ] JXN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:52.680355] [DEBUG] [LOCAL ] KAI: Loaded 13606 existing records from primary storage. [2025-10-20T22:10:52.680577] [INFO ] [MAIN ] KAI: Loaded 13606 primary data records. [2025-10-20T22:10:52.898445] [DEBUG] [LOCAL ] K: Loaded 50372 existing records from primary storage. [2025-10-20T22:10:52.919826] [DEBUG] [LOCAL ] JXN: Loaded 44551 existing records from primary storage. [2025-10-20T22:10:52.919975] [INFO ] [MAIN ] K: Loaded 50372 primary data records. [2025-10-20T22:10:53.025691] [INFO ] [MAIN ] JXN: Loaded 44551 primary data records. [2025-10-20T22:10:53.697685] [DEBUG] [MAIN ] KAI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:53.699327] [INFO ] [MAIN ] KAI: Generated 1794 derivative records from 13606 primary records. [2025-10-20T22:10:53.699670] [DEBUG] [LOCAL ] KAI: Saving derivative data. [2025-10-20T22:10:53.716990] [DEBUG] [LOCAL ] KAI: Derivative data saved (1794 records). [2025-10-20T22:10:53.717544] [INFO ] [MAIN ] KAI: Successfully saved derivative data. [2025-10-20T22:10:53.719463] [INFO ] [MAIN ] Derivative Progress: 991/1877 symbols completed. KAI took 1.1s. Estimated remaining: 5m 34s. Rate: 159.2 symbols/min [2025-10-20T22:10:53.719962] [INFO ] [MAIN ] KAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:53.739362] [DEBUG] [MAIN ] K: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:53.741920] [INFO ] [MAIN ] K: Generated 1794 derivative records from 50372 primary records. [2025-10-20T22:10:53.742106] [DEBUG] [LOCAL ] K: Saving derivative data. [2025-10-20T22:10:53.770567] [DEBUG] [MAIN ] JXN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:53.773569] [INFO ] [MAIN ] JXN: Generated 1794 derivative records from 44551 primary records. [2025-10-20T22:10:53.776192] [DEBUG] [LOCAL ] JXN: Saving derivative data. [2025-10-20T22:10:53.776083] [DEBUG] [LOCAL ] K: Derivative data saved (1794 records). [2025-10-20T22:10:53.777169] [INFO ] [MAIN ] K: Successfully saved derivative data. [2025-10-20T22:10:53.779150] [INFO ] [MAIN ] Derivative Progress: 992/1877 symbols completed. K took 1.1s. Estimated remaining: 5m 33s. Rate: 159.3 symbols/min [2025-10-20T22:10:53.788621] [INFO ] [MAIN ] KB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:53.803663] [DEBUG] [LOCAL ] JXN: Derivative data saved (1794 records). [2025-10-20T22:10:53.804109] [INFO ] [MAIN ] JXN: Successfully saved derivative data. [2025-10-20T22:10:53.806469] [INFO ] [MAIN ] Derivative Progress: 993/1877 symbols completed. JXN took 1.1s. Estimated remaining: 5m 33s. Rate: 159.5 symbols/min [2025-10-20T22:10:53.806967] [INFO ] [MAIN ] KBDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:53.822414] [DEBUG] [LOCAL ] KAR: Loaded 39864 existing records from primary storage. [2025-10-20T22:10:53.822868] [INFO ] [MAIN ] KAR: Loaded 39864 primary data records. [2025-10-20T22:10:53.936174] [DEBUG] [LOCAL ] KBDC: Loaded 22651 existing records from primary storage. [2025-10-20T22:10:53.955931] [INFO ] [MAIN ] KBDC: Loaded 22651 primary data records. [2025-10-20T22:10:53.967905] [DEBUG] [LOCAL ] KB: Loaded 23270 existing records from primary storage. [2025-10-20T22:10:53.993709] [INFO ] [MAIN ] KB: Loaded 23270 primary data records. [2025-10-20T22:10:54.772449] [DEBUG] [MAIN ] KAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:54.773235] [INFO ] [MAIN ] KAR: Generated 1794 derivative records from 39864 primary records. [2025-10-20T22:10:54.773524] [DEBUG] [LOCAL ] KAR: Saving derivative data. [2025-10-20T22:10:54.790956] [DEBUG] [LOCAL ] KAR: Derivative data saved (1794 records). [2025-10-20T22:10:54.791179] [INFO ] [MAIN ] KAR: Successfully saved derivative data. [2025-10-20T22:10:54.793077] [INFO ] [MAIN ] Derivative Progress: 994/1877 symbols completed. KAR took 1.1s. Estimated remaining: 5m 33s. Rate: 159.2 symbols/min [2025-10-20T22:10:54.793456] [INFO ] [MAIN ] KBH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:54.813461] [DEBUG] [MAIN ] KB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:54.826027] [INFO ] [MAIN ] KB: Generated 1794 derivative records from 23270 primary records. [2025-10-20T22:10:54.826239] [DEBUG] [LOCAL ] KB: Saving derivative data. [2025-10-20T22:10:54.826490] [DEBUG] [MAIN ] KBDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:54.827686] [INFO ] [MAIN ] KBDC: Generated 1794 derivative records from 22651 primary records. [2025-10-20T22:10:54.827944] [DEBUG] [LOCAL ] KBDC: Saving derivative data. [2025-10-20T22:10:54.851115] [DEBUG] [LOCAL ] KB: Derivative data saved (1794 records). [2025-10-20T22:10:54.855419] [INFO ] [MAIN ] KB: Successfully saved derivative data. [2025-10-20T22:10:54.858730] [INFO ] [MAIN ] Derivative Progress: 995/1877 symbols completed. KB took 1.1s. Estimated remaining: 5m 32s. Rate: 159.3 symbols/min [2025-10-20T22:10:54.860215] [INFO ] [MAIN ] KBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:54.861344] [DEBUG] [LOCAL ] KBDC: Derivative data saved (1794 records). [2025-10-20T22:10:54.861726] [INFO ] [MAIN ] KBDC: Successfully saved derivative data. [2025-10-20T22:10:54.863863] [INFO ] [MAIN ] Derivative Progress: 996/1877 symbols completed. KBDC took 1.1s. Estimated remaining: 5m 31s. Rate: 159.5 symbols/min [2025-10-20T22:10:54.864067] [INFO ] [MAIN ] KD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:54.896250] [DEBUG] [LOCAL ] KBH: Loaded 48759 existing records from primary storage. [2025-10-20T22:10:54.896488] [INFO ] [MAIN ] KBH: Loaded 48759 primary data records. [2025-10-20T22:10:55.249662] [DEBUG] [LOCAL ] KBR: Loaded 48051 existing records from primary storage. [2025-10-20T22:10:55.249885] [INFO ] [MAIN ] KBR: Loaded 48051 primary data records. [2025-10-20T22:10:55.250580] [DEBUG] [LOCAL ] KD: Loaded 52354 existing records from primary storage. [2025-10-20T22:10:55.250696] [INFO ] [MAIN ] KD: Loaded 52354 primary data records. [2025-10-20T22:10:55.893636] [DEBUG] [MAIN ] KBH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:55.894842] [INFO ] [MAIN ] KBH: Generated 1794 derivative records from 48759 primary records. [2025-10-20T22:10:55.895124] [DEBUG] [LOCAL ] KBH: Saving derivative data. [2025-10-20T22:10:55.912250] [DEBUG] [LOCAL ] KBH: Derivative data saved (1794 records). [2025-10-20T22:10:55.912600] [INFO ] [MAIN ] KBH: Successfully saved derivative data. [2025-10-20T22:10:55.914941] [INFO ] [MAIN ] Derivative Progress: 997/1877 symbols completed. KBH took 1.1s. Estimated remaining: 5m 32s. Rate: 159.2 symbols/min [2025-10-20T22:10:55.915204] [INFO ] [MAIN ] KEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:55.935921] [DEBUG] [LOCAL ] KEN: Loaded 4695 existing records from primary storage. [2025-10-20T22:10:55.938082] [DEBUG] [MAIN ] KBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:55.937859] [INFO ] [MAIN ] KEN: Loaded 4695 primary data records. [2025-10-20T22:10:55.950253] [INFO ] [MAIN ] KBR: Generated 1794 derivative records from 48051 primary records. [2025-10-20T22:10:55.969864] [DEBUG] [LOCAL ] KBR: Saving derivative data. [2025-10-20T22:10:56.038426] [DEBUG] [MAIN ] KD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:56.039181] [INFO ] [MAIN ] KD: Generated 1794 derivative records from 52354 primary records. [2025-10-20T22:10:56.052126] [DEBUG] [LOCAL ] KD: Saving derivative data. [2025-10-20T22:10:56.078605] [DEBUG] [LOCAL ] KBR: Derivative data saved (1794 records). [2025-10-20T22:10:56.100514] [INFO ] [MAIN ] KBR: Successfully saved derivative data. [2025-10-20T22:10:56.103360] [INFO ] [MAIN ] Derivative Progress: 998/1877 symbols completed. KBR took 1.2s. Estimated remaining: 5m 31s. Rate: 159.3 symbols/min [2025-10-20T22:10:56.101571] [DEBUG] [LOCAL ] KD: Derivative data saved (1794 records). [2025-10-20T22:10:56.113778] [INFO ] [MAIN ] KD: Successfully saved derivative data. [2025-10-20T22:10:56.138367] [INFO ] [MAIN ] KEP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:56.161532] [INFO ] [MAIN ] Derivative Progress: 999/1877 symbols completed. KD took 1.3s. Estimated remaining: 5m 30s. Rate: 159.4 symbols/min [2025-10-20T22:10:56.285556] [INFO ] [MAIN ] KEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:56.339596] [DEBUG] [LOCAL ] KEP: Loaded 32390 existing records from primary storage. [2025-10-20T22:10:56.339874] [DEBUG] [MAIN ] KEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:56.340620] [INFO ] [MAIN ] KEN: Generated 1794 derivative records from 4695 primary records. [2025-10-20T22:10:56.340787] [DEBUG] [LOCAL ] KEN: Saving derivative data. [2025-10-20T22:10:56.340674] [INFO ] [MAIN ] KEP: Loaded 32390 primary data records. [2025-10-20T22:10:56.462903] [DEBUG] [LOCAL ] KEX: Loaded 38824 existing records from primary storage. [2025-10-20T22:10:56.475441] [INFO ] [MAIN ] KEX: Loaded 38824 primary data records. [2025-10-20T22:10:56.475322] [DEBUG] [LOCAL ] KEN: Derivative data saved (1794 records). [2025-10-20T22:10:56.479305] [INFO ] [MAIN ] KEN: Successfully saved derivative data. [2025-10-20T22:10:56.493205] [INFO ] [MAIN ] Derivative Progress: 1000/1877 symbols completed. KEN took 0.6s. Estimated remaining: 5m 30s. Rate: 159.4 symbols/min [2025-10-20T22:10:56.514158] [INFO ] [MAIN ] KEYS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:57.053611] [DEBUG] [LOCAL ] KEYS: Loaded 43295 existing records from primary storage. [2025-10-20T22:10:57.053966] [INFO ] [MAIN ] KEYS: Loaded 43295 primary data records. [2025-10-20T22:10:57.099903] [DEBUG] [MAIN ] KEP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:57.129558] [INFO ] [MAIN ] KEP: Generated 1794 derivative records from 32390 primary records. [2025-10-20T22:10:57.130741] [DEBUG] [LOCAL ] KEP: Saving derivative data. [2025-10-20T22:10:57.198889] [DEBUG] [LOCAL ] KEP: Derivative data saved (1794 records). [2025-10-20T22:10:57.203114] [INFO ] [MAIN ] KEP: Successfully saved derivative data. [2025-10-20T22:10:57.223368] [INFO ] [MAIN ] Derivative Progress: 1001/1877 symbols completed. KEP took 1.1s. Estimated remaining: 5m 30s. Rate: 159.3 symbols/min [2025-10-20T22:10:57.239139] [INFO ] [MAIN ] KF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:57.244830] [DEBUG] [MAIN ] KEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:57.280172] [INFO ] [MAIN ] KEX: Generated 1794 derivative records from 38824 primary records. [2025-10-20T22:10:57.423966] [DEBUG] [LOCAL ] KEX: Saving derivative data. [2025-10-20T22:10:57.427401] [DEBUG] [LOCAL ] KF: Loaded 2366 existing records from primary storage. [2025-10-20T22:10:57.429080] [INFO ] [MAIN ] KF: Loaded 2366 primary data records. [2025-10-20T22:10:57.523672] [DEBUG] [LOCAL ] KEX: Derivative data saved (1794 records). [2025-10-20T22:10:57.535183] [INFO ] [MAIN ] KEX: Successfully saved derivative data. [2025-10-20T22:10:57.553814] [INFO ] [MAIN ] Derivative Progress: 1002/1877 symbols completed. KEX took 1.3s. Estimated remaining: 5m 30s. Rate: 159.3 symbols/min [2025-10-20T22:10:57.554061] [INFO ] [MAIN ] KFRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:57.795632] [DEBUG] [LOCAL ] KFRC: Loaded 20727 existing records from primary storage. [2025-10-20T22:10:57.796060] [INFO ] [MAIN ] KFRC: Loaded 20727 primary data records. [2025-10-20T22:10:57.923812] [DEBUG] [MAIN ] KEYS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:57.974750] [INFO ] [MAIN ] KEYS: Generated 1794 derivative records from 43295 primary records. [2025-10-20T22:10:58.007293] [DEBUG] [LOCAL ] KEYS: Saving derivative data. [2025-10-20T22:10:58.146666] [DEBUG] [LOCAL ] KEYS: Derivative data saved (1794 records). [2025-10-20T22:10:58.147565] [INFO ] [MAIN ] KEYS: Successfully saved derivative data. [2025-10-20T22:10:58.150017] [INFO ] [MAIN ] Derivative Progress: 1003/1877 symbols completed. KEYS took 1.6s. Estimated remaining: 5m 29s. Rate: 159.2 symbols/min [2025-10-20T22:10:58.167491] [DEBUG] [MAIN ] KF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:58.168289] [INFO ] [MAIN ] KF: Generated 1794 derivative records from 2366 primary records. [2025-10-20T22:10:58.168486] [DEBUG] [LOCAL ] KF: Saving derivative data. [2025-10-20T22:10:58.182801] [DEBUG] [LOCAL ] KF: Derivative data saved (1794 records). [2025-10-20T22:10:58.182960] [INFO ] [MAIN ] KF: Successfully saved derivative data. [2025-10-20T22:10:58.185146] [INFO ] [MAIN ] Derivative Progress: 1004/1877 symbols completed. KF took 0.9s. Estimated remaining: 5m 29s. Rate: 159.4 symbols/min [2025-10-20T22:10:58.202026] [DEBUG] [MAIN ] KFRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:58.202675] [INFO ] [MAIN ] KFRC: Generated 1794 derivative records from 20727 primary records. [2025-10-20T22:10:58.202738] [DEBUG] [LOCAL ] KFRC: Saving derivative data. [2025-10-20T22:10:58.218214] [DEBUG] [LOCAL ] KFRC: Derivative data saved (1794 records). [2025-10-20T22:10:58.218302] [INFO ] [MAIN ] KFRC: Successfully saved derivative data. [2025-10-20T22:10:58.220885] [INFO ] [MAIN ] Derivative Progress: 1005/1877 symbols completed. KFRC took 0.7s. Estimated remaining: 5m 28s. Rate: 159.5 symbols/min [2025-10-20T22:10:58.221843] [INFO ] [MAIN ] KFS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:58.221909] [INFO ] [MAIN ] KGC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:58.223973] [INFO ] [MAIN ] KFY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:58.235385] [DEBUG] [LOCAL ] KFS: Loaded 8699 existing records from primary storage. [2025-10-20T22:10:58.235526] [INFO ] [MAIN ] KFS: Loaded 8699 primary data records. [2025-10-20T22:10:58.463072] [DEBUG] [LOCAL ] KFY: Loaded 29744 existing records from primary storage. [2025-10-20T22:10:58.564309] [DEBUG] [LOCAL ] KGC: Loaded 53700 existing records from primary storage. [2025-10-20T22:10:58.569772] [INFO ] [MAIN ] KFY: Loaded 29744 primary data records. [2025-10-20T22:10:58.569670] [INFO ] [MAIN ] KGC: Loaded 53700 primary data records. [2025-10-20T22:10:59.203014] [DEBUG] [MAIN ] KFS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:59.204253] [INFO ] [MAIN ] KFS: Generated 1794 derivative records from 8699 primary records. [2025-10-20T22:10:59.204459] [DEBUG] [LOCAL ] KFS: Saving derivative data. [2025-10-20T22:10:59.222953] [DEBUG] [LOCAL ] KFS: Derivative data saved (1794 records). [2025-10-20T22:10:59.223232] [INFO ] [MAIN ] KFS: Successfully saved derivative data. [2025-10-20T22:10:59.225664] [INFO ] [MAIN ] Derivative Progress: 1006/1877 symbols completed. KFS took 1.0s. Estimated remaining: 5m 28s. Rate: 159.2 symbols/min [2025-10-20T22:10:59.226156] [INFO ] [MAIN ] KGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:59.247602] [DEBUG] [MAIN ] KGC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:59.251509] [INFO ] [MAIN ] KGC: Generated 1794 derivative records from 53700 primary records. [2025-10-20T22:10:59.251798] [DEBUG] [LOCAL ] KGC: Saving derivative data. [2025-10-20T22:10:59.274773] [DEBUG] [MAIN ] KFY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:10:59.281792] [DEBUG] [LOCAL ] KGC: Derivative data saved (1794 records). [2025-10-20T22:10:59.282017] [INFO ] [MAIN ] KGC: Successfully saved derivative data. [2025-10-20T22:10:59.282774] [INFO ] [MAIN ] KFY: Generated 1794 derivative records from 29744 primary records. [2025-10-20T22:10:59.285677] [INFO ] [MAIN ] Derivative Progress: 1007/1877 symbols completed. KGC took 1.1s. Estimated remaining: 5m 28s. Rate: 159.4 symbols/min [2025-10-20T22:10:59.285805] [DEBUG] [LOCAL ] KFY: Saving derivative data. [2025-10-20T22:10:59.286032] [INFO ] [MAIN ] KIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:59.313013] [DEBUG] [LOCAL ] KFY: Derivative data saved (1794 records). [2025-10-20T22:10:59.313182] [INFO ] [MAIN ] KFY: Successfully saved derivative data. [2025-10-20T22:10:59.315471] [INFO ] [MAIN ] Derivative Progress: 1008/1877 symbols completed. KFY took 1.1s. Estimated remaining: 5m 27s. Rate: 159.5 symbols/min [2025-10-20T22:10:59.315795] [INFO ] [MAIN ] KIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:10:59.331588] [DEBUG] [LOCAL ] KGS: Loaded 46120 existing records from primary storage. [2025-10-20T22:10:59.331910] [INFO ] [MAIN ] KGS: Loaded 46120 primary data records. [2025-10-20T22:10:59.561081] [DEBUG] [LOCAL ] KIO: Loaded 21766 existing records from primary storage. [2025-10-20T22:10:59.568170] [INFO ] [MAIN ] KIO: Loaded 21766 primary data records. [2025-10-20T22:10:59.813364] [DEBUG] [LOCAL ] KIM: Loaded 53012 existing records from primary storage. [2025-10-20T22:10:59.831632] [INFO ] [MAIN ] KIM: Loaded 53012 primary data records. [2025-10-20T22:11:00.420135] [DEBUG] [MAIN ] KGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:00.421205] [INFO ] [MAIN ] KGS: Generated 1794 derivative records from 46120 primary records. [2025-10-20T22:11:00.421423] [DEBUG] [LOCAL ] KGS: Saving derivative data. [2025-10-20T22:11:00.438907] [DEBUG] [LOCAL ] KGS: Derivative data saved (1794 records). [2025-10-20T22:11:00.440022] [INFO ] [MAIN ] KGS: Successfully saved derivative data. [2025-10-20T22:11:00.442180] [INFO ] [MAIN ] Derivative Progress: 1009/1877 symbols completed. KGS took 1.2s. Estimated remaining: 5m 27s. Rate: 159.2 symbols/min [2025-10-20T22:11:00.443426] [INFO ] [MAIN ] KKR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:00.459814] [DEBUG] [MAIN ] KIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:00.460882] [INFO ] [MAIN ] KIO: Generated 1794 derivative records from 21766 primary records. [2025-10-20T22:11:00.461101] [DEBUG] [LOCAL ] KIO: Saving derivative data. [2025-10-20T22:11:00.481673] [DEBUG] [LOCAL ] KIO: Derivative data saved (1794 records). [2025-10-20T22:11:00.481934] [INFO ] [MAIN ] KIO: Successfully saved derivative data. [2025-10-20T22:11:00.484460] [INFO ] [MAIN ] Derivative Progress: 1010/1877 symbols completed. KIO took 1.2s. Estimated remaining: 5m 26s. Rate: 159.4 symbols/min [2025-10-20T22:11:00.484817] [INFO ] [MAIN ] KKRS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:00.498975] [DEBUG] [MAIN ] KIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:00.499894] [INFO ] [MAIN ] KIM: Generated 1794 derivative records from 53012 primary records. [2025-10-20T22:11:00.500066] [DEBUG] [LOCAL ] KIM: Saving derivative data. [2025-10-20T22:11:00.531448] [DEBUG] [LOCAL ] KKRS: Loaded 5590 existing records from primary storage. [2025-10-20T22:11:00.535820] [INFO ] [MAIN ] KKRS: Loaded 5590 primary data records. [2025-10-20T22:11:00.541433] [DEBUG] [LOCAL ] KIM: Derivative data saved (1794 records). [2025-10-20T22:11:00.590282] [INFO ] [MAIN ] KIM: Successfully saved derivative data. [2025-10-20T22:11:00.616336] [INFO ] [MAIN ] Derivative Progress: 1011/1877 symbols completed. KIM took 1.3s. Estimated remaining: 5m 26s. Rate: 159.5 symbols/min [2025-10-20T22:11:00.658922] [DEBUG] [LOCAL ] KKR: Loaded 53503 existing records from primary storage. [2025-10-20T22:11:00.652612] [INFO ] [MAIN ] KLC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:00.664564] [INFO ] [MAIN ] KKR: Loaded 53503 primary data records. [2025-10-20T22:11:01.218333] [DEBUG] [LOCAL ] KLC: Loaded 36293 existing records from primary storage. [2025-10-20T22:11:01.218746] [INFO ] [MAIN ] KLC: Loaded 36293 primary data records. [2025-10-20T22:11:01.571410] [DEBUG] [MAIN ] KKRS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:01.574057] [INFO ] [MAIN ] KKRS: Generated 1794 derivative records from 5590 primary records. [2025-10-20T22:11:01.574461] [DEBUG] [LOCAL ] KKRS: Saving derivative data. [2025-10-20T22:11:01.596629] [DEBUG] [LOCAL ] KKRS: Derivative data saved (1794 records). [2025-10-20T22:11:01.597110] [INFO ] [MAIN ] KKRS: Successfully saved derivative data. [2025-10-20T22:11:01.599208] [INFO ] [MAIN ] Derivative Progress: 1012/1877 symbols completed. KKRS took 1.1s. Estimated remaining: 5m 26s. Rate: 159.2 symbols/min [2025-10-20T22:11:01.599367] [INFO ] [MAIN ] KMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:01.614698] [DEBUG] [MAIN ] KKR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:01.615781] [INFO ] [MAIN ] KKR: Generated 1794 derivative records from 53503 primary records. [2025-10-20T22:11:01.616205] [DEBUG] [LOCAL ] KKR: Saving derivative data. [2025-10-20T22:11:01.635411] [DEBUG] [LOCAL ] KKR: Derivative data saved (1794 records). [2025-10-20T22:11:01.636029] [INFO ] [MAIN ] KKR: Successfully saved derivative data. [2025-10-20T22:11:01.638739] [INFO ] [MAIN ] Derivative Progress: 1013/1877 symbols completed. KKR took 1.2s. Estimated remaining: 5m 25s. Rate: 159.3 symbols/min [2025-10-20T22:11:01.639106] [INFO ] [MAIN ] KMPB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:01.648067] [DEBUG] [LOCAL ] KMPB: Loaded 3558 existing records from primary storage. [2025-10-20T22:11:01.648380] [INFO ] [MAIN ] KMPB: Loaded 3558 primary data records. [2025-10-20T22:11:01.742580] [DEBUG] [MAIN ] KLC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:01.784977] [INFO ] [MAIN ] KLC: Generated 1794 derivative records from 36293 primary records. [2025-10-20T22:11:01.806273] [DEBUG] [LOCAL ] KLC: Saving derivative data. [2025-10-20T22:11:01.846395] [DEBUG] [LOCAL ] KMI: Loaded 53693 existing records from primary storage. [2025-10-20T22:11:01.863865] [INFO ] [MAIN ] KMI: Loaded 53693 primary data records. [2025-10-20T22:11:02.168422] [DEBUG] [LOCAL ] KLC: Derivative data saved (1794 records). [2025-10-20T22:11:02.168852] [INFO ] [MAIN ] KLC: Successfully saved derivative data. [2025-10-20T22:11:02.172604] [INFO ] [MAIN ] Derivative Progress: 1014/1877 symbols completed. KLC took 1.5s. Estimated remaining: 5m 25s. Rate: 159.3 symbols/min [2025-10-20T22:11:02.189186] [INFO ] [MAIN ] KMPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:02.397164] [DEBUG] [MAIN ] KMPB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:02.399302] [INFO ] [MAIN ] KMPB: Generated 1794 derivative records from 3558 primary records. [2025-10-20T22:11:02.399545] [DEBUG] [LOCAL ] KMPB: Saving derivative data. [2025-10-20T22:11:02.424377] [DEBUG] [LOCAL ] KMPB: Derivative data saved (1794 records). [2025-10-20T22:11:02.424559] [INFO ] [MAIN ] KMPB: Successfully saved derivative data. [2025-10-20T22:11:02.426573] [INFO ] [MAIN ] Derivative Progress: 1015/1877 symbols completed. KMPB took 0.8s. Estimated remaining: 5m 25s. Rate: 159.3 symbols/min [2025-10-20T22:11:02.427042] [INFO ] [MAIN ] KMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:02.452598] [DEBUG] [LOCAL ] KMPR: Loaded 40452 existing records from primary storage. [2025-10-20T22:11:02.457880] [INFO ] [MAIN ] KMPR: Loaded 40452 primary data records. [2025-10-20T22:11:02.458199] [DEBUG] [MAIN ] KMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:02.479741] [INFO ] [MAIN ] KMI: Generated 1794 derivative records from 53693 primary records. [2025-10-20T22:11:02.495184] [DEBUG] [LOCAL ] KMI: Saving derivative data. [2025-10-20T22:11:02.578797] [DEBUG] [LOCAL ] KMI: Derivative data saved (1794 records). [2025-10-20T22:11:02.603635] [INFO ] [MAIN ] KMI: Successfully saved derivative data. [2025-10-20T22:11:02.659148] [INFO ] [MAIN ] Derivative Progress: 1016/1877 symbols completed. KMI took 1.1s. Estimated remaining: 5m 24s. Rate: 159.4 symbols/min [2025-10-20T22:11:02.697027] [INFO ] [MAIN ] KMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:02.814472] [DEBUG] [LOCAL ] KMT: Loaded 42948 existing records from primary storage. [2025-10-20T22:11:02.814984] [INFO ] [MAIN ] KMT: Loaded 42948 primary data records. [2025-10-20T22:11:03.150962] [DEBUG] [LOCAL ] KMX: Loaded 53284 existing records from primary storage. [2025-10-20T22:11:03.151494] [INFO ] [MAIN ] KMX: Loaded 53284 primary data records. [2025-10-20T22:11:03.239537] [DEBUG] [MAIN ] KMPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:03.269993] [INFO ] [MAIN ] KMPR: Generated 1794 derivative records from 40452 primary records. [2025-10-20T22:11:03.270172] [DEBUG] [LOCAL ] KMPR: Saving derivative data. [2025-10-20T22:11:03.324831] [DEBUG] [LOCAL ] KMPR: Derivative data saved (1794 records). [2025-10-20T22:11:03.325262] [INFO ] [MAIN ] KMPR: Successfully saved derivative data. [2025-10-20T22:11:03.328240] [INFO ] [MAIN ] Derivative Progress: 1017/1877 symbols completed. KMPR took 1.1s. Estimated remaining: 5m 24s. Rate: 159.3 symbols/min [2025-10-20T22:11:03.343780] [INFO ] [MAIN ] KN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:03.503019] [DEBUG] [MAIN ] KMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:03.505106] [INFO ] [MAIN ] KMT: Generated 1794 derivative records from 42948 primary records. [2025-10-20T22:11:03.505291] [DEBUG] [LOCAL ] KMT: Saving derivative data. [2025-10-20T22:11:03.527856] [DEBUG] [LOCAL ] KMT: Derivative data saved (1794 records). [2025-10-20T22:11:03.528060] [INFO ] [MAIN ] KMT: Successfully saved derivative data. [2025-10-20T22:11:03.530192] [INFO ] [MAIN ] Derivative Progress: 1018/1877 symbols completed. KMT took 1.1s. Estimated remaining: 5m 23s. Rate: 159.3 symbols/min [2025-10-20T22:11:03.557105] [DEBUG] [MAIN ] KMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:03.557825] [INFO ] [MAIN ] KMX: Generated 1794 derivative records from 53284 primary records. [2025-10-20T22:11:03.557889] [DEBUG] [LOCAL ] KMX: Saving derivative data. [2025-10-20T22:11:03.584284] [DEBUG] [LOCAL ] KN: Loaded 38488 existing records from primary storage. [2025-10-20T22:11:03.584816] [INFO ] [MAIN ] KN: Loaded 38488 primary data records. [2025-10-20T22:11:03.655450] [DEBUG] [LOCAL ] KMX: Derivative data saved (1794 records). [2025-10-20T22:11:03.677209] [INFO ] [MAIN ] KMX: Successfully saved derivative data. [2025-10-20T22:11:03.693480] [INFO ] [MAIN ] Derivative Progress: 1019/1877 symbols completed. KMX took 1.0s. Estimated remaining: 5m 23s. Rate: 159.4 symbols/min [2025-10-20T22:11:03.920721] [DEBUG] [MAIN ] KN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:03.921361] [INFO ] [MAIN ] KN: Generated 1794 derivative records from 38488 primary records. [2025-10-20T22:11:03.921431] [DEBUG] [LOCAL ] KN: Saving derivative data. [2025-10-20T22:11:03.936401] [DEBUG] [LOCAL ] KN: Derivative data saved (1794 records). [2025-10-20T22:11:03.936520] [INFO ] [MAIN ] KN: Successfully saved derivative data. [2025-10-20T22:11:03.938804] [INFO ] [MAIN ] Derivative Progress: 1020/1877 symbols completed. KN took 0.6s. Estimated remaining: 5m 22s. Rate: 159.5 symbols/min [2025-10-20T22:11:03.939272] [INFO ] [MAIN ] KNF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:03.939380] [INFO ] [MAIN ] KNSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:03.939687] [INFO ] [MAIN ] KNOP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:03.959252] [DEBUG] [LOCAL ] KNOP: Loaded 10181 existing records from primary storage. [2025-10-20T22:11:03.959421] [INFO ] [MAIN ] KNOP: Loaded 10181 primary data records. [2025-10-20T22:11:04.065884] [DEBUG] [LOCAL ] KNSL: Loaded 18516 existing records from primary storage. [2025-10-20T22:11:04.097603] [INFO ] [MAIN ] KNSL: Loaded 18516 primary data records. [2025-10-20T22:11:04.109317] [DEBUG] [LOCAL ] KNF: Loaded 36667 existing records from primary storage. [2025-10-20T22:11:04.172892] [INFO ] [MAIN ] KNF: Loaded 36667 primary data records. [2025-10-20T22:11:04.913504] [DEBUG] [MAIN ] KNOP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:04.914326] [INFO ] [MAIN ] KNOP: Generated 1794 derivative records from 10181 primary records. [2025-10-20T22:11:04.914497] [DEBUG] [LOCAL ] KNOP: Saving derivative data. [2025-10-20T22:11:04.932413] [DEBUG] [LOCAL ] KNOP: Derivative data saved (1794 records). [2025-10-20T22:11:04.940411] [INFO ] [MAIN ] KNOP: Successfully saved derivative data. [2025-10-20T22:11:04.947201] [INFO ] [MAIN ] Derivative Progress: 1021/1877 symbols completed. KNOP took 1.0s. Estimated remaining: 5m 23s. Rate: 159.2 symbols/min [2025-10-20T22:11:04.947356] [INFO ] [MAIN ] KNTK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:04.948601] [DEBUG] [MAIN ] KNSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:04.949703] [INFO ] [MAIN ] KNSL: Generated 1794 derivative records from 18516 primary records. [2025-10-20T22:11:04.949870] [DEBUG] [LOCAL ] KNSL: Saving derivative data. [2025-10-20T22:11:04.969508] [DEBUG] [LOCAL ] KNSL: Derivative data saved (1794 records). [2025-10-20T22:11:04.969733] [INFO ] [MAIN ] KNSL: Successfully saved derivative data. [2025-10-20T22:11:04.972130] [INFO ] [MAIN ] Derivative Progress: 1022/1877 symbols completed. KNSL took 1.0s. Estimated remaining: 5m 22s. Rate: 159.4 symbols/min [2025-10-20T22:11:04.984899] [INFO ] [MAIN ] KNX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:04.985814] [DEBUG] [MAIN ] KNF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:04.986357] [INFO ] [MAIN ] KNF: Generated 1794 derivative records from 36667 primary records. [2025-10-20T22:11:04.986411] [DEBUG] [LOCAL ] KNF: Saving derivative data. [2025-10-20T22:11:05.002586] [DEBUG] [LOCAL ] KNF: Derivative data saved (1794 records). [2025-10-20T22:11:05.002815] [INFO ] [MAIN ] KNF: Successfully saved derivative data. [2025-10-20T22:11:05.005451] [INFO ] [MAIN ] Derivative Progress: 1023/1877 symbols completed. KNF took 1.1s. Estimated remaining: 5m 21s. Rate: 159.5 symbols/min [2025-10-20T22:11:05.005860] [INFO ] [MAIN ] KOF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:05.045037] [DEBUG] [LOCAL ] KNTK: Loaded 47220 existing records from primary storage. [2025-10-20T22:11:05.045261] [INFO ] [MAIN ] KNTK: Loaded 47220 primary data records. [2025-10-20T22:11:05.164080] [DEBUG] [LOCAL ] KOF: Loaded 26742 existing records from primary storage. [2025-10-20T22:11:05.182209] [INFO ] [MAIN ] KOF: Loaded 26742 primary data records. [2025-10-20T22:11:05.376684] [DEBUG] [LOCAL ] KNX: Loaded 52639 existing records from primary storage. [2025-10-20T22:11:05.510009] [INFO ] [MAIN ] KNX: Loaded 52639 primary data records. [2025-10-20T22:11:06.088942] [DEBUG] [MAIN ] KNTK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:06.090203] [INFO ] [MAIN ] KNTK: Generated 1794 derivative records from 47220 primary records. [2025-10-20T22:11:06.090847] [DEBUG] [LOCAL ] KNTK: Saving derivative data. [2025-10-20T22:11:06.125480] [DEBUG] [LOCAL ] KNTK: Derivative data saved (1794 records). [2025-10-20T22:11:06.126183] [INFO ] [MAIN ] KNTK: Successfully saved derivative data. [2025-10-20T22:11:06.128349] [INFO ] [MAIN ] Derivative Progress: 1024/1877 symbols completed. KNTK took 1.2s. Estimated remaining: 5m 21s. Rate: 159.2 symbols/min [2025-10-20T22:11:06.126092] [DEBUG] [MAIN ] KOF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:06.128571] [INFO ] [MAIN ] KOP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:06.130068] [INFO ] [MAIN ] KOF: Generated 1794 derivative records from 26742 primary records. [2025-10-20T22:11:06.130168] [DEBUG] [LOCAL ] KOF: Saving derivative data. [2025-10-20T22:11:06.148912] [DEBUG] [LOCAL ] KOF: Derivative data saved (1794 records). [2025-10-20T22:11:06.152914] [INFO ] [MAIN ] KOF: Successfully saved derivative data. [2025-10-20T22:11:06.155235] [INFO ] [MAIN ] Derivative Progress: 1025/1877 symbols completed. KOF took 1.1s. Estimated remaining: 5m 21s. Rate: 159.3 symbols/min [2025-10-20T22:11:06.155476] [INFO ] [MAIN ] KORE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:06.166555] [DEBUG] [LOCAL ] KOP: Loaded 17454 existing records from primary storage. [2025-10-20T22:11:06.166842] [INFO ] [MAIN ] KOP: Loaded 17454 primary data records. [2025-10-20T22:11:06.182522] [DEBUG] [LOCAL ] KORE: Loaded 2176 existing records from primary storage. [2025-10-20T22:11:06.182875] [DEBUG] [MAIN ] KNX: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:06.194862] [INFO ] [MAIN ] KORE: Loaded 2176 primary data records. [2025-10-20T22:11:06.194789] [INFO ] [MAIN ] KNX: Generated 1793 derivative records from 52639 primary records. [2025-10-20T22:11:06.220036] [DEBUG] [LOCAL ] KNX: Saving derivative data. [2025-10-20T22:11:06.394302] [DEBUG] [LOCAL ] KNX: Derivative data saved (1793 records). [2025-10-20T22:11:06.421707] [INFO ] [MAIN ] KNX: Successfully saved derivative data. [2025-10-20T22:11:06.429507] [INFO ] [MAIN ] Derivative Progress: 1026/1877 symbols completed. KNX took 1.4s. Estimated remaining: 5m 20s. Rate: 159.4 symbols/min [2025-10-20T22:11:06.446121] [INFO ] [MAIN ] KOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:06.853868] [DEBUG] [MAIN ] KOP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:06.855873] [INFO ] [MAIN ] KOP: Generated 1794 derivative records from 17454 primary records. [2025-10-20T22:11:06.856077] [DEBUG] [LOCAL ] KOP: Saving derivative data. [2025-10-20T22:11:06.884624] [DEBUG] [LOCAL ] KOP: Derivative data saved (1794 records). [2025-10-20T22:11:06.900445] [INFO ] [MAIN ] KOP: Successfully saved derivative data. [2025-10-20T22:11:06.884939] [DEBUG] [MAIN ] KORE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:06.901253] [INFO ] [MAIN ] KORE: Generated 1794 derivative records from 2176 primary records. [2025-10-20T22:11:06.903178] [INFO ] [MAIN ] Derivative Progress: 1027/1877 symbols completed. KOP took 0.8s. Estimated remaining: 5m 20s. Rate: 159.3 symbols/min [2025-10-20T22:11:06.903364] [INFO ] [MAIN ] KR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:06.904414] [DEBUG] [LOCAL ] KOS: Loaded 53095 existing records from primary storage. [2025-10-20T22:11:06.905039] [DEBUG] [LOCAL ] KORE: Saving derivative data. [2025-10-20T22:11:06.904972] [INFO ] [MAIN ] KOS: Loaded 53095 primary data records. [2025-10-20T22:11:06.984099] [DEBUG] [LOCAL ] KORE: Derivative data saved (1794 records). [2025-10-20T22:11:06.990535] [INFO ] [MAIN ] KORE: Successfully saved derivative data. [2025-10-20T22:11:07.006995] [INFO ] [MAIN ] Derivative Progress: 1028/1877 symbols completed. KORE took 0.8s. Estimated remaining: 5m 19s. Rate: 159.5 symbols/min [2025-10-20T22:11:07.028893] [INFO ] [MAIN ] KRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:07.309231] [DEBUG] [LOCAL ] KR: Loaded 53681 existing records from primary storage. [2025-10-20T22:11:07.309461] [INFO ] [MAIN ] KR: Loaded 53681 primary data records. [2025-10-20T22:11:07.310246] [DEBUG] [LOCAL ] KRC: Loaded 48217 existing records from primary storage. [2025-10-20T22:11:07.331534] [INFO ] [MAIN ] KRC: Loaded 48217 primary data records. [2025-10-20T22:11:07.510794] [DEBUG] [MAIN ] KOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:07.532872] [INFO ] [MAIN ] KOS: Generated 1794 derivative records from 53095 primary records. [2025-10-20T22:11:07.573137] [DEBUG] [LOCAL ] KOS: Saving derivative data. [2025-10-20T22:11:07.698268] [DEBUG] [LOCAL ] KOS: Derivative data saved (1794 records). [2025-10-20T22:11:07.940851] [INFO ] [MAIN ] KOS: Successfully saved derivative data. [2025-10-20T22:11:08.005326] [INFO ] [MAIN ] Derivative Progress: 1029/1877 symbols completed. KOS took 1.6s. Estimated remaining: 5m 20s. Rate: 159.2 symbols/min [2025-10-20T22:11:08.006283] [INFO ] [MAIN ] KREF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:08.037170] [DEBUG] [MAIN ] KR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:08.050282] [INFO ] [MAIN ] KR: Generated 1794 derivative records from 53681 primary records. [2025-10-20T22:11:08.061136] [DEBUG] [LOCAL ] KR: Saving derivative data. [2025-10-20T22:11:08.061004] [DEBUG] [MAIN ] KRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:08.061929] [INFO ] [MAIN ] KRC: Generated 1794 derivative records from 48217 primary records. [2025-10-20T22:11:08.062041] [DEBUG] [LOCAL ] KRC: Saving derivative data. [2025-10-20T22:11:08.091333] [DEBUG] [LOCAL ] KRC: Derivative data saved (1794 records). [2025-10-20T22:11:08.091547] [INFO ] [MAIN ] KRC: Successfully saved derivative data. [2025-10-20T22:11:08.096404] [INFO ] [MAIN ] Derivative Progress: 1030/1877 symbols completed. KRC took 1.1s. Estimated remaining: 5m 19s. Rate: 159.3 symbols/min [2025-10-20T22:11:08.096685] [INFO ] [MAIN ] KRG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:08.098666] [DEBUG] [LOCAL ] KR: Derivative data saved (1794 records). [2025-10-20T22:11:08.112678] [INFO ] [MAIN ] KR: Successfully saved derivative data. [2025-10-20T22:11:08.115184] [INFO ] [MAIN ] Derivative Progress: 1031/1877 symbols completed. KR took 1.2s. Estimated remaining: 5m 18s. Rate: 159.5 symbols/min [2025-10-20T22:11:08.115440] [INFO ] [MAIN ] KRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:08.118279] [DEBUG] [LOCAL ] KREF: Loaded 38036 existing records from primary storage. [2025-10-20T22:11:08.118418] [INFO ] [MAIN ] KREF: Loaded 38036 primary data records. [2025-10-20T22:11:08.356212] [DEBUG] [LOCAL ] KRO: Loaded 26644 existing records from primary storage. [2025-10-20T22:11:08.426323] [INFO ] [MAIN ] KRO: Loaded 26644 primary data records. [2025-10-20T22:11:08.487194] [DEBUG] [LOCAL ] KRG: Loaded 49298 existing records from primary storage. [2025-10-20T22:11:08.505069] [INFO ] [MAIN ] KRG: Loaded 49298 primary data records. [2025-10-20T22:11:09.126053] [DEBUG] [MAIN ] KREF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:09.127407] [INFO ] [MAIN ] KREF: Generated 1794 derivative records from 38036 primary records. [2025-10-20T22:11:09.128093] [DEBUG] [LOCAL ] KREF: Saving derivative data. [2025-10-20T22:11:09.147286] [DEBUG] [LOCAL ] KREF: Derivative data saved (1794 records). [2025-10-20T22:11:09.147745] [INFO ] [MAIN ] KREF: Successfully saved derivative data. [2025-10-20T22:11:09.149876] [INFO ] [MAIN ] Derivative Progress: 1032/1877 symbols completed. KREF took 1.1s. Estimated remaining: 5m 18s. Rate: 159.2 symbols/min [2025-10-20T22:11:09.152245] [INFO ] [MAIN ] KRP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:09.170050] [DEBUG] [MAIN ] KRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:09.172255] [INFO ] [MAIN ] KRO: Generated 1794 derivative records from 26644 primary records. [2025-10-20T22:11:09.173886] [DEBUG] [LOCAL ] KRO: Saving derivative data. [2025-10-20T22:11:09.201342] [DEBUG] [MAIN ] KRG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:09.205328] [INFO ] [MAIN ] KRG: Generated 1794 derivative records from 49298 primary records. [2025-10-20T22:11:09.207667] [DEBUG] [LOCAL ] KRG: Saving derivative data. [2025-10-20T22:11:09.207548] [DEBUG] [LOCAL ] KRO: Derivative data saved (1794 records). [2025-10-20T22:11:09.232517] [DEBUG] [LOCAL ] KRG: Derivative data saved (1794 records). [2025-10-20T22:11:09.232612] [INFO ] [MAIN ] KRG: Successfully saved derivative data. [2025-10-20T22:11:09.228781] [INFO ] [MAIN ] KRO: Successfully saved derivative data. [2025-10-20T22:11:09.234904] [INFO ] [MAIN ] Derivative Progress: 1033/1877 symbols completed. KRO took 1.1s. Estimated remaining: 5m 18s. Rate: 159.3 symbols/min [2025-10-20T22:11:09.237072] [INFO ] [MAIN ] Derivative Progress: 1034/1877 symbols completed. KRG took 1.1s. Estimated remaining: 5m 17s. Rate: 159.5 symbols/min [2025-10-20T22:11:09.250227] [DEBUG] [LOCAL ] KRP: Loaded 36203 existing records from primary storage. [2025-10-20T22:11:09.250362] [INFO ] [MAIN ] KRP: Loaded 36203 primary data records. [2025-10-20T22:11:09.556448] [DEBUG] [MAIN ] KRP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:09.557077] [INFO ] [MAIN ] KRP: Generated 1794 derivative records from 36203 primary records. [2025-10-20T22:11:09.557141] [DEBUG] [LOCAL ] KRP: Saving derivative data. [2025-10-20T22:11:09.572422] [DEBUG] [LOCAL ] KRP: Derivative data saved (1794 records). [2025-10-20T22:11:09.572486] [INFO ] [MAIN ] KRP: Successfully saved derivative data. [2025-10-20T22:11:09.574341] [INFO ] [MAIN ] Derivative Progress: 1035/1877 symbols completed. KRP took 0.4s. Estimated remaining: 5m 17s. Rate: 159.5 symbols/min [2025-10-20T22:11:09.574920] [INFO ] [MAIN ] KSS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:09.575286] [INFO ] [MAIN ] KTF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:09.575328] [INFO ] [MAIN ] KTB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:09.588247] [DEBUG] [LOCAL ] KTF: Loaded 11669 existing records from primary storage. [2025-10-20T22:11:09.588448] [INFO ] [MAIN ] KTF: Loaded 11669 primary data records. [2025-10-20T22:11:09.935518] [DEBUG] [LOCAL ] KTB: Loaded 40517 existing records from primary storage. [2025-10-20T22:11:09.963624] [INFO ] [MAIN ] KTB: Loaded 40517 primary data records. [2025-10-20T22:11:09.987305] [DEBUG] [LOCAL ] KSS: Loaded 53591 existing records from primary storage. [2025-10-20T22:11:10.004025] [INFO ] [MAIN ] KSS: Loaded 53591 primary data records. [2025-10-20T22:11:10.598706] [DEBUG] [MAIN ] KTF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:10.600230] [INFO ] [MAIN ] KTF: Generated 1794 derivative records from 11669 primary records. [2025-10-20T22:11:10.600605] [DEBUG] [LOCAL ] KTF: Saving derivative data. [2025-10-20T22:11:10.619293] [DEBUG] [LOCAL ] KTF: Derivative data saved (1794 records). [2025-10-20T22:11:10.619397] [INFO ] [MAIN ] KTF: Successfully saved derivative data. [2025-10-20T22:11:10.621485] [INFO ] [MAIN ] Derivative Progress: 1036/1877 symbols completed. KTF took 1.0s. Estimated remaining: 5m 17s. Rate: 159.2 symbols/min [2025-10-20T22:11:10.621627] [INFO ] [MAIN ] KVUE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:10.638540] [DEBUG] [MAIN ] KTB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:10.639530] [INFO ] [MAIN ] KTB: Generated 1794 derivative records from 40517 primary records. [2025-10-20T22:11:10.639794] [DEBUG] [LOCAL ] KTB: Saving derivative data. [2025-10-20T22:11:10.659567] [DEBUG] [LOCAL ] KTB: Derivative data saved (1794 records). [2025-10-20T22:11:10.659935] [INFO ] [MAIN ] KTB: Successfully saved derivative data. [2025-10-20T22:11:10.661713] [INFO ] [MAIN ] Derivative Progress: 1037/1877 symbols completed. KTB took 1.1s. Estimated remaining: 5m 16s. Rate: 159.3 symbols/min [2025-10-20T22:11:10.662029] [INFO ] [MAIN ] KVYO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:10.675430] [DEBUG] [MAIN ] KSS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:10.676193] [INFO ] [MAIN ] KSS: Generated 1794 derivative records from 53591 primary records. [2025-10-20T22:11:10.676362] [DEBUG] [LOCAL ] KSS: Saving derivative data. [2025-10-20T22:11:10.705716] [DEBUG] [LOCAL ] KSS: Derivative data saved (1794 records). [2025-10-20T22:11:10.705961] [INFO ] [MAIN ] KSS: Successfully saved derivative data. [2025-10-20T22:11:10.708003] [INFO ] [MAIN ] Derivative Progress: 1038/1877 symbols completed. KSS took 1.1s. Estimated remaining: 5m 16s. Rate: 159.5 symbols/min [2025-10-20T22:11:10.708238] [INFO ] [MAIN ] KW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:10.728181] [DEBUG] [LOCAL ] KVUE: Loaded 53667 existing records from primary storage. [2025-10-20T22:11:10.728534] [INFO ] [MAIN ] KVUE: Loaded 53667 primary data records. [2025-10-20T22:11:11.073035] [DEBUG] [LOCAL ] KW: Loaded 40515 existing records from primary storage. [2025-10-20T22:11:11.074362] [DEBUG] [LOCAL ] KVYO: Loaded 50869 existing records from primary storage. [2025-10-20T22:11:11.074560] [INFO ] [MAIN ] KW: Loaded 40515 primary data records. [2025-10-20T22:11:11.074776] [INFO ] [MAIN ] KVYO: Loaded 50869 primary data records. [2025-10-20T22:11:11.756196] [DEBUG] [MAIN ] KVUE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:11.758615] [INFO ] [MAIN ] KVUE: Generated 1794 derivative records from 53667 primary records. [2025-10-20T22:11:11.758845] [DEBUG] [LOCAL ] KVUE: Saving derivative data. [2025-10-20T22:11:11.782881] [DEBUG] [LOCAL ] KVUE: Derivative data saved (1794 records). [2025-10-20T22:11:11.783335] [INFO ] [MAIN ] KVUE: Successfully saved derivative data. [2025-10-20T22:11:11.785417] [INFO ] [MAIN ] Derivative Progress: 1039/1877 symbols completed. KVUE took 1.2s. Estimated remaining: 5m 16s. Rate: 159.2 symbols/min [2025-10-20T22:11:11.785557] [INFO ] [MAIN ] KWR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:11.837157] [DEBUG] [MAIN ] KW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:11.839138] [DEBUG] [LOCAL ] KWR: Loaded 15952 existing records from primary storage. [2025-10-20T22:11:11.839731] [INFO ] [MAIN ] KWR: Loaded 15952 primary data records. [2025-10-20T22:11:11.840543] [INFO ] [MAIN ] KW: Generated 1794 derivative records from 40515 primary records. [2025-10-20T22:11:11.839595] [DEBUG] [MAIN ] KVYO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:11.852451] [DEBUG] [LOCAL ] KW: Saving derivative data. [2025-10-20T22:11:11.854387] [INFO ] [MAIN ] KVYO: Generated 1794 derivative records from 50869 primary records. [2025-10-20T22:11:11.909258] [DEBUG] [LOCAL ] KW: Derivative data saved (1794 records). [2025-10-20T22:11:11.909356] [DEBUG] [LOCAL ] KVYO: Saving derivative data. [2025-10-20T22:11:11.909628] [INFO ] [MAIN ] KW: Successfully saved derivative data. [2025-10-20T22:11:11.920287] [INFO ] [MAIN ] Derivative Progress: 1040/1877 symbols completed. KW took 1.2s. Estimated remaining: 5m 15s. Rate: 159.3 symbols/min [2025-10-20T22:11:11.934561] [INFO ] [MAIN ] KYN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:12.012110] [DEBUG] [LOCAL ] KVYO: Derivative data saved (1794 records). [2025-10-20T22:11:12.026286] [INFO ] [MAIN ] KVYO: Successfully saved derivative data. [2025-10-20T22:11:12.038720] [INFO ] [MAIN ] Derivative Progress: 1041/1877 symbols completed. KVYO took 1.4s. Estimated remaining: 5m 15s. Rate: 159.4 symbols/min [2025-10-20T22:11:12.068157] [INFO ] [MAIN ] L: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:12.208839] [DEBUG] [LOCAL ] KYN: Loaded 30326 existing records from primary storage. [2025-10-20T22:11:12.209003] [INFO ] [MAIN ] KYN: Loaded 30326 primary data records. [2025-10-20T22:11:12.411106] [DEBUG] [LOCAL ] L: Loaded 41243 existing records from primary storage. [2025-10-20T22:11:12.518964] [INFO ] [MAIN ] L: Loaded 41243 primary data records. [2025-10-20T22:11:12.633531] [DEBUG] [MAIN ] KWR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:12.651473] [INFO ] [MAIN ] KWR: Generated 1794 derivative records from 15952 primary records. [2025-10-20T22:11:12.651783] [DEBUG] [LOCAL ] KWR: Saving derivative data. [2025-10-20T22:11:12.852104] [DEBUG] [LOCAL ] KWR: Derivative data saved (1794 records). [2025-10-20T22:11:12.865950] [INFO ] [MAIN ] KWR: Successfully saved derivative data. [2025-10-20T22:11:12.869371] [INFO ] [MAIN ] Derivative Progress: 1042/1877 symbols completed. KWR took 1.1s. Estimated remaining: 5m 15s. Rate: 159.2 symbols/min [2025-10-20T22:11:12.869759] [INFO ] [MAIN ] LAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:12.890364] [DEBUG] [MAIN ] KYN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:12.892197] [INFO ] [MAIN ] KYN: Generated 1794 derivative records from 30326 primary records. [2025-10-20T22:11:12.892330] [DEBUG] [LOCAL ] KYN: Saving derivative data. [2025-10-20T22:11:12.908496] [DEBUG] [LOCAL ] KYN: Derivative data saved (1794 records). [2025-10-20T22:11:12.908791] [INFO ] [MAIN ] KYN: Successfully saved derivative data. [2025-10-20T22:11:12.911031] [INFO ] [MAIN ] Derivative Progress: 1043/1877 symbols completed. KYN took 1.0s. Estimated remaining: 5m 14s. Rate: 159.4 symbols/min [2025-10-20T22:11:12.911242] [INFO ] [MAIN ] LAD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:12.942823] [DEBUG] [MAIN ] L: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:12.943513] [INFO ] [MAIN ] L: Generated 1794 derivative records from 41243 primary records. [2025-10-20T22:11:12.943637] [DEBUG] [LOCAL ] L: Saving derivative data. [2025-10-20T22:11:12.964908] [DEBUG] [LOCAL ] L: Derivative data saved (1794 records). [2025-10-20T22:11:12.965183] [INFO ] [MAIN ] L: Successfully saved derivative data. [2025-10-20T22:11:12.967181] [INFO ] [MAIN ] Derivative Progress: 1044/1877 symbols completed. L took 0.9s. Estimated remaining: 5m 13s. Rate: 159.5 symbols/min [2025-10-20T22:11:12.974498] [INFO ] [MAIN ] LADR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:12.992414] [DEBUG] [LOCAL ] LAD: Loaded 23301 existing records from primary storage. [2025-10-20T22:11:12.992698] [INFO ] [MAIN ] LAD: Loaded 23301 primary data records. [2025-10-20T22:11:12.993672] [DEBUG] [LOCAL ] LAC: Loaded 52554 existing records from primary storage. [2025-10-20T22:11:13.014832] [INFO ] [MAIN ] LAC: Loaded 52554 primary data records. [2025-10-20T22:11:13.574306] [DEBUG] [LOCAL ] LADR: Loaded 39435 existing records from primary storage. [2025-10-20T22:11:13.662292] [INFO ] [MAIN ] LADR: Loaded 39435 primary data records. [2025-10-20T22:11:13.986331] [DEBUG] [MAIN ] LAD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:13.989679] [INFO ] [MAIN ] LAD: Generated 1794 derivative records from 23301 primary records. [2025-10-20T22:11:13.991906] [DEBUG] [LOCAL ] LAD: Saving derivative data. [2025-10-20T22:11:14.023267] [DEBUG] [LOCAL ] LAD: Derivative data saved (1794 records). [2025-10-20T22:11:14.023413] [DEBUG] [MAIN ] LAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:14.023588] [INFO ] [MAIN ] LAD: Successfully saved derivative data. [2025-10-20T22:11:14.026047] [INFO ] [MAIN ] Derivative Progress: 1045/1877 symbols completed. LAD took 1.1s. Estimated remaining: 5m 14s. Rate: 159.2 symbols/min [2025-10-20T22:11:14.027272] [INFO ] [MAIN ] LAC: Generated 1794 derivative records from 52554 primary records. [2025-10-20T22:11:14.027503] [INFO ] [MAIN ] LANV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:14.027831] [DEBUG] [LOCAL ] LAC: Saving derivative data. [2025-10-20T22:11:14.064801] [DEBUG] [LOCAL ] LANV: Loaded 2795 existing records from primary storage. [2025-10-20T22:11:14.062315] [DEBUG] [LOCAL ] LAC: Derivative data saved (1794 records). [2025-10-20T22:11:14.065510] [INFO ] [MAIN ] LAC: Successfully saved derivative data. [2025-10-20T22:11:14.064987] [INFO ] [MAIN ] LANV: Loaded 2795 primary data records. [2025-10-20T22:11:14.065329] [DEBUG] [MAIN ] LADR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:14.067726] [INFO ] [MAIN ] Derivative Progress: 1046/1877 symbols completed. LAC took 1.2s. Estimated remaining: 5m 13s. Rate: 159.3 symbols/min [2025-10-20T22:11:14.089953] [INFO ] [MAIN ] LADR: Generated 1794 derivative records from 39435 primary records. [2025-10-20T22:11:14.106143] [DEBUG] [LOCAL ] LADR: Saving derivative data. [2025-10-20T22:11:14.149393] [INFO ] [MAIN ] LAW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:14.196392] [DEBUG] [LOCAL ] LADR: Derivative data saved (1794 records). [2025-10-20T22:11:14.216972] [INFO ] [MAIN ] LADR: Successfully saved derivative data. [2025-10-20T22:11:14.242042] [INFO ] [MAIN ] Derivative Progress: 1047/1877 symbols completed. LADR took 1.3s. Estimated remaining: 5m 12s. Rate: 159.4 symbols/min [2025-10-20T22:11:14.247765] [INFO ] [MAIN ] LAZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:14.328143] [DEBUG] [LOCAL ] LAW: Loaded 14803 existing records from primary storage. [2025-10-20T22:11:14.389372] [INFO ] [MAIN ] LAW: Loaded 14803 primary data records. [2025-10-20T22:11:14.731500] [DEBUG] [LOCAL ] LAZ: Loaded 44543 existing records from primary storage. [2025-10-20T22:11:14.731963] [INFO ] [MAIN ] LAZ: Loaded 44543 primary data records. [2025-10-20T22:11:14.851770] [DEBUG] [MAIN ] LANV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:14.868488] [INFO ] [MAIN ] LANV: Generated 1793 derivative records from 2795 primary records. [2025-10-20T22:11:14.868718] [DEBUG] [LOCAL ] LANV: Saving derivative data. [2025-10-20T22:11:15.038686] [DEBUG] [LOCAL ] LANV: Derivative data saved (1793 records). [2025-10-20T22:11:15.063624] [INFO ] [MAIN ] LANV: Successfully saved derivative data. [2025-10-20T22:11:15.067266] [INFO ] [MAIN ] Derivative Progress: 1048/1877 symbols completed. LANV took 1.0s. Estimated remaining: 5m 12s. Rate: 159.2 symbols/min [2025-10-20T22:11:15.082721] [DEBUG] [MAIN ] LAW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:15.084026] [INFO ] [MAIN ] LAW: Generated 1794 derivative records from 14803 primary records. [2025-10-20T22:11:15.084255] [DEBUG] [LOCAL ] LAW: Saving derivative data. [2025-10-20T22:11:15.101079] [DEBUG] [LOCAL ] LAW: Derivative data saved (1794 records). [2025-10-20T22:11:15.101257] [INFO ] [MAIN ] LAW: Successfully saved derivative data. [2025-10-20T22:11:15.103480] [INFO ] [MAIN ] Derivative Progress: 1049/1877 symbols completed. LAW took 1.0s. Estimated remaining: 5m 12s. Rate: 159.4 symbols/min [2025-10-20T22:11:15.120972] [DEBUG] [MAIN ] LAZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:15.121536] [INFO ] [MAIN ] LAZ: Generated 1794 derivative records from 44543 primary records. [2025-10-20T22:11:15.121606] [DEBUG] [LOCAL ] LAZ: Saving derivative data. [2025-10-20T22:11:15.136300] [DEBUG] [LOCAL ] LAZ: Derivative data saved (1794 records). [2025-10-20T22:11:15.136359] [INFO ] [MAIN ] LAZ: Successfully saved derivative data. [2025-10-20T22:11:15.138390] [INFO ] [MAIN ] Derivative Progress: 1050/1877 symbols completed. LAZ took 0.9s. Estimated remaining: 5m 11s. Rate: 159.5 symbols/min [2025-10-20T22:11:15.138814] [INFO ] [MAIN ] LB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:15.138935] [INFO ] [MAIN ] LC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:15.139488] [INFO ] [MAIN ] LBRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:15.197362] [DEBUG] [LOCAL ] LB: Loaded 37212 existing records from primary storage. [2025-10-20T22:11:15.197625] [INFO ] [MAIN ] LB: Loaded 37212 primary data records. [2025-10-20T22:11:15.326972] [DEBUG] [LOCAL ] LBRT: Loaded 53003 existing records from primary storage. [2025-10-20T22:11:15.329411] [INFO ] [MAIN ] LBRT: Loaded 53003 primary data records. [2025-10-20T22:11:15.372823] [DEBUG] [LOCAL ] LC: Loaded 48258 existing records from primary storage. [2025-10-20T22:11:15.448229] [INFO ] [MAIN ] LC: Loaded 48258 primary data records. [2025-10-20T22:11:16.224082] [DEBUG] [MAIN ] LB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:16.226420] [INFO ] [MAIN ] LB: Generated 1794 derivative records from 37212 primary records. [2025-10-20T22:11:16.226973] [DEBUG] [LOCAL ] LB: Saving derivative data. [2025-10-20T22:11:16.260415] [DEBUG] [LOCAL ] LB: Derivative data saved (1794 records). [2025-10-20T22:11:16.260577] [INFO ] [MAIN ] LB: Successfully saved derivative data. [2025-10-20T22:11:16.261473] [DEBUG] [MAIN ] LBRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:16.262346] [INFO ] [MAIN ] LBRT: Generated 1794 derivative records from 53003 primary records. [2025-10-20T22:11:16.264166] [INFO ] [MAIN ] Derivative Progress: 1051/1877 symbols completed. LB took 1.1s. Estimated remaining: 5m 11s. Rate: 159.2 symbols/min [2025-10-20T22:11:16.264500] [DEBUG] [LOCAL ] LBRT: Saving derivative data. [2025-10-20T22:11:16.264380] [INFO ] [MAIN ] LCII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:16.284583] [DEBUG] [LOCAL ] LBRT: Derivative data saved (1794 records). [2025-10-20T22:11:16.284721] [INFO ] [MAIN ] LBRT: Successfully saved derivative data. [2025-10-20T22:11:16.295058] [INFO ] [MAIN ] Derivative Progress: 1052/1877 symbols completed. LBRT took 1.2s. Estimated remaining: 5m 11s. Rate: 159.4 symbols/min [2025-10-20T22:11:16.295247] [INFO ] [MAIN ] LDI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:16.301317] [DEBUG] [MAIN ] LC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:16.302145] [INFO ] [MAIN ] LC: Generated 1794 derivative records from 48258 primary records. [2025-10-20T22:11:16.302767] [DEBUG] [LOCAL ] LC: Saving derivative data. [2025-10-20T22:11:16.328861] [DEBUG] [LOCAL ] LC: Derivative data saved (1794 records). [2025-10-20T22:11:16.329078] [INFO ] [MAIN ] LC: Successfully saved derivative data. [2025-10-20T22:11:16.331171] [INFO ] [MAIN ] Derivative Progress: 1053/1877 symbols completed. LC took 1.2s. Estimated remaining: 5m 10s. Rate: 159.5 symbols/min [2025-10-20T22:11:16.331460] [INFO ] [MAIN ] LDOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:16.346714] [DEBUG] [LOCAL ] LCII: Loaded 27217 existing records from primary storage. [2025-10-20T22:11:16.346822] [INFO ] [MAIN ] LCII: Loaded 27217 primary data records. [2025-10-20T22:11:16.542046] [DEBUG] [LOCAL ] LDI: Loaded 44983 existing records from primary storage. [2025-10-20T22:11:16.562281] [INFO ] [MAIN ] LDI: Loaded 44983 primary data records. [2025-10-20T22:11:16.772128] [DEBUG] [LOCAL ] LDOS: Loaded 44754 existing records from primary storage. [2025-10-20T22:11:16.795153] [INFO ] [MAIN ] LDOS: Loaded 44754 primary data records. [2025-10-20T22:11:17.322511] [DEBUG] [MAIN ] LCII: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:17.323411] [INFO ] [MAIN ] LCII: Generated 1793 derivative records from 27217 primary records. [2025-10-20T22:11:17.323877] [DEBUG] [LOCAL ] LCII: Saving derivative data. [2025-10-20T22:11:17.355793] [DEBUG] [MAIN ] LDI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:17.356500] [INFO ] [MAIN ] LDI: Generated 1794 derivative records from 44983 primary records. [2025-10-20T22:11:17.355499] [DEBUG] [LOCAL ] LCII: Derivative data saved (1793 records). [2025-10-20T22:11:17.356719] [INFO ] [MAIN ] LCII: Successfully saved derivative data. [2025-10-20T22:11:17.358909] [INFO ] [MAIN ] Derivative Progress: 1054/1877 symbols completed. LCII took 1.1s. Estimated remaining: 5m 10s. Rate: 159.2 symbols/min [2025-10-20T22:11:17.359031] [DEBUG] [LOCAL ] LDI: Saving derivative data. [2025-10-20T22:11:17.359123] [INFO ] [MAIN ] LDP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:17.377834] [DEBUG] [LOCAL ] LDI: Derivative data saved (1794 records). [2025-10-20T22:11:17.393805] [INFO ] [MAIN ] LDI: Successfully saved derivative data. [2025-10-20T22:11:17.394145] [DEBUG] [MAIN ] LDOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:17.397186] [INFO ] [MAIN ] LDOS: Generated 1794 derivative records from 44754 primary records. [2025-10-20T22:11:17.396423] [INFO ] [MAIN ] Derivative Progress: 1055/1877 symbols completed. LDI took 1.1s. Estimated remaining: 5m 9s. Rate: 159.4 symbols/min [2025-10-20T22:11:17.397536] [INFO ] [MAIN ] LEA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:17.397425] [DEBUG] [LOCAL ] LDOS: Saving derivative data. [2025-10-20T22:11:17.412031] [DEBUG] [LOCAL ] LDP: Loaded 12697 existing records from primary storage. [2025-10-20T22:11:17.412142] [INFO ] [MAIN ] LDP: Loaded 12697 primary data records. [2025-10-20T22:11:17.425993] [DEBUG] [LOCAL ] LDOS: Derivative data saved (1794 records). [2025-10-20T22:11:17.433111] [INFO ] [MAIN ] LDOS: Successfully saved derivative data. [2025-10-20T22:11:17.444912] [INFO ] [MAIN ] Derivative Progress: 1056/1877 symbols completed. LDOS took 1.1s. Estimated remaining: 5m 9s. Rate: 159.5 symbols/min [2025-10-20T22:11:17.455883] [INFO ] [MAIN ] LEG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:17.777601] [DEBUG] [LOCAL ] LEA: Loaded 41313 existing records from primary storage. [2025-10-20T22:11:17.777858] [INFO ] [MAIN ] LEA: Loaded 41313 primary data records. [2025-10-20T22:11:17.898716] [DEBUG] [LOCAL ] LEG: Loaded 50094 existing records from primary storage. [2025-10-20T22:11:17.904517] [INFO ] [MAIN ] LEG: Loaded 50094 primary data records. [2025-10-20T22:11:18.402773] [DEBUG] [MAIN ] LDP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:18.403623] [INFO ] [MAIN ] LDP: Generated 1794 derivative records from 12697 primary records. [2025-10-20T22:11:18.404207] [DEBUG] [LOCAL ] LDP: Saving derivative data. [2025-10-20T22:11:18.422594] [DEBUG] [LOCAL ] LDP: Derivative data saved (1794 records). [2025-10-20T22:11:18.423243] [INFO ] [MAIN ] LDP: Successfully saved derivative data. [2025-10-20T22:11:18.425445] [INFO ] [MAIN ] Derivative Progress: 1057/1877 symbols completed. LDP took 1.1s. Estimated remaining: 5m 9s. Rate: 159.3 symbols/min [2025-10-20T22:11:18.425806] [INFO ] [MAIN ] LEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:18.446933] [DEBUG] [MAIN ] LEA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:18.447709] [INFO ] [MAIN ] LEA: Generated 1794 derivative records from 41313 primary records. [2025-10-20T22:11:18.447860] [DEBUG] [LOCAL ] LEA: Saving derivative data. [2025-10-20T22:11:18.469589] [DEBUG] [LOCAL ] LEA: Derivative data saved (1794 records). [2025-10-20T22:11:18.477478] [INFO ] [MAIN ] LEA: Successfully saved derivative data. [2025-10-20T22:11:18.484201] [INFO ] [MAIN ] Derivative Progress: 1058/1877 symbols completed. LEA took 1.1s. Estimated remaining: 5m 8s. Rate: 159.4 symbols/min [2025-10-20T22:11:18.484611] [DEBUG] [MAIN ] LEG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:18.484722] [INFO ] [MAIN ] LEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:18.485903] [INFO ] [MAIN ] LEG: Generated 1794 derivative records from 50094 primary records. [2025-10-20T22:11:18.486113] [DEBUG] [LOCAL ] LEG: Saving derivative data. [2025-10-20T22:11:18.502124] [DEBUG] [LOCAL ] LEG: Derivative data saved (1794 records). [2025-10-20T22:11:18.516825] [INFO ] [MAIN ] LEG: Successfully saved derivative data. [2025-10-20T22:11:18.519216] [INFO ] [MAIN ] Derivative Progress: 1059/1877 symbols completed. LEG took 1.1s. Estimated remaining: 5m 8s. Rate: 159.5 symbols/min [2025-10-20T22:11:18.522886] [INFO ] [MAIN ] LEVI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:18.530778] [DEBUG] [LOCAL ] LEO: Loaded 16617 existing records from primary storage. [2025-10-20T22:11:18.530918] [INFO ] [MAIN ] LEO: Loaded 16617 primary data records. [2025-10-20T22:11:18.651615] [DEBUG] [LOCAL ] LEN: Loaded 53202 existing records from primary storage. [2025-10-20T22:11:18.652250] [INFO ] [MAIN ] LEN: Loaded 53202 primary data records. [2025-10-20T22:11:19.173359] [DEBUG] [LOCAL ] LEVI: Loaded 51821 existing records from primary storage. [2025-10-20T22:11:19.175332] [INFO ] [MAIN ] LEVI: Loaded 51821 primary data records. [2025-10-20T22:11:19.535899] [DEBUG] [MAIN ] LEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:19.537579] [INFO ] [MAIN ] LEO: Generated 1794 derivative records from 16617 primary records. [2025-10-20T22:11:19.538367] [DEBUG] [LOCAL ] LEO: Saving derivative data. [2025-10-20T22:11:19.555860] [DEBUG] [LOCAL ] LEO: Derivative data saved (1794 records). [2025-10-20T22:11:19.556927] [INFO ] [MAIN ] LEO: Successfully saved derivative data. [2025-10-20T22:11:19.559594] [INFO ] [MAIN ] Derivative Progress: 1060/1877 symbols completed. LEO took 1.1s. Estimated remaining: 5m 8s. Rate: 159.3 symbols/min [2025-10-20T22:11:19.559920] [INFO ] [MAIN ] LFT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:19.575267] [DEBUG] [MAIN ] LEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:19.575972] [INFO ] [MAIN ] LEN: Generated 1794 derivative records from 53202 primary records. [2025-10-20T22:11:19.576220] [DEBUG] [LOCAL ] LEN: Saving derivative data. [2025-10-20T22:11:19.597284] [DEBUG] [LOCAL ] LEN: Derivative data saved (1794 records). [2025-10-20T22:11:19.597755] [INFO ] [MAIN ] LEN: Successfully saved derivative data. [2025-10-20T22:11:19.599998] [INFO ] [MAIN ] Derivative Progress: 1061/1877 symbols completed. LEN took 1.2s. Estimated remaining: 5m 7s. Rate: 159.4 symbols/min [2025-10-20T22:11:19.600795] [INFO ] [MAIN ] LGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:19.608417] [DEBUG] [LOCAL ] LFT: Loaded 18230 existing records from primary storage. [2025-10-20T22:11:19.608527] [INFO ] [MAIN ] LFT: Loaded 18230 primary data records. [2025-10-20T22:11:19.730671] [DEBUG] [MAIN ] LEVI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:19.784137] [INFO ] [MAIN ] LEVI: Generated 1794 derivative records from 51821 primary records. [2025-10-20T22:11:19.784448] [DEBUG] [LOCAL ] LGI: Loaded 10042 existing records from primary storage. [2025-10-20T22:11:19.909020] [DEBUG] [LOCAL ] LEVI: Saving derivative data. [2025-10-20T22:11:19.922298] [INFO ] [MAIN ] LGI: Loaded 10042 primary data records. [2025-10-20T22:11:19.994385] [DEBUG] [LOCAL ] LEVI: Derivative data saved (1794 records). [2025-10-20T22:11:20.034577] [INFO ] [MAIN ] LEVI: Successfully saved derivative data. [2025-10-20T22:11:20.054502] [INFO ] [MAIN ] Derivative Progress: 1062/1877 symbols completed. LEVI took 1.5s. Estimated remaining: 5m 7s. Rate: 159.4 symbols/min [2025-10-20T22:11:20.071201] [INFO ] [MAIN ] LH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:20.288850] [DEBUG] [MAIN ] LFT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:20.298741] [INFO ] [MAIN ] LFT: Generated 1794 derivative records from 18230 primary records. [2025-10-20T22:11:20.299438] [DEBUG] [LOCAL ] LFT: Saving derivative data. [2025-10-20T22:11:20.327297] [DEBUG] [LOCAL ] LFT: Derivative data saved (1794 records). [2025-10-20T22:11:20.328153] [DEBUG] [LOCAL ] LH: Loaded 37275 existing records from primary storage. [2025-10-20T22:11:20.328451] [INFO ] [MAIN ] LH: Loaded 37275 primary data records. [2025-10-20T22:11:20.327578] [INFO ] [MAIN ] LFT: Successfully saved derivative data. [2025-10-20T22:11:20.330609] [INFO ] [MAIN ] Derivative Progress: 1063/1877 symbols completed. LFT took 0.8s. Estimated remaining: 5m 6s. Rate: 159.4 symbols/min [2025-10-20T22:11:20.467039] [DEBUG] [MAIN ] LGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:20.494571] [INFO ] [MAIN ] LGI: Generated 1794 derivative records from 10042 primary records. [2025-10-20T22:11:20.502240] [DEBUG] [LOCAL ] LGI: Saving derivative data. [2025-10-20T22:11:20.649597] [DEBUG] [LOCAL ] LGI: Derivative data saved (1794 records). [2025-10-20T22:11:20.649858] [INFO ] [MAIN ] LGI: Successfully saved derivative data. [2025-10-20T22:11:20.651831] [INFO ] [MAIN ] Derivative Progress: 1064/1877 symbols completed. LGI took 1.1s. Estimated remaining: 5m 6s. Rate: 159.4 symbols/min [2025-10-20T22:11:20.672770] [DEBUG] [MAIN ] LH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:20.673331] [INFO ] [MAIN ] LH: Generated 1794 derivative records from 37275 primary records. [2025-10-20T22:11:20.673390] [DEBUG] [LOCAL ] LH: Saving derivative data. [2025-10-20T22:11:20.688254] [DEBUG] [LOCAL ] LH: Derivative data saved (1794 records). [2025-10-20T22:11:20.688312] [INFO ] [MAIN ] LH: Successfully saved derivative data. [2025-10-20T22:11:20.690150] [INFO ] [MAIN ] Derivative Progress: 1065/1877 symbols completed. LH took 0.6s. Estimated remaining: 5m 5s. Rate: 159.6 symbols/min [2025-10-20T22:11:20.690503] [INFO ] [MAIN ] LHX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:20.690567] [INFO ] [MAIN ] LII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:20.690666] [INFO ] [MAIN ] LION: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:20.737117] [DEBUG] [LOCAL ] LII: Loaded 27796 existing records from primary storage. [2025-10-20T22:11:20.737338] [INFO ] [MAIN ] LII: Loaded 27796 primary data records. [2025-10-20T22:11:20.858368] [DEBUG] [LOCAL ] LION: Loaded 46794 existing records from primary storage. [2025-10-20T22:11:20.870668] [INFO ] [MAIN ] LION: Loaded 46794 primary data records. [2025-10-20T22:11:21.041121] [DEBUG] [LOCAL ] LHX: Loaded 48528 existing records from primary storage. [2025-10-20T22:11:21.065694] [INFO ] [MAIN ] LHX: Loaded 48528 primary data records. [2025-10-20T22:11:21.741127] [DEBUG] [MAIN ] LII: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:21.754092] [INFO ] [MAIN ] LII: Generated 1794 derivative records from 27796 primary records. [2025-10-20T22:11:21.754596] [DEBUG] [MAIN ] LION: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:21.754353] [DEBUG] [LOCAL ] LII: Saving derivative data. [2025-10-20T22:11:21.755927] [INFO ] [MAIN ] LION: Generated 1794 derivative records from 46794 primary records. [2025-10-20T22:11:21.756179] [DEBUG] [LOCAL ] LION: Saving derivative data. [2025-10-20T22:11:21.781255] [DEBUG] [LOCAL ] LII: Derivative data saved (1794 records). [2025-10-20T22:11:21.781367] [INFO ] [MAIN ] LII: Successfully saved derivative data. [2025-10-20T22:11:21.785808] [INFO ] [MAIN ] Derivative Progress: 1066/1877 symbols completed. LII took 1.1s. Estimated remaining: 5m 6s. Rate: 159.3 symbols/min [2025-10-20T22:11:21.786250] [INFO ] [MAIN ] LITB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:21.791061] [DEBUG] [LOCAL ] LION: Derivative data saved (1794 records). [2025-10-20T22:11:21.791752] [INFO ] [MAIN ] LION: Successfully saved derivative data. [2025-10-20T22:11:21.794295] [INFO ] [MAIN ] Derivative Progress: 1067/1877 symbols completed. LION took 1.1s. Estimated remaining: 5m 5s. Rate: 159.4 symbols/min [2025-10-20T22:11:21.807176] [DEBUG] [MAIN ] LHX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:21.806910] [INFO ] [MAIN ] LLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:21.808187] [INFO ] [MAIN ] LHX: Generated 1794 derivative records from 48528 primary records. [2025-10-20T22:11:21.809407] [DEBUG] [LOCAL ] LHX: Saving derivative data. [2025-10-20T22:11:21.821118] [DEBUG] [LOCAL ] LITB: Loaded 3871 existing records from primary storage. [2025-10-20T22:11:21.826999] [INFO ] [MAIN ] LITB: Loaded 3871 primary data records. [2025-10-20T22:11:21.828092] [DEBUG] [LOCAL ] LHX: Derivative data saved (1794 records). [2025-10-20T22:11:21.864877] [INFO ] [MAIN ] LHX: Successfully saved derivative data. [2025-10-20T22:11:21.866993] [INFO ] [MAIN ] Derivative Progress: 1068/1877 symbols completed. LHX took 1.2s. Estimated remaining: 5m 4s. Rate: 159.5 symbols/min [2025-10-20T22:11:21.874287] [INFO ] [MAIN ] LMND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:22.171970] [DEBUG] [LOCAL ] LLY: Loaded 53228 existing records from primary storage. [2025-10-20T22:11:22.172577] [INFO ] [MAIN ] LLY: Loaded 53228 primary data records. [2025-10-20T22:11:22.295477] [DEBUG] [LOCAL ] LMND: Loaded 52135 existing records from primary storage. [2025-10-20T22:11:22.301056] [INFO ] [MAIN ] LMND: Loaded 52135 primary data records. [2025-10-20T22:11:22.577330] [DEBUG] [MAIN ] LITB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:22.609563] [INFO ] [MAIN ] LITB: Generated 1794 derivative records from 3871 primary records. [2025-10-20T22:11:22.613048] [DEBUG] [LOCAL ] LITB: Saving derivative data. [2025-10-20T22:11:22.665918] [DEBUG] [LOCAL ] LITB: Derivative data saved (1794 records). [2025-10-20T22:11:22.817131] [INFO ] [MAIN ] LITB: Successfully saved derivative data. [2025-10-20T22:11:22.819502] [INFO ] [MAIN ] Derivative Progress: 1069/1877 symbols completed. LITB took 1.0s. Estimated remaining: 5m 4s. Rate: 159.3 symbols/min [2025-10-20T22:11:22.819910] [INFO ] [MAIN ] LMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:22.846367] [DEBUG] [MAIN ] LLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:22.847239] [INFO ] [MAIN ] LLY: Generated 1794 derivative records from 53228 primary records. [2025-10-20T22:11:22.847457] [DEBUG] [LOCAL ] LLY: Saving derivative data. [2025-10-20T22:11:22.864054] [DEBUG] [LOCAL ] LLY: Derivative data saved (1794 records). [2025-10-20T22:11:22.875554] [INFO ] [MAIN ] LLY: Successfully saved derivative data. [2025-10-20T22:11:22.877631] [INFO ] [MAIN ] Derivative Progress: 1070/1877 symbols completed. LLY took 1.1s. Estimated remaining: 5m 4s. Rate: 159.4 symbols/min [2025-10-20T22:11:22.878114] [INFO ] [MAIN ] LNC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:22.891590] [DEBUG] [MAIN ] LMND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:22.892344] [INFO ] [MAIN ] LMND: Generated 1794 derivative records from 52135 primary records. [2025-10-20T22:11:22.892468] [DEBUG] [LOCAL ] LMND: Saving derivative data. [2025-10-20T22:11:22.923327] [DEBUG] [LOCAL ] LMND: Derivative data saved (1794 records). [2025-10-20T22:11:22.924743] [DEBUG] [LOCAL ] LMT: Loaded 49622 existing records from primary storage. [2025-10-20T22:11:22.924962] [INFO ] [MAIN ] LMT: Loaded 49622 primary data records. [2025-10-20T22:11:22.925039] [INFO ] [MAIN ] LMND: Successfully saved derivative data. [2025-10-20T22:11:22.927294] [INFO ] [MAIN ] Derivative Progress: 1071/1877 symbols completed. LMND took 1.1s. Estimated remaining: 5m 3s. Rate: 159.6 symbols/min [2025-10-20T22:11:22.958135] [INFO ] [MAIN ] LND: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:23.046298] [DEBUG] [LOCAL ] LND: Loaded 4966 existing records from primary storage. [2025-10-20T22:11:23.046528] [INFO ] [MAIN ] LND: Loaded 4966 primary data records. [2025-10-20T22:11:23.384690] [DEBUG] [LOCAL ] LNC: Loaded 51511 existing records from primary storage. [2025-10-20T22:11:23.433274] [INFO ] [MAIN ] LNC: Loaded 51511 primary data records. [2025-10-20T22:11:23.918747] [DEBUG] [MAIN ] LMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:23.921850] [INFO ] [MAIN ] LMT: Generated 1794 derivative records from 49622 primary records. [2025-10-20T22:11:23.922010] [DEBUG] [LOCAL ] LMT: Saving derivative data. [2025-10-20T22:11:23.936500] [DEBUG] [MAIN ] LND: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:23.944886] [INFO ] [MAIN ] LND: Generated 1794 derivative records from 4966 primary records. [2025-10-20T22:11:23.946320] [DEBUG] [LOCAL ] LND: Saving derivative data. [2025-10-20T22:11:23.954056] [DEBUG] [LOCAL ] LMT: Derivative data saved (1794 records). [2025-10-20T22:11:23.954345] [INFO ] [MAIN ] LMT: Successfully saved derivative data. [2025-10-20T22:11:23.956510] [INFO ] [MAIN ] Derivative Progress: 1072/1877 symbols completed. LMT took 1.1s. Estimated remaining: 5m 3s. Rate: 159.3 symbols/min [2025-10-20T22:11:23.962874] [INFO ] [MAIN ] LNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:23.971279] [DEBUG] [LOCAL ] LND: Derivative data saved (1794 records). [2025-10-20T22:11:23.971520] [INFO ] [MAIN ] LND: Successfully saved derivative data. [2025-10-20T22:11:23.973367] [INFO ] [MAIN ] Derivative Progress: 1073/1877 symbols completed. LND took 1.0s. Estimated remaining: 5m 3s. Rate: 159.4 symbols/min [2025-10-20T22:11:23.973834] [INFO ] [MAIN ] LNN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:23.990360] [DEBUG] [MAIN ] LNC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:23.991044] [INFO ] [MAIN ] LNC: Generated 1794 derivative records from 51511 primary records. [2025-10-20T22:11:23.991324] [DEBUG] [LOCAL ] LNC: Saving derivative data. [2025-10-20T22:11:24.009254] [DEBUG] [LOCAL ] LNC: Derivative data saved (1794 records). [2025-10-20T22:11:24.009726] [INFO ] [MAIN ] LNC: Successfully saved derivative data. [2025-10-20T22:11:24.012139] [INFO ] [MAIN ] Derivative Progress: 1074/1877 symbols completed. LNC took 1.1s. Estimated remaining: 5m 2s. Rate: 159.6 symbols/min [2025-10-20T22:11:24.012538] [INFO ] [MAIN ] LOAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:24.017857] [DEBUG] [LOCAL ] LNN: Loaded 11478 existing records from primary storage. [2025-10-20T22:11:24.017993] [INFO ] [MAIN ] LNN: Loaded 11478 primary data records. [2025-10-20T22:11:24.345776] [DEBUG] [LOCAL ] LNG: Loaded 51168 existing records from primary storage. [2025-10-20T22:11:24.354679] [INFO ] [MAIN ] LNG: Loaded 51168 primary data records. [2025-10-20T22:11:24.417405] [DEBUG] [LOCAL ] LOAR: Loaded 42010 existing records from primary storage. [2025-10-20T22:11:24.448147] [INFO ] [MAIN ] LOAR: Loaded 42010 primary data records. [2025-10-20T22:11:24.998779] [DEBUG] [MAIN ] LNN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:25.000776] [INFO ] [MAIN ] LNN: Generated 1794 derivative records from 11478 primary records. [2025-10-20T22:11:25.001129] [DEBUG] [LOCAL ] LNN: Saving derivative data. [2025-10-20T22:11:25.019101] [DEBUG] [LOCAL ] LNN: Derivative data saved (1794 records). [2025-10-20T22:11:25.019582] [INFO ] [MAIN ] LNN: Successfully saved derivative data. [2025-10-20T22:11:25.022899] [INFO ] [MAIN ] Derivative Progress: 1075/1877 symbols completed. LNN took 1.0s. Estimated remaining: 5m 2s. Rate: 159.3 symbols/min [2025-10-20T22:11:25.023201] [INFO ] [MAIN ] LOB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:25.046440] [DEBUG] [MAIN ] LNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:25.047516] [INFO ] [MAIN ] LNG: Generated 1794 derivative records from 51168 primary records. [2025-10-20T22:11:25.047793] [DEBUG] [LOCAL ] LNG: Saving derivative data. [2025-10-20T22:11:25.082165] [DEBUG] [LOCAL ] LNG: Derivative data saved (1794 records). [2025-10-20T22:11:25.082512] [INFO ] [MAIN ] LNG: Successfully saved derivative data. [2025-10-20T22:11:25.084974] [INFO ] [MAIN ] Derivative Progress: 1076/1877 symbols completed. LNG took 1.1s. Estimated remaining: 5m 1s. Rate: 159.5 symbols/min [2025-10-20T22:11:25.086558] [INFO ] [MAIN ] LOCL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:25.116263] [DEBUG] [MAIN ] LOAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:25.118059] [INFO ] [MAIN ] LOAR: Generated 1794 derivative records from 42010 primary records. [2025-10-20T22:11:25.118220] [DEBUG] [LOCAL ] LOAR: Saving derivative data. [2025-10-20T22:11:25.117104] [DEBUG] [LOCAL ] LOB: Loaded 26844 existing records from primary storage. [2025-10-20T22:11:25.118872] [INFO ] [MAIN ] LOB: Loaded 26844 primary data records. [2025-10-20T22:11:25.145354] [DEBUG] [LOCAL ] LOCL: Loaded 3814 existing records from primary storage. [2025-10-20T22:11:25.192963] [INFO ] [MAIN ] LOCL: Loaded 3814 primary data records. [2025-10-20T22:11:25.334522] [DEBUG] [LOCAL ] LOAR: Derivative data saved (1794 records). [2025-10-20T22:11:25.345053] [INFO ] [MAIN ] LOAR: Successfully saved derivative data. [2025-10-20T22:11:25.358682] [INFO ] [MAIN ] Derivative Progress: 1077/1877 symbols completed. LOAR took 1.3s. Estimated remaining: 5m 1s. Rate: 159.5 symbols/min [2025-10-20T22:11:25.405805] [INFO ] [MAIN ] LOMA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:25.962948] [DEBUG] [LOCAL ] LOMA: Loaded 32769 existing records from primary storage. [2025-10-20T22:11:25.965063] [INFO ] [MAIN ] LOMA: Loaded 32769 primary data records. [2025-10-20T22:11:26.339577] [DEBUG] [MAIN ] LOB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:26.341369] [INFO ] [MAIN ] LOB: Generated 1794 derivative records from 26844 primary records. [2025-10-20T22:11:26.342002] [DEBUG] [LOCAL ] LOB: Saving derivative data. [2025-10-20T22:11:26.381556] [DEBUG] [LOCAL ] LOB: Derivative data saved (1794 records). [2025-10-20T22:11:26.381883] [DEBUG] [MAIN ] LOCL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:26.382887] [INFO ] [MAIN ] LOCL: Generated 1794 derivative records from 3814 primary records. [2025-10-20T22:11:26.383141] [DEBUG] [LOCAL ] LOCL: Saving derivative data. [2025-10-20T22:11:26.382958] [INFO ] [MAIN ] LOB: Successfully saved derivative data. [2025-10-20T22:11:26.385629] [INFO ] [MAIN ] Derivative Progress: 1078/1877 symbols completed. LOB took 1.4s. Estimated remaining: 5m 1s. Rate: 159.2 symbols/min [2025-10-20T22:11:26.402434] [DEBUG] [LOCAL ] LOCL: Derivative data saved (1794 records). [2025-10-20T22:11:26.402731] [INFO ] [MAIN ] LOCL: Successfully saved derivative data. [2025-10-20T22:11:26.404690] [INFO ] [MAIN ] Derivative Progress: 1079/1877 symbols completed. LOCL took 1.3s. Estimated remaining: 5m 0s. Rate: 159.4 symbols/min [2025-10-20T22:11:26.422777] [DEBUG] [MAIN ] LOMA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:26.423475] [INFO ] [MAIN ] LOMA: Generated 1794 derivative records from 32769 primary records. [2025-10-20T22:11:26.423541] [DEBUG] [LOCAL ] LOMA: Saving derivative data. [2025-10-20T22:11:26.439393] [DEBUG] [LOCAL ] LOMA: Derivative data saved (1794 records). [2025-10-20T22:11:26.439477] [INFO ] [MAIN ] LOMA: Successfully saved derivative data. [2025-10-20T22:11:26.442133] [INFO ] [MAIN ] Derivative Progress: 1080/1877 symbols completed. LOMA took 1.0s. Estimated remaining: 4m 60s. Rate: 159.5 symbols/min [2025-10-20T22:11:26.442623] [INFO ] [MAIN ] LPG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:26.442531] [INFO ] [MAIN ] LOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:26.442980] [INFO ] [MAIN ] LPL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:26.496436] [DEBUG] [LOCAL ] LPL: Loaded 28753 existing records from primary storage. [2025-10-20T22:11:26.509766] [INFO ] [MAIN ] LPL: Loaded 28753 primary data records. [2025-10-20T22:11:26.515696] [DEBUG] [LOCAL ] LPG: Loaded 39541 existing records from primary storage. [2025-10-20T22:11:26.549883] [INFO ] [MAIN ] LPG: Loaded 39541 primary data records. [2025-10-20T22:11:26.734783] [DEBUG] [LOCAL ] LOW: Loaded 52993 existing records from primary storage. [2025-10-20T22:11:26.833365] [INFO ] [MAIN ] LOW: Loaded 52993 primary data records. [2025-10-20T22:11:27.539603] [DEBUG] [MAIN ] LPL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:27.540695] [INFO ] [MAIN ] LPL: Generated 1794 derivative records from 28753 primary records. [2025-10-20T22:11:27.540941] [DEBUG] [LOCAL ] LPL: Saving derivative data. [2025-10-20T22:11:27.561899] [DEBUG] [LOCAL ] LPL: Derivative data saved (1794 records). [2025-10-20T22:11:27.569883] [INFO ] [MAIN ] LPL: Successfully saved derivative data. [2025-10-20T22:11:27.576446] [INFO ] [MAIN ] Derivative Progress: 1081/1877 symbols completed. LPL took 1.1s. Estimated remaining: 4m 60s. Rate: 159.2 symbols/min [2025-10-20T22:11:27.577847] [DEBUG] [MAIN ] LPG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:27.577474] [INFO ] [MAIN ] LPX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:27.579088] [INFO ] [MAIN ] LPG: Generated 1794 derivative records from 39541 primary records. [2025-10-20T22:11:27.579237] [DEBUG] [LOCAL ] LPG: Saving derivative data. [2025-10-20T22:11:27.611963] [DEBUG] [LOCAL ] LPG: Derivative data saved (1794 records). [2025-10-20T22:11:27.610141] [DEBUG] [MAIN ] LOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:27.612862] [INFO ] [MAIN ] LOW: Generated 1794 derivative records from 52993 primary records. [2025-10-20T22:11:27.612930] [INFO ] [MAIN ] LPG: Successfully saved derivative data. [2025-10-20T22:11:27.613038] [DEBUG] [LOCAL ] LOW: Saving derivative data. [2025-10-20T22:11:27.615233] [INFO ] [MAIN ] Derivative Progress: 1082/1877 symbols completed. LPG took 1.2s. Estimated remaining: 4m 59s. Rate: 159.3 symbols/min [2025-10-20T22:11:27.615438] [INFO ] [MAIN ] LRN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:27.633268] [DEBUG] [LOCAL ] LOW: Derivative data saved (1794 records). [2025-10-20T22:11:27.633569] [INFO ] [MAIN ] LOW: Successfully saved derivative data. [2025-10-20T22:11:27.635722] [INFO ] [MAIN ] Derivative Progress: 1083/1877 symbols completed. LOW took 1.2s. Estimated remaining: 4m 59s. Rate: 159.5 symbols/min [2025-10-20T22:11:27.635934] [INFO ] [MAIN ] LSPD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:27.668054] [DEBUG] [LOCAL ] LPX: Loaded 40152 existing records from primary storage. [2025-10-20T22:11:27.686216] [INFO ] [MAIN ] LPX: Loaded 40152 primary data records. [2025-10-20T22:11:27.823097] [DEBUG] [LOCAL ] LRN: Loaded 39505 existing records from primary storage. [2025-10-20T22:11:27.839359] [DEBUG] [LOCAL ] LSPD: Loaded 42440 existing records from primary storage. [2025-10-20T22:11:27.844739] [INFO ] [MAIN ] LSPD: Loaded 42440 primary data records. [2025-10-20T22:11:27.839459] [INFO ] [MAIN ] LRN: Loaded 39505 primary data records. [2025-10-20T22:11:28.657049] [DEBUG] [MAIN ] LPX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:28.658409] [INFO ] [MAIN ] LPX: Generated 1794 derivative records from 40152 primary records. [2025-10-20T22:11:28.658866] [DEBUG] [LOCAL ] LPX: Saving derivative data. [2025-10-20T22:11:28.676635] [DEBUG] [LOCAL ] LPX: Derivative data saved (1794 records). [2025-10-20T22:11:28.677855] [INFO ] [MAIN ] LPX: Successfully saved derivative data. [2025-10-20T22:11:28.683023] [INFO ] [MAIN ] Derivative Progress: 1084/1877 symbols completed. LPX took 1.1s. Estimated remaining: 4m 59s. Rate: 159.2 symbols/min [2025-10-20T22:11:28.689792] [INFO ] [MAIN ] LTH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:28.706705] [DEBUG] [MAIN ] LRN: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:28.712193] [INFO ] [MAIN ] LRN: Generated 1793 derivative records from 39505 primary records. [2025-10-20T22:11:28.712710] [DEBUG] [MAIN ] LSPD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:28.713360] [INFO ] [MAIN ] LSPD: Generated 1794 derivative records from 42440 primary records. [2025-10-20T22:11:28.712765] [DEBUG] [LOCAL ] LRN: Saving derivative data. [2025-10-20T22:11:28.713486] [DEBUG] [LOCAL ] LSPD: Saving derivative data. [2025-10-20T22:11:28.741027] [DEBUG] [LOCAL ] LSPD: Derivative data saved (1794 records). [2025-10-20T22:11:28.744218] [INFO ] [MAIN ] LSPD: Successfully saved derivative data. [2025-10-20T22:11:28.745298] [DEBUG] [LOCAL ] LRN: Derivative data saved (1793 records). [2025-10-20T22:11:28.748110] [INFO ] [MAIN ] Derivative Progress: 1085/1877 symbols completed. LSPD took 1.1s. Estimated remaining: 4m 58s. Rate: 159.3 symbols/min [2025-10-20T22:11:28.748259] [INFO ] [MAIN ] LTM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:28.745438] [INFO ] [MAIN ] LRN: Successfully saved derivative data. [2025-10-20T22:11:28.750573] [INFO ] [MAIN ] Derivative Progress: 1086/1877 symbols completed. LRN took 1.1s. Estimated remaining: 4m 58s. Rate: 159.5 symbols/min [2025-10-20T22:11:28.751094] [INFO ] [MAIN ] LU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:28.801694] [DEBUG] [LOCAL ] LTH: Loaded 51686 existing records from primary storage. [2025-10-20T22:11:28.802018] [INFO ] [MAIN ] LTH: Loaded 51686 primary data records. [2025-10-20T22:11:28.933586] [DEBUG] [LOCAL ] LTM: Loaded 39040 existing records from primary storage. [2025-10-20T22:11:28.986542] [INFO ] [MAIN ] LTM: Loaded 39040 primary data records. [2025-10-20T22:11:29.129094] [DEBUG] [LOCAL ] LU: Loaded 49163 existing records from primary storage. [2025-10-20T22:11:29.168715] [INFO ] [MAIN ] LU: Loaded 49163 primary data records. [2025-10-20T22:11:29.828085] [DEBUG] [MAIN ] LTH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:29.829369] [INFO ] [MAIN ] LTH: Generated 1794 derivative records from 51686 primary records. [2025-10-20T22:11:29.829637] [DEBUG] [LOCAL ] LTH: Saving derivative data. [2025-10-20T22:11:29.846897] [DEBUG] [LOCAL ] LTH: Derivative data saved (1794 records). [2025-10-20T22:11:29.847010] [INFO ] [MAIN ] LTH: Successfully saved derivative data. [2025-10-20T22:11:29.849159] [INFO ] [MAIN ] Derivative Progress: 1087/1877 symbols completed. LTH took 1.2s. Estimated remaining: 4m 58s. Rate: 159.2 symbols/min [2025-10-20T22:11:29.849404] [INFO ] [MAIN ] LUCK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:29.865897] [DEBUG] [MAIN ] LTM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:29.866760] [INFO ] [MAIN ] LTM: Generated 1794 derivative records from 39040 primary records. [2025-10-20T22:11:29.867076] [DEBUG] [LOCAL ] LTM: Saving derivative data. [2025-10-20T22:11:29.883534] [DEBUG] [LOCAL ] LTM: Derivative data saved (1794 records). [2025-10-20T22:11:29.889008] [INFO ] [MAIN ] LTM: Successfully saved derivative data. [2025-10-20T22:11:29.893026] [INFO ] [MAIN ] Derivative Progress: 1088/1877 symbols completed. LTM took 1.1s. Estimated remaining: 4m 57s. Rate: 159.3 symbols/min [2025-10-20T22:11:29.893383] [INFO ] [MAIN ] LUMN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:29.918274] [DEBUG] [LOCAL ] LUCK: Loaded 24157 existing records from primary storage. [2025-10-20T22:11:29.918588] [DEBUG] [MAIN ] LU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:29.919285] [INFO ] [MAIN ] LU: Generated 1794 derivative records from 49163 primary records. [2025-10-20T22:11:29.919386] [INFO ] [MAIN ] LUCK: Loaded 24157 primary data records. [2025-10-20T22:11:29.919511] [DEBUG] [LOCAL ] LU: Saving derivative data. [2025-10-20T22:11:30.050110] [DEBUG] [LOCAL ] LU: Derivative data saved (1794 records). [2025-10-20T22:11:30.053668] [INFO ] [MAIN ] LU: Successfully saved derivative data. [2025-10-20T22:11:30.061433] [INFO ] [MAIN ] Derivative Progress: 1089/1877 symbols completed. LU took 1.3s. Estimated remaining: 4m 57s. Rate: 159.4 symbols/min [2025-10-20T22:11:30.080054] [INFO ] [MAIN ] LUV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:30.280178] [DEBUG] [LOCAL ] LUMN: Loaded 53645 existing records from primary storage. [2025-10-20T22:11:30.280916] [INFO ] [MAIN ] LUMN: Loaded 53645 primary data records. [2025-10-20T22:11:30.609542] [DEBUG] [LOCAL ] LUV: Loaded 53689 existing records from primary storage. [2025-10-20T22:11:30.610451] [INFO ] [MAIN ] LUV: Loaded 53689 primary data records. [2025-10-20T22:11:30.736745] [DEBUG] [MAIN ] LUCK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:30.747857] [INFO ] [MAIN ] LUCK: Generated 1794 derivative records from 24157 primary records. [2025-10-20T22:11:30.748061] [DEBUG] [LOCAL ] LUCK: Saving derivative data. [2025-10-20T22:11:30.839445] [DEBUG] [LOCAL ] LUCK: Derivative data saved (1794 records). [2025-10-20T22:11:30.923623] [INFO ] [MAIN ] LUCK: Successfully saved derivative data. [2025-10-20T22:11:30.939031] [INFO ] [MAIN ] Derivative Progress: 1090/1877 symbols completed. LUCK took 1.1s. Estimated remaining: 4m 57s. Rate: 159.2 symbols/min [2025-10-20T22:11:30.941685] [INFO ] [MAIN ] LVS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:30.957514] [DEBUG] [MAIN ] LUMN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:30.958400] [INFO ] [MAIN ] LUMN: Generated 1794 derivative records from 53645 primary records. [2025-10-20T22:11:30.958758] [DEBUG] [LOCAL ] LUMN: Saving derivative data. [2025-10-20T22:11:30.975901] [DEBUG] [LOCAL ] LUMN: Derivative data saved (1794 records). [2025-10-20T22:11:30.976161] [INFO ] [MAIN ] LUMN: Successfully saved derivative data. [2025-10-20T22:11:30.978573] [INFO ] [MAIN ] Derivative Progress: 1091/1877 symbols completed. LUMN took 1.1s. Estimated remaining: 4m 56s. Rate: 159.4 symbols/min [2025-10-20T22:11:30.978937] [INFO ] [MAIN ] LVWR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:30.996778] [DEBUG] [MAIN ] LUV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:31.001988] [INFO ] [MAIN ] LUV: Generated 1794 derivative records from 53689 primary records. [2025-10-20T22:11:31.002250] [DEBUG] [LOCAL ] LUV: Saving derivative data. [2025-10-20T22:11:31.037830] [DEBUG] [LOCAL ] LVWR: Loaded 18569 existing records from primary storage. [2025-10-20T22:11:31.037984] [INFO ] [MAIN ] LVWR: Loaded 18569 primary data records. [2025-10-20T22:11:31.037218] [DEBUG] [LOCAL ] LUV: Derivative data saved (1794 records). [2025-10-20T22:11:31.038088] [INFO ] [MAIN ] LUV: Successfully saved derivative data. [2025-10-20T22:11:31.040144] [INFO ] [MAIN ] Derivative Progress: 1092/1877 symbols completed. LUV took 1.0s. Estimated remaining: 4m 55s. Rate: 159.5 symbols/min [2025-10-20T22:11:31.072620] [INFO ] [MAIN ] LW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:31.164445] [DEBUG] [LOCAL ] LVS: Loaded 53570 existing records from primary storage. [2025-10-20T22:11:31.164724] [INFO ] [MAIN ] LVS: Loaded 53570 primary data records. [2025-10-20T22:11:31.721499] [DEBUG] [LOCAL ] LW: Loaded 51191 existing records from primary storage. [2025-10-20T22:11:31.730275] [INFO ] [MAIN ] LW: Loaded 51191 primary data records. [2025-10-20T22:11:32.089387] [DEBUG] [MAIN ] LVWR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:32.092292] [INFO ] [MAIN ] LVWR: Generated 1794 derivative records from 18569 primary records. [2025-10-20T22:11:32.093295] [DEBUG] [LOCAL ] LVWR: Saving derivative data. [2025-10-20T22:11:32.123880] [DEBUG] [LOCAL ] LVWR: Derivative data saved (1794 records). [2025-10-20T22:11:32.124498] [DEBUG] [MAIN ] LVS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:32.124679] [INFO ] [MAIN ] LVWR: Successfully saved derivative data. [2025-10-20T22:11:32.126872] [INFO ] [MAIN ] Derivative Progress: 1093/1877 symbols completed. LVWR took 1.1s. Estimated remaining: 4m 55s. Rate: 159.2 symbols/min [2025-10-20T22:11:32.127992] [INFO ] [MAIN ] LVS: Generated 1794 derivative records from 53570 primary records. [2025-10-20T22:11:32.128263] [DEBUG] [LOCAL ] LVS: Saving derivative data. [2025-10-20T22:11:32.144602] [DEBUG] [LOCAL ] LVS: Derivative data saved (1794 records). [2025-10-20T22:11:32.144996] [INFO ] [MAIN ] LVS: Successfully saved derivative data. [2025-10-20T22:11:32.147241] [INFO ] [MAIN ] Derivative Progress: 1094/1877 symbols completed. LVS took 1.2s. Estimated remaining: 4m 55s. Rate: 159.3 symbols/min [2025-10-20T22:11:32.162183] [DEBUG] [MAIN ] LW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:32.162898] [INFO ] [MAIN ] LW: Generated 1794 derivative records from 51191 primary records. [2025-10-20T22:11:32.162962] [DEBUG] [LOCAL ] LW: Saving derivative data. [2025-10-20T22:11:32.178908] [DEBUG] [LOCAL ] LW: Derivative data saved (1794 records). [2025-10-20T22:11:32.179006] [INFO ] [MAIN ] LW: Successfully saved derivative data. [2025-10-20T22:11:32.181132] [INFO ] [MAIN ] Derivative Progress: 1095/1877 symbols completed. LW took 1.1s. Estimated remaining: 4m 54s. Rate: 159.5 symbols/min [2025-10-20T22:11:32.181705] [INFO ] [MAIN ] LXFR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:32.181787] [INFO ] [MAIN ] LXU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:32.181603] [INFO ] [MAIN ] LXP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:32.226702] [DEBUG] [LOCAL ] LXFR: Loaded 16472 existing records from primary storage. [2025-10-20T22:11:32.227038] [INFO ] [MAIN ] LXFR: Loaded 16472 primary data records. [2025-10-20T22:11:32.227684] [DEBUG] [LOCAL ] LXU: Loaded 31669 existing records from primary storage. [2025-10-20T22:11:32.259517] [INFO ] [MAIN ] LXU: Loaded 31669 primary data records. [2025-10-20T22:11:32.505853] [DEBUG] [LOCAL ] LXP: Loaded 50793 existing records from primary storage. [2025-10-20T22:11:32.542418] [INFO ] [MAIN ] LXP: Loaded 50793 primary data records. [2025-10-20T22:11:33.248243] [DEBUG] [MAIN ] LXU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:33.247731] [DEBUG] [MAIN ] LXFR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:33.249318] [INFO ] [MAIN ] LXU: Generated 1794 derivative records from 31669 primary records. [2025-10-20T22:11:33.250056] [DEBUG] [LOCAL ] LXU: Saving derivative data. [2025-10-20T22:11:33.249999] [INFO ] [MAIN ] LXFR: Generated 1794 derivative records from 16472 primary records. [2025-10-20T22:11:33.250750] [DEBUG] [LOCAL ] LXFR: Saving derivative data. [2025-10-20T22:11:33.275174] [DEBUG] [LOCAL ] LXU: Derivative data saved (1794 records). [2025-10-20T22:11:33.279474] [INFO ] [MAIN ] LXU: Successfully saved derivative data. [2025-10-20T22:11:33.282241] [INFO ] [MAIN ] Derivative Progress: 1096/1877 symbols completed. LXU took 1.1s. Estimated remaining: 4m 54s. Rate: 159.2 symbols/min [2025-10-20T22:11:33.282400] [INFO ] [MAIN ] LYB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:33.285209] [DEBUG] [LOCAL ] LXFR: Derivative data saved (1794 records). [2025-10-20T22:11:33.285355] [INFO ] [MAIN ] LXFR: Successfully saved derivative data. [2025-10-20T22:11:33.288202] [INFO ] [MAIN ] Derivative Progress: 1097/1877 symbols completed. LXFR took 1.1s. Estimated remaining: 4m 54s. Rate: 159.3 symbols/min [2025-10-20T22:11:33.288469] [INFO ] [MAIN ] LYG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:33.306732] [DEBUG] [MAIN ] LXP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:33.307592] [INFO ] [MAIN ] LXP: Generated 1794 derivative records from 50793 primary records. [2025-10-20T22:11:33.307781] [DEBUG] [LOCAL ] LXP: Saving derivative data. [2025-10-20T22:11:33.327862] [DEBUG] [LOCAL ] LXP: Derivative data saved (1794 records). [2025-10-20T22:11:33.341062] [INFO ] [MAIN ] LXP: Successfully saved derivative data. [2025-10-20T22:11:33.343570] [INFO ] [MAIN ] Derivative Progress: 1098/1877 symbols completed. LXP took 1.2s. Estimated remaining: 4m 53s. Rate: 159.5 symbols/min [2025-10-20T22:11:33.343969] [INFO ] [MAIN ] LYV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:33.378203] [DEBUG] [LOCAL ] LYB: Loaded 53621 existing records from primary storage. [2025-10-20T22:11:33.378569] [INFO ] [MAIN ] LYB: Loaded 53621 primary data records. [2025-10-20T22:11:33.513570] [DEBUG] [LOCAL ] LYG: Loaded 52666 existing records from primary storage. [2025-10-20T22:11:33.514220] [INFO ] [MAIN ] LYG: Loaded 52666 primary data records. [2025-10-20T22:11:33.890188] [DEBUG] [LOCAL ] LYV: Loaded 51086 existing records from primary storage. [2025-10-20T22:11:33.902360] [INFO ] [MAIN ] LYV: Loaded 51086 primary data records. [2025-10-20T22:11:34.462073] [DEBUG] [MAIN ] LYB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:34.463179] [INFO ] [MAIN ] LYB: Generated 1794 derivative records from 53621 primary records. [2025-10-20T22:11:34.472096] [DEBUG] [LOCAL ] LYB: Saving derivative data. [2025-10-20T22:11:34.494014] [DEBUG] [MAIN ] LYG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:34.497168] [INFO ] [MAIN ] LYG: Generated 1794 derivative records from 52666 primary records. [2025-10-20T22:11:34.497436] [DEBUG] [LOCAL ] LYG: Saving derivative data. [2025-10-20T22:11:34.499159] [DEBUG] [LOCAL ] LYB: Derivative data saved (1794 records). [2025-10-20T22:11:34.507088] [INFO ] [MAIN ] LYB: Successfully saved derivative data. [2025-10-20T22:11:34.509975] [INFO ] [MAIN ] Derivative Progress: 1099/1877 symbols completed. LYB took 1.2s. Estimated remaining: 4m 53s. Rate: 159.2 symbols/min [2025-10-20T22:11:34.518912] [DEBUG] [LOCAL ] LYG: Derivative data saved (1794 records). [2025-10-20T22:11:34.514438] [INFO ] [MAIN ] LZB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:34.520861] [INFO ] [MAIN ] LYG: Successfully saved derivative data. [2025-10-20T22:11:34.523878] [INFO ] [MAIN ] Derivative Progress: 1100/1877 symbols completed. LYG took 1.2s. Estimated remaining: 4m 53s. Rate: 159.3 symbols/min [2025-10-20T22:11:34.524220] [INFO ] [MAIN ] LZM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:34.538779] [DEBUG] [MAIN ] LYV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:34.539781] [INFO ] [MAIN ] LYV: Generated 1794 derivative records from 51086 primary records. [2025-10-20T22:11:34.539909] [DEBUG] [LOCAL ] LYV: Saving derivative data. [2025-10-20T22:11:34.563200] [DEBUG] [LOCAL ] LYV: Derivative data saved (1794 records). [2025-10-20T22:11:34.563533] [INFO ] [MAIN ] LYV: Successfully saved derivative data. [2025-10-20T22:11:34.565889] [INFO ] [MAIN ] Derivative Progress: 1101/1877 symbols completed. LYV took 1.2s. Estimated remaining: 4m 52s. Rate: 159.4 symbols/min [2025-10-20T22:11:34.566155] [INFO ] [MAIN ] M: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:34.581796] [DEBUG] [LOCAL ] LZM: Loaded 14708 existing records from primary storage. [2025-10-20T22:11:34.581920] [INFO ] [MAIN ] LZM: Loaded 14708 primary data records. [2025-10-20T22:11:34.698106] [DEBUG] [LOCAL ] LZB: Loaded 35168 existing records from primary storage. [2025-10-20T22:11:34.709376] [INFO ] [MAIN ] LZB: Loaded 35168 primary data records. [2025-10-20T22:11:35.119795] [DEBUG] [LOCAL ] M: Loaded 53544 existing records from primary storage. [2025-10-20T22:11:35.222588] [INFO ] [MAIN ] M: Loaded 53544 primary data records. [2025-10-20T22:11:35.561444] [DEBUG] [MAIN ] LZM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:35.562619] [INFO ] [MAIN ] LZM: Generated 1794 derivative records from 14708 primary records. [2025-10-20T22:11:35.562995] [DEBUG] [LOCAL ] LZM: Saving derivative data. [2025-10-20T22:11:35.581864] [DEBUG] [LOCAL ] LZM: Derivative data saved (1794 records). [2025-10-20T22:11:35.583029] [INFO ] [MAIN ] LZM: Successfully saved derivative data. [2025-10-20T22:11:35.585806] [INFO ] [MAIN ] Derivative Progress: 1102/1877 symbols completed. LZM took 1.1s. Estimated remaining: 4m 52s. Rate: 159.2 symbols/min [2025-10-20T22:11:35.586287] [INFO ] [MAIN ] MA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:35.605920] [DEBUG] [MAIN ] LZB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:35.606829] [INFO ] [MAIN ] LZB: Generated 1794 derivative records from 35168 primary records. [2025-10-20T22:11:35.607003] [DEBUG] [LOCAL ] LZB: Saving derivative data. [2025-10-20T22:11:35.626479] [DEBUG] [LOCAL ] LZB: Derivative data saved (1794 records). [2025-10-20T22:11:35.626690] [INFO ] [MAIN ] LZB: Successfully saved derivative data. [2025-10-20T22:11:35.628988] [INFO ] [MAIN ] Derivative Progress: 1103/1877 symbols completed. LZB took 1.1s. Estimated remaining: 4m 52s. Rate: 159.3 symbols/min [2025-10-20T22:11:35.629321] [INFO ] [MAIN ] MAA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:35.643599] [DEBUG] [MAIN ] M: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:35.644412] [INFO ] [MAIN ] M: Generated 1794 derivative records from 53544 primary records. [2025-10-20T22:11:35.644636] [DEBUG] [LOCAL ] M: Saving derivative data. [2025-10-20T22:11:35.678176] [DEBUG] [LOCAL ] M: Derivative data saved (1794 records). [2025-10-20T22:11:35.678329] [INFO ] [MAIN ] M: Successfully saved derivative data. [2025-10-20T22:11:35.681560] [INFO ] [MAIN ] Derivative Progress: 1104/1877 symbols completed. M took 1.1s. Estimated remaining: 4m 51s. Rate: 159.4 symbols/min [2025-10-20T22:11:35.681805] [INFO ] [MAIN ] MAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:35.711914] [DEBUG] [LOCAL ] MA: Loaded 52467 existing records from primary storage. [2025-10-20T22:11:35.712138] [INFO ] [MAIN ] MA: Loaded 52467 primary data records. [2025-10-20T22:11:35.828574] [DEBUG] [LOCAL ] MAA: Loaded 42908 existing records from primary storage. [2025-10-20T22:11:35.839356] [INFO ] [MAIN ] MAA: Loaded 42908 primary data records. [2025-10-20T22:11:36.186452] [DEBUG] [LOCAL ] MAC: Loaded 49690 existing records from primary storage. [2025-10-20T22:11:36.197832] [INFO ] [MAIN ] MAC: Loaded 49690 primary data records. [2025-10-20T22:11:36.700521] [DEBUG] [MAIN ] MA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:36.702155] [INFO ] [MAIN ] MA: Generated 1794 derivative records from 52467 primary records. [2025-10-20T22:11:36.702613] [DEBUG] [LOCAL ] MA: Saving derivative data. [2025-10-20T22:11:36.721485] [DEBUG] [LOCAL ] MA: Derivative data saved (1794 records). [2025-10-20T22:11:36.721796] [INFO ] [MAIN ] MA: Successfully saved derivative data. [2025-10-20T22:11:36.723953] [INFO ] [MAIN ] Derivative Progress: 1105/1877 symbols completed. MA took 1.1s. Estimated remaining: 4m 51s. Rate: 159.2 symbols/min [2025-10-20T22:11:36.724207] [INFO ] [MAIN ] MAGN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:36.742832] [DEBUG] [MAIN ] MAA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:36.743728] [INFO ] [MAIN ] MAA: Generated 1794 derivative records from 42908 primary records. [2025-10-20T22:11:36.743847] [DEBUG] [LOCAL ] MAA: Saving derivative data. [2025-10-20T22:11:36.762031] [DEBUG] [LOCAL ] MAA: Derivative data saved (1794 records). [2025-10-20T22:11:36.769911] [INFO ] [MAIN ] MAA: Successfully saved derivative data. [2025-10-20T22:11:36.777411] [INFO ] [MAIN ] Derivative Progress: 1106/1877 symbols completed. MAA took 1.1s. Estimated remaining: 4m 50s. Rate: 159.3 symbols/min [2025-10-20T22:11:36.777729] [INFO ] [MAIN ] MAIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:36.778553] [DEBUG] [MAIN ] MAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:36.779637] [INFO ] [MAIN ] MAC: Generated 1794 derivative records from 49690 primary records. [2025-10-20T22:11:36.779769] [DEBUG] [LOCAL ] MAC: Saving derivative data. [2025-10-20T22:11:36.806075] [DEBUG] [LOCAL ] MAC: Derivative data saved (1794 records). [2025-10-20T22:11:36.806261] [INFO ] [MAIN ] MAC: Successfully saved derivative data. [2025-10-20T22:11:36.808666] [INFO ] [MAIN ] Derivative Progress: 1107/1877 symbols completed. MAC took 1.1s. Estimated remaining: 4m 50s. Rate: 159.4 symbols/min [2025-10-20T22:11:36.808895] [INFO ] [MAIN ] MAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:36.821735] [DEBUG] [LOCAL ] MAGN: Loaded 36341 existing records from primary storage. [2025-10-20T22:11:36.821870] [INFO ] [MAIN ] MAGN: Loaded 36341 primary data records. [2025-10-20T22:11:37.144272] [DEBUG] [LOCAL ] MAIN: Loaded 41956 existing records from primary storage. [2025-10-20T22:11:37.144520] [INFO ] [MAIN ] MAIN: Loaded 41956 primary data records. [2025-10-20T22:11:37.254688] [DEBUG] [LOCAL ] MAN: Loaded 44330 existing records from primary storage. [2025-10-20T22:11:37.266243] [INFO ] [MAIN ] MAN: Loaded 44330 primary data records. [2025-10-20T22:11:37.816828] [DEBUG] [MAIN ] MAGN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:37.819576] [INFO ] [MAIN ] MAGN: Generated 1794 derivative records from 36341 primary records. [2025-10-20T22:11:37.820195] [DEBUG] [LOCAL ] MAGN: Saving derivative data. [2025-10-20T22:11:37.838990] [DEBUG] [LOCAL ] MAGN: Derivative data saved (1794 records). [2025-10-20T22:11:37.839332] [INFO ] [MAIN ] MAGN: Successfully saved derivative data. [2025-10-20T22:11:37.841700] [INFO ] [MAIN ] Derivative Progress: 1108/1877 symbols completed. MAGN took 1.1s. Estimated remaining: 4m 50s. Rate: 159.2 symbols/min [2025-10-20T22:11:37.858290] [DEBUG] [MAIN ] MAIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:37.859094] [INFO ] [MAIN ] MAIN: Generated 1794 derivative records from 41956 primary records. [2025-10-20T22:11:37.859264] [DEBUG] [LOCAL ] MAIN: Saving derivative data. [2025-10-20T22:11:37.889254] [DEBUG] [LOCAL ] MAIN: Derivative data saved (1794 records). [2025-10-20T22:11:37.889438] [INFO ] [MAIN ] MAIN: Successfully saved derivative data. [2025-10-20T22:11:37.891573] [INFO ] [MAIN ] Derivative Progress: 1109/1877 symbols completed. MAIN took 1.1s. Estimated remaining: 4m 49s. Rate: 159.3 symbols/min [2025-10-20T22:11:37.889774] [DEBUG] [MAIN ] MAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:37.892464] [INFO ] [MAIN ] MAN: Generated 1794 derivative records from 44330 primary records. [2025-10-20T22:11:37.892600] [DEBUG] [LOCAL ] MAN: Saving derivative data. [2025-10-20T22:11:37.907565] [DEBUG] [LOCAL ] MAN: Derivative data saved (1794 records). [2025-10-20T22:11:37.907626] [INFO ] [MAIN ] MAN: Successfully saved derivative data. [2025-10-20T22:11:37.909591] [INFO ] [MAIN ] Derivative Progress: 1110/1877 symbols completed. MAN took 1.1s. Estimated remaining: 4m 49s. Rate: 159.4 symbols/min [2025-10-20T22:11:37.909980] [INFO ] [MAIN ] MANU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:37.910046] [INFO ] [MAIN ] MAS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:37.910557] [INFO ] [MAIN ] MATV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:37.945144] [DEBUG] [LOCAL ] MANU: Loaded 27281 existing records from primary storage. [2025-10-20T22:11:37.945257] [INFO ] [MAIN ] MANU: Loaded 27281 primary data records. [2025-10-20T22:11:38.032965] [DEBUG] [LOCAL ] MATV: Loaded 35365 existing records from primary storage. [2025-10-20T22:11:38.049735] [INFO ] [MAIN ] MATV: Loaded 35365 primary data records. [2025-10-20T22:11:38.184401] [DEBUG] [LOCAL ] MAS: Loaded 52281 existing records from primary storage. [2025-10-20T22:11:38.267498] [INFO ] [MAIN ] MAS: Loaded 52281 primary data records. [2025-10-20T22:11:38.891928] [DEBUG] [MAIN ] MANU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:38.892706] [INFO ] [MAIN ] MANU: Generated 1794 derivative records from 27281 primary records. [2025-10-20T22:11:38.893997] [DEBUG] [LOCAL ] MANU: Saving derivative data. [2025-10-20T22:11:38.923771] [DEBUG] [LOCAL ] MANU: Derivative data saved (1794 records). [2025-10-20T22:11:38.924084] [INFO ] [MAIN ] MANU: Successfully saved derivative data. [2025-10-20T22:11:38.924266] [DEBUG] [MAIN ] MATV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:38.926234] [INFO ] [MAIN ] Derivative Progress: 1111/1877 symbols completed. MANU took 1.0s. Estimated remaining: 4m 49s. Rate: 159.2 symbols/min [2025-10-20T22:11:38.926371] [INFO ] [MAIN ] MATX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:38.927710] [INFO ] [MAIN ] MATV: Generated 1794 derivative records from 35365 primary records. [2025-10-20T22:11:38.927828] [DEBUG] [LOCAL ] MATV: Saving derivative data. [2025-10-20T22:11:38.945555] [DEBUG] [LOCAL ] MATV: Derivative data saved (1794 records). [2025-10-20T22:11:38.945793] [INFO ] [MAIN ] MATV: Successfully saved derivative data. [2025-10-20T22:11:38.948264] [INFO ] [MAIN ] Derivative Progress: 1112/1877 symbols completed. MATV took 1.0s. Estimated remaining: 4m 48s. Rate: 159.3 symbols/min [2025-10-20T22:11:38.948514] [INFO ] [MAIN ] MAX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:38.961826] [DEBUG] [MAIN ] MAS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:38.962531] [INFO ] [MAIN ] MAS: Generated 1794 derivative records from 52281 primary records. [2025-10-20T22:11:38.962803] [DEBUG] [LOCAL ] MAS: Saving derivative data. [2025-10-20T22:11:38.986217] [DEBUG] [LOCAL ] MAS: Derivative data saved (1794 records). [2025-10-20T22:11:38.986359] [INFO ] [MAIN ] MAS: Successfully saved derivative data. [2025-10-20T22:11:38.988565] [INFO ] [MAIN ] Derivative Progress: 1113/1877 symbols completed. MAS took 1.1s. Estimated remaining: 4m 47s. Rate: 159.5 symbols/min [2025-10-20T22:11:38.988856] [INFO ] [MAIN ] MBC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:39.009232] [DEBUG] [LOCAL ] MATX: Loaded 32025 existing records from primary storage. [2025-10-20T22:11:39.009389] [INFO ] [MAIN ] MATX: Loaded 32025 primary data records. [2025-10-20T22:11:39.062439] [DEBUG] [LOCAL ] MAX: Loaded 30945 existing records from primary storage. [2025-10-20T22:11:39.099753] [INFO ] [MAIN ] MAX: Loaded 30945 primary data records. [2025-10-20T22:11:39.509411] [DEBUG] [LOCAL ] MBC: Loaded 43967 existing records from primary storage. [2025-10-20T22:11:39.652016] [INFO ] [MAIN ] MBC: Loaded 43967 primary data records. [2025-10-20T22:11:39.982175] [DEBUG] [MAIN ] MATX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:39.983369] [INFO ] [MAIN ] MATX: Generated 1794 derivative records from 32025 primary records. [2025-10-20T22:11:39.984929] [DEBUG] [LOCAL ] MATX: Saving derivative data. [2025-10-20T22:11:40.009040] [DEBUG] [LOCAL ] MATX: Derivative data saved (1794 records). [2025-10-20T22:11:40.009179] [INFO ] [MAIN ] MATX: Successfully saved derivative data. [2025-10-20T22:11:40.015970] [INFO ] [MAIN ] Derivative Progress: 1114/1877 symbols completed. MATX took 1.1s. Estimated remaining: 4m 48s. Rate: 159.2 symbols/min [2025-10-20T22:11:40.016344] [INFO ] [MAIN ] MBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:40.016723] [DEBUG] [MAIN ] MAX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:40.017830] [INFO ] [MAIN ] MAX: Generated 1794 derivative records from 30945 primary records. [2025-10-20T22:11:40.019099] [DEBUG] [LOCAL ] MAX: Saving derivative data. [2025-10-20T22:11:40.035459] [DEBUG] [LOCAL ] MAX: Derivative data saved (1794 records). [2025-10-20T22:11:40.035766] [INFO ] [MAIN ] MAX: Successfully saved derivative data. [2025-10-20T22:11:40.038181] [INFO ] [MAIN ] Derivative Progress: 1115/1877 symbols completed. MAX took 1.1s. Estimated remaining: 4m 47s. Rate: 159.3 symbols/min [2025-10-20T22:11:40.038326] [INFO ] [MAIN ] MCB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:40.071393] [DEBUG] [MAIN ] MBC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:40.071848] [DEBUG] [LOCAL ] MBI: Loaded 24304 existing records from primary storage. [2025-10-20T22:11:40.072759] [INFO ] [MAIN ] MBC: Generated 1794 derivative records from 43967 primary records. [2025-10-20T22:11:40.077391] [DEBUG] [LOCAL ] MBC: Saving derivative data. [2025-10-20T22:11:40.077319] [DEBUG] [LOCAL ] MCB: Loaded 11359 existing records from primary storage. [2025-10-20T22:11:40.077085] [INFO ] [MAIN ] MBI: Loaded 24304 primary data records. [2025-10-20T22:11:40.077883] [INFO ] [MAIN ] MCB: Loaded 11359 primary data records. [2025-10-20T22:11:40.234578] [DEBUG] [LOCAL ] MBC: Derivative data saved (1794 records). [2025-10-20T22:11:40.240716] [INFO ] [MAIN ] MBC: Successfully saved derivative data. [2025-10-20T22:11:40.281540] [INFO ] [MAIN ] Derivative Progress: 1116/1877 symbols completed. MBC took 1.3s. Estimated remaining: 4m 46s. Rate: 159.4 symbols/min [2025-10-20T22:11:40.329480] [INFO ] [MAIN ] MCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:40.695389] [DEBUG] [LOCAL ] MCI: Loaded 6235 existing records from primary storage. [2025-10-20T22:11:40.696196] [INFO ] [MAIN ] MCI: Loaded 6235 primary data records. [2025-10-20T22:11:41.022555] [DEBUG] [MAIN ] MBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:41.036150] [INFO ] [MAIN ] MBI: Generated 1794 derivative records from 24304 primary records. [2025-10-20T22:11:41.036745] [DEBUG] [LOCAL ] MBI: Saving derivative data. [2025-10-20T22:11:41.037174] [DEBUG] [MAIN ] MCB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:41.038484] [INFO ] [MAIN ] MCB: Generated 1794 derivative records from 11359 primary records. [2025-10-20T22:11:41.038829] [DEBUG] [LOCAL ] MCB: Saving derivative data. [2025-10-20T22:11:41.067806] [DEBUG] [LOCAL ] MBI: Derivative data saved (1794 records). [2025-10-20T22:11:41.073149] [INFO ] [MAIN ] MBI: Successfully saved derivative data. [2025-10-20T22:11:41.072248] [DEBUG] [LOCAL ] MCB: Derivative data saved (1794 records). [2025-10-20T22:11:41.076108] [INFO ] [MAIN ] Derivative Progress: 1117/1877 symbols completed. MBI took 1.1s. Estimated remaining: 4m 46s. Rate: 159.2 symbols/min [2025-10-20T22:11:41.076606] [INFO ] [MAIN ] MCK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:41.076478] [INFO ] [MAIN ] MCB: Successfully saved derivative data. [2025-10-20T22:11:41.079128] [INFO ] [MAIN ] Derivative Progress: 1118/1877 symbols completed. MCB took 1.0s. Estimated remaining: 4m 46s. Rate: 159.4 symbols/min [2025-10-20T22:11:41.079583] [INFO ] [MAIN ] MCN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:41.092844] [DEBUG] [MAIN ] MCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:41.093456] [INFO ] [MAIN ] MCI: Generated 1794 derivative records from 6235 primary records. [2025-10-20T22:11:41.093532] [DEBUG] [LOCAL ] MCI: Saving derivative data. [2025-10-20T22:11:41.109507] [DEBUG] [LOCAL ] MCI: Derivative data saved (1794 records). [2025-10-20T22:11:41.111856] [INFO ] [MAIN ] MCI: Successfully saved derivative data. [2025-10-20T22:11:41.113931] [INFO ] [MAIN ] Derivative Progress: 1119/1877 symbols completed. MCI took 0.8s. Estimated remaining: 4m 45s. Rate: 159.5 symbols/min [2025-10-20T22:11:41.114147] [INFO ] [MAIN ] MCR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:41.119196] [DEBUG] [LOCAL ] MCN: Loaded 8069 existing records from primary storage. [2025-10-20T22:11:41.119292] [INFO ] [MAIN ] MCN: Loaded 8069 primary data records. [2025-10-20T22:11:41.245795] [DEBUG] [LOCAL ] MCK: Loaded 38322 existing records from primary storage. [2025-10-20T22:11:41.251231] [INFO ] [MAIN ] MCK: Loaded 38322 primary data records. [2025-10-20T22:11:41.262105] [DEBUG] [LOCAL ] MCR: Loaded 8879 existing records from primary storage. [2025-10-20T22:11:41.293447] [INFO ] [MAIN ] MCR: Loaded 8879 primary data records. [2025-10-20T22:11:42.117406] [DEBUG] [MAIN ] MCN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:42.118750] [INFO ] [MAIN ] MCN: Generated 1794 derivative records from 8069 primary records. [2025-10-20T22:11:42.119127] [DEBUG] [LOCAL ] MCN: Saving derivative data. [2025-10-20T22:11:42.135296] [DEBUG] [LOCAL ] MCN: Derivative data saved (1794 records). [2025-10-20T22:11:42.135498] [INFO ] [MAIN ] MCN: Successfully saved derivative data. [2025-10-20T22:11:42.137596] [INFO ] [MAIN ] Derivative Progress: 1120/1877 symbols completed. MCN took 1.1s. Estimated remaining: 4m 45s. Rate: 159.3 symbols/min [2025-10-20T22:11:42.137910] [INFO ] [MAIN ] MCS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:42.169742] [DEBUG] [MAIN ] MCK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:42.175503] [INFO ] [MAIN ] MCK: Generated 1794 derivative records from 38322 primary records. [2025-10-20T22:11:42.175766] [DEBUG] [LOCAL ] MCK: Saving derivative data. [2025-10-20T22:11:42.176142] [DEBUG] [MAIN ] MCR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:42.177688] [INFO ] [MAIN ] MCR: Generated 1794 derivative records from 8879 primary records. [2025-10-20T22:11:42.185049] [DEBUG] [LOCAL ] MCR: Saving derivative data. [2025-10-20T22:11:42.193373] [DEBUG] [LOCAL ] MCK: Derivative data saved (1794 records). [2025-10-20T22:11:42.193470] [INFO ] [MAIN ] MCK: Successfully saved derivative data. [2025-10-20T22:11:42.195845] [INFO ] [MAIN ] Derivative Progress: 1121/1877 symbols completed. MCK took 1.1s. Estimated remaining: 4m 45s. Rate: 159.4 symbols/min [2025-10-20T22:11:42.196401] [INFO ] [MAIN ] MCY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:42.218527] [DEBUG] [LOCAL ] MCS: Loaded 21228 existing records from primary storage. [2025-10-20T22:11:42.219727] [DEBUG] [LOCAL ] MCR: Derivative data saved (1794 records). [2025-10-20T22:11:42.220022] [INFO ] [MAIN ] MCS: Loaded 21228 primary data records. [2025-10-20T22:11:42.220212] [INFO ] [MAIN ] MCR: Successfully saved derivative data. [2025-10-20T22:11:42.247886] [INFO ] [MAIN ] Derivative Progress: 1122/1877 symbols completed. MCR took 1.1s. Estimated remaining: 4m 44s. Rate: 159.5 symbols/min [2025-10-20T22:11:42.269997] [INFO ] [MAIN ] MD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:42.354619] [DEBUG] [LOCAL ] MCY: Loaded 23809 existing records from primary storage. [2025-10-20T22:11:42.360685] [INFO ] [MAIN ] MCY: Loaded 23809 primary data records. [2025-10-20T22:11:42.716763] [DEBUG] [LOCAL ] MD: Loaded 38589 existing records from primary storage. [2025-10-20T22:11:42.741583] [INFO ] [MAIN ] MD: Loaded 38589 primary data records. [2025-10-20T22:11:43.207521] [DEBUG] [MAIN ] MCS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:43.209534] [INFO ] [MAIN ] MCS: Generated 1794 derivative records from 21228 primary records. [2025-10-20T22:11:43.210094] [DEBUG] [LOCAL ] MCS: Saving derivative data. [2025-10-20T22:11:43.242249] [DEBUG] [LOCAL ] MCS: Derivative data saved (1794 records). [2025-10-20T22:11:43.243028] [DEBUG] [MAIN ] MCY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:43.242776] [INFO ] [MAIN ] MCS: Successfully saved derivative data. [2025-10-20T22:11:43.245313] [INFO ] [MAIN ] Derivative Progress: 1123/1877 symbols completed. MCS took 1.1s. Estimated remaining: 4m 44s. Rate: 159.3 symbols/min [2025-10-20T22:11:43.246238] [INFO ] [MAIN ] MCY: Generated 1794 derivative records from 23809 primary records. [2025-10-20T22:11:43.246388] [DEBUG] [LOCAL ] MCY: Saving derivative data. [2025-10-20T22:11:43.263678] [DEBUG] [LOCAL ] MCY: Derivative data saved (1794 records). [2025-10-20T22:11:43.264425] [INFO ] [MAIN ] MCY: Successfully saved derivative data. [2025-10-20T22:11:43.266930] [INFO ] [MAIN ] Derivative Progress: 1124/1877 symbols completed. MCY took 1.1s. Estimated remaining: 4m 43s. Rate: 159.4 symbols/min [2025-10-20T22:11:43.279277] [DEBUG] [MAIN ] MD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:43.279854] [INFO ] [MAIN ] MD: Generated 1794 derivative records from 38589 primary records. [2025-10-20T22:11:43.279912] [DEBUG] [LOCAL ] MD: Saving derivative data. [2025-10-20T22:11:43.294419] [DEBUG] [LOCAL ] MD: Derivative data saved (1794 records). [2025-10-20T22:11:43.294473] [INFO ] [MAIN ] MD: Successfully saved derivative data. [2025-10-20T22:11:43.296566] [INFO ] [MAIN ] Derivative Progress: 1125/1877 symbols completed. MD took 1.0s. Estimated remaining: 4m 43s. Rate: 159.5 symbols/min [2025-10-20T22:11:43.296976] [INFO ] [MAIN ] MDST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:43.297075] [INFO ] [MAIN ] MDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:43.297351] [INFO ] [MAIN ] MDU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:43.307446] [DEBUG] [LOCAL ] MDST: Loaded 6125 existing records from primary storage. [2025-10-20T22:11:43.307894] [INFO ] [MAIN ] MDST: Loaded 6125 primary data records. [2025-10-20T22:11:43.683902] [DEBUG] [LOCAL ] MDT: Loaded 53678 existing records from primary storage. [2025-10-20T22:11:43.684891] [DEBUG] [LOCAL ] MDU: Loaded 48046 existing records from primary storage. [2025-10-20T22:11:43.685066] [INFO ] [MAIN ] MDU: Loaded 48046 primary data records. [2025-10-20T22:11:43.685604] [INFO ] [MAIN ] MDT: Loaded 53678 primary data records. [2025-10-20T22:11:44.312722] [DEBUG] [MAIN ] MDST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:44.314298] [INFO ] [MAIN ] MDST: Generated 1794 derivative records from 6125 primary records. [2025-10-20T22:11:44.314816] [DEBUG] [LOCAL ] MDST: Saving derivative data. [2025-10-20T22:11:44.337128] [DEBUG] [LOCAL ] MDST: Derivative data saved (1794 records). [2025-10-20T22:11:44.337825] [INFO ] [MAIN ] MDST: Successfully saved derivative data. [2025-10-20T22:11:44.340289] [INFO ] [MAIN ] Derivative Progress: 1126/1877 symbols completed. MDST took 1.0s. Estimated remaining: 4m 43s. Rate: 159.3 symbols/min [2025-10-20T22:11:44.340881] [INFO ] [MAIN ] MDV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:44.378320] [DEBUG] [MAIN ] MDU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:44.379169] [DEBUG] [MAIN ] MDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:44.383759] [INFO ] [MAIN ] MDU: Generated 1794 derivative records from 48046 primary records. [2025-10-20T22:11:44.383959] [DEBUG] [LOCAL ] MDU: Saving derivative data. [2025-10-20T22:11:44.385598] [INFO ] [MAIN ] MDT: Generated 1794 derivative records from 53678 primary records. [2025-10-20T22:11:44.386129] [DEBUG] [LOCAL ] MDV: Loaded 8358 existing records from primary storage. [2025-10-20T22:11:44.393231] [DEBUG] [LOCAL ] MDT: Saving derivative data. [2025-10-20T22:11:44.393875] [INFO ] [MAIN ] MDV: Loaded 8358 primary data records. [2025-10-20T22:11:44.473026] [DEBUG] [LOCAL ] MDT: Derivative data saved (1794 records). [2025-10-20T22:11:44.482498] [DEBUG] [LOCAL ] MDU: Derivative data saved (1794 records). [2025-10-20T22:11:44.500008] [INFO ] [MAIN ] MDU: Successfully saved derivative data. [2025-10-20T22:11:44.522581] [INFO ] [MAIN ] Derivative Progress: 1127/1877 symbols completed. MDU took 1.2s. Estimated remaining: 4m 42s. Rate: 159.4 symbols/min [2025-10-20T22:11:44.511869] [INFO ] [MAIN ] MDT: Successfully saved derivative data. [2025-10-20T22:11:44.526589] [INFO ] [MAIN ] MEC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:44.529676] [INFO ] [MAIN ] Derivative Progress: 1128/1877 symbols completed. MDT took 1.2s. Estimated remaining: 4m 42s. Rate: 159.5 symbols/min [2025-10-20T22:11:44.535542] [INFO ] [MAIN ] MED: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:44.742333] [DEBUG] [LOCAL ] MEC: Loaded 16516 existing records from primary storage. [2025-10-20T22:11:44.742609] [INFO ] [MAIN ] MEC: Loaded 16516 primary data records. [2025-10-20T22:11:44.808188] [DEBUG] [LOCAL ] MED: Loaded 21123 existing records from primary storage. [2025-10-20T22:11:44.817279] [INFO ] [MAIN ] MED: Loaded 21123 primary data records. [2025-10-20T22:11:45.264364] [DEBUG] [MAIN ] MDV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:45.265716] [INFO ] [MAIN ] MDV: Generated 1794 derivative records from 8358 primary records. [2025-10-20T22:11:45.265861] [DEBUG] [LOCAL ] MDV: Saving derivative data. [2025-10-20T22:11:45.461413] [DEBUG] [LOCAL ] MDV: Derivative data saved (1794 records). [2025-10-20T22:11:45.462027] [INFO ] [MAIN ] MDV: Successfully saved derivative data. [2025-10-20T22:11:45.464460] [INFO ] [MAIN ] Derivative Progress: 1129/1877 symbols completed. MDV took 1.1s. Estimated remaining: 4m 42s. Rate: 159.3 symbols/min [2025-10-20T22:11:45.464830] [INFO ] [MAIN ] MEG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:45.485489] [DEBUG] [MAIN ] MEC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:45.486334] [INFO ] [MAIN ] MEC: Generated 1794 derivative records from 16516 primary records. [2025-10-20T22:11:45.486601] [DEBUG] [LOCAL ] MEC: Saving derivative data. [2025-10-20T22:11:45.503797] [DEBUG] [LOCAL ] MEC: Derivative data saved (1794 records). [2025-10-20T22:11:45.511696] [INFO ] [MAIN ] MEC: Successfully saved derivative data. [2025-10-20T22:11:45.514109] [INFO ] [MAIN ] Derivative Progress: 1130/1877 symbols completed. MEC took 1.0s. Estimated remaining: 4m 41s. Rate: 159.4 symbols/min [2025-10-20T22:11:45.514335] [INFO ] [MAIN ] MEGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:45.540398] [DEBUG] [MAIN ] MED: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:45.542225] [INFO ] [MAIN ] MED: Generated 1794 derivative records from 21123 primary records. [2025-10-20T22:11:45.542929] [DEBUG] [LOCAL ] MEG: Loaded 31609 existing records from primary storage. [2025-10-20T22:11:45.542325] [DEBUG] [LOCAL ] MED: Saving derivative data. [2025-10-20T22:11:45.543037] [INFO ] [MAIN ] MEG: Loaded 31609 primary data records. [2025-10-20T22:11:45.650762] [DEBUG] [LOCAL ] MED: Derivative data saved (1794 records). [2025-10-20T22:11:45.674963] [INFO ] [MAIN ] MED: Successfully saved derivative data. [2025-10-20T22:11:45.677392] [INFO ] [MAIN ] Derivative Progress: 1131/1877 symbols completed. MED took 1.1s. Estimated remaining: 4m 41s. Rate: 159.5 symbols/min [2025-10-20T22:11:45.674854] [DEBUG] [LOCAL ] MEGI: Loaded 19796 existing records from primary storage. [2025-10-20T22:11:45.682816] [INFO ] [MAIN ] MEI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:45.693155] [INFO ] [MAIN ] MEGI: Loaded 19796 primary data records. [2025-10-20T22:11:46.194603] [DEBUG] [LOCAL ] MEI: Loaded 31501 existing records from primary storage. [2025-10-20T22:11:46.203172] [INFO ] [MAIN ] MEI: Loaded 31501 primary data records. [2025-10-20T22:11:46.527173] [DEBUG] [MAIN ] MEG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:46.528710] [INFO ] [MAIN ] MEG: Generated 1794 derivative records from 31609 primary records. [2025-10-20T22:11:46.528907] [DEBUG] [LOCAL ] MEG: Saving derivative data. [2025-10-20T22:11:46.558705] [DEBUG] [MAIN ] MEGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:46.559438] [INFO ] [MAIN ] MEGI: Generated 1794 derivative records from 19796 primary records. [2025-10-20T22:11:46.560820] [DEBUG] [LOCAL ] MEGI: Saving derivative data. [2025-10-20T22:11:46.581090] [DEBUG] [LOCAL ] MEG: Derivative data saved (1794 records). [2025-10-20T22:11:46.581360] [INFO ] [MAIN ] MEG: Successfully saved derivative data. [2025-10-20T22:11:46.581415] [DEBUG] [LOCAL ] MEGI: Derivative data saved (1794 records). [2025-10-20T22:11:46.583489] [INFO ] [MAIN ] Derivative Progress: 1132/1877 symbols completed. MEG took 1.1s. Estimated remaining: 4m 41s. Rate: 159.3 symbols/min [2025-10-20T22:11:46.583771] [INFO ] [MAIN ] MEGI: Successfully saved derivative data. [2025-10-20T22:11:46.586339] [INFO ] [MAIN ] Derivative Progress: 1133/1877 symbols completed. MEGI took 1.1s. Estimated remaining: 4m 40s. Rate: 159.4 symbols/min [2025-10-20T22:11:46.583701] [INFO ] [MAIN ] MET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:46.586612] [INFO ] [MAIN ] MFA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:46.605352] [DEBUG] [MAIN ] MEI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:46.606064] [INFO ] [MAIN ] MEI: Generated 1794 derivative records from 31501 primary records. [2025-10-20T22:11:46.606242] [DEBUG] [LOCAL ] MEI: Saving derivative data. [2025-10-20T22:11:46.622137] [DEBUG] [LOCAL ] MEI: Derivative data saved (1794 records). [2025-10-20T22:11:46.622308] [INFO ] [MAIN ] MEI: Successfully saved derivative data. [2025-10-20T22:11:46.624512] [INFO ] [MAIN ] Derivative Progress: 1134/1877 symbols completed. MEI took 0.9s. Estimated remaining: 4m 39s. Rate: 159.6 symbols/min [2025-10-20T22:11:46.624771] [INFO ] [MAIN ] MFAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:46.640960] [DEBUG] [LOCAL ] MFAN: Loaded 4316 existing records from primary storage. [2025-10-20T22:11:46.641205] [INFO ] [MAIN ] MFAN: Loaded 4316 primary data records. [2025-10-20T22:11:46.772972] [DEBUG] [LOCAL ] MFA: Loaded 45743 existing records from primary storage. [2025-10-20T22:11:46.788137] [INFO ] [MAIN ] MFA: Loaded 45743 primary data records. [2025-10-20T22:11:46.799575] [DEBUG] [LOCAL ] MET: Loaded 53307 existing records from primary storage. [2025-10-20T22:11:46.815925] [INFO ] [MAIN ] MET: Loaded 53307 primary data records. [2025-10-20T22:11:47.654983] [DEBUG] [MAIN ] MFAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:47.668698] [INFO ] [MAIN ] MFAN: Generated 1794 derivative records from 4316 primary records. [2025-10-20T22:11:47.668947] [DEBUG] [LOCAL ] MFAN: Saving derivative data. [2025-10-20T22:11:47.669481] [DEBUG] [MAIN ] MFA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:47.670693] [INFO ] [MAIN ] MFA: Generated 1794 derivative records from 45743 primary records. [2025-10-20T22:11:47.670913] [DEBUG] [LOCAL ] MFA: Saving derivative data. [2025-10-20T22:11:47.692739] [DEBUG] [LOCAL ] MFAN: Derivative data saved (1794 records). [2025-10-20T22:11:47.693610] [INFO ] [MAIN ] MFAN: Successfully saved derivative data. [2025-10-20T22:11:47.703441] [DEBUG] [LOCAL ] MFA: Derivative data saved (1794 records). [2025-10-20T22:11:47.702409] [INFO ] [MAIN ] Derivative Progress: 1135/1877 symbols completed. MFAN took 1.1s. Estimated remaining: 4m 39s. Rate: 159.3 symbols/min [2025-10-20T22:11:47.703667] [INFO ] [MAIN ] MFA: Successfully saved derivative data. [2025-10-20T22:11:47.706127] [INFO ] [MAIN ] Derivative Progress: 1136/1877 symbols completed. MFA took 1.1s. Estimated remaining: 4m 39s. Rate: 159.4 symbols/min [2025-10-20T22:11:47.706285] [INFO ] [MAIN ] MFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:47.703851] [INFO ] [MAIN ] MFAO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:47.720035] [DEBUG] [MAIN ] MET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:47.720705] [INFO ] [MAIN ] MET: Generated 1794 derivative records from 53307 primary records. [2025-10-20T22:11:47.720886] [DEBUG] [LOCAL ] MET: Saving derivative data. [2025-10-20T22:11:47.738036] [DEBUG] [LOCAL ] MET: Derivative data saved (1794 records). [2025-10-20T22:11:47.738475] [INFO ] [MAIN ] MET: Successfully saved derivative data. [2025-10-20T22:11:47.741295] [INFO ] [MAIN ] Derivative Progress: 1137/1877 symbols completed. MET took 1.2s. Estimated remaining: 4m 38s. Rate: 159.6 symbols/min [2025-10-20T22:11:47.741511] [INFO ] [MAIN ] MFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:47.745163] [DEBUG] [LOCAL ] MFAO: Loaded 3969 existing records from primary storage. [2025-10-20T22:11:47.745345] [INFO ] [MAIN ] MFAO: Loaded 3969 primary data records. [2025-10-20T22:11:48.085449] [DEBUG] [LOCAL ] MFC: Loaded 51048 existing records from primary storage. [2025-10-20T22:11:48.085734] [INFO ] [MAIN ] MFC: Loaded 51048 primary data records. [2025-10-20T22:11:48.189456] [DEBUG] [LOCAL ] MFG: Loaded 49621 existing records from primary storage. [2025-10-20T22:11:48.211087] [INFO ] [MAIN ] MFG: Loaded 49621 primary data records. [2025-10-20T22:11:48.741825] [DEBUG] [MAIN ] MFAO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:48.742616] [INFO ] [MAIN ] MFAO: Generated 1793 derivative records from 3969 primary records. [2025-10-20T22:11:48.743606] [DEBUG] [LOCAL ] MFAO: Saving derivative data. [2025-10-20T22:11:48.764519] [DEBUG] [LOCAL ] MFAO: Derivative data saved (1793 records). [2025-10-20T22:11:48.764904] [INFO ] [MAIN ] MFAO: Successfully saved derivative data. [2025-10-20T22:11:48.767515] [INFO ] [MAIN ] Derivative Progress: 1138/1877 symbols completed. MFAO took 1.1s. Estimated remaining: 4m 38s. Rate: 159.3 symbols/min [2025-10-20T22:11:48.785770] [DEBUG] [MAIN ] MFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:48.786558] [INFO ] [MAIN ] MFC: Generated 1794 derivative records from 51048 primary records. [2025-10-20T22:11:48.786879] [DEBUG] [LOCAL ] MFC: Saving derivative data. [2025-10-20T22:11:48.802868] [DEBUG] [LOCAL ] MFC: Derivative data saved (1794 records). [2025-10-20T22:11:48.803118] [INFO ] [MAIN ] MFC: Successfully saved derivative data. [2025-10-20T22:11:48.805068] [INFO ] [MAIN ] Derivative Progress: 1139/1877 symbols completed. MFC took 1.1s. Estimated remaining: 4m 38s. Rate: 159.4 symbols/min [2025-10-20T22:11:48.820474] [DEBUG] [MAIN ] MFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:48.821172] [INFO ] [MAIN ] MFG: Generated 1794 derivative records from 49621 primary records. [2025-10-20T22:11:48.821235] [DEBUG] [LOCAL ] MFG: Saving derivative data. [2025-10-20T22:11:48.836895] [DEBUG] [LOCAL ] MFG: Derivative data saved (1794 records). [2025-10-20T22:11:48.837015] [INFO ] [MAIN ] MFG: Successfully saved derivative data. [2025-10-20T22:11:48.840143] [INFO ] [MAIN ] Derivative Progress: 1140/1877 symbols completed. MFG took 1.1s. Estimated remaining: 4m 37s. Rate: 159.6 symbols/min [2025-10-20T22:11:48.840625] [INFO ] [MAIN ] MFSG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:48.840787] [INFO ] [MAIN ] MFSI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:48.840574] [INFO ] [MAIN ] MFM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:48.850044] [DEBUG] [LOCAL ] MFSG: Loaded 5899 existing records from primary storage. [2025-10-20T22:11:48.850295] [INFO ] [MAIN ] MFSG: Loaded 5899 primary data records. [2025-10-20T22:11:48.935013] [DEBUG] [LOCAL ] MFM: Loaded 14274 existing records from primary storage. [2025-10-20T22:11:48.952481] [INFO ] [MAIN ] MFM: Loaded 14274 primary data records. [2025-10-20T22:11:48.972478] [DEBUG] [LOCAL ] MFSI: Loaded 12471 existing records from primary storage. [2025-10-20T22:11:49.052715] [INFO ] [MAIN ] MFSI: Loaded 12471 primary data records. [2025-10-20T22:11:49.829701] [DEBUG] [MAIN ] MFSG: Filtered 7001 bins outside market hours, 7822 incomplete bins [2025-10-20T22:11:49.842384] [INFO ] [MAIN ] MFSG: Generated 1792 derivative records from 5899 primary records. [2025-10-20T22:11:49.842900] [DEBUG] [LOCAL ] MFSG: Saving derivative data. [2025-10-20T22:11:49.843710] [DEBUG] [MAIN ] MFM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:49.844899] [INFO ] [MAIN ] MFM: Generated 1794 derivative records from 14274 primary records. [2025-10-20T22:11:49.845067] [DEBUG] [LOCAL ] MFM: Saving derivative data. [2025-10-20T22:11:49.863311] [DEBUG] [LOCAL ] MFSG: Derivative data saved (1792 records). [2025-10-20T22:11:49.863535] [INFO ] [MAIN ] MFSG: Successfully saved derivative data. [2025-10-20T22:11:49.865905] [INFO ] [MAIN ] Derivative Progress: 1141/1877 symbols completed. MFSG took 1.0s. Estimated remaining: 4m 37s. Rate: 159.3 symbols/min [2025-10-20T22:11:49.866338] [INFO ] [MAIN ] MFSV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:49.884064] [DEBUG] [LOCAL ] MFM: Derivative data saved (1794 records). [2025-10-20T22:11:49.884294] [INFO ] [MAIN ] MFM: Successfully saved derivative data. [2025-10-20T22:11:49.886624] [INFO ] [MAIN ] Derivative Progress: 1142/1877 symbols completed. MFM took 1.0s. Estimated remaining: 4m 37s. Rate: 159.5 symbols/min [2025-10-20T22:11:49.887023] [INFO ] [MAIN ] MG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:49.889782] [DEBUG] [LOCAL ] MFSV: Loaded 2924 existing records from primary storage. [2025-10-20T22:11:49.902064] [INFO ] [MAIN ] MFSV: Loaded 2924 primary data records. [2025-10-20T22:11:49.913967] [DEBUG] [MAIN ] MFSI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:49.934091] [INFO ] [MAIN ] MFSI: Generated 1793 derivative records from 12471 primary records. [2025-10-20T22:11:49.987288] [DEBUG] [LOCAL ] MFSI: Saving derivative data. [2025-10-20T22:11:50.015466] [DEBUG] [LOCAL ] MG: Loaded 15611 existing records from primary storage. [2025-10-20T22:11:50.015816] [INFO ] [MAIN ] MG: Loaded 15611 primary data records. [2025-10-20T22:11:50.066365] [DEBUG] [LOCAL ] MFSI: Derivative data saved (1793 records). [2025-10-20T22:11:50.105133] [INFO ] [MAIN ] MFSI: Successfully saved derivative data. [2025-10-20T22:11:50.150341] [INFO ] [MAIN ] Derivative Progress: 1143/1877 symbols completed. MFSI took 1.3s. Estimated remaining: 4m 36s. Rate: 159.5 symbols/min [2025-10-20T22:11:50.282358] [INFO ] [MAIN ] MGA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:50.563188] [DEBUG] [LOCAL ] MGA: Loaded 50442 existing records from primary storage. [2025-10-20T22:11:50.563980] [INFO ] [MAIN ] MGA: Loaded 50442 primary data records. [2025-10-20T22:11:50.622111] [DEBUG] [MAIN ] MFSV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:50.644982] [INFO ] [MAIN ] MFSV: Generated 1793 derivative records from 2924 primary records. [2025-10-20T22:11:50.645123] [DEBUG] [LOCAL ] MFSV: Saving derivative data. [2025-10-20T22:11:50.910682] [DEBUG] [MAIN ] MG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:50.913063] [INFO ] [MAIN ] MG: Generated 1794 derivative records from 15611 primary records. [2025-10-20T22:11:50.915183] [DEBUG] [LOCAL ] MFSV: Derivative data saved (1793 records). [2025-10-20T22:11:50.915376] [INFO ] [MAIN ] MFSV: Successfully saved derivative data. [2025-10-20T22:11:50.917371] [INFO ] [MAIN ] Derivative Progress: 1144/1877 symbols completed. MFSV took 1.1s. Estimated remaining: 4m 36s. Rate: 159.4 symbols/min [2025-10-20T22:11:50.914340] [DEBUG] [LOCAL ] MG: Saving derivative data. [2025-10-20T22:11:50.917659] [INFO ] [MAIN ] MGF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:50.939490] [DEBUG] [LOCAL ] MG: Derivative data saved (1794 records). [2025-10-20T22:11:50.942724] [DEBUG] [LOCAL ] MGF: Loaded 4141 existing records from primary storage. [2025-10-20T22:11:50.939881] [INFO ] [MAIN ] MG: Successfully saved derivative data. [2025-10-20T22:11:50.946257] [INFO ] [MAIN ] Derivative Progress: 1145/1877 symbols completed. MG took 1.1s. Estimated remaining: 4m 35s. Rate: 159.5 symbols/min [2025-10-20T22:11:50.946421] [INFO ] [MAIN ] MGF: Loaded 4141 primary data records. [2025-10-20T22:11:50.991253] [INFO ] [MAIN ] MGM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:51.304221] [DEBUG] [MAIN ] MGA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:51.305496] [INFO ] [MAIN ] MGA: Generated 1794 derivative records from 50442 primary records. [2025-10-20T22:11:51.306142] [DEBUG] [LOCAL ] MGA: Saving derivative data. [2025-10-20T22:11:51.338773] [DEBUG] [LOCAL ] MGA: Derivative data saved (1794 records). [2025-10-20T22:11:51.339034] [INFO ] [MAIN ] MGA: Successfully saved derivative data. [2025-10-20T22:11:51.341501] [INFO ] [MAIN ] Derivative Progress: 1146/1877 symbols completed. MGA took 1.1s. Estimated remaining: 4m 35s. Rate: 159.5 symbols/min [2025-10-20T22:11:51.341851] [INFO ] [MAIN ] MGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:51.361705] [DEBUG] [LOCAL ] MGM: Loaded 53607 existing records from primary storage. [2025-10-20T22:11:51.364220] [INFO ] [MAIN ] MGM: Loaded 53607 primary data records. [2025-10-20T22:11:51.408203] [DEBUG] [LOCAL ] MGR: Loaded 5576 existing records from primary storage. [2025-10-20T22:11:51.437423] [INFO ] [MAIN ] MGR: Loaded 5576 primary data records. [2025-10-20T22:11:51.710836] [DEBUG] [MAIN ] MGF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:51.826752] [INFO ] [MAIN ] MGF: Generated 1794 derivative records from 4141 primary records. [2025-10-20T22:11:51.866263] [DEBUG] [LOCAL ] MGF: Saving derivative data. [2025-10-20T22:11:52.031011] [DEBUG] [LOCAL ] MGF: Derivative data saved (1794 records). [2025-10-20T22:11:52.031543] [INFO ] [MAIN ] MGF: Successfully saved derivative data. [2025-10-20T22:11:52.034051] [INFO ] [MAIN ] Derivative Progress: 1147/1877 symbols completed. MGF took 1.1s. Estimated remaining: 4m 35s. Rate: 159.4 symbols/min [2025-10-20T22:11:52.034482] [INFO ] [MAIN ] MGRB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:52.043445] [DEBUG] [LOCAL ] MGRB: Loaded 4005 existing records from primary storage. [2025-10-20T22:11:52.043740] [INFO ] [MAIN ] MGRB: Loaded 4005 primary data records. [2025-10-20T22:11:52.135503] [DEBUG] [MAIN ] MGM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:52.146703] [INFO ] [MAIN ] MGM: Generated 1794 derivative records from 53607 primary records. [2025-10-20T22:11:52.158692] [DEBUG] [LOCAL ] MGM: Saving derivative data. [2025-10-20T22:11:52.231226] [DEBUG] [LOCAL ] MGM: Derivative data saved (1794 records). [2025-10-20T22:11:52.230543] [DEBUG] [MAIN ] MGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:52.241742] [INFO ] [MAIN ] MGM: Successfully saved derivative data. [2025-10-20T22:11:52.242700] [INFO ] [MAIN ] MGR: Generated 1794 derivative records from 5576 primary records. [2025-10-20T22:11:52.250032] [DEBUG] [LOCAL ] MGR: Saving derivative data. [2025-10-20T22:11:52.249969] [INFO ] [MAIN ] Derivative Progress: 1148/1877 symbols completed. MGM took 1.3s. Estimated remaining: 4m 34s. Rate: 159.4 symbols/min [2025-10-20T22:11:52.409359] [INFO ] [MAIN ] MGRD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:52.436474] [DEBUG] [LOCAL ] MGR: Derivative data saved (1794 records). [2025-10-20T22:11:52.436967] [INFO ] [MAIN ] MGR: Successfully saved derivative data. [2025-10-20T22:11:52.440572] [DEBUG] [LOCAL ] MGRD: Loaded 2218 existing records from primary storage. [2025-10-20T22:11:52.441066] [INFO ] [MAIN ] MGRD: Loaded 2218 primary data records. [2025-10-20T22:11:52.443154] [INFO ] [MAIN ] Derivative Progress: 1149/1877 symbols completed. MGR took 1.1s. Estimated remaining: 4m 34s. Rate: 159.5 symbols/min [2025-10-20T22:11:52.456950] [INFO ] [MAIN ] MGRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:52.564776] [DEBUG] [LOCAL ] MGRE: Loaded 6471 existing records from primary storage. [2025-10-20T22:11:52.593410] [INFO ] [MAIN ] MGRE: Loaded 6471 primary data records. [2025-10-20T22:11:52.986570] [DEBUG] [MAIN ] MGRB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:53.031568] [INFO ] [MAIN ] MGRB: Generated 1794 derivative records from 4005 primary records. [2025-10-20T22:11:53.105153] [DEBUG] [LOCAL ] MGRB: Saving derivative data. [2025-10-20T22:11:53.121479] [DEBUG] [LOCAL ] MGRB: Derivative data saved (1794 records). [2025-10-20T22:11:53.122165] [INFO ] [MAIN ] MGRB: Successfully saved derivative data. [2025-10-20T22:11:53.125186] [INFO ] [MAIN ] Derivative Progress: 1150/1877 symbols completed. MGRB took 1.1s. Estimated remaining: 4m 34s. Rate: 159.4 symbols/min [2025-10-20T22:11:53.125558] [INFO ] [MAIN ] MGY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:53.154412] [DEBUG] [MAIN ] MGRD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:53.155889] [INFO ] [MAIN ] MGRD: Generated 1794 derivative records from 2218 primary records. [2025-10-20T22:11:53.156346] [DEBUG] [LOCAL ] MGRD: Saving derivative data. [2025-10-20T22:11:53.188119] [DEBUG] [LOCAL ] MGRD: Derivative data saved (1794 records). [2025-10-20T22:11:53.202872] [INFO ] [MAIN ] MGRD: Successfully saved derivative data. [2025-10-20T22:11:53.206086] [INFO ] [MAIN ] Derivative Progress: 1151/1877 symbols completed. MGRD took 0.8s. Estimated remaining: 4m 33s. Rate: 159.5 symbols/min [2025-10-20T22:11:53.202478] [DEBUG] [MAIN ] MGRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:53.206359] [INFO ] [MAIN ] MHD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:53.207361] [INFO ] [MAIN ] MGRE: Generated 1794 derivative records from 6471 primary records. [2025-10-20T22:11:53.208117] [DEBUG] [LOCAL ] MGRE: Saving derivative data. [2025-10-20T22:11:53.224500] [DEBUG] [LOCAL ] MGRE: Derivative data saved (1794 records). [2025-10-20T22:11:53.240256] [INFO ] [MAIN ] MGRE: Successfully saved derivative data. [2025-10-20T22:11:53.242416] [INFO ] [MAIN ] Derivative Progress: 1152/1877 symbols completed. MGRE took 0.8s. Estimated remaining: 4m 33s. Rate: 159.6 symbols/min [2025-10-20T22:11:53.242762] [INFO ] [MAIN ] MHF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:53.246442] [DEBUG] [LOCAL ] MGY: Loaded 52437 existing records from primary storage. [2025-10-20T22:11:53.246620] [INFO ] [MAIN ] MGY: Loaded 52437 primary data records. [2025-10-20T22:11:53.358117] [DEBUG] [LOCAL ] MHF: Loaded 9023 existing records from primary storage. [2025-10-20T22:11:53.362059] [DEBUG] [LOCAL ] MHD: Loaded 12053 existing records from primary storage. [2025-10-20T22:11:53.362480] [INFO ] [MAIN ] MHF: Loaded 9023 primary data records. [2025-10-20T22:11:53.398866] [INFO ] [MAIN ] MHD: Loaded 12053 primary data records. [2025-10-20T22:11:54.182598] [DEBUG] [MAIN ] MGY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:54.187754] [INFO ] [MAIN ] MGY: Generated 1794 derivative records from 52437 primary records. [2025-10-20T22:11:54.188694] [DEBUG] [LOCAL ] MGY: Saving derivative data. [2025-10-20T22:11:54.203430] [DEBUG] [MAIN ] MHF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:54.225330] [INFO ] [MAIN ] MHF: Generated 1794 derivative records from 9023 primary records. [2025-10-20T22:11:54.230140] [DEBUG] [LOCAL ] MHF: Saving derivative data. [2025-10-20T22:11:54.230434] [DEBUG] [MAIN ] MHD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:54.234030] [INFO ] [MAIN ] MHD: Generated 1794 derivative records from 12053 primary records. [2025-10-20T22:11:54.242000] [DEBUG] [LOCAL ] MHD: Saving derivative data. [2025-10-20T22:11:54.244404] [DEBUG] [LOCAL ] MGY: Derivative data saved (1794 records). [2025-10-20T22:11:54.262787] [DEBUG] [LOCAL ] MHF: Derivative data saved (1794 records). [2025-10-20T22:11:54.264971] [INFO ] [MAIN ] MHF: Successfully saved derivative data. [2025-10-20T22:11:54.267350] [INFO ] [MAIN ] Derivative Progress: 1153/1877 symbols completed. MHF took 1.0s. Estimated remaining: 4m 33s. Rate: 159.4 symbols/min [2025-10-20T22:11:54.264830] [INFO ] [MAIN ] MGY: Successfully saved derivative data. [2025-10-20T22:11:54.269504] [DEBUG] [LOCAL ] MHD: Derivative data saved (1794 records). [2025-10-20T22:11:54.269782] [INFO ] [MAIN ] MHD: Successfully saved derivative data. [2025-10-20T22:11:54.273236] [INFO ] [MAIN ] Derivative Progress: 1154/1877 symbols completed. MHD took 1.1s. Estimated remaining: 4m 32s. Rate: 159.5 symbols/min [2025-10-20T22:11:54.275785] [INFO ] [MAIN ] Derivative Progress: 1155/1877 symbols completed. MGY took 1.1s. Estimated remaining: 4m 31s. Rate: 159.6 symbols/min [2025-10-20T22:11:54.276166] [INFO ] [MAIN ] MHK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:54.276353] [INFO ] [MAIN ] MHN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:54.276257] [INFO ] [MAIN ] MHO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:54.290137] [DEBUG] [LOCAL ] MHN: Loaded 7475 existing records from primary storage. [2025-10-20T22:11:54.290416] [INFO ] [MAIN ] MHN: Loaded 7475 primary data records. [2025-10-20T22:11:54.417111] [DEBUG] [LOCAL ] MHO: Loaded 23788 existing records from primary storage. [2025-10-20T22:11:54.437282] [INFO ] [MAIN ] MHO: Loaded 23788 primary data records. [2025-10-20T22:11:54.606597] [DEBUG] [LOCAL ] MHK: Loaded 42112 existing records from primary storage. [2025-10-20T22:11:54.739187] [INFO ] [MAIN ] MHK: Loaded 42112 primary data records. [2025-10-20T22:11:55.293061] [DEBUG] [MAIN ] MHN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:55.294551] [INFO ] [MAIN ] MHN: Generated 1794 derivative records from 7475 primary records. [2025-10-20T22:11:55.294855] [DEBUG] [LOCAL ] MHN: Saving derivative data. [2025-10-20T22:11:55.314249] [DEBUG] [LOCAL ] MHN: Derivative data saved (1794 records). [2025-10-20T22:11:55.314427] [INFO ] [MAIN ] MHN: Successfully saved derivative data. [2025-10-20T22:11:55.316392] [INFO ] [MAIN ] Derivative Progress: 1156/1877 symbols completed. MHN took 1.0s. Estimated remaining: 4m 31s. Rate: 159.4 symbols/min [2025-10-20T22:11:55.316757] [INFO ] [MAIN ] MIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:55.330743] [DEBUG] [MAIN ] MHO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:55.331664] [INFO ] [MAIN ] MHO: Generated 1794 derivative records from 23788 primary records. [2025-10-20T22:11:55.331845] [DEBUG] [LOCAL ] MHO: Saving derivative data. [2025-10-20T22:11:55.356197] [DEBUG] [LOCAL ] MHO: Derivative data saved (1794 records). [2025-10-20T22:11:55.361507] [INFO ] [MAIN ] MHO: Successfully saved derivative data. [2025-10-20T22:11:55.361878] [DEBUG] [LOCAL ] MIN: Loaded 13919 existing records from primary storage. [2025-10-20T22:11:55.365289] [INFO ] [MAIN ] Derivative Progress: 1157/1877 symbols completed. MHO took 1.1s. Estimated remaining: 4m 31s. Rate: 159.5 symbols/min [2025-10-20T22:11:55.365676] [INFO ] [MAIN ] MIR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:55.378908] [INFO ] [MAIN ] MIN: Loaded 13919 primary data records. [2025-10-20T22:11:55.380204] [DEBUG] [MAIN ] MHK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:55.393938] [INFO ] [MAIN ] MHK: Generated 1794 derivative records from 42112 primary records. [2025-10-20T22:11:55.404720] [DEBUG] [LOCAL ] MHK: Saving derivative data. [2025-10-20T22:11:55.494510] [DEBUG] [LOCAL ] MHK: Derivative data saved (1794 records). [2025-10-20T22:11:55.505585] [INFO ] [MAIN ] MHK: Successfully saved derivative data. [2025-10-20T22:11:55.507889] [INFO ] [MAIN ] Derivative Progress: 1158/1877 symbols completed. MHK took 1.2s. Estimated remaining: 4m 30s. Rate: 159.6 symbols/min [2025-10-20T22:11:55.524234] [INFO ] [MAIN ] MITP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:55.735014] [DEBUG] [LOCAL ] MITP: Loaded 2277 existing records from primary storage. [2025-10-20T22:11:55.735342] [INFO ] [MAIN ] MITP: Loaded 2277 primary data records. [2025-10-20T22:11:55.860303] [DEBUG] [LOCAL ] MIR: Loaded 51381 existing records from primary storage. [2025-10-20T22:11:55.872028] [INFO ] [MAIN ] MIR: Loaded 51381 primary data records. [2025-10-20T22:11:56.366282] [DEBUG] [MAIN ] MIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:56.368232] [INFO ] [MAIN ] MIN: Generated 1794 derivative records from 13919 primary records. [2025-10-20T22:11:56.368359] [DEBUG] [LOCAL ] MIN: Saving derivative data. [2025-10-20T22:11:56.385466] [DEBUG] [LOCAL ] MIN: Derivative data saved (1794 records). [2025-10-20T22:11:56.388720] [INFO ] [MAIN ] MIN: Successfully saved derivative data. [2025-10-20T22:11:56.391304] [INFO ] [MAIN ] Derivative Progress: 1159/1877 symbols completed. MIN took 1.1s. Estimated remaining: 4m 30s. Rate: 159.4 symbols/min [2025-10-20T22:11:56.391573] [INFO ] [MAIN ] MITT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:56.405762] [DEBUG] [MAIN ] MITP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:56.406465] [INFO ] [MAIN ] MITP: Generated 1794 derivative records from 2277 primary records. [2025-10-20T22:11:56.406694] [DEBUG] [LOCAL ] MITP: Saving derivative data. [2025-10-20T22:11:56.423360] [DEBUG] [LOCAL ] MITP: Derivative data saved (1794 records). [2025-10-20T22:11:56.423739] [INFO ] [MAIN ] MITP: Successfully saved derivative data. [2025-10-20T22:11:56.425674] [INFO ] [MAIN ] Derivative Progress: 1160/1877 symbols completed. MITP took 0.9s. Estimated remaining: 4m 30s. Rate: 159.5 symbols/min [2025-10-20T22:11:56.426826] [INFO ] [MAIN ] MIY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:56.448960] [DEBUG] [MAIN ] MIR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:56.449834] [INFO ] [MAIN ] MIR: Generated 1794 derivative records from 51381 primary records. [2025-10-20T22:11:56.450002] [DEBUG] [LOCAL ] MIR: Saving derivative data. [2025-10-20T22:11:56.458468] [DEBUG] [LOCAL ] MITT: Loaded 22548 existing records from primary storage. [2025-10-20T22:11:56.461867] [INFO ] [MAIN ] MITT: Loaded 22548 primary data records. [2025-10-20T22:11:56.462112] [DEBUG] [LOCAL ] MIY: Loaded 6244 existing records from primary storage. [2025-10-20T22:11:56.497014] [INFO ] [MAIN ] MIY: Loaded 6244 primary data records. [2025-10-20T22:11:56.497376] [DEBUG] [LOCAL ] MIR: Derivative data saved (1794 records). [2025-10-20T22:11:56.519617] [INFO ] [MAIN ] MIR: Successfully saved derivative data. [2025-10-20T22:11:56.521876] [INFO ] [MAIN ] Derivative Progress: 1161/1877 symbols completed. MIR took 1.2s. Estimated remaining: 4m 29s. Rate: 159.7 symbols/min [2025-10-20T22:11:56.532118] [INFO ] [MAIN ] MKC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:57.155577] [DEBUG] [LOCAL ] MKC: Loaded 52529 existing records from primary storage. [2025-10-20T22:11:57.156116] [INFO ] [MAIN ] MKC: Loaded 52529 primary data records. [2025-10-20T22:11:57.348555] [DEBUG] [MAIN ] MIY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:57.468935] [INFO ] [MAIN ] MIY: Generated 1794 derivative records from 6244 primary records. [2025-10-20T22:11:57.477502] [DEBUG] [LOCAL ] MIY: Saving derivative data. [2025-10-20T22:11:57.483106] [DEBUG] [MAIN ] MITT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:57.484551] [INFO ] [MAIN ] MITT: Generated 1794 derivative records from 22548 primary records. [2025-10-20T22:11:57.485368] [DEBUG] [LOCAL ] MITT: Saving derivative data. [2025-10-20T22:11:57.516502] [DEBUG] [LOCAL ] MITT: Derivative data saved (1794 records). [2025-10-20T22:11:57.516982] [INFO ] [MAIN ] MITT: Successfully saved derivative data. [2025-10-20T22:11:57.519957] [INFO ] [MAIN ] Derivative Progress: 1162/1877 symbols completed. MITT took 1.1s. Estimated remaining: 4m 29s. Rate: 159.4 symbols/min [2025-10-20T22:11:57.517589] [DEBUG] [LOCAL ] MIY: Derivative data saved (1794 records). [2025-10-20T22:11:57.520366] [INFO ] [MAIN ] MKL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:57.520515] [INFO ] [MAIN ] MIY: Successfully saved derivative data. [2025-10-20T22:11:57.523093] [INFO ] [MAIN ] Derivative Progress: 1163/1877 symbols completed. MIY took 1.1s. Estimated remaining: 4m 28s. Rate: 159.6 symbols/min [2025-10-20T22:11:57.524931] [INFO ] [MAIN ] MLI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:57.544272] [DEBUG] [MAIN ] MKC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:57.544919] [INFO ] [MAIN ] MKC: Generated 1794 derivative records from 52529 primary records. [2025-10-20T22:11:57.545079] [DEBUG] [LOCAL ] MKC: Saving derivative data. [2025-10-20T22:11:57.562175] [DEBUG] [LOCAL ] MKC: Derivative data saved (1794 records). [2025-10-20T22:11:57.570807] [INFO ] [MAIN ] MKC: Successfully saved derivative data. [2025-10-20T22:11:57.572706] [INFO ] [MAIN ] Derivative Progress: 1164/1877 symbols completed. MKC took 1.0s. Estimated remaining: 4m 28s. Rate: 159.7 symbols/min [2025-10-20T22:11:57.573077] [INFO ] [MAIN ] MLM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:57.600789] [DEBUG] [LOCAL ] MKL: Loaded 34079 existing records from primary storage. [2025-10-20T22:11:57.601520] [INFO ] [MAIN ] MKL: Loaded 34079 primary data records. [2025-10-20T22:11:57.697344] [DEBUG] [LOCAL ] MLI: Loaded 43423 existing records from primary storage. [2025-10-20T22:11:57.722859] [INFO ] [MAIN ] MLI: Loaded 43423 primary data records. [2025-10-20T22:11:57.723334] [DEBUG] [LOCAL ] MLM: Loaded 29020 existing records from primary storage. [2025-10-20T22:11:57.733951] [INFO ] [MAIN ] MLM: Loaded 29020 primary data records. [2025-10-20T22:11:58.544998] [DEBUG] [MAIN ] MKL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:58.557843] [INFO ] [MAIN ] MKL: Generated 1794 derivative records from 34079 primary records. [2025-10-20T22:11:58.558513] [DEBUG] [LOCAL ] MKL: Saving derivative data. [2025-10-20T22:11:58.572194] [DEBUG] [MAIN ] MLI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:58.571702] [DEBUG] [MAIN ] MLM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:58.581012] [INFO ] [MAIN ] MLM: Generated 1794 derivative records from 29020 primary records. [2025-10-20T22:11:58.573107] [INFO ] [MAIN ] MLI: Generated 1793 derivative records from 43423 primary records. [2025-10-20T22:11:58.582355] [DEBUG] [LOCAL ] MLM: Saving derivative data. [2025-10-20T22:11:58.585395] [DEBUG] [LOCAL ] MLI: Saving derivative data. [2025-10-20T22:11:58.589568] [DEBUG] [LOCAL ] MKL: Derivative data saved (1794 records). [2025-10-20T22:11:58.590354] [INFO ] [MAIN ] MKL: Successfully saved derivative data. [2025-10-20T22:11:58.599340] [INFO ] [MAIN ] Derivative Progress: 1165/1877 symbols completed. MKL took 1.1s. Estimated remaining: 4m 28s. Rate: 159.4 symbols/min [2025-10-20T22:11:58.603111] [INFO ] [MAIN ] MLNK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:58.618160] [DEBUG] [LOCAL ] MLM: Derivative data saved (1794 records). [2025-10-20T22:11:58.620879] [INFO ] [MAIN ] MLM: Successfully saved derivative data. [2025-10-20T22:11:58.624360] [INFO ] [MAIN ] Derivative Progress: 1166/1877 symbols completed. MLM took 1.1s. Estimated remaining: 4m 27s. Rate: 159.6 symbols/min [2025-10-20T22:11:58.624551] [INFO ] [MAIN ] MLP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:58.622413] [DEBUG] [LOCAL ] MLI: Derivative data saved (1793 records). [2025-10-20T22:11:58.625145] [INFO ] [MAIN ] MLI: Successfully saved derivative data. [2025-10-20T22:11:58.627393] [INFO ] [MAIN ] Derivative Progress: 1167/1877 symbols completed. MLI took 1.1s. Estimated remaining: 4m 27s. Rate: 159.7 symbols/min [2025-10-20T22:11:58.627582] [INFO ] [MAIN ] MLR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:58.632722] [DEBUG] [LOCAL ] MLP: Loaded 3092 existing records from primary storage. [2025-10-20T22:11:58.642250] [INFO ] [MAIN ] MLP: Loaded 3092 primary data records. [2025-10-20T22:11:58.718868] [DEBUG] [LOCAL ] MLR: Loaded 12314 existing records from primary storage. [2025-10-20T22:11:58.725475] [INFO ] [MAIN ] MLR: Loaded 12314 primary data records. [2025-10-20T22:11:58.813519] [DEBUG] [LOCAL ] MLNK: Loaded 27135 existing records from primary storage. [2025-10-20T22:11:58.862366] [INFO ] [MAIN ] MLNK: Loaded 27135 primary data records. [2025-10-20T22:11:59.614581] [DEBUG] [MAIN ] MLP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:59.616011] [INFO ] [MAIN ] MLP: Generated 1794 derivative records from 3092 primary records. [2025-10-20T22:11:59.616159] [DEBUG] [LOCAL ] MLP: Saving derivative data. [2025-10-20T22:11:59.629893] [DEBUG] [MAIN ] MLR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:11:59.637989] [INFO ] [MAIN ] MLR: Generated 1793 derivative records from 12314 primary records. [2025-10-20T22:11:59.642584] [DEBUG] [LOCAL ] MLR: Saving derivative data. [2025-10-20T22:11:59.646285] [DEBUG] [LOCAL ] MLP: Derivative data saved (1794 records). [2025-10-20T22:11:59.646956] [INFO ] [MAIN ] MLP: Successfully saved derivative data. [2025-10-20T22:11:59.649092] [INFO ] [MAIN ] Derivative Progress: 1168/1877 symbols completed. MLP took 1.0s. Estimated remaining: 4m 27s. Rate: 159.5 symbols/min [2025-10-20T22:11:59.681799] [DEBUG] [LOCAL ] MLR: Derivative data saved (1793 records). [2025-10-20T22:11:59.681896] [INFO ] [MAIN ] MLR: Successfully saved derivative data. [2025-10-20T22:11:59.683949] [INFO ] [MAIN ] Derivative Progress: 1169/1877 symbols completed. MLR took 1.1s. Estimated remaining: 4m 26s. Rate: 159.6 symbols/min [2025-10-20T22:11:59.680814] [DEBUG] [MAIN ] MLNK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:11:59.685017] [INFO ] [MAIN ] MLNK: Generated 1794 derivative records from 27135 primary records. [2025-10-20T22:11:59.685115] [DEBUG] [LOCAL ] MLNK: Saving derivative data. [2025-10-20T22:11:59.701368] [DEBUG] [LOCAL ] MLNK: Derivative data saved (1794 records). [2025-10-20T22:11:59.701515] [INFO ] [MAIN ] MLNK: Successfully saved derivative data. [2025-10-20T22:11:59.703595] [INFO ] [MAIN ] Derivative Progress: 1170/1877 symbols completed. MLNK took 1.1s. Estimated remaining: 4m 26s. Rate: 159.7 symbols/min [2025-10-20T22:11:59.704099] [INFO ] [MAIN ] MMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:59.704276] [INFO ] [MAIN ] MMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:59.704431] [INFO ] [MAIN ] MMD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:11:59.722176] [DEBUG] [LOCAL ] MMD: Loaded 10770 existing records from primary storage. [2025-10-20T22:11:59.722311] [INFO ] [MAIN ] MMD: Loaded 10770 primary data records. [2025-10-20T22:11:59.782908] [DEBUG] [LOCAL ] MMI: Loaded 15464 existing records from primary storage. [2025-10-20T22:11:59.826234] [INFO ] [MAIN ] MMI: Loaded 15464 primary data records. [2025-10-20T22:12:00.100428] [DEBUG] [LOCAL ] MMC: Loaded 51958 existing records from primary storage. [2025-10-20T22:12:00.157222] [INFO ] [MAIN ] MMC: Loaded 51958 primary data records. [2025-10-20T22:12:00.702939] [DEBUG] [MAIN ] MMD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:00.706521] [INFO ] [MAIN ] MMD: Generated 1794 derivative records from 10770 primary records. [2025-10-20T22:12:00.706975] [DEBUG] [LOCAL ] MMD: Saving derivative data. [2025-10-20T22:12:00.732389] [DEBUG] [MAIN ] MMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:00.737108] [DEBUG] [LOCAL ] MMD: Derivative data saved (1794 records). [2025-10-20T22:12:00.737265] [INFO ] [MAIN ] MMD: Successfully saved derivative data. [2025-10-20T22:12:00.740340] [INFO ] [MAIN ] MMI: Generated 1794 derivative records from 15464 primary records. [2025-10-20T22:12:00.740750] [DEBUG] [LOCAL ] MMI: Saving derivative data. [2025-10-20T22:12:00.739562] [INFO ] [MAIN ] Derivative Progress: 1171/1877 symbols completed. MMD took 1.0s. Estimated remaining: 4m 26s. Rate: 159.5 symbols/min [2025-10-20T22:12:00.741770] [INFO ] [MAIN ] MMM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:00.759499] [DEBUG] [LOCAL ] MMI: Derivative data saved (1794 records). [2025-10-20T22:12:00.760310] [INFO ] [MAIN ] MMI: Successfully saved derivative data. [2025-10-20T22:12:00.762623] [INFO ] [MAIN ] Derivative Progress: 1172/1877 symbols completed. MMI took 1.1s. Estimated remaining: 4m 25s. Rate: 159.6 symbols/min [2025-10-20T22:12:00.762840] [INFO ] [MAIN ] MMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:00.776728] [DEBUG] [MAIN ] MMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:00.777382] [INFO ] [MAIN ] MMC: Generated 1794 derivative records from 51958 primary records. [2025-10-20T22:12:00.777568] [DEBUG] [LOCAL ] MMC: Saving derivative data. [2025-10-20T22:12:00.794021] [DEBUG] [LOCAL ] MMC: Derivative data saved (1794 records). [2025-10-20T22:12:00.794249] [INFO ] [MAIN ] MMC: Successfully saved derivative data. [2025-10-20T22:12:00.796622] [INFO ] [MAIN ] Derivative Progress: 1173/1877 symbols completed. MMC took 1.1s. Estimated remaining: 4m 24s. Rate: 159.7 symbols/min [2025-10-20T22:12:00.797176] [INFO ] [MAIN ] MMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:00.829279] [DEBUG] [LOCAL ] MMT: Loaded 10181 existing records from primary storage. [2025-10-20T22:12:00.829447] [INFO ] [MAIN ] MMT: Loaded 10181 primary data records. [2025-10-20T22:12:00.932421] [DEBUG] [LOCAL ] MMS: Loaded 33590 existing records from primary storage. [2025-10-20T22:12:00.958237] [INFO ] [MAIN ] MMS: Loaded 33590 primary data records. [2025-10-20T22:12:00.991051] [DEBUG] [LOCAL ] MMM: Loaded 53112 existing records from primary storage. [2025-10-20T22:12:01.069459] [INFO ] [MAIN ] MMM: Loaded 53112 primary data records. [2025-10-20T22:12:01.810093] [DEBUG] [MAIN ] MMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:01.825809] [DEBUG] [MAIN ] MMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:01.827481] [INFO ] [MAIN ] MMS: Generated 1794 derivative records from 33590 primary records. [2025-10-20T22:12:01.826808] [INFO ] [MAIN ] MMT: Generated 1794 derivative records from 10181 primary records. [2025-10-20T22:12:01.827770] [DEBUG] [LOCAL ] MMT: Saving derivative data. [2025-10-20T22:12:01.830496] [DEBUG] [LOCAL ] MMS: Saving derivative data. [2025-10-20T22:12:01.871071] [DEBUG] [LOCAL ] MMT: Derivative data saved (1794 records). [2025-10-20T22:12:01.877026] [DEBUG] [LOCAL ] MMS: Derivative data saved (1794 records). [2025-10-20T22:12:01.877638] [DEBUG] [MAIN ] MMM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:01.871639] [INFO ] [MAIN ] MMT: Successfully saved derivative data. [2025-10-20T22:12:01.878813] [INFO ] [MAIN ] MMM: Generated 1794 derivative records from 53112 primary records. [2025-10-20T22:12:01.880819] [INFO ] [MAIN ] Derivative Progress: 1174/1877 symbols completed. MMT took 1.1s. Estimated remaining: 4m 24s. Rate: 159.5 symbols/min [2025-10-20T22:12:01.878694] [INFO ] [MAIN ] MMS: Successfully saved derivative data. [2025-10-20T22:12:01.881052] [DEBUG] [LOCAL ] MMM: Saving derivative data. [2025-10-20T22:12:01.881249] [INFO ] [MAIN ] MMU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:01.884578] [INFO ] [MAIN ] Derivative Progress: 1175/1877 symbols completed. MMS took 1.1s. Estimated remaining: 4m 24s. Rate: 159.6 symbols/min [2025-10-20T22:12:01.884869] [INFO ] [MAIN ] MNR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:01.904855] [DEBUG] [LOCAL ] MMM: Derivative data saved (1794 records). [2025-10-20T22:12:01.905177] [INFO ] [MAIN ] MMM: Successfully saved derivative data. [2025-10-20T22:12:01.907426] [INFO ] [MAIN ] Derivative Progress: 1176/1877 symbols completed. MMM took 1.2s. Estimated remaining: 4m 23s. Rate: 159.7 symbols/min [2025-10-20T22:12:01.907818] [INFO ] [MAIN ] MNSO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:01.927483] [DEBUG] [LOCAL ] MMU: Loaded 16694 existing records from primary storage. [2025-10-20T22:12:01.927633] [INFO ] [MAIN ] MMU: Loaded 16694 primary data records. [2025-10-20T22:12:02.000913] [DEBUG] [LOCAL ] MNR: Loaded 25017 existing records from primary storage. [2025-10-20T22:12:02.024737] [INFO ] [MAIN ] MNR: Loaded 25017 primary data records. [2025-10-20T22:12:02.316936] [DEBUG] [LOCAL ] MNSO: Loaded 44180 existing records from primary storage. [2025-10-20T22:12:02.438976] [INFO ] [MAIN ] MNSO: Loaded 44180 primary data records. [2025-10-20T22:12:02.986453] [DEBUG] [MAIN ] MNR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:02.985275] [DEBUG] [MAIN ] MMU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:02.987638] [INFO ] [MAIN ] MMU: Generated 1794 derivative records from 16694 primary records. [2025-10-20T22:12:02.988597] [INFO ] [MAIN ] MNR: Generated 1794 derivative records from 25017 primary records. [2025-10-20T22:12:02.988846] [DEBUG] [LOCAL ] MNR: Saving derivative data. [2025-10-20T22:12:02.988910] [DEBUG] [LOCAL ] MMU: Saving derivative data. [2025-10-20T22:12:03.014208] [DEBUG] [LOCAL ] MNR: Derivative data saved (1794 records). [2025-10-20T22:12:03.017529] [INFO ] [MAIN ] MNR: Successfully saved derivative data. [2025-10-20T22:12:03.019698] [INFO ] [MAIN ] Derivative Progress: 1177/1877 symbols completed. MNR took 1.1s. Estimated remaining: 4m 23s. Rate: 159.5 symbols/min [2025-10-20T22:12:03.021969] [INFO ] [MAIN ] MOD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:03.023520] [DEBUG] [LOCAL ] MMU: Derivative data saved (1794 records). [2025-10-20T22:12:03.024173] [INFO ] [MAIN ] MMU: Successfully saved derivative data. [2025-10-20T22:12:03.026093] [INFO ] [MAIN ] Derivative Progress: 1178/1877 symbols completed. MMU took 1.1s. Estimated remaining: 4m 23s. Rate: 159.6 symbols/min [2025-10-20T22:12:03.026442] [INFO ] [MAIN ] MODG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:03.043630] [DEBUG] [MAIN ] MNSO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:03.044404] [INFO ] [MAIN ] MNSO: Generated 1794 derivative records from 44180 primary records. [2025-10-20T22:12:03.044515] [DEBUG] [LOCAL ] MNSO: Saving derivative data. [2025-10-20T22:12:03.061193] [DEBUG] [LOCAL ] MNSO: Derivative data saved (1794 records). [2025-10-20T22:12:03.061305] [INFO ] [MAIN ] MNSO: Successfully saved derivative data. [2025-10-20T22:12:03.063750] [INFO ] [MAIN ] Derivative Progress: 1179/1877 symbols completed. MNSO took 1.2s. Estimated remaining: 4m 22s. Rate: 159.7 symbols/min [2025-10-20T22:12:03.063995] [INFO ] [MAIN ] MOH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:03.107310] [DEBUG] [LOCAL ] MOD: Loaded 44610 existing records from primary storage. [2025-10-20T22:12:03.108152] [INFO ] [MAIN ] MOD: Loaded 44610 primary data records. [2025-10-20T22:12:03.188777] [DEBUG] [LOCAL ] MODG: Loaded 51551 existing records from primary storage. [2025-10-20T22:12:03.204115] [INFO ] [MAIN ] MODG: Loaded 51551 primary data records. [2025-10-20T22:12:03.315891] [DEBUG] [LOCAL ] MOH: Loaded 43003 existing records from primary storage. [2025-10-20T22:12:03.343404] [INFO ] [MAIN ] MOH: Loaded 43003 primary data records. [2025-10-20T22:12:04.087266] [DEBUG] [MAIN ] MOD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:04.087017] [DEBUG] [MAIN ] MODG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:04.088580] [INFO ] [MAIN ] MOD: Generated 1794 derivative records from 44610 primary records. [2025-10-20T22:12:04.089462] [INFO ] [MAIN ] MODG: Generated 1794 derivative records from 51551 primary records. [2025-10-20T22:12:04.088758] [DEBUG] [LOCAL ] MOD: Saving derivative data. [2025-10-20T22:12:04.089970] [DEBUG] [LOCAL ] MODG: Saving derivative data. [2025-10-20T22:12:04.124719] [DEBUG] [LOCAL ] MODG: Derivative data saved (1794 records). [2025-10-20T22:12:04.120734] [DEBUG] [LOCAL ] MOD: Derivative data saved (1794 records). [2025-10-20T22:12:04.125093] [INFO ] [MAIN ] MOD: Successfully saved derivative data. [2025-10-20T22:12:04.127176] [INFO ] [MAIN ] Derivative Progress: 1180/1877 symbols completed. MOD took 1.1s. Estimated remaining: 4m 22s. Rate: 159.5 symbols/min [2025-10-20T22:12:04.127425] [INFO ] [MAIN ] MOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:04.135348] [INFO ] [MAIN ] MODG: Successfully saved derivative data. [2025-10-20T22:12:04.141044] [DEBUG] [MAIN ] MOH: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:12:04.143315] [INFO ] [MAIN ] Derivative Progress: 1181/1877 symbols completed. MODG took 1.1s. Estimated remaining: 4m 22s. Rate: 159.6 symbols/min [2025-10-20T22:12:04.144097] [INFO ] [MAIN ] MOH: Generated 1793 derivative records from 43003 primary records. [2025-10-20T22:12:04.144328] [INFO ] [MAIN ] MOV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:04.144437] [DEBUG] [LOCAL ] MOH: Saving derivative data. [2025-10-20T22:12:04.161196] [DEBUG] [LOCAL ] MOH: Derivative data saved (1793 records). [2025-10-20T22:12:04.161347] [INFO ] [MAIN ] MOH: Successfully saved derivative data. [2025-10-20T22:12:04.163723] [INFO ] [MAIN ] Derivative Progress: 1182/1877 symbols completed. MOH took 1.1s. Estimated remaining: 4m 21s. Rate: 159.7 symbols/min [2025-10-20T22:12:04.164020] [INFO ] [MAIN ] MP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:04.184169] [DEBUG] [LOCAL ] MOV: Loaded 21500 existing records from primary storage. [2025-10-20T22:12:04.184410] [INFO ] [MAIN ] MOV: Loaded 21500 primary data records. [2025-10-20T22:12:04.388166] [DEBUG] [LOCAL ] MOS: Loaded 53512 existing records from primary storage. [2025-10-20T22:12:04.440338] [INFO ] [MAIN ] MOS: Loaded 53512 primary data records. [2025-10-20T22:12:04.580316] [DEBUG] [LOCAL ] MP: Loaded 53744 existing records from primary storage. [2025-10-20T22:12:04.605685] [INFO ] [MAIN ] MP: Loaded 53744 primary data records. [2025-10-20T22:12:05.281798] [DEBUG] [MAIN ] MOV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:05.284285] [INFO ] [MAIN ] MOV: Generated 1794 derivative records from 21500 primary records. [2025-10-20T22:12:05.285273] [DEBUG] [LOCAL ] MOV: Saving derivative data. [2025-10-20T22:12:05.305250] [DEBUG] [LOCAL ] MOV: Derivative data saved (1794 records). [2025-10-20T22:12:05.305486] [INFO ] [MAIN ] MOV: Successfully saved derivative data. [2025-10-20T22:12:05.307581] [INFO ] [MAIN ] Derivative Progress: 1183/1877 symbols completed. MOV took 1.2s. Estimated remaining: 4m 21s. Rate: 159.5 symbols/min [2025-10-20T22:12:05.329726] [DEBUG] [MAIN ] MOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:05.330889] [INFO ] [MAIN ] MOS: Generated 1794 derivative records from 53512 primary records. [2025-10-20T22:12:05.330973] [DEBUG] [LOCAL ] MOS: Saving derivative data. [2025-10-20T22:12:05.366478] [DEBUG] [LOCAL ] MOS: Derivative data saved (1794 records). [2025-10-20T22:12:05.366816] [INFO ] [MAIN ] MOS: Successfully saved derivative data. [2025-10-20T22:12:05.368893] [INFO ] [MAIN ] Derivative Progress: 1184/1877 symbols completed. MOS took 1.2s. Estimated remaining: 4m 21s. Rate: 159.6 symbols/min [2025-10-20T22:12:05.369131] [DEBUG] [MAIN ] MP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:05.370026] [INFO ] [MAIN ] MP: Generated 1794 derivative records from 53744 primary records. [2025-10-20T22:12:05.370089] [DEBUG] [LOCAL ] MP: Saving derivative data. [2025-10-20T22:12:05.385125] [DEBUG] [LOCAL ] MP: Derivative data saved (1794 records). [2025-10-20T22:12:05.385181] [INFO ] [MAIN ] MP: Successfully saved derivative data. [2025-10-20T22:12:05.387323] [INFO ] [MAIN ] Derivative Progress: 1185/1877 symbols completed. MP took 1.2s. Estimated remaining: 4m 20s. Rate: 159.7 symbols/min [2025-10-20T22:12:05.387871] [INFO ] [MAIN ] MPLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:05.388022] [INFO ] [MAIN ] MPA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:05.387784] [INFO ] [MAIN ] MPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:05.397276] [DEBUG] [LOCAL ] MPA: Loaded 5274 existing records from primary storage. [2025-10-20T22:12:05.397536] [INFO ] [MAIN ] MPA: Loaded 5274 primary data records. [2025-10-20T22:12:05.743885] [DEBUG] [LOCAL ] MPLX: Loaded 50822 existing records from primary storage. [2025-10-20T22:12:05.744220] [INFO ] [MAIN ] MPLX: Loaded 50822 primary data records. [2025-10-20T22:12:05.868331] [DEBUG] [LOCAL ] MPC: Loaded 51981 existing records from primary storage. [2025-10-20T22:12:05.869561] [INFO ] [MAIN ] MPC: Loaded 51981 primary data records. [2025-10-20T22:12:06.422715] [DEBUG] [MAIN ] MPA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:06.423630] [INFO ] [MAIN ] MPA: Generated 1794 derivative records from 5274 primary records. [2025-10-20T22:12:06.423848] [DEBUG] [LOCAL ] MPA: Saving derivative data. [2025-10-20T22:12:06.441294] [DEBUG] [LOCAL ] MPA: Derivative data saved (1794 records). [2025-10-20T22:12:06.441899] [INFO ] [MAIN ] MPA: Successfully saved derivative data. [2025-10-20T22:12:06.443982] [INFO ] [MAIN ] Derivative Progress: 1186/1877 symbols completed. MPA took 1.1s. Estimated remaining: 4m 20s. Rate: 159.5 symbols/min [2025-10-20T22:12:06.444619] [INFO ] [MAIN ] MPV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:06.469279] [DEBUG] [LOCAL ] MPV: Loaded 4405 existing records from primary storage. [2025-10-20T22:12:06.469082] [DEBUG] [MAIN ] MPLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:06.470314] [INFO ] [MAIN ] MPLX: Generated 1794 derivative records from 50822 primary records. [2025-10-20T22:12:06.470426] [DEBUG] [LOCAL ] MPLX: Saving derivative data. [2025-10-20T22:12:06.470686] [INFO ] [MAIN ] MPV: Loaded 4405 primary data records. [2025-10-20T22:12:06.590670] [DEBUG] [LOCAL ] MPLX: Derivative data saved (1794 records). [2025-10-20T22:12:06.590826] [INFO ] [MAIN ] MPLX: Successfully saved derivative data. [2025-10-20T22:12:06.594503] [INFO ] [MAIN ] Derivative Progress: 1187/1877 symbols completed. MPLX took 1.2s. Estimated remaining: 4m 19s. Rate: 159.5 symbols/min [2025-10-20T22:12:06.595099] [INFO ] [MAIN ] MPW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:06.683509] [DEBUG] [MAIN ] MPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:06.689290] [INFO ] [MAIN ] MPC: Generated 1794 derivative records from 51981 primary records. [2025-10-20T22:12:06.794070] [DEBUG] [LOCAL ] MPC: Saving derivative data. [2025-10-20T22:12:06.826621] [DEBUG] [LOCAL ] MPC: Derivative data saved (1794 records). [2025-10-20T22:12:06.840534] [INFO ] [MAIN ] MPC: Successfully saved derivative data. [2025-10-20T22:12:06.843604] [INFO ] [MAIN ] Derivative Progress: 1188/1877 symbols completed. MPC took 1.5s. Estimated remaining: 4m 19s. Rate: 159.6 symbols/min [2025-10-20T22:12:06.844273] [INFO ] [MAIN ] MPX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:06.852154] [DEBUG] [LOCAL ] MPX: Loaded 4023 existing records from primary storage. [2025-10-20T22:12:06.853350] [INFO ] [MAIN ] MPX: Loaded 4023 primary data records. [2025-10-20T22:12:06.923048] [DEBUG] [LOCAL ] MPW: Loaded 53498 existing records from primary storage. [2025-10-20T22:12:06.945966] [INFO ] [MAIN ] MPW: Loaded 53498 primary data records. [2025-10-20T22:12:07.379301] [DEBUG] [MAIN ] MPV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:07.429566] [INFO ] [MAIN ] MPV: Generated 1794 derivative records from 4405 primary records. [2025-10-20T22:12:07.494800] [DEBUG] [LOCAL ] MPV: Saving derivative data. [2025-10-20T22:12:07.560385] [DEBUG] [LOCAL ] MPV: Derivative data saved (1794 records). [2025-10-20T22:12:07.560561] [INFO ] [MAIN ] MPV: Successfully saved derivative data. [2025-10-20T22:12:07.562878] [INFO ] [MAIN ] Derivative Progress: 1189/1877 symbols completed. MPV took 1.1s. Estimated remaining: 4m 19s. Rate: 159.5 symbols/min [2025-10-20T22:12:07.563233] [INFO ] [MAIN ] MQT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:07.575892] [DEBUG] [LOCAL ] MQT: Loaded 8156 existing records from primary storage. [2025-10-20T22:12:07.576235] [INFO ] [MAIN ] MQT: Loaded 8156 primary data records. [2025-10-20T22:12:07.720975] [DEBUG] [MAIN ] MPX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:07.753876] [INFO ] [MAIN ] MPX: Generated 1794 derivative records from 4023 primary records. [2025-10-20T22:12:07.762367] [DEBUG] [LOCAL ] MPX: Saving derivative data. [2025-10-20T22:12:07.891149] [DEBUG] [MAIN ] MPW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:07.912030] [INFO ] [MAIN ] MPW: Generated 1794 derivative records from 53498 primary records. [2025-10-20T22:12:07.918675] [DEBUG] [LOCAL ] MPW: Saving derivative data. [2025-10-20T22:12:07.920102] [DEBUG] [LOCAL ] MPX: Derivative data saved (1794 records). [2025-10-20T22:12:07.920276] [INFO ] [MAIN ] MPX: Successfully saved derivative data. [2025-10-20T22:12:07.922582] [INFO ] [MAIN ] Derivative Progress: 1190/1877 symbols completed. MPX took 1.1s. Estimated remaining: 4m 18s. Rate: 159.5 symbols/min [2025-10-20T22:12:07.934733] [INFO ] [MAIN ] MQY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:07.940353] [DEBUG] [LOCAL ] MPW: Derivative data saved (1794 records). [2025-10-20T22:12:07.940495] [INFO ] [MAIN ] MPW: Successfully saved derivative data. [2025-10-20T22:12:07.942771] [INFO ] [MAIN ] Derivative Progress: 1191/1877 symbols completed. MPW took 1.3s. Estimated remaining: 4m 18s. Rate: 159.6 symbols/min [2025-10-20T22:12:07.943058] [INFO ] [MAIN ] MRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:07.959252] [DEBUG] [LOCAL ] MQY: Loaded 16735 existing records from primary storage. [2025-10-20T22:12:07.959397] [INFO ] [MAIN ] MQY: Loaded 16735 primary data records. [2025-10-20T22:12:08.108213] [DEBUG] [MAIN ] MQT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:08.115106] [INFO ] [MAIN ] MQT: Generated 1794 derivative records from 8156 primary records. [2025-10-20T22:12:08.115381] [DEBUG] [LOCAL ] MQT: Saving derivative data. [2025-10-20T22:12:08.194414] [DEBUG] [LOCAL ] MRC: Loaded 37338 existing records from primary storage. [2025-10-20T22:12:08.303253] [INFO ] [MAIN ] MRC: Loaded 37338 primary data records. [2025-10-20T22:12:08.321252] [DEBUG] [LOCAL ] MQT: Derivative data saved (1794 records). [2025-10-20T22:12:08.343613] [INFO ] [MAIN ] MQT: Successfully saved derivative data. [2025-10-20T22:12:08.355978] [INFO ] [MAIN ] Derivative Progress: 1192/1877 symbols completed. MQT took 0.8s. Estimated remaining: 4m 18s. Rate: 159.6 symbols/min [2025-10-20T22:12:08.368433] [INFO ] [MAIN ] MRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:08.681464] [DEBUG] [MAIN ] MQY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:08.683081] [INFO ] [MAIN ] MQY: Generated 1794 derivative records from 16735 primary records. [2025-10-20T22:12:08.683280] [DEBUG] [LOCAL ] MQY: Saving derivative data. [2025-10-20T22:12:08.713593] [DEBUG] [LOCAL ] MRK: Loaded 53790 existing records from primary storage. [2025-10-20T22:12:08.716638] [INFO ] [MAIN ] MRK: Loaded 53790 primary data records. [2025-10-20T22:12:08.718414] [DEBUG] [LOCAL ] MQY: Derivative data saved (1794 records). [2025-10-20T22:12:08.729840] [INFO ] [MAIN ] MQY: Successfully saved derivative data. [2025-10-20T22:12:08.751961] [INFO ] [MAIN ] Derivative Progress: 1193/1877 symbols completed. MQY took 0.8s. Estimated remaining: 4m 17s. Rate: 159.6 symbols/min [2025-10-20T22:12:08.757487] [INFO ] [MAIN ] MS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:09.072310] [DEBUG] [MAIN ] MRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:09.073160] [INFO ] [MAIN ] MRC: Generated 1794 derivative records from 37338 primary records. [2025-10-20T22:12:09.073300] [DEBUG] [LOCAL ] MRC: Saving derivative data. [2025-10-20T22:12:09.091187] [DEBUG] [LOCAL ] MRC: Derivative data saved (1794 records). [2025-10-20T22:12:09.106835] [INFO ] [MAIN ] MRC: Successfully saved derivative data. [2025-10-20T22:12:09.109120] [INFO ] [MAIN ] Derivative Progress: 1194/1877 symbols completed. MRC took 1.2s. Estimated remaining: 4m 17s. Rate: 159.6 symbols/min [2025-10-20T22:12:09.109913] [INFO ] [MAIN ] MSA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:09.153708] [DEBUG] [LOCAL ] MS: Loaded 53688 existing records from primary storage. [2025-10-20T22:12:09.154093] [DEBUG] [MAIN ] MRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:09.154312] [INFO ] [MAIN ] MS: Loaded 53688 primary data records. [2025-10-20T22:12:09.155101] [INFO ] [MAIN ] MRK: Generated 1794 derivative records from 53790 primary records. [2025-10-20T22:12:09.155330] [DEBUG] [LOCAL ] MRK: Saving derivative data. [2025-10-20T22:12:09.185225] [DEBUG] [LOCAL ] MSA: Loaded 18915 existing records from primary storage. [2025-10-20T22:12:09.195990] [INFO ] [MAIN ] MSA: Loaded 18915 primary data records. [2025-10-20T22:12:09.316749] [DEBUG] [LOCAL ] MRK: Derivative data saved (1794 records). [2025-10-20T22:12:09.356474] [INFO ] [MAIN ] MRK: Successfully saved derivative data. [2025-10-20T22:12:09.393192] [INFO ] [MAIN ] Derivative Progress: 1195/1877 symbols completed. MRK took 1.0s. Estimated remaining: 4m 16s. Rate: 159.6 symbols/min [2025-10-20T22:12:09.426298] [INFO ] [MAIN ] MSB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:09.685580] [DEBUG] [LOCAL ] MSB: Loaded 7950 existing records from primary storage. [2025-10-20T22:12:09.799338] [INFO ] [MAIN ] MSB: Loaded 7950 primary data records. [2025-10-20T22:12:10.129368] [DEBUG] [MAIN ] MSA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:10.131412] [INFO ] [MAIN ] MSA: Generated 1794 derivative records from 18915 primary records. [2025-10-20T22:12:10.143561] [DEBUG] [LOCAL ] MSA: Saving derivative data. [2025-10-20T22:12:10.144157] [DEBUG] [MAIN ] MS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:10.145830] [INFO ] [MAIN ] MS: Generated 1794 derivative records from 53688 primary records. [2025-10-20T22:12:10.145921] [DEBUG] [LOCAL ] MS: Saving derivative data. [2025-10-20T22:12:10.169197] [DEBUG] [LOCAL ] MSA: Derivative data saved (1794 records). [2025-10-20T22:12:10.170020] [INFO ] [MAIN ] MSA: Successfully saved derivative data. [2025-10-20T22:12:10.172146] [INFO ] [MAIN ] Derivative Progress: 1196/1877 symbols completed. MSA took 1.1s. Estimated remaining: 4m 16s. Rate: 159.5 symbols/min [2025-10-20T22:12:10.177636] [INFO ] [MAIN ] MSCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:10.179932] [DEBUG] [LOCAL ] MS: Derivative data saved (1794 records). [2025-10-20T22:12:10.180449] [INFO ] [MAIN ] MS: Successfully saved derivative data. [2025-10-20T22:12:10.182371] [INFO ] [MAIN ] Derivative Progress: 1197/1877 symbols completed. MS took 1.4s. Estimated remaining: 4m 16s. Rate: 159.6 symbols/min [2025-10-20T22:12:10.182520] [INFO ] [MAIN ] MSD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:10.206250] [DEBUG] [MAIN ] MSB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:10.207290] [INFO ] [MAIN ] MSB: Generated 1794 derivative records from 7950 primary records. [2025-10-20T22:12:10.215900] [DEBUG] [LOCAL ] MSB: Saving derivative data. [2025-10-20T22:12:10.237751] [DEBUG] [LOCAL ] MSB: Derivative data saved (1794 records). [2025-10-20T22:12:10.238301] [INFO ] [MAIN ] MSB: Successfully saved derivative data. [2025-10-20T22:12:10.238208] [DEBUG] [LOCAL ] MSD: Loaded 16558 existing records from primary storage. [2025-10-20T22:12:10.240818] [INFO ] [MAIN ] MSD: Loaded 16558 primary data records. [2025-10-20T22:12:10.240723] [INFO ] [MAIN ] Derivative Progress: 1198/1877 symbols completed. MSB took 0.8s. Estimated remaining: 4m 15s. Rate: 159.7 symbols/min [2025-10-20T22:12:10.295283] [DEBUG] [LOCAL ] MSCI: Loaded 34427 existing records from primary storage. [2025-10-20T22:12:10.327122] [INFO ] [MAIN ] MSCI: Loaded 34427 primary data records. [2025-10-20T22:12:10.898106] [DEBUG] [MAIN ] MSD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:10.900354] [INFO ] [MAIN ] MSD: Generated 1794 derivative records from 16558 primary records. [2025-10-20T22:12:10.901077] [DEBUG] [LOCAL ] MSD: Saving derivative data. [2025-10-20T22:12:10.924808] [DEBUG] [MAIN ] MSCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:10.928452] [INFO ] [MAIN ] MSCI: Generated 1794 derivative records from 34427 primary records. [2025-10-20T22:12:10.929504] [DEBUG] [LOCAL ] MSD: Derivative data saved (1794 records). [2025-10-20T22:12:10.929696] [DEBUG] [LOCAL ] MSCI: Saving derivative data. [2025-10-20T22:12:10.929617] [INFO ] [MAIN ] MSD: Successfully saved derivative data. [2025-10-20T22:12:10.932339] [INFO ] [MAIN ] Derivative Progress: 1199/1877 symbols completed. MSD took 0.7s. Estimated remaining: 4m 15s. Rate: 159.6 symbols/min [2025-10-20T22:12:10.947435] [DEBUG] [LOCAL ] MSCI: Derivative data saved (1794 records). [2025-10-20T22:12:10.947532] [INFO ] [MAIN ] MSCI: Successfully saved derivative data. [2025-10-20T22:12:10.949698] [INFO ] [MAIN ] Derivative Progress: 1200/1877 symbols completed. MSCI took 0.8s. Estimated remaining: 4m 14s. Rate: 159.7 symbols/min [2025-10-20T22:12:10.950316] [INFO ] [MAIN ] MSGE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:10.950242] [INFO ] [MAIN ] MSDL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:10.950468] [INFO ] [MAIN ] MSGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:10.985100] [DEBUG] [LOCAL ] MSGS: Loaded 17291 existing records from primary storage. [2025-10-20T22:12:10.985586] [DEBUG] [LOCAL ] MSGE: Loaded 27014 existing records from primary storage. [2025-10-20T22:12:10.985706] [INFO ] [MAIN ] MSGS: Loaded 17291 primary data records. [2025-10-20T22:12:11.006568] [INFO ] [MAIN ] MSGE: Loaded 27014 primary data records. [2025-10-20T22:12:11.187866] [DEBUG] [LOCAL ] MSDL: Loaded 38684 existing records from primary storage. [2025-10-20T22:12:11.209049] [INFO ] [MAIN ] MSDL: Loaded 38684 primary data records. [2025-10-20T22:12:11.952177] [DEBUG] [MAIN ] MSGE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:11.954471] [INFO ] [MAIN ] MSGE: Generated 1794 derivative records from 27014 primary records. [2025-10-20T22:12:11.954700] [DEBUG] [LOCAL ] MSGE: Saving derivative data. [2025-10-20T22:12:11.953169] [DEBUG] [MAIN ] MSGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:11.956431] [INFO ] [MAIN ] MSGS: Generated 1794 derivative records from 17291 primary records. [2025-10-20T22:12:11.956681] [DEBUG] [LOCAL ] MSGS: Saving derivative data. [2025-10-20T22:12:11.985689] [DEBUG] [LOCAL ] MSGE: Derivative data saved (1794 records). [2025-10-20T22:12:11.987032] [INFO ] [MAIN ] MSGE: Successfully saved derivative data. [2025-10-20T22:12:11.989340] [INFO ] [MAIN ] Derivative Progress: 1201/1877 symbols completed. MSGE took 1.0s. Estimated remaining: 4m 14s. Rate: 159.5 symbols/min [2025-10-20T22:12:11.989494] [INFO ] [MAIN ] MSI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:11.991469] [DEBUG] [LOCAL ] MSGS: Derivative data saved (1794 records). [2025-10-20T22:12:11.991665] [INFO ] [MAIN ] MSGS: Successfully saved derivative data. [2025-10-20T22:12:11.993944] [INFO ] [MAIN ] Derivative Progress: 1202/1877 symbols completed. MSGS took 1.0s. Estimated remaining: 4m 14s. Rate: 159.6 symbols/min [2025-10-20T22:12:11.994234] [INFO ] [MAIN ] MSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:12.011515] [DEBUG] [MAIN ] MSDL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:12.012334] [INFO ] [MAIN ] MSDL: Generated 1794 derivative records from 38684 primary records. [2025-10-20T22:12:12.012505] [DEBUG] [LOCAL ] MSDL: Saving derivative data. [2025-10-20T22:12:12.028952] [DEBUG] [LOCAL ] MSDL: Derivative data saved (1794 records). [2025-10-20T22:12:12.029238] [INFO ] [MAIN ] MSDL: Successfully saved derivative data. [2025-10-20T22:12:12.031470] [INFO ] [MAIN ] Derivative Progress: 1203/1877 symbols completed. MSDL took 1.1s. Estimated remaining: 4m 13s. Rate: 159.7 symbols/min [2025-10-20T22:12:12.031862] [INFO ] [MAIN ] MT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:12.077722] [DEBUG] [LOCAL ] MSI: Loaded 39863 existing records from primary storage. [2025-10-20T22:12:12.078429] [DEBUG] [LOCAL ] MSM: Loaded 35537 existing records from primary storage. [2025-10-20T22:12:12.078584] [INFO ] [MAIN ] MSI: Loaded 39863 primary data records. [2025-10-20T22:12:12.079006] [INFO ] [MAIN ] MSM: Loaded 35537 primary data records. [2025-10-20T22:12:12.500800] [DEBUG] [LOCAL ] MT: Loaded 50451 existing records from primary storage. [2025-10-20T22:12:12.662736] [INFO ] [MAIN ] MT: Loaded 50451 primary data records. [2025-10-20T22:12:13.096123] [DEBUG] [MAIN ] MSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:13.095822] [DEBUG] [MAIN ] MSI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:12:13.098066] [INFO ] [MAIN ] MSM: Generated 1794 derivative records from 35537 primary records. [2025-10-20T22:12:13.097040] [INFO ] [MAIN ] MSI: Generated 1793 derivative records from 39863 primary records. [2025-10-20T22:12:13.098202] [DEBUG] [LOCAL ] MSI: Saving derivative data. [2025-10-20T22:12:13.099900] [DEBUG] [LOCAL ] MSM: Saving derivative data. [2025-10-20T22:12:13.130463] [DEBUG] [LOCAL ] MSI: Derivative data saved (1793 records). [2025-10-20T22:12:13.131945] [DEBUG] [LOCAL ] MSM: Derivative data saved (1794 records). [2025-10-20T22:12:13.130809] [INFO ] [MAIN ] MSI: Successfully saved derivative data. [2025-10-20T22:12:13.132091] [INFO ] [MAIN ] MSM: Successfully saved derivative data. [2025-10-20T22:12:13.134323] [INFO ] [MAIN ] Derivative Progress: 1204/1877 symbols completed. MSI took 1.1s. Estimated remaining: 4m 13s. Rate: 159.5 symbols/min [2025-10-20T22:12:13.136596] [INFO ] [MAIN ] Derivative Progress: 1205/1877 symbols completed. MSM took 1.1s. Estimated remaining: 4m 13s. Rate: 159.6 symbols/min [2025-10-20T22:12:13.136956] [INFO ] [MAIN ] MTD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:13.134683] [INFO ] [MAIN ] MTB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:13.154990] [DEBUG] [MAIN ] MT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:13.155837] [INFO ] [MAIN ] MT: Generated 1794 derivative records from 50451 primary records. [2025-10-20T22:12:13.159805] [DEBUG] [LOCAL ] MT: Saving derivative data. [2025-10-20T22:12:13.183245] [DEBUG] [LOCAL ] MT: Derivative data saved (1794 records). [2025-10-20T22:12:13.183624] [DEBUG] [LOCAL ] MTD: Loaded 15324 existing records from primary storage. [2025-10-20T22:12:13.183850] [INFO ] [MAIN ] MT: Successfully saved derivative data. [2025-10-20T22:12:13.186005] [INFO ] [MAIN ] Derivative Progress: 1206/1877 symbols completed. MT took 1.2s. Estimated remaining: 4m 12s. Rate: 159.7 symbols/min [2025-10-20T22:12:13.186268] [INFO ] [MAIN ] MTDR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:13.186097] [INFO ] [MAIN ] MTD: Loaded 15324 primary data records. [2025-10-20T22:12:13.391264] [DEBUG] [LOCAL ] MTB: Loaded 46699 existing records from primary storage. [2025-10-20T22:12:13.391700] [INFO ] [MAIN ] MTB: Loaded 46699 primary data records. [2025-10-20T22:12:13.656526] [DEBUG] [LOCAL ] MTDR: Loaded 51765 existing records from primary storage. [2025-10-20T22:12:13.667905] [INFO ] [MAIN ] MTDR: Loaded 51765 primary data records. [2025-10-20T22:12:14.221413] [DEBUG] [MAIN ] MTD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:14.222720] [INFO ] [MAIN ] MTD: Generated 1794 derivative records from 15324 primary records. [2025-10-20T22:12:14.223322] [DEBUG] [LOCAL ] MTD: Saving derivative data. [2025-10-20T22:12:14.242226] [DEBUG] [LOCAL ] MTD: Derivative data saved (1794 records). [2025-10-20T22:12:14.244855] [INFO ] [MAIN ] MTD: Successfully saved derivative data. [2025-10-20T22:12:14.246974] [INFO ] [MAIN ] Derivative Progress: 1207/1877 symbols completed. MTD took 1.1s. Estimated remaining: 4m 12s. Rate: 159.5 symbols/min [2025-10-20T22:12:14.247530] [INFO ] [MAIN ] MTG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:14.262099] [DEBUG] [MAIN ] MTB: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:12:14.263024] [INFO ] [MAIN ] MTB: Generated 1793 derivative records from 46699 primary records. [2025-10-20T22:12:14.263329] [DEBUG] [LOCAL ] MTB: Saving derivative data. [2025-10-20T22:12:14.283302] [DEBUG] [LOCAL ] MTB: Derivative data saved (1793 records). [2025-10-20T22:12:14.283800] [INFO ] [MAIN ] MTB: Successfully saved derivative data. [2025-10-20T22:12:14.286128] [INFO ] [MAIN ] Derivative Progress: 1208/1877 symbols completed. MTB took 1.2s. Estimated remaining: 4m 11s. Rate: 159.6 symbols/min [2025-10-20T22:12:14.298881] [INFO ] [MAIN ] MTH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:14.299566] [DEBUG] [MAIN ] MTDR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:14.300489] [INFO ] [MAIN ] MTDR: Generated 1794 derivative records from 51765 primary records. [2025-10-20T22:12:14.300680] [DEBUG] [LOCAL ] MTDR: Saving derivative data. [2025-10-20T22:12:14.331037] [DEBUG] [LOCAL ] MTDR: Derivative data saved (1794 records). [2025-10-20T22:12:14.331171] [INFO ] [MAIN ] MTDR: Successfully saved derivative data. [2025-10-20T22:12:14.333454] [INFO ] [MAIN ] Derivative Progress: 1209/1877 symbols completed. MTDR took 1.1s. Estimated remaining: 4m 11s. Rate: 159.7 symbols/min [2025-10-20T22:12:14.333825] [INFO ] [MAIN ] MTN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:14.352925] [DEBUG] [LOCAL ] MTG: Loaded 51055 existing records from primary storage. [2025-10-20T22:12:14.353281] [INFO ] [MAIN ] MTG: Loaded 51055 primary data records. [2025-10-20T22:12:14.694401] [DEBUG] [LOCAL ] MTH: Loaded 43889 existing records from primary storage. [2025-10-20T22:12:14.694786] [INFO ] [MAIN ] MTH: Loaded 43889 primary data records. [2025-10-20T22:12:14.810463] [DEBUG] [LOCAL ] MTN: Loaded 39520 existing records from primary storage. [2025-10-20T22:12:14.826674] [INFO ] [MAIN ] MTN: Loaded 39520 primary data records. [2025-10-20T22:12:15.351854] [DEBUG] [MAIN ] MTG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:15.353382] [INFO ] [MAIN ] MTG: Generated 1794 derivative records from 51055 primary records. [2025-10-20T22:12:15.353490] [DEBUG] [LOCAL ] MTG: Saving derivative data. [2025-10-20T22:12:15.371479] [DEBUG] [LOCAL ] MTG: Derivative data saved (1794 records). [2025-10-20T22:12:15.371638] [INFO ] [MAIN ] MTG: Successfully saved derivative data. [2025-10-20T22:12:15.373882] [INFO ] [MAIN ] Derivative Progress: 1210/1877 symbols completed. MTG took 1.1s. Estimated remaining: 4m 11s. Rate: 159.5 symbols/min [2025-10-20T22:12:15.374150] [INFO ] [MAIN ] MTUS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:15.393520] [DEBUG] [MAIN ] MTH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:15.394275] [INFO ] [MAIN ] MTH: Generated 1794 derivative records from 43889 primary records. [2025-10-20T22:12:15.394443] [DEBUG] [LOCAL ] MTH: Saving derivative data. [2025-10-20T22:12:15.418755] [DEBUG] [LOCAL ] MTH: Derivative data saved (1794 records). [2025-10-20T22:12:15.418961] [INFO ] [MAIN ] MTH: Successfully saved derivative data. [2025-10-20T22:12:15.420918] [INFO ] [MAIN ] Derivative Progress: 1211/1877 symbols completed. MTH took 1.1s. Estimated remaining: 4m 10s. Rate: 159.6 symbols/min [2025-10-20T22:12:15.421078] [INFO ] [MAIN ] MTW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:15.439491] [DEBUG] [MAIN ] MTN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:15.440686] [INFO ] [MAIN ] MTN: Generated 1794 derivative records from 39520 primary records. [2025-10-20T22:12:15.448577] [DEBUG] [LOCAL ] MTN: Saving derivative data. [2025-10-20T22:12:15.450830] [DEBUG] [LOCAL ] MTUS: Loaded 27547 existing records from primary storage. [2025-10-20T22:12:15.457278] [INFO ] [MAIN ] MTUS: Loaded 27547 primary data records. [2025-10-20T22:12:15.517810] [DEBUG] [LOCAL ] MTW: Loaded 26563 existing records from primary storage. [2025-10-20T22:12:15.531671] [INFO ] [MAIN ] MTW: Loaded 26563 primary data records. [2025-10-20T22:12:15.526328] [DEBUG] [LOCAL ] MTN: Derivative data saved (1794 records). [2025-10-20T22:12:15.641134] [INFO ] [MAIN ] MTN: Successfully saved derivative data. [2025-10-20T22:12:15.671858] [INFO ] [MAIN ] Derivative Progress: 1212/1877 symbols completed. MTN took 1.3s. Estimated remaining: 4m 10s. Rate: 159.7 symbols/min [2025-10-20T22:12:15.691837] [INFO ] [MAIN ] MTX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:16.090224] [DEBUG] [LOCAL ] MTX: Loaded 22705 existing records from primary storage. [2025-10-20T22:12:16.090750] [INFO ] [MAIN ] MTX: Loaded 22705 primary data records. [2025-10-20T22:12:16.428388] [DEBUG] [MAIN ] MTUS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:16.429815] [INFO ] [MAIN ] MTUS: Generated 1794 derivative records from 27547 primary records. [2025-10-20T22:12:16.428878] [DEBUG] [MAIN ] MTW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:16.431574] [INFO ] [MAIN ] MTW: Generated 1794 derivative records from 26563 primary records. [2025-10-20T22:12:16.431817] [DEBUG] [LOCAL ] MTW: Saving derivative data. [2025-10-20T22:12:16.430845] [DEBUG] [LOCAL ] MTUS: Saving derivative data. [2025-10-20T22:12:16.464332] [DEBUG] [LOCAL ] MTW: Derivative data saved (1794 records). [2025-10-20T22:12:16.464551] [INFO ] [MAIN ] MTW: Successfully saved derivative data. [2025-10-20T22:12:16.466711] [INFO ] [MAIN ] Derivative Progress: 1213/1877 symbols completed. MTW took 1.0s. Estimated remaining: 4m 10s. Rate: 159.5 symbols/min [2025-10-20T22:12:16.466899] [DEBUG] [LOCAL ] MTUS: Derivative data saved (1794 records). [2025-10-20T22:12:16.467100] [INFO ] [MAIN ] MTUS: Successfully saved derivative data. [2025-10-20T22:12:16.469244] [INFO ] [MAIN ] Derivative Progress: 1214/1877 symbols completed. MTUS took 1.1s. Estimated remaining: 4m 9s. Rate: 159.6 symbols/min [2025-10-20T22:12:16.484517] [DEBUG] [MAIN ] MTX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:16.485077] [INFO ] [MAIN ] MTX: Generated 1794 derivative records from 22705 primary records. [2025-10-20T22:12:16.485138] [DEBUG] [LOCAL ] MTX: Saving derivative data. [2025-10-20T22:12:16.500009] [DEBUG] [LOCAL ] MTX: Derivative data saved (1794 records). [2025-10-20T22:12:16.500065] [INFO ] [MAIN ] MTX: Successfully saved derivative data. [2025-10-20T22:12:16.502177] [INFO ] [MAIN ] Derivative Progress: 1215/1877 symbols completed. MTX took 0.8s. Estimated remaining: 4m 9s. Rate: 159.8 symbols/min [2025-10-20T22:12:16.502544] [INFO ] [MAIN ] MTZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:16.502744] [INFO ] [MAIN ] MUC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:16.502683] [INFO ] [MAIN ] MUA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:16.522379] [DEBUG] [LOCAL ] MUA: Loaded 11500 existing records from primary storage. [2025-10-20T22:12:16.522885] [INFO ] [MAIN ] MUA: Loaded 11500 primary data records. [2025-10-20T22:12:16.584456] [DEBUG] [LOCAL ] MUC: Loaded 20776 existing records from primary storage. [2025-10-20T22:12:16.604555] [INFO ] [MAIN ] MUC: Loaded 20776 primary data records. [2025-10-20T22:12:16.828489] [DEBUG] [LOCAL ] MTZ: Loaded 42828 existing records from primary storage. [2025-10-20T22:12:16.970713] [INFO ] [MAIN ] MTZ: Loaded 42828 primary data records. [2025-10-20T22:12:17.485614] [DEBUG] [MAIN ] MUA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:17.487173] [INFO ] [MAIN ] MUA: Generated 1794 derivative records from 11500 primary records. [2025-10-20T22:12:17.489567] [DEBUG] [LOCAL ] MUA: Saving derivative data. [2025-10-20T22:12:17.516001] [DEBUG] [MAIN ] MUC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:17.518710] [INFO ] [MAIN ] MUC: Generated 1794 derivative records from 20776 primary records. [2025-10-20T22:12:17.519070] [DEBUG] [LOCAL ] MUC: Saving derivative data. [2025-10-20T22:12:17.520114] [DEBUG] [LOCAL ] MUA: Derivative data saved (1794 records). [2025-10-20T22:12:17.521054] [INFO ] [MAIN ] MUA: Successfully saved derivative data. [2025-10-20T22:12:17.523032] [INFO ] [MAIN ] Derivative Progress: 1216/1877 symbols completed. MUA took 1.0s. Estimated remaining: 4m 9s. Rate: 159.5 symbols/min [2025-10-20T22:12:17.523267] [INFO ] [MAIN ] MUE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:17.540758] [DEBUG] [LOCAL ] MUC: Derivative data saved (1794 records). [2025-10-20T22:12:17.541531] [INFO ] [MAIN ] MUC: Successfully saved derivative data. [2025-10-20T22:12:17.543742] [INFO ] [MAIN ] Derivative Progress: 1217/1877 symbols completed. MUC took 1.0s. Estimated remaining: 4m 8s. Rate: 159.7 symbols/min [2025-10-20T22:12:17.544154] [INFO ] [MAIN ] MUFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:17.557873] [DEBUG] [MAIN ] MTZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:17.558601] [INFO ] [MAIN ] MTZ: Generated 1794 derivative records from 42828 primary records. [2025-10-20T22:12:17.561730] [DEBUG] [LOCAL ] MTZ: Saving derivative data. [2025-10-20T22:12:17.583204] [DEBUG] [LOCAL ] MTZ: Derivative data saved (1794 records). [2025-10-20T22:12:17.583338] [INFO ] [MAIN ] MTZ: Successfully saved derivative data. [2025-10-20T22:12:17.583666] [DEBUG] [LOCAL ] MUE: Loaded 12237 existing records from primary storage. [2025-10-20T22:12:17.583850] [INFO ] [MAIN ] MUE: Loaded 12237 primary data records. [2025-10-20T22:12:17.586176] [INFO ] [MAIN ] Derivative Progress: 1218/1877 symbols completed. MTZ took 1.1s. Estimated remaining: 4m 7s. Rate: 159.8 symbols/min [2025-10-20T22:12:17.607540] [INFO ] [MAIN ] MUJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:17.794101] [DEBUG] [LOCAL ] MUJ: Loaded 13886 existing records from primary storage. [2025-10-20T22:12:17.922236] [INFO ] [MAIN ] MUJ: Loaded 13886 primary data records. [2025-10-20T22:12:17.946220] [DEBUG] [LOCAL ] MUFG: Loaded 52417 existing records from primary storage. [2025-10-20T22:12:17.963295] [INFO ] [MAIN ] MUFG: Loaded 52417 primary data records. [2025-10-20T22:12:18.707606] [DEBUG] [MAIN ] MUE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:18.709248] [INFO ] [MAIN ] MUE: Generated 1794 derivative records from 12237 primary records. [2025-10-20T22:12:18.709475] [DEBUG] [LOCAL ] MUE: Saving derivative data. [2025-10-20T22:12:18.727429] [DEBUG] [LOCAL ] MUE: Derivative data saved (1794 records). [2025-10-20T22:12:18.729001] [INFO ] [MAIN ] MUE: Successfully saved derivative data. [2025-10-20T22:12:18.731523] [INFO ] [MAIN ] Derivative Progress: 1219/1877 symbols completed. MUE took 1.2s. Estimated remaining: 4m 8s. Rate: 159.5 symbols/min [2025-10-20T22:12:18.732616] [INFO ] [MAIN ] MUR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:18.754731] [DEBUG] [MAIN ] MUJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:18.756106] [INFO ] [MAIN ] MUJ: Generated 1794 derivative records from 13886 primary records. [2025-10-20T22:12:18.756296] [DEBUG] [LOCAL ] MUJ: Saving derivative data. [2025-10-20T22:12:18.783664] [DEBUG] [LOCAL ] MUJ: Derivative data saved (1794 records). [2025-10-20T22:12:18.790507] [INFO ] [MAIN ] MUJ: Successfully saved derivative data. [2025-10-20T22:12:18.817217] [INFO ] [MAIN ] Derivative Progress: 1220/1877 symbols completed. MUJ took 1.2s. Estimated remaining: 4m 7s. Rate: 159.6 symbols/min [2025-10-20T22:12:18.823828] [DEBUG] [MAIN ] MUFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:18.823412] [INFO ] [MAIN ] MUSA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:18.824817] [INFO ] [MAIN ] MUFG: Generated 1794 derivative records from 52417 primary records. [2025-10-20T22:12:18.825325] [DEBUG] [LOCAL ] MUFG: Saving derivative data. [2025-10-20T22:12:18.842994] [DEBUG] [LOCAL ] MUFG: Derivative data saved (1794 records). [2025-10-20T22:12:18.843400] [INFO ] [MAIN ] MUFG: Successfully saved derivative data. [2025-10-20T22:12:18.845776] [INFO ] [MAIN ] Derivative Progress: 1221/1877 symbols completed. MUFG took 1.3s. Estimated remaining: 4m 6s. Rate: 159.7 symbols/min [2025-10-20T22:12:18.863750] [INFO ] [MAIN ] MUX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:18.865414] [DEBUG] [LOCAL ] MUR: Loaded 52913 existing records from primary storage. [2025-10-20T22:12:18.865586] [INFO ] [MAIN ] MUR: Loaded 52913 primary data records. [2025-10-20T22:12:19.015060] [DEBUG] [LOCAL ] MUSA: Loaded 24721 existing records from primary storage. [2025-10-20T22:12:19.015319] [INFO ] [MAIN ] MUSA: Loaded 24721 primary data records. [2025-10-20T22:12:19.407406] [DEBUG] [LOCAL ] MUX: Loaded 43712 existing records from primary storage. [2025-10-20T22:12:19.413954] [INFO ] [MAIN ] MUX: Loaded 43712 primary data records. [2025-10-20T22:12:20.052136] [DEBUG] [MAIN ] MUR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:20.053829] [INFO ] [MAIN ] MUR: Generated 1794 derivative records from 52913 primary records. [2025-10-20T22:12:20.054202] [DEBUG] [LOCAL ] MUR: Saving derivative data. [2025-10-20T22:12:20.075392] [DEBUG] [LOCAL ] MUR: Derivative data saved (1794 records). [2025-10-20T22:12:20.075879] [INFO ] [MAIN ] MUR: Successfully saved derivative data. [2025-10-20T22:12:20.085712] [INFO ] [MAIN ] Derivative Progress: 1222/1877 symbols completed. MUR took 1.4s. Estimated remaining: 4m 7s. Rate: 159.4 symbols/min [2025-10-20T22:12:20.091516] [INFO ] [MAIN ] MVF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:20.091389] [DEBUG] [MAIN ] MUSA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:20.093594] [INFO ] [MAIN ] MUSA: Generated 1794 derivative records from 24721 primary records. [2025-10-20T22:12:20.094144] [DEBUG] [LOCAL ] MUSA: Saving derivative data. [2025-10-20T22:12:20.115095] [DEBUG] [LOCAL ] MUSA: Derivative data saved (1794 records). [2025-10-20T22:12:20.115258] [INFO ] [MAIN ] MUSA: Successfully saved derivative data. [2025-10-20T22:12:20.117584] [INFO ] [MAIN ] Derivative Progress: 1223/1877 symbols completed. MUSA took 1.3s. Estimated remaining: 4m 6s. Rate: 159.5 symbols/min [2025-10-20T22:12:20.117817] [INFO ] [MAIN ] MVO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:20.126111] [DEBUG] [LOCAL ] MVF: Loaded 12674 existing records from primary storage. [2025-10-20T22:12:20.127193] [INFO ] [MAIN ] MVF: Loaded 12674 primary data records. [2025-10-20T22:12:20.183391] [DEBUG] [MAIN ] MUX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:20.204399] [INFO ] [MAIN ] MUX: Generated 1794 derivative records from 43712 primary records. [2025-10-20T22:12:20.202313] [DEBUG] [LOCAL ] MVO: Loaded 8946 existing records from primary storage. [2025-10-20T22:12:20.204573] [DEBUG] [LOCAL ] MUX: Saving derivative data. [2025-10-20T22:12:20.204677] [INFO ] [MAIN ] MVO: Loaded 8946 primary data records. [2025-10-20T22:12:20.368988] [DEBUG] [LOCAL ] MUX: Derivative data saved (1794 records). [2025-10-20T22:12:20.382584] [INFO ] [MAIN ] MUX: Successfully saved derivative data. [2025-10-20T22:12:20.400283] [INFO ] [MAIN ] Derivative Progress: 1224/1877 symbols completed. MUX took 1.5s. Estimated remaining: 4m 6s. Rate: 159.6 symbols/min [2025-10-20T22:12:20.495362] [INFO ] [MAIN ] MVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:20.822378] [DEBUG] [LOCAL ] MVT: Loaded 9962 existing records from primary storage. [2025-10-20T22:12:20.822765] [INFO ] [MAIN ] MVT: Loaded 9962 primary data records. [2025-10-20T22:12:21.182465] [DEBUG] [MAIN ] MVF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:21.184118] [INFO ] [MAIN ] MVF: Generated 1794 derivative records from 12674 primary records. [2025-10-20T22:12:21.187486] [DEBUG] [LOCAL ] MVF: Saving derivative data. [2025-10-20T22:12:21.209542] [DEBUG] [MAIN ] MVO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:21.217542] [INFO ] [MAIN ] MVO: Generated 1794 derivative records from 8946 primary records. [2025-10-20T22:12:21.219429] [DEBUG] [LOCAL ] MVO: Saving derivative data. [2025-10-20T22:12:21.219369] [DEBUG] [LOCAL ] MVF: Derivative data saved (1794 records). [2025-10-20T22:12:21.220415] [INFO ] [MAIN ] MVF: Successfully saved derivative data. [2025-10-20T22:12:21.222352] [INFO ] [MAIN ] Derivative Progress: 1225/1877 symbols completed. MVF took 1.1s. Estimated remaining: 4m 5s. Rate: 159.4 symbols/min [2025-10-20T22:12:21.222669] [INFO ] [MAIN ] MWA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:21.238321] [DEBUG] [LOCAL ] MVO: Derivative data saved (1794 records). [2025-10-20T22:12:21.238592] [INFO ] [MAIN ] MVO: Successfully saved derivative data. [2025-10-20T22:12:21.240817] [INFO ] [MAIN ] Derivative Progress: 1226/1877 symbols completed. MVO took 1.1s. Estimated remaining: 4m 5s. Rate: 159.6 symbols/min [2025-10-20T22:12:21.241270] [INFO ] [MAIN ] MXF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:21.261920] [DEBUG] [MAIN ] MVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:21.265716] [DEBUG] [LOCAL ] MXF: Loaded 5568 existing records from primary storage. [2025-10-20T22:12:21.262618] [INFO ] [MAIN ] MVT: Generated 1794 derivative records from 9962 primary records. [2025-10-20T22:12:21.265854] [INFO ] [MAIN ] MXF: Loaded 5568 primary data records. [2025-10-20T22:12:21.265925] [DEBUG] [LOCAL ] MVT: Saving derivative data. [2025-10-20T22:12:21.457125] [DEBUG] [LOCAL ] MWA: Loaded 45836 existing records from primary storage. [2025-10-20T22:12:21.457369] [DEBUG] [LOCAL ] MVT: Derivative data saved (1794 records). [2025-10-20T22:12:21.482163] [INFO ] [MAIN ] MWA: Loaded 45836 primary data records. [2025-10-20T22:12:21.487782] [INFO ] [MAIN ] MVT: Successfully saved derivative data. [2025-10-20T22:12:21.595601] [INFO ] [MAIN ] Derivative Progress: 1227/1877 symbols completed. MVT took 1.1s. Estimated remaining: 4m 4s. Rate: 159.6 symbols/min [2025-10-20T22:12:21.651244] [INFO ] [MAIN ] MYD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:21.801631] [DEBUG] [LOCAL ] MYD: Loaded 13274 existing records from primary storage. [2025-10-20T22:12:21.960156] [INFO ] [MAIN ] MYD: Loaded 13274 primary data records. [2025-10-20T22:12:22.305030] [DEBUG] [MAIN ] MXF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:22.315887] [INFO ] [MAIN ] MXF: Generated 1794 derivative records from 5568 primary records. [2025-10-20T22:12:22.316141] [DEBUG] [LOCAL ] MXF: Saving derivative data. [2025-10-20T22:12:22.333863] [DEBUG] [LOCAL ] MXF: Derivative data saved (1794 records). [2025-10-20T22:12:22.335542] [INFO ] [MAIN ] MXF: Successfully saved derivative data. [2025-10-20T22:12:22.338292] [INFO ] [MAIN ] Derivative Progress: 1228/1877 symbols completed. MXF took 1.1s. Estimated remaining: 4m 4s. Rate: 159.4 symbols/min [2025-10-20T22:12:22.353274] [DEBUG] [MAIN ] MWA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:22.354872] [INFO ] [MAIN ] MWA: Generated 1794 derivative records from 45836 primary records. [2025-10-20T22:12:22.354976] [DEBUG] [LOCAL ] MWA: Saving derivative data. [2025-10-20T22:12:22.371075] [DEBUG] [LOCAL ] MWA: Derivative data saved (1794 records). [2025-10-20T22:12:22.371292] [INFO ] [MAIN ] MWA: Successfully saved derivative data. [2025-10-20T22:12:22.373443] [INFO ] [MAIN ] Derivative Progress: 1229/1877 symbols completed. MWA took 1.2s. Estimated remaining: 4m 4s. Rate: 159.5 symbols/min [2025-10-20T22:12:22.389187] [DEBUG] [MAIN ] MYD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:22.389763] [INFO ] [MAIN ] MYD: Generated 1794 derivative records from 13274 primary records. [2025-10-20T22:12:22.389823] [DEBUG] [LOCAL ] MYD: Saving derivative data. [2025-10-20T22:12:22.404306] [DEBUG] [LOCAL ] MYD: Derivative data saved (1794 records). [2025-10-20T22:12:22.404366] [INFO ] [MAIN ] MYD: Successfully saved derivative data. [2025-10-20T22:12:22.406801] [INFO ] [MAIN ] Derivative Progress: 1230/1877 symbols completed. MYD took 0.8s. Estimated remaining: 4m 3s. Rate: 159.7 symbols/min [2025-10-20T22:12:22.407288] [INFO ] [MAIN ] MYE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:22.407384] [INFO ] [MAIN ] MYN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:22.408133] [INFO ] [MAIN ] MYI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:22.422334] [DEBUG] [LOCAL ] MYN: Loaded 9964 existing records from primary storage. [2025-10-20T22:12:22.422553] [INFO ] [MAIN ] MYN: Loaded 9964 primary data records. [2025-10-20T22:12:22.554224] [DEBUG] [LOCAL ] MYI: Loaded 18937 existing records from primary storage. [2025-10-20T22:12:22.558309] [INFO ] [MAIN ] MYI: Loaded 18937 primary data records. [2025-10-20T22:12:22.657181] [DEBUG] [LOCAL ] MYE: Loaded 24224 existing records from primary storage. [2025-10-20T22:12:22.682767] [INFO ] [MAIN ] MYE: Loaded 24224 primary data records. [2025-10-20T22:12:23.437382] [DEBUG] [MAIN ] MYN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:23.438466] [INFO ] [MAIN ] MYN: Generated 1794 derivative records from 9964 primary records. [2025-10-20T22:12:23.438846] [DEBUG] [LOCAL ] MYN: Saving derivative data. [2025-10-20T22:12:23.459012] [DEBUG] [LOCAL ] MYN: Derivative data saved (1794 records). [2025-10-20T22:12:23.459366] [INFO ] [MAIN ] MYN: Successfully saved derivative data. [2025-10-20T22:12:23.474414] [INFO ] [MAIN ] Derivative Progress: 1231/1877 symbols completed. MYN took 1.1s. Estimated remaining: 4m 3s. Rate: 159.4 symbols/min [2025-10-20T22:12:23.475671] [INFO ] [MAIN ] NABL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:23.476969] [DEBUG] [MAIN ] MYI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:23.477967] [INFO ] [MAIN ] MYI: Generated 1794 derivative records from 18937 primary records. [2025-10-20T22:12:23.478205] [DEBUG] [LOCAL ] MYI: Saving derivative data. [2025-10-20T22:12:23.490888] [DEBUG] [MAIN ] MYE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:23.492883] [INFO ] [MAIN ] MYE: Generated 1794 derivative records from 24224 primary records. [2025-10-20T22:12:23.499970] [DEBUG] [LOCAL ] MYE: Saving derivative data. [2025-10-20T22:12:23.521189] [DEBUG] [LOCAL ] MYI: Derivative data saved (1794 records). [2025-10-20T22:12:23.524334] [INFO ] [MAIN ] MYI: Successfully saved derivative data. [2025-10-20T22:12:23.524240] [DEBUG] [LOCAL ] MYE: Derivative data saved (1794 records). [2025-10-20T22:12:23.526840] [INFO ] [MAIN ] Derivative Progress: 1232/1877 symbols completed. MYI took 1.1s. Estimated remaining: 4m 3s. Rate: 159.5 symbols/min [2025-10-20T22:12:23.527175] [INFO ] [MAIN ] MYE: Successfully saved derivative data. [2025-10-20T22:12:23.529566] [INFO ] [MAIN ] Derivative Progress: 1233/1877 symbols completed. MYE took 1.1s. Estimated remaining: 4m 2s. Rate: 159.7 symbols/min [2025-10-20T22:12:23.527306] [INFO ] [MAIN ] NAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:23.529756] [INFO ] [MAIN ] NAD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:23.581335] [DEBUG] [LOCAL ] NAC: Loaded 24479 existing records from primary storage. [2025-10-20T22:12:23.582375] [DEBUG] [LOCAL ] NABL: Loaded 39071 existing records from primary storage. [2025-10-20T22:12:23.582488] [INFO ] [MAIN ] NABL: Loaded 39071 primary data records. [2025-10-20T22:12:23.581496] [INFO ] [MAIN ] NAC: Loaded 24479 primary data records. [2025-10-20T22:12:23.807458] [DEBUG] [LOCAL ] NAD: Loaded 33743 existing records from primary storage. [2025-10-20T22:12:23.825287] [INFO ] [MAIN ] NAD: Loaded 33743 primary data records. [2025-10-20T22:12:24.643925] [DEBUG] [MAIN ] NAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:24.645349] [INFO ] [MAIN ] NAC: Generated 1794 derivative records from 24479 primary records. [2025-10-20T22:12:24.644423] [DEBUG] [MAIN ] NABL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:24.646329] [INFO ] [MAIN ] NABL: Generated 1794 derivative records from 39071 primary records. [2025-10-20T22:12:24.646535] [DEBUG] [LOCAL ] NAC: Saving derivative data. [2025-10-20T22:12:24.646804] [DEBUG] [LOCAL ] NABL: Saving derivative data. [2025-10-20T22:12:24.680539] [DEBUG] [LOCAL ] NAC: Derivative data saved (1794 records). [2025-10-20T22:12:24.689163] [INFO ] [MAIN ] NAC: Successfully saved derivative data. [2025-10-20T22:12:24.689241] [DEBUG] [LOCAL ] NABL: Derivative data saved (1794 records). [2025-10-20T22:12:24.696253] [INFO ] [MAIN ] NABL: Successfully saved derivative data. [2025-10-20T22:12:24.696141] [INFO ] [MAIN ] Derivative Progress: 1234/1877 symbols completed. NAC took 1.2s. Estimated remaining: 4m 2s. Rate: 159.4 symbols/min [2025-10-20T22:12:24.699077] [INFO ] [MAIN ] NAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:24.698947] [DEBUG] [MAIN ] NAD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:24.700500] [INFO ] [MAIN ] NAD: Generated 1794 derivative records from 33743 primary records. [2025-10-20T22:12:24.699186] [INFO ] [MAIN ] Derivative Progress: 1235/1877 symbols completed. NABL took 1.2s. Estimated remaining: 4m 1s. Rate: 159.5 symbols/min [2025-10-20T22:12:24.700731] [INFO ] [MAIN ] NAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:24.700813] [DEBUG] [LOCAL ] NAD: Saving derivative data. [2025-10-20T22:12:24.721000] [DEBUG] [LOCAL ] NAD: Derivative data saved (1794 records). [2025-10-20T22:12:24.721208] [INFO ] [MAIN ] NAD: Successfully saved derivative data. [2025-10-20T22:12:24.723306] [INFO ] [MAIN ] Derivative Progress: 1236/1877 symbols completed. NAD took 1.2s. Estimated remaining: 4m 1s. Rate: 159.6 symbols/min [2025-10-20T22:12:24.723547] [INFO ] [MAIN ] NATL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:24.728629] [DEBUG] [LOCAL ] NAN: Loaded 8464 existing records from primary storage. [2025-10-20T22:12:24.728846] [INFO ] [MAIN ] NAN: Loaded 8464 primary data records. [2025-10-20T22:12:25.095259] [DEBUG] [LOCAL ] NAT: Loaded 49239 existing records from primary storage. [2025-10-20T22:12:25.095805] [DEBUG] [LOCAL ] NATL: Loaded 32454 existing records from primary storage. [2025-10-20T22:12:25.097226] [INFO ] [MAIN ] NATL: Loaded 32454 primary data records. [2025-10-20T22:12:25.097148] [INFO ] [MAIN ] NAT: Loaded 49239 primary data records. [2025-10-20T22:12:25.745335] [DEBUG] [MAIN ] NAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:25.747088] [INFO ] [MAIN ] NAN: Generated 1794 derivative records from 8464 primary records. [2025-10-20T22:12:25.747682] [DEBUG] [LOCAL ] NAN: Saving derivative data. [2025-10-20T22:12:25.767350] [DEBUG] [LOCAL ] NAN: Derivative data saved (1794 records). [2025-10-20T22:12:25.767486] [INFO ] [MAIN ] NAN: Successfully saved derivative data. [2025-10-20T22:12:25.769609] [INFO ] [MAIN ] Derivative Progress: 1237/1877 symbols completed. NAN took 1.1s. Estimated remaining: 4m 1s. Rate: 159.4 symbols/min [2025-10-20T22:12:25.769825] [INFO ] [MAIN ] NAZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:25.777593] [DEBUG] [LOCAL ] NAZ: Loaded 2677 existing records from primary storage. [2025-10-20T22:12:25.778234] [INFO ] [MAIN ] NAZ: Loaded 2677 primary data records. [2025-10-20T22:12:25.920918] [DEBUG] [MAIN ] NATL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:25.927973] [INFO ] [MAIN ] NATL: Generated 1794 derivative records from 32454 primary records. [2025-10-20T22:12:25.933604] [DEBUG] [MAIN ] NAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:25.944028] [DEBUG] [LOCAL ] NATL: Saving derivative data. [2025-10-20T22:12:25.963332] [INFO ] [MAIN ] NAT: Generated 1794 derivative records from 49239 primary records. [2025-10-20T22:12:25.963506] [DEBUG] [LOCAL ] NAT: Saving derivative data. [2025-10-20T22:12:26.014013] [DEBUG] [LOCAL ] NATL: Derivative data saved (1794 records). [2025-10-20T22:12:26.018110] [INFO ] [MAIN ] NATL: Successfully saved derivative data. [2025-10-20T22:12:26.124124] [INFO ] [MAIN ] Derivative Progress: 1238/1877 symbols completed. NATL took 1.4s. Estimated remaining: 4m 0s. Rate: 159.4 symbols/min [2025-10-20T22:12:26.139835] [INFO ] [MAIN ] NBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:26.149074] [DEBUG] [LOCAL ] NAT: Derivative data saved (1794 records). [2025-10-20T22:12:26.149503] [INFO ] [MAIN ] NAT: Successfully saved derivative data. [2025-10-20T22:12:26.152124] [INFO ] [MAIN ] Derivative Progress: 1239/1877 symbols completed. NAT took 1.5s. Estimated remaining: 3m 60s. Rate: 159.5 symbols/min [2025-10-20T22:12:26.152637] [INFO ] [MAIN ] NBHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:26.163328] [DEBUG] [LOCAL ] NBB: Loaded 9040 existing records from primary storage. [2025-10-20T22:12:26.163686] [INFO ] [MAIN ] NBB: Loaded 9040 primary data records. [2025-10-20T22:12:26.368847] [DEBUG] [LOCAL ] NBHC: Loaded 23824 existing records from primary storage. [2025-10-20T22:12:26.488182] [INFO ] [MAIN ] NBHC: Loaded 23824 primary data records. [2025-10-20T22:12:26.488810] [DEBUG] [MAIN ] NAZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:26.521830] [INFO ] [MAIN ] NAZ: Generated 1794 derivative records from 2677 primary records. [2025-10-20T22:12:26.522064] [DEBUG] [LOCAL ] NAZ: Saving derivative data. [2025-10-20T22:12:26.585033] [DEBUG] [LOCAL ] NAZ: Derivative data saved (1794 records). [2025-10-20T22:12:26.602094] [INFO ] [MAIN ] NAZ: Successfully saved derivative data. [2025-10-20T22:12:26.612270] [INFO ] [MAIN ] Derivative Progress: 1240/1877 symbols completed. NAZ took 0.8s. Estimated remaining: 3m 60s. Rate: 159.5 symbols/min [2025-10-20T22:12:26.612566] [INFO ] [MAIN ] NBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:26.850631] [DEBUG] [LOCAL ] NBR: Loaded 34623 existing records from primary storage. [2025-10-20T22:12:26.851338] [INFO ] [MAIN ] NBR: Loaded 34623 primary data records. [2025-10-20T22:12:26.882329] [DEBUG] [MAIN ] NBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:26.884125] [INFO ] [MAIN ] NBB: Generated 1794 derivative records from 9040 primary records. [2025-10-20T22:12:26.897631] [DEBUG] [LOCAL ] NBB: Saving derivative data. [2025-10-20T22:12:27.025238] [DEBUG] [LOCAL ] NBB: Derivative data saved (1794 records). [2025-10-20T22:12:27.045682] [INFO ] [MAIN ] NBB: Successfully saved derivative data. [2025-10-20T22:12:27.053308] [INFO ] [MAIN ] Derivative Progress: 1241/1877 symbols completed. NBB took 0.9s. Estimated remaining: 3m 59s. Rate: 159.5 symbols/min [2025-10-20T22:12:27.059011] [INFO ] [MAIN ] NBXG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:27.278076] [DEBUG] [MAIN ] NBHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:27.278857] [INFO ] [MAIN ] NBHC: Generated 1794 derivative records from 23824 primary records. [2025-10-20T22:12:27.279749] [DEBUG] [LOCAL ] NBHC: Saving derivative data. [2025-10-20T22:12:27.303380] [DEBUG] [LOCAL ] NBHC: Derivative data saved (1794 records). [2025-10-20T22:12:27.303835] [INFO ] [MAIN ] NBHC: Successfully saved derivative data. [2025-10-20T22:12:27.306171] [INFO ] [MAIN ] Derivative Progress: 1242/1877 symbols completed. NBHC took 1.2s. Estimated remaining: 3m 59s. Rate: 159.5 symbols/min [2025-10-20T22:12:27.314912] [INFO ] [MAIN ] NC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:27.315617] [DEBUG] [LOCAL ] NBXG: Loaded 21838 existing records from primary storage. [2025-10-20T22:12:27.315970] [INFO ] [MAIN ] NBXG: Loaded 21838 primary data records. [2025-10-20T22:12:27.387887] [DEBUG] [LOCAL ] NC: Loaded 2000 existing records from primary storage. [2025-10-20T22:12:27.412083] [INFO ] [MAIN ] NC: Loaded 2000 primary data records. [2025-10-20T22:12:27.934447] [DEBUG] [MAIN ] NBR: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:12:27.945143] [INFO ] [MAIN ] NBR: Generated 1793 derivative records from 34623 primary records. [2025-10-20T22:12:27.945484] [DEBUG] [LOCAL ] NBR: Saving derivative data. [2025-10-20T22:12:27.966634] [DEBUG] [LOCAL ] NBR: Derivative data saved (1793 records). [2025-10-20T22:12:27.967726] [INFO ] [MAIN ] NBR: Successfully saved derivative data. [2025-10-20T22:12:27.969820] [INFO ] [MAIN ] Derivative Progress: 1243/1877 symbols completed. NBR took 1.4s. Estimated remaining: 3m 59s. Rate: 159.4 symbols/min [2025-10-20T22:12:27.994852] [DEBUG] [MAIN ] NBXG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:27.996028] [INFO ] [MAIN ] NBXG: Generated 1794 derivative records from 21838 primary records. [2025-10-20T22:12:27.996168] [DEBUG] [LOCAL ] NBXG: Saving derivative data. [2025-10-20T22:12:28.010347] [DEBUG] [MAIN ] NC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:28.018524] [INFO ] [MAIN ] NC: Generated 1794 derivative records from 2000 primary records. [2025-10-20T22:12:28.022564] [DEBUG] [LOCAL ] NC: Saving derivative data. [2025-10-20T22:12:28.026893] [DEBUG] [LOCAL ] NBXG: Derivative data saved (1794 records). [2025-10-20T22:12:28.027788] [INFO ] [MAIN ] NBXG: Successfully saved derivative data. [2025-10-20T22:12:28.030350] [INFO ] [MAIN ] Derivative Progress: 1244/1877 symbols completed. NBXG took 1.0s. Estimated remaining: 3m 58s. Rate: 159.5 symbols/min [2025-10-20T22:12:28.043312] [DEBUG] [LOCAL ] NC: Derivative data saved (1794 records). [2025-10-20T22:12:28.043404] [INFO ] [MAIN ] NC: Successfully saved derivative data. [2025-10-20T22:12:28.045686] [INFO ] [MAIN ] Derivative Progress: 1245/1877 symbols completed. NC took 0.7s. Estimated remaining: 3m 57s. Rate: 159.7 symbols/min [2025-10-20T22:12:28.046187] [INFO ] [MAIN ] NCDL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:28.046253] [INFO ] [MAIN ] NCLH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:28.046117] [INFO ] [MAIN ] NCA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:28.064333] [DEBUG] [LOCAL ] NCA: Loaded 9920 existing records from primary storage. [2025-10-20T22:12:28.064469] [INFO ] [MAIN ] NCA: Loaded 9920 primary data records. [2025-10-20T22:12:28.184368] [DEBUG] [LOCAL ] NCDL: Loaded 21728 existing records from primary storage. [2025-10-20T22:12:28.199913] [INFO ] [MAIN ] NCDL: Loaded 21728 primary data records. [2025-10-20T22:12:28.541046] [DEBUG] [LOCAL ] NCLH: Loaded 53753 existing records from primary storage. [2025-10-20T22:12:28.552615] [INFO ] [MAIN ] NCLH: Loaded 53753 primary data records. [2025-10-20T22:12:29.119005] [DEBUG] [MAIN ] NCA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:29.120773] [INFO ] [MAIN ] NCA: Generated 1794 derivative records from 9920 primary records. [2025-10-20T22:12:29.121068] [DEBUG] [LOCAL ] NCA: Saving derivative data. [2025-10-20T22:12:29.138357] [DEBUG] [LOCAL ] NCA: Derivative data saved (1794 records). [2025-10-20T22:12:29.140716] [INFO ] [MAIN ] NCA: Successfully saved derivative data. [2025-10-20T22:12:29.142924] [INFO ] [MAIN ] Derivative Progress: 1246/1877 symbols completed. NCA took 1.1s. Estimated remaining: 3m 57s. Rate: 159.4 symbols/min [2025-10-20T22:12:29.143239] [INFO ] [MAIN ] NCV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:29.156568] [DEBUG] [MAIN ] NCDL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:29.157296] [INFO ] [MAIN ] NCDL: Generated 1794 derivative records from 21728 primary records. [2025-10-20T22:12:29.157552] [DEBUG] [LOCAL ] NCDL: Saving derivative data. [2025-10-20T22:12:29.173778] [DEBUG] [LOCAL ] NCDL: Derivative data saved (1794 records). [2025-10-20T22:12:29.173959] [INFO ] [MAIN ] NCDL: Successfully saved derivative data. [2025-10-20T22:12:29.176118] [INFO ] [MAIN ] Derivative Progress: 1247/1877 symbols completed. NCDL took 1.1s. Estimated remaining: 3m 57s. Rate: 159.5 symbols/min [2025-10-20T22:12:29.176461] [INFO ] [MAIN ] NCZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:29.203513] [DEBUG] [MAIN ] NCLH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:29.203187] [DEBUG] [LOCAL ] NCV: Loaded 13877 existing records from primary storage. [2025-10-20T22:12:29.207520] [INFO ] [MAIN ] NCLH: Generated 1794 derivative records from 53753 primary records. [2025-10-20T22:12:29.207855] [INFO ] [MAIN ] NCV: Loaded 13877 primary data records. [2025-10-20T22:12:29.207777] [DEBUG] [LOCAL ] NCLH: Saving derivative data. [2025-10-20T22:12:29.257483] [DEBUG] [LOCAL ] NCZ: Loaded 12277 existing records from primary storage. [2025-10-20T22:12:29.280185] [INFO ] [MAIN ] NCZ: Loaded 12277 primary data records. [2025-10-20T22:12:29.341146] [DEBUG] [LOCAL ] NCLH: Derivative data saved (1794 records). [2025-10-20T22:12:29.364831] [INFO ] [MAIN ] NCLH: Successfully saved derivative data. [2025-10-20T22:12:29.398314] [INFO ] [MAIN ] Derivative Progress: 1248/1877 symbols completed. NCLH took 1.4s. Estimated remaining: 3m 56s. Rate: 159.6 symbols/min [2025-10-20T22:12:29.436145] [INFO ] [MAIN ] NDMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:29.855198] [DEBUG] [LOCAL ] NDMO: Loaded 16985 existing records from primary storage. [2025-10-20T22:12:29.855470] [INFO ] [MAIN ] NDMO: Loaded 16985 primary data records. [2025-10-20T22:12:30.175461] [DEBUG] [MAIN ] NCV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:30.176544] [INFO ] [MAIN ] NCV: Generated 1794 derivative records from 13877 primary records. [2025-10-20T22:12:30.176810] [DEBUG] [LOCAL ] NCV: Saving derivative data. [2025-10-20T22:12:30.209969] [DEBUG] [LOCAL ] NCV: Derivative data saved (1794 records). [2025-10-20T22:12:30.210997] [DEBUG] [MAIN ] NCZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:30.212023] [INFO ] [MAIN ] NCZ: Generated 1794 derivative records from 12277 primary records. [2025-10-20T22:12:30.211193] [INFO ] [MAIN ] NCV: Successfully saved derivative data. [2025-10-20T22:12:30.214320] [DEBUG] [LOCAL ] NCZ: Saving derivative data. [2025-10-20T22:12:30.214220] [INFO ] [MAIN ] Derivative Progress: 1249/1877 symbols completed. NCV took 1.1s. Estimated remaining: 3m 56s. Rate: 159.4 symbols/min [2025-10-20T22:12:30.214662] [INFO ] [MAIN ] NE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:30.233876] [DEBUG] [LOCAL ] NCZ: Derivative data saved (1794 records). [2025-10-20T22:12:30.234102] [INFO ] [MAIN ] NCZ: Successfully saved derivative data. [2025-10-20T22:12:30.237778] [INFO ] [MAIN ] Derivative Progress: 1250/1877 symbols completed. NCZ took 1.1s. Estimated remaining: 3m 56s. Rate: 159.6 symbols/min [2025-10-20T22:12:30.238189] [INFO ] [MAIN ] NEA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:30.252441] [DEBUG] [MAIN ] NDMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:30.253126] [INFO ] [MAIN ] NDMO: Generated 1794 derivative records from 16985 primary records. [2025-10-20T22:12:30.253797] [DEBUG] [LOCAL ] NDMO: Saving derivative data. [2025-10-20T22:12:30.270365] [DEBUG] [LOCAL ] NDMO: Derivative data saved (1794 records). [2025-10-20T22:12:30.270549] [INFO ] [MAIN ] NDMO: Successfully saved derivative data. [2025-10-20T22:12:30.272561] [INFO ] [MAIN ] Derivative Progress: 1251/1877 symbols completed. NDMO took 0.8s. Estimated remaining: 3m 55s. Rate: 159.7 symbols/min [2025-10-20T22:12:30.272930] [INFO ] [MAIN ] NEE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:30.315637] [DEBUG] [LOCAL ] NEA: Loaded 35207 existing records from primary storage. [2025-10-20T22:12:30.330397] [INFO ] [MAIN ] NEA: Loaded 35207 primary data records. [2025-10-20T22:12:30.386753] [DEBUG] [LOCAL ] NE: Loaded 52353 existing records from primary storage. [2025-10-20T22:12:30.407319] [INFO ] [MAIN ] NE: Loaded 52353 primary data records. [2025-10-20T22:12:30.775507] [DEBUG] [LOCAL ] NEE: Loaded 53707 existing records from primary storage. [2025-10-20T22:12:30.800035] [INFO ] [MAIN ] NEE: Loaded 53707 primary data records. [2025-10-20T22:12:31.362842] [DEBUG] [MAIN ] NEA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:31.364951] [INFO ] [MAIN ] NEA: Generated 1794 derivative records from 35207 primary records. [2025-10-20T22:12:31.366130] [DEBUG] [LOCAL ] NEA: Saving derivative data. [2025-10-20T22:12:31.393618] [DEBUG] [MAIN ] NE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:31.399702] [INFO ] [MAIN ] NE: Generated 1794 derivative records from 52353 primary records. [2025-10-20T22:12:31.401182] [DEBUG] [LOCAL ] NE: Saving derivative data. [2025-10-20T22:12:31.401284] [DEBUG] [LOCAL ] NEA: Derivative data saved (1794 records). [2025-10-20T22:12:31.402177] [INFO ] [MAIN ] NEA: Successfully saved derivative data. [2025-10-20T22:12:31.405286] [INFO ] [MAIN ] Derivative Progress: 1252/1877 symbols completed. NEA took 1.2s. Estimated remaining: 3m 55s. Rate: 159.4 symbols/min [2025-10-20T22:12:31.405933] [INFO ] [MAIN ] NEM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:31.423873] [DEBUG] [LOCAL ] NE: Derivative data saved (1794 records). [2025-10-20T22:12:31.424195] [INFO ] [MAIN ] NE: Successfully saved derivative data. [2025-10-20T22:12:31.426456] [INFO ] [MAIN ] Derivative Progress: 1253/1877 symbols completed. NE took 1.2s. Estimated remaining: 3m 55s. Rate: 159.5 symbols/min [2025-10-20T22:12:31.427253] [INFO ] [MAIN ] NET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:31.446619] [DEBUG] [MAIN ] NEE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:31.447427] [INFO ] [MAIN ] NEE: Generated 1794 derivative records from 53707 primary records. [2025-10-20T22:12:31.447553] [DEBUG] [LOCAL ] NEE: Saving derivative data. [2025-10-20T22:12:31.463912] [DEBUG] [LOCAL ] NEE: Derivative data saved (1794 records). [2025-10-20T22:12:31.464109] [INFO ] [MAIN ] NEE: Successfully saved derivative data. [2025-10-20T22:12:31.466301] [INFO ] [MAIN ] Derivative Progress: 1254/1877 symbols completed. NEE took 1.2s. Estimated remaining: 3m 54s. Rate: 159.7 symbols/min [2025-10-20T22:12:31.466464] [INFO ] [MAIN ] NEU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:31.525294] [DEBUG] [LOCAL ] NEM: Loaded 53708 existing records from primary storage. [2025-10-20T22:12:31.525596] [DEBUG] [LOCAL ] NEU: Loaded 8633 existing records from primary storage. [2025-10-20T22:12:31.525994] [INFO ] [MAIN ] NEM: Loaded 53708 primary data records. [2025-10-20T22:12:31.526089] [INFO ] [MAIN ] NEU: Loaded 8633 primary data records. [2025-10-20T22:12:31.691517] [DEBUG] [LOCAL ] NET: Loaded 52843 existing records from primary storage. [2025-10-20T22:12:31.722823] [INFO ] [MAIN ] NET: Loaded 52843 primary data records. [2025-10-20T22:12:32.491008] [DEBUG] [MAIN ] NEU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:32.493045] [INFO ] [MAIN ] NEU: Generated 1794 derivative records from 8633 primary records. [2025-10-20T22:12:32.493231] [DEBUG] [LOCAL ] NEU: Saving derivative data. [2025-10-20T22:12:32.524267] [DEBUG] [LOCAL ] NEU: Derivative data saved (1794 records). [2025-10-20T22:12:32.524469] [INFO ] [MAIN ] NEU: Successfully saved derivative data. [2025-10-20T22:12:32.524184] [DEBUG] [MAIN ] NEM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:32.526306] [INFO ] [MAIN ] Derivative Progress: 1255/1877 symbols completed. NEU took 1.1s. Estimated remaining: 3m 54s. Rate: 159.4 symbols/min [2025-10-20T22:12:32.526577] [INFO ] [MAIN ] NEXA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:32.527440] [INFO ] [MAIN ] NEM: Generated 1794 derivative records from 53708 primary records. [2025-10-20T22:12:32.527631] [DEBUG] [LOCAL ] NEM: Saving derivative data. [2025-10-20T22:12:32.541458] [DEBUG] [MAIN ] NET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:32.549786] [INFO ] [MAIN ] NET: Generated 1794 derivative records from 52843 primary records. [2025-10-20T22:12:32.549983] [DEBUG] [LOCAL ] NET: Saving derivative data. [2025-10-20T22:12:32.560359] [DEBUG] [LOCAL ] NEM: Derivative data saved (1794 records). [2025-10-20T22:12:32.571768] [INFO ] [MAIN ] NEM: Successfully saved derivative data. [2025-10-20T22:12:32.575923] [DEBUG] [LOCAL ] NET: Derivative data saved (1794 records). [2025-10-20T22:12:32.581396] [INFO ] [MAIN ] Derivative Progress: 1256/1877 symbols completed. NEM took 1.2s. Estimated remaining: 3m 54s. Rate: 159.5 symbols/min [2025-10-20T22:12:32.581822] [DEBUG] [LOCAL ] NEXA: Loaded 5494 existing records from primary storage. [2025-10-20T22:12:32.581589] [INFO ] [MAIN ] NET: Successfully saved derivative data. [2025-10-20T22:12:32.581984] [INFO ] [MAIN ] NFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:32.584537] [INFO ] [MAIN ] Derivative Progress: 1257/1877 symbols completed. NET took 1.2s. Estimated remaining: 3m 53s. Rate: 159.7 symbols/min [2025-10-20T22:12:32.584612] [INFO ] [MAIN ] NEXA: Loaded 5494 primary data records. [2025-10-20T22:12:32.597435] [INFO ] [MAIN ] NFJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:32.891503] [DEBUG] [LOCAL ] NFJ: Loaded 20008 existing records from primary storage. [2025-10-20T22:12:32.896945] [INFO ] [MAIN ] NFJ: Loaded 20008 primary data records. [2025-10-20T22:12:33.026280] [DEBUG] [LOCAL ] NFG: Loaded 38404 existing records from primary storage. [2025-10-20T22:12:33.026771] [INFO ] [MAIN ] NFG: Loaded 38404 primary data records. [2025-10-20T22:12:33.578203] [DEBUG] [MAIN ] NEXA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:33.579002] [INFO ] [MAIN ] NEXA: Generated 1794 derivative records from 5494 primary records. [2025-10-20T22:12:33.579607] [DEBUG] [LOCAL ] NEXA: Saving derivative data. [2025-10-20T22:12:33.595716] [DEBUG] [LOCAL ] NEXA: Derivative data saved (1794 records). [2025-10-20T22:12:33.596193] [INFO ] [MAIN ] NEXA: Successfully saved derivative data. [2025-10-20T22:12:33.598254] [INFO ] [MAIN ] Derivative Progress: 1258/1877 symbols completed. NEXA took 1.1s. Estimated remaining: 3m 53s. Rate: 159.4 symbols/min [2025-10-20T22:12:33.616579] [DEBUG] [MAIN ] NFJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:33.617387] [INFO ] [MAIN ] NFJ: Generated 1794 derivative records from 20008 primary records. [2025-10-20T22:12:33.617529] [DEBUG] [LOCAL ] NFJ: Saving derivative data. [2025-10-20T22:12:33.635316] [DEBUG] [LOCAL ] NFJ: Derivative data saved (1794 records). [2025-10-20T22:12:33.635519] [INFO ] [MAIN ] NFJ: Successfully saved derivative data. [2025-10-20T22:12:33.647790] [DEBUG] [MAIN ] NFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:33.649847] [INFO ] [MAIN ] Derivative Progress: 1259/1877 symbols completed. NFJ took 1.1s. Estimated remaining: 3m 52s. Rate: 159.6 symbols/min [2025-10-20T22:12:33.650565] [INFO ] [MAIN ] NFG: Generated 1794 derivative records from 38404 primary records. [2025-10-20T22:12:33.650777] [DEBUG] [LOCAL ] NFG: Saving derivative data. [2025-10-20T22:12:33.665462] [DEBUG] [LOCAL ] NFG: Derivative data saved (1794 records). [2025-10-20T22:12:33.665520] [INFO ] [MAIN ] NFG: Successfully saved derivative data. [2025-10-20T22:12:33.667788] [INFO ] [MAIN ] Derivative Progress: 1260/1877 symbols completed. NFG took 1.1s. Estimated remaining: 3m 52s. Rate: 159.7 symbols/min [2025-10-20T22:12:33.668500] [INFO ] [MAIN ] NGG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:33.668605] [INFO ] [MAIN ] NGL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:33.668785] [INFO ] [MAIN ] NGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:33.686579] [DEBUG] [LOCAL ] NGS: Loaded 11109 existing records from primary storage. [2025-10-20T22:12:33.695950] [INFO ] [MAIN ] NGS: Loaded 11109 primary data records. [2025-10-20T22:12:33.830203] [DEBUG] [LOCAL ] NGL: Loaded 29965 existing records from primary storage. [2025-10-20T22:12:33.858958] [INFO ] [MAIN ] NGL: Loaded 29965 primary data records. [2025-10-20T22:12:33.907557] [DEBUG] [LOCAL ] NGG: Loaded 46845 existing records from primary storage. [2025-10-20T22:12:34.003198] [INFO ] [MAIN ] NGG: Loaded 46845 primary data records. [2025-10-20T22:12:34.685553] [DEBUG] [MAIN ] NGS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:34.686598] [INFO ] [MAIN ] NGS: Generated 1794 derivative records from 11109 primary records. [2025-10-20T22:12:34.686877] [DEBUG] [LOCAL ] NGS: Saving derivative data. [2025-10-20T22:12:34.717319] [DEBUG] [LOCAL ] NGS: Derivative data saved (1794 records). [2025-10-20T22:12:34.715620] [DEBUG] [MAIN ] NGL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:34.717542] [INFO ] [MAIN ] NGS: Successfully saved derivative data. [2025-10-20T22:12:34.718288] [INFO ] [MAIN ] NGL: Generated 1794 derivative records from 29965 primary records. [2025-10-20T22:12:34.718398] [DEBUG] [LOCAL ] NGL: Saving derivative data. [2025-10-20T22:12:34.720465] [INFO ] [MAIN ] Derivative Progress: 1261/1877 symbols completed. NGS took 1.1s. Estimated remaining: 3m 52s. Rate: 159.4 symbols/min [2025-10-20T22:12:34.721122] [INFO ] [MAIN ] NGVC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:34.740000] [DEBUG] [LOCAL ] NGL: Derivative data saved (1794 records). [2025-10-20T22:12:34.740879] [INFO ] [MAIN ] NGL: Successfully saved derivative data. [2025-10-20T22:12:34.743539] [INFO ] [MAIN ] Derivative Progress: 1262/1877 symbols completed. NGL took 1.1s. Estimated remaining: 3m 51s. Rate: 159.6 symbols/min [2025-10-20T22:12:34.743743] [INFO ] [MAIN ] NGVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:34.757760] [DEBUG] [MAIN ] NGG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:34.758690] [INFO ] [MAIN ] NGG: Generated 1794 derivative records from 46845 primary records. [2025-10-20T22:12:34.764359] [DEBUG] [LOCAL ] NGG: Saving derivative data. [2025-10-20T22:12:34.794208] [DEBUG] [LOCAL ] NGG: Derivative data saved (1794 records). [2025-10-20T22:12:34.794454] [INFO ] [MAIN ] NGG: Successfully saved derivative data. [2025-10-20T22:12:34.794883] [DEBUG] [LOCAL ] NGVC: Loaded 22829 existing records from primary storage. [2025-10-20T22:12:34.795067] [INFO ] [MAIN ] NGVC: Loaded 22829 primary data records. [2025-10-20T22:12:34.797179] [INFO ] [MAIN ] Derivative Progress: 1263/1877 symbols completed. NGG took 1.1s. Estimated remaining: 3m 51s. Rate: 159.7 symbols/min [2025-10-20T22:12:34.828722] [INFO ] [MAIN ] NHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:34.873966] [DEBUG] [LOCAL ] NGVT: Loaded 24042 existing records from primary storage. [2025-10-20T22:12:34.874065] [INFO ] [MAIN ] NGVT: Loaded 24042 primary data records. [2025-10-20T22:12:35.446137] [DEBUG] [LOCAL ] NHI: Loaded 22020 existing records from primary storage. [2025-10-20T22:12:35.456467] [INFO ] [MAIN ] NHI: Loaded 22020 primary data records. [2025-10-20T22:12:35.774564] [DEBUG] [MAIN ] NGVC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:35.775396] [INFO ] [MAIN ] NGVC: Generated 1794 derivative records from 22829 primary records. [2025-10-20T22:12:35.775604] [DEBUG] [LOCAL ] NGVC: Saving derivative data. [2025-10-20T22:12:35.794219] [DEBUG] [LOCAL ] NGVC: Derivative data saved (1794 records). [2025-10-20T22:12:35.794338] [INFO ] [MAIN ] NGVC: Successfully saved derivative data. [2025-10-20T22:12:35.796373] [INFO ] [MAIN ] Derivative Progress: 1264/1877 symbols completed. NGVC took 1.1s. Estimated remaining: 3m 51s. Rate: 159.5 symbols/min [2025-10-20T22:12:35.796603] [INFO ] [MAIN ] NI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:35.814399] [DEBUG] [MAIN ] NGVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:35.815114] [INFO ] [MAIN ] NGVT: Generated 1794 derivative records from 24042 primary records. [2025-10-20T22:12:35.815440] [DEBUG] [LOCAL ] NGVT: Saving derivative data. [2025-10-20T22:12:35.831879] [DEBUG] [LOCAL ] NGVT: Derivative data saved (1794 records). [2025-10-20T22:12:35.832176] [INFO ] [MAIN ] NGVT: Successfully saved derivative data. [2025-10-20T22:12:35.834555] [INFO ] [MAIN ] Derivative Progress: 1265/1877 symbols completed. NGVT took 1.1s. Estimated remaining: 3m 50s. Rate: 159.6 symbols/min [2025-10-20T22:12:35.834968] [INFO ] [MAIN ] NIC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:35.851034] [DEBUG] [MAIN ] NHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:35.851710] [INFO ] [MAIN ] NHI: Generated 1794 derivative records from 22020 primary records. [2025-10-20T22:12:35.851857] [DEBUG] [LOCAL ] NHI: Saving derivative data. [2025-10-20T22:12:35.882279] [DEBUG] [LOCAL ] NHI: Derivative data saved (1794 records). [2025-10-20T22:12:35.882561] [INFO ] [MAIN ] NHI: Successfully saved derivative data. [2025-10-20T22:12:35.889015] [INFO ] [MAIN ] Derivative Progress: 1266/1877 symbols completed. NHI took 1.1s. Estimated remaining: 3m 50s. Rate: 159.7 symbols/min [2025-10-20T22:12:35.889166] [INFO ] [MAIN ] NIE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:35.886738] [DEBUG] [LOCAL ] NIC: Loaded 8898 existing records from primary storage. [2025-10-20T22:12:35.889812] [INFO ] [MAIN ] NIC: Loaded 8898 primary data records. [2025-10-20T22:12:36.033202] [DEBUG] [LOCAL ] NI: Loaded 53410 existing records from primary storage. [2025-10-20T22:12:36.038582] [INFO ] [MAIN ] NI: Loaded 53410 primary data records. [2025-10-20T22:12:36.075424] [DEBUG] [LOCAL ] NIE: Loaded 11838 existing records from primary storage. [2025-10-20T22:12:36.148766] [INFO ] [MAIN ] NIE: Loaded 11838 primary data records. [2025-10-20T22:12:36.919868] [DEBUG] [MAIN ] NIC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:36.925758] [DEBUG] [MAIN ] NI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:36.927344] [INFO ] [MAIN ] NIC: Generated 1794 derivative records from 8898 primary records. [2025-10-20T22:12:36.926579] [INFO ] [MAIN ] NI: Generated 1794 derivative records from 53410 primary records. [2025-10-20T22:12:36.927548] [DEBUG] [LOCAL ] NIC: Saving derivative data. [2025-10-20T22:12:36.928592] [DEBUG] [LOCAL ] NI: Saving derivative data. [2025-10-20T22:12:36.946978] [DEBUG] [LOCAL ] NI: Derivative data saved (1794 records). [2025-10-20T22:12:36.961596] [DEBUG] [LOCAL ] NIC: Derivative data saved (1794 records). [2025-10-20T22:12:36.961879] [INFO ] [MAIN ] NIC: Successfully saved derivative data. [2025-10-20T22:12:36.971596] [INFO ] [MAIN ] Derivative Progress: 1267/1877 symbols completed. NIC took 1.1s. Estimated remaining: 3m 50s. Rate: 159.4 symbols/min [2025-10-20T22:12:36.960880] [INFO ] [MAIN ] NI: Successfully saved derivative data. [2025-10-20T22:12:36.971852] [INFO ] [MAIN ] NIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:36.979056] [INFO ] [MAIN ] Derivative Progress: 1268/1877 symbols completed. NI took 1.2s. Estimated remaining: 3m 49s. Rate: 159.6 symbols/min [2025-10-20T22:12:36.979952] [DEBUG] [MAIN ] NIE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:36.979664] [INFO ] [MAIN ] NINE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:36.980889] [INFO ] [MAIN ] NIE: Generated 1794 derivative records from 11838 primary records. [2025-10-20T22:12:36.981440] [DEBUG] [LOCAL ] NIE: Saving derivative data. [2025-10-20T22:12:37.001157] [DEBUG] [LOCAL ] NIE: Derivative data saved (1794 records). [2025-10-20T22:12:37.001445] [INFO ] [MAIN ] NIE: Successfully saved derivative data. [2025-10-20T22:12:37.001380] [DEBUG] [LOCAL ] NIM: Loaded 3520 existing records from primary storage. [2025-10-20T22:12:37.003845] [INFO ] [MAIN ] Derivative Progress: 1269/1877 symbols completed. NIE took 1.1s. Estimated remaining: 3m 48s. Rate: 159.7 symbols/min [2025-10-20T22:12:37.001611] [INFO ] [MAIN ] NIM: Loaded 3520 primary data records. [2025-10-20T22:12:37.004074] [INFO ] [MAIN ] NIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:37.132507] [DEBUG] [LOCAL ] NINE: Loaded 28597 existing records from primary storage. [2025-10-20T22:12:37.132737] [INFO ] [MAIN ] NINE: Loaded 28597 primary data records. [2025-10-20T22:12:37.640292] [DEBUG] [LOCAL ] NIO: Loaded 53802 existing records from primary storage. [2025-10-20T22:12:37.649613] [INFO ] [MAIN ] NIO: Loaded 53802 primary data records. [2025-10-20T22:12:37.987877] [DEBUG] [MAIN ] NIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:37.990445] [INFO ] [MAIN ] NIM: Generated 1794 derivative records from 3520 primary records. [2025-10-20T22:12:37.990604] [DEBUG] [LOCAL ] NIM: Saving derivative data. [2025-10-20T22:12:38.010068] [DEBUG] [MAIN ] NINE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:38.011916] [INFO ] [MAIN ] NINE: Generated 1794 derivative records from 28597 primary records. [2025-10-20T22:12:38.019165] [DEBUG] [LOCAL ] NINE: Saving derivative data. [2025-10-20T22:12:38.018806] [DEBUG] [LOCAL ] NIM: Derivative data saved (1794 records). [2025-10-20T22:12:38.019496] [INFO ] [MAIN ] NIM: Successfully saved derivative data. [2025-10-20T22:12:38.022147] [INFO ] [MAIN ] Derivative Progress: 1270/1877 symbols completed. NIM took 1.1s. Estimated remaining: 3m 48s. Rate: 159.5 symbols/min [2025-10-20T22:12:38.022394] [INFO ] [MAIN ] NJR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:38.038631] [DEBUG] [LOCAL ] NINE: Derivative data saved (1794 records). [2025-10-20T22:12:38.038768] [INFO ] [MAIN ] NINE: Successfully saved derivative data. [2025-10-20T22:12:38.041105] [INFO ] [MAIN ] Derivative Progress: 1271/1877 symbols completed. NINE took 1.1s. Estimated remaining: 3m 48s. Rate: 159.6 symbols/min [2025-10-20T22:12:38.041341] [INFO ] [MAIN ] NKX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:38.057829] [DEBUG] [MAIN ] NIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:38.058729] [INFO ] [MAIN ] NIO: Generated 1794 derivative records from 53802 primary records. [2025-10-20T22:12:38.058795] [DEBUG] [LOCAL ] NIO: Saving derivative data. [2025-10-20T22:12:38.078531] [DEBUG] [LOCAL ] NIO: Derivative data saved (1794 records). [2025-10-20T22:12:38.078756] [INFO ] [MAIN ] NIO: Successfully saved derivative data. [2025-10-20T22:12:38.080942] [INFO ] [MAIN ] Derivative Progress: 1272/1877 symbols completed. NIO took 1.1s. Estimated remaining: 3m 47s. Rate: 159.7 symbols/min [2025-10-20T22:12:38.081174] [INFO ] [MAIN ] NLOP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:38.095404] [DEBUG] [LOCAL ] NKX: Loaded 13697 existing records from primary storage. [2025-10-20T22:12:38.095749] [INFO ] [MAIN ] NKX: Loaded 13697 primary data records. [2025-10-20T22:12:38.213059] [DEBUG] [LOCAL ] NJR: Loaded 38008 existing records from primary storage. [2025-10-20T22:12:38.216581] [INFO ] [MAIN ] NJR: Loaded 38008 primary data records. [2025-10-20T22:12:38.263540] [DEBUG] [LOCAL ] NLOP: Loaded 11699 existing records from primary storage. [2025-10-20T22:12:38.268846] [INFO ] [MAIN ] NLOP: Loaded 11699 primary data records. [2025-10-20T22:12:39.078674] [DEBUG] [MAIN ] NKX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:39.079530] [INFO ] [MAIN ] NKX: Generated 1794 derivative records from 13697 primary records. [2025-10-20T22:12:39.079803] [DEBUG] [LOCAL ] NKX: Saving derivative data. [2025-10-20T22:12:39.107385] [DEBUG] [LOCAL ] NKX: Derivative data saved (1794 records). [2025-10-20T22:12:39.112575] [DEBUG] [MAIN ] NJR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:39.120817] [INFO ] [MAIN ] NJR: Generated 1794 derivative records from 38008 primary records. [2025-10-20T22:12:39.125801] [DEBUG] [LOCAL ] NJR: Saving derivative data. [2025-10-20T22:12:39.112773] [INFO ] [MAIN ] NKX: Successfully saved derivative data. [2025-10-20T22:12:39.125731] [DEBUG] [MAIN ] NLOP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:39.128880] [INFO ] [MAIN ] NLOP: Generated 1794 derivative records from 11699 primary records. [2025-10-20T22:12:39.129314] [DEBUG] [LOCAL ] NLOP: Saving derivative data. [2025-10-20T22:12:39.127809] [INFO ] [MAIN ] Derivative Progress: 1273/1877 symbols completed. NKX took 1.1s. Estimated remaining: 3m 47s. Rate: 159.5 symbols/min [2025-10-20T22:12:39.158673] [DEBUG] [LOCAL ] NLOP: Derivative data saved (1794 records). [2025-10-20T22:12:39.158905] [DEBUG] [LOCAL ] NJR: Derivative data saved (1794 records). [2025-10-20T22:12:39.158976] [INFO ] [MAIN ] NLOP: Successfully saved derivative data. [2025-10-20T22:12:39.159064] [INFO ] [MAIN ] NJR: Successfully saved derivative data. [2025-10-20T22:12:39.161158] [INFO ] [MAIN ] Derivative Progress: 1274/1877 symbols completed. NLOP took 1.1s. Estimated remaining: 3m 47s. Rate: 159.6 symbols/min [2025-10-20T22:12:39.163371] [INFO ] [MAIN ] Derivative Progress: 1275/1877 symbols completed. NJR took 1.1s. Estimated remaining: 3m 46s. Rate: 159.7 symbols/min [2025-10-20T22:12:39.163905] [INFO ] [MAIN ] NMAI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:39.163854] [INFO ] [MAIN ] NLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:39.164044] [INFO ] [MAIN ] NMCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:39.183320] [DEBUG] [LOCAL ] NMAI: Loaded 12634 existing records from primary storage. [2025-10-20T22:12:39.183493] [INFO ] [MAIN ] NMAI: Loaded 12634 primary data records. [2025-10-20T22:12:39.234802] [DEBUG] [LOCAL ] NMCO: Loaded 16596 existing records from primary storage. [2025-10-20T22:12:39.261441] [INFO ] [MAIN ] NMCO: Loaded 16596 primary data records. [2025-10-20T22:12:39.453464] [DEBUG] [LOCAL ] NLY: Loaded 53564 existing records from primary storage. [2025-10-20T22:12:39.463983] [INFO ] [MAIN ] NLY: Loaded 53564 primary data records. [2025-10-20T22:12:40.158844] [DEBUG] [MAIN ] NMAI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:40.160005] [INFO ] [MAIN ] NMAI: Generated 1794 derivative records from 12634 primary records. [2025-10-20T22:12:40.159285] [DEBUG] [MAIN ] NMCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:40.160274] [DEBUG] [LOCAL ] NMAI: Saving derivative data. [2025-10-20T22:12:40.161523] [INFO ] [MAIN ] NMCO: Generated 1794 derivative records from 16596 primary records. [2025-10-20T22:12:40.161911] [DEBUG] [LOCAL ] NMCO: Saving derivative data. [2025-10-20T22:12:40.191742] [DEBUG] [LOCAL ] NMAI: Derivative data saved (1794 records). [2025-10-20T22:12:40.206292] [DEBUG] [LOCAL ] NMCO: Derivative data saved (1794 records). [2025-10-20T22:12:40.206766] [INFO ] [MAIN ] NMAI: Successfully saved derivative data. [2025-10-20T22:12:40.207107] [INFO ] [MAIN ] NMCO: Successfully saved derivative data. [2025-10-20T22:12:40.209479] [INFO ] [MAIN ] Derivative Progress: 1276/1877 symbols completed. NMAI took 1.0s. Estimated remaining: 3m 46s. Rate: 159.5 symbols/min [2025-10-20T22:12:40.211865] [DEBUG] [MAIN ] NLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:40.212098] [INFO ] [MAIN ] NMG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:40.212240] [INFO ] [MAIN ] Derivative Progress: 1277/1877 symbols completed. NMCO took 1.0s. Estimated remaining: 3m 46s. Rate: 159.6 symbols/min [2025-10-20T22:12:40.212973] [INFO ] [MAIN ] NMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:40.213783] [INFO ] [MAIN ] NLY: Generated 1794 derivative records from 53564 primary records. [2025-10-20T22:12:40.214115] [DEBUG] [LOCAL ] NLY: Saving derivative data. [2025-10-20T22:12:40.234677] [DEBUG] [LOCAL ] NLY: Derivative data saved (1794 records). [2025-10-20T22:12:40.234796] [INFO ] [MAIN ] NLY: Successfully saved derivative data. [2025-10-20T22:12:40.236915] [INFO ] [MAIN ] Derivative Progress: 1278/1877 symbols completed. NLY took 1.1s. Estimated remaining: 3m 45s. Rate: 159.7 symbols/min [2025-10-20T22:12:40.237114] [INFO ] [MAIN ] NMM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:40.240171] [DEBUG] [LOCAL ] NMI: Loaded 4049 existing records from primary storage. [2025-10-20T22:12:40.240738] [INFO ] [MAIN ] NMI: Loaded 4049 primary data records. [2025-10-20T22:12:40.309202] [DEBUG] [LOCAL ] NMG: Loaded 11574 existing records from primary storage. [2025-10-20T22:12:40.325535] [INFO ] [MAIN ] NMG: Loaded 11574 primary data records. [2025-10-20T22:12:40.404059] [DEBUG] [LOCAL ] NMM: Loaded 14378 existing records from primary storage. [2025-10-20T22:12:40.436722] [INFO ] [MAIN ] NMM: Loaded 14378 primary data records. [2025-10-20T22:12:41.240134] [DEBUG] [MAIN ] NMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:41.241140] [INFO ] [MAIN ] NMI: Generated 1794 derivative records from 4049 primary records. [2025-10-20T22:12:41.241525] [DEBUG] [LOCAL ] NMI: Saving derivative data. [2025-10-20T22:12:41.257905] [DEBUG] [LOCAL ] NMI: Derivative data saved (1794 records). [2025-10-20T22:12:41.258287] [INFO ] [MAIN ] NMI: Successfully saved derivative data. [2025-10-20T22:12:41.261133] [INFO ] [MAIN ] Derivative Progress: 1279/1877 symbols completed. NMI took 1.0s. Estimated remaining: 3m 45s. Rate: 159.5 symbols/min [2025-10-20T22:12:41.262002] [INFO ] [MAIN ] NMR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:41.281074] [DEBUG] [MAIN ] NMG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:41.294319] [INFO ] [MAIN ] NMG: Generated 1794 derivative records from 11574 primary records. [2025-10-20T22:12:41.294515] [DEBUG] [LOCAL ] NMG: Saving derivative data. [2025-10-20T22:12:41.295869] [DEBUG] [MAIN ] NMM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:41.296784] [INFO ] [MAIN ] NMM: Generated 1794 derivative records from 14378 primary records. [2025-10-20T22:12:41.304890] [DEBUG] [LOCAL ] NMM: Saving derivative data. [2025-10-20T22:12:41.312997] [DEBUG] [LOCAL ] NMG: Derivative data saved (1794 records). [2025-10-20T22:12:41.322227] [INFO ] [MAIN ] NMG: Successfully saved derivative data. [2025-10-20T22:12:41.331666] [INFO ] [MAIN ] Derivative Progress: 1280/1877 symbols completed. NMG took 1.1s. Estimated remaining: 3m 44s. Rate: 159.6 symbols/min [2025-10-20T22:12:41.332886] [INFO ] [MAIN ] NMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:41.340056] [DEBUG] [LOCAL ] NMM: Derivative data saved (1794 records). [2025-10-20T22:12:41.340837] [INFO ] [MAIN ] NMM: Successfully saved derivative data. [2025-10-20T22:12:41.353899] [INFO ] [MAIN ] Derivative Progress: 1281/1877 symbols completed. NMM took 1.1s. Estimated remaining: 3m 44s. Rate: 159.7 symbols/min [2025-10-20T22:12:41.356050] [INFO ] [MAIN ] NMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:41.356774] [DEBUG] [LOCAL ] NMR: Loaded 35622 existing records from primary storage. [2025-10-20T22:12:41.357365] [INFO ] [MAIN ] NMR: Loaded 35622 primary data records. [2025-10-20T22:12:41.394790] [DEBUG] [LOCAL ] NMS: Loaded 2879 existing records from primary storage. [2025-10-20T22:12:41.400112] [INFO ] [MAIN ] NMS: Loaded 2879 primary data records. [2025-10-20T22:12:41.494443] [DEBUG] [LOCAL ] NMT: Loaded 2818 existing records from primary storage. [2025-10-20T22:12:41.502247] [INFO ] [MAIN ] NMT: Loaded 2818 primary data records. [2025-10-20T22:12:42.284129] [DEBUG] [MAIN ] NMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:42.296992] [INFO ] [MAIN ] NMS: Generated 1794 derivative records from 2879 primary records. [2025-10-20T22:12:42.297271] [DEBUG] [MAIN ] NMR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:42.298027] [INFO ] [MAIN ] NMR: Generated 1794 derivative records from 35622 primary records. [2025-10-20T22:12:42.298172] [DEBUG] [LOCAL ] NMR: Saving derivative data. [2025-10-20T22:12:42.297445] [DEBUG] [LOCAL ] NMS: Saving derivative data. [2025-10-20T22:12:42.322385] [DEBUG] [LOCAL ] NMR: Derivative data saved (1794 records). [2025-10-20T22:12:42.337309] [INFO ] [MAIN ] NMR: Successfully saved derivative data. [2025-10-20T22:12:42.337254] [DEBUG] [LOCAL ] NMS: Derivative data saved (1794 records). [2025-10-20T22:12:42.344102] [INFO ] [MAIN ] Derivative Progress: 1282/1877 symbols completed. NMR took 1.1s. Estimated remaining: 3m 44s. Rate: 159.5 symbols/min [2025-10-20T22:12:42.344205] [INFO ] [MAIN ] NMS: Successfully saved derivative data. [2025-10-20T22:12:42.344469] [INFO ] [MAIN ] NMZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:42.346978] [INFO ] [MAIN ] Derivative Progress: 1283/1877 symbols completed. NMS took 1.0s. Estimated remaining: 3m 43s. Rate: 159.7 symbols/min [2025-10-20T22:12:42.344843] [DEBUG] [MAIN ] NMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:42.347099] [INFO ] [MAIN ] NNI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:42.348383] [INFO ] [MAIN ] NMT: Generated 1794 derivative records from 2818 primary records. [2025-10-20T22:12:42.348534] [DEBUG] [LOCAL ] NMT: Saving derivative data. [2025-10-20T22:12:42.365844] [DEBUG] [LOCAL ] NMT: Derivative data saved (1794 records). [2025-10-20T22:12:42.366101] [INFO ] [MAIN ] NMT: Successfully saved derivative data. [2025-10-20T22:12:42.368242] [INFO ] [MAIN ] Derivative Progress: 1284/1877 symbols completed. NMT took 1.0s. Estimated remaining: 3m 43s. Rate: 159.8 symbols/min [2025-10-20T22:12:42.368464] [INFO ] [MAIN ] NNN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:42.379031] [DEBUG] [LOCAL ] NNI: Loaded 10532 existing records from primary storage. [2025-10-20T22:12:42.379141] [INFO ] [MAIN ] NNI: Loaded 10532 primary data records. [2025-10-20T22:12:42.466200] [DEBUG] [LOCAL ] NMZ: Loaded 21584 existing records from primary storage. [2025-10-20T22:12:42.477134] [INFO ] [MAIN ] NMZ: Loaded 21584 primary data records. [2025-10-20T22:12:42.883437] [DEBUG] [LOCAL ] NNN: Loaded 48230 existing records from primary storage. [2025-10-20T22:12:42.913877] [INFO ] [MAIN ] NNN: Loaded 48230 primary data records. [2025-10-20T22:12:43.344036] [DEBUG] [MAIN ] NNI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:43.345745] [INFO ] [MAIN ] NNI: Generated 1794 derivative records from 10532 primary records. [2025-10-20T22:12:43.346029] [DEBUG] [LOCAL ] NNI: Saving derivative data. [2025-10-20T22:12:43.365523] [DEBUG] [LOCAL ] NNI: Derivative data saved (1794 records). [2025-10-20T22:12:43.378157] [INFO ] [MAIN ] NNI: Successfully saved derivative data. [2025-10-20T22:12:43.380434] [INFO ] [MAIN ] Derivative Progress: 1285/1877 symbols completed. NNI took 1.0s. Estimated remaining: 3m 43s. Rate: 159.6 symbols/min [2025-10-20T22:12:43.380584] [INFO ] [MAIN ] NNY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:43.381457] [DEBUG] [MAIN ] NMZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:43.382373] [INFO ] [MAIN ] NMZ: Generated 1794 derivative records from 21584 primary records. [2025-10-20T22:12:43.382512] [DEBUG] [LOCAL ] NMZ: Saving derivative data. [2025-10-20T22:12:43.400113] [DEBUG] [LOCAL ] NMZ: Derivative data saved (1794 records). [2025-10-20T22:12:43.400548] [INFO ] [MAIN ] NMZ: Successfully saved derivative data. [2025-10-20T22:12:43.402721] [INFO ] [MAIN ] Derivative Progress: 1286/1877 symbols completed. NMZ took 1.1s. Estimated remaining: 3m 42s. Rate: 159.7 symbols/min [2025-10-20T22:12:43.403030] [INFO ] [MAIN ] NOA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:43.406162] [DEBUG] [LOCAL ] NNY: Loaded 5657 existing records from primary storage. [2025-10-20T22:12:43.406264] [INFO ] [MAIN ] NNY: Loaded 5657 primary data records. [2025-10-20T22:12:43.579187] [DEBUG] [MAIN ] NNN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:43.710867] [INFO ] [MAIN ] NNN: Generated 1794 derivative records from 48230 primary records. [2025-10-20T22:12:43.607254] [DEBUG] [LOCAL ] NOA: Loaded 15970 existing records from primary storage. [2025-10-20T22:12:43.724393] [DEBUG] [LOCAL ] NNN: Saving derivative data. [2025-10-20T22:12:43.724766] [INFO ] [MAIN ] NOA: Loaded 15970 primary data records. [2025-10-20T22:12:43.834139] [DEBUG] [LOCAL ] NNN: Derivative data saved (1794 records). [2025-10-20T22:12:43.851043] [INFO ] [MAIN ] NNN: Successfully saved derivative data. [2025-10-20T22:12:43.873997] [INFO ] [MAIN ] Derivative Progress: 1287/1877 symbols completed. NNN took 1.5s. Estimated remaining: 3m 42s. Rate: 159.7 symbols/min [2025-10-20T22:12:43.886807] [INFO ] [MAIN ] NOAH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:44.072295] [DEBUG] [LOCAL ] NOAH: Loaded 20177 existing records from primary storage. [2025-10-20T22:12:44.071947] [DEBUG] [MAIN ] NNY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:44.073060] [INFO ] [MAIN ] NNY: Generated 1794 derivative records from 5657 primary records. [2025-10-20T22:12:44.073214] [DEBUG] [LOCAL ] NNY: Saving derivative data. [2025-10-20T22:12:44.073264] [INFO ] [MAIN ] NOAH: Loaded 20177 primary data records. [2025-10-20T22:12:44.162595] [DEBUG] [LOCAL ] NNY: Derivative data saved (1794 records). [2025-10-20T22:12:44.173104] [INFO ] [MAIN ] NNY: Successfully saved derivative data. [2025-10-20T22:12:44.188026] [INFO ] [MAIN ] Derivative Progress: 1288/1877 symbols completed. NNY took 0.8s. Estimated remaining: 3m 41s. Rate: 159.7 symbols/min [2025-10-20T22:12:44.393962] [DEBUG] [MAIN ] NOA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:44.394752] [INFO ] [MAIN ] NOA: Generated 1794 derivative records from 15970 primary records. [2025-10-20T22:12:44.395593] [DEBUG] [LOCAL ] NOA: Saving derivative data. [2025-10-20T22:12:44.412768] [DEBUG] [LOCAL ] NOA: Derivative data saved (1794 records). [2025-10-20T22:12:44.412969] [INFO ] [MAIN ] NOA: Successfully saved derivative data. [2025-10-20T22:12:44.415366] [INFO ] [MAIN ] Derivative Progress: 1289/1877 symbols completed. NOA took 1.0s. Estimated remaining: 3m 41s. Rate: 159.7 symbols/min [2025-10-20T22:12:44.432746] [DEBUG] [MAIN ] NOAH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:44.433281] [INFO ] [MAIN ] NOAH: Generated 1794 derivative records from 20177 primary records. [2025-10-20T22:12:44.433338] [DEBUG] [LOCAL ] NOAH: Saving derivative data. [2025-10-20T22:12:44.448060] [DEBUG] [LOCAL ] NOAH: Derivative data saved (1794 records). [2025-10-20T22:12:44.448117] [INFO ] [MAIN ] NOAH: Successfully saved derivative data. [2025-10-20T22:12:44.450136] [INFO ] [MAIN ] Derivative Progress: 1290/1877 symbols completed. NOAH took 0.6s. Estimated remaining: 3m 40s. Rate: 159.8 symbols/min [2025-10-20T22:12:44.450866] [INFO ] [MAIN ] NOG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:44.451100] [INFO ] [MAIN ] NOK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:44.450797] [INFO ] [MAIN ] NOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:44.514971] [DEBUG] [LOCAL ] NOC: Loaded 41627 existing records from primary storage. [2025-10-20T22:12:44.515226] [INFO ] [MAIN ] NOC: Loaded 41627 primary data records. [2025-10-20T22:12:44.621424] [DEBUG] [LOCAL ] NOG: Loaded 51365 existing records from primary storage. [2025-10-20T22:12:44.647315] [INFO ] [MAIN ] NOG: Loaded 51365 primary data records. [2025-10-20T22:12:44.663910] [DEBUG] [LOCAL ] NOK: Loaded 53622 existing records from primary storage. [2025-10-20T22:12:44.685492] [INFO ] [MAIN ] NOK: Loaded 53622 primary data records. [2025-10-20T22:12:45.500662] [DEBUG] [MAIN ] NOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:45.501973] [INFO ] [MAIN ] NOC: Generated 1794 derivative records from 41627 primary records. [2025-10-20T22:12:45.503037] [DEBUG] [LOCAL ] NOC: Saving derivative data. [2025-10-20T22:12:45.519875] [DEBUG] [LOCAL ] NOC: Derivative data saved (1794 records). [2025-10-20T22:12:45.519991] [INFO ] [MAIN ] NOC: Successfully saved derivative data. [2025-10-20T22:12:45.521681] [INFO ] [MAIN ] Derivative Progress: 1291/1877 symbols completed. NOC took 1.1s. Estimated remaining: 3m 40s. Rate: 159.6 symbols/min [2025-10-20T22:12:45.522036] [INFO ] [MAIN ] NOMD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:45.542138] [DEBUG] [MAIN ] NOK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:45.544064] [INFO ] [MAIN ] NOK: Generated 1794 derivative records from 53622 primary records. [2025-10-20T22:12:45.544339] [DEBUG] [LOCAL ] NOK: Saving derivative data. [2025-10-20T22:12:45.558550] [DEBUG] [MAIN ] NOG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:45.567547] [INFO ] [MAIN ] NOG: Generated 1794 derivative records from 51365 primary records. [2025-10-20T22:12:45.567805] [DEBUG] [LOCAL ] NOG: Saving derivative data. [2025-10-20T22:12:45.575817] [DEBUG] [LOCAL ] NOK: Derivative data saved (1794 records). [2025-10-20T22:12:45.575941] [INFO ] [MAIN ] NOK: Successfully saved derivative data. [2025-10-20T22:12:45.578468] [INFO ] [MAIN ] Derivative Progress: 1292/1877 symbols completed. NOK took 1.1s. Estimated remaining: 3m 40s. Rate: 159.7 symbols/min [2025-10-20T22:12:45.598566] [INFO ] [MAIN ] NOTE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:45.603458] [DEBUG] [LOCAL ] NOG: Derivative data saved (1794 records). [2025-10-20T22:12:45.603567] [INFO ] [MAIN ] NOG: Successfully saved derivative data. [2025-10-20T22:12:45.606177] [INFO ] [MAIN ] Derivative Progress: 1293/1877 symbols completed. NOG took 1.2s. Estimated remaining: 3m 39s. Rate: 159.8 symbols/min [2025-10-20T22:12:45.606409] [INFO ] [MAIN ] NOV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:45.622731] [DEBUG] [LOCAL ] NOMD: Loaded 43107 existing records from primary storage. [2025-10-20T22:12:45.622867] [INFO ] [MAIN ] NOMD: Loaded 43107 primary data records. [2025-10-20T22:12:45.956055] [DEBUG] [LOCAL ] NOTE: Loaded 39914 existing records from primary storage. [2025-10-20T22:12:45.971070] [INFO ] [MAIN ] NOTE: Loaded 39914 primary data records. [2025-10-20T22:12:46.011846] [DEBUG] [LOCAL ] NOV: Loaded 53173 existing records from primary storage. [2025-10-20T22:12:46.039419] [INFO ] [MAIN ] NOV: Loaded 53173 primary data records. [2025-10-20T22:12:46.643224] [DEBUG] [MAIN ] NOMD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:46.644549] [INFO ] [MAIN ] NOMD: Generated 1794 derivative records from 43107 primary records. [2025-10-20T22:12:46.644871] [DEBUG] [LOCAL ] NOMD: Saving derivative data. [2025-10-20T22:12:46.664388] [DEBUG] [LOCAL ] NOMD: Derivative data saved (1794 records). [2025-10-20T22:12:46.664827] [INFO ] [MAIN ] NOMD: Successfully saved derivative data. [2025-10-20T22:12:46.666945] [INFO ] [MAIN ] Derivative Progress: 1294/1877 symbols completed. NOMD took 1.1s. Estimated remaining: 3m 39s. Rate: 159.6 symbols/min [2025-10-20T22:12:46.667259] [INFO ] [MAIN ] NOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:46.684561] [DEBUG] [MAIN ] NOTE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:46.686926] [INFO ] [MAIN ] NOTE: Generated 1794 derivative records from 39914 primary records. [2025-10-20T22:12:46.687838] [DEBUG] [LOCAL ] NOTE: Saving derivative data. [2025-10-20T22:12:46.707626] [DEBUG] [MAIN ] NOV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:46.713669] [INFO ] [MAIN ] NOV: Generated 1794 derivative records from 53173 primary records. [2025-10-20T22:12:46.716321] [DEBUG] [LOCAL ] NOV: Saving derivative data. [2025-10-20T22:12:46.716238] [DEBUG] [LOCAL ] NOTE: Derivative data saved (1794 records). [2025-10-20T22:12:46.716786] [INFO ] [MAIN ] NOTE: Successfully saved derivative data. [2025-10-20T22:12:46.718906] [INFO ] [MAIN ] Derivative Progress: 1295/1877 symbols completed. NOTE took 1.1s. Estimated remaining: 3m 39s. Rate: 159.7 symbols/min [2025-10-20T22:12:46.719327] [INFO ] [MAIN ] NPCT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:46.746974] [DEBUG] [LOCAL ] NOV: Derivative data saved (1794 records). [2025-10-20T22:12:46.747307] [INFO ] [MAIN ] NOV: Successfully saved derivative data. [2025-10-20T22:12:46.749676] [INFO ] [MAIN ] Derivative Progress: 1296/1877 symbols completed. NOV took 1.1s. Estimated remaining: 3m 38s. Rate: 159.8 symbols/min [2025-10-20T22:12:46.753677] [INFO ] [MAIN ] NPFD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:46.763104] [DEBUG] [LOCAL ] NPCT: Loaded 16647 existing records from primary storage. [2025-10-20T22:12:46.763484] [INFO ] [MAIN ] NPCT: Loaded 16647 primary data records. [2025-10-20T22:12:46.834036] [DEBUG] [LOCAL ] NOW: Loaded 47924 existing records from primary storage. [2025-10-20T22:12:46.850125] [INFO ] [MAIN ] NOW: Loaded 47924 primary data records. [2025-10-20T22:12:46.888974] [DEBUG] [LOCAL ] NPFD: Loaded 10026 existing records from primary storage. [2025-10-20T22:12:46.931584] [INFO ] [MAIN ] NPFD: Loaded 10026 primary data records. [2025-10-20T22:12:47.752625] [DEBUG] [MAIN ] NPCT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:47.758094] [DEBUG] [MAIN ] NOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:47.759469] [INFO ] [MAIN ] NPCT: Generated 1794 derivative records from 16647 primary records. [2025-10-20T22:12:47.761468] [INFO ] [MAIN ] NOW: Generated 1794 derivative records from 47924 primary records. [2025-10-20T22:12:47.760070] [DEBUG] [LOCAL ] NPCT: Saving derivative data. [2025-10-20T22:12:47.762423] [DEBUG] [LOCAL ] NOW: Saving derivative data. [2025-10-20T22:12:47.810674] [DEBUG] [LOCAL ] NPCT: Derivative data saved (1794 records). [2025-10-20T22:12:47.814849] [INFO ] [MAIN ] NPCT: Successfully saved derivative data. [2025-10-20T22:12:47.814693] [DEBUG] [LOCAL ] NOW: Derivative data saved (1794 records). [2025-10-20T22:12:47.816785] [INFO ] [MAIN ] Derivative Progress: 1297/1877 symbols completed. NPCT took 1.1s. Estimated remaining: 3m 38s. Rate: 159.6 symbols/min [2025-10-20T22:12:47.816885] [INFO ] [MAIN ] NOW: Successfully saved derivative data. [2025-10-20T22:12:47.817008] [INFO ] [MAIN ] NPK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:47.809614] [DEBUG] [MAIN ] NPFD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:47.818136] [INFO ] [MAIN ] NPFD: Generated 1794 derivative records from 10026 primary records. [2025-10-20T22:12:47.820331] [INFO ] [MAIN ] Derivative Progress: 1298/1877 symbols completed. NOW took 1.2s. Estimated remaining: 3m 38s. Rate: 159.7 symbols/min [2025-10-20T22:12:47.820861] [DEBUG] [LOCAL ] NPFD: Saving derivative data. [2025-10-20T22:12:47.821064] [INFO ] [MAIN ] NPKI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:47.839263] [DEBUG] [LOCAL ] NPFD: Derivative data saved (1794 records). [2025-10-20T22:12:47.839506] [INFO ] [MAIN ] NPFD: Successfully saved derivative data. [2025-10-20T22:12:47.841518] [INFO ] [MAIN ] Derivative Progress: 1299/1877 symbols completed. NPFD took 1.1s. Estimated remaining: 3m 37s. Rate: 159.8 symbols/min [2025-10-20T22:12:47.841780] [INFO ] [MAIN ] NPO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:47.846772] [DEBUG] [LOCAL ] NPK: Loaded 8500 existing records from primary storage. [2025-10-20T22:12:47.846964] [INFO ] [MAIN ] NPK: Loaded 8500 primary data records. [2025-10-20T22:12:47.987208] [DEBUG] [LOCAL ] NPO: Loaded 13626 existing records from primary storage. [2025-10-20T22:12:48.013799] [INFO ] [MAIN ] NPO: Loaded 13626 primary data records. [2025-10-20T22:12:48.240459] [DEBUG] [LOCAL ] NPKI: Loaded 40492 existing records from primary storage. [2025-10-20T22:12:48.265759] [INFO ] [MAIN ] NPKI: Loaded 40492 primary data records. [2025-10-20T22:12:48.829964] [DEBUG] [MAIN ] NPK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:48.831426] [INFO ] [MAIN ] NPK: Generated 1794 derivative records from 8500 primary records. [2025-10-20T22:12:48.831663] [DEBUG] [LOCAL ] NPK: Saving derivative data. [2025-10-20T22:12:48.850917] [DEBUG] [LOCAL ] NPK: Derivative data saved (1794 records). [2025-10-20T22:12:48.851244] [INFO ] [MAIN ] NPK: Successfully saved derivative data. [2025-10-20T22:12:48.853287] [INFO ] [MAIN ] Derivative Progress: 1300/1877 symbols completed. NPK took 1.0s. Estimated remaining: 3m 37s. Rate: 159.6 symbols/min [2025-10-20T22:12:48.861120] [INFO ] [MAIN ] NPV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:48.866918] [DEBUG] [MAIN ] NPO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:48.867589] [INFO ] [MAIN ] NPO: Generated 1794 derivative records from 13626 primary records. [2025-10-20T22:12:48.867891] [DEBUG] [LOCAL ] NPO: Saving derivative data. [2025-10-20T22:12:48.890611] [DEBUG] [LOCAL ] NPO: Derivative data saved (1794 records). [2025-10-20T22:12:48.895339] [INFO ] [MAIN ] NPO: Successfully saved derivative data. [2025-10-20T22:12:48.895263] [DEBUG] [LOCAL ] NPV: Loaded 5369 existing records from primary storage. [2025-10-20T22:12:48.897494] [INFO ] [MAIN ] Derivative Progress: 1301/1877 symbols completed. NPO took 1.1s. Estimated remaining: 3m 36s. Rate: 159.7 symbols/min [2025-10-20T22:12:48.897996] [INFO ] [MAIN ] NPV: Loaded 5369 primary data records. [2025-10-20T22:12:48.897852] [INFO ] [MAIN ] NPWR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:48.980190] [DEBUG] [MAIN ] NPKI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:49.013807] [INFO ] [MAIN ] NPKI: Generated 1794 derivative records from 40492 primary records. [2025-10-20T22:12:49.029538] [DEBUG] [LOCAL ] NPKI: Saving derivative data. [2025-10-20T22:12:49.105549] [DEBUG] [LOCAL ] NPKI: Derivative data saved (1794 records). [2025-10-20T22:12:49.205722] [INFO ] [MAIN ] NPKI: Successfully saved derivative data. [2025-10-20T22:12:49.230506] [INFO ] [MAIN ] Derivative Progress: 1302/1877 symbols completed. NPKI took 1.4s. Estimated remaining: 3m 36s. Rate: 159.7 symbols/min [2025-10-20T22:12:49.230859] [INFO ] [MAIN ] NQP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:49.252580] [DEBUG] [LOCAL ] NPWR: Loaded 43398 existing records from primary storage. [2025-10-20T22:12:49.253765] [INFO ] [MAIN ] NPWR: Loaded 43398 primary data records. [2025-10-20T22:12:49.289079] [DEBUG] [LOCAL ] NQP: Loaded 9208 existing records from primary storage. [2025-10-20T22:12:49.294517] [INFO ] [MAIN ] NQP: Loaded 9208 primary data records. [2025-10-20T22:12:49.848998] [DEBUG] [MAIN ] NPV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:49.864063] [INFO ] [MAIN ] NPV: Generated 1794 derivative records from 5369 primary records. [2025-10-20T22:12:49.864810] [DEBUG] [LOCAL ] NPV: Saving derivative data. [2025-10-20T22:12:49.885379] [DEBUG] [LOCAL ] NPV: Derivative data saved (1794 records). [2025-10-20T22:12:49.886175] [INFO ] [MAIN ] NPV: Successfully saved derivative data. [2025-10-20T22:12:49.888286] [INFO ] [MAIN ] Derivative Progress: 1303/1877 symbols completed. NPV took 1.0s. Estimated remaining: 3m 36s. Rate: 159.7 symbols/min [2025-10-20T22:12:49.907631] [DEBUG] [MAIN ] NPWR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:49.909120] [INFO ] [MAIN ] NPWR: Generated 1794 derivative records from 43398 primary records. [2025-10-20T22:12:49.909803] [DEBUG] [LOCAL ] NPWR: Saving derivative data. [2025-10-20T22:12:49.933901] [DEBUG] [MAIN ] NQP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:49.936299] [INFO ] [MAIN ] NQP: Generated 1794 derivative records from 9208 primary records. [2025-10-20T22:12:49.936571] [DEBUG] [LOCAL ] NQP: Saving derivative data. [2025-10-20T22:12:49.945860] [DEBUG] [LOCAL ] NPWR: Derivative data saved (1794 records). [2025-10-20T22:12:49.949207] [INFO ] [MAIN ] NPWR: Successfully saved derivative data. [2025-10-20T22:12:49.951560] [INFO ] [MAIN ] Derivative Progress: 1304/1877 symbols completed. NPWR took 1.1s. Estimated remaining: 3m 35s. Rate: 159.8 symbols/min [2025-10-20T22:12:49.955504] [DEBUG] [LOCAL ] NQP: Derivative data saved (1794 records). [2025-10-20T22:12:49.955739] [INFO ] [MAIN ] NQP: Successfully saved derivative data. [2025-10-20T22:12:49.958058] [INFO ] [MAIN ] Derivative Progress: 1305/1877 symbols completed. NQP took 0.7s. Estimated remaining: 3m 35s. Rate: 159.9 symbols/min [2025-10-20T22:12:49.958496] [INFO ] [MAIN ] NRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:49.959023] [INFO ] [MAIN ] NREF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:49.958436] [INFO ] [MAIN ] NRDY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:49.981331] [DEBUG] [LOCAL ] NREF: Loaded 7833 existing records from primary storage. [2025-10-20T22:12:49.981449] [INFO ] [MAIN ] NREF: Loaded 7833 primary data records. [2025-10-20T22:12:50.053850] [DEBUG] [LOCAL ] NRK: Loaded 20401 existing records from primary storage. [2025-10-20T22:12:50.077915] [INFO ] [MAIN ] NRK: Loaded 20401 primary data records. [2025-10-20T22:12:50.286608] [DEBUG] [LOCAL ] NRDY: Loaded 32396 existing records from primary storage. [2025-10-20T22:12:50.422904] [INFO ] [MAIN ] NRDY: Loaded 32396 primary data records. [2025-10-20T22:12:50.965018] [DEBUG] [MAIN ] NREF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:50.966300] [INFO ] [MAIN ] NREF: Generated 1794 derivative records from 7833 primary records. [2025-10-20T22:12:50.966510] [DEBUG] [LOCAL ] NREF: Saving derivative data. [2025-10-20T22:12:50.979857] [DEBUG] [MAIN ] NRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:50.992484] [INFO ] [MAIN ] NRK: Generated 1794 derivative records from 20401 primary records. [2025-10-20T22:12:50.998331] [DEBUG] [LOCAL ] NRK: Saving derivative data. [2025-10-20T22:12:51.000557] [DEBUG] [LOCAL ] NREF: Derivative data saved (1794 records). [2025-10-20T22:12:51.011165] [INFO ] [MAIN ] NREF: Successfully saved derivative data. [2025-10-20T22:12:51.015940] [INFO ] [MAIN ] Derivative Progress: 1306/1877 symbols completed. NREF took 1.1s. Estimated remaining: 3m 35s. Rate: 159.7 symbols/min [2025-10-20T22:12:51.016756] [INFO ] [MAIN ] NRP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:51.026009] [DEBUG] [LOCAL ] NRK: Derivative data saved (1794 records). [2025-10-20T22:12:51.026134] [INFO ] [MAIN ] NRK: Successfully saved derivative data. [2025-10-20T22:12:51.034113] [DEBUG] [MAIN ] NRDY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:51.033755] [INFO ] [MAIN ] Derivative Progress: 1307/1877 symbols completed. NRK took 1.1s. Estimated remaining: 3m 34s. Rate: 159.8 symbols/min [2025-10-20T22:12:51.035476] [INFO ] [MAIN ] NRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:51.036381] [INFO ] [MAIN ] NRDY: Generated 1794 derivative records from 32396 primary records. [2025-10-20T22:12:51.036893] [DEBUG] [LOCAL ] NRDY: Saving derivative data. [2025-10-20T22:12:51.040382] [DEBUG] [LOCAL ] NRP: Loaded 4208 existing records from primary storage. [2025-10-20T22:12:51.040801] [INFO ] [MAIN ] NRP: Loaded 4208 primary data records. [2025-10-20T22:12:51.138710] [DEBUG] [LOCAL ] NRDY: Derivative data saved (1794 records). [2025-10-20T22:12:51.143934] [INFO ] [MAIN ] NRDY: Successfully saved derivative data. [2025-10-20T22:12:51.133528] [DEBUG] [LOCAL ] NRT: Loaded 4894 existing records from primary storage. [2025-10-20T22:12:51.149462] [INFO ] [MAIN ] NRT: Loaded 4894 primary data records. [2025-10-20T22:12:51.151848] [INFO ] [MAIN ] Derivative Progress: 1308/1877 symbols completed. NRDY took 1.2s. Estimated remaining: 3m 34s. Rate: 159.9 symbols/min [2025-10-20T22:12:51.197612] [INFO ] [MAIN ] NRUC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:51.366403] [DEBUG] [LOCAL ] NRUC: Loaded 3117 existing records from primary storage. [2025-10-20T22:12:51.507844] [INFO ] [MAIN ] NRUC: Loaded 3117 primary data records. [2025-10-20T22:12:51.984072] [DEBUG] [MAIN ] NRP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:51.985149] [INFO ] [MAIN ] NRP: Generated 1794 derivative records from 4208 primary records. [2025-10-20T22:12:51.992763] [DEBUG] [LOCAL ] NRP: Saving derivative data. [2025-10-20T22:12:51.998555] [DEBUG] [MAIN ] NRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:51.999349] [INFO ] [MAIN ] NRT: Generated 1794 derivative records from 4894 primary records. [2025-10-20T22:12:51.999592] [DEBUG] [LOCAL ] NRT: Saving derivative data. [2025-10-20T22:12:52.026883] [DEBUG] [LOCAL ] NRT: Derivative data saved (1794 records). [2025-10-20T22:12:52.028055] [INFO ] [MAIN ] NRT: Successfully saved derivative data. [2025-10-20T22:12:52.030292] [INFO ] [MAIN ] Derivative Progress: 1309/1877 symbols completed. NRT took 1.0s. Estimated remaining: 3m 33s. Rate: 159.7 symbols/min [2025-10-20T22:12:52.031994] [INFO ] [MAIN ] NTB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:52.033793] [DEBUG] [LOCAL ] NRP: Derivative data saved (1794 records). [2025-10-20T22:12:52.034261] [INFO ] [MAIN ] NRP: Successfully saved derivative data. [2025-10-20T22:12:52.037114] [INFO ] [MAIN ] Derivative Progress: 1310/1877 symbols completed. NRP took 1.0s. Estimated remaining: 3m 33s. Rate: 159.8 symbols/min [2025-10-20T22:12:52.038723] [INFO ] [MAIN ] NTST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:52.054557] [DEBUG] [MAIN ] NRUC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:52.060370] [INFO ] [MAIN ] NRUC: Generated 1794 derivative records from 3117 primary records. [2025-10-20T22:12:52.060502] [DEBUG] [LOCAL ] NRUC: Saving derivative data. [2025-10-20T22:12:52.083184] [DEBUG] [LOCAL ] NTB: Loaded 20386 existing records from primary storage. [2025-10-20T22:12:52.082699] [DEBUG] [LOCAL ] NRUC: Derivative data saved (1794 records). [2025-10-20T22:12:52.084151] [INFO ] [MAIN ] NTB: Loaded 20386 primary data records. [2025-10-20T22:12:52.084395] [INFO ] [MAIN ] NRUC: Successfully saved derivative data. [2025-10-20T22:12:52.097981] [INFO ] [MAIN ] Derivative Progress: 1311/1877 symbols completed. NRUC took 0.9s. Estimated remaining: 3m 32s. Rate: 159.9 symbols/min [2025-10-20T22:12:52.131032] [INFO ] [MAIN ] NUV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:52.429736] [DEBUG] [LOCAL ] NTST: Loaded 44149 existing records from primary storage. [2025-10-20T22:12:52.430504] [INFO ] [MAIN ] NTST: Loaded 44149 primary data records. [2025-10-20T22:12:52.527248] [DEBUG] [LOCAL ] NUV: Loaded 23604 existing records from primary storage. [2025-10-20T22:12:52.544908] [INFO ] [MAIN ] NUV: Loaded 23604 primary data records. [2025-10-20T22:12:53.056128] [DEBUG] [MAIN ] NTB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:53.056948] [INFO ] [MAIN ] NTB: Generated 1794 derivative records from 20386 primary records. [2025-10-20T22:12:53.057678] [DEBUG] [LOCAL ] NTB: Saving derivative data. [2025-10-20T22:12:53.075027] [DEBUG] [LOCAL ] NTB: Derivative data saved (1794 records). [2025-10-20T22:12:53.075256] [INFO ] [MAIN ] NTB: Successfully saved derivative data. [2025-10-20T22:12:53.078257] [INFO ] [MAIN ] Derivative Progress: 1312/1877 symbols completed. NTB took 1.0s. Estimated remaining: 3m 32s. Rate: 159.7 symbols/min [2025-10-20T22:12:53.078999] [INFO ] [MAIN ] NUW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:53.097736] [DEBUG] [MAIN ] NTST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:53.101395] [INFO ] [MAIN ] NTST: Generated 1794 derivative records from 44149 primary records. [2025-10-20T22:12:53.101537] [DEBUG] [LOCAL ] NTST: Saving derivative data. [2025-10-20T22:12:53.105253] [DEBUG] [LOCAL ] NUW: Loaded 6681 existing records from primary storage. [2025-10-20T22:12:53.108123] [INFO ] [MAIN ] NUW: Loaded 6681 primary data records. [2025-10-20T22:12:53.214096] [DEBUG] [LOCAL ] NTST: Derivative data saved (1794 records). [2025-10-20T22:12:53.220075] [INFO ] [MAIN ] NTST: Successfully saved derivative data. [2025-10-20T22:12:53.219952] [DEBUG] [MAIN ] NUV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:53.242248] [INFO ] [MAIN ] NUV: Generated 1794 derivative records from 23604 primary records. [2025-10-20T22:12:53.233533] [INFO ] [MAIN ] Derivative Progress: 1313/1877 symbols completed. NTST took 1.2s. Estimated remaining: 3m 32s. Rate: 159.8 symbols/min [2025-10-20T22:12:53.246130] [DEBUG] [LOCAL ] NUV: Saving derivative data. [2025-10-20T22:12:53.246244] [INFO ] [MAIN ] NVG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:53.336420] [DEBUG] [LOCAL ] NUV: Derivative data saved (1794 records). [2025-10-20T22:12:53.340044] [INFO ] [MAIN ] NUV: Successfully saved derivative data. [2025-10-20T22:12:53.343920] [INFO ] [MAIN ] Derivative Progress: 1314/1877 symbols completed. NUV took 1.2s. Estimated remaining: 3m 31s. Rate: 159.9 symbols/min [2025-10-20T22:12:53.448070] [INFO ] [MAIN ] NVGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:53.490433] [DEBUG] [LOCAL ] NVG: Loaded 27492 existing records from primary storage. [2025-10-20T22:12:53.491391] [INFO ] [MAIN ] NVG: Loaded 27492 primary data records. [2025-10-20T22:12:53.540105] [DEBUG] [MAIN ] NUW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:53.567534] [INFO ] [MAIN ] NUW: Generated 1794 derivative records from 6681 primary records. [2025-10-20T22:12:53.588799] [DEBUG] [LOCAL ] NUW: Saving derivative data. [2025-10-20T22:12:53.605588] [DEBUG] [LOCAL ] NVGS: Loaded 30187 existing records from primary storage. [2025-10-20T22:12:53.622862] [INFO ] [MAIN ] NVGS: Loaded 30187 primary data records. [2025-10-20T22:12:53.662022] [DEBUG] [LOCAL ] NUW: Derivative data saved (1794 records). [2025-10-20T22:12:53.713206] [INFO ] [MAIN ] NUW: Successfully saved derivative data. [2025-10-20T22:12:53.735436] [INFO ] [MAIN ] Derivative Progress: 1315/1877 symbols completed. NUW took 0.7s. Estimated remaining: 3m 31s. Rate: 159.9 symbols/min [2025-10-20T22:12:53.784619] [INFO ] [MAIN ] NVO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:54.206382] [DEBUG] [MAIN ] NVG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:54.207307] [INFO ] [MAIN ] NVG: Generated 1794 derivative records from 27492 primary records. [2025-10-20T22:12:54.207546] [DEBUG] [LOCAL ] NVG: Saving derivative data. [2025-10-20T22:12:54.240441] [DEBUG] [LOCAL ] NVG: Derivative data saved (1794 records). [2025-10-20T22:12:54.241632] [INFO ] [MAIN ] NVG: Successfully saved derivative data. [2025-10-20T22:12:54.243553] [INFO ] [MAIN ] Derivative Progress: 1316/1877 symbols completed. NVG took 1.0s. Estimated remaining: 3m 31s. Rate: 159.8 symbols/min [2025-10-20T22:12:54.240327] [DEBUG] [LOCAL ] NVO: Loaded 53784 existing records from primary storage. [2025-10-20T22:12:54.243929] [INFO ] [MAIN ] NVO: Loaded 53784 primary data records. [2025-10-20T22:12:54.243792] [INFO ] [MAIN ] NVS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:54.341797] [DEBUG] [MAIN ] NVGS: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:12:54.366492] [INFO ] [MAIN ] NVGS: Generated 1793 derivative records from 30187 primary records. [2025-10-20T22:12:54.369976] [DEBUG] [LOCAL ] NVGS: Saving derivative data. [2025-10-20T22:12:54.591218] [DEBUG] [LOCAL ] NVGS: Derivative data saved (1793 records). [2025-10-20T22:12:54.591520] [INFO ] [MAIN ] NVGS: Successfully saved derivative data. [2025-10-20T22:12:54.593769] [INFO ] [MAIN ] Derivative Progress: 1317/1877 symbols completed. NVGS took 1.1s. Estimated remaining: 3m 30s. Rate: 159.8 symbols/min [2025-10-20T22:12:54.594003] [INFO ] [MAIN ] NXC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:54.602253] [DEBUG] [LOCAL ] NXC: Loaded 4121 existing records from primary storage. [2025-10-20T22:12:54.602621] [INFO ] [MAIN ] NXC: Loaded 4121 primary data records. [2025-10-20T22:12:54.711118] [DEBUG] [LOCAL ] NVS: Loaded 51400 existing records from primary storage. [2025-10-20T22:12:54.722527] [INFO ] [MAIN ] NVS: Loaded 51400 primary data records. [2025-10-20T22:12:55.202955] [DEBUG] [MAIN ] NVO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:55.216877] [INFO ] [MAIN ] NVO: Generated 1794 derivative records from 53784 primary records. [2025-10-20T22:12:55.217364] [DEBUG] [LOCAL ] NVO: Saving derivative data. [2025-10-20T22:12:55.236470] [DEBUG] [LOCAL ] NVO: Derivative data saved (1794 records). [2025-10-20T22:12:55.236705] [INFO ] [MAIN ] NVO: Successfully saved derivative data. [2025-10-20T22:12:55.238778] [INFO ] [MAIN ] Derivative Progress: 1318/1877 symbols completed. NVO took 1.5s. Estimated remaining: 3m 30s. Rate: 159.7 symbols/min [2025-10-20T22:12:55.258169] [DEBUG] [MAIN ] NXC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:55.259034] [INFO ] [MAIN ] NXC: Generated 1794 derivative records from 4121 primary records. [2025-10-20T22:12:55.259240] [DEBUG] [LOCAL ] NXC: Saving derivative data. [2025-10-20T22:12:55.287212] [DEBUG] [LOCAL ] NXC: Derivative data saved (1794 records). [2025-10-20T22:12:55.287382] [DEBUG] [MAIN ] NVS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:55.288064] [INFO ] [MAIN ] NVS: Generated 1794 derivative records from 51400 primary records. [2025-10-20T22:12:55.288143] [DEBUG] [LOCAL ] NVS: Saving derivative data. [2025-10-20T22:12:55.288701] [INFO ] [MAIN ] NXC: Successfully saved derivative data. [2025-10-20T22:12:55.290731] [INFO ] [MAIN ] Derivative Progress: 1319/1877 symbols completed. NXC took 0.7s. Estimated remaining: 3m 29s. Rate: 159.8 symbols/min [2025-10-20T22:12:55.305294] [DEBUG] [LOCAL ] NVS: Derivative data saved (1794 records). [2025-10-20T22:12:55.305358] [INFO ] [MAIN ] NVS: Successfully saved derivative data. [2025-10-20T22:12:55.307488] [INFO ] [MAIN ] Derivative Progress: 1320/1877 symbols completed. NVS took 1.1s. Estimated remaining: 3m 29s. Rate: 160.0 symbols/min [2025-10-20T22:12:55.308099] [INFO ] [MAIN ] NXDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:55.308162] [INFO ] [MAIN ] NXG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:55.308243] [INFO ] [MAIN ] NXJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:55.323979] [DEBUG] [LOCAL ] NXG: Loaded 10497 existing records from primary storage. [2025-10-20T22:12:55.327450] [INFO ] [MAIN ] NXG: Loaded 10497 primary data records. [2025-10-20T22:12:55.412127] [DEBUG] [LOCAL ] NXDT: Loaded 14961 existing records from primary storage. [2025-10-20T22:12:55.419559] [DEBUG] [LOCAL ] NXJ: Loaded 14894 existing records from primary storage. [2025-10-20T22:12:55.419814] [INFO ] [MAIN ] NXJ: Loaded 14894 primary data records. [2025-10-20T22:12:55.419990] [INFO ] [MAIN ] NXDT: Loaded 14961 primary data records. [2025-10-20T22:12:56.297042] [DEBUG] [MAIN ] NXG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:56.300848] [INFO ] [MAIN ] NXG: Generated 1794 derivative records from 10497 primary records. [2025-10-20T22:12:56.301136] [DEBUG] [LOCAL ] NXG: Saving derivative data. [2025-10-20T22:12:56.320759] [DEBUG] [LOCAL ] NXG: Derivative data saved (1794 records). [2025-10-20T22:12:56.321604] [INFO ] [MAIN ] NXG: Successfully saved derivative data. [2025-10-20T22:12:56.331767] [INFO ] [MAIN ] Derivative Progress: 1321/1877 symbols completed. NXG took 1.0s. Estimated remaining: 3m 29s. Rate: 159.8 symbols/min [2025-10-20T22:12:56.331919] [INFO ] [MAIN ] NXN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:56.350348] [DEBUG] [MAIN ] NXJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:56.351675] [DEBUG] [MAIN ] NXDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:56.352507] [INFO ] [MAIN ] NXDT: Generated 1794 derivative records from 14961 primary records. [2025-10-20T22:12:56.351326] [INFO ] [MAIN ] NXJ: Generated 1794 derivative records from 14894 primary records. [2025-10-20T22:12:56.353205] [DEBUG] [LOCAL ] NXDT: Saving derivative data. [2025-10-20T22:12:56.354167] [DEBUG] [LOCAL ] NXJ: Saving derivative data. [2025-10-20T22:12:56.363693] [DEBUG] [LOCAL ] NXN: Loaded 2284 existing records from primary storage. [2025-10-20T22:12:56.367676] [INFO ] [MAIN ] NXN: Loaded 2284 primary data records. [2025-10-20T22:12:56.423054] [DEBUG] [LOCAL ] NXDT: Derivative data saved (1794 records). [2025-10-20T22:12:56.436343] [DEBUG] [LOCAL ] NXJ: Derivative data saved (1794 records). [2025-10-20T22:12:56.444118] [INFO ] [MAIN ] NXJ: Successfully saved derivative data. [2025-10-20T22:12:56.458481] [INFO ] [MAIN ] NXDT: Successfully saved derivative data. [2025-10-20T22:12:56.460849] [INFO ] [MAIN ] Derivative Progress: 1322/1877 symbols completed. NXJ took 1.2s. Estimated remaining: 3m 28s. Rate: 159.8 symbols/min [2025-10-20T22:12:56.474891] [INFO ] [MAIN ] Derivative Progress: 1323/1877 symbols completed. NXDT took 1.2s. Estimated remaining: 3m 28s. Rate: 159.9 symbols/min [2025-10-20T22:12:56.467567] [INFO ] [MAIN ] NXP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:56.494858] [INFO ] [MAIN ] NZF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:56.708889] [DEBUG] [LOCAL ] NXP: Loaded 12235 existing records from primary storage. [2025-10-20T22:12:56.709062] [INFO ] [MAIN ] NXP: Loaded 12235 primary data records. [2025-10-20T22:12:56.789679] [DEBUG] [LOCAL ] NZF: Loaded 28787 existing records from primary storage. [2025-10-20T22:12:56.812137] [INFO ] [MAIN ] NZF: Loaded 28787 primary data records. [2025-10-20T22:12:57.330738] [DEBUG] [MAIN ] NXN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:57.332686] [INFO ] [MAIN ] NXN: Generated 1794 derivative records from 2284 primary records. [2025-10-20T22:12:57.332786] [DEBUG] [LOCAL ] NXN: Saving derivative data. [2025-10-20T22:12:57.348042] [DEBUG] [LOCAL ] NXN: Derivative data saved (1794 records). [2025-10-20T22:12:57.348271] [INFO ] [MAIN ] NXN: Successfully saved derivative data. [2025-10-20T22:12:57.350331] [INFO ] [MAIN ] Derivative Progress: 1324/1877 symbols completed. NXN took 1.0s. Estimated remaining: 3m 28s. Rate: 159.8 symbols/min [2025-10-20T22:12:57.350635] [INFO ] [MAIN ] OBDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:57.368582] [DEBUG] [MAIN ] NXP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:57.369478] [INFO ] [MAIN ] NXP: Generated 1794 derivative records from 12235 primary records. [2025-10-20T22:12:57.369721] [DEBUG] [LOCAL ] NXP: Saving derivative data. [2025-10-20T22:12:57.385805] [DEBUG] [LOCAL ] NXP: Derivative data saved (1794 records). [2025-10-20T22:12:57.386719] [INFO ] [MAIN ] NXP: Successfully saved derivative data. [2025-10-20T22:12:57.389737] [INFO ] [MAIN ] Derivative Progress: 1325/1877 symbols completed. NXP took 0.9s. Estimated remaining: 3m 27s. Rate: 159.9 symbols/min [2025-10-20T22:12:57.390928] [INFO ] [MAIN ] OBK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:57.404569] [DEBUG] [MAIN ] NZF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:57.405303] [INFO ] [MAIN ] NZF: Generated 1794 derivative records from 28787 primary records. [2025-10-20T22:12:57.405422] [DEBUG] [LOCAL ] NZF: Saving derivative data. [2025-10-20T22:12:57.433973] [DEBUG] [LOCAL ] NZF: Derivative data saved (1794 records). [2025-10-20T22:12:57.437588] [INFO ] [MAIN ] NZF: Successfully saved derivative data. [2025-10-20T22:12:57.441251] [INFO ] [MAIN ] Derivative Progress: 1326/1877 symbols completed. NZF took 0.9s. Estimated remaining: 3m 27s. Rate: 160.0 symbols/min [2025-10-20T22:12:57.441563] [INFO ] [MAIN ] OCFT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:57.450338] [DEBUG] [LOCAL ] OBK: Loaded 16500 existing records from primary storage. [2025-10-20T22:12:57.452576] [INFO ] [MAIN ] OBK: Loaded 16500 primary data records. [2025-10-20T22:12:57.537795] [DEBUG] [LOCAL ] OBDC: Loaded 51730 existing records from primary storage. [2025-10-20T22:12:57.552395] [DEBUG] [LOCAL ] OCFT: Loaded 7276 existing records from primary storage. [2025-10-20T22:12:57.563285] [INFO ] [MAIN ] OBDC: Loaded 51730 primary data records. [2025-10-20T22:12:57.571433] [INFO ] [MAIN ] OCFT: Loaded 7276 primary data records. [2025-10-20T22:12:58.435788] [DEBUG] [MAIN ] OBK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:58.436878] [INFO ] [MAIN ] OBK: Generated 1794 derivative records from 16500 primary records. [2025-10-20T22:12:58.437160] [DEBUG] [LOCAL ] OBK: Saving derivative data. [2025-10-20T22:12:58.463730] [DEBUG] [LOCAL ] OBK: Derivative data saved (1794 records). [2025-10-20T22:12:58.470953] [DEBUG] [MAIN ] OCFT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:58.472356] [INFO ] [MAIN ] OCFT: Generated 1794 derivative records from 7276 primary records. [2025-10-20T22:12:58.469791] [INFO ] [MAIN ] OBK: Successfully saved derivative data. [2025-10-20T22:12:58.472727] [DEBUG] [LOCAL ] OCFT: Saving derivative data. [2025-10-20T22:12:58.475547] [INFO ] [MAIN ] Derivative Progress: 1327/1877 symbols completed. OBK took 1.1s. Estimated remaining: 3m 27s. Rate: 159.8 symbols/min [2025-10-20T22:12:58.484111] [INFO ] [MAIN ] ODC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:58.497583] [DEBUG] [MAIN ] OBDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:58.504029] [DEBUG] [LOCAL ] OCFT: Derivative data saved (1794 records). [2025-10-20T22:12:58.504917] [INFO ] [MAIN ] OBDC: Generated 1794 derivative records from 51730 primary records. [2025-10-20T22:12:58.505181] [DEBUG] [LOCAL ] OBDC: Saving derivative data. [2025-10-20T22:12:58.505303] [INFO ] [MAIN ] OCFT: Successfully saved derivative data. [2025-10-20T22:12:58.507878] [INFO ] [MAIN ] Derivative Progress: 1328/1877 symbols completed. OCFT took 1.1s. Estimated remaining: 3m 26s. Rate: 159.9 symbols/min [2025-10-20T22:12:58.510290] [INFO ] [MAIN ] OEC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:58.529457] [DEBUG] [LOCAL ] ODC: Loaded 8628 existing records from primary storage. [2025-10-20T22:12:58.532116] [INFO ] [MAIN ] ODC: Loaded 8628 primary data records. [2025-10-20T22:12:58.556401] [DEBUG] [LOCAL ] OBDC: Derivative data saved (1794 records). [2025-10-20T22:12:58.572293] [INFO ] [MAIN ] OBDC: Successfully saved derivative data. [2025-10-20T22:12:58.593346] [INFO ] [MAIN ] Derivative Progress: 1329/1877 symbols completed. OBDC took 1.2s. Estimated remaining: 3m 26s. Rate: 160.0 symbols/min [2025-10-20T22:12:58.593598] [INFO ] [MAIN ] OIA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:58.845891] [DEBUG] [LOCAL ] OIA: Loaded 13084 existing records from primary storage. [2025-10-20T22:12:58.859198] [INFO ] [MAIN ] OIA: Loaded 13084 primary data records. [2025-10-20T22:12:58.930868] [DEBUG] [LOCAL ] OEC: Loaded 37290 existing records from primary storage. [2025-10-20T22:12:58.945460] [INFO ] [MAIN ] OEC: Loaded 37290 primary data records. [2025-10-20T22:12:59.516956] [DEBUG] [MAIN ] ODC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:59.518896] [INFO ] [MAIN ] ODC: Generated 1794 derivative records from 8628 primary records. [2025-10-20T22:12:59.519089] [DEBUG] [LOCAL ] ODC: Saving derivative data. [2025-10-20T22:12:59.534569] [DEBUG] [LOCAL ] ODC: Derivative data saved (1794 records). [2025-10-20T22:12:59.534749] [INFO ] [MAIN ] ODC: Successfully saved derivative data. [2025-10-20T22:12:59.538428] [INFO ] [MAIN ] Derivative Progress: 1330/1877 symbols completed. ODC took 1.1s. Estimated remaining: 3m 25s. Rate: 159.8 symbols/min [2025-10-20T22:12:59.539311] [INFO ] [MAIN ] OIS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:59.554127] [DEBUG] [MAIN ] OIA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:59.554864] [INFO ] [MAIN ] OIA: Generated 1794 derivative records from 13084 primary records. [2025-10-20T22:12:59.555055] [DEBUG] [LOCAL ] OIA: Saving derivative data. [2025-10-20T22:12:59.572051] [DEBUG] [LOCAL ] OIA: Derivative data saved (1794 records). [2025-10-20T22:12:59.572287] [INFO ] [MAIN ] OIA: Successfully saved derivative data. [2025-10-20T22:12:59.574283] [INFO ] [MAIN ] Derivative Progress: 1331/1877 symbols completed. OIA took 1.0s. Estimated remaining: 3m 25s. Rate: 159.9 symbols/min [2025-10-20T22:12:59.574517] [INFO ] [MAIN ] OKLO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:59.588003] [DEBUG] [MAIN ] OEC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:12:59.588673] [INFO ] [MAIN ] OEC: Generated 1794 derivative records from 37290 primary records. [2025-10-20T22:12:59.588750] [DEBUG] [LOCAL ] OEC: Saving derivative data. [2025-10-20T22:12:59.613586] [DEBUG] [LOCAL ] OEC: Derivative data saved (1794 records). [2025-10-20T22:12:59.613915] [INFO ] [MAIN ] OEC: Successfully saved derivative data. [2025-10-20T22:12:59.615874] [INFO ] [MAIN ] Derivative Progress: 1332/1877 symbols completed. OEC took 1.1s. Estimated remaining: 3m 24s. Rate: 160.0 symbols/min [2025-10-20T22:12:59.616117] [INFO ] [MAIN ] OLP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:12:59.630539] [DEBUG] [LOCAL ] OIS: Loaded 39917 existing records from primary storage. [2025-10-20T22:12:59.630708] [INFO ] [MAIN ] OIS: Loaded 39917 primary data records. [2025-10-20T22:12:59.754952] [DEBUG] [LOCAL ] OLP: Loaded 10116 existing records from primary storage. [2025-10-20T22:12:59.760630] [INFO ] [MAIN ] OLP: Loaded 10116 primary data records. [2025-10-20T22:13:00.088364] [DEBUG] [LOCAL ] OKLO: Loaded 53819 existing records from primary storage. [2025-10-20T22:13:00.110624] [INFO ] [MAIN ] OKLO: Loaded 53819 primary data records. [2025-10-20T22:13:00.595804] [DEBUG] [MAIN ] OIS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:00.597421] [INFO ] [MAIN ] OIS: Generated 1794 derivative records from 39917 primary records. [2025-10-20T22:13:00.599117] [DEBUG] [LOCAL ] OIS: Saving derivative data. [2025-10-20T22:13:00.627052] [DEBUG] [MAIN ] OLP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:00.626858] [DEBUG] [LOCAL ] OIS: Derivative data saved (1794 records). [2025-10-20T22:13:00.627844] [INFO ] [MAIN ] OLP: Generated 1794 derivative records from 10116 primary records. [2025-10-20T22:13:00.627928] [INFO ] [MAIN ] OIS: Successfully saved derivative data. [2025-10-20T22:13:00.630381] [INFO ] [MAIN ] Derivative Progress: 1333/1877 symbols completed. OIS took 1.1s. Estimated remaining: 3m 24s. Rate: 159.8 symbols/min [2025-10-20T22:13:00.628103] [DEBUG] [LOCAL ] OLP: Saving derivative data. [2025-10-20T22:13:00.660304] [DEBUG] [LOCAL ] OLP: Derivative data saved (1794 records). [2025-10-20T22:13:00.659595] [DEBUG] [MAIN ] OKLO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:00.660394] [INFO ] [MAIN ] OLP: Successfully saved derivative data. [2025-10-20T22:13:00.662362] [INFO ] [MAIN ] Derivative Progress: 1334/1877 symbols completed. OLP took 1.0s. Estimated remaining: 3m 24s. Rate: 159.9 symbols/min [2025-10-20T22:13:00.663105] [INFO ] [MAIN ] OKLO: Generated 1794 derivative records from 53819 primary records. [2025-10-20T22:13:00.663190] [DEBUG] [LOCAL ] OKLO: Saving derivative data. [2025-10-20T22:13:00.678449] [DEBUG] [LOCAL ] OKLO: Derivative data saved (1794 records). [2025-10-20T22:13:00.678505] [INFO ] [MAIN ] OKLO: Successfully saved derivative data. [2025-10-20T22:13:00.680448] [INFO ] [MAIN ] Derivative Progress: 1335/1877 symbols completed. OKLO took 1.1s. Estimated remaining: 3m 23s. Rate: 160.0 symbols/min [2025-10-20T22:13:00.681106] [INFO ] [MAIN ] OMF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:00.681042] [INFO ] [MAIN ] ONL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:00.680873] [INFO ] [MAIN ] ONIT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:00.707040] [DEBUG] [LOCAL ] ONIT: Loaded 7641 existing records from primary storage. [2025-10-20T22:13:00.707277] [INFO ] [MAIN ] ONIT: Loaded 7641 primary data records. [2025-10-20T22:13:00.784448] [DEBUG] [LOCAL ] ONL: Loaded 29214 existing records from primary storage. [2025-10-20T22:13:00.810908] [INFO ] [MAIN ] ONL: Loaded 29214 primary data records. [2025-10-20T22:13:01.155538] [DEBUG] [LOCAL ] OMF: Loaded 46485 existing records from primary storage. [2025-10-20T22:13:01.199100] [INFO ] [MAIN ] OMF: Loaded 46485 primary data records. [2025-10-20T22:13:01.699886] [DEBUG] [MAIN ] ONIT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:01.700865] [INFO ] [MAIN ] ONIT: Generated 1794 derivative records from 7641 primary records. [2025-10-20T22:13:01.701294] [DEBUG] [LOCAL ] ONIT: Saving derivative data. [2025-10-20T22:13:01.719981] [DEBUG] [LOCAL ] ONIT: Derivative data saved (1794 records). [2025-10-20T22:13:01.720885] [INFO ] [MAIN ] ONIT: Successfully saved derivative data. [2025-10-20T22:13:01.723273] [INFO ] [MAIN ] Derivative Progress: 1336/1877 symbols completed. ONIT took 1.0s. Estimated remaining: 3m 23s. Rate: 159.8 symbols/min [2025-10-20T22:13:01.723840] [INFO ] [MAIN ] ONTF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:01.738903] [DEBUG] [MAIN ] ONL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:01.740242] [INFO ] [MAIN ] ONL: Generated 1794 derivative records from 29214 primary records. [2025-10-20T22:13:01.740551] [DEBUG] [LOCAL ] ONL: Saving derivative data. [2025-10-20T22:13:01.761968] [DEBUG] [LOCAL ] ONL: Derivative data saved (1794 records). [2025-10-20T22:13:01.769272] [INFO ] [MAIN ] ONL: Successfully saved derivative data. [2025-10-20T22:13:01.772785] [INFO ] [MAIN ] Derivative Progress: 1337/1877 symbols completed. ONL took 1.1s. Estimated remaining: 3m 23s. Rate: 159.9 symbols/min [2025-10-20T22:13:01.785782] [INFO ] [MAIN ] OOMA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:01.786836] [DEBUG] [MAIN ] OMF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:01.787627] [INFO ] [MAIN ] OMF: Generated 1794 derivative records from 46485 primary records. [2025-10-20T22:13:01.795932] [DEBUG] [LOCAL ] OMF: Saving derivative data. [2025-10-20T22:13:01.796791] [DEBUG] [LOCAL ] ONTF: Loaded 22672 existing records from primary storage. [2025-10-20T22:13:01.797541] [INFO ] [MAIN ] ONTF: Loaded 22672 primary data records. [2025-10-20T22:13:01.834475] [DEBUG] [LOCAL ] OMF: Derivative data saved (1794 records). [2025-10-20T22:13:01.863744] [INFO ] [MAIN ] OMF: Successfully saved derivative data. [2025-10-20T22:13:01.886460] [INFO ] [MAIN ] Derivative Progress: 1338/1877 symbols completed. OMF took 1.2s. Estimated remaining: 3m 22s. Rate: 160.0 symbols/min [2025-10-20T22:13:01.898545] [INFO ] [MAIN ] OPAD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:02.009626] [DEBUG] [LOCAL ] OOMA: Loaded 18427 existing records from primary storage. [2025-10-20T22:13:02.015283] [INFO ] [MAIN ] OOMA: Loaded 18427 primary data records. [2025-10-20T22:13:02.245124] [DEBUG] [LOCAL ] OPAD: Loaded 25848 existing records from primary storage. [2025-10-20T22:13:02.262116] [INFO ] [MAIN ] OPAD: Loaded 25848 primary data records. [2025-10-20T22:13:02.804742] [DEBUG] [MAIN ] ONTF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:02.806773] [INFO ] [MAIN ] ONTF: Generated 1794 derivative records from 22672 primary records. [2025-10-20T22:13:02.806876] [DEBUG] [LOCAL ] ONTF: Saving derivative data. [2025-10-20T22:13:02.824529] [DEBUG] [LOCAL ] ONTF: Derivative data saved (1794 records). [2025-10-20T22:13:02.824882] [INFO ] [MAIN ] ONTF: Successfully saved derivative data. [2025-10-20T22:13:02.827508] [INFO ] [MAIN ] Derivative Progress: 1339/1877 symbols completed. ONTF took 1.1s. Estimated remaining: 3m 22s. Rate: 159.8 symbols/min [2025-10-20T22:13:02.827852] [INFO ] [MAIN ] OPFI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:02.847200] [DEBUG] [MAIN ] OOMA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:02.848501] [INFO ] [MAIN ] OOMA: Generated 1794 derivative records from 18427 primary records. [2025-10-20T22:13:02.848638] [DEBUG] [LOCAL ] OOMA: Saving derivative data. [2025-10-20T22:13:02.866573] [DEBUG] [LOCAL ] OOMA: Derivative data saved (1794 records). [2025-10-20T22:13:02.866765] [INFO ] [MAIN ] OOMA: Successfully saved derivative data. [2025-10-20T22:13:02.868965] [INFO ] [MAIN ] Derivative Progress: 1340/1877 symbols completed. OOMA took 1.1s. Estimated remaining: 3m 21s. Rate: 159.9 symbols/min [2025-10-20T22:13:02.869126] [INFO ] [MAIN ] OPP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:02.885324] [DEBUG] [MAIN ] OPAD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:02.886165] [INFO ] [MAIN ] OPAD: Generated 1794 derivative records from 25848 primary records. [2025-10-20T22:13:02.897313] [DEBUG] [LOCAL ] OPAD: Saving derivative data. [2025-10-20T22:13:02.921599] [DEBUG] [LOCAL ] OPP: Loaded 12377 existing records from primary storage. [2025-10-20T22:13:02.922522] [INFO ] [MAIN ] OPP: Loaded 12377 primary data records. [2025-10-20T22:13:02.922300] [DEBUG] [LOCAL ] OPAD: Derivative data saved (1794 records). [2025-10-20T22:13:02.943442] [INFO ] [MAIN ] OPAD: Successfully saved derivative data. [2025-10-20T22:13:02.960959] [INFO ] [MAIN ] Derivative Progress: 1341/1877 symbols completed. OPAD took 1.1s. Estimated remaining: 3m 21s. Rate: 160.0 symbols/min [2025-10-20T22:13:02.968401] [INFO ] [MAIN ] OR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:03.053402] [DEBUG] [LOCAL ] OPFI: Loaded 46172 existing records from primary storage. [2025-10-20T22:13:03.054047] [INFO ] [MAIN ] OPFI: Loaded 46172 primary data records. [2025-10-20T22:13:03.490127] [DEBUG] [LOCAL ] OR: Loaded 46804 existing records from primary storage. [2025-10-20T22:13:03.490905] [INFO ] [MAIN ] OR: Loaded 46804 primary data records. [2025-10-20T22:13:03.996299] [DEBUG] [MAIN ] OPP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:03.997563] [INFO ] [MAIN ] OPP: Generated 1794 derivative records from 12377 primary records. [2025-10-20T22:13:03.997886] [DEBUG] [LOCAL ] OPP: Saving derivative data. [2025-10-20T22:13:04.017446] [DEBUG] [LOCAL ] OPP: Derivative data saved (1794 records). [2025-10-20T22:13:04.031088] [INFO ] [MAIN ] OPP: Successfully saved derivative data. [2025-10-20T22:13:04.033330] [INFO ] [MAIN ] Derivative Progress: 1342/1877 symbols completed. OPP took 1.2s. Estimated remaining: 3m 21s. Rate: 159.8 symbols/min [2025-10-20T22:13:04.033877] [INFO ] [MAIN ] ORN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:04.033761] [DEBUG] [MAIN ] OPFI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:13:04.035627] [INFO ] [MAIN ] OPFI: Generated 1793 derivative records from 46172 primary records. [2025-10-20T22:13:04.035908] [DEBUG] [LOCAL ] OPFI: Saving derivative data. [2025-10-20T22:13:04.055807] [DEBUG] [LOCAL ] OPFI: Derivative data saved (1793 records). [2025-10-20T22:13:04.057155] [INFO ] [MAIN ] OPFI: Successfully saved derivative data. [2025-10-20T22:13:04.061320] [INFO ] [MAIN ] Derivative Progress: 1343/1877 symbols completed. OPFI took 1.2s. Estimated remaining: 3m 20s. Rate: 159.9 symbols/min [2025-10-20T22:13:04.062523] [INFO ] [MAIN ] OWLT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:04.086130] [DEBUG] [MAIN ] OR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:04.087033] [INFO ] [MAIN ] OR: Generated 1794 derivative records from 46804 primary records. [2025-10-20T22:13:04.087141] [DEBUG] [LOCAL ] OR: Saving derivative data. [2025-10-20T22:13:04.116080] [DEBUG] [LOCAL ] OR: Derivative data saved (1794 records). [2025-10-20T22:13:04.116234] [INFO ] [MAIN ] OR: Successfully saved derivative data. [2025-10-20T22:13:04.114363] [DEBUG] [LOCAL ] ORN: Loaded 30941 existing records from primary storage. [2025-10-20T22:13:04.118577] [INFO ] [MAIN ] Derivative Progress: 1344/1877 symbols completed. OR took 1.2s. Estimated remaining: 3m 20s. Rate: 160.0 symbols/min [2025-10-20T22:13:04.118887] [INFO ] [MAIN ] ORN: Loaded 30941 primary data records. [2025-10-20T22:13:04.118801] [INFO ] [MAIN ] PAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:04.179902] [DEBUG] [LOCAL ] OWLT: Loaded 9504 existing records from primary storage. [2025-10-20T22:13:04.180362] [INFO ] [MAIN ] OWLT: Loaded 9504 primary data records. [2025-10-20T22:13:04.402000] [DEBUG] [LOCAL ] PAC: Loaded 11741 existing records from primary storage. [2025-10-20T22:13:04.417443] [INFO ] [MAIN ] PAC: Loaded 11741 primary data records. [2025-10-20T22:13:05.137053] [DEBUG] [MAIN ] ORN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:05.138045] [INFO ] [MAIN ] ORN: Generated 1794 derivative records from 30941 primary records. [2025-10-20T22:13:05.138236] [DEBUG] [LOCAL ] ORN: Saving derivative data. [2025-10-20T22:13:05.156834] [DEBUG] [LOCAL ] ORN: Derivative data saved (1794 records). [2025-10-20T22:13:05.157487] [INFO ] [MAIN ] ORN: Successfully saved derivative data. [2025-10-20T22:13:05.160862] [INFO ] [MAIN ] Derivative Progress: 1345/1877 symbols completed. ORN took 1.1s. Estimated remaining: 3m 20s. Rate: 159.8 symbols/min [2025-10-20T22:13:05.161145] [INFO ] [MAIN ] PACK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:05.176516] [DEBUG] [MAIN ] OWLT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:05.178425] [INFO ] [MAIN ] OWLT: Generated 1794 derivative records from 9504 primary records. [2025-10-20T22:13:05.179182] [DEBUG] [LOCAL ] OWLT: Saving derivative data. [2025-10-20T22:13:05.202411] [DEBUG] [LOCAL ] OWLT: Derivative data saved (1794 records). [2025-10-20T22:13:05.207209] [INFO ] [MAIN ] OWLT: Successfully saved derivative data. [2025-10-20T22:13:05.210297] [INFO ] [MAIN ] Derivative Progress: 1346/1877 symbols completed. OWLT took 1.1s. Estimated remaining: 3m 19s. Rate: 159.9 symbols/min [2025-10-20T22:13:05.210776] [INFO ] [MAIN ] PACS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:05.211258] [DEBUG] [MAIN ] PAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:05.221002] [INFO ] [MAIN ] PAC: Generated 1794 derivative records from 11741 primary records. [2025-10-20T22:13:05.221306] [DEBUG] [LOCAL ] PAC: Saving derivative data. [2025-10-20T22:13:05.254253] [DEBUG] [LOCAL ] PAC: Derivative data saved (1794 records). [2025-10-20T22:13:05.252710] [DEBUG] [LOCAL ] PACK: Loaded 32000 existing records from primary storage. [2025-10-20T22:13:05.254477] [INFO ] [MAIN ] PAC: Successfully saved derivative data. [2025-10-20T22:13:05.257375] [INFO ] [MAIN ] PACK: Loaded 32000 primary data records. [2025-10-20T22:13:05.257253] [INFO ] [MAIN ] Derivative Progress: 1347/1877 symbols completed. PAC took 1.1s. Estimated remaining: 3m 19s. Rate: 160.0 symbols/min [2025-10-20T22:13:05.257776] [INFO ] [MAIN ] PAI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:05.370271] [DEBUG] [LOCAL ] PAI: Loaded 2814 existing records from primary storage. [2025-10-20T22:13:05.382437] [INFO ] [MAIN ] PAI: Loaded 2814 primary data records. [2025-10-20T22:13:05.419426] [DEBUG] [LOCAL ] PACS: Loaded 34736 existing records from primary storage. [2025-10-20T22:13:05.473367] [INFO ] [MAIN ] PACS: Loaded 34736 primary data records. [2025-10-20T22:13:06.267973] [DEBUG] [MAIN ] PACK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:06.269083] [INFO ] [MAIN ] PACK: Generated 1794 derivative records from 32000 primary records. [2025-10-20T22:13:06.269680] [DEBUG] [LOCAL ] PACK: Saving derivative data. [2025-10-20T22:13:06.286097] [DEBUG] [LOCAL ] PACK: Derivative data saved (1794 records). [2025-10-20T22:13:06.287971] [INFO ] [MAIN ] PACK: Successfully saved derivative data. [2025-10-20T22:13:06.290320] [INFO ] [MAIN ] Derivative Progress: 1348/1877 symbols completed. PACK took 1.1s. Estimated remaining: 3m 19s. Rate: 159.8 symbols/min [2025-10-20T22:13:06.309833] [DEBUG] [MAIN ] PAI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:06.311487] [INFO ] [MAIN ] PAI: Generated 1794 derivative records from 2814 primary records. [2025-10-20T22:13:06.312109] [DEBUG] [LOCAL ] PAI: Saving derivative data. [2025-10-20T22:13:06.341556] [DEBUG] [MAIN ] PACS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:06.342417] [INFO ] [MAIN ] PACS: Generated 1794 derivative records from 34736 primary records. [2025-10-20T22:13:06.341244] [DEBUG] [LOCAL ] PAI: Derivative data saved (1794 records). [2025-10-20T22:13:06.342529] [INFO ] [MAIN ] PAI: Successfully saved derivative data. [2025-10-20T22:13:06.345042] [DEBUG] [LOCAL ] PACS: Saving derivative data. [2025-10-20T22:13:06.344967] [INFO ] [MAIN ] Derivative Progress: 1349/1877 symbols completed. PAI took 1.1s. Estimated remaining: 3m 18s. Rate: 159.9 symbols/min [2025-10-20T22:13:06.361451] [DEBUG] [LOCAL ] PACS: Derivative data saved (1794 records). [2025-10-20T22:13:06.361561] [INFO ] [MAIN ] PACS: Successfully saved derivative data. [2025-10-20T22:13:06.363595] [INFO ] [MAIN ] Derivative Progress: 1350/1877 symbols completed. PACS took 1.2s. Estimated remaining: 3m 18s. Rate: 160.0 symbols/min [2025-10-20T22:13:06.364340] [INFO ] [MAIN ] PAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:06.364942] [INFO ] [MAIN ] PAXS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:06.364469] [INFO ] [MAIN ] PAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:06.403134] [DEBUG] [LOCAL ] PAXS: Loaded 21022 existing records from primary storage. [2025-10-20T22:13:06.412790] [INFO ] [MAIN ] PAXS: Loaded 21022 primary data records. [2025-10-20T22:13:06.413358] [DEBUG] [LOCAL ] PAM: Loaded 28940 existing records from primary storage. [2025-10-20T22:13:06.435064] [INFO ] [MAIN ] PAM: Loaded 28940 primary data records. [2025-10-20T22:13:06.583638] [DEBUG] [LOCAL ] PAR: Loaded 34290 existing records from primary storage. [2025-10-20T22:13:06.619348] [INFO ] [MAIN ] PAR: Loaded 34290 primary data records. [2025-10-20T22:13:07.463595] [DEBUG] [MAIN ] PAM: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:13:07.465133] [DEBUG] [MAIN ] PAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:07.464635] [DEBUG] [MAIN ] PAXS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:07.466258] [INFO ] [MAIN ] PAM: Generated 1793 derivative records from 28940 primary records. [2025-10-20T22:13:07.466668] [DEBUG] [LOCAL ] PAM: Saving derivative data. [2025-10-20T22:13:07.468679] [INFO ] [MAIN ] PAXS: Generated 1794 derivative records from 21022 primary records. [2025-10-20T22:13:07.485468] [DEBUG] [LOCAL ] PAM: Derivative data saved (1793 records). [2025-10-20T22:13:07.477266] [DEBUG] [LOCAL ] PAXS: Saving derivative data. [2025-10-20T22:13:07.469961] [INFO ] [MAIN ] PAR: Generated 1794 derivative records from 34290 primary records. [2025-10-20T22:13:07.485793] [INFO ] [MAIN ] PAM: Successfully saved derivative data. [2025-10-20T22:13:07.485870] [DEBUG] [LOCAL ] PAR: Saving derivative data. [2025-10-20T22:13:07.491228] [INFO ] [MAIN ] Derivative Progress: 1351/1877 symbols completed. PAM took 1.1s. Estimated remaining: 3m 18s. Rate: 159.8 symbols/min [2025-10-20T22:13:07.506129] [INFO ] [MAIN ] PAY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:07.524270] [DEBUG] [LOCAL ] PAXS: Derivative data saved (1794 records). [2025-10-20T22:13:07.525758] [INFO ] [MAIN ] PAXS: Successfully saved derivative data. [2025-10-20T22:13:07.528630] [INFO ] [MAIN ] Derivative Progress: 1352/1877 symbols completed. PAXS took 1.2s. Estimated remaining: 3m 17s. Rate: 159.9 symbols/min [2025-10-20T22:13:07.528900] [INFO ] [MAIN ] PBA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:07.525519] [DEBUG] [LOCAL ] PAR: Derivative data saved (1794 records). [2025-10-20T22:13:07.529686] [INFO ] [MAIN ] PAR: Successfully saved derivative data. [2025-10-20T22:13:07.532485] [INFO ] [MAIN ] Derivative Progress: 1353/1877 symbols completed. PAR took 1.2s. Estimated remaining: 3m 16s. Rate: 160.0 symbols/min [2025-10-20T22:13:07.532865] [INFO ] [MAIN ] PBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:07.587154] [DEBUG] [LOCAL ] PAY: Loaded 38632 existing records from primary storage. [2025-10-20T22:13:07.587482] [INFO ] [MAIN ] PAY: Loaded 38632 primary data records. [2025-10-20T22:13:07.704251] [DEBUG] [LOCAL ] PBA: Loaded 49232 existing records from primary storage. [2025-10-20T22:13:07.717351] [INFO ] [MAIN ] PBA: Loaded 49232 primary data records. [2025-10-20T22:13:07.905198] [DEBUG] [LOCAL ] PBR: Loaded 53713 existing records from primary storage. [2025-10-20T22:13:08.086763] [INFO ] [MAIN ] PBR: Loaded 53713 primary data records. [2025-10-20T22:13:08.637925] [DEBUG] [MAIN ] PAY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:08.643602] [INFO ] [MAIN ] PAY: Generated 1794 derivative records from 38632 primary records. [2025-10-20T22:13:08.645523] [DEBUG] [LOCAL ] PAY: Saving derivative data. [2025-10-20T22:13:08.661702] [DEBUG] [MAIN ] PBA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:08.670138] [INFO ] [MAIN ] PBA: Generated 1794 derivative records from 49232 primary records. [2025-10-20T22:13:08.674768] [DEBUG] [LOCAL ] PBA: Saving derivative data. [2025-10-20T22:13:08.688901] [DEBUG] [LOCAL ] PAY: Derivative data saved (1794 records). [2025-10-20T22:13:08.689160] [INFO ] [MAIN ] PAY: Successfully saved derivative data. [2025-10-20T22:13:08.691950] [INFO ] [MAIN ] Derivative Progress: 1354/1877 symbols completed. PAY took 1.2s. Estimated remaining: 3m 16s. Rate: 159.8 symbols/min [2025-10-20T22:13:08.695020] [INFO ] [MAIN ] PBT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:08.700585] [DEBUG] [LOCAL ] PBA: Derivative data saved (1794 records). [2025-10-20T22:13:08.701029] [INFO ] [MAIN ] PBA: Successfully saved derivative data. [2025-10-20T22:13:08.703428] [INFO ] [MAIN ] Derivative Progress: 1355/1877 symbols completed. PBA took 1.2s. Estimated remaining: 3m 16s. Rate: 159.9 symbols/min [2025-10-20T22:13:08.703766] [INFO ] [MAIN ] PCF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:08.729305] [DEBUG] [MAIN ] PBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:08.730418] [INFO ] [MAIN ] PBR: Generated 1794 derivative records from 53713 primary records. [2025-10-20T22:13:08.730845] [DEBUG] [LOCAL ] PBR: Saving derivative data. [2025-10-20T22:13:08.735615] [DEBUG] [LOCAL ] PCF: Loaded 9084 existing records from primary storage. [2025-10-20T22:13:08.743774] [INFO ] [MAIN ] PCF: Loaded 9084 primary data records. [2025-10-20T22:13:08.744471] [DEBUG] [LOCAL ] PBT: Loaded 17542 existing records from primary storage. [2025-10-20T22:13:08.757252] [INFO ] [MAIN ] PBT: Loaded 17542 primary data records. [2025-10-20T22:13:08.820522] [DEBUG] [LOCAL ] PBR: Derivative data saved (1794 records). [2025-10-20T22:13:08.870237] [INFO ] [MAIN ] PBR: Successfully saved derivative data. [2025-10-20T22:13:08.890763] [INFO ] [MAIN ] Derivative Progress: 1356/1877 symbols completed. PBR took 1.4s. Estimated remaining: 3m 15s. Rate: 159.9 symbols/min [2025-10-20T22:13:08.912729] [INFO ] [MAIN ] PCM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:09.104122] [DEBUG] [LOCAL ] PCM: Loaded 6851 existing records from primary storage. [2025-10-20T22:13:09.237140] [INFO ] [MAIN ] PCM: Loaded 6851 primary data records. [2025-10-20T22:13:09.865345] [DEBUG] [MAIN ] PCF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:09.867118] [INFO ] [MAIN ] PCF: Generated 1794 derivative records from 9084 primary records. [2025-10-20T22:13:09.867816] [DEBUG] [LOCAL ] PCF: Saving derivative data. [2025-10-20T22:13:09.890875] [DEBUG] [LOCAL ] PCF: Derivative data saved (1794 records). [2025-10-20T22:13:09.891489] [INFO ] [MAIN ] PCF: Successfully saved derivative data. [2025-10-20T22:13:09.906777] [INFO ] [MAIN ] Derivative Progress: 1357/1877 symbols completed. PCF took 1.2s. Estimated remaining: 3m 15s. Rate: 159.7 symbols/min [2025-10-20T22:13:09.907703] [DEBUG] [MAIN ] PBT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:09.908880] [INFO ] [MAIN ] PBT: Generated 1794 derivative records from 17542 primary records. [2025-10-20T22:13:09.909006] [INFO ] [MAIN ] PCN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:09.909923] [DEBUG] [LOCAL ] PBT: Saving derivative data. [2025-10-20T22:13:09.929494] [DEBUG] [LOCAL ] PBT: Derivative data saved (1794 records). [2025-10-20T22:13:09.930396] [INFO ] [MAIN ] PBT: Successfully saved derivative data. [2025-10-20T22:13:09.933079] [INFO ] [MAIN ] Derivative Progress: 1358/1877 symbols completed. PBT took 1.2s. Estimated remaining: 3m 15s. Rate: 159.8 symbols/min [2025-10-20T22:13:09.933346] [INFO ] [MAIN ] PCOR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:09.948572] [DEBUG] [MAIN ] PCM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:09.949582] [INFO ] [MAIN ] PCM: Generated 1794 derivative records from 6851 primary records. [2025-10-20T22:13:09.949698] [DEBUG] [LOCAL ] PCM: Saving derivative data. [2025-10-20T22:13:09.986830] [DEBUG] [LOCAL ] PCM: Derivative data saved (1794 records). [2025-10-20T22:13:09.987039] [INFO ] [MAIN ] PCM: Successfully saved derivative data. [2025-10-20T22:13:09.991544] [DEBUG] [LOCAL ] PCN: Loaded 25417 existing records from primary storage. [2025-10-20T22:13:09.991801] [INFO ] [MAIN ] PCN: Loaded 25417 primary data records. [2025-10-20T22:13:09.990689] [INFO ] [MAIN ] Derivative Progress: 1359/1877 symbols completed. PCM took 1.1s. Estimated remaining: 3m 14s. Rate: 159.9 symbols/min [2025-10-20T22:13:10.003964] [INFO ] [MAIN ] PCQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:10.312011] [DEBUG] [LOCAL ] PCQ: Loaded 12261 existing records from primary storage. [2025-10-20T22:13:10.326993] [DEBUG] [LOCAL ] PCOR: Loaded 47617 existing records from primary storage. [2025-10-20T22:13:10.327401] [INFO ] [MAIN ] PCOR: Loaded 47617 primary data records. [2025-10-20T22:13:10.327269] [INFO ] [MAIN ] PCQ: Loaded 12261 primary data records. [2025-10-20T22:13:10.992917] [DEBUG] [MAIN ] PCN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:10.995813] [INFO ] [MAIN ] PCN: Generated 1794 derivative records from 25417 primary records. [2025-10-20T22:13:10.998872] [DEBUG] [LOCAL ] PCN: Saving derivative data. [2025-10-20T22:13:11.019053] [DEBUG] [LOCAL ] PCN: Derivative data saved (1794 records). [2025-10-20T22:13:11.019339] [INFO ] [MAIN ] PCN: Successfully saved derivative data. [2025-10-20T22:13:11.022389] [INFO ] [MAIN ] Derivative Progress: 1360/1877 symbols completed. PCN took 1.1s. Estimated remaining: 3m 14s. Rate: 159.7 symbols/min [2025-10-20T22:13:11.022614] [INFO ] [MAIN ] PD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:11.050547] [DEBUG] [MAIN ] PCQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:11.064616] [INFO ] [MAIN ] PCQ: Generated 1794 derivative records from 12261 primary records. [2025-10-20T22:13:11.065469] [DEBUG] [LOCAL ] PCQ: Saving derivative data. [2025-10-20T22:13:11.065311] [DEBUG] [MAIN ] PCOR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:11.066348] [INFO ] [MAIN ] PCOR: Generated 1794 derivative records from 47617 primary records. [2025-10-20T22:13:11.067285] [DEBUG] [LOCAL ] PCOR: Saving derivative data. [2025-10-20T22:13:11.096480] [DEBUG] [LOCAL ] PCQ: Derivative data saved (1794 records). [2025-10-20T22:13:11.105157] [INFO ] [MAIN ] PCQ: Successfully saved derivative data. [2025-10-20T22:13:11.105262] [DEBUG] [LOCAL ] PCOR: Derivative data saved (1794 records). [2025-10-20T22:13:11.105452] [INFO ] [MAIN ] PCOR: Successfully saved derivative data. [2025-10-20T22:13:11.107591] [INFO ] [MAIN ] Derivative Progress: 1361/1877 symbols completed. PCQ took 1.1s. Estimated remaining: 3m 14s. Rate: 159.8 symbols/min [2025-10-20T22:13:11.123448] [INFO ] [MAIN ] PDI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:11.123581] [INFO ] [MAIN ] Derivative Progress: 1362/1877 symbols completed. PCOR took 1.2s. Estimated remaining: 3m 13s. Rate: 159.9 symbols/min [2025-10-20T22:13:11.126460] [INFO ] [MAIN ] PDM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:11.149673] [DEBUG] [LOCAL ] PD: Loaded 48336 existing records from primary storage. [2025-10-20T22:13:11.150156] [INFO ] [MAIN ] PD: Loaded 48336 primary data records. [2025-10-20T22:13:11.525514] [DEBUG] [LOCAL ] PDM: Loaded 39915 existing records from primary storage. [2025-10-20T22:13:11.527778] [DEBUG] [LOCAL ] PDI: Loaded 50190 existing records from primary storage. [2025-10-20T22:13:11.526854] [INFO ] [MAIN ] PDM: Loaded 39915 primary data records. [2025-10-20T22:13:11.528108] [INFO ] [MAIN ] PDI: Loaded 50190 primary data records. [2025-10-20T22:13:12.412940] [DEBUG] [MAIN ] PD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:12.415704] [INFO ] [MAIN ] PD: Generated 1794 derivative records from 48336 primary records. [2025-10-20T22:13:12.416169] [DEBUG] [LOCAL ] PD: Saving derivative data. [2025-10-20T22:13:12.442754] [DEBUG] [LOCAL ] PD: Derivative data saved (1794 records). [2025-10-20T22:13:12.443270] [INFO ] [MAIN ] PD: Successfully saved derivative data. [2025-10-20T22:13:12.445871] [INFO ] [MAIN ] Derivative Progress: 1363/1877 symbols completed. PD took 1.4s. Estimated remaining: 3m 13s. Rate: 159.6 symbols/min [2025-10-20T22:13:12.470259] [DEBUG] [MAIN ] PDM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:12.471357] [INFO ] [MAIN ] PDM: Generated 1794 derivative records from 39915 primary records. [2025-10-20T22:13:12.484462] [DEBUG] [LOCAL ] PDM: Saving derivative data. [2025-10-20T22:13:12.485104] [DEBUG] [MAIN ] PDI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:12.487136] [INFO ] [MAIN ] PDI: Generated 1794 derivative records from 50190 primary records. [2025-10-20T22:13:12.487505] [DEBUG] [LOCAL ] PDI: Saving derivative data. [2025-10-20T22:13:12.537882] [DEBUG] [LOCAL ] PDI: Derivative data saved (1794 records). [2025-10-20T22:13:12.535683] [DEBUG] [LOCAL ] PDM: Derivative data saved (1794 records). [2025-10-20T22:13:12.538121] [INFO ] [MAIN ] PDI: Successfully saved derivative data. [2025-10-20T22:13:12.541102] [INFO ] [MAIN ] Derivative Progress: 1364/1877 symbols completed. PDI took 1.4s. Estimated remaining: 3m 13s. Rate: 159.7 symbols/min [2025-10-20T22:13:12.541310] [INFO ] [MAIN ] PDM: Successfully saved derivative data. [2025-10-20T22:13:12.544300] [INFO ] [MAIN ] Derivative Progress: 1365/1877 symbols completed. PDM took 1.4s. Estimated remaining: 3m 12s. Rate: 159.9 symbols/min [2025-10-20T22:13:12.545688] [INFO ] [MAIN ] PDS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:12.545823] [INFO ] [MAIN ] PDT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:12.545550] [INFO ] [MAIN ] PDO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:12.582130] [DEBUG] [LOCAL ] PDT: Loaded 17512 existing records from primary storage. [2025-10-20T22:13:12.582994] [INFO ] [MAIN ] PDT: Loaded 17512 primary data records. [2025-10-20T22:13:12.622511] [DEBUG] [LOCAL ] PDS: Loaded 16958 existing records from primary storage. [2025-10-20T22:13:12.647499] [INFO ] [MAIN ] PDS: Loaded 16958 primary data records. [2025-10-20T22:13:12.702621] [DEBUG] [LOCAL ] PDO: Loaded 31542 existing records from primary storage. [2025-10-20T22:13:12.790781] [INFO ] [MAIN ] PDO: Loaded 31542 primary data records. [2025-10-20T22:13:13.663317] [DEBUG] [MAIN ] PDT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:13.678204] [INFO ] [MAIN ] PDT: Generated 1794 derivative records from 17512 primary records. [2025-10-20T22:13:13.677281] [DEBUG] [MAIN ] PDS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:13.679676] [INFO ] [MAIN ] PDS: Generated 1794 derivative records from 16958 primary records. [2025-10-20T22:13:13.679789] [DEBUG] [LOCAL ] PDS: Saving derivative data. [2025-10-20T22:13:13.683917] [DEBUG] [LOCAL ] PDT: Saving derivative data. [2025-10-20T22:13:13.728628] [DEBUG] [LOCAL ] PDS: Derivative data saved (1794 records). [2025-10-20T22:13:13.729414] [INFO ] [MAIN ] PDS: Successfully saved derivative data. [2025-10-20T22:13:13.728800] [DEBUG] [LOCAL ] PDT: Derivative data saved (1794 records). [2025-10-20T22:13:13.729334] [DEBUG] [MAIN ] PDO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:13.731386] [INFO ] [MAIN ] Derivative Progress: 1366/1877 symbols completed. PDS took 1.2s. Estimated remaining: 3m 12s. Rate: 159.6 symbols/min [2025-10-20T22:13:13.732467] [INFO ] [MAIN ] PDO: Generated 1794 derivative records from 31542 primary records. [2025-10-20T22:13:13.732634] [INFO ] [MAIN ] PDX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:13.731591] [INFO ] [MAIN ] PDT: Successfully saved derivative data. [2025-10-20T22:13:13.733309] [DEBUG] [LOCAL ] PDO: Saving derivative data. [2025-10-20T22:13:13.737326] [INFO ] [MAIN ] Derivative Progress: 1367/1877 symbols completed. PDT took 1.2s. Estimated remaining: 3m 12s. Rate: 159.7 symbols/min [2025-10-20T22:13:13.737560] [INFO ] [MAIN ] PEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:13.763054] [DEBUG] [LOCAL ] PDO: Derivative data saved (1794 records). [2025-10-20T22:13:13.763162] [INFO ] [MAIN ] PDO: Successfully saved derivative data. [2025-10-20T22:13:13.765349] [INFO ] [MAIN ] Derivative Progress: 1368/1877 symbols completed. PDO took 1.2s. Estimated remaining: 3m 11s. Rate: 159.8 symbols/min [2025-10-20T22:13:13.765739] [INFO ] [MAIN ] PERF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:13.777958] [DEBUG] [LOCAL ] PEO: Loaded 11179 existing records from primary storage. [2025-10-20T22:13:13.777692] [DEBUG] [LOCAL ] PDX: Loaded 14168 existing records from primary storage. [2025-10-20T22:13:13.778104] [INFO ] [MAIN ] PDX: Loaded 14168 primary data records. [2025-10-20T22:13:13.778261] [INFO ] [MAIN ] PEO: Loaded 11179 primary data records. [2025-10-20T22:13:13.914411] [DEBUG] [LOCAL ] PERF: Loaded 14457 existing records from primary storage. [2025-10-20T22:13:13.962431] [INFO ] [MAIN ] PERF: Loaded 14457 primary data records. [2025-10-20T22:13:14.779319] [DEBUG] [MAIN ] PDX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:14.780241] [INFO ] [MAIN ] PDX: Generated 1794 derivative records from 14168 primary records. [2025-10-20T22:13:14.780341] [DEBUG] [LOCAL ] PDX: Saving derivative data. [2025-10-20T22:13:14.797833] [DEBUG] [MAIN ] PEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:14.807379] [INFO ] [MAIN ] PEO: Generated 1794 derivative records from 11179 primary records. [2025-10-20T22:13:14.810375] [DEBUG] [LOCAL ] PEO: Saving derivative data. [2025-10-20T22:13:14.814386] [DEBUG] [LOCAL ] PDX: Derivative data saved (1794 records). [2025-10-20T22:13:14.814674] [INFO ] [MAIN ] PDX: Successfully saved derivative data. [2025-10-20T22:13:14.830600] [INFO ] [MAIN ] Derivative Progress: 1369/1877 symbols completed. PDX took 1.1s. Estimated remaining: 3m 11s. Rate: 159.6 symbols/min [2025-10-20T22:13:14.839152] [DEBUG] [MAIN ] PERF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:14.843686] [INFO ] [MAIN ] PFH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:14.848639] [INFO ] [MAIN ] PERF: Generated 1794 derivative records from 14457 primary records. [2025-10-20T22:13:14.849760] [DEBUG] [LOCAL ] PERF: Saving derivative data. [2025-10-20T22:13:14.849475] [DEBUG] [LOCAL ] PEO: Derivative data saved (1794 records). [2025-10-20T22:13:14.850549] [INFO ] [MAIN ] PEO: Successfully saved derivative data. [2025-10-20T22:13:14.852979] [INFO ] [MAIN ] Derivative Progress: 1370/1877 symbols completed. PEO took 1.1s. Estimated remaining: 3m 10s. Rate: 159.7 symbols/min [2025-10-20T22:13:14.860268] [INFO ] [MAIN ] PFL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:14.868898] [DEBUG] [LOCAL ] PERF: Derivative data saved (1794 records). [2025-10-20T22:13:14.869046] [INFO ] [MAIN ] PERF: Successfully saved derivative data. [2025-10-20T22:13:14.871078] [INFO ] [MAIN ] Derivative Progress: 1371/1877 symbols completed. PERF took 1.1s. Estimated remaining: 3m 10s. Rate: 159.8 symbols/min [2025-10-20T22:13:14.872974] [INFO ] [MAIN ] PFLT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:14.877558] [DEBUG] [LOCAL ] PFH: Loaded 6502 existing records from primary storage. [2025-10-20T22:13:14.877726] [INFO ] [MAIN ] PFH: Loaded 6502 primary data records. [2025-10-20T22:13:14.979795] [DEBUG] [LOCAL ] PFL: Loaded 14544 existing records from primary storage. [2025-10-20T22:13:14.986176] [INFO ] [MAIN ] PFL: Loaded 14544 primary data records. [2025-10-20T22:13:15.385136] [DEBUG] [LOCAL ] PFLT: Loaded 44339 existing records from primary storage. [2025-10-20T22:13:15.406949] [INFO ] [MAIN ] PFLT: Loaded 44339 primary data records. [2025-10-20T22:13:15.875164] [DEBUG] [MAIN ] PFH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:15.889206] [DEBUG] [MAIN ] PFL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:15.888520] [INFO ] [MAIN ] PFH: Generated 1794 derivative records from 6502 primary records. [2025-10-20T22:13:15.890622] [INFO ] [MAIN ] PFL: Generated 1794 derivative records from 14544 primary records. [2025-10-20T22:13:15.890812] [DEBUG] [LOCAL ] PFH: Saving derivative data. [2025-10-20T22:13:15.891429] [DEBUG] [LOCAL ] PFL: Saving derivative data. [2025-10-20T22:13:15.921199] [DEBUG] [LOCAL ] PFH: Derivative data saved (1794 records). [2025-10-20T22:13:15.922590] [DEBUG] [LOCAL ] PFL: Derivative data saved (1794 records). [2025-10-20T22:13:15.922786] [INFO ] [MAIN ] PFH: Successfully saved derivative data. [2025-10-20T22:13:15.924983] [INFO ] [MAIN ] Derivative Progress: 1372/1877 symbols completed. PFH took 1.1s. Estimated remaining: 3m 10s. Rate: 159.6 symbols/min [2025-10-20T22:13:15.925389] [INFO ] [MAIN ] PFN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:15.922708] [INFO ] [MAIN ] PFL: Successfully saved derivative data. [2025-10-20T22:13:15.927782] [INFO ] [MAIN ] Derivative Progress: 1373/1877 symbols completed. PFL took 1.1s. Estimated remaining: 3m 9s. Rate: 159.7 symbols/min [2025-10-20T22:13:15.927913] [INFO ] [MAIN ] PFO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:15.950428] [DEBUG] [LOCAL ] PFO: Loaded 6099 existing records from primary storage. [2025-10-20T22:13:15.947245] [DEBUG] [MAIN ] PFLT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:15.951338] [INFO ] [MAIN ] PFO: Loaded 6099 primary data records. [2025-10-20T22:13:15.951209] [INFO ] [MAIN ] PFLT: Generated 1794 derivative records from 44339 primary records. [2025-10-20T22:13:15.951430] [DEBUG] [LOCAL ] PFLT: Saving derivative data. [2025-10-20T22:13:16.092157] [DEBUG] [LOCAL ] PFN: Loaded 22398 existing records from primary storage. [2025-10-20T22:13:16.096056] [DEBUG] [LOCAL ] PFLT: Derivative data saved (1794 records). [2025-10-20T22:13:16.101402] [INFO ] [MAIN ] PFN: Loaded 22398 primary data records. [2025-10-20T22:13:16.106790] [INFO ] [MAIN ] PFLT: Successfully saved derivative data. [2025-10-20T22:13:16.113979] [INFO ] [MAIN ] Derivative Progress: 1374/1877 symbols completed. PFLT took 1.2s. Estimated remaining: 3m 9s. Rate: 159.8 symbols/min [2025-10-20T22:13:16.136781] [INFO ] [MAIN ] PFSI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:16.655508] [DEBUG] [LOCAL ] PFSI: Loaded 29172 existing records from primary storage. [2025-10-20T22:13:16.656120] [INFO ] [MAIN ] PFSI: Loaded 29172 primary data records. [2025-10-20T22:13:16.863295] [DEBUG] [MAIN ] PFO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:16.981179] [INFO ] [MAIN ] PFO: Generated 1794 derivative records from 6099 primary records. [2025-10-20T22:13:16.981962] [DEBUG] [LOCAL ] PFO: Saving derivative data. [2025-10-20T22:13:17.004033] [DEBUG] [LOCAL ] PFO: Derivative data saved (1794 records). [2025-10-20T22:13:17.004536] [INFO ] [MAIN ] PFO: Successfully saved derivative data. [2025-10-20T22:13:17.006977] [INFO ] [MAIN ] Derivative Progress: 1375/1877 symbols completed. PFO took 1.1s. Estimated remaining: 3m 9s. Rate: 159.6 symbols/min [2025-10-20T22:13:17.007695] [INFO ] [MAIN ] PGP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:17.034689] [DEBUG] [LOCAL ] PGP: Loaded 5690 existing records from primary storage. [2025-10-20T22:13:17.035075] [INFO ] [MAIN ] PGP: Loaded 5690 primary data records. [2025-10-20T22:13:17.047191] [DEBUG] [MAIN ] PFN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:17.087309] [INFO ] [MAIN ] PFN: Generated 1794 derivative records from 22398 primary records. [2025-10-20T22:13:17.104688] [DEBUG] [LOCAL ] PFN: Saving derivative data. [2025-10-20T22:13:17.248324] [DEBUG] [LOCAL ] PFN: Derivative data saved (1794 records). [2025-10-20T22:13:17.250558] [INFO ] [MAIN ] PFN: Successfully saved derivative data. [2025-10-20T22:13:17.378282] [INFO ] [MAIN ] Derivative Progress: 1376/1877 symbols completed. PFN took 1.5s. Estimated remaining: 3m 8s. Rate: 159.6 symbols/min [2025-10-20T22:13:17.379746] [INFO ] [MAIN ] PGZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:17.400380] [DEBUG] [MAIN ] PFSI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:17.402396] [INFO ] [MAIN ] PFSI: Generated 1794 derivative records from 29172 primary records. [2025-10-20T22:13:17.403015] [DEBUG] [LOCAL ] PFSI: Saving derivative data. [2025-10-20T22:13:17.425335] [DEBUG] [LOCAL ] PFSI: Derivative data saved (1794 records). [2025-10-20T22:13:17.421106] [DEBUG] [LOCAL ] PGZ: Loaded 5586 existing records from primary storage. [2025-10-20T22:13:17.426039] [INFO ] [MAIN ] PFSI: Successfully saved derivative data. [2025-10-20T22:13:17.428210] [INFO ] [MAIN ] Derivative Progress: 1377/1877 symbols completed. PFSI took 1.3s. Estimated remaining: 3m 8s. Rate: 159.7 symbols/min [2025-10-20T22:13:17.428373] [INFO ] [MAIN ] PHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:17.428438] [INFO ] [MAIN ] PGZ: Loaded 5586 primary data records. [2025-10-20T22:13:17.794688] [DEBUG] [MAIN ] PGP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:17.810435] [INFO ] [MAIN ] PGP: Generated 1794 derivative records from 5690 primary records. [2025-10-20T22:13:17.812591] [DEBUG] [LOCAL ] PGP: Saving derivative data. [2025-10-20T22:13:17.831068] [DEBUG] [LOCAL ] PHG: Loaded 45631 existing records from primary storage. [2025-10-20T22:13:17.831498] [DEBUG] [LOCAL ] PGP: Derivative data saved (1794 records). [2025-10-20T22:13:17.832434] [INFO ] [MAIN ] PGP: Successfully saved derivative data. [2025-10-20T22:13:17.834819] [INFO ] [MAIN ] PHG: Loaded 45631 primary data records. [2025-10-20T22:13:17.834678] [INFO ] [MAIN ] Derivative Progress: 1378/1877 symbols completed. PGP took 0.8s. Estimated remaining: 3m 7s. Rate: 159.7 symbols/min [2025-10-20T22:13:18.224575] [DEBUG] [MAIN ] PGZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:18.225374] [INFO ] [MAIN ] PGZ: Generated 1794 derivative records from 5586 primary records. [2025-10-20T22:13:18.225508] [DEBUG] [LOCAL ] PGZ: Saving derivative data. [2025-10-20T22:13:18.242843] [DEBUG] [LOCAL ] PGZ: Derivative data saved (1794 records). [2025-10-20T22:13:18.242978] [INFO ] [MAIN ] PGZ: Successfully saved derivative data. [2025-10-20T22:13:18.245321] [INFO ] [MAIN ] Derivative Progress: 1379/1877 symbols completed. PGZ took 0.9s. Estimated remaining: 3m 7s. Rate: 159.7 symbols/min [2025-10-20T22:13:18.264426] [DEBUG] [MAIN ] PHG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:18.265087] [INFO ] [MAIN ] PHG: Generated 1794 derivative records from 45631 primary records. [2025-10-20T22:13:18.265148] [DEBUG] [LOCAL ] PHG: Saving derivative data. [2025-10-20T22:13:18.280677] [DEBUG] [LOCAL ] PHG: Derivative data saved (1794 records). [2025-10-20T22:13:18.280741] [INFO ] [MAIN ] PHG: Successfully saved derivative data. [2025-10-20T22:13:18.282625] [INFO ] [MAIN ] Derivative Progress: 1380/1877 symbols completed. PHG took 0.9s. Estimated remaining: 3m 7s. Rate: 159.8 symbols/min [2025-10-20T22:13:18.283297] [INFO ] [MAIN ] PHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:18.283224] [INFO ] [MAIN ] PHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:18.284199] [INFO ] [MAIN ] PHK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:18.302341] [DEBUG] [LOCAL ] PHI: Loaded 15165 existing records from primary storage. [2025-10-20T22:13:18.302595] [INFO ] [MAIN ] PHI: Loaded 15165 primary data records. [2025-10-20T22:13:18.506400] [DEBUG] [LOCAL ] PHR: Loaded 37955 existing records from primary storage. [2025-10-20T22:13:18.505465] [DEBUG] [LOCAL ] PHK: Loaded 26939 existing records from primary storage. [2025-10-20T22:13:18.528212] [INFO ] [MAIN ] PHK: Loaded 26939 primary data records. [2025-10-20T22:13:18.533890] [INFO ] [MAIN ] PHR: Loaded 37955 primary data records. [2025-10-20T22:13:19.324025] [DEBUG] [MAIN ] PHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:19.325152] [INFO ] [MAIN ] PHI: Generated 1794 derivative records from 15165 primary records. [2025-10-20T22:13:19.328069] [DEBUG] [LOCAL ] PHI: Saving derivative data. [2025-10-20T22:13:19.347882] [DEBUG] [LOCAL ] PHI: Derivative data saved (1794 records). [2025-10-20T22:13:19.348445] [INFO ] [MAIN ] PHI: Successfully saved derivative data. [2025-10-20T22:13:19.351140] [INFO ] [MAIN ] Derivative Progress: 1381/1877 symbols completed. PHI took 1.1s. Estimated remaining: 3m 6s. Rate: 159.6 symbols/min [2025-10-20T22:13:19.351624] [INFO ] [MAIN ] PIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:19.371790] [DEBUG] [MAIN ] PHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:19.373618] [INFO ] [MAIN ] PHR: Generated 1794 derivative records from 37955 primary records. [2025-10-20T22:13:19.375277] [DEBUG] [LOCAL ] PHR: Saving derivative data. [2025-10-20T22:13:19.380730] [DEBUG] [LOCAL ] PIM: Loaded 4832 existing records from primary storage. [2025-10-20T22:13:19.406142] [DEBUG] [MAIN ] PHK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:19.408723] [INFO ] [MAIN ] PHK: Generated 1794 derivative records from 26939 primary records. [2025-10-20T22:13:19.408841] [DEBUG] [LOCAL ] PHK: Saving derivative data. [2025-10-20T22:13:19.407847] [INFO ] [MAIN ] PIM: Loaded 4832 primary data records. [2025-10-20T22:13:19.437072] [DEBUG] [LOCAL ] PHR: Derivative data saved (1794 records). [2025-10-20T22:13:19.437253] [INFO ] [MAIN ] PHR: Successfully saved derivative data. [2025-10-20T22:13:19.451765] [INFO ] [MAIN ] Derivative Progress: 1382/1877 symbols completed. PHR took 1.2s. Estimated remaining: 3m 6s. Rate: 159.7 symbols/min [2025-10-20T22:13:19.468014] [DEBUG] [LOCAL ] PHK: Derivative data saved (1794 records). [2025-10-20T22:13:19.473480] [INFO ] [MAIN ] PINE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:19.468205] [INFO ] [MAIN ] PHK: Successfully saved derivative data. [2025-10-20T22:13:19.488618] [INFO ] [MAIN ] Derivative Progress: 1383/1877 symbols completed. PHK took 1.2s. Estimated remaining: 3m 5s. Rate: 159.8 symbols/min [2025-10-20T22:13:19.488914] [INFO ] [MAIN ] PKE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:19.619348] [DEBUG] [LOCAL ] PINE: Loaded 14337 existing records from primary storage. [2025-10-20T22:13:19.748439] [INFO ] [MAIN ] PINE: Loaded 14337 primary data records. [2025-10-20T22:13:19.754466] [DEBUG] [LOCAL ] PKE: Loaded 14258 existing records from primary storage. [2025-10-20T22:13:19.776524] [INFO ] [MAIN ] PKE: Loaded 14258 primary data records. [2025-10-20T22:13:20.468763] [DEBUG] [MAIN ] PIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:20.470491] [INFO ] [MAIN ] PIM: Generated 1794 derivative records from 4832 primary records. [2025-10-20T22:13:20.470813] [DEBUG] [LOCAL ] PIM: Saving derivative data. [2025-10-20T22:13:20.490929] [DEBUG] [LOCAL ] PIM: Derivative data saved (1794 records). [2025-10-20T22:13:20.491472] [INFO ] [MAIN ] PIM: Successfully saved derivative data. [2025-10-20T22:13:20.494522] [INFO ] [MAIN ] Derivative Progress: 1384/1877 symbols completed. PIM took 1.1s. Estimated remaining: 3m 5s. Rate: 159.6 symbols/min [2025-10-20T22:13:20.495873] [INFO ] [MAIN ] PKST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:20.534398] [DEBUG] [MAIN ] PINE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:20.534850] [DEBUG] [MAIN ] PKE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:20.542501] [INFO ] [MAIN ] PINE: Generated 1794 derivative records from 14337 primary records. [2025-10-20T22:13:20.542760] [DEBUG] [LOCAL ] PINE: Saving derivative data. [2025-10-20T22:13:20.541543] [INFO ] [MAIN ] PKE: Generated 1794 derivative records from 14258 primary records. [2025-10-20T22:13:20.543056] [DEBUG] [LOCAL ] PKE: Saving derivative data. [2025-10-20T22:13:20.580445] [DEBUG] [LOCAL ] PKE: Derivative data saved (1794 records). [2025-10-20T22:13:20.580620] [INFO ] [MAIN ] PKE: Successfully saved derivative data. [2025-10-20T22:13:20.585049] [INFO ] [MAIN ] Derivative Progress: 1385/1877 symbols completed. PKE took 1.1s. Estimated remaining: 3m 5s. Rate: 159.7 symbols/min [2025-10-20T22:13:20.586865] [DEBUG] [LOCAL ] PKST: Loaded 18304 existing records from primary storage. [2025-10-20T22:13:20.587402] [INFO ] [MAIN ] PKX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:20.587832] [INFO ] [MAIN ] PKST: Loaded 18304 primary data records. [2025-10-20T22:13:20.588814] [DEBUG] [LOCAL ] PINE: Derivative data saved (1794 records). [2025-10-20T22:13:20.611348] [INFO ] [MAIN ] PINE: Successfully saved derivative data. [2025-10-20T22:13:20.632013] [INFO ] [MAIN ] Derivative Progress: 1386/1877 symbols completed. PINE took 1.2s. Estimated remaining: 3m 4s. Rate: 159.8 symbols/min [2025-10-20T22:13:20.661423] [INFO ] [MAIN ] PLOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:20.916117] [DEBUG] [LOCAL ] PKX: Loaded 22849 existing records from primary storage. [2025-10-20T22:13:20.926924] [INFO ] [MAIN ] PKX: Loaded 22849 primary data records. [2025-10-20T22:13:20.973509] [DEBUG] [LOCAL ] PLOW: Loaded 19918 existing records from primary storage. [2025-10-20T22:13:21.004350] [INFO ] [MAIN ] PLOW: Loaded 19918 primary data records. [2025-10-20T22:13:21.608432] [DEBUG] [MAIN ] PKST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:21.610802] [INFO ] [MAIN ] PKST: Generated 1794 derivative records from 18304 primary records. [2025-10-20T22:13:21.612007] [DEBUG] [LOCAL ] PKST: Saving derivative data. [2025-10-20T22:13:21.635241] [DEBUG] [LOCAL ] PKST: Derivative data saved (1794 records). [2025-10-20T22:13:21.635832] [INFO ] [MAIN ] PKST: Successfully saved derivative data. [2025-10-20T22:13:21.639516] [INFO ] [MAIN ] Derivative Progress: 1387/1877 symbols completed. PKST took 1.1s. Estimated remaining: 3m 4s. Rate: 159.6 symbols/min [2025-10-20T22:13:21.640222] [INFO ] [MAIN ] PLYM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:21.660042] [DEBUG] [MAIN ] PKX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:21.661344] [INFO ] [MAIN ] PKX: Generated 1794 derivative records from 22849 primary records. [2025-10-20T22:13:21.661482] [DEBUG] [LOCAL ] PKX: Saving derivative data. [2025-10-20T22:13:21.680800] [DEBUG] [LOCAL ] PKX: Derivative data saved (1794 records). [2025-10-20T22:13:21.681518] [INFO ] [MAIN ] PKX: Successfully saved derivative data. [2025-10-20T22:13:21.697763] [INFO ] [MAIN ] Derivative Progress: 1388/1877 symbols completed. PKX took 1.1s. Estimated remaining: 3m 4s. Rate: 159.7 symbols/min [2025-10-20T22:13:21.709268] [INFO ] [MAIN ] PML: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:21.709146] [DEBUG] [MAIN ] PLOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:21.711500] [INFO ] [MAIN ] PLOW: Generated 1794 derivative records from 19918 primary records. [2025-10-20T22:13:21.711947] [DEBUG] [LOCAL ] PLOW: Saving derivative data. [2025-10-20T22:13:21.730595] [DEBUG] [LOCAL ] PLOW: Derivative data saved (1794 records). [2025-10-20T22:13:21.730832] [INFO ] [MAIN ] PLOW: Successfully saved derivative data. [2025-10-20T22:13:21.733216] [INFO ] [MAIN ] Derivative Progress: 1389/1877 symbols completed. PLOW took 1.1s. Estimated remaining: 3m 3s. Rate: 159.8 symbols/min [2025-10-20T22:13:21.733513] [INFO ] [MAIN ] PMM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:21.753639] [DEBUG] [LOCAL ] PLYM: Loaded 34451 existing records from primary storage. [2025-10-20T22:13:21.754285] [INFO ] [MAIN ] PLYM: Loaded 34451 primary data records. [2025-10-20T22:13:21.881437] [DEBUG] [LOCAL ] PML: Loaded 22073 existing records from primary storage. [2025-10-20T22:13:21.885671] [DEBUG] [LOCAL ] PMM: Loaded 9961 existing records from primary storage. [2025-10-20T22:13:21.885985] [INFO ] [MAIN ] PMM: Loaded 9961 primary data records. [2025-10-20T22:13:21.903701] [INFO ] [MAIN ] PML: Loaded 22073 primary data records. [2025-10-20T22:13:22.764208] [DEBUG] [MAIN ] PLYM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:22.765877] [INFO ] [MAIN ] PLYM: Generated 1794 derivative records from 34451 primary records. [2025-10-20T22:13:22.766351] [DEBUG] [LOCAL ] PLYM: Saving derivative data. [2025-10-20T22:13:22.793567] [DEBUG] [LOCAL ] PLYM: Derivative data saved (1794 records). [2025-10-20T22:13:22.794568] [INFO ] [MAIN ] PLYM: Successfully saved derivative data. [2025-10-20T22:13:22.797717] [INFO ] [MAIN ] Derivative Progress: 1390/1877 symbols completed. PLYM took 1.2s. Estimated remaining: 3m 3s. Rate: 159.6 symbols/min [2025-10-20T22:13:22.798399] [INFO ] [MAIN ] PMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:22.816878] [DEBUG] [MAIN ] PMM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:22.839339] [INFO ] [MAIN ] PMM: Generated 1794 derivative records from 9961 primary records. [2025-10-20T22:13:22.841004] [DEBUG] [LOCAL ] PMM: Saving derivative data. [2025-10-20T22:13:22.847515] [DEBUG] [LOCAL ] PMO: Loaded 9680 existing records from primary storage. [2025-10-20T22:13:22.848731] [DEBUG] [MAIN ] PML: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:22.856078] [INFO ] [MAIN ] PML: Generated 1794 derivative records from 22073 primary records. [2025-10-20T22:13:22.857890] [DEBUG] [LOCAL ] PML: Saving derivative data. [2025-10-20T22:13:22.854888] [INFO ] [MAIN ] PMO: Loaded 9680 primary data records. [2025-10-20T22:13:22.880350] [DEBUG] [LOCAL ] PMM: Derivative data saved (1794 records). [2025-10-20T22:13:22.902223] [INFO ] [MAIN ] PMM: Successfully saved derivative data. [2025-10-20T22:13:22.904830] [INFO ] [MAIN ] Derivative Progress: 1391/1877 symbols completed. PMM took 1.2s. Estimated remaining: 3m 3s. Rate: 159.7 symbols/min [2025-10-20T22:13:22.925149] [INFO ] [MAIN ] PNI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:22.978307] [DEBUG] [LOCAL ] PML: Derivative data saved (1794 records). [2025-10-20T22:13:22.985893] [INFO ] [MAIN ] PML: Successfully saved derivative data. [2025-10-20T22:13:23.006086] [INFO ] [MAIN ] Derivative Progress: 1392/1877 symbols completed. PML took 1.3s. Estimated remaining: 3m 2s. Rate: 159.8 symbols/min [2025-10-20T22:13:23.011155] [INFO ] [MAIN ] PNNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:23.010856] [DEBUG] [LOCAL ] PNI: Loaded 6908 existing records from primary storage. [2025-10-20T22:13:23.011824] [INFO ] [MAIN ] PNI: Loaded 6908 primary data records. [2025-10-20T22:13:23.525241] [DEBUG] [LOCAL ] PNNT: Loaded 33472 existing records from primary storage. [2025-10-20T22:13:23.538912] [INFO ] [MAIN ] PNNT: Loaded 33472 primary data records. [2025-10-20T22:13:23.858023] [DEBUG] [MAIN ] PMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:23.859142] [INFO ] [MAIN ] PMO: Generated 1794 derivative records from 9680 primary records. [2025-10-20T22:13:23.859406] [DEBUG] [LOCAL ] PMO: Saving derivative data. [2025-10-20T22:13:23.876975] [DEBUG] [LOCAL ] PMO: Derivative data saved (1794 records). [2025-10-20T22:13:23.877385] [INFO ] [MAIN ] PMO: Successfully saved derivative data. [2025-10-20T22:13:23.887170] [INFO ] [MAIN ] Derivative Progress: 1393/1877 symbols completed. PMO took 1.1s. Estimated remaining: 3m 2s. Rate: 159.6 symbols/min [2025-10-20T22:13:23.893376] [DEBUG] [MAIN ] PNI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:23.894164] [INFO ] [MAIN ] PNI: Generated 1794 derivative records from 6908 primary records. [2025-10-20T22:13:23.894412] [DEBUG] [LOCAL ] PNI: Saving derivative data. [2025-10-20T22:13:23.908846] [DEBUG] [LOCAL ] PNI: Derivative data saved (1794 records). [2025-10-20T22:13:23.909030] [INFO ] [MAIN ] PNI: Successfully saved derivative data. [2025-10-20T22:13:23.911101] [INFO ] [MAIN ] Derivative Progress: 1394/1877 symbols completed. PNI took 1.0s. Estimated remaining: 3m 1s. Rate: 159.7 symbols/min [2025-10-20T22:13:23.925685] [DEBUG] [MAIN ] PNNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:23.926198] [INFO ] [MAIN ] PNNT: Generated 1794 derivative records from 33472 primary records. [2025-10-20T22:13:23.926255] [DEBUG] [LOCAL ] PNNT: Saving derivative data. [2025-10-20T22:13:23.940702] [DEBUG] [LOCAL ] PNNT: Derivative data saved (1794 records). [2025-10-20T22:13:23.940760] [INFO ] [MAIN ] PNNT: Successfully saved derivative data. [2025-10-20T22:13:23.942940] [INFO ] [MAIN ] Derivative Progress: 1395/1877 symbols completed. PNNT took 0.9s. Estimated remaining: 3m 1s. Rate: 159.8 symbols/min [2025-10-20T22:13:23.943273] [INFO ] [MAIN ] PPT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:23.943326] [INFO ] [MAIN ] PRH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:23.943428] [INFO ] [MAIN ] PRCS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:23.951054] [DEBUG] [LOCAL ] PRCS: Loaded 2786 existing records from primary storage. [2025-10-20T22:13:23.953755] [INFO ] [MAIN ] PRCS: Loaded 2786 primary data records. [2025-10-20T22:13:23.953676] [DEBUG] [LOCAL ] PRH: Loaded 4740 existing records from primary storage. [2025-10-20T22:13:23.965034] [INFO ] [MAIN ] PRH: Loaded 4740 primary data records. [2025-10-20T22:13:24.123979] [DEBUG] [LOCAL ] PPT: Loaded 19733 existing records from primary storage. [2025-10-20T22:13:24.172402] [INFO ] [MAIN ] PPT: Loaded 19733 primary data records. [2025-10-20T22:13:25.002124] [DEBUG] [MAIN ] PRH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:25.008472] [INFO ] [MAIN ] PRH: Generated 1794 derivative records from 4740 primary records. [2025-10-20T22:13:25.008574] [DEBUG] [LOCAL ] PRH: Saving derivative data. [2025-10-20T22:13:25.007586] [DEBUG] [MAIN ] PRCS: Filtered 7001 bins outside market hours, 7822 incomplete bins [2025-10-20T22:13:25.010406] [INFO ] [MAIN ] PRCS: Generated 1792 derivative records from 2786 primary records. [2025-10-20T22:13:25.010576] [DEBUG] [LOCAL ] PRCS: Saving derivative data. [2025-10-20T22:13:25.034780] [DEBUG] [LOCAL ] PRH: Derivative data saved (1794 records). [2025-10-20T22:13:25.042428] [INFO ] [MAIN ] PRH: Successfully saved derivative data. [2025-10-20T22:13:25.042328] [DEBUG] [LOCAL ] PRCS: Derivative data saved (1792 records). [2025-10-20T22:13:25.044619] [INFO ] [MAIN ] Derivative Progress: 1396/1877 symbols completed. PRH took 1.1s. Estimated remaining: 3m 1s. Rate: 159.6 symbols/min [2025-10-20T22:13:25.052695] [INFO ] [MAIN ] PRCS: Successfully saved derivative data. [2025-10-20T22:13:25.057815] [INFO ] [MAIN ] PRIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:25.060805] [DEBUG] [MAIN ] PPT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:25.061877] [INFO ] [MAIN ] PPT: Generated 1794 derivative records from 19733 primary records. [2025-10-20T22:13:25.061958] [DEBUG] [LOCAL ] PPT: Saving derivative data. [2025-10-20T22:13:25.060403] [INFO ] [MAIN ] Derivative Progress: 1397/1877 symbols completed. PRCS took 1.1s. Estimated remaining: 3m 0s. Rate: 159.7 symbols/min [2025-10-20T22:13:25.062710] [INFO ] [MAIN ] PRKS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:25.079484] [DEBUG] [LOCAL ] PPT: Derivative data saved (1794 records). [2025-10-20T22:13:25.079621] [INFO ] [MAIN ] PPT: Successfully saved derivative data. [2025-10-20T22:13:25.081766] [INFO ] [MAIN ] Derivative Progress: 1398/1877 symbols completed. PPT took 1.1s. Estimated remaining: 2m 60s. Rate: 159.8 symbols/min [2025-10-20T22:13:25.081966] [INFO ] [MAIN ] PRM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:25.124215] [DEBUG] [LOCAL ] PRIM: Loaded 44184 existing records from primary storage. [2025-10-20T22:13:25.136485] [INFO ] [MAIN ] PRIM: Loaded 44184 primary data records. [2025-10-20T22:13:25.178201] [DEBUG] [LOCAL ] PRKS: Loaded 42462 existing records from primary storage. [2025-10-20T22:13:25.203418] [INFO ] [MAIN ] PRKS: Loaded 42462 primary data records. [2025-10-20T22:13:25.333416] [DEBUG] [LOCAL ] PRM: Loaded 44358 existing records from primary storage. [2025-10-20T22:13:25.340516] [INFO ] [MAIN ] PRM: Loaded 44358 primary data records. [2025-10-20T22:13:26.079943] [DEBUG] [MAIN ] PRIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:26.083061] [INFO ] [MAIN ] PRIM: Generated 1794 derivative records from 44184 primary records. [2025-10-20T22:13:26.083398] [DEBUG] [LOCAL ] PRIM: Saving derivative data. [2025-10-20T22:13:26.113215] [DEBUG] [LOCAL ] PRIM: Derivative data saved (1794 records). [2025-10-20T22:13:26.114770] [INFO ] [MAIN ] PRIM: Successfully saved derivative data. [2025-10-20T22:13:26.116744] [INFO ] [MAIN ] Derivative Progress: 1399/1877 symbols completed. PRIM took 1.1s. Estimated remaining: 2m 60s. Rate: 159.6 symbols/min [2025-10-20T22:13:26.117351] [DEBUG] [MAIN ] PRKS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:26.117625] [INFO ] [MAIN ] PRMB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:26.118581] [INFO ] [MAIN ] PRKS: Generated 1794 derivative records from 42462 primary records. [2025-10-20T22:13:26.131594] [DEBUG] [LOCAL ] PRKS: Saving derivative data. [2025-10-20T22:13:26.131545] [DEBUG] [MAIN ] PRM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:26.132988] [INFO ] [MAIN ] PRM: Generated 1794 derivative records from 44358 primary records. [2025-10-20T22:13:26.133331] [DEBUG] [LOCAL ] PRM: Saving derivative data. [2025-10-20T22:13:26.149476] [DEBUG] [LOCAL ] PRKS: Derivative data saved (1794 records). [2025-10-20T22:13:26.161667] [INFO ] [MAIN ] PRKS: Successfully saved derivative data. [2025-10-20T22:13:26.166273] [INFO ] [MAIN ] Derivative Progress: 1400/1877 symbols completed. PRKS took 1.1s. Estimated remaining: 2m 59s. Rate: 159.7 symbols/min [2025-10-20T22:13:26.164335] [DEBUG] [LOCAL ] PRM: Derivative data saved (1794 records). [2025-10-20T22:13:26.166700] [INFO ] [MAIN ] PRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:26.166399] [INFO ] [MAIN ] PRM: Successfully saved derivative data. [2025-10-20T22:13:26.169110] [INFO ] [MAIN ] Derivative Progress: 1401/1877 symbols completed. PRM took 1.1s. Estimated remaining: 2m 59s. Rate: 159.8 symbols/min [2025-10-20T22:13:26.169674] [INFO ] [MAIN ] PRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:26.177380] [DEBUG] [LOCAL ] PRT: Loaded 5153 existing records from primary storage. [2025-10-20T22:13:26.177848] [INFO ] [MAIN ] PRT: Loaded 5153 primary data records. [2025-10-20T22:13:26.399074] [DEBUG] [LOCAL ] PRMB: Loaded 53123 existing records from primary storage. [2025-10-20T22:13:26.462851] [INFO ] [MAIN ] PRMB: Loaded 53123 primary data records. [2025-10-20T22:13:26.582710] [DEBUG] [LOCAL ] PRO: Loaded 40179 existing records from primary storage. [2025-10-20T22:13:26.603579] [INFO ] [MAIN ] PRO: Loaded 40179 primary data records. [2025-10-20T22:13:27.224039] [DEBUG] [MAIN ] PRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:27.225983] [INFO ] [MAIN ] PRT: Generated 1794 derivative records from 5153 primary records. [2025-10-20T22:13:27.226760] [DEBUG] [LOCAL ] PRT: Saving derivative data. [2025-10-20T22:13:27.243703] [DEBUG] [LOCAL ] PRT: Derivative data saved (1794 records). [2025-10-20T22:13:27.243916] [INFO ] [MAIN ] PRT: Successfully saved derivative data. [2025-10-20T22:13:27.246072] [INFO ] [MAIN ] Derivative Progress: 1402/1877 symbols completed. PRT took 1.1s. Estimated remaining: 2m 59s. Rate: 159.6 symbols/min [2025-10-20T22:13:27.246768] [INFO ] [MAIN ] PSBD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:27.270009] [DEBUG] [MAIN ] PRMB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:27.271055] [INFO ] [MAIN ] PRMB: Generated 1794 derivative records from 53123 primary records. [2025-10-20T22:13:27.271158] [DEBUG] [LOCAL ] PRMB: Saving derivative data. [2025-10-20T22:13:27.287884] [DEBUG] [LOCAL ] PSBD: Loaded 8153 existing records from primary storage. [2025-10-20T22:13:27.292622] [DEBUG] [LOCAL ] PRMB: Derivative data saved (1794 records). [2025-10-20T22:13:27.292100] [INFO ] [MAIN ] PSBD: Loaded 8153 primary data records. [2025-10-20T22:13:27.313490] [INFO ] [MAIN ] PRMB: Successfully saved derivative data. [2025-10-20T22:13:27.338550] [INFO ] [MAIN ] Derivative Progress: 1403/1877 symbols completed. PRMB took 1.2s. Estimated remaining: 2m 58s. Rate: 159.7 symbols/min [2025-10-20T22:13:27.364804] [INFO ] [MAIN ] PSF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:27.497066] [DEBUG] [LOCAL ] PSF: Loaded 7015 existing records from primary storage. [2025-10-20T22:13:27.611208] [INFO ] [MAIN ] PSF: Loaded 7015 primary data records. [2025-10-20T22:13:27.603432] [DEBUG] [MAIN ] PRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:27.644010] [INFO ] [MAIN ] PRO: Generated 1794 derivative records from 40179 primary records. [2025-10-20T22:13:27.664740] [DEBUG] [LOCAL ] PRO: Saving derivative data. [2025-10-20T22:13:27.945244] [DEBUG] [LOCAL ] PRO: Derivative data saved (1794 records). [2025-10-20T22:13:27.945595] [INFO ] [MAIN ] PRO: Successfully saved derivative data. [2025-10-20T22:13:27.947750] [INFO ] [MAIN ] Derivative Progress: 1404/1877 symbols completed. PRO took 1.8s. Estimated remaining: 2m 58s. Rate: 159.6 symbols/min [2025-10-20T22:13:27.948229] [INFO ] [MAIN ] PSFE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:27.970067] [DEBUG] [MAIN ] PSBD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:27.978626] [INFO ] [MAIN ] PSBD: Generated 1794 derivative records from 8153 primary records. [2025-10-20T22:13:27.979184] [DEBUG] [LOCAL ] PSBD: Saving derivative data. [2025-10-20T22:13:28.007404] [DEBUG] [LOCAL ] PSFE: Loaded 29874 existing records from primary storage. [2025-10-20T22:13:28.020832] [INFO ] [MAIN ] PSFE: Loaded 29874 primary data records. [2025-10-20T22:13:28.021621] [DEBUG] [LOCAL ] PSBD: Derivative data saved (1794 records). [2025-10-20T22:13:28.021833] [INFO ] [MAIN ] PSBD: Successfully saved derivative data. [2025-10-20T22:13:28.024020] [INFO ] [MAIN ] Derivative Progress: 1405/1877 symbols completed. PSBD took 0.8s. Estimated remaining: 2m 57s. Rate: 159.7 symbols/min [2025-10-20T22:13:28.024373] [DEBUG] [MAIN ] PSF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:28.046196] [INFO ] [MAIN ] PSF: Generated 1794 derivative records from 7015 primary records. [2025-10-20T22:13:28.056754] [INFO ] [MAIN ] PSO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:28.061997] [DEBUG] [LOCAL ] PSF: Saving derivative data. [2025-10-20T22:13:28.142608] [DEBUG] [LOCAL ] PSF: Derivative data saved (1794 records). [2025-10-20T22:13:28.146392] [INFO ] [MAIN ] PSF: Successfully saved derivative data. [2025-10-20T22:13:28.153911] [INFO ] [MAIN ] Derivative Progress: 1406/1877 symbols completed. PSF took 0.8s. Estimated remaining: 2m 57s. Rate: 159.8 symbols/min [2025-10-20T22:13:28.174891] [INFO ] [MAIN ] PSQH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:28.411886] [DEBUG] [LOCAL ] PSQH: Loaded 35389 existing records from primary storage. [2025-10-20T22:13:28.413519] [INFO ] [MAIN ] PSQH: Loaded 35389 primary data records. [2025-10-20T22:13:28.413442] [DEBUG] [LOCAL ] PSO: Loaded 36830 existing records from primary storage. [2025-10-20T22:13:28.413700] [INFO ] [MAIN ] PSO: Loaded 36830 primary data records. [2025-10-20T22:13:29.026341] [DEBUG] [MAIN ] PSFE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:29.028499] [INFO ] [MAIN ] PSFE: Generated 1794 derivative records from 29874 primary records. [2025-10-20T22:13:29.029842] [DEBUG] [LOCAL ] PSFE: Saving derivative data. [2025-10-20T22:13:29.051971] [DEBUG] [LOCAL ] PSFE: Derivative data saved (1794 records). [2025-10-20T22:13:29.052330] [INFO ] [MAIN ] PSFE: Successfully saved derivative data. [2025-10-20T22:13:29.054511] [INFO ] [MAIN ] Derivative Progress: 1407/1877 symbols completed. PSFE took 1.1s. Estimated remaining: 2m 57s. Rate: 159.6 symbols/min [2025-10-20T22:13:29.055238] [INFO ] [MAIN ] PSTL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:29.076389] [DEBUG] [MAIN ] PSQH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:29.081989] [INFO ] [MAIN ] PSQH: Generated 1794 derivative records from 35389 primary records. [2025-10-20T22:13:29.082184] [DEBUG] [LOCAL ] PSQH: Saving derivative data. [2025-10-20T22:13:29.095513] [DEBUG] [MAIN ] PSO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:29.096557] [INFO ] [MAIN ] PSO: Generated 1794 derivative records from 36830 primary records. [2025-10-20T22:13:29.096670] [DEBUG] [LOCAL ] PSO: Saving derivative data. [2025-10-20T22:13:29.131048] [DEBUG] [LOCAL ] PSTL: Loaded 20025 existing records from primary storage. [2025-10-20T22:13:29.132676] [INFO ] [MAIN ] PSTL: Loaded 20025 primary data records. [2025-10-20T22:13:29.130672] [DEBUG] [LOCAL ] PSQH: Derivative data saved (1794 records). [2025-10-20T22:13:29.134730] [DEBUG] [LOCAL ] PSO: Derivative data saved (1794 records). [2025-10-20T22:13:29.134861] [INFO ] [MAIN ] PSO: Successfully saved derivative data. [2025-10-20T22:13:29.134948] [INFO ] [MAIN ] PSQH: Successfully saved derivative data. [2025-10-20T22:13:29.142388] [INFO ] [MAIN ] Derivative Progress: 1408/1877 symbols completed. PSQH took 1.0s. Estimated remaining: 2m 56s. Rate: 159.7 symbols/min [2025-10-20T22:13:29.171217] [INFO ] [MAIN ] Derivative Progress: 1409/1877 symbols completed. PSO took 1.1s. Estimated remaining: 2m 56s. Rate: 159.8 symbols/min [2025-10-20T22:13:29.450389] [DEBUG] [MAIN ] PSTL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:29.451001] [INFO ] [MAIN ] PSTL: Generated 1794 derivative records from 20025 primary records. [2025-10-20T22:13:29.451062] [DEBUG] [LOCAL ] PSTL: Saving derivative data. [2025-10-20T22:13:29.465873] [DEBUG] [LOCAL ] PSTL: Derivative data saved (1794 records). [2025-10-20T22:13:29.465930] [INFO ] [MAIN ] PSTL: Successfully saved derivative data. [2025-10-20T22:13:29.467806] [INFO ] [MAIN ] Derivative Progress: 1410/1877 symbols completed. PSTL took 0.4s. Estimated remaining: 2m 55s. Rate: 159.8 symbols/min [2025-10-20T22:13:29.468240] [INFO ] [MAIN ] PTA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:29.468342] [INFO ] [MAIN ] PTY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:29.468760] [INFO ] [MAIN ] PUK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:29.486681] [DEBUG] [LOCAL ] PTA: Loaded 15320 existing records from primary storage. [2025-10-20T22:13:29.486900] [INFO ] [MAIN ] PTA: Loaded 15320 primary data records. [2025-10-20T22:13:29.823631] [DEBUG] [LOCAL ] PTY: Loaded 37923 existing records from primary storage. [2025-10-20T22:13:29.822545] [DEBUG] [LOCAL ] PUK: Loaded 44463 existing records from primary storage. [2025-10-20T22:13:29.831771] [INFO ] [MAIN ] PTY: Loaded 37923 primary data records. [2025-10-20T22:13:29.832154] [INFO ] [MAIN ] PUK: Loaded 44463 primary data records. [2025-10-20T22:13:30.483350] [DEBUG] [MAIN ] PTA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:30.484293] [INFO ] [MAIN ] PTA: Generated 1794 derivative records from 15320 primary records. [2025-10-20T22:13:30.484477] [DEBUG] [LOCAL ] PTA: Saving derivative data. [2025-10-20T22:13:30.503908] [DEBUG] [LOCAL ] PTA: Derivative data saved (1794 records). [2025-10-20T22:13:30.504218] [INFO ] [MAIN ] PTA: Successfully saved derivative data. [2025-10-20T22:13:30.506454] [INFO ] [MAIN ] Derivative Progress: 1411/1877 symbols completed. PTA took 1.0s. Estimated remaining: 2m 55s. Rate: 159.6 symbols/min [2025-10-20T22:13:30.506827] [INFO ] [MAIN ] PVL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:30.518528] [DEBUG] [LOCAL ] PVL: Loaded 4742 existing records from primary storage. [2025-10-20T22:13:30.530594] [INFO ] [MAIN ] PVL: Loaded 4742 primary data records. [2025-10-20T22:13:30.548674] [DEBUG] [MAIN ] PUK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:30.549550] [INFO ] [MAIN ] PUK: Generated 1794 derivative records from 44463 primary records. [2025-10-20T22:13:30.549983] [DEBUG] [MAIN ] PTY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:30.549670] [DEBUG] [LOCAL ] PUK: Saving derivative data. [2025-10-20T22:13:30.578292] [INFO ] [MAIN ] PTY: Generated 1794 derivative records from 37923 primary records. [2025-10-20T22:13:30.654129] [DEBUG] [LOCAL ] PTY: Saving derivative data. [2025-10-20T22:13:30.665791] [DEBUG] [LOCAL ] PUK: Derivative data saved (1794 records). [2025-10-20T22:13:30.684706] [INFO ] [MAIN ] PUK: Successfully saved derivative data. [2025-10-20T22:13:30.686761] [INFO ] [MAIN ] Derivative Progress: 1412/1877 symbols completed. PUK took 1.2s. Estimated remaining: 2m 55s. Rate: 159.7 symbols/min [2025-10-20T22:13:30.710055] [DEBUG] [LOCAL ] PTY: Derivative data saved (1794 records). [2025-10-20T22:13:30.710173] [INFO ] [MAIN ] PTY: Successfully saved derivative data. [2025-10-20T22:13:30.710261] [INFO ] [MAIN ] PX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:30.734186] [INFO ] [MAIN ] Derivative Progress: 1413/1877 symbols completed. PTY took 1.3s. Estimated remaining: 2m 54s. Rate: 159.8 symbols/min [2025-10-20T22:13:30.734315] [INFO ] [MAIN ] QD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:30.909358] [DEBUG] [LOCAL ] QD: Loaded 27590 existing records from primary storage. [2025-10-20T22:13:30.909741] [INFO ] [MAIN ] QD: Loaded 27590 primary data records. [2025-10-20T22:13:31.013796] [DEBUG] [LOCAL ] PX: Loaded 34272 existing records from primary storage. [2025-10-20T22:13:31.024437] [INFO ] [MAIN ] PX: Loaded 34272 primary data records. [2025-10-20T22:13:31.143767] [DEBUG] [MAIN ] PVL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:31.186439] [INFO ] [MAIN ] PVL: Generated 1794 derivative records from 4742 primary records. [2025-10-20T22:13:31.307560] [DEBUG] [LOCAL ] PVL: Saving derivative data. [2025-10-20T22:13:31.554249] [DEBUG] [LOCAL ] PVL: Derivative data saved (1794 records). [2025-10-20T22:13:31.554564] [INFO ] [MAIN ] PVL: Successfully saved derivative data. [2025-10-20T22:13:31.556533] [INFO ] [MAIN ] Derivative Progress: 1414/1877 symbols completed. PVL took 1.0s. Estimated remaining: 2m 54s. Rate: 159.7 symbols/min [2025-10-20T22:13:31.556905] [INFO ] [MAIN ] QGEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:31.583280] [DEBUG] [MAIN ] QD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:31.583977] [INFO ] [MAIN ] QD: Generated 1794 derivative records from 27590 primary records. [2025-10-20T22:13:31.584261] [DEBUG] [LOCAL ] QD: Saving derivative data. [2025-10-20T22:13:31.602772] [DEBUG] [LOCAL ] QD: Derivative data saved (1794 records). [2025-10-20T22:13:31.612907] [INFO ] [MAIN ] QD: Successfully saved derivative data. [2025-10-20T22:13:31.614968] [INFO ] [MAIN ] Derivative Progress: 1415/1877 symbols completed. QD took 0.9s. Estimated remaining: 2m 54s. Rate: 159.8 symbols/min [2025-10-20T22:13:31.615393] [INFO ] [MAIN ] QSR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:31.628925] [DEBUG] [MAIN ] PX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:31.629559] [INFO ] [MAIN ] PX: Generated 1794 derivative records from 34272 primary records. [2025-10-20T22:13:31.629695] [DEBUG] [LOCAL ] PX: Saving derivative data. [2025-10-20T22:13:31.663043] [DEBUG] [LOCAL ] PX: Derivative data saved (1794 records). [2025-10-20T22:13:31.661868] [DEBUG] [LOCAL ] QGEN: Loaded 49088 existing records from primary storage. [2025-10-20T22:13:31.663286] [INFO ] [MAIN ] PX: Successfully saved derivative data. [2025-10-20T22:13:31.663372] [INFO ] [MAIN ] QGEN: Loaded 49088 primary data records. [2025-10-20T22:13:31.665848] [INFO ] [MAIN ] Derivative Progress: 1416/1877 symbols completed. PX took 1.0s. Estimated remaining: 2m 53s. Rate: 159.9 symbols/min [2025-10-20T22:13:31.666311] [INFO ] [MAIN ] QTWO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:32.003019] [DEBUG] [LOCAL ] QSR: Loaded 52497 existing records from primary storage. [2025-10-20T22:13:32.003340] [INFO ] [MAIN ] QSR: Loaded 52497 primary data records. [2025-10-20T22:13:32.097353] [DEBUG] [LOCAL ] QTWO: Loaded 39535 existing records from primary storage. [2025-10-20T22:13:32.122272] [INFO ] [MAIN ] QTWO: Loaded 39535 primary data records. [2025-10-20T22:13:32.697773] [DEBUG] [MAIN ] QGEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:32.700402] [INFO ] [MAIN ] QGEN: Generated 1794 derivative records from 49088 primary records. [2025-10-20T22:13:32.700697] [DEBUG] [LOCAL ] QGEN: Saving derivative data. [2025-10-20T22:13:32.723805] [DEBUG] [LOCAL ] QGEN: Derivative data saved (1794 records). [2025-10-20T22:13:32.723994] [INFO ] [MAIN ] QGEN: Successfully saved derivative data. [2025-10-20T22:13:32.726543] [INFO ] [MAIN ] Derivative Progress: 1417/1877 symbols completed. QGEN took 1.2s. Estimated remaining: 2m 53s. Rate: 159.7 symbols/min [2025-10-20T22:13:32.726771] [INFO ] [MAIN ] QUAD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:32.747126] [DEBUG] [MAIN ] QSR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:32.748316] [INFO ] [MAIN ] QSR: Generated 1794 derivative records from 52497 primary records. [2025-10-20T22:13:32.748742] [DEBUG] [LOCAL ] QSR: Saving derivative data. [2025-10-20T22:13:32.771832] [DEBUG] [LOCAL ] QSR: Derivative data saved (1794 records). [2025-10-20T22:13:32.781992] [DEBUG] [LOCAL ] QUAD: Loaded 23177 existing records from primary storage. [2025-10-20T22:13:32.782195] [INFO ] [MAIN ] QUAD: Loaded 23177 primary data records. [2025-10-20T22:13:32.781518] [INFO ] [MAIN ] QSR: Successfully saved derivative data. [2025-10-20T22:13:32.791890] [INFO ] [MAIN ] Derivative Progress: 1418/1877 symbols completed. QSR took 1.2s. Estimated remaining: 2m 52s. Rate: 159.7 symbols/min [2025-10-20T22:13:32.802583] [DEBUG] [MAIN ] QTWO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:32.816898] [INFO ] [MAIN ] QVCC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:32.828945] [INFO ] [MAIN ] QTWO: Generated 1794 derivative records from 39535 primary records. [2025-10-20T22:13:32.829022] [DEBUG] [LOCAL ] QTWO: Saving derivative data. [2025-10-20T22:13:32.900876] [DEBUG] [LOCAL ] QTWO: Derivative data saved (1794 records). [2025-10-20T22:13:32.913355] [INFO ] [MAIN ] QTWO: Successfully saved derivative data. [2025-10-20T22:13:32.921278] [INFO ] [MAIN ] Derivative Progress: 1419/1877 symbols completed. QTWO took 1.3s. Estimated remaining: 2m 52s. Rate: 159.8 symbols/min [2025-10-20T22:13:32.938788] [INFO ] [MAIN ] QVCD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:32.942838] [DEBUG] [LOCAL ] QVCC: Loaded 8741 existing records from primary storage. [2025-10-20T22:13:32.943550] [INFO ] [MAIN ] QVCC: Loaded 8741 primary data records. [2025-10-20T22:13:33.100218] [DEBUG] [LOCAL ] QVCD: Loaded 4608 existing records from primary storage. [2025-10-20T22:13:33.230106] [INFO ] [MAIN ] QVCD: Loaded 4608 primary data records. [2025-10-20T22:13:33.784508] [DEBUG] [MAIN ] QUAD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:33.785305] [INFO ] [MAIN ] QUAD: Generated 1794 derivative records from 23177 primary records. [2025-10-20T22:13:33.785556] [DEBUG] [LOCAL ] QUAD: Saving derivative data. [2025-10-20T22:13:33.815714] [DEBUG] [LOCAL ] QUAD: Derivative data saved (1794 records). [2025-10-20T22:13:33.815910] [INFO ] [MAIN ] QUAD: Successfully saved derivative data. [2025-10-20T22:13:33.818546] [INFO ] [MAIN ] Derivative Progress: 1420/1877 symbols completed. QUAD took 1.1s. Estimated remaining: 2m 52s. Rate: 159.7 symbols/min [2025-10-20T22:13:33.819189] [INFO ] [MAIN ] RA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:33.818871] [DEBUG] [MAIN ] QVCC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:33.820032] [INFO ] [MAIN ] QVCC: Generated 1794 derivative records from 8741 primary records. [2025-10-20T22:13:33.820569] [DEBUG] [LOCAL ] QVCC: Saving derivative data. [2025-10-20T22:13:33.836070] [DEBUG] [LOCAL ] QVCC: Derivative data saved (1794 records). [2025-10-20T22:13:33.836248] [INFO ] [MAIN ] QVCC: Successfully saved derivative data. [2025-10-20T22:13:33.838295] [INFO ] [MAIN ] Derivative Progress: 1421/1877 symbols completed. QVCC took 1.0s. Estimated remaining: 2m 51s. Rate: 159.8 symbols/min [2025-10-20T22:13:33.838483] [INFO ] [MAIN ] RACE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:33.859865] [DEBUG] [MAIN ] QVCD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:33.860559] [INFO ] [MAIN ] QVCD: Generated 1794 derivative records from 4608 primary records. [2025-10-20T22:13:33.860733] [DEBUG] [LOCAL ] QVCD: Saving derivative data. [2025-10-20T22:13:33.883207] [DEBUG] [LOCAL ] QVCD: Derivative data saved (1794 records). [2025-10-20T22:13:33.881885] [DEBUG] [LOCAL ] RA: Loaded 21865 existing records from primary storage. [2025-10-20T22:13:33.883419] [INFO ] [MAIN ] QVCD: Successfully saved derivative data. [2025-10-20T22:13:33.883498] [INFO ] [MAIN ] RA: Loaded 21865 primary data records. [2025-10-20T22:13:33.897103] [INFO ] [MAIN ] Derivative Progress: 1422/1877 symbols completed. QVCD took 1.0s. Estimated remaining: 2m 51s. Rate: 159.9 symbols/min [2025-10-20T22:13:33.929310] [INFO ] [MAIN ] RBOT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:34.036412] [DEBUG] [LOCAL ] RACE: Loaded 32933 existing records from primary storage. [2025-10-20T22:13:34.057573] [INFO ] [MAIN ] RACE: Loaded 32933 primary data records. [2025-10-20T22:13:34.213571] [DEBUG] [LOCAL ] RBOT: Loaded 6187 existing records from primary storage. [2025-10-20T22:13:34.285791] [INFO ] [MAIN ] RBOT: Loaded 6187 primary data records. [2025-10-20T22:13:34.880385] [DEBUG] [MAIN ] RA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:34.881674] [INFO ] [MAIN ] RA: Generated 1794 derivative records from 21865 primary records. [2025-10-20T22:13:34.881996] [DEBUG] [LOCAL ] RA: Saving derivative data. [2025-10-20T22:13:34.898139] [DEBUG] [LOCAL ] RA: Derivative data saved (1794 records). [2025-10-20T22:13:34.898365] [INFO ] [MAIN ] RA: Successfully saved derivative data. [2025-10-20T22:13:34.900819] [INFO ] [MAIN ] Derivative Progress: 1423/1877 symbols completed. RA took 1.1s. Estimated remaining: 2m 51s. Rate: 159.7 symbols/min [2025-10-20T22:13:34.918945] [DEBUG] [MAIN ] RACE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:34.919748] [INFO ] [MAIN ] RACE: Generated 1794 derivative records from 32933 primary records. [2025-10-20T22:13:34.919922] [DEBUG] [LOCAL ] RACE: Saving derivative data. [2025-10-20T22:13:34.938522] [DEBUG] [LOCAL ] RACE: Derivative data saved (1794 records). [2025-10-20T22:13:34.938666] [INFO ] [MAIN ] RACE: Successfully saved derivative data. [2025-10-20T22:13:34.940820] [INFO ] [MAIN ] Derivative Progress: 1424/1877 symbols completed. RACE took 1.1s. Estimated remaining: 2m 50s. Rate: 159.8 symbols/min [2025-10-20T22:13:34.953380] [DEBUG] [MAIN ] RBOT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:34.954035] [INFO ] [MAIN ] RBOT: Generated 1794 derivative records from 6187 primary records. [2025-10-20T22:13:34.954097] [DEBUG] [LOCAL ] RBOT: Saving derivative data. [2025-10-20T22:13:34.967998] [DEBUG] [LOCAL ] RBOT: Derivative data saved (1794 records). [2025-10-20T22:13:34.968056] [INFO ] [MAIN ] RBOT: Successfully saved derivative data. [2025-10-20T22:13:34.970119] [INFO ] [MAIN ] Derivative Progress: 1425/1877 symbols completed. RBOT took 1.0s. Estimated remaining: 2m 50s. Rate: 159.9 symbols/min [2025-10-20T22:13:34.970634] [INFO ] [MAIN ] RBRK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:34.970824] [INFO ] [MAIN ] RCD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:34.970756] [INFO ] [MAIN ] RCI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:34.978608] [DEBUG] [LOCAL ] RCD: Loaded 3357 existing records from primary storage. [2025-10-20T22:13:34.978824] [INFO ] [MAIN ] RCD: Loaded 3357 primary data records. [2025-10-20T22:13:35.334624] [DEBUG] [LOCAL ] RBRK: Loaded 53169 existing records from primary storage. [2025-10-20T22:13:35.335796] [INFO ] [MAIN ] RBRK: Loaded 53169 primary data records. [2025-10-20T22:13:35.337386] [DEBUG] [LOCAL ] RCI: Loaded 48624 existing records from primary storage. [2025-10-20T22:13:35.343114] [INFO ] [MAIN ] RCI: Loaded 48624 primary data records. [2025-10-20T22:13:36.003746] [DEBUG] [MAIN ] RCD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:36.005547] [INFO ] [MAIN ] RCD: Generated 1794 derivative records from 3357 primary records. [2025-10-20T22:13:36.005896] [DEBUG] [LOCAL ] RCD: Saving derivative data. [2025-10-20T22:13:36.022253] [DEBUG] [LOCAL ] RCD: Derivative data saved (1794 records). [2025-10-20T22:13:36.022378] [INFO ] [MAIN ] RCD: Successfully saved derivative data. [2025-10-20T22:13:36.024747] [INFO ] [MAIN ] Derivative Progress: 1426/1877 symbols completed. RCD took 1.1s. Estimated remaining: 2m 49s. Rate: 159.7 symbols/min [2025-10-20T22:13:36.025124] [INFO ] [MAIN ] RCS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:36.041168] [DEBUG] [LOCAL ] RCS: Loaded 12476 existing records from primary storage. [2025-10-20T22:13:36.044220] [INFO ] [MAIN ] RCS: Loaded 12476 primary data records. [2025-10-20T22:13:36.121284] [DEBUG] [MAIN ] RBRK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:36.147496] [DEBUG] [MAIN ] RCI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:36.148415] [INFO ] [MAIN ] RCI: Generated 1794 derivative records from 48624 primary records. [2025-10-20T22:13:36.158798] [DEBUG] [LOCAL ] RCI: Saving derivative data. [2025-10-20T22:13:36.174462] [INFO ] [MAIN ] RBRK: Generated 1794 derivative records from 53169 primary records. [2025-10-20T22:13:36.185153] [DEBUG] [LOCAL ] RBRK: Saving derivative data. [2025-10-20T22:13:36.210106] [DEBUG] [LOCAL ] RCI: Derivative data saved (1794 records). [2025-10-20T22:13:36.234316] [INFO ] [MAIN ] RCI: Successfully saved derivative data. [2025-10-20T22:13:36.277078] [INFO ] [MAIN ] Derivative Progress: 1427/1877 symbols completed. RCI took 1.3s. Estimated remaining: 2m 49s. Rate: 159.7 symbols/min [2025-10-20T22:13:36.281489] [INFO ] [MAIN ] RDY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:36.291092] [DEBUG] [LOCAL ] RBRK: Derivative data saved (1794 records). [2025-10-20T22:13:36.291320] [INFO ] [MAIN ] RBRK: Successfully saved derivative data. [2025-10-20T22:13:36.293266] [INFO ] [MAIN ] Derivative Progress: 1428/1877 symbols completed. RBRK took 1.3s. Estimated remaining: 2m 49s. Rate: 159.8 symbols/min [2025-10-20T22:13:36.293414] [INFO ] [MAIN ] RELX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:36.456926] [DEBUG] [MAIN ] RCS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:36.457546] [INFO ] [MAIN ] RCS: Generated 1794 derivative records from 12476 primary records. [2025-10-20T22:13:36.457607] [DEBUG] [LOCAL ] RCS: Saving derivative data. [2025-10-20T22:13:36.473498] [DEBUG] [LOCAL ] RCS: Derivative data saved (1794 records). [2025-10-20T22:13:36.487306] [INFO ] [MAIN ] RCS: Successfully saved derivative data. [2025-10-20T22:13:36.503009] [INFO ] [MAIN ] Derivative Progress: 1429/1877 symbols completed. RCS took 0.5s. Estimated remaining: 2m 48s. Rate: 159.9 symbols/min [2025-10-20T22:13:36.504148] [DEBUG] [LOCAL ] RDY: Loaded 46400 existing records from primary storage. [2025-10-20T22:13:36.504313] [INFO ] [MAIN ] RERE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:36.504407] [INFO ] [MAIN ] RDY: Loaded 46400 primary data records. [2025-10-20T22:13:36.505627] [DEBUG] [LOCAL ] RELX: Loaded 48572 existing records from primary storage. [2025-10-20T22:13:36.526966] [INFO ] [MAIN ] RELX: Loaded 48572 primary data records. [2025-10-20T22:13:37.131568] [DEBUG] [LOCAL ] RERE: Loaded 41702 existing records from primary storage. [2025-10-20T22:13:37.131843] [INFO ] [MAIN ] RERE: Loaded 41702 primary data records. [2025-10-20T22:13:37.474343] [DEBUG] [MAIN ] RDY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:37.476152] [INFO ] [MAIN ] RDY: Generated 1794 derivative records from 46400 primary records. [2025-10-20T22:13:37.476424] [DEBUG] [LOCAL ] RDY: Saving derivative data. [2025-10-20T22:13:37.478361] [DEBUG] [MAIN ] RELX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:37.494040] [DEBUG] [LOCAL ] RDY: Derivative data saved (1794 records). [2025-10-20T22:13:37.494199] [INFO ] [MAIN ] RDY: Successfully saved derivative data. [2025-10-20T22:13:37.495087] [INFO ] [MAIN ] RELX: Generated 1794 derivative records from 48572 primary records. [2025-10-20T22:13:37.497303] [DEBUG] [LOCAL ] RELX: Saving derivative data. [2025-10-20T22:13:37.497112] [INFO ] [MAIN ] Derivative Progress: 1430/1877 symbols completed. RDY took 1.2s. Estimated remaining: 2m 48s. Rate: 159.7 symbols/min [2025-10-20T22:13:37.498338] [INFO ] [MAIN ] REVG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:37.517724] [DEBUG] [LOCAL ] RELX: Derivative data saved (1794 records). [2025-10-20T22:13:37.519131] [INFO ] [MAIN ] RELX: Successfully saved derivative data. [2025-10-20T22:13:37.521373] [INFO ] [MAIN ] Derivative Progress: 1431/1877 symbols completed. RELX took 1.2s. Estimated remaining: 2m 47s. Rate: 159.8 symbols/min [2025-10-20T22:13:37.521709] [INFO ] [MAIN ] RFI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:37.535243] [DEBUG] [MAIN ] RERE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:37.536023] [INFO ] [MAIN ] RERE: Generated 1794 derivative records from 41702 primary records. [2025-10-20T22:13:37.536139] [DEBUG] [LOCAL ] RERE: Saving derivative data. [2025-10-20T22:13:37.555041] [DEBUG] [LOCAL ] RERE: Derivative data saved (1794 records). [2025-10-20T22:13:37.555221] [INFO ] [MAIN ] RERE: Successfully saved derivative data. [2025-10-20T22:13:37.557349] [INFO ] [MAIN ] Derivative Progress: 1432/1877 symbols completed. RERE took 1.1s. Estimated remaining: 2m 47s. Rate: 159.9 symbols/min [2025-10-20T22:13:37.565880] [INFO ] [MAIN ] RFL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:37.571418] [DEBUG] [LOCAL ] RFI: Loaded 10384 existing records from primary storage. [2025-10-20T22:13:37.571614] [INFO ] [MAIN ] RFI: Loaded 10384 primary data records. [2025-10-20T22:13:37.696725] [DEBUG] [LOCAL ] REVG: Loaded 40371 existing records from primary storage. [2025-10-20T22:13:37.696954] [DEBUG] [LOCAL ] RFL: Loaded 11457 existing records from primary storage. [2025-10-20T22:13:37.697173] [INFO ] [MAIN ] REVG: Loaded 40371 primary data records. [2025-10-20T22:13:37.697330] [INFO ] [MAIN ] RFL: Loaded 11457 primary data records. [2025-10-20T22:13:38.560313] [DEBUG] [MAIN ] RFI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:38.561945] [INFO ] [MAIN ] RFI: Generated 1794 derivative records from 10384 primary records. [2025-10-20T22:13:38.562148] [DEBUG] [LOCAL ] RFI: Saving derivative data. [2025-10-20T22:13:38.581881] [DEBUG] [LOCAL ] RFI: Derivative data saved (1794 records). [2025-10-20T22:13:38.583115] [INFO ] [MAIN ] RFI: Successfully saved derivative data. [2025-10-20T22:13:38.585060] [INFO ] [MAIN ] Derivative Progress: 1433/1877 symbols completed. RFI took 1.1s. Estimated remaining: 2m 47s. Rate: 159.7 symbols/min [2025-10-20T22:13:38.586631] [INFO ] [MAIN ] RFM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:38.600130] [DEBUG] [MAIN ] REVG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:38.601076] [INFO ] [MAIN ] REVG: Generated 1794 derivative records from 40371 primary records. [2025-10-20T22:13:38.601234] [DEBUG] [LOCAL ] REVG: Saving derivative data. [2025-10-20T22:13:38.615806] [DEBUG] [MAIN ] RFL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:38.627889] [INFO ] [MAIN ] RFL: Generated 1794 derivative records from 11457 primary records. [2025-10-20T22:13:38.628234] [DEBUG] [LOCAL ] RFL: Saving derivative data. [2025-10-20T22:13:38.628138] [DEBUG] [LOCAL ] RFM: Loaded 4195 existing records from primary storage. [2025-10-20T22:13:38.635616] [DEBUG] [LOCAL ] REVG: Derivative data saved (1794 records). [2025-10-20T22:13:38.635970] [INFO ] [MAIN ] RFM: Loaded 4195 primary data records. [2025-10-20T22:13:38.635886] [INFO ] [MAIN ] REVG: Successfully saved derivative data. [2025-10-20T22:13:38.644952] [INFO ] [MAIN ] Derivative Progress: 1434/1877 symbols completed. REVG took 1.1s. Estimated remaining: 2m 46s. Rate: 159.8 symbols/min [2025-10-20T22:13:38.660470] [INFO ] [MAIN ] RFMZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:38.690407] [DEBUG] [LOCAL ] RFL: Derivative data saved (1794 records). [2025-10-20T22:13:38.697458] [INFO ] [MAIN ] RFL: Successfully saved derivative data. [2025-10-20T22:13:38.699920] [INFO ] [MAIN ] Derivative Progress: 1435/1877 symbols completed. RFL took 1.1s. Estimated remaining: 2m 46s. Rate: 159.9 symbols/min [2025-10-20T22:13:38.712520] [INFO ] [MAIN ] RGT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:38.769212] [DEBUG] [LOCAL ] RFMZ: Loaded 8876 existing records from primary storage. [2025-10-20T22:13:38.787078] [INFO ] [MAIN ] RFMZ: Loaded 8876 primary data records. [2025-10-20T22:13:38.781875] [DEBUG] [LOCAL ] RGT: Loaded 2730 existing records from primary storage. [2025-10-20T22:13:38.818264] [INFO ] [MAIN ] RGT: Loaded 2730 primary data records. [2025-10-20T22:13:39.613812] [DEBUG] [MAIN ] RFM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:39.615241] [INFO ] [MAIN ] RFM: Generated 1794 derivative records from 4195 primary records. [2025-10-20T22:13:39.615426] [DEBUG] [LOCAL ] RFM: Saving derivative data. [2025-10-20T22:13:39.616447] [DEBUG] [MAIN ] RFMZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:39.617681] [INFO ] [MAIN ] RFMZ: Generated 1794 derivative records from 8876 primary records. [2025-10-20T22:13:39.618068] [DEBUG] [LOCAL ] RFMZ: Saving derivative data. [2025-10-20T22:13:39.633691] [DEBUG] [LOCAL ] RFM: Derivative data saved (1794 records). [2025-10-20T22:13:39.641940] [INFO ] [MAIN ] RFM: Successfully saved derivative data. [2025-10-20T22:13:39.644098] [INFO ] [MAIN ] Derivative Progress: 1436/1877 symbols completed. RFM took 1.1s. Estimated remaining: 2m 46s. Rate: 159.7 symbols/min [2025-10-20T22:13:39.648846] [INFO ] [MAIN ] RIO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:39.660691] [DEBUG] [LOCAL ] RFMZ: Derivative data saved (1794 records). [2025-10-20T22:13:39.660787] [INFO ] [MAIN ] RFMZ: Successfully saved derivative data. [2025-10-20T22:13:39.667829] [INFO ] [MAIN ] Derivative Progress: 1437/1877 symbols completed. RFMZ took 1.0s. Estimated remaining: 2m 45s. Rate: 159.8 symbols/min [2025-10-20T22:13:39.668169] [DEBUG] [MAIN ] RGT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:39.669221] [INFO ] [MAIN ] RIV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:39.669119] [INFO ] [MAIN ] RGT: Generated 1794 derivative records from 2730 primary records. [2025-10-20T22:13:39.669825] [DEBUG] [LOCAL ] RGT: Saving derivative data. [2025-10-20T22:13:39.684202] [DEBUG] [LOCAL ] RGT: Derivative data saved (1794 records). [2025-10-20T22:13:39.684379] [INFO ] [MAIN ] RGT: Successfully saved derivative data. [2025-10-20T22:13:39.686458] [INFO ] [MAIN ] Derivative Progress: 1438/1877 symbols completed. RGT took 1.0s. Estimated remaining: 2m 45s. Rate: 159.9 symbols/min [2025-10-20T22:13:39.697781] [DEBUG] [LOCAL ] RIV: Loaded 12682 existing records from primary storage. [2025-10-20T22:13:39.697898] [INFO ] [MAIN ] RIV: Loaded 12682 primary data records. [2025-10-20T22:13:40.033836] [DEBUG] [LOCAL ] RIO: Loaded 53402 existing records from primary storage. [2025-10-20T22:13:40.041894] [INFO ] [MAIN ] RIO: Loaded 53402 primary data records. [2025-10-20T22:13:40.373334] [DEBUG] [MAIN ] RIV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:40.374405] [INFO ] [MAIN ] RIV: Generated 1794 derivative records from 12682 primary records. [2025-10-20T22:13:40.374499] [DEBUG] [LOCAL ] RIV: Saving derivative data. [2025-10-20T22:13:40.392032] [DEBUG] [LOCAL ] RIV: Derivative data saved (1794 records). [2025-10-20T22:13:40.392100] [INFO ] [MAIN ] RIV: Successfully saved derivative data. [2025-10-20T22:13:40.394397] [INFO ] [MAIN ] Derivative Progress: 1439/1877 symbols completed. RIV took 0.7s. Estimated remaining: 2m 44s. Rate: 159.8 symbols/min [2025-10-20T22:13:40.412497] [DEBUG] [MAIN ] RIO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:40.413141] [INFO ] [MAIN ] RIO: Generated 1794 derivative records from 53402 primary records. [2025-10-20T22:13:40.413201] [DEBUG] [LOCAL ] RIO: Saving derivative data. [2025-10-20T22:13:40.428218] [DEBUG] [LOCAL ] RIO: Derivative data saved (1794 records). [2025-10-20T22:13:40.428278] [INFO ] [MAIN ] RIO: Successfully saved derivative data. [2025-10-20T22:13:40.430581] [INFO ] [MAIN ] Derivative Progress: 1440/1877 symbols completed. RIO took 0.8s. Estimated remaining: 2m 44s. Rate: 159.9 symbols/min [2025-10-20T22:13:40.431155] [INFO ] [MAIN ] RLJ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:40.431207] [INFO ] [MAIN ] RLTY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:40.431323] [INFO ] [MAIN ] RLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:40.446419] [DEBUG] [LOCAL ] RLTY: Loaded 11193 existing records from primary storage. [2025-10-20T22:13:40.446630] [INFO ] [MAIN ] RLTY: Loaded 11193 primary data records. [2025-10-20T22:13:40.810035] [DEBUG] [LOCAL ] RLX: Loaded 48187 existing records from primary storage. [2025-10-20T22:13:40.809372] [DEBUG] [LOCAL ] RLJ: Loaded 48004 existing records from primary storage. [2025-10-20T22:13:40.810182] [INFO ] [MAIN ] RLJ: Loaded 48004 primary data records. [2025-10-20T22:13:40.810266] [INFO ] [MAIN ] RLX: Loaded 48187 primary data records. [2025-10-20T22:13:41.420470] [DEBUG] [MAIN ] RLTY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:41.421310] [INFO ] [MAIN ] RLTY: Generated 1794 derivative records from 11193 primary records. [2025-10-20T22:13:41.421465] [DEBUG] [LOCAL ] RLTY: Saving derivative data. [2025-10-20T22:13:41.439339] [DEBUG] [LOCAL ] RLTY: Derivative data saved (1794 records). [2025-10-20T22:13:41.439772] [INFO ] [MAIN ] RLTY: Successfully saved derivative data. [2025-10-20T22:13:41.442209] [INFO ] [MAIN ] Derivative Progress: 1441/1877 symbols completed. RLTY took 1.0s. Estimated remaining: 2m 44s. Rate: 159.7 symbols/min [2025-10-20T22:13:41.442407] [INFO ] [MAIN ] RM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:41.462503] [DEBUG] [LOCAL ] RM: Loaded 7548 existing records from primary storage. [2025-10-20T22:13:41.468588] [DEBUG] [MAIN ] RLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:41.467421] [INFO ] [MAIN ] RM: Loaded 7548 primary data records. [2025-10-20T22:13:41.469994] [INFO ] [MAIN ] RLX: Generated 1794 derivative records from 48187 primary records. [2025-10-20T22:13:41.470810] [DEBUG] [LOCAL ] RLX: Saving derivative data. [2025-10-20T22:13:41.569335] [DEBUG] [MAIN ] RLJ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:41.600475] [INFO ] [MAIN ] RLJ: Generated 1794 derivative records from 48004 primary records. [2025-10-20T22:13:41.600572] [DEBUG] [LOCAL ] RLJ: Saving derivative data. [2025-10-20T22:13:41.637772] [DEBUG] [LOCAL ] RLX: Derivative data saved (1794 records). [2025-10-20T22:13:41.659481] [INFO ] [MAIN ] RLX: Successfully saved derivative data. [2025-10-20T22:13:41.673260] [INFO ] [MAIN ] Derivative Progress: 1442/1877 symbols completed. RLX took 1.2s. Estimated remaining: 2m 43s. Rate: 159.8 symbols/min [2025-10-20T22:13:41.711954] [DEBUG] [LOCAL ] RLJ: Derivative data saved (1794 records). [2025-10-20T22:13:41.706470] [INFO ] [MAIN ] RMAX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:41.712103] [INFO ] [MAIN ] RLJ: Successfully saved derivative data. [2025-10-20T22:13:41.714303] [INFO ] [MAIN ] Derivative Progress: 1443/1877 symbols completed. RLJ took 1.3s. Estimated remaining: 2m 43s. Rate: 159.9 symbols/min [2025-10-20T22:13:41.838967] [INFO ] [MAIN ] RMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:41.865847] [DEBUG] [LOCAL ] RMI: Loaded 4709 existing records from primary storage. [2025-10-20T22:13:41.866364] [INFO ] [MAIN ] RMI: Loaded 4709 primary data records. [2025-10-20T22:13:41.916566] [DEBUG] [LOCAL ] RMAX: Loaded 21719 existing records from primary storage. [2025-10-20T22:13:41.942712] [INFO ] [MAIN ] RMAX: Loaded 21719 primary data records. [2025-10-20T22:13:42.508332] [DEBUG] [MAIN ] RM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:42.511361] [INFO ] [MAIN ] RM: Generated 1794 derivative records from 7548 primary records. [2025-10-20T22:13:42.511537] [DEBUG] [LOCAL ] RM: Saving derivative data. [2025-10-20T22:13:42.530823] [DEBUG] [LOCAL ] RM: Derivative data saved (1794 records). [2025-10-20T22:13:42.531003] [INFO ] [MAIN ] RM: Successfully saved derivative data. [2025-10-20T22:13:42.533557] [INFO ] [MAIN ] Derivative Progress: 1444/1877 symbols completed. RM took 1.1s. Estimated remaining: 2m 43s. Rate: 159.8 symbols/min [2025-10-20T22:13:42.534233] [INFO ] [MAIN ] RMM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:42.561211] [DEBUG] [MAIN ] RMI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:13:42.574055] [DEBUG] [MAIN ] RMAX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:42.580781] [INFO ] [MAIN ] RMI: Generated 1793 derivative records from 4709 primary records. [2025-10-20T22:13:42.580150] [INFO ] [MAIN ] RMAX: Generated 1794 derivative records from 21719 primary records. [2025-10-20T22:13:42.580945] [DEBUG] [LOCAL ] RMI: Saving derivative data. [2025-10-20T22:13:42.581807] [DEBUG] [LOCAL ] RMM: Loaded 10826 existing records from primary storage. [2025-10-20T22:13:42.581920] [DEBUG] [LOCAL ] RMAX: Saving derivative data. [2025-10-20T22:13:42.583188] [INFO ] [MAIN ] RMM: Loaded 10826 primary data records. [2025-10-20T22:13:42.650566] [DEBUG] [LOCAL ] RMAX: Derivative data saved (1794 records). [2025-10-20T22:13:42.658321] [INFO ] [MAIN ] RMAX: Successfully saved derivative data. [2025-10-20T22:13:42.661163] [INFO ] [MAIN ] Derivative Progress: 1445/1877 symbols completed. RMAX took 1.0s. Estimated remaining: 2m 42s. Rate: 159.8 symbols/min [2025-10-20T22:13:42.674504] [INFO ] [MAIN ] RMMZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:42.674703] [DEBUG] [LOCAL ] RMI: Derivative data saved (1793 records). [2025-10-20T22:13:42.689546] [INFO ] [MAIN ] RMI: Successfully saved derivative data. [2025-10-20T22:13:42.718222] [INFO ] [MAIN ] Derivative Progress: 1446/1877 symbols completed. RMI took 0.9s. Estimated remaining: 2m 42s. Rate: 159.9 symbols/min [2025-10-20T22:13:42.732089] [INFO ] [MAIN ] RMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:42.764352] [DEBUG] [LOCAL ] RMMZ: Loaded 4802 existing records from primary storage. [2025-10-20T22:13:42.816717] [INFO ] [MAIN ] RMMZ: Loaded 4802 primary data records. [2025-10-20T22:13:43.050535] [DEBUG] [LOCAL ] RMT: Loaded 17390 existing records from primary storage. [2025-10-20T22:13:43.089957] [INFO ] [MAIN ] RMT: Loaded 17390 primary data records. [2025-10-20T22:13:43.610383] [DEBUG] [MAIN ] RMM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:43.612684] [INFO ] [MAIN ] RMM: Generated 1794 derivative records from 10826 primary records. [2025-10-20T22:13:43.613113] [DEBUG] [LOCAL ] RMM: Saving derivative data. [2025-10-20T22:13:43.630140] [DEBUG] [LOCAL ] RMM: Derivative data saved (1794 records). [2025-10-20T22:13:43.630667] [INFO ] [MAIN ] RMM: Successfully saved derivative data. [2025-10-20T22:13:43.632565] [INFO ] [MAIN ] Derivative Progress: 1447/1877 symbols completed. RMM took 1.1s. Estimated remaining: 2m 41s. Rate: 159.8 symbols/min [2025-10-20T22:13:43.633206] [INFO ] [MAIN ] RNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:43.652458] [DEBUG] [MAIN ] RMMZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:43.653236] [INFO ] [MAIN ] RMMZ: Generated 1794 derivative records from 4802 primary records. [2025-10-20T22:13:43.653454] [DEBUG] [LOCAL ] RMMZ: Saving derivative data. [2025-10-20T22:13:43.670848] [DEBUG] [LOCAL ] RMMZ: Derivative data saved (1794 records). [2025-10-20T22:13:43.671750] [INFO ] [MAIN ] RMMZ: Successfully saved derivative data. [2025-10-20T22:13:43.674301] [INFO ] [MAIN ] Derivative Progress: 1448/1877 symbols completed. RMMZ took 1.0s. Estimated remaining: 2m 41s. Rate: 159.9 symbols/min [2025-10-20T22:13:43.674515] [INFO ] [MAIN ] RNGR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:43.687958] [DEBUG] [MAIN ] RMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:43.688869] [INFO ] [MAIN ] RMT: Generated 1794 derivative records from 17390 primary records. [2025-10-20T22:13:43.688929] [DEBUG] [LOCAL ] RMT: Saving derivative data. [2025-10-20T22:13:43.714006] [DEBUG] [LOCAL ] RMT: Derivative data saved (1794 records). [2025-10-20T22:13:43.714101] [INFO ] [MAIN ] RMT: Successfully saved derivative data. [2025-10-20T22:13:43.716297] [INFO ] [MAIN ] Derivative Progress: 1449/1877 symbols completed. RMT took 1.0s. Estimated remaining: 2m 41s. Rate: 160.0 symbols/min [2025-10-20T22:13:43.716663] [INFO ] [MAIN ] RNP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:43.751613] [DEBUG] [LOCAL ] RNGR: Loaded 20833 existing records from primary storage. [2025-10-20T22:13:43.752716] [DEBUG] [LOCAL ] RNG: Loaded 48033 existing records from primary storage. [2025-10-20T22:13:43.758999] [INFO ] [MAIN ] RNG: Loaded 48033 primary data records. [2025-10-20T22:13:43.758907] [INFO ] [MAIN ] RNGR: Loaded 20833 primary data records. [2025-10-20T22:13:43.780324] [DEBUG] [LOCAL ] RNP: Loaded 16118 existing records from primary storage. [2025-10-20T22:13:43.855029] [INFO ] [MAIN ] RNP: Loaded 16118 primary data records. [2025-10-20T22:13:44.700767] [DEBUG] [MAIN ] RNGR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:44.703485] [INFO ] [MAIN ] RNGR: Generated 1794 derivative records from 20833 primary records. [2025-10-20T22:13:44.703868] [DEBUG] [LOCAL ] RNGR: Saving derivative data. [2025-10-20T22:13:44.732118] [DEBUG] [MAIN ] RNP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:44.736621] [INFO ] [MAIN ] RNP: Generated 1794 derivative records from 16118 primary records. [2025-10-20T22:13:44.738017] [DEBUG] [LOCAL ] RNGR: Derivative data saved (1794 records). [2025-10-20T22:13:44.750183] [DEBUG] [LOCAL ] RNP: Saving derivative data. [2025-10-20T22:13:44.751706] [DEBUG] [MAIN ] RNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:44.751820] [INFO ] [MAIN ] RNGR: Successfully saved derivative data. [2025-10-20T22:13:44.754515] [INFO ] [MAIN ] Derivative Progress: 1450/1877 symbols completed. RNGR took 1.1s. Estimated remaining: 2m 40s. Rate: 159.8 symbols/min [2025-10-20T22:13:44.752487] [INFO ] [MAIN ] RNG: Generated 1794 derivative records from 48033 primary records. [2025-10-20T22:13:44.769768] [DEBUG] [LOCAL ] RNP: Derivative data saved (1794 records). [2025-10-20T22:13:44.761798] [DEBUG] [LOCAL ] RNG: Saving derivative data. [2025-10-20T22:13:44.765685] [INFO ] [MAIN ] RPT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:44.770986] [INFO ] [MAIN ] RNP: Successfully saved derivative data. [2025-10-20T22:13:44.773461] [INFO ] [MAIN ] Derivative Progress: 1451/1877 symbols completed. RNP took 1.1s. Estimated remaining: 2m 40s. Rate: 159.9 symbols/min [2025-10-20T22:13:44.773994] [INFO ] [MAIN ] RQI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:44.790849] [DEBUG] [LOCAL ] RNG: Derivative data saved (1794 records). [2025-10-20T22:13:44.791209] [INFO ] [MAIN ] RNG: Successfully saved derivative data. [2025-10-20T22:13:44.793258] [INFO ] [MAIN ] Derivative Progress: 1452/1877 symbols completed. RNG took 1.2s. Estimated remaining: 2m 39s. Rate: 160.0 symbols/min [2025-10-20T22:13:44.798237] [INFO ] [MAIN ] RSF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:44.809085] [DEBUG] [LOCAL ] RPT: Loaded 19109 existing records from primary storage. [2025-10-20T22:13:44.809259] [INFO ] [MAIN ] RPT: Loaded 19109 primary data records. [2025-10-20T22:13:44.843291] [DEBUG] [LOCAL ] RSF: Loaded 4311 existing records from primary storage. [2025-10-20T22:13:44.876577] [INFO ] [MAIN ] RSF: Loaded 4311 primary data records. [2025-10-20T22:13:45.053258] [DEBUG] [LOCAL ] RQI: Loaded 29007 existing records from primary storage. [2025-10-20T22:13:45.075301] [INFO ] [MAIN ] RQI: Loaded 29007 primary data records. [2025-10-20T22:13:45.809571] [DEBUG] [MAIN ] RSF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:45.810687] [INFO ] [MAIN ] RSF: Generated 1794 derivative records from 4311 primary records. [2025-10-20T22:13:45.811082] [DEBUG] [MAIN ] RPT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:45.811766] [DEBUG] [LOCAL ] RSF: Saving derivative data. [2025-10-20T22:13:45.811713] [INFO ] [MAIN ] RPT: Generated 1794 derivative records from 19109 primary records. [2025-10-20T22:13:45.819844] [DEBUG] [LOCAL ] RPT: Saving derivative data. [2025-10-20T22:13:45.830876] [DEBUG] [LOCAL ] RSF: Derivative data saved (1794 records). [2025-10-20T22:13:45.831026] [INFO ] [MAIN ] RSF: Successfully saved derivative data. [2025-10-20T22:13:45.833363] [INFO ] [MAIN ] Derivative Progress: 1453/1877 symbols completed. RSF took 1.0s. Estimated remaining: 2m 39s. Rate: 159.8 symbols/min [2025-10-20T22:13:45.862843] [DEBUG] [LOCAL ] RPT: Derivative data saved (1794 records). [2025-10-20T22:13:45.863060] [DEBUG] [MAIN ] RQI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:45.863951] [INFO ] [MAIN ] RPT: Successfully saved derivative data. [2025-10-20T22:13:45.863861] [INFO ] [MAIN ] RQI: Generated 1794 derivative records from 29007 primary records. [2025-10-20T22:13:45.866143] [DEBUG] [LOCAL ] RQI: Saving derivative data. [2025-10-20T22:13:45.866038] [INFO ] [MAIN ] Derivative Progress: 1454/1877 symbols completed. RPT took 1.1s. Estimated remaining: 2m 39s. Rate: 159.9 symbols/min [2025-10-20T22:13:45.882204] [DEBUG] [LOCAL ] RQI: Derivative data saved (1794 records). [2025-10-20T22:13:45.882266] [INFO ] [MAIN ] RQI: Successfully saved derivative data. [2025-10-20T22:13:45.884428] [INFO ] [MAIN ] Derivative Progress: 1455/1877 symbols completed. RQI took 1.1s. Estimated remaining: 2m 38s. Rate: 160.0 symbols/min [2025-10-20T22:13:45.884806] [INFO ] [MAIN ] RSKD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:45.885002] [INFO ] [MAIN ] RTO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:45.884909] [INFO ] [MAIN ] RVLV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:45.952710] [DEBUG] [LOCAL ] RSKD: Loaded 33373 existing records from primary storage. [2025-10-20T22:13:45.953957] [INFO ] [MAIN ] RSKD: Loaded 33373 primary data records. [2025-10-20T22:13:45.953796] [DEBUG] [LOCAL ] RTO: Loaded 39868 existing records from primary storage. [2025-10-20T22:13:45.985454] [INFO ] [MAIN ] RTO: Loaded 39868 primary data records. [2025-10-20T22:13:46.111544] [DEBUG] [LOCAL ] RVLV: Loaded 47502 existing records from primary storage. [2025-10-20T22:13:46.165991] [INFO ] [MAIN ] RVLV: Loaded 47502 primary data records. [2025-10-20T22:13:46.939761] [DEBUG] [MAIN ] RTO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:46.941338] [INFO ] [MAIN ] RTO: Generated 1794 derivative records from 39868 primary records. [2025-10-20T22:13:46.941850] [DEBUG] [LOCAL ] RTO: Saving derivative data. [2025-10-20T22:13:46.941749] [DEBUG] [MAIN ] RSKD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:46.944863] [INFO ] [MAIN ] RSKD: Generated 1794 derivative records from 33373 primary records. [2025-10-20T22:13:46.944984] [DEBUG] [LOCAL ] RSKD: Saving derivative data. [2025-10-20T22:13:46.985206] [DEBUG] [LOCAL ] RTO: Derivative data saved (1794 records). [2025-10-20T22:13:46.990908] [DEBUG] [MAIN ] RVLV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:46.985423] [INFO ] [MAIN ] RTO: Successfully saved derivative data. [2025-10-20T22:13:46.995490] [INFO ] [MAIN ] Derivative Progress: 1456/1877 symbols completed. RTO took 1.1s. Estimated remaining: 2m 38s. Rate: 159.8 symbols/min [2025-10-20T22:13:46.993010] [DEBUG] [LOCAL ] RSKD: Derivative data saved (1794 records). [2025-10-20T22:13:46.996315] [INFO ] [MAIN ] RVLV: Generated 1794 derivative records from 47502 primary records. [2025-10-20T22:13:46.996594] [INFO ] [MAIN ] RVT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:46.996867] [INFO ] [MAIN ] RSKD: Successfully saved derivative data. [2025-10-20T22:13:46.997045] [DEBUG] [LOCAL ] RVLV: Saving derivative data. [2025-10-20T22:13:47.002996] [INFO ] [MAIN ] Derivative Progress: 1457/1877 symbols completed. RSKD took 1.1s. Estimated remaining: 2m 38s. Rate: 159.9 symbols/min [2025-10-20T22:13:47.004762] [INFO ] [MAIN ] RY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:47.021850] [DEBUG] [LOCAL ] RVLV: Derivative data saved (1794 records). [2025-10-20T22:13:47.022061] [INFO ] [MAIN ] RVLV: Successfully saved derivative data. [2025-10-20T22:13:47.024368] [INFO ] [MAIN ] Derivative Progress: 1458/1877 symbols completed. RVLV took 1.1s. Estimated remaining: 2m 37s. Rate: 160.0 symbols/min [2025-10-20T22:13:47.024611] [INFO ] [MAIN ] RYAM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:47.048205] [DEBUG] [LOCAL ] RVT: Loaded 26373 existing records from primary storage. [2025-10-20T22:13:47.048346] [INFO ] [MAIN ] RVT: Loaded 26373 primary data records. [2025-10-20T22:13:47.277164] [DEBUG] [LOCAL ] RYAM: Loaded 35098 existing records from primary storage. [2025-10-20T22:13:47.384608] [INFO ] [MAIN ] RYAM: Loaded 35098 primary data records. [2025-10-20T22:13:47.379237] [DEBUG] [LOCAL ] RY: Loaded 47633 existing records from primary storage. [2025-10-20T22:13:47.414011] [INFO ] [MAIN ] RY: Loaded 47633 primary data records. [2025-10-20T22:13:48.069937] [DEBUG] [MAIN ] RVT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:48.071815] [INFO ] [MAIN ] RVT: Generated 1794 derivative records from 26373 primary records. [2025-10-20T22:13:48.073431] [DEBUG] [LOCAL ] RVT: Saving derivative data. [2025-10-20T22:13:48.091040] [DEBUG] [LOCAL ] RVT: Derivative data saved (1794 records). [2025-10-20T22:13:48.091153] [INFO ] [MAIN ] RVT: Successfully saved derivative data. [2025-10-20T22:13:48.093326] [INFO ] [MAIN ] Derivative Progress: 1459/1877 symbols completed. RVT took 1.1s. Estimated remaining: 2m 37s. Rate: 159.8 symbols/min [2025-10-20T22:13:48.095121] [INFO ] [MAIN ] RYI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:48.119602] [DEBUG] [MAIN ] RY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:48.126513] [INFO ] [MAIN ] RY: Generated 1794 derivative records from 47633 primary records. [2025-10-20T22:13:48.126909] [DEBUG] [LOCAL ] RY: Saving derivative data. [2025-10-20T22:13:48.155157] [DEBUG] [LOCAL ] RY: Derivative data saved (1794 records). [2025-10-20T22:13:48.156238] [INFO ] [MAIN ] RY: Successfully saved derivative data. [2025-10-20T22:13:48.156019] [DEBUG] [LOCAL ] RYI: Loaded 21736 existing records from primary storage. [2025-10-20T22:13:48.159787] [INFO ] [MAIN ] Derivative Progress: 1460/1877 symbols completed. RY took 1.2s. Estimated remaining: 2m 37s. Rate: 159.9 symbols/min [2025-10-20T22:13:48.156492] [INFO ] [MAIN ] RYI: Loaded 21736 primary data records. [2025-10-20T22:13:48.172125] [INFO ] [MAIN ] RZB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:48.245926] [DEBUG] [MAIN ] RYAM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:48.278353] [INFO ] [MAIN ] RYAM: Generated 1794 derivative records from 35098 primary records. [2025-10-20T22:13:48.287874] [DEBUG] [LOCAL ] RYAM: Saving derivative data. [2025-10-20T22:13:48.329316] [DEBUG] [LOCAL ] RZB: Loaded 5937 existing records from primary storage. [2025-10-20T22:13:48.372822] [INFO ] [MAIN ] RZB: Loaded 5937 primary data records. [2025-10-20T22:13:48.609413] [DEBUG] [LOCAL ] RYAM: Derivative data saved (1794 records). [2025-10-20T22:13:48.615928] [INFO ] [MAIN ] RYAM: Successfully saved derivative data. [2025-10-20T22:13:48.629623] [INFO ] [MAIN ] Derivative Progress: 1461/1877 symbols completed. RYAM took 1.6s. Estimated remaining: 2m 36s. Rate: 159.8 symbols/min [2025-10-20T22:13:48.662478] [INFO ] [MAIN ] RZC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:48.819916] [DEBUG] [LOCAL ] RZC: Loaded 6586 existing records from primary storage. [2025-10-20T22:13:48.821059] [INFO ] [MAIN ] RZC: Loaded 6586 primary data records. [2025-10-20T22:13:49.030999] [DEBUG] [MAIN ] RYI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:49.141672] [INFO ] [MAIN ] RYI: Generated 1794 derivative records from 21736 primary records. [2025-10-20T22:13:49.151203] [DEBUG] [LOCAL ] RYI: Saving derivative data. [2025-10-20T22:13:49.175237] [DEBUG] [LOCAL ] RYI: Derivative data saved (1794 records). [2025-10-20T22:13:49.188295] [INFO ] [MAIN ] RYI: Successfully saved derivative data. [2025-10-20T22:13:49.191321] [INFO ] [MAIN ] Derivative Progress: 1462/1877 symbols completed. RYI took 1.1s. Estimated remaining: 2m 36s. Rate: 159.8 symbols/min [2025-10-20T22:13:49.192104] [INFO ] [MAIN ] S: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:49.193938] [DEBUG] [MAIN ] RZB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:49.197073] [INFO ] [MAIN ] RZB: Generated 1794 derivative records from 5937 primary records. [2025-10-20T22:13:49.197263] [DEBUG] [LOCAL ] RZB: Saving derivative data. [2025-10-20T22:13:49.221171] [DEBUG] [LOCAL ] RZB: Derivative data saved (1794 records). [2025-10-20T22:13:49.221795] [INFO ] [MAIN ] RZB: Successfully saved derivative data. [2025-10-20T22:13:49.224890] [INFO ] [MAIN ] Derivative Progress: 1463/1877 symbols completed. RZB took 1.1s. Estimated remaining: 2m 35s. Rate: 159.9 symbols/min [2025-10-20T22:13:49.225675] [INFO ] [MAIN ] SA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:49.244562] [DEBUG] [MAIN ] RZC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:49.245537] [INFO ] [MAIN ] RZC: Generated 1794 derivative records from 6586 primary records. [2025-10-20T22:13:49.245681] [DEBUG] [LOCAL ] RZC: Saving derivative data. [2025-10-20T22:13:49.278344] [DEBUG] [LOCAL ] RZC: Derivative data saved (1794 records). [2025-10-20T22:13:49.278549] [INFO ] [MAIN ] RZC: Successfully saved derivative data. [2025-10-20T22:13:49.281330] [INFO ] [MAIN ] Derivative Progress: 1464/1877 symbols completed. RZC took 0.6s. Estimated remaining: 2m 35s. Rate: 160.0 symbols/min [2025-10-20T22:13:49.281736] [INFO ] [MAIN ] SABA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:49.317469] [DEBUG] [LOCAL ] S: Loaded 53538 existing records from primary storage. [2025-10-20T22:13:49.322532] [DEBUG] [LOCAL ] SABA: Loaded 9248 existing records from primary storage. [2025-10-20T22:13:49.322810] [INFO ] [MAIN ] S: Loaded 53538 primary data records. [2025-10-20T22:13:49.322754] [INFO ] [MAIN ] SABA: Loaded 9248 primary data records. [2025-10-20T22:13:49.409131] [DEBUG] [LOCAL ] SA: Loaded 44579 existing records from primary storage. [2025-10-20T22:13:49.463757] [INFO ] [MAIN ] SA: Loaded 44579 primary data records. [2025-10-20T22:13:50.346665] [DEBUG] [MAIN ] S: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:50.347569] [INFO ] [MAIN ] S: Generated 1794 derivative records from 53538 primary records. [2025-10-20T22:13:50.347909] [DEBUG] [LOCAL ] S: Saving derivative data. [2025-10-20T22:13:50.370415] [DEBUG] [LOCAL ] S: Derivative data saved (1794 records). [2025-10-20T22:13:50.370676] [INFO ] [MAIN ] S: Successfully saved derivative data. [2025-10-20T22:13:50.372590] [INFO ] [MAIN ] Derivative Progress: 1465/1877 symbols completed. S took 1.2s. Estimated remaining: 2m 35s. Rate: 159.8 symbols/min [2025-10-20T22:13:50.385973] [INFO ] [MAIN ] SAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:50.400755] [DEBUG] [MAIN ] SABA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:50.402134] [INFO ] [MAIN ] SABA: Generated 1794 derivative records from 9248 primary records. [2025-10-20T22:13:50.401127] [DEBUG] [MAIN ] SA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:50.402244] [DEBUG] [LOCAL ] SABA: Saving derivative data. [2025-10-20T22:13:50.403302] [INFO ] [MAIN ] SA: Generated 1794 derivative records from 44579 primary records. [2025-10-20T22:13:50.404387] [DEBUG] [LOCAL ] SA: Saving derivative data. [2025-10-20T22:13:50.428692] [DEBUG] [LOCAL ] SABA: Derivative data saved (1794 records). [2025-10-20T22:13:50.430684] [INFO ] [MAIN ] SABA: Successfully saved derivative data. [2025-10-20T22:13:50.435774] [INFO ] [MAIN ] Derivative Progress: 1466/1877 symbols completed. SABA took 1.2s. Estimated remaining: 2m 34s. Rate: 159.9 symbols/min [2025-10-20T22:13:50.440208] [DEBUG] [LOCAL ] SA: Derivative data saved (1794 records). [2025-10-20T22:13:50.440915] [INFO ] [MAIN ] SA: Successfully saved derivative data. [2025-10-20T22:13:50.443233] [INFO ] [MAIN ] Derivative Progress: 1467/1877 symbols completed. SA took 1.2s. Estimated remaining: 2m 34s. Rate: 160.0 symbols/min [2025-10-20T22:13:50.440773] [INFO ] [MAIN ] SAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:50.443539] [INFO ] [MAIN ] SAR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:50.476966] [DEBUG] [LOCAL ] SAR: Loaded 17251 existing records from primary storage. [2025-10-20T22:13:50.477229] [INFO ] [MAIN ] SAR: Loaded 17251 primary data records. [2025-10-20T22:13:50.564312] [DEBUG] [LOCAL ] SAN: Loaded 52969 existing records from primary storage. [2025-10-20T22:13:50.570526] [INFO ] [MAIN ] SAN: Loaded 52969 primary data records. [2025-10-20T22:13:50.943583] [DEBUG] [LOCAL ] SAP: Loaded 49871 existing records from primary storage. [2025-10-20T22:13:50.948698] [INFO ] [MAIN ] SAP: Loaded 49871 primary data records. [2025-10-20T22:13:51.508372] [DEBUG] [MAIN ] SAR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:51.509909] [INFO ] [MAIN ] SAR: Generated 1794 derivative records from 17251 primary records. [2025-10-20T22:13:51.511290] [DEBUG] [LOCAL ] SAR: Saving derivative data. [2025-10-20T22:13:51.543233] [DEBUG] [MAIN ] SAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:51.544741] [INFO ] [MAIN ] SAN: Generated 1794 derivative records from 52969 primary records. [2025-10-20T22:13:51.544986] [DEBUG] [LOCAL ] SAN: Saving derivative data. [2025-10-20T22:13:51.543879] [DEBUG] [LOCAL ] SAR: Derivative data saved (1794 records). [2025-10-20T22:13:51.545291] [INFO ] [MAIN ] SAR: Successfully saved derivative data. [2025-10-20T22:13:51.547970] [INFO ] [MAIN ] Derivative Progress: 1468/1877 symbols completed. SAR took 1.1s. Estimated remaining: 2m 34s. Rate: 159.8 symbols/min [2025-10-20T22:13:51.569020] [DEBUG] [LOCAL ] SAN: Derivative data saved (1794 records). [2025-10-20T22:13:51.569146] [INFO ] [MAIN ] SAN: Successfully saved derivative data. [2025-10-20T22:13:51.571423] [INFO ] [MAIN ] Derivative Progress: 1469/1877 symbols completed. SAN took 1.2s. Estimated remaining: 2m 33s. Rate: 159.9 symbols/min [2025-10-20T22:13:51.584259] [DEBUG] [MAIN ] SAP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:51.584999] [INFO ] [MAIN ] SAP: Generated 1794 derivative records from 49871 primary records. [2025-10-20T22:13:51.585061] [DEBUG] [LOCAL ] SAP: Saving derivative data. [2025-10-20T22:13:51.600616] [DEBUG] [LOCAL ] SAP: Derivative data saved (1794 records). [2025-10-20T22:13:51.600718] [INFO ] [MAIN ] SAP: Successfully saved derivative data. [2025-10-20T22:13:51.602905] [INFO ] [MAIN ] Derivative Progress: 1470/1877 symbols completed. SAP took 1.2s. Estimated remaining: 2m 33s. Rate: 160.0 symbols/min [2025-10-20T22:13:51.603591] [INFO ] [MAIN ] SB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:51.603498] [INFO ] [MAIN ] SARO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:51.603740] [INFO ] [MAIN ] SBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:51.616093] [DEBUG] [LOCAL ] SBI: Loaded 7716 existing records from primary storage. [2025-10-20T22:13:51.616374] [INFO ] [MAIN ] SBI: Loaded 7716 primary data records. [2025-10-20T22:13:52.001039] [DEBUG] [LOCAL ] SB: Loaded 32611 existing records from primary storage. [2025-10-20T22:13:52.001891] [DEBUG] [LOCAL ] SARO: Loaded 48307 existing records from primary storage. [2025-10-20T22:13:52.002685] [INFO ] [MAIN ] SB: Loaded 32611 primary data records. [2025-10-20T22:13:52.002911] [INFO ] [MAIN ] SARO: Loaded 48307 primary data records. [2025-10-20T22:13:52.647923] [DEBUG] [MAIN ] SBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:52.649109] [INFO ] [MAIN ] SBI: Generated 1794 derivative records from 7716 primary records. [2025-10-20T22:13:52.649769] [DEBUG] [LOCAL ] SBI: Saving derivative data. [2025-10-20T22:13:52.667236] [DEBUG] [LOCAL ] SBI: Derivative data saved (1794 records). [2025-10-20T22:13:52.668346] [INFO ] [MAIN ] SBI: Successfully saved derivative data. [2025-10-20T22:13:52.671126] [INFO ] [MAIN ] Derivative Progress: 1471/1877 symbols completed. SBI took 1.1s. Estimated remaining: 2m 32s. Rate: 159.8 symbols/min [2025-10-20T22:13:52.671401] [INFO ] [MAIN ] SBR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:52.683972] [DEBUG] [LOCAL ] SBR: Loaded 7870 existing records from primary storage. [2025-10-20T22:13:52.684274] [INFO ] [MAIN ] SBR: Loaded 7870 primary data records. [2025-10-20T22:13:52.803052] [DEBUG] [MAIN ] SB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:52.814994] [INFO ] [MAIN ] SB: Generated 1794 derivative records from 32611 primary records. [2025-10-20T22:13:52.820008] [DEBUG] [LOCAL ] SB: Saving derivative data. [2025-10-20T22:13:52.820519] [DEBUG] [MAIN ] SARO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:52.827019] [INFO ] [MAIN ] SARO: Generated 1794 derivative records from 48307 primary records. [2025-10-20T22:13:52.880122] [DEBUG] [LOCAL ] SARO: Saving derivative data. [2025-10-20T22:13:53.048762] [DEBUG] [LOCAL ] SB: Derivative data saved (1794 records). [2025-10-20T22:13:53.050350] [INFO ] [MAIN ] SB: Successfully saved derivative data. [2025-10-20T22:13:53.053091] [INFO ] [MAIN ] Derivative Progress: 1472/1877 symbols completed. SB took 1.4s. Estimated remaining: 2m 32s. Rate: 159.8 symbols/min [2025-10-20T22:13:53.053420] [INFO ] [MAIN ] SBS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:53.050150] [DEBUG] [LOCAL ] SARO: Derivative data saved (1794 records). [2025-10-20T22:13:53.054566] [INFO ] [MAIN ] SARO: Successfully saved derivative data. [2025-10-20T22:13:53.056850] [INFO ] [MAIN ] Derivative Progress: 1473/1877 symbols completed. SARO took 1.5s. Estimated remaining: 2m 32s. Rate: 159.9 symbols/min [2025-10-20T22:13:53.057469] [INFO ] [MAIN ] SCCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:53.084025] [DEBUG] [MAIN ] SBR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:53.085061] [INFO ] [MAIN ] SBR: Generated 1794 derivative records from 7870 primary records. [2025-10-20T22:13:53.096193] [DEBUG] [LOCAL ] SBR: Saving derivative data. [2025-10-20T22:13:53.112308] [DEBUG] [LOCAL ] SBR: Derivative data saved (1794 records). [2025-10-20T22:13:53.112459] [INFO ] [MAIN ] SBR: Successfully saved derivative data. [2025-10-20T22:13:53.114740] [INFO ] [MAIN ] Derivative Progress: 1474/1877 symbols completed. SBR took 0.4s. Estimated remaining: 2m 31s. Rate: 160.0 symbols/min [2025-10-20T22:13:53.114916] [INFO ] [MAIN ] SCD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:53.141472] [DEBUG] [LOCAL ] SBS: Loaded 49323 existing records from primary storage. [2025-10-20T22:13:53.141616] [INFO ] [MAIN ] SBS: Loaded 49323 primary data records. [2025-10-20T22:13:53.269636] [DEBUG] [LOCAL ] SCCO: Loaded 49602 existing records from primary storage. [2025-10-20T22:13:53.270271] [DEBUG] [LOCAL ] SCD: Loaded 13520 existing records from primary storage. [2025-10-20T22:13:53.270502] [INFO ] [MAIN ] SCCO: Loaded 49602 primary data records. [2025-10-20T22:13:53.270730] [INFO ] [MAIN ] SCD: Loaded 13520 primary data records. [2025-10-20T22:13:54.160960] [DEBUG] [MAIN ] SBS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:54.162574] [INFO ] [MAIN ] SBS: Generated 1794 derivative records from 49323 primary records. [2025-10-20T22:13:54.162799] [DEBUG] [LOCAL ] SBS: Saving derivative data. [2025-10-20T22:13:54.194615] [DEBUG] [LOCAL ] SBS: Derivative data saved (1794 records). [2025-10-20T22:13:54.194861] [INFO ] [MAIN ] SBS: Successfully saved derivative data. [2025-10-20T22:13:54.194545] [DEBUG] [MAIN ] SCCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:54.196876] [INFO ] [MAIN ] Derivative Progress: 1475/1877 symbols completed. SBS took 1.1s. Estimated remaining: 2m 31s. Rate: 159.7 symbols/min [2025-10-20T22:13:54.205303] [INFO ] [MAIN ] SCCO: Generated 1794 derivative records from 49602 primary records. [2025-10-20T22:13:54.205637] [DEBUG] [LOCAL ] SCCO: Saving derivative data. [2025-10-20T22:13:54.210971] [DEBUG] [MAIN ] SCD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:54.205545] [INFO ] [MAIN ] SCM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:54.212429] [INFO ] [MAIN ] SCD: Generated 1794 derivative records from 13520 primary records. [2025-10-20T22:13:54.212833] [DEBUG] [LOCAL ] SCD: Saving derivative data. [2025-10-20T22:13:54.229718] [DEBUG] [LOCAL ] SCCO: Derivative data saved (1794 records). [2025-10-20T22:13:54.230248] [INFO ] [MAIN ] SCCO: Successfully saved derivative data. [2025-10-20T22:13:54.244857] [DEBUG] [LOCAL ] SCD: Derivative data saved (1794 records). [2025-10-20T22:13:54.245135] [INFO ] [MAIN ] SCD: Successfully saved derivative data. [2025-10-20T22:13:54.247320] [INFO ] [MAIN ] Derivative Progress: 1476/1877 symbols completed. SCD took 1.1s. Estimated remaining: 2m 31s. Rate: 159.8 symbols/min [2025-10-20T22:13:54.249932] [INFO ] [MAIN ] SCS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:54.250005] [INFO ] [MAIN ] Derivative Progress: 1477/1877 symbols completed. SCCO took 1.2s. Estimated remaining: 2m 30s. Rate: 159.9 symbols/min [2025-10-20T22:13:54.250603] [INFO ] [MAIN ] SD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:54.268084] [DEBUG] [LOCAL ] SCM: Loaded 20080 existing records from primary storage. [2025-10-20T22:13:54.268258] [INFO ] [MAIN ] SCM: Loaded 20080 primary data records. [2025-10-20T22:13:54.442746] [DEBUG] [LOCAL ] SD: Loaded 28534 existing records from primary storage. [2025-10-20T22:13:54.456024] [INFO ] [MAIN ] SD: Loaded 28534 primary data records. [2025-10-20T22:13:54.685842] [DEBUG] [LOCAL ] SCS: Loaded 43377 existing records from primary storage. [2025-10-20T22:13:54.705302] [INFO ] [MAIN ] SCS: Loaded 43377 primary data records. [2025-10-20T22:13:55.232175] [DEBUG] [MAIN ] SCM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:55.233956] [INFO ] [MAIN ] SCM: Generated 1794 derivative records from 20080 primary records. [2025-10-20T22:13:55.234458] [DEBUG] [LOCAL ] SCM: Saving derivative data. [2025-10-20T22:13:55.252150] [DEBUG] [LOCAL ] SCM: Derivative data saved (1794 records). [2025-10-20T22:13:55.252581] [INFO ] [MAIN ] SCM: Successfully saved derivative data. [2025-10-20T22:13:55.255840] [INFO ] [MAIN ] Derivative Progress: 1478/1877 symbols completed. SCM took 1.1s. Estimated remaining: 2m 30s. Rate: 159.8 symbols/min [2025-10-20T22:13:55.256978] [INFO ] [MAIN ] SDHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:55.272181] [DEBUG] [MAIN ] SD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:55.273121] [INFO ] [MAIN ] SD: Generated 1794 derivative records from 28534 primary records. [2025-10-20T22:13:55.273678] [DEBUG] [LOCAL ] SD: Saving derivative data. [2025-10-20T22:13:55.294342] [DEBUG] [LOCAL ] SD: Derivative data saved (1794 records). [2025-10-20T22:13:55.294556] [INFO ] [MAIN ] SD: Successfully saved derivative data. [2025-10-20T22:13:55.296461] [INFO ] [MAIN ] Derivative Progress: 1479/1877 symbols completed. SD took 1.0s. Estimated remaining: 2m 29s. Rate: 159.9 symbols/min [2025-10-20T22:13:55.296861] [INFO ] [MAIN ] SDHY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:55.314273] [DEBUG] [MAIN ] SCS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:55.314524] [DEBUG] [LOCAL ] SDHC: Loaded 10386 existing records from primary storage. [2025-10-20T22:13:55.315429] [INFO ] [MAIN ] SDHC: Loaded 10386 primary data records. [2025-10-20T22:13:55.315327] [INFO ] [MAIN ] SCS: Generated 1794 derivative records from 43377 primary records. [2025-10-20T22:13:55.315612] [DEBUG] [LOCAL ] SCS: Saving derivative data. [2025-10-20T22:13:55.375452] [DEBUG] [LOCAL ] SCS: Derivative data saved (1794 records). [2025-10-20T22:13:55.399538] [INFO ] [MAIN ] SCS: Successfully saved derivative data. [2025-10-20T22:13:55.401460] [INFO ] [MAIN ] Derivative Progress: 1480/1877 symbols completed. SCS took 1.2s. Estimated remaining: 2m 29s. Rate: 159.9 symbols/min [2025-10-20T22:13:55.413121] [INFO ] [MAIN ] SDRL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:55.445285] [DEBUG] [LOCAL ] SDHY: Loaded 11160 existing records from primary storage. [2025-10-20T22:13:55.493285] [INFO ] [MAIN ] SDHY: Loaded 11160 primary data records. [2025-10-20T22:13:55.864688] [DEBUG] [LOCAL ] SDRL: Loaded 41446 existing records from primary storage. [2025-10-20T22:13:55.970338] [INFO ] [MAIN ] SDRL: Loaded 41446 primary data records. [2025-10-20T22:13:56.300563] [DEBUG] [MAIN ] SDHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:56.302079] [INFO ] [MAIN ] SDHC: Generated 1794 derivative records from 10386 primary records. [2025-10-20T22:13:56.314088] [DEBUG] [LOCAL ] SDHC: Saving derivative data. [2025-10-20T22:13:56.314554] [DEBUG] [MAIN ] SDHY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:56.316119] [INFO ] [MAIN ] SDHY: Generated 1794 derivative records from 11160 primary records. [2025-10-20T22:13:56.317060] [DEBUG] [LOCAL ] SDHY: Saving derivative data. [2025-10-20T22:13:56.339086] [DEBUG] [LOCAL ] SDHC: Derivative data saved (1794 records). [2025-10-20T22:13:56.339295] [INFO ] [MAIN ] SDHC: Successfully saved derivative data. [2025-10-20T22:13:56.342146] [INFO ] [MAIN ] Derivative Progress: 1481/1877 symbols completed. SDHC took 1.1s. Estimated remaining: 2m 29s. Rate: 159.8 symbols/min [2025-10-20T22:13:56.347821] [INFO ] [MAIN ] SEG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:56.349441] [DEBUG] [LOCAL ] SDHY: Derivative data saved (1794 records). [2025-10-20T22:13:56.349605] [INFO ] [MAIN ] SDHY: Successfully saved derivative data. [2025-10-20T22:13:56.351407] [INFO ] [MAIN ] Derivative Progress: 1482/1877 symbols completed. SDHY took 1.1s. Estimated remaining: 2m 28s. Rate: 159.9 symbols/min [2025-10-20T22:13:56.351529] [INFO ] [MAIN ] SEMR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:56.373228] [DEBUG] [MAIN ] SDRL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:56.373999] [INFO ] [MAIN ] SDRL: Generated 1794 derivative records from 41446 primary records. [2025-10-20T22:13:56.374134] [DEBUG] [LOCAL ] SDRL: Saving derivative data. [2025-10-20T22:13:56.393022] [DEBUG] [LOCAL ] SEG: Loaded 12718 existing records from primary storage. [2025-10-20T22:13:56.393752] [INFO ] [MAIN ] SEG: Loaded 12718 primary data records. [2025-10-20T22:13:56.395048] [DEBUG] [LOCAL ] SDRL: Derivative data saved (1794 records). [2025-10-20T22:13:56.430841] [INFO ] [MAIN ] SDRL: Successfully saved derivative data. [2025-10-20T22:13:56.439489] [INFO ] [MAIN ] Derivative Progress: 1483/1877 symbols completed. SDRL took 1.0s. Estimated remaining: 2m 28s. Rate: 160.0 symbols/min [2025-10-20T22:13:56.597194] [DEBUG] [LOCAL ] SEMR: Loaded 42107 existing records from primary storage. [2025-10-20T22:13:56.597439] [INFO ] [MAIN ] SEMR: Loaded 42107 primary data records. [2025-10-20T22:13:57.033483] [DEBUG] [MAIN ] SEG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:57.034766] [INFO ] [MAIN ] SEG: Generated 1794 derivative records from 12718 primary records. [2025-10-20T22:13:57.034935] [DEBUG] [LOCAL ] SEG: Saving derivative data. [2025-10-20T22:13:57.052748] [DEBUG] [LOCAL ] SEG: Derivative data saved (1794 records). [2025-10-20T22:13:57.053201] [INFO ] [MAIN ] SEG: Successfully saved derivative data. [2025-10-20T22:13:57.055927] [INFO ] [MAIN ] Derivative Progress: 1484/1877 symbols completed. SEG took 0.7s. Estimated remaining: 2m 27s. Rate: 159.9 symbols/min [2025-10-20T22:13:57.071790] [DEBUG] [MAIN ] SEMR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:57.072536] [INFO ] [MAIN ] SEMR: Generated 1794 derivative records from 42107 primary records. [2025-10-20T22:13:57.072601] [DEBUG] [LOCAL ] SEMR: Saving derivative data. [2025-10-20T22:13:57.087871] [DEBUG] [LOCAL ] SEMR: Derivative data saved (1794 records). [2025-10-20T22:13:57.087927] [INFO ] [MAIN ] SEMR: Successfully saved derivative data. [2025-10-20T22:13:57.090033] [INFO ] [MAIN ] Derivative Progress: 1485/1877 symbols completed. SEMR took 0.7s. Estimated remaining: 2m 27s. Rate: 160.0 symbols/min [2025-10-20T22:13:57.090791] [INFO ] [MAIN ] SFB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:57.091483] [INFO ] [MAIN ] SFL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:57.090580] [INFO ] [MAIN ] SES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:57.100312] [DEBUG] [LOCAL ] SFB: Loaded 3778 existing records from primary storage. [2025-10-20T22:13:57.100707] [INFO ] [MAIN ] SFB: Loaded 3778 primary data records. [2025-10-20T22:13:57.473319] [DEBUG] [LOCAL ] SFL: Loaded 45331 existing records from primary storage. [2025-10-20T22:13:57.474295] [DEBUG] [LOCAL ] SES: Loaded 51598 existing records from primary storage. [2025-10-20T22:13:57.474737] [INFO ] [MAIN ] SFL: Loaded 45331 primary data records. [2025-10-20T22:13:57.475523] [INFO ] [MAIN ] SES: Loaded 51598 primary data records. [2025-10-20T22:13:58.153784] [DEBUG] [MAIN ] SFB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:58.155583] [INFO ] [MAIN ] SFB: Generated 1794 derivative records from 3778 primary records. [2025-10-20T22:13:58.156696] [DEBUG] [LOCAL ] SFB: Saving derivative data. [2025-10-20T22:13:58.174470] [DEBUG] [LOCAL ] SFB: Derivative data saved (1794 records). [2025-10-20T22:13:58.174635] [INFO ] [MAIN ] SFB: Successfully saved derivative data. [2025-10-20T22:13:58.177971] [INFO ] [MAIN ] Derivative Progress: 1486/1877 symbols completed. SFB took 1.1s. Estimated remaining: 2m 27s. Rate: 159.8 symbols/min [2025-10-20T22:13:58.178463] [INFO ] [MAIN ] SGHC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:58.196179] [DEBUG] [MAIN ] SFL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:58.200473] [INFO ] [MAIN ] SFL: Generated 1794 derivative records from 45331 primary records. [2025-10-20T22:13:58.200737] [DEBUG] [LOCAL ] SFL: Saving derivative data. [2025-10-20T22:13:58.230371] [DEBUG] [LOCAL ] SFL: Derivative data saved (1794 records). [2025-10-20T22:13:58.230598] [INFO ] [MAIN ] SFL: Successfully saved derivative data. [2025-10-20T22:13:58.232413] [INFO ] [MAIN ] Derivative Progress: 1487/1877 symbols completed. SFL took 1.1s. Estimated remaining: 2m 26s. Rate: 159.9 symbols/min [2025-10-20T22:13:58.226181] [DEBUG] [MAIN ] SES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:58.232634] [INFO ] [MAIN ] SGU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:58.233822] [INFO ] [MAIN ] SES: Generated 1794 derivative records from 51598 primary records. [2025-10-20T22:13:58.234136] [DEBUG] [LOCAL ] SES: Saving derivative data. [2025-10-20T22:13:58.252737] [DEBUG] [LOCAL ] SES: Derivative data saved (1794 records). [2025-10-20T22:13:58.266399] [INFO ] [MAIN ] SES: Successfully saved derivative data. [2025-10-20T22:13:58.270726] [INFO ] [MAIN ] Derivative Progress: 1488/1877 symbols completed. SES took 1.2s. Estimated remaining: 2m 26s. Rate: 160.0 symbols/min [2025-10-20T22:13:58.271068] [INFO ] [MAIN ] SHCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:58.276001] [DEBUG] [LOCAL ] SGU: Loaded 6541 existing records from primary storage. [2025-10-20T22:13:58.276198] [INFO ] [MAIN ] SGU: Loaded 6541 primary data records. [2025-10-20T22:13:58.404096] [DEBUG] [LOCAL ] SGHC: Loaded 47247 existing records from primary storage. [2025-10-20T22:13:58.418777] [INFO ] [MAIN ] SGHC: Loaded 47247 primary data records. [2025-10-20T22:13:58.458411] [DEBUG] [LOCAL ] SHCO: Loaded 25358 existing records from primary storage. [2025-10-20T22:13:58.484935] [INFO ] [MAIN ] SHCO: Loaded 25358 primary data records. [2025-10-20T22:13:59.277842] [DEBUG] [MAIN ] SGU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:59.278987] [INFO ] [MAIN ] SGU: Generated 1794 derivative records from 6541 primary records. [2025-10-20T22:13:59.280622] [DEBUG] [LOCAL ] SGU: Saving derivative data. [2025-10-20T22:13:59.300326] [DEBUG] [LOCAL ] SGU: Derivative data saved (1794 records). [2025-10-20T22:13:59.300575] [INFO ] [MAIN ] SGU: Successfully saved derivative data. [2025-10-20T22:13:59.302758] [INFO ] [MAIN ] Derivative Progress: 1489/1877 symbols completed. SGU took 1.1s. Estimated remaining: 2m 26s. Rate: 159.8 symbols/min [2025-10-20T22:13:59.303369] [INFO ] [MAIN ] SHEL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:59.317772] [DEBUG] [MAIN ] SGHC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:59.319299] [INFO ] [MAIN ] SGHC: Generated 1794 derivative records from 47247 primary records. [2025-10-20T22:13:59.319488] [DEBUG] [LOCAL ] SGHC: Saving derivative data. [2025-10-20T22:13:59.354475] [DEBUG] [MAIN ] SHCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:13:59.355947] [DEBUG] [LOCAL ] SGHC: Derivative data saved (1794 records). [2025-10-20T22:13:59.356309] [INFO ] [MAIN ] SGHC: Successfully saved derivative data. [2025-10-20T22:13:59.359426] [INFO ] [MAIN ] Derivative Progress: 1490/1877 symbols completed. SGHC took 1.2s. Estimated remaining: 2m 25s. Rate: 159.9 symbols/min [2025-10-20T22:13:59.360463] [INFO ] [MAIN ] SHCO: Generated 1794 derivative records from 25358 primary records. [2025-10-20T22:13:59.360728] [DEBUG] [LOCAL ] SHCO: Saving derivative data. [2025-10-20T22:13:59.361338] [INFO ] [MAIN ] SHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:59.379684] [DEBUG] [LOCAL ] SHCO: Derivative data saved (1794 records). [2025-10-20T22:13:59.379814] [INFO ] [MAIN ] SHCO: Successfully saved derivative data. [2025-10-20T22:13:59.382488] [INFO ] [MAIN ] Derivative Progress: 1491/1877 symbols completed. SHCO took 1.1s. Estimated remaining: 2m 25s. Rate: 160.0 symbols/min [2025-10-20T22:13:59.382784] [INFO ] [MAIN ] SHOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:13:59.411344] [DEBUG] [LOCAL ] SHOC: Loaded 3140 existing records from primary storage. [2025-10-20T22:13:59.411675] [INFO ] [MAIN ] SHOC: Loaded 3140 primary data records. [2025-10-20T22:13:59.510460] [DEBUG] [LOCAL ] SHG: Loaded 26655 existing records from primary storage. [2025-10-20T22:13:59.523064] [INFO ] [MAIN ] SHG: Loaded 26655 primary data records. [2025-10-20T22:13:59.529813] [DEBUG] [LOCAL ] SHEL: Loaded 53675 existing records from primary storage. [2025-10-20T22:13:59.546692] [INFO ] [MAIN ] SHEL: Loaded 53675 primary data records. [2025-10-20T22:14:00.441439] [DEBUG] [MAIN ] SHOC: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:00.442568] [INFO ] [MAIN ] SHOC: Generated 1793 derivative records from 3140 primary records. [2025-10-20T22:14:00.442939] [DEBUG] [LOCAL ] SHOC: Saving derivative data. [2025-10-20T22:14:00.458344] [DEBUG] [LOCAL ] SHOC: Derivative data saved (1793 records). [2025-10-20T22:14:00.458490] [INFO ] [MAIN ] SHOC: Successfully saved derivative data. [2025-10-20T22:14:00.460359] [INFO ] [MAIN ] Derivative Progress: 1492/1877 symbols completed. SHOC took 1.1s. Estimated remaining: 2m 25s. Rate: 159.8 symbols/min [2025-10-20T22:14:00.460629] [INFO ] [MAIN ] SID: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:00.478488] [DEBUG] [MAIN ] SHEL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:00.480938] [INFO ] [MAIN ] SHEL: Generated 1794 derivative records from 53675 primary records. [2025-10-20T22:14:00.481184] [DEBUG] [LOCAL ] SHEL: Saving derivative data. [2025-10-20T22:14:00.502105] [DEBUG] [MAIN ] SHG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:00.509922] [INFO ] [MAIN ] SHG: Generated 1794 derivative records from 26655 primary records. [2025-10-20T22:14:00.511172] [DEBUG] [LOCAL ] SHG: Saving derivative data. [2025-10-20T22:14:00.511084] [DEBUG] [LOCAL ] SHEL: Derivative data saved (1794 records). [2025-10-20T22:14:00.512052] [INFO ] [MAIN ] SHEL: Successfully saved derivative data. [2025-10-20T22:14:00.521259] [INFO ] [MAIN ] Derivative Progress: 1493/1877 symbols completed. SHEL took 1.2s. Estimated remaining: 2m 24s. Rate: 159.9 symbols/min [2025-10-20T22:14:00.525304] [INFO ] [MAIN ] SII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:00.529750] [DEBUG] [LOCAL ] SHG: Derivative data saved (1794 records). [2025-10-20T22:14:00.530341] [INFO ] [MAIN ] SHG: Successfully saved derivative data. [2025-10-20T22:14:00.532441] [INFO ] [MAIN ] Derivative Progress: 1494/1877 symbols completed. SHG took 1.2s. Estimated remaining: 2m 24s. Rate: 160.0 symbols/min [2025-10-20T22:14:00.532760] [INFO ] [MAIN ] SILA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:00.574818] [DEBUG] [LOCAL ] SID: Loaded 42841 existing records from primary storage. [2025-10-20T22:14:00.575047] [INFO ] [MAIN ] SID: Loaded 42841 primary data records. [2025-10-20T22:14:00.574015] [DEBUG] [LOCAL ] SII: Loaded 24165 existing records from primary storage. [2025-10-20T22:14:00.575222] [INFO ] [MAIN ] SII: Loaded 24165 primary data records. [2025-10-20T22:14:00.793231] [DEBUG] [LOCAL ] SILA: Loaded 30253 existing records from primary storage. [2025-10-20T22:14:00.801022] [INFO ] [MAIN ] SILA: Loaded 30253 primary data records. [2025-10-20T22:14:01.592527] [DEBUG] [MAIN ] SID: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:01.593889] [INFO ] [MAIN ] SID: Generated 1794 derivative records from 42841 primary records. [2025-10-20T22:14:01.594353] [DEBUG] [LOCAL ] SID: Saving derivative data. [2025-10-20T22:14:01.594273] [DEBUG] [MAIN ] SII: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:01.597950] [INFO ] [MAIN ] SII: Generated 1794 derivative records from 24165 primary records. [2025-10-20T22:14:01.609490] [DEBUG] [LOCAL ] SII: Saving derivative data. [2025-10-20T22:14:01.641844] [DEBUG] [LOCAL ] SII: Derivative data saved (1794 records). [2025-10-20T22:14:01.641985] [INFO ] [MAIN ] SII: Successfully saved derivative data. [2025-10-20T22:14:01.642090] [DEBUG] [LOCAL ] SID: Derivative data saved (1794 records). [2025-10-20T22:14:01.644626] [INFO ] [MAIN ] SID: Successfully saved derivative data. [2025-10-20T22:14:01.644934] [DEBUG] [MAIN ] SILA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:01.648031] [INFO ] [MAIN ] SILA: Generated 1794 derivative records from 30253 primary records. [2025-10-20T22:14:01.644314] [INFO ] [MAIN ] Derivative Progress: 1495/1877 symbols completed. SII took 1.1s. Estimated remaining: 2m 23s. Rate: 159.8 symbols/min [2025-10-20T22:14:01.648337] [INFO ] [MAIN ] SITE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:01.648422] [INFO ] [MAIN ] Derivative Progress: 1496/1877 symbols completed. SID took 1.2s. Estimated remaining: 2m 23s. Rate: 159.9 symbols/min [2025-10-20T22:14:01.648217] [DEBUG] [LOCAL ] SILA: Saving derivative data. [2025-10-20T22:14:01.648820] [INFO ] [MAIN ] SJT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:01.666391] [DEBUG] [LOCAL ] SILA: Derivative data saved (1794 records). [2025-10-20T22:14:01.666473] [INFO ] [MAIN ] SILA: Successfully saved derivative data. [2025-10-20T22:14:01.668514] [INFO ] [MAIN ] Derivative Progress: 1497/1877 symbols completed. SILA took 1.1s. Estimated remaining: 2m 23s. Rate: 160.0 symbols/min [2025-10-20T22:14:01.668724] [INFO ] [MAIN ] SKE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:01.692425] [DEBUG] [LOCAL ] SJT: Loaded 22785 existing records from primary storage. [2025-10-20T22:14:01.692699] [INFO ] [MAIN ] SJT: Loaded 22785 primary data records. [2025-10-20T22:14:01.825864] [DEBUG] [LOCAL ] SITE: Loaded 35597 existing records from primary storage. [2025-10-20T22:14:01.839014] [INFO ] [MAIN ] SITE: Loaded 35597 primary data records. [2025-10-20T22:14:02.138138] [DEBUG] [LOCAL ] SKE: Loaded 37233 existing records from primary storage. [2025-10-20T22:14:02.161349] [INFO ] [MAIN ] SKE: Loaded 37233 primary data records. [2025-10-20T22:14:02.679889] [DEBUG] [MAIN ] SJT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:02.685443] [DEBUG] [MAIN ] SITE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:02.686328] [INFO ] [MAIN ] SJT: Generated 1794 derivative records from 22785 primary records. [2025-10-20T22:14:02.686519] [DEBUG] [LOCAL ] SJT: Saving derivative data. [2025-10-20T22:14:02.687786] [INFO ] [MAIN ] SITE: Generated 1794 derivative records from 35597 primary records. [2025-10-20T22:14:02.687917] [DEBUG] [LOCAL ] SITE: Saving derivative data. [2025-10-20T22:14:02.704394] [DEBUG] [LOCAL ] SJT: Derivative data saved (1794 records). [2025-10-20T22:14:02.704550] [INFO ] [MAIN ] SJT: Successfully saved derivative data. [2025-10-20T22:14:02.713767] [INFO ] [MAIN ] Derivative Progress: 1498/1877 symbols completed. SJT took 1.1s. Estimated remaining: 2m 22s. Rate: 159.8 symbols/min [2025-10-20T22:14:02.723032] [DEBUG] [LOCAL ] SITE: Derivative data saved (1794 records). [2025-10-20T22:14:02.724089] [INFO ] [MAIN ] SITE: Successfully saved derivative data. [2025-10-20T22:14:02.726968] [INFO ] [MAIN ] Derivative Progress: 1499/1877 symbols completed. SITE took 1.1s. Estimated remaining: 2m 22s. Rate: 159.9 symbols/min [2025-10-20T22:14:02.741192] [DEBUG] [MAIN ] SKE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:02.742146] [INFO ] [MAIN ] SKE: Generated 1794 derivative records from 37233 primary records. [2025-10-20T22:14:02.742231] [DEBUG] [LOCAL ] SKE: Saving derivative data. [2025-10-20T22:14:02.764414] [DEBUG] [LOCAL ] SKE: Derivative data saved (1794 records). [2025-10-20T22:14:02.764583] [INFO ] [MAIN ] SKE: Successfully saved derivative data. [2025-10-20T22:14:02.767475] [INFO ] [MAIN ] Derivative Progress: 1500/1877 symbols completed. SKE took 1.1s. Estimated remaining: 2m 21s. Rate: 160.0 symbols/min [2025-10-20T22:14:02.768061] [INFO ] [MAIN ] SKIL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:02.768317] [INFO ] [MAIN ] SKM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:02.768173] [INFO ] [MAIN ] SKLZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:02.789144] [DEBUG] [LOCAL ] SKLZ: Loaded 11630 existing records from primary storage. [2025-10-20T22:14:02.788871] [DEBUG] [LOCAL ] SKIL: Loaded 6812 existing records from primary storage. [2025-10-20T22:14:02.789433] [INFO ] [MAIN ] SKIL: Loaded 6812 primary data records. [2025-10-20T22:14:02.789518] [INFO ] [MAIN ] SKLZ: Loaded 11630 primary data records. [2025-10-20T22:14:03.016034] [DEBUG] [LOCAL ] SKM: Loaded 35239 existing records from primary storage. [2025-10-20T22:14:03.019637] [INFO ] [MAIN ] SKM: Loaded 35239 primary data records. [2025-10-20T22:14:03.788735] [DEBUG] [MAIN ] SKIL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:03.789766] [INFO ] [MAIN ] SKIL: Generated 1794 derivative records from 6812 primary records. [2025-10-20T22:14:03.790080] [DEBUG] [LOCAL ] SKIL: Saving derivative data. [2025-10-20T22:14:03.790438] [DEBUG] [MAIN ] SKLZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:03.792454] [INFO ] [MAIN ] SKLZ: Generated 1794 derivative records from 11630 primary records. [2025-10-20T22:14:03.799601] [DEBUG] [LOCAL ] SKLZ: Saving derivative data. [2025-10-20T22:14:03.824769] [DEBUG] [LOCAL ] SKLZ: Derivative data saved (1794 records). [2025-10-20T22:14:03.826606] [INFO ] [MAIN ] SKLZ: Successfully saved derivative data. [2025-10-20T22:14:03.831175] [INFO ] [MAIN ] Derivative Progress: 1501/1877 symbols completed. SKLZ took 1.1s. Estimated remaining: 2m 21s. Rate: 159.8 symbols/min [2025-10-20T22:14:03.832013] [DEBUG] [LOCAL ] SKIL: Derivative data saved (1794 records). [2025-10-20T22:14:03.832225] [INFO ] [MAIN ] SKYH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:03.832615] [INFO ] [MAIN ] SKIL: Successfully saved derivative data. [2025-10-20T22:14:03.835795] [INFO ] [MAIN ] Derivative Progress: 1502/1877 symbols completed. SKIL took 1.1s. Estimated remaining: 2m 21s. Rate: 159.9 symbols/min [2025-10-20T22:14:03.848750] [INFO ] [MAIN ] SLF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:03.849812] [DEBUG] [MAIN ] SKM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:03.850536] [INFO ] [MAIN ] SKM: Generated 1794 derivative records from 35239 primary records. [2025-10-20T22:14:03.850695] [DEBUG] [LOCAL ] SKM: Saving derivative data. [2025-10-20T22:14:03.871304] [DEBUG] [LOCAL ] SKM: Derivative data saved (1794 records). [2025-10-20T22:14:03.871524] [INFO ] [MAIN ] SKM: Successfully saved derivative data. [2025-10-20T22:14:03.873552] [INFO ] [MAIN ] Derivative Progress: 1503/1877 symbols completed. SKM took 1.1s. Estimated remaining: 2m 20s. Rate: 160.0 symbols/min [2025-10-20T22:14:03.873904] [INFO ] [MAIN ] SLQT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:03.881388] [DEBUG] [LOCAL ] SKYH: Loaded 15998 existing records from primary storage. [2025-10-20T22:14:03.881530] [INFO ] [MAIN ] SKYH: Loaded 15998 primary data records. [2025-10-20T22:14:04.196876] [DEBUG] [LOCAL ] SLF: Loaded 38690 existing records from primary storage. [2025-10-20T22:14:04.208704] [INFO ] [MAIN ] SLF: Loaded 38690 primary data records. [2025-10-20T22:14:04.261070] [DEBUG] [LOCAL ] SLQT: Loaded 43489 existing records from primary storage. [2025-10-20T22:14:04.281106] [INFO ] [MAIN ] SLQT: Loaded 43489 primary data records. [2025-10-20T22:14:04.846384] [DEBUG] [MAIN ] SKYH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:04.847722] [INFO ] [MAIN ] SKYH: Generated 1794 derivative records from 15998 primary records. [2025-10-20T22:14:04.848758] [DEBUG] [LOCAL ] SKYH: Saving derivative data. [2025-10-20T22:14:04.867895] [DEBUG] [LOCAL ] SKYH: Derivative data saved (1794 records). [2025-10-20T22:14:04.868758] [INFO ] [MAIN ] SKYH: Successfully saved derivative data. [2025-10-20T22:14:04.871111] [INFO ] [MAIN ] Derivative Progress: 1504/1877 symbols completed. SKYH took 1.0s. Estimated remaining: 2m 20s. Rate: 159.8 symbols/min [2025-10-20T22:14:04.871360] [INFO ] [MAIN ] SMBK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:04.894225] [DEBUG] [LOCAL ] SMBK: Loaded 8274 existing records from primary storage. [2025-10-20T22:14:04.911755] [DEBUG] [MAIN ] SLF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:04.911935] [INFO ] [MAIN ] SMBK: Loaded 8274 primary data records. [2025-10-20T22:14:04.912499] [DEBUG] [MAIN ] SLQT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:04.925333] [INFO ] [MAIN ] SLQT: Generated 1794 derivative records from 43489 primary records. [2025-10-20T22:14:04.913335] [INFO ] [MAIN ] SLF: Generated 1794 derivative records from 38690 primary records. [2025-10-20T22:14:04.934717] [DEBUG] [LOCAL ] SLF: Saving derivative data. [2025-10-20T22:14:04.945190] [DEBUG] [LOCAL ] SLQT: Saving derivative data. [2025-10-20T22:14:05.016312] [DEBUG] [LOCAL ] SLF: Derivative data saved (1794 records). [2025-10-20T22:14:05.021599] [INFO ] [MAIN ] SLF: Successfully saved derivative data. [2025-10-20T22:14:05.029956] [INFO ] [MAIN ] Derivative Progress: 1505/1877 symbols completed. SLF took 1.2s. Estimated remaining: 2m 20s. Rate: 159.9 symbols/min [2025-10-20T22:14:05.053029] [DEBUG] [LOCAL ] SLQT: Derivative data saved (1794 records). [2025-10-20T22:14:05.053206] [INFO ] [MAIN ] SLQT: Successfully saved derivative data. [2025-10-20T22:14:05.038469] [INFO ] [MAIN ] SMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:05.055180] [INFO ] [MAIN ] Derivative Progress: 1506/1877 symbols completed. SLQT took 1.2s. Estimated remaining: 2m 19s. Rate: 160.0 symbols/min [2025-10-20T22:14:05.066769] [INFO ] [MAIN ] SMFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:05.253325] [DEBUG] [LOCAL ] SMC: Loaded 11111 existing records from primary storage. [2025-10-20T22:14:05.253466] [INFO ] [MAIN ] SMC: Loaded 11111 primary data records. [2025-10-20T22:14:05.605823] [DEBUG] [LOCAL ] SMFG: Loaded 48395 existing records from primary storage. [2025-10-20T22:14:05.606272] [INFO ] [MAIN ] SMFG: Loaded 48395 primary data records. [2025-10-20T22:14:05.729819] [DEBUG] [MAIN ] SMBK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:05.730857] [INFO ] [MAIN ] SMBK: Generated 1794 derivative records from 8274 primary records. [2025-10-20T22:14:05.730973] [DEBUG] [LOCAL ] SMBK: Saving derivative data. [2025-10-20T22:14:05.795774] [DEBUG] [LOCAL ] SMBK: Derivative data saved (1794 records). [2025-10-20T22:14:05.819763] [INFO ] [MAIN ] SMBK: Successfully saved derivative data. [2025-10-20T22:14:05.827013] [INFO ] [MAIN ] Derivative Progress: 1507/1877 symbols completed. SMBK took 1.0s. Estimated remaining: 2m 19s. Rate: 159.9 symbols/min [2025-10-20T22:14:05.932130] [INFO ] [MAIN ] SMHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:05.957430] [DEBUG] [MAIN ] SMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:05.958902] [INFO ] [MAIN ] SMC: Generated 1794 derivative records from 11111 primary records. [2025-10-20T22:14:05.959495] [DEBUG] [LOCAL ] SMC: Saving derivative data. [2025-10-20T22:14:05.979733] [DEBUG] [LOCAL ] SMC: Derivative data saved (1794 records). [2025-10-20T22:14:05.979883] [INFO ] [MAIN ] SMC: Successfully saved derivative data. [2025-10-20T22:14:05.981994] [INFO ] [MAIN ] Derivative Progress: 1508/1877 symbols completed. SMC took 0.9s. Estimated remaining: 2m 18s. Rate: 159.9 symbols/min [2025-10-20T22:14:05.988993] [DEBUG] [LOCAL ] SMHI: Loaded 13011 existing records from primary storage. [2025-10-20T22:14:05.989388] [INFO ] [MAIN ] SMHI: Loaded 13011 primary data records. [2025-10-20T22:14:05.989248] [INFO ] [MAIN ] SMRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:06.339385] [DEBUG] [MAIN ] SMFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:06.342132] [INFO ] [MAIN ] SMFG: Generated 1794 derivative records from 48395 primary records. [2025-10-20T22:14:06.343127] [DEBUG] [LOCAL ] SMFG: Saving derivative data. [2025-10-20T22:14:06.374088] [DEBUG] [LOCAL ] SMFG: Derivative data saved (1794 records). [2025-10-20T22:14:06.375485] [DEBUG] [LOCAL ] SMRT: Loaded 43776 existing records from primary storage. [2025-10-20T22:14:06.375771] [INFO ] [MAIN ] SMFG: Successfully saved derivative data. [2025-10-20T22:14:06.378165] [INFO ] [MAIN ] SMRT: Loaded 43776 primary data records. [2025-10-20T22:14:06.378058] [INFO ] [MAIN ] Derivative Progress: 1509/1877 symbols completed. SMFG took 1.3s. Estimated remaining: 2m 18s. Rate: 159.9 symbols/min [2025-10-20T22:14:06.378809] [INFO ] [MAIN ] SMWB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:06.697638] [DEBUG] [LOCAL ] SMWB: Loaded 31939 existing records from primary storage. [2025-10-20T22:14:06.698226] [INFO ] [MAIN ] SMWB: Loaded 31939 primary data records. [2025-10-20T22:14:06.749063] [DEBUG] [MAIN ] SMHI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:06.799871] [INFO ] [MAIN ] SMHI: Generated 1793 derivative records from 13011 primary records. [2025-10-20T22:14:06.821741] [DEBUG] [LOCAL ] SMHI: Saving derivative data. [2025-10-20T22:14:07.030180] [DEBUG] [LOCAL ] SMHI: Derivative data saved (1793 records). [2025-10-20T22:14:07.030539] [INFO ] [MAIN ] SMHI: Successfully saved derivative data. [2025-10-20T22:14:07.032967] [INFO ] [MAIN ] Derivative Progress: 1510/1877 symbols completed. SMHI took 1.1s. Estimated remaining: 2m 18s. Rate: 159.8 symbols/min [2025-10-20T22:14:07.033230] [INFO ] [MAIN ] SNDA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:07.043761] [DEBUG] [LOCAL ] SNDA: Loaded 4488 existing records from primary storage. [2025-10-20T22:14:07.056756] [INFO ] [MAIN ] SNDA: Loaded 4488 primary data records. [2025-10-20T22:14:07.056697] [DEBUG] [MAIN ] SMRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:07.093008] [INFO ] [MAIN ] SMRT: Generated 1794 derivative records from 43776 primary records. [2025-10-20T22:14:07.132150] [DEBUG] [LOCAL ] SMRT: Saving derivative data. [2025-10-20T22:14:07.212791] [DEBUG] [LOCAL ] SMRT: Derivative data saved (1794 records). [2025-10-20T22:14:07.249455] [INFO ] [MAIN ] SMRT: Successfully saved derivative data. [2025-10-20T22:14:07.396108] [INFO ] [MAIN ] Derivative Progress: 1511/1877 symbols completed. SMRT took 1.4s. Estimated remaining: 2m 17s. Rate: 159.8 symbols/min [2025-10-20T22:14:07.404199] [INFO ] [MAIN ] SNN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:07.422868] [DEBUG] [MAIN ] SMWB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:07.423915] [INFO ] [MAIN ] SMWB: Generated 1794 derivative records from 31939 primary records. [2025-10-20T22:14:07.424385] [DEBUG] [LOCAL ] SMWB: Saving derivative data. [2025-10-20T22:14:07.442855] [DEBUG] [LOCAL ] SMWB: Derivative data saved (1794 records). [2025-10-20T22:14:07.443547] [INFO ] [MAIN ] SMWB: Successfully saved derivative data. [2025-10-20T22:14:07.445705] [INFO ] [MAIN ] Derivative Progress: 1512/1877 symbols completed. SMWB took 1.1s. Estimated remaining: 2m 17s. Rate: 159.9 symbols/min [2025-10-20T22:14:07.446065] [INFO ] [MAIN ] SOBO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:07.473217] [DEBUG] [MAIN ] SNDA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:07.473896] [INFO ] [MAIN ] SNDA: Generated 1794 derivative records from 4488 primary records. [2025-10-20T22:14:07.474090] [DEBUG] [LOCAL ] SNDA: Saving derivative data. [2025-10-20T22:14:07.488934] [DEBUG] [LOCAL ] SNDA: Derivative data saved (1794 records). [2025-10-20T22:14:07.489097] [INFO ] [MAIN ] SNDA: Successfully saved derivative data. [2025-10-20T22:14:07.491698] [INFO ] [MAIN ] Derivative Progress: 1513/1877 symbols completed. SNDA took 0.5s. Estimated remaining: 2m 16s. Rate: 160.0 symbols/min [2025-10-20T22:14:07.507491] [DEBUG] [LOCAL ] SNN: Loaded 44407 existing records from primary storage. [2025-10-20T22:14:07.507740] [INFO ] [MAIN ] SNN: Loaded 44407 primary data records. [2025-10-20T22:14:07.707508] [DEBUG] [LOCAL ] SOBO: Loaded 39419 existing records from primary storage. [2025-10-20T22:14:07.812693] [INFO ] [MAIN ] SOBO: Loaded 39419 primary data records. [2025-10-20T22:14:08.132753] [DEBUG] [MAIN ] SNN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:08.133761] [INFO ] [MAIN ] SNN: Generated 1794 derivative records from 44407 primary records. [2025-10-20T22:14:08.133835] [DEBUG] [LOCAL ] SNN: Saving derivative data. [2025-10-20T22:14:08.150587] [DEBUG] [LOCAL ] SNN: Derivative data saved (1794 records). [2025-10-20T22:14:08.150886] [INFO ] [MAIN ] SNN: Successfully saved derivative data. [2025-10-20T22:14:08.152802] [INFO ] [MAIN ] Derivative Progress: 1514/1877 symbols completed. SNN took 0.7s. Estimated remaining: 2m 16s. Rate: 159.9 symbols/min [2025-10-20T22:14:08.168067] [DEBUG] [MAIN ] SOBO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:08.168664] [INFO ] [MAIN ] SOBO: Generated 1794 derivative records from 39419 primary records. [2025-10-20T22:14:08.168726] [DEBUG] [LOCAL ] SOBO: Saving derivative data. [2025-10-20T22:14:08.183458] [DEBUG] [LOCAL ] SOBO: Derivative data saved (1794 records). [2025-10-20T22:14:08.183513] [INFO ] [MAIN ] SOBO: Successfully saved derivative data. [2025-10-20T22:14:08.185665] [INFO ] [MAIN ] Derivative Progress: 1515/1877 symbols completed. SOBO took 0.7s. Estimated remaining: 2m 16s. Rate: 160.0 symbols/min [2025-10-20T22:14:08.186305] [INFO ] [MAIN ] SOJD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:08.186252] [INFO ] [MAIN ] SOJC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:08.186569] [INFO ] [MAIN ] SOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:08.194312] [DEBUG] [LOCAL ] SOJC: Loaded 4689 existing records from primary storage. [2025-10-20T22:14:08.194486] [INFO ] [MAIN ] SOJC: Loaded 4689 primary data records. [2025-10-20T22:14:08.294482] [DEBUG] [LOCAL ] SOJD: Loaded 12905 existing records from primary storage. [2025-10-20T22:14:08.305250] [INFO ] [MAIN ] SOJD: Loaded 12905 primary data records. [2025-10-20T22:14:08.727326] [DEBUG] [LOCAL ] SOC: Loaded 51208 existing records from primary storage. [2025-10-20T22:14:08.857161] [INFO ] [MAIN ] SOC: Loaded 51208 primary data records. [2025-10-20T22:14:09.251209] [DEBUG] [MAIN ] SOJC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:09.253604] [INFO ] [MAIN ] SOJC: Generated 1794 derivative records from 4689 primary records. [2025-10-20T22:14:09.254631] [DEBUG] [LOCAL ] SOJC: Saving derivative data. [2025-10-20T22:14:09.286867] [DEBUG] [LOCAL ] SOJC: Derivative data saved (1794 records). [2025-10-20T22:14:09.287255] [DEBUG] [MAIN ] SOJD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:09.288105] [INFO ] [MAIN ] SOJD: Generated 1794 derivative records from 12905 primary records. [2025-10-20T22:14:09.287665] [INFO ] [MAIN ] SOJC: Successfully saved derivative data. [2025-10-20T22:14:09.290420] [INFO ] [MAIN ] Derivative Progress: 1516/1877 symbols completed. SOJC took 1.1s. Estimated remaining: 2m 16s. Rate: 159.8 symbols/min [2025-10-20T22:14:09.290621] [INFO ] [MAIN ] SOJE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:09.288216] [DEBUG] [LOCAL ] SOJD: Saving derivative data. [2025-10-20T22:14:09.310595] [DEBUG] [LOCAL ] SOJD: Derivative data saved (1794 records). [2025-10-20T22:14:09.311061] [INFO ] [MAIN ] SOJD: Successfully saved derivative data. [2025-10-20T22:14:09.313139] [INFO ] [MAIN ] Derivative Progress: 1517/1877 symbols completed. SOJD took 1.1s. Estimated remaining: 2m 15s. Rate: 159.9 symbols/min [2025-10-20T22:14:09.313525] [INFO ] [MAIN ] SOL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:09.318523] [DEBUG] [LOCAL ] SOJE: Loaded 8803 existing records from primary storage. [2025-10-20T22:14:09.318881] [INFO ] [MAIN ] SOJE: Loaded 8803 primary data records. [2025-10-20T22:14:09.457466] [DEBUG] [LOCAL ] SOL: Loaded 13587 existing records from primary storage. [2025-10-20T22:14:09.468177] [INFO ] [MAIN ] SOL: Loaded 13587 primary data records. [2025-10-20T22:14:09.468500] [DEBUG] [MAIN ] SOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:09.500823] [INFO ] [MAIN ] SOC: Generated 1794 derivative records from 51208 primary records. [2025-10-20T22:14:09.565743] [DEBUG] [LOCAL ] SOC: Saving derivative data. [2025-10-20T22:14:09.764740] [DEBUG] [LOCAL ] SOC: Derivative data saved (1794 records). [2025-10-20T22:14:09.769393] [INFO ] [MAIN ] SOC: Successfully saved derivative data. [2025-10-20T22:14:09.788418] [INFO ] [MAIN ] Derivative Progress: 1518/1877 symbols completed. SOC took 1.6s. Estimated remaining: 2m 15s. Rate: 159.9 symbols/min [2025-10-20T22:14:09.804886] [INFO ] [MAIN ] SONY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:09.997332] [DEBUG] [MAIN ] SOJE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:09.998340] [INFO ] [MAIN ] SOJE: Generated 1794 derivative records from 8803 primary records. [2025-10-20T22:14:09.998700] [DEBUG] [LOCAL ] SOJE: Saving derivative data. [2025-10-20T22:14:10.031158] [DEBUG] [LOCAL ] SOJE: Derivative data saved (1794 records). [2025-10-20T22:14:10.032029] [INFO ] [MAIN ] SOJE: Successfully saved derivative data. [2025-10-20T22:14:10.063234] [DEBUG] [MAIN ] SOL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:10.064132] [INFO ] [MAIN ] SOL: Generated 1794 derivative records from 13587 primary records. [2025-10-20T22:14:10.062803] [INFO ] [MAIN ] Derivative Progress: 1519/1877 symbols completed. SOJE took 0.8s. Estimated remaining: 2m 14s. Rate: 159.9 symbols/min [2025-10-20T22:14:10.064244] [DEBUG] [LOCAL ] SOL: Saving derivative data. [2025-10-20T22:14:10.064347] [INFO ] [MAIN ] SOR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:10.065497] [DEBUG] [LOCAL ] SONY: Loaded 53379 existing records from primary storage. [2025-10-20T22:14:10.066628] [INFO ] [MAIN ] SONY: Loaded 53379 primary data records. [2025-10-20T22:14:10.146324] [DEBUG] [LOCAL ] SOL: Derivative data saved (1794 records). [2025-10-20T22:14:10.170380] [DEBUG] [LOCAL ] SOR: Loaded 2827 existing records from primary storage. [2025-10-20T22:14:10.184102] [INFO ] [MAIN ] SOL: Successfully saved derivative data. [2025-10-20T22:14:10.187329] [INFO ] [MAIN ] SOR: Loaded 2827 primary data records. [2025-10-20T22:14:10.189424] [INFO ] [MAIN ] Derivative Progress: 1520/1877 symbols completed. SOL took 0.9s. Estimated remaining: 2m 14s. Rate: 160.0 symbols/min [2025-10-20T22:14:10.200464] [INFO ] [MAIN ] SOS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:10.549778] [DEBUG] [LOCAL ] SOS: Loaded 9555 existing records from primary storage. [2025-10-20T22:14:10.674807] [INFO ] [MAIN ] SOS: Loaded 9555 primary data records. [2025-10-20T22:14:11.013917] [DEBUG] [MAIN ] SONY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:11.020962] [INFO ] [MAIN ] SONY: Generated 1794 derivative records from 53379 primary records. [2025-10-20T22:14:11.021163] [DEBUG] [LOCAL ] SONY: Saving derivative data. [2025-10-20T22:14:11.020128] [DEBUG] [MAIN ] SOR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:11.022875] [INFO ] [MAIN ] SOR: Generated 1794 derivative records from 2827 primary records. [2025-10-20T22:14:11.023313] [DEBUG] [LOCAL ] SOR: Saving derivative data. [2025-10-20T22:14:11.052354] [DEBUG] [LOCAL ] SOR: Derivative data saved (1794 records). [2025-10-20T22:14:11.052732] [INFO ] [MAIN ] SOR: Successfully saved derivative data. [2025-10-20T22:14:11.054977] [INFO ] [MAIN ] Derivative Progress: 1521/1877 symbols completed. SOR took 1.0s. Estimated remaining: 2m 14s. Rate: 159.9 symbols/min [2025-10-20T22:14:11.056918] [INFO ] [MAIN ] SPB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:11.058278] [DEBUG] [LOCAL ] SONY: Derivative data saved (1794 records). [2025-10-20T22:14:11.058663] [INFO ] [MAIN ] SONY: Successfully saved derivative data. [2025-10-20T22:14:11.060998] [INFO ] [MAIN ] Derivative Progress: 1522/1877 symbols completed. SONY took 1.3s. Estimated remaining: 2m 13s. Rate: 160.0 symbols/min [2025-10-20T22:14:11.061231] [INFO ] [MAIN ] SPCE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:11.075066] [DEBUG] [MAIN ] SOS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:11.075886] [INFO ] [MAIN ] SOS: Generated 1794 derivative records from 9555 primary records. [2025-10-20T22:14:11.075948] [DEBUG] [LOCAL ] SOS: Saving derivative data. [2025-10-20T22:14:11.091037] [DEBUG] [LOCAL ] SOS: Derivative data saved (1794 records). [2025-10-20T22:14:11.091104] [INFO ] [MAIN ] SOS: Successfully saved derivative data. [2025-10-20T22:14:11.092984] [INFO ] [MAIN ] Derivative Progress: 1523/1877 symbols completed. SOS took 0.9s. Estimated remaining: 2m 13s. Rate: 160.1 symbols/min [2025-10-20T22:14:11.093237] [INFO ] [MAIN ] SPE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:11.119909] [DEBUG] [LOCAL ] SPE: Loaded 6265 existing records from primary storage. [2025-10-20T22:14:11.129670] [INFO ] [MAIN ] SPE: Loaded 6265 primary data records. [2025-10-20T22:14:11.174785] [DEBUG] [LOCAL ] SPB: Loaded 34136 existing records from primary storage. [2025-10-20T22:14:11.249288] [INFO ] [MAIN ] SPB: Loaded 34136 primary data records. [2025-10-20T22:14:11.250449] [DEBUG] [LOCAL ] SPCE: Loaded 50018 existing records from primary storage. [2025-10-20T22:14:11.302607] [INFO ] [MAIN ] SPCE: Loaded 50018 primary data records. [2025-10-20T22:14:12.088452] [DEBUG] [MAIN ] SPE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:12.089510] [INFO ] [MAIN ] SPE: Generated 1794 derivative records from 6265 primary records. [2025-10-20T22:14:12.089774] [DEBUG] [LOCAL ] SPE: Saving derivative data. [2025-10-20T22:14:12.108525] [DEBUG] [LOCAL ] SPE: Derivative data saved (1794 records). [2025-10-20T22:14:12.108747] [INFO ] [MAIN ] SPE: Successfully saved derivative data. [2025-10-20T22:14:12.110877] [INFO ] [MAIN ] Derivative Progress: 1524/1877 symbols completed. SPE took 1.0s. Estimated remaining: 2m 12s. Rate: 159.9 symbols/min [2025-10-20T22:14:12.124798] [DEBUG] [MAIN ] SPCE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:12.123750] [INFO ] [MAIN ] SPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:12.125561] [INFO ] [MAIN ] SPCE: Generated 1794 derivative records from 50018 primary records. [2025-10-20T22:14:12.126684] [DEBUG] [LOCAL ] SPCE: Saving derivative data. [2025-10-20T22:14:12.155556] [DEBUG] [LOCAL ] SPCE: Derivative data saved (1794 records). [2025-10-20T22:14:12.155916] [INFO ] [MAIN ] SPCE: Successfully saved derivative data. [2025-10-20T22:14:12.155796] [DEBUG] [MAIN ] SPB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:12.158168] [INFO ] [MAIN ] Derivative Progress: 1525/1877 symbols completed. SPCE took 1.1s. Estimated remaining: 2m 12s. Rate: 160.0 symbols/min [2025-10-20T22:14:12.158950] [INFO ] [MAIN ] SPB: Generated 1794 derivative records from 34136 primary records. [2025-10-20T22:14:12.159168] [INFO ] [MAIN ] SPHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:12.159279] [DEBUG] [LOCAL ] SPB: Saving derivative data. [2025-10-20T22:14:12.179441] [DEBUG] [LOCAL ] SPB: Derivative data saved (1794 records). [2025-10-20T22:14:12.179702] [INFO ] [MAIN ] SPB: Successfully saved derivative data. [2025-10-20T22:14:12.181862] [INFO ] [MAIN ] Derivative Progress: 1526/1877 symbols completed. SPB took 1.1s. Estimated remaining: 2m 12s. Rate: 160.1 symbols/min [2025-10-20T22:14:12.182070] [INFO ] [MAIN ] SPIR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:12.190045] [DEBUG] [LOCAL ] SPH: Loaded 16660 existing records from primary storage. [2025-10-20T22:14:12.190230] [INFO ] [MAIN ] SPH: Loaded 16660 primary data records. [2025-10-20T22:14:12.510084] [DEBUG] [LOCAL ] SPHR: Loaded 40682 existing records from primary storage. [2025-10-20T22:14:12.510984] [INFO ] [MAIN ] SPHR: Loaded 40682 primary data records. [2025-10-20T22:14:12.627368] [DEBUG] [LOCAL ] SPIR: Loaded 37644 existing records from primary storage. [2025-10-20T22:14:12.627724] [INFO ] [MAIN ] SPIR: Loaded 37644 primary data records. [2025-10-20T22:14:13.179172] [DEBUG] [MAIN ] SPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:13.180849] [INFO ] [MAIN ] SPH: Generated 1794 derivative records from 16660 primary records. [2025-10-20T22:14:13.181177] [DEBUG] [LOCAL ] SPH: Saving derivative data. [2025-10-20T22:14:13.199140] [DEBUG] [LOCAL ] SPH: Derivative data saved (1794 records). [2025-10-20T22:14:13.199396] [INFO ] [MAIN ] SPH: Successfully saved derivative data. [2025-10-20T22:14:13.201686] [INFO ] [MAIN ] Derivative Progress: 1527/1877 symbols completed. SPH took 1.1s. Estimated remaining: 2m 11s. Rate: 159.9 symbols/min [2025-10-20T22:14:13.201924] [INFO ] [MAIN ] SPMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:13.218957] [DEBUG] [MAIN ] SPHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:13.219754] [INFO ] [MAIN ] SPHR: Generated 1794 derivative records from 40682 primary records. [2025-10-20T22:14:13.221399] [DEBUG] [LOCAL ] SPHR: Saving derivative data. [2025-10-20T22:14:13.238757] [DEBUG] [LOCAL ] SPMC: Loaded 5386 existing records from primary storage. [2025-10-20T22:14:13.240842] [INFO ] [MAIN ] SPMC: Loaded 5386 primary data records. [2025-10-20T22:14:13.273510] [DEBUG] [LOCAL ] SPHR: Derivative data saved (1794 records). [2025-10-20T22:14:13.273817] [INFO ] [MAIN ] SPHR: Successfully saved derivative data. [2025-10-20T22:14:13.275758] [INFO ] [MAIN ] Derivative Progress: 1528/1877 symbols completed. SPHR took 1.1s. Estimated remaining: 2m 11s. Rate: 160.0 symbols/min [2025-10-20T22:14:13.332293] [DEBUG] [MAIN ] SPIR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:13.338093] [INFO ] [MAIN ] SPIR: Generated 1794 derivative records from 37644 primary records. [2025-10-20T22:14:13.356268] [DEBUG] [LOCAL ] SPIR: Saving derivative data. [2025-10-20T22:14:13.442776] [DEBUG] [LOCAL ] SPIR: Derivative data saved (1794 records). [2025-10-20T22:14:13.552695] [INFO ] [MAIN ] SPIR: Successfully saved derivative data. [2025-10-20T22:14:13.562585] [INFO ] [MAIN ] Derivative Progress: 1529/1877 symbols completed. SPIR took 1.4s. Estimated remaining: 2m 10s. Rate: 160.0 symbols/min [2025-10-20T22:14:13.585120] [DEBUG] [MAIN ] SPMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:13.585690] [INFO ] [MAIN ] SPMC: Generated 1794 derivative records from 5386 primary records. [2025-10-20T22:14:13.585752] [DEBUG] [LOCAL ] SPMC: Saving derivative data. [2025-10-20T22:14:13.599618] [DEBUG] [LOCAL ] SPMC: Derivative data saved (1794 records). [2025-10-20T22:14:13.599700] [INFO ] [MAIN ] SPMC: Successfully saved derivative data. [2025-10-20T22:14:13.601750] [INFO ] [MAIN ] Derivative Progress: 1530/1877 symbols completed. SPMC took 0.4s. Estimated remaining: 2m 10s. Rate: 160.1 symbols/min [2025-10-20T22:14:13.602286] [INFO ] [MAIN ] SPRU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:13.602356] [INFO ] [MAIN ] SPXX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:13.602423] [INFO ] [MAIN ] SPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:13.620241] [DEBUG] [LOCAL ] SPXX: Loaded 9266 existing records from primary storage. [2025-10-20T22:14:13.619956] [DEBUG] [LOCAL ] SPRU: Loaded 10032 existing records from primary storage. [2025-10-20T22:14:13.620385] [INFO ] [MAIN ] SPRU: Loaded 10032 primary data records. [2025-10-20T22:14:13.620442] [INFO ] [MAIN ] SPXX: Loaded 9266 primary data records. [2025-10-20T22:14:14.207304] [DEBUG] [LOCAL ] SPR: Loaded 40983 existing records from primary storage. [2025-10-20T22:14:14.220031] [INFO ] [MAIN ] SPR: Loaded 40983 primary data records. [2025-10-20T22:14:14.541177] [DEBUG] [MAIN ] SPRU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:14.553883] [INFO ] [MAIN ] SPRU: Generated 1794 derivative records from 10032 primary records. [2025-10-20T22:14:14.554269] [DEBUG] [LOCAL ] SPRU: Saving derivative data. [2025-10-20T22:14:14.554494] [DEBUG] [MAIN ] SPXX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:14.556670] [INFO ] [MAIN ] SPXX: Generated 1794 derivative records from 9266 primary records. [2025-10-20T22:14:14.564408] [DEBUG] [LOCAL ] SPXX: Saving derivative data. [2025-10-20T22:14:14.571624] [DEBUG] [LOCAL ] SPRU: Derivative data saved (1794 records). [2025-10-20T22:14:14.579253] [INFO ] [MAIN ] SPRU: Successfully saved derivative data. [2025-10-20T22:14:14.581541] [INFO ] [MAIN ] Derivative Progress: 1531/1877 symbols completed. SPRU took 1.0s. Estimated remaining: 2m 10s. Rate: 159.9 symbols/min [2025-10-20T22:14:14.584341] [INFO ] [MAIN ] SQM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:14.588366] [DEBUG] [LOCAL ] SPXX: Derivative data saved (1794 records). [2025-10-20T22:14:14.588897] [INFO ] [MAIN ] SPXX: Successfully saved derivative data. [2025-10-20T22:14:14.590967] [INFO ] [MAIN ] Derivative Progress: 1532/1877 symbols completed. SPXX took 1.0s. Estimated remaining: 2m 9s. Rate: 160.0 symbols/min [2025-10-20T22:14:14.591438] [INFO ] [MAIN ] SQNS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:14.608590] [DEBUG] [MAIN ] SPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:14.609251] [INFO ] [MAIN ] SPR: Generated 1794 derivative records from 40983 primary records. [2025-10-20T22:14:14.609449] [DEBUG] [LOCAL ] SPR: Saving derivative data. [2025-10-20T22:14:14.641951] [DEBUG] [LOCAL ] SPR: Derivative data saved (1794 records). [2025-10-20T22:14:14.642088] [INFO ] [MAIN ] SPR: Successfully saved derivative data. [2025-10-20T22:14:14.644490] [INFO ] [MAIN ] Derivative Progress: 1533/1877 symbols completed. SPR took 1.0s. Estimated remaining: 2m 9s. Rate: 160.1 symbols/min [2025-10-20T22:14:14.645232] [INFO ] [MAIN ] SREA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:14.655217] [DEBUG] [LOCAL ] SQNS: Loaded 27927 existing records from primary storage. [2025-10-20T22:14:14.655359] [INFO ] [MAIN ] SQNS: Loaded 27927 primary data records. [2025-10-20T22:14:14.764127] [DEBUG] [LOCAL ] SQM: Loaded 48976 existing records from primary storage. [2025-10-20T22:14:14.769747] [DEBUG] [LOCAL ] SREA: Loaded 9830 existing records from primary storage. [2025-10-20T22:14:14.780439] [INFO ] [MAIN ] SREA: Loaded 9830 primary data records. [2025-10-20T22:14:14.780876] [INFO ] [MAIN ] SQM: Loaded 48976 primary data records. [2025-10-20T22:14:15.636979] [DEBUG] [MAIN ] SQM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:15.636608] [DEBUG] [MAIN ] SQNS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:15.638018] [INFO ] [MAIN ] SQM: Generated 1794 derivative records from 48976 primary records. [2025-10-20T22:14:15.638113] [DEBUG] [LOCAL ] SQM: Saving derivative data. [2025-10-20T22:14:15.639498] [INFO ] [MAIN ] SQNS: Generated 1794 derivative records from 27927 primary records. [2025-10-20T22:14:15.640990] [DEBUG] [LOCAL ] SQNS: Saving derivative data. [2025-10-20T22:14:15.676191] [DEBUG] [LOCAL ] SQM: Derivative data saved (1794 records). [2025-10-20T22:14:15.680158] [INFO ] [MAIN ] SQM: Successfully saved derivative data. [2025-10-20T22:14:15.685373] [DEBUG] [MAIN ] SREA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:15.682734] [INFO ] [MAIN ] Derivative Progress: 1534/1877 symbols completed. SQM took 1.1s. Estimated remaining: 2m 9s. Rate: 159.9 symbols/min [2025-10-20T22:14:15.687316] [INFO ] [MAIN ] SRFM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:15.686344] [DEBUG] [LOCAL ] SQNS: Derivative data saved (1794 records). [2025-10-20T22:14:15.687226] [INFO ] [MAIN ] SREA: Generated 1794 derivative records from 9830 primary records. [2025-10-20T22:14:15.687695] [INFO ] [MAIN ] SQNS: Successfully saved derivative data. [2025-10-20T22:14:15.690054] [INFO ] [MAIN ] Derivative Progress: 1535/1877 symbols completed. SQNS took 1.1s. Estimated remaining: 2m 8s. Rate: 160.0 symbols/min [2025-10-20T22:14:15.687596] [DEBUG] [LOCAL ] SREA: Saving derivative data. [2025-10-20T22:14:15.691333] [INFO ] [MAIN ] SRG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:15.708045] [DEBUG] [LOCAL ] SREA: Derivative data saved (1794 records). [2025-10-20T22:14:15.708225] [INFO ] [MAIN ] SREA: Successfully saved derivative data. [2025-10-20T22:14:15.710442] [INFO ] [MAIN ] Derivative Progress: 1536/1877 symbols completed. SREA took 1.1s. Estimated remaining: 2m 8s. Rate: 160.1 symbols/min [2025-10-20T22:14:15.710613] [INFO ] [MAIN ] SRI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:15.734127] [DEBUG] [LOCAL ] SRG: Loaded 14970 existing records from primary storage. [2025-10-20T22:14:15.734407] [INFO ] [MAIN ] SRG: Loaded 14970 primary data records. [2025-10-20T22:14:15.865465] [DEBUG] [LOCAL ] SRI: Loaded 26431 existing records from primary storage. [2025-10-20T22:14:15.866348] [DEBUG] [LOCAL ] SRFM: Loaded 39590 existing records from primary storage. [2025-10-20T22:14:15.865695] [INFO ] [MAIN ] SRI: Loaded 26431 primary data records. [2025-10-20T22:14:15.866481] [INFO ] [MAIN ] SRFM: Loaded 39590 primary data records. [2025-10-20T22:14:16.658787] [DEBUG] [MAIN ] SRG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:16.660110] [INFO ] [MAIN ] SRG: Generated 1794 derivative records from 14970 primary records. [2025-10-20T22:14:16.660270] [DEBUG] [LOCAL ] SRG: Saving derivative data. [2025-10-20T22:14:16.678619] [DEBUG] [LOCAL ] SRG: Derivative data saved (1794 records). [2025-10-20T22:14:16.679596] [INFO ] [MAIN ] SRG: Successfully saved derivative data. [2025-10-20T22:14:16.682575] [INFO ] [MAIN ] Derivative Progress: 1537/1877 symbols completed. SRG took 1.0s. Estimated remaining: 2m 8s. Rate: 160.0 symbols/min [2025-10-20T22:14:16.690473] [INFO ] [MAIN ] SRV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:16.696231] [DEBUG] [MAIN ] SRI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:16.704456] [INFO ] [MAIN ] SRI: Generated 1794 derivative records from 26431 primary records. [2025-10-20T22:14:16.709311] [DEBUG] [LOCAL ] SRI: Saving derivative data. [2025-10-20T22:14:16.709543] [DEBUG] [MAIN ] SRFM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:16.712092] [INFO ] [MAIN ] SRFM: Generated 1794 derivative records from 39590 primary records. [2025-10-20T22:14:16.712218] [DEBUG] [LOCAL ] SRFM: Saving derivative data. [2025-10-20T22:14:16.745062] [DEBUG] [LOCAL ] SRV: Loaded 7460 existing records from primary storage. [2025-10-20T22:14:16.746065] [INFO ] [MAIN ] SRV: Loaded 7460 primary data records. [2025-10-20T22:14:16.749498] [DEBUG] [LOCAL ] SRI: Derivative data saved (1794 records). [2025-10-20T22:14:16.746856] [DEBUG] [LOCAL ] SRFM: Derivative data saved (1794 records). [2025-10-20T22:14:16.754735] [INFO ] [MAIN ] SRI: Successfully saved derivative data. [2025-10-20T22:14:16.754874] [INFO ] [MAIN ] SRFM: Successfully saved derivative data. [2025-10-20T22:14:16.756961] [INFO ] [MAIN ] Derivative Progress: 1538/1877 symbols completed. SRI took 1.0s. Estimated remaining: 2m 7s. Rate: 160.1 symbols/min [2025-10-20T22:14:16.809437] [INFO ] [MAIN ] Derivative Progress: 1539/1877 symbols completed. SRFM took 1.1s. Estimated remaining: 2m 7s. Rate: 160.1 symbols/min [2025-10-20T22:14:16.798998] [INFO ] [MAIN ] SSL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:16.819797] [INFO ] [MAIN ] SST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:17.067022] [DEBUG] [LOCAL ] SST: Loaded 11489 existing records from primary storage. [2025-10-20T22:14:17.080905] [INFO ] [MAIN ] SST: Loaded 11489 primary data records. [2025-10-20T22:14:17.285520] [DEBUG] [LOCAL ] SSL: Loaded 41522 existing records from primary storage. [2025-10-20T22:14:17.391046] [INFO ] [MAIN ] SSL: Loaded 41522 primary data records. [2025-10-20T22:14:17.710007] [DEBUG] [MAIN ] SRV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:17.712489] [INFO ] [MAIN ] SRV: Generated 1794 derivative records from 7460 primary records. [2025-10-20T22:14:17.712579] [DEBUG] [LOCAL ] SRV: Saving derivative data. [2025-10-20T22:14:17.731219] [DEBUG] [LOCAL ] SRV: Derivative data saved (1794 records). [2025-10-20T22:14:17.731402] [INFO ] [MAIN ] SRV: Successfully saved derivative data. [2025-10-20T22:14:17.741015] [INFO ] [MAIN ] Derivative Progress: 1540/1877 symbols completed. SRV took 1.1s. Estimated remaining: 2m 6s. Rate: 160.0 symbols/min [2025-10-20T22:14:17.741174] [INFO ] [MAIN ] STEM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:17.746779] [DEBUG] [MAIN ] SST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:17.747488] [INFO ] [MAIN ] SST: Generated 1794 derivative records from 11489 primary records. [2025-10-20T22:14:17.747571] [DEBUG] [LOCAL ] SST: Saving derivative data. [2025-10-20T22:14:17.763793] [DEBUG] [LOCAL ] SST: Derivative data saved (1794 records). [2025-10-20T22:14:17.764137] [INFO ] [MAIN ] SST: Successfully saved derivative data. [2025-10-20T22:14:17.766088] [INFO ] [MAIN ] Derivative Progress: 1541/1877 symbols completed. SST took 0.9s. Estimated remaining: 2m 6s. Rate: 160.1 symbols/min [2025-10-20T22:14:17.766349] [INFO ] [MAIN ] STEW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:17.783873] [DEBUG] [MAIN ] SSL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:17.784512] [INFO ] [MAIN ] SSL: Generated 1794 derivative records from 41522 primary records. [2025-10-20T22:14:17.784629] [DEBUG] [LOCAL ] SSL: Saving derivative data. [2025-10-20T22:14:17.804427] [DEBUG] [LOCAL ] SSL: Derivative data saved (1794 records). [2025-10-20T22:14:17.812556] [INFO ] [MAIN ] SSL: Successfully saved derivative data. [2025-10-20T22:14:17.820358] [INFO ] [MAIN ] Derivative Progress: 1542/1877 symbols completed. SSL took 1.0s. Estimated remaining: 2m 5s. Rate: 160.2 symbols/min [2025-10-20T22:14:17.820526] [INFO ] [MAIN ] STK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:17.818417] [DEBUG] [LOCAL ] STEW: Loaded 15442 existing records from primary storage. [2025-10-20T22:14:17.821067] [INFO ] [MAIN ] STEW: Loaded 15442 primary data records. [2025-10-20T22:14:17.944251] [DEBUG] [LOCAL ] STEM: Loaded 38621 existing records from primary storage. [2025-10-20T22:14:17.944457] [INFO ] [MAIN ] STEM: Loaded 38621 primary data records. [2025-10-20T22:14:18.102274] [DEBUG] [LOCAL ] STK: Loaded 9842 existing records from primary storage. [2025-10-20T22:14:18.119690] [INFO ] [MAIN ] STK: Loaded 9842 primary data records. [2025-10-20T22:14:18.783769] [DEBUG] [MAIN ] STEW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:18.785278] [INFO ] [MAIN ] STEW: Generated 1794 derivative records from 15442 primary records. [2025-10-20T22:14:18.785482] [DEBUG] [LOCAL ] STEW: Saving derivative data. [2025-10-20T22:14:18.803497] [DEBUG] [LOCAL ] STEW: Derivative data saved (1794 records). [2025-10-20T22:14:18.805173] [INFO ] [MAIN ] STEW: Successfully saved derivative data. [2025-10-20T22:14:18.808169] [INFO ] [MAIN ] Derivative Progress: 1543/1877 symbols completed. STEW took 1.0s. Estimated remaining: 2m 5s. Rate: 160.0 symbols/min [2025-10-20T22:14:18.821525] [DEBUG] [MAIN ] STEM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:18.822351] [INFO ] [MAIN ] STEM: Generated 1794 derivative records from 38621 primary records. [2025-10-20T22:14:18.822519] [DEBUG] [LOCAL ] STEM: Saving derivative data. [2025-10-20T22:14:18.840059] [DEBUG] [LOCAL ] STEM: Derivative data saved (1794 records). [2025-10-20T22:14:18.852472] [INFO ] [MAIN ] STEM: Successfully saved derivative data. [2025-10-20T22:14:18.854490] [INFO ] [MAIN ] Derivative Progress: 1544/1877 symbols completed. STEM took 1.1s. Estimated remaining: 2m 5s. Rate: 160.1 symbols/min [2025-10-20T22:14:18.852414] [DEBUG] [MAIN ] STK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:18.855263] [INFO ] [MAIN ] STK: Generated 1794 derivative records from 9842 primary records. [2025-10-20T22:14:18.855415] [DEBUG] [LOCAL ] STK: Saving derivative data. [2025-10-20T22:14:18.870155] [DEBUG] [LOCAL ] STK: Derivative data saved (1794 records). [2025-10-20T22:14:18.870213] [INFO ] [MAIN ] STK: Successfully saved derivative data. [2025-10-20T22:14:18.872494] [INFO ] [MAIN ] Derivative Progress: 1545/1877 symbols completed. STK took 1.1s. Estimated remaining: 2m 4s. Rate: 160.2 symbols/min [2025-10-20T22:14:18.873060] [INFO ] [MAIN ] STNG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:18.873126] [INFO ] [MAIN ] STRV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:18.873248] [INFO ] [MAIN ] STN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:18.887325] [DEBUG] [LOCAL ] STRV: Loaded 11011 existing records from primary storage. [2025-10-20T22:14:18.887606] [INFO ] [MAIN ] STRV: Loaded 11011 primary data records. [2025-10-20T22:14:19.030272] [DEBUG] [LOCAL ] STN: Loaded 22776 existing records from primary storage. [2025-10-20T22:14:19.046005] [INFO ] [MAIN ] STN: Loaded 22776 primary data records. [2025-10-20T22:14:19.315141] [DEBUG] [LOCAL ] STNG: Loaded 43623 existing records from primary storage. [2025-10-20T22:14:19.326868] [INFO ] [MAIN ] STNG: Loaded 43623 primary data records. [2025-10-20T22:14:19.854889] [DEBUG] [MAIN ] STRV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:19.855780] [INFO ] [MAIN ] STRV: Generated 1793 derivative records from 11011 primary records. [2025-10-20T22:14:19.856191] [DEBUG] [LOCAL ] STRV: Saving derivative data. [2025-10-20T22:14:19.874063] [DEBUG] [LOCAL ] STRV: Derivative data saved (1793 records). [2025-10-20T22:14:19.874523] [INFO ] [MAIN ] STRV: Successfully saved derivative data. [2025-10-20T22:14:19.876915] [INFO ] [MAIN ] Derivative Progress: 1546/1877 symbols completed. STRV took 1.0s. Estimated remaining: 2m 4s. Rate: 160.0 symbols/min [2025-10-20T22:14:19.877223] [INFO ] [MAIN ] STVN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:19.894078] [DEBUG] [MAIN ] STN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:19.894773] [INFO ] [MAIN ] STN: Generated 1794 derivative records from 22776 primary records. [2025-10-20T22:14:19.895032] [DEBUG] [LOCAL ] STN: Saving derivative data. [2025-10-20T22:14:19.919938] [DEBUG] [LOCAL ] STN: Derivative data saved (1794 records). [2025-10-20T22:14:19.920120] [INFO ] [MAIN ] STN: Successfully saved derivative data. [2025-10-20T22:14:19.934678] [DEBUG] [MAIN ] STNG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:19.934326] [INFO ] [MAIN ] Derivative Progress: 1547/1877 symbols completed. STN took 1.1s. Estimated remaining: 2m 4s. Rate: 160.1 symbols/min [2025-10-20T22:14:19.935451] [INFO ] [MAIN ] STNG: Generated 1794 derivative records from 43623 primary records. [2025-10-20T22:14:19.944440] [DEBUG] [LOCAL ] STVN: Loaded 26823 existing records from primary storage. [2025-10-20T22:14:19.944552] [DEBUG] [LOCAL ] STNG: Saving derivative data. [2025-10-20T22:14:19.944628] [INFO ] [MAIN ] STXD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:19.945708] [INFO ] [MAIN ] STVN: Loaded 26823 primary data records. [2025-10-20T22:14:20.046725] [DEBUG] [LOCAL ] STXD: Loaded 1245 existing records from primary storage. [2025-10-20T22:14:20.064057] [INFO ] [MAIN ] STXD: Loaded 1245 primary data records. [2025-10-20T22:14:20.130720] [DEBUG] [LOCAL ] STNG: Derivative data saved (1794 records). [2025-10-20T22:14:20.232972] [INFO ] [MAIN ] STNG: Successfully saved derivative data. [2025-10-20T22:14:20.253849] [INFO ] [MAIN ] Derivative Progress: 1548/1877 symbols completed. STNG took 1.4s. Estimated remaining: 2m 3s. Rate: 160.1 symbols/min [2025-10-20T22:14:20.353101] [INFO ] [MAIN ] STXE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:20.443130] [DEBUG] [LOCAL ] STXE: Loaded 2087 existing records from primary storage. [2025-10-20T22:14:20.444708] [INFO ] [MAIN ] STXE: Loaded 2087 primary data records. [2025-10-20T22:14:20.924940] [DEBUG] [MAIN ] STVN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:20.926216] [INFO ] [MAIN ] STVN: Generated 1794 derivative records from 26823 primary records. [2025-10-20T22:14:20.926402] [DEBUG] [LOCAL ] STVN: Saving derivative data. [2025-10-20T22:14:20.942867] [DEBUG] [LOCAL ] STVN: Derivative data saved (1794 records). [2025-10-20T22:14:20.943717] [INFO ] [MAIN ] STVN: Successfully saved derivative data. [2025-10-20T22:14:20.946045] [INFO ] [MAIN ] Derivative Progress: 1549/1877 symbols completed. STVN took 1.1s. Estimated remaining: 2m 3s. Rate: 160.0 symbols/min [2025-10-20T22:14:20.946296] [INFO ] [MAIN ] STXK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:20.964846] [DEBUG] [LOCAL ] STXK: Loaded 1512 existing records from primary storage. [2025-10-20T22:14:20.965287] [DEBUG] [MAIN ] STXD: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:20.966069] [INFO ] [MAIN ] STXD: Generated 1793 derivative records from 1245 primary records. [2025-10-20T22:14:20.965437] [INFO ] [MAIN ] STXK: Loaded 1512 primary data records. [2025-10-20T22:14:20.986716] [DEBUG] [LOCAL ] STXD: Saving derivative data. [2025-10-20T22:14:21.067073] [DEBUG] [LOCAL ] STXD: Derivative data saved (1793 records). [2025-10-20T22:14:21.083600] [INFO ] [MAIN ] STXD: Successfully saved derivative data. [2025-10-20T22:14:21.108175] [INFO ] [MAIN ] Derivative Progress: 1550/1877 symbols completed. STXD took 1.2s. Estimated remaining: 2m 3s. Rate: 160.1 symbols/min [2025-10-20T22:14:21.123865] [INFO ] [MAIN ] STXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:21.171399] [DEBUG] [MAIN ] STXE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:21.178940] [INFO ] [MAIN ] STXE: Generated 1794 derivative records from 2087 primary records. [2025-10-20T22:14:21.179699] [DEBUG] [LOCAL ] STXE: Saving derivative data. [2025-10-20T22:14:21.310236] [DEBUG] [LOCAL ] STXE: Derivative data saved (1794 records). [2025-10-20T22:14:21.310327] [INFO ] [MAIN ] STXE: Successfully saved derivative data. [2025-10-20T22:14:21.310185] [DEBUG] [LOCAL ] STXT: Loaded 2492 existing records from primary storage. [2025-10-20T22:14:21.313324] [INFO ] [MAIN ] STXT: Loaded 2492 primary data records. [2025-10-20T22:14:21.313246] [INFO ] [MAIN ] Derivative Progress: 1551/1877 symbols completed. STXE took 1.0s. Estimated remaining: 2m 2s. Rate: 160.1 symbols/min [2025-10-20T22:14:21.313591] [INFO ] [MAIN ] STXV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:21.420274] [DEBUG] [LOCAL ] STXV: Loaded 1488 existing records from primary storage. [2025-10-20T22:14:21.421154] [INFO ] [MAIN ] STXV: Loaded 1488 primary data records. [2025-10-20T22:14:21.807448] [DEBUG] [MAIN ] STXK: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:21.926243] [INFO ] [MAIN ] STXK: Generated 1793 derivative records from 1512 primary records. [2025-10-20T22:14:21.927327] [DEBUG] [LOCAL ] STXK: Saving derivative data. [2025-10-20T22:14:21.942786] [DEBUG] [LOCAL ] STXK: Derivative data saved (1793 records). [2025-10-20T22:14:21.943528] [INFO ] [MAIN ] STXK: Successfully saved derivative data. [2025-10-20T22:14:21.945937] [INFO ] [MAIN ] Derivative Progress: 1552/1877 symbols completed. STXK took 1.0s. Estimated remaining: 2m 2s. Rate: 160.1 symbols/min [2025-10-20T22:14:21.946123] [INFO ] [MAIN ] SUI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:21.967676] [DEBUG] [MAIN ] STXT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:21.968436] [INFO ] [MAIN ] STXT: Generated 1793 derivative records from 2492 primary records. [2025-10-20T22:14:21.968575] [DEBUG] [LOCAL ] STXT: Saving derivative data. [2025-10-20T22:14:22.005801] [DEBUG] [MAIN ] STXV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:22.007290] [INFO ] [MAIN ] STXV: Generated 1793 derivative records from 1488 primary records. [2025-10-20T22:14:22.007702] [DEBUG] [LOCAL ] STXV: Saving derivative data. [2025-10-20T22:14:22.017257] [DEBUG] [LOCAL ] STXT: Derivative data saved (1793 records). [2025-10-20T22:14:22.018249] [INFO ] [MAIN ] STXT: Successfully saved derivative data. [2025-10-20T22:14:22.023936] [INFO ] [MAIN ] Derivative Progress: 1553/1877 symbols completed. STXT took 0.9s. Estimated remaining: 2m 1s. Rate: 160.2 symbols/min [2025-10-20T22:14:22.027022] [DEBUG] [LOCAL ] STXV: Derivative data saved (1793 records). [2025-10-20T22:14:22.027834] [INFO ] [MAIN ] STXV: Successfully saved derivative data. [2025-10-20T22:14:22.028065] [INFO ] [MAIN ] SUN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:22.029982] [INFO ] [MAIN ] Derivative Progress: 1554/1877 symbols completed. STXV took 0.7s. Estimated remaining: 2m 1s. Rate: 160.3 symbols/min [2025-10-20T22:14:22.030573] [INFO ] [MAIN ] SUPV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:22.045716] [DEBUG] [LOCAL ] SUI: Loaded 42016 existing records from primary storage. [2025-10-20T22:14:22.045889] [INFO ] [MAIN ] SUI: Loaded 42016 primary data records. [2025-10-20T22:14:22.426877] [DEBUG] [LOCAL ] SUN: Loaded 38845 existing records from primary storage. [2025-10-20T22:14:22.427316] [INFO ] [MAIN ] SUN: Loaded 38845 primary data records. [2025-10-20T22:14:22.428892] [DEBUG] [LOCAL ] SUPV: Loaded 48009 existing records from primary storage. [2025-10-20T22:14:22.459835] [INFO ] [MAIN ] SUPV: Loaded 48009 primary data records. [2025-10-20T22:14:23.043774] [DEBUG] [MAIN ] SUI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:23.044506] [INFO ] [MAIN ] SUI: Generated 1794 derivative records from 42016 primary records. [2025-10-20T22:14:23.045579] [DEBUG] [LOCAL ] SUI: Saving derivative data. [2025-10-20T22:14:23.066357] [DEBUG] [LOCAL ] SUI: Derivative data saved (1794 records). [2025-10-20T22:14:23.066591] [INFO ] [MAIN ] SUI: Successfully saved derivative data. [2025-10-20T22:14:23.069077] [INFO ] [MAIN ] Derivative Progress: 1555/1877 symbols completed. SUI took 1.1s. Estimated remaining: 2m 1s. Rate: 160.1 symbols/min [2025-10-20T22:14:23.070718] [INFO ] [MAIN ] SUZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:23.109216] [DEBUG] [MAIN ] SUN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:23.108904] [DEBUG] [MAIN ] SUPV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:23.110330] [INFO ] [MAIN ] SUN: Generated 1794 derivative records from 38845 primary records. [2025-10-20T22:14:23.111082] [INFO ] [MAIN ] SUPV: Generated 1794 derivative records from 48009 primary records. [2025-10-20T22:14:23.111278] [DEBUG] [LOCAL ] SUPV: Saving derivative data. [2025-10-20T22:14:23.111470] [DEBUG] [LOCAL ] SUN: Saving derivative data. [2025-10-20T22:14:23.144442] [DEBUG] [LOCAL ] SUPV: Derivative data saved (1794 records). [2025-10-20T22:14:23.145413] [INFO ] [MAIN ] SUPV: Successfully saved derivative data. [2025-10-20T22:14:23.147441] [INFO ] [MAIN ] Derivative Progress: 1556/1877 symbols completed. SUPV took 1.1s. Estimated remaining: 2m 0s. Rate: 160.2 symbols/min [2025-10-20T22:14:23.150310] [INFO ] [MAIN ] SVV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:23.156915] [DEBUG] [LOCAL ] SUN: Derivative data saved (1794 records). [2025-10-20T22:14:23.170582] [INFO ] [MAIN ] SUN: Successfully saved derivative data. [2025-10-20T22:14:23.173888] [DEBUG] [LOCAL ] SUZ: Loaded 45627 existing records from primary storage. [2025-10-20T22:14:23.172872] [INFO ] [MAIN ] Derivative Progress: 1557/1877 symbols completed. SUN took 1.1s. Estimated remaining: 1m 60s. Rate: 160.2 symbols/min [2025-10-20T22:14:23.174170] [INFO ] [MAIN ] SW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:23.174735] [INFO ] [MAIN ] SUZ: Loaded 45627 primary data records. [2025-10-20T22:14:23.329567] [DEBUG] [LOCAL ] SVV: Loaded 40326 existing records from primary storage. [2025-10-20T22:14:23.348266] [INFO ] [MAIN ] SVV: Loaded 40326 primary data records. [2025-10-20T22:14:23.758117] [DEBUG] [LOCAL ] SW: Loaded 53179 existing records from primary storage. [2025-10-20T22:14:23.776539] [INFO ] [MAIN ] SW: Loaded 53179 primary data records. [2025-10-20T22:14:24.323617] [DEBUG] [MAIN ] SUZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:24.413946] [INFO ] [MAIN ] SUZ: Generated 1794 derivative records from 45627 primary records. [2025-10-20T22:14:24.414537] [DEBUG] [LOCAL ] SUZ: Saving derivative data. [2025-10-20T22:14:24.437681] [DEBUG] [LOCAL ] SUZ: Derivative data saved (1794 records). [2025-10-20T22:14:24.438253] [INFO ] [MAIN ] SUZ: Successfully saved derivative data. [2025-10-20T22:14:24.440466] [INFO ] [MAIN ] Derivative Progress: 1558/1877 symbols completed. SUZ took 1.4s. Estimated remaining: 1m 60s. Rate: 160.0 symbols/min [2025-10-20T22:14:24.455241] [DEBUG] [MAIN ] SVV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:24.456205] [INFO ] [MAIN ] SVV: Generated 1794 derivative records from 40326 primary records. [2025-10-20T22:14:24.456900] [DEBUG] [LOCAL ] SVV: Saving derivative data. [2025-10-20T22:14:24.477281] [DEBUG] [LOCAL ] SVV: Derivative data saved (1794 records). [2025-10-20T22:14:24.478127] [INFO ] [MAIN ] SVV: Successfully saved derivative data. [2025-10-20T22:14:24.480523] [INFO ] [MAIN ] Derivative Progress: 1559/1877 symbols completed. SVV took 1.3s. Estimated remaining: 1m 59s. Rate: 160.1 symbols/min [2025-10-20T22:14:24.496576] [DEBUG] [MAIN ] SW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:24.497623] [INFO ] [MAIN ] SW: Generated 1794 derivative records from 53179 primary records. [2025-10-20T22:14:24.497727] [DEBUG] [LOCAL ] SW: Saving derivative data. [2025-10-20T22:14:24.517072] [DEBUG] [LOCAL ] SW: Derivative data saved (1794 records). [2025-10-20T22:14:24.517205] [INFO ] [MAIN ] SW: Successfully saved derivative data. [2025-10-20T22:14:24.519513] [INFO ] [MAIN ] Derivative Progress: 1560/1877 symbols completed. SW took 1.3s. Estimated remaining: 1m 59s. Rate: 160.2 symbols/min [2025-10-20T22:14:24.520051] [INFO ] [MAIN ] SWK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:24.520311] [INFO ] [MAIN ] SWZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:24.520150] [INFO ] [MAIN ] SWX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:24.530412] [DEBUG] [LOCAL ] SWZ: Loaded 6016 existing records from primary storage. [2025-10-20T22:14:24.530749] [INFO ] [MAIN ] SWZ: Loaded 6016 primary data records. [2025-10-20T22:14:24.762874] [DEBUG] [LOCAL ] SWX: Loaded 30791 existing records from primary storage. [2025-10-20T22:14:24.922998] [INFO ] [MAIN ] SWX: Loaded 30791 primary data records. [2025-10-20T22:14:25.007776] [DEBUG] [LOCAL ] SWK: Loaded 52434 existing records from primary storage. [2025-10-20T22:14:25.022371] [INFO ] [MAIN ] SWK: Loaded 52434 primary data records. [2025-10-20T22:14:25.564030] [DEBUG] [MAIN ] SWZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:25.566396] [INFO ] [MAIN ] SWZ: Generated 1794 derivative records from 6016 primary records. [2025-10-20T22:14:25.567187] [DEBUG] [LOCAL ] SWZ: Saving derivative data. [2025-10-20T22:14:25.583981] [DEBUG] [LOCAL ] SWZ: Derivative data saved (1794 records). [2025-10-20T22:14:25.584077] [INFO ] [MAIN ] SWZ: Successfully saved derivative data. [2025-10-20T22:14:25.586461] [INFO ] [MAIN ] Derivative Progress: 1561/1877 symbols completed. SWZ took 1.1s. Estimated remaining: 1m 59s. Rate: 160.0 symbols/min [2025-10-20T22:14:25.586881] [INFO ] [MAIN ] SXC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:25.605887] [DEBUG] [MAIN ] SWX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:25.608023] [INFO ] [MAIN ] SWX: Generated 1794 derivative records from 30791 primary records. [2025-10-20T22:14:25.621156] [DEBUG] [LOCAL ] SWX: Saving derivative data. [2025-10-20T22:14:25.621416] [DEBUG] [MAIN ] SWK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:25.629927] [INFO ] [MAIN ] SWK: Generated 1794 derivative records from 52434 primary records. [2025-10-20T22:14:25.642915] [DEBUG] [LOCAL ] SWK: Saving derivative data. [2025-10-20T22:14:25.655670] [DEBUG] [LOCAL ] SWX: Derivative data saved (1794 records). [2025-10-20T22:14:25.658266] [INFO ] [MAIN ] SWX: Successfully saved derivative data. [2025-10-20T22:14:25.663059] [INFO ] [MAIN ] Derivative Progress: 1562/1877 symbols completed. SWX took 1.1s. Estimated remaining: 1m 58s. Rate: 160.1 symbols/min [2025-10-20T22:14:25.663953] [INFO ] [MAIN ] SXI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:25.680868] [DEBUG] [LOCAL ] SXC: Loaded 41615 existing records from primary storage. [2025-10-20T22:14:25.681874] [INFO ] [MAIN ] SXC: Loaded 41615 primary data records. [2025-10-20T22:14:25.681712] [DEBUG] [LOCAL ] SWK: Derivative data saved (1794 records). [2025-10-20T22:14:25.682032] [INFO ] [MAIN ] SWK: Successfully saved derivative data. [2025-10-20T22:14:25.684503] [INFO ] [MAIN ] Derivative Progress: 1563/1877 symbols completed. SWK took 1.2s. Estimated remaining: 1m 58s. Rate: 160.2 symbols/min [2025-10-20T22:14:25.684683] [INFO ] [MAIN ] SXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:25.767579] [DEBUG] [LOCAL ] SXI: Loaded 12056 existing records from primary storage. [2025-10-20T22:14:25.784404] [INFO ] [MAIN ] SXI: Loaded 12056 primary data records. [2025-10-20T22:14:25.996292] [DEBUG] [LOCAL ] SXT: Loaded 29870 existing records from primary storage. [2025-10-20T22:14:26.007699] [INFO ] [MAIN ] SXT: Loaded 29870 primary data records. [2025-10-20T22:14:26.681810] [DEBUG] [MAIN ] SXC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:26.683425] [INFO ] [MAIN ] SXC: Generated 1794 derivative records from 41615 primary records. [2025-10-20T22:14:26.684506] [DEBUG] [LOCAL ] SXC: Saving derivative data. [2025-10-20T22:14:26.702850] [DEBUG] [LOCAL ] SXC: Derivative data saved (1794 records). [2025-10-20T22:14:26.703045] [INFO ] [MAIN ] SXC: Successfully saved derivative data. [2025-10-20T22:14:26.704953] [INFO ] [MAIN ] Derivative Progress: 1564/1877 symbols completed. SXC took 1.1s. Estimated remaining: 1m 57s. Rate: 160.0 symbols/min [2025-10-20T22:14:26.713247] [INFO ] [MAIN ] SYF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:26.719428] [DEBUG] [MAIN ] SXI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:26.721436] [INFO ] [MAIN ] SXI: Generated 1794 derivative records from 12056 primary records. [2025-10-20T22:14:26.721794] [DEBUG] [LOCAL ] SXI: Saving derivative data. [2025-10-20T22:14:26.740357] [DEBUG] [LOCAL ] SXI: Derivative data saved (1794 records). [2025-10-20T22:14:26.740724] [INFO ] [MAIN ] SXI: Successfully saved derivative data. [2025-10-20T22:14:26.742947] [INFO ] [MAIN ] Derivative Progress: 1565/1877 symbols completed. SXI took 1.1s. Estimated remaining: 1m 57s. Rate: 160.1 symbols/min [2025-10-20T22:14:26.743439] [INFO ] [MAIN ] SYK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:26.762152] [DEBUG] [MAIN ] SXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:26.763009] [INFO ] [MAIN ] SXT: Generated 1794 derivative records from 29870 primary records. [2025-10-20T22:14:26.778393] [DEBUG] [LOCAL ] SXT: Saving derivative data. [2025-10-20T22:14:26.795752] [DEBUG] [LOCAL ] SXT: Derivative data saved (1794 records). [2025-10-20T22:14:26.796090] [INFO ] [MAIN ] SXT: Successfully saved derivative data. [2025-10-20T22:14:26.798372] [INFO ] [MAIN ] Derivative Progress: 1566/1877 symbols completed. SXT took 1.1s. Estimated remaining: 1m 56s. Rate: 160.2 symbols/min [2025-10-20T22:14:26.798772] [INFO ] [MAIN ] SYY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:26.826662] [DEBUG] [LOCAL ] SYF: Loaded 53453 existing records from primary storage. [2025-10-20T22:14:26.826893] [INFO ] [MAIN ] SYF: Loaded 53453 primary data records. [2025-10-20T22:14:27.019180] [DEBUG] [LOCAL ] SYK: Loaded 46886 existing records from primary storage. [2025-10-20T22:14:27.048128] [INFO ] [MAIN ] SYK: Loaded 46886 primary data records. [2025-10-20T22:14:27.235681] [DEBUG] [LOCAL ] SYY: Loaded 53207 existing records from primary storage. [2025-10-20T22:14:27.249499] [INFO ] [MAIN ] SYY: Loaded 53207 primary data records. [2025-10-20T22:14:27.847608] [DEBUG] [MAIN ] SYF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:27.849066] [INFO ] [MAIN ] SYF: Generated 1794 derivative records from 53453 primary records. [2025-10-20T22:14:27.849404] [DEBUG] [LOCAL ] SYF: Saving derivative data. [2025-10-20T22:14:27.866496] [DEBUG] [LOCAL ] SYF: Derivative data saved (1794 records). [2025-10-20T22:14:27.866662] [INFO ] [MAIN ] SYF: Successfully saved derivative data. [2025-10-20T22:14:27.869011] [INFO ] [MAIN ] Derivative Progress: 1567/1877 symbols completed. SYF took 1.2s. Estimated remaining: 1m 56s. Rate: 160.0 symbols/min [2025-10-20T22:14:27.869295] [INFO ] [MAIN ] T: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:27.903278] [DEBUG] [MAIN ] SYK: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:27.909247] [INFO ] [MAIN ] SYK: Generated 1793 derivative records from 46886 primary records. [2025-10-20T22:14:27.909685] [DEBUG] [LOCAL ] SYK: Saving derivative data. [2025-10-20T22:14:27.910410] [DEBUG] [MAIN ] SYY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:27.911596] [INFO ] [MAIN ] SYY: Generated 1794 derivative records from 53207 primary records. [2025-10-20T22:14:27.911981] [DEBUG] [LOCAL ] SYY: Saving derivative data. [2025-10-20T22:14:27.955289] [DEBUG] [LOCAL ] SYK: Derivative data saved (1793 records). [2025-10-20T22:14:27.956912] [INFO ] [MAIN ] SYK: Successfully saved derivative data. [2025-10-20T22:14:27.959060] [INFO ] [MAIN ] Derivative Progress: 1568/1877 symbols completed. SYK took 1.2s. Estimated remaining: 1m 56s. Rate: 160.1 symbols/min [2025-10-20T22:14:27.959344] [INFO ] [MAIN ] TAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:27.956815] [DEBUG] [LOCAL ] SYY: Derivative data saved (1794 records). [2025-10-20T22:14:27.959534] [INFO ] [MAIN ] SYY: Successfully saved derivative data. [2025-10-20T22:14:27.961935] [INFO ] [MAIN ] Derivative Progress: 1569/1877 symbols completed. SYY took 1.2s. Estimated remaining: 1m 55s. Rate: 160.2 symbols/min [2025-10-20T22:14:27.962297] [INFO ] [MAIN ] TAK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:27.982727] [DEBUG] [LOCAL ] T: Loaded 53798 existing records from primary storage. [2025-10-20T22:14:27.983055] [INFO ] [MAIN ] T: Loaded 53798 primary data records. [2025-10-20T22:14:28.310362] [DEBUG] [LOCAL ] TAC: Loaded 43339 existing records from primary storage. [2025-10-20T22:14:28.310565] [INFO ] [MAIN ] TAC: Loaded 43339 primary data records. [2025-10-20T22:14:28.429661] [DEBUG] [LOCAL ] TAK: Loaded 50251 existing records from primary storage. [2025-10-20T22:14:28.434357] [INFO ] [MAIN ] TAK: Loaded 50251 primary data records. [2025-10-20T22:14:28.989112] [DEBUG] [MAIN ] T: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:28.991388] [INFO ] [MAIN ] T: Generated 1794 derivative records from 53798 primary records. [2025-10-20T22:14:28.991831] [DEBUG] [LOCAL ] T: Saving derivative data. [2025-10-20T22:14:29.009936] [DEBUG] [LOCAL ] T: Derivative data saved (1794 records). [2025-10-20T22:14:29.010553] [INFO ] [MAIN ] T: Successfully saved derivative data. [2025-10-20T22:14:29.012808] [INFO ] [MAIN ] Derivative Progress: 1570/1877 symbols completed. T took 1.1s. Estimated remaining: 1m 55s. Rate: 160.0 symbols/min [2025-10-20T22:14:29.012990] [INFO ] [MAIN ] TAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:29.030263] [DEBUG] [MAIN ] TAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:29.031005] [INFO ] [MAIN ] TAC: Generated 1794 derivative records from 43339 primary records. [2025-10-20T22:14:29.031305] [DEBUG] [LOCAL ] TAC: Saving derivative data. [2025-10-20T22:14:29.050369] [DEBUG] [LOCAL ] TAC: Derivative data saved (1794 records). [2025-10-20T22:14:29.050587] [INFO ] [MAIN ] TAC: Successfully saved derivative data. [2025-10-20T22:14:29.052483] [INFO ] [MAIN ] Derivative Progress: 1571/1877 symbols completed. TAC took 1.1s. Estimated remaining: 1m 55s. Rate: 160.1 symbols/min [2025-10-20T22:14:29.052781] [INFO ] [MAIN ] TALO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:29.067072] [DEBUG] [MAIN ] TAK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:29.068016] [INFO ] [MAIN ] TAK: Generated 1794 derivative records from 50251 primary records. [2025-10-20T22:14:29.068165] [DEBUG] [LOCAL ] TAK: Saving derivative data. [2025-10-20T22:14:29.099269] [DEBUG] [LOCAL ] TAK: Derivative data saved (1794 records). [2025-10-20T22:14:29.099578] [INFO ] [MAIN ] TAK: Successfully saved derivative data. [2025-10-20T22:14:29.103163] [INFO ] [MAIN ] Derivative Progress: 1572/1877 symbols completed. TAK took 1.1s. Estimated remaining: 1m 54s. Rate: 160.2 symbols/min [2025-10-20T22:14:29.103525] [INFO ] [MAIN ] TAP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:29.153413] [DEBUG] [LOCAL ] TAL: Loaded 51731 existing records from primary storage. [2025-10-20T22:14:29.153748] [INFO ] [MAIN ] TAL: Loaded 51731 primary data records. [2025-10-20T22:14:29.154481] [DEBUG] [LOCAL ] TALO: Loaded 51031 existing records from primary storage. [2025-10-20T22:14:29.154739] [INFO ] [MAIN ] TALO: Loaded 51031 primary data records. [2025-10-20T22:14:29.770376] [DEBUG] [LOCAL ] TAP: Loaded 52985 existing records from primary storage. [2025-10-20T22:14:29.784724] [INFO ] [MAIN ] TAP: Loaded 52985 primary data records. [2025-10-20T22:14:30.123157] [DEBUG] [MAIN ] TALO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:30.124951] [DEBUG] [MAIN ] TAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:30.126005] [INFO ] [MAIN ] TAL: Generated 1794 derivative records from 51731 primary records. [2025-10-20T22:14:30.126729] [INFO ] [MAIN ] TALO: Generated 1794 derivative records from 51031 primary records. [2025-10-20T22:14:30.126912] [DEBUG] [LOCAL ] TALO: Saving derivative data. [2025-10-20T22:14:30.126985] [DEBUG] [LOCAL ] TAL: Saving derivative data. [2025-10-20T22:14:30.159871] [DEBUG] [LOCAL ] TALO: Derivative data saved (1794 records). [2025-10-20T22:14:30.161137] [INFO ] [MAIN ] TALO: Successfully saved derivative data. [2025-10-20T22:14:30.163776] [INFO ] [MAIN ] Derivative Progress: 1573/1877 symbols completed. TALO took 1.1s. Estimated remaining: 1m 54s. Rate: 160.0 symbols/min [2025-10-20T22:14:30.161418] [DEBUG] [LOCAL ] TAL: Derivative data saved (1794 records). [2025-10-20T22:14:30.164096] [INFO ] [MAIN ] TAL: Successfully saved derivative data. [2025-10-20T22:14:30.166066] [INFO ] [MAIN ] Derivative Progress: 1574/1877 symbols completed. TAL took 1.2s. Estimated remaining: 1m 54s. Rate: 160.1 symbols/min [2025-10-20T22:14:30.182338] [DEBUG] [MAIN ] TAP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:30.182998] [INFO ] [MAIN ] TAP: Generated 1794 derivative records from 52985 primary records. [2025-10-20T22:14:30.183060] [DEBUG] [LOCAL ] TAP: Saving derivative data. [2025-10-20T22:14:30.198103] [DEBUG] [LOCAL ] TAP: Derivative data saved (1794 records). [2025-10-20T22:14:30.198164] [INFO ] [MAIN ] TAP: Successfully saved derivative data. [2025-10-20T22:14:30.200336] [INFO ] [MAIN ] Derivative Progress: 1575/1877 symbols completed. TAP took 1.1s. Estimated remaining: 1m 53s. Rate: 160.2 symbols/min [2025-10-20T22:14:30.200878] [INFO ] [MAIN ] TBBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:30.200809] [INFO ] [MAIN ] TBB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:30.201106] [INFO ] [MAIN ] TBI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:30.232897] [DEBUG] [LOCAL ] TBB: Loaded 11444 existing records from primary storage. [2025-10-20T22:14:30.245630] [INFO ] [MAIN ] TBB: Loaded 11444 primary data records. [2025-10-20T22:14:30.245330] [DEBUG] [LOCAL ] TBI: Loaded 22807 existing records from primary storage. [2025-10-20T22:14:30.246004] [INFO ] [MAIN ] TBI: Loaded 22807 primary data records. [2025-10-20T22:14:30.481159] [DEBUG] [LOCAL ] TBBB: Loaded 35683 existing records from primary storage. [2025-10-20T22:14:30.505758] [INFO ] [MAIN ] TBBB: Loaded 35683 primary data records. [2025-10-20T22:14:31.269716] [DEBUG] [MAIN ] TBI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:31.271362] [INFO ] [MAIN ] TBI: Generated 1794 derivative records from 22807 primary records. [2025-10-20T22:14:31.285602] [DEBUG] [LOCAL ] TBI: Saving derivative data. [2025-10-20T22:14:31.286162] [DEBUG] [MAIN ] TBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:31.286978] [INFO ] [MAIN ] TBB: Generated 1794 derivative records from 11444 primary records. [2025-10-20T22:14:31.287184] [DEBUG] [LOCAL ] TBB: Saving derivative data. [2025-10-20T22:14:31.328210] [DEBUG] [LOCAL ] TBI: Derivative data saved (1794 records). [2025-10-20T22:14:31.324265] [DEBUG] [LOCAL ] TBB: Derivative data saved (1794 records). [2025-10-20T22:14:31.328523] [INFO ] [MAIN ] TBB: Successfully saved derivative data. [2025-10-20T22:14:31.328734] [INFO ] [MAIN ] TBI: Successfully saved derivative data. [2025-10-20T22:14:31.330889] [INFO ] [MAIN ] Derivative Progress: 1576/1877 symbols completed. TBI took 1.1s. Estimated remaining: 1m 53s. Rate: 160.0 symbols/min [2025-10-20T22:14:31.346130] [INFO ] [MAIN ] Derivative Progress: 1577/1877 symbols completed. TBB took 1.1s. Estimated remaining: 1m 52s. Rate: 160.1 symbols/min [2025-10-20T22:14:31.346288] [DEBUG] [MAIN ] TBBB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:31.345867] [INFO ] [MAIN ] TBN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:31.347296] [INFO ] [MAIN ] TBBB: Generated 1794 derivative records from 35683 primary records. [2025-10-20T22:14:31.347355] [DEBUG] [LOCAL ] TBBB: Saving derivative data. [2025-10-20T22:14:31.347906] [INFO ] [MAIN ] TCBX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:31.365296] [DEBUG] [LOCAL ] TBBB: Derivative data saved (1794 records). [2025-10-20T22:14:31.368287] [DEBUG] [LOCAL ] TBN: Loaded 3997 existing records from primary storage. [2025-10-20T22:14:31.368404] [INFO ] [MAIN ] TBN: Loaded 3997 primary data records. [2025-10-20T22:14:31.368540] [INFO ] [MAIN ] TBBB: Successfully saved derivative data. [2025-10-20T22:14:31.382007] [INFO ] [MAIN ] Derivative Progress: 1578/1877 symbols completed. TBBB took 1.2s. Estimated remaining: 1m 52s. Rate: 160.2 symbols/min [2025-10-20T22:14:31.382191] [INFO ] [MAIN ] TD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:31.460947] [DEBUG] [LOCAL ] TCBX: Loaded 10256 existing records from primary storage. [2025-10-20T22:14:31.472377] [INFO ] [MAIN ] TCBX: Loaded 10256 primary data records. [2025-10-20T22:14:32.033263] [DEBUG] [LOCAL ] TD: Loaded 52287 existing records from primary storage. [2025-10-20T22:14:32.033828] [INFO ] [MAIN ] TD: Loaded 52287 primary data records. [2025-10-20T22:14:32.232852] [DEBUG] [MAIN ] TBN: Filtered 7001 bins outside market hours, 7822 incomplete bins [2025-10-20T22:14:32.233947] [INFO ] [MAIN ] TBN: Generated 1792 derivative records from 3997 primary records. [2025-10-20T22:14:32.335406] [DEBUG] [LOCAL ] TBN: Saving derivative data. [2025-10-20T22:14:32.377542] [DEBUG] [LOCAL ] TBN: Derivative data saved (1792 records). [2025-10-20T22:14:32.378536] [INFO ] [MAIN ] TBN: Successfully saved derivative data. [2025-10-20T22:14:32.395809] [INFO ] [MAIN ] Derivative Progress: 1579/1877 symbols completed. TBN took 1.0s. Estimated remaining: 1m 52s. Rate: 160.0 symbols/min [2025-10-20T22:14:32.397240] [INFO ] [MAIN ] TDC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:32.396819] [DEBUG] [MAIN ] TCBX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:32.399023] [INFO ] [MAIN ] TCBX: Generated 1794 derivative records from 10256 primary records. [2025-10-20T22:14:32.399411] [DEBUG] [LOCAL ] TCBX: Saving derivative data. [2025-10-20T22:14:32.418472] [DEBUG] [LOCAL ] TCBX: Derivative data saved (1794 records). [2025-10-20T22:14:32.419059] [INFO ] [MAIN ] TCBX: Successfully saved derivative data. [2025-10-20T22:14:32.422431] [INFO ] [MAIN ] Derivative Progress: 1580/1877 symbols completed. TCBX took 1.1s. Estimated remaining: 1m 51s. Rate: 160.1 symbols/min [2025-10-20T22:14:32.423030] [INFO ] [MAIN ] TDF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:32.443872] [DEBUG] [LOCAL ] TDF: Loaded 6264 existing records from primary storage. [2025-10-20T22:14:32.449332] [INFO ] [MAIN ] TDF: Loaded 6264 primary data records. [2025-10-20T22:14:32.463191] [DEBUG] [MAIN ] TD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:32.494541] [INFO ] [MAIN ] TD: Generated 1794 derivative records from 52287 primary records. [2025-10-20T22:14:32.517545] [DEBUG] [LOCAL ] TD: Saving derivative data. [2025-10-20T22:14:32.624141] [DEBUG] [LOCAL ] TDC: Loaded 46477 existing records from primary storage. [2025-10-20T22:14:32.633423] [DEBUG] [LOCAL ] TD: Derivative data saved (1794 records). [2025-10-20T22:14:32.663040] [INFO ] [MAIN ] TD: Successfully saved derivative data. [2025-10-20T22:14:32.665282] [INFO ] [MAIN ] Derivative Progress: 1581/1877 symbols completed. TD took 1.3s. Estimated remaining: 1m 51s. Rate: 160.1 symbols/min [2025-10-20T22:14:32.638606] [INFO ] [MAIN ] TDC: Loaded 46477 primary data records. [2025-10-20T22:14:32.665441] [INFO ] [MAIN ] TDG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:33.115466] [DEBUG] [LOCAL ] TDG: Loaded 22139 existing records from primary storage. [2025-10-20T22:14:33.175320] [INFO ] [MAIN ] TDG: Loaded 22139 primary data records. [2025-10-20T22:14:33.500240] [DEBUG] [MAIN ] TDF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:33.501409] [INFO ] [MAIN ] TDF: Generated 1794 derivative records from 6264 primary records. [2025-10-20T22:14:33.501511] [DEBUG] [LOCAL ] TDF: Saving derivative data. [2025-10-20T22:14:33.519123] [DEBUG] [LOCAL ] TDF: Derivative data saved (1794 records). [2025-10-20T22:14:33.520225] [INFO ] [MAIN ] TDF: Successfully saved derivative data. [2025-10-20T22:14:33.522372] [INFO ] [MAIN ] Derivative Progress: 1582/1877 symbols completed. TDF took 1.1s. Estimated remaining: 1m 51s. Rate: 160.0 symbols/min [2025-10-20T22:14:33.523682] [INFO ] [MAIN ] TDOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:33.537285] [DEBUG] [MAIN ] TDC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:33.537924] [INFO ] [MAIN ] TDC: Generated 1794 derivative records from 46477 primary records. [2025-10-20T22:14:33.541318] [DEBUG] [LOCAL ] TDC: Saving derivative data. [2025-10-20T22:14:33.564452] [DEBUG] [LOCAL ] TDC: Derivative data saved (1794 records). [2025-10-20T22:14:33.564934] [INFO ] [MAIN ] TDC: Successfully saved derivative data. [2025-10-20T22:14:33.567086] [INFO ] [MAIN ] Derivative Progress: 1583/1877 symbols completed. TDC took 1.2s. Estimated remaining: 1m 50s. Rate: 160.1 symbols/min [2025-10-20T22:14:33.567434] [INFO ] [MAIN ] TDS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:33.598208] [DEBUG] [MAIN ] TDG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:33.598927] [INFO ] [MAIN ] TDG: Generated 1794 derivative records from 22139 primary records. [2025-10-20T22:14:33.599112] [DEBUG] [LOCAL ] TDG: Saving derivative data. [2025-10-20T22:14:33.617400] [DEBUG] [LOCAL ] TDG: Derivative data saved (1794 records). [2025-10-20T22:14:33.617815] [INFO ] [MAIN ] TDG: Successfully saved derivative data. [2025-10-20T22:14:33.620220] [INFO ] [MAIN ] Derivative Progress: 1584/1877 symbols completed. TDG took 1.0s. Estimated remaining: 1m 50s. Rate: 160.2 symbols/min [2025-10-20T22:14:33.636537] [INFO ] [MAIN ] TDW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:33.638176] [DEBUG] [LOCAL ] TDOC: Loaded 53368 existing records from primary storage. [2025-10-20T22:14:33.638295] [INFO ] [MAIN ] TDOC: Loaded 53368 primary data records. [2025-10-20T22:14:33.799788] [DEBUG] [LOCAL ] TDS: Loaded 42428 existing records from primary storage. [2025-10-20T22:14:33.805486] [INFO ] [MAIN ] TDS: Loaded 42428 primary data records. [2025-10-20T22:14:34.096844] [DEBUG] [LOCAL ] TDW: Loaded 45039 existing records from primary storage. [2025-10-20T22:14:34.113123] [INFO ] [MAIN ] TDW: Loaded 45039 primary data records. [2025-10-20T22:14:34.659535] [DEBUG] [MAIN ] TDOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:34.660520] [INFO ] [MAIN ] TDOC: Generated 1794 derivative records from 53368 primary records. [2025-10-20T22:14:34.660718] [DEBUG] [LOCAL ] TDOC: Saving derivative data. [2025-10-20T22:14:34.681020] [DEBUG] [LOCAL ] TDOC: Derivative data saved (1794 records). [2025-10-20T22:14:34.681241] [INFO ] [MAIN ] TDOC: Successfully saved derivative data. [2025-10-20T22:14:34.695809] [INFO ] [MAIN ] Derivative Progress: 1585/1877 symbols completed. TDOC took 1.2s. Estimated remaining: 1m 50s. Rate: 160.0 symbols/min [2025-10-20T22:14:34.696077] [INFO ] [MAIN ] TDY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:34.697241] [DEBUG] [MAIN ] TDS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:34.698186] [INFO ] [MAIN ] TDS: Generated 1794 derivative records from 42428 primary records. [2025-10-20T22:14:34.698376] [DEBUG] [LOCAL ] TDS: Saving derivative data. [2025-10-20T22:14:34.718689] [DEBUG] [LOCAL ] TDS: Derivative data saved (1794 records). [2025-10-20T22:14:34.719097] [INFO ] [MAIN ] TDS: Successfully saved derivative data. [2025-10-20T22:14:34.721399] [INFO ] [MAIN ] Derivative Progress: 1586/1877 symbols completed. TDS took 1.2s. Estimated remaining: 1m 49s. Rate: 160.1 symbols/min [2025-10-20T22:14:34.721697] [INFO ] [MAIN ] TEAF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:34.739859] [DEBUG] [MAIN ] TDW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:34.740458] [INFO ] [MAIN ] TDW: Generated 1794 derivative records from 45039 primary records. [2025-10-20T22:14:34.740636] [DEBUG] [LOCAL ] TDW: Saving derivative data. [2025-10-20T22:14:34.766164] [DEBUG] [LOCAL ] TDY: Loaded 22191 existing records from primary storage. [2025-10-20T22:14:34.765541] [DEBUG] [LOCAL ] TDW: Derivative data saved (1794 records). [2025-10-20T22:14:34.766353] [INFO ] [MAIN ] TDW: Successfully saved derivative data. [2025-10-20T22:14:34.766424] [INFO ] [MAIN ] TDY: Loaded 22191 primary data records. [2025-10-20T22:14:34.768622] [INFO ] [MAIN ] Derivative Progress: 1587/1877 symbols completed. TDW took 1.1s. Estimated remaining: 1m 49s. Rate: 160.1 symbols/min [2025-10-20T22:14:34.790476] [INFO ] [MAIN ] TECK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:34.838672] [DEBUG] [LOCAL ] TEAF: Loaded 7386 existing records from primary storage. [2025-10-20T22:14:34.855469] [INFO ] [MAIN ] TEAF: Loaded 7386 primary data records. [2025-10-20T22:14:35.395469] [DEBUG] [LOCAL ] TECK: Loaded 53435 existing records from primary storage. [2025-10-20T22:14:35.395843] [INFO ] [MAIN ] TECK: Loaded 53435 primary data records. [2025-10-20T22:14:35.714239] [DEBUG] [MAIN ] TDY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:35.715901] [INFO ] [MAIN ] TDY: Generated 1794 derivative records from 22191 primary records. [2025-10-20T22:14:35.716723] [DEBUG] [LOCAL ] TDY: Saving derivative data. [2025-10-20T22:14:35.734817] [DEBUG] [LOCAL ] TDY: Derivative data saved (1794 records). [2025-10-20T22:14:35.735113] [INFO ] [MAIN ] TDY: Successfully saved derivative data. [2025-10-20T22:14:35.747700] [DEBUG] [MAIN ] TEAF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:35.750443] [INFO ] [MAIN ] TEAF: Generated 1794 derivative records from 7386 primary records. [2025-10-20T22:14:35.749579] [INFO ] [MAIN ] Derivative Progress: 1588/1877 symbols completed. TDY took 1.1s. Estimated remaining: 1m 48s. Rate: 160.0 symbols/min [2025-10-20T22:14:35.750544] [DEBUG] [LOCAL ] TEAF: Saving derivative data. [2025-10-20T22:14:35.767260] [DEBUG] [LOCAL ] TEAF: Derivative data saved (1794 records). [2025-10-20T22:14:35.767444] [INFO ] [MAIN ] TEAF: Successfully saved derivative data. [2025-10-20T22:14:35.770272] [INFO ] [MAIN ] Derivative Progress: 1589/1877 symbols completed. TEAF took 1.0s. Estimated remaining: 1m 48s. Rate: 160.1 symbols/min [2025-10-20T22:14:35.786440] [DEBUG] [MAIN ] TECK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:35.787086] [INFO ] [MAIN ] TECK: Generated 1794 derivative records from 53435 primary records. [2025-10-20T22:14:35.787147] [DEBUG] [LOCAL ] TECK: Saving derivative data. [2025-10-20T22:14:35.802118] [DEBUG] [LOCAL ] TECK: Derivative data saved (1794 records). [2025-10-20T22:14:35.802176] [INFO ] [MAIN ] TECK: Successfully saved derivative data. [2025-10-20T22:14:35.805579] [INFO ] [MAIN ] Derivative Progress: 1590/1877 symbols completed. TECK took 1.0s. Estimated remaining: 1m 48s. Rate: 160.2 symbols/min [2025-10-20T22:14:35.806174] [INFO ] [MAIN ] TEF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:35.806485] [INFO ] [MAIN ] TEI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:35.806916] [INFO ] [MAIN ] TEL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:35.841819] [DEBUG] [LOCAL ] TEI: Loaded 21267 existing records from primary storage. [2025-10-20T22:14:35.841995] [INFO ] [MAIN ] TEI: Loaded 21267 primary data records. [2025-10-20T22:14:35.989767] [DEBUG] [LOCAL ] TEF: Loaded 36882 existing records from primary storage. [2025-10-20T22:14:36.016771] [DEBUG] [LOCAL ] TEL: Loaded 51047 existing records from primary storage. [2025-10-20T22:14:36.005509] [INFO ] [MAIN ] TEF: Loaded 36882 primary data records. [2025-10-20T22:14:36.027346] [INFO ] [MAIN ] TEL: Loaded 51047 primary data records. [2025-10-20T22:14:36.848204] [DEBUG] [MAIN ] TEI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:36.851414] [INFO ] [MAIN ] TEI: Generated 1794 derivative records from 21267 primary records. [2025-10-20T22:14:36.851800] [DEBUG] [LOCAL ] TEI: Saving derivative data. [2025-10-20T22:14:36.868677] [DEBUG] [LOCAL ] TEI: Derivative data saved (1794 records). [2025-10-20T22:14:36.868921] [INFO ] [MAIN ] TEI: Successfully saved derivative data. [2025-10-20T22:14:36.871178] [INFO ] [MAIN ] Derivative Progress: 1591/1877 symbols completed. TEI took 1.1s. Estimated remaining: 1m 47s. Rate: 160.0 symbols/min [2025-10-20T22:14:36.871672] [INFO ] [MAIN ] TEN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:36.892820] [DEBUG] [MAIN ] TEL: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:36.906260] [INFO ] [MAIN ] TEL: Generated 1793 derivative records from 51047 primary records. [2025-10-20T22:14:36.906589] [DEBUG] [LOCAL ] TEL: Saving derivative data. [2025-10-20T22:14:36.907025] [DEBUG] [MAIN ] TEF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:36.908602] [INFO ] [MAIN ] TEF: Generated 1794 derivative records from 36882 primary records. [2025-10-20T22:14:36.908981] [DEBUG] [LOCAL ] TEF: Saving derivative data. [2025-10-20T22:14:36.932499] [DEBUG] [LOCAL ] TEL: Derivative data saved (1793 records). [2025-10-20T22:14:36.932937] [INFO ] [MAIN ] TEL: Successfully saved derivative data. [2025-10-20T22:14:36.942335] [INFO ] [MAIN ] Derivative Progress: 1592/1877 symbols completed. TEL took 1.1s. Estimated remaining: 1m 47s. Rate: 160.1 symbols/min [2025-10-20T22:14:36.944058] [INFO ] [MAIN ] TEO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:36.961840] [DEBUG] [LOCAL ] TEF: Derivative data saved (1794 records). [2025-10-20T22:14:36.962088] [INFO ] [MAIN ] TEF: Successfully saved derivative data. [2025-10-20T22:14:36.964301] [INFO ] [MAIN ] Derivative Progress: 1593/1877 symbols completed. TEF took 1.2s. Estimated remaining: 1m 46s. Rate: 160.2 symbols/min [2025-10-20T22:14:36.965232] [DEBUG] [LOCAL ] TEN: Loaded 30597 existing records from primary storage. [2025-10-20T22:14:36.964489] [INFO ] [MAIN ] TEVA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:36.965529] [INFO ] [MAIN ] TEN: Loaded 30597 primary data records. [2025-10-20T22:14:37.164425] [DEBUG] [LOCAL ] TEO: Loaded 22830 existing records from primary storage. [2025-10-20T22:14:37.267628] [INFO ] [MAIN ] TEO: Loaded 22830 primary data records. [2025-10-20T22:14:37.465434] [DEBUG] [LOCAL ] TEVA: Loaded 53622 existing records from primary storage. [2025-10-20T22:14:37.474130] [INFO ] [MAIN ] TEVA: Loaded 53622 primary data records. [2025-10-20T22:14:37.915925] [DEBUG] [MAIN ] TEN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:37.925259] [INFO ] [MAIN ] TEN: Generated 1794 derivative records from 30597 primary records. [2025-10-20T22:14:37.925552] [DEBUG] [LOCAL ] TEN: Saving derivative data. [2025-10-20T22:14:37.943063] [DEBUG] [LOCAL ] TEN: Derivative data saved (1794 records). [2025-10-20T22:14:37.943907] [INFO ] [MAIN ] TEN: Successfully saved derivative data. [2025-10-20T22:14:37.946089] [INFO ] [MAIN ] Derivative Progress: 1594/1877 symbols completed. TEN took 1.1s. Estimated remaining: 1m 46s. Rate: 160.0 symbols/min [2025-10-20T22:14:37.946229] [INFO ] [MAIN ] TEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:37.967044] [DEBUG] [MAIN ] TEO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:37.967882] [INFO ] [MAIN ] TEO: Generated 1794 derivative records from 22830 primary records. [2025-10-20T22:14:37.968237] [DEBUG] [LOCAL ] TEO: Saving derivative data. [2025-10-20T22:14:37.985891] [DEBUG] [LOCAL ] TEO: Derivative data saved (1794 records). [2025-10-20T22:14:37.986907] [INFO ] [MAIN ] TEO: Successfully saved derivative data. [2025-10-20T22:14:37.989879] [INFO ] [MAIN ] Derivative Progress: 1595/1877 symbols completed. TEO took 1.0s. Estimated remaining: 1m 46s. Rate: 160.1 symbols/min [2025-10-20T22:14:37.990150] [INFO ] [MAIN ] TFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:38.013407] [DEBUG] [MAIN ] TEVA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:38.014263] [INFO ] [MAIN ] TEVA: Generated 1794 derivative records from 53622 primary records. [2025-10-20T22:14:38.014410] [DEBUG] [LOCAL ] TEVA: Saving derivative data. [2025-10-20T22:14:38.030336] [DEBUG] [LOCAL ] TEVA: Derivative data saved (1794 records). [2025-10-20T22:14:38.031193] [INFO ] [MAIN ] TEVA: Successfully saved derivative data. [2025-10-20T22:14:38.033571] [INFO ] [MAIN ] Derivative Progress: 1596/1877 symbols completed. TEVA took 1.1s. Estimated remaining: 1m 45s. Rate: 160.2 symbols/min [2025-10-20T22:14:38.046767] [INFO ] [MAIN ] TFII: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:38.048416] [DEBUG] [LOCAL ] TEX: Loaded 43641 existing records from primary storage. [2025-10-20T22:14:38.048726] [INFO ] [MAIN ] TEX: Loaded 43641 primary data records. [2025-10-20T22:14:38.391775] [DEBUG] [LOCAL ] TFC: Loaded 53667 existing records from primary storage. [2025-10-20T22:14:38.390532] [DEBUG] [LOCAL ] TFII: Loaded 32631 existing records from primary storage. [2025-10-20T22:14:38.392055] [INFO ] [MAIN ] TFII: Loaded 32631 primary data records. [2025-10-20T22:14:38.392215] [INFO ] [MAIN ] TFC: Loaded 53667 primary data records. [2025-10-20T22:14:39.049405] [DEBUG] [MAIN ] TEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:39.051055] [INFO ] [MAIN ] TEX: Generated 1794 derivative records from 43641 primary records. [2025-10-20T22:14:39.051556] [DEBUG] [LOCAL ] TEX: Saving derivative data. [2025-10-20T22:14:39.069433] [DEBUG] [LOCAL ] TEX: Derivative data saved (1794 records). [2025-10-20T22:14:39.069845] [INFO ] [MAIN ] TEX: Successfully saved derivative data. [2025-10-20T22:14:39.072320] [INFO ] [MAIN ] Derivative Progress: 1597/1877 symbols completed. TEX took 1.1s. Estimated remaining: 1m 45s. Rate: 160.0 symbols/min [2025-10-20T22:14:39.073385] [INFO ] [MAIN ] TFIN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:39.091257] [DEBUG] [MAIN ] TFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:39.091977] [INFO ] [MAIN ] TFC: Generated 1794 derivative records from 53667 primary records. [2025-10-20T22:14:39.092079] [DEBUG] [LOCAL ] TFC: Saving derivative data. [2025-10-20T22:14:39.115630] [DEBUG] [LOCAL ] TFC: Derivative data saved (1794 records). [2025-10-20T22:14:39.115832] [INFO ] [MAIN ] TFC: Successfully saved derivative data. [2025-10-20T22:14:39.132366] [INFO ] [MAIN ] Derivative Progress: 1598/1877 symbols completed. TFC took 1.1s. Estimated remaining: 1m 45s. Rate: 160.1 symbols/min [2025-10-20T22:14:39.138467] [DEBUG] [MAIN ] TFII: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:39.139207] [INFO ] [MAIN ] TFII: Generated 1794 derivative records from 32631 primary records. [2025-10-20T22:14:39.139311] [INFO ] [MAIN ] TFPM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:39.139737] [DEBUG] [LOCAL ] TFIN: Loaded 23845 existing records from primary storage. [2025-10-20T22:14:39.139889] [DEBUG] [LOCAL ] TFII: Saving derivative data. [2025-10-20T22:14:39.140471] [INFO ] [MAIN ] TFIN: Loaded 23845 primary data records. [2025-10-20T22:14:39.257443] [DEBUG] [LOCAL ] TFII: Derivative data saved (1794 records). [2025-10-20T22:14:39.268245] [INFO ] [MAIN ] TFII: Successfully saved derivative data. [2025-10-20T22:14:39.281037] [INFO ] [MAIN ] Derivative Progress: 1599/1877 symbols completed. TFII took 1.2s. Estimated remaining: 1m 44s. Rate: 160.1 symbols/min [2025-10-20T22:14:39.307821] [INFO ] [MAIN ] TFX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:39.484009] [DEBUG] [LOCAL ] TFPM: Loaded 39797 existing records from primary storage. [2025-10-20T22:14:39.484604] [INFO ] [MAIN ] TFPM: Loaded 39797 primary data records. [2025-10-20T22:14:39.619412] [DEBUG] [LOCAL ] TFX: Loaded 35956 existing records from primary storage. [2025-10-20T22:14:39.620085] [INFO ] [MAIN ] TFX: Loaded 35956 primary data records. [2025-10-20T22:14:39.784011] [DEBUG] [MAIN ] TFIN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:39.921515] [INFO ] [MAIN ] TFIN: Generated 1794 derivative records from 23845 primary records. [2025-10-20T22:14:39.928721] [DEBUG] [LOCAL ] TFIN: Saving derivative data. [2025-10-20T22:14:39.985359] [DEBUG] [LOCAL ] TFIN: Derivative data saved (1794 records). [2025-10-20T22:14:39.985735] [INFO ] [MAIN ] TFIN: Successfully saved derivative data. [2025-10-20T22:14:39.987769] [INFO ] [MAIN ] Derivative Progress: 1600/1877 symbols completed. TFIN took 0.9s. Estimated remaining: 1m 44s. Rate: 160.1 symbols/min [2025-10-20T22:14:39.987905] [INFO ] [MAIN ] TG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:40.173090] [DEBUG] [LOCAL ] TG: Loaded 10704 existing records from primary storage. [2025-10-20T22:14:40.176815] [INFO ] [MAIN ] TG: Loaded 10704 primary data records. [2025-10-20T22:14:40.376233] [DEBUG] [MAIN ] TFPM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:40.524133] [INFO ] [MAIN ] TFPM: Generated 1794 derivative records from 39797 primary records. [2025-10-20T22:14:40.532863] [DEBUG] [LOCAL ] TFPM: Saving derivative data. [2025-10-20T22:14:40.562416] [DEBUG] [LOCAL ] TFPM: Derivative data saved (1794 records). [2025-10-20T22:14:40.567548] [INFO ] [MAIN ] TFPM: Successfully saved derivative data. [2025-10-20T22:14:40.570190] [INFO ] [MAIN ] Derivative Progress: 1601/1877 symbols completed. TFPM took 1.4s. Estimated remaining: 1m 44s. Rate: 160.0 symbols/min [2025-10-20T22:14:40.570630] [DEBUG] [MAIN ] TFX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:40.571607] [INFO ] [MAIN ] TFX: Generated 1794 derivative records from 35956 primary records. [2025-10-20T22:14:40.571757] [DEBUG] [LOCAL ] TFX: Saving derivative data. [2025-10-20T22:14:40.571903] [INFO ] [MAIN ] TGLS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:40.590413] [DEBUG] [LOCAL ] TFX: Derivative data saved (1794 records). [2025-10-20T22:14:40.590828] [INFO ] [MAIN ] TFX: Successfully saved derivative data. [2025-10-20T22:14:40.593039] [INFO ] [MAIN ] Derivative Progress: 1602/1877 symbols completed. TFX took 1.3s. Estimated remaining: 1m 43s. Rate: 160.1 symbols/min [2025-10-20T22:14:40.593485] [INFO ] [MAIN ] TGNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:40.619123] [DEBUG] [MAIN ] TG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:40.619835] [INFO ] [MAIN ] TG: Generated 1794 derivative records from 10704 primary records. [2025-10-20T22:14:40.619953] [DEBUG] [LOCAL ] TG: Saving derivative data. [2025-10-20T22:14:40.635609] [DEBUG] [LOCAL ] TG: Derivative data saved (1794 records). [2025-10-20T22:14:40.644578] [INFO ] [MAIN ] TG: Successfully saved derivative data. [2025-10-20T22:14:40.646898] [INFO ] [MAIN ] Derivative Progress: 1603/1877 symbols completed. TG took 0.7s. Estimated remaining: 1m 43s. Rate: 160.2 symbols/min [2025-10-20T22:14:40.647995] [DEBUG] [LOCAL ] TGLS: Loaded 27616 existing records from primary storage. [2025-10-20T22:14:40.648100] [INFO ] [MAIN ] TGLS: Loaded 27616 primary data records. [2025-10-20T22:14:40.842393] [DEBUG] [LOCAL ] TGNA: Loaded 48621 existing records from primary storage. [2025-10-20T22:14:40.863858] [INFO ] [MAIN ] TGNA: Loaded 48621 primary data records. [2025-10-20T22:14:41.306787] [DEBUG] [MAIN ] TGLS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:41.307868] [INFO ] [MAIN ] TGLS: Generated 1794 derivative records from 27616 primary records. [2025-10-20T22:14:41.307979] [DEBUG] [LOCAL ] TGLS: Saving derivative data. [2025-10-20T22:14:41.327723] [DEBUG] [LOCAL ] TGLS: Derivative data saved (1794 records). [2025-10-20T22:14:41.327879] [INFO ] [MAIN ] TGLS: Successfully saved derivative data. [2025-10-20T22:14:41.330042] [INFO ] [MAIN ] Derivative Progress: 1604/1877 symbols completed. TGLS took 0.8s. Estimated remaining: 1m 42s. Rate: 160.1 symbols/min [2025-10-20T22:14:41.346079] [DEBUG] [MAIN ] TGNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:41.346625] [INFO ] [MAIN ] TGNA: Generated 1794 derivative records from 48621 primary records. [2025-10-20T22:14:41.346707] [DEBUG] [LOCAL ] TGNA: Saving derivative data. [2025-10-20T22:14:41.361418] [DEBUG] [LOCAL ] TGNA: Derivative data saved (1794 records). [2025-10-20T22:14:41.361475] [INFO ] [MAIN ] TGNA: Successfully saved derivative data. [2025-10-20T22:14:41.363599] [INFO ] [MAIN ] Derivative Progress: 1605/1877 symbols completed. TGNA took 0.8s. Estimated remaining: 1m 42s. Rate: 160.2 symbols/min [2025-10-20T22:14:41.364498] [INFO ] [MAIN ] TGT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:41.364441] [INFO ] [MAIN ] TGS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:41.364675] [INFO ] [MAIN ] THC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:41.397361] [DEBUG] [LOCAL ] TGS: Loaded 33811 existing records from primary storage. [2025-10-20T22:14:41.397487] [INFO ] [MAIN ] TGS: Loaded 33811 primary data records. [2025-10-20T22:14:41.612638] [DEBUG] [LOCAL ] THC: Loaded 46820 existing records from primary storage. [2025-10-20T22:14:41.725799] [INFO ] [MAIN ] THC: Loaded 46820 primary data records. [2025-10-20T22:14:41.811978] [DEBUG] [LOCAL ] TGT: Loaded 53777 existing records from primary storage. [2025-10-20T22:14:41.835982] [INFO ] [MAIN ] TGT: Loaded 53777 primary data records. [2025-10-20T22:14:42.371589] [DEBUG] [MAIN ] TGS: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:42.372758] [INFO ] [MAIN ] TGS: Generated 1793 derivative records from 33811 primary records. [2025-10-20T22:14:42.373033] [DEBUG] [LOCAL ] TGS: Saving derivative data. [2025-10-20T22:14:42.391054] [DEBUG] [LOCAL ] TGS: Derivative data saved (1793 records). [2025-10-20T22:14:42.392115] [INFO ] [MAIN ] TGS: Successfully saved derivative data. [2025-10-20T22:14:42.395158] [INFO ] [MAIN ] Derivative Progress: 1606/1877 symbols completed. TGS took 1.0s. Estimated remaining: 1m 42s. Rate: 160.0 symbols/min [2025-10-20T22:14:42.396258] [INFO ] [MAIN ] THG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:42.410051] [DEBUG] [MAIN ] THC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:42.411028] [INFO ] [MAIN ] THC: Generated 1794 derivative records from 46820 primary records. [2025-10-20T22:14:42.411202] [DEBUG] [LOCAL ] THC: Saving derivative data. [2025-10-20T22:14:42.442524] [DEBUG] [LOCAL ] THC: Derivative data saved (1794 records). [2025-10-20T22:14:42.447838] [DEBUG] [MAIN ] TGT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:42.448625] [INFO ] [MAIN ] TGT: Generated 1794 derivative records from 53777 primary records. [2025-10-20T22:14:42.447665] [INFO ] [MAIN ] THC: Successfully saved derivative data. [2025-10-20T22:14:42.448753] [DEBUG] [LOCAL ] TGT: Saving derivative data. [2025-10-20T22:14:42.451405] [INFO ] [MAIN ] Derivative Progress: 1607/1877 symbols completed. THC took 1.1s. Estimated remaining: 1m 41s. Rate: 160.1 symbols/min [2025-10-20T22:14:42.451608] [INFO ] [MAIN ] THIR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:42.474665] [DEBUG] [LOCAL ] TGT: Derivative data saved (1794 records). [2025-10-20T22:14:42.475695] [DEBUG] [LOCAL ] THG: Loaded 21672 existing records from primary storage. [2025-10-20T22:14:42.475341] [INFO ] [MAIN ] TGT: Successfully saved derivative data. [2025-10-20T22:14:42.475913] [INFO ] [MAIN ] THG: Loaded 21672 primary data records. [2025-10-20T22:14:42.478086] [INFO ] [MAIN ] Derivative Progress: 1608/1877 symbols completed. TGT took 1.1s. Estimated remaining: 1m 41s. Rate: 160.2 symbols/min [2025-10-20T22:14:42.491417] [INFO ] [MAIN ] THLV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:42.491579] [DEBUG] [LOCAL ] THIR: Loaded 2407 existing records from primary storage. [2025-10-20T22:14:42.528097] [INFO ] [MAIN ] THIR: Loaded 2407 primary data records. [2025-10-20T22:14:42.612230] [DEBUG] [LOCAL ] THLV: Loaded 1505 existing records from primary storage. [2025-10-20T22:14:42.625234] [INFO ] [MAIN ] THLV: Loaded 1505 primary data records. [2025-10-20T22:14:43.439450] [DEBUG] [MAIN ] THG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:43.440381] [INFO ] [MAIN ] THG: Generated 1794 derivative records from 21672 primary records. [2025-10-20T22:14:43.442902] [DEBUG] [LOCAL ] THG: Saving derivative data. [2025-10-20T22:14:43.485233] [DEBUG] [LOCAL ] THG: Derivative data saved (1794 records). [2025-10-20T22:14:43.485993] [DEBUG] [MAIN ] THIR: Filtered 7001 bins outside market hours, 7831 incomplete bins [2025-10-20T22:14:43.485695] [DEBUG] [MAIN ] THLV: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:43.487381] [INFO ] [MAIN ] THLV: Generated 1793 derivative records from 1505 primary records. [2025-10-20T22:14:43.487520] [INFO ] [MAIN ] THG: Successfully saved derivative data. [2025-10-20T22:14:43.487476] [DEBUG] [LOCAL ] THLV: Saving derivative data. [2025-10-20T22:14:43.489412] [INFO ] [MAIN ] Derivative Progress: 1609/1877 symbols completed. THG took 1.1s. Estimated remaining: 1m 40s. Rate: 160.0 symbols/min [2025-10-20T22:14:43.489604] [INFO ] [MAIN ] THO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:43.486815] [INFO ] [MAIN ] THIR: Generated 1783 derivative records from 2407 primary records. [2025-10-20T22:14:43.489816] [DEBUG] [LOCAL ] THIR: Saving derivative data. [2025-10-20T22:14:43.513927] [DEBUG] [LOCAL ] THLV: Derivative data saved (1793 records). [2025-10-20T22:14:43.516004] [INFO ] [MAIN ] THLV: Successfully saved derivative data. [2025-10-20T22:14:43.520066] [INFO ] [MAIN ] Derivative Progress: 1610/1877 symbols completed. THLV took 1.0s. Estimated remaining: 1m 40s. Rate: 160.1 symbols/min [2025-10-20T22:14:43.520205] [DEBUG] [LOCAL ] THIR: Derivative data saved (1783 records). [2025-10-20T22:14:43.520436] [INFO ] [MAIN ] THIR: Successfully saved derivative data. [2025-10-20T22:14:43.523109] [INFO ] [MAIN ] Derivative Progress: 1611/1877 symbols completed. THIR took 1.1s. Estimated remaining: 1m 40s. Rate: 160.2 symbols/min [2025-10-20T22:14:43.523308] [INFO ] [MAIN ] THR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:43.520526] [INFO ] [MAIN ] THQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:43.558439] [DEBUG] [LOCAL ] THR: Loaded 19630 existing records from primary storage. [2025-10-20T22:14:43.559198] [INFO ] [MAIN ] THR: Loaded 19630 primary data records. [2025-10-20T22:14:43.643763] [DEBUG] [LOCAL ] THO: Loaded 41224 existing records from primary storage. [2025-10-20T22:14:43.619928] [DEBUG] [LOCAL ] THQ: Loaded 25401 existing records from primary storage. [2025-10-20T22:14:43.653119] [INFO ] [MAIN ] THO: Loaded 41224 primary data records. [2025-10-20T22:14:43.665527] [INFO ] [MAIN ] THQ: Loaded 25401 primary data records. [2025-10-20T22:14:44.545789] [DEBUG] [MAIN ] THR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:44.546961] [INFO ] [MAIN ] THR: Generated 1794 derivative records from 19630 primary records. [2025-10-20T22:14:44.559362] [DEBUG] [LOCAL ] THR: Saving derivative data. [2025-10-20T22:14:44.560130] [DEBUG] [MAIN ] THO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:14:44.562020] [INFO ] [MAIN ] THO: Generated 1793 derivative records from 41224 primary records. [2025-10-20T22:14:44.562107] [DEBUG] [LOCAL ] THO: Saving derivative data. [2025-10-20T22:14:44.586554] [DEBUG] [LOCAL ] THR: Derivative data saved (1794 records). [2025-10-20T22:14:44.600322] [INFO ] [MAIN ] THR: Successfully saved derivative data. [2025-10-20T22:14:44.604952] [INFO ] [MAIN ] Derivative Progress: 1612/1877 symbols completed. THR took 1.1s. Estimated remaining: 1m 39s. Rate: 160.0 symbols/min [2025-10-20T22:14:44.608212] [INFO ] [MAIN ] THS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:44.608995] [DEBUG] [MAIN ] THQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:44.610716] [INFO ] [MAIN ] THQ: Generated 1794 derivative records from 25401 primary records. [2025-10-20T22:14:44.611021] [DEBUG] [LOCAL ] THQ: Saving derivative data. [2025-10-20T22:14:44.610911] [DEBUG] [LOCAL ] THO: Derivative data saved (1793 records). [2025-10-20T22:14:44.611534] [INFO ] [MAIN ] THO: Successfully saved derivative data. [2025-10-20T22:14:44.614263] [INFO ] [MAIN ] Derivative Progress: 1613/1877 symbols completed. THO took 1.1s. Estimated remaining: 1m 39s. Rate: 160.1 symbols/min [2025-10-20T22:14:44.614726] [INFO ] [MAIN ] THW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:44.630759] [DEBUG] [LOCAL ] THQ: Derivative data saved (1794 records). [2025-10-20T22:14:44.630957] [INFO ] [MAIN ] THQ: Successfully saved derivative data. [2025-10-20T22:14:44.633266] [INFO ] [MAIN ] Derivative Progress: 1614/1877 symbols completed. THQ took 1.1s. Estimated remaining: 1m 38s. Rate: 160.2 symbols/min [2025-10-20T22:14:44.633564] [INFO ] [MAIN ] TIMB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:44.658035] [DEBUG] [LOCAL ] THW: Loaded 19667 existing records from primary storage. [2025-10-20T22:14:44.658228] [INFO ] [MAIN ] THW: Loaded 19667 primary data records. [2025-10-20T22:14:44.782999] [DEBUG] [LOCAL ] THS: Loaded 38682 existing records from primary storage. [2025-10-20T22:14:44.789276] [INFO ] [MAIN ] THS: Loaded 38682 primary data records. [2025-10-20T22:14:44.872108] [DEBUG] [LOCAL ] TIMB: Loaded 35027 existing records from primary storage. [2025-10-20T22:14:44.885943] [INFO ] [MAIN ] TIMB: Loaded 35027 primary data records. [2025-10-20T22:14:45.653945] [DEBUG] [MAIN ] THW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:45.654917] [INFO ] [MAIN ] THW: Generated 1794 derivative records from 19667 primary records. [2025-10-20T22:14:45.656075] [DEBUG] [LOCAL ] THW: Saving derivative data. [2025-10-20T22:14:45.675462] [DEBUG] [LOCAL ] THW: Derivative data saved (1794 records). [2025-10-20T22:14:45.689828] [INFO ] [MAIN ] THW: Successfully saved derivative data. [2025-10-20T22:14:45.690262] [DEBUG] [MAIN ] TIMB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:45.691029] [INFO ] [MAIN ] TIMB: Generated 1794 derivative records from 35027 primary records. [2025-10-20T22:14:45.693186] [INFO ] [MAIN ] Derivative Progress: 1615/1877 symbols completed. THW took 1.1s. Estimated remaining: 1m 38s. Rate: 160.0 symbols/min [2025-10-20T22:14:45.693404] [INFO ] [MAIN ] TISI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:45.691241] [DEBUG] [LOCAL ] TIMB: Saving derivative data. [2025-10-20T22:14:45.707907] [DEBUG] [MAIN ] THS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:45.719746] [INFO ] [MAIN ] THS: Generated 1794 derivative records from 38682 primary records. [2025-10-20T22:14:45.722862] [DEBUG] [LOCAL ] THS: Saving derivative data. [2025-10-20T22:14:45.724998] [DEBUG] [LOCAL ] TIMB: Derivative data saved (1794 records). [2025-10-20T22:14:45.725147] [INFO ] [MAIN ] TIMB: Successfully saved derivative data. [2025-10-20T22:14:45.728313] [INFO ] [MAIN ] Derivative Progress: 1616/1877 symbols completed. TIMB took 1.1s. Estimated remaining: 1m 38s. Rate: 160.1 symbols/min [2025-10-20T22:14:45.728563] [INFO ] [MAIN ] TIXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:45.731537] [DEBUG] [LOCAL ] TISI: Loaded 3305 existing records from primary storage. [2025-10-20T22:14:45.743158] [INFO ] [MAIN ] TISI: Loaded 3305 primary data records. [2025-10-20T22:14:45.772600] [DEBUG] [LOCAL ] THS: Derivative data saved (1794 records). [2025-10-20T22:14:45.795053] [INFO ] [MAIN ] THS: Successfully saved derivative data. [2025-10-20T22:14:45.817990] [INFO ] [MAIN ] Derivative Progress: 1617/1877 symbols completed. THS took 1.2s. Estimated remaining: 1m 37s. Rate: 160.2 symbols/min [2025-10-20T22:14:45.829937] [INFO ] [MAIN ] TJX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:45.949246] [DEBUG] [LOCAL ] TIXT: Loaded 32585 existing records from primary storage. [2025-10-20T22:14:45.952902] [INFO ] [MAIN ] TIXT: Loaded 32585 primary data records. [2025-10-20T22:14:46.305350] [DEBUG] [LOCAL ] TJX: Loaded 53664 existing records from primary storage. [2025-10-20T22:14:46.307372] [INFO ] [MAIN ] TJX: Loaded 53664 primary data records. [2025-10-20T22:14:46.829107] [DEBUG] [MAIN ] TISI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:46.830366] [INFO ] [MAIN ] TISI: Generated 1794 derivative records from 3305 primary records. [2025-10-20T22:14:46.830691] [DEBUG] [LOCAL ] TISI: Saving derivative data. [2025-10-20T22:14:46.847235] [DEBUG] [LOCAL ] TISI: Derivative data saved (1794 records). [2025-10-20T22:14:46.847980] [INFO ] [MAIN ] TISI: Successfully saved derivative data. [2025-10-20T22:14:46.850151] [INFO ] [MAIN ] Derivative Progress: 1618/1877 symbols completed. TISI took 1.2s. Estimated remaining: 1m 37s. Rate: 160.0 symbols/min [2025-10-20T22:14:46.867631] [DEBUG] [MAIN ] TIXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:46.868436] [INFO ] [MAIN ] TIXT: Generated 1794 derivative records from 32585 primary records. [2025-10-20T22:14:46.868585] [DEBUG] [LOCAL ] TIXT: Saving derivative data. [2025-10-20T22:14:46.885213] [DEBUG] [LOCAL ] TIXT: Derivative data saved (1794 records). [2025-10-20T22:14:46.885515] [INFO ] [MAIN ] TIXT: Successfully saved derivative data. [2025-10-20T22:14:46.887694] [INFO ] [MAIN ] Derivative Progress: 1619/1877 symbols completed. TIXT took 1.2s. Estimated remaining: 1m 37s. Rate: 160.1 symbols/min [2025-10-20T22:14:46.902865] [DEBUG] [MAIN ] TJX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:46.903485] [INFO ] [MAIN ] TJX: Generated 1794 derivative records from 53664 primary records. [2025-10-20T22:14:46.903545] [DEBUG] [LOCAL ] TJX: Saving derivative data. [2025-10-20T22:14:46.918945] [DEBUG] [LOCAL ] TJX: Derivative data saved (1794 records). [2025-10-20T22:14:46.919048] [INFO ] [MAIN ] TJX: Successfully saved derivative data. [2025-10-20T22:14:46.921869] [INFO ] [MAIN ] Derivative Progress: 1620/1877 symbols completed. TJX took 1.1s. Estimated remaining: 1m 36s. Rate: 160.2 symbols/min [2025-10-20T22:14:46.922289] [INFO ] [MAIN ] TK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:46.922395] [INFO ] [MAIN ] TKO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:46.922490] [INFO ] [MAIN ] TKC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:46.976064] [DEBUG] [LOCAL ] TKC: Loaded 39869 existing records from primary storage. [2025-10-20T22:14:46.976283] [INFO ] [MAIN ] TKC: Loaded 39869 primary data records. [2025-10-20T22:14:47.117426] [DEBUG] [LOCAL ] TKO: Loaded 43606 existing records from primary storage. [2025-10-20T22:14:47.105988] [DEBUG] [LOCAL ] TK: Loaded 39202 existing records from primary storage. [2025-10-20T22:14:47.146155] [INFO ] [MAIN ] TK: Loaded 39202 primary data records. [2025-10-20T22:14:47.146027] [INFO ] [MAIN ] TKO: Loaded 43606 primary data records. [2025-10-20T22:14:47.939004] [DEBUG] [MAIN ] TKC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:47.941401] [INFO ] [MAIN ] TKC: Generated 1794 derivative records from 39869 primary records. [2025-10-20T22:14:47.941540] [DEBUG] [LOCAL ] TKC: Saving derivative data. [2025-10-20T22:14:47.960388] [DEBUG] [LOCAL ] TKC: Derivative data saved (1794 records). [2025-10-20T22:14:47.960819] [INFO ] [MAIN ] TKC: Successfully saved derivative data. [2025-10-20T22:14:47.962958] [INFO ] [MAIN ] Derivative Progress: 1621/1877 symbols completed. TKC took 1.0s. Estimated remaining: 1m 36s. Rate: 160.0 symbols/min [2025-10-20T22:14:47.963231] [INFO ] [MAIN ] TKR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:47.984571] [DEBUG] [MAIN ] TK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:47.985496] [INFO ] [MAIN ] TK: Generated 1794 derivative records from 39202 primary records. [2025-10-20T22:14:47.985623] [DEBUG] [LOCAL ] TK: Saving derivative data. [2025-10-20T22:14:48.000360] [DEBUG] [MAIN ] TKO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:48.016149] [DEBUG] [LOCAL ] TK: Derivative data saved (1794 records). [2025-10-20T22:14:48.012925] [INFO ] [MAIN ] TKO: Generated 1794 derivative records from 43606 primary records. [2025-10-20T22:14:48.016268] [INFO ] [MAIN ] TK: Successfully saved derivative data. [2025-10-20T22:14:48.018720] [INFO ] [MAIN ] Derivative Progress: 1622/1877 symbols completed. TK took 1.1s. Estimated remaining: 1m 36s. Rate: 160.1 symbols/min [2025-10-20T22:14:48.019119] [INFO ] [MAIN ] TLK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:48.018815] [DEBUG] [LOCAL ] TKO: Saving derivative data. [2025-10-20T22:14:48.048013] [DEBUG] [LOCAL ] TKO: Derivative data saved (1794 records). [2025-10-20T22:14:48.059901] [INFO ] [MAIN ] TKO: Successfully saved derivative data. [2025-10-20T22:14:48.062822] [INFO ] [MAIN ] Derivative Progress: 1623/1877 symbols completed. TKO took 1.1s. Estimated remaining: 1m 35s. Rate: 160.2 symbols/min [2025-10-20T22:14:48.063213] [INFO ] [MAIN ] TLYS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:48.064548] [DEBUG] [LOCAL ] TKR: Loaded 39466 existing records from primary storage. [2025-10-20T22:14:48.064803] [INFO ] [MAIN ] TKR: Loaded 39466 primary data records. [2025-10-20T22:14:48.199481] [DEBUG] [LOCAL ] TLYS: Loaded 16437 existing records from primary storage. [2025-10-20T22:14:48.200261] [DEBUG] [LOCAL ] TLK: Loaded 38763 existing records from primary storage. [2025-10-20T22:14:48.211043] [INFO ] [MAIN ] TLYS: Loaded 16437 primary data records. [2025-10-20T22:14:48.226628] [INFO ] [MAIN ] TLK: Loaded 38763 primary data records. [2025-10-20T22:14:49.060576] [DEBUG] [MAIN ] TKR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:49.062360] [INFO ] [MAIN ] TKR: Generated 1794 derivative records from 39466 primary records. [2025-10-20T22:14:49.064790] [DEBUG] [LOCAL ] TKR: Saving derivative data. [2025-10-20T22:14:49.080905] [DEBUG] [MAIN ] TLK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:49.089777] [INFO ] [MAIN ] TLK: Generated 1794 derivative records from 38763 primary records. [2025-10-20T22:14:49.090768] [DEBUG] [LOCAL ] TLK: Saving derivative data. [2025-10-20T22:14:49.099927] [DEBUG] [LOCAL ] TKR: Derivative data saved (1794 records). [2025-10-20T22:14:49.109722] [INFO ] [MAIN ] TKR: Successfully saved derivative data. [2025-10-20T22:14:49.112374] [INFO ] [MAIN ] Derivative Progress: 1624/1877 symbols completed. TKR took 1.1s. Estimated remaining: 1m 35s. Rate: 160.0 symbols/min [2025-10-20T22:14:49.118769] [INFO ] [MAIN ] TM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:49.133632] [DEBUG] [MAIN ] TLYS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:49.136246] [DEBUG] [LOCAL ] TLK: Derivative data saved (1794 records). [2025-10-20T22:14:49.136015] [INFO ] [MAIN ] TLYS: Generated 1794 derivative records from 16437 primary records. [2025-10-20T22:14:49.136337] [DEBUG] [LOCAL ] TLYS: Saving derivative data. [2025-10-20T22:14:49.136422] [INFO ] [MAIN ] TLK: Successfully saved derivative data. [2025-10-20T22:14:49.139102] [INFO ] [MAIN ] Derivative Progress: 1625/1877 symbols completed. TLK took 1.1s. Estimated remaining: 1m 34s. Rate: 160.1 symbols/min [2025-10-20T22:14:49.139310] [INFO ] [MAIN ] TME: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:49.156050] [DEBUG] [LOCAL ] TLYS: Derivative data saved (1794 records). [2025-10-20T22:14:49.156356] [INFO ] [MAIN ] TLYS: Successfully saved derivative data. [2025-10-20T22:14:49.158823] [INFO ] [MAIN ] Derivative Progress: 1626/1877 symbols completed. TLYS took 1.1s. Estimated remaining: 1m 34s. Rate: 160.2 symbols/min [2025-10-20T22:14:49.159107] [INFO ] [MAIN ] TMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:49.201246] [DEBUG] [LOCAL ] TM: Loaded 36215 existing records from primary storage. [2025-10-20T22:14:49.201402] [INFO ] [MAIN ] TM: Loaded 36215 primary data records. [2025-10-20T22:14:49.355200] [DEBUG] [LOCAL ] TME: Loaded 53210 existing records from primary storage. [2025-10-20T22:14:49.373497] [INFO ] [MAIN ] TME: Loaded 53210 primary data records. [2025-10-20T22:14:49.584724] [DEBUG] [LOCAL ] TMO: Loaded 52402 existing records from primary storage. [2025-10-20T22:14:49.615290] [INFO ] [MAIN ] TMO: Loaded 52402 primary data records. [2025-10-20T22:14:50.212232] [DEBUG] [MAIN ] TM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:50.213453] [INFO ] [MAIN ] TM: Generated 1794 derivative records from 36215 primary records. [2025-10-20T22:14:50.213676] [DEBUG] [LOCAL ] TM: Saving derivative data. [2025-10-20T22:14:50.230733] [DEBUG] [LOCAL ] TM: Derivative data saved (1794 records). [2025-10-20T22:14:50.230849] [INFO ] [MAIN ] TM: Successfully saved derivative data. [2025-10-20T22:14:50.232816] [INFO ] [MAIN ] Derivative Progress: 1627/1877 symbols completed. TM took 1.1s. Estimated remaining: 1m 34s. Rate: 160.0 symbols/min [2025-10-20T22:14:50.234358] [INFO ] [MAIN ] TNC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:50.251842] [DEBUG] [MAIN ] TME: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:50.252873] [INFO ] [MAIN ] TME: Generated 1794 derivative records from 53210 primary records. [2025-10-20T22:14:50.253140] [DEBUG] [LOCAL ] TME: Saving derivative data. [2025-10-20T22:14:50.275832] [DEBUG] [LOCAL ] TME: Derivative data saved (1794 records). [2025-10-20T22:14:50.275931] [INFO ] [MAIN ] TME: Successfully saved derivative data. [2025-10-20T22:14:50.284410] [INFO ] [MAIN ] Derivative Progress: 1628/1877 symbols completed. TME took 1.1s. Estimated remaining: 1m 33s. Rate: 160.1 symbols/min [2025-10-20T22:14:50.284573] [INFO ] [MAIN ] TNET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:50.282002] [DEBUG] [LOCAL ] TNC: Loaded 14395 existing records from primary storage. [2025-10-20T22:14:50.285381] [INFO ] [MAIN ] TNC: Loaded 14395 primary data records. [2025-10-20T22:14:50.319856] [DEBUG] [MAIN ] TMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:50.331714] [INFO ] [MAIN ] TMO: Generated 1794 derivative records from 52402 primary records. [2025-10-20T22:14:50.337059] [DEBUG] [LOCAL ] TMO: Saving derivative data. [2025-10-20T22:14:50.422521] [DEBUG] [LOCAL ] TMO: Derivative data saved (1794 records). [2025-10-20T22:14:50.430990] [INFO ] [MAIN ] TMO: Successfully saved derivative data. [2025-10-20T22:14:50.439298] [INFO ] [MAIN ] Derivative Progress: 1629/1877 symbols completed. TMO took 1.3s. Estimated remaining: 1m 33s. Rate: 160.2 symbols/min [2025-10-20T22:14:50.444682] [INFO ] [MAIN ] TNK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:50.680888] [DEBUG] [LOCAL ] TNET: Loaded 28275 existing records from primary storage. [2025-10-20T22:14:50.681260] [INFO ] [MAIN ] TNET: Loaded 28275 primary data records. [2025-10-20T22:14:50.881714] [DEBUG] [LOCAL ] TNK: Loaded 35462 existing records from primary storage. [2025-10-20T22:14:50.907686] [INFO ] [MAIN ] TNK: Loaded 35462 primary data records. [2025-10-20T22:14:51.242326] [DEBUG] [MAIN ] TNC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:51.367195] [INFO ] [MAIN ] TNC: Generated 1794 derivative records from 14395 primary records. [2025-10-20T22:14:51.367717] [DEBUG] [LOCAL ] TNC: Saving derivative data. [2025-10-20T22:14:51.386166] [DEBUG] [LOCAL ] TNC: Derivative data saved (1794 records). [2025-10-20T22:14:51.386435] [INFO ] [MAIN ] TNC: Successfully saved derivative data. [2025-10-20T22:14:51.388812] [INFO ] [MAIN ] Derivative Progress: 1630/1877 symbols completed. TNC took 1.2s. Estimated remaining: 1m 33s. Rate: 160.0 symbols/min [2025-10-20T22:14:51.388959] [INFO ] [MAIN ] TNL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:51.413326] [DEBUG] [MAIN ] TNET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:51.414593] [INFO ] [MAIN ] TNET: Generated 1794 derivative records from 28275 primary records. [2025-10-20T22:14:51.414991] [DEBUG] [LOCAL ] TNET: Saving derivative data. [2025-10-20T22:14:51.432254] [DEBUG] [LOCAL ] TNET: Derivative data saved (1794 records). [2025-10-20T22:14:51.442562] [INFO ] [MAIN ] TNET: Successfully saved derivative data. [2025-10-20T22:14:51.444860] [INFO ] [MAIN ] Derivative Progress: 1631/1877 symbols completed. TNET took 1.2s. Estimated remaining: 1m 32s. Rate: 160.1 symbols/min [2025-10-20T22:14:51.445080] [INFO ] [MAIN ] TOL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:51.475299] [DEBUG] [MAIN ] TNK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:51.477743] [INFO ] [MAIN ] TNK: Generated 1794 derivative records from 35462 primary records. [2025-10-20T22:14:51.478795] [DEBUG] [LOCAL ] TNL: Loaded 41798 existing records from primary storage. [2025-10-20T22:14:51.478946] [DEBUG] [LOCAL ] TNK: Saving derivative data. [2025-10-20T22:14:51.479551] [INFO ] [MAIN ] TNL: Loaded 41798 primary data records. [2025-10-20T22:14:51.587708] [DEBUG] [LOCAL ] TNK: Derivative data saved (1794 records). [2025-10-20T22:14:51.611531] [INFO ] [MAIN ] TNK: Successfully saved derivative data. [2025-10-20T22:14:51.646780] [INFO ] [MAIN ] Derivative Progress: 1632/1877 symbols completed. TNK took 1.2s. Estimated remaining: 1m 32s. Rate: 160.1 symbols/min [2025-10-20T22:14:51.657389] [INFO ] [MAIN ] TOST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:51.851042] [DEBUG] [LOCAL ] TOL: Loaded 50629 existing records from primary storage. [2025-10-20T22:14:51.851395] [INFO ] [MAIN ] TOL: Loaded 50629 primary data records. [2025-10-20T22:14:52.200392] [DEBUG] [LOCAL ] TOST: Loaded 53690 existing records from primary storage. [2025-10-20T22:14:52.201184] [INFO ] [MAIN ] TOST: Loaded 53690 primary data records. [2025-10-20T22:14:52.520182] [DEBUG] [MAIN ] TNL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:52.569537] [INFO ] [MAIN ] TNL: Generated 1794 derivative records from 41798 primary records. [2025-10-20T22:14:52.578065] [DEBUG] [LOCAL ] TNL: Saving derivative data. [2025-10-20T22:14:52.601457] [DEBUG] [LOCAL ] TNL: Derivative data saved (1794 records). [2025-10-20T22:14:52.602799] [INFO ] [MAIN ] TNL: Successfully saved derivative data. [2025-10-20T22:14:52.605393] [INFO ] [MAIN ] Derivative Progress: 1633/1877 symbols completed. TNL took 1.2s. Estimated remaining: 1m 32s. Rate: 160.0 symbols/min [2025-10-20T22:14:52.621206] [DEBUG] [MAIN ] TOL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:52.622407] [INFO ] [MAIN ] TOL: Generated 1794 derivative records from 50629 primary records. [2025-10-20T22:14:52.623771] [DEBUG] [LOCAL ] TOL: Saving derivative data. [2025-10-20T22:14:52.645834] [DEBUG] [LOCAL ] TOL: Derivative data saved (1794 records). [2025-10-20T22:14:52.646202] [INFO ] [MAIN ] TOL: Successfully saved derivative data. [2025-10-20T22:14:52.649802] [INFO ] [MAIN ] Derivative Progress: 1634/1877 symbols completed. TOL took 1.2s. Estimated remaining: 1m 31s. Rate: 160.1 symbols/min [2025-10-20T22:14:52.666959] [DEBUG] [MAIN ] TOST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:52.667746] [INFO ] [MAIN ] TOST: Generated 1794 derivative records from 53690 primary records. [2025-10-20T22:14:52.667810] [DEBUG] [LOCAL ] TOST: Saving derivative data. [2025-10-20T22:14:52.683482] [DEBUG] [LOCAL ] TOST: Derivative data saved (1794 records). [2025-10-20T22:14:52.683609] [INFO ] [MAIN ] TOST: Successfully saved derivative data. [2025-10-20T22:14:52.685578] [INFO ] [MAIN ] Derivative Progress: 1635/1877 symbols completed. TOST took 1.0s. Estimated remaining: 1m 31s. Rate: 160.2 symbols/min [2025-10-20T22:14:52.686134] [INFO ] [MAIN ] TPB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:52.686672] [INFO ] [MAIN ] TPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:52.686297] [INFO ] [MAIN ] TPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:52.713586] [DEBUG] [LOCAL ] TPB: Loaded 26151 existing records from primary storage. [2025-10-20T22:14:52.713755] [INFO ] [MAIN ] TPB: Loaded 26151 primary data records. [2025-10-20T22:14:53.045858] [DEBUG] [LOCAL ] TPC: Loaded 39183 existing records from primary storage. [2025-10-20T22:14:53.058981] [INFO ] [MAIN ] TPC: Loaded 39183 primary data records. [2025-10-20T22:14:52.937230] [DEBUG] [LOCAL ] TPH: Loaded 44829 existing records from primary storage. [2025-10-20T22:14:53.079777] [INFO ] [MAIN ] TPH: Loaded 44829 primary data records. [2025-10-20T22:14:53.686943] [DEBUG] [MAIN ] TPB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:53.687920] [INFO ] [MAIN ] TPB: Generated 1794 derivative records from 26151 primary records. [2025-10-20T22:14:53.688211] [DEBUG] [LOCAL ] TPB: Saving derivative data. [2025-10-20T22:14:53.706504] [DEBUG] [LOCAL ] TPB: Derivative data saved (1794 records). [2025-10-20T22:14:53.707406] [INFO ] [MAIN ] TPB: Successfully saved derivative data. [2025-10-20T22:14:53.709438] [INFO ] [MAIN ] Derivative Progress: 1636/1877 symbols completed. TPB took 1.0s. Estimated remaining: 1m 30s. Rate: 160.0 symbols/min [2025-10-20T22:14:53.709789] [INFO ] [MAIN ] TPL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:53.727162] [DEBUG] [MAIN ] TPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:53.729457] [INFO ] [MAIN ] TPC: Generated 1794 derivative records from 39183 primary records. [2025-10-20T22:14:53.729682] [DEBUG] [LOCAL ] TPC: Saving derivative data. [2025-10-20T22:14:53.765818] [DEBUG] [MAIN ] TPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:53.773198] [DEBUG] [LOCAL ] TPL: Loaded 12450 existing records from primary storage. [2025-10-20T22:14:53.772598] [DEBUG] [LOCAL ] TPC: Derivative data saved (1794 records). [2025-10-20T22:14:53.773414] [INFO ] [MAIN ] TPH: Generated 1794 derivative records from 44829 primary records. [2025-10-20T22:14:53.773517] [INFO ] [MAIN ] TPL: Loaded 12450 primary data records. [2025-10-20T22:14:53.773766] [DEBUG] [LOCAL ] TPH: Saving derivative data. [2025-10-20T22:14:53.773611] [INFO ] [MAIN ] TPC: Successfully saved derivative data. [2025-10-20T22:14:53.776171] [INFO ] [MAIN ] Derivative Progress: 1637/1877 symbols completed. TPC took 1.1s. Estimated remaining: 1m 30s. Rate: 160.1 symbols/min [2025-10-20T22:14:53.782029] [INFO ] [MAIN ] TPR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:53.849876] [DEBUG] [LOCAL ] TPH: Derivative data saved (1794 records). [2025-10-20T22:14:53.856369] [INFO ] [MAIN ] TPH: Successfully saved derivative data. [2025-10-20T22:14:53.879940] [INFO ] [MAIN ] Derivative Progress: 1638/1877 symbols completed. TPH took 1.2s. Estimated remaining: 1m 30s. Rate: 160.1 symbols/min [2025-10-20T22:14:53.903011] [INFO ] [MAIN ] TPVG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:54.153100] [DEBUG] [LOCAL ] TPVG: Loaded 27321 existing records from primary storage. [2025-10-20T22:14:54.153422] [INFO ] [MAIN ] TPVG: Loaded 27321 primary data records. [2025-10-20T22:14:54.255096] [DEBUG] [LOCAL ] TPR: Loaded 53371 existing records from primary storage. [2025-10-20T22:14:54.266837] [INFO ] [MAIN ] TPR: Loaded 53371 primary data records. [2025-10-20T22:14:54.838592] [DEBUG] [MAIN ] TPL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:54.839793] [INFO ] [MAIN ] TPL: Generated 1794 derivative records from 12450 primary records. [2025-10-20T22:14:54.840035] [DEBUG] [LOCAL ] TPL: Saving derivative data. [2025-10-20T22:14:54.861601] [DEBUG] [LOCAL ] TPL: Derivative data saved (1794 records). [2025-10-20T22:14:54.862141] [INFO ] [MAIN ] TPL: Successfully saved derivative data. [2025-10-20T22:14:54.864094] [INFO ] [MAIN ] Derivative Progress: 1639/1877 symbols completed. TPL took 1.2s. Estimated remaining: 1m 29s. Rate: 160.0 symbols/min [2025-10-20T22:14:54.866028] [INFO ] [MAIN ] TPYP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:54.883050] [DEBUG] [MAIN ] TPVG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:54.883818] [INFO ] [MAIN ] TPVG: Generated 1794 derivative records from 27321 primary records. [2025-10-20T22:14:54.886171] [DEBUG] [LOCAL ] TPVG: Saving derivative data. [2025-10-20T22:14:54.907100] [DEBUG] [LOCAL ] TPYP: Loaded 7931 existing records from primary storage. [2025-10-20T22:14:54.908343] [INFO ] [MAIN ] TPYP: Loaded 7931 primary data records. [2025-10-20T22:14:54.909281] [DEBUG] [LOCAL ] TPVG: Derivative data saved (1794 records). [2025-10-20T22:14:54.909405] [INFO ] [MAIN ] TPVG: Successfully saved derivative data. [2025-10-20T22:14:54.943391] [INFO ] [MAIN ] Derivative Progress: 1640/1877 symbols completed. TPVG took 1.0s. Estimated remaining: 1m 29s. Rate: 160.1 symbols/min [2025-10-20T22:14:54.956455] [INFO ] [MAIN ] TPZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:55.004721] [DEBUG] [MAIN ] TPR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:55.022096] [INFO ] [MAIN ] TPR: Generated 1794 derivative records from 53371 primary records. [2025-10-20T22:14:55.030920] [DEBUG] [LOCAL ] TPR: Saving derivative data. [2025-10-20T22:14:55.073386] [DEBUG] [LOCAL ] TPZ: Loaded 4653 existing records from primary storage. [2025-10-20T22:14:55.094125] [DEBUG] [LOCAL ] TPR: Derivative data saved (1794 records). [2025-10-20T22:14:55.102566] [INFO ] [MAIN ] TPZ: Loaded 4653 primary data records. [2025-10-20T22:14:55.130266] [INFO ] [MAIN ] TPR: Successfully saved derivative data. [2025-10-20T22:14:55.287124] [INFO ] [MAIN ] Derivative Progress: 1641/1877 symbols completed. TPR took 1.5s. Estimated remaining: 1m 28s. Rate: 160.1 symbols/min [2025-10-20T22:14:55.310638] [INFO ] [MAIN ] TR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:55.568129] [DEBUG] [LOCAL ] TR: Loaded 19094 existing records from primary storage. [2025-10-20T22:14:55.568700] [INFO ] [MAIN ] TR: Loaded 19094 primary data records. [2025-10-20T22:14:55.769149] [DEBUG] [MAIN ] TPYP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:55.900009] [INFO ] [MAIN ] TPYP: Generated 1794 derivative records from 7931 primary records. [2025-10-20T22:14:55.900383] [DEBUG] [LOCAL ] TPYP: Saving derivative data. [2025-10-20T22:14:55.917411] [DEBUG] [LOCAL ] TPYP: Derivative data saved (1794 records). [2025-10-20T22:14:55.918870] [INFO ] [MAIN ] TPYP: Successfully saved derivative data. [2025-10-20T22:14:55.922974] [INFO ] [MAIN ] Derivative Progress: 1642/1877 symbols completed. TPYP took 1.1s. Estimated remaining: 1m 28s. Rate: 160.0 symbols/min [2025-10-20T22:14:55.924819] [INFO ] [MAIN ] TRAK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:55.940538] [DEBUG] [MAIN ] TPZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:55.941537] [INFO ] [MAIN ] TPZ: Generated 1794 derivative records from 4653 primary records. [2025-10-20T22:14:55.941911] [DEBUG] [LOCAL ] TPZ: Saving derivative data. [2025-10-20T22:14:55.964317] [DEBUG] [LOCAL ] TRAK: Loaded 9660 existing records from primary storage. [2025-10-20T22:14:55.965580] [INFO ] [MAIN ] TRAK: Loaded 9660 primary data records. [2025-10-20T22:14:55.980502] [DEBUG] [LOCAL ] TPZ: Derivative data saved (1794 records). [2025-10-20T22:14:56.006617] [INFO ] [MAIN ] TPZ: Successfully saved derivative data. [2025-10-20T22:14:56.021492] [INFO ] [MAIN ] Derivative Progress: 1643/1877 symbols completed. TPZ took 1.1s. Estimated remaining: 1m 28s. Rate: 160.1 symbols/min [2025-10-20T22:14:56.062403] [INFO ] [MAIN ] TRC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:56.087552] [DEBUG] [MAIN ] TR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:56.091866] [INFO ] [MAIN ] TR: Generated 1794 derivative records from 19094 primary records. [2025-10-20T22:14:56.092087] [DEBUG] [LOCAL ] TR: Saving derivative data. [2025-10-20T22:14:56.180182] [DEBUG] [LOCAL ] TR: Derivative data saved (1794 records). [2025-10-20T22:14:56.200332] [INFO ] [MAIN ] TR: Successfully saved derivative data. [2025-10-20T22:14:56.202491] [INFO ] [MAIN ] Derivative Progress: 1644/1877 symbols completed. TR took 0.9s. Estimated remaining: 1m 27s. Rate: 160.1 symbols/min [2025-10-20T22:14:56.321171] [DEBUG] [LOCAL ] TRC: Loaded 15518 existing records from primary storage. [2025-10-20T22:14:56.321296] [INFO ] [MAIN ] TRC: Loaded 15518 primary data records. [2025-10-20T22:14:56.321639] [INFO ] [MAIN ] TREX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:56.703518] [DEBUG] [LOCAL ] TREX: Loaded 49841 existing records from primary storage. [2025-10-20T22:14:56.703694] [INFO ] [MAIN ] TREX: Loaded 49841 primary data records. [2025-10-20T22:14:56.703415] [DEBUG] [MAIN ] TRAK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:56.709748] [INFO ] [MAIN ] TRAK: Generated 1794 derivative records from 9660 primary records. [2025-10-20T22:14:56.716687] [DEBUG] [LOCAL ] TRAK: Saving derivative data. [2025-10-20T22:14:56.856388] [DEBUG] [LOCAL ] TRAK: Derivative data saved (1794 records). [2025-10-20T22:14:56.873551] [INFO ] [MAIN ] TRAK: Successfully saved derivative data. [2025-10-20T22:14:56.915250] [INFO ] [MAIN ] Derivative Progress: 1645/1877 symbols completed. TRAK took 1.0s. Estimated remaining: 1m 27s. Rate: 160.0 symbols/min [2025-10-20T22:14:57.034530] [INFO ] [MAIN ] TRGP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:57.062871] [DEBUG] [MAIN ] TRC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:57.064289] [INFO ] [MAIN ] TRC: Generated 1794 derivative records from 15518 primary records. [2025-10-20T22:14:57.064474] [DEBUG] [LOCAL ] TRC: Saving derivative data. [2025-10-20T22:14:57.082307] [DEBUG] [LOCAL ] TRC: Derivative data saved (1794 records). [2025-10-20T22:14:57.082569] [INFO ] [MAIN ] TRC: Successfully saved derivative data. [2025-10-20T22:14:57.084990] [INFO ] [MAIN ] Derivative Progress: 1646/1877 symbols completed. TRC took 1.0s. Estimated remaining: 1m 27s. Rate: 160.1 symbols/min [2025-10-20T22:14:57.085274] [INFO ] [MAIN ] TRN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:57.126267] [DEBUG] [MAIN ] TREX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:57.126964] [INFO ] [MAIN ] TREX: Generated 1794 derivative records from 49841 primary records. [2025-10-20T22:14:57.127024] [DEBUG] [LOCAL ] TREX: Saving derivative data. [2025-10-20T22:14:57.150059] [DEBUG] [LOCAL ] TREX: Derivative data saved (1794 records). [2025-10-20T22:14:57.150835] [INFO ] [MAIN ] TREX: Successfully saved derivative data. [2025-10-20T22:14:57.152989] [INFO ] [MAIN ] Derivative Progress: 1647/1877 symbols completed. TREX took 0.8s. Estimated remaining: 1m 26s. Rate: 160.2 symbols/min [2025-10-20T22:14:57.153211] [INFO ] [MAIN ] TRNO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:57.180750] [DEBUG] [LOCAL ] TRGP: Loaded 50800 existing records from primary storage. [2025-10-20T22:14:57.181536] [DEBUG] [LOCAL ] TRN: Loaded 36521 existing records from primary storage. [2025-10-20T22:14:57.181707] [INFO ] [MAIN ] TRN: Loaded 36521 primary data records. [2025-10-20T22:14:57.181035] [INFO ] [MAIN ] TRGP: Loaded 50800 primary data records. [2025-10-20T22:14:57.710758] [DEBUG] [LOCAL ] TRNO: Loaded 43861 existing records from primary storage. [2025-10-20T22:14:57.838358] [INFO ] [MAIN ] TRNO: Loaded 43861 primary data records. [2025-10-20T22:14:58.176840] [DEBUG] [MAIN ] TRN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:58.177329] [DEBUG] [MAIN ] TRGP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:58.178887] [INFO ] [MAIN ] TRN: Generated 1794 derivative records from 36521 primary records. [2025-10-20T22:14:58.178164] [INFO ] [MAIN ] TRGP: Generated 1794 derivative records from 50800 primary records. [2025-10-20T22:14:58.179053] [DEBUG] [LOCAL ] TRGP: Saving derivative data. [2025-10-20T22:14:58.179567] [DEBUG] [LOCAL ] TRN: Saving derivative data. [2025-10-20T22:14:58.204069] [DEBUG] [LOCAL ] TRGP: Derivative data saved (1794 records). [2025-10-20T22:14:58.211128] [INFO ] [MAIN ] TRGP: Successfully saved derivative data. [2025-10-20T22:14:58.213966] [INFO ] [MAIN ] Derivative Progress: 1648/1877 symbols completed. TRGP took 1.2s. Estimated remaining: 1m 26s. Rate: 160.0 symbols/min [2025-10-20T22:14:58.214517] [DEBUG] [LOCAL ] TRN: Derivative data saved (1794 records). [2025-10-20T22:14:58.215120] [INFO ] [MAIN ] TRN: Successfully saved derivative data. [2025-10-20T22:14:58.217139] [INFO ] [MAIN ] Derivative Progress: 1649/1877 symbols completed. TRN took 1.1s. Estimated remaining: 1m 25s. Rate: 160.1 symbols/min [2025-10-20T22:14:58.233267] [DEBUG] [MAIN ] TRNO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:58.233831] [INFO ] [MAIN ] TRNO: Generated 1794 derivative records from 43861 primary records. [2025-10-20T22:14:58.233889] [DEBUG] [LOCAL ] TRNO: Saving derivative data. [2025-10-20T22:14:58.248571] [DEBUG] [LOCAL ] TRNO: Derivative data saved (1794 records). [2025-10-20T22:14:58.248628] [INFO ] [MAIN ] TRNO: Successfully saved derivative data. [2025-10-20T22:14:58.250527] [INFO ] [MAIN ] Derivative Progress: 1650/1877 symbols completed. TRNO took 1.1s. Estimated remaining: 1m 25s. Rate: 160.2 symbols/min [2025-10-20T22:14:58.250958] [INFO ] [MAIN ] TROX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:58.251404] [INFO ] [MAIN ] TRP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:58.251229] [INFO ] [MAIN ] TRU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:58.324964] [DEBUG] [LOCAL ] TROX: Loaded 52248 existing records from primary storage. [2025-10-20T22:14:58.326220] [INFO ] [MAIN ] TROX: Loaded 52248 primary data records. [2025-10-20T22:14:58.433472] [DEBUG] [LOCAL ] TRP: Loaded 52150 existing records from primary storage. [2025-10-20T22:14:58.457397] [INFO ] [MAIN ] TRP: Loaded 52150 primary data records. [2025-10-20T22:14:58.458139] [DEBUG] [LOCAL ] TRU: Loaded 51460 existing records from primary storage. [2025-10-20T22:14:58.503712] [INFO ] [MAIN ] TRU: Loaded 51460 primary data records. [2025-10-20T22:14:59.338475] [DEBUG] [MAIN ] TROX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:59.349383] [INFO ] [MAIN ] TROX: Generated 1794 derivative records from 52248 primary records. [2025-10-20T22:14:59.349737] [DEBUG] [LOCAL ] TROX: Saving derivative data. [2025-10-20T22:14:59.355801] [DEBUG] [MAIN ] TRP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:59.357260] [INFO ] [MAIN ] TRP: Generated 1794 derivative records from 52150 primary records. [2025-10-20T22:14:59.368972] [DEBUG] [LOCAL ] TRP: Saving derivative data. [2025-10-20T22:14:59.373590] [DEBUG] [LOCAL ] TROX: Derivative data saved (1794 records). [2025-10-20T22:14:59.374078] [INFO ] [MAIN ] TROX: Successfully saved derivative data. [2025-10-20T22:14:59.377738] [INFO ] [MAIN ] Derivative Progress: 1651/1877 symbols completed. TROX took 1.1s. Estimated remaining: 1m 25s. Rate: 160.0 symbols/min [2025-10-20T22:14:59.385307] [INFO ] [MAIN ] TRV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:59.407358] [DEBUG] [LOCAL ] TRP: Derivative data saved (1794 records). [2025-10-20T22:14:59.407571] [INFO ] [MAIN ] TRP: Successfully saved derivative data. [2025-10-20T22:14:59.409762] [INFO ] [MAIN ] Derivative Progress: 1652/1877 symbols completed. TRP took 1.2s. Estimated remaining: 1m 24s. Rate: 160.1 symbols/min [2025-10-20T22:14:59.410377] [INFO ] [MAIN ] TS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:59.410160] [DEBUG] [MAIN ] TRU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:14:59.411466] [INFO ] [MAIN ] TRU: Generated 1794 derivative records from 51460 primary records. [2025-10-20T22:14:59.411590] [DEBUG] [LOCAL ] TRU: Saving derivative data. [2025-10-20T22:14:59.430887] [DEBUG] [LOCAL ] TRU: Derivative data saved (1794 records). [2025-10-20T22:14:59.431289] [INFO ] [MAIN ] TRU: Successfully saved derivative data. [2025-10-20T22:14:59.433570] [INFO ] [MAIN ] Derivative Progress: 1653/1877 symbols completed. TRU took 1.2s. Estimated remaining: 1m 24s. Rate: 160.2 symbols/min [2025-10-20T22:14:59.433763] [INFO ] [MAIN ] TSE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:14:59.487697] [DEBUG] [LOCAL ] TRV: Loaded 47484 existing records from primary storage. [2025-10-20T22:14:59.488017] [INFO ] [MAIN ] TRV: Loaded 47484 primary data records. [2025-10-20T22:14:59.591779] [DEBUG] [LOCAL ] TS: Loaded 51678 existing records from primary storage. [2025-10-20T22:14:59.597192] [INFO ] [MAIN ] TS: Loaded 51678 primary data records. [2025-10-20T22:14:59.645964] [DEBUG] [LOCAL ] TSE: Loaded 27846 existing records from primary storage. [2025-10-20T22:14:59.715920] [INFO ] [MAIN ] TSE: Loaded 27846 primary data records. [2025-10-20T22:15:00.514872] [DEBUG] [MAIN ] TRV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:00.515965] [INFO ] [MAIN ] TRV: Generated 1794 derivative records from 47484 primary records. [2025-10-20T22:15:00.516077] [DEBUG] [LOCAL ] TRV: Saving derivative data. [2025-10-20T22:15:00.539294] [DEBUG] [LOCAL ] TRV: Derivative data saved (1794 records). [2025-10-20T22:15:00.539696] [INFO ] [MAIN ] TRV: Successfully saved derivative data. [2025-10-20T22:15:00.541488] [INFO ] [MAIN ] Derivative Progress: 1654/1877 symbols completed. TRV took 1.2s. Estimated remaining: 1m 24s. Rate: 160.0 symbols/min [2025-10-20T22:15:00.550123] [INFO ] [MAIN ] TSI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:00.570114] [DEBUG] [MAIN ] TS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:00.571105] [DEBUG] [MAIN ] TSE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:00.572127] [INFO ] [MAIN ] TS: Generated 1794 derivative records from 51678 primary records. [2025-10-20T22:15:00.573225] [INFO ] [MAIN ] TSE: Generated 1794 derivative records from 27846 primary records. [2025-10-20T22:15:00.573587] [DEBUG] [LOCAL ] TSE: Saving derivative data. [2025-10-20T22:15:00.573377] [DEBUG] [LOCAL ] TS: Saving derivative data. [2025-10-20T22:15:00.605125] [DEBUG] [LOCAL ] TS: Derivative data saved (1794 records). [2025-10-20T22:15:00.609289] [INFO ] [MAIN ] TS: Successfully saved derivative data. [2025-10-20T22:15:00.611444] [INFO ] [MAIN ] Derivative Progress: 1655/1877 symbols completed. TS took 1.2s. Estimated remaining: 1m 23s. Rate: 160.1 symbols/min [2025-10-20T22:15:00.611931] [DEBUG] [LOCAL ] TSE: Derivative data saved (1794 records). [2025-10-20T22:15:00.612188] [INFO ] [MAIN ] TSE: Successfully saved derivative data. [2025-10-20T22:15:00.612320] [INFO ] [MAIN ] TSLX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:00.614509] [INFO ] [MAIN ] Derivative Progress: 1656/1877 symbols completed. TSE took 1.2s. Estimated remaining: 1m 23s. Rate: 160.1 symbols/min [2025-10-20T22:15:00.619060] [INFO ] [MAIN ] TSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:00.619386] [DEBUG] [LOCAL ] TSI: Loaded 8213 existing records from primary storage. [2025-10-20T22:15:00.619549] [INFO ] [MAIN ] TSI: Loaded 8213 primary data records. [2025-10-20T22:15:00.935491] [DEBUG] [LOCAL ] TSLX: Loaded 35864 existing records from primary storage. [2025-10-20T22:15:00.949545] [INFO ] [MAIN ] TSLX: Loaded 35864 primary data records. [2025-10-20T22:15:01.090033] [DEBUG] [LOCAL ] TSM: Loaded 53819 existing records from primary storage. [2025-10-20T22:15:01.095538] [INFO ] [MAIN ] TSM: Loaded 53819 primary data records. [2025-10-20T22:15:01.509520] [DEBUG] [MAIN ] TSI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:01.517812] [INFO ] [MAIN ] TSI: Generated 1794 derivative records from 8213 primary records. [2025-10-20T22:15:01.622899] [DEBUG] [LOCAL ] TSI: Saving derivative data. [2025-10-20T22:15:01.648883] [DEBUG] [LOCAL ] TSI: Derivative data saved (1794 records). [2025-10-20T22:15:01.649722] [INFO ] [MAIN ] TSI: Successfully saved derivative data. [2025-10-20T22:15:01.651538] [INFO ] [MAIN ] Derivative Progress: 1657/1877 symbols completed. TSI took 1.1s. Estimated remaining: 1m 23s. Rate: 160.0 symbols/min [2025-10-20T22:15:01.652336] [INFO ] [MAIN ] TSN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:01.670719] [DEBUG] [MAIN ] TSLX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:01.671983] [INFO ] [MAIN ] TSLX: Generated 1794 derivative records from 35864 primary records. [2025-10-20T22:15:01.672127] [DEBUG] [LOCAL ] TSLX: Saving derivative data. [2025-10-20T22:15:01.689511] [DEBUG] [LOCAL ] TSLX: Derivative data saved (1794 records). [2025-10-20T22:15:01.690008] [INFO ] [MAIN ] TSLX: Successfully saved derivative data. [2025-10-20T22:15:01.691918] [INFO ] [MAIN ] Derivative Progress: 1658/1877 symbols completed. TSLX took 1.1s. Estimated remaining: 1m 22s. Rate: 160.1 symbols/min [2025-10-20T22:15:01.692479] [INFO ] [MAIN ] TSQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:01.712313] [DEBUG] [MAIN ] TSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:01.713165] [INFO ] [MAIN ] TSM: Generated 1794 derivative records from 53819 primary records. [2025-10-20T22:15:01.713274] [DEBUG] [LOCAL ] TSM: Saving derivative data. [2025-10-20T22:15:01.744756] [DEBUG] [LOCAL ] TSQ: Loaded 11603 existing records from primary storage. [2025-10-20T22:15:01.748056] [DEBUG] [LOCAL ] TSM: Derivative data saved (1794 records). [2025-10-20T22:15:01.748227] [INFO ] [MAIN ] TSM: Successfully saved derivative data. [2025-10-20T22:15:01.747409] [INFO ] [MAIN ] TSQ: Loaded 11603 primary data records. [2025-10-20T22:15:01.750252] [INFO ] [MAIN ] Derivative Progress: 1659/1877 symbols completed. TSM took 1.1s. Estimated remaining: 1m 22s. Rate: 160.1 symbols/min [2025-10-20T22:15:01.771845] [INFO ] [MAIN ] TT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:01.848973] [DEBUG] [LOCAL ] TSN: Loaded 53079 existing records from primary storage. [2025-10-20T22:15:01.870308] [INFO ] [MAIN ] TSN: Loaded 53079 primary data records. [2025-10-20T22:15:02.435906] [DEBUG] [LOCAL ] TT: Loaded 47092 existing records from primary storage. [2025-10-20T22:15:02.436527] [INFO ] [MAIN ] TT: Loaded 47092 primary data records. [2025-10-20T22:15:02.830454] [DEBUG] [MAIN ] TSQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:02.831605] [INFO ] [MAIN ] TSQ: Generated 1794 derivative records from 11603 primary records. [2025-10-20T22:15:02.831921] [DEBUG] [LOCAL ] TSQ: Saving derivative data. [2025-10-20T22:15:02.847889] [DEBUG] [MAIN ] TSN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:02.856823] [INFO ] [MAIN ] TSN: Generated 1794 derivative records from 53079 primary records. [2025-10-20T22:15:02.857999] [DEBUG] [LOCAL ] TSN: Saving derivative data. [2025-10-20T22:15:02.874514] [DEBUG] [LOCAL ] TSQ: Derivative data saved (1794 records). [2025-10-20T22:15:02.878402] [INFO ] [MAIN ] TSQ: Successfully saved derivative data. [2025-10-20T22:15:02.882627] [INFO ] [MAIN ] Derivative Progress: 1660/1877 symbols completed. TSQ took 1.2s. Estimated remaining: 1m 21s. Rate: 160.0 symbols/min [2025-10-20T22:15:02.883321] [INFO ] [MAIN ] TTC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:02.886332] [DEBUG] [LOCAL ] TSN: Derivative data saved (1794 records). [2025-10-20T22:15:02.886482] [INFO ] [MAIN ] TSN: Successfully saved derivative data. [2025-10-20T22:15:02.888801] [INFO ] [MAIN ] Derivative Progress: 1661/1877 symbols completed. TSN took 1.2s. Estimated remaining: 1m 21s. Rate: 160.0 symbols/min [2025-10-20T22:15:02.889475] [INFO ] [MAIN ] TTE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:02.908103] [DEBUG] [MAIN ] TT: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:02.908877] [INFO ] [MAIN ] TT: Generated 1793 derivative records from 47092 primary records. [2025-10-20T22:15:02.909063] [DEBUG] [LOCAL ] TT: Saving derivative data. [2025-10-20T22:15:02.927597] [DEBUG] [LOCAL ] TT: Derivative data saved (1793 records). [2025-10-20T22:15:02.938441] [INFO ] [MAIN ] TT: Successfully saved derivative data. [2025-10-20T22:15:02.940949] [INFO ] [MAIN ] Derivative Progress: 1662/1877 symbols completed. TT took 1.2s. Estimated remaining: 1m 21s. Rate: 160.1 symbols/min [2025-10-20T22:15:02.941301] [INFO ] [MAIN ] TTI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:02.972357] [DEBUG] [LOCAL ] TTC: Loaded 41748 existing records from primary storage. [2025-10-20T22:15:02.972618] [INFO ] [MAIN ] TTC: Loaded 41748 primary data records. [2025-10-20T22:15:03.102816] [DEBUG] [LOCAL ] TTE: Loaded 51594 existing records from primary storage. [2025-10-20T22:15:03.103004] [INFO ] [MAIN ] TTE: Loaded 51594 primary data records. [2025-10-20T22:15:03.449112] [DEBUG] [LOCAL ] TTI: Loaded 46578 existing records from primary storage. [2025-10-20T22:15:03.455979] [INFO ] [MAIN ] TTI: Loaded 46578 primary data records. [2025-10-20T22:15:04.000396] [DEBUG] [MAIN ] TTC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:04.001889] [INFO ] [MAIN ] TTC: Generated 1794 derivative records from 41748 primary records. [2025-10-20T22:15:04.002211] [DEBUG] [LOCAL ] TTC: Saving derivative data. [2025-10-20T22:15:04.023030] [DEBUG] [LOCAL ] TTC: Derivative data saved (1794 records). [2025-10-20T22:15:04.023415] [INFO ] [MAIN ] TTC: Successfully saved derivative data. [2025-10-20T22:15:04.025917] [INFO ] [MAIN ] Derivative Progress: 1663/1877 symbols completed. TTC took 1.1s. Estimated remaining: 1m 20s. Rate: 159.9 symbols/min [2025-10-20T22:15:04.041104] [DEBUG] [MAIN ] TTE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:04.042186] [INFO ] [MAIN ] TTE: Generated 1794 derivative records from 51594 primary records. [2025-10-20T22:15:04.054706] [DEBUG] [LOCAL ] TTE: Saving derivative data. [2025-10-20T22:15:04.055034] [DEBUG] [MAIN ] TTI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:04.056797] [INFO ] [MAIN ] TTI: Generated 1794 derivative records from 46578 primary records. [2025-10-20T22:15:04.056886] [DEBUG] [LOCAL ] TTI: Saving derivative data. [2025-10-20T22:15:04.087400] [DEBUG] [LOCAL ] TTI: Derivative data saved (1794 records). [2025-10-20T22:15:04.087497] [INFO ] [MAIN ] TTI: Successfully saved derivative data. [2025-10-20T22:15:04.086370] [DEBUG] [LOCAL ] TTE: Derivative data saved (1794 records). [2025-10-20T22:15:04.089752] [INFO ] [MAIN ] Derivative Progress: 1664/1877 symbols completed. TTI took 1.1s. Estimated remaining: 1m 20s. Rate: 160.0 symbols/min [2025-10-20T22:15:04.089894] [INFO ] [MAIN ] TTE: Successfully saved derivative data. [2025-10-20T22:15:04.092050] [INFO ] [MAIN ] Derivative Progress: 1665/1877 symbols completed. TTE took 1.2s. Estimated remaining: 1m 19s. Rate: 160.1 symbols/min [2025-10-20T22:15:04.092446] [INFO ] [MAIN ] TUYA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:04.092578] [INFO ] [MAIN ] TU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:04.093240] [INFO ] [MAIN ] TV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:04.152083] [DEBUG] [LOCAL ] TUYA: Loaded 45927 existing records from primary storage. [2025-10-20T22:15:04.152328] [INFO ] [MAIN ] TUYA: Loaded 45927 primary data records. [2025-10-20T22:15:04.286986] [DEBUG] [LOCAL ] TV: Loaded 41489 existing records from primary storage. [2025-10-20T22:15:04.303002] [INFO ] [MAIN ] TV: Loaded 41489 primary data records. [2025-10-20T22:15:04.303820] [DEBUG] [LOCAL ] TU: Loaded 49805 existing records from primary storage. [2025-10-20T22:15:04.349751] [INFO ] [MAIN ] TU: Loaded 49805 primary data records. [2025-10-20T22:15:05.166742] [DEBUG] [MAIN ] TUYA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:05.167533] [INFO ] [MAIN ] TUYA: Generated 1794 derivative records from 45927 primary records. [2025-10-20T22:15:05.167632] [DEBUG] [LOCAL ] TUYA: Saving derivative data. [2025-10-20T22:15:05.186034] [DEBUG] [LOCAL ] TUYA: Derivative data saved (1794 records). [2025-10-20T22:15:05.186236] [INFO ] [MAIN ] TUYA: Successfully saved derivative data. [2025-10-20T22:15:05.188003] [INFO ] [MAIN ] Derivative Progress: 1666/1877 symbols completed. TUYA took 1.1s. Estimated remaining: 1m 19s. Rate: 159.9 symbols/min [2025-10-20T22:15:05.195827] [INFO ] [MAIN ] TVC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:05.203888] [DEBUG] [MAIN ] TV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:05.204963] [INFO ] [MAIN ] TV: Generated 1794 derivative records from 41489 primary records. [2025-10-20T22:15:05.205124] [DEBUG] [LOCAL ] TV: Saving derivative data. [2025-10-20T22:15:05.219369] [DEBUG] [MAIN ] TU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:05.220447] [INFO ] [MAIN ] TU: Generated 1794 derivative records from 49805 primary records. [2025-10-20T22:15:05.227387] [DEBUG] [LOCAL ] TU: Saving derivative data. [2025-10-20T22:15:05.234600] [DEBUG] [LOCAL ] TVC: Loaded 3148 existing records from primary storage. [2025-10-20T22:15:05.239039] [DEBUG] [LOCAL ] TV: Derivative data saved (1794 records). [2025-10-20T22:15:05.239329] [INFO ] [MAIN ] TVC: Loaded 3148 primary data records. [2025-10-20T22:15:05.285555] [DEBUG] [LOCAL ] TU: Derivative data saved (1794 records). [2025-10-20T22:15:05.284522] [INFO ] [MAIN ] TV: Successfully saved derivative data. [2025-10-20T22:15:05.320008] [INFO ] [MAIN ] Derivative Progress: 1667/1877 symbols completed. TV took 1.2s. Estimated remaining: 1m 19s. Rate: 160.0 symbols/min [2025-10-20T22:15:05.325278] [INFO ] [MAIN ] TU: Successfully saved derivative data. [2025-10-20T22:15:05.325385] [INFO ] [MAIN ] TVE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:05.344019] [INFO ] [MAIN ] Derivative Progress: 1668/1877 symbols completed. TU took 1.3s. Estimated remaining: 1m 18s. Rate: 160.1 symbols/min [2025-10-20T22:15:05.344304] [INFO ] [MAIN ] TWI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:05.436250] [DEBUG] [LOCAL ] TVE: Loaded 2584 existing records from primary storage. [2025-10-20T22:15:05.562628] [INFO ] [MAIN ] TVE: Loaded 2584 primary data records. [2025-10-20T22:15:05.740200] [DEBUG] [LOCAL ] TWI: Loaded 33967 existing records from primary storage. [2025-10-20T22:15:05.764510] [INFO ] [MAIN ] TWI: Loaded 33967 primary data records. [2025-10-20T22:15:06.080744] [DEBUG] [MAIN ] TVC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:06.181410] [INFO ] [MAIN ] TVC: Generated 1794 derivative records from 3148 primary records. [2025-10-20T22:15:06.194777] [DEBUG] [LOCAL ] TVC: Saving derivative data. [2025-10-20T22:15:06.210568] [DEBUG] [LOCAL ] TVC: Derivative data saved (1794 records). [2025-10-20T22:15:06.212044] [INFO ] [MAIN ] TVC: Successfully saved derivative data. [2025-10-20T22:15:06.213940] [INFO ] [MAIN ] Derivative Progress: 1669/1877 symbols completed. TVC took 1.0s. Estimated remaining: 1m 18s. Rate: 160.0 symbols/min [2025-10-20T22:15:06.214261] [INFO ] [MAIN ] TWLO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:06.233135] [DEBUG] [MAIN ] TVE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:06.233857] [INFO ] [MAIN ] TVE: Generated 1794 derivative records from 2584 primary records. [2025-10-20T22:15:06.234062] [DEBUG] [LOCAL ] TVE: Saving derivative data. [2025-10-20T22:15:06.249326] [DEBUG] [LOCAL ] TVE: Derivative data saved (1794 records). [2025-10-20T22:15:06.249757] [INFO ] [MAIN ] TVE: Successfully saved derivative data. [2025-10-20T22:15:06.251798] [INFO ] [MAIN ] Derivative Progress: 1670/1877 symbols completed. TVE took 0.9s. Estimated remaining: 1m 18s. Rate: 160.0 symbols/min [2025-10-20T22:15:06.252587] [INFO ] [MAIN ] TWN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:06.272687] [DEBUG] [LOCAL ] TWN: Loaded 3258 existing records from primary storage. [2025-10-20T22:15:06.273105] [DEBUG] [MAIN ] TWI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:06.287700] [INFO ] [MAIN ] TWN: Loaded 3258 primary data records. [2025-10-20T22:15:06.273914] [INFO ] [MAIN ] TWI: Generated 1794 derivative records from 33967 primary records. [2025-10-20T22:15:06.301837] [DEBUG] [LOCAL ] TWI: Saving derivative data. [2025-10-20T22:15:06.411610] [DEBUG] [LOCAL ] TWI: Derivative data saved (1794 records). [2025-10-20T22:15:06.427148] [INFO ] [MAIN ] TWI: Successfully saved derivative data. [2025-10-20T22:15:06.456637] [DEBUG] [LOCAL ] TWLO: Loaded 52836 existing records from primary storage. [2025-10-20T22:15:06.434747] [INFO ] [MAIN ] Derivative Progress: 1671/1877 symbols completed. TWI took 1.1s. Estimated remaining: 1m 17s. Rate: 160.1 symbols/min [2025-10-20T22:15:06.489870] [INFO ] [MAIN ] TWO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:06.489745] [INFO ] [MAIN ] TWLO: Loaded 52836 primary data records. [2025-10-20T22:15:06.992772] [DEBUG] [LOCAL ] TWO: Loaded 48624 existing records from primary storage. [2025-10-20T22:15:07.005542] [INFO ] [MAIN ] TWO: Loaded 48624 primary data records. [2025-10-20T22:15:07.006197] [DEBUG] [MAIN ] TWN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:07.043629] [INFO ] [MAIN ] TWN: Generated 1794 derivative records from 3258 primary records. [2025-10-20T22:15:07.065094] [DEBUG] [LOCAL ] TWN: Saving derivative data. [2025-10-20T22:15:07.133525] [DEBUG] [LOCAL ] TWN: Derivative data saved (1794 records). [2025-10-20T22:15:07.133699] [INFO ] [MAIN ] TWN: Successfully saved derivative data. [2025-10-20T22:15:07.135756] [INFO ] [MAIN ] Derivative Progress: 1672/1877 symbols completed. TWN took 0.9s. Estimated remaining: 1m 17s. Rate: 160.0 symbols/min [2025-10-20T22:15:07.160255] [INFO ] [MAIN ] TX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:07.160879] [DEBUG] [MAIN ] TWLO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:07.172891] [INFO ] [MAIN ] TWLO: Generated 1793 derivative records from 52836 primary records. [2025-10-20T22:15:07.227741] [DEBUG] [LOCAL ] TWLO: Saving derivative data. [2025-10-20T22:15:07.370357] [DEBUG] [LOCAL ] TWLO: Derivative data saved (1793 records). [2025-10-20T22:15:07.379406] [INFO ] [MAIN ] TWLO: Successfully saved derivative data. [2025-10-20T22:15:07.380359] [DEBUG] [LOCAL ] TX: Loaded 23414 existing records from primary storage. [2025-10-20T22:15:07.380691] [INFO ] [MAIN ] TX: Loaded 23414 primary data records. [2025-10-20T22:15:07.382737] [INFO ] [MAIN ] Derivative Progress: 1673/1877 symbols completed. TWLO took 1.2s. Estimated remaining: 1m 16s. Rate: 160.0 symbols/min [2025-10-20T22:15:07.394527] [INFO ] [MAIN ] TXNM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:07.715115] [DEBUG] [MAIN ] TWO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:07.716301] [INFO ] [MAIN ] TWO: Generated 1794 derivative records from 48624 primary records. [2025-10-20T22:15:07.717035] [DEBUG] [LOCAL ] TWO: Saving derivative data. [2025-10-20T22:15:07.742854] [DEBUG] [LOCAL ] TXNM: Loaded 43149 existing records from primary storage. [2025-10-20T22:15:07.747682] [DEBUG] [LOCAL ] TWO: Derivative data saved (1794 records). [2025-10-20T22:15:07.747767] [INFO ] [MAIN ] TXNM: Loaded 43149 primary data records. [2025-10-20T22:15:07.747880] [INFO ] [MAIN ] TWO: Successfully saved derivative data. [2025-10-20T22:15:07.749898] [INFO ] [MAIN ] Derivative Progress: 1674/1877 symbols completed. TWO took 1.3s. Estimated remaining: 1m 16s. Rate: 160.0 symbols/min [2025-10-20T22:15:07.750485] [INFO ] [MAIN ] TXO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:08.100662] [DEBUG] [MAIN ] TX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:08.101117] [DEBUG] [LOCAL ] TXO: Loaded 19974 existing records from primary storage. [2025-10-20T22:15:08.102001] [INFO ] [MAIN ] TX: Generated 1794 derivative records from 23414 primary records. [2025-10-20T22:15:08.102317] [INFO ] [MAIN ] TXO: Loaded 19974 primary data records. [2025-10-20T22:15:08.102245] [DEBUG] [LOCAL ] TX: Saving derivative data. [2025-10-20T22:15:08.206617] [DEBUG] [LOCAL ] TX: Derivative data saved (1794 records). [2025-10-20T22:15:08.223058] [INFO ] [MAIN ] TX: Successfully saved derivative data. [2025-10-20T22:15:08.225179] [INFO ] [MAIN ] Derivative Progress: 1675/1877 symbols completed. TX took 1.1s. Estimated remaining: 1m 16s. Rate: 160.0 symbols/min [2025-10-20T22:15:08.226056] [INFO ] [MAIN ] TXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:08.467537] [DEBUG] [MAIN ] TXNM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:08.468555] [INFO ] [MAIN ] TXNM: Generated 1794 derivative records from 43149 primary records. [2025-10-20T22:15:08.469388] [DEBUG] [LOCAL ] TXNM: Saving derivative data. [2025-10-20T22:15:08.487127] [DEBUG] [LOCAL ] TXNM: Derivative data saved (1794 records). [2025-10-20T22:15:08.487510] [INFO ] [MAIN ] TXNM: Successfully saved derivative data. [2025-10-20T22:15:08.489884] [INFO ] [MAIN ] Derivative Progress: 1676/1877 symbols completed. TXNM took 1.1s. Estimated remaining: 1m 15s. Rate: 160.1 symbols/min [2025-10-20T22:15:08.504877] [INFO ] [MAIN ] TY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:08.507875] [DEBUG] [LOCAL ] TXT: Loaded 49541 existing records from primary storage. [2025-10-20T22:15:08.508419] [INFO ] [MAIN ] TXT: Loaded 49541 primary data records. [2025-10-20T22:15:08.641040] [DEBUG] [LOCAL ] TY: Loaded 7553 existing records from primary storage. [2025-10-20T22:15:08.642013] [DEBUG] [MAIN ] TXO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:08.641749] [INFO ] [MAIN ] TY: Loaded 7553 primary data records. [2025-10-20T22:15:08.659009] [INFO ] [MAIN ] TXO: Generated 1794 derivative records from 19974 primary records. [2025-10-20T22:15:08.730623] [DEBUG] [LOCAL ] TXO: Saving derivative data. [2025-10-20T22:15:08.986897] [DEBUG] [LOCAL ] TXO: Derivative data saved (1794 records). [2025-10-20T22:15:09.002988] [INFO ] [MAIN ] TXO: Successfully saved derivative data. [2025-10-20T22:15:09.016560] [INFO ] [MAIN ] Derivative Progress: 1677/1877 symbols completed. TXO took 1.3s. Estimated remaining: 1m 15s. Rate: 160.0 symbols/min [2025-10-20T22:15:09.062850] [INFO ] [MAIN ] TYG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:09.200633] [DEBUG] [LOCAL ] TYG: Loaded 13996 existing records from primary storage. [2025-10-20T22:15:09.201091] [INFO ] [MAIN ] TYG: Loaded 13996 primary data records. [2025-10-20T22:15:09.348592] [DEBUG] [MAIN ] TXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:09.371036] [INFO ] [MAIN ] TXT: Generated 1794 derivative records from 49541 primary records. [2025-10-20T22:15:09.371142] [DEBUG] [LOCAL ] TXT: Saving derivative data. [2025-10-20T22:15:09.534592] [DEBUG] [LOCAL ] TXT: Derivative data saved (1794 records). [2025-10-20T22:15:09.534931] [INFO ] [MAIN ] TXT: Successfully saved derivative data. [2025-10-20T22:15:09.537037] [INFO ] [MAIN ] Derivative Progress: 1678/1877 symbols completed. TXT took 1.3s. Estimated remaining: 1m 15s. Rate: 160.0 symbols/min [2025-10-20T22:15:09.550857] [DEBUG] [MAIN ] TY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:09.552588] [INFO ] [MAIN ] TY: Generated 1794 derivative records from 7553 primary records. [2025-10-20T22:15:09.552701] [DEBUG] [LOCAL ] TY: Saving derivative data. [2025-10-20T22:15:09.569280] [DEBUG] [LOCAL ] TY: Derivative data saved (1794 records). [2025-10-20T22:15:09.569444] [INFO ] [MAIN ] TY: Successfully saved derivative data. [2025-10-20T22:15:09.571492] [INFO ] [MAIN ] Derivative Progress: 1679/1877 symbols completed. TY took 1.1s. Estimated remaining: 1m 14s. Rate: 160.1 symbols/min [2025-10-20T22:15:09.590113] [DEBUG] [MAIN ] TYG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:09.590764] [INFO ] [MAIN ] TYG: Generated 1794 derivative records from 13996 primary records. [2025-10-20T22:15:09.590847] [DEBUG] [LOCAL ] TYG: Saving derivative data. [2025-10-20T22:15:09.605695] [DEBUG] [LOCAL ] TYG: Derivative data saved (1794 records). [2025-10-20T22:15:09.605752] [INFO ] [MAIN ] TYG: Successfully saved derivative data. [2025-10-20T22:15:09.607987] [INFO ] [MAIN ] Derivative Progress: 1680/1877 symbols completed. TYG took 0.5s. Estimated remaining: 1m 14s. Rate: 160.1 symbols/min [2025-10-20T22:15:09.608382] [INFO ] [MAIN ] TYL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:09.609039] [INFO ] [MAIN ] U: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:09.608597] [INFO ] [MAIN ] UA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:09.639169] [DEBUG] [LOCAL ] TYL: Loaded 22229 existing records from primary storage. [2025-10-20T22:15:09.639303] [INFO ] [MAIN ] TYL: Loaded 22229 primary data records. [2025-10-20T22:15:09.856419] [DEBUG] [LOCAL ] UA: Loaded 50486 existing records from primary storage. [2025-10-20T22:15:09.881064] [DEBUG] [LOCAL ] U: Loaded 53799 existing records from primary storage. [2025-10-20T22:15:09.874523] [INFO ] [MAIN ] UA: Loaded 50486 primary data records. [2025-10-20T22:15:09.987599] [INFO ] [MAIN ] U: Loaded 53799 primary data records. [2025-10-20T22:15:10.678405] [DEBUG] [MAIN ] TYL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:10.681112] [INFO ] [MAIN ] TYL: Generated 1794 derivative records from 22229 primary records. [2025-10-20T22:15:10.681362] [DEBUG] [LOCAL ] TYL: Saving derivative data. [2025-10-20T22:15:10.706889] [DEBUG] [LOCAL ] TYL: Derivative data saved (1794 records). [2025-10-20T22:15:10.707026] [INFO ] [MAIN ] TYL: Successfully saved derivative data. [2025-10-20T22:15:10.713909] [INFO ] [MAIN ] Derivative Progress: 1681/1877 symbols completed. TYL took 1.1s. Estimated remaining: 1m 14s. Rate: 160.0 symbols/min [2025-10-20T22:15:10.714581] [INFO ] [MAIN ] UAA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:10.714273] [DEBUG] [MAIN ] UA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:10.716091] [INFO ] [MAIN ] UA: Generated 1794 derivative records from 50486 primary records. [2025-10-20T22:15:10.716554] [DEBUG] [LOCAL ] UA: Saving derivative data. [2025-10-20T22:15:10.736728] [DEBUG] [LOCAL ] UA: Derivative data saved (1794 records). [2025-10-20T22:15:10.737054] [INFO ] [MAIN ] UA: Successfully saved derivative data. [2025-10-20T22:15:10.740772] [INFO ] [MAIN ] Derivative Progress: 1682/1877 symbols completed. UA took 1.1s. Estimated remaining: 1m 13s. Rate: 160.1 symbols/min [2025-10-20T22:15:10.753820] [INFO ] [MAIN ] UAN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:10.754131] [DEBUG] [MAIN ] U: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:10.755293] [INFO ] [MAIN ] U: Generated 1794 derivative records from 53799 primary records. [2025-10-20T22:15:10.755491] [DEBUG] [LOCAL ] U: Saving derivative data. [2025-10-20T22:15:10.774029] [DEBUG] [LOCAL ] U: Derivative data saved (1794 records). [2025-10-20T22:15:10.774288] [INFO ] [MAIN ] U: Successfully saved derivative data. [2025-10-20T22:15:10.776460] [INFO ] [MAIN ] Derivative Progress: 1683/1877 symbols completed. U took 1.2s. Estimated remaining: 1m 13s. Rate: 160.1 symbols/min [2025-10-20T22:15:10.776830] [INFO ] [MAIN ] UBER: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:10.794014] [DEBUG] [LOCAL ] UAN: Loaded 5412 existing records from primary storage. [2025-10-20T22:15:10.794261] [INFO ] [MAIN ] UAN: Loaded 5412 primary data records. [2025-10-20T22:15:10.918342] [DEBUG] [LOCAL ] UAA: Loaded 53499 existing records from primary storage. [2025-10-20T22:15:10.923827] [INFO ] [MAIN ] UAA: Loaded 53499 primary data records. [2025-10-20T22:15:11.334763] [DEBUG] [LOCAL ] UBER: Loaded 53806 existing records from primary storage. [2025-10-20T22:15:11.339325] [INFO ] [MAIN ] UBER: Loaded 53806 primary data records. [2025-10-20T22:15:11.784376] [DEBUG] [MAIN ] UAN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:11.785545] [INFO ] [MAIN ] UAN: Generated 1794 derivative records from 5412 primary records. [2025-10-20T22:15:11.785902] [DEBUG] [LOCAL ] UAN: Saving derivative data. [2025-10-20T22:15:11.809084] [DEBUG] [LOCAL ] UAN: Derivative data saved (1794 records). [2025-10-20T22:15:11.810298] [INFO ] [MAIN ] UAN: Successfully saved derivative data. [2025-10-20T22:15:11.821632] [INFO ] [MAIN ] Derivative Progress: 1684/1877 symbols completed. UAN took 1.1s. Estimated remaining: 1m 12s. Rate: 160.0 symbols/min [2025-10-20T22:15:11.826864] [INFO ] [MAIN ] UBS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:11.827807] [DEBUG] [MAIN ] UAA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:11.829951] [INFO ] [MAIN ] UAA: Generated 1794 derivative records from 53499 primary records. [2025-10-20T22:15:11.830194] [DEBUG] [LOCAL ] UAA: Saving derivative data. [2025-10-20T22:15:11.846403] [DEBUG] [LOCAL ] UAA: Derivative data saved (1794 records). [2025-10-20T22:15:11.846620] [INFO ] [MAIN ] UAA: Successfully saved derivative data. [2025-10-20T22:15:11.848525] [INFO ] [MAIN ] Derivative Progress: 1685/1877 symbols completed. UAA took 1.1s. Estimated remaining: 1m 12s. Rate: 160.1 symbols/min [2025-10-20T22:15:11.848972] [INFO ] [MAIN ] UCB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:11.868718] [DEBUG] [MAIN ] UBER: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:11.869582] [INFO ] [MAIN ] UBER: Generated 1794 derivative records from 53806 primary records. [2025-10-20T22:15:11.869826] [DEBUG] [LOCAL ] UBER: Saving derivative data. [2025-10-20T22:15:11.887208] [DEBUG] [LOCAL ] UBER: Derivative data saved (1794 records). [2025-10-20T22:15:11.887424] [INFO ] [MAIN ] UBER: Successfully saved derivative data. [2025-10-20T22:15:11.889986] [INFO ] [MAIN ] Derivative Progress: 1686/1877 symbols completed. UBER took 1.1s. Estimated remaining: 1m 12s. Rate: 160.1 symbols/min [2025-10-20T22:15:11.890198] [INFO ] [MAIN ] UDR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:11.928595] [DEBUG] [LOCAL ] UCB: Loaded 37801 existing records from primary storage. [2025-10-20T22:15:11.928984] [INFO ] [MAIN ] UCB: Loaded 37801 primary data records. [2025-10-20T22:15:12.011564] [DEBUG] [LOCAL ] UBS: Loaded 52297 existing records from primary storage. [2025-10-20T22:15:12.017958] [INFO ] [MAIN ] UBS: Loaded 52297 primary data records. [2025-10-20T22:15:12.409269] [DEBUG] [LOCAL ] UDR: Loaded 52109 existing records from primary storage. [2025-10-20T22:15:12.425799] [INFO ] [MAIN ] UDR: Loaded 52109 primary data records. [2025-10-20T22:15:12.973300] [DEBUG] [MAIN ] UCB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:12.975231] [INFO ] [MAIN ] UCB: Generated 1794 derivative records from 37801 primary records. [2025-10-20T22:15:12.975400] [DEBUG] [LOCAL ] UCB: Saving derivative data. [2025-10-20T22:15:12.994396] [DEBUG] [LOCAL ] UCB: Derivative data saved (1794 records). [2025-10-20T22:15:12.994617] [INFO ] [MAIN ] UCB: Successfully saved derivative data. [2025-10-20T22:15:12.996714] [INFO ] [MAIN ] Derivative Progress: 1687/1877 symbols completed. UCB took 1.1s. Estimated remaining: 1m 11s. Rate: 160.0 symbols/min [2025-10-20T22:15:12.996985] [INFO ] [MAIN ] UE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:13.010726] [DEBUG] [MAIN ] UBS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:13.011940] [INFO ] [MAIN ] UBS: Generated 1794 derivative records from 52297 primary records. [2025-10-20T22:15:13.012238] [DEBUG] [LOCAL ] UBS: Saving derivative data. [2025-10-20T22:15:13.031823] [DEBUG] [LOCAL ] UBS: Derivative data saved (1794 records). [2025-10-20T22:15:13.032021] [INFO ] [MAIN ] UBS: Successfully saved derivative data. [2025-10-20T22:15:13.046320] [INFO ] [MAIN ] Derivative Progress: 1688/1877 symbols completed. UBS took 1.2s. Estimated remaining: 1m 11s. Rate: 160.0 symbols/min [2025-10-20T22:15:13.046678] [DEBUG] [MAIN ] UDR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:13.047004] [INFO ] [MAIN ] UFI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:13.048200] [INFO ] [MAIN ] UDR: Generated 1794 derivative records from 52109 primary records. [2025-10-20T22:15:13.048451] [DEBUG] [LOCAL ] UDR: Saving derivative data. [2025-10-20T22:15:13.075414] [DEBUG] [LOCAL ] UDR: Derivative data saved (1794 records). [2025-10-20T22:15:13.075588] [INFO ] [MAIN ] UDR: Successfully saved derivative data. [2025-10-20T22:15:13.077707] [INFO ] [MAIN ] Derivative Progress: 1689/1877 symbols completed. UDR took 1.2s. Estimated remaining: 1m 10s. Rate: 160.1 symbols/min [2025-10-20T22:15:13.077891] [INFO ] [MAIN ] UGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:13.082528] [DEBUG] [LOCAL ] UFI: Loaded 8161 existing records from primary storage. [2025-10-20T22:15:13.082692] [INFO ] [MAIN ] UFI: Loaded 8161 primary data records. [2025-10-20T22:15:13.201392] [DEBUG] [LOCAL ] UE: Loaded 42603 existing records from primary storage. [2025-10-20T22:15:13.207300] [INFO ] [MAIN ] UE: Loaded 42603 primary data records. [2025-10-20T22:15:13.552456] [DEBUG] [LOCAL ] UGI: Loaded 50511 existing records from primary storage. [2025-10-20T22:15:13.570914] [INFO ] [MAIN ] UGI: Loaded 50511 primary data records. [2025-10-20T22:15:14.063948] [DEBUG] [MAIN ] UFI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:14.065597] [INFO ] [MAIN ] UFI: Generated 1794 derivative records from 8161 primary records. [2025-10-20T22:15:14.078182] [DEBUG] [LOCAL ] UFI: Saving derivative data. [2025-10-20T22:15:14.079710] [DEBUG] [MAIN ] UE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:14.081783] [INFO ] [MAIN ] UE: Generated 1794 derivative records from 42603 primary records. [2025-10-20T22:15:14.088176] [DEBUG] [LOCAL ] UE: Saving derivative data. [2025-10-20T22:15:14.095828] [DEBUG] [LOCAL ] UFI: Derivative data saved (1794 records). [2025-10-20T22:15:14.096881] [INFO ] [MAIN ] UFI: Successfully saved derivative data. [2025-10-20T22:15:14.099074] [INFO ] [MAIN ] Derivative Progress: 1690/1877 symbols completed. UFI took 1.1s. Estimated remaining: 1m 10s. Rate: 160.0 symbols/min [2025-10-20T22:15:14.106613] [INFO ] [MAIN ] UGP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:14.114844] [DEBUG] [LOCAL ] UE: Derivative data saved (1794 records). [2025-10-20T22:15:14.115063] [INFO ] [MAIN ] UE: Successfully saved derivative data. [2025-10-20T22:15:14.117203] [INFO ] [MAIN ] Derivative Progress: 1691/1877 symbols completed. UE took 1.1s. Estimated remaining: 1m 10s. Rate: 160.1 symbols/min [2025-10-20T22:15:14.117798] [INFO ] [MAIN ] UHAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:14.134280] [DEBUG] [MAIN ] UGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:14.134996] [INFO ] [MAIN ] UGI: Generated 1794 derivative records from 50511 primary records. [2025-10-20T22:15:14.135259] [DEBUG] [LOCAL ] UGI: Saving derivative data. [2025-10-20T22:15:14.155156] [DEBUG] [LOCAL ] UGI: Derivative data saved (1794 records). [2025-10-20T22:15:14.155302] [INFO ] [MAIN ] UGI: Successfully saved derivative data. [2025-10-20T22:15:14.157522] [INFO ] [MAIN ] Derivative Progress: 1692/1877 symbols completed. UGI took 1.1s. Estimated remaining: 1m 9s. Rate: 160.1 symbols/min [2025-10-20T22:15:14.157864] [INFO ] [MAIN ] UHS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:14.174251] [DEBUG] [LOCAL ] UHAL: Loaded 16899 existing records from primary storage. [2025-10-20T22:15:14.174469] [INFO ] [MAIN ] UHAL: Loaded 16899 primary data records. [2025-10-20T22:15:14.278743] [DEBUG] [LOCAL ] UGP: Loaded 42987 existing records from primary storage. [2025-10-20T22:15:14.295494] [INFO ] [MAIN ] UGP: Loaded 42987 primary data records. [2025-10-20T22:15:14.619622] [DEBUG] [LOCAL ] UHS: Loaded 39895 existing records from primary storage. [2025-10-20T22:15:14.630737] [INFO ] [MAIN ] UHS: Loaded 39895 primary data records. [2025-10-20T22:15:15.174798] [DEBUG] [MAIN ] UHAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:15.175974] [INFO ] [MAIN ] UHAL: Generated 1794 derivative records from 16899 primary records. [2025-10-20T22:15:15.177032] [DEBUG] [LOCAL ] UHAL: Saving derivative data. [2025-10-20T22:15:15.213140] [DEBUG] [LOCAL ] UHAL: Derivative data saved (1794 records). [2025-10-20T22:15:15.213485] [DEBUG] [MAIN ] UGP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:15.213629] [INFO ] [MAIN ] UHAL: Successfully saved derivative data. [2025-10-20T22:15:15.215666] [INFO ] [MAIN ] Derivative Progress: 1693/1877 symbols completed. UHAL took 1.1s. Estimated remaining: 1m 9s. Rate: 160.0 symbols/min [2025-10-20T22:15:15.216909] [INFO ] [MAIN ] UGP: Generated 1794 derivative records from 42987 primary records. [2025-10-20T22:15:15.217072] [DEBUG] [LOCAL ] UGP: Saving derivative data. [2025-10-20T22:15:15.235262] [DEBUG] [LOCAL ] UGP: Derivative data saved (1794 records). [2025-10-20T22:15:15.247549] [INFO ] [MAIN ] UGP: Successfully saved derivative data. [2025-10-20T22:15:15.249769] [INFO ] [MAIN ] Derivative Progress: 1694/1877 symbols completed. UGP took 1.1s. Estimated remaining: 1m 9s. Rate: 160.0 symbols/min [2025-10-20T22:15:15.247858] [DEBUG] [MAIN ] UHS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:15.250617] [INFO ] [MAIN ] UHS: Generated 1794 derivative records from 39895 primary records. [2025-10-20T22:15:15.250791] [DEBUG] [LOCAL ] UHS: Saving derivative data. [2025-10-20T22:15:15.265900] [DEBUG] [LOCAL ] UHS: Derivative data saved (1794 records). [2025-10-20T22:15:15.265962] [INFO ] [MAIN ] UHS: Successfully saved derivative data. [2025-10-20T22:15:15.268025] [INFO ] [MAIN ] Derivative Progress: 1695/1877 symbols completed. UHS took 1.1s. Estimated remaining: 1m 8s. Rate: 160.1 symbols/min [2025-10-20T22:15:15.268417] [INFO ] [MAIN ] UHT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:15.268475] [INFO ] [MAIN ] UI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:15.268540] [INFO ] [MAIN ] UIS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:15.284460] [DEBUG] [LOCAL ] UHT: Loaded 10229 existing records from primary storage. [2025-10-20T22:15:15.284680] [INFO ] [MAIN ] UHT: Loaded 10229 primary data records. [2025-10-20T22:15:15.328763] [DEBUG] [LOCAL ] UI: Loaded 14122 existing records from primary storage. [2025-10-20T22:15:15.339682] [INFO ] [MAIN ] UI: Loaded 14122 primary data records. [2025-10-20T22:15:15.555708] [DEBUG] [LOCAL ] UIS: Loaded 36775 existing records from primary storage. [2025-10-20T22:15:15.574375] [INFO ] [MAIN ] UIS: Loaded 36775 primary data records. [2025-10-20T22:15:16.312722] [DEBUG] [MAIN ] UHT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:16.326454] [INFO ] [MAIN ] UHT: Generated 1794 derivative records from 10229 primary records. [2025-10-20T22:15:16.327103] [DEBUG] [MAIN ] UI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:16.326841] [DEBUG] [LOCAL ] UHT: Saving derivative data. [2025-10-20T22:15:16.328964] [INFO ] [MAIN ] UI: Generated 1794 derivative records from 14122 primary records. [2025-10-20T22:15:16.329477] [DEBUG] [LOCAL ] UI: Saving derivative data. [2025-10-20T22:15:16.364568] [DEBUG] [LOCAL ] UI: Derivative data saved (1794 records). [2025-10-20T22:15:16.367843] [INFO ] [MAIN ] UI: Successfully saved derivative data. [2025-10-20T22:15:16.370481] [INFO ] [MAIN ] Derivative Progress: 1696/1877 symbols completed. UI took 1.1s. Estimated remaining: 1m 8s. Rate: 160.0 symbols/min [2025-10-20T22:15:16.371142] [INFO ] [MAIN ] UL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:16.373978] [DEBUG] [LOCAL ] UHT: Derivative data saved (1794 records). [2025-10-20T22:15:16.388276] [INFO ] [MAIN ] UHT: Successfully saved derivative data. [2025-10-20T22:15:16.390777] [INFO ] [MAIN ] Derivative Progress: 1697/1877 symbols completed. UHT took 1.1s. Estimated remaining: 1m 7s. Rate: 160.0 symbols/min [2025-10-20T22:15:16.391196] [DEBUG] [MAIN ] UIS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:16.392320] [INFO ] [MAIN ] UIS: Generated 1794 derivative records from 36775 primary records. [2025-10-20T22:15:16.391380] [INFO ] [MAIN ] ULS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:16.392437] [DEBUG] [LOCAL ] UIS: Saving derivative data. [2025-10-20T22:15:16.421047] [DEBUG] [LOCAL ] UIS: Derivative data saved (1794 records). [2025-10-20T22:15:16.421311] [INFO ] [MAIN ] UIS: Successfully saved derivative data. [2025-10-20T22:15:16.424288] [INFO ] [MAIN ] Derivative Progress: 1698/1877 symbols completed. UIS took 1.2s. Estimated remaining: 1m 7s. Rate: 160.1 symbols/min [2025-10-20T22:15:16.424722] [INFO ] [MAIN ] UMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:16.479621] [DEBUG] [LOCAL ] ULS: Loaded 38818 existing records from primary storage. [2025-10-20T22:15:16.479952] [INFO ] [MAIN ] ULS: Loaded 38818 primary data records. [2025-10-20T22:15:16.627368] [DEBUG] [LOCAL ] UL: Loaded 53076 existing records from primary storage. [2025-10-20T22:15:16.627597] [INFO ] [MAIN ] UL: Loaded 53076 primary data records. [2025-10-20T22:15:17.075818] [DEBUG] [LOCAL ] UMC: Loaded 52683 existing records from primary storage. [2025-10-20T22:15:17.081486] [INFO ] [MAIN ] UMC: Loaded 52683 primary data records. [2025-10-20T22:15:17.650130] [DEBUG] [MAIN ] ULS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:17.650904] [INFO ] [MAIN ] ULS: Generated 1794 derivative records from 38818 primary records. [2025-10-20T22:15:17.651073] [DEBUG] [LOCAL ] ULS: Saving derivative data. [2025-10-20T22:15:17.669231] [DEBUG] [LOCAL ] ULS: Derivative data saved (1794 records). [2025-10-20T22:15:17.670214] [INFO ] [MAIN ] ULS: Successfully saved derivative data. [2025-10-20T22:15:17.672268] [INFO ] [MAIN ] Derivative Progress: 1699/1877 symbols completed. ULS took 1.3s. Estimated remaining: 1m 7s. Rate: 159.9 symbols/min [2025-10-20T22:15:17.672532] [INFO ] [MAIN ] UMH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:17.686332] [DEBUG] [MAIN ] UL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:17.687033] [INFO ] [MAIN ] UL: Generated 1794 derivative records from 53076 primary records. [2025-10-20T22:15:17.687209] [DEBUG] [LOCAL ] UL: Saving derivative data. [2025-10-20T22:15:17.704675] [DEBUG] [LOCAL ] UL: Derivative data saved (1794 records). [2025-10-20T22:15:17.705622] [INFO ] [MAIN ] UL: Successfully saved derivative data. [2025-10-20T22:15:17.707625] [INFO ] [MAIN ] Derivative Progress: 1700/1877 symbols completed. UL took 1.3s. Estimated remaining: 1m 6s. Rate: 160.0 symbols/min [2025-10-20T22:15:17.708108] [INFO ] [MAIN ] UNF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:17.721820] [DEBUG] [MAIN ] UMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:17.731184] [INFO ] [MAIN ] UMC: Generated 1794 derivative records from 52683 primary records. [2025-10-20T22:15:17.731417] [DEBUG] [LOCAL ] UMC: Saving derivative data. [2025-10-20T22:15:17.748893] [DEBUG] [LOCAL ] UMC: Derivative data saved (1794 records). [2025-10-20T22:15:17.758754] [INFO ] [MAIN ] UMC: Successfully saved derivative data. [2025-10-20T22:15:17.760920] [INFO ] [MAIN ] Derivative Progress: 1701/1877 symbols completed. UMC took 1.3s. Estimated remaining: 1m 6s. Rate: 160.1 symbols/min [2025-10-20T22:15:17.761121] [INFO ] [MAIN ] UNFI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:17.763607] [DEBUG] [LOCAL ] UMH: Loaded 34230 existing records from primary storage. [2025-10-20T22:15:17.764341] [INFO ] [MAIN ] UMH: Loaded 34230 primary data records. [2025-10-20T22:15:17.812561] [DEBUG] [LOCAL ] UNF: Loaded 11812 existing records from primary storage. [2025-10-20T22:15:17.860719] [INFO ] [MAIN ] UNF: Loaded 11812 primary data records. [2025-10-20T22:15:18.375878] [DEBUG] [LOCAL ] UNFI: Loaded 42612 existing records from primary storage. [2025-10-20T22:15:18.388999] [INFO ] [MAIN ] UNFI: Loaded 42612 primary data records. [2025-10-20T22:15:18.777298] [DEBUG] [MAIN ] UNF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:18.777857] [DEBUG] [MAIN ] UMH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:18.778590] [INFO ] [MAIN ] UNF: Generated 1794 derivative records from 11812 primary records. [2025-10-20T22:15:18.779311] [INFO ] [MAIN ] UMH: Generated 1794 derivative records from 34230 primary records. [2025-10-20T22:15:18.779591] [DEBUG] [LOCAL ] UMH: Saving derivative data. [2025-10-20T22:15:18.779485] [DEBUG] [LOCAL ] UNF: Saving derivative data. [2025-10-20T22:15:18.807424] [DEBUG] [LOCAL ] UNF: Derivative data saved (1794 records). [2025-10-20T22:15:18.807542] [INFO ] [MAIN ] UNF: Successfully saved derivative data. [2025-10-20T22:15:18.810938] [INFO ] [MAIN ] Derivative Progress: 1702/1877 symbols completed. UNF took 1.1s. Estimated remaining: 1m 6s. Rate: 159.9 symbols/min [2025-10-20T22:15:18.815320] [DEBUG] [LOCAL ] UMH: Derivative data saved (1794 records). [2025-10-20T22:15:18.814160] [INFO ] [MAIN ] UNH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:18.815628] [INFO ] [MAIN ] UMH: Successfully saved derivative data. [2025-10-20T22:15:18.818314] [INFO ] [MAIN ] Derivative Progress: 1703/1877 symbols completed. UMH took 1.1s. Estimated remaining: 1m 5s. Rate: 160.0 symbols/min [2025-10-20T22:15:18.818822] [INFO ] [MAIN ] UNM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:18.837702] [DEBUG] [MAIN ] UNFI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:18.838552] [INFO ] [MAIN ] UNFI: Generated 1794 derivative records from 42612 primary records. [2025-10-20T22:15:18.838852] [DEBUG] [LOCAL ] UNFI: Saving derivative data. [2025-10-20T22:15:18.857024] [DEBUG] [LOCAL ] UNFI: Derivative data saved (1794 records). [2025-10-20T22:15:18.857277] [INFO ] [MAIN ] UNFI: Successfully saved derivative data. [2025-10-20T22:15:18.859968] [INFO ] [MAIN ] Derivative Progress: 1704/1877 symbols completed. UNFI took 1.1s. Estimated remaining: 1m 5s. Rate: 160.1 symbols/min [2025-10-20T22:15:18.860257] [INFO ] [MAIN ] UNMA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:18.876725] [DEBUG] [LOCAL ] UNMA: Loaded 3457 existing records from primary storage. [2025-10-20T22:15:18.888811] [INFO ] [MAIN ] UNMA: Loaded 3457 primary data records. [2025-10-20T22:15:19.035390] [DEBUG] [LOCAL ] UNM: Loaded 48078 existing records from primary storage. [2025-10-20T22:15:19.036026] [DEBUG] [LOCAL ] UNH: Loaded 53819 existing records from primary storage. [2025-10-20T22:15:19.041375] [INFO ] [MAIN ] UNM: Loaded 48078 primary data records. [2025-10-20T22:15:19.064477] [INFO ] [MAIN ] UNH: Loaded 53819 primary data records. [2025-10-20T22:15:19.840976] [DEBUG] [MAIN ] UNMA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:19.841979] [INFO ] [MAIN ] UNMA: Generated 1794 derivative records from 3457 primary records. [2025-10-20T22:15:19.842161] [DEBUG] [LOCAL ] UNMA: Saving derivative data. [2025-10-20T22:15:19.857804] [DEBUG] [LOCAL ] UNMA: Derivative data saved (1794 records). [2025-10-20T22:15:19.858004] [INFO ] [MAIN ] UNMA: Successfully saved derivative data. [2025-10-20T22:15:19.860153] [INFO ] [MAIN ] Derivative Progress: 1705/1877 symbols completed. UNMA took 1.0s. Estimated remaining: 1m 5s. Rate: 159.9 symbols/min [2025-10-20T22:15:19.860301] [INFO ] [MAIN ] UNP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:19.886279] [DEBUG] [MAIN ] UNH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:19.891760] [INFO ] [MAIN ] UNH: Generated 1794 derivative records from 53819 primary records. [2025-10-20T22:15:19.891855] [DEBUG] [LOCAL ] UNH: Saving derivative data. [2025-10-20T22:15:19.892098] [DEBUG] [MAIN ] UNM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:19.892760] [INFO ] [MAIN ] UNM: Generated 1794 derivative records from 48078 primary records. [2025-10-20T22:15:19.893466] [DEBUG] [LOCAL ] UNM: Saving derivative data. [2025-10-20T22:15:19.923505] [DEBUG] [LOCAL ] UNM: Derivative data saved (1794 records). [2025-10-20T22:15:19.924832] [DEBUG] [LOCAL ] UNH: Derivative data saved (1794 records). [2025-10-20T22:15:19.925071] [INFO ] [MAIN ] UNM: Successfully saved derivative data. [2025-10-20T22:15:19.925119] [INFO ] [MAIN ] UNH: Successfully saved derivative data. [2025-10-20T22:15:19.927277] [INFO ] [MAIN ] Derivative Progress: 1706/1877 symbols completed. UNM took 1.1s. Estimated remaining: 1m 4s. Rate: 160.0 symbols/min [2025-10-20T22:15:19.927416] [INFO ] [MAIN ] UP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:19.929586] [INFO ] [MAIN ] Derivative Progress: 1707/1877 symbols completed. UNH took 1.1s. Estimated remaining: 1m 4s. Rate: 160.1 symbols/min [2025-10-20T22:15:19.930004] [INFO ] [MAIN ] UPS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:19.970691] [DEBUG] [LOCAL ] UNP: Loaded 53362 existing records from primary storage. [2025-10-20T22:15:19.970988] [INFO ] [MAIN ] UNP: Loaded 53362 primary data records. [2025-10-20T22:15:20.138454] [DEBUG] [LOCAL ] UP: Loaded 44329 existing records from primary storage. [2025-10-20T22:15:20.189841] [INFO ] [MAIN ] UP: Loaded 44329 primary data records. [2025-10-20T22:15:20.404103] [DEBUG] [LOCAL ] UPS: Loaded 53720 existing records from primary storage. [2025-10-20T22:15:20.410100] [INFO ] [MAIN ] UPS: Loaded 53720 primary data records. [2025-10-20T22:15:20.990539] [DEBUG] [MAIN ] UNP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:20.991677] [INFO ] [MAIN ] UNP: Generated 1794 derivative records from 53362 primary records. [2025-10-20T22:15:20.992159] [DEBUG] [LOCAL ] UNP: Saving derivative data. [2025-10-20T22:15:21.012592] [DEBUG] [LOCAL ] UNP: Derivative data saved (1794 records). [2025-10-20T22:15:21.027026] [INFO ] [MAIN ] UNP: Successfully saved derivative data. [2025-10-20T22:15:21.027446] [DEBUG] [MAIN ] UP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:21.029723] [INFO ] [MAIN ] Derivative Progress: 1708/1877 symbols completed. UNP took 1.2s. Estimated remaining: 1m 3s. Rate: 159.9 symbols/min [2025-10-20T22:15:21.042846] [INFO ] [MAIN ] UP: Generated 1794 derivative records from 44329 primary records. [2025-10-20T22:15:21.043086] [DEBUG] [MAIN ] UPS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:21.043689] [INFO ] [MAIN ] UPS: Generated 1794 derivative records from 53720 primary records. [2025-10-20T22:15:21.043818] [DEBUG] [LOCAL ] UPS: Saving derivative data. [2025-10-20T22:15:21.043765] [DEBUG] [LOCAL ] UP: Saving derivative data. [2025-10-20T22:15:21.073160] [DEBUG] [LOCAL ] UPS: Derivative data saved (1794 records). [2025-10-20T22:15:21.074196] [DEBUG] [LOCAL ] UP: Derivative data saved (1794 records). [2025-10-20T22:15:21.074287] [INFO ] [MAIN ] UPS: Successfully saved derivative data. [2025-10-20T22:15:21.077554] [INFO ] [MAIN ] Derivative Progress: 1709/1877 symbols completed. UPS took 1.1s. Estimated remaining: 1m 3s. Rate: 160.0 symbols/min [2025-10-20T22:15:21.077666] [INFO ] [MAIN ] UP: Successfully saved derivative data. [2025-10-20T22:15:21.080310] [INFO ] [MAIN ] Derivative Progress: 1710/1877 symbols completed. UP took 1.2s. Estimated remaining: 1m 3s. Rate: 160.1 symbols/min [2025-10-20T22:15:21.080788] [INFO ] [MAIN ] USAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:21.080883] [INFO ] [MAIN ] USA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:21.080731] [INFO ] [MAIN ] URI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:21.103968] [DEBUG] [LOCAL ] USAC: Loaded 21832 existing records from primary storage. [2025-10-20T22:15:21.119151] [INFO ] [MAIN ] USAC: Loaded 21832 primary data records. [2025-10-20T22:15:21.213988] [DEBUG] [LOCAL ] URI: Loaded 34532 existing records from primary storage. [2025-10-20T22:15:21.237998] [INFO ] [MAIN ] URI: Loaded 34532 primary data records. [2025-10-20T22:15:21.279832] [DEBUG] [LOCAL ] USA: Loaded 36476 existing records from primary storage. [2025-10-20T22:15:21.327845] [INFO ] [MAIN ] USA: Loaded 36476 primary data records. [2025-10-20T22:15:22.063588] [DEBUG] [MAIN ] USAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:22.064604] [INFO ] [MAIN ] USAC: Generated 1794 derivative records from 21832 primary records. [2025-10-20T22:15:22.064865] [DEBUG] [LOCAL ] USAC: Saving derivative data. [2025-10-20T22:15:22.096174] [DEBUG] [LOCAL ] USAC: Derivative data saved (1794 records). [2025-10-20T22:15:22.096373] [INFO ] [MAIN ] USAC: Successfully saved derivative data. [2025-10-20T22:15:22.096104] [DEBUG] [MAIN ] URI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:22.099826] [INFO ] [MAIN ] Derivative Progress: 1711/1877 symbols completed. USAC took 1.0s. Estimated remaining: 1m 2s. Rate: 159.9 symbols/min [2025-10-20T22:15:22.097164] [INFO ] [MAIN ] URI: Generated 1794 derivative records from 34532 primary records. [2025-10-20T22:15:22.100923] [INFO ] [MAIN ] USB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:22.101848] [DEBUG] [LOCAL ] URI: Saving derivative data. [2025-10-20T22:15:22.116356] [DEBUG] [MAIN ] USA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:22.128778] [INFO ] [MAIN ] USA: Generated 1794 derivative records from 36476 primary records. [2025-10-20T22:15:22.129407] [DEBUG] [LOCAL ] USA: Saving derivative data. [2025-10-20T22:15:22.133533] [DEBUG] [LOCAL ] URI: Derivative data saved (1794 records). [2025-10-20T22:15:22.144746] [INFO ] [MAIN ] URI: Successfully saved derivative data. [2025-10-20T22:15:22.148407] [INFO ] [MAIN ] Derivative Progress: 1712/1877 symbols completed. URI took 1.1s. Estimated remaining: 1m 2s. Rate: 160.0 symbols/min [2025-10-20T22:15:22.148589] [INFO ] [MAIN ] USFD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:22.150275] [DEBUG] [LOCAL ] USA: Derivative data saved (1794 records). [2025-10-20T22:15:22.150733] [INFO ] [MAIN ] USA: Successfully saved derivative data. [2025-10-20T22:15:22.152857] [INFO ] [MAIN ] Derivative Progress: 1713/1877 symbols completed. USA took 1.1s. Estimated remaining: 1m 1s. Rate: 160.1 symbols/min [2025-10-20T22:15:22.153626] [INFO ] [MAIN ] USNA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:22.201116] [DEBUG] [LOCAL ] USNA: Loaded 18032 existing records from primary storage. [2025-10-20T22:15:22.202117] [DEBUG] [LOCAL ] USB: Loaded 53688 existing records from primary storage. [2025-10-20T22:15:22.202239] [INFO ] [MAIN ] USB: Loaded 53688 primary data records. [2025-10-20T22:15:22.202414] [INFO ] [MAIN ] USNA: Loaded 18032 primary data records. [2025-10-20T22:15:22.514769] [DEBUG] [LOCAL ] USFD: Loaded 51212 existing records from primary storage. [2025-10-20T22:15:22.648010] [INFO ] [MAIN ] USFD: Loaded 51212 primary data records. [2025-10-20T22:15:23.174816] [DEBUG] [MAIN ] USB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:23.188284] [INFO ] [MAIN ] USB: Generated 1794 derivative records from 53688 primary records. [2025-10-20T22:15:23.188603] [DEBUG] [LOCAL ] USB: Saving derivative data. [2025-10-20T22:15:23.189501] [DEBUG] [MAIN ] USNA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:23.191163] [INFO ] [MAIN ] USNA: Generated 1794 derivative records from 18032 primary records. [2025-10-20T22:15:23.191543] [DEBUG] [LOCAL ] USNA: Saving derivative data. [2025-10-20T22:15:23.218793] [DEBUG] [LOCAL ] USB: Derivative data saved (1794 records). [2025-10-20T22:15:23.226898] [DEBUG] [LOCAL ] USNA: Derivative data saved (1794 records). [2025-10-20T22:15:23.227099] [INFO ] [MAIN ] USNA: Successfully saved derivative data. [2025-10-20T22:15:23.229472] [INFO ] [MAIN ] Derivative Progress: 1714/1877 symbols completed. USNA took 1.1s. Estimated remaining: 1m 1s. Rate: 159.9 symbols/min [2025-10-20T22:15:23.229899] [INFO ] [MAIN ] USPH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:23.229799] [INFO ] [MAIN ] USB: Successfully saved derivative data. [2025-10-20T22:15:23.233497] [INFO ] [MAIN ] Derivative Progress: 1715/1877 symbols completed. USB took 1.1s. Estimated remaining: 1m 1s. Rate: 160.0 symbols/min [2025-10-20T22:15:23.235234] [INFO ] [MAIN ] UTF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:23.249126] [DEBUG] [MAIN ] USFD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:23.249750] [INFO ] [MAIN ] USFD: Generated 1794 derivative records from 51212 primary records. [2025-10-20T22:15:23.249979] [DEBUG] [LOCAL ] USFD: Saving derivative data. [2025-10-20T22:15:23.269909] [DEBUG] [LOCAL ] USFD: Derivative data saved (1794 records). [2025-10-20T22:15:23.269989] [INFO ] [MAIN ] USFD: Successfully saved derivative data. [2025-10-20T22:15:23.272583] [INFO ] [MAIN ] Derivative Progress: 1716/1877 symbols completed. USFD took 1.1s. Estimated remaining: 1m 0s. Rate: 160.1 symbols/min [2025-10-20T22:15:23.279117] [INFO ] [MAIN ] UTI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:23.282209] [DEBUG] [LOCAL ] USPH: Loaded 17179 existing records from primary storage. [2025-10-20T22:15:23.282369] [INFO ] [MAIN ] USPH: Loaded 17179 primary data records. [2025-10-20T22:15:23.430081] [DEBUG] [LOCAL ] UTF: Loaded 30922 existing records from primary storage. [2025-10-20T22:15:23.444804] [INFO ] [MAIN ] UTF: Loaded 30922 primary data records. [2025-10-20T22:15:23.715880] [DEBUG] [LOCAL ] UTI: Loaded 40371 existing records from primary storage. [2025-10-20T22:15:23.738864] [INFO ] [MAIN ] UTI: Loaded 40371 primary data records. [2025-10-20T22:15:24.283342] [DEBUG] [MAIN ] USPH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:24.284476] [INFO ] [MAIN ] USPH: Generated 1794 derivative records from 17179 primary records. [2025-10-20T22:15:24.284684] [DEBUG] [LOCAL ] USPH: Saving derivative data. [2025-10-20T22:15:24.305678] [DEBUG] [LOCAL ] USPH: Derivative data saved (1794 records). [2025-10-20T22:15:24.319089] [INFO ] [MAIN ] USPH: Successfully saved derivative data. [2025-10-20T22:15:24.319028] [DEBUG] [MAIN ] UTF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:24.322237] [INFO ] [MAIN ] Derivative Progress: 1717/1877 symbols completed. USPH took 1.1s. Estimated remaining: 1m 0s. Rate: 159.9 symbols/min [2025-10-20T22:15:24.322407] [INFO ] [MAIN ] UTL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:24.320220] [INFO ] [MAIN ] UTF: Generated 1794 derivative records from 30922 primary records. [2025-10-20T22:15:24.322824] [DEBUG] [LOCAL ] UTF: Saving derivative data. [2025-10-20T22:15:24.341059] [DEBUG] [LOCAL ] UTF: Derivative data saved (1794 records). [2025-10-20T22:15:24.341307] [INFO ] [MAIN ] UTF: Successfully saved derivative data. [2025-10-20T22:15:24.343476] [INFO ] [MAIN ] Derivative Progress: 1718/1877 symbols completed. UTF took 1.1s. Estimated remaining: 59.6s. Rate: 160.0 symbols/min [2025-10-20T22:15:24.349263] [INFO ] [MAIN ] UTZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:24.367804] [DEBUG] [LOCAL ] UTL: Loaded 11878 existing records from primary storage. [2025-10-20T22:15:24.368425] [INFO ] [MAIN ] UTL: Loaded 11878 primary data records. [2025-10-20T22:15:24.368365] [DEBUG] [MAIN ] UTI: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:24.381281] [INFO ] [MAIN ] UTI: Generated 1793 derivative records from 40371 primary records. [2025-10-20T22:15:24.410953] [DEBUG] [LOCAL ] UTI: Saving derivative data. [2025-10-20T22:15:24.537196] [DEBUG] [LOCAL ] UTI: Derivative data saved (1793 records). [2025-10-20T22:15:24.574864] [INFO ] [MAIN ] UTI: Successfully saved derivative data. [2025-10-20T22:15:24.599851] [INFO ] [MAIN ] Derivative Progress: 1719/1877 symbols completed. UTI took 1.3s. Estimated remaining: 59.2s. Rate: 160.1 symbols/min [2025-10-20T22:15:24.600189] [INFO ] [MAIN ] UVE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:24.771928] [DEBUG] [LOCAL ] UTZ: Loaded 44488 existing records from primary storage. [2025-10-20T22:15:24.772408] [INFO ] [MAIN ] UTZ: Loaded 44488 primary data records. [2025-10-20T22:15:24.895458] [DEBUG] [LOCAL ] UVE: Loaded 22011 existing records from primary storage. [2025-10-20T22:15:24.895751] [INFO ] [MAIN ] UVE: Loaded 22011 primary data records. [2025-10-20T22:15:25.441226] [DEBUG] [MAIN ] UTL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:25.443438] [INFO ] [MAIN ] UTL: Generated 1794 derivative records from 11878 primary records. [2025-10-20T22:15:25.444611] [DEBUG] [LOCAL ] UTL: Saving derivative data. [2025-10-20T22:15:25.462476] [DEBUG] [LOCAL ] UTL: Derivative data saved (1794 records). [2025-10-20T22:15:25.463424] [INFO ] [MAIN ] UTL: Successfully saved derivative data. [2025-10-20T22:15:25.467219] [INFO ] [MAIN ] Derivative Progress: 1720/1877 symbols completed. UTL took 1.1s. Estimated remaining: 58.9s. Rate: 159.9 symbols/min [2025-10-20T22:15:25.468253] [INFO ] [MAIN ] UVV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:25.482439] [DEBUG] [MAIN ] UTZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:25.483175] [INFO ] [MAIN ] UTZ: Generated 1794 derivative records from 44488 primary records. [2025-10-20T22:15:25.483362] [DEBUG] [LOCAL ] UTZ: Saving derivative data. [2025-10-20T22:15:25.499853] [DEBUG] [LOCAL ] UTZ: Derivative data saved (1794 records). [2025-10-20T22:15:25.500021] [INFO ] [MAIN ] UTZ: Successfully saved derivative data. [2025-10-20T22:15:25.502033] [INFO ] [MAIN ] Derivative Progress: 1721/1877 symbols completed. UTZ took 1.2s. Estimated remaining: 58.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:25.503230] [INFO ] [MAIN ] UWMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:25.523745] [DEBUG] [MAIN ] UVE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:25.524557] [INFO ] [MAIN ] UVE: Generated 1794 derivative records from 22011 primary records. [2025-10-20T22:15:25.524697] [DEBUG] [LOCAL ] UVE: Saving derivative data. [2025-10-20T22:15:25.533573] [DEBUG] [LOCAL ] UVV: Loaded 22837 existing records from primary storage. [2025-10-20T22:15:25.544141] [INFO ] [MAIN ] UVV: Loaded 22837 primary data records. [2025-10-20T22:15:25.570002] [DEBUG] [LOCAL ] UVE: Derivative data saved (1794 records). [2025-10-20T22:15:25.585618] [INFO ] [MAIN ] UVE: Successfully saved derivative data. [2025-10-20T22:15:25.620937] [INFO ] [MAIN ] Derivative Progress: 1722/1877 symbols completed. UVE took 1.0s. Estimated remaining: 58.1s. Rate: 160.1 symbols/min [2025-10-20T22:15:25.644029] [INFO ] [MAIN ] UZD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:25.766471] [DEBUG] [LOCAL ] UZD: Loaded 5930 existing records from primary storage. [2025-10-20T22:15:25.826864] [DEBUG] [LOCAL ] UWMC: Loaded 52957 existing records from primary storage. [2025-10-20T22:15:25.825764] [INFO ] [MAIN ] UZD: Loaded 5930 primary data records. [2025-10-20T22:15:25.884147] [INFO ] [MAIN ] UWMC: Loaded 52957 primary data records. [2025-10-20T22:15:26.574084] [DEBUG] [MAIN ] UVV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:26.575742] [INFO ] [MAIN ] UVV: Generated 1794 derivative records from 22837 primary records. [2025-10-20T22:15:26.576071] [DEBUG] [LOCAL ] UVV: Saving derivative data. [2025-10-20T22:15:26.593276] [DEBUG] [LOCAL ] UVV: Derivative data saved (1794 records). [2025-10-20T22:15:26.594401] [INFO ] [MAIN ] UVV: Successfully saved derivative data. [2025-10-20T22:15:26.596415] [INFO ] [MAIN ] Derivative Progress: 1723/1877 symbols completed. UVV took 1.1s. Estimated remaining: 57.8s. Rate: 159.9 symbols/min [2025-10-20T22:15:26.616559] [DEBUG] [MAIN ] UZD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:26.617816] [INFO ] [MAIN ] UZD: Generated 1794 derivative records from 5930 primary records. [2025-10-20T22:15:26.617927] [DEBUG] [LOCAL ] UZD: Saving derivative data. [2025-10-20T22:15:26.635416] [DEBUG] [LOCAL ] UZD: Derivative data saved (1794 records). [2025-10-20T22:15:26.636163] [INFO ] [MAIN ] UZD: Successfully saved derivative data. [2025-10-20T22:15:26.638488] [INFO ] [MAIN ] Derivative Progress: 1724/1877 symbols completed. UZD took 1.0s. Estimated remaining: 57.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:26.651739] [DEBUG] [MAIN ] UWMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:26.652307] [INFO ] [MAIN ] UWMC: Generated 1794 derivative records from 52957 primary records. [2025-10-20T22:15:26.652368] [DEBUG] [LOCAL ] UWMC: Saving derivative data. [2025-10-20T22:15:26.667991] [DEBUG] [LOCAL ] UWMC: Derivative data saved (1794 records). [2025-10-20T22:15:26.668110] [INFO ] [MAIN ] UWMC: Successfully saved derivative data. [2025-10-20T22:15:26.670335] [INFO ] [MAIN ] Derivative Progress: 1725/1877 symbols completed. UWMC took 1.2s. Estimated remaining: 57.0s. Rate: 160.1 symbols/min [2025-10-20T22:15:26.670763] [INFO ] [MAIN ] UZE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:26.670863] [INFO ] [MAIN ] V: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:26.670964] [INFO ] [MAIN ] VAC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:26.681352] [DEBUG] [LOCAL ] UZE: Loaded 7056 existing records from primary storage. [2025-10-20T22:15:26.681476] [INFO ] [MAIN ] UZE: Loaded 7056 primary data records. [2025-10-20T22:15:27.002164] [DEBUG] [LOCAL ] VAC: Loaded 33877 existing records from primary storage. [2025-10-20T22:15:27.015870] [INFO ] [MAIN ] VAC: Loaded 33877 primary data records. [2025-10-20T22:15:27.143496] [DEBUG] [LOCAL ] V: Loaded 53752 existing records from primary storage. [2025-10-20T22:15:27.150142] [INFO ] [MAIN ] V: Loaded 53752 primary data records. [2025-10-20T22:15:27.661158] [DEBUG] [MAIN ] UZE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:27.661932] [INFO ] [MAIN ] UZE: Generated 1794 derivative records from 7056 primary records. [2025-10-20T22:15:27.662084] [DEBUG] [LOCAL ] UZE: Saving derivative data. [2025-10-20T22:15:27.678552] [DEBUG] [LOCAL ] UZE: Derivative data saved (1794 records). [2025-10-20T22:15:27.678927] [INFO ] [MAIN ] UZE: Successfully saved derivative data. [2025-10-20T22:15:27.681268] [INFO ] [MAIN ] Derivative Progress: 1726/1877 symbols completed. UZE took 1.0s. Estimated remaining: 56.6s. Rate: 159.9 symbols/min [2025-10-20T22:15:27.681414] [INFO ] [MAIN ] VAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:27.702833] [DEBUG] [MAIN ] VAC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:27.703496] [INFO ] [MAIN ] VAC: Generated 1794 derivative records from 33877 primary records. [2025-10-20T22:15:27.703831] [DEBUG] [LOCAL ] VAC: Saving derivative data. [2025-10-20T22:15:27.723387] [DEBUG] [LOCAL ] VAC: Derivative data saved (1794 records). [2025-10-20T22:15:27.723728] [INFO ] [MAIN ] VAC: Successfully saved derivative data. [2025-10-20T22:15:27.725541] [INFO ] [MAIN ] Derivative Progress: 1727/1877 symbols completed. VAC took 1.1s. Estimated remaining: 56.2s. Rate: 160.0 symbols/min [2025-10-20T22:15:27.725818] [INFO ] [MAIN ] VALE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:27.749156] [DEBUG] [MAIN ] V: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:27.749947] [INFO ] [MAIN ] V: Generated 1794 derivative records from 53752 primary records. [2025-10-20T22:15:27.750097] [DEBUG] [LOCAL ] V: Saving derivative data. [2025-10-20T22:15:27.767353] [DEBUG] [LOCAL ] V: Derivative data saved (1794 records). [2025-10-20T22:15:27.767598] [INFO ] [MAIN ] V: Successfully saved derivative data. [2025-10-20T22:15:27.770066] [INFO ] [MAIN ] Derivative Progress: 1728/1877 symbols completed. V took 1.1s. Estimated remaining: 55.8s. Rate: 160.1 symbols/min [2025-10-20T22:15:27.785219] [INFO ] [MAIN ] VATE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:27.786472] [DEBUG] [LOCAL ] VAL: Loaded 50021 existing records from primary storage. [2025-10-20T22:15:27.786567] [INFO ] [MAIN ] VAL: Loaded 50021 primary data records. [2025-10-20T22:15:27.884077] [DEBUG] [LOCAL ] VATE: Loaded 5107 existing records from primary storage. [2025-10-20T22:15:27.895010] [INFO ] [MAIN ] VATE: Loaded 5107 primary data records. [2025-10-20T22:15:28.171400] [DEBUG] [LOCAL ] VALE: Loaded 53693 existing records from primary storage. [2025-10-20T22:15:28.206606] [INFO ] [MAIN ] VALE: Loaded 53693 primary data records. [2025-10-20T22:15:28.781687] [DEBUG] [MAIN ] VAL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:28.794356] [DEBUG] [MAIN ] VATE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:28.795112] [INFO ] [MAIN ] VAL: Generated 1794 derivative records from 50021 primary records. [2025-10-20T22:15:28.795868] [INFO ] [MAIN ] VATE: Generated 1794 derivative records from 5107 primary records. [2025-10-20T22:15:28.796018] [DEBUG] [LOCAL ] VAL: Saving derivative data. [2025-10-20T22:15:28.796183] [DEBUG] [LOCAL ] VATE: Saving derivative data. [2025-10-20T22:15:28.820915] [DEBUG] [LOCAL ] VATE: Derivative data saved (1794 records). [2025-10-20T22:15:28.821032] [INFO ] [MAIN ] VATE: Successfully saved derivative data. [2025-10-20T22:15:28.830193] [INFO ] [MAIN ] Derivative Progress: 1729/1877 symbols completed. VATE took 1.0s. Estimated remaining: 55.5s. Rate: 159.9 symbols/min [2025-10-20T22:15:28.830701] [INFO ] [MAIN ] VBF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:28.831625] [DEBUG] [LOCAL ] VAL: Derivative data saved (1794 records). [2025-10-20T22:15:28.832060] [INFO ] [MAIN ] VAL: Successfully saved derivative data. [2025-10-20T22:15:28.834401] [INFO ] [MAIN ] Derivative Progress: 1730/1877 symbols completed. VAL took 1.2s. Estimated remaining: 55.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:28.834609] [INFO ] [MAIN ] VCV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:28.847833] [DEBUG] [MAIN ] VALE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:28.850586] [INFO ] [MAIN ] VALE: Generated 1794 derivative records from 53693 primary records. [2025-10-20T22:15:28.850739] [DEBUG] [LOCAL ] VALE: Saving derivative data. [2025-10-20T22:15:28.872714] [DEBUG] [LOCAL ] VBF: Loaded 7267 existing records from primary storage. [2025-10-20T22:15:28.873769] [DEBUG] [LOCAL ] VALE: Derivative data saved (1794 records). [2025-10-20T22:15:28.873960] [INFO ] [MAIN ] VBF: Loaded 7267 primary data records. [2025-10-20T22:15:28.874035] [INFO ] [MAIN ] VALE: Successfully saved derivative data. [2025-10-20T22:15:28.876193] [INFO ] [MAIN ] Derivative Progress: 1731/1877 symbols completed. VALE took 1.2s. Estimated remaining: 54.7s. Rate: 160.1 symbols/min [2025-10-20T22:15:28.907040] [INFO ] [MAIN ] VEEV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:28.936926] [DEBUG] [LOCAL ] VCV: Loaded 13517 existing records from primary storage. [2025-10-20T22:15:28.962701] [INFO ] [MAIN ] VCV: Loaded 13517 primary data records. [2025-10-20T22:15:29.498296] [DEBUG] [LOCAL ] VEEV: Loaded 47245 existing records from primary storage. [2025-10-20T22:15:29.498930] [INFO ] [MAIN ] VEEV: Loaded 47245 primary data records. [2025-10-20T22:15:29.824314] [DEBUG] [MAIN ] VBF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:29.826976] [INFO ] [MAIN ] VBF: Generated 1794 derivative records from 7267 primary records. [2025-10-20T22:15:29.828624] [DEBUG] [LOCAL ] VBF: Saving derivative data. [2025-10-20T22:15:29.864214] [DEBUG] [MAIN ] VCV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:29.866633] [DEBUG] [LOCAL ] VBF: Derivative data saved (1794 records). [2025-10-20T22:15:29.865792] [INFO ] [MAIN ] VCV: Generated 1794 derivative records from 13517 primary records. [2025-10-20T22:15:29.867133] [INFO ] [MAIN ] VBF: Successfully saved derivative data. [2025-10-20T22:15:29.866795] [DEBUG] [LOCAL ] VCV: Saving derivative data. [2025-10-20T22:15:29.870477] [INFO ] [MAIN ] Derivative Progress: 1732/1877 symbols completed. VBF took 1.0s. Estimated remaining: 54.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:29.871799] [INFO ] [MAIN ] VEL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:29.887990] [DEBUG] [LOCAL ] VCV: Derivative data saved (1794 records). [2025-10-20T22:15:29.888166] [INFO ] [MAIN ] VCV: Successfully saved derivative data. [2025-10-20T22:15:29.891085] [INFO ] [MAIN ] Derivative Progress: 1733/1877 symbols completed. VCV took 1.1s. Estimated remaining: 54.0s. Rate: 160.0 symbols/min [2025-10-20T22:15:29.891313] [INFO ] [MAIN ] VET: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:29.904374] [DEBUG] [LOCAL ] VEL: Loaded 12088 existing records from primary storage. [2025-10-20T22:15:29.904741] [INFO ] [MAIN ] VEL: Loaded 12088 primary data records. [2025-10-20T22:15:29.938691] [DEBUG] [MAIN ] VEEV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:29.962028] [INFO ] [MAIN ] VEEV: Generated 1794 derivative records from 47245 primary records. [2025-10-20T22:15:29.989928] [DEBUG] [LOCAL ] VEEV: Saving derivative data. [2025-10-20T22:15:30.054982] [DEBUG] [LOCAL ] VEEV: Derivative data saved (1794 records). [2025-10-20T22:15:30.065870] [INFO ] [MAIN ] VEEV: Successfully saved derivative data. [2025-10-20T22:15:30.092862] [INFO ] [MAIN ] Derivative Progress: 1734/1877 symbols completed. VEEV took 1.2s. Estimated remaining: 53.6s. Rate: 160.1 symbols/min [2025-10-20T22:15:30.124402] [INFO ] [MAIN ] VFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:30.282412] [DEBUG] [LOCAL ] VET: Loaded 44200 existing records from primary storage. [2025-10-20T22:15:30.282797] [INFO ] [MAIN ] VET: Loaded 44200 primary data records. [2025-10-20T22:15:30.666280] [DEBUG] [LOCAL ] VFC: Loaded 53607 existing records from primary storage. [2025-10-20T22:15:30.666829] [INFO ] [MAIN ] VFC: Loaded 53607 primary data records. [2025-10-20T22:15:30.760418] [DEBUG] [MAIN ] VEL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:30.790489] [INFO ] [MAIN ] VEL: Generated 1794 derivative records from 12088 primary records. [2025-10-20T22:15:30.790794] [DEBUG] [LOCAL ] VEL: Saving derivative data. [2025-10-20T22:15:31.010953] [DEBUG] [LOCAL ] VEL: Derivative data saved (1794 records). [2025-10-20T22:15:31.011231] [INFO ] [MAIN ] VEL: Successfully saved derivative data. [2025-10-20T22:15:31.013363] [INFO ] [MAIN ] Derivative Progress: 1735/1877 symbols completed. VEL took 1.1s. Estimated remaining: 53.3s. Rate: 160.0 symbols/min [2025-10-20T22:15:31.014224] [INFO ] [MAIN ] VGI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:31.025581] [DEBUG] [LOCAL ] VGI: Loaded 5227 existing records from primary storage. [2025-10-20T22:15:31.026591] [INFO ] [MAIN ] VGI: Loaded 5227 primary data records. [2025-10-20T22:15:31.076823] [DEBUG] [MAIN ] VET: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:31.097895] [INFO ] [MAIN ] VET: Generated 1794 derivative records from 44200 primary records. [2025-10-20T22:15:31.112765] [DEBUG] [LOCAL ] VET: Saving derivative data. [2025-10-20T22:15:31.186675] [DEBUG] [LOCAL ] VET: Derivative data saved (1794 records). [2025-10-20T22:15:31.192161] [INFO ] [MAIN ] VET: Successfully saved derivative data. [2025-10-20T22:15:31.208788] [INFO ] [MAIN ] Derivative Progress: 1736/1877 symbols completed. VET took 1.3s. Estimated remaining: 52.9s. Rate: 160.0 symbols/min [2025-10-20T22:15:31.238802] [INFO ] [MAIN ] VGM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:31.259234] [DEBUG] [MAIN ] VFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:31.259901] [INFO ] [MAIN ] VFC: Generated 1794 derivative records from 53607 primary records. [2025-10-20T22:15:31.260281] [DEBUG] [LOCAL ] VFC: Saving derivative data. [2025-10-20T22:15:31.391520] [DEBUG] [LOCAL ] VFC: Derivative data saved (1794 records). [2025-10-20T22:15:31.392152] [INFO ] [MAIN ] VFC: Successfully saved derivative data. [2025-10-20T22:15:31.394531] [INFO ] [MAIN ] Derivative Progress: 1737/1877 symbols completed. VFC took 1.3s. Estimated remaining: 52.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:31.394735] [INFO ] [MAIN ] VHI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:31.409849] [DEBUG] [LOCAL ] VGM: Loaded 18689 existing records from primary storage. [2025-10-20T22:15:31.410217] [INFO ] [MAIN ] VGM: Loaded 18689 primary data records. [2025-10-20T22:15:31.433573] [DEBUG] [LOCAL ] VHI: Loaded 2733 existing records from primary storage. [2025-10-20T22:15:31.468187] [INFO ] [MAIN ] VHI: Loaded 2733 primary data records. [2025-10-20T22:15:31.894232] [DEBUG] [MAIN ] VGI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:32.014989] [INFO ] [MAIN ] VGI: Generated 1794 derivative records from 5227 primary records. [2025-10-20T22:15:32.015803] [DEBUG] [LOCAL ] VGI: Saving derivative data. [2025-10-20T22:15:32.033094] [DEBUG] [LOCAL ] VGI: Derivative data saved (1794 records). [2025-10-20T22:15:32.033683] [INFO ] [MAIN ] VGI: Successfully saved derivative data. [2025-10-20T22:15:32.036374] [INFO ] [MAIN ] Derivative Progress: 1738/1877 symbols completed. VGI took 1.0s. Estimated remaining: 52.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:32.060930] [DEBUG] [MAIN ] VGM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:32.075124] [INFO ] [MAIN ] VGM: Generated 1794 derivative records from 18689 primary records. [2025-10-20T22:15:32.075193] [DEBUG] [LOCAL ] VGM: Saving derivative data. [2025-10-20T22:15:32.075040] [DEBUG] [MAIN ] VHI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:32.076333] [INFO ] [MAIN ] VHI: Generated 1794 derivative records from 2733 primary records. [2025-10-20T22:15:32.076586] [DEBUG] [LOCAL ] VHI: Saving derivative data. [2025-10-20T22:15:32.101770] [DEBUG] [LOCAL ] VGM: Derivative data saved (1794 records). [2025-10-20T22:15:32.102323] [INFO ] [MAIN ] VGM: Successfully saved derivative data. [2025-10-20T22:15:32.106039] [INFO ] [MAIN ] Derivative Progress: 1739/1877 symbols completed. VGM took 0.9s. Estimated remaining: 51.7s. Rate: 160.1 symbols/min [2025-10-20T22:15:32.108018] [DEBUG] [LOCAL ] VHI: Derivative data saved (1794 records). [2025-10-20T22:15:32.108082] [INFO ] [MAIN ] VHI: Successfully saved derivative data. [2025-10-20T22:15:32.110357] [INFO ] [MAIN ] Derivative Progress: 1740/1877 symbols completed. VHI took 0.7s. Estimated remaining: 51.3s. Rate: 160.1 symbols/min [2025-10-20T22:15:32.110784] [INFO ] [MAIN ] VIK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:32.110894] [INFO ] [MAIN ] VIPS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:32.110737] [INFO ] [MAIN ] VICI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:32.200338] [DEBUG] [LOCAL ] VICI: Loaded 53653 existing records from primary storage. [2025-10-20T22:15:32.201045] [INFO ] [MAIN ] VICI: Loaded 53653 primary data records. [2025-10-20T22:15:32.213303] [DEBUG] [LOCAL ] VIK: Loaded 52392 existing records from primary storage. [2025-10-20T22:15:32.227765] [INFO ] [MAIN ] VIK: Loaded 52392 primary data records. [2025-10-20T22:15:32.327401] [DEBUG] [LOCAL ] VIPS: Loaded 51269 existing records from primary storage. [2025-10-20T22:15:32.408283] [INFO ] [MAIN ] VIPS: Loaded 51269 primary data records. [2025-10-20T22:15:33.159730] [DEBUG] [MAIN ] VICI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:33.161714] [INFO ] [MAIN ] VICI: Generated 1794 derivative records from 53653 primary records. [2025-10-20T22:15:33.162519] [DEBUG] [LOCAL ] VICI: Saving derivative data. [2025-10-20T22:15:33.206151] [DEBUG] [MAIN ] VIK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:33.205601] [DEBUG] [LOCAL ] VICI: Derivative data saved (1794 records). [2025-10-20T22:15:33.207108] [INFO ] [MAIN ] VIK: Generated 1794 derivative records from 52392 primary records. [2025-10-20T22:15:33.207294] [DEBUG] [LOCAL ] VIK: Saving derivative data. [2025-10-20T22:15:33.207175] [INFO ] [MAIN ] VICI: Successfully saved derivative data. [2025-10-20T22:15:33.209105] [INFO ] [MAIN ] Derivative Progress: 1741/1877 symbols completed. VICI took 1.1s. Estimated remaining: 51.0s. Rate: 160.0 symbols/min [2025-10-20T22:15:33.210181] [DEBUG] [MAIN ] VIPS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:33.210574] [INFO ] [MAIN ] VIRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:33.218944] [INFO ] [MAIN ] VIPS: Generated 1794 derivative records from 51269 primary records. [2025-10-20T22:15:33.224010] [DEBUG] [LOCAL ] VIPS: Saving derivative data. [2025-10-20T22:15:33.229016] [DEBUG] [LOCAL ] VIK: Derivative data saved (1794 records). [2025-10-20T22:15:33.229361] [INFO ] [MAIN ] VIK: Successfully saved derivative data. [2025-10-20T22:15:33.238986] [INFO ] [MAIN ] Derivative Progress: 1742/1877 symbols completed. VIK took 1.1s. Estimated remaining: 50.6s. Rate: 160.1 symbols/min [2025-10-20T22:15:33.242819] [INFO ] [MAIN ] VIST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:33.247442] [DEBUG] [LOCAL ] VIPS: Derivative data saved (1794 records). [2025-10-20T22:15:33.247581] [INFO ] [MAIN ] VIPS: Successfully saved derivative data. [2025-10-20T22:15:33.250121] [INFO ] [MAIN ] Derivative Progress: 1743/1877 symbols completed. VIPS took 1.1s. Estimated remaining: 50.2s. Rate: 160.1 symbols/min [2025-10-20T22:15:33.250308] [INFO ] [MAIN ] VIV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:33.310043] [DEBUG] [LOCAL ] VIRT: Loaded 46585 existing records from primary storage. [2025-10-20T22:15:33.310202] [INFO ] [MAIN ] VIRT: Loaded 46585 primary data records. [2025-10-20T22:15:33.439219] [DEBUG] [LOCAL ] VIV: Loaded 44697 existing records from primary storage. [2025-10-20T22:15:33.456070] [INFO ] [MAIN ] VIV: Loaded 44697 primary data records. [2025-10-20T22:15:33.467282] [DEBUG] [LOCAL ] VIST: Loaded 46676 existing records from primary storage. [2025-10-20T22:15:33.499340] [INFO ] [MAIN ] VIST: Loaded 46676 primary data records. [2025-10-20T22:15:34.249835] [DEBUG] [MAIN ] VIRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:34.251767] [INFO ] [MAIN ] VIRT: Generated 1794 derivative records from 46585 primary records. [2025-10-20T22:15:34.251884] [DEBUG] [LOCAL ] VIRT: Saving derivative data. [2025-10-20T22:15:34.268359] [DEBUG] [LOCAL ] VIRT: Derivative data saved (1794 records). [2025-10-20T22:15:34.268727] [INFO ] [MAIN ] VIRT: Successfully saved derivative data. [2025-10-20T22:15:34.270443] [INFO ] [MAIN ] Derivative Progress: 1744/1877 symbols completed. VIRT took 1.1s. Estimated remaining: 49.9s. Rate: 160.0 symbols/min [2025-10-20T22:15:34.270739] [INFO ] [MAIN ] VKQ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:34.304794] [DEBUG] [MAIN ] VIST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:34.305042] [DEBUG] [MAIN ] VIV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:34.306468] [INFO ] [MAIN ] VIV: Generated 1794 derivative records from 44697 primary records. [2025-10-20T22:15:34.305887] [INFO ] [MAIN ] VIST: Generated 1794 derivative records from 46676 primary records. [2025-10-20T22:15:34.306554] [DEBUG] [LOCAL ] VIV: Saving derivative data. [2025-10-20T22:15:34.311480] [DEBUG] [LOCAL ] VIST: Saving derivative data. [2025-10-20T22:15:34.349151] [DEBUG] [LOCAL ] VIV: Derivative data saved (1794 records). [2025-10-20T22:15:34.348051] [DEBUG] [LOCAL ] VIST: Derivative data saved (1794 records). [2025-10-20T22:15:34.349279] [INFO ] [MAIN ] VIV: Successfully saved derivative data. [2025-10-20T22:15:34.351295] [INFO ] [MAIN ] Derivative Progress: 1745/1877 symbols completed. VIV took 1.1s. Estimated remaining: 49.5s. Rate: 160.1 symbols/min [2025-10-20T22:15:34.352138] [DEBUG] [LOCAL ] VKQ: Loaded 17880 existing records from primary storage. [2025-10-20T22:15:34.351554] [INFO ] [MAIN ] VIST: Successfully saved derivative data. [2025-10-20T22:15:34.352323] [INFO ] [MAIN ] VKQ: Loaded 17880 primary data records. [2025-10-20T22:15:34.354500] [INFO ] [MAIN ] Derivative Progress: 1746/1877 symbols completed. VIST took 1.1s. Estimated remaining: 49.1s. Rate: 160.1 symbols/min [2025-10-20T22:15:34.351444] [INFO ] [MAIN ] VLN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:34.366209] [INFO ] [MAIN ] VLO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:34.697195] [DEBUG] [LOCAL ] VLN: Loaded 30887 existing records from primary storage. [2025-10-20T22:15:34.698388] [INFO ] [MAIN ] VLN: Loaded 30887 primary data records. [2025-10-20T22:15:34.921691] [DEBUG] [LOCAL ] VLO: Loaded 52982 existing records from primary storage. [2025-10-20T22:15:34.927890] [INFO ] [MAIN ] VLO: Loaded 52982 primary data records. [2025-10-20T22:15:35.257363] [DEBUG] [MAIN ] VKQ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:35.364988] [INFO ] [MAIN ] VKQ: Generated 1794 derivative records from 17880 primary records. [2025-10-20T22:15:35.373425] [DEBUG] [LOCAL ] VKQ: Saving derivative data. [2025-10-20T22:15:35.390753] [DEBUG] [LOCAL ] VKQ: Derivative data saved (1794 records). [2025-10-20T22:15:35.391543] [INFO ] [MAIN ] VKQ: Successfully saved derivative data. [2025-10-20T22:15:35.393935] [INFO ] [MAIN ] Derivative Progress: 1747/1877 symbols completed. VKQ took 1.1s. Estimated remaining: 48.8s. Rate: 160.0 symbols/min [2025-10-20T22:15:35.397389] [INFO ] [MAIN ] VLRS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:35.412567] [DEBUG] [MAIN ] VLN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:35.413443] [INFO ] [MAIN ] VLN: Generated 1794 derivative records from 30887 primary records. [2025-10-20T22:15:35.413576] [DEBUG] [LOCAL ] VLN: Saving derivative data. [2025-10-20T22:15:35.430459] [DEBUG] [LOCAL ] VLN: Derivative data saved (1794 records). [2025-10-20T22:15:35.430793] [INFO ] [MAIN ] VLN: Successfully saved derivative data. [2025-10-20T22:15:35.432689] [INFO ] [MAIN ] Derivative Progress: 1748/1877 symbols completed. VLN took 1.1s. Estimated remaining: 48.4s. Rate: 160.1 symbols/min [2025-10-20T22:15:35.432843] [INFO ] [MAIN ] VLT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:35.459514] [DEBUG] [LOCAL ] VLT: Loaded 4389 existing records from primary storage. [2025-10-20T22:15:35.464661] [INFO ] [MAIN ] VLT: Loaded 4389 primary data records. [2025-10-20T22:15:35.465140] [DEBUG] [MAIN ] VLO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:35.509080] [INFO ] [MAIN ] VLO: Generated 1794 derivative records from 52982 primary records. [2025-10-20T22:15:35.522397] [DEBUG] [LOCAL ] VLO: Saving derivative data. [2025-10-20T22:15:35.561971] [DEBUG] [LOCAL ] VLRS: Loaded 39472 existing records from primary storage. [2025-10-20T22:15:35.574808] [INFO ] [MAIN ] VLRS: Loaded 39472 primary data records. [2025-10-20T22:15:35.658606] [DEBUG] [LOCAL ] VLO: Derivative data saved (1794 records). [2025-10-20T22:15:35.669229] [INFO ] [MAIN ] VLO: Successfully saved derivative data. [2025-10-20T22:15:35.686205] [INFO ] [MAIN ] Derivative Progress: 1749/1877 symbols completed. VLO took 1.3s. Estimated remaining: 48.0s. Rate: 160.1 symbols/min [2025-10-20T22:15:35.708377] [INFO ] [MAIN ] VLTO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:36.168916] [DEBUG] [MAIN ] VLT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:36.169865] [DEBUG] [LOCAL ] VLTO: Loaded 46138 existing records from primary storage. [2025-10-20T22:15:36.170771] [INFO ] [MAIN ] VLT: Generated 1794 derivative records from 4389 primary records. [2025-10-20T22:15:36.171051] [DEBUG] [LOCAL ] VLT: Saving derivative data. [2025-10-20T22:15:36.170998] [INFO ] [MAIN ] VLTO: Loaded 46138 primary data records. [2025-10-20T22:15:36.225851] [DEBUG] [LOCAL ] VLT: Derivative data saved (1794 records). [2025-10-20T22:15:36.259104] [INFO ] [MAIN ] VLT: Successfully saved derivative data. [2025-10-20T22:15:36.285521] [INFO ] [MAIN ] Derivative Progress: 1750/1877 symbols completed. VLT took 0.9s. Estimated remaining: 47.6s. Rate: 160.0 symbols/min [2025-10-20T22:15:36.307603] [INFO ] [MAIN ] VMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:36.313796] [DEBUG] [MAIN ] VLRS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:36.325424] [INFO ] [MAIN ] VLRS: Generated 1794 derivative records from 39472 primary records. [2025-10-20T22:15:36.330907] [DEBUG] [LOCAL ] VLRS: Saving derivative data. [2025-10-20T22:15:36.555906] [DEBUG] [LOCAL ] VLRS: Derivative data saved (1794 records). [2025-10-20T22:15:36.556295] [INFO ] [MAIN ] VLRS: Successfully saved derivative data. [2025-10-20T22:15:36.558600] [INFO ] [MAIN ] Derivative Progress: 1751/1877 symbols completed. VLRS took 1.2s. Estimated remaining: 47.2s. Rate: 160.1 symbols/min [2025-10-20T22:15:36.559793] [INFO ] [MAIN ] VMI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:36.595781] [DEBUG] [MAIN ] VLTO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:36.596439] [INFO ] [MAIN ] VLTO: Generated 1794 derivative records from 46138 primary records. [2025-10-20T22:15:36.596517] [DEBUG] [LOCAL ] VLTO: Saving derivative data. [2025-10-20T22:15:36.612328] [DEBUG] [LOCAL ] VLTO: Derivative data saved (1794 records). [2025-10-20T22:15:36.612494] [INFO ] [MAIN ] VLTO: Successfully saved derivative data. [2025-10-20T22:15:36.614410] [INFO ] [MAIN ] Derivative Progress: 1752/1877 symbols completed. VLTO took 0.9s. Estimated remaining: 46.8s. Rate: 160.1 symbols/min [2025-10-20T22:15:36.615723] [INFO ] [MAIN ] VMO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:36.634477] [DEBUG] [LOCAL ] VMI: Loaded 14963 existing records from primary storage. [2025-10-20T22:15:36.634701] [INFO ] [MAIN ] VMI: Loaded 14963 primary data records. [2025-10-20T22:15:36.635521] [DEBUG] [LOCAL ] VMC: Loaded 42371 existing records from primary storage. [2025-10-20T22:15:36.656839] [INFO ] [MAIN ] VMC: Loaded 42371 primary data records. [2025-10-20T22:15:36.853418] [DEBUG] [LOCAL ] VMO: Loaded 20784 existing records from primary storage. [2025-10-20T22:15:36.898790] [INFO ] [MAIN ] VMO: Loaded 20784 primary data records. [2025-10-20T22:15:37.617319] [DEBUG] [MAIN ] VMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:37.617071] [DEBUG] [MAIN ] VMI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:37.618243] [INFO ] [MAIN ] VMC: Generated 1794 derivative records from 42371 primary records. [2025-10-20T22:15:37.618456] [DEBUG] [LOCAL ] VMC: Saving derivative data. [2025-10-20T22:15:37.620049] [INFO ] [MAIN ] VMI: Generated 1794 derivative records from 14963 primary records. [2025-10-20T22:15:37.620151] [DEBUG] [LOCAL ] VMI: Saving derivative data. [2025-10-20T22:15:37.650948] [DEBUG] [LOCAL ] VMC: Derivative data saved (1794 records). [2025-10-20T22:15:37.651083] [INFO ] [MAIN ] VMC: Successfully saved derivative data. [2025-10-20T22:15:37.654317] [INFO ] [MAIN ] Derivative Progress: 1753/1877 symbols completed. VMC took 1.3s. Estimated remaining: 46.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:37.655758] [DEBUG] [LOCAL ] VMI: Derivative data saved (1794 records). [2025-10-20T22:15:37.655902] [INFO ] [MAIN ] VMI: Successfully saved derivative data. [2025-10-20T22:15:37.657743] [INFO ] [MAIN ] Derivative Progress: 1754/1877 symbols completed. VMI took 1.1s. Estimated remaining: 46.1s. Rate: 160.1 symbols/min [2025-10-20T22:15:37.670411] [DEBUG] [MAIN ] VMO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:37.671086] [INFO ] [MAIN ] VMO: Generated 1794 derivative records from 20784 primary records. [2025-10-20T22:15:37.671149] [DEBUG] [LOCAL ] VMO: Saving derivative data. [2025-10-20T22:15:37.686190] [DEBUG] [LOCAL ] VMO: Derivative data saved (1794 records). [2025-10-20T22:15:37.686249] [INFO ] [MAIN ] VMO: Successfully saved derivative data. [2025-10-20T22:15:37.688626] [INFO ] [MAIN ] Derivative Progress: 1755/1877 symbols completed. VMO took 1.1s. Estimated remaining: 45.7s. Rate: 160.2 symbols/min [2025-10-20T22:15:37.689513] [INFO ] [MAIN ] VOC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:37.689363] [INFO ] [MAIN ] VNO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:37.689436] [INFO ] [MAIN ] VNT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:37.702870] [DEBUG] [LOCAL ] VOC: Loaded 9011 existing records from primary storage. [2025-10-20T22:15:37.703115] [INFO ] [MAIN ] VOC: Loaded 9011 primary data records. [2025-10-20T22:15:38.036838] [DEBUG] [LOCAL ] VNT: Loaded 43986 existing records from primary storage. [2025-10-20T22:15:38.050940] [INFO ] [MAIN ] VNT: Loaded 43986 primary data records. [2025-10-20T22:15:38.064278] [DEBUG] [LOCAL ] VNO: Loaded 48936 existing records from primary storage. [2025-10-20T22:15:38.085997] [INFO ] [MAIN ] VNO: Loaded 48936 primary data records. [2025-10-20T22:15:38.704152] [DEBUG] [MAIN ] VOC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:38.705919] [INFO ] [MAIN ] VOC: Generated 1794 derivative records from 9011 primary records. [2025-10-20T22:15:38.706237] [DEBUG] [LOCAL ] VOC: Saving derivative data. [2025-10-20T22:15:38.722440] [DEBUG] [LOCAL ] VOC: Derivative data saved (1794 records). [2025-10-20T22:15:38.722665] [INFO ] [MAIN ] VOC: Successfully saved derivative data. [2025-10-20T22:15:38.724681] [INFO ] [MAIN ] Derivative Progress: 1756/1877 symbols completed. VOC took 1.0s. Estimated remaining: 45.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:38.725228] [INFO ] [MAIN ] VOYA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:38.744511] [DEBUG] [MAIN ] VNT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:38.756919] [DEBUG] [MAIN ] VNO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:38.757680] [INFO ] [MAIN ] VNO: Generated 1794 derivative records from 48936 primary records. [2025-10-20T22:15:38.758272] [INFO ] [MAIN ] VNT: Generated 1794 derivative records from 43986 primary records. [2025-10-20T22:15:38.758412] [DEBUG] [LOCAL ] VNT: Saving derivative data. [2025-10-20T22:15:38.758453] [DEBUG] [LOCAL ] VNO: Saving derivative data. [2025-10-20T22:15:38.788318] [DEBUG] [LOCAL ] VNT: Derivative data saved (1794 records). [2025-10-20T22:15:38.789405] [DEBUG] [LOCAL ] VNO: Derivative data saved (1794 records). [2025-10-20T22:15:38.789535] [INFO ] [MAIN ] VNT: Successfully saved derivative data. [2025-10-20T22:15:38.791910] [INFO ] [MAIN ] Derivative Progress: 1757/1877 symbols completed. VNT took 1.1s. Estimated remaining: 45.0s. Rate: 160.1 symbols/min [2025-10-20T22:15:38.792071] [INFO ] [MAIN ] VNO: Successfully saved derivative data. [2025-10-20T22:15:38.794289] [INFO ] [MAIN ] Derivative Progress: 1758/1877 symbols completed. VNO took 1.1s. Estimated remaining: 44.6s. Rate: 160.2 symbols/min [2025-10-20T22:15:38.792167] [INFO ] [MAIN ] VPG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:38.794464] [INFO ] [MAIN ] VPV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:38.825795] [DEBUG] [LOCAL ] VOYA: Loaded 42073 existing records from primary storage. [2025-10-20T22:15:38.824829] [DEBUG] [LOCAL ] VPV: Loaded 5429 existing records from primary storage. [2025-10-20T22:15:38.826078] [INFO ] [MAIN ] VPV: Loaded 5429 primary data records. [2025-10-20T22:15:38.826019] [INFO ] [MAIN ] VOYA: Loaded 42073 primary data records. [2025-10-20T22:15:38.970371] [DEBUG] [LOCAL ] VPG: Loaded 17102 existing records from primary storage. [2025-10-20T22:15:39.004676] [INFO ] [MAIN ] VPG: Loaded 17102 primary data records. [2025-10-20T22:15:39.783343] [DEBUG] [MAIN ] VOYA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:39.789578] [DEBUG] [MAIN ] VPV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:39.790543] [INFO ] [MAIN ] VPV: Generated 1794 derivative records from 5429 primary records. [2025-10-20T22:15:39.789383] [INFO ] [MAIN ] VOYA: Generated 1794 derivative records from 42073 primary records. [2025-10-20T22:15:39.790635] [DEBUG] [LOCAL ] VPV: Saving derivative data. [2025-10-20T22:15:39.790880] [DEBUG] [LOCAL ] VOYA: Saving derivative data. [2025-10-20T22:15:39.819536] [DEBUG] [LOCAL ] VPV: Derivative data saved (1794 records). [2025-10-20T22:15:39.820740] [INFO ] [MAIN ] VPV: Successfully saved derivative data. [2025-10-20T22:15:39.820669] [DEBUG] [LOCAL ] VOYA: Derivative data saved (1794 records). [2025-10-20T22:15:39.821908] [INFO ] [MAIN ] VOYA: Successfully saved derivative data. [2025-10-20T22:15:39.824798] [INFO ] [MAIN ] Derivative Progress: 1759/1877 symbols completed. VOYA took 1.1s. Estimated remaining: 44.3s. Rate: 160.0 symbols/min [2025-10-20T22:15:39.827445] [INFO ] [MAIN ] VRE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:39.827513] [INFO ] [MAIN ] Derivative Progress: 1760/1877 symbols completed. VPV took 1.0s. Estimated remaining: 43.9s. Rate: 160.1 symbols/min [2025-10-20T22:15:39.828379] [INFO ] [MAIN ] VRT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:39.842069] [DEBUG] [MAIN ] VPG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:39.842960] [INFO ] [MAIN ] VPG: Generated 1794 derivative records from 17102 primary records. [2025-10-20T22:15:39.843127] [DEBUG] [LOCAL ] VPG: Saving derivative data. [2025-10-20T22:15:39.861922] [DEBUG] [LOCAL ] VPG: Derivative data saved (1794 records). [2025-10-20T22:15:39.862069] [INFO ] [MAIN ] VPG: Successfully saved derivative data. [2025-10-20T22:15:39.864853] [INFO ] [MAIN ] Derivative Progress: 1761/1877 symbols completed. VPG took 1.1s. Estimated remaining: 43.5s. Rate: 160.2 symbols/min [2025-10-20T22:15:39.865043] [INFO ] [MAIN ] VRTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:39.909626] [DEBUG] [LOCAL ] VRTS: Loaded 9391 existing records from primary storage. [2025-10-20T22:15:39.910431] [DEBUG] [LOCAL ] VRE: Loaded 35362 existing records from primary storage. [2025-10-20T22:15:39.910705] [INFO ] [MAIN ] VRTS: Loaded 9391 primary data records. [2025-10-20T22:15:39.910786] [INFO ] [MAIN ] VRE: Loaded 35362 primary data records. [2025-10-20T22:15:40.125009] [DEBUG] [LOCAL ] VRT: Loaded 53753 existing records from primary storage. [2025-10-20T22:15:40.156860] [INFO ] [MAIN ] VRT: Loaded 53753 primary data records. [2025-10-20T22:15:40.880547] [DEBUG] [MAIN ] VRTS: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:40.881678] [INFO ] [MAIN ] VRTS: Generated 1793 derivative records from 9391 primary records. [2025-10-20T22:15:40.881954] [DEBUG] [LOCAL ] VRTS: Saving derivative data. [2025-10-20T22:15:40.908422] [DEBUG] [MAIN ] VRE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:40.915505] [DEBUG] [LOCAL ] VRTS: Derivative data saved (1793 records). [2025-10-20T22:15:40.928634] [INFO ] [MAIN ] VRE: Generated 1794 derivative records from 35362 primary records. [2025-10-20T22:15:40.929173] [INFO ] [MAIN ] VRTS: Successfully saved derivative data. [2025-10-20T22:15:40.929100] [DEBUG] [LOCAL ] VRE: Saving derivative data. [2025-10-20T22:15:40.931124] [INFO ] [MAIN ] Derivative Progress: 1762/1877 symbols completed. VRTS took 1.1s. Estimated remaining: 43.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:40.932674] [INFO ] [MAIN ] VSCO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:40.932543] [DEBUG] [MAIN ] VRT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:40.942026] [INFO ] [MAIN ] VRT: Generated 1794 derivative records from 53753 primary records. [2025-10-20T22:15:40.942249] [DEBUG] [LOCAL ] VRT: Saving derivative data. [2025-10-20T22:15:40.959922] [DEBUG] [LOCAL ] VRE: Derivative data saved (1794 records). [2025-10-20T22:15:40.960125] [INFO ] [MAIN ] VRE: Successfully saved derivative data. [2025-10-20T22:15:40.964914] [INFO ] [MAIN ] Derivative Progress: 1763/1877 symbols completed. VRE took 1.1s. Estimated remaining: 42.7s. Rate: 160.1 symbols/min [2025-10-20T22:15:40.967405] [INFO ] [MAIN ] VSH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:40.970194] [DEBUG] [LOCAL ] VRT: Derivative data saved (1794 records). [2025-10-20T22:15:40.970429] [INFO ] [MAIN ] VRT: Successfully saved derivative data. [2025-10-20T22:15:40.972701] [INFO ] [MAIN ] Derivative Progress: 1764/1877 symbols completed. VRT took 1.1s. Estimated remaining: 42.3s. Rate: 160.2 symbols/min [2025-10-20T22:15:40.973075] [INFO ] [MAIN ] VST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:41.023809] [DEBUG] [LOCAL ] VSCO: Loaded 52717 existing records from primary storage. [2025-10-20T22:15:41.024189] [INFO ] [MAIN ] VSCO: Loaded 52717 primary data records. [2025-10-20T22:15:41.181688] [DEBUG] [LOCAL ] VSH: Loaded 48263 existing records from primary storage. [2025-10-20T22:15:41.214964] [INFO ] [MAIN ] VSH: Loaded 48263 primary data records. [2025-10-20T22:15:41.287602] [DEBUG] [LOCAL ] VST: Loaded 53709 existing records from primary storage. [2025-10-20T22:15:41.409485] [INFO ] [MAIN ] VST: Loaded 53709 primary data records. [2025-10-20T22:15:42.076119] [DEBUG] [MAIN ] VSCO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:42.078059] [INFO ] [MAIN ] VSCO: Generated 1794 derivative records from 52717 primary records. [2025-10-20T22:15:42.078368] [DEBUG] [LOCAL ] VSCO: Saving derivative data. [2025-10-20T22:15:42.096676] [DEBUG] [LOCAL ] VSCO: Derivative data saved (1794 records). [2025-10-20T22:15:42.096948] [INFO ] [MAIN ] VSCO: Successfully saved derivative data. [2025-10-20T22:15:42.099236] [INFO ] [MAIN ] Derivative Progress: 1765/1877 symbols completed. VSCO took 1.2s. Estimated remaining: 42.0s. Rate: 160.0 symbols/min [2025-10-20T22:15:42.100480] [INFO ] [MAIN ] VSTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:42.115319] [DEBUG] [MAIN ] VSH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:42.116632] [INFO ] [MAIN ] VSH: Generated 1794 derivative records from 48263 primary records. [2025-10-20T22:15:42.116720] [DEBUG] [LOCAL ] VSH: Saving derivative data. [2025-10-20T22:15:42.134938] [DEBUG] [LOCAL ] VSH: Derivative data saved (1794 records). [2025-10-20T22:15:42.135842] [INFO ] [MAIN ] VSH: Successfully saved derivative data. [2025-10-20T22:15:42.138016] [INFO ] [MAIN ] Derivative Progress: 1766/1877 symbols completed. VSH took 1.2s. Estimated remaining: 41.6s. Rate: 160.1 symbols/min [2025-10-20T22:15:42.138413] [INFO ] [MAIN ] VTEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:42.153191] [DEBUG] [MAIN ] VST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:42.154023] [INFO ] [MAIN ] VST: Generated 1794 derivative records from 53709 primary records. [2025-10-20T22:15:42.154147] [DEBUG] [LOCAL ] VST: Saving derivative data. [2025-10-20T22:15:42.170789] [DEBUG] [LOCAL ] VST: Derivative data saved (1794 records). [2025-10-20T22:15:42.181537] [INFO ] [MAIN ] VST: Successfully saved derivative data. [2025-10-20T22:15:42.184074] [INFO ] [MAIN ] Derivative Progress: 1767/1877 symbols completed. VST took 1.2s. Estimated remaining: 41.2s. Rate: 160.2 symbols/min [2025-10-20T22:15:42.184290] [INFO ] [MAIN ] VTLE: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:42.214747] [DEBUG] [LOCAL ] VSTS: Loaded 48911 existing records from primary storage. [2025-10-20T22:15:42.227450] [INFO ] [MAIN ] VSTS: Loaded 48911 primary data records. [2025-10-20T22:15:42.228185] [DEBUG] [LOCAL ] VTEX: Loaded 42953 existing records from primary storage. [2025-10-20T22:15:42.261132] [INFO ] [MAIN ] VTEX: Loaded 42953 primary data records. [2025-10-20T22:15:42.558865] [DEBUG] [LOCAL ] VTLE: Loaded 47796 existing records from primary storage. [2025-10-20T22:15:42.605716] [INFO ] [MAIN ] VTLE: Loaded 47796 primary data records. [2025-10-20T22:15:43.224432] [DEBUG] [MAIN ] VSTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:43.238105] [INFO ] [MAIN ] VSTS: Generated 1794 derivative records from 48911 primary records. [2025-10-20T22:15:43.238340] [DEBUG] [LOCAL ] VSTS: Saving derivative data. [2025-10-20T22:15:43.238803] [DEBUG] [MAIN ] VTEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:43.247553] [INFO ] [MAIN ] VTEX: Generated 1794 derivative records from 42953 primary records. [2025-10-20T22:15:43.251855] [DEBUG] [LOCAL ] VTEX: Saving derivative data. [2025-10-20T22:15:43.255937] [DEBUG] [LOCAL ] VSTS: Derivative data saved (1794 records). [2025-10-20T22:15:43.256759] [INFO ] [MAIN ] VSTS: Successfully saved derivative data. [2025-10-20T22:15:43.258962] [INFO ] [MAIN ] Derivative Progress: 1768/1877 symbols completed. VSTS took 1.2s. Estimated remaining: 40.9s. Rate: 160.0 symbols/min [2025-10-20T22:15:43.274249] [DEBUG] [LOCAL ] VTEX: Derivative data saved (1794 records). [2025-10-20T22:15:43.274351] [INFO ] [MAIN ] VTEX: Successfully saved derivative data. [2025-10-20T22:15:43.276355] [INFO ] [MAIN ] Derivative Progress: 1769/1877 symbols completed. VTEX took 1.1s. Estimated remaining: 40.5s. Rate: 160.1 symbols/min [2025-10-20T22:15:43.291490] [DEBUG] [MAIN ] VTLE: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:43.292072] [INFO ] [MAIN ] VTLE: Generated 1794 derivative records from 47796 primary records. [2025-10-20T22:15:43.292132] [DEBUG] [LOCAL ] VTLE: Saving derivative data. [2025-10-20T22:15:43.307495] [DEBUG] [LOCAL ] VTLE: Derivative data saved (1794 records). [2025-10-20T22:15:43.307565] [INFO ] [MAIN ] VTLE: Successfully saved derivative data. [2025-10-20T22:15:43.310018] [INFO ] [MAIN ] Derivative Progress: 1770/1877 symbols completed. VTLE took 1.1s. Estimated remaining: 40.1s. Rate: 160.2 symbols/min [2025-10-20T22:15:43.310410] [INFO ] [MAIN ] VTMX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:43.310914] [INFO ] [MAIN ] VTOL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:43.310852] [INFO ] [MAIN ] VTN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:43.329853] [DEBUG] [LOCAL ] VTN: Loaded 7075 existing records from primary storage. [2025-10-20T22:15:43.330039] [INFO ] [MAIN ] VTN: Loaded 7075 primary data records. [2025-10-20T22:15:43.423082] [DEBUG] [LOCAL ] VTOL: Loaded 19532 existing records from primary storage. [2025-10-20T22:15:43.409015] [DEBUG] [LOCAL ] VTMX: Loaded 17621 existing records from primary storage. [2025-10-20T22:15:43.440749] [INFO ] [MAIN ] VTOL: Loaded 19532 primary data records. [2025-10-20T22:15:43.440858] [INFO ] [MAIN ] VTMX: Loaded 17621 primary data records. [2025-10-20T22:15:44.323349] [DEBUG] [MAIN ] VTN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:44.324324] [INFO ] [MAIN ] VTN: Generated 1794 derivative records from 7075 primary records. [2025-10-20T22:15:44.325420] [DEBUG] [LOCAL ] VTN: Saving derivative data. [2025-10-20T22:15:44.342678] [DEBUG] [LOCAL ] VTN: Derivative data saved (1794 records). [2025-10-20T22:15:44.342968] [INFO ] [MAIN ] VTN: Successfully saved derivative data. [2025-10-20T22:15:44.362509] [INFO ] [MAIN ] Derivative Progress: 1771/1877 symbols completed. VTN took 1.1s. Estimated remaining: 39.8s. Rate: 160.0 symbols/min [2025-10-20T22:15:44.364227] [INFO ] [MAIN ] VTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:44.365860] [DEBUG] [MAIN ] VTOL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:44.366898] [INFO ] [MAIN ] VTOL: Generated 1794 derivative records from 19532 primary records. [2025-10-20T22:15:44.367012] [DEBUG] [LOCAL ] VTOL: Saving derivative data. [2025-10-20T22:15:44.391746] [DEBUG] [MAIN ] VTMX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:44.396668] [DEBUG] [LOCAL ] VTOL: Derivative data saved (1794 records). [2025-10-20T22:15:44.397404] [INFO ] [MAIN ] VTMX: Generated 1794 derivative records from 17621 primary records. [2025-10-20T22:15:44.397698] [DEBUG] [LOCAL ] VTMX: Saving derivative data. [2025-10-20T22:15:44.397600] [INFO ] [MAIN ] VTOL: Successfully saved derivative data. [2025-10-20T22:15:44.399988] [INFO ] [MAIN ] Derivative Progress: 1772/1877 symbols completed. VTOL took 1.1s. Estimated remaining: 39.4s. Rate: 160.1 symbols/min [2025-10-20T22:15:44.400627] [INFO ] [MAIN ] VTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:44.417539] [DEBUG] [LOCAL ] VTMX: Derivative data saved (1794 records). [2025-10-20T22:15:44.417765] [INFO ] [MAIN ] VTMX: Successfully saved derivative data. [2025-10-20T22:15:44.420102] [INFO ] [MAIN ] Derivative Progress: 1773/1877 symbols completed. VTMX took 1.1s. Estimated remaining: 39.0s. Rate: 160.2 symbols/min [2025-10-20T22:15:44.420289] [INFO ] [MAIN ] VVR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:44.469317] [DEBUG] [LOCAL ] VTS: Loaded 29986 existing records from primary storage. [2025-10-20T22:15:44.479621] [INFO ] [MAIN ] VTS: Loaded 29986 primary data records. [2025-10-20T22:15:44.526782] [DEBUG] [LOCAL ] VVR: Loaded 31242 existing records from primary storage. [2025-10-20T22:15:44.539951] [DEBUG] [LOCAL ] VTR: Loaded 52720 existing records from primary storage. [2025-10-20T22:15:44.552618] [INFO ] [MAIN ] VTR: Loaded 52720 primary data records. [2025-10-20T22:15:44.545191] [INFO ] [MAIN ] VVR: Loaded 31242 primary data records. [2025-10-20T22:15:45.428404] [DEBUG] [MAIN ] VTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:45.434511] [INFO ] [MAIN ] VTS: Generated 1794 derivative records from 29986 primary records. [2025-10-20T22:15:45.435766] [DEBUG] [MAIN ] VTR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:45.436729] [INFO ] [MAIN ] VTR: Generated 1794 derivative records from 52720 primary records. [2025-10-20T22:15:45.435929] [DEBUG] [LOCAL ] VTS: Saving derivative data. [2025-10-20T22:15:45.444624] [DEBUG] [LOCAL ] VTR: Saving derivative data. [2025-10-20T22:15:45.462089] [DEBUG] [MAIN ] VVR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:45.474182] [INFO ] [MAIN ] VVR: Generated 1794 derivative records from 31242 primary records. [2025-10-20T22:15:45.474291] [DEBUG] [LOCAL ] VVR: Saving derivative data. [2025-10-20T22:15:45.481787] [DEBUG] [LOCAL ] VTS: Derivative data saved (1794 records). [2025-10-20T22:15:45.482152] [INFO ] [MAIN ] VTS: Successfully saved derivative data. [2025-10-20T22:15:45.483241] [DEBUG] [LOCAL ] VTR: Derivative data saved (1794 records). [2025-10-20T22:15:45.485049] [INFO ] [MAIN ] Derivative Progress: 1774/1877 symbols completed. VTS took 1.1s. Estimated remaining: 38.6s. Rate: 160.0 symbols/min [2025-10-20T22:15:45.485431] [INFO ] [MAIN ] VTR: Successfully saved derivative data. [2025-10-20T22:15:45.492384] [INFO ] [MAIN ] VVV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:45.499793] [INFO ] [MAIN ] Derivative Progress: 1775/1877 symbols completed. VTR took 1.1s. Estimated remaining: 38.2s. Rate: 160.1 symbols/min [2025-10-20T22:15:45.502325] [DEBUG] [LOCAL ] VVR: Derivative data saved (1794 records). [2025-10-20T22:15:45.502671] [INFO ] [MAIN ] VVR: Successfully saved derivative data. [2025-10-20T22:15:45.502917] [INFO ] [MAIN ] VVX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:45.505343] [INFO ] [MAIN ] Derivative Progress: 1776/1877 symbols completed. VVR took 1.1s. Estimated remaining: 37.8s. Rate: 160.2 symbols/min [2025-10-20T22:15:45.505681] [INFO ] [MAIN ] VYX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:45.541782] [DEBUG] [LOCAL ] VVX: Loaded 23053 existing records from primary storage. [2025-10-20T22:15:45.541929] [INFO ] [MAIN ] VVX: Loaded 23053 primary data records. [2025-10-20T22:15:45.708467] [DEBUG] [LOCAL ] VYX: Loaded 48363 existing records from primary storage. [2025-10-20T22:15:45.761241] [DEBUG] [LOCAL ] VVV: Loaded 50231 existing records from primary storage. [2025-10-20T22:15:45.871394] [INFO ] [MAIN ] VVV: Loaded 50231 primary data records. [2025-10-20T22:15:45.871285] [INFO ] [MAIN ] VYX: Loaded 48363 primary data records. [2025-10-20T22:15:46.532757] [DEBUG] [MAIN ] VVX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:46.534165] [INFO ] [MAIN ] VVX: Generated 1794 derivative records from 23053 primary records. [2025-10-20T22:15:46.534669] [DEBUG] [LOCAL ] VVX: Saving derivative data. [2025-10-20T22:15:46.553608] [DEBUG] [LOCAL ] VVX: Derivative data saved (1794 records). [2025-10-20T22:15:46.554022] [INFO ] [MAIN ] VVX: Successfully saved derivative data. [2025-10-20T22:15:46.556265] [INFO ] [MAIN ] Derivative Progress: 1777/1877 symbols completed. VVX took 1.1s. Estimated remaining: 37.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:46.556625] [INFO ] [MAIN ] VZ: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:46.587026] [DEBUG] [MAIN ] VVV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:46.592399] [DEBUG] [MAIN ] VYX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:46.593365] [INFO ] [MAIN ] VVV: Generated 1794 derivative records from 50231 primary records. [2025-10-20T22:15:46.593979] [INFO ] [MAIN ] VYX: Generated 1794 derivative records from 48363 primary records. [2025-10-20T22:15:46.594360] [DEBUG] [LOCAL ] VYX: Saving derivative data. [2025-10-20T22:15:46.594171] [DEBUG] [LOCAL ] VVV: Saving derivative data. [2025-10-20T22:15:46.624188] [DEBUG] [LOCAL ] VVV: Derivative data saved (1794 records). [2025-10-20T22:15:46.627412] [DEBUG] [LOCAL ] VYX: Derivative data saved (1794 records). [2025-10-20T22:15:46.627584] [INFO ] [MAIN ] VYX: Successfully saved derivative data. [2025-10-20T22:15:46.627691] [INFO ] [MAIN ] VVV: Successfully saved derivative data. [2025-10-20T22:15:46.629763] [INFO ] [MAIN ] Derivative Progress: 1778/1877 symbols completed. VYX took 1.1s. Estimated remaining: 37.1s. Rate: 160.1 symbols/min [2025-10-20T22:15:46.631994] [INFO ] [MAIN ] W: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:46.632181] [INFO ] [MAIN ] Derivative Progress: 1779/1877 symbols completed. VVV took 1.1s. Estimated remaining: 36.7s. Rate: 160.2 symbols/min [2025-10-20T22:15:46.646419] [INFO ] [MAIN ] WAB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:46.673382] [DEBUG] [LOCAL ] VZ: Loaded 53791 existing records from primary storage. [2025-10-20T22:15:46.673903] [INFO ] [MAIN ] VZ: Loaded 53791 primary data records. [2025-10-20T22:15:47.014531] [DEBUG] [LOCAL ] W: Loaded 53421 existing records from primary storage. [2025-10-20T22:15:47.027440] [INFO ] [MAIN ] W: Loaded 53421 primary data records. [2025-10-20T22:15:47.041532] [DEBUG] [LOCAL ] WAB: Loaded 43863 existing records from primary storage. [2025-10-20T22:15:47.060210] [INFO ] [MAIN ] WAB: Loaded 43863 primary data records. [2025-10-20T22:15:47.647381] [DEBUG] [MAIN ] VZ: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:47.649115] [INFO ] [MAIN ] VZ: Generated 1794 derivative records from 53791 primary records. [2025-10-20T22:15:47.649364] [DEBUG] [LOCAL ] VZ: Saving derivative data. [2025-10-20T22:15:47.668039] [DEBUG] [LOCAL ] VZ: Derivative data saved (1794 records). [2025-10-20T22:15:47.668909] [INFO ] [MAIN ] VZ: Successfully saved derivative data. [2025-10-20T22:15:47.671114] [INFO ] [MAIN ] Derivative Progress: 1780/1877 symbols completed. VZ took 1.1s. Estimated remaining: 36.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:47.671432] [INFO ] [MAIN ] WAL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:47.685469] [DEBUG] [MAIN ] W: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:47.686158] [INFO ] [MAIN ] W: Generated 1794 derivative records from 53421 primary records. [2025-10-20T22:15:47.686303] [DEBUG] [LOCAL ] W: Saving derivative data. [2025-10-20T22:15:47.704438] [DEBUG] [LOCAL ] W: Derivative data saved (1794 records). [2025-10-20T22:15:47.704679] [INFO ] [MAIN ] W: Successfully saved derivative data. [2025-10-20T22:15:47.707535] [INFO ] [MAIN ] Derivative Progress: 1781/1877 symbols completed. W took 1.1s. Estimated remaining: 36.0s. Rate: 160.1 symbols/min [2025-10-20T22:15:47.707777] [INFO ] [MAIN ] WAT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:47.721194] [DEBUG] [MAIN ] WAB: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:47.721859] [INFO ] [MAIN ] WAB: Generated 1793 derivative records from 43863 primary records. [2025-10-20T22:15:47.722027] [DEBUG] [LOCAL ] WAB: Saving derivative data. [2025-10-20T22:15:47.740734] [DEBUG] [LOCAL ] WAB: Derivative data saved (1793 records). [2025-10-20T22:15:47.751029] [INFO ] [MAIN ] WAB: Successfully saved derivative data. [2025-10-20T22:15:47.755267] [INFO ] [MAIN ] Derivative Progress: 1782/1877 symbols completed. WAB took 1.1s. Estimated remaining: 35.6s. Rate: 160.2 symbols/min [2025-10-20T22:15:47.755739] [INFO ] [MAIN ] WBS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:47.783269] [DEBUG] [LOCAL ] WAL: Loaded 45683 existing records from primary storage. [2025-10-20T22:15:47.784064] [INFO ] [MAIN ] WAL: Loaded 45683 primary data records. [2025-10-20T22:15:47.850765] [DEBUG] [LOCAL ] WAT: Loaded 35792 existing records from primary storage. [2025-10-20T22:15:47.859913] [INFO ] [MAIN ] WAT: Loaded 35792 primary data records. [2025-10-20T22:15:48.334909] [DEBUG] [LOCAL ] WBS: Loaded 48845 existing records from primary storage. [2025-10-20T22:15:48.444992] [INFO ] [MAIN ] WBS: Loaded 48845 primary data records. [2025-10-20T22:15:48.794070] [DEBUG] [MAIN ] WAL: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:48.808360] [INFO ] [MAIN ] WAL: Generated 1793 derivative records from 45683 primary records. [2025-10-20T22:15:48.808829] [DEBUG] [LOCAL ] WAL: Saving derivative data. [2025-10-20T22:15:48.809028] [DEBUG] [MAIN ] WAT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:48.811665] [INFO ] [MAIN ] WAT: Generated 1794 derivative records from 35792 primary records. [2025-10-20T22:15:48.811932] [DEBUG] [LOCAL ] WAT: Saving derivative data. [2025-10-20T22:15:48.828559] [DEBUG] [LOCAL ] WAL: Derivative data saved (1793 records). [2025-10-20T22:15:48.828843] [INFO ] [MAIN ] WAL: Successfully saved derivative data. [2025-10-20T22:15:48.830906] [INFO ] [MAIN ] Derivative Progress: 1783/1877 symbols completed. WAL took 1.2s. Estimated remaining: 35.3s. Rate: 160.0 symbols/min [2025-10-20T22:15:48.847482] [DEBUG] [LOCAL ] WAT: Derivative data saved (1794 records). [2025-10-20T22:15:48.847717] [INFO ] [MAIN ] WAT: Successfully saved derivative data. [2025-10-20T22:15:48.849769] [INFO ] [MAIN ] Derivative Progress: 1784/1877 symbols completed. WAT took 1.1s. Estimated remaining: 34.9s. Rate: 160.1 symbols/min [2025-10-20T22:15:48.866378] [DEBUG] [MAIN ] WBS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:48.867370] [INFO ] [MAIN ] WBS: Generated 1794 derivative records from 48845 primary records. [2025-10-20T22:15:48.867446] [DEBUG] [LOCAL ] WBS: Saving derivative data. [2025-10-20T22:15:48.884732] [DEBUG] [LOCAL ] WBS: Derivative data saved (1794 records). [2025-10-20T22:15:48.884816] [INFO ] [MAIN ] WBS: Successfully saved derivative data. [2025-10-20T22:15:48.887415] [INFO ] [MAIN ] Derivative Progress: 1785/1877 symbols completed. WBS took 1.1s. Estimated remaining: 34.5s. Rate: 160.2 symbols/min [2025-10-20T22:15:48.887829] [INFO ] [MAIN ] WBX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:48.887980] [INFO ] [MAIN ] WCN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:48.888043] [INFO ] [MAIN ] WCC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:48.902588] [DEBUG] [LOCAL ] WBX: Loaded 10196 existing records from primary storage. [2025-10-20T22:15:48.902769] [INFO ] [MAIN ] WBX: Loaded 10196 primary data records. [2025-10-20T22:15:49.138214] [DEBUG] [LOCAL ] WCC: Loaded 38838 existing records from primary storage. [2025-10-20T22:15:49.247786] [INFO ] [MAIN ] WCC: Loaded 38838 primary data records. [2025-10-20T22:15:49.344543] [DEBUG] [LOCAL ] WCN: Loaded 47917 existing records from primary storage. [2025-10-20T22:15:49.368761] [INFO ] [MAIN ] WCN: Loaded 47917 primary data records. [2025-10-20T22:15:49.835021] [DEBUG] [MAIN ] WBX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:49.934203] [INFO ] [MAIN ] WBX: Generated 1794 derivative records from 10196 primary records. [2025-10-20T22:15:49.934673] [DEBUG] [LOCAL ] WBX: Saving derivative data. [2025-10-20T22:15:49.950867] [DEBUG] [LOCAL ] WBX: Derivative data saved (1794 records). [2025-10-20T22:15:49.951846] [INFO ] [MAIN ] WBX: Successfully saved derivative data. [2025-10-20T22:15:49.954957] [INFO ] [MAIN ] Derivative Progress: 1786/1877 symbols completed. WBX took 1.1s. Estimated remaining: 34.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:49.955831] [INFO ] [MAIN ] WD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:49.974329] [DEBUG] [MAIN ] WCC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:49.975199] [INFO ] [MAIN ] WCC: Generated 1794 derivative records from 38838 primary records. [2025-10-20T22:15:49.975484] [DEBUG] [LOCAL ] WCC: Saving derivative data. [2025-10-20T22:15:49.998892] [DEBUG] [LOCAL ] WCC: Derivative data saved (1794 records). [2025-10-20T22:15:49.999334] [INFO ] [MAIN ] WCC: Successfully saved derivative data. [2025-10-20T22:15:50.001318] [INFO ] [MAIN ] Derivative Progress: 1787/1877 symbols completed. WCC took 1.1s. Estimated remaining: 33.7s. Rate: 160.1 symbols/min [2025-10-20T22:15:50.001620] [INFO ] [MAIN ] WDH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:50.016002] [DEBUG] [LOCAL ] WD: Loaded 22815 existing records from primary storage. [2025-10-20T22:15:50.029714] [DEBUG] [MAIN ] WCN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:50.030602] [INFO ] [MAIN ] WCN: Generated 1794 derivative records from 47917 primary records. [2025-10-20T22:15:50.029486] [INFO ] [MAIN ] WD: Loaded 22815 primary data records. [2025-10-20T22:15:50.035280] [DEBUG] [LOCAL ] WCN: Saving derivative data. [2025-10-20T22:15:50.124415] [DEBUG] [LOCAL ] WDH: Loaded 18081 existing records from primary storage. [2025-10-20T22:15:50.124030] [DEBUG] [LOCAL ] WCN: Derivative data saved (1794 records). [2025-10-20T22:15:50.141755] [INFO ] [MAIN ] WDH: Loaded 18081 primary data records. [2025-10-20T22:15:50.141677] [INFO ] [MAIN ] WCN: Successfully saved derivative data. [2025-10-20T22:15:50.175725] [INFO ] [MAIN ] Derivative Progress: 1788/1877 symbols completed. WCN took 1.3s. Estimated remaining: 33.3s. Rate: 160.1 symbols/min [2025-10-20T22:15:50.183565] [INFO ] [MAIN ] WDI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:50.498289] [DEBUG] [LOCAL ] WDI: Loaded 21311 existing records from primary storage. [2025-10-20T22:15:50.527483] [INFO ] [MAIN ] WDI: Loaded 21311 primary data records. [2025-10-20T22:15:51.069492] [DEBUG] [MAIN ] WD: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:51.070563] [INFO ] [MAIN ] WD: Generated 1793 derivative records from 22815 primary records. [2025-10-20T22:15:51.070952] [DEBUG] [LOCAL ] WD: Saving derivative data. [2025-10-20T22:15:51.095953] [DEBUG] [LOCAL ] WD: Derivative data saved (1793 records). [2025-10-20T22:15:51.101320] [INFO ] [MAIN ] WD: Successfully saved derivative data. [2025-10-20T22:15:51.103267] [INFO ] [MAIN ] Derivative Progress: 1789/1877 symbols completed. WD took 1.1s. Estimated remaining: 33.0s. Rate: 160.0 symbols/min [2025-10-20T22:15:51.103702] [INFO ] [MAIN ] WDS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:51.104458] [DEBUG] [MAIN ] WDH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:51.106065] [INFO ] [MAIN ] WDH: Generated 1794 derivative records from 18081 primary records. [2025-10-20T22:15:51.106160] [DEBUG] [LOCAL ] WDH: Saving derivative data. [2025-10-20T22:15:51.126779] [DEBUG] [LOCAL ] WDH: Derivative data saved (1794 records). [2025-10-20T22:15:51.127036] [INFO ] [MAIN ] WDH: Successfully saved derivative data. [2025-10-20T22:15:51.129448] [INFO ] [MAIN ] Derivative Progress: 1790/1877 symbols completed. WDH took 1.1s. Estimated remaining: 32.6s. Rate: 160.1 symbols/min [2025-10-20T22:15:51.129623] [INFO ] [MAIN ] WEA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:51.143330] [DEBUG] [MAIN ] WDI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:51.144198] [INFO ] [MAIN ] WDI: Generated 1794 derivative records from 21311 primary records. [2025-10-20T22:15:51.144377] [DEBUG] [LOCAL ] WDI: Saving derivative data. [2025-10-20T22:15:51.164880] [DEBUG] [LOCAL ] WDI: Derivative data saved (1794 records). [2025-10-20T22:15:51.165081] [INFO ] [MAIN ] WDI: Successfully saved derivative data. [2025-10-20T22:15:51.167421] [INFO ] [MAIN ] Derivative Progress: 1791/1877 symbols completed. WDI took 1.0s. Estimated remaining: 32.2s. Rate: 160.2 symbols/min [2025-10-20T22:15:51.167809] [INFO ] [MAIN ] WEAV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:51.172422] [DEBUG] [LOCAL ] WEA: Loaded 7826 existing records from primary storage. [2025-10-20T22:15:51.183361] [INFO ] [MAIN ] WEA: Loaded 7826 primary data records. [2025-10-20T22:15:51.304729] [DEBUG] [LOCAL ] WDS: Loaded 43142 existing records from primary storage. [2025-10-20T22:15:51.329964] [INFO ] [MAIN ] WDS: Loaded 43142 primary data records. [2025-10-20T22:15:51.591144] [DEBUG] [LOCAL ] WEAV: Loaded 42365 existing records from primary storage. [2025-10-20T22:15:51.603211] [INFO ] [MAIN ] WEAV: Loaded 42365 primary data records. [2025-10-20T22:15:52.208082] [DEBUG] [MAIN ] WEA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:52.209195] [INFO ] [MAIN ] WEA: Generated 1794 derivative records from 7826 primary records. [2025-10-20T22:15:52.209313] [DEBUG] [LOCAL ] WEA: Saving derivative data. [2025-10-20T22:15:52.226676] [DEBUG] [LOCAL ] WEA: Derivative data saved (1794 records). [2025-10-20T22:15:52.227203] [INFO ] [MAIN ] WEA: Successfully saved derivative data. [2025-10-20T22:15:52.229352] [INFO ] [MAIN ] Derivative Progress: 1792/1877 symbols completed. WEA took 1.1s. Estimated remaining: 31.9s. Rate: 160.0 symbols/min [2025-10-20T22:15:52.229616] [INFO ] [MAIN ] WEC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:52.260677] [DEBUG] [MAIN ] WEAV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:52.261789] [INFO ] [MAIN ] WEAV: Generated 1794 derivative records from 42365 primary records. [2025-10-20T22:15:52.261032] [DEBUG] [MAIN ] WDS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:52.262040] [DEBUG] [LOCAL ] WEAV: Saving derivative data. [2025-10-20T22:15:52.262573] [INFO ] [MAIN ] WDS: Generated 1794 derivative records from 43142 primary records. [2025-10-20T22:15:52.262691] [DEBUG] [LOCAL ] WDS: Saving derivative data. [2025-10-20T22:15:52.293390] [DEBUG] [LOCAL ] WEAV: Derivative data saved (1794 records). [2025-10-20T22:15:52.298239] [DEBUG] [LOCAL ] WDS: Derivative data saved (1794 records). [2025-10-20T22:15:52.298335] [INFO ] [MAIN ] WEAV: Successfully saved derivative data. [2025-10-20T22:15:52.298426] [INFO ] [MAIN ] WDS: Successfully saved derivative data. [2025-10-20T22:15:52.300378] [INFO ] [MAIN ] Derivative Progress: 1793/1877 symbols completed. WEAV took 1.1s. Estimated remaining: 31.5s. Rate: 160.1 symbols/min [2025-10-20T22:15:52.302924] [INFO ] [MAIN ] Derivative Progress: 1794/1877 symbols completed. WDS took 1.2s. Estimated remaining: 31.1s. Rate: 160.2 symbols/min [2025-10-20T22:15:52.300560] [INFO ] [MAIN ] WELL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:52.303724] [INFO ] [MAIN ] WES: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:52.347760] [DEBUG] [LOCAL ] WEC: Loaded 52188 existing records from primary storage. [2025-10-20T22:15:52.348095] [INFO ] [MAIN ] WEC: Loaded 52188 primary data records. [2025-10-20T22:15:52.544255] [DEBUG] [LOCAL ] WES: Loaded 48673 existing records from primary storage. [2025-10-20T22:15:52.553182] [INFO ] [MAIN ] WES: Loaded 48673 primary data records. [2025-10-20T22:15:52.796471] [DEBUG] [LOCAL ] WELL: Loaded 52322 existing records from primary storage. [2025-10-20T22:15:52.808024] [INFO ] [MAIN ] WELL: Loaded 52322 primary data records. [2025-10-20T22:15:53.359274] [DEBUG] [MAIN ] WEC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:53.360457] [INFO ] [MAIN ] WEC: Generated 1794 derivative records from 52188 primary records. [2025-10-20T22:15:53.360719] [DEBUG] [LOCAL ] WEC: Saving derivative data. [2025-10-20T22:15:53.380559] [DEBUG] [LOCAL ] WEC: Derivative data saved (1794 records). [2025-10-20T22:15:53.381000] [INFO ] [MAIN ] WEC: Successfully saved derivative data. [2025-10-20T22:15:53.383074] [INFO ] [MAIN ] Derivative Progress: 1795/1877 symbols completed. WEC took 1.2s. Estimated remaining: 30.8s. Rate: 160.0 symbols/min [2025-10-20T22:15:53.383327] [INFO ] [MAIN ] WEX: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:53.397459] [DEBUG] [MAIN ] WES: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:53.398390] [INFO ] [MAIN ] WES: Generated 1794 derivative records from 48673 primary records. [2025-10-20T22:15:53.398579] [DEBUG] [LOCAL ] WES: Saving derivative data. [2025-10-20T22:15:53.414886] [DEBUG] [LOCAL ] WES: Derivative data saved (1794 records). [2025-10-20T22:15:53.415033] [INFO ] [MAIN ] WES: Successfully saved derivative data. [2025-10-20T22:15:53.418174] [INFO ] [MAIN ] Derivative Progress: 1796/1877 symbols completed. WES took 1.1s. Estimated remaining: 30.4s. Rate: 160.1 symbols/min [2025-10-20T22:15:53.420215] [INFO ] [MAIN ] WF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:53.435511] [DEBUG] [MAIN ] WELL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:53.436454] [INFO ] [MAIN ] WELL: Generated 1794 derivative records from 52322 primary records. [2025-10-20T22:15:53.445528] [DEBUG] [LOCAL ] WELL: Saving derivative data. [2025-10-20T22:15:53.473541] [DEBUG] [LOCAL ] WEX: Loaded 32742 existing records from primary storage. [2025-10-20T22:15:53.481443] [DEBUG] [LOCAL ] WELL: Derivative data saved (1794 records). [2025-10-20T22:15:53.479445] [DEBUG] [LOCAL ] WF: Loaded 10893 existing records from primary storage. [2025-10-20T22:15:53.481724] [INFO ] [MAIN ] WELL: Successfully saved derivative data. [2025-10-20T22:15:53.481946] [INFO ] [MAIN ] WEX: Loaded 32742 primary data records. [2025-10-20T22:15:53.481875] [INFO ] [MAIN ] WF: Loaded 10893 primary data records. [2025-10-20T22:15:53.484293] [INFO ] [MAIN ] Derivative Progress: 1797/1877 symbols completed. WELL took 1.2s. Estimated remaining: 30.0s. Rate: 160.1 symbols/min [2025-10-20T22:15:53.494822] [INFO ] [MAIN ] WFC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:54.197062] [DEBUG] [LOCAL ] WFC: Loaded 53748 existing records from primary storage. [2025-10-20T22:15:54.197572] [INFO ] [MAIN ] WFC: Loaded 53748 primary data records. [2025-10-20T22:15:54.416475] [DEBUG] [MAIN ] WEX: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:54.417703] [INFO ] [MAIN ] WEX: Generated 1794 derivative records from 32742 primary records. [2025-10-20T22:15:54.537276] [DEBUG] [LOCAL ] WEX: Saving derivative data. [2025-10-20T22:15:54.568614] [DEBUG] [LOCAL ] WEX: Derivative data saved (1794 records). [2025-10-20T22:15:54.569265] [INFO ] [MAIN ] WEX: Successfully saved derivative data. [2025-10-20T22:15:54.571383] [INFO ] [MAIN ] Derivative Progress: 1798/1877 symbols completed. WEX took 1.2s. Estimated remaining: 29.6s. Rate: 160.0 symbols/min [2025-10-20T22:15:54.568518] [DEBUG] [MAIN ] WF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:54.573060] [INFO ] [MAIN ] WF: Generated 1794 derivative records from 10893 primary records. [2025-10-20T22:15:54.573381] [DEBUG] [LOCAL ] WF: Saving derivative data. [2025-10-20T22:15:54.591007] [DEBUG] [LOCAL ] WF: Derivative data saved (1794 records). [2025-10-20T22:15:54.591159] [INFO ] [MAIN ] WF: Successfully saved derivative data. [2025-10-20T22:15:54.593525] [INFO ] [MAIN ] Derivative Progress: 1799/1877 symbols completed. WF took 1.2s. Estimated remaining: 29.2s. Rate: 160.1 symbols/min [2025-10-20T22:15:54.613205] [DEBUG] [MAIN ] WFC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:54.613906] [INFO ] [MAIN ] WFC: Generated 1794 derivative records from 53748 primary records. [2025-10-20T22:15:54.613966] [DEBUG] [LOCAL ] WFC: Saving derivative data. [2025-10-20T22:15:54.629774] [DEBUG] [LOCAL ] WFC: Derivative data saved (1794 records). [2025-10-20T22:15:54.629907] [INFO ] [MAIN ] WFC: Successfully saved derivative data. [2025-10-20T22:15:54.632208] [INFO ] [MAIN ] Derivative Progress: 1800/1877 symbols completed. WFC took 1.1s. Estimated remaining: 28.9s. Rate: 160.1 symbols/min [2025-10-20T22:15:54.632826] [INFO ] [MAIN ] WFG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:54.632927] [INFO ] [MAIN ] WH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:54.632728] [INFO ] [MAIN ] WGO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:54.668319] [DEBUG] [LOCAL ] WFG: Loaded 24746 existing records from primary storage. [2025-10-20T22:15:54.668468] [INFO ] [MAIN ] WFG: Loaded 24746 primary data records. [2025-10-20T22:15:54.898516] [DEBUG] [LOCAL ] WH: Loaded 45259 existing records from primary storage. [2025-10-20T22:15:55.003505] [DEBUG] [LOCAL ] WGO: Loaded 37980 existing records from primary storage. [2025-10-20T22:15:55.003620] [INFO ] [MAIN ] WH: Loaded 45259 primary data records. [2025-10-20T22:15:55.014932] [INFO ] [MAIN ] WGO: Loaded 37980 primary data records. [2025-10-20T22:15:55.659804] [DEBUG] [MAIN ] WFG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:55.661126] [INFO ] [MAIN ] WFG: Generated 1794 derivative records from 24746 primary records. [2025-10-20T22:15:55.661433] [DEBUG] [LOCAL ] WFG: Saving derivative data. [2025-10-20T22:15:55.680219] [DEBUG] [LOCAL ] WFG: Derivative data saved (1794 records). [2025-10-20T22:15:55.680922] [INFO ] [MAIN ] WFG: Successfully saved derivative data. [2025-10-20T22:15:55.685109] [INFO ] [MAIN ] Derivative Progress: 1801/1877 symbols completed. WFG took 1.1s. Estimated remaining: 28.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:55.685540] [INFO ] [MAIN ] WHD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:55.701858] [DEBUG] [MAIN ] WH: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:55.702789] [INFO ] [MAIN ] WH: Generated 1793 derivative records from 45259 primary records. [2025-10-20T22:15:55.703044] [DEBUG] [LOCAL ] WH: Saving derivative data. [2025-10-20T22:15:55.724112] [DEBUG] [MAIN ] WGO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:55.733259] [INFO ] [MAIN ] WGO: Generated 1794 derivative records from 37980 primary records. [2025-10-20T22:15:55.733441] [DEBUG] [LOCAL ] WGO: Saving derivative data. [2025-10-20T22:15:55.732502] [DEBUG] [LOCAL ] WH: Derivative data saved (1793 records). [2025-10-20T22:15:55.734154] [INFO ] [MAIN ] WH: Successfully saved derivative data. [2025-10-20T22:15:55.736389] [INFO ] [MAIN ] Derivative Progress: 1802/1877 symbols completed. WH took 1.1s. Estimated remaining: 28.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:55.743681] [INFO ] [MAIN ] WHG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:55.752522] [DEBUG] [LOCAL ] WGO: Derivative data saved (1794 records). [2025-10-20T22:15:55.753616] [INFO ] [MAIN ] WGO: Successfully saved derivative data. [2025-10-20T22:15:55.756006] [INFO ] [MAIN ] Derivative Progress: 1803/1877 symbols completed. WGO took 1.1s. Estimated remaining: 27.7s. Rate: 160.1 symbols/min [2025-10-20T22:15:55.756231] [INFO ] [MAIN ] WHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:55.769825] [DEBUG] [LOCAL ] WHG: Loaded 3075 existing records from primary storage. [2025-10-20T22:15:55.770054] [INFO ] [MAIN ] WHG: Loaded 3075 primary data records. [2025-10-20T22:15:55.895817] [DEBUG] [LOCAL ] WHD: Loaded 40110 existing records from primary storage. [2025-10-20T22:15:55.912220] [INFO ] [MAIN ] WHD: Loaded 40110 primary data records. [2025-10-20T22:15:56.234557] [DEBUG] [LOCAL ] WHR: Loaded 48938 existing records from primary storage. [2025-10-20T22:15:56.235536] [INFO ] [MAIN ] WHR: Loaded 48938 primary data records. [2025-10-20T22:15:56.828263] [DEBUG] [MAIN ] WHG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:56.829456] [INFO ] [MAIN ] WHG: Generated 1794 derivative records from 3075 primary records. [2025-10-20T22:15:56.829553] [DEBUG] [LOCAL ] WHG: Saving derivative data. [2025-10-20T22:15:56.846068] [DEBUG] [LOCAL ] WHG: Derivative data saved (1794 records). [2025-10-20T22:15:56.846204] [INFO ] [MAIN ] WHG: Successfully saved derivative data. [2025-10-20T22:15:56.849260] [INFO ] [MAIN ] Derivative Progress: 1804/1877 symbols completed. WHG took 1.1s. Estimated remaining: 27.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:56.850246] [INFO ] [MAIN ] WIA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:56.865939] [DEBUG] [MAIN ] WHD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:56.866775] [INFO ] [MAIN ] WHD: Generated 1794 derivative records from 40110 primary records. [2025-10-20T22:15:56.867084] [DEBUG] [LOCAL ] WHD: Saving derivative data. [2025-10-20T22:15:56.881569] [DEBUG] [LOCAL ] WIA: Loaded 5002 existing records from primary storage. [2025-10-20T22:15:56.884705] [INFO ] [MAIN ] WIA: Loaded 5002 primary data records. [2025-10-20T22:15:56.911366] [DEBUG] [LOCAL ] WHD: Derivative data saved (1794 records). [2025-10-20T22:15:56.935305] [INFO ] [MAIN ] WHD: Successfully saved derivative data. [2025-10-20T22:15:56.950441] [INFO ] [MAIN ] Derivative Progress: 1805/1877 symbols completed. WHD took 1.3s. Estimated remaining: 27.0s. Rate: 160.0 symbols/min [2025-10-20T22:15:56.964790] [INFO ] [MAIN ] WINN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:57.013702] [DEBUG] [MAIN ] WHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:57.014895] [INFO ] [MAIN ] WHR: Generated 1794 derivative records from 48938 primary records. [2025-10-20T22:15:57.033761] [DEBUG] [LOCAL ] WHR: Saving derivative data. [2025-10-20T22:15:57.218988] [DEBUG] [LOCAL ] WINN: Loaded 11461 existing records from primary storage. [2025-10-20T22:15:57.219150] [INFO ] [MAIN ] WINN: Loaded 11461 primary data records. [2025-10-20T22:15:57.262296] [DEBUG] [LOCAL ] WHR: Derivative data saved (1794 records). [2025-10-20T22:15:57.262524] [INFO ] [MAIN ] WHR: Successfully saved derivative data. [2025-10-20T22:15:57.265275] [INFO ] [MAIN ] Derivative Progress: 1806/1877 symbols completed. WHR took 1.5s. Estimated remaining: 26.6s. Rate: 160.0 symbols/min [2025-10-20T22:15:57.291047] [INFO ] [MAIN ] WIT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:57.612469] [DEBUG] [MAIN ] WIA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:57.614047] [INFO ] [MAIN ] WIA: Generated 1794 derivative records from 5002 primary records. [2025-10-20T22:15:57.614397] [DEBUG] [LOCAL ] WIA: Saving derivative data. [2025-10-20T22:15:57.633066] [DEBUG] [LOCAL ] WIT: Loaded 52534 existing records from primary storage. [2025-10-20T22:15:57.633308] [INFO ] [MAIN ] WIT: Loaded 52534 primary data records. [2025-10-20T22:15:57.708242] [DEBUG] [LOCAL ] WIA: Derivative data saved (1794 records). [2025-10-20T22:15:57.718077] [INFO ] [MAIN ] WIA: Successfully saved derivative data. [2025-10-20T22:15:57.733956] [INFO ] [MAIN ] Derivative Progress: 1807/1877 symbols completed. WIA took 0.9s. Estimated remaining: 26.2s. Rate: 160.0 symbols/min [2025-10-20T22:15:57.739760] [INFO ] [MAIN ] WIW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:57.853406] [DEBUG] [MAIN ] WINN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:57.983299] [INFO ] [MAIN ] WINN: Generated 1794 derivative records from 11461 primary records. [2025-10-20T22:15:57.998379] [DEBUG] [LOCAL ] WINN: Saving derivative data. [2025-10-20T22:15:58.008358] [DEBUG] [LOCAL ] WIW: Loaded 20283 existing records from primary storage. [2025-10-20T22:15:58.008835] [INFO ] [MAIN ] WIW: Loaded 20283 primary data records. [2025-10-20T22:15:58.072164] [DEBUG] [LOCAL ] WINN: Derivative data saved (1794 records). [2025-10-20T22:15:58.095490] [INFO ] [MAIN ] WINN: Successfully saved derivative data. [2025-10-20T22:15:58.141189] [INFO ] [MAIN ] Derivative Progress: 1808/1877 symbols completed. WINN took 1.2s. Estimated remaining: 25.9s. Rate: 160.0 symbols/min [2025-10-20T22:15:58.147589] [INFO ] [MAIN ] WK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:58.416709] [DEBUG] [MAIN ] WIT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:58.418374] [INFO ] [MAIN ] WIT: Generated 1794 derivative records from 52534 primary records. [2025-10-20T22:15:58.418704] [DEBUG] [LOCAL ] WIT: Saving derivative data. [2025-10-20T22:15:58.442613] [DEBUG] [LOCAL ] WK: Loaded 37168 existing records from primary storage. [2025-10-20T22:15:58.449442] [DEBUG] [LOCAL ] WIT: Derivative data saved (1794 records). [2025-10-20T22:15:58.449686] [INFO ] [MAIN ] WK: Loaded 37168 primary data records. [2025-10-20T22:15:58.450013] [INFO ] [MAIN ] WIT: Successfully saved derivative data. [2025-10-20T22:15:58.457370] [INFO ] [MAIN ] Derivative Progress: 1809/1877 symbols completed. WIT took 1.2s. Estimated remaining: 25.5s. Rate: 160.0 symbols/min [2025-10-20T22:15:58.464588] [INFO ] [MAIN ] WKC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:58.696103] [DEBUG] [MAIN ] WIW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:58.851391] [INFO ] [MAIN ] WIW: Generated 1794 derivative records from 20283 primary records. [2025-10-20T22:15:58.866514] [DEBUG] [LOCAL ] WKC: Loaded 34592 existing records from primary storage. [2025-10-20T22:15:58.866672] [DEBUG] [LOCAL ] WIW: Saving derivative data. [2025-10-20T22:15:58.867971] [INFO ] [MAIN ] WKC: Loaded 34592 primary data records. [2025-10-20T22:15:58.930115] [DEBUG] [LOCAL ] WIW: Derivative data saved (1794 records). [2025-10-20T22:15:58.937475] [INFO ] [MAIN ] WIW: Successfully saved derivative data. [2025-10-20T22:15:58.966344] [INFO ] [MAIN ] Derivative Progress: 1810/1877 symbols completed. WIW took 1.2s. Estimated remaining: 25.1s. Rate: 160.0 symbols/min [2025-10-20T22:15:58.996455] [INFO ] [MAIN ] WLK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:59.230151] [DEBUG] [MAIN ] WK: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:15:59.231160] [INFO ] [MAIN ] WK: Generated 1793 derivative records from 37168 primary records. [2025-10-20T22:15:59.231261] [DEBUG] [LOCAL ] WK: Saving derivative data. [2025-10-20T22:15:59.251120] [DEBUG] [LOCAL ] WK: Derivative data saved (1793 records). [2025-10-20T22:15:59.251859] [INFO ] [MAIN ] WK: Successfully saved derivative data. [2025-10-20T22:15:59.269295] [INFO ] [MAIN ] Derivative Progress: 1811/1877 symbols completed. WK took 1.1s. Estimated remaining: 24.7s. Rate: 160.0 symbols/min [2025-10-20T22:15:59.269800] [INFO ] [MAIN ] WLKP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:59.270855] [DEBUG] [LOCAL ] WLK: Loaded 45903 existing records from primary storage. [2025-10-20T22:15:59.271614] [INFO ] [MAIN ] WLK: Loaded 45903 primary data records. [2025-10-20T22:15:59.379344] [DEBUG] [LOCAL ] WLKP: Loaded 7204 existing records from primary storage. [2025-10-20T22:15:59.411459] [DEBUG] [MAIN ] WKC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:15:59.411249] [INFO ] [MAIN ] WLKP: Loaded 7204 primary data records. [2025-10-20T22:15:59.428958] [INFO ] [MAIN ] WKC: Generated 1794 derivative records from 34592 primary records. [2025-10-20T22:15:59.457416] [DEBUG] [LOCAL ] WKC: Saving derivative data. [2025-10-20T22:15:59.715199] [DEBUG] [LOCAL ] WKC: Derivative data saved (1794 records). [2025-10-20T22:15:59.715533] [INFO ] [MAIN ] WKC: Successfully saved derivative data. [2025-10-20T22:15:59.723310] [INFO ] [MAIN ] Derivative Progress: 1812/1877 symbols completed. WKC took 1.3s. Estimated remaining: 24.4s. Rate: 160.0 symbols/min [2025-10-20T22:15:59.729403] [INFO ] [MAIN ] WLY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:15:59.948769] [DEBUG] [LOCAL ] WLY: Loaded 29783 existing records from primary storage. [2025-10-20T22:15:59.950229] [INFO ] [MAIN ] WLY: Loaded 29783 primary data records. [2025-10-20T22:16:00.015433] [DEBUG] [MAIN ] WLK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:00.023788] [INFO ] [MAIN ] WLK: Generated 1794 derivative records from 45903 primary records. [2025-10-20T22:16:00.062910] [DEBUG] [MAIN ] WLKP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:00.057300] [DEBUG] [LOCAL ] WLK: Saving derivative data. [2025-10-20T22:16:00.063718] [INFO ] [MAIN ] WLKP: Generated 1794 derivative records from 7204 primary records. [2025-10-20T22:16:00.084131] [DEBUG] [LOCAL ] WLKP: Saving derivative data. [2025-10-20T22:16:00.129960] [DEBUG] [LOCAL ] WLK: Derivative data saved (1794 records). [2025-10-20T22:16:00.146441] [INFO ] [MAIN ] WLK: Successfully saved derivative data. [2025-10-20T22:16:00.190014] [INFO ] [MAIN ] Derivative Progress: 1813/1877 symbols completed. WLK took 1.2s. Estimated remaining: 24.0s. Rate: 160.0 symbols/min [2025-10-20T22:16:00.192004] [DEBUG] [LOCAL ] WLKP: Derivative data saved (1794 records). [2025-10-20T22:16:00.207300] [INFO ] [MAIN ] WLKP: Successfully saved derivative data. [2025-10-20T22:16:00.210078] [INFO ] [MAIN ] Derivative Progress: 1814/1877 symbols completed. WLKP took 0.9s. Estimated remaining: 23.6s. Rate: 160.1 symbols/min [2025-10-20T22:16:00.376635] [DEBUG] [MAIN ] WLY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:00.377281] [INFO ] [MAIN ] WLY: Generated 1794 derivative records from 29783 primary records. [2025-10-20T22:16:00.377341] [DEBUG] [LOCAL ] WLY: Saving derivative data. [2025-10-20T22:16:00.391966] [DEBUG] [LOCAL ] WLY: Derivative data saved (1794 records). [2025-10-20T22:16:00.392024] [INFO ] [MAIN ] WLY: Successfully saved derivative data. [2025-10-20T22:16:00.394205] [INFO ] [MAIN ] Derivative Progress: 1815/1877 symbols completed. WLY took 0.7s. Estimated remaining: 23.2s. Rate: 160.1 symbols/min [2025-10-20T22:16:00.394776] [INFO ] [MAIN ] WMK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:00.395447] [INFO ] [MAIN ] WMB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:00.395243] [INFO ] [MAIN ] WM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:00.412762] [DEBUG] [LOCAL ] WMK: Loaded 14754 existing records from primary storage. [2025-10-20T22:16:00.412906] [INFO ] [MAIN ] WMK: Loaded 14754 primary data records. [2025-10-20T22:16:00.755968] [DEBUG] [LOCAL ] WMB: Loaded 53672 existing records from primary storage. [2025-10-20T22:16:00.771140] [INFO ] [MAIN ] WMB: Loaded 53672 primary data records. [2025-10-20T22:16:00.802777] [DEBUG] [LOCAL ] WM: Loaded 50913 existing records from primary storage. [2025-10-20T22:16:00.815749] [INFO ] [MAIN ] WM: Loaded 50913 primary data records. [2025-10-20T22:16:01.439809] [DEBUG] [MAIN ] WMK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:01.441060] [INFO ] [MAIN ] WMK: Generated 1794 derivative records from 14754 primary records. [2025-10-20T22:16:01.441844] [DEBUG] [LOCAL ] WMK: Saving derivative data. [2025-10-20T22:16:01.461046] [DEBUG] [LOCAL ] WMK: Derivative data saved (1794 records). [2025-10-20T22:16:01.461862] [INFO ] [MAIN ] WMK: Successfully saved derivative data. [2025-10-20T22:16:01.464856] [INFO ] [MAIN ] Derivative Progress: 1816/1877 symbols completed. WMK took 1.1s. Estimated remaining: 22.9s. Rate: 159.9 symbols/min [2025-10-20T22:16:01.465386] [INFO ] [MAIN ] WMS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:01.509448] [DEBUG] [MAIN ] WMB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:01.523996] [DEBUG] [MAIN ] WM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:01.523132] [INFO ] [MAIN ] WMB: Generated 1794 derivative records from 53672 primary records. [2025-10-20T22:16:01.524494] [DEBUG] [LOCAL ] WMB: Saving derivative data. [2025-10-20T22:16:01.525403] [INFO ] [MAIN ] WM: Generated 1794 derivative records from 50913 primary records. [2025-10-20T22:16:01.525580] [DEBUG] [LOCAL ] WM: Saving derivative data. [2025-10-20T22:16:01.574307] [DEBUG] [LOCAL ] WM: Derivative data saved (1794 records). [2025-10-20T22:16:01.571886] [DEBUG] [LOCAL ] WMB: Derivative data saved (1794 records). [2025-10-20T22:16:01.575061] [INFO ] [MAIN ] WM: Successfully saved derivative data. [2025-10-20T22:16:01.574983] [INFO ] [MAIN ] WMB: Successfully saved derivative data. [2025-10-20T22:16:01.576899] [INFO ] [MAIN ] Derivative Progress: 1817/1877 symbols completed. WMB took 1.2s. Estimated remaining: 22.5s. Rate: 160.0 symbols/min [2025-10-20T22:16:01.592467] [INFO ] [MAIN ] Derivative Progress: 1818/1877 symbols completed. WM took 1.2s. Estimated remaining: 22.1s. Rate: 160.1 symbols/min [2025-10-20T22:16:01.591259] [INFO ] [MAIN ] WMT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:01.593020] [INFO ] [MAIN ] WNC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:01.593913] [DEBUG] [LOCAL ] WMS: Loaded 39655 existing records from primary storage. [2025-10-20T22:16:01.594200] [INFO ] [MAIN ] WMS: Loaded 39655 primary data records. [2025-10-20T22:16:01.959052] [DEBUG] [LOCAL ] WNC: Loaded 38400 existing records from primary storage. [2025-10-20T22:16:01.960092] [INFO ] [MAIN ] WNC: Loaded 38400 primary data records. [2025-10-20T22:16:02.076092] [DEBUG] [LOCAL ] WMT: Loaded 53800 existing records from primary storage. [2025-10-20T22:16:02.111901] [INFO ] [MAIN ] WMT: Loaded 53800 primary data records. [2025-10-20T22:16:02.688557] [DEBUG] [MAIN ] WMS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:02.690157] [INFO ] [MAIN ] WMS: Generated 1794 derivative records from 39655 primary records. [2025-10-20T22:16:02.690288] [DEBUG] [LOCAL ] WMS: Saving derivative data. [2025-10-20T22:16:02.709111] [DEBUG] [LOCAL ] WMS: Derivative data saved (1794 records). [2025-10-20T22:16:02.709605] [INFO ] [MAIN ] WMS: Successfully saved derivative data. [2025-10-20T22:16:02.711597] [INFO ] [MAIN ] Derivative Progress: 1819/1877 symbols completed. WMS took 1.2s. Estimated remaining: 21.8s. Rate: 159.9 symbols/min [2025-10-20T22:16:02.712259] [INFO ] [MAIN ] WOLF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:02.731990] [DEBUG] [MAIN ] WNC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:02.732804] [INFO ] [MAIN ] WNC: Generated 1794 derivative records from 38400 primary records. [2025-10-20T22:16:02.733092] [DEBUG] [LOCAL ] WNC: Saving derivative data. [2025-10-20T22:16:02.752303] [DEBUG] [LOCAL ] WNC: Derivative data saved (1794 records). [2025-10-20T22:16:02.752483] [INFO ] [MAIN ] WNC: Successfully saved derivative data. [2025-10-20T22:16:02.754308] [INFO ] [MAIN ] Derivative Progress: 1820/1877 symbols completed. WNC took 1.2s. Estimated remaining: 21.4s. Rate: 160.0 symbols/min [2025-10-20T22:16:02.754828] [INFO ] [MAIN ] WOR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:02.767996] [DEBUG] [MAIN ] WMT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:02.768613] [INFO ] [MAIN ] WMT: Generated 1794 derivative records from 53800 primary records. [2025-10-20T22:16:02.768795] [DEBUG] [LOCAL ] WMT: Saving derivative data. [2025-10-20T22:16:02.797797] [DEBUG] [LOCAL ] WMT: Derivative data saved (1794 records). [2025-10-20T22:16:02.798071] [INFO ] [MAIN ] WMT: Successfully saved derivative data. [2025-10-20T22:16:02.800462] [INFO ] [MAIN ] Derivative Progress: 1821/1877 symbols completed. WMT took 1.2s. Estimated remaining: 21.0s. Rate: 160.1 symbols/min [2025-10-20T22:16:02.800765] [INFO ] [MAIN ] WOW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:02.832233] [DEBUG] [LOCAL ] WOR: Loaded 23740 existing records from primary storage. [2025-10-20T22:16:02.832513] [INFO ] [MAIN ] WOR: Loaded 23740 primary data records. [2025-10-20T22:16:02.833389] [DEBUG] [LOCAL ] WOLF: Loaded 53128 existing records from primary storage. [2025-10-20T22:16:02.833684] [INFO ] [MAIN ] WOLF: Loaded 53128 primary data records. [2025-10-20T22:16:03.082527] [DEBUG] [LOCAL ] WOW: Loaded 31259 existing records from primary storage. [2025-10-20T22:16:03.122474] [INFO ] [MAIN ] WOW: Loaded 31259 primary data records. [2025-10-20T22:16:03.795514] [DEBUG] [MAIN ] WOLF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:03.795785] [DEBUG] [MAIN ] WOR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:03.796672] [INFO ] [MAIN ] WOLF: Generated 1794 derivative records from 53128 primary records. [2025-10-20T22:16:03.797484] [INFO ] [MAIN ] WOR: Generated 1794 derivative records from 23740 primary records. [2025-10-20T22:16:03.796931] [DEBUG] [LOCAL ] WOLF: Saving derivative data. [2025-10-20T22:16:03.797578] [DEBUG] [LOCAL ] WOR: Saving derivative data. [2025-10-20T22:16:03.834326] [DEBUG] [LOCAL ] WOLF: Derivative data saved (1794 records). [2025-10-20T22:16:03.833230] [DEBUG] [LOCAL ] WOR: Derivative data saved (1794 records). [2025-10-20T22:16:03.834529] [INFO ] [MAIN ] WOR: Successfully saved derivative data. [2025-10-20T22:16:03.836671] [INFO ] [MAIN ] WOLF: Successfully saved derivative data. [2025-10-20T22:16:03.836588] [INFO ] [MAIN ] Derivative Progress: 1822/1877 symbols completed. WOR took 1.1s. Estimated remaining: 20.6s. Rate: 159.9 symbols/min [2025-10-20T22:16:03.851176] [INFO ] [MAIN ] WPC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:03.851477] [INFO ] [MAIN ] Derivative Progress: 1823/1877 symbols completed. WOLF took 1.1s. Estimated remaining: 20.3s. Rate: 160.0 symbols/min [2025-10-20T22:16:03.852305] [INFO ] [MAIN ] WPM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:03.852622] [DEBUG] [MAIN ] WOW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:03.853456] [INFO ] [MAIN ] WOW: Generated 1794 derivative records from 31259 primary records. [2025-10-20T22:16:03.853605] [DEBUG] [LOCAL ] WOW: Saving derivative data. [2025-10-20T22:16:03.869220] [DEBUG] [LOCAL ] WOW: Derivative data saved (1794 records). [2025-10-20T22:16:03.869332] [INFO ] [MAIN ] WOW: Successfully saved derivative data. [2025-10-20T22:16:03.871387] [INFO ] [MAIN ] Derivative Progress: 1824/1877 symbols completed. WOW took 1.1s. Estimated remaining: 19.9s. Rate: 160.1 symbols/min [2025-10-20T22:16:03.871610] [INFO ] [MAIN ] WPP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:03.929701] [DEBUG] [LOCAL ] WPP: Loaded 34370 existing records from primary storage. [2025-10-20T22:16:03.929855] [INFO ] [MAIN ] WPP: Loaded 34370 primary data records. [2025-10-20T22:16:04.067043] [DEBUG] [LOCAL ] WPM: Loaded 52786 existing records from primary storage. [2025-10-20T22:16:04.069456] [INFO ] [MAIN ] WPM: Loaded 52786 primary data records. [2025-10-20T22:16:04.088253] [DEBUG] [LOCAL ] WPC: Loaded 48228 existing records from primary storage. [2025-10-20T22:16:04.164683] [INFO ] [MAIN ] WPC: Loaded 48228 primary data records. [2025-10-20T22:16:04.914712] [DEBUG] [MAIN ] WPP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:04.915862] [INFO ] [MAIN ] WPP: Generated 1794 derivative records from 34370 primary records. [2025-10-20T22:16:04.916027] [DEBUG] [LOCAL ] WPP: Saving derivative data. [2025-10-20T22:16:04.937347] [DEBUG] [LOCAL ] WPP: Derivative data saved (1794 records). [2025-10-20T22:16:04.963513] [INFO ] [MAIN ] WPP: Successfully saved derivative data. [2025-10-20T22:16:04.966154] [INFO ] [MAIN ] Derivative Progress: 1825/1877 symbols completed. WPP took 1.1s. Estimated remaining: 19.5s. Rate: 159.9 symbols/min [2025-10-20T22:16:04.966303] [INFO ] [MAIN ] WRB: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:04.964431] [DEBUG] [MAIN ] WPM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:04.967762] [INFO ] [MAIN ] WPM: Generated 1794 derivative records from 52786 primary records. [2025-10-20T22:16:04.968144] [DEBUG] [MAIN ] WPC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:04.968219] [DEBUG] [LOCAL ] WPM: Saving derivative data. [2025-10-20T22:16:04.969058] [INFO ] [MAIN ] WPC: Generated 1794 derivative records from 48228 primary records. [2025-10-20T22:16:04.969537] [DEBUG] [LOCAL ] WPC: Saving derivative data. [2025-10-20T22:16:04.999850] [DEBUG] [LOCAL ] WPC: Derivative data saved (1794 records). [2025-10-20T22:16:05.001406] [INFO ] [MAIN ] WPC: Successfully saved derivative data. [2025-10-20T22:16:05.003841] [INFO ] [MAIN ] Derivative Progress: 1826/1877 symbols completed. WPC took 1.2s. Estimated remaining: 19.1s. Rate: 160.0 symbols/min [2025-10-20T22:16:05.004346] [INFO ] [MAIN ] WRBY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:05.006001] [DEBUG] [LOCAL ] WPM: Derivative data saved (1794 records). [2025-10-20T22:16:05.006544] [INFO ] [MAIN ] WPM: Successfully saved derivative data. [2025-10-20T22:16:05.009330] [INFO ] [MAIN ] Derivative Progress: 1827/1877 symbols completed. WPM took 1.2s. Estimated remaining: 18.7s. Rate: 160.1 symbols/min [2025-10-20T22:16:05.009743] [INFO ] [MAIN ] WS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:05.050728] [DEBUG] [LOCAL ] WRB: Loaded 50781 existing records from primary storage. [2025-10-20T22:16:05.051024] [INFO ] [MAIN ] WRB: Loaded 50781 primary data records. [2025-10-20T22:16:05.196884] [DEBUG] [LOCAL ] WS: Loaded 23540 existing records from primary storage. [2025-10-20T22:16:05.253157] [INFO ] [MAIN ] WS: Loaded 23540 primary data records. [2025-10-20T22:16:05.491331] [DEBUG] [LOCAL ] WRBY: Loaded 50728 existing records from primary storage. [2025-10-20T22:16:05.507090] [INFO ] [MAIN ] WRBY: Loaded 50728 primary data records. [2025-10-20T22:16:06.013929] [DEBUG] [MAIN ] WRB: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:06.014929] [INFO ] [MAIN ] WRB: Generated 1794 derivative records from 50781 primary records. [2025-10-20T22:16:06.015118] [DEBUG] [LOCAL ] WRB: Saving derivative data. [2025-10-20T22:16:06.033820] [DEBUG] [LOCAL ] WRB: Derivative data saved (1794 records). [2025-10-20T22:16:06.034157] [INFO ] [MAIN ] WRB: Successfully saved derivative data. [2025-10-20T22:16:06.036210] [INFO ] [MAIN ] Derivative Progress: 1828/1877 symbols completed. WRB took 1.1s. Estimated remaining: 18.4s. Rate: 159.9 symbols/min [2025-10-20T22:16:06.057707] [DEBUG] [MAIN ] WS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:06.058881] [INFO ] [MAIN ] WS: Generated 1794 derivative records from 23540 primary records. [2025-10-20T22:16:06.059105] [DEBUG] [LOCAL ] WS: Saving derivative data. [2025-10-20T22:16:06.096014] [DEBUG] [MAIN ] WRBY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:06.098147] [INFO ] [MAIN ] WRBY: Generated 1794 derivative records from 50728 primary records. [2025-10-20T22:16:06.098461] [DEBUG] [LOCAL ] WRBY: Saving derivative data. [2025-10-20T22:16:06.100389] [DEBUG] [LOCAL ] WS: Derivative data saved (1794 records). [2025-10-20T22:16:06.100695] [INFO ] [MAIN ] WS: Successfully saved derivative data. [2025-10-20T22:16:06.111566] [INFO ] [MAIN ] Derivative Progress: 1829/1877 symbols completed. WS took 1.1s. Estimated remaining: 18.0s. Rate: 160.0 symbols/min [2025-10-20T22:16:06.118736] [DEBUG] [LOCAL ] WRBY: Derivative data saved (1794 records). [2025-10-20T22:16:06.118833] [INFO ] [MAIN ] WRBY: Successfully saved derivative data. [2025-10-20T22:16:06.121176] [INFO ] [MAIN ] Derivative Progress: 1830/1877 symbols completed. WRBY took 1.1s. Estimated remaining: 17.6s. Rate: 160.1 symbols/min [2025-10-20T22:16:06.121719] [INFO ] [MAIN ] WSR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:06.121628] [INFO ] [MAIN ] WSM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:06.121786] [INFO ] [MAIN ] WSO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:06.164595] [DEBUG] [LOCAL ] WSR: Loaded 22806 existing records from primary storage. [2025-10-20T22:16:06.164803] [INFO ] [MAIN ] WSR: Loaded 22806 primary data records. [2025-10-20T22:16:06.289553] [DEBUG] [LOCAL ] WSO: Loaded 25899 existing records from primary storage. [2025-10-20T22:16:06.293173] [INFO ] [MAIN ] WSO: Loaded 25899 primary data records. [2025-10-20T22:16:06.446638] [DEBUG] [LOCAL ] WSM: Loaded 48892 existing records from primary storage. [2025-10-20T22:16:06.470693] [INFO ] [MAIN ] WSM: Loaded 48892 primary data records. [2025-10-20T22:16:07.180197] [DEBUG] [MAIN ] WSR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:07.181155] [INFO ] [MAIN ] WSR: Generated 1794 derivative records from 22806 primary records. [2025-10-20T22:16:07.181437] [DEBUG] [LOCAL ] WSR: Saving derivative data. [2025-10-20T22:16:07.200171] [DEBUG] [LOCAL ] WSR: Derivative data saved (1794 records). [2025-10-20T22:16:07.213756] [INFO ] [MAIN ] WSR: Successfully saved derivative data. [2025-10-20T22:16:07.213579] [DEBUG] [MAIN ] WSO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:07.215233] [INFO ] [MAIN ] WSO: Generated 1794 derivative records from 25899 primary records. [2025-10-20T22:16:07.216027] [DEBUG] [LOCAL ] WSO: Saving derivative data. [2025-10-20T22:16:07.219850] [INFO ] [MAIN ] Derivative Progress: 1831/1877 symbols completed. WSR took 1.1s. Estimated remaining: 17.3s. Rate: 159.9 symbols/min [2025-10-20T22:16:07.235471] [INFO ] [MAIN ] WST: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:07.235350] [DEBUG] [MAIN ] WSM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:07.253944] [INFO ] [MAIN ] WSM: Generated 1794 derivative records from 48892 primary records. [2025-10-20T22:16:07.256040] [DEBUG] [LOCAL ] WSO: Derivative data saved (1794 records). [2025-10-20T22:16:07.256425] [DEBUG] [LOCAL ] WSM: Saving derivative data. [2025-10-20T22:16:07.256623] [INFO ] [MAIN ] WSO: Successfully saved derivative data. [2025-10-20T22:16:07.259749] [INFO ] [MAIN ] Derivative Progress: 1832/1877 symbols completed. WSO took 1.1s. Estimated remaining: 16.9s. Rate: 160.0 symbols/min [2025-10-20T22:16:07.261229] [INFO ] [MAIN ] WT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:07.280508] [DEBUG] [LOCAL ] WSM: Derivative data saved (1794 records). [2025-10-20T22:16:07.280753] [INFO ] [MAIN ] WSM: Successfully saved derivative data. [2025-10-20T22:16:07.283406] [INFO ] [MAIN ] Derivative Progress: 1833/1877 symbols completed. WSM took 1.2s. Estimated remaining: 16.5s. Rate: 160.1 symbols/min [2025-10-20T22:16:07.283742] [INFO ] [MAIN ] WTI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:07.325191] [DEBUG] [LOCAL ] WST: Loaded 36673 existing records from primary storage. [2025-10-20T22:16:07.325411] [INFO ] [MAIN ] WST: Loaded 36673 primary data records. [2025-10-20T22:16:07.456958] [DEBUG] [LOCAL ] WT: Loaded 48165 existing records from primary storage. [2025-10-20T22:16:07.462736] [INFO ] [MAIN ] WT: Loaded 48165 primary data records. [2025-10-20T22:16:07.775765] [DEBUG] [LOCAL ] WTI: Loaded 44707 existing records from primary storage. [2025-10-20T22:16:07.777148] [INFO ] [MAIN ] WTI: Loaded 44707 primary data records. [2025-10-20T22:16:08.339572] [DEBUG] [MAIN ] WST: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:08.341080] [INFO ] [MAIN ] WST: Generated 1794 derivative records from 36673 primary records. [2025-10-20T22:16:08.341974] [DEBUG] [LOCAL ] WST: Saving derivative data. [2025-10-20T22:16:08.361303] [DEBUG] [LOCAL ] WST: Derivative data saved (1794 records). [2025-10-20T22:16:08.361929] [INFO ] [MAIN ] WST: Successfully saved derivative data. [2025-10-20T22:16:08.378083] [DEBUG] [MAIN ] WT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:08.376755] [INFO ] [MAIN ] Derivative Progress: 1834/1877 symbols completed. WST took 1.1s. Estimated remaining: 16.1s. Rate: 159.9 symbols/min [2025-10-20T22:16:08.379151] [INFO ] [MAIN ] WT: Generated 1794 derivative records from 48165 primary records. [2025-10-20T22:16:08.378319] [INFO ] [MAIN ] WTRG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:08.379378] [DEBUG] [LOCAL ] WT: Saving derivative data. [2025-10-20T22:16:08.397503] [DEBUG] [LOCAL ] WT: Derivative data saved (1794 records). [2025-10-20T22:16:08.397738] [INFO ] [MAIN ] WT: Successfully saved derivative data. [2025-10-20T22:16:08.399990] [INFO ] [MAIN ] Derivative Progress: 1835/1877 symbols completed. WT took 1.1s. Estimated remaining: 15.8s. Rate: 160.0 symbols/min [2025-10-20T22:16:08.400521] [INFO ] [MAIN ] WTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:08.417111] [DEBUG] [MAIN ] WTI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:08.417988] [INFO ] [MAIN ] WTI: Generated 1794 derivative records from 44707 primary records. [2025-10-20T22:16:08.418218] [DEBUG] [LOCAL ] WTI: Saving derivative data. [2025-10-20T22:16:08.439083] [DEBUG] [LOCAL ] WTI: Derivative data saved (1794 records). [2025-10-20T22:16:08.439329] [INFO ] [MAIN ] WTI: Successfully saved derivative data. [2025-10-20T22:16:08.441341] [INFO ] [MAIN ] Derivative Progress: 1836/1877 symbols completed. WTI took 1.2s. Estimated remaining: 15.4s. Rate: 160.1 symbols/min [2025-10-20T22:16:08.441663] [INFO ] [MAIN ] WTTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:08.448713] [DEBUG] [LOCAL ] WTS: Loaded 17355 existing records from primary storage. [2025-10-20T22:16:08.448874] [INFO ] [MAIN ] WTS: Loaded 17355 primary data records. [2025-10-20T22:16:08.599440] [DEBUG] [LOCAL ] WTRG: Loaded 50966 existing records from primary storage. [2025-10-20T22:16:08.610149] [INFO ] [MAIN ] WTRG: Loaded 50966 primary data records. [2025-10-20T22:16:08.942271] [DEBUG] [LOCAL ] WTTR: Loaded 44633 existing records from primary storage. [2025-10-20T22:16:08.953433] [INFO ] [MAIN ] WTTR: Loaded 44633 primary data records. [2025-10-20T22:16:09.592680] [DEBUG] [MAIN ] WTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:09.594508] [INFO ] [MAIN ] WTS: Generated 1794 derivative records from 17355 primary records. [2025-10-20T22:16:09.594818] [DEBUG] [LOCAL ] WTS: Saving derivative data. [2025-10-20T22:16:09.623786] [DEBUG] [LOCAL ] WTS: Derivative data saved (1794 records). [2025-10-20T22:16:09.624639] [INFO ] [MAIN ] WTS: Successfully saved derivative data. [2025-10-20T22:16:09.626737] [INFO ] [MAIN ] Derivative Progress: 1837/1877 symbols completed. WTS took 1.2s. Estimated remaining: 15.0s. Rate: 159.9 symbols/min [2025-10-20T22:16:09.627119] [INFO ] [MAIN ] WU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:09.652470] [DEBUG] [MAIN ] WTRG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:09.653905] [INFO ] [MAIN ] WTRG: Generated 1794 derivative records from 50966 primary records. [2025-10-20T22:16:09.654039] [DEBUG] [LOCAL ] WTRG: Saving derivative data. [2025-10-20T22:16:09.692989] [DEBUG] [LOCAL ] WTRG: Derivative data saved (1794 records). [2025-10-20T22:16:09.710725] [DEBUG] [MAIN ] WTTR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:09.712045] [INFO ] [MAIN ] WTTR: Generated 1794 derivative records from 44633 primary records. [2025-10-20T22:16:09.710989] [INFO ] [MAIN ] WTRG: Successfully saved derivative data. [2025-10-20T22:16:09.712253] [DEBUG] [LOCAL ] WTTR: Saving derivative data. [2025-10-20T22:16:09.714937] [INFO ] [MAIN ] Derivative Progress: 1838/1877 symbols completed. WTRG took 1.3s. Estimated remaining: 14.6s. Rate: 159.9 symbols/min [2025-10-20T22:16:09.715230] [INFO ] [MAIN ] WWW: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:09.764338] [DEBUG] [LOCAL ] WTTR: Derivative data saved (1794 records). [2025-10-20T22:16:09.764614] [INFO ] [MAIN ] WTTR: Successfully saved derivative data. [2025-10-20T22:16:09.766583] [INFO ] [MAIN ] Derivative Progress: 1839/1877 symbols completed. WTTR took 1.3s. Estimated remaining: 14.2s. Rate: 160.0 symbols/min [2025-10-20T22:16:09.769246] [INFO ] [MAIN ] WY: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:09.768917] [DEBUG] [LOCAL ] WU: Loaded 53492 existing records from primary storage. [2025-10-20T22:16:09.770017] [INFO ] [MAIN ] WU: Loaded 53492 primary data records. [2025-10-20T22:16:10.194634] [DEBUG] [LOCAL ] WWW: Loaded 49279 existing records from primary storage. [2025-10-20T22:16:10.196297] [INFO ] [MAIN ] WWW: Loaded 49279 primary data records. [2025-10-20T22:16:10.259744] [DEBUG] [LOCAL ] WY: Loaded 53216 existing records from primary storage. [2025-10-20T22:16:10.284851] [INFO ] [MAIN ] WY: Loaded 53216 primary data records. [2025-10-20T22:16:10.943139] [DEBUG] [MAIN ] WU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:10.944268] [INFO ] [MAIN ] WU: Generated 1794 derivative records from 53492 primary records. [2025-10-20T22:16:10.944512] [DEBUG] [LOCAL ] WU: Saving derivative data. [2025-10-20T22:16:10.962220] [DEBUG] [LOCAL ] WU: Derivative data saved (1794 records). [2025-10-20T22:16:10.963550] [INFO ] [MAIN ] WU: Successfully saved derivative data. [2025-10-20T22:16:10.965727] [INFO ] [MAIN ] Derivative Progress: 1840/1877 symbols completed. WU took 1.3s. Estimated remaining: 13.9s. Rate: 159.8 symbols/min [2025-10-20T22:16:10.966088] [INFO ] [MAIN ] XFLT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:10.987018] [DEBUG] [MAIN ] WWW: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:10.995224] [INFO ] [MAIN ] WWW: Generated 1794 derivative records from 49279 primary records. [2025-10-20T22:16:10.995324] [DEBUG] [LOCAL ] WWW: Saving derivative data. [2025-10-20T22:16:11.000670] [DEBUG] [MAIN ] WY: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:11.001483] [INFO ] [MAIN ] WY: Generated 1794 derivative records from 53216 primary records. [2025-10-20T22:16:11.012850] [DEBUG] [LOCAL ] WY: Saving derivative data. [2025-10-20T22:16:11.025094] [DEBUG] [LOCAL ] WWW: Derivative data saved (1794 records). [2025-10-20T22:16:11.025395] [INFO ] [MAIN ] WWW: Successfully saved derivative data. [2025-10-20T22:16:11.035134] [INFO ] [MAIN ] Derivative Progress: 1841/1877 symbols completed. WWW took 1.3s. Estimated remaining: 13.5s. Rate: 159.9 symbols/min [2025-10-20T22:16:11.038095] [INFO ] [MAIN ] XHR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:11.042251] [DEBUG] [LOCAL ] WY: Derivative data saved (1794 records). [2025-10-20T22:16:11.042756] [INFO ] [MAIN ] WY: Successfully saved derivative data. [2025-10-20T22:16:11.044696] [INFO ] [MAIN ] Derivative Progress: 1842/1877 symbols completed. WY took 1.3s. Estimated remaining: 13.1s. Rate: 160.0 symbols/min [2025-10-20T22:16:11.044936] [INFO ] [MAIN ] XOM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:11.055797] [DEBUG] [LOCAL ] XFLT: Loaded 29989 existing records from primary storage. [2025-10-20T22:16:11.056072] [INFO ] [MAIN ] XFLT: Loaded 29989 primary data records. [2025-10-20T22:16:11.387413] [DEBUG] [LOCAL ] XHR: Loaded 43474 existing records from primary storage. [2025-10-20T22:16:11.387749] [INFO ] [MAIN ] XHR: Loaded 43474 primary data records. [2025-10-20T22:16:11.514544] [DEBUG] [LOCAL ] XOM: Loaded 53793 existing records from primary storage. [2025-10-20T22:16:11.556619] [INFO ] [MAIN ] XOM: Loaded 53793 primary data records. [2025-10-20T22:16:12.015902] [DEBUG] [MAIN ] XFLT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:12.016825] [INFO ] [MAIN ] XFLT: Generated 1794 derivative records from 29989 primary records. [2025-10-20T22:16:12.017040] [DEBUG] [LOCAL ] XFLT: Saving derivative data. [2025-10-20T22:16:12.033574] [DEBUG] [LOCAL ] XFLT: Derivative data saved (1794 records). [2025-10-20T22:16:12.034683] [INFO ] [MAIN ] XFLT: Successfully saved derivative data. [2025-10-20T22:16:12.036746] [INFO ] [MAIN ] Derivative Progress: 1843/1877 symbols completed. XFLT took 1.1s. Estimated remaining: 12.8s. Rate: 159.8 symbols/min [2025-10-20T22:16:12.054578] [DEBUG] [MAIN ] XHR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:12.055262] [INFO ] [MAIN ] XHR: Generated 1794 derivative records from 43474 primary records. [2025-10-20T22:16:12.055404] [DEBUG] [LOCAL ] XHR: Saving derivative data. [2025-10-20T22:16:12.072164] [DEBUG] [LOCAL ] XHR: Derivative data saved (1794 records). [2025-10-20T22:16:12.072931] [INFO ] [MAIN ] XHR: Successfully saved derivative data. [2025-10-20T22:16:12.075311] [INFO ] [MAIN ] Derivative Progress: 1844/1877 symbols completed. XHR took 1.0s. Estimated remaining: 12.4s. Rate: 159.9 symbols/min [2025-10-20T22:16:12.087907] [DEBUG] [MAIN ] XOM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:12.088458] [INFO ] [MAIN ] XOM: Generated 1794 derivative records from 53793 primary records. [2025-10-20T22:16:12.088516] [DEBUG] [LOCAL ] XOM: Saving derivative data. [2025-10-20T22:16:12.103453] [DEBUG] [LOCAL ] XOM: Derivative data saved (1794 records). [2025-10-20T22:16:12.103510] [INFO ] [MAIN ] XOM: Successfully saved derivative data. [2025-10-20T22:16:12.105608] [INFO ] [MAIN ] Derivative Progress: 1845/1877 symbols completed. XOM took 1.1s. Estimated remaining: 12.0s. Rate: 160.0 symbols/min [2025-10-20T22:16:12.106286] [INFO ] [MAIN ] XPER: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:12.106386] [INFO ] [MAIN ] XPEV: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:12.106496] [INFO ] [MAIN ] XPO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:12.164703] [DEBUG] [LOCAL ] XPER: Loaded 30766 existing records from primary storage. [2025-10-20T22:16:12.164975] [INFO ] [MAIN ] XPER: Loaded 30766 primary data records. [2025-10-20T22:16:12.305073] [DEBUG] [LOCAL ] XPO: Loaded 47569 existing records from primary storage. [2025-10-20T22:16:12.309009] [INFO ] [MAIN ] XPO: Loaded 47569 primary data records. [2025-10-20T22:16:12.327631] [DEBUG] [LOCAL ] XPEV: Loaded 53624 existing records from primary storage. [2025-10-20T22:16:12.327897] [INFO ] [MAIN ] XPEV: Loaded 53624 primary data records. [2025-10-20T22:16:13.231086] [DEBUG] [MAIN ] XPER: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:13.232035] [INFO ] [MAIN ] XPER: Generated 1794 derivative records from 30766 primary records. [2025-10-20T22:16:13.233804] [DEBUG] [LOCAL ] XPER: Saving derivative data. [2025-10-20T22:16:13.250432] [DEBUG] [LOCAL ] XPER: Derivative data saved (1794 records). [2025-10-20T22:16:13.250930] [INFO ] [MAIN ] XPER: Successfully saved derivative data. [2025-10-20T22:16:13.252786] [INFO ] [MAIN ] Derivative Progress: 1846/1877 symbols completed. XPER took 1.1s. Estimated remaining: 11.6s. Rate: 159.8 symbols/min [2025-10-20T22:16:13.253325] [INFO ] [MAIN ] XPOF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:13.276793] [DEBUG] [MAIN ] XPO: Filtered 7001 bins outside market hours, 7821 incomplete bins [2025-10-20T22:16:13.285454] [INFO ] [MAIN ] XPO: Generated 1793 derivative records from 47569 primary records. [2025-10-20T22:16:13.290623] [DEBUG] [LOCAL ] XPO: Saving derivative data. [2025-10-20T22:16:13.290898] [DEBUG] [MAIN ] XPEV: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:13.292150] [INFO ] [MAIN ] XPEV: Generated 1794 derivative records from 53624 primary records. [2025-10-20T22:16:13.292586] [DEBUG] [LOCAL ] XPEV: Saving derivative data. [2025-10-20T22:16:13.317048] [DEBUG] [LOCAL ] XPO: Derivative data saved (1793 records). [2025-10-20T22:16:13.327911] [INFO ] [MAIN ] XPO: Successfully saved derivative data. [2025-10-20T22:16:13.335017] [INFO ] [MAIN ] Derivative Progress: 1847/1877 symbols completed. XPO took 1.2s. Estimated remaining: 11.3s. Rate: 159.9 symbols/min [2025-10-20T22:16:13.332822] [DEBUG] [LOCAL ] XPEV: Derivative data saved (1794 records). [2025-10-20T22:16:13.335221] [INFO ] [MAIN ] XPEV: Successfully saved derivative data. [2025-10-20T22:16:13.337713] [INFO ] [MAIN ] XPRO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:13.337796] [INFO ] [MAIN ] Derivative Progress: 1848/1877 symbols completed. XPEV took 1.2s. Estimated remaining: 10.9s. Rate: 160.0 symbols/min [2025-10-20T22:16:13.338215] [INFO ] [MAIN ] XYF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:13.354226] [DEBUG] [LOCAL ] XPOF: Loaded 37197 existing records from primary storage. [2025-10-20T22:16:13.354362] [INFO ] [MAIN ] XPOF: Loaded 37197 primary data records. [2025-10-20T22:16:13.523778] [DEBUG] [LOCAL ] XYF: Loaded 27638 existing records from primary storage. [2025-10-20T22:16:13.540272] [INFO ] [MAIN ] XYF: Loaded 27638 primary data records. [2025-10-20T22:16:13.621968] [DEBUG] [LOCAL ] XPRO: Loaded 45873 existing records from primary storage. [2025-10-20T22:16:13.645206] [INFO ] [MAIN ] XPRO: Loaded 45873 primary data records. [2025-10-20T22:16:14.371403] [DEBUG] [MAIN ] XPOF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:14.372676] [INFO ] [MAIN ] XPOF: Generated 1794 derivative records from 37197 primary records. [2025-10-20T22:16:14.372776] [DEBUG] [LOCAL ] XPOF: Saving derivative data. [2025-10-20T22:16:14.392798] [DEBUG] [LOCAL ] XPOF: Derivative data saved (1794 records). [2025-10-20T22:16:14.393077] [INFO ] [MAIN ] XPOF: Successfully saved derivative data. [2025-10-20T22:16:14.395110] [INFO ] [MAIN ] Derivative Progress: 1849/1877 symbols completed. XPOF took 1.1s. Estimated remaining: 10.5s. Rate: 159.8 symbols/min [2025-10-20T22:16:14.395415] [INFO ] [MAIN ] XYL: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:14.411384] [DEBUG] [MAIN ] XYF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:14.412230] [INFO ] [MAIN ] XYF: Generated 1794 derivative records from 27638 primary records. [2025-10-20T22:16:14.412420] [DEBUG] [LOCAL ] XYF: Saving derivative data. [2025-10-20T22:16:14.433611] [DEBUG] [MAIN ] XPRO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:14.441691] [INFO ] [MAIN ] XPRO: Generated 1794 derivative records from 45873 primary records. [2025-10-20T22:16:14.443146] [DEBUG] [LOCAL ] XPRO: Saving derivative data. [2025-10-20T22:16:14.442985] [DEBUG] [LOCAL ] XYF: Derivative data saved (1794 records). [2025-10-20T22:16:14.444345] [INFO ] [MAIN ] XYF: Successfully saved derivative data. [2025-10-20T22:16:14.446392] [INFO ] [MAIN ] Derivative Progress: 1850/1877 symbols completed. XYF took 1.1s. Estimated remaining: 10.1s. Rate: 159.9 symbols/min [2025-10-20T22:16:14.454979] [INFO ] [MAIN ] YALA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:14.463246] [DEBUG] [LOCAL ] XPRO: Derivative data saved (1794 records). [2025-10-20T22:16:14.463519] [INFO ] [MAIN ] XPRO: Successfully saved derivative data. [2025-10-20T22:16:14.465864] [INFO ] [MAIN ] Derivative Progress: 1851/1877 symbols completed. XPRO took 1.1s. Estimated remaining: 9.8s. Rate: 160.0 symbols/min [2025-10-20T22:16:14.466138] [INFO ] [MAIN ] YELP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:14.521452] [DEBUG] [LOCAL ] XYL: Loaded 47669 existing records from primary storage. [2025-10-20T22:16:14.531943] [INFO ] [MAIN ] XYL: Loaded 47669 primary data records. [2025-10-20T22:16:14.532499] [DEBUG] [LOCAL ] YALA: Loaded 36606 existing records from primary storage. [2025-10-20T22:16:14.557137] [INFO ] [MAIN ] YALA: Loaded 36606 primary data records. [2025-10-20T22:16:14.681353] [DEBUG] [LOCAL ] YELP: Loaded 45389 existing records from primary storage. [2025-10-20T22:16:14.717188] [INFO ] [MAIN ] YELP: Loaded 45389 primary data records. [2025-10-20T22:16:15.505342] [DEBUG] [MAIN ] XYL: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:15.507867] [INFO ] [MAIN ] XYL: Generated 1794 derivative records from 47669 primary records. [2025-10-20T22:16:15.508057] [DEBUG] [LOCAL ] XYL: Saving derivative data. [2025-10-20T22:16:15.540285] [DEBUG] [LOCAL ] XYL: Derivative data saved (1794 records). [2025-10-20T22:16:15.540399] [INFO ] [MAIN ] XYL: Successfully saved derivative data. [2025-10-20T22:16:15.542086] [INFO ] [MAIN ] Derivative Progress: 1852/1877 symbols completed. XYL took 1.1s. Estimated remaining: 9.4s. Rate: 159.8 symbols/min [2025-10-20T22:16:15.542317] [INFO ] [MAIN ] YETI: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:15.543453] [DEBUG] [MAIN ] YALA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:15.544944] [INFO ] [MAIN ] YALA: Generated 1794 derivative records from 36606 primary records. [2025-10-20T22:16:15.552817] [DEBUG] [LOCAL ] YALA: Saving derivative data. [2025-10-20T22:16:15.558089] [DEBUG] [MAIN ] YELP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:15.567065] [INFO ] [MAIN ] YELP: Generated 1794 derivative records from 45389 primary records. [2025-10-20T22:16:15.572267] [DEBUG] [LOCAL ] YELP: Saving derivative data. [2025-10-20T22:16:15.588813] [DEBUG] [LOCAL ] YALA: Derivative data saved (1794 records). [2025-10-20T22:16:15.589686] [INFO ] [MAIN ] YALA: Successfully saved derivative data. [2025-10-20T22:16:15.595423] [DEBUG] [LOCAL ] YELP: Derivative data saved (1794 records). [2025-10-20T22:16:15.595333] [INFO ] [MAIN ] Derivative Progress: 1853/1877 symbols completed. YALA took 1.1s. Estimated remaining: 9.0s. Rate: 159.9 symbols/min [2025-10-20T22:16:15.595743] [INFO ] [MAIN ] YEXT: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:15.596254] [INFO ] [MAIN ] YELP: Successfully saved derivative data. [2025-10-20T22:16:15.598964] [INFO ] [MAIN ] Derivative Progress: 1854/1877 symbols completed. YELP took 1.1s. Estimated remaining: 8.6s. Rate: 160.0 symbols/min [2025-10-20T22:16:15.599606] [INFO ] [MAIN ] YMM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:15.648007] [DEBUG] [LOCAL ] YETI: Loaded 52543 existing records from primary storage. [2025-10-20T22:16:15.648344] [INFO ] [MAIN ] YETI: Loaded 52543 primary data records. [2025-10-20T22:16:15.766429] [DEBUG] [LOCAL ] YEXT: Loaded 43147 existing records from primary storage. [2025-10-20T22:16:15.790436] [INFO ] [MAIN ] YEXT: Loaded 43147 primary data records. [2025-10-20T22:16:16.100077] [DEBUG] [LOCAL ] YMM: Loaded 52587 existing records from primary storage. [2025-10-20T22:16:16.130392] [INFO ] [MAIN ] YMM: Loaded 52587 primary data records. [2025-10-20T22:16:16.609248] [DEBUG] [MAIN ] YETI: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:16.611109] [INFO ] [MAIN ] YETI: Generated 1794 derivative records from 52543 primary records. [2025-10-20T22:16:16.611261] [DEBUG] [LOCAL ] YETI: Saving derivative data. [2025-10-20T22:16:16.628917] [DEBUG] [LOCAL ] YETI: Derivative data saved (1794 records). [2025-10-20T22:16:16.629209] [INFO ] [MAIN ] YETI: Successfully saved derivative data. [2025-10-20T22:16:16.631281] [INFO ] [MAIN ] Derivative Progress: 1855/1877 symbols completed. YETI took 1.1s. Estimated remaining: 8.3s. Rate: 159.8 symbols/min [2025-10-20T22:16:16.631548] [INFO ] [MAIN ] YOU: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:16.650017] [DEBUG] [MAIN ] YEXT: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:16.650788] [INFO ] [MAIN ] YEXT: Generated 1794 derivative records from 43147 primary records. [2025-10-20T22:16:16.650944] [DEBUG] [LOCAL ] YEXT: Saving derivative data. [2025-10-20T22:16:16.667302] [DEBUG] [LOCAL ] YEXT: Derivative data saved (1794 records). [2025-10-20T22:16:16.667465] [INFO ] [MAIN ] YEXT: Successfully saved derivative data. [2025-10-20T22:16:16.670020] [INFO ] [MAIN ] Derivative Progress: 1856/1877 symbols completed. YEXT took 1.1s. Estimated remaining: 7.9s. Rate: 159.9 symbols/min [2025-10-20T22:16:16.670469] [INFO ] [MAIN ] YPF: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:16.686962] [DEBUG] [MAIN ] YMM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:16.687688] [INFO ] [MAIN ] YMM: Generated 1794 derivative records from 52587 primary records. [2025-10-20T22:16:16.687855] [DEBUG] [LOCAL ] YMM: Saving derivative data. [2025-10-20T22:16:16.713980] [DEBUG] [LOCAL ] YMM: Derivative data saved (1794 records). [2025-10-20T22:16:16.714218] [INFO ] [MAIN ] YMM: Successfully saved derivative data. [2025-10-20T22:16:16.716448] [INFO ] [MAIN ] Derivative Progress: 1857/1877 symbols completed. YMM took 1.1s. Estimated remaining: 7.5s. Rate: 160.0 symbols/min [2025-10-20T22:16:16.716763] [INFO ] [MAIN ] YRD: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:16.737048] [DEBUG] [LOCAL ] YOU: Loaded 49573 existing records from primary storage. [2025-10-20T22:16:16.737319] [INFO ] [MAIN ] YOU: Loaded 49573 primary data records. [2025-10-20T22:16:16.856766] [DEBUG] [LOCAL ] YRD: Loaded 11354 existing records from primary storage. [2025-10-20T22:16:16.874281] [INFO ] [MAIN ] YRD: Loaded 11354 primary data records. [2025-10-20T22:16:16.946666] [DEBUG] [LOCAL ] YPF: Loaded 50760 existing records from primary storage. [2025-10-20T22:16:17.017990] [INFO ] [MAIN ] YPF: Loaded 50760 primary data records. [2025-10-20T22:16:17.725232] [DEBUG] [MAIN ] YOU: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:17.726098] [INFO ] [MAIN ] YOU: Generated 1794 derivative records from 49573 primary records. [2025-10-20T22:16:17.726291] [DEBUG] [LOCAL ] YOU: Saving derivative data. [2025-10-20T22:16:17.748003] [DEBUG] [LOCAL ] YOU: Derivative data saved (1794 records). [2025-10-20T22:16:17.749981] [INFO ] [MAIN ] YOU: Successfully saved derivative data. [2025-10-20T22:16:17.752012] [INFO ] [MAIN ] Derivative Progress: 1858/1877 symbols completed. YOU took 1.1s. Estimated remaining: 7.1s. Rate: 159.8 symbols/min [2025-10-20T22:16:17.765785] [DEBUG] [MAIN ] YRD: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:17.766668] [INFO ] [MAIN ] YRD: Generated 1794 derivative records from 11354 primary records. [2025-10-20T22:16:17.766847] [DEBUG] [LOCAL ] YRD: Saving derivative data. [2025-10-20T22:16:17.784189] [DEBUG] [LOCAL ] YRD: Derivative data saved (1794 records). [2025-10-20T22:16:17.784374] [INFO ] [MAIN ] YRD: Successfully saved derivative data. [2025-10-20T22:16:17.786697] [INFO ] [MAIN ] Derivative Progress: 1859/1877 symbols completed. YRD took 1.1s. Estimated remaining: 6.8s. Rate: 159.9 symbols/min [2025-10-20T22:16:17.799796] [DEBUG] [MAIN ] YPF: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:17.800384] [INFO ] [MAIN ] YPF: Generated 1794 derivative records from 50760 primary records. [2025-10-20T22:16:17.800463] [DEBUG] [LOCAL ] YPF: Saving derivative data. [2025-10-20T22:16:17.815746] [DEBUG] [LOCAL ] YPF: Derivative data saved (1794 records). [2025-10-20T22:16:17.815811] [INFO ] [MAIN ] YPF: Successfully saved derivative data. [2025-10-20T22:16:17.818342] [INFO ] [MAIN ] Derivative Progress: 1860/1877 symbols completed. YPF took 1.1s. Estimated remaining: 6.4s. Rate: 160.0 symbols/min [2025-10-20T22:16:17.818938] [INFO ] [MAIN ] YUMC: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:17.818846] [INFO ] [MAIN ] YSG: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:17.819361] [INFO ] [MAIN ] YUM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:17.845565] [DEBUG] [LOCAL ] YSG: Loaded 25657 existing records from primary storage. [2025-10-20T22:16:17.845792] [INFO ] [MAIN ] YSG: Loaded 25657 primary data records. [2025-10-20T22:16:18.186813] [DEBUG] [LOCAL ] YUMC: Loaded 52595 existing records from primary storage. [2025-10-20T22:16:18.196755] [DEBUG] [LOCAL ] YUM: Loaded 51173 existing records from primary storage. [2025-10-20T22:16:18.204798] [INFO ] [MAIN ] YUMC: Loaded 52595 primary data records. [2025-10-20T22:16:18.205305] [INFO ] [MAIN ] YUM: Loaded 51173 primary data records. [2025-10-20T22:16:18.881334] [DEBUG] [MAIN ] YSG: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:18.882417] [INFO ] [MAIN ] YSG: Generated 1794 derivative records from 25657 primary records. [2025-10-20T22:16:18.882576] [DEBUG] [LOCAL ] YSG: Saving derivative data. [2025-10-20T22:16:18.899948] [DEBUG] [LOCAL ] YSG: Derivative data saved (1794 records). [2025-10-20T22:16:18.900344] [INFO ] [MAIN ] YSG: Successfully saved derivative data. [2025-10-20T22:16:18.902454] [INFO ] [MAIN ] Derivative Progress: 1861/1877 symbols completed. YSG took 1.1s. Estimated remaining: 6.0s. Rate: 159.8 symbols/min [2025-10-20T22:16:18.902734] [INFO ] [MAIN ] ZBH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:18.923494] [DEBUG] [MAIN ] YUMC: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:18.924405] [INFO ] [MAIN ] YUMC: Generated 1794 derivative records from 52595 primary records. [2025-10-20T22:16:18.924634] [DEBUG] [LOCAL ] YUMC: Saving derivative data. [2025-10-20T22:16:18.938241] [DEBUG] [MAIN ] YUM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:18.954210] [INFO ] [MAIN ] YUM: Generated 1794 derivative records from 51173 primary records. [2025-10-20T22:16:18.954310] [DEBUG] [LOCAL ] YUM: Saving derivative data. [2025-10-20T22:16:18.953406] [DEBUG] [LOCAL ] YUMC: Derivative data saved (1794 records). [2025-10-20T22:16:18.954789] [INFO ] [MAIN ] YUMC: Successfully saved derivative data. [2025-10-20T22:16:18.956753] [INFO ] [MAIN ] Derivative Progress: 1862/1877 symbols completed. YUMC took 1.1s. Estimated remaining: 5.6s. Rate: 159.9 symbols/min [2025-10-20T22:16:18.957382] [INFO ] [MAIN ] ZEPP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:18.973552] [DEBUG] [LOCAL ] YUM: Derivative data saved (1794 records). [2025-10-20T22:16:18.986112] [INFO ] [MAIN ] YUM: Successfully saved derivative data. [2025-10-20T22:16:18.988059] [INFO ] [MAIN ] Derivative Progress: 1863/1877 symbols completed. YUM took 1.2s. Estimated remaining: 5.3s. Rate: 160.0 symbols/min [2025-10-20T22:16:18.988385] [INFO ] [MAIN ] ZETA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:19.018864] [DEBUG] [LOCAL ] ZEPP: Loaded 19430 existing records from primary storage. [2025-10-20T22:16:19.019039] [INFO ] [MAIN ] ZEPP: Loaded 19430 primary data records. [2025-10-20T22:16:19.020050] [DEBUG] [LOCAL ] ZBH: Loaded 50791 existing records from primary storage. [2025-10-20T22:16:19.031904] [INFO ] [MAIN ] ZBH: Loaded 50791 primary data records. [2025-10-20T22:16:19.351513] [DEBUG] [LOCAL ] ZETA: Loaded 53618 existing records from primary storage. [2025-10-20T22:16:19.376140] [INFO ] [MAIN ] ZETA: Loaded 53618 primary data records. [2025-10-20T22:16:19.985120] [DEBUG] [MAIN ] ZBH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:19.999198] [DEBUG] [MAIN ] ZEPP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:19.999962] [INFO ] [MAIN ] ZEPP: Generated 1794 derivative records from 19430 primary records. [2025-10-20T22:16:19.998285] [INFO ] [MAIN ] ZBH: Generated 1794 derivative records from 50791 primary records. [2025-10-20T22:16:20.000877] [DEBUG] [LOCAL ] ZBH: Saving derivative data. [2025-10-20T22:16:20.000998] [DEBUG] [LOCAL ] ZEPP: Saving derivative data. [2025-10-20T22:16:20.041964] [DEBUG] [MAIN ] ZETA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:20.042692] [INFO ] [MAIN ] ZETA: Generated 1794 derivative records from 53618 primary records. [2025-10-20T22:16:20.039095] [DEBUG] [LOCAL ] ZBH: Derivative data saved (1794 records). [2025-10-20T22:16:20.042784] [DEBUG] [LOCAL ] ZETA: Saving derivative data. [2025-10-20T22:16:20.047244] [DEBUG] [LOCAL ] ZEPP: Derivative data saved (1794 records). [2025-10-20T22:16:20.046051] [INFO ] [MAIN ] ZBH: Successfully saved derivative data. [2025-10-20T22:16:20.047607] [INFO ] [MAIN ] ZEPP: Successfully saved derivative data. [2025-10-20T22:16:20.049553] [INFO ] [MAIN ] Derivative Progress: 1864/1877 symbols completed. ZBH took 1.1s. Estimated remaining: 4.9s. Rate: 159.8 symbols/min [2025-10-20T22:16:20.051981] [INFO ] [MAIN ] Derivative Progress: 1865/1877 symbols completed. ZEPP took 1.1s. Estimated remaining: 4.5s. Rate: 159.9 symbols/min [2025-10-20T22:16:20.052379] [INFO ] [MAIN ] ZH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:20.049737] [INFO ] [MAIN ] ZGN: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:20.067976] [DEBUG] [LOCAL ] ZETA: Derivative data saved (1794 records). [2025-10-20T22:16:20.068094] [INFO ] [MAIN ] ZETA: Successfully saved derivative data. [2025-10-20T22:16:20.070187] [INFO ] [MAIN ] Derivative Progress: 1866/1877 symbols completed. ZETA took 1.1s. Estimated remaining: 4.1s. Rate: 160.0 symbols/min [2025-10-20T22:16:20.070508] [INFO ] [MAIN ] ZIM: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:20.096835] [DEBUG] [LOCAL ] ZH: Loaded 24968 existing records from primary storage. [2025-10-20T22:16:20.097051] [INFO ] [MAIN ] ZH: Loaded 24968 primary data records. [2025-10-20T22:16:20.280817] [DEBUG] [LOCAL ] ZGN: Loaded 40935 existing records from primary storage. [2025-10-20T22:16:20.321674] [INFO ] [MAIN ] ZGN: Loaded 40935 primary data records. [2025-10-20T22:16:20.336411] [DEBUG] [LOCAL ] ZIM: Loaded 53475 existing records from primary storage. [2025-10-20T22:16:20.336532] [INFO ] [MAIN ] ZIM: Loaded 53475 primary data records. [2025-10-20T22:16:21.126066] [DEBUG] [MAIN ] ZH: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:21.127302] [INFO ] [MAIN ] ZH: Generated 1794 derivative records from 24968 primary records. [2025-10-20T22:16:21.127691] [DEBUG] [LOCAL ] ZH: Saving derivative data. [2025-10-20T22:16:21.145757] [DEBUG] [LOCAL ] ZH: Derivative data saved (1794 records). [2025-10-20T22:16:21.145989] [INFO ] [MAIN ] ZH: Successfully saved derivative data. [2025-10-20T22:16:21.148324] [INFO ] [MAIN ] Derivative Progress: 1867/1877 symbols completed. ZH took 1.1s. Estimated remaining: 3.8s. Rate: 159.8 symbols/min [2025-10-20T22:16:21.151627] [INFO ] [MAIN ] ZIP: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:21.167006] [DEBUG] [MAIN ] ZIM: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:21.175212] [INFO ] [MAIN ] ZIM: Generated 1794 derivative records from 53475 primary records. [2025-10-20T22:16:21.180298] [DEBUG] [LOCAL ] ZIM: Saving derivative data. [2025-10-20T22:16:21.181148] [DEBUG] [MAIN ] ZGN: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:21.182014] [INFO ] [MAIN ] ZGN: Generated 1794 derivative records from 40935 primary records. [2025-10-20T22:16:21.189234] [DEBUG] [LOCAL ] ZGN: Saving derivative data. [2025-10-20T22:16:21.209731] [DEBUG] [LOCAL ] ZGN: Derivative data saved (1794 records). [2025-10-20T22:16:21.210428] [INFO ] [MAIN ] ZGN: Successfully saved derivative data. [2025-10-20T22:16:21.212784] [INFO ] [MAIN ] Derivative Progress: 1868/1877 symbols completed. ZGN took 1.2s. Estimated remaining: 3.4s. Rate: 159.9 symbols/min [2025-10-20T22:16:21.213512] [INFO ] [MAIN ] ZK: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:21.225328] [DEBUG] [LOCAL ] ZIM: Derivative data saved (1794 records). [2025-10-20T22:16:21.225460] [INFO ] [MAIN ] ZIM: Successfully saved derivative data. [2025-10-20T22:16:21.228031] [INFO ] [MAIN ] Derivative Progress: 1869/1877 symbols completed. ZIM took 1.2s. Estimated remaining: 3.0s. Rate: 160.0 symbols/min [2025-10-20T22:16:21.228292] [INFO ] [MAIN ] ZKH: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:21.247323] [DEBUG] [LOCAL ] ZIP: Loaded 43557 existing records from primary storage. [2025-10-20T22:16:21.247530] [INFO ] [MAIN ] ZIP: Loaded 43557 primary data records. [2025-10-20T22:16:21.315270] [DEBUG] [LOCAL ] ZKH: Loaded 7399 existing records from primary storage. [2025-10-20T22:16:21.340436] [INFO ] [MAIN ] ZKH: Loaded 7399 primary data records. [2025-10-20T22:16:21.535423] [DEBUG] [LOCAL ] ZK: Loaded 39023 existing records from primary storage. [2025-10-20T22:16:21.556171] [INFO ] [MAIN ] ZK: Loaded 39023 primary data records. [2025-10-20T22:16:22.191206] [DEBUG] [MAIN ] ZIP: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:22.192369] [INFO ] [MAIN ] ZIP: Generated 1794 derivative records from 43557 primary records. [2025-10-20T22:16:22.205130] [DEBUG] [LOCAL ] ZIP: Saving derivative data. [2025-10-20T22:16:22.205634] [DEBUG] [MAIN ] ZKH: Filtered 7001 bins outside market hours, 7824 incomplete bins [2025-10-20T22:16:22.206342] [INFO ] [MAIN ] ZKH: Generated 1790 derivative records from 7399 primary records. [2025-10-20T22:16:22.207171] [DEBUG] [LOCAL ] ZKH: Saving derivative data. [2025-10-20T22:16:22.223376] [DEBUG] [LOCAL ] ZIP: Derivative data saved (1794 records). [2025-10-20T22:16:22.224461] [INFO ] [MAIN ] ZIP: Successfully saved derivative data. [2025-10-20T22:16:22.226312] [INFO ] [MAIN ] Derivative Progress: 1870/1877 symbols completed. ZIP took 1.1s. Estimated remaining: 2.6s. Rate: 159.8 symbols/min [2025-10-20T22:16:22.232218] [INFO ] [MAIN ] ZTO: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:22.249566] [DEBUG] [LOCAL ] ZKH: Derivative data saved (1790 records). [2025-10-20T22:16:22.254892] [INFO ] [MAIN ] ZKH: Successfully saved derivative data. [2025-10-20T22:16:22.254828] [DEBUG] [MAIN ] ZK: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:22.257061] [INFO ] [MAIN ] Derivative Progress: 1871/1877 symbols completed. ZKH took 1.0s. Estimated remaining: 2.3s. Rate: 159.9 symbols/min [2025-10-20T22:16:22.257871] [INFO ] [MAIN ] ZK: Generated 1794 derivative records from 39023 primary records. [2025-10-20T22:16:22.258007] [INFO ] [MAIN ] ZTR: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:22.258175] [DEBUG] [LOCAL ] ZK: Saving derivative data. [2025-10-20T22:16:22.275529] [DEBUG] [LOCAL ] ZK: Derivative data saved (1794 records). [2025-10-20T22:16:22.275767] [INFO ] [MAIN ] ZK: Successfully saved derivative data. [2025-10-20T22:16:22.277989] [INFO ] [MAIN ] Derivative Progress: 1872/1877 symbols completed. ZK took 1.1s. Estimated remaining: 1.9s. Rate: 160.0 symbols/min [2025-10-20T22:16:22.278383] [INFO ] [MAIN ] ZTS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:22.291393] [DEBUG] [LOCAL ] ZTR: Loaded 16614 existing records from primary storage. [2025-10-20T22:16:22.291584] [INFO ] [MAIN ] ZTR: Loaded 16614 primary data records. [2025-10-20T22:16:22.504016] [DEBUG] [LOCAL ] ZTO: Loaded 49462 existing records from primary storage. [2025-10-20T22:16:22.626980] [INFO ] [MAIN ] ZTO: Loaded 49462 primary data records. [2025-10-20T22:16:22.783560] [DEBUG] [LOCAL ] ZTS: Loaded 53138 existing records from primary storage. [2025-10-20T22:16:22.819959] [INFO ] [MAIN ] ZTS: Loaded 53138 primary data records. [2025-10-20T22:16:23.318267] [DEBUG] [MAIN ] ZTR: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:23.319271] [INFO ] [MAIN ] ZTR: Generated 1794 derivative records from 16614 primary records. [2025-10-20T22:16:23.319459] [DEBUG] [LOCAL ] ZTR: Saving derivative data. [2025-10-20T22:16:23.336932] [DEBUG] [LOCAL ] ZTR: Derivative data saved (1794 records). [2025-10-20T22:16:23.337389] [INFO ] [MAIN ] ZTR: Successfully saved derivative data. [2025-10-20T22:16:23.339779] [INFO ] [MAIN ] Derivative Progress: 1873/1877 symbols completed. ZTR took 1.1s. Estimated remaining: 1.5s. Rate: 159.8 symbols/min [2025-10-20T22:16:23.356891] [DEBUG] [MAIN ] ZTO: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:23.357571] [INFO ] [MAIN ] ZTO: Generated 1794 derivative records from 49462 primary records. [2025-10-20T22:16:23.357740] [DEBUG] [LOCAL ] ZTO: Saving derivative data. [2025-10-20T22:16:23.374789] [DEBUG] [LOCAL ] ZTO: Derivative data saved (1794 records). [2025-10-20T22:16:23.375475] [INFO ] [MAIN ] ZTO: Successfully saved derivative data. [2025-10-20T22:16:23.378290] [INFO ] [MAIN ] Derivative Progress: 1874/1877 symbols completed. ZTO took 1.1s. Estimated remaining: 1.1s. Rate: 159.9 symbols/min [2025-10-20T22:16:23.390909] [DEBUG] [MAIN ] ZTS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:23.391466] [INFO ] [MAIN ] ZTS: Generated 1794 derivative records from 53138 primary records. [2025-10-20T22:16:23.391527] [DEBUG] [LOCAL ] ZTS: Saving derivative data. [2025-10-20T22:16:23.406610] [DEBUG] [LOCAL ] ZTS: Derivative data saved (1794 records). [2025-10-20T22:16:23.406698] [INFO ] [MAIN ] ZTS: Successfully saved derivative data. [2025-10-20T22:16:23.409146] [INFO ] [MAIN ] Derivative Progress: 1875/1877 symbols completed. ZTS took 1.1s. Estimated remaining: 0.8s. Rate: 160.0 symbols/min [2025-10-20T22:16:23.409430] [INFO ] [MAIN ] ZVIA: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:23.409392] [INFO ] [MAIN ] ZWS: Starting derivative data generation with 30-minute intervals. [2025-10-20T22:16:23.451953] [DEBUG] [LOCAL ] ZVIA: Loaded 35831 existing records from primary storage. [2025-10-20T22:16:23.452436] [INFO ] [MAIN ] ZVIA: Loaded 35831 primary data records. [2025-10-20T22:16:23.566997] [DEBUG] [LOCAL ] ZWS: Loaded 42373 existing records from primary storage. [2025-10-20T22:16:23.577848] [INFO ] [MAIN ] ZWS: Loaded 42373 primary data records. [2025-10-20T22:16:24.127308] [DEBUG] [MAIN ] ZVIA: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:24.128146] [INFO ] [MAIN ] ZVIA: Generated 1794 derivative records from 35831 primary records. [2025-10-20T22:16:24.128461] [DEBUG] [LOCAL ] ZVIA: Saving derivative data. [2025-10-20T22:16:24.146935] [DEBUG] [LOCAL ] ZVIA: Derivative data saved (1794 records). [2025-10-20T22:16:24.147101] [INFO ] [MAIN ] ZVIA: Successfully saved derivative data. [2025-10-20T22:16:24.149220] [INFO ] [MAIN ] Derivative Progress: 1876/1877 symbols completed. ZVIA took 0.7s. Estimated remaining: 0.4s. Rate: 159.9 symbols/min [2025-10-20T22:16:24.163082] [DEBUG] [MAIN ] ZWS: Filtered 7001 bins outside market hours, 7820 incomplete bins [2025-10-20T22:16:24.163766] [INFO ] [MAIN ] ZWS: Generated 1794 derivative records from 42373 primary records. [2025-10-20T22:16:24.163828] [DEBUG] [LOCAL ] ZWS: Saving derivative data. [2025-10-20T22:16:24.178954] [DEBUG] [LOCAL ] ZWS: Derivative data saved (1794 records). [2025-10-20T22:16:24.179016] [INFO ] [MAIN ] ZWS: Successfully saved derivative data. [2025-10-20T22:16:24.181247] [INFO ] [MAIN ] Derivative Progress: 1877/1877 symbols completed. ZWS took 0.8s. Estimated remaining: 0.0s. Rate: 160.0 symbols/min [2025-10-20T22:16:24.183932] [INFO ] [MAIN ] Parallel derivative generation completed in 11m 44s [2025-10-20T22:16:24.184096] [INFO ] [MAIN ] Derivative generation success rate: 100.0% (1877/1877 symbols) [2025-10-20T22:16:24.945208] [DEBUG] [EMAIL ] Email sent to 'chris.punches@silogroup.org' with subject: 'COMPLETE_OK: TDM Derivative Generation Completed (1877/1877 @ 100.00%)' [2025-10-20T22:16:24.945748] [INFO ] [MAIN ] Both primary update and derivative generation completed successfully. [2025-10-20T22:16:24.946854] [WARN ] [STATE ] State set to: COMPLETED_OK [2025-10-20T22:16:24.947046] [INFO ] [STATE ] Last completion updated: 2025-10-20T22:16:24.946929